Synchronizing Sound and Player Actions

When we start any game, we first evaluate the look and feel. They impact how a game plays in the most fundamental ways. Look and feel feed our primary senses of touch and sight. On a list of priorities, a game’s sound won’t matter until after the player accepts those two components. Sound, although important, affects the game experience in different, less obvious ways.

Depending on the genre, the importance of sound varies. In a racing game or shooter, it’s not uncommon for players to listen to their own music. For an RPG or narrative game, constant dialogue requires both quiet and attention. In these single player, narrative driven games, studios can overlook the implementation of very small, yet impactful sound cues. Thoughtful sound design, much in line with an informative HUD, both supplements and changes how we play games.

Gears of War – Synchronizing Visuals and Sounds

The team at Epic Games utilizes sound in the Gears of War campaigns to reaffirm what players see. Distinctive music plays throughout a combat encounter, which tells players of nearby enemies. You know when you cleared a Locust horde based on what you hear. When you don’t see any enemies, it matches the silence. The music combined with individual chimes or tones denote clear beginnings and ends. You can sample the combat music below:

Sound and visual sync also keeps the player and the game focused on the same thing. If Marcus, your character, or any of the game’s characters discuss a visual, a button prompt appears. When pressed, the camera shifts to item or location they discuss. This prompt helps you refocus and follow along with the conversation. Let’s call this the “Look Here Button.”

In one sequence, you’re scrounging for ammo, vision focused on the floor. Then Marcus responds to a comment about a collapsed building across the river. But you’re not there. Without a Look Here Button to shift the camera, you now need to stop, search and listen. Gears of War’s camera-shifting prompt syncs player action with the in-game character. Since Marcus and the player can participate in different activities, games need a way to bring the two together.

The “Walk and Talk”

For dialogue, Gears of War uses the “Walk and Talk” design of stripping away mechanics and limiting movement during conversations. When Marcus and Dom need to discuss an objective, the walk speed slows to a crawl and the player can’t shoot. You regain control when the discussion ends.

The movement limitation prevents the player from progressing too far forward and missing out on story information. For example, if you prepare to enter a building, you won’t reach the entrance before Dom finishes outlining the plan. The limitations control the pace and prevent an awkward separation of talking characters.

Horizon Zero Dawn – Disjointed Dialogue

Horizon Zero Dawn can learn a lot from Gears of War to improve the player/character connection. The lack of the Look Here Button in key, story related situations dampens the mysterious narrative. And without limitations in dialogue sections to regulate audio scripting, conversations feel robotic.

I entered a ruin buried beneath the surface after climbing down a rocky shaft. At the entrance of the ruin, I started collecting herbs and looting boxes. With my vision focused on the floor, Aloy, the main character, began commenting on a dead body nearby. I stopped looting and spun my camera around in the dark ruin and couldn’t find the body. After Aloy finished talking, I found the body and returned to looting.

Without the Look Here Button to point me towards what Aloy commented on, I fumbled to catch up. These situations plague Horizon Zero Dawn, even in simpler sections. While exploring an open field, Aloy commented on the construction of a nearby robotic monster. I did not see this monster, so I didn’t know what she spoke about. The Look Here Button synchronizes the player and the character when each focus on different things.

Overlapping Audio

The Look Here Button controls the pace and focus, much like the limitations in movement speed. Although games like Gears of War make the player wait for any dialogue to finish, Horizon removes those restrictions. Without the control, it leads to interrupted or overlapping dialogue.

Horizon’s version of the Walk and Talk lets you move at any speed in any direction. This freedom failed within the first few hours of the game. I walked alongside and elderly woman and instead of following, I split from them and explored a corner on the opposite end. Despite our separation, the elderly woman kept talking and Aloy responded as if they stood beside each other. I couldn’t hear the elderly woman at all.

In another instance, as someone spoke to me, I ran ahead and cut off their sentence. I missed out on the entire conversation because I reached a point where another conversation triggered. Depending on the game, the conversation might not end, and instead the audio tracks overlap. Overlapping audio ruins many moments in a game like Lighting Returns: Final Fantasy XIII because of the freedom given to the player. The Walk and Talk slowness feels restrictive and sometimes frustrating, but it exists to prevent overlapping or missed dialogue.

Overlapping audio complicates listening to Horizon’s audio logs. When you find the audio logs, the freedom to walk while listening risks triggering Aloy’s scripted dialogue. As I listened to a log and wandered into another room, Aloy commented on the furniture. Now I can’t hear anything because two audio tracks overlap. To compensate, I started listening to logs without moving.

The standstill now eliminates the opportunity for the logs to complement the visual surroundings. Player actions, Aloy’s observations and audio scripting don’t work together. They function as individual pieces, which creates a disjointed listening experience. Audio needs to consider the player’s actions.

Grand Theft Auto V – Considering Player Behaviour

Grand Theft Auto V understands players don’t think or play in straight lines. At the minimum, people follow the predetermined path, but deviate if something appears along the way. A nearby cave along a forest path invites players to explore. This detour can disrupt pacing or scripted audio. Grand Theft Auto V recognizes players will stop following the predicted path, so it pauses its conversations.

The Walk and Talk solves any audio scripting issues by implementing restrictions. Grand Theft Auto V forgoes the restrictions in favour of pausing any dialogue. In one mission, as I drove my partner to our destination, he told me a story. When I left the car to find another vehicle, he said something like, “Hey, where are you going?” When we hopped into the new car, he resumed his story from where he left off.

The small consideration for any interruptions during his story made him feel like a real person. He reacted to my terrible driving and waited for me when I left the car. I didn’t need to change how I played to accommodate the game. In Horizon Zero Dawn, I avoided combat or movement during any scripted audio sequences. It lacked true restrictions, but instead I imposed them on myself. If games aim to synchronize player actions with the character, players can act freely without missing out on dialogue or story.