Sunday, February 12, 2012

The Game Audio Explosion - A Guide to Great Game Sound Part I: Pre-production and Sound Design


I. FAR BEYOND BLEEPS AND LOOPS

The new console era is upon us. It has been met by developers everywhere with

great anticipation, promise, ...and yet, reluctance. Programmers have spent a

large portion of the past decade squeezing every last bit of potential from our

PS2s, Xbox's and Gamecubes.

Now, after tricking these machines into performing beyond their expectations,

the shackles of technology have been lifted yet again. But will the next

generation consoles guarantee better audio?

No. We can certainly expect more audio due to an increase in available

memory, and the ability to add additional content within BD-ROM and dual

layer DVD-ROM formats. But what makes audio sound good doesn't

necessarily have anything to do with performance and delivery specs. Surely,

our ability to manipulate audio will improve, but it will mean nothing if the

content doesn't deliver. This article focuses on sound creation, and will enable

you to pave the way for effective and successful interactive game sound.

You have the ability to put the creative spark in motion regardless of which

game format you are developing. Knowing and preparing your sound team as

well as understanding the processes through which they work, will ultimately

help you to keep the audio on track, both artistically and financially.

II. THE AUDIO TEAM

A few years back, I was scoring a short animated film. One of the animators for

this film held a day job at a well-known entertainment company that had just

released a CG movie about dinosaurs. I asked him what he did on that project,

to which he replied, "I did all the toenails."

I couldn't help but think of the army of people responsible for the teeth, eyes,

scales, and so on. None-the-less, I saw the movie and it was visually stunning.

Realistically, game budgets will not allow for such an extravagant audio team,

but it does illustrate a good principle; that your audio personnel have well-

defined roles with which to focus their efforts. Collectively, your audio will be

that much better for it.

Game budgets once mandated that production costs stay low, so it wasn't

unusual to find that one or two people produced all of a game's audio. Today,

the stakes are much higher, and so are the budgets. Consumer expectations

have grown, requiring a movie-like experience within the confines of their

homes. The interactive market has become a battlefield for franchise

superiority. Bland, over-used audio must not be the exposed link in the armor

of any publisher or developer.

Whether you are using an in-house audio department or outsourcing the audio

completely, it is important that individuals have well-defined roles that do not

cross over into the other aspects of sound production. If the Audio Director is

splitting time as the Sound Designer, and the Sound Designer is also the

Composer, you can be sure that none of these shared jobs will get the proper

attention they require. It is important to obtain a list of your entire audio team

that breaks down the responsibilities of each member. Use your sound budget

to fortify any areas in sound production that need particular emphasis. We will

discuss more on budgets later, but for now let's start at the beginning.

III. COMMUNICATING THE VISION: PRE-PRODUCTION

A. EARLY BIRD CATCHES THE WORM

By their very nature, creative people are passionate about what they do. You

shouldn't have difficulty finding the enthusiasm amongst your sound team. Yet

this inherent motivation is not something to be left without guidance. You will

be doing your budget as well as your team's morale, a disservice by letting

your sound team simply "have at it". When it comes time to add sound, the

sound designers have both an advantage and a disadvantage compared to the

other production team members.

The advantage is, that by the time the game is ready for audio creation, the

game has taken real shape and personality. This helps to guide the direction of

the sound effects design. The disadvantage is, that since the sound design is

one of the last stages to be developed, previously fallen deadlines become the

responsibility of the sound design team to make up. By bringing your sound

designers up to speed early, you can avoid costly third and fourth revisions.

B. THE DOCUMENTS, PLEASE!

Giving the sound team the most recent build to play, only gives them a partial

picture of the artistic direction of the game. The sound team, like the art

department, must understand the metamorphosis of the game's characters and

landscapes.

To do this, compile a book or digital archive that chronologically depicts the

artwork, from the earliest sketches to the final in-game representations.

Arrange an in-depth meeting between the sound designers, composer and the

Art Director to discuss the game's development from an artistic standpoint.

This will help your audio team create the proper palette of sounds in much the

same way an artist creates a palette of colors.

For story-driven games, distributing copies of the script will be necessary to

illustrate the motivation and goal of the game. While this is critical for

composers, the sound designers will benefit by the added sense of immersion

into the game.

Perhaps the best form of communicating the vision will come from the Game

Designer. The game designer works tirelessly in his pursuit to create "the best

game ever". He is never short of words when describing the intent of the game.

Though his work is creative, his methods are mostly technical. No one

understands the abilities of the characters in such detail as the game designer,

as the great number of technical documents he produces will attest. These

documents are invaluable to the audio team. By thoroughly examining level

overviews and enemy specs, both sound designers and composers can create

complimentary aural depictions. Bosses that are slow but powerful, or enemies

that are stealthy will be revealed in great detail within these documents,

providing the backdrop from which the sound designers can create.

C. THE DEMOS - GETTING ON THE SAME PAGE

Once the above preproduction steps have been completed, it's time for the

sound design team and composer to begin creating demos from game capture.

Create three to four movies 60 to 90 seconds in length from different levels in

the game. Be sure to include the ambient portion prior to the action in order

to hear the game shift from low to high levels of activity. However, this may

not be possible for some arcade style games.

Once the sound design and music are complete, a mix of all the audio content

should be performed by the Sound Lead or Audio Director in either stereo,

surround or both, and exported with the movies for review.

It is important to have in place a team of reviewers that appropriately represent

those who have creative input. These might include, but are not limited to, the

Developing Producer, Publishing Producer, Executive Producer, Associate

Producer, Game Designer, Art Director, Audio Director and a franchise

representative if applicable. A robust review team will help generate an

accurate and collective review. If changes in the demonstration audio are

required and then subsequently agreed upon, your audio is ready for

production.

IV. SOUND DESIGN PRODUCTION

A. EMOTIONAL RESPONSE TO SOUND

From the beginning we have been programmed to respond to sound.

A mother's voice, a church bell, or police sirens conjure an emotional

response. Sounds help us to decipher the world around us. They warn us of

danger, call us to action and bring peace and tranquility to our lives. The more

expressive the sound is, the greater our emotional response to it. Sound

effects correctly placed in a game should evoke this response while defining

the environment, circumstance and personas on screen. Due to the random

nature by which sounds are triggered in a game, they must effectively co-exist

without losing definition or character when multiple sounds occur in close

proximity to each other. Let us examine some general observations in game

sound design.

B. BEWARE OF SONIC SLUDGE!

There is a finite amount of sound data that the ear can properly interpret

before fatigue sets in. It is the role of the sound programmer or director to

prioritize which sounds are most important and at what times they are

important. The sound designer on the other hand, must always create content

that will be effective, regardless of the circumstances that exist at the time a

sound is played. Good sound effects should work well alone and in

combination with many other sounds. This is a challenging task, but careful

forethought and planning will produce a rich, dynamic and satisfying

interactive soundscape.

The key to preventing sonic fatigue is to create sound effects that vary in

volume and frequency in relation to each other. A single sound effect that is

loud and contains equal amounts of low, middle and high frequencies may be

effective when played alone, but if all the sound effects are loud and contain a

similar frequency spectrum, it becomes difficult to decipher one sound from

the next.

In most cases, the sound designer delivers the sounds at a reasonably loud

volume, to allow the audio director or programmer to appropriately mix those

sounds into the game, setting the playback volume for each sound. However, it

is the job of the sound designer to emphasize different frequencies according

to the requirements of each sound. To do this, the designer must know which

sounds are likely to be played together at any given time, then selectively

decide which sounds will emphasize specific frequencies. Higher frequencies

provide detail. Upper middle frequencies provide presence, while lower

frequencies depict power or energy. Too much emphasis on high and upper-

middle frequencies will lead to fatigue, while too many sounds containing

lower or sub frequencies, will become muddy and detract from the overall

detail of the sound design. The goal is to create individual sounds that do not

compete, but compliment. With this in mind, the sound designer must

appropriately focus on the frequencies that will best suit each sound effect.

This process essentially carves out any unnecessary sound space to allow

additional room for other sound effects to be heard. When volumes and

frequencies are selectively assigned, the sound effects will breathe and

compliment each other regardless of when they play.

C. PACING - BUILDING TOWARD CLIMACTIC MOMENTS

Now let's examine the sound design from the "Big Picture" perspective. Game

and level design documents will provide the structure of the game in terms of

moments of emphasis. Generally, these structures take the form of peaks and

valleys that convey changes in difficulty as the game progresses. Usually, the

peaks represent a boss fight, though not necessarily so. When examined as a

whole, the sound design should appropriately compliment these arching

structures, and allow, from a sound perspective, a sense of building toward

these peak moments. If the sound designer has examined the enemies and

situations thoroughly, the overall sound design will naturally fall into place,

appropriately following the peaks and valleys within the game. However, if for

example, minions sound as powerful as bosses, some adjustment will be

necessary to bring down the emphasis of these weaker and less difficult

enemies. By not doing so will result in sound design that does not match the

arching pattern of the game. To put it simply, there can be "too much of a

good thing". Let's now look at the specific areas of game sound design.

D. AMBIENCE - DEFINING ENVIROMENTS THROUGH SOUND

Initially, ambient sound should effectively portray the setting, location and time

frame of the game or its various levels. For instance, percussion and double

reed music, a multitude of bartering voices and distant clanking iron would

suggest a medieval marketplace. As the game progresses the role of the

ambient sound is to support the circumstances with which the player is

involved. Does the sound within the environment evoke danger or safety?

Activity or inactivity? Conversely, ambience can be used to deceive the player

through suggesting a false circumstance, such as creating a sense of calm

before an ambush. Under all these conditions, good ambient sound should

portray a living environment.

The psychological impact of ambient sounds can add much to the onscreen

imagery, though not physically present in the scenery. For instance a distant,

sustained cry of an infant suggests vulnerability or insecurity. A broken fence

rattling in the wind of an abandoned city, suggests to the player a previous

traumatic event. These are subtle examples used to arouse awareness in the

player. More obvious sounds should be used to cue the player of his direct

proximity to danger. Dark drones or muffled enemy vocalizations will prepare

the player for fierce combat ahead. Fear, anticipation and anxiety are easily

evoked by the careful placement of ambient sounds.

E. IMPACTS AND DESTRUCTION - BREATHING DEATH INTO THE NON-LIVING

Early on, comic books depicted the sound of the action scenes through the use

of words that sonically mimicked the action. Over time, words like "thud" "pow"

and "zap" lost their effectiveness. Comic book writers had to jog their

imaginations to express sounds in more creative and exciting ways, such as

"Kathwaaap', "fwuuuhmp" and so on. Similarly, the sound effects in early

games experienced a renaissance as memory increased and streaming

technology allowed for more and varied sounds to be launched under the

animations. However, no increase in playback performance will ensure the

effectiveness of the sound effects, if the sounds are not expressive.

From a sound perspective, impacts and destruction must primarily convey

suffering and submission. These terms apply naturally to the vocal efforts

triggered under an opponent or avatar under attack, but are more abstract

when applied to inanimate objects. Since the human voice is the most

expressive instrument in existence, applying human-like characteristics to the

'non-living', will help give the sounds a more life-like and expressive quality.

Twisting, screeching metal, the deep thud and release of broken concrete and

wood that creaks, pops and splinters convey expressive responses to the

forces applied to them, in much the same way a grunt, moan and exhale

expresses human injury.

Additionally, impacts and destruction sounds should proportionately depict the

transference of energy between the weapon and the target. A metallic ping

with a ricochet is an effective response to a bullet on metal, in which the

transfer of energy between a low-mass object at high speed can be observed.

A missile explosion, on the other hand, is more powerful and slower to

develop, therefore requiring an equally proportionate response. The sound of

larger impacts with destruction should develop through three basic phases:

Attack, Sustain and Release.

The Attack is the first and shortest event of the three. It is important to note

that the attack is not the sound of the weapon or projectile. In this case, a

missile, contains it's own dry explosion sound that is launched under the

animation of the missile explosion. Therefore the attack will be the impact

sound based on the material composition of the target. Since the attack and

the dry explosion of the missile will happen simultaneously, the attack should

have a short period of 'lead-in' or silence to allow the peak, or initial part of

the explosion of the missile to be heard uncompromised by the attack of the

material impact.

Next is the Sustain, which introduces the debris and material breakdown

created by the explosion. Over this phase, detail should be observed. The

sustain should sound less dense than the attack so that the specific details of

the destruction can adequately be heard.

The final phase is the Release, which is a response to the destruction that

should characterize a kind of 'submission'. This phase of the destruction

should contain lighter falling debris based on the materials destroyed,

movement of dust and earth and perhaps steam.

When all three of these phases are exhibited, the destructions will sound more

expressive and compliment the weapons by adequately portraying their

explosive energy.

For "The Incredible Hulk - Ultimate Destruction" we maximized the detail and

movement of large, explosive forces by dynamically altering the stereo field

throughout the three phases of the destruction. The attack phase was almost

entirely monophonic, while a quickly widening stereo field was applied to the

sustain, finally resting on a wide and fixed stereo field for the release. The

result was destruction that moved rapidly over a wide area, thereby adequately

portraying the Hulk's enormous power.

F. WEAPONS - KNOW THYSELF, KNOW THY ENEMY!

It is a lesser-known fact that a gunshot at close range, sounds less threatening

than from 40 or even 80 yards away. Since most people have never fired a gun,

their expectations for the sound of gunshots as depicted by the entertainment

media are very high. Therefore, even in games based on historical simulation,

some amount of sonic sweetening will be necessary. In the case of a "period"

war game, multiple recordings of the specific weapon should be blended

together to create a satisfying gunshot. These might include mixing together

the various distances recorded for the gunshot, as well as the dry trigger and

shell discharge sounds for the specific firearm. Sounds created this way will be

sonically interesting while retaining the historical accuracy of the weapon.

For science-fiction or fantasy games, the imagination is the sound designer's

only limitation. As mentioned previously, the design documents will shed light

on the abilities of the enemies and characters within the game. The weapons

detailed in this document should explain the amount of damage incurred by

each weapon. It is important that these sounds appropriately match the

damage potential, since the player will, to some extent, be judging the amount

of damage from each weapon by the sound it creates. For example, weapons

that contain a charge-up sound before firing, indicates to the player that a

great amount of force is forthcoming. Likewise a weapon that produces a large

discharge noise would produce the same result.

From a stylistic perspective, weapons are an extension of the personalities of

each character and should compliment the character's physical attributes,

abilities and in some cases, their heritage or history. For instance, the sounds

of swords, knives and shuriken should be as stealthy as the master ninja who

wields them. The character of these sounds should compliment the physical

qualities exhibited by the ninja and reflect the mastery of the ninja tradition.

With this in mind you should expect the sounds to be light but fierce, focused

and evoke quickness of movement.

G. VEHICLES

Since vehicle sounds typically respond to controller movements, and not

animations, they can be difficult to perform in a plausible manner. Developers

for racing games are likely to have robust code for manipulating vehicle

sounds. Since we are focusing on sound production, and not programming,

let's examine the basic elements that make up vehicle sounds.

In most cases the sound designer will provide four separate engine sounds per

vehicle: an idle loop, acceleration, a steady thrust loop and a deceleration

(engine decompression or braking). The idle will simply indicate that the

vehicle is engaged. The acceleration and deceleration sounds should be

designed to seamlessly crossfade into, and out of the steady thrust loop via

programming. This formula is effective for simple vehicles with a low threshold

of speed in which the vehicle will quickly reach maximum velocity until the

button or trigger is released.

If the visual perspectives of the vehicle can be changed, so too should the

sounds that accompany the vehicle. This will ensure a greater sense of realism.

For instance, if inside and outside perspectives are available, subtle shifts in

the observed engine sounds should be present to support the change in

perspective. An inside perspective will result in a de-emphasis of the higher

frequencies that are present within the engine sounds, giving those sounds the

muffled quality one would expect when listening to the engine from inside.

One way to perform this, is for the sound designer to supply separate versions

of the engine sounds based on the perspective observed. If the sound designer

has access to recordings from the various perspectives, this will be easy to

supply. However if these sound perspectives are not available, or if the vehicle

is fictitious, separate mixes that include changes in equalization should be

performed in order to support the visual perspectives.

For added realism, intermittent sounds can be supplied to add feedback based

on the driving conditions or the state of the vehicle while operating. For

instance, wheel-based vehicles will contain surface noises used to indicate the

terrain (tarmac, gravel etc.). Metallic rattling and scraping is used to indicate

the state of a vehicle that is damaged. The addition of these and other

intermittent sounds add a heightened sense of realism and immersion when

operating the vehicle.

H. MENUS - LESS IS MORE

As games have become more sophisticated, so too have the menus. Player's

can customize a variety of options as well as view or purchase an array of

unlock-able content. This, of course requires more navigation. In most cases,

sounds will accompany the navigation to provide greater sensory feedback. No

matter how enjoyable these sounds may be, their repetition will soon become

an annoyance. It is always safe to create short and subtle sonic events that are

felt rather than heard.




Steve Kutay is the co-founder of Radius360, an award-winning audio post production company, specializing in sound for film and games, located in Los Angeles, California. You may contact Steve at steve.kutay@radius360.com.




No comments:

Post a Comment