Introduction

This is the final part of our Unity+FMOD+GVR tutorial series and we are covering the implementation into Unity.  The videos are separated into two parts, implementing ambience and implementing the gunshot interaction. We hope these tutorials are helpful for you and please contact us at [email protected] if you have questions or comments.   Note that GVR has been rebranded by Google as Resonance Audio. Thanks for watching!

If you missed the first two articles in this series, you can find them here:

Part 1 – Introduction to Spatial Audio

Part 2 – FMOD Setup

In the previous video we set up an FMOD project with the GVR plugins. In this video we are going to set up the Unity project for VR and implement the ambience events. I’m building this using a Vive, but GVR and FMOD will work with any VR device and since the SteamVR prefabs use OpenVR, you can use any headset that supports OpenVR, including an Oculus.

Download the FMOD and Unity Projects

You can download the completed Unity Project and FMOD project used in these tutorials here:

DOWNLOADS

Setup

If you are starting from scratch, the first thing we need to do is download the viking village project and the steamVR prefabs (If you are using the finished project this is already done.)  Go to the Unity Asset Store and download them.  Search “steamvr” for the steam VR package, download and import it. Then search “viking unity” and download and import the viking village project from Unity.

If the API Updater give you a warning, click “I made a back up, go ahead” (the version on the store is the backup). When the project loads Steam VR will pop up with a window select “accept all,” this ensures that the project is set up and optimized for VR. 

Next, you will need the FMOD studio Unity integration.  www.fmod.com/downloads. You will need the same version of the Unity integration as the version of FMOD studio you are using. The integration includes the necessary GVR assets that FMOD needs. Import it by going to Assets->Import Package->Custom Package… and browse to where you downloaded the integration.  Then go to FMOD->Event Settings and in project path, and browse to your FMOD project. Also, in the Event settings, scroll down to plugins and add “gvraudio”.

Editing the Level

Open the viking village scene in Assets-> Scenes.

If you start seeing warnings relating to lighting in your console. Go to Window->lighting->settings-> and click generate. You will see a blue bar in the bottom corner of the screen showing the progress of the light maps. It is ok to continue working on the level as this is happening. (I didn’t in the video because I was screen recording and didn’t want to overuse the GPU).

I’m going to enable world boundaries, delete the canvas, event system, mouse lock and quality manager, this holds the previous player controller. Next I’m going to import the SteamVR prefab in SteamVR-> prefabs.  Import the steamvr and the camera rig prefabs. The camera rig is a basic setup that links into the Vive that we are going to extend for our purposes.

Next up, we need to specify our FMOD listener go into the hierarchy and open up the camera rig and camera(head) and open the ears in the inspector. Disable the unity audio listener and click “add component” and type in FMOD listener, then select the script.

Adding the Ambience Objects

Now we are going to add our ambience. Right click in the hierarchy and create an empty gameobject, name it ambience, and add the FMOD studio event emitter, set the play and stop to object start and destroy, respectively, and then select your event from the FMOD event browser.

Now lets test our level! Hit the play button, and we should be able to turn and hear the soundfield changing. Its important to note that it doesn’t matter where in the scene this soundfield is because sound fields do not take into account location; only rotation.

Water

We want to set up some more sources so that the ambience changes as we move throughout the level. So lets add another object and add the FMOD emitter for the ocean emitters. Again set the play and stop to start and destroy, then add our event.  On this one it is important to place it accurately, so I’m going to put it  below the listener and close to the shore.  Next, we’ll make a prefab out of it by right clicking in the project window and right-click->create->prefab, then drag the object from the hierarchy to the new prefab. Now drag the new prefab into the scene line the shore with a few more emitters. I’m going to add a new empty game object and place all of the water sounds in there, just to keep my hierarchy cleaner.

Torches

There are quite a few torches throughout the level that we are going to add sounds to, luckily they are prefabs, so we only need to edit one and then apply the changes to all of them. Select one of the torches and go the the prop_torch_01 object and add an FMOD emitter and again link the event and start destroy for start and stop.

Its important in vr to be very precise as to where you are placing your sound emitters. In 3d games everything is playing on a single plane, so if we were designing a car the wheels driving over dirt and the engine could come from the same source and sound fine, but in vr the wheels need a separate source from the engine because in reality those 2 sounds are coming from 2 different places in space. Likewise with a giant monster – in vr you shouldn’t place the foot falls at the same place the roar comes from.

Back to the torch, once you have the source placed on the torch you can go to the pf_torch_stick – the top level of the prefab and hit apply. It will add our changes to every prefab of this object – it will apply ALL changes except for the position and rotation of the transform so be careful when editing. There are 2 other torches in the level that weren’t updated, select one of them and add the event.

Cranes

We also have a sound for the giant cranes. I’m going to add this event to the wheel object and the sound will come from the center of the wheel. Again update the prefab and it will add our sound to the other 2 cranes.

Frogs

I’m also going to add a few frogs in the grass to the level, make a new empty gameobject and add the fmod emitter link up the sound.  I’m going to duplicate this object (ctrl+d) and change it to the other frog sound and move it to another patch of grass. I’m not creating a prefab of this because I only want one of each, but if I wanted more I would make a prefab for both.

Bird

The last piece of ambience we have is a bird.  We’re going to animate the position of the bird so it is flying around the area.  Create a cube, and I’m going to put mine on the wall to start. Add component-> animator and add the FMOD emitter. In our project create an animator controller, name it bird and link it to our bird object. Open the animator window right click and add an event. Then in the project add an animation, link it to the event and make sure to set it to loop in the inspector.  Open the animation window and select the bird. In the animation window add property->transform->position. I’m going to make my animation 1 minute, so scroll wheel to zoom out and move the end keyframe to 60:00. Move the time bar and then set the position.  I’m going to have the bird fly around our head.

Adding Movement

Save the scene and test it out. Be sure to turn off the record mode in the animation window.

 For the last part of this video, we’re going to add a way to move through the level. On the camera rig, I’m going to put it on the left controller, go to the controller left object – it should have the steamvr_tracked object on it already and add 2 components, the steam teleporter script and and steam laser pointer. Change the color to red and the teleport on click to true.

Part 2 – Adding the Gun Interaction

In order to add the gun interaction we need to get a gun model.  Go to the Asset store, and to get the one I used, search “gun 1911”. Download and import the package.  Then, add one of the models in the package the right controller and adjust the transform, so it is accurate to the controller. Then turn off the controller.

Add a line renderer, uncheck world space, set the width to .1, and the positions to (0, 0.7, 0) and 0, 0.7, 100).  This will create a line from the gun barrel to 100 meters in from the of the gun barrel. In order to get the line to be red, we will create a new Material.  In the project right-click->create->Material and set the albedo to red and emission to red. Add the material to the renderer.

Coding the Gun

Create a new C# script, I called it Controller_Gun and attach it to the right controller object.  In your code editor (I used Visual Studio) delete the start and update functions and remove the Monobehaviour parent class.  Replace it with SteamVR_TrackedController.  We want to extend the OnTriggerClicked function to do something when the controller trigger is pressed.  In this script we will also need to add an FMOD event reference and play a oneshot in the OnTriggerClicked function.

Back in the Inspector, link up the FMOD fire event. save and test it out.

We are going to add another script, called Target, that will be placed on everything that can be shot with the gun.  This one will be a MonoBehaviour, be again we don’t need Update or Start.  Add an FMOD Event Ref and create a new function called PlayHitSound that takes a vector3 and it will play an FMOD OneShot at the given location. The objects that will have this script will need a collider and the appropriate FMOD event linked to the script.  I started with the terrain_near (it has a mesh collider) and linked the dirt hit event from FMOD.

Back in the Controller_Gun script we need to add the interaction by using a ray cast and get data from what we hit.  If the object we hit has a Target script, we call the PlayHitSound script on it to play the appropriate hit sound.

Make sure to save and test at this point.

Making the environment responsive

The next part consists of going through each object you want to interact with the gun and making sure it has a collider and a Target script with the appropriate sound. Much of the level has been made with prefabs and is easy to add to one and apply changes to all.  You can also multi-select and add component to add the same component to all selected items. (this is already done in the completed project available for download above)

Adding GVR Audio Rooms

The last thing we are going to cover is adding reverb zones. I added 2 to the level: one in the big storage area and one in the blacksmith. In order to do this add a new empty child object to the building you want to have reverb and add the fmod gvr audio room component, then approximate the size of the room.  Any sound emitted in this zone will have reverb applied based on the parameters set in the inspector. In Unity front wall is positive Z, back wall is negative Z, left wall  is negative X, right wall is positive X, ceiling is positive Y and floor is negative Y.

Thats it, you now have a interactive VR level! Thanks for watching! again, if you have questions or comments contact us at [email protected].

Introduction

This video covers the set up of the FMOD project with the GVR plugins for spatialized audio. It covers setting up the FMOD master bus to route all spatialized audio through the GVR source, adding an ambisonic soundfield event as well as point sources for the ambience, and setting up a gunshot event with a new FMOD 1.10 feature called smart presets.  In the following videos, we will be integrating this FMOD project with the Viking Village in Unity and modifying it for VR.  Note that GVR has been rebranded by Google as Resonance Audio.

Setting Up FMOD Master Bus

In a new project the first thing we need to do is go to the mixing desk (ctrl+2 on Windows Cmd+2 on Mac) and click on the master bus and then before the fader, click on the plus button and go to Add Effect -> Plugin-effects -> Google -> GVR Listener this will allow us to hear what is happening to the audio that is routed through the GVR plugins we will add on our events. We also want to set to master output to stereo, because VR audio is built to sound best in headphones.

The Ambisonic Event

The first event we are going to make is an ambisonic soundfield event for our backdrop ambience. In a new event, add the soundfield file to the timeline – it needs to be a first-order ambisonic file. On the master go to Add Effect -> Plugin-effects -> Google -> GVR Soundfield. This plugin converts the ambisonic soundfield into stereo and re-reroutes the audio to the GVR listener.

You can preview the rotation of the player using the scroll wheel in the 3d preview pane (works best listening on headphones.)

The Point Source Events

All of the ambience events will have a GVR Source (on each event’s master bus Add plugin -> Plugin-Effects -> Google -> GVR Source)

In the water event, add the tracks and then turn up the spread on the GVR source – this changes the audio to make it seem less like a point source and more spread out over an area. The GVR Source also has the ability to occlude the sound by turning up the occlusion knob.

In the torch event, add the torch sounds and create a loop region around the 

multi-event. Set the GVR distance falloff to a small number and around 100. Same process for the crane, frogs and bird.

The Gunshots

Next we are going to create our gunshot event.  Add a GVR Source to the event, adjust the falloff curve and directivity pattern.

Along with our gunshot we are going to add a sound when the bullet hits. Import all of the hit sounds into a different event for each material.

Add the GVR source plugin to each hit event, but this time I’m going to disable the distance attenuation. I do have to set the min and max distance even though I am not using the falloff curve in GVR.  We are going to create a custom falloff curve using the built in distance parameter as well as an effect.

On the track add a gain effect. This will be our custom falloff curve.  In your event add a parameter -> new parameter at the bottom, this is a new browser that has all of the parameters you have made so far.  Set the min and max to 0 and 100 respectively.  After the gain curve is set how you want it right click and select convert to preset and then add the preset to the other gun events.

Finishing Up

The last thing we are going to do for this project is select all of your events and add them to the master bank. Right click(on all events) -> add to bank->master.  Then build your banks (F7)

In this video we covered ambisonic events, the GVR plugins: sound field, source, and listener as well as a new feature to 1.10, presets.  In the next video I’ll show you how to link this all up in Unity.

Download the FMOD and Unity Projects

You can download the Unity Project and FMOD project used in these tutorials here:

DOWNLOADS

The Churnwalker

Intro

Teaser Trailers

Hero Spotlight

The People

Actor: Roger Jackson

Writers/Producers: Sarah Arellano and Jay Osipa (SEMC)

Creative Director: Dale Crowley (Somatone)

Recording Engineer/Vocal Processing: Michael Brinkman (Somatone)

Sound designer: Dan Johnson (Somatone)

The Language – “All Secrets are Worth Knowing”

 

The Actor

 

The Voice

The Recording

We recorded the vocal audio here at Somatone, in Emeryville, California. Sarah Arellano and Jason Osipa from SuperEvilMegeCorp, produced, Dale Crowley, of Somatone, directed, and Michael Brinkman, of Somatone, was the Recording Engineer.

While Roger Jackson was in the booth, we ran the Krotos Dehumaniser2 plugin on an aux bus so that everyone in the studio, including the actor, could hear the effect the plugin was creating, and the actor could work with it in real-time. This gave Roger total control over the final product, by allowing him to play off of the effected voice, and create the sounds that he felt worked for this character. But in the end, what we had was a clean, raw version of his recording, that we could tweak, and play with, to bring out the Churn.

Roger gave us a lot to work with, both in terms of great acting, and in setting the sound/tone of the character overall. Some of the best ideas for the direction of the Churnwalker’s voice, came from playing around in the booth, and letting Roger go wild. From creating creepy noises, to delivering spine-chilling lines, (and spitting Wu-Tang verses between takes)

Roger created a character that fans are going to LOVE. In terms of playing VainGlory, it will be just as much fun to be killed by the Churnwalker, as it will be to BE the Churnwalker! Thanks to GREAT writing and character development, The Churnwalker is set to be one of the coolest characters in MOBA games today!

The Tech

Michael Brinkman handled all of the vocal processing for the Churnwalker character at Somatone. The project involved several layers of the same audio file being processed in different ways simultaneously. First, we started with the raw line of dialogue, then ran it through the Krotos Dehumaiser2 plugin, and came up with our own customized preset that included just 2 nodes. One for granular synthesis, and one for pitch shifting. This gave a really dark, sinister feeling to the voice, but also kept it clear and recognizable.

Next, we took a copy of the same clean line of dialogue, and did a little modern day backmasking. First, reversing the audio with the standard reverse plugin in ProTools Audio Suite. Then we added a decent amount of reverb with Truverb, and bounced that out as a new audio file. Once finished with the reverb, we again reversed the file using the same audio suite reverse plugin. The final product was a line of dialogue where each word begins in a reverb tail before developing into the word. This is a common effect in horror films for ghostly and/or evil characters.

The third layer was something that Roger did on the fly in the vocal booth. We all decided we loved it, so we added it to the character. It was a sound similar to the chittering of the Predator from the movies. With a little reverb that included some glittery artifacts, we ended up with a nice effect that made you wonder what kind of mouth the Churnwalker is speaking with under that mask.

The fourth and final layer of the Churnwalker’s voice, is the Churn itself. The tear in reality. The break in known existence that allows the Churnwalker to move freely among us. For this, we chose a set of samples of thunder that were really deeply rumbling, and sat mostly in the midrange, with a big notch in the area of the vocal EQ so the words were still clear, and understandable. Each line of dialogue for the Churnwalker had to be hand laid and edited with both the Predator clicks and the churn breaks, line by line. A process we believe was definitely worth it in the end.

The Result

We are honored to be working with Firelight to bring you a series of feature and tutorial videos for the new release of FMOD 1.10.

It is packed with some fantastic new capabilities, broadly grouped under three categories:  Live Update, Per Platform Optimizations, and Smart Presets.  These greatly increase the speed and flexibility of working in FMOD, and one of my favorites is the new Live Update feature that allows you to swap out audio files without requiring a new build of the game!

I call this Hot-swapping, and the scenario where this is most useful is when working with clients who send us builds of the game with Live Update.  We can test out different sounds realtime, while the game is running, without needing to ask the developer for a new build.  This feature alone makes the update to 1.10 a no brainer, but with all the other improvements and features, it seems like this is one of the most important updates to FMOD Studio in a long time.

To Download FMOD 1.10 for free, simply head over to the FMOD Website.

Here is the introduction and overview of the videos that follow:

And here is the first video, which covers the new capabilities of Smart Presets

Large projects can have thousands of events, creating a huge challenge for developers.  Smart presets provide an innovative workflow solution to managing parameters, effects, and automation across events in you project. This feature greatly improves scalability while also reducing the project’s memory footprint. Smart Presets allow you to link any number of events and have changes propagate across your project.

Smart Presets make it easier to:

• Manage large projects
• Reduce memory footprint
• Edit once and change everywhere
• Provide a dynamic and scalable workflow that is fast and easy to iterate on.

Live Update

Live Update allows you to create audio content and monitor the results while connected to your game in real time. You can mix, iterate, test and optimize your entire project in real-time with instant feedback. Interactive media provides a unique challenge for sound designers. Live update provides a powerful workflow to improve the audio design of your game.

Live Update allows you to:

• Create, Mix, Test, and Optimize in-game
• It allows for fast iteration, and fast results
• All in a workflow designed for games.

Per Platform Optimizations

Getting the audio right for each platform requires an understanding of the key features and limitations each platform, such as: what audio codecs are available, is there a limit to download size, how much RAM is available, how powerful is the CPU, and other questions that you need to understand to deliver the best audio experience you can. FMOD studio makes it easy to develop and optimize by making one project and exporting to multiple platforms. In the project we can set encoding formats and loading strategy for each platform and, new in 1.10, we can set which content and DSP plays on which platform.

The general range of platforms runs from Desktop, with the least limitations, to Mobile, with the most limitations.  On a desktop you generally have lots of memory, a powerful CPU, and unlimited download size. Whereas on mobile, you have limited memory, a less powerful CPU, a small download size and the added limitation of battery capacity. These limitations affect how we design and implement audio for each platform.

FMOD’s platform optimization tools allow us to:

• Have a single project for all platforms
• Have centralized management and testing environment
• Simplify the development and maintenance of a project
• And makes it easy to reuse content on multiple platforms

For more info go to the FMOD Website.

Introduction

Welcome to the first of a three part series of in depth tutorials about the art and technology of creating audio for virtual reality.  In this first part, we will be giving a general overview and talking about some of the key concepts of audio in 360 degree surround.  In the following articles, we will go into the details of setting up that audio in FMOD, and then integrating that audio into Unity.  We don’t leave out any of the steps, and even include the FMOD and Unity packages for you to play around with!

In this series of articles, we have created videos that demonstrate the pipeline for working with the Google VR (GVR) audio plugins in FMOD and Unity. We will be using the Viking Village Demo from Unity and a Vive demonstrate, but FMOD and GVR will work with any device.  This video is a collaboration between FMOD, Google and Somatone we would like express our gratitude to everyone who helped to make these videos possible.

VR Audio Concepts

There are a few concepts and I want to go over before we dive in, which I cover in this video.

The first concept is Ambisonics. Ambisonics is the technique that describes creating the full-sphere of sound around a listener; this includes both the stereo panning of the horizontal plane as well as the up and down of the vertical plane.  We will be using First Order Ambisonic files. these are 4 channel files in which the channels do not correspond to speakers, and for the purposes of VR, should be decoded to a stereo output and heard in headphones.

GVR in FMOD

The next concept is the different GVR plugins. There are 3 plugins in FMOD: the listener, source, and soundfield.

• The Listener takes information from both the source and soundfield and applies binaural spatialization and room effects, then converts the information to a stereo output. This should put on the master bus in FMOD.
• The Source takes mono sound files and applies distance attenuation and directivity patterns, then sends that information to the Listener.
• The Soundfield decodes first order ambisonic files based on rotation and sends it to the Listener.

GVR Effects

Spatial Audio in GVR is comprised of 3 effects:

• Interaural time differences – This is the difference in time from when one ear a sound to when the other hears that same sound.
• Interaural level differences – This is the difference in volume between one ear and the other.

• Spectral filtering – This is the changes our outer ear makes to a sound depending on the angle in which it reflects off of the outer ear and into our inner ear. This is the primary way we hear elevation of sounds.

These effects that our ears do for us in the real world must be recreated in a virtual world.

Based on these effects Google has created HRTFs or head related transfer functions that recreate the effects and packaged them into the GVR audio plugins so we can spatialize sounds in virtual reality. The plugins also include distance attenuation curves and can link to the GVR audio room which will simulate early and late reflections (or reverb) in real time.

Additional GVR Settings

The plugins also have settings for directivity patterns, occlusion and spread.

•  The directivity is the way the sound propagates from a source, these work like virtual microphones and mimic the common pickup patterns patterns: omni, cardioid and figure eight. It also includes a sharpness factor – which is the width of the directivity pattern.
• The occlusion factor simulates an object blocking the direct sound from a source and affects high and low frequencies differently and mimics how occlusion works in the reality.
• The spread describes the point the sound comes from. Sounds like gunshots and ricochets can come from small point sources and have no spread, but water sounds will come from wider sources and have higher spread.

Final Thoughts – Key Concepts in VR Audio

Lastly, the videos above cover a few general rules for designing sound in a VR space to make your world seem more believable.

• Use mono source files, unless it is an ambisonic sound field or it is not going to be spatialized (ex: non-diagetic music, internal player VO).
• Only use reverb in enclosed spaces
• Place audio sources as accurately as possible (voice sources should come from the mouth, not the center of the body)
• Use complex waveforms covering a wide spectrum – fundamental waveforms like sine waves do not exist naturally and do not spatialize well.
• If you want a listener to locate a source, play a sound multiple times, make sure to include higher frequencies as they are easier for our ears to pinpoint, and animate it, people have an easier time locating moving sources as opposed to stationary sources.

Stay Tuned…

We will be releasing the remaining  parts of this tutorial series over the next few weeks.  In the next installment, we go into the FMOD setup for different types of VR audio and how to use the GVR plugin.  In the final two installments, we explain in detail how to integrate FMOD into Unity, including the projects themselves and the code required to get some really cool audio hooked up.

We had the opportunity to sit down with Associate Producer Jonathan Cuff of IUGO Mobile Entertainment and ask a few questions about their new top-selling game; The Walking Dead: Road to Survival. Check out the interview below for some great insights into all of the aspects of production, including our musical contributions, and how it all came together to create a hit game topping the charts in the iOS store (more…)

Check out our exclusive interview with Jon Loucks, a Game Designer for DropForge Games. We helped this company step out of the box with music and sounds for their new Loot & Legends, a flash to the past when true gamers played D&D in their basements with pizza to go around!  (more…)

By Eric Van Amerongen, Sr. Sound Designer, Somatone Interactive

Audio for games: It’s what we do. It’s what we breathe day in and day out here at Somatone. If you’re a producer and want your game to be engaging, creative, cohesive, competitive on the market and most importantly, FUN! for your user’s experience, then factoring in great audio is an important component of your production. From our experience, great audio not only begins with its inception, but also continues through to its final implementation, which is the most important part of game audio production. (more…)

How does a project go from temp sounds to completed game with an original score, sound effects, and voice-over? At Somatone, we have the process down to an art – and we have the artists following a proven process. Our interactive audio artists will work with you from game design doc to launch to make sure your game sounds just the way you imagined it. (more…)

At Somatone we love any discussions of game audio and/or sound production, because generally they aren’t talked about enough. So when we saw Jaime Cross’s article about game audio we were pretty excited. (more…)