Introduction

This video covers the set up of the FMOD project with the GVR plugins for spatialized audio. It covers setting up the FMOD master bus to route all spatialized audio through the GVR source, adding an ambisonic soundfield event as well as point sources for the ambience, and setting up a gunshot event with a new FMOD 1.10 feature called smart presets.  In the following videos, we will be integrating this FMOD project with the Viking Village in Unity and modifying it for VR.  Note that GVR has been rebranded by Google as Resonance Audio.

Setting Up FMOD Master Bus

In a new project the first thing we need to do is go to the mixing desk (ctrl+2 on Windows Cmd+2 on Mac) and click on the master bus and then before the fader, click on the plus button and go to Add Effect -> Plugin-effects -> Google -> GVR Listener this will allow us to hear what is happening to the audio that is routed through the GVR plugins we will add on our events. We also want to set to master output to stereo, because VR audio is built to sound best in headphones.

The Ambisonic Event

The first event we are going to make is an ambisonic soundfield event for our backdrop ambience. In a new event, add the soundfield file to the timeline – it needs to be a first-order ambisonic file. On the master go to Add Effect -> Plugin-effects -> Google -> GVR Soundfield. This plugin converts the ambisonic soundfield into stereo and re-reroutes the audio to the GVR listener.

You can preview the rotation of the player using the scroll wheel in the 3d preview pane (works best listening on headphones.)

The Point Source Events

All of the ambience events will have a GVR Source (on each event’s master bus Add plugin -> Plugin-Effects -> Google -> GVR Source)

In the water event, add the tracks and then turn up the spread on the GVR source – this changes the audio to make it seem less like a point source and more spread out over an area. The GVR Source also has the ability to occlude the sound by turning up the occlusion knob.

In the torch event, add the torch sounds and create a loop region around the 

multi-event. Set the GVR distance falloff to a small number and around 100. Same process for the crane, frogs and bird.

The Gunshots

Next we are going to create our gunshot event.  Add a GVR Source to the event, adjust the falloff curve and directivity pattern.

Along with our gunshot we are going to add a sound when the bullet hits. Import all of the hit sounds into a different event for each material.

Add the GVR source plugin to each hit event, but this time I’m going to disable the distance attenuation. I do have to set the min and max distance even though I am not using the falloff curve in GVR.  We are going to create a custom falloff curve using the built in distance parameter as well as an effect.

On the track add a gain effect. This will be our custom falloff curve.  In your event add a parameter -> new parameter at the bottom, this is a new browser that has all of the parameters you have made so far.  Set the min and max to 0 and 100 respectively.  After the gain curve is set how you want it right click and select convert to preset and then add the preset to the other gun events.

Finishing Up

The last thing we are going to do for this project is select all of your events and add them to the master bank. Right click(on all events) -> add to bank->master.  Then build your banks (F7)

In this video we covered ambisonic events, the GVR plugins: sound field, source, and listener as well as a new feature to 1.10, presets.  In the next video I’ll show you how to link this all up in Unity.

Download the FMOD and Unity Projects

You can download the Unity Project and FMOD project used in these tutorials here:

DOWNLOADS

Introduction

Welcome to the first of a three part series of in depth tutorials about the art and technology of creating audio for virtual reality.  In this first part, we will be giving a general overview and talking about some of the key concepts of audio in 360 degree surround.  In the following articles, we will go into the details of setting up that audio in FMOD, and then integrating that audio into Unity.  We don’t leave out any of the steps, and even include the FMOD and Unity packages for you to play around with!

In this series of articles, we have created videos that demonstrate the pipeline for working with the Google VR (GVR) audio plugins in FMOD and Unity. We will be using the Viking Village Demo from Unity and a Vive demonstrate, but FMOD and GVR will work with any device.  This video is a collaboration between FMOD, Google and Somatone we would like express our gratitude to everyone who helped to make these videos possible.

 

 

VR Audio Concepts

There are a few concepts and I want to go over before we dive in, which I cover in this video.

 

The first concept is Ambisonics. Ambisonics is the technique that describes creating the full-sphere of sound around a listener; this includes both the stereo panning of the horizontal plane as well as the up and down of the vertical plane.  We will be using First Order Ambisonic files. these are 4 channel files in which the channels do not correspond to speakers, and for the purposes of VR, should be decoded to a stereo output and heard in headphones.

GVR in FMOD

The next concept is the different GVR plugins. There are 3 plugins in FMOD: the listener, source, and soundfield.

• The Listener takes information from both the source and soundfield and applies binaural spatialization and room effects, then converts the information to a stereo output. This should put on the master bus in FMOD.
• The Source takes mono sound files and applies distance attenuation and directivity patterns, then sends that information to the Listener.
• The Soundfield decodes first order ambisonic files based on rotation and sends it to the Listener.

GVR Effects

Spatial Audio in GVR is comprised of 3 effects:

• Interaural time differences – This is the difference in time from when one ear a sound to when the other hears that same sound.
• Interaural level differences – This is the difference in volume between one ear and the other.

• Spectral filtering – This is the changes our outer ear makes to a sound depending on the angle in which it reflects off of the outer ear and into our inner ear. This is the primary way we hear elevation of sounds.

These effects that our ears do for us in the real world must be recreated in a virtual world.

Based on these effects Google has created HRTFs or head related transfer functions that recreate the effects and packaged them into the GVR audio plugins so we can spatialize sounds in virtual reality. The plugins also include distance attenuation curves and can link to the GVR audio room which will simulate early and late reflections (or reverb) in real time.

Additional GVR Settings

The plugins also have settings for directivity patterns, occlusion and spread.

•  The directivity is the way the sound propagates from a source, these work like virtual microphones and mimic the common pickup patterns patterns: omni, cardioid and figure eight. It also includes a sharpness factor – which is the width of the directivity pattern.
• The occlusion factor simulates an object blocking the direct sound from a source and affects high and low frequencies differently and mimics how occlusion works in the reality.
• The spread describes the point the sound comes from. Sounds like gunshots and ricochets can come from small point sources and have no spread, but water sounds will come from wider sources and have higher spread.

Final Thoughts – Key Concepts in VR Audio

Lastly, the videos above cover a few general rules for designing sound in a VR space to make your world seem more believable.

• Use mono source files, unless it is an ambisonic sound field or it is not going to be spatialized (ex: non-diagetic music, internal player VO).
• Only use reverb in enclosed spaces
• Place audio sources as accurately as possible (voice sources should come from the mouth, not the center of the body)
• Use complex waveforms covering a wide spectrum – fundamental waveforms like sine waves do not exist naturally and do not spatialize well.
• If you want a listener to locate a source, play a sound multiple times, make sure to include higher frequencies as they are easier for our ears to pinpoint, and animate it, people have an easier time locating moving sources as opposed to stationary sources.

Stay Tuned…

We will be releasing the remaining  parts of this tutorial series over the next few weeks.  In the next installment, we go into the FMOD setup for different types of VR audio and how to use the GVR plugin.  In the final two installments, we explain in detail how to integrate FMOD into Unity, including the projects themselves and the code required to get some really cool audio hooked up.