Mixing 2D and 3D music with Wwise Callbacks
About Blue Fire
First of all, in case you haven’t heard about it, Blue Fire is an action-adventure platformer made by Robi Studios, currently available on all platforms. Here’s a trailer.
Who I am
My name is Conrado Laje and I have been working at Robi Studios as a Technical Sound Designer since 2019. In Blue Fire I’ve made all the Sound Design, implementation, and the design of the audio systems.
In this blog I’ll be making a deep dive into a small feature -but one of my favorites- of Blue Fire’s sound design. I’m talking about mixing 2D non-diegetic music with positional 3D music emitters. Here’s an example in-game:
The Onops
As it can be heard, the “Onops” (these cute little characters that are wandering throughout the world of Penumbra) are playing the flute in sync on top of the incidental exploration music, composed by the great Ariel Contreras Esquivel.
I found this “technique” pretty much underused in game audio in general. It’s a great way to blur the lines between diegetic and non-diegetic music, without compromising the player’s immersion. Even further, I found that it glues the audio together, making the player feel part of the world, encouraging them to explore every corner of the game.
Wwise Callbacks
The communication between Wwise and the Game Engine that you are using, in this case Unreal Engine 4, usually goes one way. The engine tells Wwise what and when to play an Event via functions (Post Event, Post event at location, etc). But what if we need Wwise to tell Unreal to do something? Callbacks.
Callbacks allow you to get a call back from an Event you have posted. You can retrieve various kinds of information about a playing music, like the Duration, Music Sync Beat, Bar, Markers, etc. In this specific case, I only needed to retrieve the time location of some markers (Custom Cues) that I placed in the Music Segments. That told Unreal when to call the Event of the flute sounds that are placed in the world.
As it can be seen in the image above, the music segments have these markers on them that will indicate Unreal when to start playing the flutes. These sounds are placed in the Onops location, and have 3d positioning, making it so that you don’t hear the flute layer until you are somewhat close. Some of the musician Onop’s are a little bit out of the way, even hidden, and I found it charming to find them only by searching for that spatialized flute layer.
The Flute Melodies
Because these sounds have to be spatialized, they can’t be in the Interactive Music Hierarchy in Wwise. They must be inside the Actor Mixer Hierarchy.
The sounds are sent to a special “Diegetic Music” bus that has a sidechain compressor with the “Non Diegetic” Music Bus. That causes the “normal” music to diminish a few dB to make room for the Onops melodies.
In the States property of the Onop_Flutes, when the NPC_Talk state is “On” it will drop the Voice Volume to -108 dB, because the Onop’s will stop playing the flute when interacting with the player.
You can even interrupt their beautiful flute solo with a sword attack. This will trigger a “Flute_Mistake” sfx that will Set the Voice Volume of the Melodies to -96 and Reset the Voice Volume after a 1.3 seconds Delay.
Unreal Engine
In Unreal, we just have to connect a Delegate (that red wired node) to a Custom Event. When the Wwise Custom Cue is called, this Event will fire its logic. In this case I’m switching on Callback Type because I’m only interested in Custom “Sync User Cues”. Then I Call a Dispatcher that will be Binded in NPC_Master_Onop. All this is in the actual Player Character, where I have the main event of Music playback.
In the Onop Master is where the actual Post Ak Event is located. Because each actual Onop is a child of this master, they inherit this logic. In each child I just have to override the Ak Event, to play the correct flute melody.
The Quest
The quest consists in finding the 4 musicians who were in a flute quartet together long ago, and are now distanced because of a big fight with the band’s director, Polale. When you talk with all of them, they forgive each other and then you can find them reunited in their village, Rust City, playing a really beautiful quartet interpretation of the Rust City song.
This is a little audio joke/easter egg that I find pretty funny and that I enjoyed a lot when recording the flutes myself. Playing with the expectations of the player, who thinks that they will find the quartet playing gorgeously, and find out that they are out of tune and sounding awful, is more funny to me than it probably should be. It fits the Onop’s style of comedy set all along the game.
And that concludes the first chapter of the “Behind the sound of Blue Fire”. Please let me know if you find it interesting, and if you have any other aspect of the game’s sound that I should cover, I’m receiving recommendations. Till the next one!
