Raytraced Audio Update - Materials, Reverb, Room Centers, Bots, Optimisations

Hi all, I've been working on some exciting features and improvements for the next Raytraced Audio demo. Here's a recap of what I've done.

Reverb and Materials

I've been researching how reverb parameters like diffusion/density/decay/etc are influenced by the materials and shape of the environment around you.

I'm now setting each reverb parameter based on:

• materials around you

• length of returning rays

• standard deviation of the length of returning rays

• material smoothness / scattering

• material absorption for high/low frequencies

• speed of sound

• percentage of rays that return to you

• percentage of rays that escape outdoors

As an example, diffusion is currently calculated as:

diffusion = 1.0 - 0.4 x materialSmoothness - 0.5 x outdoorPercent;

It's not perfect yet and will take a lot of iteration to get this right, but it's no longer simply blending between reverb presets based on the size of the room you're in.

You'll be able to modify these reverb formulas in the next demo version.

Reverb Zones

If you're standing inside and a gun fires outside, the gun should be affected by the outdoor reverb preset instead. The echo of the gunshot should also be heard from the direction of the doors/windows leading outside.

However, reverb effects are expensive to run. If we have 100 active sounds, we can't run 100 reverb effects in real time. At minimum we'll need two reverb effects:

• One for outdoors

• One for sounds near you / in the same room as you

Depending on your specs, more reverb effects can be added for adjacent indoor rooms.

I've got this partially working, but it depends on some room-defining and sound-grouping features that I'll talk about in the next section.

Room Definition

One tester reported that sounds were too muffled when both listener and sound are hugging different sides of the same wall. For example:

Because of the nature of this shape of environment, not many rays will reach the sound. This would be solved with a floodfill approach - which is still on my list - but I don't want to give up on pure raytracing just yet.

To increase the coverage of the rays we cast - and also optimise raytracing (described later) - we need to artificially push the listener and all sounds away from their crevices and towards the center of the room. Then the rays will be:

• cast outwards from the center of the room, ensuring an even spread of rays

• searching for the center of the room that sounds are in, rather than towards the position of the sounds themselves

This small change means that sounds no longer become strongly muffled when they are just around a corner. In the next demo version you'll be able to toggle this feature on and off, as well as display the room bounding boxes.

If multiple sounds are all inside the same room, we can raytrace towards the room center, rather than towards each sound, which is a great optimisation.

Note that 'rooms' dont always align exactly to the shape of the room. For example if a room is divided in two by objects like shelves, then each half will be treated as separate rooms, since sounds in each half should be muffled differently. I'll write a separate post about how this room detection algorithm works.

Caching + Optimisation

Let's say you're inside a small room and there are 50 sounds playing in the room next to you. There will be:

• 960 rays cast outwards from the listener

• Each ray bounces 8 times

• On each bounce, a green LOS ray is cast towards each of the 50 sounds

That equals 960 x 8 x 50 = 384,000 rays that need to be cast, to fully update the scene.

The less rays we need to cast, the faster we can update the scene.

The first optimisation is a directional check. If the ray bounces off a wall, and a sound is on the other side of the wall, we know for certain that we can't reach that sound. Because the wall that we just bounced off, is in the way!

There are 6 sides of a room (floor, ceiling, 4 walls) and we can only perform this optimisation on the wall between us and the sound. So that eliminates 1/6th of the green rays we need to cast, and we're down to:

960 x 8 x (50 x 5/6) = 320,000 rays

Now for the 2nd optimisation. When a ray bounces off a wall, it will fire 50 green LOS rays - one towards each sound. Later another ray might hit the same voxel in the wall, and cast another 50 green LOS rays. What a waste! We already know what sounds are reachable from this position on the wall, so we should re-use that information.

If the room we're in is a 10x10x10 room, then each wall will be composed of 100 voxels, meaning there are 600 unique voxels we can bounce off. One wall is between us and the sound, so that leaves 500 voxels. That means we'll cast:

960 rays outwards from the listener, bouncing 8 times = 7,680 rays

500 voxels x 50 sounds = 25,000 line-of-sight (LOS) checks

7,680 + 25000 = 32,680 rays total

That's 1/10th of the rays we started with! However this is a best case scenario. In a larger environment it's less likely that rays will bounce off the same voxel twice, but it's still a useful optimisation to have.

If a player notices their CPU is burning up, they can box themselves into a smaller room to cool it down ;)

Bots

To simulate a warfare environment, you can add up to 100 bots to the level. They will run around, dig, build and shoot each other, making plenty of noise for you to test raytraced audio with.

Deaf Mode

I've recoloured the dots that each sound type makes:

• Red for gunfire

• Orange for explosions and bullet impacts

• Yellow for footsteps

• Cyan for destruction and building

It's pretty to look at, but the colours are definitely intense. I recommend lowering the Deaf Ray Count setting when there's a lot of action going on. I also need to add an opacity slider.

Speed of Sound

I learned recently that OpenAL does not account for the speed of sound in its simulation, i.e. it takes a while for a distant sound to be audible. So I've added this feature myself, which simply delays playing a sound until its distance / speed of sound in milliseconds has elapsed.

Directional Ambience

Directional ambience has been disabled as it flickers in direction when moving around, which sounds terrible. Instead the volume of wind/rain is controlled by how outside you are.

Miscellaneous Updates

There are new variables you can control

• Speed of sound

• Air absorption

• Bot amount

• Enable/disable the direct LOS check

• Distance falloff model

Better filter system - no longer allocating 256 filters on startup and choosing the closest one. Each sound now gets its own exact low pass filter.

Air absorption over distance - sounds are slightly muffled based on distance

Fixes:

• Crosshair, killfeed and points UI displays correctly (if you wish to fight the bots)

• Fixed some crashes and exceptions from your log files

• Fixed a crash when spam toggling Deaf Mode

That's all for now, I'm hoping to have this next demo ready soon. Thank you to everyone who has tried the demo and provided feedback, it has been incredibly useful.