Raytraced Audio

UPDATE: you can now register your interest in Raytraced Audio here.

My latest YouTube video on Raytraced Audio is finally published!

If you've come from YouTube - the GitHub repository will be updated with the latest animations and models soon. I need to sleep first :)

If you've recently joined my Patreon, read the instructions for accessing the code on GitHub here. 

Overview

Raytraced Audio was one of the key features of Sector's Edge, because it had a destructible environment. It was difficult to know if a player was inside or outside, or which sounds should be muffled, because everything could be destroyed.

So I switched to a raytraced approach, which casts rays outwards from your player to learn about the environment around you. In true Vercidium style I optimised it as much as I could, so that it could keep up with the real-time destruction of Sector's Edge.

Along the way I also created an accessibility feature that paints sound on the walls around you, allowing deaf players to see sound.

My plan - and current work - is to extract this raytraced system out of Sector's Edge and into a standalone, paid plugin that other game developers can use.

I'm looking for developers that are working on games in Unreal Engine or Godot, who would be willing to test out this plugin when it's in Alpha.

You don't need to join my Patreon - although it would be appreciated! - just send me an email. My email address can be found by clicking 'more' at the top of my YouTube page.

Raytracing Features

Muffling
In real life, sounds travel a very long distance. But if you have realistic volume falloff in games, you'll hear everything around you. Raytraced Audio determines how 'reachable' every sound is, and sounds that are less reachable will be muffled. This mimics real life, and helps you as a gamer focus on the sounds that matter.

Echo
Raytraced Audio figures out the size of the room you're in, as well as how much sound returns back to your ears. This means the correct echo effects can be applied to sounds, to create a distinct difference between indoor and outdoor environments.

Directional Ambience
When indoors, you'll hear muffled rain all around you. But if you open a window, you'll hear the rain from that window specifically. Raytraced audio identifies the areas where indoors meets outdoors - like doors and windows - and creates seamless transitions between environments of different sizes.

Permeation
Not all environments are alike. Some may be composed of thick concrete walls, and others paper thin houses. Raytraced Audio determines how much 'mass' is between the listener and each sound, so that they can be muffled correctly.

Deaf Visualisation
Deaf gamers are at a huge disadvantage in 3D games, because so much information is conveyed through audio. In the early days of working on Raytraced Audio, I had a debug overlay that showed where rays travelled and where they bounced. This has been turned into an accessibility feature that paints spatialised sound on the environment around you. This allows deaf gamers to hear 'around the corner', and quickly tell where the action is.

What's Next
Materials have been on my list for a long time, as they can improve the realism of echo and permeation. This is next on my list and will allow echo properties to be calculated in real time, with no need to define reverb presets.

4K versions of the above images are available here