Tech Research: Rendering Lots of Bubbles for VR

This is something I experimented at work in my down time. It spun off of brainstorming ways to render massive amounts of good looking bubbles (foam) for whenever I start working on bathing (probably for v0.5). The idea is that there would be lots of foamy clumps of bubbles when she showers. This foam would be used to decorate the scene as well as censor her body. The problem is finding a way to make good looking foam while keeping VR performance a consideration. I'll try to keep the discussion as not-technical as possible.

My main profession is technical artist specializing in shader work, and so I try to learn and come up with as many shader tricks to achieve my goal.

To start, foam should have volume and transparency as it's just a bunch of bubble spheres put together. The problem is that spheres in general tend to have LOTS of vertices so we can't brute force a giant mesh full of spheres (42 low poly spheres is already 12k triangles. We need THOUSANDS of bubbles):

If we try combining it into one big mesh with a single bumpy surface, we lose the interior faces which make us lose not only volume, but transparency since the internal faces would be gone:

So I thought what if we had a way to do this with billboards? Billboards are used to simplify objects by making them always face the camera. Like the distant trees you see in other games:

So now we can represent a single bubble with just 2 triangles if we make it always face the camera. A bonus is that since bubbles are spheres, we don't care what direction we look at it, it will always look good and round!

The problem with this is that normally billboard shaders work on a per object basis (unless we use instancing but that's not what we are going for). So when you see a forest of billboards, each tree has it's own "center" where it rotates around to always face the camera:

So we can't have multiple billboards inside a single mesh, since each bubble needs its own center to rotate around. They would just be sharing the mesh's origin. Instead I came up with a way to "pack" these needed centers into a single mesh object. How it works is I first create the shape of the foam with each individual bubble (can be anything):

Then I just extract the centers of each bubble:

And replace each center with a plane:

Now to store the center of these billboard planes in a way that can be read in a shader, I simply take advantage of the fact that each plane is UV unwrapped normally from corner to corner (0,0) to (1,1):

 This means that in the vertex shader, I can tell which corner of the billboard I am on and what direction it's "expanding" out from the plane center. For example, the corner with UV 1,1 is going up and right and corner with UV 0,0 is going down left. Therefore if I shrink all corners to the center of their respective planes, I can "unpack" them back out to their expanded positions later on. So that's what I do, I scale each plane to zero so each of their 4 corners are now at the same position (the plane center):

And so I wrote prototype shader (not in Unity) to test this approach and it worked wonderfully. The following is a single mesh with each "packed" plane being expanded in a billboard way:

Each plane is facing the camera (500 bubbles, 2k tris, 1 mesh w/ shader) and it looks good as I rotate around it because each billboard is a sphere and so it feels like it's 3D. Like a chainmail made of bubbles.

Now for the performance side. This method has kept the vertex pass extremely cheap but the real bottleneck in VR is the fragment pass. Meaning we need to reduce the number of pixels drawn. If I change my shader to additive, I can visualize the overdraw, the repeated drawing as the triangles of my billboard overlap each other:

This repeated drawing on top increases the fragment count as it just draws on the same pixel position over and over again (brighter and brighter). Since the bubble billboards don't need corners, we can reduce the frag by not having it render the corners of the billboard. This can be done by changing from plane for billboard to octagon for billboard. Image below shows both billboard shape types with their overdraw visualized:

The catch is that because we are now using octagons, that's 2x as many vertices per billboard, but since we are doing well with our vert count, it didn't bother me as much.

This does not get rid of overdraw completely but it does minimize it. I ran some tests using python to find out exactly how much I saved by switching to octagons rather than planes. The test groups vary by size of the billboards:

As you can see, as I increase the size of each bubble billboard, the overlap increases, which makes sense. The data collected was how many pixels and what level of overdraw is present in each image. Black was not drawn at all (0x) and white is the highest possible overdraw (5x). The results are as follows:

My results conclude that the bigger the billboards, the less you gain by switching to octagons. The sweet spot for size is for now purely subjective as I don't have this implemented into the game to trial and error framerate. But the fragment reduction varies from 43%~20% which is pretty big! This does not translate to 43% framerate improvement but under certain circumstances it should make a significant difference!

So there you have it. That's going to be my first stab at good looking realistic bubbles. If you knew Unity you may have asked, why not just use instancing using Particle Systems and feed it an array of all the points where you want a bubble billboard? This would work for static piles of foam but this foam system is also going to be used for characters. Meaning I could potentially render this shader using a SkinnedMeshRenderer and have each bubble center deform and animate along armatures. Which is just what I need to cover and animate Shinobu with bubbles!