My contributions

My most noteable features in the project are:

• Recursive Stencil Buffer Portals(C#, Unity ShaderLab)
• Custom Masked Software Occlusion Culling(C++),
  based on a research paper from Intel (2016)

Recursive Stencil Buffer Portals

For this project I was mostly responsible for the non-Euclidean recursive portals. The hardest part was getting it polished for VR and the culling optimizations needed to keep the game running smooth.

We wanted Recursive portals so that the level design would not be limited to closed spaces. But instead could look out on other area's with portals, which could also be traveled between.
After researching different techniques I decided on using stencil buffer portals as the resolution of the portal plane didn't matter. And because I could use it for an arbetrairy level of recursion.

The other difficulty with the implementation for VR, was that unity's API is still quite bolted down.
Although they are making an effort to open up the rendering pipeline via the Scriptable Rendering Pipeline (SRP for short). And they are still more flexible than UE4 and Godot for example. It had some hard edges that I needed to needed a way to curve those to my needs. One such problem was that for rendering a portal. You essentially move the camera to a different position. However, calling context.SetupCameraProperties multiple times in a frame will actually crash Unity. And setting the variables directly (eg unity_StereoMatrixVP) also doesn't work because Unity overrides them with the VR drivers' matrices.
To get around this problem I eventually settled on using my own matrix variables. And modifying all standard shaders and creating a tool which converts ShaderGraph shaders automatically to my own variables.

The SRP's extra flexibility does have additional flexibility I made use of. I used it to control the draw and cull order and override the stencil states of the each material in the scene. To only draw objects in a portal, within the stencil value.

The final difficulty which kept me busy for the rest of the project was the performance. As you would expect, rendering the whole scene again from a different point of view is quite expensive. Which is why I needed to implement several optimization techniques:

Optimization of Portals:

There are two main problems are that rendering a portal in and of itself is expensive. It essentially requires rendering the same scene again, but then from a different camera position.

The second is that the worst case complexity of rendering portals recursively is O(n^k) where k is the maximum amount of recursion levels, and n is the amount of portals in the scene. Which will explode very dramatically very quickly.

To not run performance in the ground, I needed to cull my portals efficiently. My solution is, beginning from the main camera, to first to cheap backface culling and frustum culling. Then, to make sure that portals would be culled through walls. I implemented Intel's "Masked Software Occlusion Culling". It has a SIMD optimized reference implementation and this is what I integrated into unity by writing a C-API and binding that to C# to cull out most of the remaining portals which were clearly not visible through walls. The paper by Intel describes the technique of rasterizing a triangle and storing depth information in chunks. While using a heuristic for splitting up chunks for depth tests when the difference gets too big.

In addition to culling portal's away when they are not visible, a lot of performance could be gained by narrowing the view frustum used for culling. This is because almost all the portals in the level are relatively far away, thus leaving a very small screenspace area that needs to be rendered. This means that the frustum becomes very narrow and almost becomes a box.

Challenges

Teamwork

Working with a big cross-disciplinary team was quite a challenge because we often interperet things in different ways

Because of the team composition. Which had a disproportionately large amount of designers. I decided that the task of placing occlusion meshes would fall onto level designers.

Customizing Unity

It was hard to work with VR and customize the rendering pipeline. Even the new ScriptableRenderingPipeline is quite locked down. The two things that blocked me most were the locked down culling API, and locked down default Unity matrices in VR (view/projection matrix etc.). And to get a working toolchain for level design.

Tools used

For this project I wrote multiple tools for both myself to aid in debugging. And for my peers to visualize what the portal system was doing.

My own tools

For this project I had to find ways of debugging the stencil values that the portals generated. Thus the Stencil Value Viewer was born:
I also needed to create matrices which were identical with the ones from the VR SDK, while also slightly modifying the near clipping plane distance/orientation. For this I created a tool to quickly readback GPU matrices and compare them to my implementation:

Third-party tools

I used Unity's profiler, frame debugger(really useful!). And RenderDoc to debug stencil/depth values.