izigame.me

Hello again,
Following our case study on Somerville's art, we now have a technical breakdown of sediment by our programmer Jay Steen and graphics programmer Thomas Hooper. It's a long and detailed update, so we are publishing it in two parts. The following is part 1 - we hope you enjoy it!
First a quick demonstration; in the game, Earth is invaded by an alien entity known as the Monolith, who embody a strange malleable material we call sediment, deposits of which are left scattered in the environment. Our protagonist gains the ability to control these sediment deposits:
Melting: by combining your blue power with a light source, you can melt a deposit
Regrowing: some will regrow back into their original shape
Solidifying: using your red power, you can create a burst that freezes this regrowth

There are many of these sediment deposits throughout the game, for which we built tools to design and sculpt:
Simple bounding boxes are placed
These are sculpted using sphere and box shapes (constructive solid geometry)
Detail is added to the surface by fracturing the chunks, using the same sculpting tools



Development process
Before diving into the details of how this all works, a few notes on the development process.
There were a lot of expectations for sediment, we knew it was going to be a core part of the gameplay and world, but weren’t entirely sure how we wanted it to work or look, so we started with basic functionality and visuals, tried them in-game, and evolved them to fit whatever new requirements fell out. Following are a few parts of the process that worked well:
Building it twice
Initially we wanted two distinct forms of environmental sediment, dead and discarded sediment chunks, and an alive ground-covering sediment web. We built the sediment web first, which shared a lot of functionality with sediment chunks, but was much simpler by virtue of being height-map based, instead of fully volumetric 3D. This version didn't get used in the final game, but gave us a great foundation to work from for chunks, it's always easier to build something the second time!
Parallel development
There were two distinct streams to development, the gameplay systems, and the visuals. These progressed at different rates so we needed to be careful to not break one when updating the other, but also needed the freedom to radically change each part as needs arose. Thankfully, after building the sediment web, we knew that we'd take a voxel based approach for the gameplay systems, and that storing this data in a 3D texture would work well for the rendering technique we had planned. This voxel data structure, and the code that managed it, acted as a contract between the separate systems and was built collaboratively. This, combined with code review and regular testing, meant we rarely introduced new bugs in each system.
How it works
Artists place volumes of sediment in the level, and sculpt them with box and sphere primitives, we call these volumes 'chunks'.
The core structure is a 3D grid of nodes containing a fill value; when the player shines light on the chunk, nodes near the surface have their fill amount reduced, exposing more nodes behind.
A signed distance field (SDF) representing the boundary to the filled area is calculated, and used for rendering.
To render, the volume is filled with a grid of instanced meshes, which are then deformed to match the SDF.
Gameplay implementation
Melting with Light
The gameplay our designers wanted from sediment was highly varied and specific to each scene. This led us to explore many different options for implementation. We tried using skinned meshes with custom deformations for each situation, but we couldn’t make this system flexible enough and we realised it would take too much of our artists’ time to be viable. We also had a requirement to be able to melt sediment from any angle and have it respond to the type of light hitting it, ie. spotlight or point light. With this in mind, we decided a voxel approach was the best to take.


Each voxel traces paths from their nearest faces to the light
We started by defining a grid with fixed-size voxels, each having a fill value that can be depleted by light. To respond to light, each voxel traces a ray to the light source when a light is cast on that sediment chunk. If this ray intersects with another voxel that has fill, no melting takes place and the ray tracing stops. The algorithm is based on http://www.cse.yorku.ca/~amana/research/grid.pdf If the ray can trace a path to the light, a small amount of fill is removed. This process is repeated every game frame that a light is cast on the sediment, which is very CPU intensive for anything other than small sediment chunks with low grid resolutions. For this reason, we selected Burst and Unity’s Job system to carry the brunt of the work.
The Job system allowed us to make use of multiple CPU cores and parallel processing, giving us a minimum of a 4x performance increase on our low-end target hardware. It also allowed us to schedule work from the Update() callback, and give it time to process asynchronously until LateUpdate(), where we could access the results on the main thread and feed them back to the rest of the game.
Burst is a Unity technology that compiles a subset of C# into assembly language for the target hardware. It allowed us to write our voxel traversal and other sediment gameplay logic in a high-level language while achieving assembly language performance, which made the whole feature possible! In some tests, large complex arrangements of chunks that were used in game scenes were 100 times faster when compiled with Burst than when using regular C#.
Interaction with the Character
As well as being melted by light, the sediment had to behave as if it were a solid object in the world. This meant the player character could not clip through it, and other interactable objects had to bounce off it and be frozen into it. To achieve this, it was necessary to generate physics colliders for sediment, and update them dynamically when the player interacts by melting. We also wanted to be able to script other gameplay based on melting, such as releasing an object that started out frozen into the sediment. This required us to create a querying system.
Generating Collision



We identified two choices when generating collision - using a grid of pre-existing Unity colliders, or script dynamic collision behaviour for anything that touches sediment. Unity is very flexible when it comes to scripting physics, but it does not as of version 2020.3 allow any custom shapes or access to the collision solver. This would make it difficult to achieve parity with built-in physics behaviour, and we wanted our solution to ‘just work’ without having to add components to things we wanted to collide with sediment. For this reason, we chose to generate a grid of colliders.
Box colliders were selected as they are the closest analog to the voxel grid and provide the best fit with the least wasted empty collision. We preferred overfill to underfill as it is better for objects to float slightly over the surface of the sediment than to clip into it. Due to the sheer number of voxels, it was necessary to have fewer box colliders than voxels - anything more than 1024 colliders in a single hierarchy provided unacceptable CPU performance.
To minimise the number of active colliders required, we used an octree data structure to approximate areas of fill and proxy them with a box collider. If all the voxels within a branch of 8 were totally full, it could be proxied with a single box collider, otherwise it would be replaced with 8 and the logic would recurse over each box until a maximum recursion depth was reached. The depth could be specified by designers, so some sediment chunks could have more detail than others, where needed.
Fill Querying



A query volume system was needed to script gameplay based on the sediment system that would give designers the ability to detect how much sediment there is within a given volume, and get notified when it updates. For example, in the opening home scene after the protagonist wakes up, the bookcase cannot be pulled over until the sediment around it has been melted.
In this situation, a sediment query volume is authored around the bookcase. This query volume implements an interface that submits a recurring query request to the sediment system. The sediment system then detects the sediment chunks that overlap the query volume, and submits the query to those chunks’ burst job update chain for processing. Within the burst job chain, the number of filled voxels within the query shape is evaluated for that chunk. When all chunks have finished processing, the results are aggregated for each query and the query volume is notified. This then plugs into other gameplay logic systems to enable and disable objects, change available interactions, notify AI actors, or any other actions available to designers in the gameplay logic system.
Visual Effects Triggering
Visual Effects (VFX) were needed to ground sediment in the world, aid in achieving the artistic vision and give feedback to the player when they are melting or freezing a chunk. To trigger particles, we make use of Particle System Jobs - a Unity interface that allows high performance particle system control on the CPU. These can be chained with regular jobs, allowing us to slot them seamlessly into sediment’s architecture.
Melting particles require a notification to be generated whenever voxel fill changes. During the chunk’s logical update, a parallel-writable list is populated with entries detailing voxel coordinates and fill change amounts for the voxels that have changed. This list is consumed by (among other things) the melting particle generator job. The generator job keeps track of how many particles should spawn over time for each voxel, then passes particle details to the ParticleSystemJob, generating particles when necessary.
Freeze bursts are triggered when any of the sediment is frozen (preventing its regrowth). A similar architecture is utilised - a list is generated on each frame where a freeze of voxels took place, and this is consumed by the effect jobs to spawn bursts of particles.
Emission rates are defined by the VFX artist to control this behaviour, which are applied per unit volume of voxels to ensure an even emission regardless of the chunk size or density.
Audio Feedback
We wanted to be able to provide directional melting audio feedback that responded to the amount of sediment being melted, sounding like a trickle when only small amounts are being melted and a gooey ooze when larger chunks are melted. The same list that was used to generate melting visual effects was used to achieve this for audio. These fill changes are used to generate a point cloud that is submitted to the audio middleware as a group of playback positions, giving directionality to the sound. The change amounts are totalled and this is sent through as a parameter for the melt sound, allowing sound designers to interpolate playback between smaller and larger-feeling sounds.
Light Occlusion
Chunks do internal occlusion of light when the melting update occurs, however we also need to occlude melt light using other things in the world, to prevent sediment from being meltable through walls, etc. To achieve this, each chunk generates a list of rays from each of its voxels to each of the lights being cast on it. These rays are then submitted to a RaycastBatchJob which executes each frame before the chunk’s melting update. The results of each raycast are used to inform whether a melt operation can take place, and is also used to optimise the update - if a ray is occluded, there is no need to perform the update calculations. Executing these raycasts asynchronously is the only way this feature is possible from a CPU performance perspective, as it often requires hundreds of raycasts a frame.
Editing Experience
From the start of development, we wanted this system to support WYSIWYG editing and rapid iteration. To make this possible, we defined a set of editor handles that could be used to edit the size, shape, scale and density of sediment chunks. Each time a designer interacts with these handles, the chunk is regenerated from scratch. This required the same jobs we use to build the chunk at runtime to be used during editing, so there is not much distinction between what happens during edit mode and play mode. It required a lot of up-front architectural decisions and early optimisation, but it paid off to make sediment a very flexible and easy to iterate system.