October 7, 2021

Ray Tracing versus Animation

---

There are no answers from vendors about acceleration structure performance and actual numbers of how the dynamic geometry affects ray tracing performance. There are only recommendations on how to use acceleration structure flags. We had static performance numbers in our previous post. Now it’s time to focus on acceleration structure builders.

We will draw animated characters. Because all characters are independent, they will all require a bottom-level acceleration structure. Each pixel will query two rays, one for the scene intersection and one for the shadow intersection. The resolution is 1600×900.

Acceleration structure builders have many configuration parameters, that’s why there is a lot of data in the tables. We will also split Vulkan and Direct3D12 results into different rows.

• FT (RT) – All BLAS build time with Fast Trace flag enabled and the scene tracing time (in round brackets).
• FB (RT) – The Fast Build time and (RQ scene tracing time).
• CS (RT) – Compute shader BLAS build time (CS scene tracing time).
• FTU (RT) – All BLAS Update time with Fast Trace flag enabled and (RQ scene tracing time).
• FBU (RT) – The Fast Build Update time and (RQ scene tracing time).
• CSU (RT) – Compute shader BLAS Update time (CS scene tracing time).
• BLAS (Scratch) – The memory required for the BLAS buffers and (the scratch buffer size).

There are the results with 81 instances of the 52K triangle model. The overall number of animated vertices/triangles is 2.9M/4.2M. The number of joints is 4212:

Here is what we have:

• It’s impossible to build BLAS for animated characters in each frame. The BLAS update mode must be used. Otherwise, the BLAS builder will significantly reduce the FPS.
• AMD Vulkan BLAS builder is much slower than the Direct3D12 builder. The best AMD BLAS build time is twice bigger than on Nvidia.
• The ray queries on Apple M1 are twice faster than the CS solution, which is interesting considering that there was no difference in our previous test.
• There is no coherent buffer memory in Metal shading language. That is why we are performing the BVH update step by using atomics. And it affects performance negatively.
• AMD has an enormous memory consumption for the BLAS and Scratch buffers.
• Our GPU BVH builder outperforms any available implementation in full rebuild mode.

Now let’s find the ray tracing limits by increasing the number of characters. This time we will use a simplified model of 1500 triangles. The overall number of characters is 2401. The number of triangles in the scene is 3.6M. The number of joints is 125K. It’s getting difficult for the CPU to animate all characters independently, so we will reuse the joint transformations across multiple instances.

• The number of instances doesn’t affect Nvidia and Tellusim BVH builder time. Only the number of triangles does.
• There are no mistakes with AMD Vulkan and Metal timings. The BLAS build time is measured in seconds.
• Vulkan API is performing all BLAS build/update in a single API call. But AMD is performing BLAS build in a non-parallel way on Vulkan. Metal API is doing the same.
• Nvidia GPU is only 50-70% utilized on Windows in the second test. But it’s working at full speed on Linux.

Here are the videos of these two tests captured in BLAS update mode. Otherwise, the CS solution is faster than HW ray tracing.

How about 10000 BLAS instances? It will be 520K joints and 15M triangles. CPU cannot process half of million joints well. So we will stop the animation. Nvidia requires 1 GB BLAS buffer for all instances. The BLAS buffer for AMD is 2.3 GB, and the scratch buffer requires an additional 3 GB of memory. We are not counting pre-transformed vertices (600 MB in the case of 48 bytes per vertex). BLAS builder will work in the update mode. Nvidia 2080 Ti provides 43 FPS in 2K video mode due to underutilization, but the overall timing is already more than 10 ms, so it’s not better than 100 FPS. Radeon 6700 XT shows 49 FPS with 100% utilization.

Each animated, morphed, or deformed object requires memory for Vertex and BLAS data. Sometimes it can be a lot of data and a lot of memory bandwidth. And it affects performance. By contrast, rasterization doesn’t not require anything except original geometry and joint transformations. And all transformed data goes directly to the rasterizer. Tellusim engine is fully GPU-driven, including animation blend trees. The video with 10K animated characters is running with 135 FPS on Nvidia 2080 Ti. The performance on Radeon 6700 XT is more than 120 FPS. It works with 20 FPS and animation on mobile Apple M1. Each character has a unique animation and casts a global shadow as well.

How to make 10K animated characters scene with Tellusim Engine:

// load object
ObjectMesh object(scene);
if(!object.load("object.glb", &material, ObjectMesh::FlagTriSkiBasTex | ObjectMesh::FlagAnimation)) return 1;

// create nodes
uint32_t size = 100;
Array nodes(size * size);
for(uint32_t i = 0; i < nodes.size(); i++) {
  nodes[i] = NodeObject(graph, object);
  nodes[i].setGlobalTransform(Matrix4x3d::translate((i % size) * step, (i / size) * step, 0.0));
}

// somewhere in the update loop
Random random(0);
for(NodeObject node : nodes) {
  ObjectFrame frame;
  uint32_t index = random.geti32(0, object.getNumAnimations() - 1);
  frame.append(ObjectFrame(object.getAnimation(index), time + random.getf32(0.0f, 32.0f)));
  node.setFrame(frame);
}
scene.updateGraph(graph);
graph.updateObjectTree();
graph.updateNodes();

That’s all that’s required. CPU only does some random number generation. Everything else is delegated to the GPU.

Tellusim Animation Demo: Tellusim_Animation.zip