PC Gaming Tech: Understanding Ray Tracing and NVIDIA’s DLSS

NVIDIA’s new RTX Turing-based cards are more than just numbers hitters. These cards have some exciting technologies under the hood; namely dedicated RT cores for real-time ray tracing, a first for consumer graphics, and Tensor cores for NGX that, to put it simply, allow it to perform DLSS or deep learning super sampling (more on that later). If developers code their games for the new RTX platform and optimized it for both ray tracing and DLSS, NVIDIA is claiming these games will gain massive performance and visual improvements over last-gen graphics cards including their own’s GTX 1080 Ti.

Ray Tracing?

It sounds like some esoteric graphics rendering technique, but you’ve probably already enjoyed the results of it on the big screen. Modern movies use ray tracing to generate or bolster special effects, which is how you get photorealistic shadows, reflections, and refractions. It’s how filmmakers create blazing conflagrations and blend them with the real world.

In a nutshell, ray tracing involves following the path of light beams backward from your eye to the objects that the light ray interacts with. However, this requires immense computational power, which is why it’s been restricted mostly to the post-production stages of a movie, where filmmakers can take their time to render scenes and take advantage of render farms. And of course, budgets.

Before Turing, real-time ray tracing was a pipe dream for games. But the new RTX cards meant that developers could now make a new generation of games that can generate photo-realistic scenes in real-time.

Okay, but how does ray tracing works?

Ray tracing hews more closely to how light works in the real world. Light rays hit 3D objects and bounce from one object to another before reaching our eyes. On the other hand, light may also be obscured by some objects, creating shadows. Then there is reflection and refraction, which can be challenging to simulate in computer graphics.

Ray tracing works backward from your eye, retracing the path of light from your eye to the object. In effect, it traces a light ray’s path through each pixel on a 2D screen back into a 3D model of the scene.

It can also capture reflection, shadows, and refraction by mining color and lighting information at the intersection point between a light ray and an object. This data contributes to determining pixel color and the level of illumination, and if a ray bounces off or passes through the surfaces of different objects before reaching the light source, color and lighting information from all these objects contribute to the final pixel color as well.

These techniques, along with further refinements along the years, are why ray tracing is the technique of choice for movie-making today. It effectively captures the way light works, which is why you get scenes that are truer to life, replete with details like softer shadows and light that interacts more realistically with the environment.

Deep Learning Super Sampling (DLSS)

Turing’s second final highlight is NVIDIA NGX, which is a new deep learning technology stack that is a part of NVIDIA’s RTX platform. NGX utilizes deep neural networks to perform AI-based functions capable of accelerating and enhancing graphics, among other things.

NGX relies on the Turing Tensor cores for deep learning-based operations, and it does not work on older architectures prior to Turing.

NGX encompasses many things, but the one NVIDIA gave the most attention to is something called deep learning super sampling, or DLSS. This can be thought of as a new method of anti-aliasing that helps reduce jagged lines and prevent blocky images. However, the key difference is that it doesn’t run on the shader cores, which frees them up to do other work.

In a sense, this is free AA, where you get better looking graphics without the usual performance hit. This is an important milestone, because turning MSAA on in a game like Deus Ex: Mankind Divided  is enough to cripple some of the most powerful systems, and DLSS offers a possible way around that.

In modern games, rendered frames go through post-processing and image enhancements that combine input from multiple rendered frames in order to remove visual artifacts such as aliasing while still preserving detail.

DLSS applies AI to this process and is supposedly capable of a much higher quality than temporal anti-aliasing (TAA), a shader-based algorithm that combines two frames using motion vectors to determine where to sample the previous frame.

TAA renders at the final target resolution and then combines frames, losing detail in the process. However, NVIDIA says DLSS permits faster rendering at a lower input sample count, and infers a result that should rival TAA but requires approximately half the shading work in the process.

The interesting part is that DLSS can be “trained”, where it learns how to produce the desired output based on large numbers of super high quality images. NVIDIA says it collected reference images rendered using 64x super sampling, where each pixel is shaded at 64 different offsets instead of just one. This results in a high level of detail and excellent anti-aliasing results.

The DLSS network is then trained by trying to match the 64x SS output frames with its own, measuring the differences between the two, and making the necessary adjustments.

Eventually, DLSS learns to produce results that come close to that of 64x SS, while avoiding problems that can arise in more challenging scenes, such as blurring, disocclusion (where a previously occluded object becomes visible), and unwanted transparency.

The biggest bonus is that RTX cards will supposedly run up to twice as fast as previous-generation GPUs using conventional anti-aliasing, assuming the game supports DLSS.

The biggest caveat though, is that DLSS is still relatively new, and “training” it will take sometime. It’s a future-proof technology that, in theory at least, will make games of the near future look even more gorgeous than the ones supporting it today.

What does this mean for games?

Basically, games that support the RTX cards will look either more beautiful, run faster or both. Depending on how much optimization a developer wants to do for their games. Metro Exodus provides an illuminating example, and with ray tracing enabled, the rooms are far darker than with ray tracing off. In practice, this could give developers more freedom to tweak the atmospheric lighting to create their desired mood, in addition to concealing enemies in the shadows.

Similarly, in Shadow of the Tomb Raider, the shadows blend together far more realistically, and you can distinguish between different gradations of shadow, where the words penumbra and umbra take on real meaning.

In a nutshell, your games stand to look far better with NVIDIA’s RTX technology. Unfortunately, the only two Turing cards, the RTX 2080 and RTX 2080 Ti are pricey. If you aren’t thinking of jumping into the 4K bandwagon yet, there’s the relatively more affordable RTX 2070 to consider if you’re still keen in future-proofing your gaming rig.