Why Ray Tracing Is More Important Than You Realize

*Updated July 2023

Many gamers seem to believe that real-time ray tracing is nothing more than a fancier reflections technology, or even worse, they see it as a technically inferior form of lighting that just makes scenes darker somehow. They cannot be more wrong. Ray tracing is not only the biggest leap in 3D game graphics technology in at least two decades, and not only can it lead to hugely improved immersion when used well, it can also lead to gameplay improvements and innovations. It is a technology here to stay, it is the present and the future, and in this article we will explain how while debunking other common misconceptions, because it is unbelievably misunderstood.

What is ray tracing exactly? Well, we’ve been playing games with it for decades, but in a different form. Static lighting in most games since the late 1990s is path traced, but pre-baked (offline) so that no calculations are done during runtime. This is also how Pixar movies are famously rendered. Path tracing, which is just ray tracing but with multiple ray bounces being calculated, is physically accurate lighting – in the 2010s video games adopted physically based rendering, so physically accurate lighting is the next step in photorealism.

In path traced rendering, light is made up of rays or photons – as in reality – which shoot around in the environment, bouncing around, lighting the environment indirectly (global illumination), taking into account material permutations (e.g. picking up the refracted color from a surface), giving accurate reflections with no limitations, even causing actual optical magnification and illusions like rainbows naturally. All things raster either can’t do or uses janky trickery to fake.

Static lighting in all forms is terrible for games. You move around in games, so when light (whether direct illumination, indirect illumination aka global illumination, refractions, reflections, shadows) doesn’t move with you, the scene breaks. Path tracing solves most age-old rendering problems.

Ray tracing is a feature in Vulkan and DirectX 12 Ultimate. NVIDIA helped create this version of real-time ray traced rendering, but now it’s a Khronos Group and Microsoft standard. Any graphics card meeting those requirements can run ray tracing. This includes AMD’s RDNA 2 and later as well as Intel Arc and later.

And no, NVIDIA isn’t the only brand with cards that can effectively use ray tracing. Every respectable RDNA 2 card can at least handle Unreal Engine 5’s Lumen using its highest performance settings at respectable frame rates on a reasonable resolution for the card (1080p minimum). RDNA 3 is about twice as fast in ray tracing, and Intel is making similar strides.

This misconception has largely died down it seems, but it should be addressed regardless. Sure, in a number of games, only reflections are ray traced while everything else is still rasterized. Ray traced reflections can reflect anything rather than just what’s on-screen, but this is of course more expensive than screen space reflections. Still, Metro Exodus: Enhanced Edition showed early on that ray tracing isn’t just reflections… granted so did baked path tracing in the 90s.

As you can see, from a graphical standpoint, this technology does a hell of a lot more than just improve reflections. In Metro, the most complete ray traced effect is global illumination, which is just indirect lighting. Most games use static indirect lighting – that is light that is bounced off of surfaces, like how if you turn on a flashlight in a pitch black room in reality, it lights up the whole room rather than only the area within the light beam. Without global illumination, only the area within the light beam is illuminated while the rest of the room is pitch black.

But the future is fully path traced rendering with no more rasterization at all. We’re already close to this with Portal with RTX and to a lesser extent Cyberpunk 2077. 

Path tracing is actually the more efficient rendering technology than raster, because all rendering happens in a single pass and the number of rays scales linearly with regards to performance. Raster rendering is janky and full of workarounds, path tracing “just works” to a much greater extent.

But it isn’t only good for photorealism, as we’ll now examine.

 

DOOM 3 heavily experimented with dynamic lighting and using it to improve gameplay in a straightforward way; shadows cast by enemies can often give their position away. This, combined with the artistic lighting design, made light a gameplay element. Ray traced lighting (including shadows) takes that to another level; in fact even older, more primitive global illumination techniques that don’t use hardware ray tracing have demonstrated this.

For example, the purely software-driven global illumination and eye adaptation/HDR in Kingdom Come: Deliverance and Red Dead Redemption 2 allow the player to spot fires or other dynamic lights in the distance via indirect lighting, thus identifying people or camps where simpler lighting techniques wouldn’t allow you to see anything. This is really a life or death difference in a game as realistic as Kingdom Come, where bandits tend to be more active at night and like to set up ambushes along roads. Thanks to the game’s global illumination, a keen eye can potentially lead to you seeing them before you stumble blindly into their camp and get killed. So when playing these games, I’ll keep an eye out for dynamic indirect light, like the light from a campfire illuminating nearby vegetation. Without global illumination, that indirect light wouldn’t exist. And it’s limited to just one bounce in those games, not 8 or infinite like better solutions.

This gameplay improvement from ray tracing even extends to simpler scenarios like clearing a room in any FPS game; players like to hide in unnaturally dark corners (due to no or inadequate baked indirect lighting) in order to get the jump on you, but with ray traced global illumination (RTGI), they could be indirectly illuminated by any light source, since light travels and bounces realistically. These improvements aren’t exclusive to first person games either; a properly ray traced, globally illuminated dungeon would make for not only a big atmospheric improvement but can again seriously impact enemy detection (including AI detection in an advanced enough game) in a dungeon crawler or RPG, and most kinds of games really.

Here’s an illustration of how RTGI can save your life: the game below is Ground Branch, an Unreal Engine 4 game that actually uses its screen space global illumination, which is limited to 1 bounce. The game exclusively or almost exclusively uses dynamic lighting. You can see the areas underneath the trees and awnings are unnaturally dark – that single bounce of indirect light is highly unrealistic and inadequate. Light should illuminate these areas, but it doesn’t. So someone hiding in there is effectively invisible and could’ve easily killed me. If this game used a good RTGI solution with many bounces, then those misplaced pitch black parts of this brightly lit scene would be properly illuminated as in reality, allowing me to see whoever is back there.

Now imagine these improvements in a game like the first two Thief games, where it is absolutely vital to both hear and see enemies before they see you. Thief, despite them being late 1990s games with primitive lighting techniques, has more complex mechanical light simulation than the vast majority of games due to its importance on gameplay. But it’s faked because it’s an old game. Hiding in the dark and spotting enemies and important objects is a must. A proper ray tracing implementation would make such a game infinitely more immersive, make it rely less on an HUD element and more on the visuals and atmosphere for sneaking around, and can even result in things like the player spotting a hard to see valuable object due to a reflection or glimmer caused by the ray traced lighting and reflection. The AI’s ability to detect the player could also be significantly altered and made more dynamic by ray tracing.

How about a daytime winter raid in a shooter, RTT, or TBT game? The sun is out and the skies clear, but the snow and reflective ice being heavy on the ground can make for a blinding atmosphere. This combined with a game that simulates camouflage (or a multiplayer game where it happens naturally) would result in characters wearing white camo having a particularly big advantage with proper ray tracing.

How about a night time raid in a tactical shooter, where you might want to avoid reflective equipment because it can give your position away if a light or the moon catches it?

Or perhaps you’ll spot a target through a detailed ray traced reflection that wouldn’t be reflected by any other means? And how about scopes or binoculars reflecting light in FPS games? In the Battlefield games that have this feature, scopes are basically just big spotlights visible in daytime. That’s how they’re coded. This is a cheap imitation that is actually unfair since it pretends to reflect light even when no light should be reflected at all. With proper ray tracing, this feature can be implemented correctly, resulting in snipers and spotters having to choose positions where sunlight won’t reflect off of their optics.

Also, do you see a pattern here? There is a bizarre, wild misconception among PC gamers that ray tracing just makes scenes darker. I’m not sure where this comes from… deficient single bounce global illumination implementations, as shown in the above Ground Branch example, aren’t in mainstream games except for those that also use baked indirect lighting to make up for those deficiencies (like RDR 2). Quake II RTX isn’t flat out darker than the original, though some places are. Portal with RTX isn’t darker.

In any case, proper path traced lighting (ideally more than 5 bounces, the more the merrier) makes a game brighter if said game relies on dynamic lighting and has minimal baked indirect (ambient) lighting. Most games aren’t afraid to rely on baked ambient lighting though, so for those games a proper path traced lighting system with adequate bounces basically just converts the game from static to dynamic. More realistic no matter what.

 

Target identification and planning around this isn’t the only gameplay improvement brought upon by ray tracing. Who knows what else developers will come up with? A proper ray tracing implementation can add additional challenge to a racing simulator during heavy rain when the sun is out, resulting in a wet, highly reflective track to the point where in some areas, the sun will reflect heavily off the track thus making track lines impossible to see, making it more difficult to hit an apex or even judge braking.

At most there is a temporary performance concern with some but not all ray tracing effects. People talk about it as if no hardware will ever be able to run these effects, when NVIDIA Ada Lovelace (RTX 4000 series) and AMD RDNA 3 (RX 7000 series) are around twice as fast as their technical predecessors in ray traced rendering. Next gen will be twice as fast yet again. But of course, most PC gamers are still using GTX 1060s (2016) or GTX 1650s (2019, even slower) for some reason, apparently without any plans to upgrade ever.

Ray tracing has made incredible strides in a short amount of time, most notably through NVIDIA’s cutting edge lighting algorithm, ReSTIR. This is one of the biggest performance saving technologies in recent years at least, and it is revolutionary. Here’s an easy example: prior to ReSTIR, games were limited to just a handful of dynamic light sources per scene (maybe around a dozen) while remaining performant, with CPU bottlenecks being a concern. These are referred to as “hero lights.” Everything else would be a static light.

With RTXDI however – that is, ReSTIR direct illumination – you can forget about practical limits on number of dynamic shadow casting lights. It’s unbelievable that this feature is not talked about more… though I say the same about PhysX. Amazing, revolutionary technologies that no one cares about.

 

Who doesn’t want all light sources in a game to be dynamic? So that when anything in the environment moves, the scene doesn’t break? Sign me up. NVIDIA’s proprietary licensing will further limit its adoption, but the technology runs on any DX12 Ultimate / Vulkan-RT capable card, where it always improves performance. Not all ray tracing is hard to run. RTXDI enables a game to go from a dozen to hundreds or even thousands of light sources without sacrificing performance. All it requires is a video card with RT cores, so an RTX 2060 (2018) or newer. Consoles are well beyond this, and the Steam Deck can do it too.

This reminds me of how people mistakenly believe that Unreal Engine 5’s Nanite is a demanding feature, when in fact it increases geometry performance tenfold which allows developers to practically ignore polygon budget. Rather, PC gamers blindly believe that Nanite just means “insanely high polygon count which I can’t run on my 7 year old video card” which is false. It’s the fastest geometry rendering pipeline from any game on any card that meets the minimum requirements (DX11), like how RTXDI is the fastest direct lighting solution for all cards with RT cores.

ReSTIR doesn’t just stop at direct illumination. There’s ReSTIR global illumination as well, and as a whole ReSTIR path tracing. ReSTIR provides such a performance improvement that it’s the only reason we can have playable path traced graphics with any hardware today, in Portal with RTX and Cyberpunk 2077. Without ReSTIR path tracing and even with DLSS 3 frame generation, forget about it.

That shows just how far we’ve come in performance optimization in a short time span: real time ray tracing became mainstream in 2018 via DirectX Ray Tracing/Vulkan Ray Tracing, but fully path traced rendering was a pipe dream. That pipe dream is now feasible as of 2022. Show me another game tech leap this substantial in a 4 year period, in the 3D rendering era (1996 and later).

Others are innovating with high performance ray tracing solutions as well. Unreal Engine 5’s Lumen on performance mode is faster than Unreal Engine 4’s screen space global illumination (as well as voxel based global illumination solutions like in CryEngine), while providing infinite bounces unlike the 1 bounce in those solutions! This is thanks to radiance caching, a more recent optimized global illumination method challenging older probe-based solutions like RTXGI. AMD has their own. These don’t provide the quality of ReSTIR GI but are much faster, such that the performance target for Lumen on performance mode is 1080p 60 FPS on PlayStation 5.

Other recent technological leaps that improve ray tracing performance include: NVIDIA’s denoisers which greatly reduce the amount of rays needed to achieve an accurate image, NVIDIA Shader Execution Reordering (SER), NVIDIA Micro-Meshes (both displaced micro-meshes and opacity micro maps) which will hopefully become a DX12 Ultimate/Vulkan standard too. This is in addition to the jaw-dropping performance increasing technologies we’ve seen on the raster side of things since the inception of DX12 Ultimate, such as mesh shader, variable rate shading, texture-space shading, sampler feedback, DX12 GPU upload heaps so that the CPU can finally address all VRAM at once like Vulkan has allowed for years.

It Makes Game Development Easier

With ray traced lighting, game artists can just place the light sources where they like, and leave it at that. The lighting takes care of itself automatically. “It just works.” No need to spend lots of time baking lights, level designers just have to position light sources how they want them and adjust their power and nearby material properties to their liking, and then move on. In a way, this makes video game scene lighting more like cinema/studio scene lighting, and amusingly enough the film industry would kill for static lighting to exist in reality while forward thinking gamers largely want it eliminated from gaming.

 

Like every other game technology, there are games with good and bad implementations. Too many people have let poor ray tracing implementations like Battlefield V’s color their perception of the technology. The potential atmospheric and gameplay improvements are both immense and obvious, as showcased above.

I would rank the importance of ray traced effects in the following order:

1. RTXDI (ReSTIR direct illumination) – being able to have every single light source in a game be dynamic (as well as physically accurate) is revolutionary. Without RTXDI, even an RTX 4090 + Ryzen 7 7800X3D can’t handle a high amount of dynamic lights without huge performance penalty.
2. Global Illumination – dynamic, physically accurate indirect lighting can significantly improve gameplay as well as atmosphere as mentioned above. It most easily affects enemy detection. For lower end systems, AMD’s FidelityFX Two-Level Radiance Caching seems like a very promising solution, similar to Unreal Engine 5’s Lumen on performance mode. For higher end systems, ReSTIR global illumination is where it’s at.
3. Audio – no one seems to think of this except for AMD, some VR game studios, and the creators of Returnal. Take everything described above about path tracing and global illumination and apply it to sound. Dynamic reverb is terrible in every game when compared to how it is in reality: think of a very echoey place and how no game can achieve that. Reverb doesn’t really travel in games, and sound occluding through materials barely exists. We can do this in real time now, and it’ll be infinitely less demanding than ray traced lighting since the calculations are far simpler. Path traced, physically accurate dynamic reverb and occlusion… this can also massively improve both atmosphere and gameplay. This technology is underdeveloped and there are none that take advantage of the latest ray tracing technology such as DXR/Vulkan-RT. It also warrants the return of sound cards – sound cards with HBM3, RT cores, and compute units.
4. Reflections – reflections reflecting the whole world rather than only what’s on-screen is a gargantuan improvement, albeit a smaller gameplay improvement than the above. Still, from a gameplay perspective it gives you opportunities to detect off-screen enemies through reflections. And it’s nice to be able to have this without rendering the entire scene twice like in Unreal Engine 1 and some other really old games.
5. Shadows – ray traced vs dynamic rasterized shadows has less of a gameplay difference than the above, but it’s technically still possible for a more accurate ray traced shadow to improve gameplay by aiding enemy detection.
6. Translucency – any game that has you looking through translucent objects (glass being most common) will benefit significantly from ray traced translucency and the use of NVIDIA’s opacity micro maps to speed this up. Otherwise, translucency is just rendered poorly and looks broken and fake.
7. Particles – purely eye candy. After this I can’t single out any particular thing I want path traced more than anything else. We need everything to path traced, and we will get there eventually. This was known to be inevitable for decades.

What’s less obvious to people are the potential gameplay improvements mentioned above. For the most part, they will become reality in time, as some can be experienced right now. Rather than dreading this technological improvement just because your computer can’t currently run it, let’s rejoice over this bit of evolution in game design, as evolution is exactly what this backwards industry needs.