Deep Learning Super Sampling (DLSS) is an AI rendering technique developed by Nvidia that can boost frame rates in games while generating sharp images. DLSS uses the power of a deep neural network to increase rendering resolution and graphics performance.
How DLSS Works
DLSS relies on a deep convolutional neural network that is trained on thousands of high resolution images of game scenes. This deep learning model learns to make intelligent approximations about the detailed content needed to render a high quality image.
During gameplay, DLSS renders a lower resolution version of the image and then uses the deep learning model to fill in details and upscale the image to a higher target resolution. This gives a major performance boost compared to rendering at native resolution, while still maintaining crisp image quality.
The DLSS model focuses on key details like edges and textures when upscaling the image. This avoids the blurriness typically associated with upscaling techniques. The model also takes temporal data from previous frames into account for smoother transitions.
DLSS requires specific Tensor Core processors present on RTX graphics cards to run the deep learning model efficiently. The DLSS model is continually improved through ongoing training as more gameplay data becomes available.
DLSS Versions
There are now three main versions of DLSS available:
- DLSS 1.0 – Original version trained per game. Quality not consistent.
- DLSS 2.0 – Universal model trained on a generalized dataset. Higher quality and easier integration.
- DLSS 3.0 – Adds optical flow accelerated super resolution for even sharper quality.
DLSS 1.0 required per-game training and lacked consistency in quality across titles. DLSS 2.0 improved this with a generalized AI model that could be more easily implemented across many games.
DLSS 3.0 enhances image reconstruction through the addition of temporal data. This leverages optical flow motion vectors to boost both image quality and frame rates.
Benefits of DLSS
There are several advantages to using DLSS:
- Higher FPS: By rendering at a lower internal resolution, significant performance gains of up to 2-3x fps are possible without perceptible loss in visual quality.
- Higher resolutions: Games can be run at 4K or higher resolutions while maintaining smooth frame rates. DLSS enables playing at settings that would normally be too demanding.
- Ray tracing: DLSS helps offset the performance hit from enabling ray traced lighting effects. This makes real-time ray tracing possible on consumer gaming GPUs.
- AI enhanced graphics: The DLSS model intelligently enhances details, edges, and texture quality in a way that goes beyond traditional upscaling.
- Easy implementation: With DLSS 2.0, integration into games is relatively simple compared to optimizing rendering techniques from scratch. DLSS also works across multiple hardware generations.
DLSS in Game Performance
DLSS has proven to provide significant performance benefits in games while generating similar or higher visual quality compared to native resolution rendering.
Some examples of performance gains seen in games using DLSS:
- Death Stranding: 4K with DLSS enabled gives up to 2x higher average fps and higher image quality over native 4K.
- Control: With DLSS at 1440p resolution, frame rates can more than double from 50fps to over 100fps.
- Cyberpunk 2077: DLSS enables getting up to 60% higher performance at 4K resolution with no compromise in visual fidelity.
- Red Dead Redemption 2: Using DLSS boosts average frame rates by over 70% at 3840 x 2160 resolution.
Nvidia has made continual improvements to DLSS image reconstruction quality over successive versions. DLSS 3.0 integrates new Optical Flow Accelerator hardware to deliver even sharper image details and high efficiency. This further showcases the capabilities of AI to enhance real-time graphics.
DLSS in Professional Creative Applications
While its debut has been in gaming, Nvidia has expanded DLSS to professional creative workflows as well. For applications like video editing and 3D modelling, DLSS can provide AI-enhanced previews and significant time savings.
Creative software that has integrated DLSS includes Autodesk Maya, Arnold, and 3ds Max, as well as Blackmagic Design DaVinci Resolve. In Autodesk tools, DLSS allows for higher resolution previews of complex scenes and assets while modelling or animating. In DaVinci Resolve, DLSS can accelerate render previews, particularly for resolutions above 4K.
Advantages like being able to iterate faster with high-fidelity previews demonstrates the value DLSS can provide beyond real-time graphics applications.
The Future and Limitations of DLSS
DLSS shows how AI and deep learning techniques can improve graphics workload efficiency. It delivers substantial performance gains that scale across generations of GPU hardware. Nvidia is also expanding DLSS into new usage scenarios like video streaming.
However, DLSS still relies on developer integration and training data for optimal quality. There are overhead costs in running the DLSS neural network which require dedicated hardware acceleration. Scenes with lots of unpredictable motion can cause some instability in image reconstruction.
As machine learning techniques continue advancing, DLSS helps blaze a trail for applying AI to graphics rendering. While not a magic bullet to replace native resolution quality, DLSS provides an innovative way to maximize both performance and visual fidelity.