Using DLAA mode for implementing anti-aliasing in custom engine, the rendered image flickers significantly.

[wsyOverflow]

sl-dlaa.mp4

Version and Platform:

• streamline v2.4.10
• GPU: RTX4090
• Driver: 555.99

I referred to the documentation and the implementation of the Streamline_Sample. Below, I will describe the specific implementation process.

1. Initialize

I use manual hooking to enable sl feature and call slSetVulkanInfo to hook vulkan API. Then I know that kFeatureDLSS is supported by calling slIsFeatureSupported.

2. Setup DLSSOptions

sl::DLSSOptions dlss_options{};
dlss_options.mode = sl::DLSSMode::eDLAA;
dlss_options.outputWidth = ;
dlss_options.outputHeight =;
dlss_options.colorBuffersHDR = sl::Boolean::eTrue;
dlss_options.useAutoExposure = sl::Boolean::eFalse;

3. Setup Constants

3.1 Camera

I used the default column-major matrices and the left-handed coordinate system of OpenGL. When configuring the camera-related properties, I converted the matrices to row-major order.

3.2 Motion Vector

Motion vector is computed in uv space as following codes, thus it is already in [-1, 1].

vec2 velocity = vec2(vertex.clip_position.xy / vertex.clip_position.w - last_clip_pos.xy) * 0.5;

• constants.jitterOffset
  I make jitterOffset be within [-0.5, 0.5]
• constants.mvecScale: [-1, 1]
  Because motion vector in my custom engine points from previous frame to current frame, which is opposite to the description of dlss documentation. Therefore, I set mvecScale to [-1, -1].
  In addition, Y-axis of screen coordinate in mu custome engine is opposite to that in dlss documention. Therefore, I need to set mvecScale to [-1, 1].
• constants.motionVectorsJittered = sl::Boolean::eTrue
  Motion vector contains jitter offset.

I also compare the motion vectors with and without correctness operation described as below.

if (correct_y) {
    constants.jitterOffset = { jitter.x, -jitter.y }; // Negate Y-axis
    constants.mvecScale = { -1.f, 1.f };  // Negate the direction of motion vector and negate Y-axis
} else {
    constants.jitterOffset = { jitter.x, jitter.y };
    constants.mvecScale = { 1.f, 1.f };
}

• no correct y

no-correct-y.mp4

• correct y

correct-y.mp4

I didn't find any difference.

[jake-nv]

Those mvecs look a little suspect. It should be the same format as FG and look similar to https://github.com/NVIDIAGameWorks/Streamline/blob/main/docs/ProgrammingGuideDLSS_G.md#buffers-to-tag.

If you haven’t already, read the DLSS-SR SDK docs (https://developer.nvidia.com/rtx/dlss). It covers the “native” NGX API rather than SL, but goes into a lot more detail.

[wsyOverflow]

Those mvecs look a little suspect. It should be the same format as FG and look similar to https://github.com/NVIDIAGameWorks/Streamline/blob/main/docs/ProgrammingGuideDLSS_G.md#buffers-to-tag.

If you haven’t already, read the DLSS-SR SDK docs (https://developer.nvidia.com/rtx/dlss). It covers the “native” NGX API rather than SL, but goes into a lot more detail.

I'm sorry, I only wrote part of the information earlier. I just finished my description. Please take a look. I have carefully reviewed the documentation you mentioned, including the documents within the DLSS SDK. I also feel that the flickering issue might be related to the motion vector, but despite trying many modifications, it had no effect. I wonder if you can identify any implementation issues from my description.

[jake-nv]

You should turn auto-exposure on (if you’re not tagging an exposure buffer). Otherwise, only the mvecs stick out.

All your inputs should be similar to slSample. A tool like Nsight Graphics or renderdoc could help you compare your integration with the sample and debug what input is the issue. If you have any post-processing, etc you might try turning that off to help narrow things down.

[wsyOverflow]

Those mvecs look a little suspect. It should be the same format as FG and look similar to https://github.com/NVIDIAGameWorks/Streamline/blob/main/docs/ProgrammingGuideDLSS_G.md#buffers-to-tag.

If you haven’t already, read the DLSS-SR SDK docs (https://developer.nvidia.com/rtx/dlss). It covers the “native” NGX API rather than SL, but goes into a lot more detail.

I found when I disable constants.motionVectorsJittered

You should turn auto-exposure on (if you’re not tagging an exposure buffer). Otherwise, only the mvecs stick out.

All your inputs should be similar to slSample. A tool like Nsight Graphics or renderdoc could help you compare your integration with the sample and debug what input is the issue. If you have any post-processing, etc you might try turning that off to help narrow things down.

Thank you for your response. I will try your suggestions.

I just find that when constants.motionVectorsJittered is set to false, the flickering disappears. However, my motion vector includes jitter. This is the same issue I encountered when implementing the DLSS-SR SDK before, https://forums.developer.nvidia.com/t/motion-vector-flag-nvsdk-ngx-dlss-feature-flags-mvjittered-seems-to-work-abnormally/294576. It seems to be related to the handling of jitter, but I don't know the exact reason.

disable-jitter-include.mp4

[jake-nv]

I’ve found some internal threads regarding confusion around those values, but not in a form that’s readily digestible.
I’ll create a task for people more familiar with the algorithm to elaborate that section of the documentation.

[wsyOverflow]

I’ve found some internal threads regarding confusion around those values, but not in a form that’s readily digestible. I’ll create a task for people more familiar with the algorithm to elaborate that section of the documentation.

Thank you for your support and look forward to your update.

[wsyOverflow]

When the camera moves, the aliasing effect in DLAA mode is very obvious. I tried many modifications, such as changing the direction of the motion vector, but none improved the situation. I hope to get some optimization suggestions.

2024-06-21.190351.mp4

[wsyOverflow]

Regarding the previously mentioned constants.motionVectorsJittered, I realized that I was mistaken. In my code, the motion vector I calculated does not include jitter, so there is no issue with it.