[FFmpeg-devel] [PATCH] vulkan_av1: add workaround for NVIDIA drivers tested on broken CTS

[Lynne <dev@lynne.ee>]

[-- Attachment #1: Type: text/plain, Size: 421 bytes --]

The first release of the CTS for AV1 decoding had incorrect
offsets for the OrderHints values.
The CTS will be fixed, and eventually, the drivers will be
updated to the proper spec-conforming behaviour, but we still
need to add a workaround as this will take months.
Only NVIDIA use these values at all, so limit the workaround
to only NVIDIA.

Meant to be applied on top of jkqxz's previous 2 patches.

Patch attached.


[-- Attachment #2: 0001-vulkan_av1-add-workaround-for-NVIDIA-drivers-tested-.patch --]
[-- Type: text/x-diff, Size: 4175 bytes --]

From bbd2cc90206e59098accced3ff3a8896e0bfa269 Mon Sep 17 00:00:00 2001
From: Lynne <dev@lynne.ee>
Date: Sun, 14 Apr 2024 14:41:26 +0200
Subject: [PATCH] vulkan_av1: add workaround for NVIDIA drivers tested on
 broken CTS

The first release of the CTS for AV1 decoding had incorrect
offsets for the OrderHints values.
The CTS will be fixed, and eventually, the drivers will be
updated to the proper spec-conforming behaviour, but we still
need to add a workaround as this will take months.
Only NVIDIA use these values at all, so limit the workaround
to only NVIDIA.
---
 libavcodec/vulkan_av1.c    | 20 +++++++++++++++-----
 libavcodec/vulkan_decode.c |  8 ++++++++
 libavcodec/vulkan_decode.h |  4 ++++
 3 files changed, 27 insertions(+), 5 deletions(-)

diff --git a/libavcodec/vulkan_av1.c b/libavcodec/vulkan_av1.c
index 8d532445a1..ff663c8cec 100644
--- a/libavcodec/vulkan_av1.c
+++ b/libavcodec/vulkan_av1.c
@@ -97,9 +97,15 @@ static int vk_av1_fill_pict(AVCodecContext *avctx, const AV1Frame **ref_src,
         .RefFrameSignBias = hp->ref_frame_sign_bias_mask,
     };
 
-    if (saved_order_hints)
-        for (int i = 0; i < AV1_TOTAL_REFS_PER_FRAME; i++)
-            vkav1_std_ref->SavedOrderHints[i] = saved_order_hints[i];
+    if (saved_order_hints) {
+        if (dec->quirk_av1_offset) {
+            for (int i = 1; i < AV1_TOTAL_REFS_PER_FRAME; i++)
+                vkav1_std_ref->SavedOrderHints[i - 1] = saved_order_hints[i];
+        } else {
+            for (int i = 0; i < AV1_TOTAL_REFS_PER_FRAME; i++)
+                vkav1_std_ref->SavedOrderHints[i] = saved_order_hints[i];
+        }
+    }
 
     *vkav1_ref = (VkVideoDecodeAV1DpbSlotInfoKHR) {
         .sType = VK_STRUCTURE_TYPE_VIDEO_DECODE_AV1_DPB_SLOT_INFO_KHR,
@@ -490,8 +496,12 @@ static int vk_av1_start_frame(AVCodecContext          *avctx,
         }
     }
 
-    for (int i = 0; i < STD_VIDEO_AV1_NUM_REF_FRAMES; i++) {
-        ap->std_pic_info.OrderHints[i] = pic->order_hints[i];
+    if (dec->quirk_av1_offset) {
+        for (int i = 1; i < STD_VIDEO_AV1_NUM_REF_FRAMES; i++)
+            ap->std_pic_info.OrderHints[i - 1] = pic->order_hints[i];
+    } else {
+        for (int i = 0; i < STD_VIDEO_AV1_NUM_REF_FRAMES; i++)
+            ap->std_pic_info.OrderHints[i] = pic->order_hints[i];
     }
 
     for (int i = 0; i < STD_VIDEO_AV1_TOTAL_REFS_PER_FRAME; i++)
diff --git a/libavcodec/vulkan_decode.c b/libavcodec/vulkan_decode.c
index 9c6c2d4efb..5cb328d8ca 100644
--- a/libavcodec/vulkan_decode.c
+++ b/libavcodec/vulkan_decode.c
@@ -1115,6 +1115,7 @@ int ff_vk_decode_init(AVCodecContext *avctx)
     FFVulkanFunctions *vk;
     const VkVideoProfileInfoKHR *profile;
     const FFVulkanDecodeDescriptor *vk_desc;
+    const VkPhysicalDeviceDriverProperties *driver_props;
 
     VkVideoDecodeH264SessionParametersCreateInfoKHR h264_params = {
         .sType = VK_STRUCTURE_TYPE_VIDEO_DECODE_H264_SESSION_PARAMETERS_CREATE_INFO_KHR,
@@ -1276,6 +1277,13 @@ int ff_vk_decode_init(AVCodecContext *avctx)
         return AVERROR_EXTERNAL;
     }
 
+    driver_props = &dec->shared_ctx->s.driver_props;
+    if (driver_props->driverID == VK_DRIVER_ID_NVIDIA_PROPRIETARY &&
+        driver_props->conformanceVersion.major == 1 &&
+        driver_props->conformanceVersion.minor == 3 &&
+        driver_props->conformanceVersion.subminor == 8)
+        dec->quirk_av1_offset = 1;
+
     ff_vk_decode_flush(avctx);
 
     av_log(avctx, AV_LOG_VERBOSE, "Vulkan decoder initialization sucessful\n");
diff --git a/libavcodec/vulkan_decode.h b/libavcodec/vulkan_decode.h
index 7ba8b239cb..076af93499 100644
--- a/libavcodec/vulkan_decode.h
+++ b/libavcodec/vulkan_decode.h
@@ -72,6 +72,10 @@ typedef struct FFVulkanDecodeContext {
     int external_fg;   /* Oddity  #2 - hardware can't apply film grain */
     uint32_t frame_id_alloc_mask; /* For AV1 only */
 
+    /* Workaround for NVIDIA drivers tested with CTS version 1.3.8 for AV1.
+     * The tests were incorrect as the OrderHints were offset by 1. */
+    int quirk_av1_offset;
+
     /* Thread-local state below */
     struct HEVCHeaderSet *hevc_headers;
     size_t hevc_headers_size;
-- 
2.43.0.381.gb435a96ce8


[-- Attachment #3: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".