[FFmpeg-devel] [PATCH v2] avcodec/cuvidddec: Guess pixel format based on probed bit depth and chroma sampling

[Roman Arzumanyan <r.arzumanyan@visionlabs.ai>]

[-- Attachment #1: Type: text/plain, Size: 186 bytes --]

Hello world,

Please find v2 of the previous patch attached.
Both bit depth and chroma sampling (420 and 444 as supported by Nvdec) are
now taken into account when selecting sw_pix_fmt.

[-- Attachment #2: 0001-Guess-pixel-format-based-on-bit-depth-and-chroma-sam.patch --]
[-- Type: text/x-patch, Size: 2415 bytes --]

From a0c0c8497e75987ae771a466c9f0fce5c3ef106c Mon Sep 17 00:00:00 2001
From: Roman Arzumanyan <r.arzumanyan@visionlabs.ai>
Date: Thu, 1 Aug 2024 16:35:22 +0300
Subject: [PATCH] Guess pixel format based on bit depth and chroma sampling:

---
 libavcodec/cuviddec.c | 25 ++++++++++++++++++++++---
 1 file changed, 22 insertions(+), 3 deletions(-)

diff --git a/libavcodec/cuviddec.c b/libavcodec/cuviddec.c
index f88ad75e88..a7185fecbf 100644
--- a/libavcodec/cuviddec.c
+++ b/libavcodec/cuviddec.c
@@ -834,22 +834,41 @@ static av_cold int cuvid_decode_init(AVCodecContext *avctx)
     int ret = 0;
 
     enum AVPixelFormat pix_fmts[3] = { AV_PIX_FMT_CUDA,
-                                       AV_PIX_FMT_NV12,
+                                       AV_PIX_FMT_NONE,
                                        AV_PIX_FMT_NONE };
 
     int probed_width = avctx->coded_width ? avctx->coded_width : 1280;
     int probed_height = avctx->coded_height ? avctx->coded_height : 720;
-    int probed_bit_depth = 8;
+    int probed_bit_depth = 8, is_yuv444 = 0;
 
     const AVPixFmtDescriptor *probe_desc = av_pix_fmt_desc_get(avctx->pix_fmt);
     if (probe_desc && probe_desc->nb_components)
         probed_bit_depth = probe_desc->comp[0].depth;
 
+    if (probe_desc && !probe_desc->log2_chroma_w && !probe_desc->log2_chroma_h)
+        is_yuv444 = 1;
+
+    // Pick pixel format based on bit depth and chroma sampling.
+    // Only 420 and 444 sampling are supported by HW so far, no need to check for 422.
+    switch (probed_bit_depth) {
+    case 8:
+        pix_fmts[1] = is_yuv444 ? AV_PIX_FMT_YUV444P : AV_PIX_FMT_NV12;
+        break;        
+    case 10:
+        pix_fmts[1] = is_yuv444 ? AV_PIX_FMT_YUV444P10 : AV_PIX_FMT_P010;
+        break;
+    case 12:
+        pix_fmts[1] = is_yuv444 ? AV_PIX_FMT_YUV444P16 : AV_PIX_FMT_P016;
+        break;
+    default:
+        break;
+    }
+
     ctx->pkt = avctx->internal->in_pkt;
     // Accelerated transcoding scenarios with 'ffmpeg' require that the
     // pix_fmt be set to AV_PIX_FMT_CUDA early. The sw_pix_fmt, and the
     // pix_fmt for non-accelerated transcoding, do not need to be correct
-    // but need to be set to something. We arbitrarily pick NV12.
+    // but need to be set to something.
     ret = ff_get_format(avctx, pix_fmts);
     if (ret < 0) {
         av_log(avctx, AV_LOG_ERROR, "ff_get_format failed: %d\n", ret);
-- 
2.34.1


[-- Attachment #3: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".