[FFmpeg-devel] [PATCH 0/3] V5: VAAPI: Add high bit depth encode/decode support

[FFmpeg-devel] [PATCH 0/3] V5: VAAPI: Add high bit depth encode/decode support

[Philip Langdale]

This changeset fills in support for the remaining high bit depth
formats the VAAPI exposes. This requires adding more weird Microsoft
pixel formats and then mapping them in the VAAPI code. I've also
enabled hw mapping between VAAPI and vulkan for sufficiently simple
formats.

V5: Switched to using alpha-less variants of Y410 and Y412 (called XV30
    and XV36 in libdrm) to avoid forcing the overhead of alpha handling
    in situations where it can't actually be used.

Philip Langdale (3):
  lavu/pixfmt: Add P012, Y212, XV30, and XV36 formats
  lavc/vaapi: Add support for remaining 10/12bit profiles
  lavu/hwcontext_vulkan: support mapping VUYX, P012, and XV36

 libavcodec/hevcdec.c             |  8 +++
 libavcodec/vaapi_decode.c        | 13 +++++
 libavcodec/vaapi_encode.c        |  4 ++
 libavcodec/vaapi_encode_h265.c   |  4 ++
 libavcodec/vaapi_encode_vp9.c    |  1 +
 libavcodec/vaapi_hevc.c          | 11 +++-
 libavcodec/vp9.c                 |  2 +
 libavutil/hwcontext_vaapi.c      | 25 +++++++++
 libavutil/hwcontext_vulkan.c     | 10 ++++
 libavutil/pixdesc.c              | 95 +++++++++++++++++++++++++++++++-
 libavutil/pixfmt.h               | 16 ++++++
 tests/ref/fate/imgutils          |  8 +++
 tests/ref/fate/sws-pixdesc-query | 38 +++++++++++++
 13 files changed, 233 insertions(+), 2 deletions(-)

-- 
2.34.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 1/3] lavu/pixfmt: Add P012, Y212, XV30, and XV36 formats

[Philip Langdale]

These are the formats we want/need to use when dealing with the Intel
VAAPI decoder for 12bit 4:2:0, 12bit 4:2:2, 10bit 4:4:4 and 12bit 4:4:4
respectively.

As with the already supported Y210 and YUVX (XVUY) formats, they are
based on formats Microsoft picked as their preferred 4:2:2 and 4:4:4
video formats, and Intel ran with it.

P12 and Y212 are simply an extension of 10 bit formats to say 12 bits
will be used, with 4 unused bits instead of 6.

XV30, and XV36, as exotic as they sound, are variants of Y410 and Y412
where the alpha channel is left formally undefined. We prefer these
over the alpha versions because the hardware cannot actually do
anything with the alpha channel and respecting it is just overhead.

Y412/XV46 is a normal looking packed 4 channel format where each
channel is 16bits wide but only the 12msb are used (like P012).

Y410/XV30 packs three 10bit channels in 32bits with 2bits of alpha,
like A/X2RGB10 style formats. This annoying layout forced me to define
the BE version as a bitstream format. It seems like our pixdesc
infrastructure can handle the LE version being byte-defined, but not
when it's reversed. If there's a better way to handle this, please
let me know. Our existing X2 formats all have the 2 bits at the MSB
end, but this format places them at the LSB end and that seems to be
the root of the problem.

Signed-off-by: Philip Langdale <philipl@overt.org>
---
 libavutil/pixdesc.c              | 95 +++++++++++++++++++++++++++++++-
 libavutil/pixfmt.h               | 16 ++++++
 tests/ref/fate/imgutils          |  8 +++
 tests/ref/fate/sws-pixdesc-query | 38 +++++++++++++
 4 files changed, 156 insertions(+), 1 deletion(-)

diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c
index 79ebfd3f16..d7c6ebfdc4 100644
--- a/libavutil/pixdesc.c
+++ b/libavutil/pixdesc.c
@@ -2147,6 +2147,30 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = {
         },
         .flags = AV_PIX_FMT_FLAG_PLANAR | AV_PIX_FMT_FLAG_BE,
     },
+    [AV_PIX_FMT_P012LE] = {
+        .name = "p012le",
+        .nb_components = 3,
+        .log2_chroma_w = 1,
+        .log2_chroma_h = 1,
+        .comp = {
+            { 0, 2, 0, 4, 12 },        /* Y */
+            { 1, 4, 0, 4, 12 },        /* U */
+            { 1, 4, 2, 4, 12 },        /* V */
+        },
+        .flags = AV_PIX_FMT_FLAG_PLANAR,
+    },
+    [AV_PIX_FMT_P012BE] = {
+        .name = "p012be",
+        .nb_components = 3,
+        .log2_chroma_w = 1,
+        .log2_chroma_h = 1,
+        .comp = {
+            { 0, 2, 0, 4, 12 },        /* Y */
+            { 1, 4, 0, 4, 12 },        /* U */
+            { 1, 4, 2, 4, 12 },        /* V */
+        },
+        .flags = AV_PIX_FMT_FLAG_PLANAR | AV_PIX_FMT_FLAG_BE,
+    },
     [AV_PIX_FMT_P016LE] = {
         .name = "p016le",
         .nb_components = 3,
@@ -2543,6 +2567,75 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = {
         .flags = AV_PIX_FMT_FLAG_RGB | AV_PIX_FMT_FLAG_ALPHA |
                  AV_PIX_FMT_FLAG_FLOAT,
     },
+    [AV_PIX_FMT_Y212LE] = {
+        .name = "y212le",
+        .nb_components = 3,
+        .log2_chroma_w = 1,
+        .log2_chroma_h = 0,
+        .comp = {
+            { 0, 4, 0, 4, 12 },        /* Y */
+            { 0, 8, 2, 4, 12 },        /* U */
+            { 0, 8, 6, 4, 12 },        /* V */
+        },
+    },
+    [AV_PIX_FMT_Y212BE] = {
+        .name = "y212be",
+        .nb_components = 3,
+        .log2_chroma_w = 1,
+        .log2_chroma_h = 0,
+        .comp = {
+            { 0, 4, 0, 4, 12 },        /* Y */
+            { 0, 8, 2, 4, 12 },        /* U */
+            { 0, 8, 6, 4, 12 },        /* V */
+        },
+        .flags = AV_PIX_FMT_FLAG_BE,
+    },
+    [AV_PIX_FMT_XV30LE] = {
+        .name = "xv30le",
+        .nb_components= 3,
+        .log2_chroma_w= 0,
+        .log2_chroma_h= 0,
+        .comp = {
+            { 0, 4, 1, 2, 10 },       /* Y */
+            { 0, 4, 0, 0, 10 },       /* U */
+            { 0, 4, 2, 4, 10 },       /* V */
+        },
+    },
+    [AV_PIX_FMT_XV30BE] = {
+        .name = "xv30be",
+        .nb_components= 3,
+        .log2_chroma_w= 0,
+        .log2_chroma_h= 0,
+        .comp = {
+            { 0, 32, 10, 0, 10 },       /* Y */
+            { 0, 32,  0, 0, 10 },       /* U */
+            { 0, 32, 20, 0, 10 },       /* V */
+        },
+        .flags = AV_PIX_FMT_FLAG_BE | AV_PIX_FMT_FLAG_BITSTREAM,
+    },
+    [AV_PIX_FMT_XV36LE] = {
+        .name = "xv36le",
+        .nb_components= 3,
+        .log2_chroma_w= 0,
+        .log2_chroma_h= 0,
+        .comp = {
+            { 0, 8, 2, 4, 12 },       /* Y */
+            { 0, 8, 0, 4, 12 },       /* U */
+            { 0, 8, 4, 4, 12 },       /* V */
+        },
+    },
+    [AV_PIX_FMT_XV36BE] = {
+        .name = "xv36be",
+        .nb_components= 3,
+        .log2_chroma_w= 0,
+        .log2_chroma_h= 0,
+        .comp = {
+            { 0, 8, 2, 4, 12 },       /* Y */
+            { 0, 8, 0, 4, 12 },       /* U */
+            { 0, 8, 4, 4, 12 },       /* V */
+        },
+        .flags = AV_PIX_FMT_FLAG_BE,
+    },
 };
 
 static const char * const color_range_names[] = {
@@ -2778,7 +2871,7 @@ void ff_check_pixfmt_descriptors(void){
 
         if (!d->name && !d->nb_components && !d->log2_chroma_w && !d->log2_chroma_h && !d->flags)
             continue;
-//         av_log(NULL, AV_LOG_DEBUG, "Checking: %s\n", d->name);
+        av_log(NULL, AV_LOG_INFO, "Checking: %s\n", d->name);
         av_assert0(d->log2_chroma_w <= 3);
         av_assert0(d->log2_chroma_h <= 3);
         av_assert0(d->nb_components <= 4);
diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h
index 7d45561395..a1c4c9fb75 100644
--- a/libavutil/pixfmt.h
+++ b/libavutil/pixfmt.h
@@ -374,6 +374,18 @@ enum AVPixelFormat {
 
     AV_PIX_FMT_VUYX,        ///< packed VUYX 4:4:4, 32bpp, Variant of VUYA where alpha channel is left undefined
 
+    AV_PIX_FMT_P012LE,      ///< like NV12, with 12bpp per component, data in the high bits, zeros in the low bits, little-endian
+    AV_PIX_FMT_P012BE,      ///< like NV12, with 12bpp per component, data in the high bits, zeros in the low bits, big-endian
+
+    AV_PIX_FMT_Y212BE,      ///< packed YUV 4:2:2 like YUYV422, 24bpp, data in the high bits, zeros in the low bits, big-endian
+    AV_PIX_FMT_Y212LE,      ///< packed YUV 4:2:2 like YUYV422, 24bpp, data in the high bits, zeros in the low bits, little-endian
+
+    AV_PIX_FMT_XV30BE,      ///< packed XVYU 4:4:4, 32bpp, (msb)2X 10V 10Y 10U(lsb), big-endian, variant of Y410 where alpha channel is left undefined
+    AV_PIX_FMT_XV30LE,      ///< packed XVYU 4:4:4, 32bpp, (msb)2X 10V 10Y 10U(lsb), little-endian, variant of Y410 where alpha channel is left undefined
+
+    AV_PIX_FMT_XV36BE,      ///< packed XVYU 4:4:4, 48bpp, data in the high bits, zeros in the low bits, big-endian, variant of Y412 where alpha channel is left undefined
+    AV_PIX_FMT_XV36LE,      ///< packed XVYU 4:4:4, 48bpp, data in the high bits, zeros in the low bits, little-endian, variant of Y412 where alpha channel is left undefined
+
     AV_PIX_FMT_NB         ///< number of pixel formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions
 };
 
@@ -460,9 +472,13 @@ enum AVPixelFormat {
 #define AV_PIX_FMT_NV20       AV_PIX_FMT_NE(NV20BE,  NV20LE)
 #define AV_PIX_FMT_AYUV64     AV_PIX_FMT_NE(AYUV64BE, AYUV64LE)
 #define AV_PIX_FMT_P010       AV_PIX_FMT_NE(P010BE,  P010LE)
+#define AV_PIX_FMT_P012       AV_PIX_FMT_NE(P012BE,  P012LE)
 #define AV_PIX_FMT_P016       AV_PIX_FMT_NE(P016BE,  P016LE)
 
 #define AV_PIX_FMT_Y210       AV_PIX_FMT_NE(Y210BE,  Y210LE)
+#define AV_PIX_FMT_Y212       AV_PIX_FMT_NE(Y212BE,  Y212LE)
+#define AV_PIX_FMT_XV30       AV_PIX_FMT_NE(XV30BE,  XV30LE)
+#define AV_PIX_FMT_XV36       AV_PIX_FMT_NE(XV36BE,  XV36LE)
 #define AV_PIX_FMT_X2RGB10    AV_PIX_FMT_NE(X2RGB10BE, X2RGB10LE)
 #define AV_PIX_FMT_X2BGR10    AV_PIX_FMT_NE(X2BGR10BE, X2BGR10LE)
 
diff --git a/tests/ref/fate/imgutils b/tests/ref/fate/imgutils
index 47b73b1b64..de73513e7c 100644
--- a/tests/ref/fate/imgutils
+++ b/tests/ref/fate/imgutils
@@ -250,3 +250,11 @@ vuya            planes: 1, linesizes: 256   0   0   0, plane_sizes: 12288     0
 rgbaf16be       planes: 1, linesizes: 512   0   0   0, plane_sizes: 24576     0     0     0, plane_offsets:     0     0     0, total_size: 24576
 rgbaf16le       planes: 1, linesizes: 512   0   0   0, plane_sizes: 24576     0     0     0, plane_offsets:     0     0     0, total_size: 24576
 vuyx            planes: 1, linesizes: 256   0   0   0, plane_sizes: 12288     0     0     0, plane_offsets:     0     0     0, total_size: 12288
+p012le          planes: 2, linesizes: 128 128   0   0, plane_sizes:  6144  3072     0     0, plane_offsets:  6144     0     0, total_size: 9216
+p012be          planes: 2, linesizes: 128 128   0   0, plane_sizes:  6144  3072     0     0, plane_offsets:  6144     0     0, total_size: 9216
+y212be          planes: 1, linesizes: 256   0   0   0, plane_sizes: 12288     0     0     0, plane_offsets:     0     0     0, total_size: 12288
+y212le          planes: 1, linesizes: 256   0   0   0, plane_sizes: 12288     0     0     0, plane_offsets:     0     0     0, total_size: 12288
+xv30be          planes: 1, linesizes: 256   0   0   0, plane_sizes: 12288     0     0     0, plane_offsets:     0     0     0, total_size: 12288
+xv30le          planes: 1, linesizes: 256   0   0   0, plane_sizes: 12288     0     0     0, plane_offsets:     0     0     0, total_size: 12288
+xv36be          planes: 1, linesizes: 512   0   0   0, plane_sizes: 24576     0     0     0, plane_offsets:     0     0     0, total_size: 24576
+xv36le          planes: 1, linesizes: 512   0   0   0, plane_sizes: 24576     0     0     0, plane_offsets:     0     0     0, total_size: 24576
diff --git a/tests/ref/fate/sws-pixdesc-query b/tests/ref/fate/sws-pixdesc-query
index f54372d364..20fc596ce9 100644
--- a/tests/ref/fate/sws-pixdesc-query
+++ b/tests/ref/fate/sws-pixdesc-query
@@ -63,6 +63,8 @@ isNBPS:
   nv20le
   p010be
   p010le
+  p012be
+  p012le
   p210be
   p210le
   p410be
@@ -71,10 +73,16 @@ isNBPS:
   x2bgr10le
   x2rgb10be
   x2rgb10le
+  xv30be
+  xv30le
+  xv36be
+  xv36le
   xyz12be
   xyz12le
   y210be
   y210le
+  y212be
+  y212le
   yuv420p10be
   yuv420p10le
   yuv420p12be
@@ -149,6 +157,7 @@ isBE:
   grayf32be
   nv20be
   p010be
+  p012be
   p016be
   p210be
   p216be
@@ -162,8 +171,11 @@ isBE:
   rgbaf16be
   x2bgr10be
   x2rgb10be
+  xv30be
+  xv36be
   xyz12be
   y210be
+  y212be
   ya16be
   yuv420p10be
   yuv420p12be
@@ -206,6 +218,8 @@ isYUV:
   nv42
   p010be
   p010le
+  p012be
+  p012le
   p016be
   p016le
   p210be
@@ -220,10 +234,16 @@ isYUV:
   uyyvyy411
   vuya
   vuyx
+  xv30be
+  xv30le
+  xv36be
+  xv36le
   xyz12be
   xyz12le
   y210be
   y210le
+  y212be
+  y212le
   ya16be
   ya16le
   ya8
@@ -310,6 +330,8 @@ isPlanarYUV:
   nv42
   p010be
   p010le
+  p012be
+  p012le
   p016be
   p016le
   p210be
@@ -401,6 +423,8 @@ isSemiPlanarYUV:
   nv42
   p010be
   p010le
+  p012be
+  p012le
   p016be
   p016le
   p210be
@@ -759,10 +783,16 @@ Packed:
   x2bgr10le
   x2rgb10be
   x2rgb10le
+  xv30be
+  xv30le
+  xv36be
+  xv36le
   xyz12be
   xyz12le
   y210be
   y210le
+  y212be
+  y212le
   ya16be
   ya16le
   ya8
@@ -801,6 +831,8 @@ Planar:
   nv42
   p010be
   p010le
+  p012be
+  p012le
   p016be
   p016le
   p210be
@@ -973,14 +1005,20 @@ usePal:
 DataInHighBits:
   p010be
   p010le
+  p012be
+  p012le
   p210be
   p210le
   p410be
   p410le
+  xv36be
+  xv36le
   xyz12be
   xyz12le
   y210be
   y210le
+  y212be
+  y212le
 
 SwappedChroma:
   nv21
-- 
2.34.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles

[Philip Langdale]

With the necessary pixel formats defined, we can now expose support for
the remaining 10/12bit combinations that VAAPI can handle.

Specifically, we are adding support for:

* HEVC
** 12bit 420
** 10bit 422
** 12bit 422
** 10bit 444
** 12bit 444

* VP9
** 10bit 444
** 12bit 444

These obviously require actual hardware support to be usable, but where
that exists, it is now enabled.

Note that unlike YUVA/YUVX, the Intel driver does not formally expose
support for the alphaless formats XV30 and XV360, and so we are
implicitly discarding the alpha from the decoder and passing undefined
values for the alpha to the encoder. If a future encoder iteration was
to actually do something with the alpha bits, we would need to use a
formal alpha capable format or the encoder would need to explicitly
accept the alphaless format.

Signed-off-by: Philip Langdale <philipl@overt.org>
---
 libavcodec/hevcdec.c           |  8 ++++++++
 libavcodec/vaapi_decode.c      | 13 +++++++++++++
 libavcodec/vaapi_encode.c      |  4 ++++
 libavcodec/vaapi_encode_h265.c |  4 ++++
 libavcodec/vaapi_encode_vp9.c  |  1 +
 libavcodec/vaapi_hevc.c        | 11 ++++++++++-
 libavcodec/vp9.c               |  2 ++
 libavutil/hwcontext_vaapi.c    | 25 +++++++++++++++++++++++++
 8 files changed, 67 insertions(+), 1 deletion(-)

diff --git a/libavcodec/hevcdec.c b/libavcodec/hevcdec.c
index 477d6d9d36..fd23e791a4 100644
--- a/libavcodec/hevcdec.c
+++ b/libavcodec/hevcdec.c
@@ -481,11 +481,19 @@ static enum AVPixelFormat get_format(HEVCContext *s, const HEVCSPS *sps)
 #endif
     case AV_PIX_FMT_YUV420P12:
     case AV_PIX_FMT_YUV444P12:
+#if CONFIG_HEVC_VAAPI_HWACCEL
+       *fmt++ = AV_PIX_FMT_VAAPI;
+#endif
 #if CONFIG_HEVC_VDPAU_HWACCEL
         *fmt++ = AV_PIX_FMT_VDPAU;
 #endif
 #if CONFIG_HEVC_NVDEC_HWACCEL
         *fmt++ = AV_PIX_FMT_CUDA;
+#endif
+        break;
+    case AV_PIX_FMT_YUV422P12:
+#if CONFIG_HEVC_VAAPI_HWACCEL
+       *fmt++ = AV_PIX_FMT_VAAPI;
 #endif
         break;
     }
diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c
index 8c13a4f098..134f10eca5 100644
--- a/libavcodec/vaapi_decode.c
+++ b/libavcodec/vaapi_decode.c
@@ -262,6 +262,9 @@ static const struct {
     MAP(YUY2, YUYV422),
 #ifdef VA_FOURCC_Y210
     MAP(Y210,    Y210),
+#endif
+#ifdef VA_FOURCC_Y212
+    MAP(Y212,    Y212),
 #endif
     // 4:4:0
     MAP(422V, YUV440P),
@@ -269,11 +272,20 @@ static const struct {
     MAP(444P, YUV444P),
 #ifdef VA_FOURCC_XYUV
     MAP(XYUV, VUYX),
+#endif
+#ifdef VA_FOURCC_Y410
+    MAP(Y410,    XV30),
+#endif
+#ifdef VA_FOURCC_Y412
+    MAP(Y412,    XV36),
 #endif
     // 4:2:0 10-bit
 #ifdef VA_FOURCC_P010
     MAP(P010, P010),
 #endif
+#ifdef VA_FOURCC_P012
+    MAP(P012, P012),
+#endif
 #ifdef VA_FOURCC_I010
     MAP(I010, YUV420P10),
 #endif
@@ -417,6 +429,7 @@ static const struct {
 #if VA_CHECK_VERSION(0, 39, 0)
     MAP(VP9,         VP9_1,           VP9Profile1 ),
     MAP(VP9,         VP9_2,           VP9Profile2 ),
+    MAP(VP9,         VP9_3,           VP9Profile3 ),
 #endif
 #if VA_CHECK_VERSION(1, 8, 0)
     MAP(AV1,         AV1_MAIN,        AV1Profile0),
diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
index 2dc5c96f7b..9a58661b51 100644
--- a/libavcodec/vaapi_encode.c
+++ b/libavcodec/vaapi_encode.c
@@ -1305,7 +1305,11 @@ static const VAAPIEncodeRTFormat vaapi_encode_rt_formats[] = {
     { "YUV420",    VA_RT_FORMAT_YUV420,        8, 3, 1, 1 },
     { "YUV422",    VA_RT_FORMAT_YUV422,        8, 3, 1, 0 },
 #if VA_CHECK_VERSION(1, 2, 0)
+    { "YUV420_12", VA_RT_FORMAT_YUV420_12,    12, 3, 1, 1 },
     { "YUV422_10", VA_RT_FORMAT_YUV422_10,    10, 3, 1, 0 },
+    { "YUV422_12", VA_RT_FORMAT_YUV422_12,    12, 3, 1, 0 },
+    { "YUV444_10", VA_RT_FORMAT_YUV444_10,    10, 3, 0, 0 },
+    { "YUV444_12", VA_RT_FORMAT_YUV444_12,    12, 3, 0, 0 },
 #endif
     { "YUV444",    VA_RT_FORMAT_YUV444,        8, 3, 0, 0 },
     { "XYUV",      VA_RT_FORMAT_YUV444,        8, 3, 0, 0 },
diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c
index 967d71e998..90f44acff1 100644
--- a/libavcodec/vaapi_encode_h265.c
+++ b/libavcodec/vaapi_encode_h265.c
@@ -1276,9 +1276,13 @@ static const VAAPIEncodeProfile vaapi_encode_h265_profiles[] = {
     { FF_PROFILE_HEVC_REXT,    10, 3, 1, 1, VAProfileHEVCMain10     },
 #endif
 #if VA_CHECK_VERSION(1, 2, 0)
+    { FF_PROFILE_HEVC_REXT,    12, 3, 1, 1, VAProfileHEVCMain12 },
     { FF_PROFILE_HEVC_REXT,     8, 3, 1, 0, VAProfileHEVCMain422_10 },
     { FF_PROFILE_HEVC_REXT,    10, 3, 1, 0, VAProfileHEVCMain422_10 },
+    { FF_PROFILE_HEVC_REXT,    12, 3, 1, 0, VAProfileHEVCMain422_12 },
     { FF_PROFILE_HEVC_REXT,     8, 3, 0, 0, VAProfileHEVCMain444 },
+    { FF_PROFILE_HEVC_REXT,    10, 3, 0, 0, VAProfileHEVCMain444_10 },
+    { FF_PROFILE_HEVC_REXT,    12, 3, 0, 0, VAProfileHEVCMain444_12 },
 #endif
     { FF_PROFILE_UNKNOWN }
 };
diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c
index 9530b2f462..38c167d2b6 100644
--- a/libavcodec/vaapi_encode_vp9.c
+++ b/libavcodec/vaapi_encode_vp9.c
@@ -230,6 +230,7 @@ static const VAAPIEncodeProfile vaapi_encode_vp9_profiles[] = {
     { FF_PROFILE_VP9_0,  8, 3, 1, 1, VAProfileVP9Profile0 },
     { FF_PROFILE_VP9_1,  8, 3, 0, 0, VAProfileVP9Profile1 },
     { FF_PROFILE_VP9_2, 10, 3, 1, 1, VAProfileVP9Profile2 },
+    { FF_PROFILE_VP9_3, 10, 3, 0, 0, VAProfileVP9Profile3 },
     { FF_PROFILE_UNKNOWN }
 };
 
diff --git a/libavcodec/vaapi_hevc.c b/libavcodec/vaapi_hevc.c
index d82975979a..20fb36adfa 100644
--- a/libavcodec/vaapi_hevc.c
+++ b/libavcodec/vaapi_hevc.c
@@ -567,15 +567,24 @@ VAProfile ff_vaapi_parse_hevc_rext_profile(AVCodecContext *avctx)
     }
 
 #if VA_CHECK_VERSION(1, 2, 0)
-    if (!strcmp(profile->name, "Main 4:2:2 10") ||
+    if (!strcmp(profile->name, "Main 12") ||
+        !strcmp(profile->name, "Main 12 Intra"))
+        return VAProfileHEVCMain12;
+    else if (!strcmp(profile->name, "Main 4:2:2 10") ||
         !strcmp(profile->name, "Main 4:2:2 10 Intra"))
         return VAProfileHEVCMain422_10;
+    else if (!strcmp(profile->name, "Main 4:2:2 12") ||
+        !strcmp(profile->name, "Main 4:2:2 12 Intra"))
+        return VAProfileHEVCMain422_12;
     else if (!strcmp(profile->name, "Main 4:4:4") ||
              !strcmp(profile->name, "Main 4:4:4 Intra"))
         return VAProfileHEVCMain444;
     else if (!strcmp(profile->name, "Main 4:4:4 10") ||
              !strcmp(profile->name, "Main 4:4:4 10 Intra"))
         return VAProfileHEVCMain444_10;
+    else if (!strcmp(profile->name, "Main 4:4:4 12") ||
+             !strcmp(profile->name, "Main 4:4:4 12 Intra"))
+        return VAProfileHEVCMain444_12;
 #else
     av_log(avctx, AV_LOG_WARNING, "HEVC profile %s is "
            "not supported with this VA version.\n", profile->name);
diff --git a/libavcodec/vp9.c b/libavcodec/vp9.c
index 621627ddc5..8105956775 100644
--- a/libavcodec/vp9.c
+++ b/libavcodec/vp9.c
@@ -235,6 +235,8 @@ static int update_size(AVCodecContext *avctx, int w, int h)
 #endif
             break;
         case AV_PIX_FMT_YUV444P:
+        case AV_PIX_FMT_YUV444P10:
+        case AV_PIX_FMT_YUV444P12:
 #if CONFIG_VP9_VAAPI_HWACCEL
             *fmtp++ = AV_PIX_FMT_VAAPI;
 #endif
diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c
index 78205425ee..9ba5225ad2 100644
--- a/libavutil/hwcontext_vaapi.c
+++ b/libavutil/hwcontext_vaapi.c
@@ -121,6 +121,9 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = {
     MAP(YUY2, YUV422,  YUYV422, 0),
 #ifdef VA_FOURCC_Y210
     MAP(Y210, YUV422_10,  Y210, 0),
+#endif
+#ifdef VA_FOURCC_Y212
+    MAP(Y212, YUV422_12,  Y212, 0),
 #endif
     MAP(411P, YUV411,  YUV411P, 0),
     MAP(422V, YUV422,  YUV440P, 0),
@@ -131,6 +134,9 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = {
     MAP(Y800, YUV400,  GRAY8,   0),
 #ifdef VA_FOURCC_P010
     MAP(P010, YUV420_10BPP, P010, 0),
+#endif
+#ifdef VA_FOURCC_P012
+    MAP(P012, YUV420_12, P012, 0),
 #endif
     MAP(BGRA, RGB32,   BGRA, 0),
     MAP(BGRX, RGB32,   BGR0, 0),
@@ -145,6 +151,16 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = {
 #ifdef VA_FOURCC_X2R10G10B10
     MAP(X2R10G10B10, RGB32_10, X2RGB10, 0),
 #endif
+#ifdef VA_FOURCC_Y410
+    // libva doesn't include a fourcc for XV30 and the driver only declares
+    // support for Y410, so we must fudge the mapping here.
+    MAP(Y410, YUV444_10,  XV30, 0),
+#endif
+#ifdef VA_FOURCC_Y412
+    // libva doesn't include a fourcc for XV36 and the driver only declares
+    // support for Y412, so we must fudge the mapping here.
+    MAP(Y412, YUV444_12,  XV36, 0),
+#endif
 };
 #undef MAP
 
@@ -1000,6 +1016,9 @@ static const struct {
     DRM_MAP(NV12, 1, DRM_FORMAT_NV12),
 #if defined(VA_FOURCC_P010) && defined(DRM_FORMAT_R16)
     DRM_MAP(P010, 2, DRM_FORMAT_R16, DRM_FORMAT_RG1616),
+#endif
+#if defined(VA_FOURCC_P012) && defined(DRM_FORMAT_R16)
+    DRM_MAP(P012, 2, DRM_FORMAT_R16, DRM_FORMAT_RG1616),
 #endif
     DRM_MAP(BGRA, 1, DRM_FORMAT_ARGB8888),
     DRM_MAP(BGRX, 1, DRM_FORMAT_XRGB8888),
@@ -1014,6 +1033,12 @@ static const struct {
 #if defined(VA_FOURCC_XYUV) && defined(DRM_FORMAT_XYUV8888)
     DRM_MAP(XYUV, 1, DRM_FORMAT_XYUV8888),
 #endif
+#if defined(VA_FOURCC_Y412) && defined(DRM_FORMAT_XVYU2101010)
+    DRM_MAP(Y410, 1, DRM_FORMAT_XVYU2101010),
+#endif
+#if defined(VA_FOURCC_Y412) && defined(DRM_FORMAT_XVYU12_16161616)
+    DRM_MAP(Y412, 1, DRM_FORMAT_XVYU12_16161616),
+#endif
 };
 #undef DRM_MAP
 
-- 
2.34.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 3/3] lavu/hwcontext_vulkan: support mapping VUYX, P012, and XV36

[Philip Langdale]

If we want to be able to map between VAAPI and Vulkan (to do Vulkan
filtering), we need to have matching formats on each side.

The mappings here are not exact. In the same way that P010 is still
mapped to full 16 bit formats, P012 has to be mapped that way as well.

Similarly, VUYX has to be mapped to an alpha-equipped format, and XV36
has to be mapped to a fully 16bit alpha-equipped format.

While Vulkan seems to fundamentally lack formats with an undefined,
but physically present, alpha channel, it has have 10X6 and 12X4
formats that you could imagine using for P010, P012 and XV36, but these
formats don't support the STORAGE usage flag. Today, hwcontext_vulkan
requires all formats to be storable because it wants to be able to use
them to create writable images. Until that changes, which might happen,
we have to restrict the set of formats we use.

Finally, when mapping a Vulkan image back to vaapi, I observed that
the VK_FORMAT_R16G16B16A16_UNORM format we have to use for XV36 going
to Vulkan is mapped to Y416 when going to vaapi (which makes sense as
it's the exact matching format) so I had to add an entry for it even
though we don't use it directly.

Signed-off-by: Philip Langdale <philipl@overt.org>
---
 libavutil/hwcontext_vulkan.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c
index 237caa4bc0..cabf77db95 100644
--- a/libavutil/hwcontext_vulkan.c
+++ b/libavutil/hwcontext_vulkan.c
@@ -173,6 +173,7 @@ static const struct {
     { AV_PIX_FMT_NV12, { VK_FORMAT_R8_UNORM, VK_FORMAT_R8G8_UNORM } },
     { AV_PIX_FMT_NV21, { VK_FORMAT_R8_UNORM, VK_FORMAT_R8G8_UNORM } },
     { AV_PIX_FMT_P010, { VK_FORMAT_R16_UNORM, VK_FORMAT_R16G16_UNORM } },
+    { AV_PIX_FMT_P012, { VK_FORMAT_R16_UNORM, VK_FORMAT_R16G16_UNORM } },
     { AV_PIX_FMT_P016, { VK_FORMAT_R16_UNORM, VK_FORMAT_R16G16_UNORM } },
 
     { AV_PIX_FMT_NV16, { VK_FORMAT_R8_UNORM, VK_FORMAT_R8G8_UNORM } },
@@ -210,6 +211,9 @@ static const struct {
     { AV_PIX_FMT_YUVA444P12, { VK_FORMAT_R16_UNORM, VK_FORMAT_R16_UNORM, VK_FORMAT_R16_UNORM, VK_FORMAT_R16_UNORM } },
     { AV_PIX_FMT_YUVA444P16, { VK_FORMAT_R16_UNORM, VK_FORMAT_R16_UNORM, VK_FORMAT_R16_UNORM, VK_FORMAT_R16_UNORM } },
 
+    { AV_PIX_FMT_VUYX,   { VK_FORMAT_R8G8B8A8_UNORM } },
+    { AV_PIX_FMT_XV36,   { VK_FORMAT_R16G16B16A16_UNORM } },
+
     { AV_PIX_FMT_BGRA,   { VK_FORMAT_B8G8R8A8_UNORM } },
     { AV_PIX_FMT_RGBA,   { VK_FORMAT_R8G8B8A8_UNORM } },
     { AV_PIX_FMT_RGB24,  { VK_FORMAT_R8G8B8_UNORM } },
@@ -2629,6 +2633,12 @@ static const struct {
     { DRM_FORMAT_XRGB8888, VK_FORMAT_B8G8R8A8_UNORM },
     { DRM_FORMAT_ABGR8888, VK_FORMAT_R8G8B8A8_UNORM },
     { DRM_FORMAT_XBGR8888, VK_FORMAT_R8G8B8A8_UNORM },
+
+    { DRM_FORMAT_XYUV8888, VK_FORMAT_R8G8B8A8_UNORM     },
+    { DRM_FORMAT_XVYU12_16161616, VK_FORMAT_R16G16B16A16_UNORM} ,
+    // As we had to map XV36 to a 16bit Vulkan format, reverse mapping will
+    // end up yielding Y416 as the DRM format, so we need to recognise it.
+    { DRM_FORMAT_Y416,     VK_FORMAT_R16G16B16A16_UNORM },
 };
 
 static inline VkFormat drm_to_vulkan_fmt(uint32_t drm_fourcc)
-- 
2.34.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/3] V5: VAAPI: Add high bit depth encode/decode support

[Philip Langdale]

On Thu, 25 Aug 2022 19:17:33 -0700
Philip Langdale <philipl@overt.org> wrote:

> This changeset fills in support for the remaining high bit depth
> formats the VAAPI exposes. This requires adding more weird Microsoft
> pixel formats and then mapping them in the VAAPI code. I've also
> enabled hw mapping between VAAPI and vulkan for sufficiently simple
> formats.
> 
> V5: Switched to using alpha-less variants of Y410 and Y412 (called
> XV30 and XV36 in libdrm) to avoid forcing the overhead of alpha
> handling in situations where it can't actually be used.
> 
> Philip Langdale (3):
>   lavu/pixfmt: Add P012, Y212, XV30, and XV36 formats
>   lavc/vaapi: Add support for remaining 10/12bit profiles
>   lavu/hwcontext_vulkan: support mapping VUYX, P012, and XV36
> 
>  libavcodec/hevcdec.c             |  8 +++
>  libavcodec/vaapi_decode.c        | 13 +++++
>  libavcodec/vaapi_encode.c        |  4 ++
>  libavcodec/vaapi_encode_h265.c   |  4 ++
>  libavcodec/vaapi_encode_vp9.c    |  1 +
>  libavcodec/vaapi_hevc.c          | 11 +++-
>  libavcodec/vp9.c                 |  2 +
>  libavutil/hwcontext_vaapi.c      | 25 +++++++++
>  libavutil/hwcontext_vulkan.c     | 10 ++++
>  libavutil/pixdesc.c              | 95
> +++++++++++++++++++++++++++++++- libavutil/pixfmt.h               |
> 16 ++++++ tests/ref/fate/imgutils          |  8 +++
>  tests/ref/fate/sws-pixdesc-query | 38 +++++++++++++
>  13 files changed, 233 insertions(+), 2 deletions(-)
> 

I will push these this weekend if there aren't any comments.


--phil
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/3] lavu/pixfmt: Add P012, Y212, XV30, and XV36 formats

[James Almer]

On 8/25/2022 11:17 PM, Philip Langdale wrote:
>   static const char * const color_range_names[] = {
> @@ -2778,7 +2871,7 @@ void ff_check_pixfmt_descriptors(void){
>   
>           if (!d->name && !d->nb_components && !d->log2_chroma_w && !d->log2_chroma_h && !d->flags)
>               continue;
> -//         av_log(NULL, AV_LOG_DEBUG, "Checking: %s\n", d->name);
> +        av_log(NULL, AV_LOG_INFO, "Checking: %s\n", d->name);
>           av_assert0(d->log2_chroma_w <= 3);
>           av_assert0(d->log2_chroma_h <= 3);
>           av_assert0(d->nb_components <= 4);

Unintended change?
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/3] lavu/pixfmt: Add P012, Y212, XV30, and XV36 formats

[Philip Langdale]

On Mon, 5 Sep 2022 11:51:56 -0300
James Almer <jamrial@gmail.com> wrote:

> On 8/25/2022 11:17 PM, Philip Langdale wrote:
> >   static const char * const color_range_names[] = {
> > @@ -2778,7 +2871,7 @@ void ff_check_pixfmt_descriptors(void){
> >   
> >           if (!d->name && !d->nb_components && !d->log2_chroma_w &&
> > !d->log2_chroma_h && !d->flags) continue;
> > -//         av_log(NULL, AV_LOG_DEBUG, "Checking: %s\n", d->name);
> > +        av_log(NULL, AV_LOG_INFO, "Checking: %s\n", d->name);
> >           av_assert0(d->log2_chroma_w <= 3);
> >           av_assert0(d->log2_chroma_h <= 3);
> >           av_assert0(d->nb_components <= 4);  
> 
> Unintended change?

Actually intended. Because if there is a problem and the assert fails,
you can't tell what format was being tested. It seems you want it on
all the time, otherwise the test isn't helpful out-of-the-box.

--phil
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".