* [FFmpeg-devel] [PATCH 0/3] V2: VAAPI: Add high bit depth encode/decode support
@ 2022-08-14 21:33 Philip Langdale
2022-08-14 21:33 ` [FFmpeg-devel] [PATCH 1/3] lavu/pixfmt: Add Y216, Y410, and Y416 formats Philip Langdale
` (2 more replies)
0 siblings, 3 replies; 16+ messages in thread
From: Philip Langdale @ 2022-08-14 21:33 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Philip Langdale
This changeset fills in support for the remaining high bit depth
formats the VAAPI exposes. This requires adding more weird Microsoft
pixel formats and then mapping them in the VAAPI code. I've also
enabled hw mapping between VAAPI and vulkan for sufficiently simple
formats.
* Updated to add change to support Vulkan mapping.
Philip Langdale (3):
lavu/pixfmt: Add Y216, Y410, and Y416 formats
lavc/vaapi: Add support for remaining 10/12bit profiles
lavu/hwcontext_[vaapi|vulkan]: support mapping VUYA and Y416
libavcodec/hevcdec.c | 8 ++++
libavcodec/vaapi_decode.c | 13 ++++++
libavcodec/vaapi_encode.c | 19 ++++++--
libavcodec/vaapi_encode_h265.c | 10 ++++-
libavcodec/vaapi_encode_vp9.c | 4 +-
libavcodec/vaapi_hevc.c | 11 ++++-
libavutil/hwcontext_vaapi.c | 16 +++++++
libavutil/hwcontext_vulkan.c | 6 +++
libavutil/pixdesc.c | 77 +++++++++++++++++++++++++++++++-
libavutil/pixfmt.h | 12 +++++
tests/ref/fate/imgutils | 6 +++
tests/ref/fate/sws-pixdesc-query | 25 +++++++++++
12 files changed, 198 insertions(+), 9 deletions(-)
--
2.34.1
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 16+ messages in thread* [FFmpeg-devel] [PATCH 1/3] lavu/pixfmt: Add Y216, Y410, and Y416 formats 2022-08-14 21:33 [FFmpeg-devel] [PATCH 0/3] V2: VAAPI: Add high bit depth encode/decode support Philip Langdale @ 2022-08-14 21:33 ` Philip Langdale 2022-08-15 6:12 ` Xiang, Haihao 2022-08-14 21:33 ` [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles Philip Langdale 2022-08-14 21:33 ` [FFmpeg-devel] [PATCH 3/3] lavu/hwcontext_[vaapi|vulkan]: support mapping VUYA and Y416 Philip Langdale 2 siblings, 1 reply; 16+ messages in thread From: Philip Langdale @ 2022-08-14 21:33 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Philip Langdale These are the formats returned by the Intel VAAPI decoder for 12bit 4:2:2, 10bit 4:4:4, and 12bit 4:4:4 respectively. As with the already supported Y210 and YUVA (AVUY) formats, they are the formats Microsoft picked as their preferred 4:2:2 and 4:4:4 video formats, and Intel ran with it. Y216 is simply an extension of Y210 to say all 16bits will be used, and Y416 is a normal looking packed 4 channel format. Y410 is an annoying format that packs three 10bit channels into 32bits with 2bits of alpha. As a result, I had to define Y410 as a bitstream format, even though each pixel is byte-aligned. If it is in-fact possible to define as a normal byte-aligned format, please let me know how. As with VUYA, I have kept the formal definition of Y410 and Y416 as formats with alpha channels to maintain fidelity. The Intel folks say they prefer this and they would rather explicitly use a format defined as not having alpha than to silently drop it. The hardware decoder does at least ensure the alpha channel is set to full opacity. Signed-off-by: Philip Langdale <philipl@overt.org> --- libavutil/pixdesc.c | 77 +++++++++++++++++++++++++++++++- libavutil/pixfmt.h | 12 +++++ tests/ref/fate/imgutils | 6 +++ tests/ref/fate/sws-pixdesc-query | 25 +++++++++++ 4 files changed, 119 insertions(+), 1 deletion(-) diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c index f7558ff8b9..5dee3a95d3 100644 --- a/libavutil/pixdesc.c +++ b/libavutil/pixdesc.c @@ -2532,6 +2532,81 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { .flags = AV_PIX_FMT_FLAG_RGB | AV_PIX_FMT_FLAG_ALPHA | AV_PIX_FMT_FLAG_FLOAT, }, + [AV_PIX_FMT_Y216LE] = { + .name = "y216le", + .nb_components = 3, + .log2_chroma_w = 1, + .log2_chroma_h = 0, + .comp = { + { 0, 4, 0, 0, 16 }, /* Y */ + { 0, 8, 2, 0, 16 }, /* U */ + { 0, 8, 6, 0, 16 }, /* V */ + }, + }, + [AV_PIX_FMT_Y216BE] = { + .name = "y216be", + .nb_components = 3, + .log2_chroma_w = 1, + .log2_chroma_h = 0, + .comp = { + { 0, 4, 0, 0, 16 }, /* Y */ + { 0, 8, 2, 0, 16 }, /* U */ + { 0, 8, 6, 0, 16 }, /* V */ + }, + .flags = AV_PIX_FMT_FLAG_BE, + }, + [AV_PIX_FMT_Y410LE] = { + .name = "y410le", + .nb_components= 4, + .log2_chroma_w= 0, + .log2_chroma_h= 0, + .comp = { + { 0, 32, 10, 0, 10 }, /* Y */ + { 0, 32, 0, 0, 10 }, /* U */ + { 0, 32, 20, 0, 10 }, /* V */ + { 0, 32, 30, 0, 2 }, /* A */ + }, + .flags = AV_PIX_FMT_FLAG_ALPHA | AV_PIX_FMT_FLAG_BITSTREAM, + }, + [AV_PIX_FMT_Y410BE] = { + .name = "y410be", + .nb_components= 4, + .log2_chroma_w= 0, + .log2_chroma_h= 0, + .comp = { + { 0, 32, 10, 0, 10 }, /* Y */ + { 0, 32, 0, 0, 10 }, /* U */ + { 0, 32, 20, 0, 10 }, /* V */ + { 0, 32, 30, 0, 2 }, /* A */ + }, + .flags = AV_PIX_FMT_FLAG_BE | AV_PIX_FMT_FLAG_ALPHA | AV_PIX_FMT_FLAG_BITSTREAM, + }, + [AV_PIX_FMT_Y416LE] = { + .name = "y416le", + .nb_components= 4, + .log2_chroma_w= 0, + .log2_chroma_h= 0, + .comp = { + { 0, 8, 2, 0, 16 }, /* Y */ + { 0, 8, 0, 0, 16 }, /* U */ + { 0, 8, 4, 0, 16 }, /* V */ + { 0, 8, 6, 0, 16 }, /* A */ + }, + .flags = AV_PIX_FMT_FLAG_ALPHA, + }, + [AV_PIX_FMT_Y416BE] = { + .name = "y416be", + .nb_components= 4, + .log2_chroma_w= 0, + .log2_chroma_h= 0, + .comp = { + { 0, 8, 2, 0, 16 }, /* Y */ + { 0, 8, 0, 0, 16 }, /* U */ + { 0, 8, 4, 0, 16 }, /* V */ + { 0, 8, 6, 0, 16 }, /* A */ + }, + .flags = AV_PIX_FMT_FLAG_BE | AV_PIX_FMT_FLAG_ALPHA, + }, }; static const char * const color_range_names[] = { @@ -2767,7 +2842,7 @@ void ff_check_pixfmt_descriptors(void){ if (!d->name && !d->nb_components && !d->log2_chroma_w && !d->log2_chroma_h && !d->flags) continue; -// av_log(NULL, AV_LOG_DEBUG, "Checking: %s\n", d->name); + av_log(NULL, AV_LOG_INFO, "Checking: %s\n", d->name); av_assert0(d->log2_chroma_w <= 3); av_assert0(d->log2_chroma_h <= 3); av_assert0(d->nb_components <= 4); diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h index 86c9bdefeb..485655f0c0 100644 --- a/libavutil/pixfmt.h +++ b/libavutil/pixfmt.h @@ -372,6 +372,15 @@ enum AVPixelFormat { AV_PIX_FMT_RGBAF16BE, ///< IEEE-754 half precision packed RGBA 16:16:16:16, 64bpp, RGBARGBA..., big-endian AV_PIX_FMT_RGBAF16LE, ///< IEEE-754 half precision packed RGBA 16:16:16:16, 64bpp, RGBARGBA..., little-endian + AV_PIX_FMT_Y216BE, ///< packed YUV 4:2:2 like YUYV422, 32bpp, big-endian + AV_PIX_FMT_Y216LE, ///< packed YUV 4:2:2 like YUYV422, 32bpp, big-endian + + AV_PIX_FMT_Y410BE, ///< packed AVYU 2:10:10:10, 32bpp, (msb)2A 10V 10Y 10U(lsb), big-endian + AV_PIX_FMT_Y410LE, ///< packed AVYU 2:10:10:10, 32bpp, (msb)2A 10V 10Y 10U(lsb), little-endian + + AV_PIX_FMT_Y416BE, ///< packed AVYU 16:16:16:16, 64bpp, big-endian + AV_PIX_FMT_Y416LE, ///< packed AVYU 16:16:16:16, 64bpp, little-endian + AV_PIX_FMT_NB ///< number of pixel formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions }; @@ -461,6 +470,9 @@ enum AVPixelFormat { #define AV_PIX_FMT_P016 AV_PIX_FMT_NE(P016BE, P016LE) #define AV_PIX_FMT_Y210 AV_PIX_FMT_NE(Y210BE, Y210LE) +#define AV_PIX_FMT_Y216 AV_PIX_FMT_NE(Y216BE, Y216LE) +#define AV_PIX_FMT_Y410 AV_PIX_FMT_NE(Y410BE, Y410LE) +#define AV_PIX_FMT_Y416 AV_PIX_FMT_NE(Y416BE, Y416LE) #define AV_PIX_FMT_X2RGB10 AV_PIX_FMT_NE(X2RGB10BE, X2RGB10LE) #define AV_PIX_FMT_X2BGR10 AV_PIX_FMT_NE(X2BGR10BE, X2BGR10LE) diff --git a/tests/ref/fate/imgutils b/tests/ref/fate/imgutils index 01c9877de5..ea959c26b1 100644 --- a/tests/ref/fate/imgutils +++ b/tests/ref/fate/imgutils @@ -249,3 +249,9 @@ p416le planes: 2, linesizes: 128 256 0 0, plane_sizes: 6144 12288 vuya planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 rgbaf16be planes: 1, linesizes: 512 0 0 0, plane_sizes: 24576 0 0 0, plane_offsets: 0 0 0, total_size: 24576 rgbaf16le planes: 1, linesizes: 512 0 0 0, plane_sizes: 24576 0 0 0, plane_offsets: 0 0 0, total_size: 24576 +y216be planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 +y216le planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 +y410be planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 +y410le planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 +y416be planes: 1, linesizes: 512 0 0 0, plane_sizes: 24576 0 0 0, plane_offsets: 0 0 0, total_size: 24576 +y416le planes: 1, linesizes: 512 0 0 0, plane_sizes: 24576 0 0 0, plane_offsets: 0 0 0, total_size: 24576 diff --git a/tests/ref/fate/sws-pixdesc-query b/tests/ref/fate/sws-pixdesc-query index f79d99e513..2dea9c5a3c 100644 --- a/tests/ref/fate/sws-pixdesc-query +++ b/tests/ref/fate/sws-pixdesc-query @@ -23,6 +23,10 @@ is16BPS: rgba64le rgbaf16be rgbaf16le + y216be + y216le + y416be + y416le ya16be ya16le yuv420p16be @@ -75,6 +79,8 @@ isNBPS: xyz12le y210be y210le + y410be + y410le yuv420p10be yuv420p10le yuv420p12be @@ -164,6 +170,9 @@ isBE: x2rgb10be xyz12be y210be + y216be + y410be + y416be ya16be yuv420p10be yuv420p12be @@ -223,6 +232,12 @@ isYUV: xyz12le y210be y210le + y216be + y216le + y410be + y410le + y416be + y416le ya16be ya16le ya8 @@ -665,6 +680,10 @@ ALPHA: rgbaf16be rgbaf16le vuya + y410be + y410le + y416be + y416le ya16be ya16le ya8 @@ -761,6 +780,12 @@ Packed: xyz12le y210be y210le + y216be + y216le + y410be + y410le + y416be + y416le ya16be ya16le ya8 -- 2.34.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [FFmpeg-devel] [PATCH 1/3] lavu/pixfmt: Add Y216, Y410, and Y416 formats 2022-08-14 21:33 ` [FFmpeg-devel] [PATCH 1/3] lavu/pixfmt: Add Y216, Y410, and Y416 formats Philip Langdale @ 2022-08-15 6:12 ` Xiang, Haihao 2022-08-15 17:42 ` Philip Langdale 0 siblings, 1 reply; 16+ messages in thread From: Xiang, Haihao @ 2022-08-15 6:12 UTC (permalink / raw) To: ffmpeg-devel; +Cc: philipl On Sun, 2022-08-14 at 14:33 -0700, Philip Langdale wrote: > These are the formats returned by the Intel VAAPI decoder for 12bit > 4:2:2, 10bit 4:4:4, and 12bit 4:4:4 respectively. As with the already > supported Y210 and YUVA (AVUY) formats, they are the formats Microsoft > picked as their preferred 4:2:2 and 4:4:4 video formats, and Intel ran > with it. > > Y216 is simply an extension of Y210 to say all 16bits will be used, and > Y416 is a normal looking packed 4 channel format. Y410 is an annoying > format that packs three 10bit channels into 32bits with 2bits of alpha. > > As a result, I had to define Y410 as a bitstream format, even though > each pixel is byte-aligned. If it is in-fact possible to define as a > normal byte-aligned format, please let me know how. > > As with VUYA, I have kept the formal definition of Y410 and Y416 as > formats with alpha channels to maintain fidelity. The Intel folks say > they prefer this and they would rather explicitly use a format defined > as not having alpha than to silently drop it. The hardware decoder > does at least ensure the alpha channel is set to full opacity. > > Signed-off-by: Philip Langdale <philipl@overt.org> > --- > libavutil/pixdesc.c | 77 +++++++++++++++++++++++++++++++- > libavutil/pixfmt.h | 12 +++++ > tests/ref/fate/imgutils | 6 +++ > tests/ref/fate/sws-pixdesc-query | 25 +++++++++++ > 4 files changed, 119 insertions(+), 1 deletion(-) > > diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c > index f7558ff8b9..5dee3a95d3 100644 > --- a/libavutil/pixdesc.c > +++ b/libavutil/pixdesc.c > @@ -2532,6 +2532,81 @@ static const AVPixFmtDescriptor > av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { > .flags = AV_PIX_FMT_FLAG_RGB | AV_PIX_FMT_FLAG_ALPHA | > AV_PIX_FMT_FLAG_FLOAT, > }, > + [AV_PIX_FMT_Y216LE] = { > + .name = "y216le", > + .nb_components = 3, > + .log2_chroma_w = 1, > + .log2_chroma_h = 0, > + .comp = { > + { 0, 4, 0, 0, 16 }, /* Y */ > + { 0, 8, 2, 0, 16 }, /* U */ > + { 0, 8, 6, 0, 16 }, /* V */ > + }, > + }, > + [AV_PIX_FMT_Y216BE] = { > + .name = "y216be", > + .nb_components = 3, > + .log2_chroma_w = 1, > + .log2_chroma_h = 0, > + .comp = { > + { 0, 4, 0, 0, 16 }, /* Y */ > + { 0, 8, 2, 0, 16 }, /* U */ > + { 0, 8, 6, 0, 16 }, /* V */ > + }, > + .flags = AV_PIX_FMT_FLAG_BE, > + }, > + [AV_PIX_FMT_Y410LE] = { > + .name = "y410le", > + .nb_components= 4, > + .log2_chroma_w= 0, > + .log2_chroma_h= 0, > + .comp = { > + { 0, 32, 10, 0, 10 }, /* Y */ > + { 0, 32, 0, 0, 10 }, /* U */ > + { 0, 32, 20, 0, 10 }, /* V */ > + { 0, 32, 30, 0, 2 }, /* A */ > + }, > + .flags = AV_PIX_FMT_FLAG_ALPHA | AV_PIX_FMT_FLAG_BITSTREAM, > + }, > + [AV_PIX_FMT_Y410BE] = { > + .name = "y410be", > + .nb_components= 4, > + .log2_chroma_w= 0, > + .log2_chroma_h= 0, > + .comp = { > + { 0, 32, 10, 0, 10 }, /* Y */ > + { 0, 32, 0, 0, 10 }, /* U */ > + { 0, 32, 20, 0, 10 }, /* V */ > + { 0, 32, 30, 0, 2 }, /* A */ > + }, > + .flags = AV_PIX_FMT_FLAG_BE | AV_PIX_FMT_FLAG_ALPHA | > AV_PIX_FMT_FLAG_BITSTREAM, > + }, > + [AV_PIX_FMT_Y416LE] = { > + .name = "y416le", > + .nb_components= 4, > + .log2_chroma_w= 0, > + .log2_chroma_h= 0, > + .comp = { > + { 0, 8, 2, 0, 16 }, /* Y */ > + { 0, 8, 0, 0, 16 }, /* U */ > + { 0, 8, 4, 0, 16 }, /* V */ > + { 0, 8, 6, 0, 16 }, /* A */ > + }, > + .flags = AV_PIX_FMT_FLAG_ALPHA, > + }, > + [AV_PIX_FMT_Y416BE] = { > + .name = "y416be", > + .nb_components= 4, > + .log2_chroma_w= 0, > + .log2_chroma_h= 0, > + .comp = { > + { 0, 8, 2, 0, 16 }, /* Y */ > + { 0, 8, 0, 0, 16 }, /* U */ > + { 0, 8, 4, 0, 16 }, /* V */ > + { 0, 8, 6, 0, 16 }, /* A */ > + }, > + .flags = AV_PIX_FMT_FLAG_BE | AV_PIX_FMT_FLAG_ALPHA, > + }, > }; > > static const char * const color_range_names[] = { > @@ -2767,7 +2842,7 @@ void ff_check_pixfmt_descriptors(void){ > > if (!d->name && !d->nb_components && !d->log2_chroma_w && !d- > >log2_chroma_h && !d->flags) > continue; > -// av_log(NULL, AV_LOG_DEBUG, "Checking: %s\n", d->name); > + av_log(NULL, AV_LOG_INFO, "Checking: %s\n", d->name); > av_assert0(d->log2_chroma_w <= 3); > av_assert0(d->log2_chroma_h <= 3); > av_assert0(d->nb_components <= 4); > diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h > index 86c9bdefeb..485655f0c0 100644 > --- a/libavutil/pixfmt.h > +++ b/libavutil/pixfmt.h > @@ -372,6 +372,15 @@ enum AVPixelFormat { > AV_PIX_FMT_RGBAF16BE, ///< IEEE-754 half precision packed RGBA > 16:16:16:16, 64bpp, RGBARGBA..., big-endian > AV_PIX_FMT_RGBAF16LE, ///< IEEE-754 half precision packed RGBA > 16:16:16:16, 64bpp, RGBARGBA..., little-endian > > + AV_PIX_FMT_Y216BE, ///< packed YUV 4:2:2 like YUYV422, 32bpp, big- > endian > + AV_PIX_FMT_Y216LE, ///< packed YUV 4:2:2 like YUYV422, 32bpp, big- > endian > + > + AV_PIX_FMT_Y410BE, ///< packed AVYU 2:10:10:10, 32bpp, (msb)2A 10V > 10Y 10U(lsb), big-endian > + AV_PIX_FMT_Y410LE, ///< packed AVYU 2:10:10:10, 32bpp, (msb)2A 10V > 10Y 10U(lsb), little-endian > + > + AV_PIX_FMT_Y416BE, ///< packed AVYU 16:16:16:16, 64bpp, big-endian > + AV_PIX_FMT_Y416LE, ///< packed AVYU 16:16:16:16, 64bpp, little- > endian > + > AV_PIX_FMT_NB ///< number of pixel formats, DO NOT USE THIS if > you want to link with shared libav* because the number of formats might differ > between versions > }; > > @@ -461,6 +470,9 @@ enum AVPixelFormat { > #define AV_PIX_FMT_P016 AV_PIX_FMT_NE(P016BE, P016LE) > > #define AV_PIX_FMT_Y210 AV_PIX_FMT_NE(Y210BE, Y210LE) > +#define AV_PIX_FMT_Y216 AV_PIX_FMT_NE(Y216BE, Y216LE) > +#define AV_PIX_FMT_Y410 AV_PIX_FMT_NE(Y410BE, Y410LE) > +#define AV_PIX_FMT_Y416 AV_PIX_FMT_NE(Y416BE, Y416LE) > #define AV_PIX_FMT_X2RGB10 AV_PIX_FMT_NE(X2RGB10BE, X2RGB10LE) > #define AV_PIX_FMT_X2BGR10 AV_PIX_FMT_NE(X2BGR10BE, X2BGR10LE) > > diff --git a/tests/ref/fate/imgutils b/tests/ref/fate/imgutils > index 01c9877de5..ea959c26b1 100644 > --- a/tests/ref/fate/imgutils > +++ b/tests/ref/fate/imgutils > @@ -249,3 +249,9 @@ p416le planes: 2, linesizes: 128 256 0 0, > plane_sizes: 6144 12288 > vuya planes: 1, linesizes: 256 0 0 0, plane_sizes: > 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 > rgbaf16be planes: 1, linesizes: 512 0 0 0, plane_sizes: > 24576 0 0 0, plane_offsets: 0 0 0, total_size: 24576 > rgbaf16le planes: 1, linesizes: 512 0 0 0, plane_sizes: > 24576 0 0 0, plane_offsets: 0 0 0, total_size: 24576 > +y216be planes: 1, linesizes: 256 0 0 0, plane_sizes: > 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 > +y216le planes: 1, linesizes: 256 0 0 0, plane_sizes: > 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 > +y410be planes: 1, linesizes: 256 0 0 0, plane_sizes: > 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 > +y410le planes: 1, linesizes: 256 0 0 0, plane_sizes: > 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 > +y416be planes: 1, linesizes: 512 0 0 0, plane_sizes: > 24576 0 0 0, plane_offsets: 0 0 0, total_size: 24576 > +y416le planes: 1, linesizes: 512 0 0 0, plane_sizes: > 24576 0 0 0, plane_offsets: 0 0 0, total_size: 24576 > diff --git a/tests/ref/fate/sws-pixdesc-query b/tests/ref/fate/sws-pixdesc- > query > index f79d99e513..2dea9c5a3c 100644 > --- a/tests/ref/fate/sws-pixdesc-query > +++ b/tests/ref/fate/sws-pixdesc-query > @@ -23,6 +23,10 @@ is16BPS: > rgba64le > rgbaf16be > rgbaf16le > + y216be > + y216le > + y416be > + y416le > ya16be > ya16le > yuv420p16be > @@ -75,6 +79,8 @@ isNBPS: > xyz12le > y210be > y210le > + y410be > + y410le > yuv420p10be > yuv420p10le > yuv420p12be > @@ -164,6 +170,9 @@ isBE: > x2rgb10be > xyz12be > y210be > + y216be > + y410be > + y416be > ya16be > yuv420p10be > yuv420p12be > @@ -223,6 +232,12 @@ isYUV: > xyz12le > y210be > y210le > + y216be > + y216le > + y410be > + y410le > + y416be > + y416le > ya16be > ya16le > ya8 > @@ -665,6 +680,10 @@ ALPHA: > rgbaf16be > rgbaf16le > vuya > + y410be > + y410le > + y416be > + y416le > ya16be > ya16le > ya8 > @@ -761,6 +780,12 @@ Packed: > xyz12le > y210be > y210le > + y216be > + y216le > + y410be > + y410le > + y416be > + y416le > ya16be > ya16le > ya8 Hi Philip, May we add new formats P012, Y212 and Y412 for 12bit contents ? I agree with Mark's comment in https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200619015248.21873-1-fei.w.wang@intel.com/ " Tracking it separately does not seem fun - it looks to me like it would require adding a new bit depth field to AVFrame. FFmpeg has always used pixfmt as defining both the memory layout and which bits are used in that (so, for example, ARGB and 0RGB are not the same thing), unlike most of the graphics APIs which tend to define those two separately. " The bit depth is known if using p012 for 12bit contents in the command below: $ ffmpeg -init_hw_device vaapi=vaapi:/dev/dri/renderD128 -f lavfi -i yuvtestsrc -vf "format=p012,hwupload" -c:v hevc_vaapi -f null - If using p016 for 12bit contents, how do we know the bit depth is 12 when converting yuv444 to p016 in the command below ? $ ffmpeg -init_hw_device vaapi=vaapi:/dev/dri/renderD128 -f lavfi -i yuvtestsrc -vf "format=p016,hwupload" -c:v hevc_vaapi -f null - Thanks Haihao _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [FFmpeg-devel] [PATCH 1/3] lavu/pixfmt: Add Y216, Y410, and Y416 formats 2022-08-15 6:12 ` Xiang, Haihao @ 2022-08-15 17:42 ` Philip Langdale 2022-08-15 23:06 ` Philip Langdale 0 siblings, 1 reply; 16+ messages in thread From: Philip Langdale @ 2022-08-15 17:42 UTC (permalink / raw) To: Xiang, Haihao; +Cc: FFmpeg development discussions and patches On Mon, 15 Aug 2022 06:12:20 +0000 "Xiang, Haihao" <haihao.xiang-at-intel.com@ffmpeg.org> wrote: > > Hi Philip, > > May we add new formats P012, Y212 and Y412 for 12bit contents ? I > agree with Mark's comment in > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200619015248.21873-1-fei.w.wang@intel.com/ > > > " > Tracking it separately does not seem fun - it looks to me like it > would require adding a new bit depth field to AVFrame. > > FFmpeg has always used pixfmt as defining both the memory layout and > which bits are used in that (so, for example, ARGB and 0RGB are not > the same thing), unlike most of the graphics APIs which tend to > define those two separately. > " I went through this same conversation a few years ago around the behaviour of nvdec, which uses p016 and yuv444p16 for 12bit content. All the same arguments were made about not introducing new pixel formats to just indicate stream depth and how we could add a new attribute but it would be a bunch of work, and to avoid bogging down and doing nothing, I just had nvdec/nvenc work with P016 (and honestly, nvidia don't even use P010 internally, it's P016 for >= 10bit) and it was fine. On the decoder side, there's no real problem, and on the encoder side, I think the key insight is that the encoder doesn't encoder to the frame's pixel format depth, it encodes to the depth _specified by the profile_. So even if you give it real 16bit frames in P016, if you are encoding Main12, you are going to get 12bit. Same for any future Main14. I'd argue this is the most correct way to handle it anyway. If the default profile we guess based on format isn't right, specify the one you actually want. > The bit depth is known if using p012 for 12bit contents in the > command below: > > $ ffmpeg -init_hw_device vaapi=vaapi:/dev/dri/renderD128 -f lavfi -i > yuvtestsrc -vf "format=p012,hwupload" -c:v hevc_vaapi -f null - > > If using p016 for 12bit contents, how do we know the bit depth is 12 > when converting yuv444 to p016 in the command below ? > > $ ffmpeg -init_hw_device vaapi=vaapi:/dev/dri/renderD128 -f lavfi -i > yuvtestsrc -vf "format=p016,hwupload" -c:v hevc_vaapi -f null - That's also my answer to your command lines: Specify the profile if the default that we guess from the format isn't what you want. It's not the say that I don't think a bit depth field on AVFrames is a terrible idea, but it's a huge project and not one that I'm interested in blocking on. We'd get nothing done otherwise. I'm very comfortable with the heuristics I've added in this patch set vs the heuristics we had before. It'll be a while before we get 14bit and profile selection actually becomes ambiguous. I don't know if QSV has different constraints that make this approach impossible but it works fine for VAAPI, and I would imagine that when implementing Microsoft's directx interfaces you have to do something similar as they fix the set of usable formats for you. --phil _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [FFmpeg-devel] [PATCH 1/3] lavu/pixfmt: Add Y216, Y410, and Y416 formats 2022-08-15 17:42 ` Philip Langdale @ 2022-08-15 23:06 ` Philip Langdale 0 siblings, 0 replies; 16+ messages in thread From: Philip Langdale @ 2022-08-15 23:06 UTC (permalink / raw) To: Xiang, Haihao; +Cc: FFmpeg development discussions and patches On Mon, 15 Aug 2022 10:42:11 -0700 Philip Langdale <philipl@overt.org> wrote: > On Mon, 15 Aug 2022 06:12:20 +0000 > "Xiang, Haihao" <haihao.xiang-at-intel.com@ffmpeg.org> wrote: > > > > > Hi Philip, > > > > May we add new formats P012, Y212 and Y412 for 12bit contents ? I > > agree with Mark's comment in > > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200619015248.21873-1-fei.w.wang@intel.com/ > > > > > > " > > Tracking it separately does not seem fun - it looks to me like it > > would require adding a new bit depth field to AVFrame. > > > > FFmpeg has always used pixfmt as defining both the memory layout and > > which bits are used in that (so, for example, ARGB and 0RGB are not > > the same thing), unlike most of the graphics APIs which tend to > > define those two separately. > > " > > I went through this same conversation a few years ago around the > behaviour of nvdec, which uses p016 and yuv444p16 for 12bit content. > All the same arguments were made about not introducing new pixel > formats to just indicate stream depth and how we could add a new > attribute but it would be a bunch of work, and to avoid bogging down > and doing nothing, I just had nvdec/nvenc work with P016 (and > honestly, nvidia don't even use P010 internally, it's P016 for >= > 10bit) and it was fine. On the decoder side, there's no real problem, > and on the encoder side, I think the key insight is that the encoder > doesn't encoder to the frame's pixel format depth, it encodes to the > depth _specified by the profile_. So even if you give it real 16bit > frames in P016, if you are encoding Main12, you are going to get > 12bit. Same for any future Main14. I'd argue this is the most correct > way to handle it anyway. If the default profile we guess based on > format isn't right, specify the one you actually want. Hi Hai Hao, So, Mark has pointed out to me that this isn't the same situation as nvidia, because the driver is actually exposing support for specific P012, Y212, Y412 formats. I didn't realise this, which is why I took the approach I did. With this new information, I will redo it to add those formats instead of the 16bit ones (which don't have a reason to exist in ffmpeg for now). Thanks! --phil _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 16+ messages in thread
* [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles 2022-08-14 21:33 [FFmpeg-devel] [PATCH 0/3] V2: VAAPI: Add high bit depth encode/decode support Philip Langdale 2022-08-14 21:33 ` [FFmpeg-devel] [PATCH 1/3] lavu/pixfmt: Add Y216, Y410, and Y416 formats Philip Langdale @ 2022-08-14 21:33 ` Philip Langdale 2022-08-15 22:10 ` Michael Niedermayer 2022-08-14 21:33 ` [FFmpeg-devel] [PATCH 3/3] lavu/hwcontext_[vaapi|vulkan]: support mapping VUYA and Y416 Philip Langdale 2 siblings, 1 reply; 16+ messages in thread From: Philip Langdale @ 2022-08-14 21:33 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Philip Langdale With the necessary pixel formats defined, we can now expose support for the remaining 10/12bit combinations that VAAPI can handle. Specifically, we are adding support for: * HEVC ** 12bit 420 ** 10bit 422 ** 12bit 422 ** 10bit 444 ** 12bit 444 * VP9 ** 10bit 422 ** 10bit 444 These obviously require actual hardware support to be usable, but where that exists, it is now enabled. I had to make some adjustments to the encode logic for matching bit depth as the existing code assumed that the picture depth and the pixel format depth were always the same, which is not true for 12bit content which uses 16bit pixel formats. Signed-off-by: Philip Langdale <philipl@overt.org> --- libavcodec/hevcdec.c | 8 ++++++++ libavcodec/vaapi_decode.c | 13 +++++++++++++ libavcodec/vaapi_encode.c | 19 +++++++++++++++---- libavcodec/vaapi_encode_h265.c | 10 ++++++++-- libavcodec/vaapi_encode_vp9.c | 4 +++- libavcodec/vaapi_hevc.c | 11 ++++++++++- libavutil/hwcontext_vaapi.c | 12 ++++++++++++ 7 files changed, 69 insertions(+), 8 deletions(-) diff --git a/libavcodec/hevcdec.c b/libavcodec/hevcdec.c index f1be8af2cd..1a895800a6 100644 --- a/libavcodec/hevcdec.c +++ b/libavcodec/hevcdec.c @@ -481,11 +481,19 @@ static enum AVPixelFormat get_format(HEVCContext *s, const HEVCSPS *sps) #endif case AV_PIX_FMT_YUV420P12: case AV_PIX_FMT_YUV444P12: +#if CONFIG_HEVC_VAAPI_HWACCEL + *fmt++ = AV_PIX_FMT_VAAPI; +#endif #if CONFIG_HEVC_VDPAU_HWACCEL *fmt++ = AV_PIX_FMT_VDPAU; #endif #if CONFIG_HEVC_NVDEC_HWACCEL *fmt++ = AV_PIX_FMT_CUDA; +#endif + break; + case AV_PIX_FMT_YUV422P12: +#if CONFIG_HEVC_VAAPI_HWACCEL + *fmt++ = AV_PIX_FMT_VAAPI; #endif break; } diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c index bc2d3ed803..9ba7dfd886 100644 --- a/libavcodec/vaapi_decode.c +++ b/libavcodec/vaapi_decode.c @@ -262,16 +262,28 @@ static const struct { MAP(YUY2, YUYV422), #ifdef VA_FOURCC_Y210 MAP(Y210, Y210), +#endif +#ifdef VA_FOURCC_Y216 + MAP(Y216, Y216), #endif // 4:4:0 MAP(422V, YUV440P), // 4:4:4 MAP(444P, YUV444P), MAP(AYUV, VUYA), +#ifdef VA_FOURCC_Y410 + MAP(Y410, Y410), +#endif +#ifdef VA_FOURCC_Y416 + MAP(Y416, Y416), +#endif // 4:2:0 10-bit #ifdef VA_FOURCC_P010 MAP(P010, P010), #endif +#ifdef VA_FOURCC_P016 + MAP(P016, P016), +#endif #ifdef VA_FOURCC_I010 MAP(I010, YUV420P10), #endif @@ -415,6 +427,7 @@ static const struct { #if VA_CHECK_VERSION(0, 39, 0) MAP(VP9, VP9_1, VP9Profile1 ), MAP(VP9, VP9_2, VP9Profile2 ), + MAP(VP9, VP9_3, VP9Profile3 ), #endif #if VA_CHECK_VERSION(1, 8, 0) MAP(AV1, AV1_MAIN, AV1Profile0), diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index f13daa5cff..1cf11c4cd5 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -1305,7 +1305,11 @@ static const VAAPIEncodeRTFormat vaapi_encode_rt_formats[] = { { "YUV420", VA_RT_FORMAT_YUV420, 8, 3, 1, 1 }, { "YUV422", VA_RT_FORMAT_YUV422, 8, 3, 1, 0 }, #if VA_CHECK_VERSION(1, 2, 0) + { "YUV420_12", VA_RT_FORMAT_YUV420_12, 12, 3, 1, 1 }, { "YUV422_10", VA_RT_FORMAT_YUV422_10, 10, 3, 1, 0 }, + { "YUV422_12", VA_RT_FORMAT_YUV422_12, 12, 3, 1, 0 }, + { "YUV444_10", VA_RT_FORMAT_YUV444_10, 10, 4, 0, 0 }, + { "YUV444_12", VA_RT_FORMAT_YUV444_12, 12, 4, 0, 0 }, #endif { "YUV444", VA_RT_FORMAT_YUV444, 8, 3, 0, 0 }, { "AYUV", VA_RT_FORMAT_YUV444, 8, 4, 0, 0 }, @@ -1342,7 +1346,7 @@ static av_cold int vaapi_encode_profile_entrypoint(AVCodecContext *avctx) VAConfigAttrib rt_format_attr; const VAAPIEncodeRTFormat *rt_format; const char *profile_string, *entrypoint_string; - int i, j, n, depth, err; + int i, j, n, depth, logical_depth, err; if (ctx->low_power) { @@ -1365,7 +1369,9 @@ static av_cold int vaapi_encode_profile_entrypoint(AVCodecContext *avctx) } depth = desc->comp[0].depth; for (i = 1; i < desc->nb_components; i++) { - if (desc->comp[i].depth != depth) { + // We do not apply this depth requirement to the fourth component as + // that will be the alpha channel when present, which can be smaller. + if (i < 3 && desc->comp[i].depth != depth) { av_log(avctx, AV_LOG_ERROR, "Invalid input pixfmt (%s).\n", desc->name); return AVERROR(EINVAL); @@ -1391,8 +1397,11 @@ static av_cold int vaapi_encode_profile_entrypoint(AVCodecContext *avctx) av_assert0(ctx->codec->profiles); for (i = 0; (ctx->codec->profiles[i].av_profile != FF_PROFILE_UNKNOWN); i++) { + profile = &ctx->codec->profiles[i]; - if (depth != profile->depth || + // Look for a 16bit format if the profile depth is > 10bit + logical_depth = profile->depth > 10 ? 16 : profile->depth; + if (depth != logical_depth || desc->nb_components != profile->nb_components) continue; if (desc->nb_components > 1 && @@ -1476,7 +1485,9 @@ static av_cold int vaapi_encode_profile_entrypoint(AVCodecContext *avctx) for (i = 0; i < FF_ARRAY_ELEMS(vaapi_encode_rt_formats); i++) { rt_format = &vaapi_encode_rt_formats[i]; - if (rt_format->depth == depth && + // Look for a 16bit pixel format if the rt format depth is > 10bit + logical_depth = rt_format->depth > 10 ? 16 : rt_format->depth; + if (logical_depth == depth && rt_format->nb_components == profile->nb_components && rt_format->log2_chroma_w == profile->log2_chroma_w && rt_format->log2_chroma_h == profile->log2_chroma_h) diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 1de323af78..1a65fb9ebf 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -283,7 +283,9 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) return AVERROR(EINVAL); } } - bit_depth = desc->comp[0].depth; + // Bit depth must be taken from the profile, as the pixel format will be + // 16bit for >= 12bit content. + bit_depth = ctx->profile->depth; // VPS @@ -1276,10 +1278,14 @@ static const VAAPIEncodeProfile vaapi_encode_h265_profiles[] = { { FF_PROFILE_HEVC_REXT, 10, 3, 1, 1, VAProfileHEVCMain10 }, #endif #if VA_CHECK_VERSION(1, 2, 0) + { FF_PROFILE_HEVC_REXT, 12, 3, 1, 1, VAProfileHEVCMain12 }, { FF_PROFILE_HEVC_REXT, 8, 3, 1, 0, VAProfileHEVCMain422_10 }, { FF_PROFILE_HEVC_REXT, 10, 3, 1, 0, VAProfileHEVCMain422_10 }, - // Four channels because this uses the AYUV format which has Alpha + { FF_PROFILE_HEVC_REXT, 12, 3, 1, 0, VAProfileHEVCMain422_12 }, + // Four channels because these use formats which have Alpha { FF_PROFILE_HEVC_REXT, 8, 4, 0, 0, VAProfileHEVCMain444 }, + { FF_PROFILE_HEVC_REXT, 10, 4, 0, 0, VAProfileHEVCMain444_10 }, + { FF_PROFILE_HEVC_REXT, 12, 4, 0, 0, VAProfileHEVCMain444_12 }, #endif { FF_PROFILE_UNKNOWN } }; diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c index 9b455e10c9..e6c7f01f11 100644 --- a/libavcodec/vaapi_encode_vp9.c +++ b/libavcodec/vaapi_encode_vp9.c @@ -228,9 +228,11 @@ static av_cold int vaapi_encode_vp9_configure(AVCodecContext *avctx) static const VAAPIEncodeProfile vaapi_encode_vp9_profiles[] = { { FF_PROFILE_VP9_0, 8, 3, 1, 1, VAProfileVP9Profile0 }, - // Four channels because this uses the AYUV format which has Alpha + // Four channels because this uses a format which has Alpha { FF_PROFILE_VP9_1, 8, 4, 0, 0, VAProfileVP9Profile1 }, { FF_PROFILE_VP9_2, 10, 3, 1, 1, VAProfileVP9Profile2 }, + // Four channels because this uses a format which has Alpha + { FF_PROFILE_VP9_3, 10, 4, 0, 0, VAProfileVP9Profile3 }, { FF_PROFILE_UNKNOWN } }; diff --git a/libavcodec/vaapi_hevc.c b/libavcodec/vaapi_hevc.c index d82975979a..20fb36adfa 100644 --- a/libavcodec/vaapi_hevc.c +++ b/libavcodec/vaapi_hevc.c @@ -567,15 +567,24 @@ VAProfile ff_vaapi_parse_hevc_rext_profile(AVCodecContext *avctx) } #if VA_CHECK_VERSION(1, 2, 0) - if (!strcmp(profile->name, "Main 4:2:2 10") || + if (!strcmp(profile->name, "Main 12") || + !strcmp(profile->name, "Main 12 Intra")) + return VAProfileHEVCMain12; + else if (!strcmp(profile->name, "Main 4:2:2 10") || !strcmp(profile->name, "Main 4:2:2 10 Intra")) return VAProfileHEVCMain422_10; + else if (!strcmp(profile->name, "Main 4:2:2 12") || + !strcmp(profile->name, "Main 4:2:2 12 Intra")) + return VAProfileHEVCMain422_12; else if (!strcmp(profile->name, "Main 4:4:4") || !strcmp(profile->name, "Main 4:4:4 Intra")) return VAProfileHEVCMain444; else if (!strcmp(profile->name, "Main 4:4:4 10") || !strcmp(profile->name, "Main 4:4:4 10 Intra")) return VAProfileHEVCMain444_10; + else if (!strcmp(profile->name, "Main 4:4:4 12") || + !strcmp(profile->name, "Main 4:4:4 12 Intra")) + return VAProfileHEVCMain444_12; #else av_log(avctx, AV_LOG_WARNING, "HEVC profile %s is " "not supported with this VA version.\n", profile->name); diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index 2ee5145727..c3e79907fd 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -121,6 +121,9 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = { MAP(YUY2, YUV422, YUYV422, 0), #ifdef VA_FOURCC_Y210 MAP(Y210, YUV422_10, Y210, 0), +#endif +#ifdef VA_FOURCC_Y216 + MAP(Y216, YUV422_12, Y216, 0), #endif MAP(411P, YUV411, YUV411P, 0), MAP(422V, YUV422, YUV440P, 0), @@ -129,6 +132,9 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = { MAP(Y800, YUV400, GRAY8, 0), #ifdef VA_FOURCC_P010 MAP(P010, YUV420_10BPP, P010, 0), +#endif +#ifdef VA_FOURCC_P016 + MAP(P016, YUV420_12, P016, 0), #endif MAP(BGRA, RGB32, BGRA, 0), MAP(BGRX, RGB32, BGR0, 0), @@ -143,6 +149,12 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = { #ifdef VA_FOURCC_X2R10G10B10 MAP(X2R10G10B10, RGB32_10, X2RGB10, 0), #endif +#ifdef VA_FOURCC_Y410 + MAP(Y410, YUV444_10, Y410, 0), +#endif +#ifdef VA_FOURCC_Y416 + MAP(Y416, YUV444_12, Y416, 0), +#endif }; #undef MAP -- 2.34.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles 2022-08-14 21:33 ` [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles Philip Langdale @ 2022-08-15 22:10 ` Michael Niedermayer 2022-08-15 22:23 ` Philip Langdale 0 siblings, 1 reply; 16+ messages in thread From: Michael Niedermayer @ 2022-08-15 22:10 UTC (permalink / raw) To: FFmpeg development discussions and patches [-- Attachment #1.1: Type: text/plain, Size: 1501 bytes --] On Sun, Aug 14, 2022 at 02:33:12PM -0700, Philip Langdale wrote: > With the necessary pixel formats defined, we can now expose support for > the remaining 10/12bit combinations that VAAPI can handle. > > Specifically, we are adding support for: > > * HEVC > ** 12bit 420 > ** 10bit 422 > ** 12bit 422 > ** 10bit 444 > ** 12bit 444 > > * VP9 > ** 10bit 422 > ** 10bit 444 > > These obviously require actual hardware support to be usable, but where > that exists, it is now enabled. > > I had to make some adjustments to the encode logic for matching bit > depth as the existing code assumed that the picture depth and the pixel > format depth were always the same, which is not true for 12bit content > which uses 16bit pixel formats. breaks build on ubuntu x86-64, assuming i did not miss any patch make CC libavutil/hwcontext_vaapi.o libavutil/hwcontext_vaapi.c:103:9: error: ‘VA_RT_FORMAT_YUV420_12’ undeclared here (not in a function); did you mean ‘VA_RT_FORMAT_YUV420’? VA_RT_FORMAT_ ## rt, \ ^ libavutil/hwcontext_vaapi.c:137:5: note: in expansion of macro ‘MAP’ MAP(P016, YUV420_12, P016, 0), ^~~ ffbuild/common.mak:81: recipe for target 'libavutil/hwcontext_vaapi.o' failed make: *** [libavutil/hwcontext_vaapi.o] Error 1 [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Republics decline into democracies and democracies degenerate into despotisms. -- Aristotle [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] [-- Attachment #2: Type: text/plain, Size: 251 bytes --] _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles 2022-08-15 22:10 ` Michael Niedermayer @ 2022-08-15 22:23 ` Philip Langdale 2022-08-16 17:29 ` Michael Niedermayer 0 siblings, 1 reply; 16+ messages in thread From: Philip Langdale @ 2022-08-15 22:23 UTC (permalink / raw) To: FFmpeg development discussions and patches On Tue, 16 Aug 2022 00:10:49 +0200 Michael Niedermayer <michael@niedermayer.cc> wrote: > On Sun, Aug 14, 2022 at 02:33:12PM -0700, Philip Langdale wrote: > > With the necessary pixel formats defined, we can now expose support > > for the remaining 10/12bit combinations that VAAPI can handle. > > > > Specifically, we are adding support for: > > > > * HEVC > > ** 12bit 420 > > ** 10bit 422 > > ** 12bit 422 > > ** 10bit 444 > > ** 12bit 444 > > > > * VP9 > > ** 10bit 422 > > ** 10bit 444 > > > > These obviously require actual hardware support to be usable, but > > where that exists, it is now enabled. > > > > I had to make some adjustments to the encode logic for matching bit > > depth as the existing code assumed that the picture depth and the > > pixel format depth were always the same, which is not true for > > 12bit content which uses 16bit pixel formats. > > breaks build on ubuntu x86-64, assuming i did not miss any patch > make > CC libavutil/hwcontext_vaapi.o > libavutil/hwcontext_vaapi.c:103:9: error: ‘VA_RT_FORMAT_YUV420_12’ > undeclared here (not in a function); did you mean > ‘VA_RT_FORMAT_YUV420’? VA_RT_FORMAT_ ## rt, \ ^ > libavutil/hwcontext_vaapi.c:137:5: note: in expansion of macro ‘MAP’ > MAP(P016, YUV420_12, P016, 0), > ^~~ > ffbuild/common.mak:81: recipe for target > 'libavutil/hwcontext_vaapi.o' failed make: *** > [libavutil/hwcontext_vaapi.o] Error 1 > I guess there's probably a libva version dependency I need to guard for. What version do you have installed? Thanks, --phil _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles 2022-08-15 22:23 ` Philip Langdale @ 2022-08-16 17:29 ` Michael Niedermayer 2022-08-16 19:52 ` Philip Langdale 0 siblings, 1 reply; 16+ messages in thread From: Michael Niedermayer @ 2022-08-16 17:29 UTC (permalink / raw) To: FFmpeg development discussions and patches [-- Attachment #1.1: Type: text/plain, Size: 3347 bytes --] On Mon, Aug 15, 2022 at 03:23:29PM -0700, Philip Langdale wrote: > On Tue, 16 Aug 2022 00:10:49 +0200 > Michael Niedermayer <michael@niedermayer.cc> wrote: > > > On Sun, Aug 14, 2022 at 02:33:12PM -0700, Philip Langdale wrote: > > > With the necessary pixel formats defined, we can now expose support > > > for the remaining 10/12bit combinations that VAAPI can handle. > > > > > > Specifically, we are adding support for: > > > > > > * HEVC > > > ** 12bit 420 > > > ** 10bit 422 > > > ** 12bit 422 > > > ** 10bit 444 > > > ** 12bit 444 > > > > > > * VP9 > > > ** 10bit 422 > > > ** 10bit 444 > > > > > > These obviously require actual hardware support to be usable, but > > > where that exists, it is now enabled. > > > > > > I had to make some adjustments to the encode logic for matching bit > > > depth as the existing code assumed that the picture depth and the > > > pixel format depth were always the same, which is not true for > > > 12bit content which uses 16bit pixel formats. > > > > breaks build on ubuntu x86-64, assuming i did not miss any patch > > make > > CC libavutil/hwcontext_vaapi.o > > libavutil/hwcontext_vaapi.c:103:9: error: ‘VA_RT_FORMAT_YUV420_12’ > > undeclared here (not in a function); did you mean > > ‘VA_RT_FORMAT_YUV420’? VA_RT_FORMAT_ ## rt, \ ^ > > libavutil/hwcontext_vaapi.c:137:5: note: in expansion of macro ‘MAP’ > > MAP(P016, YUV420_12, P016, 0), > > ^~~ > > ffbuild/common.mak:81: recipe for target > > 'libavutil/hwcontext_vaapi.o' failed make: *** > > [libavutil/hwcontext_vaapi.o] Error 1 > > > > I guess there's probably a libva version dependency I need to guard > for. What version do you have installed? ii libva-dev:amd64 2.1.0-3 amd64 Video Acceleration (VA) API for Linux -- development files ii libva-drm2:amd64 2.1.0-3 amd64 Video Acceleration (VA) API for Linux -- DRM runtime ii libva-glx2:amd64 2.1.0-3 amd64 Video Acceleration (VA) API for Linux -- GLX runtime ii libva-wayland2:amd64 2.1.0-3 amd64 Video Acceleration (VA) API for Linux -- Wayland runtime ii libva-x11-2:amd64 2.1.0-3 amd64 Video Acceleration (VA) API for Linux -- X11 runtime ii libva2:amd64 2.1.0-3 amd64 Video Acceleration (VA) API for Linux -- runtime thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Breaking DRM is a little like attempting to break through a door even though the window is wide open and the only thing in the house is a bunch of things you dont want and which you would get tomorrow for free anyway [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] [-- Attachment #2: Type: text/plain, Size: 251 bytes --] _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles 2022-08-16 17:29 ` Michael Niedermayer @ 2022-08-16 19:52 ` Philip Langdale 2022-08-16 19:55 ` Philip Langdale 0 siblings, 1 reply; 16+ messages in thread From: Philip Langdale @ 2022-08-16 19:52 UTC (permalink / raw) To: ffmpeg-devel On Tue, 16 Aug 2022 19:29:24 +0200 Michael Niedermayer <michael@niedermayer.cc> wrote: > > > CC libavutil/hwcontext_vaapi.o > > > libavutil/hwcontext_vaapi.c:103:9: error: ‘VA_RT_FORMAT_YUV420_12’ > > > undeclared here (not in a function); did you mean > > > ‘VA_RT_FORMAT_YUV420’? VA_RT_FORMAT_ ## rt, \ ^ > > > libavutil/hwcontext_vaapi.c:137:5: note: in expansion of macro > > > ‘MAP’ MAP(P016, YUV420_12, P016, 0), > > > ^~~ > > > ffbuild/common.mak:81: recipe for target > > > 'libavutil/hwcontext_vaapi.o' failed make: *** > > > [libavutil/hwcontext_vaapi.o] Error 1 > > > > > > > I guess there's probably a libva version dependency I need to guard > > for. What version do you have installed? > > ii libva-dev:amd64 > 2.1.0-3 amd64 That's older than what I have, although I'd have thought it would be new enough. Regardless, I've included the appropriate #ifdef guard in the latest version of the patchset. Thanks, --phil _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles 2022-08-16 19:52 ` Philip Langdale @ 2022-08-16 19:55 ` Philip Langdale 2022-08-17 14:34 ` Michael Niedermayer 0 siblings, 1 reply; 16+ messages in thread From: Philip Langdale @ 2022-08-16 19:55 UTC (permalink / raw) To: ffmpeg-devel On Tue, 16 Aug 2022 12:52:52 -0700 Philip Langdale <philipl@overt.org> wrote: > On Tue, 16 Aug 2022 19:29:24 +0200 > Michael Niedermayer <michael@niedermayer.cc> wrote: > > > > > CC libavutil/hwcontext_vaapi.o > > > > libavutil/hwcontext_vaapi.c:103:9: error: > > > > ‘VA_RT_FORMAT_YUV420_12’ undeclared here (not in a function); > > > > did you mean ‘VA_RT_FORMAT_YUV420’? VA_RT_FORMAT_ ## rt, \ ^ > > > > libavutil/hwcontext_vaapi.c:137:5: note: in expansion of macro > > > > ‘MAP’ MAP(P016, YUV420_12, P016, 0), > > > > ^~~ > > > > ffbuild/common.mak:81: recipe for target > > > > 'libavutil/hwcontext_vaapi.o' failed make: *** > > > > [libavutil/hwcontext_vaapi.o] Error 1 > > > > > > > > > > I guess there's probably a libva version dependency I need to > > > guard for. What version do you have installed? > > > > ii libva-dev:amd64 > > 2.1.0-3 amd64 > > That's older than what I have, although I'd have thought it would be > new enough. Regardless, I've included the appropriate #ifdef guard in > the latest version of the patchset. > > Thanks, Ooops. I thought that said 2.10.0, not 2.1.0. 2.1.0 is ancient (2018) but it turns out even 2.10.0 is too old. For the record it requires 2.13.0. --phil _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles 2022-08-16 19:55 ` Philip Langdale @ 2022-08-17 14:34 ` Michael Niedermayer 0 siblings, 0 replies; 16+ messages in thread From: Michael Niedermayer @ 2022-08-17 14:34 UTC (permalink / raw) To: FFmpeg development discussions and patches [-- Attachment #1.1: Type: text/plain, Size: 1673 bytes --] On Tue, Aug 16, 2022 at 12:55:38PM -0700, Philip Langdale wrote: > On Tue, 16 Aug 2022 12:52:52 -0700 > Philip Langdale <philipl@overt.org> wrote: > > > On Tue, 16 Aug 2022 19:29:24 +0200 > > Michael Niedermayer <michael@niedermayer.cc> wrote: > > > > > > > CC libavutil/hwcontext_vaapi.o > > > > > libavutil/hwcontext_vaapi.c:103:9: error: > > > > > ‘VA_RT_FORMAT_YUV420_12’ undeclared here (not in a function); > > > > > did you mean ‘VA_RT_FORMAT_YUV420’? VA_RT_FORMAT_ ## rt, \ ^ > > > > > libavutil/hwcontext_vaapi.c:137:5: note: in expansion of macro > > > > > ‘MAP’ MAP(P016, YUV420_12, P016, 0), > > > > > ^~~ > > > > > ffbuild/common.mak:81: recipe for target > > > > > 'libavutil/hwcontext_vaapi.o' failed make: *** > > > > > [libavutil/hwcontext_vaapi.o] Error 1 > > > > > > > > > > > > > I guess there's probably a libva version dependency I need to > > > > guard for. What version do you have installed? > > > > > > ii libva-dev:amd64 > > > 2.1.0-3 amd64 > > > > That's older than what I have, although I'd have thought it would be > > new enough. Regardless, I've included the appropriate #ifdef guard in > > the latest version of the patchset. > > > > Thanks, > > Ooops. I thought that said 2.10.0, not 2.1.0. 2.1.0 is ancient (2018) > but it turns out even 2.10.0 is too old. For the record it requires > 2.13.0. its not ancient its ubuntu LTS ;) or rather whats in it [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Opposition brings concord. Out of discord comes the fairest harmony. -- Heraclitus [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] [-- Attachment #2: Type: text/plain, Size: 251 bytes --] _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 16+ messages in thread
* [FFmpeg-devel] [PATCH 3/3] lavu/hwcontext_[vaapi|vulkan]: support mapping VUYA and Y416 2022-08-14 21:33 [FFmpeg-devel] [PATCH 0/3] V2: VAAPI: Add high bit depth encode/decode support Philip Langdale 2022-08-14 21:33 ` [FFmpeg-devel] [PATCH 1/3] lavu/pixfmt: Add Y216, Y410, and Y416 formats Philip Langdale 2022-08-14 21:33 ` [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles Philip Langdale @ 2022-08-14 21:33 ` Philip Langdale 2 siblings, 0 replies; 16+ messages in thread From: Philip Langdale @ 2022-08-14 21:33 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Philip Langdale These two of the Microsoft formats used by Intel VAAPI are sufficiently conventional that we can add simple mappings for them in the hwcontexts to enable back and forth mapping so that Vulkan filters can be used with vaapi decoding/encoding of these formats. Signed-off-by: Philip Langdale <philipl@overt.org> --- libavutil/hwcontext_vaapi.c | 4 ++++ libavutil/hwcontext_vulkan.c | 6 ++++++ 2 files changed, 10 insertions(+) diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index c3e79907fd..d2d327567e 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -1021,6 +1021,10 @@ static const struct { #endif DRM_MAP(ARGB, 1, DRM_FORMAT_BGRA8888), DRM_MAP(XRGB, 1, DRM_FORMAT_BGRX8888), +#ifdef VA_FOURCC_Y416 + DRM_MAP(AYUV, 1, DRM_FORMAT_AYUV), + DRM_MAP(Y416, 1, DRM_FORMAT_Y416), +#endif }; #undef DRM_MAP diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 237caa4bc0..be08238ff3 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -210,6 +210,9 @@ static const struct { { AV_PIX_FMT_YUVA444P12, { VK_FORMAT_R16_UNORM, VK_FORMAT_R16_UNORM, VK_FORMAT_R16_UNORM, VK_FORMAT_R16_UNORM } }, { AV_PIX_FMT_YUVA444P16, { VK_FORMAT_R16_UNORM, VK_FORMAT_R16_UNORM, VK_FORMAT_R16_UNORM, VK_FORMAT_R16_UNORM } }, + { AV_PIX_FMT_VUYA, { VK_FORMAT_R8G8B8A8_UNORM } }, + { AV_PIX_FMT_Y416, { VK_FORMAT_R16G16B16A16_UNORM } }, + { AV_PIX_FMT_BGRA, { VK_FORMAT_B8G8R8A8_UNORM } }, { AV_PIX_FMT_RGBA, { VK_FORMAT_R8G8B8A8_UNORM } }, { AV_PIX_FMT_RGB24, { VK_FORMAT_R8G8B8_UNORM } }, @@ -2629,6 +2632,9 @@ static const struct { { DRM_FORMAT_XRGB8888, VK_FORMAT_B8G8R8A8_UNORM }, { DRM_FORMAT_ABGR8888, VK_FORMAT_R8G8B8A8_UNORM }, { DRM_FORMAT_XBGR8888, VK_FORMAT_R8G8B8A8_UNORM }, + + { DRM_FORMAT_AYUV, VK_FORMAT_R8G8B8A8_UNORM }, + { DRM_FORMAT_Y416, VK_FORMAT_R16G16B16A16_UNORM }, }; static inline VkFormat drm_to_vulkan_fmt(uint32_t drm_fourcc) -- 2.34.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 16+ messages in thread
* [FFmpeg-devel] [PATCH 0/3] V3: VAAPI: Add high bit depth encode/decode support @ 2022-08-16 4:36 Philip Langdale 2022-08-16 4:36 ` [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles Philip Langdale 0 siblings, 1 reply; 16+ messages in thread From: Philip Langdale @ 2022-08-16 4:36 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Philip Langdale This changeset fills in support for the remaining high bit depth formats the VAAPI exposes. This requires adding more weird Microsoft pixel formats and then mapping them in the VAAPI code. I've also enabled hw mapping between VAAPI and vulkan for sufficiently simple formats. * Updated to use explicit 12bit pixel formats after I learned that the driver supports them and doesn't force the use of 16bit formats. Philip Langdale (3): lavu/pixfmt: Add P012, Y212, Y410, and Y412 formats lavc/vaapi: Add support for remaining 10/12bit profiles lavu/hwcontext_[vaapi|vulkan]: support mapping VUYA, P012, and Y412 libavcodec/hevcdec.c | 8 +++ libavcodec/vaapi_decode.c | 13 ++++ libavcodec/vaapi_encode.c | 10 ++- libavcodec/vaapi_encode_h265.c | 6 +- libavcodec/vaapi_encode_vp9.c | 4 +- libavcodec/vaapi_hevc.c | 11 +++- libavcodec/vp9.c | 2 + libavutil/hwcontext_vaapi.c | 21 +++++++ libavutil/hwcontext_vulkan.c | 10 +++ libavutil/pixdesc.c | 101 ++++++++++++++++++++++++++++++- libavutil/pixfmt.h | 16 +++++ tests/ref/fate/imgutils | 8 +++ tests/ref/fate/sws-pixdesc-query | 42 +++++++++++++ 13 files changed, 246 insertions(+), 6 deletions(-) -- 2.34.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 16+ messages in thread
* [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles 2022-08-16 4:36 [FFmpeg-devel] [PATCH 0/3] V3: VAAPI: Add high bit depth encode/decode support Philip Langdale @ 2022-08-16 4:36 ` Philip Langdale 0 siblings, 0 replies; 16+ messages in thread From: Philip Langdale @ 2022-08-16 4:36 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Philip Langdale With the necessary pixel formats defined, we can now expose support for the remaining 10/12bit combinations that VAAPI can handle. Specifically, we are adding support for: * HEVC ** 12bit 420 ** 10bit 422 ** 12bit 422 ** 10bit 444 ** 12bit 444 * VP9 ** 10bit 444 ** 12bit 444 These obviously require actual hardware support to be usable, but where that exists, it is now enabled. I had to tweak the format matching logic slightly to account for the reduced size alpha channel in Y410, as we can't assume all channels are the same depth as the first channel. Signed-off-by: Philip Langdale <philipl@overt.org> --- libavcodec/hevcdec.c | 8 ++++++++ libavcodec/vaapi_decode.c | 13 +++++++++++++ libavcodec/vaapi_encode.c | 10 ++++++++-- libavcodec/vaapi_encode_h265.c | 6 +++++- libavcodec/vaapi_encode_vp9.c | 4 +++- libavcodec/vaapi_hevc.c | 11 ++++++++++- libavcodec/vp9.c | 2 ++ libavutil/hwcontext_vaapi.c | 12 ++++++++++++ 8 files changed, 61 insertions(+), 5 deletions(-) diff --git a/libavcodec/hevcdec.c b/libavcodec/hevcdec.c index f1be8af2cd..1a895800a6 100644 --- a/libavcodec/hevcdec.c +++ b/libavcodec/hevcdec.c @@ -481,11 +481,19 @@ static enum AVPixelFormat get_format(HEVCContext *s, const HEVCSPS *sps) #endif case AV_PIX_FMT_YUV420P12: case AV_PIX_FMT_YUV444P12: +#if CONFIG_HEVC_VAAPI_HWACCEL + *fmt++ = AV_PIX_FMT_VAAPI; +#endif #if CONFIG_HEVC_VDPAU_HWACCEL *fmt++ = AV_PIX_FMT_VDPAU; #endif #if CONFIG_HEVC_NVDEC_HWACCEL *fmt++ = AV_PIX_FMT_CUDA; +#endif + break; + case AV_PIX_FMT_YUV422P12: +#if CONFIG_HEVC_VAAPI_HWACCEL + *fmt++ = AV_PIX_FMT_VAAPI; #endif break; } diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c index bc2d3ed803..8c6fd45f3d 100644 --- a/libavcodec/vaapi_decode.c +++ b/libavcodec/vaapi_decode.c @@ -262,16 +262,28 @@ static const struct { MAP(YUY2, YUYV422), #ifdef VA_FOURCC_Y210 MAP(Y210, Y210), +#endif +#ifdef VA_FOURCC_Y212 + MAP(Y212, Y212), #endif // 4:4:0 MAP(422V, YUV440P), // 4:4:4 MAP(444P, YUV444P), MAP(AYUV, VUYA), +#ifdef VA_FOURCC_Y410 + MAP(Y410, Y410), +#endif +#ifdef VA_FOURCC_Y412 + MAP(Y412, Y412), +#endif // 4:2:0 10-bit #ifdef VA_FOURCC_P010 MAP(P010, P010), #endif +#ifdef VA_FOURCC_P012 + MAP(P012, P012), +#endif #ifdef VA_FOURCC_I010 MAP(I010, YUV420P10), #endif @@ -415,6 +427,7 @@ static const struct { #if VA_CHECK_VERSION(0, 39, 0) MAP(VP9, VP9_1, VP9Profile1 ), MAP(VP9, VP9_2, VP9Profile2 ), + MAP(VP9, VP9_3, VP9Profile3 ), #endif #if VA_CHECK_VERSION(1, 8, 0) MAP(AV1, AV1_MAIN, AV1Profile0), diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index f13daa5cff..6112b98624 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -1305,7 +1305,11 @@ static const VAAPIEncodeRTFormat vaapi_encode_rt_formats[] = { { "YUV420", VA_RT_FORMAT_YUV420, 8, 3, 1, 1 }, { "YUV422", VA_RT_FORMAT_YUV422, 8, 3, 1, 0 }, #if VA_CHECK_VERSION(1, 2, 0) + { "YUV420_12", VA_RT_FORMAT_YUV420_12, 12, 3, 1, 1 }, { "YUV422_10", VA_RT_FORMAT_YUV422_10, 10, 3, 1, 0 }, + { "YUV422_12", VA_RT_FORMAT_YUV422_12, 12, 3, 1, 0 }, + { "YUV444_10", VA_RT_FORMAT_YUV444_10, 10, 4, 0, 0 }, + { "YUV444_12", VA_RT_FORMAT_YUV444_12, 12, 4, 0, 0 }, #endif { "YUV444", VA_RT_FORMAT_YUV444, 8, 3, 0, 0 }, { "AYUV", VA_RT_FORMAT_YUV444, 8, 4, 0, 0 }, @@ -1365,7 +1369,9 @@ static av_cold int vaapi_encode_profile_entrypoint(AVCodecContext *avctx) } depth = desc->comp[0].depth; for (i = 1; i < desc->nb_components; i++) { - if (desc->comp[i].depth != depth) { + // We do not apply this depth requirement to the fourth component as + // that will be the alpha channel when present, which can be smaller. + if (i < 3 && desc->comp[i].depth != depth) { av_log(avctx, AV_LOG_ERROR, "Invalid input pixfmt (%s).\n", desc->name); return AVERROR(EINVAL); @@ -1387,10 +1393,10 @@ static av_cold int vaapi_encode_profile_entrypoint(AVCodecContext *avctx) err = AVERROR_EXTERNAL; goto fail; } - av_assert0(ctx->codec->profiles); for (i = 0; (ctx->codec->profiles[i].av_profile != FF_PROFILE_UNKNOWN); i++) { + profile = &ctx->codec->profiles[i]; if (depth != profile->depth || desc->nb_components != profile->nb_components) diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 1de323af78..c2e91a8449 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -1276,10 +1276,14 @@ static const VAAPIEncodeProfile vaapi_encode_h265_profiles[] = { { FF_PROFILE_HEVC_REXT, 10, 3, 1, 1, VAProfileHEVCMain10 }, #endif #if VA_CHECK_VERSION(1, 2, 0) + { FF_PROFILE_HEVC_REXT, 12, 3, 1, 1, VAProfileHEVCMain12 }, { FF_PROFILE_HEVC_REXT, 8, 3, 1, 0, VAProfileHEVCMain422_10 }, { FF_PROFILE_HEVC_REXT, 10, 3, 1, 0, VAProfileHEVCMain422_10 }, - // Four channels because this uses the AYUV format which has Alpha + { FF_PROFILE_HEVC_REXT, 12, 3, 1, 0, VAProfileHEVCMain422_12 }, + // Four channels because these use formats which have Alpha { FF_PROFILE_HEVC_REXT, 8, 4, 0, 0, VAProfileHEVCMain444 }, + { FF_PROFILE_HEVC_REXT, 10, 4, 0, 0, VAProfileHEVCMain444_10 }, + { FF_PROFILE_HEVC_REXT, 12, 4, 0, 0, VAProfileHEVCMain444_12 }, #endif { FF_PROFILE_UNKNOWN } }; diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c index 9b455e10c9..e6c7f01f11 100644 --- a/libavcodec/vaapi_encode_vp9.c +++ b/libavcodec/vaapi_encode_vp9.c @@ -228,9 +228,11 @@ static av_cold int vaapi_encode_vp9_configure(AVCodecContext *avctx) static const VAAPIEncodeProfile vaapi_encode_vp9_profiles[] = { { FF_PROFILE_VP9_0, 8, 3, 1, 1, VAProfileVP9Profile0 }, - // Four channels because this uses the AYUV format which has Alpha + // Four channels because this uses a format which has Alpha { FF_PROFILE_VP9_1, 8, 4, 0, 0, VAProfileVP9Profile1 }, { FF_PROFILE_VP9_2, 10, 3, 1, 1, VAProfileVP9Profile2 }, + // Four channels because this uses a format which has Alpha + { FF_PROFILE_VP9_3, 10, 4, 0, 0, VAProfileVP9Profile3 }, { FF_PROFILE_UNKNOWN } }; diff --git a/libavcodec/vaapi_hevc.c b/libavcodec/vaapi_hevc.c index d82975979a..20fb36adfa 100644 --- a/libavcodec/vaapi_hevc.c +++ b/libavcodec/vaapi_hevc.c @@ -567,15 +567,24 @@ VAProfile ff_vaapi_parse_hevc_rext_profile(AVCodecContext *avctx) } #if VA_CHECK_VERSION(1, 2, 0) - if (!strcmp(profile->name, "Main 4:2:2 10") || + if (!strcmp(profile->name, "Main 12") || + !strcmp(profile->name, "Main 12 Intra")) + return VAProfileHEVCMain12; + else if (!strcmp(profile->name, "Main 4:2:2 10") || !strcmp(profile->name, "Main 4:2:2 10 Intra")) return VAProfileHEVCMain422_10; + else if (!strcmp(profile->name, "Main 4:2:2 12") || + !strcmp(profile->name, "Main 4:2:2 12 Intra")) + return VAProfileHEVCMain422_12; else if (!strcmp(profile->name, "Main 4:4:4") || !strcmp(profile->name, "Main 4:4:4 Intra")) return VAProfileHEVCMain444; else if (!strcmp(profile->name, "Main 4:4:4 10") || !strcmp(profile->name, "Main 4:4:4 10 Intra")) return VAProfileHEVCMain444_10; + else if (!strcmp(profile->name, "Main 4:4:4 12") || + !strcmp(profile->name, "Main 4:4:4 12 Intra")) + return VAProfileHEVCMain444_12; #else av_log(avctx, AV_LOG_WARNING, "HEVC profile %s is " "not supported with this VA version.\n", profile->name); diff --git a/libavcodec/vp9.c b/libavcodec/vp9.c index 621627ddc5..8105956775 100644 --- a/libavcodec/vp9.c +++ b/libavcodec/vp9.c @@ -235,6 +235,8 @@ static int update_size(AVCodecContext *avctx, int w, int h) #endif break; case AV_PIX_FMT_YUV444P: + case AV_PIX_FMT_YUV444P10: + case AV_PIX_FMT_YUV444P12: #if CONFIG_VP9_VAAPI_HWACCEL *fmtp++ = AV_PIX_FMT_VAAPI; #endif diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index 2ee5145727..c3e79907fd 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -121,6 +121,9 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = { MAP(YUY2, YUV422, YUYV422, 0), #ifdef VA_FOURCC_Y210 MAP(Y210, YUV422_10, Y210, 0), +#endif +#ifdef VA_FOURCC_Y216 + MAP(Y216, YUV422_12, Y216, 0), #endif MAP(411P, YUV411, YUV411P, 0), MAP(422V, YUV422, YUV440P, 0), @@ -129,6 +132,9 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = { MAP(Y800, YUV400, GRAY8, 0), #ifdef VA_FOURCC_P010 MAP(P010, YUV420_10BPP, P010, 0), +#endif +#ifdef VA_FOURCC_P016 + MAP(P016, YUV420_12, P016, 0), #endif MAP(BGRA, RGB32, BGRA, 0), MAP(BGRX, RGB32, BGR0, 0), @@ -143,6 +149,12 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = { #ifdef VA_FOURCC_X2R10G10B10 MAP(X2R10G10B10, RGB32_10, X2RGB10, 0), #endif +#ifdef VA_FOURCC_Y410 + MAP(Y410, YUV444_10, Y410, 0), +#endif +#ifdef VA_FOURCC_Y416 + MAP(Y416, YUV444_12, Y416, 0), +#endif }; #undef MAP -- 2.34.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 16+ messages in thread
* [FFmpeg-devel] [PATCH 0/3] V4: VAAPI: Add high bit depth encode/decode support @ 2022-08-16 5:02 Philip Langdale 2022-08-16 5:02 ` [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles Philip Langdale 0 siblings, 1 reply; 16+ messages in thread From: Philip Langdale @ 2022-08-16 5:02 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Philip Langdale This changeset fills in support for the remaining high bit depth formats the VAAPI exposes. This requires adding more weird Microsoft pixel formats and then mapping them in the VAAPI code. I've also enabled hw mapping between VAAPI and vulkan for sufficiently simple formats. * V2: Added vaapi/vulkan mapping support * V3: Updated to use explicit 12bit pixel formats after I learned tha the driver supports them and doesn't force the use of 16bit formats. * V4: Fixed incorrect chunk in wrong change Philip Langdale (3): lavu/pixfmt: Add P012, Y212, Y410, and Y412 formats lavc/vaapi: Add support for remaining 10/12bit profiles lavu/hwcontext_[vaapi|vulkan]: support mapping VUYA, P012, and Y412 libavcodec/hevcdec.c | 8 +++ libavcodec/vaapi_decode.c | 13 ++++ libavcodec/vaapi_encode.c | 10 ++- libavcodec/vaapi_encode_h265.c | 6 +- libavcodec/vaapi_encode_vp9.c | 4 +- libavcodec/vaapi_hevc.c | 11 +++- libavcodec/vp9.c | 2 + libavutil/hwcontext_vaapi.c | 21 +++++++ libavutil/hwcontext_vulkan.c | 10 +++ libavutil/pixdesc.c | 101 ++++++++++++++++++++++++++++++- libavutil/pixfmt.h | 16 +++++ tests/ref/fate/imgutils | 8 +++ tests/ref/fate/sws-pixdesc-query | 42 +++++++++++++ 13 files changed, 246 insertions(+), 6 deletions(-) -- 2.34.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 16+ messages in thread
* [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles 2022-08-16 5:02 [FFmpeg-devel] [PATCH 0/3] V4: VAAPI: Add high bit depth encode/decode support Philip Langdale @ 2022-08-16 5:02 ` Philip Langdale 0 siblings, 0 replies; 16+ messages in thread From: Philip Langdale @ 2022-08-16 5:02 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Philip Langdale With the necessary pixel formats defined, we can now expose support for the remaining 10/12bit combinations that VAAPI can handle. Specifically, we are adding support for: * HEVC ** 12bit 420 ** 10bit 422 ** 12bit 422 ** 10bit 444 ** 12bit 444 * VP9 ** 10bit 444 ** 12bit 444 These obviously require actual hardware support to be usable, but where that exists, it is now enabled. I had to tweak the format matching logic slightly to account for the reduced size alpha channel in Y410, as we can't assume all channels are the same depth as the first channel. Signed-off-by: Philip Langdale <philipl@overt.org> --- libavcodec/hevcdec.c | 8 ++++++++ libavcodec/vaapi_decode.c | 13 +++++++++++++ libavcodec/vaapi_encode.c | 8 +++++++- libavcodec/vaapi_encode_h265.c | 6 +++++- libavcodec/vaapi_encode_vp9.c | 4 +++- libavcodec/vaapi_hevc.c | 11 ++++++++++- libavcodec/vp9.c | 2 ++ libavutil/hwcontext_vaapi.c | 12 ++++++++++++ 8 files changed, 60 insertions(+), 4 deletions(-) diff --git a/libavcodec/hevcdec.c b/libavcodec/hevcdec.c index f1be8af2cd..1a895800a6 100644 --- a/libavcodec/hevcdec.c +++ b/libavcodec/hevcdec.c @@ -481,11 +481,19 @@ static enum AVPixelFormat get_format(HEVCContext *s, const HEVCSPS *sps) #endif case AV_PIX_FMT_YUV420P12: case AV_PIX_FMT_YUV444P12: +#if CONFIG_HEVC_VAAPI_HWACCEL + *fmt++ = AV_PIX_FMT_VAAPI; +#endif #if CONFIG_HEVC_VDPAU_HWACCEL *fmt++ = AV_PIX_FMT_VDPAU; #endif #if CONFIG_HEVC_NVDEC_HWACCEL *fmt++ = AV_PIX_FMT_CUDA; +#endif + break; + case AV_PIX_FMT_YUV422P12: +#if CONFIG_HEVC_VAAPI_HWACCEL + *fmt++ = AV_PIX_FMT_VAAPI; #endif break; } diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c index bc2d3ed803..8c6fd45f3d 100644 --- a/libavcodec/vaapi_decode.c +++ b/libavcodec/vaapi_decode.c @@ -262,16 +262,28 @@ static const struct { MAP(YUY2, YUYV422), #ifdef VA_FOURCC_Y210 MAP(Y210, Y210), +#endif +#ifdef VA_FOURCC_Y212 + MAP(Y212, Y212), #endif // 4:4:0 MAP(422V, YUV440P), // 4:4:4 MAP(444P, YUV444P), MAP(AYUV, VUYA), +#ifdef VA_FOURCC_Y410 + MAP(Y410, Y410), +#endif +#ifdef VA_FOURCC_Y412 + MAP(Y412, Y412), +#endif // 4:2:0 10-bit #ifdef VA_FOURCC_P010 MAP(P010, P010), #endif +#ifdef VA_FOURCC_P012 + MAP(P012, P012), +#endif #ifdef VA_FOURCC_I010 MAP(I010, YUV420P10), #endif @@ -415,6 +427,7 @@ static const struct { #if VA_CHECK_VERSION(0, 39, 0) MAP(VP9, VP9_1, VP9Profile1 ), MAP(VP9, VP9_2, VP9Profile2 ), + MAP(VP9, VP9_3, VP9Profile3 ), #endif #if VA_CHECK_VERSION(1, 8, 0) MAP(AV1, AV1_MAIN, AV1Profile0), diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index f13daa5cff..e45ba1938a 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -1305,7 +1305,11 @@ static const VAAPIEncodeRTFormat vaapi_encode_rt_formats[] = { { "YUV420", VA_RT_FORMAT_YUV420, 8, 3, 1, 1 }, { "YUV422", VA_RT_FORMAT_YUV422, 8, 3, 1, 0 }, #if VA_CHECK_VERSION(1, 2, 0) + { "YUV420_12", VA_RT_FORMAT_YUV420_12, 12, 3, 1, 1 }, { "YUV422_10", VA_RT_FORMAT_YUV422_10, 10, 3, 1, 0 }, + { "YUV422_12", VA_RT_FORMAT_YUV422_12, 12, 3, 1, 0 }, + { "YUV444_10", VA_RT_FORMAT_YUV444_10, 10, 4, 0, 0 }, + { "YUV444_12", VA_RT_FORMAT_YUV444_12, 12, 4, 0, 0 }, #endif { "YUV444", VA_RT_FORMAT_YUV444, 8, 3, 0, 0 }, { "AYUV", VA_RT_FORMAT_YUV444, 8, 4, 0, 0 }, @@ -1365,7 +1369,9 @@ static av_cold int vaapi_encode_profile_entrypoint(AVCodecContext *avctx) } depth = desc->comp[0].depth; for (i = 1; i < desc->nb_components; i++) { - if (desc->comp[i].depth != depth) { + // We do not apply this depth requirement to the fourth component as + // that will be the alpha channel when present, which can be smaller. + if (i < 3 && desc->comp[i].depth != depth) { av_log(avctx, AV_LOG_ERROR, "Invalid input pixfmt (%s).\n", desc->name); return AVERROR(EINVAL); diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 1de323af78..c2e91a8449 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -1276,10 +1276,14 @@ static const VAAPIEncodeProfile vaapi_encode_h265_profiles[] = { { FF_PROFILE_HEVC_REXT, 10, 3, 1, 1, VAProfileHEVCMain10 }, #endif #if VA_CHECK_VERSION(1, 2, 0) + { FF_PROFILE_HEVC_REXT, 12, 3, 1, 1, VAProfileHEVCMain12 }, { FF_PROFILE_HEVC_REXT, 8, 3, 1, 0, VAProfileHEVCMain422_10 }, { FF_PROFILE_HEVC_REXT, 10, 3, 1, 0, VAProfileHEVCMain422_10 }, - // Four channels because this uses the AYUV format which has Alpha + { FF_PROFILE_HEVC_REXT, 12, 3, 1, 0, VAProfileHEVCMain422_12 }, + // Four channels because these use formats which have Alpha { FF_PROFILE_HEVC_REXT, 8, 4, 0, 0, VAProfileHEVCMain444 }, + { FF_PROFILE_HEVC_REXT, 10, 4, 0, 0, VAProfileHEVCMain444_10 }, + { FF_PROFILE_HEVC_REXT, 12, 4, 0, 0, VAProfileHEVCMain444_12 }, #endif { FF_PROFILE_UNKNOWN } }; diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c index 9b455e10c9..e6c7f01f11 100644 --- a/libavcodec/vaapi_encode_vp9.c +++ b/libavcodec/vaapi_encode_vp9.c @@ -228,9 +228,11 @@ static av_cold int vaapi_encode_vp9_configure(AVCodecContext *avctx) static const VAAPIEncodeProfile vaapi_encode_vp9_profiles[] = { { FF_PROFILE_VP9_0, 8, 3, 1, 1, VAProfileVP9Profile0 }, - // Four channels because this uses the AYUV format which has Alpha + // Four channels because this uses a format which has Alpha { FF_PROFILE_VP9_1, 8, 4, 0, 0, VAProfileVP9Profile1 }, { FF_PROFILE_VP9_2, 10, 3, 1, 1, VAProfileVP9Profile2 }, + // Four channels because this uses a format which has Alpha + { FF_PROFILE_VP9_3, 10, 4, 0, 0, VAProfileVP9Profile3 }, { FF_PROFILE_UNKNOWN } }; diff --git a/libavcodec/vaapi_hevc.c b/libavcodec/vaapi_hevc.c index d82975979a..20fb36adfa 100644 --- a/libavcodec/vaapi_hevc.c +++ b/libavcodec/vaapi_hevc.c @@ -567,15 +567,24 @@ VAProfile ff_vaapi_parse_hevc_rext_profile(AVCodecContext *avctx) } #if VA_CHECK_VERSION(1, 2, 0) - if (!strcmp(profile->name, "Main 4:2:2 10") || + if (!strcmp(profile->name, "Main 12") || + !strcmp(profile->name, "Main 12 Intra")) + return VAProfileHEVCMain12; + else if (!strcmp(profile->name, "Main 4:2:2 10") || !strcmp(profile->name, "Main 4:2:2 10 Intra")) return VAProfileHEVCMain422_10; + else if (!strcmp(profile->name, "Main 4:2:2 12") || + !strcmp(profile->name, "Main 4:2:2 12 Intra")) + return VAProfileHEVCMain422_12; else if (!strcmp(profile->name, "Main 4:4:4") || !strcmp(profile->name, "Main 4:4:4 Intra")) return VAProfileHEVCMain444; else if (!strcmp(profile->name, "Main 4:4:4 10") || !strcmp(profile->name, "Main 4:4:4 10 Intra")) return VAProfileHEVCMain444_10; + else if (!strcmp(profile->name, "Main 4:4:4 12") || + !strcmp(profile->name, "Main 4:4:4 12 Intra")) + return VAProfileHEVCMain444_12; #else av_log(avctx, AV_LOG_WARNING, "HEVC profile %s is " "not supported with this VA version.\n", profile->name); diff --git a/libavcodec/vp9.c b/libavcodec/vp9.c index 621627ddc5..8105956775 100644 --- a/libavcodec/vp9.c +++ b/libavcodec/vp9.c @@ -235,6 +235,8 @@ static int update_size(AVCodecContext *avctx, int w, int h) #endif break; case AV_PIX_FMT_YUV444P: + case AV_PIX_FMT_YUV444P10: + case AV_PIX_FMT_YUV444P12: #if CONFIG_VP9_VAAPI_HWACCEL *fmtp++ = AV_PIX_FMT_VAAPI; #endif diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index 2ee5145727..6c057aa5fd 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -121,6 +121,9 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = { MAP(YUY2, YUV422, YUYV422, 0), #ifdef VA_FOURCC_Y210 MAP(Y210, YUV422_10, Y210, 0), +#endif +#ifdef VA_FOURCC_Y212 + MAP(Y212, YUV422_12, Y212, 0), #endif MAP(411P, YUV411, YUV411P, 0), MAP(422V, YUV422, YUV440P, 0), @@ -129,6 +132,9 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = { MAP(Y800, YUV400, GRAY8, 0), #ifdef VA_FOURCC_P010 MAP(P010, YUV420_10BPP, P010, 0), +#endif +#ifdef VA_FOURCC_P012 + MAP(P012, YUV420_12, P012, 0), #endif MAP(BGRA, RGB32, BGRA, 0), MAP(BGRX, RGB32, BGR0, 0), @@ -143,6 +149,12 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = { #ifdef VA_FOURCC_X2R10G10B10 MAP(X2R10G10B10, RGB32_10, X2RGB10, 0), #endif +#ifdef VA_FOURCC_Y410 + MAP(Y410, YUV444_10, Y410, 0), +#endif +#ifdef VA_FOURCC_Y412 + MAP(Y412, YUV444_12, Y412, 0), +#endif }; #undef MAP -- 2.34.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 16+ messages in thread
* [FFmpeg-devel] [PATCH 0/3] V5: VAAPI: Add high bit depth encode/decode support
@ 2022-08-26 2:17 Philip Langdale
2022-08-26 2:17 ` [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles Philip Langdale
0 siblings, 1 reply; 16+ messages in thread
From: Philip Langdale @ 2022-08-26 2:17 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Philip Langdale
This changeset fills in support for the remaining high bit depth
formats the VAAPI exposes. This requires adding more weird Microsoft
pixel formats and then mapping them in the VAAPI code. I've also
enabled hw mapping between VAAPI and vulkan for sufficiently simple
formats.
V5: Switched to using alpha-less variants of Y410 and Y412 (called XV30
and XV36 in libdrm) to avoid forcing the overhead of alpha handling
in situations where it can't actually be used.
Philip Langdale (3):
lavu/pixfmt: Add P012, Y212, XV30, and XV36 formats
lavc/vaapi: Add support for remaining 10/12bit profiles
lavu/hwcontext_vulkan: support mapping VUYX, P012, and XV36
libavcodec/hevcdec.c | 8 +++
libavcodec/vaapi_decode.c | 13 +++++
libavcodec/vaapi_encode.c | 4 ++
libavcodec/vaapi_encode_h265.c | 4 ++
libavcodec/vaapi_encode_vp9.c | 1 +
libavcodec/vaapi_hevc.c | 11 +++-
libavcodec/vp9.c | 2 +
libavutil/hwcontext_vaapi.c | 25 +++++++++
libavutil/hwcontext_vulkan.c | 10 ++++
libavutil/pixdesc.c | 95 +++++++++++++++++++++++++++++++-
libavutil/pixfmt.h | 16 ++++++
tests/ref/fate/imgutils | 8 +++
tests/ref/fate/sws-pixdesc-query | 38 +++++++++++++
13 files changed, 233 insertions(+), 2 deletions(-)
--
2.34.1
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 16+ messages in thread* [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles 2022-08-26 2:17 [FFmpeg-devel] [PATCH 0/3] V5: VAAPI: Add high bit depth encode/decode support Philip Langdale @ 2022-08-26 2:17 ` Philip Langdale 0 siblings, 0 replies; 16+ messages in thread From: Philip Langdale @ 2022-08-26 2:17 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Philip Langdale With the necessary pixel formats defined, we can now expose support for the remaining 10/12bit combinations that VAAPI can handle. Specifically, we are adding support for: * HEVC ** 12bit 420 ** 10bit 422 ** 12bit 422 ** 10bit 444 ** 12bit 444 * VP9 ** 10bit 444 ** 12bit 444 These obviously require actual hardware support to be usable, but where that exists, it is now enabled. Note that unlike YUVA/YUVX, the Intel driver does not formally expose support for the alphaless formats XV30 and XV360, and so we are implicitly discarding the alpha from the decoder and passing undefined values for the alpha to the encoder. If a future encoder iteration was to actually do something with the alpha bits, we would need to use a formal alpha capable format or the encoder would need to explicitly accept the alphaless format. Signed-off-by: Philip Langdale <philipl@overt.org> --- libavcodec/hevcdec.c | 8 ++++++++ libavcodec/vaapi_decode.c | 13 +++++++++++++ libavcodec/vaapi_encode.c | 4 ++++ libavcodec/vaapi_encode_h265.c | 4 ++++ libavcodec/vaapi_encode_vp9.c | 1 + libavcodec/vaapi_hevc.c | 11 ++++++++++- libavcodec/vp9.c | 2 ++ libavutil/hwcontext_vaapi.c | 25 +++++++++++++++++++++++++ 8 files changed, 67 insertions(+), 1 deletion(-) diff --git a/libavcodec/hevcdec.c b/libavcodec/hevcdec.c index 477d6d9d36..fd23e791a4 100644 --- a/libavcodec/hevcdec.c +++ b/libavcodec/hevcdec.c @@ -481,11 +481,19 @@ static enum AVPixelFormat get_format(HEVCContext *s, const HEVCSPS *sps) #endif case AV_PIX_FMT_YUV420P12: case AV_PIX_FMT_YUV444P12: +#if CONFIG_HEVC_VAAPI_HWACCEL + *fmt++ = AV_PIX_FMT_VAAPI; +#endif #if CONFIG_HEVC_VDPAU_HWACCEL *fmt++ = AV_PIX_FMT_VDPAU; #endif #if CONFIG_HEVC_NVDEC_HWACCEL *fmt++ = AV_PIX_FMT_CUDA; +#endif + break; + case AV_PIX_FMT_YUV422P12: +#if CONFIG_HEVC_VAAPI_HWACCEL + *fmt++ = AV_PIX_FMT_VAAPI; #endif break; } diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c index 8c13a4f098..134f10eca5 100644 --- a/libavcodec/vaapi_decode.c +++ b/libavcodec/vaapi_decode.c @@ -262,6 +262,9 @@ static const struct { MAP(YUY2, YUYV422), #ifdef VA_FOURCC_Y210 MAP(Y210, Y210), +#endif +#ifdef VA_FOURCC_Y212 + MAP(Y212, Y212), #endif // 4:4:0 MAP(422V, YUV440P), @@ -269,11 +272,20 @@ static const struct { MAP(444P, YUV444P), #ifdef VA_FOURCC_XYUV MAP(XYUV, VUYX), +#endif +#ifdef VA_FOURCC_Y410 + MAP(Y410, XV30), +#endif +#ifdef VA_FOURCC_Y412 + MAP(Y412, XV36), #endif // 4:2:0 10-bit #ifdef VA_FOURCC_P010 MAP(P010, P010), #endif +#ifdef VA_FOURCC_P012 + MAP(P012, P012), +#endif #ifdef VA_FOURCC_I010 MAP(I010, YUV420P10), #endif @@ -417,6 +429,7 @@ static const struct { #if VA_CHECK_VERSION(0, 39, 0) MAP(VP9, VP9_1, VP9Profile1 ), MAP(VP9, VP9_2, VP9Profile2 ), + MAP(VP9, VP9_3, VP9Profile3 ), #endif #if VA_CHECK_VERSION(1, 8, 0) MAP(AV1, AV1_MAIN, AV1Profile0), diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 2dc5c96f7b..9a58661b51 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -1305,7 +1305,11 @@ static const VAAPIEncodeRTFormat vaapi_encode_rt_formats[] = { { "YUV420", VA_RT_FORMAT_YUV420, 8, 3, 1, 1 }, { "YUV422", VA_RT_FORMAT_YUV422, 8, 3, 1, 0 }, #if VA_CHECK_VERSION(1, 2, 0) + { "YUV420_12", VA_RT_FORMAT_YUV420_12, 12, 3, 1, 1 }, { "YUV422_10", VA_RT_FORMAT_YUV422_10, 10, 3, 1, 0 }, + { "YUV422_12", VA_RT_FORMAT_YUV422_12, 12, 3, 1, 0 }, + { "YUV444_10", VA_RT_FORMAT_YUV444_10, 10, 3, 0, 0 }, + { "YUV444_12", VA_RT_FORMAT_YUV444_12, 12, 3, 0, 0 }, #endif { "YUV444", VA_RT_FORMAT_YUV444, 8, 3, 0, 0 }, { "XYUV", VA_RT_FORMAT_YUV444, 8, 3, 0, 0 }, diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 967d71e998..90f44acff1 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -1276,9 +1276,13 @@ static const VAAPIEncodeProfile vaapi_encode_h265_profiles[] = { { FF_PROFILE_HEVC_REXT, 10, 3, 1, 1, VAProfileHEVCMain10 }, #endif #if VA_CHECK_VERSION(1, 2, 0) + { FF_PROFILE_HEVC_REXT, 12, 3, 1, 1, VAProfileHEVCMain12 }, { FF_PROFILE_HEVC_REXT, 8, 3, 1, 0, VAProfileHEVCMain422_10 }, { FF_PROFILE_HEVC_REXT, 10, 3, 1, 0, VAProfileHEVCMain422_10 }, + { FF_PROFILE_HEVC_REXT, 12, 3, 1, 0, VAProfileHEVCMain422_12 }, { FF_PROFILE_HEVC_REXT, 8, 3, 0, 0, VAProfileHEVCMain444 }, + { FF_PROFILE_HEVC_REXT, 10, 3, 0, 0, VAProfileHEVCMain444_10 }, + { FF_PROFILE_HEVC_REXT, 12, 3, 0, 0, VAProfileHEVCMain444_12 }, #endif { FF_PROFILE_UNKNOWN } }; diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c index 9530b2f462..38c167d2b6 100644 --- a/libavcodec/vaapi_encode_vp9.c +++ b/libavcodec/vaapi_encode_vp9.c @@ -230,6 +230,7 @@ static const VAAPIEncodeProfile vaapi_encode_vp9_profiles[] = { { FF_PROFILE_VP9_0, 8, 3, 1, 1, VAProfileVP9Profile0 }, { FF_PROFILE_VP9_1, 8, 3, 0, 0, VAProfileVP9Profile1 }, { FF_PROFILE_VP9_2, 10, 3, 1, 1, VAProfileVP9Profile2 }, + { FF_PROFILE_VP9_3, 10, 3, 0, 0, VAProfileVP9Profile3 }, { FF_PROFILE_UNKNOWN } }; diff --git a/libavcodec/vaapi_hevc.c b/libavcodec/vaapi_hevc.c index d82975979a..20fb36adfa 100644 --- a/libavcodec/vaapi_hevc.c +++ b/libavcodec/vaapi_hevc.c @@ -567,15 +567,24 @@ VAProfile ff_vaapi_parse_hevc_rext_profile(AVCodecContext *avctx) } #if VA_CHECK_VERSION(1, 2, 0) - if (!strcmp(profile->name, "Main 4:2:2 10") || + if (!strcmp(profile->name, "Main 12") || + !strcmp(profile->name, "Main 12 Intra")) + return VAProfileHEVCMain12; + else if (!strcmp(profile->name, "Main 4:2:2 10") || !strcmp(profile->name, "Main 4:2:2 10 Intra")) return VAProfileHEVCMain422_10; + else if (!strcmp(profile->name, "Main 4:2:2 12") || + !strcmp(profile->name, "Main 4:2:2 12 Intra")) + return VAProfileHEVCMain422_12; else if (!strcmp(profile->name, "Main 4:4:4") || !strcmp(profile->name, "Main 4:4:4 Intra")) return VAProfileHEVCMain444; else if (!strcmp(profile->name, "Main 4:4:4 10") || !strcmp(profile->name, "Main 4:4:4 10 Intra")) return VAProfileHEVCMain444_10; + else if (!strcmp(profile->name, "Main 4:4:4 12") || + !strcmp(profile->name, "Main 4:4:4 12 Intra")) + return VAProfileHEVCMain444_12; #else av_log(avctx, AV_LOG_WARNING, "HEVC profile %s is " "not supported with this VA version.\n", profile->name); diff --git a/libavcodec/vp9.c b/libavcodec/vp9.c index 621627ddc5..8105956775 100644 --- a/libavcodec/vp9.c +++ b/libavcodec/vp9.c @@ -235,6 +235,8 @@ static int update_size(AVCodecContext *avctx, int w, int h) #endif break; case AV_PIX_FMT_YUV444P: + case AV_PIX_FMT_YUV444P10: + case AV_PIX_FMT_YUV444P12: #if CONFIG_VP9_VAAPI_HWACCEL *fmtp++ = AV_PIX_FMT_VAAPI; #endif diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index 78205425ee..9ba5225ad2 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -121,6 +121,9 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = { MAP(YUY2, YUV422, YUYV422, 0), #ifdef VA_FOURCC_Y210 MAP(Y210, YUV422_10, Y210, 0), +#endif +#ifdef VA_FOURCC_Y212 + MAP(Y212, YUV422_12, Y212, 0), #endif MAP(411P, YUV411, YUV411P, 0), MAP(422V, YUV422, YUV440P, 0), @@ -131,6 +134,9 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = { MAP(Y800, YUV400, GRAY8, 0), #ifdef VA_FOURCC_P010 MAP(P010, YUV420_10BPP, P010, 0), +#endif +#ifdef VA_FOURCC_P012 + MAP(P012, YUV420_12, P012, 0), #endif MAP(BGRA, RGB32, BGRA, 0), MAP(BGRX, RGB32, BGR0, 0), @@ -145,6 +151,16 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = { #ifdef VA_FOURCC_X2R10G10B10 MAP(X2R10G10B10, RGB32_10, X2RGB10, 0), #endif +#ifdef VA_FOURCC_Y410 + // libva doesn't include a fourcc for XV30 and the driver only declares + // support for Y410, so we must fudge the mapping here. + MAP(Y410, YUV444_10, XV30, 0), +#endif +#ifdef VA_FOURCC_Y412 + // libva doesn't include a fourcc for XV36 and the driver only declares + // support for Y412, so we must fudge the mapping here. + MAP(Y412, YUV444_12, XV36, 0), +#endif }; #undef MAP @@ -1000,6 +1016,9 @@ static const struct { DRM_MAP(NV12, 1, DRM_FORMAT_NV12), #if defined(VA_FOURCC_P010) && defined(DRM_FORMAT_R16) DRM_MAP(P010, 2, DRM_FORMAT_R16, DRM_FORMAT_RG1616), +#endif +#if defined(VA_FOURCC_P012) && defined(DRM_FORMAT_R16) + DRM_MAP(P012, 2, DRM_FORMAT_R16, DRM_FORMAT_RG1616), #endif DRM_MAP(BGRA, 1, DRM_FORMAT_ARGB8888), DRM_MAP(BGRX, 1, DRM_FORMAT_XRGB8888), @@ -1014,6 +1033,12 @@ static const struct { #if defined(VA_FOURCC_XYUV) && defined(DRM_FORMAT_XYUV8888) DRM_MAP(XYUV, 1, DRM_FORMAT_XYUV8888), #endif +#if defined(VA_FOURCC_Y412) && defined(DRM_FORMAT_XVYU2101010) + DRM_MAP(Y410, 1, DRM_FORMAT_XVYU2101010), +#endif +#if defined(VA_FOURCC_Y412) && defined(DRM_FORMAT_XVYU12_16161616) + DRM_MAP(Y412, 1, DRM_FORMAT_XVYU12_16161616), +#endif }; #undef DRM_MAP -- 2.34.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2022-08-26 2:18 UTC | newest] Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-08-14 21:33 [FFmpeg-devel] [PATCH 0/3] V2: VAAPI: Add high bit depth encode/decode support Philip Langdale 2022-08-14 21:33 ` [FFmpeg-devel] [PATCH 1/3] lavu/pixfmt: Add Y216, Y410, and Y416 formats Philip Langdale 2022-08-15 6:12 ` Xiang, Haihao 2022-08-15 17:42 ` Philip Langdale 2022-08-15 23:06 ` Philip Langdale 2022-08-14 21:33 ` [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles Philip Langdale 2022-08-15 22:10 ` Michael Niedermayer 2022-08-15 22:23 ` Philip Langdale 2022-08-16 17:29 ` Michael Niedermayer 2022-08-16 19:52 ` Philip Langdale 2022-08-16 19:55 ` Philip Langdale 2022-08-17 14:34 ` Michael Niedermayer 2022-08-14 21:33 ` [FFmpeg-devel] [PATCH 3/3] lavu/hwcontext_[vaapi|vulkan]: support mapping VUYA and Y416 Philip Langdale 2022-08-16 4:36 [FFmpeg-devel] [PATCH 0/3] V3: VAAPI: Add high bit depth encode/decode support Philip Langdale 2022-08-16 4:36 ` [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles Philip Langdale 2022-08-16 5:02 [FFmpeg-devel] [PATCH 0/3] V4: VAAPI: Add high bit depth encode/decode support Philip Langdale 2022-08-16 5:02 ` [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles Philip Langdale 2022-08-26 2:17 [FFmpeg-devel] [PATCH 0/3] V5: VAAPI: Add high bit depth encode/decode support Philip Langdale 2022-08-26 2:17 ` [FFmpeg-devel] [PATCH 2/3] lavc/vaapi: Add support for remaining 10/12bit profiles Philip Langdale
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel This inbox may be cloned and mirrored by anyone: git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \ ffmpegdev@gitmailbox.com public-inbox-index ffmpegdev Example config snippet for mirrors. AGPL code for this site: git clone https://public-inbox.org/public-inbox.git