[FFmpeg-devel] [PATCH 0/3] vaapi: Add support for decoding 8bit 4:4:4 content

[FFmpeg-devel] [PATCH 0/3] vaapi: Add support for decoding 8bit 4:4:4 content

[Philip Langdale]

Recent Intel hardware is capable of decoding 8bit 4:4:4 HEVC and VP9,
but it uses a packed format that Microsoft wants, and which we don't
currently know about.

This patch series adds support for the AYUV format and declares support
in VAAPI.

Mark has noted that due to there never being any actual Alpha produced
by the hardware, we should use an XYUV format. I've not done so here
because libva does declare an explicit XYUV format, which I assume we
will see used in the future, and I didn't want to fudge the formats so
that they lose the nominal distinction. But I'm happy to change that if
desired.

Philip Langdale (3):
  lavu/pixfmt: Add packed 4:4:4 format
  lavu/hwcontext_vaapi: Map the AYUV format
  lavc/vaapi: Declare support for decoding 8bit 4:4:4 content

 Changelog                        |  1 +
 doc/APIchanges                   |  3 +++
 libavcodec/hevcdec.c             |  3 +++
 libavcodec/vaapi_decode.c        |  2 ++
 libavcodec/version.h             |  2 +-
 libavcodec/vp9.c                 |  5 +++++
 libavutil/hwcontext_vaapi.c      |  1 +
 libavutil/pixdesc.c              | 13 +++++++++++++
 libavutil/pixfmt.h               |  2 ++
 libavutil/version.h              |  4 ++--
 tests/ref/fate/imgutils          |  1 +
 tests/ref/fate/sws-pixdesc-query |  4 ++++
 12 files changed, 38 insertions(+), 3 deletions(-)

-- 
2.34.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 1/3] lavu/pixfmt: Add packed 4:4:4 format

[Philip Langdale]

The "AYUV" format is defined by Microsoft as their preferred format for
4:4:4 content, and so it is the format used by Intel VAAPI and QSV.

As Microsoft like to define their byte ordering in little-endian
fashion, the memory order is reversed, and so our pix_fmt, which
follows memory order, has a reversed name (VUYA).
---
 doc/APIchanges                   |  3 +++
 libavutil/pixdesc.c              | 13 +++++++++++++
 libavutil/pixfmt.h               |  2 ++
 libavutil/version.h              |  2 +-
 tests/ref/fate/imgutils          |  1 +
 tests/ref/fate/sws-pixdesc-query |  4 ++++
 6 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/doc/APIchanges b/doc/APIchanges
index b3563cd528..35e6fda6bc 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -14,6 +14,9 @@ libavutil:     2021-04-27
 
 API changes, most recent first:
 
+2021-11-22 - xxxxxxxxxx - lavu 57.31.100 - pixfmt.h
+  Add AV_PIX_FMT_VUYA.
+
 2022-07-xx - xxxxxxxxxx - lavu 57.30.100 - frame.h
   Add AVFrame.duration, deprecate AVFrame.pkt_duration.
 
diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c
index 6e57a82cb6..e078fd5320 100644
--- a/libavutil/pixdesc.c
+++ b/libavutil/pixdesc.c
@@ -2491,6 +2491,19 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = {
         },
         .flags = AV_PIX_FMT_FLAG_PLANAR,
     },
+    [AV_PIX_FMT_VUYA] = {
+        .name = "vuya",
+        .nb_components = 4,
+        .log2_chroma_w = 0,
+        .log2_chroma_h = 0,
+        .comp = {
+            { 0, 4, 2, 0, 8 },        /* Y */
+            { 0, 4, 1, 0, 8 },        /* U */
+            { 0, 4, 0, 0, 8 },        /* V */
+            { 0, 4, 3, 0, 8 },        /* A */
+        },
+        .flags = AV_PIX_FMT_FLAG_ALPHA,
+    },
 };
 
 static const char * const color_range_names[] = {
diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h
index 2d3927cc3f..9d1fdaf82d 100644
--- a/libavutil/pixfmt.h
+++ b/libavutil/pixfmt.h
@@ -367,6 +367,8 @@ enum AVPixelFormat {
     AV_PIX_FMT_P416BE,      ///< interleaved chroma YUV 4:4:4, 48bpp, big-endian
     AV_PIX_FMT_P416LE,      ///< interleaved chroma YUV 4:4:4, 48bpp, little-endian
 
+    AV_PIX_FMT_VUYA,        ///< packed VUYA 4:4:4, 32bpp, VUYAVUYA...
+
     AV_PIX_FMT_NB         ///< number of pixel formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions
 };
 
diff --git a/libavutil/version.h b/libavutil/version.h
index ee4c531b80..e9eefcdb2c 100644
--- a/libavutil/version.h
+++ b/libavutil/version.h
@@ -79,7 +79,7 @@
  */
 
 #define LIBAVUTIL_VERSION_MAJOR  57
-#define LIBAVUTIL_VERSION_MINOR  30
+#define LIBAVUTIL_VERSION_MINOR  31
 #define LIBAVUTIL_VERSION_MICRO 100
 
 #define LIBAVUTIL_VERSION_INT   AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \
diff --git a/tests/ref/fate/imgutils b/tests/ref/fate/imgutils
index b23d1b4c39..4ec66febb8 100644
--- a/tests/ref/fate/imgutils
+++ b/tests/ref/fate/imgutils
@@ -246,3 +246,4 @@ p216be          planes: 2, linesizes: 128 128   0   0, plane_sizes:  6144  6144
 p216le          planes: 2, linesizes: 128 128   0   0, plane_sizes:  6144  6144     0     0, plane_offsets:  6144     0     0, total_size: 12288
 p416be          planes: 2, linesizes: 128 256   0   0, plane_sizes:  6144 12288     0     0, plane_offsets:  6144     0     0, total_size: 18432
 p416le          planes: 2, linesizes: 128 256   0   0, plane_sizes:  6144 12288     0     0, plane_offsets:  6144     0     0, total_size: 18432
+vuya            planes: 1, linesizes: 256   0   0   0, plane_sizes: 12288     0     0     0, plane_offsets:     0     0     0, total_size: 12288
diff --git a/tests/ref/fate/sws-pixdesc-query b/tests/ref/fate/sws-pixdesc-query
index 76104bc5a6..bd0f1fcb82 100644
--- a/tests/ref/fate/sws-pixdesc-query
+++ b/tests/ref/fate/sws-pixdesc-query
@@ -215,6 +215,7 @@ isYUV:
   p416le
   uyvy422
   uyyvyy411
+  vuya
   xyz12be
   xyz12le
   y210be
@@ -654,6 +655,7 @@ ALPHA:
   rgb32_1
   rgba64be
   rgba64le
+  vuya
   ya16be
   ya16le
   ya8
@@ -739,6 +741,7 @@ Packed:
   rgba64le
   uyvy422
   uyyvyy411
+  vuya
   x2bgr10be
   x2bgr10le
   x2rgb10be
@@ -967,5 +970,6 @@ DataInHighBits:
 SwappedChroma:
   nv21
   nv42
+  vuya
   yvyu422
 
-- 
2.34.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 2/3] lavu/hwcontext_vaapi: Map the AYUV format

[Philip Langdale]

This is the format used by Intel VAAPI for 8bit 4:4:4 content.
---
 libavutil/hwcontext_vaapi.c | 1 +
 libavutil/version.h         | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c
index c3a98bc4b1..2ee5145727 100644
--- a/libavutil/hwcontext_vaapi.c
+++ b/libavutil/hwcontext_vaapi.c
@@ -125,6 +125,7 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = {
     MAP(411P, YUV411,  YUV411P, 0),
     MAP(422V, YUV422,  YUV440P, 0),
     MAP(444P, YUV444,  YUV444P, 0),
+    MAP(AYUV, YUV444,  VUYA,    0),
     MAP(Y800, YUV400,  GRAY8,   0),
 #ifdef VA_FOURCC_P010
     MAP(P010, YUV420_10BPP, P010, 0),
diff --git a/libavutil/version.h b/libavutil/version.h
index e9eefcdb2c..21f11f3d6b 100644
--- a/libavutil/version.h
+++ b/libavutil/version.h
@@ -80,7 +80,7 @@
 
 #define LIBAVUTIL_VERSION_MAJOR  57
 #define LIBAVUTIL_VERSION_MINOR  31
-#define LIBAVUTIL_VERSION_MICRO 100
+#define LIBAVUTIL_VERSION_MICRO 101
 
 #define LIBAVUTIL_VERSION_INT   AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \
                                                LIBAVUTIL_VERSION_MINOR, \
-- 
2.34.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 3/3] lavc/vaapi: Declare support for decoding 8bit 4:4:4 content

[Philip Langdale]

Now that we have a combination of capable hardware (new enough Intel)
and a mutually understood format ("AYUV"), we can declare support for
decoding 8bit 4:4:4 content via VAAPI.

This requires listing AYUV as a supported format, and then adding VAAPI
as a supported hwaccel for the relevant codecs (HEVC and VP9). I also
had to add VP9Profile1 to the set of supported profiles for VAAPI as it
was never relevant before.
---
 Changelog                 | 1 +
 libavcodec/hevcdec.c      | 3 +++
 libavcodec/vaapi_decode.c | 2 ++
 libavcodec/version.h      | 2 +-
 libavcodec/vp9.c          | 5 +++++
 5 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/Changelog b/Changelog
index 92780c9953..a5539d5ec3 100644
--- a/Changelog
+++ b/Changelog
@@ -7,6 +7,7 @@ version <next>:
 - ffmpeg -shortest_buf_duration option
 - ffmpeg now requires threading to be built
 - ffmpeg now runs every muxer in a separate thread
+- VAAPI hwaccel for 8bit 444 HEVC and VP9
 
 
 version 5.1:
diff --git a/libavcodec/hevcdec.c b/libavcodec/hevcdec.c
index 2d06d09b75..7070ed3f02 100644
--- a/libavcodec/hevcdec.c
+++ b/libavcodec/hevcdec.c
@@ -452,6 +452,9 @@ static enum AVPixelFormat get_format(HEVCContext *s, const HEVCSPS *sps)
 #endif
         break;
     case AV_PIX_FMT_YUV444P:
+#if CONFIG_HEVC_VAAPI_HWACCEL
+        *fmt++ = AV_PIX_FMT_VAAPI;
+#endif
 #if CONFIG_HEVC_VDPAU_HWACCEL
         *fmt++ = AV_PIX_FMT_VDPAU;
 #endif
diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c
index a7abddb06b..db48efc3ed 100644
--- a/libavcodec/vaapi_decode.c
+++ b/libavcodec/vaapi_decode.c
@@ -267,6 +267,7 @@ static const struct {
     MAP(422V, YUV440P),
     // 4:4:4
     MAP(444P, YUV444P),
+    MAP(AYUV, VUYA),
     // 4:2:0 10-bit
 #ifdef VA_FOURCC_P010
     MAP(P010, P010),
@@ -410,6 +411,7 @@ static const struct {
     MAP(VP9,         VP9_0,           VP9Profile0 ),
 #endif
 #if VA_CHECK_VERSION(0, 39, 0)
+    MAP(VP9,         VP9_1,           VP9Profile1 ),
     MAP(VP9,         VP9_2,           VP9Profile2 ),
 #endif
 #if VA_CHECK_VERSION(1, 8, 0)
diff --git a/libavcodec/version.h b/libavcodec/version.h
index f2f14eaed1..ecdbc51c74 100644
--- a/libavcodec/version.h
+++ b/libavcodec/version.h
@@ -30,7 +30,7 @@
 #include "version_major.h"
 
 #define LIBAVCODEC_VERSION_MINOR  39
-#define LIBAVCODEC_VERSION_MICRO 100
+#define LIBAVCODEC_VERSION_MICRO 101
 
 #define LIBAVCODEC_VERSION_INT  AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \
                                                LIBAVCODEC_VERSION_MINOR, \
diff --git a/libavcodec/vp9.c b/libavcodec/vp9.c
index 12ec87992c..834fbebb01 100644
--- a/libavcodec/vp9.c
+++ b/libavcodec/vp9.c
@@ -231,6 +231,11 @@ static int update_size(AVCodecContext *avctx, int w, int h)
 #endif
 #if CONFIG_VP9_VDPAU_HWACCEL
             *fmtp++ = AV_PIX_FMT_VDPAU;
+#endif
+            break;
+        case AV_PIX_FMT_YUV444P:
+#if CONFIG_VP9_VAAPI_HWACCEL
+            *fmtp++ = AV_PIX_FMT_VAAPI;
 #endif
             break;
         }
-- 
2.34.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/3] vaapi: Add support for decoding 8bit 4:4:4 content

[Philip Langdale]

On Sat, 23 Jul 2022 11:53:12 -0700
Philip Langdale <philipl@overt.org> wrote:

> Recent Intel hardware is capable of decoding 8bit 4:4:4 HEVC and VP9,
> but it uses a packed format that Microsoft wants, and which we don't
> currently know about.
> 
> This patch series adds support for the AYUV format and declares
> support in VAAPI.
> 
> Mark has noted that due to there never being any actual Alpha produced
> by the hardware, we should use an XYUV format. I've not done so here
> because libva does declare an explicit XYUV format, which I assume we
> will see used in the future, and I didn't want to fudge the formats so
> that they lose the nominal distinction. But I'm happy to change that
> if desired.
> 
> Philip Langdale (3):
>   lavu/pixfmt: Add packed 4:4:4 format
>   lavu/hwcontext_vaapi: Map the AYUV format
>   lavc/vaapi: Declare support for decoding 8bit 4:4:4 content
> 
>  Changelog                        |  1 +
>  doc/APIchanges                   |  3 +++
>  libavcodec/hevcdec.c             |  3 +++
>  libavcodec/vaapi_decode.c        |  2 ++
>  libavcodec/version.h             |  2 +-
>  libavcodec/vp9.c                 |  5 +++++
>  libavutil/hwcontext_vaapi.c      |  1 +
>  libavutil/pixdesc.c              | 13 +++++++++++++
>  libavutil/pixfmt.h               |  2 ++
>  libavutil/version.h              |  4 ++--
>  tests/ref/fate/imgutils          |  1 +
>  tests/ref/fate/sws-pixdesc-query |  4 ++++
>  12 files changed, 38 insertions(+), 3 deletions(-)
> 

I will push this in the next few days if I don't get any comments or
objections. Hai Hao confirmed in IRC that the Intel driver does
explicitly set the alpha value to 0xFF so it is correct to treat it
as an alpha value rather than undefined.

Thanks,

--phil
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/3] vaapi: Add support for decoding 8bit 4:4:4 content

[Philip Langdale]

On Fri, 29 Jul 2022 10:33:59 -0700
Philip Langdale <philipl@overt.org> wrote:

> On Sat, 23 Jul 2022 11:53:12 -0700
> Philip Langdale <philipl@overt.org> wrote:
> 
> > Recent Intel hardware is capable of decoding 8bit 4:4:4 HEVC and
> > VP9, but it uses a packed format that Microsoft wants, and which we
> > don't currently know about.
> > 
> > This patch series adds support for the AYUV format and declares
> > support in VAAPI.
> > 
> > Mark has noted that due to there never being any actual Alpha
> > produced by the hardware, we should use an XYUV format. I've not
> > done so here because libva does declare an explicit XYUV format,
> > which I assume we will see used in the future, and I didn't want to
> > fudge the formats so that they lose the nominal distinction. But
> > I'm happy to change that if desired.
> > 
> > Philip Langdale (3):
> >   lavu/pixfmt: Add packed 4:4:4 format
> >   lavu/hwcontext_vaapi: Map the AYUV format
> >   lavc/vaapi: Declare support for decoding 8bit 4:4:4 content
> > 
> >  Changelog                        |  1 +
> >  doc/APIchanges                   |  3 +++
> >  libavcodec/hevcdec.c             |  3 +++
> >  libavcodec/vaapi_decode.c        |  2 ++
> >  libavcodec/version.h             |  2 +-
> >  libavcodec/vp9.c                 |  5 +++++
> >  libavutil/hwcontext_vaapi.c      |  1 +
> >  libavutil/pixdesc.c              | 13 +++++++++++++
> >  libavutil/pixfmt.h               |  2 ++
> >  libavutil/version.h              |  4 ++--
> >  tests/ref/fate/imgutils          |  1 +
> >  tests/ref/fate/sws-pixdesc-query |  4 ++++
> >  12 files changed, 38 insertions(+), 3 deletions(-)
> >   
> 
> I will push this in the next few days if I don't get any comments or
> objections. Hai Hao confirmed in IRC that the Intel driver does
> explicitly set the alpha value to 0xFF so it is correct to treat it
> as an alpha value rather than undefined.
> 
> Thanks,
> 
> --phil

Pushed.

--phil
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".