[FFmpeg-devel] [PATCH 0/2] lavc/vaapi: More 4:4:4 changes

[FFmpeg-devel] [PATCH 0/2] lavc/vaapi: More 4:4:4 changes

[Philip Langdale]

jamrial sent me off down a rabbit hole when he asked me to test his
swscale change, and here are the results.

Philip Langdale (2):
  lavc/vaapi_decode: add missing flag when picking best pixel format
  lavc/vaapi_encode: enable 8bit 4:4:4 encoding for HEVC and VP9

 libavcodec/vaapi_decode.c      | 1 +
 libavcodec/vaapi_encode.c      | 1 +
 libavcodec/vaapi_encode_h265.c | 2 ++
 libavcodec/vaapi_encode_vp9.c  | 2 ++
 4 files changed, 6 insertions(+)

-- 
2.34.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 1/2] lavc/vaapi_decode: add missing flag when picking best pixel format

[Philip Langdale]

vaapi_decode_find_best_format currently does not set the
VA_SURFACE_ATTRIB_SETTABLE flag on the pixel format attribute that it
returns.

Without this flag, the attribute will be ignored by vaCreateSurfaces,
meaning that the driver's default logic for picking a pixel format will
kick in.

So far, this hasn't produced visible problems, but when trying to
decode 4:4:4 content, at least on Intel, the driver will pick the
444P planar format, even though the decoder can only return the AYUV
packed format.

The hwcontext_vaapi code that sets surface attributes when picking
formats does not have this bug.

Applications may use their own logic for finding the best format, and
so may not hit this bug. eg: mpv is unaffected.

Signed-off-by: Philip Langdale <philipl@overt.org>
---
 libavcodec/vaapi_decode.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c
index db48efc3ed..38813eb8e4 100644
--- a/libavcodec/vaapi_decode.c
+++ b/libavcodec/vaapi_decode.c
@@ -358,6 +358,7 @@ static int vaapi_decode_find_best_format(AVCodecContext *avctx,
 
         ctx->pixel_format_attribute = (VASurfaceAttrib) {
             .type          = VASurfaceAttribPixelFormat,
+            .flags         = VA_SURFACE_ATTRIB_SETTABLE,
             .value.value.i = best_fourcc,
         };
 
-- 
2.34.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/2] lavc/vaapi_decode: add missing flag when picking best pixel format

[Wang, Fei W]

On Thu, 2022-08-04 at 20:59 -0700, Philip Langdale wrote:
> vaapi_decode_find_best_format currently does not set the
> VA_SURFACE_ATTRIB_SETTABLE flag on the pixel format attribute that it
> returns.
> 
> Without this flag, the attribute will be ignored by vaCreateSurfaces,
> meaning that the driver's default logic for picking a pixel format
> will
> kick in.
> 
> So far, this hasn't produced visible problems, but when trying to
> decode 4:4:4 content, at least on Intel, the driver will pick the
> 444P planar format, even though the decoder can only return the AYUV
> packed format.
> 
> The hwcontext_vaapi code that sets surface attributes when picking
> formats does not have this bug.
> 
> Applications may use their own logic for finding the best format, and
> so may not hit this bug. eg: mpv is unaffected.
> 
> Signed-off-by: Philip Langdale <philipl@overt.org>
> ---
>  libavcodec/vaapi_decode.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c
> index db48efc3ed..38813eb8e4 100644
> --- a/libavcodec/vaapi_decode.c
> +++ b/libavcodec/vaapi_decode.c
> @@ -358,6 +358,7 @@ static int
> vaapi_decode_find_best_format(AVCodecContext *avctx,
>  
>          ctx->pixel_format_attribute = (VASurfaceAttrib) {
>              .type          = VASurfaceAttribPixelFormat,
> +            .flags         = VA_SURFACE_ATTRIB_SETTABLE,

Better to fill .value.type with VAGenericValueTypeInteger together:
https://github.com/intel/media-driver/blob/4c95e8ef1e98cac661412d02f108e4e1c94d3556/media_driver/linux/common/ddi/media_libva.cpp#L2780

Thanks
Fei

>              .value.value.i = best_fourcc,
>          };
>  
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/2] lavc/vaapi_decode: add missing flag when picking best pixel format

[Philip Langdale]

On Fri, 5 Aug 2022 05:16:19 +0000
"Wang, Fei W" <fei.w.wang-at-intel.com@ffmpeg.org> wrote:

> 
> Better to fill .value.type with VAGenericValueTypeInteger together:
> https://github.com/intel/media-driver/blob/4c95e8ef1e98cac661412d02f108e4e1c94d3556/media_driver/linux/common/ddi/media_libva.cpp#L2780
> 

Thanks! I forgot about that.

--phil
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 2/2] lavc/vaapi_encode: enable 8bit 4:4:4 encoding for HEVC and VP9

[Philip Langdale]

Sufficiently recent Intel hardware is able to do encoding of 8bit 4:4:4
content in HEVC and VP9. The main requirement here is that the frames
must be provided in the AYUV format.

Enabling support is done by adding the appropriate encoding profiles
and noting that AYUV is officially a four channel format with alpha so
we must state that we expect all four channels.

Also, there are currently very limited ways to get data into the right
format. While our VUYA format exists, someone still has to write the
swscale code to be able to convert to it. Without that, you're stuck
only able to encode frames that came out of the vaapi decoder in the
right format.

Signed-off-by: Philip Langdale <philipl@overt.org>
---
 libavcodec/vaapi_encode.c      | 1 +
 libavcodec/vaapi_encode_h265.c | 2 ++
 libavcodec/vaapi_encode_vp9.c  | 2 ++
 3 files changed, 5 insertions(+)

diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
index 284ce29888..f13daa5cff 100644
--- a/libavcodec/vaapi_encode.c
+++ b/libavcodec/vaapi_encode.c
@@ -1308,6 +1308,7 @@ static const VAAPIEncodeRTFormat vaapi_encode_rt_formats[] = {
     { "YUV422_10", VA_RT_FORMAT_YUV422_10,    10, 3, 1, 0 },
 #endif
     { "YUV444",    VA_RT_FORMAT_YUV444,        8, 3, 0, 0 },
+    { "AYUV",      VA_RT_FORMAT_YUV444,        8, 4, 0, 0 },
     { "YUV411",    VA_RT_FORMAT_YUV411,        8, 3, 2, 0 },
 #if VA_CHECK_VERSION(0, 38, 1)
     { "YUV420_10", VA_RT_FORMAT_YUV420_10BPP, 10, 3, 1, 1 },
diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c
index d5375add22..1de323af78 100644
--- a/libavcodec/vaapi_encode_h265.c
+++ b/libavcodec/vaapi_encode_h265.c
@@ -1278,6 +1278,8 @@ static const VAAPIEncodeProfile vaapi_encode_h265_profiles[] = {
 #if VA_CHECK_VERSION(1, 2, 0)
     { FF_PROFILE_HEVC_REXT,     8, 3, 1, 0, VAProfileHEVCMain422_10 },
     { FF_PROFILE_HEVC_REXT,    10, 3, 1, 0, VAProfileHEVCMain422_10 },
+    // Four channels because this uses the AYUV format which has Alpha
+    { FF_PROFILE_HEVC_REXT,     8, 4, 0, 0, VAProfileHEVCMain444 },
 #endif
     { FF_PROFILE_UNKNOWN }
 };
diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c
index 892ad770c6..9b455e10c9 100644
--- a/libavcodec/vaapi_encode_vp9.c
+++ b/libavcodec/vaapi_encode_vp9.c
@@ -228,6 +228,8 @@ static av_cold int vaapi_encode_vp9_configure(AVCodecContext *avctx)
 
 static const VAAPIEncodeProfile vaapi_encode_vp9_profiles[] = {
     { FF_PROFILE_VP9_0,  8, 3, 1, 1, VAProfileVP9Profile0 },
+    // Four channels because this uses the AYUV format which has Alpha
+    { FF_PROFILE_VP9_1,  8, 4, 0, 0, VAProfileVP9Profile1 },
     { FF_PROFILE_VP9_2, 10, 3, 1, 1, VAProfileVP9Profile2 },
     { FF_PROFILE_UNKNOWN }
 };
-- 
2.34.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2 0/2] lavc/vaapi: More 4:4:4 changes

[Philip Langdale]

Since v1:
* Added missing value type in decoder fix. Thanks Fei.
* Tested with jamrial's pending VUYA swscale output patch.

Philip Langdale (2):
  lavc/vaapi_decode: add missing flag when picking best pixel format
  lavc/vaapi_encode: enable 8bit 4:4:4 encoding for HEVC and VP9

 libavcodec/vaapi_decode.c      | 2 ++
 libavcodec/vaapi_encode.c      | 1 +
 libavcodec/vaapi_encode_h265.c | 2 ++
 libavcodec/vaapi_encode_vp9.c  | 2 ++
 4 files changed, 7 insertions(+)

-- 
2.34.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 2/2] lavc/vaapi_encode: enable 8bit 4:4:4 encoding for HEVC and VP9

[Philip Langdale]

Sufficiently recent Intel hardware is able to do encoding of 8bit 4:4:4
content in HEVC and VP9. The main requirement here is that the frames
must be provided in the AYUV format.

Enabling support is done by adding the appropriate encoding profiles
and noting that AYUV is officially a four channel format with alpha so
we must state that we expect all four channels.

Also, there are currently very limited ways to get data into the right
format. While our VUYA format exists, the swscale code that can convert
to VUYA is pending, but I've tested it and it works as expected.

Signed-off-by: Philip Langdale <philipl@overt.org>
---
 libavcodec/vaapi_encode.c      | 1 +
 libavcodec/vaapi_encode_h265.c | 2 ++
 libavcodec/vaapi_encode_vp9.c  | 2 ++
 3 files changed, 5 insertions(+)

diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
index 284ce29888..f13daa5cff 100644
--- a/libavcodec/vaapi_encode.c
+++ b/libavcodec/vaapi_encode.c
@@ -1308,6 +1308,7 @@ static const VAAPIEncodeRTFormat vaapi_encode_rt_formats[] = {
     { "YUV422_10", VA_RT_FORMAT_YUV422_10,    10, 3, 1, 0 },
 #endif
     { "YUV444",    VA_RT_FORMAT_YUV444,        8, 3, 0, 0 },
+    { "AYUV",      VA_RT_FORMAT_YUV444,        8, 4, 0, 0 },
     { "YUV411",    VA_RT_FORMAT_YUV411,        8, 3, 2, 0 },
 #if VA_CHECK_VERSION(0, 38, 1)
     { "YUV420_10", VA_RT_FORMAT_YUV420_10BPP, 10, 3, 1, 1 },
diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c
index d5375add22..1de323af78 100644
--- a/libavcodec/vaapi_encode_h265.c
+++ b/libavcodec/vaapi_encode_h265.c
@@ -1278,6 +1278,8 @@ static const VAAPIEncodeProfile vaapi_encode_h265_profiles[] = {
 #if VA_CHECK_VERSION(1, 2, 0)
     { FF_PROFILE_HEVC_REXT,     8, 3, 1, 0, VAProfileHEVCMain422_10 },
     { FF_PROFILE_HEVC_REXT,    10, 3, 1, 0, VAProfileHEVCMain422_10 },
+    // Four channels because this uses the AYUV format which has Alpha
+    { FF_PROFILE_HEVC_REXT,     8, 4, 0, 0, VAProfileHEVCMain444 },
 #endif
     { FF_PROFILE_UNKNOWN }
 };
diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c
index 892ad770c6..9b455e10c9 100644
--- a/libavcodec/vaapi_encode_vp9.c
+++ b/libavcodec/vaapi_encode_vp9.c
@@ -228,6 +228,8 @@ static av_cold int vaapi_encode_vp9_configure(AVCodecContext *avctx)
 
 static const VAAPIEncodeProfile vaapi_encode_vp9_profiles[] = {
     { FF_PROFILE_VP9_0,  8, 3, 1, 1, VAProfileVP9Profile0 },
+    // Four channels because this uses the AYUV format which has Alpha
+    { FF_PROFILE_VP9_1,  8, 4, 0, 0, VAProfileVP9Profile1 },
     { FF_PROFILE_VP9_2, 10, 3, 1, 1, VAProfileVP9Profile2 },
     { FF_PROFILE_UNKNOWN }
 };
-- 
2.34.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/2] lavc/vaapi_encode: enable 8bit 4:4:4 encoding for HEVC and VP9

[Xiang, Haihao]

On Fri, 2022-08-05 at 20:27 -0700, Philip Langdale wrote:
> Sufficiently recent Intel hardware is able to do encoding of 8bit 4:4:4
> content in HEVC and VP9. The main requirement here is that the frames
> must be provided in the AYUV format.
> 
> Enabling support is done by adding the appropriate encoding profiles
> and noting that AYUV is officially a four channel format with alpha so
> we must state that we expect all four channels.
> 
> Also, there are currently very limited ways to get data into the right
> format. While our VUYA format exists, the swscale code that can convert
> to VUYA is pending, but I've tested it and it works as expected.
> 
> Signed-off-by: Philip Langdale <philipl@overt.org>
> ---
>  libavcodec/vaapi_encode.c      | 1 +
>  libavcodec/vaapi_encode_h265.c | 2 ++
>  libavcodec/vaapi_encode_vp9.c  | 2 ++
>  3 files changed, 5 insertions(+)
> 
> diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
> index 284ce29888..f13daa5cff 100644
> --- a/libavcodec/vaapi_encode.c
> +++ b/libavcodec/vaapi_encode.c
> @@ -1308,6 +1308,7 @@ static const VAAPIEncodeRTFormat
> vaapi_encode_rt_formats[] = {
>      { "YUV422_10", VA_RT_FORMAT_YUV422_10,    10, 3, 1, 0 },
>  #endif
>      { "YUV444",    VA_RT_FORMAT_YUV444,        8, 3, 0, 0 },
> +    { "AYUV",      VA_RT_FORMAT_YUV444,        8, 4, 0, 0 },
>      { "YUV411",    VA_RT_FORMAT_YUV411,        8, 3, 2, 0 },
>  #if VA_CHECK_VERSION(0, 38, 1)
>      { "YUV420_10", VA_RT_FORMAT_YUV420_10BPP, 10, 3, 1, 1 },
> diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c
> index d5375add22..1de323af78 100644
> --- a/libavcodec/vaapi_encode_h265.c
> +++ b/libavcodec/vaapi_encode_h265.c
> @@ -1278,6 +1278,8 @@ static const VAAPIEncodeProfile
> vaapi_encode_h265_profiles[] = {
>  #if VA_CHECK_VERSION(1, 2, 0)
>      { FF_PROFILE_HEVC_REXT,     8, 3, 1, 0, VAProfileHEVCMain422_10 },
>      { FF_PROFILE_HEVC_REXT,    10, 3, 1, 0, VAProfileHEVCMain422_10 },
> +    // Four channels because this uses the AYUV format which has Alpha
> +    { FF_PROFILE_HEVC_REXT,     8, 4, 0, 0, VAProfileHEVCMain444 },
>  #endif
>      { FF_PROFILE_UNKNOWN }
>  };
> diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c
> index 892ad770c6..9b455e10c9 100644
> --- a/libavcodec/vaapi_encode_vp9.c
> +++ b/libavcodec/vaapi_encode_vp9.c
> @@ -228,6 +228,8 @@ static av_cold int
> vaapi_encode_vp9_configure(AVCodecContext *avctx)
>  
>  static const VAAPIEncodeProfile vaapi_encode_vp9_profiles[] = {
>      { FF_PROFILE_VP9_0,  8, 3, 1, 1, VAProfileVP9Profile0 },
> +    // Four channels because this uses the AYUV format which has Alpha
> +    { FF_PROFILE_VP9_1,  8, 4, 0, 0, VAProfileVP9Profile1 },
>      { FF_PROFILE_VP9_2, 10, 3, 1, 1, VAProfileVP9Profile2 },
>      { FF_PROFILE_UNKNOWN }
>  };

LGTM, thx

-Haihao

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".