[FFmpeg-devel] [PATCH 1/3] lavu/hwcontext_vaapi: Add a new quirk

[FFmpeg-devel] [PATCH 1/3] lavu/hwcontext_vaapi: Add a new quirk

[Xiang, Haihao]

From: Haihao Xiang <haihao.xiang@intel.com>

libva2 doesn't require a fixed surface-array any more, but some
driver/hardware combinations which rely on this are still used. To
reduce the impact to users, add a quirk for the driver/hardware
combination which supports dynamic surface pool.

Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
---
 libavutil/hwcontext_vaapi.c | 7 +++++++
 libavutil/hwcontext_vaapi.h | 6 ++++++
 2 files changed, 13 insertions(+)

diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c
index 56d03aa4cd..dae5dd4a11 100644
--- a/libavutil/hwcontext_vaapi.c
+++ b/libavutil/hwcontext_vaapi.c
@@ -390,6 +390,13 @@ static const struct {
         "Splitted-Desktop Systems VDPAU backend for VA-API",
         AV_VAAPI_DRIVER_QUIRK_SURFACE_ATTRIBUTES,
     },
+#if CONFIG_VAAPI_1
+    {
+        "New Intel iHD",
+        "Intel iHD driver for Intel(R) Gen Graphics",
+        AV_VAAPI_DRIVER_QUIRK_DYNAMIC_SURFACE_POOL,
+    },
+#endif
 };
 
 static int vaapi_device_init(AVHWDeviceContext *hwdev)
diff --git a/libavutil/hwcontext_vaapi.h b/libavutil/hwcontext_vaapi.h
index 0b2e071cb3..07014fd526 100644
--- a/libavutil/hwcontext_vaapi.h
+++ b/libavutil/hwcontext_vaapi.h
@@ -58,6 +58,12 @@ enum {
      * and the results of the vaQuerySurfaceAttributes() call will be faked.
      */
     AV_VAAPI_DRIVER_QUIRK_SURFACE_ATTRIBUTES = (1 << 3),
+
+    /**
+     * The driver (and the underlying HW) supports dynamic surface pool.
+     * The vaCreateContext() call doesn't require a fixed surface-array.
+     */
+    AV_VAAPI_DRIVER_QUIRK_DYNAMIC_SURFACE_POOL = (1 << 4),
 };
 
 /**
-- 
2.34.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 2/3] lavc/vaapi_decode: Use dynamic frame pool if possible

[Xiang, Haihao]

From: Haihao Xiang <haihao.xiang@intel.com>

This allows a downstream element stores more frames from VAAPI
decoders and fixes the broke cases for the driver/hardware
combination which doesn't rely on fixed surface-array.

For example:
$ ffmpeg -hwaccel vaapi -hwaccel_output_format vaapi \
-i input.mp4 -c:v hevc_vaapi -f null -
...
[h264 @ 0x557a075a1400] get_buffer() failed
[h264 @ 0x557a075a1400] thread_get_buffer() failed
[h264 @ 0x557a075a1400] decode_slice_header error
[h264 @ 0x557a075a1400] no frame!

Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
---
 libavcodec/vaapi_decode.c | 36 ++++++++++++++++++++----------------
 1 file changed, 20 insertions(+), 16 deletions(-)

diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c
index cca94b5336..c690e25342 100644
--- a/libavcodec/vaapi_decode.c
+++ b/libavcodec/vaapi_decode.c
@@ -598,22 +598,26 @@ static int vaapi_decode_make_config(AVCodecContext *avctx,
         if (err < 0)
             goto fail;
 
-        frames->initial_pool_size = 1;
-        // Add per-codec number of surfaces used for storing reference frames.
-        switch (avctx->codec_id) {
-        case AV_CODEC_ID_H264:
-        case AV_CODEC_ID_HEVC:
-        case AV_CODEC_ID_AV1:
-            frames->initial_pool_size += 16;
-            break;
-        case AV_CODEC_ID_VP9:
-            frames->initial_pool_size += 8;
-            break;
-        case AV_CODEC_ID_VP8:
-            frames->initial_pool_size += 3;
-            break;
-        default:
-            frames->initial_pool_size += 2;
+        if (hwctx->driver_quirks & AV_VAAPI_DRIVER_QUIRK_DYNAMIC_SURFACE_POOL)
+            frames->initial_pool_size = 0;
+        else {
+            frames->initial_pool_size = 1;
+            // Add per-codec number of surfaces used for storing reference frames.
+            switch (avctx->codec_id) {
+            case AV_CODEC_ID_H264:
+            case AV_CODEC_ID_HEVC:
+            case AV_CODEC_ID_AV1:
+                frames->initial_pool_size += 16;
+                break;
+            case AV_CODEC_ID_VP9:
+                frames->initial_pool_size += 8;
+                break;
+            case AV_CODEC_ID_VP8:
+                frames->initial_pool_size += 3;
+                break;
+            default:
+                frames->initial_pool_size += 2;
+            }
         }
     }
 
-- 
2.34.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 3/3] lavfi/vaapi_vpp: Use dynamic frame pool in outlink if possible

[Xiang, Haihao]

From: Haihao Xiang <haihao.xiang@intel.com>

This can fix the broken cases for the driver/hardware combination which
doesn't rely on fixed surface-array.

For example:
$ ffmpeg -hwaccel_output_format vaapi -hwaccel vaapi -i input.mp4 \
-vf 'scale_vaapi=w=720:h=480' -c:v hevc_vaapi -f null -

Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
---
 libavfilter/vaapi_vpp.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/libavfilter/vaapi_vpp.c b/libavfilter/vaapi_vpp.c
index 59961bfa4a..0755d674ac 100644
--- a/libavfilter/vaapi_vpp.c
+++ b/libavfilter/vaapi_vpp.c
@@ -203,7 +203,10 @@ int ff_vaapi_vpp_config_output(AVFilterLink *outlink)
     output_frames->width     = ctx->output_width;
     output_frames->height    = ctx->output_height;
 
-    output_frames->initial_pool_size = 4;
+    if (ctx->hwctx->driver_quirks & AV_VAAPI_DRIVER_QUIRK_DYNAMIC_SURFACE_POOL)
+        output_frames->initial_pool_size = 0;
+    else
+        output_frames->initial_pool_size = 4;
 
     err = ff_filter_init_hw_frames(avctx, outlink, 10);
     if (err < 0)
@@ -219,6 +222,8 @@ int ff_vaapi_vpp_config_output(AVFilterLink *outlink)
     va_frames = output_frames->hwctx;
 
     av_assert0(ctx->va_context == VA_INVALID_ID);
+    av_assert0(output_frames->initial_pool_size ||
+               (va_frames->surface_ids == NULL && va_frames->nb_surfaces == 0));
     vas = vaCreateContext(ctx->hwctx->display, ctx->va_config,
                           ctx->output_width, ctx->output_height,
                           VA_PROGRESSIVE,
-- 
2.34.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/3] lavu/hwcontext_vaapi: Add a new quirk

[Xiang, Haihao]

On Do, 2024-03-28 at 10:17 +0800, Xiang, Haihao wrote:
> From: Haihao Xiang <haihao.xiang@intel.com>
> 
> libva2 doesn't require a fixed surface-array any more, but some
> driver/hardware combinations which rely on this are still used. To
> reduce the impact to users, add a quirk for the driver/hardware
> combination which supports dynamic surface pool.
> 
> Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
> ---
>  libavutil/hwcontext_vaapi.c | 7 +++++++
>  libavutil/hwcontext_vaapi.h | 6 ++++++
>  2 files changed, 13 insertions(+)
> 
> diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c
> index 56d03aa4cd..dae5dd4a11 100644
> --- a/libavutil/hwcontext_vaapi.c
> +++ b/libavutil/hwcontext_vaapi.c
> @@ -390,6 +390,13 @@ static const struct {
>          "Splitted-Desktop Systems VDPAU backend for VA-API",
>          AV_VAAPI_DRIVER_QUIRK_SURFACE_ATTRIBUTES,
>      },
> +#if CONFIG_VAAPI_1
> +    {
> +        "New Intel iHD",
> +        "Intel iHD driver for Intel(R) Gen Graphics",
> +        AV_VAAPI_DRIVER_QUIRK_DYNAMIC_SURFACE_POOL,
> +    },
> +#endif
>  };
>  
>  static int vaapi_device_init(AVHWDeviceContext *hwdev)
> diff --git a/libavutil/hwcontext_vaapi.h b/libavutil/hwcontext_vaapi.h
> index 0b2e071cb3..07014fd526 100644
> --- a/libavutil/hwcontext_vaapi.h
> +++ b/libavutil/hwcontext_vaapi.h
> @@ -58,6 +58,12 @@ enum {
>       * and the results of the vaQuerySurfaceAttributes() call will be faked.
>       */
>      AV_VAAPI_DRIVER_QUIRK_SURFACE_ATTRIBUTES = (1 << 3),
> +
> +    /**
> +     * The driver (and the underlying HW) supports dynamic surface pool.
> +     * The vaCreateContext() call doesn't require a fixed surface-array.
> +     */
> +    AV_VAAPI_DRIVER_QUIRK_DYNAMIC_SURFACE_POOL = (1 << 4),
>  };
>  
>  /**

I will merge this patchset if there are no objections.

Thanks
Haihao

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/3] lavu/hwcontext_vaapi: Add a new quirk

[Mark Thompson]

On 28/03/2024 02:17, Xiang, Haihao wrote:
> From: Haihao Xiang <haihao.xiang@intel.com>
> 
> libva2 doesn't require a fixed surface-array any more, but some
> driver/hardware combinations which rely on this are still used. To
> reduce the impact to users, add a quirk for the driver/hardware
> combination which supports dynamic surface pool.
> 
> Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
> ---
>   libavutil/hwcontext_vaapi.c | 7 +++++++
>   libavutil/hwcontext_vaapi.h | 6 ++++++
>   2 files changed, 13 insertions(+)
> 
> diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c
> index 56d03aa4cd..dae5dd4a11 100644
> --- a/libavutil/hwcontext_vaapi.c
> +++ b/libavutil/hwcontext_vaapi.c
> @@ -390,6 +390,13 @@ static const struct {
>           "Splitted-Desktop Systems VDPAU backend for VA-API",
>           AV_VAAPI_DRIVER_QUIRK_SURFACE_ATTRIBUTES,
>       },
> +#if CONFIG_VAAPI_1
> +    {
> +        "New Intel iHD",
> +        "Intel iHD driver for Intel(R) Gen Graphics",
> +        AV_VAAPI_DRIVER_QUIRK_DYNAMIC_SURFACE_POOL,
> +    },
> +#endif
>   };
>   
>   static int vaapi_device_init(AVHWDeviceContext *hwdev)
> diff --git a/libavutil/hwcontext_vaapi.h b/libavutil/hwcontext_vaapi.h
> index 0b2e071cb3..07014fd526 100644
> --- a/libavutil/hwcontext_vaapi.h
> +++ b/libavutil/hwcontext_vaapi.h
> @@ -58,6 +58,12 @@ enum {
>        * and the results of the vaQuerySurfaceAttributes() call will be faked.
>        */
>       AV_VAAPI_DRIVER_QUIRK_SURFACE_ATTRIBUTES = (1 << 3),
> +
> +    /**
> +     * The driver (and the underlying HW) supports dynamic surface pool.
> +     * The vaCreateContext() call doesn't require a fixed surface-array.
> +     */
> +    AV_VAAPI_DRIVER_QUIRK_DYNAMIC_SURFACE_POOL = (1 << 4),
>   };
>   
>   /**

I do not think a vendor-specific quirk like this is a reasonable answer, but I can see that your company is invested in making sure that your current driver doesn't hit this problem.

Given that, I give up on arguing for trying to preserve compatibility here.  Let's just use dynamic pools unconditionally and see if anything breaks.

Is there any reason not to drop support for libva < 2.0 at the same time?  (Making CONFIG_VAAPI_1 always true.)  It is of similar age to C17, which we are intending to require soon as well.

Thanks,

- Mark
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/3] lavu/hwcontext_vaapi: Add a new quirk

[Xiang, Haihao]

On Wo, 2024-04-03 at 20:21 +0100, Mark Thompson wrote:
> On 28/03/2024 02:17, Xiang, Haihao wrote:
> > From: Haihao Xiang <haihao.xiang@intel.com>
> > 
> > libva2 doesn't require a fixed surface-array any more, but some
> > driver/hardware combinations which rely on this are still used. To
> > reduce the impact to users, add a quirk for the driver/hardware
> > combination which supports dynamic surface pool.
> > 
> > Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
> > ---
> >   libavutil/hwcontext_vaapi.c | 7 +++++++
> >   libavutil/hwcontext_vaapi.h | 6 ++++++
> >   2 files changed, 13 insertions(+)
> > 
> > diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c
> > index 56d03aa4cd..dae5dd4a11 100644
> > --- a/libavutil/hwcontext_vaapi.c
> > +++ b/libavutil/hwcontext_vaapi.c
> > @@ -390,6 +390,13 @@ static const struct {
> >           "Splitted-Desktop Systems VDPAU backend for VA-API",
> >           AV_VAAPI_DRIVER_QUIRK_SURFACE_ATTRIBUTES,
> >       },
> > +#if CONFIG_VAAPI_1
> > +    {
> > +        "New Intel iHD",
> > +        "Intel iHD driver for Intel(R) Gen Graphics",
> > +        AV_VAAPI_DRIVER_QUIRK_DYNAMIC_SURFACE_POOL,
> > +    },
> > +#endif
> >   };
> >   
> >   static int vaapi_device_init(AVHWDeviceContext *hwdev)
> > diff --git a/libavutil/hwcontext_vaapi.h b/libavutil/hwcontext_vaapi.h
> > index 0b2e071cb3..07014fd526 100644
> > --- a/libavutil/hwcontext_vaapi.h
> > +++ b/libavutil/hwcontext_vaapi.h
> > @@ -58,6 +58,12 @@ enum {
> >        * and the results of the vaQuerySurfaceAttributes() call will be
> > faked.
> >        */
> >       AV_VAAPI_DRIVER_QUIRK_SURFACE_ATTRIBUTES = (1 << 3),
> > +
> > +    /**
> > +     * The driver (and the underlying HW) supports dynamic surface pool.
> > +     * The vaCreateContext() call doesn't require a fixed surface-array.
> > +     */
> > +    AV_VAAPI_DRIVER_QUIRK_DYNAMIC_SURFACE_POOL = (1 << 4),
> >   };
> >   
> >   /**
> 
> I do not think a vendor-specific quirk like this is a reasonable answer, but I
> can see that your company is invested in making sure that your current driver
> doesn't hit this problem.
> 
> Given that, I give up on arguing for trying to preserve compatibility here. 
> Let's just use dynamic pools unconditionally and see if anything breaks.

Thanks, I'll update the patchset to use dynamic pools for all drivers when libva
>= 2.0. 

> 
> Is there any reason not to drop support for libva < 2.0 at the same time? 
> (Making CONFIG_VAAPI_1 always true.)  It is of similar age to C17, which we
> are intending to require soon as well.

We are considering to drop the support for libva < 2.0, but we can't be sure
whether user is still using libva < 2.0.

If there isn't any objection about dropping the support for libva < 2.0, We will
use a separate patch to drop the support.

BRs
Haihao

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".