Re: [FFmpeg-devel] [PATCH v2 11/12] lavc/vaapi_decode: use dynamic frame pool for output frames with libva2

["Xiang, Haihao" <haihao.xiang-at-intel.com@ffmpeg.org>]

On Wo, 2024-01-31 at 02:26 +0000, Xiang, Haihao wrote:
> On Di, 2024-01-30 at 19:07 +0000, Mark Thompson wrote:
> > On 30/01/2024 06:30, Xiang, Haihao wrote:
> > > On Ma, 2024-01-29 at 21:58 +0000, Mark Thompson wrote:
> > > > On 26/01/2024 07:25, Xiang, Haihao wrote:
> > > > > On Wo, 2023-12-20 at 15:10 +0800, Xiang, Haihao wrote:
> > > > > > From: Haihao Xiang <haihao.xiang@intel.com>
> > > > > > 
> > > > > > This allows a downstream element stores more frames from VAAPI
> > > > > > decoders and fixes error in get_buffer()
> > > > > > 
> > > > > > $ ffmpeg -hwaccel vaapi -hwaccel_output_format vaapi -i
> > > > > > input_100frames.mp4 \
> > > > > >    -vf reverse -an -f null -
> > > > > > ...
> > > > > > [h264 @ 0x557a075a1400] get_buffer() failed
> > > > > > [h264 @ 0x557a075a1400] thread_get_buffer() failed
> > > > > > [h264 @ 0x557a075a1400] decode_slice_header error
> > > > > > [h264 @ 0x557a075a1400] no frame!
> > > > > > 
> > > > > > Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
> > > > > > ---
> > > > > >    libavcodec/vaapi_decode.c | 36 ++++++++++++++++++++--------------
> > > > > > --
> > > > > >    1 file changed, 20 insertions(+), 16 deletions(-)
> > > > > > 
> > > > > > diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c
> > > > > > index ceac769c52..8cc29e96f9 100644
> > > > > > --- a/libavcodec/vaapi_decode.c
> > > > > > +++ b/libavcodec/vaapi_decode.c
> > > > > > @@ -601,22 +601,26 @@ static int
> > > > > > vaapi_decode_make_config(AVCodecContext
> > > > > > *avctx,
> > > > > >            if (err < 0)
> > > > > >                goto fail;
> > > > > >    
> > > > > > -        frames->initial_pool_size = 1;
> > > > > > -        // Add per-codec number of surfaces used for storing
> > > > > > reference
> > > > > > frames.
> > > > > > -        switch (avctx->codec_id) {
> > > > > > -        case AV_CODEC_ID_H264:
> > > > > > -        case AV_CODEC_ID_HEVC:
> > > > > > -        case AV_CODEC_ID_AV1:
> > > > > > -            frames->initial_pool_size += 16;
> > > > > > -            break;
> > > > > > -        case AV_CODEC_ID_VP9:
> > > > > > -            frames->initial_pool_size += 8;
> > > > > > -            break;
> > > > > > -        case AV_CODEC_ID_VP8:
> > > > > > -            frames->initial_pool_size += 3;
> > > > > > -            break;
> > > > > > -        default:
> > > > > > -            frames->initial_pool_size += 2;
> > > > > > +        if (CONFIG_VAAPI_1)
> > > > > > +            frames->initial_pool_size = 0;
> > > > > > +        else {
> > > > > > +            frames->initial_pool_size = 1;
> > > > > > +            // Add per-codec number of surfaces used for storing
> > > > > > reference
> > > > > > frames.
> > > > > > +            switch (avctx->codec_id) {
> > > > > > +            case AV_CODEC_ID_H264:
> > > > > > +            case AV_CODEC_ID_HEVC:
> > > > > > +            case AV_CODEC_ID_AV1:
> > > > > > +                frames->initial_pool_size += 16;
> > > > > > +                break;
> > > > > > +            case AV_CODEC_ID_VP9:
> > > > > > +                frames->initial_pool_size += 8;
> > > > > > +                break;
> > > > > > +            case AV_CODEC_ID_VP8:
> > > > > > +                frames->initial_pool_size += 3;
> > > > > > +                break;
> > > > > > +            default:
> > > > > > +                frames->initial_pool_size += 2;
> > > > > > +            }
> > > > > >            }
> > > > > >        }
> > > > > >    
> > > > > 
> > > > > Hi Mark,
> > > > > 
> > > > > Do you have any comment about dynamic frame pool used in vaapi ?
> > > > 
> > > > Are we completely sure that there are no driver/hardware combinations
> > > > which
> > > > rely on this still used?
> > > 
> > > I tested this patch with i965, iHD and radeonsi drivers on Linux and
> > > vaon12
> > > driver on Windows. But honestly I am not sure whether there is a driver
> > > which
> > > works with fixed frame pool only.
> > 
> > How does the vaon12 driver work with this given that some D3D12 devices
> > require a fixed array texture?
> 
> Honestly I don't know. I don't have such HWs for testing. 
> 
> > 
> > Note that the interesting test here is not the most recent version of any of
> > these things.  Rather, it is the older versions which exist in a
> > distribution
> > configuration which we still want to support, for example Ubuntu 20.04.
> 
> This patch is based on libva2, FFmpeg still works with libva. For older
> versions
> , user may use libva.
> 
> > 
> > I'm also unclear to what degree this might depend on the hardware being
> > used. 
> > Certainly in D3D12 whether the fixed array texture is required depends on
> > the
> > actual hardware support.
> 
> How about add a quirk for workable driver(s) only ? We won't be concerned by
> other drivers. 
> 
> diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c
> index e6f45f8fde..aecdd55728 100644
> --- a/libavutil/hwcontext_vaapi.c
> +++ b/libavutil/hwcontext_vaapi.c
> @@ -374,7 +374,7 @@ static const struct {
>      {
>          "Intel iHD",
>          "ubit",
> -        AV_VAAPI_DRIVER_QUIRK_ATTRIB_MEMTYPE,
> +        AV_VAAPI_DRIVER_QUIRK_ATTRIB_MEMTYPE |
> AV_VAAPI_DRIVER_QUIRK_DYNAMIC_SURFACE_POOL,
>      },
>      {
>          "VDPAU wrapper",
> diff --git a/libavutil/hwcontext_vaapi.h b/libavutil/hwcontext_vaapi.h
> index 259c6f5dbd..c4d2709224 100644
> --- a/libavutil/hwcontext_vaapi.h
> +++ b/libavutil/hwcontext_vaapi.h
> @@ -60,6 +60,12 @@ enum {
>       * and the results of the vaQuerySurfaceAttributes() call will be faked.
>       */
>      AV_VAAPI_DRIVER_QUIRK_SURFACE_ATTRIBUTES = (1 << 3),
> +
> +    /**
> +     * The driver (and the underlying HW) supports dynamic surface pool.
> +     * The vaCreateContext() call doesn't require a fixed array surfaces.
> +     */
> +    AV_VAAPI_DRIVER_QUIRK_DYNAMIC_SURFACE_POOL = (1 << 4),
>  };
> 
>  /**

Hi Mark,

Is there any comment or concern about adding a quirk for workable drivers ? We
may use a dynamic frame pool in vaapi decoders and filters for workable drivers
only.

Note since commit e0da916b, a command below doesn't work with the current fixed
frame pool used in vaapi filters. 

$ ffmpeg -hwaccel_output_format vaapi -hwaccel vaapi -i input.mp4 -vf
'scale_vaapi=w=720:h=480' -c:v hevc_vaapi -f null -
[...]
[vf#0:0 @ 0x562847b01050] Error while filtering: Cannot allocate memory
[vf#0:0 @ 0x562847b01050] Task finished with error code: -12 (Cannot allocate
memory)
[vf#0:0 @ 0x562847b01050] Terminating thread with return code -12 (Cannot
allocate memory)
[...]

Thanks
Haihao


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".