Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
 help / color / mirror / Atom feed
* [FFmpeg-devel] [PATCH V2 1/3] libavcodec/vaapi_encode: Add new API adaption to vaapi_encode
@ 2022-01-05  2:48 Wenbin Chen
  2022-01-05  2:48 ` [FFmpeg-devel] [PATCH V2 2/3] libavcodec/vaapi_encode: Change the way to call async to increase performance Wenbin Chen
  2022-01-05  2:48 ` [FFmpeg-devel] [PATCH V2 3/3] libavcodec/vaapi_encode: Add async_depth to vaapi_encoder " Wenbin Chen
  0 siblings, 2 replies; 8+ messages in thread
From: Wenbin Chen @ 2022-01-05  2:48 UTC (permalink / raw)
  To: ffmpeg-devel

Add vaSyncBuffer to VAAPI encoder. Old version API vaSyncSurface wait
surface to complete. When surface is used for multiple operation, it
waits all operations to finish. vaSyncBuffer only wait one channel to
finish.

Add wait param to vaapi_encode_wait() to prepare for the async_depth
option. "wait=1" means wait until operation ready. "wait=0" means
query operation's status. If it is ready return 0, if it is still
in progress return EAGAIN.

Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
---
 libavcodec/vaapi_encode.c | 47 +++++++++++++++++++++++++++++++++------
 1 file changed, 40 insertions(+), 7 deletions(-)

diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
index 3bf379b1a0..b87b58a42b 100644
--- a/libavcodec/vaapi_encode.c
+++ b/libavcodec/vaapi_encode.c
@@ -134,7 +134,8 @@ static int vaapi_encode_make_misc_param_buffer(AVCodecContext *avctx,
 }
 
 static int vaapi_encode_wait(AVCodecContext *avctx,
-                             VAAPIEncodePicture *pic)
+                             VAAPIEncodePicture *pic,
+                             uint8_t wait)
 {
     VAAPIEncodeContext *ctx = avctx->priv_data;
     VAStatus vas;
@@ -150,11 +151,43 @@ static int vaapi_encode_wait(AVCodecContext *avctx,
            "(input surface %#x).\n", pic->display_order,
            pic->encode_order, pic->input_surface);
 
-    vas = vaSyncSurface(ctx->hwctx->display, pic->input_surface);
-    if (vas != VA_STATUS_SUCCESS) {
-        av_log(avctx, AV_LOG_ERROR, "Failed to sync to picture completion: "
-               "%d (%s).\n", vas, vaErrorStr(vas));
+#if VA_CHECK_VERSION(1, 9, 0)
+    // Try vaSyncBuffer.
+    vas = vaSyncBuffer(ctx->hwctx->display,
+                       pic->output_buffer,
+                       wait ? VA_TIMEOUT_INFINITE : 0);
+    if (vas == VA_STATUS_ERROR_TIMEDOUT) {
+        return AVERROR(EAGAIN);
+    } else if (vas != VA_STATUS_SUCCESS && vas != VA_STATUS_ERROR_UNIMPLEMENTED) {
+        av_log(avctx, AV_LOG_ERROR, "Failed to sync to output buffer completion: "
+                "%d (%s).\n", vas, vaErrorStr(vas));
         return AVERROR(EIO);
+    } else if (vas == VA_STATUS_ERROR_UNIMPLEMENTED)
+    // If vaSyncBuffer is not implemented, try old version API.
+#endif
+    {
+        if (!wait) {
+            VASurfaceStatus surface_status;
+            vas = vaQuerySurfaceStatus(ctx->hwctx->display,
+                                    pic->input_surface,
+                                    &surface_status);
+            if (vas == VA_STATUS_SUCCESS &&
+                surface_status != VASurfaceReady &&
+                surface_status != VASurfaceSkipped) {
+                return AVERROR(EAGAIN);
+            } else if (vas != VA_STATUS_SUCCESS) {
+                av_log(avctx, AV_LOG_ERROR, "Failed to query surface status: "
+                    "%d (%s).\n", vas, vaErrorStr(vas));
+                return AVERROR(EIO);
+            }
+        } else {
+            vas = vaSyncSurface(ctx->hwctx->display, pic->input_surface);
+            if (vas != VA_STATUS_SUCCESS) {
+                av_log(avctx, AV_LOG_ERROR, "Failed to sync to picture completion: "
+                    "%d (%s).\n", vas, vaErrorStr(vas));
+                return AVERROR(EIO);
+            }
+        }
     }
 
     // Input is definitely finished with now.
@@ -633,7 +666,7 @@ static int vaapi_encode_output(AVCodecContext *avctx,
     uint8_t *ptr;
     int err;
 
-    err = vaapi_encode_wait(avctx, pic);
+    err = vaapi_encode_wait(avctx, pic, 1);
     if (err < 0)
         return err;
 
@@ -695,7 +728,7 @@ fail:
 static int vaapi_encode_discard(AVCodecContext *avctx,
                                 VAAPIEncodePicture *pic)
 {
-    vaapi_encode_wait(avctx, pic);
+    vaapi_encode_wait(avctx, pic, 1);
 
     if (pic->output_buffer_ref) {
         av_log(avctx, AV_LOG_DEBUG, "Discard output for pic "
-- 
2.25.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 8+ messages in thread
* [FFmpeg-devel] [PATCH V2 2/3] libavcodec/vaapi_encode: Change the way to call async to increase performance
  2022-01-05  2:48 [FFmpeg-devel] [PATCH V2 1/3] libavcodec/vaapi_encode: Add new API adaption to vaapi_encode Wenbin Chen
@ 2022-01-05  2:48 ` Wenbin Chen
  2022-01-05  5:45   ` Andreas Rheinhardt
  2022-02-07  4:07   ` Xiang, Haihao
  2022-01-05  2:48 ` [FFmpeg-devel] [PATCH V2 3/3] libavcodec/vaapi_encode: Add async_depth to vaapi_encoder " Wenbin Chen
  1 sibling, 2 replies; 8+ messages in thread
From: Wenbin Chen @ 2022-01-05  2:48 UTC (permalink / raw)
  To: ffmpeg-devel

Fix: #7706. After commit 5fdcf85bbffe7451c2, vaapi encoder's performance
decrease. The reason is that vaRenderPicture() and vaSyncBuffer() are
called at the same time (vaRenderPicture() always followed by a
vaSyncBuffer()). When we encode stream with B frames, we need buffer to
reorder frames, so we can send serveral frames to HW at once to increase
performance. Now I changed them to be called in a asynchronous way, which
will make better use of hardware. 1080p transcoding increases about 17%
fps on my environment.

This change fits vaSyncBuffer(), so if driver does not support
vaSyncBuffer, it will keep previous operation.

Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
---
 libavcodec/vaapi_encode.c | 64 ++++++++++++++++++++++++++++++++-------
 libavcodec/vaapi_encode.h |  5 +++
 2 files changed, 58 insertions(+), 11 deletions(-)

diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
index b87b58a42b..9a3b3ba4ad 100644
--- a/libavcodec/vaapi_encode.c
+++ b/libavcodec/vaapi_encode.c
@@ -984,8 +984,10 @@ static int vaapi_encode_pick_next(AVCodecContext *avctx,
     if (!pic && ctx->end_of_stream) {
         --b_counter;
         pic = ctx->pic_end;
-        if (pic->encode_issued)
+        if (pic->encode_complete)
             return AVERROR_EOF;
+        else if (pic->encode_issued)
+            return AVERROR(EAGAIN);
     }
 
     if (!pic) {
@@ -1210,18 +1212,45 @@ int ff_vaapi_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt)
             return AVERROR(EAGAIN);
     }
 
-    pic = NULL;
-    err = vaapi_encode_pick_next(avctx, &pic);
-    if (err < 0)
-        return err;
-    av_assert0(pic);
+#if VA_CHECK_VERSION(1, 9, 0)
+    if (ctx->has_sync_buffer_func) {
+        while (av_fifo_size(ctx->encode_fifo) <=
+               MAX_PICTURE_REFERENCES * sizeof(VAAPIEncodePicture *)) {
+            pic = NULL;
+            err = vaapi_encode_pick_next(avctx, &pic);
+            if (err < 0)
+                break;
+
+            av_assert0(pic);
+            pic->encode_order = ctx->encode_order +
+                (av_fifo_size(ctx->encode_fifo) / sizeof(VAAPIEncodePicture *));
+            err = vaapi_encode_issue(avctx, pic);
+            if (err < 0) {
+                av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err);
+                return err;
+            }
+            av_fifo_generic_write(ctx->encode_fifo, &pic, sizeof(pic), NULL);
+        }
+        if (!av_fifo_size(ctx->encode_fifo))
+            return err;
+        av_fifo_generic_read(ctx->encode_fifo, &pic, sizeof(pic), NULL);
+        ctx->encode_order = pic->encode_order + 1;
+    } else
+#endif
+    {
+        pic = NULL;
+        err = vaapi_encode_pick_next(avctx, &pic);
+        if (err < 0)
+            return err;
+        av_assert0(pic);
 
-    pic->encode_order = ctx->encode_order++;
+        pic->encode_order = ctx->encode_order++;
 
-    err = vaapi_encode_issue(avctx, pic);
-    if (err < 0) {
-        av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err);
-        return err;
+        err = vaapi_encode_issue(avctx, pic);
+        if (err < 0) {
+            av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err);
+            return err;
+        }
     }
 
     err = vaapi_encode_output(avctx, pic, pkt);
@@ -2555,6 +2584,18 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx)
         }
     }
 
+#if VA_CHECK_VERSION(1, 9, 0)
+    //check vaSyncBuffer function
+    vas = vaSyncBuffer(ctx->hwctx->display, 0, 0);
+    if (vas != VA_STATUS_ERROR_UNIMPLEMENTED) {
+        ctx->has_sync_buffer_func = 1;
+        ctx->encode_fifo = av_fifo_alloc((MAX_PICTURE_REFERENCES + 1) *
+                                        sizeof(VAAPIEncodePicture *));
+        if (!ctx->encode_fifo)
+            return AVERROR(ENOMEM);
+    }
+#endif
+
     return 0;
 
 fail:
@@ -2592,6 +2633,7 @@ av_cold int ff_vaapi_encode_close(AVCodecContext *avctx)
 
     av_freep(&ctx->codec_sequence_params);
     av_freep(&ctx->codec_picture_params);
+    av_fifo_freep(&ctx->encode_fifo);
 
     av_buffer_unref(&ctx->recon_frames_ref);
     av_buffer_unref(&ctx->input_frames_ref);
diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h
index b41604a883..560a1c42a9 100644
--- a/libavcodec/vaapi_encode.h
+++ b/libavcodec/vaapi_encode.h
@@ -29,6 +29,7 @@
 
 #include "libavutil/hwcontext.h"
 #include "libavutil/hwcontext_vaapi.h"
+#include "libavutil/fifo.h"
 
 #include "avcodec.h"
 #include "hwconfig.h"
@@ -345,6 +346,10 @@ typedef struct VAAPIEncodeContext {
     int             roi_warned;
 
     AVFrame         *frame;
+    //Store buffered pic
+    AVFifoBuffer *encode_fifo;
+    //Whether the driver support vaSyncBuffer
+    int has_sync_buffer_func;
 } VAAPIEncodeContext;
 
 enum {
-- 
2.25.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 8+ messages in thread
* [FFmpeg-devel] [PATCH V2 3/3] libavcodec/vaapi_encode: Add async_depth to vaapi_encoder to increase performance
  2022-01-05  2:48 [FFmpeg-devel] [PATCH V2 1/3] libavcodec/vaapi_encode: Add new API adaption to vaapi_encode Wenbin Chen
  2022-01-05  2:48 ` [FFmpeg-devel] [PATCH V2 2/3] libavcodec/vaapi_encode: Change the way to call async to increase performance Wenbin Chen
@ 2022-01-05  2:48 ` Wenbin Chen
  1 sibling, 0 replies; 8+ messages in thread
From: Wenbin Chen @ 2022-01-05  2:48 UTC (permalink / raw)
  To: ffmpeg-devel

Add async_depth to increase encoder's performance. Reuse encode_fifo as
async buffer. Encoder puts all reordered frame to HW and then check
fifo size. If fifo < async_depth and the top frame is not ready, it will
return AVERROR(EAGAIN) to require more frames.

1080p transcoding (no B frames) with -async_depth=4 can increase 20%
performance on my environment.
The async increases performance but also introduces frame delay.

Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
---
 libavcodec/vaapi_encode.c | 19 ++++++++++++++-----
 libavcodec/vaapi_encode.h | 12 ++++++++++--
 2 files changed, 24 insertions(+), 7 deletions(-)

diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
index 9a3b3ba4ad..f9ffca0475 100644
--- a/libavcodec/vaapi_encode.c
+++ b/libavcodec/vaapi_encode.c
@@ -1158,7 +1158,8 @@ static int vaapi_encode_send_frame(AVCodecContext *avctx, AVFrame *frame)
         if (ctx->input_order == ctx->decode_delay)
             ctx->dts_pts_diff = pic->pts - ctx->first_pts;
         if (ctx->output_delay > 0)
-            ctx->ts_ring[ctx->input_order % (3 * ctx->output_delay)] = pic->pts;
+            ctx->ts_ring[ctx->input_order %
+                        (3 * ctx->output_delay + ctx->async_depth)] = pic->pts;
 
         pic->display_order = ctx->input_order;
         ++ctx->input_order;
@@ -1214,8 +1215,8 @@ int ff_vaapi_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt)
 
 #if VA_CHECK_VERSION(1, 9, 0)
     if (ctx->has_sync_buffer_func) {
-        while (av_fifo_size(ctx->encode_fifo) <=
-               MAX_PICTURE_REFERENCES * sizeof(VAAPIEncodePicture *)) {
+        while (av_fifo_size(ctx->encode_fifo) <
+                MAX_ASYNC_DEPTH * sizeof(VAAPIEncodePicture *)) {
             pic = NULL;
             err = vaapi_encode_pick_next(avctx, &pic);
             if (err < 0)
@@ -1233,6 +1234,14 @@ int ff_vaapi_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt)
         }
         if (!av_fifo_size(ctx->encode_fifo))
             return err;
+        if (av_fifo_size(ctx->encode_fifo) <
+            ctx->async_depth * sizeof(VAAPIEncodePicture *) &&
+            !ctx->end_of_stream) {
+            av_fifo_generic_peek(ctx->encode_fifo, &pic, sizeof(pic), NULL);
+            err = vaapi_encode_wait(avctx, pic, 0);
+            if (err < 0)
+                return err;
+        }
         av_fifo_generic_read(ctx->encode_fifo, &pic, sizeof(pic), NULL);
         ctx->encode_order = pic->encode_order + 1;
     } else
@@ -1268,7 +1277,7 @@ int ff_vaapi_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt)
             pkt->dts = ctx->ts_ring[pic->encode_order] - ctx->dts_pts_diff;
     } else {
         pkt->dts = ctx->ts_ring[(pic->encode_order - ctx->decode_delay) %
-                                (3 * ctx->output_delay)];
+                                (3 * ctx->output_delay + ctx->async_depth)];
     }
     av_log(avctx, AV_LOG_DEBUG, "Output packet: pts %"PRId64" dts %"PRId64".\n",
            pkt->pts, pkt->dts);
@@ -2589,7 +2598,7 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx)
     vas = vaSyncBuffer(ctx->hwctx->display, 0, 0);
     if (vas != VA_STATUS_ERROR_UNIMPLEMENTED) {
         ctx->has_sync_buffer_func = 1;
-        ctx->encode_fifo = av_fifo_alloc((MAX_PICTURE_REFERENCES + 1) *
+        ctx->encode_fifo = av_fifo_alloc(MAX_ASYNC_DEPTH *
                                         sizeof(VAAPIEncodePicture *));
         if (!ctx->encode_fifo)
             return AVERROR(ENOMEM);
diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h
index 560a1c42a9..1a5824e702 100644
--- a/libavcodec/vaapi_encode.h
+++ b/libavcodec/vaapi_encode.h
@@ -48,6 +48,7 @@ enum {
     MAX_TILE_ROWS          = 22,
     // A.4.1: table A.6 allows at most 20 tile columns for any level.
     MAX_TILE_COLS          = 20,
+    MAX_ASYNC_DEPTH        = 64,
 };
 
 extern const AVCodecHWConfigInternal *const ff_vaapi_encode_hw_configs[];
@@ -298,7 +299,8 @@ typedef struct VAAPIEncodeContext {
     // Timestamp handling.
     int64_t         first_pts;
     int64_t         dts_pts_diff;
-    int64_t         ts_ring[MAX_REORDER_DELAY * 3];
+    int64_t         ts_ring[MAX_REORDER_DELAY * 3 +
+                            MAX_ASYNC_DEPTH];
 
     // Slice structure.
     int slice_block_rows;
@@ -350,6 +352,8 @@ typedef struct VAAPIEncodeContext {
     AVFifoBuffer *encode_fifo;
     //Whether the driver support vaSyncBuffer
     int has_sync_buffer_func;
+    //Max number of frame buffered in encoder.
+    int async_depth;
 } VAAPIEncodeContext;
 
 enum {
@@ -460,7 +464,11 @@ int ff_vaapi_encode_close(AVCodecContext *avctx);
     { "b_depth", \
       "Maximum B-frame reference depth", \
       OFFSET(common.desired_b_depth), AV_OPT_TYPE_INT, \
-      { .i64 = 1 }, 1, INT_MAX, FLAGS }
+      { .i64 = 1 }, 1, INT_MAX, FLAGS }, \
+    { "async_depth", "Maximum processing parallelism. " \
+      "Increase this to improve single channel performance", \
+      OFFSET(common.async_depth), AV_OPT_TYPE_INT, \
+      { .i64 = 4 }, 0, MAX_ASYNC_DEPTH, FLAGS }
 
 #define VAAPI_ENCODE_RC_MODE(name, desc) \
     { #name, desc, 0, AV_OPT_TYPE_CONST, { .i64 = RC_MODE_ ## name }, \
-- 
2.25.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 8+ messages in thread
* Re: [FFmpeg-devel] [PATCH V2 2/3] libavcodec/vaapi_encode: Change the way to call async to increase performance
  2022-01-05  2:48 ` [FFmpeg-devel] [PATCH V2 2/3] libavcodec/vaapi_encode: Change the way to call async to increase performance Wenbin Chen
@ 2022-01-05  5:45   ` Andreas Rheinhardt
  2022-01-05  7:59     ` Chen, Wenbin
  2022-02-07  4:07   ` Xiang, Haihao
  1 sibling, 1 reply; 8+ messages in thread
From: Andreas Rheinhardt @ 2022-01-05  5:45 UTC (permalink / raw)
  To: ffmpeg-devel

Wenbin Chen:
> Fix: #7706. After commit 5fdcf85bbffe7451c2, vaapi encoder's performance
> decrease. The reason is that vaRenderPicture() and vaSyncBuffer() are
> called at the same time (vaRenderPicture() always followed by a
> vaSyncBuffer()). When we encode stream with B frames, we need buffer to
> reorder frames, so we can send serveral frames to HW at once to increase
> performance. Now I changed them to be called in a asynchronous way, which
> will make better use of hardware. 1080p transcoding increases about 17%
> fps on my environment.
> 
> This change fits vaSyncBuffer(), so if driver does not support
> vaSyncBuffer, it will keep previous operation.
> 
> Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
> ---
>  libavcodec/vaapi_encode.c | 64 ++++++++++++++++++++++++++++++++-------
>  libavcodec/vaapi_encode.h |  5 +++
>  2 files changed, 58 insertions(+), 11 deletions(-)
> 
> diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
> index b87b58a42b..9a3b3ba4ad 100644
> --- a/libavcodec/vaapi_encode.c
> +++ b/libavcodec/vaapi_encode.c
> @@ -984,8 +984,10 @@ static int vaapi_encode_pick_next(AVCodecContext *avctx,
>      if (!pic && ctx->end_of_stream) {
>          --b_counter;
>          pic = ctx->pic_end;
> -        if (pic->encode_issued)
> +        if (pic->encode_complete)
>              return AVERROR_EOF;
> +        else if (pic->encode_issued)
> +            return AVERROR(EAGAIN);
>      }
>  
>      if (!pic) {
> @@ -1210,18 +1212,45 @@ int ff_vaapi_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt)
>              return AVERROR(EAGAIN);
>      }
>  
> -    pic = NULL;
> -    err = vaapi_encode_pick_next(avctx, &pic);
> -    if (err < 0)
> -        return err;
> -    av_assert0(pic);
> +#if VA_CHECK_VERSION(1, 9, 0)
> +    if (ctx->has_sync_buffer_func) {
> +        while (av_fifo_size(ctx->encode_fifo) <=
> +               MAX_PICTURE_REFERENCES * sizeof(VAAPIEncodePicture *)) {
> +            pic = NULL;
> +            err = vaapi_encode_pick_next(avctx, &pic);
> +            if (err < 0)
> +                break;
> +
> +            av_assert0(pic);
> +            pic->encode_order = ctx->encode_order +
> +                (av_fifo_size(ctx->encode_fifo) / sizeof(VAAPIEncodePicture *));
> +            err = vaapi_encode_issue(avctx, pic);
> +            if (err < 0) {
> +                av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err);
> +                return err;
> +            }
> +            av_fifo_generic_write(ctx->encode_fifo, &pic, sizeof(pic), NULL);
> +        }
> +        if (!av_fifo_size(ctx->encode_fifo))
> +            return err;
> +        av_fifo_generic_read(ctx->encode_fifo, &pic, sizeof(pic), NULL);
> +        ctx->encode_order = pic->encode_order + 1;
> +    } else
> +#endif
> +    {
> +        pic = NULL;
> +        err = vaapi_encode_pick_next(avctx, &pic);
> +        if (err < 0)
> +            return err;
> +        av_assert0(pic);
>  
> -    pic->encode_order = ctx->encode_order++;
> +        pic->encode_order = ctx->encode_order++;
>  
> -    err = vaapi_encode_issue(avctx, pic);
> -    if (err < 0) {
> -        av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err);
> -        return err;
> +        err = vaapi_encode_issue(avctx, pic);
> +        if (err < 0) {
> +            av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err);
> +            return err;
> +        }
>      }
>  
>      err = vaapi_encode_output(avctx, pic, pkt);
> @@ -2555,6 +2584,18 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx)
>          }
>      }
>  
> +#if VA_CHECK_VERSION(1, 9, 0)
> +    //check vaSyncBuffer function
> +    vas = vaSyncBuffer(ctx->hwctx->display, 0, 0);
> +    if (vas != VA_STATUS_ERROR_UNIMPLEMENTED) {
> +        ctx->has_sync_buffer_func = 1;
> +        ctx->encode_fifo = av_fifo_alloc((MAX_PICTURE_REFERENCES + 1) *
> +                                        sizeof(VAAPIEncodePicture *));
> +        if (!ctx->encode_fifo)
> +            return AVERROR(ENOMEM);
> +    }
> +#endif
> +
>      return 0;
>  
>  fail:
> @@ -2592,6 +2633,7 @@ av_cold int ff_vaapi_encode_close(AVCodecContext *avctx)
>  
>      av_freep(&ctx->codec_sequence_params);
>      av_freep(&ctx->codec_picture_params);
> +    av_fifo_freep(&ctx->encode_fifo);

Is it guaranteed that the fifo is empty at this point? I don't think so.

>  
>      av_buffer_unref(&ctx->recon_frames_ref);
>      av_buffer_unref(&ctx->input_frames_ref);
> diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h
> index b41604a883..560a1c42a9 100644
> --- a/libavcodec/vaapi_encode.h
> +++ b/libavcodec/vaapi_encode.h
> @@ -29,6 +29,7 @@
>  
>  #include "libavutil/hwcontext.h"
>  #include "libavutil/hwcontext_vaapi.h"
> +#include "libavutil/fifo.h"
>  
>  #include "avcodec.h"
>  #include "hwconfig.h"
> @@ -345,6 +346,10 @@ typedef struct VAAPIEncodeContext {
>      int             roi_warned;
>  
>      AVFrame         *frame;
> +    //Store buffered pic
> +    AVFifoBuffer *encode_fifo;
> +    //Whether the driver support vaSyncBuffer
> +    int has_sync_buffer_func;
>  } VAAPIEncodeContext;
>  
>  enum {
> 

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 8+ messages in thread
* Re: [FFmpeg-devel] [PATCH V2 2/3] libavcodec/vaapi_encode: Change the way to call async to increase performance
  2022-01-05  5:45   ` Andreas Rheinhardt
@ 2022-01-05  7:59     ` Chen, Wenbin
  2022-01-06  8:41       ` Andreas Rheinhardt
  0 siblings, 1 reply; 8+ messages in thread
From: Chen, Wenbin @ 2022-01-05  7:59 UTC (permalink / raw)
  To: FFmpeg development discussions and patches

> Wenbin Chen:
> > Fix: #7706. After commit 5fdcf85bbffe7451c2, vaapi encoder's performance
> > decrease. The reason is that vaRenderPicture() and vaSyncBuffer() are
> > called at the same time (vaRenderPicture() always followed by a
> > vaSyncBuffer()). When we encode stream with B frames, we need buffer to
> > reorder frames, so we can send serveral frames to HW at once to increase
> > performance. Now I changed them to be called in a asynchronous way,
> which
> > will make better use of hardware. 1080p transcoding increases about 17%
> > fps on my environment.
> >
> > This change fits vaSyncBuffer(), so if driver does not support
> > vaSyncBuffer, it will keep previous operation.
> >
> > Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
> > ---
> >  libavcodec/vaapi_encode.c | 64 ++++++++++++++++++++++++++++++++-----
> --
> >  libavcodec/vaapi_encode.h |  5 +++
> >  2 files changed, 58 insertions(+), 11 deletions(-)
> >
> > diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
> > index b87b58a42b..9a3b3ba4ad 100644
> > --- a/libavcodec/vaapi_encode.c
> > +++ b/libavcodec/vaapi_encode.c
> > @@ -984,8 +984,10 @@ static int
> vaapi_encode_pick_next(AVCodecContext *avctx,
> >      if (!pic && ctx->end_of_stream) {
> >          --b_counter;
> >          pic = ctx->pic_end;
> > -        if (pic->encode_issued)
> > +        if (pic->encode_complete)
> >              return AVERROR_EOF;
> > +        else if (pic->encode_issued)
> > +            return AVERROR(EAGAIN);
> >      }
> >
> >      if (!pic) {
> > @@ -1210,18 +1212,45 @@ int
> ff_vaapi_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt)
> >              return AVERROR(EAGAIN);
> >      }
> >
> > -    pic = NULL;
> > -    err = vaapi_encode_pick_next(avctx, &pic);
> > -    if (err < 0)
> > -        return err;
> > -    av_assert0(pic);
> > +#if VA_CHECK_VERSION(1, 9, 0)
> > +    if (ctx->has_sync_buffer_func) {
> > +        while (av_fifo_size(ctx->encode_fifo) <=
> > +               MAX_PICTURE_REFERENCES * sizeof(VAAPIEncodePicture *)) {
> > +            pic = NULL;
> > +            err = vaapi_encode_pick_next(avctx, &pic);
> > +            if (err < 0)
> > +                break;
> > +
> > +            av_assert0(pic);
> > +            pic->encode_order = ctx->encode_order +
> > +                (av_fifo_size(ctx->encode_fifo) / sizeof(VAAPIEncodePicture *));
> > +            err = vaapi_encode_issue(avctx, pic);
> > +            if (err < 0) {
> > +                av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err);
> > +                return err;
> > +            }
> > +            av_fifo_generic_write(ctx->encode_fifo, &pic, sizeof(pic), NULL);
> > +        }
> > +        if (!av_fifo_size(ctx->encode_fifo))
> > +            return err;
> > +        av_fifo_generic_read(ctx->encode_fifo, &pic, sizeof(pic), NULL);
> > +        ctx->encode_order = pic->encode_order + 1;
> > +    } else
> > +#endif
> > +    {
> > +        pic = NULL;
> > +        err = vaapi_encode_pick_next(avctx, &pic);
> > +        if (err < 0)
> > +            return err;
> > +        av_assert0(pic);
> >
> > -    pic->encode_order = ctx->encode_order++;
> > +        pic->encode_order = ctx->encode_order++;
> >
> > -    err = vaapi_encode_issue(avctx, pic);
> > -    if (err < 0) {
> > -        av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err);
> > -        return err;
> > +        err = vaapi_encode_issue(avctx, pic);
> > +        if (err < 0) {
> > +            av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err);
> > +            return err;
> > +        }
> >      }
> >
> >      err = vaapi_encode_output(avctx, pic, pkt);
> > @@ -2555,6 +2584,18 @@ av_cold int
> ff_vaapi_encode_init(AVCodecContext *avctx)
> >          }
> >      }
> >
> > +#if VA_CHECK_VERSION(1, 9, 0)
> > +    //check vaSyncBuffer function
> > +    vas = vaSyncBuffer(ctx->hwctx->display, 0, 0);
> > +    if (vas != VA_STATUS_ERROR_UNIMPLEMENTED) {
> > +        ctx->has_sync_buffer_func = 1;
> > +        ctx->encode_fifo = av_fifo_alloc((MAX_PICTURE_REFERENCES + 1) *
> > +                                        sizeof(VAAPIEncodePicture *));
> > +        if (!ctx->encode_fifo)
> > +            return AVERROR(ENOMEM);
> > +    }
> > +#endif
> > +
> >      return 0;
> >
> >  fail:
> > @@ -2592,6 +2633,7 @@ av_cold int
> ff_vaapi_encode_close(AVCodecContext *avctx)
> >
> >      av_freep(&ctx->codec_sequence_params);
> >      av_freep(&ctx->codec_picture_params);
> > +    av_fifo_freep(&ctx->encode_fifo);
> 
> Is it guaranteed that the fifo is empty at this point? I don't think so.

I don't check the fifo size, because in ff_vaapi_encode_close() all pics
are already freed and encode_fifo only buffer pic.
```
    for (pic = ctx->pic_start; pic; pic = next) {
        next = pic->next;
        vaapi_encode_free(avctx, pic);
    }
```

> 
> >
> >      av_buffer_unref(&ctx->recon_frames_ref);
> >      av_buffer_unref(&ctx->input_frames_ref);
> > diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h
> > index b41604a883..560a1c42a9 100644
> > --- a/libavcodec/vaapi_encode.h
> > +++ b/libavcodec/vaapi_encode.h
> > @@ -29,6 +29,7 @@
> >
> >  #include "libavutil/hwcontext.h"
> >  #include "libavutil/hwcontext_vaapi.h"
> > +#include "libavutil/fifo.h"
> >
> >  #include "avcodec.h"
> >  #include "hwconfig.h"
> > @@ -345,6 +346,10 @@ typedef struct VAAPIEncodeContext {
> >      int             roi_warned;
> >
> >      AVFrame         *frame;
> > +    //Store buffered pic
> > +    AVFifoBuffer *encode_fifo;
> > +    //Whether the driver support vaSyncBuffer
> > +    int has_sync_buffer_func;
> >  } VAAPIEncodeContext;
> >
> >  enum {
> >
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 8+ messages in thread
* Re: [FFmpeg-devel] [PATCH V2 2/3] libavcodec/vaapi_encode: Change the way to call async to increase performance
  2022-01-05  7:59     ` Chen, Wenbin
@ 2022-01-06  8:41       ` Andreas Rheinhardt
  0 siblings, 0 replies; 8+ messages in thread
From: Andreas Rheinhardt @ 2022-01-06  8:41 UTC (permalink / raw)
  To: ffmpeg-devel

Chen, Wenbin:
>> Wenbin Chen:
>>> Fix: #7706. After commit 5fdcf85bbffe7451c2, vaapi encoder's performance
>>> decrease. The reason is that vaRenderPicture() and vaSyncBuffer() are
>>> called at the same time (vaRenderPicture() always followed by a
>>> vaSyncBuffer()). When we encode stream with B frames, we need buffer to
>>> reorder frames, so we can send serveral frames to HW at once to increase
>>> performance. Now I changed them to be called in a asynchronous way,
>> which
>>> will make better use of hardware. 1080p transcoding increases about 17%
>>> fps on my environment.
>>>
>>> This change fits vaSyncBuffer(), so if driver does not support
>>> vaSyncBuffer, it will keep previous operation.
>>>
>>> Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
>>> ---
>>>  libavcodec/vaapi_encode.c | 64 ++++++++++++++++++++++++++++++++-----
>> --
>>>  libavcodec/vaapi_encode.h |  5 +++
>>>  2 files changed, 58 insertions(+), 11 deletions(-)
>>>
>>> diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
>>> index b87b58a42b..9a3b3ba4ad 100644
>>> --- a/libavcodec/vaapi_encode.c
>>> +++ b/libavcodec/vaapi_encode.c
>>> @@ -984,8 +984,10 @@ static int
>> vaapi_encode_pick_next(AVCodecContext *avctx,
>>>      if (!pic && ctx->end_of_stream) {
>>>          --b_counter;
>>>          pic = ctx->pic_end;
>>> -        if (pic->encode_issued)
>>> +        if (pic->encode_complete)
>>>              return AVERROR_EOF;
>>> +        else if (pic->encode_issued)
>>> +            return AVERROR(EAGAIN);
>>>      }
>>>
>>>      if (!pic) {
>>> @@ -1210,18 +1212,45 @@ int
>> ff_vaapi_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt)
>>>              return AVERROR(EAGAIN);
>>>      }
>>>
>>> -    pic = NULL;
>>> -    err = vaapi_encode_pick_next(avctx, &pic);
>>> -    if (err < 0)
>>> -        return err;
>>> -    av_assert0(pic);
>>> +#if VA_CHECK_VERSION(1, 9, 0)
>>> +    if (ctx->has_sync_buffer_func) {
>>> +        while (av_fifo_size(ctx->encode_fifo) <=
>>> +               MAX_PICTURE_REFERENCES * sizeof(VAAPIEncodePicture *)) {
>>> +            pic = NULL;
>>> +            err = vaapi_encode_pick_next(avctx, &pic);
>>> +            if (err < 0)
>>> +                break;
>>> +
>>> +            av_assert0(pic);
>>> +            pic->encode_order = ctx->encode_order +
>>> +                (av_fifo_size(ctx->encode_fifo) / sizeof(VAAPIEncodePicture *));
>>> +            err = vaapi_encode_issue(avctx, pic);
>>> +            if (err < 0) {
>>> +                av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err);
>>> +                return err;
>>> +            }
>>> +            av_fifo_generic_write(ctx->encode_fifo, &pic, sizeof(pic), NULL);
>>> +        }
>>> +        if (!av_fifo_size(ctx->encode_fifo))
>>> +            return err;
>>> +        av_fifo_generic_read(ctx->encode_fifo, &pic, sizeof(pic), NULL);
>>> +        ctx->encode_order = pic->encode_order + 1;
>>> +    } else
>>> +#endif
>>> +    {
>>> +        pic = NULL;
>>> +        err = vaapi_encode_pick_next(avctx, &pic);
>>> +        if (err < 0)
>>> +            return err;
>>> +        av_assert0(pic);
>>>
>>> -    pic->encode_order = ctx->encode_order++;
>>> +        pic->encode_order = ctx->encode_order++;
>>>
>>> -    err = vaapi_encode_issue(avctx, pic);
>>> -    if (err < 0) {
>>> -        av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err);
>>> -        return err;
>>> +        err = vaapi_encode_issue(avctx, pic);
>>> +        if (err < 0) {
>>> +            av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err);
>>> +            return err;
>>> +        }
>>>      }
>>>
>>>      err = vaapi_encode_output(avctx, pic, pkt);
>>> @@ -2555,6 +2584,18 @@ av_cold int
>> ff_vaapi_encode_init(AVCodecContext *avctx)
>>>          }
>>>      }
>>>
>>> +#if VA_CHECK_VERSION(1, 9, 0)
>>> +    //check vaSyncBuffer function
>>> +    vas = vaSyncBuffer(ctx->hwctx->display, 0, 0);
>>> +    if (vas != VA_STATUS_ERROR_UNIMPLEMENTED) {
>>> +        ctx->has_sync_buffer_func = 1;
>>> +        ctx->encode_fifo = av_fifo_alloc((MAX_PICTURE_REFERENCES + 1) *
>>> +                                        sizeof(VAAPIEncodePicture *));
>>> +        if (!ctx->encode_fifo)
>>> +            return AVERROR(ENOMEM);
>>> +    }
>>> +#endif
>>> +
>>>      return 0;
>>>
>>>  fail:
>>> @@ -2592,6 +2633,7 @@ av_cold int
>> ff_vaapi_encode_close(AVCodecContext *avctx)
>>>
>>>      av_freep(&ctx->codec_sequence_params);
>>>      av_freep(&ctx->codec_picture_params);
>>> +    av_fifo_freep(&ctx->encode_fifo);
>>
>> Is it guaranteed that the fifo is empty at this point? I don't think so.
> 
> I don't check the fifo size, because in ff_vaapi_encode_close() all pics
> are already freed and encode_fifo only buffer pic.
> ```
>     for (pic = ctx->pic_start; pic; pic = next) {
>         next = pic->next;
>         vaapi_encode_free(avctx, pic);
>     }
> ```
> 

Ok, seems like the FIFO does not have ownership of the pics. Alright then.

>>
>>>
>>>      av_buffer_unref(&ctx->recon_frames_ref);
>>>      av_buffer_unref(&ctx->input_frames_ref);
>>> diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h
>>> index b41604a883..560a1c42a9 100644
>>> --- a/libavcodec/vaapi_encode.h
>>> +++ b/libavcodec/vaapi_encode.h
>>> @@ -29,6 +29,7 @@
>>>
>>>  #include "libavutil/hwcontext.h"
>>>  #include "libavutil/hwcontext_vaapi.h"
>>> +#include "libavutil/fifo.h"
>>>
>>>  #include "avcodec.h"
>>>  #include "hwconfig.h"
>>> @@ -345,6 +346,10 @@ typedef struct VAAPIEncodeContext {
>>>      int             roi_warned;
>>>
>>>      AVFrame         *frame;
>>> +    //Store buffered pic
>>> +    AVFifoBuffer *encode_fifo;
>>> +    //Whether the driver support vaSyncBuffer
>>> +    int has_sync_buffer_func;
>>>  } VAAPIEncodeContext;
>>>
>>>  enum {
>>>
>>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 8+ messages in thread
* Re: [FFmpeg-devel] [PATCH V2 2/3] libavcodec/vaapi_encode: Change the way to call async to increase performance
  2022-01-05  2:48 ` [FFmpeg-devel] [PATCH V2 2/3] libavcodec/vaapi_encode: Change the way to call async to increase performance Wenbin Chen
  2022-01-05  5:45   ` Andreas Rheinhardt
@ 2022-02-07  4:07   ` Xiang, Haihao
  2022-02-07  5:20     ` Chen, Wenbin
  1 sibling, 1 reply; 8+ messages in thread
From: Xiang, Haihao @ 2022-02-07  4:07 UTC (permalink / raw)
  To: ffmpeg-devel

On Wed, 2022-01-05 at 10:48 +0800, Wenbin Chen wrote:
> Fix: #7706. After commit 5fdcf85bbffe7451c2, vaapi encoder's performance
> decrease. The reason is that vaRenderPicture() and vaSyncBuffer() are
> called at the same time (vaRenderPicture() always followed by a
> vaSyncBuffer()). When we encode stream with B frames, we need buffer to
> reorder frames, so we can send serveral frames to HW at once to increase
> performance. Now I changed them to be called in a asynchronous way, which
> will make better use of hardware. 1080p transcoding increases about 17%
> fps on my environment.
> 
> This change fits vaSyncBuffer(), so if driver does not support
> vaSyncBuffer, it will keep previous operation.
> 
> Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
> ---
>  libavcodec/vaapi_encode.c | 64 ++++++++++++++++++++++++++++++++-------
>  libavcodec/vaapi_encode.h |  5 +++
>  2 files changed, 58 insertions(+), 11 deletions(-)
> 
> diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
> index b87b58a42b..9a3b3ba4ad 100644
> --- a/libavcodec/vaapi_encode.c
> +++ b/libavcodec/vaapi_encode.c
> @@ -984,8 +984,10 @@ static int vaapi_encode_pick_next(AVCodecContext *avctx,
>      if (!pic && ctx->end_of_stream) {
>          --b_counter;
>          pic = ctx->pic_end;
> -        if (pic->encode_issued)
> +        if (pic->encode_complete)
>              return AVERROR_EOF;
> +        else if (pic->encode_issued)
> +            return AVERROR(EAGAIN);
>      }
>  
>      if (!pic) {
> @@ -1210,18 +1212,45 @@ int ff_vaapi_encode_receive_packet(AVCodecContext
> *avctx, AVPacket *pkt)
>              return AVERROR(EAGAIN);
>      }
>  
> -    pic = NULL;
> -    err = vaapi_encode_pick_next(avctx, &pic);
> -    if (err < 0)
> -        return err;
> -    av_assert0(pic);
> +#if VA_CHECK_VERSION(1, 9, 0)
> +    if (ctx->has_sync_buffer_func) {
> +        while (av_fifo_size(ctx->encode_fifo) <=
> +               MAX_PICTURE_REFERENCES * sizeof(VAAPIEncodePicture *)) {
> +            pic = NULL;
> +            err = vaapi_encode_pick_next(avctx, &pic);
> +            if (err < 0)
> +                break;
> +
> +            av_assert0(pic);
> +            pic->encode_order = ctx->encode_order +
> +                (av_fifo_size(ctx->encode_fifo) / sizeof(VAAPIEncodePicture
> *));
> +            err = vaapi_encode_issue(avctx, pic);
> +            if (err < 0) {
> +                av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err);
> +                return err;
> +            }
> +            av_fifo_generic_write(ctx->encode_fifo, &pic, sizeof(pic), NULL);
> +        }
> +        if (!av_fifo_size(ctx->encode_fifo))
> +            return err;
> +        av_fifo_generic_read(ctx->encode_fifo, &pic, sizeof(pic), NULL);
> +        ctx->encode_order = pic->encode_order + 1;
> +    } else
> +#endif
> +    {
> +        pic = NULL;
> +        err = vaapi_encode_pick_next(avctx, &pic);
> +        if (err < 0)
> +            return err;
> +        av_assert0(pic);
>  
> -    pic->encode_order = ctx->encode_order++;
> +        pic->encode_order = ctx->encode_order++;
>  
> -    err = vaapi_encode_issue(avctx, pic);
> -    if (err < 0) {
> -        av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err);
> -        return err;
> +        err = vaapi_encode_issue(avctx, pic);
> +        if (err < 0) {
> +            av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err);
> +            return err;
> +        }
>      }
>  
>      err = vaapi_encode_output(avctx, pic, pkt);
> @@ -2555,6 +2584,18 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx)
>          }
>      }
>  
> +#if VA_CHECK_VERSION(1, 9, 0)
> +    //check vaSyncBuffer function
> +    vas = vaSyncBuffer(ctx->hwctx->display, 0, 0);
> +    if (vas != VA_STATUS_ERROR_UNIMPLEMENTED) {
> +        ctx->has_sync_buffer_func = 1;
> +        ctx->encode_fifo = av_fifo_alloc((MAX_PICTURE_REFERENCES + 1) *
> +                                        sizeof(VAAPIEncodePicture *));
> +        if (!ctx->encode_fifo)
> +            return AVERROR(ENOMEM);
> +    }
> +#endif
> +
>      return 0;
>  
>  fail:
> @@ -2592,6 +2633,7 @@ av_cold int ff_vaapi_encode_close(AVCodecContext *avctx)
>  
>      av_freep(&ctx->codec_sequence_params);
>      av_freep(&ctx->codec_picture_params);
> +    av_fifo_freep(&ctx->encode_fifo);
>  
>      av_buffer_unref(&ctx->recon_frames_ref);
>      av_buffer_unref(&ctx->input_frames_ref);
> diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h
> index b41604a883..560a1c42a9 100644
> --- a/libavcodec/vaapi_encode.h
> +++ b/libavcodec/vaapi_encode.h
> @@ -29,6 +29,7 @@
>  
>  #include "libavutil/hwcontext.h"
>  #include "libavutil/hwcontext_vaapi.h"
> +#include "libavutil/fifo.h"
>  
>  #include "avcodec.h"
>  #include "hwconfig.h"
> @@ -345,6 +346,10 @@ typedef struct VAAPIEncodeContext {
>      int             roi_warned;
>  
>      AVFrame         *frame;
> +    //Store buffered pic
> +    AVFifoBuffer *encode_fifo;
> +    //Whether the driver support vaSyncBuffer

Could you update your patch to use the new FIFO API ?

Thanks
Haihao

> +    int has_sync_buffer_func;
>  } VAAPIEncodeContext;
>  
>  enum {
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 8+ messages in thread
* Re: [FFmpeg-devel] [PATCH V2 2/3] libavcodec/vaapi_encode: Change the way to call async to increase performance
  2022-02-07  4:07   ` Xiang, Haihao
@ 2022-02-07  5:20     ` Chen, Wenbin
  0 siblings, 0 replies; 8+ messages in thread
From: Chen, Wenbin @ 2022-02-07  5:20 UTC (permalink / raw)
  To: FFmpeg development discussions and patches

> On Wed, 2022-01-05 at 10:48 +0800, Wenbin Chen wrote:
> > Fix: #7706. After commit 5fdcf85bbffe7451c2, vaapi encoder's performance
> > decrease. The reason is that vaRenderPicture() and vaSyncBuffer() are
> > called at the same time (vaRenderPicture() always followed by a
> > vaSyncBuffer()). When we encode stream with B frames, we need buffer to
> > reorder frames, so we can send serveral frames to HW at once to increase
> > performance. Now I changed them to be called in a asynchronous way,
> which
> > will make better use of hardware. 1080p transcoding increases about 17%
> > fps on my environment.
> >
> > This change fits vaSyncBuffer(), so if driver does not support
> > vaSyncBuffer, it will keep previous operation.
> >
> > Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
> > ---
> >  libavcodec/vaapi_encode.c | 64 ++++++++++++++++++++++++++++++++-----
> --
> >  libavcodec/vaapi_encode.h |  5 +++
> >  2 files changed, 58 insertions(+), 11 deletions(-)
> >
> > diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
> > index b87b58a42b..9a3b3ba4ad 100644
> > --- a/libavcodec/vaapi_encode.c
> > +++ b/libavcodec/vaapi_encode.c
> > @@ -984,8 +984,10 @@ static int
> vaapi_encode_pick_next(AVCodecContext *avctx,
> >      if (!pic && ctx->end_of_stream) {
> >          --b_counter;
> >          pic = ctx->pic_end;
> > -        if (pic->encode_issued)
> > +        if (pic->encode_complete)
> >              return AVERROR_EOF;
> > +        else if (pic->encode_issued)
> > +            return AVERROR(EAGAIN);
> >      }
> >
> >      if (!pic) {
> > @@ -1210,18 +1212,45 @@ int
> ff_vaapi_encode_receive_packet(AVCodecContext
> > *avctx, AVPacket *pkt)
> >              return AVERROR(EAGAIN);
> >      }
> >
> > -    pic = NULL;
> > -    err = vaapi_encode_pick_next(avctx, &pic);
> > -    if (err < 0)
> > -        return err;
> > -    av_assert0(pic);
> > +#if VA_CHECK_VERSION(1, 9, 0)
> > +    if (ctx->has_sync_buffer_func) {
> > +        while (av_fifo_size(ctx->encode_fifo) <=
> > +               MAX_PICTURE_REFERENCES * sizeof(VAAPIEncodePicture *)) {
> > +            pic = NULL;
> > +            err = vaapi_encode_pick_next(avctx, &pic);
> > +            if (err < 0)
> > +                break;
> > +
> > +            av_assert0(pic);
> > +            pic->encode_order = ctx->encode_order +
> > +                (av_fifo_size(ctx->encode_fifo) / sizeof(VAAPIEncodePicture
> > *));
> > +            err = vaapi_encode_issue(avctx, pic);
> > +            if (err < 0) {
> > +                av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err);
> > +                return err;
> > +            }
> > +            av_fifo_generic_write(ctx->encode_fifo, &pic, sizeof(pic), NULL);
> > +        }
> > +        if (!av_fifo_size(ctx->encode_fifo))
> > +            return err;
> > +        av_fifo_generic_read(ctx->encode_fifo, &pic, sizeof(pic), NULL);
> > +        ctx->encode_order = pic->encode_order + 1;
> > +    } else
> > +#endif
> > +    {
> > +        pic = NULL;
> > +        err = vaapi_encode_pick_next(avctx, &pic);
> > +        if (err < 0)
> > +            return err;
> > +        av_assert0(pic);
> >
> > -    pic->encode_order = ctx->encode_order++;
> > +        pic->encode_order = ctx->encode_order++;
> >
> > -    err = vaapi_encode_issue(avctx, pic);
> > -    if (err < 0) {
> > -        av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err);
> > -        return err;
> > +        err = vaapi_encode_issue(avctx, pic);
> > +        if (err < 0) {
> > +            av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err);
> > +            return err;
> > +        }
> >      }
> >
> >      err = vaapi_encode_output(avctx, pic, pkt);
> > @@ -2555,6 +2584,18 @@ av_cold int
> ff_vaapi_encode_init(AVCodecContext *avctx)
> >          }
> >      }
> >
> > +#if VA_CHECK_VERSION(1, 9, 0)
> > +    //check vaSyncBuffer function
> > +    vas = vaSyncBuffer(ctx->hwctx->display, 0, 0);
> > +    if (vas != VA_STATUS_ERROR_UNIMPLEMENTED) {
> > +        ctx->has_sync_buffer_func = 1;
> > +        ctx->encode_fifo = av_fifo_alloc((MAX_PICTURE_REFERENCES + 1) *
> > +                                        sizeof(VAAPIEncodePicture *));
> > +        if (!ctx->encode_fifo)
> > +            return AVERROR(ENOMEM);
> > +    }
> > +#endif
> > +
> >      return 0;
> >
> >  fail:
> > @@ -2592,6 +2633,7 @@ av_cold int
> ff_vaapi_encode_close(AVCodecContext *avctx)
> >
> >      av_freep(&ctx->codec_sequence_params);
> >      av_freep(&ctx->codec_picture_params);
> > +    av_fifo_freep(&ctx->encode_fifo);
> >
> >      av_buffer_unref(&ctx->recon_frames_ref);
> >      av_buffer_unref(&ctx->input_frames_ref);
> > diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h
> > index b41604a883..560a1c42a9 100644
> > --- a/libavcodec/vaapi_encode.h
> > +++ b/libavcodec/vaapi_encode.h
> > @@ -29,6 +29,7 @@
> >
> >  #include "libavutil/hwcontext.h"
> >  #include "libavutil/hwcontext_vaapi.h"
> > +#include "libavutil/fifo.h"
> >
> >  #include "avcodec.h"
> >  #include "hwconfig.h"
> > @@ -345,6 +346,10 @@ typedef struct VAAPIEncodeContext {
> >      int             roi_warned;
> >
> >      AVFrame         *frame;
> > +    //Store buffered pic
> > +    AVFifoBuffer *encode_fifo;
> > +    //Whether the driver support vaSyncBuffer
> 
> Could you update your patch to use the new FIFO API ?
> 
> Thanks
> Haihao

Ok, I will update it.

Thanks
Wenbin
> 
> > +    int has_sync_buffer_func;
> >  } VAAPIEncodeContext;
> >
> >  enum {
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 8+ messages in thread
end of thread, other threads:[~2022-02-07  5:21 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-05  2:48 [FFmpeg-devel] [PATCH V2 1/3] libavcodec/vaapi_encode: Add new API adaption to vaapi_encode Wenbin Chen
2022-01-05  2:48 ` [FFmpeg-devel] [PATCH V2 2/3] libavcodec/vaapi_encode: Change the way to call async to increase performance Wenbin Chen
2022-01-05  5:45   ` Andreas Rheinhardt
2022-01-05  7:59     ` Chen, Wenbin
2022-01-06  8:41       ` Andreas Rheinhardt
2022-02-07  4:07   ` Xiang, Haihao
2022-02-07  5:20     ` Chen, Wenbin
2022-01-05  2:48 ` [FFmpeg-devel] [PATCH V2 3/3] libavcodec/vaapi_encode: Add async_depth to vaapi_encoder " Wenbin Chen

Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
		ffmpegdev@gitmailbox.com
	public-inbox-index ffmpegdev

Example config snippet for mirrors.


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git