[FFmpeg-devel] [PATCH] Playout to DeckLink will wait for all buffered frames before stopping.

[FFmpeg-devel] [PATCH] Playout to DeckLink will wait for all buffered frames before stopping.

[Matt via ffmpeg-devel]

[-- Attachment #1: Type: message/rfc822, Size: 3539 bytes --]

From: matthew@watchgood.com
To: ffmpeg-devel@ffmpeg.org
Cc: Matthew Rademaker <matthew@watchgood.com>
Subject: [PATCH] Playout to DeckLink will wait for all buffered frames before stopping.
Date: Tue,  8 Jul 2025 11:39:38 +1000
Message-ID: <20250708013949.3044652-1-matthew@watchgood.com>

From: Matthew Rademaker <matthew@watchgood.com>

Instead of stopping video output to the card as soon as encoding is
finished, wait until the card reports that all buffered frames have
been played out. Wait longer if there are more frames, shorter if
there are fewer so that we end ASAP.

See https://ffmpeg.org/pipermail/ffmpeg-devel/2025-March/340634.html

Signed-off-by: Matthew Rademaker <matthew@watchgood.com>
---
 libavdevice/decklink_enc.cpp | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/libavdevice/decklink_enc.cpp b/libavdevice/decklink_enc.cpp
index cb8f91730e..5751f27de8 100644
--- a/libavdevice/decklink_enc.cpp
+++ b/libavdevice/decklink_enc.cpp
@@ -20,6 +20,7 @@
  */

 #include <atomic>
+#include <unistd.h>
 using std::atomic;

 /* Include internal.h first to avoid conflict between winsock.h (used by
@@ -369,11 +370,26 @@ av_cold int ff_decklink_write_trailer(AVFormatContext *avctx)
 {
     struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data;
     struct decklink_ctx *ctx = (struct decklink_ctx *)cctx->ctx;
+    uint32_t buffered;

     if (ctx->playback_started) {
         BMDTimeValue actual;
         ctx->dlo->StopScheduledPlayback(ctx->last_pts * ctx->bmd_tb_num,
                                         &actual, ctx->bmd_tb_den);
+        av_log(avctx, AV_LOG_DEBUG, "Stopped at %ld, requested %ld\n", actual, ctx->last_pts * ctx->bmd_tb_num);
+        while (1){
+            ctx->dlo->GetBufferedVideoFrameCount(&buffered);
+            if (buffered <= 0){
+                break;
+            }
+            av_log(avctx, AV_LOG_DEBUG, "Waiting for %d buffered frames to finish\n", buffered);
+            if (buffered < 5) {
+                usleep(1);
+            } else {
+                usleep(300);
+            }
+        }
+        av_log(avctx, AV_LOG_DEBUG, "All frames returned, finishing up\n");
         ctx->dlo->DisableVideoOutput();
         if (ctx->audio)
             ctx->dlo->DisableAudioOutput();
--
2.43.0


[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] Playout to DeckLink will wait for all buffered frames before stopping.

[Devin Heitmueller]

Hi Matt,

Thanks for your contribution.  Comments inline.

On Mon, Jul 7, 2025 at 9:40 PM Matt via ffmpeg-devel
<ffmpeg-devel@ffmpeg.org> wrote:
> diff --git a/libavdevice/decklink_enc.cpp b/libavdevice/decklink_enc.cpp
> index cb8f91730e..5751f27de8 100644
> --- a/libavdevice/decklink_enc.cpp
> +++ b/libavdevice/decklink_enc.cpp
> @@ -20,6 +20,7 @@
>   */
>
>  #include <atomic>
> +#include <unistd.h>

Not needed (see below)

>  using std::atomic;
>
>  /* Include internal.h first to avoid conflict between winsock.h (used by
> @@ -369,11 +370,26 @@ av_cold int ff_decklink_write_trailer(AVFormatContext *avctx)
>  {
>      struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data;
>      struct decklink_ctx *ctx = (struct decklink_ctx *)cctx->ctx;
> +    uint32_t buffered;
>
>      if (ctx->playback_started) {
>          BMDTimeValue actual;
>          ctx->dlo->StopScheduledPlayback(ctx->last_pts * ctx->bmd_tb_num,
>                                          &actual, ctx->bmd_tb_den);
> +        av_log(avctx, AV_LOG_DEBUG, "Stopped at %ld, requested %ld\n", actual, ctx->last_pts * ctx->bmd_tb_num);
> +        while (1){
> +            ctx->dlo->GetBufferedVideoFrameCount(&buffered);
> +            if (buffered <= 0){
> +                break;
> +            }
> +            av_log(avctx, AV_LOG_DEBUG, "Waiting for %d buffered frames to finish\n", buffered);
> +            if (buffered < 5) {
> +                usleep(1);
> +            } else {
> +                usleep(300);
> +            }

The sleep calls should use av_usleep(), which is portable and you
don't need unistd.h.  Also, there should be some limit so that it
doesn't block forever if video frames are scheduled very far in the
future (e.g. after 1 second, give up and log a warning).

Devin

-- 
Devin Heitmueller, Senior Software Engineer
LTN Global Communications
o: +1 (301) 363-1001
w: https://ltnglobal.com  e: devin.heitmueller@ltnglobal.com
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] Playout to DeckLink will wait for all buffered frames before stopping.

[Marvin Scholz]

On 8 Jul 2025, at 16:54, Devin Heitmueller wrote:

> Hi Matt,
>
> Thanks for your contribution.  Comments inline.
>
> On Mon, Jul 7, 2025 at 9:40 PM Matt via ffmpeg-devel
> <ffmpeg-devel@ffmpeg.org> wrote:
>> diff --git a/libavdevice/decklink_enc.cpp b/libavdevice/decklink_enc.cpp
>> index cb8f91730e..5751f27de8 100644
>> --- a/libavdevice/decklink_enc.cpp
>> +++ b/libavdevice/decklink_enc.cpp
>> @@ -20,6 +20,7 @@
>>   */
>>
>>  #include <atomic>
>> +#include <unistd.h>
>
> Not needed (see below)
>
>>  using std::atomic;
>>
>>  /* Include internal.h first to avoid conflict between winsock.h (used by
>> @@ -369,11 +370,26 @@ av_cold int ff_decklink_write_trailer(AVFormatContext *avctx)
>>  {
>>      struct decklink_cctx *cctx = (struct decklink_cctx *)avctx->priv_data;
>>      struct decklink_ctx *ctx = (struct decklink_ctx *)cctx->ctx;
>> +    uint32_t buffered;

Can be moved inside the while block to narrow the scope.

>>
>>      if (ctx->playback_started) {
>>          BMDTimeValue actual;
>>          ctx->dlo->StopScheduledPlayback(ctx->last_pts * ctx->bmd_tb_num,
>>                                          &actual, ctx->bmd_tb_den);
>> +        av_log(avctx, AV_LOG_DEBUG, "Stopped at %ld, requested %ld\n", actual, ctx->last_pts * ctx->bmd_tb_num);
>> +        while (1){
>> +            ctx->dlo->GetBufferedVideoFrameCount(&buffered);
>> +            if (buffered <= 0){
>> +                break;
>> +            }
>> +            av_log(avctx, AV_LOG_DEBUG, "Waiting for %d buffered frames to finish\n", buffered);
>> +            if (buffered < 5) {
>> +                usleep(1);
>> +            } else {
>> +                usleep(300);
>> +            }
>
> The sleep calls should use av_usleep(), which is portable and you
> don't need unistd.h.  Also, there should be some limit so that it
> doesn't block forever if video frames are scheduled very far in the
> future (e.g. after 1 second, give up and log a warning).
>

I think even if a frame is schedule "far" in the future, if I want them
all played out then it should probably still wait, no? As what is considered
"far" is a matter of interpretation…

Also there should likely be an option to opt in to this new behavior?

> Devin
>
> -- 
> Devin Heitmueller, Senior Software Engineer
> LTN Global Communications
> o: +1 (301) 363-1001
> w: https://ltnglobal.com  e: devin.heitmueller@ltnglobal.com
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] Playout to DeckLink will wait for all buffered frames before stopping.

[Devin Heitmueller]

Hi Marvin,

On Tue, Jul 8, 2025 at 11:19 AM Marvin Scholz
<epirat07-at-gmail.com@ffmpeg.org> wrote:
> I think even if a frame is schedule "far" in the future, if I want them
> all played out then it should probably still wait, no? As what is considered
> "far" is a matter of interpretation…

That probably depends.  If the clock gets screwed up and a frame is
submitted with a PTS five hours into the future you don't want it to
stall for five hours.  The driver lets you control the number of
buffers available (and has limits on such), but IIRC not explicitly on
the timestamps submitted (which dictates when the frame gets played
out).

But yeah, "far" is a matter of interpretation.  In practice you
generally won't see frames queued more than a second in advance.  I
think we can agree on a reasonable threshold that prevents it getting
stuck in a loop for potentially hours if the clocks were bad but
doesn't cause issues in practice (e.g. 2 seconds?).

> Also there should likely be an option to opt in to this new behavior?

Agreed, this behavior should probably be configurable.  I would argue
though that the behavior provided by this patch should be the default.
AFAIK, it wasn't an intentional decision to terminate before all
buffers were output, and flushing pending data on close is the
standard behavior for other muxers.

On a personal note, I work with realtime decoding and thus the
existing behavior is my preference (since I want it to terminate
immediately).  But I think my use case is far less common than people
who simply want to play out their file based content to the Blackmagic
device.

Devin

-- 
Devin Heitmueller, Senior Software Engineer
LTN Global Communications
o: +1 (301) 363-1001
w: https://ltnglobal.com  e: devin.heitmueller@ltnglobal.com
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] Playout to DeckLink will wait for all buffered frames before stopping.

[Marvin Scholz]

On 8 Jul 2025, at 18:22, Devin Heitmueller wrote:

> Hi Marvin,
>
> On Tue, Jul 8, 2025 at 11:19 AM Marvin Scholz
> <epirat07-at-gmail.com@ffmpeg.org> wrote:
>> I think even if a frame is schedule "far" in the future, if I want them
>> all played out then it should probably still wait, no? As what is considered
>> "far" is a matter of interpretation…
>
> That probably depends.  If the clock gets screwed up and a frame is
> submitted with a PTS five hours into the future you don't want it to
> stall for five hours.  The driver lets you control the number of
> buffers available (and has limits on such), but IIRC not explicitly on
> the timestamps submitted (which dictates when the frame gets played
> out).
>
> But yeah, "far" is a matter of interpretation.  In practice you
> generally won't see frames queued more than a second in advance. I
> think we can agree on a reasonable threshold that prevents it getting
> stuck in a loop for potentially hours if the clocks were bad but
> doesn't cause issues in practice (e.g. 2 seconds?).
>
>> Also there should likely be an option to opt in to this new behavior?
>
> Agreed, this behavior should probably be configurable.  I would argue
> though that the behavior provided by this patch should be the default.
> AFAIK, it wasn't an intentional decision to terminate before all
> buffers were output, and flushing pending data on close is the
> standard behavior for other muxers.
>
> On a personal note, I work with realtime decoding and thus the
> existing behavior is my preference (since I want it to terminate
> immediately).  But I think my use case is far less common than people
> who simply want to play out their file based content to the Blackmagic
> device.

What about a drain_timeout option? This could then solve all the
mentioned cases. We could have it default to something like 1000ms,
and for your use-case you could set it to 0, to not drain at all?

>
> Devin
>
> -- 
> Devin Heitmueller, Senior Software Engineer
> LTN Global Communications
> o: +1 (301) 363-1001
> w: https://ltnglobal.com  e: devin.heitmueller@ltnglobal.com
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] Playout to DeckLink will wait for all buffered frames before stopping.

[Devin Heitmueller]

On Tue, Jul 8, 2025 at 1:08 PM Marvin Scholz
<epirat07-at-gmail.com@ffmpeg.org> wrote:
> What about a drain_timeout option? This could then solve all the
> mentioned cases. We could have it default to something like 1000ms,
> and for your use-case you could set it to 0, to not drain at all?

I have no objection to that approach.

Devin

-- 
Devin Heitmueller, Senior Software Engineer
LTN Global Communications
o: +1 (301) 363-1001
w: https://ltnglobal.com  e: devin.heitmueller@ltnglobal.com
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] Playout to DeckLink will wait for all buffered frames before stopping.

[Marton Balint]

On Tue, 8 Jul 2025, Devin Heitmueller wrote:

> On Tue, Jul 8, 2025 at 1:08 PM Marvin Scholz
> <epirat07-at-gmail.com@ffmpeg.org> wrote:
>> What about a drain_timeout option? This could then solve all the
>> mentioned cases. We could have it default to something like 1000ms,
>> and for your use-case you could set it to 0, to not drain at all?
>
> I have no objection to that approach.
>

Ok, but instead of sleeping you should use something like this:

pthread_mutex_lock(&ctx->mutex);
while (ctx->frames_buffer_available_spots < ctx->frame_buffer) {
     pthread_cond_wait(&ctx->cond, &ctx->mutex);
}
pthread_mutex_unlock(&ctx->mutex);

If you want an adjustable timeout you can use pthread_cond_timedwait().

Regards,
Marton
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".