* [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading
@ 2023-09-19 19:10 Anton Khirnov
2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 01/27] fftools/ffmpeg: move derivation of frame duration from filter framerate Anton Khirnov
` (27 more replies)
0 siblings, 28 replies; 30+ messages in thread
From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw)
To: ffmpeg-devel
Hi,
as some of you might know, I have been working on multithreading for the
ffmpeg CLI (as opposed to the libraries) for quite a while now. That
work is now finally entering the vicinity of being usable, so I've
decided to air it on the ML for comments on the big picture.
Do note that many things still do not work at all, notably
* subtitles
* bitstream filters
* streamloop
* filtergraphs with no inputs
Many others are probably broken to a larger or smaller extent.
I can, however, get some interesting results - e.g. significant
wallclock speedup in cases where singlethreaded codecs/filters can now
be run in parallel. I am also seeing 10-20% overall clock cycle increase
in my tests - for now that is a known bug that will hopefully be
addressed later.
One of the main themes of this work is separating various bits of
functionality in the code - that typically grew organically over years
(often in rather random places) - into clearly defined objects with
clearly defined responsibilities. At the core of the new world order is
the new scheduler object, living in ffmpeg_sched.[ch]. It handles all
interactions between the various ffmpeg components, inter-thread
synchronization, etc., so the individual components themselves do not
need to know anything about threading/synchronization/the rest of the
transcoding pipeline.
The patches are roughly grouped as follows:
* 01-13 Preparatory cleanup.
The big thing here is moving fps conversion from video encoding to
filtering (see commit message for 20/27 for details)
* 14-20 "Fake" threading for encoders/filters.
This moves the last two unthreaded components - encoders and filters -
into threads. As previously for decoders, this is merely preparatory,
since the main thread waits while the encoder/filter thread does its
work. The filter patch is unfortunately especially large, because
filtering has multiple entrypoints and they all need to be wrapped and
unwrapped for communication with the filtering thread.
Some features that conflict with threading are also disabled in
these patches. That will hopefully change later (though currently
I don't see how subtitle heartbeat can work in its current form).
* 21 The scheduler itself, merely added without being used
* 22-27 Actually using the scheduler.
This is split into multiple patches for clarity, but will be squashed
in the final form.
You can fetch the code from the 'ffmpeg_threading' branch in
git://git.khirnov.net/libav. I will also present a short talk about this
work at VDD. Comments, questions, suggestions, etc. are very much
welcome, both here and there.
Cheers,
--
Anton Khirnov
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 30+ messages in thread* [FFmpeg-devel] [PATCH 01/27] fftools/ffmpeg: move derivation of frame duration from filter framerate 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 02/27] fftools/ffmpeg_enc: move handling video frame duration to video_sync_process() Anton Khirnov ` (26 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel From ffmpeg_enc to ffmpeg_filter, which is a more appropriate place for it. fate-lavf-gxf* no longer spuriously duplicate the first video frame, due to different rounding. --- fftools/ffmpeg_enc.c | 9 +-------- fftools/ffmpeg_filter.c | 6 +++++- tests/ref/lavf/gxf | 2 +- tests/ref/lavf/gxf_ntsc | 2 +- tests/ref/lavf/gxf_pal | 2 +- 5 files changed, 9 insertions(+), 12 deletions(-) diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index b40a6211a9..9b8b3e0226 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -1109,16 +1109,9 @@ static int do_video_out(OutputFile *of, OutputStream *ost, AVFrame *frame) int64_t nb_frames, nb_frames_prev, i; double duration = 0; - if (frame) { - FrameData *fd = frame_data(frame); - + if (frame) duration = lrintf(frame->duration * av_q2d(frame->time_base) / av_q2d(enc->time_base)); - if (duration <= 0 && - fd->frame_rate_filter.num > 0 && fd->frame_rate_filter.den > 0) - duration = 1 / (av_q2d(fd->frame_rate_filter) * av_q2d(enc->time_base)); - } - video_sync_process(of, ost, frame, duration, &nb_frames, &nb_frames_prev); diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 9bf870b615..b6348d7f87 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -1817,8 +1817,12 @@ static int fg_output_step(OutputFilterPriv *ofp, int flush) if (ost->type == AVMEDIA_TYPE_VIDEO) { AVRational fr = av_buffersink_get_frame_rate(filter); - if (fr.num > 0 && fr.den > 0) + if (fr.num > 0 && fr.den > 0) { fd->frame_rate_filter = fr; + + if (!frame->duration) + frame->duration = av_rescale_q(1, av_inv_q(fr), frame->time_base); + } } ret = enc_frame(ost, frame); diff --git a/tests/ref/lavf/gxf b/tests/ref/lavf/gxf index e8351fab86..d5fd40c8ab 100644 --- a/tests/ref/lavf/gxf +++ b/tests/ref/lavf/gxf @@ -1,3 +1,3 @@ 0638c4d073ac224608baaba16732b68f *tests/data/lavf/lavf.gxf 795876 tests/data/lavf/lavf.gxf -tests/data/lavf/lavf.gxf CRC=0x5ade0285 +tests/data/lavf/lavf.gxf CRC=0xe032707a diff --git a/tests/ref/lavf/gxf_ntsc b/tests/ref/lavf/gxf_ntsc index 60efd80462..d375420ee6 100644 --- a/tests/ref/lavf/gxf_ntsc +++ b/tests/ref/lavf/gxf_ntsc @@ -1,3 +1,3 @@ 9a27673c85f1671ba9ff7cd33e5735de *tests/data/lavf/lavf.gxf_ntsc 794660 tests/data/lavf/lavf.gxf_ntsc -tests/data/lavf/lavf.gxf_ntsc CRC=0xdcd39443 +tests/data/lavf/lavf.gxf_ntsc CRC=0xaf956c57 diff --git a/tests/ref/lavf/gxf_pal b/tests/ref/lavf/gxf_pal index aefcd0ccab..83b0482431 100644 --- a/tests/ref/lavf/gxf_pal +++ b/tests/ref/lavf/gxf_pal @@ -1,3 +1,3 @@ 4d1bd16c6d52468c05711d8301e4e302 *tests/data/lavf/lavf.gxf_pal 795880 tests/data/lavf/lavf.gxf_pal -tests/data/lavf/lavf.gxf_pal CRC=0x1dbfef76 +tests/data/lavf/lavf.gxf_pal CRC=0x34fe5d7a -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 02/27] fftools/ffmpeg_enc: move handling video frame duration to video_sync_process() 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 01/27] fftools/ffmpeg: move derivation of frame duration from filter framerate Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 03/27] fftools/ffmpeg_enc: move remaining vsync-related code " Anton Khirnov ` (25 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel That is a more appropriate place for this. --- fftools/ffmpeg_enc.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index 9b8b3e0226..9aa00f682c 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -965,12 +965,12 @@ early_exit: * should this (and possibly previous) frame be repeated in order to conform to * desired target framerate (if any). */ -static void video_sync_process(OutputFile *of, OutputStream *ost, - AVFrame *frame, double duration, +static void video_sync_process(OutputFile *of, OutputStream *ost, AVFrame *frame, int64_t *nb_frames, int64_t *nb_frames_prev) { Encoder *e = ost->enc; - double delta0, delta, sync_ipts; + AVCodecContext *enc = ost->enc_ctx; + double delta0, delta, sync_ipts, duration; if (!frame) { *nb_frames_prev = *nb_frames = mid_pred(e->frames_prev_hist[0], @@ -979,6 +979,8 @@ static void video_sync_process(OutputFile *of, OutputStream *ost, goto finish; } + duration = lrintf(frame->duration * av_q2d(frame->time_base) / av_q2d(enc->time_base)); + sync_ipts = adjust_frame_pts_to_encoder_tb(of, ost, frame); /* delta0 is the "drift" between the input frame and * where it would fall in the output. */ @@ -1107,12 +1109,8 @@ static int do_video_out(OutputFile *of, OutputStream *ost, AVFrame *frame) Encoder *e = ost->enc; AVCodecContext *enc = ost->enc_ctx; int64_t nb_frames, nb_frames_prev, i; - double duration = 0; - if (frame) - duration = lrintf(frame->duration * av_q2d(frame->time_base) / av_q2d(enc->time_base)); - - video_sync_process(of, ost, frame, duration, + video_sync_process(of, ost, frame, &nb_frames, &nb_frames_prev); if (nb_frames_prev == 0 && ost->last_dropped) { -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 03/27] fftools/ffmpeg_enc: move remaining vsync-related code to video_sync_process() 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 01/27] fftools/ffmpeg: move derivation of frame duration from filter framerate Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 02/27] fftools/ffmpeg_enc: move handling video frame duration to video_sync_process() Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 04/27] fftools/ffmpeg_enc: simplify adjust_frame_pts_to_encoder_tb() signature Anton Khirnov ` (24 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel --- fftools/ffmpeg_enc.c | 46 +++++++++++++++++++++++--------------------- 1 file changed, 24 insertions(+), 22 deletions(-) diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index 9aa00f682c..420fe96c97 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -1048,6 +1048,30 @@ finish: e->frames_prev_hist, sizeof(e->frames_prev_hist[0]) * (FF_ARRAY_ELEMS(e->frames_prev_hist) - 1)); e->frames_prev_hist[0] = *nb_frames_prev; + + if (*nb_frames_prev == 0 && ost->last_dropped) { + ost->nb_frames_drop++; + av_log(ost, AV_LOG_VERBOSE, + "*** dropping frame %"PRId64" at ts %"PRId64"\n", + e->vsync_frame_number, e->last_frame->pts); + } + if (*nb_frames > (*nb_frames_prev && ost->last_dropped) + (*nb_frames > *nb_frames_prev)) { + if (*nb_frames > dts_error_threshold * 30) { + av_log(ost, AV_LOG_ERROR, "%"PRId64" frame duplication too large, skipping\n", *nb_frames - 1); + ost->nb_frames_drop++; + *nb_frames = 0; + return; + } + ost->nb_frames_dup += *nb_frames - (*nb_frames_prev && ost->last_dropped) - (*nb_frames > *nb_frames_prev); + av_log(ost, AV_LOG_VERBOSE, "*** %"PRId64" dup!\n", *nb_frames - 1); + if (ost->nb_frames_dup > e->dup_warning) { + av_log(ost, AV_LOG_WARNING, "More than %"PRIu64" frames duplicated\n", e->dup_warning); + e->dup_warning *= 10; + } + } + + ost->last_dropped = *nb_frames == *nb_frames_prev && frame; + ost->kf.dropped_keyframe = ost->last_dropped && frame && (frame->flags & AV_FRAME_FLAG_KEY); } static enum AVPictureType forced_kf_apply(void *logctx, KeyframeForceCtx *kf, @@ -1113,28 +1137,6 @@ static int do_video_out(OutputFile *of, OutputStream *ost, AVFrame *frame) video_sync_process(of, ost, frame, &nb_frames, &nb_frames_prev); - if (nb_frames_prev == 0 && ost->last_dropped) { - ost->nb_frames_drop++; - av_log(ost, AV_LOG_VERBOSE, - "*** dropping frame %"PRId64" at ts %"PRId64"\n", - e->vsync_frame_number, e->last_frame->pts); - } - if (nb_frames > (nb_frames_prev && ost->last_dropped) + (nb_frames > nb_frames_prev)) { - if (nb_frames > dts_error_threshold * 30) { - av_log(ost, AV_LOG_ERROR, "%"PRId64" frame duplication too large, skipping\n", nb_frames - 1); - ost->nb_frames_drop++; - return 0; - } - ost->nb_frames_dup += nb_frames - (nb_frames_prev && ost->last_dropped) - (nb_frames > nb_frames_prev); - av_log(ost, AV_LOG_VERBOSE, "*** %"PRId64" dup!\n", nb_frames - 1); - if (ost->nb_frames_dup > e->dup_warning) { - av_log(ost, AV_LOG_WARNING, "More than %"PRIu64" frames duplicated\n", e->dup_warning); - e->dup_warning *= 10; - } - } - ost->last_dropped = nb_frames == nb_frames_prev && frame; - ost->kf.dropped_keyframe = ost->last_dropped && frame && (frame->flags & AV_FRAME_FLAG_KEY); - /* duplicates frame if needed */ for (i = 0; i < nb_frames; i++) { AVFrame *in_picture; -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 04/27] fftools/ffmpeg_enc: simplify adjust_frame_pts_to_encoder_tb() signature 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (2 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 03/27] fftools/ffmpeg_enc: move remaining vsync-related code " Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 05/27] ffools/ffmpeg_filter: stop trying to handle an unreachable state Anton Khirnov ` (23 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel It does not need an OutputFile and an OutputStream, only the target timebase and the timestamp offset. --- fftools/ffmpeg_enc.c | 25 ++++++++++--------------- 1 file changed, 10 insertions(+), 15 deletions(-) diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index 420fe96c97..df79eaff59 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -917,16 +917,12 @@ static int do_audio_out(OutputFile *of, OutputStream *ost, return (ret < 0 && ret != AVERROR_EOF) ? ret : 0; } -static double adjust_frame_pts_to_encoder_tb(OutputFile *of, OutputStream *ost, - AVFrame *frame) +static double adjust_frame_pts_to_encoder_tb(AVFrame *frame, AVRational tb_dst, + int64_t start_time) { double float_pts = AV_NOPTS_VALUE; // this is identical to frame.pts but with higher precision - const int64_t start_time = (of->start_time == AV_NOPTS_VALUE) ? - 0 : of->start_time; - AVCodecContext *const enc = ost->enc_ctx; - - AVRational tb = enc->time_base; + AVRational tb = tb_dst; AVRational filter_tb = frame->time_base; const int extra_bits = av_clip(29 - av_log2(tb.den), 0, 16); @@ -943,19 +939,17 @@ static double adjust_frame_pts_to_encoder_tb(OutputFile *of, OutputStream *ost, if (float_pts != llrint(float_pts)) float_pts += FFSIGN(float_pts) * 1.0 / (1<<17); - frame->pts = av_rescale_q(frame->pts, filter_tb, enc->time_base) - - av_rescale_q(start_time, AV_TIME_BASE_Q, enc->time_base); - frame->time_base = enc->time_base; + frame->pts = av_rescale_q(frame->pts, filter_tb, tb_dst) - + av_rescale_q(start_time, AV_TIME_BASE_Q, tb_dst); + frame->time_base = tb_dst; early_exit: if (debug_ts) { av_log(NULL, AV_LOG_INFO, "filter -> pts:%s pts_time:%s exact:%f time_base:%d/%d\n", frame ? av_ts2str(frame->pts) : "NULL", - (enc && frame) ? av_ts2timestr(frame->pts, &enc->time_base) : "NULL", - float_pts, - enc ? enc->time_base.num : -1, - enc ? enc->time_base.den : -1); + av_ts2timestr(frame->pts, &tb_dst), + float_pts, tb_dst.num, tb_dst.den); } return float_pts; @@ -981,7 +975,8 @@ static void video_sync_process(OutputFile *of, OutputStream *ost, AVFrame *frame duration = lrintf(frame->duration * av_q2d(frame->time_base) / av_q2d(enc->time_base)); - sync_ipts = adjust_frame_pts_to_encoder_tb(of, ost, frame); + sync_ipts = adjust_frame_pts_to_encoder_tb(frame, enc->time_base, + of->start_time == AV_NOPTS_VALUE ? 0 : of->start_time); /* delta0 is the "drift" between the input frame and * where it would fall in the output. */ delta0 = sync_ipts - e->next_pts; -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 05/27] ffools/ffmpeg_filter: stop trying to handle an unreachable state 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (3 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 04/27] fftools/ffmpeg_enc: simplify adjust_frame_pts_to_encoder_tb() signature Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 06/27] tests/fate/ffmpeg: add tests for -force_key_frames source Anton Khirnov ` (22 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel ifilter_send_eof() will fail if the input has no real or fallback parameters, so there is no need to handle the case of some inputs being in EOF state yet having no parameters. --- fftools/ffmpeg.c | 2 +- fftools/ffmpeg.h | 2 -- fftools/ffmpeg_filter.c | 10 +++++----- 3 files changed, 6 insertions(+), 8 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index fd2ce1c2d0..14f55cbec7 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -944,7 +944,7 @@ static int choose_output(OutputStream **post) INT64_MIN : ost->last_mux_dts; } - if (!ost->initialized && !ost->inputs_done && !ost->finished) { + if (!ost->initialized && !ost->finished) { ost_min = ost; break; } diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index b059ecbb9f..2e8f1db9b6 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -583,8 +583,6 @@ typedef struct OutputStream { // parameters are set in the AVStream. int initialized; - int inputs_done; - const char *attachment_filename; int keep_pix_fmt; diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index b6348d7f87..804b9de3dc 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -2076,11 +2076,11 @@ int fg_transcode_step(FilterGraph *graph, InputStream **best_ist) } } - // graph not configured, but all inputs are either initialized or EOF - for (int i = 0; i < graph->nb_outputs; i++) - graph->outputs[i]->ost->inputs_done = 1; - - return 0; + // This state - graph is not configured, but all inputs are either + // initialized or EOF - should be unreachable because sending EOF to a + // filter without even a fallback format should fail + av_assert0(0); + return AVERROR_BUG; } *best_ist = NULL; -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 06/27] tests/fate/ffmpeg: add tests for -force_key_frames source 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (4 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 05/27] ffools/ffmpeg_filter: stop trying to handle an unreachable state Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 07/27] fftools/ffmpeg_enc: unbreak -force_key_frames source_no_drop Anton Khirnov ` (21 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel --- tests/fate/ffmpeg.mak | 20 + tests/ref/fate/force_key_frames-source | 397 +++++++++++++ tests/ref/fate/force_key_frames-source-drop | 22 + tests/ref/fate/force_key_frames-source-dup | 617 ++++++++++++++++++++ 4 files changed, 1056 insertions(+) create mode 100644 tests/ref/fate/force_key_frames-source create mode 100644 tests/ref/fate/force_key_frames-source-drop create mode 100644 tests/ref/fate/force_key_frames-source-dup diff --git a/tests/fate/ffmpeg.mak b/tests/fate/ffmpeg.mak index 04500d53a0..835770a924 100644 --- a/tests/fate/ffmpeg.mak +++ b/tests/fate/ffmpeg.mak @@ -48,6 +48,26 @@ fate-force_key_frames: CMD = enc_dec \ avi "-c mpeg4 -g 240 -qscale 10 -force_key_frames 0.5,0:00:01.5" \ framecrc "" "-skip_frame nokey" +# test -force_key_frames source with and without framerate conversion +# * we don't care about the actual video content, so replace it with +# a 2x2 black square to speed up encoding +# * force mpeg2video to only emit keyframes when explicitly requested +fate-force_key_frames-source: CMD = framecrc -i $(TARGET_SAMPLES)/h264/intra_refresh.h264 \ + -vf crop=2:2,drawbox=color=black:t=fill \ + -c:v mpeg2video -g 400 -sc_threshold 99999 \ + -force_key_frames source +fate-force_key_frames-source-drop: CMD = framecrc -i $(TARGET_SAMPLES)/h264/intra_refresh.h264 \ + -vf crop=2:2,drawbox=color=black:t=fill \ + -c:v mpeg2video -g 400 -sc_threshold 99999 \ + -force_key_frames source -r 1 +fate-force_key_frames-source-dup: CMD = framecrc -i $(TARGET_SAMPLES)/h264/intra_refresh.h264 \ + -vf crop=2:2,drawbox=color=black:t=fill \ + -c:v mpeg2video -g 400 -sc_threshold 99999 \ + -force_key_frames source -r 39 -force_fps -strict experimental + +FATE_SAMPLES_FFMPEG-$(call ENCDEC, MPEG2VIDEO H264, FRAMECRC H264, CROP_FILTER DRAWBOX_FILTER) += \ + fate-force_key_frames-source fate-force_key_frames-source-drop fate-force_key_frames-source-dup + # Tests that the video is properly autorotated using the contained # display matrix and that the generated file does not contain # a display matrix any more. diff --git a/tests/ref/fate/force_key_frames-source b/tests/ref/fate/force_key_frames-source new file mode 100644 index 0000000000..87a35e2d2c --- /dev/null +++ b/tests/ref/fate/force_key_frames-source @@ -0,0 +1,397 @@ +#tb 0: 1/25 +#media_type 0: video +#codec_id 0: mpeg2video +#dimensions 0: 2x2 +#sar 0: 0/1 +0, -1, 0, 1, 57, 0x7db00eb7, S=1, 8 +0, 0, 1, 1, 24, 0x4f1c0660, F=0x0, S=1, 8 +0, 1, 2, 1, 24, 0x53dc06a0, F=0x0, S=1, 8 +0, 2, 3, 1, 24, 0x589c06e0, F=0x0, S=1, 8 +0, 3, 4, 1, 24, 0x4a700621, F=0x0, S=1, 8 +0, 4, 5, 1, 24, 0x4f300661, F=0x0, S=1, 8 +0, 5, 6, 1, 24, 0x53f006a1, F=0x0, S=1, 8 +0, 6, 7, 1, 24, 0x58b006e1, F=0x0, S=1, 8 +0, 7, 8, 1, 24, 0x4a840622, F=0x0, S=1, 8 +0, 8, 9, 1, 24, 0x4f440662, F=0x0, S=1, 8 +0, 9, 10, 1, 24, 0x540406a2, F=0x0, S=1, 8 +0, 10, 11, 1, 24, 0x58c406e2, F=0x0, S=1, 8 +0, 11, 12, 1, 24, 0x4a980623, F=0x0, S=1, 8 +0, 12, 13, 1, 24, 0x4f580663, F=0x0, S=1, 8 +0, 13, 14, 1, 24, 0x541806a3, F=0x0, S=1, 8 +0, 14, 15, 1, 24, 0x58d806e3, F=0x0, S=1, 8 +0, 15, 16, 1, 24, 0x4aac0624, F=0x0, S=1, 8 +0, 16, 17, 1, 24, 0x4f6c0664, F=0x0, S=1, 8 +0, 17, 18, 1, 24, 0x542c06a4, F=0x0, S=1, 8 +0, 18, 19, 1, 24, 0x58ec06e4, F=0x0, S=1, 8 +0, 19, 20, 1, 24, 0x4ac00625, F=0x0, S=1, 8 +0, 20, 21, 1, 24, 0x4f800665, F=0x0, S=1, 8 +0, 21, 22, 1, 24, 0x544006a5, F=0x0, S=1, 8 +0, 22, 23, 1, 24, 0x590006e5, F=0x0, S=1, 8 +0, 23, 24, 1, 24, 0x4ad40626, F=0x0, S=1, 8 +0, 24, 25, 1, 24, 0x4f940666, F=0x0, S=1, 8 +0, 25, 26, 1, 24, 0x545406a6, F=0x0, S=1, 8 +0, 26, 27, 1, 24, 0x591406e6, F=0x0, S=1, 8 +0, 27, 28, 1, 24, 0x4ae80627, F=0x0, S=1, 8 +0, 28, 29, 1, 24, 0x4fa80667, F=0x0, S=1, 8 +0, 29, 30, 1, 24, 0x546806a7, F=0x0, S=1, 8 +0, 30, 31, 1, 24, 0x592806e7, F=0x0, S=1, 8 +0, 31, 32, 1, 24, 0x4afc0628, F=0x0, S=1, 8 +0, 32, 33, 1, 24, 0x4fbc0668, F=0x0, S=1, 8 +0, 33, 34, 1, 24, 0x547c06a8, F=0x0, S=1, 8 +0, 34, 35, 1, 24, 0x593c06e8, F=0x0, S=1, 8 +0, 35, 36, 1, 24, 0x4b100629, F=0x0, S=1, 8 +0, 36, 37, 1, 24, 0x4fd00669, F=0x0, S=1, 8 +0, 37, 38, 1, 24, 0x549006a9, F=0x0, S=1, 8 +0, 38, 39, 1, 24, 0x595006e9, F=0x0, S=1, 8 +0, 39, 40, 1, 24, 0x4b24062a, F=0x0, S=1, 8 +0, 40, 41, 1, 24, 0x4fe4066a, F=0x0, S=1, 8 +0, 41, 42, 1, 24, 0x54a406aa, F=0x0, S=1, 8 +0, 42, 43, 1, 24, 0x596406ea, F=0x0, S=1, 8 +0, 43, 44, 1, 24, 0x4b38062b, F=0x0, S=1, 8 +0, 44, 45, 1, 24, 0x4ff8066b, F=0x0, S=1, 8 +0, 45, 46, 1, 24, 0x54b806ab, F=0x0, S=1, 8 +0, 46, 47, 1, 24, 0x597806eb, F=0x0, S=1, 8 +0, 47, 48, 1, 24, 0x4b4c062c, F=0x0, S=1, 8 +0, 48, 49, 1, 24, 0x500c066c, F=0x0, S=1, 8 +0, 49, 50, 1, 24, 0x54cc06ac, F=0x0, S=1, 8 +0, 50, 51, 1, 24, 0x598c06ec, F=0x0, S=1, 8 +0, 51, 52, 1, 24, 0x4b60062d, F=0x0, S=1, 8 +0, 52, 53, 1, 24, 0x5020066d, F=0x0, S=1, 8 +0, 53, 54, 1, 24, 0x54e006ad, F=0x0, S=1, 8 +0, 54, 55, 1, 24, 0x59a006ed, F=0x0, S=1, 8 +0, 55, 56, 1, 24, 0x4b74062e, F=0x0, S=1, 8 +0, 56, 57, 1, 24, 0x5034066e, F=0x0, S=1, 8 +0, 57, 58, 1, 24, 0x54f406ae, F=0x0, S=1, 8 +0, 58, 59, 1, 24, 0x59b406ee, F=0x0, S=1, 8 +0, 59, 60, 1, 24, 0x4b88062f, F=0x0, S=1, 8 +0, 60, 61, 1, 24, 0x5048066f, F=0x0, S=1, 8 +0, 61, 62, 1, 24, 0x550806af, F=0x0, S=1, 8 +0, 62, 63, 1, 24, 0x59c806ef, F=0x0, S=1, 8 +0, 63, 64, 1, 24, 0x4b9c0630, F=0x0, S=1, 8 +0, 64, 65, 1, 24, 0x505c0670, F=0x0, S=1, 8 +0, 65, 66, 1, 24, 0x551c06b0, F=0x0, S=1, 8 +0, 66, 67, 1, 24, 0x59dc06f0, F=0x0, S=1, 8 +0, 67, 68, 1, 24, 0x4bb00631, F=0x0, S=1, 8 +0, 68, 69, 1, 24, 0x50700671, F=0x0, S=1, 8 +0, 69, 70, 1, 24, 0x553006b1, F=0x0, S=1, 8 +0, 70, 71, 1, 24, 0x59f006f1, F=0x0, S=1, 8 +0, 71, 72, 1, 24, 0x4bc40632, F=0x0, S=1, 8 +0, 72, 73, 1, 24, 0x50840672, F=0x0, S=1, 8 +0, 73, 74, 1, 24, 0x554406b2, F=0x0, S=1, 8 +0, 74, 75, 1, 24, 0x5a0406f2, F=0x0, S=1, 8 +0, 75, 76, 1, 24, 0x4bd80633, F=0x0, S=1, 8 +0, 76, 77, 1, 24, 0x50980673, F=0x0, S=1, 8 +0, 77, 78, 1, 24, 0x555806b3, F=0x0, S=1, 8 +0, 78, 79, 1, 24, 0x5a1806f3, F=0x0, S=1, 8 +0, 79, 80, 1, 24, 0x4bec0634, F=0x0, S=1, 8 +0, 80, 81, 1, 24, 0x50ac0674, F=0x0, S=1, 8 +0, 81, 82, 1, 24, 0x556c06b4, F=0x0, S=1, 8 +0, 82, 83, 1, 24, 0x5a2c06f4, F=0x0, S=1, 8 +0, 83, 84, 1, 24, 0x4c000635, F=0x0, S=1, 8 +0, 84, 85, 1, 24, 0x50c00675, F=0x0, S=1, 8 +0, 85, 86, 1, 24, 0x558006b5, F=0x0, S=1, 8 +0, 86, 87, 1, 24, 0x5a4006f5, F=0x0, S=1, 8 +0, 87, 88, 1, 24, 0x4c140636, F=0x0, S=1, 8 +0, 88, 89, 1, 24, 0x50d40676, F=0x0, S=1, 8 +0, 89, 90, 1, 24, 0x559406b6, F=0x0, S=1, 8 +0, 90, 91, 1, 24, 0x5a5406f6, F=0x0, S=1, 8 +0, 91, 92, 1, 24, 0x4c280637, F=0x0, S=1, 8 +0, 92, 93, 1, 24, 0x50e80677, F=0x0, S=1, 8 +0, 93, 94, 1, 24, 0x55a806b7, F=0x0, S=1, 8 +0, 94, 95, 1, 24, 0x5a6806f7, F=0x0, S=1, 8 +0, 95, 96, 1, 24, 0x4c3c0638, F=0x0, S=1, 8 +0, 96, 97, 1, 24, 0x50fc0678, F=0x0, S=1, 8 +0, 97, 98, 1, 24, 0x55bc06b8, F=0x0, S=1, 8 +0, 98, 99, 1, 24, 0x5a7c06f8, F=0x0, S=1, 8 +0, 99, 100, 1, 24, 0x4c500639, F=0x0, S=1, 8 +0, 100, 101, 1, 24, 0x51100679, F=0x0, S=1, 8 +0, 101, 102, 1, 24, 0x55d006b9, F=0x0, S=1, 8 +0, 102, 103, 1, 24, 0x5a9006f9, F=0x0, S=1, 8 +0, 103, 104, 1, 24, 0x4c64063a, F=0x0, S=1, 8 +0, 104, 105, 1, 24, 0x5124067a, F=0x0, S=1, 8 +0, 105, 106, 1, 24, 0x55e406ba, F=0x0, S=1, 8 +0, 106, 107, 1, 24, 0x5aa406fa, F=0x0, S=1, 8 +0, 107, 108, 1, 57, 0x85a40efb, S=1, 8 +0, 108, 109, 1, 24, 0x4f1c0660, F=0x0, S=1, 8 +0, 109, 110, 1, 24, 0x53dc06a0, F=0x0, S=1, 8 +0, 110, 111, 1, 24, 0x589c06e0, F=0x0, S=1, 8 +0, 111, 112, 1, 24, 0x4a700621, F=0x0, S=1, 8 +0, 112, 113, 1, 24, 0x4f300661, F=0x0, S=1, 8 +0, 113, 114, 1, 24, 0x53f006a1, F=0x0, S=1, 8 +0, 114, 115, 1, 24, 0x58b006e1, F=0x0, S=1, 8 +0, 115, 116, 1, 24, 0x4a840622, F=0x0, S=1, 8 +0, 116, 117, 1, 24, 0x4f440662, F=0x0, S=1, 8 +0, 117, 118, 1, 24, 0x540406a2, F=0x0, S=1, 8 +0, 118, 119, 1, 24, 0x58c406e2, F=0x0, S=1, 8 +0, 119, 120, 1, 24, 0x4a980623, F=0x0, S=1, 8 +0, 120, 121, 1, 24, 0x4f580663, F=0x0, S=1, 8 +0, 121, 122, 1, 24, 0x541806a3, F=0x0, S=1, 8 +0, 122, 123, 1, 24, 0x58d806e3, F=0x0, S=1, 8 +0, 123, 124, 1, 24, 0x4aac0624, F=0x0, S=1, 8 +0, 124, 125, 1, 24, 0x4f6c0664, F=0x0, S=1, 8 +0, 125, 126, 1, 24, 0x542c06a4, F=0x0, S=1, 8 +0, 126, 127, 1, 24, 0x58ec06e4, F=0x0, S=1, 8 +0, 127, 128, 1, 24, 0x4ac00625, F=0x0, S=1, 8 +0, 128, 129, 1, 24, 0x4f800665, F=0x0, S=1, 8 +0, 129, 130, 1, 24, 0x544006a5, F=0x0, S=1, 8 +0, 130, 131, 1, 24, 0x590006e5, F=0x0, S=1, 8 +0, 131, 132, 1, 24, 0x4ad40626, F=0x0, S=1, 8 +0, 132, 133, 1, 24, 0x4f940666, F=0x0, S=1, 8 +0, 133, 134, 1, 24, 0x545406a6, F=0x0, S=1, 8 +0, 134, 135, 1, 24, 0x591406e6, F=0x0, S=1, 8 +0, 135, 136, 1, 24, 0x4ae80627, F=0x0, S=1, 8 +0, 136, 137, 1, 24, 0x4fa80667, F=0x0, S=1, 8 +0, 137, 138, 1, 24, 0x546806a7, F=0x0, S=1, 8 +0, 138, 139, 1, 24, 0x592806e7, F=0x0, S=1, 8 +0, 139, 140, 1, 24, 0x4afc0628, F=0x0, S=1, 8 +0, 140, 141, 1, 24, 0x4fbc0668, F=0x0, S=1, 8 +0, 141, 142, 1, 24, 0x547c06a8, F=0x0, S=1, 8 +0, 142, 143, 1, 24, 0x593c06e8, F=0x0, S=1, 8 +0, 143, 144, 1, 24, 0x4b100629, F=0x0, S=1, 8 +0, 144, 145, 1, 24, 0x4fd00669, F=0x0, S=1, 8 +0, 145, 146, 1, 24, 0x549006a9, F=0x0, S=1, 8 +0, 146, 147, 1, 24, 0x595006e9, F=0x0, S=1, 8 +0, 147, 148, 1, 24, 0x4b24062a, F=0x0, S=1, 8 +0, 148, 149, 1, 24, 0x4fe4066a, F=0x0, S=1, 8 +0, 149, 150, 1, 24, 0x54a406aa, F=0x0, S=1, 8 +0, 150, 151, 1, 24, 0x596406ea, F=0x0, S=1, 8 +0, 151, 152, 1, 24, 0x4b38062b, F=0x0, S=1, 8 +0, 152, 153, 1, 24, 0x4ff8066b, F=0x0, S=1, 8 +0, 153, 154, 1, 24, 0x54b806ab, F=0x0, S=1, 8 +0, 154, 155, 1, 24, 0x597806eb, F=0x0, S=1, 8 +0, 155, 156, 1, 24, 0x4b4c062c, F=0x0, S=1, 8 +0, 156, 157, 1, 24, 0x500c066c, F=0x0, S=1, 8 +0, 157, 158, 1, 24, 0x54cc06ac, F=0x0, S=1, 8 +0, 158, 159, 1, 24, 0x598c06ec, F=0x0, S=1, 8 +0, 159, 160, 1, 24, 0x4b60062d, F=0x0, S=1, 8 +0, 160, 161, 1, 24, 0x5020066d, F=0x0, S=1, 8 +0, 161, 162, 1, 24, 0x54e006ad, F=0x0, S=1, 8 +0, 162, 163, 1, 24, 0x59a006ed, F=0x0, S=1, 8 +0, 163, 164, 1, 24, 0x4b74062e, F=0x0, S=1, 8 +0, 164, 165, 1, 24, 0x5034066e, F=0x0, S=1, 8 +0, 165, 166, 1, 24, 0x54f406ae, F=0x0, S=1, 8 +0, 166, 167, 1, 24, 0x59b406ee, F=0x0, S=1, 8 +0, 167, 168, 1, 24, 0x4b88062f, F=0x0, S=1, 8 +0, 168, 169, 1, 24, 0x5048066f, F=0x0, S=1, 8 +0, 169, 170, 1, 24, 0x550806af, F=0x0, S=1, 8 +0, 170, 171, 1, 24, 0x59c806ef, F=0x0, S=1, 8 +0, 171, 172, 1, 24, 0x4b9c0630, F=0x0, S=1, 8 +0, 172, 173, 1, 24, 0x505c0670, F=0x0, S=1, 8 +0, 173, 174, 1, 24, 0x551c06b0, F=0x0, S=1, 8 +0, 174, 175, 1, 24, 0x59dc06f0, F=0x0, S=1, 8 +0, 175, 176, 1, 24, 0x4bb00631, F=0x0, S=1, 8 +0, 176, 177, 1, 24, 0x50700671, F=0x0, S=1, 8 +0, 177, 178, 1, 24, 0x553006b1, F=0x0, S=1, 8 +0, 178, 179, 1, 24, 0x59f006f1, F=0x0, S=1, 8 +0, 179, 180, 1, 24, 0x4bc40632, F=0x0, S=1, 8 +0, 180, 181, 1, 24, 0x50840672, F=0x0, S=1, 8 +0, 181, 182, 1, 24, 0x554406b2, F=0x0, S=1, 8 +0, 182, 183, 1, 24, 0x5a0406f2, F=0x0, S=1, 8 +0, 183, 184, 1, 24, 0x4bd80633, F=0x0, S=1, 8 +0, 184, 185, 1, 24, 0x50980673, F=0x0, S=1, 8 +0, 185, 186, 1, 24, 0x555806b3, F=0x0, S=1, 8 +0, 186, 187, 1, 24, 0x5a1806f3, F=0x0, S=1, 8 +0, 187, 188, 1, 24, 0x4bec0634, F=0x0, S=1, 8 +0, 188, 189, 1, 24, 0x50ac0674, F=0x0, S=1, 8 +0, 189, 190, 1, 24, 0x556c06b4, F=0x0, S=1, 8 +0, 190, 191, 1, 24, 0x5a2c06f4, F=0x0, S=1, 8 +0, 191, 192, 1, 24, 0x4c000635, F=0x0, S=1, 8 +0, 192, 193, 1, 24, 0x50c00675, F=0x0, S=1, 8 +0, 193, 194, 1, 24, 0x558006b5, F=0x0, S=1, 8 +0, 194, 195, 1, 24, 0x5a4006f5, F=0x0, S=1, 8 +0, 195, 196, 1, 24, 0x4c140636, F=0x0, S=1, 8 +0, 196, 197, 1, 24, 0x50d40676, F=0x0, S=1, 8 +0, 197, 198, 1, 24, 0x559406b6, F=0x0, S=1, 8 +0, 198, 199, 1, 24, 0x5a5406f6, F=0x0, S=1, 8 +0, 199, 200, 1, 24, 0x4c280637, F=0x0, S=1, 8 +0, 200, 201, 1, 24, 0x50e80677, F=0x0, S=1, 8 +0, 201, 202, 1, 24, 0x55a806b7, F=0x0, S=1, 8 +0, 202, 203, 1, 24, 0x5a6806f7, F=0x0, S=1, 8 +0, 203, 204, 1, 24, 0x4c3c0638, F=0x0, S=1, 8 +0, 204, 205, 1, 24, 0x50fc0678, F=0x0, S=1, 8 +0, 205, 206, 1, 24, 0x55bc06b8, F=0x0, S=1, 8 +0, 206, 207, 1, 24, 0x5a7c06f8, F=0x0, S=1, 8 +0, 207, 208, 1, 24, 0x4c500639, F=0x0, S=1, 8 +0, 208, 209, 1, 24, 0x51100679, F=0x0, S=1, 8 +0, 209, 210, 1, 24, 0x55d006b9, F=0x0, S=1, 8 +0, 210, 211, 1, 24, 0x5a9006f9, F=0x0, S=1, 8 +0, 211, 212, 1, 24, 0x4c64063a, F=0x0, S=1, 8 +0, 212, 213, 1, 24, 0x5124067a, F=0x0, S=1, 8 +0, 213, 214, 1, 24, 0x55e406ba, F=0x0, S=1, 8 +0, 214, 215, 1, 24, 0x5aa406fa, F=0x0, S=1, 8 +0, 215, 216, 1, 24, 0x4c78063b, F=0x0, S=1, 8 +0, 216, 217, 1, 24, 0x5138067b, F=0x0, S=1, 8 +0, 217, 218, 1, 24, 0x55f806bb, F=0x0, S=1, 8 +0, 218, 219, 1, 24, 0x5ab806fb, F=0x0, S=1, 8 +0, 219, 220, 1, 24, 0x4c8c063c, F=0x0, S=1, 8 +0, 220, 221, 1, 24, 0x514c067c, F=0x0, S=1, 8 +0, 221, 222, 1, 24, 0x560c06bc, F=0x0, S=1, 8 +0, 222, 223, 1, 24, 0x5acc06fc, F=0x0, S=1, 8 +0, 223, 224, 1, 24, 0x4ca0063d, F=0x0, S=1, 8 +0, 224, 225, 1, 24, 0x5160067d, F=0x0, S=1, 8 +0, 225, 226, 1, 24, 0x562006bd, F=0x0, S=1, 8 +0, 226, 227, 1, 24, 0x5ae006fd, F=0x0, S=1, 8 +0, 227, 228, 1, 24, 0x4cb4063e, F=0x0, S=1, 8 +0, 228, 229, 1, 24, 0x5174067e, F=0x0, S=1, 8 +0, 229, 230, 1, 24, 0x563406be, F=0x0, S=1, 8 +0, 230, 231, 1, 24, 0x5af406fe, F=0x0, S=1, 8 +0, 231, 232, 1, 24, 0x4cc8063f, F=0x0, S=1, 8 +0, 232, 233, 1, 24, 0x5188067f, F=0x0, S=1, 8 +0, 233, 234, 1, 24, 0x564806bf, F=0x0, S=1, 8 +0, 234, 235, 1, 24, 0x5b0806ff, F=0x0, S=1, 8 +0, 235, 236, 1, 24, 0x4cdc0640, F=0x0, S=1, 8 +0, 236, 237, 1, 24, 0x519c0680, F=0x0, S=1, 8 +0, 237, 238, 1, 24, 0x565c06c0, F=0x0, S=1, 8 +0, 238, 239, 1, 24, 0x5b1c0700, F=0x0, S=1, 8 +0, 239, 240, 1, 24, 0x4cf00641, F=0x0, S=1, 8 +0, 240, 241, 1, 24, 0x51b00681, F=0x0, S=1, 8 +0, 241, 242, 1, 24, 0x567006c1, F=0x0, S=1, 8 +0, 242, 243, 1, 24, 0x5b300701, F=0x0, S=1, 8 +0, 243, 244, 1, 24, 0x4d040642, F=0x0, S=1, 8 +0, 244, 245, 1, 24, 0x51c40682, F=0x0, S=1, 8 +0, 245, 246, 1, 24, 0x568406c2, F=0x0, S=1, 8 +0, 246, 247, 1, 24, 0x5b440702, F=0x0, S=1, 8 +0, 247, 248, 1, 24, 0x4d180643, F=0x0, S=1, 8 +0, 248, 249, 1, 24, 0x51d80683, F=0x0, S=1, 8 +0, 249, 250, 1, 24, 0x569806c3, F=0x0, S=1, 8 +0, 250, 251, 1, 24, 0x5b580703, F=0x0, S=1, 8 +0, 251, 252, 1, 24, 0x4d2c0644, F=0x0, S=1, 8 +0, 252, 253, 1, 24, 0x51ec0684, F=0x0, S=1, 8 +0, 253, 254, 1, 24, 0x56ac06c4, F=0x0, S=1, 8 +0, 254, 255, 1, 24, 0x5b6c0704, F=0x0, S=1, 8 +0, 255, 256, 1, 24, 0x4d400645, F=0x0, S=1, 8 +0, 256, 257, 1, 24, 0x52000685, F=0x0, S=1, 8 +0, 257, 258, 1, 24, 0x56c006c5, F=0x0, S=1, 8 +0, 258, 259, 1, 24, 0x5b800705, F=0x0, S=1, 8 +0, 259, 260, 1, 24, 0x4d540646, F=0x0, S=1, 8 +0, 260, 261, 1, 24, 0x52140686, F=0x0, S=1, 8 +0, 261, 262, 1, 24, 0x56d406c6, F=0x0, S=1, 8 +0, 262, 263, 1, 24, 0x5b940706, F=0x0, S=1, 8 +0, 263, 264, 1, 24, 0x4d680647, F=0x0, S=1, 8 +0, 264, 265, 1, 24, 0x52280687, F=0x0, S=1, 8 +0, 265, 266, 1, 24, 0x56e806c7, F=0x0, S=1, 8 +0, 266, 267, 1, 24, 0x5ba80707, F=0x0, S=1, 8 +0, 267, 268, 1, 24, 0x4d7c0648, F=0x0, S=1, 8 +0, 268, 269, 1, 24, 0x523c0688, F=0x0, S=1, 8 +0, 269, 270, 1, 24, 0x56fc06c8, F=0x0, S=1, 8 +0, 270, 271, 1, 24, 0x5bbc0708, F=0x0, S=1, 8 +0, 271, 272, 1, 24, 0x4d900649, F=0x0, S=1, 8 +0, 272, 273, 1, 24, 0x52500689, F=0x0, S=1, 8 +0, 273, 274, 1, 24, 0x571006c9, F=0x0, S=1, 8 +0, 274, 275, 1, 24, 0x5bd00709, F=0x0, S=1, 8 +0, 275, 276, 1, 24, 0x4da4064a, F=0x0, S=1, 8 +0, 276, 277, 1, 24, 0x5264068a, F=0x0, S=1, 8 +0, 277, 278, 1, 24, 0x572406ca, F=0x0, S=1, 8 +0, 278, 279, 1, 24, 0x5be4070a, F=0x0, S=1, 8 +0, 279, 280, 1, 24, 0x4db8064b, F=0x0, S=1, 8 +0, 280, 281, 1, 24, 0x5278068b, F=0x0, S=1, 8 +0, 281, 282, 1, 24, 0x573806cb, F=0x0, S=1, 8 +0, 282, 283, 1, 24, 0x5bf8070b, F=0x0, S=1, 8 +0, 283, 284, 1, 24, 0x4dcc064c, F=0x0, S=1, 8 +0, 284, 285, 1, 24, 0x528c068c, F=0x0, S=1, 8 +0, 285, 286, 1, 24, 0x574c06cc, F=0x0, S=1, 8 +0, 286, 287, 1, 24, 0x5c0c070c, F=0x0, S=1, 8 +0, 287, 288, 1, 24, 0x4de0064d, F=0x0, S=1, 8 +0, 288, 289, 1, 24, 0x52a0068d, F=0x0, S=1, 8 +0, 289, 290, 1, 24, 0x576006cd, F=0x0, S=1, 8 +0, 290, 291, 1, 24, 0x5c20070d, F=0x0, S=1, 8 +0, 291, 292, 1, 24, 0x4df4064e, F=0x0, S=1, 8 +0, 292, 293, 1, 24, 0x52b4068e, F=0x0, S=1, 8 +0, 293, 294, 1, 24, 0x577406ce, F=0x0, S=1, 8 +0, 294, 295, 1, 24, 0x5c34070e, F=0x0, S=1, 8 +0, 295, 296, 1, 24, 0x4e08064f, F=0x0, S=1, 8 +0, 296, 297, 1, 24, 0x52c8068f, F=0x0, S=1, 8 +0, 297, 298, 1, 24, 0x578806cf, F=0x0, S=1, 8 +0, 298, 299, 1, 24, 0x5c48070f, F=0x0, S=1, 8 +0, 299, 300, 1, 24, 0x4e1c0650, F=0x0, S=1, 8 +0, 300, 301, 1, 24, 0x52dc0690, F=0x0, S=1, 8 +0, 301, 302, 1, 24, 0x579c06d0, F=0x0, S=1, 8 +0, 302, 303, 1, 24, 0x5c5c0710, F=0x0, S=1, 8 +0, 303, 304, 1, 24, 0x4e300651, F=0x0, S=1, 8 +0, 304, 305, 1, 24, 0x52f00691, F=0x0, S=1, 8 +0, 305, 306, 1, 24, 0x57b006d1, F=0x0, S=1, 8 +0, 306, 307, 1, 24, 0x5c700711, F=0x0, S=1, 8 +0, 307, 308, 1, 24, 0x4e440652, F=0x0, S=1, 8 +0, 308, 309, 1, 24, 0x53040692, F=0x0, S=1, 8 +0, 309, 310, 1, 24, 0x57c406d2, F=0x0, S=1, 8 +0, 310, 311, 1, 24, 0x5c840712, F=0x0, S=1, 8 +0, 311, 312, 1, 24, 0x4e580653, F=0x0, S=1, 8 +0, 312, 313, 1, 24, 0x53180693, F=0x0, S=1, 8 +0, 313, 314, 1, 24, 0x57d806d3, F=0x0, S=1, 8 +0, 314, 315, 1, 24, 0x5c980713, F=0x0, S=1, 8 +0, 315, 316, 1, 24, 0x4e6c0654, F=0x0, S=1, 8 +0, 316, 317, 1, 24, 0x532c0694, F=0x0, S=1, 8 +0, 317, 318, 1, 24, 0x57ec06d4, F=0x0, S=1, 8 +0, 318, 319, 1, 24, 0x5cac0714, F=0x0, S=1, 8 +0, 319, 320, 1, 24, 0x4e800655, F=0x0, S=1, 8 +0, 320, 321, 1, 24, 0x53400695, F=0x0, S=1, 8 +0, 321, 322, 1, 24, 0x580006d5, F=0x0, S=1, 8 +0, 322, 323, 1, 24, 0x5cc00715, F=0x0, S=1, 8 +0, 323, 324, 1, 24, 0x4e940656, F=0x0, S=1, 8 +0, 324, 325, 1, 24, 0x53540696, F=0x0, S=1, 8 +0, 325, 326, 1, 24, 0x581406d6, F=0x0, S=1, 8 +0, 326, 327, 1, 24, 0x5cd40716, F=0x0, S=1, 8 +0, 327, 328, 1, 24, 0x4ea80657, F=0x0, S=1, 8 +0, 328, 329, 1, 24, 0x53680697, F=0x0, S=1, 8 +0, 329, 330, 1, 24, 0x582806d7, F=0x0, S=1, 8 +0, 330, 331, 1, 24, 0x5ce80717, F=0x0, S=1, 8 +0, 331, 332, 1, 24, 0x4ebc0658, F=0x0, S=1, 8 +0, 332, 333, 1, 24, 0x537c0698, F=0x0, S=1, 8 +0, 333, 334, 1, 24, 0x583c06d8, F=0x0, S=1, 8 +0, 334, 335, 1, 24, 0x5cfc0718, F=0x0, S=1, 8 +0, 335, 336, 1, 24, 0x4ed00659, F=0x0, S=1, 8 +0, 336, 337, 1, 57, 0x899c0f1e, S=1, 8 +0, 337, 338, 1, 24, 0x4f1c0660, F=0x0, S=1, 8 +0, 338, 339, 1, 24, 0x53dc06a0, F=0x0, S=1, 8 +0, 339, 340, 1, 24, 0x589c06e0, F=0x0, S=1, 8 +0, 340, 341, 1, 24, 0x4a700621, F=0x0, S=1, 8 +0, 341, 342, 1, 24, 0x4f300661, F=0x0, S=1, 8 +0, 342, 343, 1, 24, 0x53f006a1, F=0x0, S=1, 8 +0, 343, 344, 1, 24, 0x58b006e1, F=0x0, S=1, 8 +0, 344, 345, 1, 24, 0x4a840622, F=0x0, S=1, 8 +0, 345, 346, 1, 24, 0x4f440662, F=0x0, S=1, 8 +0, 346, 347, 1, 24, 0x540406a2, F=0x0, S=1, 8 +0, 347, 348, 1, 24, 0x58c406e2, F=0x0, S=1, 8 +0, 348, 349, 1, 24, 0x4a980623, F=0x0, S=1, 8 +0, 349, 350, 1, 24, 0x4f580663, F=0x0, S=1, 8 +0, 350, 351, 1, 24, 0x541806a3, F=0x0, S=1, 8 +0, 351, 352, 1, 24, 0x58d806e3, F=0x0, S=1, 8 +0, 352, 353, 1, 24, 0x4aac0624, F=0x0, S=1, 8 +0, 353, 354, 1, 24, 0x4f6c0664, F=0x0, S=1, 8 +0, 354, 355, 1, 24, 0x542c06a4, F=0x0, S=1, 8 +0, 355, 356, 1, 24, 0x58ec06e4, F=0x0, S=1, 8 +0, 356, 357, 1, 24, 0x4ac00625, F=0x0, S=1, 8 +0, 357, 358, 1, 24, 0x4f800665, F=0x0, S=1, 8 +0, 358, 359, 1, 24, 0x544006a5, F=0x0, S=1, 8 +0, 359, 360, 1, 24, 0x590006e5, F=0x0, S=1, 8 +0, 360, 361, 1, 24, 0x4ad40626, F=0x0, S=1, 8 +0, 361, 362, 1, 24, 0x4f940666, F=0x0, S=1, 8 +0, 362, 363, 1, 24, 0x545406a6, F=0x0, S=1, 8 +0, 363, 364, 1, 24, 0x591406e6, F=0x0, S=1, 8 +0, 364, 365, 1, 24, 0x4ae80627, F=0x0, S=1, 8 +0, 365, 366, 1, 24, 0x4fa80667, F=0x0, S=1, 8 +0, 366, 367, 1, 24, 0x546806a7, F=0x0, S=1, 8 +0, 367, 368, 1, 24, 0x592806e7, F=0x0, S=1, 8 +0, 368, 369, 1, 57, 0x9b930fc1, S=1, 8 +0, 369, 370, 1, 24, 0x4f1c0660, F=0x0, S=1, 8 +0, 370, 371, 1, 24, 0x53dc06a0, F=0x0, S=1, 8 +0, 371, 372, 1, 24, 0x589c06e0, F=0x0, S=1, 8 +0, 372, 373, 1, 24, 0x4a700621, F=0x0, S=1, 8 +0, 373, 374, 1, 24, 0x4f300661, F=0x0, S=1, 8 +0, 374, 375, 1, 24, 0x53f006a1, F=0x0, S=1, 8 +0, 375, 376, 1, 24, 0x58b006e1, F=0x0, S=1, 8 +0, 376, 377, 1, 24, 0x4a840622, F=0x0, S=1, 8 +0, 377, 378, 1, 24, 0x4f440662, F=0x0, S=1, 8 +0, 378, 379, 1, 24, 0x540406a2, F=0x0, S=1, 8 +0, 379, 380, 1, 24, 0x58c406e2, F=0x0, S=1, 8 +0, 380, 381, 1, 24, 0x4a980623, F=0x0, S=1, 8 +0, 381, 382, 1, 24, 0x4f580663, F=0x0, S=1, 8 +0, 382, 383, 1, 24, 0x541806a3, F=0x0, S=1, 8 +0, 383, 384, 1, 24, 0x58d806e3, F=0x0, S=1, 8 +0, 384, 385, 1, 24, 0x4aac0624, F=0x0, S=1, 8 +0, 385, 386, 1, 24, 0x4f6c0664, F=0x0, S=1, 8 +0, 386, 387, 1, 24, 0x542c06a4, F=0x0, S=1, 8 +0, 387, 388, 1, 24, 0x58ec06e4, F=0x0, S=1, 8 +0, 388, 389, 1, 24, 0x4ac00625, F=0x0, S=1, 8 +0, 389, 390, 1, 24, 0x4f800665, F=0x0, S=1, 8 +0, 390, 391, 1, 24, 0x544006a5, F=0x0, S=1, 8 diff --git a/tests/ref/fate/force_key_frames-source-drop b/tests/ref/fate/force_key_frames-source-drop new file mode 100644 index 0000000000..99aa2ae826 --- /dev/null +++ b/tests/ref/fate/force_key_frames-source-drop @@ -0,0 +1,22 @@ +#tb 0: 1/1 +#media_type 0: video +#codec_id 0: mpeg2video +#dimensions 0: 2x2 +#sar 0: 0/1 +0, -1, 0, 1, 57, 0x80ba0ecd, S=1, 8 +0, 0, 1, 1, 24, 0x4f1c0660, F=0x0, S=1, 8 +0, 1, 2, 1, 24, 0x53dc06a0, F=0x0, S=1, 8 +0, 2, 3, 1, 24, 0x589c06e0, F=0x0, S=1, 8 +0, 3, 4, 1, 24, 0x4a700621, F=0x0, S=1, 8 +0, 4, 5, 1, 24, 0x4f300661, F=0x0, S=1, 8 +0, 5, 6, 1, 24, 0x53f006a1, F=0x0, S=1, 8 +0, 6, 7, 1, 24, 0x58b006e1, F=0x0, S=1, 8 +0, 7, 8, 1, 24, 0x4a840622, F=0x0, S=1, 8 +0, 8, 9, 1, 24, 0x4f440662, F=0x0, S=1, 8 +0, 9, 10, 1, 24, 0x540406a2, F=0x0, S=1, 8 +0, 10, 11, 1, 24, 0x58c406e2, F=0x0, S=1, 8 +0, 11, 12, 1, 24, 0x4a980623, F=0x0, S=1, 8 +0, 12, 13, 1, 24, 0x4f580663, F=0x0, S=1, 8 +0, 13, 14, 1, 24, 0x541806a3, F=0x0, S=1, 8 +0, 14, 15, 1, 24, 0x58d806e3, F=0x0, S=1, 8 +0, 15, 16, 1, 24, 0x4aac0624, F=0x0, S=1, 8 diff --git a/tests/ref/fate/force_key_frames-source-dup b/tests/ref/fate/force_key_frames-source-dup new file mode 100644 index 0000000000..4df86bdf4d --- /dev/null +++ b/tests/ref/fate/force_key_frames-source-dup @@ -0,0 +1,617 @@ +#tb 0: 1/39 +#media_type 0: video +#codec_id 0: mpeg2video +#dimensions 0: 2x2 +#sar 0: 0/1 +0, -1, 0, 1, 57, 0x8baa0f1a, S=1, 8 +0, 0, 1, 1, 24, 0x4f1c0660, F=0x0, S=1, 8 +0, 1, 2, 1, 24, 0x53dc06a0, F=0x0, S=1, 8 +0, 2, 3, 1, 24, 0x589c06e0, F=0x0, S=1, 8 +0, 3, 4, 1, 24, 0x4a700621, F=0x0, S=1, 8 +0, 4, 5, 1, 24, 0x4f300661, F=0x0, S=1, 8 +0, 5, 6, 1, 24, 0x53f006a1, F=0x0, S=1, 8 +0, 6, 7, 1, 24, 0x58b006e1, F=0x0, S=1, 8 +0, 7, 8, 1, 24, 0x4a840622, F=0x0, S=1, 8 +0, 8, 9, 1, 24, 0x4f440662, F=0x0, S=1, 8 +0, 9, 10, 1, 24, 0x540406a2, F=0x0, S=1, 8 +0, 10, 11, 1, 24, 0x58c406e2, F=0x0, S=1, 8 +0, 11, 12, 1, 24, 0x4a980623, F=0x0, S=1, 8 +0, 12, 13, 1, 24, 0x4f580663, F=0x0, S=1, 8 +0, 13, 14, 1, 24, 0x541806a3, F=0x0, S=1, 8 +0, 14, 15, 1, 24, 0x58d806e3, F=0x0, S=1, 8 +0, 15, 16, 1, 24, 0x4aac0624, F=0x0, S=1, 8 +0, 16, 17, 1, 24, 0x4f6c0664, F=0x0, S=1, 8 +0, 17, 18, 1, 24, 0x542c06a4, F=0x0, S=1, 8 +0, 18, 19, 1, 24, 0x58ec06e4, F=0x0, S=1, 8 +0, 19, 20, 1, 24, 0x4ac00625, F=0x0, S=1, 8 +0, 20, 21, 1, 24, 0x4f800665, F=0x0, S=1, 8 +0, 21, 22, 1, 24, 0x544006a5, F=0x0, S=1, 8 +0, 22, 23, 1, 24, 0x590006e5, F=0x0, S=1, 8 +0, 23, 24, 1, 24, 0x4ad40626, F=0x0, S=1, 8 +0, 24, 25, 1, 24, 0x4f940666, F=0x0, S=1, 8 +0, 25, 26, 1, 24, 0x545406a6, F=0x0, S=1, 8 +0, 26, 27, 1, 24, 0x591406e6, F=0x0, S=1, 8 +0, 27, 28, 1, 24, 0x4ae80627, F=0x0, S=1, 8 +0, 28, 29, 1, 24, 0x4fa80667, F=0x0, S=1, 8 +0, 29, 30, 1, 24, 0x546806a7, F=0x0, S=1, 8 +0, 30, 31, 1, 24, 0x592806e7, F=0x0, S=1, 8 +0, 31, 32, 1, 24, 0x4afc0628, F=0x0, S=1, 8 +0, 32, 33, 1, 24, 0x4fbc0668, F=0x0, S=1, 8 +0, 33, 34, 1, 24, 0x547c06a8, F=0x0, S=1, 8 +0, 34, 35, 1, 24, 0x593c06e8, F=0x0, S=1, 8 +0, 35, 36, 1, 24, 0x4b100629, F=0x0, S=1, 8 +0, 36, 37, 1, 24, 0x4fd00669, F=0x0, S=1, 8 +0, 37, 38, 1, 24, 0x549006a9, F=0x0, S=1, 8 +0, 38, 39, 1, 24, 0x595006e9, F=0x0, S=1, 8 +0, 39, 40, 1, 24, 0x4b24062a, F=0x0, S=1, 8 +0, 40, 41, 1, 24, 0x4fe4066a, F=0x0, S=1, 8 +0, 41, 42, 1, 24, 0x54a406aa, F=0x0, S=1, 8 +0, 42, 43, 1, 24, 0x596406ea, F=0x0, S=1, 8 +0, 43, 44, 1, 24, 0x4b38062b, F=0x0, S=1, 8 +0, 44, 45, 1, 24, 0x4ff8066b, F=0x0, S=1, 8 +0, 45, 46, 1, 24, 0x54b806ab, F=0x0, S=1, 8 +0, 46, 47, 1, 24, 0x597806eb, F=0x0, S=1, 8 +0, 47, 48, 1, 24, 0x4b4c062c, F=0x0, S=1, 8 +0, 48, 49, 1, 24, 0x500c066c, F=0x0, S=1, 8 +0, 49, 50, 1, 24, 0x54cc06ac, F=0x0, S=1, 8 +0, 50, 51, 1, 24, 0x598c06ec, F=0x0, S=1, 8 +0, 51, 52, 1, 24, 0x4b60062d, F=0x0, S=1, 8 +0, 52, 53, 1, 24, 0x5020066d, F=0x0, S=1, 8 +0, 53, 54, 1, 24, 0x54e006ad, F=0x0, S=1, 8 +0, 54, 55, 1, 24, 0x59a006ed, F=0x0, S=1, 8 +0, 55, 56, 1, 24, 0x4b74062e, F=0x0, S=1, 8 +0, 56, 57, 1, 24, 0x5034066e, F=0x0, S=1, 8 +0, 57, 58, 1, 24, 0x54f406ae, F=0x0, S=1, 8 +0, 58, 59, 1, 24, 0x59b406ee, F=0x0, S=1, 8 +0, 59, 60, 1, 24, 0x4b88062f, F=0x0, S=1, 8 +0, 60, 61, 1, 24, 0x5048066f, F=0x0, S=1, 8 +0, 61, 62, 1, 24, 0x550806af, F=0x0, S=1, 8 +0, 62, 63, 1, 24, 0x59c806ef, F=0x0, S=1, 8 +0, 63, 64, 1, 24, 0x4b9c0630, F=0x0, S=1, 8 +0, 64, 65, 1, 24, 0x505c0670, F=0x0, S=1, 8 +0, 65, 66, 1, 24, 0x551c06b0, F=0x0, S=1, 8 +0, 66, 67, 1, 24, 0x59dc06f0, F=0x0, S=1, 8 +0, 67, 68, 1, 24, 0x4bb00631, F=0x0, S=1, 8 +0, 68, 69, 1, 24, 0x50700671, F=0x0, S=1, 8 +0, 69, 70, 1, 24, 0x553006b1, F=0x0, S=1, 8 +0, 70, 71, 1, 24, 0x59f006f1, F=0x0, S=1, 8 +0, 71, 72, 1, 24, 0x4bc40632, F=0x0, S=1, 8 +0, 72, 73, 1, 24, 0x50840672, F=0x0, S=1, 8 +0, 73, 74, 1, 24, 0x554406b2, F=0x0, S=1, 8 +0, 74, 75, 1, 24, 0x5a0406f2, F=0x0, S=1, 8 +0, 75, 76, 1, 24, 0x4bd80633, F=0x0, S=1, 8 +0, 76, 77, 1, 24, 0x50980673, F=0x0, S=1, 8 +0, 77, 78, 1, 24, 0x555806b3, F=0x0, S=1, 8 +0, 78, 79, 1, 24, 0x5a1806f3, F=0x0, S=1, 8 +0, 79, 80, 1, 24, 0x4bec0634, F=0x0, S=1, 8 +0, 80, 81, 1, 24, 0x50ac0674, F=0x0, S=1, 8 +0, 81, 82, 1, 24, 0x556c06b4, F=0x0, S=1, 8 +0, 82, 83, 1, 24, 0x5a2c06f4, F=0x0, S=1, 8 +0, 83, 84, 1, 24, 0x4c000635, F=0x0, S=1, 8 +0, 84, 85, 1, 24, 0x50c00675, F=0x0, S=1, 8 +0, 85, 86, 1, 24, 0x558006b5, F=0x0, S=1, 8 +0, 86, 87, 1, 24, 0x5a4006f5, F=0x0, S=1, 8 +0, 87, 88, 1, 24, 0x4c140636, F=0x0, S=1, 8 +0, 88, 89, 1, 24, 0x50d40676, F=0x0, S=1, 8 +0, 89, 90, 1, 24, 0x559406b6, F=0x0, S=1, 8 +0, 90, 91, 1, 24, 0x5a5406f6, F=0x0, S=1, 8 +0, 91, 92, 1, 24, 0x4c280637, F=0x0, S=1, 8 +0, 92, 93, 1, 24, 0x50e80677, F=0x0, S=1, 8 +0, 93, 94, 1, 24, 0x55a806b7, F=0x0, S=1, 8 +0, 94, 95, 1, 24, 0x5a6806f7, F=0x0, S=1, 8 +0, 95, 96, 1, 24, 0x4c3c0638, F=0x0, S=1, 8 +0, 96, 97, 1, 24, 0x50fc0678, F=0x0, S=1, 8 +0, 97, 98, 1, 24, 0x55bc06b8, F=0x0, S=1, 8 +0, 98, 99, 1, 24, 0x5a7c06f8, F=0x0, S=1, 8 +0, 99, 100, 1, 24, 0x4c500639, F=0x0, S=1, 8 +0, 100, 101, 1, 24, 0x51100679, F=0x0, S=1, 8 +0, 101, 102, 1, 24, 0x55d006b9, F=0x0, S=1, 8 +0, 102, 103, 1, 24, 0x5a9006f9, F=0x0, S=1, 8 +0, 103, 104, 1, 24, 0x4c64063a, F=0x0, S=1, 8 +0, 104, 105, 1, 24, 0x5124067a, F=0x0, S=1, 8 +0, 105, 106, 1, 24, 0x55e406ba, F=0x0, S=1, 8 +0, 106, 107, 1, 24, 0x5aa406fa, F=0x0, S=1, 8 +0, 107, 108, 1, 24, 0x4c78063b, F=0x0, S=1, 8 +0, 108, 109, 1, 24, 0x5138067b, F=0x0, S=1, 8 +0, 109, 110, 1, 24, 0x55f806bb, F=0x0, S=1, 8 +0, 110, 111, 1, 24, 0x5ab806fb, F=0x0, S=1, 8 +0, 111, 112, 1, 24, 0x4c8c063c, F=0x0, S=1, 8 +0, 112, 113, 1, 24, 0x514c067c, F=0x0, S=1, 8 +0, 113, 114, 1, 24, 0x560c06bc, F=0x0, S=1, 8 +0, 114, 115, 1, 24, 0x5acc06fc, F=0x0, S=1, 8 +0, 115, 116, 1, 24, 0x4ca0063d, F=0x0, S=1, 8 +0, 116, 117, 1, 24, 0x5160067d, F=0x0, S=1, 8 +0, 117, 118, 1, 24, 0x562006bd, F=0x0, S=1, 8 +0, 118, 119, 1, 24, 0x5ae006fd, F=0x0, S=1, 8 +0, 119, 120, 1, 24, 0x4cb4063e, F=0x0, S=1, 8 +0, 120, 121, 1, 24, 0x5174067e, F=0x0, S=1, 8 +0, 121, 122, 1, 24, 0x563406be, F=0x0, S=1, 8 +0, 122, 123, 1, 24, 0x5af406fe, F=0x0, S=1, 8 +0, 123, 124, 1, 24, 0x4cc8063f, F=0x0, S=1, 8 +0, 124, 125, 1, 24, 0x5188067f, F=0x0, S=1, 8 +0, 125, 126, 1, 24, 0x564806bf, F=0x0, S=1, 8 +0, 126, 127, 1, 24, 0x5b0806ff, F=0x0, S=1, 8 +0, 127, 128, 1, 24, 0x4cdc0640, F=0x0, S=1, 8 +0, 128, 129, 1, 24, 0x519c0680, F=0x0, S=1, 8 +0, 129, 130, 1, 24, 0x565c06c0, F=0x0, S=1, 8 +0, 130, 131, 1, 24, 0x5b1c0700, F=0x0, S=1, 8 +0, 131, 132, 1, 24, 0x4cf00641, F=0x0, S=1, 8 +0, 132, 133, 1, 24, 0x51b00681, F=0x0, S=1, 8 +0, 133, 134, 1, 24, 0x567006c1, F=0x0, S=1, 8 +0, 134, 135, 1, 24, 0x5b300701, F=0x0, S=1, 8 +0, 135, 136, 1, 24, 0x4d040642, F=0x0, S=1, 8 +0, 136, 137, 1, 24, 0x51c40682, F=0x0, S=1, 8 +0, 137, 138, 1, 24, 0x568406c2, F=0x0, S=1, 8 +0, 138, 139, 1, 24, 0x5b440702, F=0x0, S=1, 8 +0, 139, 140, 1, 24, 0x4d180643, F=0x0, S=1, 8 +0, 140, 141, 1, 24, 0x51d80683, F=0x0, S=1, 8 +0, 141, 142, 1, 24, 0x569806c3, F=0x0, S=1, 8 +0, 142, 143, 1, 24, 0x5b580703, F=0x0, S=1, 8 +0, 143, 144, 1, 24, 0x4d2c0644, F=0x0, S=1, 8 +0, 144, 145, 1, 24, 0x51ec0684, F=0x0, S=1, 8 +0, 145, 146, 1, 24, 0x56ac06c4, F=0x0, S=1, 8 +0, 146, 147, 1, 24, 0x5b6c0704, F=0x0, S=1, 8 +0, 147, 148, 1, 24, 0x4d400645, F=0x0, S=1, 8 +0, 148, 149, 1, 24, 0x52000685, F=0x0, S=1, 8 +0, 149, 150, 1, 24, 0x56c006c5, F=0x0, S=1, 8 +0, 150, 151, 1, 24, 0x5b800705, F=0x0, S=1, 8 +0, 151, 152, 1, 24, 0x4d540646, F=0x0, S=1, 8 +0, 152, 153, 1, 24, 0x52140686, F=0x0, S=1, 8 +0, 153, 154, 1, 24, 0x56d406c6, F=0x0, S=1, 8 +0, 154, 155, 1, 24, 0x5b940706, F=0x0, S=1, 8 +0, 155, 156, 1, 24, 0x4d680647, F=0x0, S=1, 8 +0, 156, 157, 1, 24, 0x52280687, F=0x0, S=1, 8 +0, 157, 158, 1, 24, 0x56e806c7, F=0x0, S=1, 8 +0, 158, 159, 1, 24, 0x5ba80707, F=0x0, S=1, 8 +0, 159, 160, 1, 24, 0x4d7c0648, F=0x0, S=1, 8 +0, 160, 161, 1, 24, 0x523c0688, F=0x0, S=1, 8 +0, 161, 162, 1, 24, 0x56fc06c8, F=0x0, S=1, 8 +0, 162, 163, 1, 24, 0x5bbc0708, F=0x0, S=1, 8 +0, 163, 164, 1, 24, 0x4d900649, F=0x0, S=1, 8 +0, 164, 165, 1, 24, 0x52500689, F=0x0, S=1, 8 +0, 165, 166, 1, 24, 0x571006c9, F=0x0, S=1, 8 +0, 166, 167, 1, 24, 0x5bd00709, F=0x0, S=1, 8 +0, 167, 168, 1, 24, 0x4da4064a, F=0x0, S=1, 8 +0, 168, 169, 1, 57, 0xa5cf1003, S=1, 8 +0, 169, 170, 1, 24, 0x4f1c0660, F=0x0, S=1, 8 +0, 170, 171, 1, 24, 0x53dc06a0, F=0x0, S=1, 8 +0, 171, 172, 1, 24, 0x589c06e0, F=0x0, S=1, 8 +0, 172, 173, 1, 24, 0x4a700621, F=0x0, S=1, 8 +0, 173, 174, 1, 24, 0x4f300661, F=0x0, S=1, 8 +0, 174, 175, 1, 24, 0x53f006a1, F=0x0, S=1, 8 +0, 175, 176, 1, 24, 0x58b006e1, F=0x0, S=1, 8 +0, 176, 177, 1, 24, 0x4a840622, F=0x0, S=1, 8 +0, 177, 178, 1, 24, 0x4f440662, F=0x0, S=1, 8 +0, 178, 179, 1, 24, 0x540406a2, F=0x0, S=1, 8 +0, 179, 180, 1, 24, 0x58c406e2, F=0x0, S=1, 8 +0, 180, 181, 1, 24, 0x4a980623, F=0x0, S=1, 8 +0, 181, 182, 1, 24, 0x4f580663, F=0x0, S=1, 8 +0, 182, 183, 1, 24, 0x541806a3, F=0x0, S=1, 8 +0, 183, 184, 1, 24, 0x58d806e3, F=0x0, S=1, 8 +0, 184, 185, 1, 24, 0x4aac0624, F=0x0, S=1, 8 +0, 185, 186, 1, 24, 0x4f6c0664, F=0x0, S=1, 8 +0, 186, 187, 1, 24, 0x542c06a4, F=0x0, S=1, 8 +0, 187, 188, 1, 24, 0x58ec06e4, F=0x0, S=1, 8 +0, 188, 189, 1, 24, 0x4ac00625, F=0x0, S=1, 8 +0, 189, 190, 1, 24, 0x4f800665, F=0x0, S=1, 8 +0, 190, 191, 1, 24, 0x544006a5, F=0x0, S=1, 8 +0, 191, 192, 1, 24, 0x590006e5, F=0x0, S=1, 8 +0, 192, 193, 1, 24, 0x4ad40626, F=0x0, S=1, 8 +0, 193, 194, 1, 24, 0x4f940666, F=0x0, S=1, 8 +0, 194, 195, 1, 24, 0x545406a6, F=0x0, S=1, 8 +0, 195, 196, 1, 24, 0x591406e6, F=0x0, S=1, 8 +0, 196, 197, 1, 24, 0x4ae80627, F=0x0, S=1, 8 +0, 197, 198, 1, 24, 0x4fa80667, F=0x0, S=1, 8 +0, 198, 199, 1, 24, 0x546806a7, F=0x0, S=1, 8 +0, 199, 200, 1, 24, 0x592806e7, F=0x0, S=1, 8 +0, 200, 201, 1, 24, 0x4afc0628, F=0x0, S=1, 8 +0, 201, 202, 1, 24, 0x4fbc0668, F=0x0, S=1, 8 +0, 202, 203, 1, 24, 0x547c06a8, F=0x0, S=1, 8 +0, 203, 204, 1, 24, 0x593c06e8, F=0x0, S=1, 8 +0, 204, 205, 1, 24, 0x4b100629, F=0x0, S=1, 8 +0, 205, 206, 1, 24, 0x4fd00669, F=0x0, S=1, 8 +0, 206, 207, 1, 24, 0x549006a9, F=0x0, S=1, 8 +0, 207, 208, 1, 24, 0x595006e9, F=0x0, S=1, 8 +0, 208, 209, 1, 24, 0x4b24062a, F=0x0, S=1, 8 +0, 209, 210, 1, 24, 0x4fe4066a, F=0x0, S=1, 8 +0, 210, 211, 1, 24, 0x54a406aa, F=0x0, S=1, 8 +0, 211, 212, 1, 24, 0x596406ea, F=0x0, S=1, 8 +0, 212, 213, 1, 24, 0x4b38062b, F=0x0, S=1, 8 +0, 213, 214, 1, 24, 0x4ff8066b, F=0x0, S=1, 8 +0, 214, 215, 1, 24, 0x54b806ab, F=0x0, S=1, 8 +0, 215, 216, 1, 24, 0x597806eb, F=0x0, S=1, 8 +0, 216, 217, 1, 24, 0x4b4c062c, F=0x0, S=1, 8 +0, 217, 218, 1, 24, 0x500c066c, F=0x0, S=1, 8 +0, 218, 219, 1, 24, 0x54cc06ac, F=0x0, S=1, 8 +0, 219, 220, 1, 24, 0x598c06ec, F=0x0, S=1, 8 +0, 220, 221, 1, 24, 0x4b60062d, F=0x0, S=1, 8 +0, 221, 222, 1, 24, 0x5020066d, F=0x0, S=1, 8 +0, 222, 223, 1, 24, 0x54e006ad, F=0x0, S=1, 8 +0, 223, 224, 1, 24, 0x59a006ed, F=0x0, S=1, 8 +0, 224, 225, 1, 24, 0x4b74062e, F=0x0, S=1, 8 +0, 225, 226, 1, 24, 0x5034066e, F=0x0, S=1, 8 +0, 226, 227, 1, 24, 0x54f406ae, F=0x0, S=1, 8 +0, 227, 228, 1, 24, 0x59b406ee, F=0x0, S=1, 8 +0, 228, 229, 1, 24, 0x4b88062f, F=0x0, S=1, 8 +0, 229, 230, 1, 24, 0x5048066f, F=0x0, S=1, 8 +0, 230, 231, 1, 24, 0x550806af, F=0x0, S=1, 8 +0, 231, 232, 1, 24, 0x59c806ef, F=0x0, S=1, 8 +0, 232, 233, 1, 24, 0x4b9c0630, F=0x0, S=1, 8 +0, 233, 234, 1, 24, 0x505c0670, F=0x0, S=1, 8 +0, 234, 235, 1, 24, 0x551c06b0, F=0x0, S=1, 8 +0, 235, 236, 1, 24, 0x59dc06f0, F=0x0, S=1, 8 +0, 236, 237, 1, 24, 0x4bb00631, F=0x0, S=1, 8 +0, 237, 238, 1, 24, 0x50700671, F=0x0, S=1, 8 +0, 238, 239, 1, 24, 0x553006b1, F=0x0, S=1, 8 +0, 239, 240, 1, 24, 0x59f006f1, F=0x0, S=1, 8 +0, 240, 241, 1, 24, 0x4bc40632, F=0x0, S=1, 8 +0, 241, 242, 1, 24, 0x50840672, F=0x0, S=1, 8 +0, 242, 243, 1, 24, 0x554406b2, F=0x0, S=1, 8 +0, 243, 244, 1, 24, 0x5a0406f2, F=0x0, S=1, 8 +0, 244, 245, 1, 24, 0x4bd80633, F=0x0, S=1, 8 +0, 245, 246, 1, 24, 0x50980673, F=0x0, S=1, 8 +0, 246, 247, 1, 24, 0x555806b3, F=0x0, S=1, 8 +0, 247, 248, 1, 24, 0x5a1806f3, F=0x0, S=1, 8 +0, 248, 249, 1, 24, 0x4bec0634, F=0x0, S=1, 8 +0, 249, 250, 1, 24, 0x50ac0674, F=0x0, S=1, 8 +0, 250, 251, 1, 24, 0x556c06b4, F=0x0, S=1, 8 +0, 251, 252, 1, 24, 0x5a2c06f4, F=0x0, S=1, 8 +0, 252, 253, 1, 24, 0x4c000635, F=0x0, S=1, 8 +0, 253, 254, 1, 24, 0x50c00675, F=0x0, S=1, 8 +0, 254, 255, 1, 24, 0x558006b5, F=0x0, S=1, 8 +0, 255, 256, 1, 24, 0x5a4006f5, F=0x0, S=1, 8 +0, 256, 257, 1, 24, 0x4c140636, F=0x0, S=1, 8 +0, 257, 258, 1, 24, 0x50d40676, F=0x0, S=1, 8 +0, 258, 259, 1, 24, 0x559406b6, F=0x0, S=1, 8 +0, 259, 260, 1, 24, 0x5a5406f6, F=0x0, S=1, 8 +0, 260, 261, 1, 24, 0x4c280637, F=0x0, S=1, 8 +0, 261, 262, 1, 24, 0x50e80677, F=0x0, S=1, 8 +0, 262, 263, 1, 24, 0x55a806b7, F=0x0, S=1, 8 +0, 263, 264, 1, 24, 0x5a6806f7, F=0x0, S=1, 8 +0, 264, 265, 1, 24, 0x4c3c0638, F=0x0, S=1, 8 +0, 265, 266, 1, 24, 0x50fc0678, F=0x0, S=1, 8 +0, 266, 267, 1, 24, 0x55bc06b8, F=0x0, S=1, 8 +0, 267, 268, 1, 24, 0x5a7c06f8, F=0x0, S=1, 8 +0, 268, 269, 1, 24, 0x4c500639, F=0x0, S=1, 8 +0, 269, 270, 1, 24, 0x51100679, F=0x0, S=1, 8 +0, 270, 271, 1, 24, 0x55d006b9, F=0x0, S=1, 8 +0, 271, 272, 1, 24, 0x5a9006f9, F=0x0, S=1, 8 +0, 272, 273, 1, 24, 0x4c64063a, F=0x0, S=1, 8 +0, 273, 274, 1, 24, 0x5124067a, F=0x0, S=1, 8 +0, 274, 275, 1, 24, 0x55e406ba, F=0x0, S=1, 8 +0, 275, 276, 1, 24, 0x5aa406fa, F=0x0, S=1, 8 +0, 276, 277, 1, 24, 0x4c78063b, F=0x0, S=1, 8 +0, 277, 278, 1, 24, 0x5138067b, F=0x0, S=1, 8 +0, 278, 279, 1, 24, 0x55f806bb, F=0x0, S=1, 8 +0, 279, 280, 1, 24, 0x5ab806fb, F=0x0, S=1, 8 +0, 280, 281, 1, 24, 0x4c8c063c, F=0x0, S=1, 8 +0, 281, 282, 1, 24, 0x514c067c, F=0x0, S=1, 8 +0, 282, 283, 1, 24, 0x560c06bc, F=0x0, S=1, 8 +0, 283, 284, 1, 24, 0x5acc06fc, F=0x0, S=1, 8 +0, 284, 285, 1, 24, 0x4ca0063d, F=0x0, S=1, 8 +0, 285, 286, 1, 24, 0x5160067d, F=0x0, S=1, 8 +0, 286, 287, 1, 24, 0x562006bd, F=0x0, S=1, 8 +0, 287, 288, 1, 24, 0x5ae006fd, F=0x0, S=1, 8 +0, 288, 289, 1, 24, 0x4cb4063e, F=0x0, S=1, 8 +0, 289, 290, 1, 24, 0x5174067e, F=0x0, S=1, 8 +0, 290, 291, 1, 24, 0x563406be, F=0x0, S=1, 8 +0, 291, 292, 1, 24, 0x5af406fe, F=0x0, S=1, 8 +0, 292, 293, 1, 24, 0x4cc8063f, F=0x0, S=1, 8 +0, 293, 294, 1, 24, 0x5188067f, F=0x0, S=1, 8 +0, 294, 295, 1, 24, 0x564806bf, F=0x0, S=1, 8 +0, 295, 296, 1, 24, 0x5b0806ff, F=0x0, S=1, 8 +0, 296, 297, 1, 24, 0x4cdc0640, F=0x0, S=1, 8 +0, 297, 298, 1, 24, 0x519c0680, F=0x0, S=1, 8 +0, 298, 299, 1, 24, 0x565c06c0, F=0x0, S=1, 8 +0, 299, 300, 1, 24, 0x5b1c0700, F=0x0, S=1, 8 +0, 300, 301, 1, 24, 0x4cf00641, F=0x0, S=1, 8 +0, 301, 302, 1, 24, 0x51b00681, F=0x0, S=1, 8 +0, 302, 303, 1, 24, 0x567006c1, F=0x0, S=1, 8 +0, 303, 304, 1, 24, 0x5b300701, F=0x0, S=1, 8 +0, 304, 305, 1, 24, 0x4d040642, F=0x0, S=1, 8 +0, 305, 306, 1, 24, 0x51c40682, F=0x0, S=1, 8 +0, 306, 307, 1, 24, 0x568406c2, F=0x0, S=1, 8 +0, 307, 308, 1, 24, 0x5b440702, F=0x0, S=1, 8 +0, 308, 309, 1, 24, 0x4d180643, F=0x0, S=1, 8 +0, 309, 310, 1, 24, 0x51d80683, F=0x0, S=1, 8 +0, 310, 311, 1, 24, 0x569806c3, F=0x0, S=1, 8 +0, 311, 312, 1, 24, 0x5b580703, F=0x0, S=1, 8 +0, 312, 313, 1, 24, 0x4d2c0644, F=0x0, S=1, 8 +0, 313, 314, 1, 24, 0x51ec0684, F=0x0, S=1, 8 +0, 314, 315, 1, 24, 0x56ac06c4, F=0x0, S=1, 8 +0, 315, 316, 1, 24, 0x5b6c0704, F=0x0, S=1, 8 +0, 316, 317, 1, 24, 0x4d400645, F=0x0, S=1, 8 +0, 317, 318, 1, 24, 0x52000685, F=0x0, S=1, 8 +0, 318, 319, 1, 24, 0x56c006c5, F=0x0, S=1, 8 +0, 319, 320, 1, 24, 0x5b800705, F=0x0, S=1, 8 +0, 320, 321, 1, 24, 0x4d540646, F=0x0, S=1, 8 +0, 321, 322, 1, 24, 0x52140686, F=0x0, S=1, 8 +0, 322, 323, 1, 24, 0x56d406c6, F=0x0, S=1, 8 +0, 323, 324, 1, 24, 0x5b940706, F=0x0, S=1, 8 +0, 324, 325, 1, 24, 0x4d680647, F=0x0, S=1, 8 +0, 325, 326, 1, 24, 0x52280687, F=0x0, S=1, 8 +0, 326, 327, 1, 24, 0x56e806c7, F=0x0, S=1, 8 +0, 327, 328, 1, 24, 0x5ba80707, F=0x0, S=1, 8 +0, 328, 329, 1, 24, 0x4d7c0648, F=0x0, S=1, 8 +0, 329, 330, 1, 24, 0x523c0688, F=0x0, S=1, 8 +0, 330, 331, 1, 24, 0x56fc06c8, F=0x0, S=1, 8 +0, 331, 332, 1, 24, 0x5bbc0708, F=0x0, S=1, 8 +0, 332, 333, 1, 24, 0x4d900649, F=0x0, S=1, 8 +0, 333, 334, 1, 24, 0x52500689, F=0x0, S=1, 8 +0, 334, 335, 1, 24, 0x571006c9, F=0x0, S=1, 8 +0, 335, 336, 1, 24, 0x5bd00709, F=0x0, S=1, 8 +0, 336, 337, 1, 24, 0x4da4064a, F=0x0, S=1, 8 +0, 337, 338, 1, 24, 0x5264068a, F=0x0, S=1, 8 +0, 338, 339, 1, 24, 0x572406ca, F=0x0, S=1, 8 +0, 339, 340, 1, 24, 0x5be4070a, F=0x0, S=1, 8 +0, 340, 341, 1, 24, 0x4db8064b, F=0x0, S=1, 8 +0, 341, 342, 1, 24, 0x5278068b, F=0x0, S=1, 8 +0, 342, 343, 1, 24, 0x573806cb, F=0x0, S=1, 8 +0, 343, 344, 1, 24, 0x5bf8070b, F=0x0, S=1, 8 +0, 344, 345, 1, 24, 0x4dcc064c, F=0x0, S=1, 8 +0, 345, 346, 1, 24, 0x528c068c, F=0x0, S=1, 8 +0, 346, 347, 1, 24, 0x574c06cc, F=0x0, S=1, 8 +0, 347, 348, 1, 24, 0x5c0c070c, F=0x0, S=1, 8 +0, 348, 349, 1, 24, 0x4de0064d, F=0x0, S=1, 8 +0, 349, 350, 1, 24, 0x52a0068d, F=0x0, S=1, 8 +0, 350, 351, 1, 24, 0x576006cd, F=0x0, S=1, 8 +0, 351, 352, 1, 24, 0x5c20070d, F=0x0, S=1, 8 +0, 352, 353, 1, 24, 0x4df4064e, F=0x0, S=1, 8 +0, 353, 354, 1, 24, 0x52b4068e, F=0x0, S=1, 8 +0, 354, 355, 1, 24, 0x577406ce, F=0x0, S=1, 8 +0, 355, 356, 1, 24, 0x5c34070e, F=0x0, S=1, 8 +0, 356, 357, 1, 24, 0x4e08064f, F=0x0, S=1, 8 +0, 357, 358, 1, 24, 0x52c8068f, F=0x0, S=1, 8 +0, 358, 359, 1, 24, 0x578806cf, F=0x0, S=1, 8 +0, 359, 360, 1, 24, 0x5c48070f, F=0x0, S=1, 8 +0, 360, 361, 1, 24, 0x4e1c0650, F=0x0, S=1, 8 +0, 361, 362, 1, 24, 0x52dc0690, F=0x0, S=1, 8 +0, 362, 363, 1, 24, 0x579c06d0, F=0x0, S=1, 8 +0, 363, 364, 1, 24, 0x5c5c0710, F=0x0, S=1, 8 +0, 364, 365, 1, 24, 0x4e300651, F=0x0, S=1, 8 +0, 365, 366, 1, 24, 0x52f00691, F=0x0, S=1, 8 +0, 366, 367, 1, 24, 0x57b006d1, F=0x0, S=1, 8 +0, 367, 368, 1, 24, 0x5c700711, F=0x0, S=1, 8 +0, 368, 369, 1, 24, 0x4e440652, F=0x0, S=1, 8 +0, 369, 370, 1, 24, 0x53040692, F=0x0, S=1, 8 +0, 370, 371, 1, 24, 0x57c406d2, F=0x0, S=1, 8 +0, 371, 372, 1, 24, 0x5c840712, F=0x0, S=1, 8 +0, 372, 373, 1, 24, 0x4e580653, F=0x0, S=1, 8 +0, 373, 374, 1, 24, 0x53180693, F=0x0, S=1, 8 +0, 374, 375, 1, 24, 0x57d806d3, F=0x0, S=1, 8 +0, 375, 376, 1, 24, 0x5c980713, F=0x0, S=1, 8 +0, 376, 377, 1, 24, 0x4e6c0654, F=0x0, S=1, 8 +0, 377, 378, 1, 24, 0x532c0694, F=0x0, S=1, 8 +0, 378, 379, 1, 24, 0x57ec06d4, F=0x0, S=1, 8 +0, 379, 380, 1, 24, 0x5cac0714, F=0x0, S=1, 8 +0, 380, 381, 1, 24, 0x4e800655, F=0x0, S=1, 8 +0, 381, 382, 1, 24, 0x53400695, F=0x0, S=1, 8 +0, 382, 383, 1, 24, 0x580006d5, F=0x0, S=1, 8 +0, 383, 384, 1, 24, 0x5cc00715, F=0x0, S=1, 8 +0, 384, 385, 1, 24, 0x4e940656, F=0x0, S=1, 8 +0, 385, 386, 1, 24, 0x53540696, F=0x0, S=1, 8 +0, 386, 387, 1, 24, 0x581406d6, F=0x0, S=1, 8 +0, 387, 388, 1, 24, 0x5cd40716, F=0x0, S=1, 8 +0, 388, 389, 1, 24, 0x4ea80657, F=0x0, S=1, 8 +0, 389, 390, 1, 24, 0x53680697, F=0x0, S=1, 8 +0, 390, 391, 1, 24, 0x582806d7, F=0x0, S=1, 8 +0, 391, 392, 1, 24, 0x5ce80717, F=0x0, S=1, 8 +0, 392, 393, 1, 24, 0x4ebc0658, F=0x0, S=1, 8 +0, 393, 394, 1, 24, 0x537c0698, F=0x0, S=1, 8 +0, 394, 395, 1, 24, 0x583c06d8, F=0x0, S=1, 8 +0, 395, 396, 1, 24, 0x5cfc0718, F=0x0, S=1, 8 +0, 396, 397, 1, 24, 0x4ed00659, F=0x0, S=1, 8 +0, 397, 398, 1, 24, 0x53900699, F=0x0, S=1, 8 +0, 398, 399, 1, 24, 0x585006d9, F=0x0, S=1, 8 +0, 399, 400, 1, 24, 0x5d100719, F=0x0, S=1, 8 +0, 400, 401, 1, 24, 0x4ee4065a, F=0x0, S=1, 8 +0, 401, 402, 1, 24, 0x53a4069a, F=0x0, S=1, 8 +0, 402, 403, 1, 24, 0x586406da, F=0x0, S=1, 8 +0, 403, 404, 1, 24, 0x5d24071a, F=0x0, S=1, 8 +0, 404, 405, 1, 24, 0x4ef8065b, F=0x0, S=1, 8 +0, 405, 406, 1, 24, 0x53b8069b, F=0x0, S=1, 8 +0, 406, 407, 1, 24, 0x587806db, F=0x0, S=1, 8 +0, 407, 408, 1, 24, 0x5d38071b, F=0x0, S=1, 8 +0, 408, 409, 1, 24, 0x4f0c065c, F=0x0, S=1, 8 +0, 409, 410, 1, 24, 0x53cc069c, F=0x0, S=1, 8 +0, 410, 411, 1, 24, 0x588c06dc, F=0x0, S=1, 8 +0, 411, 412, 1, 24, 0x5d4c071c, F=0x0, S=1, 8 +0, 412, 413, 1, 24, 0x4f20065d, F=0x0, S=1, 8 +0, 413, 414, 1, 24, 0x53e0069d, F=0x0, S=1, 8 +0, 414, 415, 1, 24, 0x58a006dd, F=0x0, S=1, 8 +0, 415, 416, 1, 24, 0x5d60071d, F=0x0, S=1, 8 +0, 416, 417, 1, 24, 0x4f34065e, F=0x0, S=1, 8 +0, 417, 418, 1, 24, 0x53f4069e, F=0x0, S=1, 8 +0, 418, 419, 1, 24, 0x58b406de, F=0x0, S=1, 8 +0, 419, 420, 1, 24, 0x5d74071e, F=0x0, S=1, 8 +0, 420, 421, 1, 24, 0x4f48065f, F=0x0, S=1, 8 +0, 421, 422, 1, 24, 0x5408069f, F=0x0, S=1, 8 +0, 422, 423, 1, 24, 0x58c806df, F=0x0, S=1, 8 +0, 423, 424, 1, 24, 0x5d88071f, F=0x0, S=1, 8 +0, 424, 425, 1, 24, 0x4f5c0660, F=0x0, S=1, 8 +0, 425, 426, 1, 24, 0x541c06a0, F=0x0, S=1, 8 +0, 426, 427, 1, 24, 0x58dc06e0, F=0x0, S=1, 8 +0, 427, 428, 1, 24, 0x5d9c0720, F=0x0, S=1, 8 +0, 428, 429, 1, 24, 0x4f700661, F=0x0, S=1, 8 +0, 429, 430, 1, 24, 0x543006a1, F=0x0, S=1, 8 +0, 430, 431, 1, 24, 0x58f006e1, F=0x0, S=1, 8 +0, 431, 432, 1, 24, 0x5db00721, F=0x0, S=1, 8 +0, 432, 433, 1, 24, 0x4f840662, F=0x0, S=1, 8 +0, 433, 434, 1, 24, 0x544406a2, F=0x0, S=1, 8 +0, 434, 435, 1, 24, 0x590406e2, F=0x0, S=1, 8 +0, 435, 436, 1, 24, 0x5dc40722, F=0x0, S=1, 8 +0, 436, 437, 1, 24, 0x4f980663, F=0x0, S=1, 8 +0, 437, 438, 1, 24, 0x545806a3, F=0x0, S=1, 8 +0, 438, 439, 1, 24, 0x591806e3, F=0x0, S=1, 8 +0, 439, 440, 1, 24, 0x5dd80723, F=0x0, S=1, 8 +0, 440, 441, 1, 24, 0x4fac0664, F=0x0, S=1, 8 +0, 441, 442, 1, 24, 0x546c06a4, F=0x0, S=1, 8 +0, 442, 443, 1, 24, 0x592c06e4, F=0x0, S=1, 8 +0, 443, 444, 1, 24, 0x5dec0724, F=0x0, S=1, 8 +0, 444, 445, 1, 24, 0x4fc00665, F=0x0, S=1, 8 +0, 445, 446, 1, 24, 0x548006a5, F=0x0, S=1, 8 +0, 446, 447, 1, 24, 0x594006e5, F=0x0, S=1, 8 +0, 447, 448, 1, 24, 0x5e000725, F=0x0, S=1, 8 +0, 448, 449, 1, 24, 0x4fd40666, F=0x0, S=1, 8 +0, 449, 450, 1, 24, 0x549406a6, F=0x0, S=1, 8 +0, 450, 451, 1, 24, 0x595406e6, F=0x0, S=1, 8 +0, 451, 452, 1, 24, 0x5e140726, F=0x0, S=1, 8 +0, 452, 453, 1, 24, 0x4fe80667, F=0x0, S=1, 8 +0, 453, 454, 1, 24, 0x54a806a7, F=0x0, S=1, 8 +0, 454, 455, 1, 24, 0x596806e7, F=0x0, S=1, 8 +0, 455, 456, 1, 24, 0x5e280727, F=0x0, S=1, 8 +0, 456, 457, 1, 24, 0x4ffc0668, F=0x0, S=1, 8 +0, 457, 458, 1, 24, 0x54bc06a8, F=0x0, S=1, 8 +0, 458, 459, 1, 24, 0x597c06e8, F=0x0, S=1, 8 +0, 459, 460, 1, 24, 0x5e3c0728, F=0x0, S=1, 8 +0, 460, 461, 1, 24, 0x50100669, F=0x0, S=1, 8 +0, 461, 462, 1, 24, 0x54d006a9, F=0x0, S=1, 8 +0, 462, 463, 1, 24, 0x599006e9, F=0x0, S=1, 8 +0, 463, 464, 1, 24, 0x5e500729, F=0x0, S=1, 8 +0, 464, 465, 1, 24, 0x5024066a, F=0x0, S=1, 8 +0, 465, 466, 1, 24, 0x54e406aa, F=0x0, S=1, 8 +0, 466, 467, 1, 24, 0x59a406ea, F=0x0, S=1, 8 +0, 467, 468, 1, 24, 0x5e64072a, F=0x0, S=1, 8 +0, 468, 469, 1, 24, 0x5038066b, F=0x0, S=1, 8 +0, 469, 470, 1, 24, 0x54f806ab, F=0x0, S=1, 8 +0, 470, 471, 1, 24, 0x59b806eb, F=0x0, S=1, 8 +0, 471, 472, 1, 24, 0x5e78072b, F=0x0, S=1, 8 +0, 472, 473, 1, 24, 0x504c066c, F=0x0, S=1, 8 +0, 473, 474, 1, 24, 0x550c06ac, F=0x0, S=1, 8 +0, 474, 475, 1, 24, 0x59cc06ec, F=0x0, S=1, 8 +0, 475, 476, 1, 24, 0x5e8c072c, F=0x0, S=1, 8 +0, 476, 477, 1, 24, 0x5060066d, F=0x0, S=1, 8 +0, 477, 478, 1, 24, 0x552006ad, F=0x0, S=1, 8 +0, 478, 479, 1, 24, 0x59e006ed, F=0x0, S=1, 8 +0, 479, 480, 1, 24, 0x5ea0072d, F=0x0, S=1, 8 +0, 480, 481, 1, 24, 0x5074066e, F=0x0, S=1, 8 +0, 481, 482, 1, 24, 0x553406ae, F=0x0, S=1, 8 +0, 482, 483, 1, 24, 0x59f406ee, F=0x0, S=1, 8 +0, 483, 484, 1, 24, 0x5eb4072e, F=0x0, S=1, 8 +0, 484, 485, 1, 24, 0x5088066f, F=0x0, S=1, 8 +0, 485, 486, 1, 24, 0x554806af, F=0x0, S=1, 8 +0, 486, 487, 1, 24, 0x5a0806ef, F=0x0, S=1, 8 +0, 487, 488, 1, 24, 0x5ec8072f, F=0x0, S=1, 8 +0, 488, 489, 1, 24, 0x509c0670, F=0x0, S=1, 8 +0, 489, 490, 1, 24, 0x555c06b0, F=0x0, S=1, 8 +0, 490, 491, 1, 24, 0x5a1c06f0, F=0x0, S=1, 8 +0, 491, 492, 1, 24, 0x5edc0730, F=0x0, S=1, 8 +0, 492, 493, 1, 24, 0x50b00671, F=0x0, S=1, 8 +0, 493, 494, 1, 24, 0x557006b1, F=0x0, S=1, 8 +0, 494, 495, 1, 24, 0x5a3006f1, F=0x0, S=1, 8 +0, 495, 496, 1, 24, 0x5ef00731, F=0x0, S=1, 8 +0, 496, 497, 1, 24, 0x50c40672, F=0x0, S=1, 8 +0, 497, 498, 1, 24, 0x558406b2, F=0x0, S=1, 8 +0, 498, 499, 1, 24, 0x5a4406f2, F=0x0, S=1, 8 +0, 499, 500, 1, 24, 0x5f040732, F=0x0, S=1, 8 +0, 500, 501, 1, 24, 0x50d80673, F=0x0, S=1, 8 +0, 501, 502, 1, 24, 0x559806b3, F=0x0, S=1, 8 +0, 502, 503, 1, 24, 0x5a5806f3, F=0x0, S=1, 8 +0, 503, 504, 1, 24, 0x5f180733, F=0x0, S=1, 8 +0, 504, 505, 1, 24, 0x50ec0674, F=0x0, S=1, 8 +0, 505, 506, 1, 24, 0x55ac06b4, F=0x0, S=1, 8 +0, 506, 507, 1, 24, 0x5a6c06f4, F=0x0, S=1, 8 +0, 507, 508, 1, 24, 0x5f2c0734, F=0x0, S=1, 8 +0, 508, 509, 1, 24, 0x51000675, F=0x0, S=1, 8 +0, 509, 510, 1, 24, 0x55c006b5, F=0x0, S=1, 8 +0, 510, 511, 1, 24, 0x5a8006f5, F=0x0, S=1, 8 +0, 511, 512, 1, 24, 0x5f400735, F=0x0, S=1, 8 +0, 512, 513, 1, 24, 0x51140676, F=0x0, S=1, 8 +0, 513, 514, 1, 24, 0x55d406b6, F=0x0, S=1, 8 +0, 514, 515, 1, 24, 0x5a9406f6, F=0x0, S=1, 8 +0, 515, 516, 1, 24, 0x5f540736, F=0x0, S=1, 8 +0, 516, 517, 1, 24, 0x51280677, F=0x0, S=1, 8 +0, 517, 518, 1, 24, 0x55e806b7, F=0x0, S=1, 8 +0, 518, 519, 1, 24, 0x5aa806f7, F=0x0, S=1, 8 +0, 519, 520, 1, 24, 0x5f680737, F=0x0, S=1, 8 +0, 520, 521, 1, 24, 0x513c0678, F=0x0, S=1, 8 +0, 521, 522, 1, 24, 0x55fc06b8, F=0x0, S=1, 8 +0, 522, 523, 1, 24, 0x5abc06f8, F=0x0, S=1, 8 +0, 523, 524, 1, 24, 0x5f7c0738, F=0x0, S=1, 8 +0, 524, 525, 1, 24, 0x51500679, F=0x0, S=1, 8 +0, 525, 526, 1, 57, 0x896e0f04, S=1, 8 +0, 526, 527, 1, 24, 0x4f1c0660, F=0x0, S=1, 8 +0, 527, 528, 1, 24, 0x53dc06a0, F=0x0, S=1, 8 +0, 528, 529, 1, 24, 0x589c06e0, F=0x0, S=1, 8 +0, 529, 530, 1, 24, 0x4a700621, F=0x0, S=1, 8 +0, 530, 531, 1, 24, 0x4f300661, F=0x0, S=1, 8 +0, 531, 532, 1, 24, 0x53f006a1, F=0x0, S=1, 8 +0, 532, 533, 1, 24, 0x58b006e1, F=0x0, S=1, 8 +0, 533, 534, 1, 24, 0x4a840622, F=0x0, S=1, 8 +0, 534, 535, 1, 24, 0x4f440662, F=0x0, S=1, 8 +0, 535, 536, 1, 24, 0x540406a2, F=0x0, S=1, 8 +0, 536, 537, 1, 24, 0x58c406e2, F=0x0, S=1, 8 +0, 537, 538, 1, 24, 0x4a980623, F=0x0, S=1, 8 +0, 538, 539, 1, 24, 0x4f580663, F=0x0, S=1, 8 +0, 539, 540, 1, 24, 0x541806a3, F=0x0, S=1, 8 +0, 540, 541, 1, 24, 0x58d806e3, F=0x0, S=1, 8 +0, 541, 542, 1, 24, 0x4aac0624, F=0x0, S=1, 8 +0, 542, 543, 1, 24, 0x4f6c0664, F=0x0, S=1, 8 +0, 543, 544, 1, 24, 0x542c06a4, F=0x0, S=1, 8 +0, 544, 545, 1, 24, 0x58ec06e4, F=0x0, S=1, 8 +0, 545, 546, 1, 24, 0x4ac00625, F=0x0, S=1, 8 +0, 546, 547, 1, 24, 0x4f800665, F=0x0, S=1, 8 +0, 547, 548, 1, 24, 0x544006a5, F=0x0, S=1, 8 +0, 548, 549, 1, 24, 0x590006e5, F=0x0, S=1, 8 +0, 549, 550, 1, 24, 0x4ad40626, F=0x0, S=1, 8 +0, 550, 551, 1, 24, 0x4f940666, F=0x0, S=1, 8 +0, 551, 552, 1, 24, 0x545406a6, F=0x0, S=1, 8 +0, 552, 553, 1, 24, 0x591406e6, F=0x0, S=1, 8 +0, 553, 554, 1, 24, 0x4ae80627, F=0x0, S=1, 8 +0, 554, 555, 1, 24, 0x4fa80667, F=0x0, S=1, 8 +0, 555, 556, 1, 24, 0x546806a7, F=0x0, S=1, 8 +0, 556, 557, 1, 24, 0x592806e7, F=0x0, S=1, 8 +0, 557, 558, 1, 24, 0x4afc0628, F=0x0, S=1, 8 +0, 558, 559, 1, 24, 0x4fbc0668, F=0x0, S=1, 8 +0, 559, 560, 1, 24, 0x547c06a8, F=0x0, S=1, 8 +0, 560, 561, 1, 24, 0x593c06e8, F=0x0, S=1, 8 +0, 561, 562, 1, 24, 0x4b100629, F=0x0, S=1, 8 +0, 562, 563, 1, 24, 0x4fd00669, F=0x0, S=1, 8 +0, 563, 564, 1, 24, 0x549006a9, F=0x0, S=1, 8 +0, 564, 565, 1, 24, 0x595006e9, F=0x0, S=1, 8 +0, 565, 566, 1, 24, 0x4b24062a, F=0x0, S=1, 8 +0, 566, 567, 1, 24, 0x4fe4066a, F=0x0, S=1, 8 +0, 567, 568, 1, 24, 0x54a406aa, F=0x0, S=1, 8 +0, 568, 569, 1, 24, 0x596406ea, F=0x0, S=1, 8 +0, 569, 570, 1, 24, 0x4b38062b, F=0x0, S=1, 8 +0, 570, 571, 1, 24, 0x4ff8066b, F=0x0, S=1, 8 +0, 571, 572, 1, 24, 0x54b806ab, F=0x0, S=1, 8 +0, 572, 573, 1, 24, 0x597806eb, F=0x0, S=1, 8 +0, 573, 574, 1, 24, 0x4b4c062c, F=0x0, S=1, 8 +0, 574, 575, 1, 24, 0x500c066c, F=0x0, S=1, 8 +0, 575, 576, 1, 57, 0x901d0f3f, S=1, 8 +0, 576, 577, 1, 24, 0x4f1c0660, F=0x0, S=1, 8 +0, 577, 578, 1, 24, 0x53dc06a0, F=0x0, S=1, 8 +0, 578, 579, 1, 24, 0x589c06e0, F=0x0, S=1, 8 +0, 579, 580, 1, 24, 0x4a700621, F=0x0, S=1, 8 +0, 580, 581, 1, 24, 0x4f300661, F=0x0, S=1, 8 +0, 581, 582, 1, 24, 0x53f006a1, F=0x0, S=1, 8 +0, 582, 583, 1, 24, 0x58b006e1, F=0x0, S=1, 8 +0, 583, 584, 1, 24, 0x4a840622, F=0x0, S=1, 8 +0, 584, 585, 1, 24, 0x4f440662, F=0x0, S=1, 8 +0, 585, 586, 1, 24, 0x540406a2, F=0x0, S=1, 8 +0, 586, 587, 1, 24, 0x58c406e2, F=0x0, S=1, 8 +0, 587, 588, 1, 24, 0x4a980623, F=0x0, S=1, 8 +0, 588, 589, 1, 24, 0x4f580663, F=0x0, S=1, 8 +0, 589, 590, 1, 24, 0x541806a3, F=0x0, S=1, 8 +0, 590, 591, 1, 24, 0x58d806e3, F=0x0, S=1, 8 +0, 591, 592, 1, 24, 0x4aac0624, F=0x0, S=1, 8 +0, 592, 593, 1, 24, 0x4f6c0664, F=0x0, S=1, 8 +0, 593, 594, 1, 24, 0x542c06a4, F=0x0, S=1, 8 +0, 594, 595, 1, 24, 0x58ec06e4, F=0x0, S=1, 8 +0, 595, 596, 1, 24, 0x4ac00625, F=0x0, S=1, 8 +0, 596, 597, 1, 24, 0x4f800665, F=0x0, S=1, 8 +0, 597, 598, 1, 24, 0x544006a5, F=0x0, S=1, 8 +0, 598, 599, 1, 24, 0x590006e5, F=0x0, S=1, 8 +0, 599, 600, 1, 24, 0x4ad40626, F=0x0, S=1, 8 +0, 600, 601, 1, 24, 0x4f940666, F=0x0, S=1, 8 +0, 601, 602, 1, 24, 0x545406a6, F=0x0, S=1, 8 +0, 602, 603, 1, 24, 0x591406e6, F=0x0, S=1, 8 +0, 603, 604, 1, 24, 0x4ae80627, F=0x0, S=1, 8 +0, 604, 605, 1, 24, 0x4fa80667, F=0x0, S=1, 8 +0, 605, 606, 1, 24, 0x546806a7, F=0x0, S=1, 8 +0, 606, 607, 1, 24, 0x592806e7, F=0x0, S=1, 8 +0, 607, 608, 1, 24, 0x4afc0628, F=0x0, S=1, 8 +0, 608, 609, 1, 24, 0x4fbc0668, F=0x0, S=1, 8 +0, 609, 610, 1, 24, 0x547c06a8, F=0x0, S=1, 8 +0, 610, 611, 1, 24, 0x593c06e8, F=0x0, S=1, 8 -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 07/27] fftools/ffmpeg_enc: unbreak -force_key_frames source_no_drop 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (5 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 06/27] tests/fate/ffmpeg: add tests for -force_key_frames source Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 08/27] fftools/ffmpeg_enc: merge -force_key_frames source/source_no_drop Anton Khirnov ` (20 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel Unlike the 'source' mode, which preserves source keyframe-marking as-is, the 'source_no_drop' mode attempts to keep track of keyframes dropped by framerate conversion and mark the next output frame as key in such cases. However, * c75be061487 broke this functionality entirely, and made it equivalent to 'source' * even before it would only work when the frame immediately following the dropped keyframe is preserved and not dropped as well Also, drop a redundant check for 'frame' in setting dropped_keyframe, as it is redundant with the check on the above line. --- fftools/ffmpeg_enc.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index df79eaff59..1db67d1497 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -1066,7 +1066,7 @@ finish: } ost->last_dropped = *nb_frames == *nb_frames_prev && frame; - ost->kf.dropped_keyframe = ost->last_dropped && frame && (frame->flags & AV_FRAME_FLAG_KEY); + ost->kf.dropped_keyframe |= ost->last_dropped && (frame->flags & AV_FRAME_FLAG_KEY); } static enum AVPictureType forced_kf_apply(void *logctx, KeyframeForceCtx *kf, @@ -1109,8 +1109,9 @@ static enum AVPictureType forced_kf_apply(void *logctx, KeyframeForceCtx *kf, (in_picture->flags & AV_FRAME_FLAG_KEY) && !dup_idx) { goto force_keyframe; } else if (kf->type == KF_FORCE_SOURCE_NO_DROP && !dup_idx) { + int dropped_keyframe = kf->dropped_keyframe; kf->dropped_keyframe = 0; - if ((in_picture->flags & AV_FRAME_FLAG_KEY) || kf->dropped_keyframe) + if ((in_picture->flags & AV_FRAME_FLAG_KEY) || dropped_keyframe) goto force_keyframe; } -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 08/27] fftools/ffmpeg_enc: merge -force_key_frames source/source_no_drop 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (6 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 07/27] fftools/ffmpeg_enc: unbreak -force_key_frames source_no_drop Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 09/27] fftools/ffmpeg: stop accessing OutputStream.last_dropped in print_report() Anton Khirnov ` (19 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel Always use the functionality of the latter, which makes more sense as it avoids losing keyframes due to vsync code dropping frames. Deprecate the 'source_no_drop' value, as it is now redundant. --- doc/ffmpeg.texi | 5 ----- fftools/ffmpeg.h | 3 +++ fftools/ffmpeg_enc.c | 5 +---- fftools/ffmpeg_mux_init.c | 6 +++++- tests/ref/fate/force_key_frames-source-drop | 22 ++++++++++----------- 5 files changed, 20 insertions(+), 21 deletions(-) diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi index d2864ff37e..ea473e14e8 100644 --- a/doc/ffmpeg.texi +++ b/doc/ffmpeg.texi @@ -1060,7 +1060,6 @@ Deprecated see -bsf @item -force_key_frames[:@var{stream_specifier}] @var{time}[,@var{time}...] (@emph{output,per-stream}) @item -force_key_frames[:@var{stream_specifier}] expr:@var{expr} (@emph{output,per-stream}) @item -force_key_frames[:@var{stream_specifier}] source (@emph{output,per-stream}) -@item -force_key_frames[:@var{stream_specifier}] source_no_drop (@emph{output,per-stream}) @var{force_key_frames} can take arguments of the following form: @@ -1121,10 +1120,6 @@ starting from second 13: @item source If the argument is @code{source}, ffmpeg will force a key frame if the current frame being encoded is marked as a key frame in its source. - -@item source_no_drop -If the argument is @code{source_no_drop}, ffmpeg will force a key frame if -the current frame being encoded is marked as a key frame in its source. In cases where this particular source frame has to be dropped, enforce the next available frame to become a key frame instead. diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 2e8f1db9b6..eaa663e718 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -58,6 +58,7 @@ #define FFMPEG_OPT_ADRIFT_THRESHOLD 1 #define FFMPEG_OPT_ENC_TIME_BASE_NUM 1 #define FFMPEG_OPT_TOP 1 +#define FFMPEG_OPT_FORCE_KF_SOURCE_NO_DROP 1 enum VideoSyncMethod { VSYNC_AUTO = -1, @@ -484,7 +485,9 @@ typedef enum { enum { KF_FORCE_SOURCE = 1, +#if FFMPEG_OPT_FORCE_KF_SOURCE_NO_DROP KF_FORCE_SOURCE_NO_DROP = 2, +#endif }; typedef struct KeyframeForceCtx { diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index 1db67d1497..0ad3a7b6b1 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -1105,10 +1105,7 @@ static enum AVPictureType forced_kf_apply(void *logctx, KeyframeForceCtx *kf, kf->expr_const_values[FKF_N_FORCED] += 1; goto force_keyframe; } - } else if (kf->type == KF_FORCE_SOURCE && - (in_picture->flags & AV_FRAME_FLAG_KEY) && !dup_idx) { - goto force_keyframe; - } else if (kf->type == KF_FORCE_SOURCE_NO_DROP && !dup_idx) { + } else if (kf->type == KF_FORCE_SOURCE && !dup_idx) { int dropped_keyframe = kf->dropped_keyframe; kf->dropped_keyframe = 0; if ((in_picture->flags & AV_FRAME_FLAG_KEY) || dropped_keyframe) diff --git a/fftools/ffmpeg_mux_init.c b/fftools/ffmpeg_mux_init.c index 9d6f442068..f35680e355 100644 --- a/fftools/ffmpeg_mux_init.c +++ b/fftools/ffmpeg_mux_init.c @@ -2509,8 +2509,12 @@ static int process_forced_keyframes(Muxer *mux, const OptionsContext *o) // parse it only for static kf timings } else if (!strcmp(forced_keyframes, "source")) { ost->kf.type = KF_FORCE_SOURCE; +#if FFMPEG_OPT_FORCE_KF_SOURCE_NO_DROP } else if (!strcmp(forced_keyframes, "source_no_drop")) { - ost->kf.type = KF_FORCE_SOURCE_NO_DROP; + av_log(ost, AV_LOG_WARNING, "The 'source_no_drop' value for " + "-force_key_frames is deprecated, use just 'source'\n"); + ost->kf.type = KF_FORCE_SOURCE; +#endif } else { int ret = parse_forced_key_frames(ost, &ost->kf, mux, forced_keyframes); if (ret < 0) diff --git a/tests/ref/fate/force_key_frames-source-drop b/tests/ref/fate/force_key_frames-source-drop index 99aa2ae826..220c0f0f88 100644 --- a/tests/ref/fate/force_key_frames-source-drop +++ b/tests/ref/fate/force_key_frames-source-drop @@ -9,14 +9,14 @@ 0, 2, 3, 1, 24, 0x589c06e0, F=0x0, S=1, 8 0, 3, 4, 1, 24, 0x4a700621, F=0x0, S=1, 8 0, 4, 5, 1, 24, 0x4f300661, F=0x0, S=1, 8 -0, 5, 6, 1, 24, 0x53f006a1, F=0x0, S=1, 8 -0, 6, 7, 1, 24, 0x58b006e1, F=0x0, S=1, 8 -0, 7, 8, 1, 24, 0x4a840622, F=0x0, S=1, 8 -0, 8, 9, 1, 24, 0x4f440662, F=0x0, S=1, 8 -0, 9, 10, 1, 24, 0x540406a2, F=0x0, S=1, 8 -0, 10, 11, 1, 24, 0x58c406e2, F=0x0, S=1, 8 -0, 11, 12, 1, 24, 0x4a980623, F=0x0, S=1, 8 -0, 12, 13, 1, 24, 0x4f580663, F=0x0, S=1, 8 -0, 13, 14, 1, 24, 0x541806a3, F=0x0, S=1, 8 -0, 14, 15, 1, 24, 0x58d806e3, F=0x0, S=1, 8 -0, 15, 16, 1, 24, 0x4aac0624, F=0x0, S=1, 8 +0, 5, 6, 1, 57, 0x7a110e90, S=1, 8 +0, 6, 7, 1, 24, 0x4f1c0660, F=0x0, S=1, 8 +0, 7, 8, 1, 24, 0x53dc06a0, F=0x0, S=1, 8 +0, 8, 9, 1, 24, 0x589c06e0, F=0x0, S=1, 8 +0, 9, 10, 1, 24, 0x4a700621, F=0x0, S=1, 8 +0, 10, 11, 1, 24, 0x4f300661, F=0x0, S=1, 8 +0, 11, 12, 1, 24, 0x53f006a1, F=0x0, S=1, 8 +0, 12, 13, 1, 24, 0x58b006e1, F=0x0, S=1, 8 +0, 13, 14, 1, 24, 0x4a840622, F=0x0, S=1, 8 +0, 14, 15, 1, 57, 0x88850f14, S=1, 8 +0, 15, 16, 1, 57, 0x7aa20e95, S=1, 8 -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 09/27] fftools/ffmpeg: stop accessing OutputStream.last_dropped in print_report() 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (7 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 08/27] fftools/ffmpeg_enc: merge -force_key_frames source/source_no_drop Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 10/27] fftools/ffmpeg_enc: move framerate conversion state into a separate struct Anton Khirnov ` (18 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel That field is used by the framerate code to track whether any output has been generated for the last input frame(*). Its use in the last invocation of print_report() is meant to meant to account (in the number of dropped frames printed in the log) for the very last filtered frame being dropped. However, that is a highly inappropriate place to do so, as it makes assumptions about vsync logic in completely unrelated code. Move the increment to encoder flush instead. (*) the name is misleading, as the input frame has not yet been dropped and may still be output in the future --- fftools/ffmpeg.c | 3 --- fftools/ffmpeg_enc.c | 3 +++ 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 14f55cbec7..a854589bef 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -566,9 +566,6 @@ static void print_report(int is_last_report, int64_t timer_start, int64_t cur_ti pts -= copy_ts_first_pts; } } - - if (is_last_report) - nb_frames_drop += ost->last_dropped; } us = FFABS64U(pts) % AV_TIME_BASE; diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index 0ad3a7b6b1..2250d22bd3 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -761,6 +761,9 @@ static int encode_frame(OutputFile *of, OutputStream *ost, AVFrame *frame) if (frame->sample_aspect_ratio.num && !ost->frame_aspect_ratio.num) enc->sample_aspect_ratio = frame->sample_aspect_ratio; + } else if (ost->last_dropped) { + ost->nb_frames_drop++; + ost->last_dropped = 0; } update_benchmark(NULL); -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 10/27] fftools/ffmpeg_enc: move framerate conversion state into a separate struct 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (8 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 09/27] fftools/ffmpeg: stop accessing OutputStream.last_dropped in print_report() Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 11/27] fftools/ffmpeg_enc: move fps conversion code to ffmpeg_filter Anton Khirnov ` (17 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel Makes it more clear what state is specific to framerate conversion, which will be useful in the following commit. --- fftools/ffmpeg_enc.c | 65 ++++++++++++++++++++++++-------------------- 1 file changed, 35 insertions(+), 30 deletions(-) diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index 2250d22bd3..9aee18bfe1 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -41,18 +41,24 @@ #include "libavformat/avformat.h" -struct Encoder { - /* predicted pts of the next frame to be encoded */ - int64_t next_pts; - +typedef struct FPSConvContext { AVFrame *last_frame; /* number of frames emitted by the video-encoding sync code */ - int64_t vsync_frame_number; + int64_t frame_number; /* history of nb_frames_prev, i.e. the number of times the * previous frame was duplicated by vsync code in recent * do_video_out() calls */ int64_t frames_prev_hist[3]; + uint64_t dup_warning; +} FPSConvContext; + +struct Encoder { + /* predicted pts of the next frame to be encoded */ + int64_t next_pts; + + FPSConvContext fps; + AVFrame *sq_frame; // packet for receiving encoded output @@ -64,8 +70,6 @@ struct Encoder { // number of packets received from the encoder uint64_t packets_encoded; - uint64_t dup_warning; - int opened; }; @@ -76,7 +80,7 @@ void enc_free(Encoder **penc) if (!enc) return; - av_frame_free(&enc->last_frame); + av_frame_free(&enc->fps.last_frame); av_frame_free(&enc->sq_frame); av_packet_free(&enc->pkt); @@ -95,17 +99,17 @@ int enc_alloc(Encoder **penc, const AVCodec *codec) return AVERROR(ENOMEM); if (codec->type == AVMEDIA_TYPE_VIDEO) { - enc->last_frame = av_frame_alloc(); - if (!enc->last_frame) + enc->fps.last_frame = av_frame_alloc(); + if (!enc->fps.last_frame) goto fail; + + enc->fps.dup_warning = 1000; } enc->pkt = av_packet_alloc(); if (!enc->pkt) goto fail; - enc->dup_warning = 1000; - *penc = enc; return 0; @@ -966,13 +970,14 @@ static void video_sync_process(OutputFile *of, OutputStream *ost, AVFrame *frame int64_t *nb_frames, int64_t *nb_frames_prev) { Encoder *e = ost->enc; + FPSConvContext *fps = &e->fps; AVCodecContext *enc = ost->enc_ctx; double delta0, delta, sync_ipts, duration; if (!frame) { - *nb_frames_prev = *nb_frames = mid_pred(e->frames_prev_hist[0], - e->frames_prev_hist[1], - e->frames_prev_hist[2]); + *nb_frames_prev = *nb_frames = mid_pred(fps->frames_prev_hist[0], + fps->frames_prev_hist[1], + fps->frames_prev_hist[2]); goto finish; } @@ -1006,7 +1011,7 @@ static void video_sync_process(OutputFile *of, OutputStream *ost, AVFrame *frame switch (ost->vsync_method) { case VSYNC_VSCFR: - if (e->vsync_frame_number == 0 && delta0 >= 0.5) { + if (fps->frame_number == 0 && delta0 >= 0.5) { av_log(ost, AV_LOG_DEBUG, "Not duplicating %d initial frames\n", (int)lrintf(delta0)); delta = duration; delta0 = 0; @@ -1014,7 +1019,7 @@ static void video_sync_process(OutputFile *of, OutputStream *ost, AVFrame *frame } case VSYNC_CFR: // FIXME set to 0.5 after we fix some dts/pts bugs like in avidec.c - if (frame_drop_threshold && delta < frame_drop_threshold && e->vsync_frame_number) { + if (frame_drop_threshold && delta < frame_drop_threshold && fps->frame_number) { *nb_frames = 0; } else if (delta < -1.1) *nb_frames = 0; @@ -1042,16 +1047,16 @@ static void video_sync_process(OutputFile *of, OutputStream *ost, AVFrame *frame } finish: - memmove(e->frames_prev_hist + 1, - e->frames_prev_hist, - sizeof(e->frames_prev_hist[0]) * (FF_ARRAY_ELEMS(e->frames_prev_hist) - 1)); - e->frames_prev_hist[0] = *nb_frames_prev; + memmove(fps->frames_prev_hist + 1, + fps->frames_prev_hist, + sizeof(fps->frames_prev_hist[0]) * (FF_ARRAY_ELEMS(fps->frames_prev_hist) - 1)); + fps->frames_prev_hist[0] = *nb_frames_prev; if (*nb_frames_prev == 0 && ost->last_dropped) { ost->nb_frames_drop++; av_log(ost, AV_LOG_VERBOSE, "*** dropping frame %"PRId64" at ts %"PRId64"\n", - e->vsync_frame_number, e->last_frame->pts); + fps->frame_number, fps->last_frame->pts); } if (*nb_frames > (*nb_frames_prev && ost->last_dropped) + (*nb_frames > *nb_frames_prev)) { if (*nb_frames > dts_error_threshold * 30) { @@ -1062,9 +1067,9 @@ finish: } ost->nb_frames_dup += *nb_frames - (*nb_frames_prev && ost->last_dropped) - (*nb_frames > *nb_frames_prev); av_log(ost, AV_LOG_VERBOSE, "*** %"PRId64" dup!\n", *nb_frames - 1); - if (ost->nb_frames_dup > e->dup_warning) { - av_log(ost, AV_LOG_WARNING, "More than %"PRIu64" frames duplicated\n", e->dup_warning); - e->dup_warning *= 10; + if (ost->nb_frames_dup > fps->dup_warning) { + av_log(ost, AV_LOG_WARNING, "More than %"PRIu64" frames duplicated\n", fps->dup_warning); + fps->dup_warning *= 10; } } @@ -1137,8 +1142,8 @@ static int do_video_out(OutputFile *of, OutputStream *ost, AVFrame *frame) for (i = 0; i < nb_frames; i++) { AVFrame *in_picture; - if (i < nb_frames_prev && e->last_frame->buf[0]) { - in_picture = e->last_frame; + if (i < nb_frames_prev && e->fps.last_frame->buf[0]) { + in_picture = e->fps.last_frame; } else in_picture = frame; @@ -1167,12 +1172,12 @@ static int do_video_out(OutputFile *of, OutputStream *ost, AVFrame *frame) return ret; e->next_pts++; - e->vsync_frame_number++; + e->fps.frame_number++; } - av_frame_unref(e->last_frame); + av_frame_unref(e->fps.last_frame); if (frame) - av_frame_move_ref(e->last_frame, frame); + av_frame_move_ref(e->fps.last_frame, frame); return 0; } -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 11/27] fftools/ffmpeg_enc: move fps conversion code to ffmpeg_filter 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (9 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 10/27] fftools/ffmpeg_enc: move framerate conversion state into a separate struct Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 12/27] fftools/ffmpeg_filter: fail on filtering errors Anton Khirnov ` (16 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel Its function is analogous to that of the fps filter, so filtering is a more appropriate place for this. The main practical reason for this move is that it places the encoding sync queue right at the boundary between filters and encoders. This will be important when switching to threaded scheduling, as the sync queue involves multiple streams and will thus need to do nontrivial inter-thread synchronization. In addition to framerate conversion, the closely-related * encoder timebase selection * applying the start_time offset are also moved to filtering. --- fftools/ffmpeg.c | 6 +- fftools/ffmpeg.h | 7 +- fftools/ffmpeg_enc.c | 370 +++------------------------------- fftools/ffmpeg_filter.c | 432 ++++++++++++++++++++++++++++++++++++++-- 4 files changed, 442 insertions(+), 373 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index a854589bef..7c33b56cd3 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -536,7 +536,7 @@ static void print_report(int is_last_report, int64_t timer_start, int64_t cur_ti av_bprintf(&buf_script, "stream_%d_%d_q=%.1f\n", ost->file_index, ost->index, q); } - if (!vid && ost->type == AVMEDIA_TYPE_VIDEO) { + if (!vid && ost->type == AVMEDIA_TYPE_VIDEO && ost->filter) { float fps; uint64_t frame_number = atomic_load(&ost->packets_written); @@ -550,8 +550,8 @@ static void print_report(int is_last_report, int64_t timer_start, int64_t cur_ti if (is_last_report) av_bprintf(&buf, "L"); - nb_frames_dup = ost->nb_frames_dup; - nb_frames_drop = ost->nb_frames_drop; + nb_frames_dup = ost->filter->nb_frames_dup; + nb_frames_drop = ost->filter->nb_frames_drop; vid = 1; } diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index eaa663e718..15790d3e0c 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -302,6 +302,9 @@ typedef struct OutputFilter { /* pts of the last frame received from this filter, in AV_TIME_BASE_Q */ int64_t last_pts; + + uint64_t nb_frames_dup; + uint64_t nb_frames_drop; } OutputFilter; typedef struct FilterGraph { @@ -536,10 +539,6 @@ typedef struct OutputStream { Encoder *enc; AVCodecContext *enc_ctx; - uint64_t nb_frames_dup; - uint64_t nb_frames_drop; - int64_t last_dropped; - /* video only */ AVRational frame_rate; AVRational max_frame_rate; diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index 9aee18bfe1..321554ab5c 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -36,29 +36,9 @@ #include "libavcodec/avcodec.h" -// FIXME private header, used for mid_pred() -#include "libavcodec/mathops.h" - #include "libavformat/avformat.h" -typedef struct FPSConvContext { - AVFrame *last_frame; - /* number of frames emitted by the video-encoding sync code */ - int64_t frame_number; - /* history of nb_frames_prev, i.e. the number of times the - * previous frame was duplicated by vsync code in recent - * do_video_out() calls */ - int64_t frames_prev_hist[3]; - - uint64_t dup_warning; -} FPSConvContext; - struct Encoder { - /* predicted pts of the next frame to be encoded */ - int64_t next_pts; - - FPSConvContext fps; - AVFrame *sq_frame; // packet for receiving encoded output @@ -80,7 +60,6 @@ void enc_free(Encoder **penc) if (!enc) return; - av_frame_free(&enc->fps.last_frame); av_frame_free(&enc->sq_frame); av_packet_free(&enc->pkt); @@ -98,14 +77,6 @@ int enc_alloc(Encoder **penc, const AVCodec *codec) if (!enc) return AVERROR(ENOMEM); - if (codec->type == AVMEDIA_TYPE_VIDEO) { - enc->fps.last_frame = av_frame_alloc(); - if (!enc->fps.last_frame) - goto fail; - - enc->fps.dup_warning = 1000; - } - enc->pkt = av_packet_alloc(); if (!enc->pkt) goto fail; @@ -194,98 +165,6 @@ static int set_encoder_id(OutputFile *of, OutputStream *ost) return 0; } -static int enc_choose_timebase(OutputStream *ost, AVFrame *frame) -{ - const OutputFile *of = output_files[ost->file_index]; - AVCodecContext *enc = ost->enc_ctx; - AVRational tb = (AVRational){ 0, 0 }; - AVRational fr; - FrameData *fd; - - if (ost->type == AVMEDIA_TYPE_SUBTITLE) { - if (ost->enc_timebase.num) - av_log(ost, AV_LOG_WARNING, - "-enc_time_base not supported for subtitles, ignoring\n"); - enc->time_base = AV_TIME_BASE_Q; - return 0; - } - - fd = frame_data(frame); - - // apply -enc_time_base - if (ost->enc_timebase.num == ENC_TIME_BASE_DEMUX && - (fd->dec.tb.num <= 0 || fd->dec.tb.den <= 0)) { - av_log(ost, AV_LOG_ERROR, - "Demuxing timebase not available - cannot use it for encoding\n"); - return AVERROR(EINVAL); - } - - switch (ost->enc_timebase.num) { - case 0: break; - case ENC_TIME_BASE_DEMUX: tb = fd->dec.tb; break; - case ENC_TIME_BASE_FILTER: tb = frame->time_base; break; - default: tb = ost->enc_timebase; break; - } - - if (ost->type == AVMEDIA_TYPE_AUDIO) { - enc->time_base = tb.num ? tb : (AVRational){ 1, frame->sample_rate }; - return 0; - } - - fr = ost->frame_rate; - if (!fr.num) - fr = fd->frame_rate_filter; - - if (ost->is_cfr) { - if (!fr.num && !ost->max_frame_rate.num) { - fr = (AVRational){25, 1}; - av_log(ost, AV_LOG_WARNING, - "No information " - "about the input framerate is available. Falling " - "back to a default value of 25fps. Use the -r option " - "if you want a different framerate.\n"); - } - - if (ost->max_frame_rate.num && - (av_q2d(fr) > av_q2d(ost->max_frame_rate) || - !fr.den)) - fr = ost->max_frame_rate; - } - - if (fr.num > 0) { - if (enc->codec->supported_framerates && !ost->force_fps) { - int idx = av_find_nearest_q_idx(fr, enc->codec->supported_framerates); - fr = enc->codec->supported_framerates[idx]; - } - // reduce frame rate for mpeg4 to be within the spec limits - if (enc->codec_id == AV_CODEC_ID_MPEG4) { - av_reduce(&fr.num, &fr.den, - fr.num, fr.den, 65535); - } - } - - if (av_q2d(fr) > 1e3 && ost->vsync_method != VSYNC_PASSTHROUGH && - (ost->vsync_method == VSYNC_CFR || ost->vsync_method == VSYNC_VSCFR || - (ost->vsync_method == VSYNC_AUTO && !(of->format->flags & AVFMT_VARIABLE_FPS)))){ - av_log(ost, AV_LOG_WARNING, "Frame rate very high for a muxer not efficiently supporting it.\n" - "Please consider specifying a lower framerate, a different muxer or " - "setting vsync/fps_mode to vfr\n"); - } - - enc->framerate = fr; - - ost->st->avg_frame_rate = fr; - - if (!(tb.num > 0 && tb.den > 0)) - tb = av_inv_q(fr); - if (!(tb.num > 0 && tb.den > 0)) - tb = frame->time_base; - - enc->time_base = tb; - - return 0; -} - int enc_open(OutputStream *ost, AVFrame *frame) { InputStream *ist = ost->ist; @@ -317,10 +196,11 @@ int enc_open(OutputStream *ost, AVFrame *frame) dec_ctx = ist->dec_ctx; } - ret = enc_choose_timebase(ost, frame); - if (ret < 0) { - av_log(ost, AV_LOG_ERROR, "Could not choose a time base for encoding\n"); - return AVERROR(EINVAL); + // the timebase is chosen by filtering code + if (ost->type == AVMEDIA_TYPE_AUDIO || ost->type == AVMEDIA_TYPE_VIDEO) { + enc_ctx->time_base = frame->time_base; + enc_ctx->framerate = fd->frame_rate_filter; + ost->st->avg_frame_rate = fd->frame_rate_filter; } switch (enc_ctx->codec_type) { @@ -383,6 +263,11 @@ int enc_open(OutputStream *ost, AVFrame *frame) break; } case AVMEDIA_TYPE_SUBTITLE: + if (ost->enc_timebase.num) + av_log(ost, AV_LOG_WARNING, + "-enc_time_base not supported for subtitles, ignoring\n"); + enc_ctx->time_base = AV_TIME_BASE_Q; + if (!enc_ctx->width) { enc_ctx->width = ost->ist->par->width; enc_ctx->height = ost->ist->par->height; @@ -765,9 +650,6 @@ static int encode_frame(OutputFile *of, OutputStream *ost, AVFrame *frame) if (frame->sample_aspect_ratio.num && !ost->frame_aspect_ratio.num) enc->sample_aspect_ratio = frame->sample_aspect_ratio; - } else if (ost->last_dropped) { - ost->nb_frames_drop++; - ost->last_dropped = 0; } update_benchmark(NULL); @@ -892,7 +774,6 @@ static int submit_encode_frame(OutputFile *of, OutputStream *ost, static int do_audio_out(OutputFile *of, OutputStream *ost, AVFrame *frame) { - Encoder *e = ost->enc; AVCodecContext *enc = ost->enc_ctx; int ret; @@ -903,183 +784,15 @@ static int do_audio_out(OutputFile *of, OutputStream *ost, return 0; } - if (frame->pts == AV_NOPTS_VALUE) - frame->pts = e->next_pts; - else { - int64_t start_time = (of->start_time == AV_NOPTS_VALUE) ? 0 : of->start_time; - frame->pts = - av_rescale_q(frame->pts, frame->time_base, enc->time_base) - - av_rescale_q(start_time, AV_TIME_BASE_Q, enc->time_base); - } - frame->time_base = enc->time_base; - frame->duration = av_rescale_q(frame->nb_samples, (AVRational){1, frame->sample_rate}, - enc->time_base); - if (!check_recording_time(ost, frame->pts, frame->time_base)) return 0; - e->next_pts = frame->pts + frame->nb_samples; - ret = submit_encode_frame(of, ost, frame); return (ret < 0 && ret != AVERROR_EOF) ? ret : 0; } -static double adjust_frame_pts_to_encoder_tb(AVFrame *frame, AVRational tb_dst, - int64_t start_time) -{ - double float_pts = AV_NOPTS_VALUE; // this is identical to frame.pts but with higher precision - - AVRational tb = tb_dst; - AVRational filter_tb = frame->time_base; - const int extra_bits = av_clip(29 - av_log2(tb.den), 0, 16); - - if (frame->pts == AV_NOPTS_VALUE) - goto early_exit; - - tb.den <<= extra_bits; - float_pts = av_rescale_q(frame->pts, filter_tb, tb) - - av_rescale_q(start_time, AV_TIME_BASE_Q, tb); - float_pts /= 1 << extra_bits; - // when float_pts is not exactly an integer, - // avoid exact midpoints to reduce the chance of rounding differences, this - // can be removed in case the fps code is changed to work with integers - if (float_pts != llrint(float_pts)) - float_pts += FFSIGN(float_pts) * 1.0 / (1<<17); - - frame->pts = av_rescale_q(frame->pts, filter_tb, tb_dst) - - av_rescale_q(start_time, AV_TIME_BASE_Q, tb_dst); - frame->time_base = tb_dst; - -early_exit: - - if (debug_ts) { - av_log(NULL, AV_LOG_INFO, "filter -> pts:%s pts_time:%s exact:%f time_base:%d/%d\n", - frame ? av_ts2str(frame->pts) : "NULL", - av_ts2timestr(frame->pts, &tb_dst), - float_pts, tb_dst.num, tb_dst.den); - } - - return float_pts; -} - -/* Convert frame timestamps to the encoder timebase and decide how many times - * should this (and possibly previous) frame be repeated in order to conform to - * desired target framerate (if any). - */ -static void video_sync_process(OutputFile *of, OutputStream *ost, AVFrame *frame, - int64_t *nb_frames, int64_t *nb_frames_prev) -{ - Encoder *e = ost->enc; - FPSConvContext *fps = &e->fps; - AVCodecContext *enc = ost->enc_ctx; - double delta0, delta, sync_ipts, duration; - - if (!frame) { - *nb_frames_prev = *nb_frames = mid_pred(fps->frames_prev_hist[0], - fps->frames_prev_hist[1], - fps->frames_prev_hist[2]); - goto finish; - } - - duration = lrintf(frame->duration * av_q2d(frame->time_base) / av_q2d(enc->time_base)); - - sync_ipts = adjust_frame_pts_to_encoder_tb(frame, enc->time_base, - of->start_time == AV_NOPTS_VALUE ? 0 : of->start_time); - /* delta0 is the "drift" between the input frame and - * where it would fall in the output. */ - delta0 = sync_ipts - e->next_pts; - delta = delta0 + duration; - - // tracks the number of times the PREVIOUS frame should be duplicated, - // mostly for variable framerate (VFR) - *nb_frames_prev = 0; - /* by default, we output a single frame */ - *nb_frames = 1; - - if (delta0 < 0 && - delta > 0 && - ost->vsync_method != VSYNC_PASSTHROUGH && - ost->vsync_method != VSYNC_DROP) { - if (delta0 < -0.6) { - av_log(ost, AV_LOG_VERBOSE, "Past duration %f too large\n", -delta0); - } else - av_log(ost, AV_LOG_DEBUG, "Clipping frame in rate conversion by %f\n", -delta0); - sync_ipts = e->next_pts; - duration += delta0; - delta0 = 0; - } - - switch (ost->vsync_method) { - case VSYNC_VSCFR: - if (fps->frame_number == 0 && delta0 >= 0.5) { - av_log(ost, AV_LOG_DEBUG, "Not duplicating %d initial frames\n", (int)lrintf(delta0)); - delta = duration; - delta0 = 0; - e->next_pts = llrint(sync_ipts); - } - case VSYNC_CFR: - // FIXME set to 0.5 after we fix some dts/pts bugs like in avidec.c - if (frame_drop_threshold && delta < frame_drop_threshold && fps->frame_number) { - *nb_frames = 0; - } else if (delta < -1.1) - *nb_frames = 0; - else if (delta > 1.1) { - *nb_frames = llrintf(delta); - if (delta0 > 1.1) - *nb_frames_prev = llrintf(delta0 - 0.6); - } - frame->duration = 1; - break; - case VSYNC_VFR: - if (delta <= -0.6) - *nb_frames = 0; - else if (delta > 0.6) - e->next_pts = llrint(sync_ipts); - frame->duration = duration; - break; - case VSYNC_DROP: - case VSYNC_PASSTHROUGH: - frame->duration = duration; - e->next_pts = llrint(sync_ipts); - break; - default: - av_assert0(0); - } - -finish: - memmove(fps->frames_prev_hist + 1, - fps->frames_prev_hist, - sizeof(fps->frames_prev_hist[0]) * (FF_ARRAY_ELEMS(fps->frames_prev_hist) - 1)); - fps->frames_prev_hist[0] = *nb_frames_prev; - - if (*nb_frames_prev == 0 && ost->last_dropped) { - ost->nb_frames_drop++; - av_log(ost, AV_LOG_VERBOSE, - "*** dropping frame %"PRId64" at ts %"PRId64"\n", - fps->frame_number, fps->last_frame->pts); - } - if (*nb_frames > (*nb_frames_prev && ost->last_dropped) + (*nb_frames > *nb_frames_prev)) { - if (*nb_frames > dts_error_threshold * 30) { - av_log(ost, AV_LOG_ERROR, "%"PRId64" frame duplication too large, skipping\n", *nb_frames - 1); - ost->nb_frames_drop++; - *nb_frames = 0; - return; - } - ost->nb_frames_dup += *nb_frames - (*nb_frames_prev && ost->last_dropped) - (*nb_frames > *nb_frames_prev); - av_log(ost, AV_LOG_VERBOSE, "*** %"PRId64" dup!\n", *nb_frames - 1); - if (ost->nb_frames_dup > fps->dup_warning) { - av_log(ost, AV_LOG_WARNING, "More than %"PRIu64" frames duplicated\n", fps->dup_warning); - fps->dup_warning *= 10; - } - } - - ost->last_dropped = *nb_frames == *nb_frames_prev && frame; - ost->kf.dropped_keyframe |= ost->last_dropped && (frame->flags & AV_FRAME_FLAG_KEY); -} - static enum AVPictureType forced_kf_apply(void *logctx, KeyframeForceCtx *kf, - AVRational tb, const AVFrame *in_picture, - int dup_idx) + AVRational tb, const AVFrame *in_picture) { double pts_time; @@ -1113,11 +826,8 @@ static enum AVPictureType forced_kf_apply(void *logctx, KeyframeForceCtx *kf, kf->expr_const_values[FKF_N_FORCED] += 1; goto force_keyframe; } - } else if (kf->type == KF_FORCE_SOURCE && !dup_idx) { - int dropped_keyframe = kf->dropped_keyframe; - kf->dropped_keyframe = 0; - if ((in_picture->flags & AV_FRAME_FLAG_KEY) || dropped_keyframe) - goto force_keyframe; + } else if (kf->type == KF_FORCE_SOURCE && (in_picture->flags & AV_FRAME_FLAG_KEY)) { + goto force_keyframe; } return AV_PICTURE_TYPE_NONE; @@ -1128,58 +838,26 @@ force_keyframe: } /* May modify/reset frame */ -static int do_video_out(OutputFile *of, OutputStream *ost, AVFrame *frame) +static int do_video_out(OutputFile *of, OutputStream *ost, AVFrame *in_picture) { int ret; - Encoder *e = ost->enc; AVCodecContext *enc = ost->enc_ctx; - int64_t nb_frames, nb_frames_prev, i; - video_sync_process(of, ost, frame, - &nb_frames, &nb_frames_prev); + if (!check_recording_time(ost, in_picture->pts, ost->enc_ctx->time_base)) + return 0; - /* duplicates frame if needed */ - for (i = 0; i < nb_frames; i++) { - AVFrame *in_picture; - - if (i < nb_frames_prev && e->fps.last_frame->buf[0]) { - in_picture = e->fps.last_frame; - } else - in_picture = frame; - - if (!in_picture) - return 0; - - in_picture->pts = e->next_pts; - - if (!check_recording_time(ost, in_picture->pts, ost->enc_ctx->time_base)) - return 0; - - in_picture->quality = enc->global_quality; - in_picture->pict_type = forced_kf_apply(ost, &ost->kf, enc->time_base, in_picture, i); + in_picture->quality = enc->global_quality; + in_picture->pict_type = forced_kf_apply(ost, &ost->kf, enc->time_base, in_picture); #if FFMPEG_OPT_TOP - if (ost->top_field_first >= 0) { - in_picture->flags &= ~AV_FRAME_FLAG_TOP_FIELD_FIRST; - in_picture->flags |= AV_FRAME_FLAG_TOP_FIELD_FIRST * (!!ost->top_field_first); - } + if (ost->top_field_first >= 0) { + in_picture->flags &= ~AV_FRAME_FLAG_TOP_FIELD_FIRST; + in_picture->flags |= AV_FRAME_FLAG_TOP_FIELD_FIRST * (!!ost->top_field_first); + } #endif - ret = submit_encode_frame(of, ost, in_picture); - if (ret == AVERROR_EOF) - break; - else if (ret < 0) - return ret; - - e->next_pts++; - e->fps.frame_number++; - } - - av_frame_unref(e->fps.last_frame); - if (frame) - av_frame_move_ref(e->fps.last_frame, frame); - - return 0; + ret = submit_encode_frame(of, ost, in_picture); + return (ret == AVERROR_EOF) ? 0 : ret; } int enc_frame(OutputStream *ost, AVFrame *frame) diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 804b9de3dc..92f6a6236d 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -38,6 +38,9 @@ #include "libavutil/samplefmt.h" #include "libavutil/timestamp.h" +// FIXME private header, used for mid_pred() +#include "libavcodec/mathops.h" + typedef struct FilterGraphPriv { FilterGraph fg; @@ -54,6 +57,8 @@ typedef struct FilterGraphPriv { // frame for temporarily holding output from the filtergraph AVFrame *frame; + // frame for sending output to the encoder + AVFrame *frame_enc; } FilterGraphPriv; static FilterGraphPriv *fgp_from_fg(FilterGraph *fg) @@ -134,6 +139,26 @@ static InputFilterPriv *ifp_from_ifilter(InputFilter *ifilter) return (InputFilterPriv*)ifilter; } +typedef struct FPSConvContext { + AVFrame *last_frame; + /* number of frames emitted by the video-encoding sync code */ + int64_t frame_number; + /* history of nb_frames_prev, i.e. the number of times the + * previous frame was duplicated by vsync code in recent + * do_video_out() calls */ + int64_t frames_prev_hist[3]; + + uint64_t dup_warning; + + int last_dropped; + int dropped_keyframe; + + AVRational framerate; + AVRational framerate_max; + const AVRational *framerate_supported; + int framerate_clip; +} FPSConvContext; + typedef struct OutputFilterPriv { OutputFilter ofilter; @@ -145,7 +170,13 @@ typedef struct OutputFilterPriv { int sample_rate; AVChannelLayout ch_layout; - AVRational time_base; + // time base in which the output is sent to our downstream + // does not need to match the filtersink's timebase + AVRational tb_out; + // at least one frame with the above timebase was sent + // to our downstream, so it cannot change anymore + int tb_out_locked; + AVRational sample_aspect_ratio; // those are only set if no format is specified and the encoder gives us multiple options @@ -154,6 +185,12 @@ typedef struct OutputFilterPriv { const AVChannelLayout *ch_layouts; const int *sample_rates; + AVRational enc_timebase; + // offset for output timestamps, in AV_TIME_BASE_Q + int64_t ts_offset; + int64_t next_pts; + FPSConvContext fps; + // set to 1 after at least one frame passed through this output int got_frame; } OutputFilterPriv; @@ -627,6 +664,7 @@ static int set_channel_layout(OutputFilterPriv *f, OutputStream *ost) int ofilter_bind_ost(OutputFilter *ofilter, OutputStream *ost) { + const OutputFile *of = output_files[ost->file_index]; OutputFilterPriv *ofp = ofp_from_ofilter(ofilter); FilterGraph *fg = ofilter->graph; FilterGraphPriv *fgp = fgp_from_fg(fg); @@ -637,6 +675,9 @@ int ofilter_bind_ost(OutputFilter *ofilter, OutputStream *ost) ofilter->ost = ost; av_freep(&ofilter->linklabel); + ofp->ts_offset = of->start_time == AV_NOPTS_VALUE ? 0 : of->start_time; + ofp->enc_timebase = ost->enc_timebase; + switch (ost->enc_ctx->codec_type) { case AVMEDIA_TYPE_VIDEO: ofp->width = ost->enc_ctx->width; @@ -673,6 +714,21 @@ int ofilter_bind_ost(OutputFilter *ofilter, OutputStream *ost) fgp->disable_conversions |= ost->keep_pix_fmt; + ofp->fps.last_frame = av_frame_alloc(); + if (!ofp->fps.last_frame) + return AVERROR(ENOMEM); + + ofp->fps.framerate = ost->frame_rate; + ofp->fps.framerate_max = ost->max_frame_rate; + ofp->fps.framerate_supported = ost->force_fps ? + NULL : c->supported_framerates; + + // reduce frame rate for mpeg4 to be within the spec limits + if (c->id == AV_CODEC_ID_MPEG4) + ofp->fps.framerate_clip = 65535; + + ofp->fps.dup_warning = 1000; + break; case AVMEDIA_TYPE_AUDIO: if (ost->enc_ctx->sample_fmt != AV_SAMPLE_FMT_NONE) { @@ -777,6 +833,8 @@ void fg_free(FilterGraph **pfg) OutputFilter *ofilter = fg->outputs[j]; OutputFilterPriv *ofp = ofp_from_ofilter(ofilter); + av_frame_free(&ofp->fps.last_frame); + av_freep(&ofilter->linklabel); av_freep(&ofilter->name); av_channel_layout_uninit(&ofp->ch_layout); @@ -786,6 +844,7 @@ void fg_free(FilterGraph **pfg) av_freep(&fgp->graph_desc); av_frame_free(&fgp->frame); + av_frame_free(&fgp->frame_enc); av_freep(pfg); } @@ -828,8 +887,9 @@ int fg_create(FilterGraph **pfg, char *graph_desc) snprintf(fgp->log_name, sizeof(fgp->log_name), "fc#%d", fg->index); - fgp->frame = av_frame_alloc(); - if (!fgp->frame) + fgp->frame = av_frame_alloc(); + fgp->frame_enc = av_frame_alloc(); + if (!fgp->frame || !fgp->frame_enc) return AVERROR(ENOMEM); /* this graph is only used for determining the kinds of inputs @@ -1630,7 +1690,16 @@ static int configure_filtergraph(FilterGraph *fg) ofp->width = av_buffersink_get_w(sink); ofp->height = av_buffersink_get_h(sink); - ofp->time_base = av_buffersink_get_time_base(sink); + // If the timing parameters are not locked yet, get the tentative values + // here but don't lock them. They will only be used if no output frames + // are ever produced. + if (!ofp->tb_out_locked) { + AVRational fr = av_buffersink_get_frame_rate(sink); + if (ofp->fps.framerate.num <= 0 && ofp->fps.framerate.den <= 0 && + fr.num > 0 && fr.den > 0) + ofp->fps.framerate = fr; + ofp->tb_out = av_buffersink_get_time_base(sink); + } ofp->sample_aspect_ratio = av_buffersink_get_sample_aspect_ratio(sink); ofp->sample_rate = av_buffersink_get_sample_rate(sink); @@ -1765,6 +1834,313 @@ void fg_send_command(FilterGraph *fg, double time, const char *target, } } +static int choose_out_timebase(OutputFilterPriv *ofp, AVFrame *frame) +{ + OutputFilter *ofilter = &ofp->ofilter; + FPSConvContext *fps = &ofp->fps; + AVRational tb = (AVRational){ 0, 0 }; + AVRational fr; + FrameData *fd; + + fd = frame_data(frame); + + // apply -enc_time_base + if (ofp->enc_timebase.num == ENC_TIME_BASE_DEMUX && + (fd->dec.tb.num <= 0 || fd->dec.tb.den <= 0)) { + av_log(ofilter->ost, AV_LOG_ERROR, + "Demuxing timebase not available - cannot use it for encoding\n"); + return AVERROR(EINVAL); + } + + switch (ofp->enc_timebase.num) { + case 0: break; + case ENC_TIME_BASE_DEMUX: tb = fd->dec.tb; break; + case ENC_TIME_BASE_FILTER: tb = frame->time_base; break; + default: tb = ofp->enc_timebase; break; + } + + if (ofilter->type == AVMEDIA_TYPE_AUDIO) { + tb = tb.num ? tb : (AVRational){ 1, frame->sample_rate }; + goto finish; + } + + fr = fps->framerate; + if (!fr.num) { + AVRational fr_sink = av_buffersink_get_frame_rate(ofp->filter); + if (fr_sink.num > 0 && fr_sink.den > 0) + fr = fr_sink; + } + + if (ofilter->ost->is_cfr) { + if (!fr.num && !fps->framerate_max.num) { + fr = (AVRational){25, 1}; + av_log(ofilter->ost, AV_LOG_WARNING, + "No information " + "about the input framerate is available. Falling " + "back to a default value of 25fps. Use the -r option " + "if you want a different framerate.\n"); + } + + if (fps->framerate_max.num && + (av_q2d(fr) > av_q2d(fps->framerate_max) || + !fr.den)) + fr = fps->framerate_max; + } + + if (fr.num > 0) { + if (fps->framerate_supported) { + int idx = av_find_nearest_q_idx(fr, fps->framerate_supported); + fr = fps->framerate_supported[idx]; + } + if (fps->framerate_clip) { + av_reduce(&fr.num, &fr.den, + fr.num, fr.den, fps->framerate_clip); + } + } + + if (!(tb.num > 0 && tb.den > 0)) + tb = av_inv_q(fr); + if (!(tb.num > 0 && tb.den > 0)) + tb = frame->time_base; + +finish: + ofp->tb_out = tb; + fps->framerate = fr; + ofp->tb_out_locked = 1; + + return 0; +} + +static double adjust_frame_pts_to_encoder_tb(AVFrame *frame, AVRational tb_dst, + int64_t start_time) +{ + double float_pts = AV_NOPTS_VALUE; // this is identical to frame.pts but with higher precision + + AVRational tb = tb_dst; + AVRational filter_tb = frame->time_base; + const int extra_bits = av_clip(29 - av_log2(tb.den), 0, 16); + + if (frame->pts == AV_NOPTS_VALUE) + goto early_exit; + + tb.den <<= extra_bits; + float_pts = av_rescale_q(frame->pts, filter_tb, tb) - + av_rescale_q(start_time, AV_TIME_BASE_Q, tb); + float_pts /= 1 << extra_bits; + // when float_pts is not exactly an integer, + // avoid exact midpoints to reduce the chance of rounding differences, this + // can be removed in case the fps code is changed to work with integers + if (float_pts != llrint(float_pts)) + float_pts += FFSIGN(float_pts) * 1.0 / (1<<17); + + frame->pts = av_rescale_q(frame->pts, filter_tb, tb_dst) - + av_rescale_q(start_time, AV_TIME_BASE_Q, tb_dst); + frame->time_base = tb_dst; + +early_exit: + + if (debug_ts) { + av_log(NULL, AV_LOG_INFO, "filter -> pts:%s pts_time:%s exact:%f time_base:%d/%d\n", + frame ? av_ts2str(frame->pts) : "NULL", + av_ts2timestr(frame->pts, &tb_dst), + float_pts, tb_dst.num, tb_dst.den); + } + + return float_pts; +} + +/* Convert frame timestamps to the encoder timebase and decide how many times + * should this (and possibly previous) frame be repeated in order to conform to + * desired target framerate (if any). + */ +static void video_sync_process(OutputFilterPriv *ofp, AVFrame *frame, + int64_t *nb_frames, int64_t *nb_frames_prev) +{ + OutputFilter *ofilter = &ofp->ofilter; + OutputStream *ost = ofilter->ost; + FPSConvContext *fps = &ofp->fps; + double delta0, delta, sync_ipts, duration; + + if (!frame) { + *nb_frames_prev = *nb_frames = mid_pred(fps->frames_prev_hist[0], + fps->frames_prev_hist[1], + fps->frames_prev_hist[2]); + + if (!*nb_frames && fps->last_dropped) { + ofilter->nb_frames_drop++; + fps->last_dropped++; + } + + goto finish; + } + + duration = lrintf(frame->duration * av_q2d(frame->time_base) / av_q2d(ofp->tb_out)); + + sync_ipts = adjust_frame_pts_to_encoder_tb(frame, ofp->tb_out, ofp->ts_offset); + /* delta0 is the "drift" between the input frame and + * where it would fall in the output. */ + delta0 = sync_ipts - ofp->next_pts; + delta = delta0 + duration; + + // tracks the number of times the PREVIOUS frame should be duplicated, + // mostly for variable framerate (VFR) + *nb_frames_prev = 0; + /* by default, we output a single frame */ + *nb_frames = 1; + + if (delta0 < 0 && + delta > 0 && + ost->vsync_method != VSYNC_PASSTHROUGH && + ost->vsync_method != VSYNC_DROP) { + if (delta0 < -0.6) { + av_log(ost, AV_LOG_VERBOSE, "Past duration %f too large\n", -delta0); + } else + av_log(ost, AV_LOG_DEBUG, "Clipping frame in rate conversion by %f\n", -delta0); + sync_ipts = ofp->next_pts; + duration += delta0; + delta0 = 0; + } + + switch (ost->vsync_method) { + case VSYNC_VSCFR: + if (fps->frame_number == 0 && delta0 >= 0.5) { + av_log(ost, AV_LOG_DEBUG, "Not duplicating %d initial frames\n", (int)lrintf(delta0)); + delta = duration; + delta0 = 0; + ofp->next_pts = llrint(sync_ipts); + } + case VSYNC_CFR: + // FIXME set to 0.5 after we fix some dts/pts bugs like in avidec.c + if (frame_drop_threshold && delta < frame_drop_threshold && fps->frame_number) { + *nb_frames = 0; + } else if (delta < -1.1) + *nb_frames = 0; + else if (delta > 1.1) { + *nb_frames = llrintf(delta); + if (delta0 > 1.1) + *nb_frames_prev = llrintf(delta0 - 0.6); + } + frame->duration = 1; + break; + case VSYNC_VFR: + if (delta <= -0.6) + *nb_frames = 0; + else if (delta > 0.6) + ofp->next_pts = llrint(sync_ipts); + frame->duration = duration; + break; + case VSYNC_DROP: + case VSYNC_PASSTHROUGH: + frame->duration = duration; + ofp->next_pts = llrint(sync_ipts); + break; + default: + av_assert0(0); + } + +finish: + memmove(fps->frames_prev_hist + 1, + fps->frames_prev_hist, + sizeof(fps->frames_prev_hist[0]) * (FF_ARRAY_ELEMS(fps->frames_prev_hist) - 1)); + fps->frames_prev_hist[0] = *nb_frames_prev; + + if (*nb_frames_prev == 0 && fps->last_dropped) { + ofilter->nb_frames_drop++; + av_log(ost, AV_LOG_VERBOSE, + "*** dropping frame %"PRId64" at ts %"PRId64"\n", + fps->frame_number, fps->last_frame->pts); + } + if (*nb_frames > (*nb_frames_prev && fps->last_dropped) + (*nb_frames > *nb_frames_prev)) { + if (*nb_frames > dts_error_threshold * 30) { + av_log(ost, AV_LOG_ERROR, "%"PRId64" frame duplication too large, skipping\n", *nb_frames - 1); + ofilter->nb_frames_drop++; + *nb_frames = 0; + return; + } + ofilter->nb_frames_dup += *nb_frames - (*nb_frames_prev && fps->last_dropped) - (*nb_frames > *nb_frames_prev); + av_log(ost, AV_LOG_VERBOSE, "*** %"PRId64" dup!\n", *nb_frames - 1); + if (ofilter->nb_frames_dup > fps->dup_warning) { + av_log(ost, AV_LOG_WARNING, "More than %"PRIu64" frames duplicated\n", fps->dup_warning); + fps->dup_warning *= 10; + } + } + + fps->last_dropped = *nb_frames == *nb_frames_prev && frame; + fps->dropped_keyframe |= fps->last_dropped && (frame->flags & AV_FRAME_FLAG_KEY); +} + +static int fg_output_frame(OutputFilterPriv *ofp, AVFrame *frame) +{ + FilterGraphPriv *fgp = fgp_from_fg(ofp->ofilter.graph); + OutputStream *ost = ofp->ofilter.ost; + AVFrame *frame_prev = ofp->fps.last_frame; + enum AVMediaType type = ofp->ofilter.type; + + int64_t nb_frames = 1, nb_frames_prev = 0; + + if (type == AVMEDIA_TYPE_VIDEO) + video_sync_process(ofp, frame, &nb_frames, &nb_frames_prev); + + for (int64_t i = 0; i < nb_frames; i++) { + AVFrame *frame_out; + int ret; + + if (type == AVMEDIA_TYPE_VIDEO) { + AVFrame *frame_in = (i < nb_frames_prev && frame_prev->buf[0]) ? + frame_prev : frame; + if (!frame_in) + break; + + frame_out = fgp->frame_enc; + ret = av_frame_ref(frame_out, frame_in); + if (ret < 0) + return ret; + + frame_out->pts = ofp->next_pts; + + if (ofp->fps.dropped_keyframe) { + frame_out->flags |= AV_FRAME_FLAG_KEY; + ofp->fps.dropped_keyframe = 0; + } + } else { + frame->pts = (frame->pts == AV_NOPTS_VALUE) ? ofp->next_pts : + av_rescale_q(frame->pts, frame->time_base, ofp->tb_out) - + av_rescale_q(ofp->ts_offset, AV_TIME_BASE_Q, ofp->tb_out); + + frame->time_base = ofp->tb_out; + frame->duration = av_rescale_q(frame->nb_samples, + (AVRational){ 1, frame->sample_rate }, + ofp->tb_out); + + ofp->next_pts = frame->pts + frame->duration; + + frame_out = frame; + } + + ret = enc_frame(ost, frame_out); + av_frame_unref(frame_out); + if (ret < 0) + return ret; + + if (type == AVMEDIA_TYPE_VIDEO) { + ofp->fps.frame_number++; + ofp->next_pts++; + + if (i == nb_frames_prev && frame) + frame->flags &= ~AV_FRAME_FLAG_KEY; + } + + ofp->got_frame = 1; + } + + if (frame && frame_prev) { + av_frame_unref(frame_prev); + av_frame_move_ref(frame_prev, frame); + } + + return 0; +} + static int fg_output_step(OutputFilterPriv *ofp, int flush) { FilterGraphPriv *fgp = fgp_from_fg(ofp->ofilter.graph); @@ -1782,9 +2158,8 @@ static int fg_output_step(OutputFilterPriv *ofp, int flush) "Error in av_buffersink_get_frame_flags(): %s\n", av_err2str(ret)); } else if (flush && ret == AVERROR_EOF && ofp->got_frame && av_buffersink_get_type(filter) == AVMEDIA_TYPE_VIDEO) { - ret = enc_frame(ost, NULL); - if (ret < 0) - return ret; + ret = fg_output_frame(ofp, NULL); + return (ret < 0) ? ret : 1; } return 1; @@ -1794,14 +2169,26 @@ static int fg_output_step(OutputFilterPriv *ofp, int flush) return 0; } + frame->time_base = av_buffersink_get_time_base(filter); + if (frame->pts != AV_NOPTS_VALUE) { - AVRational tb = av_buffersink_get_time_base(filter); - ost->filter->last_pts = av_rescale_q(frame->pts, tb, AV_TIME_BASE_Q); - frame->time_base = tb; + ost->filter->last_pts = av_rescale_q(frame->pts, frame->time_base, + AV_TIME_BASE_Q); if (debug_ts) av_log(fgp, AV_LOG_INFO, "filter_raw -> pts:%s pts_time:%s time_base:%d/%d\n", - av_ts2str(frame->pts), av_ts2timestr(frame->pts, &tb), tb.num, tb.den); + av_ts2str(frame->pts), av_ts2timestr(frame->pts, &frame->time_base), + frame->time_base.num, frame->time_base.den); + } + + // Choose the output timebase the first time we get a frame. + if (!ofp->tb_out_locked) { + ret = choose_out_timebase(ofp, frame); + if (ret < 0) { + av_log(ost, AV_LOG_ERROR, "Could not choose an output time base\n"); + av_frame_unref(frame); + return ret; + } } fd = frame_data(frame); @@ -1816,22 +2203,20 @@ static int fg_output_step(OutputFilterPriv *ofp, int flush) fd->bits_per_raw_sample = 0; if (ost->type == AVMEDIA_TYPE_VIDEO) { - AVRational fr = av_buffersink_get_frame_rate(filter); - if (fr.num > 0 && fr.den > 0) { - fd->frame_rate_filter = fr; - - if (!frame->duration) + if (!frame->duration) { + AVRational fr = av_buffersink_get_frame_rate(filter); + if (fr.num > 0 && fr.den > 0) frame->duration = av_rescale_q(1, av_inv_q(fr), frame->time_base); } + + fd->frame_rate_filter = ofp->fps.framerate; } - ret = enc_frame(ost, frame); + ret = fg_output_frame(ofp, frame); av_frame_unref(frame); if (ret < 0) return ret; - ofp->got_frame = 1; - return 0; } @@ -2098,8 +2483,9 @@ int fg_transcode_step(FilterGraph *graph, InputStream **best_ist) // at least initialize the encoder with a dummy frame if (!ofp->got_frame) { AVFrame *frame = fgp->frame; + FrameData *fd; - frame->time_base = ofp->time_base; + frame->time_base = ofp->tb_out; frame->format = ofp->format; frame->width = ofp->width; @@ -2113,6 +2499,12 @@ int fg_transcode_step(FilterGraph *graph, InputStream **best_ist) return ret; } + fd = frame_data(frame); + if (!fd) + return AVERROR(ENOMEM); + + fd->frame_rate_filter = ofp->fps.framerate; + av_assert0(!frame->buf[0]); av_log(ofilter->ost, AV_LOG_WARNING, -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 12/27] fftools/ffmpeg_filter: fail on filtering errors 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (10 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 11/27] fftools/ffmpeg_enc: move fps conversion code to ffmpeg_filter Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 13/27] fftools/ffmpeg_enc: constify the frame passed to enc_open() Anton Khirnov ` (15 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel These should be considered serious errors - don't just print a log message and continue as if nothing happened. --- fftools/ffmpeg_filter.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 92f6a6236d..b9a4a4e400 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -2152,17 +2152,17 @@ static int fg_output_step(OutputFilterPriv *ofp, int flush) ret = av_buffersink_get_frame_flags(filter, frame, AV_BUFFERSINK_FLAG_NO_REQUEST); - if (ret < 0) { - if (ret != AVERROR(EAGAIN) && ret != AVERROR_EOF) { - av_log(fgp, AV_LOG_WARNING, - "Error in av_buffersink_get_frame_flags(): %s\n", av_err2str(ret)); - } else if (flush && ret == AVERROR_EOF && ofp->got_frame && - av_buffersink_get_type(filter) == AVMEDIA_TYPE_VIDEO) { - ret = fg_output_frame(ofp, NULL); - return (ret < 0) ? ret : 1; - } - + if (flush && ret == AVERROR_EOF && ofp->got_frame && + ost->type == AVMEDIA_TYPE_VIDEO) { + ret = fg_output_frame(ofp, NULL); + return (ret < 0) ? ret : 1; + } else if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) { return 1; + } else if (ret < 0) { + av_log(fgp, AV_LOG_WARNING, + "Error in retrieving a frame from the filtergraph: %s\n", + av_err2str(ret)); + return ret; } if (ost->finished) { av_frame_unref(frame); -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 13/27] fftools/ffmpeg_enc: constify the frame passed to enc_open() 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (11 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 12/27] fftools/ffmpeg_filter: fail on filtering errors Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 14/27] fftools/ffmpeg_filter: move filtering to a separate thread Anton Khirnov ` (14 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel --- fftools/ffmpeg.h | 2 +- fftools/ffmpeg_enc.c | 7 +++---- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 15790d3e0c..0983d026cd 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -817,7 +817,7 @@ int dec_packet(InputStream *ist, const AVPacket *pkt, int no_eof); int enc_alloc(Encoder **penc, const AVCodec *codec); void enc_free(Encoder **penc); -int enc_open(OutputStream *ost, AVFrame *frame); +int enc_open(OutputStream *ost, const AVFrame *frame); int enc_subtitle(OutputFile *of, OutputStream *ost, const AVSubtitle *sub); int enc_frame(OutputStream *ost, AVFrame *frame); int enc_flush(void); diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index 321554ab5c..c300a11f28 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -165,7 +165,7 @@ static int set_encoder_id(OutputFile *of, OutputStream *ost) return 0; } -int enc_open(OutputStream *ost, AVFrame *frame) +int enc_open(OutputStream *ost, const AVFrame *frame) { InputStream *ist = ost->ist; Encoder *e = ost->enc; @@ -183,9 +183,8 @@ int enc_open(OutputStream *ost, AVFrame *frame) av_assert0(frame || (enc->type != AVMEDIA_TYPE_VIDEO && enc->type != AVMEDIA_TYPE_AUDIO)); if (frame) { - fd = frame_data(frame); - if (!fd) - return AVERROR(ENOMEM); + av_assert0(frame->opaque_ref); + fd = (FrameData*)frame->opaque_ref->data; } ret = set_encoder_id(output_files[ost->file_index], ost); -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 14/27] fftools/ffmpeg_filter: move filtering to a separate thread 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (12 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 13/27] fftools/ffmpeg_enc: constify the frame passed to enc_open() Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 15/27] fftools/ffmpeg_mux: add muxing thread private data Anton Khirnov ` (13 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel As previously for decoding, this is merely "scaffolding" for moving to a fully threaded architecture and does not yet make filtering truly parallel - the main thread will currently wait for the filtering thread to finish its work before continuing. That will change in future commits after encoders are also moved to threads and a thread-aware scheduler is added. --- fftools/ffmpeg.h | 1 - fftools/ffmpeg_dec.c | 39 +- fftools/ffmpeg_filter.c | 835 ++++++++++++++++++++++++++++++++++------ 3 files changed, 733 insertions(+), 142 deletions(-) diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 0983d026cd..5a8f52ce00 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -735,7 +735,6 @@ FrameData *frame_data(AVFrame *frame); int ifilter_send_frame(InputFilter *ifilter, AVFrame *frame, int keep_reference); int ifilter_send_eof(InputFilter *ifilter, int64_t pts, AVRational tb); -int ifilter_sub2video(InputFilter *ifilter, const AVFrame *frame); void ifilter_sub2video_heartbeat(InputFilter *ifilter, int64_t pts, AVRational tb); /** diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c index 1de8234a97..fb7b404020 100644 --- a/fftools/ffmpeg_dec.c +++ b/fftools/ffmpeg_dec.c @@ -146,11 +146,12 @@ fail: static int send_frame_to_filters(InputStream *ist, AVFrame *decoded_frame) { - int i, ret; + int i, ret = 0; - av_assert1(ist->nb_filters > 0); /* ensure ret is initialized */ for (i = 0; i < ist->nb_filters; i++) { - ret = ifilter_send_frame(ist->filters[i], decoded_frame, i < ist->nb_filters - 1); + ret = ifilter_send_frame(ist->filters[i], decoded_frame, + i < ist->nb_filters - 1 || + ist->dec->type == AVMEDIA_TYPE_SUBTITLE); if (ret == AVERROR_EOF) ret = 0; /* ignore */ if (ret < 0) { @@ -379,15 +380,6 @@ static int video_frame_process(InputStream *ist, AVFrame *frame) return 0; } -static void sub2video_flush(InputStream *ist) -{ - for (int i = 0; i < ist->nb_filters; i++) { - int ret = ifilter_sub2video(ist->filters[i], NULL); - if (ret != AVERROR_EOF && ret < 0) - av_log(NULL, AV_LOG_WARNING, "Flush the frame error.\n"); - } -} - static int process_subtitle(InputStream *ist, AVFrame *frame) { Decoder *d = ist->decoder; @@ -425,14 +417,9 @@ static int process_subtitle(InputStream *ist, AVFrame *frame) if (!subtitle) return 0; - for (int i = 0; i < ist->nb_filters; i++) { - ret = ifilter_sub2video(ist->filters[i], frame); - if (ret < 0) { - av_log(ist, AV_LOG_ERROR, "Error sending a subtitle for filtering: %s\n", - av_err2str(ret)); - return ret; - } - } + ret = send_frame_to_filters(ist, frame); + if (ret < 0) + return ret; subtitle = (AVSubtitle*)frame->buf[0]->data; if (!subtitle->num_rects) @@ -823,14 +810,10 @@ finish: return ret; // signal EOF to our downstreams - if (ist->dec->type == AVMEDIA_TYPE_SUBTITLE) - sub2video_flush(ist); - else { - ret = send_filter_eof(ist); - if (ret < 0) { - av_log(NULL, AV_LOG_FATAL, "Error marking filters as finished\n"); - return ret; - } + ret = send_filter_eof(ist); + if (ret < 0) { + av_log(NULL, AV_LOG_FATAL, "Error marking filters as finished\n"); + return ret; } return AVERROR_EOF; diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index b9a4a4e400..cfd13dd81a 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -21,6 +21,7 @@ #include <stdint.h> #include "ffmpeg.h" +#include "thread_queue.h" #include "libavfilter/avfilter.h" #include "libavfilter/buffersink.h" @@ -41,6 +42,14 @@ // FIXME private header, used for mid_pred() #include "libavcodec/mathops.h" +enum FrameOpaque { + FRAME_OPAQUE_REAP_FILTERS = 1, + FRAME_OPAQUE_CHOOSE_INPUT, + FRAME_OPAQUE_SUB_HEARTBEAT, + FRAME_OPAQUE_EOF, + FRAME_OPAQUE_SEND_COMMAND, +}; + typedef struct FilterGraphPriv { FilterGraph fg; @@ -53,12 +62,50 @@ typedef struct FilterGraphPriv { int is_meta; int disable_conversions; + int nb_inputs_bound; + int nb_outputs_bound; + const char *graph_desc; // frame for temporarily holding output from the filtergraph AVFrame *frame; // frame for sending output to the encoder AVFrame *frame_enc; + + pthread_t thread; + /** + * Queue for sending frames from the main thread to the filtergraph. Has + * nb_inputs+1 streams - the first nb_inputs stream correspond to + * filtergraph inputs. Frames on those streams may have their opaque set to + * - FRAME_OPAQUE_EOF: frame contains no data, but pts+timebase of the + * EOF event for the correspondint stream. Will be immediately followed by + * this stream being send-closed. + * - FRAME_OPAQUE_SUB_HEARTBEAT: frame contains no data, but pts+timebase of + * a subtitle heartbeat event. Will only be sent for sub2video streams. + * + * The last stream is "control" - the main thread sends empty AVFrames with + * opaque set to + * - FRAME_OPAQUE_REAP_FILTERS: a request to retrieve all frame available + * from filtergraph outputs. These frames are sent to corresponding + * streams in queue_out. Finally an empty frame is sent to the control + * stream in queue_out. + * - FRAME_OPAQUE_CHOOSE_INPUT: same as above, but in case no frames are + * available the terminating empty frame's opaque will contain the index+1 + * of the filtergraph input to which more input frames should be supplied. + */ + ThreadQueue *queue_in; + /** + * Queue for sending frames from the filtergraph back to the main thread. + * Has nb_outputs+1 streams - the first nb_outputs stream correspond to + * filtergraph outputs. + * + * The last stream is "control" - see documentation for queue_in for more + * details. + */ + ThreadQueue *queue_out; + // submitting frames to filter thread returned EOF + // this only happens on thread exit, so is not per-input + int eof_in; } FilterGraphPriv; static FilterGraphPriv *fgp_from_fg(FilterGraph *fg) @@ -71,9 +118,27 @@ static const FilterGraphPriv *cfgp_from_cfg(const FilterGraph *fg) return (const FilterGraphPriv*)fg; } +// data that is local to the filter thread and not visible outside of it +typedef struct FilterGraphThread { + AVFrame *frame; + + // Temporary buffer for output frames, since on filtergraph reset + // we cannot send them to encoders immediately. + // The output index is stored in frame opaque. + AVFifo *frame_queue_out; + + int got_frame; + + // EOF status of each input/output, as received by the thread + uint8_t *eof_in; + uint8_t *eof_out; +} FilterGraphThread; + typedef struct InputFilterPriv { InputFilter ifilter; + int index; + AVFilterContext *filter; InputStream *ist; @@ -162,6 +227,8 @@ typedef struct FPSConvContext { typedef struct OutputFilterPriv { OutputFilter ofilter; + int index; + AVFilterContext *filter; /* desired output stream properties */ @@ -200,7 +267,25 @@ static OutputFilterPriv *ofp_from_ofilter(OutputFilter *ofilter) return (OutputFilterPriv*)ofilter; } -static int configure_filtergraph(FilterGraph *fg); +typedef struct FilterCommand { + char *target; + char *command; + char *arg; + + double time; + int all_filters; +} FilterCommand; + +static void filter_command_free(void *opaque, uint8_t *data) +{ + FilterCommand *fc = (FilterCommand*)data; + + av_freep(&fc->target); + av_freep(&fc->command); + av_freep(&fc->arg); + + av_free(data); +} static int sub2video_get_blank_frame(InputFilterPriv *ifp) { @@ -570,6 +655,59 @@ static int ifilter_has_all_input_formats(FilterGraph *fg) return 1; } +static void *filter_thread(void *arg); + +// start the filtering thread once all inputs and outputs are bound +static int fg_thread_try_start(FilterGraphPriv *fgp) +{ + FilterGraph *fg = &fgp->fg; + ObjPool *op; + int ret = 0; + + if (fgp->nb_inputs_bound < fg->nb_inputs || + fgp->nb_outputs_bound < fg->nb_outputs) + return 0; + + op = objpool_alloc_frames(); + if (!op) + return AVERROR(ENOMEM); + + fgp->queue_in = tq_alloc(fg->nb_inputs + 1, 1, op, frame_move); + if (!fgp->queue_in) { + objpool_free(&op); + return AVERROR(ENOMEM); + } + + // at least one output is mandatory + op = objpool_alloc_frames(); + if (!op) + goto fail; + + fgp->queue_out = tq_alloc(fg->nb_outputs + 1, 1, op, frame_move); + if (!fgp->queue_out) { + objpool_free(&op); + goto fail; + } + + ret = pthread_create(&fgp->thread, NULL, filter_thread, fgp); + if (ret) { + ret = AVERROR(ret); + av_log(NULL, AV_LOG_ERROR, "pthread_create() for filtergraph %d failed: %s\n", + fg->index, av_err2str(ret)); + goto fail; + } + + return 0; +fail: + if (ret >= 0) + ret = AVERROR(ENOMEM); + + tq_free(&fgp->queue_in); + tq_free(&fgp->queue_out); + + return ret; +} + static char *describe_filter_link(FilterGraph *fg, AVFilterInOut *inout, int in) { AVFilterContext *ctx = inout->filter_ctx; @@ -594,6 +732,7 @@ static OutputFilter *ofilter_alloc(FilterGraph *fg) ofilter = &ofp->ofilter; ofilter->graph = fg; ofp->format = -1; + ofp->index = fg->nb_outputs - 1; ofilter->last_pts = AV_NOPTS_VALUE; return ofilter; @@ -602,6 +741,7 @@ static OutputFilter *ofilter_alloc(FilterGraph *fg) static int ifilter_bind_ist(InputFilter *ifilter, InputStream *ist) { InputFilterPriv *ifp = ifp_from_ifilter(ifilter); + FilterGraphPriv *fgp = fgp_from_fg(ifilter->graph); int ret; av_assert0(!ifp->ist); @@ -619,7 +759,10 @@ static int ifilter_bind_ist(InputFilter *ifilter, InputStream *ist) return AVERROR(ENOMEM); } - return 0; + fgp->nb_inputs_bound++; + av_assert0(fgp->nb_inputs_bound <= ifilter->graph->nb_inputs); + + return fg_thread_try_start(fgp); } static int set_channel_layout(OutputFilterPriv *f, OutputStream *ost) @@ -751,24 +894,10 @@ int ofilter_bind_ost(OutputFilter *ofilter, OutputStream *ost) break; } - // if we have all input parameters and all outputs are bound, - // the graph can now be configured - if (ifilter_has_all_input_formats(fg)) { - int ret; + fgp->nb_outputs_bound++; + av_assert0(fgp->nb_outputs_bound <= fg->nb_outputs); - for (int i = 0; i < fg->nb_outputs; i++) - if (!fg->outputs[i]->ost) - return 0; - - ret = configure_filtergraph(fg); - if (ret < 0) { - av_log(fg, AV_LOG_ERROR, "Error configuring filter graph: %s\n", - av_err2str(ret)); - return ret; - } - } - - return 0; + return fg_thread_try_start(fgp); } static InputFilter *ifilter_alloc(FilterGraph *fg) @@ -787,6 +916,7 @@ static InputFilter *ifilter_alloc(FilterGraph *fg) if (!ifp->frame) return NULL; + ifp->index = fg->nb_inputs - 1; ifp->format = -1; ifp->fallback.format = -1; @@ -797,6 +927,34 @@ static InputFilter *ifilter_alloc(FilterGraph *fg) return ifilter; } +static int fg_thread_stop(FilterGraphPriv *fgp) +{ + void *ret; + + if (!fgp->queue_in) + return 0; + + for (int i = 0; i <= fgp->fg.nb_inputs; i++) { + InputFilterPriv *ifp = i < fgp->fg.nb_inputs ? + ifp_from_ifilter(fgp->fg.inputs[i]) : NULL; + + if (ifp) + ifp->eof = 1; + + tq_send_finish(fgp->queue_in, i); + } + + for (int i = 0; i <= fgp->fg.nb_outputs; i++) + tq_receive_finish(fgp->queue_out, i); + + pthread_join(fgp->thread, &ret); + + tq_free(&fgp->queue_in); + tq_free(&fgp->queue_out); + + return (int)(intptr_t)ret; +} + void fg_free(FilterGraph **pfg) { FilterGraph *fg = *pfg; @@ -806,6 +964,8 @@ void fg_free(FilterGraph **pfg) return; fgp = fgp_from_fg(fg); + fg_thread_stop(fgp); + avfilter_graph_free(&fg->graph); for (int j = 0; j < fg->nb_inputs; j++) { InputFilter *ifilter = fg->inputs[j]; @@ -1603,7 +1763,7 @@ static int graph_is_meta(AVFilterGraph *graph) return 1; } -static int configure_filtergraph(FilterGraph *fg) +static int configure_filtergraph(FilterGraph *fg, const FilterGraphThread *fgt) { FilterGraphPriv *fgp = fgp_from_fg(fg); AVBufferRef *hw_device; @@ -1727,7 +1887,7 @@ static int configure_filtergraph(FilterGraph *fg) /* send the EOFs for the finished inputs */ for (i = 0; i < fg->nb_inputs; i++) { InputFilterPriv *ifp = ifp_from_ifilter(fg->inputs[i]); - if (ifp->eof) { + if (fgt->eof_in[i]) { ret = av_buffersrc_add_frame(ifp->filter, NULL); if (ret < 0) goto fail; @@ -1810,8 +1970,8 @@ int filtergraph_is_simple(const FilterGraph *fg) return fgp->is_simple; } -void fg_send_command(FilterGraph *fg, double time, const char *target, - const char *command, const char *arg, int all_filters) +static void send_command(FilterGraph *fg, double time, const char *target, + const char *command, const char *arg, int all_filters) { int ret; @@ -2069,16 +2229,16 @@ finish: fps->dropped_keyframe |= fps->last_dropped && (frame->flags & AV_FRAME_FLAG_KEY); } -static int fg_output_frame(OutputFilterPriv *ofp, AVFrame *frame) +static int fg_output_frame(OutputFilterPriv *ofp, FilterGraphThread *fgt, + AVFrame *frame, int buffer) { FilterGraphPriv *fgp = fgp_from_fg(ofp->ofilter.graph); - OutputStream *ost = ofp->ofilter.ost; AVFrame *frame_prev = ofp->fps.last_frame; enum AVMediaType type = ofp->ofilter.type; - int64_t nb_frames = 1, nb_frames_prev = 0; + int64_t nb_frames = !!frame, nb_frames_prev = 0; - if (type == AVMEDIA_TYPE_VIDEO) + if (type == AVMEDIA_TYPE_VIDEO && (frame || fgt->got_frame)) video_sync_process(ofp, frame, &nb_frames, &nb_frames_prev); for (int64_t i = 0; i < nb_frames; i++) { @@ -2117,10 +2277,31 @@ static int fg_output_frame(OutputFilterPriv *ofp, AVFrame *frame) frame_out = frame; } - ret = enc_frame(ost, frame_out); - av_frame_unref(frame_out); - if (ret < 0) - return ret; + if (buffer) { + AVFrame *f = av_frame_alloc(); + + if (!f) { + av_frame_unref(frame_out); + return AVERROR(ENOMEM); + } + + av_frame_move_ref(f, frame_out); + f->opaque = (void*)(intptr_t)ofp->index; + + ret = av_fifo_write(fgt->frame_queue_out, &f, 1); + if (ret < 0) { + av_frame_free(&f); + return AVERROR(ENOMEM); + } + } else { + // return the frame to the main thread + ret = tq_send(fgp->queue_out, ofp->index, frame_out); + if (ret < 0) { + av_frame_unref(frame_out); + fgt->eof_out[ofp->index] = 1; + return ret == AVERROR_EOF ? 0 : ret; + } + } if (type == AVMEDIA_TYPE_VIDEO) { ofp->fps.frame_number++; @@ -2130,7 +2311,7 @@ static int fg_output_frame(OutputFilterPriv *ofp, AVFrame *frame) frame->flags &= ~AV_FRAME_FLAG_KEY; } - ofp->got_frame = 1; + fgt->got_frame = 1; } if (frame && frame_prev) { @@ -2138,23 +2319,27 @@ static int fg_output_frame(OutputFilterPriv *ofp, AVFrame *frame) av_frame_move_ref(frame_prev, frame); } + if (!frame) { + tq_send_finish(fgp->queue_out, ofp->index); + fgt->eof_out[ofp->index] = 1; + } + return 0; } -static int fg_output_step(OutputFilterPriv *ofp, int flush) +static int fg_output_step(OutputFilterPriv *ofp, FilterGraphThread *fgt, + AVFrame *frame, int buffer) { FilterGraphPriv *fgp = fgp_from_fg(ofp->ofilter.graph); OutputStream *ost = ofp->ofilter.ost; - AVFrame *frame = fgp->frame; AVFilterContext *filter = ofp->filter; FrameData *fd; int ret; ret = av_buffersink_get_frame_flags(filter, frame, AV_BUFFERSINK_FLAG_NO_REQUEST); - if (flush && ret == AVERROR_EOF && ofp->got_frame && - ost->type == AVMEDIA_TYPE_VIDEO) { - ret = fg_output_frame(ofp, NULL); + if (ret == AVERROR_EOF && !buffer && !fgt->eof_out[ofp->index]) { + ret = fg_output_frame(ofp, fgt, NULL, buffer); return (ret < 0) ? ret : 1; } else if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) { return 1; @@ -2164,22 +2349,18 @@ static int fg_output_step(OutputFilterPriv *ofp, int flush) av_err2str(ret)); return ret; } - if (ost->finished) { + + if (fgt->eof_out[ofp->index]) { av_frame_unref(frame); return 0; } frame->time_base = av_buffersink_get_time_base(filter); - if (frame->pts != AV_NOPTS_VALUE) { - ost->filter->last_pts = av_rescale_q(frame->pts, frame->time_base, - AV_TIME_BASE_Q); - - if (debug_ts) - av_log(fgp, AV_LOG_INFO, "filter_raw -> pts:%s pts_time:%s time_base:%d/%d\n", - av_ts2str(frame->pts), av_ts2timestr(frame->pts, &frame->time_base), - frame->time_base.num, frame->time_base.den); - } + if (debug_ts) + av_log(fgp, AV_LOG_INFO, "filter_raw -> pts:%s pts_time:%s time_base:%d/%d\n", + av_ts2str(frame->pts), av_ts2timestr(frame->pts, &frame->time_base), + frame->time_base.num, frame->time_base.den); // Choose the output timebase the first time we get a frame. if (!ofp->tb_out_locked) { @@ -2212,7 +2393,7 @@ static int fg_output_step(OutputFilterPriv *ofp, int flush) fd->frame_rate_filter = ofp->fps.framerate; } - ret = fg_output_frame(ofp, frame); + ret = fg_output_frame(ofp, fgt, frame, buffer); av_frame_unref(frame); if (ret < 0) return ret; @@ -2220,18 +2401,38 @@ static int fg_output_step(OutputFilterPriv *ofp, int flush) return 0; } -int reap_filters(FilterGraph *fg, int flush) +/* retrieve all frames available at filtergraph outputs and either send them to + * the main thread (buffer=0) or buffer them for later (buffer=1) */ +static int read_frames(FilterGraph *fg, FilterGraphThread *fgt, + AVFrame *frame, int buffer) { + FilterGraphPriv *fgp = fgp_from_fg(fg); + int ret = 0; + if (!fg->graph) return 0; + // process buffered frames + if (!buffer) { + AVFrame *f; + + while (av_fifo_read(fgt->frame_queue_out, &f, 1) >= 0) { + int out_idx = (intptr_t)f->opaque; + f->opaque = NULL; + ret = tq_send(fgp->queue_out, out_idx, f); + av_frame_free(&f); + if (ret < 0 && ret != AVERROR_EOF) + return ret; + } + } + /* Reap all buffers present in the buffer sinks */ for (int i = 0; i < fg->nb_outputs; i++) { OutputFilterPriv *ofp = ofp_from_ofilter(fg->outputs[i]); int ret = 0; while (!ret) { - ret = fg_output_step(ofp, flush); + ret = fg_output_step(ofp, fgt, frame, buffer); if (ret < 0) return ret; } @@ -2240,7 +2441,7 @@ int reap_filters(FilterGraph *fg, int flush) return 0; } -void ifilter_sub2video_heartbeat(InputFilter *ifilter, int64_t pts, AVRational tb) +static void sub2video_heartbeat(InputFilter *ifilter, int64_t pts, AVRational tb) { InputFilterPriv *ifp = ifp_from_ifilter(ifilter); int64_t pts2; @@ -2266,11 +2467,17 @@ void ifilter_sub2video_heartbeat(InputFilter *ifilter, int64_t pts, AVRational t sub2video_push_ref(ifp, pts2); } -int ifilter_sub2video(InputFilter *ifilter, const AVFrame *frame) +static int sub2video_frame(InputFilter *ifilter, const AVFrame *frame) { InputFilterPriv *ifp = ifp_from_ifilter(ifilter); int ret; + // heartbeat frame + if (frame && !frame->buf[0]) { + sub2video_heartbeat(ifilter, frame->pts, frame->time_base); + return 0; + } + if (ifilter->graph->graph) { if (!frame) { if (ifp->sub2video.end_pts < INT64_MAX) @@ -2299,12 +2506,13 @@ int ifilter_sub2video(InputFilter *ifilter, const AVFrame *frame) return 0; } -int ifilter_send_eof(InputFilter *ifilter, int64_t pts, AVRational tb) +static int send_eof(FilterGraphThread *fgt, InputFilter *ifilter, + int64_t pts, AVRational tb) { InputFilterPriv *ifp = ifp_from_ifilter(ifilter); int ret; - ifp->eof = 1; + fgt->eof_in[ifp->index] = 1; if (ifp->filter) { pts = av_rescale_q_rnd(pts, tb, ifp->time_base, @@ -2328,7 +2536,7 @@ int ifilter_send_eof(InputFilter *ifilter, int64_t pts, AVRational tb) return ret; if (ifilter_has_all_input_formats(ifilter->graph)) { - ret = configure_filtergraph(ifilter->graph); + ret = configure_filtergraph(ifilter->graph, fgt); if (ret < 0) { av_log(NULL, AV_LOG_ERROR, "Error initializing filters!\n"); return ret; @@ -2347,10 +2555,10 @@ int ifilter_send_eof(InputFilter *ifilter, int64_t pts, AVRational tb) return 0; } -int ifilter_send_frame(InputFilter *ifilter, AVFrame *frame, int keep_reference) +static int send_frame(FilterGraph *fg, FilterGraphThread *fgt, + InputFilter *ifilter, AVFrame *frame) { InputFilterPriv *ifp = ifp_from_ifilter(ifilter); - FilterGraph *fg = ifilter->graph; AVFrameSideData *sd; int need_reinit, ret; @@ -2390,10 +2598,13 @@ int ifilter_send_frame(InputFilter *ifilter, AVFrame *frame, int keep_reference) /* (re)init the graph if possible, otherwise buffer the frame and return */ if (need_reinit || !fg->graph) { + AVFrame *tmp = av_frame_alloc(); + + if (!tmp) + return AVERROR(ENOMEM); + if (!ifilter_has_all_input_formats(fg)) { - AVFrame *tmp = av_frame_clone(frame); - if (!tmp) - return AVERROR(ENOMEM); + av_frame_move_ref(tmp, frame); ret = av_fifo_write(ifp->frame_queue, &tmp, 1); if (ret < 0) @@ -2402,27 +2613,18 @@ int ifilter_send_frame(InputFilter *ifilter, AVFrame *frame, int keep_reference) return ret; } - ret = reap_filters(fg, 0); - if (ret < 0 && ret != AVERROR_EOF) { - av_log(fg, AV_LOG_ERROR, "Error while filtering: %s\n", av_err2str(ret)); + ret = read_frames(fg, fgt, tmp, 1); + av_frame_free(&tmp); + if (ret < 0) return ret; - } - ret = configure_filtergraph(fg); + ret = configure_filtergraph(fg, fgt); if (ret < 0) { av_log(fg, AV_LOG_ERROR, "Error reinitializing filters!\n"); return ret; } } - if (keep_reference) { - ret = av_frame_ref(ifp->frame, frame); - if (ret < 0) - return ret; - } else - av_frame_move_ref(ifp->frame, frame); - frame = ifp->frame; - frame->pts = av_rescale_q(frame->pts, frame->time_base, ifp->time_base); frame->duration = av_rescale_q(frame->duration, frame->time_base, ifp->time_base); frame->time_base = ifp->time_base; @@ -2444,20 +2646,53 @@ int ifilter_send_frame(InputFilter *ifilter, AVFrame *frame, int keep_reference) return 0; } -int fg_transcode_step(FilterGraph *graph, InputStream **best_ist) +static int choose_input(const FilterGraph *fg, const FilterGraphThread *fgt) { - FilterGraphPriv *fgp = fgp_from_fg(graph); - int i, ret; int nb_requests, nb_requests_max = 0; - InputStream *ist; + int best_input = -1; - if (!graph->graph) { - for (int i = 0; i < graph->nb_inputs; i++) { - InputFilter *ifilter = graph->inputs[i]; + for (int i = 0; i < fg->nb_inputs; i++) { + InputFilter *ifilter = fg->inputs[i]; + InputFilterPriv *ifp = ifp_from_ifilter(ifilter); + InputStream *ist = ifp->ist; + + if (input_files[ist->file_index]->eagain || fgt->eof_in[i]) + continue; + + nb_requests = av_buffersrc_get_nb_failed_requests(ifp->filter); + if (nb_requests > nb_requests_max) { + nb_requests_max = nb_requests; + best_input = i; + } + } + + return best_input; +} + +static int msg_process(FilterGraphPriv *fgp, FilterGraphThread *fgt, + AVFrame *frame) +{ + const enum FrameOpaque msg = (intptr_t)frame->opaque; + FilterGraph *fg = &fgp->fg; + int graph_eof = 0; + int ret; + + frame->opaque = NULL; + av_assert0(msg > 0); + av_assert0(msg == FRAME_OPAQUE_SEND_COMMAND || !frame->buf[0]); + + if (!fg->graph) { + // graph not configured yet, ignore all messages other than choosing + // the input to read from + if (msg != FRAME_OPAQUE_CHOOSE_INPUT) + goto done; + + for (int i = 0; i < fg->nb_inputs; i++) { + InputFilter *ifilter = fg->inputs[i]; InputFilterPriv *ifp = ifp_from_ifilter(ifilter); - if (ifp->format < 0 && !ifp->eof) { - *best_ist = ifp->ist; - return 0; + if (ifp->format < 0 && !fgt->eof_in[i]) { + frame->opaque = (void*)(intptr_t)(i + 1); + goto done; } } @@ -2468,16 +2703,309 @@ int fg_transcode_step(FilterGraph *graph, InputStream **best_ist) return AVERROR_BUG; } - *best_ist = NULL; - ret = avfilter_graph_request_oldest(graph->graph); - if (ret >= 0) - return reap_filters(graph, 0); + if (msg == FRAME_OPAQUE_SEND_COMMAND) { + FilterCommand *fc = (FilterCommand*)frame->buf[0]->data; + send_command(fg, fc->time, fc->target, fc->command, fc->arg, fc->all_filters); + av_frame_unref(frame); + goto done; + } - if (ret == AVERROR_EOF) { - reap_filters(graph, 1); - for (int i = 0; i < graph->nb_outputs; i++) { - OutputFilter *ofilter = graph->outputs[i]; - OutputFilterPriv *ofp = ofp_from_ofilter(ofilter); + if (msg == FRAME_OPAQUE_CHOOSE_INPUT) { + ret = avfilter_graph_request_oldest(fg->graph); + + graph_eof = ret == AVERROR_EOF; + + if (ret == AVERROR(EAGAIN)) { + frame->opaque = (void*)(intptr_t)(choose_input(fg, fgt) + 1); + goto done; + } else if (ret < 0 && !graph_eof) + return ret; + } + + ret = read_frames(fg, fgt, frame, 0); + if (ret < 0) { + av_log(fg, AV_LOG_ERROR, "Error sending filtered frames for encoding\n"); + return ret; + } + + if (graph_eof) + return AVERROR_EOF; + + // signal to the main thread that we are done processing the message +done: + ret = tq_send(fgp->queue_out, fg->nb_outputs, frame); + if (ret < 0) { + if (ret != AVERROR_EOF) + av_log(fg, AV_LOG_ERROR, "Error communicating with the main thread\n"); + return ret; + } + + return 0; +} + +static void fg_thread_set_name(const FilterGraph *fg) +{ + char name[16]; + if (filtergraph_is_simple(fg)) { + OutputStream *ost = fg->outputs[0]->ost; + snprintf(name, sizeof(name), "%cf#%d:%d", + av_get_media_type_string(ost->type)[0], + ost->file_index, ost->index); + } else { + snprintf(name, sizeof(name), "fc%d", fg->index); + } + + ff_thread_setname(name); +} + +static void fg_thread_uninit(FilterGraphThread *fgt) +{ + if (fgt->frame_queue_out) { + AVFrame *frame; + while (av_fifo_read(fgt->frame_queue_out, &frame, 1) >= 0) + av_frame_free(&frame); + av_fifo_freep2(&fgt->frame_queue_out); + } + + av_frame_free(&fgt->frame); + av_freep(&fgt->eof_in); + av_freep(&fgt->eof_out); + + memset(fgt, 0, sizeof(*fgt)); +} + +static int fg_thread_init(FilterGraphThread *fgt, const FilterGraph *fg) +{ + memset(fgt, 0, sizeof(*fgt)); + + fgt->frame = av_frame_alloc(); + if (!fgt->frame) + goto fail; + + fgt->eof_in = av_calloc(fg->nb_inputs, sizeof(*fgt->eof_in)); + if (!fgt->eof_in) + goto fail; + + fgt->eof_out = av_calloc(fg->nb_outputs, sizeof(*fgt->eof_out)); + if (!fgt->eof_out) + goto fail; + + fgt->frame_queue_out = av_fifo_alloc2(1, sizeof(AVFrame*), AV_FIFO_FLAG_AUTO_GROW); + if (!fgt->frame_queue_out) + goto fail; + + return 0; + +fail: + fg_thread_uninit(fgt); + return AVERROR(ENOMEM); +} + +static void *filter_thread(void *arg) +{ + FilterGraphPriv *fgp = arg; + FilterGraph *fg = &fgp->fg; + + FilterGraphThread fgt; + int ret = 0, input_status = 0; + + ret = fg_thread_init(&fgt, fg); + if (ret < 0) + goto finish; + + fg_thread_set_name(fg); + + // if we have all input parameters the graph can now be configured + if (ifilter_has_all_input_formats(fg)) { + ret = configure_filtergraph(fg, &fgt); + if (ret < 0) { + av_log(fg, AV_LOG_ERROR, "Error configuring filter graph: %s\n", + av_err2str(ret)); + goto finish; + } + } + + while (1) { + InputFilter *ifilter; + InputFilterPriv *ifp; + int input_idx, eof_frame; + + input_status = tq_receive(fgp->queue_in, &input_idx, fgt.frame); + if (input_idx < 0 || + (input_idx == fg->nb_inputs && input_status < 0)) { + av_log(fg, AV_LOG_VERBOSE, "Filtering thread received EOF\n"); + break; + } + + // message on the control stream + if (input_idx == fg->nb_inputs) { + ret = msg_process(fgp, &fgt, fgt.frame); + if (ret < 0) + goto finish; + + continue; + } + + // we received an input frame or EOF + ifilter = fg->inputs[input_idx]; + ifp = ifp_from_ifilter(ifilter); + eof_frame = input_status >= 0 && (intptr_t)fgt.frame->opaque == FRAME_OPAQUE_EOF; + if (ifp->type_src == AVMEDIA_TYPE_SUBTITLE) { + if (input_status >= 0 && (intptr_t)fgt.frame->opaque == FRAME_OPAQUE_SUB_HEARTBEAT) + sub2video_heartbeat(ifilter, fgt.frame->pts, fgt.frame->time_base); + else + ret = sub2video_frame(ifilter, fgt.frame->buf[0] ? fgt.frame : NULL); + } else if (input_status >= 0 && fgt.frame->buf[0]) { + ret = send_frame(fg, &fgt, ifilter, fgt.frame); + } else { + int64_t pts = input_status >= 0 ? fgt.frame->pts : AV_NOPTS_VALUE; + AVRational tb = input_status >= 0 ? fgt.frame->time_base : (AVRational){ 1, 1 }; + ret = send_eof(&fgt, ifilter, pts, tb); + } + av_frame_unref(fgt.frame); + if (ret < 0) + break; + + if (eof_frame) { + // an EOF frame is immediately followed by sender closing + // the corresponding stream, so retrieve that event + input_status = tq_receive(fgp->queue_in, &input_idx, fgt.frame); + av_assert0(input_status == AVERROR_EOF && input_idx == ifp->index); + } + + // signal to the main thread that we are done + ret = tq_send(fgp->queue_out, fg->nb_outputs, fgt.frame); + if (ret < 0) { + if (ret == AVERROR_EOF) + break; + + av_log(fg, AV_LOG_ERROR, "Error communicating with the main thread\n"); + goto finish; + } + } + +finish: + // EOF is normal termination + if (ret == AVERROR_EOF) + ret = 0; + + for (int i = 0; i <= fg->nb_inputs; i++) + tq_receive_finish(fgp->queue_in, i); + for (int i = 0; i <= fg->nb_outputs; i++) + tq_send_finish(fgp->queue_out, i); + + fg_thread_uninit(&fgt); + + av_log(fg, AV_LOG_VERBOSE, "Terminating filtering thread\n"); + + return (void*)(intptr_t)ret; +} + +static int thread_send_frame(FilterGraphPriv *fgp, InputFilter *ifilter, + AVFrame *frame, enum FrameOpaque type) +{ + InputFilterPriv *ifp = ifp_from_ifilter(ifilter); + int output_idx, ret; + + if (ifp->eof) { + av_frame_unref(frame); + return AVERROR_EOF; + } + + frame->opaque = (void*)(intptr_t)type; + + ret = tq_send(fgp->queue_in, ifp->index, frame); + if (ret < 0) { + ifp->eof = 1; + av_frame_unref(frame); + return ret; + } + + if (type == FRAME_OPAQUE_EOF) + tq_send_finish(fgp->queue_in, ifp->index); + + // wait for the frame to be processed + ret = tq_receive(fgp->queue_out, &output_idx, frame); + av_assert0(output_idx == fgp->fg.nb_outputs || ret == AVERROR_EOF); + + return ret; +} + +int ifilter_send_frame(InputFilter *ifilter, AVFrame *frame, int keep_reference) +{ + FilterGraphPriv *fgp = fgp_from_fg(ifilter->graph); + int ret; + + if (keep_reference) { + ret = av_frame_ref(fgp->frame, frame); + if (ret < 0) + return ret; + } else + av_frame_move_ref(fgp->frame, frame); + + return thread_send_frame(fgp, ifilter, fgp->frame, 0); +} + +int ifilter_send_eof(InputFilter *ifilter, int64_t pts, AVRational tb) +{ + FilterGraphPriv *fgp = fgp_from_fg(ifilter->graph); + int ret; + + fgp->frame->pts = pts; + fgp->frame->time_base = tb; + + ret = thread_send_frame(fgp, ifilter, fgp->frame, FRAME_OPAQUE_EOF); + + return ret == AVERROR_EOF ? 0 : ret; +} + +void ifilter_sub2video_heartbeat(InputFilter *ifilter, int64_t pts, AVRational tb) +{ + FilterGraphPriv *fgp = fgp_from_fg(ifilter->graph); + + fgp->frame->pts = pts; + fgp->frame->time_base = tb; + + thread_send_frame(fgp, ifilter, fgp->frame, FRAME_OPAQUE_SUB_HEARTBEAT); +} + +int fg_transcode_step(FilterGraph *graph, InputStream **best_ist) +{ + FilterGraphPriv *fgp = fgp_from_fg(graph); + int ret, got_frames = 0; + + if (fgp->eof_in) + return AVERROR_EOF; + + // signal to the filtering thread to return all frames it can + av_assert0(!fgp->frame->buf[0]); + fgp->frame->opaque = (void*)(intptr_t)(best_ist ? + FRAME_OPAQUE_CHOOSE_INPUT : + FRAME_OPAQUE_REAP_FILTERS); + + ret = tq_send(fgp->queue_in, graph->nb_inputs, fgp->frame); + if (ret < 0) { + fgp->eof_in = 1; + goto finish; + } + + while (1) { + OutputFilter *ofilter; + OutputFilterPriv *ofp; + OutputStream *ost; + int output_idx; + + ret = tq_receive(fgp->queue_out, &output_idx, fgp->frame); + + // EOF on the whole queue or the control stream + if (output_idx < 0 || + (ret < 0 && output_idx == graph->nb_outputs)) + goto finish; + + // EOF for a specific stream + if (ret < 0) { + ofilter = graph->outputs[output_idx]; + ofp = ofp_from_ofilter(ofilter); // we are finished and no frames were ever seen at this output, // at least initialize the encoder with a dummy frame @@ -2515,30 +3043,111 @@ int fg_transcode_step(FilterGraph *graph, InputStream **best_ist) av_frame_unref(frame); } - close_output_stream(ofilter->ost); - } - return 0; - } - if (ret != AVERROR(EAGAIN)) - return ret; - - for (i = 0; i < graph->nb_inputs; i++) { - InputFilter *ifilter = graph->inputs[i]; - InputFilterPriv *ifp = ifp_from_ifilter(ifilter); - - ist = ifp->ist; - if (input_files[ist->file_index]->eagain || ifp->eof) + close_output_stream(graph->outputs[output_idx]->ost); continue; - nb_requests = av_buffersrc_get_nb_failed_requests(ifp->filter); - if (nb_requests > nb_requests_max) { - nb_requests_max = nb_requests; - *best_ist = ist; } + + // request was fully processed by the filtering thread, + // return the input stream to read from, if needed + if (output_idx == graph->nb_outputs) { + int input_idx = (intptr_t)fgp->frame->opaque - 1; + av_assert0(input_idx <= graph->nb_inputs); + + if (best_ist) { + *best_ist = (input_idx >= 0 && input_idx < graph->nb_inputs) ? + ifp_from_ifilter(graph->inputs[input_idx])->ist : NULL; + + if (input_idx < 0 && !got_frames) { + for (int i = 0; i < graph->nb_outputs; i++) + graph->outputs[i]->ost->unavailable = 1; + } + } + break; + } + + // got a frame from the filtering thread, send it for encoding + ofilter = graph->outputs[output_idx]; + ost = ofilter->ost; + ofp = ofp_from_ofilter(ofilter); + + if (ost->finished) { + av_frame_unref(fgp->frame); + tq_receive_finish(fgp->queue_out, output_idx); + continue; + } + + if (fgp->frame->pts != AV_NOPTS_VALUE) { + ofilter->last_pts = av_rescale_q(fgp->frame->pts, + fgp->frame->time_base, + AV_TIME_BASE_Q); + } + + ret = enc_frame(ost, fgp->frame); + av_frame_unref(fgp->frame); + if (ret < 0) + goto finish; + + ofp->got_frame = 1; + got_frames = 1; } - if (!*best_ist) - for (i = 0; i < graph->nb_outputs; i++) - graph->outputs[i]->ost->unavailable = 1; +finish: + if (ret < 0) { + fgp->eof_in = 1; + for (int i = 0; i < graph->nb_outputs; i++) + close_output_stream(graph->outputs[i]->ost); + } - return 0; + return ret; +} + +int reap_filters(FilterGraph *fg, int flush) +{ + return fg_transcode_step(fg, NULL); +} + +void fg_send_command(FilterGraph *fg, double time, const char *target, + const char *command, const char *arg, int all_filters) +{ + FilterGraphPriv *fgp = fgp_from_fg(fg); + AVBufferRef *buf; + FilterCommand *fc; + int output_idx, ret; + + if (!fgp->queue_in) + return; + + fc = av_mallocz(sizeof(*fc)); + if (!fc) + return; + + buf = av_buffer_create((uint8_t*)fc, sizeof(*fc), filter_command_free, NULL, 0); + if (!buf) { + av_freep(&fc); + return; + } + + fc->target = av_strdup(target); + fc->command = av_strdup(command); + fc->arg = av_strdup(arg); + if (!fc->target || !fc->command || !fc->arg) { + av_buffer_unref(&buf); + return; + } + + fc->time = time; + fc->all_filters = all_filters; + + fgp->frame->buf[0] = buf; + fgp->frame->opaque = (void*)(intptr_t)FRAME_OPAQUE_SEND_COMMAND; + + ret = tq_send(fgp->queue_in, fg->nb_inputs + 1, fgp->frame); + if (ret < 0) { + av_frame_unref(fgp->frame); + return; + } + + // wait for the frame to be processed + ret = tq_receive(fgp->queue_out, &output_idx, fgp->frame); + av_assert0(output_idx == fgp->fg.nb_outputs || ret == AVERROR_EOF); } -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 15/27] fftools/ffmpeg_mux: add muxing thread private data 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (13 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 14/27] fftools/ffmpeg_filter: move filtering to a separate thread Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 16/27] fftools/ffmpeg_demux: switch from AVThreadMessageQueue to ThreadQueue Anton Khirnov ` (12 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel To be used for data that never needs to be visible outside of the muxer thread. Start by moving the muxed AVPacket in there. --- fftools/ffmpeg_mux.c | 44 +++++++++++++++++++++++++++++++++++--------- 1 file changed, 35 insertions(+), 9 deletions(-) diff --git a/fftools/ffmpeg_mux.c b/fftools/ffmpeg_mux.c index 7a924dba6c..033894ae86 100644 --- a/fftools/ffmpeg_mux.c +++ b/fftools/ffmpeg_mux.c @@ -38,6 +38,10 @@ #include "libavformat/avformat.h" #include "libavformat/avio.h" +typedef struct MuxThreadContext { + AVPacket *pkt; +} MuxThreadContext; + int want_sdp = 1; static Muxer *mux_from_of(OutputFile *of) @@ -209,18 +213,40 @@ static void thread_set_name(OutputFile *of) ff_thread_setname(name); } +static void mux_thread_uninit(MuxThreadContext *mt) +{ + av_packet_free(&mt->pkt); + + memset(mt, 0, sizeof(*mt)); +} + +static int mux_thread_init(MuxThreadContext *mt) +{ + memset(mt, 0, sizeof(*mt)); + + mt->pkt = av_packet_alloc(); + if (!mt->pkt) + goto fail; + + return 0; + +fail: + mux_thread_uninit(mt); + return AVERROR(ENOMEM); +} + static void *muxer_thread(void *arg) { Muxer *mux = arg; OutputFile *of = &mux->of; - AVPacket *pkt = NULL; + + MuxThreadContext mt; + int ret = 0; - pkt = av_packet_alloc(); - if (!pkt) { - ret = AVERROR(ENOMEM); + ret = mux_thread_init(&mt); + if (ret < 0) goto finish; - } thread_set_name(of); @@ -228,7 +254,7 @@ static void *muxer_thread(void *arg) OutputStream *ost; int stream_idx, stream_eof = 0; - ret = tq_receive(mux->tq, &stream_idx, pkt); + ret = tq_receive(mux->tq, &stream_idx, mt.pkt); if (stream_idx < 0) { av_log(mux, AV_LOG_VERBOSE, "All streams finished\n"); ret = 0; @@ -236,8 +262,8 @@ static void *muxer_thread(void *arg) } ost = of->streams[stream_idx]; - ret = sync_queue_process(mux, ost, ret < 0 ? NULL : pkt, &stream_eof); - av_packet_unref(pkt); + ret = sync_queue_process(mux, ost, ret < 0 ? NULL : mt.pkt, &stream_eof); + av_packet_unref(mt.pkt); if (ret == AVERROR_EOF) { if (stream_eof) { tq_receive_finish(mux->tq, stream_idx); @@ -253,7 +279,7 @@ static void *muxer_thread(void *arg) } finish: - av_packet_free(&pkt); + mux_thread_uninit(&mt); for (unsigned int i = 0; i < mux->fc->nb_streams; i++) tq_receive_finish(mux->tq, i); -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 16/27] fftools/ffmpeg_demux: switch from AVThreadMessageQueue to ThreadQueue 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (14 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 15/27] fftools/ffmpeg_mux: add muxing thread private data Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 17/27] XXX: disable fix_sub_duration_heartbeat Anton Khirnov ` (11 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel * the code is made shorter and simpler * avoids constantly allocating and freeing AVPackets, thanks to ThreadQueue integration with ObjPool * is consistent with decoding/filtering/muxing * reduces the diff in the future switch to thread-aware scheduling This makes ifile_get_packet() always block. Any potential issues caused by this will be resolved by the switch to thread-aware scheduling in future commits. --- fftools/ffmpeg.c | 32 +++++------ fftools/ffmpeg.h | 14 ++++- fftools/ffmpeg_demux.c | 116 +++++++++++++++------------------------- fftools/ffmpeg_filter.c | 2 +- 4 files changed, 71 insertions(+), 93 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 7c33b56cd3..7d6972f689 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -1029,9 +1029,6 @@ static int check_keyboard_interaction(int64_t cur_time) static void reset_eagain(void) { - int i; - for (i = 0; i < nb_input_files; i++) - input_files[i]->eagain = 0; for (OutputStream *ost = ost_iter(NULL); ost; ost = ost_iter(ost)) ost->unavailable = 0; } @@ -1055,19 +1052,14 @@ static void decode_flush(InputFile *ifile) * this function should be called again * - AVERROR_EOF -- this function should not be called again */ -static int process_input(int file_index) +static int process_input(int file_index, AVPacket *pkt) { InputFile *ifile = input_files[file_index]; InputStream *ist; - AVPacket *pkt; int ret, i; - ret = ifile_get_packet(ifile, &pkt); + ret = ifile_get_packet(ifile, pkt); - if (ret == AVERROR(EAGAIN)) { - ifile->eagain = 1; - return ret; - } if (ret == 1) { /* the input file is looped: flush the decoders */ decode_flush(ifile); @@ -1114,7 +1106,7 @@ static int process_input(int file_index) ret = process_input_packet(ist, pkt, 0); - av_packet_free(&pkt); + av_packet_unref(pkt); return ret < 0 ? ret : 0; } @@ -1124,7 +1116,7 @@ static int process_input(int file_index) * * @return 0 for success, <0 for error */ -static int transcode_step(OutputStream *ost) +static int transcode_step(OutputStream *ost, AVPacket *demux_pkt) { InputStream *ist = NULL; int ret; @@ -1139,10 +1131,8 @@ static int transcode_step(OutputStream *ost) av_assert0(ist); } - ret = process_input(ist->file_index); + ret = process_input(ist->file_index, demux_pkt); if (ret == AVERROR(EAGAIN)) { - if (input_files[ist->file_index]->eagain) - ost->unavailable = 1; return 0; } @@ -1168,12 +1158,19 @@ static int transcode(int *err_rate_exceeded) int ret = 0, i; InputStream *ist; int64_t timer_start; + AVPacket *demux_pkt = NULL; print_stream_maps(); *err_rate_exceeded = 0; atomic_store(&transcode_init_done, 1); + demux_pkt = av_packet_alloc(); + if (!demux_pkt) { + ret = AVERROR(ENOMEM); + goto fail; + } + if (stdin_interaction) { av_log(NULL, AV_LOG_INFO, "Press [q] to stop, [?] for help\n"); } @@ -1201,7 +1198,7 @@ static int transcode(int *err_rate_exceeded) break; } - ret = transcode_step(ost); + ret = transcode_step(ost, demux_pkt); if (ret < 0 && ret != AVERROR_EOF) { av_log(NULL, AV_LOG_ERROR, "Error while filtering: %s\n", av_err2str(ret)); break; @@ -1242,6 +1239,9 @@ static int transcode(int *err_rate_exceeded) /* dump report by using the first video and audio streams */ print_report(1, timer_start, av_gettime_relative()); +fail: + av_packet_free(&demux_pkt); + return ret; } diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 5a8f52ce00..88b8ed12c0 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -101,6 +101,17 @@ typedef struct { } AudioChannelMap; #endif +/** + * AVPacket.opaque values we use. + */ +enum PacketOpaque { + /** + * Sent by demuxers after seeking, so that decoders should be flushed. + * The packet wit this value is otherwise always empty. + */ + PKT_OPAQUE_SEEK = 1, +}; + typedef struct DemuxPktData { // estimated dts in AV_TIME_BASE_Q, // to be used when real dts is missing @@ -406,7 +417,6 @@ typedef struct InputFile { AVFormatContext *ctx; int eof_reached; /* true if eof reached */ - int eagain; /* true if last read attempt returned EAGAIN */ int64_t input_ts_offset; int input_sync_ref; /** @@ -856,7 +866,7 @@ void ifile_close(InputFile **f); * caller should flush decoders and read from this demuxer again * - a negative error code on failure */ -int ifile_get_packet(InputFile *f, AVPacket **pkt); +int ifile_get_packet(InputFile *f, AVPacket *pkt); int ist_output_add(InputStream *ist, OutputStream *ost); int ist_filter_add(InputStream *ist, InputFilter *ifilter, int is_simple); diff --git a/fftools/ffmpeg_demux.c b/fftools/ffmpeg_demux.c index c01852d4cf..0955956117 100644 --- a/fftools/ffmpeg_demux.c +++ b/fftools/ffmpeg_demux.c @@ -20,6 +20,8 @@ #include <stdint.h> #include "ffmpeg.h" +#include "objpool.h" +#include "thread_queue.h" #include "libavutil/avassert.h" #include "libavutil/avstring.h" @@ -32,7 +34,6 @@ #include "libavutil/time.h" #include "libavutil/timestamp.h" #include "libavutil/thread.h" -#include "libavutil/threadmessage.h" #include "libavcodec/packet.h" @@ -109,19 +110,13 @@ typedef struct Demuxer { double readrate_initial_burst; - AVThreadMessageQueue *in_thread_queue; + ThreadQueue *thread_queue; int thread_queue_size; pthread_t thread; - int non_blocking; int read_started; } Demuxer; -typedef struct DemuxMsg { - AVPacket *pkt; - int looping; -} DemuxMsg; - static DemuxStream *ds_from_ist(InputStream *ist) { return (DemuxStream*)ist; @@ -456,26 +451,16 @@ static int ts_fixup(Demuxer *d, AVPacket *pkt) return 0; } -// process an input packet into a message to send to the consumer thread -// src is always cleared by this function -static int input_packet_process(Demuxer *d, DemuxMsg *msg, AVPacket *src) +static int input_packet_process(Demuxer *d, AVPacket *pkt) { InputFile *f = &d->f; - InputStream *ist = f->streams[src->stream_index]; + InputStream *ist = f->streams[pkt->stream_index]; DemuxStream *ds = ds_from_ist(ist); - AVPacket *pkt; int ret = 0; - pkt = av_packet_alloc(); - if (!pkt) { - av_packet_unref(src); - return AVERROR(ENOMEM); - } - av_packet_move_ref(pkt, src); - ret = ts_fixup(d, pkt); if (ret < 0) - goto fail; + return ret; ds->data_size += pkt->size; ds->nb_packets++; @@ -493,10 +478,8 @@ static int input_packet_process(Demuxer *d, DemuxMsg *msg, AVPacket *src) continue; dst_data = av_packet_new_side_data(pkt, src_sd->type, src_sd->size); - if (!dst_data) { - ret = AVERROR(ENOMEM); - goto fail; - } + if (!dst_data) + return AVERROR(ENOMEM); memcpy(dst_data, src_sd->data, src_sd->size); } @@ -513,13 +496,7 @@ static int input_packet_process(Demuxer *d, DemuxMsg *msg, AVPacket *src) av_ts2timestr(input_files[ist->file_index]->ts_offset, &AV_TIME_BASE_Q)); } - msg->pkt = pkt; - pkt = NULL; - -fail: - av_packet_free(&pkt); - - return ret; + return 0; } static void readrate_sleep(Demuxer *d) @@ -569,7 +546,6 @@ static void *input_thread(void *arg) Demuxer *d = arg; InputFile *f = &d->f; AVPacket *pkt; - unsigned flags = d->non_blocking ? AV_THREAD_MESSAGE_NONBLOCK : 0; int ret = 0; pkt = av_packet_alloc(); @@ -585,8 +561,6 @@ static void *input_thread(void *arg) d->wallclock_start = av_gettime_relative(); while (1) { - DemuxMsg msg = { NULL }; - ret = av_read_frame(f->ctx, pkt); if (ret == AVERROR(EAGAIN)) { @@ -596,8 +570,8 @@ static void *input_thread(void *arg) if (ret < 0) { if (d->loop) { /* signal looping to the consumer thread */ - msg.looping = 1; - ret = av_thread_message_queue_send(d->in_thread_queue, &msg, 0); + pkt->opaque = (void*)(intptr_t)PKT_OPAQUE_SEEK; + ret = tq_send(d->thread_queue, 0, pkt); if (ret >= 0) ret = seek_to_start(d); if (ret >= 0) @@ -640,35 +614,26 @@ static void *input_thread(void *arg) } } - ret = input_packet_process(d, &msg, pkt); + ret = input_packet_process(d, pkt); if (ret < 0) break; if (f->readrate) readrate_sleep(d); - ret = av_thread_message_queue_send(d->in_thread_queue, &msg, flags); - if (flags && ret == AVERROR(EAGAIN)) { - flags = 0; - ret = av_thread_message_queue_send(d->in_thread_queue, &msg, flags); - av_log(f, AV_LOG_WARNING, - "Thread message queue blocking; consider raising the " - "thread_queue_size option (current value: %d)\n", - d->thread_queue_size); - } + ret = tq_send(d->thread_queue, 0, pkt); if (ret < 0) { if (ret != AVERROR_EOF) av_log(f, AV_LOG_ERROR, "Unable to send packet to main thread: %s\n", av_err2str(ret)); - av_packet_free(&msg.pkt); break; } } finish: av_assert0(ret < 0); - av_thread_message_queue_set_err_recv(d->in_thread_queue, ret); + tq_send_finish(d->thread_queue, 0); av_packet_free(&pkt); @@ -680,16 +645,16 @@ finish: static void thread_stop(Demuxer *d) { InputFile *f = &d->f; - DemuxMsg msg; - if (!d->in_thread_queue) + if (!d->thread_queue) return; - av_thread_message_queue_set_err_send(d->in_thread_queue, AVERROR_EOF); - while (av_thread_message_queue_recv(d->in_thread_queue, &msg, 0) >= 0) - av_packet_free(&msg.pkt); + + tq_receive_finish(d->thread_queue, 0); pthread_join(d->thread, NULL); - av_thread_message_queue_free(&d->in_thread_queue); + + tq_free(&d->thread_queue); + av_thread_message_queue_free(&f->audio_duration_queue); } @@ -697,18 +662,20 @@ static int thread_start(Demuxer *d) { int ret; InputFile *f = &d->f; + ObjPool *op; if (d->thread_queue_size <= 0) d->thread_queue_size = (nb_input_files > 1 ? 8 : 1); - if (nb_input_files > 1 && - (f->ctx->pb ? !f->ctx->pb->seekable : - strcmp(f->ctx->iformat->name, "lavfi"))) - d->non_blocking = 1; - ret = av_thread_message_queue_alloc(&d->in_thread_queue, - d->thread_queue_size, sizeof(DemuxMsg)); - if (ret < 0) - return ret; + op = objpool_alloc_packets(); + if (!op) + return AVERROR(ENOMEM); + + d->thread_queue = tq_alloc(1, d->thread_queue_size, op, pkt_move); + if (!d->thread_queue) { + objpool_free(&op); + return AVERROR(ENOMEM); + } if (d->loop) { int nb_audio_dec = 0; @@ -738,31 +705,32 @@ static int thread_start(Demuxer *d) return 0; fail: - av_thread_message_queue_free(&d->in_thread_queue); + tq_free(&d->thread_queue); return ret; } -int ifile_get_packet(InputFile *f, AVPacket **pkt) +int ifile_get_packet(InputFile *f, AVPacket *pkt) { Demuxer *d = demuxer_from_ifile(f); - DemuxMsg msg; - int ret; + int ret, dummy; - if (!d->in_thread_queue) { + if (!d->thread_queue) { ret = thread_start(d); if (ret < 0) return ret; } - ret = av_thread_message_queue_recv(d->in_thread_queue, &msg, - d->non_blocking ? - AV_THREAD_MESSAGE_NONBLOCK : 0); + ret = tq_receive(d->thread_queue, &dummy, pkt); if (ret < 0) return ret; - if (msg.looping) - return 1; - *pkt = msg.pkt; + if (pkt->opaque) { + av_assert0((intptr_t)pkt->opaque == PKT_OPAQUE_SEEK && + !pkt->data && !pkt->side_data_elems); + pkt->opaque = NULL; + return 1; + } + return 0; } diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index cfd13dd81a..04c4b4ea7b 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -2656,7 +2656,7 @@ static int choose_input(const FilterGraph *fg, const FilterGraphThread *fgt) InputFilterPriv *ifp = ifp_from_ifilter(ifilter); InputStream *ist = ifp->ist; - if (input_files[ist->file_index]->eagain || fgt->eof_in[i]) + if (fgt->eof_in[i]) continue; nb_requests = av_buffersrc_get_nb_failed_requests(ifp->filter); -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 17/27] XXX: disable fix_sub_duration_heartbeat 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (15 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 16/27] fftools/ffmpeg_demux: switch from AVThreadMessageQueue to ThreadQueue Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 18/27] XXX fftools/ffmpeg_enc: temporarily disable side data copying Anton Khirnov ` (10 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel It conflicts with threading work. --- fftools/ffmpeg.c | 2 ++ tests/fate/ffmpeg.mak | 24 ++++++++++++------------ 2 files changed, 14 insertions(+), 12 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 7d6972f689..e084318864 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -770,6 +770,7 @@ int subtitle_wrap_frame(AVFrame *frame, AVSubtitle *subtitle, int copy) int trigger_fix_sub_duration_heartbeat(OutputStream *ost, const AVPacket *pkt) { +#if 0 OutputFile *of = output_files[ost->file_index]; int64_t signal_pts = av_rescale_q(pkt->pts, pkt->time_base, AV_TIME_BASE_Q); @@ -795,6 +796,7 @@ int trigger_fix_sub_duration_heartbeat(OutputStream *ost, const AVPacket *pkt) if ((ret = fix_sub_duration_heartbeat(ist, signal_pts)) < 0) return ret; } +#endif return 0; } diff --git a/tests/fate/ffmpeg.mak b/tests/fate/ffmpeg.mak index 835770a924..ebc1e1f189 100644 --- a/tests/fate/ffmpeg.mak +++ b/tests/fate/ffmpeg.mak @@ -139,18 +139,18 @@ fate-ffmpeg-fix_sub_duration: CMD = fmtstdout srt -fix_sub_duration \ # Basic test for fix_sub_duration_heartbeat, which causes a buffered subtitle # to be pushed out when a video keyframe is received from an encoder. -FATE_SAMPLES_FFMPEG-$(call FILTERDEMDECENCMUX, MOVIE, MPEGVIDEO, \ - MPEG2VIDEO, SUBRIP, SRT, LAVFI_INDEV \ - MPEGVIDEO_PARSER CCAPTION_DECODER \ - MPEG2VIDEO_ENCODER NULL_MUXER PIPE_PROTOCOL) \ - += fate-ffmpeg-fix_sub_duration_heartbeat -fate-ffmpeg-fix_sub_duration_heartbeat: CMD = fmtstdout srt -fix_sub_duration \ - -real_time 1 -f lavfi \ - -i "movie=$(TARGET_SAMPLES)/sub/Closedcaption_rollup.m2v[out0+subcc]" \ - -map 0:v -map 0:s -fix_sub_duration_heartbeat:v:0 \ - -c:v mpeg2video -b:v 2M -g 30 -sc_threshold 1000000000 \ - -c:s srt \ - -f null - +#FATE_SAMPLES_FFMPEG-$(call FILTERDEMDECENCMUX, MOVIE, MPEGVIDEO, \ +# MPEG2VIDEO, SUBRIP, SRT, LAVFI_INDEV \ +# MPEGVIDEO_PARSER CCAPTION_DECODER \ +# MPEG2VIDEO_ENCODER NULL_MUXER PIPE_PROTOCOL) \ +# += fate-ffmpeg-fix_sub_duration_heartbeat +#fate-ffmpeg-fix_sub_duration_heartbeat: CMD = fmtstdout srt -fix_sub_duration \ +# -real_time 1 -f lavfi \ +# -i "movie=$(TARGET_SAMPLES)/sub/Closedcaption_rollup.m2v[out0+subcc]" \ +# -map 0:v -map 0:s -fix_sub_duration_heartbeat:v:0 \ +# -c:v mpeg2video -b:v 2M -g 30 -sc_threshold 1000000000 \ +# -c:s srt \ +# -f null - # FIXME: the integer AAC decoder does not produce the same output on all platforms # so until that is fixed we use the volume filter to silence the data -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 18/27] XXX fftools/ffmpeg_enc: temporarily disable side data copying 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (16 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 17/27] XXX: disable fix_sub_duration_heartbeat Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 19/27] XXX ffmpeg temporarily disable -stream_loop Anton Khirnov ` (9 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel It conflicts with threading work. --- fftools/ffmpeg_enc.c | 2 ++ .../fate/matroska-mastering-display-metadata | 24 ++------------- tests/ref/lavf/mpg | 6 ++-- tests/ref/seek/lavf-mpg | 30 +++++++++---------- 4 files changed, 23 insertions(+), 39 deletions(-) diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index c300a11f28..d8d7c3416d 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -345,6 +345,7 @@ int enc_open(OutputStream *ost, const AVFrame *frame) return ret; } +#if 0 if (ost->enc_ctx->nb_coded_side_data) { int i; @@ -380,6 +381,7 @@ int enc_open(OutputStream *ost, const AVFrame *frame) } } } +#endif // copy timebase while removing common factors if (ost->st->time_base.num <= 0 || ost->st->time_base.den <= 0) diff --git a/tests/ref/fate/matroska-mastering-display-metadata b/tests/ref/fate/matroska-mastering-display-metadata index 3726469213..6d3c4f1023 100644 --- a/tests/ref/fate/matroska-mastering-display-metadata +++ b/tests/ref/fate/matroska-mastering-display-metadata @@ -1,5 +1,5 @@ -9d0fb8123a2e90e85153428a91d1ee9d *tests/data/fate/matroska-mastering-display-metadata.matroska -1669589 tests/data/fate/matroska-mastering-display-metadata.matroska +1ed4248debff46761cf6230c55768dd9 *tests/data/fate/matroska-mastering-display-metadata.matroska +1669465 tests/data/fate/matroska-mastering-display-metadata.matroska #extradata 0: 4, 0x040901a3 #extradata 3: 200, 0x506463a8 #tb 0: 1/1000 @@ -24,7 +24,7 @@ #sar 3: 1/1 0, 0, 0, 16, 57008, 0x43416399, S=2, 8, 88 1, 0, 0, 16, 2403, 0xaa818522 -3, 0, 0, 16, 274117, 0xc439610f, S=2, 8, 88 +3, 0, 0, 16, 274117, 0xc439610f 0, 17, 17, 16, 57248, 0xa06cd7b5 1, 17, 17, 16, 2403, 0xe1a991e5 2, 17, 17, 16, 1602, 0x5d868171 @@ -74,22 +74,4 @@ codec_name=pcm_s16be [STREAM] index=3 codec_name=ffv1 -[SIDE_DATA] -side_data_type=Content light level metadata -max_content=1000 -max_average=100 -[/SIDE_DATA] -[SIDE_DATA] -side_data_type=Mastering display metadata -red_x=17/25 -red_y=8/25 -green_x=53/200 -green_y=69/100 -blue_x=3/20 -blue_y=3/50 -white_point_x=3127/10000 -white_point_y=329/1000 -min_luminance=0/1 -max_luminance=1000/1 -[/SIDE_DATA] [/STREAM] diff --git a/tests/ref/lavf/mpg b/tests/ref/lavf/mpg index 332b7114b5..8779871ebf 100644 --- a/tests/ref/lavf/mpg +++ b/tests/ref/lavf/mpg @@ -1,9 +1,9 @@ -01bbdea588da51ab4a9d1d26f3443c96 *tests/data/lavf/lavf.mpg +f9b99341af206109dc6d48fc7c57fb5c *tests/data/lavf/lavf.mpg 372736 tests/data/lavf/lavf.mpg tests/data/lavf/lavf.mpg CRC=0x000e23ae -87b447b78a7d1141b9d41bb3aa50434d *tests/data/lavf/lavf.mpg +d2b1947240fd0d8da0561c785788cc32 *tests/data/lavf/lavf.mpg 389120 tests/data/lavf/lavf.mpg tests/data/lavf/lavf.mpg CRC=0x60ba4ab9 -284f41c914df75c12de01e223d65f87f *tests/data/lavf/lavf.mpg +a4cc793cd26616ac980514c4399e36cb *tests/data/lavf/lavf.mpg 372736 tests/data/lavf/lavf.mpg tests/data/lavf/lavf.mpg CRC=0x000e23ae diff --git a/tests/ref/seek/lavf-mpg b/tests/ref/seek/lavf-mpg index e804b84739..e8b8899fb6 100644 --- a/tests/ref/seek/lavf-mpg +++ b/tests/ref/seek/lavf-mpg @@ -2,51 +2,51 @@ ret: 0 st: 1 flags:1 dts: 0.529089 pts: 0.529089 pos: 2048 size: 208 ret: 0 st:-1 flags:0 ts:-1.000000 ret: 0 st: 1 flags:1 dts: 0.529089 pts: 0.529089 pos: 2048 size: 208 ret: 0 st:-1 flags:1 ts: 1.894167 -ret: 0 st: 1 flags:1 dts: 1.051544 pts: 1.051544 pos: 342028 size: 314 +ret: 0 st: 0 flags:1 dts: 1.460000 pts: 1.500000 pos: 346112 size: 24937 ret: 0 st: 0 flags:0 ts: 0.788333 -ret: 0 st: 0 flags:0 dts: 0.820000 pts: 0.860000 pos: 118784 size: 14717 +ret: 0 st: 0 flags:0 dts: 0.820000 pts: 0.860000 pos: 126976 size: 14717 ret: 0 st: 0 flags:1 ts:-0.317500 ret: 0 st: 1 flags:1 dts: 0.529089 pts: 0.529089 pos: 2048 size: 208 ret: 0 st: 1 flags:0 ts: 2.576667 -ret: 0 st: 1 flags:1 dts: 1.312767 pts: 1.312767 pos: 368652 size: 379 +ret: 0 st: 1 flags:1 dts: 1.312767 pts: 1.312767 pos: 51200 size: 367 ret: 0 st: 1 flags:1 ts: 1.470833 -ret: 0 st: 1 flags:1 dts: 1.312767 pts: 1.312767 pos: 368652 size: 379 +ret: 0 st: 1 flags:1 dts: 1.312767 pts: 1.312767 pos: 51200 size: 367 ret: 0 st:-1 flags:0 ts: 0.365002 ret: 0 st: 1 flags:1 dts: 0.529089 pts: 0.529089 pos: 2048 size: 208 ret: 0 st:-1 flags:1 ts:-0.740831 ret: 0 st: 1 flags:1 dts: 0.529089 pts: 0.529089 pos: 2048 size: 208 ret: 0 st: 0 flags:0 ts: 2.153333 -ret: 0 st: 1 flags:1 dts: 1.051544 pts: 1.051544 pos: 342028 size: 314 +ret: 0 st: 0 flags:1 dts: 1.460000 pts: 1.500000 pos: 346112 size: 24937 ret: 0 st: 0 flags:1 ts: 1.047500 -ret: 0 st: 0 flags:0 dts: 1.020000 pts: 1.060000 pos: 196608 size: 17639 +ret: 0 st: 0 flags:0 dts: 1.020000 pts: 1.060000 pos: 202752 size: 17651 ret: 0 st: 1 flags:0 ts:-0.058333 ret: 0 st: 1 flags:1 dts: 0.529089 pts: 0.529089 pos: 2048 size: 208 ret: 0 st: 1 flags:1 ts: 2.835833 -ret: 0 st: 1 flags:1 dts: 1.312767 pts: 1.312767 pos: 368652 size: 379 +ret: 0 st: 1 flags:1 dts: 1.312767 pts: 1.312767 pos: 51200 size: 367 ret: 0 st:-1 flags:0 ts: 1.730004 -ret: 0 st: 1 flags:1 dts: 1.051544 pts: 1.051544 pos: 342028 size: 314 +ret: 0 st: 0 flags:1 dts: 1.460000 pts: 1.500000 pos: 346112 size: 24937 ret: 0 st:-1 flags:1 ts: 0.624171 -ret: 0 st: 0 flags:0 dts: 0.620000 pts: 0.660000 pos: 55296 size: 14239 +ret: 0 st: 0 flags:0 dts: 0.620000 pts: 0.660000 pos: 63488 size: 14239 ret: 0 st: 0 flags:0 ts:-0.481667 ret: 0 st: 1 flags:1 dts: 0.529089 pts: 0.529089 pos: 2048 size: 208 ret: 0 st: 0 flags:1 ts: 2.412500 -ret: 0 st: 1 flags:1 dts: 1.051544 pts: 1.051544 pos: 342028 size: 314 +ret: 0 st: 0 flags:1 dts: 1.460000 pts: 1.500000 pos: 346112 size: 24937 ret: 0 st: 1 flags:0 ts: 1.306667 -ret: 0 st: 1 flags:1 dts: 1.312767 pts: 1.312767 pos: 368652 size: 379 +ret: 0 st: 1 flags:1 dts: 1.312767 pts: 1.312767 pos: 51200 size: 367 ret: 0 st: 1 flags:1 ts: 0.200844 ret: 0 st: 1 flags:1 dts: 0.529089 pts: 0.529089 pos: 2048 size: 208 ret: 0 st:-1 flags:0 ts:-0.904994 ret: 0 st: 1 flags:1 dts: 0.529089 pts: 0.529089 pos: 2048 size: 208 ret: 0 st:-1 flags:1 ts: 1.989173 -ret: 0 st: 1 flags:1 dts: 1.051544 pts: 1.051544 pos: 342028 size: 314 +ret: 0 st: 0 flags:1 dts: 1.460000 pts: 1.500000 pos: 346112 size: 24937 ret: 0 st: 0 flags:0 ts: 0.883344 -ret: 0 st: 0 flags:0 dts: 0.900000 pts: 0.940000 pos: 147456 size: 12755 +ret: 0 st: 0 flags:0 dts: 0.900000 pts: 0.940000 pos: 153600 size: 12767 ret: 0 st: 0 flags:1 ts:-0.222489 ret: 0 st: 1 flags:1 dts: 0.529089 pts: 0.529089 pos: 2048 size: 208 ret: 0 st: 1 flags:0 ts: 2.671678 -ret: 0 st: 1 flags:1 dts: 1.312767 pts: 1.312767 pos: 368652 size: 379 +ret: 0 st: 1 flags:1 dts: 1.312767 pts: 1.312767 pos: 51200 size: 367 ret: 0 st: 1 flags:1 ts: 1.565844 -ret: 0 st: 1 flags:1 dts: 1.312767 pts: 1.312767 pos: 368652 size: 379 +ret: 0 st: 1 flags:1 dts: 1.312767 pts: 1.312767 pos: 51200 size: 367 ret: 0 st:-1 flags:0 ts: 0.460008 ret: 0 st: 1 flags:1 dts: 0.529089 pts: 0.529089 pos: 2048 size: 208 ret: 0 st:-1 flags:1 ts:-0.645825 -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 19/27] XXX ffmpeg temporarily disable -stream_loop 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (17 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 18/27] XXX fftools/ffmpeg_enc: temporarily disable side data copying Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 20/27] WIP: fftools/ffmpeg_enc: move encoding to a separate thread Anton Khirnov ` (8 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel It conflicts with threading work. --- fftools/ffmpeg.h | 6 +++--- fftools/ffmpeg_dec.c | 4 ++++ fftools/ffmpeg_demux.c | 8 +++++++- tests/fate/ffmpeg.mak | 12 ++++++------ 4 files changed, 20 insertions(+), 10 deletions(-) diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 88b8ed12c0..a4fd825749 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -45,7 +45,7 @@ #include "libavutil/pixfmt.h" #include "libavutil/rational.h" #include "libavutil/thread.h" -#include "libavutil/threadmessage.h" +//#include "libavutil/threadmessage.h" #include "libswresample/swresample.h" @@ -438,8 +438,8 @@ typedef struct InputFile { /* when looping the input file, this queue is used by decoders to report * the last frame duration back to the demuxer thread */ - AVThreadMessageQueue *audio_duration_queue; - int audio_duration_queue_size; + //AVThreadMessageQueue *audio_duration_queue; + //int audio_duration_queue_size; } InputFile; enum forced_keyframes_const { diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c index fb7b404020..36163195ca 100644 --- a/fftools/ffmpeg_dec.c +++ b/fftools/ffmpeg_dec.c @@ -702,6 +702,7 @@ static void *decoder_thread(void *arg) if (!flush_buffers) break; +#if 0 /* report last frame duration to the demuxer thread */ if (ist->dec->type == AVMEDIA_TYPE_AUDIO) { LastFrameDuration dur; @@ -713,6 +714,7 @@ static void *decoder_thread(void *arg) av_thread_message_queue_send(ifile->audio_duration_queue, &dur, 0); } +#endif avcodec_flush_buffers(ist->dec_ctx); } else if (ret < 0) { @@ -738,10 +740,12 @@ finish: tq_receive_finish(d->queue_in, 0); tq_send_finish (d->queue_out, 0); +#if 0 // make sure the demuxer does not get stuck waiting for audio durations // that will never arrive if (ifile->audio_duration_queue && ist->dec->type == AVMEDIA_TYPE_AUDIO) av_thread_message_queue_set_err_recv(ifile->audio_duration_queue, AVERROR_EOF); +#endif dec_thread_uninit(&dt); diff --git a/fftools/ffmpeg_demux.c b/fftools/ffmpeg_demux.c index 0955956117..ea74b45663 100644 --- a/fftools/ffmpeg_demux.c +++ b/fftools/ffmpeg_demux.c @@ -150,6 +150,7 @@ static void report_new_stream(Demuxer *d, const AVPacket *pkt) d->nb_streams_warn = pkt->stream_index + 1; } +#if 0 static void ifile_duration_update(Demuxer *d, DemuxStream *ds, int64_t last_duration) { @@ -217,6 +218,7 @@ static int seek_to_start(Demuxer *d) return ret; } +#endif static void ts_discontinuity_detect(Demuxer *d, InputStream *ist, AVPacket *pkt) @@ -568,6 +570,7 @@ static void *input_thread(void *arg) continue; } if (ret < 0) { +#if 0 if (d->loop) { /* signal looping to the consumer thread */ pkt->opaque = (void*)(intptr_t)PKT_OPAQUE_SEEK; @@ -579,6 +582,7 @@ static void *input_thread(void *arg) /* fallthrough to the error path */ } +#endif if (ret == AVERROR_EOF) av_log(d, AV_LOG_VERBOSE, "EOF while reading input\n"); @@ -655,7 +659,7 @@ static void thread_stop(Demuxer *d) tq_free(&d->thread_queue); - av_thread_message_queue_free(&f->audio_duration_queue); + //av_thread_message_queue_free(&f->audio_duration_queue); } static int thread_start(Demuxer *d) @@ -677,6 +681,7 @@ static int thread_start(Demuxer *d) return AVERROR(ENOMEM); } +#if 0 if (d->loop) { int nb_audio_dec = 0; @@ -694,6 +699,7 @@ static int thread_start(Demuxer *d) f->audio_duration_queue_size = nb_audio_dec; } } +#endif if ((ret = pthread_create(&d->thread, NULL, input_thread, d))) { av_log(d, AV_LOG_ERROR, "pthread_create failed: %s. Try to increase `ulimit -v` or decrease `ulimit -s`.\n", strerror(ret)); diff --git a/tests/fate/ffmpeg.mak b/tests/fate/ffmpeg.mak index ebc1e1f189..a5dac5476a 100644 --- a/tests/fate/ffmpeg.mak +++ b/tests/fate/ffmpeg.mak @@ -154,10 +154,10 @@ fate-ffmpeg-fix_sub_duration: CMD = fmtstdout srt -fix_sub_duration \ # FIXME: the integer AAC decoder does not produce the same output on all platforms # so until that is fixed we use the volume filter to silence the data -FATE_SAMPLES_FFMPEG-$(call FRAMECRC, MATROSKA, H264 AAC_FIXED, PCM_S32LE_ENCODER VOLUME_FILTER) += fate-ffmpeg-streamloop-transcode-av -fate-ffmpeg-streamloop-transcode-av: CMD = \ - framecrc -auto_conversion_filters -stream_loop 3 -c:a aac_fixed -i $(TARGET_SAMPLES)/mkv/1242-small.mkv \ - -af volume=0:precision=fixed -c:a pcm_s32le +#FATE_SAMPLES_FFMPEG-$(call FRAMECRC, MATROSKA, H264 AAC_FIXED, PCM_S32LE_ENCODER VOLUME_FILTER) += fate-ffmpeg-streamloop-transcode-av +#fate-ffmpeg-streamloop-transcode-av: CMD = \ +# framecrc -auto_conversion_filters -stream_loop 3 -c:a aac_fixed -i $(TARGET_SAMPLES)/mkv/1242-small.mkv \ +# -af volume=0:precision=fixed -c:a pcm_s32le FATE_STREAMCOPY-$(call REMUX, MP4 MOV, EAC3_DEMUXER) += fate-copy-trac3074 fate-copy-trac3074: CMD = transcode eac3 $(TARGET_SAMPLES)/eac3/csi_miami_stereo_128_spx.eac3\ @@ -194,8 +194,8 @@ FATE_STREAMCOPY-$(call REMUX, PSP MOV, H264_PARSER H264_DECODER) += fate-copy-ps fate-copy-psp: CMD = transcode "mov" $(TARGET_SAMPLES)/h264/wwwq_cut.mp4\ psp "-c copy" "-codec copy" -FATE_STREAMCOPY-$(call FRAMEMD5, FLV, H264) += fate-ffmpeg-streamloop-copy -fate-ffmpeg-streamloop-copy: CMD = framemd5 -stream_loop 2 -i $(TARGET_SAMPLES)/flv/streamloop.flv -c copy +#FATE_STREAMCOPY-$(call FRAMEMD5, FLV, H264) += fate-ffmpeg-streamloop-copy +#fate-ffmpeg-streamloop-copy: CMD = framemd5 -stream_loop 2 -i $(TARGET_SAMPLES)/flv/streamloop.flv -c copy tests/data/audio_shorter_than_video.nut: TAG = GEN tests/data/audio_shorter_than_video.nut: tests/data/vsynth_lena.yuv -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 20/27] WIP: fftools/ffmpeg_enc: move encoding to a separate thread 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (18 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 19/27] XXX ffmpeg temporarily disable -stream_loop Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 21/27] WIP fftools/ffmpeg: add thread-aware transcode scheduling infrastructure Anton Khirnov ` (7 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel As for the analogous decoding change, this is only a preparatory step to a fully threaded architecture and does not yet make encoding truly parallel. The main thread will currently submit a frame and wait until it has been fully processed by the encoder before moving on. That will change in future commits after filters are moved to threads and a thread-aware scheduler is added. WIP: resolve all // XXX left in the code Also, if an encoder with a sync queue receives EOF it will terminate after processing everything it currently has, even though the sync queue might still be triggered by other threads. --- fftools/ffmpeg_enc.c | 384 ++++++++++++++++++++++++++++++++++++++----- 1 file changed, 340 insertions(+), 44 deletions(-) diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index d8d7c3416d..ea542173c5 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -20,6 +20,7 @@ #include <stdint.h> #include "ffmpeg.h" +#include "thread_queue.h" #include "libavutil/avassert.h" #include "libavutil/avstring.h" @@ -43,6 +44,7 @@ struct Encoder { // packet for receiving encoded output AVPacket *pkt; + AVFrame *sub_frame; // combined size of all the packets received from the encoder uint64_t data_size; @@ -51,8 +53,48 @@ struct Encoder { uint64_t packets_encoded; int opened; + int finished; + + pthread_t thread; + /** + * Queue for sending frames from the main thread to + * the encoder thread. + */ + ThreadQueue *queue_in; + /** + * Queue for sending encoded packets from the encoder thread + * to the main thread. + * + * An empty packet is sent to signal that a previously sent + * frame has been fully processed. + */ + ThreadQueue *queue_out; }; +// data that is local to the decoder thread and not visible outside of it +typedef struct EncoderThread { + AVFrame *frame; + AVPacket *pkt; +} EncoderThread; + +static int enc_thread_stop(Encoder *e) +{ + void *ret; + + if (!e->queue_in) + return 0; + + tq_send_finish(e->queue_in, 0); + tq_receive_finish(e->queue_out, 0); + + pthread_join(e->thread, &ret); + + tq_free(&e->queue_in); + tq_free(&e->queue_out); + + return (int)(intptr_t)ret; +} + void enc_free(Encoder **penc) { Encoder *enc = *penc; @@ -60,7 +102,10 @@ void enc_free(Encoder **penc) if (!enc) return; + enc_thread_stop(enc); + av_frame_free(&enc->sq_frame); + av_frame_free(&enc->sub_frame); av_packet_free(&enc->pkt); @@ -77,6 +122,12 @@ int enc_alloc(Encoder **penc, const AVCodec *codec) if (!enc) return AVERROR(ENOMEM); + if (codec->type == AVMEDIA_TYPE_SUBTITLE) { + enc->sub_frame = av_frame_alloc(); + if (!enc->sub_frame) + goto fail; + } + enc->pkt = av_packet_alloc(); if (!enc->pkt) goto fail; @@ -165,6 +216,52 @@ static int set_encoder_id(OutputFile *of, OutputStream *ost) return 0; } +static void *encoder_thread(void *arg); + +static int enc_thread_start(OutputStream *ost) +{ + Encoder *e = ost->enc; + ObjPool *op; + int ret = 0; + + op = objpool_alloc_frames(); + if (!op) + return AVERROR(ENOMEM); + + e->queue_in = tq_alloc(1, 1, op, frame_move); + if (!e->queue_in) { + objpool_free(&op); + return AVERROR(ENOMEM); + } + + op = objpool_alloc_packets(); + if (!op) + goto fail; + + e->queue_out = tq_alloc(1, 4, op, pkt_move); + if (!e->queue_out) { + objpool_free(&op); + goto fail; + } + + ret = pthread_create(&e->thread, NULL, encoder_thread, ost); + if (ret) { + ret = AVERROR(ret); + av_log(ost, AV_LOG_ERROR, "pthread_create() failed: %s\n", + av_err2str(ret)); + goto fail; + } + + return 0; +fail: + if (ret >= 0) + ret = AVERROR(ENOMEM); + + tq_free(&e->queue_in); + tq_free(&e->queue_out); + return ret; +} + int enc_open(OutputStream *ost, const AVFrame *frame) { InputStream *ist = ost->ist; @@ -387,6 +484,13 @@ int enc_open(OutputStream *ost, const AVFrame *frame) if (ost->st->time_base.num <= 0 || ost->st->time_base.den <= 0) ost->st->time_base = av_add_q(ost->enc_ctx->time_base, (AVRational){0, 1}); + ret = enc_thread_start(ost); + if (ret < 0) { + av_log(ost, AV_LOG_ERROR, "Error starting encoder thread: %s\n", + av_err2str(ret)); + return ret; + } + ret = of_stream_init(of, ost); if (ret < 0) return ret; @@ -400,19 +504,18 @@ static int check_recording_time(OutputStream *ost, int64_t ts, AVRational tb) if (of->recording_time != INT64_MAX && av_compare_ts(ts, tb, of->recording_time, AV_TIME_BASE_Q) >= 0) { - close_output_stream(ost); return 0; } return 1; } -int enc_subtitle(OutputFile *of, OutputStream *ost, const AVSubtitle *sub) +static int do_subtitle_out(OutputFile *of, OutputStream *ost, const AVSubtitle *sub, + AVPacket *pkt) { Encoder *e = ost->enc; int subtitle_out_max_size = 1024 * 1024; int subtitle_out_size, nb, i, ret; AVCodecContext *enc; - AVPacket *pkt = e->pkt; int64_t pts; if (sub->pts == AV_NOPTS_VALUE) { @@ -442,8 +545,9 @@ int enc_subtitle(OutputFile *of, OutputStream *ost, const AVSubtitle *sub) for (i = 0; i < nb; i++) { AVSubtitle local_sub = *sub; + // XXX if (!check_recording_time(ost, pts, AV_TIME_BASE_Q)) - return 0; + return AVERROR_EOF; ret = av_new_packet(pkt, subtitle_out_max_size); if (ret < 0) @@ -484,9 +588,11 @@ int enc_subtitle(OutputFile *of, OutputStream *ost, const AVSubtitle *sub) } pkt->dts = pkt->pts; - ret = of_output_packet(of, ost, pkt); - if (ret < 0) + ret = tq_send(e->queue_out, 0, pkt); + if (ret < 0) { + av_packet_unref(pkt); return ret; + } } return 0; @@ -624,11 +730,11 @@ static int update_video_stats(OutputStream *ost, const AVPacket *pkt, int write_ return 0; } -static int encode_frame(OutputFile *of, OutputStream *ost, AVFrame *frame) +static int encode_frame(OutputFile *of, OutputStream *ost, AVFrame *frame, + AVPacket *pkt) { Encoder *e = ost->enc; AVCodecContext *enc = ost->enc_ctx; - AVPacket *pkt = e->pkt; const char *type_desc = av_get_media_type_string(enc->codec_type); const char *action = frame ? "encode" : "flush"; int ret; @@ -678,11 +784,9 @@ static int encode_frame(OutputFile *of, OutputStream *ost, AVFrame *frame) if (ret == AVERROR(EAGAIN)) { av_assert0(frame); // should never happen during flushing return 0; - } else if (ret == AVERROR_EOF) { - ret = of_output_packet(of, ost, NULL); - return ret < 0 ? ret : AVERROR_EOF; } else if (ret < 0) { - av_log(ost, AV_LOG_ERROR, "%s encoding failed\n", type_desc); + if (ret != AVERROR_EOF) + av_log(ost, AV_LOG_ERROR, "%s encoding failed\n", type_desc); return ret; } @@ -706,33 +810,36 @@ static int encode_frame(OutputFile *of, OutputStream *ost, AVFrame *frame) av_ts2str(pkt->duration), av_ts2timestr(pkt->duration, &enc->time_base)); } - if ((ret = trigger_fix_sub_duration_heartbeat(ost, pkt)) < 0) { - av_log(NULL, AV_LOG_ERROR, - "Subtitle heartbeat logic failed in %s! (%s)\n", - __func__, av_err2str(ret)); - return ret; - } + // XXX + //if ((ret = trigger_fix_sub_duration_heartbeat(ost, pkt)) < 0) { + // av_log(NULL, AV_LOG_ERROR, + // "Subtitle heartbeat logic failed in %s! (%s)\n", + // __func__, av_err2str(ret)); + // return ret; + //} e->data_size += pkt->size; e->packets_encoded++; - ret = of_output_packet(of, ost, pkt); - if (ret < 0) + ret = tq_send(e->queue_out, 0, pkt); + if (ret < 0) { + av_packet_unref(pkt); return ret; + } } av_assert0(0); } static int submit_encode_frame(OutputFile *of, OutputStream *ost, - AVFrame *frame) + AVFrame *frame, AVPacket *pkt) { Encoder *e = ost->enc; int ret; if (ost->sq_idx_encode < 0) - return encode_frame(of, ost, frame); + return encode_frame(of, ost, frame, pkt); if (frame) { ret = av_frame_ref(e->sq_frame, frame); @@ -761,22 +868,22 @@ static int submit_encode_frame(OutputFile *of, OutputStream *ost, return (ret == AVERROR(EAGAIN)) ? 0 : ret; } - ret = encode_frame(of, ost, enc_frame); + ret = encode_frame(of, ost, enc_frame, pkt); if (enc_frame) av_frame_unref(enc_frame); if (ret < 0) { - if (ret == AVERROR_EOF) - close_output_stream(ost); + // XXX + //if (ret == AVERROR_EOF) + // close_output_stream(ost); return ret; } } } static int do_audio_out(OutputFile *of, OutputStream *ost, - AVFrame *frame) + AVFrame *frame, AVPacket *pkt) { AVCodecContext *enc = ost->enc_ctx; - int ret; if (!(enc->codec->capabilities & AV_CODEC_CAP_PARAM_CHANGE) && enc->ch_layout.nb_channels != frame->ch_layout.nb_channels) { @@ -785,11 +892,12 @@ static int do_audio_out(OutputFile *of, OutputStream *ost, return 0; } + // XXX if (!check_recording_time(ost, frame->pts, frame->time_base)) - return 0; + return AVERROR_EOF; - ret = submit_encode_frame(of, ost, frame); - return (ret < 0 && ret != AVERROR_EOF) ? ret : 0; + // XXX check EOF handling + return submit_encode_frame(of, ost, frame, pkt); } static enum AVPictureType forced_kf_apply(void *logctx, KeyframeForceCtx *kf, @@ -839,13 +947,14 @@ force_keyframe: } /* May modify/reset frame */ -static int do_video_out(OutputFile *of, OutputStream *ost, AVFrame *in_picture) +static int do_video_out(OutputFile *of, OutputStream *ost, + AVFrame *in_picture, AVPacket *pkt) { - int ret; AVCodecContext *enc = ost->enc_ctx; + // XXX if (!check_recording_time(ost, in_picture->pts, ost->enc_ctx->time_base)) - return 0; + return AVERROR_EOF; in_picture->quality = enc->global_quality; in_picture->pict_type = forced_kf_apply(ost, &ost->kf, enc->time_base, in_picture); @@ -857,26 +966,210 @@ static int do_video_out(OutputFile *of, OutputStream *ost, AVFrame *in_picture) } #endif - ret = submit_encode_frame(of, ost, in_picture); - return (ret == AVERROR_EOF) ? 0 : ret; + // XXX check EOF handling + return submit_encode_frame(of, ost, in_picture, pkt); +} + +static int frame_encode(OutputStream *ost, AVFrame *frame, AVPacket *pkt) +{ + OutputFile *of = output_files[ost->file_index]; + enum AVMediaType type = ost->type; + int ret; + + if (type == AVMEDIA_TYPE_SUBTITLE) { + // no flushing for subtitles + return frame ? + do_subtitle_out(of, ost, (AVSubtitle*)frame->buf[0]->data, pkt) : 0; + } + + // XXX + if (frame) { + ret = (type == AVMEDIA_TYPE_VIDEO) ? do_video_out(of, ost, frame, pkt) : + do_audio_out(of, ost, frame, pkt); + if (ret < 0) + return ret; + } + + return frame ? 0 : submit_encode_frame(of, ost, NULL, pkt); +} + +static void enc_thread_set_name(const OutputStream *ost) +{ + char name[16]; + snprintf(name, sizeof(name), "enc%d:%d:%s", ost->file_index, ost->index, + ost->enc_ctx->codec->name); + ff_thread_setname(name); +} + +static void enc_thread_uninit(EncoderThread *et) +{ + av_packet_free(&et->pkt); + av_frame_free(&et->frame); + + memset(et, 0, sizeof(*et)); +} + +static int enc_thread_init(EncoderThread *et) +{ + memset(et, 0, sizeof(*et)); + + et->frame = av_frame_alloc(); + if (!et->frame) + goto fail; + + et->pkt = av_packet_alloc(); + if (!et->pkt) + goto fail; + + return 0; + +fail: + enc_thread_uninit(et); + return AVERROR(ENOMEM); +} + +static void *encoder_thread(void *arg) +{ + OutputStream *ost = arg; + OutputFile *of = output_files[ost->file_index]; + Encoder *e = ost->enc; + EncoderThread et; + int ret = 0, input_status = 0; + + ret = enc_thread_init(&et); + if (ret < 0) + goto finish; + + enc_thread_set_name(ost); + + while (!input_status) { + int dummy; + + input_status = tq_receive(e->queue_in, &dummy, et.frame); + if (input_status < 0) + av_log(ost, AV_LOG_VERBOSE, "Encoder thread received EOF\n"); + + ret = frame_encode(ost, input_status >= 0 ? et.frame : NULL, et.pkt); + + av_packet_unref(et.pkt); + av_frame_unref(et.frame); + + if (ret < 0) { + if (ret == AVERROR_EOF) + av_log(ost, AV_LOG_VERBOSE, "Encoder returned EOF, finishing\n"); + else + av_log(ost, AV_LOG_ERROR, "Error encoding a frame: %s\n", + av_err2str(ret)); + break; + } + + // signal to the consumer thread that the frame was encoded + ret = tq_send(e->queue_out, 0, et.pkt); + if (ret < 0) { + if (ret != AVERROR_EOF) + av_log(ost, AV_LOG_ERROR, + "Error communicating with the main thread\n"); + break; + } + } + + // EOF is normal thread termination + if (ret == AVERROR_EOF) + ret = 0; + +finish: + if (ost->sq_idx_encode >= 0) + sq_send(of->sq_encode, ost->sq_idx_encode, SQFRAME(NULL)); + + tq_receive_finish(e->queue_in, 0); + tq_send_finish (e->queue_out, 0); + + enc_thread_uninit(&et); + + av_log(ost, AV_LOG_VERBOSE, "Terminating encoder thread\n"); + + return (void*)(intptr_t)ret; } int enc_frame(OutputStream *ost, AVFrame *frame) { OutputFile *of = output_files[ost->file_index]; - int ret; + Encoder *e = ost->enc; + int ret, thread_ret; ret = enc_open(ost, frame); if (ret < 0) return ret; - return ost->enc_ctx->codec_type == AVMEDIA_TYPE_VIDEO ? - do_video_out(of, ost, frame) : do_audio_out(of, ost, frame); + // thread already joined + // XXX check EOF handling + if (!e->queue_in) + return AVERROR_EOF; + + // send the frame/EOF to the encoder thread + if (frame) { + ret = tq_send(e->queue_in, 0, frame); + if (ret < 0) + goto finish; + } else + tq_send_finish(e->queue_in, 0); + + // retrieve all encoded data for the frame + while (1) { + int dummy; + + ret = tq_receive(e->queue_out, &dummy, e->pkt); + if (ret < 0) + break; + + // frame fully encoded + if (!e->pkt->data && !e->pkt->side_data_elems) + return 0; + + // process the encoded packet + ret = of_output_packet(of, ost, e->pkt); + if (ret < 0) + goto finish; + } + +finish: + thread_ret = enc_thread_stop(e); + if (thread_ret < 0) { + av_log(ost, AV_LOG_ERROR, "Encoder thread returned error: %s\n", + av_err2str(thread_ret)); + ret = err_merge(ret, thread_ret); + } + + if (ret < 0 && ret != AVERROR_EOF) + return ret; + + // signal EOF to the muxer + return of_output_packet(of, ost, NULL); +} + +int enc_subtitle(OutputFile *of, OutputStream *ost, const AVSubtitle *sub) +{ + Encoder *e = ost->enc; + AVFrame *f = e->sub_frame; + int ret; + + // XXX the queue for transferring data to the encoder thread runs + // on AVFrames, so we wrap AVSubtitle in an AVBufferRef and put + // that inside the frame + // eventually, subtitles should be switched to use AVFrames natively + ret = subtitle_wrap_frame(f, sub, 1); + if (ret < 0) + return ret; + + ret = enc_frame(ost, f); + av_frame_unref(f); + + return ret; } int enc_flush(void) { - int ret; + int ret = 0; for (OutputStream *ost = ost_iter(NULL); ost; ost = ost_iter(ost)) { OutputFile *of = output_files[ost->file_index]; @@ -887,16 +1180,19 @@ int enc_flush(void) for (OutputStream *ost = ost_iter(NULL); ost; ost = ost_iter(ost)) { Encoder *e = ost->enc; AVCodecContext *enc = ost->enc_ctx; - OutputFile *of = output_files[ost->file_index]; + int err; if (!enc || !e->opened || (enc->codec_type != AVMEDIA_TYPE_VIDEO && enc->codec_type != AVMEDIA_TYPE_AUDIO)) continue; - ret = submit_encode_frame(of, ost, NULL); - if (ret != AVERROR_EOF) - return ret; + err = enc_frame(ost, NULL); + // XXX check EOF handling + if (err != AVERROR_EOF && ret < 0) + ret = err_merge(ret, err); + + av_assert0(!e->queue_in); } - return 0; + return ret; } -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 21/27] WIP fftools/ffmpeg: add thread-aware transcode scheduling infrastructure 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (19 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 20/27] WIP: fftools/ffmpeg_enc: move encoding to a separate thread Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 22/27] WIP fftools/ffmpeg_demux: convert to the scheduler Anton Khirnov ` (6 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel See the comment block at the top of fftools/ffmpeg_sched.h for more details on what this scheduler is for. This commit adds the scheduling code itself, along with minimal integration with the rest of the program: * allocating and freeing the scheduler * passing it throughout the call stack in order to register the individual components (demuxers/decoders/filtergraphs/encoders/muxers) with the scheduler The scheduler is not actually used as of this commit, so it should not result in any change in behavior. That will change in future commits. --- fftools/Makefile | 1 + fftools/ffmpeg.c | 18 +- fftools/ffmpeg.h | 26 +- fftools/ffmpeg_dec.c | 10 +- fftools/ffmpeg_demux.c | 44 +- fftools/ffmpeg_enc.c | 13 +- fftools/ffmpeg_filter.c | 38 +- fftools/ffmpeg_mux.c | 15 +- fftools/ffmpeg_mux.h | 10 + fftools/ffmpeg_mux_init.c | 70 +- fftools/ffmpeg_opt.c | 22 +- fftools/ffmpeg_sched.c | 1703 +++++++++++++++++++++++++++++++++++++ fftools/ffmpeg_sched.h | 414 +++++++++ 13 files changed, 2332 insertions(+), 52 deletions(-) create mode 100644 fftools/ffmpeg_sched.c create mode 100644 fftools/ffmpeg_sched.h diff --git a/fftools/Makefile b/fftools/Makefile index 56820e6bc8..d6a8913a7f 100644 --- a/fftools/Makefile +++ b/fftools/Makefile @@ -18,6 +18,7 @@ OBJS-ffmpeg += \ fftools/ffmpeg_mux.o \ fftools/ffmpeg_mux_init.o \ fftools/ffmpeg_opt.o \ + fftools/ffmpeg_sched.o \ fftools/objpool.o \ fftools/sync_queue.o \ fftools/thread_queue.o \ diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index e084318864..995424ca93 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -99,6 +99,7 @@ #include "cmdutils.h" #include "ffmpeg.h" +#include "ffmpeg_sched.h" #include "sync_queue.h" const char program_name[] = "ffmpeg"; @@ -1155,7 +1156,7 @@ static int transcode_step(OutputStream *ost, AVPacket *demux_pkt) /* * The following code is the main loop of the file converter */ -static int transcode(int *err_rate_exceeded) +static int transcode(Scheduler *sch, int *err_rate_exceeded) { int ret = 0, i; InputStream *ist; @@ -1293,6 +1294,8 @@ static int64_t getmaxrss(void) int main(int argc, char **argv) { + Scheduler *sch = NULL; + int ret, err_rate_exceeded; BenchmarkTimeStamps ti; @@ -1310,8 +1313,14 @@ int main(int argc, char **argv) show_banner(argc, argv, options); + sch = sch_alloc(); + if (!sch) { + ret = AVERROR(ENOMEM); + goto finish; + } + /* parse options and open all input/output files */ - ret = ffmpeg_parse_options(argc, argv); + ret = ffmpeg_parse_options(argc, argv, sch); if (ret < 0) goto finish; @@ -1329,7 +1338,7 @@ int main(int argc, char **argv) } current_time = ti = get_benchmark_time_stamps(); - ret = transcode(&err_rate_exceeded); + ret = transcode(sch, &err_rate_exceeded); if (ret >= 0 && do_benchmark) { int64_t utime, stime, rtime; current_time = get_benchmark_time_stamps(); @@ -1349,5 +1358,8 @@ finish: ret = 0; ffmpeg_cleanup(ret); + + sch_free(&sch); + return ret; } diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index a4fd825749..278216e5ff 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -27,6 +27,7 @@ #include <signal.h> #include "cmdutils.h" +#include "ffmpeg_sched.h" #include "sync_queue.h" #include "libavformat/avformat.h" @@ -731,7 +732,8 @@ int parse_and_set_vsync(const char *arg, int *vsync_var, int file_idx, int st_id int check_filter_outputs(void); int filtergraph_is_simple(const FilterGraph *fg); int init_simple_filtergraph(InputStream *ist, OutputStream *ost, - char *graph_desc); + char *graph_desc, + Scheduler *sch, unsigned sch_idx_enc); int init_complex_filtergraph(FilterGraph *fg); int copy_av_subtitle(AVSubtitle *dst, const AVSubtitle *src); @@ -754,7 +756,8 @@ void ifilter_sub2video_heartbeat(InputFilter *ifilter, int64_t pts, AVRational t */ int ifilter_parameters_from_dec(InputFilter *ifilter, const AVCodecContext *dec); -int ofilter_bind_ost(OutputFilter *ofilter, OutputStream *ost); +int ofilter_bind_ost(OutputFilter *ofilter, OutputStream *ost, + unsigned sched_idx_enc); /** * Create a new filtergraph in the global filtergraph list. @@ -762,7 +765,7 @@ int ofilter_bind_ost(OutputFilter *ofilter, OutputStream *ost); * @param graph_desc Graph description; an av_malloc()ed string, filtergraph * takes ownership of it. */ -int fg_create(FilterGraph **pfg, char *graph_desc); +int fg_create(FilterGraph **pfg, char *graph_desc, Scheduler *sch); void fg_free(FilterGraph **pfg); @@ -786,7 +789,7 @@ void fg_send_command(FilterGraph *fg, double time, const char *target, */ int reap_filters(FilterGraph *fg, int flush); -int ffmpeg_parse_options(int argc, char **argv); +int ffmpeg_parse_options(int argc, char **argv, Scheduler *sch); void enc_stats_write(OutputStream *ost, EncStats *es, const AVFrame *frame, const AVPacket *pkt, @@ -809,7 +812,7 @@ AVBufferRef *hw_device_for_filter(void); int hwaccel_retrieve_data(AVCodecContext *avctx, AVFrame *input); -int dec_open(InputStream *ist); +int dec_open(InputStream *ist, Scheduler *sch, unsigned sch_idx); void dec_free(Decoder **pdec); /** @@ -823,7 +826,8 @@ void dec_free(Decoder **pdec); */ int dec_packet(InputStream *ist, const AVPacket *pkt, int no_eof); -int enc_alloc(Encoder **penc, const AVCodec *codec); +int enc_alloc(Encoder **penc, const AVCodec *codec, + Scheduler *sch, unsigned sch_idx); void enc_free(Encoder **penc); int enc_open(OutputStream *ost, const AVFrame *frame); @@ -839,7 +843,7 @@ int enc_flush(void); */ int of_stream_init(OutputFile *of, OutputStream *ost); int of_write_trailer(OutputFile *of); -int of_open(const OptionsContext *o, const char *filename); +int of_open(const OptionsContext *o, const char *filename, Scheduler *sch); void of_free(OutputFile **pof); void of_enc_stats_close(void); @@ -853,7 +857,7 @@ int of_streamcopy(OutputStream *ost, const AVPacket *pkt, int64_t dts); int64_t of_filesize(OutputFile *of); -int ifile_open(const OptionsContext *o, const char *filename); +int ifile_open(const OptionsContext *o, const char *filename, Scheduler *sch); void ifile_close(InputFile **f); /** @@ -961,4 +965,10 @@ static inline void frame_move(void *dst, void *src) av_frame_move_ref(dst, src); } +void *muxer_thread(void *arg); +void *decoder_thread(void *arg); +void *encoder_thread(void *arg); + +int print_sdp(const char *filename); + #endif /* FFTOOLS_FFMPEG_H */ diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c index 36163195ca..dc8d0374a3 100644 --- a/fftools/ffmpeg_dec.c +++ b/fftools/ffmpeg_dec.c @@ -51,6 +51,9 @@ struct Decoder { AVFrame *sub_prev[2]; AVFrame *sub_heartbeat; + Scheduler *sch; + unsigned sch_idx; + pthread_t thread; /** * Queue for sending coded packets from the main thread to @@ -667,7 +670,7 @@ fail: return AVERROR(ENOMEM); } -static void *decoder_thread(void *arg) +void *decoder_thread(void *arg) { InputStream *ist = arg; InputFile *ifile = input_files[ist->file_index]; @@ -1048,7 +1051,7 @@ static int hw_device_setup_for_decode(InputStream *ist) return 0; } -int dec_open(InputStream *ist) +int dec_open(InputStream *ist, Scheduler *sch, unsigned sch_idx) { Decoder *d; const AVCodec *codec = ist->dec; @@ -1066,6 +1069,9 @@ int dec_open(InputStream *ist) return ret; d = ist->decoder; + d->sch = sch; + d->sch_idx = sch_idx; + if (codec->type == AVMEDIA_TYPE_SUBTITLE && ist->fix_sub_duration) { for (int i = 0; i < FF_ARRAY_ELEMS(d->sub_prev); i++) { d->sub_prev[i] = av_frame_alloc(); diff --git a/fftools/ffmpeg_demux.c b/fftools/ffmpeg_demux.c index ea74b45663..074546d517 100644 --- a/fftools/ffmpeg_demux.c +++ b/fftools/ffmpeg_demux.c @@ -20,6 +20,7 @@ #include <stdint.h> #include "ffmpeg.h" +#include "ffmpeg_sched.h" #include "objpool.h" #include "thread_queue.h" @@ -59,6 +60,9 @@ typedef struct DemuxStream { // name used for logging char log_name[32]; + int sch_idx_stream; + int sch_idx_dec; + double ts_scale; int streamcopy_needed; @@ -110,6 +114,7 @@ typedef struct Demuxer { double readrate_initial_burst; + Scheduler *sch; ThreadQueue *thread_queue; int thread_queue_size; pthread_t thread; @@ -824,7 +829,9 @@ void ifile_close(InputFile **pf) static int ist_use(InputStream *ist, int decoding_needed) { + Demuxer *d = demuxer_from_ifile(input_files[ist->file_index]); DemuxStream *ds = ds_from_ist(ist); + int ret; if (ist->user_set_discard == AVDISCARD_ALL) { av_log(ist, AV_LOG_ERROR, "Cannot %s a disabled input stream\n", @@ -832,13 +839,30 @@ static int ist_use(InputStream *ist, int decoding_needed) return AVERROR(EINVAL); } + if (ds->sch_idx_stream < 0) { + ret = sch_add_demux_stream(d->sch, d->f.index); + if (ret < 0) + return ret; + ds->sch_idx_stream = ret; + } + ist->discard = 0; ist->st->discard = ist->user_set_discard; ist->decoding_needed |= decoding_needed; ds->streamcopy_needed |= !decoding_needed; - if (decoding_needed && !avcodec_is_open(ist->dec_ctx)) { - int ret = dec_open(ist); + if (decoding_needed && ds->sch_idx_dec < 0) { + ret = sch_add_dec(d->sch, decoder_thread, ist); + if (ret < 0) + return ret; + ds->sch_idx_dec = ret; + + ret = sch_connect(d->sch, SCH_DSTREAM(d->f.index, ds->sch_idx_stream), + SCH_DEC(ds->sch_idx_dec), NULL, NULL); + if (ret < 0) + return ret; + + ret = dec_open(ist, d->sch, ds->sch_idx_dec); if (ret < 0) return ret; } @@ -848,6 +872,7 @@ static int ist_use(InputStream *ist, int decoding_needed) int ist_output_add(InputStream *ist, OutputStream *ost) { + DemuxStream *ds = ds_from_ist(ist); int ret; ret = ist_use(ist, ost->enc ? DECODING_FOR_OST : 0); @@ -860,11 +885,12 @@ int ist_output_add(InputStream *ist, OutputStream *ost) ist->outputs[ist->nb_outputs - 1] = ost; - return 0; + return ost->enc ? ds->sch_idx_dec : ds->sch_idx_stream; } int ist_filter_add(InputStream *ist, InputFilter *ifilter, int is_simple) { + DemuxStream *ds = ds_from_ist(ist); int ret; ret = ist_use(ist, is_simple ? DECODING_FOR_OST : DECODING_FOR_FILTER); @@ -882,7 +908,7 @@ int ist_filter_add(InputStream *ist, InputFilter *ifilter, int is_simple) if (ret < 0) return ret; - return 0; + return ds->sch_idx_dec; } static int choose_decoder(const OptionsContext *o, AVFormatContext *s, AVStream *st, @@ -1009,6 +1035,9 @@ static DemuxStream *demux_stream_alloc(Demuxer *d, AVStream *st) if (!ds) return NULL; + ds->sch_idx_stream = -1; + ds->sch_idx_dec = -1; + ds->ist.st = st; ds->ist.file_index = f->index; ds->ist.index = st->index; @@ -1339,7 +1368,7 @@ static Demuxer *demux_alloc(void) return d; } -int ifile_open(const OptionsContext *o, const char *filename) +int ifile_open(const OptionsContext *o, const char *filename, Scheduler *sch) { Demuxer *d; InputFile *f; @@ -1366,6 +1395,11 @@ int ifile_open(const OptionsContext *o, const char *filename) f = &d->f; + ret = sch_add_demux(sch, input_thread, d); + if (ret < 0) + return ret; + d->sch = sch; + if (stop_time != INT64_MAX && recording_time != INT64_MAX) { stop_time = INT64_MAX; av_log(d, AV_LOG_WARNING, "-t and -to cannot be used together; using -t.\n"); diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index ea542173c5..9bede78a1e 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -55,6 +55,9 @@ struct Encoder { int opened; int finished; + Scheduler *sch; + unsigned sch_idx; + pthread_t thread; /** * Queue for sending frames from the main thread to @@ -112,7 +115,8 @@ void enc_free(Encoder **penc) av_freep(penc); } -int enc_alloc(Encoder **penc, const AVCodec *codec) +int enc_alloc(Encoder **penc, const AVCodec *codec, + Scheduler *sch, unsigned sch_idx) { Encoder *enc; @@ -132,6 +136,9 @@ int enc_alloc(Encoder **penc, const AVCodec *codec) if (!enc->pkt) goto fail; + enc->sch = sch; + enc->sch_idx = sch_idx; + *penc = enc; return 0; @@ -216,8 +223,6 @@ static int set_encoder_id(OutputFile *of, OutputStream *ost) return 0; } -static void *encoder_thread(void *arg); - static int enc_thread_start(OutputStream *ost) { Encoder *e = ost->enc; @@ -1028,7 +1033,7 @@ fail: return AVERROR(ENOMEM); } -static void *encoder_thread(void *arg) +void *encoder_thread(void *arg) { OutputStream *ost = arg; OutputFile *of = output_files[ost->file_index]; diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 04c4b4ea7b..e8e78f5454 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -72,6 +72,9 @@ typedef struct FilterGraphPriv { // frame for sending output to the encoder AVFrame *frame_enc; + Scheduler *sch; + unsigned sch_idx; + pthread_t thread; /** * Queue for sending frames from the main thread to the filtergraph. Has @@ -742,14 +745,20 @@ static int ifilter_bind_ist(InputFilter *ifilter, InputStream *ist) { InputFilterPriv *ifp = ifp_from_ifilter(ifilter); FilterGraphPriv *fgp = fgp_from_fg(ifilter->graph); - int ret; + int ret, dec_idx; av_assert0(!ifp->ist); ifp->ist = ist; ifp->type_src = ist->st->codecpar->codec_type; - ret = ist_filter_add(ist, ifilter, filtergraph_is_simple(ifilter->graph)); + dec_idx = ist_filter_add(ist, ifilter, filtergraph_is_simple(ifilter->graph)); + if (dec_idx < 0) + return dec_idx; + + ret = sch_connect(fgp->sch, SCH_DEC(dec_idx), + SCH_FILTER_IN(fgp->sch_idx, ifp->index), + NULL, NULL); if (ret < 0) return ret; @@ -805,13 +814,15 @@ static int set_channel_layout(OutputFilterPriv *f, OutputStream *ost) return 0; } -int ofilter_bind_ost(OutputFilter *ofilter, OutputStream *ost) +int ofilter_bind_ost(OutputFilter *ofilter, OutputStream *ost, + unsigned sched_idx_enc) { const OutputFile *of = output_files[ost->file_index]; OutputFilterPriv *ofp = ofp_from_ofilter(ofilter); FilterGraph *fg = ofilter->graph; FilterGraphPriv *fgp = fgp_from_fg(fg); const AVCodec *c = ost->enc_ctx->codec; + int ret; av_assert0(!ofilter->ost); @@ -894,6 +905,11 @@ int ofilter_bind_ost(OutputFilter *ofilter, OutputStream *ost) break; } + ret = sch_connect(fgp->sch, SCH_FILTER_OUT(fgp->sch_idx, ofp->index), + SCH_ENC(sched_idx_enc), NULL, NULL); + if (ret < 0) + return ret; + fgp->nb_outputs_bound++; av_assert0(fgp->nb_outputs_bound <= fg->nb_outputs); @@ -1023,7 +1039,7 @@ static const AVClass fg_class = { .category = AV_CLASS_CATEGORY_FILTER, }; -int fg_create(FilterGraph **pfg, char *graph_desc) +int fg_create(FilterGraph **pfg, char *graph_desc, Scheduler *sch) { FilterGraphPriv *fgp; FilterGraph *fg; @@ -1044,6 +1060,7 @@ int fg_create(FilterGraph **pfg, char *graph_desc) fg->index = nb_filtergraphs - 1; fgp->graph_desc = graph_desc; fgp->disable_conversions = !auto_conversion_filters; + fgp->sch = sch; snprintf(fgp->log_name, sizeof(fgp->log_name), "fc#%d", fg->index); @@ -1103,6 +1120,12 @@ int fg_create(FilterGraph **pfg, char *graph_desc) goto fail; } + ret = sch_add_filtergraph(sch, fg->nb_inputs, fg->nb_outputs, + filter_thread, fgp); + if (ret < 0) + goto fail; + fgp->sch_idx = ret; + fail: avfilter_inout_free(&inputs); avfilter_inout_free(&outputs); @@ -1115,13 +1138,14 @@ fail: } int init_simple_filtergraph(InputStream *ist, OutputStream *ost, - char *graph_desc) + char *graph_desc, + Scheduler *sch, unsigned sched_idx_enc) { FilterGraph *fg; FilterGraphPriv *fgp; int ret; - ret = fg_create(&fg, graph_desc); + ret = fg_create(&fg, graph_desc, sch); if (ret < 0) return ret; fgp = fgp_from_fg(fg); @@ -1147,7 +1171,7 @@ int init_simple_filtergraph(InputStream *ist, OutputStream *ost, if (ret < 0) return ret; - ret = ofilter_bind_ost(fg->outputs[0], ost); + ret = ofilter_bind_ost(fg->outputs[0], ost, sched_idx_enc); if (ret < 0) return ret; diff --git a/fftools/ffmpeg_mux.c b/fftools/ffmpeg_mux.c index 033894ae86..9628728d95 100644 --- a/fftools/ffmpeg_mux.c +++ b/fftools/ffmpeg_mux.c @@ -235,7 +235,7 @@ fail: return AVERROR(ENOMEM); } -static void *muxer_thread(void *arg) +void *muxer_thread(void *arg) { Muxer *mux = arg; OutputFile *of = &mux->of; @@ -557,7 +557,7 @@ static int thread_start(Muxer *mux) return 0; } -static int print_sdp(void) +int print_sdp(const char *filename) { char sdp[16384]; int i; @@ -590,19 +590,18 @@ static int print_sdp(void) if (ret < 0) goto fail; - if (!sdp_filename) { + if (!filename) { printf("SDP:\n%s\n", sdp); fflush(stdout); } else { - ret = avio_open2(&sdp_pb, sdp_filename, AVIO_FLAG_WRITE, &int_cb, NULL); + ret = avio_open2(&sdp_pb, filename, AVIO_FLAG_WRITE, &int_cb, NULL); if (ret < 0) { - av_log(NULL, AV_LOG_ERROR, "Failed to open sdp file '%s'\n", sdp_filename); + av_log(NULL, AV_LOG_ERROR, "Failed to open sdp file '%s'\n", filename); goto fail; } avio_print(sdp_pb, sdp); avio_closep(&sdp_pb); - av_freep(&sdp_filename); } // SDP successfully written, allow muxer threads to start @@ -638,7 +637,7 @@ int mux_check_init(Muxer *mux) nb_output_dumped++; if (sdp_filename || want_sdp) { - ret = print_sdp(); + ret = print_sdp(sdp_filename); if (ret < 0) { av_log(NULL, AV_LOG_ERROR, "Error writing the SDP.\n"); return ret; @@ -961,6 +960,8 @@ void of_free(OutputFile **pof) ost_free(&of->streams[i]); av_freep(&of->streams); + av_freep(&mux->sch_stream_idx); + av_dict_free(&mux->opts); av_packet_free(&mux->sq_pkt); diff --git a/fftools/ffmpeg_mux.h b/fftools/ffmpeg_mux.h index a2bb4dfc7d..d5aba6db36 100644 --- a/fftools/ffmpeg_mux.h +++ b/fftools/ffmpeg_mux.h @@ -24,6 +24,7 @@ #include <stdatomic.h> #include <stdint.h> +#include "ffmpeg_sched.h" #include "thread_queue.h" #include "libavformat/avformat.h" @@ -50,6 +51,9 @@ typedef struct MuxStream { EncStats stats; + int sched_idx; + int sched_idx_enc; + int64_t max_frames; /* @@ -94,6 +98,12 @@ typedef struct Muxer { AVFormatContext *fc; + Scheduler *sch; + + // OutputStream indices indexed by scheduler stream indices + int *sch_stream_idx; + int nb_sch_stream_idx; + pthread_t thread; ThreadQueue *tq; diff --git a/fftools/ffmpeg_mux_init.c b/fftools/ffmpeg_mux_init.c index f35680e355..3380cbeb5c 100644 --- a/fftools/ffmpeg_mux_init.c +++ b/fftools/ffmpeg_mux_init.c @@ -23,6 +23,7 @@ #include "cmdutils.h" #include "ffmpeg.h" #include "ffmpeg_mux.h" +#include "ffmpeg_sched.h" #include "fopen_utf8.h" #include "libavformat/avformat.h" @@ -436,6 +437,9 @@ static MuxStream *mux_stream_alloc(Muxer *mux, enum AVMediaType type) ms->ost.class = &output_stream_class; + ms->sched_idx = -1; + ms->sched_idx_enc = -1; + snprintf(ms->log_name, sizeof(ms->log_name), "%cost#%d:%d", type_str ? *type_str : '?', mux->of.index, ms->ost.index); @@ -1123,6 +1127,22 @@ static int ost_add(Muxer *mux, const OptionsContext *o, enum AVMediaType type, if (!ms) return AVERROR(ENOMEM); + // only streams with sources (i.e. not attachments) + // are handled by the scheduler + if (ist || ofilter) { + ret = GROW_ARRAY(mux->sch_stream_idx, mux->nb_sch_stream_idx); + if (ret < 0) + return ret; + + ret = sch_add_mux_stream(mux->sch, mux->of.index); + if (ret < 0) + return ret; + + av_assert0(ret == mux->nb_sch_stream_idx - 1); + mux->sch_stream_idx[ret] = ms->ost.index; + ms->sched_idx = ret; + } + ost = &ms->ost; if (o->streamid) { @@ -1166,7 +1186,12 @@ static int ost_add(Muxer *mux, const OptionsContext *o, enum AVMediaType type, if (!ost->enc_ctx) return AVERROR(ENOMEM); - ret = enc_alloc(&ost->enc, enc); + ret = sch_add_enc(mux->sch, encoder_thread, ost, NULL); + if (ret < 0) + return ret; + ms->sched_idx_enc = ret; + + ret = enc_alloc(&ost->enc, enc, mux->sch, ms->sched_idx_enc); if (ret < 0) return ret; @@ -1421,23 +1446,48 @@ static int ost_add(Muxer *mux, const OptionsContext *o, enum AVMediaType type, (type == AVMEDIA_TYPE_VIDEO || type == AVMEDIA_TYPE_AUDIO)) { if (ofilter) { ost->filter = ofilter; - ret = ofilter_bind_ost(ofilter, ost); + ret = ofilter_bind_ost(ofilter, ost, ms->sched_idx_enc); if (ret < 0) return ret; } else { - ret = init_simple_filtergraph(ost->ist, ost, filters); + ret = init_simple_filtergraph(ost->ist, ost, filters, + mux->sch, ms->sched_idx_enc); if (ret < 0) { av_log(ost, AV_LOG_ERROR, "Error initializing a simple filtergraph\n"); return ret; } } + + ret = sch_connect(mux->sch, SCH_ENC(ms->sched_idx_enc), + SCH_MSTREAM(ost->file_index, ms->sched_idx), + NULL, NULL); + if (ret < 0) + return ret; } else if (ost->ist) { - ret = ist_output_add(ost->ist, ost); - if (ret < 0) { + int sched_idx = ist_output_add(ost->ist, ost); + if (sched_idx < 0) { av_log(ost, AV_LOG_ERROR, "Error binding an input stream\n"); - return ret; + return sched_idx; + } + + if (ost->enc) { + ret = sch_connect(mux->sch, SCH_DEC(sched_idx), SCH_ENC(ms->sched_idx_enc), + NULL, NULL); + if (ret < 0) + return ret; + + ret = sch_connect(mux->sch, SCH_ENC(ms->sched_idx_enc), + SCH_MSTREAM(ost->file_index, ms->sched_idx), + NULL, NULL); + if (ret < 0) + return ret; + } else { + ret = sch_connect(mux->sch, SCH_DSTREAM(ost->ist->file_index, sched_idx), + SCH_MSTREAM(ost->file_index, ms->sched_idx), NULL, NULL); + if (ret < 0) + return ret; } } @@ -2617,7 +2667,7 @@ static Muxer *mux_alloc(void) return mux; } -int of_open(const OptionsContext *o, const char *filename) +int of_open(const OptionsContext *o, const char *filename, Scheduler *sch) { Muxer *mux; AVFormatContext *oc; @@ -2687,6 +2737,12 @@ int of_open(const OptionsContext *o, const char *filename) AVFMT_FLAG_BITEXACT); } + err = sch_add_mux(sch, muxer_thread, NULL, mux, + !strcmp(oc->oformat->name, "rtp")); + if (err < 0) + return err; + mux->sch = sch; + /* create all output streams for this file */ err = create_streams(mux, o); if (err < 0) diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 304471dd03..d463306546 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -28,6 +28,7 @@ #endif #include "ffmpeg.h" +#include "ffmpeg_sched.h" #include "cmdutils.h" #include "opt_common.h" #include "sync_queue.h" @@ -1157,20 +1158,22 @@ static int opt_audio_qscale(void *optctx, const char *opt, const char *arg) static int opt_filter_complex(void *optctx, const char *opt, const char *arg) { + Scheduler *sch = optctx; char *graph_desc = av_strdup(arg); if (!graph_desc) return AVERROR(ENOMEM); - return fg_create(NULL, graph_desc); + return fg_create(NULL, graph_desc, sch); } static int opt_filter_complex_script(void *optctx, const char *opt, const char *arg) { + Scheduler *sch = optctx; char *graph_desc = file_read(arg); if (!graph_desc) return AVERROR(EINVAL); - return fg_create(NULL, graph_desc); + return fg_create(NULL, graph_desc, sch); } void show_help_default(const char *opt, const char *arg) @@ -1262,8 +1265,9 @@ static const OptionGroupDef groups[] = { [GROUP_INFILE] = { "input url", "i", OPT_INPUT }, }; -static int open_files(OptionGroupList *l, const char *inout, - int (*open_file)(const OptionsContext*, const char*)) +static int open_files(OptionGroupList *l, const char *inout, Scheduler *sch, + int (*open_file)(const OptionsContext*, const char*, + Scheduler*)) { int i, ret; @@ -1283,7 +1287,7 @@ static int open_files(OptionGroupList *l, const char *inout, } av_log(NULL, AV_LOG_DEBUG, "Opening an %s file: %s.\n", inout, g->arg); - ret = open_file(&o, g->arg); + ret = open_file(&o, g->arg, sch); uninit_options(&o); if (ret < 0) { av_log(NULL, AV_LOG_ERROR, "Error opening %s file %s.\n", @@ -1296,7 +1300,7 @@ static int open_files(OptionGroupList *l, const char *inout, return 0; } -int ffmpeg_parse_options(int argc, char **argv) +int ffmpeg_parse_options(int argc, char **argv, Scheduler *sch) { OptionParseContext octx; const char *errmsg = NULL; @@ -1313,7 +1317,7 @@ int ffmpeg_parse_options(int argc, char **argv) } /* apply global options */ - ret = parse_optgroup(NULL, &octx.global_opts); + ret = parse_optgroup(sch, &octx.global_opts); if (ret < 0) { errmsg = "parsing global options"; goto fail; @@ -1323,7 +1327,7 @@ int ffmpeg_parse_options(int argc, char **argv) term_init(); /* open input files */ - ret = open_files(&octx.groups[GROUP_INFILE], "input", ifile_open); + ret = open_files(&octx.groups[GROUP_INFILE], "input", sch, ifile_open); if (ret < 0) { errmsg = "opening input files"; goto fail; @@ -1337,7 +1341,7 @@ int ffmpeg_parse_options(int argc, char **argv) } /* open output files */ - ret = open_files(&octx.groups[GROUP_OUTFILE], "output", of_open); + ret = open_files(&octx.groups[GROUP_OUTFILE], "output", sch, of_open); if (ret < 0) { errmsg = "opening output files"; goto fail; diff --git a/fftools/ffmpeg_sched.c b/fftools/ffmpeg_sched.c new file mode 100644 index 0000000000..de7070906a --- /dev/null +++ b/fftools/ffmpeg_sched.c @@ -0,0 +1,1703 @@ +/* + * Inter-thread scheduling/synchronization. + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <stdatomic.h> +#include <stddef.h> +#include <stdint.h> + +#include "cmdutils.h" +#include "ffmpeg_sched.h" +#include "sync_queue.h" +#include "thread_queue.h" + +// TODO: try to get rid of this +#include "ffmpeg.h" + +#include "libavcodec/bsf.h" +#include "libavcodec/packet.h" + +#include "libavutil/avassert.h" +#include "libavutil/error.h" +#include "libavutil/frame.h" +#include "libavutil/mem.h" +#include "libavutil/thread.h" + +// 100 ms +// XXX: some other value? make this dynamic? +#define SCHEDULE_TOLERANCE (100 * 1000) + +enum QueueType { + QUEUE_PACKETS, + QUEUE_FRAMES, +}; + +typedef struct SchTask { + SchThreadFunc func; + void *func_arg; + + pthread_t thread; + int thread_running; +} SchTask; + +typedef struct SchDec { + SchedulerNode src; + SchedulerNode *dst; + uint8_t *dst_finished; + unsigned nb_dst; + + SchTask task; + // Queue for receiving input packets, one stream. + ThreadQueue *queue; + + // temporary storage used by sch_dec_send() + AVFrame *send_frame; +} SchDec; + +typedef struct SchSyncQueue { + SyncQueue *sq; + AVFrame *frame; + pthread_mutex_t lock; + + unsigned *enc_idx; + unsigned nb_enc_idx; +} SchSyncQueue; + +typedef struct SchEnc { + SchedulerNode src; + SchedulerNode dst; + + // [0] - index of the sync queue in Scheduler.sq_enc, + // [1] - index of this encoder in the sq + int sq_idx[2]; + + pthread_mutex_t open_lock; + pthread_cond_t open_cond; + int (*open_cb)(void *opaque, const AVFrame *frame); + int can_open; + int opened; + + SchTask task; + // Queue for receiving input frames, one stream. + ThreadQueue *queue; +} SchEnc; + +typedef struct SchDemuxStream { + SchedulerNode *dst; + uint8_t *dst_finished; + unsigned nb_dst; +} SchDemuxStream; + +typedef struct SchDemux { + SchDemuxStream *streams; + unsigned nb_streams; + + SchTask task; + + pthread_mutex_t demux_lock; + pthread_cond_t demux_cond; + atomic_int can_demux; + int terminate; + + int finished; + + // temporary storage used by sch_demux_send() + AVPacket *send_pkt; + + // the following must not be accessed outside of schedule_update_locked() + int can_demux_prev; + int can_demux_next; +} SchDemux; + +typedef struct SchMuxStream { + SchedulerNode src; + SchedulerNode src_sched; + + // XXX + int (*hook)(void *opaque, void *item); + void *opaque; + + AVBSFContext *bsf_ctx; + AVPacket *bsf_pkt; + + //////////////////////////////////////////////////////////// + // The following are protected by Scheduler.schedule_lock // + + /* dts of the last packet sent to this stream + in AV_TIME_BASE_Q */ + int64_t last_dts; + int source_blocked; + // this stream no longer accepts input + int finished; + //////////////////////////////////////////////////////////// +} SchMuxStream; + +typedef struct SchMux { + SchMuxStream *streams; + unsigned nb_streams; + unsigned nb_streams_ready; + + int (*init)(void *arg); + + SchTask task; + ThreadQueue *queue; +} SchMux; + +typedef struct SchFilterIn { + SchedulerNode src; + SchedulerNode src_sched; +} SchFilterIn; + +typedef struct SchFilterOut { + SchedulerNode dst; +} SchFilterOut; + +typedef struct SchFilterGraph { + SchFilterIn *inputs; + unsigned nb_inputs; + + SchFilterOut *outputs; + unsigned nb_outputs; + + SchTask task; + ThreadQueue *queue; + + // protected by schedule_lock + int best_input; +} SchFilterGraph; + +struct Scheduler { + SchDemux *demux; + unsigned nb_demux; + + SchMux *mux; + unsigned nb_mux; + + unsigned nb_mux_ready; + pthread_mutex_t mux_ready_lock; + + unsigned nb_mux_done; + pthread_mutex_t mux_done_lock; + pthread_cond_t mux_done_cond; + + + SchDec *dec; + unsigned nb_dec; + + SchEnc *enc; + unsigned nb_enc; + + SchSyncQueue *sq_enc; + unsigned nb_sq_enc; + + SchFilterGraph *filters; + unsigned nb_filters; + + char *sdp_filename; + int sdp_auto; + + int transcode_started; + + pthread_mutex_t schedule_lock; +}; + +static int queue_alloc(ThreadQueue **ptq, unsigned nb_streams, unsigned queue_size, + enum QueueType type) +{ + ThreadQueue *tq; + ObjPool *op; + + op = (type == QUEUE_PACKETS) ? objpool_alloc_packets() : + objpool_alloc_frames(); + if (!op) + return AVERROR(ENOMEM); + + tq = tq_alloc(nb_streams, queue_size, op, + (type == QUEUE_PACKETS) ? pkt_move : frame_move); + if (!tq) { + objpool_free(&op); + return AVERROR(ENOMEM); + } + + *ptq = tq; + return 0; +} + +static int task_stop(SchTask *task) +{ + int ret; + void *thread_ret; + + if (!task->thread_running) + return 0; + + ret = pthread_join(task->thread, &thread_ret); + av_assert0(ret == 0); + + task->thread_running = 0; + + return (intptr_t)thread_ret; +} + +static int task_start(SchTask *task) +{ + int ret; + + av_assert0(!task->thread_running); + + ret = pthread_create(&task->thread, NULL, task->func, task->func_arg); + if (ret) { + av_log(NULL, AV_LOG_ERROR, "pthread_create() failed: %s\n", strerror(ret)); + return AVERROR(ret); + } + + task->thread_running = 1; + return 0; +} + +int sch_stop(Scheduler *sch) +{ + int ret = 0, err; + + // XXX ensure no threads can get stuck + + for (unsigned i = 0; i < sch->nb_demux; i++) { + SchDemux *d = &sch->demux[i]; + + pthread_mutex_lock(&d->demux_lock); + + d->terminate = 1; + atomic_store(&d->can_demux, 0); + pthread_cond_signal(&d->demux_cond); + + pthread_mutex_unlock(&d->demux_lock); + + err = task_stop(&d->task); + ret = err_merge(ret, err); + } + + for (unsigned i = 0; i < sch->nb_dec; i++) { + SchDec *dec = &sch->dec[i]; + + // XXX should not be needed? + //tq_send_finish(dec->queue, 0); + + err = task_stop(&dec->task); + ret = err_merge(ret, err); + } + + for (unsigned i = 0; i < sch->nb_filters; i++) { + SchFilterGraph *fg = &sch->filters[i]; + + err = task_stop(&fg->task); + ret = err_merge(ret, err); + } + + for (unsigned i = 0; i < sch->nb_enc; i++) { + SchEnc *enc = &sch->enc[i]; + + err = task_stop(&enc->task); + ret = err_merge(ret, err); + } + + for (unsigned i = 0; i < sch->nb_mux; i++) { + SchMux *mux = &sch->mux[i]; + + err = task_stop(&mux->task); + ret = err_merge(ret, err); + } + + return ret; +} + +void sch_free(Scheduler **psch) +{ + Scheduler *sch = *psch; + + if (!sch) + return; + + sch_stop(sch); + + for (unsigned i = 0; i < sch->nb_demux; i++) { + SchDemux *d = &sch->demux[i]; + + for (unsigned j = 0; j < d->nb_streams; j++) { + SchDemuxStream *ds = &d->streams[j]; + av_freep(&ds->dst); + av_freep(&ds->dst_finished); + } + av_freep(&d->streams); + + av_packet_free(&d->send_pkt); + + pthread_mutex_destroy(&d->demux_lock); + pthread_cond_destroy(&d->demux_cond); + } + av_freep(&sch->demux); + + for (unsigned i = 0; i < sch->nb_mux; i++) { + SchMux *mux = &sch->mux[i]; + + for (unsigned j = 0; j < mux->nb_streams; j++) { + SchMuxStream *ms = &mux->streams[j]; + + av_bsf_free(&ms->bsf_ctx); + av_packet_free(&ms->bsf_pkt); + } + + av_freep(&mux->streams); + + tq_free(&mux->queue); + } + av_freep(&sch->mux); + + for (unsigned i = 0; i < sch->nb_dec; i++) { + SchDec *dec = &sch->dec[i]; + + tq_free(&dec->queue); + + av_freep(&dec->dst); + av_freep(&dec->dst_finished); + + av_frame_free(&dec->send_frame); + } + av_freep(&sch->dec); + + for (unsigned i = 0; i < sch->nb_enc; i++) { + SchEnc *enc = &sch->enc[i]; + + tq_free(&enc->queue); + } + av_freep(&sch->enc); + + for (unsigned i = 0; i < sch->nb_sq_enc; i++) { + SchSyncQueue *sq = &sch->sq_enc[i]; + sq_free(&sq->sq); + av_frame_free(&sq->frame); + pthread_mutex_destroy(&sq->lock); + av_freep(&sq->enc_idx); + } + av_freep(&sch->sq_enc); + + for (unsigned i = 0; i < sch->nb_filters; i++) { + SchFilterGraph *fg = &sch->filters[i]; + + tq_free(&fg->queue); + + av_freep(&fg->inputs); + av_freep(&fg->outputs); + } + av_freep(&sch->filters); + + av_freep(&sch->sdp_filename); + + pthread_mutex_destroy(&sch->mux_ready_lock); + + pthread_mutex_destroy(&sch->mux_done_lock); + pthread_cond_destroy(&sch->mux_done_cond); + + av_freep(psch); +} + +Scheduler *sch_alloc(void) +{ + Scheduler *sch; + int ret; + + sch = av_mallocz(sizeof(*sch)); + if (!sch) + return NULL; + + sch->sdp_auto = 1; + + ret = pthread_mutex_init(&sch->mux_ready_lock, NULL); + if (ret) + goto fail; + + ret = pthread_mutex_init(&sch->mux_done_lock, NULL); + if (ret) + goto fail; + + ret = pthread_cond_init(&sch->mux_done_cond, NULL); + if (ret) + goto fail; + + return sch; +fail: + sch_free(&sch); + return NULL; +} + +int sch_sdp_filename(Scheduler *sch, const char *sdp_filename) +{ + av_freep(&sch->sdp_filename); + sch->sdp_filename = av_strdup(sdp_filename); + return sch->sdp_filename ? 0 : AVERROR(ENOMEM); +} + +int sch_add_mux(Scheduler *sch, SchThreadFunc func, int (*init)(void *), + void *arg, int sdp_auto) +{ + SchMux *mux; + int ret; + + ret = GROW_ARRAY(sch->mux, sch->nb_mux); + if (ret < 0) + return ret; + + mux = &sch->mux[sch->nb_mux - 1]; + mux->task.func = func; + mux->task.func_arg = arg; + + mux->init = init; + + sch->sdp_auto &= sdp_auto; + + return mux - sch->mux; +} + +int sch_add_mux_stream(Scheduler *sch, unsigned mux_idx) +{ + SchMux *mux; + unsigned stream_idx; + int ret; + + av_assert0(mux_idx < sch->nb_mux); + mux = &sch->mux[mux_idx]; + + ret = GROW_ARRAY(mux->streams, mux->nb_streams); + if (ret < 0) + return ret; + stream_idx = mux->nb_streams - 1; + + mux->streams[stream_idx].last_dts = AV_NOPTS_VALUE; + + return stream_idx; +} + +int sch_add_mux_stream_bsf(Scheduler *sch, unsigned mux_idx, unsigned stream_idx, + const char *bsf) +{ + SchMuxStream *ms; + int ret; + + av_assert0(mux_idx < sch->nb_mux && stream_idx < sch->mux[mux_idx].nb_streams); + ms = &sch->mux[mux_idx].streams[stream_idx]; + + av_assert0(!ms->bsf_ctx); + + ret = av_bsf_list_parse_str(bsf, &ms->bsf_ctx); + if (ret < 0) + return ret; + + ms->bsf_pkt = av_packet_alloc(); + if (!ms->bsf_pkt) + return AVERROR(ENOMEM); + + return 0; +} + +int sch_add_demux(Scheduler *sch, SchThreadFunc func, void *func_arg) +{ + SchDemux *d; + int ret; + + ret = GROW_ARRAY(sch->demux, sch->nb_demux); + if (ret < 0) + return ret; + + d = &sch->demux[sch->nb_demux - 1]; + d->task.func = func; + d->task.func_arg = func_arg; + + d->send_pkt = av_packet_alloc(); + if (!d->send_pkt) + return AVERROR(ENOMEM); + + ret = pthread_mutex_init(&d->demux_lock, NULL); + if (ret) + return AVERROR(errno); + + ret = pthread_cond_init(&d->demux_cond, NULL); + if (ret) + return AVERROR(errno); + + return d - sch->demux; +} + +int sch_add_demux_stream(Scheduler *sch, unsigned demux_idx) +{ + SchDemux *d; + int ret; + + av_assert0(demux_idx < sch->nb_demux); + d = &sch->demux[demux_idx]; + + ret = GROW_ARRAY(d->streams, d->nb_streams); + return ret < 0 ? ret : d->nb_streams - 1; +} + +int sch_add_dec(Scheduler *sch, SchThreadFunc func, void *func_arg) +{ + SchDec *dec; + int ret; + + ret = GROW_ARRAY(sch->dec, sch->nb_dec); + if (ret < 0) + return ret; + + dec = &sch->dec[sch->nb_dec - 1]; + dec->task.func = func; + dec->task.func_arg = func_arg; + + dec->send_frame = av_frame_alloc(); + if (!dec->send_frame) + return AVERROR(ENOMEM); + + ret = queue_alloc(&dec->queue, 1, 1, QUEUE_PACKETS); + if (ret < 0) + return ret; + + return dec - sch->dec; +} + +int sch_add_enc(Scheduler *sch, SchThreadFunc func, void *func_arg, + int (*open_cb)(void *opaque, const AVFrame *frame)) +{ + SchEnc *enc; + int ret; + + ret = GROW_ARRAY(sch->enc, sch->nb_enc); + if (ret < 0) + return ret; + + enc = &sch->enc[sch->nb_enc - 1]; + enc->task.func = func; + enc->task.func_arg = func_arg; + enc->open_cb = open_cb; + + ret = pthread_mutex_init(&enc->open_lock, NULL); + if (ret) + return AVERROR(ret); + + ret = pthread_cond_init(&enc->open_cond, NULL); + if (ret) + return AVERROR(ret); + + enc->sq_idx[0] = -1; + enc->sq_idx[1] = -1; + + ret = queue_alloc(&enc->queue, 1, 1, QUEUE_FRAMES); + if (ret < 0) + return ret; + + return enc - sch->enc; +} + +int sch_add_filtergraph(Scheduler *sch, unsigned nb_inputs, unsigned nb_outputs, + SchThreadFunc func, void *func_arg) +{ + SchFilterGraph *fg; + int ret; + + ret = GROW_ARRAY(sch->filters, sch->nb_filters); + if (ret < 0) + return ret; + fg = &sch->filters[sch->nb_filters - 1]; + + if (nb_inputs) { + fg->inputs = av_calloc(nb_inputs, sizeof(*fg->inputs)); + if (!fg->inputs) + return AVERROR(ENOMEM); + fg->nb_inputs = nb_inputs; + } + + if (nb_outputs) { + fg->outputs = av_calloc(nb_outputs, sizeof(*fg->outputs)); + if (!fg->outputs) + return AVERROR(ENOMEM); + fg->nb_outputs = nb_outputs; + } + + fg->best_input = nb_inputs ? 0 : -1; + fg->task.func = func; + fg->task.func_arg = func_arg; + + ret = queue_alloc(&fg->queue, fg->nb_inputs, 1, QUEUE_FRAMES); + if (ret < 0) + return ret; + + return fg - sch->filters; +} + +int sch_add_sq_enc(Scheduler *sch, uint64_t buf_size_us, void *logctx) +{ + SchSyncQueue *sq; + int ret; + + ret = GROW_ARRAY(sch->sq_enc, sch->nb_sq_enc); + if (ret < 0) + return ret; + sq = &sch->sq_enc[sch->nb_sq_enc - 1]; + + sq->sq = sq_alloc(SYNC_QUEUE_FRAMES, buf_size_us, logctx); + if (!sq->sq) + return AVERROR(ENOMEM); + + sq->frame = av_frame_alloc(); + if (!sq->frame) + return AVERROR(ENOMEM); + + ret = pthread_mutex_init(&sq->lock, NULL); + if (ret) + return AVERROR(ret); + + return sq - sch->sq_enc; +} + +int sch_sq_add_enc(Scheduler *sch, unsigned sq_idx, unsigned enc_idx, + int limiting, uint64_t max_frames) +{ + SchSyncQueue *sq; + SchEnc *enc; + int ret; + + av_assert0(sq_idx < sch->nb_sq_enc); + sq = &sch->sq_enc[sq_idx]; + + av_assert0(enc_idx < sch->nb_enc); + enc = &sch->enc[enc_idx]; + + ret = GROW_ARRAY(sq->enc_idx, sq->nb_enc_idx); + if (ret < 0) + return ret; + sq->enc_idx[sq->nb_enc_idx - 1] = enc_idx; + + ret = sq_add_stream(sq->sq, limiting); + if (ret < 0) + return ret; + + enc->sq_idx[0] = sq_idx; + enc->sq_idx[1] = ret; + + if (max_frames != INT64_MAX) + sq_limit_frames(sq->sq, enc->sq_idx[1], max_frames); + + return 0; +} + +int sch_connect(Scheduler *sch, SchedulerNode src, SchedulerNode dst, + int (*hook)(void *opaque, void *item), void *opaque) +{ + int ret; + + // XXX hack + if (hook) + av_assert0(src.type == SCH_NODE_TYPE_DEMUX && dst.type == SCH_NODE_TYPE_MUX); + + switch (src.type) { + case SCH_NODE_TYPE_DEMUX: { + SchDemuxStream *ds; + + av_assert0(src.idx < sch->nb_demux && + src.idx_stream < sch->demux[src.idx].nb_streams); + ds = &sch->demux[src.idx].streams[src.idx_stream]; + + ret = GROW_ARRAY(ds->dst, ds->nb_dst); + if (ret < 0) + return ret; + + ds->dst[ds->nb_dst - 1] = dst; + + // demuxed packets go to decoding or streamcopy + switch (dst.type) { + case SCH_NODE_TYPE_DEC: { + SchDec *dec; + + av_assert0(dst.idx < sch->nb_dec); + dec = &sch->dec[dst.idx]; + + av_assert0(!dec->src.type); + dec->src = src; + break; + } + case SCH_NODE_TYPE_MUX: { + SchMuxStream *ms; + + av_assert0(dst.idx < sch->nb_mux && + dst.idx_stream < sch->mux[dst.idx].nb_streams); + ms = &sch->mux[dst.idx].streams[dst.idx_stream]; + + av_assert0(!ms->src.type); + ms->src = src; + + ms->hook = hook; + ms->opaque = opaque; + + break; + } + default: av_assert0(0); + } + + break; + } + case SCH_NODE_TYPE_DEC: { + SchDec *dec; + + av_assert0(src.idx < sch->nb_dec); + dec = &sch->dec[src.idx]; + + ret = GROW_ARRAY(dec->dst, dec->nb_dst); + if (ret < 0) + return ret; + + dec->dst[dec->nb_dst - 1] = dst; + + // decoded frames go to filters or encoding + switch (dst.type) { + case SCH_NODE_TYPE_FILTER_IN: { + SchFilterIn *fi; + + av_assert0(dst.idx < sch->nb_filters && + dst.idx_stream < sch->filters[dst.idx].nb_inputs); + fi = &sch->filters[dst.idx].inputs[dst.idx_stream]; + + av_assert0(!fi->src.type); + fi->src = src; + break; + } + case SCH_NODE_TYPE_ENC: { + SchEnc *enc; + + av_assert0(dst.idx < sch->nb_enc); + enc = &sch->enc[dst.idx]; + + av_assert0(!enc->src.type); + enc->src = src; + break; + } + default: av_assert0(0); + } + + break; + } + case SCH_NODE_TYPE_FILTER_OUT: { + SchFilterOut *fo; + SchEnc *enc; + + av_assert0(src.idx < sch->nb_filters && + src.idx_stream < sch->filters[src.idx].nb_outputs); + // filtered frames go to encoding + av_assert0(dst.type == SCH_NODE_TYPE_ENC && + dst.idx < sch->nb_enc); + + fo = &sch->filters[src.idx].outputs[src.idx_stream]; + enc = &sch->enc[dst.idx]; + + av_assert0(!fo->dst.type && !enc->src.type); + fo->dst = dst; + enc->src = src; + + break; + } + case SCH_NODE_TYPE_ENC: { + SchEnc *enc; + SchMuxStream *ms; + + av_assert0(src.idx < sch->nb_enc); + // encoding packets go to muxing + av_assert0(dst.type == SCH_NODE_TYPE_MUX && + dst.idx < sch->nb_mux && + dst.idx_stream < sch->mux[dst.idx].nb_streams); + enc = &sch->enc[src.idx]; + ms = &sch->mux[dst.idx].streams[dst.idx_stream]; + + av_assert0(!enc->dst.type && !ms->src.type); + enc->dst = dst; + ms->src = src; + + break; + } + default: av_assert0(0); + } + + return 0; +} + +static int mux_init(Scheduler *sch, SchMux *mux) +{ + int ret; + + ret = mux->init(mux->task.func_arg); + if (ret < 0) + return ret; + + sch->nb_mux_ready++; + + // XXX: test this + if (sch->sdp_filename || sch->sdp_auto) { + if (sch->nb_mux_ready < sch->nb_mux) + return 0; + + ret = print_sdp(sch->sdp_filename); + if (ret < 0) { + av_log(NULL, AV_LOG_ERROR, "Error writing the SDP.\n"); + return ret; + } + + /* SDP is written only after all the muxers are ready, so now we + * start ALL the threads */ + for (int i = 0; i < sch->nb_mux; i++) { + ret = task_start(&sch->mux[i].task); + if (ret < 0) + return ret; + } + } else { + ret = task_start(&mux->task); + if (ret < 0) + return ret; + } + + // XXX +#if 0 + /* flush the muxing queues */ + for (int i = 0; i < fc->nb_streams; i++) { + OutputStream *ost = mux->of.streams[i]; + MuxStream *ms = ms_from_ost(ost); + AVPacket *pkt; + + while (av_fifo_read(ms->muxing_queue, &pkt, 1) >= 0) { + ret = thread_submit_packet(mux, ost, pkt); + if (pkt) { + ms->muxing_queue_data_size -= pkt->size; + av_packet_free(&pkt); + } + if (ret < 0) + return ret; + } + } +#endif + + return 0; +} + +static int64_t trailing_dts(const Scheduler *sch) +{ + int64_t min_dts = INT64_MAX; + + for (unsigned i = 0; i < sch->nb_mux; i++) { + const SchMux *mux = &sch->mux[i]; + + for (int j = 0; j < mux->nb_streams; j++) { + const SchMuxStream *ms = &mux->streams[j]; + + if (ms->finished) + continue; + if (ms->last_dts == AV_NOPTS_VALUE) + return AV_NOPTS_VALUE; + + min_dts = FFMIN(min_dts, ms->last_dts); + } + } + + return min_dts == INT64_MAX ? AV_NOPTS_VALUE : min_dts; +} + +static void schedule_update_locked(Scheduler *sch) +{ + int64_t dts = trailing_dts(sch); + + // initialize our internal state + // XXX handle filtergraphs with no inputs here + for (unsigned i = 0; i < sch->nb_demux; i++) { + SchDemux *d = &sch->demux[i]; + d->can_demux_prev = atomic_load(&d->can_demux); + d->can_demux_next = 0; + } + + // figure out the sources that are allowed to proceed + for (unsigned i = 0; i < sch->nb_mux; i++) { + SchMux *mux = &sch->mux[i]; + + for (int j = 0; j < mux->nb_streams; j++) { + SchMuxStream *ms = &mux->streams[j]; + SchDemux *d; + + // unblock sources for output streams that are not finished + // and not too far ahead of the trailing stream + if (ms->finished) + continue; + if (dts == AV_NOPTS_VALUE && ms->last_dts != AV_NOPTS_VALUE) + continue; + if (dts != AV_NOPTS_VALUE && ms->last_dts - dts >= SCHEDULE_TOLERANCE) + continue; + + // for outputs fed from filtergraphs, consider that filtergraph's + // best_input information, in other cases there is a well-defined + // source demuxer + if (ms->src_sched.type == SCH_NODE_TYPE_FILTER_OUT) { + SchFilterGraph *fg = &sch->filters[ms->src_sched.idx]; + SchFilterIn *fi; + + // XXX handle filtergraphs with no inputs here + if (fg->best_input < 0) + continue; + fi = &fg->inputs[fg->best_input]; + + d = &sch->demux[fi->src_sched.idx]; + } else + d = &sch->demux[ms->src_sched.idx]; + + d->can_demux_next = 1; + } + } + + for (unsigned i = 0; i < sch->nb_demux; i++) { + SchDemux *d = &sch->demux[i]; + + if (d->can_demux_prev == d->can_demux_next) + continue; + + pthread_mutex_lock(&d->demux_lock); + + atomic_store(&d->can_demux, d->can_demux_next); + pthread_cond_signal(&d->demux_cond); + + pthread_mutex_unlock(&d->demux_lock); + } +} + +int sch_start(Scheduler *sch) +{ + int ret; + + sch->transcode_started = 1; + + // XXX add comprehensive logging + + for (unsigned i = 0; i < sch->nb_mux; i++) { + SchMux *mux = &sch->mux[i]; + + for (unsigned j = 0; j < mux->nb_streams; j++) { + SchMuxStream *ms = &mux->streams[j]; + + switch (ms->src.type) { + case SCH_NODE_TYPE_ENC: { + SchEnc *enc = &sch->enc[ms->src.idx]; + if (enc->src.type == SCH_NODE_TYPE_DEC) { + ms->src_sched = sch->dec[enc->src.idx].src; + av_assert0(ms->src_sched.type == SCH_NODE_TYPE_DEMUX); + } else { + ms->src_sched = enc->src; + av_assert0(ms->src_sched.type == SCH_NODE_TYPE_FILTER_OUT); + } + break; + } + case SCH_NODE_TYPE_DEMUX: + ms->src_sched = ms->src; + break; + default: + av_log(NULL, AV_LOG_ERROR, + "Muxer stream #%u:%u not connected to a source\n", i, j); + return AVERROR(EINVAL); + } + } + + // XXX should be special buffering queue + ret = queue_alloc(&mux->queue, mux->nb_streams, 1, QUEUE_PACKETS); + if (ret < 0) + return ret; + + if (mux->nb_streams_ready == mux->nb_streams) { + ret = mux_init(sch, mux); + if (ret < 0) + return ret; + } + } + + for (unsigned i = 0; i < sch->nb_enc; i++) { + SchEnc *enc = &sch->enc[i]; + + if (!enc->src.type) { + av_log(NULL, AV_LOG_ERROR, + "Encoder %u not connected to a source\n", i); + return AVERROR(EINVAL); + } + if (!enc->dst.type) { + av_log(NULL, AV_LOG_ERROR, + "Encoder %u not connected to a sink\n", i); + return AVERROR(EINVAL); + } + + ret = task_start(&enc->task); + if (ret < 0) + return ret; + } + + for (unsigned i = 0; i < sch->nb_filters; i++) { + SchFilterGraph *fg = &sch->filters[i]; + + for (unsigned j = 0; j < fg->nb_inputs; j++) { + SchFilterIn *fi = &fg->inputs[j]; + + if (!fi->src.type) { + av_log(NULL, AV_LOG_ERROR, + "Filtergraph %u input %u not connected to a source\n", i, j); + return AVERROR(EINVAL); + } + + fi->src_sched = sch->dec[fi->src.idx].src; + } + + for (unsigned j = 0; j < fg->nb_outputs; j++) { + SchFilterOut *fo = &fg->outputs[j]; + + if (!fo->dst.type) { + av_log(NULL, AV_LOG_ERROR, + "Filtergraph %u output %u not connected to a sink\n", i, j); + return AVERROR(EINVAL); + } + } + + ret = task_start(&fg->task); + if (ret < 0) + return ret; + } + + for (unsigned i = 0; i < sch->nb_dec; i++) { + SchDec *dec = &sch->dec[i]; + + if (!dec->src.type) { + av_log(NULL, AV_LOG_ERROR, + "Decoder %u not connected to a source\n", i); + return AVERROR(EINVAL); + } + if (!dec->nb_dst) { + av_log(NULL, AV_LOG_ERROR, + "Decoder %u not connected to any sink\n", i); + return AVERROR(EINVAL); + } + + dec->dst_finished = av_calloc(dec->nb_dst, sizeof(*dec->dst_finished)); + if (!dec->dst_finished) + return AVERROR(ENOMEM); + + ret = task_start(&dec->task); + if (ret < 0) + return ret; + } + + for (unsigned i = 0; i < sch->nb_demux; i++) { + SchDemux *d = &sch->demux[i]; + + // XXX: is this useful? + if (!d->nb_streams) + continue; + + for (unsigned j = 0; j < d->nb_streams; j++) { + SchDemuxStream *ds = &d->streams[j]; + + if (!ds->nb_dst) { + av_log(NULL, AV_LOG_ERROR, + "Demuxer stream #%u:%u not connected to any sink\n", i, j); + return AVERROR(EINVAL); + } + + ds->dst_finished = av_calloc(ds->nb_dst, sizeof(*ds->dst_finished)); + if (!ds->dst_finished) + return AVERROR(ENOMEM); + } + + ret = task_start(&d->task); + if (ret < 0) + return ret; + } + + pthread_mutex_lock(&sch->schedule_lock); + schedule_update_locked(sch); + pthread_mutex_unlock(&sch->schedule_lock); + + return 0; +} + +int sch_wait(Scheduler *sch, uint64_t timeout_us) +{ + int ret; + + pthread_mutex_lock(&sch->mux_done_lock); + + if (sch->nb_mux_done < sch->nb_mux) { + struct timespec tv = { .tv_sec = timeout_us / 1000000, + .tv_nsec = (timeout_us % 1000000) * 1000 }; + pthread_cond_timedwait(&sch->mux_done_cond, &sch->mux_done_lock, &tv); + } + + ret = sch->nb_mux_done == sch->nb_mux; + + pthread_mutex_unlock(&sch->mux_done_lock); + + return ret; +} + +static void enc_allow_open(SchEnc *enc) +{ + pthread_mutex_lock(&enc->open_lock); + + enc->can_open = 1; + pthread_cond_signal(&enc->open_cond); + + pthread_mutex_unlock(&enc->open_lock); +} + +static int sch_enc_open(Scheduler *sch, SchEnc *enc, const AVFrame *frame) +{ + int ret; + + // this is called from the sending thread (filter/decoder), + // so wait until the encoding thread is ready to be opened + pthread_mutex_lock(&enc->open_lock); + + while (!enc->can_open) + pthread_cond_wait(&enc->open_cond, &enc->open_lock); + + pthread_mutex_unlock(&enc->open_lock); + + ret = enc->open_cb(enc->task.func_arg, frame); + if (ret < 0) + return ret; + + // ret>0 signals audio frame size, which means sync queue should + // have been enabled during encoder creation + if (ret > 0) { + av_assert0(enc->sq_idx[0] >= 0); + sq_frame_samples(sch->sq_enc[enc->sq_idx[0]].sq, enc->sq_idx[1], ret); + } + + return 0; +} + +static int send_to_enc_thread(Scheduler *sch, SchEnc *enc, AVFrame *frame) +{ + if (frame) + return tq_send(enc->queue, 0, frame); + + tq_send_finish(enc->queue, 0); + return 0; +} + +static int send_to_sq(Scheduler *sch, SchSyncQueue *sq, + AVFrame *frame, unsigned stream_idx) +{ + int ret = 0; + + pthread_mutex_lock(&sq->lock); + + ret = sq_send(sq->sq, stream_idx, SQFRAME(frame)); + if (ret < 0) + goto finish; + + while (1) { + SchEnc *enc; + + // TODO: the SQ API should be extended to allow returning EOF + // for individual streams + ret = sq_receive(sq->sq, -1, SQFRAME(sq->frame)); + if (ret == AVERROR(EAGAIN)) { + ret = 0; + goto finish; + } else if (ret < 0) { + // close all encoders fed from this sync queue + for (unsigned i = 0; i < sq->nb_enc_idx; i++) { + int err = send_to_enc_thread(sch, &sch->enc[sq->enc_idx[i]], NULL); + + // if the sync queue error is EOF and closing the encoder + // produces a more serious error, make sure to pick the latter + ret = err_merge((ret == AVERROR_EOF && err < 0) ? 0 : ret, err); + } + goto finish; + } + + enc = &sch->enc[sq->enc_idx[ret]]; + ret = send_to_enc_thread(sch, enc, sq->frame); + av_frame_unref(sq->frame); + if (ret < 0) { + sq_send(sq->sq, enc->sq_idx[1], SQFRAME(NULL)); + goto finish; + } + } + +finish: + pthread_mutex_unlock(&sq->lock); + + return ret; +} + +static int send_to_enc(Scheduler *sch, SchEnc *enc, AVFrame *frame) +{ + if (enc->open_cb && frame && !enc->opened) { + int ret = sch_enc_open(sch, enc, frame); + if (ret < 0) + return ret; + enc->opened = 1; + } + + return (enc->sq_idx[0] >= 0) ? + send_to_sq(sch, &sch->sq_enc[enc->sq_idx[0]], frame, enc->sq_idx[1]) : + send_to_enc_thread(sch, enc, frame); +} + +static int send_to_mux(Scheduler *sch, SchMux *mux, unsigned stream_idx, + AVPacket *pkt) +{ + SchMuxStream *ms = &mux->streams[stream_idx]; + int64_t dts = AV_NOPTS_VALUE; + + if (pkt) { + int ret; + + if (pkt->dts != AV_NOPTS_VALUE) + dts = av_rescale_q(pkt->dts, pkt->time_base, AV_TIME_BASE_Q); + + ret = tq_send(mux->queue, stream_idx, pkt); + if (ret < 0) + return ret; + } else + tq_send_finish(mux->queue, stream_idx); + + // TODO: use atomics to check whether this changes trailing dts + // to avoid locking unnecesarily + if (dts != AV_NOPTS_VALUE || !pkt) { + pthread_mutex_lock(&sch->schedule_lock); + + if (pkt) ms->last_dts = dts; + else ms->finished = 1; + + schedule_update_locked(sch); + + pthread_mutex_unlock(&sch->schedule_lock); + } + + return 0; +} + +static int +demux_stream_send_to_dst(Scheduler *sch, const SchedulerNode dst, + uint8_t *dst_finished, AVPacket *pkt) +{ + int ret; + + if (*dst_finished) + return AVERROR_EOF; + + if (!pkt) + goto finish; + + if (dst.type == SCH_NODE_TYPE_MUX) { + SchMux *mux = &sch->mux[dst.idx]; + SchMuxStream *ms = &mux->streams[dst.idx_stream]; + + // XXX check if this can be dropped + if (ms->hook) { + ret = ms->hook(ms->opaque, pkt); + if (ret == AVERROR_EOF) + goto finish; + else if (ret < 0) + return (ret == AVERROR(EAGAIN)) ? 0 : ret; + } + + ret = send_to_mux(sch, mux, dst.idx_stream, pkt); + } else + ret = tq_send(sch->dec[dst.idx].queue, 0, pkt); + + if (ret == AVERROR_EOF) + goto finish; + + return ret; + +finish: + if (dst.type == SCH_NODE_TYPE_MUX) + send_to_mux(sch, &sch->mux[dst.idx], dst.idx_stream, pkt); + else + tq_send_finish(sch->dec[dst.idx].queue, 0); + + *dst_finished = 1; + return AVERROR_EOF; +} + +static int demux_send_for_stream(Scheduler *sch, SchDemux *d, + SchDemuxStream *ds, AVPacket *pkt) +{ + unsigned nb_done = 0; + + for (unsigned i = 0; i < ds->nb_dst; i++) { + AVPacket *to_send = pkt; + uint8_t *finished = &ds->dst_finished[i]; + + int ret; + + // sending a packet consumes it, so make a temporary reference if needed + if (pkt && !*finished && i < ds->nb_dst - 1) { + to_send = d->send_pkt; + + ret = av_packet_ref(to_send, pkt); + if (ret < 0) + return ret; + } + + ret = demux_stream_send_to_dst(sch, ds->dst[i], finished, to_send); + if (to_send) + av_packet_unref(to_send); + if (ret == AVERROR_EOF) + nb_done++; + else if (ret < 0) + return ret; + } + + // XXX sub2video_heartbeat() + + return (nb_done == ds->nb_dst) ? AVERROR_EOF : 0; +} + +int sch_demux_send(Scheduler *sch, unsigned demux_idx, AVPacket *pkt) +{ + SchDemux *d; + int ret = 0, terminate = 0; + + av_assert0(demux_idx < sch->nb_demux); + d = &sch->demux[demux_idx]; + + if (d->finished) + return AVERROR_EXIT; + + // sleep until the scheduling algorithm allows us to proceed + if (!atomic_load(&d->can_demux)) { + + pthread_mutex_lock(&d->demux_lock); + + while (!atomic_load(&d->can_demux) && !d->terminate) + pthread_cond_wait(&d->demux_cond, &d->demux_lock); + + terminate = d->terminate; + + pthread_mutex_unlock(&d->demux_lock); + } + + if (!pkt || terminate) { + for (unsigned i = 0; i < d->nb_streams; i++) { + ret = demux_send_for_stream(sch, d, &d->streams[i], NULL); + av_assert0(ret >= 0 || ret == AVERROR_EOF); + } + + pthread_mutex_lock(&sch->schedule_lock); + + d->finished = 1; + schedule_update_locked(sch); + + pthread_mutex_unlock(&sch->schedule_lock); + + return pkt ? AVERROR_EXIT : 0; + } + + + av_assert0(pkt->stream_index < d->nb_streams); + + return demux_send_for_stream(sch, d, &d->streams[pkt->stream_index], pkt); +} + +int sch_mux_receive(Scheduler *sch, unsigned mux_idx, AVPacket *pkt) +{ + SchMux *mux; + int ret, stream_idx; + + av_assert0(mux_idx < sch->nb_mux); + mux = &sch->mux[mux_idx]; + + ret = tq_receive(mux->queue, &stream_idx, pkt); + pkt->stream_index = stream_idx; + return ret; +} + +void sch_mux_receive_finish(Scheduler *sch, unsigned mux_idx, int stream_idx) +{ + SchMux *mux; + + av_assert0(mux_idx < sch->nb_mux); + mux = &sch->mux[mux_idx]; + + pthread_mutex_lock(&sch->schedule_lock); + + if (stream_idx >= 0) { + av_assert0(stream_idx < mux->nb_streams); + tq_receive_finish(mux->queue, stream_idx); + mux->streams[stream_idx].finished = 1; + } else { + // the muxer as a whole is done + for (unsigned i = 0; i < mux->nb_streams; i++) { + tq_receive_finish(mux->queue, i); + mux->streams[i].finished = 1; + } + + pthread_mutex_lock(&sch->mux_done_lock); + + av_assert0(sch->nb_mux_done < sch->nb_mux); + sch->nb_mux_done++; + + pthread_cond_signal(&sch->mux_done_cond); + + pthread_mutex_unlock(&sch->mux_done_lock); + } + + schedule_update_locked(sch); + + pthread_mutex_unlock(&sch->schedule_lock); +} + +int sch_mux_stream_ready(Scheduler *sch, unsigned mux_idx, unsigned stream_idx) +{ + SchMux *mux; + int ret = 0; + + av_assert0(mux_idx < sch->nb_mux); + mux = &sch->mux[mux_idx]; + + av_assert0(stream_idx < mux->nb_streams); + + pthread_mutex_lock(&sch->mux_ready_lock); + + av_assert0(mux->nb_streams_ready < mux->nb_streams); + + // this may be called during initialization - do not start + // threads before sch_start() is called + if (++mux->nb_streams_ready == mux->nb_streams && sch->transcode_started) + ret = mux_init(sch, mux); + + pthread_mutex_unlock(&sch->mux_ready_lock); + + return ret; +} + +int sch_dec_receive(Scheduler *sch, unsigned dec_idx, AVPacket *pkt) +{ + SchDec *dec; + int ret, dummy; + + av_assert0(dec_idx < sch->nb_dec); + dec = &sch->dec[dec_idx]; + + ret = tq_receive(dec->queue, &dummy, pkt); + av_assert0(dummy <= 0); + + return ret; +} + +static int send_to_filter(Scheduler *sch, SchFilterGraph *fg, + unsigned in_idx, AVFrame *frame) +{ + if (frame) + return tq_send(fg->queue, in_idx, frame); + + tq_send_finish(fg->queue, in_idx); + return 0; +} + +static int dec_send_to_dst(Scheduler *sch, const SchedulerNode dst, + uint8_t *dst_finished, AVFrame *frame) +{ + int ret; + + if (*dst_finished) + return AVERROR_EOF; + + if (!frame) + goto finish; + + ret = (dst.type == SCH_NODE_TYPE_FILTER_IN) ? + send_to_filter(sch, &sch->filters[dst.idx], dst.idx_stream, frame) : + send_to_enc(sch, &sch->enc[dst.idx], frame); + if (ret == AVERROR_EOF) + goto finish; + + return ret; + +finish: + if (dst.type == SCH_NODE_TYPE_FILTER_IN) + send_to_filter(sch, &sch->filters[dst.idx], dst.idx_stream, NULL); + else + send_to_enc(sch, &sch->enc[dst.idx], NULL); + + *dst_finished = 1; + + return AVERROR_EOF; +} + +int sch_dec_send(Scheduler *sch, unsigned dec_idx, AVFrame *frame) +{ + SchDec *dec; + int ret = 0; + unsigned nb_done = 0; + + av_assert0(dec_idx < sch->nb_dec); + dec = &sch->dec[dec_idx]; + + for (unsigned i = 0; i < dec->nb_dst; i++) { + uint8_t *finished = &dec->dst_finished[i]; + + AVFrame *to_send = frame; + + // sending a frame consumes it, so make a temporary reference if needed + if (frame && !*finished && i < dec->nb_dst - 1) { + to_send = dec->send_frame; + + // frame may sometimes contain props only, + // e.g. to signal EOF timestamp + ret = frame->buf[0] ? av_frame_ref(to_send, frame) : + av_frame_copy_props(to_send, frame); + if (ret < 0) + return ret; + } + + ret = dec_send_to_dst(sch, dec->dst[i], finished, to_send); + if (to_send) + av_frame_unref(to_send); + if (ret == AVERROR_EOF) { + nb_done++; + ret = 0; + continue; + } else if (ret < 0) + goto finish; + } + +finish: + // close the decoder's input queue at the end + if (!frame) { + tq_receive_finish(dec->queue, 0); + return 0; + } + + return ret < 0 ? ret : + (nb_done == dec->nb_dst) ? AVERROR_EOF : 0; +} + +int sch_enc_receive(Scheduler *sch, unsigned enc_idx, AVFrame *frame) +{ + SchEnc *enc; + int ret, dummy; + + av_assert0(enc_idx < sch->nb_enc); + enc = &sch->enc[enc_idx]; + + if (!enc->can_open) + enc_allow_open(enc); + + ret = tq_receive(enc->queue, &dummy, frame); + av_assert0(dummy <= 0); + + return ret; +} + +int sch_enc_send(Scheduler *sch, unsigned enc_idx, AVPacket *pkt) +{ + SchEnc *enc; + int ret; + + av_assert0(enc_idx < sch->nb_enc); + enc = &sch->enc[enc_idx]; + + // XXX this is for cases when sch_enc_receive() was never called + // should it be here or in sch_stop()? + if (!enc->can_open) { + av_assert0(!pkt); + enc_allow_open(enc); + } + + ret = send_to_mux(sch, &sch->mux[enc->dst.idx], enc->dst.idx_stream, pkt); + if (pkt) + av_packet_unref(pkt); + else { + // close the encoder's input queue at the end + tq_receive_finish(enc->queue, 0); + } + + return ret; +} + +int sch_filter_receive(Scheduler *sch, unsigned fg_idx, + int *in_idx, AVFrame *frame) +{ + SchFilterGraph *fg; + int ret; + + av_assert0(fg_idx < sch->nb_filters); + fg = &sch->filters[fg_idx]; + + av_assert0((*in_idx >= 0 || !fg->nb_inputs) && + *in_idx < (int)fg->nb_inputs); + + // update scheduling to account for desired input stream, if it changed + if (*in_idx != fg->best_input) { + pthread_mutex_lock(&sch->schedule_lock); + + fg->best_input = *in_idx; + schedule_update_locked(sch); + + pthread_mutex_unlock(&sch->schedule_lock); + } + + // XXX: handle graphs with no inputs + + ret = tq_receive(fg->queue, in_idx, frame); + av_assert0(*in_idx < (int)fg->nb_inputs); + + return ret; +} + +int sch_filter_send(Scheduler *sch, unsigned fg_idx, int out_idx, AVFrame *frame) +{ + SchFilterGraph *fg; + int ret; + + av_assert0(fg_idx < sch->nb_filters); + fg = &sch->filters[fg_idx]; + + if (out_idx < 0) { + av_assert0(!frame); + + for (unsigned i = 0; i < fg->nb_inputs; i++) + tq_receive_finish(fg->queue, i); + + // XXX update schedule here? + + for (unsigned i = 0; i < fg->nb_outputs; i++) { + SchEnc *enc = &sch->enc[fg->outputs[i].dst.idx]; + int err = send_to_enc(sch, enc, NULL); + ret = err_merge(ret, err); + } + + return ret; + } + + av_assert0(out_idx < fg->nb_outputs); + ret = send_to_enc(sch, &sch->enc[fg->outputs[out_idx].dst.idx], frame); + + if (frame) + av_frame_unref(frame); + + return ret; +} diff --git a/fftools/ffmpeg_sched.h b/fftools/ffmpeg_sched.h new file mode 100644 index 0000000000..d1f1a006b4 --- /dev/null +++ b/fftools/ffmpeg_sched.h @@ -0,0 +1,414 @@ +/* + * Inter-thread scheduling/synchronization. + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef FFTOOLS_FFMPEG_SCHED_H +#define FFTOOLS_FFMPEG_SCHED_H + +/* + * This file contains the API for the transcode scheduler. + * + * Overall architecture of the transcoding process involves instances of the + * following components: + * - demuxers, each containing any number of streams; demuxed packets belonging + * to some stream are sent to any number of decoders (transcoding) or + * muxers (streamcopy); + * - decoders, which receive encoded packets from some demuxed stream, decode + * them, and send decoded frames to any number of filtergraph inputs + * (audio/video) or encoders (subtitles); + * - filtergraphs, which receive decoded frames from some decoder on every + * input, filter them, and send filtered frames from each output to some + * encoder; a generic filtergraph may have zero or more inputs (0 in case the + * filtergraph contains a lavfi source filter), and one or more outputs; the + * inputs and outputs need not have matching media types; + * - encoders, which receive decoded frames from some decoder (subtitles) or + * some filtergraph output (audio/video), encode them, and send encoded + * packets to some muxed stream; + * - muxers, each containing any number of muxed streams; each muxed stream + * receives encoded packets from some demuxed stream (streamcopy) or some + * encoder (transcoding); those packets are interleaved and written out by the + * muxer. + * + * There must be at least one muxer instance, otherwise the transcode produces + * no output and is meaningless. Otherwise, in a generic transcoding scenario + * there may be arbitrary number of instances of any of the above components, + * interconnected in various ways. + * + * The code tries to keep all the output streams across all the muxers in sync + * (i.e. at the same DTS), which is accomplished by varying the rates at which + * packets are read from different demuxers. Note that the degree of control we + * have over synchronization is fundamentally limited - if some demuxed streams + * in the same input are interleaved at different rates than that at which they + * are to be muxed (e.g. because an input file is badly interleaved, or the user + * changed their speed by mismatching amounts), then there will be increasing + * amounts of buffering followed by eventual transcoding failure. + * + * N.B. 1: there are meaningful transcode scenarios with no demuxers, e.g. + * - encoding and muxing output from filtergraph(s) that have no inputs; + * - creating a file that contains nothing but attachments and/or metadata. + * + * N.B. 2: a filtergraph output could, in principle, feed multiple encoders, but + * this is unnecessary because the (a)split filter provides the same + * functionality. + * + * The scheduler, in the above model, is the master object that oversees and + * facilitates the transcoding process. The basic idea is that all instances + * of the abovementioned components communicate only with the scheduler and not + * with each other. The scheduler is then the single place containing the + * knowledge about the whole transcoding pipeline. + */ + +typedef struct Scheduler Scheduler; + +enum SchedulerNodeType { + SCH_NODE_TYPE_NONE = 0, + SCH_NODE_TYPE_DEMUX, + SCH_NODE_TYPE_MUX, + SCH_NODE_TYPE_DEC, + SCH_NODE_TYPE_ENC, + SCH_NODE_TYPE_FILTER_IN, + SCH_NODE_TYPE_FILTER_OUT, +}; + +typedef struct SchedulerNode { + enum SchedulerNodeType type; + unsigned idx; + unsigned idx_stream; +} SchedulerNode; + +typedef void* (*SchThreadFunc)(void *arg); + +#define SCH_DSTREAM(file, stream) \ + (SchedulerNode){ .type = SCH_NODE_TYPE_DEMUX, \ + .idx = file, .idx_stream = stream } +#define SCH_MSTREAM(file, stream) \ + (SchedulerNode){ .type = SCH_NODE_TYPE_MUX, \ + .idx = file, .idx_stream = stream } +#define SCH_DEC(decoder) \ + (SchedulerNode){ .type = SCH_NODE_TYPE_DEC, \ + .idx = decoder } +#define SCH_ENC(encoder) \ + (SchedulerNode){ .type = SCH_NODE_TYPE_ENC, \ + .idx = encoder } +#define SCH_FILTER_IN(filter, input) \ + (SchedulerNode){ .type = SCH_NODE_TYPE_FILTER_IN, \ + .idx = filter, .idx_stream = input } +#define SCH_FILTER_OUT(filter, output) \ + (SchedulerNode){ .type = SCH_NODE_TYPE_FILTER_OUT, \ + .idx = filter, .idx_stream = output } + +Scheduler *sch_alloc(void); +void sch_free(Scheduler **sch); + +int sch_start(Scheduler *sch); +int sch_stop(Scheduler *sch); + +/** + * Wait until transcoding terminates or the specified timeout elapses. + * + * @param timeout_us Amount of time in microseconds after which this function + * will timeout. + * + * @retval 0 waiting timed out, transcoding is not finished + * @retval 1 transcoding is finished + */ +int sch_wait(Scheduler *sch, uint64_t timeout_us); + +/** + * Add a demuxer to the scheduler. + * + * @retval ">=0" Index of the newly-created demuxer. + * @retval "<0" Error code. + */ +int sch_add_demux(Scheduler *sch, SchThreadFunc func, void *func_arg); +/** + * Add a demuxed stream for a previously added demuxer. + * + * @param demux_idx index previously returned by sch_add_demux() + * + * @retval ">=0" Index of the newly-created demuxed stream. + * @retval "<0" Error code. + */ +int sch_add_demux_stream(Scheduler *sch, unsigned demux_idx); + +/** + * Add a decoder to the scheduler. + * + * @retval ">=0" Index of the newly-created decoder. + * @retval "<0" Error code. + */ +int sch_add_dec(Scheduler *sch, SchThreadFunc func, void *func_arg); + +/** + * Add a filtergraph to the scheduler. + * + * @param nb_inputs Number of filtergraph inputs. + * @param nb_outputs number of filtergraph outputs + * + * @retval ">=0" Index of the newly-created filtergraph. + * @retval "<0" Error code. + */ +int sch_add_filtergraph(Scheduler *sch, unsigned nb_inputs, unsigned nb_outputs, + SchThreadFunc func, void *func_arg); + +/** + * Add a muxer to the scheduler. + * + * Note that muxer thread startup is more complicated than for other components, + * because + * - muxer streams fed by audio/video encoders become initialized dynamically at + * runtime, after those encoders receive their first frame and initialize + * themselves, followed by calling sch_mux_stream_ready() + * - the header can be written after all the streams for a muxer are initialized + * - we may need to write an SDP, which must happen + * - AFTER all the headers are written + * - BEFORE any packets are written by any muxer + * - with all the muxers quiescent + * To avoid complicated muxer-thread synchronization dances, we postpone + * starting the muxer threads until after the SDP is written. The sequence of + * events is then as follows: + * - After sch_mux_stream_ready() is called for all the streams in a given muxer, + * the header for that muxer is written (care is taken that headers for + * different muxers are not written concurrently, since they write file + * information to stderr). If SDP is not wanted, the muxer thread then starts + * and muxing begins. + * - When SDP _is_ wanted, no muxer threads start until the header for the last + * muxer is written. After that, the SDP is written, after which all the muxer + * threads are started at once. + * + * In order, for the above to work, the scheduler needs to be able to invoke + * just writing the header, which is the reason the init parameter exists. + * + * @param init Callback that is called to initialize the muxer and write the + * header. Called after sch_mux_stream_ready() is called for all the + * streams in the muxer. + * @param sdp_auto Determines automatic SDP writing - see sch_sdp_filename(). + * + * @retval ">=0" Index of the newly-created muxer. + * @retval "<0" Error code. + */ +int sch_add_mux(Scheduler *sch, SchThreadFunc func, int (*init)(void *), + void *arg, int sdp_auto); +/** + * Add a muxed stream for a previously added muxer. + * + * @param mux_idx index previously returned by sch_add_mux() + * + * @retval ">=0" Index of the newly-created muxed stream. + * @retval "<0" Error code. + */ +int sch_add_mux_stream(Scheduler *sch, unsigned mux_idx); + +/** + * Signal to the scheduler that the specified muxed stream is initialized and + * ready. Muxing is started once all the streams are ready. + */ +int sch_mux_stream_ready(Scheduler *sch, unsigned mux_idx, unsigned stream_idx); + +/** + * Set the file path for the SDP. + * + * The SDP is written when either of the following is true: + * - this function is called at least once + * - sdp_auto=1 is passed to EVERY call of sch_add_mux() + */ +int sch_sdp_filename(Scheduler *sch, const char *sdp_filename); + +/** + * Add an encoder to the scheduler. + * + * @param open_cb This callback, if specified, will be called when the first + * frame is obtained for this encoder. For audio encoders with a + * fixed frame size (which use a sync queue in the scheduler to + * rechunk frames), it must return that frame size on success. + * Otherwise (non-audio, variable frame size) it should return 0. + * + * @retval ">=0" Index of the newly-created encoder. + * @retval "<0" Error code. + */ +int sch_add_enc(Scheduler *sch, SchThreadFunc func, void *func_arg, + int (*open_cb)(void *func_arg, const AVFrame *frame)); + +/** + * Add an pre-encoding sync queue to the scheduler. + * + * @param buf_size_us Sync queue buffering size, passed to sq_alloc(). + * @param logctx Logging context for the sync queue. passed to sq_alloc(). + * + * @retval ">=0" Index of the newly-created sync queue. + * @retval "<0" Error code. + */ +int sch_add_sq_enc(Scheduler *sch, uint64_t buf_size_us, void *logctx); +int sch_sq_add_enc(Scheduler *sch, unsigned sq_idx, unsigned enc_idx, + int limiting, uint64_t max_frames); + +/** + * XXX: TODO + */ +int sch_add_mux_stream_bsf(Scheduler *sch, unsigned mux_idx, unsigned stream_idx, + const char *bsf); + +int sch_connect(Scheduler *sch, SchedulerNode src, SchedulerNode dst, + int (*hook)(void *opaque, void *item), void *opaque); + +/** + * Called by demuxing threads to send a demuxed packet or EOF to all its + * consumers. The stream is indentified by the packet's stream_index field. + * + * Every demuxer task must call this function with pkt=NULL exactly once, right + * before it exits - this call always succeeeds. + * + * @param demux_idx demuxer index + * @param pkt A demuxed packet to send or NULL to signal end of demuxing. + * If non-NULL, on success the packet is consumed and cleared + * by this function + * + * @retval "non-negative value" success + * @retval AVERROR_EOF all consumers for the stream are done + * @retval AVERROR_EXIT all consumers are done, should terminate demuxing + * @retval "anoter negative error code" other failure + */ +int sch_demux_send(Scheduler *sch, unsigned demux_idx, AVPacket *pkt); + +/** + * Called by decoders to receive a packet for decoding. + * + * @param dec_idx decoder index + * @param pkt Input packet will be written here on success. An empty packet + * signals that the decoder should be flushed, but more packets will + * follow (e.g. after seeking). + * + * @retval "non-negative value" success + * @retval AVERROR_EOF no more packets will arrive, should terminate decoding + * @retval "another negative error code" other failure + */ +int sch_dec_receive(Scheduler *sch, unsigned dec_idx, AVPacket *pkt); + +/** + * Called by decoder tasks to send a decoded frame or EOF downstream. + * + * Every decoder task must call this function with frame=NULL exactly once, right + * before it exits - this call always succeeeds. + * + * @param dec_idx Decoder index previously returned by sch_add_dec(). + * @param frame Decoded frame or NULL to signal end of decoding. + * If non-NULL, on success the frame is consumed and cleared + * by this function + * + * @retval ">=0" success + * @retval AVERROR_EOF all consumers are done, should terminate decoding + * @retval "another negative error code" other failure + */ +int sch_dec_send(Scheduler *sch, unsigned dec_idx, AVFrame *frame); + +/** + * Called by filtergraph tasks to obtain frames for filtering. Will wait for a + * frame to become available and return it in frame. + * + * @param fg_idx Filtergraph index previously returned by sch_add_filtergraph(). + * @param[in,out] in_idx On input contains the index of the input on which a frame + * is most desired. On output contains input index of the + * actually returned frame or EOF. + */ +int sch_filter_receive(Scheduler *sch, unsigned fg_idx, + int *in_idx, AVFrame *frame); +/** + * Called by filtergraph tasks to send a filtered frame or EOF to consumers. + * + * Every filter task must call this function with out_idx<0 exactly once, + * right before it exits - that call always succeeds. + * + * @param fg_idx Filtergraph index previously returned by sch_add_filtergraph(). + * @param out_idx Index of the output which produced the frame. When negative, + * this call signals EOF on all streams, i.e. the filtering + * task is being terminated - in that case frame must be NULL. + * @param frame The frame to send to consumers. When NULL, signals that no more + * frames will be produced for the specified output (or all outputs + * when out_idx<0). When non-NULL, the frame is always consumed and + * cleared by this function. + * + * @retval "non-negative value" success + * @retval AVERROR_EOF all consumers are done + * @retval "anoter negative error code" other failure + */ +int sch_filter_send(Scheduler *sch, unsigned fg_idx, int out_idx, AVFrame *frame); + +/** + * Called by encoder tasks to obtain frames for encoding. Will wait for a frame + * to become available and return it in frame. + * + * @param enc_idx Encoder index previously returned by sch_add_enc(). + * @param frame Newly-received frame will be stored here. Must be clean on + * entrance to this function. + * + * @retval 0 A frame was successfully delivered into frame. + * @retval AVERROR_EOF No more frames will be delivered, the encoder should + * flush everything and terminate. + * + */ +int sch_enc_receive(Scheduler *sch, unsigned enc_idx, AVFrame *frame); +/** + * Called by encoder tasks to send encoded packets or EOF downstream. + * Will not block. + * + * Every encoder task must call this function with pkt=NULL exactly once, + * right before it exits - that call always succeeds. + * + * @param enc_idx Encoder index previously returned by sch_add_enc(). + * @param pkt An encoded packet or NULL to signal EOF. When non-NULL, the + * packet will be consumed and cleared by this function on + * success. + * + * @retval 0 success + * @retval "<0" Error code. + */ +int sch_enc_send (Scheduler *sch, unsigned enc_idx, AVPacket *pkt); + +/** + * Called by muxer tasks to obtain packets for muxing. Will wait for a packet + * for any muxed stream to become available and return it in pkt. + * + * @param mux_idx Muxer index previously returned by sch_add_mux(). + * @param pkt Newly-received packet will be stored here. Must be clean + * on entrance to this function. + * + * @retval 0 A packet was successfully delivered into pkt. Its stream_index + * corresponds to a stream index previously returned from + * sch_add_mux_stream(). + * @retval AVERROR_EOF When pkt->stream_index is non-negative, this signals that + * no more packets will be delivered for this stream index. + * Otherwise this indicates that no more packets will be + * delivered for any stream and the muxer should therefore + * flush everything and terminate. + */ +int sch_mux_receive(Scheduler *sch, unsigned mux_idx, AVPacket *pkt); + +/** + * Called by muxer tasks to signal that a single stream or all streams will no + * longer accept input. + * + * Every muxer task must call this function with stream_idx=-1 exactly once, + * right before it exits. + * + * @param stream_idx A non-negative stream index to mark one stream as finished, + * -1 to mark all. + */ +void sch_mux_receive_finish(Scheduler *sch, unsigned mux_idx, int stream_idx); + +#endif /* FFTOOLS_FFMPEG_SCHED_H */ -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 22/27] WIP fftools/ffmpeg_demux: convert to the scheduler 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (20 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 21/27] WIP fftools/ffmpeg: add thread-aware transcode scheduling infrastructure Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 23/27] WIP fftools/ffmpeg_dec: " Anton Khirnov ` (5 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel --- fftools/ffmpeg.c | 2 +- fftools/ffmpeg.h | 12 ---- fftools/ffmpeg_demux.c | 131 +++++++++++++++++++---------------------- 3 files changed, 60 insertions(+), 85 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 995424ca93..00e57c4382 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -1061,7 +1061,7 @@ static int process_input(int file_index, AVPacket *pkt) InputStream *ist; int ret, i; - ret = ifile_get_packet(ifile, pkt); + ret = 0; if (ret == 1) { /* the input file is looped: flush the decoders */ diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 278216e5ff..4646c05bea 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -860,18 +860,6 @@ int64_t of_filesize(OutputFile *of); int ifile_open(const OptionsContext *o, const char *filename, Scheduler *sch); void ifile_close(InputFile **f); -/** - * Get next input packet from the demuxer. - * - * @param pkt the packet is written here when this function returns 0 - * @return - * - 0 when a packet has been read successfully - * - 1 when stream end was reached, but the stream is looped; - * caller should flush decoders and read from this demuxer again - * - a negative error code on failure - */ -int ifile_get_packet(InputFile *f, AVPacket *pkt); - int ist_output_add(InputStream *ist, OutputStream *ost); int ist_filter_add(InputStream *ist, InputFilter *ifilter, int is_simple); diff --git a/fftools/ffmpeg_demux.c b/fftools/ffmpeg_demux.c index 074546d517..9c0e54979e 100644 --- a/fftools/ffmpeg_demux.c +++ b/fftools/ffmpeg_demux.c @@ -21,8 +21,6 @@ #include "ffmpeg.h" #include "ffmpeg_sched.h" -#include "objpool.h" -#include "thread_queue.h" #include "libavutil/avassert.h" #include "libavutil/avstring.h" @@ -34,7 +32,6 @@ #include "libavutil/pixdesc.h" #include "libavutil/time.h" #include "libavutil/timestamp.h" -#include "libavutil/thread.h" #include "libavcodec/packet.h" @@ -65,6 +62,9 @@ typedef struct DemuxStream { double ts_scale; + // scheduler returned EOF for this stream + int finished; + int streamcopy_needed; int wrap_correction_done; @@ -115,11 +115,10 @@ typedef struct Demuxer { double readrate_initial_burst; Scheduler *sch; - ThreadQueue *thread_queue; - int thread_queue_size; - pthread_t thread; int read_started; + int nb_streams_used; + int nb_streams_finished; } Demuxer; static DemuxStream *ds_from_ist(InputStream *ist) @@ -503,6 +502,8 @@ static int input_packet_process(Demuxer *d, AVPacket *pkt) av_ts2timestr(input_files[ist->file_index]->ts_offset, &AV_TIME_BASE_Q)); } + pkt->stream_index = ds->sch_idx_stream; + return 0; } @@ -565,9 +566,14 @@ static void *input_thread(void *arg) discard_unused_programs(f); + // XXX + d->read_started = 1; + d->wallclock_start = av_gettime_relative(); while (1) { + DemuxStream *ds; + ret = av_read_frame(f->ctx, pkt); if (ret == AVERROR(EAGAIN)) { @@ -575,25 +581,32 @@ static void *input_thread(void *arg) continue; } if (ret < 0) { -#if 0 if (d->loop) { - /* signal looping to the consumer thread */ - pkt->opaque = (void*)(intptr_t)PKT_OPAQUE_SEEK; - ret = tq_send(d->thread_queue, 0, pkt); - if (ret >= 0) - ret = seek_to_start(d); + /* signal looping to our consumers */ + for (int i = 0; i < f->nb_streams; i++) { + pkt->opaque = (void*)(intptr_t)PKT_OPAQUE_SEEK; + pkt->stream_index = i; + + ret = sch_demux_send(d->sch, f->index, pkt); + // XXX + if (ret >= 0) + //ret = seek_to_start(d); + if (ret < 0) + break; + } if (ret >= 0) continue; /* fallthrough to the error path */ } -#endif if (ret == AVERROR_EOF) av_log(d, AV_LOG_VERBOSE, "EOF while reading input\n"); - else + else { av_log(d, AV_LOG_ERROR, "Error during demuxing: %s\n", av_err2str(ret)); + ret = exit_on_error ? ret : 0; + } break; } @@ -605,8 +618,9 @@ static void *input_thread(void *arg) /* the following test is needed in case new streams appear dynamically in stream : we ignore them */ - if (pkt->stream_index >= f->nb_streams || - f->streams[pkt->stream_index]->discard) { + ds = pkt->stream_index < f->nb_streams ? + ds_from_ist(f->streams[pkt->stream_index]) : NULL; + if (!ds || ds->ist.discard || ds->finished) { report_new_stream(d, pkt); av_packet_unref(pkt); continue; @@ -630,40 +644,47 @@ static void *input_thread(void *arg) if (f->readrate) readrate_sleep(d); - ret = tq_send(d->thread_queue, 0, pkt); - if (ret < 0) { - if (ret != AVERROR_EOF) + ret = sch_demux_send(d->sch, f->index, pkt); + if (ret == AVERROR_EOF) { + av_packet_unref(pkt); + + av_log(ds, AV_LOG_VERBOSE, "All consumers done\n"); + ds->finished = 1; + + if (++d->nb_streams_finished == d->nb_streams_used) { + av_log(f, AV_LOG_VERBOSE, "All streams' consumers done\n"); + break; + } + continue; + } else if (ret < 0) { + if (ret != AVERROR_EXIT) av_log(f, AV_LOG_ERROR, - "Unable to send packet to main thread: %s\n", + "Unable to send demuxed packet to consumers: %s\n", av_err2str(ret)); break; } } + // EOF/EXIT is normal termination + if (ret == AVERROR_EOF || ret == AVERROR_EXIT) + ret = 0; + finish: - av_assert0(ret < 0); - tq_send_finish(d->thread_queue, 0); + sch_demux_send(d->sch, f->index, NULL); av_packet_free(&pkt); av_log(d, AV_LOG_VERBOSE, "Terminating demuxer thread\n"); - return NULL; + return (void*)(intptr_t)ret; } +// XXX +#if 0 static void thread_stop(Demuxer *d) { InputFile *f = &d->f; - if (!d->thread_queue) - return; - - tq_receive_finish(d->thread_queue, 0); - - pthread_join(d->thread, NULL); - - tq_free(&d->thread_queue); - //av_thread_message_queue_free(&f->audio_duration_queue); } @@ -671,22 +692,7 @@ static int thread_start(Demuxer *d) { int ret; InputFile *f = &d->f; - ObjPool *op; - if (d->thread_queue_size <= 0) - d->thread_queue_size = (nb_input_files > 1 ? 8 : 1); - - op = objpool_alloc_packets(); - if (!op) - return AVERROR(ENOMEM); - - d->thread_queue = tq_alloc(1, d->thread_queue_size, op, pkt_move); - if (!d->thread_queue) { - objpool_free(&op); - return AVERROR(ENOMEM); - } - -#if 0 if (d->loop) { int nb_audio_dec = 0; @@ -704,20 +710,6 @@ static int thread_start(Demuxer *d) f->audio_duration_queue_size = nb_audio_dec; } } -#endif - - if ((ret = pthread_create(&d->thread, NULL, input_thread, d))) { - av_log(d, AV_LOG_ERROR, "pthread_create failed: %s. Try to increase `ulimit -v` or decrease `ulimit -s`.\n", strerror(ret)); - ret = AVERROR(ret); - goto fail; - } - - d->read_started = 1; - - return 0; -fail: - tq_free(&d->thread_queue); - return ret; } int ifile_get_packet(InputFile *f, AVPacket *pkt) @@ -725,12 +717,6 @@ int ifile_get_packet(InputFile *f, AVPacket *pkt) Demuxer *d = demuxer_from_ifile(f); int ret, dummy; - if (!d->thread_queue) { - ret = thread_start(d); - if (ret < 0) - return ret; - } - ret = tq_receive(d->thread_queue, &dummy, pkt); if (ret < 0) return ret; @@ -744,6 +730,7 @@ int ifile_get_packet(InputFile *f, AVPacket *pkt) return 0; } +#endif static void demux_final_stats(Demuxer *d) { @@ -813,8 +800,6 @@ void ifile_close(InputFile **pf) if (!f) return; - thread_stop(d); - if (d->read_started) demux_final_stats(d); @@ -846,7 +831,11 @@ static int ist_use(InputStream *ist, int decoding_needed) ds->sch_idx_stream = ret; } - ist->discard = 0; + if (ist->discard) { + ist->discard = 0; + d->nb_streams_used++; + } + ist->st->discard = ist->user_set_discard; ist->decoding_needed |= decoding_needed; ds->streamcopy_needed |= !decoding_needed; @@ -1647,8 +1636,6 @@ int ifile_open(const OptionsContext *o, const char *filename, Scheduler *sch) "since neither -readrate nor -re were given\n"); } - d->thread_queue_size = o->thread_queue_size; - /* Add all the streams from the given input file to the demuxer */ for (int i = 0; i < ic->nb_streams; i++) { ret = ist_add(o, d, ic->streams[i]); -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 23/27] WIP fftools/ffmpeg_dec: convert to the scheduler 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (21 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 22/27] WIP fftools/ffmpeg_demux: convert to the scheduler Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 24/27] WIP fftools/ffmpeg_filter: " Anton Khirnov ` (4 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel --- fftools/ffmpeg.c | 9 +-- fftools/ffmpeg.h | 11 --- fftools/ffmpeg_dec.c | 189 +++++++++---------------------------------- 3 files changed, 42 insertions(+), 167 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 00e57c4382..a09a9e1200 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -810,11 +810,6 @@ static int process_input_packet(InputStream *ist, const AVPacket *pkt, int no_eo int ret = 0; int eof_reached = 0; - if (ist->decoding_needed) { - ret = dec_packet(ist, pkt, no_eof); - if (ret < 0 && ret != AVERROR_EOF) - return ret; - } if (ret == AVERROR_EOF || (!pkt && !ist->decoding_needed)) eof_reached = 1; @@ -1036,6 +1031,7 @@ static void reset_eagain(void) ost->unavailable = 0; } +#if 0 static void decode_flush(InputFile *ifile) { for (int i = 0; i < ifile->nb_streams; i++) { @@ -1047,6 +1043,7 @@ static void decode_flush(InputFile *ifile) dec_packet(ist, NULL, 1); } } +#endif /* * Return @@ -1063,11 +1060,13 @@ static int process_input(int file_index, AVPacket *pkt) ret = 0; +#if 0 if (ret == 1) { /* the input file is looped: flush the decoders */ decode_flush(ifile); return AVERROR(EAGAIN); } +#endif if (ret < 0) { if (ret != AVERROR_EOF) { av_log(ifile, AV_LOG_ERROR, diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 4646c05bea..841f8d0d68 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -815,17 +815,6 @@ int hwaccel_retrieve_data(AVCodecContext *avctx, AVFrame *input); int dec_open(InputStream *ist, Scheduler *sch, unsigned sch_idx); void dec_free(Decoder **pdec); -/** - * Submit a packet for decoding - * - * When pkt==NULL and no_eof=0, there will be no more input. Flush decoders and - * mark all downstreams as finished. - * - * When pkt==NULL and no_eof=1, the stream was reset (e.g. after a seek). Flush - * decoders and await further input. - */ -int dec_packet(InputStream *ist, const AVPacket *pkt, int no_eof); - int enc_alloc(Encoder **penc, const AVCodec *codec, Scheduler *sch, unsigned sch_idx); void enc_free(Encoder **penc); diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c index dc8d0374a3..400fa666b9 100644 --- a/fftools/ffmpeg_dec.c +++ b/fftools/ffmpeg_dec.c @@ -53,24 +53,6 @@ struct Decoder { Scheduler *sch; unsigned sch_idx; - - pthread_t thread; - /** - * Queue for sending coded packets from the main thread to - * the decoder thread. - * - * An empty packet is sent to flush the decoder without terminating - * decoding. - */ - ThreadQueue *queue_in; - /** - * Queue for sending decoded frames from the decoder thread - * to the main thread. - * - * An empty frame is sent to signal that a single packet has been fully - * processed. - */ - ThreadQueue *queue_out; }; // data that is local to the decoder thread and not visible outside of it @@ -79,24 +61,6 @@ typedef struct DecThreadContext { AVPacket *pkt; } DecThreadContext; -static int dec_thread_stop(Decoder *d) -{ - void *ret; - - if (!d->queue_in) - return 0; - - tq_send_finish(d->queue_in, 0); - tq_receive_finish(d->queue_out, 0); - - pthread_join(d->thread, &ret); - - tq_free(&d->queue_in); - tq_free(&d->queue_out); - - return (intptr_t)ret; -} - void dec_free(Decoder **pdec) { Decoder *dec = *pdec; @@ -104,8 +68,6 @@ void dec_free(Decoder **pdec) if (!dec) return; - dec_thread_stop(dec); - av_frame_free(&dec->frame); av_packet_free(&dec->pkt); @@ -147,25 +109,6 @@ fail: return AVERROR(ENOMEM); } -static int send_frame_to_filters(InputStream *ist, AVFrame *decoded_frame) -{ - int i, ret = 0; - - for (i = 0; i < ist->nb_filters; i++) { - ret = ifilter_send_frame(ist->filters[i], decoded_frame, - i < ist->nb_filters - 1 || - ist->dec->type == AVMEDIA_TYPE_SUBTITLE); - if (ret == AVERROR_EOF) - ret = 0; /* ignore */ - if (ret < 0) { - av_log(NULL, AV_LOG_ERROR, - "Failed to inject frame into filter network: %s\n", av_err2str(ret)); - break; - } - } - return ret; -} - static AVRational audio_samplerate_update(void *logctx, Decoder *d, const AVFrame *frame) { @@ -420,7 +363,8 @@ static int process_subtitle(InputStream *ist, AVFrame *frame) if (!subtitle) return 0; - ret = send_frame_to_filters(ist, frame); + // XXX + //ret = send_frame_to_filters(ist, frame); if (ret < 0) return ret; @@ -495,7 +439,7 @@ static int transcode_subtitles(InputStream *ist, const AVPacket *pkt, ist->frames_decoded++; - // XXX the queue for transferring data back to the main thread runs + // XXX the queue for transferring data to consumers runs // on AVFrames, so we wrap AVSubtitle in an AVBufferRef and put that // inside the frame // eventually, subtitles should be switched to use AVFrames natively @@ -508,26 +452,11 @@ static int transcode_subtitles(InputStream *ist, const AVPacket *pkt, frame->width = ist->dec_ctx->width; frame->height = ist->dec_ctx->height; - ret = tq_send(d->queue_out, 0, frame); + ret = sch_dec_send(d->sch, d->sch_idx, frame); if (ret < 0) av_frame_unref(frame); - return ret; -} - -static int send_filter_eof(InputStream *ist) -{ - Decoder *d = ist->decoder; - int i, ret; - - for (i = 0; i < ist->nb_filters; i++) { - int64_t end_pts = d->last_frame_pts == AV_NOPTS_VALUE ? AV_NOPTS_VALUE : - d->last_frame_pts + d->last_frame_duration_est; - ret = ifilter_send_eof(ist->filters[i], end_pts, d->last_frame_tb); - if (ret < 0) - return ret; - } - return 0; + return ret == AVERROR_EOF ? AVERROR_EXIT : ret; } static int packet_decode(InputStream *ist, const AVPacket *pkt, AVFrame *frame) @@ -629,9 +558,9 @@ static int packet_decode(InputStream *ist, const AVPacket *pkt, AVFrame *frame) ist->frames_decoded++; - ret = tq_send(d->queue_out, 0, frame); + ret = sch_dec_send(d->sch, d->sch_idx, frame); if (ret < 0) - return ret; + return ret == AVERROR_EOF ? AVERROR_EXIT : ret; } } @@ -685,9 +614,9 @@ void *decoder_thread(void *arg) dec_thread_set_name(ist); while (!input_status) { - int dummy, flush_buffers; + int flush_buffers; - input_status = tq_receive(d->queue_in, &dummy, dt.pkt); + input_status = sch_dec_receive(d->sch, d->sch_idx, dt.pkt); flush_buffers = input_status >= 0 && !dt.pkt->buf; if (!dt.pkt->buf) av_log(ist, AV_LOG_VERBOSE, "Decoder thread received %s packet\n", @@ -698,6 +627,14 @@ void *decoder_thread(void *arg) av_packet_unref(dt.pkt); av_frame_unref(dt.frame); + // AVERROR_EOF - EOF from the decoder + // AVERROR_EXIT - EOF from the scheduler + // we treat them differently when flushing + if (ret == AVERROR_EXIT) { + ret = AVERROR_EOF; + flush_buffers = 0; + } + if (ret == AVERROR_EOF) { av_log(ist, AV_LOG_VERBOSE, "Decoder returned EOF, %s\n", flush_buffers ? "resetting" : "finishing"); @@ -725,23 +662,32 @@ void *decoder_thread(void *arg) av_err2str(ret)); break; } - - // signal to the consumer thread that the entire packet was processed - ret = tq_send(d->queue_out, 0, dt.frame); - if (ret < 0) { - if (ret != AVERROR_EOF) - av_log(ist, AV_LOG_ERROR, "Error communicating with the main thread\n"); - break; - } } // EOF is normal thread termination if (ret == AVERROR_EOF) ret = 0; + // on success send EOF timestamp to our downstreams + if (ret >= 0) { + av_frame_unref(dt.frame); + + dt.frame->pts = d->last_frame_pts == AV_NOPTS_VALUE ? AV_NOPTS_VALUE : + d->last_frame_pts + d->last_frame_duration_est; + dt.frame->time_base = d->last_frame_tb; + + // XXX check EOF/EXIT handling + ret = sch_dec_send(d->sch, d->sch_idx, dt.frame); + if (ret < 0 && ret != AVERROR_EOF) { + av_log(NULL, AV_LOG_FATAL, + "Error signalling EOF timestamp: %s\n", av_err2str(ret)); + goto finish; + } + ret = 0; + } + finish: - tq_receive_finish(d->queue_in, 0); - tq_send_finish (d->queue_out, 0); + sch_dec_send(d->sch, d->sch_idx, NULL); #if 0 // make sure the demuxer does not get stuck waiting for audio durations @@ -757,15 +703,12 @@ finish: return (void*)(intptr_t)ret; } +#if 0 int dec_packet(InputStream *ist, const AVPacket *pkt, int no_eof) { Decoder *d = ist->decoder; int ret = 0, thread_ret; - // thread already joined - if (!d->queue_in) - return AVERROR_EOF; - // send the packet/flush request/EOF to the decoder thread if (pkt || no_eof) { av_packet_unref(d->pkt); @@ -816,59 +759,10 @@ finish: if (ret < 0 && ret != AVERROR_EOF) return ret; - // signal EOF to our downstreams - ret = send_filter_eof(ist); - if (ret < 0) { - av_log(NULL, AV_LOG_FATAL, "Error marking filters as finished\n"); - return ret; - } return AVERROR_EOF; } - -static int dec_thread_start(InputStream *ist) -{ - Decoder *d = ist->decoder; - ObjPool *op; - int ret = 0; - - op = objpool_alloc_packets(); - if (!op) - return AVERROR(ENOMEM); - - d->queue_in = tq_alloc(1, 1, op, pkt_move); - if (!d->queue_in) { - objpool_free(&op); - return AVERROR(ENOMEM); - } - - op = objpool_alloc_frames(); - if (!op) - goto fail; - - d->queue_out = tq_alloc(1, 4, op, frame_move); - if (!d->queue_out) { - objpool_free(&op); - goto fail; - } - - ret = pthread_create(&d->thread, NULL, decoder_thread, ist); - if (ret) { - ret = AVERROR(ret); - av_log(ist, AV_LOG_ERROR, "pthread_create() failed: %s\n", - av_err2str(ret)); - goto fail; - } - - return 0; -fail: - if (ret >= 0) - ret = AVERROR(ENOMEM); - - tq_free(&d->queue_in); - tq_free(&d->queue_out); - return ret; -} +#endif static enum AVPixelFormat get_format(AVCodecContext *s, const enum AVPixelFormat *pix_fmts) { @@ -1121,12 +1015,5 @@ int dec_open(InputStream *ist, Scheduler *sch, unsigned sch_idx) if (ret < 0) return ret; - ret = dec_thread_start(ist); - if (ret < 0) { - av_log(ist, AV_LOG_ERROR, "Error starting decoder thread: %s\n", - av_err2str(ret)); - return ret; - } - return 0; } -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 24/27] WIP fftools/ffmpeg_filter: convert to the scheduler 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (22 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 23/27] WIP fftools/ffmpeg_dec: " Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 25/27] WIP fftools/ffmpeg_enc: " Anton Khirnov ` (3 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel --- fftools/ffmpeg.h | 22 -- fftools/ffmpeg_filter.c | 533 ++++++++++++---------------------------- 2 files changed, 152 insertions(+), 403 deletions(-) diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 841f8d0d68..eb4e8e27a9 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -312,9 +312,6 @@ typedef struct OutputFilter { enum AVMediaType type; - /* pts of the last frame received from this filter, in AV_TIME_BASE_Q */ - int64_t last_pts; - uint64_t nb_frames_dup; uint64_t nb_frames_drop; } OutputFilter; @@ -745,8 +742,6 @@ int subtitle_wrap_frame(AVFrame *frame, AVSubtitle *subtitle, int copy); */ FrameData *frame_data(AVFrame *frame); -int ifilter_send_frame(InputFilter *ifilter, AVFrame *frame, int keep_reference); -int ifilter_send_eof(InputFilter *ifilter, int64_t pts, AVRational tb); void ifilter_sub2video_heartbeat(InputFilter *ifilter, int64_t pts, AVRational tb); /** @@ -769,26 +764,9 @@ int fg_create(FilterGraph **pfg, char *graph_desc, Scheduler *sch); void fg_free(FilterGraph **pfg); -/** - * Perform a step of transcoding for the specified filter graph. - * - * @param[in] graph filter graph to consider - * @param[out] best_ist input stream where a frame would allow to continue - * @return 0 for success, <0 for error - */ -int fg_transcode_step(FilterGraph *graph, InputStream **best_ist); - void fg_send_command(FilterGraph *fg, double time, const char *target, const char *command, const char *arg, int all_filters); -/** - * Get and encode new output from specified filtergraph, without causing - * activity. - * - * @return 0 for success, <0 for severe errors - */ -int reap_filters(FilterGraph *fg, int flush); - int ffmpeg_parse_options(int argc, char **argv, Scheduler *sch); void enc_stats_write(OutputStream *ost, EncStats *es, diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index e8e78f5454..c588dde452 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -21,7 +21,6 @@ #include <stdint.h> #include "ffmpeg.h" -#include "thread_queue.h" #include "libavfilter/avfilter.h" #include "libavfilter/buffersink.h" @@ -46,7 +45,6 @@ enum FrameOpaque { FRAME_OPAQUE_REAP_FILTERS = 1, FRAME_OPAQUE_CHOOSE_INPUT, FRAME_OPAQUE_SUB_HEARTBEAT, - FRAME_OPAQUE_EOF, FRAME_OPAQUE_SEND_COMMAND, }; @@ -62,8 +60,7 @@ typedef struct FilterGraphPriv { int is_meta; int disable_conversions; - int nb_inputs_bound; - int nb_outputs_bound; + unsigned nb_outputs_done; const char *graph_desc; @@ -75,14 +72,10 @@ typedef struct FilterGraphPriv { Scheduler *sch; unsigned sch_idx; - pthread_t thread; /** * Queue for sending frames from the main thread to the filtergraph. Has * nb_inputs+1 streams - the first nb_inputs stream correspond to * filtergraph inputs. Frames on those streams may have their opaque set to - * - FRAME_OPAQUE_EOF: frame contains no data, but pts+timebase of the - * EOF event for the correspondint stream. Will be immediately followed by - * this stream being send-closed. * - FRAME_OPAQUE_SUB_HEARTBEAT: frame contains no data, but pts+timebase of * a subtitle heartbeat event. Will only be sent for sub2video streams. * @@ -96,7 +89,7 @@ typedef struct FilterGraphPriv { * available the terminating empty frame's opaque will contain the index+1 * of the filtergraph input to which more input frames should be supplied. */ - ThreadQueue *queue_in; + /** * Queue for sending frames from the filtergraph back to the main thread. * Has nb_outputs+1 streams - the first nb_outputs stream correspond to @@ -105,7 +98,7 @@ typedef struct FilterGraphPriv { * The last stream is "control" - see documentation for queue_in for more * details. */ - ThreadQueue *queue_out; + //ThreadQueue *queue_out; // submitting frames to filter thread returned EOF // this only happens on thread exit, so is not per-input int eof_in; @@ -130,6 +123,7 @@ typedef struct FilterGraphThread { // The output index is stored in frame opaque. AVFifo *frame_queue_out; + // set to 1 after at least one frame passed through this output int got_frame; // EOF status of each input/output, as received by the thread @@ -260,9 +254,6 @@ typedef struct OutputFilterPriv { int64_t ts_offset; int64_t next_pts; FPSConvContext fps; - - // set to 1 after at least one frame passed through this output - int got_frame; } OutputFilterPriv; static OutputFilterPriv *ofp_from_ofilter(OutputFilter *ofilter) @@ -660,57 +651,6 @@ static int ifilter_has_all_input_formats(FilterGraph *fg) static void *filter_thread(void *arg); -// start the filtering thread once all inputs and outputs are bound -static int fg_thread_try_start(FilterGraphPriv *fgp) -{ - FilterGraph *fg = &fgp->fg; - ObjPool *op; - int ret = 0; - - if (fgp->nb_inputs_bound < fg->nb_inputs || - fgp->nb_outputs_bound < fg->nb_outputs) - return 0; - - op = objpool_alloc_frames(); - if (!op) - return AVERROR(ENOMEM); - - fgp->queue_in = tq_alloc(fg->nb_inputs + 1, 1, op, frame_move); - if (!fgp->queue_in) { - objpool_free(&op); - return AVERROR(ENOMEM); - } - - // at least one output is mandatory - op = objpool_alloc_frames(); - if (!op) - goto fail; - - fgp->queue_out = tq_alloc(fg->nb_outputs + 1, 1, op, frame_move); - if (!fgp->queue_out) { - objpool_free(&op); - goto fail; - } - - ret = pthread_create(&fgp->thread, NULL, filter_thread, fgp); - if (ret) { - ret = AVERROR(ret); - av_log(NULL, AV_LOG_ERROR, "pthread_create() for filtergraph %d failed: %s\n", - fg->index, av_err2str(ret)); - goto fail; - } - - return 0; -fail: - if (ret >= 0) - ret = AVERROR(ENOMEM); - - tq_free(&fgp->queue_in); - tq_free(&fgp->queue_out); - - return ret; -} - static char *describe_filter_link(FilterGraph *fg, AVFilterInOut *inout, int in) { AVFilterContext *ctx = inout->filter_ctx; @@ -736,7 +676,6 @@ static OutputFilter *ofilter_alloc(FilterGraph *fg) ofilter->graph = fg; ofp->format = -1; ofp->index = fg->nb_outputs - 1; - ofilter->last_pts = AV_NOPTS_VALUE; return ofilter; } @@ -768,10 +707,7 @@ static int ifilter_bind_ist(InputFilter *ifilter, InputStream *ist) return AVERROR(ENOMEM); } - fgp->nb_inputs_bound++; - av_assert0(fgp->nb_inputs_bound <= ifilter->graph->nb_inputs); - - return fg_thread_try_start(fgp); + return 0; } static int set_channel_layout(OutputFilterPriv *f, OutputStream *ost) @@ -910,10 +846,7 @@ int ofilter_bind_ost(OutputFilter *ofilter, OutputStream *ost, if (ret < 0) return ret; - fgp->nb_outputs_bound++; - av_assert0(fgp->nb_outputs_bound <= fg->nb_outputs); - - return fg_thread_try_start(fgp); + return 0; } static InputFilter *ifilter_alloc(FilterGraph *fg) @@ -943,34 +876,6 @@ static InputFilter *ifilter_alloc(FilterGraph *fg) return ifilter; } -static int fg_thread_stop(FilterGraphPriv *fgp) -{ - void *ret; - - if (!fgp->queue_in) - return 0; - - for (int i = 0; i <= fgp->fg.nb_inputs; i++) { - InputFilterPriv *ifp = i < fgp->fg.nb_inputs ? - ifp_from_ifilter(fgp->fg.inputs[i]) : NULL; - - if (ifp) - ifp->eof = 1; - - tq_send_finish(fgp->queue_in, i); - } - - for (int i = 0; i <= fgp->fg.nb_outputs; i++) - tq_receive_finish(fgp->queue_out, i); - - pthread_join(fgp->thread, &ret); - - tq_free(&fgp->queue_in); - tq_free(&fgp->queue_out); - - return (int)(intptr_t)ret; -} - void fg_free(FilterGraph **pfg) { FilterGraph *fg = *pfg; @@ -980,8 +885,6 @@ void fg_free(FilterGraph **pfg) return; fgp = fgp_from_fg(fg); - fg_thread_stop(fgp); - avfilter_graph_free(&fg->graph); for (int j = 0; j < fg->nb_inputs; j++) { InputFilter *ifilter = fg->inputs[j]; @@ -2253,8 +2156,56 @@ finish: fps->dropped_keyframe |= fps->last_dropped && (frame->flags & AV_FRAME_FLAG_KEY); } +static int close_output(OutputFilterPriv *ofp, FilterGraphThread *fgt) +{ + FilterGraphPriv *fgp = fgp_from_fg(ofp->ofilter.graph); + int ret; + + // we are finished and no frames were ever seen at this output, + // at least initialize the encoder with a dummy frame + if (!fgt->got_frame) { + AVFrame *frame = fgt->frame; + FrameData *fd; + + frame->time_base = ofp->tb_out; + frame->format = ofp->format; + + frame->width = ofp->width; + frame->height = ofp->height; + frame->sample_aspect_ratio = ofp->sample_aspect_ratio; + + frame->sample_rate = ofp->sample_rate; + if (ofp->ch_layout.nb_channels) { + ret = av_channel_layout_copy(&frame->ch_layout, &ofp->ch_layout); + if (ret < 0) + return ret; + } + + fd = frame_data(frame); + if (!fd) + return AVERROR(ENOMEM); + + fd->frame_rate_filter = ofp->fps.framerate; + + av_assert0(!frame->buf[0]); + + av_log(ofp->ofilter.ost, AV_LOG_WARNING, + "No filtered frames for output stream, trying to " + "initialize anyway.\n"); + + ret = sch_filter_send(fgp->sch, fgp->sch_idx, ofp->index, frame); + av_frame_unref(frame); + if (ret < 0) + return ret; + } + + fgt->eof_out[ofp->index] = 1; + + return sch_filter_send(fgp->sch, fgp->sch_idx, ofp->index, NULL); +} + static int fg_output_frame(OutputFilterPriv *ofp, FilterGraphThread *fgt, - AVFrame *frame, int buffer) + AVFrame *frame) { FilterGraphPriv *fgp = fgp_from_fg(ofp->ofilter.graph); AVFrame *frame_prev = ofp->fps.last_frame; @@ -2301,28 +2252,17 @@ static int fg_output_frame(OutputFilterPriv *ofp, FilterGraphThread *fgt, frame_out = frame; } - if (buffer) { - AVFrame *f = av_frame_alloc(); - - if (!f) { - av_frame_unref(frame_out); - return AVERROR(ENOMEM); - } - - av_frame_move_ref(f, frame_out); - f->opaque = (void*)(intptr_t)ofp->index; - - ret = av_fifo_write(fgt->frame_queue_out, &f, 1); - if (ret < 0) { - av_frame_free(&f); - return AVERROR(ENOMEM); - } - } else { - // return the frame to the main thread - ret = tq_send(fgp->queue_out, ofp->index, frame_out); + { + // send the frame to consumers + ret = sch_filter_send(fgp->sch, fgp->sch_idx, ofp->index, frame_out); if (ret < 0) { av_frame_unref(frame_out); - fgt->eof_out[ofp->index] = 1; + + if (!fgt->eof_out[ofp->index]) { + fgt->eof_out[ofp->index] = 1; + fgp->nb_outputs_done++; + } + return ret == AVERROR_EOF ? 0 : ret; } } @@ -2343,16 +2283,14 @@ static int fg_output_frame(OutputFilterPriv *ofp, FilterGraphThread *fgt, av_frame_move_ref(frame_prev, frame); } - if (!frame) { - tq_send_finish(fgp->queue_out, ofp->index); - fgt->eof_out[ofp->index] = 1; - } + if (!frame) + return close_output(ofp, fgt); return 0; } static int fg_output_step(OutputFilterPriv *ofp, FilterGraphThread *fgt, - AVFrame *frame, int buffer) + AVFrame *frame) { FilterGraphPriv *fgp = fgp_from_fg(ofp->ofilter.graph); OutputStream *ost = ofp->ofilter.ost; @@ -2362,8 +2300,8 @@ static int fg_output_step(OutputFilterPriv *ofp, FilterGraphThread *fgt, ret = av_buffersink_get_frame_flags(filter, frame, AV_BUFFERSINK_FLAG_NO_REQUEST); - if (ret == AVERROR_EOF && !buffer && !fgt->eof_out[ofp->index]) { - ret = fg_output_frame(ofp, fgt, NULL, buffer); + if (ret == AVERROR_EOF && !fgt->eof_out[ofp->index]) { + ret = fg_output_frame(ofp, fgt, NULL); return (ret < 0) ? ret : 1; } else if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) { return 1; @@ -2417,7 +2355,7 @@ static int fg_output_step(OutputFilterPriv *ofp, FilterGraphThread *fgt, fd->frame_rate_filter = ofp->fps.framerate; } - ret = fg_output_frame(ofp, fgt, frame, buffer); + ret = fg_output_frame(ofp, fgt, frame); av_frame_unref(frame); if (ret < 0) return ret; @@ -2425,44 +2363,29 @@ static int fg_output_step(OutputFilterPriv *ofp, FilterGraphThread *fgt, return 0; } -/* retrieve all frames available at filtergraph outputs and either send them to - * the main thread (buffer=0) or buffer them for later (buffer=1) */ +/* retrieve all frames available at filtergraph outputs + * and send them to consumers */ static int read_frames(FilterGraph *fg, FilterGraphThread *fgt, - AVFrame *frame, int buffer) + AVFrame *frame) { FilterGraphPriv *fgp = fgp_from_fg(fg); - int ret = 0; if (!fg->graph) return 0; - // process buffered frames - if (!buffer) { - AVFrame *f; - - while (av_fifo_read(fgt->frame_queue_out, &f, 1) >= 0) { - int out_idx = (intptr_t)f->opaque; - f->opaque = NULL; - ret = tq_send(fgp->queue_out, out_idx, f); - av_frame_free(&f); - if (ret < 0 && ret != AVERROR_EOF) - return ret; - } - } - /* Reap all buffers present in the buffer sinks */ for (int i = 0; i < fg->nb_outputs; i++) { OutputFilterPriv *ofp = ofp_from_ofilter(fg->outputs[i]); int ret = 0; while (!ret) { - ret = fg_output_step(ofp, fgt, frame, buffer); + ret = fg_output_step(ofp, fgt, frame); if (ret < 0) return ret; } } - return 0; + return (fgp->nb_outputs_done == fg->nb_outputs) ? AVERROR_EOF : 0; } static void sub2video_heartbeat(InputFilter *ifilter, int64_t pts, AVRational tb) @@ -2536,6 +2459,9 @@ static int send_eof(FilterGraphThread *fgt, InputFilter *ifilter, InputFilterPriv *ifp = ifp_from_ifilter(ifilter); int ret; + if (fgt->eof_in[ifp->index]) + return 0; + fgt->eof_in[ifp->index] = 1; if (ifp->filter) { @@ -2637,7 +2563,7 @@ static int send_frame(FilterGraph *fg, FilterGraphThread *fgt, return ret; } - ret = read_frames(fg, fgt, tmp, 1); + ret = read_frames(fg, fgt, tmp); av_frame_free(&tmp); if (ret < 0) return ret; @@ -2674,6 +2600,29 @@ static int choose_input(const FilterGraph *fg, const FilterGraphThread *fgt) { int nb_requests, nb_requests_max = 0; int best_input = -1; + int ret; + + if (!fg->graph) { + for (int i = 0; i < fg->nb_inputs; i++) { + InputFilterPriv *ifp = ifp_from_ifilter(fg->inputs[i]); + if (ifp->format < 0 && !fgt->eof_in[i]) + return i; + } + + // This state - graph is not configured, but all inputs are either + // initialized or EOF - should be unreachable because sending EOF to a + // filter without even a fallback format should fail + av_assert0(0); + return AVERROR_BUG; + } + + ret = avfilter_graph_request_oldest(fg->graph); + if (ret < 0 && ret != AVERROR(EAGAIN)) + return ret; + + // we drain all frames from the filtergraph after each input frame, so + // we should never get success here + av_assert0(ret < 0); for (int i = 0; i < fg->nb_inputs; i++) { InputFilter *ifilter = fg->inputs[i]; @@ -2690,9 +2639,12 @@ static int choose_input(const FilterGraph *fg, const FilterGraphThread *fgt) } } + av_assert0(best_input >= 0); + return best_input; } +#if 0 static int msg_process(FilterGraphPriv *fgp, FilterGraphThread *fgt, AVFrame *frame) { @@ -2766,6 +2718,7 @@ done: return 0; } +#endif static void fg_thread_set_name(const FilterGraph *fg) { @@ -2852,28 +2805,53 @@ static void *filter_thread(void *arg) while (1) { InputFilter *ifilter; InputFilterPriv *ifp; - int input_idx, eof_frame; + int input_idx; - input_status = tq_receive(fgp->queue_in, &input_idx, fgt.frame); - if (input_idx < 0 || - (input_idx == fg->nb_inputs && input_status < 0)) { + // XXX + if (!fg->nb_inputs) { + abort(); + goto no_inputs; + } + + input_idx = choose_input(fg, &fgt); + // XXX + if (input_idx < 0) + goto finish; + + input_status = sch_filter_receive(fgp->sch, fgp->sch_idx, + &input_idx, fgt.frame); + if (input_idx < 0) { av_log(fg, AV_LOG_VERBOSE, "Filtering thread received EOF\n"); break; } + // XXX +#if 0 // message on the control stream if (input_idx == fg->nb_inputs) { ret = msg_process(fgp, &fgt, fgt.frame); if (ret < 0) goto finish; +#elif 0 + enum FrameOpaque msg = (intptr_t)fgt.frame->opaque; + + fgt.frame->opaque = NULL; + av_assert0(msg > 0); + av_assert0(msg == FRAME_OPAQUE_SEND_COMMAND); + + { + FilterCommand *fc = (FilterCommand*)frame->buf[0]->data; + send_command(fg, fc->time, fc->target, fc->command, fc->arg, fc->all_filters); + av_frame_unref(frame); + } + continue; } +#endif - // we received an input frame or EOF ifilter = fg->inputs[input_idx]; ifp = ifp_from_ifilter(ifilter); - eof_frame = input_status >= 0 && (intptr_t)fgt.frame->opaque == FRAME_OPAQUE_EOF; if (ifp->type_src == AVMEDIA_TYPE_SUBTITLE) { if (input_status >= 0 && (intptr_t)fgt.frame->opaque == FRAME_OPAQUE_SUB_HEARTBEAT) sub2video_heartbeat(ifilter, fgt.frame->pts, fgt.frame->time_base); @@ -2890,20 +2868,17 @@ static void *filter_thread(void *arg) if (ret < 0) break; - if (eof_frame) { - // an EOF frame is immediately followed by sender closing - // the corresponding stream, so retrieve that event - input_status = tq_receive(fgp->queue_in, &input_idx, fgt.frame); - av_assert0(input_status == AVERROR_EOF && input_idx == ifp->index); - } - - // signal to the main thread that we are done - ret = tq_send(fgp->queue_out, fg->nb_outputs, fgt.frame); +no_inputs: + // retrieve all newly avalable frames + // XXX: handle filtergraphs connected to an unlimited source + ret = read_frames(fg, &fgt, fgt.frame); if (ret < 0) { if (ret == AVERROR_EOF) - break; + av_log(fg, AV_LOG_VERBOSE, "All consumers returned EOF\n"); + else + av_log(fg, AV_LOG_ERROR, "Error sending frames to consumers: %s\n", + av_err2str(ret)); - av_log(fg, AV_LOG_ERROR, "Error communicating with the main thread\n"); goto finish; } } @@ -2913,10 +2888,9 @@ finish: if (ret == AVERROR_EOF) ret = 0; - for (int i = 0; i <= fg->nb_inputs; i++) - tq_receive_finish(fgp->queue_in, i); - for (int i = 0; i <= fg->nb_outputs; i++) - tq_send_finish(fgp->queue_out, i); + // XXX close_output() here? + + sch_filter_send(fgp->sch, fgp->sch_idx, -1, NULL); fg_thread_uninit(&fgt); @@ -2925,64 +2899,6 @@ finish: return (void*)(intptr_t)ret; } -static int thread_send_frame(FilterGraphPriv *fgp, InputFilter *ifilter, - AVFrame *frame, enum FrameOpaque type) -{ - InputFilterPriv *ifp = ifp_from_ifilter(ifilter); - int output_idx, ret; - - if (ifp->eof) { - av_frame_unref(frame); - return AVERROR_EOF; - } - - frame->opaque = (void*)(intptr_t)type; - - ret = tq_send(fgp->queue_in, ifp->index, frame); - if (ret < 0) { - ifp->eof = 1; - av_frame_unref(frame); - return ret; - } - - if (type == FRAME_OPAQUE_EOF) - tq_send_finish(fgp->queue_in, ifp->index); - - // wait for the frame to be processed - ret = tq_receive(fgp->queue_out, &output_idx, frame); - av_assert0(output_idx == fgp->fg.nb_outputs || ret == AVERROR_EOF); - - return ret; -} - -int ifilter_send_frame(InputFilter *ifilter, AVFrame *frame, int keep_reference) -{ - FilterGraphPriv *fgp = fgp_from_fg(ifilter->graph); - int ret; - - if (keep_reference) { - ret = av_frame_ref(fgp->frame, frame); - if (ret < 0) - return ret; - } else - av_frame_move_ref(fgp->frame, frame); - - return thread_send_frame(fgp, ifilter, fgp->frame, 0); -} - -int ifilter_send_eof(InputFilter *ifilter, int64_t pts, AVRational tb) -{ - FilterGraphPriv *fgp = fgp_from_fg(ifilter->graph); - int ret; - - fgp->frame->pts = pts; - fgp->frame->time_base = tb; - - ret = thread_send_frame(fgp, ifilter, fgp->frame, FRAME_OPAQUE_EOF); - - return ret == AVERROR_EOF ? 0 : ret; -} - void ifilter_sub2video_heartbeat(InputFilter *ifilter, int64_t pts, AVRational tb) { FilterGraphPriv *fgp = fgp_from_fg(ifilter->graph); @@ -2990,144 +2906,10 @@ void ifilter_sub2video_heartbeat(InputFilter *ifilter, int64_t pts, AVRational t fgp->frame->pts = pts; fgp->frame->time_base = tb; + // XXX +#if 0 thread_send_frame(fgp, ifilter, fgp->frame, FRAME_OPAQUE_SUB_HEARTBEAT); -} - -int fg_transcode_step(FilterGraph *graph, InputStream **best_ist) -{ - FilterGraphPriv *fgp = fgp_from_fg(graph); - int ret, got_frames = 0; - - if (fgp->eof_in) - return AVERROR_EOF; - - // signal to the filtering thread to return all frames it can - av_assert0(!fgp->frame->buf[0]); - fgp->frame->opaque = (void*)(intptr_t)(best_ist ? - FRAME_OPAQUE_CHOOSE_INPUT : - FRAME_OPAQUE_REAP_FILTERS); - - ret = tq_send(fgp->queue_in, graph->nb_inputs, fgp->frame); - if (ret < 0) { - fgp->eof_in = 1; - goto finish; - } - - while (1) { - OutputFilter *ofilter; - OutputFilterPriv *ofp; - OutputStream *ost; - int output_idx; - - ret = tq_receive(fgp->queue_out, &output_idx, fgp->frame); - - // EOF on the whole queue or the control stream - if (output_idx < 0 || - (ret < 0 && output_idx == graph->nb_outputs)) - goto finish; - - // EOF for a specific stream - if (ret < 0) { - ofilter = graph->outputs[output_idx]; - ofp = ofp_from_ofilter(ofilter); - - // we are finished and no frames were ever seen at this output, - // at least initialize the encoder with a dummy frame - if (!ofp->got_frame) { - AVFrame *frame = fgp->frame; - FrameData *fd; - - frame->time_base = ofp->tb_out; - frame->format = ofp->format; - - frame->width = ofp->width; - frame->height = ofp->height; - frame->sample_aspect_ratio = ofp->sample_aspect_ratio; - - frame->sample_rate = ofp->sample_rate; - if (ofp->ch_layout.nb_channels) { - ret = av_channel_layout_copy(&frame->ch_layout, &ofp->ch_layout); - if (ret < 0) - return ret; - } - - fd = frame_data(frame); - if (!fd) - return AVERROR(ENOMEM); - - fd->frame_rate_filter = ofp->fps.framerate; - - av_assert0(!frame->buf[0]); - - av_log(ofilter->ost, AV_LOG_WARNING, - "No filtered frames for output stream, trying to " - "initialize anyway.\n"); - - enc_open(ofilter->ost, frame); - av_frame_unref(frame); - } - - close_output_stream(graph->outputs[output_idx]->ost); - continue; - } - - // request was fully processed by the filtering thread, - // return the input stream to read from, if needed - if (output_idx == graph->nb_outputs) { - int input_idx = (intptr_t)fgp->frame->opaque - 1; - av_assert0(input_idx <= graph->nb_inputs); - - if (best_ist) { - *best_ist = (input_idx >= 0 && input_idx < graph->nb_inputs) ? - ifp_from_ifilter(graph->inputs[input_idx])->ist : NULL; - - if (input_idx < 0 && !got_frames) { - for (int i = 0; i < graph->nb_outputs; i++) - graph->outputs[i]->ost->unavailable = 1; - } - } - break; - } - - // got a frame from the filtering thread, send it for encoding - ofilter = graph->outputs[output_idx]; - ost = ofilter->ost; - ofp = ofp_from_ofilter(ofilter); - - if (ost->finished) { - av_frame_unref(fgp->frame); - tq_receive_finish(fgp->queue_out, output_idx); - continue; - } - - if (fgp->frame->pts != AV_NOPTS_VALUE) { - ofilter->last_pts = av_rescale_q(fgp->frame->pts, - fgp->frame->time_base, - AV_TIME_BASE_Q); - } - - ret = enc_frame(ost, fgp->frame); - av_frame_unref(fgp->frame); - if (ret < 0) - goto finish; - - ofp->got_frame = 1; - got_frames = 1; - } - -finish: - if (ret < 0) { - fgp->eof_in = 1; - for (int i = 0; i < graph->nb_outputs; i++) - close_output_stream(graph->outputs[i]->ost); - } - - return ret; -} - -int reap_filters(FilterGraph *fg, int flush) -{ - return fg_transcode_step(fg, NULL); +#endif } void fg_send_command(FilterGraph *fg, double time, const char *target, @@ -3138,9 +2920,6 @@ void fg_send_command(FilterGraph *fg, double time, const char *target, FilterCommand *fc; int output_idx, ret; - if (!fgp->queue_in) - return; - fc = av_mallocz(sizeof(*fc)); if (!fc) return; @@ -3165,13 +2944,5 @@ void fg_send_command(FilterGraph *fg, double time, const char *target, fgp->frame->buf[0] = buf; fgp->frame->opaque = (void*)(intptr_t)FRAME_OPAQUE_SEND_COMMAND; - ret = tq_send(fgp->queue_in, fg->nb_inputs + 1, fgp->frame); - if (ret < 0) { - av_frame_unref(fgp->frame); - return; - } - - // wait for the frame to be processed - ret = tq_receive(fgp->queue_out, &output_idx, fgp->frame); - av_assert0(output_idx == fgp->fg.nb_outputs || ret == AVERROR_EOF); + // XXX actually implement the command } -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 25/27] WIP fftools/ffmpeg_enc: convert to the scheduler 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (23 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 24/27] WIP fftools/ffmpeg_filter: " Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 26/27] WIP fftools/ffmpeg_mux: " Anton Khirnov ` (2 subsequent siblings) 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel --- fftools/ffmpeg.c | 1 - fftools/ffmpeg.h | 4 +- fftools/ffmpeg_dec.c | 1 + fftools/ffmpeg_enc.c | 231 ++++++-------------------------------- fftools/ffmpeg_mux_init.c | 43 ++----- 5 files changed, 45 insertions(+), 235 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index a09a9e1200..69e73b84ed 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -1228,7 +1228,6 @@ static int transcode(Scheduler *sch, int *err_rate_exceeded) } else if (err_rate) av_log(ist, AV_LOG_VERBOSE, "Decode error rate %g\n", err_rate); } - ret = err_merge(ret, enc_flush()); term_exit(); diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index eb4e8e27a9..66b5e398bc 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -797,10 +797,8 @@ int enc_alloc(Encoder **penc, const AVCodec *codec, Scheduler *sch, unsigned sch_idx); void enc_free(Encoder **penc); -int enc_open(OutputStream *ost, const AVFrame *frame); +int enc_open(void *opaque, const AVFrame *frame); int enc_subtitle(OutputFile *of, OutputStream *ost, const AVSubtitle *sub); -int enc_frame(OutputStream *ost, AVFrame *frame); -int enc_flush(void); /* * Initialize muxing state for the given stream, should be called diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c index 400fa666b9..d2dc27303d 100644 --- a/fftools/ffmpeg_dec.c +++ b/fftools/ffmpeg_dec.c @@ -377,6 +377,7 @@ static int process_subtitle(InputStream *ist, AVFrame *frame) if (!ost->enc || ost->type != AVMEDIA_TYPE_SUBTITLE) continue; + // XXX ret = enc_subtitle(output_files[ost->file_index], ost, subtitle); if (ret < 0) return ret; diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index 9bede78a1e..0944cddf1f 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -57,21 +57,6 @@ struct Encoder { Scheduler *sch; unsigned sch_idx; - - pthread_t thread; - /** - * Queue for sending frames from the main thread to - * the encoder thread. - */ - ThreadQueue *queue_in; - /** - * Queue for sending encoded packets from the encoder thread - * to the main thread. - * - * An empty packet is sent to signal that a previously sent - * frame has been fully processed. - */ - ThreadQueue *queue_out; }; // data that is local to the decoder thread and not visible outside of it @@ -80,24 +65,6 @@ typedef struct EncoderThread { AVPacket *pkt; } EncoderThread; -static int enc_thread_stop(Encoder *e) -{ - void *ret; - - if (!e->queue_in) - return 0; - - tq_send_finish(e->queue_in, 0); - tq_receive_finish(e->queue_out, 0); - - pthread_join(e->thread, &ret); - - tq_free(&e->queue_in); - tq_free(&e->queue_out); - - return (int)(intptr_t)ret; -} - void enc_free(Encoder **penc) { Encoder *enc = *penc; @@ -105,8 +72,6 @@ void enc_free(Encoder **penc) if (!enc) return; - enc_thread_stop(enc); - av_frame_free(&enc->sq_frame); av_frame_free(&enc->sub_frame); @@ -223,52 +188,9 @@ static int set_encoder_id(OutputFile *of, OutputStream *ost) return 0; } -static int enc_thread_start(OutputStream *ost) -{ - Encoder *e = ost->enc; - ObjPool *op; - int ret = 0; - - op = objpool_alloc_frames(); - if (!op) - return AVERROR(ENOMEM); - - e->queue_in = tq_alloc(1, 1, op, frame_move); - if (!e->queue_in) { - objpool_free(&op); - return AVERROR(ENOMEM); - } - - op = objpool_alloc_packets(); - if (!op) - goto fail; - - e->queue_out = tq_alloc(1, 4, op, pkt_move); - if (!e->queue_out) { - objpool_free(&op); - goto fail; - } - - ret = pthread_create(&e->thread, NULL, encoder_thread, ost); - if (ret) { - ret = AVERROR(ret); - av_log(ost, AV_LOG_ERROR, "pthread_create() failed: %s\n", - av_err2str(ret)); - goto fail; - } - - return 0; -fail: - if (ret >= 0) - ret = AVERROR(ENOMEM); - - tq_free(&e->queue_in); - tq_free(&e->queue_out); - return ret; -} - -int enc_open(OutputStream *ost, const AVFrame *frame) +int enc_open(void *opaque, const AVFrame *frame) { + OutputStream *ost = opaque; InputStream *ist = ost->ist; Encoder *e = ost->enc; AVCodecContext *enc_ctx = ost->enc_ctx; @@ -276,6 +198,7 @@ int enc_open(OutputStream *ost, const AVFrame *frame) const AVCodec *enc = enc_ctx->codec; OutputFile *of = output_files[ost->file_index]; FrameData *fd; + int frame_samples = 0; int ret; if (e->opened) @@ -425,11 +348,8 @@ int enc_open(OutputStream *ost, const AVFrame *frame) return AVERROR(ENOMEM); } - if (ost->enc_ctx->frame_size) { - av_assert0(ost->sq_idx_encode >= 0); - sq_frame_samples(output_files[ost->file_index]->sq_encode, - ost->sq_idx_encode, ost->enc_ctx->frame_size); - } + if (ost->enc_ctx->frame_size) + frame_samples = ost->enc_ctx->frame_size; ret = check_avoptions(ost->encoder_opts); if (ret < 0) @@ -489,18 +409,11 @@ int enc_open(OutputStream *ost, const AVFrame *frame) if (ost->st->time_base.num <= 0 || ost->st->time_base.den <= 0) ost->st->time_base = av_add_q(ost->enc_ctx->time_base, (AVRational){0, 1}); - ret = enc_thread_start(ost); - if (ret < 0) { - av_log(ost, AV_LOG_ERROR, "Error starting encoder thread: %s\n", - av_err2str(ret)); - return ret; - } - ret = of_stream_init(of, ost); if (ret < 0) return ret; - return 0; + return frame_samples; } static int check_recording_time(OutputStream *ost, int64_t ts, AVRational tb) @@ -593,7 +506,7 @@ static int do_subtitle_out(OutputFile *of, OutputStream *ost, const AVSubtitle * } pkt->dts = pkt->pts; - ret = tq_send(e->queue_out, 0, pkt); + ret = sch_enc_send(e->sch, e->sch_idx, pkt); if (ret < 0) { av_packet_unref(pkt); return ret; @@ -827,7 +740,7 @@ static int encode_frame(OutputFile *of, OutputStream *ost, AVFrame *frame, e->packets_encoded++; - ret = tq_send(e->queue_out, 0, pkt); + ret = sch_enc_send(e->sch, e->sch_idx, pkt); if (ret < 0) { av_packet_unref(pkt); return ret; @@ -1047,13 +960,32 @@ void *encoder_thread(void *arg) enc_thread_set_name(ost); - while (!input_status) { - int dummy; + /* Open the subtitle encoders immediately. AVFrame-based encoders + * are opened through a callback from the scheduler once they get + * their first frame */ + if (ost->type != AVMEDIA_TYPE_VIDEO && ost->type != AVMEDIA_TYPE_AUDIO) { + ret = enc_open(ost, NULL); + if (ret < 0) + goto finish; + } - input_status = tq_receive(e->queue_in, &dummy, et.frame); - if (input_status < 0) + while (!input_status) { + input_status = sch_enc_receive(e->sch, e->sch_idx, et.frame); + if (input_status == AVERROR_EOF) { av_log(ost, AV_LOG_VERBOSE, "Encoder thread received EOF\n"); + if (!e->opened) { + av_log(ost, AV_LOG_ERROR, "Could not open encoder before EOF\n"); + ret = AVERROR(EINVAL); + goto finish; + } + } else if (input_status < 0) { + ret = input_status; + av_log(ost, AV_LOG_ERROR, "Error receiving a frame for encoding: %s\n", + av_err2str(ret)); + goto finish; + } + ret = frame_encode(ost, input_status >= 0 ? et.frame : NULL, et.pkt); av_packet_unref(et.pkt); @@ -1067,15 +999,6 @@ void *encoder_thread(void *arg) av_err2str(ret)); break; } - - // signal to the consumer thread that the frame was encoded - ret = tq_send(e->queue_out, 0, et.pkt); - if (ret < 0) { - if (ret != AVERROR_EOF) - av_log(ost, AV_LOG_ERROR, - "Error communicating with the main thread\n"); - break; - } } // EOF is normal thread termination @@ -1086,8 +1009,7 @@ finish: if (ost->sq_idx_encode >= 0) sq_send(of->sq_encode, ost->sq_idx_encode, SQFRAME(NULL)); - tq_receive_finish(e->queue_in, 0); - tq_send_finish (e->queue_out, 0); + sch_enc_send(e->sch, e->sch_idx, NULL); enc_thread_uninit(&et); @@ -1096,62 +1018,6 @@ finish: return (void*)(intptr_t)ret; } -int enc_frame(OutputStream *ost, AVFrame *frame) -{ - OutputFile *of = output_files[ost->file_index]; - Encoder *e = ost->enc; - int ret, thread_ret; - - ret = enc_open(ost, frame); - if (ret < 0) - return ret; - - // thread already joined - // XXX check EOF handling - if (!e->queue_in) - return AVERROR_EOF; - - // send the frame/EOF to the encoder thread - if (frame) { - ret = tq_send(e->queue_in, 0, frame); - if (ret < 0) - goto finish; - } else - tq_send_finish(e->queue_in, 0); - - // retrieve all encoded data for the frame - while (1) { - int dummy; - - ret = tq_receive(e->queue_out, &dummy, e->pkt); - if (ret < 0) - break; - - // frame fully encoded - if (!e->pkt->data && !e->pkt->side_data_elems) - return 0; - - // process the encoded packet - ret = of_output_packet(of, ost, e->pkt); - if (ret < 0) - goto finish; - } - -finish: - thread_ret = enc_thread_stop(e); - if (thread_ret < 0) { - av_log(ost, AV_LOG_ERROR, "Encoder thread returned error: %s\n", - av_err2str(thread_ret)); - ret = err_merge(ret, thread_ret); - } - - if (ret < 0 && ret != AVERROR_EOF) - return ret; - - // signal EOF to the muxer - return of_output_packet(of, ost, NULL); -} - int enc_subtitle(OutputFile *of, OutputStream *ost, const AVSubtitle *sub) { Encoder *e = ost->enc; @@ -1166,38 +1032,11 @@ int enc_subtitle(OutputFile *of, OutputStream *ost, const AVSubtitle *sub) if (ret < 0) return ret; + // XXX +#if 0 ret = enc_frame(ost, f); +#endif av_frame_unref(f); return ret; } - -int enc_flush(void) -{ - int ret = 0; - - for (OutputStream *ost = ost_iter(NULL); ost; ost = ost_iter(ost)) { - OutputFile *of = output_files[ost->file_index]; - if (ost->sq_idx_encode >= 0) - sq_send(of->sq_encode, ost->sq_idx_encode, SQFRAME(NULL)); - } - - for (OutputStream *ost = ost_iter(NULL); ost; ost = ost_iter(ost)) { - Encoder *e = ost->enc; - AVCodecContext *enc = ost->enc_ctx; - int err; - - if (!enc || !e->opened || - (enc->codec_type != AVMEDIA_TYPE_VIDEO && enc->codec_type != AVMEDIA_TYPE_AUDIO)) - continue; - - err = enc_frame(ost, NULL); - // XXX check EOF handling - if (err != AVERROR_EOF && ret < 0) - ret = err_merge(ret, err); - - av_assert0(!e->queue_in); - } - - return ret; -} diff --git a/fftools/ffmpeg_mux_init.c b/fftools/ffmpeg_mux_init.c index 3380cbeb5c..9fb6a74393 100644 --- a/fftools/ffmpeg_mux_init.c +++ b/fftools/ffmpeg_mux_init.c @@ -1186,7 +1186,8 @@ static int ost_add(Muxer *mux, const OptionsContext *o, enum AVMediaType type, if (!ost->enc_ctx) return AVERROR(ENOMEM); - ret = sch_add_enc(mux->sch, encoder_thread, ost, NULL); + ret = sch_add_enc(mux->sch, encoder_thread, ost, + ost->type == AVMEDIA_TYPE_SUBTITLE ? NULL : enc_open); if (ret < 0) return ret; ms->sched_idx_enc = ret; @@ -2621,23 +2622,6 @@ static int validate_enc_avopt(Muxer *mux, const AVDictionary *codec_avopt) return 0; } -static int init_output_stream_nofilter(OutputStream *ost) -{ - int ret = 0; - - if (ost->enc_ctx) { - ret = enc_open(ost, NULL); - if (ret < 0) - return ret; - } else { - ret = of_stream_init(output_files[ost->file_index], ost); - if (ret < 0) - return ret; - } - - return ret; -} - static const char *output_file_item_name(void *obj) { const Muxer *mux = obj; @@ -2822,26 +2806,15 @@ int of_open(const OptionsContext *o, const char *filename, Scheduler *sch) of->url = filename; - /* initialize stream copy and subtitle/data streams. - * Encoded AVFrame based streams will get initialized when the first AVFrame - * is received in do_video_out - */ + /* initialize streamcopy streams. */ for (int i = 0; i < of->nb_streams; i++) { OutputStream *ost = of->streams[i]; - if (ost->filter) - continue; - - err = init_output_stream_nofilter(ost); - if (err < 0) - return err; - } - - /* write the header for files with no streams */ - if (of->format->flags & AVFMT_NOSTREAMS && oc->nb_streams == 0) { - int ret = mux_check_init(mux); - if (ret < 0) - return ret; + if (!ost->enc) { + err = of_stream_init(of, ost); + if (err < 0) + return err; + } } return 0; -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 26/27] WIP fftools/ffmpeg_mux: convert to the scheduler 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (24 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 25/27] WIP fftools/ffmpeg_enc: " Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 27/27] WIP: ffmpeg: switch to scheduler Anton Khirnov 2023-09-20 14:49 ` [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Michael Niedermayer 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel --- fftools/ffmpeg.c | 26 ------ fftools/ffmpeg.h | 8 -- fftools/ffmpeg_mux.c | 184 ++++++++++++-------------------------- fftools/ffmpeg_mux.h | 14 ++- fftools/ffmpeg_mux_init.c | 33 +++---- fftools/ffmpeg_opt.c | 6 +- 6 files changed, 80 insertions(+), 191 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 69e73b84ed..e82d88b3e0 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -806,38 +806,12 @@ int trigger_fix_sub_duration_heartbeat(OutputStream *ost, const AVPacket *pkt) static int process_input_packet(InputStream *ist, const AVPacket *pkt, int no_eof) { InputFile *f = input_files[ist->file_index]; - int64_t dts_est = AV_NOPTS_VALUE; int ret = 0; int eof_reached = 0; if (ret == AVERROR_EOF || (!pkt && !ist->decoding_needed)) eof_reached = 1; - if (pkt && pkt->opaque_ref) { - DemuxPktData *pd = (DemuxPktData*)pkt->opaque_ref->data; - dts_est = pd->dts_est; - } - - if (f->recording_time != INT64_MAX) { - int64_t start_time = 0; - if (copy_ts) { - start_time += f->start_time != AV_NOPTS_VALUE ? f->start_time : 0; - start_time += start_at_zero ? 0 : f->start_time_effective; - } - if (dts_est >= f->recording_time + start_time) - pkt = NULL; - } - - for (int oidx = 0; oidx < ist->nb_outputs; oidx++) { - OutputStream *ost = ist->outputs[oidx]; - if (ost->enc || (!pkt && no_eof)) - continue; - - ret = of_streamcopy(ost, pkt, dts_est); - if (ret < 0) - return ret; - } - return !eof_reached; } diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 66b5e398bc..e3aea75415 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -665,7 +665,6 @@ extern FilterGraph **filtergraphs; extern int nb_filtergraphs; extern char *vstats_filename; -extern char *sdp_filename; extern float dts_delta_threshold; extern float dts_error_threshold; @@ -813,13 +812,6 @@ void of_free(OutputFile **pof); void of_enc_stats_close(void); -int of_output_packet(OutputFile *of, OutputStream *ost, AVPacket *pkt); - -/** - * @param dts predicted packet dts in AV_TIME_BASE_Q - */ -int of_streamcopy(OutputStream *ost, const AVPacket *pkt, int64_t dts); - int64_t of_filesize(OutputFile *of); int ifile_open(const OptionsContext *o, const char *filename, Scheduler *sch); diff --git a/fftools/ffmpeg_mux.c b/fftools/ffmpeg_mux.c index 9628728d95..ba90079266 100644 --- a/fftools/ffmpeg_mux.c +++ b/fftools/ffmpeg_mux.c @@ -22,16 +22,13 @@ #include "ffmpeg.h" #include "ffmpeg_mux.h" -#include "objpool.h" #include "sync_queue.h" -#include "thread_queue.h" #include "libavutil/fifo.h" #include "libavutil/intreadwrite.h" #include "libavutil/log.h" #include "libavutil/mem.h" #include "libavutil/timestamp.h" -#include "libavutil/thread.h" #include "libavcodec/packet.h" @@ -42,8 +39,6 @@ typedef struct MuxThreadContext { AVPacket *pkt; } MuxThreadContext; -int want_sdp = 1; - static Muxer *mux_from_of(OutputFile *of) { return (Muxer*)of; @@ -254,19 +249,22 @@ void *muxer_thread(void *arg) OutputStream *ost; int stream_idx, stream_eof = 0; - ret = tq_receive(mux->tq, &stream_idx, mt.pkt); + ret = sch_mux_receive(mux->sch, of->index, mt.pkt); + stream_idx = mt.pkt->stream_index; if (stream_idx < 0) { av_log(mux, AV_LOG_VERBOSE, "All streams finished\n"); ret = 0; break; } - ost = of->streams[stream_idx]; + ost = of->streams[mux->sch_stream_idx[stream_idx]]; + mt.pkt->stream_index = ost->index; + ret = sync_queue_process(mux, ost, ret < 0 ? NULL : mt.pkt, &stream_eof); av_packet_unref(mt.pkt); if (ret == AVERROR_EOF) { if (stream_eof) { - tq_receive_finish(mux->tq, stream_idx); + sch_mux_receive_finish(mux->sch, of->index, stream_idx); } else { av_log(mux, AV_LOG_VERBOSE, "Muxer returned EOF\n"); ret = 0; @@ -278,17 +276,19 @@ void *muxer_thread(void *arg) } } + // XXX move av_write_trailer() here? + finish: mux_thread_uninit(&mt); - for (unsigned int i = 0; i < mux->fc->nb_streams; i++) - tq_receive_finish(mux->tq, i); + sch_mux_receive_finish(mux->sch, of->index, -1); av_log(mux, AV_LOG_VERBOSE, "Terminating muxer thread\n"); return (void*)(intptr_t)ret; } +#if 0 static int thread_submit_packet(Muxer *mux, OutputStream *ost, AVPacket *pkt) { int ret = 0; @@ -296,7 +296,8 @@ static int thread_submit_packet(Muxer *mux, OutputStream *ost, AVPacket *pkt) if (!pkt || ost->finished & MUXER_FINISHED) goto finish; - ret = tq_send(mux->tq, ost->index, pkt); + // XXX + //ret = tq_send(mux->tq, ost->index, pkt); if (ret < 0) goto finish; @@ -306,8 +307,9 @@ finish: if (pkt) av_packet_unref(pkt); + // XXX ost->finished |= MUXER_FINISHED; - tq_send_finish(mux->tq, ost->index); + //tq_send_finish(mux->tq, ost->index); return ret == AVERROR_EOF ? 0 : ret; } @@ -428,57 +430,48 @@ fail: av_log(ost, AV_LOG_ERROR, "Error %s\n", err_msg); return exit_on_error ? ret : 0; } +#endif -int of_streamcopy(OutputStream *ost, const AVPacket *pkt, int64_t dts) +static int of_streamcopy(OutputStream *ost, AVPacket *pkt, int64_t dts) { OutputFile *of = output_files[ost->file_index]; MuxStream *ms = ms_from_ost(ost); int64_t start_time = (of->start_time == AV_NOPTS_VALUE) ? 0 : of->start_time; int64_t ts_offset; - AVPacket *opkt = ms->pkt; - int ret; - - av_packet_unref(opkt); if (of->recording_time != INT64_MAX && dts >= of->recording_time + start_time) - pkt = NULL; - - // EOF: flush output bitstream filters. - if (!pkt) - return of_output_packet(of, ost, NULL); + return AVERROR_EOF; if (!ms->streamcopy_started && !(pkt->flags & AV_PKT_FLAG_KEY) && !ms->copy_initial_nonkeyframes) - return 0; + return AVERROR(EAGAIN); if (!ms->streamcopy_started) { if (!ms->copy_prior_start && (pkt->pts == AV_NOPTS_VALUE ? dts < ms->ts_copy_start : pkt->pts < av_rescale_q(ms->ts_copy_start, AV_TIME_BASE_Q, pkt->time_base))) - return 0; + return AVERROR(EAGAIN); if (of->start_time != AV_NOPTS_VALUE && dts < of->start_time) - return 0; + return AVERROR(EAGAIN); } - ret = av_packet_ref(opkt, pkt); - if (ret < 0) - return ret; - - ts_offset = av_rescale_q(start_time, AV_TIME_BASE_Q, opkt->time_base); + ts_offset = av_rescale_q(start_time, AV_TIME_BASE_Q, pkt->time_base); if (pkt->pts != AV_NOPTS_VALUE) - opkt->pts -= ts_offset; + pkt->pts -= ts_offset; if (pkt->dts == AV_NOPTS_VALUE) { - opkt->dts = av_rescale_q(dts, AV_TIME_BASE_Q, opkt->time_base); + pkt->dts = av_rescale_q(dts, AV_TIME_BASE_Q, pkt->time_base); } else if (ost->st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) { - opkt->pts = opkt->dts - ts_offset; + pkt->pts = pkt->dts - ts_offset; } - opkt->dts -= ts_offset; + pkt->dts -= ts_offset; + // XXX +#if 0 { int ret = trigger_fix_sub_duration_heartbeat(ost, pkt); if (ret < 0) { @@ -488,73 +481,42 @@ int of_streamcopy(OutputStream *ost, const AVPacket *pkt, int64_t dts) return ret; } } - - ret = of_output_packet(of, ost, opkt); - if (ret < 0) - return ret; +#endif ms->streamcopy_started = 1; return 0; } -static int thread_stop(Muxer *mux) +int of_streamcopy_hook(void *opaque, void *item) { - void *ret; + OutputStream *ost = opaque; + const InputStream *ist = ost->ist; + const InputFile *f = input_files[ist->file_index]; - if (!mux || !mux->tq) - return 0; + AVPacket *pkt = item; + int64_t dts_est = AV_NOPTS_VALUE; - for (unsigned int i = 0; i < mux->fc->nb_streams; i++) - tq_send_finish(mux->tq, i); + // nothing for us to do about seek signals + if ((intptr_t)pkt->opaque == PKT_OPAQUE_SEEK) + return AVERROR(EAGAIN); - pthread_join(mux->thread, &ret); - - tq_free(&mux->tq); - - return (int)(intptr_t)ret; -} - -static int thread_start(Muxer *mux) -{ - AVFormatContext *fc = mux->fc; - ObjPool *op; - int ret; - - op = objpool_alloc_packets(); - if (!op) - return AVERROR(ENOMEM); - - mux->tq = tq_alloc(fc->nb_streams, mux->thread_queue_size, op, pkt_move); - if (!mux->tq) { - objpool_free(&op); - return AVERROR(ENOMEM); + if (pkt->opaque_ref) { + DemuxPktData *pd = (DemuxPktData*)pkt->opaque_ref->data; + dts_est = pd->dts_est; } - ret = pthread_create(&mux->thread, NULL, muxer_thread, (void*)mux); - if (ret) { - tq_free(&mux->tq); - return AVERROR(ret); - } - - /* flush the muxing queues */ - for (int i = 0; i < fc->nb_streams; i++) { - OutputStream *ost = mux->of.streams[i]; - MuxStream *ms = ms_from_ost(ost); - AVPacket *pkt; - - while (av_fifo_read(ms->muxing_queue, &pkt, 1) >= 0) { - ret = thread_submit_packet(mux, ost, pkt); - if (pkt) { - ms->muxing_queue_data_size -= pkt->size; - av_packet_free(&pkt); - } - if (ret < 0) - return ret; + if (f->recording_time != INT64_MAX) { + int64_t start_time = 0; + if (copy_ts) { + start_time += f->start_time != AV_NOPTS_VALUE ? f->start_time : 0; + start_time += start_at_zero ? 0 : f->start_time_effective; } + if (dts_est >= f->recording_time + start_time) + return AVERROR_EOF; } - return 0; + return of_streamcopy(ost, pkt, dts_est); } int print_sdp(const char *filename) @@ -565,11 +527,6 @@ int print_sdp(const char *filename) AVIOContext *sdp_pb; AVFormatContext **avc; - for (i = 0; i < nb_output_files; i++) { - if (!mux_from_of(output_files[i])->header_written) - return 0; - } - avc = av_malloc_array(nb_output_files, sizeof(*avc)); if (!avc) return AVERROR(ENOMEM); @@ -604,25 +561,17 @@ int print_sdp(const char *filename) avio_closep(&sdp_pb); } - // SDP successfully written, allow muxer threads to start - ret = 1; - fail: av_freep(&avc); return ret; } -int mux_check_init(Muxer *mux) +int mux_check_init(void *arg) { + Muxer *mux = arg; OutputFile *of = &mux->of; AVFormatContext *fc = mux->fc; - int ret, i; - - for (i = 0; i < fc->nb_streams; i++) { - OutputStream *ost = of->streams[i]; - if (!ost->initialized) - return 0; - } + int ret; ret = avformat_write_header(fc, &mux->opts); if (ret < 0) { @@ -636,26 +585,6 @@ int mux_check_init(Muxer *mux) av_dump_format(fc, of->index, fc->url, 1); nb_output_dumped++; - if (sdp_filename || want_sdp) { - ret = print_sdp(sdp_filename); - if (ret < 0) { - av_log(NULL, AV_LOG_ERROR, "Error writing the SDP.\n"); - return ret; - } else if (ret == 1) { - /* SDP is written only after all the muxers are ready, so now we - * start ALL the threads */ - for (i = 0; i < nb_output_files; i++) { - ret = thread_start(mux_from_of(output_files[i])); - if (ret < 0) - return ret; - } - } - } else { - ret = thread_start(mux_from_of(of)); - if (ret < 0) - return ret; - } - return 0; } @@ -711,9 +640,10 @@ int of_stream_init(OutputFile *of, OutputStream *ost) ost->st->time_base); } - ost->initialized = 1; + if (ms->sched_idx >= 0) + return sch_mux_stream_ready(mux->sch, of->index, ms->sched_idx); - return mux_check_init(mux); + return 0; } static int check_written(OutputFile *of) @@ -827,15 +757,13 @@ int of_write_trailer(OutputFile *of) AVFormatContext *fc = mux->fc; int ret, mux_result = 0; - if (!mux->tq) { + if (!mux->header_written) { av_log(mux, AV_LOG_ERROR, "Nothing was written into output file, because " "at least one of its streams received no packets.\n"); return AVERROR(EINVAL); } - mux_result = thread_stop(mux); - ret = av_write_trailer(fc); if (ret < 0) { av_log(mux, AV_LOG_ERROR, "Error writing trailer: %s\n", av_err2str(ret)); @@ -951,8 +879,6 @@ void of_free(OutputFile **pof) return; mux = mux_from_of(of); - thread_stop(mux); - sq_free(&of->sq_encode); sq_free(&mux->sq_mux); diff --git a/fftools/ffmpeg_mux.h b/fftools/ffmpeg_mux.h index d5aba6db36..311b612018 100644 --- a/fftools/ffmpeg_mux.h +++ b/fftools/ffmpeg_mux.h @@ -25,7 +25,6 @@ #include <stdint.h> #include "ffmpeg_sched.h" -#include "thread_queue.h" #include "libavformat/avformat.h" @@ -33,7 +32,6 @@ #include "libavutil/dict.h" #include "libavutil/fifo.h" -#include "libavutil/thread.h" typedef struct MuxStream { OutputStream ost; @@ -104,9 +102,6 @@ typedef struct Muxer { int *sch_stream_idx; int nb_sch_stream_idx; - pthread_t thread; - ThreadQueue *tq; - AVDictionary *opts; int thread_queue_size; @@ -120,14 +115,15 @@ typedef struct Muxer { AVPacket *sq_pkt; } Muxer; -/* whether we want to print an SDP, set in of_open() */ -extern int want_sdp; - -int mux_check_init(Muxer *mux); +int mux_check_init(void *arg); static MuxStream *ms_from_ost(OutputStream *ost) { return (MuxStream*)ost; } +/* XXX explain why this is needed + * (so it's called in the sending thread rather than the muxer) */ +int of_streamcopy_hook(void *opaque, void *item); + #endif /* FFTOOLS_FFMPEG_MUX_H */ diff --git a/fftools/ffmpeg_mux_init.c b/fftools/ffmpeg_mux_init.c index 9fb6a74393..122a7344e4 100644 --- a/fftools/ffmpeg_mux_init.c +++ b/fftools/ffmpeg_mux_init.c @@ -1375,7 +1375,7 @@ static int ost_add(Muxer *mux, const OptionsContext *o, enum AVMediaType type, MATCH_PER_STREAM_OPT(bitstream_filters, str, bsfs, oc, st); if (bsfs && *bsfs) { - ret = av_bsf_list_parse_str(bsfs, &ms->bsf_ctx); + ret = sch_add_mux_stream_bsf(mux->sch, mux->of.index, ost->index, bsfs); if (ret < 0) { av_log(ost, AV_LOG_ERROR, "Error parsing bitstream filter sequence '%s': %s\n", bsfs, av_err2str(ret)); return ret; @@ -1486,7 +1486,8 @@ static int ost_add(Muxer *mux, const OptionsContext *o, enum AVMediaType type, return ret; } else { ret = sch_connect(mux->sch, SCH_DSTREAM(ost->ist->file_index, sched_idx), - SCH_MSTREAM(ost->file_index, ms->sched_idx), NULL, NULL); + SCH_MSTREAM(ost->file_index, ms->sched_idx), + of_streamcopy_hook, ost); if (ret < 0) return ret; } @@ -1931,11 +1932,17 @@ static int setup_sync_queues(Muxer *mux, AVFormatContext *oc, int64_t buf_size_u * - at least one encoded audio/video stream is frame-limited, since * that has similar semantics to 'shortest' * - at least one audio encoder requires constant frame sizes + * + * Note that encoding sync queues are handled in the scheduler, because + * different encoders run in different threads and need external + * synchronization, while muxer sync queues can be handled inside the muxer */ if ((of->shortest && nb_av_enc > 1) || limit_frames_av_enc || nb_audio_fs) { - of->sq_encode = sq_alloc(SYNC_QUEUE_FRAMES, buf_size_us, mux); - if (!of->sq_encode) - return AVERROR(ENOMEM); + int sq_idx, ret; + + sq_idx = sch_add_sq_enc(mux->sch, buf_size_us, mux); + if (sq_idx < 0) + return sq_idx; for (int i = 0; i < oc->nb_streams; i++) { OutputStream *ost = of->streams[i]; @@ -1945,13 +1952,11 @@ static int setup_sync_queues(Muxer *mux, AVFormatContext *oc, int64_t buf_size_u if (!IS_AV_ENC(ost, type)) continue; - ost->sq_idx_encode = sq_add_stream(of->sq_encode, - of->shortest || ms->max_frames < INT64_MAX); - if (ost->sq_idx_encode < 0) - return ost->sq_idx_encode; - - if (ms->max_frames != INT64_MAX) - sq_limit_frames(of->sq_encode, ost->sq_idx_encode, ms->max_frames); + ret = sch_sq_add_enc(mux->sch, sq_idx, ms->sched_idx_enc, + of->shortest || ms->max_frames < INT64_MAX, + ms->max_frames); + if (ret < 0) + return ret; } } @@ -2704,8 +2709,6 @@ int of_open(const OptionsContext *o, const char *filename, Scheduler *sch) av_strlcat(mux->log_name, "/", sizeof(mux->log_name)); av_strlcat(mux->log_name, oc->oformat->name, sizeof(mux->log_name)); - if (strcmp(oc->oformat->name, "rtp")) - want_sdp = 0; of->format = oc->oformat; if (recording_time != INT64_MAX) @@ -2721,7 +2724,7 @@ int of_open(const OptionsContext *o, const char *filename, Scheduler *sch) AVFMT_FLAG_BITEXACT); } - err = sch_add_mux(sch, muxer_thread, NULL, mux, + err = sch_add_mux(sch, muxer_thread, mux_check_init, mux, !strcmp(oc->oformat->name, "rtp")); if (err < 0) return err; diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index d463306546..6177a96a4e 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -64,7 +64,6 @@ const char *const opt_name_top_field_first[] = {"top", NULL}; HWDevice *filter_hw_device; char *vstats_filename; -char *sdp_filename; float audio_drift_threshold = 0.1; float dts_delta_threshold = 10; @@ -580,9 +579,8 @@ fail: static int opt_sdp_file(void *optctx, const char *opt, const char *arg) { - av_free(sdp_filename); - sdp_filename = av_strdup(arg); - return 0; + Scheduler *sch = optctx; + return sch_sdp_filename(sch, arg); } #if CONFIG_VAAPI -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* [FFmpeg-devel] [PATCH 27/27] WIP: ffmpeg: switch to scheduler 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (25 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 26/27] WIP fftools/ffmpeg_mux: " Anton Khirnov @ 2023-09-19 19:10 ` Anton Khirnov 2023-09-20 14:49 ` [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Michael Niedermayer 27 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-19 19:10 UTC (permalink / raw) To: ffmpeg-devel --- fftools/ffmpeg.c | 237 ++--------------------------------------------- fftools/ffmpeg.h | 6 -- 2 files changed, 10 insertions(+), 233 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index e82d88b3e0..f3e150e453 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -137,6 +137,8 @@ static struct termios oldtty; static int restore_tty; #endif +// XXX should be triggered from the demuxer after every input packets +#if 0 /* sub2video hack: Convert subtitles to video with alpha to insert them in filter graphs. This is a temporary solution until libavfilter gets real subtitles support. @@ -160,6 +162,7 @@ static void sub2video_heartbeat(InputFile *infile, int64_t pts, AVRational tb) } /* end of sub2video hack */ +#endif static void term_exit_sigsafe(void) { @@ -802,19 +805,6 @@ int trigger_fix_sub_duration_heartbeat(OutputStream *ost, const AVPacket *pkt) return 0; } -/* pkt = NULL means EOF (needed to flush decoder buffers) */ -static int process_input_packet(InputStream *ist, const AVPacket *pkt, int no_eof) -{ - InputFile *f = input_files[ist->file_index]; - int ret = 0; - int eof_reached = 0; - - if (ret == AVERROR_EOF || (!pkt && !ist->decoding_needed)) - eof_reached = 1; - - return !eof_reached; -} - static void print_stream_maps(void) { av_log(NULL, AV_LOG_INFO, "Stream mapping:\n"); @@ -891,43 +881,6 @@ static void print_stream_maps(void) } } -/** - * Select the output stream to process. - * - * @retval 0 an output stream was selected - * @retval AVERROR(EAGAIN) need to wait until more input is available - * @retval AVERROR_EOF no more streams need output - */ -static int choose_output(OutputStream **post) -{ - int64_t opts_min = INT64_MAX; - OutputStream *ost_min = NULL; - - for (OutputStream *ost = ost_iter(NULL); ost; ost = ost_iter(ost)) { - int64_t opts; - - if (ost->filter && ost->filter->last_pts != AV_NOPTS_VALUE) { - opts = ost->filter->last_pts; - } else { - opts = ost->last_mux_dts == AV_NOPTS_VALUE ? - INT64_MIN : ost->last_mux_dts; - } - - if (!ost->initialized && !ost->finished) { - ost_min = ost; - break; - } - if (!ost->finished && opts < opts_min) { - opts_min = opts; - ost_min = ost; - } - } - if (!ost_min) - return AVERROR_EOF; - *post = ost_min; - return ost_min->unavailable ? AVERROR(EAGAIN) : 0; -} - static void set_tty_echo(int on) { #if HAVE_TERMIOS_H @@ -999,153 +952,22 @@ static int check_keyboard_interaction(int64_t cur_time) return 0; } -static void reset_eagain(void) -{ - for (OutputStream *ost = ost_iter(NULL); ost; ost = ost_iter(ost)) - ost->unavailable = 0; -} - -#if 0 -static void decode_flush(InputFile *ifile) -{ - for (int i = 0; i < ifile->nb_streams; i++) { - InputStream *ist = ifile->streams[i]; - - if (ist->discard || !ist->decoding_needed) - continue; - - dec_packet(ist, NULL, 1); - } -} -#endif - -/* - * Return - * - 0 -- one packet was read and processed - * - AVERROR(EAGAIN) -- no packets were available for selected file, - * this function should be called again - * - AVERROR_EOF -- this function should not be called again - */ -static int process_input(int file_index, AVPacket *pkt) -{ - InputFile *ifile = input_files[file_index]; - InputStream *ist; - int ret, i; - - ret = 0; - -#if 0 - if (ret == 1) { - /* the input file is looped: flush the decoders */ - decode_flush(ifile); - return AVERROR(EAGAIN); - } -#endif - if (ret < 0) { - if (ret != AVERROR_EOF) { - av_log(ifile, AV_LOG_ERROR, - "Error retrieving a packet from demuxer: %s\n", av_err2str(ret)); - if (exit_on_error) - return ret; - } - - for (i = 0; i < ifile->nb_streams; i++) { - ist = ifile->streams[i]; - if (!ist->discard) { - ret = process_input_packet(ist, NULL, 0); - if (ret>0) - return 0; - else if (ret < 0) - return ret; - } - - /* mark all outputs that don't go through lavfi as finished */ - for (int oidx = 0; oidx < ist->nb_outputs; oidx++) { - OutputStream *ost = ist->outputs[oidx]; - OutputFile *of = output_files[ost->file_index]; - - ret = of_output_packet(of, ost, NULL); - if (ret < 0) - return ret; - } - } - - ifile->eof_reached = 1; - return AVERROR(EAGAIN); - } - - reset_eagain(); - - ist = ifile->streams[pkt->stream_index]; - - sub2video_heartbeat(ifile, pkt->pts, pkt->time_base); - - ret = process_input_packet(ist, pkt, 0); - - av_packet_unref(pkt); - - return ret < 0 ? ret : 0; -} - -/** - * Run a single step of transcoding. - * - * @return 0 for success, <0 for error - */ -static int transcode_step(OutputStream *ost, AVPacket *demux_pkt) -{ - InputStream *ist = NULL; - int ret; - - if (ost->filter) { - if ((ret = fg_transcode_step(ost->filter->graph, &ist)) < 0) - return ret; - if (!ist) - return 0; - } else { - ist = ost->ist; - av_assert0(ist); - } - - ret = process_input(ist->file_index, demux_pkt); - if (ret == AVERROR(EAGAIN)) { - return 0; - } - - if (ret < 0) - return ret == AVERROR_EOF ? 0 : ret; - - // process_input() above might have caused output to become available - // in multiple filtergraphs, so we process all of them - for (int i = 0; i < nb_filtergraphs; i++) { - ret = reap_filters(filtergraphs[i], 0); - if (ret < 0) - return ret; - } - - return 0; -} - /* * The following code is the main loop of the file converter */ static int transcode(Scheduler *sch, int *err_rate_exceeded) { int ret = 0, i; - InputStream *ist; int64_t timer_start; - AVPacket *demux_pkt = NULL; print_stream_maps(); *err_rate_exceeded = 0; atomic_store(&transcode_init_done, 1); - demux_pkt = av_packet_alloc(); - if (!demux_pkt) { - ret = AVERROR(ENOMEM); - goto fail; - } + ret = sch_start(sch); + if (ret < 0) + return ret; if (stdin_interaction) { av_log(NULL, AV_LOG_INFO, "Press [q] to stop, [?] for help\n"); @@ -1153,8 +975,7 @@ static int transcode(Scheduler *sch, int *err_rate_exceeded) timer_start = av_gettime_relative(); - while (!received_sigterm) { - OutputStream *ost; + while (!sch_wait(sch, stats_period)) { int64_t cur_time= av_gettime_relative(); /* if 'q' pressed, exits */ @@ -1162,48 +983,11 @@ static int transcode(Scheduler *sch, int *err_rate_exceeded) if (check_keyboard_interaction(cur_time) < 0) break; - ret = choose_output(&ost); - if (ret == AVERROR(EAGAIN)) { - reset_eagain(); - av_usleep(10000); - ret = 0; - continue; - } else if (ret < 0) { - av_log(NULL, AV_LOG_VERBOSE, "No more output streams to write to, finishing.\n"); - ret = 0; - break; - } - - ret = transcode_step(ost, demux_pkt); - if (ret < 0 && ret != AVERROR_EOF) { - av_log(NULL, AV_LOG_ERROR, "Error while filtering: %s\n", av_err2str(ret)); - break; - } - /* dump report by using the output first video and audio streams */ print_report(0, timer_start, cur_time); } - /* at the end of stream, we must flush the decoder buffers */ - for (ist = ist_iter(NULL); ist; ist = ist_iter(ist)) { - float err_rate; - - if (!input_files[ist->file_index]->eof_reached) { - int err = process_input_packet(ist, NULL, 0); - ret = err_merge(ret, err); - } - - err_rate = (ist->frames_decoded || ist->decode_errors) ? - ist->decode_errors / (ist->frames_decoded + ist->decode_errors) : 0.f; - if (err_rate > max_error_rate) { - av_log(ist, AV_LOG_FATAL, "Decode error rate %g exceeds maximum %g\n", - err_rate, max_error_rate); - *err_rate_exceeded = 1; - } else if (err_rate) - av_log(ist, AV_LOG_VERBOSE, "Decode error rate %g\n", err_rate); - } - - term_exit(); + ret = sch_stop(sch); /* write the trailer if needed */ for (i = 0; i < nb_output_files; i++) { @@ -1211,12 +995,11 @@ static int transcode(Scheduler *sch, int *err_rate_exceeded) ret = err_merge(ret, err); } + term_exit(); + /* dump report by using the first video and audio streams */ print_report(1, timer_start, av_gettime_relative()); -fail: - av_packet_free(&demux_pkt); - return ret; } diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index e3aea75415..3f2cb7bfac 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -586,12 +586,6 @@ typedef struct OutputStream { AVDictionary *swr_opts; char *apad; OSTFinished finished; /* no more packets should be written for this stream */ - int unavailable; /* true if the steram is unavailable (possibly temporarily) */ - - // init_output_stream() has been called for this stream - // The encoder and the bitstream filters have been initialized and the stream - // parameters are set in the AVStream. - int initialized; const char *attachment_filename; -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov ` (26 preceding siblings ...) 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 27/27] WIP: ffmpeg: switch to scheduler Anton Khirnov @ 2023-09-20 14:49 ` Michael Niedermayer 2023-09-25 10:17 ` Anton Khirnov 27 siblings, 1 reply; 30+ messages in thread From: Michael Niedermayer @ 2023-09-20 14:49 UTC (permalink / raw) To: FFmpeg development discussions and patches [-- Attachment #1.1: Type: text/plain, Size: 603 bytes --] Hi On Tue, Sep 19, 2023 at 09:10:27PM +0200, Anton Khirnov wrote: [...] > You can fetch the code from the 'ffmpeg_threading' branch in > git://git.khirnov.net/libav. I will also present a short talk about this > work at VDD. Comments, questions, suggestions, etc. are very much > welcome, both here and there. please record the talk and make it available for people who arent physically or virtually there at the time Thx! [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Observe your enemies, for they first find out your faults. -- Antisthenes [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] [-- Attachment #2: Type: text/plain, Size: 251 bytes --] _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading 2023-09-20 14:49 ` [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Michael Niedermayer @ 2023-09-25 10:17 ` Anton Khirnov 0 siblings, 0 replies; 30+ messages in thread From: Anton Khirnov @ 2023-09-25 10:17 UTC (permalink / raw) To: FFmpeg development discussions and patches Quoting Michael Niedermayer (2023-09-20 16:49:42) > Hi > > On Tue, Sep 19, 2023 at 09:10:27PM +0200, Anton Khirnov wrote: > [...] > > You can fetch the code from the 'ffmpeg_threading' branch in > > git://git.khirnov.net/libav. I will also present a short talk about this > > work at VDD. Comments, questions, suggestions, etc. are very much > > welcome, both here and there. > > please record the talk and make it available for people who arent physically > or virtually there at the time Recording of the talk can be seen at https://youtu.be/Z4DS3jiZhfo?t=1221 -- Anton Khirnov _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 30+ messages in thread
end of thread, other threads:[~2023-09-25 10:17 UTC | newest] Thread overview: 30+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-09-19 19:10 [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 01/27] fftools/ffmpeg: move derivation of frame duration from filter framerate Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 02/27] fftools/ffmpeg_enc: move handling video frame duration to video_sync_process() Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 03/27] fftools/ffmpeg_enc: move remaining vsync-related code " Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 04/27] fftools/ffmpeg_enc: simplify adjust_frame_pts_to_encoder_tb() signature Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 05/27] ffools/ffmpeg_filter: stop trying to handle an unreachable state Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 06/27] tests/fate/ffmpeg: add tests for -force_key_frames source Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 07/27] fftools/ffmpeg_enc: unbreak -force_key_frames source_no_drop Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 08/27] fftools/ffmpeg_enc: merge -force_key_frames source/source_no_drop Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 09/27] fftools/ffmpeg: stop accessing OutputStream.last_dropped in print_report() Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 10/27] fftools/ffmpeg_enc: move framerate conversion state into a separate struct Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 11/27] fftools/ffmpeg_enc: move fps conversion code to ffmpeg_filter Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 12/27] fftools/ffmpeg_filter: fail on filtering errors Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 13/27] fftools/ffmpeg_enc: constify the frame passed to enc_open() Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 14/27] fftools/ffmpeg_filter: move filtering to a separate thread Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 15/27] fftools/ffmpeg_mux: add muxing thread private data Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 16/27] fftools/ffmpeg_demux: switch from AVThreadMessageQueue to ThreadQueue Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 17/27] XXX: disable fix_sub_duration_heartbeat Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 18/27] XXX fftools/ffmpeg_enc: temporarily disable side data copying Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 19/27] XXX ffmpeg temporarily disable -stream_loop Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 20/27] WIP: fftools/ffmpeg_enc: move encoding to a separate thread Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 21/27] WIP fftools/ffmpeg: add thread-aware transcode scheduling infrastructure Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 22/27] WIP fftools/ffmpeg_demux: convert to the scheduler Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 23/27] WIP fftools/ffmpeg_dec: " Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 24/27] WIP fftools/ffmpeg_filter: " Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 25/27] WIP fftools/ffmpeg_enc: " Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 26/27] WIP fftools/ffmpeg_mux: " Anton Khirnov 2023-09-19 19:10 ` [FFmpeg-devel] [PATCH 27/27] WIP: ffmpeg: switch to scheduler Anton Khirnov 2023-09-20 14:49 ` [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading Michael Niedermayer 2023-09-25 10:17 ` Anton Khirnov
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel This inbox may be cloned and mirrored by anyone: git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \ ffmpegdev@gitmailbox.com public-inbox-index ffmpegdev Example config snippet for mirrors. AGPL code for this site: git clone https://public-inbox.org/public-inbox.git