Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
 help / color / mirror / Atom feed
From: Michael Niedermayer <michael@niedermayer.cc>
To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Subject: Re: [FFmpeg-devel] [PATCH 18/24] fftools/ffmpeg: add thread-aware transcode scheduling infrastructure
Date: Sat, 4 Nov 2023 19:44:09 +0100
Message-ID: <20231104184409.GI3543730@pb2> (raw)
In-Reply-To: <20231104092125.10213-19-anton@khirnov.net>


[-- Attachment #1.1: Type: text/plain, Size: 4462 bytes --]

On Sat, Nov 04, 2023 at 08:56:27AM +0100, Anton Khirnov wrote:
> See the comment block at the top of fftools/ffmpeg_sched.h for more
> details on what this scheduler is for.
> 
> This commit adds the scheduling code itself, along with minimal
> integration with the rest of the program:
> * allocating and freeing the scheduler
> * passing it throughout the call stack in order to register the
>   individual components (demuxers/decoders/filtergraphs/encoders/muxers)
>   with the scheduler
> 
> The scheduler is not actually used as of this commit, so it should not
> result in any change in behavior. That will change in future commits.
> ---
>  fftools/Makefile          |    1 +
>  fftools/ffmpeg.c          |   18 +-
>  fftools/ffmpeg.h          |   24 +-
>  fftools/ffmpeg_dec.c      |   10 +-
>  fftools/ffmpeg_demux.c    |   46 +-
>  fftools/ffmpeg_enc.c      |   13 +-
>  fftools/ffmpeg_filter.c   |   37 +-
>  fftools/ffmpeg_mux.c      |   17 +-
>  fftools/ffmpeg_mux.h      |   11 +
>  fftools/ffmpeg_mux_init.c |   82 +-
>  fftools/ffmpeg_opt.c      |   22 +-
>  fftools/ffmpeg_sched.c    | 2072 +++++++++++++++++++++++++++++++++++++
>  fftools/ffmpeg_sched.h    |  461 +++++++++
>  13 files changed, 2758 insertions(+), 56 deletions(-)
>  create mode 100644 fftools/ffmpeg_sched.c
>  create mode 100644 fftools/ffmpeg_sched.h

Is fate intended to pass for each point in the patchset ?

make -j32 fate-h264-skip-nointra
seems to fail after this

[h264 @ 0x5598133bccc0] non-existing PPS 0 referenced
    Last message repeated 5 times
[h264 @ 0x5598133bccc0] decode_slice_header error
[h264 @ 0x5598133bccc0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0x5598133bccc0] decode_slice_header error
[h264 @ 0x5598133bccc0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0x5598133bccc0] decode_slice_header error
[h264 @ 0x5598133bccc0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0x5598133bccc0] decode_slice_header error
[h264 @ 0x5598133bccc0] non-existing PPS 0 referenced
    Last message repeated 3 times
[h264 @ 0x5598133bccc0] mmco: unref short failure
    Last message repeated 1 times
[h264 @ 0x5598133bccc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
[mpegts @ 0x5598133b7080] PES packet size mismatch
[mpegts @ 0x5598133b7080] Packet corrupt (stream = 2, dts = 212088417).
[mpegts @ 0x5598133b7080] PES packet size mismatch
[mpegts @ 0x5598133b7080] Packet corrupt (stream = 1, dts = 212089857).
[mpegts @ 0x5598133b7080] PES packet size mismatch
[mpegts @ 0x5598133b7080] Packet corrupt (stream = 2, dts = 212088417).
[mpegts @ 0x5598133b7080] PES packet size mismatch
[mpegts @ 0x5598133b7080] Packet corrupt (stream = 1, dts = 212089857).
Input #0, mpegts, from 'fatesamples/fate/fate-suite//h264/h264_intra_first-small.ts':
  Duration: 00:00:02.48, start: 2355.577967, bitrate: 4961 kb/s
  Program 41
  Stream #0:0[0x12b5]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 50 tbr, 90k tbn
  Stream #0:1[0x12b4](swe): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 224 kb/s
  Stream #0:2[0x12b2](nor): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 224 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))
  Stream #0:1 -> #0:1 (mp2 (native) -> pcm_s16le (native))
[aost#0:1/pcm_s16le @ 0x5598134536c0] Too many packets buffered for output stream 0:1.
[aost#0:1/pcm_s16le @ 0x5598134536c0] Error submitting a packet to the muxer: No space left on deviceError while filtering: No space left on device
[out#0/framecrc @ 0x5598133bedc0] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=       0kB time=00:00:00.19 bitrate=   0.0kbits/s speed=68.4x
Conversion failed!
threads=1
tests/Makefile:307: recipe for target 'fate-h264-skip-nointra' failed
make: *** [fate-h264-skip-nointra] Error 228


[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Dictatorship: All citizens are under surveillance, all their steps and
actions recorded, for the politicians to enforce control.
Democracy: All politicians are under surveillance, all their steps and
actions recorded, for the citizens to enforce control.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

  parent reply	other threads:[~2023-11-04 18:44 UTC|newest]

Thread overview: 51+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-11-04  7:56 [FFmpeg-devel] [PATCH] ffmpeg CLI multithreading Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 01/24] lavf/mux: do not apply max_interleave_delta to subtitles Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 02/24] lavfi/af_amix: make sure the output does not depend on input ordering Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 03/24] lavc/8bps: fix exporting palette after 63767b79a570404628b2521b83104108b7b6884c Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 04/24] fftools/ffmpeg: move a few inline function into a new header Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 05/24] fftools/thread_queue: do not return elements for receive-finished streams Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 06/24] fftools/thread_queue: count receive-finished streams as finished Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 07/24] fftools/ffmpeg: rework keeping track of file duration for -stream_loop Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 08/24] fftools/ffmpeg_filter: remove an unnecessary sub2video_push_ref() call Anton Khirnov
2023-11-04 14:19   ` Nicolas George
2023-11-09 10:42     ` Anton Khirnov
2023-11-09 10:47       ` Nicolas George
2023-11-09 21:29         ` Anton Khirnov
2023-11-17  9:44           ` Nicolas George
2023-11-17 11:52             ` Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 09/24] fftools/ffmpeg_filter: track input/output index in {Input, Output}FilterPriv Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 10/24] fftools/ffmpeg_filter: move filtering to a separate thread Anton Khirnov
2023-11-04 19:54   ` Michael Niedermayer
2023-11-09 11:45     ` [FFmpeg-devel] [PATCH v2 " Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 11/24] fftools/ffmpeg_filter: buffer sub2video heartbeat frames like other frames Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 12/24] fftools/ffmpeg_filter: reindent Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 13/24] fftools/ffmpeg_mux: add muxing thread private data Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 14/24] fftools/ffmpeg_mux: move bitstream filtering to the muxer thread Anton Khirnov
2023-11-04 13:39   ` James Almer
2023-11-09 11:41     ` Anton Khirnov
2023-11-09 11:47       ` James Almer
2023-11-09 12:00         ` Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 15/24] fftools/ffmpeg_demux: switch from AVThreadMessageQueue to ThreadQueue Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 16/24] fftools/ffmpeg: disable -fix_sub_duration_heartbeat Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 17/24] fftools/ffmpeg_enc: move encoding to a separate thread Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 18/24] fftools/ffmpeg: add thread-aware transcode scheduling infrastructure Anton Khirnov
2023-11-04 13:53   ` James Almer
2023-11-09 11:06     ` Anton Khirnov
2023-11-04 18:44   ` Michael Niedermayer [this message]
2023-11-09 11:36     ` [FFmpeg-devel] [PATCH v2 " Anton Khirnov
2023-11-11 15:21       ` [FFmpeg-devel] [PATCH v3 " Anton Khirnov
2023-11-09 11:39     ` [FFmpeg-devel] [PATCH " Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 19/24] fftools/ffmpeg_demux: convert to the scheduler Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 20/24] fftools/ffmpeg_dec: " Anton Khirnov
2023-11-04 18:30   ` Michael Niedermayer
2023-11-11 15:24     ` Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 21/24] fftools/ffmpeg_filter: " Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 22/24] fftools/ffmpeg_enc: " Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 23/24] fftools/ffmpeg_mux: " Anton Khirnov
2023-11-04  7:56 ` [FFmpeg-devel] [PATCH 24/24] ffmpeg: switch to scheduler Anton Khirnov
2023-11-09 11:46 ` [FFmpeg-devel] [PATCH] ffmpeg CLI multithreading Anton Khirnov
2023-11-09 12:46   ` Paul B Mahol
2023-11-11 20:10 ` Michael Niedermayer
2023-11-17 10:46   ` Anton Khirnov
2023-11-13 12:34 ` Jan Ekström
2023-11-13 12:38   ` James Almer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20231104184409.GI3543730@pb2 \
    --to=michael@niedermayer.cc \
    --cc=ffmpeg-devel@ffmpeg.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
		ffmpegdev@gitmailbox.com
	public-inbox-index ffmpegdev

Example config snippet for mirrors.


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git