From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <ffmpeg-devel-bounces@ffmpeg.org>
Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100])
	by master.gitmailbox.com (Postfix) with ESMTP id DC4664767B
	for <ffmpegdev@gitmailbox.com>; Tue, 19 Sep 2023 19:21:15 +0000 (UTC)
Received: from [127.0.1.1] (localhost [127.0.0.1])
	by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0219568C917;
	Tue, 19 Sep 2023 22:21:00 +0300 (EEST)
Received: from mail1.khirnov.net (quelana.khirnov.net [94.230.150.81])
 by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1F82468C90F
 for <ffmpeg-devel@ffmpeg.org>; Tue, 19 Sep 2023 22:20:53 +0300 (EEST)
Received: from localhost (mail1.khirnov.net [IPv6:::1])
 by mail1.khirnov.net (Postfix) with ESMTP id BD40A5109
 for <ffmpeg-devel@ffmpeg.org>; Tue, 19 Sep 2023 21:20:52 +0200 (CEST)
Received: from mail1.khirnov.net ([IPv6:::1])
 by localhost (mail1.khirnov.net [IPv6:::1]) (amavis, port 10024) with ESMTP
 id CvdXv6Hn4mXz for <ffmpeg-devel@ffmpeg.org>;
 Tue, 19 Sep 2023 21:20:47 +0200 (CEST)
Received: from libav.khirnov.net (libav.khirnov.net
 [IPv6:2a00:c500:561:201::7])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "libav.khirnov.net",
 Issuer "smtp.khirnov.net SMTP CA" (verified OK))
 by mail1.khirnov.net (Postfix) with ESMTPS id 261055101
 for <ffmpeg-devel@ffmpeg.org>; Tue, 19 Sep 2023 21:20:42 +0200 (CEST)
Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1])
 by libav.khirnov.net (Postfix) with ESMTP id F2BD93A0212
 for <ffmpeg-devel@ffmpeg.org>; Tue, 19 Sep 2023 21:20:41 +0200 (CEST)
From: Anton Khirnov <anton@khirnov.net>
To: ffmpeg-devel@ffmpeg.org
Date: Tue, 19 Sep 2023 21:10:27 +0200
Message-Id: <20230919191044.18873-1-anton@khirnov.net>
X-Mailer: git-send-email 2.40.1
MIME-Version: 1.0
Subject: [FFmpeg-devel] [RFC/PATCH] ffmpeg CLI multithreading
X-BeenThere: ffmpeg-devel@ffmpeg.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: FFmpeg development discussions and patches <ffmpeg-devel.ffmpeg.org>
List-Unsubscribe: <https://ffmpeg.org/mailman/options/ffmpeg-devel>,
 <mailto:ffmpeg-devel-request@ffmpeg.org?subject=unsubscribe>
List-Archive: <https://ffmpeg.org/pipermail/ffmpeg-devel>
List-Post: <mailto:ffmpeg-devel@ffmpeg.org>
List-Help: <mailto:ffmpeg-devel-request@ffmpeg.org?subject=help>
List-Subscribe: <https://ffmpeg.org/mailman/listinfo/ffmpeg-devel>,
 <mailto:ffmpeg-devel-request@ffmpeg.org?subject=subscribe>
Reply-To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: ffmpeg-devel-bounces@ffmpeg.org
Sender: "ffmpeg-devel" <ffmpeg-devel-bounces@ffmpeg.org>
Archived-At: <https://master.gitmailbox.com/ffmpegdev/20230919191044.18873-1-anton@khirnov.net/>
List-Archive: <https://master.gitmailbox.com/ffmpegdev/>
List-Post: <mailto:ffmpegdev@gitmailbox.com>

Hi,
as some of you might know, I have been working on multithreading for the
ffmpeg CLI (as opposed to the libraries) for quite a while now. That
work is now finally entering the vicinity of being usable, so I've
decided to air it on the ML for comments on the big picture.

Do note that many things still do not work at all, notably
* subtitles
* bitstream filters
* streamloop
* filtergraphs with no inputs
Many others are probably broken to a larger or smaller extent.

I can, however, get some interesting results - e.g. significant
wallclock speedup in cases where singlethreaded codecs/filters can now
be run in parallel. I am also seeing 10-20% overall clock cycle increase
in my tests - for now that is a known bug that will hopefully be
addressed later.

One of the main themes of this work is separating various bits of
functionality in the code - that typically grew organically over years
(often in rather random places) - into clearly defined objects with
clearly defined responsibilities. At the core of the new world order is
the new scheduler object, living in ffmpeg_sched.[ch]. It handles all
interactions between the various ffmpeg components, inter-thread
synchronization, etc., so the individual components themselves do not
need to know anything about threading/synchronization/the rest of the
transcoding pipeline.

The patches are roughly grouped as follows:
* 01-13 Preparatory cleanup.
  The big thing here is moving fps conversion from video encoding to
  filtering (see commit message for 20/27 for details)
* 14-20 "Fake" threading for encoders/filters.
  This moves the last two unthreaded components - encoders and filters -
  into threads. As previously for decoders, this is merely preparatory,
  since the main thread waits while the encoder/filter thread does its
  work. The filter patch is unfortunately especially large, because
  filtering has multiple entrypoints and they all need to be wrapped and
  unwrapped for communication with the filtering thread.
  Some features that conflict with threading are also disabled in
  these patches. That will hopefully change later (though currently
  I don't see how subtitle heartbeat can work in its current form).
* 21 The scheduler itself, merely added without being used
* 22-27 Actually using the scheduler.
  This is split into multiple patches for clarity, but will be squashed
  in the final form.

You can fetch the code from the 'ffmpeg_threading' branch in
git://git.khirnov.net/libav. I will also present a short talk about this
work at VDD. Comments, questions, suggestions, etc. are very much
welcome, both here and there.

Cheers,
-- 
Anton Khirnov

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".