From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 51DC9461D3 for ; Wed, 6 Dec 2023 17:31:42 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1BC3768CF9B; Wed, 6 Dec 2023 19:31:40 +0200 (EET) Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2CCAE68CEED; Wed, 6 Dec 2023 19:31:28 +0200 (EET) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id BC8712405ED; Wed, 6 Dec 2023 18:31:27 +0100 (CET) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavis, port 10024) with ESMTP id MxWRRcNSYGmz; Wed, 6 Dec 2023 18:31:27 +0100 (CET) Received: from lain.khirnov.net (lain.khirnov.net [IPv6:2001:67c:1138:4306::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "lain.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id 3D60D2400AC; Wed, 6 Dec 2023 18:31:27 +0100 (CET) Received: by lain.khirnov.net (Postfix, from userid 1000) id 273151601B9; Wed, 6 Dec 2023 18:31:27 +0100 (CET) From: Anton Khirnov To: FFmpeg development discussions and patches , tc@ffmpeg.org In-Reply-To: <20231206103002.30084-1-anton@khirnov.net> References: <20231206103002.30084-1-anton@khirnov.net> Mail-Followup-To: FFmpeg development discussions and patches Date: Wed, 06 Dec 2023 18:31:27 +0100 Message-ID: <170188388713.8914.8464001514053857598@lain.khirnov.net> User-Agent: alot/0.8.1 MIME-Version: 1.0 Subject: Re: [FFmpeg-devel] [PATCH v3] ffmpeg CLI multithreading X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Hi all, As usual when someone disagrees with him, Nicolas converged to being utterly unreasonable and deaf to all arguments. I see no point in discussing this with him any further and intend to push the set tomorrow, unless somebody else has substantial objections. I've considered asking for a TC vote, but as he is not suggesting any viable alternative, there is really nothing to vote on. So the purpose of this email is just summarizing the dispute, so others can understand it more easily. The issue concerns sub2video code, which allows converting bitmap subtitles to video streams, mainly for hardsubbing purposes. As subtitles are typically sparse in time, the demuxer that produces the subtitle stream emits "heartbeat" frames that are sent to the filtering code just like real subtitle frames. The code in ffmpeg_filter.c then decides whether these heartbeat frames should be sent to the filtergraph or ignored. The problem is that this decision is currently made in a way that depends on what frames previously arrived on _other_ filtergraph inputs (e.g. on video frames in a graph with a subtitle and a video input). However, the inputs are not synchronized, and the interleaving of frames on different inputs is effectively arbitrary. E.g. it depends on the video decoder delay (and thus on the number of frame threads, when frame threading is used). The reason this arbitrariness has not become a major issue until now, is that it is deterministic for a given run on a given machine (sub2video FATE tests do not use a frame-threaded decoder, and so do not exhibit the problem). With ffmpeg CLI becoming fully parallel, the results become non-deterministic and change from run to run, which forces me to do something about this. My solution in patch 01/10 changes the filtering code to always send the heartbeat frames to the filtergraph. This not only makes the results deterministic, but also improves subtitle timing in FATE tests. Nicolas presented a testcase that consists of taking a video+subtitle streams from a single source, offsetting them against each other by a fixed delay, and overlaying subtitle onto video. After my patch, this results in the filtergraph buffering a number of heartbeat frames proportional to the offset, which causes higher memory consumption. However, * the testcase suffers from the above problem - its output can change significantly depending on the number of decoder frame threads; this is fixed by my patch; * the extra buffering added by the patch is similar to what would be added by the muxer interleaving queue, were the streams remuxed rather than overlaid; * the buffering can be avoided entirely by opening the input twice. I thus consider his argument (that the patch is "breaking" the testcase) invalid, as the testcase is * contrived; * already broken; * actually fixed by my patch. Nicolas has also NOT suggested any viable alternative approach. -- Anton Khirnov _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".