From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id B26AD49FDD for ; Tue, 19 Mar 2024 11:06:13 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C128E68D35D; Tue, 19 Mar 2024 13:06:10 +0200 (EET) Received: from mail-ed1-f47.google.com (mail-ed1-f47.google.com [209.85.208.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BF98368D303 for ; Tue, 19 Mar 2024 13:06:04 +0200 (EET) Received: by mail-ed1-f47.google.com with SMTP id 4fb4d7f45d1cf-56b857bac38so766521a12.0 for ; Tue, 19 Mar 2024 04:06:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710846364; x=1711451164; darn=ffmpeg.org; h=user-agent:in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date:from:to:cc :subject:date:message-id:reply-to; bh=iXLTh/hRgFFkwUKGlJ64WsB+Wl88VxPeXCpYuQPxXoc=; b=Cuu/SEQS2jIPhnkpDHVdWfd4mNc2/Z7WM2tKtfpYBHtHUIDts1Rl4Ogxws9+bC9B+r Pqhs1sRhV1k+CgzOc895W8cyQNAKDcYF4AwODXHDDbx4pGqHn8NMmbP3oShFneZQfkeJ PHiUKdgqYO3Zl8vioNjB7TOKTPJgmBzt8x+BhXzQdNABJLLmyVYkqyXKknSawBiz7FcW 1J+PJWEqCpwgFmye2rsRQ5iwivZh/uCKwSvgr7AaED+4AajQTp1tIgoVJ9jAwEVf8SVx VXGFEwVAriNzB7jXx3N50Vnt99e0uwNrMV/gfCjNd6vzbSskDlFCp1vciHoznCUItrYk UobA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710846364; x=1711451164; h=user-agent:in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=iXLTh/hRgFFkwUKGlJ64WsB+Wl88VxPeXCpYuQPxXoc=; b=d67sHAGq5VcMoIurRbrV6FZiTjEU4Q8ydUCpST6Jx12WoHj3PQjT0J2qVzoTVJ8UUx PFpwXIbBIgJUG3nngc5HPwj1qaqbe/u9+XTRDcz7Y8l85vzyFADfIuIkfHXZY0w5rZ5i WKSCVKVDgMd7l1fwQCzrOLyUd/bChqMesZ1GGabji4tObRlr/ed/eiJtUmBBHRMkmHS2 BrXyb2DF40FoXFExPjxcbUqPG/jjxY0x8xpbnhPrw6AOmMF6StyvblcWpoqazdmt3TQA v5s7AKlOB3vCpGEx44mHVUYAF9/bqca9ftehK5XM5p9JzuWFld7qZBmfWb2pyn9aSZZi JRpg== X-Gm-Message-State: AOJu0Yxbrfg/eyzIvNwRB5+XOv+hpxQ+8i3YjZDwqQkdlj22awjKpyL5 x1M6RtkWsx1324ufWH5mkrfDLI/4Mpwyt09bFC/lK+TVV17nRUMTTt0Nhhbl X-Google-Smtp-Source: AGHT+IGNetc15XTX8O/+518xAWpJ27ewcVQXXHiqxqWE4Pgtp81yc0kY6bFdt8zr4DlEyMBN5Rx+MA== X-Received: by 2002:a05:6402:448c:b0:568:1599:b854 with SMTP id er12-20020a056402448c00b005681599b854mr6401228edb.42.1710846363492; Tue, 19 Mar 2024 04:06:03 -0700 (PDT) Received: from mariano ([188.210.239.9]) by smtp.gmail.com with ESMTPSA id fg9-20020a056402548900b00568d3cae00dsm2527281edb.94.2024.03.19.04.06.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Mar 2024 04:06:03 -0700 (PDT) Received: by mariano (Postfix, from userid 1000) id 44463BFCDC; Tue, 19 Mar 2024 12:06:01 +0100 (CET) Date: Tue, 19 Mar 2024 12:06:01 +0100 From: Stefano Sabatini To: FFmpeg development discussions and patches Message-ID: Mail-Followup-To: FFmpeg development discussions and patches , Nicolas Gaullier References: <20240311155719.1874201-1-nicolas.gaullier@cji.paris> <20240311155719.1874201-2-nicolas.gaullier@cji.paris> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240311155719.1874201-2-nicolas.gaullier@cji.paris> User-Agent: Mutt/2.1.4 (2021-12-11) Subject: Re: [FFmpeg-devel] [PATCH v3 1/1] avformat/demux: Add durationprobesize AVOption X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nicolas Gaullier Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Sorry for the slow replay. On date Monday 2024-03-11 16:57:19 +0100, Nicolas Gaullier wrote: > Yet another probesize used to get the durations when > estimate_timings_from_pts is required. It is aimed at users interested > in better durations probing for itself, or because using > avformat_find_stream_info indirectly and requiring exact values: for > concatdec for example, especially if streamcopying above it. > The current code is a performance trade-off that can fail to get video > stream durations in a scenario with high bitrates and buffering for > files ending cleanly (as opposed to live captures): the physical gap > between the last video packet and the last audio packet is very high in > such a case. > > Default behaviour is unchanged: 250k up to 250k << 6 (step by step). > Setting this new option has two effects: > - override the maximum probesize (currently 250k << 6) > - reduce the number of steps to 1 instead of 6, this is to avoid > detecting the audio "too early" and failing to reach a video packet. > Even if a single audio stream duration is found but not the other > audio/video stream durations, there will be a retry, so at the end the > full user-overriden probesize will be used as expected by the user. > > Signed-off-by: Nicolas Gaullier > --- > doc/APIchanges | 3 +++ > doc/formats.texi | 16 +++++++++++++++- > libavformat/avformat.h | 12 ++++++++++++ > libavformat/demux.c | 13 ++++++++----- > libavformat/options_table.h | 1 + > libavformat/version.h | 2 +- > 6 files changed, 40 insertions(+), 7 deletions(-) > > diff --git a/doc/APIchanges b/doc/APIchanges > index cf58c8c5f0..c97acf60d1 100644 > --- a/doc/APIchanges > +++ b/doc/APIchanges > @@ -2,6 +2,9 @@ The last version increases of all libraries were on 2024-03-07 > > API changes, most recent first: > > +2024-03-11 - xxxxxxxxxx - lavf 61.1.100 - avformat.h > + Add AVFormatContext.duration_probesize nit++: missing dot at end of sentence > + > 2024-03-08 - xxxxxxxxxx - lavc 61.1.100 - avcodec.h > Add AVCodecContext.[nb_]side_data_prefer_packet. > > diff --git a/doc/formats.texi b/doc/formats.texi > index 69fc1457a4..9cada03a6e 100644 > --- a/doc/formats.texi > +++ b/doc/formats.texi > @@ -225,9 +225,23 @@ Specifies the maximum number of streams. This can be used to reject files that > would require too many resources due to a large number of streams. > > @item skip_estimate_duration_from_pts @var{bool} (@emph{input}) > -Skip estimation of input duration when calculated using PTS. > +Skip estimation of input duration if it requires an additional probing for pts at end of file. consistency: PTS > At present, applicable for MPEG-PS and MPEG-TS. > > +@item durationprobesize @var{integer} (@emph{input}) I'd go with duration_probesize, as it looks more readable (although we also have fpsprobesize), but most option names are splitting the name and this looks way more readable. > +Set probing size for input duration estimation when it actually requires an additional probing > +for pts at end of file. consistency: PTS also specify the measure unit of the value > +At present, applicable for MPEG-PS and MPEG-TS. I'm still confused why this should not be applicated to other formats as well. > +It is aimed at users interested in better durations probing for itself, or indirectly > +for specific use cases like using the concat demuxer. > +Files with high bitrates and ending cleanly (as opposed to live captures), can lead > +to a large physical gap between the last video packet and the last audio packet, > +so many bytes have to be read in order to get a video stream duration. > +Setting this value has a performance impact even for small files because the probing size is fixed. > +Default behaviour is a trade-off, largely adaptive: the probing size may range from > +250K up to 16M, but it is not extended to get streams durations at all costs. If this is bytes, then it might be confusing using K and M, maybe I'd use Ki and Mi to prevent the possible confusion. > +Must be an integer not lesser than 1, or 0 for default behaviour. > + > @item strict, f_strict @var{integer} (@emph{input/output}) > Specify how strictly to follow the standards. @code{f_strict} is deprecated and > should be used only via the @command{ffmpeg} tool. > diff --git a/libavformat/avformat.h b/libavformat/avformat.h > index de40397676..9042a62b70 100644 > --- a/libavformat/avformat.h > +++ b/libavformat/avformat.h > @@ -1667,6 +1667,8 @@ typedef struct AVFormatContext { > * Skip duration calcuation in estimate_timings_from_pts. > * - encoding: unused > * - decoding: set by user > + * > + * @sa duration_probesize nit: let's favor long forms to help readability (seealso or equivalent) > */ > int skip_estimate_duration_from_pts; > > @@ -1870,6 +1872,16 @@ typedef struct AVFormatContext { > * @return 0 on success, a negative AVERROR code on failure > */ > int (*io_close2)(struct AVFormatContext *s, AVIOContext *pb); > + > + /** > + * Maximum number of bytes read from input in order to determine stream durations > + * when using estimate_timings_from_pts in avformat_find_stream_info(). > + * Demuxing only, set by the caller before avformat_find_stream_info(). > + * Can be set to 0 to let avformat choose using a heuristic. > + * > + * @sa skip_estimate_duration_from_pts ditto > + */ > + int64_t duration_probesize; > } AVFormatContext; > > /** > diff --git a/libavformat/demux.c b/libavformat/demux.c > index 4c50eb5568..68b0c51720 100644 > --- a/libavformat/demux.c > +++ b/libavformat/demux.c > @@ -1817,8 +1817,9 @@ static void estimate_timings_from_bit_rate(AVFormatContext *ic) > "Estimating duration from bitrate, this may be inaccurate\n"); > } > > -#define DURATION_MAX_READ_SIZE 250000LL > -#define DURATION_MAX_RETRY 6 > +#define DURATION_MAX_READ_SIZE_DEFAULT 250000LL > +#define DURATION_MAX_RETRY_DEFAULT 6 > +#define DURATION_MAX_RETRY_USER 1 nit: DURATION_DEFAULT_MAX_READ_SIZE DURATION_DEFAULT_MAX_RETRY_DEFAULT DURATION_MAX_RETRY 1 > > /* only usable for MPEG-PS streams */ > static void estimate_timings_from_pts(AVFormatContext *ic, int64_t old_offset) > @@ -1826,6 +1827,8 @@ static void estimate_timings_from_pts(AVFormatContext *ic, int64_t old_offset) > FFFormatContext *const si = ffformatcontext(ic); > AVPacket *const pkt = si->pkt; > int num, den, read_size, ret; > + int64_t duration_max_read_size = ic->duration_probesize ? ic->duration_probesize >> DURATION_MAX_RETRY_USER : DURATION_MAX_READ_SIZE_DEFAULT; nit: I find the right shift followed by the leftshift a bit confusing, but probably there is no simple way to prevent it > + int duration_max_retry = ic->duration_probesize ? DURATION_MAX_RETRY_USER : DURATION_MAX_RETRY_DEFAULT; > int found_duration = 0; > int is_end; > int64_t filesize, offset, duration; > @@ -1861,7 +1864,7 @@ static void estimate_timings_from_pts(AVFormatContext *ic, int64_t old_offset) > filesize = ic->pb ? avio_size(ic->pb) : 0; > do { > is_end = found_duration; > - offset = filesize - (DURATION_MAX_READ_SIZE << retry); > + offset = filesize - (duration_max_read_size << retry); > if (offset < 0) > offset = 0; > > @@ -1870,7 +1873,7 @@ static void estimate_timings_from_pts(AVFormatContext *ic, int64_t old_offset) > for (;;) { > AVStream *st; > FFStream *sti; > - if (read_size >= DURATION_MAX_READ_SIZE << (FFMAX(retry - 1, 0))) > + if (read_size >= duration_max_read_size << (FFMAX(retry - 1, 0))) > break; > > do { > @@ -1924,7 +1927,7 @@ static void estimate_timings_from_pts(AVFormatContext *ic, int64_t old_offset) > } > } while (!is_end && > offset && > - ++retry <= DURATION_MAX_RETRY); > + ++retry <= duration_max_retry); > > av_opt_set_int(ic, "skip_changes", 0, AV_OPT_SEARCH_CHILDREN); > > diff --git a/libavformat/options_table.h b/libavformat/options_table.h > index b9dca147f9..0cc921a28d 100644 > --- a/libavformat/options_table.h > +++ b/libavformat/options_table.h > @@ -108,6 +108,7 @@ static const AVOption avformat_options[] = { > {"max_streams", "maximum number of streams", OFFSET(max_streams), AV_OPT_TYPE_INT, { .i64 = 1000 }, 0, INT_MAX, D }, > {"skip_estimate_duration_from_pts", "skip duration calculation in estimate_timings_from_pts", OFFSET(skip_estimate_duration_from_pts), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, D}, > {"max_probe_packets", "Maximum number of packets to probe a codec", OFFSET(max_probe_packets), AV_OPT_TYPE_INT, { .i64 = 2500 }, 0, INT_MAX, D }, > +{"durationprobesize", "maximum number of bytes to probe the durations of the streams in estimate_timings_from_pts", OFFSET(duration_probesize), AV_OPT_TYPE_INT64, {.i64 = 0 }, 0, INT64_MAX, D}, duration_probesize ? [...] Thanks _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".