From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 15E3B47249 for ; Mon, 2 Oct 2023 09:37:42 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BFED568CC4A; Mon, 2 Oct 2023 12:37:40 +0300 (EEST) Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 68ABF68CAFF for ; Mon, 2 Oct 2023 12:37:34 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 218BB240490 for ; Mon, 2 Oct 2023 11:37:34 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavis, port 10024) with ESMTP id OjAG8TekpfCM for ; Mon, 2 Oct 2023 11:37:33 +0200 (CEST) Received: from lain.khirnov.net (lain.khirnov.net [IPv6:2001:67c:1138:4306::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "lain.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id 6ADAF240445 for ; Mon, 2 Oct 2023 11:37:33 +0200 (CEST) Received: by lain.khirnov.net (Postfix, from userid 1000) id 567E91601B9; Mon, 2 Oct 2023 11:37:33 +0200 (CEST) From: Anton Khirnov To: FFmpeg development discussions and patches In-Reply-To: <20230906143832.54604-1-jamrial@gmail.com> References: <20230906143832.54604-1-jamrial@gmail.com> Mail-Followup-To: FFmpeg development discussions and patches Date: Mon, 02 Oct 2023 11:37:33 +0200 Message-ID: <169623945333.6638.1744815521200399312@lain.khirnov.net> User-Agent: alot/0.8.1 MIME-Version: 1.0 Subject: Re: [FFmpeg-devel] [PATCH] [RFC]avformat: introduce AVStreamGroup X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Quoting James Almer (2023-09-06 16:38:32) > Signed-off-by: James Almer > --- > This is an initial proof of concept for AVStream groups, something that's > needed for quite a few existing and upcoming formats that lavf has no way to > currently export. Said formats define a single video or audio stream composed > by merging several individualy multiplexed streams within a media file. > This is the case of HEIF, a format defining a tiled image where each tile is a > separate image (either hevc, av1, etc) all of which need to be decoded > individualy and then stitched together for presentation using container level > information; IAMF, a new audio format where several individual streams (mono or > stereo) need to be decoded individually and then combined to form audio streams > with different channel layouts; and MPEG-TS programs, currently exported as > AVProgram, which this new general purpose API would replace. > There may be others too, like something ISOBMFF specific and not HEIF related, > I'm told. > > A new struct, AVStreamGroup, would cover all these cases by grouping the > relevant streams and propagating format specific metadata for the purpose of > combining them into what's expected for presentation (Something a filter for > example would have to take care of). > > Missing from this first version is something like a type field, which could be > an enum listing the different currently known formats the user would then use > to interpret the attached metadata, defined perhaps in codecpar.extradata > > I'd like to hear opinions and suggestions to improve and properly handle this. > > libavformat/avformat.c | 26 ++++++++ > libavformat/avformat.h | 143 ++++++++++++++++++++++++++++++++++++++++- > libavformat/dump.c | 65 +++++++++++++++++-- > libavformat/internal.h | 28 ++++++++ > libavformat/options.c | 77 ++++++++++++++++++++++ > 5 files changed, 332 insertions(+), 7 deletions(-) > > diff --git a/libavformat/avformat.h b/libavformat/avformat.h > index 1916aa2dc5..d18eafb933 100644 > --- a/libavformat/avformat.h > +++ b/libavformat/avformat.h > @@ -1007,6 +1007,59 @@ typedef struct AVStream { > int pts_wrap_bits; > } AVStream; > > +typedef struct AVStreamGroup { > + /** > + * A class for @ref avoptions. Set on stream creation. ^^^^^^ group > + */ > + const AVClass *av_class; > + > + /** > + * Group index in AVFormatContext. > + */ > + int index; unsigned? > + > + /** > + * Format-specific group ID. > + * decoding: set by libavformat > + * encoding: set by the user, replaced by libavformat if left unset > + */ > + int id; might want to make this 64bit > + > + /** > + * Codec parameters associated with this stream group. Allocated and freed > + * by libavformat in avformat_new_stream_group() and avformat_free_context() > + * respectively. > + * > + * - demuxing: filled by libavformat on stream group creation or in > + * avformat_find_stream_info() > + * - muxing: filled by the caller before avformat_write_header() > + */ > + AVCodecParameters *codecpar; > + > + void *priv_data; Do we really need this? > + > + /** > + * Number of elements in AVStreamGroup.stream_index. > + * > + * Set by av_stream_group_add_stream() and av_stream_group_new_stream(), must not > + * be modified by any other code. > + */ > + int nb_stream_indexes; > + > + /** > + * A list of indexes of streams in the group. New entries are created with > + * av_stream_group_add_stream() and av_stream_group_new_stream(). > + * > + * - demuxing: entries are created by libavformat in avformat_open_input(). > + * If AVFMTCTX_NOHEADER is set in ctx_flags, then new entries may also > + * appear in av_read_frame(). > + * - muxing: entries are created by the user before avformat_write_header(). > + * > + * Freed by libavformat in avformat_free_context(). > + */ > + int *stream_index; unsigned for both? > @@ -1844,6 +1940,51 @@ const AVClass *av_stream_get_class(void); > */ > AVStream *avformat_new_stream(AVFormatContext *s, const AVCodec *c); > > +/** > + * Add a new stream to a stream group. > + * > + * When demuxing, it may be called by the demuxer in read_header(). If the > + * flag AVFMTCTX_NOHEADER is set in s.ctx_flags, then it may also > + * be called in read_packet(). > + * > + * When muxing, may be called by the user before avformat_write_header() after > + * having allocated a new group with avformat_new_stream_group(). > + * > + * User is required to call avformat_free_context() to clean up the allocation > + * by av_stream_group_new_stream(). > + * > + * This is functionally the same as avformat_new_stream() while also adding the > + * newly allocated stream to the group belonging to the media file. > + * > + * @param stg stream group belonging to a media file. > + * > + * @return newly created stream or NULL on error. > + * @see av_stream_group_add_stream, avformat_new_stream_group. > + */ > +AVStream *av_stream_group_new_stream(AVStreamGroup *stg); Is there a big enough advantage to having this as a separate function? > + > +/** > + * Add an already allocated stream to a stream group. > + * > + * When demuxing, it may be called by the demuxer in read_header(). If the > + * flag AVFMTCTX_NOHEADER is set in s.ctx_flags, then it may also > + * be called in read_packet(). > + * > + * When muxing, may be called by the user before avformat_write_header() after > + * having allocated a new group with avformat_new_stream_group() and stream with > + * avformat_new_stream(). > + * > + * User is required to call avformat_free_context() to clean up the allocation > + * by av_stream_group_add_stream(). > + * > + * @param stg stream group belonging to a media file. > + * @param st stream in the media file to add to the group. > + * > + * @return 0 on success, or a negative AVERROR otherwise. > + * @see avformat_new_stream, av_stream_group_new_stream, avformat_new_stream_group. > + */ > +int av_stream_group_add_stream(AVStreamGroup *stg, const AVStream *st); It'd be nice to have the streamgroup-related functions consistenly namespaced. E.g. * avformat_stream_group_add() * avformat_stream_group_add_stream() * ff_stream_group_free() etc. alternatively for the first two: * avformat_stream_group_create() * avformat_stream_group_extend() -- Anton Khirnov _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".