From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 46D3B47B19 for ; Mon, 2 Oct 2023 12:10:14 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id ABBAA68CDF8; Mon, 2 Oct 2023 15:10:12 +0300 (EEST) Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id F0F8E68CDE3 for ; Mon, 2 Oct 2023 15:10:06 +0300 (EEST) Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-1c3bd829b86so131269735ad.0 for ; Mon, 02 Oct 2023 05:10:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696248605; x=1696853405; darn=ffmpeg.org; h=content-transfer-encoding:in-reply-to:autocrypt:from :content-language:references:to:subject:user-agent:mime-version:date :message-id:from:to:cc:subject:date:message-id:reply-to; bh=CJO9uTrBUAyzA9O/YaAp2lVTBuSC+vE1ORRapQFFV8A=; b=CMkviKpXDRiJcvfOo81rny2VFJsjcWbazpswYV85SxQ37bWxpIzET47TnVfCCAGwIy wtUcVrgeDBmXEE85IHikqgHmgC83A8dW/8zpkEgzvD8Q/vAHPPITHPrvdgI1J/P6ma6L oo39YHYEEx+ZZ/7afn1hYGrphrb6/x4SzwY18xb1NHtqNlmB8VOMBiHlVRILYPIjuP31 lnTLKuTfrIdDto4O2aQ8J2YupmfH2ECRkowJV6V5v9AzVo/ce7PKbVDvQNvi/yBQ+gGy 0KK2y2si8mePGgIxXej3Fgp0UfmIrwJIMS1tw+uFjJ1CLwRs4qMbX5BoI1l77BKULCX5 aofg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696248605; x=1696853405; h=content-transfer-encoding:in-reply-to:autocrypt:from :content-language:references:to:subject:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=CJO9uTrBUAyzA9O/YaAp2lVTBuSC+vE1ORRapQFFV8A=; b=URekq8P7o2DXgPa52TP7hrxRK8f30TYLp7yofRyX9dW9UFObpO+/NzUZ1FC6W84T3o a+FVOwB8zHTrRgNKDsIbrMAX63awdIKFN+cxwwyjKYuKd+8jDJslay2in3dYjHgew+m0 3h6evx2qjnlyriUTuwNMUXZYITJmpHOr0jnYgRhIU8NH6C+MVIjo67h2B0dzqV0HDZhW goWBFokR47XgP4axT0FBMIYx5lodS5McT1jY/FCVs3euCih4veAfwKc7N50JXOUJBvFq Zt1cYte8lEQCNiASMhr8CtjAvWYC6H9GPGo016lM3LI9PzWuYH8w2VfwqXLgZUTdRuVC p0Tw== X-Gm-Message-State: AOJu0Ywv4KBGZivacepzh0s6q77jPocHD1Y8Fmb8+6H+2s5Xk4koDXJh f7BPHXa2WOECR2fTvAhUD6eO8hW3i5g= X-Google-Smtp-Source: AGHT+IHbecBcE2PO3kA7zZblDT2/kTb7uDoV1F6XrR6k20I5kn9MELELfYw9zrW8S+VKTUsqPr7fpA== X-Received: by 2002:a17:902:ea0e:b0:1c5:fa2d:b251 with SMTP id s14-20020a170902ea0e00b001c5fa2db251mr11698667plg.5.1696248604561; Mon, 02 Oct 2023 05:10:04 -0700 (PDT) Received: from [192.168.0.10] (host197.190-225-105.telecom.net.ar. [190.225.105.197]) by smtp.gmail.com with ESMTPSA id iz2-20020a170902ef8200b001c5bcc9d916sm4754164plb.176.2023.10.02.05.10.02 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 02 Oct 2023 05:10:03 -0700 (PDT) Message-ID: <40fa74ce-1d31-4f06-9352-436fdd1c2bef@gmail.com> Date: Mon, 2 Oct 2023 09:10:11 -0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: ffmpeg-devel@ffmpeg.org References: <20230906143832.54604-1-jamrial@gmail.com> <169623945333.6638.1744815521200399312@lain.khirnov.net> Content-Language: en-US From: James Almer Autocrypt: addr=jamrial@gmail.com; keydata= xsBNBFjZtqABCADLW+vdEoZaJZDsIO6geYFTOcn1unsEHefj9zn+3oTHlDFFzO47mzHsSfbK 9JE2xpOJEVnC8FAF5Sayi/pVwV+mtQUV3n5dgVeVBYF9GUQwOGFCpK8X54RRqhkgknbunOEE 0CtgAJgmpFmmmHgq02GvEspx1h/rh4apqwQR6QX4Favb+x9+i9ytVpwVcBX94vo2toyP7h/K BWfadQmb8ltgE1kshfg+SQs/H5bTV5Z1DuEASf02ZL/1qYB/sdTgWPLv9XMUHHsRFmMY8TMx wJSkP+Af3AiYQPJYz1B1D4tt98T/NoiVdin10zATakPjV8hXaobuRmxgakkUASXudydDABEB AAHNH0phbWVzIEFsbWVyIDxqYW1yaWFsQGdtYWlsLmNvbT7CwJIEEwEIADwCGwMGCwkIBwMC BhUIAgkKCwQWAgMBAh4BAheAFiEEd1EujP2UoWlX5pp6FGMBrXN2WeAFAmJoLUUCGQEACgkQ FGMBrXN2WeAFVQf9GtGhniRs1PzNUOgJktCnv6j4BbLieaIPYPEFXKDHOgjqQE2zVMYXnoXl Jam928ii902a8OY06r9ywn/R8ApD1/3NY/v64O71CY9scz5XyH2au8wIZ6HwFy3/f7sqjdGD uctY8Qs7rjT7NkoC5lmgMu2v2k03dGtM9AAf5AK5gU+H0EUw7vmKKiXzUqt5kvBuf4CEwXvH AQT1SMJ52rIlDWB7FQFyZeUbOAK2IgY/KNedfK6nsgd/eQVnlofPd2XoddE7kP6iys7jJefw DD3g3rZyDTq7in5dyk5glaNpWZpbHGBs+9SCYLnfQ8XvWqPFOD+gj0plamKANgOvavKTxM7A TQRY2bagAQgA69YtILj8kYxmqPr/M8+MXT7wVoOWVW9lvSmPquCELaDy/NIS7D06VC5EuE/6 JlJXZMTn37NLlyWhzwOgXuXw5w2tyoQQBuvqGiXJijuXwXH7HKdzrc6rpYtAqt5w05hzNrFS KrS0izG64VpWrfproy3BsL+8TBm9brLhhNPynVRqVukbbGzlATTzNQGZ14TTi2/dL6DkMQnM qn4jX9UEe4GdGQBP50bUJSSmeiIkyNLWA+znuN2PZEz930ZwNrF9GtDVw7mzcmpCZ7spldE2 tutbpy9D1bIqxyqBrYDSezyzL2adR1qgHyOTMCHg2AYNkrIQHrSyJxKTpZ1/hqOp8wARAQAB wsBfBBgBAgAJBQJY2bagAhsMAAoJEBRjAa1zdlnghekH/0Yb0iYJ74oID2f/Fj+AJKS2ekQF P2xOr8lpGzgp/+yWUvPtqbX0A33anBJdYwxaAC0NataX3tfZ+oJkzXqfmqhIHMPYHdZesJA2 Bk9hU/33mDl5s5U66/z0uelWzwKVHoQ2O6or4+qF3HJFSJLCe9uvWJ3zXf9F342Ftj73sfx+ 3xkw/IXsN1RqbYqDlzpoEQ99SIEfY/8Jjwnd3sIPfqkuyeaYfe6GJDqKawdCEP1oRRlbXEAp TJgYz8r3nPhGv9cdHNDCk44ISbsqVuxIEnLqi4fTPZaGupiQhT+srl268TTAp2TQW7+6Ce/b NPQorMquzS/LZoyALpmsYi/miMc= In-Reply-To: <169623945333.6638.1744815521200399312@lain.khirnov.net> Subject: Re: [FFmpeg-devel] [PATCH] [RFC]avformat: introduce AVStreamGroup X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On 10/2/2023 6:37 AM, Anton Khirnov wrote: > Quoting James Almer (2023-09-06 16:38:32) >> Signed-off-by: James Almer >> --- >> This is an initial proof of concept for AVStream groups, something that's >> needed for quite a few existing and upcoming formats that lavf has no way to >> currently export. Said formats define a single video or audio stream composed >> by merging several individualy multiplexed streams within a media file. >> This is the case of HEIF, a format defining a tiled image where each tile is a >> separate image (either hevc, av1, etc) all of which need to be decoded >> individualy and then stitched together for presentation using container level >> information; IAMF, a new audio format where several individual streams (mono or >> stereo) need to be decoded individually and then combined to form audio streams >> with different channel layouts; and MPEG-TS programs, currently exported as >> AVProgram, which this new general purpose API would replace. >> There may be others too, like something ISOBMFF specific and not HEIF related, >> I'm told. >> >> A new struct, AVStreamGroup, would cover all these cases by grouping the >> relevant streams and propagating format specific metadata for the purpose of >> combining them into what's expected for presentation (Something a filter for >> example would have to take care of). >> >> Missing from this first version is something like a type field, which could be >> an enum listing the different currently known formats the user would then use >> to interpret the attached metadata, defined perhaps in codecpar.extradata >> >> I'd like to hear opinions and suggestions to improve and properly handle this. >> >> libavformat/avformat.c | 26 ++++++++ >> libavformat/avformat.h | 143 ++++++++++++++++++++++++++++++++++++++++- >> libavformat/dump.c | 65 +++++++++++++++++-- >> libavformat/internal.h | 28 ++++++++ >> libavformat/options.c | 77 ++++++++++++++++++++++ >> 5 files changed, 332 insertions(+), 7 deletions(-) >> > >> diff --git a/libavformat/avformat.h b/libavformat/avformat.h >> index 1916aa2dc5..d18eafb933 100644 >> --- a/libavformat/avformat.h >> +++ b/libavformat/avformat.h >> @@ -1007,6 +1007,59 @@ typedef struct AVStream { >> int pts_wrap_bits; >> } AVStream; >> >> +typedef struct AVStreamGroup { >> + /** >> + * A class for @ref avoptions. Set on stream creation. > ^^^^^^ > group > >> + */ >> + const AVClass *av_class; >> + >> + /** >> + * Group index in AVFormatContext. >> + */ >> + int index; > > unsigned? Made it int to have it consistent with AVStream, but ok. > >> + >> + /** >> + * Format-specific group ID. >> + * decoding: set by libavformat >> + * encoding: set by the user, replaced by libavformat if left unset >> + */ >> + int id; > > might want to make this 64bit Ok. > >> + >> + /** >> + * Codec parameters associated with this stream group. Allocated and freed >> + * by libavformat in avformat_new_stream_group() and avformat_free_context() >> + * respectively. >> + * >> + * - demuxing: filled by libavformat on stream group creation or in >> + * avformat_find_stream_info() >> + * - muxing: filled by the caller before avformat_write_header() >> + */ >> + AVCodecParameters *codecpar; >> + >> + void *priv_data; > > Do we really need this? It's a single pointer, and some demuxers may actually make use of it, like they do for the AVStream's counterpart. A git grep "st->priv_data" has a lot of hits. > >> + >> + /** >> + * Number of elements in AVStreamGroup.stream_index. >> + * >> + * Set by av_stream_group_add_stream() and av_stream_group_new_stream(), must not >> + * be modified by any other code. >> + */ >> + int nb_stream_indexes; >> + >> + /** >> + * A list of indexes of streams in the group. New entries are created with >> + * av_stream_group_add_stream() and av_stream_group_new_stream(). >> + * >> + * - demuxing: entries are created by libavformat in avformat_open_input(). >> + * If AVFMTCTX_NOHEADER is set in ctx_flags, then new entries may also >> + * appear in av_read_frame(). >> + * - muxing: entries are created by the user before avformat_write_header(). >> + * >> + * Freed by libavformat in avformat_free_context(). >> + */ >> + int *stream_index; > > unsigned for both? Ok. > >> @@ -1844,6 +1940,51 @@ const AVClass *av_stream_get_class(void); >> */ >> AVStream *avformat_new_stream(AVFormatContext *s, const AVCodec *c); >> >> +/** >> + * Add a new stream to a stream group. >> + * >> + * When demuxing, it may be called by the demuxer in read_header(). If the >> + * flag AVFMTCTX_NOHEADER is set in s.ctx_flags, then it may also >> + * be called in read_packet(). >> + * >> + * When muxing, may be called by the user before avformat_write_header() after >> + * having allocated a new group with avformat_new_stream_group(). >> + * >> + * User is required to call avformat_free_context() to clean up the allocation >> + * by av_stream_group_new_stream(). >> + * >> + * This is functionally the same as avformat_new_stream() while also adding the >> + * newly allocated stream to the group belonging to the media file. >> + * >> + * @param stg stream group belonging to a media file. >> + * >> + * @return newly created stream or NULL on error. >> + * @see av_stream_group_add_stream, avformat_new_stream_group. >> + */ >> +AVStream *av_stream_group_new_stream(AVStreamGroup *stg); > > Is there a big enough advantage to having this as a separate function? I figured it would be nice to have it for the sake of convenience. For formats like HEIF, new streams should in theory be created once the group they will belong to is known. I have no strong attachment to this function, so it can go if you think it's superfluous. > >> + >> +/** >> + * Add an already allocated stream to a stream group. >> + * >> + * When demuxing, it may be called by the demuxer in read_header(). If the >> + * flag AVFMTCTX_NOHEADER is set in s.ctx_flags, then it may also >> + * be called in read_packet(). >> + * >> + * When muxing, may be called by the user before avformat_write_header() after >> + * having allocated a new group with avformat_new_stream_group() and stream with >> + * avformat_new_stream(). >> + * >> + * User is required to call avformat_free_context() to clean up the allocation >> + * by av_stream_group_add_stream(). >> + * >> + * @param stg stream group belonging to a media file. >> + * @param st stream in the media file to add to the group. >> + * >> + * @return 0 on success, or a negative AVERROR otherwise. >> + * @see avformat_new_stream, av_stream_group_new_stream, avformat_new_stream_group. >> + */ >> +int av_stream_group_add_stream(AVStreamGroup *stg, const AVStream *st); > > It'd be nice to have the streamgroup-related functions consistenly > namespaced. > > E.g. > * avformat_stream_group_add() > * avformat_stream_group_add_stream() > * ff_stream_group_free() > etc. > > alternatively for the first two: > * avformat_stream_group_create() > * avformat_stream_group_extend() I named avformat_new_stream_group() essentially the same as avformat_new_stream(), then namespaced the functions that take a AVStreamGroup as input. I don't particularly like _extend(), but i guess i could do something like AVStreamGroup *avformat_stream_group_create(AVFormatContext *s) int avformat_stream_group_add_stream(AVStreamGroup *stg, const AVStream *st); _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".