From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 2D86E438BE for ; Sat, 2 Jul 2022 19:35:06 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C2A5A68B8C5; Sat, 2 Jul 2022 22:35:03 +0300 (EEST) Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9E96C68B844 for ; Sat, 2 Jul 2022 22:34:57 +0300 (EEST) Received: by mail-wr1-f51.google.com with SMTP id v14so7754150wra.5 for ; Sat, 02 Jul 2022 12:34:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=zD/8hZ7qLoohJAenPyIc+VC0crQHiDnJMbr5mCWEY4E=; b=J5Z8xxCAVYdUpQ7f/K5lR6djWZMtPfm6xTr3SbwCOhlroNnt47XzM8zoQa5UmGDQQ7 7q3WGk5apBnX28+10Obd0woVgZ8gyQG6nEqVgUgaS1M+gmdaSfGI6Y/TzZYxXYhTO11e GZpi6AtC1OgtxYIn5yf7krMYE8c22oXToIdYJiLGqbZo+LpQVdfuvnEXW6Gxhv4DvkD7 fTvKdC/Qa+r+Ua8NwMdP1eGZj/J1Y3LJ5YzntOftoPO2fTrbxhE7v7+9ML/CqiKEFF9V tIc/fWuWKmWxK+bhL+iXWpDJIITisI4hf0ienDGsoxKFVL0Tg0OorkOXUAbeKSxQHZiZ TS6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=zD/8hZ7qLoohJAenPyIc+VC0crQHiDnJMbr5mCWEY4E=; b=va2KGgEqIInlfwm0x7atQZQ/O2RKNJmSRk66FSKJjeUt2I2WJIOnn2eA9fTlFYDWHJ 7RuN4ugC7WH3dNoV40Z28Euww0TPDBFR8txiZ+C6kw0jXUTsM9sNuczHYSf6dlPo1w9U 98UA3Vx1hyQmyEYWY/qAXzjEpDUxx/yoGdQp7iF666AXRCnbJpkxSgc0HO6bdqsIGklC 8jo3w2vmmd9D6oYA0srGHJNREXmCO+HrG/sOtNOZ1RGKtHge9K3jOVUcEvRj/TyHOPn+ EqaQxBpBBE9O1Nj2d5BVULrtUeaTzHvDptDNCCXuw7mZ02Wk93lRu3Qaz/kWMWePfS5A NKMw== X-Gm-Message-State: AJIora94irO5qgal04TkKpBDhvUpSBIqLfc1BrW5GsfH4EdHIS7zREK0 dkzuAZbccMvbwyXSNclW+fBbKoKVuAt5zK6Jy0RZBmygT/Q= X-Google-Smtp-Source: AGRyM1u/is9lDkCuJa8lpqEbdZ5TKukBn76SPmMNyQT+NFkJJiuTQOTRBP/LlDRLmJXo1hC/Sgna3sDi3I6Ken1OxKI= X-Received: by 2002:a05:6000:1547:b0:21b:a2e2:6151 with SMTP id 7-20020a056000154700b0021ba2e26151mr19059159wry.481.1656790496994; Sat, 02 Jul 2022 12:34:56 -0700 (PDT) MIME-Version: 1.0 References: <20220630210434.1551769-1-vigneshv@google.com> <20220630210434.1551769-2-vigneshv@google.com> <165675449715.31466.4432489889968017112@lain.khirnov.net> In-Reply-To: From: =?UTF-8?B?SmFuIEVrc3Ryw7Zt?= Date: Sat, 2 Jul 2022 22:34:44 +0300 Message-ID: To: FFmpeg development discussions and patches Subject: Re: [FFmpeg-devel] [PATCH 2/2] avformat/mov: Support parsing of still AVIF Alpha Channel X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On Sat, Jul 2, 2022 at 7:32 PM Vignesh Venkatasubramanian wrote: > > On Sat, Jul 2, 2022 at 2:35 AM Anton Khirnov wrote: > > > > Quoting Vignesh Venkatasubramanian (2022-06-30 23:04:34) > > > Parse the alpha channel for still AVIF images and expose it as a > > > separate track. This is the simplest way of supporting AVIF alpha > > > channel in a codec independent manner (similar to how ffmpeg > > > supports animated AVIF with alpha channel). > > > > > > One can use the alphamerge filter to get a transparent image with > > > a single command. For example: > > > > > > ffmpeg -i image_with_alpha.avif -filter_complex alphamerge image_with_alpha.png > > > > > > Signed-off-by: Vignesh Venkatasubramanian > > > --- > > > libavformat/isom.h | 1 + > > > libavformat/mov.c | 66 ++++++++++++++++++++++++++++++++++++++++++++++ > > > 2 files changed, 67 insertions(+) > > > > I am against this patch, this is a digusting hack. > > > > These are not two streams, it is a single stream and should be exposed > > as such. > > > > Yes, while it is a hack, it is also the simplest way of supporting > AVIF images with alpha. Since AVIF alpha images require two separate > decoders, i could not think of any other solution that does not > involve modifying the underlying av1 codec itself. > > If there are any alternative solutions where we can expose a single > stream that needs two codecs, then i am open to implementing that. > > Otherwise, this solution improves the current situation where there is > no way for the user to extract the alpha channel (albeit in a hacky > way). I have discussed this on IRC, and as far as I can see we have multiple cases where an additional image has to be decoded by a separate decoder context to get alpha, namely: - vp8/9 (currently handled by AV_PKT_DATA_MATROSKA_BLOCKADDITIONAL) - HEIF-like formats (such as AVIF), which are codec agnostic (you can put JPEG, HEVC, AV1 etc into HEIF) This means the logic (preferably) should not be codec-specific, and would allow to implement at least two separate My proposal - which Anton didn't really like - was to add a side data entry AV_PKT_DATA_AUXILIARY_IMAGE , which could have type (ALPHA, DEPTH etc) as well as the relevant packet(s) - as well as if any of specifications define codec initialization data, that. Then libavcodec/decode or so could initialize a secondary decoder context for alpha images and add the third plane if parameters match what's required. This would also make remux possible, since containers that do not support such auxiliary images would ignore the alpha plane, and those that do would properly pass it through. As for encoding, not fully sure how it should be integrated, if any encoders actually at this moment do proper alpha coding, or do all API clients have to separately encode with one context the primary image, and the alpha with another? That said, the multi-stream implementation is already merged on the muxer side, so that can be utilized in the mean time, even though the auxiliary images are not technically a separate stream within HEIF-likes. Jan _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".