From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 0CBDE43351 for ; Wed, 1 Jun 2022 18:06:06 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7296668B79A; Wed, 1 Jun 2022 21:06:04 +0300 (EEST) Received: from mail-yb1-f173.google.com (mail-yb1-f173.google.com [209.85.219.173]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id EE0AD68B6EB for ; Wed, 1 Jun 2022 21:05:57 +0300 (EEST) Received: by mail-yb1-f173.google.com with SMTP id r82so4248771ybc.13 for ; Wed, 01 Jun 2022 11:05:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=opW+7YuHY/Q9oqN0x9JSOwWBlS/zotEDg+LGHvYSpXo=; b=hZuGF+jlml6IDr261DinWHU1X1Iem76JFHNi6yzAA3EmbpgMtcTbajfzklJqWpHgN4 u8F+QVOOZopV0OuD+C3P7o0Rdu/EJSSKu7ujR9mYVCspJWR/G0O9iczGtdmxnBjLRXjR cbqgDoMnAKyhR/EA2Au4HB8s0veGIWFgrrrOr9p5YK/tVG3l3yn9il6w8uVbu2Dh3xVN 7wQkAmgi1iRxifJAvBFQLxnZgvJVko50iPPZRpfPZ/0cF1gYC2/1EWO8AqlLRDiue0km +WL/CJzo3MU4b3ZkPpf19d+Mv49/hh9oSpx7GPARLXeuEGMFIIEmWPsZ/xQPK+mTdvvH saXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=opW+7YuHY/Q9oqN0x9JSOwWBlS/zotEDg+LGHvYSpXo=; b=7jWF1PsdQW2QIs7A+MejePkvsN+QFiZMlcCkh6FgT4Bb2jaAt/WAYa9nBw0HmY/Vb7 KzOWVNiB6D2qT67RRHaJJOfiuXitd2mdxFAPv+LNEn7Do6ezgAWnnbfmYY0CXVH9+kxF SEXb3BMAxbrqMQa1hb1zHYN13GtcW1qG+DoVM8cem20I5rSYzKHU/IB4TpK9hyAnpd9Z A7sMZ6c3XkO44yu56u+daf0KiSfNgcjf5L4wjoIyhh7tDzO7VQc688Fz6dWn5QyfMW6y Akk07BEibYjEEkq8ADwsU2G9HcaF6W1ZsyyB1iKHZURsTsmFwNGaMzaGE0rxE+02ByDC RpzQ== X-Gm-Message-State: AOAM532l79C93qVSZGlofQ+zyAOQUZq+OTSBvNa/8p8PsU3ocF43OpkR 5RZ/oct6iekKh6F8kAMs7IUWdqFREAyCs7YhbNdFmA== X-Google-Smtp-Source: ABdhPJyTbLHTn0Lj447sLKWQQAnXhuly0xV6i0NC8pN5nLH88zemn+oJWnAg0nirdPsB5LEWwidZQgzuGA3uhz0cT+Q= X-Received: by 2002:a5b:881:0:b0:65b:74dd:e36c with SMTP id e1-20020a5b0881000000b0065b74dde36cmr1067285ybq.172.1654106756124; Wed, 01 Jun 2022 11:05:56 -0700 (PDT) MIME-Version: 1.0 References: <20220518215620.1718203-1-vigneshv@google.com> In-Reply-To: From: Vignesh Venkatasubramanian Date: Wed, 1 Jun 2022 11:05:44 -0700 Message-ID: To: James Zern Subject: Re: [FFmpeg-devel] [PATCH] avformat/movenc: Support alpha channel for AVIF X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On Wed, Jun 1, 2022 at 10:52 AM James Zern wrote: > > On Wed, May 18, 2022 at 2:56 PM Vignesh Venkatasubramanian > wrote: > > > > AVIF specification allows for alpha channel as an auxillary item (in > > auxiliary > Done. > > case of still images) or as an auxillary track (in case of animated > > images). Add support for both of these. The AVIF muxer will take > > exactly two streams (when alpha is present) as input (first one being > > the YUV planes and the second one being the alpha plane). > > > > The input has to come from two different images (one of it color and > > the other one being alpha), or it can come from a single file > > source with the alpha channel extracted using the "alphaextract" > > filter. > > > > Example using alphaextract: > > ffmpeg -i rgba.png -filter_complex "[0:v]alphaextract[a]" -map 0 -map "[a]" -still-picture 1 avif_with_alpha.avif > > > > Example using two sources (first source can be in any pixel format and > > the second source has to be in monochrome grey pixel format): > > ffmpeg -i color.avif -i grey.avif -map 0 -map 1 -c copy avif_with_alpha.avif > > > > The generated files pass the compliance checks in Compliance Warden: > > https://github.com/gpac/ComplianceWarden > > > > libavif (the reference avif library) is able to decode the files > > generated using this patch. > > > > They also play back properly (with transparent background) in: > > 1) Chrome > > 2) Firefox (only still AVIF, no animation support) > > > > Signed-off-by: Vignesh Venkatasubramanian > > --- > > libavformat/movenc.c | 185 +++++++++++++++++++++++++++++-------------- > > libavformat/movenc.h | 4 +- > > 2 files changed, 128 insertions(+), 61 deletions(-) > > > > diff --git a/libavformat/movenc.c b/libavformat/movenc.c > > index de971f94e8..00e42b7abb 100644 > > --- a/libavformat/movenc.c > > +++ b/libavformat/movenc.c > > @@ -2852,7 +2852,7 @@ static int mov_write_hdlr_tag(AVFormatContext *s, AVIOContext *pb, MOVTrack *tra > > hdlr = (track->mode == MODE_MOV) ? "mhlr" : "\0\0\0\0"; > > if (track->par->codec_type == AVMEDIA_TYPE_VIDEO) { > > if (track->mode == MODE_AVIF) { > > - hdlr_type = "pict"; > > + hdlr_type = (track == &mov->tracks[0]) ? "pict" : "auxv"; > > descr = "PictureHandler"; > > } else { > > hdlr_type = "vide"; > > @@ -2940,57 +2940,83 @@ static int mov_write_iloc_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatConte > > avio_wb32(pb, 0); /* Version & flags */ > > avio_w8(pb, (4 << 4) + 4); /* offset_size(4) and length_size(4) */ > > avio_w8(pb, 0); /* base_offset_size(4) and reserved(4) */ > > - avio_wb16(pb, 1); /* item_count */ > > + avio_wb16(pb, s->nb_streams); /* item_count */ > > This assumes at most 2 streams for AVIF, the use of nb_streams here and > elsewhere carry any risk for other formats? > For AVIF, we check and ensure we only have at most 2 streams in mov_init. iloc box is written only for AVIF as of now. So there is no risk for other formats. > > [...] > > > > @@ -6874,14 +6920,23 @@ static int mov_init(AVFormatContext *s) > > > > /* AVIF output must have exactly one video stream */ > > This is no longer true. > Updated. > > if (mov->mode == MODE_AVIF) { > > - if (s->nb_streams > 1) { > > - av_log(s, AV_LOG_ERROR, "AVIF output requires exactly one stream\n"); > > + if (s->nb_streams > 2) { > > + av_log(s, AV_LOG_ERROR, "AVIF output requires exactly one or two streams\n"); > > return AVERROR(EINVAL); > > } > > - if (s->streams[0]->codecpar->codec_type != AVMEDIA_TYPE_VIDEO) { > > - av_log(s, AV_LOG_ERROR, "AVIF output requires one video stream\n"); > > + if (s->streams[0]->codecpar->codec_type != AVMEDIA_TYPE_VIDEO && > > + (s->nb_streams > 1 && s->streams[1]->codecpar->codec_type != AVMEDIA_TYPE_VIDEO)) { > > + av_log(s, AV_LOG_ERROR, "AVIF output supports only video streams\n"); > > return AVERROR(EINVAL); > > } > > + if (s->nb_streams > 1) { > > + const AVPixFmtDescriptor *pixdesc = > > + av_pix_fmt_desc_get(s->streams[1]->codecpar->format); > > + if (pixdesc->nb_components != 1) { > > + av_log(s, AV_LOG_ERROR, "Second stream for AVIF (alpha) output must have exactly one plane\n"); > > + return AVERROR(EINVAL); > > + } > > + } > > s->streams[0]->disposition |= AV_DISPOSITION_DEFAULT; > > } > > > > @@ -7543,18 +7598,25 @@ static int avif_write_trailer(AVFormatContext *s) > > { > > AVIOContext *pb = s->pb; > > MOVMuxContext *mov = s->priv_data; > > - int64_t pos_backup, mdat_pos; > > + int64_t pos_backup, extent_offsets[2]; > > uint8_t *buf; > > - int buf_size, moov_size; > > + int buf_size, moov_size, i; > > > > if (mov->moov_written) return 0; > > > > mov->is_animated_avif = s->streams[0]->nb_frames > 1; > > + if (mov->is_animated_avif && s->nb_streams > 1) { > > + // For animated avif with alpha channel, we need to write a the tref > > ...write a tref... Done. -- Vignesh _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".