From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 489D2458B2 for ; Fri, 26 May 2023 20:55:37 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1040D68C1E8; Fri, 26 May 2023 23:55:20 +0300 (EEST) Received: from mail-yw1-f180.google.com (mail-yw1-f180.google.com [209.85.128.180]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7D6D168C1DB for ; Fri, 26 May 2023 23:55:11 +0300 (EEST) Received: by mail-yw1-f180.google.com with SMTP id 00721157ae682-561d1c4846cso2269757b3.1 for ; Fri, 26 May 2023 13:55:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685134510; x=1687726510; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1TWARR3bMptgM2dmljVXkiG0/oJEiycRqY5RDc3CiKo=; b=l3CtiN0PaT7w4b+hHwVSTFcUbbkc8yYjCPYpy0qpSSnXOXcx1Quc++dj0HIz8rv1OO c01Mgzv5QBDc5n3TzmyG3U79hA+DRumSIq8JKBaBJgjRKwvglmqGvdvgixSBl7FrdMT6 2DpN3033Lf46ZW4lr37waLZTmd5DL7rhqeBUwkGPi3HRGvxb39/pgTa8lnAdJ25jbMiX hrnvgmVyc026tpvD8vgBoneadneA6VFxLFD6XsZ6ZgP8VIhy4cMJuWB7rRf2hwjewzas ZyOk12JjeXOF+3SHwcIKDQTH1pLfPNacgJvo4ShF5lbmlp/W/En+Ag74r5ZtzQ1K2qoZ BnAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685134510; x=1687726510; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1TWARR3bMptgM2dmljVXkiG0/oJEiycRqY5RDc3CiKo=; b=YjWqcBLbLcyGREPnciYT3zkYG5lrXPhx9tSj16UDAE/ZHLTLAjMJw003WqKAl1CJS5 Pp4PKKQqg4hSortqrXl+d/NuXr0irIn9C5SjbvhrVQ5E5ludWRL7vo0RLUkLWjr8/NQP yns6mHkCDZLqxUluF80HcC/iqDvxtdYzDbDc5Bj9+E0XZT6WgQzEntqPl3MPFxos129/ 2CyV9k26qyKWB+eYlrAUonksF7i55PBpKDTAMqA+gmFv316dLdvGv7zf81BXL8UC29Af nxCBqgP5kuRSr3b2q1uLSHlnKuQWk5dfPSJsirxOjnFkoPazGWBhMPO95hRCQrZOyC7n yyNA== X-Gm-Message-State: AC+VfDx9Dnb/toOqnMfmrqMHN84pYgzuNWvKcN8rO0YyZIgi7F38vXJp LFQu644ScGfuOm69LislHflfNyTZOQY= X-Google-Smtp-Source: ACHHUZ4yLw203M77VVz/TjOVZQn9qSEaGHt24KGqvsFcB4+0+kZ+SvITGCGfcFCdym0rJ0OtxyHF+w== X-Received: by 2002:a81:1b97:0:b0:565:9e6f:8b50 with SMTP id b145-20020a811b97000000b005659e6f8b50mr2924709ywb.3.1685134510108; Fri, 26 May 2023 13:55:10 -0700 (PDT) Received: from gauss.local (c-98-224-219-15.hsd1.mi.comcast.net. [98.224.219.15]) by smtp.gmail.com with ESMTPSA id m186-20020a0dfcc3000000b0054bfc94a10dsm1457972ywf.47.2023.05.26.13.55.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 May 2023 13:55:09 -0700 (PDT) From: Leo Izen To: ffmpeg-devel@ffmpeg.org Date: Fri, 26 May 2023 16:55:05 -0400 Message-Id: <20230526205505.23569-3-leo.izen@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230526205505.23569-1-leo.izen@gmail.com> References: <20230526205505.23569-1-leo.izen@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 2/2] avformat/jpegxl_anim_dec: add animated JPEG XL demuxer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Leo Izen Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Animated JPEG XL files requires a separate demuxer than image2, because the timebase information is set by the demuxer. Should the timebase of an animated JPEG XL file be incompatible with the timebase set by the image2pipe demuxer (usually 1/25 unless set otherwise), rescaling will fail. Adding a separate demuxer for animated JPEG XL files allows the timebase to be set correctly. Signed-off-by: Leo Izen --- MAINTAINERS | 1 + libavformat/Makefile | 1 + libavformat/allformats.c | 1 + libavformat/img2dec.c | 2 +- libavformat/jpegxl_anim_dec.c | 266 ++++++++++++++++++++++++++++++++++ libavformat/jpegxl_probe.c | 19 +-- libavformat/jpegxl_probe.h | 7 +- libavformat/version.h | 2 +- 8 files changed, 287 insertions(+), 12 deletions(-) create mode 100644 libavformat/jpegxl_anim_dec.c diff --git a/MAINTAINERS b/MAINTAINERS index f95be01dc6..07852486e4 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -432,6 +432,7 @@ Muxers/Demuxers: ipmovie.c Mike Melanson ircam* Paul B Mahol iss.c Stefan Gehrer + jpegxl_anim_dec.c Leo Izen jpegxl_probe.* Leo Izen jvdec.c Peter Ross kvag.c Zane van Iperen diff --git a/libavformat/Makefile b/libavformat/Makefile index f8ad7c6a11..05434a0f82 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -316,6 +316,7 @@ OBJS-$(CONFIG_IVF_MUXER) += ivfenc.o OBJS-$(CONFIG_IVR_DEMUXER) += rmdec.o rm.o rmsipr.o OBJS-$(CONFIG_JACOSUB_DEMUXER) += jacosubdec.o subtitles.o OBJS-$(CONFIG_JACOSUB_MUXER) += jacosubenc.o rawenc.o +OBJS-$(CONFIG_JPEGXL_ANIM_DEMUXER) += jpegxl_anim_dec.o jpegxl_probe.o OBJS-$(CONFIG_JV_DEMUXER) += jvdec.o OBJS-$(CONFIG_KUX_DEMUXER) += flvdec.o OBJS-$(CONFIG_KVAG_DEMUXER) += kvag.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index efdb34e29d..96443a7272 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -238,6 +238,7 @@ extern const AVInputFormat ff_ivr_demuxer; extern const AVInputFormat ff_jacosub_demuxer; extern const FFOutputFormat ff_jacosub_muxer; extern const AVInputFormat ff_jv_demuxer; +extern const AVInputFormat ff_jpegxl_anim_demuxer; extern const AVInputFormat ff_kux_demuxer; extern const AVInputFormat ff_kvag_demuxer; extern const FFOutputFormat ff_kvag_muxer; diff --git a/libavformat/img2dec.c b/libavformat/img2dec.c index c037b6aa88..b986d3a502 100644 --- a/libavformat/img2dec.c +++ b/libavformat/img2dec.c @@ -850,7 +850,7 @@ static int jpegxl_probe(const AVProbeData *p) if (AV_RL16(b) != FF_JPEGXL_CODESTREAM_SIGNATURE_LE) return 0; #if CONFIG_IMAGE_JPEGXL_PIPE_DEMUXER - if (ff_jpegxl_verify_codestream_header(p->buf, p->buf_size) >= 0) + if (ff_jpegxl_verify_codestream_header(p->buf, p->buf_size, 1) >= 0) return AVPROBE_SCORE_MAX - 2; #endif return 0; diff --git a/libavformat/jpegxl_anim_dec.c b/libavformat/jpegxl_anim_dec.c new file mode 100644 index 0000000000..6ea6c46d8f --- /dev/null +++ b/libavformat/jpegxl_anim_dec.c @@ -0,0 +1,266 @@ +/* + * Animated JPEG XL Demuxer + * Copyright (c) 2023 Leo Izen (thebombzen) + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * Animated JPEG XL Demuxer + * @see ISO/IEC 18181-1 and 18181-2 + */ + +#include +#include + +#define BITSTREAM_READER_LE +#include "libavcodec/get_bits.h" + +#include "libavutil/intreadwrite.h" +#include "libavutil/opt.h" + +#include "avformat.h" +#include "internal.h" +#include "jpegxl_probe.h" + +typedef struct JXLAnimDemuxContext { + AVBufferRef *initial; +} JXLAnimDemuxContext; + +/* + * copies as much of the codestream into the buffer as possible + * pass a shorter buflen to request less + * returns the number of bytes consumed from input, may be greater than input_len + * if the input doesn't end on an ISOBMFF-box boundary + */ +static int jpegxl_collect_codestream_header(const uint8_t *input_buffer, int input_len, uint8_t *buffer, int buflen, int *copied) { + const uint8_t *b = input_buffer; + *copied = 0; + + while (1) { + uint64_t size; + uint32_t tag; + int head_size = 8; + + if (b - input_buffer >= input_len - 16) + break; + + size = AV_RB32(b); + b += 4; + if (size == 1) { + size = AV_RB64(b); + b += 8; + head_size = 16; + } + /* invalid ISOBMFF size */ + if (size > 0 && size <= head_size) + return AVERROR_INVALIDDATA; + if (size > 0) + size -= head_size; + + tag = AV_RL32(b); + b += 4; + if (tag == MKTAG('j', 'x', 'l', 'p')) { + b += 4; + size -= 4; + } + + if (tag == MKTAG('j', 'x', 'l', 'c') || tag == MKTAG('j', 'x', 'l', 'p')) { + /* + * size = 0 means "until EOF". this is legal but uncommon + * here we just set it to the remaining size of the probe buffer + * which at this point should always be nonnegative + */ + if (size == 0 || size > input_len - (b - input_buffer)) + size = input_len - (b - input_buffer); + + if (size > buflen - *copied) + size = buflen - *copied; + /* + * arbitrary chunking of the payload makes this memcpy hard to avoid + * in practice this will only be performed one or two times at most + */ + memcpy(buffer + *copied, b, size); + *copied += size; + } + b += size; + if (b >= input_buffer + input_len || *copied >= buflen) + break; + } + + return b - input_buffer; +} + +static int jpegxl_anim_probe(const AVProbeData *p) +{ + uint8_t buffer[4096]; + int copied; + + /* this is a raw codestream */ + if (AV_RL16(p->buf) == FF_JPEGXL_CODESTREAM_SIGNATURE_LE) { + if (ff_jpegxl_verify_codestream_header(p->buf, p->buf_size, 1) >= 1) + return AVPROBE_SCORE_MAX; + + return 0; + } + + /* not a JPEG XL file at all */ + if (AV_RL64(p->buf) != FF_JPEGXL_CONTAINER_SIGNATURE_LE) + return 0; + + if (jpegxl_collect_codestream_header(p->buf, p->buf_size, buffer, sizeof(buffer), &copied) <= 0 || copied <= 0) + return 0; + + if (ff_jpegxl_verify_codestream_header(buffer, copied, 0) >= 1) + return AVPROBE_SCORE_MAX; + + return 0; +} + +static int jpegxl_anim_read_header(AVFormatContext *s) +{ + JXLAnimDemuxContext *ctx = s->priv_data; + AVIOContext *pb = s->pb; + AVStream *st; + int offset = 0; + uint8_t head[256]; + int headsize = 0; + int ctrl; + AVRational tb; + GetBitContext gbi, *gb = &gbi; + + uint64_t sig16 = avio_rl16(pb); + if (sig16 == FF_JPEGXL_CODESTREAM_SIGNATURE_LE) { + AV_WL16(head, sig16); + headsize = avio_read(s->pb, head + 2, sizeof(head) - 2); + if (headsize < 0) + return headsize; + headsize += 2; + ctx->initial = av_buffer_alloc(headsize); + if (!ctx->initial) + return AVERROR(ENOMEM); + memcpy(ctx->initial->data, head, headsize); + } else { + uint64_t sig64 = avio_rl64(pb); + sig64 = (sig64 << 16) | sig16; + if (sig64 != FF_JPEGXL_CONTAINER_SIGNATURE_LE) + return AVERROR_INVALIDDATA; + avio_skip(pb, 2); // first box always 12 bytes + while (1) { + int copied; + uint8_t buf[4096]; + int read = avio_read(pb, buf, sizeof(buf)); + if (read < 0) + return read; + if (!ctx->initial) { + ctx->initial = av_buffer_alloc(read + 12); + if (!ctx->initial) + return AVERROR(ENOMEM); + AV_WL64(ctx->initial->data, FF_JPEGXL_CONTAINER_SIGNATURE_LE); + AV_WL32(ctx->initial->data + 8, 0x0a870a0d); + } else { + /* this only should be happening zero or one times in practice */ + if (av_buffer_realloc(&ctx->initial, ctx->initial->size + read) < 0) + return AVERROR(ENOMEM); + } + jpegxl_collect_codestream_header(buf, read, head + headsize, sizeof(head) - headsize, &copied); + memcpy(ctx->initial->data + (ctx->initial->size - read), buf, read); + headsize += copied; + if (headsize >= sizeof(head) || read < sizeof(buf)) + break; + } + } + /* offset in bits of the animation header */ + offset = ff_jpegxl_verify_codestream_header(head, headsize, 0); + if (offset <= 0) + return AVERROR_INVALIDDATA; + if (init_get_bits8(gb, head, headsize) < 0) + return AVERROR_INVALIDDATA; + skip_bits_long(gb, offset); + + st = avformat_new_stream(s, NULL); + if (!st) + return AVERROR(ENOMEM); + + st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; + st->codecpar->codec_id = AV_CODEC_ID_JPEGXL; + ctrl = get_bits(gb, 2); + tb.den = (const uint32_t[]){100, 1000, 1, 1}[ctrl] + get_bits_long(gb, (const uint32_t[]){0, 0, 10, 30}[ctrl]); + ctrl = get_bits(gb, 2); + tb.num = (const uint32_t[]){1, 1001, 1, 1}[ctrl] + get_bits_long(gb, (const uint32_t[]){0, 0, 8, 10}[ctrl]); + avpriv_set_pts_info(st, 1, tb.num, tb.den); + + return 0; +} + +/* the decoder requires the full input file as a single packet */ +static int jpegxl_anim_read_packet(AVFormatContext *s, AVPacket *pkt) +{ + JXLAnimDemuxContext *ctx = s->priv_data; + AVIOContext *pb = s->pb; + int ret; + int64_t size; + size_t offset = 0; + + if ((size = avio_size(pb)) < 0) + return size; + + /* animated JXL this big should not exist */ + if (size > INT_MAX) + return AVERROR_INVALIDDATA; + + if (ctx->initial && size < ctx->initial->size) + size = ctx->initial->size; + + if ((ret = av_new_packet(pkt, size) < 0)) + return ret; + + if (ctx->initial) { + offset = ctx->initial->size; + memcpy(pkt->data, ctx->initial->data, offset); + av_buffer_unref(&ctx->initial); + } + + if ((ret = avio_read(pb, pkt->data + offset, size - offset)) < 0) + return ret; + + return 0; +} + +static int jpegxl_anim_close(AVFormatContext *s) +{ + JXLAnimDemuxContext *ctx = s->priv_data; + if (ctx->initial) + av_buffer_unref(&ctx->initial); + + return 0; +} + +const AVInputFormat ff_jpegxl_anim_demuxer = { + .name = "jpegxl_anim", + .long_name = NULL_IF_CONFIG_SMALL("Animated JPEG XL"), + .priv_data_size = sizeof(JXLAnimDemuxContext), + .read_probe = jpegxl_anim_probe, + .read_header = jpegxl_anim_read_header, + .read_packet = jpegxl_anim_read_packet, + .read_close = jpegxl_anim_close, + .flags_internal = FF_FMT_INIT_CLEANUP, + .flags = AVFMT_GENERIC_INDEX, + .mime_type = "image/jxl", + .extensions = "jxl", +}; diff --git a/libavformat/jpegxl_probe.c b/libavformat/jpegxl_probe.c index 3de002f004..a3845b037d 100644 --- a/libavformat/jpegxl_probe.c +++ b/libavformat/jpegxl_probe.c @@ -208,7 +208,7 @@ static void jpegxl_skip_bit_depth(GetBitContext *gb) * validate a Jpeg XL Extra Channel Info bundle * @return >= 0 upon valid, < 0 upon invalid */ -static int jpegxl_read_extra_channel_info(GetBitContext *gb) +static int jpegxl_read_extra_channel_info(GetBitContext *gb, int validate_level) { int all_default = jxl_bits(1); uint32_t type, name_len = 0; @@ -217,7 +217,7 @@ static int jpegxl_read_extra_channel_info(GetBitContext *gb) type = jxl_enum(); if (type > 63) return -1; /* enum types cannot be 64+ */ - if (type == FF_JPEGXL_CT_BLACK) + if (type == FF_JPEGXL_CT_BLACK && validate_level) return -1; jpegxl_skip_bit_depth(gb); jxl_u32(0, 3, 4, 1, 0, 0, 0, 3); /* dim-shift */ @@ -242,12 +242,12 @@ static int jpegxl_read_extra_channel_info(GetBitContext *gb) return 0; } -/* verify that a codestream header is valid */ -int ff_jpegxl_verify_codestream_header(const uint8_t *buf, int buflen) +int ff_jpegxl_verify_codestream_header(const uint8_t *buf, int buflen, int validate_level) { GetBitContext gbi, *gb = &gbi; int all_default, extra_fields = 0; int xyb_encoded = 1, have_icc_profile = 0; + int animation_offset = 0; uint32_t num_extra_channels; uint64_t extensions; int ret; @@ -259,7 +259,7 @@ int ff_jpegxl_verify_codestream_header(const uint8_t *buf, int buflen) if (jxl_bits(16) != FF_JPEGXL_CODESTREAM_SIGNATURE_LE) return -1; - if (jpegxl_read_size_header(gb) < 0) + if (jpegxl_read_size_header(gb) < 0 && validate_level) return -1; all_default = jxl_bits(1); @@ -285,6 +285,7 @@ int ff_jpegxl_verify_codestream_header(const uint8_t *buf, int buflen) /* animation header */ if (jxl_bits(1)) { + animation_offset = get_bits_count(gb); jxl_u32(100, 1000, 1, 1, 0, 0, 10, 30); jxl_u32(1, 1001, 1, 1, 0, 0, 8, 10); jxl_u32(0, 0, 0, 0, 0, 3, 16, 32); @@ -296,14 +297,14 @@ int ff_jpegxl_verify_codestream_header(const uint8_t *buf, int buflen) jpegxl_skip_bit_depth(gb); /* modular_16bit_buffers must equal 1 */ - if (!jxl_bits(1)) + if (!jxl_bits(1) && validate_level) return -1; num_extra_channels = jxl_u32(0, 1, 2, 1, 0, 0, 4, 12); - if (num_extra_channels > 4) + if (num_extra_channels > 4 && validate_level) return -1; for (uint32_t i = 0; i < num_extra_channels; i++) { - if (jpegxl_read_extra_channel_info(gb) < 0) + if (jpegxl_read_extra_channel_info(gb, validate_level) < 0) return -1; } @@ -392,5 +393,5 @@ int ff_jpegxl_verify_codestream_header(const uint8_t *buf, int buflen) if (get_bits_left(gb) < 0) return -1; - return 0; + return animation_offset; } diff --git a/libavformat/jpegxl_probe.h b/libavformat/jpegxl_probe.h index 2960e81e11..496445fbce 100644 --- a/libavformat/jpegxl_probe.h +++ b/libavformat/jpegxl_probe.h @@ -27,6 +27,11 @@ #define FF_JPEGXL_CODESTREAM_SIGNATURE_LE 0x0aff #define FF_JPEGXL_CONTAINER_SIGNATURE_LE 0x204c584a0c000000 -int ff_jpegxl_verify_codestream_header(const uint8_t *buf, int buflen); +/** + * @brief verify that a codestream header is valid + * @return Negative upon error, 0 upon verifying that the codestream is not animated, + * and 1 upon verifying that it is animated + */ +int ff_jpegxl_verify_codestream_header(const uint8_t *buf, int buflen, int validate_level); #endif /* AVFORMAT_JPEGXL_PROBE_H */ diff --git a/libavformat/version.h b/libavformat/version.h index e2634b85ae..4bde82abb4 100644 --- a/libavformat/version.h +++ b/libavformat/version.h @@ -31,7 +31,7 @@ #include "version_major.h" -#define LIBAVFORMAT_VERSION_MINOR 5 +#define LIBAVFORMAT_VERSION_MINOR 6 #define LIBAVFORMAT_VERSION_MICRO 100 #define LIBAVFORMAT_VERSION_INT AV_VERSION_INT(LIBAVFORMAT_VERSION_MAJOR, \ -- 2.40.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".