* [FFmpeg-devel] [PATCH v14 0/4] Jpeg XL Patch Set
@ 2022-04-12 5:53 Leo Izen
2022-04-12 5:53 ` [FFmpeg-devel] [PATCH v14 1/4] avcodec/jpegxl: add Jpeg XL image codec Leo Izen
` (3 more replies)
0 siblings, 4 replies; 14+ messages in thread
From: Leo Izen @ 2022-04-12 5:53 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Leo Izen
Changes:
v14:
- Make changes requested by Anton from v13
-- make encoder return refcounted packet
-- use av_realloc for realloc loop
-- use -1 as default instead of -0.0
- Remove unnecessary JXL_DEC_FRAME subscribe event from decoder
- Update FFCodec declarations to use the new macros, rebase onto master
v13:
- Make changes requested by Anton and Andreas from v12
v12:
- Remove the parser in order to avoid avpriv in the ABI
- Add a lightweight version of it to avformat to help the prober
- Remove the FATE test for the parser which no longer exists
Leo Izen (4):
avcodec/jpegxl: add Jpeg XL image codec
avcodec/libjxl: add Jpeg XL decoding via libjxl
avcodec/libjxl: add Jpeg XL encoding via libjxl
avformat/image2: add Jpeg XL as image2 format
MAINTAINERS | 3 +
configure | 6 +
doc/general_contents.texi | 7 +
libavcodec/Makefile | 2 +
libavcodec/allcodecs.c | 2 +
libavcodec/codec_desc.c | 9 +
libavcodec/codec_id.h | 1 +
libavcodec/libjxl.c | 70 +++++++
libavcodec/libjxl.h | 48 +++++
libavcodec/libjxldec.c | 274 ++++++++++++++++++++++++++
libavcodec/libjxlenc.c | 382 +++++++++++++++++++++++++++++++++++
libavformat/Makefile | 1 +
libavformat/allformats.c | 1 +
libavformat/img2.c | 1 +
libavformat/img2dec.c | 18 ++
libavformat/img2enc.c | 6 +-
libavformat/jpegxl_probe.c | 393 +++++++++++++++++++++++++++++++++++++
libavformat/jpegxl_probe.h | 32 +++
libavformat/mov.c | 1 +
19 files changed, 1254 insertions(+), 3 deletions(-)
create mode 100644 libavcodec/libjxl.c
create mode 100644 libavcodec/libjxl.h
create mode 100644 libavcodec/libjxldec.c
create mode 100644 libavcodec/libjxlenc.c
create mode 100644 libavformat/jpegxl_probe.c
create mode 100644 libavformat/jpegxl_probe.h
--
2.35.1
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 14+ messages in thread* [FFmpeg-devel] [PATCH v14 1/4] avcodec/jpegxl: add Jpeg XL image codec 2022-04-12 5:53 [FFmpeg-devel] [PATCH v14 0/4] Jpeg XL Patch Set Leo Izen @ 2022-04-12 5:53 ` Leo Izen 2022-04-12 5:53 ` [FFmpeg-devel] [PATCH v14 2/4] avcodec/libjxl: add Jpeg XL decoding via libjxl Leo Izen ` (2 subsequent siblings) 3 siblings, 0 replies; 14+ messages in thread From: Leo Izen @ 2022-04-12 5:53 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Leo Izen This commit adds support to libavcodec to read encoded Jpeg XL images. Jpeg XL is intended to be an extended-life replacement to legacy mjpeg. --- MAINTAINERS | 1 + libavcodec/codec_desc.c | 9 +++++++++ libavcodec/codec_id.h | 1 + 3 files changed, 11 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 8c71605339..859a5005d4 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -618,6 +618,7 @@ Haihao Xiang (haihao) 1F0C 31E8 B4FE F7A4 4DC1 DC99 E0F5 76D4 76FC 437F Jaikrishnan Menon 61A1 F09F 01C9 2D45 78E1 C862 25DC 8831 AF70 D368 James Almer 7751 2E8C FD94 A169 57E6 9A7A 1463 01AD 7376 59E0 Jean Delvare 7CA6 9F44 60F1 BDC4 1FD2 C858 A552 6B9B B3CD 4E6A +Leo Izen (thebombzen) B6FD 3CFC 7ACF 83FC 9137 6945 5A71 C331 FD2F A19A Loren Merritt ABD9 08F4 C920 3F65 D8BE 35D7 1540 DAA7 060F 56DE Lynne FE50 139C 6805 72CA FD52 1F8D A2FE A5F0 3F03 4464 Michael Niedermayer 9FF2 128B 147E F673 0BAD F133 611E C787 040B 0FAB diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c index c08854cc93..e7f0f6a8d4 100644 --- a/libavcodec/codec_desc.c +++ b/libavcodec/codec_desc.c @@ -1870,6 +1870,15 @@ static const AVCodecDescriptor codec_descriptors[] = { .long_name = NULL_IF_CONFIG_SMALL("Vizrt Binary Image"), .props = AV_CODEC_PROP_LOSSY, }, + { + .id = AV_CODEC_ID_JPEGXL, + .type = AVMEDIA_TYPE_VIDEO, + .name = "jpegxl", + .long_name = NULL_IF_CONFIG_SMALL("JPEG XL"), + .props = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSY | + AV_CODEC_PROP_LOSSLESS, + .mime_types= MT("image/jxl"), + }, /* various PCM "codecs" */ { diff --git a/libavcodec/codec_id.h b/libavcodec/codec_id.h index 43c72ce8e4..8b317fa121 100644 --- a/libavcodec/codec_id.h +++ b/libavcodec/codec_id.h @@ -309,6 +309,7 @@ enum AVCodecID { AV_CODEC_ID_SGA_VIDEO, AV_CODEC_ID_GEM, AV_CODEC_ID_VBN, + AV_CODEC_ID_JPEGXL, /* various PCM "codecs" */ AV_CODEC_ID_FIRST_AUDIO = 0x10000, ///< A dummy id pointing at the start of audio codecs -- 2.35.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 14+ messages in thread
* [FFmpeg-devel] [PATCH v14 2/4] avcodec/libjxl: add Jpeg XL decoding via libjxl 2022-04-12 5:53 [FFmpeg-devel] [PATCH v14 0/4] Jpeg XL Patch Set Leo Izen 2022-04-12 5:53 ` [FFmpeg-devel] [PATCH v14 1/4] avcodec/jpegxl: add Jpeg XL image codec Leo Izen @ 2022-04-12 5:53 ` Leo Izen 2022-04-15 11:08 ` Andreas Rheinhardt 2022-04-12 5:53 ` [FFmpeg-devel] [PATCH v14 3/4] avcodec/libjxl: add Jpeg XL encoding " Leo Izen 2022-04-12 5:53 ` [FFmpeg-devel] [PATCH v14 4/4] avformat/image2: add Jpeg XL as image2 format Leo Izen 3 siblings, 1 reply; 14+ messages in thread From: Leo Izen @ 2022-04-12 5:53 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Leo Izen This commit adds decoding support to libavcodec for Jpeg XL images via the external library libjxl. --- MAINTAINERS | 1 + configure | 5 + doc/general_contents.texi | 7 + libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 1 + libavcodec/libjxl.c | 70 ++++++++++ libavcodec/libjxl.h | 48 +++++++ libavcodec/libjxldec.c | 274 ++++++++++++++++++++++++++++++++++++++ 8 files changed, 407 insertions(+) create mode 100644 libavcodec/libjxl.c create mode 100644 libavcodec/libjxl.h create mode 100644 libavcodec/libjxldec.c diff --git a/MAINTAINERS b/MAINTAINERS index 859a5005d4..faea84ebf1 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -194,6 +194,7 @@ Codecs: libcodec2.c Tomas Härdin libdirac* David Conrad libdavs2.c Huiwen Ren + libjxl*.c, libjxl.h Leo Izen libgsm.c Michel Bardiaux libkvazaar.c Arttu Ylä-Outinen libopenh264enc.c Martin Storsjo, Linjie Fu diff --git a/configure b/configure index 9c8965852b..c36d5dbc8b 100755 --- a/configure +++ b/configure @@ -240,6 +240,7 @@ External library support: --enable-libiec61883 enable iec61883 via libiec61883 [no] --enable-libilbc enable iLBC de/encoding via libilbc [no] --enable-libjack enable JACK audio sound server [no] + --enable-libjxl enable JPEG XL decoding via libjxl [no] --enable-libklvanc enable Kernel Labs VANC processing [no] --enable-libkvazaar enable HEVC encoding via libkvazaar [no] --enable-liblensfun enable lensfun lens correction [no] @@ -1833,6 +1834,7 @@ EXTERNAL_LIBRARY_LIST=" libiec61883 libilbc libjack + libjxl libklvanc libkvazaar libmodplug @@ -3331,6 +3333,7 @@ libgsm_ms_decoder_deps="libgsm" libgsm_ms_encoder_deps="libgsm" libilbc_decoder_deps="libilbc" libilbc_encoder_deps="libilbc" +libjxl_decoder_deps="libjxl libjxl_threads" libkvazaar_encoder_deps="libkvazaar" libmodplug_demuxer_deps="libmodplug" libmp3lame_encoder_deps="libmp3lame" @@ -6543,6 +6546,8 @@ enabled libgsm && { for gsm_hdr in "gsm.h" "gsm/gsm.h"; do check_lib libgsm "${gsm_hdr}" gsm_create -lgsm && break; done || die "ERROR: libgsm not found"; } enabled libilbc && require libilbc ilbc.h WebRtcIlbcfix_InitDecode -lilbc $pthreads_extralibs +enabled libjxl && require_pkg_config libjxl "libjxl >= 0.7.0" jxl/decode.h JxlDecoderVersion && + require_pkg_config libjxl_threads "libjxl_threads >= 0.7.0" jxl/thread_parallel_runner.h JxlThreadParallelRunner enabled libklvanc && require libklvanc libklvanc/vanc.h klvanc_context_create -lklvanc enabled libkvazaar && require_pkg_config libkvazaar "kvazaar >= 0.8.1" kvazaar.h kvz_api_get enabled liblensfun && require_pkg_config liblensfun lensfun lensfun.h lf_db_new diff --git a/doc/general_contents.texi b/doc/general_contents.texi index 238568f2bb..93a90a5e52 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -171,6 +171,13 @@ Go to @url{https://github.com/TimothyGu/libilbc} and follow the instructions for installing the library. Then pass @code{--enable-libilbc} to configure to enable it. +@section libjxl + +JPEG XL is an image format intended to fully replace legacy JPEG for an extended +period of life. See @url{https://jpegxl.info/} for more information, and see +@url{https://github.com/libjxl/libjxl} for the library source. You can pass +@code{--enable-libjxl} to configure in order enable the libjxl wrapper. + @section libvpx FFmpeg can make use of the libvpx library for VP8/VP9 decoding and encoding. diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 90f46035d9..8fc3b3cc5f 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1061,6 +1061,7 @@ OBJS-$(CONFIG_LIBGSM_MS_DECODER) += libgsmdec.o OBJS-$(CONFIG_LIBGSM_MS_ENCODER) += libgsmenc.o OBJS-$(CONFIG_LIBILBC_DECODER) += libilbc.o OBJS-$(CONFIG_LIBILBC_ENCODER) += libilbc.o +OBJS-$(CONFIG_LIBJXL_DECODER) += libjxldec.o libjxl.o OBJS-$(CONFIG_LIBKVAZAAR_ENCODER) += libkvazaar.o OBJS-$(CONFIG_LIBMP3LAME_ENCODER) += libmp3lame.o OBJS-$(CONFIG_LIBOPENCORE_AMRNB_DECODER) += libopencore-amr.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 585918da93..07f5bafd27 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -751,6 +751,7 @@ extern const FFCodec ff_libgsm_ms_encoder; extern const FFCodec ff_libgsm_ms_decoder; extern const FFCodec ff_libilbc_encoder; extern const FFCodec ff_libilbc_decoder; +extern const FFCodec ff_libjxl_decoder; extern const FFCodec ff_libmp3lame_encoder; extern const FFCodec ff_libopencore_amrnb_encoder; extern const FFCodec ff_libopencore_amrnb_decoder; diff --git a/libavcodec/libjxl.c b/libavcodec/libjxl.c new file mode 100644 index 0000000000..204d91d8a8 --- /dev/null +++ b/libavcodec/libjxl.c @@ -0,0 +1,70 @@ +/* + * JPEG XL de/encoding via libjxl, common support implementation + * Copyright (c) 2021 Leo Izen <leo.izen@gmail.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * JPEG XL via libjxl common support implementation + */ + +#include "libavutil/cpu.h" +#include "libavutil/mem.h" + +#include <jxl/memory_manager.h> +#include "libjxl.h" + +size_t ff_libjxl_get_threadcount(int threads) +{ + if (threads <= 0) + return av_cpu_count(); + if (threads == 1) + return 0; + return threads; +} + +/** + * Wrapper around av_malloc used as a jpegxl_alloc_func. + * + * @param opaque opaque pointer for jpegxl_alloc_func, always ignored + * @param size Size in bytes for the memory block to be allocated + * @return Pointer to the allocated block, or `NULL` if it cannot be allocated + */ +static void *libjxl_av_malloc(void *opaque, size_t size) +{ + return av_malloc(size); +} + +/** + * Wrapper around av_free used as a jpegxl_free_func. + * + * @param opaque opaque pointer for jpegxl_free_func, always ignored + * @param address Pointer to the allocated block, to free. `NULL` permitted as a no-op. + */ +static void libjxl_av_free(void *opaque, void *address) +{ + av_free(address); +} + +void ff_libjxl_init_memory_manager(JxlMemoryManager *manager) +{ + manager->opaque = NULL; + manager->alloc = &libjxl_av_malloc; + manager->free = &libjxl_av_free; +} diff --git a/libavcodec/libjxl.h b/libavcodec/libjxl.h new file mode 100644 index 0000000000..5387c438fd --- /dev/null +++ b/libavcodec/libjxl.h @@ -0,0 +1,48 @@ +/* + * JPEG XL de/encoding via libjxl, common support header + * Copyright (c) 2021 Leo Izen <leo.izen@gmail.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * JPEG XL via libjxl common support header + */ + +#ifndef AVCODEC_LIBJXL_H +#define AVCODEC_LIBJXL_H + +#include <jxl/memory_manager.h> + +/** + * Transform threadcount in ffmpeg to one used by libjxl. + * + * @param threads ffmpeg's threads AVOption + * @return thread count for libjxl's parallel runner + */ +size_t ff_libjxl_get_threadcount(int threads); + +/** + * Initialize and populate a JxlMemoryManager + * with av_malloc() and av_free() so libjxl will use these + * functions. + * @param manager a pointer to a JxlMemoryManager struct + */ +void ff_libjxl_init_memory_manager(JxlMemoryManager *manager); + +#endif /* AVCODEC_LIBJXL_H */ diff --git a/libavcodec/libjxldec.c b/libavcodec/libjxldec.c new file mode 100644 index 0000000000..a7d205b081 --- /dev/null +++ b/libavcodec/libjxldec.c @@ -0,0 +1,274 @@ +/* + * JPEG XL decoding support via libjxl + * Copyright (c) 2021 Leo Izen <leo.izen@gmail.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * JPEG XL decoder using libjxl + */ + +#include "libavutil/avassert.h" +#include "libavutil/buffer.h" +#include "libavutil/common.h" +#include "libavutil/error.h" +#include "libavutil/mem.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/frame.h" + +#include "avcodec.h" +#include "codec_internal.h" +#include "internal.h" + +#include <jxl/decode.h> +#include <jxl/thread_parallel_runner.h> +#include "libjxl.h" + +typedef struct LibJxlDecodeContext { + void *runner; + JxlDecoder *decoder; + JxlBasicInfo basic_info; + JxlPixelFormat jxl_pixfmt; + JxlDecoderStatus events; + AVBufferRef *iccp; +} LibJxlDecodeContext; + +static int libjxl_init_jxl_decoder(AVCodecContext *avctx) +{ + LibJxlDecodeContext *ctx = avctx->priv_data; + + ctx->events = JXL_DEC_BASIC_INFO | JXL_DEC_FULL_IMAGE | JXL_DEC_COLOR_ENCODING; + if (JxlDecoderSubscribeEvents(ctx->decoder, ctx->events) != JXL_DEC_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Error subscribing to JXL events\n"); + return AVERROR_EXTERNAL; + } + + if (JxlDecoderSetParallelRunner(ctx->decoder, JxlThreadParallelRunner, ctx->runner) != JXL_DEC_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to set JxlThreadParallelRunner\n"); + return AVERROR_EXTERNAL; + } + + memset(&ctx->basic_info, 0, sizeof(JxlBasicInfo)); + memset(&ctx->jxl_pixfmt, 0, sizeof(JxlPixelFormat)); + + return 0; +} + +static av_cold int libjxl_decode_init(AVCodecContext *avctx) +{ + LibJxlDecodeContext *ctx = avctx->priv_data; + JxlMemoryManager manager; + + ff_libjxl_init_memory_manager(&manager); + ctx->decoder = JxlDecoderCreate(&manager); + if (!ctx->decoder) { + av_log(avctx, AV_LOG_ERROR, "Failed to create JxlDecoder\n"); + return AVERROR_EXTERNAL; + } + + ctx->runner = JxlThreadParallelRunnerCreate(&manager, ff_libjxl_get_threadcount(avctx->thread_count)); + if (!ctx->runner) { + av_log(avctx, AV_LOG_ERROR, "Failed to create JxlThreadParallelRunner\n"); + return AVERROR_EXTERNAL; + } + + return libjxl_init_jxl_decoder(avctx); +} + +static enum AVPixelFormat libjxl_get_pix_fmt(AVCodecContext *avctx, JxlBasicInfo *basic_info, JxlPixelFormat *format) +{ + format->endianness = JXL_NATIVE_ENDIAN; + format->num_channels = basic_info->num_color_channels + (basic_info->alpha_bits > 0); + /* Gray */ + if (basic_info->num_color_channels == 1) { + if (basic_info->bits_per_sample <= 8) { + format->data_type = JXL_TYPE_UINT8; + return basic_info->alpha_bits ? AV_PIX_FMT_YA8 : AV_PIX_FMT_GRAY8; + } + if (basic_info->exponent_bits_per_sample || basic_info->bits_per_sample > 16) { + if (basic_info->alpha_bits) + return AV_PIX_FMT_NONE; + format->data_type = JXL_TYPE_FLOAT; + return AV_PIX_FMT_GRAYF32; + } + format->data_type = JXL_TYPE_UINT16; + return basic_info->alpha_bits ? AV_PIX_FMT_YA16 : AV_PIX_FMT_GRAY16; + } + /* rgb only */ + /* libjxl only supports packed RGB and gray output at the moment */ + if (basic_info->num_color_channels == 3) { + if (basic_info->bits_per_sample <= 8) { + format->data_type = JXL_TYPE_UINT8; + return basic_info->alpha_bits ? AV_PIX_FMT_RGBA : AV_PIX_FMT_RGB24; + } + if (basic_info->bits_per_sample > 16) + av_log(avctx, AV_LOG_WARNING, "Downsampling larger integer to 16-bit via libjxl\n"); + if (basic_info->exponent_bits_per_sample) + av_log(avctx, AV_LOG_WARNING, "Downsampling float to 16-bit integer via libjxl\n"); + format->data_type = JXL_TYPE_UINT16; + return basic_info->alpha_bits ? AV_PIX_FMT_RGBA64 : AV_PIX_FMT_RGB48; + } + + return AV_PIX_FMT_NONE; +} + +static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_frame, AVPacket *avpkt) +{ + LibJxlDecodeContext *ctx = avctx->priv_data; + uint8_t *buf = avpkt->data; + size_t remaining = avpkt->size, iccp_len; + JxlDecoderStatus jret; + int ret; + *got_frame = 0; + + while (1) { + + jret = JxlDecoderSetInput(ctx->decoder, buf, remaining); + + if (jret == JXL_DEC_ERROR) { + /* this should never happen here unless there's a bug in libjxl */ + av_log(avctx, AV_LOG_ERROR, "Unknown libjxl decode error\n"); + return AVERROR_EXTERNAL; + } + + jret = JxlDecoderProcessInput(ctx->decoder); + /* + * JxlDecoderReleaseInput returns the number + * of bytes remaining to be read, rather than + * the number of bytes that it did read + */ + remaining = JxlDecoderReleaseInput(ctx->decoder); + buf = avpkt->data + avpkt->size - remaining; + + switch(jret) { + case JXL_DEC_ERROR: + av_log(avctx, AV_LOG_ERROR, "Unknown libjxl decode error\n"); + return AVERROR_INVALIDDATA; + case JXL_DEC_NEED_MORE_INPUT: + if (remaining == 0) { + av_log(avctx, AV_LOG_ERROR, "Unexpected end of JXL codestream\n"); + return AVERROR_INVALIDDATA; + } + av_log(avctx, AV_LOG_DEBUG, "NEED_MORE_INPUT event emitted\n"); + continue; + case JXL_DEC_BASIC_INFO: + av_log(avctx, AV_LOG_DEBUG, "BASIC_INFO event emitted\n"); + if (JxlDecoderGetBasicInfo(ctx->decoder, &ctx->basic_info) != JXL_DEC_SUCCESS) { + /* + * this should never happen + * if it does it is likely a libjxl decoder bug + */ + av_log(avctx, AV_LOG_ERROR, "Bad libjxl basic info event\n"); + return AVERROR_EXTERNAL; + } + avctx->pix_fmt = libjxl_get_pix_fmt(avctx, &ctx->basic_info, &ctx->jxl_pixfmt); + if (avctx->pix_fmt == AV_PIX_FMT_NONE) { + av_log(avctx, AV_LOG_ERROR, "Bad libjxl pixel format\n"); + return AVERROR_EXTERNAL; + } + ret = ff_set_dimensions(avctx, ctx->basic_info.xsize, ctx->basic_info.ysize); + if (ret < 0) + return ret; + continue; + case JXL_DEC_COLOR_ENCODING: + av_log(avctx, AV_LOG_DEBUG, "COLOR_ENCODING event emitted\n"); + jret = JxlDecoderGetICCProfileSize(ctx->decoder, &ctx->jxl_pixfmt, JXL_COLOR_PROFILE_TARGET_ORIGINAL, &iccp_len); + if (jret == JXL_DEC_SUCCESS && iccp_len > 0) { + av_buffer_unref(&ctx->iccp); + ctx->iccp = av_buffer_alloc(iccp_len); + if (!ctx->iccp) + return AVERROR(ENOMEM); + jret = JxlDecoderGetColorAsICCProfile(ctx->decoder, &ctx->jxl_pixfmt, JXL_COLOR_PROFILE_TARGET_ORIGINAL, ctx->iccp->data, iccp_len); + if (jret != JXL_DEC_SUCCESS) + av_buffer_unref(&ctx->iccp); + } + continue; + case JXL_DEC_NEED_IMAGE_OUT_BUFFER: + av_log(avctx, AV_LOG_DEBUG, "NEED_IMAGE_OUT_BUFFER event emitted\n"); + ret = ff_get_buffer(avctx, frame, 0); + if (ret < 0) + return ret; + ctx->jxl_pixfmt.align = frame->linesize[0]; + if (JxlDecoderSetImageOutBuffer(ctx->decoder, &ctx->jxl_pixfmt, frame->data[0], frame->buf[0]->size) != JXL_DEC_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Bad libjxl dec need image out buffer event\n"); + return AVERROR_EXTERNAL; + } + continue; + case JXL_DEC_FULL_IMAGE: + /* full image is one frame, even if animated */ + av_log(avctx, AV_LOG_DEBUG, "FULL_IMAGE event emitted\n"); + *got_frame = 1; + frame->pict_type = AV_PICTURE_TYPE_I; + frame->key_frame = 1; + if (ctx->iccp) { + AVFrameSideData *sd = av_frame_new_side_data_from_buf(frame, AV_FRAME_DATA_ICC_PROFILE, ctx->iccp); + if (!sd) + return AVERROR(ENOMEM); + /* ownership is transfered, and it is not ref-ed */ + ctx->iccp = NULL; + } + return avpkt->size - remaining; + case JXL_DEC_SUCCESS: + av_log(avctx, AV_LOG_DEBUG, "SUCCESS event emitted\n"); + /* + * The file has finished decoding + * reset the decoder to let us + * reuse it again for the next image + */ + JxlDecoderReset(ctx->decoder); + libjxl_init_jxl_decoder(avctx); + buf = avpkt->data; + remaining = avpkt->size; + continue; + default: + av_log(avctx, AV_LOG_ERROR, "Bad libjxl event: %d\n", jret); + return AVERROR_EXTERNAL; + } + } +} + +static av_cold int libjxl_decode_close(AVCodecContext *avctx) +{ + LibJxlDecodeContext *ctx = avctx->priv_data; + + if (ctx->runner) + JxlThreadParallelRunnerDestroy(ctx->runner); + ctx->runner = NULL; + if (ctx->decoder) + JxlDecoderDestroy(ctx->decoder); + ctx->decoder = NULL; + av_buffer_unref(&ctx->iccp); + + return 0; +} + +const FFCodec ff_libjxl_decoder = { + .p.name = "libjxl", + .p.long_name = NULL_IF_CONFIG_SMALL("libjxl JPEG XL"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_JPEGXL, + .priv_data_size = sizeof(LibJxlDecodeContext), + .init = libjxl_decode_init, + FF_CODEC_DECODE_CB(libjxl_decode_frame), + .close = libjxl_decode_close, + .p.capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_OTHER_THREADS, + .caps_internal = FF_CODEC_CAP_AUTO_THREADS | FF_CODEC_CAP_INIT_CLEANUP, + .p.wrapper_name = "libjxl", +}; -- 2.35.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [FFmpeg-devel] [PATCH v14 2/4] avcodec/libjxl: add Jpeg XL decoding via libjxl 2022-04-12 5:53 ` [FFmpeg-devel] [PATCH v14 2/4] avcodec/libjxl: add Jpeg XL decoding via libjxl Leo Izen @ 2022-04-15 11:08 ` Andreas Rheinhardt 2022-04-15 19:31 ` Leo Izen 0 siblings, 1 reply; 14+ messages in thread From: Andreas Rheinhardt @ 2022-04-15 11:08 UTC (permalink / raw) To: ffmpeg-devel Leo Izen: > This commit adds decoding support to libavcodec > for Jpeg XL images via the external library libjxl. > --- > MAINTAINERS | 1 + > configure | 5 + > doc/general_contents.texi | 7 + > libavcodec/Makefile | 1 + > libavcodec/allcodecs.c | 1 + > libavcodec/libjxl.c | 70 ++++++++++ > libavcodec/libjxl.h | 48 +++++++ > libavcodec/libjxldec.c | 274 ++++++++++++++++++++++++++++++++++++++ > 8 files changed, 407 insertions(+) > create mode 100644 libavcodec/libjxl.c > create mode 100644 libavcodec/libjxl.h > create mode 100644 libavcodec/libjxldec.c > > diff --git a/MAINTAINERS b/MAINTAINERS > index 859a5005d4..faea84ebf1 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -194,6 +194,7 @@ Codecs: > libcodec2.c Tomas Härdin > libdirac* David Conrad > libdavs2.c Huiwen Ren > + libjxl*.c, libjxl.h Leo Izen > libgsm.c Michel Bardiaux > libkvazaar.c Arttu Ylä-Outinen > libopenh264enc.c Martin Storsjo, Linjie Fu > diff --git a/configure b/configure > index 9c8965852b..c36d5dbc8b 100755 > --- a/configure > +++ b/configure > @@ -240,6 +240,7 @@ External library support: > --enable-libiec61883 enable iec61883 via libiec61883 [no] > --enable-libilbc enable iLBC de/encoding via libilbc [no] > --enable-libjack enable JACK audio sound server [no] > + --enable-libjxl enable JPEG XL decoding via libjxl [no] > --enable-libklvanc enable Kernel Labs VANC processing [no] > --enable-libkvazaar enable HEVC encoding via libkvazaar [no] > --enable-liblensfun enable lensfun lens correction [no] > @@ -1833,6 +1834,7 @@ EXTERNAL_LIBRARY_LIST=" > libiec61883 > libilbc > libjack > + libjxl > libklvanc > libkvazaar > libmodplug > @@ -3331,6 +3333,7 @@ libgsm_ms_decoder_deps="libgsm" > libgsm_ms_encoder_deps="libgsm" > libilbc_decoder_deps="libilbc" > libilbc_encoder_deps="libilbc" > +libjxl_decoder_deps="libjxl libjxl_threads" > libkvazaar_encoder_deps="libkvazaar" > libmodplug_demuxer_deps="libmodplug" > libmp3lame_encoder_deps="libmp3lame" > @@ -6543,6 +6546,8 @@ enabled libgsm && { for gsm_hdr in "gsm.h" "gsm/gsm.h"; do > check_lib libgsm "${gsm_hdr}" gsm_create -lgsm && break; > done || die "ERROR: libgsm not found"; } > enabled libilbc && require libilbc ilbc.h WebRtcIlbcfix_InitDecode -lilbc $pthreads_extralibs > +enabled libjxl && require_pkg_config libjxl "libjxl >= 0.7.0" jxl/decode.h JxlDecoderVersion && > + require_pkg_config libjxl_threads "libjxl_threads >= 0.7.0" jxl/thread_parallel_runner.h JxlThreadParallelRunner > enabled libklvanc && require libklvanc libklvanc/vanc.h klvanc_context_create -lklvanc > enabled libkvazaar && require_pkg_config libkvazaar "kvazaar >= 0.8.1" kvazaar.h kvz_api_get > enabled liblensfun && require_pkg_config liblensfun lensfun lensfun.h lf_db_new > diff --git a/doc/general_contents.texi b/doc/general_contents.texi > index 238568f2bb..93a90a5e52 100644 > --- a/doc/general_contents.texi > +++ b/doc/general_contents.texi > @@ -171,6 +171,13 @@ Go to @url{https://github.com/TimothyGu/libilbc} and follow the instructions for > installing the library. Then pass @code{--enable-libilbc} to configure to > enable it. > > +@section libjxl > + > +JPEG XL is an image format intended to fully replace legacy JPEG for an extended > +period of life. See @url{https://jpegxl.info/} for more information, and see > +@url{https://github.com/libjxl/libjxl} for the library source. You can pass > +@code{--enable-libjxl} to configure in order enable the libjxl wrapper. > + > @section libvpx > > FFmpeg can make use of the libvpx library for VP8/VP9 decoding and encoding. > diff --git a/libavcodec/Makefile b/libavcodec/Makefile > index 90f46035d9..8fc3b3cc5f 100644 > --- a/libavcodec/Makefile > +++ b/libavcodec/Makefile > @@ -1061,6 +1061,7 @@ OBJS-$(CONFIG_LIBGSM_MS_DECODER) += libgsmdec.o > OBJS-$(CONFIG_LIBGSM_MS_ENCODER) += libgsmenc.o > OBJS-$(CONFIG_LIBILBC_DECODER) += libilbc.o > OBJS-$(CONFIG_LIBILBC_ENCODER) += libilbc.o > +OBJS-$(CONFIG_LIBJXL_DECODER) += libjxldec.o libjxl.o > OBJS-$(CONFIG_LIBKVAZAAR_ENCODER) += libkvazaar.o > OBJS-$(CONFIG_LIBMP3LAME_ENCODER) += libmp3lame.o > OBJS-$(CONFIG_LIBOPENCORE_AMRNB_DECODER) += libopencore-amr.o > diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c > index 585918da93..07f5bafd27 100644 > --- a/libavcodec/allcodecs.c > +++ b/libavcodec/allcodecs.c > @@ -751,6 +751,7 @@ extern const FFCodec ff_libgsm_ms_encoder; > extern const FFCodec ff_libgsm_ms_decoder; > extern const FFCodec ff_libilbc_encoder; > extern const FFCodec ff_libilbc_decoder; > +extern const FFCodec ff_libjxl_decoder; > extern const FFCodec ff_libmp3lame_encoder; > extern const FFCodec ff_libopencore_amrnb_encoder; > extern const FFCodec ff_libopencore_amrnb_decoder; > diff --git a/libavcodec/libjxl.c b/libavcodec/libjxl.c > new file mode 100644 > index 0000000000..204d91d8a8 > --- /dev/null > +++ b/libavcodec/libjxl.c > @@ -0,0 +1,70 @@ > +/* > + * JPEG XL de/encoding via libjxl, common support implementation > + * Copyright (c) 2021 Leo Izen <leo.izen@gmail.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +/** > + * @file > + * JPEG XL via libjxl common support implementation > + */ > + > +#include "libavutil/cpu.h" > +#include "libavutil/mem.h" > + > +#include <jxl/memory_manager.h> > +#include "libjxl.h" > + > +size_t ff_libjxl_get_threadcount(int threads) > +{ > + if (threads <= 0) > + return av_cpu_count(); > + if (threads == 1) > + return 0; > + return threads; > +} > + > +/** > + * Wrapper around av_malloc used as a jpegxl_alloc_func. > + * > + * @param opaque opaque pointer for jpegxl_alloc_func, always ignored > + * @param size Size in bytes for the memory block to be allocated > + * @return Pointer to the allocated block, or `NULL` if it cannot be allocated > + */ > +static void *libjxl_av_malloc(void *opaque, size_t size) > +{ > + return av_malloc(size); > +} > + > +/** > + * Wrapper around av_free used as a jpegxl_free_func. > + * > + * @param opaque opaque pointer for jpegxl_free_func, always ignored > + * @param address Pointer to the allocated block, to free. `NULL` permitted as a no-op. > + */ > +static void libjxl_av_free(void *opaque, void *address) > +{ > + av_free(address); > +} > + > +void ff_libjxl_init_memory_manager(JxlMemoryManager *manager) > +{ > + manager->opaque = NULL; > + manager->alloc = &libjxl_av_malloc; > + manager->free = &libjxl_av_free; > +} > diff --git a/libavcodec/libjxl.h b/libavcodec/libjxl.h > new file mode 100644 > index 0000000000..5387c438fd > --- /dev/null > +++ b/libavcodec/libjxl.h > @@ -0,0 +1,48 @@ > +/* > + * JPEG XL de/encoding via libjxl, common support header > + * Copyright (c) 2021 Leo Izen <leo.izen@gmail.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +/** > + * @file > + * JPEG XL via libjxl common support header > + */ > + > +#ifndef AVCODEC_LIBJXL_H > +#define AVCODEC_LIBJXL_H > + > +#include <jxl/memory_manager.h> > + > +/** > + * Transform threadcount in ffmpeg to one used by libjxl. > + * > + * @param threads ffmpeg's threads AVOption > + * @return thread count for libjxl's parallel runner > + */ > +size_t ff_libjxl_get_threadcount(int threads); > + > +/** > + * Initialize and populate a JxlMemoryManager > + * with av_malloc() and av_free() so libjxl will use these > + * functions. > + * @param manager a pointer to a JxlMemoryManager struct > + */ > +void ff_libjxl_init_memory_manager(JxlMemoryManager *manager); > + > +#endif /* AVCODEC_LIBJXL_H */ > diff --git a/libavcodec/libjxldec.c b/libavcodec/libjxldec.c > new file mode 100644 > index 0000000000..a7d205b081 > --- /dev/null > +++ b/libavcodec/libjxldec.c > @@ -0,0 +1,274 @@ > +/* > + * JPEG XL decoding support via libjxl > + * Copyright (c) 2021 Leo Izen <leo.izen@gmail.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +/** > + * @file > + * JPEG XL decoder using libjxl > + */ > + > +#include "libavutil/avassert.h" > +#include "libavutil/buffer.h" > +#include "libavutil/common.h" > +#include "libavutil/error.h" > +#include "libavutil/mem.h" > +#include "libavutil/pixdesc.h" > +#include "libavutil/pixfmt.h" > +#include "libavutil/frame.h" > + > +#include "avcodec.h" > +#include "codec_internal.h" > +#include "internal.h" > + > +#include <jxl/decode.h> > +#include <jxl/thread_parallel_runner.h> > +#include "libjxl.h" > + > +typedef struct LibJxlDecodeContext { > + void *runner; > + JxlDecoder *decoder; > + JxlBasicInfo basic_info; > + JxlPixelFormat jxl_pixfmt; > + JxlDecoderStatus events; > + AVBufferRef *iccp; > +} LibJxlDecodeContext; > + > +static int libjxl_init_jxl_decoder(AVCodecContext *avctx) > +{ > + LibJxlDecodeContext *ctx = avctx->priv_data; > + > + ctx->events = JXL_DEC_BASIC_INFO | JXL_DEC_FULL_IMAGE | JXL_DEC_COLOR_ENCODING; > + if (JxlDecoderSubscribeEvents(ctx->decoder, ctx->events) != JXL_DEC_SUCCESS) { > + av_log(avctx, AV_LOG_ERROR, "Error subscribing to JXL events\n"); > + return AVERROR_EXTERNAL; > + } > + > + if (JxlDecoderSetParallelRunner(ctx->decoder, JxlThreadParallelRunner, ctx->runner) != JXL_DEC_SUCCESS) { > + av_log(avctx, AV_LOG_ERROR, "Failed to set JxlThreadParallelRunner\n"); > + return AVERROR_EXTERNAL; > + } > + > + memset(&ctx->basic_info, 0, sizeof(JxlBasicInfo)); > + memset(&ctx->jxl_pixfmt, 0, sizeof(JxlPixelFormat)); > + > + return 0; > +} > + > +static av_cold int libjxl_decode_init(AVCodecContext *avctx) > +{ > + LibJxlDecodeContext *ctx = avctx->priv_data; > + JxlMemoryManager manager; > + > + ff_libjxl_init_memory_manager(&manager); > + ctx->decoder = JxlDecoderCreate(&manager); > + if (!ctx->decoder) { > + av_log(avctx, AV_LOG_ERROR, "Failed to create JxlDecoder\n"); > + return AVERROR_EXTERNAL; > + } > + > + ctx->runner = JxlThreadParallelRunnerCreate(&manager, ff_libjxl_get_threadcount(avctx->thread_count)); > + if (!ctx->runner) { > + av_log(avctx, AV_LOG_ERROR, "Failed to create JxlThreadParallelRunner\n"); > + return AVERROR_EXTERNAL; > + } > + > + return libjxl_init_jxl_decoder(avctx); > +} > + > +static enum AVPixelFormat libjxl_get_pix_fmt(AVCodecContext *avctx, JxlBasicInfo *basic_info, JxlPixelFormat *format) > +{ > + format->endianness = JXL_NATIVE_ENDIAN; > + format->num_channels = basic_info->num_color_channels + (basic_info->alpha_bits > 0); > + /* Gray */ > + if (basic_info->num_color_channels == 1) { > + if (basic_info->bits_per_sample <= 8) { > + format->data_type = JXL_TYPE_UINT8; > + return basic_info->alpha_bits ? AV_PIX_FMT_YA8 : AV_PIX_FMT_GRAY8; > + } > + if (basic_info->exponent_bits_per_sample || basic_info->bits_per_sample > 16) { > + if (basic_info->alpha_bits) > + return AV_PIX_FMT_NONE; > + format->data_type = JXL_TYPE_FLOAT; > + return AV_PIX_FMT_GRAYF32; > + } > + format->data_type = JXL_TYPE_UINT16; > + return basic_info->alpha_bits ? AV_PIX_FMT_YA16 : AV_PIX_FMT_GRAY16; > + } > + /* rgb only */ > + /* libjxl only supports packed RGB and gray output at the moment */ > + if (basic_info->num_color_channels == 3) { > + if (basic_info->bits_per_sample <= 8) { > + format->data_type = JXL_TYPE_UINT8; > + return basic_info->alpha_bits ? AV_PIX_FMT_RGBA : AV_PIX_FMT_RGB24; > + } > + if (basic_info->bits_per_sample > 16) > + av_log(avctx, AV_LOG_WARNING, "Downsampling larger integer to 16-bit via libjxl\n"); > + if (basic_info->exponent_bits_per_sample) > + av_log(avctx, AV_LOG_WARNING, "Downsampling float to 16-bit integer via libjxl\n"); > + format->data_type = JXL_TYPE_UINT16; > + return basic_info->alpha_bits ? AV_PIX_FMT_RGBA64 : AV_PIX_FMT_RGB48; > + } > + > + return AV_PIX_FMT_NONE; > +} > + > +static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_frame, AVPacket *avpkt) > +{ > + LibJxlDecodeContext *ctx = avctx->priv_data; > + uint8_t *buf = avpkt->data; > + size_t remaining = avpkt->size, iccp_len; > + JxlDecoderStatus jret; > + int ret; > + *got_frame = 0; > + > + while (1) { > + > + jret = JxlDecoderSetInput(ctx->decoder, buf, remaining); > + > + if (jret == JXL_DEC_ERROR) { > + /* this should never happen here unless there's a bug in libjxl */ > + av_log(avctx, AV_LOG_ERROR, "Unknown libjxl decode error\n"); > + return AVERROR_EXTERNAL; > + } > + > + jret = JxlDecoderProcessInput(ctx->decoder); > + /* > + * JxlDecoderReleaseInput returns the number > + * of bytes remaining to be read, rather than > + * the number of bytes that it did read > + */ > + remaining = JxlDecoderReleaseInput(ctx->decoder); > + buf = avpkt->data + avpkt->size - remaining; > + > + switch(jret) { > + case JXL_DEC_ERROR: > + av_log(avctx, AV_LOG_ERROR, "Unknown libjxl decode error\n"); > + return AVERROR_INVALIDDATA; > + case JXL_DEC_NEED_MORE_INPUT: > + if (remaining == 0) { > + av_log(avctx, AV_LOG_ERROR, "Unexpected end of JXL codestream\n"); > + return AVERROR_INVALIDDATA; > + } > + av_log(avctx, AV_LOG_DEBUG, "NEED_MORE_INPUT event emitted\n"); > + continue; > + case JXL_DEC_BASIC_INFO: > + av_log(avctx, AV_LOG_DEBUG, "BASIC_INFO event emitted\n"); > + if (JxlDecoderGetBasicInfo(ctx->decoder, &ctx->basic_info) != JXL_DEC_SUCCESS) { > + /* > + * this should never happen > + * if it does it is likely a libjxl decoder bug > + */ > + av_log(avctx, AV_LOG_ERROR, "Bad libjxl basic info event\n"); > + return AVERROR_EXTERNAL; > + } > + avctx->pix_fmt = libjxl_get_pix_fmt(avctx, &ctx->basic_info, &ctx->jxl_pixfmt); > + if (avctx->pix_fmt == AV_PIX_FMT_NONE) { > + av_log(avctx, AV_LOG_ERROR, "Bad libjxl pixel format\n"); > + return AVERROR_EXTERNAL; > + } > + ret = ff_set_dimensions(avctx, ctx->basic_info.xsize, ctx->basic_info.ysize); > + if (ret < 0) > + return ret; > + continue; > + case JXL_DEC_COLOR_ENCODING: > + av_log(avctx, AV_LOG_DEBUG, "COLOR_ENCODING event emitted\n"); > + jret = JxlDecoderGetICCProfileSize(ctx->decoder, &ctx->jxl_pixfmt, JXL_COLOR_PROFILE_TARGET_ORIGINAL, &iccp_len); > + if (jret == JXL_DEC_SUCCESS && iccp_len > 0) { > + av_buffer_unref(&ctx->iccp); > + ctx->iccp = av_buffer_alloc(iccp_len); > + if (!ctx->iccp) > + return AVERROR(ENOMEM); > + jret = JxlDecoderGetColorAsICCProfile(ctx->decoder, &ctx->jxl_pixfmt, JXL_COLOR_PROFILE_TARGET_ORIGINAL, ctx->iccp->data, iccp_len); > + if (jret != JXL_DEC_SUCCESS) > + av_buffer_unref(&ctx->iccp); > + } > + continue; > + case JXL_DEC_NEED_IMAGE_OUT_BUFFER: > + av_log(avctx, AV_LOG_DEBUG, "NEED_IMAGE_OUT_BUFFER event emitted\n"); > + ret = ff_get_buffer(avctx, frame, 0); > + if (ret < 0) > + return ret; > + ctx->jxl_pixfmt.align = frame->linesize[0]; > + if (JxlDecoderSetImageOutBuffer(ctx->decoder, &ctx->jxl_pixfmt, frame->data[0], frame->buf[0]->size) != JXL_DEC_SUCCESS) { > + av_log(avctx, AV_LOG_ERROR, "Bad libjxl dec need image out buffer event\n"); > + return AVERROR_EXTERNAL; > + } > + continue; > + case JXL_DEC_FULL_IMAGE: > + /* full image is one frame, even if animated */ > + av_log(avctx, AV_LOG_DEBUG, "FULL_IMAGE event emitted\n"); > + *got_frame = 1; Only set this after you know that you return successfully (i.e. after the av_frame_new_side_data_from_buf()). > + frame->pict_type = AV_PICTURE_TYPE_I; > + frame->key_frame = 1; > + if (ctx->iccp) { > + AVFrameSideData *sd = av_frame_new_side_data_from_buf(frame, AV_FRAME_DATA_ICC_PROFILE, ctx->iccp); > + if (!sd) > + return AVERROR(ENOMEM); > + /* ownership is transfered, and it is not ref-ed */ > + ctx->iccp = NULL; > + } > + return avpkt->size - remaining; If this decoder is supposed to produce multiple frames from one packet of input, it needs to use the receive_frame-callback. For video decoders, it is only checked whether the return value is >= 0 or not (see lines 451-456 in decode.c). > + case JXL_DEC_SUCCESS: > + av_log(avctx, AV_LOG_DEBUG, "SUCCESS event emitted\n"); > + /* > + * The file has finished decoding > + * reset the decoder to let us > + * reuse it again for the next image > + */ > + JxlDecoderReset(ctx->decoder); > + libjxl_init_jxl_decoder(avctx); You are only resetting the decoder on success. What happens if any of the errors happened? Would the decoder need to be reset before decoding the next picture? > + buf = avpkt->data; > + remaining = avpkt->size; > + continue; Didn't you already feed this data to the decoder? This somehow looks like a recipe for an infinite loop. > + default: > + av_log(avctx, AV_LOG_ERROR, "Bad libjxl event: %d\n", jret); > + return AVERROR_EXTERNAL; > + } > + } > +} > + > +static av_cold int libjxl_decode_close(AVCodecContext *avctx) > +{ > + LibJxlDecodeContext *ctx = avctx->priv_data; > + > + if (ctx->runner) > + JxlThreadParallelRunnerDestroy(ctx->runner); > + ctx->runner = NULL; > + if (ctx->decoder) > + JxlDecoderDestroy(ctx->decoder); > + ctx->decoder = NULL; > + av_buffer_unref(&ctx->iccp); > + > + return 0; > +} > + > +const FFCodec ff_libjxl_decoder = { > + .p.name = "libjxl", > + .p.long_name = NULL_IF_CONFIG_SMALL("libjxl JPEG XL"), > + .p.type = AVMEDIA_TYPE_VIDEO, > + .p.id = AV_CODEC_ID_JPEGXL, > + .priv_data_size = sizeof(LibJxlDecodeContext), > + .init = libjxl_decode_init, > + FF_CODEC_DECODE_CB(libjxl_decode_frame), > + .close = libjxl_decode_close, > + .p.capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_OTHER_THREADS, > + .caps_internal = FF_CODEC_CAP_AUTO_THREADS | FF_CODEC_CAP_INIT_CLEANUP, > + .p.wrapper_name = "libjxl", > +}; - Andreas PS: I apologize in advance if any of the above points has already been raised and shown to be non-issues. _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [FFmpeg-devel] [PATCH v14 2/4] avcodec/libjxl: add Jpeg XL decoding via libjxl 2022-04-15 11:08 ` Andreas Rheinhardt @ 2022-04-15 19:31 ` Leo Izen 0 siblings, 0 replies; 14+ messages in thread From: Leo Izen @ 2022-04-15 19:31 UTC (permalink / raw) To: ffmpeg-devel On 4/15/22 07:08, Andreas Rheinhardt wrote: > Leo Izen: > >> + return avpkt->size - remaining; > If this decoder is supposed to produce multiple frames from one packet > of input, it needs to use the receive_frame-callback. For video > decoders, it is only checked whether the return value is >= 0 or not > (see lines 451-456 in decode.c). It isn't, because I've chosen not to support animated JXL at this time. As far as I'm aware an animated JXL file will just decode to its first frame with this decoder. >> + case JXL_DEC_SUCCESS: >> + av_log(avctx, AV_LOG_DEBUG, "SUCCESS event emitted\n"); >> + /* >> + * The file has finished decoding >> + * reset the decoder to let us >> + * reuse it again for the next image >> + */ >> + JxlDecoderReset(ctx->decoder); >> + libjxl_init_jxl_decoder(avctx); > You are only resetting the decoder on success. What happens if any of > the errors happened? Would the decoder need to be reset before decoding > the next picture? This was already brought up by Anton. The JXL_DEC_SUCCESS event isn't fired until the start of the next packet (if there is one). It is the "finished decoding" event which we never actually get to for single images because JXL_DEC_FULL_IMAGE is always fired first. If this is fired it means we're in an image2 sequence and this is the next frame, at which point the data will have been swapped out. Resetting the decoder is necessary here to make the library re-read the header info and not whine. I agree that it's a bit confusing and I should probably add a block comment explaining this. >> + buf = avpkt->data; >> + remaining = avpkt->size; >> + continue; > Didn't you already feed this data to the decoder? This somehow looks > like a recipe for an infinite loop. No, see up. - Leo Izen (thebombzen) _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 14+ messages in thread
* [FFmpeg-devel] [PATCH v14 3/4] avcodec/libjxl: add Jpeg XL encoding via libjxl 2022-04-12 5:53 [FFmpeg-devel] [PATCH v14 0/4] Jpeg XL Patch Set Leo Izen 2022-04-12 5:53 ` [FFmpeg-devel] [PATCH v14 1/4] avcodec/jpegxl: add Jpeg XL image codec Leo Izen 2022-04-12 5:53 ` [FFmpeg-devel] [PATCH v14 2/4] avcodec/libjxl: add Jpeg XL decoding via libjxl Leo Izen @ 2022-04-12 5:53 ` Leo Izen 2022-04-14 17:49 ` Anton Khirnov 2022-04-12 5:53 ` [FFmpeg-devel] [PATCH v14 4/4] avformat/image2: add Jpeg XL as image2 format Leo Izen 3 siblings, 1 reply; 14+ messages in thread From: Leo Izen @ 2022-04-12 5:53 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Leo Izen This commit adds encoding support to libavcodec for Jpeg XL images via the external library libjxl. --- configure | 3 +- libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 1 + libavcodec/libjxlenc.c | 382 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 386 insertions(+), 1 deletion(-) create mode 100644 libavcodec/libjxlenc.c diff --git a/configure b/configure index c36d5dbc8b..76bd8857db 100755 --- a/configure +++ b/configure @@ -240,7 +240,7 @@ External library support: --enable-libiec61883 enable iec61883 via libiec61883 [no] --enable-libilbc enable iLBC de/encoding via libilbc [no] --enable-libjack enable JACK audio sound server [no] - --enable-libjxl enable JPEG XL decoding via libjxl [no] + --enable-libjxl enable JPEG XL de/encoding via libjxl [no] --enable-libklvanc enable Kernel Labs VANC processing [no] --enable-libkvazaar enable HEVC encoding via libkvazaar [no] --enable-liblensfun enable lensfun lens correction [no] @@ -3334,6 +3334,7 @@ libgsm_ms_encoder_deps="libgsm" libilbc_decoder_deps="libilbc" libilbc_encoder_deps="libilbc" libjxl_decoder_deps="libjxl libjxl_threads" +libjxl_encoder_deps="libjxl libjxl_threads" libkvazaar_encoder_deps="libkvazaar" libmodplug_demuxer_deps="libmodplug" libmp3lame_encoder_deps="libmp3lame" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 8fc3b3cc5f..a8cb606186 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1062,6 +1062,7 @@ OBJS-$(CONFIG_LIBGSM_MS_ENCODER) += libgsmenc.o OBJS-$(CONFIG_LIBILBC_DECODER) += libilbc.o OBJS-$(CONFIG_LIBILBC_ENCODER) += libilbc.o OBJS-$(CONFIG_LIBJXL_DECODER) += libjxldec.o libjxl.o +OBJS-$(CONFIG_LIBJXL_ENCODER) += libjxlenc.o libjxl.o OBJS-$(CONFIG_LIBKVAZAAR_ENCODER) += libkvazaar.o OBJS-$(CONFIG_LIBMP3LAME_ENCODER) += libmp3lame.o OBJS-$(CONFIG_LIBOPENCORE_AMRNB_DECODER) += libopencore-amr.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 07f5bafd27..c47133aa18 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -752,6 +752,7 @@ extern const FFCodec ff_libgsm_ms_decoder; extern const FFCodec ff_libilbc_encoder; extern const FFCodec ff_libilbc_decoder; extern const FFCodec ff_libjxl_decoder; +extern const FFCodec ff_libjxl_encoder; extern const FFCodec ff_libmp3lame_encoder; extern const FFCodec ff_libopencore_amrnb_encoder; extern const FFCodec ff_libopencore_amrnb_decoder; diff --git a/libavcodec/libjxlenc.c b/libavcodec/libjxlenc.c new file mode 100644 index 0000000000..a20de91661 --- /dev/null +++ b/libavcodec/libjxlenc.c @@ -0,0 +1,382 @@ +/* + * JPEG XL encoding support via libjxl + * Copyright (c) 2021 Leo Izen <leo.izen@gmail.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * JPEG XL encoder using libjxl + */ + +#include <string.h> + +#include "libavutil/avutil.h" +#include "libavutil/error.h" +#include "libavutil/frame.h" +#include "libavutil/libm.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/version.h" + +#include "avcodec.h" +#include "encode.h" +#include "codec_internal.h" + +#include <jxl/encode.h> +#include <jxl/thread_parallel_runner.h> +#include "libjxl.h" + +typedef struct LibJxlEncodeContext { + AVClass *class; + void *runner; + JxlEncoder *encoder; + JxlEncoderFrameSettings *options; + int effort; + float distance; + int modular; + uint8_t *buffer; + size_t buffer_size; +} LibJxlEncodeContext; + +/** + * Map a quality setting for -qscale roughly from libjpeg + * quality numbers to libjxl's butteraugli distance for + * photographic content. + * + * Setting distance explicitly is preferred, but this will + * allow qscale to be used as a fallback. + * + * This function is continuous and injective on [0, 100] which + * makes it monotonic. + * + * @param quality 0.0 to 100.0 quality setting, libjpeg quality + * @return Butteraugli distance between 0.0 and 15.0 + */ +static float quality_to_distance(float quality) +{ + if (quality >= 100.0) + return 0.0; + else if (quality >= 90.0) + return (100.0 - quality) * 0.10; + else if (quality >= 30.0) + return 0.1 + (100.0 - quality) * 0.09; + else if (quality > 0.0) + return 15.0 + (59.0 * quality - 4350.0) * quality / 9000.0; + else + return 15.0; +} + +/** + * Initalize the decoder on a per-frame basis. All of these need to be set + * once each time the decoder is reset, which it must be each frame to make + * the image2 muxer work. + * + * @return 0 upon success, negative on failure. + */ +static int libjxl_init_jxl_encoder(AVCodecContext *avctx) +{ + LibJxlEncodeContext *ctx = avctx->priv_data; + + /* reset the encoder every frame for image2 muxer */ + JxlEncoderReset(ctx->encoder); + + ctx->options = JxlEncoderFrameSettingsCreate(ctx->encoder, NULL); + if (!ctx->options) { + av_log(avctx, AV_LOG_ERROR, "Failed to create JxlEncoderOptions\n"); + return AVERROR_EXTERNAL; + } + + /* This needs to be set each time the decoder is reset */ + if (JxlEncoderSetParallelRunner(ctx->encoder, JxlThreadParallelRunner, ctx->runner) + != JXL_ENC_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to set JxlThreadParallelRunner\n"); + return AVERROR_EXTERNAL; + } + + /* these shouldn't fail, libjxl bug notwithstanding */ + if (JxlEncoderFrameSettingsSetOption(ctx->options, JXL_ENC_FRAME_SETTING_EFFORT, ctx->effort) + != JXL_ENC_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to set effort to: %d\n", ctx->effort); + return AVERROR_EXTERNAL; + } + + /* check for negative zero, our default */ + if (ctx->distance < 0.0) { + /* use ffmpeg.c -q option if passed */ + if (avctx->flags & AV_CODEC_FLAG_QSCALE) + ctx->distance = quality_to_distance((float)avctx->global_quality / FF_QP2LAMBDA); + else + /* default 1.0 matches cjxl */ + ctx->distance = 1.0; + } + + /* + * 0.01 is the minimum distance accepted for lossy + * interpreting any positive value less than this as minimum + */ + if (ctx->distance > 0.0 && ctx->distance < 0.01) + ctx->distance = 0.01; + if (JxlEncoderOptionsSetDistance(ctx->options, ctx->distance) != JXL_ENC_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to set distance: %f\n", ctx->distance); + return AVERROR_EXTERNAL; + } + + /* + * In theory the library should automatically enable modular if necessary, + * but it appears it won't at the moment due to a bug. This will still + * work even if that is patched. + */ + if (JxlEncoderFrameSettingsSetOption(ctx->options, JXL_ENC_FRAME_SETTING_MODULAR, + ctx->modular || ctx->distance <= 0.0 ? 1 : -1) != JXL_ENC_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to set modular\n"); + return AVERROR_EXTERNAL; + } + + return 0; +} + +/** + * Global encoder initialization. This only needs to be run once, + * not every frame. + */ +static av_cold int libjxl_encode_init(AVCodecContext *avctx) +{ + LibJxlEncodeContext *ctx = avctx->priv_data; + JxlMemoryManager manager; + + ff_libjxl_init_memory_manager(&manager); + ctx->encoder = JxlEncoderCreate(&manager); + if (!ctx->encoder) { + av_log(avctx, AV_LOG_ERROR, "Failed to create JxlEncoder\n"); + return AVERROR_EXTERNAL; + } + + ctx->runner = JxlThreadParallelRunnerCreate(&manager, ff_libjxl_get_threadcount(avctx->thread_count)); + if (!ctx->runner) { + av_log(avctx, AV_LOG_ERROR, "Failed to create JxlThreadParallelRunner\n"); + return AVERROR_EXTERNAL; + } + + ctx->buffer_size = 4096; + ctx->buffer = av_realloc(NULL, ctx->buffer_size); + + if (!ctx->buffer) { + av_log(avctx, AV_LOG_ERROR, "Could not allocate encoding buffer\n"); + return AVERROR(ENOMEM); + } + + return 0; +} + +/** + * Encode an entire frame. Currently animation, is not supported by + * this encoder, so this will always reinitialize a new still image + * and encode a one-frame image (for image2 and image2pipe). + */ +static int libjxl_encode_frame(AVCodecContext *avctx, AVPacket *pkt, const AVFrame *frame, int *got_packet) +{ + LibJxlEncodeContext *ctx = avctx->priv_data; + AVFrameSideData *sd; + const AVPixFmtDescriptor *pix_desc = av_pix_fmt_desc_get(frame->format); + JxlBasicInfo info; + JxlColorEncoding jxl_color; + JxlPixelFormat jxl_fmt; + JxlEncoderStatus jret; + int ret; + size_t available = ctx->buffer_size; + size_t bytes_written = 0; + uint8_t *next_out = ctx->buffer; + + ret = libjxl_init_jxl_encoder(avctx); + if (ret) { + av_log(avctx, AV_LOG_ERROR, "Error frame-initializing JxlEncoder\n"); + return ret; + } + + /* populate the basic info settings */ + JxlEncoderInitBasicInfo(&info); + jxl_fmt.num_channels = pix_desc->nb_components; + info.xsize = frame->width; + info.ysize = frame->height; + info.num_extra_channels = (jxl_fmt.num_channels + 1) % 2; + info.num_color_channels = jxl_fmt.num_channels - info.num_extra_channels; + info.bits_per_sample = av_get_bits_per_pixel(pix_desc) / jxl_fmt.num_channels; + info.alpha_bits = (info.num_extra_channels > 0) * info.bits_per_sample; + if (pix_desc->flags & AV_PIX_FMT_FLAG_FLOAT) { + info.exponent_bits_per_sample = info.bits_per_sample > 16 ? 8 : 5; + info.alpha_exponent_bits = info.alpha_bits ? info.exponent_bits_per_sample : 0; + jxl_fmt.data_type = info.bits_per_sample > 16 ? JXL_TYPE_FLOAT : JXL_TYPE_FLOAT16; + JxlColorEncodingSetToLinearSRGB(&jxl_color, info.num_color_channels == 1); + } else { + info.exponent_bits_per_sample = 0; + info.alpha_exponent_bits = 0; + jxl_fmt.data_type = info.bits_per_sample <= 8 ? JXL_TYPE_UINT8 : JXL_TYPE_UINT16; + JxlColorEncodingSetToSRGB(&jxl_color, info.num_color_channels == 1); + } + + if (info.bits_per_sample > 16 + || info.xsize > (1 << 18) || info.ysize > (1 << 18) + || (info.xsize << 4) * (info.ysize << 4) > (1 << 20)) { + /* + * must upgrade codestream to level 10, from level 5 + * the encoder will not do this automatically + */ + if (JxlEncoderSetCodestreamLevel(ctx->encoder, 10) != JXL_ENC_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Could not upgrade JXL Codestream level.\n"); + return AVERROR_EXTERNAL; + } + } + + /* bitexact lossless requires there to be no XYB transform */ + info.uses_original_profile = ctx->distance == 0.0; + + sd = av_frame_get_side_data(frame, AV_FRAME_DATA_ICC_PROFILE); + if (sd && sd->size && JxlEncoderSetICCProfile(ctx->encoder, sd->data, sd->size) != JXL_ENC_SUCCESS) { + av_log(avctx, AV_LOG_WARNING, "Could not set ICC Profile\n"); + } else if (info.uses_original_profile) { + /* + * the color encoding is not used if uses_original_profile is false + * this just works around a bug in libjxl 0.7.0 and lower + */ + if (JxlEncoderSetColorEncoding(ctx->encoder, &jxl_color) != JXL_ENC_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to set JxlColorEncoding\n"); + return AVERROR_EXTERNAL; + } + } + + if (JxlEncoderSetBasicInfo(ctx->encoder, &info) != JXL_ENC_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to set JxlBasicInfo\n"); + return AVERROR_EXTERNAL; + } + + jxl_fmt.endianness = JXL_NATIVE_ENDIAN; + jxl_fmt.align = frame->linesize[0]; + + if (JxlEncoderAddImageFrame(ctx->options, &jxl_fmt, frame->data[0], jxl_fmt.align * info.ysize) != JXL_ENC_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to add Image Frame\n"); + return AVERROR_EXTERNAL; + } + + /* + * Run this after the last frame in the image has been passed. + * TODO support animation + */ + JxlEncoderCloseInput(ctx->encoder); + + while (1) { + jret = JxlEncoderProcessOutput(ctx->encoder, &next_out, &available); + if (jret == JXL_ENC_ERROR) { + av_log(avctx, AV_LOG_ERROR, "Unspecified libjxl error occurred\n"); + return AVERROR_EXTERNAL; + } + bytes_written = ctx->buffer_size - available; + /* all data passed has been encoded */ + if (jret == JXL_ENC_SUCCESS) + break; + if (jret == JXL_ENC_NEED_MORE_OUTPUT) { + /* + * at the moment, libjxl has no way to + * tell us how much space it actually needs + * so we need to malloc loop + */ + uint8_t *temp; + size_t new_size = ctx->buffer_size * 2; + temp = av_realloc(ctx->buffer, new_size); + if (!temp) + return AVERROR(ENOMEM); + ctx->buffer = temp; + ctx->buffer_size = new_size; + next_out = ctx->buffer + bytes_written; + available = new_size - bytes_written; + continue; + } + } + + ret = ff_get_encode_buffer(avctx, pkt, bytes_written, 0); + if (ret < 0) + return ret; + + memcpy(pkt->data, ctx->buffer, bytes_written); + *got_packet = 1; + + return 0; +} + +static av_cold int libjxl_encode_close(AVCodecContext *avctx) +{ + LibJxlEncodeContext *ctx = avctx->priv_data; + + if (ctx->runner) + JxlThreadParallelRunnerDestroy(ctx->runner); + ctx->runner = NULL; + + /* + * destroying the decoder also frees + * ctx->options so we don't need to + */ + if (ctx->encoder) + JxlEncoderDestroy(ctx->encoder); + ctx->encoder = NULL; + + av_freep(&ctx->buffer); + + return 0; +} + +#define OFFSET(x) offsetof(LibJxlEncodeContext, x) +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM + +static const AVOption libjxl_encode_options[] = { + { "effort", "Encoding effort", OFFSET(effort), AV_OPT_TYPE_INT, { .i64 = 7 }, 1, 9, VE }, + { "distance", "Maximum Butteraugli distance (quality setting, " + "lower = better, zero = lossless, default 1.0)", OFFSET(distance), AV_OPT_TYPE_FLOAT, { .dbl = -1.0 }, -1.0, 15.0, VE }, + { "modular", "Force modular mode", OFFSET(modular), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE }, + { NULL }, +}; + +static const AVClass libjxl_encode_class = { + .class_name = "libjxl", + .item_name = av_default_item_name, + .option = libjxl_encode_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFCodec ff_libjxl_encoder = { + .p.name = "libjxl", + .p.long_name = NULL_IF_CONFIG_SMALL("libjxl JPEG XL"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_JPEGXL, + .priv_data_size = sizeof(LibJxlEncodeContext), + .init = libjxl_encode_init, + FF_CODEC_ENCODE_CB(libjxl_encode_frame), + .close = libjxl_encode_close, + .p.capabilities = AV_CODEC_CAP_OTHER_THREADS, + .caps_internal = FF_CODEC_CAP_AUTO_THREADS | FF_CODEC_CAP_INIT_CLEANUP, + .p.pix_fmts = (const enum AVPixelFormat[]) { + AV_PIX_FMT_RGB24, AV_PIX_FMT_RGBA, + AV_PIX_FMT_RGB48, AV_PIX_FMT_RGBA64, + AV_PIX_FMT_GRAY8, AV_PIX_FMT_YA8, + AV_PIX_FMT_GRAY16, AV_PIX_FMT_YA16, + AV_PIX_FMT_GRAYF32, + AV_PIX_FMT_NONE + }, + .p.priv_class = &libjxl_encode_class, + .p.wrapper_name = "libjxl", +}; -- 2.35.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [FFmpeg-devel] [PATCH v14 3/4] avcodec/libjxl: add Jpeg XL encoding via libjxl 2022-04-12 5:53 ` [FFmpeg-devel] [PATCH v14 3/4] avcodec/libjxl: add Jpeg XL encoding " Leo Izen @ 2022-04-14 17:49 ` Anton Khirnov 2022-04-15 1:37 ` Leo Izen 0 siblings, 1 reply; 14+ messages in thread From: Anton Khirnov @ 2022-04-14 17:49 UTC (permalink / raw) To: FFmpeg development discussions and patches; +Cc: Leo Izen Quoting Leo Izen (2022-04-12 07:53:32) > + > + while (1) { > + jret = JxlEncoderProcessOutput(ctx->encoder, &next_out, &available); > + if (jret == JXL_ENC_ERROR) { > + av_log(avctx, AV_LOG_ERROR, "Unspecified libjxl error occurred\n"); > + return AVERROR_EXTERNAL; > + } > + bytes_written = ctx->buffer_size - available; > + /* all data passed has been encoded */ > + if (jret == JXL_ENC_SUCCESS) > + break; > + if (jret == JXL_ENC_NEED_MORE_OUTPUT) { Is it possible for the encoder to return anything other than the three codes you're handling? Won't you get into an infinite loop if it does? -- Anton Khirnov _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [FFmpeg-devel] [PATCH v14 3/4] avcodec/libjxl: add Jpeg XL encoding via libjxl 2022-04-14 17:49 ` Anton Khirnov @ 2022-04-15 1:37 ` Leo Izen 2022-04-15 10:39 ` Anton Khirnov 0 siblings, 1 reply; 14+ messages in thread From: Leo Izen @ 2022-04-15 1:37 UTC (permalink / raw) To: FFmpeg development discussions and patches On 4/14/22 13:49, Anton Khirnov wrote: > Quoting Leo Izen (2022-04-12 07:53:32) >> + >> + while (1) { >> + jret = JxlEncoderProcessOutput(ctx->encoder, &next_out, &available); >> + if (jret == JXL_ENC_ERROR) { >> + av_log(avctx, AV_LOG_ERROR, "Unspecified libjxl error occurred\n"); >> + return AVERROR_EXTERNAL; >> + } >> + bytes_written = ctx->buffer_size - available; >> + /* all data passed has been encoded */ >> + if (jret == JXL_ENC_SUCCESS) >> + break; >> + if (jret == JXL_ENC_NEED_MORE_OUTPUT) { > Is it possible for the encoder to return anything other than the three > codes you're handling? Won't you get into an infinite loop if it does? > It's just these three, according to the documentation. https://libjxl.readthedocs.io/en/latest/api_encoder.html -Leo Izen (thebombzen) _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [FFmpeg-devel] [PATCH v14 3/4] avcodec/libjxl: add Jpeg XL encoding via libjxl 2022-04-15 1:37 ` Leo Izen @ 2022-04-15 10:39 ` Anton Khirnov 2022-04-15 19:32 ` Leo Izen 0 siblings, 1 reply; 14+ messages in thread From: Anton Khirnov @ 2022-04-15 10:39 UTC (permalink / raw) To: FFmpeg development discussions and patches Quoting Leo Izen (2022-04-15 03:37:20) > > On 4/14/22 13:49, Anton Khirnov wrote: > > Quoting Leo Izen (2022-04-12 07:53:32) > >> + > >> + while (1) { > >> + jret = JxlEncoderProcessOutput(ctx->encoder, &next_out, &available); > >> + if (jret == JXL_ENC_ERROR) { > >> + av_log(avctx, AV_LOG_ERROR, "Unspecified libjxl error occurred\n"); > >> + return AVERROR_EXTERNAL; > >> + } > >> + bytes_written = ctx->buffer_size - available; > >> + /* all data passed has been encoded */ > >> + if (jret == JXL_ENC_SUCCESS) > >> + break; > >> + if (jret == JXL_ENC_NEED_MORE_OUTPUT) { > > Is it possible for the encoder to return anything other than the three > > codes you're handling? Won't you get into an infinite loop if it does? > > > It's just these three, according to the documentation. > https://libjxl.readthedocs.io/en/latest/api_encoder.html So the end of the loop should be unreachable, as you always break, continue or return before. I would put return AVERROR_EXTERNAL there just in case. Beyond that I have no more comments on the set. -- Anton Khirnov _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [FFmpeg-devel] [PATCH v14 3/4] avcodec/libjxl: add Jpeg XL encoding via libjxl 2022-04-15 10:39 ` Anton Khirnov @ 2022-04-15 19:32 ` Leo Izen 0 siblings, 0 replies; 14+ messages in thread From: Leo Izen @ 2022-04-15 19:32 UTC (permalink / raw) To: ffmpeg-devel On 4/15/22 06:39, Anton Khirnov wrote: > Quoting Leo Izen (2022-04-15 03:37:20) >> On 4/14/22 13:49, Anton Khirnov wrote: >>> Quoting Leo Izen (2022-04-12 07:53:32) >>>> + >>>> + while (1) { >>>> + jret = JxlEncoderProcessOutput(ctx->encoder, &next_out, &available); >>>> + if (jret == JXL_ENC_ERROR) { >>>> + av_log(avctx, AV_LOG_ERROR, "Unspecified libjxl error occurred\n"); >>>> + return AVERROR_EXTERNAL; >>>> + } >>>> + bytes_written = ctx->buffer_size - available; >>>> + /* all data passed has been encoded */ >>>> + if (jret == JXL_ENC_SUCCESS) >>>> + break; >>>> + if (jret == JXL_ENC_NEED_MORE_OUTPUT) { >>> Is it possible for the encoder to return anything other than the three >>> codes you're handling? Won't you get into an infinite loop if it does? >>> >> It's just these three, according to the documentation. >> https://libjxl.readthedocs.io/en/latest/api_encoder.html > So the end of the loop should be unreachable, as you always break, > continue or return before. I would put return AVERROR_EXTERNAL there > just in case. > > Beyond that I have no more comments on the set. > I suppose a failsafe in case of a bug or futureproofing wouldn't be a bad call. I can add one. - Leo Izen (thebombzen) _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 14+ messages in thread
* [FFmpeg-devel] [PATCH v14 4/4] avformat/image2: add Jpeg XL as image2 format 2022-04-12 5:53 [FFmpeg-devel] [PATCH v14 0/4] Jpeg XL Patch Set Leo Izen ` (2 preceding siblings ...) 2022-04-12 5:53 ` [FFmpeg-devel] [PATCH v14 3/4] avcodec/libjxl: add Jpeg XL encoding " Leo Izen @ 2022-04-12 5:53 ` Leo Izen 2022-04-15 11:34 ` Andreas Rheinhardt 3 siblings, 1 reply; 14+ messages in thread From: Leo Izen @ 2022-04-12 5:53 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Leo Izen This commit adds support to libavformat for muxing and demuxing Jpeg XL images as image2 streams. --- MAINTAINERS | 1 + libavformat/Makefile | 1 + libavformat/allformats.c | 1 + libavformat/img2.c | 1 + libavformat/img2dec.c | 18 ++ libavformat/img2enc.c | 6 +- libavformat/jpegxl_probe.c | 393 +++++++++++++++++++++++++++++++++++++ libavformat/jpegxl_probe.h | 32 +++ libavformat/mov.c | 1 + 9 files changed, 451 insertions(+), 3 deletions(-) create mode 100644 libavformat/jpegxl_probe.c create mode 100644 libavformat/jpegxl_probe.h diff --git a/MAINTAINERS b/MAINTAINERS index faea84ebf1..46723972dc 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -439,6 +439,7 @@ Muxers/Demuxers: ipmovie.c Mike Melanson ircam* Paul B Mahol iss.c Stefan Gehrer + jpegxl_probe.* Leo Izen jvdec.c Peter Ross kvag.c Zane van Iperen libmodplug.c Clément Bœsch diff --git a/libavformat/Makefile b/libavformat/Makefile index d7182d6bd8..beecdf5a66 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -272,6 +272,7 @@ OBJS-$(CONFIG_IMAGE_GIF_PIPE_DEMUXER) += img2dec.o img2.o OBJS-$(CONFIG_IMAGE_J2K_PIPE_DEMUXER) += img2dec.o img2.o OBJS-$(CONFIG_IMAGE_JPEG_PIPE_DEMUXER) += img2dec.o img2.o OBJS-$(CONFIG_IMAGE_JPEGLS_PIPE_DEMUXER) += img2dec.o img2.o +OBJS-$(CONFIG_IMAGE_JPEGXL_PIPE_DEMUXER) += img2dec.o img2.o jpegxl_probe.o OBJS-$(CONFIG_IMAGE_PAM_PIPE_DEMUXER) += img2dec.o img2.o OBJS-$(CONFIG_IMAGE_PBM_PIPE_DEMUXER) += img2dec.o img2.o OBJS-$(CONFIG_IMAGE_PCX_PIPE_DEMUXER) += img2dec.o img2.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index 7c1d0ac38f..63876c468f 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -510,6 +510,7 @@ extern const AVInputFormat ff_image_gif_pipe_demuxer; extern const AVInputFormat ff_image_j2k_pipe_demuxer; extern const AVInputFormat ff_image_jpeg_pipe_demuxer; extern const AVInputFormat ff_image_jpegls_pipe_demuxer; +extern const AVInputFormat ff_image_jpegxl_pipe_demuxer; extern const AVInputFormat ff_image_pam_pipe_demuxer; extern const AVInputFormat ff_image_pbm_pipe_demuxer; extern const AVInputFormat ff_image_pcx_pipe_demuxer; diff --git a/libavformat/img2.c b/libavformat/img2.c index fe2ca7bfff..566ef873ca 100644 --- a/libavformat/img2.c +++ b/libavformat/img2.c @@ -88,6 +88,7 @@ const IdStrMap ff_img_tags[] = { { AV_CODEC_ID_GEM, "ximg" }, { AV_CODEC_ID_GEM, "timg" }, { AV_CODEC_ID_VBN, "vbn" }, + { AV_CODEC_ID_JPEGXL, "jxl" }, { AV_CODEC_ID_NONE, NULL } }; diff --git a/libavformat/img2dec.c b/libavformat/img2dec.c index 551b9d508e..627bb67212 100644 --- a/libavformat/img2dec.c +++ b/libavformat/img2dec.c @@ -36,6 +36,7 @@ #include "avio_internal.h" #include "internal.h" #include "img2.h" +#include "jpegxl_probe.h" #include "libavcodec/mjpeg.h" #include "libavcodec/vbn.h" #include "libavcodec/xwd.h" @@ -837,6 +838,22 @@ static int jpegls_probe(const AVProbeData *p) return 0; } +static int jpegxl_probe(const AVProbeData *p) +{ + const uint8_t *b = p->buf; + + /* ISOBMFF-based container */ + /* 0x4a584c20 == "JXL " */ + if (AV_RL64(b) == FF_JPEGXL_CONTAINER_SIGNATURE_LE) + return AVPROBE_SCORE_EXTENSION + 1; + /* Raw codestreams all start with 0xff0a */ + if (AV_RL16(b) != FF_JPEGXL_CODESTREAM_SIGNATURE_LE) + return 0; + if (ff_jpegxl_verify_codestream_header(p->buf, p->buf_size) >= 0) + return AVPROBE_SCORE_MAX - 2; + return 0; +} + static int pcx_probe(const AVProbeData *p) { const uint8_t *b = p->buf; @@ -1176,6 +1193,7 @@ IMAGEAUTO_DEMUXER(gif, GIF) IMAGEAUTO_DEMUXER_EXT(j2k, JPEG2000, J2K) IMAGEAUTO_DEMUXER_EXT(jpeg, MJPEG, JPEG) IMAGEAUTO_DEMUXER(jpegls, JPEGLS) +IMAGEAUTO_DEMUXER(jpegxl, JPEGXL) IMAGEAUTO_DEMUXER(pam, PAM) IMAGEAUTO_DEMUXER(pbm, PBM) IMAGEAUTO_DEMUXER(pcx, PCX) diff --git a/libavformat/img2enc.c b/libavformat/img2enc.c index ae351963d9..5ed97bb833 100644 --- a/libavformat/img2enc.c +++ b/libavformat/img2enc.c @@ -263,9 +263,9 @@ static const AVClass img2mux_class = { const AVOutputFormat ff_image2_muxer = { .name = "image2", .long_name = NULL_IF_CONFIG_SMALL("image2 sequence"), - .extensions = "bmp,dpx,exr,jls,jpeg,jpg,ljpg,pam,pbm,pcx,pfm,pgm,pgmyuv,png," - "ppm,sgi,tga,tif,tiff,jp2,j2c,j2k,xwd,sun,ras,rs,im1,im8,im24," - "sunras,vbn,xbm,xface,pix,y", + .extensions = "bmp,dpx,exr,jls,jpeg,jpg,jxl,ljpg,pam,pbm,pcx,pfm,pgm,pgmyuv," + "png,ppm,sgi,tga,tif,tiff,jp2,j2c,j2k,xwd,sun,ras,rs,im1,im8," + "im24,sunras,vbn,xbm,xface,pix,y", .priv_data_size = sizeof(VideoMuxData), .video_codec = AV_CODEC_ID_MJPEG, .write_header = write_header, diff --git a/libavformat/jpegxl_probe.c b/libavformat/jpegxl_probe.c new file mode 100644 index 0000000000..d3d3822fee --- /dev/null +++ b/libavformat/jpegxl_probe.c @@ -0,0 +1,393 @@ +/* + * Jpeg XL header verification + * Copyright (c) 2022 Leo Izen <leo.izen@gmail.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "jpegxl_probe.h" + +#define BITSTREAM_READER_LE +#include "libavcodec/get_bits.h" + +enum JpegXLExtraChannelType { + FF_JPEGXL_CT_ALPHA = 0, + FF_JPEGXL_CT_DEPTH, + FF_JPEGXL_CT_SPOT_COLOR, + FF_JPEGXL_CT_SELECTION_MASK, + FF_JPEGXL_CT_BLACK, + FF_JPEGXL_CT_CFA, + FF_JPEGXL_CT_THERMAL, + FF_JPEGXL_CT_NON_OPTIONAL = 15, + FF_JPEGXL_CT_OPTIONAL +}; + +enum JpegXLColorSpace { + FF_JPEGXL_CS_RGB = 0, + FF_JPEGXL_CS_GRAY, + FF_JPEGXL_CS_XYB, + FF_JPEGXL_CS_UNKNOWN +}; + +enum JpegXLWhitePoint { + FF_JPEGXL_WP_D65 = 1, + FF_JPEGXL_WP_CUSTOM, + FF_JPEGXL_WP_E = 10, + FF_JPEGXL_WP_DCI = 11 +}; + +enum JpegXLPrimaries { + FF_JPEGXL_PR_SRGB = 1, + FF_JPEGXL_PR_CUSTOM, + FF_JPEGXL_PR_2100 = 9, + FF_JPEGXL_PR_P3 = 11, +}; + +#define jxl_bits(n) get_bits_long(gb, (n)) +#define jxl_bits_skip(n) skip_bits_long(gb, (n)) +#define jxl_u32(c0, c1, c2, c3, u0, u1, u2, u3) jpegxl_u32(gb, \ + (const uint32_t[]){c0, c1, c2, c3}, (const uint32_t[]){u0, u1, u2, u3}) +#define jxl_u64() jpegxl_u64(gb) +#define jxl_enum() jxl_u32(0, 1, 2, 18, 0, 0, 4, 6) + +/* read a U32(c_i + u(u_i)) */ +static uint32_t jpegxl_u32(GetBitContext *gb, + const uint32_t constants[4], const uint32_t ubits[4]) +{ + uint32_t ret, choice = jxl_bits(2); + + ret = constants[choice]; + if (ubits[choice]) + ret += jxl_bits(ubits[choice]); + + return ret; +} + +/* read a U64() */ +static uint64_t jpegxl_u64(GetBitContext *gb) +{ + uint64_t shift = 12, ret; + + switch (jxl_bits(2)) { + case 0: + ret = 0; + break; + case 1: + ret = 1 + jxl_bits(4); + break; + case 2: + ret = 17 + jxl_bits(8); + break; + case 3: + ret = jxl_bits(12); + while (jxl_bits(1)) { + if (shift < 60) { + ret |= jxl_bits(8) << shift; + shift += 8; + } else { + ret |= jxl_bits(4) << shift; + break; + } + } + break; + } + + return ret; +} + +static uint32_t jpegxl_width_from_ratio(uint32_t height, int ratio) +{ + uint64_t height64 = height; + switch (ratio) { + case 1: + return height; + case 2: + return (uint32_t)((height64 * 12) / 10); + case 3: + return (uint32_t)((height64 * 4) / 3); + case 4: + return (uint32_t)((height64 * 3) / 2); + case 5: + return (uint32_t)((height64 * 16) / 9); + case 6: + return (uint32_t)((height64 * 5) / 4); + case 7: + return (uint32_t)(height64 * 2); + default: + break; + } + + return 0; /* manual width */ +} + +/** + * validate a Jpeg XL Size Header + * @return >= 0 upon valid size, < 0 upon invalid size found + */ +static int jpegxl_read_size_header(GetBitContext *gb) +{ + uint32_t width, height; + + if (jxl_bits(1)) { + /* small size header */ + height = (jxl_bits(5) + 1) << 3; + width = jpegxl_width_from_ratio(height, jxl_bits(3)); + if (!width) + width = (jxl_bits(5) + 1) << 3; + } else { + /* large size header */ + height = 1 + jxl_u32(0, 0, 0, 0, 9, 13, 18, 30); + width = jpegxl_width_from_ratio(height, jxl_bits(3)); + if (!width) + width = 1 + jxl_u32(0, 0, 0, 0, 9, 13, 18, 30); + } + if (width > (1 << 18) || height > (1 << 18) + || (width >> 4) * (height >> 4) > (1 << 20)) + return -1; + + return 0; +} + +/** + * validate a Jpeg XL Preview Header + * @return >= 0 upon valid size, < 0 upon invalid size found + */ +static int jpegxl_read_preview_header(GetBitContext *gb) +{ + uint32_t width, height; + + if (jxl_bits(1)) { + /* coded height and width divided by eight */ + height = jxl_u32(16, 32, 1, 33, 0, 0, 5, 9) << 3; + width = jpegxl_width_from_ratio(height, jxl_bits(3)); + if (!width) + width = jxl_u32(16, 32, 1, 33, 0, 0, 5, 9) << 3; + } else { + /* full height and width coded */ + height = jxl_u32(1, 65, 321, 1345, 6, 8, 10, 12); + width = jpegxl_width_from_ratio(height, jxl_bits(3)); + if (!width) + width = jxl_u32(1, 65, 321, 1345, 6, 8, 10, 12); + } + if (width > 4096 || height > 4096) + return -1; + + return 0; +} + +/** + * skip a Jpeg XL BitDepth Header. These cannot be invalid. + */ +static void jpegxl_skip_bit_depth(GetBitContext *gb) +{ + if (jxl_bits(1)) { + /* float samples */ + jxl_u32(32, 16, 24, 1, 0, 0, 0, 6); /* mantissa */ + jxl_bits_skip(4); /* exponent */ + } else { + /* integer samples */ + jxl_u32(8, 10, 12, 1, 0, 0, 0, 6); + } +} + +/** + * validate a Jpeg XL Preview Header + * @return >= 0 upon valid, < 0 upon invalid + */ +static int jpegxl_read_extra_channel_info(GetBitContext *gb) +{ + int all_default = jxl_bits(1); + uint32_t type, name_len = 0; + + if (!all_default) { + type = jxl_enum(); + if (type > 63) + return -1; /* enum types cannot be 64+ */ + if (type == FF_JPEGXL_CT_BLACK) + return -1; + jpegxl_skip_bit_depth(gb); + jxl_u32(0, 3, 4, 1, 0, 0, 0, 3); /* dim-shift */ + /* max of name_len is 1071 = 48 + 2^10 - 1 */ + name_len = jxl_u32(0, 0, 16, 48, 0, 4, 5, 10); + } else { + type = FF_JPEGXL_CT_ALPHA; + } + + /* skip over the name */ + jxl_bits_skip(8 * name_len); + + if (!all_default && type == FF_JPEGXL_CT_ALPHA) + jxl_bits_skip(1); + + if (type == FF_JPEGXL_CT_SPOT_COLOR) + jxl_bits_skip(16 * 4); + + if (type == FF_JPEGXL_CT_CFA) + jxl_u32(1, 0, 3, 19, 0, 2, 4, 8); + + return 0; +} + +/* verify that a codestream header is valid */ +int ff_jpegxl_verify_codestream_header(const uint8_t *buf, int buflen) +{ + GetBitContext gbi, *gb = &gbi; + int all_default, extra_fields = 0; + int xyb_encoded = 1, have_icc_profile = 0; + uint32_t num_extra_channels; + uint64_t extensions; + + init_get_bits8(gb, buf, buflen); + + if (jxl_bits(16) != FF_JPEGXL_CODESTREAM_SIGNATURE_LE) + return -1; + + if (jpegxl_read_size_header(gb) < 0) + return -1; + + all_default = jxl_bits(1); + if (!all_default) + extra_fields = jxl_bits(1); + + if (extra_fields) { + jxl_bits_skip(3); /* orientation */ + + /* + * intrinstic size + * any size header here is valid, but as it + * is variable length we have to read it + */ + if (jxl_bits(1)) + jpegxl_read_size_header(gb); + + /* preview header */ + if (jxl_bits(1)) { + if (jpegxl_read_preview_header(gb) < 0) + return -1; + } + + /* animation header */ + if (jxl_bits(1)) { + jxl_u32(100, 1000, 1, 1, 0, 0, 10, 30); + jxl_u32(1, 1001, 1, 1, 0, 0, 8, 10); + jxl_u32(0, 0, 0, 0, 0, 3, 16, 32); + jxl_bits_skip(1); + } + } + + if (!all_default) { + jpegxl_skip_bit_depth(gb); + + /* modular_16bit_buffers must equal 1 */ + if (!jxl_bits(1)) + return -1; + + num_extra_channels = jxl_u32(0, 1, 2, 1, 0, 0, 4, 12); + if (num_extra_channels > 4) + return -1; + for (uint32_t i = 0; i < num_extra_channels; i++) { + if (jpegxl_read_extra_channel_info(gb) < 0) + return -1; + } + + xyb_encoded = jxl_bits(1); + + /* color encoding bundle */ + if (!jxl_bits(1)) { + uint32_t color_space; + have_icc_profile = jxl_bits(1); + color_space = jxl_enum(); + if (color_space > 63) + return -1; + + if (!have_icc_profile) { + if (color_space != FF_JPEGXL_CS_XYB) { + uint32_t white_point = jxl_enum(); + if (white_point > 63) + return -1; + if (white_point == FF_JPEGXL_WP_CUSTOM) { + /* ux and uy values */ + jxl_u32(0, 524288, 1048576, 2097152, 19, 19, 20, 21); + jxl_u32(0, 524288, 1048576, 2097152, 19, 19, 20, 21); + } + if (color_space != FF_JPEGXL_CS_GRAY) { + /* primaries */ + uint32_t primaries = jxl_enum(); + if (primaries > 63) + return -1; + if (primaries == FF_JPEGXL_PR_CUSTOM) { + /* ux/uy values for r,g,b */ + for (int i = 0; i < 6; i++) + jxl_u32(0, 524288, 1048576, 2097152, 19, 19, 20, 21); + } + } + } + + /* transfer characteristics */ + if (jxl_bits(1)) { + /* gamma */ + jxl_bits_skip(24); + } else { + /* transfer function */ + if (jxl_enum() > 63) + return -1; + } + + /* rendering intent */ + if (jxl_enum() > 63) + return -1; + } + } + + /* tone mapping bundle */ + if (extra_fields && !jxl_bits(1)) + jxl_bits_skip(16 + 16 + 1 + 16); + + extensions = jxl_u64(); + if (extensions) { + for (int i = 0; i < 64; i++) { + if (extensions & (UINT64_C(1) << i)) + jxl_u64(); + } + } + } + + /* default transform */ + if (!jxl_bits(1)) { + /* opsin inverse matrix */ + if (xyb_encoded && !jxl_bits(1)) + jxl_bits_skip(16 * 16); + /* cw_mask and default weights */ + if (jxl_bits(1)) + jxl_bits_skip(16 * 15); + if (jxl_bits(1)) + jxl_bits_skip(16 * 55); + if (jxl_bits(1)) + jxl_bits_skip(16 * 210); + } + + if (!have_icc_profile) { + int bits_remaining = 7 - (gb->index - 1) % 8; + if (bits_remaining && jxl_bits(bits_remaining)) + return -1; + } + + if (gb->index > gb->size_in_bits) + return -1; + + return 0; +} diff --git a/libavformat/jpegxl_probe.h b/libavformat/jpegxl_probe.h new file mode 100644 index 0000000000..2960e81e11 --- /dev/null +++ b/libavformat/jpegxl_probe.h @@ -0,0 +1,32 @@ +/* + * Jpeg XL header verification + * Copyright (c) 2022 Leo Izen <leo.izen@gmail.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVFORMAT_JPEGXL_PROBE_H +#define AVFORMAT_JPEGXL_PROBE_H + +#include <stdint.h> + +#define FF_JPEGXL_CODESTREAM_SIGNATURE_LE 0x0aff +#define FF_JPEGXL_CONTAINER_SIGNATURE_LE 0x204c584a0c000000 + +int ff_jpegxl_verify_codestream_header(const uint8_t *buf, int buflen); + +#endif /* AVFORMAT_JPEGXL_PROBE_H */ diff --git a/libavformat/mov.c b/libavformat/mov.c index 6c847de164..c4b8873b0a 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -7697,6 +7697,7 @@ static int mov_probe(const AVProbeData *p) if (tag == MKTAG('f','t','y','p') && ( AV_RL32(p->buf + offset + 8) == MKTAG('j','p','2',' ') || AV_RL32(p->buf + offset + 8) == MKTAG('j','p','x',' ') + || AV_RL32(p->buf + offset + 8) == MKTAG('j','x','l',' ') )) { score = FFMAX(score, 5); } else { -- 2.35.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [FFmpeg-devel] [PATCH v14 4/4] avformat/image2: add Jpeg XL as image2 format 2022-04-12 5:53 ` [FFmpeg-devel] [PATCH v14 4/4] avformat/image2: add Jpeg XL as image2 format Leo Izen @ 2022-04-15 11:34 ` Andreas Rheinhardt 2022-04-15 19:38 ` Leo Izen 0 siblings, 1 reply; 14+ messages in thread From: Andreas Rheinhardt @ 2022-04-15 11:34 UTC (permalink / raw) To: ffmpeg-devel Leo Izen: > This commit adds support to libavformat for muxing > and demuxing Jpeg XL images as image2 streams. > --- > MAINTAINERS | 1 + > libavformat/Makefile | 1 + > libavformat/allformats.c | 1 + > libavformat/img2.c | 1 + > libavformat/img2dec.c | 18 ++ > libavformat/img2enc.c | 6 +- > libavformat/jpegxl_probe.c | 393 +++++++++++++++++++++++++++++++++++++ > libavformat/jpegxl_probe.h | 32 +++ > libavformat/mov.c | 1 + > 9 files changed, 451 insertions(+), 3 deletions(-) > create mode 100644 libavformat/jpegxl_probe.c > create mode 100644 libavformat/jpegxl_probe.h > > diff --git a/MAINTAINERS b/MAINTAINERS > index faea84ebf1..46723972dc 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -439,6 +439,7 @@ Muxers/Demuxers: > ipmovie.c Mike Melanson > ircam* Paul B Mahol > iss.c Stefan Gehrer > + jpegxl_probe.* Leo Izen > jvdec.c Peter Ross > kvag.c Zane van Iperen > libmodplug.c Clément Bœsch > diff --git a/libavformat/Makefile b/libavformat/Makefile > index d7182d6bd8..beecdf5a66 100644 > --- a/libavformat/Makefile > +++ b/libavformat/Makefile > @@ -272,6 +272,7 @@ OBJS-$(CONFIG_IMAGE_GIF_PIPE_DEMUXER) += img2dec.o img2.o > OBJS-$(CONFIG_IMAGE_J2K_PIPE_DEMUXER) += img2dec.o img2.o > OBJS-$(CONFIG_IMAGE_JPEG_PIPE_DEMUXER) += img2dec.o img2.o > OBJS-$(CONFIG_IMAGE_JPEGLS_PIPE_DEMUXER) += img2dec.o img2.o > +OBJS-$(CONFIG_IMAGE_JPEGXL_PIPE_DEMUXER) += img2dec.o img2.o jpegxl_probe.o > OBJS-$(CONFIG_IMAGE_PAM_PIPE_DEMUXER) += img2dec.o img2.o > OBJS-$(CONFIG_IMAGE_PBM_PIPE_DEMUXER) += img2dec.o img2.o > OBJS-$(CONFIG_IMAGE_PCX_PIPE_DEMUXER) += img2dec.o img2.o > diff --git a/libavformat/allformats.c b/libavformat/allformats.c > index 7c1d0ac38f..63876c468f 100644 > --- a/libavformat/allformats.c > +++ b/libavformat/allformats.c > @@ -510,6 +510,7 @@ extern const AVInputFormat ff_image_gif_pipe_demuxer; > extern const AVInputFormat ff_image_j2k_pipe_demuxer; > extern const AVInputFormat ff_image_jpeg_pipe_demuxer; > extern const AVInputFormat ff_image_jpegls_pipe_demuxer; > +extern const AVInputFormat ff_image_jpegxl_pipe_demuxer; > extern const AVInputFormat ff_image_pam_pipe_demuxer; > extern const AVInputFormat ff_image_pbm_pipe_demuxer; > extern const AVInputFormat ff_image_pcx_pipe_demuxer; > diff --git a/libavformat/img2.c b/libavformat/img2.c > index fe2ca7bfff..566ef873ca 100644 > --- a/libavformat/img2.c > +++ b/libavformat/img2.c > @@ -88,6 +88,7 @@ const IdStrMap ff_img_tags[] = { > { AV_CODEC_ID_GEM, "ximg" }, > { AV_CODEC_ID_GEM, "timg" }, > { AV_CODEC_ID_VBN, "vbn" }, > + { AV_CODEC_ID_JPEGXL, "jxl" }, > { AV_CODEC_ID_NONE, NULL } > }; > > diff --git a/libavformat/img2dec.c b/libavformat/img2dec.c > index 551b9d508e..627bb67212 100644 > --- a/libavformat/img2dec.c > +++ b/libavformat/img2dec.c > @@ -36,6 +36,7 @@ > #include "avio_internal.h" > #include "internal.h" > #include "img2.h" > +#include "jpegxl_probe.h" > #include "libavcodec/mjpeg.h" > #include "libavcodec/vbn.h" > #include "libavcodec/xwd.h" > @@ -837,6 +838,22 @@ static int jpegls_probe(const AVProbeData *p) > return 0; > } > > +static int jpegxl_probe(const AVProbeData *p) > +{ > + const uint8_t *b = p->buf; > + > + /* ISOBMFF-based container */ > + /* 0x4a584c20 == "JXL " */ > + if (AV_RL64(b) == FF_JPEGXL_CONTAINER_SIGNATURE_LE) > + return AVPROBE_SCORE_EXTENSION + 1; > + /* Raw codestreams all start with 0xff0a */ > + if (AV_RL16(b) != FF_JPEGXL_CODESTREAM_SIGNATURE_LE) > + return 0; > + if (ff_jpegxl_verify_codestream_header(p->buf, p->buf_size) >= 0) This will give a linking failure if the image_jpegxl_pipe_demuxer is disabled. > + return AVPROBE_SCORE_MAX - 2; > + return 0; > +} > + > static int pcx_probe(const AVProbeData *p) > { > const uint8_t *b = p->buf; > @@ -1176,6 +1193,7 @@ IMAGEAUTO_DEMUXER(gif, GIF) > IMAGEAUTO_DEMUXER_EXT(j2k, JPEG2000, J2K) > IMAGEAUTO_DEMUXER_EXT(jpeg, MJPEG, JPEG) > IMAGEAUTO_DEMUXER(jpegls, JPEGLS) > +IMAGEAUTO_DEMUXER(jpegxl, JPEGXL) > IMAGEAUTO_DEMUXER(pam, PAM) > IMAGEAUTO_DEMUXER(pbm, PBM) > IMAGEAUTO_DEMUXER(pcx, PCX) > diff --git a/libavformat/img2enc.c b/libavformat/img2enc.c > index ae351963d9..5ed97bb833 100644 > --- a/libavformat/img2enc.c > +++ b/libavformat/img2enc.c > @@ -263,9 +263,9 @@ static const AVClass img2mux_class = { > const AVOutputFormat ff_image2_muxer = { > .name = "image2", > .long_name = NULL_IF_CONFIG_SMALL("image2 sequence"), > - .extensions = "bmp,dpx,exr,jls,jpeg,jpg,ljpg,pam,pbm,pcx,pfm,pgm,pgmyuv,png," > - "ppm,sgi,tga,tif,tiff,jp2,j2c,j2k,xwd,sun,ras,rs,im1,im8,im24," > - "sunras,vbn,xbm,xface,pix,y", > + .extensions = "bmp,dpx,exr,jls,jpeg,jpg,jxl,ljpg,pam,pbm,pcx,pfm,pgm,pgmyuv," > + "png,ppm,sgi,tga,tif,tiff,jp2,j2c,j2k,xwd,sun,ras,rs,im1,im8," > + "im24,sunras,vbn,xbm,xface,pix,y", > .priv_data_size = sizeof(VideoMuxData), > .video_codec = AV_CODEC_ID_MJPEG, > .write_header = write_header, > diff --git a/libavformat/jpegxl_probe.c b/libavformat/jpegxl_probe.c > new file mode 100644 > index 0000000000..d3d3822fee > --- /dev/null > +++ b/libavformat/jpegxl_probe.c > @@ -0,0 +1,393 @@ > +/* > + * Jpeg XL header verification > + * Copyright (c) 2022 Leo Izen <leo.izen@gmail.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include "jpegxl_probe.h" > + > +#define BITSTREAM_READER_LE > +#include "libavcodec/get_bits.h" > + > +enum JpegXLExtraChannelType { > + FF_JPEGXL_CT_ALPHA = 0, > + FF_JPEGXL_CT_DEPTH, > + FF_JPEGXL_CT_SPOT_COLOR, > + FF_JPEGXL_CT_SELECTION_MASK, > + FF_JPEGXL_CT_BLACK, > + FF_JPEGXL_CT_CFA, > + FF_JPEGXL_CT_THERMAL, > + FF_JPEGXL_CT_NON_OPTIONAL = 15, > + FF_JPEGXL_CT_OPTIONAL > +}; > + > +enum JpegXLColorSpace { > + FF_JPEGXL_CS_RGB = 0, > + FF_JPEGXL_CS_GRAY, > + FF_JPEGXL_CS_XYB, > + FF_JPEGXL_CS_UNKNOWN > +}; > + > +enum JpegXLWhitePoint { > + FF_JPEGXL_WP_D65 = 1, > + FF_JPEGXL_WP_CUSTOM, > + FF_JPEGXL_WP_E = 10, > + FF_JPEGXL_WP_DCI = 11 > +}; > + > +enum JpegXLPrimaries { > + FF_JPEGXL_PR_SRGB = 1, > + FF_JPEGXL_PR_CUSTOM, > + FF_JPEGXL_PR_2100 = 9, > + FF_JPEGXL_PR_P3 = 11, > +}; > + > +#define jxl_bits(n) get_bits_long(gb, (n)) > +#define jxl_bits_skip(n) skip_bits_long(gb, (n)) > +#define jxl_u32(c0, c1, c2, c3, u0, u1, u2, u3) jpegxl_u32(gb, \ > + (const uint32_t[]){c0, c1, c2, c3}, (const uint32_t[]){u0, u1, u2, u3}) > +#define jxl_u64() jpegxl_u64(gb) > +#define jxl_enum() jxl_u32(0, 1, 2, 18, 0, 0, 4, 6) > + > +/* read a U32(c_i + u(u_i)) */ > +static uint32_t jpegxl_u32(GetBitContext *gb, > + const uint32_t constants[4], const uint32_t ubits[4]) > +{ > + uint32_t ret, choice = jxl_bits(2); > + > + ret = constants[choice]; > + if (ubits[choice]) > + ret += jxl_bits(ubits[choice]); > + > + return ret; > +} > + > +/* read a U64() */ > +static uint64_t jpegxl_u64(GetBitContext *gb) > +{ > + uint64_t shift = 12, ret; > + > + switch (jxl_bits(2)) { > + case 0: > + ret = 0; > + break; > + case 1: > + ret = 1 + jxl_bits(4); > + break; > + case 2: > + ret = 17 + jxl_bits(8); > + break; > + case 3: > + ret = jxl_bits(12); > + while (jxl_bits(1)) { > + if (shift < 60) { > + ret |= jxl_bits(8) << shift; > + shift += 8; > + } else { > + ret |= jxl_bits(4) << shift; > + break; > + } > + } > + break; > + } > + > + return ret; > +} > + > +static uint32_t jpegxl_width_from_ratio(uint32_t height, int ratio) > +{ > + uint64_t height64 = height; > + switch (ratio) { > + case 1: > + return height; > + case 2: > + return (uint32_t)((height64 * 12) / 10); > + case 3: > + return (uint32_t)((height64 * 4) / 3); > + case 4: > + return (uint32_t)((height64 * 3) / 2); > + case 5: > + return (uint32_t)((height64 * 16) / 9); > + case 6: > + return (uint32_t)((height64 * 5) / 4); > + case 7: > + return (uint32_t)(height64 * 2); > + default: > + break; > + } > + > + return 0; /* manual width */ > +} > + > +/** > + * validate a Jpeg XL Size Header > + * @return >= 0 upon valid size, < 0 upon invalid size found > + */ > +static int jpegxl_read_size_header(GetBitContext *gb) > +{ > + uint32_t width, height; > + > + if (jxl_bits(1)) { > + /* small size header */ > + height = (jxl_bits(5) + 1) << 3; > + width = jpegxl_width_from_ratio(height, jxl_bits(3)); > + if (!width) > + width = (jxl_bits(5) + 1) << 3; > + } else { > + /* large size header */ > + height = 1 + jxl_u32(0, 0, 0, 0, 9, 13, 18, 30); > + width = jpegxl_width_from_ratio(height, jxl_bits(3)); > + if (!width) > + width = 1 + jxl_u32(0, 0, 0, 0, 9, 13, 18, 30); > + } > + if (width > (1 << 18) || height > (1 << 18) > + || (width >> 4) * (height >> 4) > (1 << 20)) > + return -1; > + > + return 0; > +} > + > +/** > + * validate a Jpeg XL Preview Header > + * @return >= 0 upon valid size, < 0 upon invalid size found > + */ > +static int jpegxl_read_preview_header(GetBitContext *gb) > +{ > + uint32_t width, height; > + > + if (jxl_bits(1)) { > + /* coded height and width divided by eight */ > + height = jxl_u32(16, 32, 1, 33, 0, 0, 5, 9) << 3; > + width = jpegxl_width_from_ratio(height, jxl_bits(3)); > + if (!width) > + width = jxl_u32(16, 32, 1, 33, 0, 0, 5, 9) << 3; > + } else { > + /* full height and width coded */ > + height = jxl_u32(1, 65, 321, 1345, 6, 8, 10, 12); > + width = jpegxl_width_from_ratio(height, jxl_bits(3)); > + if (!width) > + width = jxl_u32(1, 65, 321, 1345, 6, 8, 10, 12); > + } > + if (width > 4096 || height > 4096) > + return -1; > + > + return 0; > +} > + > +/** > + * skip a Jpeg XL BitDepth Header. These cannot be invalid. > + */ > +static void jpegxl_skip_bit_depth(GetBitContext *gb) > +{ > + if (jxl_bits(1)) { > + /* float samples */ > + jxl_u32(32, 16, 24, 1, 0, 0, 0, 6); /* mantissa */ > + jxl_bits_skip(4); /* exponent */ > + } else { > + /* integer samples */ > + jxl_u32(8, 10, 12, 1, 0, 0, 0, 6); > + } > +} > + > +/** > + * validate a Jpeg XL Preview Header > + * @return >= 0 upon valid, < 0 upon invalid > + */ > +static int jpegxl_read_extra_channel_info(GetBitContext *gb) > +{ > + int all_default = jxl_bits(1); > + uint32_t type, name_len = 0; > + > + if (!all_default) { > + type = jxl_enum(); > + if (type > 63) > + return -1; /* enum types cannot be 64+ */ > + if (type == FF_JPEGXL_CT_BLACK) > + return -1; > + jpegxl_skip_bit_depth(gb); > + jxl_u32(0, 3, 4, 1, 0, 0, 0, 3); /* dim-shift */ > + /* max of name_len is 1071 = 48 + 2^10 - 1 */ > + name_len = jxl_u32(0, 0, 16, 48, 0, 4, 5, 10); > + } else { > + type = FF_JPEGXL_CT_ALPHA; > + } > + > + /* skip over the name */ > + jxl_bits_skip(8 * name_len); > + > + if (!all_default && type == FF_JPEGXL_CT_ALPHA) > + jxl_bits_skip(1); > + > + if (type == FF_JPEGXL_CT_SPOT_COLOR) > + jxl_bits_skip(16 * 4); > + > + if (type == FF_JPEGXL_CT_CFA) > + jxl_u32(1, 0, 3, 19, 0, 2, 4, 8); > + > + return 0; > +} > + > +/* verify that a codestream header is valid */ > +int ff_jpegxl_verify_codestream_header(const uint8_t *buf, int buflen) > +{ > + GetBitContext gbi, *gb = &gbi; > + int all_default, extra_fields = 0; > + int xyb_encoded = 1, have_icc_profile = 0; > + uint32_t num_extra_channels; > + uint64_t extensions; > + > + init_get_bits8(gb, buf, buflen); > + > + if (jxl_bits(16) != FF_JPEGXL_CODESTREAM_SIGNATURE_LE) > + return -1; > + > + if (jpegxl_read_size_header(gb) < 0) > + return -1; > + > + all_default = jxl_bits(1); > + if (!all_default) > + extra_fields = jxl_bits(1); > + > + if (extra_fields) { > + jxl_bits_skip(3); /* orientation */ > + > + /* > + * intrinstic size > + * any size header here is valid, but as it > + * is variable length we have to read it > + */ > + if (jxl_bits(1)) > + jpegxl_read_size_header(gb); > + > + /* preview header */ > + if (jxl_bits(1)) { > + if (jpegxl_read_preview_header(gb) < 0) > + return -1; > + } > + > + /* animation header */ > + if (jxl_bits(1)) { > + jxl_u32(100, 1000, 1, 1, 0, 0, 10, 30); > + jxl_u32(1, 1001, 1, 1, 0, 0, 8, 10); > + jxl_u32(0, 0, 0, 0, 0, 3, 16, 32); > + jxl_bits_skip(1); > + } > + } > + > + if (!all_default) { > + jpegxl_skip_bit_depth(gb); > + > + /* modular_16bit_buffers must equal 1 */ > + if (!jxl_bits(1)) > + return -1; > + > + num_extra_channels = jxl_u32(0, 1, 2, 1, 0, 0, 4, 12); > + if (num_extra_channels > 4) > + return -1; > + for (uint32_t i = 0; i < num_extra_channels; i++) { > + if (jpegxl_read_extra_channel_info(gb) < 0) > + return -1; > + } > + > + xyb_encoded = jxl_bits(1); > + > + /* color encoding bundle */ > + if (!jxl_bits(1)) { > + uint32_t color_space; > + have_icc_profile = jxl_bits(1); > + color_space = jxl_enum(); > + if (color_space > 63) > + return -1; > + > + if (!have_icc_profile) { > + if (color_space != FF_JPEGXL_CS_XYB) { > + uint32_t white_point = jxl_enum(); > + if (white_point > 63) > + return -1; > + if (white_point == FF_JPEGXL_WP_CUSTOM) { > + /* ux and uy values */ > + jxl_u32(0, 524288, 1048576, 2097152, 19, 19, 20, 21); > + jxl_u32(0, 524288, 1048576, 2097152, 19, 19, 20, 21); > + } > + if (color_space != FF_JPEGXL_CS_GRAY) { > + /* primaries */ > + uint32_t primaries = jxl_enum(); > + if (primaries > 63) > + return -1; > + if (primaries == FF_JPEGXL_PR_CUSTOM) { > + /* ux/uy values for r,g,b */ > + for (int i = 0; i < 6; i++) > + jxl_u32(0, 524288, 1048576, 2097152, 19, 19, 20, 21); > + } > + } > + } > + > + /* transfer characteristics */ > + if (jxl_bits(1)) { > + /* gamma */ > + jxl_bits_skip(24); > + } else { > + /* transfer function */ > + if (jxl_enum() > 63) > + return -1; > + } > + > + /* rendering intent */ > + if (jxl_enum() > 63) > + return -1; > + } > + } > + > + /* tone mapping bundle */ > + if (extra_fields && !jxl_bits(1)) > + jxl_bits_skip(16 + 16 + 1 + 16); > + > + extensions = jxl_u64(); > + if (extensions) { > + for (int i = 0; i < 64; i++) { > + if (extensions & (UINT64_C(1) << i)) > + jxl_u64(); > + } > + } > + } > + > + /* default transform */ > + if (!jxl_bits(1)) { > + /* opsin inverse matrix */ > + if (xyb_encoded && !jxl_bits(1)) > + jxl_bits_skip(16 * 16); > + /* cw_mask and default weights */ > + if (jxl_bits(1)) > + jxl_bits_skip(16 * 15); > + if (jxl_bits(1)) > + jxl_bits_skip(16 * 55); > + if (jxl_bits(1)) > + jxl_bits_skip(16 * 210); > + } > + > + if (!have_icc_profile) { > + int bits_remaining = 7 - (gb->index - 1) % 8; Try to avoid accessing GetBitContext members directly; here: Use get_bits_count(). > + if (bits_remaining && jxl_bits(bits_remaining)) > + return -1; > + } > + > + if (gb->index > gb->size_in_bits) get_bits_left(gb) < 0 > + return -1; > + > + return 0; > +} For the record: I'm not really ok with duplicating this code in lavf and lavc. > diff --git a/libavformat/jpegxl_probe.h b/libavformat/jpegxl_probe.h > new file mode 100644 > index 0000000000..2960e81e11 > --- /dev/null > +++ b/libavformat/jpegxl_probe.h > @@ -0,0 +1,32 @@ > +/* > + * Jpeg XL header verification > + * Copyright (c) 2022 Leo Izen <leo.izen@gmail.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#ifndef AVFORMAT_JPEGXL_PROBE_H > +#define AVFORMAT_JPEGXL_PROBE_H > + > +#include <stdint.h> > + > +#define FF_JPEGXL_CODESTREAM_SIGNATURE_LE 0x0aff > +#define FF_JPEGXL_CONTAINER_SIGNATURE_LE 0x204c584a0c000000 > + > +int ff_jpegxl_verify_codestream_header(const uint8_t *buf, int buflen); > + > +#endif /* AVFORMAT_JPEGXL_PROBE_H */ > diff --git a/libavformat/mov.c b/libavformat/mov.c > index 6c847de164..c4b8873b0a 100644 > --- a/libavformat/mov.c > +++ b/libavformat/mov.c > @@ -7697,6 +7697,7 @@ static int mov_probe(const AVProbeData *p) > if (tag == MKTAG('f','t','y','p') && > ( AV_RL32(p->buf + offset + 8) == MKTAG('j','p','2',' ') > || AV_RL32(p->buf + offset + 8) == MKTAG('j','p','x',' ') > + || AV_RL32(p->buf + offset + 8) == MKTAG('j','x','l',' ') > )) { > score = FFMAX(score, 5); > } else { _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [FFmpeg-devel] [PATCH v14 4/4] avformat/image2: add Jpeg XL as image2 format 2022-04-15 11:34 ` Andreas Rheinhardt @ 2022-04-15 19:38 ` Leo Izen 2022-04-15 19:41 ` Andreas Rheinhardt 0 siblings, 1 reply; 14+ messages in thread From: Leo Izen @ 2022-04-15 19:38 UTC (permalink / raw) To: ffmpeg-devel On 4/15/22 07:34, Andreas Rheinhardt wrote: > Leo Izen: >> +static int jpegxl_probe(const AVProbeData *p) >> +{ >> + const uint8_t *b = p->buf; >> + >> + /* ISOBMFF-based container */ >> + /* 0x4a584c20 == "JXL " */ >> + if (AV_RL64(b) == FF_JPEGXL_CONTAINER_SIGNATURE_LE) >> + return AVPROBE_SCORE_EXTENSION + 1; >> + /* Raw codestreams all start with 0xff0a */ >> + if (AV_RL16(b) != FF_JPEGXL_CODESTREAM_SIGNATURE_LE) >> + return 0; >> + if (ff_jpegxl_verify_codestream_header(p->buf, p->buf_size) >= 0) > This will give a linking failure if the image_jpegxl_pipe_demuxer is > disabled. > I thought of that, and I tested it, and it doesn't. It produces a compiler warning that the static function jpegxl_probe is never called, which means the linker probably throws away the function entirely, thus eliminating a linking error. Though I admit that's a guess. > For the record: I'm not really ok with duplicating this code in lavf and > lavc. > This code was removed from avcodec when I moved it over to avformat, for now, in order to avoid an avpriv in avcodec's ABI. Essentially we have to decide to either duplicate code in the future, or move it to avcodec and add an avpriv to the ABI, but that is a decision that can be made down the road when Lynne finishes her proper parser. I believe this is a better solution now than the other way around since we can later change our minds once there will actually be duplicated code, but an avpriv cannot be removed once added without waiting the necessary ABI change period. - Leo Izen (thebombzen) _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [FFmpeg-devel] [PATCH v14 4/4] avformat/image2: add Jpeg XL as image2 format 2022-04-15 19:38 ` Leo Izen @ 2022-04-15 19:41 ` Andreas Rheinhardt 0 siblings, 0 replies; 14+ messages in thread From: Andreas Rheinhardt @ 2022-04-15 19:41 UTC (permalink / raw) To: ffmpeg-devel Leo Izen: > > On 4/15/22 07:34, Andreas Rheinhardt wrote: >> Leo Izen: >>> +static int jpegxl_probe(const AVProbeData *p) >>> +{ >>> + const uint8_t *b = p->buf; >>> + >>> + /* ISOBMFF-based container */ >>> + /* 0x4a584c20 == "JXL " */ >>> + if (AV_RL64(b) == FF_JPEGXL_CONTAINER_SIGNATURE_LE) >>> + return AVPROBE_SCORE_EXTENSION + 1; >>> + /* Raw codestreams all start with 0xff0a */ >>> + if (AV_RL16(b) != FF_JPEGXL_CODESTREAM_SIGNATURE_LE) >>> + return 0; >>> + if (ff_jpegxl_verify_codestream_header(p->buf, p->buf_size) >= 0) >> This will give a linking failure if the image_jpegxl_pipe_demuxer is >> disabled. >> > I thought of that, and I tested it, and it doesn't. It produces a > compiler warning that the static function jpegxl_probe is never called, > which means the linker probably throws away the function entirely, thus > eliminating a linking error. Though I admit that's a guess. That only works when optimizations are enabled. Try again with -O0. - Andreas _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2022-04-15 19:41 UTC | newest] Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-04-12 5:53 [FFmpeg-devel] [PATCH v14 0/4] Jpeg XL Patch Set Leo Izen 2022-04-12 5:53 ` [FFmpeg-devel] [PATCH v14 1/4] avcodec/jpegxl: add Jpeg XL image codec Leo Izen 2022-04-12 5:53 ` [FFmpeg-devel] [PATCH v14 2/4] avcodec/libjxl: add Jpeg XL decoding via libjxl Leo Izen 2022-04-15 11:08 ` Andreas Rheinhardt 2022-04-15 19:31 ` Leo Izen 2022-04-12 5:53 ` [FFmpeg-devel] [PATCH v14 3/4] avcodec/libjxl: add Jpeg XL encoding " Leo Izen 2022-04-14 17:49 ` Anton Khirnov 2022-04-15 1:37 ` Leo Izen 2022-04-15 10:39 ` Anton Khirnov 2022-04-15 19:32 ` Leo Izen 2022-04-12 5:53 ` [FFmpeg-devel] [PATCH v14 4/4] avformat/image2: add Jpeg XL as image2 format Leo Izen 2022-04-15 11:34 ` Andreas Rheinhardt 2022-04-15 19:38 ` Leo Izen 2022-04-15 19:41 ` Andreas Rheinhardt
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel This inbox may be cloned and mirrored by anyone: git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \ ffmpegdev@gitmailbox.com public-inbox-index ffmpegdev Example config snippet for mirrors. AGPL code for this site: git clone https://public-inbox.org/public-inbox.git