From: Daniel Stadelmann <code@ffmpeg.org> To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH] avcodec/libmpeghdec: Add MPEG-H 3DA Fraunhofer IIS mpeghdec decoder (PR #20129) Date: Wed, 6 Aug 2025 09:54:20 +0300 (EEST) Message-ID: <20250806065420.E68D868BA68@ffbox0-bg.ffmpeg.org> (raw) PR #20129 opened by Daniel Stadelmann (dstadelmann-iis) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20129 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20129.patch Adds a wrapper around the Fraunhofer IIS MPEG-H 3D Audio mpeghdec [1] decoder shared library. [1] https://github.com/Fraunhofer-IIS/mpeghdec Signed-off-by: Stadelmann, Daniel <daniel.stadelmann@iis.fraunhofer.de> From 137f4ac50efb868f0f7419c0842c5bf6913d27b9 Mon Sep 17 00:00:00 2001 From: "Stadelmann, Daniel" <daniel.stadelmann@iis.fraunhofer.de> Date: Fri, 1 Aug 2025 07:15:37 +0200 Subject: [PATCH] avcodec/libmpeghdec: Add MPEG-H 3DA Fraunhofer IIS mpeghdec decoder Adds a wrapper around the Fraunhofer IIS MPEG-H 3D Audio mpeghdec [1] decoder shared library. [1] https://github.com/Fraunhofer-IIS/mpeghdec Signed-off-by: Stadelmann, Daniel <daniel.stadelmann@iis.fraunhofer.de> --- Changelog | 1 + MAINTAINERS | 1 + configure | 4 + doc/decoders.texi | 21 +++ doc/general_contents.texi | 10 ++ libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 1 + libavcodec/libmpeghdec.c | 306 ++++++++++++++++++++++++++++++++++++++ libavcodec/version.h | 4 +- 9 files changed, 347 insertions(+), 2 deletions(-) create mode 100644 libavcodec/libmpeghdec.c diff --git a/Changelog b/Changelog index 04db8e15b2..42304c4098 100644 --- a/Changelog +++ b/Changelog @@ -26,6 +26,7 @@ version <next>: - OpenHarmony hardware decoder/encoder - Colordetect filter - Add vf_scale_d3d11 filter +- MPEG-H 3D Audio decoding via mpeghdec version 7.1: diff --git a/MAINTAINERS b/MAINTAINERS index 650300dadc..ccf41b6099 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -208,6 +208,7 @@ Codecs: libjxl*.c, libjxl.h Leo Izen libgsm.c Michel Bardiaux libkvazaar.c Arttu Ylä-Outinen + libmpeghdec.c Fraunhofer IIS (CC mpeg-h-techsupport@iis.fraunhofer.de) (B https://github.com/Fraunhofer-IIS/mpeghdec/issues) libopenh264enc.c Martin Storsjo, Linjie Fu libopenjpegenc.c Michael Bradshaw libtheoraenc.c David Conrad diff --git a/configure b/configure index 00934b00be..1dbadd01b0 100755 --- a/configure +++ b/configure @@ -249,6 +249,7 @@ External library support: --enable-liblensfun enable lensfun lens correction [no] --enable-libmodplug enable ModPlug via libmodplug [no] --enable-libmp3lame enable MP3 encoding via libmp3lame [no] + --enable-libmpeghdec enable MPEG-H 3DA decoding via libmpeghdec [no] --enable-liboapv enable APV encoding via liboapv [no] --enable-libopencore-amrnb enable AMR-NB de/encoding via libopencore-amrnb [no] --enable-libopencore-amrwb enable AMR-WB decoding via libopencore-amrwb [no] @@ -1952,6 +1953,7 @@ EXTERNAL_LIBRARY_LIST=" liblcevc_dec libmodplug libmp3lame + libmpeghdec libmysofa liboapv libopencv @@ -3611,6 +3613,7 @@ liblc3_encoder_select="audio_frame_queue" libmodplug_demuxer_deps="libmodplug" libmp3lame_encoder_deps="libmp3lame" libmp3lame_encoder_select="audio_frame_queue mpegaudioheader" +libmpeghdec_decoder_deps="libmpeghdec" liboapv_encoder_deps="liboapv" libopencore_amrnb_decoder_deps="libopencore_amrnb" libopencore_amrnb_encoder_deps="libopencore_amrnb" @@ -7085,6 +7088,7 @@ fi enabled libmodplug && require_pkg_config libmodplug libmodplug libmodplug/modplug.h ModPlug_Load enabled libmp3lame && require "libmp3lame >= 3.98.3" lame/lame.h lame_set_VBR_quality -lmp3lame $libm_extralibs +enabled libmpeghdec && require_pkg_config libmpeghdec "mpeghdec >= 3.0.0" mpeghdec/mpeghdecoder.h mpeghdecoder_init enabled libmysofa && { check_pkg_config libmysofa libmysofa mysofa.h mysofa_neighborhood_init_withstepdefine || require libmysofa mysofa.h mysofa_neighborhood_init_withstepdefine -lmysofa $zlib_extralibs; } enabled libnpp && { check_lib libnpp npp.h nppGetLibVersion -lnppig -lnppicc -lnppc -lnppidei -lnppif || diff --git a/doc/decoders.texi b/doc/decoders.texi index 13eb40f4dd..68088246ab 100644 --- a/doc/decoders.texi +++ b/doc/decoders.texi @@ -356,6 +356,27 @@ value is 0 (disabled). @end table +@section libmpeghdec + +libmpeghdec decoder wrapper. + +libmpeghdec allows libmpeghdec to decode the MPEG-H 3D audio codec. +Requires the presence of the libmpeghdec headers and library during +configuration. You need to explicitly configure the build with +@code{--enable-libmpeghdec}. + +@subsection Options + +The following option is supported by the libmpeghdec wrapper. + +@table @option +@item cicp @var{number} (1 - 19) + +Enables decoding into the given CICP (channel layout). The default +value is 2 (Stereo). + +@end table + @section libopencore-amrnb libopencore-amrnb decoder wrapper. diff --git a/doc/general_contents.texi b/doc/general_contents.texi index b9dab580f1..9608ce219a 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -250,6 +250,14 @@ Go to @url{http://sourceforge.net/projects/opencore-amr/} and follow the instructions for installing the library. Then pass @code{--enable-libfdk-aac} to configure to enable it. +@subsection Fraunhofer MPEG-H 3D Audio decoder library + +FFmpeg can make use of the Fraunhofer MPEG-H decoder library for MPEG-H 3DA decoding. + +Go to @url{https://github.com/Fraunhofer-IIS/mpeghdec} and follow the +instructions for installing the library. +Then pass @code{--enable-libmpeghdec} to configure to enable it. + @subsection LC3 library FFmpeg can make use of the Google LC3 library for LC3 decoding & encoding. @@ -1344,6 +1352,8 @@ following image formats are supported: @item MP3 (MPEG audio layer 3) @tab E @tab IX @tab encoding supported through external library LAME, ADU MP3 and MP3onMP4 also supported @item MPEG-4 Audio Lossless Coding (ALS) @tab @tab X +@item MPEG-H 3D Audio @tab @tab E + @tab decoding supported through external library libmpeghdec @item MobiClip FastAudio @tab @tab X @item Musepack SV7 @tab @tab X @item Musepack SV8 @tab @tab X diff --git a/libavcodec/Makefile b/libavcodec/Makefile index f935cfbe0d..a1705cdd7b 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1160,6 +1160,7 @@ OBJS-$(CONFIG_LIBKVAZAAR_ENCODER) += libkvazaar.o OBJS-$(CONFIG_LIBLC3_ENCODER) += liblc3enc.o OBJS-$(CONFIG_LIBLC3_DECODER) += liblc3dec.o OBJS-$(CONFIG_LIBMP3LAME_ENCODER) += libmp3lame.o +OBJS-$(CONFIG_LIBMPEGHDEC_DECODER) += libmpeghdec.o OBJS-$(CONFIG_LIBOAPV_ENCODER) += liboapvenc.o OBJS-$(CONFIG_LIBOPENCORE_AMRNB_DECODER) += libopencore-amr.o OBJS-$(CONFIG_LIBOPENCORE_AMRNB_ENCODER) += libopencore-amr.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 3aaa351157..22239b3521 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -789,6 +789,7 @@ extern const FFCodec ff_libjxl_encoder; extern const FFCodec ff_liblc3_encoder; extern const FFCodec ff_liblc3_decoder; extern const FFCodec ff_libmp3lame_encoder; +extern const FFCodec ff_libmpeghdec_decoder; extern const FFCodec ff_liboapv_encoder; extern const FFCodec ff_libopencore_amrnb_encoder; extern const FFCodec ff_libopencore_amrnb_decoder; diff --git a/libavcodec/libmpeghdec.c b/libavcodec/libmpeghdec.c new file mode 100644 index 0000000000..b5d9e623ed --- /dev/null +++ b/libavcodec/libmpeghdec.c @@ -0,0 +1,306 @@ +/* + * MPEG-H 3D Audio Decoder Wrapper + * Copyright (C) 2025 Fraunhofer Institute for Integrated Circuits IIS + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/* + * Please note that this FFmpeg Software is licensed under the LGPL-2.1 + * but is combined with software that is licensed under different terms, namely + * the "Software License for The Fraunhofer FDK MPEG-H Software". Fraunhofer + * as the initial licensor does not interpret the LGPL-2.1 as requiring + * distribution of the MPEG-H Software under the LGPL-2.1 if being distributed + * together with this FFmpeg Software. Therefore, downstream distribution of + * FFmpegg Software does not imply any right to redistribute the MPEG-H Software + * under the LGPL-2.1. + */ +#include <string.h> +#include <mpeghdec/mpeghdecoder.h> + +#include "libavutil/channel_layout.h" +#include "libavutil/frame.h" +#include "libavutil/mem.h" +#include "libavutil/opt.h" + +#include "codec_internal.h" +#include "decode.h" + +#define MAX_LOST_FRAMES 2 +// 32-bit int * 22.2 * max framesize * (max delay frames + 1) +#define MAX_OUTBUF_SIZE (sizeof(int32_t) * 24 * 3072 * (MAX_LOST_FRAMES + 1)) + +typedef struct MPEGH3DADecContext { + AVClass *av_class; + // pointer to the decoder + HANDLE_MPEGH_DECODER_CONTEXT decoder; + + // Command-line options + int cicp; + + // Internal values + int32_t *decoder_buffer; + int decoder_buffer_size; +} MPEGH3DADecContext; + +static const AVOption mpegh3da_options[] = { + { "cicp", + "Target channel layout (CICP index)", + offsetof(MPEGH3DADecContext, cicp), + AV_OPT_TYPE_INT, + { .i64 = 2 }, + 1, + 19, + AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_DECODING_PARAM, + "cicp" }, + { NULL } +}; + +// private class definition for ffmpeg +static const AVClass mpegh3da_class = { + .class_name = "MPEG-H 3D Audio Decoder", + .item_name = av_default_item_name, + .option = mpegh3da_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +static av_cold int mpegh3dadec_close(AVCodecContext *avctx) +{ + MPEGH3DADecContext *s = avctx->priv_data; + + if (s->decoder) + mpeghdecoder_destroy(s->decoder); + s->decoder = NULL; + av_freep(&s->decoder_buffer); + + return 0; +} + +// lookup for ffmpeg channel layout for given cicp, see: +// https://github.com/Fraunhofer-IIS/mpeghdec/wiki/MPEG-H-decoder-target-layouts +static av_cold int set_channel_layout(AVChannelLayout *outLayout, int cicp) +{ + switch (cicp) { + case 0: + // default channel layout + return av_channel_layout_copy( + outLayout, &(AVChannelLayout) AV_CHANNEL_LAYOUT_STEREO); + case 1: + return av_channel_layout_copy( + outLayout, &(AVChannelLayout) AV_CHANNEL_LAYOUT_MONO); + case 2: + return av_channel_layout_copy( + outLayout, &(AVChannelLayout) AV_CHANNEL_LAYOUT_STEREO); + case 3: + return av_channel_layout_copy( + outLayout, &(AVChannelLayout) AV_CHANNEL_LAYOUT_SURROUND); + case 4: + return av_channel_layout_copy( + outLayout, &(AVChannelLayout) AV_CHANNEL_LAYOUT_4POINT0); + case 5: + return av_channel_layout_copy( + outLayout, &(AVChannelLayout) AV_CHANNEL_LAYOUT_5POINT0); + case 6: + return av_channel_layout_copy( + outLayout, &(AVChannelLayout) AV_CHANNEL_LAYOUT_5POINT1); + case 7: + return av_channel_layout_copy( + outLayout, &(AVChannelLayout) AV_CHANNEL_LAYOUT_7POINT1_WIDE); + case 9: + return av_channel_layout_copy(outLayout, + &(AVChannelLayout) AV_CHANNEL_LAYOUT_2_1); + case 10: + return av_channel_layout_copy(outLayout, + &(AVChannelLayout) AV_CHANNEL_LAYOUT_2_2); + case 11: + return av_channel_layout_copy( + outLayout, &(AVChannelLayout) AV_CHANNEL_LAYOUT_6POINT1); + case 12: + return av_channel_layout_copy( + outLayout, &(AVChannelLayout) AV_CHANNEL_LAYOUT_7POINT1); + case 13: + return av_channel_layout_copy( + outLayout, &(AVChannelLayout) AV_CHANNEL_LAYOUT_22POINT2); + case 14: + return av_channel_layout_copy( + outLayout, &(AVChannelLayout) AV_CHANNEL_LAYOUT_5POINT1POINT2); + case 15: + return av_channel_layout_copy( + outLayout, + &(AVChannelLayout) AV_CHANNEL_LAYOUT_MASK( + 12, (AV_CH_LAYOUT_5POINT1POINT2 | AV_CH_SIDE_SURROUND_LEFT | + AV_CH_SIDE_SURROUND_RIGHT | AV_CH_TOP_BACK_CENTER | + AV_CH_LOW_FREQUENCY_2))); + case 16: + return av_channel_layout_copy( + outLayout, &(AVChannelLayout) AV_CHANNEL_LAYOUT_5POINT1POINT4_BACK); + case 17: + return av_channel_layout_copy( + outLayout, &(AVChannelLayout) AV_CHANNEL_LAYOUT_MASK( + 12, (AV_CH_LAYOUT_5POINT1POINT4_BACK | + AV_CH_TOP_FRONT_CENTER | AV_CH_TOP_CENTER))); + case 18: + return av_channel_layout_copy( + outLayout, &(AVChannelLayout) AV_CHANNEL_LAYOUT_MASK( + 14, (AV_CH_LAYOUT_7POINT1POINT4_BACK | + AV_CH_TOP_FRONT_CENTER | AV_CH_TOP_CENTER))); + case 19: + return av_channel_layout_copy( + outLayout, &(AVChannelLayout) AV_CHANNEL_LAYOUT_7POINT1POINT4_BACK); + } + return AVERROR(EINVAL); +} + +static av_cold int mpegh3dadec_init(AVCodecContext *avctx) +{ + MPEGH3DADecContext *s = avctx->priv_data; + + if (!s->cicp) { + av_log(avctx, AV_LOG_ERROR, + "Target layout CICP needs to be specified\n"); + return AVERROR(EINVAL); + } + + s->decoder = NULL; + + avctx->delay = 0; + avctx->sample_fmt = AV_SAMPLE_FMT_S32; + avctx->sample_rate = 48000; + if (set_channel_layout(&avctx->ch_layout, s->cicp) || + avctx->ch_layout.nb_channels == 0) { + av_log(avctx, AV_LOG_ERROR, "Unsupported CICP index: %u.\n", s->cicp); + mpegh3dadec_close(avctx); + return AVERROR(EINVAL); + } + + s->decoder_buffer_size = MAX_OUTBUF_SIZE; + s->decoder_buffer = av_malloc(s->decoder_buffer_size); + if (!s->decoder_buffer) { + mpegh3dadec_close(avctx); + return AVERROR(ENOMEM); + } + + // initialize the decoder + s->decoder = mpeghdecoder_init(s->cicp); + if (s->decoder == NULL) { + av_log(avctx, AV_LOG_ERROR, "MPEG-H decoder library init failed.\n"); + mpegh3dadec_close(avctx); + return AVERROR_EXTERNAL; + } + + if (avctx->extradata_size) { + if (mpeghdecoder_setMhaConfig(s->decoder, avctx->extradata, + avctx->extradata_size)) { + av_log(avctx, AV_LOG_ERROR, "Unable to set MHA configuration\n"); + mpegh3dadec_close(avctx); + return AVERROR_INVALIDDATA; + } + } + + return 0; +} + +static int mpegh3dadec_decode_frame(AVCodecContext *avctx, AVFrame *frame, + int *got_frame_ptr, AVPacket *avpkt) +{ + MPEGH3DADecContext *s = avctx->priv_data; + int ret; + MPEGH_DECODER_ERROR err; + MPEGH_DECODER_OUTPUT_INFO out_info; + + if (!avctx->sample_rate) { + av_log(avctx, AV_LOG_ERROR, "Audio sample rate is not set"); + return AVERROR_INVALIDDATA; + } + + if (avpkt->data != NULL && avpkt->size > 0) { + if ((err = mpeghdecoder_processTimescale(s->decoder, avpkt->data, + avpkt->size, avpkt->pts, + avctx->sample_rate))) { + av_log(avctx, AV_LOG_ERROR, "mpeghdecoder_process() failed: %x\n", + err); + return AVERROR_INVALIDDATA; + } + } else { + // we are flushing + err = mpeghdecoder_flushAndGet(s->decoder); + + if (err != MPEGH_DEC_OK && err != MPEGH_DEC_FEED_DATA) + av_log(avctx, AV_LOG_WARNING, + "mpeghdecoder_flushAndGet() failed: %d\n", err); + } + + err = mpeghdecoder_getSamples(s->decoder, s->decoder_buffer, + s->decoder_buffer_size / sizeof(int32_t), + &out_info); + if (err == MPEGH_DEC_FEED_DATA) { + // no frames to produce at the moment + return avpkt->size; + } else if (err) { + av_log(avctx, AV_LOG_ERROR, "mpeghdecoder_getSamples() failed: %x\n", + err); + return AVERROR_UNKNOWN; + } + + frame->nb_samples = avctx->frame_size = out_info.numSamplesPerChannel; + frame->sample_rate = avctx->sample_rate = out_info.sampleRate; + frame->pts = out_info.ticks; + frame->time_base.num = 1; + frame->time_base.den = out_info.sampleRate; + + if ((ret = ff_get_buffer(avctx, frame, 0)) < 0) + return ret; + + memcpy(frame->extended_data[0], s->decoder_buffer, + avctx->ch_layout.nb_channels * avctx->frame_size * + av_get_bytes_per_sample(avctx->sample_fmt)); + + *got_frame_ptr = 1; + return ret = avpkt->size; +} + +static av_cold void mpegh3dadec_flush(AVCodecContext *avctx) +{ + MPEGH_DECODER_ERROR err; + MPEGH3DADecContext *s = avctx->priv_data; + + if (!s->decoder) + return; + + err = mpeghdecoder_flush(s->decoder); + + if (err != MPEGH_DEC_OK && err != MPEGH_DEC_FEED_DATA) + av_log(avctx, AV_LOG_WARNING, "mpeghdecoder_flush failed: %d\n", err); +} + +const FFCodec ff_libmpeghdec_decoder = { + .p.name = "libmpeghdec", + CODEC_LONG_NAME("libmpeghdec (MPEG-H 3D Audio)"), + .p.priv_class = &mpegh3da_class, + .p.type = AVMEDIA_TYPE_AUDIO, + .p.id = AV_CODEC_ID_MPEGH_3D_AUDIO, + .p.capabilities = + AV_CODEC_CAP_DR1 | AV_CODEC_CAP_DELAY | AV_CODEC_CAP_CHANNEL_CONF, + .priv_data_size = sizeof(MPEGH3DADecContext), + .init = mpegh3dadec_init, + FF_CODEC_DECODE_CB(mpegh3dadec_decode_frame), + .close = mpegh3dadec_close, + .flush = mpegh3dadec_flush, + CODEC_SAMPLEFMTS(AV_SAMPLE_FMT_S32), + .caps_internal = 0, + .p.wrapper_name = "libmpeghdec", +}; diff --git a/libavcodec/version.h b/libavcodec/version.h index fb4cb6c86b..230d5fa13e 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -29,8 +29,8 @@ #include "version_major.h" -#define LIBAVCODEC_VERSION_MINOR 8 -#define LIBAVCODEC_VERSION_MICRO 101 +#define LIBAVCODEC_VERSION_MINOR 9 +#define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ LIBAVCODEC_VERSION_MINOR, \ -- 2.49.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
reply other threads:[~2025-08-06 6:54 UTC|newest] Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20250806065420.E68D868BA68@ffbox0-bg.ffmpeg.org \ --to=code@ffmpeg.org \ --cc=ffmpeg-devel@ffmpeg.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel This inbox may be cloned and mirrored by anyone: git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \ ffmpegdev@gitmailbox.com public-inbox-index ffmpegdev Example config snippet for mirrors. AGPL code for this site: git clone https://public-inbox.org/public-inbox.git