Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
 help / color / mirror / Atom feed
* [FFmpeg-devel] [PATCH 1/2] avcodec/liblc3: Add encoding/decoding support of LC3 audio codec
@ 2024-03-28 21:35 Antoine Soulier via ffmpeg-devel
  2024-03-28 21:35 ` [FFmpeg-devel] [PATCH 2/2] avformat/lc3: Add file format for LC3/LC3plus transport Antoine Soulier via ffmpeg-devel
  0 siblings, 1 reply; 17+ messages in thread
From: Antoine Soulier via ffmpeg-devel @ 2024-03-28 21:35 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: Antoine Soulier

The LC3 audio codec is the default codec of Bluetooth LE audio.
This is a wrapper over the liblc3 library (https://github.com/google/liblc3).

Signed-off-by: Antoine Soulier <asoulier@google.com>
---
 Changelog                 |   4 +
 configure                 |   6 ++
 doc/encoders.texi         |  57 ++++++++++++
 doc/general_contents.texi |  11 ++-
 libavcodec/Makefile       |   2 +
 libavcodec/allcodecs.c    |   2 +
 libavcodec/codec_desc.c   |   7 ++
 libavcodec/codec_id.h     |   1 +
 libavcodec/liblc3dec.c    | 135 +++++++++++++++++++++++++++
 libavcodec/liblc3enc.c    | 190 ++++++++++++++++++++++++++++++++++++++
 10 files changed, 414 insertions(+), 1 deletion(-)
 create mode 100644 libavcodec/liblc3dec.c
 create mode 100644 libavcodec/liblc3enc.c

diff --git a/Changelog b/Changelog
index e83a00e35c..83a4cf7888 100644
--- a/Changelog
+++ b/Changelog
@@ -1,6 +1,10 @@
 Entries are sorted chronologically from oldest to youngest within each release,
 releases are sorted from youngest to oldest.
 
+version <next>:
+- LC3/LC3plus decoding/encoding using external library liblc3
+
+
 version 7.0:
 - DXV DXT1 encoder
 - LEAD MCMP decoder
diff --git a/configure b/configure
index 2d46ef0b9c..e5d9ba9f53 100755
--- a/configure
+++ b/configure
@@ -244,6 +244,7 @@ External library support:
   --enable-libjxl          enable JPEG XL de/encoding via libjxl [no]
   --enable-libklvanc       enable Kernel Labs VANC processing [no]
   --enable-libkvazaar      enable HEVC encoding via libkvazaar [no]
+  --enable-liblc3          enable LC3 de/encoding via liblc3 [no]
   --enable-liblensfun      enable lensfun lens correction [no]
   --enable-libmodplug      enable ModPlug via libmodplug [no]
   --enable-libmp3lame      enable MP3 encoding via libmp3lame [no]
@@ -1926,6 +1927,7 @@ EXTERNAL_LIBRARY_LIST="
     libjxl
     libklvanc
     libkvazaar
+    liblc3
     libmodplug
     libmp3lame
     libmysofa
@@ -3499,6 +3501,9 @@ libilbc_encoder_deps="libilbc"
 libjxl_decoder_deps="libjxl libjxl_threads"
 libjxl_encoder_deps="libjxl libjxl_threads"
 libkvazaar_encoder_deps="libkvazaar"
+liblc3_decoder_deps="liblc3"
+liblc3_encoder_deps="liblc3"
+liblc3_encoder_select="audio_frame_queue"
 libmodplug_demuxer_deps="libmodplug"
 libmp3lame_encoder_deps="libmp3lame"
 libmp3lame_encoder_select="audio_frame_queue mpegaudioheader"
@@ -6869,6 +6874,7 @@ enabled libjxl            && require_pkg_config libjxl "libjxl >= 0.7.0" jxl/dec
                              require_pkg_config libjxl_threads "libjxl_threads >= 0.7.0" jxl/thread_parallel_runner.h JxlThreadParallelRunner
 enabled libklvanc         && require libklvanc libklvanc/vanc.h klvanc_context_create -lklvanc
 enabled libkvazaar        && require_pkg_config libkvazaar "kvazaar >= 2.0.0" kvazaar.h kvz_api_get
+enabled liblc3            && require_pkg_config liblc3 "lc3 >= 1.1.0" lc3.h lc3_hr_setup_encoder
 enabled liblensfun        && require_pkg_config liblensfun lensfun lensfun.h lf_db_create
 
 if enabled libmfx && enabled libvpl; then
diff --git a/doc/encoders.texi b/doc/encoders.texi
index 7c223ed74c..66847191e1 100644
--- a/doc/encoders.texi
+++ b/doc/encoders.texi
@@ -814,6 +814,63 @@ ffmpeg -i input.wav -c:a libfdk_aac -profile:a aac_he -b:a 64k output.m4a
 @end example
 @end itemize
 
+@anchor{liblc3-enc}
+@section liblc3
+
+liblc3 LC3 (Low Complexity Communication Codec) encoder wrapper.
+
+Requires the presence of the liblc3 headers and library during configuration.
+You need to explicitly configure the build with @code{--enable-liblc3}.
+
+This encoder has support for the Bluetooth SIG LC3 codec for the LE Audio
+protocol, and the following features of LC3plus:
+@itemize
+@item
+Frame duration of 2.5 and 5ms.
+@item
+High-Resolution mode, 48 KHz, and 96 kHz sampling rates.
+@end itemize
+
+For more information see the liblc3 project at
+@url{https://github.com/google/liblc3}.
+
+@subsection Options
+
+The following options are mapped on the shared FFmpeg codec options.
+
+@table @option
+@item b @var{bitrate}
+Set the bit rate in bits/s. This will determine the fixed size of the encoded
+frames, for a selected frame duration.
+
+@item ar @var{frequency}
+Set the audio sampling rate (in Hz).
+
+@item channels
+Set the number of audio channels.
+
+@item frame_duration
+Set the audio frame duration in milliseconds. Default value is 10ms.
+Allowed frame durations are 2.5ms, 5ms, 7.5ms and 10ms.
+LC3 (Bluetooth LE Audio), allows 7.5ms and 10ms; and LC3plus 2.5ms, 5ms
+and 10ms.
+
+The 10ms frame duration is available in LC3 and LC3 plus standard.
+In this mode, the produced bitstream can be referenced either as LC3 or LC3plus.
+
+@item high_resolution @var{boolean}
+Enable the high-resolution mode if set to 1. The high-resolution mode is
+available with all LC3plus frame durations and for a sampling rate of 48 KHz,
+and 96 KHz.
+
+The encoder automatically turns off this mode at lower sampling rates and
+activates it at 96 KHz.
+
+This mode should be preferred at high bitrates. In this mode, the audio
+bandwidth is always up to the Nyquist frequency, compared to LC3 at 48 KHz,
+which limits the bandwidth to 20 KHz.
+@end table
+
 @anchor{libmp3lame}
 @section libmp3lame
 
diff --git a/doc/general_contents.texi b/doc/general_contents.texi
index f269cbd1a9..e7cf4f8239 100644
--- a/doc/general_contents.texi
+++ b/doc/general_contents.texi
@@ -237,6 +237,14 @@ Go to @url{http://sourceforge.net/projects/opencore-amr/} and follow the
 instructions for installing the library.
 Then pass @code{--enable-libfdk-aac} to configure to enable it.
 
+@subsection LC3 library
+
+FFmpeg can make use of the Google LC3 library for LC3 decoding & encoding.
+
+Go to @url{https://github.com/google/liblc3/} and follow the instructions for
+installing the library.
+Then pass @code{--enable-liblc3} to configure to enable it.
+
 @section OpenH264
 
 FFmpeg can make use of the OpenH264 library for H.264 decoding and encoding.
@@ -1300,7 +1308,8 @@ following image formats are supported:
     @tab encoding and decoding supported through external library libilbc
 @item IMC (Intel Music Coder)  @tab     @tab  X
 @item Interplay ACM            @tab     @tab  X
-@item MACE (Macintosh Audio Compression/Expansion) 3:1  @tab     @tab  X
+@item LC3                    @tab E  @tab  E
+    @tab supported through external library liblc3
 @item MACE (Macintosh Audio Compression/Expansion) 6:1  @tab     @tab  X
 @item Marian's A-pac audio     @tab     @tab  X
 @item MI-SC4 (Micronas SC-4 Audio)  @tab     @tab  X
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index 9ce6d445c1..e70811dbd6 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -1123,6 +1123,8 @@ OBJS-$(CONFIG_LIBILBC_ENCODER)            += libilbc.o
 OBJS-$(CONFIG_LIBJXL_DECODER)             += libjxldec.o libjxl.o
 OBJS-$(CONFIG_LIBJXL_ENCODER)             += libjxlenc.o libjxl.o
 OBJS-$(CONFIG_LIBKVAZAAR_ENCODER)         += libkvazaar.o
+OBJS-$(CONFIG_LIBLC3_ENCODER)             += liblc3enc.o
+OBJS-$(CONFIG_LIBLC3_DECODER)             += liblc3dec.o
 OBJS-$(CONFIG_LIBMP3LAME_ENCODER)         += libmp3lame.o
 OBJS-$(CONFIG_LIBOPENCORE_AMRNB_DECODER)  += libopencore-amr.o
 OBJS-$(CONFIG_LIBOPENCORE_AMRNB_ENCODER)  += libopencore-amr.o
diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c
index 2386b450a6..f4705651fb 100644
--- a/libavcodec/allcodecs.c
+++ b/libavcodec/allcodecs.c
@@ -776,6 +776,8 @@ extern const FFCodec ff_libilbc_encoder;
 extern const FFCodec ff_libilbc_decoder;
 extern const FFCodec ff_libjxl_decoder;
 extern const FFCodec ff_libjxl_encoder;
+extern const FFCodec ff_liblc3_encoder;
+extern const FFCodec ff_liblc3_decoder;
 extern const FFCodec ff_libmp3lame_encoder;
 extern const FFCodec ff_libopencore_amrnb_encoder;
 extern const FFCodec ff_libopencore_amrnb_decoder;
diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c
index 3bab86db62..7dba61dc8b 100644
--- a/libavcodec/codec_desc.c
+++ b/libavcodec/codec_desc.c
@@ -3425,6 +3425,13 @@ static const AVCodecDescriptor codec_descriptors[] = {
         .long_name = NULL_IF_CONFIG_SMALL("QOA (Quite OK Audio)"),
         .props     = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSY,
     },
+    {
+        .id        = AV_CODEC_ID_LC3,
+        .type      = AVMEDIA_TYPE_AUDIO,
+        .name      = "lc3",
+        .long_name = NULL_IF_CONFIG_SMALL("LC3 (Low Complexity Communication Codec)"),
+        .props     = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSY,
+    },
 
     /* subtitle codecs */
     {
diff --git a/libavcodec/codec_id.h b/libavcodec/codec_id.h
index c8dc21da74..0ab1e34a61 100644
--- a/libavcodec/codec_id.h
+++ b/libavcodec/codec_id.h
@@ -543,6 +543,7 @@ enum AVCodecID {
     AV_CODEC_ID_AC4,
     AV_CODEC_ID_OSQ,
     AV_CODEC_ID_QOA,
+    AV_CODEC_ID_LC3,
 
     /* subtitle codecs */
     AV_CODEC_ID_FIRST_SUBTITLE = 0x17000,          ///< A dummy ID pointing at the start of subtitle codecs.
diff --git a/libavcodec/liblc3dec.c b/libavcodec/liblc3dec.c
new file mode 100644
index 0000000000..3f747e70c9
--- /dev/null
+++ b/libavcodec/liblc3dec.c
@@ -0,0 +1,135 @@
+/*
+ * LC3 decoder wrapper
+ * Copyright (C) 2024  Antoine Soulier <asoulier@google.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * Permission to use, copy, modify, and/or distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#include <lc3.h>
+
+#include "libavutil/intreadwrite.h"
+
+#include "avcodec.h"
+#include "codec.h"
+#include "codec_internal.h"
+#include "decode.h"
+#include "internal.h"
+
+#define DECODER_MAX_CHANNELS  2
+
+typedef struct LibLC3DecContext {
+    int frame_us, srate_hz;
+    bool hr_mode;
+    void *decoder_mem;
+    lc3_decoder_t decoder[DECODER_MAX_CHANNELS];
+} LibLC3DecContext;
+
+static av_cold int liblc3_decode_init(AVCodecContext *avctx)
+{
+    LibLC3DecContext *liblc3 = avctx->priv_data;
+    int channels = avctx->ch_layout.nb_channels;
+    unsigned decoder_size;
+
+    if (avctx->extradata_size < 2)
+        return AVERROR_INVALIDDATA;
+
+    liblc3->frame_us = AV_RL16(avctx->extradata + 0);
+    liblc3->srate_hz = avctx->sample_rate;
+    liblc3->hr_mode  = avctx->extradata_size >= 6 &&
+                       AV_RL16(avctx->extradata + 4);
+
+    av_log(avctx, AV_LOG_INFO,
+           "Decoding %.1f ms frames\n", liblc3->frame_us / 1000.f);
+    if (liblc3->hr_mode)
+        av_log(avctx, AV_LOG_INFO, "High-resolution mode enabled\n");
+
+    decoder_size = lc3_hr_decoder_size(
+        liblc3->hr_mode, liblc3->frame_us, liblc3->srate_hz);
+    if (!decoder_size)
+        return AVERROR_INVALIDDATA;
+
+    liblc3->decoder_mem = av_malloc_array(channels, decoder_size);
+    if (!liblc3->decoder_mem)
+        return AVERROR(ENOMEM);
+
+    for (int ch = 0; ch < channels; ch++) {
+        liblc3->decoder[ch] = lc3_hr_setup_decoder(
+            liblc3->hr_mode, liblc3->frame_us, liblc3->srate_hz, 0,
+            (char *)liblc3->decoder_mem + ch * decoder_size);
+    }
+
+    avctx->sample_fmt = AV_SAMPLE_FMT_FLTP;
+    avctx->delay = lc3_hr_delay_samples(
+        liblc3->hr_mode, liblc3->frame_us, liblc3->srate_hz);
+    avctx->internal->skip_samples = avctx->delay;
+
+    return 0;
+}
+
+static av_cold int liblc3_decode_close(AVCodecContext *avctx)
+{
+    LibLC3DecContext *liblc3 = avctx->priv_data;
+
+    av_freep(&liblc3->decoder_mem);
+
+    return 0;
+}
+
+static int liblc3_decode(AVCodecContext *avctx, AVFrame *frame,
+                         int *got_frame_ptr, AVPacket *avpkt)
+{
+    LibLC3DecContext *liblc3 = avctx->priv_data;
+    int channels = avctx->ch_layout.nb_channels;
+    uint8_t *in = avpkt->data;
+    int block_bytes, ret;
+
+    frame->nb_samples = av_rescale(
+        liblc3->frame_us, liblc3->srate_hz, 1000*1000);
+    if ((ret = ff_get_buffer(avctx, frame, 0)) < 0)
+        return ret;
+
+    block_bytes = avpkt->size;
+    for (int ch = 0; ch < channels; ch++) {
+        int frame_bytes = block_bytes / channels
+            + (ch < block_bytes % channels);
+
+        ret = lc3_decode(liblc3->decoder[ch], in, frame_bytes,
+                         LC3_PCM_FORMAT_FLOAT, frame->data[ch], 1);
+        if (ret < 0)
+            return AVERROR_INVALIDDATA;
+
+        in += frame_bytes;
+    }
+
+    frame->nb_samples = FFMIN(frame->nb_samples, avpkt->duration);
+
+    *got_frame_ptr = 1;
+
+    return avpkt->size;
+}
+
+const FFCodec ff_liblc3_decoder = {
+    .p.name         = "liblc3",
+    CODEC_LONG_NAME("LC3 (Low Complexity Communication Codec)"),
+    .p.type         = AVMEDIA_TYPE_AUDIO,
+    .p.id           = AV_CODEC_ID_LC3,
+    .p.capabilities = AV_CODEC_CAP_DR1,
+    .p.wrapper_name = "liblc3",
+    .priv_data_size = sizeof(LibLC3DecContext),
+    .caps_internal  = FF_CODEC_CAP_INIT_CLEANUP,
+    .init           = liblc3_decode_init,
+    .close          = liblc3_decode_close,
+    FF_CODEC_DECODE_CB(liblc3_decode),
+};
diff --git a/libavcodec/liblc3enc.c b/libavcodec/liblc3enc.c
new file mode 100644
index 0000000000..9a0953e960
--- /dev/null
+++ b/libavcodec/liblc3enc.c
@@ -0,0 +1,190 @@
+/*
+ * LC3 encoder wrapper
+ * Copyright (C) 2024  Antoine Soulier <asoulier@google.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * Permission to use, copy, modify, and/or distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#include <lc3.h>
+
+#include "libavutil/intreadwrite.h"
+#include "libavutil/opt.h"
+
+#include "avcodec.h"
+#include "codec.h"
+#include "codec_internal.h"
+#include "encode.h"
+
+#define ENCODER_MAX_CHANNELS  2
+
+typedef struct LibLC3EncOpts {
+    float frame_duration;
+    int hr_mode;
+} LibLC3EncOpts;
+
+typedef struct LibLC3EncContext {
+    const AVClass *av_class;
+    LibLC3EncOpts opts;
+    int block_bytes;
+    void *encoder_mem;
+    lc3_encoder_t encoder[ENCODER_MAX_CHANNELS];
+} LibLC3EncContext;
+
+static av_cold int liblc3_encode_init(AVCodecContext *avctx)
+{
+    LibLC3EncContext *liblc3 = avctx->priv_data;
+    bool hr_mode = liblc3->opts.hr_mode;
+    int frame_us = liblc3->opts.frame_duration * 1000;
+    int srate_hz = avctx->sample_rate;
+    int channels = avctx->ch_layout.nb_channels;
+    int effective_bit_rate;
+    unsigned encoder_size;
+
+    if (frame_us != 2500 && frame_us !=  5000 &&
+        frame_us != 7500 && frame_us != 10000   ) {
+        av_log(avctx, AV_LOG_ERROR,
+               "Unsupported frame duration %.1f ms\n", frame_us / 1000.f);
+        return AVERROR(EINVAL);
+    }
+
+    hr_mode |= srate_hz > 48000;
+    hr_mode &= srate_hz >= 48000;
+
+    if (frame_us == 7500 && hr_mode) {
+        av_log(avctx, AV_LOG_ERROR,
+               "High-resolution mode not supported with 7.5 ms frames\n");
+        return AVERROR(EINVAL);
+    }
+
+    av_log(avctx, AV_LOG_INFO, "Encoding %.1f ms frames\n", frame_us / 1000.f);
+    if (hr_mode)
+        av_log(avctx, AV_LOG_INFO, "High-resolution mode enabled\n");
+
+    liblc3->block_bytes = lc3_hr_frame_block_bytes(
+        hr_mode, frame_us, srate_hz, channels, avctx->bit_rate);
+
+    effective_bit_rate = lc3_hr_resolve_bitrate(
+        hr_mode, frame_us, srate_hz, liblc3->block_bytes);
+
+    if (avctx->bit_rate != effective_bit_rate)
+        av_log(avctx, AV_LOG_WARNING,
+               "Bitrate changed to %d bps\n", effective_bit_rate);
+    avctx->bit_rate = effective_bit_rate;
+
+    encoder_size = lc3_hr_encoder_size(hr_mode, frame_us, srate_hz);
+    if (!encoder_size)
+        return AVERROR(EINVAL);
+
+    liblc3->encoder_mem = av_malloc_array(channels, encoder_size);
+    if (!liblc3->encoder_mem)
+        return AVERROR(ENOMEM);
+
+    for (int ch = 0; ch < channels; ch++) {
+        liblc3->encoder[ch] = lc3_hr_setup_encoder(
+            hr_mode, frame_us, srate_hz, 0,
+            (char *)liblc3->encoder_mem + ch * encoder_size);
+    }
+
+    avctx->extradata = av_mallocz(6 + AV_INPUT_BUFFER_PADDING_SIZE);
+    if (!avctx->extradata)
+        return AVERROR(ENOMEM);
+
+    AV_WL16(avctx->extradata + 0, frame_us);
+    AV_WL16(avctx->extradata + 2, 0);
+    AV_WL16(avctx->extradata + 4, hr_mode);
+    avctx->extradata_size = 6;
+
+    avctx->frame_size = av_rescale(frame_us, srate_hz, 1000*1000);
+
+    return 0;
+}
+
+static av_cold int liblc3_encode_close(AVCodecContext *avctx)
+{
+    LibLC3EncContext *liblc3 = avctx->priv_data;
+
+    av_freep(&liblc3->encoder_mem);
+
+    return 0;
+}
+
+static int liblc3_encode(AVCodecContext *avctx, AVPacket *avpkt,
+                         const AVFrame *av_frame, int *got_packet_ptr)
+{
+    LibLC3EncContext *liblc3 = avctx->priv_data;
+    int block_bytes = liblc3->block_bytes;
+    int channels = avctx->ch_layout.nb_channels;
+    uint8_t *data_ptr;
+    int ret;
+
+    if ((ret = ff_get_encode_buffer(avctx, avpkt, block_bytes, 0)) < 0)
+        return ret;
+
+    data_ptr = avpkt->data;
+    for (int ch = 0; ch < channels; ch++) {
+        int frame_bytes = block_bytes / channels
+                + (ch < block_bytes % channels);
+
+        lc3_encode(liblc3->encoder[ch],
+                   LC3_PCM_FORMAT_FLOAT, av_frame->data[ch], 1,
+                   frame_bytes, data_ptr);
+
+        data_ptr += frame_bytes;
+    }
+
+    *got_packet_ptr = 1;
+
+    return 0;
+}
+
+#define OFFSET(x) offsetof(LibLC3EncContext, opts.x)
+#define FLAGS AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_ENCODING_PARAM
+static const AVOption options[] = {
+    { "frame_duration", "Duration of a frame in milliseconds",
+        OFFSET(frame_duration), AV_OPT_TYPE_FLOAT,
+        { .dbl = 10.0 }, 2.5, 10.0, FLAGS },
+    { "high_resolution", "Enable High-Resolution mode (48 KHz or 96 KHz)",
+        OFFSET(hr_mode), AV_OPT_TYPE_BOOL,
+        { .i64 = 0 }, 0, 1, FLAGS },
+    { NULL }
+};
+
+static const AVClass class = {
+    .class_name = "liblc3 encoder",
+    .item_name  = av_default_item_name,
+    .option     = options,
+    .version    = LIBAVUTIL_VERSION_INT,
+};
+
+const FFCodec ff_liblc3_encoder = {
+    .p.name         = "liblc3",
+    CODEC_LONG_NAME("LC3 (Low Complexity Communication Codec)"),
+    .p.type         = AVMEDIA_TYPE_AUDIO,
+    .p.id           = AV_CODEC_ID_LC3,
+    .p.capabilities = AV_CODEC_CAP_DR1,
+    .p.ch_layouts = (const AVChannelLayout[])
+        { { AV_CHANNEL_ORDER_UNSPEC, 1 },
+          { AV_CHANNEL_ORDER_UNSPEC, 2 }, { 0 } },
+    .p.supported_samplerates = (const int [])
+        { 96000, 48000, 32000, 24000, 16000, 8000, 0 },
+    .p.sample_fmts = (const enum AVSampleFormat[])
+        { AV_SAMPLE_FMT_FLTP, AV_SAMPLE_FMT_NONE },
+    .p.priv_class   = &class,
+    .p.wrapper_name = "liblc3",
+    .priv_data_size = sizeof(LibLC3EncContext),
+    .init           = liblc3_encode_init,
+    .close          = liblc3_encode_close,
+    FF_CODEC_ENCODE_CB(liblc3_encode),
+};
-- 
2.44.0.478.gd926399ef9-goog

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 17+ messages in thread
* [FFmpeg-devel] [PATCH 1/2] avcodec/liblc3: Add encoding/decoding support of LC3 audio codec
@ 2024-03-28 22:33 Antoine Soulier via ffmpeg-devel
  2024-03-28 22:33 ` [FFmpeg-devel] [PATCH 2/2] avformat/lc3: Add file format for LC3/LC3plus transport Antoine Soulier via ffmpeg-devel
  0 siblings, 1 reply; 17+ messages in thread
From: Antoine Soulier via ffmpeg-devel @ 2024-03-28 22:33 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: Antoine Soulier

The LC3 audio codec is the default codec of Bluetooth LE audio.
This is a wrapper over the liblc3 library (https://github.com/google/liblc3).

Signed-off-by: Antoine Soulier <asoulier@google.com>
---
 Changelog                 |   4 +
 configure                 |   6 ++
 doc/encoders.texi         |  57 ++++++++++++
 doc/general_contents.texi |  11 ++-
 libavcodec/Makefile       |   2 +
 libavcodec/allcodecs.c    |   2 +
 libavcodec/codec_desc.c   |   7 ++
 libavcodec/codec_id.h     |   1 +
 libavcodec/liblc3dec.c    | 141 ++++++++++++++++++++++++++++
 libavcodec/liblc3enc.c    | 190 ++++++++++++++++++++++++++++++++++++++
 10 files changed, 420 insertions(+), 1 deletion(-)
 create mode 100644 libavcodec/liblc3dec.c
 create mode 100644 libavcodec/liblc3enc.c

diff --git a/Changelog b/Changelog
index e83a00e35c..83a4cf7888 100644
--- a/Changelog
+++ b/Changelog
@@ -1,6 +1,10 @@
 Entries are sorted chronologically from oldest to youngest within each release,
 releases are sorted from youngest to oldest.
 
+version <next>:
+- LC3/LC3plus decoding/encoding using external library liblc3
+
+
 version 7.0:
 - DXV DXT1 encoder
 - LEAD MCMP decoder
diff --git a/configure b/configure
index 2d46ef0b9c..e5d9ba9f53 100755
--- a/configure
+++ b/configure
@@ -244,6 +244,7 @@ External library support:
   --enable-libjxl          enable JPEG XL de/encoding via libjxl [no]
   --enable-libklvanc       enable Kernel Labs VANC processing [no]
   --enable-libkvazaar      enable HEVC encoding via libkvazaar [no]
+  --enable-liblc3          enable LC3 de/encoding via liblc3 [no]
   --enable-liblensfun      enable lensfun lens correction [no]
   --enable-libmodplug      enable ModPlug via libmodplug [no]
   --enable-libmp3lame      enable MP3 encoding via libmp3lame [no]
@@ -1926,6 +1927,7 @@ EXTERNAL_LIBRARY_LIST="
     libjxl
     libklvanc
     libkvazaar
+    liblc3
     libmodplug
     libmp3lame
     libmysofa
@@ -3499,6 +3501,9 @@ libilbc_encoder_deps="libilbc"
 libjxl_decoder_deps="libjxl libjxl_threads"
 libjxl_encoder_deps="libjxl libjxl_threads"
 libkvazaar_encoder_deps="libkvazaar"
+liblc3_decoder_deps="liblc3"
+liblc3_encoder_deps="liblc3"
+liblc3_encoder_select="audio_frame_queue"
 libmodplug_demuxer_deps="libmodplug"
 libmp3lame_encoder_deps="libmp3lame"
 libmp3lame_encoder_select="audio_frame_queue mpegaudioheader"
@@ -6869,6 +6874,7 @@ enabled libjxl            && require_pkg_config libjxl "libjxl >= 0.7.0" jxl/dec
                              require_pkg_config libjxl_threads "libjxl_threads >= 0.7.0" jxl/thread_parallel_runner.h JxlThreadParallelRunner
 enabled libklvanc         && require libklvanc libklvanc/vanc.h klvanc_context_create -lklvanc
 enabled libkvazaar        && require_pkg_config libkvazaar "kvazaar >= 2.0.0" kvazaar.h kvz_api_get
+enabled liblc3            && require_pkg_config liblc3 "lc3 >= 1.1.0" lc3.h lc3_hr_setup_encoder
 enabled liblensfun        && require_pkg_config liblensfun lensfun lensfun.h lf_db_create
 
 if enabled libmfx && enabled libvpl; then
diff --git a/doc/encoders.texi b/doc/encoders.texi
index 7c223ed74c..66847191e1 100644
--- a/doc/encoders.texi
+++ b/doc/encoders.texi
@@ -814,6 +814,63 @@ ffmpeg -i input.wav -c:a libfdk_aac -profile:a aac_he -b:a 64k output.m4a
 @end example
 @end itemize
 
+@anchor{liblc3-enc}
+@section liblc3
+
+liblc3 LC3 (Low Complexity Communication Codec) encoder wrapper.
+
+Requires the presence of the liblc3 headers and library during configuration.
+You need to explicitly configure the build with @code{--enable-liblc3}.
+
+This encoder has support for the Bluetooth SIG LC3 codec for the LE Audio
+protocol, and the following features of LC3plus:
+@itemize
+@item
+Frame duration of 2.5 and 5ms.
+@item
+High-Resolution mode, 48 KHz, and 96 kHz sampling rates.
+@end itemize
+
+For more information see the liblc3 project at
+@url{https://github.com/google/liblc3}.
+
+@subsection Options
+
+The following options are mapped on the shared FFmpeg codec options.
+
+@table @option
+@item b @var{bitrate}
+Set the bit rate in bits/s. This will determine the fixed size of the encoded
+frames, for a selected frame duration.
+
+@item ar @var{frequency}
+Set the audio sampling rate (in Hz).
+
+@item channels
+Set the number of audio channels.
+
+@item frame_duration
+Set the audio frame duration in milliseconds. Default value is 10ms.
+Allowed frame durations are 2.5ms, 5ms, 7.5ms and 10ms.
+LC3 (Bluetooth LE Audio), allows 7.5ms and 10ms; and LC3plus 2.5ms, 5ms
+and 10ms.
+
+The 10ms frame duration is available in LC3 and LC3 plus standard.
+In this mode, the produced bitstream can be referenced either as LC3 or LC3plus.
+
+@item high_resolution @var{boolean}
+Enable the high-resolution mode if set to 1. The high-resolution mode is
+available with all LC3plus frame durations and for a sampling rate of 48 KHz,
+and 96 KHz.
+
+The encoder automatically turns off this mode at lower sampling rates and
+activates it at 96 KHz.
+
+This mode should be preferred at high bitrates. In this mode, the audio
+bandwidth is always up to the Nyquist frequency, compared to LC3 at 48 KHz,
+which limits the bandwidth to 20 KHz.
+@end table
+
 @anchor{libmp3lame}
 @section libmp3lame
 
diff --git a/doc/general_contents.texi b/doc/general_contents.texi
index f269cbd1a9..e7cf4f8239 100644
--- a/doc/general_contents.texi
+++ b/doc/general_contents.texi
@@ -237,6 +237,14 @@ Go to @url{http://sourceforge.net/projects/opencore-amr/} and follow the
 instructions for installing the library.
 Then pass @code{--enable-libfdk-aac} to configure to enable it.
 
+@subsection LC3 library
+
+FFmpeg can make use of the Google LC3 library for LC3 decoding & encoding.
+
+Go to @url{https://github.com/google/liblc3/} and follow the instructions for
+installing the library.
+Then pass @code{--enable-liblc3} to configure to enable it.
+
 @section OpenH264
 
 FFmpeg can make use of the OpenH264 library for H.264 decoding and encoding.
@@ -1300,7 +1308,8 @@ following image formats are supported:
     @tab encoding and decoding supported through external library libilbc
 @item IMC (Intel Music Coder)  @tab     @tab  X
 @item Interplay ACM            @tab     @tab  X
-@item MACE (Macintosh Audio Compression/Expansion) 3:1  @tab     @tab  X
+@item LC3                    @tab E  @tab  E
+    @tab supported through external library liblc3
 @item MACE (Macintosh Audio Compression/Expansion) 6:1  @tab     @tab  X
 @item Marian's A-pac audio     @tab     @tab  X
 @item MI-SC4 (Micronas SC-4 Audio)  @tab     @tab  X
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index 9ce6d445c1..e70811dbd6 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -1123,6 +1123,8 @@ OBJS-$(CONFIG_LIBILBC_ENCODER)            += libilbc.o
 OBJS-$(CONFIG_LIBJXL_DECODER)             += libjxldec.o libjxl.o
 OBJS-$(CONFIG_LIBJXL_ENCODER)             += libjxlenc.o libjxl.o
 OBJS-$(CONFIG_LIBKVAZAAR_ENCODER)         += libkvazaar.o
+OBJS-$(CONFIG_LIBLC3_ENCODER)             += liblc3enc.o
+OBJS-$(CONFIG_LIBLC3_DECODER)             += liblc3dec.o
 OBJS-$(CONFIG_LIBMP3LAME_ENCODER)         += libmp3lame.o
 OBJS-$(CONFIG_LIBOPENCORE_AMRNB_DECODER)  += libopencore-amr.o
 OBJS-$(CONFIG_LIBOPENCORE_AMRNB_ENCODER)  += libopencore-amr.o
diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c
index 2386b450a6..f4705651fb 100644
--- a/libavcodec/allcodecs.c
+++ b/libavcodec/allcodecs.c
@@ -776,6 +776,8 @@ extern const FFCodec ff_libilbc_encoder;
 extern const FFCodec ff_libilbc_decoder;
 extern const FFCodec ff_libjxl_decoder;
 extern const FFCodec ff_libjxl_encoder;
+extern const FFCodec ff_liblc3_encoder;
+extern const FFCodec ff_liblc3_decoder;
 extern const FFCodec ff_libmp3lame_encoder;
 extern const FFCodec ff_libopencore_amrnb_encoder;
 extern const FFCodec ff_libopencore_amrnb_decoder;
diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c
index 3bab86db62..7dba61dc8b 100644
--- a/libavcodec/codec_desc.c
+++ b/libavcodec/codec_desc.c
@@ -3425,6 +3425,13 @@ static const AVCodecDescriptor codec_descriptors[] = {
         .long_name = NULL_IF_CONFIG_SMALL("QOA (Quite OK Audio)"),
         .props     = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSY,
     },
+    {
+        .id        = AV_CODEC_ID_LC3,
+        .type      = AVMEDIA_TYPE_AUDIO,
+        .name      = "lc3",
+        .long_name = NULL_IF_CONFIG_SMALL("LC3 (Low Complexity Communication Codec)"),
+        .props     = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSY,
+    },
 
     /* subtitle codecs */
     {
diff --git a/libavcodec/codec_id.h b/libavcodec/codec_id.h
index c8dc21da74..0ab1e34a61 100644
--- a/libavcodec/codec_id.h
+++ b/libavcodec/codec_id.h
@@ -543,6 +543,7 @@ enum AVCodecID {
     AV_CODEC_ID_AC4,
     AV_CODEC_ID_OSQ,
     AV_CODEC_ID_QOA,
+    AV_CODEC_ID_LC3,
 
     /* subtitle codecs */
     AV_CODEC_ID_FIRST_SUBTITLE = 0x17000,          ///< A dummy ID pointing at the start of subtitle codecs.
diff --git a/libavcodec/liblc3dec.c b/libavcodec/liblc3dec.c
new file mode 100644
index 0000000000..32d8fade91
--- /dev/null
+++ b/libavcodec/liblc3dec.c
@@ -0,0 +1,141 @@
+/*
+ * LC3 decoder wrapper
+ * Copyright (C) 2024  Antoine Soulier <asoulier@google.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * Permission to use, copy, modify, and/or distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#include <lc3.h>
+
+#include "libavutil/intreadwrite.h"
+
+#include "avcodec.h"
+#include "codec.h"
+#include "codec_internal.h"
+#include "decode.h"
+#include "internal.h"
+
+#define DECODER_MAX_CHANNELS  2
+
+typedef struct LibLC3DecContext {
+    int frame_us, srate_hz;
+    bool hr_mode;
+    void *decoder_mem;
+    lc3_decoder_t decoder[DECODER_MAX_CHANNELS];
+} LibLC3DecContext;
+
+static av_cold int liblc3_decode_init(AVCodecContext *avctx)
+{
+    LibLC3DecContext *liblc3 = avctx->priv_data;
+    int channels = avctx->ch_layout.nb_channels;
+    int ep_mode;
+    unsigned decoder_size;
+
+    if (avctx->extradata_size < 6)
+        return AVERROR_INVALIDDATA;
+
+    liblc3->frame_us = AV_RL16(avctx->extradata + 0) * 10;
+    liblc3->srate_hz = avctx->sample_rate;
+    ep_mode          = AV_RL16(avctx->extradata + 2);
+    liblc3->hr_mode  = AV_RL16(avctx->extradata + 4);
+    if (ep_mode != 0) {
+        av_log(avctx, AV_LOG_ERROR,
+               "Error protection mode is not supported.\n");
+        return AVERROR(EINVAL);
+    }
+
+    av_log(avctx, AV_LOG_INFO,
+           "Decoding %.1f ms frames.\n", liblc3->frame_us / 1000.f);
+    if (liblc3->hr_mode)
+        av_log(avctx, AV_LOG_INFO, "High-resolution mode enabled.\n");
+
+    decoder_size = lc3_hr_decoder_size(
+        liblc3->hr_mode, liblc3->frame_us, liblc3->srate_hz);
+    if (!decoder_size)
+        return AVERROR_INVALIDDATA;
+
+    liblc3->decoder_mem = av_malloc_array(channels, decoder_size);
+    if (!liblc3->decoder_mem)
+        return AVERROR(ENOMEM);
+
+    for (int ch = 0; ch < channels; ch++) {
+        liblc3->decoder[ch] = lc3_hr_setup_decoder(
+            liblc3->hr_mode, liblc3->frame_us, liblc3->srate_hz, 0,
+            (char *)liblc3->decoder_mem + ch * decoder_size);
+    }
+
+    avctx->sample_fmt = AV_SAMPLE_FMT_FLTP;
+    avctx->delay = lc3_hr_delay_samples(
+        liblc3->hr_mode, liblc3->frame_us, liblc3->srate_hz);
+    avctx->internal->skip_samples = avctx->delay;
+
+    return 0;
+}
+
+static av_cold int liblc3_decode_close(AVCodecContext *avctx)
+{
+    LibLC3DecContext *liblc3 = avctx->priv_data;
+
+    av_freep(&liblc3->decoder_mem);
+
+    return 0;
+}
+
+static int liblc3_decode(AVCodecContext *avctx, AVFrame *frame,
+                         int *got_frame_ptr, AVPacket *avpkt)
+{
+    LibLC3DecContext *liblc3 = avctx->priv_data;
+    int channels = avctx->ch_layout.nb_channels;
+    uint8_t *in = avpkt->data;
+    int block_bytes, ret;
+
+    frame->nb_samples = av_rescale(
+        liblc3->frame_us, liblc3->srate_hz, 1000*1000);
+    if ((ret = ff_get_buffer(avctx, frame, 0)) < 0)
+        return ret;
+
+    block_bytes = avpkt->size;
+    for (int ch = 0; ch < channels; ch++) {
+        int frame_bytes = block_bytes / channels
+            + (ch < block_bytes % channels);
+
+        ret = lc3_decode(liblc3->decoder[ch], in, frame_bytes,
+                         LC3_PCM_FORMAT_FLOAT, frame->data[ch], 1);
+        if (ret < 0)
+            return AVERROR_INVALIDDATA;
+
+        in += frame_bytes;
+    }
+
+    frame->nb_samples = FFMIN(frame->nb_samples, avpkt->duration);
+
+    *got_frame_ptr = 1;
+
+    return avpkt->size;
+}
+
+const FFCodec ff_liblc3_decoder = {
+    .p.name         = "liblc3",
+    CODEC_LONG_NAME("LC3 (Low Complexity Communication Codec)"),
+    .p.type         = AVMEDIA_TYPE_AUDIO,
+    .p.id           = AV_CODEC_ID_LC3,
+    .p.capabilities = AV_CODEC_CAP_DR1,
+    .p.wrapper_name = "liblc3",
+    .priv_data_size = sizeof(LibLC3DecContext),
+    .caps_internal  = FF_CODEC_CAP_INIT_CLEANUP,
+    .init           = liblc3_decode_init,
+    .close          = liblc3_decode_close,
+    FF_CODEC_DECODE_CB(liblc3_decode),
+};
diff --git a/libavcodec/liblc3enc.c b/libavcodec/liblc3enc.c
new file mode 100644
index 0000000000..7d5d92303c
--- /dev/null
+++ b/libavcodec/liblc3enc.c
@@ -0,0 +1,190 @@
+/*
+ * LC3 encoder wrapper
+ * Copyright (C) 2024  Antoine Soulier <asoulier@google.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * Permission to use, copy, modify, and/or distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#include <lc3.h>
+
+#include "libavutil/intreadwrite.h"
+#include "libavutil/opt.h"
+
+#include "avcodec.h"
+#include "codec.h"
+#include "codec_internal.h"
+#include "encode.h"
+
+#define ENCODER_MAX_CHANNELS  2
+
+typedef struct LibLC3EncOpts {
+    float frame_duration;
+    int hr_mode;
+} LibLC3EncOpts;
+
+typedef struct LibLC3EncContext {
+    const AVClass *av_class;
+    LibLC3EncOpts opts;
+    int block_bytes;
+    void *encoder_mem;
+    lc3_encoder_t encoder[ENCODER_MAX_CHANNELS];
+} LibLC3EncContext;
+
+static av_cold int liblc3_encode_init(AVCodecContext *avctx)
+{
+    LibLC3EncContext *liblc3 = avctx->priv_data;
+    bool hr_mode = liblc3->opts.hr_mode;
+    int frame_us = liblc3->opts.frame_duration * 1000;
+    int srate_hz = avctx->sample_rate;
+    int channels = avctx->ch_layout.nb_channels;
+    int effective_bit_rate;
+    unsigned encoder_size;
+
+    if (frame_us != 2500 && frame_us !=  5000 &&
+        frame_us != 7500 && frame_us != 10000   ) {
+        av_log(avctx, AV_LOG_ERROR,
+               "Unsupported frame duration %.1f ms.\n", frame_us / 1000.f);
+        return AVERROR(EINVAL);
+    }
+
+    hr_mode |= srate_hz > 48000;
+    hr_mode &= srate_hz >= 48000;
+
+    if (frame_us == 7500 && hr_mode) {
+        av_log(avctx, AV_LOG_ERROR,
+               "High-resolution mode is not supported with 7.5 ms frames.\n");
+        return AVERROR(EINVAL);
+    }
+
+    av_log(avctx, AV_LOG_INFO, "Encoding %.1f ms frames.\n", frame_us / 1000.f);
+    if (hr_mode)
+        av_log(avctx, AV_LOG_INFO, "High-resolution mode is enabled.\n");
+
+    liblc3->block_bytes = lc3_hr_frame_block_bytes(
+        hr_mode, frame_us, srate_hz, channels, avctx->bit_rate);
+
+    effective_bit_rate = lc3_hr_resolve_bitrate(
+        hr_mode, frame_us, srate_hz, liblc3->block_bytes);
+
+    if (avctx->bit_rate != effective_bit_rate)
+        av_log(avctx, AV_LOG_WARNING,
+               "Bitrate changed to %d bps.\n", effective_bit_rate);
+    avctx->bit_rate = effective_bit_rate;
+
+    encoder_size = lc3_hr_encoder_size(hr_mode, frame_us, srate_hz);
+    if (!encoder_size)
+        return AVERROR(EINVAL);
+
+    liblc3->encoder_mem = av_malloc_array(channels, encoder_size);
+    if (!liblc3->encoder_mem)
+        return AVERROR(ENOMEM);
+
+    for (int ch = 0; ch < channels; ch++) {
+        liblc3->encoder[ch] = lc3_hr_setup_encoder(
+            hr_mode, frame_us, srate_hz, 0,
+            (char *)liblc3->encoder_mem + ch * encoder_size);
+    }
+
+    avctx->extradata = av_mallocz(6 + AV_INPUT_BUFFER_PADDING_SIZE);
+    if (!avctx->extradata)
+        return AVERROR(ENOMEM);
+
+    AV_WL16(avctx->extradata + 0, frame_us / 10);
+    AV_WL16(avctx->extradata + 2, 0);
+    AV_WL16(avctx->extradata + 4, hr_mode);
+    avctx->extradata_size = 6;
+
+    avctx->frame_size = av_rescale(frame_us, srate_hz, 1000*1000);
+
+    return 0;
+}
+
+static av_cold int liblc3_encode_close(AVCodecContext *avctx)
+{
+    LibLC3EncContext *liblc3 = avctx->priv_data;
+
+    av_freep(&liblc3->encoder_mem);
+
+    return 0;
+}
+
+static int liblc3_encode(AVCodecContext *avctx, AVPacket *avpkt,
+                         const AVFrame *av_frame, int *got_packet_ptr)
+{
+    LibLC3EncContext *liblc3 = avctx->priv_data;
+    int block_bytes = liblc3->block_bytes;
+    int channels = avctx->ch_layout.nb_channels;
+    uint8_t *data_ptr;
+    int ret;
+
+    if ((ret = ff_get_encode_buffer(avctx, avpkt, block_bytes, 0)) < 0)
+        return ret;
+
+    data_ptr = avpkt->data;
+    for (int ch = 0; ch < channels; ch++) {
+        int frame_bytes = block_bytes / channels
+                + (ch < block_bytes % channels);
+
+        lc3_encode(liblc3->encoder[ch],
+                   LC3_PCM_FORMAT_FLOAT, av_frame->data[ch], 1,
+                   frame_bytes, data_ptr);
+
+        data_ptr += frame_bytes;
+    }
+
+    *got_packet_ptr = 1;
+
+    return 0;
+}
+
+#define OFFSET(x) offsetof(LibLC3EncContext, opts.x)
+#define FLAGS AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_ENCODING_PARAM
+static const AVOption options[] = {
+    { "frame_duration", "Duration of a frame in milliseconds",
+        OFFSET(frame_duration), AV_OPT_TYPE_FLOAT,
+        { .dbl = 10.0 }, 2.5, 10.0, FLAGS },
+    { "high_resolution", "Enable High-Resolution mode (48 KHz or 96 KHz)",
+        OFFSET(hr_mode), AV_OPT_TYPE_BOOL,
+        { .i64 = 0 }, 0, 1, FLAGS },
+    { NULL }
+};
+
+static const AVClass class = {
+    .class_name = "liblc3 encoder",
+    .item_name  = av_default_item_name,
+    .option     = options,
+    .version    = LIBAVUTIL_VERSION_INT,
+};
+
+const FFCodec ff_liblc3_encoder = {
+    .p.name         = "liblc3",
+    CODEC_LONG_NAME("LC3 (Low Complexity Communication Codec)"),
+    .p.type         = AVMEDIA_TYPE_AUDIO,
+    .p.id           = AV_CODEC_ID_LC3,
+    .p.capabilities = AV_CODEC_CAP_DR1,
+    .p.ch_layouts = (const AVChannelLayout[])
+        { { AV_CHANNEL_ORDER_UNSPEC, 1 },
+          { AV_CHANNEL_ORDER_UNSPEC, 2 }, { 0 } },
+    .p.supported_samplerates = (const int [])
+        { 96000, 48000, 32000, 24000, 16000, 8000, 0 },
+    .p.sample_fmts = (const enum AVSampleFormat[])
+        { AV_SAMPLE_FMT_FLTP, AV_SAMPLE_FMT_NONE },
+    .p.priv_class   = &class,
+    .p.wrapper_name = "liblc3",
+    .priv_data_size = sizeof(LibLC3EncContext),
+    .init           = liblc3_encode_init,
+    .close          = liblc3_encode_close,
+    FF_CODEC_ENCODE_CB(liblc3_encode),
+};
-- 
2.44.0.478.gd926399ef9-goog

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 17+ messages in thread
* [FFmpeg-devel] [PATCH 1/2] avcodec/liblc3: Add encoding/decoding support of LC3 audio codec
@ 2024-03-29 17:30 Antoine Soulier via ffmpeg-devel
  2024-03-29 17:30 ` [FFmpeg-devel] [PATCH 2/2] avformat/lc3: Add file format for LC3/LC3plus transport Antoine Soulier via ffmpeg-devel
  0 siblings, 1 reply; 17+ messages in thread
From: Antoine Soulier via ffmpeg-devel @ 2024-03-29 17:30 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: Antoine Soulier

The LC3 audio codec is the default codec of Bluetooth LE audio.
This is a wrapper over the liblc3 library (https://github.com/google/liblc3).

Signed-off-by: Antoine Soulier <asoulier@google.com>
---
 Changelog                 |   4 +
 configure                 |   6 ++
 doc/encoders.texi         |  57 ++++++++++++
 doc/general_contents.texi |  11 ++-
 libavcodec/Makefile       |   2 +
 libavcodec/allcodecs.c    |   2 +
 libavcodec/codec_desc.c   |   7 ++
 libavcodec/codec_id.h     |   1 +
 libavcodec/liblc3dec.c    | 141 ++++++++++++++++++++++++++++
 libavcodec/liblc3enc.c    | 190 ++++++++++++++++++++++++++++++++++++++
 10 files changed, 420 insertions(+), 1 deletion(-)
 create mode 100644 libavcodec/liblc3dec.c
 create mode 100644 libavcodec/liblc3enc.c

diff --git a/Changelog b/Changelog
index e83a00e35c..83a4cf7888 100644
--- a/Changelog
+++ b/Changelog
@@ -1,6 +1,10 @@
 Entries are sorted chronologically from oldest to youngest within each release,
 releases are sorted from youngest to oldest.
 
+version <next>:
+- LC3/LC3plus decoding/encoding using external library liblc3
+
+
 version 7.0:
 - DXV DXT1 encoder
 - LEAD MCMP decoder
diff --git a/configure b/configure
index 2d46ef0b9c..e5d9ba9f53 100755
--- a/configure
+++ b/configure
@@ -244,6 +244,7 @@ External library support:
   --enable-libjxl          enable JPEG XL de/encoding via libjxl [no]
   --enable-libklvanc       enable Kernel Labs VANC processing [no]
   --enable-libkvazaar      enable HEVC encoding via libkvazaar [no]
+  --enable-liblc3          enable LC3 de/encoding via liblc3 [no]
   --enable-liblensfun      enable lensfun lens correction [no]
   --enable-libmodplug      enable ModPlug via libmodplug [no]
   --enable-libmp3lame      enable MP3 encoding via libmp3lame [no]
@@ -1926,6 +1927,7 @@ EXTERNAL_LIBRARY_LIST="
     libjxl
     libklvanc
     libkvazaar
+    liblc3
     libmodplug
     libmp3lame
     libmysofa
@@ -3499,6 +3501,9 @@ libilbc_encoder_deps="libilbc"
 libjxl_decoder_deps="libjxl libjxl_threads"
 libjxl_encoder_deps="libjxl libjxl_threads"
 libkvazaar_encoder_deps="libkvazaar"
+liblc3_decoder_deps="liblc3"
+liblc3_encoder_deps="liblc3"
+liblc3_encoder_select="audio_frame_queue"
 libmodplug_demuxer_deps="libmodplug"
 libmp3lame_encoder_deps="libmp3lame"
 libmp3lame_encoder_select="audio_frame_queue mpegaudioheader"
@@ -6869,6 +6874,7 @@ enabled libjxl            && require_pkg_config libjxl "libjxl >= 0.7.0" jxl/dec
                              require_pkg_config libjxl_threads "libjxl_threads >= 0.7.0" jxl/thread_parallel_runner.h JxlThreadParallelRunner
 enabled libklvanc         && require libklvanc libklvanc/vanc.h klvanc_context_create -lklvanc
 enabled libkvazaar        && require_pkg_config libkvazaar "kvazaar >= 2.0.0" kvazaar.h kvz_api_get
+enabled liblc3            && require_pkg_config liblc3 "lc3 >= 1.1.0" lc3.h lc3_hr_setup_encoder
 enabled liblensfun        && require_pkg_config liblensfun lensfun lensfun.h lf_db_create
 
 if enabled libmfx && enabled libvpl; then
diff --git a/doc/encoders.texi b/doc/encoders.texi
index 7c223ed74c..66847191e1 100644
--- a/doc/encoders.texi
+++ b/doc/encoders.texi
@@ -814,6 +814,63 @@ ffmpeg -i input.wav -c:a libfdk_aac -profile:a aac_he -b:a 64k output.m4a
 @end example
 @end itemize
 
+@anchor{liblc3-enc}
+@section liblc3
+
+liblc3 LC3 (Low Complexity Communication Codec) encoder wrapper.
+
+Requires the presence of the liblc3 headers and library during configuration.
+You need to explicitly configure the build with @code{--enable-liblc3}.
+
+This encoder has support for the Bluetooth SIG LC3 codec for the LE Audio
+protocol, and the following features of LC3plus:
+@itemize
+@item
+Frame duration of 2.5 and 5ms.
+@item
+High-Resolution mode, 48 KHz, and 96 kHz sampling rates.
+@end itemize
+
+For more information see the liblc3 project at
+@url{https://github.com/google/liblc3}.
+
+@subsection Options
+
+The following options are mapped on the shared FFmpeg codec options.
+
+@table @option
+@item b @var{bitrate}
+Set the bit rate in bits/s. This will determine the fixed size of the encoded
+frames, for a selected frame duration.
+
+@item ar @var{frequency}
+Set the audio sampling rate (in Hz).
+
+@item channels
+Set the number of audio channels.
+
+@item frame_duration
+Set the audio frame duration in milliseconds. Default value is 10ms.
+Allowed frame durations are 2.5ms, 5ms, 7.5ms and 10ms.
+LC3 (Bluetooth LE Audio), allows 7.5ms and 10ms; and LC3plus 2.5ms, 5ms
+and 10ms.
+
+The 10ms frame duration is available in LC3 and LC3 plus standard.
+In this mode, the produced bitstream can be referenced either as LC3 or LC3plus.
+
+@item high_resolution @var{boolean}
+Enable the high-resolution mode if set to 1. The high-resolution mode is
+available with all LC3plus frame durations and for a sampling rate of 48 KHz,
+and 96 KHz.
+
+The encoder automatically turns off this mode at lower sampling rates and
+activates it at 96 KHz.
+
+This mode should be preferred at high bitrates. In this mode, the audio
+bandwidth is always up to the Nyquist frequency, compared to LC3 at 48 KHz,
+which limits the bandwidth to 20 KHz.
+@end table
+
 @anchor{libmp3lame}
 @section libmp3lame
 
diff --git a/doc/general_contents.texi b/doc/general_contents.texi
index f269cbd1a9..e7cf4f8239 100644
--- a/doc/general_contents.texi
+++ b/doc/general_contents.texi
@@ -237,6 +237,14 @@ Go to @url{http://sourceforge.net/projects/opencore-amr/} and follow the
 instructions for installing the library.
 Then pass @code{--enable-libfdk-aac} to configure to enable it.
 
+@subsection LC3 library
+
+FFmpeg can make use of the Google LC3 library for LC3 decoding & encoding.
+
+Go to @url{https://github.com/google/liblc3/} and follow the instructions for
+installing the library.
+Then pass @code{--enable-liblc3} to configure to enable it.
+
 @section OpenH264
 
 FFmpeg can make use of the OpenH264 library for H.264 decoding and encoding.
@@ -1300,7 +1308,8 @@ following image formats are supported:
     @tab encoding and decoding supported through external library libilbc
 @item IMC (Intel Music Coder)  @tab     @tab  X
 @item Interplay ACM            @tab     @tab  X
-@item MACE (Macintosh Audio Compression/Expansion) 3:1  @tab     @tab  X
+@item LC3                    @tab E  @tab  E
+    @tab supported through external library liblc3
 @item MACE (Macintosh Audio Compression/Expansion) 6:1  @tab     @tab  X
 @item Marian's A-pac audio     @tab     @tab  X
 @item MI-SC4 (Micronas SC-4 Audio)  @tab     @tab  X
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index 9ce6d445c1..e70811dbd6 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -1123,6 +1123,8 @@ OBJS-$(CONFIG_LIBILBC_ENCODER)            += libilbc.o
 OBJS-$(CONFIG_LIBJXL_DECODER)             += libjxldec.o libjxl.o
 OBJS-$(CONFIG_LIBJXL_ENCODER)             += libjxlenc.o libjxl.o
 OBJS-$(CONFIG_LIBKVAZAAR_ENCODER)         += libkvazaar.o
+OBJS-$(CONFIG_LIBLC3_ENCODER)             += liblc3enc.o
+OBJS-$(CONFIG_LIBLC3_DECODER)             += liblc3dec.o
 OBJS-$(CONFIG_LIBMP3LAME_ENCODER)         += libmp3lame.o
 OBJS-$(CONFIG_LIBOPENCORE_AMRNB_DECODER)  += libopencore-amr.o
 OBJS-$(CONFIG_LIBOPENCORE_AMRNB_ENCODER)  += libopencore-amr.o
diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c
index 2386b450a6..f4705651fb 100644
--- a/libavcodec/allcodecs.c
+++ b/libavcodec/allcodecs.c
@@ -776,6 +776,8 @@ extern const FFCodec ff_libilbc_encoder;
 extern const FFCodec ff_libilbc_decoder;
 extern const FFCodec ff_libjxl_decoder;
 extern const FFCodec ff_libjxl_encoder;
+extern const FFCodec ff_liblc3_encoder;
+extern const FFCodec ff_liblc3_decoder;
 extern const FFCodec ff_libmp3lame_encoder;
 extern const FFCodec ff_libopencore_amrnb_encoder;
 extern const FFCodec ff_libopencore_amrnb_decoder;
diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c
index 3bab86db62..7dba61dc8b 100644
--- a/libavcodec/codec_desc.c
+++ b/libavcodec/codec_desc.c
@@ -3425,6 +3425,13 @@ static const AVCodecDescriptor codec_descriptors[] = {
         .long_name = NULL_IF_CONFIG_SMALL("QOA (Quite OK Audio)"),
         .props     = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSY,
     },
+    {
+        .id        = AV_CODEC_ID_LC3,
+        .type      = AVMEDIA_TYPE_AUDIO,
+        .name      = "lc3",
+        .long_name = NULL_IF_CONFIG_SMALL("LC3 (Low Complexity Communication Codec)"),
+        .props     = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSY,
+    },
 
     /* subtitle codecs */
     {
diff --git a/libavcodec/codec_id.h b/libavcodec/codec_id.h
index c8dc21da74..0ab1e34a61 100644
--- a/libavcodec/codec_id.h
+++ b/libavcodec/codec_id.h
@@ -543,6 +543,7 @@ enum AVCodecID {
     AV_CODEC_ID_AC4,
     AV_CODEC_ID_OSQ,
     AV_CODEC_ID_QOA,
+    AV_CODEC_ID_LC3,
 
     /* subtitle codecs */
     AV_CODEC_ID_FIRST_SUBTITLE = 0x17000,          ///< A dummy ID pointing at the start of subtitle codecs.
diff --git a/libavcodec/liblc3dec.c b/libavcodec/liblc3dec.c
new file mode 100644
index 0000000000..32d8fade91
--- /dev/null
+++ b/libavcodec/liblc3dec.c
@@ -0,0 +1,141 @@
+/*
+ * LC3 decoder wrapper
+ * Copyright (C) 2024  Antoine Soulier <asoulier@google.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * Permission to use, copy, modify, and/or distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#include <lc3.h>
+
+#include "libavutil/intreadwrite.h"
+
+#include "avcodec.h"
+#include "codec.h"
+#include "codec_internal.h"
+#include "decode.h"
+#include "internal.h"
+
+#define DECODER_MAX_CHANNELS  2
+
+typedef struct LibLC3DecContext {
+    int frame_us, srate_hz;
+    bool hr_mode;
+    void *decoder_mem;
+    lc3_decoder_t decoder[DECODER_MAX_CHANNELS];
+} LibLC3DecContext;
+
+static av_cold int liblc3_decode_init(AVCodecContext *avctx)
+{
+    LibLC3DecContext *liblc3 = avctx->priv_data;
+    int channels = avctx->ch_layout.nb_channels;
+    int ep_mode;
+    unsigned decoder_size;
+
+    if (avctx->extradata_size < 6)
+        return AVERROR_INVALIDDATA;
+
+    liblc3->frame_us = AV_RL16(avctx->extradata + 0) * 10;
+    liblc3->srate_hz = avctx->sample_rate;
+    ep_mode          = AV_RL16(avctx->extradata + 2);
+    liblc3->hr_mode  = AV_RL16(avctx->extradata + 4);
+    if (ep_mode != 0) {
+        av_log(avctx, AV_LOG_ERROR,
+               "Error protection mode is not supported.\n");
+        return AVERROR(EINVAL);
+    }
+
+    av_log(avctx, AV_LOG_INFO,
+           "Decoding %.1f ms frames.\n", liblc3->frame_us / 1000.f);
+    if (liblc3->hr_mode)
+        av_log(avctx, AV_LOG_INFO, "High-resolution mode enabled.\n");
+
+    decoder_size = lc3_hr_decoder_size(
+        liblc3->hr_mode, liblc3->frame_us, liblc3->srate_hz);
+    if (!decoder_size)
+        return AVERROR_INVALIDDATA;
+
+    liblc3->decoder_mem = av_malloc_array(channels, decoder_size);
+    if (!liblc3->decoder_mem)
+        return AVERROR(ENOMEM);
+
+    for (int ch = 0; ch < channels; ch++) {
+        liblc3->decoder[ch] = lc3_hr_setup_decoder(
+            liblc3->hr_mode, liblc3->frame_us, liblc3->srate_hz, 0,
+            (char *)liblc3->decoder_mem + ch * decoder_size);
+    }
+
+    avctx->sample_fmt = AV_SAMPLE_FMT_FLTP;
+    avctx->delay = lc3_hr_delay_samples(
+        liblc3->hr_mode, liblc3->frame_us, liblc3->srate_hz);
+    avctx->internal->skip_samples = avctx->delay;
+
+    return 0;
+}
+
+static av_cold int liblc3_decode_close(AVCodecContext *avctx)
+{
+    LibLC3DecContext *liblc3 = avctx->priv_data;
+
+    av_freep(&liblc3->decoder_mem);
+
+    return 0;
+}
+
+static int liblc3_decode(AVCodecContext *avctx, AVFrame *frame,
+                         int *got_frame_ptr, AVPacket *avpkt)
+{
+    LibLC3DecContext *liblc3 = avctx->priv_data;
+    int channels = avctx->ch_layout.nb_channels;
+    uint8_t *in = avpkt->data;
+    int block_bytes, ret;
+
+    frame->nb_samples = av_rescale(
+        liblc3->frame_us, liblc3->srate_hz, 1000*1000);
+    if ((ret = ff_get_buffer(avctx, frame, 0)) < 0)
+        return ret;
+
+    block_bytes = avpkt->size;
+    for (int ch = 0; ch < channels; ch++) {
+        int frame_bytes = block_bytes / channels
+            + (ch < block_bytes % channels);
+
+        ret = lc3_decode(liblc3->decoder[ch], in, frame_bytes,
+                         LC3_PCM_FORMAT_FLOAT, frame->data[ch], 1);
+        if (ret < 0)
+            return AVERROR_INVALIDDATA;
+
+        in += frame_bytes;
+    }
+
+    frame->nb_samples = FFMIN(frame->nb_samples, avpkt->duration);
+
+    *got_frame_ptr = 1;
+
+    return avpkt->size;
+}
+
+const FFCodec ff_liblc3_decoder = {
+    .p.name         = "liblc3",
+    CODEC_LONG_NAME("LC3 (Low Complexity Communication Codec)"),
+    .p.type         = AVMEDIA_TYPE_AUDIO,
+    .p.id           = AV_CODEC_ID_LC3,
+    .p.capabilities = AV_CODEC_CAP_DR1,
+    .p.wrapper_name = "liblc3",
+    .priv_data_size = sizeof(LibLC3DecContext),
+    .caps_internal  = FF_CODEC_CAP_INIT_CLEANUP,
+    .init           = liblc3_decode_init,
+    .close          = liblc3_decode_close,
+    FF_CODEC_DECODE_CB(liblc3_decode),
+};
diff --git a/libavcodec/liblc3enc.c b/libavcodec/liblc3enc.c
new file mode 100644
index 0000000000..7d5d92303c
--- /dev/null
+++ b/libavcodec/liblc3enc.c
@@ -0,0 +1,190 @@
+/*
+ * LC3 encoder wrapper
+ * Copyright (C) 2024  Antoine Soulier <asoulier@google.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * Permission to use, copy, modify, and/or distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#include <lc3.h>
+
+#include "libavutil/intreadwrite.h"
+#include "libavutil/opt.h"
+
+#include "avcodec.h"
+#include "codec.h"
+#include "codec_internal.h"
+#include "encode.h"
+
+#define ENCODER_MAX_CHANNELS  2
+
+typedef struct LibLC3EncOpts {
+    float frame_duration;
+    int hr_mode;
+} LibLC3EncOpts;
+
+typedef struct LibLC3EncContext {
+    const AVClass *av_class;
+    LibLC3EncOpts opts;
+    int block_bytes;
+    void *encoder_mem;
+    lc3_encoder_t encoder[ENCODER_MAX_CHANNELS];
+} LibLC3EncContext;
+
+static av_cold int liblc3_encode_init(AVCodecContext *avctx)
+{
+    LibLC3EncContext *liblc3 = avctx->priv_data;
+    bool hr_mode = liblc3->opts.hr_mode;
+    int frame_us = liblc3->opts.frame_duration * 1000;
+    int srate_hz = avctx->sample_rate;
+    int channels = avctx->ch_layout.nb_channels;
+    int effective_bit_rate;
+    unsigned encoder_size;
+
+    if (frame_us != 2500 && frame_us !=  5000 &&
+        frame_us != 7500 && frame_us != 10000   ) {
+        av_log(avctx, AV_LOG_ERROR,
+               "Unsupported frame duration %.1f ms.\n", frame_us / 1000.f);
+        return AVERROR(EINVAL);
+    }
+
+    hr_mode |= srate_hz > 48000;
+    hr_mode &= srate_hz >= 48000;
+
+    if (frame_us == 7500 && hr_mode) {
+        av_log(avctx, AV_LOG_ERROR,
+               "High-resolution mode is not supported with 7.5 ms frames.\n");
+        return AVERROR(EINVAL);
+    }
+
+    av_log(avctx, AV_LOG_INFO, "Encoding %.1f ms frames.\n", frame_us / 1000.f);
+    if (hr_mode)
+        av_log(avctx, AV_LOG_INFO, "High-resolution mode is enabled.\n");
+
+    liblc3->block_bytes = lc3_hr_frame_block_bytes(
+        hr_mode, frame_us, srate_hz, channels, avctx->bit_rate);
+
+    effective_bit_rate = lc3_hr_resolve_bitrate(
+        hr_mode, frame_us, srate_hz, liblc3->block_bytes);
+
+    if (avctx->bit_rate != effective_bit_rate)
+        av_log(avctx, AV_LOG_WARNING,
+               "Bitrate changed to %d bps.\n", effective_bit_rate);
+    avctx->bit_rate = effective_bit_rate;
+
+    encoder_size = lc3_hr_encoder_size(hr_mode, frame_us, srate_hz);
+    if (!encoder_size)
+        return AVERROR(EINVAL);
+
+    liblc3->encoder_mem = av_malloc_array(channels, encoder_size);
+    if (!liblc3->encoder_mem)
+        return AVERROR(ENOMEM);
+
+    for (int ch = 0; ch < channels; ch++) {
+        liblc3->encoder[ch] = lc3_hr_setup_encoder(
+            hr_mode, frame_us, srate_hz, 0,
+            (char *)liblc3->encoder_mem + ch * encoder_size);
+    }
+
+    avctx->extradata = av_mallocz(6 + AV_INPUT_BUFFER_PADDING_SIZE);
+    if (!avctx->extradata)
+        return AVERROR(ENOMEM);
+
+    AV_WL16(avctx->extradata + 0, frame_us / 10);
+    AV_WL16(avctx->extradata + 2, 0);
+    AV_WL16(avctx->extradata + 4, hr_mode);
+    avctx->extradata_size = 6;
+
+    avctx->frame_size = av_rescale(frame_us, srate_hz, 1000*1000);
+
+    return 0;
+}
+
+static av_cold int liblc3_encode_close(AVCodecContext *avctx)
+{
+    LibLC3EncContext *liblc3 = avctx->priv_data;
+
+    av_freep(&liblc3->encoder_mem);
+
+    return 0;
+}
+
+static int liblc3_encode(AVCodecContext *avctx, AVPacket *avpkt,
+                         const AVFrame *av_frame, int *got_packet_ptr)
+{
+    LibLC3EncContext *liblc3 = avctx->priv_data;
+    int block_bytes = liblc3->block_bytes;
+    int channels = avctx->ch_layout.nb_channels;
+    uint8_t *data_ptr;
+    int ret;
+
+    if ((ret = ff_get_encode_buffer(avctx, avpkt, block_bytes, 0)) < 0)
+        return ret;
+
+    data_ptr = avpkt->data;
+    for (int ch = 0; ch < channels; ch++) {
+        int frame_bytes = block_bytes / channels
+                + (ch < block_bytes % channels);
+
+        lc3_encode(liblc3->encoder[ch],
+                   LC3_PCM_FORMAT_FLOAT, av_frame->data[ch], 1,
+                   frame_bytes, data_ptr);
+
+        data_ptr += frame_bytes;
+    }
+
+    *got_packet_ptr = 1;
+
+    return 0;
+}
+
+#define OFFSET(x) offsetof(LibLC3EncContext, opts.x)
+#define FLAGS AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_ENCODING_PARAM
+static const AVOption options[] = {
+    { "frame_duration", "Duration of a frame in milliseconds",
+        OFFSET(frame_duration), AV_OPT_TYPE_FLOAT,
+        { .dbl = 10.0 }, 2.5, 10.0, FLAGS },
+    { "high_resolution", "Enable High-Resolution mode (48 KHz or 96 KHz)",
+        OFFSET(hr_mode), AV_OPT_TYPE_BOOL,
+        { .i64 = 0 }, 0, 1, FLAGS },
+    { NULL }
+};
+
+static const AVClass class = {
+    .class_name = "liblc3 encoder",
+    .item_name  = av_default_item_name,
+    .option     = options,
+    .version    = LIBAVUTIL_VERSION_INT,
+};
+
+const FFCodec ff_liblc3_encoder = {
+    .p.name         = "liblc3",
+    CODEC_LONG_NAME("LC3 (Low Complexity Communication Codec)"),
+    .p.type         = AVMEDIA_TYPE_AUDIO,
+    .p.id           = AV_CODEC_ID_LC3,
+    .p.capabilities = AV_CODEC_CAP_DR1,
+    .p.ch_layouts = (const AVChannelLayout[])
+        { { AV_CHANNEL_ORDER_UNSPEC, 1 },
+          { AV_CHANNEL_ORDER_UNSPEC, 2 }, { 0 } },
+    .p.supported_samplerates = (const int [])
+        { 96000, 48000, 32000, 24000, 16000, 8000, 0 },
+    .p.sample_fmts = (const enum AVSampleFormat[])
+        { AV_SAMPLE_FMT_FLTP, AV_SAMPLE_FMT_NONE },
+    .p.priv_class   = &class,
+    .p.wrapper_name = "liblc3",
+    .priv_data_size = sizeof(LibLC3EncContext),
+    .init           = liblc3_encode_init,
+    .close          = liblc3_encode_close,
+    FF_CODEC_ENCODE_CB(liblc3_encode),
+};
-- 
2.44.0.478.gd926399ef9-goog

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 17+ messages in thread
* [FFmpeg-devel] [PATCH 1/2] avcodec/liblc3: Add encoding/decoding support of LC3 audio codec
@ 2024-04-01 21:32 Antoine Soulier via ffmpeg-devel
  2024-04-01 21:32 ` [FFmpeg-devel] [PATCH 2/2] avformat/lc3: Add file format for LC3/LC3plus transport Antoine Soulier via ffmpeg-devel
  0 siblings, 1 reply; 17+ messages in thread
From: Antoine Soulier via ffmpeg-devel @ 2024-04-01 21:32 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: Antoine Soulier

The LC3 audio codec is the default codec of Bluetooth LE audio.
This is a wrapper over the liblc3 library (https://github.com/google/liblc3).

Signed-off-by: Antoine Soulier <asoulier@google.com>
---
 Changelog                 |   4 +
 configure                 |   6 ++
 doc/encoders.texi         |  57 +++++++++++
 doc/general_contents.texi |  11 +-
 libavcodec/Makefile       |   2 +
 libavcodec/allcodecs.c    |   2 +
 libavcodec/codec_desc.c   |   7 ++
 libavcodec/codec_id.h     |   1 +
 libavcodec/liblc3dec.c    | 145 ++++++++++++++++++++++++++
 libavcodec/liblc3enc.c    | 210 ++++++++++++++++++++++++++++++++++++++
 10 files changed, 444 insertions(+), 1 deletion(-)
 create mode 100644 libavcodec/liblc3dec.c
 create mode 100644 libavcodec/liblc3enc.c

diff --git a/Changelog b/Changelog
index e83a00e35c..83a4cf7888 100644
--- a/Changelog
+++ b/Changelog
@@ -1,6 +1,10 @@
 Entries are sorted chronologically from oldest to youngest within each release,
 releases are sorted from youngest to oldest.
 
+version <next>:
+- LC3/LC3plus decoding/encoding using external library liblc3
+
+
 version 7.0:
 - DXV DXT1 encoder
 - LEAD MCMP decoder
diff --git a/configure b/configure
index 2d46ef0b9c..e5d9ba9f53 100755
--- a/configure
+++ b/configure
@@ -244,6 +244,7 @@ External library support:
   --enable-libjxl          enable JPEG XL de/encoding via libjxl [no]
   --enable-libklvanc       enable Kernel Labs VANC processing [no]
   --enable-libkvazaar      enable HEVC encoding via libkvazaar [no]
+  --enable-liblc3          enable LC3 de/encoding via liblc3 [no]
   --enable-liblensfun      enable lensfun lens correction [no]
   --enable-libmodplug      enable ModPlug via libmodplug [no]
   --enable-libmp3lame      enable MP3 encoding via libmp3lame [no]
@@ -1926,6 +1927,7 @@ EXTERNAL_LIBRARY_LIST="
     libjxl
     libklvanc
     libkvazaar
+    liblc3
     libmodplug
     libmp3lame
     libmysofa
@@ -3499,6 +3501,9 @@ libilbc_encoder_deps="libilbc"
 libjxl_decoder_deps="libjxl libjxl_threads"
 libjxl_encoder_deps="libjxl libjxl_threads"
 libkvazaar_encoder_deps="libkvazaar"
+liblc3_decoder_deps="liblc3"
+liblc3_encoder_deps="liblc3"
+liblc3_encoder_select="audio_frame_queue"
 libmodplug_demuxer_deps="libmodplug"
 libmp3lame_encoder_deps="libmp3lame"
 libmp3lame_encoder_select="audio_frame_queue mpegaudioheader"
@@ -6869,6 +6874,7 @@ enabled libjxl            && require_pkg_config libjxl "libjxl >= 0.7.0" jxl/dec
                              require_pkg_config libjxl_threads "libjxl_threads >= 0.7.0" jxl/thread_parallel_runner.h JxlThreadParallelRunner
 enabled libklvanc         && require libklvanc libklvanc/vanc.h klvanc_context_create -lklvanc
 enabled libkvazaar        && require_pkg_config libkvazaar "kvazaar >= 2.0.0" kvazaar.h kvz_api_get
+enabled liblc3            && require_pkg_config liblc3 "lc3 >= 1.1.0" lc3.h lc3_hr_setup_encoder
 enabled liblensfun        && require_pkg_config liblensfun lensfun lensfun.h lf_db_create
 
 if enabled libmfx && enabled libvpl; then
diff --git a/doc/encoders.texi b/doc/encoders.texi
index 7c223ed74c..66847191e1 100644
--- a/doc/encoders.texi
+++ b/doc/encoders.texi
@@ -814,6 +814,63 @@ ffmpeg -i input.wav -c:a libfdk_aac -profile:a aac_he -b:a 64k output.m4a
 @end example
 @end itemize
 
+@anchor{liblc3-enc}
+@section liblc3
+
+liblc3 LC3 (Low Complexity Communication Codec) encoder wrapper.
+
+Requires the presence of the liblc3 headers and library during configuration.
+You need to explicitly configure the build with @code{--enable-liblc3}.
+
+This encoder has support for the Bluetooth SIG LC3 codec for the LE Audio
+protocol, and the following features of LC3plus:
+@itemize
+@item
+Frame duration of 2.5 and 5ms.
+@item
+High-Resolution mode, 48 KHz, and 96 kHz sampling rates.
+@end itemize
+
+For more information see the liblc3 project at
+@url{https://github.com/google/liblc3}.
+
+@subsection Options
+
+The following options are mapped on the shared FFmpeg codec options.
+
+@table @option
+@item b @var{bitrate}
+Set the bit rate in bits/s. This will determine the fixed size of the encoded
+frames, for a selected frame duration.
+
+@item ar @var{frequency}
+Set the audio sampling rate (in Hz).
+
+@item channels
+Set the number of audio channels.
+
+@item frame_duration
+Set the audio frame duration in milliseconds. Default value is 10ms.
+Allowed frame durations are 2.5ms, 5ms, 7.5ms and 10ms.
+LC3 (Bluetooth LE Audio), allows 7.5ms and 10ms; and LC3plus 2.5ms, 5ms
+and 10ms.
+
+The 10ms frame duration is available in LC3 and LC3 plus standard.
+In this mode, the produced bitstream can be referenced either as LC3 or LC3plus.
+
+@item high_resolution @var{boolean}
+Enable the high-resolution mode if set to 1. The high-resolution mode is
+available with all LC3plus frame durations and for a sampling rate of 48 KHz,
+and 96 KHz.
+
+The encoder automatically turns off this mode at lower sampling rates and
+activates it at 96 KHz.
+
+This mode should be preferred at high bitrates. In this mode, the audio
+bandwidth is always up to the Nyquist frequency, compared to LC3 at 48 KHz,
+which limits the bandwidth to 20 KHz.
+@end table
+
 @anchor{libmp3lame}
 @section libmp3lame
 
diff --git a/doc/general_contents.texi b/doc/general_contents.texi
index f269cbd1a9..e7cf4f8239 100644
--- a/doc/general_contents.texi
+++ b/doc/general_contents.texi
@@ -237,6 +237,14 @@ Go to @url{http://sourceforge.net/projects/opencore-amr/} and follow the
 instructions for installing the library.
 Then pass @code{--enable-libfdk-aac} to configure to enable it.
 
+@subsection LC3 library
+
+FFmpeg can make use of the Google LC3 library for LC3 decoding & encoding.
+
+Go to @url{https://github.com/google/liblc3/} and follow the instructions for
+installing the library.
+Then pass @code{--enable-liblc3} to configure to enable it.
+
 @section OpenH264
 
 FFmpeg can make use of the OpenH264 library for H.264 decoding and encoding.
@@ -1300,7 +1308,8 @@ following image formats are supported:
     @tab encoding and decoding supported through external library libilbc
 @item IMC (Intel Music Coder)  @tab     @tab  X
 @item Interplay ACM            @tab     @tab  X
-@item MACE (Macintosh Audio Compression/Expansion) 3:1  @tab     @tab  X
+@item LC3                    @tab E  @tab  E
+    @tab supported through external library liblc3
 @item MACE (Macintosh Audio Compression/Expansion) 6:1  @tab     @tab  X
 @item Marian's A-pac audio     @tab     @tab  X
 @item MI-SC4 (Micronas SC-4 Audio)  @tab     @tab  X
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index eef936944d..6323c98add 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -1122,6 +1122,8 @@ OBJS-$(CONFIG_LIBILBC_ENCODER)            += libilbc.o
 OBJS-$(CONFIG_LIBJXL_DECODER)             += libjxldec.o libjxl.o
 OBJS-$(CONFIG_LIBJXL_ENCODER)             += libjxlenc.o libjxl.o
 OBJS-$(CONFIG_LIBKVAZAAR_ENCODER)         += libkvazaar.o
+OBJS-$(CONFIG_LIBLC3_ENCODER)             += liblc3enc.o
+OBJS-$(CONFIG_LIBLC3_DECODER)             += liblc3dec.o
 OBJS-$(CONFIG_LIBMP3LAME_ENCODER)         += libmp3lame.o
 OBJS-$(CONFIG_LIBOPENCORE_AMRNB_DECODER)  += libopencore-amr.o
 OBJS-$(CONFIG_LIBOPENCORE_AMRNB_ENCODER)  += libopencore-amr.o
diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c
index 2386b450a6..f4705651fb 100644
--- a/libavcodec/allcodecs.c
+++ b/libavcodec/allcodecs.c
@@ -776,6 +776,8 @@ extern const FFCodec ff_libilbc_encoder;
 extern const FFCodec ff_libilbc_decoder;
 extern const FFCodec ff_libjxl_decoder;
 extern const FFCodec ff_libjxl_encoder;
+extern const FFCodec ff_liblc3_encoder;
+extern const FFCodec ff_liblc3_decoder;
 extern const FFCodec ff_libmp3lame_encoder;
 extern const FFCodec ff_libopencore_amrnb_encoder;
 extern const FFCodec ff_libopencore_amrnb_decoder;
diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c
index 3bab86db62..7dba61dc8b 100644
--- a/libavcodec/codec_desc.c
+++ b/libavcodec/codec_desc.c
@@ -3425,6 +3425,13 @@ static const AVCodecDescriptor codec_descriptors[] = {
         .long_name = NULL_IF_CONFIG_SMALL("QOA (Quite OK Audio)"),
         .props     = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSY,
     },
+    {
+        .id        = AV_CODEC_ID_LC3,
+        .type      = AVMEDIA_TYPE_AUDIO,
+        .name      = "lc3",
+        .long_name = NULL_IF_CONFIG_SMALL("LC3 (Low Complexity Communication Codec)"),
+        .props     = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSY,
+    },
 
     /* subtitle codecs */
     {
diff --git a/libavcodec/codec_id.h b/libavcodec/codec_id.h
index c8dc21da74..0ab1e34a61 100644
--- a/libavcodec/codec_id.h
+++ b/libavcodec/codec_id.h
@@ -543,6 +543,7 @@ enum AVCodecID {
     AV_CODEC_ID_AC4,
     AV_CODEC_ID_OSQ,
     AV_CODEC_ID_QOA,
+    AV_CODEC_ID_LC3,
 
     /* subtitle codecs */
     AV_CODEC_ID_FIRST_SUBTITLE = 0x17000,          ///< A dummy ID pointing at the start of subtitle codecs.
diff --git a/libavcodec/liblc3dec.c b/libavcodec/liblc3dec.c
new file mode 100644
index 0000000000..75ab90f377
--- /dev/null
+++ b/libavcodec/liblc3dec.c
@@ -0,0 +1,145 @@
+/*
+ * LC3 decoder wrapper
+ * Copyright (C) 2024  Antoine Soulier <asoulier@google.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * Permission to use, copy, modify, and/or distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#include <lc3.h>
+
+#include "libavutil/intreadwrite.h"
+
+#include "avcodec.h"
+#include "codec.h"
+#include "codec_internal.h"
+#include "decode.h"
+#include "internal.h"
+
+#define DECODER_MAX_CHANNELS  2
+
+typedef struct LibLC3DecContext {
+    int frame_us, srate_hz, hr_mode;
+    void *decoder_mem;
+    lc3_decoder_t decoder[DECODER_MAX_CHANNELS];
+    int64_t length;
+} LibLC3DecContext;
+
+static av_cold int liblc3_decode_init(AVCodecContext *avctx)
+{
+    LibLC3DecContext *liblc3 = avctx->priv_data;
+    int channels = avctx->ch_layout.nb_channels;
+    int ep_mode;
+    unsigned decoder_size;
+
+    if (avctx->extradata_size < 10)
+        return AVERROR_INVALIDDATA;
+
+    liblc3->frame_us = AV_RL16(avctx->extradata + 0) * 10;
+    liblc3->srate_hz = avctx->sample_rate;
+    ep_mode          = AV_RL16(avctx->extradata + 2);
+    liblc3->hr_mode  = AV_RL16(avctx->extradata + 4);
+    liblc3->length   = AV_RL32(avctx->extradata + 6);
+    if (ep_mode != 0) {
+        av_log(avctx, AV_LOG_ERROR,
+               "Error protection mode is not supported.\n");
+        return AVERROR(EINVAL);
+    }
+
+    av_log(avctx, AV_LOG_INFO,
+           "Decoding %.1f ms frames.\n", liblc3->frame_us / 1000.f);
+    if (liblc3->hr_mode)
+        av_log(avctx, AV_LOG_INFO, "High-resolution mode enabled.\n");
+
+    decoder_size = lc3_hr_decoder_size(
+        liblc3->hr_mode, liblc3->frame_us, liblc3->srate_hz);
+    if (!decoder_size)
+        return AVERROR_INVALIDDATA;
+
+    liblc3->decoder_mem = av_malloc_array(channels, decoder_size);
+    if (!liblc3->decoder_mem)
+        return AVERROR(ENOMEM);
+
+    for (int ch = 0; ch < channels; ch++) {
+        liblc3->decoder[ch] = lc3_hr_setup_decoder(
+            liblc3->hr_mode, liblc3->frame_us, liblc3->srate_hz, 0,
+            (char *)liblc3->decoder_mem + ch * decoder_size);
+    }
+
+    avctx->sample_fmt = AV_SAMPLE_FMT_FLTP;
+    avctx->delay = lc3_hr_delay_samples(
+        liblc3->hr_mode, liblc3->frame_us, liblc3->srate_hz);
+    avctx->internal->skip_samples = avctx->delay;
+
+    return 0;
+}
+
+static av_cold int liblc3_decode_close(AVCodecContext *avctx)
+{
+    LibLC3DecContext *liblc3 = avctx->priv_data;
+
+    av_freep(&liblc3->decoder_mem);
+
+    return 0;
+}
+
+static int liblc3_decode(AVCodecContext *avctx, AVFrame *frame,
+                         int *got_frame_ptr, AVPacket *avpkt)
+{
+    LibLC3DecContext *liblc3 = avctx->priv_data;
+    int channels = avctx->ch_layout.nb_channels;
+    uint8_t *in = avpkt->data;
+    int block_bytes, ret;
+
+    frame->nb_samples = av_rescale(
+        liblc3->frame_us, liblc3->srate_hz, 1000*1000);
+    if ((ret = ff_get_buffer(avctx, frame, 0)) < 0)
+        return ret;
+
+    block_bytes = avpkt->size;
+    for (int ch = 0; ch < channels; ch++) {
+        int nbytes = block_bytes / channels + (ch < block_bytes % channels);
+
+        ret = lc3_decode(liblc3->decoder[ch], in, nbytes,
+                         LC3_PCM_FORMAT_FLOAT, frame->data[ch], 1);
+        if (ret < 0)
+            return AVERROR_INVALIDDATA;
+
+        in += nbytes;
+    }
+
+    if (liblc3->length > 0) {
+        int64_t end_pts = liblc3->length + avctx->delay;
+        frame->nb_samples = FFMIN(frame->nb_samples,
+                                  FFMAX(end_pts - frame->pts, 0));
+    }
+
+    *got_frame_ptr = 1;
+
+    return avpkt->size;
+}
+
+const FFCodec ff_liblc3_decoder = {
+    .p.name         = "liblc3",
+    CODEC_LONG_NAME("LC3 (Low Complexity Communication Codec)"),
+    .p.type         = AVMEDIA_TYPE_AUDIO,
+    .p.id           = AV_CODEC_ID_LC3,
+    .p.capabilities = AV_CODEC_CAP_DR1,
+    .p.wrapper_name = "liblc3",
+    .priv_data_size = sizeof(LibLC3DecContext),
+    .caps_internal  = FF_CODEC_CAP_INIT_CLEANUP,
+    .init           = liblc3_decode_init,
+    .close          = liblc3_decode_close,
+    FF_CODEC_DECODE_CB(liblc3_decode),
+};
diff --git a/libavcodec/liblc3enc.c b/libavcodec/liblc3enc.c
new file mode 100644
index 0000000000..9731229e50
--- /dev/null
+++ b/libavcodec/liblc3enc.c
@@ -0,0 +1,210 @@
+/*
+ * LC3 encoder wrapper
+ * Copyright (C) 2024  Antoine Soulier <asoulier@google.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * Permission to use, copy, modify, and/or distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#include <lc3.h>
+
+#include "libavutil/intreadwrite.h"
+#include "libavutil/opt.h"
+
+#include "avcodec.h"
+#include "codec.h"
+#include "codec_internal.h"
+#include "encode.h"
+
+#define ENCODER_MAX_CHANNELS  2
+
+typedef struct LibLC3EncOpts {
+    float frame_duration;
+    int hr_mode;
+} LibLC3EncOpts;
+
+typedef struct LibLC3EncContext {
+    const AVClass *av_class;
+    LibLC3EncOpts opts;
+    int block_bytes;
+    void *encoder_mem;
+    lc3_encoder_t encoder[ENCODER_MAX_CHANNELS];
+    int delay_samples;
+    int remaining_samples;
+} LibLC3EncContext;
+
+static av_cold int liblc3_encode_init(AVCodecContext *avctx)
+{
+    LibLC3EncContext *liblc3 = avctx->priv_data;
+    bool hr_mode = liblc3->opts.hr_mode;
+    int frame_us = liblc3->opts.frame_duration * 1000;
+    int srate_hz = avctx->sample_rate;
+    int channels = avctx->ch_layout.nb_channels;
+    int effective_bit_rate;
+    unsigned encoder_size;
+
+    if (frame_us != 2500 && frame_us !=  5000 &&
+        frame_us != 7500 && frame_us != 10000   ) {
+        av_log(avctx, AV_LOG_ERROR,
+               "Unsupported frame duration %.1f ms.\n", frame_us / 1000.f);
+        return AVERROR(EINVAL);
+    }
+
+    hr_mode |= srate_hz > 48000;
+    hr_mode &= srate_hz >= 48000;
+
+    if (frame_us == 7500 && hr_mode) {
+        av_log(avctx, AV_LOG_ERROR,
+               "High-resolution mode is not supported with 7.5 ms frames.\n");
+        return AVERROR(EINVAL);
+    }
+
+    av_log(avctx, AV_LOG_INFO, "Encoding %.1f ms frames.\n", frame_us / 1000.f);
+    if (hr_mode)
+        av_log(avctx, AV_LOG_INFO, "High-resolution mode is enabled.\n");
+
+    liblc3->block_bytes = lc3_hr_frame_block_bytes(
+        hr_mode, frame_us, srate_hz, channels, avctx->bit_rate);
+
+    effective_bit_rate = lc3_hr_resolve_bitrate(
+        hr_mode, frame_us, srate_hz, liblc3->block_bytes);
+
+    if (avctx->bit_rate != effective_bit_rate)
+        av_log(avctx, AV_LOG_WARNING,
+               "Bitrate changed to %d bps.\n", effective_bit_rate);
+    avctx->bit_rate = effective_bit_rate;
+
+    encoder_size = lc3_hr_encoder_size(hr_mode, frame_us, srate_hz);
+    if (!encoder_size)
+        return AVERROR(EINVAL);
+
+    liblc3->encoder_mem = av_malloc_array(channels, encoder_size);
+    if (!liblc3->encoder_mem)
+        return AVERROR(ENOMEM);
+
+    for (int ch = 0; ch < channels; ch++) {
+        liblc3->encoder[ch] = lc3_hr_setup_encoder(
+            hr_mode, frame_us, srate_hz, 0,
+            (char *)liblc3->encoder_mem + ch * encoder_size);
+    }
+
+    avctx->extradata = av_mallocz(6 + AV_INPUT_BUFFER_PADDING_SIZE);
+    if (!avctx->extradata)
+        return AVERROR(ENOMEM);
+
+    AV_WL16(avctx->extradata + 0, frame_us / 10);
+    AV_WL16(avctx->extradata + 2, 0);
+    AV_WL16(avctx->extradata + 4, hr_mode);
+    avctx->extradata_size = 6;
+
+    avctx->frame_size = av_rescale(frame_us, srate_hz, 1000*1000);
+    liblc3->delay_samples = lc3_hr_delay_samples(hr_mode, frame_us, srate_hz);
+    liblc3->remaining_samples = 0;
+
+    return 0;
+}
+
+static av_cold int liblc3_encode_close(AVCodecContext *avctx)
+{
+    LibLC3EncContext *liblc3 = avctx->priv_data;
+
+    av_freep(&liblc3->encoder_mem);
+
+    return 0;
+}
+
+static int liblc3_encode(AVCodecContext *avctx, AVPacket *pkt,
+                         const AVFrame *frame, int *got_packet_ptr)
+{
+    LibLC3EncContext *liblc3 = avctx->priv_data;
+    int block_bytes = liblc3->block_bytes;
+    int channels = avctx->ch_layout.nb_channels;
+    void *zero_frame = NULL;
+    uint8_t *data_ptr;
+    int ret;
+
+    if ((ret = ff_get_encode_buffer(avctx, pkt, block_bytes, 0)) < 0)
+        return ret;
+
+    if (frame) {
+        int padding = frame->nb_samples - frame->duration;
+        liblc3->remaining_samples = FFMAX(liblc3->delay_samples - padding, 0);
+    } else {
+        if (!liblc3->remaining_samples)
+            return 0;
+
+        liblc3->remaining_samples = 0;
+        zero_frame = av_mallocz(avctx->frame_size * sizeof(float));
+        if (!zero_frame)
+            return AVERROR(ENOMEM);
+    }
+
+    data_ptr = pkt->data;
+    for (int ch = 0; ch < channels; ch++) {
+        const float *pcm = zero_frame ? zero_frame : frame->data[ch];
+        int nbytes = block_bytes / channels + (ch < block_bytes % channels);
+
+        lc3_encode(liblc3->encoder[ch],
+                   LC3_PCM_FORMAT_FLOAT, pcm, 1, nbytes, data_ptr);
+
+        data_ptr += nbytes;
+    }
+
+    if (zero_frame)
+        av_free(zero_frame);
+
+    *got_packet_ptr = 1;
+
+    return 0;
+}
+
+#define OFFSET(x) offsetof(LibLC3EncContext, opts.x)
+#define FLAGS AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_ENCODING_PARAM
+static const AVOption options[] = {
+    { "frame_duration", "Duration of a frame in milliseconds",
+        OFFSET(frame_duration), AV_OPT_TYPE_FLOAT,
+        { .dbl = 10.0 }, 2.5, 10.0, FLAGS },
+    { "high_resolution", "Enable High-Resolution mode (48 KHz or 96 KHz)",
+        OFFSET(hr_mode), AV_OPT_TYPE_BOOL,
+        { .i64 = 0 }, 0, 1, FLAGS },
+    { NULL }
+};
+
+static const AVClass class = {
+    .class_name = "liblc3 encoder",
+    .item_name  = av_default_item_name,
+    .option     = options,
+    .version    = LIBAVUTIL_VERSION_INT,
+};
+
+const FFCodec ff_liblc3_encoder = {
+    .p.name         = "liblc3",
+    CODEC_LONG_NAME("LC3 (Low Complexity Communication Codec)"),
+    .p.type         = AVMEDIA_TYPE_AUDIO,
+    .p.id           = AV_CODEC_ID_LC3,
+    .p.capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_DELAY,
+    .p.ch_layouts = (const AVChannelLayout[])
+        { { AV_CHANNEL_ORDER_UNSPEC, 1 },
+          { AV_CHANNEL_ORDER_UNSPEC, 2 }, { 0 } },
+    .p.supported_samplerates = (const int [])
+        { 96000, 48000, 32000, 24000, 16000, 8000, 0 },
+    .p.sample_fmts = (const enum AVSampleFormat[])
+        { AV_SAMPLE_FMT_FLTP, AV_SAMPLE_FMT_NONE },
+    .p.priv_class   = &class,
+    .p.wrapper_name = "liblc3",
+    .priv_data_size = sizeof(LibLC3EncContext),
+    .init           = liblc3_encode_init,
+    .close          = liblc3_encode_close,
+    FF_CODEC_ENCODE_CB(liblc3_encode),
+};
-- 
2.44.0.478.gd926399ef9-goog

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 17+ messages in thread
* [FFmpeg-devel] [PATCH 1/2] avcodec/liblc3dec: Retrieve duration of the last packet from the demux
@ 2024-04-04 22:57 Antoine Soulier via ffmpeg-devel
  2024-04-04 22:57 ` [FFmpeg-devel] [PATCH 2/2] avformat/lc3: Add file format for LC3/LC3plus transport Antoine Soulier via ffmpeg-devel
  0 siblings, 1 reply; 17+ messages in thread
From: Antoine Soulier via ffmpeg-devel @ 2024-04-04 22:57 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: Antoine Soulier

From: Antoine SOULIER <asoulier@google.com>

Use the packet duration field to invalid last samples of the last frame.
---
 libavcodec/liblc3dec.c | 12 +++---------
 1 file changed, 3 insertions(+), 9 deletions(-)

diff --git a/libavcodec/liblc3dec.c b/libavcodec/liblc3dec.c
index 90da28679b..d250ace38a 100644
--- a/libavcodec/liblc3dec.c
+++ b/libavcodec/liblc3dec.c
@@ -34,7 +34,6 @@ typedef struct LibLC3DecContext {
     int frame_us, srate_hz, hr_mode;
     void *decoder_mem;
     lc3_decoder_t decoder[DECODER_MAX_CHANNELS];
-    int64_t length;
 } LibLC3DecContext;
 
 static av_cold int liblc3_decode_init(AVCodecContext *avctx)
@@ -44,12 +43,12 @@ static av_cold int liblc3_decode_init(AVCodecContext *avctx)
     int ep_mode;
     unsigned decoder_size;
 
-    if (avctx->extradata_size < 10)
+    if (avctx->extradata_size < 6)
         return AVERROR_INVALIDDATA;
     if (channels < 0 || channels > DECODER_MAX_CHANNELS) {
         av_log(avctx, AV_LOG_ERROR,
                "Invalid number of channels %d. Max %d channels are accepted\n",
-               channels, DECODER_MAX_CHANNES);
+               channels, DECODER_MAX_CHANNELS);
         return AVERROR(EINVAL);
     }
 
@@ -57,7 +56,6 @@ static av_cold int liblc3_decode_init(AVCodecContext *avctx)
     liblc3->srate_hz = avctx->sample_rate;
     ep_mode          = AV_RL16(avctx->extradata + 2);
     liblc3->hr_mode  = AV_RL16(avctx->extradata + 4);
-    liblc3->length   = AV_RL32(avctx->extradata + 6);
     if (ep_mode != 0) {
         av_log(avctx, AV_LOG_ERROR,
                "Error protection mode is not supported.\n");
@@ -126,11 +124,7 @@ static int liblc3_decode(AVCodecContext *avctx, AVFrame *frame,
         in += nbytes;
     }
 
-    if (liblc3->length > 0) {
-        int64_t end_pts = liblc3->length + avctx->delay;
-        frame->nb_samples = FFMIN(frame->nb_samples,
-                                  FFMAX(end_pts - frame->pts, 0));
-    }
+    frame->nb_samples = FFMIN(frame->nb_samples, avpkt->duration);
 
     *got_frame_ptr = 1;
 
-- 
2.44.0.478.gd926399ef9-goog

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 17+ messages in thread
end of thread, other threads:[~2024-04-06  7:54 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-03-28 21:35 [FFmpeg-devel] [PATCH 1/2] avcodec/liblc3: Add encoding/decoding support of LC3 audio codec Antoine Soulier via ffmpeg-devel
2024-03-28 21:35 ` [FFmpeg-devel] [PATCH 2/2] avformat/lc3: Add file format for LC3/LC3plus transport Antoine Soulier via ffmpeg-devel
2024-03-28 21:49   ` Andreas Rheinhardt
2024-03-28 22:33 [FFmpeg-devel] [PATCH 1/2] avcodec/liblc3: Add encoding/decoding support of LC3 audio codec Antoine Soulier via ffmpeg-devel
2024-03-28 22:33 ` [FFmpeg-devel] [PATCH 2/2] avformat/lc3: Add file format for LC3/LC3plus transport Antoine Soulier via ffmpeg-devel
2024-03-29 17:30 [FFmpeg-devel] [PATCH 1/2] avcodec/liblc3: Add encoding/decoding support of LC3 audio codec Antoine Soulier via ffmpeg-devel
2024-03-29 17:30 ` [FFmpeg-devel] [PATCH 2/2] avformat/lc3: Add file format for LC3/LC3plus transport Antoine Soulier via ffmpeg-devel
2024-03-30 11:45   ` Paul B Mahol
2024-03-30 21:51     ` Antoine Soulier via ffmpeg-devel
2024-03-30 23:15       ` Paul B Mahol
2024-04-01 21:32 [FFmpeg-devel] [PATCH 1/2] avcodec/liblc3: Add encoding/decoding support of LC3 audio codec Antoine Soulier via ffmpeg-devel
2024-04-01 21:32 ` [FFmpeg-devel] [PATCH 2/2] avformat/lc3: Add file format for LC3/LC3plus transport Antoine Soulier via ffmpeg-devel
2024-04-04 15:40   ` Stefano Sabatini
2024-04-04 16:51     ` Andreas Rheinhardt
2024-04-04 16:23   ` James Almer
2024-04-04 16:30   ` Andreas Rheinhardt
2024-04-04 16:54     ` Antoine Soulier via ffmpeg-devel
2024-04-04 17:09       ` Andreas Rheinhardt
2024-04-04 22:57 [FFmpeg-devel] [PATCH 1/2] avcodec/liblc3dec: Retrieve duration of the last packet from the demux Antoine Soulier via ffmpeg-devel
2024-04-04 22:57 ` [FFmpeg-devel] [PATCH 2/2] avformat/lc3: Add file format for LC3/LC3plus transport Antoine Soulier via ffmpeg-devel
2024-04-06  7:54   ` Stefano Sabatini

Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
		ffmpegdev@gitmailbox.com
	public-inbox-index ffmpegdev

Example config snippet for mirrors.


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git