[FFmpeg-devel] [PATCH] avcodec/aacenc: add strict bit rate control mode

[FFmpeg-devel] [PATCH] avcodec/aacenc: add strict bit rate control mode

[Jeremy Wu]

From: Jeremy Wu <jrwu@google.com>

In certain use cases, controlling the maximum frame size is critical. An
example is when transmitting AAC packets over Bluetooth A2DP.

While the spec allows the packets be fragmented (though UNRECOMMENDED),
in practice most headsets do not recognize nor reassemble such packets.

In this patch, we add a new mode to specify that the configured bit rate
should be followed strictly up to frame level.

Signed-off-by: Jeremy Wu <jrwu@chromium.org>
---
 doc/APIchanges       |  3 +++
 libavcodec/aacenc.c  | 11 +++++++++++
 libavcodec/avcodec.h |  4 ++++
 libavcodec/version.h |  2 +-
 4 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/doc/APIchanges b/doc/APIchanges
index 0b609e3d3b..e730a7e126 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -2,6 +2,9 @@ The last version increases of all libraries were on 2023-02-09
 
 API changes, most recent first:
 
+2023-0x-xx - xxxxxxxxxx - lavc 60.11.100 - avcodec.h
+  Add AV_CODEC_FLAG_STRICT_BITRATE.
+
 2023-04-10 - xxxxxxxxxx - lavu 58.6.100 - frame.h
   av_frame_get_plane_buffer() now accepts const AVFrame*.
 
diff --git a/libavcodec/aacenc.c b/libavcodec/aacenc.c
index ed036209e9..daf5538056 100644
--- a/libavcodec/aacenc.c
+++ b/libavcodec/aacenc.c
@@ -1106,6 +1106,17 @@ static int aac_encode_frame(AVCodecContext *avctx, AVPacket *avpkt,
         too_many_bits = FFMIN(too_many_bits, 6144 * s->channels - 3);
         too_few_bits = FFMIN(FFMAX(rate_bits - rate_bits/4, target_bits), too_many_bits);
 
+        if (avctx->flags & AV_CODEC_FLAG_STRICT_BITRATE) {
+            if (rate_bits < frame_bits) {
+                /* temporarily degrade quality and repeat until frame fits */
+                s->lambda *= 0.75f;
+                continue;
+            }
+            /* reset lambda when solution is found */
+            s->lambda = avctx->global_quality > 0 ? avctx->global_quality : 120;
+            break;
+        }
+
         /* When using ABR, be strict (but only for increasing) */
         too_few_bits = too_few_bits - too_few_bits/8;
         too_many_bits = too_many_bits + too_many_bits/2;
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 1e91b9cb53..bc9ecdff23 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -333,6 +333,10 @@ typedef struct RcOverride{
  * H.263 advanced intra coding / MPEG-4 AC prediction
  */
 #define AV_CODEC_FLAG_AC_PRED         (1 << 24)
+/**
+ * Treat specified bit rate as upper bound up to frame level.
+ */
+#define AV_CODEC_FLAG_STRICT_BITRATE  (1 << 25)
 /**
  * interlaced motion estimation
  */
diff --git a/libavcodec/version.h b/libavcodec/version.h
index 80e2ae630d..8b53586be1 100644
--- a/libavcodec/version.h
+++ b/libavcodec/version.h
@@ -29,7 +29,7 @@
 
 #include "version_major.h"
 
-#define LIBAVCODEC_VERSION_MINOR  10
+#define LIBAVCODEC_VERSION_MINOR  11
 #define LIBAVCODEC_VERSION_MICRO 100
 
 #define LIBAVCODEC_VERSION_INT  AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \
-- 
2.40.0.634.g4ca3ef3211-goog

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/aacenc: add strict bit rate control mode

[Lynne]

Apr 24, 2023, 11:27 by jrwu@chromium.org:

> From: Jeremy Wu <jrwu@google.com>
>
> In certain use cases, controlling the maximum frame size is critical. An
> example is when transmitting AAC packets over Bluetooth A2DP.
>
> While the spec allows the packets be fragmented (though UNRECOMMENDED),
> in practice most headsets do not recognize nor reassemble such packets.
>
> In this patch, we add a new mode to specify that the configured bit rate
> should be followed strictly up to frame level.
>
> Signed-off-by: Jeremy Wu <jrwu@chromium.org>
> ---
>  doc/APIchanges       |  3 +++
>  libavcodec/aacenc.c  | 11 +++++++++++
>  libavcodec/avcodec.h |  4 ++++
>  libavcodec/version.h |  2 +-
>  4 files changed, 19 insertions(+), 1 deletion(-)
>
> diff --git a/doc/APIchanges b/doc/APIchanges
> index 0b609e3d3b..e730a7e126 100644
> --- a/doc/APIchanges
> +++ b/doc/APIchanges
> @@ -2,6 +2,9 @@ The last version increases of all libraries were on 2023-02-09
>
>  2023-04-10 - xxxxxxxxxx - lavu 58.6.100 - frame.h
>  av_frame_get_plane_buffer() now accepts const AVFrame*.
>  
> diff --git a/libavcodec/aacenc.c b/libavcodec/aacenc.c
> index ed036209e9..daf5538056 100644
> --- a/libavcodec/aacenc.c
> +++ b/libavcodec/aacenc.c
> @@ -1106,6 +1106,17 @@ static int aac_encode_frame(AVCodecContext *avctx, AVPacket *avpkt,
>  too_many_bits = FFMIN(too_many_bits, 6144 * s->channels - 3);
>  too_few_bits = FFMIN(FFMAX(rate_bits - rate_bits/4, target_bits), too_many_bits);
>  
> +        if (avctx->flags & AV_CODEC_FLAG_STRICT_BITRATE) {
>

Use avctx->bit_rate_tolerance instead. By default, it's set to ~400ish kbps.
Just detect if it's set to zero to enable this code path. You can set the
variable via both the command line and the API.


> +            if (rate_bits < frame_bits) {
> +                /* temporarily degrade quality and repeat until frame fits */
> +                s->lambda *= 0.75f;
> +                continue;
> +            }
> +            /* reset lambda when solution is found */
> +            s->lambda = avctx->global_quality > 0 ? avctx->global_quality : 120;
> +            break;
> +        }
>

That's a heavy handed approach. A better way would be to use the
rate_bits/frame_bits, multiply lambda by the ratio, and use that lambda as
the starting point. That way, you'd need less reencodes to satisfy the condition.

By the way, do you have any specific use for the encoder?
I am in the process of rewriting it, and I'd like to know where it's useful,
apart from the obvious streaming use-case.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/aacenc: add strict bit rate control mode

[Jeremy Wu]

Thanks for the feedback! I was also looking at whether there is a less 
intrusive way to enable the code path.

I have uploaded a v2 patch that looks at `bit_rate_tolerance`, with 
minor changes to the option table as it was ignored in the context of audio.

I'm only looking at the Bluetooth streaming use case as of now, and the 
issue cannot be bypassed without this (common implementations use libfdk 
that supports a similar option).

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH] avcodec/aacenc: add strict bit rate control mode

[Jeremy Wu]

In certain use cases, controlling the maximum frame size is critical. An
example is when transmitting AAC packets over Bluetooth A2DP.

While the spec allows the packets be fragmented (though UNRECOMMENDED),
in practice most headsets do not recognize nor reassemble such packets.

In this patch, we add a new mode to specify that the configured bit rate
should be followed strictly up to frame level.

Signed-off-by: Jeremy Wu <jrwu@chromium.org>
---
doc/APIchanges | 3 +++
libavcodec/aacenc.c | 11 +++++++++++
libavcodec/avcodec.h | 4 ++++
libavcodec/version.h | 2 +-
4 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/doc/APIchanges b/doc/APIchanges
index 0b609e3d3b..e730a7e126 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -2,6 +2,9 @@ The last version increases of all libraries were on 
2023-02-09
API changes, most recent first:
+2023-0x-xx - xxxxxxxxxx - lavc 60.11.100 - avcodec.h
+ Add AV_CODEC_FLAG_STRICT_BITRATE.
+
2023-04-10 - xxxxxxxxxx - lavu 58.6.100 - frame.h
av_frame_get_plane_buffer() now accepts const AVFrame*.
diff --git a/libavcodec/aacenc.c b/libavcodec/aacenc.c
index ed036209e9..daf5538056 100644
--- a/libavcodec/aacenc.c
+++ b/libavcodec/aacenc.c
@@ -1106,6 +1106,17 @@ static int aac_encode_frame(AVCodecContext 
*avctx, AVPacket *avpkt,
too_many_bits = FFMIN(too_many_bits, 6144 * s->channels - 3);
too_few_bits = FFMIN(FFMAX(rate_bits - rate_bits/4, target_bits), 
too_many_bits);
+ if (avctx->flags & AV_CODEC_FLAG_STRICT_BITRATE) {
+ if (rate_bits < frame_bits) {
+ /* temporarily degrade quality and repeat until frame fits */
+ s->lambda *= 0.75f;
+ continue;
+ }
+ /* reset lambda when solution is found */
+ s->lambda = avctx->global_quality > 0 ? avctx->global_quality : 120;
+ break;
+ }
+
/* When using ABR, be strict (but only for increasing) */
too_few_bits = too_few_bits - too_few_bits/8;
too_many_bits = too_many_bits + too_many_bits/2;
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 1e91b9cb53..bc9ecdff23 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -333,6 +333,10 @@ typedef struct RcOverride{
* H.263 advanced intra coding / MPEG-4 AC prediction
*/
#define AV_CODEC_FLAG_AC_PRED (1 << 24)
+/**
+ * Treat specified bit rate as upper bound up to frame level.
+ */
+#define AV_CODEC_FLAG_STRICT_BITRATE (1 << 25)
/**
* interlaced motion estimation
*/
diff --git a/libavcodec/version.h b/libavcodec/version.h
index 80e2ae630d..8b53586be1 100644
--- a/libavcodec/version.h
+++ b/libavcodec/version.h
@@ -29,7 +29,7 @@
#include "version_major.h"
-#define LIBAVCODEC_VERSION_MINOR 10
+#define LIBAVCODEC_VERSION_MINOR 11
#define LIBAVCODEC_VERSION_MICRO 100
#define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \

-- 
2.40.0.634.g4ca3ef3211-goog

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH] avcodec/aacenc: add strict bit rate control mode

[Jeremy Wu]

In certain use cases, controlling the maximum frame size is critical. An
example is when transmitting AAC packets over Bluetooth A2DP.

While the spec allows the packets be fragmented (though UNRECOMMENDED),
in practice most headsets do not recognize nor reassemble such packets.

In this patch, we add a new mode to specify that the configured bit rate
should be followed strictly up to frame level.

Signed-off-by: Jeremy Wu <jrwu@chromium.org>
---
doc/APIchanges | 3 +++
libavcodec/aacenc.c | 11 +++++++++++
libavcodec/avcodec.h | 4 ++++
libavcodec/version.h | 2 +-
4 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/doc/APIchanges b/doc/APIchanges
index 0b609e3d3b..e730a7e126 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -2,6 +2,9 @@ The last version increases of all libraries were on 
2023-02-09
API changes, most recent first:
+2023-0x-xx - xxxxxxxxxx - lavc 60.11.100 - avcodec.h
+ Add AV_CODEC_FLAG_STRICT_BITRATE.
+
2023-04-10 - xxxxxxxxxx - lavu 58.6.100 - frame.h
av_frame_get_plane_buffer() now accepts const AVFrame*.
diff --git a/libavcodec/aacenc.c b/libavcodec/aacenc.c
index ed036209e9..daf5538056 100644
--- a/libavcodec/aacenc.c
+++ b/libavcodec/aacenc.c
@@ -1106,6 +1106,17 @@ static int aac_encode_frame(AVCodecContext 
*avctx, AVPacket *avpkt,
too_many_bits = FFMIN(too_many_bits, 6144 * s->channels - 3);
too_few_bits = FFMIN(FFMAX(rate_bits - rate_bits/4, target_bits), 
too_many_bits);
+ if (avctx->flags & AV_CODEC_FLAG_STRICT_BITRATE) {
+ if (rate_bits < frame_bits) {
+ /* temporarily degrade quality and repeat until frame fits */
+ s->lambda *= 0.75f;
+ continue;
+ }
+ /* reset lambda when solution is found */
+ s->lambda = avctx->global_quality > 0 ? avctx->global_quality : 120;
+ break;
+ }
+
/* When using ABR, be strict (but only for increasing) */
too_few_bits = too_few_bits - too_few_bits/8;
too_many_bits = too_many_bits + too_many_bits/2;
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 1e91b9cb53..bc9ecdff23 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -333,6 +333,10 @@ typedef struct RcOverride{
* H.263 advanced intra coding / MPEG-4 AC prediction
*/
#define AV_CODEC_FLAG_AC_PRED (1 << 24)
+/**
+ * Treat specified bit rate as upper bound up to frame level.
+ */
+#define AV_CODEC_FLAG_STRICT_BITRATE (1 << 25)
/**
* interlaced motion estimation
*/
diff --git a/libavcodec/version.h b/libavcodec/version.h
index 80e2ae630d..8b53586be1 100644
--- a/libavcodec/version.h
+++ b/libavcodec/version.h
@@ -29,7 +29,7 @@
#include "version_major.h"
-#define LIBAVCODEC_VERSION_MINOR 10
+#define LIBAVCODEC_VERSION_MINOR 11
#define LIBAVCODEC_VERSION_MICRO 100
#define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \

-- 
2.40.0.634.g4ca3ef3211-goog

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".