[FFmpeg-devel] [PATCH] [8.0] Backport Videotoolbox improvements (PR #20496)

[Marvin Scholz via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>]

PR #20496 opened by Marvin Scholz (ePirat)
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20496
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20496.patch


>From 8fbd564bed9ee358b7d376e80197253283c55679 Mon Sep 17 00:00:00 2001
From: Zhao Zhili <zhilizhao@tencent.com>
Date: Mon, 1 Sep 2025 01:17:08 +0800
Subject: [PATCH 1/4] avcodec/videotoolboxenc: fix the loss of precision when
 calculating quality

(cherry picked from commit 73750489a67dcf99a6142009edfe45f504d55e8e)
Signed-off-by: Marvin Scholz <epirat07@gmail.com>
---
 libavcodec/videotoolboxenc.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c
index b748ecda61..c3704810d2 100644
--- a/libavcodec/videotoolboxenc.c
+++ b/libavcodec/videotoolboxenc.c
@@ -1185,9 +1185,7 @@ static int vtenc_create_encoder(AVCodecContext   *avctx,
     VTEncContext *vtctx = avctx->priv_data;
     SInt32       bit_rate = avctx->bit_rate;
     SInt32       max_rate = avctx->rc_max_rate;
-    Float32      quality = avctx->global_quality / FF_QP2LAMBDA;
     CFNumberRef  bit_rate_num;
-    CFNumberRef  quality_num;
     CFNumberRef  bytes_per_second;
     CFNumberRef  one_second;
     CFArrayRef   data_rate_limits;
@@ -1248,10 +1246,10 @@ static int vtenc_create_encoder(AVCodecContext   *avctx,
     }
 
     if (avctx->flags & AV_CODEC_FLAG_QSCALE) {
-        quality = quality >= 100 ? 1.0 : quality / 100;
-        quality_num = CFNumberCreate(kCFAllocatorDefault,
-                                     kCFNumberFloat32Type,
-                                     &quality);
+        Float32 quality = fminf(avctx->global_quality / 100.0f / FF_QP2LAMBDA, 1.0f);
+        CFNumberRef quality_num = CFNumberCreate(kCFAllocatorDefault,
+                                                 kCFNumberFloat32Type,
+                                                 &quality);
         if (!quality_num) return AVERROR(ENOMEM);
 
         status = VTSessionSetProperty(vtctx->session,
-- 
2.49.1


>From d2eddd662f69e566e09de8ae02f3237cc32c4fd0 Mon Sep 17 00:00:00 2001
From: Zhao Zhili <zhilizhao@tencent.com>
Date: Mon, 1 Sep 2025 01:36:25 +0800
Subject: [PATCH 2/4] avcodec/videotoolboxenc: support global_quality without
 qscale

(cherry picked from commit fc45127bcc79731a2e3ec5858a00c35c6cfbc1c0)
Signed-off-by: Marvin Scholz <epirat07@gmail.com>
---
 libavcodec/videotoolboxenc.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c
index c3704810d2..e423da6933 100644
--- a/libavcodec/videotoolboxenc.c
+++ b/libavcodec/videotoolboxenc.c
@@ -1245,8 +1245,10 @@ static int vtenc_create_encoder(AVCodecContext   *avctx,
         return AVERROR_EXTERNAL;
     }
 
-    if (avctx->flags & AV_CODEC_FLAG_QSCALE) {
-        Float32 quality = fminf(avctx->global_quality / 100.0f / FF_QP2LAMBDA, 1.0f);
+    if (avctx->flags & AV_CODEC_FLAG_QSCALE || avctx->global_quality > 0) {
+        float factor = (avctx->flags & AV_CODEC_FLAG_QSCALE) ?
+                       FF_QP2LAMBDA * 100.0f : 100.0f;
+        Float32 quality = fminf(avctx->global_quality / factor, 1.0f);
         CFNumberRef quality_num = CFNumberCreate(kCFAllocatorDefault,
                                                  kCFNumberFloat32Type,
                                                  &quality);
-- 
2.49.1


>From 9da749b361a09cbbd9769a4586a10a70f1fff4ac Mon Sep 17 00:00:00 2001
From: Cameron Gutman <aicommander@gmail.com>
Date: Fri, 5 Sep 2025 23:42:52 -0500
Subject: [PATCH 3/4] avcodec/videotoolboxenc: allow low latency RC with HEVC

It is supported only for H.264 on Intel Macs, but it can be used with
both H.264 and HEVC on Apple Silicon.

Signed-off-by: Cameron Gutman <aicommander@gmail.com>
(cherry picked from commit d87210745e09f6d55a7e43f70bf9d8f81b5f739a)
Signed-off-by: Marvin Scholz <epirat07@gmail.com>
---
 libavcodec/videotoolboxenc.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c
index e423da6933..f0d3a15897 100644
--- a/libavcodec/videotoolboxenc.c
+++ b/libavcodec/videotoolboxenc.c
@@ -1709,7 +1709,8 @@ static int vtenc_configure_encoder(AVCodecContext *avctx)
 #endif
 
     // low-latency mode: eliminate frame reordering, follow a one-in-one-out encoding mode
-    if ((avctx->flags & AV_CODEC_FLAG_LOW_DELAY) && avctx->codec_id == AV_CODEC_ID_H264) {
+    if ((avctx->flags & AV_CODEC_FLAG_LOW_DELAY) &&
+        ((avctx->codec_id == AV_CODEC_ID_H264) || (TARGET_CPU_ARM64 && avctx->codec_id == AV_CODEC_ID_HEVC))) {
         CFDictionarySetValue(enc_info,
                              compat_keys.kVTVideoEncoderSpecification_EnableLowLatencyRateControl,
                              kCFBooleanTrue);
-- 
2.49.1


>From 7dd897dc6d34de9e19713f97854e74de80ffd3c9 Mon Sep 17 00:00:00 2001
From: Zhao Zhili <zhilizhao@tencent.com>
Date: Wed, 10 Sep 2025 20:29:47 +0800
Subject: [PATCH 4/4] avcodec/videotoolboxenc: ensure bitrate is set in
 low_delay mode

VideoToolbox doesn't support automatic bitrate in low delay mode.
Check bitrate and show error message so user knows what's going
wrong.

(cherry picked from commit c1dc2e2b7cc8df8a40b616793d1204be0e71103c)
Signed-off-by: Marvin Scholz <epirat07@gmail.com>
---
 libavcodec/videotoolboxenc.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c
index f0d3a15897..729072c0b9 100644
--- a/libavcodec/videotoolboxenc.c
+++ b/libavcodec/videotoolboxenc.c
@@ -1711,6 +1711,12 @@ static int vtenc_configure_encoder(AVCodecContext *avctx)
     // low-latency mode: eliminate frame reordering, follow a one-in-one-out encoding mode
     if ((avctx->flags & AV_CODEC_FLAG_LOW_DELAY) &&
         ((avctx->codec_id == AV_CODEC_ID_H264) || (TARGET_CPU_ARM64 && avctx->codec_id == AV_CODEC_ID_HEVC))) {
+        if (!avctx->bit_rate) {
+            av_log(avctx, AV_LOG_ERROR, "Doesn't support automatic bitrate in low_delay mode, "
+                                        "please specify bitrate explicitly\n");
+            status = AVERROR(EINVAL);
+            goto init_cleanup;
+        }
         CFDictionarySetValue(enc_info,
                              compat_keys.kVTVideoEncoderSpecification_EnableLowLatencyRateControl,
                              kCFBooleanTrue);
-- 
2.49.1

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org