[FFmpeg-devel] [PATCH 2/3] avcodec/mediacodecenc: workaround the alignment requirement only for H.264

[FFmpeg-devel] [PATCH 2/3] avcodec/mediacodecenc: workaround the alignment requirement only for H.264

[Zhao Zhili]

From: Zhao Zhili <zhilizhao@tencent.com>

There is no bsf for other codecs to modify crop info except H.265.
For H.265, the assumption that FFALIGN(width, 16)xFFALIGN(height, 16)
is the video resolution can be wrong, since the encoder can use CTU
larger than 16x16. In that case, use FFALIGN(width, 16) - width
as crop_right is incorrect. So disable the workaround for H.265 now.
---
 libavcodec/mediacodecenc.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/libavcodec/mediacodecenc.c b/libavcodec/mediacodecenc.c
index bbf570e7be..bfff149039 100644
--- a/libavcodec/mediacodecenc.c
+++ b/libavcodec/mediacodecenc.c
@@ -234,7 +234,8 @@ static av_cold int mediacodec_init(AVCodecContext *avctx)
     ff_AMediaFormat_setString(format, "mime", codec_mime);
     // Workaround the alignment requirement of mediacodec. We can't do it
     // silently for AV_PIX_FMT_MEDIACODEC.
-    if (avctx->pix_fmt != AV_PIX_FMT_MEDIACODEC) {
+    if (avctx->pix_fmt != AV_PIX_FMT_MEDIACODEC &&
+        avctx->codec_id == AV_CODEC_ID_H264) {
         s->width = FFALIGN(avctx->width, 16);
         s->height = FFALIGN(avctx->height, 16);
     } else {
-- 
2.42.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 3/3] avcodec/mediacodecenc: workaround the alignment requirement for H.265

[Zhao Zhili]

From: Zhao Zhili <zhilizhao@tencent.com>

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
---
 libavcodec/mediacodecenc.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/libavcodec/mediacodecenc.c b/libavcodec/mediacodecenc.c
index bfff149039..a42270551b 100644
--- a/libavcodec/mediacodecenc.c
+++ b/libavcodec/mediacodecenc.c
@@ -154,8 +154,12 @@ static int mediacodec_init_bsf(AVCodecContext *avctx)
             ret = snprintf(str, sizeof(str), "h264_metadata=crop_right=%d:crop_bottom=%d",
                            crop_right, crop_bottom);
         else if (avctx->codec_id == AV_CODEC_ID_HEVC)
-            ret = snprintf(str, sizeof(str), "hevc_metadata=crop_right=%d:crop_bottom=%d",
-                           crop_right, crop_bottom);
+            /* Encoder can use CTU size larger than 16x16, so the real crop
+             * margin can be larger than crop_right/crop_bottom. Let bsf figure
+             * out the real crop margin.
+             */
+            ret = snprintf(str, sizeof(str), "hevc_metadata=width=%d:height=%d",
+                           avctx->width, avctx->height);
         if (ret >= sizeof(str))
             return AVERROR_BUFFER_TOO_SMALL;
     }
@@ -235,7 +239,8 @@ static av_cold int mediacodec_init(AVCodecContext *avctx)
     // Workaround the alignment requirement of mediacodec. We can't do it
     // silently for AV_PIX_FMT_MEDIACODEC.
     if (avctx->pix_fmt != AV_PIX_FMT_MEDIACODEC &&
-        avctx->codec_id == AV_CODEC_ID_H264) {
+        (avctx->codec_id == AV_CODEC_ID_H264 ||
+         avctx->codec_id == AV_CODEC_ID_HEVC)) {
         s->width = FFALIGN(avctx->width, 16);
         s->height = FFALIGN(avctx->height, 16);
     } else {
-- 
2.42.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 3/3] avcodec/mediacodecenc: workaround the alignment requirement for H.265

[Zhao Zhili]

> On May 23, 2024, at 15:08, Zhao Zhili <quinkblack@foxmail.com> wrote:
> 
> From: Zhao Zhili <zhilizhao@tencent.com>
> 
> Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
> ---
> libavcodec/mediacodecenc.c | 11 ++++++++---
> 1 file changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/libavcodec/mediacodecenc.c b/libavcodec/mediacodecenc.c
> index bfff149039..a42270551b 100644
> --- a/libavcodec/mediacodecenc.c
> +++ b/libavcodec/mediacodecenc.c
> @@ -154,8 +154,12 @@ static int mediacodec_init_bsf(AVCodecContext *avctx)
>             ret = snprintf(str, sizeof(str), "h264_metadata=crop_right=%d:crop_bottom=%d",
>                            crop_right, crop_bottom);
>         else if (avctx->codec_id == AV_CODEC_ID_HEVC)
> -            ret = snprintf(str, sizeof(str), "hevc_metadata=crop_right=%d:crop_bottom=%d",
> -                           crop_right, crop_bottom);
> +            /* Encoder can use CTU size larger than 16x16, so the real crop
> +             * margin can be larger than crop_right/crop_bottom. Let bsf figure
> +             * out the real crop margin.
> +             */
> +            ret = snprintf(str, sizeof(str), "hevc_metadata=width=%d:height=%d",
> +                           avctx->width, avctx->height);
>         if (ret >= sizeof(str))
>             return AVERROR_BUFFER_TOO_SMALL;
>     }
> @@ -235,7 +239,8 @@ static av_cold int mediacodec_init(AVCodecContext *avctx)
>     // Workaround the alignment requirement of mediacodec. We can't do it
>     // silently for AV_PIX_FMT_MEDIACODEC.
>     if (avctx->pix_fmt != AV_PIX_FMT_MEDIACODEC &&
> -        avctx->codec_id == AV_CODEC_ID_H264) {
> +        (avctx->codec_id == AV_CODEC_ID_H264 ||
> +         avctx->codec_id == AV_CODEC_ID_HEVC)) {
>         s->width = FFALIGN(avctx->width, 16);
>         s->height = FFALIGN(avctx->height, 16);
>     } else {
> -- 
> 2.42.0
> 

For anyone who is interested on why the workaround is important, here is an example.

This mediacodec xml comes from a pretty new mobile made by Vivo, with CPU from MTK.
Both H.264 and H.265 encoder require alignment of 16x16. MediaCodec will fail during
configure with resolution like 1080x1920. When use surface as input, it will output a video
stream with resolution 1072x1920. It’s weird, but it’s not rare.

  <MediaCodec name="c2.mtk.avc.encoder" type="video/avc" >
            <Alias name="OMX.MTK.VIDEO.ENCODER.AVC" />
            <Limit name="size" min="160x128" max="3840x2176" />
            <Limit name="alignment" value="16x16" />
            <Limit name="block-size" value="16x16" />
            <Limit name="concurrent-instances" max="10" />
            <Limit name="performance-point-3840x2160" value="30" />
            <Limit name="performance-point-1280x719" value="140" />
            <Limit name="performance-point-1280x720" value="120" />
        </MediaCodec>

  <MediaCodec name="c2.mtk.hevc.encoder" type="video/hevc" >
            <Alias name="OMX.MTK.VIDEO.ENCODER.HEVC" />
            <Limit name="size" min="160x128" max="7680x4320" />
            <Limit name="alignment" value="16x16" />
            <Limit name="block-size" value="16x16" />
            <Limit name="quality" range="0-100" default="50" />
            <Limit name="concurrent-instances" max="10" />
            <Feature name="bitrate-modes" value="VBR,CBR,CQ" />
            <Limit name="performance-point-3840x2160" value="30" />
            <Limit name="performance-point-1280x719" value="140" />
            <Limit name="performance-point-1280x720" value="120" />
        </MediaCodec>


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".