[FFmpeg-devel] [PATCH] avcodec/vaapi_encode: avoid potential overflow (PR #20421)

[Tong Wu via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>]

PR #20421 opened by Tong Wu (tong1wu)
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20421
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20421.patch

The max_frame_size in byte will be filled in
VAEncMiscParameterBufferMaxFrameSize which receives size in bit.
This requires max_frame_size * 8.

Signed-off-by: Tong Wu <wutong1208@outlook.com>


>From 971e1dec185529f05f33076c2b6eb1b1210b91e1 Mon Sep 17 00:00:00 2001
From: Tong Wu <wutong1208@outlook.com>
Date: Thu, 4 Sep 2025 06:51:24 +0800
Subject: [PATCH] avcodec/vaapi_encode: avoid potential overflow

The max_frame_size in byte will be filled in
VAEncMiscParameterBufferMaxFrameSize which receives size in bit.
This requires max_frame_size * 8.

Signed-off-by: Tong Wu <wutong1208@outlook.com>
---
 libavcodec/vaapi_encode.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h
index 8e3eab9f27..1fa1e99a31 100644
--- a/libavcodec/vaapi_encode.h
+++ b/libavcodec/vaapi_encode.h
@@ -360,7 +360,7 @@ int ff_vaapi_encode_close(AVCodecContext *avctx);
     { "max_frame_size", \
       "Maximum frame size (in bytes)",\
       OFFSET(common.max_frame_size), AV_OPT_TYPE_INT, \
-      { .i64 = 0 }, 0, INT_MAX, FLAGS }
+      { .i64 = 0 }, 0, INT_MAX / 8, FLAGS }
 
 #define VAAPI_ENCODE_RC_MODE(name, desc) \
     { #name, desc, 0, AV_OPT_TYPE_CONST, { .i64 = RC_MODE_ ## name }, \
-- 
2.49.1

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org