[FFmpeg-devel] [PR] avcodec/libtheoraenc: make keyframe mask unsigned and handle its larger range (PR #22244)

[michaelni via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>]

PR #22244 opened by michaelni
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/22244
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/22244.patch

Fixes: left shift of 1 by 31 places cannot be represented in type 'int'
Fixes: 473579864/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LIBTHEORA_fuzzer-5835688160591872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>


>From 1e59c4baa432681ed15944039672425e98421e36 Mon Sep 17 00:00:00 2001
From: Michael Niedermayer <michael@niedermayer.cc>
Date: Sat, 21 Feb 2026 02:12:03 +0100
Subject: [PATCH] avcodec/libtheoraenc: make keyframe mask unsigned and handle
 its larger range

Fixes: left shift of 1 by 31 places cannot be represented in type 'int'
Fixes: 473579864/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LIBTHEORA_fuzzer-5835688160591872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavcodec/libtheoraenc.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavcodec/libtheoraenc.c b/libavcodec/libtheoraenc.c
index 2b70ec06a6..e089fe8f5b 100644
--- a/libavcodec/libtheoraenc.c
+++ b/libavcodec/libtheoraenc.c
@@ -53,7 +53,7 @@ typedef struct TheoraContext {
     int         stats_offset;
     int         uv_hshift;
     int         uv_vshift;
-    int         keyframe_mask;
+    unsigned    keyframe_mask;
     int         speed_level;
 } TheoraContext;
 
@@ -249,7 +249,7 @@ static av_cold int encode_init(AVCodecContext* avc_context)
         return AVERROR_EXTERNAL;
     }
 
-    h->keyframe_mask = (1 << av_ceil_log2(avc_context->gop_size)) - 1;
+    h->keyframe_mask = (1U << av_ceil_log2(avc_context->gop_size)) - 1;
     /* Clear up theora_info struct */
     th_info_clear(&t_info);
 
-- 
2.52.0

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org