From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id C9D6C4731E for ; Thu, 5 Oct 2023 19:45:27 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9A6F168CC5C; Thu, 5 Oct 2023 22:44:54 +0300 (EEST) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7614B68CB94 for ; Thu, 5 Oct 2023 22:44:46 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id 8E3C040004 for ; Thu, 5 Oct 2023 19:44:45 +0000 (UTC) From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Thu, 5 Oct 2023 21:44:40 +0200 Message-Id: <20231005194440.1678-5-michael@niedermayer.cc> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20231005194440.1678-1-michael@niedermayer.cc> References: <20231005194440.1678-1-michael@niedermayer.cc> X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 5/5] avcodec/xvididct: Make c* unsigned to avoid undefined overflows X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Fixes: signed integer overflow: 1496950099 + 728014168 cannot be represented in type 'int' Fixes: 62667/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MJPEGB_fuzzer-6511785170305024 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/xvididct.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/libavcodec/xvididct.c b/libavcodec/xvididct.c index dcea32210a2..01072d80ab7 100644 --- a/libavcodec/xvididct.c +++ b/libavcodec/xvididct.c @@ -56,13 +56,13 @@ static const int TAB35[] = { 26722, 25172, 22654, 19266, 15137, 10426, 5315 }; static int idct_row(short *in, const int *const tab, int rnd) { - const int c1 = tab[0]; - const int c2 = tab[1]; - const int c3 = tab[2]; - const int c4 = tab[3]; - const int c5 = tab[4]; - const int c6 = tab[5]; - const int c7 = tab[6]; + const unsigned c1 = tab[0]; + const unsigned c2 = tab[1]; + const unsigned c3 = tab[2]; + const unsigned c4 = tab[3]; + const unsigned c5 = tab[4]; + const unsigned c6 = tab[5]; + const unsigned c7 = tab[6]; const int right = in[5] | in[6] | in[7]; const int left = in[1] | in[2] | in[3]; @@ -102,8 +102,8 @@ static int idct_row(short *in, const int *const tab, int rnd) return 0; } } else if (!(left | right)) { - const int a0 = (rnd + c4 * (in[0] + in[4])) >> ROW_SHIFT; - const int a1 = (rnd + c4 * (in[0] - in[4])) >> ROW_SHIFT; + const int a0 = (int)(rnd + c4 * (in[0] + in[4])) >> ROW_SHIFT; + const int a1 = (int)(rnd + c4 * (in[0] - in[4])) >> ROW_SHIFT; in[0] = a0; in[3] = a0; -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".