From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 554AD45F9C for ; Sat, 22 Apr 2023 21:26:05 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5BE5668BF78; Sun, 23 Apr 2023 00:26:03 +0300 (EEST) Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [217.70.183.196]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1046768BE14 for ; Sun, 23 Apr 2023 00:25:56 +0300 (EEST) Received: (Authenticated sender: michael@niedermayer.cc) by mail.gandi.net (Postfix) with ESMTPSA id 5D6A4E0002 for ; Sat, 22 Apr 2023 21:25:56 +0000 (UTC) From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Sat, 22 Apr 2023 23:25:55 +0200 Message-Id: <20230422212555.11317-1-michael@niedermayer.cc> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH] avcodec/bonk: Avoid undefined integer overflow in predictor_calc_error() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Fixes: signed integer overflow: -159584 * 5105950 cannot be represented in type 'int' Fixes: 55165/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_BONK_fuzzer-5796023719297024 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/bonk.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavcodec/bonk.c b/libavcodec/bonk.c index 5f510e49103..f2baa9c9315 100644 --- a/libavcodec/bonk.c +++ b/libavcodec/bonk.c @@ -265,14 +265,14 @@ static inline int shift(int a, int b) static int predictor_calc_error(int *k, int *state, int order, int error) { - int i, x = error - shift_down(k[order-1] * state[order-1], LATTICE_SHIFT); + int i, x = error - shift_down(k[order-1] * (unsigned)state[order-1], LATTICE_SHIFT); int *k_ptr = &(k[order-2]), *state_ptr = &(state[order-2]); for (i = order-2; i >= 0; i--, k_ptr--, state_ptr--) { unsigned k_value = *k_ptr, state_value = *state_ptr; - x -= shift_down(k_value * state_value, LATTICE_SHIFT); + x -= shift_down(k_value * (unsigned)state_value, LATTICE_SHIFT); state_ptr[1] = state_value + shift_down(k_value * x, LATTICE_SHIFT); } -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".