Re: [FFmpeg-devel] [PATCH 05/11] avcodec/flacdec: Fix signed integre overflow

From: James Almer <jamrial@gmail.com>
To: ffmpeg-devel@ffmpeg.org
Subject: Re: [FFmpeg-devel] [PATCH 05/11] avcodec/flacdec: Fix signed integre overflow
Date: Fri, 5 May 2023 19:36:05 -0300
Message-ID: <cb0a1de4-63bc-b8cc-585a-daa075610d55@gmail.com> (raw)
In-Reply-To: <20230416164830.15664-5-michael@niedermayer.cc>

On 4/16/2023 1:48 PM, Michael Niedermayer wrote:
> Fixes: signed integer overflow: 3011809745540902265 + 6323452730883571725 cannot be represented in type 'long'
> Fixes: 45982/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FLAC_fuzzer-6687553022722048
> 
> Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>   libavcodec/flacdec.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavcodec/flacdec.c b/libavcodec/flacdec.c
> index cc778a8dff1..524a0469495 100644
> --- a/libavcodec/flacdec.c
> +++ b/libavcodec/flacdec.c
> @@ -513,7 +513,7 @@ static int decode_subframe_lpc_33bps(FLACContext *s, int64_t *decoded,
>       for (i = pred_order; i < s->blocksize; i++, decoded++) {
>           int64_t sum = 0;
>           for (j = 0; j < pred_order; j++)
> -            sum += (int64_t)coeffs[j] * decoded[j];
> +            sum += (int64_t)coeffs[j] * (uint64_t)decoded[j];

Why not instead do

sum = av_sat_add64(sum, (int64_t)coeffs[j] * decoded[j]);

Also, decoded[j] is an int64_t, so wouldn't coeffs[j] be promoted if you 
swap the order in the multiplication, thus saving the cast?

>           decoded[j] = residual[i] + (sum >> qlevel);
>       }
>   
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".