[FFmpeg-devel] [PATCH] avcodec/jpeg2000dec: Ensure calculation of buf_size cannot overflow.

[FFmpeg-devel] [PATCH] avcodec/jpeg2000dec: Ensure calculation of buf_size cannot overflow.

[etemesicaleb]

From: caleb <etemesicaleb@gmail.com>

---
 libavcodec/jpeg2000htdec.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/jpeg2000htdec.c b/libavcodec/jpeg2000htdec.c
index 51cd96e0f1..d77293ddd8 100644
--- a/libavcodec/jpeg2000htdec.c
+++ b/libavcodec/jpeg2000htdec.c
@@ -595,7 +595,7 @@ static int jpeg2000_decode_ht_cleanup_segment(const Jpeg2000DecoderContext *s,
     const uint16_t quad_width  = ff_jpeg2000_ceildivpow2(width, 1);
     const uint16_t quad_height = ff_jpeg2000_ceildivpow2(height, 1);
 
-    size_t buf_size = 4 * quad_width * quad_height;
+    size_t buf_size = 4UL * quad_width * quad_height;
 
     uint8_t *sigma_n = av_calloc(buf_size, sizeof(uint8_t));
     uint8_t *E       = av_calloc(buf_size, sizeof(uint8_t));
-- 
2.39.2

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/jpeg2000dec: Ensure calculation of buf_size cannot overflow.

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 1440 bytes --]

On Fri, Apr 28, 2023 at 07:42:48AM +0300, etemesicaleb@gmail.com wrote:
> From: caleb <etemesicaleb@gmail.com>
> 
> ---
>  libavcodec/jpeg2000htdec.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavcodec/jpeg2000htdec.c b/libavcodec/jpeg2000htdec.c
> index 51cd96e0f1..d77293ddd8 100644
> --- a/libavcodec/jpeg2000htdec.c
> +++ b/libavcodec/jpeg2000htdec.c
> @@ -595,7 +595,7 @@ static int jpeg2000_decode_ht_cleanup_segment(const Jpeg2000DecoderContext *s,
>      const uint16_t quad_width  = ff_jpeg2000_ceildivpow2(width, 1);
>      const uint16_t quad_height = ff_jpeg2000_ceildivpow2(height, 1);
>  
> -    size_t buf_size = 4 * quad_width * quad_height;
> +    size_t buf_size = 4UL * quad_width * quad_height;

The only caller asserts that width and height are <= 1024 so this change
should not be needed
4 is an int and int is by POSIX 32bit at least

if you want to make the code "nicer", i would suggest
changing the quad_width and quad_height to int. uint16_t should generally
only be used if theres a reason. (space for an array can be a reason so can
it be to intentionally truncate to 16bit) But needing only 16bit is a bad
reason. uint16_t is generally not faster, in fact C treats it like a int anyway

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

No snowflake in an avalanche ever feels responsible. -- Voltaire

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".