Re: [FFmpeg-devel] [PATCH 1/7] avcodec/sunrast: Use ptrdiff_t for stride

[James Almer <jamrial@gmail.com>]

On 9/30/2022 2:02 PM, Andreas Rheinhardt wrote:
> Fixes segfaults with negative linesizes; in particular,
> this affected the sunraster-(1|8|24)bit-(raw|rle) and
> sunraster-8bit_gray-raw FATE tests.
> 
> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
> ---
>   libavcodec/sunrast.c | 5 +++--
>   1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/sunrast.c b/libavcodec/sunrast.c
> index 77feef06e1..45b29e4d72 100644
> --- a/libavcodec/sunrast.c
> +++ b/libavcodec/sunrast.c
> @@ -31,7 +31,8 @@ static int sunrast_decode_frame(AVCodecContext *avctx, AVFrame *p,
>   {
>       const uint8_t *buf       = avpkt->data;
>       const uint8_t *buf_end   = avpkt->data + avpkt->size;
> -    unsigned int w, h, depth, type, maptype, maplength, stride, x, y, len, alen;
> +    unsigned int w, h, depth, type, maptype, maplength, x, y, len, alen;
> +    ptrdiff_t stride;
>       uint8_t *ptr, *ptr2 = NULL;
>       const uint8_t *bufstart = buf;
>       int ret;
> @@ -141,7 +142,7 @@ static int sunrast_decode_frame(AVCodecContext *avctx, AVFrame *p,
>   
>       if (type == RT_BYTE_ENCODED) {
>           int value, run;
> -        uint8_t *end = ptr + h * stride;
> +        uint8_t *end = ptr + (ptrdiff_t)h * stride;

Is the cast needed if stride is already ptrdiff_t?

>   
>           x = 0;
>           while (ptr != end && buf < buf_end) {
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".