Re: [FFmpeg-devel] [PATCH] motion_est: fixed signed_integer_overflow when computing score

From: James Almer <jamrial@gmail.com>
To: ffmpeg-devel@ffmpeg.org
Subject: Re: [FFmpeg-devel] [PATCH] motion_est: fixed signed_integer_overflow when computing score
Date: Fri, 11 Nov 2022 15:15:56 -0300
Message-ID: <09257d5c-0ffe-9336-a143-e9a9c83356a4@gmail.com> (raw)
In-Reply-To: <20221109223132.1795504-1-tfoucu@gmail.com>

On 11/9/2022 7:31 PM, Thierry Foucu wrote:
> ---
>   libavcodec/motion_est.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavcodec/motion_est.c b/libavcodec/motion_est.c
> index d17ffe42b4..95978d95dd 100644
> --- a/libavcodec/motion_est.c
> +++ b/libavcodec/motion_est.c
> @@ -1571,7 +1571,7 @@ void ff_estimate_b_frame_motion(MpegEncContext * s,
>               type= CANDIDATE_MB_TYPE_BACKWARD_I;
>           }
>   
> -        score= ((unsigned)(score*score + 128*256))>>16;
> +        score= ((uint64_t)(score)*(uint64_t)(score) + 128*256)>>16;

Is it not enough just doing

score= ((unsigned)score*score + 128*256)>>16;

If not, then use uint64_t, but like the above, you should only need to 
cast score once.

>           c->mc_mb_var_sum_temp += score;
>           s->mc_mb_var[mb_y*s->mb_stride + mb_x] = score; //FIXME use SSE
>       }
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".