[FFmpeg-devel] Re: [PATCH] libswscale/output: fix signed integer overflow in yuv2rgba64

[Michael Niedermayer via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>]

[-- Attachment #1.1: Type: text/plain, Size: 2034 bytes --]

Hi Hankang Li

On Wed, Feb 11, 2026 at 01:00:42AM -0800, Hankang Li via ffmpeg-devel wrote:
> Clamp the intermediate alpha value within 19 bits, this prevents
> overflow.
> 
> Note: This issue is only reproducible with --disable-x86asm. There is an
> existing issue in fate-sws-unscaled with --disable-x86asm (ticket
>  #21113). This change does not introduce any new regressions.
> 
> Signed-off-by: Hankang Li <hankang201222@gmail.com>
> ---
>  libswscale/output.c | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/libswscale/output.c b/libswscale/output.c
> index a3b8da54bf..afb3878170 100644
> --- a/libswscale/output.c
> +++ b/libswscale/output.c
> @@ -1288,8 +1288,8 @@ yuv2rgba64_1_c_template(SwsInternal *c, const int32_t *buf0,
>              Y2 += (1 << 13) - (1 << 29);
>  
>              if (hasAlpha) {
> -                A1 = abuf0[i * 2    ] * (1 << 11);
> -                A2 = abuf0[i * 2 + 1] * (1 << 11);
> +                A1 = av_clip_uintp2(abuf0[i * 2    ], 19) * (1 << 11);
> +                A2 = av_clip_uintp2(abuf0[i * 2 + 1], 19) * (1 << 11);
>  
>                  A1 += 1 << 13;
>                  A2 += 1 << 13;
> @@ -1337,8 +1337,8 @@ yuv2rgba64_1_c_template(SwsInternal *c, const int32_t *buf0,
>              Y2 += (1 << 13) - (1 << 29);
>  
>              if (hasAlpha) {
> -                A1 = abuf0[i * 2    ] * (1 << 11);
> -                A2 = abuf0[i * 2 + 1] * (1 << 11);
> +                A1 = av_clip_uintp2(abuf0[i * 2    ], 19) * (1 << 11);
> +                A2 = av_clip_uintp2(abuf0[i * 2 + 1], 19) * (1 << 11);
>  
>                  A1 += 1 << 13;
>                  A2 += 1 << 13;

A1/A2 are already cliped a few lines later
cliping them twice is not reasonable

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Freedom in capitalist society always remains about the same as it was in
ancient Greek republics: Freedom for slave owners. -- Vladimir Lenin

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 163 bytes --]

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org