* [FFmpeg-devel] [PATCH] libswscale/output: fix signed integer overflow in yuv2rgba64
@ 2026-02-11 9:00 Hankang Li via ffmpeg-devel
2026-02-13 2:17 ` [FFmpeg-devel] " Michael Niedermayer via ffmpeg-devel
0 siblings, 1 reply; 2+ messages in thread
From: Hankang Li via ffmpeg-devel @ 2026-02-11 9:00 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Hankang Li
Clamp the intermediate alpha value within 19 bits, this prevents
overflow.
Note: This issue is only reproducible with --disable-x86asm. There is an
existing issue in fate-sws-unscaled with --disable-x86asm (ticket
#21113). This change does not introduce any new regressions.
Signed-off-by: Hankang Li <hankang201222@gmail.com>
---
libswscale/output.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/libswscale/output.c b/libswscale/output.c
index a3b8da54bf..afb3878170 100644
--- a/libswscale/output.c
+++ b/libswscale/output.c
@@ -1288,8 +1288,8 @@ yuv2rgba64_1_c_template(SwsInternal *c, const int32_t *buf0,
Y2 += (1 << 13) - (1 << 29);
if (hasAlpha) {
- A1 = abuf0[i * 2 ] * (1 << 11);
- A2 = abuf0[i * 2 + 1] * (1 << 11);
+ A1 = av_clip_uintp2(abuf0[i * 2 ], 19) * (1 << 11);
+ A2 = av_clip_uintp2(abuf0[i * 2 + 1], 19) * (1 << 11);
A1 += 1 << 13;
A2 += 1 << 13;
@@ -1337,8 +1337,8 @@ yuv2rgba64_1_c_template(SwsInternal *c, const int32_t *buf0,
Y2 += (1 << 13) - (1 << 29);
if (hasAlpha) {
- A1 = abuf0[i * 2 ] * (1 << 11);
- A2 = abuf0[i * 2 + 1] * (1 << 11);
+ A1 = av_clip_uintp2(abuf0[i * 2 ], 19) * (1 << 11);
+ A2 = av_clip_uintp2(abuf0[i * 2 + 1], 19) * (1 << 11);
A1 += 1 << 13;
A2 += 1 << 13;
--
2.52.0
_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org
^ permalink raw reply [flat|nested] 2+ messages in thread
* [FFmpeg-devel] Re: [PATCH] libswscale/output: fix signed integer overflow in yuv2rgba64
2026-02-11 9:00 [FFmpeg-devel] [PATCH] libswscale/output: fix signed integer overflow in yuv2rgba64 Hankang Li via ffmpeg-devel
@ 2026-02-13 2:17 ` Michael Niedermayer via ffmpeg-devel
0 siblings, 0 replies; 2+ messages in thread
From: Michael Niedermayer via ffmpeg-devel @ 2026-02-13 2:17 UTC (permalink / raw)
To: FFmpeg development discussions and patches; +Cc: Michael Niedermayer
[-- Attachment #1.1: Type: text/plain, Size: 2034 bytes --]
Hi Hankang Li
On Wed, Feb 11, 2026 at 01:00:42AM -0800, Hankang Li via ffmpeg-devel wrote:
> Clamp the intermediate alpha value within 19 bits, this prevents
> overflow.
>
> Note: This issue is only reproducible with --disable-x86asm. There is an
> existing issue in fate-sws-unscaled with --disable-x86asm (ticket
> #21113). This change does not introduce any new regressions.
>
> Signed-off-by: Hankang Li <hankang201222@gmail.com>
> ---
> libswscale/output.c | 8 ++++----
> 1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/libswscale/output.c b/libswscale/output.c
> index a3b8da54bf..afb3878170 100644
> --- a/libswscale/output.c
> +++ b/libswscale/output.c
> @@ -1288,8 +1288,8 @@ yuv2rgba64_1_c_template(SwsInternal *c, const int32_t *buf0,
> Y2 += (1 << 13) - (1 << 29);
>
> if (hasAlpha) {
> - A1 = abuf0[i * 2 ] * (1 << 11);
> - A2 = abuf0[i * 2 + 1] * (1 << 11);
> + A1 = av_clip_uintp2(abuf0[i * 2 ], 19) * (1 << 11);
> + A2 = av_clip_uintp2(abuf0[i * 2 + 1], 19) * (1 << 11);
>
> A1 += 1 << 13;
> A2 += 1 << 13;
> @@ -1337,8 +1337,8 @@ yuv2rgba64_1_c_template(SwsInternal *c, const int32_t *buf0,
> Y2 += (1 << 13) - (1 << 29);
>
> if (hasAlpha) {
> - A1 = abuf0[i * 2 ] * (1 << 11);
> - A2 = abuf0[i * 2 + 1] * (1 << 11);
> + A1 = av_clip_uintp2(abuf0[i * 2 ], 19) * (1 << 11);
> + A2 = av_clip_uintp2(abuf0[i * 2 + 1], 19) * (1 << 11);
>
> A1 += 1 << 13;
> A2 += 1 << 13;
A1/A2 are already cliped a few lines later
cliping them twice is not reasonable
thx
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
Freedom in capitalist society always remains about the same as it was in
ancient Greek republics: Freedom for slave owners. -- Vladimir Lenin
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]
[-- Attachment #2: Type: text/plain, Size: 163 bytes --]
_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2026-02-13 2:18 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-02-11 9:00 [FFmpeg-devel] [PATCH] libswscale/output: fix signed integer overflow in yuv2rgba64 Hankang Li via ffmpeg-devel
2026-02-13 2:17 ` [FFmpeg-devel] " Michael Niedermayer via ffmpeg-devel
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
This inbox may be cloned and mirrored by anyone:
git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git
# If you have public-inbox 1.1+ installed, you may
# initialize and index your mirror using the following commands:
public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
ffmpegdev@gitmailbox.com
public-inbox-index ffmpegdev
Example config snippet for mirrors.
AGPL code for this site: git clone https://public-inbox.org/public-inbox.git