[FFmpeg-devel] [PR] swscale/output: Fix some integer overflows in yuv2rgba64_full*() (PR #21743)

[michaelni via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>]

PR #21743 opened by michaelni
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21743
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21743.patch

Fixes: integer overflows
Fixes: 471587361/clusterfuzz-testcase-minimized-ffmpeg_SWS_fuzzer-5015347829997568

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>


>From 7c9a6ffb976506bae8298e85e6369b7f2d10791e Mon Sep 17 00:00:00 2001
From: Michael Niedermayer <michael@niedermayer.cc>
Date: Fri, 13 Feb 2026 00:16:24 +0100
Subject: [PATCH] swscale/output: Fix some integer overflows in
 yuv2rgba64_full*()

Fixes: integer overflows
Fixes: 471587361/clusterfuzz-testcase-minimized-ffmpeg_SWS_fuzzer-5015347829997568

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libswscale/output.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/libswscale/output.c b/libswscale/output.c
index a3b8da54bf..94454860c3 100644
--- a/libswscale/output.c
+++ b/libswscale/output.c
@@ -1416,9 +1416,9 @@ yuv2rgba64_full_X_c_template(SwsInternal *c, const int16_t *lumFilter,
         Y += (1 << 13) - (1<<29); // 21
         // 8bit: 17 + 13bit = 30bit, 16bit: 17 + 13bit = 30bit
 
-        R = V * c->yuv2rgb_v2r_coeff;
-        G = V * c->yuv2rgb_v2g_coeff + U * c->yuv2rgb_u2g_coeff;
-        B =                            U * c->yuv2rgb_u2b_coeff;
+        R = (unsigned)V * c->yuv2rgb_v2r_coeff;
+        G = (unsigned)V * c->yuv2rgb_v2g_coeff + (unsigned)U * c->yuv2rgb_u2g_coeff;
+        B =                                      (unsigned)U * c->yuv2rgb_u2b_coeff;
 
         // 8bit: 30 - 22 = 8bit, 16bit: 30bit - 14 = 16bit
         output_pixel(&dest[0], av_clip_uintp2(((int)(R_B + (unsigned)Y)>>14) + (1<<15), 16));
@@ -1457,9 +1457,9 @@ yuv2rgba64_full_2_c_template(SwsInternal *c, const int32_t *buf[2],
     av_assert2(uvalpha <= 4096U);
 
     for (i = 0; i < dstW; i++) {
-        int Y  = (int)(buf0[i]     * yalpha1  + buf1[i]     * yalpha) >> 14;
-        int U  = (int)(ubuf0[i]   * uvalpha1 + ubuf1[i]     * uvalpha - (128 << 23)) >> 14;
-        int V  = (int)(vbuf0[i]   * uvalpha1 + vbuf1[i]     * uvalpha - (128 << 23)) >> 14;
+        unsigned Y  = (int)(buf0[i]     * yalpha1  + buf1[i]     * yalpha) >> 14;
+        unsigned U  = (int)(ubuf0[i]   * uvalpha1 + ubuf1[i]     * uvalpha - (128 << 23)) >> 14;
+        unsigned V  = (int)(vbuf0[i]   * uvalpha1 + vbuf1[i]     * uvalpha - (128 << 23)) >> 14;
         int R, G, B;
 
         Y -= c->yuv2rgb_y_offset;
@@ -1476,9 +1476,9 @@ yuv2rgba64_full_2_c_template(SwsInternal *c, const int32_t *buf[2],
             A += 1 << 13;
         }
 
-        output_pixel(&dest[0], av_clip_uintp2(((R_B + Y) >> 14) + (1<<15), 16));
-        output_pixel(&dest[1], av_clip_uintp2(((  G + Y) >> 14) + (1<<15), 16));
-        output_pixel(&dest[2], av_clip_uintp2(((B_R + Y) >> 14) + (1<<15), 16));
+        output_pixel(&dest[0], av_clip_uintp2(((int)(R_B + Y) >> 14) + (1<<15), 16));
+        output_pixel(&dest[1], av_clip_uintp2(((int)(  G + Y) >> 14) + (1<<15), 16));
+        output_pixel(&dest[2], av_clip_uintp2(((int)(B_R + Y) >> 14) + (1<<15), 16));
         if (eightbytes) {
             output_pixel(&dest[3], av_clip_uintp2(A, 30) >> 14);
             dest += 4;
-- 
2.52.0

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org