From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.ffmpeg.org (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id C1A694C468 for ; Mon, 13 Oct 2025 13:28:06 +0000 (UTC) Authentication-Results: ffbox; dkim=fail (body hash mismatch (got b'HbiosXp0MBGPXcem/80ED8DrAQhDrm4u3E0XT3BJB7U=', expected b'jycyr5cJqKZ3Xg+ztalfRjHx/JAnv4Cbjb/55LNXpEs=')) header.d=ffmpeg.org header.i=@ffmpeg.org header.a=rsa-sha256 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ffmpeg.org; i=@ffmpeg.org; q=dns/txt; s=mail; t=1760362073; h=mime-version : to : date : message-id : reply-to : subject : list-id : list-archive : list-archive : list-help : list-owner : list-post : list-subscribe : list-unsubscribe : from : cc : content-type : content-transfer-encoding : from; bh=HbiosXp0MBGPXcem/80ED8DrAQhDrm4u3E0XT3BJB7U=; b=R8LoArgSFsRdUG5Lo0JWLhZQk0XbTn8EpdFmJmEPRuKKh9uSGYEmtSQxngnlRpslFoxCt 0EsR4AyvcsZOECa84i5zIBiLfZewTYt5KL5Cf4cENj2xyAJS6/yecX8GYQUmlOKSFAM5C0h B3tnQcMGZ7JZONKcwiwH8ikxcOMXtRzv27hHmZMindERJopYn8IifwwCwOu845XCexdt5bY waqe00+j2G1tscaltvP8a35txCnNNg2aOsKL/8NT5y24WjyaePTHjsSeBgTWksaTnrLbNwE WMatKoVoL3Zh5MnGVJvEg4YlSO81/9PplJnxBl2f9aBqbQsA5R9zIWfcG72g== Received: from [172.19.0.4] (unknown [172.19.0.4]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTP id 030E668F208; Mon, 13 Oct 2025 16:27:53 +0300 (EEST) ARC-Seal: i=1; cv=none; a=rsa-sha256; d=ffmpeg.org; s=arc; t=1760362070; b=rAXnoxXSkXtJytmhugd89ZwTRGORmC1LjxVYn0k+NEl1AFeQdUfaXrO8/cWQ8jqhGnvXK ZGpp4t0lHYB9Fnpdr/3vrKLiX8ws75Pt3E9509Py1dnxlNWnCeZ2KqrAReLqo7GhmqQsspS NyhSLjEYx/Mocy0pZP/WVZ+qsGMqTUPeu9hJHVJbCdywNWgOqiOSDlYQAZDj1UFTTjzVGI/ PJpZsgcxgaxlULpq1JqBkXAW1B5BVAc4jrzha4m59I8Ip3hWFAftMhHyCBMdAyyCsZ3h9mU wBnUX9je47IlfPhYBYzP/iCcJAmeRZGGOGNdWXSbapt41GYafleClsJdV3Cw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=ffmpeg.org; s=arc; t=1760362070; h=from : sender : reply-to : subject : date : message-id : to : cc : mime-version : content-type : content-transfer-encoding : content-id : content-description : resent-date : resent-from : resent-sender : resent-to : resent-cc : resent-message-id : in-reply-to : references : list-id : list-help : list-unsubscribe : list-subscribe : list-post : list-owner : list-archive; bh=EX6kjpzWCKPyhiVzP+JfL8jlCVhQMhrg6m/5M5dKGL0=; b=DefBXcWHa7wIfnd2KkEJJu+AFlzybgMgQ3BcVaVpNpOcr0IUXDeF8XyKUxdCZvi67XmTC lFSxZxkReHtg5X2XCLbcIrndnl3sLgT52mFKytQBrO1Z+lp85caKcm7R6dhQLQ0aR+26KKZ MmsQVUwHTiCuJj1Xl1olUcmu4MRqyPP56aZv8SpnYgWMDnTNqv3+myziz7W3LNc8hl8hIgv XccJ643Zf6XdFhzniVnhF785VR3ilzx+iW0Wf9sLoyWHuzwn8lnIcWXWnk6NqFvIK6PZcyK ssYtA1vb2N+Fe5d3Uq7dK3qleycmZxifx81x+GNNFdtmC7TumblelXO4jORA== ARC-Authentication-Results: i=1; ffmpeg.org; dkim=pass header.d=ffmpeg.org header.i=@ffmpeg.org; arc=none; dmarc=pass header.from=ffmpeg.org policy.dmarc=quarantine Authentication-Results: ffmpeg.org; dkim=pass header.d=ffmpeg.org header.i=@ffmpeg.org; arc=none (Message is not ARC signed); dmarc=pass (Used From Domain Record) header.from=ffmpeg.org policy.dmarc=quarantine DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ffmpeg.org; i=@ffmpeg.org; q=dns/txt; s=mail; t=1760362064; h=content-type : mime-version : content-transfer-encoding : from : to : reply-to : subject : date : from; bh=jycyr5cJqKZ3Xg+ztalfRjHx/JAnv4Cbjb/55LNXpEs=; b=SNSljOfMunl4iX8ETgVTiuBSoZ5h1w1tJPvrGi/WNLmYnsVw+GjHHhCCkwv3a8l5QxrUp SqDjS542Zgwhgq0zp4B6zEU8u8ZrP+v26u8knVAGQHZcJfqE2dQrMG1ClR7RuPI4o+jDzBA uvSjUKCxj4QMNtRnl1LjuvmDSEDXRPCdxcHYUfo/inwjq8JqcyMCVRF+lvuqMcvA1qVCx4S qG3TXbOKu4NFmjypU8Rgj0bmDHXpIcca4fdE3J9XdtLKNm0LrYG7zJEVuP82TwxgOLMikOr unC5sJLIV++ac1EJKBglNm1gacP1NqP8Fo7VtW1FxNy6/u7MVuQpMYaPMEAg== Received: from be50bb5a3685 (code.ffmpeg.org [188.245.149.3]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTPS id E589D68F1D0 for ; Mon, 13 Oct 2025 16:27:43 +0300 (EEST) MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Date: Mon, 13 Oct 2025 13:27:43 -0000 Message-ID: <176036206413.25.15905318861432080946@bf907ddaa564> Message-ID-Hash: GPPREOLELOKB32CRYGYITXJPOESWAVMD X-Message-ID-Hash: GPPREOLELOKB32CRYGYITXJPOESWAVMD X-MailFrom: code@ffmpeg.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; header-match-ffmpeg-devel.ffmpeg.org-0; header-match-ffmpeg-devel.ffmpeg.org-1; header-match-ffmpeg-devel.ffmpeg.org-2; header-match-ffmpeg-devel.ffmpeg.org-3; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list Reply-To: FFmpeg development discussions and patches Subject: [FFmpeg-devel] [PATCH] swscale/output: Fix unsigned cast position in yuv2* (PR #20698) List-Id: FFmpeg development discussions and patches Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: michaelni via ffmpeg-devel Cc: michaelni Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Archived-At: List-Archive: List-Post: PR #20698 opened by michaelni URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20698 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20698.patch >>From 93794583405909cfe8e7abffdd84c2454959ff18 Mon Sep 17 00:00:00 2001 From: Michael Niedermayer Date: Mon, 13 Oct 2025 14:32:45 +0200 Subject: [PATCH 1/2] swscale/output: Fix integer overflow in yuv2ya16_X_c_template() Found-by: colod colod Signed-off-by: Michael Niedermayer --- libswscale/output.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libswscale/output.c b/libswscale/output.c index b873276c00..cb6630bd73 100644 --- a/libswscale/output.c +++ b/libswscale/output.c @@ -1028,7 +1028,7 @@ yuv2ya16_X_c_template(SwsInternal *c, const int16_t *lumFilter, int A = 0xffff; for (j = 0; j < lumFilterSize; j++) - Y += (unsigned)(lumSrc[j][i] * lumFilter[j]); + Y += lumSrc[j][i] * (unsigned)lumFilter[j]; Y >>= 15; Y += (1<<3) + 0x8000; @@ -1037,7 +1037,7 @@ yuv2ya16_X_c_template(SwsInternal *c, const int16_t *lumFilter, if (hasAlpha) { A = -0x40000000 + (1<<14); for (j = 0; j < lumFilterSize; j++) - A += (unsigned)(alpSrc[j][i] * lumFilter[j]); + A += alpSrc[j][i] * (unsigned)lumFilter[j]; A >>= 15; A += 0x8000; -- 2.49.1 >>From 9d0761453e5ecaa1344d63e2fcc9bd3cc6d4d0f6 Mon Sep 17 00:00:00 2001 From: Michael Niedermayer Date: Mon, 13 Oct 2025 14:46:16 +0200 Subject: [PATCH 2/2] swscale/output: Fix unsigned cast position in yuv2* Fixes: signed overflow Signed-off-by: Michael Niedermayer --- libswscale/output.c | 90 ++++++++++++++++++++++----------------------- 1 file changed, 45 insertions(+), 45 deletions(-) diff --git a/libswscale/output.c b/libswscale/output.c index cb6630bd73..877db04f36 100644 --- a/libswscale/output.c +++ b/libswscale/output.c @@ -503,8 +503,8 @@ static void yuv2nv12cX_c(enum AVPixelFormat dstFormat, const uint8_t *chrDither, int v = chrDither[(i + 3) & 7] << 12; int j; for (j=0; j>19); @@ -516,8 +516,8 @@ static void yuv2nv12cX_c(enum AVPixelFormat dstFormat, const uint8_t *chrDither, int v = chrDither[(i + 3) & 7] << 12; int j; for (j=0; j>19); @@ -577,8 +577,8 @@ static void yuv2p01xcX_c(int big_endian, const uint8_t *chrDither, int v = 1 << (shift - 1); for (j = 0; j < chrFilterSize; j++) { - u += (unsigned)(chrUSrc[j][i] * chrFilter[j]); - v += (unsigned)(chrVSrc[j][i] * chrFilter[j]); + u += chrUSrc[j][i] * (unsigned)chrFilter[j]; + v += chrVSrc[j][i] * (unsigned)chrFilter[j]; } output_pixel(&dest[2*i] , u); @@ -678,8 +678,8 @@ yuv2mono_X_c_template(SwsInternal *c, const int16_t *lumFilter, int Y2 = 1 << 18; for (j = 0; j < lumFilterSize; j++) { - Y1 += (unsigned)(lumSrc[j][i] * lumFilter[j]); - Y2 += (unsigned)(lumSrc[j][i+1] * lumFilter[j]); + Y1 += lumSrc[j][i] * (unsigned)lumFilter[j]; + Y2 += lumSrc[j][i+1] * (unsigned)lumFilter[j]; } Y1 >>= 19; Y2 >>= 19; @@ -896,12 +896,12 @@ yuv2422_X_c_template(SwsInternal *c, const int16_t *lumFilter, int V = 1 << 18; for (j = 0; j < lumFilterSize; j++) { - Y1 += (unsigned)(lumSrc[j][i * 2] * lumFilter[j]); - Y2 += (unsigned)(lumSrc[j][i * 2 + 1] * lumFilter[j]); + Y1 += lumSrc[j][i * 2] * (unsigned)lumFilter[j]; + Y2 += lumSrc[j][i * 2 + 1] * (unsigned)lumFilter[j]; } for (j = 0; j < chrFilterSize; j++) { - U += (unsigned)(chrUSrc[j][i] * chrFilter[j]); - V += (unsigned)(chrVSrc[j][i] * chrFilter[j]); + U += chrUSrc[j][i] * (unsigned)chrFilter[j]; + V += chrVSrc[j][i] * (unsigned)chrFilter[j]; } Y1 >>= 19; Y2 >>= 19; @@ -1802,12 +1802,12 @@ yuv2rgb_X_c_template(SwsInternal *c, const int16_t *lumFilter, const void *r, *g, *b; for (j = 0; j < lumFilterSize; j++) { - Y1 += (unsigned)(lumSrc[j][i * 2] * lumFilter[j]); - Y2 += (unsigned)(lumSrc[j][i * 2 + 1] * lumFilter[j]); + Y1 += lumSrc[j][i * 2] * (unsigned)lumFilter[j]; + Y2 += lumSrc[j][i * 2 + 1] * (unsigned)lumFilter[j]; } for (j = 0; j < chrFilterSize; j++) { - U += (unsigned)(chrUSrc[j][i] * chrFilter[j]); - V += (unsigned)(chrVSrc[j][i] * chrFilter[j]); + U += chrUSrc[j][i] * (unsigned)chrFilter[j]; + V += chrVSrc[j][i] * (unsigned)chrFilter[j]; } Y1 >>= 19; Y2 >>= 19; @@ -1817,8 +1817,8 @@ yuv2rgb_X_c_template(SwsInternal *c, const int16_t *lumFilter, A1 = 1 << 18; A2 = 1 << 18; for (j = 0; j < lumFilterSize; j++) { - A1 += (unsigned)(alpSrc[j][i * 2 ] * lumFilter[j]); - A2 += (unsigned)(alpSrc[j][i * 2 + 1] * lumFilter[j]); + A1 += alpSrc[j][i * 2 ] * (unsigned)lumFilter[j]; + A2 += alpSrc[j][i * 2 + 1] * (unsigned)lumFilter[j]; } A1 >>= 19; A2 >>= 19; @@ -2179,11 +2179,11 @@ yuv2rgb_full_X_c_template(SwsInternal *c, const int16_t *lumFilter, int V = (1<<9)-(128 << 19); for (j = 0; j < lumFilterSize; j++) { - Y += (unsigned)(lumSrc[j][i] * lumFilter[j]); + Y += lumSrc[j][i] * (unsigned)lumFilter[j]; } for (j = 0; j < chrFilterSize; j++) { - U += (unsigned)(chrUSrc[j][i] * chrFilter[j]); - V += (unsigned)(chrVSrc[j][i] * chrFilter[j]); + U += chrUSrc[j][i] * (unsigned)chrFilter[j]; + V += chrVSrc[j][i] * (unsigned)chrFilter[j]; } Y >>= 10; U >>= 10; @@ -2191,7 +2191,7 @@ yuv2rgb_full_X_c_template(SwsInternal *c, const int16_t *lumFilter, if (hasAlpha) { A = 1 << 18; for (j = 0; j < lumFilterSize; j++) { - A += (unsigned)(alpSrc[j][i] * lumFilter[j]); + A += alpSrc[j][i] * (unsigned)lumFilter[j]; } A >>= 19; if (A & 0x100) @@ -2360,11 +2360,11 @@ yuv2gbrp_full_X_c(SwsInternal *c, const int16_t *lumFilter, int R, G, B; for (j = 0; j < lumFilterSize; j++) - Y += (unsigned)(lumSrc[j][i] * lumFilter[j]); + Y += lumSrc[j][i] * (unsigned)lumFilter[j]; for (j = 0; j < chrFilterSize; j++) { - U += (unsigned)(chrUSrc[j][i] * chrFilter[j]); - V += (unsigned)(chrVSrc[j][i] * chrFilter[j]); + U += chrUSrc[j][i] * (unsigned)chrFilter[j]; + V += chrVSrc[j][i] * (unsigned)chrFilter[j]; } Y >>= 10; @@ -2375,7 +2375,7 @@ yuv2gbrp_full_X_c(SwsInternal *c, const int16_t *lumFilter, A = 1 << 18; for (j = 0; j < lumFilterSize; j++) - A += (unsigned)(alpSrc[j][i] * lumFilter[j]); + A += alpSrc[j][i] * (unsigned)lumFilter[j]; if (A & 0xF8000000) A = av_clip_uintp2(A, 27); @@ -2679,7 +2679,7 @@ yuv2ya8_X_c(SwsInternal *c, const int16_t *lumFilter, int Y = 1 << 18, A = 1 << 18; for (j = 0; j < lumFilterSize; j++) - Y += (unsigned)(lumSrc[j][i] * lumFilter[j]); + Y += lumSrc[j][i] * (unsigned)lumFilter[j]; Y >>= 19; if (Y & 0x100) @@ -2687,7 +2687,7 @@ yuv2ya8_X_c(SwsInternal *c, const int16_t *lumFilter, if (hasAlpha) { for (j = 0; j < lumFilterSize; j++) - A += (unsigned)(alpSrc[j][i] * lumFilter[j]); + A += alpSrc[j][i] * (unsigned)lumFilter[j]; A >>= 19; @@ -2793,11 +2793,11 @@ yuv2v30_X_c_template(SwsInternal *c, const int16_t *lumFilter, int j; for (j = 0; j < lumFilterSize; j++) - Y += (unsigned)(lumSrc[j][i] * lumFilter[j]); + Y += lumSrc[j][i] * (unsigned)lumFilter[j]; for (j = 0; j < chrFilterSize; j++) { - U += (unsigned)(chrUSrc[j][i] * chrFilter[j]); - V += (unsigned)(chrVSrc[j][i] * chrFilter[j]); + U += chrUSrc[j][i] * (unsigned)chrFilter[j]; + V += chrVSrc[j][i] * (unsigned)chrFilter[j]; } Y = av_clip_uintp2(Y >> 17, 10); @@ -2847,11 +2847,11 @@ yuv2xv36_X_c(SwsInternal *c, const int16_t *lumFilter, int j; for (j = 0; j < lumFilterSize; j++) - Y += (unsigned)(lumSrc[j][i] * lumFilter[j]); + Y += lumSrc[j][i] * (unsigned)lumFilter[j]; for (j = 0; j < chrFilterSize; j++) { - U += (unsigned)(chrUSrc[j][i] * chrFilter[j]); - V += (unsigned)(chrVSrc[j][i] * chrFilter[j]); + U += chrUSrc[j][i] * (unsigned)chrFilter[j]; + V += chrVSrc[j][i] * (unsigned)chrFilter[j]; } output_pixels(dest + 8 * i + 2, Y, 15, 12, 4) @@ -3011,13 +3011,13 @@ yuv2ayuv_X_c_template(SwsInternal *c, const int16_t *lumFilter, int V = 1 << 18, A = 255; for (j = 0; j < lumFilterSize; j++) - Y += (unsigned)(lumSrc[j][i] * lumFilter[j]); + Y += lumSrc[j][i] * (unsigned)lumFilter[j]; for (j = 0; j < chrFilterSize; j++) - U += (unsigned)(chrUSrc[j][i] * chrFilter[j]); + U += chrUSrc[j][i] * (unsigned)chrFilter[j]; for (j = 0; j < chrFilterSize; j++) - V += (unsigned)(chrVSrc[j][i] * chrFilter[j]); + V += chrVSrc[j][i] * (unsigned)chrFilter[j]; Y >>= 19; U >>= 19; @@ -3034,7 +3034,7 @@ yuv2ayuv_X_c_template(SwsInternal *c, const int16_t *lumFilter, A = 1 << 18; for (j = 0; j < lumFilterSize; j++) - A += (unsigned)(alpSrc[j][i] * lumFilter[j]); + A += alpSrc[j][i] * (unsigned)lumFilter[j]; A >>= 19; @@ -3105,13 +3105,13 @@ AYUVPACKEDWRAPPER(uyva, AV_PIX_FMT_UYVA) int U = 1 << (shift - 1), V = 1 << (shift - 1); \ \ for (j = 0; j < lumFilterSize; j++) { \ - Y1 += (unsigned)(lumSrc[j][i * 2] * lumFilter[j]); \ - Y2 += (unsigned)(lumSrc[j][i * 2 + 1] * lumFilter[j]); \ + Y1 += lumSrc[j][i * 2] * (unsigned)lumFilter[j]; \ + Y2 += lumSrc[j][i * 2 + 1] * (unsigned)lumFilter[j]; \ } \ \ for (j = 0; j < chrFilterSize; j++) { \ - U += (unsigned)(chrUSrc[j][i] * chrFilter[j]); \ - V += (unsigned)(chrVSrc[j][i] * chrFilter[j]); \ + U += chrUSrc[j][i] * (unsigned)chrFilter[j]; \ + V += chrVSrc[j][i] * (unsigned)chrFilter[j]; \ } \ \ output_pixel(dest + 8 * i + 0, Y1, bits); \ @@ -3259,13 +3259,13 @@ yuv2vyu444_X_c(SwsInternal *c, const int16_t *lumFilter, int V = 1 << 18; for (j = 0; j < lumFilterSize; j++) - Y += (unsigned)(lumSrc[j][i] * lumFilter[j]); + Y += lumSrc[j][i] * (unsigned)lumFilter[j]; for (j = 0; j < chrFilterSize; j++) - U += (unsigned)(chrUSrc[j][i] * chrFilter[j]); + U += chrUSrc[j][i] * (unsigned)chrFilter[j]; for (j = 0; j < chrFilterSize; j++) - V += (unsigned)(chrVSrc[j][i] * chrFilter[j]); + V += chrVSrc[j][i] * (unsigned)chrFilter[j]; Y >>= 19; U >>= 19; -- 2.49.1 _______________________________________________ ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org