From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 54EFB44CEF for ; Tue, 15 Nov 2022 22:17:16 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4A10368BD8D; Wed, 16 Nov 2022 00:17:07 +0200 (EET) Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [217.70.183.196]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5D06968BBD9 for ; Wed, 16 Nov 2022 00:17:00 +0200 (EET) Received: (Authenticated sender: michael@niedermayer.cc) by mail.gandi.net (Postfix) with ESMTPSA id 87BA5E0003 for ; Tue, 15 Nov 2022 22:16:59 +0000 (UTC) From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Tue, 15 Nov 2022 23:16:57 +0100 Message-Id: <20221115221657.23290-2-michael@niedermayer.cc> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221115221657.23290-1-michael@niedermayer.cc> References: <20221115221657.23290-1-michael@niedermayer.cc> Subject: [FFmpeg-devel] [PATCH 2/2] swscale/input: Use more unsigned intermediates X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Same principle as previous commit, with sufficiently huge rgb2yuv table values this produces wrong results and undefined behavior. The unsigned produces the same incorrect results. That is probably ok as these cases with huge values seem not to occur in any real use case. There are more cases but someone is refactoring them, so i didnt yet change them. Fixes: signed integer overflow Signed-off-by: Michael Niedermayer --- libswscale/input.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/libswscale/input.c b/libswscale/input.c index d7dbedd82f..d5676062a2 100644 --- a/libswscale/input.c +++ b/libswscale/input.c @@ -84,9 +84,9 @@ rgb64ToUV_half_c_template(uint16_t *dstU, uint16_t *dstV, int32_t rv = rgb2yuv[RV_IDX], gv = rgb2yuv[GV_IDX], bv = rgb2yuv[BV_IDX]; av_assert1(src1==src2); for (i = 0; i < width; i++) { - int r_b = (input_pixel(&src1[8 * i + 0]) + input_pixel(&src1[8 * i + 4]) + 1) >> 1; - int g = (input_pixel(&src1[8 * i + 1]) + input_pixel(&src1[8 * i + 5]) + 1) >> 1; - int b_r = (input_pixel(&src1[8 * i + 2]) + input_pixel(&src1[8 * i + 6]) + 1) >> 1; + unsigned r_b = (input_pixel(&src1[8 * i + 0]) + input_pixel(&src1[8 * i + 4]) + 1) >> 1; + unsigned g = (input_pixel(&src1[8 * i + 1]) + input_pixel(&src1[8 * i + 5]) + 1) >> 1; + unsigned b_r = (input_pixel(&src1[8 * i + 2]) + input_pixel(&src1[8 * i + 6]) + 1) >> 1; dstU[i]= (ru*r + gu*g + bu*b + (0x10001<<(RGB2YUV_SHIFT-1))) >> RGB2YUV_SHIFT; dstV[i]= (rv*r + gv*g + bv*b + (0x10001<<(RGB2YUV_SHIFT-1))) >> RGB2YUV_SHIFT; @@ -158,9 +158,9 @@ static av_always_inline void rgb48ToUV_c_template(uint16_t *dstU, int32_t rv = rgb2yuv[RV_IDX], gv = rgb2yuv[GV_IDX], bv = rgb2yuv[BV_IDX]; av_assert1(src1 == src2); for (i = 0; i < width; i++) { - int r_b = input_pixel(&src1[i * 3 + 0]); - int g = input_pixel(&src1[i * 3 + 1]); - int b_r = input_pixel(&src1[i * 3 + 2]); + unsigned r_b = input_pixel(&src1[i * 3 + 0]); + unsigned g = input_pixel(&src1[i * 3 + 1]); + unsigned b_r = input_pixel(&src1[i * 3 + 2]); dstU[i] = (ru*r + gu*g + bu*b + (0x10001 << (RGB2YUV_SHIFT - 1))) >> RGB2YUV_SHIFT; dstV[i] = (rv*r + gv*g + bv*b + (0x10001 << (RGB2YUV_SHIFT - 1))) >> RGB2YUV_SHIFT; @@ -180,12 +180,12 @@ static av_always_inline void rgb48ToUV_half_c_template(uint16_t *dstU, int32_t rv = rgb2yuv[RV_IDX], gv = rgb2yuv[GV_IDX], bv = rgb2yuv[BV_IDX]; av_assert1(src1 == src2); for (i = 0; i < width; i++) { - int r_b = (input_pixel(&src1[6 * i + 0]) + - input_pixel(&src1[6 * i + 3]) + 1) >> 1; - int g = (input_pixel(&src1[6 * i + 1]) + - input_pixel(&src1[6 * i + 4]) + 1) >> 1; - int b_r = (input_pixel(&src1[6 * i + 2]) + - input_pixel(&src1[6 * i + 5]) + 1) >> 1; + unsigned r_b = (input_pixel(&src1[6 * i + 0]) + + input_pixel(&src1[6 * i + 3]) + 1) >> 1; + unsigned g = (input_pixel(&src1[6 * i + 1]) + + input_pixel(&src1[6 * i + 4]) + 1) >> 1; + unsigned b_r = (input_pixel(&src1[6 * i + 2]) + + input_pixel(&src1[6 * i + 5]) + 1) >> 1; dstU[i] = (ru*r + gu*g + bu*b + (0x10001 << (RGB2YUV_SHIFT - 1))) >> RGB2YUV_SHIFT; dstV[i] = (rv*r + gv*g + bv*b + (0x10001 << (RGB2YUV_SHIFT - 1))) >> RGB2YUV_SHIFT; -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".