[FFmpeg-devel] [PATCH] checkasm/h264dsp: use int64_t scale values

[FFmpeg-devel] [PATCH] checkasm/h264dsp: use int64_t scale values

[James Almer]

Fixes "signed integer overflow: [varies] * 104858 cannot be represented in type 'int'" errors
under ubsan.

Signed-off-by: James Almer <jamrial@gmail.com>
---
 tests/checkasm/h264dsp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/checkasm/h264dsp.c b/tests/checkasm/h264dsp.c
index cb180cc44f..0cc1f32740 100644
--- a/tests/checkasm/h264dsp.c
+++ b/tests/checkasm/h264dsp.c
@@ -83,7 +83,7 @@ static void dct4x4_##size(dctcoef *coef)                                     \
     }                                                                        \
     for (y = 0; y < 4; y++) {                                                \
         for (x = 0; x < 4; x++) {                                            \
-            static const int scale[] = { 13107 * 10, 8066 * 13, 5243 * 16 }; \
+            const int64_t scale[] = { 13107 * 10, 8066 * 13, 5243 * 16 };    \
             const int idx = (y & 1) + (x & 1);                               \
             coef[y*4 + x] = (coef[y*4 + x] * scale[idx] + (1 << 14)) >> 15;  \
         }                                                                    \
-- 
2.45.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".