[FFmpeg-devel] [PATCH] lavc/vvc: Avoid overflow in coeff scale intermediate

[FFmpeg-devel] [PATCH] lavc/vvc: Avoid overflow in coeff scale intermediate

[Frank Plowman]

Make intermediate result 64-bits to avoid an overflow before the right
shift.

Signed-off-by: Frank Plowman <post@frankplowman.com>
---
 libavcodec/vvc/intra.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/vvc/intra.c b/libavcodec/vvc/intra.c
index e515fb9710..5ac7d02c80 100644
--- a/libavcodec/vvc/intra.c
+++ b/libavcodec/vvc/intra.c
@@ -416,7 +416,7 @@ static const uint8_t* derive_scale_m(const VVCLocalContext *lc, const TransformB
 static av_always_inline int scale_coeff(const TransformBlock *tb, int coeff,
     const int scale, const int scale_m, const int log2_transform_range)
 {
-    coeff = (coeff * scale * scale_m + tb->bd_offset) >> tb->bd_shift;
+    coeff = ((int64_t) coeff * scale * scale_m + tb->bd_offset) >> tb->bd_shift;
     coeff = av_clip_intp2(coeff, log2_transform_range);
     return coeff;
 }
-- 
2.44.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".