[FFmpeg-devel] [PATCH] lavc/vvc: Clamp shift RHS

[FFmpeg-devel] [PATCH] lavc/vvc: Clamp shift RHS

[post]

From: Frank Plowman <post@frankplowman.com>

Resolves the following undefined behavior sanitiser error:
runtime error: shift exponent 32 is too large for 32-bit type 'int'

Signed-off-by: Frank Plowman <post@frankplowman.com>
---
 libavcodec/vvc/vvc_intra_template.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/vvc/vvc_intra_template.c b/libavcodec/vvc/vvc_intra_template.c
index 9fb47549d5..a078885266 100644
--- a/libavcodec/vvc/vvc_intra_template.c
+++ b/libavcodec/vvc/vvc_intra_template.c
@@ -969,7 +969,7 @@ static void FUNC(pred_angular_h)(uint8_t *_src, const uint8_t *_top, const uint8
         int pos = (1 + ref_idx) * intra_pred_angle;
         int wt;
         if (need_pdpc)
-            wt = (32 >> ((y * 2) >> nscale));
+            wt = (32 >> FFMIN(31, (y * 2) >> nscale));
 
         for (int x = 0; x < w; x++) {
             const int idx  = (pos >> 5) + ref_idx;
-- 
2.43.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] lavc/vvc: Clamp shift RHS

[James Almer]

On 1/26/2024 6:13 AM, post@frankplowman.com wrote:
> From: Frank Plowman <post@frankplowman.com>
> 
> Resolves the following undefined behavior sanitiser error:
> runtime error: shift exponent 32 is too large for 32-bit type 'int'
> 
> Signed-off-by: Frank Plowman <post@frankplowman.com>
> ---
>   libavcodec/vvc/vvc_intra_template.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavcodec/vvc/vvc_intra_template.c b/libavcodec/vvc/vvc_intra_template.c
> index 9fb47549d5..a078885266 100644
> --- a/libavcodec/vvc/vvc_intra_template.c
> +++ b/libavcodec/vvc/vvc_intra_template.c
> @@ -969,7 +969,7 @@ static void FUNC(pred_angular_h)(uint8_t *_src, const uint8_t *_top, const uint8
>           int pos = (1 + ref_idx) * intra_pred_angle;
>           int wt;
>           if (need_pdpc)
> -            wt = (32 >> ((y * 2) >> nscale));
> +            wt = (32 >> FFMIN(31, (y * 2) >> nscale));
>   
>           for (int x = 0; x < w; x++) {
>               const int idx  = (pos >> 5) + ref_idx;

Applied, thanks.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".