[FFmpeg-devel] [PATCH] lavc/dovi_rpu: Fix UB for possible left shift of negative values

[Thilo Borgmann <thilo.borgmann@mail.de>]

[-- Attachment #1: Type: text/plain, Size: 134 bytes --]

Hi,

avoid left-shifting possibly negative values in case of RPU_COEFF_FLOAT. Changed RPU_COEFF_FIXED as well for consistency.

-Thilo

[-- Attachment #2: 0001-lavc-dovi_rpu-Fix-UB-for-possible-left-shift-of-nega.patch --]
[-- Type: text/plain, Size: 1528 bytes --]

From 7eca2659b588c404f058eccef478c889555957fd Mon Sep 17 00:00:00 2001
From: Michael Goulet <mgoulet@fb.com>
Date: Thu, 16 Jun 2022 10:25:28 +0200
Subject: [PATCH] lavc/dovi_rpu: Fix UB for possible left shift of negative
 values

It is undefined to left-shift a negative value in the RPU_COEFF_FLOAT case.
Changed the RPU_COEFF_FIXED case as well for consistency although its all positive values there.
---
 libavcodec/dovi_rpu.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavcodec/dovi_rpu.c b/libavcodec/dovi_rpu.c
index a87562c8a3..833ce9e705 100644
--- a/libavcodec/dovi_rpu.c
+++ b/libavcodec/dovi_rpu.c
@@ -153,7 +153,7 @@ static inline uint64_t get_ue_coef(GetBitContext *gb, const AVDOVIRpuDataHeader
     case RPU_COEFF_FIXED:
         ipart = get_ue_golomb_long(gb);
         fpart.u32 = get_bits_long(gb, hdr->coef_log2_denom);
-        return (ipart << hdr->coef_log2_denom) + fpart.u32;
+        return ipart * (1 << hdr->coef_log2_denom) + fpart.u32;
 
     case RPU_COEFF_FLOAT:
         fpart.u32 = get_bits_long(gb, 32);
@@ -172,7 +172,7 @@ static inline int64_t get_se_coef(GetBitContext *gb, const AVDOVIRpuDataHeader *
     case RPU_COEFF_FIXED:
         ipart = get_se_golomb_long(gb);
         fpart.u32 = get_bits_long(gb, hdr->coef_log2_denom);
-        return (ipart << hdr->coef_log2_denom) + fpart.u32;
+        return ipart * (1 << hdr->coef_log2_denom) + fpart.u32;
 
     case RPU_COEFF_FLOAT:
         fpart.u32 = get_bits_long(gb, 32);
-- 
2.20.1 (Apple Git-117)


[-- Attachment #3: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".