[FFmpeg-devel] [PATCH 1/5] avcodec/apedec: Fix integer overflow in filter_3800()

[FFmpeg-devel] [PATCH 1/5] avcodec/apedec: Fix integer overflow in filter_3800()

[Michael Niedermayer]

Fixes: signed integer overflow: -2147448926 + -198321 cannot be represented in type 'int'
Fixes: 48798/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5739619273015296
Fixes: 48798/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-6744428485672960

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavcodec/apedec.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c
index 4e3ddfea01b..c08d13d6c23 100644
--- a/libavcodec/apedec.c
+++ b/libavcodec/apedec.c
@@ -934,7 +934,7 @@ static av_always_inline int filter_3800(APEPredictor *p,
     p->coeffsB[filter][0] += (((d3 >> 29) & 4) - 2) * sign;
     p->coeffsB[filter][1] -= (((d4 >> 30) & 2) - 1) * sign;
 
-    p->filterB[filter] = p->lastA[filter] + (predictionB >> shift);
+    p->filterB[filter] = p->lastA[filter] + (unsigned)(predictionB >> shift);
     p->filterA[filter] = p->filterB[filter] + (unsigned)((int)(p->filterA[filter] * 31U) >> 5);
 
     return p->filterA[filter];
-- 
2.17.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 2/5] avcodec/dstdec: Check for overflow in build_filter()

[Michael Niedermayer]

Fixes: signed integer overflow: 1917019860 + 265558963 cannot be represented in type 'int'
Fixes: 48798/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DST_fuzzer-4833165046317056

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavcodec/dstdec.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/libavcodec/dstdec.c b/libavcodec/dstdec.c
index 6bdd6c885c8..4b1762db33c 100644
--- a/libavcodec/dstdec.c
+++ b/libavcodec/dstdec.c
@@ -215,7 +215,7 @@ static uint8_t prob_dst_x_bit(int c)
     return (ff_reverse[c & 127] >> 1) + 1;
 }
 
-static void build_filter(int16_t table[DST_MAX_ELEMENTS][16][256], const Table *fsets)
+static int build_filter(int16_t table[DST_MAX_ELEMENTS][16][256], const Table *fsets)
 {
     int i, j, k, l;
 
@@ -226,14 +226,17 @@ static void build_filter(int16_t table[DST_MAX_ELEMENTS][16][256], const Table *
             int total = av_clip(length - j * 8, 0, 8);
 
             for (k = 0; k < 256; k++) {
-                int v = 0;
+                int64_t v = 0;
 
                 for (l = 0; l < total; l++)
                     v += (((k >> l) & 1) * 2 - 1) * fsets->coeff[i][j * 8 + l];
+                if ((int16_t)v != v)
+                    return AVERROR_INVALIDDATA;
                 table[i][j][k] = v;
             }
         }
     }
+    return 0;
 }
 
 static int decode_frame(AVCodecContext *avctx, AVFrame *frame,
@@ -328,7 +331,9 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *frame,
         return AVERROR_INVALIDDATA;
     ac_init(ac, gb);
 
-    build_filter(s->filter, &s->fsets);
+    ret = build_filter(s->filter, &s->fsets);
+    if (ret < 0)
+        return ret;
 
     memset(s->status, 0xAA, sizeof(s->status));
     memset(dsd, 0, frame->nb_samples * 4 * channels);
-- 
2.17.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/5] avcodec/dstdec: Check for overflow in build_filter()

[Rémi Denis-Courmont]

Hi,

Down-casting to a signed type (here, int16_t) is implementation-defined. And while normal compilers do the expected thing, with modulo-2^n complement, sanitizers tend to dislike it.

AFAIK, the clean solution is via an union whence you assign the uint16_t member, and then read the int16_t member. Fortunately, GCC and LLVM are able to optimise that construct back to a single sign-extension.

Br,
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/5] avcodec/dstdec: Check for overflow in build_filter()

[Andreas Rheinhardt]

Rémi Denis-Courmont:
> Hi,
> 
> Down-casting to a signed type (here, int16_t) is implementation-defined. And while normal compilers do the expected thing, with modulo-2^n complement, sanitizers tend to dislike it.
> 

1. We expect the implementation-defined behaviour for signed types to
match the typical two's complement behaviour, see
https://ffmpeg.org/developer.html#C-language-features
2. In this case, there is no real-implementation-defined behaviour:
While the value (int16_t)v is implementation defined, whether it
coincides with v is not (it does so if and only if the value of v is
representable in an int16_t).
3. What sanitizers dislike it? After all, this is an explicit cast, so
e.g. UBSan would never report it as suspicious.

> AFAIK, the clean solution is via an union whence you assign the uint16_t member, and then read the int16_t member. Fortunately, GCC and LLVM are able to optimise that construct back to a single sign-extension.
> 

Type-punning via unions is implementation-defined, too. It will work
with two's complement types (as long as the number where all value bits
are unset and the sign bit is set is not a trap representation (all
exact-width intN_t are of this type)).

- Andreas
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 3/5] avcodec/exr: Check preview psize

[Michael Niedermayer]

Fixes: signed integer overflow: 17121181824 * 538976288 cannot be represented in type 'long long'
Fixes: 48798/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EXR_fuzzer-5915330316206080

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavcodec/exr.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/libavcodec/exr.c b/libavcodec/exr.c
index 235f6fa6cdd..c924406f131 100644
--- a/libavcodec/exr.c
+++ b/libavcodec/exr.c
@@ -1942,9 +1942,12 @@ static int decode_header(EXRContext *s, AVFrame *frame)
                                                      "preview", 16)) >= 0) {
             uint32_t pw = bytestream2_get_le32(gb);
             uint32_t ph = bytestream2_get_le32(gb);
-            int64_t psize = 4LL * pw * ph;
+            uint64_t psize = pw * ph;
+            if (psize > INT64_MAX / 4)
+                return AVERROR_INVALIDDATA;
+            psize *= 4;
 
-            if (psize >= bytestream2_get_bytes_left(gb))
+            if ((int64_t)psize >= bytestream2_get_bytes_left(gb))
                 return AVERROR_INVALIDDATA;
 
             bytestream2_skip(gb, psize);
-- 
2.17.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 3/5] avcodec/exr: Check preview psize

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 812 bytes --]

On Sun, Sep 11, 2022 at 12:30:44AM +0200, Michael Niedermayer wrote:
> Fixes: signed integer overflow: 17121181824 * 538976288 cannot be represented in type 'long long'
> Fixes: 48798/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EXR_fuzzer-5915330316206080
> 
> Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>  libavcodec/exr.c | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)

will apply

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Whats the most studid thing your enemy could do ? Blow himself up
Whats the most studid thing you could do ? Give up your rights and
freedom because your enemy blew himself up.


[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 4/5] avcodec/mobiclip: Check quantizer for overflow

[Michael Niedermayer]

Fixes: signed integer overflow: 127 + 2147483536 cannot be represented in type 'int'
Fixes: 48798/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MOBICLIP_fuzzer-6014034970804224

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavcodec/mobiclip.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavcodec/mobiclip.c b/libavcodec/mobiclip.c
index 5348f3bd6c3..aca462428c1 100644
--- a/libavcodec/mobiclip.c
+++ b/libavcodec/mobiclip.c
@@ -330,7 +330,7 @@ static av_cold int mobiclip_init(AVCodecContext *avctx)
     return 0;
 }
 
-static int setup_qtables(AVCodecContext *avctx, int quantizer)
+static int setup_qtables(AVCodecContext *avctx, int64_t quantizer)
 {
     MobiClipContext *s = avctx->priv_data;
     int qx, qy;
@@ -1256,7 +1256,7 @@ static int mobiclip_decode(AVCodecContext *avctx, AVFrame *rframe,
         frame->key_frame = 0;
         s->dct_tab_idx = 0;
 
-        ret = setup_qtables(avctx, s->quantizer + get_se_golomb(gb));
+        ret = setup_qtables(avctx, s->quantizer + (int64_t)get_se_golomb(gb));
         if (ret < 0)
             return ret;
 
-- 
2.17.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 4/5] avcodec/mobiclip: Check quantizer for overflow

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 699 bytes --]

On Sun, Sep 11, 2022 at 12:30:45AM +0200, Michael Niedermayer wrote:
> Fixes: signed integer overflow: 127 + 2147483536 cannot be represented in type 'int'
> Fixes: 48798/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MOBICLIP_fuzzer-6014034970804224
> 
> Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>  libavcodec/mobiclip.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)

will apply

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Never trust a computer, one day, it may think you are the virus. -- Compn

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 5/5] avcodec/tta: Check 24bit scaling for overflow

[Michael Niedermayer]

Fixes: signed integer overflow: -8427924 * 256 cannot be represented in type 'int'
Fixes: 48798/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TTA_fuzzer-5409428670644224

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavcodec/tta.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/libavcodec/tta.c b/libavcodec/tta.c
index 6fb8d7a74f9..d66e25af059 100644
--- a/libavcodec/tta.c
+++ b/libavcodec/tta.c
@@ -377,8 +377,15 @@ static int tta_decode_frame(AVCodecContext *avctx, AVFrame *frame,
     case 3: {
         // shift samples for 24-bit sample format
         int32_t *samples = (int32_t *)frame->data[0];
-        for (i = 0; i < framelen * s->channels; i++)
-            *samples++ *= 256;
+        int overflow = 0;
+
+        for (i = 0; i < framelen * s->channels; i++) {
+            int scaled = *samples * 256U;
+            overflow += (scaled >> 8 != *samples);
+            *samples++ = scaled;
+        }
+        if (overflow)
+            av_log(avctx, AV_LOG_WARNING, "%d overflows occurred on 24bit upscale\n", overflow);
         // reset decode buffer
         s->decode_buffer = NULL;
         break;
-- 
2.17.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 5/5] avcodec/tta: Check 24bit scaling for overflow

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 807 bytes --]

On Sun, Sep 11, 2022 at 12:30:46AM +0200, Michael Niedermayer wrote:
> Fixes: signed integer overflow: -8427924 * 256 cannot be represented in type 'int'
> Fixes: 48798/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TTA_fuzzer-5409428670644224
> 
> Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>  libavcodec/tta.c | 11 +++++++++--
>  1 file changed, 9 insertions(+), 2 deletions(-)

will apply

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Let us carefully observe those good qualities wherein our enemies excel us
and endeavor to excel them, by avoiding what is faulty, and imitating what
is excellent in them. -- Plutarch

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/5] avcodec/apedec: Fix integer overflow in filter_3800()

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 930 bytes --]

On Sun, Sep 11, 2022 at 12:30:42AM +0200, Michael Niedermayer wrote:
> Fixes: signed integer overflow: -2147448926 + -198321 cannot be represented in type 'int'
> Fixes: 48798/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5739619273015296
> Fixes: 48798/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-6744428485672960
> 
> Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>  libavcodec/apedec.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

will apply

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Rewriting code that is poorly written but fully understood is good.
Rewriting code that one doesnt understand is a sign that one is less smart
than the original author, trying to rewrite it will not make it better.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".