Re: [FFmpeg-devel] [PATCH 4/4] avcodec/osq: avoid using too large numbers for shifts and integers in update_residue_parameter()

From: Michael Niedermayer <michael@niedermayer.cc>
To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Subject: Re: [FFmpeg-devel] [PATCH 4/4] avcodec/osq: avoid using too large numbers for shifts and integers in update_residue_parameter()
Date: Fri, 15 Sep 2023 16:38:07 +0200
Message-ID: <20230915143807.GU8640@pb2> (raw)
In-Reply-To: <CAPYw7P6uZ+uUDQcEBy+WL44bi5FMe9KyRox8E9AKf=75bzQp7w@mail.gmail.com>

[-- Attachment #1.1: Type: text/plain, Size: 1023 bytes --]

On Fri, Sep 15, 2023 at 03:54:19PM +0200, Paul B Mahol wrote:
> On Fri, Sep 15, 2023 at 3:12 PM Michael Niedermayer <michael@niedermayer.cc>
> wrote:
> 
> > The code should be changed to not use floats in the VLC parameters
> > This patch merely fixes undefined behavior
> >
> > Fixes: 2.96539e+09 is outside the range of representable values of type
> > 'int'
> > Fixes: Assertion n>=0 && n<=32 failed at libavcodec/get_bits.h:423
> > Fixes:
> > 62241/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_OSQ_fuzzer-4525761925873664
> >
> 
> 
> NAK
> 
> Breaks decoding.

Please provide sample that works before and fails after this.

That said, it has to be pointed out that the current code in osq is
buggy in multiply ways. The use of floats for computing vlc parameters is
not portable (unless theres alot of luck).

thx

[....]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

I have often repented speaking, but never of holding my tongue.
-- Xenocrates

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".