Re: [FFmpeg-devel] [PATCH 2/3] avcodec/x86: disable hevc 12b luma deblock

From: "J. Dekker" <jdek@itanimul.li>
To: ffmpeg-devel@ffmpeg.org
Subject: Re: [FFmpeg-devel] [PATCH 2/3] avcodec/x86: disable hevc 12b luma deblock
Date: Sat, 24 Feb 2024 10:49:21 +0100
Message-ID: <87cysmjifp.fsf@itanimul.li> (raw)
In-Reply-To: <CAFXK13dZYj4aa0eoPsUbt=RPwZhJNg+tk2P83Z=P_gw8SgxOFw@mail.gmail.com>

Nuo Mi <nuomi2021@gmail.com> writes:

> On Wed, Feb 21, 2024 at 7:10 PM J. Dekker <jdek@itanimul.li> wrote:
>
>> Over/underflow in some cases.
>>
>> Signed-off-by: J. Dekker <jdek@itanimul.li>
>> ---
>>  libavcodec/x86/hevcdsp_init.c | 9 +++++----
>>  1 file changed, 5 insertions(+), 4 deletions(-)
>>
>> diff --git a/libavcodec/x86/hevcdsp_init.c b/libavcodec/x86/hevcdsp_init.c
>> index 31e81eb11f..11cb1b3bfd 100644
>> --- a/libavcodec/x86/hevcdsp_init.c
>> +++ b/libavcodec/x86/hevcdsp_init.c
>> @@ -1205,10 +1205,11 @@ void ff_hevc_dsp_init_x86(HEVCDSPContext *c, const
>> int bit_depth)
>>          if (EXTERNAL_SSE2(cpu_flags)) {
>>              c->hevc_v_loop_filter_chroma =
>> ff_hevc_v_loop_filter_chroma_12_sse2;
>>              c->hevc_h_loop_filter_chroma =
>> ff_hevc_h_loop_filter_chroma_12_sse2;
>> -            if (ARCH_X86_64) {
>> -                c->hevc_v_loop_filter_luma =
>> ff_hevc_v_loop_filter_luma_12_sse2;
>> -                c->hevc_h_loop_filter_luma =
>> ff_hevc_h_loop_filter_luma_12_sse2;
>> -            }
>> +            // FIXME: 12-bit luma deblock over/underflows in some cases
>> +            // if (ARCH_X86_64) {
>> +            //     c->hevc_v_loop_filter_luma =
>> ff_hevc_v_loop_filter_luma_12_sse2;
>> +            //     c->hevc_h_loop_filter_luma =
>> ff_hevc_h_loop_filter_luma_12_sse2;
>> +            // }
>>              SAO_BAND_INIT(12, sse2);
>>              SAO_EDGE_INIT(12, sse2);
>>
> Hi Dekker,
> VVC will utilize this function as well.
> Could you please share the HEVC clip or data that caused the overflow?
> We'll make efforts to address it during the VVC porting
>

You can just run ./tests/checkasm/checkasm --test=hevc_deblock to
find a failing case. My guess is that delta0 overflows before the right
shift, see the ARM64 asm which specfically widens this calculation on 12
bit variant but I'm not 100%, I don't know x86 asm.

-- 
jd
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".