Re: [FFmpeg-devel] [PATCH] avcodec/x86/hevc: fix luma 12b overflow

From: James Almer <jamrial@gmail.com>
To: ffmpeg-devel@ffmpeg.org
Subject: Re: [FFmpeg-devel] [PATCH] avcodec/x86/hevc: fix luma 12b overflow
Date: Sun, 25 Feb 2024 13:28:09 -0300
Message-ID: <f3fe04e4-6880-40a9-a06f-3e42f30dd6cf@gmail.com> (raw)
In-Reply-To: <CAEEMt2nLXPTGdv8j+1SKNBAgWy-G4PaFkSqd7ZzNLiG4iAHO8g@mail.gmail.com>

On 2/25/2024 1:22 PM, Ronald S. Bultje wrote:
> On Sun, Feb 25, 2024 at 10:56 AM Ronald S. Bultje <rsbultje@gmail.com>
> wrote:
> 
>> Hi,
>>
>> On Sun, Feb 25, 2024 at 3:28 AM J. Dekker <jdek@itanimul.li> wrote:
>>
>>> Weak filter can overflow in delta0 calculation before >> 4 in int16.
>>>
>>> Signed-off-by: J. Dekker <jdek@itanimul.li>
>>> ---
>>>
>>>   I do not know x86 simd at all, so this is just an attempt to fix
>>>   the implementation rather than write extremely performant code.
>>>
>>>   Suggestions welcome.
>>>
>>
>> https://pastebin.com/KvcbQ2nK
>>
> 
> Attached a slightly adjusted version which does sse2 in 16bit also.
> 
> Ronald

> diff --git a/libavcodec/x86/hevc_deblock.asm b/libavcodec/x86/hevc_deblock.asm
> index 85ee4800bb..869301caff 100644
> --- a/libavcodec/x86/hevc_deblock.asm
> +++ b/libavcodec/x86/hevc_deblock.asm
> @@ -31,6 +31,7 @@ cextern pw_1023
>  pw_pixel_max_12: times 8 dw ((1 << 12)-1)
>  pw_m2:           times 8 dw -2
>  pd_1 :           times 4 dd  1
> +pd_8 :           times 8 dd  8

This is unused.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".