[FFmpeg-devel] [PATCH] checkasm: Increase the tolerance for ac3_sum_square_butterfly_float

[FFmpeg-devel] [PATCH] checkasm: Increase the tolerance for ac3_sum_square_butterfly_float

[Martin Storsjö via ffmpeg-devel]

Increase the tolerance from 10 ulp to 11 ulp. This fixes occasional
errors for some inputs; the errors could be reproduced on
aarch64/neon builds, with "checkasm --test=ac3dsp 3446175925".
---
 tests/checkasm/ac3dsp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/checkasm/ac3dsp.c b/tests/checkasm/ac3dsp.c
index 442e965f3b..8c682d03cd 100644
--- a/tests/checkasm/ac3dsp.c
+++ b/tests/checkasm/ac3dsp.c
@@ -181,7 +181,7 @@ static void check_ac3_sum_square_butterfly_float(AC3DSPContext *c) {
         call_ref(v1, lt, rt, ELEMS);
         call_new(v2, lt, rt, ELEMS);
 
-        if (!float_near_ulp_array(v1, v2, 10, 4))
+        if (!float_near_ulp_array(v1, v2, 11, 4))
             fail();
 
         bench_new(v2, lt, rt, ELEMS);
-- 
2.39.3 (Apple Git-146)

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] checkasm: Increase the tolerance for ac3_sum_square_butterfly_float

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 664 bytes --]

On Wed, Jul 24, 2024 at 12:01:32AM +0300, Martin Storsjö via ffmpeg-devel wrote:
> Increase the tolerance from 10 ulp to 11 ulp. This fixes occasional
> errors for some inputs; the errors could be reproduced on
> aarch64/neon builds, with "checkasm --test=ac3dsp 3446175925".
> ---
>  tests/checkasm/ac3dsp.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

probably ok

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Many things microsoft did are stupid, but not doing something just because
microsoft did it is even more stupid. If everything ms did were stupid they
would be bankrupt already.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] checkasm: Increase the tolerance for ac3_sum_square_butterfly_float

[Martin Storsjö]

On Tue, 23 Jul 2024, Michael Niedermayer wrote:

> On Wed, Jul 24, 2024 at 12:01:32AM +0300, Martin Storsjö via ffmpeg-devel wrote:
>> Increase the tolerance from 10 ulp to 11 ulp. This fixes occasional
>> errors for some inputs; the errors could be reproduced on
>> aarch64/neon builds, with "checkasm --test=ac3dsp 3446175925".
>> ---
>>  tests/checkasm/ac3dsp.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> probably ok

Thanks, pushed it now.

// Martin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".