From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <ffmpeg-devel-bounces@ffmpeg.org>
Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100])
	by master.gitmailbox.com (Postfix) with ESMTPS id A085F4E73A
	for <ffmpegdev@gitmailbox.com>; Fri, 14 Mar 2025 20:44:34 +0000 (UTC)
Received: from [127.0.1.1] (localhost [127.0.0.1])
	by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6257B68E341;
	Fri, 14 Mar 2025 22:44:30 +0200 (EET)
Received: from haasn.dev (haasn.dev [78.46.187.166])
 by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 62A8D68D88B
 for <ffmpeg-devel@ffmpeg.org>; Fri, 14 Mar 2025 22:44:23 +0200 (EET)
Received: from haasn.dev (unknown [10.30.1.1])
 by haasn.dev (Postfix) with UTF8SMTP id 0EF05406F0
 for <ffmpeg-devel@ffmpeg.org>; Fri, 14 Mar 2025 21:44:23 +0100 (CET)
Date: Fri, 14 Mar 2025 21:44:22 +0100
Message-ID: <20250314214422.GB518999@haasn.xyz>
From: Niklas Haas <ffmpeg@haasn.xyz>
To: ffmpeg-devel@ffmpeg.org
In-Reply-To: <20250314010930.GB892569@haasn.xyz>
References: <20241226193325.288834126EE@natalya.videolan.org>
 <20250126182659.GE2768425@pb2> <20250313235425.GC4991@pb2>
 <20250314010930.GB892569@haasn.xyz>
MIME-Version: 1.0
Content-Disposition: inline
Subject: Re: [FFmpeg-devel] [FFmpeg-cvslog] swscale: use 16-bit intermediate
 precision for RGB/XYZ conversion
X-BeenThere: ffmpeg-devel@ffmpeg.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: FFmpeg development discussions and patches <ffmpeg-devel.ffmpeg.org>
List-Unsubscribe: <https://ffmpeg.org/mailman/options/ffmpeg-devel>,
 <mailto:ffmpeg-devel-request@ffmpeg.org?subject=unsubscribe>
List-Archive: <https://ffmpeg.org/pipermail/ffmpeg-devel>
List-Post: <mailto:ffmpeg-devel@ffmpeg.org>
List-Help: <mailto:ffmpeg-devel-request@ffmpeg.org?subject=help>
List-Subscribe: <https://ffmpeg.org/mailman/listinfo/ffmpeg-devel>,
 <mailto:ffmpeg-devel-request@ffmpeg.org?subject=subscribe>
Reply-To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: ffmpeg-devel-bounces@ffmpeg.org
Sender: "ffmpeg-devel" <ffmpeg-devel-bounces@ffmpeg.org>
Archived-At: <https://master.gitmailbox.com/ffmpegdev/20250314214422.GB518999@haasn.xyz/>
List-Archive: <https://master.gitmailbox.com/ffmpegdev/>
List-Post: <mailto:ffmpegdev@gitmailbox.com>

On Fri, 14 Mar 2025 01:09:30 +0100 Niklas Haas <ffmpeg@haasn.xyz> wrote:
> On Fri, 14 Mar 2025 00:54:25 +0100 Michael Niedermayer <michael@niedermayer.cc> wrote:
> > Hi
> >
> > On Sun, Jan 26, 2025 at 07:26:59PM +0100, Michael Niedermayer wrote:
> > > On Thu, Dec 26, 2024 at 07:33:23PM +0000, Niklas Haas wrote:
> > > > ffmpeg | branch: master | Niklas Haas <git@haasn.dev> | Mon Dec 16 14:49:39 2024 +0100| [af6d52eec66961f6a502b0f2f390c12226d087cd] | committer: Niklas Haas
> > > >
> > > > swscale: use 16-bit intermediate precision for RGB/XYZ conversion
> > > >
> > > > The current logic uses 12-bit linear light math, which is woefully insufficient
> > > > and leads to nasty postarization artifacts. This patch simply switches the
> > > > internal logic to 16-bit precision.
> > > >
> > > > This raises the memory requirement of these tables from 32 kB to 272 kB.
> > > >
> > > > All relevant FATE tests updated for improved accuracy.
> > > >
> > > > Fixes: #4829
> > > > Signed-off-by: Niklas Haas <git@haasn.dev>
> > > > Sponsored-by: Sovereign Tech Fund
> > > >
> > > > > http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=af6d52eec66961f6a502b0f2f390c12226d087cd
> > > > ---
> > >
> > > this breaks on x86-32
> >
> > ping
> >
> > the fate tests for a major architecture are broken since this
> > This impacts future bisections over that range for example
>
> My bad, missed this email for some reason.
>
> Will investigate and fix tomorrow.

So, I found the bug, but I don't know how to fix it properly.

The problem is that tha XYZ/RGB conversion tables are calculated using excess
precision from the x87 FPU. This results in slightly different rounding behavior
in exactly one case (i == 3415).

Fixing this requires setting -mfpmath=sse at compile time. Strangely, I was
not able to force the compiler to discard the excess precision even with
forcing -fexcess-precision=standard and storing every single subexpression to
a `volatile double`.

How do we deal with such cases generally? I can think of several options, such
as hard-coding the table values (unappealing), or somehow using integer math
(very unappealing, how do we calculate an integer power without massive loss
of precision?)

>
> >
> > you can trivially test this on x86-64 with something like:
> > --cross-prefix=/usr/i686-linux-gnu/bin/ --cc='ccache i686-linux-gnu-gcc-7' --arch=x86_32  --target-os=linux --enable-cross-compile
> > if you have the correponding packages installed
> >
> > before af6d52eec66961f6a502b0f2f390c12226d087cd
> > TEST    filter-pixdesc-xyz12be
> >
> > after af6d52eec66961f6a502b0f2f390c12226d087cd:
> > --- src/tests/ref/fate/filter-pixdesc-xyz12be	2025-03-14 00:46:40.445681223 +0100
> > +++ tests/data/fate/filter-pixdesc-xyz12be	2025-03-14 00:47:08.301903796 +0100
> > @@ -1 +1 @@
> > -pixdesc-xyz12be     1508a33dea936c45d9ee13f7743af00d
> > +pixdesc-xyz12be     198f43f452bc55f4ca1e0e0171de5c4c
> > Test filter-pixdesc-xyz12be failed. Look at tests/data/fate/filter-pixdesc-xyz12be.err for details.
> > make: *** [src/tests/Makefile:311: fate-filter-pixdesc-xyz12be] Error 1
> >
> > in master:
> > --- src/tests/ref/fate/filter-pixdesc-xyz12be	2025-03-14 00:46:40.445681223 +0100
> > +++ tests/data/fate/filter-pixdesc-xyz12be	2025-03-14 00:48:43.042660972 +0100
> > @@ -1 +1 @@
> > -pixdesc-xyz12be     1508a33dea936c45d9ee13f7743af00d
> > +pixdesc-xyz12be     198f43f452bc55f4ca1e0e0171de5c4c
> > Test filter-pixdesc-xyz12be failed. Look at tests/data/fate/filter-pixdesc-xyz12be.err for details.
> > make: *** [src/tests/Makefile:315: fate-filter-pixdesc-xyz12be] Error 1
> >
> >
> > thx
> >
> > [...]
> >
> > --
> > Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> >
> > Never trust a computer, one day, it may think you are the virus. -- Compn
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".