Hi

On Mon, Mar 24, 2025 at 02:53:27AM +0100, Michael Niedermayer wrote:
> Hi Jerome
> 
> On Sat, Mar 22, 2025 at 06:54:25PM +0100, Jerome Martinez wrote:
> > Le 22/03/2025 à 18:45, Michael Niedermayer a écrit :
> > > [...]
> > > Also I failed to find any worthy gain from adjusting mul_count so
> > > while the code in the encoder looks complex ATM alot of that can be
> > > dropped later if for example we choose to never put mul_count > 1 into
> > > the specification and ATM it makes no sense to put that in as theres no
> > > significant gain with the material i tested
> > 
> > I would prefer we don't put mul_count > 1 related code in FFmpeg, even if it
> > is still experimental, without a use case demonstrating that this is useful.
> 
> Ok, ive done some more tests
> 
> mul_count>1 is for fixed point in float and decimal mantisse*exponent
> formats. This is also what the paper concentrated on that ive looked at.

And before people ask about the paper:
its "ALP: Adaptive Lossless floating-Point Compression"

And before people ask, why we are not using it:
1. its about compressing 1D academic data not images, things like
    Barometric Pressure (kPa), Temperature (C°), Monetary (Stocks), ...
2. its primarly about 64bit double precission floats
3. with 64bit doubles they tested their method against zstd and zstd beats
   them 3.1 to 3.0 compression rate wise.
4. now to 32bit floats, which they tested too, and for that they beat zstd
   they achieve 28.1bits per float and zstd 29.7bit/float, all other variants
   they compare against are above 33.4bits per float.

So how do we compare to zstd?
Using ACES_OT_VWG run through -vf tonemap=hable,format=gbrp16 -pix_fmt gbrpf32
(to have some sort of simulation of 16bit fixed point sensor data in 32 bit float)
and using 256 slices to maximize the effect of the remap tables (the file with 40
in it uses 40 slices to compare)
rawdump is simply storing the remap tables as raw 32bit floats
rawvideo is rawvideo instead of ffv1
zst is zstd -k -19
noremapstored is simply the ffv1 data without the remap table stored

-rw-r----- 1 michael michael  483868924 Mär 24 13:24 float-303503-fixed-40-noremapstored.nut
-rw-r----- 1 michael michael  491567660 Mär 24 01:14 float-303503-fixed-256-noremapstored.nut.zst
-rw-r----- 1 michael michael  491835695 Mär 24 01:14 float-303503-fixed-256-noremapstored.nut
-rw-r----- 1 michael michael  499059484 Mär 24 12:52 float-303503-fixed-40.nut
-rw-r----- 1 michael michael  545695302 Mär 24 01:13 float-303503-fixed-256.nut
-rw-r----- 1 michael michael  600666368 Mär 24 12:26 float-303503-fixed-256-rawdump2.nut.zst
-rw-r----- 1 michael michael  807719501 Mär 24 12:54 float-303503-fixed-rawvideo.nut.zst
-rw-r----- 1 michael michael 1085998190 Mär 24 12:26 float-303503-fixed-256-rawdump2.nut
-rw-r----- 1 michael michael 1990659135 Mär 24 12:54 float-303503-fixed-rawvideo.nut

From this you can see, that for this example (assuming i made no mistake)
1. our code to store remap tables needs about half the space that zstd needs
2. ffv1 needs only 60% the space zstd of rawvideo needs
3. the remap tables are 3% of the file with big slices but over 10% with small

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

In fact, the RIAA has been known to suggest that students drop out
of college or go to community college in order to be able to afford
settlements. -- The RIAA