From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id 407514BFA5 for ; Fri, 7 Mar 2025 01:13:26 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9614F68F4BB; Fri, 7 Mar 2025 03:13:22 +0200 (EET) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 28B5968EF63 for ; Fri, 7 Mar 2025 03:13:16 +0200 (EET) Received: by mail.gandi.net (Postfix) with ESMTPSA id 83A87443CD for ; Fri, 7 Mar 2025 01:13:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niedermayer.cc; s=gm1; t=1741309995; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=GtyPaDRJQI2cLn/33yhpyLr+Cktbbb3weG6JY2P3U3I=; b=lExMheux0qaOKNiAVM42A4zQjgaDxqBHvoYT2ORi0qAzXqfTuMvpKV5CGG+dFICnT218KP lH0FfhNePYF+Sbruki+3z0xDcjpOluXZ0B1N74YMThRACgVnS0rVuWjB7pndf/7ll8eaj3 3s5KCSKGhlRQmjos8w2/n8MWiNBf6fDzUv88334FvuT2g6S/wNMg53pbylKRNrYIsD+g57 jsx0bxXPOQ32ftsZf6Fvdl/ierxLIJj11fksXy87aKIPbAO08Nwfmigi149yAQ/+xwOvY/ /cAS8lDxXdJoAWwevznKl0OsxpeXYxRhQPEW0WVnT2ZvgcXA3w6VisX8bECDtg== Date: Fri, 7 Mar 2025 02:13:14 +0100 From: Michael Niedermayer To: FFmpeg development discussions and patches Message-ID: <20250307011314.GY4991@pb2> References: <20250306001552.GV4991@pb2> <20250306163751.GW4991@pb2> <825ffaa6-7b03-439c-b9aa-b0714d8738c2@mediaarea.net> MIME-Version: 1.0 In-Reply-To: <825ffaa6-7b03-439c-b9aa-b0714d8738c2@mediaarea.net> X-GND-State: clean X-GND-Score: -85 X-GND-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdelfedtucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuifetpfffkfdpucggtfgfnhhsuhgsshgtrhhisggvnecuuegrihhlohhuthemuceftddunecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenfghrlhcuvffnffculdduhedmnecujfgurhepfffhvffukfhfgggtuggjsehgtderredttddunecuhfhrohhmpefoihgthhgrvghlucfpihgvuggvrhhmrgihvghruceomhhitghhrggvlhesnhhivgguvghrmhgrhigvrhdrtggtqeenucggtffrrghtthgvrhhnpeevueefgeefudetuefggfeivdetteekgfdukedtjeeuffevheegleduleffudfgheenucffohhmrghinhepfhhfmhhpvghgrdhorhhgnecukfhppeeguddrieeirdeijedruddufeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepihhnvghtpeeguddrieeirdeijedruddufedphhgvlhhopehlohgtrghlhhhoshhtpdhmrghilhhfrhhomhepmhhitghhrggvlhesnhhivgguvghrmhgrhigvrhdrtggtpdhnsggprhgtphhtthhopedupdhrtghpthhtohepfhhfmhhpvghgqdguvghvvghlsehffhhmphgvghdrohhrgh X-GND-Sasl: michael@niedermayer.cc Subject: Re: [FFmpeg-devel] [RFC] FFV1 float support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: multipart/mixed; boundary="===============0369114408503996072==" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: --===============0369114408503996072== Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="YXwyRHx8h8vbEtPe" Content-Disposition: inline --YXwyRHx8h8vbEtPe Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi Jerome On Thu, Mar 06, 2025 at 10:20:58PM +0100, Jerome Martinez wrote: > Le 06/03/2025 =E0 17:37, Michael Niedermayer a =E9crit=A0: > > On Thu, Mar 06, 2025 at 03:14:49AM +0100, Lynne wrote: > > > On 06/03/2025 01:15, Michael Niedermayer wrote: > > > > Hi everyone > > > >=20 > > > > Current FFv1 code with my patchset supports 16bit floats. The imple= mentation > > > > is quite simple. Which is good > > > >=20 > > > > I have tried improving compression, the gains are incremental, but = its not > > > > large overall. For example 44% space gain on the remapping table is= just > > > > 0.1% overall. > > > >=20 > > > > I have few float samples. Its mainly one high quality slideshow of = unrelated > > > > 16bit float images. These excercise a wide range fo things includin= g negative > > > > color values. > > > > I think I have only one single (non slideshow) video of float 16 sa= mples, > > > >=20 > > > > It turns out the most efficient way to store floats that i found, i= s to remap > > > > them to integers. Not to store tham as sign, exponent, mantisse, i = tried that > > > > for many hours. >=20 > I also tried some stuff but IMO the remap to integer is what is currently > the best for keeping the spec simple & with a good compression, and a more > complex way is not needed until we find something which really makes a > difference in term of compression ratio. > So let's keep it simple, just a remap to int, and we already get all the > advantage of e.g. the incoming GPU implementation. yes, i fully agree >=20 >=20 > > > > [...] > > > > What about the mapping itself, it uses a rather simple rle coder. i= ve spend > > > > most of the day today tuning that and failing to really beat that. > > > > Using context from the previous image didnt work with the slideshow= material > > > > i have nor that one video i have. I tried using sign and exponent a= s context, > > > > tried previous run, relations of runs, low bits and many more, but = no or > > > > insignificant improvments of yesterdays implementation was achieved. > > > I did mention this once before, but you should look into the Daala/Op= us way > > > of storing rawbits into a separate chunk at the end of the bitstream.= It > > > avoids polluting the range context with equiprobable bits. > > This may fit into my LSB work but that is not float specific and its not > > needed for float support. > >=20 > > I originally thought we would do floats through RAW storing LSB, and sp= ecial > > handling the "always" 0 and 1 bits. It makes sense but just remapping o= nly > > the used values to a compact list eliminates all the constant bits for = free > > and we never have more than 16bits per sample > >=20 > > We still can implement storing LSB raw, i have written 2 implementations > > the first without range coder was this: > > https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2024-October/334718.html > >=20 > > I didnt like it, but it seems more popular >=20 > I am still in favor of this simple and independent way to store the LSB > bits, the other tentatives seem to only add complexity and performance > issues while not providing enough gain. >=20 >=20 > >=20 > > Also given the raw bits are a known and constant length. It could make > > sense to put them first if they are stored non interleaved. >=20 > Putting them last is better IMO, we can compute the byte offset with the > constant length while storing some config in the slice header. > Actually, there are many possibilities, I suggest: > - permitting both MSB & LSB, because we may have MSB 0 and we would no st= ore > them (see below) > - after the count of bits not compressed, we add a flag indicating if we > store them or not (meaning they are 0) > - permitting a configuration per stream (Configuration Record) or per sli= ce > (Slice Header); if a decision is to keep only one place for keeping it > simple, it would be in the slice header. >=20 > The rational is that we may have: > - 0 padding for the LSB is when we compress integer from DCP, we have inp= ut > with 16-bit TIFF but only 12 relevant bits, and we need to keep the bit > depth of 16-bit if we want to say that we do it lossless and that we can > reverse, especially because 99.999% of frames are with 4 bits of padding = but > you may have a frame for whatever reason (often a bug, but if we do lossl= ess > we need to store buggy frames) which is not 0, so a performant storage of > 0-filled LSB while permitting to store the actual bits if not 0. > - 0 padding for the MSB is when we compress float mapped to integer and > float range is from 0.0 to 1.0 only, sign & most of mantissa are always 0= so > we can store them as a flag, but we prepare the stream to have values out= of > range (<0 or > 1). > - and we don't know that before we start to compress each frame/slice We have a generic remap table with the float code. 1. This is not really float specific, it could be used with integers 2. Its per slice, so each slice can have a different table or none Any bits that are always 0 or 1 will be perfectly optimized out using that table. No mattter if LSB, MSB or in the middle even if every color is a multiple of 13 the remap table will still perfectly remove that and produce a continous range that is one thirteenth If you disagree, please explain how thats not already solving every variant and more ? thx [...] --=20 Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Those who are too smart to engage in politics are punished by being governed by those who are dumber. -- Plato=20 --YXwyRHx8h8vbEtPe Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iF0EABEKAB0WIQSf8hKLFH72cwut8TNhHseHBAsPqwUCZ8pIIwAKCRBhHseHBAsP qznkAJ99zDiZJkzjLTOyK9LmM1HHanK1jwCgj4CQ7dYkonkzPb7QU78ctbeMZf4= =UuMi -----END PGP SIGNATURE----- --YXwyRHx8h8vbEtPe-- --===============0369114408503996072== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". --===============0369114408503996072==--