From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id 78A0C4BF2E for ; Thu, 6 Mar 2025 16:38:03 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F213B68F20A; Thu, 6 Mar 2025 18:37:58 +0200 (EET) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A378068B3AF for ; Thu, 6 Mar 2025 18:37:52 +0200 (EET) Received: by mail.gandi.net (Postfix) with ESMTPSA id 09297204D1 for ; Thu, 6 Mar 2025 16:37:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niedermayer.cc; s=gm1; t=1741279072; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Q3t3x/phvgwY3xsMcPCZSF/2o6JZDx7Bl1oINxvhfA4=; b=mffwYNyzSlLVaZ1GWnCj5hojK9Nt1k/TJVgR6ODcH08Wgc4AlyEs8CXHl/WiBb3I0r2NoS t+uJfguME2Z9c8JWFXNdUKZbEF5jfC/kpB1JbNzpVnDMCK3brx5bKF4rU7qQtunDsQ6ND6 MXuUfNF1Xt3tFDzYZpLXwZ+Zi83Y4XyIQp8R9coUonRhjR5NJOWYjTrAlymHyTgg2Mkr57 Oztn4BicciiCMDHAsVvzThaQM587bSyzV+RUmAyPyMdQ+thA99KSX4eO2D5oSLhF+rOBIi ye1Xpump7iiNGnJVjc5e5d8lMnLHht898NH9ng5DIu2TjWs+lFfYzg7w2KD5Uw== Date: Thu, 6 Mar 2025 17:37:51 +0100 From: Michael Niedermayer To: FFmpeg development discussions and patches Message-ID: <20250306163751.GW4991@pb2> References: <20250306001552.GV4991@pb2> MIME-Version: 1.0 In-Reply-To: X-GND-State: clean X-GND-Score: -85 X-GND-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddutdekvdeiucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuifetpfffkfdpucggtfgfnhhsuhgsshgtrhhisggvnecuuegrihhlohhuthemuceftddunecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenfghrlhcuvffnffculdduhedmnecujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefoihgthhgrvghlucfpihgvuggvrhhmrgihvghruceomhhitghhrggvlhesnhhivgguvghrmhgrhigvrhdrtggtqeenucggtffrrghtthgvrhhnpeffledtfeevfeffheeuuefhtdejieelueeftdeitdfgheetgefffeefteekffdthfenucffohhmrghinhepfhhfmhhpvghgrdhorhhgnecukfhppeeguddrieeirdeijedruddufeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepihhnvghtpeeguddrieeirdeijedruddufedphhgvlhhopehlohgtrghlhhhoshhtpdhmrghilhhfrhhomhepmhhitghhrggvlhesnhhivgguvghrmhgrhigvrhdrtggtpdhnsggprhgtphhtthhopedupdhrtghpthhtohepfhhfmhhpvghgqdguvghvvghlsehffhhmphgvghdrohhrgh X-GND-Sasl: michael@niedermayer.cc Subject: Re: [FFmpeg-devel] [RFC] FFV1 float support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: multipart/mixed; boundary="===============1381529940846576576==" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: --===============1381529940846576576== Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="ZuTAjMrl77sZyLGZ" Content-Disposition: inline --ZuTAjMrl77sZyLGZ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi On Thu, Mar 06, 2025 at 03:14:49AM +0100, Lynne wrote: > On 06/03/2025 01:15, Michael Niedermayer wrote: > > Hi everyone > >=20 > > Current FFv1 code with my patchset supports 16bit floats. The implement= ation > > is quite simple. Which is good > >=20 > > I have tried improving compression, the gains are incremental, but its = not > > large overall. For example 44% space gain on the remapping table is just > > 0.1% overall. > >=20 > > I have few float samples. Its mainly one high quality slideshow of unre= lated > > 16bit float images. These excercise a wide range fo things including ne= gative > > color values. > > I think I have only one single (non slideshow) video of float 16 sample= s, > >=20 > > It turns out the most efficient way to store floats that i found, is to= remap > > them to integers. Not to store tham as sign, exponent, mantisse, i trie= d that > > for many hours. > >=20 > > Storing them using a remapping, has a very nice side effect though, and= that is > > it will be easy to add 32bit and 64bit float support (once there is some > > sample data) > > Because a image of 64bit floats, after its split into slices of up to 2= 56x256 > > can always be mapped into 16bit integers within each slice. >=20 > *how*? Unlike ints, you cannot simply shift and extract bits from floats = and > treat them as floats. well, if the input is IEEE 754 32bit or 64bit floats or some 16bit floats. they can be read with AV_RL16/32/64 and are then 16/32/64bit ints. So we ha= ve slices of 256x256 or smaller of such integers. A 256x256 slice has at maximum 64k distinct values, these values can always be stored in 16bit, once they are remapped to a compact list. There are no floats after the remapping and before its inverted in the deco= der directly doing anything with floats is not bit exactly defined in C so we have to work with integers anyway in a lossless codec. >=20 > > What about the mapping itself, it uses a rather simple rle coder. ive s= pend > > most of the day today tuning that and failing to really beat that. > > Using context from the previous image didnt work with the slideshow mat= erial > > i have nor that one video i have. I tried using sign and exponent as co= ntext, > > tried previous run, relations of runs, low bits and many more, but no or > > insignificant improvments of yesterdays implementation was achieved. >=20 > I did mention this once before, but you should look into the Daala/Opus w= ay > of storing rawbits into a separate chunk at the end of the bitstream. It > avoids polluting the range context with equiprobable bits. This may fit into my LSB work but that is not float specific and its not needed for float support. I originally thought we would do floats through RAW storing LSB, and special handling the "always" 0 and 1 bits. It makes sense but just remapping only the used values to a compact list eliminates all the constant bits for free and we never have more than 16bits per sample We still can implement storing LSB raw, i have written 2 implementations the first without range coder was this: https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2024-October/334718.html I didnt like it, but it seems more popular Also given the raw bits are a known and constant length. It could make sense to put them first if they are stored non interleaved. but IIRC, the implementation for LSB refered in the link decorrelates after LSB is split, it should give more compression doing it the other way around. That said the LSB patch still isnt needed for float support, it could improve speed at the expense of compression for 16 and 32bit samples (float= or other) thx [...] --=20 Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Republics decline into democracies and democracies degenerate into despotisms. -- Aristotle --ZuTAjMrl77sZyLGZ Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iF0EABEKAB0WIQSf8hKLFH72cwut8TNhHseHBAsPqwUCZ8nPWwAKCRBhHseHBAsP q5y7AJ9pKCrWs99eG0Az3fi0OeZxVEKXwgCfXl56c45YNTSrq2A7cv8IhunGe8k= =DHis -----END PGP SIGNATURE----- --ZuTAjMrl77sZyLGZ-- --===============1381529940846576576== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". --===============1381529940846576576==--