From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <ffmpeg-devel-bounces@ffmpeg.org> Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id D16C844ADE for <ffmpegdev@gitmailbox.com>; Sat, 22 Mar 2025 17:45:31 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3C094687C16; Sat, 22 Mar 2025 19:45:28 +0200 (EET) Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [217.70.183.198]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D3FAE687B8A for <ffmpeg-devel@ffmpeg.org>; Sat, 22 Mar 2025 19:45:21 +0200 (EET) Received: by mail.gandi.net (Postfix) with ESMTPSA id 4246F442C3 for <ffmpeg-devel@ffmpeg.org>; Sat, 22 Mar 2025 17:45:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niedermayer.cc; s=gm1; t=1742665521; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=YPC28mt/HIULiw947lGP1YnDGH+XuZTJqs7aiPeS07Q=; b=mpH15EtZMQcLLekKZbITrksp8c05ze+4NutffhURe+wfkcEpUHS+8sNfgLeSHTeuyPmjkW qdShc+phudavnk0akeH4yeBnNBUcm8kAxA6XLRiG5OANikB9APo8WiE59t3KPDyxWlUXpK /B0/BFYqArBrJ0mRWuoy22vDZcK1n2EaC+b+pEX88BghyG2QGCw+MwfoJZUl5bQ/ujYBwm kJtA3oEI8TyDxDzavcJ9uuckYXJoY2kyz7dNj7dAic6gRtBGk4PwHQKA/yRnMfVkfkSO3M qTJkR0kI6Ca8SD3CkFWtzD9Xa1Bv6dt4bYPCXZevw5THasbmdsOEWY4obFk8Vw== Date: Sat, 22 Mar 2025 18:45:20 +0100 From: Michael Niedermayer <michael@niedermayer.cc> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org> Message-ID: <20250322174520.GN4991@pb2> References: <20250320223028.3469435-1-michael@niedermayer.cc> <5f802aaa-958b-49c3-9b4e-3b36e97b244c@lynne.ee> <20250321201349.GF4991@pb2> <20250321211250.GG4991@pb2> <20250321213658.GH4991@pb2> <20250321222245.GI4991@pb2> MIME-Version: 1.0 In-Reply-To: <20250321222245.GI4991@pb2> X-GND-State: clean X-GND-Score: -70 X-GND-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgdduheegiedtucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuifetpfffkfdpucggtfgfnhhsuhgsshgtrhhisggvnecuuegrihhlohhuthemuceftddunecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenfghrlhcuvffnffculdeftddmnecujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefoihgthhgrvghlucfpihgvuggvrhhmrgihvghruceomhhitghhrggvlhesnhhivgguvghrmhgrhigvrhdrtggtqeenucggtffrrghtthgvrhhnpeeigeektdejudffjefhteegjedtgeettefggedthfejgfevhfetgeekjedtvdfhveenucfkphepgedurdeiiedrieejrdduudefnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehinhgvthepgedurdeiiedrieejrdduudefpdhhvghloheplhhotggrlhhhohhsthdpmhgrihhlfhhrohhmpehmihgthhgrvghlsehnihgvuggvrhhmrgihvghrrdgttgdpnhgspghrtghpthhtohepuddprhgtphhtthhopehffhhmphgvghdquggvvhgvlhesfhhfmhhpvghgrdhorhhg X-GND-Sasl: michael@niedermayer.cc Subject: Re: [FFmpeg-devel] [PATCH] avcodec/ffv1: Implement 2D RLE for remap X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches <ffmpeg-devel.ffmpeg.org> List-Unsubscribe: <https://ffmpeg.org/mailman/options/ffmpeg-devel>, <mailto:ffmpeg-devel-request@ffmpeg.org?subject=unsubscribe> List-Archive: <https://ffmpeg.org/pipermail/ffmpeg-devel> List-Post: <mailto:ffmpeg-devel@ffmpeg.org> List-Help: <mailto:ffmpeg-devel-request@ffmpeg.org?subject=help> List-Subscribe: <https://ffmpeg.org/mailman/listinfo/ffmpeg-devel>, <mailto:ffmpeg-devel-request@ffmpeg.org?subject=subscribe> Reply-To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org> Content-Type: multipart/mixed; boundary="===============9170598178088546418==" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" <ffmpeg-devel-bounces@ffmpeg.org> Archived-At: <https://master.gitmailbox.com/ffmpegdev/20250322174520.GN4991@pb2/> List-Archive: <https://master.gitmailbox.com/ffmpegdev/> List-Post: <mailto:ffmpegdev@gitmailbox.com> --===============9170598178088546418== Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="EMT7SBGHGqDayLsM" Content-Disposition: inline --EMT7SBGHGqDayLsM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Mar 21, 2025 at 11:22:45PM +0100, Michael Niedermayer wrote: > On Fri, Mar 21, 2025 at 10:36:58PM +0100, Michael Niedermayer wrote: > > On Fri, Mar 21, 2025 at 10:12:50PM +0100, Michael Niedermayer wrote: > > > Hi > > >=20 > > > On Fri, Mar 21, 2025 at 09:13:49PM +0100, Michael Niedermayer wrote: > > > > On Fri, Mar 21, 2025 at 12:07:30AM +0100, Lynne wrote: > > > > > On 20/03/2025 23:30, Michael Niedermayer wrote: > > > > > > This performs about as good as the non LRU system for 16bit and > > > > > > better than then the LRU system for 16 converted to 32. So > > > > > > its basically performing best in all cases we have atm making > > > > > > the LRU system unneeded. > > > > >=20 > > > > > Test on *real* 32-bit content, please. You can generate some by u= sing the > > > > > tonemap filter, or any of the others that support it. > > > >=20 > > > > iam happy to test tonemap output but > > > > tonemap output is not "real content" either > > >=20 > > > tested the previous LRU code and this with ACES_OT_VWG run through to= nemap > > > this still performs better than the previous LRU code. > >=20 > > heres the test results, > > the try1 and try256 case try hardcoded mul values of 1 and 256, they > > perform worse than the automatically selected ones > > noremapstor simply does not store the remap table and thus shows how bi= g that > > table is (its quite huge with the tonemap output) > > the rest shows that the LRU code performs worse in every tested case > > that gz file is just a sanity check to ensure that we arent writing tons > > of low entropy data. > >=20 > > -rw-r----- 1 michael michael 694591360 Mar 21 21:57 float-303503-c1-m2= -s40-tmf32-nolsb-retrrr-noremapstor.nut > > -rw-r----- 1 michael michael 916492722 Mar 21 21:54 float-303503-c1-m2= -s40-tmf32-nolsb-retrrr.nut.gz > > -rw-r----- 1 michael michael 917135003 Mar 21 21:54 float-303503-c1-m2= -s40-tmf32-nolsb-retrrr.nut > > -rw-r----- 1 michael michael 921698263 Mar 21 22:03 float-303503-c1-m2= -s40-tmf32-nolsb-retrrr-try256.nut > > -rw-r----- 1 michael michael 921725671 Mar 21 22:04 float-303503-c1-m2= -s40-tmf32-nolsb-retrrr-tryLRU.nut > > -rw-r----- 1 michael michael 921729598 Mar 21 22:01 float-303503-c1-m2= -s40-tmf32-nolsb-retrrr-try1.nut > > -rw-r----- 1 michael michael 928459175 Mar 21 22:23 float-303503-c1-m2= -s40-tmf32-nolsb-retrrr-try-linear.nut > > -rw-r----- 1 michael michael 932903780 Mar 21 22:22 float-303503-c1-m2= -s40-tmf32-nolsb-retrrr-tryLRU-linear.nut > > -rw-r----- 1 michael michael 1100100630 Mar 21 22:24 float-303503-c1-m2= -s40-tmf32-nolsb-retrrr-try-gamma.nut > > -rw-r----- 1 michael michael 1101005617 Mar 21 22:22 float-303503-c1-m2= -s40-tmf32-nolsb-retrrr-tryLRU-gamma.nut > > -rw-r----- 1 michael michael 1150326564 Mar 21 22:23 float-303503-c1-m2= -s40-tmf32-nolsb-retrrr-try-hable.nut > > -rw-r----- 1 michael michael 1153310394 Mar 21 22:22 float-303503-c1-m2= -s40-tmf32-nolsb-retrrr-tryLRU-hable.nut >=20 > and of course my testing had a bug, i set the wrong remap mode >=20 > -rw-r----- 1 michael michael 694591360 Mar 21 21:57 float-303503-c1-m2-s= 40-tmf32-nolsb-retrrr-noremapstor.nut > -rw-r----- 1 michael michael 915326963 Mar 21 22:55 float-303503-c1-m3-s= 40-tmf32-nolsb-retrrr-tryLRU.nut > -rw-r----- 1 michael michael 917135003 Mar 21 21:54 float-303503-c1-m2-s= 40-tmf32-nolsb-retrrr.nut > -rw-r----- 1 michael michael 921698263 Mar 21 22:03 float-303503-c1-m2-s= 40-tmf32-nolsb-retrrr-try256.nut > -rw-r----- 1 michael michael 921729598 Mar 21 22:01 float-303503-c1-m2-s= 40-tmf32-nolsb-retrrr-try1.nut > -rw-r----- 1 michael michael 922576142 Mar 21 22:54 float-303503-c1-m3-s= 40-tmf32-nolsb-retrrr-tryLRU-linear.nut > -rw-r----- 1 michael michael 928459175 Mar 21 22:23 float-303503-c1-m2-s= 40-tmf32-nolsb-retrrr-try-linear.nut > -rw-r----- 1 michael michael 1100100630 Mar 21 22:24 float-303503-c1-m2-s= 40-tmf32-nolsb-retrrr-try-gamma.nut > -rw-r----- 1 michael michael 1114541572 Mar 21 22:52 float-303503-c1-m3-s= 40-tmf32-nolsb-retrrr-tryLRU-gamma.nut > -rw-r----- 1 michael michael 1150326564 Mar 21 22:23 float-303503-c1-m2-s= 40-tmf32-nolsb-retrrr-try-hable.nut > -rw-r----- 1 michael michael 1157209215 Mar 21 22:52 float-303503-c1-m3-s= 40-tmf32-nolsb-retrrr-tryLRU-hable.nut >=20 > so for the linear and none tonemaps, LRU is 0.2% and 0.6% better, for the= others its worse. > That said, for the 2D RLE only 1 mul value and only powers of 2 of that i= s searched. >=20 > That said, we can try other algorithms. My main goal ATM is though to keep > things simple, ideally 1 simple algorithm. > Both the LRU/RLE and 2d RLE algorithms are quite simple > the 2d RLE is very simple on the decoder side and on the encoder side its= the > encoders choice is it wants to optimize the parameters or just treat it l= ike > a normal RLE > also the 2d RLE allows us to tune the step (=3Dmul) for the whole range w= hich > allows storing various lower than 32 bit floating point formats efficient= ly > (thats tested and confirmed with that float16 input) > and we can also store a step per exponent which should allow storing fixed > point formats efficiently. (thats not tested) > For true float32 that value is just 1 and its just RLE. (which we confirm= ed > to work too now) >=20 > The same remap system can also be used with non float data like 16bit int= egers > where the mul value would allow very compact removial of any fixed pattern > in the LSB while allowing efficient coding of exceptions. >=20 > How much better than this simple way of storing the remap table can we do? > I dont know, maybe i have a new idea when i wake up tomorrow :) > but ATM i like it as its simple and seems to work well for its simplicity Also spend most of the day today working on this more. Ive simplified the remap decoder code, fixed a bunch of issues in the encoder and tested my heuristic that chooses the mul value vs brute force of all power of 2 values. Basically brute force is not worth the extra time. Also I failed to find any worthy gain from adjusting mul_count so while the code in the encoder looks complex ATM alot of that can be dropped later if for example we choose to never put mul_count > 1 into the specification and ATM it makes no sense to put that in as theres no significant gain with the material i tested The remap encoder code is also more flexible now as it can easily and clean= ly be run several times (to compare effects of parameters) I do intend to apply my current code for float32 support and the remap code in the next 24h (with whatever bugfixes it has at the time) thx [...] --=20 Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Whats the most studid thing your enemy could do ? Blow himself up Whats the most studid thing you could do ? Give up your rights and freedom because your enemy blew himself up. --EMT7SBGHGqDayLsM Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iF0EABEKAB0WIQSf8hKLFH72cwut8TNhHseHBAsPqwUCZ973KgAKCRBhHseHBAsP q3BDAJ4q9fDRhkGg2qY6cM0rOKlEWLQziQCgjzbfkFVbojQLatLD8G1+mZXDFss= =c4bJ -----END PGP SIGNATURE----- --EMT7SBGHGqDayLsM-- --===============9170598178088546418== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". --===============9170598178088546418==--