From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.ffmpeg.org (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id 7AAFC4BFA3 for ; Fri, 23 May 2025 00:57:29 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTP id 2277568D481; Fri, 23 May 2025 03:57:25 +0300 (EEST) Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [217.70.183.196]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTPS id 4801968C2F6 for ; Fri, 23 May 2025 03:57:18 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id 8478943AF0 for ; Fri, 23 May 2025 00:57:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niedermayer.cc; s=gm1; t=1747961837; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=lUzW0mD3VQx4urCy1745wD1n1r43BuTY9TBCD6WLDuw=; b=ScjWfHsTv3QhpQQ811/X2Xxkopvan1UMhnODFhIlDy1eq819Jjj3AZ66hNgP49WZLWKbLw CSbVsq54ivmwshmqguGDFI2xESizS0nGDHWFBQO0tKPnujeBxHNI+kODPNGq44EF5dZhIi NHHw6STYXQ6EfUUFCMHJtCbarJ+cBEhrFddaIp4kWEVr8/88pA/M5xpJxQKynN3uYVdHXk j3JI24Ks/1dyM8I+KvR6XidHLeXUCgvVuP+LaZ3oJFCUJXlM9q8813ONIzW7K2q53ig+li 8YfeUaAAiSFigdaPgr0vCz+WIgzLKMtUmJbbid0ndG/AgYc6zAFP0euQnpJs7w== Date: Fri, 23 May 2025 02:57:16 +0200 From: Michael Niedermayer To: FFmpeg development discussions and patches Message-ID: <20250523005716.GV29660@pb2> References: <20250523004559.GU29660@pb2> MIME-Version: 1.0 In-Reply-To: <20250523004559.GU29660@pb2> X-GND-State: clean X-GND-Score: -85 X-GND-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtddtgdejgeehucdltddurdegfedvrddttddmucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuifetpfffkfdpucggtfgfnhhsuhgsshgtrhhisggvnecuuegrihhlohhuthemuceftddunecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenfghrlhcuvffnffculdduhedmnecujfgurhepfffhvffukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpefoihgthhgrvghlucfpihgvuggvrhhmrgihvghruceomhhitghhrggvlhesnhhivgguvghrmhgrhigvrhdrtggtqeenucggtffrrghtthgvrhhnpeelkeeggfffiedufeejueffjeduhedttdduledtheevveevtdeiueelhfdtuedtkeenucfkphepgedurdeiiedrieejrdduudefnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehinhgvthepgedurdeiiedrieejrdduudefpdhhvghloheplhhotggrlhhhohhsthdpmhgrihhlfhhrohhmpehmihgthhgrvghlsehnihgvuggvrhhmrgihvghrrdgttgdpnhgspghrtghpthhtohepuddprhgtphhtthhopehffhhmphgvghdquggvvhgvlhesfhhfmhhpvghgrdhorhhg X-GND-Sasl: michael@niedermayer.cc Subject: Re: [FFmpeg-devel] [FEATURE PROPOSAL] Extracting codec-level data to binary files X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: multipart/mixed; boundary="===============8281849307183297764==" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: --===============8281849307183297764== Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="8iwxiGWjJkDeRCkD" Content-Disposition: inline --8iwxiGWjJkDeRCkD Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, May 23, 2025 at 02:45:59AM +0200, Michael Niedermayer wrote: > Hi Ronald >=20 > On Thu, May 22, 2025 at 07:59:06AM -0400, Ronald S. Bultje wrote: > > Hi, > >=20 > > On Wed, May 21, 2025 at 9:34=E2=80=AFAM Timoth=C3=A9e < > > timothee.informatique@regaud-chapuy.fr> wrote: > >=20 > > > Hello, > > > > > > I am interested in expanding ffmpeg's capabilities to extract > > > low-level data from video codecs. Specifically, I'd like to implement > > > functionality that would allow exporting frame data, macroblock > > > information, quantization tables, and similar codec-specific elements > > > to binary files for further analysis. > > > > > > After searching through the documentation and existing features, I > > > haven't found similar functionality, though I may have missed > > > something. Has this been implemented before, or are there related > > > features I should examine? > >=20 > >=20 > > Some older codecs implement minor variants for this, e.g. grep > > for AV_FRAME_DATA_MOTION_VECTORS, which attaches a frame's motion vecto= rs > > to the picture data. I believe there's an example app and possibly a fi= lter > > to overlay MVs on top of the video frame based on this concept. You cou= ld > > extend this to cover other (macro)block info. There used to be a varian= t of > > this for quant-tables also but I can't find it, maybe it was removed. >=20 > For motion vectors: > ./ffplay -flags2 +export_mvs -i matrixbench_mpeg2.mpg -vf codecview=3Dmv= =3Dpf+bf+bb >=20 > For macroblock segmentation and type vissualization + also motion vectors: > ffplay-3.4.13 -debug vis_mb_type matrixbench_mpeg2.mpg -vf codecview=3Dm= v=3Dpf+bf+bb >=20 > For QP vissualization + also motion vectors: > ffplay-3.4.13 -debug vis_qp matrixbench_mpeg2.mpg -vf codecview=3Dmv=3Dp= f+bf+bb >=20 > For qp values dumped on the console > ./ffplay -debug qp -i matrixbench_mpeg2.mpg And this can easily be extended to other codecs, ATM it should work with all 16x16 MB based codecs like msmpeg4*/wmv*/mpeg1/2/4/h263/h264 mbtype and qp vissualization need codecview to be extended or versions arou= nd 3.4 which implemented it differently Implementing vissualization as done currently with sidedata and codecview is simple and efficient. It also would allow exporting the data to json by writing a codec2json filter in place of codecview Also all decoders already have all this data parsed and available so its simpler than trying to do it in a decoder independant way I would thus suggest implementations of this for modern codecs to follow the same path as the existing code. thx [...] --=20 Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB "You are 36 times more likely to die in a bathtub than at the hands of a terrorist. Also, you are 2.5 times more likely to become a president and 2 times more likely to become an astronaut, than to die in a terrorist attack." -- Thoughty2 --8iwxiGWjJkDeRCkD Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iF0EABEKAB0WIQSf8hKLFH72cwut8TNhHseHBAsPqwUCaC/H6AAKCRBhHseHBAsP q+aTAJ9Cp7Pqrb1JzYtYjoJYK65/adPyyACeNS/DILstWwdKZNJ80elBuMIadtk= =mQxP -----END PGP SIGNATURE----- --8iwxiGWjJkDeRCkD-- --===============8281849307183297764== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". --===============8281849307183297764==--