From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.ffmpeg.org (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id 0FE8F4AC1A for ; Fri, 18 Jul 2025 15:49:05 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTP id 63F7868D81A; Fri, 18 Jul 2025 18:49:01 +0300 (EEST) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTPS id 94E5868CFA8 for ; Fri, 18 Jul 2025 18:48:54 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id A1F8A442A2 for ; Fri, 18 Jul 2025 15:48:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niedermayer.cc; s=gm1; t=1752853733; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=6+V75xnjWLt6r5BzT2TS9PyE0VnrxHR9hP5VgQEDRYQ=; b=Uuzf4E0sHZrr302EnzjHFzhcsH5YEezFLj9muhxytBQ0Nf6kjbgZ8wMMiEjldGnY1Hha1l YsYRZ+osOxRfCtIoDA7u7mW8X7+8iPlgfbEJzmRkfpslmHr2mPFaNQZy8E8MnLmQREBTWD S2O8F7q0LBv3MBH6GCjjYB3vIZg4JKXB9u3dAsMakoqu8IhF+VuAvQ6ItYaAThwZnMcdGd lQ+JldZM70To3vYyMShArihIdp9Z6ohu1H6XGbfsFh4uQnrrrajNgI+I0cGiVvDXbNsaG/ 76Ac9ORWpzjsHlRXg0ht6AmHp7Oe1ZYyB5RfGxpz8vsBnePQb7JqnKQkNFBenw== Date: Fri, 18 Jul 2025 17:48:52 +0200 From: Michael Niedermayer To: FFmpeg development discussions and patches Message-ID: <20250718154852.GC29660@pb2> References: <20250718103055.1172733-1-timothee.informatique@regaud-chapuy.fr> MIME-Version: 1.0 In-Reply-To: <20250718103055.1172733-1-timothee.informatique@regaud-chapuy.fr> X-GND-State: clean X-GND-Score: -85 X-GND-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdefgdeifeekiecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfitefpfffkpdcuggftfghnshhusghstghrihgsvgenuceurghilhhouhhtmecufedtudenucesvcftvggtihhpihgvnhhtshculddquddttddmnegfrhhlucfvnfffucdludehmdenucfjughrpeffhffvuffkfhggtggujgesghdtreertddtudenucfhrhhomhepofhitghhrggvlhcupfhivgguvghrmhgrhigvrhcuoehmihgthhgrvghlsehnihgvuggvrhhmrgihvghrrdgttgeqnecuggftrfgrthhtvghrnhepudetvdfhudeuudegudefgfehhfevvdfggfffkefhvdfgvdetffdtjeekheetfeehnecukfhppeeguddrieeirdeihedrudejieenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepihhnvghtpeeguddrieeirdeihedrudejiedphhgvlhhopehlohgtrghlhhhoshhtpdhmrghilhhfrhhomhepmhhitghhrggvlhesnhhivgguvghrmhgrhigvrhdrtggtpdhnsggprhgtphhtthhopedupdhrtghpthhtohepfhhfmhhpvghgqdguvghvvghlsehffhhmphgvghdrohhrgh X-GND-Sasl: michael@niedermayer.cc Subject: Re: [FFmpeg-devel] [PATCH 1/4] avutil: add generic side data for video coding info X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: multipart/mixed; boundary="===============2592396811693302550==" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: --===============2592396811693302550== Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="3YanZ7dyj/TysNsI" Content-Disposition: inline --3YanZ7dyj/TysNsI Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi On Fri, Jul 18, 2025 at 12:30:52PM +0200, Timoth=E9e Regaud wrote: > From: Timothee Regaud >=20 > Adds the generic data structures to libavutil. The design is recursive to= support other codecs, even though the implementation is only for H.264 for= now. >=20 > Signed-off-by: Timothee Regaud > --- > libavutil/Makefile | 1 + > libavutil/frame.h | 7 ++ > libavutil/side_data.c | 1 + > libavutil/video_coding_info.h | 163 ++++++++++++++++++++++++++++++++++ > 4 files changed, 172 insertions(+) > create mode 100644 libavutil/video_coding_info.h >=20 > diff --git a/libavutil/Makefile b/libavutil/Makefile > index 94a56bb72f..44e51ab7ae 100644 > --- a/libavutil/Makefile > +++ b/libavutil/Makefile > @@ -93,6 +93,7 @@ HEADERS =3D adler32.h = \ > tree.h \ > twofish.h \ > uuid.h \ > + video_coding_info.h \ > version.h \ > video_enc_params.h \ > xtea.h \ > diff --git a/libavutil/frame.h b/libavutil/frame.h > index c50cd263d9..f4404472a0 100644 > --- a/libavutil/frame.h > +++ b/libavutil/frame.h > @@ -254,6 +254,13 @@ enum AVFrameSideDataType { > * libavutil/tdrdi.h. > */ > AV_FRAME_DATA_3D_REFERENCE_DISPLAYS, > + > + /** > + * Detailed block-level coding information. The data is an AVVideoCo= dingInfo > + * structure. This is exported by video decoders and can be used by = filters > + * for analysis and visualization. > + */ > + AV_FRAME_DATA_VIDEO_CODING_INFO, > }; > =20 > enum AVActiveFormatDescription { > diff --git a/libavutil/side_data.c b/libavutil/side_data.c > index fa2a2c2a13..b938ef6f52 100644 > --- a/libavutil/side_data.c > +++ b/libavutil/side_data.c > @@ -56,6 +56,7 @@ static const AVSideDataDescriptor sd_props[] =3D { > [AV_FRAME_DATA_SEI_UNREGISTERED] =3D { "H.26[45] User Dat= a Unregistered SEI message", AV_SIDE_DATA_PROP_MULTI }, > [AV_FRAME_DATA_VIDEO_HINT] =3D { "Encoding video hi= nt", AV_SIDE_DATA_PROP_SIZE_DEPENDENT }, > [AV_FRAME_DATA_3D_REFERENCE_DISPLAYS] =3D { "3D Reference Disp= lays Information", AV_SIDE_DATA_PROP_GLOBAL }, > + [AV_FRAME_DATA_VIDEO_CODING_INFO] =3D { "Video Coding Info= ", AV_SIDE_DATA_PROP_SIZE_DEPENDENT }, > }; > =20 > const AVSideDataDescriptor *av_frame_side_data_desc(enum AVFrameSideData= Type type) > diff --git a/libavutil/video_coding_info.h b/libavutil/video_coding_info.h > new file mode 100644 > index 0000000000..17e9345892 > --- /dev/null > +++ b/libavutil/video_coding_info.h > @@ -0,0 +1,163 @@ > +/* > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1= 301 USA > + */ > + > +#ifndef AVUTIL_VIDEO_CODING_INFO_H > +#define AVUTIL_VIDEO_CODING_INFO_H > + > +#include > +#include > + > +/** > + * @file > + * @ingroup lavu_frame > + * Structures for describing block-level video coding information. > + */ > + > +/** > + * @defgroup lavu_video_coding_info Video Coding Info > + * @ingroup lavu_frame > + * > + * @{ > + * Structures for describing block-level video coding information, to be > + * attached to an AVFrame as side data. > + * > + * All pointer-like members in these structures are offsets relative to = the > + * start of the AVVideoCodingInfo struct to ensure the side data is > + * self-contained and relocatable. This is critical as the underlying bu= ffer > + * may be moved in memory. > + */ > + > +/** > + * Structure to hold inter-prediction information for a block. > + */ > +typedef struct AVBlockInterInfo { > + /** > + * Offsets to motion vectors for list 0 and list 1, relative to the > + * start of the AVVideoCodingInfo struct. > + * The data for each list is an array of [x, y] pairs of int16_t. > + * The number of vectors is given by num_mv. > + * An offset of 0 indicates this data is not present. > + */ > + size_t mv_offset[2]; int16 is not enough, with growing picture sizes and growing precission of motion vectors also the MV precssion is needed somewhere somehow or they could not be vissualized by generic code > + > + /** > + * Offsets to reference indices for list 0 and list 1, relative to t= he > + * start of the AVVideoCodingInfo struct. > + * The data is an array of int8_t. A value of -1 indicates the refer= ence > + * is not used for a specific partition. > + * An offset of 0 indicates this data is not present. > + */ > + size_t ref_idx_offset[2]; > + /** > + * Number of motion vectors for list 0 and list 1. > + */ > + uint8_t num_mv[2]; > +} AVBlockInterInfo; weighted bi pred needs the weights too and for more than 1 MV, the question becomes what the other vectors are, bipred ?, affine MC ?, ... Also if you want to be really generic you need to allow blocks that dont span accross the luma and chroma planes but allow different block structures (and motion vectors) per plane Iam not sure how generic we want to be and how useful that is. But it seemes you want your patch to be quite generic ? I think its more important to allow this to be extensible than suporting everything we can think of. That is maybe store the size of the struct also somewhere so that elements can be added to their end without breaking anything. At least for the main block structure I mean a future codec might allow non rectangular blocks but we dont want to think about that today. Maybe its best to keep this as simple as possible but extensible > + > +/** > + * Structure to hold intra-prediction information for a block. > + */ > +typedef struct AVBlockIntraInfo { > + /** > + * Offset to an array of intra prediction modes, relative to the > + * start of the AVVideoCodingInfo struct. > + * The number of modes is given by num_pred_modes. > + */ > + size_t pred_mode_offset; > + > + /** > + * Number of intra prediction modes. > + */ > + uint8_t num_pred_modes; > + > + /** > + * Chroma intra prediction mode. > + */ > + uint8_t chroma_pred_mode; > +} AVBlockIntraInfo; classifying the predition in directional, DC, and non directional and for directional the direction. Could be usefull. Otherwise the prediction mode number requires codec specific knowledge to interpret thx [...] --=20 Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB While the State exists there can be no freedom; when there is freedom there will be no State. -- Vladimir Lenin --3YanZ7dyj/TysNsI Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iF0EABEKAB0WIQSf8hKLFH72cwut8TNhHseHBAsPqwUCaHps3QAKCRBhHseHBAsP q8WrAJ4k2rl9LVphCzmypBx5olOKegvl3wCfWGHYL1J/nMiL7cQD3w3dAuVJ4TY= =yYsh -----END PGP SIGNATURE----- --3YanZ7dyj/TysNsI-- --===============2592396811693302550== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". --===============2592396811693302550==--