From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 3EB7449A0B for ; Mon, 26 Feb 2024 09:06:49 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9BDCD68C769; Mon, 26 Feb 2024 11:06:46 +0200 (EET) Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.11]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B4F8368C308 for ; Mon, 26 Feb 2024 11:06:39 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708938405; x=1740474405; h=from:to:subject:date:message-id:references:in-reply-to: content-transfer-encoding:mime-version; bh=LZn5q7wbTx/CwMmZtEsJD8yl7mC0opdjA5/fcSB8VYQ=; b=df+rCEo5z6Cn9b8gvVGI//NPnPjpS0H2CRQoOttbk0PaWpclQdQtis+7 TQzP5WN4O7xrCEH7qaCn2yAZJNP4ZqBVcTmW+CxNs2rauAHTEX1DX0JPf 5ZHgiUsnVaslPjwQbXpqAI3mPbmoCD+JLUuf8FE+y/wtjnirkLG/+K3VO akJXmMeN5/lntSVrGDRyN9n/fZ/r/8qJLiuAcicpycPV7YRYD3bms+yvI 0pvNmD146hrKqnbvZZ5imLNFhC1py0Da9JiFagwaBh67aDKtu4yRAIsGb +agiYfo/pevk6uFFUSkOSr2ghopHftxNkv2lqraTHLaxbtI3azkJnYbyM A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="13854565" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="13854565" Received: from orviesa010.jf.intel.com ([10.64.159.150]) by fmvoesa105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 01:06:37 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6595587" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by orviesa010.jf.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 26 Feb 2024 01:06:37 -0800 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 26 Feb 2024 01:06:36 -0800 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Mon, 26 Feb 2024 01:06:36 -0800 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.169) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Mon, 26 Feb 2024 01:06:36 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ihLDtC+FoFtPXYSq8nILRa9Wo9K5f+TqO+FqoXOwCg2rGy5f2OQWT77Q1+KNG8TV02URgCP8uU4sM5N5Z6AHgZMqfT9jraNk6iJC7NPq2Scszl5P6yV0P7bNJR51CPr0KZr1MlfH0OV2gxLoGf971QaDCEAzHmgsPd+LBF+iX1+9AMxQqcSmQNP+0VAGpzEJ1QtwtTVizZS6PJlzOP6jwyg1nqZRsMFKEOMIy24/XCT5JNiVRxzrySYJv1ijSTyDgsJRqm61uOBNeNONAY6ZDvp06rKlwJdzFKJnv67ocapP0yWnNG9IVB57h8TyzouKtIwlV+rF5kivZEF2bMFo5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TW8bsoCxPQ2hoWFkc2mYpXddayndT+g5j7AKN0Y4RJE=; b=XF8N5LGA//lg12YhcisfN9s1yPp/oNyXoLTsyGjf0BYklubvcPYyIDGmtQPtPmjGIgyy34VW9pfiksrsqeYHwGh45RZ9DtmqfEXKjP9ku+NuIwc4LuMMGNTyQ4YwuVpAFEGA7z77ak4jUMONaBVW8H8T/7WgcaG8pC1OxtubaxtAPJrdOT4vIPVJCAyUbbeJwCOiQFM6IExRogqB5XGsi4C92u1xTSmNu/gvzWPJG6N+BGhnuWKv5TvhHpMTQlzESSAaaLEfwCAknZPXPwKfQUVGj8s7dVwygKjpiNUaE3IYZrpFljjCTjegUCgEcXVxn3kTLa5Q8pi2fTboNGxmLw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from CH3PR11MB8659.namprd11.prod.outlook.com (2603:10b6:610:1cf::5) by SN7PR11MB7092.namprd11.prod.outlook.com (2603:10b6:806:29b::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.13; Mon, 26 Feb 2024 09:06:33 +0000 Received: from CH3PR11MB8659.namprd11.prod.outlook.com ([fe80::4c7a:ec78:f0:ac0a]) by CH3PR11MB8659.namprd11.prod.outlook.com ([fe80::4c7a:ec78:f0:ac0a%3]) with mapi id 15.20.7339.022; Mon, 26 Feb 2024 09:06:33 +0000 From: "Wu, Tong1" To: FFmpeg development discussions and patches Thread-Topic: [FFmpeg-devel] [PATCH v5 4/9] avcodec/vaapi_encode: extract rc parameter configuration to base layer Thread-Index: AQHaYqPJSMu6gw0/yEKYMHytjFcGUbEWJOcggACeqwCABZysQA== Date: Mon, 26 Feb 2024 09:06:33 +0000 Message-ID: References: <20240218084529.554-1-tong1.wu@intel.com> <20240218084529.554-4-tong1.wu@intel.com> <2cc1de88-09a9-41ff-b6fb-686e81027254@jkqxz.net> In-Reply-To: <2cc1de88-09a9-41ff-b6fb-686e81027254@jkqxz.net> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: CH3PR11MB8659:EE_|SN7PR11MB7092:EE_ x-ms-office365-filtering-correlation-id: ab6683e1-f3dc-4282-8faf-08dc36aa3abf x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Lf0fRAPOPhr/QblKVn4v3NvH1XlpqK6Nz42Na0q9uYhW/2x67pxEK9ZSv2Y4FWtncn8gDhDaLWpYOEt/RXl56Ijab9d+zQc2kOzSE/KB+xy596u9yaajw5ciU2HP4DpI2G3zd9f5JMF2BczIfvEUEFrao3dg5ENe0sAdFBbtv7p8l+gRQk3RULuDhaw68kMKRnYKEDQSpBp/3dzHKV+wb9VjOyMgxiqzOCtDjvoLs+/w1Dn+2ML+EaFt9Fh+SJ/OFCQRX4Q/wNe28MNlHXbszR5ftPIUfj3zIV13delrw3t6LgNbr78d91TbACsT21erz4+fcm5zfyoYOmhGh3ACff93/z+1aWQCbhPtoBYTlPyatelIBtvn1dk5N2sMLIsoN6jYcjzRlOkt89loFWEP2qDMupAPoubcK75uwvmAA8a0KkckK+1Yo2MMWoglv7rp5Hfl2L7n3rG/9mRc5vGzZaD4fNDJA+NYEHEKEJ039LK/Nf2KxlaghUl0Jc9DELBpAgl6pNVCyB36v8l8+vmciVnwihqi1A0BzGqX5oo9rI29j5sWTnscefORa8cBR9w+t1byyLp1gtTT4Gb5tSCHTRnA/D9tyCWWShZIIH0aA7IdlzSR/dsPglKB8nh2cGR4K1aQxS54JfF6rUMY9L0Apf+yo/9YKi5HLt5WT3E0iXTBeUVHHSckmIAmasx+2TuRIF7AvTcw2oyNrLbsFBo68g== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR11MB8659.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(38070700009); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?HxiNTKP8QMlAz+rNr7kfhHtbI2lOqiOR7z0uv0YDrP8s3VBTKGtzUfyXdhmy?= =?us-ascii?Q?uInI5bfZDPHZQc7EkFbskSjpmzZ5XtD1f3FwfV35nti6sknTQv+yjkOdfpzk?= =?us-ascii?Q?X7KRey4W0pTIOFXaFd8pCp7PIubIqKtqJ+S5eYCfpr05XpcCgkS3saQANukQ?= =?us-ascii?Q?ewkxm5WqLikJlMfnoH8/aEoif1Ol/g3JF+fB7KJNPIwzVRpInG0chQI+wnPt?= =?us-ascii?Q?QwfQ/OpEGtCEBgxwhrpwNoF7Z2sQb5IN6PQRozZQrxWifbURQ4Ndes9ce3pC?= =?us-ascii?Q?7ArUs3IL7Weviw910tGOdLCQwak5LsYzXJUt9v6mENoVAp4p0jA0j62RYoJJ?= =?us-ascii?Q?3/7NIE4w+BWQ2ZB7EHbkQZtKCRXm0c3x7ycZtgzLtW//ej1qIlMRwIQ+ri9u?= =?us-ascii?Q?YWKNaVcrlIuzg4HusR7hAyjl7NyGrfZaYn6D9Rstx5MDpueqWUZitRTeUWE3?= =?us-ascii?Q?PeJ5BwPZHQFTOCMwRzZZzRHA4CKMCu3Asz+1SeSH7Q6dJEJtrSKLl10t4AcM?= =?us-ascii?Q?6M/RQOvkqDtPW5Ci0TDLmkp41rl2pMwXXwTjEXGFef7OGZfS88iUENRZAzxp?= =?us-ascii?Q?bhq1kS6K/jTh551gsVP+bhosEO83LxPywHOhLMFS1PUEAiPL2MeHzRQWMNHP?= =?us-ascii?Q?Mihq6ZI9HkAW93eHv0AfGm0vH8SZxl1uyUgZYjaJrV7NYpTWgIM7oeRUmso4?= =?us-ascii?Q?gTFoItkv820Sy4J+orQliyPRMLX2coVjcMFxaJHtm88vniVNAVImDvT2dE7l?= =?us-ascii?Q?9932MY1HxATRNCCdPcaLj8U4uLamnp0J4Tap1Ncqot71Wws70S5k66Ocnw2v?= =?us-ascii?Q?iM+D+Rj40hlT7QTuDmj+muxqSNrqNc4h/wb5QrBnLOs2NxBFNMzaEpB0tK5J?= =?us-ascii?Q?t5m/SsXcXjJV3FmnHI8kfUnbgY2MxGfB4D+opRnUjsNQXJT7IeJ51ebXwQiy?= =?us-ascii?Q?5ktY9IZEQsSinEdoQzTEKxMvjoK6mD7+J8tddkizTS4+vlk+uuT6PEabiTmZ?= =?us-ascii?Q?Lcg0pQePaAt9FufKjrkL3rd51TFcOxRIMT4tg8ja/gDIjhfi8yPx6H+he9oO?= =?us-ascii?Q?5U5twYMuCPQImHYqzQj2GNk+rAUnoorjC3NBjXy0DzslMHV9XW+rJEpKUE8u?= =?us-ascii?Q?hCEkwF/UUuj4E9q1eQeAjpsiconKPfeddL3YJtrDN8zEsCoVuQznKcl0CBqe?= =?us-ascii?Q?S3Ay9LwTpByALxtM4Fn2Nighl3PKzg3GZKxmvPnwyI04hhANuVzUW1F/3FED?= =?us-ascii?Q?EU2jdHny0IQINmSW+jq1Rk60g1W9yEIwq7+JM203KJ9LxkeqN5wkAxOJA05h?= =?us-ascii?Q?/ZIMjw0SunjWJvt0V3zAvpoG7qhEEI0H0yHJjPTa36k7+MuvU3tIaJqiPF0n?= =?us-ascii?Q?pawHIpAqElgG06pY74XPWHhduZ0u0j906BwMGfY+Ga+ZJ6q4rZ4s/dXxhCgd?= =?us-ascii?Q?IcShgUF8JHGAcYQLALbrrlsonvAx4SJdKtNYluAEOdzphDOsVkcdk4sdWU+y?= =?us-ascii?Q?v9Lyo5WIOmYTdKQHafTDxo5UUQvdSUwt/BDigMzfeCTx12np7xcV3iZhOBy2?= =?us-ascii?Q?w4JJ7Xp6g/rKB2SpKoQ=3D?= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CH3PR11MB8659.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: ab6683e1-f3dc-4282-8faf-08dc36aa3abf X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Feb 2024 09:06:33.7257 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ON0mmkOmyI7ir4B9FHKq2RtWxtroBwQT3gJl0lEPx4+9Q9z5CKIdiIgpXkQLBBtxUvNDZZOoWIo1A0H73N/M/w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR11MB7092 X-OriginatorOrg: intel.com Subject: Re: [FFmpeg-devel] [PATCH v5 4/9] avcodec/vaapi_encode: extract rc parameter configuration to base layer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: >-----Original Message----- >From: ffmpeg-devel On Behalf Of Mark >Thompson >Sent: Friday, February 23, 2024 3:22 AM >To: ffmpeg-devel@ffmpeg.org >Subject: Re: [FFmpeg-devel] [PATCH v5 4/9] avcodec/vaapi_encode: extract rc >parameter configuration to base layer > >On 22/02/2024 10:10, Wu, Tong1 wrote: >>> On 18/02/2024 08:45, tong1.wu-at-intel.com@ffmpeg.org wrote: >>>> From: Tong Wu >>>> >>>> VAAPI and D3D12VA can share rate control configuration codes. Hence, it >>>> can be moved to base layer for simplification. >>>> >>>> Signed-off-by: Tong Wu >>>> --- >>>> libavcodec/hw_base_encode.c | 151 ++++++++++++++++++++++++ >>>> libavcodec/hw_base_encode.h | 34 ++++++ >>>> libavcodec/vaapi_encode.c | 210 ++++++--------------------------- >>>> libavcodec/vaapi_encode.h | 14 +-- >>>> libavcodec/vaapi_encode_av1.c | 2 +- >>>> libavcodec/vaapi_encode_h264.c | 2 +- >>>> libavcodec/vaapi_encode_vp9.c | 2 +- >>>> 7 files changed, 227 insertions(+), 188 deletions(-) >>>> >>>> ... >>>> diff --git a/libavcodec/hw_base_encode.h >b/libavcodec/hw_base_encode.h >>>> index b836b22e6b..4072b514d3 100644 >>>> --- a/libavcodec/hw_base_encode.h >>>> +++ b/libavcodec/hw_base_encode.h >>>> @@ -72,6 +72,37 @@ enum { >>>> RC_MODE_MAX = RC_MODE_AVBR, >>>> }; >>>> >>>> +typedef struct HWBaseEncodeRCMode { >>>> + // Mode from above enum (RC_MODE_*). >>>> + int mode; >>>> + // Name. >>>> + const char *name; >>>> + // Uses bitrate parameters. >>>> + int bitrate; >>>> + // Supports maxrate distinct from bitrate. >>>> + int maxrate; >>>> + // Uses quality value. >>>> + int quality; >>>> + // Supports HRD/VBV parameters. >>>> + int hrd; >>>> +} HWBaseEncodeRCMode; >>>> + >>>> +typedef struct HWBaseEncodeRCConfigure >>>> +{ >>>> + int64_t rc_bits_per_second; >>>> + int rc_target_percentage; >>>> + int rc_window_size; >>>> + >>>> + int rc_quality; >>>> + >>>> + int64_t hrd_buffer_size; >>>> + int64_t hrd_initial_buffer_fullness; >>>> + >>>> + int fr_num; >>>> + int fr_den; >>>> +} HWBaseEncodeRCConfigure; >>> >>> The set of fields here needs more thought to match up the common parts >of >>> the APIs. >>> >>> Just have target rate and maxrate and let VAAPI deal with the percentage >stuff >>> maybe? Not sure what to do with window_size which isn't present at all in >>> D3D12. The convergence/accuracy stuff for VAAPI AVBR is also unclear. >>> >> >> Could you please explain more about your thoughts on why the percentage >stuff should not be in the common part? I think D3D12 can share the same >code in terms of percentage stuff and hrd stuff. I can let VAAPI do the AVBR >stuff and remove window_size from common part and keep everything else >as-is. > >The libavcodec API is a better match for D3D12 than VAAPI is: e.g. for D3D12 >VBR you have TargetAvgBitRate = AVCodecContext.bit_rate, PeakBitRate = >AVCodecContext.rc_max_rate etc. with all fields nicely matching. > >If you use the VAAPI fields here then you are translating the rate into a >percentage (with rounding error because it's an integer) and then back again >for no reason, hence I think you should prefer the libavcodec/D3D12 fields and >do the conversion for VAAPI in its own code. OK, that makes sense. I think if that's the case, probably it's not necessary to have this patch to extract the code to a common function. Just let VAAPI and D3D12 do their own jobs separately. I'm going to drop this patch from this patchset. > >>> (Do you know why QVBR is missing the VBV parameters in D3D12? On the >face >>> of it that doesn't make any sense, but maybe I'm missing something.) >>> >> >> It is presented in QVBR1 structure(defined in DirectX-header but not yet in >Windows SDK). >> I can add this support afterwards maybe along with AV1 implementation. > >Ah, so it was a bug and has been fixed. That's good! > >>>> + >>>> + >>>> typedef struct HWBaseEncodePicture { >>>> struct HWBaseEncodePicture *next; >>>> >>>> @@ -242,6 +273,9 @@ int >>> ff_hw_base_encode_set_output_property(AVCodecContext *avctx, >>> HWBaseEncodePic >>>> >>>> int ff_hw_base_encode_receive_packet(AVCodecContext *avctx, >AVPacket >>> *pkt); >>>> >>>> +int ff_hw_base_rc_mode_configure(AVCodecContext *avctx, const >>> HWBaseEncodeRCMode *rc_mode, >>>> + int default_quality, HWBaseEncodeRCConfigure >*rc_conf); >>>> + >>>> int ff_hw_base_encode_init(AVCodecContext *avctx); >>>> >>>> int ff_hw_base_encode_close(AVCodecContext *avctx); >>>> diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c >>>> ... >>> >>> Thanks, >>> >>> - Mark >> >> Thanks for your review and I'll reply to your comments regarding D3D12 >HEVC encoder patch later since there're indeed a lot of stuff you pointed out >that I need to take care of again. > >Sure, thank you! > >- Mark >_______________________________________________ >ffmpeg-devel mailing list >ffmpeg-devel@ffmpeg.org >https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > >To unsubscribe, visit link above, or email >ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".