From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id C5DA2453D5 for ; Sat, 28 Jan 2023 08:37:07 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E905268A0BF; Sat, 28 Jan 2023 10:37:03 +0200 (EET) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5E0A968BD65 for ; Sat, 28 Jan 2023 10:36:56 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1674895021; x=1706431021; h=from:to:subject:date:message-id:references:in-reply-to: content-id:content-transfer-encoding:mime-version; bh=0LqyNnHfdozKtLhuCdfZJs0CDkKBl6DYWsoaa9veAZM=; b=F/R21gpHwN+wjctoR9iltOOQYccoRD1p6+xxa88nGXHwledoRhLBwLmu WMccTsbAZntbNHPiw6WaBQYrkVgfJ00VL+/mHNy5wmKqjVzIz/0bdJkA+ 7dsrTYcoOvJGSlS2cvtkxgXgtSd4LhYDeUVzFnKfBTFQl2lrgBnCBG2yZ 9n+BQR6combl6FQuhF6LgoT+JDDLPart/ERA7/e5C+I6vR6p6f7uzo9Oo na4GoLWUXJNBRrYgfBU+Wp7LJnwCEXBGOmTj6LDb3NaeN5jCfwPWaOSJw BfQntS8duST82VhHUwyygSeyw+S9lN+i6eGGyMJQa2IXEnFyJRgiYT4Hh Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10603"; a="389654644" X-IronPort-AV: E=Sophos;i="5.97,253,1669104000"; d="scan'208";a="389654644" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jan 2023 00:36:54 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10603"; a="752256518" X-IronPort-AV: E=Sophos;i="5.97,253,1669104000"; d="scan'208";a="752256518" Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by FMSMGA003.fm.intel.com with ESMTP; 28 Jan 2023 00:36:53 -0800 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16; Sat, 28 Jan 2023 00:36:53 -0800 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16 via Frontend Transport; Sat, 28 Jan 2023 00:36:53 -0800 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.169) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.16; Sat, 28 Jan 2023 00:36:52 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DBljcxCg4mbJgK2N50JnKFeFZU37i+08mbd/PmgqbVlZK+uQC1yz9Ou3hN3RWlS+Te6C1fkzva6z3NiP7MEVOI6oRqBtPKKOJxgbdKMhtWtDfGEg6LwjtaFjtxitix99c73r7fOWYIGtV7bBCXCOA+unUJwnn7H8Qwfx+TQ0LNl9cnlmKa2jrEi/qCCo1sTjjAK+cmBJcIwd2QfbZiOCHk03II3N1kqvU3n56W3p0N8wDQhzgMZiw1wOqAojF7F2aYRPutTrnPbrtQlvgQOAJXZd53p3WhG4Y1M+mkzf1kSDxMq+KHPMt1xqZkUeWKfO+gvlqLAroomOF9h4CwiLJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0LqyNnHfdozKtLhuCdfZJs0CDkKBl6DYWsoaa9veAZM=; b=D2XrF+xwt1r2RHNWENw/GL3GWFdXWrqla2N9Y+or34ZZboLJ6AFrZfacg7zQMOuud+QcvTXifYUcA69cwO/DSprterGe3InApr9H45kYy2xTP1mWgMVZn43SQdnuIL8gs9uaVb9tEIciuZlwG3buO7P2nM6Eqr/TW6RMmXfyCq8SadPqZUUa/3Q6NKxdAQT5sp5l1ypaXDcA2JDjm/tMS9vn4yke2OkACynzwK11TkX8c9/cEMY6TO+IQgnrZz1zyGMTwIRIcMUFGoISwPGauz3KzO/cfwBC0x/ztknZFe1/xVOWisol418/NWTFLUv7V7aNZiWY23rVPddmdV8FzA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from BN9PR11MB5515.namprd11.prod.outlook.com (2603:10b6:408:104::8) by IA1PR11MB7822.namprd11.prod.outlook.com (2603:10b6:208:3f2::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6043.28; Sat, 28 Jan 2023 08:36:50 +0000 Received: from BN9PR11MB5515.namprd11.prod.outlook.com ([fe80::36ba:13c8:6a29:c7af]) by BN9PR11MB5515.namprd11.prod.outlook.com ([fe80::36ba:13c8:6a29:c7af%7]) with mapi id 15.20.6043.028; Sat, 28 Jan 2023 08:36:50 +0000 From: "Xiang, Haihao" To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [FFmpeg-devel][PATCH] avfilter: add VA-API variants of the stack filters Thread-Index: AQHZK+BUkmHAS1/1x06sjzaDpyARNa6zj48A Date: Sat, 28 Jan 2023 08:36:49 +0000 Message-ID: References: <20230119083006.3697233-1-haihao.xiang@intel.com> In-Reply-To: <20230119083006.3697233-1-haihao.xiang@intel.com> Accept-Language: en-AS, zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Evolution 3.36.5-0ubuntu1 authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: BN9PR11MB5515:EE_|IA1PR11MB7822:EE_ x-ms-office365-filtering-correlation-id: 68f90863-659e-45f9-f489-08db010accae x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: j3i30/pWFAK8h6e9BjHD2luEgxyu/WqTt5qb7D0gwjB+d0fHIXmsVfiBvI6BrZ0yJ+nm8pMrzXklq+CF8LFSAnUniJ+CkV7jSv8eE27bgQqmMcT9JGLF5YHbRTYOo/bllbH8Jsu/S5tcXR9jYCt7oZ2SBPzkfmGFgiGJ2/Gn/RxV+VfZR1Szbpxb7d60BPP9yPPEQuF9CCLsivoBlp1i+ZTmIEAzFkkaWlP4F3Qj4DgPNvSiRYA0DRzcyAukI/g+SWanK2WgE/HWqkFYk4iKK0gpiByXL0kWi8dLzsgFWL8rd7xYV9+UXcLYkVz0I7JLo1Jl/UXI4UfGOF9bQcTTazDHaktS6Q1+AaXY9yPTRPqAd3kqBOSXe6wcCqV0a7w6p0+Y6sq5OF4L9B25gRYYfUBeFefQQJy5OYe7OqrevpA22DOJILVcgzle+ZigYoLLm7qUow7YD0+Z6r3inSTowMNrdzXb2Y1cLP8HqakXz+mw+3twTTd+QK14GZmwpMU92QZDuK66wL/XwBVl6kvQ3gHziYMalCXewnoVfUnvIBSSDFmXK+PX2shviPaAfwjMPi7gIHimASG+i7wJhCEsLpDOEdPoD7f6SKVSkrV6ZjYVlYIgnRYDymIlFV26/dZio2VUBGAAoLPxPznPcEF69wI30NErJo/deh08RQi33JLLxh7CPmp66fnhZQp14/7j9sKEYrOzIKWQUb2MlgTdK8ivH68d1TVIPq3IQWqWMXo= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN9PR11MB5515.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230025)(376002)(366004)(136003)(39860400002)(346002)(396003)(451199018)(8936002)(478600001)(66446008)(64756008)(2906002)(5660300002)(6916009)(8676002)(30864003)(41300700001)(91956017)(66476007)(66556008)(66946007)(76116006)(316002)(71200400001)(6486002)(6506007)(83380400001)(186003)(26005)(6512007)(2616005)(36756003)(86362001)(82960400001)(122000001)(38100700002)(38070700005)(2004002)(579004)(559001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?B?dTBlQWIrQkZxZGQzaVRTaGwxR3FtMEl2RjlaS1hXYk1TKzIwUEpuOHVzUXR5?= =?utf-8?B?Y29zWTdjNy9RTkx6dHBMTU1zWmFPNjF2cTA2M3pZbk8xeU9FRlYxMTZuUlhQ?= =?utf-8?B?OUZmYlVaaGsyWitMcWFMZGFDdndjOEtvaE9NVmkrUEJ4L2JUSXRYbkpNR293?= =?utf-8?B?K1RjR0djUCtSRjE2Uzk2bmMrNXdHVXI2enY5TjV4ek4vdjVndVFBaHhoMU9N?= =?utf-8?B?TXU3Q0EwWTBoZzdDNmp2R3NkQjlLV0F0U2F4YXlvQTJGaEhTdnM1cERhKzgz?= =?utf-8?B?bFhUS1pJNlFHYmtsZ05DamxUd3c0MXEzc3dhY2o5VTBHK29kVTBEUmsydXVP?= =?utf-8?B?SzN4eEllQjNKbVo5SkJ6K2lrU0lMQmRNUk5RU2tVMFFvNUVablBBUkF0S1BE?= =?utf-8?B?Y3hrYlhrQk9nTVBxakVBYmV2bnpYUU1DYS9wem0vY3RuaTBpeDFhYjZtY0lS?= =?utf-8?B?QlEyLzVYRWNWM2xFb1lPS0VKb01WMkxtaGJvOE02bVBQYVJtMG5IYUdIRE1P?= =?utf-8?B?WXpLYTNPS0s0Vmp3eXFZcnN1YmJ1NHZpRUcyZ0NoRDdoSFZheVRVSTJGYUtN?= =?utf-8?B?N1l5SWRBRXh5eWU3QWZSNUdlYnY1dW41TjhURFlxWDJTOENNOHhwOTVDczlh?= =?utf-8?B?c2lCU09CQkR0V2FaTkdOVUdnZDB1RmhxNWpZaFBsUis5NWhqNERHWmNPalh1?= =?utf-8?B?Qjd2YSsxWlZNek5iYm5sU0J0YmNLOWd6YkdaY1RhNmF4YURTVk1pVDZLNndh?= =?utf-8?B?alk4eURsNjFKdUVLamNSZ3diTi9ucDEyZWhsaXZlZHNFOEJ1d3pMYWx2bnlD?= =?utf-8?B?eVZMWTExUWtFWi9UTkFsSER6ZGlsQ1pOQmJ4aEFiTW1HWUxwcENQN01IdEZ1?= =?utf-8?B?LzF3V1pyMXQ0N2g3NHFGZEQ4a1lnK21YemFFN2I3c1pKSDkvTmhJdTBmN1Nh?= =?utf-8?B?cklJQkJuVng1bEtqNFQ1WXc5N3ZhZG1FdXNJWm5udExJVmpKNDVTc1hXK2Mz?= =?utf-8?B?ZTRkRDZ3aUQ2bDJ4M3lxZVlFVFBCeENaRmJqakZKUzRMVlZWYVVadnVTV25J?= =?utf-8?B?T2w0ck9FY0l2R0V2RFVzbmxpTTFOL0p0aGVEWndncThYMnF2bmhORURtSGp0?= =?utf-8?B?eHh5UURSVjNnN05tREZoWnJkS0txN2FtVHNNOFFMUlp3Z2JHWHhXWlJKN3Jn?= =?utf-8?B?cldxMWlYRnpLRkZmQXNVZXNlclFBWEhqazJKaHpQNURDaG13S1MrMnFTczh3?= =?utf-8?B?TmJYbTN0UEEwdnhFN1FLV2pnY0w2NFZvYU1QcWMvNnl6RGltVU9xNFlOSi8x?= =?utf-8?B?dDFaQzN2TGoxWEFZczBHZlRiQkRVS3RCK1YzR2o5ZTd6WFRPaFlScWNLdCsz?= =?utf-8?B?U0hGYnhpY2M2aU16S2Vkdkhybmp1WmQ4SUw2Uks3NXRoSWR3OVFyMWhFZnJk?= =?utf-8?B?N2wxR2pLN2xWaXpwb2ErYThNc2RLd3VJQndOVVIwcjVjekxYc1Erd1BjaEpw?= =?utf-8?B?WFE5T1dsblcxeC9wMHlsL21VcExxVk45UGRmQmxuZ3RkK0d4WW52MENjZ0JP?= =?utf-8?B?Q3hwZzVzVjVEWG90S256akRNSXhlbXAxVktwY21LbUgvUm9ia0dueUhlOTUx?= =?utf-8?B?QjJTRWV2OE9EZmhoYmlMYVNoRVMyaVgrWkhNWkFFcjBwSUZPcTc5dHBvcGox?= =?utf-8?B?VGQ1cXd5UWxSMFk0WVF6dUlTZC9kcjVpOXhKMHhUMURzRkk5RlFTS3pNTFBT?= =?utf-8?B?UVBvWW9IMmFkOFdsaWVrSE1JMXRRL1E4V1RwdW5ZalR1UFU3QWRyZ2lUS1J2?= =?utf-8?B?Rk43c2d2WXM3SzQzNkZIbXQxdHFpOEdzRlFzcVVYV1lpM0xXbGgxV2ZlbmY2?= =?utf-8?B?OU41azFjWllJaUVGaGV1VFpFQU0zV1VXbXk0Ujk1Z3loSTFlUmo4ZjJ5Wk04?= =?utf-8?B?YW10NTRVU1BIY0wxK1RocGpWcWxGempjcXhaQ0ZvQ2FGL1NPN3RVSVdaRlUx?= =?utf-8?B?YnlxbUVMWFl6dllzaVRPQ0ZpWjhPL2FmY1VZN3FjWGVRc0kyZVdWVUNGc1Fu?= =?utf-8?B?NW1kdlk4WjVaOEU1SFo1Vk9rc0szNld5RStIVVpvZUNYUDhCZ0lHdkEySld0?= =?utf-8?B?aTV5YkhTTjd4emcrNHg0TEJsakV5VE9PRnFGa3JqaTlEemxIQjdDZGZmOXEy?= =?utf-8?B?eVE9PQ==?= Content-ID: MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5515.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 68f90863-659e-45f9-f489-08db010accae X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Jan 2023 08:36:49.8062 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: IS+boMwreZai2+5m2kF1MyCraX9ss/DMHMhiBFmt5a3yEUK2dcfY267op7LjoOYDBBthgAX+s8Jasi0C8qgJ+A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR11MB7822 X-OriginatorOrg: intel.com Subject: Re: [FFmpeg-devel] [PATCH] avfilter: add VA-API variants of the stack filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On Do, 2023-01-19 at 16:30 +0800, Xiang, Haihao wrote: > From: Haihao Xiang > > Include hstack_vaapi, vstack_vaapi and xstack_vaapi. They may accept > input streams with different sizes. libva2 (VA-API 1.0+) is required. > > Example: > $ ffmpeg -hwaccel vaapi -hwaccel_output_format vaapi -i input.h265 > -filter_complex > "[0:v][0:v]hstack_vaapi" -c:v h264_vaapi out.h264 > > $ ffmpeg \ > -hwaccel vaapi -hwaccel_output_format vaapi -i input.h264 \ > -hwaccel vaapi -hwaccel_output_format vaapi -i input.h264 \ > -hwaccel vaapi -hwaccel_output_format vaapi -i input.h264 \ > -hwaccel vaapi -hwaccel_output_format vaapi -i input.h264 \ > -filter_complex > "[0:v][1:v][2:v][3:v]xstack_vaapi=inputs=4:fill=0x000000:layout=0_0_1920x1080| > w0_0_1920x1080|0_h0_1920x1080|w0_h0_1920x1080" \ > -c:v hevc_vaapi out.h265 > > Signed-off-by: Haihao Xiang > --- > Changelog | 1 + > configure | 3 + > doc/filters.texi | 81 +++++ > libavfilter/Makefile | 3 + > libavfilter/allfilters.c | 3 + > libavfilter/version.h | 2 +- > libavfilter/vf_stack_vaapi.c | 574 +++++++++++++++++++++++++++++++++++ > 7 files changed, 666 insertions(+), 1 deletion(-) > create mode 100644 libavfilter/vf_stack_vaapi.c > > diff --git a/Changelog b/Changelog > index 5c01e8365e..a6c0a08433 100644 > --- a/Changelog > +++ b/Changelog > @@ -29,6 +29,7 @@ version : > - corr video filter > - adrc audio filter > - afdelaysrc audio filter > +- hstack_vaapi, vstack_vaapi and xstack_vaapi filters > > > version 5.1: > diff --git a/configure b/configure > index 6e88c32223..a4620b1005 100755 > --- a/configure > +++ b/configure > @@ -3765,6 +3765,9 @@ xfade_opencl_filter_deps="opencl" > yadif_cuda_filter_deps="ffnvcodec" > yadif_cuda_filter_deps_any="cuda_nvcc cuda_llvm" > yadif_videotoolbox_filter_deps="metal corevideo videotoolbox" > +hstack_vaapi_filter_deps="vaapi_1" > +vstack_vaapi_filter_deps="vaapi_1" > +xstack_vaapi_filter_deps="vaapi_1" > > # examples > avio_list_dir_deps="avformat avutil" > diff --git a/doc/filters.texi b/doc/filters.texi > index be70a2396b..89c8420f7f 100644 > --- a/doc/filters.texi > +++ b/doc/filters.texi > @@ -14915,6 +14915,7 @@ Set the scaling dimension: @code{2} for @code{hq2x}, > @code{3} for > Default is @code{3}. > @end table > > +@anchor{hstack} > @section hstack > Stack input videos horizontally. > > @@ -24366,6 +24367,7 @@ Example: > ffmpeg -i ref.mpg -vf vmafmotion -f null - > @end example > > +@anchor{vstack} > @section vstack > Stack input videos vertically. > > @@ -24847,6 +24849,7 @@ minimum values, and @code{1} maximum values. > > This filter supports all above options as @ref{commands}, excluding option > @code{inputs}. > > +@anchor{xstack} > @section xstack > Stack video inputs into custom layout. > > @@ -26663,6 +26666,84 @@ tonemap_vaapi=format=p010:t=bt2020-10 > @end example > @end itemize > > +@section hstack_vaapi > +Stack input videos horizontally. > + > +This is the VA-API variant of the @ref{hstack} filter, each input stream may > +have different height, this filter will scale down/up each input stream while > +keeping the orignal aspect. > + > +It accepts the following options: > + > +@table @option > +@item inputs > +See @ref{hstack}. > + > +@item shortest > +See @ref{hstack}. > + > +@item height > +Set height of output. If set to 0, this filter will set height of output to > +height of the first input stream. Default value is 0. > +@end table > + > +@section vstack_vaapi > +Stack input videos vertically. > + > +This is the VA-API variant of the @ref{vstack} filter, each input stream may > +have different width, this filter will scale down/up each input stream while > +keeping the orignal aspect. > + > +It accepts the following options: > + > +@table @option > +@item inputs > +See @ref{vstack}. > + > +@item shortest > +See @ref{vstack}. > + > +@item width > +Set width of output. If set to 0, this filter will set width of output to > +width of the first input stream. Default value is 0. > +@end table > + > +@section xstack_vaapi > +Stack video inputs into custom layout. > + > +This is the VA-API variant of the @ref{xstack} filter, each input stream may > +have different size, this filter will scale down/up each input stream to the > +given output size, or the size of the first input stream. > + > +It accepts the following options: > + > +@table @option > +@item inputs > +See @ref{xstack}. > + > +@item shortest > +See @ref{xstack}. > + > +@item layout > +See @ref{xstack}. > +Moreover, this permits the user to supply output size for each input stream. > +@example > +xstack_vaapi=inputs=4:layout=0_0_1920x1080|0_h0_1920x1080|w0_0_1920x1080|w0_h > 0_1920x1080 > +@end example > + > +@item grid > +See @ref{xstack}. > + > +@item grid_tile_size > +Set output size for each input stream when @option{grid} is set. If this > option > +is not set, this filter will set output size by default to the size of the > +first input stream. For the syntax of this option, check the > +@ref{video size syntax,,"Video size" section in the ffmpeg-utils > manual,ffmpeg-utils}. > + > +@item fill > +See @ref{xstack}. > +@end table > + > @c man end VAAPI VIDEO FILTERS > > @chapter Video Sources > diff --git a/libavfilter/Makefile b/libavfilter/Makefile > index 211ff4daaa..a6f32b0e3e 100644 > --- a/libavfilter/Makefile > +++ b/libavfilter/Makefile > @@ -557,6 +557,9 @@ OBJS-$(CONFIG_YAEPBLUR_FILTER) += > vf_yaepblur.o > OBJS-$(CONFIG_ZMQ_FILTER) += f_zmq.o > OBJS-$(CONFIG_ZOOMPAN_FILTER) += vf_zoompan.o > OBJS-$(CONFIG_ZSCALE_FILTER) += vf_zscale.o > +OBJS-$(CONFIG_HSTACK_VAAPI_FILTER) += vf_stack_vaapi.o framesync.o > vaapi_vpp.o > +OBJS-$(CONFIG_VSTACK_VAAPI_FILTER) += vf_stack_vaapi.o framesync.o > vaapi_vpp.o > +OBJS-$(CONFIG_XSTACK_VAAPI_FILTER) += vf_stack_vaapi.o framesync.o > vaapi_vpp.o > > OBJS-$(CONFIG_ALLRGB_FILTER) += vsrc_testsrc.o > OBJS-$(CONFIG_ALLYUV_FILTER) += vsrc_testsrc.o > diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c > index 1ab3c8319a..bd1c14dccd 100644 > --- a/libavfilter/allfilters.c > +++ b/libavfilter/allfilters.c > @@ -522,6 +522,9 @@ extern const AVFilter ff_vf_yaepblur; > extern const AVFilter ff_vf_zmq; > extern const AVFilter ff_vf_zoompan; > extern const AVFilter ff_vf_zscale; > +extern const AVFilter ff_vf_hstack_vaapi; > +extern const AVFilter ff_vf_vstack_vaapi; > +extern const AVFilter ff_vf_xstack_vaapi; > > extern const AVFilter ff_vsrc_allrgb; > extern const AVFilter ff_vsrc_allyuv; > diff --git a/libavfilter/version.h b/libavfilter/version.h > index a56ba3bb6d..09b9954597 100644 > --- a/libavfilter/version.h > +++ b/libavfilter/version.h > @@ -31,7 +31,7 @@ > > #include "version_major.h" > > -#define LIBAVFILTER_VERSION_MINOR 54 > +#define LIBAVFILTER_VERSION_MINOR 55 > #define LIBAVFILTER_VERSION_MICRO 100 > > > diff --git a/libavfilter/vf_stack_vaapi.c b/libavfilter/vf_stack_vaapi.c > new file mode 100644 > index 0000000000..403fbb19eb > --- /dev/null > +++ b/libavfilter/vf_stack_vaapi.c > @@ -0,0 +1,574 @@ > +/* > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 > USA > + */ > + > +/** > + * @file > + * Hardware accelerated hstack, vstack and xstack filters based on VA-API > + */ > + > +#include "config_components.h" > + > +#include "libavutil/opt.h" > +#include "libavutil/common.h" > +#include "libavutil/pixdesc.h" > +#include "libavutil/eval.h" > +#include "libavutil/hwcontext.h" > +#include "libavutil/avstring.h" > +#include "libavutil/avassert.h" > +#include "libavutil/imgutils.h" > +#include "libavutil/mathematics.h" > +#include "libavutil/parseutils.h" > +#include "libavutil/mem.h" > + > +#include "internal.h" > +#include "filters.h" > +#include "formats.h" > +#include "video.h" > +#include "framesync.h" > +#include "vaapi_vpp.h" > + > +#define OFFSET(x) offsetof(StackVAAPIContext, x) > +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_FILTERING_PARAM) > + > +enum { > + STACK_VAAPI_H = 0, > + STACK_VAAPI_V = 1, > + STACK_VAAPI_X = 2 > +}; > + > +typedef struct StackVAAPIContext { > + VAAPIVPPContext vppctx; /**< must be the first field */ > + > + FFFrameSync fs; > + int mode; > + VARectangle *rects; > + uint8_t fillcolor[4]; > + int fillcolor_enable; > + > + /* Options */ > + int nb_inputs; > + int shortest; > + int tile_width; > + int tile_height; > + int nb_grid_columns; > + int nb_grid_rows; > + char *layout; > + char *fillcolor_str; > +} StackVAAPIContext; > + > +static int process_frame(FFFrameSync *fs) > +{ > + AVFilterContext *avctx = fs->parent; > + AVFilterLink *outlink = avctx->outputs[0]; > + StackVAAPIContext *sctx = fs->opaque; > + VAAPIVPPContext *vppctx = fs->opaque; > + AVFrame *oframe, *iframe; > + VAProcPipelineParameterBuffer *params = NULL; > + VARectangle *irect = NULL; > + int ret = 0; > + > + if (vppctx->va_context == VA_INVALID_ID) > + return AVERROR(EINVAL); > + > + oframe = ff_get_video_buffer(outlink, outlink->w, outlink->h); > + if (!oframe) > + return AVERROR(ENOMEM); > + > + irect = av_calloc(avctx->nb_inputs, sizeof(*irect)); > + params = av_calloc(avctx->nb_inputs, sizeof(*params)); > + if (!irect || !params) { > + ret = AVERROR(ENOMEM); > + goto fail; > + } > + > + for (int i = 0; i < avctx->nb_inputs; i++) { > + ret = ff_framesync_get_frame(fs, i, &iframe, 0); > + if (ret) > + goto fail; > + > + if (i == 0) { > + ret = av_frame_copy_props(oframe, iframe); > + if (ret < 0) > + goto fail; > + } > + > + ret = ff_vaapi_vpp_init_params(avctx, ¶ms[i], iframe, oframe); > + if (ret) > + goto fail; > + > + av_log(avctx, AV_LOG_DEBUG, "stack input %d: %s, %ux%u > (%"PRId64").\n", > + i, av_get_pix_fmt_name(iframe->format), > + iframe->width, iframe->height, iframe->pts); > + irect[i].x = 0; > + irect[i].y = 0; > + irect[i].width = iframe->width; > + irect[i].height = iframe->height; > + params[i].surface_region = &irect[i]; > + params[i].surface = (VASurfaceID)(uintptr_t)iframe->data[3]; > + params[i].output_region = &sctx->rects[i]; > + > + if (sctx->fillcolor_enable) > + params[i].output_background_color = (sctx->fillcolor[3] << 24 | > + sctx->fillcolor[0] << 16 | > + sctx->fillcolor[1] << 8 | > + sctx->fillcolor[2]); > + } > + > + oframe->pts = av_rescale_q(sctx->fs.pts, sctx->fs.time_base, outlink- > >time_base); > + oframe->sample_aspect_ratio = outlink->sample_aspect_ratio; > + > + ret = ff_vaapi_vpp_render_pictures(avctx, params, avctx->nb_inputs, > oframe); > + if (ret) > + goto fail; > + > + av_freep(&irect); > + av_freep(¶ms); > + return ff_filter_frame(outlink, oframe); > + > +fail: > + av_freep(&irect); > + av_freep(¶ms); > + av_frame_free(&oframe); > + return ret; > +} > + > +static int init_framesync(AVFilterContext *avctx) > +{ > + StackVAAPIContext *sctx = avctx->priv; > + int ret; > + > + ret = ff_framesync_init(&sctx->fs, avctx, avctx->nb_inputs); > + if (ret < 0) > + return ret; > + > + sctx->fs.on_event = process_frame; > + sctx->fs.opaque = sctx; > + > + for (int i = 0; i < sctx->nb_inputs; i++) { > + FFFrameSyncIn *in = &sctx->fs.in[i]; > + > + in->before = EXT_STOP; > + in->after = sctx->shortest ? EXT_STOP : EXT_INFINITY; > + in->sync = 1; > + in->time_base = avctx->inputs[i]->time_base; > + } > + > + return ff_framesync_configure(&sctx->fs); > +} > + > +#define SET_INPUT_REGION(rect, rx, ry, rw, rh) do { \ > + rect->x = rx; \ > + rect->y = ry; \ > + rect->width = rw; \ > + rect->height = rh; \ > + } while (0) > + > +static int config_output(AVFilterLink *outlink) > +{ > + AVFilterContext *avctx = outlink->src; > + StackVAAPIContext *sctx = avctx->priv; > + VAAPIVPPContext *vppctx = avctx->priv; > + AVFilterLink *inlink0 = avctx->inputs[0]; > + AVHWFramesContext *hwfc0 = NULL; > + int width, height, ret; > + > + if (inlink0->format != AV_PIX_FMT_VAAPI || !inlink0->hw_frames_ctx || > !inlink0->hw_frames_ctx->data) { > + av_log(avctx, AV_LOG_ERROR, "Software pixel format is not > supported.\n"); > + return AVERROR(EINVAL); > + } > + > + hwfc0 = (AVHWFramesContext *)inlink0->hw_frames_ctx->data; > + > + for (int i = 1; i < sctx->nb_inputs; i++) { > + AVFilterLink *inlink = avctx->inputs[i]; > + AVHWFramesContext *hwfc = NULL; > + > + if (inlink->format != AV_PIX_FMT_VAAPI || !inlink->hw_frames_ctx || > !inlink->hw_frames_ctx->data) { > + av_log(avctx, AV_LOG_ERROR, "Software pixel format is not > supported.\n"); > + return AVERROR(EINVAL); > + } > + > + hwfc = (AVHWFramesContext *)inlink->hw_frames_ctx->data; > + > + if (hwfc0->sw_format != hwfc->sw_format) { > + av_log(avctx, AV_LOG_ERROR, "All inputs should have the same > underlying software pixel format.\n"); > + return AVERROR(EINVAL); > + } > + > + if (hwfc0->device_ctx != hwfc->device_ctx) { > + av_log(avctx, AV_LOG_ERROR, "All inputs should have the same > underlying vaapi devices.\n"); > + return AVERROR(EINVAL); > + } > + } > + > + ff_vaapi_vpp_config_input(inlink0); > + vppctx->output_format = hwfc0->sw_format; > + > + if (sctx->mode == STACK_VAAPI_H) { > + height = sctx->tile_height; > + width = 0; > + > + if (!height) > + height = inlink0->h; > + > + for (int i = 0; i < sctx->nb_inputs; i++) { > + AVFilterLink *inlink = avctx->inputs[i]; > + VARectangle *rect = &sctx->rects[i]; > + > + SET_INPUT_REGION(rect, width, 0, av_rescale(height, inlink->w, > inlink->h), height); > + width += av_rescale(height, inlink->w, inlink->h); > + } > + } else if (sctx->mode == STACK_VAAPI_V) { > + height = 0; > + width = sctx->tile_width; > + > + if (!width) > + width = inlink0->w; > + > + for (int i = 0; i < sctx->nb_inputs; i++) { > + AVFilterLink *inlink = avctx->inputs[i]; > + VARectangle *rect = &sctx->rects[i]; > + > + SET_INPUT_REGION(rect, 0, height, width, av_rescale(width, > inlink->h, inlink->w)); > + height += av_rescale(width, inlink->h, inlink->w); > + } > + } else if (sctx->nb_grid_rows && sctx->nb_grid_columns) { > + int xpos = 0, ypos = 0; > + int ow, oh, k = 0; > + > + ow = sctx->tile_width; > + oh = sctx->tile_height; > + > + if (!ow || !oh) { > + ow = avctx->inputs[0]->w; > + oh = avctx->inputs[0]->h; > + } > + > + for (int i = 0; i < sctx->nb_grid_columns; i++) { > + ypos = 0; > + > + for (int j = 0; j < sctx->nb_grid_rows; j++) { > + VARectangle *rect = &sctx->rects[k++]; > + > + SET_INPUT_REGION(rect, xpos, ypos, ow, oh); > + ypos += oh; > + } > + > + xpos += ow; > + } > + > + width = ow * sctx->nb_grid_columns; > + height = oh * sctx->nb_grid_rows; > + } else { > + char *arg, *p = sctx->layout, *saveptr = NULL; > + char *arg2, *p2, *saveptr2 = NULL; > + char *arg3, *p3, *saveptr3 = NULL; > + int xpos, ypos, size; > + int ow, oh; > + > + width = avctx->inputs[0]->w; > + height = avctx->inputs[0]->h; > + > + for (int i = 0; i < sctx->nb_inputs; i++) { > + AVFilterLink *inlink = avctx->inputs[i]; > + VARectangle *rect = &sctx->rects[i]; > + > + ow = inlink->w; > + oh = inlink->h; > + > + if (!(arg = av_strtok(p, "|", &saveptr))) > + return AVERROR(EINVAL); > + > + p = NULL; > + p2 = arg; > + xpos = ypos = 0; > + > + for (int j = 0; j < 3; j++) { > + if (!(arg2 = av_strtok(p2, "_", &saveptr2))) { > + if (j == 2) > + break; > + else > + return AVERROR(EINVAL); > + } > + > + p2 = NULL; > + p3 = arg2; > + > + if (j == 2) { > + if ((ret = av_parse_video_size(&ow, &oh, p3)) < 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid size '%s'\n", > p3); > + return ret; > + } > + > + break; > + } > + > + while ((arg3 = av_strtok(p3, "+", &saveptr3))) { > + p3 = NULL; > + if (sscanf(arg3, "w%d", &size) == 1) { > + if (size == i || size < 0 || size >= sctx->nb_inputs) > + return AVERROR(EINVAL); > + > + if (!j) > + xpos += sctx->rects[size].width; > + else > + ypos += sctx->rects[size].width; > + } else if (sscanf(arg3, "h%d", &size) == 1) { > + if (size == i || size < 0 || size >= sctx->nb_inputs) > + return AVERROR(EINVAL); > + > + if (!j) > + xpos += sctx->rects[size].height; > + else > + ypos += sctx->rects[size].height; > + } else if (sscanf(arg3, "%d", &size) == 1) { > + if (size < 0) > + return AVERROR(EINVAL); > + > + if (!j) > + xpos += size; > + else > + ypos += size; > + } else { > + return AVERROR(EINVAL); > + } > + } > + } > + > + SET_INPUT_REGION(rect, xpos, ypos, ow, oh); > + width = FFMAX(width, xpos + ow); > + height = FFMAX(height, ypos + oh); > + } > + > + } > + > + outlink->w = width; > + outlink->h = height; > + outlink->frame_rate = inlink0->frame_rate; > + outlink->sample_aspect_ratio = inlink0->sample_aspect_ratio; > + > + for (int i = 1; i < sctx->nb_inputs; i++) { > + AVFilterLink *inlink = avctx->inputs[i]; > + if (outlink->frame_rate.num != inlink->frame_rate.num || > + outlink->frame_rate.den != inlink->frame_rate.den) { > + av_log(avctx, AV_LOG_VERBOSE, > + "Video inputs have different frame rates, output will be > VFR\n"); > + outlink->frame_rate = av_make_q(1, 0); > + break; > + } > + } > + > + ret = init_framesync(avctx); > + if (ret < 0) > + return ret; > + > + outlink->time_base = sctx->fs.time_base; > + > + vppctx->output_width = width; > + vppctx->output_height = height; > + > + return ff_vaapi_vpp_config_output(outlink); > +} > + > +static int vaapi_stack_init(AVFilterContext *avctx) > +{ > + StackVAAPIContext *sctx = avctx->priv; > + VAAPIVPPContext *vppctx = avctx->priv; > + int ret; > + > + if (!strcmp(avctx->filter->name, "hstack_vaapi")) > + sctx->mode = STACK_VAAPI_H; > + else if (!strcmp(avctx->filter->name, "vstack_vaapi")) > + sctx->mode = STACK_VAAPI_V; > + else { > + int is_grid; > + > + av_assert0(strcmp(avctx->filter->name, "xstack_vaapi") == 0); > + sctx->mode = STACK_VAAPI_X; > + is_grid = sctx->nb_grid_rows && sctx->nb_grid_columns; > + > + if (sctx->layout && is_grid) { > + av_log(avctx, AV_LOG_ERROR, "Both layout and grid were specified. > Only one is allowed.\n"); > + return AVERROR(EINVAL); > + } > + > + if (!sctx->layout && !is_grid) { > + if (sctx->nb_inputs == 2) { > + sctx->nb_grid_rows = 1; > + sctx->nb_grid_columns = 2; > + is_grid = 1; > + } else { > + av_log(avctx, AV_LOG_ERROR, "No layout or grid > specified.\n"); > + return AVERROR(EINVAL); > + } > + } > + > + if (is_grid) > + sctx->nb_inputs = sctx->nb_grid_rows * sctx->nb_grid_columns; > + > + if (strcmp(sctx->fillcolor_str, "none") && > + av_parse_color(sctx->fillcolor, sctx->fillcolor_str, -1, avctx) > >= 0) { > + sctx->fillcolor_enable = 1; > + } else { > + sctx->fillcolor_enable = 0; > + } > + } > + > + for (int i = 0; i < sctx->nb_inputs; i++) { > + AVFilterPad pad = { 0 }; > + > + pad.type = AVMEDIA_TYPE_VIDEO; > + pad.name = av_asprintf("input%d", i); > + > + if (!pad.name) > + return AVERROR(ENOMEM); > + > + if ((ret = ff_append_inpad_free_name(avctx, &pad)) < 0) > + return ret; > + } > + > + /* stack region */ > + sctx->rects = av_calloc(sctx->nb_inputs, sizeof(*sctx->rects)); > + if (!sctx->rects) > + return AVERROR(ENOMEM); > + > + ff_vaapi_vpp_ctx_init(avctx); > + vppctx->output_format = AV_PIX_FMT_NONE; > + > + return 0; > +} > + > +static av_cold void vaapi_stack_uninit(AVFilterContext *avctx) > +{ > + StackVAAPIContext *sctx = avctx->priv; > + > + ff_framesync_uninit(&sctx->fs); > + av_freep(&sctx->rects); > +} > + > +static int vaapi_stack_activate(AVFilterContext *avctx) > +{ > + StackVAAPIContext *sctx = avctx->priv; > + return ff_framesync_activate(&sctx->fs); > +} > + > +static int vaapi_stack_query_formats(AVFilterContext *avctx) > +{ > + static const enum AVPixelFormat pixel_formats[] = { > + AV_PIX_FMT_VAAPI, > + AV_PIX_FMT_NONE, > + }; > + > + return ff_set_common_formats_from_list(avctx, pixel_formats); > +} > + > +static const AVFilterPad vaapi_stack_outputs[] = { > + { > + .name = "default", > + .type = AVMEDIA_TYPE_VIDEO, > + .config_props = config_output, > + }, > +}; > + > +#define STACK_COMMON_OPTS \ > + { "inputs", "Set number of inputs", OFFSET(nb_inputs), AV_OPT_TYPE_INT, { > .i64 = 2 }, 2, UINT16_MAX, .flags = FLAGS }, \ > + { "shortest", "Force termination when the shortest input terminates", > OFFSET(shortest), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS }, > + > +#if CONFIG_HSTACK_VAAPI_FILTER > + > +static const AVOption hstack_vaapi_options[] = { > + STACK_COMMON_OPTS > + > + { "height", "Set output height (0 to use the height of input 0)", > OFFSET(tile_height), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, UINT16_MAX, FLAGS }, > + { NULL } > +}; > + > +AVFILTER_DEFINE_CLASS(hstack_vaapi); > + > +const AVFilter ff_vf_hstack_vaapi = { > + .name = "hstack_vaapi", > + .description = NULL_IF_CONFIG_SMALL("VA-API hstack."), > + .priv_size = sizeof(StackVAAPIContext), > + .priv_class = &hstack_vaapi_class, > + .init = vaapi_stack_init, > + .uninit = vaapi_stack_uninit, > + .activate = vaapi_stack_activate, > + FILTER_QUERY_FUNC(vaapi_stack_query_formats), > + FILTER_OUTPUTS(vaapi_stack_outputs), > + .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, > + .flags = AVFILTER_FLAG_DYNAMIC_INPUTS, > +}; > + > +#endif > + > +#if CONFIG_VSTACK_VAAPI_FILTER > + > +static const AVOption vstack_vaapi_options[] = { > + STACK_COMMON_OPTS > + > + { "width", "Set output width (0 to use the width of input 0)", > OFFSET(tile_width), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, UINT16_MAX, FLAGS }, > + { NULL } > +}; > + > +AVFILTER_DEFINE_CLASS(vstack_vaapi); > + > +const AVFilter ff_vf_vstack_vaapi = { > + .name = "vstack_vaapi", > + .description = NULL_IF_CONFIG_SMALL("VA-API vstack."), > + .priv_size = sizeof(StackVAAPIContext), > + .priv_class = &vstack_vaapi_class, > + .init = vaapi_stack_init, > + .uninit = vaapi_stack_uninit, > + .activate = vaapi_stack_activate, > + FILTER_QUERY_FUNC(vaapi_stack_query_formats), > + FILTER_OUTPUTS(vaapi_stack_outputs), > + .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, > + .flags = AVFILTER_FLAG_DYNAMIC_INPUTS, > +}; > + > +#endif > + > +#if CONFIG_XSTACK_VAAPI_FILTER > + > +static const AVOption xstack_vaapi_options[] = { > + STACK_COMMON_OPTS > + > + { "layout", "Set custom layout", OFFSET(layout), AV_OPT_TYPE_STRING, > {.str = NULL}, 0, 0, .flags = FLAGS }, > + { "grid", "set fixed size grid layout", OFFSET(nb_grid_columns), > AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, .flags = FLAGS }, > + { "grid_tile_size", "set tile size in grid layout", OFFSET(tile_width), > AV_OPT_TYPE_IMAGE_SIZE, {.str = NULL}, 0, 0, .flags = FLAGS }, > + { "fill", "Set the color for unused pixels", OFFSET(fillcolor_str), > AV_OPT_TYPE_STRING, {.str = "none"}, .flags = FLAGS }, > + { NULL } > +}; > + > +AVFILTER_DEFINE_CLASS(xstack_vaapi); > + > +const AVFilter ff_vf_xstack_vaapi = { > + .name = "xstack_vaapi", > + .description = NULL_IF_CONFIG_SMALL("VA-API xstack."), > + .priv_size = sizeof(StackVAAPIContext), > + .priv_class = &xstack_vaapi_class, > + .init = vaapi_stack_init, > + .uninit = vaapi_stack_uninit, > + .activate = vaapi_stack_activate, > + FILTER_OUTPUTS(vaapi_stack_outputs), > + FILTER_QUERY_FUNC(vaapi_stack_query_formats), > + .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, > + .flags = AVFILTER_FLAG_DYNAMIC_INPUTS, > +}; > + > +#endif I will push it soon if I don't hear anything against it. Thanks Haihao _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".