From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 4C21441103 for ; Wed, 15 Jun 2022 02:49:06 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BD09E68B646; Wed, 15 Jun 2022 05:49:03 +0300 (EEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1DBD168B5C4 for ; Wed, 15 Jun 2022 05:48:55 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1655261342; x=1686797342; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=1cgpW/crJAVPnofEbJK1hJ6BbyxPNx6z+URUSXgjcwo=; b=RI5HDwX38Fpy/+naMpz3SefeI61aZVbjZ0UIUmyCmQ6uFCf3y6D/l5UC j2U47ze7KXCWk+c7pPmAckVQqSjMAYs8tzSDvYHipZNlsk9HNtNk6AtEI x+qcZHP3m2oH29jhEGI+GaOVjmcejMXTfoJqNFHLOsw66nUumJJ+lrXNQ lqA3JfK3ENzpNtQfx26bTCGLI46N00xFSXAytGWv1xlqZPGgxHSJlVkEe It2eLHPrzAK8St+4y4Y3P5PYQ1pPl2PHD0tAvgScY3MACci/2MAHOkRMF CdW+mHUSKGSfLJWekOW9D10n+j6omW5ApxK3ATF/gFO8jIXPoXvq4rLiy Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10378"; a="279528396" X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="279528396" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jun 2022 19:48:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="911363389" Received: from orsmsx606.amr.corp.intel.com ([10.22.229.19]) by fmsmga005.fm.intel.com with ESMTP; 14 Jun 2022 19:48:52 -0700 Received: from orsmsx601.amr.corp.intel.com (10.22.229.14) by ORSMSX606.amr.corp.intel.com (10.22.229.19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Tue, 14 Jun 2022 19:48:51 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by orsmsx601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Tue, 14 Jun 2022 19:48:51 -0700 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.49) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Tue, 14 Jun 2022 19:48:51 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DYyeOYlkxFhrUmEAi+LmGkZX0ImRvmjzlsnA93yhgKblgqSz7V7PyGw4bo5a2fago2dNhDR1U8WNlPKnIZo5zyGimatJ2vRIAtT3ClBLje/3SU7hMkbvW1yY78/KRTot5NdjzgBi9oPT6dWDX4HBMeqqobryHCqHpEmRuD43NxfJclrEeNIuYifWAOnSLLRFZUXPv1WvuHK+M2sErjfDxEgmXZjglz5NzkkA6L3xnmQF3WCUfh9SXCfBcTDkgqJmd6iRyt+sbijFCeBUc7c4Cubjjiytct3HL9dUt1M7fBnz45nkZxZxW5JOM5TmQicQt7m4BzJnGRKhblJfUVsYQw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=erSketIykUc76hzdgHBucKVnxbiS+aJ+VJoqsZiVj2g=; b=SCX+kfcPHBL6msAbE0zW3Cdjqje22gvwazL34ZdwxrmDPqEdKehmtuzTZ99XMQMv4w3e4CmrOdo8GkLSqidI8DqI9xBRMi0npaa9zYK20oc8hZ/w2Jubb4J/ZT0vaunTh9ZUZGFr9fP5OYqNIFTT8yJMXJsQSec1SKsaHjleWecKtFt98sTv5hBpbzlw/WAbKOtRkbZYHFmyj7LopCJV/QkSxzIj2nlociVZbo0H8s1UadBOnt2vJqC+TpKueh8m/wRDkh3CAXWHvhmwPJTVDC1+DI/DgEKE5PFAKnglCLdTwXX0Y54Yd0SdGms+BGF6Ey6NTRGBw+WAfiV3F1dtbw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from SN6PR11MB2990.namprd11.prod.outlook.com (2603:10b6:805:cf::21) by DS0PR11MB6471.namprd11.prod.outlook.com (2603:10b6:8:c1::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5332.20; Wed, 15 Jun 2022 02:48:49 +0000 Received: from SN6PR11MB2990.namprd11.prod.outlook.com ([fe80::c8d3:c41:710c:5b5b]) by SN6PR11MB2990.namprd11.prod.outlook.com ([fe80::c8d3:c41:710c:5b5b%6]) with mapi id 15.20.5332.022; Wed, 15 Jun 2022 02:48:49 +0000 From: "Wu, Tong1" To: FFmpeg development discussions and patches Thread-Topic: [FFmpeg-devel] [PATCH] avcodec/qsvenc: make QSV encoder encode VAAPI and D3D11 frames directly Thread-Index: AQHYelBIrfmrHmmeCkGerD9R7hwrYa1E9t6AgAA7mgCAACpZAIABSDAAgAKxQwCABnlDwA== Date: Wed, 15 Jun 2022 02:48:49 +0000 Message-ID: References: <20220607092216.405-1-tong1.wu@intel.com> <078794290ef4b70ba19dfd436313a58c2ab8bdff.camel@intel.com> In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.6.500.17 dlp-reaction: no-action authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 96389829-9dad-422c-45ef-08da4e79936b x-ms-traffictypediagnostic: DS0PR11MB6471:EE_ x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: jb3QdfGvrkfJOh8a8UInE2eYom/lsXsxyo3tVMMaqhWybXdrrfpU9xiHYHkVmR76leiU3foric6dn4pUY8bPBvbsp8K5o1sJtSJt7tDX/xDzUiUriAHbv0V9IPwfDixbFLzmizX7NREeEI9LRZ1geLGMvXJ7qt1wklpPtNtqgEPz8R2lD8piY4IRt3FmuEmMOhTLOPr5hlgxmRIMtEHaYSRbqkcf1dmV+TAPf0yx0N778E8EpW/Y3gBCVzLGUZCTTRf56eD6/OfZ+BTCIWDnsJtgVPuZ79ppa0t3GXVk0IoCV6u4oZEKeIdUKeCWLr1RGevXitn3SQ1svkwIT/0rfLxiVUu48nqQH6NOf2nAtFfLHVDQ2YUn2hvxCpNqgm54scG06oIpifCZuNOYAQ2N9KfbclywNVZxvcIE6v30sme/iT4BRZEj8uEQYuQmdiGaz3LII+USw2j5novlTdOuK+RB4xVR5okUGjYtnH+CED2Fm8F+fjqXnH7ayI4S9PGBnBl1fhqhL+I1M6Bn7SJCuqSEPANngKLxg0Y/jR+IO9Z9h6bTTP6B45LtHImkRYOmw10BwJxQyQmZ9ZTPbdzlZcU5uXRhAxObA2WI4yqDMBJwqznxoT8Vfr88f/xDUgzlBWuMHq6SOgjUdwnzLJbVI1NZvmmFgE0onEyiJGq3Jxy/l8DwL6xnf9XnH3edheFEdk39isrZvafcPNULPRgk3JEjZ7FD4lysDmgNiat8G7YUynrZrqRfYV94zWFxG4VguFzeaa+bSZuawx5bYQQ4huoT8SoNO3SHmDTnbiC4l4Q= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR11MB2990.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(366004)(26005)(316002)(2906002)(86362001)(107886003)(9686003)(83380400001)(82960400001)(66556008)(66446008)(8676002)(33656002)(38070700005)(71200400001)(52536014)(966005)(122000001)(6916009)(508600001)(53546011)(8936002)(5660300002)(38100700002)(76116006)(64756008)(7696005)(4326008)(55016003)(186003)(6506007)(66476007)(66946007); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?OdG9qX/cFevUHLX2lcpaJ2LFv2QI37UIS1irH3nRCwKr/GM6syoyiuz1V016?= =?us-ascii?Q?rbj1RzQTy1SYdlAXjjN/v7mNT7lZL/Bg+DLP3ieCww5WkiAbR2o45izrg0uX?= =?us-ascii?Q?2xyIHruCX+LgfpzBUH4EZWrMEtPLW2CV4WGQ5+4bsFuh1aIyDDlzDYld6D56?= =?us-ascii?Q?6LU70aIrz0m0P+HTmJibrO1mTzm+PXpo/Bx0XowRAFV1SuLSxfLz34KIFf3q?= =?us-ascii?Q?ATgVeDWPm9XN6g/WkXMQ91u6sl2HqenpjrMk1uA/LgtOF2aJq75KkTqMUYaF?= =?us-ascii?Q?rVwuNcomOJA0wt27Lc9ekvke6EB/gDyKDxqKKCoKEuJShW8dYzrKEpgXAUFQ?= =?us-ascii?Q?fgpMWZgT54G2wZE5J2QY9NOyTSylOi7ubGJlYT8EElnAxJISCTicNLRfcc8A?= =?us-ascii?Q?mFHVQ+HfOnj142OYLYGM+0Hi6mITIuQsqADqFi2Dhf5pDZU0m3ELf39rKN3I?= =?us-ascii?Q?xF9CppPHYjeaB4b92bqjAks5VNP0cuWWcTavoJPYyYyd29ATh8MT3qFF2OF9?= =?us-ascii?Q?7Z0wZPqXVFh3g32jZ7zNS08OCpDRUjXIf2abCEmgSUU/bz+sUQo9V4dg83Ir?= =?us-ascii?Q?eLN70MPl8+bMpclFe3ao/quOGuvpH8DRHWIQO0/R9D9V75rBcB8TpuptDIfu?= =?us-ascii?Q?A56O8YJr9YDXuznPtnqw1+6BbYe1S/oFHm/zK+9DJ78SNhR91+ZI4qTseuY7?= =?us-ascii?Q?zcYolsp9qFfQhkwPVhZq50sMTS/4UNjcCqaU8oJ3tVS3IX+p4+JZKqoSzxqU?= =?us-ascii?Q?uInCrxncy5QKDZlYoiE+9D9He+qEeJkePnxqVDUOAFymoDq/bkrGV1+P/6e9?= =?us-ascii?Q?1lE2APB/1oozQKI1VG69WcFdHfifCsklE1q24kpThPCqIy6TDNOl8QDGs9XL?= =?us-ascii?Q?ZVouc9rM4bhkH81PrvVNMK39Cbw0WZUfid3TgvhWwOL66oz9rOqQag5wY3pL?= =?us-ascii?Q?qnArO2XpwD6lZy9vhoMi678ktBIAqsvBbozYZtm0ny6w6uLNBc2iGdw4EiH3?= =?us-ascii?Q?l00R2wT8oxMnydPq1tUeAcK95hQDHPr7Yf+eRR3R/ztljMCcrhRaWiqUD5co?= =?us-ascii?Q?D01lo/DQVgTbvRz2EH+5Epnt3nO1NDjGf2hXGxPMhVtiIkaFPM8igPfsHMUY?= =?us-ascii?Q?r4hM6HXiP9jLj6r8ColEOii+9Z1G8XhQwG8iSbMQAc7yn9kBTZJowvUHvmxX?= =?us-ascii?Q?D6r21O/AfdMcKL+9gLkn3TgsZkRj01rq8bPzUvgIwPwdoWmQSOODqNxmmZQ2?= =?us-ascii?Q?yy3hgoG8RGWerRF4qybkhNCOiFVkKt+HaR2BX+/76nrrmslLWPTLw+Qx0+UT?= =?us-ascii?Q?eg9xiTnbY3E7CwxmrBzpq3HG3exIMMtAySRKjj2amZOsOvfFClj1Nian4Vq0?= =?us-ascii?Q?xupk3nP1STcbLyqfxeuIU5imgmUHBam+SeXWkGGHQm9q395DIuFt+1z//5++?= =?us-ascii?Q?WdtEfNcgTa55KB1Egfw2lh7ZhMwSVeICXIl1L1MD4V5F3j6C/pd2cPMv1JIa?= =?us-ascii?Q?eBSczzWiMTULJPGCeQPDSoHdvsIZuJJWH0QTQpQMv5xdCU3M89Ns/QH6b7a+?= =?us-ascii?Q?IftsWr28PUfczAox++9dODWcaAY+kPASRqAmK+SiN34uL+4+6H1PbXQPkvTH?= =?us-ascii?Q?kd9vpj0Lj3hu9WngeboXsyU+PpCo4TQ9ERGRYYsdZYfudVrkpTTfnXnqlkc6?= =?us-ascii?Q?Ce4VSCUlMPofBfSEHviT3jbtCo8pwnu8G2DQEqeBzw73I60aXYE6HtMEsxCs?= =?us-ascii?Q?DTN+9TZV8w=3D=3D?= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR11MB2990.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 96389829-9dad-422c-45ef-08da4e79936b X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Jun 2022 02:48:49.7480 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 8t9swl1+2DutqKJM6VK2HlwXn90/7Lq2JMSpMP2+MLk2D1i8lVSTmusYkwFj7k81t9p6KXLA68tWr44ZVUzM0w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR11MB6471 X-OriginatorOrg: intel.com Subject: Re: [FFmpeg-devel] [PATCH] avcodec/qsvenc: make QSV encoder encode VAAPI and D3D11 frames directly X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: "Chen, Wenbin" Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: > > -----Original Message----- > > From: ffmpeg-devel On Behalf Of > > Xiang, Haihao > > Sent: Thursday, June 9, 2022 8:48 AM > > To: ffmpeg-devel@ffmpeg.org > > Cc: Wu, Tong1 ; Chen, Wenbin > > > > Subject: Re: [FFmpeg-devel] [PATCH] avcodec/qsvenc: make QSV encoder > > encode VAAPI and D3D11 frames directly > > > > On Wed, 2022-06-08 at 11:13 +0000, Soft Works wrote: > > > > -----Original Message----- > > > > From: ffmpeg-devel On Behalf Of > > Xiang, > > > > Haihao > > > > Sent: Wednesday, June 8, 2022 10:42 AM > > > > To: ffmpeg-devel@ffmpeg.org > > > > Cc: Wu, Tong1 ; Chen, Wenbin > > > > > > Subject: Re: [FFmpeg-devel] [PATCH] avcodec/qsvenc: make QSV > > encoder encode > > > > VAAPI and D3D11 frames directly > > > > > > > > On Wed, 2022-06-08 at 05:08 +0000, Soft Works wrote: > > > > > > -----Original Message----- > > > > > > From: ffmpeg-devel On > > Behalf Of Tong > > > > > > Wu > > > > > > Sent: Tuesday, June 7, 2022 11:22 AM > > > > > > To: ffmpeg-devel@ffmpeg.org > > > > > > Cc: Tong Wu ; Wenbin Chen > > > > > > > > Subject: [FFmpeg-devel] [PATCH] avcodec/qsvenc: make QSV > > encoder encode > > > > > > VAAPI > > > > > > and D3D11 frames directly > > > > > > [..] > > > > > > > > > 2.35.1.windows.2 > > > > > > > > > > Hi, > > > > > > > > > > thanks for submitting this patch. Though, I'm afraid, but this > > > > > > > > > > - fundamentally contradicts the logic of ffmpeg's handling of > > hw > > > > > > > > acceleration, > > > > > hw device and hw frames contexts > > > > > - adds code to an encoder, doing things an encoder is not > > supposed to do- > > > > > > > > qsv > > > > > encoders and decoders have their own context => QSV > > > > > > > > nvdec and nvenc have CUDA but nvenc can also support D3D11va, it > > sounds make > > > > sense for me to support D3D11va/vaapi in qsvenc too as > > d3d11va/vaapi are > > > > used > > > > internally in MediaSDK. > > > > > > Can you please post a command line showing nvenc working with input > > > from a D3D11VA decoder and without using any > > hwmap/hwupload/hwdownload > > > filters? > > > > > > > According to the code below, nvenc may accept d3d11 frames directly, > > > > https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/nvenc.c#L46- > > L72 > > > > so the command below should work > > > > $> ffmpeg -y -hwaccel_output_format d3d11 -hwaccel d3d11va -i > > input.mp4 -c:v > > hevc_nvenc out.mp4 > > Right, it does work. Thanks for the command, I had tried like that before, but > in a "wrong" branch. > > > Now I took a bit of a deeper look into it and the ability of NVENC to encode > from plain D3D11 frames. There are quite a few differences between NVENC > and QSVENC. > > > HW Frames Contexts > ------------------ > > QSVENV > > MSDK cannot work with VAAPI frames, D3D9 frames or D3D11 frames > directly. > An application is always required to wrap such frames via mfxSurface and > manage a collection of mfxSurface descriptions. > It's an abstraction that allows coding against the MSDK API independent from > the underlying technology. > The technical representation of this in ffmpeg is the QSVFramesContext. > When there's an input of plain VAAPI or D3D11 frames (hw frames context), > then it is required to derive a new QSVFramesContext from the input hw > frames context (e.g. via hwmap filter) where the procedure of deriving > means to set up a new QSVFramesContext which does the required wrapping > (or "mapping" as ffmpeg calls it). > > I think that the way how this logic is reflected in ffmpeg is thought out very > well and provides a high degree of flexibility. > > > NVENC > > The situation is very different here. Nvidia provides platform independency > not by wrapping platform-specific GPU frame types, but instead uses its own > custom type - CUDA memory/frames. This is what decoders are outputting, > filters are using for input/output and encoders take as input. > > What I do not know, is whether it would be possible to map D3D11 frames to > CUDA frames and vice versa. In case, that would be the preferable way IMO > to deal with different hw frame types. > At least this isn't implemented at this time. The only possible frames > derivation/mapping is from and to Vulkan. > > Hence I can't say whether the NVENC implementation to take D3D11 frames > directly has been done out of laziness or whether it was the only possible > way. In case when it wouldn't be possible to map D3D11 frames to CUDA > frames, and only NVENC encoders would be able to process D3D11 frames, > then it would have been the only option of course. > > But in any way, it's not the same as with QSVENC, because NVENC can take > D3D11 frames as input directly without wrapping/mapping first. > > ---------------- > > There are more differences, but I don't want to drive it too far. > > What stands at the bottom line is: > > - NVENC can take D3D11 frames context directly > - QSVENC can't - it needs to map it to a QSVFramesContext first > > > Concluding opinion: > > An encoder should not include (duplicated) code for creating a derived > frames context. > The same goal (= getting those command lines working that Haihao has > posted) could be achieved by auto-inserting a hwmap filter in those cases, > which would probably take a few lines of code only. > > We don't have a precedence for auto-insertion of a hwmap filter, but we do > that in many other cases, so that would seem to me at least an option to > think about. > > I'm curious about other opinions... > > Thanks, > softworkz Thanks for the opinion. It makes sense to me. Looks like it's a more feasible way and will have more investigation on adding automap. Thanks, Tong _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".