From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 8108A431BA for ; Tue, 24 May 2022 15:29:22 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 52AB668B511; Tue, 24 May 2022 18:29:21 +0300 (EEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11olkn2095.outbound.protection.outlook.com [40.92.19.95]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 44E5B68B354 for ; Tue, 24 May 2022 18:29:15 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FhEvnMGVxl44CGqTz9sCPzYkIQWd4ljDEcfanRrw7VuWwbR8plyGiN/Ds+jSKyc6iQXn3dV5MHhSf8nEQBhcgL5NnBPNbvJ3pa4SLokcduqGBQnLGPIvDIpq3pKFgNMcECfE9k6dbf9Z3/8J0PltpaJjTL3xk+9GwV3G6HKYLvJSdB87ET+1Zpppqv8a4Np04aGWK7vkpN5h4SBTKMV8WZrsBAz4CToZTuPuf6XpRomUqP8LScdodkF2pHWncN4awsq781JW63CQZ/LI9mEQHpuxpz6gKpLpcDAsPA+zCcnGmhNAGERsIYlPd+vkZNRqMiZqmopr875YIXzzCOTVPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kfsFU7h60ekNCwpqBfc9KZzaCZFPTcN0ePlA0PIykoY=; b=LOoa6mhDzVqGuYzdP2oZm2LIDeZpTe2Me/9i6FLox2qFrlXoA8ljTUWT9jlqt9E+/jjAJjc563VXPyJeRi0OBK3GGwXre70fAmzi5TATu05CckFmpszFXvCO+5rbFEDNufOyvB7rEwIk8mYUWvjmDdHCZPLS62aohL59UX5oqJa63iGW8cPI9DMK5Fz1Lm7bOjrOcqZnA91u2NoIUuwwqIbueH20BAy24vf9129Kl6MKoXQLbUZ5nsDjetFzRY8c40OVVjSrR50JYkaHKLtoBqafvpx+CghGjrAqTDERZcYgQATtkgcJEiLxJrjinOdyt36pEIX8mevmWBBmfBhoSQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kfsFU7h60ekNCwpqBfc9KZzaCZFPTcN0ePlA0PIykoY=; b=ZpCDc8Wzgb+dveRdpXCY27muPs4Uew6QxD5mx6wRlGSI99/ruO52J76nS2wei6FtRtsbC5Eh7bmNHGPD0EH0XFwttJF9QMzUED+7pvjzl8ZuoLS8myPDbzn9X8SRXKcowe4fZh1UtvoKTiiEhRFvYYwP/Ns0zHCePdPPVc0WzQjIypj8DdfORb5rbcR35AfkZVRSAzuYHhKTxbmdEpP+IqWgtKbRfPk1+DHOZLXBeC0JOUldJN8AEYu9Bb1S6I9U9B6lLfPLKz7AXtUcAMetrySoZcAWsVMgmGkMAUKwhQdcntkfYxSLUTCLxGx5r41Ijm5yhqTZ6ZGh6bz8e7Xrwg== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by BL3P223MB0241.NAMP223.PROD.OUTLOOK.COM (2603:10b6:208:34e::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13; Tue, 24 May 2022 15:29:12 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::c536:493f:7cda:53dc]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::c536:493f:7cda:53dc%3]) with mapi id 15.20.5273.023; Tue, 24 May 2022 15:29:12 +0000 From: Soft Works To: FFmpeg development discussions and patches Thread-Topic: [FFmpeg-devel] [PATCH 2/2] libavfi/dnn: add LibTorch as one of DNN backend Thread-Index: AQHYbomLzmIMQM3SmEm2rxIs+uAn3K0sOCoAgAHY1gCAAA10gIAAAcJQ Date: Tue, 24 May 2022 15:29:12 +0000 Message-ID: References: <20220523092918.9548-1-ting.fu@intel.com> <20220523092918.9548-2-ting.fu@intel.com> <01371577-00b1-4b3f-b964-59e2359f2a69@www.fastmail.com> <8f3540a8-f902-4f6d-a761-9bd59fc094df@www.fastmail.com> In-Reply-To: <8f3540a8-f902-4f6d-a761-9bd59fc094df@www.fastmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tmn: [vojOqBNdmYJnTRXMt3yNqjNrx4amX8RY] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: f1183361-0b2b-4832-6360-08da3d9a279e x-ms-traffictypediagnostic: BL3P223MB0241:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 9mYR5nRXH/JsZ3WJHOj1GqiBsE08tFq4R+bu4ZLbUmXm2tKdU6p8VhqGdSdA3SmA2LAJvjOR7AWkRAGzz5Bl4wqzA3GyX7j6BSorwDt561iP8PE8Vcguy1kOiK+2gm+vBL87LYmbsCSm+pR0jAz7bZqwynPfAr6aO36vSwvBR0dIM7PGdpYOFaKxqM1N5WyfAUNxgResDMdJGvaxCtojoh/+h+sJ66H/YBfg1sy9jP5R1pPn71ZDxn7x/9YlAu4oOaSR4z2MKOTxWDDnKKBWfsC/Mu+p09MkoVHNXOUwL9CptzYYtXC+onxIFc/UbjuVLcKkYvpVOZHYYcjqRyihnZlhUbDvbB5jFBza3KReJmKPRC27BX5dltcVdWE/KtLKOacaECiJkZWHn9u1evJLep0babOt78TnhNSfqobKbc95raCwZ4r8BLVS7A8DNhKTPTzy0AyuucjxGsraBJJ0cjNFAXblKAJMWc1sGwqeGQRrvtbiw7BjHQImfM3a+JycERRfCpqfGIKqKWVXanVVIaiOfy+2xIjJrJNkbUgpl8AWkbNcTiOHvU283T/cRKmiUU//1L9a1oVEgupOmzPa3w== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?B?TWNUbSt2dWQ4My9YTUkyWUIrYmN2STVSVzdqK0Z2MStmOVRaRFNxRStqMmJl?= =?utf-8?B?Vm1NbXd1dFNEQ1Z1RFFENDBWQjhxZEJ2eDdhd0Jpai9TMlVneUEyYjBrVXo1?= =?utf-8?B?TUJFS3lPQUg1WE5PZnJYM1crNXYvQjVPR1l1aEJndXlENExab1MzQWI4b3Er?= =?utf-8?B?Q2daZjMwaUN5ZXU0RTBiTDNmdlk0VHNoN0M2aEtCSmpRMGRKOGdqRDJyeVF6?= =?utf-8?B?bGFQYzRMdURseHNHekxsOGFNL3VhWjJQdG1JcXY2bWhnV1dFdytNNXpVWkt2?= =?utf-8?B?S29OWnJyY3NTT1FpbjN0bkQxeDUxNEdrTGFheng3MVJFQjVJUUdVcWE1SEg3?= =?utf-8?B?eGU0Uml5cjhKVmlhOGRUUlFIMzE5NjBTMVVoNXFQYUQ4WFhlYlorek5xaVpi?= =?utf-8?B?MVliTHhvRHhJdHNwYVNNZjN4cDlFR283aEhEbFBlUWtVdjljUWZFTHFDbVBx?= =?utf-8?B?TDJ0R2cySzdKcC9HVTdMUXRtbE92SFg4K0JRUnU5N2ZhSThERVpwdDh5SThv?= =?utf-8?B?b1BrWUE4UWJoQjVmeStONU9NR0NQUDNBSSsyVjZKeC9BbWZSUFlRVDhraHRS?= =?utf-8?B?MWRGVXdkR2Q1dWMyenNTVVM3eDhoZVhLanMwNW12MkFNWGxNS3RzSEFhaHJp?= =?utf-8?B?cVJ6akl0MGRBTEx3bG90b0xNVUZEUlk5UDc4VzBlRFBSMEZUYUQzWi84MFN4?= =?utf-8?B?djc4cHo4YitFOXVMNFdKVWxVMnVnZUdKeVJVNGZ1UFhXYVBkbURZMU96L05M?= =?utf-8?B?UzgxZ2pJdTdhbG1WUGVxc1lRL2tDcEREZ1BzVlZDMzQ2djArUmhhNXljeEdC?= =?utf-8?B?YTJiL002cWYrSTI4ajRaZkU1V3NYKzFQVnhseFFwaUlGZzdCajB0L2ZLeDVN?= =?utf-8?B?YVVxVEVxR0hDd0hFMTQwWmllUWx0dkdkME5sYkYyNU90aEJHdmtkYjVzNmxh?= =?utf-8?B?REhwN3FNQ2VUcnd2aFNYZDkvNTFOQlRCZG91REMvM2NiOFNEdzA3OUhQS2oy?= =?utf-8?B?dTE4b2tVVFVWTHFwTmJJckJyblk0eTFvdUxOeTk5SXkyZXN3QVdkbWgvSnR2?= =?utf-8?B?ank4ZWFOQ2ROaVAwQWs0aEZQd2RDc2s3ZzFzVmljNzlSc2ZGV3hLb0llY3Fl?= =?utf-8?B?MWNVcXRVZnladUtJbVFVTFNEa3k5a3RCUFNjUmJzdTROdU14VFFzUE52K2ZP?= =?utf-8?B?b3hRajZUL1ZldEVobzlBQ2sxbGFEbmQwUHBhWEVOZ3llTTljQ0FKeldGc2xE?= =?utf-8?B?aDNQc01lTXhCNjZ1QlpUeEdyZWdpYk1zQnYrN3FFdkFwVDJ4aEkwYWVVRG5x?= =?utf-8?B?c0R5SmtpTXQ5K29SMHZHU1BFVzZCY2o5TWt4QlU3TmlqVk9WY0t1Mk1sbWdt?= =?utf-8?B?US8rTks1RThZcFprbXk5ZVNpOE00UlZRemlRR3RRVWxLR0I2K1J1RlZZTmNy?= =?utf-8?B?MmZ0dk1TSWpaem80UlFsVk9ucjlOdW95R2dkam1PNnRzTXFXNFJVWEpqcGhq?= =?utf-8?B?MDAzRnJ2UEpvQ3dhSk4wM1JuT3MveHRheGIyYWNQV3R0T3NaK3JPN2NIbG9Y?= =?utf-8?B?RTVpN3pLUWRrcFJ4bE5IVUpNVE9uNFZUa3FKVnMxK1hkem55dDJ4dEhzb1hv?= =?utf-8?B?YXVvNTcwZ2tVeHIwbjAza0ZqSC8rSEk2djJKbjBZaDFZaU14dXlZZmUwM2Vq?= =?utf-8?B?SjZIZGJ1bkNMNG5wQlh1M05wbE9sczlHWitZZDJ3c2tzQjRXUFRWb0lhYUh4?= =?utf-8?B?T1dtSWhKOUV1d0ZPWDUrR3IwUmxReWx1OEV1UHFUeFhROGYwQUdON1ZOS1RJ?= =?utf-8?B?d3hoZlUrV3VuUWhGV1locHFtdDB2Rys3Unpuc3ZwNDcwcFNUQ2VPenFSaVBK?= =?utf-8?B?OXJ6THZIYkQ5SDVjQm5qTGVValdITXJPMzN3OTlDNlYxQjRjQkxOZ3VVT2dS?= =?utf-8?Q?EYiXBgJELIs=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: f1183361-0b2b-4832-6360-08da3d9a279e X-MS-Exchange-CrossTenant-originalarrivaltime: 24 May 2022 15:29:12.4970 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3P223MB0241 Subject: Re: [FFmpeg-devel] [PATCH 2/2] libavfi/dnn: add LibTorch as one of DNN backend X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: > -----Original Message----- > From: ffmpeg-devel On Behalf Of Jean- > Baptiste Kempf > Sent: Tuesday, May 24, 2022 4:52 PM > To: ffmpeg-devel > Subject: Re: [FFmpeg-devel] [PATCH 2/2] libavfi/dnn: add LibTorch as one > of DNN backend > > Hello, > > On Tue, 24 May 2022, at 16:03, Fu, Ting wrote: > > I am trying to add this backend since we got some users who have > > interest in doing PyTorch model(BasicVSR model) inference with FFmpeg. > > I think you are missing my point here. > We already have 3 backends (TF, Native, OpenVino) in FFmpeg. > Those are not to support different hardware, but different tastes for > users, who prefer one API to another one. It's not just about taste. Many models can only work with a specific backend and converting is often between difficult and impossible. > Where does it end? How many of those backends will we get? 10? >From my pov, the best solution not only for this but also for other use cases would be - as suggested a while ago - a plug-in model for filters. > What's the value to do that development inside ffmpeg? That's connected to my question about 1-way or 2-way interaction with those APIs. When it's just about feeding video frames into such APIs, then there wouldn't be much reason for having this integrated into ffmpeg. But as soon as you want to make modifications to video frames, how could it be implemented otherwise? I mean, none of those APIs are capable to do video processing like ffmpeg can do. In any case, there needs to be some way to interact with those APIs and at both sides, at a certain point, you need to have uncompressed images to work with, and when that memory could be shared between ffmpeg and the AI API, it saves memory and you get rid of encoding/decoding and load/save for sharing the image between ffmpeg and the AI api. Also, that kind of integration allows processing with lower latency, which is crucial when working with live video. I've never used it like this, but I'm sure I will. Kind regards, softworkz _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".