From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id BE449431F5 for ; Wed, 25 May 2022 03:50:43 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D41F168B4E7; Wed, 25 May 2022 06:50:40 +0300 (EEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6C09C68B4C3 for ; Wed, 25 May 2022 06:50:34 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1653450639; x=1684986639; h=from:to:subject:date:message-id:references:in-reply-to: content-transfer-encoding:mime-version; bh=FYMj+4FHindxNmXt7sOT/QRpcGce0zt2Ue0GSNd9b1I=; b=hUb/3qQimaaDDeWzGv2RfT3ipfSt5wmiVD3rum0ThKvshV4Aeyw6Ikle PPq5sAmx9DdoNrcgiS1UshHho8e2paYr6EFmKegWPhOjw9MKxwDc/cb9s qJlA9Xsj38kHKS8Xsr53ahdtxVJMCw5pbP1j4KFhku1hluvsUOIpkBuoe gpg1lO20bE5C5u9jwk8uWwPpWHxO/JUhsSq6U+r5fNmECkhKNe2JY67Zu 2qzvNG9a7mMa2silrzU6F2kSSFmU9M8ZeafP4XVDWXxwuuLGdSI5Lzxcj GPnrCFFmYtAtFn8K33zPcRPKgT8BuRzGOww6JLHaMbTTGPj7KDVaQv/Hb g==; X-IronPort-AV: E=McAfee;i="6400,9594,10357"; a="273435836" X-IronPort-AV: E=Sophos;i="5.91,250,1647327600"; d="scan'208";a="273435836" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 May 2022 20:50:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,250,1647327600"; d="scan'208";a="608947102" Received: from orsmsx603.amr.corp.intel.com ([10.22.229.16]) by orsmga001.jf.intel.com with ESMTP; 24 May 2022 20:50:30 -0700 Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Tue, 24 May 2022 20:50:30 -0700 Received: from orsmsx604.amr.corp.intel.com (10.22.229.17) by ORSMSX611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Tue, 24 May 2022 20:50:29 -0700 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx604.amr.corp.intel.com (10.22.229.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Tue, 24 May 2022 20:50:29 -0700 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.46) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Tue, 24 May 2022 20:50:29 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CDbqy2C0WDBO/0qiV0042OyHxDq9A+Gu9uvqk0WxkXuzkPVVnVNw6t6HPBTE4WBKAcLdctDnSjGCxldv2jbPv5/8Ww126pdiX2O5edcn2/UjfQJ0RMFdyeLh482adGFIVsD5zjTqfw0s/Me9vgJwtqki+oVnqOejNEiL1CDjDSZ2/sEKsu0ztj+rbL7bU9u2uETLDdQLMFipLlsnwjYx7XPjFilu5KilUvKyW8hdqUJZdnbiFpadkDeHa9VHaDdijebpzzV2bYIznrhVItnnyNgW0gkxkib2ylTxj9paQA922RvRCuh+78nmPqPNRXDOeLs5vW/V8Sk3aIXNWENu4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=oWCp3qOXD0cjoyhUGHVTZU5moBC4ZliSFaWeGq7bv+Y=; b=ix/TvUKdKiqercjaB//j//Rbl9rqg74ZRuQKavG9htfDWiWyvvnYtysZbUWtZGvECEgupCZTzewiqWxeyayQ4zjVBTJq3bP09G6I6Jehzdb1V6Vz3RHaZ/YB3Zq6XcPBFokO5P32K17co3eDpghZv3598R+uwoDU3L2IlhTOXRtO7WZP18Yui/P6ivpTK77IDDVUQ3nylECGORdD386CAK3qKtb4TzOGLII4inRcXUbfksALqSBqt4EkCLA6KdXyeCXwC3R/nKkzKGIIs+cVe1D0bZx5FeTTBUdAMr2i3FZoM18qls8DRvW1eewmJVttNfsDFIyhSWpeLRP1IcpyIA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from SN6PR11MB3117.namprd11.prod.outlook.com (2603:10b6:805:d7::32) by BN6PR11MB1908.namprd11.prod.outlook.com (2603:10b6:404:103::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5273.14; Wed, 25 May 2022 03:50:27 +0000 Received: from SN6PR11MB3117.namprd11.prod.outlook.com ([fe80::bc81:218:55ac:793b]) by SN6PR11MB3117.namprd11.prod.outlook.com ([fe80::bc81:218:55ac:793b%5]) with mapi id 15.20.5250.018; Wed, 25 May 2022 03:50:27 +0000 From: "Fu, Ting" To: FFmpeg development discussions and patches Thread-Topic: [FFmpeg-devel] [PATCH 2/2] libavfi/dnn: add LibTorch as one of DNN backend Thread-Index: AQHYbomVST9trs8LaUOR/SdRrLdfHa0sOCoAgAHUnpCAABGrgIAA0Ucg Date: Wed, 25 May 2022 03:50:27 +0000 Message-ID: References: <20220523092918.9548-1-ting.fu@intel.com> <20220523092918.9548-2-ting.fu@intel.com> <01371577-00b1-4b3f-b964-59e2359f2a69@www.fastmail.com> <8f3540a8-f902-4f6d-a761-9bd59fc094df@www.fastmail.com> In-Reply-To: <8f3540a8-f902-4f6d-a761-9bd59fc094df@www.fastmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.6.500.17 dlp-product: dlpe-windows dlp-reaction: no-action authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6b4d8a4a-8ffb-4164-6483-08da3e01b4e4 x-ms-traffictypediagnostic: BN6PR11MB1908:EE_ x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: RzrdIA0SWWEnH2pwwcEpMLQSqZKln8Od27Pd1aHKYFj34SFH8xh2APLgl/3nWY6Ph4mqu8iWLXsrhgWt4S2Xw+f9A9acTWAh4ca2nT3GO+sbFLuPPrKLh8t8kpv0eHCoRzFFe5Qw/AMF/j+q5Bc8s2qvBTHsyFVIlp3ZQCp0qhICOodFdJm7HOh+RcdMZxrC0tYUFe8kvphg6u3d/BHYRj6ia6L7kx4U3vZlHG4Io/Ry3aCtvEqXavN1f5lpHrfNgtNN5LYLbOBGcY5pDKWmSbSkk3ICVuJAfL5VYSW5yXEGIYFnI2IafI5iwbylWnWMF0nVnLsnmDcEvSblOSIELQCOcHgbuJZKnI0/h7EwLJw0vkoafhoMmM6jRf8/vZoo920gDS4MXsQw5zj5QsREyZQBYsz9n6z3RrXjbtfvHOOceqtcTjVEMHyqdFWny2L3T/4bqNtiHl0+4P4HD6PbdeVSWE3p02rZrinX1CZxQJu70oK/khrfW4SITqozTGHzPy3tnIakk4B7G3UWKWT9YGCgtZCgtMEfFB3PgWkBxW+3dEwCOd7bC7lgKKkDgFFznONzMDI+0OdhsTTttCuRTVt72LnNYROHX9wKS06ZcPDQmJbcWFmHvwJ6VrEYiBC4qFVc61ON0bEaTCDvxtfR+RXMcnYSTvN4j83N0X6Ml/lFri4nOQEzjCAPcOBhDmsOlJGKOx+4p8ECQVE2fcYwLrDRZcajpM/pTX2q/l/yYe0HGgc9+fmh3uqkvdR3quhka8QVCJuMC7UZVp/UW0Z88TkcSOjAu4cjkjv/inIhXNo= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR11MB3117.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(186003)(122000001)(2906002)(66476007)(5660300002)(508600001)(38070700005)(71200400001)(9686003)(86362001)(66946007)(52536014)(7696005)(76116006)(66556008)(8676002)(316002)(55016003)(8936002)(33656002)(966005)(66446008)(64756008)(83380400001)(26005)(6506007)(82960400001)(38100700002)(6916009)(53546011); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?+QVj2bmbb63P30XvlbaMOj4K19t6zP6SlK3plPRWhUzJz0waf0LmkFzr54ah?= =?us-ascii?Q?WEBvFBY0vZLJRJe1QIlUn6sMhfz44BSu5f5tB3Jo8dYlgYp+LvaSlK+CjhJu?= =?us-ascii?Q?0XHogjHknB30kfeYz134zVjAPjghAum7nb0yPqE9QiAZ5s0H+AXQU6mODph9?= =?us-ascii?Q?i9bGwb5kDRG2Th7oev4d0oTSwDGjfjrYGPY70bohksmyhP9MaurqWGGQnRKx?= =?us-ascii?Q?Az9NFwp/sCcR1Pk9Sd4Y/jvdjeI3Kd1Kwf7FyfBRV1XvS2R4zBxBa9HtsdgJ?= =?us-ascii?Q?nmIESBVViW4tdMx4iLI0I4tj3DRWyOBLwG9Jq+6VADc+U1RqID4zVGOd9d5C?= =?us-ascii?Q?O6GEczspUz9QqulKRNSVQS23s8i6wQFRK4O6Wp4Fs5vwWJ0JSU5oiwsZPH5O?= =?us-ascii?Q?YuR3Ms+UEhzg78k88gS3c3sG8kX2YEQcjbNlUYvz2RIrb1qUiN1xKdWZ2hQk?= =?us-ascii?Q?RxdavGjj9fABcXRdxvXQUQ2eS0sUB2pUI4WkZHlyrBcInjerGV6Uoi7rTeFO?= =?us-ascii?Q?drmC1SGz3r9H/TYk+bGSUTJhsA57510ZJCuI1+WdPj95u+wi5dylwczVjc9B?= =?us-ascii?Q?IR0xh9Us9lBPeXAo5jSR73erE1vC88mf1E9ZD+2VgYVRse31t+4H6rAXhRKP?= =?us-ascii?Q?eUt0cpdOUHyFusHncLrsWYVoH33FqyC31/2DN/4FwpXh6OewNu3WOSHNq5B8?= =?us-ascii?Q?K5R8UBZDCkH55JkpOGI2O2w5kJXVqLBy6S5HIuT9zK5eSON7v4ejbTC2/mOX?= =?us-ascii?Q?XQjtY1dV+wv3qoiDXDICepcpkqyya01cNrEuJi7FfV1sb5M2JFjwsDnv+WEg?= =?us-ascii?Q?cPLY+sBb4k04bHHx7XbSJrvF5bx/qHOYPiNhwniqxmrsAKMl0k01VCKBYA52?= =?us-ascii?Q?LGUPItJlY3sn9+BWIY3YZfReRQZZhmy30/dv1FJEwoPx6AuVuC0ilcZyHZKi?= =?us-ascii?Q?lp9BvsieOifDONF/EMHRP76hWjB1+wqNKdQQaFKvXo0Pyw/WBnFD/uHCO/oO?= =?us-ascii?Q?f+YG4c5K+3bli7YM6JXtlry5p3mP5KRbgsMQpXxYkB1jCzYfc3pjUdyZXt4N?= =?us-ascii?Q?GA3ugLtIjWPXlyX1Zd1dKJTqRmFq+BwoHMUGh9Cet1uQuCK3WKSkFWtZrDJn?= =?us-ascii?Q?bVYLPGD0A7D03vvuYH+jJDmEkAOiN/scEPn3tTMBfDv1i2CIbl9hWwocpDsw?= =?us-ascii?Q?gcQUYZjnwZ1VgInk8Yw3BUJkgXzIMHf4yilNH3Mx/E/b4EiBhe14NWK2P+Ij?= =?us-ascii?Q?OPT8W9yjeCd4QfmLK2w3TAHCQSFwtgASv563FlbFY/KGRzMhIXw+qNcunKup?= =?us-ascii?Q?zikE7pG4TxgburLBDkLQcIHRapp5OltdDPEV14EinnUtN37zLOqPdhdvuaz+?= =?us-ascii?Q?qSLTCXovDyFkmiRjMKEXfiYEll8LWndvnxcDOAFkRI1Jbdo49GblNe5zEK/H?= =?us-ascii?Q?/TVIEJRNh+Q83WrfFteyM309TWN5jQodH9HGnbEMP3FRMaInhl7LulMejCs6?= =?us-ascii?Q?pQ2ZZQXFrCVSEXWXi6hrRx3X75qVMId9RRoh0rxWa21xz3z2BEJ4w83CD0VU?= =?us-ascii?Q?Xl7HCVhlaySEQAFoavYxv/gQcvpdbRHJm+erjI+qH93SL9IGCHnRKuCD99ic?= =?us-ascii?Q?UFYgtiINFx9Z4WLch2GhQWK3XsjG+fJzkFUnWgxJFJERFG0thWNdDz8whrsC?= =?us-ascii?Q?R/gP2fIzAx98YaYPB2tCrScj8Y76djc/hP4fVJ1unLGwFWQecod2vEYpg4ib?= =?us-ascii?Q?Tbrc/wRB4A=3D=3D?= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR11MB3117.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6b4d8a4a-8ffb-4164-6483-08da3e01b4e4 X-MS-Exchange-CrossTenant-originalarrivaltime: 25 May 2022 03:50:27.6946 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 6mZ/4KGAguA4+3MMfjKF9McQnzSyEvmEa7QNRhRvvGaTs1xA1t09zoK9XkjUXz8VylV67OGdCUVPa3UXPcdmdg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR11MB1908 X-OriginatorOrg: intel.com Subject: Re: [FFmpeg-devel] [PATCH 2/2] libavfi/dnn: add LibTorch as one of DNN backend X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: > -----Original Message----- > From: ffmpeg-devel On Behalf Of > Jean-Baptiste Kempf > Sent: Tuesday, May 24, 2022 10:52 PM > To: ffmpeg-devel > Subject: Re: [FFmpeg-devel] [PATCH 2/2] libavfi/dnn: add LibTorch as one of > DNN backend > > Hello, > > On Tue, 24 May 2022, at 16:03, Fu, Ting wrote: > > I am trying to add this backend since we got some users who have > > interest in doing PyTorch model(BasicVSR model) inference with FFmpeg. > > I think you are missing my point here. > We already have 3 backends (TF, Native, OpenVino) in FFmpeg. > Those are not to support different hardware, but different tastes for users, Hi Jean-Baptiste, Yes, you are right, we already got three backends with FFmpeg DNN. But for now, the native backend is barely workable, due to its layers and operations weak support. And we do support different hardware. Like, the OpenVINO backend supports inference with Intel GPU. For now, the TensorFlow and OpenVINO backend support some models, which include Super Resolution model, object detect model, object classify model. I think it's not only a teste difference for users, but an option for them to choose for their work implementation. AFAIK, there are some individuals and organizations who are using FFmpeg DNN. > who prefer one API to another one. > Where does it end? How many of those backends will we get? 10? > > What's the value to do that development inside ffmpeg? > I think you are concerning why we need such backend. Because the users want to infer the BasicVSR and other VSR(video super solution) model. Those models are most implemented with PyTorch. And it can cause several issues if we convert such model to the other AI model file. Besides, the video codec is an advantage of FFmpeg framework, which can support various of hardware acceleration. We would like to utilize this framework to enhance the performance of AI inference and improve the user experience. What I want to emphasis is that the LibTorch backend is not for adding patches but an actual requirement. Thank you. Ting FU > -- > Jean-Baptiste Kempf - President > +33 672 704 734 > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org > with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".