From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id AFE7A431F0 for ; Wed, 25 May 2022 03:20:20 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 408BD68B4B3; Wed, 25 May 2022 06:20:17 +0300 (EEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3331C68B449 for ; Wed, 25 May 2022 06:20:09 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1653448815; x=1684984815; h=from:to:subject:date:message-id:references:in-reply-to: content-transfer-encoding:mime-version; bh=5jekXYB7T24/pt4uIY68/LmeRv4jqfH2xBjK2eVQ3W0=; b=HHFVGIKnzqmBm5McFPU96lkKOr16i1RtvJ18wjSulBWuy1OG3ZJyV3rT Xum871Yvx0g06Zx9sUPRztQWRVT8ysHa0lvDL+rZM4QwIoOV31USnbn9R jxXHCOXqN/jMfJNBE3ZvzhSm1Q4hjaiaVwMJzSWzBQqVlcm1oH0+11oDh S6SOE7RHDZp8BIxkHmJD5DS7DiDcMPnJt3aAY4kB0IbA7eWiltUKJBA4o 1To9Kygsnuss1RYJbkyVldXrEamM0dtLaJpyGQj6hU1tL/46PHPlDp/7m +M7I5VCU4KbxunTNimt+h03GVxkcBIvxyyry+JJTyeZ7+cND/hG7keUqj w==; X-IronPort-AV: E=McAfee;i="6400,9594,10357"; a="273431745" X-IronPort-AV: E=Sophos;i="5.91,250,1647327600"; d="scan'208";a="273431745" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 May 2022 20:20:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,250,1647327600"; d="scan'208";a="601623339" Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by orsmga008.jf.intel.com with ESMTP; 24 May 2022 20:20:06 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Tue, 24 May 2022 20:20:06 -0700 Received: from fmsmsx603.amr.corp.intel.com (10.18.126.83) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Tue, 24 May 2022 20:20:05 -0700 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Tue, 24 May 2022 20:20:05 -0700 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.105) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Tue, 24 May 2022 20:20:05 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FtL9pfVkV4vGSow4n0X3RP8PkCOOhLvGEdfP8DIhk3eCcBMi4Tf5dsrM0DhnQjYAG9CEItcO8e1PYLZdvRDMCTdrb82EYL0uRzBAR/vifGbz3rhjgddQi4cyw0HTs4bHsaKQl6j2v1GC77HbdNtaqXiPNIiXQqxAIk5bzvYjI29/07hg5x45B1m25LV/IHIzPqp8gKiac7N42XPhIcu81mghyon/BPTMBN14TXFpK6D37rG2E0QE7vBNJTtKuaTHI6XqYRZBSUw0zRJJpEAs3JVEr0gLmIN0q5sFz1oDBR+RPljWfMLttttHvhl8ndoWRRZBLc1CBaZAqJHI99CCyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5jekXYB7T24/pt4uIY68/LmeRv4jqfH2xBjK2eVQ3W0=; b=m0lJreJfkN10JlXsxuP6ytmzyq+KZy1rBe5fs/BdSLnRTa/IQtvSIoe3Som6obbPCvlZGk2e/8DJW1V9QjLq6w0asU3p8vptSLRuVl1RzyUR27ohJifqbPHmoaP1vPNfLsJbmT4qy1YDlj0BNPdefUibPif8h5eS7AV9GDxrwfIyEdDujCKjTPX8FvnvAvqXqmYCHDJYUJ5lmf91iIl8ieaAXx2x0gW0A0i0q0KnmImWlNNIq9OQpU0DTgsMJ+E1+g1QybKpWSjgkwJIvTGedMS0ATNRshCPO/mBRdG5+85sPxUjS5hE76ySpbPIMxNbaKopQqYztYrASsdadhK2Yw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from SN6PR11MB3117.namprd11.prod.outlook.com (2603:10b6:805:d7::32) by SA2PR11MB5065.namprd11.prod.outlook.com (2603:10b6:806:115::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5273.14; Wed, 25 May 2022 03:20:04 +0000 Received: from SN6PR11MB3117.namprd11.prod.outlook.com ([fe80::bc81:218:55ac:793b]) by SN6PR11MB3117.namprd11.prod.outlook.com ([fe80::bc81:218:55ac:793b%5]) with mapi id 15.20.5250.018; Wed, 25 May 2022 03:20:04 +0000 From: "Fu, Ting" To: FFmpeg development discussions and patches Thread-Topic: [FFmpeg-devel] [PATCH 2/2] libavfi/dnn: add LibTorch as one of DNN backend Thread-Index: AQHYbomVST9trs8LaUOR/SdRrLdfHa0sOCoAgAHUnpCAAAnigIAA1Tjw Date: Wed, 25 May 2022 03:20:04 +0000 Message-ID: References: <20220523092918.9548-1-ting.fu@intel.com> <20220523092918.9548-2-ting.fu@intel.com> <01371577-00b1-4b3f-b964-59e2359f2a69@www.fastmail.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.6.500.17 dlp-product: dlpe-windows dlp-reaction: no-action authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e1e8f3f0-2e2b-484d-66a7-08da3dfd75f2 x-ms-traffictypediagnostic: SA2PR11MB5065:EE_ x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: wz7h0FeR80BJj2WWJd0S00JuPZZM79juBuNYSDIcb4y7fXFWAAw/mapybL1FHAEo28vMDbj2EwGZ4KJrRPTHgzL9ZBvm/1OCTb6wDUZXwpHwQ3mmStvSXoUyf0O6zRKg1J9DBs/idbgWcTi89wKsBEJqNS94N8TKBZ7Y2+dHZQ2Ktg4VX+he+dmWaHffChMRSUOZqbilrK0ZxjQ6SkTigIjmeEawNYs0Z1BoXWHynn5ySLOhOIXZUHIwdzmO2b7YHdrrWE/TyTXntmgDROg8oB8lUoIRe8c1jeW36xNBlVAPHH9i5rlWJtuNeO7EEnjUDmE5iBSRDlhobiXFOFavdaXNQ5iEzG7Apmn1wijo5AKBZ61w7lMAyI9EwhHyBrCEimlBcpUiAzXqNViVk/Lbetudj/HtnD3skjEKoALD4DSAcBdDvALbq4MQXD5QFEzANfA9ECmWcKuJV4YAmEcAxzpj+ldHLkseRnpcDuYqwfUMBolLfHWmiEZ6og17gPPB8g+LtW1gCRZNHHUbM2eC29epUzaEORv3MEunic3YsWLKQfdFkegE8+ysnS/1kZaRaJ92N0Ho7zOkf2/oUAhH8ZNxuZqSumqglRbDuJGJ65IR50GbDUMIABng9hIUOlV7/pkKuIUTJZ2/MVwYlkqynayySzpFbApL/Y8wnx3oeSeXaTaEWCqQpSegMkJkwni8wtXGDA8FOn0kZWjtC3Ii6I+l0+j4ykkJKn3XiaYMRUbxtmJSGvCEALOwK8VejO35u+LGJD9ABmp+5n1zv4xaNRP3zFrOxB+zbtx90rXN8VY= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR11MB3117.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(55016003)(9686003)(5660300002)(26005)(508600001)(966005)(53546011)(52536014)(2906002)(7696005)(33656002)(6506007)(8936002)(8676002)(316002)(83380400001)(38070700005)(6916009)(38100700002)(122000001)(82960400001)(66946007)(66476007)(66556008)(76116006)(186003)(86362001)(64756008)(66446008)(71200400001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?+ZFogzahaY48fD44kYa/Bm3vO9IFag8A2bRkn7Lw48ErRM+/7HW/PNpuaviN?= =?us-ascii?Q?bkCrqy1iDn8Fi4DLS+LDZV4SlwNfFXeyKjl+GA8l6SS1ngcdWrUUqv0Ro9d3?= =?us-ascii?Q?t7Oldxvd4rF5b6EqvCCtCtxPMqclayv5gDXS7cRI5iXwDZvJ5L9/Llg4/ZpH?= =?us-ascii?Q?HbFVrh6C/GcHac9dKypIzLjUjvj3iyPQWKujAc8H1Y3jXyMXimimoguG7ycg?= =?us-ascii?Q?2KI/rudHEVloXAD7RTlBc6R5S8Ac1jj4xgTOt3blYcX2CY2uaf8m5JbhL73s?= =?us-ascii?Q?AMaMkt2oHHhFcPvi6Iu9hrJYq0879e7ArQvhObYzbj8KxU6+Pj2xR41dP1kD?= =?us-ascii?Q?KbNxYpaZqoMCqzOS1mhZJuJgGR4FKtMQMjZ7l854pZn0qrIWDzL0AnaJ/SKa?= =?us-ascii?Q?d5K4DF9ECGw0VHisEY5FxOfamRTDJzvTqB7gejUmcspTvJOIuYOl8xFtqv9C?= =?us-ascii?Q?+U84tbwNSVX0yLcAY+twLBSvDrkl3LLkDIzOHCVvLnbpP58oUSDLmAa8OxiF?= =?us-ascii?Q?Fb+MLBYWDwy5hwqa9YR+n/A4dzAHvc4+hmsdw642u9Bgng3tTiPIdFOYt7ts?= =?us-ascii?Q?ROZ+7HEXmjW1/CvXw9vWp5OWCgRTuF3XHzjNiwpnjSnTUnCQsnMpZgaxAIfk?= =?us-ascii?Q?nUT7dkjpEEwSBbVuUaXUE2wklt4GFzup0pQqPoBFPS6SLOj9gCNUE9HicDiu?= =?us-ascii?Q?40EbCSijVyMj+y4RrkHFn/spduYcFGd2UU+VdpupZxs3fgm2gRWyulFfTDLU?= =?us-ascii?Q?apj1j7JTaFbAQhtb7zKtPHYEqsHJ2E65Bw3uz+1tNlZ8xvNv9BdIxmPitoCh?= =?us-ascii?Q?vEzE5pGSEousGtCd1JxQDvzdDqD4HrEoI4gYcxrRRmEwwUnNbP9AQlrDzgNA?= =?us-ascii?Q?xeIq6lwLF8irweFJb8E1vj6MQOfEjvFq+AV5MDr1gC8e+yYET2A+XIWyzHwh?= =?us-ascii?Q?XvspiMeiU45Mrz0MA8rYZ6Gd6U3CCPW9hIzkXaTFcC5JLWd2WE121FJgWuMz?= =?us-ascii?Q?cA4TCRosMXI42E4G1fxJTDyxTgy4JRA1lFkN8rMXwj38lNEXxSQeUoa/+Pt9?= =?us-ascii?Q?vBf1YnII1N6GUyVzb+BTpX3IPvWauE40tzfNtqordIvaor6x2lFHPDyEPkKN?= =?us-ascii?Q?62XapXJz1nJ0CIqsXgHseHw4SWfBd7xSPv69sgkbfJhtia/4RXR45cv2Nzyc?= =?us-ascii?Q?642n/vr2yLjjNag4X4bkJ5XRZsYZcfOk7h54lwoIrL14fSRfw5ugBsiZ5VgB?= =?us-ascii?Q?hpmJRNd5ZcUi5eR4P4qlAJ2N72WvRH40lXnXYccMhhaQ+Ov8sSNo1yl1A32C?= =?us-ascii?Q?BtyTmrrirmSauzZ4cK3TwvSsS0vIDZTeoW5pBtGHV2lnrhHRk+7blBSAL4Di?= =?us-ascii?Q?Eg9TcyzV69K7b8B+SXMP3+iE3NFsBlS3rPlaMddpEcMYQ8xubKJ7wDPZDeYH?= =?us-ascii?Q?CyLCtn9EJ+AUkp4iXfwLx8At4TV28n3PoB/FOmWUD2JKI15/WdBiICXFc8OW?= =?us-ascii?Q?XbE5jvvD+Afs1yLnzofe1XGAGSgWUKtk09WBykB+Zbc9dM/0rIl3lTSBJo4I?= =?us-ascii?Q?927WJROMqzPQYYaOP8vwy5bDiBtucWymH3LpVhPth5aKsj1xQG/RcAZElmUu?= =?us-ascii?Q?el+jVd/CWkFDQIRXjiL9IIlhqrE0wSSrb3YWQlbVcmDjpd0gu85HNNrGY2yH?= =?us-ascii?Q?C39M0NCcRZGrMFEoRBIRDJWeS8BkH9BsJoAR9FkKcDER0bvghiyKS916ihdp?= =?us-ascii?Q?mA5ivu8/YQ=3D=3D?= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR11MB3117.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: e1e8f3f0-2e2b-484d-66a7-08da3dfd75f2 X-MS-Exchange-CrossTenant-originalarrivaltime: 25 May 2022 03:20:04.1030 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: +KfD9tCEFDX3Yfum4SnoLu1C2WbRmqQeiyS7USqSte+Eg6iTcClHcHRhIX1FpXu0vvC4CmOu3VlafaxGeK2oVg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA2PR11MB5065 X-OriginatorOrg: intel.com Subject: Re: [FFmpeg-devel] [PATCH 2/2] libavfi/dnn: add LibTorch as one of DNN backend X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: > -----Original Message----- > From: ffmpeg-devel On Behalf Of Soft > Works > Sent: Tuesday, May 24, 2022 10:24 PM > To: FFmpeg development discussions and patches devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 2/2] libavfi/dnn: add LibTorch as one of > DNN backend > > > > > -----Original Message----- > > From: ffmpeg-devel On Behalf Of Fu, > > Ting > > Sent: Tuesday, May 24, 2022 4:03 PM > > To: FFmpeg development discussions and patches > > > > Subject: Re: [FFmpeg-devel] [PATCH 2/2] libavfi/dnn: add LibTorch as > > one of DNN backend > > > > Hi Jean-Baptiste, > > > > I am trying to add this backend since we got some users who have > > interest in doing PyTorch model(BasicVSR model) inference with FFmpeg. > > And as we all know, the PyTorch is one of the most popular AI > > inference engines and it has large number of models. So, I think if > > LibTorch is one of FFmpeg DNN backend, would help the PyTorch users a > lot. > > > > PS, ONNX is not in my plan. I am going to improve the LibTorch backend > > performance and make it compatible with more models in next steps. > > > > Thank you. > > Ting FU > > Hi Ting, > > I've never looked at the DNN part in ffmpeg, so just out of curiosity: > > Is this working 1-way or 2-way? What I mean is whether this is just about > feeding images to the AI engines or does the ffmpeg filter get some data in > return for each frame that is processed? Hi Softworkz, Since the DNN is a part of FFmpeg libavfilter, so it can work with other filters. Other filters can get the output(metadata or just frames) from DNN. > > So for example, in case of object identification/tracking, is it possible to get > identified rectangles back from the inference result, attach it to an AVFrame > so that a downstream filter could paint those rectangles on each video frame? > Yes, for your example object identification, we preserved the output in structure AVFrameSideData of AVFrame. So, the following filters can use such data. And for now, the AVFrameSideData we saved contains bounding box, the object position info, and the object category and confidence. Thank you. Ting FU > Thanks, > softworkz > > > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org > with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".