From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id 42AAD45483 for ; Tue, 18 Feb 2025 10:09:42 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 129E468BE13; Tue, 18 Feb 2025 12:09:38 +0200 (EET) Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 266B268BAFC for ; Tue, 18 Feb 2025 12:09:30 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739873377; x=1771409377; h=from:to:subject:date:message-id:references:in-reply-to: content-transfer-encoding:mime-version; bh=VsUUkLSYLTBth2VW1Dt/nDu45xtFp2dRmLp9Yk6tgEg=; b=XWnLTX1fxNGgKHqs0tM8y5/TNr3dm/75FPsmlxolbPLAOwgE4CYSmc9Q BTTchy85rQnsi6YOuWIs5rA1ot61CMZL8zzBPg/fISkZX7VkBBOhmeTEa mlqUVJ/mMZRc20crunuvITcnwz1tXyukoS4MC5a/16oOj+c6Nv8ZpfWca aBPnMhKsPcQVN9sjXDkwWA6Yfvji9XOFT8ilb6FaQSRzm7CeTsnBBoQvY F1LFv38pxTYbOfRaJdh7ctos2rSNx+39HKP+4cnQtvB2Af6xU2PqCCCN3 CB7gNTv7KArq9LL7qMxr4+f6D8qeqiZZZI+Zqdy3vLYREwh97wW5i76EU A==; X-CSE-ConnectionGUID: dP9QnpanQiyNd6nKJbF0iA== X-CSE-MsgGUID: K2Xce71uS4amq9p2Eo4u2A== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="50775513" X-IronPort-AV: E=Sophos;i="6.13,295,1732608000"; d="scan'208";a="50775513" Received: from orviesa007.jf.intel.com ([10.64.159.147]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2025 02:09:29 -0800 X-CSE-ConnectionGUID: UqwnEHO8R2uTZkwpFIYtdA== X-CSE-MsgGUID: MskeCoj9S12CUtuOsEfIKQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="114820050" Received: from orsmsx901.amr.corp.intel.com ([10.22.229.23]) by orviesa007.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2025 02:09:29 -0800 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Tue, 18 Feb 2025 02:09:28 -0800 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Tue, 18 Feb 2025 02:09:28 -0800 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.174) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Tue, 18 Feb 2025 02:09:27 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=u6yqirjw6NZ/fetvfjJeKPGZUjxI8IoRFHJfrGXVT54nqq2Wbb9clEzMhoniA32Yqe1sl5GCQg/3zf6d1hTMYjEj2yUUlSgW/L1JHljCm48xBJ+S1UZMuIr58fUudIeJvlxeF7h8ruBKGaYUrb84busu6bQqgrVMZODm5m/4sNq/NeNGYsgUJMa8mrQauN9SdWap6xj0S0PSAUav/poF5sliKt4ihdbjm++1cpzodOuO09gudGtE6rgW87IdSyj+jCc1tD3ZljiE3wx1axrnsKTQDiw26tPfYG1BMKRZaz30ih4OOpbNI5lPpsAnEoUjZ8oSLKVve/+dhRV4pYqIgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VsUUkLSYLTBth2VW1Dt/nDu45xtFp2dRmLp9Yk6tgEg=; b=gPwmKi0OxH7fa+nprteRm5zScLGdRb4MuSgEeKNYo6r6+eT4eIbne/2YXdA4IV14olX2zCsS5jzlejc+wYlScvRem+GPIHQWR36ycqDxFOrJbeJufiX+ESucfvUoDhg1Uy/yvJSYOoEhxrlYLvR8LRSdn6Lv8uJmBHJao/t3atyrevBMnvWwPd/qyjHVygqkFuK45iXBspH/TaJauJy6tOtiFTnW9I54h1EJP6oBydnqpT72cBXgeegphXNavoBK9vbXZu2A3EcMAK+CIhGUZ6IYoZgFqR45DGJ9UGW6cZVjtXWbuIqZgIiwO4B/aAb+Iz/bmi+EFHDKAGL8xGifvA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from PH7PR11MB5957.namprd11.prod.outlook.com (2603:10b6:510:1e0::14) by PH7PR11MB6747.namprd11.prod.outlook.com (2603:10b6:510:1b5::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.18; Tue, 18 Feb 2025 10:09:12 +0000 Received: from PH7PR11MB5957.namprd11.prod.outlook.com ([fe80::9ca7:5dd5:8a5c:d73e]) by PH7PR11MB5957.namprd11.prod.outlook.com ([fe80::9ca7:5dd5:8a5c:d73e%4]) with mapi id 15.20.8445.019; Tue, 18 Feb 2025 10:09:12 +0000 From: "Guo, Yejun" To: FFmpeg development discussions and patches Thread-Topic: [FFmpeg-devel] [PATCH] avfilter/dnn: add zero-shot image classification using CLIP models Thread-Index: AdtyhuRcwdeGikTTTrap4GomiMz7rQABg6HQA2r0iPAAR8jiHQAlTBCQ Date: Tue, 18 Feb 2025 10:09:12 +0000 Message-ID: References: <001701db728d$09fa37b0$1deea710$@gmail.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: PH7PR11MB5957:EE_|PH7PR11MB6747:EE_ x-ms-office365-filtering-correlation-id: 222b6370-b306-4194-fc10-08dd50044b21 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; ARA:13230040|376014|1800799024|4022899009|366016|7053199007|13003099007|38070700018; x-microsoft-antispam-message-info: =?utf-8?B?eU1NWUc0bDl4akhnWnY4dDVHOElDMnpTMlVKd25FZDVtd2pkV2N6VFA0SXBF?= =?utf-8?B?Um5YRGkwbHc4K0dtNHo5T2JsNGxuOW1nNTZlejNiZGY4WlZGcmRoN2lGVU44?= =?utf-8?B?UGxLN1p3RnlRZmc0ZGJ4czRGQVdGZ2VQRUJzUkU3dkJ4WGxxK3N6dEU3RE0v?= =?utf-8?B?TTU2WG1pRlNYQzBsWmxRMzhSRkNZSUlWTjBmZ2IrZ0piTnpPSldkbnRXcnA4?= =?utf-8?B?QVBpYmsxT2VwaFN0QXRrdTFKcDducTVqTGEzRmtyaVJkbUVLMm91dlVOZldG?= =?utf-8?B?M1dpTVJubnAxVVlJRE4wZDVISlVtOERoYUVGR2V0eXo2RU1GYWwvMFV1OG1n?= =?utf-8?B?SUZQVUlTRVBuaHc3YnVMNHhsejRUV1lybGp0T3dmZ3hvbmk3WGVsdTlvcW52?= =?utf-8?B?WXBNOXk4N25rVlR6TkJqeGF6VFkvUHJHQ2FycCtkT2JnelZ6cUZIbXkreGZs?= =?utf-8?B?cnJ5dndKUXZkaU4rbkxrclV2dTRvK2hEUkVsMkpyVmlDVHpsT2R6V3Z3UXJw?= =?utf-8?B?NU9UZGk4YVErRUhtQVh3cU5lU2VQejVwd1lwaDE1emlzdVJLYmpYd1VnT2M5?= =?utf-8?B?ZmNUenBFam4vV0pkdjRxNFlFdmxDNHdyZDVyS3JSWTBoTlJVSmJTU2NmdVgx?= =?utf-8?B?dmpmWEZKanZwUDRTSUZtSGtUOFd4QmJUcTA3b2Vsb3hDWHcxTnBtNEtMWDR0?= =?utf-8?B?NWxEZGxnYlkxUk1LY2JvelhMRWcxeTBNcHY2d2gxUjVkQnk4dzdPQkpuTWpC?= =?utf-8?B?eHh0bjc4S1luZDJ2anA0Z0FxMnhoWjRXRFBXejBqamJMR1hnSkRzUXkvTWZm?= =?utf-8?B?SzRxYkN3OCt4MzQ5ajcwL2NZRzJGT05JRzFlQUVCd2xoeklFQlFNb0ptOC8z?= =?utf-8?B?cTFERUFRUHpPZlQxR0ZqeFpTbllodkRZRzFRTHBhSEg2R1RLbDRJYWRTTHQw?= =?utf-8?B?SVpZOU9jdENxdlVMbk54M3g0ZTdUZEFGN1lJSzdqclJPdW92S0ZtZHhaZlRh?= =?utf-8?B?Zjc0cTMyK2JES2NDaUROdWtpYllodXRmdzZXcGRLZWxUQjF5RklCbVNaRndG?= =?utf-8?B?ejRGWTA0dUx5TkhRU0hYOUFOWEV2YXM1SGl4N01WaTdHb1gvWkJEOHhTdkx0?= =?utf-8?B?QVZsR1hQc3VyMW5wcUZUQzd4N1VXZlk0aDlHb0lGRVVaaklnRkJHRWxHWFhU?= =?utf-8?B?dnZDOTdWRkFZWXRRNnFtdW5xa2Q0Q2tmL2NwWUJyT1J2NThhYU5tZGVTOFhv?= =?utf-8?B?RVBkeHZLOVRnVkFQK1BUb2tqVzA5ZHdMTjlXYTNzNExwK1poZjlrd09BMm9G?= =?utf-8?B?T2pTRlgxK1d1VDI5VGVQbExTS2pDa01CTmI0Wm9EbzFmM2lHWS9OM2t0K3Ba?= =?utf-8?B?dXZ5ckcyRmJLVm43Y1I1ZjV3UkxtMmJPQzhNWDR3UG83d2tCMWhmc2dnZjdy?= =?utf-8?B?YWY3OVRJd094L1hMWHVqeVphUU9BNXBnekhxMFl0a1FweUdNYWtEdUt6VllQ?= =?utf-8?B?aERGYkNSaWRYUGxFK3lGWDVTaEUrZTgzMkw1K1RHcklTeXF4SVgzV0Q3cFAr?= =?utf-8?B?SHZKb2wxQUNqV2o4UC9hSXFPUmt6SGZ4V1NtYVJGSk1CNCtvSnE4bzhnTVNV?= =?utf-8?B?d2R2L3E1OGdHZytCekw5eDJjZFRuSm90dVdsci9aM0JMdjMxQ3doUVJyN3lq?= =?utf-8?B?UWNGWlg4M2xPK1FYTTRiWWF3blhXdk5BZmZRcXA1Y1FEcEcrelBGSi90YTND?= =?utf-8?B?OTNnS1ppdUlhU2NTakd3cTlYSTFDQlptUHpZV3M3aWFBcnpuZCtDT2o5VlBT?= =?utf-8?B?aXpIek1DdGN6Y3k4OW5XZjB6Yk9OUDQ1VW5qdzlJUjFmemZFZ3NtdWc2SmRi?= =?utf-8?Q?b+NbET1S8tJIl?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH7PR11MB5957.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(1800799024)(4022899009)(366016)(7053199007)(13003099007)(38070700018); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?B?bFNBVmhRenJVL0o0OVJWa2k1L0FsV1RSM0k0MUNuK3NTb1V0YURMd2w4ZU5u?= =?utf-8?B?UDJQYzNpTXc0SUZYWFhnNC9yelZOZElsRmNwT2ZHRFpKc0dNYnFyQnNPVDhi?= =?utf-8?B?L3RGOE9rckxxdXpKdS9wZWRGY2ZXM0Zhc1hWOFlYZ2FmMmVibFZDR0ZPY1Ri?= =?utf-8?B?NVFJMVBYN1NsVEVUdHFicUxFZEF5YTl1a25SNDFma0lCUWtqdTQ1YzdrK0pl?= =?utf-8?B?S3Q3aWhHWFErdXJObGZab0lkWHVEbGNxdTJLRldJTDNHTTZzV1RCTXB0L1Nu?= =?utf-8?B?aytKYkRISU9ZVVd1MllBRE9Yd05zWXNKU3BYaE9ZREJwMzdJWDN1cTk2dnVQ?= =?utf-8?B?dkJpODJVSjJyU0txVTZCQ1lsb2xOUlRnNSs4RzFwWXJlZHVqOGo3cENVanJi?= =?utf-8?B?VzRTdWNSTmthVDF2MVFKSEJiTDNLQnVRY2w4VW5waGI5cVFEcjlRRzh0S0s1?= =?utf-8?B?Ukh3UXlYeWxvUmtZS2c0cExmQ1piT2dCQjgwRG0xbXUwQ2Jrd3FnUWYzbkNW?= =?utf-8?B?eXNRS2VkUDB4YklXZlZ6TGRsUzBtZFFhYnE2RG1IOGtGYStqRzdROW54Njlm?= =?utf-8?B?bytUUW43T2duVEt4SGFiWnU0eGFtN2k4ODFSenFGajFqTFVONmVRVDU3VnU5?= =?utf-8?B?bktsUVpieGczTCtTMlo2UmZxOThBWVU1b3l6bmc0SWZCY2Y0NEpoQ0E5b2c5?= =?utf-8?B?QWhESHBKTlYwVU4waFp6NExRejRRWWZSU2JYK3ptTVcxdE5HRmRGS3BpMCta?= =?utf-8?B?WWxiK2JHeHcrM2ZRRkRNTC9FVWNFenFTR0duUFp5b3hnQUVWZmZGWGFuTkdk?= =?utf-8?B?ZHpwZWRRM2NqangvNU5Jd0RsczBIRWc3dW9yYnBaZmZtVDA3OW5XV1dFejQx?= =?utf-8?B?ZUpQQTlsV0laZlZuaFpGRWRETldzaHhsbVNYODNEcmRQTVBKK2IrdC8zaGVm?= =?utf-8?B?OFpyRTV6U2JUZjlTTERmUDVDMHNicjcrT21KWXk5ZTVIWVRlZ0FRYllGNTdw?= =?utf-8?B?UklhdGgxckJtVURSOUwrdXRGU3pSVG01ekIxMVJjdnRuK0dkWWlHUGoyYkY0?= =?utf-8?B?Vm1Rc1pJOHNnaW83RlBoVXZSem9seEY2QnN2UHQ3TzlqdXBxUEt3djFHL2Vw?= =?utf-8?B?WkU0bXhCVVFFbThka3g4N280ekNtNFByMDU4eEpPKzR5ckEya1lYYUR4RUht?= =?utf-8?B?Z241c3FWWStoVlBubk1yR042OE1RdFBncTg0eTVxOGdSeVNFbDNzdVFucDMr?= =?utf-8?B?eXZxSFpxNzBBQWFUVE1kVWFQcWljRmdqVWRRYTJDRGxkQnhFTU1FT0x4R0l4?= =?utf-8?B?VyswYXI0TXd5ako1dGpZSnQ4Y1grY3ZpbDY1cmNFWWI4YTNDSzB1ZFc5aXpx?= =?utf-8?B?WHhXY2x4SEpiendVNUZONTg2a1ByK1Q0dExuakl5R213V1pjSnQ1UEFoVXVD?= =?utf-8?B?MmRqbUhCZFdCYmpueWpnSXFNRzllUHErNUJTaEovUEZET0FSQTF2dGJ3aUU4?= =?utf-8?B?TU9XNE5peUZRQzdUT2hwbStCcklkTjY3ZXM0cVhYZHJ0YmFhREFzVXVvZUNz?= =?utf-8?B?SE8vQ1JXcy9WTVdiSUxvS054WlA4cXJjeTBpWExzNnE5Zll2SXlRTmFFM3Nr?= =?utf-8?B?WjFndkhOZnFqL1VsaEZETmtWeUlpaXhWRWlPMnJKZHNGd0hTeEdRUzI3dHhq?= =?utf-8?B?VGVNOTZnVmpQTm0wNk5UdkN2L1M5WkNWbFBuTG5taHd0TWlqZ2d6eVcvcUZO?= =?utf-8?B?cXprZkNoZ3hLQWlZdE9OMm1iWDJmaDJ4KzZDays3OThhbForRGMvRy9FNnN5?= =?utf-8?B?c2txckttbkFZMUtIRlJsRkRMc1hLRW5SaXNJbXcrUlo0S1FMYjJFRVlKeUYz?= =?utf-8?B?Sk9ueUdrazVZTzl0Z3lzdlpSY1k4WUlhYkZDMFdsaTc0U2tOREFpQkJRVWR0?= =?utf-8?B?ZWd2WGtScHJCVjliZUJuRGZPZDVENEwxVVdFYjhiRXF5TzVzYUkrYmJaVUxJ?= =?utf-8?B?TXgzUjBBT2drd3BOWTYvWUhVeWNLVVJlUTZ4ajVicFB1Z3ZCL0xHaUxIMDNN?= =?utf-8?B?c1BaVTNQSnJYMnI3eGxCblNNMXFGbXl0cGxhQ0ZxVkpJQldJMkJpNm51YVpS?= =?utf-8?Q?5HV75vyokuYBWHpF0lD6xHg49?= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PH7PR11MB5957.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 222b6370-b306-4194-fc10-08dd50044b21 X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Feb 2025 10:09:12.6531 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: FUc8ZRVgOnHPjpi9RM3Z028IIMgskoHZ2AuDH2kPfe/5GIyMTvCvbVN+iYlDA/cpdGREoX8g796H5884Q7LOpQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB6747 X-OriginatorOrg: intel.com Subject: Re: [FFmpeg-devel] [PATCH] avfilter/dnn: add zero-shot image classification using CLIP models X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: > -----Original Message----- > From: ffmpeg-devel On Behalf Of > Maximilian Kaindl > Sent: Tuesday, February 18, 2025 12:29 AM > To: FFmpeg development discussions and patches devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] avfilter/dnn: add zero-shot image > classification using CLIP models > > Hello Yejun Guo, > > yes i can do that and submit it in another patch. Do you also have some > feedback for the clip backend? I have already made some small changes (cuda > accel and new preprocessing) that i will submit along with the other patch, but > i would like to hear your thoughts. > Could you share why we need a new backend? > Thanks > > ________________________________ > From: ffmpeg-devel on behalf of Guo, > Yejun > Sent: Sunday, February 16, 2025 7:09 AM > To: FFmpeg development discussions and patches devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] avfilter/dnn: add zero-shot image > classification using CLIP models > > > > > -----Original Message----- > > From: ffmpeg-devel On Behalf Of > > m.kaindl0208@gmail.com > > Sent: Thursday, January 30, 2025 4:33 AM > > To: ffmpeg-devel@ffmpeg.org > > Subject: [FFmpeg-devel] [PATCH] avfilter/dnn: add zero-shot image > > classification using CLIP models > > > > Add a new filter 'dnn_clip' that performs zero-shot image > > classification using CLIP (Contrastive Language-Image Pre-Training) models. > The filter supports: > > For image classification with new dnn models, we'd better add the new model > support with dnn_classify at https://ffmpeg.org/ffmpeg- > filters.html#dnn_005fclassify > > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org > with subject "unsubscribe". > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org > with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".