From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.ffmpeg.org (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id E5AFB4E834 for ; Wed, 4 Feb 2026 13:01:56 +0000 (UTC) Authentication-Results: ffbox; dkim=fail (body hash mismatch (got b'hZTH7kH3y+leyLf/KTxSHg+siTkYP1g54PXWWZSsDLE=', expected b'DHsP3QJaEHxUPClWZR+rlR6ygylaHnRND8nvRJ2u34A=')) header.d=gmail.com header.a=rsa-sha256 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ffmpeg.org; i=@ffmpeg.org; q=dns/txt; s=mail; t=1770210027; h=to : date : message-id : mime-version : reply-to : subject : list-id : list-archive : list-archive : list-help : list-owner : list-post : list-subscribe : list-unsubscribe : from : cc : content-type : content-transfer-encoding : from; bh=GpCNjo9qKbC2zVWnmF41kUnTZBAWAM/qvahST67hkUs=; b=RQj9k9RMpKkPO4oxPiELUQj2GDtW/pSpHS+HmG8lKtUMRj5oOl5fhdxk8IByXX0grMBEr 3E6vaGAZKsMAs7hHmMsF6IKFFicVt5wAqlQkGhcPJrOqPCzUPv8t5jLdpf1g7NhaZnYRYzq misQ0qImYAdeSJvGyadpi8BzEOS2NmG6s35PvwU9GUrAooogZ9rPV4dpP4uNuiFXyfNB+Ri iKj4YypvWzGbNSKrkqUE4qCr/POnV22PujRKlo91ZhxRMy5X16kHRyfLS3V/XU5N86UHFNP DGY2WEH3XtertoZE/1NLD8243k5U0LyStRm3bV2k9LyIeXpNgM6CMIF7KwhA== Received: from [172.20.0.4] (unknown [172.20.0.4]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTP id C1EBF691106; Wed, 4 Feb 2026 15:00:27 +0200 (EET) ARC-Seal: i=1; cv=none; a=rsa-sha256; d=ffmpeg.org; s=arc; t=1770210007; b=rYkdNmDwPf/2AzNitG9t3rpNd0LB6tkbz7nxgrkYHUE4Y1/TKW8vbGiPQcH6YvUShddC2 Qz83cVJ0FMrh4uaXaG/QT+P7rKTLZMTh/t2VnsFTe2Ht094M3EW9Lallua236WvaGaCGKyn 6eWIskpGUDrBUpKy8AfcuxAwyoDU+15W7gE7IiCi69Eem+bFIDm3koFf1LLawTXEC6tAYfw lCF7ULtzwbaUZTCnqRYuvMwC2SI30woC1IN/CQ049ujsgcOCqLGOtykw0XoTVr7bl8y+SLC XGKAfOrHLmd39oJxBy7vEjyZx6lP8ixM/boMT2z/9D21rUo59H9ll40C//UQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=ffmpeg.org; s=arc; t=1770210007; h=from : sender : reply-to : subject : date : message-id : to : cc : mime-version : content-type : content-transfer-encoding : content-id : content-description : resent-date : resent-from : resent-sender : resent-to : resent-cc : resent-message-id : in-reply-to : references : list-id : list-help : list-unsubscribe : list-subscribe : list-post : list-owner : list-archive; bh=hZTH7kH3y+leyLf/KTxSHg+siTkYP1g54PXWWZSsDLE=; b=l7vpUTW34fNFORMu21dtz/K00s6oLgCplQfIJzlPnWYOJ1+jQivctNFTlHncJijs7vibj e64TM+1My3R2hM8C6gG3lgp22SMSb4gUHG7NVkfIPnkrkrf82mhRnWxQ/RHly94GcVDV+nP huCnjuqj4GT+VY64VPOaRt0Xpj0vpktkGImeig8JzOiLkf9iDNGAWeZd+MUBuRbanD68tJm lW2O8vWlb1mRAI8VDvCzCCvARWzCPFEESUqwP8YHlr6BhLWA7FzTVXlgDb3kY6rtmcKoAus oCyX56Ws3lMGRF6/PGP4OWMebLPGMQ/whYliRcghTi3qpBse9celT90Sm5TQ== ARC-Authentication-Results: i=1; ffmpeg.org; dkim=pass header.d=gmail.com; arc=none; dmarc=pass header.from=gmail.com policy.dmarc=quarantine Authentication-Results: ffmpeg.org; dkim=pass header.d=gmail.com; arc=none (Message is not ARC signed); dmarc=pass (Used From Domain Record) header.from=gmail.com policy.dmarc=quarantine Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTPS id C623C690FB6 for ; Wed, 4 Feb 2026 14:59:52 +0200 (EET) Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-2a7bced39cfso71526455ad.1 for ; Wed, 04 Feb 2026 04:59:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1770209990; x=1770814790; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=DHsP3QJaEHxUPClWZR+rlR6ygylaHnRND8nvRJ2u34A=; b=avtWxwG63NTx0wb32qmnDizpw7ylDS2MVJ+cEs3bf22Z0ECNxLj2Gljc7Z9HCE4g0P B1u0vupcS/pbVGuEqMerGrtfBgzIKbWMSi2ZIt8hs5wxGYpVZ5u7b89o1WbCh94bT2P/ XMImDvV8kPinifaHQk3igCydrk2nJrwUXpiC8jqJcpT4oBNwCNO7WbfRSOqrFFDp2xb3 rAYc7Ft8i6lc7ir2mF841KFdO61pMSrPBXH2j3FRcSgNcfG8K3cSOD0apTGL8jdUV5v3 1eTuWdbrY2BynrM/VrxXeeEiY26VjuH+btqit884xZ2T5dIneFvuXoosEnIKCJ5uRVc3 omOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1770209990; x=1770814790; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=DHsP3QJaEHxUPClWZR+rlR6ygylaHnRND8nvRJ2u34A=; b=kX0k7CNwWnNzgkia9fu5F7MsXnoxpOznE/r/QyeluzC9b0jQLb/W8cvYLwcQYMHtIf DMvC8xeK0078A+2M5FWFE4F7Sxsn3FoUU6F65DJAv91ZxAHtEv6mG9yr/MWzF9ygJnOR aW1AyyW2QBEKRv0ohmMDukTw2XUp4YayUgesDBr8NUQEPdF42u1Xvx5pJIljLkP9DjOz ek6UqnBiiaPEQgCD7gBR4v3reP5IAP9jYw9MiXHp9UF+AaEcQsguCUZVxDqJhfO0s90O E1i1ipwEjVbW8yvhTF66vUGnQa21cJdwxE/eeaNJWNf/WDsvrJ7QMxdoSALnphwW9V6O c2OQ== X-Gm-Message-State: AOJu0YxrARR3McSRbMNyvVx1jZZKBNXhPkZ3yQl9fF4BVNNQWlL2Tqfb oLiYyPKk9a+98AOVrd7YfQzYt80Xbks5BwPNqyAldGmPf68EF5gffm7eMbOSgQ== X-Gm-Gg: AZuq6aKCnzl08RufDXA2v110KGbJCU8OINDsM83SHc5t0rib63oT43vGDz1D0BkDW+x oTrdtkTTsc/GLqrGskUrELyigfCTRg1ByslTTPPFvwDyVD8ayOgE1ACfdNVMmfLw5ZdGcQ9KHjg 4H0c6QVgxGo4BuhLYPaaAzRT3rRMc6Re74YjPG7wlxmvqcrP8Ij2CaDVM3wyDFK1qJ135+xpmcH oWEu3i0BCzvdlZx/gzvR2oTwvT5jyQ+x7Hgid3hYMD+HGeSN0yCkTVuDs4dm5UwM/Ol9sAjZNuw OBMejC6iG2vsYRNG8vfcKT6o+sifJf5fplAtiA/zzfAb42hTIDw+VxajNQLWxzavZBRMpDFSJ4c yZrwDJ5Vn4sHesX7c1zIYAEyx6/f98iO2hR0SUu1oUNJJeAi1bCQjgHPItIIIKgaaLL/i78QA/Q jwz5aBXALE0AncDXscMxJ4I8rKlh4/gsJ09ISaWvK5mg== X-Received: by 2002:a17:903:2ec6:b0:295:596f:8507 with SMTP id d9443c01a7336-2a933ad2d25mr31489015ad.0.1770209990147; Wed, 04 Feb 2026 04:59:50 -0800 (PST) Received: from raja-rathour-ASUS-TUF-Gaming-A15 ([2409:40d0:10c8:e57b:8ea7:d9fb:7c87:caae]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2a93394e5d4sm21637515ad.56.2026.02.04.04.59.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Feb 2026 04:59:49 -0800 (PST) To: ffmpeg-devel@ffmpeg.org Date: Wed, 4 Feb 2026 18:26:57 +0530 Message-ID: <20260204125940.24503-1-imraja729@gmail.com> X-Mailer: git-send-email 2.51.0 MIME-Version: 1.0 Message-ID-Hash: AGILCZMWQCSCGOKHEAGCBBDZ6JAENKIE X-Message-ID-Hash: AGILCZMWQCSCGOKHEAGCBBDZ6JAENKIE X-MailFrom: SRS0=bbLt=AI=gmail.com=imraja729@ffmpeg.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; header-match-ffmpeg-devel.ffmpeg.org-0; header-match-ffmpeg-devel.ffmpeg.org-1; header-match-ffmpeg-devel.ffmpeg.org-2; header-match-ffmpeg-devel.ffmpeg.org-3; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list Reply-To: FFmpeg development discussions and patches Subject: [FFmpeg-devel] [PATCH] avfilter/dnn: add LibTorch backend for DNN processing List-Id: FFmpeg development discussions and patches Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Raja Rathour via ffmpeg-devel Cc: Raja Rathour Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Archived-At: List-Archive: List-Post: This patch implements a new LibTorch backend for the DNN filter framework in FFmpeg. Key features: Support for asynchronous inference using the DNNAsyncExecModule. C++ exception handling for robust model loading and execution. NCHW tensor mapping for efficient data flow. Note: Explicit pointer casts for av_mallocz are included as required for C++ compilation within the FFmpeg build system. Signed-off-by: Raja Rathour --- Changelog | 2 ++ libavfilter/dnn/dnn_backend_torch.cpp | 31 +++++++++++++-------------- 2 files changed, 17 insertions(+), 16 deletions(-) diff --git a/Changelog b/Changelog index a9d68b369e..56efc3036f 100644 --- a/Changelog +++ b/Changelog @@ -2256,3 +2256,5 @@ version 0.3.1: added avi/divx support version 0.3: initial public release + +- LibTorch backend for DNN filters \ No newline at end of file diff --git a/libavfilter/dnn/dnn_backend_torch.cpp b/libavfilter/dnn/dnn_backend_torch.cpp index 33809bf983..47a3cba042 100644 --- a/libavfilter/dnn/dnn_backend_torch.cpp +++ b/libavfilter/dnn/dnn_backend_torch.cpp @@ -515,10 +515,13 @@ static DNNModel *dnn_load_model_th(DnnContext *ctx, DNNFunctionType func_type, A try { th_model->jit_model = new torch::jit::Module; - (*th_model->jit_model) = torch::jit::load(ctx->model_filename); - th_model->jit_model->to(device); + *th_model->jit_model = torch::jit::load(ctx->model_filename, device); + av_log(ctx, AV_LOG_VERBOSE, "LibTorch model loaded on device: %s\n", device_name); } catch (const c10::Error& e) { - av_log(ctx, AV_LOG_ERROR, "Failed to load torch model\n"); + av_log(ctx, AV_LOG_ERROR, "LibTorch error loading model '%s': %s\n", + ctx->model_filename, e.what()); + delete th_model->jit_model; + th_model->jit_model = NULL; goto fail; } @@ -591,45 +594,42 @@ static int dnn_execute_model_th(const DNNModel *model, DNNExecBaseParams *exec_p ret = ff_check_exec_params(ctx, DNN_TH, model->func_type, exec_params); if (ret != 0) { - av_log(ctx, AV_LOG_ERROR, "exec parameter checking fail.\n"); + av_log(ctx, AV_LOG_ERROR, "Exec parameter checking failed.\n"); return ret; } - task = (TaskItem *)av_malloc(sizeof(TaskItem)); + task = (TaskItem *)av_mallocz(sizeof(*task)); if (!task) { - av_log(ctx, AV_LOG_ERROR, "unable to alloc memory for task item.\n"); + av_log(ctx, AV_LOG_ERROR, "Unable to allocate memory for task item.\n"); return AVERROR(ENOMEM); } ret = ff_dnn_fill_task(task, exec_params, th_model, 0, 1); if (ret != 0) { av_freep(&task); - av_log(ctx, AV_LOG_ERROR, "unable to fill task.\n"); + av_log(ctx, AV_LOG_ERROR, "Unable to fill task.\n"); return ret; } - ret = ff_queue_push_back(th_model->task_queue, task); - if (ret < 0) { + if (ff_queue_push_back(th_model->task_queue, task) < 0) { av_freep(&task); - av_log(ctx, AV_LOG_ERROR, "unable to push back task_queue.\n"); - return ret; + av_log(ctx, AV_LOG_ERROR, "Unable to push back task_queue.\n"); + return AVERROR(ENOMEM); } ret = extract_lltask_from_task(task, th_model->lltask_queue); if (ret != 0) { - av_log(ctx, AV_LOG_ERROR, "unable to extract last level task from task.\n"); return ret; } request = (THRequestItem *)ff_safe_queue_pop_front(th_model->request_queue); if (!request) { - av_log(ctx, AV_LOG_ERROR, "unable to get infer request.\n"); - return AVERROR(EINVAL); + av_log(ctx, AV_LOG_ERROR, "No inference request available in the queue.\n"); + return AVERROR(EAGAIN); } return execute_model_th(request, th_model->lltask_queue); } - static DNNAsyncStatusType dnn_get_result_th(const DNNModel *model, AVFrame **in, AVFrame **out) { THModel *th_model = (THModel *)model; @@ -642,7 +642,6 @@ static int dnn_flush_th(const DNNModel *model) THRequestItem *request; if (ff_queue_size(th_model->lltask_queue) == 0) - // no pending task need to flush return 0; request = (THRequestItem *)ff_safe_queue_pop_front(th_model->request_queue); -- 2.51.0 _______________________________________________ ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org