From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.ffmpeg.org (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id 5831E4E1C4 for ; Wed, 14 Jan 2026 10:08:44 +0000 (UTC) Authentication-Results: ffbox; dkim=fail (body hash mismatch (got b'/c1zage7Wr8638DHw19+7sqwwnVNdcneiBZANPKSRvY=', expected b'3X/Cq1VgC9DzZTUu/ARecOeHfd9eSWMC5WAzhIxRHqA=')) header.d=gmail.com header.a=rsa-sha256 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ffmpeg.org; i=@ffmpeg.org; q=dns/txt; s=mail; t=1768385304; h=to : date : message-id : mime-version : reply-to : subject : list-id : list-archive : list-archive : list-help : list-owner : list-post : list-subscribe : list-unsubscribe : from : cc : content-type : content-transfer-encoding : from; bh=zHrhlcsw00PT+k0BQZ+eydZs33kEHxtFhCidPktZDdU=; b=FOK7/pvHeRWnc6CWrwTOhSNOFK0YsB4XfN/bBiFjLyQlzcWO3jFjyQRe9yWdBWHpz8uRz o2XjZTiz1Wu2Qr3e/quzmWL6oOEdhXXUumS8LDxLEd81HO9bap3zFoJ+m1Ky0wLTvZG8uPX gk9w7Rzfd9UX5qypL1e+JExHMQqrZ/gJaeM0rqmh7UdsFpDd+NgQPZSR8Qxz6rU5DcvdTxL do71XlRgJTU7wndYItI6BgvyuxJNZs2rDgdDFiRROJUO3zILWFBWBvBPReE7HC8xMd78VeC 6AT72d0oFumQUm08jsUYTsAC+PAmNQo8gLc7lLxdtOwO5eeRUearxuebBL/Q== Received: from [172.20.0.4] (unknown [172.20.0.4]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTP id 535B3690E0D; Wed, 14 Jan 2026 12:08:24 +0200 (EET) ARC-Seal: i=1; cv=none; a=rsa-sha256; d=ffmpeg.org; s=arc; t=1768385286; b=PtipPcigNXR6fuqVjtOG2y8FdAeFnVWgbS4aKTZzGa5+W6F5CmgVDcVpyfqQgxSTAwnDK E50Uo9HdIFISUD2yxZ4yQe3fwWL/5Dw1fZZpc0WSDDG+V5L9P7FN16Wdd1vC2R6kserCbVJ jp1LBZA/zkK/nSir+PzVSIHctS6OM6e1SUY2gIzSlJ6FqdpGVDwVCRGLVx2ui3ELRLIhwsT Kx9+otZLMCBiOIppn4MZE1D4HG2zCQm+Y6RQwckqQ3hnX2a7sVDaAwX5LuC+EcwX670rcnB 9x0qBOzy3j8uXe4mLzbzUWBEMT04pbAih8doVuRFBzWxbA0GaxoIhea99k8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=ffmpeg.org; s=arc; t=1768385286; h=from : sender : reply-to : subject : date : message-id : to : cc : mime-version : content-type : content-transfer-encoding : content-id : content-description : resent-date : resent-from : resent-sender : resent-to : resent-cc : resent-message-id : in-reply-to : references : list-id : list-help : list-unsubscribe : list-subscribe : list-post : list-owner : list-archive; bh=/c1zage7Wr8638DHw19+7sqwwnVNdcneiBZANPKSRvY=; b=DQKVC4JJasQzTQfFV1SASVoRtOP0nKsafyvgv6vUKGWu8c+TGoAjayA3fzG12Q2OD6UOi 8trZO3i7fP2VkhQ3faHq4Jd2pzscUAsbRFslNTCPPt6JCooByZ7BTrWZW2gW3bx+N1KtFQO TxGYOY8Y/8oS37x/HZT50UqfeX6FHzsTU2JBvyhmYJLEPCrllmFPLaiJ07hN0oa812wDwa3 laF955RUKbeUKBSFfsaLSwF5RWvx6zMLT2WNLDQ45PqGcZoe7BWUpWGb4hl+R2jsb48F+dY 5pyORn4QaUBmY/BmDeLeIbRJsofY6jfqKna7nNEwqbhVnhXBDZKlmO1t6LDQ== ARC-Authentication-Results: i=1; ffmpeg.org; dkim=pass header.d=gmail.com; arc=none; dmarc=pass header.from=gmail.com policy.dmarc=quarantine Authentication-Results: ffmpeg.org; dkim=pass header.d=gmail.com; arc=none (Message is not ARC signed); dmarc=pass (Used From Domain Record) header.from=gmail.com policy.dmarc=quarantine Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTPS id 610BB690BEB for ; Wed, 14 Jan 2026 12:07:54 +0200 (EET) Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-2a1022dda33so51468855ad.2 for ; Wed, 14 Jan 2026 02:07:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1768385272; x=1768990072; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=3X/Cq1VgC9DzZTUu/ARecOeHfd9eSWMC5WAzhIxRHqA=; b=D1b60PDyKGUfDshjdkCIjKNWLQhhwnP7bsMwU7J3zqLqIRP/fpATGD8AXBO1RIS8zN FL88mbXwgcbX7SLXL0QBX6mQ9zr+Q5UE8Xb0bdR0wEV4FmCHKo+To2OpYwdcFsmpSaah JhioeovfZ4ZKogukZuWe3GGrmMXA7Ya5uA/a26iHQ6wlunCRF52jloD9W1AEuUBpZzj+ hnJT5gRbo8OPDlRZZ4xbnCYw07sVF7mbs/Sv6iQ42P/5dNlOA4BHptmhqqUVqAwThosE 1qQCBhxVrKhJKJwzz83MSNa+xB85UBodvdiy+NKPNojXQCp1NLg2PPjxtw7a/avct6IU aMzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1768385272; x=1768990072; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=3X/Cq1VgC9DzZTUu/ARecOeHfd9eSWMC5WAzhIxRHqA=; b=defjYX5l+uCNZ1fj+saPSHEaVFJpujCV/2lKJafj136tVJ0ULHT5y1qI+aiADzRgce 0hjLHDETsuxD8GP59b8rChxqFc6m/DCFqhw4TdqArOjDZNVUTWKBfxvnMSB+tpmWpFmZ AYL0fN753SaiibjR90bSUF/ti5MOcbUNKq7z8U0BWibyHo/JnKL0bFgpZoEYWUQF+8D6 9sRc+lXolZRXeLIK+P49hvAuAMSgbrJ44G4PtfE1G8crZHKbBf88P8Vz3oowmLLrh5MY H9fuM2UOFqatEj8bKj2GJPhWHNzHgplnW5UJbyOq1m6wpjYdzC/jhHOtOuQfLNZ0j5ok Upxg== X-Gm-Message-State: AOJu0YzRWs8nr3qe9Dt4+LGo6LMjuZk95rerUks5G7xB3cCfG3eI9EX6 2oft+fur04LL8AJ7NV3UgtNXsmnZQtZAzHFTOS0gPZbDwAEHFff/vaqfqUzW4Q== X-Gm-Gg: AY/fxX6aYovLBR50f3z11cCWs32KtFBgdvuPAf6GWmf/tEQSPyn8ceyu+UdfmTZFqH7 OykbfQCSIzyAqdNpFK1tui2T1TEKcM5JolpBsS5zY5wtjSoKI9VPTS8m4SuHqvJGEENd/uhJI13 jyfPUz+IWdPBw55fVUUwh0lFzNgQvX/2giHBvVBVl/0brC/r617E6mCSBMaGSEV0h2Z4jJQAzXX gZqB0PwSzvTW275whK8fr1g8JV+1d132bFewLiIgjUnY4WQT+dPp1/Qf7/4khCXolWVc0J87O4Q 5V/hDrZ+LRLcLAwvbxfNWJ/fslIw3BM+gZEDmeAX4RcZihu3H5HsoDAITJbY4HFUKl5PfYm147C 8/IU/FZ9UpmbNqcvC8xozXEHlQLr56UZnfcjTm+4a7ioMfM95xwpQUnV5eNxwwrVoJ3lR5OhrSK 2kooRTSLq/vZIntx6qxZq1UFFxRSTpi+NoziLm0j8= X-Received: by 2002:a17:903:11c9:b0:2a3:1d78:7505 with SMTP id d9443c01a7336-2a599e5683fmr19966225ad.56.1768385272196; Wed, 14 Jan 2026 02:07:52 -0800 (PST) Received: from raja-rathour-ASUS-TUF-Gaming-A15 ([103.240.193.224]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2a3e3cd2b3asm221038305ad.88.2026.01.14.02.07.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Jan 2026 02:07:51 -0800 (PST) To: ffmpeg-devel@ffmpeg.org Date: Wed, 14 Jan 2026 15:37:45 +0530 Message-ID: <20260114100745.447204-1-imraja729@gmail.com> X-Mailer: git-send-email 2.51.0 MIME-Version: 1.0 Message-ID-Hash: OWXTWQJOJQZSAM2MU56R6UK26T7O6DOM X-Message-ID-Hash: OWXTWQJOJQZSAM2MU56R6UK26T7O6DOM X-MailFrom: SRS0=K6ea=7T=gmail.com=imraja729@ffmpeg.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; header-match-ffmpeg-devel.ffmpeg.org-0; header-match-ffmpeg-devel.ffmpeg.org-1; header-match-ffmpeg-devel.ffmpeg.org-2; header-match-ffmpeg-devel.ffmpeg.org-3; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list Reply-To: FFmpeg development discussions and patches Subject: [FFmpeg-devel] [PATCH v2] avfilter/dnn_backend_torch: enable async execution, memory safety & dynamic shapes List-Id: FFmpeg development discussions and patches Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Raja Rathour via ffmpeg-devel Cc: imraja729@gmail.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Archived-At: List-Archive: List-Post: This patch overhauls the LibTorch backend to support modern FFmpeg DNN features: 1. Async Execution: Implements non-blocking inference using ff_dnn_start_inference_async. 2. Memory Safety: Fixes a critical memory leak by introducing a persistent input buffer in THInferRequest. 3. Dynamic Shapes: Adds support for changing input resolutions by reallocating buffers on the fly. 4. Robustness: Fixes device selection crashes on parameter-less models. Signed-off-by: Raja Rathour --- libavfilter/dnn/dnn_backend_torch.cpp | 293 +++++++++----------------- 1 file changed, 101 insertions(+), 192 deletions(-) diff --git a/libavfilter/dnn/dnn_backend_torch.cpp b/libavfilter/dnn/dnn_backend_torch.cpp index 33809bf983..4e9a08c75a 100644 --- a/libavfilter/dnn/dnn_backend_torch.cpp +++ b/libavfilter/dnn/dnn_backend_torch.cpp @@ -25,10 +25,6 @@ #include #include -#include -#include -#include -#include extern "C" { #include "dnn_io_proc.h" @@ -46,16 +42,13 @@ typedef struct THModel { SafeQueue *request_queue; Queue *task_queue; Queue *lltask_queue; - SafeQueue *pending_queue; ///< requests waiting for inference - std::thread *worker_thread; ///< background worker thread - std::mutex *mutex; ///< mutex for the condition variable - std::condition_variable *cond; ///< condition variable for worker wakeup - std::atomic worker_stop; ///< signal for thread exit } THModel; typedef struct THInferRequest { torch::Tensor *output; torch::Tensor *input_tensor; + float *input_data; // Persistent buffer to prevent leaks + size_t input_data_size; // Track size for dynamic resizing } THInferRequest; typedef struct THRequestItem { @@ -104,7 +97,10 @@ static void th_free_request(THInferRequest *request) delete(request->input_tensor); request->input_tensor = NULL; } - return; + if (request->input_data) { + av_freep(&request->input_data); + request->input_data_size = 0; + } } static inline void destroy_request_item(THRequestItem **arg) @@ -124,75 +120,24 @@ static inline void destroy_request_item(THRequestItem **arg) static void dnn_free_model_th(DNNModel **model) { THModel *th_model; - if (!model || !*model) - return; - - th_model = (THModel *)(*model); - - /* 1. Stop and join the worker thread if it exists */ - if (th_model->worker_thread) { - { - std::lock_guard lock(*th_model->mutex); - th_model->worker_stop = true; - } - th_model->cond->notify_all(); - th_model->worker_thread->join(); - delete th_model->worker_thread; - th_model->worker_thread = NULL; - } - - /* 2. Safely delete C++ synchronization objects */ - if (th_model->mutex) { - delete th_model->mutex; - th_model->mutex = NULL; - } - if (th_model->cond) { - delete th_model->cond; - th_model->cond = NULL; - } - - /* 3. Clean up the pending queue */ - if (th_model->pending_queue) { - while (ff_safe_queue_size(th_model->pending_queue) > 0) { - THRequestItem *item = (THRequestItem *)ff_safe_queue_pop_front(th_model->pending_queue); - destroy_request_item(&item); - } - ff_safe_queue_destroy(th_model->pending_queue); - } - - /* 4. Clean up standard backend queues */ - if (th_model->request_queue) { + if (*model) { + th_model = (THModel *)*model; while (ff_safe_queue_size(th_model->request_queue) != 0) { THRequestItem *item = (THRequestItem *)ff_safe_queue_pop_front(th_model->request_queue); destroy_request_item(&item); - } - ff_safe_queue_destroy(th_model->request_queue); - } - - if (th_model->lltask_queue) { - while (ff_queue_size(th_model->lltask_queue) != 0) { - LastLevelTaskItem *item = (LastLevelTaskItem *)ff_queue_pop_front(th_model->lltask_queue); av_freep(&item); } - ff_queue_destroy(th_model->lltask_queue); - } - - if (th_model->task_queue) { + ff_safe_queue_destroy(th_model->request_queue); while (ff_queue_size(th_model->task_queue) != 0) { TaskItem *item = (TaskItem *)ff_queue_pop_front(th_model->task_queue); - av_frame_free(&item->in_frame); - av_frame_free(&item->out_frame); av_freep(&item); } ff_queue_destroy(th_model->task_queue); + if (th_model->jit_model) + delete th_model->jit_model; + av_freep(&th_model); + *model = NULL; } - - /* 5. Final model cleanup */ - if (th_model->jit_model) - delete th_model->jit_model; - - av_freep(&th_model); - *model = NULL; } static int get_input_th(DNNModel *model, DNNData *input, const char *input_name) @@ -207,131 +152,133 @@ static int get_input_th(DNNModel *model, DNNData *input, const char *input_name) return 0; } -static void deleter(void *arg) -{ - av_freep(&arg); -} - static int fill_model_input_th(THModel *th_model, THRequestItem *request) { - LastLevelTaskItem *lltask = NULL; - TaskItem *task = NULL; - THInferRequest *infer_request = NULL; DNNData input = { 0 }; - DnnContext *ctx = th_model->ctx; + THInferRequest *infer_request = NULL; + TaskItem *task = NULL; + LastLevelTaskItem *lltask = NULL; int ret, width_idx, height_idx, channel_idx; + size_t cur_size; lltask = (LastLevelTaskItem *)ff_queue_pop_front(th_model->lltask_queue); if (!lltask) { - ret = AVERROR(EINVAL); - goto err; + return AVERROR(EINVAL); } request->lltask = lltask; task = lltask->task; infer_request = request->infer_request; ret = get_input_th(&th_model->model, &input, NULL); - if ( ret != 0) { - goto err; + if (ret != 0) { + return ret; } width_idx = dnn_get_width_idx_by_layout(input.layout); height_idx = dnn_get_height_idx_by_layout(input.layout); channel_idx = dnn_get_channel_idx_by_layout(input.layout); input.dims[height_idx] = task->in_frame->height; input.dims[width_idx] = task->in_frame->width; - input.data = av_malloc(input.dims[height_idx] * input.dims[width_idx] * - input.dims[channel_idx] * sizeof(float)); - if (!input.data) - return AVERROR(ENOMEM); - infer_request->input_tensor = new torch::Tensor(); - infer_request->output = new torch::Tensor(); + + cur_size = input.dims[height_idx] * input.dims[width_idx] * + input.dims[channel_idx] * sizeof(float); + + /* FIX: Reuse memory instead of allocating every time (Fixes Leak) */ + if (!infer_request->input_data || infer_request->input_data_size < cur_size) { + av_freep(&infer_request->input_data); + infer_request->input_data = (float *)av_malloc(cur_size); + if (!infer_request->input_data) + return AVERROR(ENOMEM); + infer_request->input_data_size = cur_size; + } + + input.data = infer_request->input_data; + + if (!infer_request->input_tensor) + infer_request->input_tensor = new torch::Tensor(); + if (!infer_request->output) + infer_request->output = new torch::Tensor(); switch (th_model->model.func_type) { case DFT_PROCESS_FRAME: - input.scale = 255; if (task->do_ioproc) { - if (th_model->model.frame_pre_proc != NULL) { - th_model->model.frame_pre_proc(task->in_frame, &input, th_model->model.filter_ctx); + if (th_model->model.frame_to_dnn_data) { + ret = th_model->model.frame_to_dnn_data(task->in_frame, &input, th_model->model.model_data); } else { - ff_proc_from_frame_to_dnn(task->in_frame, &input, ctx); + ret = ff_dnn_frame_to_dnn_data(task->in_frame, &input, th_model->ctx); } } + if (ret != 0) { + return ret; + } break; default: - avpriv_report_missing_feature(NULL, "model function type %d", th_model->model.func_type); + avpriv_report_missing_feature(th_model->ctx, "model function type %d", th_model->model.func_type); break; } *infer_request->input_tensor = torch::from_blob(input.data, {1, input.dims[channel_idx], input.dims[height_idx], input.dims[width_idx]}, - deleter, torch::kFloat32); - return 0; + nullptr, torch::kFloat32); -err: - th_free_request(infer_request); - return ret; + return 0; } static int th_start_inference(void *args) { THRequestItem *request = (THRequestItem *)args; - THInferRequest *infer_request = NULL; - LastLevelTaskItem *lltask = NULL; - TaskItem *task = NULL; - THModel *th_model = NULL; - DnnContext *ctx = NULL; + THInferRequest *infer_request = request->infer_request; + LastLevelTaskItem *lltask = request->lltask; + TaskItem *task = lltask->task; + THModel *th_model = (THModel *)task->model; + DnnContext *ctx = th_model->ctx; std::vector inputs; - torch::NoGradGuard no_grad; - - if (!request) { - av_log(NULL, AV_LOG_ERROR, "THRequestItem is NULL\n"); - return AVERROR(EINVAL); - } - infer_request = request->infer_request; - lltask = request->lltask; - task = lltask->task; - th_model = (THModel *)task->model; - ctx = th_model->ctx; - if (ctx->torch_option.optimize) - torch::jit::setGraphExecutorOptimize(true); - else - torch::jit::setGraphExecutorOptimize(false); + torch::jit::setGraphExecutorOptimize(!!ctx->torch_option.optimize); if (!infer_request->input_tensor || !infer_request->output) { av_log(ctx, AV_LOG_ERROR, "input or output tensor is NULL\n"); return DNN_GENERIC_ERROR; } - // Transfer tensor to the same device as model - c10::Device device = (*th_model->jit_model->parameters().begin()).device(); + + /* FIX: Use the context device directly instead of querying model parameters */ + const char *device_name = ctx->device ? ctx->device : "cpu"; + c10::Device device = c10::Device(device_name); + if (infer_request->input_tensor->device() != device) *infer_request->input_tensor = infer_request->input_tensor->to(device); + inputs.push_back(*infer_request->input_tensor); - - *infer_request->output = th_model->jit_model->forward(inputs).toTensor(); + + try { + *infer_request->output = th_model->jit_model->forward(inputs).toTensor(); + } catch (const c10::Error& e) { + av_log(ctx, AV_LOG_ERROR, "Torch forward pass failed: %s\n", e.what()); + return DNN_GENERIC_ERROR; + } return 0; } static void infer_completion_callback(void *args) { - THRequestItem *request = (THRequestItem*)args; + THRequestItem *request = (THRequestItem *)args; LastLevelTaskItem *lltask = request->lltask; TaskItem *task = lltask->task; - DNNData outputs = { 0 }; + DNNData *outputs = &task->output; THInferRequest *infer_request = request->infer_request; THModel *th_model = (THModel *)task->model; torch::Tensor *output = infer_request->output; c10::IntArrayRef sizes = output->sizes(); - outputs.order = DCO_RGB; - outputs.layout = DL_NCHW; - outputs.dt = DNN_FLOAT; + outputs->order = DCO_RGB; + outputs->layout = DL_NCHW; + outputs->dt = DNN_FLOAT; + if (sizes.size() == 4) { // 4 dimensions: [batch_size, channel, height, width] // this format of data is normally used for video frame SR - outputs.dims[0] = sizes.at(0); // N - outputs.dims[1] = sizes.at(1); // C - outputs.dims[2] = sizes.at(2); // H - outputs.dims[3] = sizes.at(3); // W + outputs->dims[0] = sizes.at(0); // N + outputs->dims[1] = sizes.at(1); // C + outputs->dims[2] = sizes.at(2); // H + outputs->dims[3] = sizes.at(3); // W } else { avpriv_report_missing_feature(th_model->ctx, "Support of this kind of model"); goto err; @@ -343,70 +290,40 @@ static void infer_completion_callback(void *args) { // Post process can only deal with CPU memory. if (output->device() != torch::kCPU) *output = output->to(torch::kCPU); - outputs.scale = 255; - outputs.data = output->data_ptr(); - if (th_model->model.frame_post_proc != NULL) { - th_model->model.frame_post_proc(task->out_frame, &outputs, th_model->model.filter_ctx); + outputs->scale = 255; + outputs->data = output->data_ptr(); + if (th_model->model.dnn_data_to_frame) { + th_model->model.dnn_data_to_frame(task->out_frame, outputs, th_model->model.model_data); } else { - ff_proc_from_dnn_to_frame(task->out_frame, &outputs, th_model->ctx); + ff_dnn_dnn_data_to_frame(task->out_frame, outputs, th_model->ctx); } - } else { - task->out_frame->width = outputs.dims[dnn_get_width_idx_by_layout(outputs.layout)]; - task->out_frame->height = outputs.dims[dnn_get_height_idx_by_layout(outputs.layout)]; } break; default: avpriv_report_missing_feature(th_model->ctx, "model function type %d", th_model->model.func_type); goto err; } + task->inference_done++; av_freep(&request->lltask); + err: th_free_request(infer_request); if (ff_safe_queue_push_back(th_model->request_queue, request) < 0) { destroy_request_item(&request); - av_log(th_model->ctx, AV_LOG_ERROR, "Unable to push back request_queue when failed to start inference.\n"); - } -} - -static void th_worker_thread(THModel *th_model) { - while (true) { - THRequestItem *request = NULL; - { - std::unique_lock lock(*th_model->mutex); - th_model->cond->wait(lock, [&]{ - return th_model->worker_stop || ff_safe_queue_size(th_model->pending_queue) > 0; - }); - - if (th_model->worker_stop && ff_safe_queue_size(th_model->pending_queue) == 0) - break; - - request = (THRequestItem *)ff_safe_queue_pop_front(th_model->pending_queue); - } - - if (request) { - th_start_inference(request); - infer_completion_callback(request); - } } } static int execute_model_th(THRequestItem *request, Queue *lltask_queue) { - THModel *th_model = NULL; + THModel *th_model; LastLevelTaskItem *lltask; - TaskItem *task = NULL; + TaskItem *task; int ret = 0; - if (ff_queue_size(lltask_queue) == 0) { - destroy_request_item(&request); - return 0; - } - lltask = (LastLevelTaskItem *)ff_queue_peek_front(lltask_queue); if (lltask == NULL) { - av_log(NULL, AV_LOG_ERROR, "Failed to get LastLevelTaskItem\n"); ret = AVERROR(EINVAL); goto err; } @@ -414,20 +331,25 @@ static int execute_model_th(THRequestItem *request, Queue *lltask_queue) th_model = (THModel *)task->model; ret = fill_model_input_th(th_model, request); - if ( ret != 0) { + if (ret != 0) { goto err; } + + /* FIX: Use modern FFmpeg Async Infrastructure */ if (task->async) { - std::lock_guard lock(*th_model->mutex); - if (ff_safe_queue_push_back(th_model->pending_queue, request) < 0) { - return AVERROR(ENOMEM); - } - th_model->cond->notify_one(); + ret = ff_dnn_start_inference_async(th_model->ctx, &request->exec_module); + if (ret < 0) + goto err; return 0; + } else { + ret = th_start_inference((void *)(request)); + if (ret != 0) + goto err; + infer_completion_callback(request); + return (task->inference_done == task->inference_todo) ? 0 : DNN_GENERIC_ERROR; } err: - th_free_request(request->infer_request); if (ff_safe_queue_push_back(th_model->request_queue, request) < 0) { destroy_request_item(&request); } @@ -479,12 +401,9 @@ err: static THInferRequest *th_create_inference_request(void) { - THInferRequest *request = (THInferRequest *)av_malloc(sizeof(THInferRequest)); - if (!request) { + THInferRequest *request = (THInferRequest *)av_mallocz(sizeof(THInferRequest)); + if (!request) return NULL; - } - request->input_tensor = NULL; - request->output = NULL; return request; } @@ -556,16 +475,6 @@ static DNNModel *dnn_load_model_th(DnnContext *ctx, DNNFunctionType func_type, A goto fail; } - th_model->pending_queue = ff_safe_queue_create(); - if (!th_model->pending_queue) { - goto fail; - } - - th_model->mutex = new std::mutex(); - th_model->cond = new std::condition_variable(); - th_model->worker_stop = false; - th_model->worker_thread = new std::thread(th_worker_thread, th_model); - model->get_input = &get_input_th; model->get_output = &get_output_th; model->filter_ctx = filter_ctx; @@ -662,4 +571,4 @@ extern const DNNModule ff_dnn_backend_torch = { .get_result = dnn_get_result_th, .flush = dnn_flush_th, .free_model = dnn_free_model_th, -}; +}; \ No newline at end of file -- 2.51.0 _______________________________________________ ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org