From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id B73014E2DE for ; Mon, 10 Mar 2025 19:54:25 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9361A68DFE9; Mon, 10 Mar 2025 21:54:19 +0200 (EET) Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3641568DFE9 for ; Mon, 10 Mar 2025 21:54:13 +0200 (EET) Received: by mail-wr1-f48.google.com with SMTP id ffacd0b85a97d-391211ea598so2629326f8f.1 for ; Mon, 10 Mar 2025 12:54:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1741636452; x=1742241252; darn=ffmpeg.org; h=thread-index:content-language:content-transfer-encoding :mime-version:message-id:date:subject:to:from:from:to:cc:subject :date:message-id:reply-to; bh=0bG8tbAJfQnY7IrNdwI2mSQ8CJQbRYvB0POFbs0gcE4=; b=KK1xRSsXPnn/hzC0BhhnxaYVaMqN74iV7LsF168KLM5w37NlwRyhvXM+G31+5rQ56f ZsRQ0LHWdAupDBW34v4PmAhQhmyqTLJ37Fp8OtvmlmcE89Ki80cmQ2iFqPVcbvtw2IWd TyeyN88V2vmLMgqT5nav3xPKlvmS2mjtBWEbnLAyYIXPh/o5B2SvmtT0Ux9iCH2H5K1M /klWBYn1yv1oGSdQ26/1/gY9EcCm/Ge6LXifccr/yulhSCE7hb1BCWhwwAsaOCSUvKZF zmey60lWyHAYntikiUBvDxgPBlWixK/XQ+bO/zcasHZepR5Fu8MENJztwR0N5LtlVx+B JxsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741636452; x=1742241252; h=thread-index:content-language:content-transfer-encoding :mime-version:message-id:date:subject:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=0bG8tbAJfQnY7IrNdwI2mSQ8CJQbRYvB0POFbs0gcE4=; b=TcooY5ui9khflTDubQGU0sg6u4sgiawHEk7ndVbEYEscZjcKn7IJiJ1Vht6Ph9F3La c5XYRlo+ZTIeit3XjfqfAbyZmMYQVWz/hDmDGPZ9AzqyOmzYTl+FeTGUVQOA/TEhDg+f 7y9cY3eTs/5vYbtFlH+VUdTyRiMKJn0kjTdjGKZs7b9Meo1wra5jGghkwjoKHU/V8VUW BdM5y2N/keuqtTRMnuwbF2jSNb2NUAp79XiuBZYP46q0sXm7+1WWIZUrMMwIZmzy1iB7 cWHy4DFjMhDYcgPUyYgpAEkc/vjoaUAwI9FV1BbqwEi/GMZ45XCDLqsx/MVHehRElMtS eoWA== X-Gm-Message-State: AOJu0YzNQw0QeLTcOO+WpRmSeGx/xox+vBuaqGG6cCyJXgbieVWYcyjm S8FFhyvrnGyjAC3/wLnJt/GSwrHcMLjXrcolVd7KP1lSiuINZzcEDA8wKw== X-Gm-Gg: ASbGncsfPwXv0+8dfhnRSi4iYf5wf4yDvpibHlcpflSxG1RdOtGfRm7gWa78M9D6V8l Qj9Vrno81k/N+cUsC7O81m9cYlsOIiDX0IZmfQYOWb8efARMr7FKd/NK8HYnOcLR+eL2WxPGt0r CyZ2u+6DfWX72SGypyU4fQ/jzzdXFVYKNIy7rKGyJPcfdxRUc6zQCezIR8KjD2+QiljhmGU1PTr L/iHXNgWG6hLIr33BIcuSjE2Mg54Nvac3X0952Bt1SKsvZ5i+MvSQSO8ea62Gw8muB8x1/tlm8P 0DjGhytv+qb0gbWpOZ1PhzhTQPZYch6CZwM5e9v+SNcUrw6rVH2pUdyZoNxCC5PBFol+qkBUF33 KiBbrPU9fOZfXQd7C X-Google-Smtp-Source: AGHT+IFt/AAA48Y3NAkZCXpDVuIasmuhuRN0DyqkMaMyzn2M6GuZq8PgFid4EkOirD4N6JIqiThSig== X-Received: by 2002:a5d:6d04:0:b0:391:3173:4fa2 with SMTP id ffacd0b85a97d-39132d201bfmr9229366f8f.5.1741636452250; Mon, 10 Mar 2025 12:54:12 -0700 (PDT) Received: from MK2 (80-108-16-220.cable.dynamic.surfer.at. [80.108.16.220]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3912c019557sm16318913f8f.50.2025.03.10.12.54.11 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 10 Mar 2025 12:54:11 -0700 (PDT) From: To: Date: Mon, 10 Mar 2025 20:54:11 +0100 Message-ID: <003e01db91f6$31ed28d0$95c77a70$@gmail.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 16.0 Content-Language: en-at Thread-Index: AduR9iIW3y7LqDBnQXGlhXmnRbMbHA== Subject: [FFmpeg-devel] [PATCH v2 FFmpeg 10/20] libavfilter/dnn_filter_common: add support for new loading function with ff_dnn_init_with_tokenizer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Signed-off-by: MaximilianKaindl --- libavfilter/dnn_filter_common.c | 33 ++++++++++++++++++++++++++++++++- libavfilter/dnn_filter_common.h | 1 + 2 files changed, 33 insertions(+), 1 deletion(-) diff --git a/libavfilter/dnn_filter_common.c b/libavfilter/dnn_filter_common.c index c4ad000409..899e869cca 100644 --- a/libavfilter/dnn_filter_common.c +++ b/libavfilter/dnn_filter_common.c @@ -75,7 +75,7 @@ void *ff_dnn_filter_child_next(void *obj, void *prev) return ff_dnn_child_next(&base->dnnctx, prev); } -int ff_dnn_init(DnnContext *ctx, DNNFunctionType func_type, AVFilterContext *filter_ctx) +static int ff_dnn_init_priv(DnnContext *ctx, DNNFunctionType func_type, AVFilterContext *filter_ctx) { DNNBackendType backend = ctx->backend_type; @@ -91,6 +91,14 @@ int ff_dnn_init(DnnContext *ctx, DNNFunctionType func_type, AVFilterContext *fil if (ctx->model_outputnames) av_log(filter_ctx, AV_LOG_WARNING, "LibTorch backend do not require outputname(s), "\ "all outputname(s) will be ignored.\n"); + +#if (CONFIG_LIBTOKENIZERS == 0) + if ((func_type == DFT_ANALYTICS_CLIP || func_type == DFT_ANALYTICS_CLAP)) { + av_log(ctx, AV_LOG_ERROR, + "tokenizers-cpp is not included. CLIP/CLAP Classification requires tokenizers-cpp library. Include it with configure.\n"); + return AVERROR(EINVAL); + } +#endif ctx->nb_outputs = 1; } else if (backend == DNN_TF) { if (!ctx->model_inputname) { @@ -131,13 +139,36 @@ int ff_dnn_init(DnnContext *ctx, DNNFunctionType func_type, AVFilterContext *fil } } } + return 0; +} +int ff_dnn_init(DnnContext *ctx, DNNFunctionType func_type, AVFilterContext *filter_ctx) +{ + int ret = ff_dnn_init_priv(ctx, func_type, filter_ctx); + if (ret < 0) { + return ret; + } ctx->model = (ctx->dnn_module->load_model)(ctx, func_type, filter_ctx); if (!ctx->model) { av_log(filter_ctx, AV_LOG_ERROR, "could not load DNN model\n"); return AVERROR(EINVAL); } + return 0; +} +int ff_dnn_init_with_tokenizer(DnnContext *ctx, DNNFunctionType func_type, char **labels, int label_count, + int *softmax_units, int softmax_units_count, char *tokenizer_path, AVFilterContext *filter_ctx) +{ + int ret = ff_dnn_init_priv(ctx, func_type, filter_ctx); + if (ret < 0) { + return ret; + } + ctx->model = (ctx->dnn_module->load_model_with_tokenizer)(ctx, func_type, labels, label_count, softmax_units, + softmax_units_count, tokenizer_path, filter_ctx); + if (!ctx->model) { + av_log(filter_ctx, AV_LOG_ERROR, "could not load DNN model\n"); + return AVERROR(EINVAL); + } return 0; } diff --git a/libavfilter/dnn_filter_common.h b/libavfilter/dnn_filter_common.h index fffa676a9e..301441a37a 100644 --- a/libavfilter/dnn_filter_common.h +++ b/libavfilter/dnn_filter_common.h @@ -55,6 +55,7 @@ void *ff_dnn_filter_child_next(void *obj, void *prev); int ff_dnn_filter_init_child_class(AVFilterContext *filter); int ff_dnn_init(DnnContext *ctx, DNNFunctionType func_type, AVFilterContext *filter_ctx); +int ff_dnn_init_with_tokenizer(DnnContext *ctx, DNNFunctionType func_type, char** labels, int label_count, int* softmax_units, int softmax_units_count, char* tokenizer_path, AVFilterContext *filter_ctx); int ff_dnn_set_frame_proc(DnnContext *ctx, FramePrePostProc pre_proc, FramePrePostProc post_proc); int ff_dnn_set_detect_post_proc(DnnContext *ctx, DetectPostProc post_proc); int ff_dnn_set_classify_post_proc(DnnContext *ctx, ClassifyPostProc post_proc); -- 2.34.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".