From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 50D8848622 for ; Tue, 12 Dec 2023 02:34:03 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7C98368D0F9; Tue, 12 Dec 2023 04:34:00 +0200 (EET) Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.126]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 28F1168D0F9 for ; Tue, 12 Dec 2023 04:33:52 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1702348438; x=1733884438; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=sAO5zK+LdKAT7gYfnXZCujXZe50HbDYU1O1ccgvZi0s=; b=ebbw61AcFI17gjMmvnY0W5yUV4SsnwxbwcYGfBr6DGlsIMAQDyhBNpVN t2dkyuA59iAI2tVxFlaiBr414ad1L5QZ8UwoDB9dsKul1Y9Cwk5IVN7/7 FyYWUZSmmQWL9mcRtpt/ogrUTBhE9vSVwTJ8gcGVnc3Xymx85jHnE3C0T 3QYsbaZx7XmHacPKBlr/XXr35+tRiE2QsikAsJYXXgnAmbtdqkrOlzcAR jPnAj2xBRQ3CgcwFfikQNm2Uxy4h53YVSWiMxynxArenLsQs2/mJW4+Qx TpWXgjPlSKAEp5kPABFD3uKWvoCWHQImngylukEpV/10FuneIw4uXmJ7d w==; X-IronPort-AV: E=McAfee;i="6600,9927,10921"; a="379738942" X-IronPort-AV: E=Sophos;i="6.04,269,1695711600"; d="scan'208";a="379738942" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Dec 2023 18:33:37 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10921"; a="1020503531" X-IronPort-AV: E=Sophos;i="6.04,269,1695711600"; d="scan'208";a="1020503531" Received: from wenbin-z390-aorus-ultra.sh.intel.com ([10.239.156.43]) by fmsmga006.fm.intel.com with ESMTP; 11 Dec 2023 18:33:37 -0800 From: wenbin.chen-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Tue, 12 Dec 2023 10:33:32 +0800 Message-Id: <20231212023334.2506376-2-wenbin.chen@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231212023334.2506376-1-wenbin.chen@intel.com> References: <20231212023334.2506376-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 2/4] libavfilter/vf_dnn_detect: Add input pad X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: From: Wenbin Chen Add input pad to get model input resolution. Detection models always have fixed input size. And the output coordinators are based on the input resolution, so we need to get input size to map coordinators to our real output frames. Signed-off-by: Wenbin Chen --- libavfilter/dnn/dnn_backend_openvino.c | 24 ++++++++++++++++------ libavfilter/vf_dnn_detect.c | 28 +++++++++++++++++++++++++- 2 files changed, 45 insertions(+), 7 deletions(-) diff --git a/libavfilter/dnn/dnn_backend_openvino.c b/libavfilter/dnn/dnn_backend_openvino.c index 089e028818..671a995c70 100644 --- a/libavfilter/dnn/dnn_backend_openvino.c +++ b/libavfilter/dnn/dnn_backend_openvino.c @@ -1073,9 +1073,15 @@ static int get_input_ov(void *model, DNNData *input, const char *input_name) return AVERROR(ENOSYS); } - input->channels = dims[1]; - input->height = input_resizable ? -1 : dims[2]; - input->width = input_resizable ? -1 : dims[3]; + if (dims[1] <= 3) { // NCHW + input->channels = dims[1]; + input->height = input_resizable ? -1 : dims[2]; + input->width = input_resizable ? -1 : dims[3]; + } else { // NHWC + input->height = input_resizable ? -1 : dims[1]; + input->width = input_resizable ? -1 : dims[2]; + input->channels = dims[3]; + } input->dt = precision_to_datatype(precision); return 0; @@ -1105,9 +1111,15 @@ static int get_input_ov(void *model, DNNData *input, const char *input_name) return DNN_GENERIC_ERROR; } - input->channels = dims.dims[1]; - input->height = input_resizable ? -1 : dims.dims[2]; - input->width = input_resizable ? -1 : dims.dims[3]; + if (dims[1] <= 3) { // NCHW + input->channels = dims[1]; + input->height = input_resizable ? -1 : dims[2]; + input->width = input_resizable ? -1 : dims[3]; + } else { // NHWC + input->height = input_resizable ? -1 : dims[1]; + input->width = input_resizable ? -1 : dims[2]; + input->channels = dims[3]; + } input->dt = precision_to_datatype(precision); return 0; } diff --git a/libavfilter/vf_dnn_detect.c b/libavfilter/vf_dnn_detect.c index 373dda58bf..86f61c9907 100644 --- a/libavfilter/vf_dnn_detect.c +++ b/libavfilter/vf_dnn_detect.c @@ -699,13 +699,39 @@ static av_cold void dnn_detect_uninit(AVFilterContext *context) free_detect_labels(ctx); } +static int config_input(AVFilterLink *inlink) +{ + AVFilterContext *context = inlink->dst; + DnnDetectContext *ctx = context->priv; + DNNData model_input; + int ret; + + ret = ff_dnn_get_input(&ctx->dnnctx, &model_input); + if (ret != 0) { + av_log(ctx, AV_LOG_ERROR, "could not get input from the model\n"); + return ret; + } + ctx->scale_width = model_input.width == -1 ? inlink->w : model_input.width; + ctx->scale_height = model_input.height == -1 ? inlink->h : model_input.height; + + return 0; +} + +static const AVFilterPad dnn_detect_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_input, + }, +}; + const AVFilter ff_vf_dnn_detect = { .name = "dnn_detect", .description = NULL_IF_CONFIG_SMALL("Apply DNN detect filter to the input."), .priv_size = sizeof(DnnDetectContext), .init = dnn_detect_init, .uninit = dnn_detect_uninit, - FILTER_INPUTS(ff_video_default_filterpad), + FILTER_INPUTS(dnn_detect_inputs), FILTER_OUTPUTS(ff_video_default_filterpad), FILTER_PIXFMTS_ARRAY(pix_fmts), .priv_class = &dnn_detect_class, -- 2.34.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".