From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id C3F324E05D for ; Sat, 8 Mar 2025 14:58:12 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 655D568F3F6; Sat, 8 Mar 2025 16:58:08 +0200 (EET) Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 48AE468DC9D for ; Sat, 8 Mar 2025 16:58:02 +0200 (EET) Received: by mail-wr1-f46.google.com with SMTP id ffacd0b85a97d-3911748893aso1845390f8f.3 for ; Sat, 08 Mar 2025 06:58:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1741445881; x=1742050681; darn=ffmpeg.org; h=content-language:thread-index:content-transfer-encoding :mime-version:message-id:date:subject:in-reply-to:references:to:from :from:to:cc:subject:date:message-id:reply-to; bh=NOcrlMHbVo4x18b9n4JcTOUOXr80VCAXlU6ieX7BpGQ=; b=CW5DT9EJcIh71j+N86h4lA5XjT1D0TNIqdjRYn5b8ve5BOPtg8UqPdE11NXOvxVU6f mBxeeW3vqTVAG4/Mk+6Fs0EwCVj2qbZ2YoMypiWfC+tpOfFrL93L6Vmd8mnXOdAFHvLZ JmX3db2QwjUCrjR2TJYJdgZJsjIKSt5XEKRHBFTaCQcfPX9Qz4MIkouJuqguKNXAPrfR Vwi94o/NjisTvEFlDjTpnYKnSa1kTYgZBUzVs4vTUPSISyzm4/P6dBxuMGElnEhakqei C410Ouxjeq+nbGMMEc/fzoZI+6aQ+L+dW2OQcXV7zJY8uAmWMEXX4okCdckHPlKcvlap zD9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741445881; x=1742050681; h=content-language:thread-index:content-transfer-encoding :mime-version:message-id:date:subject:in-reply-to:references:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NOcrlMHbVo4x18b9n4JcTOUOXr80VCAXlU6ieX7BpGQ=; b=Qg040/qOfR6g2+L8qQE30LollMI0j26VBf3FwtqhTlVfhJs1QWB7RIvvfq0THWYPaf CXCeTUZJ013nqFlKzs+utPjrgWgL04Z5Zyk7dosYk8cCKrs0d+VH0zIdN8CbDCfmbQW9 DdxETJUAdr2syIC7kjWL42oHbHL4X05mxDHy6uSWNYdvaRBsI67tDqerygvjJScT3VUG hWioJlM3Ntcr5jhZyP8Hay5CWGAiNzAWdFeZ11xwedEI8gpp2CHGp6XNmlj+DAtQRoFF I1SN1ta5qDzc79WnGgAJcIL53wAxt35NcNDxZwZGCtPMWApKBWZld/Ii+dYVeigG6ASB Nl1Q== X-Gm-Message-State: AOJu0YxsqaCyc5GBwMHCoxYlCnrmKuV6AynFhONmhW9OSMm3NLdoSRRB Q8/J2uMEY1zwzWBJ+oX4V/nAZhlKB3K7BLfnyn0p0hgpCLqsziO9/RY7Cg== X-Gm-Gg: ASbGncvwG72plmTpGbSPtm2D2fXaf+QthNfsAhZYQEA8LqvCIfjjjtQyiMKTnDueLps 9y8NZsnM6han0BVZOFiAcubE1S3SQ/K23/rbOucNn12iPQJrGT/JUXJVyGpnzI4l7w38WgkECCS cplWGP9ovyLyHK+3nsOPH1Y8IIbmk8/X6sJO9uI2/CoIU6FyYjwYgHM4MaokQ8N+KOXGp7Jjfpo T5DrQ3XZ9csP4DvTuY33USTMEGwEi10WpAf99s/+o9lK1E6dK2TxUsjWlqjnxQkhsJP3kQFtr2S NbSfctgPwW7b0efyAPC7s9he+S7eFZUl49BooKbEhuVIdNsj164Iy9azrC0nLBcK0w8R2bx2UUq CE5u62TEF0T7vgbul X-Google-Smtp-Source: AGHT+IE2eI5cxbu0aaZBZqJfZUI3AGk7tO2Y1/1LW5sM3LpmJ1v3D7QWGGVawZG2mXoCLtLYAUMbtw== X-Received: by 2002:a05:6000:1f82:b0:391:3fde:1da with SMTP id ffacd0b85a97d-3913fde033cmr1149262f8f.16.1741445881053; Sat, 08 Mar 2025 06:58:01 -0800 (PST) Received: from MK2 (80-108-16-220.cable.dynamic.surfer.at. [80.108.16.220]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3912bfdfba9sm9225334f8f.39.2025.03.08.06.58.00 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 08 Mar 2025 06:58:00 -0800 (PST) From: To: References: In-Reply-To: Date: Sat, 8 Mar 2025 15:58:04 +0100 Message-ID: <007301db903a$7ea1afb0$7be50f10$@gmail.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 16.0 Thread-Index: AduQOERHTOCnyzHnS4K9or9LHo+23AAAh3/w Content-Language: en-at Subject: [FFmpeg-devel] [PATCH FFmpeg 1/15] libavutil: add detectionbbox util functions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Those functions will be used by classify in the upcoming patches. Try the new filters using my Github Repo https://github.com/MaximilianKaindl/DeepFFMPEGVideoClassification. Any Feedback is appreciated! Signed-off-by: MaximilianKaindl --- libavutil/detection_bbox.c | 54 ++++++++++++++++++++++++++++++++++++++ libavutil/detection_bbox.h | 31 ++++++++++++++++++++++ 2 files changed, 85 insertions(+) diff --git a/libavutil/detection_bbox.c b/libavutil/detection_bbox.c index cb157b355b..378233121d 100644 --- a/libavutil/detection_bbox.c +++ b/libavutil/detection_bbox.c @@ -18,6 +18,7 @@ #include "detection_bbox.h" #include "mem.h" +#include "libavutil/avstring.h" AVDetectionBBoxHeader *av_detection_bbox_alloc(uint32_t nb_bboxes, size_t *out_size) { @@ -71,3 +72,56 @@ AVDetectionBBoxHeader *av_detection_bbox_create_side_data(AVFrame *frame, uint32 return header; } + +int av_detection_bbox_fill_with_best_labels(char **labels, float +*probabilities, int num_labels, AVDetectionBBox *bbox, int max_classes_per_box, float confidence_threshold) { + int i, j, minpos, ret; + float min; + + if (!labels || !probabilities || !bbox) { + return AVERROR(EINVAL); + } + + for (i = 0; i < num_labels; i++) { + if (probabilities[i] >= confidence_threshold) { + if (bbox->classify_count >= max_classes_per_box) { + // Find lowest probability classification + min = av_q2d(bbox->classify_confidences[0]); + minpos = 0; + for (j = 1; j < bbox->classify_count; j++) { + float prob = av_q2d(bbox->classify_confidences[j]); + if (prob < min) { + min = prob; + minpos = j; + } + } + + if (probabilities[i] > min) { + ret = av_detection_bbox_set_content(bbox, labels[i], minpos, probabilities[i]); + if (ret < 0) + return ret; + } + } else { + ret = av_detection_bbox_set_content(bbox, labels[i], bbox->classify_count, probabilities[i]); + if (ret < 0) + return ret; + bbox->classify_count++; + } + } + } + return 0; +} + +int av_detection_bbox_set_content(AVDetectionBBox *bbox, char *label, +int index, float probability) { + // Set probability + bbox->classify_confidences[index] = av_make_q((int)(probability * +10000), 10000); + + // Copy label with size checking + if (av_strlcpy(bbox->classify_labels[index], label, AV_DETECTION_BBOX_LABEL_NAME_MAX_SIZE) >= + AV_DETECTION_BBOX_LABEL_NAME_MAX_SIZE) { + av_log(NULL, AV_LOG_WARNING, "Label truncated in set_prob_and_label_of_bbox\n"); + } + + return 0; +} diff --git a/libavutil/detection_bbox.h b/libavutil/detection_bbox.h index 011988052c..27d749ad59 100644 --- a/libavutil/detection_bbox.h +++ b/libavutil/detection_bbox.h @@ -105,4 +105,35 @@ AVDetectionBBoxHeader *av_detection_bbox_alloc(uint32_t nb_bboxes, size_t *out_s * AV_FRAME_DATA_DETECTION_BBOXES and initializes the variables. */ AVDetectionBBoxHeader *av_detection_bbox_create_side_data(AVFrame *frame, uint32_t nb_bboxes); + +/** + * Fills an AVDetectionBBox structure with the best labels based on probabilities. + * + * This function selects up to max_classes_per_box labels with the +highest probabilities + * that exceed the given confidence threshold, and assigns them to the bounding box. + * + * @param labels Array of label strings + * @param probabilities Array of probability values corresponding to +each label + * @param num_labels Number of elements in the labels and probabilities +arrays + * @param bbox Pointer to the AVDetectionBBox structure to be filled + * @param max_classes_per_box Maximum number of classes to assign to +the bounding box + * @param confidence_threshold Minimum probability value required for a +label to be considered + * @return 0 on success, negative error code on failure */ int +av_detection_bbox_fill_with_best_labels(char **labels, float +*probabilities, int num_labels, AVDetectionBBox *bbox, int +max_classes_per_box, float confidence_threshold); + +/** + * Sets the content of an AVDetectionBBox at the specified index. + * + * This function assigns a label and its associated probability to the +specified index + * in the bounding box's internal storage. + * + * @param bbox Pointer to the AVDetectionBBox structure to modify + * @param label The class label to assign (will be copied internally) + * @param index The index at which to store the label and probability + * @param probability The confidence score/probability for this label + * @return 0 on success + */ +int av_detection_bbox_set_content(AVDetectionBBox *bbox, char *label, +int index, float probability); + #endif -- 2.34.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".