* Re: [FFmpeg-devel] [PATCH 1/2] lafi/vf_edgedetect: Move some common functions into seperate file [not found] <af635ff1-fc5c-d39a-4c39-424d5eefbe6f@mail.de> @ 2022-04-04 10:25 ` Thilo Borgmann [not found] ` <00ca6f6f-fc18-8b6b-1ff0-f24db7f3714b@mail.de> 1 sibling, 0 replies; 10+ messages in thread From: Thilo Borgmann @ 2022-04-04 10:25 UTC (permalink / raw) To: ffmpeg-devel [-- Attachment #1: Type: text/plain, Size: 40 bytes --] Hi, v3 updated to current HEAD. -Thilo [-- Attachment #2: v3-0001-lafi-vf_edgedetect-Move-some-common-functions-int.patch --] [-- Type: text/plain, Size: 21266 bytes --] From ba103f0a51557aae4f1c242ccc394da6ddce64fb Mon Sep 17 00:00:00 2001 From: Thilo Borgmann <thilo.borgmann@mail.de> Date: Tue, 30 Nov 2021 00:16:52 +0100 Subject: [PATCH v3 1/2] lafi/vf_edgedetect: Move some common functions into seperate file --- libavfilter/Makefile | 2 +- libavfilter/edge_common.c | 181 +++++++++++++++++++++++++++++++++++ libavfilter/edge_common.h | 107 +++++++++++++++++++++ libavfilter/vf_edgedetect.c | 183 ++---------------------------------- 4 files changed, 296 insertions(+), 177 deletions(-) create mode 100644 libavfilter/edge_common.c create mode 100644 libavfilter/edge_common.h diff --git a/libavfilter/Makefile b/libavfilter/Makefile index c4c946a988..006e59b2bd 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -265,7 +265,7 @@ OBJS-$(CONFIG_DRAWBOX_FILTER) += vf_drawbox.o OBJS-$(CONFIG_DRAWGRAPH_FILTER) += f_drawgraph.o OBJS-$(CONFIG_DRAWGRID_FILTER) += vf_drawbox.o OBJS-$(CONFIG_DRAWTEXT_FILTER) += vf_drawtext.o -OBJS-$(CONFIG_EDGEDETECT_FILTER) += vf_edgedetect.o +OBJS-$(CONFIG_EDGEDETECT_FILTER) += vf_edgedetect.o edge_common.o OBJS-$(CONFIG_ELBG_FILTER) += vf_elbg.o OBJS-$(CONFIG_ENTROPY_FILTER) += vf_entropy.o OBJS-$(CONFIG_EPX_FILTER) += vf_epx.o diff --git a/libavfilter/edge_common.c b/libavfilter/edge_common.c new file mode 100644 index 0000000000..d72e8521cd --- /dev/null +++ b/libavfilter/edge_common.c @@ -0,0 +1,181 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "edge_common.h" + +// Internal helper for ff_sobel() +static int get_rounded_direction(int gx, int gy) +{ + /* reference angles: + * tan( pi/8) = sqrt(2)-1 + * tan(3pi/8) = sqrt(2)+1 + * Gy/Gx is the tangent of the angle (theta), so Gy/Gx is compared against + * <ref-angle>, or more simply Gy against <ref-angle>*Gx + * + * Gx and Gy bounds = [-1020;1020], using 16-bit arithmetic: + * round((sqrt(2)-1) * (1<<16)) = 27146 + * round((sqrt(2)+1) * (1<<16)) = 158218 + */ + if (gx) { + int tanpi8gx, tan3pi8gx; + + if (gx < 0) + gx = -gx, gy = -gy; + gy *= (1 << 16); + tanpi8gx = 27146 * gx; + tan3pi8gx = 158218 * gx; + if (gy > -tan3pi8gx && gy < -tanpi8gx) return DIRECTION_45UP; + if (gy > -tanpi8gx && gy < tanpi8gx) return DIRECTION_HORIZONTAL; + if (gy > tanpi8gx && gy < tan3pi8gx) return DIRECTION_45DOWN; + } + return DIRECTION_VERTICAL; +} + +// Simple sobel operator to get rounded gradients +void ff_sobel(int w, int h, + uint16_t *dst, int dst_linesize, + int8_t *dir, int dir_linesize, + const uint8_t *src, int src_linesize) +{ + int i, j; + + for (j = 1; j < h - 1; j++) { + dst += dst_linesize; + dir += dir_linesize; + src += src_linesize; + for (i = 1; i < w - 1; i++) { + const int gx = + -1*src[-src_linesize + i-1] + 1*src[-src_linesize + i+1] + -2*src[ i-1] + 2*src[ i+1] + -1*src[ src_linesize + i-1] + 1*src[ src_linesize + i+1]; + const int gy = + -1*src[-src_linesize + i-1] + 1*src[ src_linesize + i-1] + -2*src[-src_linesize + i ] + 2*src[ src_linesize + i ] + -1*src[-src_linesize + i+1] + 1*src[ src_linesize + i+1]; + + dst[i] = FFABS(gx) + FFABS(gy); + dir[i] = get_rounded_direction(gx, gy); + } + } +} + +// Filters rounded gradients to drop all non-maxima +// Expects gradients generated by ff_sobel() +// Expects zero's destination buffer +void ff_non_maximum_suppression(int w, int h, + uint8_t *dst, int dst_linesize, + const int8_t *dir, int dir_linesize, + const uint16_t *src, int src_linesize) +{ + int i, j; + +#define COPY_MAXIMA(ay, ax, by, bx) do { \ + if (src[i] > src[(ay)*src_linesize + i+(ax)] && \ + src[i] > src[(by)*src_linesize + i+(bx)]) \ + dst[i] = av_clip_uint8(src[i]); \ +} while (0) + + for (j = 1; j < h - 1; j++) { + dst += dst_linesize; + dir += dir_linesize; + src += src_linesize; + for (i = 1; i < w - 1; i++) { + switch (dir[i]) { + case DIRECTION_45UP: COPY_MAXIMA( 1, -1, -1, 1); break; + case DIRECTION_45DOWN: COPY_MAXIMA(-1, -1, 1, 1); break; + case DIRECTION_HORIZONTAL: COPY_MAXIMA( 0, -1, 0, 1); break; + case DIRECTION_VERTICAL: COPY_MAXIMA(-1, 0, 1, 0); break; + } + } + } +} + +// Filter to keep all pixels > high, and keep all pixels > low where all surrounding pixels > high +void ff_double_threshold(int low, int high, int w, int h, + uint8_t *dst, int dst_linesize, + const uint8_t *src, int src_linesize) +{ + int i, j; + + for (j = 0; j < h; j++) { + for (i = 0; i < w; i++) { + if (src[i] > high) { + dst[i] = src[i]; + continue; + } + + if (!(!i || i == w - 1 || !j || j == h - 1) && + src[i] > low && + (src[-src_linesize + i-1] > high || + src[-src_linesize + i ] > high || + src[-src_linesize + i+1] > high || + src[ i-1] > high || + src[ i+1] > high || + src[ src_linesize + i-1] > high || + src[ src_linesize + i ] > high || + src[ src_linesize + i+1] > high)) + dst[i] = src[i]; + else + dst[i] = 0; + } + dst += dst_linesize; + src += src_linesize; + } +} + +// Applies gaussian blur, using 5x5 kernels, sigma = 1.4 +void ff_gaussian_blur(int w, int h, + uint8_t *dst, int dst_linesize, + const uint8_t *src, int src_linesize) +{ + int i, j; + + memcpy(dst, src, w); dst += dst_linesize; src += src_linesize; + memcpy(dst, src, w); dst += dst_linesize; src += src_linesize; + for (j = 2; j < h - 2; j++) { + dst[0] = src[0]; + dst[1] = src[1]; + for (i = 2; i < w - 2; i++) { + /* Gaussian mask of size 5x5 with sigma = 1.4 */ + dst[i] = ((src[-2*src_linesize + i-2] + src[2*src_linesize + i-2]) * 2 + + (src[-2*src_linesize + i-1] + src[2*src_linesize + i-1]) * 4 + + (src[-2*src_linesize + i ] + src[2*src_linesize + i ]) * 5 + + (src[-2*src_linesize + i+1] + src[2*src_linesize + i+1]) * 4 + + (src[-2*src_linesize + i+2] + src[2*src_linesize + i+2]) * 2 + + + (src[ -src_linesize + i-2] + src[ src_linesize + i-2]) * 4 + + (src[ -src_linesize + i-1] + src[ src_linesize + i-1]) * 9 + + (src[ -src_linesize + i ] + src[ src_linesize + i ]) * 12 + + (src[ -src_linesize + i+1] + src[ src_linesize + i+1]) * 9 + + (src[ -src_linesize + i+2] + src[ src_linesize + i+2]) * 4 + + + src[i-2] * 5 + + src[i-1] * 12 + + src[i ] * 15 + + src[i+1] * 12 + + src[i+2] * 5) / 159; + } + dst[i ] = src[i ]; + dst[i + 1] = src[i + 1]; + + dst += dst_linesize; + src += src_linesize; + } + memcpy(dst, src, w); dst += dst_linesize; src += src_linesize; + memcpy(dst, src, w); +} diff --git a/libavfilter/edge_common.h b/libavfilter/edge_common.h new file mode 100644 index 0000000000..87c143f2b8 --- /dev/null +++ b/libavfilter/edge_common.h @@ -0,0 +1,107 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * common functions for edge detection + */ + +#ifndef AVFILTER_EDGE_COMMON_H +#define AVFILTER_EDGE_COMMON_H + +#include "avfilter.h" + +/** + * @brief Rounded directions used in av_image_sobel() + */ +enum AVRoundedDirection { + DIRECTION_45UP, + DIRECTION_45DOWN, + DIRECTION_HORIZONTAL, + DIRECTION_VERTICAL, +}; + +/** + * Simple sobel operator to get rounded gradients + * + * @param w the width of the image in pixels + * @param h the height of the image in pixels + * @param dst data pointers to magnitude image + * @param dst_linesize linesizes for the magnitude image + * @param dir data pointers to direction image + * @param dir_linesize linesizes for the direction image + * @param src data pointers to source image + * @param src_linesize linesizes for the source image + */ +void ff_sobel(int w, int h, + uint16_t *dst, int dst_linesize, + int8_t *dir, int dir_linesize, + const uint8_t *src, int src_linesize); + +/** + * Filters rounded gradients to drop all non-maxima pixels in the magnitude image + * Expects gradients generated by av_image_sobel() + * Expects zero's in the destination buffer dst + * + * @param w the width of the image in pixels + * @param h the height of the image in pixels + * @param dst data pointers to magnitude image + * @param dst_linesize linesizes for the magnitude image + * @param dir data pointers to direction image + * @param dir_linesize linesizes for the direction image + * @param src data pointers to source image + * @param src_linesize linesizes for the source image + */ +void ff_non_maximum_suppression(int w, int h, + uint8_t *dst, int dst_linesize, + const int8_t *dir, int dir_linesize, + const uint16_t *src, int src_linesize); + +/** + * Filters all pixels in src to keep all pixels > high, + * and keep all pixels > low where all surrounding pixels > high + * + * @param low the low threshold value + * @param high the hegh threshold value + * @param w the width of the image in pixels + * @param h the height of the image in pixels + * @param dst data pointers to destination image + * @param dst_linesize linesizes for the destination image + * @param src data pointers to source image + * @param src_linesize linesizes for the source image + */ +void ff_double_threshold(int low, int high, int w, int h, + uint8_t *dst, int dst_linesize, + const uint8_t *src, int src_linesize); + +/** + * Applies gaussian blur. + * 5x5 kernels, sigma = 1.4 + * + * @param w the width of the image in pixels + * @param h the height of the image in pixels + * @param dst data pointers to destination image + * @param dst_linesize linesizes for the destination image + * @param src data pointers to source image + * @param src_linesize linesizes for the source image + */ +void ff_gaussian_blur(int w, int h, + uint8_t *dst, int dst_linesize, + const uint8_t *src, int src_linesize); + +#endif diff --git a/libavfilter/vf_edgedetect.c b/libavfilter/vf_edgedetect.c index 3eea34e325..90390ceb3e 100644 --- a/libavfilter/vf_edgedetect.c +++ b/libavfilter/vf_edgedetect.c @@ -32,6 +32,7 @@ #include "formats.h" #include "internal.h" #include "video.h" +#include "edge_common.h" #define PLANE_R 0x4 #define PLANE_G 0x1 @@ -139,176 +140,6 @@ static int config_props(AVFilterLink *inlink) return 0; } -static void gaussian_blur(AVFilterContext *ctx, int w, int h, - uint8_t *dst, int dst_linesize, - const uint8_t *src, int src_linesize) -{ - int i, j; - - memcpy(dst, src, w); dst += dst_linesize; src += src_linesize; - if (h > 1) { - memcpy(dst, src, w); dst += dst_linesize; src += src_linesize; - } - for (j = 2; j < h - 2; j++) { - dst[0] = src[0]; - if (w > 1) - dst[1] = src[1]; - for (i = 2; i < w - 2; i++) { - /* Gaussian mask of size 5x5 with sigma = 1.4 */ - dst[i] = ((src[-2*src_linesize + i-2] + src[2*src_linesize + i-2]) * 2 - + (src[-2*src_linesize + i-1] + src[2*src_linesize + i-1]) * 4 - + (src[-2*src_linesize + i ] + src[2*src_linesize + i ]) * 5 - + (src[-2*src_linesize + i+1] + src[2*src_linesize + i+1]) * 4 - + (src[-2*src_linesize + i+2] + src[2*src_linesize + i+2]) * 2 - - + (src[ -src_linesize + i-2] + src[ src_linesize + i-2]) * 4 - + (src[ -src_linesize + i-1] + src[ src_linesize + i-1]) * 9 - + (src[ -src_linesize + i ] + src[ src_linesize + i ]) * 12 - + (src[ -src_linesize + i+1] + src[ src_linesize + i+1]) * 9 - + (src[ -src_linesize + i+2] + src[ src_linesize + i+2]) * 4 - - + src[i-2] * 5 - + src[i-1] * 12 - + src[i ] * 15 - + src[i+1] * 12 - + src[i+2] * 5) / 159; - } - if (w > 2) - dst[i ] = src[i ]; - if (w > 3) - dst[i + 1] = src[i + 1]; - - dst += dst_linesize; - src += src_linesize; - } - if (h > 2) { - memcpy(dst, src, w); dst += dst_linesize; src += src_linesize; - } - if (h > 3) - memcpy(dst, src, w); -} - -enum { - DIRECTION_45UP, - DIRECTION_45DOWN, - DIRECTION_HORIZONTAL, - DIRECTION_VERTICAL, -}; - -static int get_rounded_direction(int gx, int gy) -{ - /* reference angles: - * tan( pi/8) = sqrt(2)-1 - * tan(3pi/8) = sqrt(2)+1 - * Gy/Gx is the tangent of the angle (theta), so Gy/Gx is compared against - * <ref-angle>, or more simply Gy against <ref-angle>*Gx - * - * Gx and Gy bounds = [-1020;1020], using 16-bit arithmetic: - * round((sqrt(2)-1) * (1<<16)) = 27146 - * round((sqrt(2)+1) * (1<<16)) = 158218 - */ - if (gx) { - int tanpi8gx, tan3pi8gx; - - if (gx < 0) - gx = -gx, gy = -gy; - gy *= (1 << 16); - tanpi8gx = 27146 * gx; - tan3pi8gx = 158218 * gx; - if (gy > -tan3pi8gx && gy < -tanpi8gx) return DIRECTION_45UP; - if (gy > -tanpi8gx && gy < tanpi8gx) return DIRECTION_HORIZONTAL; - if (gy > tanpi8gx && gy < tan3pi8gx) return DIRECTION_45DOWN; - } - return DIRECTION_VERTICAL; -} - -static void sobel(int w, int h, - uint16_t *dst, int dst_linesize, - int8_t *dir, int dir_linesize, - const uint8_t *src, int src_linesize) -{ - int i, j; - - for (j = 1; j < h - 1; j++) { - dst += dst_linesize; - dir += dir_linesize; - src += src_linesize; - for (i = 1; i < w - 1; i++) { - const int gx = - -1*src[-src_linesize + i-1] + 1*src[-src_linesize + i+1] - -2*src[ i-1] + 2*src[ i+1] - -1*src[ src_linesize + i-1] + 1*src[ src_linesize + i+1]; - const int gy = - -1*src[-src_linesize + i-1] + 1*src[ src_linesize + i-1] - -2*src[-src_linesize + i ] + 2*src[ src_linesize + i ] - -1*src[-src_linesize + i+1] + 1*src[ src_linesize + i+1]; - - dst[i] = FFABS(gx) + FFABS(gy); - dir[i] = get_rounded_direction(gx, gy); - } - } -} - -static void non_maximum_suppression(int w, int h, - uint8_t *dst, int dst_linesize, - const int8_t *dir, int dir_linesize, - const uint16_t *src, int src_linesize) -{ - int i, j; - -#define COPY_MAXIMA(ay, ax, by, bx) do { \ - if (src[i] > src[(ay)*src_linesize + i+(ax)] && \ - src[i] > src[(by)*src_linesize + i+(bx)]) \ - dst[i] = av_clip_uint8(src[i]); \ -} while (0) - - for (j = 1; j < h - 1; j++) { - dst += dst_linesize; - dir += dir_linesize; - src += src_linesize; - for (i = 1; i < w - 1; i++) { - switch (dir[i]) { - case DIRECTION_45UP: COPY_MAXIMA( 1, -1, -1, 1); break; - case DIRECTION_45DOWN: COPY_MAXIMA(-1, -1, 1, 1); break; - case DIRECTION_HORIZONTAL: COPY_MAXIMA( 0, -1, 0, 1); break; - case DIRECTION_VERTICAL: COPY_MAXIMA(-1, 0, 1, 0); break; - } - } - } -} - -static void double_threshold(int low, int high, int w, int h, - uint8_t *dst, int dst_linesize, - const uint8_t *src, int src_linesize) -{ - int i, j; - - for (j = 0; j < h; j++) { - for (i = 0; i < w; i++) { - if (src[i] > high) { - dst[i] = src[i]; - continue; - } - - if (!(!i || i == w - 1 || !j || j == h - 1) && - src[i] > low && - (src[-src_linesize + i-1] > high || - src[-src_linesize + i ] > high || - src[-src_linesize + i+1] > high || - src[ i-1] > high || - src[ i+1] > high || - src[ src_linesize + i-1] > high || - src[ src_linesize + i ] > high || - src[ src_linesize + i+1] > high)) - dst[i] = src[i]; - else - dst[i] = 0; - } - dst += dst_linesize; - src += src_linesize; - } -} - static void color_mix(int w, int h, uint8_t *dst, int dst_linesize, const uint8_t *src, int src_linesize) @@ -360,12 +191,12 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) } /* gaussian filter to reduce noise */ - gaussian_blur(ctx, width, height, - tmpbuf, width, - in->data[p], in->linesize[p]); + ff_gaussian_blur(width, height, + tmpbuf, width, + in->data[p], in->linesize[p]); /* compute the 16-bits gradients and directions for the next step */ - sobel(width, height, + ff_sobel(width, height, gradients, width, directions,width, tmpbuf, width); @@ -373,13 +204,13 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) /* non_maximum_suppression() will actually keep & clip what's necessary and * ignore the rest, so we need a clean output buffer */ memset(tmpbuf, 0, width * height); - non_maximum_suppression(width, height, + ff_non_maximum_suppression(width, height, tmpbuf, width, directions,width, gradients, width); /* keep high values, or low values surrounded by high values */ - double_threshold(edgedetect->low_u8, edgedetect->high_u8, + ff_double_threshold(edgedetect->low_u8, edgedetect->high_u8, width, height, out->data[p], out->linesize[p], tmpbuf, width); -- 2.20.1 (Apple Git-117) [-- Attachment #3: Type: text/plain, Size: 251 bytes --] _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 10+ messages in thread
[parent not found: <00ca6f6f-fc18-8b6b-1ff0-f24db7f3714b@mail.de>]
* Re: [FFmpeg-devel] [PATCH 2/2] lavfi: Add blurriness filter [not found] ` <00ca6f6f-fc18-8b6b-1ff0-f24db7f3714b@mail.de> @ 2022-04-04 10:25 ` Thilo Borgmann 2022-04-04 10:42 ` Paul B Mahol 0 siblings, 1 reply; 10+ messages in thread From: Thilo Borgmann @ 2022-04-04 10:25 UTC (permalink / raw) To: ffmpeg-devel [-- Attachment #1: Type: text/plain, Size: 119 bytes --] Hi, v3 updated to current HEAD. Named blurdetect filter now. Minor fixes on allocation and removed -f option. -Thilo [-- Attachment #2: v3-0002-lavfi-Add-blurdetect-filter.patch --] [-- Type: text/plain, Size: 15672 bytes --] From 1c81cf6f0586a3cc670e6d8354f35e5bdbe2a282 Mon Sep 17 00:00:00 2001 From: Thilo Borgmann <thilo.borgmann@mail.de> Date: Mon, 4 Apr 2022 12:21:14 +0200 Subject: [PATCH v3 2/2] lavfi: Add blurdetect filter --- doc/filters.texi | 49 +++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_blurdetect.c | 368 ++++++++++++++++++++++++++++++++++++ 4 files changed, 419 insertions(+) create mode 100644 libavfilter/vf_blurdetect.c diff --git a/doc/filters.texi b/doc/filters.texi index 636c80dbff..f15fb719d5 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -7990,6 +7990,55 @@ tblend=all_mode=grainextract @subsection Commands This filter supports same @ref{commands} as options. +@anchor{blurdetect} +@section blurdetect + +Determines blurriness of frames without altering the input frames. + +Based on Marziliano, Pina, et al. "A no-reference perceptual blur metric." +Allows for a block-based abbreviation. + +The filter accepts the following options: + +@table @option +@item low +@item high +Set low and high threshold values used by the Canny thresholding +algorithm. + +The high threshold selects the "strong" edge pixels, which are then +connected through 8-connectivity with the "weak" edge pixels selected +by the low threshold. + +@var{low} and @var{high} threshold values must be chosen in the range +[0,1], and @var{low} should be lesser or equal to @var{high}. + +Default value for @var{low} is @code{20/255}, and default value for @var{high} +is @code{50/255}. + +@item radius +Define the radius to search around an edge pixel for local maxima. + +@item block_pct +Determine blurriness only for the most significant blocks, given in percentage. + +@item block_width +Determine blurriness for blocks of width @var{block_width}. If set to any value smaller 1, no blocks are used and the whole image is processed as one no matter of @var{block_height}. + +@item block_height +Determine blurriness for blocks of height @var{block_height}. If set to any value smaller 1, no blocks are used and the whole image is processed as one no matter of @var{block_width}. +@end table + +@subsection Examples + +@itemize +@item +Determine blur for 80% of most significant 32x32 blocks: +@example +blurdetect=block_width=32:block_height=32:block_pct=80 +@end example +@end itemize + @section bm3d Denoise frames using Block-Matching 3D algorithm. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 006e59b2bd..6332a6f799 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -195,6 +195,7 @@ OBJS-$(CONFIG_BLACKDETECT_FILTER) += vf_blackdetect.o OBJS-$(CONFIG_BLACKFRAME_FILTER) += vf_blackframe.o OBJS-$(CONFIG_BLEND_FILTER) += vf_blend.o framesync.o OBJS-$(CONFIG_BLEND_VULKAN_FILTER) += vf_blend_vulkan.o framesync.o vulkan.o vulkan_filter.o +OBJS-$(CONFIG_BLURDETECT_FILTER) += vf_blurdetect.o edge_common.o OBJS-$(CONFIG_BM3D_FILTER) += vf_bm3d.o framesync.o OBJS-$(CONFIG_BOXBLUR_FILTER) += vf_boxblur.o boxblur.o OBJS-$(CONFIG_BOXBLUR_OPENCL_FILTER) += vf_avgblur_opencl.o opencl.o \ diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 9fbaaacf47..2667d153ad 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -183,6 +183,7 @@ extern const AVFilter ff_vf_blackdetect; extern const AVFilter ff_vf_blackframe; extern const AVFilter ff_vf_blend; extern const AVFilter ff_vf_blend_vulkan; +extern const AVFilter ff_vf_blurdetect; extern const AVFilter ff_vf_bm3d; extern const AVFilter ff_vf_boxblur; extern const AVFilter ff_vf_boxblur_opencl; diff --git a/libavfilter/vf_blurdetect.c b/libavfilter/vf_blurdetect.c new file mode 100644 index 0000000000..050cec9529 --- /dev/null +++ b/libavfilter/vf_blurdetect.c @@ -0,0 +1,368 @@ +/* + * Copyright (c) 2021 Thilo Borgmann <thilo.borgmann _at_ mail.de> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * No-reference blurdetect filter + * + * Implementing: + * Marziliano, Pina, et al. "A no-reference perceptual blur metric." Proceedings. + * International conference on image processing. Vol. 3. IEEE, 2002. + * https://infoscience.epfl.ch/record/111802/files/14%20A%20no-reference%20perceptual%20blur%20metric.pdf + * + * @author Thilo Borgmann <thilo.borgmann _at_ mail.de> + */ + +#include "libavutil/avassert.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "libavutil/pixelutils.h" +#include "libavutil/motion_vector.h" +#include "avfilter.h" +#include "formats.h" +#include "internal.h" +#include "video.h" +#include "edge_common.h" + +static int comp(const void *a,const void *b) +{ + float x = *(float*)a; + float y = *(float*)b; + if (x > y) return 1; + if (x < y) return -1; + return 0; +} + +typedef struct BLRContext { + const AVClass *class; + + float low, high; + uint8_t low_u8, high_u8; + int radius; // radius during local maxima detection + int block_pct; // percentage of "sharpest" blocks in the image to use for bluriness calculation + int block_width; // width for block abbreviation + int block_height; // height for block abbreviation + + double blur_total; + uint64_t nb_frames; + + float *blks; + uint8_t *filterbuf; + uint8_t *tmpbuf; + uint16_t *gradients; + char *directions; +} BLRContext; + +#define OFFSET(x) offsetof(BLRContext, x) +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM +static const AVOption blr_options[] = { + { "high", "set high threshold", OFFSET(high), AV_OPT_TYPE_FLOAT, {.dbl=30/255.}, 0, 1, FLAGS }, + { "low", "set low threshold", OFFSET(low), AV_OPT_TYPE_FLOAT, {.dbl=15/255.}, 0, 1, FLAGS }, + { "radius", "search radius for maxima detection", OFFSET(radius), AV_OPT_TYPE_INT, {.i64=50}, 1, 100, FLAGS }, + { "block_pct", "block pooling threshold when calculating blurriness", OFFSET(block_pct), AV_OPT_TYPE_INT, {.i64=80}, 1, 100, FLAGS }, + { "block_width", "block size for block-based abbreviation of blurriness", OFFSET(block_width), AV_OPT_TYPE_INT, {.i64=-1}, -1, INT_MAX, FLAGS }, + { "block_height", "block size for block-based abbreviation of blurriness", OFFSET(block_height), AV_OPT_TYPE_INT, {.i64=-1}, -1, INT_MAX, FLAGS }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(blr); + +static av_cold int blr_init(AVFilterContext *ctx) +{ + BLRContext *blr = ctx->priv; + + blr->low_u8 = blr->low * 255. + .5; + blr->high_u8 = blr->high * 255. + .5; + + return 0; +} + +static int blr_config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + BLRContext *blr = ctx->priv; + const int bufsize = inlink->w * inlink->h; + + if (blr->block_width < 1 || blr->block_height < 1) { + blr->block_width = inlink->w; + blr->block_height = inlink->h; + } + + blr->tmpbuf = av_malloc(bufsize); + blr->filterbuf = av_malloc(bufsize); + blr->gradients = av_calloc(bufsize, sizeof(*blr->gradients)); + blr->directions = av_malloc(bufsize); + blr->blks = av_calloc((inlink->w / blr->block_width) * (inlink->h / blr->block_height), + sizeof(*blr->blks)); + + if (!blr->tmpbuf || !blr->filterbuf || !blr->gradients || + !blr->directions || !blr->blks) + return AVERROR(ENOMEM); + + return 0; +} + +// edge width is defined as the distance between surrounding maxima of the edge pixel +static float edge_width(BLRContext *blr, int i, int j, int8_t dir, int w, int h, + int edge, const uint8_t *src, int src_linesize) +{ + float width = 0; + int dX, dY; + int sign; + int tmp; + int p1; + int p2; + int k, x, y; + int edge1; + int edge2; + float luma1 = 0.0; // average luma difference per edge pixel + float luma2 = 0.0; + int radius = blr->radius; + + switch(dir) + { + case DIRECTION_HORIZONTAL: dX = 1; dY = 0; break; + case DIRECTION_VERTICAL: dX = 0; dY = 1; break; + case DIRECTION_45UP: dX = 1; dY = -1; break; + case DIRECTION_45DOWN: dX = 1; dY = 1; break; + } + if (dir == DIRECTION_HORIZONTAL) return 0; + + // determines if search in direction dX/dY is looking for a maximum or minimum + sign = src[j * src_linesize + i] > src[(j - dY) * src_linesize + i - dX] ? 1 : -1; + + // search in -(dX/dY) direction + for (k = 0; k < radius; k++) + { + x = i - k*dX; + y = j - k*dY; + p1 = y * src_linesize + x; + x -= dX; + y -= dY; + p2 = y * src_linesize + x; + if (x < 0 || x >= w || y < 0 || y >= h) + return 0; + + tmp = (src[p1] - src[p2]) * sign; + + if (tmp <= 0) // local maximum found + break; + + luma1 += tmp; + } + if (k > 0) luma1 /= k; + edge1 = k; + width += k; + + // search in +(dX/dY) direction + for (k = 0; k < radius; k++) + { + x = i + k * dX; + y = j + k * dY; + p1 = y * src_linesize + x; + x += dX; + y += dY; + p2 = y * src_linesize + x; + if (x < 0 || x >= w || y < 0 || y >= h) + return 0; + + tmp = (src[p1] - src[p2]) * sign; + + if (tmp >= 0) // local maximum found + break; + + luma2 -= tmp; + } + if (k > 0) luma2 /= k; + edge2 = k; + width += k; + + // for 45 degree directions approximate edge width in pixel units: 0.7 ~= sqrt(2)/2 + if (dir == DIRECTION_45UP || dir == DIRECTION_45DOWN) + width *= 0.7; + + return width; +} + +static float calculate_blur(BLRContext *blr, int w, int h, + uint8_t* dir, int dir_linesize, + uint8_t* dst, int dst_linesize, + uint8_t* src, int src_linesize) +{ + float total_width = 0.0; + int block_count; + float block_total_width; + + int i, j; + int blkcnt = 0; + + float *blks = blr->blks; + int block_pct = blr->block_pct; + float block_pool_threshold = block_pct / 100.0; + + int block_width = blr->block_width; + int block_height = blr->block_height; + int brows = h / block_height; + int bcols = w / block_width; + + for (int blkj = 0; blkj < brows; blkj++) { + for (int blki = 0; blki < bcols; blki++) { + block_total_width = 0; + block_count = 0; + for (int inj = 0; inj < block_height; inj++) { + for (int ini = 0; ini < block_width; ini++) { + i = blki * block_width + ini; + j = blkj * block_height + inj; + + if (dst[j * dst_linesize + i] > 0) { + float width = edge_width(blr, i, j, dir[j*dir_linesize+i], + w, h, dst[j*dst_linesize+i], + src, src_linesize); + if (width > 0.001) { // throw away zeros + block_count++; + block_total_width += width; + } + } + } + } + // if not enough edge pixels in a block, consider it smooth + if (block_total_width >= 2) { + blks[blkcnt] = block_total_width / block_count; + blkcnt++; + } + } + } + + // simple block pooling by sorting and keeping the sharper blocks + qsort(blks, blkcnt, sizeof(*blks), comp); + blkcnt = ceil(blkcnt * block_pool_threshold); + for (int i = 0; i < blkcnt; i++) { + total_width += blks[i]; + } + + return total_width / blkcnt; +} + +static void set_meta(AVDictionary **metadata, const char *key, float d) +{ + char value[128]; + snprintf(value, sizeof(value), "%f", d); + av_dict_set(metadata, key, value, 0); +} + +static int blr_filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx = inlink->dst; + BLRContext *blr = ctx->priv; + + const int w = inlink->w; + const int h = inlink->h; + + uint8_t *tmpbuf = blr->tmpbuf; + uint8_t *filterbuf = blr->filterbuf; + uint16_t *gradients = blr->gradients; + int8_t *directions = blr->directions; + + float blur; + AVDictionary **metadata; + metadata = &in->metadata; + + // gaussian filter to reduce noise + ff_gaussian_blur(w, h, + filterbuf, w, + in->data[0], in->linesize[0]); + + // compute the 16-bits gradients and directions for the next step + ff_sobel(w, h, gradients, w, directions, w, filterbuf, w); + + // non_maximum_suppression() will actually keep & clip what's necessary and + // ignore the rest, so we need a clean output buffer + memset(tmpbuf, 0, w * h); + ff_non_maximum_suppression(w, h, tmpbuf, w, directions, w, gradients, w); + + + // keep high values, or low values surrounded by high values + ff_double_threshold(blr->low_u8, blr->high_u8, w, h, + tmpbuf, w, tmpbuf, w); + + + blur = calculate_blur(blr, w, h, directions, w, + tmpbuf, w, filterbuf, w); + + blr->blur_total += blur; + + // write stats + av_log(ctx, AV_LOG_VERBOSE, "blur: %.7f\n", blur); + + set_meta(metadata, "lavfi.blur", blur); + + blr->nb_frames++; + + return ff_filter_frame(ctx->outputs[0], in); +} + +static av_cold void blr_uninit(AVFilterContext *ctx) +{ + BLRContext *blr = ctx->priv; + + if (blr->nb_frames > 0) { + int nb_frames = blr->nb_frames; + av_log(ctx, AV_LOG_INFO, "blur mean: %.7f\n", + blr->blur_total / nb_frames); + } + + av_freep(&blr->tmpbuf); + av_freep(&blr->filterbuf); + av_freep(&blr->gradients); + av_freep(&blr->directions); + av_freep(&blr->blks); +} + +static const enum AVPixelFormat pix_fmts[] = { AV_PIX_FMT_YUV420P, AV_PIX_FMT_GRAY8, AV_PIX_FMT_NONE }; + +static const AVFilterPad blr_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = blr_config_input, + .filter_frame = blr_filter_frame, + }, +}; + +static const AVFilterPad blr_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +AVFilter ff_vf_blurdetect = { + .name = "blurdetect", + .description = NULL_IF_CONFIG_SMALL("Blurdetect filter."), + .priv_size = sizeof(BLRContext), + .init = blr_init, + .uninit = blr_uninit, + FILTER_PIXFMTS_ARRAY(pix_fmts), + FILTER_INPUTS(blr_inputs), + FILTER_OUTPUTS(blr_outputs), + .priv_class = &blr_class, + .flags = AVFILTER_FLAG_METADATA_ONLY, +}; + -- 2.20.1 (Apple Git-117) [-- Attachment #3: Type: text/plain, Size: 251 bytes --] _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [FFmpeg-devel] [PATCH 2/2] lavfi: Add blurriness filter 2022-04-04 10:25 ` [FFmpeg-devel] [PATCH 2/2] lavfi: Add blurriness filter Thilo Borgmann @ 2022-04-04 10:42 ` Paul B Mahol 2022-04-05 8:56 ` Thilo Borgmann 0 siblings, 1 reply; 10+ messages in thread From: Paul B Mahol @ 2022-04-04 10:42 UTC (permalink / raw) To: FFmpeg development discussions and patches On Mon, Apr 4, 2022 at 12:25 PM Thilo Borgmann <thilo.borgmann@mail.de> wrote: > Hi, > > v3 updated to current HEAD. > > Named blurdetect filter now. > Minor fixes on allocation and removed -f option. > Please make this per plane filtering, with default to measure only first plane. > -Thilo_______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [FFmpeg-devel] [PATCH 2/2] lavfi: Add blurriness filter 2022-04-04 10:42 ` Paul B Mahol @ 2022-04-05 8:56 ` Thilo Borgmann 2022-04-14 8:08 ` Thilo Borgmann 0 siblings, 1 reply; 10+ messages in thread From: Thilo Borgmann @ 2022-04-05 8:56 UTC (permalink / raw) To: ffmpeg-devel [-- Attachment #1: Type: text/plain, Size: 319 bytes --] Hi, >> v3 updated to current HEAD. >> >> Named blurdetect filter now. >> Minor fixes on allocation and removed -f option. >> > > Please make this per plane filtering, with default to measure only first > plane. done in v4. (Will add Changelog, version.h and fate test once the filter itself looks ok) Thanks, Thilo [-- Attachment #2: v4-0002-lavfi-Add-blurdetect-filter.patch --] [-- Type: text/plain, Size: 16780 bytes --] From 6411603fe86d4fb6a781aec2ff48e48be12d269e Mon Sep 17 00:00:00 2001 From: Thilo Borgmann <thilo.borgmann@mail.de> Date: Tue, 5 Apr 2022 10:54:19 +0200 Subject: [PATCH v4 2/2] lavfi: Add blurdetect filter --- doc/filters.texi | 52 +++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_blurdetect.c | 394 ++++++++++++++++++++++++++++++++++++ 4 files changed, 448 insertions(+) create mode 100644 libavfilter/vf_blurdetect.c diff --git a/doc/filters.texi b/doc/filters.texi index 636c80dbff..c6d4537804 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -7990,6 +7990,58 @@ tblend=all_mode=grainextract @subsection Commands This filter supports same @ref{commands} as options. +@anchor{blurdetect} +@section blurdetect + +Determines blurriness of frames without altering the input frames. + +Based on Marziliano, Pina, et al. "A no-reference perceptual blur metric." +Allows for a block-based abbreviation. + +The filter accepts the following options: + +@table @option +@item low +@item high +Set low and high threshold values used by the Canny thresholding +algorithm. + +The high threshold selects the "strong" edge pixels, which are then +connected through 8-connectivity with the "weak" edge pixels selected +by the low threshold. + +@var{low} and @var{high} threshold values must be chosen in the range +[0,1], and @var{low} should be lesser or equal to @var{high}. + +Default value for @var{low} is @code{20/255}, and default value for @var{high} +is @code{50/255}. + +@item radius +Define the radius to search around an edge pixel for local maxima. + +@item block_pct +Determine blurriness only for the most significant blocks, given in percentage. + +@item block_width +Determine blurriness for blocks of width @var{block_width}. If set to any value smaller 1, no blocks are used and the whole image is processed as one no matter of @var{block_height}. + +@item block_height +Determine blurriness for blocks of height @var{block_height}. If set to any value smaller 1, no blocks are used and the whole image is processed as one no matter of @var{block_width}. + +@item planes +Set planes to filter. Default is first only. +@end table + +@subsection Examples + +@itemize +@item +Determine blur for 80% of most significant 32x32 blocks: +@example +blurdetect=block_width=32:block_height=32:block_pct=80 +@end example +@end itemize + @section bm3d Denoise frames using Block-Matching 3D algorithm. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 006e59b2bd..6332a6f799 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -195,6 +195,7 @@ OBJS-$(CONFIG_BLACKDETECT_FILTER) += vf_blackdetect.o OBJS-$(CONFIG_BLACKFRAME_FILTER) += vf_blackframe.o OBJS-$(CONFIG_BLEND_FILTER) += vf_blend.o framesync.o OBJS-$(CONFIG_BLEND_VULKAN_FILTER) += vf_blend_vulkan.o framesync.o vulkan.o vulkan_filter.o +OBJS-$(CONFIG_BLURDETECT_FILTER) += vf_blurdetect.o edge_common.o OBJS-$(CONFIG_BM3D_FILTER) += vf_bm3d.o framesync.o OBJS-$(CONFIG_BOXBLUR_FILTER) += vf_boxblur.o boxblur.o OBJS-$(CONFIG_BOXBLUR_OPENCL_FILTER) += vf_avgblur_opencl.o opencl.o \ diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 9fbaaacf47..2667d153ad 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -183,6 +183,7 @@ extern const AVFilter ff_vf_blackdetect; extern const AVFilter ff_vf_blackframe; extern const AVFilter ff_vf_blend; extern const AVFilter ff_vf_blend_vulkan; +extern const AVFilter ff_vf_blurdetect; extern const AVFilter ff_vf_bm3d; extern const AVFilter ff_vf_boxblur; extern const AVFilter ff_vf_boxblur_opencl; diff --git a/libavfilter/vf_blurdetect.c b/libavfilter/vf_blurdetect.c new file mode 100644 index 0000000000..71e46fa962 --- /dev/null +++ b/libavfilter/vf_blurdetect.c @@ -0,0 +1,394 @@ +/* + * Copyright (c) 2021 Thilo Borgmann <thilo.borgmann _at_ mail.de> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * No-reference blurdetect filter + * + * Implementing: + * Marziliano, Pina, et al. "A no-reference perceptual blur metric." Proceedings. + * International conference on image processing. Vol. 3. IEEE, 2002. + * https://infoscience.epfl.ch/record/111802/files/14%20A%20no-reference%20perceptual%20blur%20metric.pdf + * + * @author Thilo Borgmann <thilo.borgmann _at_ mail.de> + */ + +#include "libavutil/avassert.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "libavutil/pixelutils.h" +#include "libavutil/motion_vector.h" +#include "avfilter.h" +#include "formats.h" +#include "internal.h" +#include "video.h" +#include "edge_common.h" + +static int comp(const void *a,const void *b) +{ + float x = *(float*)a; + float y = *(float*)b; + if (x > y) return 1; + if (x < y) return -1; + return 0; +} + +typedef struct BLRContext { + const AVClass *class; + + int hsub, vsub; + int nb_planes; + + float low, high; + uint8_t low_u8, high_u8; + int radius; // radius during local maxima detection + int block_pct; // percentage of "sharpest" blocks in the image to use for bluriness calculation + int block_width; // width for block abbreviation + int block_height; // height for block abbreviation + int planes; // number of planes to filter + + double blur_total; + uint64_t nb_frames; + + float *blks; + uint8_t *filterbuf; + uint8_t *tmpbuf; + uint16_t *gradients; + char *directions; +} BLRContext; + +#define OFFSET(x) offsetof(BLRContext, x) +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM +static const AVOption blr_options[] = { + { "high", "set high threshold", OFFSET(high), AV_OPT_TYPE_FLOAT, {.dbl=30/255.}, 0, 1, FLAGS }, + { "low", "set low threshold", OFFSET(low), AV_OPT_TYPE_FLOAT, {.dbl=15/255.}, 0, 1, FLAGS }, + { "radius", "search radius for maxima detection", OFFSET(radius), AV_OPT_TYPE_INT, {.i64=50}, 1, 100, FLAGS }, + { "block_pct", "block pooling threshold when calculating blurriness", OFFSET(block_pct), AV_OPT_TYPE_INT, {.i64=80}, 1, 100, FLAGS }, + { "block_width", "block size for block-based abbreviation of blurriness", OFFSET(block_width), AV_OPT_TYPE_INT, {.i64=-1}, -1, INT_MAX, FLAGS }, + { "block_height", "block size for block-based abbreviation of blurriness", OFFSET(block_height), AV_OPT_TYPE_INT, {.i64=-1}, -1, INT_MAX, FLAGS }, + { "planes", "set planes to filter", OFFSET(planes), AV_OPT_TYPE_INT, {.i64=1}, 0, 15, FLAGS }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(blr); + +static av_cold int blr_init(AVFilterContext *ctx) +{ + BLRContext *blr = ctx->priv; + + blr->low_u8 = blr->low * 255. + .5; + blr->high_u8 = blr->high * 255. + .5; + + return 0; +} + +static int blr_config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + BLRContext *blr = ctx->priv; + const int bufsize = inlink->w * inlink->h; + const AVPixFmtDescriptor *pix_desc; + + pix_desc = av_pix_fmt_desc_get(inlink->format); + blr->hsub = pix_desc->log2_chroma_w; + blr->vsub = pix_desc->log2_chroma_h; + blr->nb_planes = av_pix_fmt_count_planes(inlink->format); + + if (blr->block_width < 1 || blr->block_height < 1) { + blr->block_width = inlink->w; + blr->block_height = inlink->h; + } + + blr->tmpbuf = av_malloc(bufsize); + blr->filterbuf = av_malloc(bufsize); + blr->gradients = av_calloc(bufsize, sizeof(*blr->gradients)); + blr->directions = av_malloc(bufsize); + blr->blks = av_calloc((inlink->w / blr->block_width) * (inlink->h / blr->block_height), + sizeof(*blr->blks)); + + if (!blr->tmpbuf || !blr->filterbuf || !blr->gradients || + !blr->directions || !blr->blks) + return AVERROR(ENOMEM); + + return 0; +} + +// edge width is defined as the distance between surrounding maxima of the edge pixel +static float edge_width(BLRContext *blr, int i, int j, int8_t dir, int w, int h, + int edge, const uint8_t *src, int src_linesize) +{ + float width = 0; + int dX, dY; + int sign; + int tmp; + int p1; + int p2; + int k, x, y; + int edge1; + int edge2; + float luma1 = 0.0; // average luma difference per edge pixel + float luma2 = 0.0; + int radius = blr->radius; + + switch(dir) + { + case DIRECTION_HORIZONTAL: dX = 1; dY = 0; break; + case DIRECTION_VERTICAL: dX = 0; dY = 1; break; + case DIRECTION_45UP: dX = 1; dY = -1; break; + case DIRECTION_45DOWN: dX = 1; dY = 1; break; + } + if (dir == DIRECTION_HORIZONTAL) return 0; + + // determines if search in direction dX/dY is looking for a maximum or minimum + sign = src[j * src_linesize + i] > src[(j - dY) * src_linesize + i - dX] ? 1 : -1; + + // search in -(dX/dY) direction + for (k = 0; k < radius; k++) + { + x = i - k*dX; + y = j - k*dY; + p1 = y * src_linesize + x; + x -= dX; + y -= dY; + p2 = y * src_linesize + x; + if (x < 0 || x >= w || y < 0 || y >= h) + return 0; + + tmp = (src[p1] - src[p2]) * sign; + + if (tmp <= 0) // local maximum found + break; + + luma1 += tmp; + } + if (k > 0) luma1 /= k; + edge1 = k; + width += k; + + // search in +(dX/dY) direction + for (k = 0; k < radius; k++) + { + x = i + k * dX; + y = j + k * dY; + p1 = y * src_linesize + x; + x += dX; + y += dY; + p2 = y * src_linesize + x; + if (x < 0 || x >= w || y < 0 || y >= h) + return 0; + + tmp = (src[p1] - src[p2]) * sign; + + if (tmp >= 0) // local maximum found + break; + + luma2 -= tmp; + } + if (k > 0) luma2 /= k; + edge2 = k; + width += k; + + // for 45 degree directions approximate edge width in pixel units: 0.7 ~= sqrt(2)/2 + if (dir == DIRECTION_45UP || dir == DIRECTION_45DOWN) + width *= 0.7; + + return width; +} + +static float calculate_blur(BLRContext *blr, int w, int h, int hsub, int vsub, + uint8_t* dir, int dir_linesize, + uint8_t* dst, int dst_linesize, + uint8_t* src, int src_linesize) +{ + float total_width = 0.0; + int block_count; + double block_total_width; + + int i, j; + int blkcnt = 0; + + float *blks = blr->blks; + float block_pool_threshold = blr->block_pct / 100.0; + + int block_width = AV_CEIL_RSHIFT(blr->block_width, hsub); + int block_height = AV_CEIL_RSHIFT(blr->block_height, vsub); + int brows = h / block_height; + int bcols = w / block_width; + + for (int blkj = 0; blkj < brows; blkj++) { + for (int blki = 0; blki < bcols; blki++) { + block_total_width = 0.0; + block_count = 0; + for (int inj = 0; inj < block_height; inj++) { + for (int ini = 0; ini < block_width; ini++) { + i = blki * block_width + ini; + j = blkj * block_height + inj; + + if (dst[j * dst_linesize + i] > 0) { + float width = edge_width(blr, i, j, dir[j*dir_linesize+i], + w, h, dst[j*dst_linesize+i], + src, src_linesize); + if (width > 0.001) { // throw away zeros + block_count++; + block_total_width += width; + } + } + } + } + // if not enough edge pixels in a block, consider it smooth + if (block_total_width >= 2) { + blks[blkcnt] = block_total_width / block_count; + blkcnt++; + } + } + } + + // simple block pooling by sorting and keeping the sharper blocks + qsort(blks, blkcnt, sizeof(*blks), comp); + blkcnt = ceil(blkcnt * block_pool_threshold); + for (int i = 0; i < blkcnt; i++) { + total_width += blks[i]; + } + + return total_width / blkcnt; +} + +static void set_meta(AVDictionary **metadata, const char *key, float d) +{ + char value[128]; + snprintf(value, sizeof(value), "%f", d); + av_dict_set(metadata, key, value, 0); +} + +static int blr_filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx = inlink->dst; + BLRContext *blr = ctx->priv; + AVFilterLink *outlink = ctx->outputs[0]; + + const int inw = inlink->w; + const int inh = inlink->h; + + uint8_t *tmpbuf = blr->tmpbuf; + uint8_t *filterbuf = blr->filterbuf; + uint16_t *gradients = blr->gradients; + int8_t *directions = blr->directions; + + float blur = 0.0f; + int nplanes = 0; + AVDictionary **metadata; + metadata = &in->metadata; + + for (int plane = 0; plane < blr->nb_planes; plane++) { + int hsub = plane == 1 || plane == 2 ? blr->hsub : 0; + int vsub = plane == 1 || plane == 2 ? blr->vsub : 0; + int w = AV_CEIL_RSHIFT(inw, hsub); + int h = AV_CEIL_RSHIFT(inh, vsub); + + if (!((1 << plane) & blr->planes)) + continue; + + nplanes++; + + // gaussian filter to reduce noise + ff_gaussian_blur(w, h, + filterbuf, w, + in->data[plane], in->linesize[plane]); + + // compute the 16-bits gradients and directions for the next step + ff_sobel(w, h, gradients, w, directions, w, filterbuf, w); + + // non_maximum_suppression() will actually keep & clip what's necessary and + // ignore the rest, so we need a clean output buffer + memset(tmpbuf, 0, inw * inh); + ff_non_maximum_suppression(w, h, tmpbuf, w, directions, w, gradients, w); + + + // keep high values, or low values surrounded by high values + ff_double_threshold(blr->low_u8, blr->high_u8, w, h, + tmpbuf, w, tmpbuf, w); + + blur += calculate_blur(blr, w, h, hsub, vsub, directions, w, + tmpbuf, w, filterbuf, w); + } + + if (nplanes) + blur /= nplanes; + + blr->blur_total += blur; + + // write stats + av_log(ctx, AV_LOG_VERBOSE, "blur: %.7f\n", blur); + + set_meta(metadata, "lavfi.blur", blur); + + blr->nb_frames++; + + return ff_filter_frame(outlink, in); +} + +static av_cold void blr_uninit(AVFilterContext *ctx) +{ + BLRContext *blr = ctx->priv; + + if (blr->nb_frames > 0) { + int nb_frames = blr->nb_frames; + av_log(ctx, AV_LOG_INFO, "blur mean: %.7f\n", + blr->blur_total / nb_frames); + } + + av_freep(&blr->tmpbuf); + av_freep(&blr->filterbuf); + av_freep(&blr->gradients); + av_freep(&blr->directions); + av_freep(&blr->blks); +} + +static const enum AVPixelFormat pix_fmts[] = { AV_PIX_FMT_YUV420P, AV_PIX_FMT_GRAY8, AV_PIX_FMT_NONE }; + +static const AVFilterPad blr_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = blr_config_input, + .filter_frame = blr_filter_frame, + }, +}; + +static const AVFilterPad blr_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +AVFilter ff_vf_blurdetect = { + .name = "blurdetect", + .description = NULL_IF_CONFIG_SMALL("Blurdetect filter."), + .priv_size = sizeof(BLRContext), + .init = blr_init, + .uninit = blr_uninit, + FILTER_PIXFMTS_ARRAY(pix_fmts), + FILTER_INPUTS(blr_inputs), + FILTER_OUTPUTS(blr_outputs), + .priv_class = &blr_class, + .flags = AVFILTER_FLAG_METADATA_ONLY, +}; + -- 2.20.1 (Apple Git-117) [-- Attachment #3: Type: text/plain, Size: 251 bytes --] _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [FFmpeg-devel] [PATCH 2/2] lavfi: Add blurriness filter 2022-04-05 8:56 ` Thilo Borgmann @ 2022-04-14 8:08 ` Thilo Borgmann 2022-04-23 13:32 ` Thilo Borgmann 0 siblings, 1 reply; 10+ messages in thread From: Thilo Borgmann @ 2022-04-14 8:08 UTC (permalink / raw) To: ffmpeg-devel Am 05.04.22 um 10:56 schrieb Thilo Borgmann: > Hi, > >>> v3 updated to current HEAD. >>> >>> Named blurdetect filter now. >>> Minor fixes on allocation and removed -f option. >>> >> >> Please make this per plane filtering, with default to measure only first >> plane. > > done in v4. > > (Will add Changelog, version.h and fate test once the filter itself looks ok) Ping. -Thilo _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [FFmpeg-devel] [PATCH 2/2] lavfi: Add blurriness filter 2022-04-14 8:08 ` Thilo Borgmann @ 2022-04-23 13:32 ` Thilo Borgmann 2022-04-24 17:28 ` Thilo Borgmann 0 siblings, 1 reply; 10+ messages in thread From: Thilo Borgmann @ 2022-04-23 13:32 UTC (permalink / raw) To: ffmpeg-devel [-- Attachment #1: Type: text/plain, Size: 379 bytes --] Hi, >>>> v3 updated to current HEAD. >>>> >>>> Named blurdetect filter now. >>>> Minor fixes on allocation and removed -f option. >>>> >>> >>> Please make this per plane filtering, with default to measure only first >>> plane. >> >> done in v4. >> >> (Will add Changelog, version.h and fate test once the filter itself looks ok) > > Ping. v5 according to IRC comments. -Thilo [-- Attachment #2: v5-0002-lavfi-Add-blurdetect-filter.patch --] [-- Type: text/plain, Size: 17036 bytes --] From be0bed07d36d957caf328f91a6cd0402c27fb6b4 Mon Sep 17 00:00:00 2001 From: Thilo Borgmann <thilo.borgmann@mail.de> Date: Sat, 23 Apr 2022 15:29:36 +0200 Subject: [PATCH v5 2/2] lavfi: Add blurdetect filter --- doc/filters.texi | 52 +++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_blurdetect.c | 395 ++++++++++++++++++++++++++++++++++++ 4 files changed, 449 insertions(+) create mode 100644 libavfilter/vf_blurdetect.c diff --git a/doc/filters.texi b/doc/filters.texi index 636c80dbff..c6d4537804 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -7990,6 +7990,58 @@ tblend=all_mode=grainextract @subsection Commands This filter supports same @ref{commands} as options. +@anchor{blurdetect} +@section blurdetect + +Determines blurriness of frames without altering the input frames. + +Based on Marziliano, Pina, et al. "A no-reference perceptual blur metric." +Allows for a block-based abbreviation. + +The filter accepts the following options: + +@table @option +@item low +@item high +Set low and high threshold values used by the Canny thresholding +algorithm. + +The high threshold selects the "strong" edge pixels, which are then +connected through 8-connectivity with the "weak" edge pixels selected +by the low threshold. + +@var{low} and @var{high} threshold values must be chosen in the range +[0,1], and @var{low} should be lesser or equal to @var{high}. + +Default value for @var{low} is @code{20/255}, and default value for @var{high} +is @code{50/255}. + +@item radius +Define the radius to search around an edge pixel for local maxima. + +@item block_pct +Determine blurriness only for the most significant blocks, given in percentage. + +@item block_width +Determine blurriness for blocks of width @var{block_width}. If set to any value smaller 1, no blocks are used and the whole image is processed as one no matter of @var{block_height}. + +@item block_height +Determine blurriness for blocks of height @var{block_height}. If set to any value smaller 1, no blocks are used and the whole image is processed as one no matter of @var{block_width}. + +@item planes +Set planes to filter. Default is first only. +@end table + +@subsection Examples + +@itemize +@item +Determine blur for 80% of most significant 32x32 blocks: +@example +blurdetect=block_width=32:block_height=32:block_pct=80 +@end example +@end itemize + @section bm3d Denoise frames using Block-Matching 3D algorithm. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 006e59b2bd..6332a6f799 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -195,6 +195,7 @@ OBJS-$(CONFIG_BLACKDETECT_FILTER) += vf_blackdetect.o OBJS-$(CONFIG_BLACKFRAME_FILTER) += vf_blackframe.o OBJS-$(CONFIG_BLEND_FILTER) += vf_blend.o framesync.o OBJS-$(CONFIG_BLEND_VULKAN_FILTER) += vf_blend_vulkan.o framesync.o vulkan.o vulkan_filter.o +OBJS-$(CONFIG_BLURDETECT_FILTER) += vf_blurdetect.o edge_common.o OBJS-$(CONFIG_BM3D_FILTER) += vf_bm3d.o framesync.o OBJS-$(CONFIG_BOXBLUR_FILTER) += vf_boxblur.o boxblur.o OBJS-$(CONFIG_BOXBLUR_OPENCL_FILTER) += vf_avgblur_opencl.o opencl.o \ diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 9fbaaacf47..2667d153ad 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -183,6 +183,7 @@ extern const AVFilter ff_vf_blackdetect; extern const AVFilter ff_vf_blackframe; extern const AVFilter ff_vf_blend; extern const AVFilter ff_vf_blend_vulkan; +extern const AVFilter ff_vf_blurdetect; extern const AVFilter ff_vf_bm3d; extern const AVFilter ff_vf_boxblur; extern const AVFilter ff_vf_boxblur_opencl; diff --git a/libavfilter/vf_blurdetect.c b/libavfilter/vf_blurdetect.c new file mode 100644 index 0000000000..c74f45ccb8 --- /dev/null +++ b/libavfilter/vf_blurdetect.c @@ -0,0 +1,395 @@ +/* + * Copyright (c) 2021 Thilo Borgmann <thilo.borgmann _at_ mail.de> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * No-reference blurdetect filter + * + * Implementing: + * Marziliano, Pina, et al. "A no-reference perceptual blur metric." Proceedings. + * International conference on image processing. Vol. 3. IEEE, 2002. + * https://infoscience.epfl.ch/record/111802/files/14%20A%20no-reference%20perceptual%20blur%20metric.pdf + * + * @author Thilo Borgmann <thilo.borgmann _at_ mail.de> + */ + +#include "libavutil/avassert.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "libavutil/pixelutils.h" +#include "libavutil/motion_vector.h" +#include "libavutil/qsort.h" +#include "avfilter.h" +#include "formats.h" +#include "internal.h" +#include "video.h" +#include "edge_common.h" + +static int comp(const float *a,const float *b) +{ + return FFDIFFSIGN(*a, *b); +} + +typedef struct BLRContext { + const AVClass *class; + + int hsub, vsub; + int nb_planes; + + float low, high; + uint8_t low_u8, high_u8; + int radius; // radius during local maxima detection + int block_pct; // percentage of "sharpest" blocks in the image to use for bluriness calculation + int block_width; // width for block abbreviation + int block_height; // height for block abbreviation + int planes; // number of planes to filter + + double blur_total; + uint64_t nb_frames; + + float *blks; + uint8_t *filterbuf; + uint8_t *tmpbuf; + uint16_t *gradients; + char *directions; +} BLRContext; + +#define OFFSET(x) offsetof(BLRContext, x) +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM +static const AVOption blurdetect_options[] = { + { "high", "set high threshold", OFFSET(high), AV_OPT_TYPE_FLOAT, {.dbl=30/255.}, 0, 1, FLAGS }, + { "low", "set low threshold", OFFSET(low), AV_OPT_TYPE_FLOAT, {.dbl=15/255.}, 0, 1, FLAGS }, + { "radius", "search radius for maxima detection", OFFSET(radius), AV_OPT_TYPE_INT, {.i64=50}, 1, 100, FLAGS }, + { "block_pct", "block pooling threshold when calculating blurriness", OFFSET(block_pct), AV_OPT_TYPE_INT, {.i64=80}, 1, 100, FLAGS }, + { "block_width", "block size for block-based abbreviation of blurriness", OFFSET(block_width), AV_OPT_TYPE_INT, {.i64=-1}, -1, INT_MAX, FLAGS }, + { "block_height", "block size for block-based abbreviation of blurriness", OFFSET(block_height), AV_OPT_TYPE_INT, {.i64=-1}, -1, INT_MAX, FLAGS }, + { "planes", "set planes to filter", OFFSET(planes), AV_OPT_TYPE_INT, {.i64=1}, 0, 15, FLAGS }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(blurdetect); + +static av_cold int blurdetect_init(AVFilterContext *ctx) +{ + BLRContext *s = ctx->priv; + + s->low_u8 = s->low * 255. + .5; + s->high_u8 = s->high * 255. + .5; + + return 0; +} + +static int blurdetect_config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + BLRContext *s = ctx->priv; + const int bufsize = inlink->w * inlink->h; + const AVPixFmtDescriptor *pix_desc; + + pix_desc = av_pix_fmt_desc_get(inlink->format); + s->hsub = pix_desc->log2_chroma_w; + s->vsub = pix_desc->log2_chroma_h; + s->nb_planes = av_pix_fmt_count_planes(inlink->format); + + if (s->block_width < 1 || s->block_height < 1) { + s->block_width = inlink->w; + s->block_height = inlink->h; + } + + s->tmpbuf = av_malloc(bufsize); + s->filterbuf = av_malloc(bufsize); + s->gradients = av_calloc(bufsize, sizeof(*s->gradients)); + s->directions = av_malloc(bufsize); + s->blks = av_calloc((inlink->w / s->block_width) * (inlink->h / s->block_height), + sizeof(*s->blks)); + + if (!s->tmpbuf || !s->filterbuf || !s->gradients || !s->directions || !s->blks) + return AVERROR(ENOMEM); + + return 0; +} + +// edge width is defined as the distance between surrounding maxima of the edge pixel +static float edge_width(BLRContext *blr, int i, int j, int8_t dir, int w, int h, + int edge, const uint8_t *src, int src_linesize) +{ + float width = 0; + int dX, dY; + int sign; + int tmp; + int p1; + int p2; + int k, x, y; + int edge1; + int edge2; + float luma1 = 0.0; // average luma difference per edge pixel + float luma2 = 0.0; + int radius = blr->radius; + + switch(dir) { + case DIRECTION_HORIZONTAL: dX = 1; dY = 0; break; + case DIRECTION_VERTICAL: dX = 0; dY = 1; break; + case DIRECTION_45UP: dX = 1; dY = -1; break; + case DIRECTION_45DOWN: dX = 1; dY = 1; break; + } + if (dir == DIRECTION_HORIZONTAL) return 0; + + // determines if search in direction dX/dY is looking for a maximum or minimum + sign = src[j * src_linesize + i] > src[(j - dY) * src_linesize + i - dX] ? 1 : -1; + + // search in -(dX/dY) direction + for (k = 0; k < radius; k++) { + x = i - k*dX; + y = j - k*dY; + p1 = y * src_linesize + x; + x -= dX; + y -= dY; + p2 = y * src_linesize + x; + if (x < 0 || x >= w || y < 0 || y >= h) + return 0; + + tmp = (src[p1] - src[p2]) * sign; + + if (tmp <= 0) // local maximum found + break; + + luma1 += tmp; + } + if (k > 0) luma1 /= k; + edge1 = k; + width += k; + + // search in +(dX/dY) direction + for (k = 0; k < radius; k++) { + x = i + k * dX; + y = j + k * dY; + p1 = y * src_linesize + x; + x += dX; + y += dY; + p2 = y * src_linesize + x; + if (x < 0 || x >= w || y < 0 || y >= h) + return 0; + + tmp = (src[p1] - src[p2]) * sign; + + if (tmp >= 0) // local maximum found + break; + + luma2 -= tmp; + } + if (k > 0) luma2 /= k; + edge2 = k; + width += k; + + // for 45 degree directions approximate edge width in pixel units: 0.7 ~= sqrt(2)/2 + if (dir == DIRECTION_45UP || dir == DIRECTION_45DOWN) + width *= 0.7; + + return width; +} + +static float calculate_blur(BLRContext *s, int w, int h, int hsub, int vsub, + uint8_t* dir, int dir_linesize, + uint8_t* dst, int dst_linesize, + uint8_t* src, int src_linesize) +{ + float total_width = 0.0; + int block_count; + double block_total_width; + + int i, j; + int blkcnt = 0; + + float *blks = s->blks; + float block_pool_threshold = s->block_pct / 100.0; + + int block_width = AV_CEIL_RSHIFT(s->block_width, hsub); + int block_height = AV_CEIL_RSHIFT(s->block_height, vsub); + int brows = h / block_height; + int bcols = w / block_width; + + for (int blkj = 0; blkj < brows; blkj++) { + for (int blki = 0; blki < bcols; blki++) { + block_total_width = 0.0; + block_count = 0; + for (int inj = 0; inj < block_height; inj++) { + for (int ini = 0; ini < block_width; ini++) { + i = blki * block_width + ini; + j = blkj * block_height + inj; + + if (dst[j * dst_linesize + i] > 0) { + float width = edge_width(s, i, j, dir[j*dir_linesize+i], + w, h, dst[j*dst_linesize+i], + src, src_linesize); + if (width > 0.001) { // throw away zeros + block_count++; + block_total_width += width; + } + } + } + } + // if not enough edge pixels in a block, consider it smooth + if (block_total_width >= 2) { + blks[blkcnt] = block_total_width / block_count; + blkcnt++; + } + } + } + + // simple block pooling by sorting and keeping the sharper blocks + AV_QSORT(blks, blkcnt, float, comp); + blkcnt = ceil(blkcnt * block_pool_threshold); + for (int i = 0; i < blkcnt; i++) { + total_width += blks[i]; + } + + return total_width / blkcnt; +} + +static void set_meta(AVDictionary **metadata, const char *key, float d) +{ + char value[128]; + snprintf(value, sizeof(value), "%f", d); + av_dict_set(metadata, key, value, 0); +} + +static int blurdetect_filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx = inlink->dst; + BLRContext *s = ctx->priv; + AVFilterLink *outlink = ctx->outputs[0]; + + const int inw = inlink->w; + const int inh = inlink->h; + + uint8_t *tmpbuf = s->tmpbuf; + uint8_t *filterbuf = s->filterbuf; + uint16_t *gradients = s->gradients; + int8_t *directions = s->directions; + + float blur = 0.0f; + int nplanes = 0; + AVDictionary **metadata; + metadata = &in->metadata; + + for (int plane = 0; plane < s->nb_planes; plane++) { + int hsub = plane == 1 || plane == 2 ? s->hsub : 0; + int vsub = plane == 1 || plane == 2 ? s->vsub : 0; + int w = AV_CEIL_RSHIFT(inw, hsub); + int h = AV_CEIL_RSHIFT(inh, vsub); + + if (!((1 << plane) & s->planes)) + continue; + + nplanes++; + + // gaussian filter to reduce noise + ff_gaussian_blur(w, h, + filterbuf, w, + in->data[plane], in->linesize[plane]); + + // compute the 16-bits gradients and directions for the next step + ff_sobel(w, h, gradients, w, directions, w, filterbuf, w); + + // non_maximum_suppression() will actually keep & clip what's necessary and + // ignore the rest, so we need a clean output buffer + memset(tmpbuf, 0, inw * inh); + ff_non_maximum_suppression(w, h, tmpbuf, w, directions, w, gradients, w); + + + // keep high values, or low values surrounded by high values + ff_double_threshold(s->low_u8, s->high_u8, w, h, + tmpbuf, w, tmpbuf, w); + + blur += calculate_blur(s, w, h, hsub, vsub, directions, w, + tmpbuf, w, filterbuf, w); + } + + if (nplanes) + blur /= nplanes; + + s->blur_total += blur; + + // write stats + av_log(ctx, AV_LOG_VERBOSE, "blur: %.7f\n", blur); + + set_meta(metadata, "lavfi.blur", blur); + + s->nb_frames = inlink->frame_count_in; + + return ff_filter_frame(outlink, in); +} + +static av_cold void blurdetect_uninit(AVFilterContext *ctx) +{ + BLRContext *s = ctx->priv; + + if (s->nb_frames > 0) { + av_log(ctx, AV_LOG_INFO, "blur mean: %.7f\n", + s->blur_total / s->nb_frames); + } + + av_freep(&s->tmpbuf); + av_freep(&s->filterbuf); + av_freep(&s->gradients); + av_freep(&s->directions); + av_freep(&s->blks); +} + +static const enum AVPixelFormat pix_fmts[] = { + AV_PIX_FMT_GRAY8, + AV_PIX_FMT_GBRP, AV_PIX_FMT_GBRAP, + AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV444P, AV_PIX_FMT_YUV440P, + AV_PIX_FMT_YUV411P, AV_PIX_FMT_YUV410P, + AV_PIX_FMT_YUVJ440P, AV_PIX_FMT_YUVJ411P, AV_PIX_FMT_YUVJ420P, + AV_PIX_FMT_YUVJ422P, AV_PIX_FMT_YUVJ444P, + AV_PIX_FMT_YUVA444P, AV_PIX_FMT_YUVA422P, AV_PIX_FMT_YUVA420P, + AV_PIX_FMT_NONE }; + +static const AVFilterPad blurdetect_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = blurdetect_config_input, + .filter_frame = blurdetect_filter_frame, + }, +}; + +static const AVFilterPad blurdetect_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +const AVFilter ff_vf_blurdetect = { + .name = "blurdetect", + .description = NULL_IF_CONFIG_SMALL("Blurdetect filter."), + .priv_size = sizeof(BLRContext), + .init = blurdetect_init, + .uninit = blurdetect_uninit, + FILTER_PIXFMTS_ARRAY(pix_fmts), + FILTER_INPUTS(blurdetect_inputs), + FILTER_OUTPUTS(blurdetect_outputs), + .priv_class = &blurdetect_class, + .flags = AVFILTER_FLAG_METADATA_ONLY, +}; + -- 2.20.1 (Apple Git-117) [-- Attachment #3: Type: text/plain, Size: 251 bytes --] _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [FFmpeg-devel] [PATCH 2/2] lavfi: Add blurriness filter 2022-04-23 13:32 ` Thilo Borgmann @ 2022-04-24 17:28 ` Thilo Borgmann 2022-04-24 22:03 ` Thilo Borgmann 0 siblings, 1 reply; 10+ messages in thread From: Thilo Borgmann @ 2022-04-24 17:28 UTC (permalink / raw) To: ffmpeg-devel [-- Attachment #1: Type: text/plain, Size: 561 bytes --] Am 23.04.22 um 15:32 schrieb Thilo Borgmann: > Hi, > >>>>> v3 updated to current HEAD. >>>>> >>>>> Named blurdetect filter now. >>>>> Minor fixes on allocation and removed -f option. >>>>> >>>> >>>> Please make this per plane filtering, with default to measure only first >>>> plane. >>> >>> done in v4. >>> >>> (Will add Changelog, version.h and fate test once the filter itself looks ok) >> >> Ping. > > v5 according to IRC comments. v6 (1/2 and 2/2) according to IRC comments. LGTM'd there, applying soon. Added Changelog, version.h, FATE. Thanks! Thilo [-- Attachment #2: v6-0001-lafi-vf_edgedetect-Move-some-common-functions-int.patch --] [-- Type: text/plain, Size: 21266 bytes --] From c90120255bff305ce76c7677c9063d94c3bb99f7 Mon Sep 17 00:00:00 2001 From: Thilo Borgmann <thilo.borgmann@mail.de> Date: Tue, 30 Nov 2021 00:16:52 +0100 Subject: [PATCH v6 1/2] lafi/vf_edgedetect: Move some common functions into seperate file --- libavfilter/Makefile | 2 +- libavfilter/edge_common.c | 181 +++++++++++++++++++++++++++++++++++ libavfilter/edge_common.h | 107 +++++++++++++++++++++ libavfilter/vf_edgedetect.c | 183 ++---------------------------------- 4 files changed, 296 insertions(+), 177 deletions(-) create mode 100644 libavfilter/edge_common.c create mode 100644 libavfilter/edge_common.h diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 32521a4836..38ca379e5a 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -266,7 +266,7 @@ OBJS-$(CONFIG_DRAWBOX_FILTER) += vf_drawbox.o OBJS-$(CONFIG_DRAWGRAPH_FILTER) += f_drawgraph.o OBJS-$(CONFIG_DRAWGRID_FILTER) += vf_drawbox.o OBJS-$(CONFIG_DRAWTEXT_FILTER) += vf_drawtext.o -OBJS-$(CONFIG_EDGEDETECT_FILTER) += vf_edgedetect.o +OBJS-$(CONFIG_EDGEDETECT_FILTER) += vf_edgedetect.o edge_common.o OBJS-$(CONFIG_ELBG_FILTER) += vf_elbg.o OBJS-$(CONFIG_ENTROPY_FILTER) += vf_entropy.o OBJS-$(CONFIG_EPX_FILTER) += vf_epx.o diff --git a/libavfilter/edge_common.c b/libavfilter/edge_common.c new file mode 100644 index 0000000000..d72e8521cd --- /dev/null +++ b/libavfilter/edge_common.c @@ -0,0 +1,181 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "edge_common.h" + +// Internal helper for ff_sobel() +static int get_rounded_direction(int gx, int gy) +{ + /* reference angles: + * tan( pi/8) = sqrt(2)-1 + * tan(3pi/8) = sqrt(2)+1 + * Gy/Gx is the tangent of the angle (theta), so Gy/Gx is compared against + * <ref-angle>, or more simply Gy against <ref-angle>*Gx + * + * Gx and Gy bounds = [-1020;1020], using 16-bit arithmetic: + * round((sqrt(2)-1) * (1<<16)) = 27146 + * round((sqrt(2)+1) * (1<<16)) = 158218 + */ + if (gx) { + int tanpi8gx, tan3pi8gx; + + if (gx < 0) + gx = -gx, gy = -gy; + gy *= (1 << 16); + tanpi8gx = 27146 * gx; + tan3pi8gx = 158218 * gx; + if (gy > -tan3pi8gx && gy < -tanpi8gx) return DIRECTION_45UP; + if (gy > -tanpi8gx && gy < tanpi8gx) return DIRECTION_HORIZONTAL; + if (gy > tanpi8gx && gy < tan3pi8gx) return DIRECTION_45DOWN; + } + return DIRECTION_VERTICAL; +} + +// Simple sobel operator to get rounded gradients +void ff_sobel(int w, int h, + uint16_t *dst, int dst_linesize, + int8_t *dir, int dir_linesize, + const uint8_t *src, int src_linesize) +{ + int i, j; + + for (j = 1; j < h - 1; j++) { + dst += dst_linesize; + dir += dir_linesize; + src += src_linesize; + for (i = 1; i < w - 1; i++) { + const int gx = + -1*src[-src_linesize + i-1] + 1*src[-src_linesize + i+1] + -2*src[ i-1] + 2*src[ i+1] + -1*src[ src_linesize + i-1] + 1*src[ src_linesize + i+1]; + const int gy = + -1*src[-src_linesize + i-1] + 1*src[ src_linesize + i-1] + -2*src[-src_linesize + i ] + 2*src[ src_linesize + i ] + -1*src[-src_linesize + i+1] + 1*src[ src_linesize + i+1]; + + dst[i] = FFABS(gx) + FFABS(gy); + dir[i] = get_rounded_direction(gx, gy); + } + } +} + +// Filters rounded gradients to drop all non-maxima +// Expects gradients generated by ff_sobel() +// Expects zero's destination buffer +void ff_non_maximum_suppression(int w, int h, + uint8_t *dst, int dst_linesize, + const int8_t *dir, int dir_linesize, + const uint16_t *src, int src_linesize) +{ + int i, j; + +#define COPY_MAXIMA(ay, ax, by, bx) do { \ + if (src[i] > src[(ay)*src_linesize + i+(ax)] && \ + src[i] > src[(by)*src_linesize + i+(bx)]) \ + dst[i] = av_clip_uint8(src[i]); \ +} while (0) + + for (j = 1; j < h - 1; j++) { + dst += dst_linesize; + dir += dir_linesize; + src += src_linesize; + for (i = 1; i < w - 1; i++) { + switch (dir[i]) { + case DIRECTION_45UP: COPY_MAXIMA( 1, -1, -1, 1); break; + case DIRECTION_45DOWN: COPY_MAXIMA(-1, -1, 1, 1); break; + case DIRECTION_HORIZONTAL: COPY_MAXIMA( 0, -1, 0, 1); break; + case DIRECTION_VERTICAL: COPY_MAXIMA(-1, 0, 1, 0); break; + } + } + } +} + +// Filter to keep all pixels > high, and keep all pixels > low where all surrounding pixels > high +void ff_double_threshold(int low, int high, int w, int h, + uint8_t *dst, int dst_linesize, + const uint8_t *src, int src_linesize) +{ + int i, j; + + for (j = 0; j < h; j++) { + for (i = 0; i < w; i++) { + if (src[i] > high) { + dst[i] = src[i]; + continue; + } + + if (!(!i || i == w - 1 || !j || j == h - 1) && + src[i] > low && + (src[-src_linesize + i-1] > high || + src[-src_linesize + i ] > high || + src[-src_linesize + i+1] > high || + src[ i-1] > high || + src[ i+1] > high || + src[ src_linesize + i-1] > high || + src[ src_linesize + i ] > high || + src[ src_linesize + i+1] > high)) + dst[i] = src[i]; + else + dst[i] = 0; + } + dst += dst_linesize; + src += src_linesize; + } +} + +// Applies gaussian blur, using 5x5 kernels, sigma = 1.4 +void ff_gaussian_blur(int w, int h, + uint8_t *dst, int dst_linesize, + const uint8_t *src, int src_linesize) +{ + int i, j; + + memcpy(dst, src, w); dst += dst_linesize; src += src_linesize; + memcpy(dst, src, w); dst += dst_linesize; src += src_linesize; + for (j = 2; j < h - 2; j++) { + dst[0] = src[0]; + dst[1] = src[1]; + for (i = 2; i < w - 2; i++) { + /* Gaussian mask of size 5x5 with sigma = 1.4 */ + dst[i] = ((src[-2*src_linesize + i-2] + src[2*src_linesize + i-2]) * 2 + + (src[-2*src_linesize + i-1] + src[2*src_linesize + i-1]) * 4 + + (src[-2*src_linesize + i ] + src[2*src_linesize + i ]) * 5 + + (src[-2*src_linesize + i+1] + src[2*src_linesize + i+1]) * 4 + + (src[-2*src_linesize + i+2] + src[2*src_linesize + i+2]) * 2 + + + (src[ -src_linesize + i-2] + src[ src_linesize + i-2]) * 4 + + (src[ -src_linesize + i-1] + src[ src_linesize + i-1]) * 9 + + (src[ -src_linesize + i ] + src[ src_linesize + i ]) * 12 + + (src[ -src_linesize + i+1] + src[ src_linesize + i+1]) * 9 + + (src[ -src_linesize + i+2] + src[ src_linesize + i+2]) * 4 + + + src[i-2] * 5 + + src[i-1] * 12 + + src[i ] * 15 + + src[i+1] * 12 + + src[i+2] * 5) / 159; + } + dst[i ] = src[i ]; + dst[i + 1] = src[i + 1]; + + dst += dst_linesize; + src += src_linesize; + } + memcpy(dst, src, w); dst += dst_linesize; src += src_linesize; + memcpy(dst, src, w); +} diff --git a/libavfilter/edge_common.h b/libavfilter/edge_common.h new file mode 100644 index 0000000000..87c143f2b8 --- /dev/null +++ b/libavfilter/edge_common.h @@ -0,0 +1,107 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * common functions for edge detection + */ + +#ifndef AVFILTER_EDGE_COMMON_H +#define AVFILTER_EDGE_COMMON_H + +#include "avfilter.h" + +/** + * @brief Rounded directions used in av_image_sobel() + */ +enum AVRoundedDirection { + DIRECTION_45UP, + DIRECTION_45DOWN, + DIRECTION_HORIZONTAL, + DIRECTION_VERTICAL, +}; + +/** + * Simple sobel operator to get rounded gradients + * + * @param w the width of the image in pixels + * @param h the height of the image in pixels + * @param dst data pointers to magnitude image + * @param dst_linesize linesizes for the magnitude image + * @param dir data pointers to direction image + * @param dir_linesize linesizes for the direction image + * @param src data pointers to source image + * @param src_linesize linesizes for the source image + */ +void ff_sobel(int w, int h, + uint16_t *dst, int dst_linesize, + int8_t *dir, int dir_linesize, + const uint8_t *src, int src_linesize); + +/** + * Filters rounded gradients to drop all non-maxima pixels in the magnitude image + * Expects gradients generated by av_image_sobel() + * Expects zero's in the destination buffer dst + * + * @param w the width of the image in pixels + * @param h the height of the image in pixels + * @param dst data pointers to magnitude image + * @param dst_linesize linesizes for the magnitude image + * @param dir data pointers to direction image + * @param dir_linesize linesizes for the direction image + * @param src data pointers to source image + * @param src_linesize linesizes for the source image + */ +void ff_non_maximum_suppression(int w, int h, + uint8_t *dst, int dst_linesize, + const int8_t *dir, int dir_linesize, + const uint16_t *src, int src_linesize); + +/** + * Filters all pixels in src to keep all pixels > high, + * and keep all pixels > low where all surrounding pixels > high + * + * @param low the low threshold value + * @param high the hegh threshold value + * @param w the width of the image in pixels + * @param h the height of the image in pixels + * @param dst data pointers to destination image + * @param dst_linesize linesizes for the destination image + * @param src data pointers to source image + * @param src_linesize linesizes for the source image + */ +void ff_double_threshold(int low, int high, int w, int h, + uint8_t *dst, int dst_linesize, + const uint8_t *src, int src_linesize); + +/** + * Applies gaussian blur. + * 5x5 kernels, sigma = 1.4 + * + * @param w the width of the image in pixels + * @param h the height of the image in pixels + * @param dst data pointers to destination image + * @param dst_linesize linesizes for the destination image + * @param src data pointers to source image + * @param src_linesize linesizes for the source image + */ +void ff_gaussian_blur(int w, int h, + uint8_t *dst, int dst_linesize, + const uint8_t *src, int src_linesize); + +#endif diff --git a/libavfilter/vf_edgedetect.c b/libavfilter/vf_edgedetect.c index 3eea34e325..90390ceb3e 100644 --- a/libavfilter/vf_edgedetect.c +++ b/libavfilter/vf_edgedetect.c @@ -32,6 +32,7 @@ #include "formats.h" #include "internal.h" #include "video.h" +#include "edge_common.h" #define PLANE_R 0x4 #define PLANE_G 0x1 @@ -139,176 +140,6 @@ static int config_props(AVFilterLink *inlink) return 0; } -static void gaussian_blur(AVFilterContext *ctx, int w, int h, - uint8_t *dst, int dst_linesize, - const uint8_t *src, int src_linesize) -{ - int i, j; - - memcpy(dst, src, w); dst += dst_linesize; src += src_linesize; - if (h > 1) { - memcpy(dst, src, w); dst += dst_linesize; src += src_linesize; - } - for (j = 2; j < h - 2; j++) { - dst[0] = src[0]; - if (w > 1) - dst[1] = src[1]; - for (i = 2; i < w - 2; i++) { - /* Gaussian mask of size 5x5 with sigma = 1.4 */ - dst[i] = ((src[-2*src_linesize + i-2] + src[2*src_linesize + i-2]) * 2 - + (src[-2*src_linesize + i-1] + src[2*src_linesize + i-1]) * 4 - + (src[-2*src_linesize + i ] + src[2*src_linesize + i ]) * 5 - + (src[-2*src_linesize + i+1] + src[2*src_linesize + i+1]) * 4 - + (src[-2*src_linesize + i+2] + src[2*src_linesize + i+2]) * 2 - - + (src[ -src_linesize + i-2] + src[ src_linesize + i-2]) * 4 - + (src[ -src_linesize + i-1] + src[ src_linesize + i-1]) * 9 - + (src[ -src_linesize + i ] + src[ src_linesize + i ]) * 12 - + (src[ -src_linesize + i+1] + src[ src_linesize + i+1]) * 9 - + (src[ -src_linesize + i+2] + src[ src_linesize + i+2]) * 4 - - + src[i-2] * 5 - + src[i-1] * 12 - + src[i ] * 15 - + src[i+1] * 12 - + src[i+2] * 5) / 159; - } - if (w > 2) - dst[i ] = src[i ]; - if (w > 3) - dst[i + 1] = src[i + 1]; - - dst += dst_linesize; - src += src_linesize; - } - if (h > 2) { - memcpy(dst, src, w); dst += dst_linesize; src += src_linesize; - } - if (h > 3) - memcpy(dst, src, w); -} - -enum { - DIRECTION_45UP, - DIRECTION_45DOWN, - DIRECTION_HORIZONTAL, - DIRECTION_VERTICAL, -}; - -static int get_rounded_direction(int gx, int gy) -{ - /* reference angles: - * tan( pi/8) = sqrt(2)-1 - * tan(3pi/8) = sqrt(2)+1 - * Gy/Gx is the tangent of the angle (theta), so Gy/Gx is compared against - * <ref-angle>, or more simply Gy against <ref-angle>*Gx - * - * Gx and Gy bounds = [-1020;1020], using 16-bit arithmetic: - * round((sqrt(2)-1) * (1<<16)) = 27146 - * round((sqrt(2)+1) * (1<<16)) = 158218 - */ - if (gx) { - int tanpi8gx, tan3pi8gx; - - if (gx < 0) - gx = -gx, gy = -gy; - gy *= (1 << 16); - tanpi8gx = 27146 * gx; - tan3pi8gx = 158218 * gx; - if (gy > -tan3pi8gx && gy < -tanpi8gx) return DIRECTION_45UP; - if (gy > -tanpi8gx && gy < tanpi8gx) return DIRECTION_HORIZONTAL; - if (gy > tanpi8gx && gy < tan3pi8gx) return DIRECTION_45DOWN; - } - return DIRECTION_VERTICAL; -} - -static void sobel(int w, int h, - uint16_t *dst, int dst_linesize, - int8_t *dir, int dir_linesize, - const uint8_t *src, int src_linesize) -{ - int i, j; - - for (j = 1; j < h - 1; j++) { - dst += dst_linesize; - dir += dir_linesize; - src += src_linesize; - for (i = 1; i < w - 1; i++) { - const int gx = - -1*src[-src_linesize + i-1] + 1*src[-src_linesize + i+1] - -2*src[ i-1] + 2*src[ i+1] - -1*src[ src_linesize + i-1] + 1*src[ src_linesize + i+1]; - const int gy = - -1*src[-src_linesize + i-1] + 1*src[ src_linesize + i-1] - -2*src[-src_linesize + i ] + 2*src[ src_linesize + i ] - -1*src[-src_linesize + i+1] + 1*src[ src_linesize + i+1]; - - dst[i] = FFABS(gx) + FFABS(gy); - dir[i] = get_rounded_direction(gx, gy); - } - } -} - -static void non_maximum_suppression(int w, int h, - uint8_t *dst, int dst_linesize, - const int8_t *dir, int dir_linesize, - const uint16_t *src, int src_linesize) -{ - int i, j; - -#define COPY_MAXIMA(ay, ax, by, bx) do { \ - if (src[i] > src[(ay)*src_linesize + i+(ax)] && \ - src[i] > src[(by)*src_linesize + i+(bx)]) \ - dst[i] = av_clip_uint8(src[i]); \ -} while (0) - - for (j = 1; j < h - 1; j++) { - dst += dst_linesize; - dir += dir_linesize; - src += src_linesize; - for (i = 1; i < w - 1; i++) { - switch (dir[i]) { - case DIRECTION_45UP: COPY_MAXIMA( 1, -1, -1, 1); break; - case DIRECTION_45DOWN: COPY_MAXIMA(-1, -1, 1, 1); break; - case DIRECTION_HORIZONTAL: COPY_MAXIMA( 0, -1, 0, 1); break; - case DIRECTION_VERTICAL: COPY_MAXIMA(-1, 0, 1, 0); break; - } - } - } -} - -static void double_threshold(int low, int high, int w, int h, - uint8_t *dst, int dst_linesize, - const uint8_t *src, int src_linesize) -{ - int i, j; - - for (j = 0; j < h; j++) { - for (i = 0; i < w; i++) { - if (src[i] > high) { - dst[i] = src[i]; - continue; - } - - if (!(!i || i == w - 1 || !j || j == h - 1) && - src[i] > low && - (src[-src_linesize + i-1] > high || - src[-src_linesize + i ] > high || - src[-src_linesize + i+1] > high || - src[ i-1] > high || - src[ i+1] > high || - src[ src_linesize + i-1] > high || - src[ src_linesize + i ] > high || - src[ src_linesize + i+1] > high)) - dst[i] = src[i]; - else - dst[i] = 0; - } - dst += dst_linesize; - src += src_linesize; - } -} - static void color_mix(int w, int h, uint8_t *dst, int dst_linesize, const uint8_t *src, int src_linesize) @@ -360,12 +191,12 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) } /* gaussian filter to reduce noise */ - gaussian_blur(ctx, width, height, - tmpbuf, width, - in->data[p], in->linesize[p]); + ff_gaussian_blur(width, height, + tmpbuf, width, + in->data[p], in->linesize[p]); /* compute the 16-bits gradients and directions for the next step */ - sobel(width, height, + ff_sobel(width, height, gradients, width, directions,width, tmpbuf, width); @@ -373,13 +204,13 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) /* non_maximum_suppression() will actually keep & clip what's necessary and * ignore the rest, so we need a clean output buffer */ memset(tmpbuf, 0, width * height); - non_maximum_suppression(width, height, + ff_non_maximum_suppression(width, height, tmpbuf, width, directions,width, gradients, width); /* keep high values, or low values surrounded by high values */ - double_threshold(edgedetect->low_u8, edgedetect->high_u8, + ff_double_threshold(edgedetect->low_u8, edgedetect->high_u8, width, height, out->data[p], out->linesize[p], tmpbuf, width); -- 2.20.1 (Apple Git-117) [-- Attachment #3: v6-0002-lavfi-Add-blurdetect-filter.patch --] [-- Type: text/plain, Size: 19168 bytes --] From 9179d43e0c00145e42142fe9dbd170cb3b563a8d Mon Sep 17 00:00:00 2001 From: Thilo Borgmann <thilo.borgmann@mail.de> Date: Sun, 24 Apr 2022 19:12:24 +0200 Subject: [PATCH v6 2/2] lavfi: Add blurdetect filter --- Changelog | 1 + doc/filters.texi | 52 +++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/version.h | 2 +- libavfilter/vf_blurdetect.c | 396 ++++++++++++++++++++ tests/fate/filter-video.mak | 3 + tests/ref/fate/filter-refcmp-blurdetect-yuv | 10 + 8 files changed, 465 insertions(+), 1 deletion(-) create mode 100644 libavfilter/vf_blurdetect.c create mode 100644 tests/ref/fate/filter-refcmp-blurdetect-yuv diff --git a/Changelog b/Changelog index 7a63e3d1ee..4d467eb741 100644 --- a/Changelog +++ b/Changelog @@ -13,6 +13,7 @@ version 5.1: - pixelize video filter - colormap video filter - colorchart video source filter +- blurdetect filter version 5.0: diff --git a/doc/filters.texi b/doc/filters.texi index c8699b9099..499f3adcd9 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -7997,6 +7997,58 @@ tblend=all_mode=grainextract @subsection Commands This filter supports same @ref{commands} as options. +@anchor{blurdetect} +@section blurdetect + +Determines blurriness of frames without altering the input frames. + +Based on Marziliano, Pina, et al. "A no-reference perceptual blur metric." +Allows for a block-based abbreviation. + +The filter accepts the following options: + +@table @option +@item low +@item high +Set low and high threshold values used by the Canny thresholding +algorithm. + +The high threshold selects the "strong" edge pixels, which are then +connected through 8-connectivity with the "weak" edge pixels selected +by the low threshold. + +@var{low} and @var{high} threshold values must be chosen in the range +[0,1], and @var{low} should be lesser or equal to @var{high}. + +Default value for @var{low} is @code{20/255}, and default value for @var{high} +is @code{50/255}. + +@item radius +Define the radius to search around an edge pixel for local maxima. + +@item block_pct +Determine blurriness only for the most significant blocks, given in percentage. + +@item block_width +Determine blurriness for blocks of width @var{block_width}. If set to any value smaller 1, no blocks are used and the whole image is processed as one no matter of @var{block_height}. + +@item block_height +Determine blurriness for blocks of height @var{block_height}. If set to any value smaller 1, no blocks are used and the whole image is processed as one no matter of @var{block_width}. + +@item planes +Set planes to filter. Default is first only. +@end table + +@subsection Examples + +@itemize +@item +Determine blur for 80% of most significant 32x32 blocks: +@example +blurdetect=block_width=32:block_height=32:block_pct=80 +@end example +@end itemize + @section bm3d Denoise frames using Block-Matching 3D algorithm. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 38ca379e5a..1db097b464 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -195,6 +195,7 @@ OBJS-$(CONFIG_BLACKDETECT_FILTER) += vf_blackdetect.o OBJS-$(CONFIG_BLACKFRAME_FILTER) += vf_blackframe.o OBJS-$(CONFIG_BLEND_FILTER) += vf_blend.o framesync.o OBJS-$(CONFIG_BLEND_VULKAN_FILTER) += vf_blend_vulkan.o framesync.o vulkan.o vulkan_filter.o +OBJS-$(CONFIG_BLURDETECT_FILTER) += vf_blurdetect.o edge_common.o OBJS-$(CONFIG_BM3D_FILTER) += vf_bm3d.o framesync.o OBJS-$(CONFIG_BOXBLUR_FILTER) += vf_boxblur.o boxblur.o OBJS-$(CONFIG_BOXBLUR_OPENCL_FILTER) += vf_avgblur_opencl.o opencl.o \ diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 36fa3ae8d7..2ad523fd0f 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -183,6 +183,7 @@ extern const AVFilter ff_vf_blackdetect; extern const AVFilter ff_vf_blackframe; extern const AVFilter ff_vf_blend; extern const AVFilter ff_vf_blend_vulkan; +extern const AVFilter ff_vf_blurdetect; extern const AVFilter ff_vf_bm3d; extern const AVFilter ff_vf_boxblur; extern const AVFilter ff_vf_boxblur_opencl; diff --git a/libavfilter/version.h b/libavfilter/version.h index 9add1658e5..8f1b16969a 100644 --- a/libavfilter/version.h +++ b/libavfilter/version.h @@ -31,7 +31,7 @@ #include "version_major.h" -#define LIBAVFILTER_VERSION_MINOR 36 +#define LIBAVFILTER_VERSION_MINOR 37 #define LIBAVFILTER_VERSION_MICRO 100 diff --git a/libavfilter/vf_blurdetect.c b/libavfilter/vf_blurdetect.c new file mode 100644 index 0000000000..2588dde3d9 --- /dev/null +++ b/libavfilter/vf_blurdetect.c @@ -0,0 +1,396 @@ +/* + * Copyright (c) 2021 Thilo Borgmann <thilo.borgmann _at_ mail.de> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * No-reference blurdetect filter + * + * Implementing: + * Marziliano, Pina, et al. "A no-reference perceptual blur metric." Proceedings. + * International conference on image processing. Vol. 3. IEEE, 2002. + * https://infoscience.epfl.ch/record/111802/files/14%20A%20no-reference%20perceptual%20blur%20metric.pdf + * + * @author Thilo Borgmann <thilo.borgmann _at_ mail.de> + */ + +#include "libavutil/avassert.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "libavutil/pixelutils.h" +#include "libavutil/motion_vector.h" +#include "libavutil/qsort.h" +#include "avfilter.h" +#include "formats.h" +#include "internal.h" +#include "video.h" +#include "edge_common.h" + +static int comp(const float *a,const float *b) +{ + return FFDIFFSIGN(*a, *b); +} + +typedef struct BLRContext { + const AVClass *class; + + int hsub, vsub; + int nb_planes; + + float low, high; + uint8_t low_u8, high_u8; + int radius; // radius during local maxima detection + int block_pct; // percentage of "sharpest" blocks in the image to use for bluriness calculation + int block_width; // width for block abbreviation + int block_height; // height for block abbreviation + int planes; // number of planes to filter + + double blur_total; + uint64_t nb_frames; + + float *blks; + uint8_t *filterbuf; + uint8_t *tmpbuf; + uint16_t *gradients; + char *directions; +} BLRContext; + +#define OFFSET(x) offsetof(BLRContext, x) +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM +static const AVOption blurdetect_options[] = { + { "high", "set high threshold", OFFSET(high), AV_OPT_TYPE_FLOAT, {.dbl=30/255.}, 0, 1, FLAGS }, + { "low", "set low threshold", OFFSET(low), AV_OPT_TYPE_FLOAT, {.dbl=15/255.}, 0, 1, FLAGS }, + { "radius", "search radius for maxima detection", OFFSET(radius), AV_OPT_TYPE_INT, {.i64=50}, 1, 100, FLAGS }, + { "block_pct", "block pooling threshold when calculating blurriness", OFFSET(block_pct), AV_OPT_TYPE_INT, {.i64=80}, 1, 100, FLAGS }, + { "block_width", "block size for block-based abbreviation of blurriness", OFFSET(block_width), AV_OPT_TYPE_INT, {.i64=-1}, -1, INT_MAX, FLAGS }, + { "block_height", "block size for block-based abbreviation of blurriness", OFFSET(block_height), AV_OPT_TYPE_INT, {.i64=-1}, -1, INT_MAX, FLAGS }, + { "planes", "set planes to filter", OFFSET(planes), AV_OPT_TYPE_INT, {.i64=1}, 0, 15, FLAGS }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(blurdetect); + +static av_cold int blurdetect_init(AVFilterContext *ctx) +{ + BLRContext *s = ctx->priv; + + s->low_u8 = s->low * 255. + .5; + s->high_u8 = s->high * 255. + .5; + + return 0; +} + +static int blurdetect_config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + BLRContext *s = ctx->priv; + const int bufsize = inlink->w * inlink->h; + const AVPixFmtDescriptor *pix_desc; + + pix_desc = av_pix_fmt_desc_get(inlink->format); + s->hsub = pix_desc->log2_chroma_w; + s->vsub = pix_desc->log2_chroma_h; + s->nb_planes = av_pix_fmt_count_planes(inlink->format); + + if (s->block_width < 1 || s->block_height < 1) { + s->block_width = inlink->w; + s->block_height = inlink->h; + } + + s->tmpbuf = av_malloc(bufsize); + s->filterbuf = av_malloc(bufsize); + s->gradients = av_calloc(bufsize, sizeof(*s->gradients)); + s->directions = av_malloc(bufsize); + s->blks = av_calloc((inlink->w / s->block_width) * (inlink->h / s->block_height), + sizeof(*s->blks)); + + if (!s->tmpbuf || !s->filterbuf || !s->gradients || !s->directions || !s->blks) + return AVERROR(ENOMEM); + + return 0; +} + +// edge width is defined as the distance between surrounding maxima of the edge pixel +static float edge_width(BLRContext *blr, int i, int j, int8_t dir, int w, int h, + int edge, const uint8_t *src, int src_linesize) +{ + float width = 0; + int dX, dY; + int sign; + int tmp; + int p1; + int p2; + int k, x, y; + int edge1; + int edge2; + float luma1 = 0.0; // average luma difference per edge pixel + float luma2 = 0.0; + int radius = blr->radius; + + switch(dir) { + case DIRECTION_HORIZONTAL: dX = 1; dY = 0; break; + case DIRECTION_VERTICAL: dX = 0; dY = 1; break; + case DIRECTION_45UP: dX = 1; dY = -1; break; + case DIRECTION_45DOWN: dX = 1; dY = 1; break; + } + if (dir == DIRECTION_HORIZONTAL) return 0; + + // determines if search in direction dX/dY is looking for a maximum or minimum + sign = src[j * src_linesize + i] > src[(j - dY) * src_linesize + i - dX] ? 1 : -1; + + // search in -(dX/dY) direction + for (k = 0; k < radius; k++) { + x = i - k*dX; + y = j - k*dY; + p1 = y * src_linesize + x; + x -= dX; + y -= dY; + p2 = y * src_linesize + x; + if (x < 0 || x >= w || y < 0 || y >= h) + return 0; + + tmp = (src[p1] - src[p2]) * sign; + + if (tmp <= 0) // local maximum found + break; + + luma1 += tmp; + } + if (k > 0) luma1 /= k; + edge1 = k; + width += k; + + // search in +(dX/dY) direction + for (k = 0; k < radius; k++) { + x = i + k * dX; + y = j + k * dY; + p1 = y * src_linesize + x; + x += dX; + y += dY; + p2 = y * src_linesize + x; + if (x < 0 || x >= w || y < 0 || y >= h) + return 0; + + tmp = (src[p1] - src[p2]) * sign; + + if (tmp >= 0) // local maximum found + break; + + luma2 -= tmp; + } + if (k > 0) luma2 /= k; + edge2 = k; + width += k; + + // for 45 degree directions approximate edge width in pixel units: 0.7 ~= sqrt(2)/2 + if (dir == DIRECTION_45UP || dir == DIRECTION_45DOWN) + width *= 0.7; + + return width; +} + +static float calculate_blur(BLRContext *s, int w, int h, int hsub, int vsub, + uint8_t* dir, int dir_linesize, + uint8_t* dst, int dst_linesize, + uint8_t* src, int src_linesize) +{ + float total_width = 0.0; + int block_count; + double block_total_width; + + int i, j; + int blkcnt = 0; + + float *blks = s->blks; + float block_pool_threshold = s->block_pct / 100.0; + + int block_width = AV_CEIL_RSHIFT(s->block_width, hsub); + int block_height = AV_CEIL_RSHIFT(s->block_height, vsub); + int brows = h / block_height; + int bcols = w / block_width; + + for (int blkj = 0; blkj < brows; blkj++) { + for (int blki = 0; blki < bcols; blki++) { + block_total_width = 0.0; + block_count = 0; + for (int inj = 0; inj < block_height; inj++) { + for (int ini = 0; ini < block_width; ini++) { + i = blki * block_width + ini; + j = blkj * block_height + inj; + + if (dst[j * dst_linesize + i] > 0) { + float width = edge_width(s, i, j, dir[j*dir_linesize+i], + w, h, dst[j*dst_linesize+i], + src, src_linesize); + if (width > 0.001) { // throw away zeros + block_count++; + block_total_width += width; + } + } + } + } + // if not enough edge pixels in a block, consider it smooth + if (block_total_width >= 2) { + blks[blkcnt] = block_total_width / block_count; + blkcnt++; + } + } + } + + // simple block pooling by sorting and keeping the sharper blocks + AV_QSORT(blks, blkcnt, float, comp); + blkcnt = ceil(blkcnt * block_pool_threshold); + for (int i = 0; i < blkcnt; i++) { + total_width += blks[i]; + } + + return total_width / blkcnt; +} + +static void set_meta(AVDictionary **metadata, const char *key, float d) +{ + char value[128]; + snprintf(value, sizeof(value), "%f", d); + av_dict_set(metadata, key, value, 0); +} + +static int blurdetect_filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx = inlink->dst; + BLRContext *s = ctx->priv; + AVFilterLink *outlink = ctx->outputs[0]; + + const int inw = inlink->w; + const int inh = inlink->h; + + uint8_t *tmpbuf = s->tmpbuf; + uint8_t *filterbuf = s->filterbuf; + uint16_t *gradients = s->gradients; + int8_t *directions = s->directions; + + float blur = 0.0f; + int nplanes = 0; + AVDictionary **metadata; + metadata = &in->metadata; + + for (int plane = 0; plane < s->nb_planes; plane++) { + int hsub = plane == 1 || plane == 2 ? s->hsub : 0; + int vsub = plane == 1 || plane == 2 ? s->vsub : 0; + int w = AV_CEIL_RSHIFT(inw, hsub); + int h = AV_CEIL_RSHIFT(inh, vsub); + + if (!((1 << plane) & s->planes)) + continue; + + nplanes++; + + // gaussian filter to reduce noise + ff_gaussian_blur(w, h, + filterbuf, w, + in->data[plane], in->linesize[plane]); + + // compute the 16-bits gradients and directions for the next step + ff_sobel(w, h, gradients, w, directions, w, filterbuf, w); + + // non_maximum_suppression() will actually keep & clip what's necessary and + // ignore the rest, so we need a clean output buffer + memset(tmpbuf, 0, inw * inh); + ff_non_maximum_suppression(w, h, tmpbuf, w, directions, w, gradients, w); + + + // keep high values, or low values surrounded by high values + ff_double_threshold(s->low_u8, s->high_u8, w, h, + tmpbuf, w, tmpbuf, w); + + blur += calculate_blur(s, w, h, hsub, vsub, directions, w, + tmpbuf, w, filterbuf, w); + } + + if (nplanes) + blur /= nplanes; + + s->blur_total += blur; + + // write stats + av_log(ctx, AV_LOG_VERBOSE, "blur: %.7f\n", blur); + + set_meta(metadata, "lavfi.blur", blur); + + s->nb_frames = inlink->frame_count_in; + + return ff_filter_frame(outlink, in); +} + +static av_cold void blurdetect_uninit(AVFilterContext *ctx) +{ + BLRContext *s = ctx->priv; + + if (s->nb_frames > 0) { + av_log(ctx, AV_LOG_INFO, "blur mean: %.7f\n", + s->blur_total / s->nb_frames); + } + + av_freep(&s->tmpbuf); + av_freep(&s->filterbuf); + av_freep(&s->gradients); + av_freep(&s->directions); + av_freep(&s->blks); +} + +static const enum AVPixelFormat pix_fmts[] = { + AV_PIX_FMT_GRAY8, + AV_PIX_FMT_GBRP, AV_PIX_FMT_GBRAP, + AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV444P, AV_PIX_FMT_YUV440P, + AV_PIX_FMT_YUV411P, AV_PIX_FMT_YUV410P, + AV_PIX_FMT_YUVJ440P, AV_PIX_FMT_YUVJ411P, AV_PIX_FMT_YUVJ420P, + AV_PIX_FMT_YUVJ422P, AV_PIX_FMT_YUVJ444P, + AV_PIX_FMT_YUVA444P, AV_PIX_FMT_YUVA422P, AV_PIX_FMT_YUVA420P, + AV_PIX_FMT_NONE +}; + +static const AVFilterPad blurdetect_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = blurdetect_config_input, + .filter_frame = blurdetect_filter_frame, + }, +}; + +static const AVFilterPad blurdetect_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +const AVFilter ff_vf_blurdetect = { + .name = "blurdetect", + .description = NULL_IF_CONFIG_SMALL("Blurdetect filter."), + .priv_size = sizeof(BLRContext), + .init = blurdetect_init, + .uninit = blurdetect_uninit, + FILTER_PIXFMTS_ARRAY(pix_fmts), + FILTER_INPUTS(blurdetect_inputs), + FILTER_OUTPUTS(blurdetect_outputs), + .priv_class = &blurdetect_class, + .flags = AVFILTER_FLAG_METADATA_ONLY, +}; + diff --git a/tests/fate/filter-video.mak b/tests/fate/filter-video.mak index 6c0d8df032..68f4c084f8 100644 --- a/tests/fate/filter-video.mak +++ b/tests/fate/filter-video.mak @@ -867,6 +867,9 @@ fate-filter-meta-4560-rotate0: CMD = framecrc -auto_conversion_filters -flags +b REFCMP_DEPS = FFMPEG LAVFI_INDEV TESTSRC2_FILTER AVGBLUR_FILTER METADATA_FILTER +FATE_FILTER-$(call ALLYES, $(REFCMP_DEPS) BLURDETECT_FILTER) += fate-filter-refcmp-blurdetect-yuv +fate-filter-refcmp-blurdetect-yuv: CMD = cmp_metadata blurdetect yuv420p 0.015 + FATE_FILTER-$(call ALLYES, $(REFCMP_DEPS) PSNR_FILTER) += fate-filter-refcmp-psnr-rgb fate-filter-refcmp-psnr-rgb: CMD = refcmp_metadata psnr rgb24 0.002 diff --git a/tests/ref/fate/filter-refcmp-blurdetect-yuv b/tests/ref/fate/filter-refcmp-blurdetect-yuv new file mode 100644 index 0000000000..a64a9a07ba --- /dev/null +++ b/tests/ref/fate/filter-refcmp-blurdetect-yuv @@ -0,0 +1,10 @@ +frame:0 pts:0 pts_time:0 +lavfi.blur=4.480640 +frame:1 pts:1 pts_time:1 +lavfi.blur=4.569398 +frame:2 pts:2 pts_time:2 +lavfi.blur=4.609386 +frame:3 pts:3 pts_time:3 +lavfi.blur=4.357340 +frame:4 pts:4 pts_time:4 +lavfi.blur=4.392871 -- 2.20.1 (Apple Git-117) [-- Attachment #4: Type: text/plain, Size: 251 bytes --] _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [FFmpeg-devel] [PATCH 2/2] lavfi: Add blurriness filter 2022-04-24 17:28 ` Thilo Borgmann @ 2022-04-24 22:03 ` Thilo Borgmann 2022-04-24 22:08 ` Thilo Borgmann 0 siblings, 1 reply; 10+ messages in thread From: Thilo Borgmann @ 2022-04-24 22:03 UTC (permalink / raw) To: ffmpeg-devel [-- Attachment #1: Type: text/plain, Size: 682 bytes --] Am 24.04.22 um 19:28 schrieb Thilo Borgmann: > Am 23.04.22 um 15:32 schrieb Thilo Borgmann: >> Hi, >> >>>>>> v3 updated to current HEAD. >>>>>> >>>>>> Named blurdetect filter now. >>>>>> Minor fixes on allocation and removed -f option. >>>>>> >>>>> >>>>> Please make this per plane filtering, with default to measure only first >>>>> plane. >>>> >>>> done in v4. >>>> >>>> (Will add Changelog, version.h and fate test once the filter itself looks ok) >>> >>> Ping. >> >> v5 according to IRC comments. > > v6 (1/2 and 2/2) according to IRC comments. LGTM'd there, applying soon. > > Added Changelog, version.h, FATE. v7: removed useless debug garbage. Still applying soon. -Thilo [-- Attachment #2: v7-0002-lavfi-Add-blurdetect-filter.patch --] [-- Type: text/plain, Size: 19120 bytes --] From 2879cfd94358aa977dd7f4314f931fb782750f96 Mon Sep 17 00:00:00 2001 From: Thilo Borgmann <thilo.borgmann@mail.de> Date: Mon, 25 Apr 2022 00:01:51 +0200 Subject: [PATCH v7 2/2] lavfi: Add blurdetect filter --- Changelog | 1 + doc/filters.texi | 52 +++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/version.h | 2 +- libavfilter/vf_blurdetect.c | 395 ++++++++++++++++++++ tests/fate/filter-video.mak | 3 + tests/ref/fate/filter-refcmp-blurdetect-yuv | 10 + 8 files changed, 464 insertions(+), 1 deletion(-) create mode 100644 libavfilter/vf_blurdetect.c create mode 100644 tests/ref/fate/filter-refcmp-blurdetect-yuv diff --git a/Changelog b/Changelog index 7a63e3d1ee..4d467eb741 100644 --- a/Changelog +++ b/Changelog @@ -13,6 +13,7 @@ version 5.1: - pixelize video filter - colormap video filter - colorchart video source filter +- blurdetect filter version 5.0: diff --git a/doc/filters.texi b/doc/filters.texi index c8699b9099..499f3adcd9 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -7997,6 +7997,58 @@ tblend=all_mode=grainextract @subsection Commands This filter supports same @ref{commands} as options. +@anchor{blurdetect} +@section blurdetect + +Determines blurriness of frames without altering the input frames. + +Based on Marziliano, Pina, et al. "A no-reference perceptual blur metric." +Allows for a block-based abbreviation. + +The filter accepts the following options: + +@table @option +@item low +@item high +Set low and high threshold values used by the Canny thresholding +algorithm. + +The high threshold selects the "strong" edge pixels, which are then +connected through 8-connectivity with the "weak" edge pixels selected +by the low threshold. + +@var{low} and @var{high} threshold values must be chosen in the range +[0,1], and @var{low} should be lesser or equal to @var{high}. + +Default value for @var{low} is @code{20/255}, and default value for @var{high} +is @code{50/255}. + +@item radius +Define the radius to search around an edge pixel for local maxima. + +@item block_pct +Determine blurriness only for the most significant blocks, given in percentage. + +@item block_width +Determine blurriness for blocks of width @var{block_width}. If set to any value smaller 1, no blocks are used and the whole image is processed as one no matter of @var{block_height}. + +@item block_height +Determine blurriness for blocks of height @var{block_height}. If set to any value smaller 1, no blocks are used and the whole image is processed as one no matter of @var{block_width}. + +@item planes +Set planes to filter. Default is first only. +@end table + +@subsection Examples + +@itemize +@item +Determine blur for 80% of most significant 32x32 blocks: +@example +blurdetect=block_width=32:block_height=32:block_pct=80 +@end example +@end itemize + @section bm3d Denoise frames using Block-Matching 3D algorithm. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 38ca379e5a..1db097b464 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -195,6 +195,7 @@ OBJS-$(CONFIG_BLACKDETECT_FILTER) += vf_blackdetect.o OBJS-$(CONFIG_BLACKFRAME_FILTER) += vf_blackframe.o OBJS-$(CONFIG_BLEND_FILTER) += vf_blend.o framesync.o OBJS-$(CONFIG_BLEND_VULKAN_FILTER) += vf_blend_vulkan.o framesync.o vulkan.o vulkan_filter.o +OBJS-$(CONFIG_BLURDETECT_FILTER) += vf_blurdetect.o edge_common.o OBJS-$(CONFIG_BM3D_FILTER) += vf_bm3d.o framesync.o OBJS-$(CONFIG_BOXBLUR_FILTER) += vf_boxblur.o boxblur.o OBJS-$(CONFIG_BOXBLUR_OPENCL_FILTER) += vf_avgblur_opencl.o opencl.o \ diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 36fa3ae8d7..2ad523fd0f 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -183,6 +183,7 @@ extern const AVFilter ff_vf_blackdetect; extern const AVFilter ff_vf_blackframe; extern const AVFilter ff_vf_blend; extern const AVFilter ff_vf_blend_vulkan; +extern const AVFilter ff_vf_blurdetect; extern const AVFilter ff_vf_bm3d; extern const AVFilter ff_vf_boxblur; extern const AVFilter ff_vf_boxblur_opencl; diff --git a/libavfilter/version.h b/libavfilter/version.h index 9add1658e5..8f1b16969a 100644 --- a/libavfilter/version.h +++ b/libavfilter/version.h @@ -31,7 +31,7 @@ #include "version_major.h" -#define LIBAVFILTER_VERSION_MINOR 36 +#define LIBAVFILTER_VERSION_MINOR 37 #define LIBAVFILTER_VERSION_MICRO 100 diff --git a/libavfilter/vf_blurdetect.c b/libavfilter/vf_blurdetect.c new file mode 100644 index 0000000000..7c0f5ce68a --- /dev/null +++ b/libavfilter/vf_blurdetect.c @@ -0,0 +1,395 @@ +/* + * Copyright (c) 2021 Thilo Borgmann <thilo.borgmann _at_ mail.de> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * No-reference blurdetect filter + * + * Implementing: + * Marziliano, Pina, et al. "A no-reference perceptual blur metric." Proceedings. + * International conference on image processing. Vol. 3. IEEE, 2002. + * https://infoscience.epfl.ch/record/111802/files/14%20A%20no-reference%20perceptual%20blur%20metric.pdf + * + * @author Thilo Borgmann <thilo.borgmann _at_ mail.de> + */ + +#include "libavutil/avassert.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "libavutil/pixelutils.h" +#include "libavutil/motion_vector.h" +#include "libavutil/qsort.h" +#include "avfilter.h" +#include "formats.h" +#include "internal.h" +#include "video.h" +#include "edge_common.h" + +static int comp(const float *a,const float *b) +{ + return FFDIFFSIGN(*a, *b); +} + +typedef struct BLRContext { + const AVClass *class; + + int hsub, vsub; + int nb_planes; + + float low, high; + uint8_t low_u8, high_u8; + int radius; // radius during local maxima detection + int block_pct; // percentage of "sharpest" blocks in the image to use for bluriness calculation + int block_width; // width for block abbreviation + int block_height; // height for block abbreviation + int planes; // number of planes to filter + + double blur_total; + uint64_t nb_frames; + + float *blks; + uint8_t *filterbuf; + uint8_t *tmpbuf; + uint16_t *gradients; + char *directions; +} BLRContext; + +#define OFFSET(x) offsetof(BLRContext, x) +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM +static const AVOption blurdetect_options[] = { + { "high", "set high threshold", OFFSET(high), AV_OPT_TYPE_FLOAT, {.dbl=30/255.}, 0, 1, FLAGS }, + { "low", "set low threshold", OFFSET(low), AV_OPT_TYPE_FLOAT, {.dbl=15/255.}, 0, 1, FLAGS }, + { "radius", "search radius for maxima detection", OFFSET(radius), AV_OPT_TYPE_INT, {.i64=50}, 1, 100, FLAGS }, + { "block_pct", "block pooling threshold when calculating blurriness", OFFSET(block_pct), AV_OPT_TYPE_INT, {.i64=80}, 1, 100, FLAGS }, + { "block_width", "block size for block-based abbreviation of blurriness", OFFSET(block_width), AV_OPT_TYPE_INT, {.i64=-1}, -1, INT_MAX, FLAGS }, + { "block_height", "block size for block-based abbreviation of blurriness", OFFSET(block_height), AV_OPT_TYPE_INT, {.i64=-1}, -1, INT_MAX, FLAGS }, + { "planes", "set planes to filter", OFFSET(planes), AV_OPT_TYPE_INT, {.i64=1}, 0, 15, FLAGS }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(blurdetect); + +static av_cold int blurdetect_init(AVFilterContext *ctx) +{ + BLRContext *s = ctx->priv; + + s->low_u8 = s->low * 255. + .5; + s->high_u8 = s->high * 255. + .5; + + return 0; +} + +static int blurdetect_config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + BLRContext *s = ctx->priv; + const int bufsize = inlink->w * inlink->h; + const AVPixFmtDescriptor *pix_desc; + + pix_desc = av_pix_fmt_desc_get(inlink->format); + s->hsub = pix_desc->log2_chroma_w; + s->vsub = pix_desc->log2_chroma_h; + s->nb_planes = av_pix_fmt_count_planes(inlink->format); + + if (s->block_width < 1 || s->block_height < 1) { + s->block_width = inlink->w; + s->block_height = inlink->h; + } + + s->tmpbuf = av_malloc(bufsize); + s->filterbuf = av_malloc(bufsize); + s->gradients = av_calloc(bufsize, sizeof(*s->gradients)); + s->directions = av_malloc(bufsize); + s->blks = av_calloc((inlink->w / s->block_width) * (inlink->h / s->block_height), + sizeof(*s->blks)); + + if (!s->tmpbuf || !s->filterbuf || !s->gradients || !s->directions || !s->blks) + return AVERROR(ENOMEM); + + return 0; +} + +// edge width is defined as the distance between surrounding maxima of the edge pixel +static float edge_width(BLRContext *blr, int i, int j, int8_t dir, int w, int h, + int edge, const uint8_t *src, int src_linesize) +{ + float width = 0; + int dX, dY; + int sign; + int tmp; + int p1; + int p2; + int k, x, y; + int edge1; + int edge2; + float luma1 = 0.0; // average luma difference per edge pixel + float luma2 = 0.0; + int radius = blr->radius; + + switch(dir) { + case DIRECTION_HORIZONTAL: dX = 1; dY = 0; break; + case DIRECTION_VERTICAL: dX = 0; dY = 1; break; + case DIRECTION_45UP: dX = 1; dY = -1; break; + case DIRECTION_45DOWN: dX = 1; dY = 1; break; + } + + // determines if search in direction dX/dY is looking for a maximum or minimum + sign = src[j * src_linesize + i] > src[(j - dY) * src_linesize + i - dX] ? 1 : -1; + + // search in -(dX/dY) direction + for (k = 0; k < radius; k++) { + x = i - k*dX; + y = j - k*dY; + p1 = y * src_linesize + x; + x -= dX; + y -= dY; + p2 = y * src_linesize + x; + if (x < 0 || x >= w || y < 0 || y >= h) + return 0; + + tmp = (src[p1] - src[p2]) * sign; + + if (tmp <= 0) // local maximum found + break; + + luma1 += tmp; + } + if (k > 0) luma1 /= k; + edge1 = k; + width += k; + + // search in +(dX/dY) direction + for (k = 0; k < radius; k++) { + x = i + k * dX; + y = j + k * dY; + p1 = y * src_linesize + x; + x += dX; + y += dY; + p2 = y * src_linesize + x; + if (x < 0 || x >= w || y < 0 || y >= h) + return 0; + + tmp = (src[p1] - src[p2]) * sign; + + if (tmp >= 0) // local maximum found + break; + + luma2 -= tmp; + } + if (k > 0) luma2 /= k; + edge2 = k; + width += k; + + // for 45 degree directions approximate edge width in pixel units: 0.7 ~= sqrt(2)/2 + if (dir == DIRECTION_45UP || dir == DIRECTION_45DOWN) + width *= 0.7; + + return width; +} + +static float calculate_blur(BLRContext *s, int w, int h, int hsub, int vsub, + uint8_t* dir, int dir_linesize, + uint8_t* dst, int dst_linesize, + uint8_t* src, int src_linesize) +{ + float total_width = 0.0; + int block_count; + double block_total_width; + + int i, j; + int blkcnt = 0; + + float *blks = s->blks; + float block_pool_threshold = s->block_pct / 100.0; + + int block_width = AV_CEIL_RSHIFT(s->block_width, hsub); + int block_height = AV_CEIL_RSHIFT(s->block_height, vsub); + int brows = h / block_height; + int bcols = w / block_width; + + for (int blkj = 0; blkj < brows; blkj++) { + for (int blki = 0; blki < bcols; blki++) { + block_total_width = 0.0; + block_count = 0; + for (int inj = 0; inj < block_height; inj++) { + for (int ini = 0; ini < block_width; ini++) { + i = blki * block_width + ini; + j = blkj * block_height + inj; + + if (dst[j * dst_linesize + i] > 0) { + float width = edge_width(s, i, j, dir[j*dir_linesize+i], + w, h, dst[j*dst_linesize+i], + src, src_linesize); + if (width > 0.001) { // throw away zeros + block_count++; + block_total_width += width; + } + } + } + } + // if not enough edge pixels in a block, consider it smooth + if (block_total_width >= 2) { + blks[blkcnt] = block_total_width / block_count; + blkcnt++; + } + } + } + + // simple block pooling by sorting and keeping the sharper blocks + AV_QSORT(blks, blkcnt, float, comp); + blkcnt = ceil(blkcnt * block_pool_threshold); + for (int i = 0; i < blkcnt; i++) { + total_width += blks[i]; + } + + return total_width / blkcnt; +} + +static void set_meta(AVDictionary **metadata, const char *key, float d) +{ + char value[128]; + snprintf(value, sizeof(value), "%f", d); + av_dict_set(metadata, key, value, 0); +} + +static int blurdetect_filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx = inlink->dst; + BLRContext *s = ctx->priv; + AVFilterLink *outlink = ctx->outputs[0]; + + const int inw = inlink->w; + const int inh = inlink->h; + + uint8_t *tmpbuf = s->tmpbuf; + uint8_t *filterbuf = s->filterbuf; + uint16_t *gradients = s->gradients; + int8_t *directions = s->directions; + + float blur = 0.0f; + int nplanes = 0; + AVDictionary **metadata; + metadata = &in->metadata; + + for (int plane = 0; plane < s->nb_planes; plane++) { + int hsub = plane == 1 || plane == 2 ? s->hsub : 0; + int vsub = plane == 1 || plane == 2 ? s->vsub : 0; + int w = AV_CEIL_RSHIFT(inw, hsub); + int h = AV_CEIL_RSHIFT(inh, vsub); + + if (!((1 << plane) & s->planes)) + continue; + + nplanes++; + + // gaussian filter to reduce noise + ff_gaussian_blur(w, h, + filterbuf, w, + in->data[plane], in->linesize[plane]); + + // compute the 16-bits gradients and directions for the next step + ff_sobel(w, h, gradients, w, directions, w, filterbuf, w); + + // non_maximum_suppression() will actually keep & clip what's necessary and + // ignore the rest, so we need a clean output buffer + memset(tmpbuf, 0, inw * inh); + ff_non_maximum_suppression(w, h, tmpbuf, w, directions, w, gradients, w); + + + // keep high values, or low values surrounded by high values + ff_double_threshold(s->low_u8, s->high_u8, w, h, + tmpbuf, w, tmpbuf, w); + + blur += calculate_blur(s, w, h, hsub, vsub, directions, w, + tmpbuf, w, filterbuf, w); + } + + if (nplanes) + blur /= nplanes; + + s->blur_total += blur; + + // write stats + av_log(ctx, AV_LOG_VERBOSE, "blur: %.7f\n", blur); + + set_meta(metadata, "lavfi.blur", blur); + + s->nb_frames = inlink->frame_count_in; + + return ff_filter_frame(outlink, in); +} + +static av_cold void blurdetect_uninit(AVFilterContext *ctx) +{ + BLRContext *s = ctx->priv; + + if (s->nb_frames > 0) { + av_log(ctx, AV_LOG_INFO, "blur mean: %.7f\n", + s->blur_total / s->nb_frames); + } + + av_freep(&s->tmpbuf); + av_freep(&s->filterbuf); + av_freep(&s->gradients); + av_freep(&s->directions); + av_freep(&s->blks); +} + +static const enum AVPixelFormat pix_fmts[] = { + AV_PIX_FMT_GRAY8, + AV_PIX_FMT_GBRP, AV_PIX_FMT_GBRAP, + AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV444P, AV_PIX_FMT_YUV440P, + AV_PIX_FMT_YUV411P, AV_PIX_FMT_YUV410P, + AV_PIX_FMT_YUVJ440P, AV_PIX_FMT_YUVJ411P, AV_PIX_FMT_YUVJ420P, + AV_PIX_FMT_YUVJ422P, AV_PIX_FMT_YUVJ444P, + AV_PIX_FMT_YUVA444P, AV_PIX_FMT_YUVA422P, AV_PIX_FMT_YUVA420P, + AV_PIX_FMT_NONE +}; + +static const AVFilterPad blurdetect_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = blurdetect_config_input, + .filter_frame = blurdetect_filter_frame, + }, +}; + +static const AVFilterPad blurdetect_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +const AVFilter ff_vf_blurdetect = { + .name = "blurdetect", + .description = NULL_IF_CONFIG_SMALL("Blurdetect filter."), + .priv_size = sizeof(BLRContext), + .init = blurdetect_init, + .uninit = blurdetect_uninit, + FILTER_PIXFMTS_ARRAY(pix_fmts), + FILTER_INPUTS(blurdetect_inputs), + FILTER_OUTPUTS(blurdetect_outputs), + .priv_class = &blurdetect_class, + .flags = AVFILTER_FLAG_METADATA_ONLY, +}; + diff --git a/tests/fate/filter-video.mak b/tests/fate/filter-video.mak index 6c0d8df032..68f4c084f8 100644 --- a/tests/fate/filter-video.mak +++ b/tests/fate/filter-video.mak @@ -867,6 +867,9 @@ fate-filter-meta-4560-rotate0: CMD = framecrc -auto_conversion_filters -flags +b REFCMP_DEPS = FFMPEG LAVFI_INDEV TESTSRC2_FILTER AVGBLUR_FILTER METADATA_FILTER +FATE_FILTER-$(call ALLYES, $(REFCMP_DEPS) BLURDETECT_FILTER) += fate-filter-refcmp-blurdetect-yuv +fate-filter-refcmp-blurdetect-yuv: CMD = cmp_metadata blurdetect yuv420p 0.015 + FATE_FILTER-$(call ALLYES, $(REFCMP_DEPS) PSNR_FILTER) += fate-filter-refcmp-psnr-rgb fate-filter-refcmp-psnr-rgb: CMD = refcmp_metadata psnr rgb24 0.002 diff --git a/tests/ref/fate/filter-refcmp-blurdetect-yuv b/tests/ref/fate/filter-refcmp-blurdetect-yuv new file mode 100644 index 0000000000..a64a9a07ba --- /dev/null +++ b/tests/ref/fate/filter-refcmp-blurdetect-yuv @@ -0,0 +1,10 @@ +frame:0 pts:0 pts_time:0 +lavfi.blur=4.480640 +frame:1 pts:1 pts_time:1 +lavfi.blur=4.569398 +frame:2 pts:2 pts_time:2 +lavfi.blur=4.609386 +frame:3 pts:3 pts_time:3 +lavfi.blur=4.357340 +frame:4 pts:4 pts_time:4 +lavfi.blur=4.392871 -- 2.20.1 (Apple Git-117) [-- Attachment #3: Type: text/plain, Size: 251 bytes --] _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [FFmpeg-devel] [PATCH 2/2] lavfi: Add blurriness filter 2022-04-24 22:03 ` Thilo Borgmann @ 2022-04-24 22:08 ` Thilo Borgmann 2022-04-25 18:52 ` Thilo Borgmann 0 siblings, 1 reply; 10+ messages in thread From: Thilo Borgmann @ 2022-04-24 22:08 UTC (permalink / raw) To: ffmpeg-devel [-- Attachment #1: Type: text/plain, Size: 791 bytes --] Am 25.04.22 um 00:03 schrieb Thilo Borgmann: > Am 24.04.22 um 19:28 schrieb Thilo Borgmann: >> Am 23.04.22 um 15:32 schrieb Thilo Borgmann: >>> Hi, >>> >>>>>>> v3 updated to current HEAD. >>>>>>> >>>>>>> Named blurdetect filter now. >>>>>>> Minor fixes on allocation and removed -f option. >>>>>>> >>>>>> >>>>>> Please make this per plane filtering, with default to measure only first >>>>>> plane. >>>>> >>>>> done in v4. >>>>> >>>>> (Will add Changelog, version.h and fate test once the filter itself looks ok) >>>> >>>> Ping. >>> >>> v5 according to IRC comments. >> >> v6 (1/2 and 2/2) according to IRC comments. LGTM'd there, applying soon. >> >> Added Changelog, version.h, FATE. > > v7: removed useless debug garbage. Still applying soon. v8: and now even with adapted FATE. -Thilo [-- Attachment #2: v8-0002-lavfi-Add-blurdetect-filter.patch --] [-- Type: text/plain, Size: 19120 bytes --] From 72f1631e8994e15eac58c5c9a8c0fcf14ef627ad Mon Sep 17 00:00:00 2001 From: Thilo Borgmann <thilo.borgmann@mail.de> Date: Mon, 25 Apr 2022 00:07:04 +0200 Subject: [PATCH v8 2/2] lavfi: Add blurdetect filter --- Changelog | 1 + doc/filters.texi | 52 +++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/version.h | 2 +- libavfilter/vf_blurdetect.c | 395 ++++++++++++++++++++ tests/fate/filter-video.mak | 3 + tests/ref/fate/filter-refcmp-blurdetect-yuv | 10 + 8 files changed, 464 insertions(+), 1 deletion(-) create mode 100644 libavfilter/vf_blurdetect.c create mode 100644 tests/ref/fate/filter-refcmp-blurdetect-yuv diff --git a/Changelog b/Changelog index 7a63e3d1ee..4d467eb741 100644 --- a/Changelog +++ b/Changelog @@ -13,6 +13,7 @@ version 5.1: - pixelize video filter - colormap video filter - colorchart video source filter +- blurdetect filter version 5.0: diff --git a/doc/filters.texi b/doc/filters.texi index c8699b9099..499f3adcd9 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -7997,6 +7997,58 @@ tblend=all_mode=grainextract @subsection Commands This filter supports same @ref{commands} as options. +@anchor{blurdetect} +@section blurdetect + +Determines blurriness of frames without altering the input frames. + +Based on Marziliano, Pina, et al. "A no-reference perceptual blur metric." +Allows for a block-based abbreviation. + +The filter accepts the following options: + +@table @option +@item low +@item high +Set low and high threshold values used by the Canny thresholding +algorithm. + +The high threshold selects the "strong" edge pixels, which are then +connected through 8-connectivity with the "weak" edge pixels selected +by the low threshold. + +@var{low} and @var{high} threshold values must be chosen in the range +[0,1], and @var{low} should be lesser or equal to @var{high}. + +Default value for @var{low} is @code{20/255}, and default value for @var{high} +is @code{50/255}. + +@item radius +Define the radius to search around an edge pixel for local maxima. + +@item block_pct +Determine blurriness only for the most significant blocks, given in percentage. + +@item block_width +Determine blurriness for blocks of width @var{block_width}. If set to any value smaller 1, no blocks are used and the whole image is processed as one no matter of @var{block_height}. + +@item block_height +Determine blurriness for blocks of height @var{block_height}. If set to any value smaller 1, no blocks are used and the whole image is processed as one no matter of @var{block_width}. + +@item planes +Set planes to filter. Default is first only. +@end table + +@subsection Examples + +@itemize +@item +Determine blur for 80% of most significant 32x32 blocks: +@example +blurdetect=block_width=32:block_height=32:block_pct=80 +@end example +@end itemize + @section bm3d Denoise frames using Block-Matching 3D algorithm. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 38ca379e5a..1db097b464 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -195,6 +195,7 @@ OBJS-$(CONFIG_BLACKDETECT_FILTER) += vf_blackdetect.o OBJS-$(CONFIG_BLACKFRAME_FILTER) += vf_blackframe.o OBJS-$(CONFIG_BLEND_FILTER) += vf_blend.o framesync.o OBJS-$(CONFIG_BLEND_VULKAN_FILTER) += vf_blend_vulkan.o framesync.o vulkan.o vulkan_filter.o +OBJS-$(CONFIG_BLURDETECT_FILTER) += vf_blurdetect.o edge_common.o OBJS-$(CONFIG_BM3D_FILTER) += vf_bm3d.o framesync.o OBJS-$(CONFIG_BOXBLUR_FILTER) += vf_boxblur.o boxblur.o OBJS-$(CONFIG_BOXBLUR_OPENCL_FILTER) += vf_avgblur_opencl.o opencl.o \ diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 36fa3ae8d7..2ad523fd0f 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -183,6 +183,7 @@ extern const AVFilter ff_vf_blackdetect; extern const AVFilter ff_vf_blackframe; extern const AVFilter ff_vf_blend; extern const AVFilter ff_vf_blend_vulkan; +extern const AVFilter ff_vf_blurdetect; extern const AVFilter ff_vf_bm3d; extern const AVFilter ff_vf_boxblur; extern const AVFilter ff_vf_boxblur_opencl; diff --git a/libavfilter/version.h b/libavfilter/version.h index 9add1658e5..8f1b16969a 100644 --- a/libavfilter/version.h +++ b/libavfilter/version.h @@ -31,7 +31,7 @@ #include "version_major.h" -#define LIBAVFILTER_VERSION_MINOR 36 +#define LIBAVFILTER_VERSION_MINOR 37 #define LIBAVFILTER_VERSION_MICRO 100 diff --git a/libavfilter/vf_blurdetect.c b/libavfilter/vf_blurdetect.c new file mode 100644 index 0000000000..7c0f5ce68a --- /dev/null +++ b/libavfilter/vf_blurdetect.c @@ -0,0 +1,395 @@ +/* + * Copyright (c) 2021 Thilo Borgmann <thilo.borgmann _at_ mail.de> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * No-reference blurdetect filter + * + * Implementing: + * Marziliano, Pina, et al. "A no-reference perceptual blur metric." Proceedings. + * International conference on image processing. Vol. 3. IEEE, 2002. + * https://infoscience.epfl.ch/record/111802/files/14%20A%20no-reference%20perceptual%20blur%20metric.pdf + * + * @author Thilo Borgmann <thilo.borgmann _at_ mail.de> + */ + +#include "libavutil/avassert.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "libavutil/pixelutils.h" +#include "libavutil/motion_vector.h" +#include "libavutil/qsort.h" +#include "avfilter.h" +#include "formats.h" +#include "internal.h" +#include "video.h" +#include "edge_common.h" + +static int comp(const float *a,const float *b) +{ + return FFDIFFSIGN(*a, *b); +} + +typedef struct BLRContext { + const AVClass *class; + + int hsub, vsub; + int nb_planes; + + float low, high; + uint8_t low_u8, high_u8; + int radius; // radius during local maxima detection + int block_pct; // percentage of "sharpest" blocks in the image to use for bluriness calculation + int block_width; // width for block abbreviation + int block_height; // height for block abbreviation + int planes; // number of planes to filter + + double blur_total; + uint64_t nb_frames; + + float *blks; + uint8_t *filterbuf; + uint8_t *tmpbuf; + uint16_t *gradients; + char *directions; +} BLRContext; + +#define OFFSET(x) offsetof(BLRContext, x) +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM +static const AVOption blurdetect_options[] = { + { "high", "set high threshold", OFFSET(high), AV_OPT_TYPE_FLOAT, {.dbl=30/255.}, 0, 1, FLAGS }, + { "low", "set low threshold", OFFSET(low), AV_OPT_TYPE_FLOAT, {.dbl=15/255.}, 0, 1, FLAGS }, + { "radius", "search radius for maxima detection", OFFSET(radius), AV_OPT_TYPE_INT, {.i64=50}, 1, 100, FLAGS }, + { "block_pct", "block pooling threshold when calculating blurriness", OFFSET(block_pct), AV_OPT_TYPE_INT, {.i64=80}, 1, 100, FLAGS }, + { "block_width", "block size for block-based abbreviation of blurriness", OFFSET(block_width), AV_OPT_TYPE_INT, {.i64=-1}, -1, INT_MAX, FLAGS }, + { "block_height", "block size for block-based abbreviation of blurriness", OFFSET(block_height), AV_OPT_TYPE_INT, {.i64=-1}, -1, INT_MAX, FLAGS }, + { "planes", "set planes to filter", OFFSET(planes), AV_OPT_TYPE_INT, {.i64=1}, 0, 15, FLAGS }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(blurdetect); + +static av_cold int blurdetect_init(AVFilterContext *ctx) +{ + BLRContext *s = ctx->priv; + + s->low_u8 = s->low * 255. + .5; + s->high_u8 = s->high * 255. + .5; + + return 0; +} + +static int blurdetect_config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + BLRContext *s = ctx->priv; + const int bufsize = inlink->w * inlink->h; + const AVPixFmtDescriptor *pix_desc; + + pix_desc = av_pix_fmt_desc_get(inlink->format); + s->hsub = pix_desc->log2_chroma_w; + s->vsub = pix_desc->log2_chroma_h; + s->nb_planes = av_pix_fmt_count_planes(inlink->format); + + if (s->block_width < 1 || s->block_height < 1) { + s->block_width = inlink->w; + s->block_height = inlink->h; + } + + s->tmpbuf = av_malloc(bufsize); + s->filterbuf = av_malloc(bufsize); + s->gradients = av_calloc(bufsize, sizeof(*s->gradients)); + s->directions = av_malloc(bufsize); + s->blks = av_calloc((inlink->w / s->block_width) * (inlink->h / s->block_height), + sizeof(*s->blks)); + + if (!s->tmpbuf || !s->filterbuf || !s->gradients || !s->directions || !s->blks) + return AVERROR(ENOMEM); + + return 0; +} + +// edge width is defined as the distance between surrounding maxima of the edge pixel +static float edge_width(BLRContext *blr, int i, int j, int8_t dir, int w, int h, + int edge, const uint8_t *src, int src_linesize) +{ + float width = 0; + int dX, dY; + int sign; + int tmp; + int p1; + int p2; + int k, x, y; + int edge1; + int edge2; + float luma1 = 0.0; // average luma difference per edge pixel + float luma2 = 0.0; + int radius = blr->radius; + + switch(dir) { + case DIRECTION_HORIZONTAL: dX = 1; dY = 0; break; + case DIRECTION_VERTICAL: dX = 0; dY = 1; break; + case DIRECTION_45UP: dX = 1; dY = -1; break; + case DIRECTION_45DOWN: dX = 1; dY = 1; break; + } + + // determines if search in direction dX/dY is looking for a maximum or minimum + sign = src[j * src_linesize + i] > src[(j - dY) * src_linesize + i - dX] ? 1 : -1; + + // search in -(dX/dY) direction + for (k = 0; k < radius; k++) { + x = i - k*dX; + y = j - k*dY; + p1 = y * src_linesize + x; + x -= dX; + y -= dY; + p2 = y * src_linesize + x; + if (x < 0 || x >= w || y < 0 || y >= h) + return 0; + + tmp = (src[p1] - src[p2]) * sign; + + if (tmp <= 0) // local maximum found + break; + + luma1 += tmp; + } + if (k > 0) luma1 /= k; + edge1 = k; + width += k; + + // search in +(dX/dY) direction + for (k = 0; k < radius; k++) { + x = i + k * dX; + y = j + k * dY; + p1 = y * src_linesize + x; + x += dX; + y += dY; + p2 = y * src_linesize + x; + if (x < 0 || x >= w || y < 0 || y >= h) + return 0; + + tmp = (src[p1] - src[p2]) * sign; + + if (tmp >= 0) // local maximum found + break; + + luma2 -= tmp; + } + if (k > 0) luma2 /= k; + edge2 = k; + width += k; + + // for 45 degree directions approximate edge width in pixel units: 0.7 ~= sqrt(2)/2 + if (dir == DIRECTION_45UP || dir == DIRECTION_45DOWN) + width *= 0.7; + + return width; +} + +static float calculate_blur(BLRContext *s, int w, int h, int hsub, int vsub, + uint8_t* dir, int dir_linesize, + uint8_t* dst, int dst_linesize, + uint8_t* src, int src_linesize) +{ + float total_width = 0.0; + int block_count; + double block_total_width; + + int i, j; + int blkcnt = 0; + + float *blks = s->blks; + float block_pool_threshold = s->block_pct / 100.0; + + int block_width = AV_CEIL_RSHIFT(s->block_width, hsub); + int block_height = AV_CEIL_RSHIFT(s->block_height, vsub); + int brows = h / block_height; + int bcols = w / block_width; + + for (int blkj = 0; blkj < brows; blkj++) { + for (int blki = 0; blki < bcols; blki++) { + block_total_width = 0.0; + block_count = 0; + for (int inj = 0; inj < block_height; inj++) { + for (int ini = 0; ini < block_width; ini++) { + i = blki * block_width + ini; + j = blkj * block_height + inj; + + if (dst[j * dst_linesize + i] > 0) { + float width = edge_width(s, i, j, dir[j*dir_linesize+i], + w, h, dst[j*dst_linesize+i], + src, src_linesize); + if (width > 0.001) { // throw away zeros + block_count++; + block_total_width += width; + } + } + } + } + // if not enough edge pixels in a block, consider it smooth + if (block_total_width >= 2) { + blks[blkcnt] = block_total_width / block_count; + blkcnt++; + } + } + } + + // simple block pooling by sorting and keeping the sharper blocks + AV_QSORT(blks, blkcnt, float, comp); + blkcnt = ceil(blkcnt * block_pool_threshold); + for (int i = 0; i < blkcnt; i++) { + total_width += blks[i]; + } + + return total_width / blkcnt; +} + +static void set_meta(AVDictionary **metadata, const char *key, float d) +{ + char value[128]; + snprintf(value, sizeof(value), "%f", d); + av_dict_set(metadata, key, value, 0); +} + +static int blurdetect_filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx = inlink->dst; + BLRContext *s = ctx->priv; + AVFilterLink *outlink = ctx->outputs[0]; + + const int inw = inlink->w; + const int inh = inlink->h; + + uint8_t *tmpbuf = s->tmpbuf; + uint8_t *filterbuf = s->filterbuf; + uint16_t *gradients = s->gradients; + int8_t *directions = s->directions; + + float blur = 0.0f; + int nplanes = 0; + AVDictionary **metadata; + metadata = &in->metadata; + + for (int plane = 0; plane < s->nb_planes; plane++) { + int hsub = plane == 1 || plane == 2 ? s->hsub : 0; + int vsub = plane == 1 || plane == 2 ? s->vsub : 0; + int w = AV_CEIL_RSHIFT(inw, hsub); + int h = AV_CEIL_RSHIFT(inh, vsub); + + if (!((1 << plane) & s->planes)) + continue; + + nplanes++; + + // gaussian filter to reduce noise + ff_gaussian_blur(w, h, + filterbuf, w, + in->data[plane], in->linesize[plane]); + + // compute the 16-bits gradients and directions for the next step + ff_sobel(w, h, gradients, w, directions, w, filterbuf, w); + + // non_maximum_suppression() will actually keep & clip what's necessary and + // ignore the rest, so we need a clean output buffer + memset(tmpbuf, 0, inw * inh); + ff_non_maximum_suppression(w, h, tmpbuf, w, directions, w, gradients, w); + + + // keep high values, or low values surrounded by high values + ff_double_threshold(s->low_u8, s->high_u8, w, h, + tmpbuf, w, tmpbuf, w); + + blur += calculate_blur(s, w, h, hsub, vsub, directions, w, + tmpbuf, w, filterbuf, w); + } + + if (nplanes) + blur /= nplanes; + + s->blur_total += blur; + + // write stats + av_log(ctx, AV_LOG_VERBOSE, "blur: %.7f\n", blur); + + set_meta(metadata, "lavfi.blur", blur); + + s->nb_frames = inlink->frame_count_in; + + return ff_filter_frame(outlink, in); +} + +static av_cold void blurdetect_uninit(AVFilterContext *ctx) +{ + BLRContext *s = ctx->priv; + + if (s->nb_frames > 0) { + av_log(ctx, AV_LOG_INFO, "blur mean: %.7f\n", + s->blur_total / s->nb_frames); + } + + av_freep(&s->tmpbuf); + av_freep(&s->filterbuf); + av_freep(&s->gradients); + av_freep(&s->directions); + av_freep(&s->blks); +} + +static const enum AVPixelFormat pix_fmts[] = { + AV_PIX_FMT_GRAY8, + AV_PIX_FMT_GBRP, AV_PIX_FMT_GBRAP, + AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV444P, AV_PIX_FMT_YUV440P, + AV_PIX_FMT_YUV411P, AV_PIX_FMT_YUV410P, + AV_PIX_FMT_YUVJ440P, AV_PIX_FMT_YUVJ411P, AV_PIX_FMT_YUVJ420P, + AV_PIX_FMT_YUVJ422P, AV_PIX_FMT_YUVJ444P, + AV_PIX_FMT_YUVA444P, AV_PIX_FMT_YUVA422P, AV_PIX_FMT_YUVA420P, + AV_PIX_FMT_NONE +}; + +static const AVFilterPad blurdetect_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = blurdetect_config_input, + .filter_frame = blurdetect_filter_frame, + }, +}; + +static const AVFilterPad blurdetect_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +const AVFilter ff_vf_blurdetect = { + .name = "blurdetect", + .description = NULL_IF_CONFIG_SMALL("Blurdetect filter."), + .priv_size = sizeof(BLRContext), + .init = blurdetect_init, + .uninit = blurdetect_uninit, + FILTER_PIXFMTS_ARRAY(pix_fmts), + FILTER_INPUTS(blurdetect_inputs), + FILTER_OUTPUTS(blurdetect_outputs), + .priv_class = &blurdetect_class, + .flags = AVFILTER_FLAG_METADATA_ONLY, +}; + diff --git a/tests/fate/filter-video.mak b/tests/fate/filter-video.mak index 6c0d8df032..68f4c084f8 100644 --- a/tests/fate/filter-video.mak +++ b/tests/fate/filter-video.mak @@ -867,6 +867,9 @@ fate-filter-meta-4560-rotate0: CMD = framecrc -auto_conversion_filters -flags +b REFCMP_DEPS = FFMPEG LAVFI_INDEV TESTSRC2_FILTER AVGBLUR_FILTER METADATA_FILTER +FATE_FILTER-$(call ALLYES, $(REFCMP_DEPS) BLURDETECT_FILTER) += fate-filter-refcmp-blurdetect-yuv +fate-filter-refcmp-blurdetect-yuv: CMD = cmp_metadata blurdetect yuv420p 0.015 + FATE_FILTER-$(call ALLYES, $(REFCMP_DEPS) PSNR_FILTER) += fate-filter-refcmp-psnr-rgb fate-filter-refcmp-psnr-rgb: CMD = refcmp_metadata psnr rgb24 0.002 diff --git a/tests/ref/fate/filter-refcmp-blurdetect-yuv b/tests/ref/fate/filter-refcmp-blurdetect-yuv new file mode 100644 index 0000000000..426321db0c --- /dev/null +++ b/tests/ref/fate/filter-refcmp-blurdetect-yuv @@ -0,0 +1,10 @@ +frame:0 pts:0 pts_time:0 +lavfi.blur=4.499666 +frame:1 pts:1 pts_time:1 +lavfi.blur=4.677492 +frame:2 pts:2 pts_time:2 +lavfi.blur=4.735711 +frame:3 pts:3 pts_time:3 +lavfi.blur=4.532343 +frame:4 pts:4 pts_time:4 +lavfi.blur=4.532660 -- 2.20.1 (Apple Git-117) [-- Attachment #3: Type: text/plain, Size: 251 bytes --] _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [FFmpeg-devel] [PATCH 2/2] lavfi: Add blurriness filter 2022-04-24 22:08 ` Thilo Borgmann @ 2022-04-25 18:52 ` Thilo Borgmann 0 siblings, 0 replies; 10+ messages in thread From: Thilo Borgmann @ 2022-04-25 18:52 UTC (permalink / raw) To: ffmpeg-devel Am 25.04.22 um 00:08 schrieb Thilo Borgmann: > Am 25.04.22 um 00:03 schrieb Thilo Borgmann: >> Am 24.04.22 um 19:28 schrieb Thilo Borgmann: >>> Am 23.04.22 um 15:32 schrieb Thilo Borgmann: >>>> Hi, >>>> >>>>>>>> v3 updated to current HEAD. >>>>>>>> >>>>>>>> Named blurdetect filter now. >>>>>>>> Minor fixes on allocation and removed -f option. >>>>>>>> >>>>>>> >>>>>>> Please make this per plane filtering, with default to measure only first >>>>>>> plane. >>>>>> >>>>>> done in v4. >>>>>> >>>>>> (Will add Changelog, version.h and fate test once the filter itself looks ok) >>>>> >>>>> Ping. >>>> >>>> v5 according to IRC comments. >>> >>> v6 (1/2 and 2/2) according to IRC comments. LGTM'd there, applying soon. >>> >>> Added Changelog, version.h, FATE. >> >> v7: removed useless debug garbage. Still applying soon. > > v8: and now even with adapted FATE. Pushed, thanks! -Thilo _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2022-04-25 18:53 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <af635ff1-fc5c-d39a-4c39-424d5eefbe6f@mail.de>
2022-04-04 10:25 ` [FFmpeg-devel] [PATCH 1/2] lafi/vf_edgedetect: Move some common functions into seperate file Thilo Borgmann
[not found] ` <00ca6f6f-fc18-8b6b-1ff0-f24db7f3714b@mail.de>
2022-04-04 10:25 ` [FFmpeg-devel] [PATCH 2/2] lavfi: Add blurriness filter Thilo Borgmann
2022-04-04 10:42 ` Paul B Mahol
2022-04-05 8:56 ` Thilo Borgmann
2022-04-14 8:08 ` Thilo Borgmann
2022-04-23 13:32 ` Thilo Borgmann
2022-04-24 17:28 ` Thilo Borgmann
2022-04-24 22:03 ` Thilo Borgmann
2022-04-24 22:08 ` Thilo Borgmann
2022-04-25 18:52 ` Thilo Borgmann
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel This inbox may be cloned and mirrored by anyone: git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \ ffmpegdev@gitmailbox.com public-inbox-index ffmpegdev Example config snippet for mirrors. AGPL code for this site: git clone https://public-inbox.org/public-inbox.git