[FFmpeg-devel] [PATCH v2 0/3] consider chroma subsampling for bwdif (including CUDA and Vulkan)

[FFmpeg-devel] [PATCH v2 0/3] consider chroma subsampling for bwdif (including CUDA and Vulkan)

[Cosmin Stejerean via ffmpeg-devel]

From: Cosmin Stejerean <cosmin@cosmin.at>

This fixes the issue reported in #10688. In v2 the checks for chroma
plane are combined together and the fixes are extended to cover the
Vulkan and CUDA implementations of bwdif.

I have not had a chance to replicate the issue on CUDA or Vulkan but
based on the dimension checks in those filters I believe they would
have the same out of bounds issue.

I kept the changes in separate patches since I've only properly tested
the first one. I'll work on testing the CUDA and Vulkan
implementations as well but meanwhile wanted to get some feedback on
the overall approach.

Cosmin Stejerean (3):
  avfilter/vf_bwdif: consider chroma subsampling when enforcing minimum
    dimensions
  avfilter/vf_bwdif_cuda: consider chroma subsampling when enforcing
    minimum dimensions
  avfilter/vf_bwdif_vulkan: consider chroma subsampling when enforcing
    minimum dimensions

 libavfilter/vf_bwdif.c        | 13 ++++++++++---
 libavfilter/vf_bwdif_cuda.c   | 13 ++++++++++---
 libavfilter/vf_bwdif_vulkan.c | 14 ++++++++++----
 3 files changed, 30 insertions(+), 10 deletions(-)

-- 
2.42.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2 2/3] avfilter/vf_bwdif_cuda: consider chroma subsampling when enforcing minimum dimensions

[Cosmin Stejerean via ffmpeg-devel]

From: Cosmin Stejerean <cosmin@cosmin.at>

Signed-off-by: Cosmin Stejerean <cosmin@cosmin.at>
---
 libavfilter/vf_bwdif_cuda.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/libavfilter/vf_bwdif_cuda.c b/libavfilter/vf_bwdif_cuda.c
index a5ecfbadb6..7d9bf861b7 100644
--- a/libavfilter/vf_bwdif_cuda.c
+++ b/libavfilter/vf_bwdif_cuda.c
@@ -296,13 +296,20 @@ static int config_output(AVFilterLink *link)
         link->frame_rate = av_mul_q(ctx->inputs[0]->frame_rate,
                                     (AVRational){2, 1});
 
-    if (link->w < 3 || link->h < 3) {
-        av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or lines is not supported\n");
+    const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(output_frames->sw_format);
+
+    int h = link->h;
+    int w = link->w;
+    int h_chroma = AV_CEIL_RSHIFT(h, desc->log2_chroma_h);
+    int w_chroma = AV_CEIL_RSHIFT(w, desc->log2_chroma_w);
+
+    if (w < 3 || w_chroma < 3 || h < 3 || h_chroma < 3) {
+        av_log(ctx, AV_LOG_ERROR, "Video with planes less than 3 columns or lines is not supported\n");
         ret = AVERROR(EINVAL);
         goto exit;
     }
 
-    y->csp = av_pix_fmt_desc_get(output_frames->sw_format);
+    y->csp = desc;
     y->filter = filter;
 
     ret = CHECK_CU(cu->cuCtxPushCurrent(s->hwctx->cuda_ctx));
-- 
2.42.1


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2 1/3] avfilter/vf_bwdif: consider chroma subsampling when enforcing minimum dimensions

[Cosmin Stejerean via ffmpeg-devel]

From: Cosmin Stejerean <cosmin@cosmin.at>

Fixes #10688

Signed-off-by: Cosmin Stejerean <cosmin@cosmin.at>
---
 libavfilter/vf_bwdif.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/libavfilter/vf_bwdif.c b/libavfilter/vf_bwdif.c
index 137cd5ef13..80aa85a48b 100644
--- a/libavfilter/vf_bwdif.c
+++ b/libavfilter/vf_bwdif.c
@@ -191,12 +191,19 @@ static int config_props(AVFilterLink *link)
         return ret;
     }
 
-    if (link->w < 3 || link->h < 4) {
-        av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or 4 lines is not supported\n");
+    const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(link->format);
+
+    int h = link->h;
+    int w = link->w;
+    int h_chroma = AV_CEIL_RSHIFT(h, desc->log2_chroma_h);
+    int w_chroma = AV_CEIL_RSHIFT(w, desc->log2_chroma_w);
+
+    if (w < 3 || w_chroma < 3 || h < 4 || h_chroma < 4) {
+        av_log(ctx, AV_LOG_ERROR, "Video with planes less than 3 columns or 4 lines is not supported\n");
         return AVERROR(EINVAL);
     }
 
-    yadif->csp = av_pix_fmt_desc_get(link->format);
+    yadif->csp = desc;
     yadif->filter = filter;
     ff_bwdif_init_filter_line(&s->dsp, yadif->csp->comp[0].depth);
 
-- 
2.42.1


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/3] avfilter/vf_bwdif: consider chroma subsampling when enforcing minimum dimensions

[Thomas Mundt]

Am Do., 30. Nov. 2023 um 01:23 Uhr schrieb Cosmin Stejerean via
ffmpeg-devel <ffmpeg-devel@ffmpeg.org>:

> From: Cosmin Stejerean <cosmin@cosmin.at>
>
> Fixes #10688
>
> Signed-off-by: Cosmin Stejerean <cosmin@cosmin.at>
> ---
>  libavfilter/vf_bwdif.c | 13 ++++++++++---
>  1 file changed, 10 insertions(+), 3 deletions(-)
>
> diff --git a/libavfilter/vf_bwdif.c b/libavfilter/vf_bwdif.c
> index 137cd5ef13..80aa85a48b 100644
> --- a/libavfilter/vf_bwdif.c
> +++ b/libavfilter/vf_bwdif.c
> @@ -191,12 +191,19 @@ static int config_props(AVFilterLink *link)
>          return ret;
>      }
>
> -    if (link->w < 3 || link->h < 4) {
> -        av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or 4
> lines is not supported\n");
> +    const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(link->format);
> +
> +    int h = link->h;
> +    int w = link->w;
> +    int h_chroma = AV_CEIL_RSHIFT(h, desc->log2_chroma_h);
> +    int w_chroma = AV_CEIL_RSHIFT(w, desc->log2_chroma_w);
> +
> +    if (w < 3 || w_chroma < 3 || h < 4 || h_chroma < 4) {
> +        av_log(ctx, AV_LOG_ERROR, "Video with planes less than 3 columns
> or 4 lines is not supported\n");
>          return AVERROR(EINVAL);
>      }
>
> -    yadif->csp = av_pix_fmt_desc_get(link->format);
> +    yadif->csp = desc;
>      yadif->filter = filter;
>      ff_bwdif_init_filter_line(&s->dsp, yadif->csp->comp[0].depth);
>
> I think mixed declarations are not allowed.
Also log2_chroma_w/h should never be negative, so why not just do:

if (AV_CEIL_RSHIFT(link->w,  yadif->csp->log2_chroma_w) < 3 ||
    AV_CEIL_RSHIFT(link->h,  yadif->csp->log2_chroma_h) < 4)

Regards,
Thomas
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/3] avfilter/vf_bwdif: consider chroma subsampling when enforcing minimum dimensions

[Cosmin Stejerean via ffmpeg-devel]

On Nov 30, 2023, at 04:37, Thomas Mundt <tmundt75@gmail.com> wrote:


Am Do., 30. Nov. 2023 um 01:23 Uhr schrieb Cosmin Stejerean via ffmpeg-devel <ffmpeg-devel@ffmpeg.org <mailto:ffmpeg-devel@ffmpeg.org> >:
From: Cosmin Stejerean <cosmin@cosmin.at <mailto:cosmin@cosmin.at> >

Fixes #10688

Signed-off-by: Cosmin Stejerean <cosmin@cosmin.at <mailto:cosmin@cosmin.at> >
---
 libavfilter/vf_bwdif.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/libavfilter/vf_bwdif.c b/libavfilter/vf_bwdif.c
index 137cd5ef13..80aa85a48b 100644
--- a/libavfilter/vf_bwdif.c
+++ b/libavfilter/vf_bwdif.c
@@ -191,12 +191,19 @@ static int config_props(AVFilterLink *link)
         return ret;
     }

-    if (link->w < 3 || link->h < 4) {
-        av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or 4 lines is not supported\n");
+    const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(link->format);
+
+    int h = link->h;
+    int w = link->w;
+    int h_chroma = AV_CEIL_RSHIFT(h, desc->log2_chroma_h);
+    int w_chroma = AV_CEIL_RSHIFT(w, desc->log2_chroma_w);
+
+    if (w < 3 || w_chroma < 3 || h < 4 || h_chroma < 4) {
+        av_log(ctx, AV_LOG_ERROR, "Video with planes less than 3 columns or 4 lines is not supported\n");
         return AVERROR(EINVAL);
     }

-    yadif->csp = av_pix_fmt_desc_get(link->format);
+    yadif->csp = desc;
     yadif->filter = filter;
     ff_bwdif_init_filter_line(&s->dsp, yadif->csp->comp[0].depth);

I think mixed declarations are not allowed.
Also log2_chroma_w/h should never be negative, so why not just do:

if (AV_CEIL_RSHIFT(link->w,  yadif->csp->log2_chroma_w) < 3 ||
    AV_CEIL_RSHIFT(link->h,  yadif->csp->log2_chroma_h) < 4)


Thank you for the prompt feedback, makes a lot of sense to me, will update in v3. 

- Cosmin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2 3/3] avfilter/vf_bwdif_vulkan: consider chroma subsampling when enforcing minimum dimensions

[Cosmin Stejerean via ffmpeg-devel]

From: Cosmin Stejerean <cosmin@cosmin.at>

Signed-off-by: Cosmin Stejerean <cosmin@cosmin.at>
---
 libavfilter/vf_bwdif_vulkan.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/libavfilter/vf_bwdif_vulkan.c b/libavfilter/vf_bwdif_vulkan.c
index 690a89c4ba..6a886d11ac 100644
--- a/libavfilter/vf_bwdif_vulkan.c
+++ b/libavfilter/vf_bwdif_vulkan.c
@@ -362,13 +362,19 @@ static int bwdif_vulkan_config_output(AVFilterLink *outlink)
         outlink->frame_rate = av_mul_q(avctx->inputs[0]->frame_rate,
                                        (AVRational){2, 1});
 
-    if (outlink->w < 4 || outlink->h < 4) {
-        av_log(avctx, AV_LOG_ERROR, "Video of less than 4 columns or lines is not "
-               "supported\n");
+    const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(vkctx->frames->sw_format);
+
+    int h = link->h;
+    int w = link->w;
+    int h_chroma = AV_CEIL_RSHIFT(h, desc->log2_chroma_h);
+    int w_chroma = AV_CEIL_RSHIFT(w, desc->log2_chroma_w);
+
+    if (w < 4 || w_chroma < 4 || h < 4 || h_chroma < 4) {
+        av_log(ctx, AV_LOG_ERROR, "Video with planes less than 4 columns or lines is not supported\n");
         return AVERROR(EINVAL);
     }
 
-    y->csp = av_pix_fmt_desc_get(vkctx->frames->sw_format);
+    y->csp = desc;
     y->filter = bwdif_vulkan_filter_frame;
 
     return init_filter(avctx);
-- 
2.42.1


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".