[FFmpeg-devel] [PATCH] avfilter/vf_libplacebo: introduce fit_mode option (PR #20467)

[FFmpeg-devel] [PATCH] avfilter/vf_libplacebo: introduce fit_mode option (PR #20467)

[Niklas Haas via ffmpeg-devel]

PR #20467 opened by Niklas Haas (haasn)
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20467
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20467.patch

The semantics of these keywords are well-defined by the CSS 'object-fit'
property. This is arguably more user-friendly and less obtuse than the
existing `normalize_sar` and `pad_crop_ratio` options. Additionally, this
comes with two new (useful) behaviors, `none` and `scale_down`, neither of
which map elegantly to the existing options.

One additional benefit of this option is that, unlike `normalize_sar`, it
does *not* also imply `reset_sar`; meaning that users can now choose to
have an anamorphic base layer and still have the overlay images scaled to fit
on top of it according to the chosen strategy.

See-Also: https://drafts.csswg.org/css-images/#the-object-fit


>From 65288dc0bbad64a9b59df6ce2535da16aa0d67e0 Mon Sep 17 00:00:00 2001
From: Niklas Haas <git@haasn.dev>
Date: Mon, 8 Sep 2025 18:28:19 +0200
Subject: [PATCH] avfilter/vf_libplacebo: introduce fit_mode option

The semantics of these keywords are well-defined by the CSS 'object-fit'
property. This is arguably more user-friendly and less obtuse than the
existing `normalize_sar` and `pad_crop_ratio` options. Additionally, this
comes with two new (useful) behaviors, `none` and `scale_down`, neither of
which map elegantly to the existing options.

One additional benefit of this option is that, unlike `normalize_sar`, it
does *not* also imply `reset_sar`; meaning that users can now choose to
have an anamorphic base layer and still have the overlay images scaled to fit
on top of it according to the chosen strategy.

See-Also: https://drafts.csswg.org/css-images/#the-object-fit
---
 doc/filters.texi            | 36 ++++++++++++++++++++++++-
 libavfilter/vf_libplacebo.c | 53 ++++++++++++++++++++++++++++++++++---
 2 files changed, 84 insertions(+), 5 deletions(-)

diff --git a/doc/filters.texi b/doc/filters.texi
index 5b52fc5521..f2966ffa94 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -16447,7 +16447,7 @@ e.g. anamorphic video sources, are forwarded to the output pixel aspect ratio.
 @item normalize_sar
 Like @option{reset_sar}, but instead of stretching the video content to fill
 the new output aspect ratio, the content is instead padded or cropped as
-necessary.
+necessary. Mutually exclusive with @option{fit_mode}. Disabled by default.
 
 @item pad_crop_ratio
 Specifies a ratio (between @code{0.0} and @code{1.0}) between padding and
@@ -16457,6 +16457,40 @@ content with black borders, while a value of @code{1.0} always crops off parts
 of the content. Intermediate values are possible, leading to a mix of the two
 approaches.
 
+@item fit_mode
+Specify the content fit strategy according to a list of predefined modes.
+Determines how the input image is to be placed inside the destination crop
+rectangle (as defined by @code{pos_x/y} and @code{pos_w/h}). The names and
+their implementations are taken from the CSS 'object-fit' property. Note that
+this option is mutually exclusive with @option{normalize_sar}. Defaults to
+@code{fill}. Valid values are:
+
+@table @samp
+@item fill
+Stretch the input to the output rectangle, ignoring aspect ratio mismatches.
+Note that unless @option{reset_sar} is also enabled, the output will still
+have the correct pixel aspect ratio tagged.
+
+@item contain
+Scale the input to fit inside the output, preserving aspect ratio by padding.
+Equivalent to @option{normalize_sar} with @option{pad_crop_ratio} set to
+@code{0.0}.
+
+@item cover
+Scale the input to fill the output, preserving aspect ratio by cropping.
+Equivalent to @option{normalize_sar} with @option{pad_crop_ratio} set to
+@code{1.0}.
+
+@item none
+Don't scale the input. The input will be placed inside the output rectangle at
+its natural size; which may result in additional padding or cropping.
+
+@item scale_down
+Scale the input down as much as needed to fit inside the output. Equivalent
+to either @code{contain} or @code{none}, depending on whether the input is
+larger than the output or not.
+@end table
+
 @item fillcolor
 Set the color used to fill the output area not covered by the output image, for
 example as a result of @option{normalize_sar}. For the general syntax of this
diff --git a/libavfilter/vf_libplacebo.c b/libavfilter/vf_libplacebo.c
index f4b5ac6b03..d4ad3349ff 100644
--- a/libavfilter/vf_libplacebo.c
+++ b/libavfilter/vf_libplacebo.c
@@ -152,6 +152,15 @@ typedef struct LibplaceboInput {
     int status;
 } LibplaceboInput;
 
+enum fit_mode {
+    FIT_FILL,
+    FIT_CONTAIN,
+    FIT_COVER,
+    FIT_NONE,
+    FIT_SCALE_DOWN,
+    FIT_MODE_NB,
+};
+
 typedef struct LibplaceboContext {
     /* lavfi vulkan*/
     FFVulkanContext vkctx;
@@ -196,6 +205,7 @@ typedef struct LibplaceboContext {
     int force_divisible_by;
     int reset_sar;
     int normalize_sar;
+    int fit_mode;
     int apply_filmgrain;
     int apply_dovi;
     int colorspace;
@@ -543,6 +553,11 @@ static int libplacebo_init(AVFilterContext *avctx)
     LibplaceboContext *s = avctx->priv;
     const AVVulkanDeviceContext *vkhwctx = NULL;
 
+    if (s->normalize_sar && s->fit_mode != FIT_FILL) {
+        av_log(avctx, AV_LOG_WARNING, "normalize_sar has no effect when using "
+               "a fit mode other than 'fill'\n");
+    }
+
     /* Create libplacebo log context */
     s->log = pl_log_create(PL_API_VER, pl_log_params(
         .log_level = get_log_level(),
@@ -841,6 +856,7 @@ static void update_crops(AVFilterContext *ctx, LibplaceboInput *in,
 {
     FilterLink     *outl = ff_filter_link(ctx->outputs[0]);
     LibplaceboContext *s = ctx->priv;
+    const AVFilterLink *outlink = ctx->outputs[0];
     const AVFilterLink *inlink = ctx->inputs[in->idx];
     const AVFrame *ref = ref_frame(&in->mix);
 
@@ -900,10 +916,33 @@ static void update_crops(AVFilterContext *ctx, LibplaceboInput *in,
             target->crop.y0 = av_expr_eval(s->pos_y_pexpr, s->var_values, NULL);
             target->crop.x1 = target->crop.x0 + s->var_values[VAR_POS_W];
             target->crop.y1 = target->crop.y0 + s->var_values[VAR_POS_H];
-            if (s->normalize_sar) {
-                float aspect = pl_rect2df_aspect(&image->crop);
-                aspect *= av_q2d(inlink->sample_aspect_ratio);
-                pl_rect2df_aspect_set(&target->crop, aspect, s->pad_crop_ratio);
+
+            /* Effective visual crop */
+            pl_rect2df fixed = image->crop;
+            float aspect = pl_rect2df_aspect(&fixed);
+            aspect *= av_q2d(inlink->sample_aspect_ratio);
+            aspect /= av_q2d(outlink->sample_aspect_ratio);
+            pl_rect2df_aspect_set(&fixed, aspect, 0.5);
+
+            switch (s->fit_mode) {
+            case FIT_FILL:
+                if (s->normalize_sar)
+                    pl_rect2df_aspect_set(&target->crop, aspect, s->pad_crop_ratio);
+                break;
+            case FIT_CONTAIN:
+                pl_rect2df_aspect_set(&target->crop, aspect, 0.0);
+                break;
+            case FIT_COVER:
+                pl_rect2df_aspect_set(&target->crop, aspect, 1.0);
+                break;
+            case FIT_NONE: {
+                const float sx = fabsf(pl_rect_w(fixed)) / pl_rect_w(target->crop);
+                const float sy = fabsf(pl_rect_h(fixed)) / pl_rect_h(target->crop);
+                pl_rect2df_stretch(&target->crop, sx, sy);
+                break;
+            }
+            case FIT_SCALE_DOWN:
+                pl_rect2df_aspect_fit(&target->crop, &fixed, 0.0);
             }
         }
     }
@@ -1540,6 +1579,12 @@ static const AVOption libplacebo_options[] = {
     { "reset_sar", "force SAR normalization to 1:1 by adjusting pos_x/y/w/h", OFFSET(reset_sar), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, STATIC },
     { "normalize_sar", "like reset_sar, but pad/crop instead of stretching the video", OFFSET(normalize_sar), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, STATIC },
     { "pad_crop_ratio", "ratio between padding and cropping when normalizing SAR (0=pad, 1=crop)", OFFSET(pad_crop_ratio), AV_OPT_TYPE_FLOAT, {.dbl=0.0}, 0.0, 1.0, DYNAMIC },
+    { "fit_mode", "content fit strategy for placing input layers in the output", OFFSET(fit_mode), AV_OPT_TYPE_INT, {.i64 = FIT_FILL }, 0, FIT_MODE_NB - 1, STATIC, .unit = "fit_mode" },
+        { "fill",       "Stretch content, ignoring aspect ratio",               0, AV_OPT_TYPE_CONST, {.i64 = FIT_FILL },       0, 0, STATIC, .unit = "fit_mode" },
+        { "contain",    "Stretch content, padding to preserve aspect",          0, AV_OPT_TYPE_CONST, {.i64 = FIT_CONTAIN },    0, 0, STATIC, .unit = "fit_mode" },
+        { "cover",      "Stretch content, cropping to preserve aspect",         0, AV_OPT_TYPE_CONST, {.i64 = FIT_COVER },      0, 0, STATIC, .unit = "fit_mode" },
+        { "none",       "Keep input unscaled, padding and cropping as needed",  0, AV_OPT_TYPE_CONST, {.i64 = FIT_NONE },       0, 0, STATIC, .unit = "fit_mode" },
+        { "scale_down", "Downscale only if larger, padding to preserve aspect", 0, AV_OPT_TYPE_CONST, {.i64 = FIT_SCALE_DOWN }, 0, 0, STATIC, .unit = "fit_mode" },
     { "fillcolor", "Background fill color", OFFSET(fillcolor), AV_OPT_TYPE_COLOR, {.str = "black@0"}, .flags = DYNAMIC },
     { "corner_rounding", "Corner rounding radius", OFFSET(corner_rounding), AV_OPT_TYPE_FLOAT, {.dbl = 0.0}, 0.0, 1.0, .flags = DYNAMIC },
     { "lut", "Path to custom LUT file to apply", OFFSET(lut_filename), AV_OPT_TYPE_STRING, { .str = NULL }, .flags = STATIC },
-- 
2.49.1

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org