[FFmpeg-devel] Re: [PATCH RFC] libavfilter: expose expression variable metadata via AVFilter struct

[Nicolas George via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>]

Len Woodward via ffmpeg-devel (HE12026-01-09):
> ## Prior Discussion
> 
> A search of the mailing list archives found no prior proposals for this
> specific feature. The closest related discussion is the [2013 Scripting
> RFC](https://ffmpeg.org/pipermail/ffmpeg-devel/2013-November/151069.html),
> which raised the broader challenge of binding the lavfi API to other
> languages. This proposal addresses one narrow aspect of that: making filter
> expression variables discoverable.
> 
> ## Problem
> 
> Filters like `drawtext`, `geq`, and `overlay` accept expressions
> referencing variables such as `w`, `h`, `t`, `n`, etc. These variables are
> documented in FFmpeg's official documentation and used in countless filter
> strings worldwide—but they're not programmatically discoverable.
> 
> Applications building filter graphs programmatically (GUIs, language
> bindings, video editors) currently must duplicate FFmpeg's internal
> `var_names` arrays and manually keep them in sync. This is fragile and
> creates maintenance burden across the ecosystem.
> 
> ## Why These Are Already Stable API
> 
> These variables are documented, user-facing, and cannot be changed without
> breaking existing filter strings. The stability contract already
> exists—this proposal simply extends programmatic access to match what
> string-based users already have.
> 
> ## Proposed Solution
> 
> Add an `expression_vars` field to the `AVFilter` struct, following the
> pattern established by `AVOption` for filter options.

Hi. I think it is an excellent idea. See comments below.

> 
> **Public API addition (avfilter.h):**
> 
> ```c
> typedef struct AVFilterExprVar {
>     const char *name;    /* Variable name (e.g., "text_w") */
>     const char *help;    /* Description (e.g., "Width of rendered text") */
> } AVFilterExprVar;

Filters are the main users of expressions and variables, but they are
not the only ones. Therefore, I think this API should exist in
libavutil, not libavfilter.

> 
> typedef struct AVFilter {
>     // ... existing fields ...
> 
>     /**
>      * NULL-terminated array of expression variables accepted by this
> filter,
>      * or NULL if the filter does not use expression evaluation.
>      */
>     const AVFilterExprVar *expression_vars;
> } AVFilter;
> ```
> 
> **Filter implementation (e.g., vf_drawtext.c):**
> 
> ```c
> // Existing array - unchanged, used internally by av_expr_parse(),
> // ff_print_eval_expr(), and other internal functions
> static const char *const var_names[] = {
>     "dar", "h", "w", "n", "t", "text_h", "text_w",
>     // ...
>     NULL
> };
> 
> // New array - public metadata with documentation
> static const AVFilterExprVar drawtext_expr_vars[] = {
>     { "dar",    "Display aspect ratio" },
>     { "h",      "Video height" },
>     { "w",      "Video width" },
>     { "n",      "Frame number (starting at 0)" },
>     { "t",      "Timestamp in seconds" },
>     { "text_h", "Height of rendered text" },
>     { "text_w", "Width of rendered text" },
>     // ...
>     { NULL }
> };
> 
> const AVFilter ff_vf_drawtext = {
>     .name            = "drawtext",
>     // ... existing fields ...
>     .expression_vars = drawtext_expr_vars,
> };
> ```
> 
> **Application usage:**
> 
> ```c
> const AVFilter *f = avfilter_get_by_name("drawtext");
> if (f->expression_vars) {
>     for (const AVFilterExprVar *v = f->expression_vars; v->name; v++)
>         printf("  %s - %s\n", v->name, v->help);
> }
> ```
> 
> ## Why This Approach
> 
> 1. **Follows established precedent**: Mirrors how `AVOption` exposes filter
> options. Developers already understand this pattern.
> 2. **Discoverable**: Given any `AVFilter*`, check if it has expression
> variables. No need for per-filter symbol knowledge.
> 3. **Self-documenting**: Help text (already written in `.texi` files)
> becomes machine-readable.
> 4. **Purely additive**: Existing internal functions (`av_expr_parse()`,
> `ff_print_eval_expr()`, `ff_print_formatted_eval_expr()`, etc.) continue
> using `var_names` unchanged. No signature changes, no risk to existing code
> paths.
> 5. **Minor release compatible**: This can land in a minor release since it
> adds functionality without modifying existing behavior.

> 6. **Extensible**: The struct can later gain fields (type hints, flags)
> without breaking users.

You are mistaken on this point: adding fields to the structure would be
compatible at the API level but at the ABI level it would invalidate the
“v++” in the loop in “application usage” above.

That can be fixed by using a function to access each element, something
like “av_expr_get_variable(exprctx, index)”. Since we are in the
proximity of parsing and evaluating an expression, the overhead is not a
concern.

> 
> ## On Array Duplication
> 
> Yes, this means filters have two arrays with overlapping content. This is
> intentional:
> 
> - Both arrays live in the same file, adjacent to each other—drift is easy
> to spot during review
> - The existing `var_names` array is used by multiple internal functions;
> changing all of them is a larger undertaking better suited for a major
> release
> - The cost is a few hundred bytes of static storage per filter
> 
> If source-level duplication is a concern, an X-macro can generate both
> arrays from a single definition:
> 
> ```c
> #define DRAWTEXT_EXPR_VARS \
>     X("dar",    "Display aspect ratio") \
>     X("h",      "Video height") \
>     X("w",      "Video width")
>     // ...
> 
> #define X(name, help) name,
> static const char *const var_names[] = { DRAWTEXT_EXPR_VARS NULL };
> #undef X
> 
> #define X(name, help) { name, help },
> static const AVFilterExprVar drawtext_expr_vars[] = { DRAWTEXT_EXPR_VARS {
> NULL } };
> #undef X
> ```
> 
> This eliminates any possibility of drift between arrays. Either approach
> works—maintainer preference.
> 

> A future major release could consolidate to a single array by updating
> internal functions to accept `AVFilterExprVar*`, but that's a separate,
> larger effort. This proposal delivers value now without that scope.

I agree, the AVExpr API needs to be cleaned up and it would take care of
the duplication issue.

> 
> ## Simpler Alternatives
> 
> If extending `AVFilter` is undesirable, these alternatives also solve the
> immediate problem:
> 
> **Export existing symbols:**
> ```diff
> -static const char *const var_names[] = {
> +const char *const ff_drawtext_var_names[] = {
> ```
> 
> **Per-filter accessor:**
> ```c
> const char *const *avfilter_drawtext_get_var_names(void);
> ```
> 
> Both work, but lack discoverability and metadata. If committing to API
> stability for expression variables, the `AVFilter` struct approach provides
> better long-term value.

I think the add to AVFilter is preferable. Or maybe AVFilterContext,
depending on the details of how we want to design it.

> ## Scope
> 
> I'm prepared to submit patches for:
> 
> 1. `AVFilterExprVar` struct and `AVFilter` field addition
> 2. Migration of `drawtext` as proof of concept
> 3. Migration of other expression-enabled filters if desired
> 4. Documentation updates
> 
> Happy to start with `drawtext` only, or cover all filters—whichever
> maintainers prefer.

Thanks.

Regards,

-- 
  Nicolas George
_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org