[FFmpeg-devel] [PATCH RFC] libavfilter: expose expression variable metadata via AVFilter struct

[FFmpeg-devel] [PATCH RFC] libavfilter: expose expression variable metadata via AVFilter struct

[Len Woodward via ffmpeg-devel]

## Prior Discussion

A search of the mailing list archives found no prior proposals for this
specific feature. The closest related discussion is the [2013 Scripting
RFC](https://ffmpeg.org/pipermail/ffmpeg-devel/2013-November/151069.html),
which raised the broader challenge of binding the lavfi API to other
languages. This proposal addresses one narrow aspect of that: making filter
expression variables discoverable.

## Problem

Filters like `drawtext`, `geq`, and `overlay` accept expressions
referencing variables such as `w`, `h`, `t`, `n`, etc. These variables are
documented in FFmpeg's official documentation and used in countless filter
strings worldwide—but they're not programmatically discoverable.

Applications building filter graphs programmatically (GUIs, language
bindings, video editors) currently must duplicate FFmpeg's internal
`var_names` arrays and manually keep them in sync. This is fragile and
creates maintenance burden across the ecosystem.

## Why These Are Already Stable API

These variables are documented, user-facing, and cannot be changed without
breaking existing filter strings. The stability contract already
exists—this proposal simply extends programmatic access to match what
string-based users already have.

## Proposed Solution

Add an `expression_vars` field to the `AVFilter` struct, following the
pattern established by `AVOption` for filter options.

**Public API addition (avfilter.h):**

```c
typedef struct AVFilterExprVar {
    const char *name;    /* Variable name (e.g., "text_w") */
    const char *help;    /* Description (e.g., "Width of rendered text") */
} AVFilterExprVar;

typedef struct AVFilter {
    // ... existing fields ...

    /**
     * NULL-terminated array of expression variables accepted by this
filter,
     * or NULL if the filter does not use expression evaluation.
     */
    const AVFilterExprVar *expression_vars;
} AVFilter;
```

**Filter implementation (e.g., vf_drawtext.c):**

```c
// Existing array - unchanged, used internally by av_expr_parse(),
// ff_print_eval_expr(), and other internal functions
static const char *const var_names[] = {
    "dar", "h", "w", "n", "t", "text_h", "text_w",
    // ...
    NULL
};

// New array - public metadata with documentation
static const AVFilterExprVar drawtext_expr_vars[] = {
    { "dar",    "Display aspect ratio" },
    { "h",      "Video height" },
    { "w",      "Video width" },
    { "n",      "Frame number (starting at 0)" },
    { "t",      "Timestamp in seconds" },
    { "text_h", "Height of rendered text" },
    { "text_w", "Width of rendered text" },
    // ...
    { NULL }
};

const AVFilter ff_vf_drawtext = {
    .name            = "drawtext",
    // ... existing fields ...
    .expression_vars = drawtext_expr_vars,
};
```

**Application usage:**

```c
const AVFilter *f = avfilter_get_by_name("drawtext");
if (f->expression_vars) {
    for (const AVFilterExprVar *v = f->expression_vars; v->name; v++)
        printf("  %s - %s\n", v->name, v->help);
}
```

## Why This Approach

1. **Follows established precedent**: Mirrors how `AVOption` exposes filter
options. Developers already understand this pattern.
2. **Discoverable**: Given any `AVFilter*`, check if it has expression
variables. No need for per-filter symbol knowledge.
3. **Self-documenting**: Help text (already written in `.texi` files)
becomes machine-readable.
4. **Purely additive**: Existing internal functions (`av_expr_parse()`,
`ff_print_eval_expr()`, `ff_print_formatted_eval_expr()`, etc.) continue
using `var_names` unchanged. No signature changes, no risk to existing code
paths.
5. **Minor release compatible**: This can land in a minor release since it
adds functionality without modifying existing behavior.
6. **Extensible**: The struct can later gain fields (type hints, flags)
without breaking users.

## On Array Duplication

Yes, this means filters have two arrays with overlapping content. This is
intentional:

- Both arrays live in the same file, adjacent to each other—drift is easy
to spot during review
- The existing `var_names` array is used by multiple internal functions;
changing all of them is a larger undertaking better suited for a major
release
- The cost is a few hundred bytes of static storage per filter

If source-level duplication is a concern, an X-macro can generate both
arrays from a single definition:

```c
#define DRAWTEXT_EXPR_VARS \
    X("dar",    "Display aspect ratio") \
    X("h",      "Video height") \
    X("w",      "Video width")
    // ...

#define X(name, help) name,
static const char *const var_names[] = { DRAWTEXT_EXPR_VARS NULL };
#undef X

#define X(name, help) { name, help },
static const AVFilterExprVar drawtext_expr_vars[] = { DRAWTEXT_EXPR_VARS {
NULL } };
#undef X
```

This eliminates any possibility of drift between arrays. Either approach
works—maintainer preference.

A future major release could consolidate to a single array by updating
internal functions to accept `AVFilterExprVar*`, but that's a separate,
larger effort. This proposal delivers value now without that scope.

## Simpler Alternatives

If extending `AVFilter` is undesirable, these alternatives also solve the
immediate problem:

**Export existing symbols:**
```diff
-static const char *const var_names[] = {
+const char *const ff_drawtext_var_names[] = {
```

**Per-filter accessor:**
```c
const char *const *avfilter_drawtext_get_var_names(void);
```

Both work, but lack discoverability and metadata. If committing to API
stability for expression variables, the `AVFilter` struct approach provides
better long-term value.

## Scope

I'm prepared to submit patches for:

1. `AVFilterExprVar` struct and `AVFilter` field addition
2. Migration of `drawtext` as proof of concept
3. Migration of other expression-enabled filters if desired
4. Documentation updates

Happy to start with `drawtext` only, or cover all filters—whichever
maintainers prefer.

---

I've encountered this limitation building language bindings for FFmpeg
filters. This is a known pain point for projects that build filter graphs
programmatically—any application that validates user expressions, provides
autocomplete, or generates documentation faces this issue.

This RFC seeks feedback on the approach before submitting patches. If
there's interest, I'll follow up with a proper patch series including
Doxygen documentation and version bump.
_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

[FFmpeg-devel] Re: [PATCH RFC] libavfilter: expose expression variable metadata via AVFilter struct

[Nicolas George via ffmpeg-devel]

Len Woodward via ffmpeg-devel (HE12026-01-09):
> ## Prior Discussion
> 
> A search of the mailing list archives found no prior proposals for this
> specific feature. The closest related discussion is the [2013 Scripting
> RFC](https://ffmpeg.org/pipermail/ffmpeg-devel/2013-November/151069.html),
> which raised the broader challenge of binding the lavfi API to other
> languages. This proposal addresses one narrow aspect of that: making filter
> expression variables discoverable.
> 
> ## Problem
> 
> Filters like `drawtext`, `geq`, and `overlay` accept expressions
> referencing variables such as `w`, `h`, `t`, `n`, etc. These variables are
> documented in FFmpeg's official documentation and used in countless filter
> strings worldwide—but they're not programmatically discoverable.
> 
> Applications building filter graphs programmatically (GUIs, language
> bindings, video editors) currently must duplicate FFmpeg's internal
> `var_names` arrays and manually keep them in sync. This is fragile and
> creates maintenance burden across the ecosystem.
> 
> ## Why These Are Already Stable API
> 
> These variables are documented, user-facing, and cannot be changed without
> breaking existing filter strings. The stability contract already
> exists—this proposal simply extends programmatic access to match what
> string-based users already have.
> 
> ## Proposed Solution
> 
> Add an `expression_vars` field to the `AVFilter` struct, following the
> pattern established by `AVOption` for filter options.

Hi. I think it is an excellent idea. See comments below.

> 
> **Public API addition (avfilter.h):**
> 
> ```c
> typedef struct AVFilterExprVar {
>     const char *name;    /* Variable name (e.g., "text_w") */
>     const char *help;    /* Description (e.g., "Width of rendered text") */
> } AVFilterExprVar;

Filters are the main users of expressions and variables, but they are
not the only ones. Therefore, I think this API should exist in
libavutil, not libavfilter.

> 
> typedef struct AVFilter {
>     // ... existing fields ...
> 
>     /**
>      * NULL-terminated array of expression variables accepted by this
> filter,
>      * or NULL if the filter does not use expression evaluation.
>      */
>     const AVFilterExprVar *expression_vars;
> } AVFilter;
> ```
> 
> **Filter implementation (e.g., vf_drawtext.c):**
> 
> ```c
> // Existing array - unchanged, used internally by av_expr_parse(),
> // ff_print_eval_expr(), and other internal functions
> static const char *const var_names[] = {
>     "dar", "h", "w", "n", "t", "text_h", "text_w",
>     // ...
>     NULL
> };
> 
> // New array - public metadata with documentation
> static const AVFilterExprVar drawtext_expr_vars[] = {
>     { "dar",    "Display aspect ratio" },
>     { "h",      "Video height" },
>     { "w",      "Video width" },
>     { "n",      "Frame number (starting at 0)" },
>     { "t",      "Timestamp in seconds" },
>     { "text_h", "Height of rendered text" },
>     { "text_w", "Width of rendered text" },
>     // ...
>     { NULL }
> };
> 
> const AVFilter ff_vf_drawtext = {
>     .name            = "drawtext",
>     // ... existing fields ...
>     .expression_vars = drawtext_expr_vars,
> };
> ```
> 
> **Application usage:**
> 
> ```c
> const AVFilter *f = avfilter_get_by_name("drawtext");
> if (f->expression_vars) {
>     for (const AVFilterExprVar *v = f->expression_vars; v->name; v++)
>         printf("  %s - %s\n", v->name, v->help);
> }
> ```
> 
> ## Why This Approach
> 
> 1. **Follows established precedent**: Mirrors how `AVOption` exposes filter
> options. Developers already understand this pattern.
> 2. **Discoverable**: Given any `AVFilter*`, check if it has expression
> variables. No need for per-filter symbol knowledge.
> 3. **Self-documenting**: Help text (already written in `.texi` files)
> becomes machine-readable.
> 4. **Purely additive**: Existing internal functions (`av_expr_parse()`,
> `ff_print_eval_expr()`, `ff_print_formatted_eval_expr()`, etc.) continue
> using `var_names` unchanged. No signature changes, no risk to existing code
> paths.
> 5. **Minor release compatible**: This can land in a minor release since it
> adds functionality without modifying existing behavior.

> 6. **Extensible**: The struct can later gain fields (type hints, flags)
> without breaking users.

You are mistaken on this point: adding fields to the structure would be
compatible at the API level but at the ABI level it would invalidate the
“v++” in the loop in “application usage” above.

That can be fixed by using a function to access each element, something
like “av_expr_get_variable(exprctx, index)”. Since we are in the
proximity of parsing and evaluating an expression, the overhead is not a
concern.

> 
> ## On Array Duplication
> 
> Yes, this means filters have two arrays with overlapping content. This is
> intentional:
> 
> - Both arrays live in the same file, adjacent to each other—drift is easy
> to spot during review
> - The existing `var_names` array is used by multiple internal functions;
> changing all of them is a larger undertaking better suited for a major
> release
> - The cost is a few hundred bytes of static storage per filter
> 
> If source-level duplication is a concern, an X-macro can generate both
> arrays from a single definition:
> 
> ```c
> #define DRAWTEXT_EXPR_VARS \
>     X("dar",    "Display aspect ratio") \
>     X("h",      "Video height") \
>     X("w",      "Video width")
>     // ...
> 
> #define X(name, help) name,
> static const char *const var_names[] = { DRAWTEXT_EXPR_VARS NULL };
> #undef X
> 
> #define X(name, help) { name, help },
> static const AVFilterExprVar drawtext_expr_vars[] = { DRAWTEXT_EXPR_VARS {
> NULL } };
> #undef X
> ```
> 
> This eliminates any possibility of drift between arrays. Either approach
> works—maintainer preference.
> 

> A future major release could consolidate to a single array by updating
> internal functions to accept `AVFilterExprVar*`, but that's a separate,
> larger effort. This proposal delivers value now without that scope.

I agree, the AVExpr API needs to be cleaned up and it would take care of
the duplication issue.

> 
> ## Simpler Alternatives
> 
> If extending `AVFilter` is undesirable, these alternatives also solve the
> immediate problem:
> 
> **Export existing symbols:**
> ```diff
> -static const char *const var_names[] = {
> +const char *const ff_drawtext_var_names[] = {
> ```
> 
> **Per-filter accessor:**
> ```c
> const char *const *avfilter_drawtext_get_var_names(void);
> ```
> 
> Both work, but lack discoverability and metadata. If committing to API
> stability for expression variables, the `AVFilter` struct approach provides
> better long-term value.

I think the add to AVFilter is preferable. Or maybe AVFilterContext,
depending on the details of how we want to design it.

> ## Scope
> 
> I'm prepared to submit patches for:
> 
> 1. `AVFilterExprVar` struct and `AVFilter` field addition
> 2. Migration of `drawtext` as proof of concept
> 3. Migration of other expression-enabled filters if desired
> 4. Documentation updates
> 
> Happy to start with `drawtext` only, or cover all filters—whichever
> maintainers prefer.

Thanks.

Regards,

-- 
  Nicolas George
_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org