[FFmpeg-devel] [PATCH] avfilter: POC: enable out-of-tree filters

[FFmpeg-devel] [PATCH] avfilter: POC: enable out-of-tree filters

[Leandro Santiago]

This is a POC/prototype that aims to enable out of tree filters on
FFmpeg.

Here I name them "extra filters".

It introduces the program `jq` as a new build dependency.

To test it, create a directory, for instance, /tmp/my-shiny-filter/ and
inside it, create the following files:

`filter.json`, with the content:

```
{
  "check": "require_pkg_config json json-c json-c/json.h json_c_version_num",
  "symbols": ["ff_vf_foo", "ff_vf_bar"]
}
```

`filter.mak`, with the content:

```
OBJS += vf_shiny.o
LIBOBJS += vf_shiny.o

libavfilter/vf_shiny.o:
        $(CC) $(EXTRA_FILTER_FLAGS) -c -o $@ $(EXTRA_FILTER_FOO_LOCATION)/vf_shiny.c
```

`vf_shiny.c` file, with the content:


```
#include "libavutil/internal.h"                                                                                                                       
#include "avfilter.h"                                                                                                                                 #include "filters.h"                                                                                                                                  
#include "video.h" 

const FFFilter ff_vf_bar = {
    .p.name        = "bar",
    .p.description = NULL_IF_CONFIG_SMALL("Example filter Baz"),
    .p.flags       = AVFILTER_FLAG_METADATA_ONLY,
    FILTER_INPUTS(ff_video_default_filterpad),
    FILTER_OUTPUTS(ff_video_default_filterpad),
};

const FFFilter ff_vf_foo = {
    .p.name        = "foo",
    .p.description = NULL_IF_CONFIG_SMALL("Another foo filter"),
    .p.flags       = AVFILTER_FLAG_METADATA_ONLY,
    FILTER_INPUTS(ff_video_default_filterpad),
    FILTER_OUTPUTS(ff_video_default_filterpad),
};
```

Then, from the ffmpeg source tree, run configure specifying where the
extra filter is located:

```
./configure --extra-filter=/tmp/my-shiny-filter
make ffplay
```

Now you can use the filters:

```
./ffplay /path/to/file.webm -vf 'foo,baz'
```

What works:

- Building C based filters with no extra dependencies.
- Multiple filters in the same object file.

What does not work:

- The extra filters will not use the same CC flags used to build the
  built-in filters as I could get it to work yet.
- Due to the above limitation, you cannot include headers of extra
  dependencies, for instance, `json.h` in the example.
- You can pass arbitrary CFLAGS or LDFLAGS in the filter.json file,
  but they should be passed only then building/linking `libavfilter`,
  instead of other libraries.

What was not implemented:

- I believe it would be useful to check if the license of the filter is
  compatible with the license used to build FFmpeg.
- Only extra filters written in C (maybe C++?) are supported for now.
  One of my goals is to enable Rust as well.

Signed-off-by: Leandro Santiago <leandrosansilva@gmail.com>
---
 .gitignore               |  3 ++
 configure                | 61 ++++++++++++++++++++++++++++++++++++++++
 libavfilter/Makefile     |  4 +++
 libavfilter/allfilters.c |  1 +
 4 files changed, 69 insertions(+)

diff --git a/.gitignore b/.gitignore
index 9cfc78b414..4963e90191 100644
--- a/.gitignore
+++ b/.gitignore
@@ -43,3 +43,6 @@
 /tools/python/__pycache__/
 /libavcodec/vulkan/*.c
 /libavfilter/vulkan/*.c
+/ffbuild/extra-filters.txt
+/ffbuild/extra-filters.mak
+/libavfilter/extra_filters_extern.h
diff --git a/configure b/configure
index 750c99e3b9..6a2adc6c05 100755
--- a/configure
+++ b/configure
@@ -179,6 +179,7 @@ Individual component options:
   --enable-filter=NAME     enable filter NAME
   --disable-filter=NAME    disable filter NAME
   --disable-filters        disable all filters
+  --extra-filter=/foo/bar  add extra filter from directory. This option can be used multiple times
 
 External library support:
 
@@ -1798,6 +1799,7 @@ AVDEVICE_COMPONENTS="
 
 AVFILTER_COMPONENTS="
     filters
+    extra_filters
 "
 
 AVFORMAT_COMPONENTS="
@@ -4382,6 +4384,8 @@ do_random(){
     $action $(rand_list "$@" | awk "BEGIN { srand($random_seed) } \$1 == \"prob\" { prob = \$2; next } rand() < prob { print }")
 }
 
+rm -f ffbuild/extra-filters.txt
+
 # deprecated components (disabled by default)
 disable sonic_encoder sonic_ls_encoder
 
@@ -4457,6 +4461,10 @@ for opt do
                 die_unknown $opt
             fi
         ;;
+        --extra-filter=*)
+            filter_path="${opt#--extra-filter=}"
+            echo "$filter_path" >> ffbuild/extra-filters.txt
+        ;;
         --list-*)
             NAME="${opt#--list-}"
             is_in $NAME $COMPONENT_LIST || die_unknown $opt
@@ -4487,6 +4495,36 @@ for opt do
     esac
 done
 
+find_extra_filters_extern() {
+  # TODO: handle invalid filter
+  while read f; do
+    jq -r '.symbols[]' < "$f/filter.json" | sed 's/^ff_[avfsinkrc]\{2,5\}_\([[:alnum:]_]\{1,\}\)/\1_filter/'
+  done < ffbuild/extra-filters.txt
+}
+
+EXTRA_FILTER_LIST=$(find_extra_filters_extern)
+
+for n in extra_filters; do
+    v=$(toupper ${n%s})_LIST
+    eval enable \$$v
+    eval ${n}_if_any="\$$v"
+done
+
+FILTER_LIST="
+    $FILTER_LIST
+    $EXTRA_FILTER_LIST
+"
+
+AVFILTER_COMPONENTS_LIST="
+    $AVFILTER_COMPONENTS_LIST
+    $EXTRA_FILTER_LIST
+"
+
+ALL_COMPONENTS="
+    $ALL_COMPONENTS
+    $EXTRA_FILTER_LIST
+"
+
 for e in $env; do
     eval "export $e"
 done
@@ -7165,6 +7203,10 @@ enabled rkmpp             && { require_pkg_config rkmpp rockchip_mpp  rockchip/r
                              }
 enabled vapoursynth       && require_headers "vapoursynth/VSScript4.h vapoursynth/VapourSynth4.h"
 
+while read f; do
+  # NOTE: this eval is dangerous, as it allows arbitrary code execution!
+  eval $(jq -r '.check // true' < "$f/filter.json")
+done < ffbuild/extra-filters.txt
 
 if enabled gcrypt; then
     GCRYPT_CONFIG="${cross_prefix}libgcrypt-config"
@@ -8243,6 +8285,12 @@ for entry in $LIBRARY_LIST $PROGRAM_LIST $EXTRALIBS_LIST; do
     eval echo "EXTRALIBS-${entry}=\$${entry}_extralibs" >> ffbuild/config.mak
 done
 
+echo "" > ffbuild/extra-filters.mak
+
+while read f; do
+    echo "include $f/filter.mak" >> ffbuild/extra-filters.mak
+done < ffbuild/extra-filters.txt
+
 cat > $TMPH <<EOF
 /* Automatically generated by configure - do not modify! */
 #ifndef FFMPEG_CONFIG_H
@@ -8330,6 +8378,19 @@ cp_if_changed $TMPH libavutil/avconfig.h
 # ...
 eval "$(sed -n "s/^extern const FFFilter ff_\([avfsinkrc]\{2,5\}\)_\(.*\);/full_filter_name_\2=\1_\2/p" $source_path/libavfilter/allfilters.c)"
 
+rm -f libavfilter/extra_filters_extern.h
+
+# Handle extra filters
+while read f; do
+    eval "$(jq -r '.symbols[]' < "$f/filter.json" | sed 's/^ff_\([avfsinkrc]\{2,5\}\)_\([[:alnum:]]\{1,\}\)$/full_filter_name_\2=\1_\2/')"
+    jq -r '.symbols[]' < "$f/filter.json" | while read symbol; do
+      echo "extern const FFFilter $symbol;" >> libavfilter/extra_filters_extern.h
+      echo "EXTRA_FILTER_$(echo $symbol | sed 's/^ff_[avfsinkrc]\{2,5\}_\([[:alnum:]]\{1,\}\)$/\1/' | tr a-z A-Z)_LOCATION = $f" >> ffbuild/extra-filters.mak
+      echo "LDFLAGS += $(jq -r '.ldflags // ""' < "$f/filter.json")" >> ffbuild/extra-filters.mak
+      echo "CFLAGS += $(jq -r '.cflags // ""' < "$f/filter.json")" >> ffbuild/extra-filters.mak
+    done
+done < ffbuild/extra-filters.txt
+
 # generate the lists of enabled components
 print_enabled_components(){
     file=$1
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 7c0d879ec9..9b22aece3a 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -27,6 +27,10 @@ OBJS = allfilters.o                                                     \
 include $(SRC_PATH)/libavfilter/dnn/Makefile
 include $(SRC_PATH)/libavfilter/vulkan/Makefile
 
+# extra filters handling
+include $(SRC_PATH)/ffbuild/extra-filters.mak
+EXTRA_FILTER_FLAGS = -I$(PWD) -I$(PWD)/libavfilter $(CPPFLAGS) $(CC_DEPFLAGS)
+
 OBJS-$(HAVE_LIBC_MSVCRT)                     += file_open.o
 OBJS-$(HAVE_THREADS)                         += pthread.o
 
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 740d9ab265..e8565de5b0 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -621,6 +621,7 @@ extern  const FFFilter ff_vsrc_buffer;
 extern  const FFFilter ff_asink_abuffer;
 extern  const FFFilter ff_vsink_buffer;
 
+#include "libavfilter/extra_filters_extern.h"
 #include "libavfilter/filter_list.c"
 
 
-- 
2.48.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: POC: enable out-of-tree filters

[Leandro Santiago]

This is a follow-up to the my previous message:

https://ffmpeg.org/pipermail/ffmpeg-devel/2025-March/340895.html

On 3/13/25 13:18, Leandro Santiago wrote:
> This is a POC/prototype that aims to enable out of tree filters on
> FFmpeg.
>
> Here I name them "extra filters".
>
> It introduces the program `jq` as a new build dependency.
>
> To test it, create a directory, for instance, /tmp/my-shiny-filter/ and
> inside it, create the following files:
>
> `filter.json`, with the content:
>
> ```
> {
>   "check": "require_pkg_config json json-c json-c/json.h json_c_version_num",
>   "symbols": ["ff_vf_foo", "ff_vf_bar"]
> }
> ```
>
> `filter.mak`, with the content:
>
> ```
> OBJS += vf_shiny.o
> LIBOBJS += vf_shiny.o
>
> libavfilter/vf_shiny.o:
>         $(CC) $(EXTRA_FILTER_FLAGS) -c -o $@ $(EXTRA_FILTER_FOO_LOCATION)/vf_shiny.c
> ```
>
> `vf_shiny.c` file, with the content:
>
>
> ```
> #include "libavutil/internal.h"                                                                                                                       
> #include "avfilter.h"                                                                                                                                 #include "filters.h"                                                                                                                                  
> #include "video.h" 
>
> const FFFilter ff_vf_bar = {
>     .p.name        = "bar",
>     .p.description = NULL_IF_CONFIG_SMALL("Example filter Baz"),
>     .p.flags       = AVFILTER_FLAG_METADATA_ONLY,
>     FILTER_INPUTS(ff_video_default_filterpad),
>     FILTER_OUTPUTS(ff_video_default_filterpad),
> };
>
> const FFFilter ff_vf_foo = {
>     .p.name        = "foo",
>     .p.description = NULL_IF_CONFIG_SMALL("Another foo filter"),
>     .p.flags       = AVFILTER_FLAG_METADATA_ONLY,
>     FILTER_INPUTS(ff_video_default_filterpad),
>     FILTER_OUTPUTS(ff_video_default_filterpad),
> };
> ```
>
> Then, from the ffmpeg source tree, run configure specifying where the
> extra filter is located:
>
> ```
> ./configure --extra-filter=/tmp/my-shiny-filter
> make ffplay
> ```
>
> Now you can use the filters:
>
> ```
> ./ffplay /path/to/file.webm -vf 'foo,baz'
> ```
>
> What works:
>
> - Building C based filters with no extra dependencies.
> - Multiple filters in the same object file.
>
> What does not work:
>
> - The extra filters will not use the same CC flags used to build the
>   built-in filters as I could get it to work yet.
> - Due to the above limitation, you cannot include headers of extra
>   dependencies, for instance, `json.h` in the example.
> - You can pass arbitrary CFLAGS or LDFLAGS in the filter.json file,
>   but they should be passed only then building/linking `libavfilter`,
>   instead of other libraries.
>
> What was not implemented:
>
> - I believe it would be useful to check if the license of the filter is
>   compatible with the license used to build FFmpeg.
> - Only extra filters written in C (maybe C++?) are supported for now.
>   One of my goals is to enable Rust as well.
>
> Signed-off-by: Leandro Santiago <leandrosansilva@gmail.com>
> ---
>  .gitignore               |  3 ++
>  configure                | 61 ++++++++++++++++++++++++++++++++++++++++
>  libavfilter/Makefile     |  4 +++
>  libavfilter/allfilters.c |  1 +
>  4 files changed, 69 insertions(+)
>
> diff --git a/.gitignore b/.gitignore
> index 9cfc78b414..4963e90191 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -43,3 +43,6 @@
>  /tools/python/__pycache__/
>  /libavcodec/vulkan/*.c
>  /libavfilter/vulkan/*.c
> +/ffbuild/extra-filters.txt
> +/ffbuild/extra-filters.mak
> +/libavfilter/extra_filters_extern.h
> diff --git a/configure b/configure
> index 750c99e3b9..6a2adc6c05 100755
> --- a/configure
> +++ b/configure
> @@ -179,6 +179,7 @@ Individual component options:
>    --enable-filter=NAME     enable filter NAME
>    --disable-filter=NAME    disable filter NAME
>    --disable-filters        disable all filters
> +  --extra-filter=/foo/bar  add extra filter from directory. This option can be used multiple times
>  
>  External library support:
>  
> @@ -1798,6 +1799,7 @@ AVDEVICE_COMPONENTS="
>  
>  AVFILTER_COMPONENTS="
>      filters
> +    extra_filters
>  "
>  
>  AVFORMAT_COMPONENTS="
> @@ -4382,6 +4384,8 @@ do_random(){
>      $action $(rand_list "$@" | awk "BEGIN { srand($random_seed) } \$1 == \"prob\" { prob = \$2; next } rand() < prob { print }")
>  }
>  
> +rm -f ffbuild/extra-filters.txt
> +
>  # deprecated components (disabled by default)
>  disable sonic_encoder sonic_ls_encoder
>  
> @@ -4457,6 +4461,10 @@ for opt do
>                  die_unknown $opt
>              fi
>          ;;
> +        --extra-filter=*)
> +            filter_path="${opt#--extra-filter=}"
> +            echo "$filter_path" >> ffbuild/extra-filters.txt
> +        ;;
>          --list-*)
>              NAME="${opt#--list-}"
>              is_in $NAME $COMPONENT_LIST || die_unknown $opt
> @@ -4487,6 +4495,36 @@ for opt do
>      esac
>  done
>  
> +find_extra_filters_extern() {
> +  # TODO: handle invalid filter
> +  while read f; do
> +    jq -r '.symbols[]' < "$f/filter.json" | sed 's/^ff_[avfsinkrc]\{2,5\}_\([[:alnum:]_]\{1,\}\)/\1_filter/'
> +  done < ffbuild/extra-filters.txt
> +}
> +
> +EXTRA_FILTER_LIST=$(find_extra_filters_extern)
> +
> +for n in extra_filters; do
> +    v=$(toupper ${n%s})_LIST
> +    eval enable \$$v
> +    eval ${n}_if_any="\$$v"
> +done
> +
> +FILTER_LIST="
> +    $FILTER_LIST
> +    $EXTRA_FILTER_LIST
> +"
> +
> +AVFILTER_COMPONENTS_LIST="
> +    $AVFILTER_COMPONENTS_LIST
> +    $EXTRA_FILTER_LIST
> +"
> +
> +ALL_COMPONENTS="
> +    $ALL_COMPONENTS
> +    $EXTRA_FILTER_LIST
> +"
> +
>  for e in $env; do
>      eval "export $e"
>  done
> @@ -7165,6 +7203,10 @@ enabled rkmpp             && { require_pkg_config rkmpp rockchip_mpp  rockchip/r
>                               }
>  enabled vapoursynth       && require_headers "vapoursynth/VSScript4.h vapoursynth/VapourSynth4.h"
>  
> +while read f; do
> +  # NOTE: this eval is dangerous, as it allows arbitrary code execution!
> +  eval $(jq -r '.check // true' < "$f/filter.json")
> +done < ffbuild/extra-filters.txt
>  
>  if enabled gcrypt; then
>      GCRYPT_CONFIG="${cross_prefix}libgcrypt-config"
> @@ -8243,6 +8285,12 @@ for entry in $LIBRARY_LIST $PROGRAM_LIST $EXTRALIBS_LIST; do
>      eval echo "EXTRALIBS-${entry}=\$${entry}_extralibs" >> ffbuild/config.mak
>  done
>  
> +echo "" > ffbuild/extra-filters.mak
> +
> +while read f; do
> +    echo "include $f/filter.mak" >> ffbuild/extra-filters.mak
> +done < ffbuild/extra-filters.txt
> +
>  cat > $TMPH <<EOF
>  /* Automatically generated by configure - do not modify! */
>  #ifndef FFMPEG_CONFIG_H
> @@ -8330,6 +8378,19 @@ cp_if_changed $TMPH libavutil/avconfig.h
>  # ...
>  eval "$(sed -n "s/^extern const FFFilter ff_\([avfsinkrc]\{2,5\}\)_\(.*\);/full_filter_name_\2=\1_\2/p" $source_path/libavfilter/allfilters.c)"
>  
> +rm -f libavfilter/extra_filters_extern.h
> +
> +# Handle extra filters
> +while read f; do
> +    eval "$(jq -r '.symbols[]' < "$f/filter.json" | sed 's/^ff_\([avfsinkrc]\{2,5\}\)_\([[:alnum:]]\{1,\}\)$/full_filter_name_\2=\1_\2/')"
> +    jq -r '.symbols[]' < "$f/filter.json" | while read symbol; do
> +      echo "extern const FFFilter $symbol;" >> libavfilter/extra_filters_extern.h
> +      echo "EXTRA_FILTER_$(echo $symbol | sed 's/^ff_[avfsinkrc]\{2,5\}_\([[:alnum:]]\{1,\}\)$/\1/' | tr a-z A-Z)_LOCATION = $f" >> ffbuild/extra-filters.mak
> +      echo "LDFLAGS += $(jq -r '.ldflags // ""' < "$f/filter.json")" >> ffbuild/extra-filters.mak
> +      echo "CFLAGS += $(jq -r '.cflags // ""' < "$f/filter.json")" >> ffbuild/extra-filters.mak
> +    done
> +done < ffbuild/extra-filters.txt
> +
>  # generate the lists of enabled components
>  print_enabled_components(){
>      file=$1
> diff --git a/libavfilter/Makefile b/libavfilter/Makefile
> index 7c0d879ec9..9b22aece3a 100644
> --- a/libavfilter/Makefile
> +++ b/libavfilter/Makefile
> @@ -27,6 +27,10 @@ OBJS = allfilters.o                                                     \
>  include $(SRC_PATH)/libavfilter/dnn/Makefile
>  include $(SRC_PATH)/libavfilter/vulkan/Makefile
>  
> +# extra filters handling
> +include $(SRC_PATH)/ffbuild/extra-filters.mak
> +EXTRA_FILTER_FLAGS = -I$(PWD) -I$(PWD)/libavfilter $(CPPFLAGS) $(CC_DEPFLAGS)
> +
>  OBJS-$(HAVE_LIBC_MSVCRT)                     += file_open.o
>  OBJS-$(HAVE_THREADS)                         += pthread.o
>  
> diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
> index 740d9ab265..e8565de5b0 100644
> --- a/libavfilter/allfilters.c
> +++ b/libavfilter/allfilters.c
> @@ -621,6 +621,7 @@ extern  const FFFilter ff_vsrc_buffer;
>  extern  const FFFilter ff_asink_abuffer;
>  extern  const FFFilter ff_vsink_buffer;
>  
> +#include "libavfilter/extra_filters_extern.h"
>  #include "libavfilter/filter_list.c"
>  
>  
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: POC: enable out-of-tree filters

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 1199 bytes --]

Hi

On Thu, Mar 13, 2025 at 01:18:49PM +0100, Leandro Santiago wrote:
> This is a POC/prototype that aims to enable out of tree filters on
> FFmpeg.
> 
> Here I name them "extra filters".
> 

> It introduces the program `jq` as a new build dependency.

I dont think this dependency is needed to achieve linking to filters
in another directory
i think its worth the few hours extra time to find a clean/nice way to do it
with no new depeandancies

also there is the quite intriguing option of using "git merge" to include
external filters
What i mean here is that there could be a script lets call it ffmerge

"ffmerge available" would check some online repository and list whats
compatible with the current checkout

"ffmerge merge shiny_filter" would then merge the shiny filter repository/branch

Advanatge would be that this is more powerfull than just linking external
filters.
Disadvantage is that for this to work care needs to be taken that conflicts
do not occur

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

It is dangerous to be right in matters on which the established authorities
are wrong. -- Voltaire

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: POC: enable out-of-tree filters

[Leandro Santiago]

On 3/14/25 16:04, Michael Niedermayer wrote:
> Hi
>
> On Thu, Mar 13, 2025 at 01:18:49PM +0100, Leandro Santiago wrote:
>> This is a POC/prototype that aims to enable out of tree filters on
>> FFmpeg.
>>
>> Here I name them "extra filters".
>>
Thinking now, "extra filters" is not a good name. Maybe "external" or "oot"
>> It introduces the program `jq` as a new build dependency.
> I dont think this dependency is needed to achieve linking to filters
> in another directory
To make it clear, the "extra filters" are linked in the same way the 
current filters are: direct as part libavfilter.(so|a). They are not 
dynamically linked.
> i think its worth the few hours extra time to find a clean/nice way to do it
> with no new depeandancies

I used json as a quick way to define the filter metadata (thus the usage 
of JQ to query them), but I confess it could have been any other format. 
I used json to get the proof of concept done :-)

Ideally it would be written in a native format to the build system, if 
something like CMake or Meson was being used.

Maybe the metadata could even be in a Makefile or spread over multiple 
files with no structure. The use of json was for pure convenience. Would 
you have any format to suggest? Is there anything similar in the 
codebase I could have a look at?

This is an example of metadata file at the moment:

{
"ldflags":"-ljson-c",
"cflags":"-DFOO=234",
"check":"require_pkg_config json json-c json-c/json.h json_c_version_num",
"symbols":["ff_vf_foo","ff_vf_bar"]
}

A simpler way to represent it would be put each field in a different 
pure text file, for instance:

symbols.txt:

ff_vf_foo ff_vf_bar

The most important aspect here is that the filter metadata (symbols, 
names, flags) need to be in a format easy to parse from the `configure` 
script.

I did some improvements on my patch and I managed to get the filter to 
pass extra flags on building.

Although the use of forgejo is not yet official, I am keeping my changes 
on it for now, as I still struggle with the email workflow:

https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/13

The "example" filter can be found at:

https://gitlab.com/leandrosansilva/ffmpeg-extra-filter-example/

To use it, simply clone this repo and build ffmpeg with 
`--extra-filter=/path/to/ffmpeg-extra-filter-example`.

>
> also there is the quite intriguing option of using "git merge" to include
> external filters
> What i mean here is that there could be a script lets call it ffmerge
How would it work when using release tarballs, without git?
> "ffmerge available" would check some online repository and list whats
> compatible with the current checkout
>
> "ffmerge merge shiny_filter" would then merge the shiny filter repository/branch
Wouldn't it put more burden over the devs to maintain such 
catalog/repository and the tooling around it? I am fine with that, I 
just wonder how the community would accept it. Having no filter 
catalog/repo feels to be the simplest step one could take at the moment.
>
> Advanatge would be that this is more powerfull than just linking external
> filters.
> Disadvantage is that for this to work care needs to be taken that conflicts
> do not occur

I see that simply keeping the filter outside and linking them at build 
time put almost no burden on the ffmpeg devs.

As, provided there is no promise of a stable API, it's up to the 
developers of the "extra" filters to make their filters build.

>
> thx
>
> [...]
>
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: POC: enable out-of-tree filters

[Lynne]

On 13/03/2025 13:18, Leandro Santiago wrote:
> This is a POC/prototype that aims to enable out of tree filters on
> FFmpeg.
> 
> Here I name them "extra filters".
> 
> It introduces the program `jq` as a new build dependency.
> 
> To test it, create a directory, for instance, /tmp/my-shiny-filter/ and
> inside it, create the following files:
> 
> `filter.json`, with the content:

This is a pointless patch and a pointlessly complex addition to the 
build system. I wouldn't accept something like this.
Those who for some reason have custom filters simply distribute their 
own branch with properly integrated filters. This also lets them 
upstream it if they want to.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: POC: enable out-of-tree filters

[Nicolas George]

Lynne (HE12025-03-14):
> This is a pointless patch and a pointlessly complex addition to the build
> system. I wouldn't accept something like this.
> Those who for some reason have custom filters simply distribute their own
> branch with properly integrated filters.

That works for third-party from one third-party. This feature helps
users who want to use filters from multiple third-party. With what you
describe, they would have to merge the branches, which might not be
trivial.

On the other hand, the JSON in this makes it look like it was designed
by looking at how fashionable languages NIHed their own f5g package
system.

If instead it was designed by asking “what would a ffmpeg developer do”,
it would just have configure run ext/*/configure and then Makefile
include $(CONFIG_EXT_WHATEVER).

Regards,

-- 
  Nicolas George
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: POC: enable out-of-tree filters

[Leandro Santiago]

On 3/14/25 19:21, Nicolas George wrote:
> Lynne (HE12025-03-14):
>> This is a pointless patch and a pointlessly complex addition to the build
>> system. I wouldn't accept something like this.
>> Those who for some reason have custom filters simply distribute their own
>> branch with properly integrated filters.
> That works for third-party from one third-party. This feature helps
> users who want to use filters from multiple third-party. With what you
> describe, they would have to merge the branches, which might not be
> trivial.
>
> On the other hand, the JSON in this makes it look like it was designed
> by looking at how fashionable languages NIHed their own f5g package
> system.

Thx for the review and to make it clear:

it was not my intention to suggest my implementation (using json and so 
on) as the final one or to be mainlined.

I notice now my lack of netiquette, as it seems I was not clear on 
saying that this is a proof of concept (POC, in the subject): I wanted 
to try something (support for non mainlined filters), and wanted to 
check whether that was possible or not, and finally get feedback.

Using json+jq was the quickest, dirtiest way to accomplished that.

I personally conclude that the POC was successful: I managed to get a 
new filter, not mainlined, to be built into ffmpeg. Next steps would be 
throw the code away and implement it properly

Obviously I need the agreement from you experienced ffmpeg devs for such 
conclusion.

I should probably have called it a RFC or similar. My mistake. I still 
struggle with using the mailing list contribution model, as in a PR/MR 
model I would simply keep a WIP request until it gets to a good shape. 
With the ML model, I feel doing so is very error prone, but this is 
probably my lack of experience with it.

My goal now is to know whether there is any interest from the ffmpeg 
devs on having such feature (regardless of the implementation), as there 
seems to be lots of disagreement.

I am personally interested on it, due to the fact I am writing a filter 
in Rust that is very unlikely to ever me mainlined. Right now I need to 
constantly rebase branches, but I'd be way happier if I could simply 
rely on some "official" support in the ffmpeg codebase to use it.

Another benefit could be moving out some filters that are "not very 
loved" or that tend to "get rotten". My impression/opinion is that the 
DNN stuff could very well be moved out, to evolve independently from the 
main ffmpeg repo. I totally respect the developers who worked on it, to 
be very clear. I benefit from the DNN code and want to keep it working well.

>
> If instead it was designed by asking “what would a ffmpeg developer do”,
> it would just have configure run ext/*/configure and then Makefile
> include $(CONFIG_EXT_WHATEVER).

Thank you for the feedback. I requested some thoughts on the topic a few 
days ago [1], and I see that probably the message was not clear.

I find this `simply run ext/configure approach` interesting and will 
have a look at it and later let you know my findings.

[1] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-March/340895.html

>
> Regards,
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: POC: enable out-of-tree filters

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 1408 bytes --]

On Fri, Mar 14, 2025 at 05:13:23PM +0100, Leandro Santiago wrote:
[...]
> > 
> > also there is the quite intriguing option of using "git merge" to include
> > external filters
> > What i mean here is that there could be a script lets call it ffmerge

> How would it work when using release tarballs, without git?

it would tell the user to run it in a git checkout or an empty directory.
The script would already have the ability to merge external filter repos
so also pulling mainline is no real extra complexity


> > "ffmerge available" would check some online repository and list whats
> > compatible with the current checkout
> > 
> > "ffmerge merge shiny_filter" would then merge the shiny filter repository/branch

> Wouldn't it put more burden over the devs to maintain such
> catalog/repository and the tooling around it? I am fine with that, I just
> wonder how the community would accept it. Having no filter catalog/repo
> feels to be the simplest step one could take at the moment.

theres no need for such catalog. I just think that if external filter
support is added, that someone will then likely start maintaining a catalog
of these filters.


thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Good people do not need laws to tell them to act responsibly, while bad
people will find a way around the laws. -- Plato

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: POC: enable out-of-tree filters

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Lynne
> Sent: Freitag, 14. März 2025 17:58
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH] avfilter: POC: enable out-of-tree
> filters
> 
> On 13/03/2025 13:18, Leandro Santiago wrote:
> > This is a POC/prototype that aims to enable out of tree filters on
> > FFmpeg.
> >
> > Here I name them "extra filters".
> >
> > It introduces the program `jq` as a new build dependency.
> >
> > To test it, create a directory, for instance, /tmp/my-shiny-filter/
> and
> > inside it, create the following files:
> >
> > `filter.json`, with the content:
> 
> This is a pointless patch and a pointlessly complex addition to the
> build system. I wouldn't accept something like this.
> Those who for some reason have custom filters simply distribute their
> own branch with properly integrated filters. This also lets them
> upstream it if they want to.
> _______________________________________________

Hi Lynne,

as someone who is using a number of custom filters, I can say that the "own branch with properly integrated filters" approach is a continuing source of trouble when rebasing or trying to apply it to different base branches. Even though these aren't really 3rd party filters, having a mechanism for it would make it much easier to maintain additional filters.
I don't want to argue about the specific approach as other will know better than me, but I'm sure that it's possible to find a nice non-complex way, no?

sw




_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable out-of-tree filters

[Leandro Santiago]

[-- Attachment #1: Type: text/plain, Size: 6921 bytes --]

I've applied a few changes from your feedback, the following is how the feature looks like.

***Disclaimer***

this is still a proof of concept and for now I am looking for your feedback and help. There are lots of rough corners and stuff that does not work. I by no means intend to request the changes as they are to be merged. Call it an RFC if you will.

Also, I am not sending the patch as an attachment, and leave the e-mail body for discussion only. I am not sure whether this is a good approach or goes against the mailing list policy. This is the time I wish we could really contribute using the Pull Request more on a nice web interface.

If you folks want to experiment, I pushed my changes in this PR: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/13

***Finish disclaimer***

I changed the name convention from "extra filters" to "external filters", as it sounds more natural. (Another possibility would be "extension filters", so please let me know what you think).

The build system will automatically pick up any filters in the subdirectory `ext/libavfilter`, and ignore it if it does not exist.

The rationale behind this directory structure is to reserve the possibility of extensions of other libraries as well.

An external filter is composed by at least one Makefile, and this Makefile has some "metadata" encoded in its comments, used by the `./configure` script to generate some code. I'll show how they look like in an example.

Although this way to encode metadata looks very ugly and I am pretty sure can be improved, this is the best I could for now, and I hope you folks can point me directions on how to improve it, as you have more knowledge of the build system than I do.

Here are the metadata types:

- check: this is a command used to check if 3rd party dependencies are available and enabling them within the build system. The command here is executed by `./configure`. This entry is optional and for now there can be only one of them, due to technical limitations of my implementation.

- symbol: this entry can appear multiple times, and indicates the name of the FFFilter objects exported by the filter. There must be one entry per exported symbol. These entries are used by `./configure` to generate the list of enabled filters.

- ldflags: those are extra LDFLAGS to be used to link libavfilter. I am not happy this this one at all, as such information ideally should be encoded as Makefile variables, but I confess I could not manage to do it. I'd be happy to get directions on where in the build system to look for the right "hook", but I'll continue looking for it as well.

As an example, here is how one can create an empty filter. Just to exercise the build system, the example filter depends on an external library, json-c, which is found using pkg-config and will be dynamically linked to libavfilter. In case json-c is not found, `./configure` will fail, as expected.

```

mkdir -p ext/libavfilter/example

```

Then create the a file ext/libavfilter/example/Makefile with the content:

```

# check: require_pkg_config json json-c json-c/json.h json_c_version_num
# symbol: ff_vf_example
# ldflags: -ljson-c

OBJS += vf_example.o
CFLAGS += -DFOO=1234

libavfilter/vf_example.o:
        $(CC) $(EXTERNAL_FILTER_FLAGS) $(CPPFLAGS) $(CFLAGS) -c -o $@ $(EXTERNAL_FILTER_EXAMPLE_LOCATION)/vf_example.c

```

Then, create the file ext/libavfilter/example/Makefile/vf_example.c with the content:


```

#include "libavutil/internal.h"
#include "avfilter.h"
#include "filters.h"
#include "video.h"

#include <json.h>

static int process_command(AVFilterContext *ctx, const char *cmd, const char *args, char *res, int res_len, int flags) {
  // NOTE: I know this makes no sense, I just wanted to use some symbol from json-c
  // in a way the compiler cannot optimize it out
  return json_c_version_num();
}


const FFFilter ff_vf_example = {
    .p.name        = "example",
    .p.description = NULL_IF_CONFIG_SMALL("An external example filter"),
    .p.flags       = AVFILTER_FLAG_METADATA_ONLY,
    FILTER_INPUTS(ff_video_default_filterpad),
    FILTER_OUTPUTS(ff_video_default_filterpad),
    .process_command = process_command,
};

```

(You can skip this manually creating those files by checking out this example filter from gitlab: https://gitlab.com/leandrosansilva/ffmpeg-extra-filter-example)

Now you can compile ffmpeg as usual, with `./configure && make && make install` and use the filter.

In my tests, building ffmpeg libraries either in static and shared mode work, with the extra dependencies properly linked.


Explanation:

- The .o files for the external filters will be generated in the same place as the built-in filters, in libavfilter/. That means that two external filters cannot generate the same .o files.

- A new variable with the format EXTERNAL_FILTER_$(FILTER_SYMBOL_IN_UPPERCASE)_LOCATION will be generated for each FFFilter exported. More explanation about that below.

- The build system offers now a variable `EXTERNAL_FILTER_FLAGS` which adds the paths of the libavfilter private headers to the compiler when building the external filters. The idea here is for code for an external filter to look exactly as if it was a built-in filter: the include paths should be the same. This will make it easier for external filters to be moved as built-in and the other way around, not requiring much code changes.


TODO:

- Find a way to remove the `ldflags` metadata entry to turn it into a Makefile variable.

- Allow multiple `check` metadata entries, to make it easier to check for multiple 3rd party dependencies.

- Better error handling on malformed/invalid filter metadata.

- Find a way to encode conditional builds in the metadata. I am here thinking on filters like the `dnn` family, which can have multiple backends, which are now chosen using `--enable-openvino`, `--enable-libtorch`, `--enable-libtensorflow` or the drawtext filter that has optional dependencies. Maybe the external filters could hook new `--enable-*` options to `./configure`, but this smells like trouble when multiple filters specify the same optional dependencies.

- You can see in the example Makefile that a new "magical" variable named `EXTERNAL_FILTER_EXAMPLE_LOCATION` was created by the build system. It looks ugly to me, but I could not find a way to obtain the path of the current Makefile via make, and I needed a way to specify the path of the filter source files. If you know a cleaner way to do it, please let me know.

- Make it possible to link filters written in other languages. My "final" goal for is Rust, but it should not prevent the use of C++ or Zig, for instance. I believe this can be done in a further iteration. For now my focus is only supporting C.

- Once the implementation and "spec" is agreed, document it.

- Refactor the changes in the build system to improve legibility.


Regards,

Leandro

[-- Attachment #2: 0001-avfilter-Proof-of-Concept-enable-out-of-tree-filters.patch --]
[-- Type: text/x-patch, Size: 7179 bytes --]

From 7486f878b982867667892611e181dc4eab3298c8 Mon Sep 17 00:00:00 2001
From: Leandro Santiago <leandrosansilva@gmail.com>
Date: Wed, 12 Mar 2025 13:42:42 +0100
Subject: [PATCH] avfilter: Proof of Concept: enable out-of-tree filters

This is a POC/prototype that aims to enable out of tree filters on
FFmpeg.

Here is how to test it, with an example filter:

```
mkdir -p ext/libavfilter
pushd ext/libavfilter
git clone https://gitlab.com/leandrosansilva/ffmpeg-extra-filter-example example
popd
```

Then compile ffmpeg as usual:

```
./configure && make && make install
```

Now you can use the filters:

```
ffplay /path/to/file.webm -vf 'foo,bar'
```

What works:

- Building C based filters with no extra, or simple dependencies.
- Multiple filters in the same object file.

What is ugly:

- The filter metadata is hidden in the Makefile comments, and this is
  needed because at `./configure` time, executed before make, some code
  needs to be generated and, although such generation could in theory be
  done via make, this feels like too much change at the moment.

- Among the metadata, there are linker and compiler flags, which would
  look much better if they were simple make variables.

- At the moment it's possible for only one `check` metadata entry to be
  included in the Makefile, but we should support multiple of them, for
  the case when there are multiple extra dependencies.

What was not implemented:

- I believe it would be useful to check if the license of the filter is
  compatible with the license used to build FFmpeg.

- Only extra filters written in C (maybe C++?) are supported for now.
  One of my goals is to enable Rust as well.

- There should be a way to have optional dependencies, for filters that
  can have multiple "backends", for instance, to be chosen at build time.

Signed-off-by: Leandro Santiago <leandrosansilva@gmail.com>
---
 .gitignore               |  4 +++
 configure                | 75 ++++++++++++++++++++++++++++++++++++++++
 libavfilter/Makefile     |  4 +++
 libavfilter/allfilters.c |  1 +
 4 files changed, 84 insertions(+)

diff --git a/.gitignore b/.gitignore
index 430abaf91b..4eae911379 100644
--- a/.gitignore
+++ b/.gitignore
@@ -44,3 +44,7 @@
 /tools/python/__pycache__/
 /libavcodec/vulkan/*.c
 /libavfilter/vulkan/*.c
+/ffbuild/external-filters.txt
+/ffbuild/external-filters.mak
+/libavfilter/external_filters_extern.h
+/ext
diff --git a/configure b/configure
index d84e32196d..f7bb2cc38a 100755
--- a/configure
+++ b/configure
@@ -1798,6 +1798,7 @@ AVDEVICE_COMPONENTS="
 
 AVFILTER_COMPONENTS="
     filters
+    external_filters
 "
 
 AVFORMAT_COMPONENTS="
@@ -4489,6 +4490,50 @@ for opt do
     esac
 done
 
+symbols_from_external_filter_makefile() {
+  grep '^#.*symbol:' < "$1" | sed -e 's|#\s*symbol: \(.*\)|\1|g'
+}
+
+list_external_filter_makefiles() {
+  [ ! -d "ext/libavfilter" ] && return
+
+  for f in ext/libavfilter/*; do
+    [ -f "$f/Makefile" ] && echo $f/Makefile
+  done
+}
+
+list_external_filter_makefiles > ffbuild/external-filters.txt
+
+find_external_filters_extern() {
+  # TODO: handle invalid filter
+  while read Makefile; do
+    symbols_from_external_filter_makefile "$Makefile" | sed 's/^ff_[avfsinkrc]\{2,5\}_\([[:alnum:]_]\{1,\}\)/\1_filter/'
+  done < ffbuild/external-filters.txt
+}
+
+EXTERNAL_FILTER_LIST=$(find_external_filters_extern)
+
+for n in external_filters; do
+    v=$(toupper ${n%s})_LIST
+    eval enable \$$v
+    eval ${n}_if_any="\$$v"
+done
+
+FILTER_LIST="
+    $FILTER_LIST
+    $EXTERNAL_FILTER_LIST
+"
+
+AVFILTER_COMPONENTS_LIST="
+    $AVFILTER_COMPONENTS_LIST
+    $EXTERNAL_FILTER_LIST
+"
+
+ALL_COMPONENTS="
+    $ALL_COMPONENTS
+    $EXTERNAL_FILTER_LIST
+"
+
 for e in $env; do
     eval "export $e"
 done
@@ -7173,6 +7218,11 @@ enabled rkmpp             && { require_pkg_config rkmpp rockchip_mpp  rockchip/r
                              }
 enabled vapoursynth       && require_headers "vapoursynth/VSScript4.h vapoursynth/VapourSynth4.h"
 
+# Check for the dependencies of the external filters
+while read Makefile; do
+  # NOTE: this eval is dangerous, as it allows arbitrary code execution!
+  eval $(grep '^#.*check:' "$Makefile" | sed 's|#\s*check: \(.*\)|\1|g')
+done < ffbuild/external-filters.txt
 
 if enabled gcrypt; then
     GCRYPT_CONFIG="${cross_prefix}libgcrypt-config"
@@ -8250,12 +8300,23 @@ IGNORE_TESTS=$ignore_tests
 VERSION_TRACKING=$version_tracking
 EOF
 
+while read Makefile; do
+  # NOTE: this eval is dangerous, as it allows arbitrary code execution!
+  eval "avfilter_extralibs=\"\$avfilter_extralibs $(grep '^#.*ldflags:' "$Makefile" | sed 's|#\s*ldflags: \(.*\)|\1|g')\""
+done < ffbuild/external-filters.txt
+
 map 'eval echo "${v}_FFLIBS=\$${v}_deps" >> ffbuild/config.mak' $LIBRARY_LIST
 
 for entry in $LIBRARY_LIST $PROGRAM_LIST $EXTRALIBS_LIST; do
     eval echo "EXTRALIBS-${entry}=\$${entry}_extralibs" >> ffbuild/config.mak
 done
 
+echo "" > ffbuild/external-filters.mak
+
+while read Makefile; do
+    echo "include $Makefile" >> ffbuild/external-filters.mak
+done < ffbuild/external-filters.txt
+
 cat > $TMPH <<EOF
 /* Automatically generated by configure - do not modify! */
 #ifndef FFMPEG_CONFIG_H
@@ -8343,6 +8404,20 @@ cp_if_changed $TMPH libavutil/avconfig.h
 # ...
 eval "$(sed -n "s/^extern const FFFilter ff_\([avfsinkrc]\{2,5\}\)_\(.*\);/full_filter_name_\2=\1_\2/p" $source_path/libavfilter/allfilters.c)"
 
+rm -f libavfilter/external_filters_extern.h
+
+# register the symbols of the external filters
+while read Makefile; do
+    eval "$(symbols_from_external_filter_makefile "$Makefile" \
+      | sed 's/^ff_\([avfsinkrc]\{2,5\}\)_\([[:alnum:]]\{1,\}\)$/full_filter_name_\2=\1_\2/')"
+
+    symbols_from_external_filter_makefile "$Makefile" | while read symbol; do
+        echo "extern const FFFilter $symbol;" >> libavfilter/external_filters_extern.h
+        echo "EXTERNAL_FILTER_$(echo $symbol | sed 's/^ff_[avfsinkrc]\{2,5\}_\([[:alnum:]]\{1,\}\)$/\1/' | tr a-z A-Z)_LOCATION = $f" \
+            >> ffbuild/external-filters.mak
+    done
+done < ffbuild/external-filters.txt
+
 # generate the lists of enabled components
 print_enabled_components(){
     file=$1
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 7c0d879ec9..877b24c30f 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -27,6 +27,10 @@ OBJS = allfilters.o                                                     \
 include $(SRC_PATH)/libavfilter/dnn/Makefile
 include $(SRC_PATH)/libavfilter/vulkan/Makefile
 
+# external filters handling
+include $(SRC_PATH)/ffbuild/external-filters.mak
+EXTERNAL_FILTER_FLAGS = -I$(PWD) -I$(PWD)/libavfilter
+
 OBJS-$(HAVE_LIBC_MSVCRT)                     += file_open.o
 OBJS-$(HAVE_THREADS)                         += pthread.o
 
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 740d9ab265..c2d576e4be 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -621,6 +621,7 @@ extern  const FFFilter ff_vsrc_buffer;
 extern  const FFFilter ff_asink_abuffer;
 extern  const FFFilter ff_vsink_buffer;
 
+#include "libavfilter/external_filters_extern.h"
 #include "libavfilter/filter_list.c"
 
 
-- 
2.48.1


[-- Attachment #3: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable out-of-tree filters

[Leandro Santiago]

This is a POC/prototype that aims to enable out of tree filters on
FFmpeg.

Here is how to test it, with an example filter:

```
mkdir -p ext/libavfilter
pushd ext/libavfilter
git clone https://gitlab.com/leandrosansilva/ffmpeg-extra-filter-example example
popd
```

Then compile ffmpeg as usual:

```
./configure && make && make install
```

Now you can use the new filters:

```
ffplay /path/to/file.webm -vf 'foo,bar'
```

What works:

- Building C based filters with no extra, or simple dependencies.
- Multiple filters in the same object file.

What was not implemented:

- I believe it would be useful to check if the license of the filter is
  compatible with the license used to build FFmpeg.

- Only extra filters written in C (maybe C++?) are supported for now.
  One of my goals is to enable Rust as well.

- Documentation

- There should be a way to have optional dependencies, for filters that
  can have multiple "backends", for instance, to be chosen at build time.

Signed-off-by: Leandro Santiago <leandrosansilva@gmail.com>
---
 .gitignore               |  3 ++
 configure                | 60 ++++++++++++++++++++++++++++++++++++++--
 libavfilter/Makefile     |  4 +++
 libavfilter/allfilters.c |  1 +
 4 files changed, 66 insertions(+), 2 deletions(-)

diff --git a/.gitignore b/.gitignore
index 430abaf91b..7d66dee516 100644
--- a/.gitignore
+++ b/.gitignore
@@ -44,3 +44,6 @@
 /tools/python/__pycache__/
 /libavcodec/vulkan/*.c
 /libavfilter/vulkan/*.c
+/ffbuild/external-filters.mak
+/libavfilter/external_filters_extern.h
+/ext
diff --git a/configure b/configure
index 2fdbe8cbbe..317862fa1a 100755
--- a/configure
+++ b/configure
@@ -1798,6 +1798,7 @@ AVDEVICE_COMPONENTS="
 
 AVFILTER_COMPONENTS="
     filters
+    external_filters
 "
 
 AVFORMAT_COMPONENTS="
@@ -4086,7 +4087,7 @@ swscale_deps="avutil"
 swscale_suggest="libm stdatomic"
 
 avcodec_extralibs="pthreads_extralibs iconv_extralibs dxva2_extralibs liblcevc_dec_extralibs lcms2_extralibs"
-avfilter_extralibs="pthreads_extralibs"
+avfilter_extralibs="pthreads_extralibs external_filters_extralibs"
 avutil_extralibs="d3d11va_extralibs d3d12va_extralibs mediacodec_extralibs nanosleep_extralibs pthreads_extralibs vaapi_drm_extralibs vaapi_x11_extralibs vaapi_win32_extralibs vdpau_x11_extralibs"
 
 # programs
@@ -4324,10 +4325,15 @@ ALL_COMPONENTS="
     $AVFORMAT_COMPONENTS_LIST
 "
 
-for n in $COMPONENT_LIST; do
+enable_component() {
+    n="$1"
     v=$(toupper ${n%s})_LIST
     eval enable \$$v
     eval ${n}_if_any="\$$v"
+}
+
+for n in $COMPONENT_LIST; do
+    enable_component "$n"
 done
 
 enable $ARCH_EXT_LIST
@@ -7463,6 +7469,56 @@ elif enabled iconv; then
     check_func_headers iconv.h iconv || check_lib iconv iconv.h iconv -liconv
 fi
 
+# Handle external filters from the directory ext/libavfilter
+
+echo "" > libavfilter/external_filters_extern.h
+echo "" > ffbuild/external-filters.mak
+
+EXTERNAL_FILTER_LIST=""
+
+# This function is used by the external filters' configure.sh script to register the filters
+register_external_filter_symbol() {
+    EXT_FILTER_SYMBOL="$1"
+    EXTERNAL_FILTER_LIST="$EXTERNAL_FILTER_LIST $(echo $EXT_FILTER_SYMBOL | sed 's/^ff_[avfsinkrc]\{2,5\}_\([[:alnum:]_]\{1,\}\)/\1_filter/')"
+
+    eval "$(echo "$EXT_FILTER_SYMBOL" | sed 's/^ff_\([avfsinkrc]\{2,5\}\)_\([[:alnum:]]\{1,\}\)$/full_filter_name_\2=\1_\2/')"
+
+    echo "extern const FFFilter $EXT_FILTER_SYMBOL;" >> libavfilter/external_filters_extern.h
+
+    echo "EXTERNAL_FILTER_$(\
+        echo $EXT_FILTER_SYMBOL \
+        | sed 's/^ff_[avfsinkrc]\{2,5\}_\([[:alnum:]]\{1,\}\)$/\1/' \
+        | tr a-z A-Z \
+        )_LOCATION = $CURRENT_EXT_FILTER_LOCATION" >> ffbuild/external-filters.mak
+}
+
+if [ -d "ext/libavfilter" ]; then
+    for f in ext/libavfilter/*; do
+        [ -f "$f/configure.sh" ] || die "Mandatory file $f/configure.sh not found"
+        [ -f "$f/Makefile" ] || die "Mandatory file $f/Makefile not found"
+        CURRENT_EXT_FILTER_LOCATION="$f"
+        . "$f/configure.sh" || die "Could not configure external filter at $f"
+        echo "include $f/Makefile" >> ffbuild/external-filters.mak
+    done
+
+    enable_component external_filters
+fi
+
+FILTER_LIST="
+    $FILTER_LIST
+    $EXTERNAL_FILTER_LIST
+"
+
+AVFILTER_COMPONENTS_LIST="
+    $AVFILTER_COMPONENTS_LIST
+    $EXTERNAL_FILTER_LIST
+"
+
+ALL_COMPONENTS="
+    $ALL_COMPONENTS
+    $EXTERNAL_FILTER_LIST
+"
+
 enabled debug && add_cflags -g"$debuglevel" && add_asflags -g"$debuglevel"
 
 # add some useful compiler flags if supported
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 7c0d879ec9..877b24c30f 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -27,6 +27,10 @@ OBJS = allfilters.o                                                     \
 include $(SRC_PATH)/libavfilter/dnn/Makefile
 include $(SRC_PATH)/libavfilter/vulkan/Makefile
 
+# external filters handling
+include $(SRC_PATH)/ffbuild/external-filters.mak
+EXTERNAL_FILTER_FLAGS = -I$(PWD) -I$(PWD)/libavfilter
+
 OBJS-$(HAVE_LIBC_MSVCRT)                     += file_open.o
 OBJS-$(HAVE_THREADS)                         += pthread.o
 
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 740d9ab265..c2d576e4be 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -621,6 +621,7 @@ extern  const FFFilter ff_vsrc_buffer;
 extern  const FFFilter ff_asink_abuffer;
 extern  const FFFilter ff_vsink_buffer;
 
+#include "libavfilter/external_filters_extern.h"
 #include "libavfilter/filter_list.c"
 
 
-- 
2.48.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable out-of-tree filters

[Leandro Santiago]

In this iteration I've made the following changes, based on the received feedback:

- No external tools are needed by the build system.

- The external filters should be put in ext/libavfilter. Anything there will be included.

- The build system will execute a file called `configure.sh` in the directory of the external filter. This script has access to the functions and variables defined on `./configure`, as it's included via `.` on sh.

- I will document the "API" for the external filters as soon the approach is approved, but an example of filter can be found at https://gitlab.com/leandrosansilva/ffmpeg-extra-filter-example . If you are unhappy with the example code hosted on gitlab.com, I can move it elsewhere where it gets easier for you to have access to the code.

- Essentially, an external filter is composed by at least a `configure.sh` and a `Makefile`.

I really hope this can be the last iteration, as I ran out of ideas on how to simplify the process, so please let me know your thoughts :-)

Leandro

On 3/24/25 16:56, Leandro Santiago wrote:
> This is a POC/prototype that aims to enable out of tree filters on
> FFmpeg.
>
> Here is how to test it, with an example filter:
>
> ```
> mkdir -p ext/libavfilter
> pushd ext/libavfilter
> git clone https://gitlab.com/leandrosansilva/ffmpeg-extra-filter-example example
> popd
> ```
>
> Then compile ffmpeg as usual:
>
> ```
> ./configure && make && make install
> ```
>
> Now you can use the new filters:
>
> ```
> ffplay /path/to/file.webm -vf 'foo,bar'
> ```
>
> What works:
>
> - Building C based filters with no extra, or simple dependencies.
> - Multiple filters in the same object file.
>
> What was not implemented:
>
> - I believe it would be useful to check if the license of the filter is
>   compatible with the license used to build FFmpeg.
>
> - Only extra filters written in C (maybe C++?) are supported for now.
>   One of my goals is to enable Rust as well.
>
> - Documentation
>
> - There should be a way to have optional dependencies, for filters that
>   can have multiple "backends", for instance, to be chosen at build time.
>
> Signed-off-by: Leandro Santiago <leandrosansilva@gmail.com>
> ---
>  .gitignore               |  3 ++
>  configure                | 60 ++++++++++++++++++++++++++++++++++++++--
>  libavfilter/Makefile     |  4 +++
>  libavfilter/allfilters.c |  1 +
>  4 files changed, 66 insertions(+), 2 deletions(-)
>
> diff --git a/.gitignore b/.gitignore
> index 430abaf91b..7d66dee516 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -44,3 +44,6 @@
>  /tools/python/__pycache__/
>  /libavcodec/vulkan/*.c
>  /libavfilter/vulkan/*.c
> +/ffbuild/external-filters.mak
> +/libavfilter/external_filters_extern.h
> +/ext
> diff --git a/configure b/configure
> index 2fdbe8cbbe..317862fa1a 100755
> --- a/configure
> +++ b/configure
> @@ -1798,6 +1798,7 @@ AVDEVICE_COMPONENTS="
>  
>  AVFILTER_COMPONENTS="
>      filters
> +    external_filters
>  "
>  
>  AVFORMAT_COMPONENTS="
> @@ -4086,7 +4087,7 @@ swscale_deps="avutil"
>  swscale_suggest="libm stdatomic"
>  
>  avcodec_extralibs="pthreads_extralibs iconv_extralibs dxva2_extralibs liblcevc_dec_extralibs lcms2_extralibs"
> -avfilter_extralibs="pthreads_extralibs"
> +avfilter_extralibs="pthreads_extralibs external_filters_extralibs"
>  avutil_extralibs="d3d11va_extralibs d3d12va_extralibs mediacodec_extralibs nanosleep_extralibs pthreads_extralibs vaapi_drm_extralibs vaapi_x11_extralibs vaapi_win32_extralibs vdpau_x11_extralibs"
>  
>  # programs
> @@ -4324,10 +4325,15 @@ ALL_COMPONENTS="
>      $AVFORMAT_COMPONENTS_LIST
>  "
>  
> -for n in $COMPONENT_LIST; do
> +enable_component() {
> +    n="$1"
>      v=$(toupper ${n%s})_LIST
>      eval enable \$$v
>      eval ${n}_if_any="\$$v"
> +}
> +
> +for n in $COMPONENT_LIST; do
> +    enable_component "$n"
>  done
>  
>  enable $ARCH_EXT_LIST
> @@ -7463,6 +7469,56 @@ elif enabled iconv; then
>      check_func_headers iconv.h iconv || check_lib iconv iconv.h iconv -liconv
>  fi
>  
> +# Handle external filters from the directory ext/libavfilter
> +
> +echo "" > libavfilter/external_filters_extern.h
> +echo "" > ffbuild/external-filters.mak
> +
> +EXTERNAL_FILTER_LIST=""
> +
> +# This function is used by the external filters' configure.sh script to register the filters
> +register_external_filter_symbol() {
> +    EXT_FILTER_SYMBOL="$1"
> +    EXTERNAL_FILTER_LIST="$EXTERNAL_FILTER_LIST $(echo $EXT_FILTER_SYMBOL | sed 's/^ff_[avfsinkrc]\{2,5\}_\([[:alnum:]_]\{1,\}\)/\1_filter/')"
> +
> +    eval "$(echo "$EXT_FILTER_SYMBOL" | sed 's/^ff_\([avfsinkrc]\{2,5\}\)_\([[:alnum:]]\{1,\}\)$/full_filter_name_\2=\1_\2/')"
> +
> +    echo "extern const FFFilter $EXT_FILTER_SYMBOL;" >> libavfilter/external_filters_extern.h
> +
> +    echo "EXTERNAL_FILTER_$(\
> +        echo $EXT_FILTER_SYMBOL \
> +        | sed 's/^ff_[avfsinkrc]\{2,5\}_\([[:alnum:]]\{1,\}\)$/\1/' \
> +        | tr a-z A-Z \
> +        )_LOCATION = $CURRENT_EXT_FILTER_LOCATION" >> ffbuild/external-filters.mak
> +}
> +
> +if [ -d "ext/libavfilter" ]; then
> +    for f in ext/libavfilter/*; do
> +        [ -f "$f/configure.sh" ] || die "Mandatory file $f/configure.sh not found"
> +        [ -f "$f/Makefile" ] || die "Mandatory file $f/Makefile not found"
> +        CURRENT_EXT_FILTER_LOCATION="$f"
> +        . "$f/configure.sh" || die "Could not configure external filter at $f"
> +        echo "include $f/Makefile" >> ffbuild/external-filters.mak
> +    done
> +
> +    enable_component external_filters
> +fi
> +
> +FILTER_LIST="
> +    $FILTER_LIST
> +    $EXTERNAL_FILTER_LIST
> +"
> +
> +AVFILTER_COMPONENTS_LIST="
> +    $AVFILTER_COMPONENTS_LIST
> +    $EXTERNAL_FILTER_LIST
> +"
> +
> +ALL_COMPONENTS="
> +    $ALL_COMPONENTS
> +    $EXTERNAL_FILTER_LIST
> +"
> +
>  enabled debug && add_cflags -g"$debuglevel" && add_asflags -g"$debuglevel"
>  
>  # add some useful compiler flags if supported
> diff --git a/libavfilter/Makefile b/libavfilter/Makefile
> index 7c0d879ec9..877b24c30f 100644
> --- a/libavfilter/Makefile
> +++ b/libavfilter/Makefile
> @@ -27,6 +27,10 @@ OBJS = allfilters.o                                                     \
>  include $(SRC_PATH)/libavfilter/dnn/Makefile
>  include $(SRC_PATH)/libavfilter/vulkan/Makefile
>  
> +# external filters handling
> +include $(SRC_PATH)/ffbuild/external-filters.mak
> +EXTERNAL_FILTER_FLAGS = -I$(PWD) -I$(PWD)/libavfilter
> +
>  OBJS-$(HAVE_LIBC_MSVCRT)                     += file_open.o
>  OBJS-$(HAVE_THREADS)                         += pthread.o
>  
> diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
> index 740d9ab265..c2d576e4be 100644
> --- a/libavfilter/allfilters.c
> +++ b/libavfilter/allfilters.c
> @@ -621,6 +621,7 @@ extern  const FFFilter ff_vsrc_buffer;
>  extern  const FFFilter ff_asink_abuffer;
>  extern  const FFFilter ff_vsink_buffer;
>  
> +#include "libavfilter/external_filters_extern.h"
>  #include "libavfilter/filter_list.c"
>  
>  
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable out-of-tree filters

[Ronald S. Bultje]

Hi,

On Mon, Mar 24, 2025 at 12:20 PM Leandro Santiago <leandrosansilva@gmail.com>
wrote:

> I really hope this can be the last iteration, as I ran out of ideas on how
> to simplify the process, so please let me know your thoughts :-)


I'm not sure I understand the rationale or goal of this. It seems you're
trying to create a process for extending the source/build tree with
components not part of our git. Is this something people are interested in?
I've never heard this use case before.

Ronald
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] =?gb18030?b?u9i4tKO6ICBbUEFUQ0hdIGF2ZmlsdGVyOiBQ?= =?gb18030?q?roof_of_Concept=3A_enable_out-of-tree_filters?=

[yangyalei via ffmpeg-devel]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="gb18030", Size: 2912 bytes --]

Sorry for garbled, reissued as follows.


In the avfiltergraph of FFmpeg, it is required that all filters within the graph can obtain a format and successfully negotiate for data flow.


So, is it allowed for the graph to contain filters that cannot obtain a format? Can these filters be ignored during format negotiation, allowing only the filters that can obtain a format to negotiate successfully?


This patch addresses the problem described in this scenario.


For example:


(abuff_src@Music)(abuff_src@Ring)---amix---abuff_sink


In the graph above, We support three playback scenarios at the same time:


1. Only abuff_src@Music needs to be played, and abuff_src@Ring has no data. abuff_src@Ring is ignored during negotiation, "abuff_src@Music -- amix -- abuff_sink" link negotiation is successful, and playback can be performed;


2. Only abuff_src@Ring needs to be played, and abuff_src@Music has no data. abuff_src@Music is ignored during negotiation, "abuff_src@ring --> amix --> abuff_sink" link negotiation is successful, and playback can be performed;


3. Both abuff_src@Music and abuff_src@Ring need to be played, and all filters are negotiated successfully and mixed playback is performed.





ÑîÑÇÀÚ
269032741@qq.com



 




------------------ Ô­Ê¼Óʼþ ------------------
·¢¼þÈË:                                                                                                                        "FFmpeg development discussions and patches"                                                                                    <rsbultje@gmail.com&gt;;
·¢ËÍʱ¼ä:&nbsp;2025Äê3ÔÂ26ÈÕ(ÐÇÆÚÈý) Á賿2:05
ÊÕ¼þÈË:&nbsp;"FFmpeg development discussions and patches"<ffmpeg-devel@ffmpeg.org&gt;;

Ö÷Ìâ:&nbsp;Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable out-of-tree filters



Hi,

On Mon, Mar 24, 2025 at 12:206§2PM Leandro Santiago <leandrosansilva@gmail.com&gt;
wrote:

&gt; I really hope this can be the last iteration, as I ran out of ideas on how
&gt; to simplify the process, so please let me know your thoughts :-)


I'm not sure I understand the rationale or goal of this. It seems you're
trying to create a process for extending the source/build tree with
components not part of our git. Is this something people are interested in?
I've never heard this use case before.

Ronald
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable out-of-tree filters

[Zhao Zhili]

> On Mar 26, 2025, at 02:05, Ronald S. Bultje <rsbultje@gmail.com> wrote:
> 
> Hi,
> 
> On Mon, Mar 24, 2025 at 12:20 PM Leandro Santiago <leandrosansilva@gmail.com>
> wrote:
> 
>> I really hope this can be the last iteration, as I ran out of ideas on how
>> to simplify the process, so please let me know your thoughts :-)
> 
> 
> I'm not sure I understand the rationale or goal of this. It seems you're
> trying to create a process for extending the source/build tree with
> components not part of our git. Is this something people are interested in?
> I've never heard this use case before.

There is a similar one. Nginx has out of tree module, e.g., nginx-rtmp-module
which add RTMP support to nginx.

https://github.com/arut/nginx-rtmp-module

> 
> Ronald
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable out-of-tree filters

[softworkz .]

Hi Ronald,

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Ronald
> S. Bultje
> Sent: Dienstag, 25. März 2025 19:05
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable
> out-of-tree filters
> 
> Hi,
> 
> On Mon, Mar 24, 2025 at 12:20 PM Leandro Santiago
> <leandrosansilva@gmail.com>
> wrote:
> 
> > I really hope this can be the last iteration, as I ran out of ideas on
> how
> > to simplify the process, so please let me know your thoughts :-)
> 
> 
> I'm not sure I understand the rationale or goal of this. It seems you're
> trying to create a process for extending the source/build tree with
> components not part of our git. Is this something people are interested
> in?

Yes.

> I've never heard this use case before.


It had been discussed here (tangentially):

https://ffmpeg.org/pipermail/ffmpeg-devel/2025-January/338807.html

and here:

https://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2025-February/340030.html


It's kind of an alternative to a run-time plugin-model to which some have expressed reservations.
Interests are multifold. It makes it easier to maintain custom filters across branches and versions of the main code, it allows people to create things which don't have a chance to get into the codebase for various reasons: like being too specific for general use, being tied to a specific vendor, using remote APIs that are too volatile, being too experimental or whatever else can be. 
Needless to mention the many AI related use cases. Without providing any kind of extensibility, other solutions will fill that gap and Ffmpeg relevancy will drop in the coming years IMO.

Best,
sw
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable out-of-tree filters

[Leandro Santiago]

On 3/25/25 19:05, Ronald S. Bultje wrote:
> Hi,
>
> On Mon, Mar 24, 2025 at 12:20 PM Leandro Santiago <leandrosansilva@gmail.com>
> wrote:
>
>> I really hope this can be the last iteration, as I ran out of ideas on how
>> to simplify the process, so please let me know your thoughts :-)
>
> I'm not sure I understand the rationale or goal of this. It seems you're
> trying to create a process for extending the source/build tree with
> components not part of our git. Is this something people are interested in?
> I've never heard this use case before.

Yes, that's the goal. I for instance at the moment have been writing a filter in Rust suited to my usecase, that is very unlikely to ever be upstreamed. At the moment I need to hook into different places in the build system, and deal with rebase issues, which is painful.

I am sure other devs have similar problems.

Also, in the same way I'm using Rust, this makes it easier for people to experiment with filters written in other languages of their preference, such as C++ or Zig or whatever, without the concern of whether or not the ffmpeg devs will be willing to upstream their changes.

Another potential usage is extracting some existing "niche" filters into external repositories where they can evolve independently and in a different pace as the main ffmpeg repo.

This can also open space for a "staging" area, where new experimental filters are developed, until they get mature enough to be upstreamed. On a similar fashion as gstreamer plugins, but resolved at build time instead of runtime.

It's important to notice that I am not advocating for turning the current private "API" into a stable, public API. It should be up to the the developers of the external filters to make them build with ffmpeg.

I am not really familiar with the other libav* libraries, but the "external components" could be useful for them as well, for experimental input devices, codecs, etc..

Leandro

>
> Ronald
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable out-of-tree filters

[Leandro Santiago]

On 3/26/25 07:09, softworkz . wrote:
> Hi Ronald,
>
>> -----Original Message-----
>> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Ronald
>> S. Bultje
>> Sent: Dienstag, 25. März 2025 19:05
>> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
>> Subject: Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable
>> out-of-tree filters
>>
>> Hi,
>>
>> On Mon, Mar 24, 2025 at 12:20 PM Leandro Santiago
>> <leandrosansilva@gmail.com>
>> wrote:
>>
>>> I really hope this can be the last iteration, as I ran out of ideas on
>> how
>>> to simplify the process, so please let me know your thoughts :-)
>>
>> I'm not sure I understand the rationale or goal of this. It seems you're
>> trying to create a process for extending the source/build tree with
>> components not part of our git. Is this something people are interested
>> in?
> Yes.
>
>> I've never heard this use case before.
>
> It had been discussed here (tangentially):
>
> https://ffmpeg.org/pipermail/ffmpeg-devel/2025-January/338807.html
>
> and here:
>
> https://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2025-February/340030.html
>
>
> It's kind of an alternative to a run-time plugin-model to which some have expressed reservations.
> Interests are multifold. It makes it easier to maintain custom filters across branches and versions of the main code, it allows people to create things which don't have a chance to get into the codebase for various reasons: like being too specific for general use, being tied to a specific vendor, using remote APIs that are too volatile, being too experimental or whatever else can be. 
> Needless to mention the many AI related use cases. Without providing any kind of extensibility, other solutions will fill that gap and Ffmpeg relevancy will drop in the coming years IMO.

That's a great summary, thank you.

As a follow up, I've just ported (albeit being still very ugly and with build system workarounds) a filter written in Rust to use this implementation of "external filters", and it seems to be working, although I need to test it more. The code can be found at https://gitlab.com/leandrosansilva/ffmpeg-track-sort-filter .

In short, one can build it with:

```

cd ffmpeg
mkdir -p ext/libavfilter/
git clone https://gitlab.com/leandrosansilva/ffmpeg-track-sort-filter ext/libavfilter/track-sort
./configure --enable-libopenvino --enable-libharfbuzz --enable-libfreetype --enable-openssl --disable-static --enable-shared
make
make install

```

And then use the `tracksort` filter as described in the repository (it requires some setup, as it's quite niche, so I won't get into details here).

Leandro

>
> Best,
> sw
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable out-of-tree filters

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 2858 bytes --]

Hi Leandro

On Mon, Mar 24, 2025 at 05:20:02PM +0100, Leandro Santiago wrote:
> In this iteration I've made the following changes, based on the received feedback:
> 
> - No external tools are needed by the build system.
> 
> - The external filters should be put in ext/libavfilter. Anything there will be included.
> 
> - The build system will execute a file called `configure.sh` in the directory of the external filter. This script has access to the functions and variables defined on `./configure`, as it's included via `.` on sh.
> 
> - I will document the "API" for the external filters as soon the approach is approved, but an example of filter can be found at https://gitlab.com/leandrosansilva/ffmpeg-extra-filter-example . If you are unhappy with the example code hosted on gitlab.com, I can move it elsewhere where it gets easier for you to have access to the code.
> 
> - Essentially, an external filter is composed by at least a `configure.sh` and a `Makefile`.
> 
> I really hope this can be the last iteration, as I ran out of ideas on how to simplify the process, so please let me know your thoughts :-)

How does this compare to simply using
git merge

That is each filter developer simply maintaining a fork of ffmpeg and their
filter, in that fork. Adding lines to configure, Makefile, ...

If we take the last filter as a random example, what it chanegd looks like this:

    avfilter/interlace_vulkan: add interlace_vulkan filter

    This is a Vulkan-accelerated version of the existing interlace filter.

 configure                         |   1 +
 doc/filters.texi                  |   2 +-
 libavfilter/Makefile              |   1 +
 libavfilter/allfilters.c          |   1 +
 libavfilter/vf_interlace_vulkan.c | 313 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 317 insertions(+), 1 deletion(-)

The advantage of "git merge" wether by hand or by a automated tool
is that its not limited to what it can do. Its much more powerfull

and the changes outside adding the filter itself are very basic.
Conflicts are something that we can workaround in many ways if they
become a problem

Also it much easier alows transitioning between actually including a filter
in git master, if the community desires that for a filter later.


thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

When the tyrant has disposed of foreign enemies by conquest or treaty, and
there is nothing more to fear from them, then he is always stirring up
some war or other, in order that the people may require a leader. -- Plato

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable out-of-tree filters

[Nicolas George]

Michael Niedermayer (HE12025-03-28):
> How does this compare to simply using
> git merge

If you want to use just one extra component, git merge is indeed better.
But if you want to use multiple components, you might get merge
conflicts in configure and the makefiles.

On the other hand, if adding a component is just dropping a directory¹,
no conflict is possible except for the name of the component².

I grant you, resolving these merge conflicts would be absolutely
trivial. We are expending effort to spare very very little effort to
people.

But I can return the question to you, because it is you who wanted a
plugin system: why do you think a plugin system would be better than git
merge?

Myself, I am 100% in favor of “we are FFmpeg, we distribute a source
code and our plugin system is git merge; if you do not like it try
gstreamer”.


1: Or even a zip file that will be extracted in the build tree.

2: We could force the name of the component to be the name of the
directory or prefixed with it. Then even conflicts in name would be
easily resolved in the copy GUI dialog: “Rename”.

Regards,

-- 
  Nicolas George
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable out-of-tree filters

[softworkz .]

Hi Michael,


> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> Michael Niedermayer
> Sent: Freitag, 28. März 2025 22:38
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable
> out-of-tree filters
> 
> Hi Leandro
> 
> On Mon, Mar 24, 2025 at 05:20:02PM +0100, Leandro Santiago wrote:
> > In this iteration I've made the following changes, based on the
> received feedback:
> >
> > - No external tools are needed by the build system.
> >
> > - The external filters should be put in ext/libavfilter. Anything
> there will be included.
> >
> > - The build system will execute a file called `configure.sh` in the
> directory of the external filter. This script has access to the
> functions and variables defined on `./configure`, as it's included via
> `.` on sh.
> >
> > - I will document the "API" for the external filters as soon the
> approach is approved, but an example of filter can be found at
> https://gitlab.com/leandrosansilva/ffmpeg-extra-filter-example . If you
> are unhappy with the example code hosted on gitlab.com, I can move it
> elsewhere where it gets easier for you to have access to the code.
> >
> > - Essentially, an external filter is composed by at least a
> `configure.sh` and a `Makefile`.
> >
> > I really hope this can be the last iteration, as I ran out of ideas on
> how to simplify the process, so please let me know your thoughts :-)
> 
> How does this compare to simply using
> git merge
> 
> That is each filter developer simply maintaining a fork of ffmpeg and
> their
> filter, in that fork. Adding lines to configure, Makefile, ...
> 

[..]

> 
> The advantage of "git merge" wether by hand or by a automated tool
> is that its not limited to what it can do. Its much more powerfull

Git merge only works when there's a common baseline and the only difference is the filter commit on top that you want to merge. It cannot be used when there are different baselines, e.g. the filter is on top of a the latest master branch and you want to merge it into an older (release) branch, as that would add all the differences, not just the filter.
What you can do is cherry-picking the commit which adds the filter, but the bigger the differences of the baseline, the bigger the problems when cherry-picking. 


> and the changes outside adding the filter itself are very basic.
> Conflicts are something that we can workaround in many ways if they
> become a problem

The changes are basic in fact, but the trouble it is causing each time is beyond basic.

To give you an idea of what I'm talking about I've recorded a short screencast to illustrate what I mean:

https://gist.github.com/softworkz/750da15adb259fa13c6b32277647d54e 

Thanks,
sw




_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable out-of-tree filters

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 3156 bytes --]

Hi

On Fri, Mar 28, 2025 at 10:23:50PM +0000, softworkz . wrote:
[...]
> >
> > The advantage of "git merge" wether by hand or by a automated tool
> > is that its not limited to what it can do. Its much more powerfull
> 
> Git merge only works when there's a common baseline and the only difference is the filter commit on top that you want to merge. It cannot be used when there are different baselines, e.g. the filter is on top of a the latest master branch and you want to merge it into an older (release) branch, as that would add all the differences, not just the filter.
> What you can do is cherry-picking the commit which adds the filter, but the bigger the differences of the baseline, the bigger the problems when cherry-picking. 
> 
> 
> > and the changes outside adding the filter itself are very basic.
> > Conflicts are something that we can workaround in many ways if they
> > become a problem
> 
> The changes are basic in fact, but the trouble it is causing each time is beyond basic.
> 
> To give you an idea of what I'm talking about I've recorded a short screencast to illustrate what I mean:
> 
> https://gist.github.com/softworkz/750da15adb259fa13c6b32277647d54e 

Conflicts can only occur in areas belonging to more than one module
ATM, when adding a filter thats allfilters.c, Makefile, doc/filters.text and configure
(and very similar files for other things than filters)

As nicolas suggested, if each filter is in its own directory no conflict
is possible.
configure just needs to include the Makefile, doc/*.texi, allwhatever.c
from each of these directories

About merges and revission differences.
A filter for ffmpeg 2.0 will possibly not work with 1.0 (in the currect
designs of using the internal API/ABI)

So if you have a filter based on 1.0, one on 1.0.3 and one on 1.0.8
and you merge these with the ffmpeg release 1.0.12
you get exactly the right thing full automatically

You can cherry pick too and the effect is about the same but if filters
share a common component merging will likely be less conflicting

What iam suggesting, i guess is,
* allow new modules to live in seperate directories
    (even without any external modules this gives us fewer conflicts)
* maintain a simple list of such externally maintained modules on
  ffmpeg.org (users need a place to find these modules, especially
  in a world where mallicious code is becoming more widespread)
* really the user can use git merge directly but we could give him
  a tool that gives a clearer success/fail and leaves no failed merges

secret-plan: every halloween we release ffmpeg-monster that merges EVERY
actively maintained module which dosnt conflict. (this would also
encourage module maintainers, to maintain their code)

thx
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

If you drop bombs on a foreign country and kill a hundred thousand
innocent people, expect your government to call the consequence
"unprovoked inhuman terrorist attacks" and use it to justify dropping
more bombs and killing more people. The technology changed, the idea is old.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable out-of-tree filters

[softworkz .]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> Michael Niedermayer
> Sent: Samstag, 29. März 2025 02:17
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable
> out-of-tree filters
> 
> Hi
> 
> On Fri, Mar 28, 2025 at 10:23:50PM +0000, softworkz . wrote:
> [...]
> > >
> > > The advantage of "git merge" wether by hand or by a automated tool
> > > is that its not limited to what it can do. Its much more powerfull
> >
> > Git merge only works when there's a common baseline and the only
> difference is the filter commit on top that you want to merge. It cannot
> be used when there are different baselines, e.g. the filter is on top of
> a the latest master branch and you want to merge it into an older
> (release) branch, as that would add all the differences, not just the
> filter.
> > What you can do is cherry-picking the commit which adds the filter,
> but the bigger the differences of the baseline, the bigger the problems
> when cherry-picking.
> >
> >
> > > and the changes outside adding the filter itself are very basic.
> > > Conflicts are something that we can workaround in many ways if they
> > > become a problem
> >
> > The changes are basic in fact, but the trouble it is causing each time
> is beyond basic.
> >
> > To give you an idea of what I'm talking about I've recorded a short
> screencast to illustrate what I mean:
> >
> > https://gist.github.com/softworkz/750da15adb259fa13c6b32277647d54e
> 
> Conflicts can only occur in areas belonging to more than one module
> ATM, when adding a filter thats allfilters.c, Makefile, doc/filters.text
> and configure
> (and very similar files for other things than filters)
> 
> As nicolas suggested, if each filter is in its own directory no conflict
> is possible.
> configure just needs to include the Makefile, doc/*.texi, allwhatever.c
> from each of these directories
> 
> About merges and revission differences.
> A filter for ffmpeg 2.0 will possibly not work with 1.0 (in the currect
> designs of using the internal API/ABI)
> 
> So if you have a filter based on 1.0, one on 1.0.3 and one on 1.0.8
> and you merge these with the ffmpeg release 1.0.12
> you get exactly the right thing full automatically
> 
> You can cherry pick too and the effect is about the same but if filters
> share a common component merging will likely be less conflicting

Hi Michael,

I suppose you haven't looked at the video. What it is showing are conflicts in exactly all of those files where you think it would be easy going for Git, but unfortunately that's not the case. Even a simple one-line addition can create large conflicting blocks (many lines). This is what I'm talking about and I've created that video because it's not what you would expect to happen, but it happens all the time and it's often a much bigger annoyance than API adaptions.

It's not quite clear why it happens, maybe it has something to do with how Git identifies the context areas of changes. I'm wondering whether it could handle it better if there were one or two blank lines in-between..?

> What iam suggesting, i guess is,
> * allow new modules to live in seperate directories
>     (even without any external modules this gives us fewer conflicts)
> * maintain a simple list of such externally maintained modules on
>   ffmpeg.org (users need a place to find these modules, especially
>   in a world where mallicious code is becoming more widespread)
> * really the user can use git merge directly but we could give him
>   a tool that gives a clearer success/fail and leaves no failed merges

configure, makefile, allfilters and these kinds of files are the troublemakers - from my experience at least.


> secret-plan: every halloween we release ffmpeg-monster that merges EVERY
> actively maintained module which dosnt conflict. (this would also
> encourage module maintainers, to maintain their code)

😊 

sw
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable out-of-tree filters

[Leandro Santiago]

On 3/28/25 22:38, Michael Niedermayer wrote:
> Hi Leandro
>
> On Mon, Mar 24, 2025 at 05:20:02PM +0100, Leandro Santiago wrote:
>> In this iteration I've made the following changes, based on the received feedback:
>>
>> - No external tools are needed by the build system.
>>
>> - The external filters should be put in ext/libavfilter. Anything there will be included.
>>
>> - The build system will execute a file called `configure.sh` in the directory of the external filter. This script has access to the functions and variables defined on `./configure`, as it's included via `.` on sh.
>>
>> - I will document the "API" for the external filters as soon the approach is approved, but an example of filter can be found at https://gitlab.com/leandrosansilva/ffmpeg-extra-filter-example . If you are unhappy with the example code hosted on gitlab.com, I can move it elsewhere where it gets easier for you to have access to the code.
>>
>> - Essentially, an external filter is composed by at least a `configure.sh` and a `Makefile`.
>>
>> I really hope this can be the last iteration, as I ran out of ideas on how to simplify the process, so please let me know your thoughts :-)
> How does this compare to simply using
> git merge

The main difference is being 100% resistant to merge/rebase/cherry-pick conflicts :-)

There is of course the possibility that the filter won't compile at all (or compile and be buggy) due to the changes in libavfilter or build system.

The good side of it is that it's up to the developer of the external filter to worry about keeping track with changes in the ffmpeg core, not the other way round.

>
> That is each filter developer simply maintaining a fork of ffmpeg and their
> filter, in that fork. Adding lines to configure, Makefile, ...
>
> If we take the last filter as a random example, what it chanegd looks like this:
>
>     avfilter/interlace_vulkan: add interlace_vulkan filter
>
>     This is a Vulkan-accelerated version of the existing interlace filter.
>
>  configure                         |   1 +
>  doc/filters.texi                  |   2 +-
>  libavfilter/Makefile              |   1 +
>  libavfilter/allfilters.c          |   1 +
>  libavfilter/vf_interlace_vulkan.c | 313 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  5 files changed, 317 insertions(+), 1 deletion(-)
Interesting, the case of documentation integration is still an open issue on my proposal. I'll have a look at it.
>
> The advantage of "git merge" wether by hand or by a automated tool
> is that its not limited to what it can do. Its much more powerfull

How would that work when the user is building ffmpeg+external-filters via release tarballs?

>
> and the changes outside adding the filter itself are very basic.
> Conflicts are something that we can workaround in many ways if they
> become a problem
>
> Also it much easier alows transitioning between actually including a filter
> in git master, if the community desires that for a filter later.

For those cases, git has good support for merging unrelated repositories with unrelated histories those days. I have experienced using it on less complex codebases and it worked really well.

Do you think it would be problematic? [1]

[1] https://git-scm.com/docs/git-merge#Documentation/git-merge.txt---allow-unrelated-histories

>
>
> thx
>
> [...]
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable out-of-tree filters

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 3876 bytes --]

Hi

On Sat, Mar 29, 2025 at 01:45:38AM +0000, softworkz . wrote:
> 
> 
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> > Michael Niedermayer
> > Sent: Samstag, 29. März 2025 02:17
> > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable
> > out-of-tree filters
> > 
> > Hi
> > 
> > On Fri, Mar 28, 2025 at 10:23:50PM +0000, softworkz . wrote:
> > [...]
> > > >
> > > > The advantage of "git merge" wether by hand or by a automated tool
> > > > is that its not limited to what it can do. Its much more powerfull
> > >
> > > Git merge only works when there's a common baseline and the only
> > difference is the filter commit on top that you want to merge. It cannot
> > be used when there are different baselines, e.g. the filter is on top of
> > a the latest master branch and you want to merge it into an older
> > (release) branch, as that would add all the differences, not just the
> > filter.
> > > What you can do is cherry-picking the commit which adds the filter,
> > but the bigger the differences of the baseline, the bigger the problems
> > when cherry-picking.
> > >
> > >
> > > > and the changes outside adding the filter itself are very basic.
> > > > Conflicts are something that we can workaround in many ways if they
> > > > become a problem
> > >
> > > The changes are basic in fact, but the trouble it is causing each time
> > is beyond basic.
> > >
> > > To give you an idea of what I'm talking about I've recorded a short
> > screencast to illustrate what I mean:
> > >
> > > https://gist.github.com/softworkz/750da15adb259fa13c6b32277647d54e
> > 
> > Conflicts can only occur in areas belonging to more than one module
> > ATM, when adding a filter thats allfilters.c, Makefile, doc/filters.text
> > and configure
> > (and very similar files for other things than filters)
> > 
> > As nicolas suggested, if each filter is in its own directory no conflict
> > is possible.
> > configure just needs to include the Makefile, doc/*.texi, allwhatever.c
> > from each of these directories
> > 
> > About merges and revission differences.
> > A filter for ffmpeg 2.0 will possibly not work with 1.0 (in the currect
> > designs of using the internal API/ABI)
> > 
> > So if you have a filter based on 1.0, one on 1.0.3 and one on 1.0.8
> > and you merge these with the ffmpeg release 1.0.12
> > you get exactly the right thing full automatically
> > 
> > You can cherry pick too and the effect is about the same but if filters
> > share a common component merging will likely be less conflicting
> 
> Hi Michael,
> 
> I suppose you haven't looked at the video. What it is showing are conflicts in exactly all of those files where you think it would be easy going for Git, but unfortunately that's not the case. Even a simple one-line addition can create large conflicting blocks (many lines). This is what I'm talking about and I've created that video because it's not what you would expect to happen, but it happens all the time and it's often a much bigger annoyance than API adaptions.
> 
> It's not quite clear why it happens, maybe it has something to do with how Git identifies the context areas of changes. I'm wondering whether it could handle it better if there were one or two blank lines in-between..?

can you show an example with command line git ?
like a simple sequence of commands that result in problems, that i can
replicate to look at what happens exactly

git merge is widely used in project MUCH bigger than our codebase

thx
[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

During times of universal deceit, telling the truth becomes a
revolutionary act. -- George Orwell

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable out-of-tree filters

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 3196 bytes --]

Hi

On Sat, Mar 29, 2025 at 03:52:52PM +0100, Leandro Santiago wrote:
[...]
> The good side of it is that it's up to the developer of the external filter to worry about keeping track with changes in the ffmpeg core, not the other way round.
> 
> >
> > That is each filter developer simply maintaining a fork of ffmpeg and their
> > filter, in that fork. Adding lines to configure, Makefile, ...
> >
> > If we take the last filter as a random example, what it chanegd looks like this:
> >
> >     avfilter/interlace_vulkan: add interlace_vulkan filter
> >
> >     This is a Vulkan-accelerated version of the existing interlace filter.
> >
> >  configure                         |   1 +
> >  doc/filters.texi                  |   2 +-
> >  libavfilter/Makefile              |   1 +
> >  libavfilter/allfilters.c          |   1 +
> >  libavfilter/vf_interlace_vulkan.c | 313 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >  5 files changed, 317 insertions(+), 1 deletion(-)
> Interesting, the case of documentation integration is still an open issue on my proposal. I'll have a look at it.
> >
> > The advantage of "git merge" wether by hand or by a automated tool
> > is that its not limited to what it can do. Its much more powerfull
> 
> How would that work when the user is building ffmpeg+external-filters via release tarballs?

If one seriously wanted this, we could include the .git directory in the tarball.

But someone who wants to include any type of module at compile time will
have to work with various tools. like a compiler and configure
and some way to obtain the module into the right place.

typing
git pull --no-rebase --no-edit https://github.com/author/project branch
is simpler

ATM, i see a system we need to maintain that supports only libavfilter
and git which we dont have to maintain that supports every module
Iam lazy, so iam drawen toward git

If git fails to work, then we can look into other solutions,
can you try simply using git and check if it would work ?


> 
> >
> > and the changes outside adding the filter itself are very basic.
> > Conflicts are something that we can workaround in many ways if they
> > become a problem
> >
> > Also it much easier alows transitioning between actually including a filter
> > in git master, if the community desires that for a filter later.
> 
> For those cases, git has good support for merging unrelated repositories with unrelated histories those days. I have experienced using it on less complex codebases and it worked really well.
> 

> Do you think it would be problematic? [1]
> 
> [1] https://git-scm.com/docs/git-merge#Documentation/git-merge.txt---allow-unrelated-histories

yes

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Freedom in capitalist society always remains about the same as it was in
ancient Greek republics: Freedom for slave owners. -- Vladimir Lenin

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable out-of-tree filters

[softworkz .]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> Michael Niedermayer
> Sent: Sonntag, 30. März 2025 00:30
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept: enable
> out-of-tree filters
> 
> Hi
> 
> On Sat, Mar 29, 2025 at 01:45:38AM +0000, softworkz . wrote:
> >
> >
> > > -----Original Message-----
> > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> > > Michael Niedermayer
> > > Sent: Samstag, 29. März 2025 02:17
> > > To: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org>
> > > Subject: Re: [FFmpeg-devel] [PATCH] avfilter: Proof of Concept:
> enable
> > > out-of-tree filters
> > >
> > > Hi
> > >
> > > On Fri, Mar 28, 2025 at 10:23:50PM +0000, softworkz . wrote:
> > > [...]
> > > > >
> > > > > The advantage of "git merge" wether by hand or by a automated
> tool
> > > > > is that its not limited to what it can do. Its much more
> powerfull
> > > >
> > > > Git merge only works when there's a common baseline and the only
> > > difference is the filter commit on top that you want to merge. It
> cannot
> > > be used when there are different baselines, e.g. the filter is on
> top of
> > > a the latest master branch and you want to merge it into an older
> > > (release) branch, as that would add all the differences, not just
> the
> > > filter.
> > > > What you can do is cherry-picking the commit which adds the
> filter,
> > > but the bigger the differences of the baseline, the bigger the
> problems
> > > when cherry-picking.
> > > >
> > > >
> > > > > and the changes outside adding the filter itself are very basic.
> > > > > Conflicts are something that we can workaround in many ways if
> they
> > > > > become a problem
> > > >
> > > > The changes are basic in fact, but the trouble it is causing each
> time
> > > is beyond basic.
> > > >
> > > > To give you an idea of what I'm talking about I've recorded a
> short
> > > screencast to illustrate what I mean:
> > > >
> > > > https://gist.github.com/softworkz/750da15adb259fa13c6b32277647d54e
> > >
> > > Conflicts can only occur in areas belonging to more than one module
> > > ATM, when adding a filter thats allfilters.c, Makefile,
> doc/filters.text
> > > and configure
> > > (and very similar files for other things than filters)
> > >
> > > As nicolas suggested, if each filter is in its own directory no
> conflict
> > > is possible.
> > > configure just needs to include the Makefile, doc/*.texi,
> allwhatever.c
> > > from each of these directories
> > >
> > > About merges and revission differences.
> > > A filter for ffmpeg 2.0 will possibly not work with 1.0 (in the
> currect
> > > designs of using the internal API/ABI)
> > >
> > > So if you have a filter based on 1.0, one on 1.0.3 and one on 1.0.8
> > > and you merge these with the ffmpeg release 1.0.12
> > > you get exactly the right thing full automatically
> > >
> > > You can cherry pick too and the effect is about the same but if
> filters
> > > share a common component merging will likely be less conflicting
> >
> > Hi Michael,
> >
> > I suppose you haven't looked at the video. What it is showing are
> conflicts in exactly all of those files where you think it would be easy
> going for Git, but unfortunately that's not the case. Even a simple one-
> line addition can create large conflicting blocks (many lines). This is
> what I'm talking about and I've created that video because it's not what
> you would expect to happen, but it happens all the time and it's often a
> much bigger annoyance than API adaptions.
> >
> > It's not quite clear why it happens, maybe it has something to do with
> how Git identifies the context areas of changes. I'm wondering whether
> it could handle it better if there were one or two blank lines in-
> between..?
> 
> 
> git merge is widely used in project MUCH bigger than our codebase

When the branch to merge has the same baseline with only the additional commits on top it works fine of course. That's equivalent to the patchsets on the ML needing to be created against the master branch.
It also works fine for larger merges like when you have a beta that's far ahead of a stable branch and merge it finally into stable. But merging something from an older and possible diverged baseline on top of a newer baseline is a different story.


> can you show an example with command line git ?
> like a simple sequence of commands that result in problems, that i can
> replicate to look at what happens exactly


Sure:


git clone https://git.ffmpeg.org/ffmpeg.git

cd ffmpeg

git checkout master

git remote add softworkz https://github.com/softworkz/FFmpeg

git fetch softworkz

# Test 1
git cherry-pick 39bb78e449a1df67ed5b3b0fafa86af7e115cb36


# Test 2
git cherry-pick --abort
git cherry-pick 7dbb6890f2eb6035eb9123ce2039f6dbbc082b3d


sw

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".