[FFmpeg-devel] [PATCH v3] avformat/avisynth: fix segfault when also compiled with libvmaf

[FFmpeg-devel] [PATCH v3] avformat/avisynth: fix segfault when also compiled with libvmaf

[Daniel Bermond]

When FFmpeg is compiled with support for both avisynth and libvmaf,
a segmentation fault occurs when using avisynth (.avs) input.

This happens because both avisynthplus and vmaf have the exactly
same C++ symbol 'Cache::~Cache()'[1][2], which is a C++ destructor.
When using avisynth input, this destructor is also called in the
vmaf side, although no vmaf object was created, thus causing a
segmentation fault.

Without changing this conflicting symbol in avisynthplus and/or
vmaf upstream code, a solution is to switch the 'RTLD_LOCAL'[3]
flag for 'RTLD_DEEPBIND'[4] when calling 'dlopen()' on the
avisynthplus library. This will make the avisynthplus symbols
loaded by 'dlopen()' to be ahead of the symbols loaded in the
global scope, and consequently avoid the segmentation fault by
preventing the conflicting vmaf C++ destructor to be called.

More details can be seen in the discussion of the Arch Linux bug
report[5] about this issue.

[1] https://github.com/AviSynth/AviSynthPlus/blob/v3.7.5/avs_core/core/cache.h#L62
[2] https://github.com/Netflix/vmaf/blob/v3.0.0/libvmaf/src/svm.cpp#L75
[3] https://man.archlinux.org/man/dlopen.3#RTLD_LOCAL
[4] https://man.archlinux.org/man/dlopen.3#RTLD_DEEPBIND
[5] https://gitlab.archlinux.org/archlinux/packaging/packages/ffmpeg/-/issues/19

Signed-off-by: Daniel Bermond <danielbermond@gmail.com>
---
 libavformat/avisynth.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavformat/avisynth.c b/libavformat/avisynth.c
index cb2be10925..bef61b4a60 100644
--- a/libavformat/avisynth.c
+++ b/libavformat/avisynth.c
@@ -120,7 +120,7 @@ typedef struct AviSynthContext {
 
 static av_cold int avisynth_load_library(AviSynthContext *avs)
 {
-    avs->avs_library.library = dlopen(AVISYNTH_LIB, RTLD_NOW | RTLD_LOCAL);
+    avs->avs_library.library = dlopen(AVISYNTH_LIB, RTLD_NOW | RTLD_DEEPBIND);
     if (!avs->avs_library.library)
         return AVERROR_UNKNOWN;
 
-- 
2.50.1
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v3] avformat/avisynth: fix segfault when also compiled with libvmaf

[Kacper Michajlow]

On Mon, 28 Jul 2025 at 02:04, Daniel Bermond <danielbermond@gmail.com> wrote:
>
> When FFmpeg is compiled with support for both avisynth and libvmaf,
> a segmentation fault occurs when using avisynth (.avs) input.
>
> This happens because both avisynthplus and vmaf have the exactly
> same C++ symbol 'Cache::~Cache()'[1][2], which is a C++ destructor.
> When using avisynth input, this destructor is also called in the
> vmaf side, although no vmaf object was created, thus causing a
> segmentation fault.
>
> Without changing this conflicting symbol in avisynthplus and/or
> vmaf upstream code, a solution is to switch the 'RTLD_LOCAL'[3]

I'm not opposed to a workaround to the issues as an interim solution,
but has this issue been reported to upstream? Classes with such
generic names should be put in a namespace in the respective projects.
Additionally it would be good for those projects to be built with
-fvisibility=hidden to reduce the amount of exposed symbols and only
expose API entry points.

- Kacper

> flag for 'RTLD_DEEPBIND'[4] when calling 'dlopen()' on the
> avisynthplus library. This will make the avisynthplus symbols
> loaded by 'dlopen()' to be ahead of the symbols loaded in the
> global scope, and consequently avoid the segmentation fault by
> preventing the conflicting vmaf C++ destructor to be called.
>
> More details can be seen in the discussion of the Arch Linux bug
> report[5] about this issue.
>
> [1] https://github.com/AviSynth/AviSynthPlus/blob/v3.7.5/avs_core/core/cache.h#L62
> [2] https://github.com/Netflix/vmaf/blob/v3.0.0/libvmaf/src/svm.cpp#L75
> [3] https://man.archlinux.org/man/dlopen.3#RTLD_LOCAL
> [4] https://man.archlinux.org/man/dlopen.3#RTLD_DEEPBIND
> [5] https://gitlab.archlinux.org/archlinux/packaging/packages/ffmpeg/-/issues/19
>
> Signed-off-by: Daniel Bermond <danielbermond@gmail.com>
> ---
>  libavformat/avisynth.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/libavformat/avisynth.c b/libavformat/avisynth.c
> index cb2be10925..bef61b4a60 100644
> --- a/libavformat/avisynth.c
> +++ b/libavformat/avisynth.c
> @@ -120,7 +120,7 @@ typedef struct AviSynthContext {
>
>  static av_cold int avisynth_load_library(AviSynthContext *avs)
>  {
> -    avs->avs_library.library = dlopen(AVISYNTH_LIB, RTLD_NOW | RTLD_LOCAL);
> +    avs->avs_library.library = dlopen(AVISYNTH_LIB, RTLD_NOW | RTLD_DEEPBIND);
>      if (!avs->avs_library.library)
>          return AVERROR_UNKNOWN;
>
> --
> 2.50.1
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v3] avformat/avisynth: fix segfault when also compiled with libvmaf

[Stephen Hutchinson]

On 7/27/25 8:34 PM, Kacper Michajlow wrote:
> On Mon, 28 Jul 2025 at 02:04, Daniel Bermond <danielbermond@gmail.com> wrote:
>>
>> When FFmpeg is compiled with support for both avisynth and libvmaf,
>> a segmentation fault occurs when using avisynth (.avs) input.
>>
>> This happens because both avisynthplus and vmaf have the exactly
>> same C++ symbol 'Cache::~Cache()'[1][2], which is a C++ destructor.
>> When using avisynth input, this destructor is also called in the
>> vmaf side, although no vmaf object was created, thus causing a
>> segmentation fault.
>>
>> Without changing this conflicting symbol in avisynthplus and/or
>> vmaf upstream code, a solution is to switch the 'RTLD_LOCAL'[3]
> 
> I'm not opposed to a workaround to the issues as an interim solution,
> but has this issue been reported to upstream? Classes with such
> generic names should be put in a namespace in the respective projects.
> Additionally it would be good for those projects to be built with
> -fvisibility=hidden to reduce the amount of exposed symbols and only
> expose API entry points.
> 

As a stopgap, yeah, this is probably the cleanest solution.
It does need to be resolved upstream, though at least on the
AviSynth+ side, I don't know how/if the reason this hadn't
already been done was because it would have broken existing
plugin compatibility.  Still worth looking at, though.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v3] avformat/avisynth: fix segfault when also compiled with libvmaf

[Daniel Bermond]

On 7/27/25 21:34, Kacper Michajlow wrote:
> On Mon, 28 Jul 2025 at 02:04, Daniel Bermond <danielbermond@gmail.com> wrote:
>>
>> When FFmpeg is compiled with support for both avisynth and libvmaf,
>> a segmentation fault occurs when using avisynth (.avs) input.
>>
>> This happens because both avisynthplus and vmaf have the exactly
>> same C++ symbol 'Cache::~Cache()'[1][2], which is a C++ destructor.
>> When using avisynth input, this destructor is also called in the
>> vmaf side, although no vmaf object was created, thus causing a
>> segmentation fault.
>>
>> Without changing this conflicting symbol in avisynthplus and/or
>> vmaf upstream code, a solution is to switch the 'RTLD_LOCAL'[3]
> 
> I'm not opposed to a workaround to the issues as an interim solution,
> but has this issue been reported to upstream? Classes with such
> generic names should be put in a namespace in the respective projects.

Thank you for the reply.

I have not reported this upstream. Both projects does not follow
such rules as encapsulating classes in a unique namespaces and/or
uniquely naming the classes. They simply use generic class names
in the top namespace of the public API. This is their code
design/style/standards, present the entire source code.
The programmer/manager is supposedly to be aware of such issues
in the code design stage, and yet they decided to be like this.

> Additionally it would be good for those projects to be built with
> -fvisibility=hidden to reduce the amount of exposed symbols and only
> expose API entry points.

Compiling avisynthplus with '-fvisibility=hidden' makes ffmpeg to
give an error when trying to open an .avs input:

'Error opening input: Unknown error occurred'

No more details when using the ffmpeg '-verbose' option.

But compiling vmaf with this flag works and avoids the segfault.
Not sure if this is the best solution, as this would be needed in
every GNU/Linux distribution out there that uses avisynth + vmaf
in ffmpeg, but surely it is another good solution. Thanks for the
suggestion.

> 
> - Kacper
> 
>> flag for 'RTLD_DEEPBIND'[4] when calling 'dlopen()' on the
>> avisynthplus library. This will make the avisynthplus symbols
>> loaded by 'dlopen()' to be ahead of the symbols loaded in the
>> global scope, and consequently avoid the segmentation fault by
>> preventing the conflicting vmaf C++ destructor to be called.
>>
>> More details can be seen in the discussion of the Arch Linux bug
>> report[5] about this issue.
>>
>> [1] https://github.com/AviSynth/AviSynthPlus/blob/v3.7.5/avs_core/core/cache.h#L62
>> [2] https://github.com/Netflix/vmaf/blob/v3.0.0/libvmaf/src/svm.cpp#L75
>> [3] https://man.archlinux.org/man/dlopen.3#RTLD_LOCAL
>> [4] https://man.archlinux.org/man/dlopen.3#RTLD_DEEPBIND
>> [5] https://gitlab.archlinux.org/archlinux/packaging/packages/ffmpeg/-/issues/19
>>
>> Signed-off-by: Daniel Bermond <danielbermond@gmail.com>
>> ---
>>  libavformat/avisynth.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/libavformat/avisynth.c b/libavformat/avisynth.c
>> index cb2be10925..bef61b4a60 100644
>> --- a/libavformat/avisynth.c
>> +++ b/libavformat/avisynth.c
>> @@ -120,7 +120,7 @@ typedef struct AviSynthContext {
>>
>>  static av_cold int avisynth_load_library(AviSynthContext *avs)
>>  {
>> -    avs->avs_library.library = dlopen(AVISYNTH_LIB, RTLD_NOW | RTLD_LOCAL);
>> +    avs->avs_library.library = dlopen(AVISYNTH_LIB, RTLD_NOW | RTLD_DEEPBIND);
>>      if (!avs->avs_library.library)
>>          return AVERROR_UNKNOWN;
>>
>> --
>> 2.50.1
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

-- 
Best regards,
Daniel Bermond

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v3] avformat/avisynth: fix segfault when also compiled with libvmaf

[Stephen Hutchinson]

On 7/27/25 9:38 PM, Stephen Hutchinson wrote:
> As a stopgap, yeah, this is probably the cleanest solution.
> It does need to be resolved upstream, though at least on the
> AviSynth+ side, I don't know how/if the reason this hadn't
> already been done was because it would have broken existing
> plugin compatibility.  Still worth looking at, though.

At least on first blush, a rename to AvsCache¹ doesn't seem
to have broken ImageSeq or FFMS2, but that's not to say other
plugins don't do stupid things.

Naming will need to be checked for things that are too generic,
but this seems to fix the immediate case without having to
change FFmpeg's dlopen invocation.

¹https://github.com/AviSynth/AviSynthPlus/pull/452
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".