[FFmpeg-devel] [PATCH v2 0/3] hwcontext_vaapi: dlopen libva-x11 and libva-drm

[FFmpeg-devel] [PATCH v2 0/3] hwcontext_vaapi: dlopen libva-x11 and libva-drm

[Emil Velikov]

Greetings everyone,

As you may know the libva* set of libraries share an internal ABI
between them. In a resent libva commit, the va_fool API was removed.

Thus if one is to mix different versions of libva.so and libva-x11.so
they will get an error, leading to a crash of the whole stack.

The simple solution is to dlopen() the winsys components, like libva-x11
and libva-drm. The changes are pretty minor and allow us to handle this
king of issues.

Comments and suggestions are welcome, but please me gentle it's my first
time hacking on ffmpeg :-P

Thanks
Emil

Changes in v2:
 - add linkage against libdl

Aside:
 - Please consider backporting it to the stable branches in due time.
 - I've noticed that we leak state in the error paths, happy to send
   follow-up patches if you'd like those fixed.
 - My TODO includes reducing the massive ABI between libva* and
   backend drivers, to a single extra "registration" API entrypoint.

Emil Velikov (3):
  hwcontext_vaapi: do not link against libva-x11.so
  hwcontext_vaapi: do not link against libva-drm.so
  hwcontext_vaapi: #if guard VAAPI_DRM specifics

 configure                   |  2 +-
 libavutil/hwcontext_vaapi.c | 92 +++++++++++++++++++++++++++++++++++--
 2 files changed, 90 insertions(+), 4 deletions(-)

-- 
2.37.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 0/3] hwcontext_vaapi: dlopen libva-x11 and libva-drm

[Nicolas George]

[-- Attachment #1.1: Type: text/plain, Size: 527 bytes --]

Emil Velikov (12022-07-19):
> As you may know the libva* set of libraries share an internal ABI
> between them. In a resent libva commit, the va_fool API was removed.
> 
> Thus if one is to mix different versions of libva.so and libva-x11.so
> they will get an error, leading to a crash of the whole stack.
> 
> The simple solution is

... a configure check.

If the person who installs replaces a library with another, it is their
responsibility to check they are compatible.

Regards,

-- 
  Nicolas George

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 0/3] hwcontext_vaapi: dlopen libva-x11 and libva-drm

[Emil Velikov]

On Tue, 19 Jul 2022 at 19:16, Nicolas George <george@nsup.org> wrote:
>
> Emil Velikov (12022-07-19):
> > As you may know the libva* set of libraries share an internal ABI
> > between them. In a resent libva commit, the va_fool API was removed.
> >
> > Thus if one is to mix different versions of libva.so and libva-x11.so
> > they will get an error, leading to a crash of the whole stack.
> >
> > The simple solution is
>
> ... a configure check.
>
> If the person who installs replaces a library with another, it is their
> responsibility to check they are compatible.
>

While I wholeheartedly agree, it's not so easy to enforce compile time
decisions at runtime. In the past, I have debugged and reported issues
where Linux distributions do not enforce the above.

We do have the typical Linux distribution model (where we have dozens
upon distros) and other distribution models. IMHO checking each
instance and combination doesn't scale. We could bring awareness to
the issue in say distribution/workflow X, which sadly may come as
finger-pointing and thus alienating.

Hope that makes sense and the team is willing to consider the extra 90
lines worth of code.

Thanks in advance
Emil
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 0/3] hwcontext_vaapi: dlopen libva-x11 and libva-drm

[Mark Thompson]

On 20/07/2022 17:41, Emil Velikov wrote:
> On Tue, 19 Jul 2022 at 19:16, Nicolas George <george@nsup.org> wrote:
>>
>> Emil Velikov (12022-07-19):
>>> As you may know the libva* set of libraries share an internal ABI
>>> between them. In a resent libva commit, the va_fool API was removed.
>>>
>>> Thus if one is to mix different versions of libva.so and libva-x11.so
>>> they will get an error, leading to a crash of the whole stack.
>>>
>>> The simple solution is
>>
>> ... a configure check.
>>
>> If the person who installs replaces a library with another, it is their
>> responsibility to check they are compatible.
>>
> 
> While I wholeheartedly agree, it's not so easy to enforce compile time
> decisions at runtime. In the past, I have debugged and reported issues
> where Linux distributions do not enforce the above.
> 
> We do have the typical Linux distribution model (where we have dozens
> upon distros) and other distribution models. IMHO checking each
> instance and combination doesn't scale. We could bring awareness to
> the issue in say distribution/workflow X, which sadly may come as
> finger-pointing and thus alienating.
> 
> Hope that makes sense and the team is willing to consider the extra 90
> lines worth of code.

The argument "libfoo can be broken in some particular configuration, so lets use dlopen() to make errors happen later" seems like it applies to every library.  Why is this case so special?  Who are the users running into this specific problem and who are stuck with broken versions they can't update?

(Also, shouldn't lazy binding save people in this situation if they don't actually use the feature, as they presumably don't if barfing at runtime makes sense?)

Tbh I don't think FFmpeg libraries are the right place to be putting this sort of workaround.

Thanks,

- Mark
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 0/3] hwcontext_vaapi: dlopen libva-x11 and libva-drm

[Emil Velikov]

On Thu, 21 Jul 2022 at 21:47, Mark Thompson <sw@jkqxz.net> wrote:
>
> On 20/07/2022 17:41, Emil Velikov wrote:
> > On Tue, 19 Jul 2022 at 19:16, Nicolas George <george@nsup.org> wrote:
> >>
> >> Emil Velikov (12022-07-19):
> >>> As you may know the libva* set of libraries share an internal ABI
> >>> between them. In a resent libva commit, the va_fool API was removed.
> >>>
> >>> Thus if one is to mix different versions of libva.so and libva-x11.so
> >>> they will get an error, leading to a crash of the whole stack.
> >>>
> >>> The simple solution is
> >>
> >> ... a configure check.
> >>
> >> If the person who installs replaces a library with another, it is their
> >> responsibility to check they are compatible.
> >>
> >
> > While I wholeheartedly agree, it's not so easy to enforce compile time
> > decisions at runtime. In the past, I have debugged and reported issues
> > where Linux distributions do not enforce the above.
> >
> > We do have the typical Linux distribution model (where we have dozens
> > upon distros) and other distribution models. IMHO checking each
> > instance and combination doesn't scale. We could bring awareness to
> > the issue in say distribution/workflow X, which sadly may come as
> > finger-pointing and thus alienating.
> >
> > Hope that makes sense and the team is willing to consider the extra 90
> > lines worth of code.
>
> The argument "libfoo can be broken in some particular configuration, so lets use dlopen() to make errors happen later" seems like it applies to every library.  Why is this case so special?  Who are the users running into this specific problem and who are stuck with broken versions they can't update?
>
It's a long story, hope I don't bore you to death :-P

Even though I've been itching to hack on ffmpeg for a while, the bug
that allowed me to do that is
https://github.com/ValveSoftware/steam-for-linux/issues/8673

As a background, steam as well as some of the programs/games shipped
use libraries provided by ffmpeg. In addition, steam ships with a
steam runtime, which is effectively a partial chroot of an old Ubuntu.
For various compatibility reasons, one cannot simply update it, so the
startup scripting will try and promote a set of the host libraries (if
newer) so that they're used instead of the bundled Ubuntu ones.

What happens in the libva case is that distributions can provide only
libva.so and omit libva-x11.so. Which due to the internal ABI break
(removal of the va_fool API), means that steam and likely some games
will simply crash out.

Now let me try and draw an analogy to another set of libraries which
also share internal ABI - libdrm.so, libdrm_nouveau.so,
libdrm_amdgpu.so libdrm_intel.so, etc. To the best of my knowledge
there was no breakage in there be that internal or public ABI.

In addition, while distribution may allow you to install only some
(say libdrm.so without libdrm_intel.so), a pair of those is pulled by
the respective GL and Vulkan drivers. For example: the amdgpu GL
driver (amdgpu_dri.so) and radv Vulkand driver (libvulkan_radeon.so)
depend on libdrm_amdgpu.so and libdrm.so. Hence, in practical terms
users cannot hit a similar issue... unless they very very deliberately
try to do so.

So while one solution is to go around telling users and distributions
that they're "doing it wrong", IMHO a more pragmatic solution is to
include this brief workaround in ffmpeg. At least in the short to mid
term.
As mentioned in the cover letter (sorry again for sending the series
multiple times), I have some plans for a proper long term, which would
reside in libva. Alas as you have experienced yourself, the libva
maintainers can be rather busy, so we're looking at least a couple of
months until a new libva release is out and further X months, until it
ripples down to end-users.

> (Also, shouldn't lazy binding save people in this situation if they don't actually use the feature, as they presumably don't if barfing at runtime makes sense?)
>
I suspect you meant using RTLD_LAZY - I believe it should work. Will
test and re-spin for the next revision.

Hope the above makes sense. I tried to stay brief and on-point, but if
anyone likes further details I will be happy to provide.

Thanks
Emil
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 0/3] hwcontext_vaapi: dlopen libva-x11 and libva-drm

[Timo Rothenpieler]

On 27/07/2022 21:51, Emil Velikov wrote:
> On Thu, 21 Jul 2022 at 21:47, Mark Thompson <sw@jkqxz.net> wrote:
>>
>> On 20/07/2022 17:41, Emil Velikov wrote:
>>> On Tue, 19 Jul 2022 at 19:16, Nicolas George <george@nsup.org> wrote:
>>>>
>>>> Emil Velikov (12022-07-19):
>>>>> As you may know the libva* set of libraries share an internal ABI
>>>>> between them. In a resent libva commit, the va_fool API was removed.
>>>>>
>>>>> Thus if one is to mix different versions of libva.so and libva-x11.so
>>>>> they will get an error, leading to a crash of the whole stack.
>>>>>
>>>>> The simple solution is
>>>>
>>>> ... a configure check.
>>>>
>>>> If the person who installs replaces a library with another, it is their
>>>> responsibility to check they are compatible.
>>>>
>>>
>>> While I wholeheartedly agree, it's not so easy to enforce compile time
>>> decisions at runtime. In the past, I have debugged and reported issues
>>> where Linux distributions do not enforce the above.
>>>
>>> We do have the typical Linux distribution model (where we have dozens
>>> upon distros) and other distribution models. IMHO checking each
>>> instance and combination doesn't scale. We could bring awareness to
>>> the issue in say distribution/workflow X, which sadly may come as
>>> finger-pointing and thus alienating.
>>>
>>> Hope that makes sense and the team is willing to consider the extra 90
>>> lines worth of code.
>>
>> The argument "libfoo can be broken in some particular configuration, so lets use dlopen() to make errors happen later" seems like it applies to every library.  Why is this case so special?  Who are the users running into this specific problem and who are stuck with broken versions they can't update?
>>
> It's a long story, hope I don't bore you to death :-P
> 
> Even though I've been itching to hack on ffmpeg for a while, the bug
> that allowed me to do that is
> https://github.com/ValveSoftware/steam-for-linux/issues/8673
> 
> As a background, steam as well as some of the programs/games shipped
> use libraries provided by ffmpeg. In addition, steam ships with a
> steam runtime, which is effectively a partial chroot of an old Ubuntu.
> For various compatibility reasons, one cannot simply update it, so the
> startup scripting will try and promote a set of the host libraries (if
> newer) so that they're used instead of the bundled Ubuntu ones.

That sounds incredibly broken and will of course cause stuff to break.
I see the issue lies with that magic script, not with ffmpeg.

You could make that exact argument for literally every single external 
library, and we don't dlopen() most of them.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 0/3] hwcontext_vaapi: dlopen libva-x11 and libva-drm

[Emil Velikov]

Greetings Timo,

On Wed, 27 Jul 2022 at 21:00, Timo Rothenpieler <timo@rothenpieler.org> wrote:

> That sounds incredibly broken and will of course cause stuff to break.
> I see the issue lies with that magic script, not with ffmpeg.
>

Apologies, never meant to imply that ffmpeg is broken - far from it.
Simply put libva broke their ABI, without bumping the major version.

Hence the libva ABI should be cleaned up and fixed - as mentioned in
my original cover letter and the hunk of my previous email that got
mysteriously removed :-P
As also mentioned, the maintainers are slow, so the odds of having a
fix that reaches end users this year are very close to zero.

What I'm asking here is for a quick short-to-mid term workaround, to
be applied in ffmpeg.

While the startup script is fragile, a similar issue can happen _even
without_ it.

> You could make that exact argument for literally every single external
> library, and we don't dlopen() most of them.

I tried to clarify why and how the libva use-case varies from other
libraries, in my email that you've trimmed out. If my analysis is
incomplete or off, kindly point me to the place that comes as such.

If any developers feel like it might be easier moving some of this
discussion to IRC, do let me know. I can hop on the ffmpeg-devel.

Thanks
Emil
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 0/3] hwcontext_vaapi: dlopen libva-x11 and libva-drm

[Emil Velikov]

On Wed, 27 Jul 2022 at 20:51, Emil Velikov <emil.l.velikov@gmail.com> wrote:
>
> On Thu, 21 Jul 2022 at 21:47, Mark Thompson <sw@jkqxz.net> wrote:
> >
> > On 20/07/2022 17:41, Emil Velikov wrote:
> > > On Tue, 19 Jul 2022 at 19:16, Nicolas George <george@nsup.org> wrote:
> > >>
> > >> Emil Velikov (12022-07-19):
> > >>> As you may know the libva* set of libraries share an internal ABI
> > >>> between them. In a resent libva commit, the va_fool API was removed.
> > >>>
> > >>> Thus if one is to mix different versions of libva.so and libva-x11.so
> > >>> they will get an error, leading to a crash of the whole stack.
> > >>>
> > >>> The simple solution is
> > >>
> > >> ... a configure check.
> > >>
> > >> If the person who installs replaces a library with another, it is their
> > >> responsibility to check they are compatible.
> > >>
> > >
> > > While I wholeheartedly agree, it's not so easy to enforce compile time
> > > decisions at runtime. In the past, I have debugged and reported issues
> > > where Linux distributions do not enforce the above.
> > >
> > > We do have the typical Linux distribution model (where we have dozens
> > > upon distros) and other distribution models. IMHO checking each
> > > instance and combination doesn't scale. We could bring awareness to
> > > the issue in say distribution/workflow X, which sadly may come as
> > > finger-pointing and thus alienating.
> > >
> > > Hope that makes sense and the team is willing to consider the extra 90
> > > lines worth of code.
> >
> > The argument "libfoo can be broken in some particular configuration, so lets use dlopen() to make errors happen later" seems like it applies to every library.  Why is this case so special?  Who are the users running into this specific problem and who are stuck with broken versions they can't update?
> >
> It's a long story, hope I don't bore you to death :-P
>
> Even though I've been itching to hack on ffmpeg for a while, the bug
> that allowed me to do that is
> https://github.com/ValveSoftware/steam-for-linux/issues/8673
>
> As a background, steam as well as some of the programs/games shipped
> use libraries provided by ffmpeg. In addition, steam ships with a
> steam runtime, which is effectively a partial chroot of an old Ubuntu.
> For various compatibility reasons, one cannot simply update it, so the
> startup scripting will try and promote a set of the host libraries (if
> newer) so that they're used instead of the bundled Ubuntu ones.
>
> What happens in the libva case is that distributions can provide only
> libva.so and omit libva-x11.so. Which due to the internal ABI break
> (removal of the va_fool API), means that steam and likely some games
> will simply crash out.
>
> Now let me try and draw an analogy to another set of libraries which
> also share internal ABI - libdrm.so, libdrm_nouveau.so,
> libdrm_amdgpu.so libdrm_intel.so, etc. To the best of my knowledge
> there was no breakage in there be that internal or public ABI.
>
> In addition, while distribution may allow you to install only some
> (say libdrm.so without libdrm_intel.so), a pair of those is pulled by
> the respective GL and Vulkan drivers. For example: the amdgpu GL
> driver (amdgpu_dri.so) and radv Vulkand driver (libvulkan_radeon.so)
> depend on libdrm_amdgpu.so and libdrm.so. Hence, in practical terms
> users cannot hit a similar issue... unless they very very deliberately
> try to do so.
>
> So while one solution is to go around telling users and distributions
> that they're "doing it wrong", IMHO a more pragmatic solution is to
> include this brief workaround in ffmpeg. At least in the short to mid
> term.
> As mentioned in the cover letter (sorry again for sending the series
> multiple times), I have some plans for a proper long term, which would
> reside in libva. Alas as you have experienced yourself, the libva
> maintainers can be rather busy, so we're looking at least a couple of
> months until a new libva release is out and further X months, until it
> ripples down to end-users.
>

Mark, humble ping?

Can you kindly let me know if the above argument seems reasonable and
more importantly if it even makes sense. I am more than happy to
provide more details and elaborate, if anything is unclear.

Thanks
Emil
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2 0/3] hwcontext_vaapi: dlopen libva-x11 and libva-drm

[Emil Velikov]

Greetings everyone,

As you may know the libva* set of libraries share an internal ABI
between them. In a resent libva commit, the va_fool API was removed.

Thus if one is to mix different versions of libva.so and libva-x11.so
they will get an error, leading to a crash of the whole stack.

The simple solution is to dlopen() the winsys components, like libva-x11
and libva-drm. The changes are pretty minor and allow us to handle this
king of issues.

Comments and suggestions are welcome, but please me gentle it's my first
time hacking on ffmpeg :-P

Thanks
Emil

Aside:
 - Please consider backporting it to the stable branches in due time.
 - I've noticed that we leak state in the error paths, happy to send
   follow-up patches if you'd like those fixed.
 - My TODO includes reducing the massive ABI between libva* and
   backend drivers, to a single extra "registration" API entrypoint.

---

Changes in v2:
 - Add libdl dependency, to address underlinking with older glibc

Emil Velikov (3):
  hwcontext_vaapi: do not link against libva-x11.so
  hwcontext_vaapi: do not link against libva-drm.so
  hwcontext_vaapi: #if guard VAAPI_DRM specifics

 configure                   |  3 +-
 libavutil/hwcontext_vaapi.c | 92 +++++++++++++++++++++++++++++++++++--
 2 files changed, 91 insertions(+), 4 deletions(-)

-- 
2.37.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".