[FFmpeg-devel] [RFC] d3dva security hw+threads

[FFmpeg-devel] [RFC] d3dva security hw+threads

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 1071 bytes --]

Hi all

Theres a use after free issue in H.264 Decoding on d3d11va with multiple threads
I dont have the hardware/platform nor do i know the hw decoding code so i made
no attempt to fix this beyond asking others to ...

There where 2 patches posted by the maintainer, neither really felt "right" and
also indepandently noone else applied either or approved either.

What is the preferred way of the community to handle this (and similar issues)?
ignore it and hope it will be handled by someone else ?
document it in release notes ?
disable the code ? (and which code)

And yes iam trying to be a bit annoying here, so this moves forward and
a fix or workaround can be in git master soon and then the next releases

Thanks

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The real ebay dictionary, page 1
"Used only once"    - "Some unspecified defect prevented a second use"
"In good condition" - "Can be repaird by experienced expert"
"As is" - "You wouldnt want it even if you were payed for it, if you knew ..."

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] d3dva security hw+threads

[Timo Rothenpieler]

On 02.09.2022 01:32, Michael Niedermayer wrote:
> Hi all
> 
> Theres a use after free issue in H.264 Decoding on d3d11va with multiple threads
> I dont have the hardware/platform nor do i know the hw decoding code so i made
> no attempt to fix this beyond asking others to ...

hwaccel with multiple threads being broken is not exactly a surprise.
So we could just disable that, and always have it be one single thread?
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] d3dva security hw+threads

[Lynne]

Sep 2, 2022, 01:46 by timo@rothenpieler.org:

> On 02.09.2022 01:32, Michael Niedermayer wrote:
>
>> Hi all
>>
>> Theres a use after free issue in H.264 Decoding on d3d11va with multiple threads
>> I dont have the hardware/platform nor do i know the hw decoding code so i made
>> no attempt to fix this beyond asking others to ...
>>
>
> hwaccel with multiple threads being broken is not exactly a surprise.
> So we could just disable that, and always have it be one single thread?
>

Is it an API problem or a lavc problem?
Vulkan does allow for threaded submission, so I'd rather keep the option open.
We could add a new codec cap.

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] d3dva security hw+threads

[Anton Khirnov]

Quoting Timo Rothenpieler (2022-09-02 01:46:59)
> On 02.09.2022 01:32, Michael Niedermayer wrote:
> > Hi all
> > 
> > Theres a use after free issue in H.264 Decoding on d3d11va with multiple threads
> > I dont have the hardware/platform nor do i know the hw decoding code so i made
> > no attempt to fix this beyond asking others to ...
> 
> hwaccel with multiple threads being broken is not exactly a surprise.
> So we could just disable that, and always have it be one single thread?

We are already disabling it in a way - the frame threading code ensures
that threads run one at a time when hwaccel is being used.

-- 
Anton Khirnov
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] d3dva security hw+threads

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> Anton Khirnov
> Sent: Sunday, September 4, 2022 8:58 AM
> To: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [RFC] d3dva security hw+threads
> 
> Quoting Timo Rothenpieler (2022-09-02 01:46:59)
> > On 02.09.2022 01:32, Michael Niedermayer wrote:
> > > Hi all
> > >
> > > Theres a use after free issue in H.264 Decoding on d3d11va with
> multiple threads
> > > I dont have the hardware/platform nor do i know the hw decoding
> code so i made
> > > no attempt to fix this beyond asking others to ...
> >
> > hwaccel with multiple threads being broken is not exactly a
> surprise.
> > So we could just disable that, and always have it be one single
> thread?
> 
> We are already disabling it in a way - the frame threading code
> ensures
> that threads run one at a time when hwaccel is being used.


Is there a described way to repro? I would try whether it still 
happens after removing the lock code in hwcontext_d3d11va.c.
Those locks are not really needed and might prevent release 
of dx11 resources in proper order. It's a guess only but 
easy to try.

softworkz
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] d3dva security hw+threads

[Anton Khirnov]

Quoting Soft Works (2022-09-04 09:43:36)
> 
> 
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> > Anton Khirnov
> > Sent: Sunday, September 4, 2022 8:58 AM
> > To: FFmpeg development discussions and patches <ffmpeg-
> > devel@ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] [RFC] d3dva security hw+threads
> > 
> > Quoting Timo Rothenpieler (2022-09-02 01:46:59)
> > > On 02.09.2022 01:32, Michael Niedermayer wrote:
> > > > Hi all
> > > >
> > > > Theres a use after free issue in H.264 Decoding on d3d11va with
> > multiple threads
> > > > I dont have the hardware/platform nor do i know the hw decoding
> > code so i made
> > > > no attempt to fix this beyond asking others to ...
> > >
> > > hwaccel with multiple threads being broken is not exactly a
> > surprise.
> > > So we could just disable that, and always have it be one single
> > thread?
> > 
> > We are already disabling it in a way - the frame threading code
> > ensures
> > that threads run one at a time when hwaccel is being used.
> 
> 
> Is there a described way to repro? I would try whether it still 
> happens after removing the lock code in hwcontext_d3d11va.c.
> Those locks are not really needed and might prevent release 
> of dx11 resources in proper order. It's a guess only but 
> easy to try.

The problem is not in d3d11 locking code, but in the generic code that
does not have clear enough ownership rules. Steve already tested that my
patch from Friday fixes this.

-- 
Anton Khirnov
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] d3dva security hw+threads

[Lukas Fellechner]

> Gesendet: Freitag, 02. September 2022 um 01:46 Uhr
> Von: "Timo Rothenpieler" <timo@rothenpieler.org>
> An: ffmpeg-devel@ffmpeg.org
> Betreff: Re: [FFmpeg-devel] [RFC] d3dva security hw+threads
> On 02.09.2022 01:32, Michael Niedermayer wrote:
>> Hi all
>>
>> Theres a use after free issue in H.264 Decoding on d3d11va with multiple threads
>> I dont have the hardware/platform nor do i know the hw decoding code so i made
>> no attempt to fix this beyond asking others to ...
>
> hwaccel with multiple threads being broken is not exactly a surprise.
> So we could just disable that, and always have it be one single thread?

I am using FFmpeg as decoder library in a video player, either with sw decoding
or d3d11va. Originally, I had threads set to auto in all cases. While it worked
for some codecs such as H.264, it produced garbage output for other codecs.
I think VP8/VP9 are severly broken with threading+d3d11va. So I had to manually
set threads to 1, if using hwaccel. Only then I had stable output for all codecs.

Using multithreading together with hwaccel really does not make any sense.
All the work is done by the GPU, not by the CPU. And the GPU will parallelize
internally where possible. Adding CPU multithreading will not help at all.

IMHO the best would be to completely disable threading when hwaccel is in use.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] d3dva security hw+threads

[Anton Khirnov]

Quoting Lukas Fellechner (2022-09-05 17:50:26)
> > Gesendet: Freitag, 02. September 2022 um 01:46 Uhr
> > Von: "Timo Rothenpieler" <timo@rothenpieler.org>
> > An: ffmpeg-devel@ffmpeg.org
> > Betreff: Re: [FFmpeg-devel] [RFC] d3dva security hw+threads
> > On 02.09.2022 01:32, Michael Niedermayer wrote:
> >> Hi all
> >>
> >> Theres a use after free issue in H.264 Decoding on d3d11va with multiple threads
> >> I dont have the hardware/platform nor do i know the hw decoding code so i made
> >> no attempt to fix this beyond asking others to ...
> >
> > hwaccel with multiple threads being broken is not exactly a surprise.
> > So we could just disable that, and always have it be one single thread?
> 
> I am using FFmpeg as decoder library in a video player, either with sw decoding
> or d3d11va. Originally, I had threads set to auto in all cases. While it worked
> for some codecs such as H.264, it produced garbage output for other codecs.
> I think VP8/VP9 are severly broken with threading+d3d11va. So I had to manually
> set threads to 1, if using hwaccel. Only then I had stable output for all codecs.

Does this still happen with recent code? Using frame threading with
hwaccel is a supported use case and should work, any breakage is a bug
that should be fixed in libavcodec.

> Using multithreading together with hwaccel really does not make any sense.
> All the work is done by the GPU, not by the CPU. And the GPU will parallelize
> internally where possible. Adding CPU multithreading will not help at all.
> 
> IMHO the best would be to completely disable threading when hwaccel is in use.

The problem is that in general you cannot know beforehand whether
hwaccel will be usable. What's more, since stream parameters can change
dynamically, you can have a stream where some segments are decodable
with hwaccel, but some are not.

In this case you typically want lavc to use hwaccel wherever possible
and frame-threaded software decoding otherwise, switching back and forth
as needed. This is what is actually meant by "hwaccel with multiple
threads" --- actual hwaccel decoding is already serialized in
libavcodec.

-- 
Anton Khirnov
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] d3dva security hw+threads

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> Anton Khirnov
> Sent: Monday, September 5, 2022 7:59 AM
> To: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [RFC] d3dva security hw+threads
> 
> Quoting Soft Works (2022-09-04 09:43:36)
> >
> >
> > > -----Original Message-----
> > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> > > Anton Khirnov
> > > Sent: Sunday, September 4, 2022 8:58 AM
> > > To: FFmpeg development discussions and patches <ffmpeg-
> > > devel@ffmpeg.org>
> > > Subject: Re: [FFmpeg-devel] [RFC] d3dva security hw+threads
> > >
> > > Quoting Timo Rothenpieler (2022-09-02 01:46:59)
> > > > On 02.09.2022 01:32, Michael Niedermayer wrote:
> > > > > Hi all
> > > > >
> > > > > Theres a use after free issue in H.264 Decoding on d3d11va
> with
> > > multiple threads
> > > > > I dont have the hardware/platform nor do i know the hw
> decoding
> > > code so i made
> > > > > no attempt to fix this beyond asking others to ...
> > > >
> > > > hwaccel with multiple threads being broken is not exactly a
> > > surprise.
> > > > So we could just disable that, and always have it be one single
> > > thread?
> > >
> > > We are already disabling it in a way - the frame threading code
> > > ensures
> > > that threads run one at a time when hwaccel is being used.
> >
> >
> > Is there a described way to repro? I would try whether it still
> > happens after removing the lock code in hwcontext_d3d11va.c.
> > Those locks are not really needed and might prevent release
> > of dx11 resources in proper order. It's a guess only but
> > easy to try.
> 
> The problem is not in d3d11 locking code, but in the generic code
> that
> does not have clear enough ownership rules. Steve already tested that
> my
> patch from Friday fixes this.

Oh I see. I was missing the context. The patch makes sense to me.

Thanks,
sw


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".