[FFmpeg-devel] filter queue question

[FFmpeg-devel] filter queue question

[Alex]

Hi!
I developing custom GPU filter that require lot of time to process frames and as result overal fps is low ( around 20 fps):

ffmpeg -i 720p.mp4  -filter_complex "format=rgb24,myfilter" -f null -

But then I added actual encoding part to ffmpeg command, result fps is down to 16 fps (-4 fps, around 20%!!!):

ffmpeg -i 720p.mp4  -filter_complex "format=rgb24,myfilter" -c:v h264 -y out.mp4

If I look at timeline of overla process in each cycle:

|----decoding time---| ---> |------------------------------filtering time-------------------------|  ---> |---encoding time---|

So, basically can I process frame in my custom filter without waiting for encoding to finish?
In other word I want to process frames in my custom filter in parallel/queue to encoding process???

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] filter queue question

[Felix LeClair]

From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> on behalf of Alex <3.14pi@ukr.net>
Sent: July 5, 2022 9:00 AM
To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Subject: [FFmpeg-devel] filter queue question

Hi!
I developing custom GPU filter that require lot of time to process frames and as result overal fps is low ( around 20 fps):

ffmpeg -i 720p.mp4  -filter_complex "format=rgb24,myfilter" -f null -

But then I added actual encoding part to ffmpeg command, result fps is down to 16 fps (-4 fps, around 20%!!!):

ffmpeg -i 720p.mp4  -filter_complex "format=rgb24,myfilter" -c:v h264 -y out.mp4

If I look at timeline of overla process in each cycle:

|----decoding time---| ---> |------------------------------filtering time-------------------------|  ---> |---encoding time---|

So, basically can I process frame in my custom filter without waiting for encoding to finish?
In other word I want to process frames in my custom filter in parallel/queue to encoding process???

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".



In concept yes, but you may be better off improving the speed of the underlying filter itself.

Part of the cost of encoding is beyond the encoder itself, you have to account for file-system overhead, disk I/O speed etc.

Depending on your implementation, you may be running into issues with memory copies from system to GPU memory and back, which is quite expensive.

try testing using a "hardware decoder--> your filter-->hardware encoder" chain to keep everything in GPU memory.

Beyond that, standard GPU acceleration rules apply. Make sure your wave fronts/work groups are aligned, check for system utilization, use non-blocking/async calls when possible, etc.

-Felix (FCLC)


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] filter queue question

[Alex]

Thanks I will chek it out!

For now my filter use standart filter_frame() function callback. But how to request next frame from decoder in my filter? 


5 July 2022, 16:19:49, by "Felix LeClair" <felix.leclair123@hotmail.com>:

From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> on behalf of Alex <3.14pi@ukr.net>
Sent: July 5, 2022 9:00 AM
To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Subject: [FFmpeg-devel] filter queue question

Hi!
I developing custom GPU filter that require lot of time to process frames and as result overal fps is low ( around 20 fps):

ffmpeg -i 720p.mp4  -filter_complex "format=rgb24,myfilter" -f null -

But then I added actual encoding part to ffmpeg command, result fps is down to 16 fps (-4 fps, around 20%!!!):

ffmpeg -i 720p.mp4  -filter_complex "format=rgb24,myfilter" -c:v h264 -y out.mp4

If I look at timeline of overla process in each cycle:

|----decoding time---| ---> |------------------------------filtering time-------------------------|  ---> |---encoding time---|

So, basically can I process frame in my custom filter without waiting for encoding to finish?
In other word I want to process frames in my custom filter in parallel/queue to encoding process???

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".



In concept yes, but you may be better off improving the speed of the underlying filter itself.

Part of the cost of encoding is beyond the encoder itself, you have to account for file-system overhead, disk I/O speed etc.

Depending on your implementation, you may be running into issues with memory copies from system to GPU memory and back, which is quite expensive.

try testing using a "hardware decoder--> your filter-->hardware encoder" chain to keep everything in GPU memory.

Beyond that, standard GPU acceleration rules apply. Make sure your wave fronts/work groups are aligned, check for system utilization, use non-blocking/async calls when possible, etc.

-Felix (FCLC)


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".