[FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Zhao Zhili]

Now it's time to talk about the libavdevice/sdl issue.

SDL output is broken with ffmpeg multithread refactor. SDL 'muxer' write_header
and write_packet must be run in the same thread. And to make it work portable
and reliable, SDL 'muxer' must be run in main thread. It's a common requirement
for render to be run in main thread.

There are at least two trac tickets for the same issue: #10644 and #10649.

And there are two patches for the issue:
https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230918063728.198377-1-haihao.xiang@intel.com/
https://patchwork.ffmpeg.org/project/ffmpeg/patch/20231101090115.10655-1-angus.chen@intel.com/

There patches might work on Linux, but not portable.

A simple workaround is pipe the output of ffmpeg to ffplay:

./ffmpeg -re -i /e/video/cctv.mp4 -an -f yuv4mpegpipe - |ffplay -

To fix it, another thread can be used to drive transcode rather than main thread.
A main loop should be created on main thread, and prepared to handle any special
tasks like render. It sounds a lot of work. I'm not sure if it worth for a single use case.
However, maybe we can have a libavfilter/vsink_preview after that.

What should we do?

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Nicolas George]

[-- Attachment #1.1: Type: text/plain, Size: 353 bytes --]

Zhao Zhili (12023-12-13):
> Now it's time to talk about the libavdevice/sdl issue.
> 
> SDL output is broken with ffmpeg multithread refactor.

And OpenGL, since it depends on SDL for its stand-alone setup.

But do not expect it to be fixed, they have hated lavd for ever and now
they have all the control.

Regards,

-- 
  Nicolas George

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Zhao Zhili]

> 在 2023年12月13日，上午2:04，Nicolas George <george@nsup.org> 写道：
> 
> Zhao Zhili (12023-12-13):
>> Now it's time to talk about the libavdevice/sdl issue.
>> 
>> SDL output is broken with ffmpeg multithread refactor.
> 
> And OpenGL, since it depends on SDL for its stand-alone setup.
> 
> But do not expect it to be fixed,

I don’t expect anything, I’m planning to work on it (once it’s clear how to do that).

> they have hated lavd for ever and now
> they have all the control.
> 
> Regards,
> 
> --
>  Nicolas George
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> 

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] Mailinglist conduct [was: [RFC] fftools/ffmpeg and libavdevice/sdl issue]

[Ronald S. Bultje]

Hi Nicolas,

On Tue, Dec 12, 2023 at 1:04 PM Nicolas George <george@nsup.org> wrote:

> But do not expect it to be fixed, they have hated lavd for ever and now
> they have all the control.
>

Your message promotes conspiracy theories ("they have all control", "they
have hated lavd for ever", etc.) and suggests that individual members are
part of this conspiracy to destroy FFmpeg (ad hominem attack). This is not
appropriate conduct on any of our project's mailing lists; there are plenty
better ways to bring up your technical concerns. Your CC requests that in
your messages to the FFmpeg developer list, you will not engage in ad
hominem attacks or conspiracy theories. Your CC also suggests trying to
assume good faith even during disagreements.

Unanimously signed,
your CC [Anton, James, Jean-Baptiste, Michael, Ronald, The Illuminati :-)]
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Anton Khirnov]

Quoting Zhao Zhili (2023-12-12 18:27:39)
> Now it's time to talk about the libavdevice/sdl issue.
> 
> SDL output is broken with ffmpeg multithread refactor. SDL 'muxer' write_header
> and write_packet must be run in the same thread. And to make it work portable
> and reliable, SDL 'muxer' must be run in main thread. It's a common requirement
> for render to be run in main thread.
> 
> There are at least two trac tickets for the same issue: #10644 and #10649.
> 
> And there are two patches for the issue:
> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230918063728.198377-1-haihao.xiang@intel.com/
> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20231101090115.10655-1-angus.chen@intel.com/
> 
> There patches might work on Linux, but not portable.
> 
> A simple workaround is pipe the output of ffmpeg to ffplay:
> 
> ./ffmpeg -re -i /e/video/cctv.mp4 -an -f yuv4mpegpipe - |ffplay -
> 
> To fix it, another thread can be used to drive transcode rather than main thread.
> A main loop should be created on main thread, and prepared to handle any special
> tasks like render. It sounds a lot of work. I'm not sure if it worth for a single use case.
> However, maybe we can have a libavfilter/vsink_preview after that.
> 
> What should we do?

Honestly I don't see how this could be done in ffmpeg CLI without
disgusting hacks, but before that the question is: why is there an SDL
"muxer" and why would anyone want to use it in ffmpeg CLI? What actual
use cases does it serve that cannot be better handled otherwise?

-- 
Anton Khirnov
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Zhao Zhili]

> On Dec 13, 2023, at 17:08, Anton Khirnov <anton@khirnov.net> wrote:
> 
> Quoting Zhao Zhili (2023-12-12 18:27:39)
>> Now it's time to talk about the libavdevice/sdl issue.
>> 
>> SDL output is broken with ffmpeg multithread refactor. SDL 'muxer' write_header
>> and write_packet must be run in the same thread. And to make it work portable
>> and reliable, SDL 'muxer' must be run in main thread. It's a common requirement
>> for render to be run in main thread.
>> 
>> There are at least two trac tickets for the same issue: #10644 and #10649.
>> 
>> And there are two patches for the issue:
>> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230918063728.198377-1-haihao.xiang@intel.com/
>> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20231101090115.10655-1-angus.chen@intel.com/
>> 
>> There patches might work on Linux, but not portable.
>> 
>> A simple workaround is pipe the output of ffmpeg to ffplay:
>> 
>> ./ffmpeg -re -i /e/video/cctv.mp4 -an -f yuv4mpegpipe - |ffplay -
>> 
>> To fix it, another thread can be used to drive transcode rather than main thread.
>> A main loop should be created on main thread, and prepared to handle any special
>> tasks like render. It sounds a lot of work. I'm not sure if it worth for a single use case.
>> However, maybe we can have a libavfilter/vsink_preview after that.
>> 
>> What should we do?
> 
> Honestly I don't see how this could be done in ffmpeg CLI without
> disgusting hacks, but before that the question is: why is there an SDL
> "muxer" and why would anyone want to use it in ffmpeg CLI? What actual
> use cases does it serve that cannot be better handled otherwise?

I pasted wrong tickets, they are #10625 and #10649.
https://trac.ffmpeg.org/ticket/10625

The use case is realtime preview. The function lavd/sdl2 provides is limited. A vsink_preview
filter is more appropriate. It has the same thread issue with libavfilter.

> 
> -- 
> Anton Khirnov
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Anton Khirnov]

Quoting Zhao Zhili (2023-12-13 10:31:38)
> 
> > On Dec 13, 2023, at 17:08, Anton Khirnov <anton@khirnov.net> wrote:
> > 
> > Quoting Zhao Zhili (2023-12-12 18:27:39)
> >> Now it's time to talk about the libavdevice/sdl issue.
> >> 
> >> SDL output is broken with ffmpeg multithread refactor. SDL 'muxer' write_header
> >> and write_packet must be run in the same thread. And to make it work portable
> >> and reliable, SDL 'muxer' must be run in main thread. It's a common requirement
> >> for render to be run in main thread.
> >> 
> >> There are at least two trac tickets for the same issue: #10644 and #10649.
> >> 
> >> And there are two patches for the issue:
> >> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230918063728.198377-1-haihao.xiang@intel.com/
> >> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20231101090115.10655-1-angus.chen@intel.com/
> >> 
> >> There patches might work on Linux, but not portable.
> >> 
> >> A simple workaround is pipe the output of ffmpeg to ffplay:
> >> 
> >> ./ffmpeg -re -i /e/video/cctv.mp4 -an -f yuv4mpegpipe - |ffplay -
> >> 
> >> To fix it, another thread can be used to drive transcode rather than main thread.
> >> A main loop should be created on main thread, and prepared to handle any special
> >> tasks like render. It sounds a lot of work. I'm not sure if it worth for a single use case.
> >> However, maybe we can have a libavfilter/vsink_preview after that.
> >> 
> >> What should we do?
> > 
> > Honestly I don't see how this could be done in ffmpeg CLI without
> > disgusting hacks, but before that the question is: why is there an SDL
> > "muxer" and why would anyone want to use it in ffmpeg CLI? What actual
> > use cases does it serve that cannot be better handled otherwise?
> 
> I pasted wrong tickets, they are #10625 and #10649.
> https://trac.ffmpeg.org/ticket/10625
> 
> The use case is realtime preview. The function lavd/sdl2 provides is limited. A vsink_preview
> filter is more appropriate. It has the same thread issue with libavfilter.

The submitter claims that piping to ffplay suffers from latency, which
should not be there and so is either a bug or an improper setup.

-- 
Anton Khirnov
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Zhao Zhili]

> On Dec 13, 2023, at 18:06, Anton Khirnov <anton@khirnov.net> wrote:
> 
> Quoting Zhao Zhili (2023-12-13 10:31:38)
>> 
>>> On Dec 13, 2023, at 17:08, Anton Khirnov <anton@khirnov.net> wrote:
>>> 
>>> Quoting Zhao Zhili (2023-12-12 18:27:39)
>>>> Now it's time to talk about the libavdevice/sdl issue.
>>>> 
>>>> SDL output is broken with ffmpeg multithread refactor. SDL 'muxer' write_header
>>>> and write_packet must be run in the same thread. And to make it work portable
>>>> and reliable, SDL 'muxer' must be run in main thread. It's a common requirement
>>>> for render to be run in main thread.
>>>> 
>>>> There are at least two trac tickets for the same issue: #10644 and #10649.
>>>> 
>>>> And there are two patches for the issue:
>>>> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230918063728.198377-1-haihao.xiang@intel.com/
>>>> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20231101090115.10655-1-angus.chen@intel.com/
>>>> 
>>>> There patches might work on Linux, but not portable.
>>>> 
>>>> A simple workaround is pipe the output of ffmpeg to ffplay:
>>>> 
>>>> ./ffmpeg -re -i /e/video/cctv.mp4 -an -f yuv4mpegpipe - |ffplay -
>>>> 
>>>> To fix it, another thread can be used to drive transcode rather than main thread.
>>>> A main loop should be created on main thread, and prepared to handle any special
>>>> tasks like render. It sounds a lot of work. I'm not sure if it worth for a single use case.
>>>> However, maybe we can have a libavfilter/vsink_preview after that.
>>>> 
>>>> What should we do?
>>> 
>>> Honestly I don't see how this could be done in ffmpeg CLI without
>>> disgusting hacks, but before that the question is: why is there an SDL
>>> "muxer" and why would anyone want to use it in ffmpeg CLI? What actual
>>> use cases does it serve that cannot be better handled otherwise?
>> 
>> I pasted wrong tickets, they are #10625 and #10649.
>> https://trac.ffmpeg.org/ticket/10625
>> 
>> The use case is realtime preview. The function lavd/sdl2 provides is limited. A vsink_preview
>> filter is more appropriate. It has the same thread issue with libavfilter.
> 
> The submitter claims that piping to ffplay suffers from latency, which
> should not be there and so is either a bug or an improper setup.

The latency issue may be real or not, I’m concerned with performance. Colorspace is easy to
be handled in the same process than pipe to ffplay, although it’s missing in current implementation.

> 
> -- 
> Anton Khirnov
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org <mailto:ffmpeg-devel@ffmpeg.org>
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org <mailto:ffmpeg-devel-request@ffmpeg.org> with subject "unsubscribe".

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Nicolas George]

Zhao Zhili (12023-12-13):
> The latency issue may be real or not

The latency is obviously real and obviously unavoidable, since there is
a muxer, a protocol, an OS buffer, a protocol and a demuxer in the chain
that are not necessary with a device. We can try to make it as small as
possible, but there will always be some extra latency.

Plus, ffplay does a lot of things that might interfere with the use,
like keeping the frames in sync with their announced timestamps, like a
-re option that cannot be disabled.

Also, the long-term plan for people who work on lavd has always been to
try to replace SDL in ffplay by lavd. It has not progressed very fast,
but it is a goal. This is a step in the wrong direction.

-- 
  Nicolas George
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Anton Khirnov]

Quoting Zhao Zhili (2023-12-13 11:37:52)
> 
> 
> > On Dec 13, 2023, at 18:06, Anton Khirnov <anton@khirnov.net> wrote:
> > 
> > Quoting Zhao Zhili (2023-12-13 10:31:38)
> >> 
> >>> On Dec 13, 2023, at 17:08, Anton Khirnov <anton@khirnov.net> wrote:
> >>> 
> >>> Quoting Zhao Zhili (2023-12-12 18:27:39)
> >>>> Now it's time to talk about the libavdevice/sdl issue.
> >>>> 
> >>>> SDL output is broken with ffmpeg multithread refactor. SDL 'muxer' write_header
> >>>> and write_packet must be run in the same thread. And to make it work portable
> >>>> and reliable, SDL 'muxer' must be run in main thread. It's a common requirement
> >>>> for render to be run in main thread.
> >>>> 
> >>>> There are at least two trac tickets for the same issue: #10644 and #10649.
> >>>> 
> >>>> And there are two patches for the issue:
> >>>> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230918063728.198377-1-haihao.xiang@intel.com/
> >>>> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20231101090115.10655-1-angus.chen@intel.com/
> >>>> 
> >>>> There patches might work on Linux, but not portable.
> >>>> 
> >>>> A simple workaround is pipe the output of ffmpeg to ffplay:
> >>>> 
> >>>> ./ffmpeg -re -i /e/video/cctv.mp4 -an -f yuv4mpegpipe - |ffplay -
> >>>> 
> >>>> To fix it, another thread can be used to drive transcode rather than main thread.
> >>>> A main loop should be created on main thread, and prepared to handle any special
> >>>> tasks like render. It sounds a lot of work. I'm not sure if it worth for a single use case.
> >>>> However, maybe we can have a libavfilter/vsink_preview after that.
> >>>> 
> >>>> What should we do?
> >>> 
> >>> Honestly I don't see how this could be done in ffmpeg CLI without
> >>> disgusting hacks, but before that the question is: why is there an SDL
> >>> "muxer" and why would anyone want to use it in ffmpeg CLI? What actual
> >>> use cases does it serve that cannot be better handled otherwise?
> >> 
> >> I pasted wrong tickets, they are #10625 and #10649.
> >> https://trac.ffmpeg.org/ticket/10625
> >> 
> >> The use case is realtime preview. The function lavd/sdl2 provides is limited. A vsink_preview
> >> filter is more appropriate. It has the same thread issue with libavfilter.
> > 
> > The submitter claims that piping to ffplay suffers from latency, which
> > should not be there and so is either a bug or an improper setup.
> 
> The latency issue may be real or not, I’m concerned with performance. Colorspace is easy to
> be handled in the same process than pipe to ffplay, although it’s missing in current implementation.

I would think piping yuv4mpeg to ffplay should have very little overhead
compared to actual encoding.

IMO ffmpeg is just not a GUI program and trying to use it as one will
just unavoidably lead to tears. It's not an email client either btw.

-- 
Anton Khirnov
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Nicolas George]

Anton Khirnov (12023-12-13):
> Honestly I don't see how this could be done in ffmpeg CLI without
> disgusting hacks,

As I said: do not expect it to be fixed.

Or we could revert the whole half-baked series.

>		    but before that the question is: why is there an SDL
> "muxer" and why would anyone want to use it in ffmpeg CLI? What actual
> use cases does it serve that cannot be better handled otherwise?

No, not “before that”. You have no authority to demand an answer to
that question before helping fixing the feature you broke.

That feature was added. That means its usefulness was obvious to the
person who spent efforts implementing it, and to the persons who
discussed it at the time and helped it happen. Its usefulness is also
useful to the person who is trying to fix it right now. The fact that
the usefulness is not evident to YOU is a failure of your imagination,
not a characteristic of the feature.

If you are curious, you can ask, POLITELY, that people tell you what
they use it for.

But despite what your behavior seems to imply, you are not FFmpeg's
dictator, you cannot demand people convince YOU to keep a feature or
decide to remove or break it.

-- 
  Nicolas George
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Stefano Sabatini]

On date Wednesday 2023-12-13 10:08:45 +0100, Anton Khirnov wrote:
> Quoting Zhao Zhili (2023-12-12 18:27:39)
[...]
> Honestly I don't see how this could be done in ffmpeg CLI without
> disgusting hacks, but before that the question is:

> why is there an SDL
> "muxer" and why would anyone want to use it in ffmpeg CLI? What actual
> use cases does it serve that cannot be better handled otherwise?

As the author of the first sdl.c muxer, maybe I can answer to this
question. It was done partly as an exercise and for fun, but also
because this was useful extremely handy for testing (e.g. to display
the output of a filterchain from ffmpeg, or to display a streamed
video in realtime).

The final goal was (still is) to unify all the tools as very thin
instances of the library. Even if this might be not practical, I think
it is a good final design plan (e.g. ffprobe might be turned to a
custom muxer, ffplay would be the realtime output of a filtergraph
connected to a rendering device, ffmpeg would be a data filtergraph
processor, and you can mix rendering and encoding if you add a movie
sink to the game).
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Anton Khirnov]

Quoting Stefano Sabatini (2023-12-14 01:47:44)
> On date Wednesday 2023-12-13 10:08:45 +0100, Anton Khirnov wrote:
> > Quoting Zhao Zhili (2023-12-12 18:27:39)
> [...]
> > Honestly I don't see how this could be done in ffmpeg CLI without
> > disgusting hacks, but before that the question is:
> 
> > why is there an SDL
> > "muxer" and why would anyone want to use it in ffmpeg CLI? What actual
> > use cases does it serve that cannot be better handled otherwise?
> 
> As the author of the first sdl.c muxer, maybe I can answer to this
> question. It was done partly as an exercise and for fun, but also
> because this was useful extremely handy for testing (e.g. to display
> the output of a filterchain from ffmpeg, or to display a streamed
> video in realtime).

As mentioned elsewhere in the thread, you can just as well pipe raw
video in yuv4mpeg or nut to the video player of your choice and thus
avoid all these hacks.

> The final goal was (still is) to unify all the tools as very thin
> instances of the library. Even if this might be not practical, I think
> it is a good final design plan (e.g. ffprobe might be turned to a
> custom muxer, ffplay would be the realtime output of a filtergraph
> connected to a rendering device, ffmpeg would be a data filtergraph
> processor, and you can mix rendering and encoding if you add a movie
> sink to the game).

I have to strongly disagree. This is neither practically workable,
nor a good goal to aim at. This mindset IMO inevitably leads to (among
other problems):
* endless scope creep
* bloated, inefficient, and buggy libraries, trying (and failing) to
  support every use case under the sun
* myopic API design aimed at fulfilling the needs of precisely one
  caller; this is a problem e.g avfilter badly suffers from, and to a
  lesser extent avformat

-- 
Anton Khirnov
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Nicolas George]

Anton Khirnov (12023-12-14):
> As mentioned elsewhere in the thread, you can just as well pipe raw
> video in yuv4mpeg or nut to the video player of your choice and thus

As was already mentioned elsewhere in the thread, this solution has a
lot more drawbacks.

> avoid all these hacks.

Please refrain from calling other people's work hacks.

> I have to strongly disagree. This is neither practically workable,
> nor a good goal to aim at.

And I strongly agree with Stefano. Having the tools just thin wrappers
around the libraries is the only way to ensure the libraries are
maximally useful for other applications. Otherwise, useful code will
only reside in the tools and be only available through a clumsy
command-line interface.

>			     This mindset IMO inevitably leads to (among
> other problems):
> * endless scope creep
> * bloated, inefficient, and buggy libraries, trying (and failing) to
>   support every use case under the sun
> * myopic API design aimed at fulfilling the needs of precisely one
>   caller; this is a problem e.g avfilter badly suffers from, and to a
>   lesser extent avformat

I see no argument supporting this opinion of yours. Quite the opposite,
over the years, Stefano and others, including me, have made some headway
in this direction without hitting these pitfalls.

-- 
  Nicolas George
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Stefano Sabatini]

On date Thursday 2023-12-14 10:35:56 +0100, Nicolas George wrote:
> Anton Khirnov (12023-12-14):
[...]
> > I have to strongly disagree. This is neither practically workable,
> > nor a good goal to aim at.
> 
> And I strongly agree with Stefano. Having the tools just thin wrappers
> around the libraries is the only way to ensure the libraries are
> maximally useful for other applications. Otherwise, useful code will
> only reside in the tools and be only available through a clumsy
> command-line interface.
> 
> >			     This mindset IMO inevitably leads to (among
> > other problems):

> > * endless scope creep

Scope creep is a general tendency in software, as it tends to grow
with more functionality and use cases in mind (FFmpeg itself started
as an MPEG decoder). OTOH there is good and bad scope creep, it is bad
if the functionality goes beyond the original design and core use
case, or if the extension is not carefully designed and suffers from
assumptions which limit how the software can be used. For example,
making constraints about where the main thread can be executed.

(Unrelated note: I greatly appreciate Anton's threaded architecture
endeavor, and I'm fine with the idea that something can result broken
as a consequence of a major redesign, but I also think we should fix
what can be fixed rather than just dismiss that as "not useful". Also
this is a community project, so it's not only Anton's responsibility
to fix that, since we all benefit from his work).

> > * bloated, inefficient, and buggy libraries, trying (and failing) to
> >   support every use case under the sun

> > * myopic API design aimed at fulfilling the needs of precisely one
> >   caller; this is a problem e.g avfilter badly suffers from, and to a
> >   lesser extent avformat

Note that these two statements conflicting. If you try to support most
of the use cases, it will be flexible by definition. For example, if
we design the API to be only usable from ffmpeg.c, it will be limited
in scope and usefullness.

Desigining something which is generic, flexible, and efficient at the
same time is hard, but that does not mean that it cannot be done.
 
> I see no argument supporting this opinion of yours. Quite the opposite,
> over the years, Stefano and others, including me, have made some headway
> in this direction without hitting these pitfalls.

Same here, these sound like vague statements, and I fail to see how
they are related to that very generic design vision.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Anton Khirnov]

Quoting Stefano Sabatini (2023-12-16 16:18:07)
> On date Thursday 2023-12-14 10:35:56 +0100, Nicolas George wrote:
> > Anton Khirnov (12023-12-14):
> [...]
> > > I have to strongly disagree. This is neither practically workable,
> > > nor a good goal to aim at.
> > 
> > And I strongly agree with Stefano. Having the tools just thin wrappers
> > around the libraries is the only way to ensure the libraries are
> > maximally useful for other applications. Otherwise, useful code will
> > only reside in the tools and be only available through a clumsy
> > command-line interface.
> > 
> > >			     This mindset IMO inevitably leads to (among
> > > other problems):
> 
> > > * endless scope creep
> 
> Scope creep is a general tendency in software, as it tends to grow
> with more functionality and use cases in mind (FFmpeg itself started
> as an MPEG decoder). OTOH there is good and bad scope creep, it is bad
> if the functionality goes beyond the original design and core use
> case, or if the extension is not carefully designed and suffers from
> assumptions which limit how the software can be used. For example,
> making constraints about where the main thread can be executed.
> 
> (Unrelated note: I greatly appreciate Anton's threaded architecture
> endeavor, and I'm fine with the idea that something can result broken
> as a consequence of a major redesign, but I also think we should fix
> what can be fixed rather than just dismiss that as "not useful".

The entire question here is whether SDL muxing is useful enough to
warrant massive hacks in ffmpeg CLI.

> > > * bloated, inefficient, and buggy libraries, trying (and failing) to
> > >   support every use case under the sun
> 
> > > * myopic API design aimed at fulfilling the needs of precisely one
> > >   caller; this is a problem e.g avfilter badly suffers from, and to a
> > >   lesser extent avformat
> 
> Note that these two statements conflicting. If you try to support most
> of the use cases, it will be flexible by definition. For example, if
> we design the API to be only usable from ffmpeg.c, it will be limited
> in scope and usefullness.

There is a subtle but important difference between
* an interface that goes out of its way to explicitly support a large
  number of specific usecases
* an interface that is generic and flexible enough to be applicable to a
  wide range of cases

The crucial distinction is that the first case is about your code doing
MORE, while the second is about doing LESS.

-- 
Anton Khirnov
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 3019 bytes --]

On Mon, Dec 18, 2023 at 06:33:45PM +0100, Anton Khirnov wrote:
> Quoting Stefano Sabatini (2023-12-16 16:18:07)
> > On date Thursday 2023-12-14 10:35:56 +0100, Nicolas George wrote:
> > > Anton Khirnov (12023-12-14):
> > [...]
> > > > I have to strongly disagree. This is neither practically workable,
> > > > nor a good goal to aim at.
> > > 
> > > And I strongly agree with Stefano. Having the tools just thin wrappers
> > > around the libraries is the only way to ensure the libraries are
> > > maximally useful for other applications. Otherwise, useful code will
> > > only reside in the tools and be only available through a clumsy
> > > command-line interface.
> > > 
> > > >			     This mindset IMO inevitably leads to (among
> > > > other problems):
> > 
> > > > * endless scope creep
> > 
> > Scope creep is a general tendency in software, as it tends to grow
> > with more functionality and use cases in mind (FFmpeg itself started
> > as an MPEG decoder). OTOH there is good and bad scope creep, it is bad
> > if the functionality goes beyond the original design and core use
> > case, or if the extension is not carefully designed and suffers from
> > assumptions which limit how the software can be used. For example,
> > making constraints about where the main thread can be executed.
> > 
> > (Unrelated note: I greatly appreciate Anton's threaded architecture
> > endeavor, and I'm fine with the idea that something can result broken
> > as a consequence of a major redesign, but I also think we should fix
> > what can be fixed rather than just dismiss that as "not useful".
> 
> The entire question here is whether SDL muxing is useful enough to
> warrant massive hacks in ffmpeg CLI.

I think the first question is, does this actually need
"massive hacks in ffmpeg CLI" ?

If you ignore the recommandition that SDL should be run from the main
thread then iam not sure what change would be needed in ffmpeg CLI at all.

If you do want to run it in the main thread, well theres the option
for the muxer to launch a seperate process by some way internally.
then it has its own main thread (not great but its a clean solution)

teh 2nd question is, is SDL the only thing requireing "main thread" or
some "single thread" or other limitation ?
does any other decoder, encoder, muxer, demuxer, filter ... use an
external lib thats not fully thread safe ? or has funny limitations ?

The last option would maybe be to run some sort of AVExecutor on the
main thread and allow things like muxers to que operations on it.
The  muxers would totally unchanged be running on a random thread
but que some operation on the main thread if an external lib, driver or
other needs it.
Naively that feels relatively clean to me

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

If a bugfix only changes things apparently unrelated to the bug with no
further explanation, that is a good sign that the bugfix is wrong.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Nicolas George]

[-- Attachment #1.1: Type: text/plain, Size: 920 bytes --]

Michael Niedermayer (12023-12-18):
> I think the first question is, does this actually need
> "massive hacks in ffmpeg CLI" ?

Thank you for bringing some sanity.

> If you ignore the recommandition that SDL should be run from the main
> thread then iam not sure what change would be needed in ffmpeg CLI at all.

I must say, I am very dubious about the statement. It is probably an
over-simplification for “in the same thread” with caveats.

> The last option would maybe be to run some sort of AVExecutor on the
> main thread and allow things like muxers to que operations on it.
> The  muxers would totally unchanged be running on a random thread
> but que some operation on the main thread if an external lib, driver or
> other needs it.
> Naively that feels relatively clean to me

Of course. Having that kind of mechanism is obviously necessary anyway.

Regards,

-- 
  Nicolas George

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Rémi Denis-Courmont]

Le 18 décembre 2023 21:58:39 GMT+02:00, Michael Niedermayer <michael@niedermayer.cc> a écrit :
>On Mon, Dec 18, 2023 at 06:33:45PM +0100, Anton Khirnov wrote:
>> Quoting Stefano Sabatini (2023-12-16 16:18:07)
>> > On date Thursday 2023-12-14 10:35:56 +0100, Nicolas George wrote:
>> > > Anton Khirnov (12023-12-14):
>> > [...]
>> > > > I have to strongly disagree. This is neither practically workable,
>> > > > nor a good goal to aim at.
>> > > 
>> > > And I strongly agree with Stefano. Having the tools just thin wrappers
>> > > around the libraries is the only way to ensure the libraries are
>> > > maximally useful for other applications. Otherwise, useful code will
>> > > only reside in the tools and be only available through a clumsy
>> > > command-line interface.
>> > > 
>> > > >			     This mindset IMO inevitably leads to (among
>> > > > other problems):
>> > 
>> > > > * endless scope creep
>> > 
>> > Scope creep is a general tendency in software, as it tends to grow
>> > with more functionality and use cases in mind (FFmpeg itself started
>> > as an MPEG decoder). OTOH there is good and bad scope creep, it is bad
>> > if the functionality goes beyond the original design and core use
>> > case, or if the extension is not carefully designed and suffers from
>> > assumptions which limit how the software can be used. For example,
>> > making constraints about where the main thread can be executed.
>> > 
>> > (Unrelated note: I greatly appreciate Anton's threaded architecture
>> > endeavor, and I'm fine with the idea that something can result broken
>> > as a consequence of a major redesign, but I also think we should fix
>> > what can be fixed rather than just dismiss that as "not useful".
>> 
>> The entire question here is whether SDL muxing is useful enough to
>> warrant massive hacks in ffmpeg CLI.
>
>I think the first question is, does this actually need
>"massive hacks in ffmpeg CLI" ?

>If you ignore the recommandition that SDL should be run from the main
>thread then iam not sure what change would be needed in ffmpeg CLI at all.

As others noted earlier, that won't work for Mac and Windows.

>If you do want to run it in the main thread, well theres the option
>for the muxer to launch a seperate process by some way internally.

Starting a process from a library is not very practical. You need to locate the executable and the way to do that is different if you're working with a proper installation, or testing in the development tree.

I reckon that testing is a big motivation for the SDL code, so this can't be simply ignored.

And then you need an IPC, which is not portable, and not very different from the piping alternative proposal up-thread.

>then it has its own main thread (not great but its a clean solution)
>
>teh 2nd question is, is SDL the only thing requireing "main thread" or
>some "single thread" or other limitation ?

On Windows, it requires its own thread to use a single-threaded COM apartment. On Mac, it must run on the main thread to access GUI functionality. Main thread here means the initial thread in the address space.

>does any other decoder, encoder, muxer, demuxer, filter ... use an
>external lib thats not fully thread safe ? or has funny limitations ?

>The last option would maybe be to run some sort of AVExecutor on the
>main thread and allow things like muxers to que operations on it.
>The  muxers would totally unchanged be running on a random thread
>but que some operation on the main thread if an external lib, driver or
>other needs it.

To me, that counts as a horrible hack for a library to have, TBH. And if nothing other than SDL on Mac would even need this, then it's very much an ad-hoc kludge.

>Naively that feels relatively clean to me
>
>thx
>
>[...]
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Nicolas George]

Rémi Denis-Courmont (12023-12-19):
> As others noted earlier, that won't work for Mac and Windows.

If it works on Linux and other real Unixes, it is a lot better than if
it does not work on any platform. And we should still blame whoever
broke it rather than whoever is trying to fix it.

> Starting a process from a library is not very practical. You need to
> locate the executable and the way to do that is different if you're
> working with a proper installation, or testing in the development
> tree.

You are confusing starting a process and executing a new executable.

> And then you need an IPC, which is not portable, and not very
> different from the piping alternative proposal up-thread.

It is a lot different in the fact that it is automatic.

> To me, that counts as a horrible hack for a library to have, TBH.

What? Having an API tu run functions in the main thread is a basic
feature for any kind of threading architecture. See g_idle_add() for
example: “the callback will be invoked in whichever thread is running
that main context”.

-- 
  Nicolas George
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Rémi Denis-Courmont]

Le 19 décembre 2023 11:29:04 GMT+02:00, Nicolas George <george@nsup.org> a écrit :
>Rémi Denis-Courmont (12023-12-19):
>> As others noted earlier, that won't work for Mac and Windows.
>
>If it works on Linux and other real Unixes, it is a lot better than if
>it does not work on any platform. And we should still blame whoever
>broke it rather than whoever is trying to fix it.

Anton's objections are against the horrible hacks necessary to support Mac and Windows, as far as I understand him.

Of course it's also objectionable for SDL to be modelled as a muxer, when it's ostensibly an audio output device and a video output device - not a multiplexer. (SPU blending, lip sync and whatever may require ESs to be processed together should not be tied to SDL.)

>> Starting a process from a library is not very practical. You need to
>> locate the executable and the way to do that is different if you're
>> working with a proper installation, or testing in the development
>> tree.
>
>You are confusing starting a process and executing a new executable.

Running on the main thread (the initial thread of an address space) requires an external executable, so if somebody is confusing the two, that's not me.

Besides, starting a new process without execution of an executable, in other words, forking without executing, is essentially impossible in a multithreaded Unix-like environment, since FFmpeg is not async-fork-safe. It is also completely impossible on Windows. So the distinction is completely helpless here.

>What? Having an API tu run functions in the main thread is a basic
>feature for any kind of threading architecture.

Oh really? And the POSIX thread function to run on the main thread is what exactly?

You're conflating main-loops and threads here. Thread-safe libraries don't normally depend on a main loop, even less the ability to run idle callbacks on it.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Nicolas George]

Rémi Denis-Courmont (12023-12-19):
> Anton's objections are against the horrible hacks necessary to support
> Mac and Windows, as far as I understand him.

I have not read that. If that is true, maybe he could start with
refraining from using expressions like “horrible hacks”.

> Of course it's also objectionable for SDL to be modelled as a muxer,

Sigh. Do we have to explain this once again? Devices have to present as
muxers and demuxers in order to be usable transparently by applications
designed for plain files. And anyway, the manner frames enter or leave a
device is orthogonal to the implementation of said device, so bringing
this question in the discussion is irrelevant.

> Running on the main thread (the initial thread of an address space)
> requires an external executable

No. Or [citation needed].

> Besides, starting a new process without execution of an executable, in
> other words, forking without executing, is essentially impossible in a
> multithreaded Unix-like environment,

It is less than standards-compliant and portable, but it is doable.

> since FFmpeg is not async-fork-safe.

This is something that should be fixed, do you not think?

> Oh really? And the POSIX thread function to run on the main thread is
> what exactly?
> 
> You're conflating main-loops and threads here.

I called them “threading infrastructure”, you called them “main-loops”.
Can we stop nitpicking on words and make an honest attempt at
understanding what the other is saying or is it too much to ask?

-- 
  Nicolas George
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Rémi Denis-Courmont]

Le 19 décembre 2023 14:51:21 GMT+02:00, Nicolas George <george@nsup.org> a écrit :
>Rémi Denis-Courmont (12023-12-19):
>> Anton's objections are against the horrible hacks necessary to support
>> Mac and Windows, as far as I understand him.
>
>I have not read that. If that is true, maybe he could start with
>refraining from using expressions like “horrible hacks”.
>
>> Of course it's also objectionable for SDL to be modelled as a muxer,
>
>Sigh. Do we have to explain this once again? Devices have to present as
>muxers and demuxers in order to be usable transparently by applications
>designed for plain files. And anyway, the manner frames enter or leave a
>device is orthogonal to the implementation of said device, so bringing
>this question in the discussion is irrelevant.

That's a horrible hack of the kind that makes one infer that whoever wrote the library doesn't understand API design.

>> Running on the main thread (the initial thread of an address space)
>> requires an external executable
>
>No. Or [citation needed].

I don't care if you disagree with the definition of "main thread" in the context of SDL.

>> Besides, starting a new process without execution of an executable, in
>> other words, forking without executing, is essentially impossible in a
>> multithreaded Unix-like environment,
>
>It is less than standards-compliant and portable, but it is doable.

You could certainly engineer a custom OS that would allow this, but I don't think that's really relevant, whilst the issue at stake is support for Apple's OS. Hence "essentially impossible" as opposed to "impossible".

>
>> since FFmpeg is not async-fork-safe.
>
>This is something that should be fixed, do you not think?

First, good luck with that. Making FFmpeg work under POSIX fork-safe constraints is simply not realistic, not to mention the underlying libraries that FFmpeg would have to fork (pun unintended). If it were feasible, we wouldn't need to have this argument: somebody could just fix the SDL muxer internals without messing with the FFmpeg APIs.

Second, even if you did succeed at this,  the result would be unmaintainable, as you'd have to mind those constraints for all future code changes.

And third, you would leak memory and resources of other threads that just happened to be allocated to the parent process at the time of fork. This is highly undesirable.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 1619 bytes --]

On Tue, Dec 19, 2023 at 09:23:29AM +0200, Rémi Denis-Courmont wrote:
> 
> 
> Le 18 décembre 2023 21:58:39 GMT+02:00, Michael Niedermayer <michael@niedermayer.cc> a écrit :
> >On Mon, Dec 18, 2023 at 06:33:45PM +0100, Anton Khirnov wrote:
> >> Quoting Stefano Sabatini (2023-12-16 16:18:07)
> >> > On date Thursday 2023-12-14 10:35:56 +0100, Nicolas George wrote:
> >> > > Anton Khirnov (12023-12-14):
> >> > [...]
[...]
> >does any other decoder, encoder, muxer, demuxer, filter ... use an
> >external lib thats not fully thread safe ? or has funny limitations ?
> 
> >The last option would maybe be to run some sort of AVExecutor on the
> >main thread and allow things like muxers to que operations on it.
> >The  muxers would totally unchanged be running on a random thread
> >but que some operation on the main thread if an external lib, driver or
> >other needs it.
> 
> To me, that counts as a horrible hack for a library to have, TBH. And if nothing other than SDL on Mac would even need this, then it's very much an ad-hoc kludge.

lets take a step back here

The problem is not SDL, the problem is any GUI on the MAC am i correct ?

so the idea is that we cannot access any GUI in any code from anything in
libavformat and probably all other libs, ever

no debug with graphical output
no vissualizuation of anything
no devices
no libs that expose anything that would need a GUI for configuration

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Everything should be made as simple as possible, but not simpler.
-- Albert Einstein

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Rémi Denis-Courmont]

Le tiistaina 19. joulukuuta 2023, 18.58.40 EET Michael Niedermayer a écrit :
> so the idea is that we cannot access any GUI in any code from anything in
> libavformat and probably all other libs, ever

No. The idea is that a command line program cannot use the GUI, and a library 
can only use the GUI if the main program is a GUI program.

> no debug with graphical output
> no vissualizuation of anything
> no devices
> no libs that expose anything that would need a GUI for configuration

You can do all those things as long as you assume that the main program is a  
GUI program running the Mac-specific UI main loop. AFAICT, you can write a 
macOS OpenGL or Metal video output device and a CoreAudio audio output device, 
and any GUI program that uses FFmpeg can then use those devices.

Trying to fit this into a generic portable command line tool is not going to 
work though. Then SDL adds the extra problem that it probably only works with 
its own API driving the main loop, and not just any API layered on top of Mac 
GUI frameworks. If so, the only proper way to support it (on Mac) is to make a 
dedicated player. Of course, it's also possible to skip support for SDL on 
Mac.


Lastly, it has been made clear by the proponents of the muxer that this is but 
a convenient trick so that a real muxer can be swapped for a renderer wherever 
a muxer is expected. In other words, it is a literal kludge (from the 
wiktionary: "Any construction or practice, typically crude yet effective, 
designed to solve a problem temporarily or expediently"). That's pretty much 
the antithesis of good and sound API design.

-- 
Rémi Denis-Courmont
http://www.remlab.net/



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Nicolas George]

[-- Attachment #1.1: Type: text/plain, Size: 795 bytes --]

Rémi Denis-Courmont (12023-12-19):
> No. The idea is that a command line program cannot use the GUI, and a library 
> can only use the GUI if the main program is a GUI program.

This is braindead design at a level way beyond anything we might have in
FFmpeg.

> Lastly, it has been made clear by the proponents of the muxer that this is but 
> a convenient trick so that a real muxer can be swapped for a renderer wherever 
> a muxer is expected.

And assembly optimizations are but a convenient trick so that programs
run faster.

The “but” in the above sentence is doing heavy lifting to turn a
perfectly true sentence into a completely stupid value judgement.
Fortunately for me, I do not mean the above sentence literally, is is
but a parody.

-- 
  Nicolas George

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 896 bytes --]

On Tue, Dec 19, 2023 at 08:48:39PM +0200, Rémi Denis-Courmont wrote:
> Le tiistaina 19. joulukuuta 2023, 18.58.40 EET Michael Niedermayer a écrit :
> > so the idea is that we cannot access any GUI in any code from anything in
> > libavformat and probably all other libs, ever
> 
> No. The idea is that a command line program cannot use the GUI, and a library 
> can only use the GUI if the main program is a GUI program.

with that, the obvious thought would be to have the user interface of ffmpeg
factored out and then the user can choose between a text based command line
and GUI interface. Thats not so trivial but its a clean way to do it.

This can form the basis of a video player, video editor and so on

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Observe your enemies, for they first find out your faults. -- Antisthenes

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Alexander Strasser via ffmpeg-devel]

On 2023-12-14 01:47 +0100, Stefano Sabatini wrote:
> On date Wednesday 2023-12-13 10:08:45 +0100, Anton Khirnov wrote:
> > Quoting Zhao Zhili (2023-12-12 18:27:39)
> [...]
> > Honestly I don't see how this could be done in ffmpeg CLI without
> > disgusting hacks, but before that the question is:
>
> > why is there an SDL
> > "muxer" and why would anyone want to use it in ffmpeg CLI? What actual
> > use cases does it serve that cannot be better handled otherwise?
>
> As the author of the first sdl.c muxer, maybe I can answer to this
> question. It was done partly as an exercise and for fun, but also
> because this was useful extremely handy for testing (e.g. to display
> the output of a filterchain from ffmpeg, or to display a streamed
> video in realtime).

That is what I used it for countless times as well.

It has a lot of merit in handling compared to piping to a player,
where one needs to run 2 processes in one command line with all
the drawbacks that come with it.


> The final goal was (still is) to unify all the tools as very thin
> instances of the library. Even if this might be not practical, I think
> it is a good final design plan (e.g. ffprobe might be turned to a
> custom muxer, ffplay would be the realtime output of a filtergraph
> connected to a rendering device, ffmpeg would be a data filtergraph
> processor, and you can mix rendering and encoding if you add a movie
> sink to the game).

I agree to this perspective. Making most of the functionality
available through the FFmpeg libs and therefore available for
wider application of a broader audience seems valuable.

Such a big goal that cannot be reached in a predictable time frame
should be challenged over the years and the plans, in so far they
exist, need to be adjusted accordingly. Still that doesn't mean
the goal is not worthwhile or not achievable.


On an ending note: I don't like at all the way this discussion unraveled:
1. Someone wants to fix a reported problem after the new threading changes
2. It is questioned why a feature that exists for over a decade is there

I hope this can get back on a more constructive rail. Otherwise
I fear it just sets precedence for more to come :(


  Alexander
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".