[FFmpeg-devel] RFC: opinions on making fftools work like a library

[FFmpeg-devel] RFC: opinions on making fftools work like a library

[Zhao Zhili]

I'd like to know what do you think about making fftools work like a
library, like what ffmpeg-kit already did, but with built-in support.

https://github.com/arthenica/ffmpeg-kit

   FFmpegKit is a collection of tools to use FFmpeg in Android, iOS,
   Linux, macOS, tvOS, Flutter and React Native applications.

   It includes scripts to build FFmpeg native libraries, a wrapper
   library to run FFmpeg/FFprobe commands in applications and 8
   prebuilt binary packages available at Github, Maven Central,
   CocoaPods, pub and npm.

Pro:
1. It can be used for testing on mobile devices.
2. It can be used directly to do useful work.
3. It can combine some work from projects like ffmpeg-kit to extend the
funtion.

Cons:
1. It makes things complicated.
2. It's hard to support mobile devices.

Welcome to comments. I want to know is it:

a. Absolutely no! It's not fftools supported to do.
b. Doesn't matter/Don't care.
c. I like the idea, (but...)

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] RFC: opinions on making fftools work like a library

[Lynne]

Mar 3, 2023, 11:15 by quinkblack@foxmail.com:

> I'd like to know what do you think about making fftools work like a
> library, like what ffmpeg-kit already did, but with built-in support.
>
> https://github.com/arthenica/ffmpeg-kit
>
>  FFmpegKit is a collection of tools to use FFmpeg in Android, iOS,
>  Linux, macOS, tvOS, Flutter and React Native applications.
>
>  It includes scripts to build FFmpeg native libraries, a wrapper
>  library to run FFmpeg/FFprobe commands in applications and 8
>  prebuilt binary packages available at Github, Maven Central,
>  CocoaPods, pub and npm.
>
> Pro:
> 1. It can be used for testing on mobile devices.
> 2. It can be used directly to do useful work.
> 3. It can combine some work from projects like ffmpeg-kit to extend the
> funtion.
>
> Cons:
> 1. It makes things complicated.
> 2. It's hard to support mobile devices.
>
> Welcome to comments. I want to know is it:
>
> a. Absolutely no! It's not fftools supported to do.
> b. Doesn't matter/Don't care.
> c. I like the idea, (but...)
>

No. A high-level wrapper is a much better idea than formalizing CLI syntax.
And there are plenty of wrappers already.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] RFC: opinions on making fftools work like a library

[Steven Liu]

Zhao Zhili <quinkblack@foxmail.com> 于2023年3月3日周五 18:15写道：
>
> I'd like to know what do you think about making fftools work like a
> library, like what ffmpeg-kit already did, but with built-in support.
>
> https://github.com/arthenica/ffmpeg-kit
>
>    FFmpegKit is a collection of tools to use FFmpeg in Android, iOS,
>    Linux, macOS, tvOS, Flutter and React Native applications.
>
>    It includes scripts to build FFmpeg native libraries, a wrapper
>    library to run FFmpeg/FFprobe commands in applications and 8
>    prebuilt binary packages available at Github, Maven Central,
>    CocoaPods, pub and npm.
>
> Pro:
> 1. It can be used for testing on mobile devices.
> 2. It can be used directly to do useful work.
> 3. It can combine some work from projects like ffmpeg-kit to extend the
> funtion.
>
> Cons:
> 1. It makes things complicated.
> 2. It's hard to support mobile devices.
>
> Welcome to comments. I want to know is it:
>
> a. Absolutely no! It's not fftools supported to do.
> b. Doesn't matter/Don't care.
> c. I like the idea, (but...)

I think there have two problem when testing on mobile devices for me:
1. The ABI of libav* always be modified.
2. There have lots of error when i run fate on mobile devices

1.  just make a portable framework between ffmpeg and downstream
modules, eg. video editor, live streaming publisher.
If ffmpeg modified the ABI, just modify the framework should be ok.
But this should not part of ffmpeg.

2. I don't get lots of time to fix them. And not sure if you have same problem.

>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".


Thanks
Steven
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] RFC: opinions on making fftools work like a library

[Anton Khirnov]

Quoting Zhao Zhili (2023-03-03 19:15:03)
> I'd like to know what do you think about making fftools work like a
> library, like what ffmpeg-kit already did, but with built-in support.

Maybe you should first clarify what exactly do you mean by "work like a
library". Meaning literally turn it into a library with a C API you can
call? That would be a massive amount of work, though my current
refactorings are moving in that direction.
Or provide stronger guarantees for commandline options? Or something
else?

-- 
Anton Khirnov
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] RFC: opinions on making fftools work like a library

[Neal Gompa]

On Fri, Mar 17, 2023 at 2:12 PM Anton Khirnov <anton@khirnov.net> wrote:
>
> Quoting Zhao Zhili (2023-03-03 19:15:03)
> > I'd like to know what do you think about making fftools work like a
> > library, like what ffmpeg-kit already did, but with built-in support.
>
> Maybe you should first clarify what exactly do you mean by "work like a
> library". Meaning literally turn it into a library with a C API you can
> call? That would be a massive amount of work, though my current
> refactorings are moving in that direction.
> Or provide stronger guarantees for commandline options? Or something
> else?
>

I think the idea is to turn fftools (e.g. ffmpeg/ffplay/ffprobe) into
a generic, simple high level stable API that consumers can use. The
raw libav* APIs are difficult for most people to use, which leads to a
ton of wrappers around the fftools instead.

Yes, it would be a lot of work, but it would make using ffmpeg way easier.



-- 
真実はいつも一つ！/ Always, there's only one truth!
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] RFC: opinions on making fftools work like a library

[Rémi Denis-Courmont]

Le lauantaina 18. maaliskuuta 2023, 13.33.36 EET Neal Gompa a écrit :
> > Maybe you should first clarify what exactly do you mean by "work like a
> > library". Meaning literally turn it into a library with a C API you can
> > call? That would be a massive amount of work, though my current
> > refactorings are moving in that direction.
> > Or provide stronger guarantees for commandline options? Or something
> > else?
> 
> I think the idea is to turn fftools (e.g. ffmpeg/ffplay/ffprobe) into
> a generic, simple high level stable API that consumers can use. The
> raw libav* APIs are difficult for most people to use, which leads to a
> ton of wrappers around the fftools instead.

That sounds like a self-contradictory goal, TBH.

There are so many different settings, complex interactions between settings, 
and so much variability depending on versions and build parameters.

> Yes, it would be a lot of work, but it would make using ffmpeg way easier.

Or not. You would have to define an API for literally every command line switch 
that each of the tools exhibits. This would unavoidably result in some kind of 
high-level abstract and rather complex settings API that would be prone to 
silent breaks has the names, values and semantics of settings vary with 
versions and builds.

There may be value in doing that anyway. In particular, it could improve the 
progress and error reporting with callbacks rather than parsing the standard 
output. But it probably wouldn't be in "simplifying" the use of the tools.

-- 
レミ・デニ-クールモン
http://www.remlab.net/



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] RFC: opinions on making fftools work like a library

[Stefano Sabatini]

On date Sunday 2023-03-19 11:01:55 +0200, Rémi Denis-Courmont wrote:
> Le lauantaina 18. maaliskuuta 2023, 13.33.36 EET Neal Gompa a écrit :
> > > Maybe you should first clarify what exactly do you mean by "work like a
> > > library". Meaning literally turn it into a library with a C API you can
> > > call? That would be a massive amount of work, though my current
> > > refactorings are moving in that direction.
> > > Or provide stronger guarantees for commandline options? Or something
> > > else?
> > 
> > I think the idea is to turn fftools (e.g. ffmpeg/ffplay/ffprobe) into
> > a generic, simple high level stable API that consumers can use. The
> > raw libav* APIs are difficult for most people to use, which leads to a
> > ton of wrappers around the fftools instead.
> 
> That sounds like a self-contradictory goal, TBH.
> 
> There are so many different settings, complex interactions between settings, 
> and so much variability depending on versions and build parameters.
> 
> > Yes, it would be a lot of work, but it would make using ffmpeg way easier.
> 
> Or not. You would have to define an API for literally every command line switch 
> that each of the tools exhibits. This would unavoidably result in some kind of 
> high-level abstract and rather complex settings API that would be prone to 
> silent breaks has the names, values and semantics of settings vary with 
> versions and builds.
> 
> There may be value in doing that anyway. In particular, it could improve the 
> progress and error reporting with callbacks rather than parsing the standard 
> output. But it probably wouldn't be in "simplifying" the use of the tools.

One of the ideas at some point was to move the functionality into kind
of high-level constructs such as a libavfilter filtergraph: you would
have basically to define the input and output of the filtergraph and
then feed and read the output from it.

At the moment this has several limitations though: for example there
is all kind of custom logic into ffmpeg.c (e.g. related to timestamp
handling and global synchronization). This in theory could be achieved
by extending the filtering logic to data, so that you have pluggable
filters to handle custom data processing logic (e.g. timestamp
tweaking).

Then you could implement a player by reading from the filtergraph and
sending the output packets to your rendering engine and sending
controls to the filtergraph to stop/resume processing and changing the
filters parameters. ffprobe could be implemented as a data
filter/processor on top of a decoding filtergraph.

This is an ideal design but in practice there are still many things
missing (for example data filtering) and threading/synchronization
with the application logic might be tricky.

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] RFC: opinions on making fftools work like a library

[Anton Khirnov]

Quoting Neal Gompa (2023-03-18 12:33:36)
> generic
> simple
> stable

These goals are to a large extent in conflict.

Making it generic (meaning it covers many use cases) necessarily makes
it complex. ffmpeg CLI is used by huge numbers of users to do gazillions
wildly different things. An interface making it simpler necessarily
takes away some of the power.
Making it stable also limits your ability to change things, which makes
it harder to cover new use cases.

> Yes, it would be a lot of work, but it would make using ffmpeg way easier.

I'm definitely not opposed to making things simpler and more generic,
and a lot of my work is actually aiming at those goals, but I think
"a lot" is a massive understatement for the amount of work required. And
I don't see people lining up to actually do the work or sponsor it.

-- 
Anton Khirnov
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] RFC: opinions on making fftools work like a library

[Zhao Zhili]

> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Anton Khirnov
> Sent: 2023年3月18日 2:12
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] RFC: opinions on making fftools work like a library
> 
> Quoting Zhao Zhili (2023-03-03 19:15:03)
> > I'd like to know what do you think about making fftools work like a
> > library, like what ffmpeg-kit already did, but with built-in support.
> 
> Maybe you should first clarify what exactly do you mean by "work like a
> library". Meaning literally turn it into a library with a C API you can
> call? That would be a massive amount of work, though my current
> refactorings are moving in that direction.
> Or provide stronger guarantees for commandline options? Or something
> else?

I mean user can simply rename main() to something like ffmpeg_main() and build
it as a library, even without directly support in our build system. Sometimes
user just want to copy-paste and run ffmpeg cmdline on mobile.

Some projects already did this, but they need to tweak the source code to
reset global states each time, avoiding exit() and so on. These projects cannot
benefit from your refactor work until they rebase their projects on new version
of fftools.

> 
> --
> Anton Khirnov
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] RFC: opinions on making fftools work like a library

[Anton Khirnov]

Quoting Zhao Zhili (2023-03-19 06:04:55)
> 
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Anton Khirnov
> > Sent: 2023年3月18日 2:12
> > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] RFC: opinions on making fftools work like a library
> > 
> > Quoting Zhao Zhili (2023-03-03 19:15:03)
> > > I'd like to know what do you think about making fftools work like a
> > > library, like what ffmpeg-kit already did, but with built-in support.
> > 
> > Maybe you should first clarify what exactly do you mean by "work like a
> > library". Meaning literally turn it into a library with a C API you can
> > call? That would be a massive amount of work, though my current
> > refactorings are moving in that direction.
> > Or provide stronger guarantees for commandline options? Or something
> > else?
> 
> I mean user can simply rename main() to something like ffmpeg_main() and build
> it as a library, even without directly support in our build system. Sometimes
> user just want to copy-paste and run ffmpeg cmdline on mobile.

I don't really understand where does "mobile" come into this and why
should we bend over backwards to work around crippled platforms.

> Some projects already did this, but they need to tweak the source code to
> reset global states each time, avoiding exit() and so on. These projects cannot
> benefit from your refactor work until they rebase their projects on new version
> of fftools.

While I personally do agree that a well-written program's main() should
be reentrant, this is not exactly a mainstream view. And as I said - the
amount of work to do this properly is immense.
It is technically possible to do this "improperly", meaning just put all
global state into an FFmpegContext that's allocated in main, then pass
it to all functions. But this just papers over the actuall issues, not
to mention that making stability guarantees for such an abomination
would cripple development

-- 
Anton Khirnov
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".