Re: [FFmpeg-devel] [RFC] FFmpeg Execution Graph Visualization

["softworkz ." <softworkz-at-hotmail.com@ffmpeg.org>]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> Stefano Sabatini
> Sent: Mittwoch, 26. März 2025 23:53
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [RFC] FFmpeg Execution Graph Visualization
> 
> On date Friday 2025-03-21 20:11:29 +0000, Soft Works wrote:
> [...]
> > > What it's not clear to me is how this builds up on top of text
> > > formatters, since they are meant to render a tree structure in a
> > > generic way. From this you can have a description of a graph, but
> then
> > > you need specialized ad-hoc logic to convert it to a graph format.
> > >
> > > What am I missing?
> >
> >
> > Hehe, that's been a bit tricky indeed, but more in terms of figuring
> > out that (and how) it can fit into the writer/textformatter
> > patterns, the result is rather simple and it requires only some
> > moderate extensions to the writer/formatter APIs:
> >
> >
> > It is using a Mermaid Flowchart graph with only 3 kinds of elements:
> >
> > - Shapes
> >   i.e. "boxes", can't be nested
> > - Subgraphs
> >   Container for shapes, can be nested
> > - Links
> >   connection from one shape to another
> >
> >
> > A simple graph with 2 shapes in a subgraph looks like this:
> >
> > ...
> > flowchart TD
> >  subgraph s1["Subgraph 1"]
> >         A("Shape A") --> B("Shape B")
> >   end
> > ...
> >
> > There's a link from A to B - but: it can also be written in a
> different way:
> >
> > ...
> > flowchart TD
> >  subgraph s1["Subgraph 1"]
> >         A("Shape A")
> >         B("Shape B")
> >   end
> >
> >   A --> B
> > ...
> >
> > This does the same thing. The link definitions can be at a different
> > place in the document - which is one of the key points to make it
> > work.
> >
> > Then there are three new section flags to denote subgraph, shapes
> > and links and two fields in the section struct (source_id_key,
> > dest_id_key). For link-producing sections they indicate the names of
> > the keys to use for building a link. A section can also produce both
> > - shapes and links when the flags are set accordingly.  I also use
> > another (existing) mechanism, which is the data object that can be
> > provided on section start, only that it's a (publicly) defined
> > "SectionContext" struct, which is a somewhat alternative
> > direction.
> 
> So basically a graph can be represented as a collection of subraphs,
> each subgraphs containing shapes, and links.

Yes, and link can exist at the top level (of nesting) and otherwise at any position in the document.


> > I kept it for the moment as suggestion because I didn't
> > know which ideas you have for the API to evolve, so that's just
> > something for discussion.
> 
> One of the possible uses is to expose the data printed by
> filters. E.g. detection filters are printing the information either in
> the stderr using custom formats, this should really be converted to
> something easier to consume (whatever formats for which you don't
> need a custom parser).
> 
> The metadata muxer also might benefit from using a text writer, to
> avoid again the need for a custom parser, and probably there are more
> use cases easy to spot.
> 
> So at some point we want to make this API accessbile from the
> libraries, that is to move them into libavutil. Again, it's fine to
> expose this at the tools level first so we can experiment and refine
> the interface before moving to a stable one.
> _______________________________________________

Yes, I see quite a number use cases as well.
It's still a work in progress, but I've set up a GitHub repo for documentation and as a place of interaction. It's not for ffmpeg code, but there are tests, examples and a documentation Wiki:

https://github.com/softworkz/ffmpeg_output_apis/wiki

(a number of things are still incomplete)


I'm also exploring a way for self-printing the data schema for each use case. On the repo Wiki there are Entity-Relationship diagrams for the FFprobe and the Graphs schema. I'll follow up with more details..

sw






_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".