[FFmpeg-devel] [RFC] FFmpeg Execution Graph Visualization

* [FFmpeg-devel] [RFC] FFmpeg Execution Graph Visualization
@ 2025-03-18  2:32 Soft Works
  2025-03-18  6:43 ` Diederick C. Niehorster
  2025-03-21 18:27 ` Stefano Sabatini
  0 siblings, 2 replies; 4+ messages in thread
From: Soft Works @ 2025-03-18  2:32 UTC (permalink / raw)
  To: FFmpeg development discussions and patches

Hello everybody,

working on the subject of writing out filtergraph information obviously implies the goal of being able to visualize that data in some way. While I do have something for long, it's tailored for specific workflows and is hardly useful for most.
Anyway, it shouldn't be required to use once another software for visualize the output. This decimates the usability for any such feature by far.
Few days ago, I was thinking about ways for making this feature really useful for everybody without needing to jump through any extra hoops. We have that range of writers (now "text formatters"), and so I wondered whether there isn't some text format that is meant to directly represent a graph for visualization, which finally reminded me about

MERMAID

Had used it just twice or thrice on GitHub for simple things, and as I found out later, it's by far less capable than I would have expected (more than a decade of age, current version 11.x) --- but: it has a number of unique selling points:

- The format is simple and transparent
- It's hand-editable and easily readable
- There's quite an ecosystem around it with many integrations and tools available
  The #1 for me though is:
- It goes hand-in-hand with Markdown
  => almost everywhere Markdown is supported, Mermaid diagrams are supported as well

Practically, this means that you can paste a Mermaid diagram into any message field on GitHub (for example; forgejo supports it as well as many others) - and the diagram is rendered automatically with a kind of interactive viewer.
Enough reason to give it a try and I also had some quick success. Writing that format fits rather well into the logic of the existing text formatters without major changes.
That way I ended up with a new 

avtextformatter_mermaid ("mermaidwriter" in old terminology)

Most of the mentioned deficiencies could be remedied by CSS styling (even though it's been really tedious). 
Now well - when viewing filtergraphs, you often want to know details about the input and output streams. For that other tool I was parsing out this information from the ffmpeg log lines describing the streams on startup. But this work is happening right at the source of information, so I added input/output files and streams to the filtergraph printing as well (just as much detail as needed for graphing).
The unthankful effect: by that, it became apparent that something else is missing: encoders and decoders - added those as well and eventually got reasonable results.
Remained one last part:

Usability

Of course, it's still not a great experience when you need to copy/paste the generated diagram code in some web portal. To solve that, I've created once another textformatter(writer): mermaidhtml.
It's just an extension to the Mermaid formatter which wraps the diagram as an html file for local viewing (yet not fully offline).
The final idea would be to simplify usage as much as possible, so that all you need to do is adding a small option to the Ffmpeg command line, like -sg ('show graph') which would cause Ffmpeg to create such mermaid html file on exit and automatically launches the browser (if any) to view that file. 
While working or trying a range of commands, after another, this will also create a browser tab for each run, allowing for easy comparison between recent commands.

Examples

I've created a Gist with some examples of the output here:

https://gist.github.com/softworkz/a196b2d0e9e2df49f766abd92f508551

(also includes a zip with html file examples)

Questions

I'm curious what you think about it!

- What's good, what's bad, what should be changed/improved?

- What about the displayed information, should something be added?

- I'm folding out the buffer source/sink filters to simplify the view, is that ok?
  Should the TRIM filters be excluded as well?

- Since it's no longer just about filter graphs - what do you think about the term
  "Ffmpeg Execution Graph"?
  (other ideas welcome)

- Does anybody have some complex command lines for me to test?
  (no need to include media files, I can try to replicate 
  something similar)

Thanks,
sw

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 4+ messages in thread