Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
 help / color / mirror / Atom feed
From: Soft Works <softworkz@hotmail.com>
To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Subject: Re: [FFmpeg-devel] [PATCH 2/2] libavfi/dnn: add LibTorch as one of DNN backend
Date: Tue, 24 May 2022 15:29:12 +0000
Message-ID: <DM8P223MB03656D93AC985A52CC1649C6BAD79@DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM> (raw)
In-Reply-To: <8f3540a8-f902-4f6d-a761-9bd59fc094df@www.fastmail.com>



> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Jean-
> Baptiste Kempf
> Sent: Tuesday, May 24, 2022 4:52 PM
> To: ffmpeg-devel <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH 2/2] libavfi/dnn: add LibTorch as one
> of DNN backend
> 
> Hello,
> 
> On Tue, 24 May 2022, at 16:03, Fu, Ting wrote:
> > I am trying to add this backend since we got some users who have
> > interest in doing PyTorch model(BasicVSR model) inference with FFmpeg.
> 
> I think you are missing my point here.
> We already have 3 backends (TF, Native, OpenVino) in FFmpeg.
> Those are not to support different hardware, but different tastes for
> users, who prefer one API to another one.

It's not just about taste. Many models can only work with a specific
backend and converting is often between difficult and impossible.


> Where does it end? How many of those backends will we get? 10?

From my pov, the best solution not only for this but also for other 
use cases would be - as suggested a while ago - a plug-in model
for filters.


> What's the value to do that development inside ffmpeg?

That's connected to my question about 1-way or 2-way interaction
with those APIs.

When it's just about feeding video frames into such APIs, then
there wouldn't be much reason for having this integrated into
ffmpeg.

But as soon as you want to make modifications to video frames,
how could it be implemented otherwise? I mean, none of those
APIs are capable to do video processing like ffmpeg can do.

In any case, there needs to be some way to interact with those 
APIs and at both sides, at a certain point, you need to have
uncompressed images to work with, and when that memory could
be shared between ffmpeg and the AI API, it saves memory
and you get rid of encoding/decoding and load/save for
sharing the image between ffmpeg and the AI api.
Also, that kind of integration allows processing with 
lower latency, which is crucial when working with
live video.

I've never used it like this, but I'm sure I will.

Kind regards,
softworkz



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

  reply	other threads:[~2022-05-24 15:29 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-05-23  9:29 [FFmpeg-devel] [PATCH 1/2] libavfi/dnn: refine enum DNNColorOrder Ting Fu
2022-05-23  9:29 ` [FFmpeg-devel] [PATCH 2/2] libavfi/dnn: add LibTorch as one of DNN backend Ting Fu
2022-05-23  9:51   ` Jean-Baptiste Kempf
2022-05-24 14:03     ` Fu, Ting
2022-05-24 14:23       ` Soft Works
2022-05-25  3:20         ` Fu, Ting
2022-05-24 14:51       ` Jean-Baptiste Kempf
2022-05-24 15:29         ` Soft Works [this message]
2022-05-25  3:50         ` Fu, Ting

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=DM8P223MB03656D93AC985A52CC1649C6BAD79@DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM \
    --to=softworkz@hotmail.com \
    --cc=ffmpeg-devel@ffmpeg.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
		ffmpegdev@gitmailbox.com
	public-inbox-index ffmpegdev

Example config snippet for mirrors.


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git