[FFmpeg-devel] Re: [GSoC][RFC] GSoC 2026 Proposal (350 Hours): High-Performance LibTorch Backend Modernization

[Michael Niedermayer via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>]

[-- Attachment #1.1: Type: text/plain, Size: 2483 bytes --]

Hi Raja, Guo

On Sun, Jan 25, 2026 at 10:30:25PM +0530, Raja Rathour via ffmpeg-devel wrote:
> *GSoC 2026 Proposal: High-Performance LibTorch Backend Modernization*
> 
> *Candidate:* Raja Rathour
> *Project Type:* Large (350 Hours)
> *Mentor:* Guo Yejun

if guo agrees then someone should add this to
https://trac.ffmpeg.org/wiki/SponsoringPrograms/GSoC/2026

we need more projects listed on the above idea page ASAP because
google will look at this and say "wtf that page is nearly empty"



> *1. Problem Statement: Bridging the Integration Gap*
> 
> While the LibTorch backend logic was present in the source tree, it was
> functionally inaccessible to end-users due to a registration mismatch in
> the AVOption system. Specifically, the dnn_backend unit in
> vf_dnn_processing.c lacked the necessary constants to map the user input
> string "torch" to the internal DNN_TH backend ID. This caused the following
> failure:
> 
> *The Error:*
> [Parsed_dnn_processing_0 @ 0x...] Option 'dnn_backend' not found
> 
> *The Fix:*
> I have already diagnosed and resolved this by correctly registering the
> torch constant in the dnn_processing_options array and updating DnnContext
> offsets.
> *2. Current Progress & Functional Verification*
> 
> I have verified the end-to-end inference pipeline using a local build
> (--enable-libtorch). The following terminal output serves as proof of
> concept, showing successful 25-frame inference at 14.8x speed using the new
> LibTorch integration:
> 
> *# Verified Command Line Proof*
> 
> ./ffmpeg -f lavfi -i testsrc=duration=1 -vf
> "dnn_processing=model=model.pt:dnn_backend=torch" -f null -
> ...
> Stream mapping: Stream #0:0 -> #0:0 (wrapped_avframe -> wrapped_avframe)
> frame= 25 fps=0.0 time=00:00:01.00 speed=14.8x
> 
> This confirms that the "plumbing" between FFmpeg's filtergraph and the
> LibTorch engine (using at::from_blob for memory wrapping) is fully
> operational.
> [image: image.png]
> 
> 
> *3. 350-Hour Technical Roadmap (12-Week Plan)*

I do like the idea of improving libtorch support
I have to admit i probably never used FFmpeg with libtorch
but i used libtorch directly in another project i wrote.

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Take away the freedom of one citizen and you will be jailed, take away
the freedom of all citizens and you will be congratulated by your peers
in Parliament.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 163 bytes --]

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org