[FFmpeg-devel] [Question] Learning about VVC and wasm

[FFmpeg-devel] [Question] Learning about VVC and wasm

[Sidd via ffmpeg-devel]

Hi, I was curious on learning more about VVC and wasm within the
ffmpeg project and would like some guidance on where to find the docs
and references to go through

I am simply trying to get my hands dirty with a past GSoC project to
get the feel of working on the source code (hopefully to understand it
better)

Any help on this would be much appreciated
_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

[FFmpeg-devel] Re: [Question] Learning about VVC and wasm

[Frank Plowman via ffmpeg-devel]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 1806 bytes --]

On 13/11/2025 16:46, Sidd via ffmpeg-devel wrote:
> Hi, I was curious on learning more about VVC and wasm within the
> ffmpeg project and would like some guidance on where to find the docs
> and references to go through
> 
> I am simply trying to get my hands dirty with a past GSoC project to
> get the feel of working on the source code (hopefully to understand it
> better)
> 
> Any help on this would be much appreciated

If you have prior experience with AVC and HEVC, [1] is an overview of
what was introduced in VVC.  If not, a good place to start would be with
Iain E. Richardson's "The H.264 Advanced Video Compression Standard".
He's also got a new book out, "Coding Video: A Practical Guide to HEVC
and Beyond", but I'm not sure if that’s a good introductory text or
whether it assumes some knowledge.

As far as FFmpeg’s implementation goes, probably the most notable thing
is the wavefront parallel processing (WPP) multithreading model, which
is documented to some extent at [2].  The code for the multithreading
aspect is pretty well-contained to libavcodec/vvc/thread.c so you can
get a sense of how the decoder works at a high level there.  There
aren't many docs for the decoder though really: it's an implementation
of the standard.  In terms of the progress on the decoder and what needs
work, its C code is in a good state but it needs more assembly written
for it, particularly on x86.

If you have any specific questions about VVC or the FFmpeg
implementation, feel free to ask me.  I don't know anything about wasm.

[1] https://ieeexplore.ieee.org/document/9503377/
[2] https://archive.fosdem.org/2024/events/attachments/fosdem-2024-3538-
ffmpeg-vvc-decoder/slides/22900/VVC_decoder_in_FFmpeg_Handout-1_FP3o8rS.pdf

--
All the best,
Frank

[-- Attachment #1.1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 1091 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

[-- Attachment #2: Type: text/plain, Size: 163 bytes --]

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

[FFmpeg-devel] Re: [Question] Learning about VVC and wasm

[Sidd via ffmpeg-devel]

Hi Frank,

Thanks for the reply, it seems like there is a lot for me to cover.

Regarding the assembly, since FFmpeg is known for handcrafting it, are
there any references of help in their docs regarding this? I have gone
through their ASM lessons and know the basics of them but am quite far
from being good at writing it. And if I may ask, what exactly are the
current drawbacks of the current C implementation?

Thank you for taking your time to answer my questions.

On Thu, Nov 13, 2025 at 11:13 PM Frank Plowman via ffmpeg-devel
<ffmpeg-devel@ffmpeg.org> wrote:
>
> On 13/11/2025 16:46, Sidd via ffmpeg-devel wrote:
> > Hi, I was curious on learning more about VVC and wasm within the
> > ffmpeg project and would like some guidance on where to find the docs
> > and references to go through
> >
> > I am simply trying to get my hands dirty with a past GSoC project to
> > get the feel of working on the source code (hopefully to understand it
> > better)
> >
> > Any help on this would be much appreciated
>
> If you have prior experience with AVC and HEVC, [1] is an overview of
> what was introduced in VVC.  If not, a good place to start would be with
> Iain E. Richardson's "The H.264 Advanced Video Compression Standard".
> He's also got a new book out, "Coding Video: A Practical Guide to HEVC
> and Beyond", but I'm not sure if that’s a good introductory text or
> whether it assumes some knowledge.
>
> As far as FFmpeg’s implementation goes, probably the most notable thing
> is the wavefront parallel processing (WPP) multithreading model, which
> is documented to some extent at [2].  The code for the multithreading
> aspect is pretty well-contained to libavcodec/vvc/thread.c so you can
> get a sense of how the decoder works at a high level there.  There
> aren't many docs for the decoder though really: it's an implementation
> of the standard.  In terms of the progress on the decoder and what needs
> work, its C code is in a good state but it needs more assembly written
> for it, particularly on x86.
>
> If you have any specific questions about VVC or the FFmpeg
> implementation, feel free to ask me.  I don't know anything about wasm.
>
> [1] https://ieeexplore.ieee.org/document/9503377/
> [2] https://archive.fosdem.org/2024/events/attachments/fosdem-2024-3538-
> ffmpeg-vvc-decoder/slides/22900/VVC_decoder_in_FFmpeg_Handout-1_FP3o8rS.pdf
>
> --
> All the best,
> Frank
> _______________________________________________
> ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
> To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org
_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

[FFmpeg-devel] Re: [Question] Learning about VVC and wasm

[Frank Plowman via ffmpeg-devel]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 1340 bytes --]

On 13/11/2025 17:56, Sidd via ffmpeg-devel wrote:
> Hi Frank,
> 
> Thanks for the reply, it seems like there is a lot for me to cover.
> 
> Regarding the assembly, since FFmpeg is known for handcrafting it, are
> there any references of help in their docs regarding this? I have gone
> through their ASM lessons and know the basics of them but am quite far
> from being good at writing it. And if I may ask, what exactly are the
> current drawbacks of the current C implementation?
> 
> Thank you for taking your time to answer my questions.
> 

The assembly lessons are the only "intro to SIMD" resource specifically
for FFmpeg that I know of, but I'm sure you can find more stuff by
googling.  A more practical resource for x86 specifically is [1] on
x86inc.asm.  I think the most important thing is to practice.  Yes, the
learning curve is steep.  I can't say I'm very good at writing SIMD.

I'm not quite sure what you mean by "drawbacks of the current C
implementation" -- as opposed to what?  As I said, the C high-level
decoder logic is in a good state, but the decoder is quite slow and
needs more ASM to make it faster.

Nit: top-posting is discouraged on mailing lists such as ffmpeg-devel.

[1]
https://blogs.gnome.org/rbultje/2017/07/14/writing-x86-simd-using-x86inc-asm/

-- 
All the best,
Frank

[-- Attachment #1.1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 1091 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

[-- Attachment #2: Type: text/plain, Size: 163 bytes --]

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

[FFmpeg-devel] Re: [Question] Learning about VVC and wasm

[Sidd via ffmpeg-devel]

Ok I misunderstood about the decoder part.

Sorry I am new to mailing lists am not sure what top-posting means, if
I have not abided by any rules I do apologize and will refrain from
doing so

On Thu, Nov 13, 2025 at 11:40 PM Frank Plowman via ffmpeg-devel
<ffmpeg-devel@ffmpeg.org> wrote:
>
> On 13/11/2025 17:56, Sidd via ffmpeg-devel wrote:
> > Hi Frank,
> >
> > Thanks for the reply, it seems like there is a lot for me to cover.
> >
> > Regarding the assembly, since FFmpeg is known for handcrafting it, are
> > there any references of help in their docs regarding this? I have gone
> > through their ASM lessons and know the basics of them but am quite far
> > from being good at writing it. And if I may ask, what exactly are the
> > current drawbacks of the current C implementation?
> >
> > Thank you for taking your time to answer my questions.
> >
>
> The assembly lessons are the only "intro to SIMD" resource specifically
> for FFmpeg that I know of, but I'm sure you can find more stuff by
> googling.  A more practical resource for x86 specifically is [1] on
> x86inc.asm.  I think the most important thing is to practice.  Yes, the
> learning curve is steep.  I can't say I'm very good at writing SIMD.
>
> I'm not quite sure what you mean by "drawbacks of the current C
> implementation" -- as opposed to what?  As I said, the C high-level
> decoder logic is in a good state, but the decoder is quite slow and
> needs more ASM to make it faster.
>
> Nit: top-posting is discouraged on mailing lists such as ffmpeg-devel.
>
> [1]
> https://blogs.gnome.org/rbultje/2017/07/14/writing-x86-simd-using-x86inc-asm/
>
> --
> All the best,
> Frank_______________________________________________
> ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
> To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org
_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

[FFmpeg-devel] Re: [Question] Learning about VVC and wasm

[Rémi Denis-Courmont via ffmpeg-devel]

Le torstaina 13. marraskuuta 2025, 19.56.30 Itä-Euroopan normaaliaika Sidd via 
ffmpeg-devel a écrit :
> Regarding the assembly, since FFmpeg is known for handcrafting it, are
> there any references of help in their docs regarding this?

Generally, the ISA should cover assembler in the specifications.

For FFmpeg, WASM optimisations are embryonic at best. There is no 
documentation since there is very little experience with them as yet.

> I have gone through their ASM lessons and know the basics of them but am
> quite far from being good at writing it.

> And if I may ask, what exactly are the current drawbacks of the current C
> implementation?

FFmpeg (and also dav1d and x264) generally dislike C intrinsic optimisations 
because:
- they are generally incompatible with run-time feature detection,
- the syntax tends to be even more cumbersome than assembler,
- the compiler may mess up the register allocation and
  perform unnecessary moves or spills,
- some functionality may not even be exposed via intrinsics.

I do not know if any of those points are relevant for WASM though.

-- 
Rémi Denis-Courmont
https://www.remlab.net/



_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

[FFmpeg-devel] Re: [Question] Learning about VVC and wasm

[Zhao Zhili via ffmpeg-devel]

> On Nov 14, 2025, at 02:19, Rémi Denis-Courmont via ffmpeg-devel <ffmpeg-devel@ffmpeg.org> wrote:
> 
> Le torstaina 13. marraskuuta 2025, 19.56.30 Itä-Euroopan normaaliaika Sidd via 
> ffmpeg-devel a écrit :
>> Regarding the assembly, since FFmpeg is known for handcrafting it, are
>> there any references of help in their docs regarding this?
> 
> Generally, the ISA should cover assembler in the specifications.
> 
> For FFmpeg, WASM optimisations are embryonic at best. There is no 
> documentation since there is very little experience with them as yet.

The wasm and wasm simd has already reached a considerable level of maturity.
For H.265 decoding, I was able to achieve a speed boost of approximately 120%
compared to the version without SIMD (decoding performance increased from 54
fps to 114 fps for 1080p video), and 55% compared to the version of compiler
auto vectorize (from 74 fps to 114 fps).

emscripten has good documentation
https://emscripten.org/docs/porting/simd.html

With wasi(WebAssembly System Interface) like wasmtime runtime, ffmpeg can be
running like normal cmdline program outside of Web browser. checkasm works
as other cpu arch, perf also works as for native program.

https://docs.wasmtime.dev/examples-profiling-perf.html

wasm itself is focus on reuse native code, it’s the same for wasm simd.

From emscripten,
> There are five different ways to leverage WebAssembly SIMD in your C/C++ programs:
> Enable LLVM/Clang SIMD autovectorizer to automatically target WebAssembly SIMD, without requiring changes to C/C++ source code.
> Write SIMD code using the GCC/Clang SIMD Vector Extensions (__attribute__((vector_size(16))))
> Write SIMD code using the WebAssembly SIMD intrinsics (#include <wasm_simd128.h>)
> Compile existing SIMD code that uses the x86 SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, AVX or AVX2 intrinsics (#include <*mmintrin.h>)
> Compile existing SIMD code that uses the ARM NEON intrinsics (#include <arm_neon.h>)

Auto conversion doesn’t work for us since we have no x86 or arm intrinsics.
wasm simd intrinsics is a good balance(I’m only talk about this non-native instruction case).

The technique and how to make it work with FFmpeg isn’t a problem. The question
is how many usecase is there, I cannot predict usage.

> 
>> I have gone through their ASM lessons and know the basics of them but am
>> quite far from being good at writing it.
> 
>> And if I may ask, what exactly are the current drawbacks of the current C
>> implementation?
> 
> FFmpeg (and also dav1d and x264) generally dislike C intrinsic optimisations 
> because:
> - they are generally incompatible with run-time feature detection,
> - the syntax tends to be even more cumbersome than assembler,
> - the compiler may mess up the register allocation and
>  perform unnecessary moves or spills,
> - some functionality may not even be exposed via intrinsics.
> 
> I do not know if any of those points are relevant for WASM though.
> 
> -- 
> Rémi Denis-Courmont
> https://www.remlab.net/
> 
> 
> 
> _______________________________________________
> ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
> To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

[FFmpeg-devel] Re: [Question] Learning about VVC and wasm

[Rémi Denis-Courmont via ffmpeg-devel]

Le 14 novembre 2025 06:09:25 GMT+02:00, Zhao Zhili via ffmpeg-devel <ffmpeg-devel@ffmpeg.org> a écrit : 
>> For FFmpeg, WASM optimisations are embryonic at best. There is no 
>> documentation since there is very little experience with them as yet.
>
>The wasm and wasm simd has already reached a considerable level of maturity.

There is only HEVC IDCT for 8- and 10-bit, and SAO for 8-bit. I don't doubt that those optimisations are good and mature. But in terms of coverage, compared to x86 and AArch64. That is embryonic IMO.

There are also no WASM guidelines in the optimisation doxygen file. How to write good assembler that runs fast on x86 and on AArch64 with intrinsics is not explained or referenced either.

To be clear, I'm not *faulting* anyone, and the RISC-V stuff, that I've been involved with, is also in many ways "embryonic" if less so than WASM's.

Br,
_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

[FFmpeg-devel] Re: [Question] Learning about VVC and wasm

[Zhao Zhili via ffmpeg-devel]

> On Nov 14, 2025, at 19:30, Rémi Denis-Courmont via ffmpeg-devel <ffmpeg-devel@ffmpeg.org> wrote:
> 
> 
> 
> Le 14 novembre 2025 06:09:25 GMT+02:00, Zhao Zhili via ffmpeg-devel <ffmpeg-devel@ffmpeg.org> a écrit :
>>> For FFmpeg, WASM optimisations are embryonic at best. There is no 
>>> documentation since there is very little experience with them as yet.
>> 
>> The wasm and wasm simd has already reached a considerable level of maturity.
> 
> There is only HEVC IDCT for 8- and 10-bit, and SAO for 8-bit. I don't doubt that those optimisations are good and mature. But in terms of coverage, compared to x86 and AArch64. That is embryonic IMO.

Oh I’m talking about the general wasm/wasm simd128 technique, not binding to FFmpeg.

Current wasm implementation in FFmpeg isn’t far from a proof of concept. I have done more work on this and get better speed boost, but the work isn’t in good
shape yet.

> 
> There are also no WASM guidelines in the optimisation doxygen file. How to write good assembler that runs fast on x86 and on AArch64 with intrinsics is not explained or referenced either.
> 
> To be clear, I'm not *faulting* anyone, and the RISC-V stuff, that I've been involved with, is also in many ways "embryonic" if less so than WASM's.
> 
> Br,
> _______________________________________________
> ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
> To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org