[FFmpeg-devel] what AVCodecID to use for copying full vanc data between .mxf and .mcc

[FFmpeg-devel] what AVCodecID to use for copying full vanc data between .mxf and .mcc

[Jacob Lifshay]

I'm currently writing a .mcc muxer, it currently translates from eia-608/708 to full vanc packets before outputting a .mcc file:
https://github.com/programmerjake/FFmpeg/tree/add-mcc-mux

I want to add the ability to the mxf and mcc muxers/demuxers to keep the full vanc data when doing stream copies, what do you think is the best way to do that?

* add a new AVCodecID for mxf vbi_vanc_smpte_436M
* change the muxers/demuxers to be able to use AV_CODEC_ID_SMPTE_KLV or AV_CODEC_ID_SMPTE_2038 (they hold the full vanc packets, right?)
* use the mxf vbi_vanc_smpte_436M stream as is -- AV_CODEC_ID_NONE and with vbi_vanc_smpte_436M in the metadata

additionally, it'd be nice to be able to output a eia-608/708 stream to mxf, so that would either need a codec/filter of some sort to translate to full vanc data in whatever format the mxf muxer ends up supporting, or to have that translation built-in to the mxf muxer, like it is built-in to the mxf demuxer (with eia608_extract).

What do you think?

Jacob
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] what AVCodecID to use for copying full vanc data between .mxf and .mcc

[Devin Heitmueller]

Hi Jake,


On Tue, Jul 8, 2025 at 12:12 AM Jacob Lifshay
<programmerjake-at-gmail.com@ffmpeg.org> wrote:
>
> I'm currently writing a .mcc muxer, it currently translates from eia-608/708 to full vanc packets before outputting a .mcc file:
> https://github.com/programmerjake/FFmpeg/tree/add-mcc-mux

I took a quick look at your branch.  So this is a good start, IMHO...

Note that I took a look at the MCC demuxer last year, and it's got a
bunch of issues where the parsing makes a bunch of assumptions (e.g.
doesn't filter out other VANC types which might be present in an MCC
such as AFD).  And it only extracts 608 (i.e. doesn't handle 708).
I'm mentioning this should you try using the MCC demuxer while
developing a muxer.  The demuxer would need some love before I would
trust it for anything in production.

> I want to add the ability to the mxf and mcc muxers/demuxers to keep the full vanc data when doing stream copies, what do you think is the best way to do that?
>
> * add a new AVCodecID for mxf vbi_vanc_smpte_436M
> * change the muxers/demuxers to be able to use AV_CODEC_ID_SMPTE_KLV or AV_CODEC_ID_SMPTE_2038 (they hold the full vanc packets, right?)

I would stay away from KLV entirely.

The SMPTE 2038 standard does support all VANC data types, but support
in ffmpeg really is only oriented around passthrough (i.e. routing
2038 packets from an input TS to an output TS).  It currently doesn't
attempt to decode the 2038 payloads into something easier to work
with.  I've built stuff like bitstream filters that take in 2038
packets and use libklvanc to parse out the individual VANC sections,
but none of that is upstream (happy to send you links to where it can
be found in my public GitHub repo).  In other words, you might have
difficulty working with the structures if you have to deserialize them
and reserialize them when passing them between modules (although
libklvanc does technically support both).

> * use the mxf vbi_vanc_smpte_436M stream as is -- AV_CODEC_ID_NONE and with vbi_vanc_smpte_436M in the metadata

I can't speak to whether this would be a good idea.

> additionally, it'd be nice to be able to output a eia-608/708 stream to mxf, so that would either need a codec/filter of some sort to translate to full vanc data in whatever format the mxf muxer ends up supporting, or to have that translation built-in to the mxf muxer, like it is built-in to the mxf demuxer (with eia608_extract).

I've done this sort of thing with bitstream filters.  The BSF takes in
2038, extracts the 608, and outputs 608 packets.  You could do
something comparable for MXF.  Also, I set up the muxers to
automatically insert the BSF if needed, which makes it painless for
the user.

The 608/708 stuff is kind of a mess in terms of what is in those
payloads.  Some modules will expect 608 only, others will contain
tuples that include 708.  And there continues to be no easy way to go
back and forth between 608 packets and SEI data embedded within the
video stream (in either direction:  SEI to 608 packet streams, or 608
packet streams into SEI).

Devin

-- 
Devin Heitmueller, Senior Software Engineer
LTN Global Communications
o: +1 (301) 363-1001
w: https://ltnglobal.com  e: devin.heitmueller@ltnglobal.com
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] what AVCodecID to use for copying full vanc data between .mxf and .mcc

[Jacob Lifshay]

On Tue, Jul 8, 2025 at 5:08 PM Devin Heitmueller
<devin.heitmueller@ltnglobal.com> wrote:
> I took a quick look at your branch.  So this is a good start, IMHO...

thanks!

> Note that I took a look at the MCC demuxer last year, and it's got a
> bunch of issues where the parsing makes a bunch of assumptions (e.g.
> doesn't filter out other VANC types which might be present in an MCC
> such as AFD).  And it only extracts 608 (i.e. doesn't handle 708).

it does handle 708, since they're using the exact same triples. I've
been using it in the casparcg branch I've been working on and it seems
to work fine, giving me the 608 and 708 captions streams. I haven't
tried to use it with any .mcc files that have anything other than 708
captions vanc packets however -- do those actually exist? it seems
like the format supports them, but every file I've seen just uses it
for 708 captions. I couldn't find any by searching github for:
path:*.mcc /^[0-9:]{11}\t[^T]/

> I'm mentioning this should you try using the MCC demuxer while
> developing a muxer.  The demuxer would need some love before I would
> trust it for anything in production.
>
> > I want to add the ability to the mxf and mcc muxers/demuxers to keep the full vanc data when doing stream copies, what do you think is the best way to do that?
> >
> > * add a new AVCodecID for mxf vbi_vanc_smpte_436M

I'm thinking this is the best option, since it's a pretty simple
format so the required [de]serialization is pretty trivial. it's just
the data in Table 7 (starts on page 13) of:
https://pub.smpte.org/latest/st436/s436m-2006.pdf

(note it uses a klv array which is just u32be array len, u32be array
element size (always 1 here), and the array contents)

> > * change the muxers/demuxers to be able to use AV_CODEC_ID_SMPTE_KLV or AV_CODEC_ID_SMPTE_2038 (they hold the full vanc packets, right?)
>
> I would stay away from KLV entirely.

ok.

> In other words, you might have
> difficulty working with the structures

yeah, it looks waay overcomplicated for what I want.

> > additionally, it'd be nice to be able to output a eia-608/708 stream to mxf, so that would either need a codec/filter of some sort to translate to full vanc data in whatever format the mxf muxer ends up supporting, or to have that translation built-in to the mxf muxer, like it is built-in to the mxf demuxer (with eia608_extract).
>
> I've done this sort of thing with bitstream filters.  The BSF takes in
> 2038, extracts the 608, and outputs 608 packets.  You could do
> something comparable for MXF.

so I could write bitstream filters to interconvert between 608/708 and
mxf vbi_vanc_smpte_436M and modify the muxers to auto-insert the
bitstream filters?

> Also, I set up the muxers to
> automatically insert the BSF if needed, which makes it painless for
> the user.

so, just add the right ff_stream_add_bitstream_filter calls in the
muxers' init functions?

Thanks,
Jacob
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".