Re: [FFmpeg-devel] what AVCodecID to use for copying full vanc data between .mxf and .mcc

[Jacob Lifshay <programmerjake@gmail.com>]

On July 22, 2025 5:21:24 AM PDT, "Tomas Härdin" <git@haerdin.se> wrote:
> mån 2025-07-07 klockan 21:12 -0700 skrev Jacob Lifshay:
> > I'm currently writing a .mcc muxer, it currently translates from eia-
> > 608/708 to full vanc packets before outputting a .mcc file:
> > https://github.com/programmerjake/FFmpeg/tree/add-mcc-mux

I've since changed the mcc muxer to support 436m, see below.
> > 
> > I want to add the ability to the mxf and mcc muxers/demuxers to keep
> > the full vanc data when doing stream copies, what do you think is the
> > best way to do that?
> 
> Stream copy should be easy enough. I'm not sure whether BSFs is the way
> to go in the long run, since full ANC support may require dealing with
> full uncropped frames and the ability to encode/decode VBI data.

imo that needs the ability for libavfilter to work on subtitles streams which seems unlikely to happen anytime soon based on how softworkz last attempt went. so I decided to write bitstream filters for now because they work on subtitles streams too.

> We
> also currently lack proper 608/708 support, so I'm not sure what the
> utility of converting from 436M to 608 is as things stand at the
> moment.

we actually have waay more 608/708 support than 436M, there are codecs to decode it to subtitles and subcc can extract it from video side data. 436M (until I merge the code I'm working on) only supports mxf muxing/demuxing (which is also just as much support as any of the other fully general vanc formats afaik).

The code I'm working on adds 436m output from the mcc decoder, adds two bitstream filters for conversion between 608/708 and 436m, adds a avpriv_* API for manipulating 436m data and extracting 608/708 data, and adds a mcc encoder that supports both 608/708 and 436m input by auto-inserting the 608 to 436m bitstream filter (this is changed from my earlier email). it also tries to add 608/708 output in the mxf muxer by auto-inserting that bitstream filter but I haven't finished figuring out all the code that needs changing to support that without the output being different from manually inserting that bitstream filter so the mxf muxer only ever sees a 436m data stream.

> Also horizontal ANC data exists.

mxf 436m supports HANC. i'd have to double check the code I wrote, since iirc it checks for only vanc data in some places, i assumed the mxf 436m is only vanc since vanc was in the metadata name back when it was a AV_CODEC_ID_NONE (as it is currently upstream, my code changes that to a new codec id).

> Stream copying 436M sounds
> reasonable enough.
> 
> Consider some potential use cases:
> 
> * copying 436M data from MXF to MXF
supported upstream now, my code retains support for that.

> * decoding 8-bit 436M data to 1-bit data
1-bit data is not allowed with the essence key I'm using according to the 436M specification, there's a separate essence key for VBI data that I'm not changing how ffmpeg handles it, so it's still AV_CODEC_ID_NONE with metadata set to VBI something. the 436m manipulation API I wrote has enumerants and AVOptions for it but not decoding/encoding support, it only supports ANC for that.

https://code.ffmpeg.org/programmerjake/FFmpeg/src/branch/add-mcc-mux/libavcodec/vanc_smpte_436m.h (i may rename that -- vanc -> anc)

> * "encoding" (rendering) 1-bit 436M data to 8-bit data
same as previous

> * extracting teletext data from 436M and rendering it
that should be easily doable similarly to how 

> * combining 436M and video essence to a full frame for playout
iirc ffmpeg architecturally doesn't support merging a 608 stream into video side data since bitstream filters only have one input and avframe filters don't support subtitles. I've had to work around that with custom code in casparcg. 436m data is similar.

> * embedding/de-embedding 608/708 CC:s and transcoding to/from regular
> subtitles

there are codecs for decoding 608/708 (but no encoders) and libavfilter has custom code for de-embedding from a video stream (subcc). embedding in a video stream isn't supported due to the architectural issues.

> * "encoding" subtitles as teletext, then embedding the resulting ANC
> data in whatever way is appropriate (as video samples, as 436M data or
> whatever)
> * possibly some cursed combination of all of the above, like doing
> teletext and closed captions at the same time

I haven't thought much about teletext, I'm assuming it'll follow a very similar path to 608/708. I'm working on all this for a TV station in the USA so only needed to bother with 608/708 data.

> > * add a new AVCodecID for mxf vbi_vanc_smpte_436M
> > * change the muxers/demuxers to be able to use AV_CODEC_ID_SMPTE_KLV
> > or AV_CODEC_ID_SMPTE_2038 (they hold the full vanc packets, right?)
> 
> Having key and length as part of the "essence" for these types of
> streams sounds wrong and cursed. Shouldn't we do the same thing we do
> for audio and video, and convert either to some common format?

sure, for now I've picked 436m as the common format and left other vanc formats <-> 436m bitstream filters for others to implement.

> Otherwise, wouldn't we need to write BSFs for the outer product of
> every type of ANC "codec"?

technically you can chain bsfs using av_bsf_list_finalize so we just have to support every anc codec <-> 436m. so when those eventually get implemented by someone else you can do AV_CODEC_ID_SMPTE_KLV -> AV_CODEC_ID_SMPTE_2038 via using 436m as an intermediate format.

Jacob
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".