Re: [FFmpeg-devel] Captions SCC

[Tom Vaughan <tom@tvaughan.com>]

I suspect WebVTT became the modern CC format because...
- It is natively supported by the HLS and DASH specifications
	- https://developer.apple.com/documentation/http-live-streaming/hls-authoring-specification-for-apple-devices
	- https://reference.dashif.org/dash.js/latest/samples/captioning/caption_vtt.html
- Streaming is HTTP - based, and WebVTT is a W3C specification.
	- https://www.w3.org/TR/webvtt1/
- WebVTT is delivered as a component (separate "sidecar" file). As such a streaming title can be updated to fix or improve a bad captions track, or to add support for additional languages without re-encoding or re-muxing the encoded video files. Updated a title to add support for another language only requires encoding the additional audio track, then packaging the updated set of audio, video and WebVTT files.
- Wide industry support in HLS and DASH packaging libraries (open source and commercial)
- Wide industry support in streaming player client platforms - both software and device native players
- Better support of dynamic ad insertion (avoids timing errors that cause program captions to appear over an ad, or vice versa)

As Zach noted, WebVTT supports setting the font color and background color. It also supports cue styling, including bold, underlined and italics.
See https://developer.mozilla.org/en-US/docs/Web/API/WebVTT_API for a live example.

Tom

On 2/9/25, 9:42 AM, "ffmpeg-devel on behalf of Devlist Archive" <ffmpeg-devel-bounces@ffmpeg.org <mailto:ffmpeg-devel-bounces@ffmpeg.org> on behalf of devlist@rlb.org <mailto:devlist@rlb.org>> wrote:


>
> Not to start an argument, but WebVTT is kind of a terrible format.
> It's a lowest common denominator and loses most formatting information
> available even in 608 (which is now more than 40 years old). Stuff
> like rollup captions for live programming, color (to distinguish
> speakers) and caption positioning are pretty important to the hearing
> impaired.

From the reading I have done, the WebVTT does support some placement,
italics, and appearance information, but not all players or ripping
programs support those functions. It does appear that 708 has even more
options for the number of captions windows, font sizes, colors, etc that
can make a big difference in experience.

The new captions standard for ATSC 3.0 is actually ISMC1 profile TTML files
that are a sidecar configuration by default. I would highly recommend
focusing on 608 and 708 compatibility first, and only then make sure all
possible features of those standards can be exported to webVTT for those
who will sponsor it. ISMC1 profile TTML won't really be needed until the
majority of TV tuners ship with ATSC 3.0 which is still quite far off.
Implementation of TTML outputs from 708 tracks will require the ability to
extract the entire A/53 Part 4 encoded stream to a sidecar file so
implementing bitstream filters to both mux and demux the captions stream
from the video stream for the full A/53 Part 4 dataset will be required.

Zach




On Sun, Feb 9, 2025 at 6:03 AM Devin Heitmueller <
devin.heitmueller@ltnglobal.com <mailto:devin.heitmueller@ltnglobal.com>> wrote:


> On Sat, Feb 8, 2025 at 10:39 PM Tom Vaughan <tom@tvaughan.com <mailto:tom@tvaughan.com>> wrote:
> >
> > I'm excited to see this discussion. There are many leading video
> distributors (streaming services, etc.) that would love to see solid
> support in FFMPEG for closed captions. No promises, but my sense is that
> additional money can be raised to sponsor this development.
> >
> > WebVTT as a sidecar captions file is the modern way to handle captions.
> The latest streaming client devices / applications support WebVTT captions.
> New movies and TV shows are produced with WebVTT captions.
>
> Not to start an argument, but WebVTT is kind of a terrible format.
> It's a lowest common denominator and loses most formatting information
> available even in 608 (which is now more than 40 years old). Stuff
> like rollup captions for live programming, color (to distinguish
> speakers) and caption positioning are pretty important to the hearing
> impaired.
>
> Yesterday I watched a DVD I ripped (purely for personal use) of a TV
> show with VTT captions for something that I knew what the original 608
> looked like, and wow what a difference (the WebVTT looked dramatically
> worse than the original 608).
>
> To my point: no, I don't think normalizing everything down to WebVTT
> is a good idea.
>
> Much of the goal, at least in the work that I do, is to conform to the
> FCC requirements, which generally require that the original 608/708
> from the content provider be preserved.
>
> There can be money to support such efforts. Most of my work in
> captioning within ffmpeg was funded by my employer, and most of the
> work I did improving 608 captions in VLC was financed by contract work
> I did for commercial customers who were embedding VLC into solutions
> and needed decent caption support.
>
> Devin
>
> --
> Devin Heitmueller, Senior Software Engineer
> LTN Global Communications
> o: +1 (301) 363-1001
> w: https://ltnglobal.com e <https://ltnglobal.com&nbsp;&nbsp;e>: devin.heitmueller@ltnglobal.com <mailto:devin.heitmueller@ltnglobal.com>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org <mailto:ffmpeg-devel@ffmpeg.org>
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel <https://ffmpeg.org/mailman/listinfo/ffmpeg-devel>
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org <mailto:ffmpeg-devel-request@ffmpeg.org> with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org <mailto:ffmpeg-devel@ffmpeg.org>
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel <https://ffmpeg.org/mailman/listinfo/ffmpeg-devel>


To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org <mailto:ffmpeg-devel-request@ffmpeg.org> with subject "unsubscribe".



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".