Re: [FFmpeg-devel] Captions SCC

[Tom Vaughan <tom@tvaughan.com>]

I know what an elementary video stream is. I founded and led development of x265, which produces elementary video streams. Elementary video streams can contain SEI messages.
It's common to refer to the integration of closed captions into an elementary video stream as multiplexing. For example, see https://doc.haivision.com/MakitoX4Enc/1.7/closed-captioning

It is common for people in the industry to refer to burned-in (or open) captions as subtitles. Those are the only type of captions that would require a video encoder. 

Closed captions are pure metadata, encapsulated in Network Access Layer Units (NALUs). They are embedded by being muxed into the elementary video stream. See https://www.closedcaptioncreator.com/blog/articles/how-to-embed-608-708-closed-captioning.html

Even though embedded captions may appear as a separate data track in a media container, they are embedded in the elementary video bitstream.

Tom

On 2/7/25, 10:55 AM, "ffmpeg-devel on behalf of Soft Works" <ffmpeg-devel-bounces@ffmpeg.org <mailto:ffmpeg-devel-bounces@ffmpeg.org> on behalf of softworkz-at-hotmail.com@ffmpeg.org <mailto:softworkz-at-hotmail.com@ffmpeg.org>> wrote:

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org <mailto:ffmpeg-devel-bounces@ffmpeg.org>> On Behalf Of Tom
> Vaughan
> Sent: Friday, February 7, 2025 7:36 PM
> To: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org <mailto:devel@ffmpeg.org>>
> Subject: Re: [FFmpeg-devel] Captions SCC
>
> Soft Works said... "the CC data needs to get into the video frames -
> so that it exists as input to the video encoder.
> For the same reason, it is also not possible to add CCs at the muxer
> level. It always requires video encoding because the data needs to
> get into the video stream itself, so there's no way to get data from
> an SCC file into a video stream currently."


Hi Tom,
> You're confusing subtitles with closed captions. 
No.

> Subtitles are text that is composited into the video.
No.
> CEA-708 closed captions are
> carried as metadata in SEI NAL units, muxed into the elementary video
> stream.
I wouldn't call this "muxing" but that's not worth a debate.
(you might want to spend a thought on why it's called "elementary stream")

I said "it is also not possible to add CCs at the muxer level".

By that I mean "muxing" in ffmpeg terminology which is unambiguous.
sw




> 
> On 2/6/25, 11:48 PM, "ffmpeg-devel on behalf of Soft Works" <ffmpeg-
> devel-bounces@ffmpeg.org <mailto:devel-bounces@ffmpeg.org> <mailto:ffmpeg-devel-bounces@ffmpeg.org <mailto:ffmpeg-devel-bounces@ffmpeg.org>> on
> behalf of softworkz-at-hotmail.com@ffmpeg.org <mailto:softworkz-at-hotmail.com@ffmpeg.org> <mailto:softworkz-at-
> hotmail.com@ffmpeg.org <mailto:hotmail.com@ffmpeg.org>>> wrote:
> 
> 
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org <mailto:ffmpeg-devel-bounces@ffmpeg.org> <mailto:ffmpeg-
> devel-bounces@ffmpeg.org <mailto:devel-bounces@ffmpeg.org>>> On Behalf Of
> > Devlist Archive
> > Sent: Friday, February 7, 2025 6:13 AM
> > To: FFmpeg development discussions and patches <ffmpeg-
> > devel@ffmpeg.org <mailto:devel@ffmpeg.org> <mailto:devel@ffmpeg.org <mailto:devel@ffmpeg.org>>>
> > Subject: Re: [FFmpeg-devel] Captions SCC
> >
> Hi Zack,
> that message from "Jack" had confused me for a moment, but on re-
> reading it appears to be an AI response.
> The content is total nonsense. There is no "SCC" encoder in ffmpeg,
> and if there was one, it wouldn't help much because the CC data needs
> to get into the video frames - so that it exists as input to the
> video encoder.
> For the same reason, it is also not possible to add CCs at the muxer
> level. It always requires video encoding because the data needs to
> get into the video stream itself, so there's no way to get data from
> an SCC file into a video stream currently.
> 
> The only way in the ffmpeg architecture to get CC data encoded would
> be at the filtering level. The Subtitle Filtering patchset that I
> have submitted a while ago has a "splitcc" filter which has video as
> input and a video plus a subtitle pin/pad at the output side.
> What would be needed is a reverse pendant like a "mergecc" filter
> with video and subtitles input - plus a CEA-608/708 encoder.
> 
> softworkz
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org <mailto:ffmpeg-devel@ffmpeg.org>
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel <https://ffmpeg.org/mailman/listinfo/ffmpeg-devel>
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org <mailto:ffmpeg-devel-request@ffmpeg.org> with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org <mailto:ffmpeg-devel@ffmpeg.org>
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel <https://ffmpeg.org/mailman/listinfo/ffmpeg-devel>


To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org <mailto:ffmpeg-devel-request@ffmpeg.org> with subject "unsubscribe".



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".