Re: [FFmpeg-devel] Captions SCC

[Devlist Archive <devlist@rlb.org>]

At some point we will need to separate the embed/extract conversation from
the transcode conversation.  I have made a ticket over on Trac that covers
the information for the transcode issue.  Let's focus on what needs to
happen to be able to generate a valid CEA-608 .scc file and embed such a
file to mp4 on this thread.  The EP12 folder on that drive link contains
captions files generated by the content producer using CaptionsMaker.  That
captions maker .scc file behaves differently and fails to embed compared to
the premiere produced one.
Transcoding Ticket
https://trac.ffmpeg.org/ticket/11459#ticket
What bitstream filters are required to do this extraction/embedding of
captions information.  Can the subcc filter or some other existing
structure be the basis for implementing this work?  Is it better to aim
straight for 708 functionality or can we fix the existing 608 path to not
cause the issues it currently produces.

On Fri, Feb 7, 2025 at 12:24 PM Tom Vaughan <tom@tvaughan.com> wrote:

> I know what an elementary video stream is. I founded and led development
> of x265, which produces elementary video streams. Elementary video streams
> can contain SEI messages.
> It's common to refer to the integration of closed captions into an
> elementary video stream as multiplexing. For example, see
> https://doc.haivision.com/MakitoX4Enc/1.7/closed-captioning
>
> It is common for people in the industry to refer to burned-in (or open)
> captions as subtitles. Those are the only type of captions that would
> require a video encoder.
>
> Closed captions are pure metadata, encapsulated in Network Access Layer
> Units (NALUs). They are embedded by being muxed into the elementary video
> stream. See
> https://www.closedcaptioncreator.com/blog/articles/how-to-embed-608-708-closed-captioning.html
>
> Even though embedded captions may appear as a separate data track in a
> media container, they are embedded in the elementary video bitstream.
>
> Tom
>
> On 2/7/25, 10:55 AM, "ffmpeg-devel on behalf of Soft Works" <
> ffmpeg-devel-bounces@ffmpeg.org <mailto:ffmpeg-devel-bounces@ffmpeg.org>
> on behalf of softworkz-at-hotmail.com@ffmpeg.org <mailto:
> softworkz-at-hotmail.com@ffmpeg.org>> wrote:
>
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org <mailto:
> ffmpeg-devel-bounces@ffmpeg.org>> On Behalf Of Tom
> > Vaughan
> > Sent: Friday, February 7, 2025 7:36 PM
> > To: FFmpeg development discussions and patches <ffmpeg-
> > devel@ffmpeg.org <mailto:devel@ffmpeg.org>>
> > Subject: Re: [FFmpeg-devel] Captions SCC
> >
> > Soft Works said... "the CC data needs to get into the video frames -
> > so that it exists as input to the video encoder.
> > For the same reason, it is also not possible to add CCs at the muxer
> > level. It always requires video encoding because the data needs to
> > get into the video stream itself, so there's no way to get data from
> > an SCC file into a video stream currently."
>
>
> Hi Tom,
> > You're confusing subtitles with closed captions.
> No.
>
> > Subtitles are text that is composited into the video.
> No.
> > CEA-708 closed captions are
> > carried as metadata in SEI NAL units, muxed into the elementary video
> > stream.
> I wouldn't call this "muxing" but that's not worth a debate.
> (you might want to spend a thought on why it's called "elementary stream")
>
> I said "it is also not possible to add CCs at the muxer level".
>
> By that I mean "muxing" in ffmpeg terminology which is unambiguous.
> sw
>
>
>
>
> >
> > On 2/6/25, 11:48 PM, "ffmpeg-devel on behalf of Soft Works" <ffmpeg-
> > devel-bounces@ffmpeg.org <mailto:devel-bounces@ffmpeg.org> <mailto:
> ffmpeg-devel-bounces@ffmpeg.org <mailto:ffmpeg-devel-bounces@ffmpeg.org>>
> on
> > behalf of softworkz-at-hotmail.com@ffmpeg.org <mailto:
> softworkz-at-hotmail.com@ffmpeg.org> <mailto:softworkz-at-
> > hotmail.com@ffmpeg.org <mailto:hotmail.com@ffmpeg.org>>> wrote:
> >
> >
> > > -----Original Message-----
> > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org <mailto:
> ffmpeg-devel-bounces@ffmpeg.org> <mailto:ffmpeg-
> > devel-bounces@ffmpeg.org <mailto:devel-bounces@ffmpeg.org>>> On Behalf
> Of
> > > Devlist Archive
> > > Sent: Friday, February 7, 2025 6:13 AM
> > > To: FFmpeg development discussions and patches <ffmpeg-
> > > devel@ffmpeg.org <mailto:devel@ffmpeg.org> <mailto:devel@ffmpeg.org
> <mailto:devel@ffmpeg.org>>>
> > > Subject: Re: [FFmpeg-devel] Captions SCC
> > >
> > Hi Zack,
> > that message from "Jack" had confused me for a moment, but on re-
> > reading it appears to be an AI response.
> > The content is total nonsense. There is no "SCC" encoder in ffmpeg,
> > and if there was one, it wouldn't help much because the CC data needs
> > to get into the video frames - so that it exists as input to the
> > video encoder.
> > For the same reason, it is also not possible to add CCs at the muxer
> > level. It always requires video encoding because the data needs to
> > get into the video stream itself, so there's no way to get data from
> > an SCC file into a video stream currently.
> >
> > The only way in the ffmpeg architecture to get CC data encoded would
> > be at the filtering level. The Subtitle Filtering patchset that I
> > have submitted a while ago has a "splitcc" filter which has video as
> > input and a video plus a subtitle pin/pad at the output side.
> > What would be needed is a reverse pendant like a "mergecc" filter
> > with video and subtitles input - plus a CEA-608/708 encoder.
> >
> > softworkz
> >
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org <mailto:ffmpeg-devel@ffmpeg.org>
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel <
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel>
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request@ffmpeg.org <mailto:ffmpeg-devel-request@ffmpeg.org>
> with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org <mailto:ffmpeg-devel@ffmpeg.org>
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel <
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel>
>
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org <mailto:ffmpeg-devel-request@ffmpeg.org>
> with subject "unsubscribe".
>
>
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".