[FFmpeg-devel] [PATCH] movenc: Make hybrid_fragmented retain the fragmented form headers (PR #20734)

[FFmpeg-devel] [PATCH] movenc: Make hybrid_fragmented retain the fragmented form headers (PR #20734)

[Martin Storsjö via ffmpeg-devel]

PR #20734 opened by Martin Storsjö (mstorsjo)
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20734
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20734.patch

This makes the final file truly hybrid: Externally the file
is a regular, non-fragmented file, but internally, the fragmented
form also exists un-overwritten.

To make any use of that, first, the fragments need to be muxed in
a position independent form, i.e. with empty_moov+default_base_moof
(or the dash or cmaf meta-flags).

Making use of the fragmented form when the file is finalized is
not entirely obvious though. One can dump the contents of the
single mdat box, and get the fragmented form. (This is a neat
trick, but not something that anybody really is expected to
want to do.)

The main expected use case is accessing fragments in the form of
byte range segments, for e.g. HLS.

Previously, the start of the file would look like this:

```
- ftyp
- free
- moov
 - (moov contents)
```

After finalizing the file, it would look like this:

```
- ftyp
- free
- mdat (previously moov)
 - (moov contents)
```

In this form, the size and type of the original moov box were
overwritten, and the original moov contents is just leftover
as unused data in the mdat box.

To avoid this issue, the start of the file now looks like this:

```
- ftyp
- free
- free
 - ftyp
- moov
 - (moov contents)
```

The second, hidden ftyp box inside mdat, would normally never be
seen.

After finalizing, the difference is that the mdat box now is
extended to cover the ftyp and the whole moov including its header
(and all the following fragments).

I.e., the start of the file looks like this:

```
- ftyp
- free
- mdat
 - ftyp
 - moov
  - (moov contents)
```

This allows accessing the "ftyp+moov" pair sequentially as such,
with a byte range - this range is untouched when finalizing,
producing the same ftyp+moov pair both while writing, when the
file is fragmented, and after finalizing, when the file is
transformed to non-fragmented externally.

Note; the sequential two "free+free" boxes may look slightly
silly; it could be tempting to make the second one an mdat
from the get-go. However, some players of fragmented mp4 (in
particular, Apple's HLS player) bail out if the initialization
segment contains an mdat box - therefore, use a free box.

It could also be possible to use just one single free box with
8 bytes of padding at the start - but that would require more
changes to the finalization logic.

For a segmenting user of the muxer, the only unclarity is how
to determine the right byte range for the internal ftyp+moov
pair. Currently, this requires parsing the muxer output and skip
past anything up to the start of the non-empty free box.


From f68beed3cbec03b6282c0b8b185bb162a127349f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Martin=20Storsj=C3=B6?= <martin@martin.st>
Date: Fri, 27 Jun 2025 15:12:32 +0300
Subject: [PATCH] movenc: Make hybrid_fragmented retain the fragmented form
 headers

This makes the final file truly hybrid: Externally the file
is a regular, non-fragmented file, but internally, the fragmented
form also exists un-overwritten.

To make any use of that, first, the fragments need to be muxed in
a position independent form, i.e. with empty_moov+default_base_moof
(or the dash or cmaf meta-flags).

Making use of the fragmented form when the file is finalized is
not entirely obvious though. One can dump the contents of the
single mdat box, and get the fragmented form. (This is a neat
trick, but not something that anybody really is expected to
want to do.)

The main expected use case is accessing fragments in the form of
byte range segments, for e.g. HLS.

Previously, the start of the file would look like this:

- ftyp
- free
- moov
 - (moov contents)

After finalizing the file, it would look like this:

- ftyp
- free
- mdat (previously moov)
 - (moov contents)

In this form, the size and type of the original moov box were
overwritten, and the original moov contents is just leftover
as unused data in the mdat box.

To avoid this issue, the start of the file now looks like this:

- ftyp
- free
- free
 - ftyp
- moov
 - (moov contents)

The second, hidden ftyp box inside mdat, would normally never be
seen.

After finalizing, the difference is that the mdat box now is
extended to cover the ftyp and the whole moov including its header
(and all the following fragments).

I.e., the start of the file looks like this:

- ftyp
- free
- mdat
 - ftyp
 - moov
  - (moov contents)

This allows accessing the "ftyp+moov" pair sequentially as such,
with a byte range - this range is untouched when finalizing,
producing the same ftyp+moov pair both while writing, when the
file is fragmented, and after finalizing, when the file is
transformed to non-fragmented externally.

Note; the sequential two "free+free" boxes may look slightly
silly; it could be tempting to make the second one an mdat
from the get-go. However, some players of fragmented mp4 (in
particular, Apple's HLS player) bail out if the initialization
segment contains an mdat box - therefore, use a free box.

It could also be possible to use just one single free box with
8 bytes of padding at the start - but that would require more
changes to the finalization logic.

For a segmenting user of the muxer, the only unclarity is how
to determine the right byte range for the internal ftyp+moov
pair. Currently, this requires parsing the muxer output and skip
past anything up to the start of the non-empty free box.
---
 libavformat/movenc.c           | 16 +++++++++++++++-
 tests/fate/mov.mak             |  2 +-
 tests/ref/lavf/mov_hybrid_frag |  4 ++--
 3 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 218a285821..c9e2d31861 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -8397,6 +8397,20 @@ static int mov_write_header(AVFormatContext *s)
             avio_wb32(pb, 8); // placeholder for extended size field (64 bit)
             ffio_wfourcc(pb, mov->mode == MODE_MOV ? "wide" : "free");
             mov->mdat_pos = avio_tell(pb);
+            // The free/wide header that later will be converted into an
+            // mdat, covering the initial moov and all the fragments.
+            avio_wb32(pb, 0);
+            ffio_wfourcc(pb, mov->mode == MODE_MOV ? "wide" : "free");
+            // Write an ftyp atom, hidden in a free/wide. This is neither
+            // exposed while the file is written, as fragmented, nor when the
+            // file is finalized into non-fragmented form. However, this allows
+            // accessing a pristine, sequential ftyp+moov init segment, even
+            // after the file is finalized. It also allows dumping the whole
+            // contents of the mdat box, to get the fragmented form of the
+            // file.
+            if ((ret = mov_write_identification(pb, s)) < 0)
+                return ret;
+            update_size(pb, mov->mdat_pos);
         }
     } else if (mov->mode != MODE_AVIF) {
         if (mov->flags & FF_MOV_FLAG_FASTSTART)
@@ -8558,7 +8572,7 @@ static void mov_write_mdat_size(AVFormatContext *s)
         avio_seek(pb, mov->mdat_pos, SEEK_SET);
         avio_wb32(pb, mov->mdat_size + 8);
         if (mov->flags & FF_MOV_FLAG_HYBRID_FRAGMENTED)
-            ffio_wfourcc(pb, "mdat"); // overwrite the original moov into a mdat
+            ffio_wfourcc(pb, "mdat"); // overwrite the original free/wide into a mdat
     } else {
         /* overwrite 'wide' placeholder atom */
         avio_seek(pb, mov->mdat_pos - 8, SEEK_SET);
diff --git a/tests/fate/mov.mak b/tests/fate/mov.mak
index eb666c5865..1f2f589beb 100644
--- a/tests/fate/mov.mak
+++ b/tests/fate/mov.mak
@@ -92,7 +92,7 @@ fate-mov-frag-overlap: CMD = framemd5 -i $(TARGET_SAMPLES)/mov/frag_overlap.mp4
 
 fate-mov-mp4-frag-flush: CMD = md5 -f lavfi -i color=blue,format=rgb24,trim=duration=0.04 -f lavfi -i anullsrc,aformat=s16,atrim=duration=2 -c:v png -c:a pcm_s16le -movflags +empty_moov+hybrid_fragmented -frag_duration 1000000 -frag_interleave 1 -bitexact -f mp4
 fate-mov-mp4-frag-flush: CMP = oneline
-fate-mov-mp4-frag-flush: REF = a1ac687d15552505f3c01a43285f32d0
+fate-mov-mp4-frag-flush: REF = 48d833e4773f7542f65dadb446f8bf61
 FATE_MOV_FFMPEG-$(call ALLYES, LAVFI_INDEV COLOR_FILTER FORMAT_FILTER TRIM_FILTER \
                                ANULLSRC_FILTER AFORMAT_FILTER ATRIM_FILTER        \
                                WRAPPED_AVFRAME_DECODER PCM_S16LE_DECODER PCM_S16BE_DECODER \
diff --git a/tests/ref/lavf/mov_hybrid_frag b/tests/ref/lavf/mov_hybrid_frag
index 64f4e9085e..13cad9505a 100644
--- a/tests/ref/lavf/mov_hybrid_frag
+++ b/tests/ref/lavf/mov_hybrid_frag
@@ -1,3 +1,3 @@
-7f79311fa73287c093aa029f58b71476 *tests/data/lavf/lavf.mov_hybrid_frag
-358436 tests/data/lavf/lavf.mov_hybrid_frag
+b79a7ecf125aef1f97c8e9b7df7066a0 *tests/data/lavf/lavf.mov_hybrid_frag
+358464 tests/data/lavf/lavf.mov_hybrid_frag
 tests/data/lavf/lavf.mov_hybrid_frag CRC=0xbb2b949b
-- 
2.49.1

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org