* [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: Prepend SPS/PPS to buffering period SEI @ 2024-07-03 21:05 Josh Allmann 2024-07-06 16:37 ` Michael Niedermayer 0 siblings, 1 reply; 15+ messages in thread From: Josh Allmann @ 2024-07-03 21:05 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Josh Allmann Encoders may emit a buffering period SEI without a corresponding SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. During Annex B conversion, this may result in the SPS/PPS being inserted *after* the buffering period SEI but before the IDR NAL. Since the buffering period SEI references the SPS, the SPS/PPS needs to come first. --- libavcodec/bsf/h264_mp4toannexb.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/libavcodec/bsf/h264_mp4toannexb.c b/libavcodec/bsf/h264_mp4toannexb.c index 92af6a6881..6607f1e91a 100644 --- a/libavcodec/bsf/h264_mp4toannexb.c +++ b/libavcodec/bsf/h264_mp4toannexb.c @@ -363,6 +363,19 @@ static int h264_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *opkt) if (!new_idr && unit_type == H264_NAL_IDR_SLICE && (buf[1] & 0x80)) new_idr = 1; + /* If this is a buffering period SEI without a corresponding sps/pps + * then prepend any existing sps/pps before the SEI */ + if (unit_type == H264_NAL_SEI && buf[1] == 0 && !sps_seen && !pps_seen) { + if (s->sps_size) { + count_or_copy(&out, &out_size, s->sps, s->sps_size, PS_OUT_OF_BAND, j); + sps_seen = 1; + } + if (s->pps_size) { + count_or_copy(&out, &out_size, s->pps, s->pps_size, PS_OUT_OF_BAND, j); + pps_seen = 1; + } + } + /* prepend only to the first type 5 NAL unit of an IDR picture, if no sps/pps are already present */ if (new_idr && unit_type == H264_NAL_IDR_SLICE && !sps_seen && !pps_seen) { if (s->sps_size) -- 2.39.2 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: Prepend SPS/PPS to buffering period SEI 2024-07-03 21:05 [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: Prepend SPS/PPS to buffering period SEI Josh Allmann @ 2024-07-06 16:37 ` Michael Niedermayer 2024-07-08 22:06 ` Josh Allmann 0 siblings, 1 reply; 15+ messages in thread From: Michael Niedermayer @ 2024-07-06 16:37 UTC (permalink / raw) To: FFmpeg development discussions and patches [-- Attachment #1.1: Type: text/plain, Size: 1168 bytes --] On Wed, Jul 03, 2024 at 02:05:06PM -0700, Josh Allmann wrote: > Encoders may emit a buffering period SEI without a corresponding > SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. > > During Annex B conversion, this may result in the SPS/PPS being > inserted *after* the buffering period SEI but before the IDR NAL. > > Since the buffering period SEI references the SPS, the SPS/PPS > needs to come first. > --- > libavcodec/bsf/h264_mp4toannexb.c | 13 +++++++++++++ > 1 file changed, 13 insertions(+) breaks fate TEST h264-bsf-mp4toannexb --- ./tests/ref/fate/h264-bsf-mp4toannexb 2024-07-01 23:30:40.656213791 +0200 +++ tests/data/fate/h264-bsf-mp4toannexb 2024-07-06 12:13:56.491072296 +0200 @@ -1 +1 @@ -5f04c27cc6ee8625fe2405fb0f7da9a3 +ff2551123909f54c382294baa1bb4364 Test h264-bsf-mp4toannexb failed. Look at tests/data/fate/h264-bsf-mp4toannexb.err for details. make: *** [tests/Makefile:311: fate-h264-bsf-mp4toannexb] Error 1 [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB I am the wisest man alive, for I know one thing, and that is that I know nothing. -- Socrates [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] [-- Attachment #2: Type: text/plain, Size: 251 bytes --] _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: Prepend SPS/PPS to buffering period SEI 2024-07-06 16:37 ` Michael Niedermayer @ 2024-07-08 22:06 ` Josh Allmann 2024-07-09 19:05 ` Josh Allmann 0 siblings, 1 reply; 15+ messages in thread From: Josh Allmann @ 2024-07-08 22:06 UTC (permalink / raw) To: FFmpeg development discussions and patches On Sat, 6 Jul 2024 at 09:37, Michael Niedermayer <michael@niedermayer.cc> wrote: > > On Wed, Jul 03, 2024 at 02:05:06PM -0700, Josh Allmann wrote: > > Encoders may emit a buffering period SEI without a corresponding > > SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. > > > > During Annex B conversion, this may result in the SPS/PPS being > > inserted *after* the buffering period SEI but before the IDR NAL. > > > > Since the buffering period SEI references the SPS, the SPS/PPS > > needs to come first. > > --- > > libavcodec/bsf/h264_mp4toannexb.c | 13 +++++++++++++ > > 1 file changed, 13 insertions(+) > > breaks fate > > TEST h264-bsf-mp4toannexb > --- ./tests/ref/fate/h264-bsf-mp4toannexb 2024-07-01 23:30:40.656213791 +0200 > +++ tests/data/fate/h264-bsf-mp4toannexb 2024-07-06 12:13:56.491072296 +0200 > @@ -1 +1 @@ > -5f04c27cc6ee8625fe2405fb0f7da9a3 > +ff2551123909f54c382294baa1bb4364 > Test h264-bsf-mp4toannexb failed. Look at tests/data/fate/h264-bsf-mp4toannexb.err for details. > make: *** [tests/Makefile:311: fate-h264-bsf-mp4toannexb] Error 1 > Thanks for the heads up. Took a look into each of the failing fate tests from [0] and I think these changes are expected and OK. Each of the failing tests shows the problem that this patch fixes, which is that the SPS/PPS is written after the buffering period SEI. An easy way to eyeball the issue is that probing the Annex B output logs an error which says "non-existing SPS 0 referenced in buffering period" - in fact this is why I wrote this patch, to get to the bottom of that message. The NAL ordering can also be inspected with `bsf:v trace_headers`. There also seems to be a side benefit that makes segmenting more robust - details below. Some notes on each failing test: fate-segment-mp4-to-ts : Before this patch, the segments produced after 000.ts are not independently decodable, because only the first segment has any extradata [1]. After the patch, the segments can be decoded independently. Unless the intent of the test is to actually produce broken segments, this patch is probably correct in fixing that. Also see the `fate-h264-bsf-mp4toannexb` testcase. fate-h264-bsf-mp4toannexb : In the original version, the SPS/PPS is only written once - maybe because there are no true IDRs after the first frame, only recovery point SEIs. In the patched version, the SPS/PPS is written before each buffering period SEI, 6 times in total. I can see how this might be strictly unnecessary, but probably harmless from a spec standpoint. Also it seems to make segmented muxing more robust, since this testcase shares the same input as `fate-segment-mp4-to-ts` which produces broken segments without the patch. fate-h264_mp4toannexb_ticket2991 : This is a clean example of the problem that this patch solves: without it, a buffering period SEI comes before the SPS/PPS. Inspecting a diff of the NAL units [2], the only change is in the reordering of the NALs such that the SPS/PPS comes before the buffering period SEI, rather than after. If all that seems OK, then I can send another patch to update the fate references to match the new values. [0] https://patchwork.ffmpeg.org/check/104951/ [1] The first segment has extradata, but it is still in the wrong order without the patch. [2] https://gist.github.com/j0sh/c912056138822c4d8c9564f4062e1e7b Josh _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 15+ messages in thread
* [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: Prepend SPS/PPS to buffering period SEI 2024-07-08 22:06 ` Josh Allmann @ 2024-07-09 19:05 ` Josh Allmann 2024-07-15 17:48 ` Josh Allmann ` (2 more replies) 0 siblings, 3 replies; 15+ messages in thread From: Josh Allmann @ 2024-07-09 19:05 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Josh Allmann Encoders may emit a buffering period SEI without a corresponding SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. During Annex B conversion, this may result in the SPS/PPS being inserted *after* the buffering period SEI but before the IDR NAL. Since the buffering period SEI references the SPS, the SPS/PPS needs to come first. --- Notes: v2: Updated FATE test refs libavcodec/bsf/h264_mp4toannexb.c | 13 +++++++++++++ tests/ref/fate/h264-bsf-mp4toannexb | 2 +- tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ 4 files changed, 29 insertions(+), 16 deletions(-) diff --git a/libavcodec/bsf/h264_mp4toannexb.c b/libavcodec/bsf/h264_mp4toannexb.c index 92af6a6881..6607f1e91a 100644 --- a/libavcodec/bsf/h264_mp4toannexb.c +++ b/libavcodec/bsf/h264_mp4toannexb.c @@ -363,6 +363,19 @@ static int h264_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *opkt) if (!new_idr && unit_type == H264_NAL_IDR_SLICE && (buf[1] & 0x80)) new_idr = 1; + /* If this is a buffering period SEI without a corresponding sps/pps + * then prepend any existing sps/pps before the SEI */ + if (unit_type == H264_NAL_SEI && buf[1] == 0 && !sps_seen && !pps_seen) { + if (s->sps_size) { + count_or_copy(&out, &out_size, s->sps, s->sps_size, PS_OUT_OF_BAND, j); + sps_seen = 1; + } + if (s->pps_size) { + count_or_copy(&out, &out_size, s->pps, s->pps_size, PS_OUT_OF_BAND, j); + pps_seen = 1; + } + } + /* prepend only to the first type 5 NAL unit of an IDR picture, if no sps/pps are already present */ if (new_idr && unit_type == H264_NAL_IDR_SLICE && !sps_seen && !pps_seen) { if (s->sps_size) diff --git a/tests/ref/fate/h264-bsf-mp4toannexb b/tests/ref/fate/h264-bsf-mp4toannexb index 2049f39701..81ff568f3d 100644 --- a/tests/ref/fate/h264-bsf-mp4toannexb +++ b/tests/ref/fate/h264-bsf-mp4toannexb @@ -1 +1 @@ -5f04c27cc6ee8625fe2405fb0f7da9a3 +ff2551123909f54c382294baa1bb4364 diff --git a/tests/ref/fate/h264_mp4toannexb_ticket2991 b/tests/ref/fate/h264_mp4toannexb_ticket2991 index f8e3e920d4..9a1fbf2f8c 100644 --- a/tests/ref/fate/h264_mp4toannexb_ticket2991 +++ b/tests/ref/fate/h264_mp4toannexb_ticket2991 @@ -1,4 +1,4 @@ -05d66e60ab22ee004720e0051af0fe74 *tests/data/fate/h264_mp4toannexb_ticket2991.h264 +b6ff5910928ad0b2a7eec481dcc41594 *tests/data/fate/h264_mp4toannexb_ticket2991.h264 1985815 tests/data/fate/h264_mp4toannexb_ticket2991.h264 #extradata 0: 47, 0x3a590d55 #tb 0: 1/1200000 @@ -6,7 +6,7 @@ #codec_id 0: h264 #dimensions 0: 1280x720 #sar 0: 3/4 -0, 0, 0, 40040, 37126, 0xb020184c +0, 0, 0, 40040, 37126, 0x515c184c 0, 40040, 40040, 40040, 6920, 0x8512361a, F=0x0 0, 80081, 80081, 40040, 7550, 0x1bc56ed4, F=0x0 0, 120121, 120121, 40040, 8752, 0xb8c6f0a1, F=0x0 @@ -21,7 +21,7 @@ 0, 480485, 480485, 40040, 11234, 0x83cbd9fd, F=0x0 0, 520525, 520525, 40040, 17616, 0xfdf95104, F=0x0 0, 560566, 560566, 40040, 10689, 0x9633d32b, F=0x0 -0, 600606, 600606, 40040, 45291, 0x543c2cf6 +0, 600606, 600606, 40040, 45291, 0xa8292cf6 0, 640646, 640646, 40040, 20837, 0x051abfab, F=0x0 0, 680687, 680687, 40040, 21418, 0xe2a59d70, F=0x0 0, 720727, 720727, 40040, 15643, 0x15cf2cec, F=0x0 @@ -36,7 +36,7 @@ 0, 1081091, 1081091, 40040, 13130, 0xcbb6bb8e, F=0x0 0, 1121131, 1121131, 40040, 16180, 0x5d188a7a, F=0x0 0, 1161172, 1161172, 40040, 14961, 0x9ff2f463, F=0x0 -0, 1201212, 1201212, 40040, 54296, 0xe6ec30ed +0, 1201212, 1201212, 40040, 54296, 0x3ae830ed 0, 1241252, 1241252, 40040, 11500, 0x8c4852c9, F=0x0 0, 1281293, 1281293, 40040, 12065, 0xfb7954c3, F=0x0 0, 1321333, 1321333, 40040, 12532, 0xf0a935d3, F=0x0 @@ -51,7 +51,7 @@ 0, 1681697, 1681697, 40040, 13250, 0xfed0deb8, F=0x0 0, 1721737, 1721737, 40040, 13360, 0xbf92d476, F=0x0 0, 1761778, 1761778, 40040, 11749, 0x3041eaf1, F=0x0 -0, 1801818, 1801818, 40040, 23997, 0xdbe6d5c4 +0, 1801818, 1801818, 40040, 23997, 0x2fe2d5c4 0, 1841858, 1841858, 40040, 16065, 0xe8f715b7, F=0x0 0, 1881899, 1881899, 40040, 16441, 0x0a4e060f, F=0x0 0, 1921939, 1921939, 40040, 17395, 0xa8edecc2, F=0x0 @@ -66,7 +66,7 @@ 0, 2282303, 2282303, 40040, 13748, 0xed26aeb4, F=0x0 0, 2322343, 2322343, 40040, 15092, 0x3c983538, F=0x0 0, 2362384, 2362384, 40040, 14636, 0x9b278a6c, F=0x0 -0, 2402424, 2402424, 40040, 29134, 0xf784be18 +0, 2402424, 2402424, 40040, 29134, 0x4b80be18 0, 2442464, 2442464, 40040, 10232, 0x5408e15b, F=0x0 0, 2482505, 2482505, 40040, 9769, 0xc93cb7f9, F=0x0 0, 2522545, 2522545, 40040, 14454, 0x45230dbe, F=0x0 @@ -81,7 +81,7 @@ 0, 2882909, 2882909, 40040, 14801, 0x40bae016, F=0x0 0, 2922949, 2922949, 40040, 17303, 0x9ce1fd31, F=0x0 0, 2962990, 2962990, 40040, 17678, 0x9bd66141, F=0x0 -0, 3003030, 3003030, 40040, 48672, 0x3215ce46 +0, 3003030, 3003030, 40040, 48672, 0x8602ce46 0, 3043070, 3043070, 40040, 11894, 0x12e1fece, F=0x0 0, 3083111, 3083111, 40040, 16514, 0xc57aed05, F=0x0 0, 3123151, 3123151, 40040, 13044, 0x61914fa0, F=0x0 @@ -96,7 +96,7 @@ 0, 3483515, 3483515, 40040, 12208, 0x81a587c0, F=0x0 0, 3523555, 3523555, 40040, 14709, 0x5dffbe04, F=0x0 0, 3563596, 3563596, 40040, 14390, 0xbfd1e041, F=0x0 -0, 3603636, 3603636, 40040, 37236, 0xe7f924b1 +0, 3603636, 3603636, 40040, 37236, 0x3bf524b1 0, 3643676, 3643676, 40040, 14056, 0x24714c7c, F=0x0 0, 3683717, 3683717, 40040, 19438, 0x0c50dcd5, F=0x0 0, 3723757, 3723757, 40040, 21728, 0x7eea4a11, F=0x0 @@ -111,7 +111,7 @@ 0, 4084121, 4084121, 40040, 16878, 0x98efbae2, F=0x0 0, 4124161, 4124161, 40040, 14685, 0x1bf78d65, F=0x0 0, 4164202, 4164202, 40040, 13127, 0x0b91881d, F=0x0 -0, 4204242, 4204242, 40040, 29390, 0xf6a5ed6b +0, 4204242, 4204242, 40040, 29390, 0x4aa1ed6b 0, 4244282, 4244282, 40040, 12576, 0xe9845ded, F=0x0 0, 4284323, 4284323, 40040, 12599, 0x96a79ab8, F=0x0 0, 4324363, 4324363, 40040, 16134, 0xb4c36d3f, F=0x0 diff --git a/tests/ref/fate/segment-mp4-to-ts b/tests/ref/fate/segment-mp4-to-ts index 2994416270..54b59a09cc 100644 --- a/tests/ref/fate/segment-mp4-to-ts +++ b/tests/ref/fate/segment-mp4-to-ts @@ -4,7 +4,7 @@ #codec_id 0: h264 #dimensions 0: 640x360 #sar 0: 1/1 -0, -7200, 0, 3600, 22630, 0x9b109541, S=1, 1 +0, -7200, 0, 3600, 22630, 0xee579541, S=1, 1 0, -3600, 14400, 3600, 4021, 0xbf7cdb02, F=0x0, S=1, 1 0, 0, 7200, 3600, 1096, 0x4f162690, F=0x0, S=1, 1 0, 3600, 3600, 3600, 687, 0x00394b95, F=0x0, S=1, 1 @@ -25,7 +25,7 @@ 0, 57600, 64800, 3600, 1182, 0xbe1a4847, F=0x0, S=1, 1 0, 61200, 61200, 3600, 809, 0x8d948a4e, F=0x0, S=1, 1 0, 64800, 68400, 3600, 656, 0x4fa03c2b, F=0x0, S=1, 1 -0, 68400, 86400, 3600, 26555, 0x5629b584, S=1, 1 +0, 68400, 86400, 3600, 26606, 0x0f75c37d, S=1, 1 0, 72000, 79200, 3600, 1141, 0x761b31e8, F=0x0, S=1, 1 0, 75600, 75600, 3600, 717, 0x57746351, F=0x0, S=1, 1 0, 79200, 82800, 3600, 693, 0x78b24263, F=0x0, S=1, 1 @@ -49,7 +49,7 @@ 0, 144000, 151200, 3600, 1271, 0x46006870, F=0x0, S=1, 1 0, 147600, 147600, 3600, 849, 0x94dc99c7, F=0x2, S=1, 1 0, 151200, 154800, 3600, 753, 0xf4236cab, F=0x0, S=1, 1 -0, 154800, 172800, 3600, 25825, 0xd5464dee, S=1, 1 +0, 154800, 172800, 3600, 25876, 0xb4305be7, S=1, 1 0, 158400, 165600, 3600, 1206, 0x8ce84344, F=0x0, S=1, 1 0, 162000, 162000, 3600, 867, 0x312fa07d, F=0x0, S=1, 1 0, 165600, 169200, 3600, 719, 0x810666d1, F=0x0, S=1, 1 @@ -73,7 +73,7 @@ 0, 230400, 237600, 3600, 1545, 0x0099fc98, F=0x0, S=1, 1 0, 234000, 234000, 3600, 929, 0xfd72d049, F=0x2, S=1, 1 0, 237600, 241200, 3600, 829, 0xcfda9e96, F=0x0, S=1, 1 -0, 241200, 259200, 3600, 24220, 0x5ca21d71, S=1, 1 +0, 241200, 259200, 3600, 24271, 0x9c472b6a, S=1, 1 0, 244800, 252000, 3600, 1422, 0xcde6cc34, F=0x0, S=1, 1 0, 248400, 248400, 3600, 883, 0xedacbe25, F=0x0, S=1, 1 0, 252000, 255600, 3600, 768, 0x89d774bc, F=0x0, S=1, 1 @@ -97,7 +97,7 @@ 0, 316800, 324000, 3600, 1501, 0xb3b8f001, F=0x0, S=1, 1 0, 320400, 320400, 3600, 941, 0x92b0cb18, F=0x2, S=1, 1 0, 324000, 327600, 3600, 823, 0x3d548355, F=0x0, S=1, 1 -0, 327600, 345600, 3600, 24042, 0x441e94fb, S=1, 1 +0, 327600, 345600, 3600, 24093, 0xcc0ba2f4, S=1, 1 0, 331200, 338400, 3600, 1582, 0x4f5d1049, F=0x0, S=1, 1 0, 334800, 334800, 3600, 945, 0x4f3cc9e8, F=0x0, S=1, 1 0, 338400, 342000, 3600, 815, 0x0ca790a4, F=0x0, S=1, 1 @@ -121,7 +121,7 @@ 0, 403200, 410400, 3600, 359, 0x11bdae52, F=0x0, S=1, 1 0, 406800, 406800, 3600, 235, 0xbec26964, F=0x2, S=1, 1 0, 410400, 414000, 3600, 221, 0x8380682c, F=0x0, S=1, 1 -0, 414000, 432000, 3600, 22588, 0xf0ecf072, S=1, 1 +0, 414000, 432000, 3600, 22639, 0x1809fe6b, S=1, 1 0, 417600, 424800, 3600, 383, 0x4f3bb571, F=0x0, S=1, 1 0, 421200, 421200, 3600, 257, 0x22e87802, F=0x0, S=1, 1 0, 424800, 428400, 3600, 261, 0xdb988134, F=0x0, S=1, 1 -- 2.39.2 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: Prepend SPS/PPS to buffering period SEI 2024-07-09 19:05 ` Josh Allmann @ 2024-07-15 17:48 ` Josh Allmann 2024-07-15 17:54 ` Josh Allmann 2024-07-22 23:01 ` Josh Allmann 2024-08-01 7:58 ` Anton Khirnov 2 siblings, 1 reply; 15+ messages in thread From: Josh Allmann @ 2024-07-15 17:48 UTC (permalink / raw) To: ffmpeg-devel On Tue, 9 Jul 2024 at 12:06, Josh Allmann <joshua.allmann@gmail.com> wrote: > > Encoders may emit a buffering period SEI without a corresponding > SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. > > During Annex B conversion, this may result in the SPS/PPS being > inserted *after* the buffering period SEI but before the IDR NAL. > > Since the buffering period SEI references the SPS, the SPS/PPS > needs to come first. > --- > > Notes: > v2: Updated FATE test refs > > libavcodec/bsf/h264_mp4toannexb.c | 13 +++++++++++++ > tests/ref/fate/h264-bsf-mp4toannexb | 2 +- > tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- > tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ > 4 files changed, 29 insertions(+), 16 deletions(-) > Ping for review. Looking at the FATE output, this patch fixes a number of things - see [1] for details [1] https://ffmpeg.org//pipermail/ffmpeg-devel/2024-July/330964.html > diff --git a/libavcodec/bsf/h264_mp4toannexb.c b/libavcodec/bsf/h264_mp4toannexb.c > index 92af6a6881..6607f1e91a 100644 > --- a/libavcodec/bsf/h264_mp4toannexb.c > +++ b/libavcodec/bsf/h264_mp4toannexb.c > @@ -363,6 +363,19 @@ static int h264_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *opkt) > if (!new_idr && unit_type == H264_NAL_IDR_SLICE && (buf[1] & 0x80)) > new_idr = 1; > > + /* If this is a buffering period SEI without a corresponding sps/pps > + * then prepend any existing sps/pps before the SEI */ > + if (unit_type == H264_NAL_SEI && buf[1] == 0 && !sps_seen && !pps_seen) { > + if (s->sps_size) { > + count_or_copy(&out, &out_size, s->sps, s->sps_size, PS_OUT_OF_BAND, j); > + sps_seen = 1; > + } > + if (s->pps_size) { > + count_or_copy(&out, &out_size, s->pps, s->pps_size, PS_OUT_OF_BAND, j); > + pps_seen = 1; > + } > + } > + > /* prepend only to the first type 5 NAL unit of an IDR picture, if no sps/pps are already present */ > if (new_idr && unit_type == H264_NAL_IDR_SLICE && !sps_seen && !pps_seen) { > if (s->sps_size) > diff --git a/tests/ref/fate/h264-bsf-mp4toannexb b/tests/ref/fate/h264-bsf-mp4toannexb > index 2049f39701..81ff568f3d 100644 > --- a/tests/ref/fate/h264-bsf-mp4toannexb > +++ b/tests/ref/fate/h264-bsf-mp4toannexb > @@ -1 +1 @@ > -5f04c27cc6ee8625fe2405fb0f7da9a3 > +ff2551123909f54c382294baa1bb4364 > diff --git a/tests/ref/fate/h264_mp4toannexb_ticket2991 b/tests/ref/fate/h264_mp4toannexb_ticket2991 > index f8e3e920d4..9a1fbf2f8c 100644 > --- a/tests/ref/fate/h264_mp4toannexb_ticket2991 > +++ b/tests/ref/fate/h264_mp4toannexb_ticket2991 > @@ -1,4 +1,4 @@ > -05d66e60ab22ee004720e0051af0fe74 *tests/data/fate/h264_mp4toannexb_ticket2991.h264 > +b6ff5910928ad0b2a7eec481dcc41594 *tests/data/fate/h264_mp4toannexb_ticket2991.h264 > 1985815 tests/data/fate/h264_mp4toannexb_ticket2991.h264 > #extradata 0: 47, 0x3a590d55 > #tb 0: 1/1200000 > @@ -6,7 +6,7 @@ > #codec_id 0: h264 > #dimensions 0: 1280x720 > #sar 0: 3/4 > -0, 0, 0, 40040, 37126, 0xb020184c > +0, 0, 0, 40040, 37126, 0x515c184c > 0, 40040, 40040, 40040, 6920, 0x8512361a, F=0x0 > 0, 80081, 80081, 40040, 7550, 0x1bc56ed4, F=0x0 > 0, 120121, 120121, 40040, 8752, 0xb8c6f0a1, F=0x0 > @@ -21,7 +21,7 @@ > 0, 480485, 480485, 40040, 11234, 0x83cbd9fd, F=0x0 > 0, 520525, 520525, 40040, 17616, 0xfdf95104, F=0x0 > 0, 560566, 560566, 40040, 10689, 0x9633d32b, F=0x0 > -0, 600606, 600606, 40040, 45291, 0x543c2cf6 > +0, 600606, 600606, 40040, 45291, 0xa8292cf6 > 0, 640646, 640646, 40040, 20837, 0x051abfab, F=0x0 > 0, 680687, 680687, 40040, 21418, 0xe2a59d70, F=0x0 > 0, 720727, 720727, 40040, 15643, 0x15cf2cec, F=0x0 > @@ -36,7 +36,7 @@ > 0, 1081091, 1081091, 40040, 13130, 0xcbb6bb8e, F=0x0 > 0, 1121131, 1121131, 40040, 16180, 0x5d188a7a, F=0x0 > 0, 1161172, 1161172, 40040, 14961, 0x9ff2f463, F=0x0 > -0, 1201212, 1201212, 40040, 54296, 0xe6ec30ed > +0, 1201212, 1201212, 40040, 54296, 0x3ae830ed > 0, 1241252, 1241252, 40040, 11500, 0x8c4852c9, F=0x0 > 0, 1281293, 1281293, 40040, 12065, 0xfb7954c3, F=0x0 > 0, 1321333, 1321333, 40040, 12532, 0xf0a935d3, F=0x0 > @@ -51,7 +51,7 @@ > 0, 1681697, 1681697, 40040, 13250, 0xfed0deb8, F=0x0 > 0, 1721737, 1721737, 40040, 13360, 0xbf92d476, F=0x0 > 0, 1761778, 1761778, 40040, 11749, 0x3041eaf1, F=0x0 > -0, 1801818, 1801818, 40040, 23997, 0xdbe6d5c4 > +0, 1801818, 1801818, 40040, 23997, 0x2fe2d5c4 > 0, 1841858, 1841858, 40040, 16065, 0xe8f715b7, F=0x0 > 0, 1881899, 1881899, 40040, 16441, 0x0a4e060f, F=0x0 > 0, 1921939, 1921939, 40040, 17395, 0xa8edecc2, F=0x0 > @@ -66,7 +66,7 @@ > 0, 2282303, 2282303, 40040, 13748, 0xed26aeb4, F=0x0 > 0, 2322343, 2322343, 40040, 15092, 0x3c983538, F=0x0 > 0, 2362384, 2362384, 40040, 14636, 0x9b278a6c, F=0x0 > -0, 2402424, 2402424, 40040, 29134, 0xf784be18 > +0, 2402424, 2402424, 40040, 29134, 0x4b80be18 > 0, 2442464, 2442464, 40040, 10232, 0x5408e15b, F=0x0 > 0, 2482505, 2482505, 40040, 9769, 0xc93cb7f9, F=0x0 > 0, 2522545, 2522545, 40040, 14454, 0x45230dbe, F=0x0 > @@ -81,7 +81,7 @@ > 0, 2882909, 2882909, 40040, 14801, 0x40bae016, F=0x0 > 0, 2922949, 2922949, 40040, 17303, 0x9ce1fd31, F=0x0 > 0, 2962990, 2962990, 40040, 17678, 0x9bd66141, F=0x0 > -0, 3003030, 3003030, 40040, 48672, 0x3215ce46 > +0, 3003030, 3003030, 40040, 48672, 0x8602ce46 > 0, 3043070, 3043070, 40040, 11894, 0x12e1fece, F=0x0 > 0, 3083111, 3083111, 40040, 16514, 0xc57aed05, F=0x0 > 0, 3123151, 3123151, 40040, 13044, 0x61914fa0, F=0x0 > @@ -96,7 +96,7 @@ > 0, 3483515, 3483515, 40040, 12208, 0x81a587c0, F=0x0 > 0, 3523555, 3523555, 40040, 14709, 0x5dffbe04, F=0x0 > 0, 3563596, 3563596, 40040, 14390, 0xbfd1e041, F=0x0 > -0, 3603636, 3603636, 40040, 37236, 0xe7f924b1 > +0, 3603636, 3603636, 40040, 37236, 0x3bf524b1 > 0, 3643676, 3643676, 40040, 14056, 0x24714c7c, F=0x0 > 0, 3683717, 3683717, 40040, 19438, 0x0c50dcd5, F=0x0 > 0, 3723757, 3723757, 40040, 21728, 0x7eea4a11, F=0x0 > @@ -111,7 +111,7 @@ > 0, 4084121, 4084121, 40040, 16878, 0x98efbae2, F=0x0 > 0, 4124161, 4124161, 40040, 14685, 0x1bf78d65, F=0x0 > 0, 4164202, 4164202, 40040, 13127, 0x0b91881d, F=0x0 > -0, 4204242, 4204242, 40040, 29390, 0xf6a5ed6b > +0, 4204242, 4204242, 40040, 29390, 0x4aa1ed6b > 0, 4244282, 4244282, 40040, 12576, 0xe9845ded, F=0x0 > 0, 4284323, 4284323, 40040, 12599, 0x96a79ab8, F=0x0 > 0, 4324363, 4324363, 40040, 16134, 0xb4c36d3f, F=0x0 > diff --git a/tests/ref/fate/segment-mp4-to-ts b/tests/ref/fate/segment-mp4-to-ts > index 2994416270..54b59a09cc 100644 > --- a/tests/ref/fate/segment-mp4-to-ts > +++ b/tests/ref/fate/segment-mp4-to-ts > @@ -4,7 +4,7 @@ > #codec_id 0: h264 > #dimensions 0: 640x360 > #sar 0: 1/1 > -0, -7200, 0, 3600, 22630, 0x9b109541, S=1, 1 > +0, -7200, 0, 3600, 22630, 0xee579541, S=1, 1 > 0, -3600, 14400, 3600, 4021, 0xbf7cdb02, F=0x0, S=1, 1 > 0, 0, 7200, 3600, 1096, 0x4f162690, F=0x0, S=1, 1 > 0, 3600, 3600, 3600, 687, 0x00394b95, F=0x0, S=1, 1 > @@ -25,7 +25,7 @@ > 0, 57600, 64800, 3600, 1182, 0xbe1a4847, F=0x0, S=1, 1 > 0, 61200, 61200, 3600, 809, 0x8d948a4e, F=0x0, S=1, 1 > 0, 64800, 68400, 3600, 656, 0x4fa03c2b, F=0x0, S=1, 1 > -0, 68400, 86400, 3600, 26555, 0x5629b584, S=1, 1 > +0, 68400, 86400, 3600, 26606, 0x0f75c37d, S=1, 1 > 0, 72000, 79200, 3600, 1141, 0x761b31e8, F=0x0, S=1, 1 > 0, 75600, 75600, 3600, 717, 0x57746351, F=0x0, S=1, 1 > 0, 79200, 82800, 3600, 693, 0x78b24263, F=0x0, S=1, 1 > @@ -49,7 +49,7 @@ > 0, 144000, 151200, 3600, 1271, 0x46006870, F=0x0, S=1, 1 > 0, 147600, 147600, 3600, 849, 0x94dc99c7, F=0x2, S=1, 1 > 0, 151200, 154800, 3600, 753, 0xf4236cab, F=0x0, S=1, 1 > -0, 154800, 172800, 3600, 25825, 0xd5464dee, S=1, 1 > +0, 154800, 172800, 3600, 25876, 0xb4305be7, S=1, 1 > 0, 158400, 165600, 3600, 1206, 0x8ce84344, F=0x0, S=1, 1 > 0, 162000, 162000, 3600, 867, 0x312fa07d, F=0x0, S=1, 1 > 0, 165600, 169200, 3600, 719, 0x810666d1, F=0x0, S=1, 1 > @@ -73,7 +73,7 @@ > 0, 230400, 237600, 3600, 1545, 0x0099fc98, F=0x0, S=1, 1 > 0, 234000, 234000, 3600, 929, 0xfd72d049, F=0x2, S=1, 1 > 0, 237600, 241200, 3600, 829, 0xcfda9e96, F=0x0, S=1, 1 > -0, 241200, 259200, 3600, 24220, 0x5ca21d71, S=1, 1 > +0, 241200, 259200, 3600, 24271, 0x9c472b6a, S=1, 1 > 0, 244800, 252000, 3600, 1422, 0xcde6cc34, F=0x0, S=1, 1 > 0, 248400, 248400, 3600, 883, 0xedacbe25, F=0x0, S=1, 1 > 0, 252000, 255600, 3600, 768, 0x89d774bc, F=0x0, S=1, 1 > @@ -97,7 +97,7 @@ > 0, 316800, 324000, 3600, 1501, 0xb3b8f001, F=0x0, S=1, 1 > 0, 320400, 320400, 3600, 941, 0x92b0cb18, F=0x2, S=1, 1 > 0, 324000, 327600, 3600, 823, 0x3d548355, F=0x0, S=1, 1 > -0, 327600, 345600, 3600, 24042, 0x441e94fb, S=1, 1 > +0, 327600, 345600, 3600, 24093, 0xcc0ba2f4, S=1, 1 > 0, 331200, 338400, 3600, 1582, 0x4f5d1049, F=0x0, S=1, 1 > 0, 334800, 334800, 3600, 945, 0x4f3cc9e8, F=0x0, S=1, 1 > 0, 338400, 342000, 3600, 815, 0x0ca790a4, F=0x0, S=1, 1 > @@ -121,7 +121,7 @@ > 0, 403200, 410400, 3600, 359, 0x11bdae52, F=0x0, S=1, 1 > 0, 406800, 406800, 3600, 235, 0xbec26964, F=0x2, S=1, 1 > 0, 410400, 414000, 3600, 221, 0x8380682c, F=0x0, S=1, 1 > -0, 414000, 432000, 3600, 22588, 0xf0ecf072, S=1, 1 > +0, 414000, 432000, 3600, 22639, 0x1809fe6b, S=1, 1 > 0, 417600, 424800, 3600, 383, 0x4f3bb571, F=0x0, S=1, 1 > 0, 421200, 421200, 3600, 257, 0x22e87802, F=0x0, S=1, 1 > 0, 424800, 428400, 3600, 261, 0xdb988134, F=0x0, S=1, 1 > -- > 2.39.2 > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: Prepend SPS/PPS to buffering period SEI 2024-07-15 17:48 ` Josh Allmann @ 2024-07-15 17:54 ` Josh Allmann 0 siblings, 0 replies; 15+ messages in thread From: Josh Allmann @ 2024-07-15 17:54 UTC (permalink / raw) To: ffmpeg-devel On Mon, 15 Jul 2024 at 10:48, Josh Allmann <joshua.allmann@gmail.com> wrote: > > On Tue, 9 Jul 2024 at 12:06, Josh Allmann <joshua.allmann@gmail.com> wrote: > > > > Encoders may emit a buffering period SEI without a corresponding > > SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. > > > > During Annex B conversion, this may result in the SPS/PPS being > > inserted *after* the buffering period SEI but before the IDR NAL. > > > > Since the buffering period SEI references the SPS, the SPS/PPS > > needs to come first. > > --- > > > > Notes: > > v2: Updated FATE test refs > > > > libavcodec/bsf/h264_mp4toannexb.c | 13 +++++++++++++ > > tests/ref/fate/h264-bsf-mp4toannexb | 2 +- > > tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- > > tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ > > 4 files changed, 29 insertions(+), 16 deletions(-) > > > > Ping for review. Looking at the FATE output, this patch fixes a number > of things - see [1] for details > > [1] https://ffmpeg.org//pipermail/ffmpeg-devel/2024-July/330964.html Pasted the wrong link - please see this for a review of the FATE test changes https://ffmpeg.org//pipermail/ffmpeg-devel/2024-July/330912.html _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: Prepend SPS/PPS to buffering period SEI 2024-07-09 19:05 ` Josh Allmann 2024-07-15 17:48 ` Josh Allmann @ 2024-07-22 23:01 ` Josh Allmann 2024-07-23 0:16 ` Timo Rothenpieler 2024-08-01 7:58 ` Anton Khirnov 2 siblings, 1 reply; 15+ messages in thread From: Josh Allmann @ 2024-07-22 23:01 UTC (permalink / raw) To: ffmpeg-devel On Tue, 9 Jul 2024 at 12:06, Josh Allmann <joshua.allmann@gmail.com> wrote: > > Encoders may emit a buffering period SEI without a corresponding > SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. > > During Annex B conversion, this may result in the SPS/PPS being > inserted *after* the buffering period SEI but before the IDR NAL. > > Since the buffering period SEI references the SPS, the SPS/PPS > needs to come first. > --- > > Notes: > v2: Updated FATE test refs > > libavcodec/bsf/h264_mp4toannexb.c | 13 +++++++++++++ > tests/ref/fate/h264-bsf-mp4toannexb | 2 +- > tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- > tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ > 4 files changed, 29 insertions(+), 16 deletions(-) > Ping again for review. Looking at the FATE output, this patch fixes a number of things - see [1] for details [1] https://ffmpeg.org//pipermail/ffmpeg-devel/2024-July/330912.html _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: Prepend SPS/PPS to buffering period SEI 2024-07-22 23:01 ` Josh Allmann @ 2024-07-23 0:16 ` Timo Rothenpieler 2024-07-30 20:16 ` Josh Allmann 0 siblings, 1 reply; 15+ messages in thread From: Timo Rothenpieler @ 2024-07-23 0:16 UTC (permalink / raw) To: ffmpeg-devel On 23/07/2024 01:01, Josh Allmann wrote: > On Tue, 9 Jul 2024 at 12:06, Josh Allmann <joshua.allmann@gmail.com> wrote: >> >> Encoders may emit a buffering period SEI without a corresponding >> SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. >> >> During Annex B conversion, this may result in the SPS/PPS being >> inserted *after* the buffering period SEI but before the IDR NAL. >> >> Since the buffering period SEI references the SPS, the SPS/PPS >> needs to come first. >> --- >> >> Notes: >> v2: Updated FATE test refs >> >> libavcodec/bsf/h264_mp4toannexb.c | 13 +++++++++++++ >> tests/ref/fate/h264-bsf-mp4toannexb | 2 +- >> tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- >> tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ >> 4 files changed, 29 insertions(+), 16 deletions(-) >> > > Ping again for review. Looking at the FATE output, this patch fixes a number > of things - see [1] for details patch generally looks good to me, but I'm not closely familiar with the code there. _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: Prepend SPS/PPS to buffering period SEI 2024-07-23 0:16 ` Timo Rothenpieler @ 2024-07-30 20:16 ` Josh Allmann 0 siblings, 0 replies; 15+ messages in thread From: Josh Allmann @ 2024-07-30 20:16 UTC (permalink / raw) To: FFmpeg development discussions and patches On Mon, 22 Jul 2024 at 17:17, Timo Rothenpieler <timo@rothenpieler.org> wrote: > > On 23/07/2024 01:01, Josh Allmann wrote: > > On Tue, 9 Jul 2024 at 12:06, Josh Allmann <joshua.allmann@gmail.com> wrote: > >> > >> Encoders may emit a buffering period SEI without a corresponding > >> SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. > >> > >> During Annex B conversion, this may result in the SPS/PPS being > >> inserted *after* the buffering period SEI but before the IDR NAL. > >> > >> Since the buffering period SEI references the SPS, the SPS/PPS > >> needs to come first. > >> --- > >> > >> Notes: > >> v2: Updated FATE test refs > >> > >> libavcodec/bsf/h264_mp4toannexb.c | 13 +++++++++++++ > >> tests/ref/fate/h264-bsf-mp4toannexb | 2 +- > >> tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- > >> tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ > >> 4 files changed, 29 insertions(+), 16 deletions(-) > >> > > > > Ping again for review. Looking at the FATE output, this patch fixes a number > > of things - see [1] for details > > patch generally looks good to me, but I'm not closely familiar with the > code there. > Thanks, is there anyone else more familiar with the code who can also sign off on this patch? Josh _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: Prepend SPS/PPS to buffering period SEI 2024-07-09 19:05 ` Josh Allmann 2024-07-15 17:48 ` Josh Allmann 2024-07-22 23:01 ` Josh Allmann @ 2024-08-01 7:58 ` Anton Khirnov 2024-08-01 21:36 ` Josh Allmann 2024-08-01 21:45 ` Josh Allmann 2 siblings, 2 replies; 15+ messages in thread From: Anton Khirnov @ 2024-08-01 7:58 UTC (permalink / raw) To: FFmpeg development discussions and patches; +Cc: Josh Allmann Quoting Josh Allmann (2024-07-09 21:05:47) > Encoders may emit a buffering period SEI without a corresponding > SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. > > During Annex B conversion, this may result in the SPS/PPS being > inserted *after* the buffering period SEI but before the IDR NAL. > > Since the buffering period SEI references the SPS, the SPS/PPS > needs to come first. > --- > > Notes: > v2: Updated FATE test refs > > libavcodec/bsf/h264_mp4toannexb.c | 13 +++++++++++++ > tests/ref/fate/h264-bsf-mp4toannexb | 2 +- > tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- > tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ > 4 files changed, 29 insertions(+), 16 deletions(-) > > diff --git a/libavcodec/bsf/h264_mp4toannexb.c b/libavcodec/bsf/h264_mp4toannexb.c > index 92af6a6881..6607f1e91a 100644 > --- a/libavcodec/bsf/h264_mp4toannexb.c > +++ b/libavcodec/bsf/h264_mp4toannexb.c > @@ -363,6 +363,19 @@ static int h264_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *opkt) > if (!new_idr && unit_type == H264_NAL_IDR_SLICE && (buf[1] & 0x80)) > new_idr = 1; > > + /* If this is a buffering period SEI without a corresponding sps/pps > + * then prepend any existing sps/pps before the SEI */ > + if (unit_type == H264_NAL_SEI && buf[1] == 0 && !sps_seen && !pps_seen) { That 0 should be SEI_TYPE_BUFFERING_PERIOD, right? > + if (s->sps_size) { > + count_or_copy(&out, &out_size, s->sps, s->sps_size, PS_OUT_OF_BAND, j); > + sps_seen = 1; > + } > + if (s->pps_size) { > + count_or_copy(&out, &out_size, s->pps, s->pps_size, PS_OUT_OF_BAND, j); > + pps_seen = 1; > + } Is there a reason to insert the PPS? IIUC only the SPS is needed. -- Anton Khirnov _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 15+ messages in thread
* [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: Prepend SPS/PPS to buffering period SEI 2024-08-01 7:58 ` Anton Khirnov @ 2024-08-01 21:36 ` Josh Allmann 2024-08-07 16:13 ` Josh Allmann 2024-08-01 21:45 ` Josh Allmann 1 sibling, 1 reply; 15+ messages in thread From: Josh Allmann @ 2024-08-01 21:36 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Josh Allmann Encoders may emit a buffering period SEI without a corresponding SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. During Annex B conversion, this may result in the SPS/PPS being inserted *after* the buffering period SEI but before the IDR NAL. Since the buffering period SEI references the SPS, the SPS/PPS needs to come first. --- libavcodec/bsf/h264_mp4toannexb.c | 15 +++++++++++++++ tests/ref/fate/h264-bsf-mp4toannexb | 2 +- tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ 4 files changed, 31 insertions(+), 16 deletions(-) diff --git a/libavcodec/bsf/h264_mp4toannexb.c b/libavcodec/bsf/h264_mp4toannexb.c index 92af6a6881..dda064287e 100644 --- a/libavcodec/bsf/h264_mp4toannexb.c +++ b/libavcodec/bsf/h264_mp4toannexb.c @@ -30,6 +30,7 @@ #include "bytestream.h" #include "defs.h" #include "h264.h" +#include "sei.h" typedef struct H264BSFContext { uint8_t *sps; @@ -363,6 +364,20 @@ static int h264_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *opkt) if (!new_idr && unit_type == H264_NAL_IDR_SLICE && (buf[1] & 0x80)) new_idr = 1; + /* If this is a buffering period SEI without a corresponding sps/pps + * then prepend any existing sps/pps before the SEI */ + if (unit_type == H264_NAL_SEI && buf[1] == SEI_TYPE_BUFFERING_PERIOD && + !sps_seen && !pps_seen) { + if (s->sps_size) { + count_or_copy(&out, &out_size, s->sps, s->sps_size, PS_OUT_OF_BAND, j); + sps_seen = 1; + } + if (s->pps_size) { + count_or_copy(&out, &out_size, s->pps, s->pps_size, PS_OUT_OF_BAND, j); + pps_seen = 1; + } + } + /* prepend only to the first type 5 NAL unit of an IDR picture, if no sps/pps are already present */ if (new_idr && unit_type == H264_NAL_IDR_SLICE && !sps_seen && !pps_seen) { if (s->sps_size) diff --git a/tests/ref/fate/h264-bsf-mp4toannexb b/tests/ref/fate/h264-bsf-mp4toannexb index 2049f39701..81ff568f3d 100644 --- a/tests/ref/fate/h264-bsf-mp4toannexb +++ b/tests/ref/fate/h264-bsf-mp4toannexb @@ -1 +1 @@ -5f04c27cc6ee8625fe2405fb0f7da9a3 +ff2551123909f54c382294baa1bb4364 diff --git a/tests/ref/fate/h264_mp4toannexb_ticket2991 b/tests/ref/fate/h264_mp4toannexb_ticket2991 index f8e3e920d4..9a1fbf2f8c 100644 --- a/tests/ref/fate/h264_mp4toannexb_ticket2991 +++ b/tests/ref/fate/h264_mp4toannexb_ticket2991 @@ -1,4 +1,4 @@ -05d66e60ab22ee004720e0051af0fe74 *tests/data/fate/h264_mp4toannexb_ticket2991.h264 +b6ff5910928ad0b2a7eec481dcc41594 *tests/data/fate/h264_mp4toannexb_ticket2991.h264 1985815 tests/data/fate/h264_mp4toannexb_ticket2991.h264 #extradata 0: 47, 0x3a590d55 #tb 0: 1/1200000 @@ -6,7 +6,7 @@ #codec_id 0: h264 #dimensions 0: 1280x720 #sar 0: 3/4 -0, 0, 0, 40040, 37126, 0xb020184c +0, 0, 0, 40040, 37126, 0x515c184c 0, 40040, 40040, 40040, 6920, 0x8512361a, F=0x0 0, 80081, 80081, 40040, 7550, 0x1bc56ed4, F=0x0 0, 120121, 120121, 40040, 8752, 0xb8c6f0a1, F=0x0 @@ -21,7 +21,7 @@ 0, 480485, 480485, 40040, 11234, 0x83cbd9fd, F=0x0 0, 520525, 520525, 40040, 17616, 0xfdf95104, F=0x0 0, 560566, 560566, 40040, 10689, 0x9633d32b, F=0x0 -0, 600606, 600606, 40040, 45291, 0x543c2cf6 +0, 600606, 600606, 40040, 45291, 0xa8292cf6 0, 640646, 640646, 40040, 20837, 0x051abfab, F=0x0 0, 680687, 680687, 40040, 21418, 0xe2a59d70, F=0x0 0, 720727, 720727, 40040, 15643, 0x15cf2cec, F=0x0 @@ -36,7 +36,7 @@ 0, 1081091, 1081091, 40040, 13130, 0xcbb6bb8e, F=0x0 0, 1121131, 1121131, 40040, 16180, 0x5d188a7a, F=0x0 0, 1161172, 1161172, 40040, 14961, 0x9ff2f463, F=0x0 -0, 1201212, 1201212, 40040, 54296, 0xe6ec30ed +0, 1201212, 1201212, 40040, 54296, 0x3ae830ed 0, 1241252, 1241252, 40040, 11500, 0x8c4852c9, F=0x0 0, 1281293, 1281293, 40040, 12065, 0xfb7954c3, F=0x0 0, 1321333, 1321333, 40040, 12532, 0xf0a935d3, F=0x0 @@ -51,7 +51,7 @@ 0, 1681697, 1681697, 40040, 13250, 0xfed0deb8, F=0x0 0, 1721737, 1721737, 40040, 13360, 0xbf92d476, F=0x0 0, 1761778, 1761778, 40040, 11749, 0x3041eaf1, F=0x0 -0, 1801818, 1801818, 40040, 23997, 0xdbe6d5c4 +0, 1801818, 1801818, 40040, 23997, 0x2fe2d5c4 0, 1841858, 1841858, 40040, 16065, 0xe8f715b7, F=0x0 0, 1881899, 1881899, 40040, 16441, 0x0a4e060f, F=0x0 0, 1921939, 1921939, 40040, 17395, 0xa8edecc2, F=0x0 @@ -66,7 +66,7 @@ 0, 2282303, 2282303, 40040, 13748, 0xed26aeb4, F=0x0 0, 2322343, 2322343, 40040, 15092, 0x3c983538, F=0x0 0, 2362384, 2362384, 40040, 14636, 0x9b278a6c, F=0x0 -0, 2402424, 2402424, 40040, 29134, 0xf784be18 +0, 2402424, 2402424, 40040, 29134, 0x4b80be18 0, 2442464, 2442464, 40040, 10232, 0x5408e15b, F=0x0 0, 2482505, 2482505, 40040, 9769, 0xc93cb7f9, F=0x0 0, 2522545, 2522545, 40040, 14454, 0x45230dbe, F=0x0 @@ -81,7 +81,7 @@ 0, 2882909, 2882909, 40040, 14801, 0x40bae016, F=0x0 0, 2922949, 2922949, 40040, 17303, 0x9ce1fd31, F=0x0 0, 2962990, 2962990, 40040, 17678, 0x9bd66141, F=0x0 -0, 3003030, 3003030, 40040, 48672, 0x3215ce46 +0, 3003030, 3003030, 40040, 48672, 0x8602ce46 0, 3043070, 3043070, 40040, 11894, 0x12e1fece, F=0x0 0, 3083111, 3083111, 40040, 16514, 0xc57aed05, F=0x0 0, 3123151, 3123151, 40040, 13044, 0x61914fa0, F=0x0 @@ -96,7 +96,7 @@ 0, 3483515, 3483515, 40040, 12208, 0x81a587c0, F=0x0 0, 3523555, 3523555, 40040, 14709, 0x5dffbe04, F=0x0 0, 3563596, 3563596, 40040, 14390, 0xbfd1e041, F=0x0 -0, 3603636, 3603636, 40040, 37236, 0xe7f924b1 +0, 3603636, 3603636, 40040, 37236, 0x3bf524b1 0, 3643676, 3643676, 40040, 14056, 0x24714c7c, F=0x0 0, 3683717, 3683717, 40040, 19438, 0x0c50dcd5, F=0x0 0, 3723757, 3723757, 40040, 21728, 0x7eea4a11, F=0x0 @@ -111,7 +111,7 @@ 0, 4084121, 4084121, 40040, 16878, 0x98efbae2, F=0x0 0, 4124161, 4124161, 40040, 14685, 0x1bf78d65, F=0x0 0, 4164202, 4164202, 40040, 13127, 0x0b91881d, F=0x0 -0, 4204242, 4204242, 40040, 29390, 0xf6a5ed6b +0, 4204242, 4204242, 40040, 29390, 0x4aa1ed6b 0, 4244282, 4244282, 40040, 12576, 0xe9845ded, F=0x0 0, 4284323, 4284323, 40040, 12599, 0x96a79ab8, F=0x0 0, 4324363, 4324363, 40040, 16134, 0xb4c36d3f, F=0x0 diff --git a/tests/ref/fate/segment-mp4-to-ts b/tests/ref/fate/segment-mp4-to-ts index 2994416270..54b59a09cc 100644 --- a/tests/ref/fate/segment-mp4-to-ts +++ b/tests/ref/fate/segment-mp4-to-ts @@ -4,7 +4,7 @@ #codec_id 0: h264 #dimensions 0: 640x360 #sar 0: 1/1 -0, -7200, 0, 3600, 22630, 0x9b109541, S=1, 1 +0, -7200, 0, 3600, 22630, 0xee579541, S=1, 1 0, -3600, 14400, 3600, 4021, 0xbf7cdb02, F=0x0, S=1, 1 0, 0, 7200, 3600, 1096, 0x4f162690, F=0x0, S=1, 1 0, 3600, 3600, 3600, 687, 0x00394b95, F=0x0, S=1, 1 @@ -25,7 +25,7 @@ 0, 57600, 64800, 3600, 1182, 0xbe1a4847, F=0x0, S=1, 1 0, 61200, 61200, 3600, 809, 0x8d948a4e, F=0x0, S=1, 1 0, 64800, 68400, 3600, 656, 0x4fa03c2b, F=0x0, S=1, 1 -0, 68400, 86400, 3600, 26555, 0x5629b584, S=1, 1 +0, 68400, 86400, 3600, 26606, 0x0f75c37d, S=1, 1 0, 72000, 79200, 3600, 1141, 0x761b31e8, F=0x0, S=1, 1 0, 75600, 75600, 3600, 717, 0x57746351, F=0x0, S=1, 1 0, 79200, 82800, 3600, 693, 0x78b24263, F=0x0, S=1, 1 @@ -49,7 +49,7 @@ 0, 144000, 151200, 3600, 1271, 0x46006870, F=0x0, S=1, 1 0, 147600, 147600, 3600, 849, 0x94dc99c7, F=0x2, S=1, 1 0, 151200, 154800, 3600, 753, 0xf4236cab, F=0x0, S=1, 1 -0, 154800, 172800, 3600, 25825, 0xd5464dee, S=1, 1 +0, 154800, 172800, 3600, 25876, 0xb4305be7, S=1, 1 0, 158400, 165600, 3600, 1206, 0x8ce84344, F=0x0, S=1, 1 0, 162000, 162000, 3600, 867, 0x312fa07d, F=0x0, S=1, 1 0, 165600, 169200, 3600, 719, 0x810666d1, F=0x0, S=1, 1 @@ -73,7 +73,7 @@ 0, 230400, 237600, 3600, 1545, 0x0099fc98, F=0x0, S=1, 1 0, 234000, 234000, 3600, 929, 0xfd72d049, F=0x2, S=1, 1 0, 237600, 241200, 3600, 829, 0xcfda9e96, F=0x0, S=1, 1 -0, 241200, 259200, 3600, 24220, 0x5ca21d71, S=1, 1 +0, 241200, 259200, 3600, 24271, 0x9c472b6a, S=1, 1 0, 244800, 252000, 3600, 1422, 0xcde6cc34, F=0x0, S=1, 1 0, 248400, 248400, 3600, 883, 0xedacbe25, F=0x0, S=1, 1 0, 252000, 255600, 3600, 768, 0x89d774bc, F=0x0, S=1, 1 @@ -97,7 +97,7 @@ 0, 316800, 324000, 3600, 1501, 0xb3b8f001, F=0x0, S=1, 1 0, 320400, 320400, 3600, 941, 0x92b0cb18, F=0x2, S=1, 1 0, 324000, 327600, 3600, 823, 0x3d548355, F=0x0, S=1, 1 -0, 327600, 345600, 3600, 24042, 0x441e94fb, S=1, 1 +0, 327600, 345600, 3600, 24093, 0xcc0ba2f4, S=1, 1 0, 331200, 338400, 3600, 1582, 0x4f5d1049, F=0x0, S=1, 1 0, 334800, 334800, 3600, 945, 0x4f3cc9e8, F=0x0, S=1, 1 0, 338400, 342000, 3600, 815, 0x0ca790a4, F=0x0, S=1, 1 @@ -121,7 +121,7 @@ 0, 403200, 410400, 3600, 359, 0x11bdae52, F=0x0, S=1, 1 0, 406800, 406800, 3600, 235, 0xbec26964, F=0x2, S=1, 1 0, 410400, 414000, 3600, 221, 0x8380682c, F=0x0, S=1, 1 -0, 414000, 432000, 3600, 22588, 0xf0ecf072, S=1, 1 +0, 414000, 432000, 3600, 22639, 0x1809fe6b, S=1, 1 0, 417600, 424800, 3600, 383, 0x4f3bb571, F=0x0, S=1, 1 0, 421200, 421200, 3600, 257, 0x22e87802, F=0x0, S=1, 1 0, 424800, 428400, 3600, 261, 0xdb988134, F=0x0, S=1, 1 -- 2.39.2 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: Prepend SPS/PPS to buffering period SEI 2024-08-01 21:36 ` Josh Allmann @ 2024-08-07 16:13 ` Josh Allmann 2024-08-13 16:57 ` Josh Allmann 0 siblings, 1 reply; 15+ messages in thread From: Josh Allmann @ 2024-08-07 16:13 UTC (permalink / raw) To: ffmpeg-devel On Thu, 1 Aug 2024 at 14:37, Josh Allmann <joshua.allmann@gmail.com> wrote: > > Encoders may emit a buffering period SEI without a corresponding > SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. > > During Annex B conversion, this may result in the SPS/PPS being > inserted *after* the buffering period SEI but before the IDR NAL. > > Since the buffering period SEI references the SPS, the SPS/PPS > needs to come first. > --- > libavcodec/bsf/h264_mp4toannexb.c | 15 +++++++++++++++ > tests/ref/fate/h264-bsf-mp4toannexb | 2 +- > tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- > tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ > 4 files changed, 31 insertions(+), 16 deletions(-) > Ping for (re-)review on this patch which addresses comments from [1] Explanation for the FATE changes here [2] - it turns out that several of the FATE samples exhibit the same behavior that this patch fixes, so it is a net improvement [1] https://ffmpeg.org//pipermail/ffmpeg-devel/2024-August/331958.html [2] https://ffmpeg.org//pipermail/ffmpeg-devel/2024-July/330912.html Josh > diff --git a/libavcodec/bsf/h264_mp4toannexb.c b/libavcodec/bsf/h264_mp4toannexb.c > index 92af6a6881..dda064287e 100644 > --- a/libavcodec/bsf/h264_mp4toannexb.c > +++ b/libavcodec/bsf/h264_mp4toannexb.c > @@ -30,6 +30,7 @@ > #include "bytestream.h" > #include "defs.h" > #include "h264.h" > +#include "sei.h" > > typedef struct H264BSFContext { > uint8_t *sps; > @@ -363,6 +364,20 @@ static int h264_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *opkt) > if (!new_idr && unit_type == H264_NAL_IDR_SLICE && (buf[1] & 0x80)) > new_idr = 1; > > + /* If this is a buffering period SEI without a corresponding sps/pps > + * then prepend any existing sps/pps before the SEI */ > + if (unit_type == H264_NAL_SEI && buf[1] == SEI_TYPE_BUFFERING_PERIOD && > + !sps_seen && !pps_seen) { > + if (s->sps_size) { > + count_or_copy(&out, &out_size, s->sps, s->sps_size, PS_OUT_OF_BAND, j); > + sps_seen = 1; > + } > + if (s->pps_size) { > + count_or_copy(&out, &out_size, s->pps, s->pps_size, PS_OUT_OF_BAND, j); > + pps_seen = 1; > + } > + } > + > /* prepend only to the first type 5 NAL unit of an IDR picture, if no sps/pps are already present */ > if (new_idr && unit_type == H264_NAL_IDR_SLICE && !sps_seen && !pps_seen) { > if (s->sps_size) > diff --git a/tests/ref/fate/h264-bsf-mp4toannexb b/tests/ref/fate/h264-bsf-mp4toannexb > index 2049f39701..81ff568f3d 100644 > --- a/tests/ref/fate/h264-bsf-mp4toannexb > +++ b/tests/ref/fate/h264-bsf-mp4toannexb > @@ -1 +1 @@ > -5f04c27cc6ee8625fe2405fb0f7da9a3 > +ff2551123909f54c382294baa1bb4364 > diff --git a/tests/ref/fate/h264_mp4toannexb_ticket2991 b/tests/ref/fate/h264_mp4toannexb_ticket2991 > index f8e3e920d4..9a1fbf2f8c 100644 > --- a/tests/ref/fate/h264_mp4toannexb_ticket2991 > +++ b/tests/ref/fate/h264_mp4toannexb_ticket2991 > @@ -1,4 +1,4 @@ > -05d66e60ab22ee004720e0051af0fe74 *tests/data/fate/h264_mp4toannexb_ticket2991.h264 > +b6ff5910928ad0b2a7eec481dcc41594 *tests/data/fate/h264_mp4toannexb_ticket2991.h264 > 1985815 tests/data/fate/h264_mp4toannexb_ticket2991.h264 > #extradata 0: 47, 0x3a590d55 > #tb 0: 1/1200000 > @@ -6,7 +6,7 @@ > #codec_id 0: h264 > #dimensions 0: 1280x720 > #sar 0: 3/4 > -0, 0, 0, 40040, 37126, 0xb020184c > +0, 0, 0, 40040, 37126, 0x515c184c > 0, 40040, 40040, 40040, 6920, 0x8512361a, F=0x0 > 0, 80081, 80081, 40040, 7550, 0x1bc56ed4, F=0x0 > 0, 120121, 120121, 40040, 8752, 0xb8c6f0a1, F=0x0 > @@ -21,7 +21,7 @@ > 0, 480485, 480485, 40040, 11234, 0x83cbd9fd, F=0x0 > 0, 520525, 520525, 40040, 17616, 0xfdf95104, F=0x0 > 0, 560566, 560566, 40040, 10689, 0x9633d32b, F=0x0 > -0, 600606, 600606, 40040, 45291, 0x543c2cf6 > +0, 600606, 600606, 40040, 45291, 0xa8292cf6 > 0, 640646, 640646, 40040, 20837, 0x051abfab, F=0x0 > 0, 680687, 680687, 40040, 21418, 0xe2a59d70, F=0x0 > 0, 720727, 720727, 40040, 15643, 0x15cf2cec, F=0x0 > @@ -36,7 +36,7 @@ > 0, 1081091, 1081091, 40040, 13130, 0xcbb6bb8e, F=0x0 > 0, 1121131, 1121131, 40040, 16180, 0x5d188a7a, F=0x0 > 0, 1161172, 1161172, 40040, 14961, 0x9ff2f463, F=0x0 > -0, 1201212, 1201212, 40040, 54296, 0xe6ec30ed > +0, 1201212, 1201212, 40040, 54296, 0x3ae830ed > 0, 1241252, 1241252, 40040, 11500, 0x8c4852c9, F=0x0 > 0, 1281293, 1281293, 40040, 12065, 0xfb7954c3, F=0x0 > 0, 1321333, 1321333, 40040, 12532, 0xf0a935d3, F=0x0 > @@ -51,7 +51,7 @@ > 0, 1681697, 1681697, 40040, 13250, 0xfed0deb8, F=0x0 > 0, 1721737, 1721737, 40040, 13360, 0xbf92d476, F=0x0 > 0, 1761778, 1761778, 40040, 11749, 0x3041eaf1, F=0x0 > -0, 1801818, 1801818, 40040, 23997, 0xdbe6d5c4 > +0, 1801818, 1801818, 40040, 23997, 0x2fe2d5c4 > 0, 1841858, 1841858, 40040, 16065, 0xe8f715b7, F=0x0 > 0, 1881899, 1881899, 40040, 16441, 0x0a4e060f, F=0x0 > 0, 1921939, 1921939, 40040, 17395, 0xa8edecc2, F=0x0 > @@ -66,7 +66,7 @@ > 0, 2282303, 2282303, 40040, 13748, 0xed26aeb4, F=0x0 > 0, 2322343, 2322343, 40040, 15092, 0x3c983538, F=0x0 > 0, 2362384, 2362384, 40040, 14636, 0x9b278a6c, F=0x0 > -0, 2402424, 2402424, 40040, 29134, 0xf784be18 > +0, 2402424, 2402424, 40040, 29134, 0x4b80be18 > 0, 2442464, 2442464, 40040, 10232, 0x5408e15b, F=0x0 > 0, 2482505, 2482505, 40040, 9769, 0xc93cb7f9, F=0x0 > 0, 2522545, 2522545, 40040, 14454, 0x45230dbe, F=0x0 > @@ -81,7 +81,7 @@ > 0, 2882909, 2882909, 40040, 14801, 0x40bae016, F=0x0 > 0, 2922949, 2922949, 40040, 17303, 0x9ce1fd31, F=0x0 > 0, 2962990, 2962990, 40040, 17678, 0x9bd66141, F=0x0 > -0, 3003030, 3003030, 40040, 48672, 0x3215ce46 > +0, 3003030, 3003030, 40040, 48672, 0x8602ce46 > 0, 3043070, 3043070, 40040, 11894, 0x12e1fece, F=0x0 > 0, 3083111, 3083111, 40040, 16514, 0xc57aed05, F=0x0 > 0, 3123151, 3123151, 40040, 13044, 0x61914fa0, F=0x0 > @@ -96,7 +96,7 @@ > 0, 3483515, 3483515, 40040, 12208, 0x81a587c0, F=0x0 > 0, 3523555, 3523555, 40040, 14709, 0x5dffbe04, F=0x0 > 0, 3563596, 3563596, 40040, 14390, 0xbfd1e041, F=0x0 > -0, 3603636, 3603636, 40040, 37236, 0xe7f924b1 > +0, 3603636, 3603636, 40040, 37236, 0x3bf524b1 > 0, 3643676, 3643676, 40040, 14056, 0x24714c7c, F=0x0 > 0, 3683717, 3683717, 40040, 19438, 0x0c50dcd5, F=0x0 > 0, 3723757, 3723757, 40040, 21728, 0x7eea4a11, F=0x0 > @@ -111,7 +111,7 @@ > 0, 4084121, 4084121, 40040, 16878, 0x98efbae2, F=0x0 > 0, 4124161, 4124161, 40040, 14685, 0x1bf78d65, F=0x0 > 0, 4164202, 4164202, 40040, 13127, 0x0b91881d, F=0x0 > -0, 4204242, 4204242, 40040, 29390, 0xf6a5ed6b > +0, 4204242, 4204242, 40040, 29390, 0x4aa1ed6b > 0, 4244282, 4244282, 40040, 12576, 0xe9845ded, F=0x0 > 0, 4284323, 4284323, 40040, 12599, 0x96a79ab8, F=0x0 > 0, 4324363, 4324363, 40040, 16134, 0xb4c36d3f, F=0x0 > diff --git a/tests/ref/fate/segment-mp4-to-ts b/tests/ref/fate/segment-mp4-to-ts > index 2994416270..54b59a09cc 100644 > --- a/tests/ref/fate/segment-mp4-to-ts > +++ b/tests/ref/fate/segment-mp4-to-ts > @@ -4,7 +4,7 @@ > #codec_id 0: h264 > #dimensions 0: 640x360 > #sar 0: 1/1 > -0, -7200, 0, 3600, 22630, 0x9b109541, S=1, 1 > +0, -7200, 0, 3600, 22630, 0xee579541, S=1, 1 > 0, -3600, 14400, 3600, 4021, 0xbf7cdb02, F=0x0, S=1, 1 > 0, 0, 7200, 3600, 1096, 0x4f162690, F=0x0, S=1, 1 > 0, 3600, 3600, 3600, 687, 0x00394b95, F=0x0, S=1, 1 > @@ -25,7 +25,7 @@ > 0, 57600, 64800, 3600, 1182, 0xbe1a4847, F=0x0, S=1, 1 > 0, 61200, 61200, 3600, 809, 0x8d948a4e, F=0x0, S=1, 1 > 0, 64800, 68400, 3600, 656, 0x4fa03c2b, F=0x0, S=1, 1 > -0, 68400, 86400, 3600, 26555, 0x5629b584, S=1, 1 > +0, 68400, 86400, 3600, 26606, 0x0f75c37d, S=1, 1 > 0, 72000, 79200, 3600, 1141, 0x761b31e8, F=0x0, S=1, 1 > 0, 75600, 75600, 3600, 717, 0x57746351, F=0x0, S=1, 1 > 0, 79200, 82800, 3600, 693, 0x78b24263, F=0x0, S=1, 1 > @@ -49,7 +49,7 @@ > 0, 144000, 151200, 3600, 1271, 0x46006870, F=0x0, S=1, 1 > 0, 147600, 147600, 3600, 849, 0x94dc99c7, F=0x2, S=1, 1 > 0, 151200, 154800, 3600, 753, 0xf4236cab, F=0x0, S=1, 1 > -0, 154800, 172800, 3600, 25825, 0xd5464dee, S=1, 1 > +0, 154800, 172800, 3600, 25876, 0xb4305be7, S=1, 1 > 0, 158400, 165600, 3600, 1206, 0x8ce84344, F=0x0, S=1, 1 > 0, 162000, 162000, 3600, 867, 0x312fa07d, F=0x0, S=1, 1 > 0, 165600, 169200, 3600, 719, 0x810666d1, F=0x0, S=1, 1 > @@ -73,7 +73,7 @@ > 0, 230400, 237600, 3600, 1545, 0x0099fc98, F=0x0, S=1, 1 > 0, 234000, 234000, 3600, 929, 0xfd72d049, F=0x2, S=1, 1 > 0, 237600, 241200, 3600, 829, 0xcfda9e96, F=0x0, S=1, 1 > -0, 241200, 259200, 3600, 24220, 0x5ca21d71, S=1, 1 > +0, 241200, 259200, 3600, 24271, 0x9c472b6a, S=1, 1 > 0, 244800, 252000, 3600, 1422, 0xcde6cc34, F=0x0, S=1, 1 > 0, 248400, 248400, 3600, 883, 0xedacbe25, F=0x0, S=1, 1 > 0, 252000, 255600, 3600, 768, 0x89d774bc, F=0x0, S=1, 1 > @@ -97,7 +97,7 @@ > 0, 316800, 324000, 3600, 1501, 0xb3b8f001, F=0x0, S=1, 1 > 0, 320400, 320400, 3600, 941, 0x92b0cb18, F=0x2, S=1, 1 > 0, 324000, 327600, 3600, 823, 0x3d548355, F=0x0, S=1, 1 > -0, 327600, 345600, 3600, 24042, 0x441e94fb, S=1, 1 > +0, 327600, 345600, 3600, 24093, 0xcc0ba2f4, S=1, 1 > 0, 331200, 338400, 3600, 1582, 0x4f5d1049, F=0x0, S=1, 1 > 0, 334800, 334800, 3600, 945, 0x4f3cc9e8, F=0x0, S=1, 1 > 0, 338400, 342000, 3600, 815, 0x0ca790a4, F=0x0, S=1, 1 > @@ -121,7 +121,7 @@ > 0, 403200, 410400, 3600, 359, 0x11bdae52, F=0x0, S=1, 1 > 0, 406800, 406800, 3600, 235, 0xbec26964, F=0x2, S=1, 1 > 0, 410400, 414000, 3600, 221, 0x8380682c, F=0x0, S=1, 1 > -0, 414000, 432000, 3600, 22588, 0xf0ecf072, S=1, 1 > +0, 414000, 432000, 3600, 22639, 0x1809fe6b, S=1, 1 > 0, 417600, 424800, 3600, 383, 0x4f3bb571, F=0x0, S=1, 1 > 0, 421200, 421200, 3600, 257, 0x22e87802, F=0x0, S=1, 1 > 0, 424800, 428400, 3600, 261, 0xdb988134, F=0x0, S=1, 1 > -- > 2.39.2 > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: Prepend SPS/PPS to buffering period SEI 2024-08-07 16:13 ` Josh Allmann @ 2024-08-13 16:57 ` Josh Allmann 2024-08-14 11:45 ` Anton Khirnov 0 siblings, 1 reply; 15+ messages in thread From: Josh Allmann @ 2024-08-13 16:57 UTC (permalink / raw) To: ffmpeg-devel On Wed, 7 Aug 2024 at 09:13, Josh Allmann <joshua.allmann@gmail.com> wrote: > > On Thu, 1 Aug 2024 at 14:37, Josh Allmann <joshua.allmann@gmail.com> wrote: > > > > Encoders may emit a buffering period SEI without a corresponding > > SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. > > > > During Annex B conversion, this may result in the SPS/PPS being > > inserted *after* the buffering period SEI but before the IDR NAL. > > > > Since the buffering period SEI references the SPS, the SPS/PPS > > needs to come first. > > --- > > libavcodec/bsf/h264_mp4toannexb.c | 15 +++++++++++++++ > > tests/ref/fate/h264-bsf-mp4toannexb | 2 +- > > tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- > > tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ > > 4 files changed, 31 insertions(+), 16 deletions(-) > > > > Ping for (re-)review on this patch which addresses comments from [1] > > Explanation for the FATE changes here [2] - it turns out that several > of the FATE samples exhibit the same behavior that this patch fixes, > so it is a net improvement > > [1] https://ffmpeg.org//pipermail/ffmpeg-devel/2024-August/331958.html > [2] https://ffmpeg.org//pipermail/ffmpeg-devel/2024-July/330912.html > Gentle ping for re-review. Josh _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: Prepend SPS/PPS to buffering period SEI 2024-08-13 16:57 ` Josh Allmann @ 2024-08-14 11:45 ` Anton Khirnov 0 siblings, 0 replies; 15+ messages in thread From: Anton Khirnov @ 2024-08-14 11:45 UTC (permalink / raw) To: FFmpeg development discussions and patches Quoting Josh Allmann (2024-08-13 18:57:59) > On Wed, 7 Aug 2024 at 09:13, Josh Allmann <joshua.allmann@gmail.com> wrote: > > > > On Thu, 1 Aug 2024 at 14:37, Josh Allmann <joshua.allmann@gmail.com> wrote: > > > > > > Encoders may emit a buffering period SEI without a corresponding > > > SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. > > > > > > During Annex B conversion, this may result in the SPS/PPS being > > > inserted *after* the buffering period SEI but before the IDR NAL. > > > > > > Since the buffering period SEI references the SPS, the SPS/PPS > > > needs to come first. > > > --- > > > libavcodec/bsf/h264_mp4toannexb.c | 15 +++++++++++++++ > > > tests/ref/fate/h264-bsf-mp4toannexb | 2 +- > > > tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- > > > tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ > > > 4 files changed, 31 insertions(+), 16 deletions(-) > > > > > > > Ping for (re-)review on this patch which addresses comments from [1] > > > > Explanation for the FATE changes here [2] - it turns out that several > > of the FATE samples exhibit the same behavior that this patch fixes, > > so it is a net improvement > > > > [1] https://ffmpeg.org//pipermail/ffmpeg-devel/2024-August/331958.html > > [2] https://ffmpeg.org//pipermail/ffmpeg-devel/2024-July/330912.html > > > > Gentle ping for re-review. pushed -- Anton Khirnov _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: Prepend SPS/PPS to buffering period SEI 2024-08-01 7:58 ` Anton Khirnov 2024-08-01 21:36 ` Josh Allmann @ 2024-08-01 21:45 ` Josh Allmann 1 sibling, 0 replies; 15+ messages in thread From: Josh Allmann @ 2024-08-01 21:45 UTC (permalink / raw) To: FFmpeg development discussions and patches, Josh Allmann On Thu, 1 Aug 2024 at 00:58, Anton Khirnov <anton@khirnov.net> wrote: > Thanks for the review. > Quoting Josh Allmann (2024-07-09 21:05:47) > > Encoders may emit a buffering period SEI without a corresponding > > SPS/PPS if the SPS/PPS is carried out-of-band, eg with avcc. > > > > During Annex B conversion, this may result in the SPS/PPS being > > inserted *after* the buffering period SEI but before the IDR NAL. > > > > Since the buffering period SEI references the SPS, the SPS/PPS > > needs to come first. > > --- > > > > Notes: > > v2: Updated FATE test refs > > > > libavcodec/bsf/h264_mp4toannexb.c | 13 +++++++++++++ > > tests/ref/fate/h264-bsf-mp4toannexb | 2 +- > > tests/ref/fate/h264_mp4toannexb_ticket2991 | 18 +++++++++--------- > > tests/ref/fate/segment-mp4-to-ts | 12 ++++++------ > > 4 files changed, 29 insertions(+), 16 deletions(-) > > > > diff --git a/libavcodec/bsf/h264_mp4toannexb.c b/libavcodec/bsf/h264_mp4toannexb.c > > index 92af6a6881..6607f1e91a 100644 > > --- a/libavcodec/bsf/h264_mp4toannexb.c > > +++ b/libavcodec/bsf/h264_mp4toannexb.c > > @@ -363,6 +363,19 @@ static int h264_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *opkt) > > if (!new_idr && unit_type == H264_NAL_IDR_SLICE && (buf[1] & 0x80)) > > new_idr = 1; > > > > + /* If this is a buffering period SEI without a corresponding sps/pps > > + * then prepend any existing sps/pps before the SEI */ > > + if (unit_type == H264_NAL_SEI && buf[1] == 0 && !sps_seen && !pps_seen) { > > That 0 should be SEI_TYPE_BUFFERING_PERIOD, right? > Yes - fixed > > + if (s->sps_size) { > > + count_or_copy(&out, &out_size, s->sps, s->sps_size, PS_OUT_OF_BAND, j); > > + sps_seen = 1; > > + } > > + if (s->pps_size) { > > + count_or_copy(&out, &out_size, s->pps, s->pps_size, PS_OUT_OF_BAND, j); > > + pps_seen = 1; > > + } > > Is there a reason to insert the PPS? IIUC only the SPS is needed. > I believe it would be needed if using this bsf with the segment muxer, and segmentation happens on a recovery point (with a buffering period), eg in the test fate-segment-mp4-to-ts . Granted it is kind of incidental that this patch actually fixes that specific case. I have never seen a SPS without a PPS though. Josh _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2024-08-14 11:45 UTC | newest] Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2024-07-03 21:05 [FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: Prepend SPS/PPS to buffering period SEI Josh Allmann 2024-07-06 16:37 ` Michael Niedermayer 2024-07-08 22:06 ` Josh Allmann 2024-07-09 19:05 ` Josh Allmann 2024-07-15 17:48 ` Josh Allmann 2024-07-15 17:54 ` Josh Allmann 2024-07-22 23:01 ` Josh Allmann 2024-07-23 0:16 ` Timo Rothenpieler 2024-07-30 20:16 ` Josh Allmann 2024-08-01 7:58 ` Anton Khirnov 2024-08-01 21:36 ` Josh Allmann 2024-08-07 16:13 ` Josh Allmann 2024-08-13 16:57 ` Josh Allmann 2024-08-14 11:45 ` Anton Khirnov 2024-08-01 21:45 ` Josh Allmann
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel This inbox may be cloned and mirrored by anyone: git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \ ffmpegdev@gitmailbox.com public-inbox-index ffmpegdev Example config snippet for mirrors. AGPL code for this site: git clone https://public-inbox.org/public-inbox.git