From: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
To: ffmpeg-devel@ffmpeg.org
Cc: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Subject: [FFmpeg-devel] [PATCH 5/8] avformat/matroskaenc: Actually apply timestamp offset for Opus
Date: Thu, 1 Sep 2022 23:24:00 +0200
Message-ID: <GV1P250MB07378A596E3912045AAF51C08F7B9@GV1P250MB0737.EURP250.PROD.OUTLOOK.COM> (raw)
In-Reply-To: <GV1P250MB0737BEEBBFA5D0B1739C8D978F7B9@GV1P250MB0737.EURP250.PROD.OUTLOOK.COM>
Matroska generally requires timestamps to be nonnegative, but
there is an exception: Data that corresponds to encoder delay
and is not supposed to be output anyway can have a negative
timestamp. This is achieved by using the CodecDelay header
field: The demuxer has to subtract this value from the raw
(nonnegative) timestamps of the corresponding track.
Therefore the muxer has to add this value first to write
this raw timestamp.
Support for writing CodecDelay has been added in FFmpeg commit
d92b1b1babe69268971863649c225e1747358a74 and in Libav commit
a1aa37dd0b96710d4a17718198a3f56aea2040c1. The former simply
wrote the header field and did not apply any timestamp offsets,
leading to desynchronisation (if one uses multiple tracks).
The latter applied it at two places, but not at the one where
it actually matters, namely in mkv_write_block(), leading to
the same desynchronisation as with the former commit. It furthermore
used the wrong stream timebase to convert the delay to the
stream's timebase, as the conversion used the timebase from
before avpriv_set_pts_info().
When the latter was merged in 82e4f39883932c1b1e5c7792a1be12dec6ab603d,
it was only done in a deactivated state that still did not
offset the timestamps when muxing due to "assertion failures
and av sync errors". a1aa37dd0b96710d4a17718198a3f56aea2040c1
made it definitely more likely to run into assertion failures
(namely if the relative block timestamp doesn't fit into an int16_t).
Yet all of the above issues have been fixed (in commits
962d63157322466a9a82f9f9d84c1b6f1b582f65,
5d3953a5dcfd5f71391b7f34908517eb6f7e5146 and
4ebeab15b037a21f195696cef1f7522daf42f3ee. This commit therefore
enables applying CodecDelay, fixing ticket #7182.
There is just one slight regression from this: If one has input
with encoder delay where the first timestamp is negative, but
the pts of the part of the data that is actually intended to be
output is nonnegative, then the timestamps will currently by default
be shifted to make them nonnegative before they reach the muxer;
the muxer will then ensure that the shifted timestamps are retained.
Before this commit, the muxer did not ensure this; instead the
timestamps that the demuxer will output were shifted and
if the first timestamp of the actually intended output was zero
before shifting, then this unintentional shift just cancels
the shift performed before the packet reached the muxer.
(But notice that this only applies if all the tracks use the same
CodecDelay, or the relative sync between tracks will be impaired.)
This happens in the matroska-opus-remux and matroska-ogg-opus-remux
FATE tests. Future commits will forward the information that
the Matroska muxer has a limited capability to handle negative
timestamps so that the shifting in libavformat can take advantage
of it.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
---
libavformat/matroskaenc.c | 8 +-
tests/ref/fate/matroska-ogg-opus-remux | 108 ++++++++++----------
tests/ref/fate/matroska-opus-remux | 130 ++++++++++++-------------
3 files changed, 123 insertions(+), 123 deletions(-)
diff --git a/libavformat/matroskaenc.c b/libavformat/matroskaenc.c
index c525edb39f..0ded53dc21 100644
--- a/libavformat/matroskaenc.c
+++ b/libavformat/matroskaenc.c
@@ -1836,11 +1836,11 @@ static int mkv_write_track(AVFormatContext *s, MatroskaMuxContext *mkv,
av_log(s, AV_LOG_ERROR, "Initial padding is invalid\n");
return AVERROR(EINVAL);
}
-// track->ts_offset = av_rescale_q(par->initial_padding,
-// (AVRational){ 1, par->sample_rate },
-// st->time_base);
-
put_ebml_uint(pb, MATROSKA_ID_CODECDELAY, codecdelay);
+
+ track->ts_offset = av_rescale_q(par->initial_padding,
+ (AVRational){ 1, par->sample_rate },
+ st->time_base);
}
if (par->codec_id == AV_CODEC_ID_OPUS)
put_ebml_uint(pb, MATROSKA_ID_SEEKPREROLL, OPUS_SEEK_PREROLL);
diff --git a/tests/ref/fate/matroska-ogg-opus-remux b/tests/ref/fate/matroska-ogg-opus-remux
index 1fa776ef01..da9c8d285b 100644
--- a/tests/ref/fate/matroska-ogg-opus-remux
+++ b/tests/ref/fate/matroska-ogg-opus-remux
@@ -1,4 +1,4 @@
-a3f98769fe55bc5234cf75fb1949749a *tests/data/fate/matroska-ogg-opus-remux.matroska
+47b6b69c2ffdf5729557e90c72d241e9 *tests/data/fate/matroska-ogg-opus-remux.matroska
10200 tests/data/fate/matroska-ogg-opus-remux.matroska
#extradata 0: 19, 0x399c0471
#tb 0: 1/1000
@@ -6,54 +6,54 @@ a3f98769fe55bc5234cf75fb1949749a *tests/data/fate/matroska-ogg-opus-remux.matros
#codec_id 0: opus
#sample_rate 0: 48000
#channel_layout_name 0: stereo
-0, -7, -7, 20, 402, 0x89b1c40f
-0, 13, 13, 20, 216, 0x7bf97146
-0, 33, 33, 20, 215, 0x6cb86d8b
-0, 53, 53, 20, 218, 0x9cfd691c
-0, 73, 73, 20, 218, 0xd7fe6a94
-0, 93, 93, 20, 194, 0x35735de6
-0, 113, 113, 20, 216, 0x3ee6705a
-0, 133, 133, 20, 218, 0x67eb6cb1
-0, 153, 153, 20, 218, 0x32d0700d
-0, 173, 173, 20, 219, 0xcb7f6c60
-0, 193, 193, 20, 218, 0x9c866b33
-0, 213, 213, 20, 217, 0xfe3e6a53
-0, 233, 233, 20, 218, 0x13586833
-0, 253, 253, 20, 222, 0xbcb2669e
-0, 273, 273, 20, 218, 0x8dfc6e33
-0, 293, 293, 20, 217, 0xf5957051
-0, 313, 313, 20, 210, 0xed126e6b
-0, 333, 333, 20, 216, 0xbf947249
-0, 353, 353, 20, 203, 0x6c7e680a
-0, 373, 373, 20, 209, 0xf78f6af4
-0, 393, 393, 20, 217, 0xd60c684d
-0, 413, 413, 20, 218, 0x89056a7a
-0, 433, 433, 20, 219, 0x0bc674ad
-0, 453, 453, 20, 217, 0xb1d86d1a
-0, 473, 473, 20, 220, 0x433d685a
-0, 493, 493, 20, 364, 0x0c88be84
-0, 513, 513, 20, 221, 0x804a733d
-0, 533, 533, 20, 215, 0x6e9d6e9b
-0, 553, 553, 20, 215, 0x63016a83
-0, 573, 573, 20, 218, 0xf9a46fbe
-0, 593, 593, 20, 216, 0xa0d66c08
-0, 613, 613, 20, 216, 0xa2ca6d0a
-0, 633, 633, 20, 216, 0xf50e6f1d
-0, 653, 653, 20, 215, 0x6aaa70b6
-0, 673, 673, 20, 219, 0x7ceb6ba0
-0, 693, 693, 20, 220, 0x398d6ca9
-0, 713, 713, 20, 218, 0x7bd06ed5
-0, 733, 733, 20, 219, 0xe2906c62
-0, 753, 753, 20, 217, 0xcf316ba1
-0, 773, 773, 20, 217, 0x470b6eea
-0, 793, 793, 20, 359, 0x36c2a18a, S=1, 10
+0, 0, 0, 20, 402, 0x89b1c40f
+0, 20, 20, 20, 216, 0x7bf97146
+0, 40, 40, 20, 215, 0x6cb86d8b
+0, 60, 60, 20, 218, 0x9cfd691c
+0, 80, 80, 20, 218, 0xd7fe6a94
+0, 100, 100, 20, 194, 0x35735de6
+0, 120, 120, 20, 216, 0x3ee6705a
+0, 140, 140, 20, 218, 0x67eb6cb1
+0, 160, 160, 20, 218, 0x32d0700d
+0, 180, 180, 20, 219, 0xcb7f6c60
+0, 200, 200, 20, 218, 0x9c866b33
+0, 220, 220, 20, 217, 0xfe3e6a53
+0, 240, 240, 20, 218, 0x13586833
+0, 260, 260, 20, 222, 0xbcb2669e
+0, 280, 280, 20, 218, 0x8dfc6e33
+0, 300, 300, 20, 217, 0xf5957051
+0, 320, 320, 20, 210, 0xed126e6b
+0, 340, 340, 20, 216, 0xbf947249
+0, 360, 360, 20, 203, 0x6c7e680a
+0, 380, 380, 20, 209, 0xf78f6af4
+0, 400, 400, 20, 217, 0xd60c684d
+0, 420, 420, 20, 218, 0x89056a7a
+0, 440, 440, 20, 219, 0x0bc674ad
+0, 460, 460, 20, 217, 0xb1d86d1a
+0, 480, 480, 20, 220, 0x433d685a
+0, 500, 500, 20, 364, 0x0c88be84
+0, 520, 520, 20, 221, 0x804a733d
+0, 540, 540, 20, 215, 0x6e9d6e9b
+0, 560, 560, 20, 215, 0x63016a83
+0, 580, 580, 20, 218, 0xf9a46fbe
+0, 600, 600, 20, 216, 0xa0d66c08
+0, 620, 620, 20, 216, 0xa2ca6d0a
+0, 640, 640, 20, 216, 0xf50e6f1d
+0, 660, 660, 20, 215, 0x6aaa70b6
+0, 680, 680, 20, 219, 0x7ceb6ba0
+0, 700, 700, 20, 220, 0x398d6ca9
+0, 720, 720, 20, 218, 0x7bd06ed5
+0, 740, 740, 20, 219, 0xe2906c62
+0, 760, 760, 20, 217, 0xcf316ba1
+0, 780, 780, 20, 217, 0x470b6eea
+0, 800, 800, 20, 359, 0x36c2a18a, S=1, 10
[PACKET]
codec_type=audio
stream_index=0
-pts=-7
-pts_time=-0.007000
-dts=-7
-dts_time=-0.007000
+pts=0
+pts_time=0.000000
+dts=0
+dts_time=0.000000
duration=20
duration_time=0.020000
size=402
@@ -63,10 +63,10 @@ flags=K_
[PACKET]
codec_type=audio
stream_index=0
-pts=13
-pts_time=0.013000
-dts=13
-dts_time=0.013000
+pts=20
+pts_time=0.020000
+dts=20
+dts_time=0.020000
duration=20
duration_time=0.020000
size=216
@@ -76,10 +76,10 @@ flags=K_
[PACKET]
codec_type=audio
stream_index=0
-pts=33
-pts_time=0.033000
-dts=33
-dts_time=0.033000
+pts=40
+pts_time=0.040000
+dts=40
+dts_time=0.040000
duration=20
duration_time=0.020000
size=215
diff --git a/tests/ref/fate/matroska-opus-remux b/tests/ref/fate/matroska-opus-remux
index 61afeaa751..286bb65949 100644
--- a/tests/ref/fate/matroska-opus-remux
+++ b/tests/ref/fate/matroska-opus-remux
@@ -1,4 +1,4 @@
-551e45142f0989b281e837a3a86f0218 *tests/data/fate/matroska-opus-remux.matroska
+2ab987ba7bad94b27fae427cdff57723 *tests/data/fate/matroska-opus-remux.matroska
9355 tests/data/fate/matroska-opus-remux.matroska
#extradata 0: 19, 0x3a04048f
#tb 0: 1/1000
@@ -6,65 +6,65 @@
#codec_id 0: opus
#sample_rate 0: 48000
#channel_layout_name 0: mono
-0, -7, -7, 20, 320, 0x58b9a88d
-0, 14, 14, 20, 159, 0x6c9c4b4c
-0, 34, 34, 20, 148, 0x0caf4b5d
-0, 54, 54, 20, 139, 0xc5624226
-0, 74, 74, 20, 146, 0x633c4937
-0, 94, 94, 20, 153, 0x3d0b4f93
-0, 114, 114, 20, 158, 0xe5c55641
-0, 134, 134, 20, 156, 0xf2fd50ef
-0, 154, 154, 20, 158, 0x93b15410
-0, 174, 174, 20, 157, 0xb6f74f5f
-0, 194, 194, 20, 159, 0x9aff4957
-0, 214, 214, 20, 153, 0xfc5f4aba
-0, 234, 234, 20, 158, 0x01e44f70
-0, 254, 254, 20, 153, 0x227149cf
-0, 274, 274, 20, 155, 0x312f4cf6
-0, 294, 294, 20, 155, 0xafc54bae
-0, 314, 314, 20, 151, 0x7b4252b3
-0, 334, 334, 20, 155, 0x29074a75
-0, 354, 354, 20, 149, 0x82c44bcd
-0, 374, 374, 20, 150, 0x55c24eb5
-0, 394, 394, 20, 156, 0xf71d4f33
-0, 414, 414, 20, 153, 0x9b6c4ae5
-0, 434, 434, 20, 156, 0x75954e51
-0, 454, 454, 20, 155, 0x28ff4ff3
-0, 474, 474, 20, 153, 0xc4424969
-0, 494, 494, 20, 154, 0xfbf94cc8
-0, 514, 514, 20, 155, 0x52c549af
-0, 534, 534, 20, 150, 0x6f1e4b7a
-0, 554, 554, 20, 158, 0xabb45566
-0, 574, 574, 20, 157, 0xe61d4a99
-0, 594, 594, 20, 159, 0xf45d4fac
-0, 614, 614, 20, 159, 0xcd0553a5
-0, 634, 634, 20, 156, 0xdb244e63
-0, 654, 654, 20, 154, 0x78654c52
-0, 674, 674, 20, 154, 0x9f804cc8
-0, 694, 694, 20, 150, 0x1fdf4c80
-0, 714, 714, 20, 155, 0x1adc4f89
-0, 734, 734, 20, 155, 0x4b53511c
-0, 754, 754, 20, 151, 0x8ff2546d
-0, 774, 774, 20, 158, 0xb7e34f1b
-0, 794, 794, 20, 154, 0x4d98474b
-0, 814, 814, 20, 154, 0x14924ea8
-0, 834, 834, 20, 153, 0x8d4752bf
-0, 854, 854, 20, 149, 0x74785066
-0, 874, 874, 20, 151, 0x36c94a4c
-0, 894, 894, 20, 155, 0x82904f3b
-0, 914, 914, 20, 154, 0xd76b4a45
-0, 934, 934, 20, 159, 0x9fec548d
-0, 954, 954, 20, 154, 0x9a084dcd
-0, 974, 974, 20, 155, 0x90a54ac8
-0, 994, 994, 20, 324, 0x8e34a2f5
-0, 1014, 1014, 20, 268, 0x10f37203, S=1, 10
+0, 0, 0, 20, 320, 0x58b9a88d
+0, 21, 21, 20, 159, 0x6c9c4b4c
+0, 41, 41, 20, 148, 0x0caf4b5d
+0, 61, 61, 20, 139, 0xc5624226
+0, 81, 81, 20, 146, 0x633c4937
+0, 101, 101, 20, 153, 0x3d0b4f93
+0, 121, 121, 20, 158, 0xe5c55641
+0, 141, 141, 20, 156, 0xf2fd50ef
+0, 161, 161, 20, 158, 0x93b15410
+0, 181, 181, 20, 157, 0xb6f74f5f
+0, 201, 201, 20, 159, 0x9aff4957
+0, 221, 221, 20, 153, 0xfc5f4aba
+0, 241, 241, 20, 158, 0x01e44f70
+0, 261, 261, 20, 153, 0x227149cf
+0, 281, 281, 20, 155, 0x312f4cf6
+0, 301, 301, 20, 155, 0xafc54bae
+0, 321, 321, 20, 151, 0x7b4252b3
+0, 341, 341, 20, 155, 0x29074a75
+0, 361, 361, 20, 149, 0x82c44bcd
+0, 381, 381, 20, 150, 0x55c24eb5
+0, 401, 401, 20, 156, 0xf71d4f33
+0, 421, 421, 20, 153, 0x9b6c4ae5
+0, 441, 441, 20, 156, 0x75954e51
+0, 461, 461, 20, 155, 0x28ff4ff3
+0, 481, 481, 20, 153, 0xc4424969
+0, 501, 501, 20, 154, 0xfbf94cc8
+0, 521, 521, 20, 155, 0x52c549af
+0, 541, 541, 20, 150, 0x6f1e4b7a
+0, 561, 561, 20, 158, 0xabb45566
+0, 581, 581, 20, 157, 0xe61d4a99
+0, 601, 601, 20, 159, 0xf45d4fac
+0, 621, 621, 20, 159, 0xcd0553a5
+0, 641, 641, 20, 156, 0xdb244e63
+0, 661, 661, 20, 154, 0x78654c52
+0, 681, 681, 20, 154, 0x9f804cc8
+0, 701, 701, 20, 150, 0x1fdf4c80
+0, 721, 721, 20, 155, 0x1adc4f89
+0, 741, 741, 20, 155, 0x4b53511c
+0, 761, 761, 20, 151, 0x8ff2546d
+0, 781, 781, 20, 158, 0xb7e34f1b
+0, 801, 801, 20, 154, 0x4d98474b
+0, 821, 821, 20, 154, 0x14924ea8
+0, 841, 841, 20, 153, 0x8d4752bf
+0, 861, 861, 20, 149, 0x74785066
+0, 881, 881, 20, 151, 0x36c94a4c
+0, 901, 901, 20, 155, 0x82904f3b
+0, 921, 921, 20, 154, 0xd76b4a45
+0, 941, 941, 20, 159, 0x9fec548d
+0, 961, 961, 20, 154, 0x9a084dcd
+0, 981, 981, 20, 155, 0x90a54ac8
+0, 1001, 1001, 20, 324, 0x8e34a2f5
+0, 1021, 1021, 20, 268, 0x10f37203, S=1, 10
[PACKET]
codec_type=audio
stream_index=0
-pts=-7
-pts_time=-0.007000
-dts=-7
-dts_time=-0.007000
+pts=0
+pts_time=0.000000
+dts=0
+dts_time=0.000000
duration=20
duration_time=0.020000
size=320
@@ -74,10 +74,10 @@ flags=K_
[PACKET]
codec_type=audio
stream_index=0
-pts=14
-pts_time=0.014000
-dts=14
-dts_time=0.014000
+pts=21
+pts_time=0.021000
+dts=21
+dts_time=0.021000
duration=20
duration_time=0.020000
size=159
@@ -87,10 +87,10 @@ flags=K_
[PACKET]
codec_type=audio
stream_index=0
-pts=34
-pts_time=0.034000
-dts=34
-dts_time=0.034000
+pts=41
+pts_time=0.041000
+dts=41
+dts_time=0.041000
duration=20
duration_time=0.020000
size=148
--
2.34.1
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
next prev parent reply other threads:[~2022-09-01 21:24 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-01 20:38 [FFmpeg-devel] [PATCH 1/8] fftools/ffprobe: Report initial and trailing padding Andreas Rheinhardt
2022-09-01 21:23 ` [FFmpeg-devel] [PATCH 2/8] fate/matroska: Add tests for muxing with initial_padding Andreas Rheinhardt
2022-09-01 21:23 ` [FFmpeg-devel] [PATCH 3/8] avformat/matroskaenc: Only write DiscardPadding if nonzero Andreas Rheinhardt
2022-09-01 21:23 ` [FFmpeg-devel] [PATCH 4/8] avformat/matroskaenc: Don't override samplerate for CodecDelay Andreas Rheinhardt
2022-09-01 21:24 ` Andreas Rheinhardt [this message]
2022-09-01 21:24 ` [FFmpeg-devel] [PATCH 6/8] avformat/mux: Allow muxers to set custom min timestamp Andreas Rheinhardt
2022-09-01 21:24 ` [FFmpeg-devel] [PATCH 7/8] avformat/matroskaenc: Use " Andreas Rheinhardt
2022-09-01 21:24 ` [FFmpeg-devel] [PATCH 8/8] avformat/matroskaenc: Write CodecDelay for codecs != Opus Andreas Rheinhardt
2022-09-04 22:52 ` [FFmpeg-devel] [PATCH 1/8] fftools/ffprobe: Report initial and trailing padding Andreas Rheinhardt
2022-09-06 7:55 ` Anton Khirnov
2022-09-06 9:31 ` Andreas Rheinhardt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=GV1P250MB07378A596E3912045AAF51C08F7B9@GV1P250MB0737.EURP250.PROD.OUTLOOK.COM \
--to=andreas.rheinhardt@outlook.com \
--cc=ffmpeg-devel@ffmpeg.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
This inbox may be cloned and mirrored by anyone:
git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git
# If you have public-inbox 1.1+ installed, you may
# initialize and index your mirror using the following commands:
public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
ffmpegdev@gitmailbox.com
public-inbox-index ffmpegdev
Example config snippet for mirrors.
AGPL code for this site: git clone https://public-inbox.org/public-inbox.git