[FFmpeg-devel] Matroska and output_ts_offset

[FFmpeg-devel] Matroska and output_ts_offset

[Peter Zebühr]

Hello,

I noticed the other day that if I try to mux opus in webm with a timestamp offset set I don't get the expected timestamps out on my packets. Example:

	ffmpeg -f lavfi -i "sine=frequency=1000:duration=60" -output_ts_offset 200ms -c:a libopus sine_200ms_offset.webm

Now if I inspect the packets with ffprobe on the produced output the first packet has a PTS of 186. My expectation would be that it would be: 200 - 6.5 (opus encoder delay) ~= 193.
Looking at the output with mkvinfo also indicates that it starts at 193ms (186 + encoder delay I assume here).

At first I thought it looked like the encoder delay gets added twice, but after some more pondering I think what is happening here is:

1. Opus outputs packets starting with the priming samples, that ends up with a negative PTS (rounded to -7)
2. In mux.c the output_ts_offset takes effect before the call to "handle_avoid_negative_ts"
3. PTS -7 gets shifter by the requested ts_offset to 193
4. Matroska muxer writes the side data about encoder delay. 
5. The produced stream now effectively starts one encoder delay distance too early.

I played around locally with moving the call to "handle_avoid_negative_ts" in mux.c/write_packet to right before the ts_offset handling instead of right after and that seems to fix this particular problem. But, I am unsure what other potential consequences that would have. Would appreciate if someone more familiar with this can help fix this issue.

Regards,
Peter
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Matroska and output_ts_offset

[Andreas Rheinhardt]

Peter Zebühr:
> Hello,
> 
> I noticed the other day that if I try to mux opus in webm with a timestamp offset set I don't get the expected timestamps out on my packets. Example:
> 
> 	ffmpeg -f lavfi -i "sine=frequency=1000:duration=60" -output_ts_offset 200ms -c:a libopus sine_200ms_offset.webm
> 
> Now if I inspect the packets with ffprobe on the produced output the first packet has a PTS of 186. My expectation would be that it would be: 200 - 6.5 (opus encoder delay) ~= 193.
> Looking at the output with mkvinfo also indicates that it starts at 193ms (186 + encoder delay I assume here).
> 
> At first I thought it looked like the encoder delay gets added twice, but after some more pondering I think what is happening here is:
> 
> 1. Opus outputs packets starting with the priming samples, that ends up with a negative PTS (rounded to -7)
> 2. In mux.c the output_ts_offset takes effect before the call to "handle_avoid_negative_ts"
> 3. PTS -7 gets shifter by the requested ts_offset to 193
> 4. Matroska muxer writes the side data about encoder delay. 
> 5. The produced stream now effectively starts one encoder delay distance too early.
> 
> I played around locally with moving the call to "handle_avoid_negative_ts" in mux.c/write_packet to right before the ts_offset handling instead of right after and that seems to fix this particular problem. But, I am unsure what other potential consequences that would have. Would appreciate if someone more familiar with this can help fix this issue.
> 

The Matroska muxer is buggy wrt. to the ts_offset relating to codec
delay. You can see it in lines 1839-1841 which are commented out.
Commenting them out happened in commit
82e4f39883932c1b1e5c7792a1be12dec6ab603d, merging the libav commit that
implemented it (namely
https://github.com/FFmpeg/FFmpeg/commit/a1aa37dd0b96710d4a17718198a3f56aea2040c1).
It mentions "assertion failures and av sync errors". I can only think of
one way that it could have led to assertion failures, but this should
have been fixed in 4ebeab15b037a21f195696cef1f7522daf42f3ee (and since
then I wondered whether it can't be enabled).

- Andreas
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Matroska and output_ts_offset

[Peter Zebühr]

> On 29 Aug 2022, at 16:37, Andreas Rheinhardt <andreas.rheinhardt@outlook.com> wrote:
> 
> The Matroska muxer is buggy wrt. to the ts_offset relating to codec
> delay. You can see it in lines 1839-1841 which are commented out.
> Commenting them out happened in commit
> 82e4f39883932c1b1e5c7792a1be12dec6ab603d, merging the libav commit that
> implemented it (namely
> https://github.com/FFmpeg/FFmpeg/commit/a1aa37dd0b96710d4a17718198a3f56aea2040c1 <https://github.com/FFmpeg/FFmpeg/commit/a1aa37dd0b96710d4a17718198a3f56aea2040c1>).
> It mentions "assertion failures and av sync errors". I can only think of
> one way that it could have led to assertion failures, but this should
> have been fixed in 4ebeab15b037a21f195696cef1f7522daf42f3ee (and since
> then I wondered whether it can't be enabled).
> 
> - Andreas

Interesting,

It does look like re-enabling that commented out section would indeed shift my teimstamps forward by the encoder dalay and work with my ts_offset setting. 
I do wonder though, would that not also mean that for the cases where the stream should start at 0 it would result in streams starting 7ms in due to the handle_avoid_negative_ts already having shifted the first packets PTS from -7 to 0? And that would end up being shifted once again for the inital_padding?

/ Peter
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Matroska and output_ts_offset

[Andreas Rheinhardt]

Peter Zebühr:
>> On 29 Aug 2022, at 16:37, Andreas Rheinhardt <andreas.rheinhardt@outlook.com> wrote:
>>
>> The Matroska muxer is buggy wrt. to the ts_offset relating to codec
>> delay. You can see it in lines 1839-1841 which are commented out.
>> Commenting them out happened in commit
>> 82e4f39883932c1b1e5c7792a1be12dec6ab603d, merging the libav commit that
>> implemented it (namely
>> https://github.com/FFmpeg/FFmpeg/commit/a1aa37dd0b96710d4a17718198a3f56aea2040c1 <https://github.com/FFmpeg/FFmpeg/commit/a1aa37dd0b96710d4a17718198a3f56aea2040c1>).
>> It mentions "assertion failures and av sync errors". I can only think of
>> one way that it could have led to assertion failures, but this should
>> have been fixed in 4ebeab15b037a21f195696cef1f7522daf42f3ee (and since
>> then I wondered whether it can't be enabled).
>>
>> - Andreas
> 
> Interesting,
> 
> It does look like re-enabling that commented out section would indeed shift my teimstamps forward by the encoder dalay and work with my ts_offset setting. 
> I do wonder though, would that not also mean that for the cases where the stream should start at 0 it would result in streams starting 7ms in due to the handle_avoid_negative_ts already having shifted the first packets PTS from -7 to 0? And that would end up being shifted once again for the inital_padding?
> 

Yes, the shifting code in libavformat/mux.c does not yet know about the
Matroska muxer's ability to shift by initial_padding samples. But at
least it will shift all packets, thereby preserving sync in case there
are multiple streams. This is not done currently.

- Andreas
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".