Re: [FFmpeg-devel] Politics

Re: [FFmpeg-devel] Politics

[Daniel Cantarín]

> 
> The more the focus is moving to "a single frame" doesn't matter,
> the more will that conversation create the impression that my patchset
> would be lacking precision.
> 

This is a possibility, indeed.

Yet, the less one answers, the more it also looks like the issue is dead
and/or some argument was the last word on the issue. Neither are the
case. Also, there's the constant claim of "you didn't listened to me";
not answering surely leads to that effect, and answering puts the burden
of proof to the people stating the hypothetical issue.

OTOH, people here could be very knowledgeable and experienced, thus
actually maybe being onto something. Debating fantasy scenarios could be
a chore and even a waste of time, but asking for proper input files is
not, and this gives the oportunity to deal with such possible tests.

No input files so far, of course, as I also do believe the
"frame-perfect serious issue" is simply not real: not any problem in
your code, not any real-life use case. We'll see if anyone brings such
real example, with files to test.

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Michael
> Niedermayer
> Sent: Wednesday, December 22, 2021 8:54 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] Politics
> 
> On Wed, Dec 22, 2021 at 01:29:04PM +0000, Soft Works wrote:
> >
> >
> > > -----Original Message-----
> > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Michael
> > > Niedermayer
> > > Sent: Monday, December 20, 2021 4:24 PM
> > > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> > > Subject: Re: [FFmpeg-devel] Politics
> > >
> > > I am not sure the direction from which you approuch this is going to
> > > increase the chances this patch has.
> > >
> > > All stream types in libavformat/codec are timebase based, that was
> > > done because its exact (for some definition of exact at least)
> > >
> > > I think you should argue why this is the best way forward not why its
> > > not too bad
> > >
> > > also in a few places where a fixed timebase is used ffmpeg uses
> > > AV_TIME_BASE_Q which is micro not milli seconds. That suddenly
> > > allows exactly addressing individual frames and audio samples.
> > > And it should be easy to change to from ms, its just a *1000
> > > it would weaken the precission argument
> >
> > For the final chapter of this story, let us return to the original
> > subject which I would summarize like:
> >
> 
> [...]
> 
> > I think I know better and therefore insist
> > on having a higher precision and/or flexible timebase for subtitle
> > timings, otherwise I won't accept the patchset"
> 
> I didnt say any of this


And you weren't meant at all.
(you're actually one of the persons I'm respecting the most) 

Sorry for the misunderstanding. I should have replied to one 
of the earlier messages. I thought the fictional quote would 
make it clear who is meant.


> > Let's look at the concern first: The concern is about that with
> > a subtitle precision of milliseconds (let's say milli, even
> > though we actually have microseconds), it would not be possible
> 
> Not sure i understand you but the code you add to AVFrame says "ms" and
> thats milli not micro,
> the prefix for micro is the SI system is μ or u if you want ASCII

subtitle_pts is AV_TIMEBASE_Q.

I had recently suggested to change the other fields (subtitle_start_time
and subtitle_end_time to AV_TIMEBASE_Q as well, to have all subtitle 
timing fields at the same base.

Kind regards,
softworkz
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 2829 bytes --]

On Wed, Dec 22, 2021 at 01:29:04PM +0000, Soft Works wrote:
> 
> 
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Michael
> > Niedermayer
> > Sent: Monday, December 20, 2021 4:24 PM
> > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] Politics
> > 
> > I am not sure the direction from which you approuch this is going to
> > increase the chances this patch has.
> > 
> > All stream types in libavformat/codec are timebase based, that was
> > done because its exact (for some definition of exact at least)
> > 
> > I think you should argue why this is the best way forward not why its
> > not too bad
> > 
> > also in a few places where a fixed timebase is used ffmpeg uses
> > AV_TIME_BASE_Q which is micro not milli seconds. That suddenly
> > allows exactly addressing individual frames and audio samples.
> > And it should be easy to change to from ms, its just a *1000
> > it would weaken the precission argument
> 
> For the final chapter of this story, let us return to the original 
> subject which I would summarize like:
> 

[...]

> I think I know better and therefore insist 
> on having a higher precision and/or flexible timebase for subtitle 
> timings, otherwise I won't accept the patchset"

I didnt say any of this

what is the case is that FFmpeg tries to use flexible timebases 
for everything. And also we try to be exact
I have said nothing about accepting or rejecting this patchset,
nor have i really made my mind up on how i would vote if it comes
to a vote.
What i have made my mind up on is that if someone replaces these
timestamps with higher precission or exact ones i would support it.

Also i do want someone to work on and improve subtitles
in FFmpeg. iam not your oponent here. I do want you to stay and
continue to contribute. But the area you picked certainly is
one where people have strong oppinions and its a big change as
well. I would suggest to keep a cool head and not give up yet.
Find out exatly what the people who objected want and if you can
find some compromise that makes everyone ok with it


> 
> Let's look at the concern first: The concern is about that with 
> a subtitle precision of milliseconds (let's say milli, even
> though we actually have microseconds), it would not be possible

Not sure i understand you but the code you add to AVFrame says "ms" and 
thats milli not micro, 
the prefix for micro is the SI system is μ or u if you want ASCII

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

In fact, the RIAA has been known to suggest that students drop out
of college or go to community college in order to be able to afford
settlements. -- The RIAA

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Michael
> Niedermayer
> Sent: Monday, December 20, 2021 4:24 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] Politics
> 
> I am not sure the direction from which you approuch this is going to
> increase the chances this patch has.
> 
> All stream types in libavformat/codec are timebase based, that was
> done because its exact (for some definition of exact at least)
> 
> I think you should argue why this is the best way forward not why its
> not too bad
> 
> also in a few places where a fixed timebase is used ffmpeg uses
> AV_TIME_BASE_Q which is micro not milli seconds. That suddenly
> allows exactly addressing individual frames and audio samples.
> And it should be easy to change to from ms, its just a *1000
> it would weaken the precission argument

For the final chapter of this story, let us return to the original 
subject which I would summarize like:

"Even though the whole world is fine using millisecond precision 
for subtitle display, I think I know better and therefore insist 
on having a higher precision and/or flexible timebase for subtitle 
timings, otherwise I won't accept the patchset"


I have been waiting for a while to answer, expecting somebody 
might come to realize himself how useless this whole idea actually
is, but I guess it's time to reveal:

Let's look at the concern first: The concern is about that with 
a subtitle precision of milliseconds (let's say milli, even
though we actually have microseconds), it would not be possible
to make sure that a subtitle event would be shown exactly at a 
specific video frame.

The claim is: This could be achieved by having a high precision
(and/or a custom time base) for subtitle timings, because
this would allow to have subtitle start times that could exactly 
match the frame display time of the frame at which a subtitle
should be initially displayed.

For a moment let's put aside the argument about subtitle format
precision. Let's assume we'd have a subtitle format that allows
such precisions and maybe even custom time bases and let's assume
a player that can handle this.

Now we look at the player and a situation where the player needs
to display frame N. At this point, a range of different things
can happen, mostly specific to the implementation of the player:
Whether it reads a frame's time value or infers it from the 
frame rate or which time base a player is using internally, 
just to name a few examples.
And then - at a total different place of implementation in the
player (could be custom, or a library like libass), the player
needs to determine whether a (and which) subtitle needs to be
displayed over frame N.
Here, we have the frame time, which has undergone a number
of calculations and we have a subtitle event with our super-
precise subtitle start time. The player converts that to its 
internal time base, and then..

..how does the player determine whether the subtitle event
should be shown on frame N?

Does it check like: frame.pts == subtitle.pts? No, it doesn't!

It does something like: 

frame.pts > subtitle.pts && frame.pts < subtitle.end_pts

..because it also needs to display all subtitle events that are 
already visible.


Let's look again at the proposal: to use high precision subtitle
timings which would allow us to have subtitle start times that 
are as close as possible (or even equal) to the video frame 
time.

Now what a surprise: having-frame-equal subtitle start-times
wouldn't make it _more_ clear at which video frame the subtitle 
should be shown - it would make it more and more _unclear_ and 
non-deterministic whether it should be shown at that frame or 
the next!
Eventually, the final presentation would depend on:
client implementation details and rounding errors, which means 
the opposite of consistent, reliable or predictable.

The closer and more precise a subtitle start time would be set
to a frame's display time, the higher the chance that it 
would be shown at the wrong time (due to the >/< tests that
clients need to perform.


Everybody who is creating computer animations and who wants 
to achieve sudden changes from one frame to another knows,
that this change needs to be authored in a way that it 
happens in the timely middle between two frames (to make 
it safe against slight adjustments, rescaling, etc.)

And the same goes for subtitle authoring: When you want 
to make sure that a subtitle is shown at frame N+1 but 
not at frame N, then you set the subtitle start time 
to a time half-way between N and N+1.

And this doesn't require high-precision timing values
nor custom time bases and it's also stable against 
calculations and rounding errors that might occur
during processing.

My wish for the future would be criticism that isn't 
based on mind-farts or unrealistic hypothetical cases,
but actual problems, ideally accompanied by an example 
to demonstrate.

Kind regards,
softworkz

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Michael
> Niedermayer
> Sent: Tuesday, December 21, 2021 7:39 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] Politics
> 
> On Mon, Dec 20, 2021 at 11:20:38PM +0000, Soft Works wrote:
> [...]
> > > Enlightened by this, let's go back to to your example. The EU frame has a
> >
> > > duration of 40ms, the US frame 33.3ms. The US frame start 0.02ms later
> and
> >
> > > is fully included in the duration of the EU frame.
> >
> >
> >
> > Seems I accidentally deleted a paragraph:
> >
> >
> >
> > The two frames are almost congruent in start and to a large percentage in
> duration, and as they are meant to present the same picture
> >
> >  it cannot happen at all that only one of them would have a hard change
> (like from white to black or something  appearing or disappearing within a
> short
> 
> video frames often are not sampled accross the whole period representing the
> frame. just look at some video with fast moving things
> 
> now what can give you a sub ms change
> a scene change, an instantaneos cut from one scene to another
> a flipped light switch, an explosion, an electric arc striking something
> a camera flash, a spinning wheel with hole or a black/white pattern
> a laser pointer just gently waving over your camera or something seen by
> your camera, a fast moving object between the camera and light source
> and many more

Yes, but that invalidates your claim that videos created from computer
generated sources would always be "exact". And in fact those things 
are done in a very different way. 

I'm familiar with 3D animations since the initial release of 3D Studio 
(the DOS based one, not Max), and some friends of mine are doing this
on a professional basis, so I'm pretty familiar with the procedures.

It is important to understand that 3D animations, with its tooling
and all the techniques involved are not doing simulations of reality.
Instead, it's only about creating visuals that look like reality as 
close as possible, and the methods involved are in many ways different
from the actual physics by which our reality is driven.

So, one important thing to note is, that it doesn't work in a way like 
you described above, that an animation is created once, independent 
of presentation framerate, and then you could create videos from 
that definition at arbitrary framerates - no way to do it like that,
given the way animations are done nowadays.

Just the opposite is true: I don't know any other video creation 
discipline where the individual output frame "raster" is more important
and getting more attention and individual work than with computer 
animations.
After the creative part (and other steps) are done, the whole
animation is undergoing another stage that is all about output
frames. Every single frame of the animation is rendered for the 
given output framerate, and all timings key-frame timings and
values are specifically adjusted for that output framerate.

There are many reasons why this needs to be done; for most, 
it burns down to avoiding visual artifacts, imperfections, 
"ghost" visuals and also for having "clean cuts" (intra-scene,
not the cutting that is done even later between scenes).

This is a tedious and time-consuming process which can take 
many weeks any a large team, depending on the length of the 
output.
When an output at a different framerate is needed, that whole
process needs to be repeated, and in the end you will
have a different animation definition for the new framerate
with hundreds or thousands of tiny differences.

Another point one might find interesting is that none of
those applications is working with time bases at precisions
like we are talking about here.

I'm still wondering whether somebody will get the twist...

softworkz






_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 1580 bytes --]

On Mon, Dec 20, 2021 at 11:20:38PM +0000, Soft Works wrote:
[...]
> > Enlightened by this, let's go back to to your example. The EU frame has a
> 
> > duration of 40ms, the US frame 33.3ms. The US frame start 0.02ms later and
> 
> > is fully included in the duration of the EU frame.
> 
> 
> 
> Seems I accidentally deleted a paragraph:
> 
> 
> 
> The two frames are almost congruent in start and to a large percentage in duration, and as they are meant to present the same picture
> 
>  it cannot happen at all that only one of them would have a hard change (like from white to black or something  appearing or disappearing within a short

video frames often are not sampled accross the whole period representing the
frame. just look at some video with fast moving things

now what can give you a sub ms change
a scene change, an instantaneos cut from one scene to another
a flipped light switch, an explosion, an electric arc striking something
a camera flash, a spinning wheel with hole or a black/white pattern
a laser pointer just gently waving over your camera or something seen by
your camera, a fast moving object between the camera and light source
and many more

btw if it would be impossible to take really short duration images with a
flash then alot of images taken that way from fast moving objects like
bullets couldnt exist.

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

If you think the mosad wants you dead since a long time then you are either
wrong or dead since a long time.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Soft Works]

> -----Original Message-----

> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Soft

> Works

> Sent: Monday, December 20, 2021 11:35 PM

> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>

> Subject: Re: [FFmpeg-devel] Politics

>

>

>

> > -----Original Message-----

> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org<mailto:ffmpeg-devel-bounces@ffmpeg.org>> On Behalf Of Michael

> > Niedermayer

> > Sent: Monday, December 20, 2021 3:50 PM

> > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org<mailto:ffmpeg-devel@ffmpeg.org>>

> > Subject: Re: [FFmpeg-devel] Politics

> >

> > On Sun, Dec 19, 2021 at 06:31:38PM +0000, Soft Works wrote:

> > >

> > >

> > > > -----Original Message-----

> > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org<mailto:ffmpeg-devel-bounces@ffmpeg.org>> On Behalf Of

> Soft

> > Works

> > > > Sent: Sunday, December 19, 2021 7:23 PM

> > > > To: FFmpeg development discussions and patches <ffmpeg-<mailto:ffmpeg-devel@ffmpeg.org>

> devel@ffmpeg.org<mailto:ffmpeg-devel@ffmpeg.org>>

> > > > Subject: Re: [FFmpeg-devel] Politics

> > > >

> > > >

> > > >

> > > > > -----Original Message-----

> > > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org<mailto:ffmpeg-devel-bounces@ffmpeg.org>> On Behalf Of

> > Michael

> > > > > Niedermayer

> > > > > Sent: Sunday, December 19, 2021 4:16 PM

> > > > > To: FFmpeg development discussions and patches <ffmpeg-

> > devel@ffmpeg.org<mailto:devel@ffmpeg.org>>

> > > > > Subject: Re: [FFmpeg-devel] Politics

> > > > >

> > > > > On Sat, Dec 1

> > > 8, 2021 at 08:41:09PM +0000, Soft Works wrote:

> > >

> > > > I don't think that a camera exists that has a framerate high enough

> so

> > that

> > > > you could produce videos at both framerates just from picking

> original

> > > > frames.

> > >

> > > I forgot to mention that I mean a high speed camera which can work at

> such

> > an

> > > odd rate (least common multiple..)

> >

> > for sake of this argument being complete, teh video is computer generated

> > so sampling at any time instance is possible and exact.

>

> Is that so?

>

> First of all, we need to decide about what kind of computer generation

> we are talking exactly?

>

> Is it generating a point-in-time picture for each frame start time?

> Assuming generated video of a situation where visual events that would

> start

> at 10ms, disappear after a duration of 20ms and repeat that every 40ms.

> These things would never be visible in the EU video, but in the US

> video.

>

> You mentioned "sampling", so I guess you mean that each frame would

> represent

> the appearance averaged over the frame duration, right?

> Of course that's a much more correct way to reflect reality.

>

> Enlightened by this, let's go back to to your example. The EU frame has a

> duration of 40ms, the US frame 33.3ms. The US frame start 0.02ms later and

> is fully included in the duration of the EU frame.



Seems I accidentally deleted a paragraph:



The two frames are almost congruent in start and to a large percentage in duration, and as they are meant to present the same picture

 it cannot happen at all that only one of them would have a hard change (like from white to black or something  appearing or disappearing within a short





> If the US-to-EU conversion (or other direction) has been done in a natural

> way without human intervention and explicit cutting, then the example is

> busted once another time.

>

> Want I can't precisely answer (but wondering) , is whether it

> would even pe possible to have single audio stream in the same container

> that could work with both video..?^








_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Michael
> Niedermayer
> Sent: Monday, December 20, 2021 3:50 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] Politics
> 
> On Sun, Dec 19, 2021 at 06:31:38PM +0000, Soft Works wrote:
> >
> >
> > > -----Original Message-----
> > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Soft
> Works
> > > Sent: Sunday, December 19, 2021 7:23 PM
> > > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> > > Subject: Re: [FFmpeg-devel] Politics
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> Michael
> > > > Niedermayer
> > > > Sent: Sunday, December 19, 2021 4:16 PM
> > > > To: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org>
> > > > Subject: Re: [FFmpeg-devel] Politics
> > > >
> > > > On Sat, Dec 1
> > 8, 2021 at 08:41:09PM +0000, Soft Works wrote:
> >
> > > I don't think that a camera exists that has a framerate high enough so
> that
> > > you could produce videos at both framerates just from picking original
> > > frames.
> >
> > I forgot to mention that I mean a high speed camera which can work at such
> an
> > odd rate (least common multiple..)
> 
> for sake of this argument being complete, teh video is computer generated
> so sampling at any time instance is possible and exact.

Is that so?

First of all, we need to decide about what kind of computer generation
we are talking exactly?

Is it generating a point-in-time picture for each frame start time?
Assuming generated video of a situation where visual events that would start
at 10ms, disappear after a duration of 20ms and repeat that every 40ms.
These things would never be visible in the EU video, but in the US 
video.

You mentioned "sampling", so I guess you mean that each frame would represent 
the appearance averaged over the frame duration, right?
Of course that's a much more correct way to reflect reality.

Enlightened by this, let's go back to to your example. The EU frame has a
duration of 40ms, the US frame 33.3ms. The US frame start 0.02ms later and 
is fully included in the duration of the EU frame. 

If the US-to-EU conversion (or other direction) has been done in a natural 
way without human intervention and explicit cutting, then the example is
busted once another time.

Want I can't precisely answer (but wondering) , is whether it 
would even pe possible to have single audio stream in the same container
that could work with both video..?

Thanks,
softworkz


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Daniel Cantarín]

> 
> I am not sure the direction from which you approuch this is going to
> increase the chances this patch has.
> 

Me neither.

But I can barely talk about ffmpeg internals: they're huge, and I know
the little bits I'm familiar with, having some experience with filters.
So, whatever argument I may have come from use cases experience and not
from ffmpeg internals knowledge.

That's why I don't deny there may be issues with softworkz code. But I
can say when the critics are presenting unreal use cases as some kind of
proof of a problem.


> 
> All stream types in libavformat/codec are timebase based, that was
> done because its exact (for some definition of exact at least)
> 
> I think you should argue why this is the best way forward not why its
> not too bad
> 

No, I don't think that's the case.
I recognize what you say has ground. Thing is, what I do does too.

It's not about it being "not too bad", but about it working were no
other code is present, and nothing being broken by it. You may speak
about ffmpeg internals, I speak about real life use cases, both are
valid considerations.

Your (and others) criticism against the patchset seems legit. However,
softworkz has been dealing with such criticism from months now, he has
been sustaining that what he did was neccessary, and suspiciously every
single example against that seems to be unreal. So, softworkz may be
wrong somehow, but the devs are clearly exaggerating about the supposed
consequences of such wrongness. Why? Devs here are experts. Why would
they need to exaggerate like this? I say something's fishy.

I understand devs may have biases towards norms, standard or otherwise,
but I find it hard to believe that somebody experienced in software
development would not accept a patch if it adds use cases and doesn't
break any previously implemented one, specially when the proper way to
implement it was object of debate for a decade without anybody doing it.
That's no small detail. And this patchset is no small feat. This is
special, not norm.

I'm doing tests from my side, by the way: it's not about "just approve
it". I see the thing working, and I'm looking for problems when somebody
say there's a problem. That's why I also discuss what I believe to be
fantasy problems: I respect the devs claims, even if I don't believe in
it. But this "frame-perfect serious problem" is clearly an exaggeration,
and that says something about arguments against the patchset.

Breaking something is the real issue here IMO, and not some clearly
unachievable purity, whatever the reason of the unachievability. I
pretend to intervene here with that logic in mind. If that doesn't help,
so be it.

In any case, devs biases will be there no matter what I say. So it's not
really about me: it's about softworkz against the ffmpeg devs community.
I just happen to want the use cases, and find this patchset working.


> 
> also in a few places where a fixed timebase is used ffmpeg uses
> AV_TIME_BASE_Q which is micro not milli seconds. That suddenly
> allows exactly addressing individual frames and audio samples.
> And it should be easy to change to from ms, its just a *1000
> it would weaken the precission argument
> 

Then softworkz would most likely adapt the code. He doesn't seem to be
reluctant to apply small modifications in order to be approved. Yet,
again and again he claims it's not that easy, and we get back to unreal
examples. This part I don't understand yet, as my time and knowledge are
both limited. But, with time, I will; if the patch doesn't die, perhaps
we'll find its way to be implemented. I just hope softworkz doesn't get
burned.

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 3743 bytes --]

On Mon, Dec 20, 2021 at 11:06:44AM -0300, Daniel Cantarín wrote:
> 
> > 
> > consider a subtitle track
> > consider 2 video tracks for US 30000/1001 fps and EU 25 fps
> > 
> > the 6th frame in the US track is at  0.2002 sec, the 5th in the EU track
> > at 0.2sec
> > 
> > 
> > if these differ and you want a subtitle to either stop displaying after
> > the
> > earlier or begin displaying after the earlier reliably. Then you need a
> > timebase that can represent points within each such close encounter of
> > frame
> > times.
> > 
> 
> So, this isn't a subtitle problem if subtitle timings are correct. You
> let 0.2002 in the subtitle, players will decide the frame. You want to
> match the frame with the content, then you transcode the video properly.
> Am I losing something?
> 
> Then again, we're of course not discussing players. One may say "that's
> not ffmpeg responsability", yet players are 90% of the time the main
> reason we have to do any extra work with ffmpeg. If we discuss real
> life scenarios, we should be discussing players.
> 
> I also fail to see why would somebody want a "frame-perfect" sync
> between two videos with different FPS. That's absurd: there's no such
> thing as "frame-perfect" in such scenarios. What you would want is video
> timings to be honored. That is: FPS conversion to be faithful to video
> timings and content. I can imagine such precision dealing with frames
> being blurred and having to do lots of fine tuning in order to be able
> to perceive a single frame off in subtitles. That would be the real
> problematic situation, not any .0002 rounding.
> 
> So, I see your example as a case of "what would the player do with that
> 0.2002 rounding" (in which case, we all know there would be two
> different subtitle files/streams with different precision for each
> video if it's THAT important) instead of "we need to change the timebase
> to something 4 decimals precise multiplier to both videos".
> 
> And even if this fails and a frame is interpreted incorrectly in the
> player, good luck finding anyone saying anything about it: nobody cares
> about a single frame when it's about subtitles. People watching
> subtitles are simply not looking at that. This isn't speedrunning: it's
> translating and transcribing. People wants to understand what's going on
> in the screen, not debugging video frames.
> 
> So, I see no "serious subtitle problem" nor anything close to
> "unnaceptable" here.
> 
> Let's remark, as softworkz say, this are all fantasy scenarios, and the
> patchset doesn't show any real precision problem so far. If anyone is
> willing to share input files to test such possible precision problems,
> I'm willing to do the tests. No such files so far.

I am not sure the direction from which you approuch this is going to
increase the chances this patch has.

All stream types in libavformat/codec are timebase based, that was
done because its exact (for some definition of exact at least)

I think you should argue why this is the best way forward not why its
not too bad

also in a few places where a fixed timebase is used ffmpeg uses
AV_TIME_BASE_Q which is micro not milli seconds. That suddenly
allows exactly addressing individual frames and audio samples.
And it should be easy to change to from ms, its just a *1000
it would weaken the precission argument

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The day soldiers stop bringing you their problems is the day you have stopped 
leading them. They have either lost confidence that you can help or concluded 
you do not care. Either case is a failure of leadership. - Colin Powell

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 1358 bytes --]

On Sun, Dec 19, 2021 at 06:31:38PM +0000, Soft Works wrote:
> 
> 
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Soft Works
> > Sent: Sunday, December 19, 2021 7:23 PM
> > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] Politics
> > 
> > 
> > 
> > > -----Original Message-----
> > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Michael
> > > Niedermayer
> > > Sent: Sunday, December 19, 2021 4:16 PM
> > > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> > > Subject: Re: [FFmpeg-devel] Politics
> > >
> > > On Sat, Dec 1
> 8, 2021 at 08:41:09PM +0000, Soft Works wrote:
> 
> > I don't think that a camera exists that has a framerate high enough so that
> > you could produce videos at both framerates just from picking original
> > frames.
> 
> I forgot to mention that I mean a high speed camera which can work at such an 
> odd rate (least common multiple..)

for sake of this argument being complete, teh video is computer generated 
so sampling at any time instance is possible and exact.

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

No great genius has ever existed without some touch of madness. -- Aristotle

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Daniel Cantarín]

> 
> consider a subtitle track
> consider 2 video tracks for US 30000/1001 fps and EU 25 fps
> 
> the 6th frame in the US track is at  0.2002 sec, the 5th in the EU 
> track at 0.2sec
> 
> 
> if these differ and you want a subtitle to either stop displaying after 
> the
> earlier or begin displaying after the earlier reliably. Then you need a
> timebase that can represent points within each such close encounter of 
> frame
> times.
> 

So, this isn't a subtitle problem if subtitle timings are correct. You
let 0.2002 in the subtitle, players will decide the frame. You want to
match the frame with the content, then you transcode the video properly.
Am I losing something?

Then again, we're of course not discussing players. One may say "that's
not ffmpeg responsability", yet players are 90% of the time the main
reason we have to do any extra work with ffmpeg. If we discuss real
life scenarios, we should be discussing players.

I also fail to see why would somebody want a "frame-perfect" sync
between two videos with different FPS. That's absurd: there's no such
thing as "frame-perfect" in such scenarios. What you would want is video
timings to be honored. That is: FPS conversion to be faithful to video
timings and content. I can imagine such precision dealing with frames
being blurred and having to do lots of fine tuning in order to be able
to perceive a single frame off in subtitles. That would be the real
problematic situation, not any .0002 rounding.

So, I see your example as a case of "what would the player do with that
0.2002 rounding" (in which case, we all know there would be two
different subtitle files/streams with different precision for each
video if it's THAT important) instead of "we need to change the timebase
to something 4 decimals precise multiplier to both videos".

And even if this fails and a frame is interpreted incorrectly in the
player, good luck finding anyone saying anything about it: nobody cares
about a single frame when it's about subtitles. People watching
subtitles are simply not looking at that. This isn't speedrunning: it's
translating and transcribing. People wants to understand what's going on
in the screen, not debugging video frames.

So, I see no "serious subtitle problem" nor anything close to
"unnaceptable" here.

Let's remark, as softworkz say, this are all fantasy scenarios, and the
patchset doesn't show any real precision problem so far. If anyone is
willing to share input files to test such possible precision problems,
I'm willing to do the tests. No such files so far.


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Soft Works
> Sent: Sunday, December 19, 2021 7:23 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] Politics
> 
> 
> 
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Michael
> > Niedermayer
> > Sent: Sunday, December 19, 2021 4:16 PM
> > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] Politics
> >
> > On Sat, Dec 1
8, 2021 at 08:41:09PM +0000, Soft Works wrote:

> I don't think that a camera exists that has a framerate high enough so that
> you could produce videos at both framerates just from picking original
> frames.

I forgot to mention that I mean a high speed camera which can work at such an 
odd rate (least common multiple..)


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Michael
> Niedermayer
> Sent: Sunday, December 19, 2021 4:16 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] Politics
> 
> On Sat, Dec 18, 2021 at 08:41:09PM +0000, Soft Works wrote:
> >
> >
> > > -----Original Message-----
> > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Daniel
> > > Cantarín
> > > Sent: Saturday, December 18, 2021 9:05 PM
> > > To: ffmpeg-devel@ffmpeg.org
> > > Subject: Re: [FFmpeg-devel] Politics
> > >
> > >  >
> > >  > Then you have never seen anime translations where signage in the
> > >  > videos are translated. If the subtitles are off even by one frame in
> > >  > such a case, you will see it, especially when the translated sign is
> > >  > moving with the video, and one new subtitle event is generated for
> > >  > every video frame.
> > >  > You can't sync to audio when the element you are translating is in the
> > >  > video itself, and not audio.
> > >  >
> > >  > - Hendrik
> > >
> > > This is correct, thank you.
> > >    1. If you're translating visuals, you sync to video.
> > >    2. If such visual is moving, you may move the translation in sync.
> > >
> > > I've ignored those cases, and it's correct to remark them.
> > >
> > > Yet, I understand this is done with video editing UIs, not ffmpeg
> filters,
> > > specially as it requires to visually match XY coordinates.
> > >
> > > Also, subtitle communities I know of use timings, not frames, even
> > > when doing overlays: "overlay at X:Y:Z.000 time".
> > >
> > > And a single frame means absolutelly nothing, even in this use cases.
> > >
> > > So, overall, I fail to see the serious frame-perfect subtitle-video sync
> > > problem with the patchset.
> >
> > The more the focus is moving to "a single frame" doesn't matter,
> > the more will that conversation create the impression that my patchset
> > would be lacking precision.
> >
> > In fact we're just talking about a fantasy subject instead of an
> > existing problem.
> >
> > > But there's no need to so much debate: get us some such anime sub,
> > > I get the original video somehow, do some tests, and post the results
> > > here. I'm open to do the testing work, if others are willing to help me
> > > with input examples.
> >
> > I would also be interested in an example for this, even when it doesn't
> > prove any issues.
> >
> > I'd be really glad when somebody could provide a sample (even privately),
> > it could be something I haven't seen yet.
> 
> playing devils advocate here, meaning iam not sure if the example below makes
> sense as an argument in this debate but its a interresting case which was
> not mentioned
> 
> consider a subtitle track
> consider 2 video tracks for US 30000/1001 fps and EU 25 fps
> 
> the 6th frame in the US track is at  0.2002 sec, the 5th in the EU track at
> 0.2sec

Yes, and we urgently need to buy carrots, because tomorrow, a unicorn might 
come around the corner and then we wouldn't have anything to feed it.
Also, the whole MPEG code should be re-written because next year, there 
could be a breaking change to the specs, so we should go and rewrite the code 
to prepare for that possibility. Also we should all buy a red T-Shirt, because
aliens might arrive soon at our planet, and they will probably like red colors.
;-)

At the serious side: 

First of all, if such a unicorn video would exist, and you would want to consider
this as a realistic problem, then it would be a problem of its own and independent
of ffmpeg and there's nothing that ffmpeg could remedy.
It's the time resolution of all the subtitle format specifications which do not offer
such a fine granularity. And this won't change. Simply because it's not needed
and there wouldn't be the slightest advantage.

The idea that a high resolution for subtitle timings would be required for being
able to match video timings as closely as possible is a misconception.

In fact, you couldn't do more wrong than that. :-)
(I'm sure some will figure out why)


For the unicorn video, we'd first need to know how it was produced.

From a movie? Then the two tracks would have a different playback speed, and the 
two video couldn't use the same subtitle track anyway.

I don't think that a camera exists that has a framerate high enough so that 
you could produce videos at both framerates just from picking original frames.

In most other cases, one of the videos would have been converted from the other
one. There's high-priced studio and broadcast equipment for doing such conversions,
and it requires a mix of different strategies for this.
But these can't do magic either, and when we go back to the example above, 
we'd have to start by wondering what is happening with hard cuts when such
conversions are done.
Either a hard cut would turn into a slightly smoothed transition over 2 or 3 frames,
then subs wouldn't be affected (still hitting the smooth transition.

On the other hand, when you would do hard cuts at the nearest neighbour frames
in the second video, then that would turn it in do a _different_ video.
That different video would have cuts at different time positions. 
And that means  that a single subtitle stream cannot be used for 
both videos - no matter how high the precision of the subtitle timings 
would be - it would help anything.

Best wishes,
softworkz
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 3037 bytes --]

On Sat, Dec 18, 2021 at 08:41:09PM +0000, Soft Works wrote:
> 
> 
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Daniel
> > Cantarín
> > Sent: Saturday, December 18, 2021 9:05 PM
> > To: ffmpeg-devel@ffmpeg.org
> > Subject: Re: [FFmpeg-devel] Politics
> > 
> >  >
> >  > Then you have never seen anime translations where signage in the
> >  > videos are translated. If the subtitles are off even by one frame in
> >  > such a case, you will see it, especially when the translated sign is
> >  > moving with the video, and one new subtitle event is generated for
> >  > every video frame.
> >  > You can't sync to audio when the element you are translating is in the
> >  > video itself, and not audio.
> >  >
> >  > - Hendrik
> > 
> > This is correct, thank you.
> >    1. If you're translating visuals, you sync to video.
> >    2. If such visual is moving, you may move the translation in sync.
> > 
> > I've ignored those cases, and it's correct to remark them.
> > 
> > Yet, I understand this is done with video editing UIs, not ffmpeg filters,
> > specially as it requires to visually match XY coordinates.
> > 
> > Also, subtitle communities I know of use timings, not frames, even
> > when doing overlays: "overlay at X:Y:Z.000 time".
> > 
> > And a single frame means absolutelly nothing, even in this use cases.
> > 
> > So, overall, I fail to see the serious frame-perfect subtitle-video sync
> > problem with the patchset.
> 
> The more the focus is moving to "a single frame" doesn't matter,
> the more will that conversation create the impression that my patchset
> would be lacking precision.
> 
> In fact we're just talking about a fantasy subject instead of an 
> existing problem.
> 
> > But there's no need to so much debate: get us some such anime sub,
> > I get the original video somehow, do some tests, and post the results
> > here. I'm open to do the testing work, if others are willing to help me
> > with input examples.
> 
> I would also be interested in an example for this, even when it doesn't
> prove any issues.
> 
> I'd be really glad when somebody could provide a sample (even privately),
> it could be something I haven't seen yet.

playing devils advocate here, meaning iam not sure if the example below makes
sense as an argument in this debate but its a interresting case which was
not mentioned

consider a subtitle track
consider 2 video tracks for US 30000/1001 fps and EU 25 fps

the 6th frame in the US track is at  0.2002 sec, the 5th in the EU track at 0.2sec

if these differ and you want a subtitle to either stop displaying after the
earlier or begin displaying after the earlier reliably. Then you need a 
timebase that can represent points within each such close encounter of frame
times.

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

There will always be a question for which you do not know the correct answer.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Daniel
> Cantarín
> Sent: Saturday, December 18, 2021 9:05 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] Politics
> 
>  >
>  > Then you have never seen anime translations where signage in the
>  > videos are translated. If the subtitles are off even by one frame in
>  > such a case, you will see it, especially when the translated sign is
>  > moving with the video, and one new subtitle event is generated for
>  > every video frame.
>  > You can't sync to audio when the element you are translating is in the
>  > video itself, and not audio.
>  >
>  > - Hendrik
> 
> This is correct, thank you.
>    1. If you're translating visuals, you sync to video.
>    2. If such visual is moving, you may move the translation in sync.
> 
> I've ignored those cases, and it's correct to remark them.
> 
> Yet, I understand this is done with video editing UIs, not ffmpeg filters,
> specially as it requires to visually match XY coordinates.
> 
> Also, subtitle communities I know of use timings, not frames, even
> when doing overlays: "overlay at X:Y:Z.000 time".
> 
> And a single frame means absolutelly nothing, even in this use cases.
> 
> So, overall, I fail to see the serious frame-perfect subtitle-video sync
> problem with the patchset.

The more the focus is moving to "a single frame" doesn't matter,
the more will that conversation create the impression that my patchset
would be lacking precision.

In fact we're just talking about a fantasy subject instead of an 
existing problem.

> But there's no need to so much debate: get us some such anime sub,
> I get the original video somehow, do some tests, and post the results
> here. I'm open to do the testing work, if others are willing to help me
> with input examples.

I would also be interested in an example for this, even when it doesn't
prove any issues.

I'd be really glad when somebody could provide a sample (even privately),
it could be something I haven't seen yet.

Thanks,
sw
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Daniel Cantarín]

 >
 > Then you have never seen anime translations where signage in the
 > videos are translated. If the subtitles are off even by one frame in
 > such a case, you will see it, especially when the translated sign is
 > moving with the video, and one new subtitle event is generated for
 > every video frame.
 > You can't sync to audio when the element you are translating is in the
 > video itself, and not audio.
 >
 > - Hendrik

This is correct, thank you.
   1. If you're translating visuals, you sync to video.
   2. If such visual is moving, you may move the translation in sync.

I've ignored those cases, and it's correct to remark them.

Yet, I understand this is done with video editing UIs, not ffmpeg filters,
specially as it requires to visually match XY coordinates.

Also, subtitle communities I know of use timings, not frames, even
when doing overlays: "overlay at X:Y:Z.000 time".

And a single frame means absolutelly nothing, even in this use cases.

So, overall, I fail to see the serious frame-perfect subtitle-video sync
problem with the patchset.

But there's no need to so much debate: get us some such anime sub,
I get the original video somehow, do some tests, and post the results
here. I'm open to do the testing work, if others are willing to help me
with input examples.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Hendrik
> Leppkes
> Sent: Saturday, December 18, 2021 7:31 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] Politics
> 
> On Sat, Dec 18, 2021 at 7:16 PM Daniel Cantarín <canta@canta.com.ar> wrote:
> >
> >  > This is a very lax attitude towards a serious problem that
> >  > everyone in the fansubbing community deals with. A frame
> >  > of offset is unacceptable for such use-cases, and they have
> >  > to deal with formats which made concessions to timebases
> >  > due to historical blunders.
> >
> > Please link to some such fansub community, where they're
> > discussing some serious frame-perfect sync issue. A single
> > comment in some forum would be fine. I would like to
> > understand those interests better, as all fansub communities
> > that I know of deal with audio timings, not video frames.
> >
> > I've mentioned this before; "people talk a lot about syncing
> > subs with video frames, but rarely against actual dialogue
> > audio", here:
> > http://ffmpeg.org/pipermail/ffmpeg-devel/2021-December/289290.html
> >
> > I've never seen anyone checking sub sync against video
> > frames: they always check it agains the dialogue in my
> > experience.
> 
> Then you have never seen anime translations where signage in the
> videos are translated. If the subtitles are off even by one frame in
> such a case, you will see it, especially when the translated sign is
> moving with the video, and one new subtitle event is generated for
> every video frame.
> You can't sync to audio when the element you are translating is in the
> video itself, and not audio.

I'm all-in regarding accuracy. That's out of question.


But please @Hendrilk & @Lynne: Provide an example demonstrating that
an problem that could be solved by increasing the scale of the 
subtitle timing values.

Let's look at real problems, not hypothetical ones.

Thanks,
softworkz




_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Hendrik Leppkes]

On Sat, Dec 18, 2021 at 7:16 PM Daniel Cantarín <canta@canta.com.ar> wrote:
>
>  > This is a very lax attitude towards a serious problem that
>  > everyone in the fansubbing community deals with. A frame
>  > of offset is unacceptable for such use-cases, and they have
>  > to deal with formats which made concessions to timebases
>  > due to historical blunders.
>
> Please link to some such fansub community, where they're
> discussing some serious frame-perfect sync issue. A single
> comment in some forum would be fine. I would like to
> understand those interests better, as all fansub communities
> that I know of deal with audio timings, not video frames.
>
> I've mentioned this before; "people talk a lot about syncing
> subs with video frames, but rarely against actual dialogue
> audio", here:
> http://ffmpeg.org/pipermail/ffmpeg-devel/2021-December/289290.html
>
> I've never seen anyone checking sub sync against video
> frames: they always check it agains the dialogue in my
> experience.

Then you have never seen anime translations where signage in the
videos are translated. If the subtitles are off even by one frame in
such a case, you will see it, especially when the translated sign is
moving with the video, and one new subtitle event is generated for
every video frame.
You can't sync to audio when the element you are translating is in the
video itself, and not audio.

- Hendrik
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Daniel Cantarín]

 > This is a very lax attitude towards a serious problem that
 > everyone in the fansubbing community deals with. A frame
 > of offset is unacceptable for such use-cases, and they have
 > to deal with formats which made concessions to timebases
 > due to historical blunders.

Please link to some such fansub community, where they're
discussing some serious frame-perfect sync issue. A single
comment in some forum would be fine. I would like to
understand those interests better, as all fansub communities
that I know of deal with audio timings, not video frames.

I've mentioned this before; "people talk a lot about syncing
subs with video frames, but rarely against actual dialogue
audio", here:
http://ffmpeg.org/pipermail/ffmpeg-devel/2021-December/289290.html

I've never seen anyone checking sub sync against video
frames: they always check it agains the dialogue in my
experience.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Daniel Cantarín]

 >> as well as ATSC subtitles
 >
 > There are like 2 or 3 characters in each frame. Sometimes
 > they are shown as they are coming in, sometimes only
 > when a line is completed, sometimes needs to wait
 > for subsequent frames before emitting new characters.
 > This is really not a high-precision thing.

Can confirm.

I did implemented this using video filters, precisely because
we don't have subtitle filters and I was forced to do OCR. It
works OK, but it's unrealiable when speaking about timings.

You have 2 bytes/chars per frame, non-ascii chars use 2
bytes, and there are also commands that also use one or
two bytes. This leaves you with a max speed of 60 chars
per second for a 30 FPS video stream: a condition no other
subtitle/caption format that I know of share. When there's
fast dialogue, it desyncs. It later re-syncs: it doesn't drift
away. But dialogue faster than 60 chars (including
non-ascii and command) per second does affect the
captions timings.

But there's more. Implementations in players are chaotic,
and some commands are ignored or works erratically. Also,
there are line width issues: 30 chars max by the standard,
even when some players change that. But you have to deal
with word wrapping by doing text treatment and applying
commands. Then there's 4 modes of rendering, as you say,
which you are supposed to control but then players do what
they want, and given that this are bytes added to the video
frames you gotta encode two different video streams if you
want to have two different caption configurations.

It may be the most available format in the world. But it's
also a PITA because a lot of details, and that's why I'm so
interested in finally having subtitles in filters first, and
dealing with the internal details later.


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Lynne]

18 Dec 2021, 16:43 by softworkz@hotmail.com:

>
>
>> -----Original Message-----
>> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Lynne
>> Sent: Saturday, December 18, 2021 4:17 PM
>> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
>> Subject: Re: [FFmpeg-devel] Politics
>>
>> 18 Dec 2021, 15:28 by softworkz@hotmail.com:
>>
>> >
>> >
>> >> -----Original Message-----
>> >> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Lynne
>> >> Sent: Saturday, December 18, 2021 2:33 PM
>> >> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
>> >> Subject: Re: [FFmpeg-devel] Politics
>> >>
>> >> Dec 18, 2021, 12:34 by softworkz@hotmail.com:
>> >>
>> >> >
>> >> >
>> >> >> -----Original Message-----
>> >> >> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Paul
>> B
>> >> >> Mahol
>> >> >> Sent: Saturday, December 18, 2021 11:27 AM
>> >> >> To: FFmpeg development discussions and patches <ffmpeg-
>> devel@ffmpeg.org>
>> >> >> Subject: Re: [FFmpeg-devel] Politics
>> >> >>
>> >> >> On Wed, Dec 15, 2021 at 2:34 PM Soft Works <softworkz@hotmail.com>
>> wrote:
>> >> >>
>> >> >
>> >> > [..]
>> >> >
>> >> >> > > > Once for the facts: the subtitle_pts field in AVFrame exists
>> since
>> >> >> > > > V5 of my patchset, which I have submitted on 2021-09-12.
>> >> >> > > > This has been 3 months ago. Nobody had objected its existence
>> >> >> > > > until only 2 or 3 weeks ago.
>> >> >> > >
>> >> >> > >
>> >> >> > > This is really irrelevant, please stop insisting on hacks like
>> >> >> > subtitle_pts.
>> >> >> >
>> >> >> > New idea:
>> >> >> >
>> >> >> > I could remove the three fields (subtitle_pts, subtitle_start_time,
>> >> >> > subtitle_end_time) from AVFrame and add it to AVSubtitleArea.
>> >> >> >
>> >> >> > How about that? It would allow the frame to be "clean" at least.
>> >> >> >
>> >> >>
>> >> >>
>> >> >> Yea, much better, But still original issue is not solved.
>> >> >>
>> >> >
>> >> > Yes, correct, this changes the location but not the logic.
>> >> > But this is something I could surely do. It's a bit of work,
>> >> > but it would be safe to do without breaking everything into
>> >> > dysfunctional pieces ;-)
>> >> >
>> >> > It wouldn't be my first choice since there can be multiple
>> >> > AVSubtitleAreas while only those values from the first one
>> >> > would be relevant. But when this would help to increase the
>> >> > acceptance, then it will be fine for me.
>> >> >
>> >> > Another possibility I had thought about, might be to leave
>> >> > them at the side of AVFrame, but put them in a struct field
>> >> > of AVFrame named 'subtitle_timing', which itself would have
>> >> > the fields pts, start_time, end_time.
>> >> >
>> >> >
>> >> > I did some research regarding the use of the start_time
>> >> > field. While it is used and cannot be dropped, I found
>> >> > that the following changes would be possible with moderate
>> >> > effort:
>> >> >
>> >> > - change the time base of start_time and end_time
>> >> >  to the same like subtitle_pts (AV_TIMEBASE_Q)
>> >> > - rename subtitle_start_time to subtitle_start_offset
>> >> > - rename subtitle_end_time to subtitle_duration
>> >> >  and adapt the logic everywhere where it's used
>> >> >
>> >> > In combination with the subtitle_timing struct idea it
>> >> > could then look like:
>> >> >
>> >> > frame->subtitle_timing.pts
>> >> > frame->subtitle_timing.start_offset
>> >> > frame->subtitle_timing.duration
>> >> >
>> >> > or even eliminate the pts naming and do like:
>> >> >
>> >> > frame->subtitle_timing.start
>> >> > frame->subtitle_timing.start_offset
>> >> > frame->subtitle_timing.duration
>> >> >
>> >> > or still move them to AVSubtitleArea, which wouldn't
>> >> > be that nice to access and require to check the
>> >> > subtitle_area_nb value wherever it needs to be
>> >> > accessed:
>> >> >
>> >> > frame->subtitle_areas[0].start
>> >> > frame->subtitle_areas[0].start_offset
>> >> > frame->subtitle_areas[0].duration
>> >> >
>> >> >
>> >> > Please let me (all) know whether one of those suggestions would
>> >> > be an acceptable compromise.
>> >> >
>> >>
>> >> Renaming the fields doesn't get around the issue that they're
>> >> still overriding fields with a different meaning from the
>> >> AVFrame structure. That's not really a compromise since they're
>> >> still there.
>> >>
>> >
>> > I'm suggesting those things that are doable.
>> >
>> >
>> >> I also am not accepting a hardcoded timebase of microseconds.
>> >> Rounding really matters for subtitles, since presenting them
>> >> a frame early or late is unacceptable, so I'd like a time_base
>> >> field for the timestamps.
>> >>
>> >
>> > I can't follow. With 120fps, the frame duration is 8300 microseconds.
>> > And you say that's not enough to hit the right frame?
>> >
>> > None of the subtitle storage formats has a precision higher than
>> > milliseconds, often it's less.
>> >
>> > Finally, a fixed time base avoids frequent re-scaling and that
>> > in turn means less rounding errors.
>> >
>>
>> A timebase completely eliminates all scaling.
>> Bitmap subtitles exist,
>>
>> as well as ATSC subtitles
>>
>
> There are like 2 or 3 characters in each frame. Sometimes
> they are shown as they are coming in, sometimes only 
> when a line is completed, sometimes needs to wait
> for subsequent frames before emitting new characters.
> This is really not a high-precision thing.
>
>
> , which
>
>> are embedded in frames, and therefore have the same timestamp
>> and timebase as the frames themselves. Forcing them to be
>> rescaled to whatever timebase you thought is good enough is
>> a bad, inflexible design choice.
>>
>
> What precision do you want?
>

The exact same time_base used for the video, optimally. That
eliminates all rounding issues. Otherwise, either the format's
native timestamps, or otherwise the container's timestamps
for each packet, if any, whichever is more accurate.
But definitely not just using AV_TIME_BASE for everything and
calling it good enough.


> This is nothing like audio. You just need to hit one
> frame or the next frame, nothing in-between because there
> is nothing in between.
>
> So what's this about? Videos with 1 Million fps?
>
>> Also, some text subtitle formats have a timestamp precision
>> much greater than a millisecond, like Ogg Kate.
>>
>
> But only because it uses the same bitstream layout as for
> audio and video.
>
>> We don't
>> support it yet, but you never know. An API must be able to
>> handle everything.
>>
>
> It will handle this perfectly fine.
>
>
> I'm afraid, but IMO you are trying to construct cases which 
> do not even have a theoretical value.
>

This is a very lax attitude towards a serious problem that
everyone in the fansubbing community deals with. A frame
of offset is unacceptable for such use-cases, and they have
to deal with formats which made concessions to timebases
due to historical blunders.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Lynne
> Sent: Saturday, December 18, 2021 4:17 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] Politics
> 
> 18 Dec 2021, 15:28 by softworkz@hotmail.com:
> 
> >
> >
> >> -----Original Message-----
> >> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Lynne
> >> Sent: Saturday, December 18, 2021 2:33 PM
> >> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> >> Subject: Re: [FFmpeg-devel] Politics
> >>
> >> Dec 18, 2021, 12:34 by softworkz@hotmail.com:
> >>
> >> >
> >> >
> >> >> -----Original Message-----
> >> >> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Paul
> B
> >> >> Mahol
> >> >> Sent: Saturday, December 18, 2021 11:27 AM
> >> >> To: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org>
> >> >> Subject: Re: [FFmpeg-devel] Politics
> >> >>
> >> >> On Wed, Dec 15, 2021 at 2:34 PM Soft Works <softworkz@hotmail.com>
> wrote:
> >> >>
> >> >
> >> > [..]
> >> >
> >> >> > > > Once for the facts: the subtitle_pts field in AVFrame exists
> since
> >> >> > > > V5 of my patchset, which I have submitted on 2021-09-12.
> >> >> > > > This has been 3 months ago. Nobody had objected its existence
> >> >> > > > until only 2 or 3 weeks ago.
> >> >> > >
> >> >> > >
> >> >> > > This is really irrelevant, please stop insisting on hacks like
> >> >> > subtitle_pts.
> >> >> >
> >> >> > New idea:
> >> >> >
> >> >> > I could remove the three fields (subtitle_pts, subtitle_start_time,
> >> >> > subtitle_end_time) from AVFrame and add it to AVSubtitleArea.
> >> >> >
> >> >> > How about that? It would allow the frame to be "clean" at least.
> >> >> >
> >> >>
> >> >>
> >> >> Yea, much better, But still original issue is not solved.
> >> >>
> >> >
> >> > Yes, correct, this changes the location but not the logic.
> >> > But this is something I could surely do. It's a bit of work,
> >> > but it would be safe to do without breaking everything into
> >> > dysfunctional pieces ;-)
> >> >
> >> > It wouldn't be my first choice since there can be multiple
> >> > AVSubtitleAreas while only those values from the first one
> >> > would be relevant. But when this would help to increase the
> >> > acceptance, then it will be fine for me.
> >> >
> >> > Another possibility I had thought about, might be to leave
> >> > them at the side of AVFrame, but put them in a struct field
> >> > of AVFrame named 'subtitle_timing', which itself would have
> >> > the fields pts, start_time, end_time.
> >> >
> >> >
> >> > I did some research regarding the use of the start_time
> >> > field. While it is used and cannot be dropped, I found
> >> > that the following changes would be possible with moderate
> >> > effort:
> >> >
> >> > - change the time base of start_time and end_time
> >> >  to the same like subtitle_pts (AV_TIMEBASE_Q)
> >> > - rename subtitle_start_time to subtitle_start_offset
> >> > - rename subtitle_end_time to subtitle_duration
> >> >  and adapt the logic everywhere where it's used
> >> >
> >> > In combination with the subtitle_timing struct idea it
> >> > could then look like:
> >> >
> >> > frame->subtitle_timing.pts
> >> > frame->subtitle_timing.start_offset
> >> > frame->subtitle_timing.duration
> >> >
> >> > or even eliminate the pts naming and do like:
> >> >
> >> > frame->subtitle_timing.start
> >> > frame->subtitle_timing.start_offset
> >> > frame->subtitle_timing.duration
> >> >
> >> > or still move them to AVSubtitleArea, which wouldn't
> >> > be that nice to access and require to check the
> >> > subtitle_area_nb value wherever it needs to be
> >> > accessed:
> >> >
> >> > frame->subtitle_areas[0].start
> >> > frame->subtitle_areas[0].start_offset
> >> > frame->subtitle_areas[0].duration
> >> >
> >> >
> >> > Please let me (all) know whether one of those suggestions would
> >> > be an acceptable compromise.
> >> >
> >>
> >> Renaming the fields doesn't get around the issue that they're
> >> still overriding fields with a different meaning from the
> >> AVFrame structure. That's not really a compromise since they're
> >> still there.
> >>
> >
> > I'm suggesting those things that are doable.
> >
> >
> >> I also am not accepting a hardcoded timebase of microseconds.
> >> Rounding really matters for subtitles, since presenting them
> >> a frame early or late is unacceptable, so I'd like a time_base
> >> field for the timestamps.
> >>
> >
> > I can't follow. With 120fps, the frame duration is 8300 microseconds.
> > And you say that's not enough to hit the right frame?
> >
> > None of the subtitle storage formats has a precision higher than
> > milliseconds, often it's less.
> >
> > Finally, a fixed time base avoids frequent re-scaling and that
> > in turn means less rounding errors.
> >
> 
> A timebase completely eliminates all scaling.
> Bitmap subtitles exist, 

> as well as ATSC subtitles

There are like 2 or 3 characters in each frame. Sometimes
they are shown as they are coming in, sometimes only 
when a line is completed, sometimes needs to wait
for subsequent frames before emitting new characters.
This is really not a high-precision thing.


, which
> are embedded in frames, and therefore have the same timestamp
> and timebase as the frames themselves. Forcing them to be
> rescaled to whatever timebase you thought is good enough is
> a bad, inflexible design choice.

What precision do you want?

This is nothing like audio. You just need to hit one
frame or the next frame, nothing in-between because there
is nothing in between.

So what's this about? Videos with 1 Million fps?

> Also, some text subtitle formats have a timestamp precision
> much greater than a millisecond, like Ogg Kate. 

But only because it uses the same bitstream layout as for
audio and video.

> We don't
> support it yet, but you never know. An API must be able to
> handle everything.

It will handle this perfectly fine.


I'm afraid, but IMO you are trying to construct cases which  
do not even have a theoretical value.


Kind regards,
softworkz

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Daniel Cantarín]

> I also am not accepting a hardcoded timebase of microseconds.
> Rounding really matters for subtitles, since presenting them
> a frame early or late is unacceptable

That's simply not true.

I don't accept or deny a hardcoded microseconds timebase;
that's beyond my knowledge to judge. What I say is not true is
the other sentence: "presenting them a frame early or late is
unnaceptable".

That's a gross exaggeration, and should not be a blocker.
In my country everyone uses subtitles since decades ago, and
people cares about timing only when there's seconds of desync.
Yeah, seconds in plural: a single second most likely would be
accepted. We're talking about dozens of frames here.

Of course, I wouldn't say "innacurate is right". But I wouln't
also block other people's work using unreal standards.

Now, DRIFTING would be a different beast, and microseconds
leaks should be a serious issue for 24/7 streams. I don't see
people debating such problem, and "a sigle frame off" is a
wrong angle for talking milliseconds precision and rounding.




Also, I did some tests with softworkz code, and timings seems
working fine with production inputs. I saw several people here
talking about complex graphs with different timebases, and
changes between subtitle formats. I believe the debate would
be much less abstract if somebody could kindly put some
example here we could all test for ourselves.

I understand nobody could have a full idea of all possible
scenarios: that would be unfair to ask for. But I believe
people with observations about possible breaking use cases
would have such use cases more likely available in mind and
could share them with us.

I DID actually find problems with timings in softworkz code.
But those problems were anything like the things debated here.
You see, I've tried to use setpts and asetpts in my graphs,
yet there's no ssetpts. Other filters had the same problem.

Also, there are live-streaming muxers and codecs that does not
support subtitles. See this, for example:
https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/hls.c#L506
Or this:
https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/dashenc.c#L1066

But I believe those issues can be handled isolated once
there's subtitle filtering implemented, so I don't consider
them part of this debate per se.


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Lynne]

18 Dec 2021, 15:28 by softworkz@hotmail.com:

>
>
>> -----Original Message-----
>> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Lynne
>> Sent: Saturday, December 18, 2021 2:33 PM
>> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
>> Subject: Re: [FFmpeg-devel] Politics
>>
>> Dec 18, 2021, 12:34 by softworkz@hotmail.com:
>>
>> >
>> >
>> >> -----Original Message-----
>> >> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Paul B
>> >> Mahol
>> >> Sent: Saturday, December 18, 2021 11:27 AM
>> >> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
>> >> Subject: Re: [FFmpeg-devel] Politics
>> >>
>> >> On Wed, Dec 15, 2021 at 2:34 PM Soft Works <softworkz@hotmail.com> wrote:
>> >>
>> >
>> > [..]
>> >
>> >> > > > Once for the facts: the subtitle_pts field in AVFrame exists since
>> >> > > > V5 of my patchset, which I have submitted on 2021-09-12.
>> >> > > > This has been 3 months ago. Nobody had objected its existence
>> >> > > > until only 2 or 3 weeks ago.
>> >> > >
>> >> > >
>> >> > > This is really irrelevant, please stop insisting on hacks like
>> >> > subtitle_pts.
>> >> >
>> >> > New idea:
>> >> >
>> >> > I could remove the three fields (subtitle_pts, subtitle_start_time,
>> >> > subtitle_end_time) from AVFrame and add it to AVSubtitleArea.
>> >> >
>> >> > How about that? It would allow the frame to be "clean" at least.
>> >> >
>> >>
>> >>
>> >> Yea, much better, But still original issue is not solved.
>> >>
>> >
>> > Yes, correct, this changes the location but not the logic.
>> > But this is something I could surely do. It's a bit of work,
>> > but it would be safe to do without breaking everything into
>> > dysfunctional pieces ;-)
>> >
>> > It wouldn't be my first choice since there can be multiple
>> > AVSubtitleAreas while only those values from the first one
>> > would be relevant. But when this would help to increase the
>> > acceptance, then it will be fine for me.
>> >
>> > Another possibility I had thought about, might be to leave
>> > them at the side of AVFrame, but put them in a struct field
>> > of AVFrame named 'subtitle_timing', which itself would have
>> > the fields pts, start_time, end_time.
>> >
>> >
>> > I did some research regarding the use of the start_time
>> > field. While it is used and cannot be dropped, I found
>> > that the following changes would be possible with moderate
>> > effort:
>> >
>> > - change the time base of start_time and end_time
>> >  to the same like subtitle_pts (AV_TIMEBASE_Q)
>> > - rename subtitle_start_time to subtitle_start_offset
>> > - rename subtitle_end_time to subtitle_duration
>> >  and adapt the logic everywhere where it's used
>> >
>> > In combination with the subtitle_timing struct idea it
>> > could then look like:
>> >
>> > frame->subtitle_timing.pts
>> > frame->subtitle_timing.start_offset
>> > frame->subtitle_timing.duration
>> >
>> > or even eliminate the pts naming and do like:
>> >
>> > frame->subtitle_timing.start
>> > frame->subtitle_timing.start_offset
>> > frame->subtitle_timing.duration
>> >
>> > or still move them to AVSubtitleArea, which wouldn't
>> > be that nice to access and require to check the
>> > subtitle_area_nb value wherever it needs to be
>> > accessed:
>> >
>> > frame->subtitle_areas[0].start
>> > frame->subtitle_areas[0].start_offset
>> > frame->subtitle_areas[0].duration
>> >
>> >
>> > Please let me (all) know whether one of those suggestions would
>> > be an acceptable compromise.
>> >
>>
>> Renaming the fields doesn't get around the issue that they're
>> still overriding fields with a different meaning from the
>> AVFrame structure. That's not really a compromise since they're
>> still there.
>>
>
> I'm suggesting those things that are doable. 
>
>
>> I also am not accepting a hardcoded timebase of microseconds.
>> Rounding really matters for subtitles, since presenting them
>> a frame early or late is unacceptable, so I'd like a time_base
>> field for the timestamps.
>>
>
> I can't follow. With 120fps, the frame duration is 8300 microseconds.
> And you say that's not enough to hit the right frame?
>
> None of the subtitle storage formats has a precision higher than
> milliseconds, often it's less. 
>
> Finally, a fixed time base avoids frequent re-scaling and that
> in turn means less rounding errors.
>

A timebase completely eliminates all scaling.
Bitmap subtitles exist, as well as ATSC subtitles, which
are embedded in frames, and therefore have the same timestamp
and timebase as the frames themselves. Forcing them to be
rescaled to whatever timebase you thought is good enough is
a bad, inflexible design choice.
Also, some text subtitle formats have a timestamp precision
much greater than a millisecond, like Ogg Kate. We don't
support it yet, but you never know. An API must be able to
handle everything.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Lynne
> Sent: Saturday, December 18, 2021 2:33 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] Politics
> 
> Dec 18, 2021, 12:34 by softworkz@hotmail.com:
> 
> >
> >
> >> -----Original Message-----
> >> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Paul B
> >> Mahol
> >> Sent: Saturday, December 18, 2021 11:27 AM
> >> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> >> Subject: Re: [FFmpeg-devel] Politics
> >>
> >> On Wed, Dec 15, 2021 at 2:34 PM Soft Works <softworkz@hotmail.com> wrote:
> >>
> >
> > [..]
> >
> >> > > > Once for the facts: the subtitle_pts field in AVFrame exists since
> >> > > > V5 of my patchset, which I have submitted on 2021-09-12.
> >> > > > This has been 3 months ago. Nobody had objected its existence
> >> > > > until only 2 or 3 weeks ago.
> >> > >
> >> > >
> >> > > This is really irrelevant, please stop insisting on hacks like
> >> > subtitle_pts.
> >> >
> >> > New idea:
> >> >
> >> > I could remove the three fields (subtitle_pts, subtitle_start_time,
> >> > subtitle_end_time) from AVFrame and add it to AVSubtitleArea.
> >> >
> >> > How about that? It would allow the frame to be "clean" at least.
> >> >
> >>
> >>
> >> Yea, much better, But still original issue is not solved.
> >>
> >
> > Yes, correct, this changes the location but not the logic.
> > But this is something I could surely do. It's a bit of work,
> > but it would be safe to do without breaking everything into
> > dysfunctional pieces ;-)
> >
> > It wouldn't be my first choice since there can be multiple
> > AVSubtitleAreas while only those values from the first one
> > would be relevant. But when this would help to increase the
> > acceptance, then it will be fine for me.
> >
> > Another possibility I had thought about, might be to leave
> > them at the side of AVFrame, but put them in a struct field
> > of AVFrame named 'subtitle_timing', which itself would have
> > the fields pts, start_time, end_time.
> >
> >
> > I did some research regarding the use of the start_time
> > field. While it is used and cannot be dropped, I found
> > that the following changes would be possible with moderate
> > effort:
> >
> > - change the time base of start_time and end_time
> >  to the same like subtitle_pts (AV_TIMEBASE_Q)
> > - rename subtitle_start_time to subtitle_start_offset
> > - rename subtitle_end_time to subtitle_duration
> >  and adapt the logic everywhere where it's used
> >
> > In combination with the subtitle_timing struct idea it
> > could then look like:
> >
> > frame->subtitle_timing.pts
> > frame->subtitle_timing.start_offset
> > frame->subtitle_timing.duration
> >
> > or even eliminate the pts naming and do like:
> >
> > frame->subtitle_timing.start
> > frame->subtitle_timing.start_offset
> > frame->subtitle_timing.duration
> >
> > or still move them to AVSubtitleArea, which wouldn't
> > be that nice to access and require to check the
> > subtitle_area_nb value wherever it needs to be
> > accessed:
> >
> > frame->subtitle_areas[0].start
> > frame->subtitle_areas[0].start_offset
> > frame->subtitle_areas[0].duration
> >
> >
> > Please let me (all) know whether one of those suggestions would
> > be an acceptable compromise.
> >
> 
> Renaming the fields doesn't get around the issue that they're
> still overriding fields with a different meaning from the
> AVFrame structure. That's not really a compromise since they're
> still there.

I'm suggesting those things that are doable. 


> I also am not accepting a hardcoded timebase of microseconds.
> Rounding really matters for subtitles, since presenting them
> a frame early or late is unacceptable, so I'd like a time_base
> field for the timestamps.

I can't follow. With 120fps, the frame duration is 8300 microseconds.
And you say that's not enough to hit the right frame?

None of the subtitle storage formats has a precision higher than
milliseconds, often it's less. 

Finally, a fixed time base avoids frequent re-scaling and that
in turn means less rounding errors.

Kind regards,
softworkz

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Lynne]

Dec 18, 2021, 12:34 by softworkz@hotmail.com:

>
>
>> -----Original Message-----
>> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Paul B
>> Mahol
>> Sent: Saturday, December 18, 2021 11:27 AM
>> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
>> Subject: Re: [FFmpeg-devel] Politics
>>
>> On Wed, Dec 15, 2021 at 2:34 PM Soft Works <softworkz@hotmail.com> wrote:
>>
>
> [..]
>
>> > > > Once for the facts: the subtitle_pts field in AVFrame exists since
>> > > > V5 of my patchset, which I have submitted on 2021-09-12.
>> > > > This has been 3 months ago. Nobody had objected its existence
>> > > > until only 2 or 3 weeks ago.
>> > >
>> > >
>> > > This is really irrelevant, please stop insisting on hacks like
>> > subtitle_pts.
>> >
>> > New idea:
>> >
>> > I could remove the three fields (subtitle_pts, subtitle_start_time,
>> > subtitle_end_time) from AVFrame and add it to AVSubtitleArea.
>> >
>> > How about that? It would allow the frame to be "clean" at least.
>> >
>>
>>
>> Yea, much better, But still original issue is not solved.
>>
>
> Yes, correct, this changes the location but not the logic.
> But this is something I could surely do. It's a bit of work, 
> but it would be safe to do without breaking everything into
> dysfunctional pieces ;-)
>
> It wouldn't be my first choice since there can be multiple
> AVSubtitleAreas while only those values from the first one
> would be relevant. But when this would help to increase the
> acceptance, then it will be fine for me.
>
> Another possibility I had thought about, might be to leave
> them at the side of AVFrame, but put them in a struct field
> of AVFrame named 'subtitle_timing', which itself would have
> the fields pts, start_time, end_time.
>
>
> I did some research regarding the use of the start_time
> field. While it is used and cannot be dropped, I found
> that the following changes would be possible with moderate
> effort:
>
> - change the time base of start_time and end_time
>  to the same like subtitle_pts (AV_TIMEBASE_Q)
> - rename subtitle_start_time to subtitle_start_offset
> - rename subtitle_end_time to subtitle_duration
>  and adapt the logic everywhere where it's used
>
> In combination with the subtitle_timing struct idea it 
> could then look like:
>
> frame->subtitle_timing.pts
> frame->subtitle_timing.start_offset
> frame->subtitle_timing.duration
>
> or even eliminate the pts naming and do like:
>
> frame->subtitle_timing.start
> frame->subtitle_timing.start_offset
> frame->subtitle_timing.duration
>
> or still move them to AVSubtitleArea, which wouldn't 
> be that nice to access and require to check the
> subtitle_area_nb value wherever it needs to be
> accessed:
>
> frame->subtitle_areas[0].start
> frame->subtitle_areas[0].start_offset
> frame->subtitle_areas[0].duration
>
>
> Please let me (all) know whether one of those suggestions would
> be an acceptable compromise.
>

Renaming the fields doesn't get around the issue that they're
still overriding fields with a different meaning from the
AVFrame structure. That's not really a compromise since they're
still there.

I also am not accepting a hardcoded timebase of microseconds.
Rounding really matters for subtitles, since presenting them
a frame early or late is unacceptable, so I'd like a time_base
field for the timestamps.

Like elenril said, this is an API that'll stay for decades at the least,
and replacing and changing will be impossible once it's merged.
Just stuffing the entire AVSubtitle field in a frame simply won't do.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Paul B
> Mahol
> Sent: Saturday, December 18, 2021 11:27 AM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] Politics
> 
> On Wed, Dec 15, 2021 at 2:34 PM Soft Works <softworkz@hotmail.com> wrote:
> 

[..]

> > > > Once for the facts: the subtitle_pts field in AVFrame exists since
> > > > V5 of my patchset, which I have submitted on 2021-09-12.
> > > > This has been 3 months ago. Nobody had objected its existence
> > > > until only 2 or 3 weeks ago.
> > >
> > >
> > > This is really irrelevant, please stop insisting on hacks like
> > subtitle_pts.
> >
> > New idea:
> >
> > I could remove the three fields (subtitle_pts, subtitle_start_time,
> > subtitle_end_time) from AVFrame and add it to AVSubtitleArea.
> >
> > How about that? It would allow the frame to be "clean" at least.
> >
> 
> 
> Yea, much better, But still original issue is not solved.

Yes, correct, this changes the location but not the logic.
But this is something I could surely do. It's a bit of work, 
but it would be safe to do without breaking everything into
dysfunctional pieces ;-)

It wouldn't be my first choice since there can be multiple
AVSubtitleAreas while only those values from the first one
would be relevant. But when this would help to increase the
acceptance, then it will be fine for me.

Another possibility I had thought about, might be to leave
them at the side of AVFrame, but put them in a struct field
of AVFrame named 'subtitle_timing', which itself would have
the fields pts, start_time, end_time.


I did some research regarding the use of the start_time
field. While it is used and cannot be dropped, I found
that the following changes would be possible with moderate
effort:

- change the time base of start_time and end_time
  to the same like subtitle_pts (AV_TIMEBASE_Q)
- rename subtitle_start_time to subtitle_start_offset
- rename subtitle_end_time to subtitle_duration
  and adapt the logic everywhere where it's used

In combination with the subtitle_timing struct idea it 
could then look like:

frame->subtitle_timing.pts
frame->subtitle_timing.start_offset
frame->subtitle_timing.duration

or even eliminate the pts naming and do like:

frame->subtitle_timing.start
frame->subtitle_timing.start_offset
frame->subtitle_timing.duration

or still move them to AVSubtitleArea, which wouldn't 
be that nice to access and require to check the
subtitle_area_nb value wherever it needs to be
accessed:

frame->subtitle_areas[0].start
frame->subtitle_areas[0].start_offset
frame->subtitle_areas[0].duration


Please let me (all) know whether one of those suggestions would
be an acceptable compromise.


PS: I'll respond to the other question shortly

Thanks,
softworkz

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Paul B Mahol]

On Wed, Dec 15, 2021 at 2:34 PM Soft Works <softworkz@hotmail.com> wrote:

>
>
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Paul B
> > Mahol
> > Sent: Wednesday, December 15, 2021 10:34 AM
> > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] Politics
> >
> > On Tue, Dec 14, 2021 at 5:00 AM Soft Works <softworkz@hotmail.com>
> wrote:
> >
> > >
> > >
> > > > -----Original Message-----
> > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of
> Daniel
> > > > Cantarín
> > > > Sent: Tuesday, December 14, 2021 4:05 AM
> > > > To: ffmpeg-devel@ffmpeg.org
> > > > Subject: Re: [FFmpeg-devel] Politics
> > > >
> > > >
> > > >  >
> > > >  > As mentioned already, I have an offer to make. It might not be
> exactly
> > > >  > what you want, but it's all you can get.
> > > >  >
> > > >  > Everybody will need to make up his mind and decide whether the
> > > benefits
> > > >  > will outweigh the drawback from one's own point of view - or not.
> > > >  >
> > > >
> > > >
> > > > I don't feel I have a voice here in the same way other devs do: I
> don't
> > > > even have a week here, it would be a joke speaking as if I were at
> the
> > > > same level that people with 10+ years of experience in the community.
> > > > So let me be clear: I'm not at the same level, my words can't be
> valued
> > > > the same as other people here.
> > > >
> > > > Also, I'm very doubtful of how to express myself here, because when I
> > > > naively tried to intervene in the patch debate I later felt all I
> > > achieved
> > > > was to ignite old grudges between very tired people that was dealing
> > > > with this patch for months. For moments, I feel I should just shut my
> > > > mouth and let the thing be, without kinda embarrasing myself and
> > > > unintendly triggering others. It's very awkward and sad.
> > > >
> > > > Yet, I believe I can note some common sense and experience points
> > > > from my perspective, even without being sure about its value. My
> > > > hope is to note something perhaps other people does not realize.
> > > > I've done it before, but I see other devs involved in this thread,
> and
> > > > so I believe doing it again is in order. Specially if this thread is
> the
> > > > final frontier between accepting or rejecting the patch. So, it's
> just a
> > > > testimony from my experience, to be noted in the thread.
> > > >
> > > > I absolutelly love the use cases implemented by this patch. It's been
> > > > years since there's people wanting this, me included, and specially
> > > > the live streaming people will benefit a lot from it.
> > > >
> > > > Live streaming has problems that file handling does not. Timings are
> a
> > > > curse, you can't probe just like that so your setup needs always
> extra
> > > > parameters and filters, there's no start and finish in the same sense
> > > > as it happens with files, there's the sparseness problem (which is a
> big
> > > > deal for filters), and so on. Yet, so often people evaluate live
> > > streaming
> > > > problems with a file mindset, that most of the time this problems are
> > > > understood as some kind of "border cases" with no merit for "a hack";
> > > > where that hack is usually "something I can do right now without
> > > > breaking anything else".
> > > >
> > > > All of this stuff can be done using other tools when it comes to
> files.
> > > > But real time? Live? Forget it: that's another deal, and this patch
> does
> > > > the job. And not only that: I've tested it, and it works amazingly
> well.
> > > > It's much better than what I ever had expected to find. I was looking
> > > > for useful bits and pieces of code, ideas even, that I could use to
> > > > solve live streaming problems. What I've found feels the same as
> > > > when any of us found ffmpeg for the first time when looking for a
> > > > tool: "this is amazing, it does everything!"
> > > >
> > > > Of course, this does not means an even more deep refactor isn't
> > > > actually needed. However, we live streaming people keep waiting for
> > > > that use cases for years (there can be found discussions a decade
> > > > old), while the "proper way to do it" never happens.
> > > >
> > > > That sounds rude to the people involved. But my intention is not to
> be
> > > > rude, but to express how it feels to be on the other side of the
> > > > development cycle. I said one day "I'll do it myself", then I found
> the
> > > > thing beyond me: I don't have the time to do all that work, to learn
> all
> > > > that knowledge and articulate it in a way other people with much
> > > > more knowledge than I have could accept. It's a lot. And I believe I
> > > > can speak for lots of us when I say to have keeped our code private
> > > > knowing it will never be accepted mainline as it is, because it does
> > > > not refactors ffmpeg or libavfilter and just solves something
> quickly.
> > > >
> > > > softworkz is now clearly tired of this patchset, and so are the devs
> > > > discussing it with him. When I didn't knew that this thing had been
> > > > going on for six months, I was hoping to be able to somehow get in
> > > > the middle of the debate and try to unblock it. I was hoping to help
> > > > coding here and there, testing this and that, comparing ideas...
> > > > I was actually beginning to analize the code, in order to see if I
> was
> > > > able to remove the controversial subtitle_pts property. I realize now
> > > > I was sadly too late to the party.
> > > >
> > > > Yet, here we are, with code that actually do the job, and even
> > > > performs pretty well.
> > > >
> > > > With all this in mind, here are my feelings about this:
> > > >
> > > >    - I could not care less about subtitle_pts. I know it's
> questionable,
> > > >      but screw it: its also questionable to wait for years with live
> > > >      streams problems as some kind of second class citizens. This
> > > >      property and all around this patchset (the good and the bad) are
> > > >      but accidents of the project's history, and those will keep on
> > > >      happening no matter what. We CAN fix subtitle_pts later,
> > > >      without having to wait forever for the concensual
> > > >      implementation in order to have the use cases implemented.
> > > >      That's not a crime: not against "good programing", and neither
> > > >      against ffmpeg.
> > > >
> > > >    - I don't think subtitle_pts is the real problem here. I think
> there
> > > >      are grudges between devs, and things going on I don't fully
> > > >      understand. Sadly, as I have no more time to participate in
> > > >      this development (given that everybody's tired, and a call for
> > > >      a vote seems imminent), I can't also read the 6 months of
> > > >      debates to gain a better understanding before expressing
> > > >      an opinion. Sorry devs, you deserve better than this
> > > >      ignorance of mine. Perhaps somebody could make a list of
> > > >      relevant links to discussions before the vote?
> > > >
> > > >    - I'm more worried about not breaking anything. I saw some
> > > >      notes from Mr. Niedermayer, and could reproduce myself some
> > > >      problems. However, softworkz seems to be willing to fix that
> > > >      kind of issues, so doesn't seem to be a big deal. I see he
> > > >      actually posted a new version of his patchset minutes ago,
> > > >      so that's great. The point is, I would find this kind of things
> > > >      actually blockers, and not subtitle_pts.
> > > >
> > > >    - If this patchset get rejected, I believe a proper consensual
> > > >      design is in order. Consensual between the relevant devs,
> > > >      of course. But so far, I don't see it linked in the debates.
> > > >      Most likely it is somewhere and I just didn't saw it. But my
> > > >      point is that rejecting this kind of works should be done
> > > >      with a proper explanation, and the acceptable design
> > > >      should be part of that explanation. With that guidelines,
> > > >      perhaps others like me could progressively implement the
> > > >      thing. But, to be honest, rejecting this without a clear and
> > > >      consise designs counterpart (and not only sparse ideas, as
> > > >      we all have some of those), would be frankly rude to both
> > > >      the people who did the job and the people who wants the
> > > >      use cases: because this works, and it's right here.
> > > >
> > > >    - I would also note that perhaps tiredness would be solved
> > > >      by resting, and not either by an ultimatum or a complete
> > > >      dismissal of the propossal. I would be very happy to
> > > >      collaborate (as I could, which I fear may not be much) if
> > > >      you people happen to just find the energy to take a look
> > > >      at the thing again without so much... err... "painful
> > > >      exchanges" (I speak spanish, sorry, my english is limited).
> > > >      Finally having this use cases should be something to
> > > >      celebrate, and not something depressing. I know, this is
> > > >      some kinda hippie thinking of mine. But I feel it's worth
> > > >      the note anyways.
> > > >
> > > >
> > > > Just my 2 cents.
> > > > Thanks,
> > > > Daniel.
> > >
> > > Hi Daniel,
> > >
> > > thanks for your comments. I basically think that others should
> > > respond to this, but I'd like to add a few remarks.
> > >
> > > Once for the facts: the subtitle_pts field in AVFrame exists since
> > > V5 of my patchset, which I have submitted on 2021-09-12.
> > > This has been 3 months ago. Nobody had objected its existence
> > > until only 2 or 3 weeks ago.
> >
> >
> > This is really irrelevant, please stop insisting on hacks like
> subtitle_pts.
>
> New idea:
>
> I could remove the three fields (subtitle_pts, subtitle_start_time,
> subtitle_end_time) from AVFrame and add it to AVSubtitleArea.
>
> How about that? It would allow the frame to be "clean" at least.
>


Yea, much better, But still original issue is not solved.

Also what about subtitle formats negotiation and auto inserting filters
when needed?


>
> sw
>
>
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] Politics

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Paul B
> Mahol
> Sent: Wednesday, December 15, 2021 10:34 AM
> To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] Politics
> 
> On Tue, Dec 14, 2021 at 5:00 AM Soft Works <softworkz@hotmail.com> wrote:
> 
> >
> >
> > > -----Original Message-----
> > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Daniel
> > > Cantarín
> > > Sent: Tuesday, December 14, 2021 4:05 AM
> > > To: ffmpeg-devel@ffmpeg.org
> > > Subject: Re: [FFmpeg-devel] Politics
> > >
> > >
> > >  >
> > >  > As mentioned already, I have an offer to make. It might not be exactly
> > >  > what you want, but it's all you can get.
> > >  >
> > >  > Everybody will need to make up his mind and decide whether the
> > benefits
> > >  > will outweigh the drawback from one's own point of view - or not.
> > >  >
> > >
> > >
> > > I don't feel I have a voice here in the same way other devs do: I don't
> > > even have a week here, it would be a joke speaking as if I were at the
> > > same level that people with 10+ years of experience in the community.
> > > So let me be clear: I'm not at the same level, my words can't be valued
> > > the same as other people here.
> > >
> > > Also, I'm very doubtful of how to express myself here, because when I
> > > naively tried to intervene in the patch debate I later felt all I
> > achieved
> > > was to ignite old grudges between very tired people that was dealing
> > > with this patch for months. For moments, I feel I should just shut my
> > > mouth and let the thing be, without kinda embarrasing myself and
> > > unintendly triggering others. It's very awkward and sad.
> > >
> > > Yet, I believe I can note some common sense and experience points
> > > from my perspective, even without being sure about its value. My
> > > hope is to note something perhaps other people does not realize.
> > > I've done it before, but I see other devs involved in this thread, and
> > > so I believe doing it again is in order. Specially if this thread is the
> > > final frontier between accepting or rejecting the patch. So, it's just a
> > > testimony from my experience, to be noted in the thread.
> > >
> > > I absolutelly love the use cases implemented by this patch. It's been
> > > years since there's people wanting this, me included, and specially
> > > the live streaming people will benefit a lot from it.
> > >
> > > Live streaming has problems that file handling does not. Timings are a
> > > curse, you can't probe just like that so your setup needs always extra
> > > parameters and filters, there's no start and finish in the same sense
> > > as it happens with files, there's the sparseness problem (which is a big
> > > deal for filters), and so on. Yet, so often people evaluate live
> > streaming
> > > problems with a file mindset, that most of the time this problems are
> > > understood as some kind of "border cases" with no merit for "a hack";
> > > where that hack is usually "something I can do right now without
> > > breaking anything else".
> > >
> > > All of this stuff can be done using other tools when it comes to files.
> > > But real time? Live? Forget it: that's another deal, and this patch does
> > > the job. And not only that: I've tested it, and it works amazingly well.
> > > It's much better than what I ever had expected to find. I was looking
> > > for useful bits and pieces of code, ideas even, that I could use to
> > > solve live streaming problems. What I've found feels the same as
> > > when any of us found ffmpeg for the first time when looking for a
> > > tool: "this is amazing, it does everything!"
> > >
> > > Of course, this does not means an even more deep refactor isn't
> > > actually needed. However, we live streaming people keep waiting for
> > > that use cases for years (there can be found discussions a decade
> > > old), while the "proper way to do it" never happens.
> > >
> > > That sounds rude to the people involved. But my intention is not to be
> > > rude, but to express how it feels to be on the other side of the
> > > development cycle. I said one day "I'll do it myself", then I found the
> > > thing beyond me: I don't have the time to do all that work, to learn all
> > > that knowledge and articulate it in a way other people with much
> > > more knowledge than I have could accept. It's a lot. And I believe I
> > > can speak for lots of us when I say to have keeped our code private
> > > knowing it will never be accepted mainline as it is, because it does
> > > not refactors ffmpeg or libavfilter and just solves something quickly.
> > >
> > > softworkz is now clearly tired of this patchset, and so are the devs
> > > discussing it with him. When I didn't knew that this thing had been
> > > going on for six months, I was hoping to be able to somehow get in
> > > the middle of the debate and try to unblock it. I was hoping to help
> > > coding here and there, testing this and that, comparing ideas...
> > > I was actually beginning to analize the code, in order to see if I was
> > > able to remove the controversial subtitle_pts property. I realize now
> > > I was sadly too late to the party.
> > >
> > > Yet, here we are, with code that actually do the job, and even
> > > performs pretty well.
> > >
> > > With all this in mind, here are my feelings about this:
> > >
> > >    - I could not care less about subtitle_pts. I know it's questionable,
> > >      but screw it: its also questionable to wait for years with live
> > >      streams problems as some kind of second class citizens. This
> > >      property and all around this patchset (the good and the bad) are
> > >      but accidents of the project's history, and those will keep on
> > >      happening no matter what. We CAN fix subtitle_pts later,
> > >      without having to wait forever for the concensual
> > >      implementation in order to have the use cases implemented.
> > >      That's not a crime: not against "good programing", and neither
> > >      against ffmpeg.
> > >
> > >    - I don't think subtitle_pts is the real problem here. I think there
> > >      are grudges between devs, and things going on I don't fully
> > >      understand. Sadly, as I have no more time to participate in
> > >      this development (given that everybody's tired, and a call for
> > >      a vote seems imminent), I can't also read the 6 months of
> > >      debates to gain a better understanding before expressing
> > >      an opinion. Sorry devs, you deserve better than this
> > >      ignorance of mine. Perhaps somebody could make a list of
> > >      relevant links to discussions before the vote?
> > >
> > >    - I'm more worried about not breaking anything. I saw some
> > >      notes from Mr. Niedermayer, and could reproduce myself some
> > >      problems. However, softworkz seems to be willing to fix that
> > >      kind of issues, so doesn't seem to be a big deal. I see he
> > >      actually posted a new version of his patchset minutes ago,
> > >      so that's great. The point is, I would find this kind of things
> > >      actually blockers, and not subtitle_pts.
> > >
> > >    - If this patchset get rejected, I believe a proper consensual
> > >      design is in order. Consensual between the relevant devs,
> > >      of course. But so far, I don't see it linked in the debates.
> > >      Most likely it is somewhere and I just didn't saw it. But my
> > >      point is that rejecting this kind of works should be done
> > >      with a proper explanation, and the acceptable design
> > >      should be part of that explanation. With that guidelines,
> > >      perhaps others like me could progressively implement the
> > >      thing. But, to be honest, rejecting this without a clear and
> > >      consise designs counterpart (and not only sparse ideas, as
> > >      we all have some of those), would be frankly rude to both
> > >      the people who did the job and the people who wants the
> > >      use cases: because this works, and it's right here.
> > >
> > >    - I would also note that perhaps tiredness would be solved
> > >      by resting, and not either by an ultimatum or a complete
> > >      dismissal of the propossal. I would be very happy to
> > >      collaborate (as I could, which I fear may not be much) if
> > >      you people happen to just find the energy to take a look
> > >      at the thing again without so much... err... "painful
> > >      exchanges" (I speak spanish, sorry, my english is limited).
> > >      Finally having this use cases should be something to
> > >      celebrate, and not something depressing. I know, this is
> > >      some kinda hippie thinking of mine. But I feel it's worth
> > >      the note anyways.
> > >
> > >
> > > Just my 2 cents.
> > > Thanks,
> > > Daniel.
> >
> > Hi Daniel,
> >
> > thanks for your comments. I basically think that others should
> > respond to this, but I'd like to add a few remarks.
> >
> > Once for the facts: the subtitle_pts field in AVFrame exists since
> > V5 of my patchset, which I have submitted on 2021-09-12.
> > This has been 3 months ago. Nobody had objected its existence
> > until only 2 or 3 weeks ago.
> 
> 
> This is really irrelevant, please stop insisting on hacks like subtitle_pts.

New idea:

I could remove the three fields (subtitle_pts, subtitle_start_time, 
subtitle_end_time) from AVFrame and add it to AVSubtitleArea.

How about that? It would allow the frame to be "clean" at least.

sw



_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".