Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Kieran Kunhya via ffmpeg-devel]

On Fri, 17 Jan 2025, 21:32 Romain Beauxis, <romain.beauxis@gmail.com> wrote:

> Le ven. 17 janv. 2025 à 08:32, Kieran Kunhya
> <kieran618@googlemail.com> a écrit :
> >
> > On Fri, Jan 17, 2025 at 2:24 PM Kieran Kunhya <kieran618@googlemail.com>
> wrote:
> > >
> > > On Fri, Jan 17, 2025 at 2:00 PM Romain Beauxis <
> romain.beauxis@gmail.com> wrote:
> > > >
> > > > Hi,
> > > >
> > > > Le jeu. 16 janv. 2025 à 16:40, Kieran Kunhya
> > > > <kieran618@googlemail.com> a écrit :
> > > > >
> > > > > On Thu, Jan 16, 2025 at 9:35 PM Romain Beauxis <
> romain.beauxis@gmail.com> wrote:
> > > > > >
> > > > > > Le jeu. 16 janv. 2025 à 14:48, Kieran Kunhya
> > > > > > <kieran618@googlemail.com> a écrit :
> > > > > > >
> > > > > > > On Thu, 16 Jan 2025, 20:15 Romain Beauxis, <
> romain.beauxis@gmail.com> wrote:
> > > > > > >>
> > > > > > >> This patch implements the decoding logic for the FEC error-
> > > > > > >> correction method described in the Pro-MPEG CoP #3-R2 FEC[1]
> > > > > > >>
> > > > > > >> We are still in the process of testing this patch with our
> encoders but
> > > > > > >> I wanted to send it now in case it could use some early
> review.
> > > > > > >>
> > > > > > >> The code seems clean enough and straight forward:
> > > > > > >
> > > > > > >
> > > > > > > As someone who wrote an FEC receiver in Upipe, I'm surprised
> to see the word "straight forward" and FEC used in the same sentence. The
> Upipe implementation (which was never merged) is extremely messy and
> complex and commercial implementations are even worse as I understand.
> There are tons of edge cases.
> > > > > >
> > > > > > Yes, you're right! What I meant by straight forward is that, in
> my
> > > > > > experience, in the absence of a specific heuristics, the most
> simple
> > > > > > and elegant solution is usually the best, at least to make sure
> the
> > > > > > code is easy to read and to improve on later.
> > > > >
> > > > > This is not academia. FEC is about maximising error recovery and
> there
> > > > > are tons of edge cases to create something that works 24/7.
> > > > >
> > > > > > I should have also said that I'm hoping that this implementation
> is an
> > > > > > initial one that provides the functionality for the most "simple"
> > > > > > situations and that could be improved later.
> > > > > >
> > > > > > > The biggest question I have, is how do you decide to schedule
> packets? In the FEC spec, the incoming streams are CBR so the matrix
> duration is fixed. So if there is packet loss you know when to schedule the
> output packets because the latency is fixed. But in FFmpeg you don't know
> the incoming streams are CBR, nor do you have a concept of time. Min and
> max packets is complete guesswork in a VBR stream and the code takes a
> bursty stream and makes it even more bursty. And you of course destroy the
> CBR nature of a valid FEC stream.
> > > > > > >
> > > > > > > The other thing that's difficult is handling a change in
> stream without causing an infinite loop and keeping FEC behaviour
> Heuristics are needed for this.
> > > > > > >
> > > > > > > It's not clear from your code how you handle the matrix
> recursion. i.e the fact you recovered one packet allows you to rerun the
> recovery and potentially recover more from a different column/row.
> > > > > >
> > > > > > The way the algorithm works is:
> > > > > > * Start by buffering an initial amount of packets
> > > > >
> > > > > But you have packet loss so this number represents a variable
> amount of time.
> > > > >
> > > > > > * Read all available FEC packets on each RTP packet read,
> keeping a
> > > > > > max number on each axis
> > > > > > * On first missing packet, keep reading additional RTP packets
> and:
> > > > > >   - Try to reconstruct using the packet row if available
> > > > > >   - Or else, try to reconstruct using the packet column if
> available
> > > > > >   - Or else, try to reconstruct using the whole FEC matrix if
> both are available
> > > > > >
> > > > > > In this matrix reconstruction phase, we keep looping on the rows
> and
> > > > > > columns of the matrix while at least one packet is
> reconstructed. We
> > > > > > also return immediately if the missing packet has been
> reconstructed.
> > > > > >
> > > > > > If the packet cannot be reconstructed, we keep reading packets
> and
> > > > > > trying this algorithm. When the max number of packets is
> reached, we
> > > > > > consider this packet unrecoverable.
> > > > >
> > > > > What about out of order packets?
> > > > >
> > > > > > If you adjust the max number of packets well, this makes sure
> that we
> > > > > > (optimistically) keep one FEC matrix forward from the packet.
> > > > > >
> > > > > > When a packet is returned, we look at the next packet's
> corresponding
> > > > > > FEC row and column packets. If we have both, we know exactly the
> index
> > > > > > of the first packet in the current packet's FEC matrix and we
> drop all
> > > > > > ones before that (RTP and FEC). Otherwise, we keep a whole
> column x
> > > > > > row matrix before the packet.
> > > > >
> > > > > So this basically bursts packets because you don't account for
> packet
> > > > > loss in your buffered packet sizes, nor CBR vs VBR.
> > > > > As I said, if you stop and start the encoder this implementation
> will
> > > > > almost certainly infinite loop and/or crash.
> > > > >
> > > > > > > I also question the need for FEC in 2025. I am happy to see
> this awful protocol gone and am not sure maintaining something like this is
> possible in FFmpeg apart from use cases where latency doesn't matter (so
> why use FEC?).
> > > > > >
> > > > > > The Radio France people have a real problem with packet loss in
> their
> > > > > > RTP streams.
> > > > > >
> > > > > > The rate of packet loss seems to be low enough so that this
> > > > > > implementation would work just fine.
> > > > >
> > > > > Yes but FFmpeg code needs to work in the real world, not just for
> your
> > > > > one customer.
> > > > >
> > > > > > Their hardware encoders have support for FEC encoding so this
> seems
> > > > > > like a real use and benefit right there.
> > > > >
> > > > > There are so much more modern technologies that are vastly easier
> to
> > > > > implement. We have learnt in the last twenty years.
> > > > >
> > > > > Have you tested this implementation with iptables or tc? You'll see
> > > > > it's very difficult to create a stable implementation that doesn't
> > > > > infinite loop as well as survives an encoder restart.
> > > > > I would suggest putting this on a real domestic internet connection
> > > > > where it's very hard to create a stable implementation.
> > > >
> > > > Hi,
> > > >
> > > > While I appreciate your feedback, I wonder about the need to be
> > > > combative like I perceive here.
> > >
> > > Because in the end writing something for your simple case and
> > > disappearing, FFmpeg will get the blame for things not being good.
> > > Same with FFmpeg supporting complex protocols WebRTC, SDR etc etc.
> > >
> > > >
> > > > At the end of the day, we are looking at a probabilistic problem
> > > > (connectivity issues).
> > > >
> > > > Just like the problem, the solution is probabilistic by nature. In
> > > > some cases, it will be able to recover some packets but. for some, it
> > > > will not.
> > > >
> > > > Even if it does not cover all corner cases, this implementation does
> help.
> > > >
> > > > I have tested it with stimulated network issues and found it to be
> > > > able to restore most packets event at 10 or 20% packet drops.
> > > >
> > > > I have also tested it with a bad local broadband.
> > > >
> > > > We plan on testing it more with a concrete problem and hardware
> encoders.
> > > >
> > > > In my experience, experienced software engineer thrive to propose
> dead
> > > > simple code that is good enough to solve the problems that it
> > > > addresses in most cases.
> > >
> > > Your code is honestly way too simplistic for an FEC implementation.
> > > There is a reason other implementations are much more complex.
> > > You have variable latency (user chooses the buffer size by hand,
> > > seriously...), create burstiness, likely infinite loop with an encoder
> > > restart etc.
> > >
> > > Writing a network protocol is about making it as tolerant to the real
> > > world as possible where all sorts of things happen.
> > >
> > > > Also, interfacing with legacy systems is at the core of what we do
> > > > because it does solve real world problems. This project has plenty of
> > > > support for legacy systems.
> > > >
> > > > I will address the encoder restart, thank you for pointing this out.
> I
> > > > imagine that the SSRC can be used to detect a new stream and
> > > > bump/reset packets indexes.
> > >
> > > SSRC is mandated to be 0 so it can't be used.
> > >
> > > Regards,
> > > Kieran Kunhya
> >
> > I would seriously look at a sophisticated FEC implementation that
> > doesn't require the user to input the matrix size, has a low and
> > constant latency, handles reordered packets and is tolerant to encoder
> > restarts:
> > https://github.com/Upipe/upipe/blob/master/lib/upipe-ts/upipe_rtp_fec.c
>
> I'm just getting to look at that.
>
> Is the ffmpeg RTP input currently supposed to recover from an encoder
> restart?
>
> I did:
> * Start a RTP output:
>
> ffmpeg -i ... rtp://...
>
> * Start a RTP input reading from the input:
>
> ffmpeg -i rtp://... ...
>
> * Stop and start the RTP input process
>
> After that, the running ffmpeg instance keeps dropping packets:
>
> [rtp @ 0x14a804200] RTP: dropping old packet received too late0.215x
>     Last message repeated 7 times
>
> If ffmpeg RTP input is not able to recover from an encoder restart, I
> don't think adding support for that in the FEC decoder will help.
>
> -- Romain
>

There you go, the only way to recover from real world issues is to use
heuristics...

Kieran

>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 3980 bytes --]

Hi Devin

On Fri, Jan 17, 2025 at 12:54:10PM -0500, Devin Heitmueller wrote:
> Hello all,
> 
> I've got some experience in this particular area, so figured it might
> be helpful to offer another voice.
> 
> Kieran makes a number of excellent points, and I largely agree with
> his discussion of the problem space.

> It's an intrinsically difficult
> problem.

sure


> The data arrives on multiple sockets, leading to all sorts
> of opportunities for timing behavior and reordering issues.

how does this matter?
Either i have received a error free packet or i have not
Either i have a set of equations taht allows some kind of optimal
unique solution allowing recovery of a missing packet or i dont.

When its time to output a packet either it can be satisfied because
its determined by the avaialable packets or its not.

Where is the complexity here ?

At this level everything in multimedia is complex or nothing is.
Its more a matter of viewpoint than the exact problem at hand.


> And, in
> my experience, I certainly agree with Kieran's assertion that the
> quality of implementations vary wildly across both open source and
> commercial products (e.g. hardware encoders and transcoders) that
> attempt to implement the standard.

of course. Thats true always everywhere, isnt it?


> Not to mention the
> interoperability issues when receiving streams from all these
> different products.

And FFmpeg has always been the most compatible when it comes to our
demuxers and decoders.


> 
> Building an implementation where you are confident that it does it
> "right" generally means having a large corpus of test vectors that
> exercise edge cases both in delivery timing and packet loss, assessing
> the result to determine which packets should have been recoverable
> versus what was actually recovered.

Of course


> This is considerably beyond the
> sorts of tests ffmpeg typically does in their FATE suite (which AFAIK
> doesn't attempt to validate libavformat protocols at all).

And that is really bad. FATE should cover libavformat protocols
We need some instrumentation/connextion thing that allows us to
inject testdata into them without a real network on the other side
in a clean simple and platform independant way.


[...]
> Marking it as experimental seems like a reasonable ask, given that
> it's brand new code.  That would both allow it to be evaluated/tested
> by those who care about it, and there's a base onto which to submit
> fixes/improvements.  And, like with anything else, if it's determined
> that the implementation works very poorly and/or it gets abandoned, it
> can always be dropped from the tree.

yes

And we seem to agree 95% on all this.

The part i do not agree with and iam not convinced about is that this
cannot be done in a clean and fully working and heuristics free
way in the real world.

You sure can write an implementation that uses hundreads of heuristics
to build a short binary coding tree. for example Shannon–Fano codes
could be a start point. But that doesnt mean there isnt an optimal
and clean solution (which there is and its huffman trees)

Or for a more complex problem you can look at early MPEG encoders
how they accumulated heuristics to decide to use intra or inter MBs
or make other decissions.
This did not preclude the use of Rate Distortion to make these
decissions in later encoders with later than also psycho-vissual
metrics.
These are no longer heuristics, these are optimal solutions under
a given Rate Distortion model. This is also no nonsense
its "modern" encoders in the real world. (modern as in last few decades)

So the claim that it "requires heuristics" and cannot be done cleanly
touches me the wrong way. That claim I disagree with

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

You can kill me, but you cannot change the truth.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Kieran Kunhya via ffmpeg-devel]

On Sun, 19 Jan 2025, 22:22 Michael Niedermayer, <michael@niedermayer.cc>
wrote:

> The part i do not agree with and iam not convinced about is that this
> cannot be done in a clean and fully working and heuristics free
> way in the real world.
>

Protocols are not the same as codecs. Just because you have a hammer,
everything isn't a nail.

Have you ever written a protocol for a lossy network transport before?
Protocols are exceptionally difficult to unit test (and this one is
relatively non complex as it's unidirectional).

Let's pick a simple problem. How do you detect a source encoder restart,
considering this spec forces you to set SSRC=0? (why I don't know, but some
devices do require it). You are forced to use heuristics. I picked a
sequence number gap of 200 packets as this is twice as big as the maximum
allowed matrix. This will false positive roughly 0.3% of the time but if
you've lost 200 packets you probably will be in a resync condition anyway.

You could also use RTP timestamps in the heuristics (but some RTP stream
payloads have constant timestamps per frame or timestamps are set
incorrectly), you could try packet arrival times, but there are stream
splicers out there that splice two RTP streams without packet timing gaps.

Note that this simple issue already exists and has existed for years in the
RTP demux, a protocol that is much more important than legacy FEC. The only
way to solve this is to use heuristics.

Rinse and repeat for another hundred or so edge cases, and make sure they
don't interact. Like I said, it's hard to unit test, so remember, you'll
have to test all of these by hand.

Protocols are not easy. There's a reason Zoom is so good and all the
alternatives (OSS included) suck.

Regards,
Kieran Kunhya

>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Kieran Kunhya via ffmpeg-devel]

>
> > The data arrives on multiple sockets, leading to all sorts
> > of opportunities for timing behavior and reordering issues.
>
> how does this matter?
>

There are routers that put traffic on a different port down a different ISP
so you have to compensate for latency delays between the two links. You
can't just "buffer N packets".

Another edge case.

Kieran

>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Nicolas George]

James Almer (12025-01-17):
> I don't see how the GA has anything to do with this when we're talking about
> one person, but you're not wrong in that rejecting a feature for being
> incomplete is not ideal.
> The codebase is not lacking in partially implemented features and protocols,
> and as long as it's stable, is not invasive, and gracefully rejects
> unsupported scenarios, IMO it should be ok.

How many partially implemented or experimental features can you quote
that have been added in the recent years, when the GA has been in
authority with the current structure?

Last year, there was the software-defined radio proposal. It was
big but non-invasive and easy to remove in case of issue. It was
rejected with an arbitrary “does not belong in ffmpeg”. It did not go to
vote, because it was so obvious for everybody involved that the
opponents had the numbers with a wide margin.

A little older, that I mention because it is close to me, there was the
ground-work necessary for enhancing the utility API: error reporting,
built-in documentation, uniform API for different types of objects,
etc.: rejected “does not belong in ffmpeg”, would have been a waste of
time to go to vote.

This last point is what convinced me to stop investing time on ffmpeg as
long as the GA has authority: why spend effort when a bunch of people
who barely understand the project can send all my efforts to waste
because they consider anything that does not profit them immediately and
might hypothetically make more work for them?

This “it does not belong in ffmpeg” attitude in the GA is killing the
project. It needs to stop. We need somebody in charge who understands
what ffmpeg really is.

-- 
  Nicolas George
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Nicolas George]

Devin Heitmueller (12025-01-17):
>					It's an intrinsically difficult
> problem.  The data arrives on multiple sockets, leading to all sorts
> of opportunities for timing behavior and reordering issues.

I will add something on top of that: The architecture of network
protocols in FFmpeg is not adapted to protocols with multiple sockets at
all. All such protocols we have implement their own half-assed event
loop.

To implement anything complex would require reworking the whole system
to have protocols and demuxers work in an input-driven fashion around a
global event loop. With threads or contexts to let current output-driven
code run, it would be doable. But with the current do-nothing-ambitious
governance, and with the sequels of the botched move to threads of
fftools, it cannot happen.

> That said, I don't necessarily think we should let "perfect be the
> enemy of good" and outright reject a proposed implementation that has
> been reported to work in many use cases.

Thank you for writing the obvious.

>	  It's a good starting point and can be improved over time.

Ditto.

Regards,

-- 
  Nicolas George
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Nicolas George]

Kieran Kunhya via ffmpeg-devel (12025-01-17):
> I am the author of the only known functional OSS implementation of FEC.

Seen from here, your rejection of this proposal looks a teeny tiny
little bit like trying to make sure you stay the only one and have no
competition in providing consulting about it.

-- 
  Nicolas George
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Kieran Kunhya via ffmpeg-devel]

On Mon, Jan 20, 2025 at 2:51 PM Nicolas George <george@nsup.org> wrote:
>
> Kieran Kunhya via ffmpeg-devel (12025-01-17):
> > I am the author of the only known functional OSS implementation of FEC.
>
> Seen from here, your rejection of this proposal looks a teeny tiny
> little bit like trying to make sure you stay the only one and have no
> competition in providing consulting about it.

Quite the opposite, as you see I haven't touched the Upipe FEC code in
over ten years.

I want the author and other people in the project to understand error
recovery protocols are non trivial and require a ton of heuristics and
testing.
If a simple RTP source restart hasn't been fixed, what chance does a
complex error recovery protocol have when the author and other project
members feel it can be done "elegantly" without heuristics?
This like someone wanting to write a fast codec in Python or
Javascript. It would be shot down by this community and it's important
to understand this applies in other fields.

Error recovery protocols are complicated - this project lacks
experience in them and a half-baked one will cause a lot of issues,
security and stability being some of them. Furthermore, the issues are
hard to reproduce as they may happen once a day or even once a week on
certain networks.

I have no issue with it being marked experimental. But the idea of
incremental improvement is false. Error recovery implementations
require careful design from the outset with numerous heuristics
needed.

Regards,
Kieran Kunhya
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[James Almer]

[-- Attachment #1.1.1: Type: text/plain, Size: 3224 bytes --]

On 1/20/2025 11:30 AM, Nicolas George wrote:
> James Almer (12025-01-17):
>> I don't see how the GA has anything to do with this when we're talking about
>> one person, but you're not wrong in that rejecting a feature for being
>> incomplete is not ideal.
>> The codebase is not lacking in partially implemented features and protocols,
>> and as long as it's stable, is not invasive, and gracefully rejects
>> unsupported scenarios, IMO it should be ok.
> 
> How many partially implemented or experimental features can you quote
> that have been added in the recent years, when the GA has been in
> authority with the current structure?

Partial support for MV-HEVC, partial support for IAMF (no rendering, 
only demuxing/muxing), partial support for tiled HEIF (no stitching), 
the new VVC decoder that's afaik not 100% complete, a PTS reorder bsf 
that's missing some edge cases and support for HEVC and VVC, plus other 
stuff i don't recall right now.

Things are partially implemented all the time.

> 
> Last year, there was the software-defined radio proposal. It was
> big but non-invasive and easy to remove in case of issue. It was
> rejected with an arbitrary “does not belong in ffmpeg”. It did not go to
> vote, because it was so obvious for everybody involved that the
> opponents had the numbers with a wide margin.
> 
> A little older, that I mention because it is close to me, there was the
> ground-work necessary for enhancing the utility API: error reporting,
> built-in documentation, uniform API for different types of objects,
> etc.: rejected “does not belong in ffmpeg”, would have been a waste of
> time to go to vote.
> 
> This last point is what convinced me to stop investing time on ffmpeg as
> long as the GA has authority: why spend effort when a bunch of people
> who barely understand the project can send all my efforts to waste
> because they consider anything that does not profit them immediately and
> might hypothetically make more work for them?
> 
> This “it does not belong in ffmpeg” attitude in the GA is killing the
> project. It needs to stop. We need somebody in charge who understands
> what ffmpeg really is.
I don't believe the kitchen sink approach is better, either. Some things 
really don't belong in our libraries.

And your strings API faced opposition (and not rejection, because no 
vote happened for that) not because it didn't belong in ffmpeg (There's 
one already, also by you), but because others either didn't consider it 
worth the complexity, or didn't like its design.
The way to move that forward is to address the concerns, changing the 
design if needed, or convincing the other parties that it's good as is, 
and if no consensus is reached, the TC can then be invoked to make a 
decision. And ultimately, the GA can be also invoked if someone wants 
that decision overruled, but i consider this excessive because the TC 
was precisely elected by the GA to handle this stuff.

I'd really like if we can stop with the "Everything's fucked, nothing 
can be done" mails every other week and instead make use of the 
framework we came up with, or if needed, change it and improve it.


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Marth64]

James Almer:

> I'd really like if we can stop with the "Everything's fucked, nothing
> can be done" mails every other week and instead make use of the
> framework we came up with, or if needed, change it and improve it.

+1
or we will never move forward with anything
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Romain Beauxis]

Hi all,

Le lun. 20 janv. 2025 à 10:23, Marth64 <marth64@proxyid.net> a écrit :
>
> James Almer:
>
> > I'd really like if we can stop with the "Everything's fucked, nothing
> > can be done" mails every other week and instead make use of the
> > framework we came up with, or if needed, change it and improve it.
>
> +1
> or we will never move forward with anything

Thank you for the robust discussion y'all.

First and foremost, I'm sorry to see that no one seems to have
actually tested my changes.

Here are some testing feedback:
* I had to fix a couple of glitches from the v0, I'll post an updated
patch after sending this email.
* It has only been tested end-to-end with FFmpeg implementation. We
are scheduled to test with the client's actual encoder some time this
month.
* I have done the following:

1. From a AWS remote server, launch this process:
ffmpeg -re -i  <New Orleans WWOZ URL> -f rtp_mpegts -fec
prompeg=l=4:d=4 rtp://<my ISP IP>:3498

2. Set up port forwarding to my laptop inside my local LAN wifi.

(If you know US IPs and if you know New Orleans, I have to add that my
internet link is particularly bad too)

3. Launch on my laptop:

ffmpeg -i "rtp://127.0.0.1:8000?fec=prompeg=l=4:d=4" -c copy -y /tmp/decoded.aac

I then used macos's Network Link Conditioner with different packet
loss ratio to simulate packet losses.

After about 30 min of each case, I reported the following recovery ratio:

1% packet loss: 100% recovery ratio
5% packet loss: 98% recovery ratio
10% packet loss: 95% recovery ratio
20% packet loss: 82% recovery ratio

Frankly, these numbers seem good enough to me to merge as is, this is
already providing a pretty robust solution for users who are concerned
only about using ffmpeg end-to-end.

As for the rest of the conversation, here are some quick notes:

--> Do I intend to maintain and expand the implementation?

I would love to. In fact, I would love to be involved more with the
development process here.

It's just been really difficult to get some attention on the changes
that I have proposed.

Typically, I have another patch that I sent earlier about ogg that has
had zero review so far.

But, yeah, I have a track record of maintaining several projects over
the years and I'd like to consider this one as well.

--> Feedback and expert knowledge

In the discussion, I see a lot of knowledge being expressed that comes
from years of working on these technologies.

Though I have some under my belt as well, I'm eager to learn more, if
this can be done in a constructive way.

Here are some feedback based on what I've been able to read in the threads:

--> Heuristics vs. algorithmic

In my opinion, heuristics should be avoided as much as possible and,
if used, should be abstracted away in their own modular architectural
design.

The reason being that each particular heuristic introduces side
effects that are often hard to anticipate, complicate code proofread
and testing.

When a specific situations arises, like a specific router behavior,
instead of immediately adding a quick conditional branch, it can be
more beneficial to ask ourselves what are the consequences of this
specific behavior to the system in general and wether or not this can
be caught through some of the more generic mechanisms already in place
(or via a new one).

However, heuristics can also be hard learned lessons from battling in
the field (see expert knowledge above) so I'm not against them, just
generally very conservative.

--> Practical engineering

I have grown to like engineering based on tests and experiments like
the ones I did above.

At the end of the day, I have no specific attachment to my
implementation in particular as long as there is one that is
maintainable and that can provide the functionality that is desired.

(though I will admit it's always nice to get your work recognized and
validated).

To address some specific concerns about the proposed implementation:

--> Infinite loop

Being a "simple" implementation with no heuristics, the expected steps
upon receiving a new packet are always finite. Thus, there cannot be
any infinite loop in this implementation.

--> Encoder restart

This seems to be out wildly of the scope for those changes. The RTP
demuxer does not yet support encoder restart.

Also, to support this, you would also have to deal with PTS/DTS
discontinuities that would be akin to supporting dynamic content
switch, another thing that is not obvious to do in ffmpeg at the
moment.

In the future, I would love to contribute more on this.

Based on the conversation'd feedback regarding implementation specific
details, this seems like a hard problem, in particular because SSRC is
not usable for that.

In this specific case, it seems to me that, instead of building
heuristics that are a little bit of a shot in the dark, this
particular problem could warrant breaking the separation of concern
and using the content layer.

In a lot of cases, that the content layer would make it extremely
clear when a new stream is coming into the demuxer by looking at some
specifics of its bitstream.

And, yes, if this is impossible, for instance with raw PCM, then use
heuristics as last resort, if this makes sense.

--> Next steps?

I still have to test the implementation with my client's hardware
encoders. However, I believe that the v1 of the changes that I'm about
to send should be ready for a solid review/consideration based on my
own testing.

Meanwhile, I will be at FOSDEM the first weekend of October and would
love to continue the conversation in person. In fact, I am planning to
give a talk about dynamic content switching using ffmpeg so some of
the talk could be an interesting starting point for a larger
conversation.

Thanks,
-- Romain
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 991 bytes --]

On Mon, Jan 20, 2025 at 06:46:46AM +0000, Kieran Kunhya via ffmpeg-devel wrote:
> >
> > > The data arrives on multiple sockets, leading to all sorts
> > > of opportunities for timing behavior and reordering issues.
> >
> > how does this matter?
> >
> 
> There are routers that put traffic on a different port down a different ISP
> so you have to compensate for latency delays between the two links. You
> can't just "buffer N packets".
> 
> Another edge case.

I dont know why you assume "buffer N packets".

Packets enter this buffer when they are received
Packets should exit that (buffer + FEC) when they are needed for decompression
(for presentation to the user) or slightly prior

Removing them from the buffer earlier has a higher propability of failure
so it is strictly worse.

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The worst form of inequality is to try to make unequal things equal.
-- Aristotle

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Kieran Kunhya via ffmpeg-devel]

On Wed, Jan 22, 2025 at 3:33 PM Michael Niedermayer
<michael@niedermayer.cc> wrote:
>
> On Mon, Jan 20, 2025 at 06:46:46AM +0000, Kieran Kunhya via ffmpeg-devel wrote:
> > >
> > > > The data arrives on multiple sockets, leading to all sorts
> > > > of opportunities for timing behavior and reordering issues.
> > >
> > > how does this matter?
> > >
> >
> > There are routers that put traffic on a different port down a different ISP
> > so you have to compensate for latency delays between the two links. You
> > can't just "buffer N packets".
> >
> > Another edge case.
>
> I dont know why you assume "buffer N packets".
>
> Packets enter this buffer when they are received
> Packets should exit that (buffer + FEC) when they are needed for decompression
> (for presentation to the user) or slightly prior
>
> Removing them from the buffer earlier has a higher propability of failure
> so it is strictly worse.

Wrong, the packets leave the buffer at a fixed duration corresponding
to (an estimate of) 2 times the matrix duration (the spec was written
for CBR so this is a constant) so that latency is fixed. There is
packet loss (by definition) so a buffer of N packets may be much
larger than 2 times the matrix duration.

"when they are needed for decompression" -> this is real-time RTP, not a file.
The issue you have with FEC going down a different path is the FEC
packets can arrive a lot earlier than the latency window or a lot
later and you have to compensate for this with heuristics.

All packets can be reordered or even duplicated which as far as I know
this code doesn't cover either.

Kieran
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 2526 bytes --]

Hi

On Mon, Jan 20, 2025 at 03:00:17PM +0000, Kieran Kunhya via ffmpeg-devel wrote:
[...]

> Error recovery protocols are complicated -

everything is complicated


> this project lacks
> experience in them

The project is made of many developers, and several of them do have
experience in protocols and also error correction

Just about myself,
i have written a teletext decoder that used raw 8bit data over 20 years ago
and it was more robust than TV sets

I recently as part of the SDR work wrote a RDS decoder, which also worked
better than the decoders i compared it to. Its less than 300 lines of code
and certainly can be improved alot but there is nothing magic along the
lines you suggest here. RDS needs both demodulation and error correction

Also i have written a reed solomon decoder based on my own algorithm
that had better asymptotic speed than what i could find at the time.


> 
> I have no issue with it being marked experimental.


> But the idea of
> incremental improvement is false.

You have experience with mpeg FEC and other protocols but
you mix your practical experience with ridiculous claims


> Error recovery implementations
> require careful design from the outset with numerous heuristics
> needed.

Can you please stop this "my way or no way"

You can design things first then implement them
you can evolutionary write code
you can do it incrementally
you can write a implementation learn from it discard it and write a 2nd one
based on lessons lerned
and there are probably hundreads of other paradigms along which you can
implement something

For some people one way works better than others. For someone else
another might work better

Someone might enjoy writing, tuning and testing heuristics. Another might
have a strong background in probability and statistics and they might design
a model about all the restarts and damages and implement an optimal solution
for teh model with no heuristics. At that level they can then even tune that
model for the particular source of packets and everything adapts to be optimal
for that then as there are no or few hardcoded heuristics.
Noone ever did that before maybe, ok, but doesnt mean it cannot be done

I dont like how you talk down on others, as if what you did is the only
way things can be done.

thx


[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Those who are best at talking, realize last or never when they are wrong.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Romain Beauxis]

Le mer. 22 janv. 2025 à 10:29, Kieran Kunhya via ffmpeg-devel
<ffmpeg-devel@ffmpeg.org> a écrit :
>
> On Wed, Jan 22, 2025 at 3:33 PM Michael Niedermayer
> <michael@niedermayer.cc> wrote:
> >
> > On Mon, Jan 20, 2025 at 06:46:46AM +0000, Kieran Kunhya via ffmpeg-devel wrote:
> > > >
> > > > > The data arrives on multiple sockets, leading to all sorts
> > > > > of opportunities for timing behavior and reordering issues.
> > > >
> > > > how does this matter?
> > > >
> > >
> > > There are routers that put traffic on a different port down a different ISP
> > > so you have to compensate for latency delays between the two links. You
> > > can't just "buffer N packets".
> > >
> > > Another edge case.
> >
> > I dont know why you assume "buffer N packets".
> >
> > Packets enter this buffer when they are received
> > Packets should exit that (buffer + FEC) when they are needed for decompression
> > (for presentation to the user) or slightly prior
> >
> > Removing them from the buffer earlier has a higher propability of failure
> > so it is strictly worse.
>
> Wrong, the packets leave the buffer at a fixed duration corresponding
> to (an estimate of) 2 times the matrix duration (the spec was written
> for CBR so this is a constant) so that latency is fixed. There is
> packet loss (by definition) so a buffer of N packets may be much
> larger than 2 times the matrix duration.
>
> "when they are needed for decompression" -> this is real-time RTP, not a file.
> The issue you have with FEC going down a different path is the FEC
> packets can arrive a lot earlier than the latency window or a lot
> later and you have to compensate for this with heuristics.
>
> All packets can be reordered or even duplicated which as far as I know
> this code doesn't cover either.

Packet reordering is covered.

Packets are stored in a binary tree indexed by their sequence ID and
pulled out based on that ID as well. That takes care of reordering.

Duplicated packets are not properly covered yet but the change is
trivial, this will be in the next version of the patch, thanks for
pointing this out.

I have posted detailed feedback with testing in real conditions, do
you have any feedback on that?

Thanks,
-- Romain
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Kieran Kunhya via ffmpeg-devel]

On Wed, Jan 22, 2025 at 4:37 PM Michael Niedermayer
<michael@niedermayer.cc> wrote:
>
> Hi
>
> On Mon, Jan 20, 2025 at 03:00:17PM +0000, Kieran Kunhya via ffmpeg-devel wrote:
> [...]
>
> > Error recovery protocols are complicated -
>
> everything is complicated
>
>
> > this project lacks
> > experience in them
>
> The project is made of many developers, and several of them do have
> experience in protocols and also error correction
>
> Just about myself,
> i have written a teletext decoder that used raw 8bit data over 20 years ago
> and it was more robust than TV sets
>
> I recently as part of the SDR work wrote a RDS decoder, which also worked
> better than the decoders i compared it to. Its less than 300 lines of code
> and certainly can be improved alot but there is nothing magic along the
> lines you suggest here. RDS needs both demodulation and error correction
>
> Also i have written a reed solomon decoder based on my own algorithm
> that had better asymptotic speed than what i could find at the time.

None of these are packet based networks. Just because I can drive a
car, doesn't mean I can fly a plane.

Kieran
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Kieran Kunhya via ffmpeg-devel]

On Wed, Jan 22, 2025 at 4:53 PM Romain Beauxis <romain.beauxis@gmail.com> wrote:
>
> Le mer. 22 janv. 2025 à 10:29, Kieran Kunhya via ffmpeg-devel
> <ffmpeg-devel@ffmpeg.org> a écrit :
> >
> > On Wed, Jan 22, 2025 at 3:33 PM Michael Niedermayer
> > <michael@niedermayer.cc> wrote:
> > >
> > > On Mon, Jan 20, 2025 at 06:46:46AM +0000, Kieran Kunhya via ffmpeg-devel wrote:
> > > > >
> > > > > > The data arrives on multiple sockets, leading to all sorts
> > > > > > of opportunities for timing behavior and reordering issues.
> > > > >
> > > > > how does this matter?
> > > > >
> > > >
> > > > There are routers that put traffic on a different port down a different ISP
> > > > so you have to compensate for latency delays between the two links. You
> > > > can't just "buffer N packets".
> > > >
> > > > Another edge case.
> > >
> > > I dont know why you assume "buffer N packets".
> > >
> > > Packets enter this buffer when they are received
> > > Packets should exit that (buffer + FEC) when they are needed for decompression
> > > (for presentation to the user) or slightly prior
> > >
> > > Removing them from the buffer earlier has a higher propability of failure
> > > so it is strictly worse.
> >
> > Wrong, the packets leave the buffer at a fixed duration corresponding
> > to (an estimate of) 2 times the matrix duration (the spec was written
> > for CBR so this is a constant) so that latency is fixed. There is
> > packet loss (by definition) so a buffer of N packets may be much
> > larger than 2 times the matrix duration.
> >
> > "when they are needed for decompression" -> this is real-time RTP, not a file.
> > The issue you have with FEC going down a different path is the FEC
> > packets can arrive a lot earlier than the latency window or a lot
> > later and you have to compensate for this with heuristics.
> >
> > All packets can be reordered or even duplicated which as far as I know
> > this code doesn't cover either.
>
> Packet reordering is covered.
>
> Packets are stored in a binary tree indexed by their sequence ID and
> pulled out based on that ID as well. That takes care of reordering.
>
> Duplicated packets are not properly covered yet but the change is
> trivial, this will be in the next version of the patch, thanks for
> pointing this out.
>
> I have posted detailed feedback with testing in real conditions, do
> you have any feedback on that?

Do you plan to support a fixed duration? Buffering N packets has a
variable duration when there is packet loss.
Do you plan to support calculating the matrix size from the RTP FEC packets?
Do you support block and non-block aligned permutations?

Also saying you don't support an encoder restart is basically saying
you want to write something that only deals with steady state
behaviour.

Kieran
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Kieran Kunhya via ffmpeg-devel]

On Wed, Jan 22, 2025 at 4:37 PM Michael Niedermayer
<michael@niedermayer.cc> wrote:
>
> Hi
>
> On Mon, Jan 20, 2025 at 03:00:17PM +0000, Kieran Kunhya via ffmpeg-devel wrote:
> [...]
>
> > Error recovery protocols are complicated -
>
> everything is complicated
>
>
> > this project lacks
> > experience in them
>
> The project is made of many developers, and several of them do have
> experience in protocols and also error correction

As far as I can tell there are no error recovery protocols natively in
FFmpeg. One reason for this as Nicolas points out is the lack of an
event loop model.

Kieran
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Kieran Kunhya via ffmpeg-devel]

On Thu, Jan 16, 2025 at 8:15 PM Romain Beauxis <romain.beauxis@gmail.com> wrote:
>
> This patch implements the decoding logic for the FEC error-
> correction method described in the Pro-MPEG CoP #3-R2 FEC[1]
>
> We are still in the process of testing this patch with our encoders but
> I wanted to send it now in case it could use some early review.
>
> The code seems clean enough and straight forward:
> * The decoder stores packet using their index.
> * Packets relevant to the decoding of the current packet are kept,
>   using the received row and column FEC packets to determine the FEC
>   matrix within with the current packet belongs to.
> * Beside that, the decoder keeps a maximum number of packets and FEC
>   packets.

I don't see any evidence of wraparound handling unless I miss
something. An FEC matrix can span a wraparound.

Kieran
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Kieran Kunhya via ffmpeg-devel]

> Can you please stop this "my way or no way"

The irony is not lost on me of this sentence (from the person banning
people, censoring people, creating paranoid theories about the GA).

Kieran
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Nicolas George]

James Almer (12025-01-20):
> Partial support for MV-HEVC, partial support for IAMF (no rendering, only
> demuxing/muxing), partial support for tiled HEIF (no stitching), the new VVC
> decoder that's afaik not 100% complete, a PTS reorder bsf that's missing
> some edge cases and support for HEVC and VVC, plus other stuff i don't
> recall right now.
> 
> Things are partially implemented all the time.

So… only support for things already invented elsewhere. Nothing actually
original. Nothing that makes FFmpeg a leader rather than a follower.

> I don't believe the kitchen sink approach is better, either. Some things
> really don't belong in our libraries.

Some things, sure. That one? It produces audio, it fits in the code, it
brings features found nowhere else. Of course it belongs. If it was
proposed at the time the people who opposed were busy pretending FFmpeg
did not exist, it would have been added without anybody saying anything
except bravo.

> And your strings API faced opposition (and not rejection, because no vote
> happened for that) not because it didn't belong in ffmpeg (There's one
> already, also by you), but because others either didn't consider it worth
> the complexity, or didn't like its design.
> The way to move that forward is to address the concerns, changing the design
> if needed, or convincing the other parties that it's good as is, and if no

Please, you are trying to explain to me discussions I was directly part
of and that you only followed from afar, it is borderline insulting.

The reality of that discussion was that they were not open to
compromise, at all. They would not clarify their requirements, they
would not consider my arguments that their concerns was missing
important considerations.

And they were not concerned by a ruling by the TC, because their friends
were in it, if not themselves, and the GA put them there.

> I'd really like if we can stop with the "Everything's fucked, nothing can be
> done" mails every other week and instead make use of the framework we came
> up with, or if needed, change it and improve it.

I have not said that “everything” is fucked. Only one thing is fucked:
democracy in a project as small but appetizing as FFmpeg. And the
improvement is obvious: reaffirm Michael as a leader.

-- 
  Nicolas George
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Romain Beauxis]

Le mer. 22 janv. 2025 à 11:03, Kieran Kunhya
<kieran618@googlemail.com> a écrit :
>
> On Thu, Jan 16, 2025 at 8:15 PM Romain Beauxis <romain.beauxis@gmail.com> wrote:
> >
> > This patch implements the decoding logic for the FEC error-
> > correction method described in the Pro-MPEG CoP #3-R2 FEC[1]
> >
> > We are still in the process of testing this patch with our encoders but
> > I wanted to send it now in case it could use some early review.
> >
> > The code seems clean enough and straight forward:
> > * The decoder stores packet using their index.
> > * Packets relevant to the decoding of the current packet are kept,
> >   using the received row and column FEC packets to determine the FEC
> >   matrix within with the current packet belongs to.
> > * Beside that, the decoder keeps a maximum number of packets and FEC
> >   packets.
>
> I don't see any evidence of wraparound handling unless I miss
> something. An FEC matrix can span a wraparound.

Got it in my local branch, working on testing/cleanup.

> Do you plan to support a fixed duration? Buffering N packets has a
> variable duration when there is packet loss.

As far as I can see there is a separation of concern here.

The FEC decoder works on the demuxer/transport layer and is
independent from the content layer.

The FEC decoder parameters can be set by the user according to their
content settings to determine the delay incurred by buffering and
packet loss for a CBR content.

The scope of this patch is limited to the FEC spec which is explicitly
defined for CBR content.

> Do you plan to support calculating the matrix size from the RTP FEC packets?
> Do you support block and non-block aligned permutations?

Those two could be done together and shouldn't be too hard to add to
the current implementation.

I'll look into it and would like to have it in the next version of the patch.

> Also saying you don't support an encoder restart is basically saying
> you want to write something that only deals with steady state
> behaviour.

As I replied before, this is just not something that ffmpeg can
support as a whole at the moment.

Do you have any other concerns?

The other change that I forgot to apply is the copyright headers
cleanup. I'll have it in the next iteration as well.

This process is not clear to me, am I supposed to work with you on
getting this patch accepted?

I do appreciate the feedback when they apply but I'd like to make sure
that I'm working toward a mutually agreed goal here.

Thanks,
-- Romain
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Kieran Kunhya via ffmpeg-devel]

> The FEC decoder works on the demuxer/transport layer and is
> independent from the content layer.
>
> The FEC decoder parameters can be set by the user according to their
> content settings to determine the delay incurred by buffering and
> packet loss for a CBR content.

A buffer of N packets does not map to a fixed duration when there is
packet loss (even if that packet loss is corrected) irrespective of
CBR. This means you have bursty output throughout the process. In a
realtime application you will then have pops and clicks.

For example in the beginning you buffer 200 packets and that
corresponds to a duration of 200ms (for simplicity). Then you have
packet loss that's corrected, but then that 200 packets corresponds to
a longer period of time, say 250ms. What will you do about this gap of
50ms in a realtime application?

>This process is not clear to me, am I supposed to work with you on
> getting this patch accepted?

No, I'm just explaining the real world of protocols is a lot more
complex than your claims of "elegance". But it's clear you want to
move the goalposts (e.g ignoring encoder restarts) to maintain that
belief. You say heuristics are not needed and then just claim an issue
that obviously needs heuristics is "out of scope".

Kieran
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Romain Beauxis]

Le jeu. 23 janv. 2025 à 12:31, Kieran Kunhya
<kieran618@googlemail.com> a écrit :
>
> > The FEC decoder works on the demuxer/transport layer and is
> > independent from the content layer.
> >
> > The FEC decoder parameters can be set by the user according to their
> > content settings to determine the delay incurred by buffering and
> > packet loss for a CBR content.
>
> A buffer of N packets does not map to a fixed duration when there is
> packet loss (even if that packet loss is corrected) irrespective of
> CBR. This means you have bursty output throughout the process. In a
> realtime application you will then have pops and clicks.
>
> For example in the beginning you buffer 200 packets and that
> corresponds to a duration of 200ms (for simplicity). Then you have
> packet loss that's corrected, but then that 200 packets corresponds to
> a longer period of time, say 250ms. What will you do about this gap of
> 50ms in a realtime application?
>
> >This process is not clear to me, am I supposed to work with you on
> > getting this patch accepted?
>
> No, I'm just explaining the real world of protocols is a lot more
> complex than your claims of "elegance". But it's clear you want to
> move the goalposts (e.g ignoring encoder restarts) to maintain that
> belief. You say heuristics are not needed and then just claim an issue
> that obviously needs heuristics is "out of scope".

I understand these two points clearly.

I responded in context of the ffmpeg project with specific questions:

How do you think these two points should/would be articulated in the
context of the ffmpeg codebase?

Thanks,
-- Romain
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".