Re: [FFmpeg-devel] [PATCH] Accept a colon in the path of a URI, instead of stripping preceding characters.

Re: [FFmpeg-devel] [PATCH] Accept a colon in the path of a URI, instead of stripping preceding characters.

[Timothy Allen via ffmpeg-devel]

Good day

I wanted to offer a discussion of the referenced patch.

I have found that, when a link in an extended M3U file (as used by HLS)
includes a colon, FFmpeg will fail to load the file.

The bug has already been reported: https://trac.ffmpeg.org/ticket/10679

The error reads:
[hls @ 0x78dea4000c80] Failed to open segment 0 of playlist 0 [hls @
0x78dea4000c80] Segment 0 of playlist 0 failed too many times, skipping

The referenced patch fixes the issue. However, it is worth noting that
the patch changes the behavior of one of the unit tests:

- http://a/b/c/d;p?q g:h                  => g:h
+ http://a:b/c/d;p?q e                    => http://a:b/c/e

The original unit test derives from the following two trac tickets:
https://trac.ffmpeg.org/ticket/8813
https://trac.ffmpeg.org/ticket/8814

This is a breaking change, and, in particular, violates one specific
example given in the rfc at
https://tools.ietf.org/html/rfc3986#section-5.4 (the first example). In
particular, it will affect cases where a URL consists only of
host:port, with no scheme or path, and the base URL links to an
unrelated host. Where before, the new link would take the form of
"host:port", the new link will now use "host:port" as the last element
of the path.

I believe that this behaviour is more intuitive given modern use of
URIs (and is replicated in many browsers), but I recognise this is a
matter of taste.

Thank you,

Tim
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] Accept a colon in the path of a URI, instead of stripping preceding characters.

[Marton Balint]

On Tue, 20 May 2025, Timothy Allen via ffmpeg-devel wrote:

> Good day
>
> I wanted to offer a discussion of the referenced patch.
>
> I have found that, when a link in an extended M3U file (as used by HLS)
> includes a colon, FFmpeg will fail to load the file.
>
> The bug has already been reported: https://trac.ffmpeg.org/ticket/10679
>
> The error reads:
> [hls @ 0x78dea4000c80] Failed to open segment 0 of playlist 0 [hls @
> 0x78dea4000c80] Segment 0 of playlist 0 failed too many times, skipping
>
> The referenced patch fixes the issue. However, it is worth noting that
> the patch changes the behavior of one of the unit tests:
>
> - http://a/b/c/d;p?q g:h                  => g:h
> + http://a:b/c/d;p?q e                    => http://a:b/c/e
>
> The original unit test derives from the following two trac tickets:
> https://trac.ffmpeg.org/ticket/8813
> https://trac.ffmpeg.org/ticket/8814
>
> This is a breaking change, and, in particular, violates one specific
> example given in the rfc at
> https://tools.ietf.org/html/rfc3986#section-5.4 (the first example). In
> particular, it will affect cases where a URL consists only of
> host:port, with no scheme or path, and the base URL links to an
> unrelated host. Where before, the new link would take the form of
> "host:port", the new link will now use "host:port" as the last element
> of the path.
>
> I believe that this behaviour is more intuitive given modern use of
> URIs (and is replicated in many browsers), but I recognise this is a
> matter of taste.

Sorry, but I'd rather not break RFC compliant parsing because of 
problematic files. Feel free however to add some more context to the 
ticket, like which browsers and/or media players support such a file, 
which don't. And how the ones which do support it distinguish between an 
URL scheme and a relative URL with a colon in the first path component, 
e.g.

video:first.ts or http:first.ts

Thanks,
Marton
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] Accept a colon in the path of a URI, instead of stripping preceding characters.

[softworkz .]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Timothy
> Allen via ffmpeg-devel
> Sent: Dienstag, 20. Mai 2025 17:28
> To: ffmpeg-devel@ffmpeg.org
> Cc: Timothy Allen <tim@treehouse.org.za>
> Subject: Re: [FFmpeg-devel] [PATCH] Accept a colon in the path of a URI,
> instead of stripping preceding characters.
> 
> Good day
> 
> I wanted to offer a discussion of the referenced patch.
> 
> I have found that, when a link in an extended M3U file (as used by HLS)
> includes a colon, FFmpeg will fail to load the file.
> 
> The bug has already been reported: https://trac.ffmpeg.org/ticket/10679
> 
> The error reads:
> [hls @ 0x78dea4000c80] Failed to open segment 0 of playlist 0 [hls @
> 0x78dea4000c80] Segment 0 of playlist 0 failed too many times, skipping
> 
> The referenced patch fixes the issue. However, it is worth noting that
> the patch changes the behavior of one of the unit tests:
> 
> - http://a/b/c/d;p?q g:h                  => g:h
> + http://a:b/c/d;p?q e                    => http://a:b/c/e
> 
> The original unit test derives from the following two trac tickets:
> https://trac.ffmpeg.org/ticket/8813
> https://trac.ffmpeg.org/ticket/8814
> 
> This is a breaking change, and, in particular, violates one specific
> example given in the rfc at
> https://tools.ietf.org/html/rfc3986#section-5.4 (the first example). In
> particular, it will affect cases where a URL consists only of
> host:port, with no scheme or path, and the base URL links to an
> unrelated host. Where before, the new link would take the form of
> "host:port", the new link will now use "host:port" as the last element
> of the path.
> 
> I believe that this behaviour is more intuitive given modern use of
> URIs (and is replicated in many browsers), but I recognise this is a
> matter of taste.
> 
> Thank you,
> 
> Tim
> _______________________________________________

Hi,

I was just about to reply and suggest to replace those colons with %3A
(url-encoded) when I read the ticket, which already suggests that.

Have you tried it? It sounds like a much better way to me.

Thanks,
sw




_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] Accept a colon in the path of a URI, instead of stripping preceding characters.

[Timothy Allen via ffmpeg-devel]

On Tue, 2025-05-20 at 20:03 +0000, softworkz . wrote:
> I was just about to reply and suggest to replace those colons with
> %3A
> (url-encoded) when I read the ticket, which already suggests that.
> 
> Have you tried it? It sounds like a much better way to me.

I think this would be a common-sense solution as long as one controls
the server/content.

The reason I submitted the patch anyway was because not everyone will
control the content they're consuming, and because, although I
acknowledge it technically breaks RFCs, there is an argument that the
RFC's behaviour is surprising; we can certainly see that other
applications (Safari, in the linked ticket) break the RFC as well.

I can certainly understand if the patch is rejected -- even if it
didn't break the RFC, Postel's Law is not in vogue any longer. However,
I think the workflows that it might break ("host:port", with no scheme
or path) are much less obvious and intuitive than the workflows that it
rescues.

Regardless of the decision, thanks for reviewing the patch!

Best,

Tim
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] Accept a colon in the path of a URI, instead of stripping preceding characters.

[softworkz .]

From: Timothy Allen <tim@treehouse.org.za>
Sent: Mittwoch, 21. Mai 2025 09:29
To: softworkz . <softworkz@hotmail.com>; FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Subject: Re: [FFmpeg-devel] [PATCH] Accept a colon in the path of a URI, instead of stripping preceding characters.

On Tue, 2025-05-20 at 20:03 +0000, softworkz . wrote:
I was just about to reply and suggest to replace those colons with %3A
(url-encoded) when I read the ticket, which already suggests that.

Have you tried it? It sounds like a much better way to me.

I think this would be a common-sense solution as long as one controls the server/content.

The reason I submitted the patch anyway was because not everyone will control the content they're consuming, and because, although I acknowledge it technically breaks RFCs, there is an argument that the RFC's behaviour is surprising; we can certainly see that other applications (Safari, in the linked ticket) break the RFC as well.


Why do you think it would require control over the server side?

Almost every http server in the world will decode the URL as one of its first steps. When
you have a file on disk with a space in it like

“/httproot/my  file.html”

then a URL like “http://server/my%20file.html” will retrieve that file without any change to the http server.

Best,
sw
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] Accept a colon in the path of a URI, instead of stripping preceding characters.

[Timothy Allen via ffmpeg-devel]

On Wed, 2025-05-21 at 18:56 +0000, softworkz . wrote:
> Why do you think it would require control over the server side?

The original ticket is referring to HLS, and specifically the manifest
of HLS, which means a remotely-hosted M3U playlist.

In principle, the user could download the playlist, convert any
relative URLs to absolute URLs, and percent-encode the URLs.

In practice, for most users, the URL will simply not play (or will skip
any segments containing colons in the URL) without any indication of
why the URL is failing.

Thanks,

Tim
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] Accept a colon in the path of a URI, instead of stripping preceding characters.

[softworkz .]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Timothy
> Allen via ffmpeg-devel
> Sent: Mittwoch, 21. Mai 2025 21:09
> To: ffmpeg-devel@ffmpeg.org
> Cc: Timothy Allen <tim@treehouse.org.za>
> Subject: Re: [FFmpeg-devel] [PATCH] Accept a colon in the path of a URI,
> instead of stripping preceding characters.
> 
> On Wed, 2025-05-21 at 18:56 +0000, softworkz . wrote:
> > Why do you think it would require control over the server side?
> 
> The original ticket is referring to HLS, and specifically the manifest
> of HLS, which means a remotely-hosted M3U playlist.
> 
> In principle, the user could download the playlist, convert any
> relative URLs to absolute URLs, and percent-encode the URLs.
> 
> In practice, for most users, the URL will simply not play (or will skip
> any segments containing colons in the URL) without any indication of
> why the URL is failing.

What I mean and what the comment in the ticket is probably suggesting, 
is that the HLS demuxer should URL-encode the URL after combining the 
base url with the segment file name before making a request for the 
segment.

PS: Thanks for switching to plain-text.

Best,
sw


_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] Accept a colon in the path of a URI, instead of stripping preceding characters.

[Timothy Allen via ffmpeg-devel]

On Wed, 2025-05-21 at 19:25 +0000, softworkz . wrote:
> What I mean and what the comment in the ticket is probably
> suggesting, 
> is that the HLS demuxer should URL-encode the URL after combining the
> base url with the segment file name before making a request for the 
> segment.

Ah, sure. That was my original plan, but then I realised that the URL
canonicalisation (or decomposition) step was already there, and I was
concerned about adding an extra step that changed the dynamics of URLs
for this specific case.

Let me take another look and see if I can figure out a reasonable way
of percent-encoding the URL in the specific case of playlists (or
possibly just HLS).

Thanks!

Tim
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] Posting correctly (was: Accept a colon in the path of a URI, instead of stripping preceding characters.)

[Nicolas George]

softworkz . (HE12025-05-21):
> From: Timothy Allen <tim@treehouse.org.za>
> Sent: Mittwoch, 21. Mai 2025 09:29
> To: softworkz . <softworkz@hotmail.com>; FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH] Accept a colon in the path of a URI, instead of stripping preceding characters.
> 
> On Tue, 2025-05-20 at 20:03 +0000, softworkz . wrote:
> I was just about to reply and suggest to replace those colons with %3A
> (url-encoded) when I read the ticket, which already suggests that.
<snip>

Your message is absolutely unreadable. It is not my software that makes
it so, it is unreadable on the official archive as well:

https://ffmpeg.org/pipermail/ffmpeg-devel/2025-May/343902.html

Compare with the message you are responding to:

https://ffmpeg.org/pipermail/ffmpeg-devel/2025-May/343841.html

It is impossible to know which parts you wrote and which parts you
quote, let alone distinguish quotes from double quotes.

Before you allow yourself to badmouth the tool mailing again, you need
to make the minimum amount of effort to use it properly. If not for
yourself, at least out of basic respects for the other people who use
it. That means making sure your mails are at least as readable as the
ones from other experienced developers. You can observe them rendered in
the archive, not mangled by your local mail software. Start with having
a real attribution instead of that awful blob of headers.

And if that makes you realize the software you have been using is crap
and change it, so much the better, and then realize that with proper
software a mailing-list is an excellent tool, so much the better. But I
will not hold my breath.

-- 
  Nicolas George
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".