[FFmpeg-devel] [PATCH 1/2] http: Improve handling of Content-Range with Transfer-Encoding:chunked

[FFmpeg-devel] [PATCH 1/2] http: Improve handling of Content-Range with Transfer-Encoding:chunked

[Vittorio Giovara]

From: Justin Ruggles <justin.ruggles@gmail.com>

When Transfer-Encoding:chunked is used, the client must ignore a
Content-Length header, if present. However, it should not ignore a
Content-Range header, which also includes the full size of the
entity.
---
 libavformat/http.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/libavformat/http.c b/libavformat/http.c
index 8f39d11a88..c89f8a5517 100644
--- a/libavformat/http.c
+++ b/libavformat/http.c
@@ -132,6 +132,7 @@ typedef struct HTTPContext {
     int64_t expires;
     char *new_location;
     AVDictionary *redirect_cache;
+    uint64_t filesize_from_content_range;
 } HTTPContext;
 
 #define OFFSET(x) offsetof(HTTPContext, x)
@@ -839,7 +840,7 @@ static void parse_content_range(URLContext *h, const char *p)
         p     += 6;
         s->off = strtoull(p, NULL, 10);
         if ((slash = strchr(p, '/')) && strlen(slash) > 0)
-            s->filesize = strtoull(slash + 1, NULL, 10);
+            s->filesize_from_content_range = strtoull(slash + 1, NULL, 10);
     }
     if (s->seekable == -1 && (!s->is_akamai || s->filesize != 2147483647))
         h->is_streamed = 0; /* we _can_ in fact seek */
@@ -1341,6 +1342,7 @@ static int http_read_header(URLContext *h)
     av_freep(&s->new_location);
     s->expires = 0;
     s->chunksize = UINT64_MAX;
+    s->filesize_from_content_range = UINT64_MAX;
 
     for (;;) {
         if ((err = http_get_line(s, line, sizeof(line))) < 0)
@@ -1356,6 +1358,10 @@ static int http_read_header(URLContext *h)
         s->line_count++;
     }
 
+    // filesize from Content-Range can always be used, even if using chunked Transfer-Encoding
+    if (s->filesize_from_content_range != UINT64_MAX)
+        s->filesize = s->filesize_from_content_range;
+
     if (s->seekable == -1 && s->is_mediagateway && s->filesize == 2000000000)
         h->is_streamed = 1; /* we can in fact _not_ seek */
 
-- 
2.34.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 2/2] http: Send a Range header even when the offset is 0

[Vittorio Giovara]

From: Justin Ruggles <justin.ruggles@gmail.com>

Using Range allows for getting the full file size from the
Content-Range header in the response, even if the server sends
back the response using chunked Transfer-Encoding, which does not
allow using Content-Length.
---
 libavformat/http.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavformat/http.c b/libavformat/http.c
index c89f8a5517..c79db955e8 100644
--- a/libavformat/http.c
+++ b/libavformat/http.c
@@ -1469,10 +1469,10 @@ static int http_connect(URLContext *h, const char *path, const char *local_path,
     }
     if (!has_header(s->headers, "\r\nAccept: "))
         av_bprintf(&request, "Accept: */*\r\n");
-    // Note: we send this on purpose even when s->off is 0 when we're probing,
+    // Note: we send the Range header on purpose, even when we're probing,
     // since it allows us to detect more reliably if a (non-conforming)
     // server supports seeking by analysing the reply headers.
-    if (!has_header(s->headers, "\r\nRange: ") && !post && (s->off > 0 || s->end_off || s->seekable == -1)) {
+    if (!has_header(s->headers, "\r\nRange: ") && !post && (s->off > 0 || s->end_off || s->seekable != 0)) {
         av_bprintf(&request, "Range: bytes=%"PRIu64"-", s->off);
         if (s->end_off)
             av_bprintf(&request, "%"PRId64, s->end_off - 1);
-- 
2.34.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/2] http: Improve handling of Content-Range with Transfer-Encoding:chunked

[Aman Karmani]

On Wed, Feb 2, 2022 at 3:50 PM Vittorio Giovara <vittorio.giovara@gmail.com>
wrote:

> From: Justin Ruggles <justin.ruggles@gmail.com>
>
> When Transfer-Encoding:chunked is used, the client must ignore a
> Content-Length header, if present. However, it should not ignore a
> Content-Range header, which also includes the full size of the
> entity.
> ---
>  libavformat/http.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/libavformat/http.c b/libavformat/http.c
> index 8f39d11a88..c89f8a5517 100644
> --- a/libavformat/http.c
> +++ b/libavformat/http.c
> @@ -132,6 +132,7 @@ typedef struct HTTPContext {
>      int64_t expires;
>      char *new_location;
>      AVDictionary *redirect_cache;
> +    uint64_t filesize_from_content_range;
>  } HTTPContext;
>
>  #define OFFSET(x) offsetof(HTTPContext, x)
> @@ -839,7 +840,7 @@ static void parse_content_range(URLContext *h, const
> char *p)
>          p     += 6;
>          s->off = strtoull(p, NULL, 10);
>          if ((slash = strchr(p, '/')) && strlen(slash) > 0)
> -            s->filesize = strtoull(slash + 1, NULL, 10);
> +            s->filesize_from_content_range = strtoull(slash + 1, NULL,
> 10);
>

The size part of the range header is optional, and can be '*' as well.

See also
https://patchwork.ffmpeg.org/project/ffmpeg/patch/20211005233244.37582-1-ffmpeg@tmm1.net/


>      }
>      if (s->seekable == -1 && (!s->is_akamai || s->filesize != 2147483647))
>          h->is_streamed = 0; /* we _can_ in fact seek */
> @@ -1341,6 +1342,7 @@ static int http_read_header(URLContext *h)
>      av_freep(&s->new_location);
>      s->expires = 0;
>      s->chunksize = UINT64_MAX;
> +    s->filesize_from_content_range = UINT64_MAX;
>
>      for (;;) {
>          if ((err = http_get_line(s, line, sizeof(line))) < 0)
> @@ -1356,6 +1358,10 @@ static int http_read_header(URLContext *h)
>          s->line_count++;
>      }
>
> +    // filesize from Content-Range can always be used, even if using
> chunked Transfer-Encoding
> +    if (s->filesize_from_content_range != UINT64_MAX)
> +        s->filesize = s->filesize_from_content_range;
> +
>      if (s->seekable == -1 && s->is_mediagateway && s->filesize ==
> 2000000000)
>          h->is_streamed = 1; /* we can in fact _not_ seek */
>
> --
> 2.34.1
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/2] http: Improve handling of Content-Range with Transfer-Encoding:chunked

[Derek Buitenhuis]

On 2/3/2022 1:26 AM, Aman Karmani wrote:
> The size part of the range header is optional, and can be '*' as well.
> 
> See also
> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20211005233244.37582-1-ffmpeg@tmm1.net/

Isn't this an orthogonal problem to what this patch is changing?

- Derek
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/2] http: Improve handling of Content-Range with Transfer-Encoding:chunked

[Vittorio Giovara]

On Thu, Feb 3, 2022 at 3:12 PM Derek Buitenhuis <derek.buitenhuis@gmail.com>
wrote:

> On 2/3/2022 1:26 AM, Aman Karmani wrote:
> > The size part of the range header is optional, and can be '*' as well.
> >
> > See also
> >
> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20211005233244.37582-1-ffmpeg@tmm1.net/
>
> Isn't this an orthogonal problem to what this patch is changing?
>
> - Derek
> _______________________________________________
>

Are there any other comments/objections to the patchset?
Thanks
-- 
Vittorio
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/2] http: Improve handling of Content-Range with Transfer-Encoding:chunked

[Paul B Mahol]

On Wed, Feb 9, 2022 at 1:56 AM Vittorio Giovara <vittorio.giovara@gmail.com>
wrote:

> On Thu, Feb 3, 2022 at 3:12 PM Derek Buitenhuis <
> derek.buitenhuis@gmail.com>
> wrote:
>
> > On 2/3/2022 1:26 AM, Aman Karmani wrote:
> > > The size part of the range header is optional, and can be '*' as well.
> > >
> > > See also
> > >
> >
> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20211005233244.37582-1-ffmpeg@tmm1.net/
> >
> > Isn't this an orthogonal problem to what this patch is changing?
> >
> > - Derek
> > _______________________________________________
> >
>
> Are there any other comments/objections to the patchset?
>

Nope. Just apply it.


> Thanks
> --
> Vittorio
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 1/2] http: Improve handling of Content-Range with Transfer-Encoding:chunked

[Vittorio Giovara]

On Wed, Feb 9, 2022 at 9:49 AM Paul B Mahol <onemda@gmail.com> wrote:

> On Wed, Feb 9, 2022 at 1:56 AM Vittorio Giovara <
> vittorio.giovara@gmail.com>
> wrote:
>
> > On Thu, Feb 3, 2022 at 3:12 PM Derek Buitenhuis <
> > derek.buitenhuis@gmail.com>
> > wrote:
> >
> > > On 2/3/2022 1:26 AM, Aman Karmani wrote:
> > > > The size part of the range header is optional, and can be '*' as
> well.
> > > >
> > > > See also
> > > >
> > >
> >
> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20211005233244.37582-1-ffmpeg@tmm1.net/
> > >
> > > Isn't this an orthogonal problem to what this patch is changing?
> > >
> > > - Derek
> > > _______________________________________________
> > >
> >
> > Are there any other comments/objections to the patchset?
> >
>
> Nope. Just apply it.
>

Pushed o/
-- 
Vittorio
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".