[FFmpeg-devel] [PR] avformat/http: add -request-size option (PR #21703)

[FFmpeg-devel] [PR] avformat/http: add -request-size option (PR #21703)

[Niklas Haas via ffmpeg-devel]

PR #21703 opened by Niklas Haas (haasn)
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21703
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21703.patch

It has come to my attention that a way to limit the request range size
would be useful in general, for reasons beyond just speeding up initial
header parsing.

This patch generalizez -initial_request_size to -request_size. I decided
to continue allowing both options to be used simultaneously, so users can
e.g. set -request_size to something large like 10 MiB, but still use a smaller
size for initial header parsing (e.g. 256 KiB).


>From 72ef7e4ae316f042761e457333fee6ff7ac4a3b9 Mon Sep 17 00:00:00 2001
From: Niklas Haas <git@haasn.dev>
Date: Mon, 9 Feb 2026 16:56:25 +0100
Subject: [PATCH] avformat/http: add -request-size option

It has come to my attention that a way to limit the request range size
would be useful in general, for reasons beyond just speeding up initial
header parsing.

This patch generalizez -initial_request_size to -request_size. I decided
to continue allowing both options to be used simultaneously, so users can
e.g. set -request_size to something large like 10 MiB, but still use a smaller
size for initial header parsing (e.g. 256 KiB).
---
 doc/protocols.texi | 25 +++++++++++++++++--------
 libavformat/http.c | 15 +++++++++------
 2 files changed, 26 insertions(+), 14 deletions(-)

diff --git a/doc/protocols.texi b/doc/protocols.texi
index 9c068075d5..0d9cc3c83e 100644
--- a/doc/protocols.texi
+++ b/doc/protocols.texi
@@ -473,18 +473,27 @@ Set the Referer header. Include 'Referer: URL' header in HTTP request.
 @item multiple_requests
 Use persistent connections if set to 1, default is 0.
 
-@item initial_request_size
-Limit the size of initial requests. This is useful when dealing with formats
-that require frequent seeks during initial parsing. This lasts until the
-demuxer makes a read request larger than this size (without a seek in between),
-after which the implementation will continue using unbounded requests as usual.
-Disabled (set to 0) by default.
+@item request_size
+Limit the size of requests made. This is useful for some pathological servers
+that throttle unbounded range requests, as well as when expecting to seek
+frequently. Disabled (set to 0) by default.
 
 Note that if enabling this option, it's strongly recommended to also enable
 the @option{multiple_requests} option, as well as setting
 @option{short_seek_size} to the same value or higher. Doing so allows FFmpeg
-to reuse a single HTTP connection wherever possible, even for formats like
-MXF or MOV that require frequent small seeks during initial parsing.
+to reuse a single HTTP connection wherever possible.
+
+@item initial_request_size
+Limit the size of initial requests. Similar to @code{request_size}, but only
+used during initial format parsing. Useful for formats like MXF or MOV that
+require frequent seeks during header parsing. Lasts until the demuxer makes a
+read request larger than this size (without a seek in between), after which
+the implementation will continue using requests as usual. Disabled (set to 0)
+by default.
+
+Note that if enabling this option, it's strongly recommended to also enable
+the @option{multiple_requests} option, as well as setting
+@option{short_seek_size} to the same value or higher.
 
 @item post_data
 Set custom HTTP post data.
diff --git a/libavformat/http.c b/libavformat/http.c
index 805e3eff9f..c62892ed43 100644
--- a/libavformat/http.c
+++ b/libavformat/http.c
@@ -147,7 +147,8 @@ typedef struct HTTPContext {
     int reconnect_max_retries;
     int reconnect_delay_total_max;
     uint64_t initial_request_size;
-    int partial_requests; /* whether or not to limit requests to initial_request_size */
+    uint64_t request_size;
+    int initial_requests; /* whether or not to limit requests to initial_request_size */
     /* Connection statistics */
     int nb_connections;
     int nb_requests;
@@ -172,6 +173,7 @@ static const AVOption options[] = {
     { "user_agent", "override User-Agent header", OFFSET(user_agent), AV_OPT_TYPE_STRING, { .str = DEFAULT_USER_AGENT }, 0, 0, D },
     { "referer", "override referer header", OFFSET(referer), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, D },
     { "multiple_requests", "use persistent connections", OFFSET(multiple_requests), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, D | E },
+    { "request_size", "size (in bytes) of requests to make", OFFSET(request_size), AV_OPT_TYPE_INT64, { .i64 = 0 }, 0, INT64_MAX, D },
     { "initial_request_size", "size (in bytes) of initial requests made during probing / header parsing", OFFSET(initial_request_size), AV_OPT_TYPE_INT64, { .i64 = 0 }, 0, INT64_MAX, D },
     { "post_data", "set custom HTTP post data", OFFSET(post_data), AV_OPT_TYPE_BINARY, .flags = D | E },
     { "mime_type", "export the MIME type", OFFSET(mime_type), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, AV_OPT_FLAG_EXPORT | AV_OPT_FLAG_READONLY },
@@ -769,7 +771,7 @@ static int http_open(URLContext *h, const char *uri, int flags,
     else
         h->is_streamed = 1;
 
-    s->partial_requests = s->seekable != 0 && s->initial_request_size > 0;
+    s->initial_requests = s->seekable != 0 && s->initial_request_size > 0;
     s->filesize = UINT64_MAX;
 
     s->location = av_strdup(uri);
@@ -1472,7 +1474,7 @@ static int http_read_header(URLContext *h)
         h->is_streamed = 1; /* we can in fact _not_ seek */
 
     if (h->is_streamed)
-        s->partial_requests = 0; /* unable to use partial requests */
+        s->initial_requests = 0; /* unable to use partial requests */
 
     // add any new cookies into the existing cookie string
     cookie_string(s->cookie_dict, &s->cookies);
@@ -1583,8 +1585,9 @@ static int http_connect(URLContext *h, const char *path, const char *local_path,
     // server supports seeking by analysing the reply headers.
     if (!has_header(s->headers, "\r\nRange: ") && !post && (s->off > 0 || s->end_off || s->seekable != 0)) {
         av_bprintf(&request, "Range: bytes=%"PRIu64"-", s->off);
-        if (s->partial_requests && s->seekable != 0) {
-            uint64_t target_off = s->off + s->initial_request_size;
+        if ((s->initial_requests || s->request_size) && s->seekable != 0) {
+            uint64_t req_size = s->initial_requests ? s->initial_request_size : s->request_size;
+            uint64_t target_off = s->off + req_size;
             if (target_off < s->off) /* overflow */
                 target_off = UINT64_MAX;
             if (s->end_off)
@@ -1835,7 +1838,7 @@ static int http_read_stream(URLContext *h, uint8_t *buf, int size)
             AVDictionary *options = NULL;
             if (s->willclose)
                 ffurl_closep(&s->hd);
-            s->partial_requests = 0; /* continue streaming uninterrupted from now on */
+            s->initial_requests = 0; /* continue streaming uninterrupted from now on */
             read_ret = http_open_cnx(h, &options);
             av_dict_free(&options);
             if (read_ret == 0)
-- 
2.52.0

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org