Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
 help / color / mirror / Atom feed
From: Derek Buitenhuis <derek.buitenhuis@gmail.com>
To: ffmpeg-devel@ffmpeg.org
Subject: [FFmpeg-devel] [PATCH v2 4/9] avformat/http: Add support for Retry-After header
Date: Mon, 22 Apr 2024 15:25:42 +0100
Message-ID: <20240422142547.281064-5-derek.buitenhuis@gmail.com> (raw)
In-Reply-To: <20240422142547.281064-1-derek.buitenhuis@gmail.com>

429 and 503 codes can, and often do (e.g. all Google Cloud
Storage URLs can), return a Retry-After header with the error,
indicating how long to wait, in seconds, before retrying again.
If it is not respected by, for example, using our default backoff
stratetgy instead, chances of success are very unlikely.

This adds an AVOption to respect that header.

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
---
 libavformat/http.c    | 12 ++++++++++++
 libavformat/version.h |  2 +-
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/libavformat/http.c b/libavformat/http.c
index e7603037f4..5ed481b63a 100644
--- a/libavformat/http.c
+++ b/libavformat/http.c
@@ -138,6 +138,8 @@ typedef struct HTTPContext {
     char *new_location;
     AVDictionary *redirect_cache;
     uint64_t filesize_from_content_range;
+    int respect_retry_after;
+    unsigned int retry_after;
 } HTTPContext;
 
 #define OFFSET(x) offsetof(HTTPContext, x)
@@ -176,6 +178,7 @@ static const AVOption options[] = {
     { "reconnect_on_http_error", "list of http status codes to reconnect on", OFFSET(reconnect_on_http_error), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, D },
     { "reconnect_streamed", "auto reconnect streamed / non seekable streams", OFFSET(reconnect_streamed), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, D },
     { "reconnect_delay_max", "max reconnect delay in seconds after which to give up", OFFSET(reconnect_delay_max), AV_OPT_TYPE_INT, { .i64 = 120 }, 0, UINT_MAX/1000/1000, D },
+    { "respect_retry_after", "respect the Retry-After header when retrying connections", OFFSET(respect_retry_after), AV_OPT_TYPE_BOOL, { .i64 = 1 }, 0, 1, D },
     { "listen", "listen on HTTP", OFFSET(listen), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 2, D | E },
     { "resource", "The resource requested by a client", OFFSET(resource), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, E },
     { "reply_code", "The http status code to return to a client", OFFSET(reply_code), AV_OPT_TYPE_INT, { .i64 = 200}, INT_MIN, 599, E},
@@ -386,6 +389,13 @@ redo:
             reconnect_delay > s->reconnect_delay_max)
             goto fail;
 
+        if (s->respect_retry_after && s->retry_after > 0) {
+            reconnect_delay = s->retry_after;
+            if (reconnect_delay > s->reconnect_delay_max)
+                goto fail;
+            s->retry_after = 0;
+        }
+
         av_log(h, AV_LOG_WARNING, "Will reconnect at %"PRIu64" in %d second(s).\n", off, reconnect_delay);
         ret = ff_network_sleep_interruptible(1000U * 1000 * reconnect_delay, &h->interrupt_callback);
         if (ret != AVERROR(ETIMEDOUT))
@@ -1231,6 +1241,8 @@ static int process_line(URLContext *h, char *line, int line_count, int *parsed_h
             parse_expires(s, p);
         } else if (!av_strcasecmp(tag, "Cache-Control")) {
             parse_cache_control(s, p);
+        } else if (!av_strcasecmp(tag, "Retry-After")) {
+            s->retry_after = strtoul(p, NULL, 10);
         }
     }
     return 1;
diff --git a/libavformat/version.h b/libavformat/version.h
index 7ff1483912..ee91990360 100644
--- a/libavformat/version.h
+++ b/libavformat/version.h
@@ -32,7 +32,7 @@
 #include "version_major.h"
 
 #define LIBAVFORMAT_VERSION_MINOR   3
-#define LIBAVFORMAT_VERSION_MICRO 100
+#define LIBAVFORMAT_VERSION_MICRO 101
 
 #define LIBAVFORMAT_VERSION_INT AV_VERSION_INT(LIBAVFORMAT_VERSION_MAJOR, \
                                                LIBAVFORMAT_VERSION_MINOR, \
-- 
2.43.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

  parent reply	other threads:[~2024-04-22 14:26 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-04-22 14:25 [FFmpeg-devel] [PATCH v2 0/9] HTTP rate limiting and retry improvements Derek Buitenhuis
2024-04-22 14:25 ` [FFmpeg-devel] [PATCH v2 1/9] avutil/error: Add HTTP 429 Too Many Requests AVERROR code Derek Buitenhuis
2024-04-22 15:20   ` Derek Buitenhuis
2024-04-22 14:25 ` [FFmpeg-devel] [PATCH v2 2/9] avformat/http: Use AVERROR_HTTP_TOO_MANY_REQUESTS Derek Buitenhuis
2024-04-24 10:53   ` Martin Storsjö
2024-04-24 19:40     ` Derek Buitenhuis
2024-04-24 10:58   ` Martin Storsjö
2024-04-24 19:41     ` Derek Buitenhuis
2024-04-22 14:25 ` [FFmpeg-devel] [PATCH v2 3/9] avformat/http: Don't bail on parsing headers on "bad" HTTP codes Derek Buitenhuis
2024-04-22 14:25 ` Derek Buitenhuis [this message]
2024-04-24 11:06   ` [FFmpeg-devel] [PATCH v2 4/9] avformat/http: Add support for Retry-After header Martin Storsjö
2024-04-24 19:40     ` Derek Buitenhuis
2024-04-22 14:25 ` [FFmpeg-devel] [PATCH v2 5/9] avformat/http: Rename attempts to auth_attempts Derek Buitenhuis
2024-04-22 14:25 ` [FFmpeg-devel] [PATCH v2 6/9] avformat/http: Add options to set the max number of connection retries Derek Buitenhuis
2024-04-24 11:08   ` Martin Storsjö
2024-04-24 19:42     ` Derek Buitenhuis
2024-04-22 14:25 ` [FFmpeg-devel] [PATCH v2 7/9] avformat/http: Add option to limit total reconnect delay Derek Buitenhuis
2024-04-22 14:25 ` [FFmpeg-devel] [PATCH v2 8/9] doc/protocols: Re-order HTTP options to match http.c order Derek Buitenhuis
2024-04-22 14:25 ` [FFmpeg-devel] [PATCH v2 9/9] doc/protocols: Fill in missing HTTP options Derek Buitenhuis
2024-04-24 11:13 ` [FFmpeg-devel] [PATCH v2 0/9] HTTP rate limiting and retry improvements Martin Storsjö
2024-04-24 19:43   ` Derek Buitenhuis
2024-04-25 13:27     ` Derek Buitenhuis

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240422142547.281064-5-derek.buitenhuis@gmail.com \
    --to=derek.buitenhuis@gmail.com \
    --cc=ffmpeg-devel@ffmpeg.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
		ffmpegdev@gitmailbox.com
	public-inbox-index ffmpegdev

Example config snippet for mirrors.


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git