[FFmpeg-devel] [PATCH v3] avformat/hls: Add option to retry failed segments for hls

[FFmpeg-devel] [PATCH v3] avformat/hls: Add option to retry failed segments for hls

[gnattu]

Current HLS implementation simply skip a failed segment to catch up
the stream, but this is not optimal for some use cases like livestream
recording.
Add an option to retry a failed segment to ensure the output file is
a complete stream.

Signed-off-by: gnattu <gnattuoc@me.com>
---
Fixed commit message wrap
 libavformat/hls.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/libavformat/hls.c b/libavformat/hls.c
index e622425e80..2b977f9132 100644
--- a/libavformat/hls.c
+++ b/libavformat/hls.c
@@ -225,6 +225,7 @@ typedef struct HLSContext {
     int http_persistent;
     int http_multiple;
     int http_seekable;
+    int seg_max_retry;
     AVIOContext *playlist_pb;
     HLSCryptoContext  crypto_ctx;
 } HLSContext;
@@ -1472,6 +1473,7 @@ static int read_data(void *opaque, uint8_t *buf, int buf_size)
     int ret;
     int just_opened = 0;
     int reload_count = 0;
+    int segment_retries = 0;
     struct segment *seg;
 
 restart:
@@ -1563,9 +1565,18 @@ reload:
             av_log(v->parent, AV_LOG_WARNING, "Failed to open segment %"PRId64" of playlist %d\n",
                    v->cur_seq_no,
                    v->index);
-            v->cur_seq_no += 1;
+            if (segment_retries >= c->seg_max_retry) {
+                av_log(v->parent, AV_LOG_WARNING, "Segment %"PRId64" of playlist %d failed too many times, skipping\n",
+                       v->cur_seq_no,
+                       v->index);
+                v->cur_seq_no += 1;
+                segment_retries = 0;
+            } else {
+                segment_retries += 1;
+            }
             goto reload;
         }
+        segment_retries = 0;
         just_opened = 1;
     }
 
@@ -2549,6 +2560,8 @@ static const AVOption hls_options[] = {
         OFFSET(http_seekable), AV_OPT_TYPE_BOOL, { .i64 = -1}, -1, 1, FLAGS},
     {"seg_format_options", "Set options for segment demuxer",
         OFFSET(seg_format_opts), AV_OPT_TYPE_DICT, {.str = NULL}, 0, 0, FLAGS},
+    {"seg_max_retry", "Maximum number of times to reload a segment on error.",
+     OFFSET(seg_max_retry), AV_OPT_TYPE_INT, {.i64 = 0}, 0, INT_MAX, FLAGS},
     {NULL}
 };
 
-- 
2.37.0 (Apple Git-136)

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v3] avformat/hls: Add option to retry failed segments for hls

[Steven Liu]

gnattu <gnattuoc@me.com> 于2022年10月10日周一 20:09写道：
>
> Current HLS implementation simply skip a failed segment to catch up
> the stream, but this is not optimal for some use cases like livestream
> recording.
> Add an option to retry a failed segment to ensure the output file is
> a complete stream.
>
> Signed-off-by: gnattu <gnattuoc@me.com>
> ---
> Fixed commit message wrap
>  libavformat/hls.c | 15 ++++++++++++++-
>  1 file changed, 14 insertions(+), 1 deletion(-)
>
> diff --git a/libavformat/hls.c b/libavformat/hls.c
> index e622425e80..2b977f9132 100644
> --- a/libavformat/hls.c
> +++ b/libavformat/hls.c
> @@ -225,6 +225,7 @@ typedef struct HLSContext {
>      int http_persistent;
>      int http_multiple;
>      int http_seekable;
> +    int seg_max_retry;
>      AVIOContext *playlist_pb;
>      HLSCryptoContext  crypto_ctx;
>  } HLSContext;
> @@ -1472,6 +1473,7 @@ static int read_data(void *opaque, uint8_t *buf, int buf_size)
>      int ret;
>      int just_opened = 0;
>      int reload_count = 0;
> +    int segment_retries = 0;
>      struct segment *seg;
>
>  restart:
> @@ -1563,9 +1565,18 @@ reload:
>              av_log(v->parent, AV_LOG_WARNING, "Failed to open segment %"PRId64" of playlist %d\n",
>                     v->cur_seq_no,
>                     v->index);
> -            v->cur_seq_no += 1;
> +            if (segment_retries >= c->seg_max_retry) {
> +                av_log(v->parent, AV_LOG_WARNING, "Segment %"PRId64" of playlist %d failed too many times, skipping\n",
> +                       v->cur_seq_no,
> +                       v->index);
> +                v->cur_seq_no += 1;
> +                segment_retries = 0;
> +            } else {
> +                segment_retries += 1;
> +            }
>              goto reload;
>          }
> +        segment_retries = 0;
>          just_opened = 1;
>      }
>
> @@ -2549,6 +2560,8 @@ static const AVOption hls_options[] = {
>          OFFSET(http_seekable), AV_OPT_TYPE_BOOL, { .i64 = -1}, -1, 1, FLAGS},
>      {"seg_format_options", "Set options for segment demuxer",
>          OFFSET(seg_format_opts), AV_OPT_TYPE_DICT, {.str = NULL}, 0, 0, FLAGS},
> +    {"seg_max_retry", "Maximum number of times to reload a segment on error.",
> +     OFFSET(seg_max_retry), AV_OPT_TYPE_INT, {.i64 = 0}, 0, INT_MAX, FLAGS},
>      {NULL}
>  };
>
> --
> 2.37.0 (Apple Git-136)
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Not sure this functions is usefull. because there have a option named
"max_reload" for  playlist reload,
but this can be used for segment reload.

Perhaps there have some sence need reload segment, so this lookd ok to me.



Thanks
Steven
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v3] avformat/hls: Add option to retry failed segments for hls

[Steven Liu]

Steven Liu <lingjiujianke@gmail.com> 于2022年10月11日周二 10:55写道：
>
> gnattu <gnattuoc@me.com> 于2022年10月10日周一 20:09写道：
> >
> > Current HLS implementation simply skip a failed segment to catch up
> > the stream, but this is not optimal for some use cases like livestream
> > recording.
> > Add an option to retry a failed segment to ensure the output file is
> > a complete stream.
> >
> > Signed-off-by: gnattu <gnattuoc@me.com>
> > ---
> > Fixed commit message wrap
> >  libavformat/hls.c | 15 ++++++++++++++-
> >  1 file changed, 14 insertions(+), 1 deletion(-)
> >
> > diff --git a/libavformat/hls.c b/libavformat/hls.c
> > index e622425e80..2b977f9132 100644
> > --- a/libavformat/hls.c
> > +++ b/libavformat/hls.c
> > @@ -225,6 +225,7 @@ typedef struct HLSContext {
> >      int http_persistent;
> >      int http_multiple;
> >      int http_seekable;
> > +    int seg_max_retry;
> >      AVIOContext *playlist_pb;
> >      HLSCryptoContext  crypto_ctx;
> >  } HLSContext;
> > @@ -1472,6 +1473,7 @@ static int read_data(void *opaque, uint8_t *buf, int buf_size)
> >      int ret;
> >      int just_opened = 0;
> >      int reload_count = 0;
> > +    int segment_retries = 0;
> >      struct segment *seg;
> >
> >  restart:
> > @@ -1563,9 +1565,18 @@ reload:
> >              av_log(v->parent, AV_LOG_WARNING, "Failed to open segment %"PRId64" of playlist %d\n",
> >                     v->cur_seq_no,
> >                     v->index);
> > -            v->cur_seq_no += 1;
> > +            if (segment_retries >= c->seg_max_retry) {
> > +                av_log(v->parent, AV_LOG_WARNING, "Segment %"PRId64" of playlist %d failed too many times, skipping\n",
> > +                       v->cur_seq_no,
> > +                       v->index);
> > +                v->cur_seq_no += 1;
> > +                segment_retries = 0;
> > +            } else {
> > +                segment_retries += 1;
> > +            }
> >              goto reload;
> >          }
> > +        segment_retries = 0;
> >          just_opened = 1;
> >      }
> >
> > @@ -2549,6 +2560,8 @@ static const AVOption hls_options[] = {
> >          OFFSET(http_seekable), AV_OPT_TYPE_BOOL, { .i64 = -1}, -1, 1, FLAGS},
> >      {"seg_format_options", "Set options for segment demuxer",
> >          OFFSET(seg_format_opts), AV_OPT_TYPE_DICT, {.str = NULL}, 0, 0, FLAGS},
> > +    {"seg_max_retry", "Maximum number of times to reload a segment on error.",
> > +     OFFSET(seg_max_retry), AV_OPT_TYPE_INT, {.i64 = 0}, 0, INT_MAX, FLAGS},
BTW, Document should add describe about the option.
> >      {NULL}
> >  };
> >
> > --
> > 2.37.0 (Apple Git-136)
> >
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
> Not sure this functions is usefull. because there have a option named
> "max_reload" for  playlist reload,
> but this can be used for segment reload.
>
> Perhaps there have some sence need reload segment, so this lookd ok to me.

>
>
>
> Thanks
> Steven
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v3] avformat/hls: Add option to retry failed segments for hls

[ChenLiucheng]

> On Oct 11, 2022, at 14:19, Steven Liu <lingjiujianke@gmail.com <mailto:lingjiujianke@gmail.com>> wrote:
> 
> Steven Liu <lingjiujianke@gmail.com <mailto:lingjiujianke@gmail.com>> 于2022年10月11日周二 10:55写道：
>> 
>> gnattu <gnattuoc@me.com <mailto:gnattuoc@me.com>> 于2022年10月10日周一 20:09写道：
>>> 
>>> Current HLS implementation simply skip a failed segment to catch up
>>> the stream, but this is not optimal for some use cases like livestream
>>> recording.
>>> Add an option to retry a failed segment to ensure the output file is
>>> a complete stream.
>>> 
>>> Signed-off-by: gnattu <gnattuoc@me.com <mailto:gnattuoc@me.com>>
>>> ---
>>> Fixed commit message wrap
>>> libavformat/hls.c | 15 ++++++++++++++-
>>> 1 file changed, 14 insertions(+), 1 deletion(-)
>>> 
>>> diff --git a/libavformat/hls.c b/libavformat/hls.c
>>> index e622425e80..2b977f9132 100644
>>> --- a/libavformat/hls.c
>>> +++ b/libavformat/hls.c
>>> @@ -225,6 +225,7 @@ typedef struct HLSContext {
>>>     int http_persistent;
>>>     int http_multiple;
>>>     int http_seekable;
>>> +    int seg_max_retry;
>>>     AVIOContext *playlist_pb;
>>>     HLSCryptoContext  crypto_ctx;
>>> } HLSContext;
>>> @@ -1472,6 +1473,7 @@ static int read_data(void *opaque, uint8_t *buf, int buf_size)
>>>     int ret;
>>>     int just_opened = 0;
>>>     int reload_count = 0;
>>> +    int segment_retries = 0;
>>>     struct segment *seg;
>>> 
>>> restart:
>>> @@ -1563,9 +1565,18 @@ reload:
>>>             av_log(v->parent, AV_LOG_WARNING, "Failed to open segment %"PRId64" of playlist %d\n",
>>>                    v->cur_seq_no,
>>>                    v->index);
>>> -            v->cur_seq_no += 1;
>>> +            if (segment_retries >= c->seg_max_retry) {
>>> +                av_log(v->parent, AV_LOG_WARNING, "Segment %"PRId64" of playlist %d failed too many times, skipping\n",
>>> +                       v->cur_seq_no,
>>> +                       v->index);
>>> +                v->cur_seq_no += 1;
>>> +                segment_retries = 0;
>>> +            } else {
>>> +                segment_retries += 1;
>>> +            }
>>>             goto reload;
>>>         }
>>> +        segment_retries = 0;
>>>         just_opened = 1;
>>>     }
>>> 
>>> @@ -2549,6 +2560,8 @@ static const AVOption hls_options[] = {
>>>         OFFSET(http_seekable), AV_OPT_TYPE_BOOL, { .i64 = -1}, -1, 1, FLAGS},
>>>     {"seg_format_options", "Set options for segment demuxer",
>>>         OFFSET(seg_format_opts), AV_OPT_TYPE_DICT, {.str = NULL}, 0, 0, FLAGS},
>>> +    {"seg_max_retry", "Maximum number of times to reload a segment on error.",
>>> +     OFFSET(seg_max_retry), AV_OPT_TYPE_INT, {.i64 = 0}, 0, INT_MAX, FLAGS},
> BTW, Document should add describe about the option.
>>>     {NULL}
>>> };
>>> 
>>> --
>>> 2.37.0 (Apple Git-136)
>>> 
>>> _______________________________________________
>>> ffmpeg-devel mailing list
>>> ffmpeg-devel@ffmpeg.org <mailto:ffmpeg-devel@ffmpeg.org>
>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>> 
>>> To unsubscribe, visit link above, or email
>>> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>> 
>> Not sure this functions is usefull. because there have a option named
>> "max_reload" for  playlist reload,
>> but this can be used for segment reload.
>> 
>> Perhaps there have some sence need reload segment, so this lookd ok to me.
> 
>> 
>> 
>> 
>> Thanks
>> Steven


May I have more clarification on this? Like on which aspect it need to have more details to better describe this option? I tried to keep the help text as concise as possible and I probably missed something here. 
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v3] avformat/hls: Add option to retry failed segments for hls

[Steven Liu]

ChenLiucheng <gnattuoc@me.com> 于2022年10月11日周二 20:14写道：
>
>
>
> On Oct 11, 2022, at 14:19, Steven Liu <lingjiujianke@gmail.com> wrote:
>
> Steven Liu <lingjiujianke@gmail.com> 于2022年10月11日周二 10:55写道：
>
>
> gnattu <gnattuoc@me.com> 于2022年10月10日周一 20:09写道：
>
>
> Current HLS implementation simply skip a failed segment to catch up
> the stream, but this is not optimal for some use cases like livestream
> recording.
> Add an option to retry a failed segment to ensure the output file is
> a complete stream.
>
> Signed-off-by: gnattu <gnattuoc@me.com>
> ---
> Fixed commit message wrap
> libavformat/hls.c | 15 ++++++++++++++-
> 1 file changed, 14 insertions(+), 1 deletion(-)
>
> diff --git a/libavformat/hls.c b/libavformat/hls.c
> index e622425e80..2b977f9132 100644
> --- a/libavformat/hls.c
> +++ b/libavformat/hls.c
> @@ -225,6 +225,7 @@ typedef struct HLSContext {
>     int http_persistent;
>     int http_multiple;
>     int http_seekable;
> +    int seg_max_retry;
>     AVIOContext *playlist_pb;
>     HLSCryptoContext  crypto_ctx;
> } HLSContext;
> @@ -1472,6 +1473,7 @@ static int read_data(void *opaque, uint8_t *buf, int buf_size)
>     int ret;
>     int just_opened = 0;
>     int reload_count = 0;
> +    int segment_retries = 0;
>     struct segment *seg;
>
> restart:
> @@ -1563,9 +1565,18 @@ reload:
>             av_log(v->parent, AV_LOG_WARNING, "Failed to open segment %"PRId64" of playlist %d\n",
>                    v->cur_seq_no,
>                    v->index);
> -            v->cur_seq_no += 1;
> +            if (segment_retries >= c->seg_max_retry) {
> +                av_log(v->parent, AV_LOG_WARNING, "Segment %"PRId64" of playlist %d failed too many times, skipping\n",
> +                       v->cur_seq_no,
> +                       v->index);
> +                v->cur_seq_no += 1;
> +                segment_retries = 0;
> +            } else {
> +                segment_retries += 1;
> +            }
>             goto reload;
>         }
> +        segment_retries = 0;
>         just_opened = 1;
>     }
>
> @@ -2549,6 +2560,8 @@ static const AVOption hls_options[] = {
>         OFFSET(http_seekable), AV_OPT_TYPE_BOOL, { .i64 = -1}, -1, 1, FLAGS},
>     {"seg_format_options", "Set options for segment demuxer",
>         OFFSET(seg_format_opts), AV_OPT_TYPE_DICT, {.str = NULL}, 0, 0, FLAGS},
> +    {"seg_max_retry", "Maximum number of times to reload a segment on error.",
> +     OFFSET(seg_max_retry), AV_OPT_TYPE_INT, {.i64 = 0}, 0, INT_MAX, FLAGS},
>
> BTW, Document should add describe about the option.
>
>     {NULL}
> };
>
> --
> 2.37.0 (Apple Git-136)
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
>
> Not sure this functions is usefull. because there have a option named
> "max_reload" for  playlist reload,
> but this can be used for segment reload.
>
> Perhaps there have some sence need reload segment, so this lookd ok to me.
>
>
>
>
>
> Thanks
> Steven
>
>
>
> May I have more clarification on this? Like on which aspect it need to have more details to better describe this option? I tried to keep the help text as concise as possible and I probably missed something here.
Do you mean documents? If yes, you need modify the file
./doc/demuxers.texi and add the option description, you can reference
the "max_reload" option.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".