[FFmpeg-devel] [PATCH v2] lavfi/vsrc_ddagrab: add an option to avoid duplicating frames

[FFmpeg-devel] [PATCH v2] lavfi/vsrc_ddagrab: add an option to avoid duplicating frames

[Anton Khirnov]

Tested-by: Jiří Eliášek, Misha Aizatulin
---
Now requested an infinite timeout from
IDXGIOutputDuplication_AcquireNextFrame() when a frame is required.
---
 doc/filters.texi           | 15 +++++++++++----
 libavfilter/vsrc_ddagrab.c |  9 +++++++--
 2 files changed, 18 insertions(+), 6 deletions(-)

diff --git a/doc/filters.texi b/doc/filters.texi
index 1d70f4d934..b9b539acee 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -28713,10 +28713,10 @@ it'll always be captured.
 @item framerate
 Maximum framerate at which the desktop will be captured - the interval between
 successive frames will not be smaller than the inverse of the framerate. When
-the desktop is not being updated often enough, the filter will duplicate
-a previous frame. Note that there is no background buffering going on, so when
-the filter is not polled often enough then the actual inter-frame interval may
-be significantly larger.
+@var{dup_frames} is true (the default) and the desktop is not being updated
+often enough, the filter will duplicate a previous frame. Note that there is no
+background buffering going on, so when the filter is not polled often enough
+then the actual inter-frame interval may be significantly larger.
 
 Defaults to 30 FPS.
 
@@ -28749,6 +28749,13 @@ Passes all supported output formats to DDA and returns what DDA decides to use.
 Filter initialization will fail if 10 bit format is requested but unavailable.
 @end table
 
+@item dup_frames
+When this option is set to true (the default), the filter will duplicate frames
+when the desktop has not been updated in order to maintain approximately
+constant target framerate. When this option is set to false, the filter will
+wait for the desktop to be updated (inter-frame intervals may vary significantly
+in this case).
+
 @end table
 
 @subsection Examples
diff --git a/libavfilter/vsrc_ddagrab.c b/libavfilter/vsrc_ddagrab.c
index 9c59faf53e..7109f5752b 100644
--- a/libavfilter/vsrc_ddagrab.c
+++ b/libavfilter/vsrc_ddagrab.c
@@ -101,6 +101,7 @@ typedef struct DdagrabContext {
     int        out_fmt;
     int        allow_fallback;
     int        force_fmt;
+    int        dup_frames;
 } DdagrabContext;
 
 #define OFFSET(x) offsetof(DdagrabContext, x)
@@ -124,6 +125,8 @@ static const AVOption ddagrab_options[] = {
                                                    OFFSET(allow_fallback), AV_OPT_TYPE_BOOL,   { .i64 = 0    },       0,       1, FLAGS },
     { "force_fmt",  "exclude BGRA from format list (experimental, discouraged by Microsoft)",
                                                    OFFSET(force_fmt),  AV_OPT_TYPE_BOOL,       { .i64 = 0    },       0,       1, FLAGS },
+    { "dup_frames",     "duplicate frames to maintain framerate",
+                                                   OFFSET(dup_frames), AV_OPT_TYPE_BOOL,       { .i64 = 1    },       0,       1, FLAGS },
     { NULL }
 };
 
@@ -688,7 +691,7 @@ static int next_frame_internal(AVFilterContext *avctx, ID3D11Texture2D **desktop
 
     hr = IDXGIOutputDuplication_AcquireNextFrame(
         dda->dxgi_outdupl,
-        dda->time_timeout,
+        need_frame ? INFINITE : dda->time_timeout,
         &frame_info,
         &desktop_resource);
     if (hr == DXGI_ERROR_WAIT_TIMEOUT) {
@@ -1067,7 +1070,9 @@ static int ddagrab_request_frame(AVFilterLink *outlink)
     now -= dda->first_pts;
 
     if (!dda->probed_texture) {
-        ret = next_frame_internal(avctx, &cur_texture, 0);
+        do {
+            ret = next_frame_internal(avctx, &cur_texture, !dda->dup_frames);
+        } while (ret == AVERROR(EAGAIN) && !dda->dup_frames);
     } else {
         cur_texture = dda->probed_texture;
         dda->probed_texture = NULL;
-- 
2.42.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2] lavfi/vsrc_ddagrab: add an option to avoid duplicating frames

[Timo Rothenpieler]

On 29.01.2024 19:31, Anton Khirnov wrote:
> Tested-by: Jiří Eliášek, Misha Aizatulin
> ---
> Now requested an infinite timeout from
> IDXGIOutputDuplication_AcquireNextFrame() when a frame is required.
> ---
>   doc/filters.texi           | 15 +++++++++++----
>   libavfilter/vsrc_ddagrab.c |  9 +++++++--
>   2 files changed, 18 insertions(+), 6 deletions(-)
> 
> diff --git a/doc/filters.texi b/doc/filters.texi
> index 1d70f4d934..b9b539acee 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -28713,10 +28713,10 @@ it'll always be captured.
>   @item framerate
>   Maximum framerate at which the desktop will be captured - the interval between
>   successive frames will not be smaller than the inverse of the framerate. When
> -the desktop is not being updated often enough, the filter will duplicate
> -a previous frame. Note that there is no background buffering going on, so when
> -the filter is not polled often enough then the actual inter-frame interval may
> -be significantly larger.
> +@var{dup_frames} is true (the default) and the desktop is not being updated
> +often enough, the filter will duplicate a previous frame. Note that there is no
> +background buffering going on, so when the filter is not polled often enough
> +then the actual inter-frame interval may be significantly larger.
>   
>   Defaults to 30 FPS.
>   
> @@ -28749,6 +28749,13 @@ Passes all supported output formats to DDA and returns what DDA decides to use.
>   Filter initialization will fail if 10 bit format is requested but unavailable.
>   @end table
>   
> +@item dup_frames
> +When this option is set to true (the default), the filter will duplicate frames
> +when the desktop has not been updated in order to maintain approximately
> +constant target framerate. When this option is set to false, the filter will
> +wait for the desktop to be updated (inter-frame intervals may vary significantly
> +in this case).
> +
>   @end table
>   
>   @subsection Examples
> diff --git a/libavfilter/vsrc_ddagrab.c b/libavfilter/vsrc_ddagrab.c
> index 9c59faf53e..7109f5752b 100644
> --- a/libavfilter/vsrc_ddagrab.c
> +++ b/libavfilter/vsrc_ddagrab.c
> @@ -101,6 +101,7 @@ typedef struct DdagrabContext {
>       int        out_fmt;
>       int        allow_fallback;
>       int        force_fmt;
> +    int        dup_frames;
>   } DdagrabContext;
>   
>   #define OFFSET(x) offsetof(DdagrabContext, x)
> @@ -124,6 +125,8 @@ static const AVOption ddagrab_options[] = {
>                                                      OFFSET(allow_fallback), AV_OPT_TYPE_BOOL,   { .i64 = 0    },       0,       1, FLAGS },
>       { "force_fmt",  "exclude BGRA from format list (experimental, discouraged by Microsoft)",
>                                                      OFFSET(force_fmt),  AV_OPT_TYPE_BOOL,       { .i64 = 0    },       0,       1, FLAGS },
> +    { "dup_frames",     "duplicate frames to maintain framerate",
> +                                                   OFFSET(dup_frames), AV_OPT_TYPE_BOOL,       { .i64 = 1    },       0,       1, FLAGS },

nit: align description properly

>       { NULL }
>   };
>   
> @@ -688,7 +691,7 @@ static int next_frame_internal(AVFilterContext *avctx, ID3D11Texture2D **desktop
>   
>       hr = IDXGIOutputDuplication_AcquireNextFrame(
>           dda->dxgi_outdupl,
> -        dda->time_timeout,
> +        need_frame ? INFINITE : dda->time_timeout,

My main worry with this is that the filter could potentially be sitting 
there for a really, really long time.
And also... see below.

>           &frame_info,
>           &desktop_resource);
>       if (hr == DXGI_ERROR_WAIT_TIMEOUT) {
> @@ -1067,7 +1070,9 @@ static int ddagrab_request_frame(AVFilterLink *outlink)
>       now -= dda->first_pts;
>   
>       if (!dda->probed_texture) {
> -        ret = next_frame_internal(avctx, &cur_texture, 0);
> +        do {
> +            ret = next_frame_internal(avctx, &cur_texture, !dda->dup_frames);
> +        } while (ret == AVERROR(EAGAIN) && !dda->dup_frames);

This is unfortunately not quiet correct, as it misunderstands the 
need_frame parameter.
The need_frame parameter is meant for the initial probing period, where 
an actual _frame_ is needed, outside of the probing, mouse-location-only 
updates are perfectly valid though.

So with this implementation, if the desktop remains static, but the 
mouse is moved, this loop will keep asking for a frame, and not record 
the mouse movement.

My idea for a fix would be to just pass 0 to need_frame as it was, but 
keep the outer loop like this.
And either make the INFINITE timeout depend on the dup_frames option 
directly, or just don't pass INFINITE, and let it loop.

>       } else {
>           cur_texture = dda->probed_texture;
>           dda->probed_texture = NULL;
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2] lavfi/vsrc_ddagrab: add an option to avoid duplicating frames

[Anton Khirnov]

Quoting Timo Rothenpieler (2024-01-29 19:55:26)
> On 29.01.2024 19:31, Anton Khirnov wrote:
> >       { NULL }
> >   };
> >   
> > @@ -688,7 +691,7 @@ static int next_frame_internal(AVFilterContext *avctx, ID3D11Texture2D **desktop
> >   
> >       hr = IDXGIOutputDuplication_AcquireNextFrame(
> >           dda->dxgi_outdupl,
> > -        dda->time_timeout,
> > +        need_frame ? INFINITE : dda->time_timeout,
> 
> My main worry with this is that the filter could potentially be sitting 
> there for a really, really long time.

I agree, ideally there should be some kind of an interrupt mechanism,
but AFAIK nothing of the sort exists in lavfi currently.

-- 
Anton Khirnov
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH] lavfi/vsrc_ddagrab: add an option to avoid duplicating frames

[Anton Khirnov]

Tested-by: Jiří Eliášek, Misha Aizatulin
---
 doc/filters.texi           | 15 +++++++++++----
 libavfilter/vsrc_ddagrab.c |  7 ++++++-
 2 files changed, 17 insertions(+), 5 deletions(-)

diff --git a/doc/filters.texi b/doc/filters.texi
index 1d70f4d934..b9b539acee 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -28713,10 +28713,10 @@ it'll always be captured.
 @item framerate
 Maximum framerate at which the desktop will be captured - the interval between
 successive frames will not be smaller than the inverse of the framerate. When
-the desktop is not being updated often enough, the filter will duplicate
-a previous frame. Note that there is no background buffering going on, so when
-the filter is not polled often enough then the actual inter-frame interval may
-be significantly larger.
+@var{dup_frames} is true (the default) and the desktop is not being updated
+often enough, the filter will duplicate a previous frame. Note that there is no
+background buffering going on, so when the filter is not polled often enough
+then the actual inter-frame interval may be significantly larger.
 
 Defaults to 30 FPS.
 
@@ -28749,6 +28749,13 @@ Passes all supported output formats to DDA and returns what DDA decides to use.
 Filter initialization will fail if 10 bit format is requested but unavailable.
 @end table
 
+@item dup_frames
+When this option is set to true (the default), the filter will duplicate frames
+when the desktop has not been updated in order to maintain approximately
+constant target framerate. When this option is set to false, the filter will
+wait for the desktop to be updated (inter-frame intervals may vary significantly
+in this case).
+
 @end table
 
 @subsection Examples
diff --git a/libavfilter/vsrc_ddagrab.c b/libavfilter/vsrc_ddagrab.c
index 9c59faf53e..51e928d785 100644
--- a/libavfilter/vsrc_ddagrab.c
+++ b/libavfilter/vsrc_ddagrab.c
@@ -101,6 +101,7 @@ typedef struct DdagrabContext {
     int        out_fmt;
     int        allow_fallback;
     int        force_fmt;
+    int        dup_frames;
 } DdagrabContext;
 
 #define OFFSET(x) offsetof(DdagrabContext, x)
@@ -124,6 +125,8 @@ static const AVOption ddagrab_options[] = {
                                                    OFFSET(allow_fallback), AV_OPT_TYPE_BOOL,   { .i64 = 0    },       0,       1, FLAGS },
     { "force_fmt",  "exclude BGRA from format list (experimental, discouraged by Microsoft)",
                                                    OFFSET(force_fmt),  AV_OPT_TYPE_BOOL,       { .i64 = 0    },       0,       1, FLAGS },
+    { "dup_frames", "duplicate frames to maintain framerate",
+                                                   OFFSET(dup_frames), AV_OPT_TYPE_BOOL,       { .i64 = 1    },       0,       1, FLAGS },
     { NULL }
 };
 
@@ -1067,7 +1070,9 @@ static int ddagrab_request_frame(AVFilterLink *outlink)
     now -= dda->first_pts;
 
     if (!dda->probed_texture) {
-        ret = next_frame_internal(avctx, &cur_texture, 0);
+        do {
+            ret = next_frame_internal(avctx, &cur_texture, 0);
+        } while (ret == AVERROR(EAGAIN) && !dda->dup_frames);
     } else {
         cur_texture = dda->probed_texture;
         dda->probed_texture = NULL;
-- 
2.42.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] lavfi/vsrc_ddagrab: add an option to avoid duplicating frames

[Timo Rothenpieler]

On 29.01.2024 20:03, Anton Khirnov wrote:
> Tested-by: Jiří Eliášek, Misha Aizatulin
> ---
>   doc/filters.texi           | 15 +++++++++++----
>   libavfilter/vsrc_ddagrab.c |  7 ++++++-
>   2 files changed, 17 insertions(+), 5 deletions(-)
> 
> diff --git a/doc/filters.texi b/doc/filters.texi
> index 1d70f4d934..b9b539acee 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -28713,10 +28713,10 @@ it'll always be captured.
>   @item framerate
>   Maximum framerate at which the desktop will be captured - the interval between
>   successive frames will not be smaller than the inverse of the framerate. When
> -the desktop is not being updated often enough, the filter will duplicate
> -a previous frame. Note that there is no background buffering going on, so when
> -the filter is not polled often enough then the actual inter-frame interval may
> -be significantly larger.
> +@var{dup_frames} is true (the default) and the desktop is not being updated
> +often enough, the filter will duplicate a previous frame. Note that there is no
> +background buffering going on, so when the filter is not polled often enough
> +then the actual inter-frame interval may be significantly larger.
>   
>   Defaults to 30 FPS.
>   
> @@ -28749,6 +28749,13 @@ Passes all supported output formats to DDA and returns what DDA decides to use.
>   Filter initialization will fail if 10 bit format is requested but unavailable.
>   @end table
>   
> +@item dup_frames
> +When this option is set to true (the default), the filter will duplicate frames
> +when the desktop has not been updated in order to maintain approximately
> +constant target framerate. When this option is set to false, the filter will
> +wait for the desktop to be updated (inter-frame intervals may vary significantly
> +in this case).
> +
>   @end table
>   
>   @subsection Examples
> diff --git a/libavfilter/vsrc_ddagrab.c b/libavfilter/vsrc_ddagrab.c
> index 9c59faf53e..51e928d785 100644
> --- a/libavfilter/vsrc_ddagrab.c
> +++ b/libavfilter/vsrc_ddagrab.c
> @@ -101,6 +101,7 @@ typedef struct DdagrabContext {
>       int        out_fmt;
>       int        allow_fallback;
>       int        force_fmt;
> +    int        dup_frames;
>   } DdagrabContext;
>   
>   #define OFFSET(x) offsetof(DdagrabContext, x)
> @@ -124,6 +125,8 @@ static const AVOption ddagrab_options[] = {
>                                                      OFFSET(allow_fallback), AV_OPT_TYPE_BOOL,   { .i64 = 0    },       0,       1, FLAGS },
>       { "force_fmt",  "exclude BGRA from format list (experimental, discouraged by Microsoft)",
>                                                      OFFSET(force_fmt),  AV_OPT_TYPE_BOOL,       { .i64 = 0    },       0,       1, FLAGS },
> +    { "dup_frames", "duplicate frames to maintain framerate",
> +                                                   OFFSET(dup_frames), AV_OPT_TYPE_BOOL,       { .i64 = 1    },       0,       1, FLAGS },
>       { NULL }
>   };
>   
> @@ -1067,7 +1070,9 @@ static int ddagrab_request_frame(AVFilterLink *outlink)
>       now -= dda->first_pts;
>   
>       if (!dda->probed_texture) {
> -        ret = next_frame_internal(avctx, &cur_texture, 0);
> +        do {
> +            ret = next_frame_internal(avctx, &cur_texture, 0);
> +        } while (ret == AVERROR(EAGAIN) && !dda->dup_frames);
>       } else {
>           cur_texture = dda->probed_texture;
>           dda->probed_texture = NULL;

looks good to me
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".