[FFmpeg-devel] [PATCH 1/2] avutil/hwcontext_d3d11va: make D3D11 surface BindFlags & MiscFlags configurable

[FFmpeg-devel] [PATCH 1/2] avutil/hwcontext_d3d11va: make D3D11 surface BindFlags & MiscFlags configurable

[Dmitrii Ovchinnikov]

Certain components require ID3D11/ID3D12 textures to be created with
additional BindFlags or MiscFlags. FFmpeg currently provides no
mechanism to configure these flags externally, leaving no way to
satisfy such components without patching the library.

This patch adds private device options that can be passed through
  shader_resource=1        adds D3D11_BIND_SHADER_RESOURCE
  uav_resource=1           adds D3D11_BIND_UNORDERED_ACCESS
  render_target_resource=1 adds D3D11_BIND_RENDER_TARGET
  shared_resource=1        adds D3D11_RESOURCE_MISC_SHARED
---
 libavutil/hwcontext_d3d11va.c | 17 +++++++++++++++++
 libavutil/hwcontext_d3d11va.h |  5 +++++
 2 files changed, 22 insertions(+)

diff --git a/libavutil/hwcontext_d3d11va.c b/libavutil/hwcontext_d3d11va.c
index 1a047ce57b..fba9597392 100644
--- a/libavutil/hwcontext_d3d11va.c
+++ b/libavutil/hwcontext_d3d11va.c
@@ -288,6 +288,9 @@ static int d3d11va_frames_init(AVHWFramesContext *ctx)
         return AVERROR(EINVAL);
     }
 
+    hwctx->BindFlags |= device_hwctx->BindFlags;
+    hwctx->MiscFlags |= device_hwctx->MiscFlags;
+
     texDesc = (D3D11_TEXTURE2D_DESC){
         .Width      = ctx->width,
         .Height     = ctx->height,
@@ -705,6 +708,20 @@ static int d3d11va_device_create(AVHWDeviceContext *ctx, const char *device,
         }
     }
 #endif
+    device_hwctx->BindFlags = 0;
+    device_hwctx->MiscFlags = 0;
+
+    if (av_dict_get(opts, "shader_resouce", NULL, 0))
+        device_hwctx->BindFlags |= D3D11_BIND_SHADER_RESOURCE;
+
+    if (av_dict_get(opts, "uav_resource", NULL, 0))
+        device_hwctx->BindFlags |= D3D11_BIND_UNORDERED_ACCESS;
+
+    if (av_dict_get(opts, "render_target_resource", NULL, 0))
+        device_hwctx->BindFlags |= D3D11_BIND_RENDER_TARGET;
+
+    if (av_dict_get(opts, "shared_resource", NULL, 0))
+        device_hwctx->MiscFlags |= D3D11_RESOURCE_MISC_SHARED;
 
     return 0;
 }
diff --git a/libavutil/hwcontext_d3d11va.h b/libavutil/hwcontext_d3d11va.h
index 77d2d72f1b..e4e1f1095d 100644
--- a/libavutil/hwcontext_d3d11va.h
+++ b/libavutil/hwcontext_d3d11va.h
@@ -94,6 +94,11 @@ typedef struct AVD3D11VADeviceContext {
     void (*lock)(void *lock_ctx);
     void (*unlock)(void *lock_ctx);
     void *lock_ctx;
+    /**
+     * Need to pass value to frames context from command line
+    */
+    UINT BindFlags;
+    UINT MiscFlags;
 } AVD3D11VADeviceContext;
 
 /**
-- 
2.47.1.windows.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 2/2] avutil/hwcontext_d3d12va: added resource Flags support to command line

[Dmitrii Ovchinnikov]

Certain components require ID3D11/ID3D12 textures to be created with
additional BindFlags or MiscFlags. FFmpeg currently provides no
mechanism to configure these flags externally, leaving no way to
satisfy such components without patching the library.

This patch adds private device options that can be passed through
  uav_resource=1        adds D3D12_RESOURCE_FLAG_ALLOW_UNORDERED_ACCESS
  render_target_resource=1 adds D3D12_RESOURCE_FLAG_ALLOW_RENDER_TARGET
  shared_resource=1        adds D3D12_HEAP_FLAG_SHARED
---
 libavutil/hwcontext_d3d12va.c | 20 +++++++++++++++++++-
 libavutil/hwcontext_d3d12va.h |  5 +++++
 2 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/libavutil/hwcontext_d3d12va.c b/libavutil/hwcontext_d3d12va.c
index 6507cf69c1..67c80c290e 100644
--- a/libavutil/hwcontext_d3d12va.c
+++ b/libavutil/hwcontext_d3d12va.c
@@ -237,6 +237,9 @@ static AVBufferRef *d3d12va_pool_alloc(void *opaque, size_t size)
     AVBufferRef *buf;
     AVD3D12VAFrame *frame;
     D3D12_HEAP_PROPERTIES props = { .Type = D3D12_HEAP_TYPE_DEFAULT };
+
+    hwctx->flags |= device_hwctx->flags;
+
     D3D12_RESOURCE_DESC desc = {
         .Dimension        = D3D12_RESOURCE_DIMENSION_TEXTURE2D,
         .Alignment        = 0,
@@ -254,7 +257,7 @@ static AVBufferRef *d3d12va_pool_alloc(void *opaque, size_t size)
     if (!frame)
         return NULL;
 
-    if (FAILED(ID3D12Device_CreateCommittedResource(device_hwctx->device, &props, D3D12_HEAP_FLAG_NONE, &desc,
+    if (FAILED(ID3D12Device_CreateCommittedResource(device_hwctx->device, &props, device_hwctx->heap_flags, &desc,
         D3D12_RESOURCE_STATE_COMMON, NULL, &IID_ID3D12Resource, (void **)&frame->texture))) {
         av_log(ctx, AV_LOG_ERROR, "Could not create the texture\n");
         goto fail;
@@ -673,6 +676,21 @@ static int d3d12va_device_create(AVHWDeviceContext *hwdev, const char *device,
         }
     }
 
+    ctx->flags = 0;
+
+    // "shader_resouce" already enabled by default.
+
+    if (av_dict_get(opts, "uav_resource", NULL, 0))
+        ctx->flags |= D3D12_RESOURCE_FLAG_ALLOW_UNORDERED_ACCESS;
+
+    if (av_dict_get(opts, "render_target_resource", NULL, 0))
+        ctx->flags |= D3D12_RESOURCE_FLAG_ALLOW_RENDER_TARGET;
+
+    ctx->heap_flags = D3D12_HEAP_FLAG_NONE;
+
+    if (av_dict_get(opts, "shared_resource", NULL, 0))
+        ctx->heap_flags |= D3D12_HEAP_FLAG_SHARED;
+
     return 0;
 }
 
diff --git a/libavutil/hwcontext_d3d12va.h b/libavutil/hwcontext_d3d12va.h
index 212a6a6146..f1aea52559 100644
--- a/libavutil/hwcontext_d3d12va.h
+++ b/libavutil/hwcontext_d3d12va.h
@@ -75,6 +75,11 @@ typedef struct AVD3D12VADeviceContext {
     void (*lock)(void *lock_ctx);
     void (*unlock)(void *lock_ctx);
     void *lock_ctx;
+    /**
+     * Need to pass value to frames context from command line
+    */
+    D3D12_RESOURCE_FLAGS flags;
+    D3D12_HEAP_FLAGS heap_flags;
 } AVD3D12VADeviceContext;
 
 /**
-- 
2.47.1.windows.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] 回复: [PATCH 2/2] avutil/hwcontext_d3d12va: added resource Flags support to command line

[Wu Jianhua]

Dmitrii Ovchinnikov <ovchinnikov.dmitrii@gmail.com>:
> Certain components require ID3D11/ID3D12 textures to be created with
> additional BindFlags or MiscFlags. FFmpeg currently provides no
> mechanism to configure these flags externally, leaving no way to
> satisfy such components without patching the library.
>
> This patch adds private device options that can be passed through
>   uav_resource=1        adds D3D12_RESOURCE_FLAG_ALLOW_UNORDERED_ACCESS
>   render_target_resource=1 adds D3D12_RESOURCE_FLAG_ALLOW_RENDER_TARGET
>   shared_resource=1        adds D3D12_HEAP_FLAG_SHARED
> ---
>  libavutil/hwcontext_d3d12va.c | 20 +++++++++++++++++++-
>  libavutil/hwcontext_d3d12va.h |  5 +++++
>  2 files changed, 24 insertions(+), 1 deletion(-)
>
> diff --git a/libavutil/hwcontext_d3d12va.c b/libavutil/hwcontext_d3d12va.c
> index 6507cf69c1..67c80c290e 100644
> --- a/libavutil/hwcontext_d3d12va.c
> +++ b/libavutil/hwcontext_d3d12va.c
> @@ -237,6 +237,9 @@ static AVBufferRef *d3d12va_pool_alloc(void *opaque, size_t size)
>      AVBufferRef *buf;
>      AVD3D12VAFrame *frame;
>      D3D12_HEAP_PROPERTIES props = { .Type = D3D12_HEAP_TYPE_DEFAULT };
> +
> +    hwctx->flags |= device_hwctx->flags;
> +
>      D3D12_RESOURCE_DESC desc = {
>          .Dimension        = D3D12_RESOURCE_DIMENSION_TEXTURE2D,
>          .Alignment        = 0,
> @@ -254,7 +257,7 @@ static AVBufferRef *d3d12va_pool_alloc(void *opaque, size_t size)
>      if (!frame)
>          return NULL;
>
> -    if (FAILED(ID3D12Device_CreateCommittedResource(device_hwctx->device, &props, D3D12_HEAP_FLAG_NONE, &desc,
> +    if (FAILED(ID3D12Device_CreateCommittedResource(device_hwctx->device, &props, device_hwctx->heap_flags, &desc,
>          D3D12_RESOURCE_STATE_COMMON, NULL, &IID_ID3D12Resource, (void **)&frame->texture))) {
>          av_log(ctx, AV_LOG_ERROR, "Could not create the texture\n");
>          goto fail;
> @@ -673,6 +676,21 @@ static int d3d12va_device_create(AVHWDeviceContext *hwdev, const char *device,
>          }
>      }
>
> +    ctx->flags = 0;
> +
> +    // "shader_resouce" already enabled by default.
> +
> +    if (av_dict_get(opts, "uav_resource", NULL, 0))
> +        ctx->flags |= D3D12_RESOURCE_FLAG_ALLOW_UNORDERED_ACCESS;
> +
> +    if (av_dict_get(opts, "render_target_resource", NULL, 0))
> +        ctx->flags |= D3D12_RESOURCE_FLAG_ALLOW_RENDER_TARGET;
> +
> +    ctx->heap_flags = D3D12_HEAP_FLAG_NONE;
> +
> +    if (av_dict_get(opts, "shared_resource", NULL, 0))
> +        ctx->heap_flags |= D3D12_HEAP_FLAG_SHARED;
> +
>      return 0;
>  }
>
> diff --git a/libavutil/hwcontext_d3d12va.h b/libavutil/hwcontext_d3d12va.h
> index 212a6a6146..f1aea52559 100644
> --- a/libavutil/hwcontext_d3d12va.h
> +++ b/libavutil/hwcontext_d3d12va.h
> @@ -75,6 +75,11 @@ typedef struct AVD3D12VADeviceContext {
>      void (*lock)(void *lock_ctx);
>      void (*unlock)(void *lock_ctx);
>      void *lock_ctx;
> +    /**
> +     * Need to pass value to frames context from command line
> +    */
> +    D3D12_RESOURCE_FLAGS flags;
> +    D3D12_HEAP_FLAGS heap_flags;
>  } AVD3D12VADeviceContext;
>
>  /**
>

Hi Dmitrii,

The AVD3D12VAFramesContext includes dedicated resource flag configuration
for resource management. These flags can be directly accessed and extended
with heap flags as needed, since resource allocation is handled at the frame
level rather than globally at the device level. Decoder and encoder instances
can independently initialize their frame contexts, each with tailored resource
flags. For example, the encoder has used the following resource flags, but
the decoder hasn't.

- D3D12_RESOURCE_FLAG_VIDEO_ENCODE_REFERENCE_ONLY
- D3D12_RESOURCE_FLAG_DENY_SHADER_RESOURCE

Thanks,
Jianhua
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/2] avutil/hwcontext_d3d12va: added resource Flags support to command line

[Dmitrii Ovchinnikov]

Hi Jianhua,

The patch is designed for cases when a component doesn't allocate frames,
but only uses frames that come from outside (for example, from a decoder)
and has requirements for them. In cases where flags are configured this
way, we can avoid unnecessary surface copying, while the default behavior
remains unchanged.

For example, an AMF encoder with enabled PA wants frames to come from the
decoder with the D3D11_BIND_SHADER_RESOURCE flag (in the case of DirectX
11).

Some filters(especially shader based) may also have requirements for
necessary flags.

Sincerely, Ovchinnikov D.A.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] 回复: [PATCH 2/2] avutil/hwcontext_d3d12va: added resource Flags support to command line

[Wu Jianhua]

Dmitrii Ovchinnikov <ovchinnikov.dmitrii@gmail.com>:
> Wu Jianhuaw <toqsxw@outlook.com>
>> Dmitrii Ovchinnikov <ovchinnikov.dmitrii@gmail.com>:
>>> Certain components require ID3D11/ID3D12 textures to be created with
>>> additional BindFlags or MiscFlags. FFmpeg currently provides no
>>> mechanism to configure these flags externally, leaving no way to
>>> satisfy such components without patching the library.
>>>
>>> This patch adds private device options that can be passed through
>>>   uav_resource=1        adds D3D12_RESOURCE_FLAG_ALLOW_UNORDERED_ACCESS
>>>   render_target_resource=1 adds D3D12_RESOURCE_FLAG_ALLOW_RENDER_TARGET
>>>   shared_resource=1        adds D3D12_HEAP_FLAG_SHARED
>>> ---
>>>  libavutil/hwcontext_d3d12va.c | 20 +++++++++++++++++++-
>>>  libavutil/hwcontext_d3d12va.h |  5 +++++
>>>  2 files changed, 24 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/libavutil/hwcontext_d3d12va.c b/libavutil/hwcontext_d3d12va.c
>>> index 6507cf69c1..67c80c290e 100644
>>> --- a/libavutil/hwcontext_d3d12va.c
>>> +++ b/libavutil/hwcontext_d3d12va.c
>>> @@ -237,6 +237,9 @@ static AVBufferRef *d3d12va_pool_alloc(void *opaque, size_t size)
>>>      AVBufferRef *buf;
>>>      AVD3D12VAFrame *frame;
>>>      D3D12_HEAP_PROPERTIES props = { .Type = D3D12_HEAP_TYPE_DEFAULT };
>>> +
>>> +    hwctx->flags |= device_hwctx->flags;
>>> +
>>>      D3D12_RESOURCE_DESC desc = {
>>>          .Dimension        = D3D12_RESOURCE_DIMENSION_TEXTURE2D,
>>>          .Alignment        = 0,
>>> @@ -254,7 +257,7 @@ static AVBufferRef *d3d12va_pool_alloc(void *opaque, size_t size)
>>>      if (!frame)
>>>          return NULL;
>>>
>>> -    if (FAILED(ID3D12Device_CreateCommittedResource(device_hwctx->device, &props, D3D12_HEAP_FLAG_NONE, &desc,
>>> +    if (FAILED(ID3D12Device_CreateCommittedResource(device_hwctx->device, &props, device_hwctx->heap_flags, &desc,
>>>          D3D12_RESOURCE_STATE_COMMON, NULL, &IID_ID3D12Resource, (void **)&frame->texture))) {
>>>          av_log(ctx, AV_LOG_ERROR, "Could not create the texture\n");
>>>          goto fail;
>>> @@ -673,6 +676,21 @@ static int d3d12va_device_create(AVHWDeviceContext *hwdev, const char *device,
>>>          }
>>>      }
>>>
>>> +    ctx->flags = 0;
>>> +
>>> +    // "shader_resouce" already enabled by default.
>>> +
>>> +    if (av_dict_get(opts, "uav_resource", NULL, 0))
>>> +        ctx->flags |= D3D12_RESOURCE_FLAG_ALLOW_UNORDERED_ACCESS;
>>> +
>>> +    if (av_dict_get(opts, "render_target_resource", NULL, 0))
>>> +        ctx->flags |= D3D12_RESOURCE_FLAG_ALLOW_RENDER_TARGET;
>>> +
>>> +    ctx->heap_flags = D3D12_HEAP_FLAG_NONE;
>>> +
>>> +    if (av_dict_get(opts, "shared_resource", NULL, 0))
>>> +        ctx->heap_flags |= D3D12_HEAP_FLAG_SHARED;
>>> +
>>>      return 0;
>>>  }
>>>
>>> diff --git a/libavutil/hwcontext_d3d12va.h b/libavutil/hwcontext_d3d12va.h
>>> index 212a6a6146..f1aea52559 100644
>>> --- a/libavutil/hwcontext_d3d12va.h
>>> +++ b/libavutil/hwcontext_d3d12va.h
>>> @@ -75,6 +75,11 @@ typedef struct AVD3D12VADeviceContext {
>>>      void (*lock)(void *lock_ctx);
>>>      void (*unlock)(void *lock_ctx);
>>>      void *lock_ctx;
>>> +    /**
>>> +     * Need to pass value to frames context from command line
>>> +    */
>>> +    D3D12_RESOURCE_FLAGS flags;
>>> +    D3D12_HEAP_FLAGS heap_flags;
>>>  } AVD3D12VADeviceContext;
>>>
>>>  /**
>>>
>>
>> Hi Dmitrii,
>>
>> The AVD3D12VAFramesContext includes dedicated resource flag configuration
>> for resource management. These flags can be directly accessed and extended
>> with heap flags as needed, since resource allocation is handled at the frame
>> level rather than globally at the device level. Decoder and encoder instances
>> can independently initialize their frame contexts, each with tailored resource
>> flags. For example, the encoder has used the following resource flags, but
>> the decoder hasn't.
>>
>> - D3D12_RESOURCE_FLAG_VIDEO_ENCODE_REFERENCE_ONLY
>> - D3D12_RESOURCE_FLAG_DENY_SHADER_RESOURCE
>>
>> Thanks,
>> Jianhua
>
> Hi Jianhua,
>
> The patch is designed for cases when a component doesn't allocate frames, but only uses frames that come from outside (for example, from a decoder) and has requirements for them. In cases where flags are configured this way, we can avoid unnecessary surface copying, while the default behavior remains unchanged.
>
> For example, an AMF encoder with enabled PA wants frames to come from the decoder with the D3D11_BIND_SHADER_RESOURCE flag (in the case of DirectX 11).
>
> Some filters(especially shader based) may also have requirements for necessary flags.
>
>
> Sincerely, Ovchinnikov D.A.

Hi Dmitrii,

I do know the reason why you added this. A simple usage is that the user creates
the resource with the shared flags so he can interoperate d3d12va with amf, cuda, etc...

I noticed that the global flags are currently being set in the device context, but
the D3D12VA decoder actually uses the frame context to allocate frames. We should
set these flags in d3d12va_frames_init instead of referencing device->flags directly
in d3d12va_pool_alloc. This will align our setup with how the decoder expects context
flags to be configured.

And, it seemed that the options uav_resource=1,render_target_resource=1,shared_resource=1
are too long. Is there a better option?

Thanks,
Jianhua
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/2] avutil/hwcontext_d3d12va: added resource Flags support to command line

[Dmitrii Ovchinnikov]

>> We should set these flags in d3d12va_frames_init
>> instead of referencing device->flags directly
Sorry, I didn't fully understand this part.

Is it actually possible to pass the required flags to
the frames context without using the device context?

Or perhaps you meant something like this:
1) Add a D3D12_HEAP_FLAGS heap_flags field to AVD3D12VAFramesContext
2) Set both heap_flags and flags inside d3d12va_frames_init,
   after validating the format,
while still passing the property through the device context?

>> it seemed that the options uav_resource=1,render_target_resource=1,
>> shared_resource=1 are too long.

Maybe we could shorten the word resource to res?
Something like this: uav_res shared_res render_targ_res(rt_res)

Sincerely, Ovchinnikov D.A.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".