* [FFmpeg-devel] [PATCH 1/3] avutil/hwcontext_d3d11va: add a format check for staging texture @ 2022-04-01 9:24 Tong Wu 2022-04-01 9:24 ` [FFmpeg-devel] [PATCH 2/3] avutil/hwcontext_d3d11va: fix the uninitialized texture bindflag Tong Wu 2022-04-01 9:24 ` [FFmpeg-devel] [PATCH 3/3] avutil/hwcontext_qsv: fix D3D11VA<->qsv hwmap errors Tong Wu 0 siblings, 2 replies; 4+ messages in thread From: Tong Wu @ 2022-04-01 9:24 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Tong Wu The texDesc.Format needs to be filled in with a corresponding format. I add a format check to initialize the format in case sometimes the ctx->internal->priv is not initialized, such as during the hwmap process. For example, for sample command line: ffmpeg.exe -hwaccel qsv -c:v h264_qsv -i input.264 -vf "hwmap=derive_device=d3d11va:extra_hw_frames=16,format=d3d11,hwdownload,format=nv12" -f rawvideo -y ./output.yuv When deriving from qsv, the ctx->internal->priv remains uninitialized, causing CreateTexture2D failure. Note: this fix only solves this particular issue and does not make the sample command line work immediately. See the other patches in the patchset which fix this qsv->d3d11va command line. Signed-off-by: Tong Wu <tong1.wu@intel.com> --- libavutil/hwcontext_d3d11va.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/libavutil/hwcontext_d3d11va.c b/libavutil/hwcontext_d3d11va.c index 8ab96bad25..f69fb3fc9c 100644 --- a/libavutil/hwcontext_d3d11va.c +++ b/libavutil/hwcontext_d3d11va.c @@ -344,6 +344,8 @@ static int d3d11va_create_staging_texture(AVHWFramesContext *ctx) AVD3D11VADeviceContext *device_hwctx = ctx->device_ctx->hwctx; D3D11VAFramesContext *s = ctx->internal->priv; HRESULT hr; + int i; + D3D11_TEXTURE2D_DESC texDesc = { .Width = ctx->width, .Height = ctx->height, @@ -355,6 +357,20 @@ static int d3d11va_create_staging_texture(AVHWFramesContext *ctx) .CPUAccessFlags = D3D11_CPU_ACCESS_READ | D3D11_CPU_ACCESS_WRITE, }; + if (!texDesc.Format) { + for (i = 0; i < FF_ARRAY_ELEMS(supported_formats); i++) { + if (ctx->sw_format == supported_formats[i].pix_fmt) { + texDesc.Format = supported_formats[i].d3d_format; + break; + } + } + if (i == FF_ARRAY_ELEMS(supported_formats)) { + av_log(ctx, AV_LOG_ERROR, "Unsupported pixel format: %s\n", + av_get_pix_fmt_name(ctx->sw_format)); + return AVERROR(EINVAL); + } + } + hr = ID3D11Device_CreateTexture2D(device_hwctx->device, &texDesc, NULL, &s->staging_texture); if (FAILED(hr)) { av_log(ctx, AV_LOG_ERROR, "Could not create the staging texture (%lx)\n", (long)hr); -- 2.35.1.windows.2 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 4+ messages in thread
* [FFmpeg-devel] [PATCH 2/3] avutil/hwcontext_d3d11va: fix the uninitialized texture bindflag 2022-04-01 9:24 [FFmpeg-devel] [PATCH 1/3] avutil/hwcontext_d3d11va: add a format check for staging texture Tong Wu @ 2022-04-01 9:24 ` Tong Wu 2022-04-01 9:24 ` [FFmpeg-devel] [PATCH 3/3] avutil/hwcontext_qsv: fix D3D11VA<->qsv hwmap errors Tong Wu 1 sibling, 0 replies; 4+ messages in thread From: Tong Wu @ 2022-04-01 9:24 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Tong Wu When uploading rawvideos using d3d11va hardware framecontext, the bindflag is not initialized and will cause creating texture failure. Now fix it, assign it the value of D3D11_BIND_RENDER_TARGET. Signed-off-by: Tong Wu <tong1.wu@intel.com> --- libavutil/hwcontext_d3d11va.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/libavutil/hwcontext_d3d11va.c b/libavutil/hwcontext_d3d11va.c index f69fb3fc9c..0ec0e07d3a 100644 --- a/libavutil/hwcontext_d3d11va.c +++ b/libavutil/hwcontext_d3d11va.c @@ -254,6 +254,11 @@ static int d3d11va_frames_init(AVHWFramesContext *ctx) return AVERROR(EINVAL); } + if (!hwctx->BindFlags) { + av_log(ctx, AV_LOG_DEBUG, "Add render target bindflag for texture\n"); + hwctx->BindFlags = D3D11_BIND_RENDER_TARGET; + } + texDesc = (D3D11_TEXTURE2D_DESC){ .Width = ctx->width, .Height = ctx->height, -- 2.35.1.windows.2 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 4+ messages in thread
* [FFmpeg-devel] [PATCH 3/3] avutil/hwcontext_qsv: fix D3D11VA<->qsv hwmap errors 2022-04-01 9:24 [FFmpeg-devel] [PATCH 1/3] avutil/hwcontext_d3d11va: add a format check for staging texture Tong Wu 2022-04-01 9:24 ` [FFmpeg-devel] [PATCH 2/3] avutil/hwcontext_d3d11va: fix the uninitialized texture bindflag Tong Wu @ 2022-04-01 9:24 ` Tong Wu 2022-04-25 9:37 ` Xiang, Haihao 1 sibling, 1 reply; 4+ messages in thread From: Tong Wu @ 2022-04-01 9:24 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Tong Wu For hwmap between qsv and d3d11va, The mfxHDLPair information should be put into texture_infos when deriving from qsv context. Moreover, when uploading from rawvideo, the ways that the textures are created are different, bindflag assertions are needed to make sure the right textures are derived during the process. Now after this fix, d3d_dec->qsv_vpp->qsv_enc, d3d_dec->qsv_vpp->qsv_download->yuv, yuv->d3d_upload->qsv_vpp->qsv->download->yuv, qsv_dec->qsv_vpp->d3d_download->yuv can all work properly. For d3d_dec->qsv_vpp->qsv_enc, one sample command line: ffmpeg.exe -hwaccel qsv -c:v h264_qsv -i input.264 -vf "hwmap=derive_device=d3d11va,format=d3d11,hwmap=derive_device=qsv,format=qsv" -c:v h264_qsv -y ./output.264 Signed-off-by: Tong Wu <tong1.wu@intel.com> --- libavutil/hwcontext_qsv.c | 48 ++++++++++++++++++++++++++++++++------- 1 file changed, 40 insertions(+), 8 deletions(-) diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index 95f8071abe..e6a7ac3ef0 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -806,12 +806,23 @@ static int qsv_frames_derive_from(AVHWFramesContext *dst_ctx, #if CONFIG_D3D11VA case AV_HWDEVICE_TYPE_D3D11VA: { + dst_ctx->initial_pool_size = src_ctx->initial_pool_size; AVD3D11VAFramesContext *dst_hwctx = dst_ctx->hwctx; - mfxHDLPair *pair = (mfxHDLPair*)src_hwctx->surfaces[i].Data.MemId; - dst_hwctx->texture = (ID3D11Texture2D*)pair->first; + dst_hwctx->texture_infos = av_calloc(src_hwctx->nb_surfaces, + sizeof(*dst_hwctx->texture_infos)); if (src_hwctx->frame_type & MFX_MEMTYPE_SHARED_RESOURCE) dst_hwctx->MiscFlags = D3D11_RESOURCE_MISC_SHARED; dst_hwctx->BindFlags = qsv_get_d3d11va_bind_flags(src_hwctx->frame_type); + for (i = 0; i < src_hwctx->nb_surfaces; i++) { + mfxHDLPair* pair = (mfxHDLPair*)src_hwctx->surfaces[i].Data.MemId; + dst_hwctx->texture_infos[i].texture = (ID3D11Texture2D*)pair->first; + if (dst_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { + dst_hwctx->texture_infos[i].index = 0; + } + else { + dst_hwctx->texture_infos[i].index = (intptr_t)pair->second; + } + } } break; #endif @@ -900,9 +911,16 @@ static int qsv_map_from(AVHWFramesContext *ctx, dst->height = src->height; if (child_frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) { +#if CONFIG_D3D11VA + AVD3D11VAFramesContext* child_frames_hwctx = child_frames_ctx->hwctx; mfxHDLPair *pair = (mfxHDLPair*)surf->Data.MemId; dst->data[0] = pair->first; - dst->data[1] = pair->second; + if (child_frames_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { + dst->data[1] = 0; + } else { + dst->data[1] = pair->second; + } +#endif } else { dst->data[3] = child_data; } @@ -930,9 +948,16 @@ static int qsv_map_from(AVHWFramesContext *ctx, dummy->height = src->height; if (child_frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) { +#if CONFIG_D3D11VA + AVD3D11VAFramesContext* child_frames_hwctx = child_frames_ctx->hwctx; mfxHDLPair *pair = (mfxHDLPair*)surf->Data.MemId; dummy->data[0] = pair->first; - dummy->data[1] = pair->second; + if (child_frames_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { + dst->data[1] = 0; + } else { + dst->data[1] = pair->second; + } +#endif } else { dummy->data[3] = child_data; } @@ -1287,6 +1312,10 @@ static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, return AVERROR(ENOSYS); } + s->child_frames_ref = av_buffer_ref(dst_ctx->internal->source_frames); + if (!s->child_frames_ref) { + return AVERROR(ENOMEM); + } dst_hwctx->surfaces = s->surfaces_internal; return 0; @@ -1314,10 +1343,13 @@ static int qsv_map_to(AVHWFramesContext *dst_ctx, case AV_PIX_FMT_D3D11: { mfxHDLPair *pair = (mfxHDLPair*)hwctx->surfaces[i].Data.MemId; - if (pair->first == src->data[0] - && pair->second == src->data[1]) { - index = i; - break; + if (pair->first == src->data[0]) { + if (hwctx->frame_type & MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET + && pair->second == src->data[1] + || hwctx->frame_type & MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET) { + index = i; + break; + } } } #endif -- 2.35.1.windows.2 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [FFmpeg-devel] [PATCH 3/3] avutil/hwcontext_qsv: fix D3D11VA<->qsv hwmap errors 2022-04-01 9:24 ` [FFmpeg-devel] [PATCH 3/3] avutil/hwcontext_qsv: fix D3D11VA<->qsv hwmap errors Tong Wu @ 2022-04-25 9:37 ` Xiang, Haihao 0 siblings, 0 replies; 4+ messages in thread From: Xiang, Haihao @ 2022-04-25 9:37 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Wu, Tong1 On Fri, 2022-04-01 at 17:24 +0800, Tong Wu wrote: > For hwmap between qsv and d3d11va, The mfxHDLPair information should be > put into texture_infos when deriving from qsv context. Moreover, when > uploading from rawvideo, the ways that the textures are created are > different, bindflag assertions are needed to make sure the right > textures are derived during the process. Now after this fix, > d3d_dec->qsv_vpp->qsv_enc, d3d_dec->qsv_vpp->qsv_download->yuv, > yuv->d3d_upload->qsv_vpp->qsv->download->yuv, > qsv_dec->qsv_vpp->d3d_download->yuv can all work properly. > > For d3d_dec->qsv_vpp->qsv_enc, one sample command line: > ffmpeg.exe -hwaccel qsv -c:v h264_qsv -i input.264 > -vf > "hwmap=derive_device=d3d11va,format=d3d11,hwmap=derive_device=qsv,format=qsv" > -c:v h264_qsv -y ./output.264 The default child_device_type is dxva2 for option --enable-libmfx, I don't think it makes sense to derive a QSV device based on dxva2 child device to a d3d11vadevice. But even if initializing qsv device with d3d11va child device, the command below still doesn't work $ ffmpeg.exe -y -hwaccel qsv -init_hw_device qsv=qsv:hw,child_device=0,child_device_type=d3d11va -c:v h264_qsv -i input.h264 -vf "hwmap=derive_device=d3d11va,format=d3d11,hwdownload,format=nv12" -f null - You may try https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=5304 Thanks Haihao > > Signed-off-by: Tong Wu <tong1.wu@intel.com> > --- > libavutil/hwcontext_qsv.c | 48 ++++++++++++++++++++++++++++++++------- > 1 file changed, 40 insertions(+), 8 deletions(-) > > diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c > index 95f8071abe..e6a7ac3ef0 100644 > --- a/libavutil/hwcontext_qsv.c > +++ b/libavutil/hwcontext_qsv.c > @@ -806,12 +806,23 @@ static int qsv_frames_derive_from(AVHWFramesContext > *dst_ctx, > #if CONFIG_D3D11VA > case AV_HWDEVICE_TYPE_D3D11VA: > { > + dst_ctx->initial_pool_size = src_ctx->initial_pool_size; > AVD3D11VAFramesContext *dst_hwctx = dst_ctx->hwctx; > - mfxHDLPair *pair = (mfxHDLPair*)src_hwctx- > >surfaces[i].Data.MemId; > - dst_hwctx->texture = (ID3D11Texture2D*)pair->first; > + dst_hwctx->texture_infos = av_calloc(src_hwctx->nb_surfaces, > + sizeof(*dst_hwctx- > >texture_infos)); > if (src_hwctx->frame_type & MFX_MEMTYPE_SHARED_RESOURCE) > dst_hwctx->MiscFlags = D3D11_RESOURCE_MISC_SHARED; > dst_hwctx->BindFlags = qsv_get_d3d11va_bind_flags(src_hwctx- > >frame_type); > + for (i = 0; i < src_hwctx->nb_surfaces; i++) { > + mfxHDLPair* pair = (mfxHDLPair*)src_hwctx- > >surfaces[i].Data.MemId; > + dst_hwctx->texture_infos[i].texture = (ID3D11Texture2D*)pair- > >first; > + if (dst_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { > + dst_hwctx->texture_infos[i].index = 0; > + } > + else { > + dst_hwctx->texture_infos[i].index = (intptr_t)pair- > >second; > + } > + } > } > break; > #endif > @@ -900,9 +911,16 @@ static int qsv_map_from(AVHWFramesContext *ctx, > dst->height = src->height; > > if (child_frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) { > +#if CONFIG_D3D11VA > + AVD3D11VAFramesContext* child_frames_hwctx = child_frames_ctx- > >hwctx; > mfxHDLPair *pair = (mfxHDLPair*)surf->Data.MemId; > dst->data[0] = pair->first; > - dst->data[1] = pair->second; > + if (child_frames_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { > + dst->data[1] = 0; > + } else { > + dst->data[1] = pair->second; > + } > +#endif > } else { > dst->data[3] = child_data; > } > @@ -930,9 +948,16 @@ static int qsv_map_from(AVHWFramesContext *ctx, > dummy->height = src->height; > > if (child_frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) { > +#if CONFIG_D3D11VA > + AVD3D11VAFramesContext* child_frames_hwctx = child_frames_ctx->hwctx; > mfxHDLPair *pair = (mfxHDLPair*)surf->Data.MemId; > dummy->data[0] = pair->first; > - dummy->data[1] = pair->second; > + if (child_frames_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { > + dst->data[1] = 0; > + } else { > + dst->data[1] = pair->second; > + } > +#endif > } else { > dummy->data[3] = child_data; > } > @@ -1287,6 +1312,10 @@ static int qsv_frames_derive_to(AVHWFramesContext > *dst_ctx, > return AVERROR(ENOSYS); > } > > + s->child_frames_ref = av_buffer_ref(dst_ctx->internal->source_frames); > + if (!s->child_frames_ref) { > + return AVERROR(ENOMEM); > + } > dst_hwctx->surfaces = s->surfaces_internal; > > return 0; > @@ -1314,10 +1343,13 @@ static int qsv_map_to(AVHWFramesContext *dst_ctx, > case AV_PIX_FMT_D3D11: > { > mfxHDLPair *pair = (mfxHDLPair*)hwctx->surfaces[i].Data.MemId; > - if (pair->first == src->data[0] > - && pair->second == src->data[1]) { > - index = i; > - break; > + if (pair->first == src->data[0]) { > + if (hwctx->frame_type & > MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET > + && pair->second == src->data[1] > + || hwctx->frame_type & > MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET) { > + index = i; > + break; > + } > } > } > #endif _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2022-04-25 9:37 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-04-01 9:24 [FFmpeg-devel] [PATCH 1/3] avutil/hwcontext_d3d11va: add a format check for staging texture Tong Wu 2022-04-01 9:24 ` [FFmpeg-devel] [PATCH 2/3] avutil/hwcontext_d3d11va: fix the uninitialized texture bindflag Tong Wu 2022-04-01 9:24 ` [FFmpeg-devel] [PATCH 3/3] avutil/hwcontext_qsv: fix D3D11VA<->qsv hwmap errors Tong Wu 2022-04-25 9:37 ` Xiang, Haihao
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel This inbox may be cloned and mirrored by anyone: git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \ ffmpegdev@gitmailbox.com public-inbox-index ffmpegdev Example config snippet for mirrors. AGPL code for this site: git clone https://public-inbox.org/public-inbox.git