From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.ffmpeg.org (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id 7CE784CE3D for ; Mon, 11 Aug 2025 10:55:54 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTP id 5B65368C4D1; Mon, 11 Aug 2025 13:55:51 +0300 (EEST) Received: from mail-ed1-f54.google.com (mail-ed1-f54.google.com [209.85.208.54]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTPS id 0F28268AD0A for ; Mon, 11 Aug 2025 13:55:44 +0300 (EEST) Received: by mail-ed1-f54.google.com with SMTP id 4fb4d7f45d1cf-615398dc162so6628838a12.3 for ; Mon, 11 Aug 2025 03:55:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1754909743; x=1755514543; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=4pw7DSNMuvbgB62H1VRVbu6IWGKHnZJTU4n2bsD01xQ=; b=nsQwLNjrf/hYiPHPPkpctppXZsegSAD1pWExyeH5RtBdK8V9vWxSenpzVSAbxoNBDA t26i0p5Qzge4gzE1aMD+vizFMJGgTUfTumDJUlcV7SRigRZYMh+DISFD0huYIeV8VHm7 PRu90yMbg/9tD1xoO5IwyX6a6MhveyPUjXhJ7Ph2GKdoEh7UllDqjTioMZIHwvtOgY8T Czl/wlbifY/q5fcWRw8TFpM0gTO7ORaC+DOc9R07H9g/+mNy088gXP5dlm20Z3Dh5NH+ 7yoS8bW2FFXjhEFuG6mQVw3y7Ab0bwy/MYTxaUw+zHUvtSOIajVnQm7BM9+gIKDHhT5A RaRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754909743; x=1755514543; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=4pw7DSNMuvbgB62H1VRVbu6IWGKHnZJTU4n2bsD01xQ=; b=ZcMrJ2TEqdzNQXrtVAm7DHbedZCnN3MEyzTW+E3KuKIrS+KD9y9jPqQr1Q8NcggLir wfieaFqK3THuAW7sWT3RDuyphmEBWKVW92eJv2oo+LmFJFfC+wKxI/NCmaGI0Tgk6dkf qyJmxbWXSkY+7zm0Yv1FkSqF0Fbce4X48Dt8ldywH66r7reQM67KYPZIgeL8cIcLJ1rY 8/mS/UK5NgfepbikbT2fCZN7KEafv7G/ePk9g1TTlUZh8slLU6uJUbHnPVc/Kz6zKtcp bE4BSjeU+tu0a1GuCRWs2kw6C/hW4YPbSbRIc483lGGR+xExxvkP4Ap8SPxG0R5HDS3N HgoQ== X-Gm-Message-State: AOJu0YxAb8dyGyUixvLq9vHFtxeqJx3B7YS5q/LdvH9upajDX3neefQS 6440OJnWS0/4KZO+gDW6X3OgKjdkWC12VV8pVUKYYl/ZVze/XiT7IDgyYEMim603zo8= X-Gm-Gg: ASbGnctzEeXfPFKSL8vXdO+Uq1EvVqYN90fn2pLqd/hA3fh9mscUwNLRnax+fR+8LQs TM0ExUpfgHfn2b6sWk9YOCy7WV6dYqUKYd2zcHgMXaKk1ZzHQHiHUA3nQPwU4PD+rltF1RJLxL9 HvSzQsh9Kn8sGaVMJ/kgqXw0WUXJjhybDSsqTbiVrRKvKHgovbhPaKgGbUBuRR/byTFGYtY1+kj fjBVuLLQC8KCPh183/jbj8kIzFaWtKsPYjn8MeFihNDrbaxYWI1vyqgM+rzfpAe5V5Sv8o2aVSx Al6qgZB5S3SKpWfAmCNFvi7t5n0RNd7FV74zWjoBJM+wKRdorN9ccx/sW3k4UplKZuR7SVUKy++ H6QeOUIsdEABdbPLiE3NsTVkjOnYfob2sQxgDRpohArgA1dZPhjjcXnYeoqyo60g9v8RR5iDcXR 8IbYaMa+l0O7Y= X-Google-Smtp-Source: AGHT+IEMD40UD3e4bCQuuo5NonCfOowJEL2u9YY9wwcc7Lk3yfsPQ0NX981nhxHxtCWEEVtfoSLaRw== X-Received: by 2002:a05:6402:2350:b0:618:196b:1f8b with SMTP id 4fb4d7f45d1cf-618196b2220mr4609727a12.23.1754909742431; Mon, 11 Aug 2025 03:55:42 -0700 (PDT) Received: from localhost.localdomain (77-46-217-237.dynamic.isp.telekom.rs. [77.46.217.237]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-618200a0159sm2585349a12.6.2025.08.11.03.55.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Aug 2025 03:55:42 -0700 (PDT) From: Araz Iusubov X-Google-Original-From: Araz Iusubov To: ffmpeg-devel@ffmpeg.org Date: Mon, 11 Aug 2025 12:55:30 +0200 Message-ID: <20250811105530.617-1-Primeadvice@gmail.com> X-Mailer: git-send-email 2.49.0.windows.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH, v7] avcodec/d3d12va_encode: texture array support for HEVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Araz Iusubov Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: This patch adds support for the texture array feature used by AMD boards in the D3D12 HEVC encoder. In texture array mode, a single texture array is shared for all reference and reconstructed pictures using different subresources. The implementation ensures compatibility and has been successfully tested on AMD, Intel, and NVIDIA GPUs. v2 updates: 1. The reference to MaxL1ReferencesForB for the H.264 codec was updated to use the corresponding H.264 field instead of the HEVC one. 2. Max_subresource_array_size calculation was adjusted by removing the D3D12VA_VIDEO_ENC_ASYNC_DEPTH offset. v3 updates: 1. Fixed a type mismatch by explicitly casting AVD3D12VAFrame* to (uint8_t*) when assigning to data[0]. 2. Adjusted logging format specifier for HRESULT to use `%lx`. v4 updates: 1. Moved texture array management to hwcontext_d3d12va for proper abstraction. 2. Added `texture_array` and `texture_array_size` fields to AVD3D12VAFramesContext. 3. Implemented shared texture array allocation during `av_hwframe_ctx_init`. 4. Frames now receive unique subresource indices via `d3d12va_pool_alloc_texture_array`. 5. Removed `d3d12va_create_texture_array`, allocation is now handled entirely within hwcontext. 6. Encoder now uses subresource indices provided by hwcontext instead of managing them manually. v5 updates: No changes, resubmitted as v4 was missed by patchwork. v6 updates: 1. Minor cosmetic fixes according to review of v5 2. Bumped lavu version to 60.5.100 and updated APIchanges v7 updates: 1. Add enum AVD3D12VAFrameFlags to define the behaviours of frame allocation. 2. Renanme AVD3D12VAFramesContext.flags to AVD3D12VAFramesContext.resource_flags. 3. Add flags to AVD3D12VAFramesContext. 4. Add flags to AVD3D12VAFrame. 5. Remove AVD3D12VAFramesContext.texture_size, instead using AVHWFramesContext.initiali_pool_size. 6. Add "Static surface pool size exceeded." check. 7. Initialize the AVD3D12VAFrame fence value for synchronization. 8. Bumped lavu version to 60.10.100 and updated APIchanges. --- doc/APIchanges | 8 ++ libavcodec/d3d12va_encode.c | 175 +++++++++++++++++++++++-------- libavcodec/d3d12va_encode.h | 12 +++ libavcodec/d3d12va_encode_hevc.c | 5 +- libavutil/hwcontext_d3d12va.c | 93 +++++++++++++++- libavutil/hwcontext_d3d12va.h | 38 ++++++- libavutil/version.h | 2 +- 7 files changed, 278 insertions(+), 55 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index 1ed8337d04..9c37f7f0cd 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -2,6 +2,14 @@ The last version increases of all libraries were on 2025-03-28 API changes, most recent first: +2025-07-29 - xxxxxxxxxx - lavu 60.10.100 - hwcontext_d3d12va.h + Add support for texture array mode AVD3D12VAFrame.subresource_index, + AVD3D12VAFramesContext.texture_array + Add enum AVD3D12VAFrameFlags to define the behaviours of frame allocation. + Renanme AVD3D12VAFramesContext.flags to AVD3D12VAFramesContext.resource_flags. + Add flags to AVD3D12VAFramesContext + Add flags to AVD3D12VAFrame + 2025-07-29 - 1c85a3832af - lavc 62.10.100 - smpte_436m.h Add a new public header smpte_436m.h with API for manipulating AV_CODEC_ID_SMPTE_436M_ANC data. diff --git a/libavcodec/d3d12va_encode.c b/libavcodec/d3d12va_encode.c index 880002ce55..31b0c6b050 100644 --- a/libavcodec/d3d12va_encode.c +++ b/libavcodec/d3d12va_encode.c @@ -191,7 +191,8 @@ static int d3d12va_encode_issue(AVCodecContext *avctx, FFHWBaseEncodeContext *base_ctx = avctx->priv_data; D3D12VAEncodeContext *ctx = avctx->priv_data; D3D12VAEncodePicture *pic = base_pic->priv; - AVD3D12VAFramesContext *frames_hwctx = base_ctx->input_frames->hwctx; + AVD3D12VAFramesContext *frames_hwctx_input = base_ctx->input_frames->hwctx; + AVD3D12VAFramesContext *frames_hwctx_recon = base_ctx->recon_frames->hwctx; int err, i, j; HRESULT hr; char data[MAX_PARAM_BUFFER_SIZE]; @@ -221,7 +222,7 @@ static int d3d12va_encode_issue(AVCodecContext *avctx, D3D12_VIDEO_ENCODER_RESOLVE_METADATA_INPUT_ARGUMENTS input_metadata = { .EncoderCodec = ctx->codec->d3d12_codec, .EncoderProfile = ctx->profile->d3d12_profile, - .EncoderInputFormat = frames_hwctx->format, + .EncoderInputFormat = frames_hwctx_input->format, .EncodedPictureEffectiveResolution = ctx->resolution, }; @@ -268,6 +269,8 @@ static int d3d12va_encode_issue(AVCodecContext *avctx, av_log(avctx, AV_LOG_DEBUG, "Recon surface is %p.\n", pic->recon_surface->texture); + pic->subresource_index = ctx->is_texture_array ? pic->recon_surface->subresource_index : 0; + pic->output_buffer_ref = av_buffer_pool_get(ctx->output_buffer_pool); if (!pic->output_buffer_ref) { err = AVERROR(ENOMEM); @@ -324,11 +327,26 @@ static int d3d12va_encode_issue(AVCodecContext *avctx, goto fail; } + if (ctx->is_texture_array) { + d3d12_refs.pSubresources = av_calloc(d3d12_refs.NumTexture2Ds, + sizeof(*d3d12_refs.pSubresources)); + if (!d3d12_refs.pSubresources) { + err = AVERROR(ENOMEM); + goto fail; + } + } + i = 0; - for (j = 0; j < base_pic->nb_refs[0]; j++) - d3d12_refs.ppTexture2Ds[i++] = ((D3D12VAEncodePicture *)base_pic->refs[0][j]->priv)->recon_surface->texture; - for (j = 0; j < base_pic->nb_refs[1]; j++) - d3d12_refs.ppTexture2Ds[i++] = ((D3D12VAEncodePicture *)base_pic->refs[1][j]->priv)->recon_surface->texture; + for (j = 0; j < base_pic->nb_refs[0]; j++) { + d3d12_refs.ppTexture2Ds[i] = ((D3D12VAEncodePicture *)base_pic->refs[0][j]->priv)->recon_surface->texture; + d3d12_refs.pSubresources[i] = ctx->is_texture_array ? ((D3D12VAEncodePicture *)base_pic->refs[0][j]->priv)->subresource_index : 0; + i++; + } + for (j = 0; j < base_pic->nb_refs[1]; j++) { + d3d12_refs.ppTexture2Ds[i] = ((D3D12VAEncodePicture *)base_pic->refs[1][j]->priv)->recon_surface->texture; + d3d12_refs.pSubresources[i] = ctx->is_texture_array ? ((D3D12VAEncodePicture *)base_pic->refs[1][j]->priv)->subresource_index : 0; + i++; + } } input_args.PictureControlDesc.IntraRefreshFrameIndex = 0; @@ -342,7 +360,7 @@ static int d3d12va_encode_issue(AVCodecContext *avctx, output_args.Bitstream.pBuffer = pic->output_buffer; output_args.Bitstream.FrameStartOffset = pic->aligned_header_size; output_args.ReconstructedPicture.pReconstructedPicture = pic->recon_surface->texture; - output_args.ReconstructedPicture.ReconstructedPictureSubresource = 0; + output_args.ReconstructedPicture.ReconstructedPictureSubresource = ctx->is_texture_array ? pic->subresource_index : 0; output_args.EncoderOutputMetadata.pBuffer = pic->encoded_metadata; output_args.EncoderOutputMetadata.Offset = 0; @@ -368,52 +386,92 @@ static int d3d12va_encode_issue(AVCodecContext *avctx, goto fail; } -#define TRANSITION_BARRIER(res, before, after) \ +#define TRANSITION_BARRIER(res, subres, before, after) \ (D3D12_RESOURCE_BARRIER) { \ .Type = D3D12_RESOURCE_BARRIER_TYPE_TRANSITION, \ .Flags = D3D12_RESOURCE_BARRIER_FLAG_NONE, \ .Transition = { \ .pResource = res, \ - .Subresource = D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES, \ + .Subresource = subres, \ .StateBefore = before, \ .StateAfter = after, \ }, \ } barriers[0] = TRANSITION_BARRIER(pic->input_surface->texture, + D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES, D3D12_RESOURCE_STATE_COMMON, D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ); barriers[1] = TRANSITION_BARRIER(pic->output_buffer, + D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES, D3D12_RESOURCE_STATE_COMMON, D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE); - barriers[2] = TRANSITION_BARRIER(pic->recon_surface->texture, - D3D12_RESOURCE_STATE_COMMON, - D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE); - barriers[3] = TRANSITION_BARRIER(pic->encoded_metadata, + barriers[2] = TRANSITION_BARRIER(pic->encoded_metadata, + D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES, D3D12_RESOURCE_STATE_COMMON, D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE); - barriers[4] = TRANSITION_BARRIER(pic->resolved_metadata, + barriers[3] = TRANSITION_BARRIER(pic->resolved_metadata, + D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES, D3D12_RESOURCE_STATE_COMMON, D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE); - ID3D12VideoEncodeCommandList2_ResourceBarrier(cmd_list, 5, barriers); + ID3D12VideoEncodeCommandList2_ResourceBarrier(cmd_list, 4, barriers); - if (d3d12_refs.NumTexture2Ds) { - D3D12_RESOURCE_BARRIER refs_barriers[3]; - - for (i = 0; i < d3d12_refs.NumTexture2Ds; i++) - refs_barriers[i] = TRANSITION_BARRIER(d3d12_refs.ppTexture2Ds[i], - D3D12_RESOURCE_STATE_COMMON, - D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ); - - ID3D12VideoEncodeCommandList2_ResourceBarrier(cmd_list, d3d12_refs.NumTexture2Ds, - refs_barriers); + int barriers_ref_index = 0; + D3D12_RESOURCE_BARRIER *barriers_ref = NULL; + if (ctx->is_texture_array) { + barriers_ref = av_calloc(base_ctx->recon_frames->initial_pool_size * ctx->plane_count, + sizeof(D3D12_RESOURCE_BARRIER)); + } else { + barriers_ref = av_calloc(MAX_DPB_SIZE, sizeof(D3D12_RESOURCE_BARRIER)); + } + + if (ctx->is_texture_array) { + D3D12_RESOURCE_DESC references_tex_array_desc = { 0 }; + pic->recon_surface->texture->lpVtbl->GetDesc(pic->recon_surface->texture, &references_tex_array_desc); + + for (uint32_t reference_subresource = 0; reference_subresource < references_tex_array_desc.DepthOrArraySize; + reference_subresource++) { + + uint32_t array_size = references_tex_array_desc.DepthOrArraySize; + uint32_t mip_slice = reference_subresource % references_tex_array_desc.MipLevels; + uint32_t array_slice = (reference_subresource / references_tex_array_desc.MipLevels) % array_size; + + for (uint32_t plane_slice = 0; plane_slice < ctx->plane_count; plane_slice++) { + uint32_t outputSubresource = mip_slice + array_slice * references_tex_array_desc.MipLevels + + plane_slice * references_tex_array_desc.MipLevels * array_size; + if (reference_subresource == pic->subresource_index) { + barriers_ref[barriers_ref_index++] = TRANSITION_BARRIER(pic->recon_surface->texture, outputSubresource, + D3D12_RESOURCE_STATE_COMMON, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE); + } else { + barriers_ref[barriers_ref_index++] = TRANSITION_BARRIER(pic->recon_surface->texture, outputSubresource, + D3D12_RESOURCE_STATE_COMMON, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ); + } + } + } + } else { + barriers_ref[barriers_ref_index++] = TRANSITION_BARRIER(pic->recon_surface->texture, + D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES, + D3D12_RESOURCE_STATE_COMMON, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE); + + if (d3d12_refs.NumTexture2Ds) { + for (i = 0; i < d3d12_refs.NumTexture2Ds; i++) + barriers_ref[barriers_ref_index++] = TRANSITION_BARRIER(d3d12_refs.ppTexture2Ds[i], + D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES, + D3D12_RESOURCE_STATE_COMMON, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ); + } } + ID3D12VideoEncodeCommandList2_ResourceBarrier(cmd_list, barriers_ref_index, barriers_ref); ID3D12VideoEncodeCommandList2_EncodeFrame(cmd_list, ctx->encoder, ctx->encoder_heap, &input_args, &output_args); barriers[3] = TRANSITION_BARRIER(pic->encoded_metadata, + D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES, D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE, D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ); @@ -421,35 +479,32 @@ static int d3d12va_encode_issue(AVCodecContext *avctx, ID3D12VideoEncodeCommandList2_ResolveEncoderOutputMetadata(cmd_list, &input_metadata, &output_metadata); - if (d3d12_refs.NumTexture2Ds) { - D3D12_RESOURCE_BARRIER refs_barriers[3]; - - for (i = 0; i < d3d12_refs.NumTexture2Ds; i++) - refs_barriers[i] = TRANSITION_BARRIER(d3d12_refs.ppTexture2Ds[i], - D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ, - D3D12_RESOURCE_STATE_COMMON); + if (barriers_ref_index > 0) { + for (i = 0; i < barriers_ref_index; i++) + FFSWAP(D3D12_RESOURCE_STATES, barriers_ref[i].Transition.StateBefore, barriers_ref[i].Transition.StateAfter); - ID3D12VideoEncodeCommandList2_ResourceBarrier(cmd_list, d3d12_refs.NumTexture2Ds, - refs_barriers); + ID3D12VideoEncodeCommandList2_ResourceBarrier(cmd_list, barriers_ref_index, + barriers_ref); } barriers[0] = TRANSITION_BARRIER(pic->input_surface->texture, + D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES, D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ, D3D12_RESOURCE_STATE_COMMON); barriers[1] = TRANSITION_BARRIER(pic->output_buffer, + D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES, D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE, D3D12_RESOURCE_STATE_COMMON); - barriers[2] = TRANSITION_BARRIER(pic->recon_surface->texture, - D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE, - D3D12_RESOURCE_STATE_COMMON); - barriers[3] = TRANSITION_BARRIER(pic->encoded_metadata, + barriers[2] = TRANSITION_BARRIER(pic->encoded_metadata, + D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES, D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ, D3D12_RESOURCE_STATE_COMMON); - barriers[4] = TRANSITION_BARRIER(pic->resolved_metadata, + barriers[3] = TRANSITION_BARRIER(pic->resolved_metadata, + D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES, D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE, D3D12_RESOURCE_STATE_COMMON); - ID3D12VideoEncodeCommandList2_ResourceBarrier(cmd_list, 5, barriers); + ID3D12VideoEncodeCommandList2_ResourceBarrier(cmd_list, 4, barriers); hr = ID3D12VideoEncodeCommandList2_Close(cmd_list); if (FAILED(hr)) { @@ -488,6 +543,14 @@ static int d3d12va_encode_issue(AVCodecContext *avctx, if (d3d12_refs.ppTexture2Ds) av_freep(&d3d12_refs.ppTexture2Ds); + if (ctx->is_texture_array) { + if (d3d12_refs.pSubresources) + av_freep(&d3d12_refs.pSubresources); + } + + if (barriers_ref) + av_freep(&barriers_ref); + return 0; fail: @@ -497,6 +560,14 @@ fail: if (d3d12_refs.ppTexture2Ds) av_freep(&d3d12_refs.ppTexture2Ds); + if (ctx->is_texture_array) { + if (d3d12_refs.pSubresources) + av_freep(&d3d12_refs.pSubresources); + } + + if (barriers_ref) + av_freep(&barriers_ref); + if (ctx->codec->free_picture_params) ctx->codec->free_picture_params(pic); @@ -1340,6 +1411,7 @@ fail: static int d3d12va_encode_create_recon_frames(AVCodecContext *avctx) { FFHWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; AVD3D12VAFramesContext *hwctx; enum AVPixelFormat recon_format; int err; @@ -1360,9 +1432,12 @@ static int d3d12va_encode_create_recon_frames(AVCodecContext *avctx) base_ctx->recon_frames->width = base_ctx->surface_width; base_ctx->recon_frames->height = base_ctx->surface_height; - hwctx->flags = D3D12_RESOURCE_FLAG_VIDEO_ENCODE_REFERENCE_ONLY | + hwctx->resource_flags = D3D12_RESOURCE_FLAG_VIDEO_ENCODE_REFERENCE_ONLY | D3D12_RESOURCE_FLAG_DENY_SHADER_RESOURCE; + base_ctx->recon_frames->initial_pool_size = ctx->is_texture_array ? MAX_DPB_SIZE + 1 : 0; + hwctx->flags |= AV_D3D12VA_FRAME_FLAG_TEXTURE_ARRAY; + err = av_hwframe_ctx_init(base_ctx->recon_frames_ref); if (err < 0) { av_log(avctx, AV_LOG_ERROR, "Failed to initialise reconstructed " @@ -1395,6 +1470,7 @@ int ff_d3d12va_encode_init(AVCodecContext *avctx) FFHWBaseEncodeContext *base_ctx = avctx->priv_data; D3D12VAEncodeContext *ctx = avctx->priv_data; D3D12_FEATURE_DATA_VIDEO_FEATURE_AREA_SUPPORT support = { 0 }; + D3D12_FEATURE_DATA_FORMAT_INFO format_info = {0}; int err; HRESULT hr; @@ -1430,6 +1506,15 @@ int ff_d3d12va_encode_init(AVCodecContext *avctx) goto fail; } + format_info.Format = ((AVD3D12VAFramesContext*)base_ctx->input_frames->hwctx)->format; + if (FAILED(ID3D12VideoDevice_CheckFeatureSupport(ctx->hwctx->device, D3D12_FEATURE_FORMAT_INFO, + &format_info, sizeof(format_info)))) { + av_log(avctx, AV_LOG_ERROR, "Failed to query format plane count: 0x%x\n", hr); + err = AVERROR_EXTERNAL; + goto fail; + } + ctx->plane_count = format_info.PlaneCount; + err = d3d12va_encode_set_profile(avctx); if (err < 0) goto fail; @@ -1457,10 +1542,6 @@ int ff_d3d12va_encode_init(AVCodecContext *avctx) if (err < 0) goto fail; - err = d3d12va_encode_create_recon_frames(avctx); - if (err < 0) - goto fail; - err = d3d12va_encode_prepare_output_buffers(avctx); if (err < 0) goto fail; @@ -1486,6 +1567,10 @@ int ff_d3d12va_encode_init(AVCodecContext *avctx) goto fail; } + err = d3d12va_encode_create_recon_frames(avctx); + if (err < 0) + goto fail; + base_ctx->output_delay = base_ctx->b_per_p; base_ctx->decode_delay = base_ctx->max_b_depth; diff --git a/libavcodec/d3d12va_encode.h b/libavcodec/d3d12va_encode.h index 3b0b8153d5..c8e64ddffd 100644 --- a/libavcodec/d3d12va_encode.h +++ b/libavcodec/d3d12va_encode.h @@ -52,6 +52,8 @@ typedef struct D3D12VAEncodePicture { ID3D12Resource *encoded_metadata; ID3D12Resource *resolved_metadata; + int subresource_index; + D3D12_VIDEO_ENCODER_PICTURE_CONTROL_CODEC_DATA pic_ctl; int fence_value; @@ -189,6 +191,16 @@ typedef struct D3D12VAEncodeContext { */ AVBufferPool *output_buffer_pool; + /** + * Flag indicates if the HW is texture array mode. + */ + int is_texture_array; + + /** + * The number of planes in the input DXGI FORMAT . + */ + int plane_count; + /** * D3D12 video encoder. */ diff --git a/libavcodec/d3d12va_encode_hevc.c b/libavcodec/d3d12va_encode_hevc.c index ce5d1bf110..65ad487550 100644 --- a/libavcodec/d3d12va_encode_hevc.c +++ b/libavcodec/d3d12va_encode_hevc.c @@ -279,9 +279,8 @@ static int d3d12va_encode_hevc_init_sequence_params(AVCodecContext *avctx) } if (support.SupportFlags & D3D12_VIDEO_ENCODER_SUPPORT_FLAG_RECONSTRUCTED_FRAMES_REQUIRE_TEXTURE_ARRAYS) { - av_log(avctx, AV_LOG_ERROR, "D3D12 video encode on this device requires texture array support, " - "but it's not implemented.\n"); - return AVERROR_PATCHWELCOME; + ctx->is_texture_array = 1; + av_log(avctx, AV_LOG_DEBUG, "D3D12 video encode on this device uses texture array mode.\n"); } desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); diff --git a/libavutil/hwcontext_d3d12va.c b/libavutil/hwcontext_d3d12va.c index 6507cf69c1..179c13fa1b 100644 --- a/libavutil/hwcontext_d3d12va.c +++ b/libavutil/hwcontext_d3d12va.c @@ -49,6 +49,7 @@ typedef struct D3D12VAFramesContext { ID3D12GraphicsCommandList *command_list; AVD3D12VASyncContext sync_ctx; UINT luma_component_size; + int nb_surfaces_used; } D3D12VAFramesContext; typedef struct D3D12VADevicePriv { @@ -174,7 +175,8 @@ fail: static void d3d12va_frames_uninit(AVHWFramesContext *ctx) { - D3D12VAFramesContext *s = ctx->hwctx; + D3D12VAFramesContext *s = ctx->hwctx; + AVD3D12VAFramesContext *hwctx = ctx->hwctx; D3D12_OBJECT_RELEASE(s->sync_ctx.fence); if (s->sync_ctx.event) @@ -185,6 +187,9 @@ static void d3d12va_frames_uninit(AVHWFramesContext *ctx) D3D12_OBJECT_RELEASE(s->command_allocator); D3D12_OBJECT_RELEASE(s->command_list); D3D12_OBJECT_RELEASE(s->command_queue); + + if (hwctx->texture_array) + D3D12_OBJECT_RELEASE(hwctx->texture_array); } static int d3d12va_frames_get_constraints(AVHWDeviceContext *ctx, const void *hwconfig, AVHWFramesConstraints *constraints) @@ -220,7 +225,9 @@ static void free_texture(void *opaque, uint8_t *data) { AVD3D12VAFrame *frame = (AVD3D12VAFrame *)data; - D3D12_OBJECT_RELEASE(frame->texture); + if (!(frame->flags & AV_D3D12VA_FRAME_FLAG_TEXTURE_ARRAY)) { + D3D12_OBJECT_RELEASE(frame->texture); + } D3D12_OBJECT_RELEASE(frame->sync_ctx.fence); if (frame->sync_ctx.event) CloseHandle(frame->sync_ctx.event); @@ -228,6 +235,42 @@ static void free_texture(void *opaque, uint8_t *data) av_freep(&data); } +static AVBufferRef *d3d12va_pool_alloc_texture_array(AVHWFramesContext *ctx) +{ + AVD3D12VAFrame *frame = av_mallocz(sizeof(*frame)); + D3D12VAFramesContext *s = ctx->hwctx; + AVD3D12VAFramesContext *hwctx = ctx->hwctx; + AVD3D12VADeviceContext *device_hwctx = ctx->device_ctx->hwctx; + AVBufferRef *buf; + + frame->flags |= AV_D3D12VA_FRAME_FLAG_TEXTURE_ARRAY; + + DX_CHECK(ID3D12Device_CreateFence(device_hwctx->device, 0, D3D12_FENCE_FLAG_NONE, + &IID_ID3D12Fence, (void **)&frame->sync_ctx.fence)); + frame->sync_ctx.event = CreateEvent(NULL, FALSE, FALSE, NULL); + if (!frame->sync_ctx.event) + goto fail; + + if (s->nb_surfaces_used >= ctx->initial_pool_size) { + av_log(ctx, AV_LOG_ERROR, "Static surface pool size exceeded.\n"); + goto fail; + } + + frame->texture = hwctx->texture_array; + frame->subresource_index = s->nb_surfaces_used; + + buf = av_buffer_create((uint8_t *)frame, sizeof(*frame), free_texture, NULL, 0); + + if (!buf) + goto fail; + + s->nb_surfaces_used++; + return buf; +fail: + free_texture(NULL, (uint8_t *)frame); + return NULL; +} + static AVBufferRef *d3d12va_pool_alloc(void *opaque, size_t size) { AVHWFramesContext *ctx = (AVHWFramesContext *)opaque; @@ -236,6 +279,10 @@ static AVBufferRef *d3d12va_pool_alloc(void *opaque, size_t size) AVBufferRef *buf; AVD3D12VAFrame *frame; + + if (ctx->initial_pool_size > 0) + return d3d12va_pool_alloc_texture_array(ctx); + D3D12_HEAP_PROPERTIES props = { .Type = D3D12_HEAP_TYPE_DEFAULT }; D3D12_RESOURCE_DESC desc = { .Dimension = D3D12_RESOURCE_DIMENSION_TEXTURE2D, @@ -247,7 +294,7 @@ static AVBufferRef *d3d12va_pool_alloc(void *opaque, size_t size) .Format = hwctx->format, .SampleDesc = {.Count = 1, .Quality = 0 }, .Layout = D3D12_TEXTURE_LAYOUT_UNKNOWN, - .Flags = hwctx->flags, + .Flags = hwctx->resource_flags, }; frame = av_mallocz(sizeof(AVD3D12VAFrame)); @@ -256,7 +303,7 @@ static AVBufferRef *d3d12va_pool_alloc(void *opaque, size_t size) if (FAILED(ID3D12Device_CreateCommittedResource(device_hwctx->device, &props, D3D12_HEAP_FLAG_NONE, &desc, D3D12_RESOURCE_STATE_COMMON, NULL, &IID_ID3D12Resource, (void **)&frame->texture))) { - av_log(ctx, AV_LOG_ERROR, "Could not create the texture\n"); + av_log(ctx, AV_LOG_ERROR, "Could not create the texture array\n"); goto fail; } @@ -278,9 +325,39 @@ fail: return NULL; } +static int d3d12va_texture_array_init(AVHWFramesContext *ctx) +{ + AVD3D12VAFramesContext *hwctx = ctx->hwctx; + AVD3D12VADeviceContext *device_hwctx = ctx->device_ctx->hwctx; + + D3D12_HEAP_PROPERTIES props = { .Type = D3D12_HEAP_TYPE_DEFAULT }; + + D3D12_RESOURCE_DESC desc = { + .Dimension = D3D12_RESOURCE_DIMENSION_TEXTURE2D, + .Alignment = 0, + .Width = ctx->width, + .Height = ctx->height, + .DepthOrArraySize = ctx->initial_pool_size, + .MipLevels = 1, + .Format = hwctx->format, + .SampleDesc = {.Count = 1, .Quality = 0 }, + .Layout = D3D12_TEXTURE_LAYOUT_UNKNOWN, + .Flags = hwctx->resource_flags, + }; + + if (FAILED(ID3D12Device_CreateCommittedResource(device_hwctx->device, &props, D3D12_HEAP_FLAG_NONE, &desc, + D3D12_RESOURCE_STATE_COMMON, NULL, &IID_ID3D12Resource, (void **)&hwctx->texture_array))) { + av_log(ctx, AV_LOG_ERROR, "Could not create the texture\n"); + return AVERROR(EINVAL); + } + return 0; +} + static int d3d12va_frames_init(AVHWFramesContext *ctx) { - AVD3D12VAFramesContext *hwctx = ctx->hwctx; + AVD3D12VAFramesContext *hwctx = ctx->hwctx; + AVD3D12VADeviceContext *device_hwctx = ctx->device_ctx->hwctx; + int i; for (i = 0; i < FF_ARRAY_ELEMS(supported_formats); i++) { @@ -298,6 +375,12 @@ static int d3d12va_frames_init(AVHWFramesContext *ctx) return AVERROR(EINVAL); } + if (ctx->initial_pool_size > 0) { + int err = d3d12va_texture_array_init(ctx); + if (err < 0) + return err; + } + ffhwframesctx(ctx)->pool_internal = av_buffer_pool_init2(sizeof(AVD3D12VAFrame), ctx, d3d12va_pool_alloc, NULL); diff --git a/libavutil/hwcontext_d3d12va.h b/libavutil/hwcontext_d3d12va.h index 212a6a6146..64c30c5857 100644 --- a/libavutil/hwcontext_d3d12va.h +++ b/libavutil/hwcontext_d3d12va.h @@ -99,6 +99,19 @@ typedef struct AVD3D12VASyncContext { uint64_t fence_value; } AVD3D12VASyncContext; +/** + * Define the behaviours of frame allocation. + */ +typedef enum AVD3D12VAFrameFlags { + AV_D3D12VA_FRAME_FLAG_NONE = 0, + + /** + * The DXGI_FORMAT doesn't provide formats like P210/P410... + * If set, the frame data will be allocated by texture array + */ + AV_D3D12VA_FRAME_FLAG_TEXTURE_ARRAY = (1 << 1), +} AVD3D12VAFrameFlags; + /** * @brief D3D12VA frame descriptor for pool allocation. * @@ -111,12 +124,22 @@ typedef struct AVD3D12VAFrame { */ ID3D12Resource *texture; + /** + * In texture array mode, the index of subresource + */ + int subresource_index; + /** * The sync context for the texture * * @see: https://learn.microsoft.com/en-us/windows/win32/medfound/direct3d-12-video-overview#directx-12-fences */ AVD3D12VASyncContext sync_ctx; + + /** + * Flags for the frame allocation behaviours. + */ + AVD3D12VAFrameFlags flags; } AVD3D12VAFrame; /** @@ -136,7 +159,20 @@ typedef struct AVD3D12VAFramesContext { * * @see https://learn.microsoft.com/en-us/windows/win32/api/d3d12/ne-d3d12-d3d12_resource_flags */ - D3D12_RESOURCE_FLAGS flags; + D3D12_RESOURCE_FLAGS resource_flags; + + /** + * In texture array mode, the D3D12 uses the same texture array (resource)for all + * pictures. + */ + ID3D12Resource *texture_array; + + /** + * A combination of AVD3D12VAFrameFlags. Unless AV_D3D12VA_FRAME_FLAG_NONE is set, + * autodetected flags will be OR'd based on the device and frame features during + * av_hwframe_ctx_init(). + */ + AVD3D12VAFrameFlags flags; } AVD3D12VAFramesContext; #endif /* AVUTIL_HWCONTEXT_D3D12VA_H */ diff --git a/libavutil/version.h b/libavutil/version.h index 5cde712c9f..27b9fe73d8 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 60 -#define LIBAVUTIL_VERSION_MINOR 9 +#define LIBAVUTIL_VERSION_MINOR 10 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ -- 2.45.2.windows.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".