From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id B2B404012D for ; Wed, 22 Jan 2025 19:34:11 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C93CD68B9C7; Wed, 22 Jan 2025 21:33:23 +0200 (EET) Received: from mail-ed1-f45.google.com (mail-ed1-f45.google.com [209.85.208.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CAE9C68B9A9 for ; Wed, 22 Jan 2025 21:33:16 +0200 (EET) Received: by mail-ed1-f45.google.com with SMTP id 4fb4d7f45d1cf-5d0d32cd31aso223764a12.0 for ; Wed, 22 Jan 2025 11:33:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1737574396; x=1738179196; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IzmkkucoRwhUezxd0nwPK/qu8wN8EvsGCqkdPPchT0A=; b=mACRnqjHngvJHoJ76kqSIhl7INJnL7xukuy6kqosbNhqNawagPeOAaAK3gKr4NE6vc tWNti5LAGYFumKrs9mtqaaM1AoJZt8M4DWFz4Z2VCm5fMpy5+Z/xRW/xs0m5PMTKqp+H qmymYrYjgX+AVdsE2VRWhPAZeLKdYYst680PrkzKilm7C4mw4P+Pa7R0PAaxGARPOmhc Dng5FFxpAohEuhYek7Zu260lTr/yjIHGuCd8F9XLVpDmKucsjRpSOQw0aX6M/9JsN3Yv 1VROBpcuqPUPgtrLrJWSxkaNFe5S6Hgq59j+RLrMyRzb5OVKzlKztVfpm4oi+2cu2cg6 W92A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737574396; x=1738179196; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IzmkkucoRwhUezxd0nwPK/qu8wN8EvsGCqkdPPchT0A=; b=MXQohWSSYZ6d3OUqDBmQRXw06BrGJSGveM70h7hcxOgIpO2+ttv4YG1Sva6iJQ5PvS y/paNf3aRU2w+pBT2qGFHZBF46J+D+8RAUmhIaGX09x0JEk9uDZQ4ZWeFybirhox1cEA BN5KUvpbocjzRRnYBT9Rcwa66FeUdjz57k91Rs4R2jOIyakyWq0OFe7B7lbjStyg9sXO 8K7KNrKY7Dub5WOekkQTBJv/ponJVFzmoYabHixgk+SlGk4lyv9amlpFlRXjOpv+YHO2 iSG+FaGyfWuASjHUuElVyOLI4KoR1hrGUiJMue3mKRVE1faLFlgS0WtZxaG6TndBS0B5 xoQw== X-Gm-Message-State: AOJu0YyL2AdW/S+vg57OYU8B8ABs8SbNTwvQOakswAda2pciO3qry9H+ rfLss7v/Ztlwnt7buh0f6c6h/9hoLrS2XWomPutgc/uBeWtjsvu/VaNJu8WW X-Gm-Gg: ASbGnctsJby1YIfF21x/xmG4eEKEpUI/4HRYu7V5QiL5gO9tMR6JwgpJDUu6rYEx7F6 MR0UAbuaGHAlr+sucIzz9XTrlmOkOE1E0gH9xU0qqrNBcdggjKY7cu/U+xJniQilSu5CqXx8sOd VaEGfRW2G2LF9SEIdOytCjBhayjeZLftekm63NrD8j7/SUMWsyzyevf/a2qcxqlzk6gKjMSxg+q Glw1J6e8w9izbveAPHhtvTlKBtWgyAAIxA2i+W0wOK9MB4UB+5p/EnGiqiaxUQp4GT7TTFNTHxo rMnu2AeaBMoAWYIbP1yl/wx5 X-Google-Smtp-Source: AGHT+IEqr7zRhFC4k7IPRBFT2wBIc8O+jOhPTBOkz5rfd/ZFT1G7hM+80BWLur8mwrqWCFKsQs+d3g== X-Received: by 2002:a05:6402:1ed4:b0:5d0:c697:1f02 with SMTP id 4fb4d7f45d1cf-5db7d30092amr47616733a12.17.1737574395183; Wed, 22 Jan 2025 11:33:15 -0800 (PST) Received: from dovchinn.amd.com ([87.116.182.236]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ab384f223e7sm950721066b.111.2025.01.22.11.33.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jan 2025 11:33:14 -0800 (PST) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 22 Jan 2025 20:32:55 +0100 Message-Id: <20250122193255.1493-5-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20250122193255.1493-1-ovchinnikov.dmitrii@gmail.com> References: <20250122193255.1493-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v6 5/5] avcodec/amfenc: redesign to use hwcontext_amf. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov , Dmitrii Ovchinnikov Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Co-authored-by: Evgeny Pavlov v3: cleanup code --- libavcodec/amfenc.c | 873 +++++++++++++-------------------------- libavcodec/amfenc.h | 34 +- libavcodec/amfenc_av1.c | 8 +- libavcodec/amfenc_h264.c | 6 +- libavcodec/amfenc_hevc.c | 6 +- 5 files changed, 305 insertions(+), 622 deletions(-) diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index c8411902f4..0fa3123cac 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -22,6 +22,8 @@ #include "libavutil/avassert.h" #include "libavutil/imgutils.h" #include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_amf.h" +#include "libavutil/hwcontext_amf_internal.h" #if CONFIG_D3D11VA #include "libavutil/hwcontext_d3d11va.h" #endif @@ -38,6 +40,8 @@ #include "internal.h" #include "libavutil/mastering_display_metadata.h" +#define AMF_AV_FRAME_REF L"av_frame_ref" + static int amf_save_hdr_metadata(AVCodecContext *avctx, const AVFrame *frame, AMFHDRMetadata *hdrmeta) { AVFrameSideData *sd_display; @@ -112,287 +116,18 @@ const enum AVPixelFormat ff_amf_pix_fmts[] = { AV_PIX_FMT_DXVA2_VLD, #endif AV_PIX_FMT_P010, + AV_PIX_FMT_AMF_SURFACE, AV_PIX_FMT_NONE }; -typedef struct FormatMap { - enum AVPixelFormat av_format; - enum AMF_SURFACE_FORMAT amf_format; -} FormatMap; - -static const FormatMap format_map[] = -{ - { AV_PIX_FMT_NONE, AMF_SURFACE_UNKNOWN }, - { AV_PIX_FMT_NV12, AMF_SURFACE_NV12 }, - { AV_PIX_FMT_P010, AMF_SURFACE_P010 }, - { AV_PIX_FMT_BGR0, AMF_SURFACE_BGRA }, - { AV_PIX_FMT_RGB0, AMF_SURFACE_RGBA }, - { AV_PIX_FMT_GRAY8, AMF_SURFACE_GRAY8 }, - { AV_PIX_FMT_YUV420P, AMF_SURFACE_YUV420P }, - { AV_PIX_FMT_YUYV422, AMF_SURFACE_YUY2 }, -}; - -static enum AMF_SURFACE_FORMAT amf_av_to_amf_format(enum AVPixelFormat fmt) -{ - int i; - for (i = 0; i < amf_countof(format_map); i++) { - if (format_map[i].av_format == fmt) { - return format_map[i].amf_format; - } - } - return AMF_SURFACE_UNKNOWN; -} - -static void AMF_CDECL_CALL AMFTraceWriter_Write(AMFTraceWriter *pThis, - const wchar_t *scope, const wchar_t *message) -{ - AmfTraceWriter *tracer = (AmfTraceWriter*)pThis; - av_log(tracer->avctx, AV_LOG_DEBUG, "%ls: %ls", scope, message); // \n is provided from AMF -} - -static void AMF_CDECL_CALL AMFTraceWriter_Flush(AMFTraceWriter *pThis) -{ -} - -static AMFTraceWriterVtbl tracer_vtbl = -{ - .Write = AMFTraceWriter_Write, - .Flush = AMFTraceWriter_Flush, -}; - -static int amf_load_library(AVCodecContext *avctx) -{ - AmfContext *ctx = avctx->priv_data; - AMFInit_Fn init_fun; - AMFQueryVersion_Fn version_fun; - AMF_RESULT res; - - ctx->delayed_frame = av_frame_alloc(); - if (!ctx->delayed_frame) { - return AVERROR(ENOMEM); - } - // hardcoded to current HW queue size - will auto-realloc if too small - ctx->timestamp_list = av_fifo_alloc2(avctx->max_b_frames + 16, sizeof(int64_t), - AV_FIFO_FLAG_AUTO_GROW); - if (!ctx->timestamp_list) { - return AVERROR(ENOMEM); - } - ctx->dts_delay = 0; - - - ctx->library = dlopen(AMF_DLL_NAMEA, RTLD_NOW | RTLD_LOCAL); - AMF_RETURN_IF_FALSE(ctx, ctx->library != NULL, - AVERROR_UNKNOWN, "DLL %s failed to open\n", AMF_DLL_NAMEA); - - init_fun = (AMFInit_Fn)dlsym(ctx->library, AMF_INIT_FUNCTION_NAME); - AMF_RETURN_IF_FALSE(ctx, init_fun != NULL, AVERROR_UNKNOWN, "DLL %s failed to find function %s\n", AMF_DLL_NAMEA, AMF_INIT_FUNCTION_NAME); - - version_fun = (AMFQueryVersion_Fn)dlsym(ctx->library, AMF_QUERY_VERSION_FUNCTION_NAME); - AMF_RETURN_IF_FALSE(ctx, version_fun != NULL, AVERROR_UNKNOWN, "DLL %s failed to find function %s\n", AMF_DLL_NAMEA, AMF_QUERY_VERSION_FUNCTION_NAME); - - res = version_fun(&ctx->version); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "%s failed with error %d\n", AMF_QUERY_VERSION_FUNCTION_NAME, res); - res = init_fun(AMF_FULL_VERSION, &ctx->factory); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "%s failed with error %d\n", AMF_INIT_FUNCTION_NAME, res); - res = ctx->factory->pVtbl->GetTrace(ctx->factory, &ctx->trace); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetTrace() failed with error %d\n", res); - res = ctx->factory->pVtbl->GetDebug(ctx->factory, &ctx->debug); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetDebug() failed with error %d\n", res); - return 0; -} - -#if CONFIG_D3D11VA -static int amf_init_from_d3d11_device(AVCodecContext *avctx, AVD3D11VADeviceContext *hwctx) -{ - AmfContext *ctx = avctx->priv_data; - AMF_RESULT res; - - res = ctx->context->pVtbl->InitDX11(ctx->context, hwctx->device, AMF_DX11_1); - if (res != AMF_OK) { - if (res == AMF_NOT_SUPPORTED) - av_log(avctx, AV_LOG_ERROR, "AMF via D3D11 is not supported on the given device.\n"); - else - av_log(avctx, AV_LOG_ERROR, "AMF failed to initialise on the given D3D11 device: %d.\n", res); - return AVERROR(ENODEV); - } - - return 0; -} -#endif - -#if CONFIG_DXVA2 -static int amf_init_from_dxva2_device(AVCodecContext *avctx, AVDXVA2DeviceContext *hwctx) -{ - AmfContext *ctx = avctx->priv_data; - HANDLE device_handle; - IDirect3DDevice9 *device; - HRESULT hr; - AMF_RESULT res; - int ret; - - hr = IDirect3DDeviceManager9_OpenDeviceHandle(hwctx->devmgr, &device_handle); - if (FAILED(hr)) { - av_log(avctx, AV_LOG_ERROR, "Failed to open device handle for Direct3D9 device: %lx.\n", (unsigned long)hr); - return AVERROR_EXTERNAL; - } - - hr = IDirect3DDeviceManager9_LockDevice(hwctx->devmgr, device_handle, &device, FALSE); - if (SUCCEEDED(hr)) { - IDirect3DDeviceManager9_UnlockDevice(hwctx->devmgr, device_handle, FALSE); - ret = 0; - } else { - av_log(avctx, AV_LOG_ERROR, "Failed to lock device handle for Direct3D9 device: %lx.\n", (unsigned long)hr); - ret = AVERROR_EXTERNAL; - } - - IDirect3DDeviceManager9_CloseDeviceHandle(hwctx->devmgr, device_handle); - - if (ret < 0) - return ret; - - res = ctx->context->pVtbl->InitDX9(ctx->context, device); - - IDirect3DDevice9_Release(device); - - if (res != AMF_OK) { - if (res == AMF_NOT_SUPPORTED) - av_log(avctx, AV_LOG_ERROR, "AMF via D3D9 is not supported on the given device.\n"); - else - av_log(avctx, AV_LOG_ERROR, "AMF failed to initialise on given D3D9 device: %d.\n", res); - return AVERROR(ENODEV); - } - - return 0; -} -#endif - -static int amf_init_context(AVCodecContext *avctx) -{ - AmfContext *ctx = avctx->priv_data; - AMFContext1 *context1 = NULL; - AMF_RESULT res; - av_unused int ret; - - ctx->hwsurfaces_in_queue = 0; - - // configure AMF logger - // the return of these functions indicates old state and do not affect behaviour - ctx->trace->pVtbl->EnableWriter(ctx->trace, AMF_TRACE_WRITER_DEBUG_OUTPUT, ctx->log_to_dbg != 0 ); - if (ctx->log_to_dbg) - ctx->trace->pVtbl->SetWriterLevel(ctx->trace, AMF_TRACE_WRITER_DEBUG_OUTPUT, AMF_TRACE_TRACE); - ctx->trace->pVtbl->EnableWriter(ctx->trace, AMF_TRACE_WRITER_CONSOLE, 0); - ctx->trace->pVtbl->SetGlobalLevel(ctx->trace, AMF_TRACE_TRACE); - - // connect AMF logger to av_log - ctx->tracer.vtbl = &tracer_vtbl; - ctx->tracer.avctx = avctx; - ctx->trace->pVtbl->RegisterWriter(ctx->trace, FFMPEG_AMF_WRITER_ID,(AMFTraceWriter*)&ctx->tracer, 1); - ctx->trace->pVtbl->SetWriterLevel(ctx->trace, FFMPEG_AMF_WRITER_ID, AMF_TRACE_TRACE); - - res = ctx->factory->pVtbl->CreateContext(ctx->factory, &ctx->context); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext() failed with error %d\n", res); - - // If a device was passed to the encoder, try to initialise from that. - if (avctx->hw_frames_ctx) { - AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; - - if (amf_av_to_amf_format(frames_ctx->sw_format) == AMF_SURFACE_UNKNOWN) { - av_log(avctx, AV_LOG_ERROR, "Format of input frames context (%s) is not supported by AMF.\n", - av_get_pix_fmt_name(frames_ctx->sw_format)); - return AVERROR(EINVAL); - } - - switch (frames_ctx->device_ctx->type) { -#if CONFIG_D3D11VA - case AV_HWDEVICE_TYPE_D3D11VA: - ret = amf_init_from_d3d11_device(avctx, frames_ctx->device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif -#if CONFIG_DXVA2 - case AV_HWDEVICE_TYPE_DXVA2: - ret = amf_init_from_dxva2_device(avctx, frames_ctx->device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif - default: - av_log(avctx, AV_LOG_ERROR, "AMF initialisation from a %s frames context is not supported.\n", - av_hwdevice_get_type_name(frames_ctx->device_ctx->type)); - return AVERROR(ENOSYS); - } - - ctx->hw_frames_ctx = av_buffer_ref(avctx->hw_frames_ctx); - if (!ctx->hw_frames_ctx) - return AVERROR(ENOMEM); - - if (frames_ctx->initial_pool_size > 0) - ctx->hwsurfaces_in_queue_max = FFMIN(ctx->hwsurfaces_in_queue_max, frames_ctx->initial_pool_size - 1); - - } else if (avctx->hw_device_ctx) { - AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; - - switch (device_ctx->type) { -#if CONFIG_D3D11VA - case AV_HWDEVICE_TYPE_D3D11VA: - ret = amf_init_from_d3d11_device(avctx, device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif -#if CONFIG_DXVA2 - case AV_HWDEVICE_TYPE_DXVA2: - ret = amf_init_from_dxva2_device(avctx, device_ctx->hwctx); - if (ret < 0) - return ret; - break; -#endif - default: - av_log(avctx, AV_LOG_ERROR, "AMF initialisation from a %s device is not supported.\n", - av_hwdevice_get_type_name(device_ctx->type)); - return AVERROR(ENOSYS); - } - - ctx->hw_device_ctx = av_buffer_ref(avctx->hw_device_ctx); - if (!ctx->hw_device_ctx) - return AVERROR(ENOMEM); - - } else { - res = ctx->context->pVtbl->InitDX11(ctx->context, NULL, AMF_DX11_1); - if (res == AMF_OK) { - av_log(avctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D11.\n"); - } else { - res = ctx->context->pVtbl->InitDX9(ctx->context, NULL); - if (res == AMF_OK) { - av_log(avctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D9.\n"); - } else { - AMFGuid guid = IID_AMFContext1(); - res = ctx->context->pVtbl->QueryInterface(ctx->context, &guid, (void**)&context1); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext1() failed with error %d\n", res); - - res = context1->pVtbl->InitVulkan(context1, NULL); - context1->pVtbl->Release(context1); - if (res != AMF_OK) { - if (res == AMF_NOT_SUPPORTED) - av_log(avctx, AV_LOG_ERROR, "AMF via Vulkan is not supported on the given device.\n"); - else - av_log(avctx, AV_LOG_ERROR, "AMF failed to initialise on the given Vulkan device: %d.\n", res); - return AVERROR(ENOSYS); - } - av_log(avctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via Vulkan.\n"); - } - } - } - return 0; -} - static int amf_init_encoder(AVCodecContext *avctx) { - AmfContext *ctx = avctx->priv_data; - const wchar_t *codec_id = NULL; - AMF_RESULT res; - enum AVPixelFormat pix_fmt; + AMFEncoderContext *ctx = avctx->priv_data; + const wchar_t *codec_id = NULL; + AMF_RESULT res; + enum AVPixelFormat pix_fmt; + AVHWDeviceContext *hw_device_ctx = (AVHWDeviceContext*)ctx->device_ctx_ref->data; + AVAMFDeviceContext *amf_device_ctx = (AVAMFDeviceContext *)hw_device_ctx->hwctx; switch (avctx->codec->id) { case AV_CODEC_ID_H264: @@ -409,35 +144,32 @@ static int amf_init_encoder(AVCodecContext *avctx) } AMF_RETURN_IF_FALSE(ctx, codec_id != NULL, AVERROR(EINVAL), "Codec %d is not supported\n", avctx->codec->id); - if (ctx->hw_frames_ctx) - pix_fmt = ((AVHWFramesContext*)ctx->hw_frames_ctx->data)->sw_format; + if (avctx->hw_frames_ctx) + pix_fmt = ((AVHWFramesContext*)avctx->hw_frames_ctx->data)->sw_format; else pix_fmt = avctx->pix_fmt; if (pix_fmt == AV_PIX_FMT_P010) { - AMF_RETURN_IF_FALSE(ctx, ctx->version >= AMF_MAKE_FULL_VERSION(1, 4, 32, 0), AVERROR_UNKNOWN, "10-bit encoder is not supported by AMD GPU drivers versions lower than 23.30.\n"); + AMF_RETURN_IF_FALSE(ctx, amf_device_ctx->version >= AMF_MAKE_FULL_VERSION(1, 4, 32, 0), AVERROR_UNKNOWN, "10-bit encoder is not supported by AMD GPU drivers versions lower than 23.30.\n"); } - ctx->format = amf_av_to_amf_format(pix_fmt); + ctx->format = av_av_to_amf_format(pix_fmt); AMF_RETURN_IF_FALSE(ctx, ctx->format != AMF_SURFACE_UNKNOWN, AVERROR(EINVAL), "Format %s is not supported\n", av_get_pix_fmt_name(pix_fmt)); - res = ctx->factory->pVtbl->CreateComponent(ctx->factory, ctx->context, codec_id, &ctx->encoder); + res = amf_device_ctx->factory->pVtbl->CreateComponent(amf_device_ctx->factory, amf_device_ctx->context, codec_id, &ctx->encoder); AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_ENCODER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", codec_id, res); ctx->submitted_frame = 0; + ctx->encoded_frame = 0; + ctx->eof = 0; return 0; } int av_cold ff_amf_encode_close(AVCodecContext *avctx) { - AmfContext *ctx = avctx->priv_data; - - if (ctx->delayed_surface) { - ctx->delayed_surface->pVtbl->Release(ctx->delayed_surface); - ctx->delayed_surface = NULL; - } + AMFEncoderContext *ctx = avctx->priv_data; if (ctx->encoder) { ctx->encoder->pVtbl->Terminate(ctx->encoder); @@ -445,27 +177,7 @@ int av_cold ff_amf_encode_close(AVCodecContext *avctx) ctx->encoder = NULL; } - if (ctx->context) { - ctx->context->pVtbl->Terminate(ctx->context); - ctx->context->pVtbl->Release(ctx->context); - ctx->context = NULL; - } - av_buffer_unref(&ctx->hw_device_ctx); - av_buffer_unref(&ctx->hw_frames_ctx); - - if (ctx->trace) { - ctx->trace->pVtbl->UnregisterWriter(ctx->trace, FFMPEG_AMF_WRITER_ID); - } - if (ctx->library) { - dlclose(ctx->library); - ctx->library = NULL; - } - ctx->trace = NULL; - ctx->debug = NULL; - ctx->factory = NULL; - ctx->version = 0; - ctx->delayed_drain = 0; - av_frame_free(&ctx->delayed_frame); + av_buffer_unref(&ctx->device_ctx_ref); av_fifo_freep2(&ctx->timestamp_list); return 0; @@ -475,12 +187,12 @@ static int amf_copy_surface(AVCodecContext *avctx, const AVFrame *frame, AMFSurface* surface) { AMFPlane *plane; - uint8_t *dst_data[4]; - int dst_linesize[4]; + uint8_t *dst_data[4] = {0}; + int dst_linesize[4] = {0}; int planes; int i; - planes = surface->pVtbl->GetPlanesCount(surface); + planes = (int)surface->pVtbl->GetPlanesCount(surface); av_assert0(planes < FF_ARRAY_ELEMS(dst_data)); for (i = 0; i < planes; i++) { @@ -497,7 +209,7 @@ static int amf_copy_surface(AVCodecContext *avctx, const AVFrame *frame, static int amf_copy_buffer(AVCodecContext *avctx, AVPacket *pkt, AMFBuffer *buffer) { - AmfContext *ctx = avctx->priv_data; + AMFEncoderContext *ctx = avctx->priv_data; int ret; AMFVariantStruct var = {0}; int64_t timestamp = AV_NOPTS_VALUE; @@ -534,7 +246,6 @@ static int amf_copy_buffer(AVCodecContext *avctx, AVPacket *pkt, AMFBuffer *buff pkt->pts = var.int64Value; // original pts - AMF_RETURN_IF_FALSE(ctx, av_fifo_read(ctx->timestamp_list, ×tamp, 1) >= 0, AVERROR_UNKNOWN, "timestamp_list is empty\n"); @@ -542,6 +253,7 @@ static int amf_copy_buffer(AVCodecContext *avctx, AVPacket *pkt, AMFBuffer *buff if ((ctx->max_b_frames > 0 || ((ctx->pa_adaptive_mini_gop == 1) ? true : false)) && ctx->dts_delay == 0) { int64_t timestamp_last = AV_NOPTS_VALUE; size_t can_read = av_fifo_can_read(ctx->timestamp_list); + AMF_RETURN_IF_FALSE(ctx, can_read > 0, AVERROR_UNKNOWN, "timestamp_list is empty while max_b_frames = %d\n", avctx->max_b_frames); av_fifo_peek(ctx->timestamp_list, ×tamp_last, 1, can_read - 1); @@ -558,14 +270,50 @@ static int amf_copy_buffer(AVCodecContext *avctx, AVPacket *pkt, AMFBuffer *buff int ff_amf_encode_init(AVCodecContext *avctx) { int ret; + AMFEncoderContext *ctx = avctx->priv_data; + AVHWDeviceContext *hwdev_ctx = NULL; + + // hardcoded to current HW queue size - will auto-realloc if too small + ctx->timestamp_list = av_fifo_alloc2(avctx->max_b_frames + 16, sizeof(int64_t), + AV_FIFO_FLAG_AUTO_GROW); + if (!ctx->timestamp_list) { + return AVERROR(ENOMEM); + } + ctx->dts_delay = 0; + + ctx->hwsurfaces_in_queue = 0; - if ((ret = amf_load_library(avctx)) == 0) { - if ((ret = amf_init_context(avctx)) == 0) { - if ((ret = amf_init_encoder(avctx)) == 0) { - return 0; + if (avctx->hw_device_ctx) { + hwdev_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + if (hwdev_ctx->type == AV_HWDEVICE_TYPE_AMF) + { + ctx->device_ctx_ref = av_buffer_ref(avctx->hw_device_ctx); + } + else { + ret = av_hwdevice_ctx_create_derived(&ctx->device_ctx_ref, AV_HWDEVICE_TYPE_AMF, avctx->hw_device_ctx, 0); + AMF_RETURN_IF_FALSE(avctx, ret == 0, ret, "Failed to create derived AMF device context: %s\n", av_err2str(ret)); + } + } else if (avctx->hw_frames_ctx) { + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + if (frames_ctx->device_ref ) { + if (frames_ctx->format == AV_PIX_FMT_AMF_SURFACE) { + ctx->device_ctx_ref = av_buffer_ref(frames_ctx->device_ref); + } + else { + ret = av_hwdevice_ctx_create_derived(&ctx->device_ctx_ref, AV_HWDEVICE_TYPE_AMF, frames_ctx->device_ref, 0); + AMF_RETURN_IF_FALSE(avctx, ret == 0, ret, "Failed to create derived AMF device context: %s\n", av_err2str(ret)); } } } + else { + ret = av_hwdevice_ctx_create(&ctx->device_ctx_ref, AV_HWDEVICE_TYPE_AMF, NULL, NULL, 0); + AMF_RETURN_IF_FALSE(avctx, ret == 0, ret, "Failed to create hardware device context (AMF) : %s\n", av_err2str(ret)); + } + + if ((ret = amf_init_encoder(avctx)) == 0) { + return 0; + } + ff_amf_encode_close(avctx); return ret; } @@ -592,332 +340,270 @@ static AMF_RESULT amf_set_property_buffer(AMFSurface *object, const wchar_t *nam return res; } -static AMF_RESULT amf_get_property_buffer(AMFData *object, const wchar_t *name, AMFBuffer **val) +static AMF_RESULT amf_store_attached_frame_ref(const AVFrame *frame, AMFSurface *surface) { - AMF_RESULT res; - AMFVariantStruct var; - res = AMFVariantInit(&var); - if (res == AMF_OK) { - res = object->pVtbl->GetProperty(object, name, &var); - if (res == AMF_OK) { - if (var.type == AMF_VARIANT_INTERFACE) { - AMFGuid guid_AMFBuffer = IID_AMFBuffer(); - AMFInterface *amf_interface = AMFVariantInterface(&var); - res = amf_interface->pVtbl->QueryInterface(amf_interface, &guid_AMFBuffer, (void**)val); - } else { - res = AMF_INVALID_DATA_TYPE; - } - } - AMFVariantClear(&var); + AMF_RESULT res = AMF_FAIL; + int64_t data; + AVFrame *frame_ref = av_frame_clone(frame); + if (frame_ref) { + memcpy(&data, &frame_ref, sizeof(frame_ref)); // store pointer in 8 bytes + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_AV_FRAME_REF, data); } return res; } -static AMFBuffer *amf_create_buffer_with_frame_ref(const AVFrame *frame, AMFContext *context) +static AMF_RESULT amf_release_attached_frame_ref(AMFBuffer *buffer) { - AVFrame *frame_ref; - AMFBuffer *frame_ref_storage_buffer = NULL; - AMF_RESULT res; - - res = context->pVtbl->AllocBuffer(context, AMF_MEMORY_HOST, sizeof(frame_ref), &frame_ref_storage_buffer); - if (res == AMF_OK) { - frame_ref = av_frame_clone(frame); - if (frame_ref) { - memcpy(frame_ref_storage_buffer->pVtbl->GetNative(frame_ref_storage_buffer), &frame_ref, sizeof(frame_ref)); - } else { - frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); - frame_ref_storage_buffer = NULL; - } + AMFVariantStruct var = {0}; + AMF_RESULT res = buffer->pVtbl->GetProperty(buffer, AMF_AV_FRAME_REF, &var); + if(res == AMF_OK && var.int64Value){ + AVFrame *frame_ref; + memcpy(&frame_ref, &var.int64Value, sizeof(frame_ref)); + av_frame_free(&frame_ref); } - return frame_ref_storage_buffer; -} - -static void amf_release_buffer_with_frame_ref(AMFBuffer *frame_ref_storage_buffer) -{ - AVFrame *frame_ref; - memcpy(&frame_ref, frame_ref_storage_buffer->pVtbl->GetNative(frame_ref_storage_buffer), sizeof(frame_ref)); - av_frame_free(&frame_ref); - frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); + return res; } int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) { - AmfContext *ctx = avctx->priv_data; + AMFEncoderContext *ctx = avctx->priv_data; + AVHWDeviceContext *hw_device_ctx = (AVHWDeviceContext*)ctx->device_ctx_ref->data; + AVAMFDeviceContext *amf_device_ctx = (AVAMFDeviceContext *)hw_device_ctx->hwctx; AMFSurface *surface; AMF_RESULT res; int ret; AMF_RESULT res_query; AMFData *data = NULL; - AVFrame *frame = ctx->delayed_frame; + AVFrame *frame = av_frame_alloc(); int block_and_wait; - int query_output_data_flag = 0; - AMF_RESULT res_resubmit; + int input_full = 0; + int hw_surface = 0; + int64_t pts = 0; + int max_b_frames = ctx->max_b_frames < 0 ? 0 : ctx->max_b_frames; - if (!ctx->encoder) + if (!ctx->encoder){ + av_frame_free(&frame); return AVERROR(EINVAL); - - if (!frame->buf[0]) { - ret = ff_encode_get_frame(avctx, frame); - if (ret < 0 && ret != AVERROR_EOF) - return ret; } - - if (!frame->buf[0]) { // submit drain - if (!ctx->eof) { // submit drain one time only - if (ctx->delayed_surface != NULL) { - ctx->delayed_drain = 1; // input queue is full: resubmit Drain() in ff_amf_receive_packet - } else if(!ctx->delayed_drain) { - res = ctx->encoder->pVtbl->Drain(ctx->encoder); - if (res == AMF_INPUT_FULL) { - ctx->delayed_drain = 1; // input queue is full: resubmit Drain() in ff_amf_receive_packet - } else { - if (res == AMF_OK) { - ctx->eof = 1; // drain started + ret = ff_encode_get_frame(avctx, frame); + if(ret < 0){ + if(ret != AVERROR_EOF){ + av_frame_free(&frame); + if(ret == AVERROR(EAGAIN)){ + if(ctx->submitted_frame <= ctx->encoded_frame + max_b_frames + 1) // too soon to poll + return ret; + } + } + } + if(ret != AVERROR(EAGAIN)){ + if (!frame->buf[0]) { // submit drain + if (!ctx->eof) { // submit drain one time only + if(!ctx->delayed_drain) { + res = ctx->encoder->pVtbl->Drain(ctx->encoder); + if (res == AMF_INPUT_FULL) { + ctx->delayed_drain = 1; // input queue is full: resubmit Drain() in receive loop + } else { + if (res == AMF_OK) { + ctx->eof = 1; // drain started + } + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "Drain() failed with error %d\n", res); } - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "Drain() failed with error %d\n", res); } } - } - } else if (!ctx->delayed_surface) { // submit frame - int hw_surface = 0; + } else { // submit frame - // prepare surface from frame - switch (frame->format) { -#if CONFIG_D3D11VA - case AV_PIX_FMT_D3D11: - { - static const GUID AMFTextureArrayIndexGUID = { 0x28115527, 0xe7c3, 0x4b66, { 0x99, 0xd3, 0x4f, 0x2a, 0xe6, 0xb4, 0x7f, 0xaf } }; - ID3D11Texture2D *texture = (ID3D11Texture2D*)frame->data[0]; // actual texture - int index = (intptr_t)frame->data[1]; // index is a slice in texture array is - set to tell AMF which slice to use + // prepare surface from frame + switch (frame->format) { + #if CONFIG_D3D11VA + case AV_PIX_FMT_D3D11: + { + static const GUID AMFTextureArrayIndexGUID = { 0x28115527, 0xe7c3, 0x4b66, { 0x99, 0xd3, 0x4f, 0x2a, 0xe6, 0xb4, 0x7f, 0xaf } }; + ID3D11Texture2D *texture = (ID3D11Texture2D*)frame->data[0]; // actual texture + int index = (intptr_t)frame->data[1]; // index is a slice in texture array is - set to tell AMF which slice to use - av_assert0(frame->hw_frames_ctx && ctx->hw_frames_ctx && - frame->hw_frames_ctx->data == ctx->hw_frames_ctx->data); + av_assert0(frame->hw_frames_ctx && avctx->hw_frames_ctx && + frame->hw_frames_ctx->data == avctx->hw_frames_ctx->data); - texture->lpVtbl->SetPrivateData(texture, &AMFTextureArrayIndexGUID, sizeof(index), &index); + texture->lpVtbl->SetPrivateData(texture, &AMFTextureArrayIndexGUID, sizeof(index), &index); - res = ctx->context->pVtbl->CreateSurfaceFromDX11Native(ctx->context, texture, &surface, NULL); // wrap to AMF surface - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX11Native() failed with error %d\n", res); + res = amf_device_ctx->context->pVtbl->CreateSurfaceFromDX11Native(amf_device_ctx->context, texture, &surface, NULL); // wrap to AMF surface + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX11Native() failed with error %d\n", res); - hw_surface = 1; - } - break; -#endif -#if CONFIG_DXVA2 - case AV_PIX_FMT_DXVA2_VLD: - { - IDirect3DSurface9 *texture = (IDirect3DSurface9 *)frame->data[3]; // actual texture + hw_surface = 1; + } + break; + #endif + #if CONFIG_DXVA2 + case AV_PIX_FMT_DXVA2_VLD: + { + IDirect3DSurface9 *texture = (IDirect3DSurface9 *)frame->data[3]; // actual texture - res = ctx->context->pVtbl->CreateSurfaceFromDX9Native(ctx->context, texture, &surface, NULL); // wrap to AMF surface - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX9Native() failed with error %d\n", res); + res = amf_device_ctx->context->pVtbl->CreateSurfaceFromDX9Native(amf_device_ctx->context, texture, &surface, NULL); // wrap to AMF surface + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX9Native() failed with error %d\n", res); - hw_surface = 1; - } - break; -#endif - default: - { - res = ctx->context->pVtbl->AllocSurface(ctx->context, AMF_MEMORY_HOST, ctx->format, avctx->width, avctx->height, &surface); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "AllocSurface() failed with error %d\n", res); - amf_copy_surface(avctx, frame, surface); + hw_surface = 1; + } + break; + #endif + case AV_PIX_FMT_AMF_SURFACE: + { + surface = (AMFSurface*)frame->data[0]; + surface->pVtbl->Acquire(surface); + hw_surface = 1; + } + break; + default: + { + res = amf_device_ctx->context->pVtbl->AllocSurface(amf_device_ctx->context, AMF_MEMORY_HOST, ctx->format, avctx->width, avctx->height, &surface); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "AllocSurface() failed with error %d\n", res); + amf_copy_surface(avctx, frame, surface); + } + break; } - break; - } - - if (hw_surface) { - AMFBuffer *frame_ref_storage_buffer; - - // input HW surfaces can be vertically aligned by 16; tell AMF the real size - surface->pVtbl->SetCrop(surface, 0, 0, frame->width, frame->height); - frame_ref_storage_buffer = amf_create_buffer_with_frame_ref(frame, ctx->context); - AMF_RETURN_IF_FALSE(ctx, frame_ref_storage_buffer != NULL, AVERROR(ENOMEM), "create_buffer_with_frame_ref() returned NULL\n"); - - res = amf_set_property_buffer(surface, L"av_frame_ref", frame_ref_storage_buffer); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "SetProperty failed for \"av_frame_ref\" with error %d\n", res); - ctx->hwsurfaces_in_queue++; - frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); - } + if (hw_surface) { + amf_store_attached_frame_ref(frame, surface); + ctx->hwsurfaces_in_queue++; + // input HW surfaces can be vertically aligned by 16; tell AMF the real size + surface->pVtbl->SetCrop(surface, 0, 0, frame->width, frame->height); + } - // HDR10 metadata - if (frame->color_trc == AVCOL_TRC_SMPTE2084) { - AMFBuffer * hdrmeta_buffer = NULL; - res = ctx->context->pVtbl->AllocBuffer(ctx->context, AMF_MEMORY_HOST, sizeof(AMFHDRMetadata), &hdrmeta_buffer); - if (res == AMF_OK) { - AMFHDRMetadata * hdrmeta = (AMFHDRMetadata*)hdrmeta_buffer->pVtbl->GetNative(hdrmeta_buffer); - if (amf_save_hdr_metadata(avctx, frame, hdrmeta) == 0) { - switch (avctx->codec->id) { - case AV_CODEC_ID_H264: - AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_INPUT_HDR_METADATA, hdrmeta_buffer); break; - case AV_CODEC_ID_HEVC: - AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_INPUT_HDR_METADATA, hdrmeta_buffer); break; - case AV_CODEC_ID_AV1: - AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_INPUT_HDR_METADATA, hdrmeta_buffer); break; + // HDR10 metadata + if (frame->color_trc == AVCOL_TRC_SMPTE2084) { + AMFBuffer * hdrmeta_buffer = NULL; + res = amf_device_ctx->context->pVtbl->AllocBuffer(amf_device_ctx->context, AMF_MEMORY_HOST, sizeof(AMFHDRMetadata), &hdrmeta_buffer); + if (res == AMF_OK) { + AMFHDRMetadata * hdrmeta = (AMFHDRMetadata*)hdrmeta_buffer->pVtbl->GetNative(hdrmeta_buffer); + if (amf_save_hdr_metadata(avctx, frame, hdrmeta) == 0) { + switch (avctx->codec->id) { + case AV_CODEC_ID_H264: + AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_INPUT_HDR_METADATA, hdrmeta_buffer); break; + case AV_CODEC_ID_HEVC: + AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_INPUT_HDR_METADATA, hdrmeta_buffer); break; + case AV_CODEC_ID_AV1: + AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_INPUT_HDR_METADATA, hdrmeta_buffer); break; + } + res = amf_set_property_buffer(surface, L"av_frame_hdrmeta", hdrmeta_buffer); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "SetProperty failed for \"av_frame_hdrmeta\" with error %d\n", res); } - res = amf_set_property_buffer(surface, L"av_frame_hdrmeta", hdrmeta_buffer); - AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "SetProperty failed for \"av_frame_hdrmeta\" with error %d\n", res); + hdrmeta_buffer->pVtbl->Release(hdrmeta_buffer); } - hdrmeta_buffer->pVtbl->Release(hdrmeta_buffer); } - } - surface->pVtbl->SetPts(surface, frame->pts); - AMF_ASSIGN_PROPERTY_INT64(res, surface, PTS_PROP, frame->pts); - - switch (avctx->codec->id) { - case AV_CODEC_ID_H264: - AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_INSERT_AUD, !!ctx->aud); - switch (frame->pict_type) { - case AV_PICTURE_TYPE_I: - if (ctx->forced_idr) { - AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_INSERT_SPS, 1); - AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_INSERT_PPS, 1); - AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_IDR); - } else { - AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_I); + surface->pVtbl->SetPts(surface, frame->pts); + AMF_ASSIGN_PROPERTY_INT64(res, surface, PTS_PROP, frame->pts); + + switch (avctx->codec->id) { + case AV_CODEC_ID_H264: + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_INSERT_AUD, !!ctx->aud); + switch (frame->pict_type) { + case AV_PICTURE_TYPE_I: + if (ctx->forced_idr) { + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_INSERT_SPS, 1); + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_INSERT_PPS, 1); + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_IDR); + } else { + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_I); + } + break; + case AV_PICTURE_TYPE_P: + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_P); + break; + case AV_PICTURE_TYPE_B: + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_B); + break; } break; - case AV_PICTURE_TYPE_P: - AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_P); - break; - case AV_PICTURE_TYPE_B: - AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_B); + case AV_CODEC_ID_HEVC: + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_INSERT_AUD, !!ctx->aud); + switch (frame->pict_type) { + case AV_PICTURE_TYPE_I: + if (ctx->forced_idr) { + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_INSERT_HEADER, 1); + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_IDR); + } else { + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_I); + } + break; + case AV_PICTURE_TYPE_P: + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_P); + break; + } break; - } - break; - case AV_CODEC_ID_HEVC: - AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_INSERT_AUD, !!ctx->aud); - switch (frame->pict_type) { - case AV_PICTURE_TYPE_I: - if (ctx->forced_idr) { - AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_INSERT_HEADER, 1); - AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_IDR); - } else { - AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_I); + case AV_CODEC_ID_AV1: + if (frame->pict_type == AV_PICTURE_TYPE_I) { + if (ctx->forced_idr) { + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_AV1_FORCE_INSERT_SEQUENCE_HEADER, 1); + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE, AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE_KEY); + } else { + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE, AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE_INTRA_ONLY); + } } break; - case AV_PICTURE_TYPE_P: - AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_P); + default: break; } - break; - case AV_CODEC_ID_AV1: - if (frame->pict_type == AV_PICTURE_TYPE_I) { - if (ctx->forced_idr) { - AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_AV1_FORCE_INSERT_SEQUENCE_HEADER, 1); - AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE, AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE_KEY); - } else { - AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE, AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE_INTRA_ONLY); - } - } - break; - default: - break; - } - - // submit surface - res = ctx->encoder->pVtbl->SubmitInput(ctx->encoder, (AMFData*)surface); - if (res == AMF_INPUT_FULL) { // handle full queue - //store surface for later submission - ctx->delayed_surface = surface; - } else { - int64_t pts = frame->pts; - surface->pVtbl->Release(surface); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "SubmitInput() failed with error %d\n", res); + pts = frame->pts; + // submit surface + res = ctx->encoder->pVtbl->SubmitInput(ctx->encoder, (AMFData*)surface); + av_frame_free(&frame); + + if (res == AMF_INPUT_FULL) { // handle full queue + //store surface for later submission + input_full = 1; + } else { + surface->pVtbl->Release(surface); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "SubmitInput() failed with error %d\n", res); - av_frame_unref(frame); - ret = av_fifo_write(ctx->timestamp_list, &pts, 1); + ctx->submitted_frame++; + ret = av_fifo_write(ctx->timestamp_list, &pts, 1); - if (ctx->submitted_frame == 0) - { - ctx->use_b_frame = (ctx->max_b_frames > 0 || ((ctx->pa_adaptive_mini_gop == 1) ? true : false)); + if (ret < 0) + return ret; + if(ctx->submitted_frame <= ctx->encoded_frame + max_b_frames + 1) + return AVERROR(EAGAIN); // if frame just submiited - don't poll or wait } - ctx->submitted_frame++; - - if (ret < 0) - return ret; } } - + av_frame_free(&frame); do { block_and_wait = 0; // poll data - if (!avpkt->data && !avpkt->buf && (ctx->use_b_frame ? (ctx->submitted_frame >= 2) : true) ) { - res_query = ctx->encoder->pVtbl->QueryOutput(ctx->encoder, &data); - if (data) { - // copy data to packet - AMFBuffer *buffer; - AMFGuid guid = IID_AMFBuffer(); - query_output_data_flag = 1; - data->pVtbl->QueryInterface(data, &guid, (void**)&buffer); // query for buffer interface - ret = amf_copy_buffer(avctx, avpkt, buffer); - ctx->submitted_frame++; - buffer->pVtbl->Release(buffer); - - if (data->pVtbl->HasProperty(data, L"av_frame_ref")) { - AMFBuffer* frame_ref_storage_buffer; - res = amf_get_property_buffer(data, L"av_frame_ref", &frame_ref_storage_buffer); - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetProperty failed for \"av_frame_ref\" with error %d\n", res); - amf_release_buffer_with_frame_ref(frame_ref_storage_buffer); - ctx->hwsurfaces_in_queue--; - } - - data->pVtbl->Release(data); - - AMF_RETURN_IF_FALSE(ctx, ret >= 0, ret, "amf_copy_buffer() failed with error %d\n", ret); - } - } - res_resubmit = AMF_OK; - if (ctx->delayed_surface != NULL) { // try to resubmit frame - if (ctx->delayed_surface->pVtbl->HasProperty(ctx->delayed_surface, L"av_frame_hdrmeta")) { - AMFBuffer * hdrmeta_buffer = NULL; - res = amf_get_property_buffer((AMFData *)ctx->delayed_surface, L"av_frame_hdrmeta", &hdrmeta_buffer); - AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "GetProperty failed for \"av_frame_hdrmeta\" with error %d\n", res); - switch (avctx->codec->id) { - case AV_CODEC_ID_H264: - AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_INPUT_HDR_METADATA, hdrmeta_buffer); break; - case AV_CODEC_ID_HEVC: - AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_HEVC_INPUT_HDR_METADATA, hdrmeta_buffer); break; - case AV_CODEC_ID_AV1: - AMF_ASSIGN_PROPERTY_INTERFACE(res, ctx->encoder, AMF_VIDEO_ENCODER_AV1_INPUT_HDR_METADATA, hdrmeta_buffer); break; - } - hdrmeta_buffer->pVtbl->Release(hdrmeta_buffer); + res_query = ctx->encoder->pVtbl->QueryOutput(ctx->encoder, &data); + if (data) { + // copy data to packet + AMFBuffer *buffer; + AMFGuid guid = IID_AMFBuffer(); + data->pVtbl->QueryInterface(data, &guid, (void**)&buffer); // query for buffer interface + ret = amf_copy_buffer(avctx, avpkt, buffer); + if (amf_release_attached_frame_ref(buffer) == AMF_OK) { + ctx->hwsurfaces_in_queue--; } - res_resubmit = ctx->encoder->pVtbl->SubmitInput(ctx->encoder, (AMFData*)ctx->delayed_surface); - if (res_resubmit != AMF_INPUT_FULL) { - int64_t pts = ctx->delayed_surface->pVtbl->GetPts(ctx->delayed_surface); - ctx->delayed_surface->pVtbl->Release(ctx->delayed_surface); - ctx->delayed_surface = NULL; - av_frame_unref(ctx->delayed_frame); - AMF_RETURN_IF_FALSE(ctx, res_resubmit == AMF_OK, AVERROR_UNKNOWN, "Repeated SubmitInput() failed with error %d\n", res_resubmit); + ctx->encoded_frame++; + buffer->pVtbl->Release(buffer); + data->pVtbl->Release(data); - ctx->submitted_frame++; - ret = av_fifo_write(ctx->timestamp_list, &pts, 1); - if (ret < 0) - return ret; - } - } else if (ctx->delayed_drain) { // try to resubmit drain - res = ctx->encoder->pVtbl->Drain(ctx->encoder); - if (res != AMF_INPUT_FULL) { - ctx->delayed_drain = 0; - ctx->eof = 1; // drain started - AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "Repeated Drain() failed with error %d\n", res); - } else { - av_log(avctx, AV_LOG_WARNING, "Data acquired but delayed drain submission got AMF_INPUT_FULL- should not happen\n"); - } - } + AMF_RETURN_IF_FALSE(ctx, ret >= 0, ret, "amf_copy_buffer() failed with error %d\n", ret); - if (query_output_data_flag == 0) { - if (res_resubmit == AMF_INPUT_FULL || ctx->delayed_drain || (ctx->eof && res_query != AMF_EOF) || (ctx->hwsurfaces_in_queue >= ctx->hwsurfaces_in_queue_max)) { - block_and_wait = 1; - - // Only sleep if the driver doesn't support waiting in QueryOutput() - // or if we already have output data so we will skip calling it. - if (!ctx->query_timeout_supported || avpkt->data || avpkt->buf) { - av_usleep(1000); + if (ctx->delayed_drain) { // try to resubmit drain + res = ctx->encoder->pVtbl->Drain(ctx->encoder); + if (res != AMF_INPUT_FULL) { + ctx->delayed_drain = 0; + ctx->eof = 1; // drain started + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "Repeated Drain() failed with error %d\n", res); + } else { + av_log(avctx, AV_LOG_WARNING, "Data acquired but delayed drain submission got AMF_INPUT_FULL- should not happen\n"); } } + } else if (ctx->delayed_drain || (ctx->eof && res_query != AMF_EOF) || (ctx->hwsurfaces_in_queue >= ctx->hwsurfaces_in_queue_max) || input_full) { + block_and_wait = 1; + // Only sleep if the driver doesn't support waiting in QueryOutput() + // or if we already have output data so we will skip calling it. + if (!ctx->query_timeout_supported || avpkt->data || avpkt->buf) { + av_usleep(1000); + } } } while (block_and_wait); @@ -926,6 +612,23 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) } else if (data == NULL) { ret = AVERROR(EAGAIN); } else { + if(input_full) { + // resubmit surface + res = ctx->encoder->pVtbl->SubmitInput(ctx->encoder, (AMFData*)surface); + surface->pVtbl->Release(surface); + if (res == AMF_INPUT_FULL) { + av_log(avctx, AV_LOG_WARNING, "Data acquired but delayed SubmitInput returned AMF_INPUT_FULL- should not happen\n"); + } else { + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "SubmitInput() failed with error %d\n", res); + + ret = av_fifo_write(ctx->timestamp_list, &pts, 1); + + ctx->submitted_frame++; + + if (ret < 0) + return ret; + } + } ret = 0; } return ret; @@ -975,5 +678,7 @@ const AVCodecHWConfigInternal *const ff_amfenc_hw_configs[] = { HW_CONFIG_ENCODER_FRAMES(DXVA2_VLD, DXVA2), HW_CONFIG_ENCODER_DEVICE(NONE, DXVA2), #endif + HW_CONFIG_ENCODER_FRAMES(AMF_SURFACE, AMF), + HW_CONFIG_ENCODER_DEVICE(NONE, AMF), NULL, }; diff --git a/libavcodec/amfenc.h b/libavcodec/amfenc.h index d83ee5bf1c..f3c9a4f8f0 100644 --- a/libavcodec/amfenc.h +++ b/libavcodec/amfenc.h @@ -33,54 +33,32 @@ #define MAX_LOOKAHEAD_DEPTH 41 -/** -* AMF trace writer callback class -* Used to capture all AMF logging -*/ - -typedef struct AmfTraceWriter { - AMFTraceWriterVtbl *vtbl; - AVCodecContext *avctx; -} AmfTraceWriter; - /** * AMF encoder context */ -typedef struct AmfContext { +typedef struct AMFEncoderContext { AVClass *avclass; // access to AMF runtime - amf_handle library; ///< handle to DLL library - AMFFactory *factory; ///< pointer to AMF factory - AMFDebug *debug; ///< pointer to AMF debug interface - AMFTrace *trace; ///< pointer to AMF trace interface - - amf_uint64 version; ///< version of AMF runtime - AmfTraceWriter tracer; ///< AMF writer registered with AMF - AMFContext *context; ///< AMF context + AVBufferRef *device_ctx_ref; + //encoder AMFComponent *encoder; ///< AMF encoder object amf_bool eof; ///< flag indicating EOF happened AMF_SURFACE_FORMAT format; ///< AMF surface format - AVBufferRef *hw_device_ctx; ///< pointer to HW accelerator (decoder) - AVBufferRef *hw_frames_ctx; ///< pointer to HW accelerator (frame allocator) - int hwsurfaces_in_queue; int hwsurfaces_in_queue_max; int query_timeout_supported; // helpers to handle async calls int delayed_drain; - AMFSurface *delayed_surface; - AVFrame *delayed_frame; // shift dts back by max_b_frames in timing AVFifo *timestamp_list; int64_t dts_delay; - int submitted_frame; - amf_bool use_b_frame; - + int64_t submitted_frame; + int64_t encoded_frame; // common encoder option options int log_to_dbg; @@ -154,7 +132,7 @@ typedef struct AmfContext { int pa_adaptive_mini_gop; -} AmfContext; +} AMFEncoderContext; extern const AVCodecHWConfigInternal *const ff_amfenc_hw_configs[]; diff --git a/libavcodec/amfenc_av1.c b/libavcodec/amfenc_av1.c index bc6ad3d7fb..dc04289c0f 100644 --- a/libavcodec/amfenc_av1.c +++ b/libavcodec/amfenc_av1.c @@ -26,7 +26,7 @@ #define AMF_VIDEO_ENCODER_AV1_CAP_WIDTH_ALIGNMENT_FACTOR_LOCAL L"Av1WidthAlignmentFactor" // amf_int64; default = 1 #define AMF_VIDEO_ENCODER_AV1_CAP_HEIGHT_ALIGNMENT_FACTOR_LOCAL L"Av1HeightAlignmentFactor" // amf_int64; default = 1 -#define OFFSET(x) offsetof(AmfContext, x) +#define OFFSET(x) offsetof(AMFEncoderContext, x) #define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM static const AVOption options[] = { @@ -187,11 +187,11 @@ static av_cold int amf_encode_init_av1(AVCodecContext* avctx) { int ret = 0; AMF_RESULT res = AMF_OK; - AmfContext* ctx = avctx->priv_data; + AMFEncoderContext *ctx = avctx->priv_data; AMFVariantStruct var = { 0 }; amf_int64 profile = 0; amf_int64 profile_level = 0; - AMFBuffer* buffer; + AMFBuffer *buffer; AMFGuid guid; AMFRate framerate; AMFSize framesize = AMFConstructSize(avctx->width, avctx->height); @@ -693,7 +693,7 @@ const FFCodec ff_av1_amf_encoder = { .init = amf_encode_init_av1, FF_CODEC_RECEIVE_PACKET_CB(ff_amf_receive_packet), .close = ff_amf_encode_close, - .priv_data_size = sizeof(AmfContext), + .priv_data_size = sizeof(AMFEncoderContext), .p.priv_class = &av1_amf_class, .defaults = defaults, .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | diff --git a/libavcodec/amfenc_h264.c b/libavcodec/amfenc_h264.c index 78bd70eb7a..c5d54d97da 100644 --- a/libavcodec/amfenc_h264.c +++ b/libavcodec/amfenc_h264.c @@ -24,7 +24,7 @@ #include "codec_internal.h" #include -#define OFFSET(x) offsetof(AmfContext, x) +#define OFFSET(x) offsetof(AMFEncoderContext, x) #define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM static const AVOption options[] = { @@ -196,7 +196,7 @@ static av_cold int amf_encode_init_h264(AVCodecContext *avctx) { int ret = 0; AMF_RESULT res = AMF_OK; - AmfContext *ctx = avctx->priv_data; + AMFEncoderContext *ctx = avctx->priv_data; AMFVariantStruct var = { 0 }; amf_int64 profile = 0; amf_int64 profile_level = 0; @@ -605,7 +605,7 @@ const FFCodec ff_h264_amf_encoder = { .init = amf_encode_init_h264, FF_CODEC_RECEIVE_PACKET_CB(ff_amf_receive_packet), .close = ff_amf_encode_close, - .priv_data_size = sizeof(AmfContext), + .priv_data_size = sizeof(AMFEncoderContext), .p.priv_class = &h264_amf_class, .defaults = defaults, .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | diff --git a/libavcodec/amfenc_hevc.c b/libavcodec/amfenc_hevc.c index 0907db945c..34241cac74 100644 --- a/libavcodec/amfenc_hevc.c +++ b/libavcodec/amfenc_hevc.c @@ -23,7 +23,7 @@ #include "codec_internal.h" #include -#define OFFSET(x) offsetof(AmfContext, x) +#define OFFSET(x) offsetof(AMFEncoderContext, x) #define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM static const AVOption options[] = { @@ -160,7 +160,7 @@ static av_cold int amf_encode_init_hevc(AVCodecContext *avctx) { int ret = 0; AMF_RESULT res = AMF_OK; - AmfContext *ctx = avctx->priv_data; + AMFEncoderContext *ctx = avctx->priv_data; AMFVariantStruct var = {0}; amf_int64 profile = 0; amf_int64 profile_level = 0; @@ -534,7 +534,7 @@ const FFCodec ff_hevc_amf_encoder = { .init = amf_encode_init_hevc, FF_CODEC_RECEIVE_PACKET_CB(ff_amf_receive_packet), .close = ff_amf_encode_close, - .priv_data_size = sizeof(AmfContext), + .priv_data_size = sizeof(AMFEncoderContext), .p.priv_class = &hevc_amf_class, .defaults = defaults, .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | -- 2.47.1.windows.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".