From: epirat07@gmail.com
To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Cc: Dmitrii Ovchinnikov <ovchinnikov.dmitrii@gmail.com>
Subject: Re: [FFmpeg-devel] [PATCH v6 1/5] avutil: add hwcontext_amf.
Date: Wed, 22 Jan 2025 22:49:11 +0100
Message-ID: <D7D0123A-E8DA-4233-A63E-DA0BD7F3C6CA@gmail.com> (raw)
In-Reply-To: <20250122193255.1493-1-ovchinnikov.dmitrii@gmail.com>
On 22 Jan 2025, at 20:32, Dmitrii Ovchinnikov wrote:
> Adds hwcontext_amf, enabling a shared AMF context for encoders,
> decoders, and AMF-based filters, without copy to the host memory.
> Code also was tested in HandBrake.
>
> Benefits:
> - Optimizations for direct video memory access from CPU
> - Significant performance boost in full AMF pipelines with filters
> - Integration of GPU filters like VPP, Super Resolution, and
> Compression Artefact Removal(in future plans)
> - VCN power management control for decoders.
> - Ability to specify which VCN instance to use for decoding
> (like for encoder)
> - AMD will soon introduce full AMF API for multimedia accelerator MA35D
> - With AMF API, integration will be much easier:
> GPU and the accelerator will have the same API
> - including encoder, decoder, scaler, color converter,
> Windows and Linux.
> Learn more:
> https://www.amd.com/en/products/accelerators/alveo/ma35d.htm
That’s 404 as the l of html is missing, also not sure it belongs in a commit,
given its just a product page and no technical/API reference.
Do you have any instructions how to test this with the MA35D card, should it
just work?
>
> Changes by versions:
> v2: Header file cleanup.
> v3: Removed an unnecessary class.
> v4: code cleanup and improved error handling
> v5: Fixes related to HandBrake integration.
> v6: Sequential filters error and memory leak have been fixed.
> ---
> libavutil/Makefile | 4 +
> libavutil/hwcontext.c | 4 +
> libavutil/hwcontext.h | 1 +
> libavutil/hwcontext_amf.c | 610 +++++++++++++++++++++++++++++
> libavutil/hwcontext_amf.h | 45 +++
> libavutil/hwcontext_amf_internal.h | 44 +++
> libavutil/hwcontext_internal.h | 1 +
> libavutil/pixdesc.c | 4 +
> libavutil/pixfmt.h | 5 +
> 9 files changed, 718 insertions(+)
> create mode 100644 libavutil/hwcontext_amf.c
> create mode 100644 libavutil/hwcontext_amf.h
> create mode 100644 libavutil/hwcontext_amf_internal.h
>
> diff --git a/libavutil/Makefile b/libavutil/Makefile
> index f8031815bd..3076b575a5 100644
> --- a/libavutil/Makefile
> +++ b/libavutil/Makefile
> @@ -47,6 +47,7 @@ HEADERS = adler32.h \
> hwcontext_d3d12va.h \
> hwcontext_drm.h \
> hwcontext_dxva2.h \
> + hwcontext_amf.h \
> hwcontext_qsv.h \
> hwcontext_mediacodec.h \
> hwcontext_opencl.h \
> @@ -200,6 +201,7 @@ OBJS-$(CONFIG_CUDA) += hwcontext_cuda.o
> OBJS-$(CONFIG_D3D11VA) += hwcontext_d3d11va.o
> OBJS-$(CONFIG_D3D12VA) += hwcontext_d3d12va.o
> OBJS-$(CONFIG_DXVA2) += hwcontext_dxva2.o
> +OBJS-$(CONFIG_AMF) += hwcontext_amf.o
> OBJS-$(CONFIG_LIBDRM) += hwcontext_drm.o
> OBJS-$(CONFIG_MACOS_KPERF) += macos_kperf.o
> OBJS-$(CONFIG_MEDIACODEC) += hwcontext_mediacodec.o
> @@ -224,6 +226,8 @@ SKIPHEADERS-$(CONFIG_CUDA) += hwcontext_cuda_internal.h \
> SKIPHEADERS-$(CONFIG_D3D11VA) += hwcontext_d3d11va.h
> SKIPHEADERS-$(CONFIG_D3D12VA) += hwcontext_d3d12va.h
> SKIPHEADERS-$(CONFIG_DXVA2) += hwcontext_dxva2.h
> +SKIPHEADERS-$(CONFIG_AMF) += hwcontext_amf.h \
> + hwcontext_amf_internal
> SKIPHEADERS-$(CONFIG_QSV) += hwcontext_qsv.h
> SKIPHEADERS-$(CONFIG_OPENCL) += hwcontext_opencl.h
> SKIPHEADERS-$(CONFIG_VAAPI) += hwcontext_vaapi.h
> diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c
> index fa99a0d8a4..f06d49c45c 100644
> --- a/libavutil/hwcontext.c
> +++ b/libavutil/hwcontext.c
> @@ -65,6 +65,9 @@ static const HWContextType * const hw_table[] = {
> #endif
> #if CONFIG_VULKAN
> &ff_hwcontext_type_vulkan,
> +#endif
> +#if CONFIG_AMF
> + &ff_hwcontext_type_amf,
> #endif
> NULL,
> };
> @@ -82,6 +85,7 @@ static const char *const hw_type_names[] = {
> [AV_HWDEVICE_TYPE_VIDEOTOOLBOX] = "videotoolbox",
> [AV_HWDEVICE_TYPE_MEDIACODEC] = "mediacodec",
> [AV_HWDEVICE_TYPE_VULKAN] = "vulkan",
> + [AV_HWDEVICE_TYPE_AMF] = "amf",
> };
>
> typedef struct FFHWDeviceContext {
> diff --git a/libavutil/hwcontext.h b/libavutil/hwcontext.h
> index bac30debae..96042ba197 100644
> --- a/libavutil/hwcontext.h
> +++ b/libavutil/hwcontext.h
> @@ -38,6 +38,7 @@ enum AVHWDeviceType {
> AV_HWDEVICE_TYPE_MEDIACODEC,
> AV_HWDEVICE_TYPE_VULKAN,
> AV_HWDEVICE_TYPE_D3D12VA,
> + AV_HWDEVICE_TYPE_AMF,
> };
>
> /**
> diff --git a/libavutil/hwcontext_amf.c b/libavutil/hwcontext_amf.c
> new file mode 100644
> index 0000000000..8e0ce1927e
> --- /dev/null
> +++ b/libavutil/hwcontext_amf.c
> @@ -0,0 +1,610 @@
> +/*
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +#include "buffer.h"
> +#include "common.h"
> +#include "hwcontext.h"
> +#include "hwcontext_amf.h"
> +#include "hwcontext_internal.h"
> +#include "hwcontext_amf_internal.h"
> +#if CONFIG_VULKAN
> +#include "hwcontext_vulkan.h"
> +#endif
> +#if CONFIG_D3D11VA
> +#include "libavutil/hwcontext_d3d11va.h"
> +#endif
> +#if CONFIG_DXVA2
> +#define COBJMACROS
> +#include "libavutil/hwcontext_dxva2.h"
> +#endif
> +#include "mem.h"
> +#include "pixdesc.h"
> +#include "pixfmt.h"
> +#include "imgutils.h"
> +#include "libavutil/avassert.h"
> +#include <AMF/core/Surface.h>
> +#include <AMF/core/Trace.h>
> +#ifdef _WIN32
> +#include "compat/w32dlfcn.h"
> +#else
> +#include <dlfcn.h>
> +#endif
> +#define FFMPEG_AMF_WRITER_ID L"ffmpeg_amf"
> +
> +
> +typedef struct AmfTraceWriter {
> + AMFTraceWriterVtbl *vtblp;
> + void *avctx;
> + AMFTraceWriterVtbl vtbl;
> +} AmfTraceWriter;
> +
> +static void AMF_CDECL_CALL AMFTraceWriter_Write(AMFTraceWriter *pThis,
> + const wchar_t *scope, const wchar_t *message)
> +{
> + AmfTraceWriter *tracer = (AmfTraceWriter*)pThis;
> + av_log(tracer->avctx, AV_LOG_DEBUG, "%ls: %ls", scope, message); // \n is provided from AMF
> +}
> +
> +static void AMF_CDECL_CALL AMFTraceWriter_Flush(AMFTraceWriter *pThis)
> +{
> +}
> +
> +static AmfTraceWriter * amf_writer_alloc(void *avctx)
> +{
> + AmfTraceWriter * writer = av_mallocz(sizeof(AmfTraceWriter));
> + if (!writer)
> + return NULL;
> +
> + writer->vtblp = &writer->vtbl;
> + writer->vtblp->Write = AMFTraceWriter_Write;
> + writer->vtblp->Flush = AMFTraceWriter_Flush;
> + writer->avctx = avctx;
> +
> + return writer;
> +}
> +
> +static void amf_writer_free(void *opaque)
> +{
> + AmfTraceWriter *writer = (AmfTraceWriter *)opaque;
> + av_freep(&writer);
> +}
> +
> +/**
> + * We still need AVHWFramesContext to utilize our hardware memory
> + * otherwise, we will receive the error "HW format requires hw_frames_ctx to be non-NULL".
> + * (libavfilter\buffersrc.c function query_formats)
> +*/
> +typedef struct {
> + void *dummy;
> +} AMFFramesContext;
> +
> +typedef struct AVAMFFormatMap {
> + enum AVPixelFormat av_format;
> + enum AMF_SURFACE_FORMAT amf_format;
> +} FormatMap;
> +
> +const FormatMap format_map[] =
> +{
> + { AV_PIX_FMT_NONE, AMF_SURFACE_UNKNOWN },
> + { AV_PIX_FMT_NV12, AMF_SURFACE_NV12 },
> + { AV_PIX_FMT_BGR0, AMF_SURFACE_BGRA },
> + { AV_PIX_FMT_RGB0, AMF_SURFACE_RGBA },
> + { AV_PIX_FMT_BGRA, AMF_SURFACE_BGRA },
> + { AV_PIX_FMT_ARGB, AMF_SURFACE_ARGB },
> + { AV_PIX_FMT_RGBA, AMF_SURFACE_RGBA },
> + { AV_PIX_FMT_GRAY8, AMF_SURFACE_GRAY8 },
> + { AV_PIX_FMT_YUV420P, AMF_SURFACE_YUV420P },
> + { AV_PIX_FMT_YUYV422, AMF_SURFACE_YUY2 },
> + { AV_PIX_FMT_P010, AMF_SURFACE_P010 },
> +};
> +
> +enum AMF_SURFACE_FORMAT av_av_to_amf_format(enum AVPixelFormat fmt)
> +{
> + int i;
> + for (i = 0; i < amf_countof(format_map); i++) {
> + if (format_map[i].av_format == fmt) {
> + return format_map[i].amf_format;
> + }
> + }
> + return AMF_SURFACE_UNKNOWN;
> +}
> +
> +enum AVPixelFormat av_amf_to_av_format(enum AMF_SURFACE_FORMAT fmt)
> +{
> + int i;
> + for (i = 0; i < amf_countof(format_map); i++) {
> + if (format_map[i].amf_format == fmt) {
> + return format_map[i].av_format;
> + }
> + }
> + return AMF_SURFACE_UNKNOWN;
> +}
> +
> +static const enum AVPixelFormat supported_formats[] = {
> + AV_PIX_FMT_NV12,
> + AV_PIX_FMT_YUV420P,
> + AV_PIX_FMT_BGRA,
> + AV_PIX_FMT_RGBA,
> + AV_PIX_FMT_P010,
> +#if CONFIG_D3D11VA
> + AV_PIX_FMT_D3D11,
> +#endif
> +#if CONFIG_DXVA2
> + AV_PIX_FMT_DXVA2_VLD,
> +#endif
> +};
> +
> +static const enum AVPixelFormat supported_transfer_formats[] = {
> + AV_PIX_FMT_NV12,
> + AV_PIX_FMT_YUV420P,
> + AV_PIX_FMT_BGRA,
> + AV_PIX_FMT_RGBA,
> + AV_PIX_FMT_P010,
> + AV_PIX_FMT_NONE,
> +};
> +
> +static int amf_frames_get_constraints(AVHWDeviceContext *ctx,
> + const void *hwconfig,
> + AVHWFramesConstraints *constraints)
> +{
> + int i;
> +
> + constraints->valid_sw_formats = av_malloc_array(FF_ARRAY_ELEMS(supported_formats) + 1,
> + sizeof(*constraints->valid_sw_formats));
> + if (!constraints->valid_sw_formats)
> + return AVERROR(ENOMEM);
> +
> + for (i = 0; i < FF_ARRAY_ELEMS(supported_formats); i++)
> + constraints->valid_sw_formats[i] = supported_formats[i];
> + constraints->valid_sw_formats[FF_ARRAY_ELEMS(supported_formats)] = AV_PIX_FMT_NONE;
> +
> + constraints->valid_hw_formats = av_malloc_array(2, sizeof(*constraints->valid_hw_formats));
> + if (!constraints->valid_hw_formats)
> + return AVERROR(ENOMEM);
> +
> + constraints->valid_hw_formats[0] = AV_PIX_FMT_AMF_SURFACE;
> + constraints->valid_hw_formats[1] = AV_PIX_FMT_NONE;
> +
> + return 0;
> +}
> +
> +static void amf_dummy_free(void *opaque, uint8_t *data)
> +{
> +
> +}
> +
> +static AVBufferRef *amf_pool_alloc(void *opaque, size_t size)
> +{
> + AVHWFramesContext *hwfc = (AVHWFramesContext *)opaque;
> + AVBufferRef *buf;
> +
> + buf = av_buffer_create(NULL, 0, amf_dummy_free, hwfc, AV_BUFFER_FLAG_READONLY);
> + if (!buf) {
> + av_log(hwfc, AV_LOG_ERROR, "Failed to create buffer for AMF context.\n");
> + return NULL;
> + }
> + return buf;
> +}
> +
> +static int amf_frames_init(AVHWFramesContext *ctx)
> +{
> + int i;
> +
> + for (i = 0; i < FF_ARRAY_ELEMS(supported_formats); i++) {
> + if (ctx->sw_format == supported_formats[i])
> + break;
> + }
> + if (i == FF_ARRAY_ELEMS(supported_formats)) {
> + av_log(ctx, AV_LOG_ERROR, "Pixel format '%s' is not supported\n",
> + av_get_pix_fmt_name(ctx->sw_format));
> + return AVERROR(ENOSYS);
> + }
> +
> + ffhwframesctx(ctx)->pool_internal =
> + av_buffer_pool_init2(sizeof(AMFSurface), ctx,
> + &amf_pool_alloc, NULL);
> +
> + return 0;
> +}
> +
> +
> +static int amf_get_buffer(AVHWFramesContext *ctx, AVFrame *frame)
> +{
> + frame->buf[0] = av_buffer_pool_get(ctx->pool);
> + if (!frame->buf[0])
> + return AVERROR(ENOMEM);
> +
> + frame->data[0] = frame->buf[0]->data;
> + frame->format = AV_PIX_FMT_AMF_SURFACE;
> + frame->width = ctx->width;
> + frame->height = ctx->height;
> + return 0;
> +}
> +
> +static int amf_transfer_get_formats(AVHWFramesContext *ctx,
> + enum AVHWFrameTransferDirection dir,
> + enum AVPixelFormat **formats)
> +{
> + enum AVPixelFormat *fmts;
> + int i;
> +
> + fmts = av_malloc_array(FF_ARRAY_ELEMS(supported_transfer_formats), sizeof(*fmts));
> + if (!fmts)
> + return AVERROR(ENOMEM);
> + for (i = 0; i < FF_ARRAY_ELEMS(supported_transfer_formats); i++)
> + fmts[i] = supported_transfer_formats[i];
> +
> + *formats = fmts;
> +
> + return 0;
> +}
> +
> +static void amf_free_amfsurface(void *opaque, uint8_t *data)
> +{
> + if(!!data){
> + AMFSurface *surface = (AMFSurface*)(data);
> + surface->pVtbl->Release(surface);
> + }
> +}
> +
> +static int amf_transfer_data_to(AVHWFramesContext *ctx, AVFrame *dst,
> + const AVFrame *src)
> +{
> + AMFSurface* surface = (AMFSurface*)dst->data[0];
> + AMFPlane *plane;
> + uint8_t *dst_data[4];
> + int dst_linesize[4];
> + int planes;
> + int i;
> + int res;
> + int w = FFMIN(dst->width, src->width);
> + int h = FFMIN(dst->height, src->height);
> +
> + if (!surface) {
> + AVHWDeviceContext *hwdev_ctx = ctx->device_ctx;
> + AVAMFDeviceContext *amf_device_ctx = (AVAMFDeviceContext *)hwdev_ctx->hwctx;
> + AMF_SURFACE_FORMAT format = av_av_to_amf_format(ctx->sw_format);
> + res = amf_device_ctx->context->pVtbl->AllocSurface(amf_device_ctx->context, AMF_MEMORY_HOST, format, dst->width, dst->height, &surface);
> + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "AllocSurface() failed with error %d\n", res);
> + dst->data[0] = (uint8_t *)surface;
> + dst->buf[1] = av_buffer_create((uint8_t *)surface, sizeof(surface),
> + amf_free_amfsurface,
> + NULL,
> + AV_BUFFER_FLAG_READONLY);
> + AMF_RETURN_IF_FALSE(ctx, !!dst->buf[1], AVERROR(ENOMEM), "av_buffer_create for amf surface failed.");
> + }
> +
> + planes = (int)surface->pVtbl->GetPlanesCount(surface);
> + av_assert0(planes < FF_ARRAY_ELEMS(dst_data));
> +
> + for (i = 0; i < planes; i++) {
> + plane = surface->pVtbl->GetPlaneAt(surface, i);
> + dst_data[i] = plane->pVtbl->GetNative(plane);
> + dst_linesize[i] = plane->pVtbl->GetHPitch(plane);
> + }
> + av_image_copy2(dst_data, dst_linesize,
> + src->data, src->linesize, src->format,
> + w, h);
> +
> + return 0;
> +}
> +
> +static int amf_transfer_data_from(AVHWFramesContext *ctx, AVFrame *dst,
> + const AVFrame *src)
> +{
> + AMFSurface* surface = (AMFSurface*)src->data[0];
> + AMFPlane *plane;
> + uint8_t *src_data[4];
> + int src_linesize[4];
> + int planes;
> + int i;
> + int w = FFMIN(dst->width, src->width);
> + int h = FFMIN(dst->height, src->height);
> + int ret;
> +
> + ret = surface->pVtbl->Convert(surface, AMF_MEMORY_HOST);
> + AMF_RETURN_IF_FALSE(ctx, ret == AMF_OK, AVERROR_UNKNOWN, "Convert(amf::AMF_MEMORY_HOST) failed with error %d\n", AVERROR_UNKNOWN);
> +
> + planes = (int)surface->pVtbl->GetPlanesCount(surface);
> + av_assert0(planes < FF_ARRAY_ELEMS(src_data));
> +
> + for (i = 0; i < planes; i++) {
> + plane = surface->pVtbl->GetPlaneAt(surface, i);
> + src_data[i] = plane->pVtbl->GetNative(plane);
> + src_linesize[i] = plane->pVtbl->GetHPitch(plane);
> + }
> + av_image_copy2(dst->data, dst->linesize,
> + src_data, src_linesize, dst->format,
> + w, h);
> + return 0;
> +}
> +
> +
> +
> +static void amf_device_uninit(AVHWDeviceContext *device_ctx)
> +{
> + AVAMFDeviceContext *amf_ctx = device_ctx->hwctx;
> + AMF_RESULT res;
> + AMFTrace *trace;
> +
> + if (amf_ctx->context) {
> + amf_ctx->context->pVtbl->Terminate(amf_ctx->context);
> + amf_ctx->context->pVtbl->Release(amf_ctx->context);
> + amf_ctx->context = NULL;
> + }
> +
> + res = amf_ctx->factory->pVtbl->GetTrace(amf_ctx->factory, &trace);
> + if (res == AMF_OK) {
> + trace->pVtbl->UnregisterWriter(trace, FFMPEG_AMF_WRITER_ID);
> + }
> +
> + if(amf_ctx->library) {
> + dlclose(amf_ctx->library);
> + amf_ctx->library = NULL;
> + }
> + if (amf_ctx->trace_writer) {
> + amf_writer_free(amf_ctx->trace_writer);
> + }
> +
> + amf_ctx->version = 0;
> +}
> +
> +static int amf_device_init(AVHWDeviceContext *ctx)
> +{
> + AVAMFDeviceContext *amf_ctx = ctx->hwctx;
> + AMFContext1 *context1 = NULL;
> + AMF_RESULT res;
> +
> +#ifdef _WIN32
> + res = amf_ctx->context->pVtbl->InitDX11(amf_ctx->context, NULL, AMF_DX11_1);
> + if (res == AMF_OK || res == AMF_ALREADY_INITIALIZED) {
> + av_log(ctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D11.\n");
> + } else {
> + res = amf_ctx->context->pVtbl->InitDX9(amf_ctx->context, NULL);
> + if (res == AMF_OK) {
> + av_log(ctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via D3D9.\n");
> + } else {
> +#endif
> + AMFGuid guid = IID_AMFContext1();
> + res = amf_ctx->context->pVtbl->QueryInterface(amf_ctx->context, &guid, (void**)&context1);
> + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "CreateContext1() failed with error %d\n", res);
> +
> + res = context1->pVtbl->InitVulkan(context1, NULL);
> + context1->pVtbl->Release(context1);
> + if (res != AMF_OK && res != AMF_ALREADY_INITIALIZED) {
> + if (res == AMF_NOT_SUPPORTED)
> + av_log(ctx, AV_LOG_ERROR, "AMF via Vulkan is not supported on the given device.\n");
> + else
> + av_log(ctx, AV_LOG_ERROR, "AMF failed to initialise on the given Vulkan device: %d.\n", res);
> + return AVERROR(ENOSYS);
> + }
> + av_log(ctx, AV_LOG_VERBOSE, "AMF initialisation succeeded via Vulkan.\n");
> +#ifdef _WIN32
> + }
> + }
> +#endif
> + return 0;
> +}
> +
> +static int amf_load_library(AVAMFDeviceContext* amf_ctx, void* avcl)
> +{
> + AMFInit_Fn init_fun;
> + AMFQueryVersion_Fn version_fun;
> + AMF_RESULT res;
> +
> + amf_ctx->library = dlopen(AMF_DLL_NAMEA, RTLD_NOW | RTLD_LOCAL);
> + AMF_RETURN_IF_FALSE(avcl, amf_ctx->library != NULL,
> + AVERROR_UNKNOWN, "DLL %s failed to open\n", AMF_DLL_NAMEA);
> +
> + init_fun = (AMFInit_Fn)dlsym(amf_ctx->library, AMF_INIT_FUNCTION_NAME);
> + AMF_RETURN_IF_FALSE(avcl, init_fun != NULL, AVERROR_UNKNOWN, "DLL %s failed to find function %s\n", AMF_DLL_NAMEA, AMF_INIT_FUNCTION_NAME);
> +
> + version_fun = (AMFQueryVersion_Fn)dlsym(amf_ctx->library, AMF_QUERY_VERSION_FUNCTION_NAME);
> + AMF_RETURN_IF_FALSE(avcl, version_fun != NULL, AVERROR_UNKNOWN, "DLL %s failed to find function %s\n", AMF_DLL_NAMEA, AMF_QUERY_VERSION_FUNCTION_NAME);
> +
> + res = version_fun(&amf_ctx->version);
> + AMF_RETURN_IF_FALSE(avcl, res == AMF_OK, AVERROR_UNKNOWN, "%s failed with error %d\n", AMF_QUERY_VERSION_FUNCTION_NAME, res);
> + res = init_fun(AMF_FULL_VERSION, &amf_ctx->factory);
> + AMF_RETURN_IF_FALSE(avcl, res == AMF_OK, AVERROR_UNKNOWN, "%s failed with error %d\n", AMF_INIT_FUNCTION_NAME, res);
> + return 0;
> +}
> +
> +static int amf_device_create(AVHWDeviceContext *device_ctx,
> + const char *device,
> + AVDictionary *opts, int flags)
> +{
> + AVAMFDeviceContext *ctx = device_ctx->hwctx;
> + AMFTrace *trace;
> + int ret;
> + if ((ret = amf_load_library(ctx, device_ctx)) == 0) {
> + ret = ctx->factory->pVtbl->GetTrace(ctx->factory, &trace);
> + if (ret == AMF_OK) {
> + int level_ff = av_log_get_level();
> + int level_amf = AMF_TRACE_TRACE;
> + amf_bool enable_log = true;
> + switch(level_ff)
> + {
> + case AV_LOG_QUIET:
> + level_amf = AMF_TRACE_ERROR;
> + enable_log = false;
> + break;
> + case AV_LOG_PANIC:
> + case AV_LOG_FATAL:
> + case AV_LOG_ERROR:
> + level_amf = AMF_TRACE_ERROR;
> + break;
> + case AV_LOG_WARNING:
> + case AV_LOG_INFO:
> + level_amf = AMF_TRACE_WARNING;
> + break;
> + case AV_LOG_VERBOSE:
> + level_amf = AMF_TRACE_INFO;
> + break;
> + case AV_LOG_DEBUG:
> + level_amf = AMF_TRACE_DEBUG;
> + break;
> + case AV_LOG_TRACE:
> + level_amf = AMF_TRACE_TRACE;
> + break;
> + }
> + if(ctx->version == AMF_MAKE_FULL_VERSION(1, 4, 35, 0)){// get around a bug in trace in AMF runtime driver 24.20
> + level_amf = AMF_TRACE_WARNING;
> + }
> +
> + trace->pVtbl->EnableWriter(trace, AMF_TRACE_WRITER_CONSOLE, 0);
> + trace->pVtbl->SetGlobalLevel(trace, level_amf);
> +
> + // connect AMF logger to av_log
> + ctx->trace_writer = amf_writer_alloc(device_ctx);
> + trace->pVtbl->RegisterWriter(trace, FFMPEG_AMF_WRITER_ID, (AMFTraceWriter*)ctx->trace_writer, 1);
> + trace->pVtbl->SetWriterLevel(trace, FFMPEG_AMF_WRITER_ID, level_amf);
> + trace->pVtbl->EnableWriter(trace, FFMPEG_AMF_WRITER_ID, enable_log);
> + trace->pVtbl->SetWriterLevel(trace, AMF_TRACE_WRITER_DEBUG_OUTPUT, level_amf);
> + trace->pVtbl->EnableWriter(trace, AMF_TRACE_WRITER_DEBUG_OUTPUT, enable_log);
> + }
> +
> +
> + ret = ctx->factory->pVtbl->CreateContext(ctx->factory, &ctx->context);
> + if (ret == AMF_OK)
> + return 0;
> + av_log(device_ctx, AV_LOG_ERROR, "CreateContext() failed with error %d.\n", ret);
> + }
> + amf_device_uninit(device_ctx);
> + return ret;
> +}
> +
> +#if CONFIG_DXVA2
> +static int amf_init_from_dxva2_device(AVAMFDeviceContext * amf_ctx, AVDXVA2DeviceContext *hwctx)
> +{
> + IDirect3DDevice9 *device;
> + HANDLE device_handle;
> + HRESULT hr;
> + AMF_RESULT res;
> + int ret;
> +
> + hr = IDirect3DDeviceManager9_OpenDeviceHandle(hwctx->devmgr, &device_handle);
> + if (FAILED(hr)) {
> + av_log(hwctx, AV_LOG_ERROR, "Failed to open device handle for Direct3D9 device: %lx.\n", (unsigned long)hr);
> + return AVERROR_EXTERNAL;
> + }
> +
> + hr = IDirect3DDeviceManager9_LockDevice(hwctx->devmgr, device_handle, &device, FALSE);
> + if (SUCCEEDED(hr)) {
> + IDirect3DDeviceManager9_UnlockDevice(hwctx->devmgr, device_handle, FALSE);
> + ret = 0;
> + } else {
> + av_log(hwctx, AV_LOG_ERROR, "Failed to lock device handle for Direct3D9 device: %lx.\n", (unsigned long)hr);
> + ret = AVERROR_EXTERNAL;
> + }
> +
> +
> + IDirect3DDeviceManager9_CloseDeviceHandle(hwctx->devmgr, device_handle);
> +
> + if (ret < 0)
> + return ret;
> +
> + res = amf_ctx->context->pVtbl->InitDX9(amf_ctx->context, device);
> +
> + IDirect3DDevice9_Release(device);
> +
> + if (res != AMF_OK && res != AMF_ALREADY_INITIALIZED) {
> + if (res == AMF_NOT_SUPPORTED)
> + av_log(hwctx, AV_LOG_ERROR, "AMF via D3D9 is not supported on the given device.\n");
> + else
> + av_log(hwctx, AV_LOG_ERROR, "AMF failed to initialise on given D3D9 device: %d.\n", res);
> + return AVERROR(ENODEV);
> + }
> + return 0;
> +}
> +#endif
> +
> +#if CONFIG_D3D11VA
> +static int amf_init_from_d3d11_device(AVAMFDeviceContext* amf_ctx, AVD3D11VADeviceContext *hwctx)
> +{
> + AMF_RESULT res;
> + res = amf_ctx->context->pVtbl->InitDX11(amf_ctx->context, hwctx->device, AMF_DX11_1);
> + if (res != AMF_OK && res != AMF_ALREADY_INITIALIZED) {
> + if (res == AMF_NOT_SUPPORTED)
> + av_log(hwctx, AV_LOG_ERROR, "AMF via D3D11 is not supported on the given device.\n");
> + else
> + av_log(hwctx, AV_LOG_ERROR, "AMF failed to initialise on the given D3D11 device: %d.\n", res);
> + return AVERROR(ENODEV);
> + }
> + return 0;
> +}
> +#endif
> +
> +static int amf_device_derive(AVHWDeviceContext *device_ctx,
> + AVHWDeviceContext *child_device_ctx, AVDictionary *opts,
> + int flags)
> +{
> +#if CONFIG_DXVA2 || CONFIG_D3D11VA
> + AVAMFDeviceContext *amf_ctx = device_ctx->hwctx;
> +#endif
> + int ret;
> +
> + ret = amf_device_create(device_ctx, "", opts, flags);
> + if(ret < 0)
> + return ret;
> +
> + switch (child_device_ctx->type) {
> +
> +#if CONFIG_DXVA2
> + case AV_HWDEVICE_TYPE_DXVA2: {
> + AVDXVA2DeviceContext *child_device_hwctx = child_device_ctx->hwctx;
> + return amf_init_from_dxva2_device(amf_ctx, child_device_hwctx);
> + }
> + break;
> +#endif
> +
> +#if CONFIG_D3D11VA
> + case AV_HWDEVICE_TYPE_D3D11VA: {
> + AVD3D11VADeviceContext *child_device_hwctx = child_device_ctx->hwctx;
> + return amf_init_from_d3d11_device(amf_ctx, child_device_hwctx);
> + }
> + break;
> +#endif
> + default: {
> + av_log(child_device_ctx, AV_LOG_ERROR, "AMF initialisation from a %s device is not supported.\n",
> + av_hwdevice_get_type_name(child_device_ctx->type));
> + return AVERROR(ENOSYS);
> + }
> + }
> + return 0;
> +}
> +
> +const HWContextType ff_hwcontext_type_amf = {
> + .type = AV_HWDEVICE_TYPE_AMF,
> + .name = "AMF",
> +
> + .device_hwctx_size = sizeof(AVAMFDeviceContext),
> + .frames_hwctx_size = sizeof(AMFFramesContext),
> +
> + .device_create = amf_device_create,
> + .device_derive = amf_device_derive,
> + .device_init = amf_device_init,
> + .device_uninit = amf_device_uninit,
> + .frames_get_constraints = amf_frames_get_constraints,
> + .frames_init = amf_frames_init,
> + .frames_get_buffer = amf_get_buffer,
> + .transfer_get_formats = amf_transfer_get_formats,
> + .transfer_data_to = amf_transfer_data_to,
> + .transfer_data_from = amf_transfer_data_from,
> +
> + .pix_fmts = (const enum AVPixelFormat[]){ AV_PIX_FMT_AMF_SURFACE, AV_PIX_FMT_NONE },
> +};
> diff --git a/libavutil/hwcontext_amf.h b/libavutil/hwcontext_amf.h
> new file mode 100644
> index 0000000000..5ad2237f75
> --- /dev/null
> +++ b/libavutil/hwcontext_amf.h
> @@ -0,0 +1,45 @@
> +/*
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +
> +#ifndef AVUTIL_HWCONTEXT_AMF_H
> +#define AVUTIL_HWCONTEXT_AMF_H
> +
> +#include "pixfmt.h"
> +#include "hwcontext.h"
> +#include <AMF/core/Factory.h>
> +#include <AMF/core/Context.h>
> +#include <AMF/core/Trace.h>
> +#include <AMF/core/Debug.h>
> +
> +/**
> + * This struct is allocated as AVHWDeviceContext.hwctx
> + */
> +typedef struct AVAMFDeviceContext {
> + void * library;
> + AMFFactory *factory;
> + void *trace_writer;
> +
> + int64_t version; ///< version of AMF runtime
> + AMFContext *context;
> +} AVAMFDeviceContext;
> +
> +enum AMF_SURFACE_FORMAT av_av_to_amf_format(enum AVPixelFormat fmt);
> +enum AVPixelFormat av_amf_to_av_format(enum AMF_SURFACE_FORMAT fmt);
> +
> +#endif /* AVUTIL_HWCONTEXT_AMF_H */
> diff --git a/libavutil/hwcontext_amf_internal.h b/libavutil/hwcontext_amf_internal.h
> new file mode 100644
> index 0000000000..b991f357a6
> --- /dev/null
> +++ b/libavutil/hwcontext_amf_internal.h
> @@ -0,0 +1,44 @@
> +/*
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +
> +#ifndef AVUTIL_HWCONTEXT_AMF_INTERNAL_H
> +#define AVUTIL_HWCONTEXT_AMF_INTERNAL_H
> +#include <AMF/core/Factory.h>
> +#include <AMF/core/Context.h>
> +
> +/**
> +* Error handling helper
> +*/
> +#define AMF_RETURN_IF_FALSE(avctx, exp, ret_value, /*message,*/ ...) \
> + if (!(exp)) { \
> + av_log(avctx, AV_LOG_ERROR, __VA_ARGS__); \
> + return ret_value; \
> + }
> +
> +#define AMF_GOTO_FAIL_IF_FALSE(avctx, exp, ret_value, /*message,*/ ...) \
> + if (!(exp)) { \
> + av_log(avctx, AV_LOG_ERROR, __VA_ARGS__); \
> + ret = ret_value; \
> + goto fail; \
> + }
> +
Nit: Not a fan of these error handling macros personally.
(Also I don’t think you should log for ENOMEM)
> +#define AMF_TIME_BASE_Q (AVRational){1, AMF_SECOND}
> +
> +
> +#endif /* AVUTIL_HWCONTEXT_AMF_INTERNAL_H */
> \ No newline at end of file
> diff --git a/libavutil/hwcontext_internal.h b/libavutil/hwcontext_internal.h
> index e32b786238..db23579c9e 100644
> --- a/libavutil/hwcontext_internal.h
> +++ b/libavutil/hwcontext_internal.h
> @@ -163,5 +163,6 @@ extern const HWContextType ff_hwcontext_type_vdpau;
> extern const HWContextType ff_hwcontext_type_videotoolbox;
> extern const HWContextType ff_hwcontext_type_mediacodec;
> extern const HWContextType ff_hwcontext_type_vulkan;
> +extern const HWContextType ff_hwcontext_type_amf;
>
> #endif /* AVUTIL_HWCONTEXT_INTERNAL_H */
> diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c
> index 2148bdd5fb..6fe83cd16b 100644
> --- a/libavutil/pixdesc.c
> +++ b/libavutil/pixdesc.c
> @@ -2133,6 +2133,10 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = {
> .name = "cuda",
> .flags = AV_PIX_FMT_FLAG_HWACCEL,
> },
> + [AV_PIX_FMT_AMF_SURFACE] = {
> + .name = "amf",
> + .flags = AV_PIX_FMT_FLAG_HWACCEL,
> + },
> [AV_PIX_FMT_VYU444] = {
> .name = "vyu444",
> .nb_components = 3,
> diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h
> index b0b64b2766..a64d40ad07 100644
> --- a/libavutil/pixfmt.h
> +++ b/libavutil/pixfmt.h
> @@ -471,6 +471,11 @@ enum AVPixelFormat {
> AV_PIX_FMT_GRAYF16BE, ///< IEEE-754 half precision Y, 16bpp, big-endian
> AV_PIX_FMT_GRAYF16LE, ///< IEEE-754 half precision Y, 16bpp, little-endian
>
> + /**
> + * HW acceleration through AMF. data[0] contain AMFSurface pointer
> + */
> + AV_PIX_FMT_AMF_SURFACE,
> +
> AV_PIX_FMT_NB ///< number of pixel formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions
> };
>
> --
> 2.47.1.windows.1
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
next prev parent reply other threads:[~2025-01-22 21:49 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-01-22 19:32 Dmitrii Ovchinnikov
2025-01-22 19:32 ` [FFmpeg-devel] [PATCH v6 2/5] avcodec: add amfdec Dmitrii Ovchinnikov
2025-01-22 19:32 ` [FFmpeg-devel] [PATCH v6 3/5] avfilter/scale_amf: Add AMF VPP & super resolution filters Dmitrii Ovchinnikov
2025-01-22 19:32 ` [FFmpeg-devel] [PATCH v6 4/5] doc/filters: Add documentation for AMF filters Dmitrii Ovchinnikov
2025-01-22 19:32 ` [FFmpeg-devel] [PATCH v6 5/5] avcodec/amfenc: redesign to use hwcontext_amf Dmitrii Ovchinnikov
2025-01-22 21:49 ` epirat07 [this message]
2025-01-27 12:18 ` [FFmpeg-devel] [PATCH v6 1/5] avutil: add hwcontext_amf Dmitrii Ovchinnikov
2025-01-29 18:42 ` Dmitrii Ovchinnikov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=D7D0123A-E8DA-4233-A63E-DA0BD7F3C6CA@gmail.com \
--to=epirat07@gmail.com \
--cc=ffmpeg-devel@ffmpeg.org \
--cc=ovchinnikov.dmitrii@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
This inbox may be cloned and mirrored by anyone:
git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git
# If you have public-inbox 1.1+ installed, you may
# initialize and index your mirror using the following commands:
public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
ffmpegdev@gitmailbox.com
public-inbox-index ffmpegdev
Example config snippet for mirrors.
AGPL code for this site: git clone https://public-inbox.org/public-inbox.git