From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id F1A7646E3F for ; Wed, 14 Feb 2024 20:56:13 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B1E3B68D200; Wed, 14 Feb 2024 22:56:10 +0200 (EET) Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BC23468CD0C for ; Wed, 14 Feb 2024 22:56:04 +0200 (EET) Received: by mail-wr1-f46.google.com with SMTP id ffacd0b85a97d-33cdedb40c4so50485f8f.2 for ; Wed, 14 Feb 2024 12:56:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20230601.gappssmtp.com; s=20230601; t=1707944164; x=1708548964; darn=ffmpeg.org; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=B8eURsP67xySYyQbSqvpdaGPadRmfXvgloB3y5R2BRk=; b=OV1gBUkZH68clgCywL50oNJ4hEa/PIBdAWihCJ/slxdxCIP0pE1lN/dS5aeFNgamzx jmviX7WSN5wYnnZOT78tkXsFT5u/NVjtJUwJuuB8s/1NCPtqP3D4Mtheam1JNlbIEFao k3d9kIQTTwIRkqjlkDg8HaO8MyyGRdVNerUTZrTSPrd0tne1wjZPOcXo8jgelQ/1OXCt 7SSLLQszePBVrmFa76OsheO/epXTbuMymlCGHRCuagDUW19M4A9Ga+i8bmo1wtdjtGms fwcZJo3j6iuhLCJ4ui7EQCkTuXLXeXZZniGMUBypwFXHT1SDljBjlOqFfQXnnCc4gN7V Ea3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707944164; x=1708548964; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=B8eURsP67xySYyQbSqvpdaGPadRmfXvgloB3y5R2BRk=; b=dYUL6B+6uct7iHsA7jxCza2aSIx1Xb9UGWtkf4NpaTeJw451kDys2FGQfDTVZHv01X SgEeJSadWMg9KIHbQ2hiWVz/jhr5Elsy5/hQaFdnzV9MMziq9d/R24UNR3kdRJEq6CZ4 qhV1HEBilYPOS5E8JkJJ3zSBbP6Mshvx5nFp2FhMqowvJLw5ED3kKJlbHWxuCr3TInqP jdtJS/NWcs4FNsyD3jHpuqUQRuUMPkiSrjwswKtrDexx52WWLEpcChdeAStNt5+uLwTG OhdTsDLNSvRK2j0+xvjlAo5JQW5o6rEggoWY18SoUTfygRL7UXUfeB/uvyLSGxMuyXoL gK5Q== X-Gm-Message-State: AOJu0YzsgpRPznUal6SHoSqsTXpuR4ef5T3ID4Wm2MS4Zw9W+Mw9diXY dzWFUqUBEnmJjcZRAi6HPDYVdHOPn/lVWvQuQScGJBlkLxf1BHlqyRQcuYBqBncVM3Xq8NnsZQz 9 X-Google-Smtp-Source: AGHT+IGvHhA/STdrpYlZ2OJOUU7mKVmeLPRAk2D8IJ+1WCzpuHOjWCLqMFYfUju+19/vnP5CpSuHpg== X-Received: by 2002:adf:e6cc:0:b0:33b:10de:59a2 with SMTP id y12-20020adfe6cc000000b0033b10de59a2mr2390661wrm.15.1707944163602; Wed, 14 Feb 2024 12:56:03 -0800 (PST) Received: from [192.168.0.15] (cpc92302-cmbg19-2-0-cust1183.5-4.cable.virginm.net. [82.1.212.160]) by smtp.gmail.com with ESMTPSA id n15-20020a7bcbcf000000b0041076fc2a61sm2912362wmi.5.2024.02.14.12.56.03 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 14 Feb 2024 12:56:03 -0800 (PST) Message-ID: <3fd27a93-d2bd-41f8-8572-ef4ddfddc871@jkqxz.net> Date: Wed, 14 Feb 2024 20:56:28 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: ffmpeg-devel@ffmpeg.org References: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> From: Mark Thompson In-Reply-To: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> Subject: Re: [FFmpeg-devel] [PATCH 1/9] libavutil: add hwcontext_amf. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On 14/02/2024 01:55, Dmitrii Ovchinnikov wrote: > Adds hwcontext_amf, which allows to use shared AMF > context for the encoder, decoder and AMF-based filters, > without copy to the host memory. > It will also allow you to use some optimizations in > the interaction of components (for example, SAV) and make a more > manageable and optimal setup for using GPU devices with AMF > in the case of a fully AMF pipeline. > It will be a significant performance uplift when full AMF pipeline > with filters is used. > > We also plan to add Compression artefact removal filter in near feature. > --- > libavutil/Makefile | 3 + > libavutil/hwcontext.c | 4 + > libavutil/hwcontext.h | 1 + > libavutil/hwcontext_amf.c | 580 +++++++++++++++++++++++++++++++++ > libavutil/hwcontext_amf.h | 105 ++++++ > libavutil/hwcontext_internal.h | 1 + > libavutil/pixdesc.c | 4 + > libavutil/pixfmt.h | 4 + > 8 files changed, 702 insertions(+) > create mode 100644 libavutil/hwcontext_amf.c > create mode 100644 libavutil/hwcontext_amf.h > > ... > diff --git a/libavutil/hwcontext_amf.h b/libavutil/hwcontext_amf.h > new file mode 100644 > index 0000000000..0161b9a29c > --- /dev/null > +++ b/libavutil/hwcontext_amf.h > @@ -0,0 +1,105 @@ > +/* > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > + > +#ifndef AVUTIL_HWCONTEXT_AMF_H > +#define AVUTIL_HWCONTEXT_AMF_H > +#include > +#include > +#include > +#include "pixfmt.h" > + > +#include "libavformat/avformat.h" > +#include "libavutil/hwcontext.h" > + > +#define FFMPEG_AMF_WRITER_ID L"ffmpeg_amf" > + > +typedef struct AmfTraceWriter { > + AMFTraceWriterVtbl *vtbl; > + void *avctx; > + void *avcl; > +} AmfTraceWriter; > + > +typedef struct AVAMFDeviceContextInternal { > + amf_handle library; ///< handle to DLL library > + AMFFactory *factory; ///< pointer to AMF factory > + AMFDebug *debug; ///< pointer to AMF debug interface > + AMFTrace *trace; ///< pointer to AMF trace interface > + > + amf_uint64 version; ///< version of AMF runtime > + AMFContext *context; ///< AMF context > + AMF_MEMORY_TYPE mem_type; > +} AVAMFDeviceContextInternal; > + > +/** > + * This struct is allocated as AVHWDeviceContext.hwctx > + */ > + > +typedef struct AVAMFDeviceContext { > + AVBufferRef *internal; > +} AVAMFDeviceContext; > + > +typedef struct AMFFramesContext { > + AMFSurface * surfaces; > + int nb_surfaces; > +} AMFFramesContext; > + > +/** > +* Error handling helper > +*/ > +#define AMF_RETURN_IF_FALSE(avctx, exp, ret_value, /*message,*/ ...) \ > + if (!(exp)) { \ > + av_log(avctx, AV_LOG_ERROR, __VA_ARGS__); \ > + return ret_value; \ > + } > + > +#define AMF_GOTO_FAIL_IF_FALSE(avctx, exp, ret_value, /*message,*/ ...) \ > + if (!(exp)) { \ > + av_log(avctx, AV_LOG_ERROR, __VA_ARGS__); \ > + ret = ret_value; \ > + goto fail; \ > + } > + > +#define AMF_TIME_BASE_Q (AVRational){1, AMF_SECOND} > + > +typedef struct FormatMap { > + enum AVPixelFormat av_format; > + enum AMF_SURFACE_FORMAT amf_format; > +} FormatMap; > + > +extern const FormatMap format_map[]; > +enum AMF_SURFACE_FORMAT av_amf_av_to_amf_format(enum AVPixelFormat fmt); > +enum AVPixelFormat av_amf_to_av_format(enum AMF_SURFACE_FORMAT fmt); > +extern AmfTraceWriter av_amf_trace_writer; > + > +int av_amf_context_init(AVAMFDeviceContextInternal* internal, void* avcl); > +int av_amf_load_library(AVAMFDeviceContextInternal* internal, void* avcl); > +int av_amf_create_context( AVAMFDeviceContextInternal * internal, > + void* avcl, > + const char *device, > + AVDictionary *opts, int flags); > +int av_amf_context_internal_create(AVAMFDeviceContextInternal * internal, > + void* avcl, > + const char *device, > + AVDictionary *opts, int flags); > +void av_amf_context_internal_free(void *opaque, uint8_t *data); > +int av_amf_context_derive(AVAMFDeviceContextInternal * internal, > + AVHWDeviceContext *child_device_ctx, AVDictionary *opts, > + int flags); > + > +#endif /* AVUTIL_HWCONTEXT_AMF_H */ We need to sort out the content of the public header file before doing anything else. It needs to contain exactly what a user has to see to set up an AMF hwcontext instance: * A device hwctx structure, with notes on how to set the fields to put an existing device into the device context before calling av_hwdevice_ctx_init(). * Notes on what the buffer pool in a frames context contains, which might involve another structure to act as the buffer entry. * Optionally a frame hwctx structure, with notes on how to set the fields when putting a pool of existing frames into the frame context before calling av_hwframe_ctx_init() (omit if it would contain nothing). * Optionally a hwconfig structure, with notes on how to set the fields when requesting the capabilities of some particular operation (omit if all operations have identical capabilities). * If the pixfmt needs to define additional structures, they go here as well. It shouldn't contain anything else unless there are hard ABI requirements (e.g. to allocate some sort of non-fixed structure). > diff --git a/libavutil/hwcontext_internal.h b/libavutil/hwcontext_internal.h > index 4df516ee6a..48d2dc012c 100644 > --- a/libavutil/hwcontext_internal.h > +++ b/libavutil/hwcontext_internal.h > @@ -175,5 +175,6 @@ extern const HWContextType ff_hwcontext_type_vdpau; > extern const HWContextType ff_hwcontext_type_videotoolbox; > extern const HWContextType ff_hwcontext_type_mediacodec; > extern const HWContextType ff_hwcontext_type_vulkan; > +extern const HWContextType ff_hwcontext_type_amf; > > #endif /* AVUTIL_HWCONTEXT_INTERNAL_H */ > diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c > index f6d4d01460..ebc79b0c74 100644 > --- a/libavutil/pixdesc.c > +++ b/libavutil/pixdesc.c > @@ -2125,6 +2125,10 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { > .name = "cuda", > .flags = AV_PIX_FMT_FLAG_HWACCEL, > }, > + [AV_PIX_FMT_AMF] = { > + .name = "amf", > + .flags = AV_PIX_FMT_FLAG_HWACCEL, > + }, > [AV_PIX_FMT_AYUV64LE] = { > .name = "ayuv64le", > .nb_components = 4, > diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h > index 9c87571f49..06459be62c 100644 > --- a/libavutil/pixfmt.h > +++ b/libavutil/pixfmt.h > @@ -251,6 +251,10 @@ enum AVPixelFormat { > * exactly as for system memory frames. > */ > AV_PIX_FMT_CUDA, > + /** > + * HW acceleration through AMF. data[3] contain AMFSurface pointer > + */ > + AV_PIX_FMT_AMF, IMO naming pixel formats to be identical to the API hosting them was a mistake. It's an AMFSurface, call the format AV_PIX_FMT_AMF_SURFACE. Also no reason to copy old formats by putting your pointer in data[3], put it in data[0]. Thanks, - Mark _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".