From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id D04BD40B65 for ; Mon, 27 Dec 2021 23:46:06 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D722768B09E; Tue, 28 Dec 2021 01:46:03 +0200 (EET) Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 96A7568AF8D for ; Tue, 28 Dec 2021 01:45:57 +0200 (EET) Received: by mail-wr1-f41.google.com with SMTP id v7so34863151wrv.12 for ; Mon, 27 Dec 2021 15:45:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20210112.gappssmtp.com; s=20210112; h=message-id:date:mime-version:user-agent:subject:content-language:to :references:from:in-reply-to:content-transfer-encoding; bh=RTIcyarjEIHM+A+/fpjRj6RbxuefRyMPKnW66cspjuI=; b=afh92xl/451v1DEib1uonzfToKUVY3UfZJih0oJz1sZZONCN2ktZvFxqaiEg8F2MJ1 FsohkXPIxfqc/pgN+7gWkK1wneXlkvcEHY/MnxY8uc/OrcrHUEfPLOrOOIDYfRP1jiBE V6ss4MMQxOgv2LuTHFBqVUg8XTzjFmMeeop72+dBWywvqFStu0wfcC6k/XYfvFTf3iif H/A1bQhkw/PNHBJB0fbk5jHiz7WrN0dRw99VMYaG140iPOf1efmqYP2BBjqxjPtEpWEJ tlLzGk3upF7faiL5iJne/HqpNhaMxmy90XOianJLtWcdqD7/gz1Fhwil604FtqwkFSGu R0Iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:references:from:in-reply-to :content-transfer-encoding; bh=RTIcyarjEIHM+A+/fpjRj6RbxuefRyMPKnW66cspjuI=; b=qBTVk1tNSO9FF4K27St07U/9+cdwHPSdy3JCZup1c9sppQmrGGhFvDZDgm75lXqYR3 g9nPbQF2mJc6YuzRH/kzGnNDJK2olXAUxSDxeZk11cgcZlFRTqZkrkB8NZqdf7aYLz8H 0Zr/lsgHrbmYFnBVEooonI3wy67NNLvhifDo32gpeUrouS6HiF3ze3lJZ/jReKH2dhVZ EncJR0hkOzfyCt5qHrkmd+nDWDOOUSpw9Z4CchGLar8uHVZCUMQi07AoQZ9BZ3nbh8k0 fcEySgFNusCxoFQGN6UCpUwe32dFIpe8RmF06SoISgU28HnEnjUJtLqbY9JkhtZlJMdj yQ5g== X-Gm-Message-State: AOAM5304OcxNg8bwdqHLd5XOhIPnujqUoTvlY/mICdsvWqkMfboAvFhm kdBCBUY2hqQhkqyYZRZfaxiCni4YSMtzQA== X-Google-Smtp-Source: ABdhPJzX8XWBX2XoCF7eEd6Wwp7mK9nGDzum1GcGB7pBVt8PVm3o9pGNByJ4qQLFIx9WsTgoJGBb0A== X-Received: by 2002:adf:f252:: with SMTP id b18mr14413679wrp.341.1640648756860; Mon, 27 Dec 2021 15:45:56 -0800 (PST) Received: from [192.168.0.3] (cpc91224-cmbg18-2-0-cust201.5-4.cable.virginm.net. [81.106.228.202]) by smtp.gmail.com with ESMTPSA id w11sm16674830wrn.65.2021.12.27.15.45.56 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 27 Dec 2021 15:45:56 -0800 (PST) Message-ID: <96dc30bc-2ff0-009b-713c-b41c248d6b6b@jkqxz.net> Date: Mon, 27 Dec 2021 23:45:55 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.4.1 Content-Language: en-US To: ffmpeg-devel@ffmpeg.org References: <20211116081623.3081766-1-wenbin.chen@intel.com> <20211116081623.3081766-3-wenbin.chen@intel.com> <92735cd0-179b-cbf6-b191-e440be779b66@jkqxz.net> From: Mark Thompson In-Reply-To: Subject: Re: [FFmpeg-devel] [PATCH 3/3] libavutil/hwcontext_opencl: fix a bug for mapping qsv frame to opencl X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On 27/12/2021 20:31, Soft Works wrote:>> -----Original Message----- >> From: ffmpeg-devel On Behalf Of Mark >> Thompson >> Sent: Monday, December 27, 2021 7:51 PM >> To: ffmpeg-devel@ffmpeg.org >> Subject: Re: [FFmpeg-devel] [PATCH 3/3] libavutil/hwcontext_opencl: fix a bug >> for mapping qsv frame to opencl >> >> On 16/11/2021 08:16, Wenbin Chen wrote: >>> From: nyanmisaka >>> >>> mfxHDLPair was added to qsv, so modify qsv->opencl map function as well. >>> Now the following commandline works: >>> >>> ffmpeg -v verbose -init_hw_device vaapi=va:/dev/dri/renderD128 \ >>> -init_hw_device qsv=qs@va -init_hw_device opencl=ocl@va -filter_hw_device >> ocl \ >>> -hwaccel qsv -hwaccel_output_format qsv -hwaccel_device qs -c:v h264_qsv \ >>> -i input.264 -vf "hwmap=derive_device=opencl,format=opencl,avgblur_opencl, >> \ >>> hwmap=derive_device=qsv:reverse=1:extra_hw_frames=32,format=qsv" \ >>> -c:v h264_qsv output.264 >>> >>> Signed-off-by: nyanmisaka >>> Signed-off-by: Wenbin Chen >>> --- >>> libavutil/hwcontext_opencl.c | 3 ++- >>> 1 file changed, 2 insertions(+), 1 deletion(-) >>> >>> diff --git a/libavutil/hwcontext_opencl.c b/libavutil/hwcontext_opencl.c >>> index 26a3a24593..4b6e74ff6f 100644 >>> --- a/libavutil/hwcontext_opencl.c >>> +++ b/libavutil/hwcontext_opencl.c >>> @@ -2249,7 +2249,8 @@ static int opencl_map_from_qsv(AVHWFramesContext >> *dst_fc, AVFrame *dst, >>> #if CONFIG_LIBMFX >>> if (src->format == AV_PIX_FMT_QSV) { >>> mfxFrameSurface1 *mfx_surface = (mfxFrameSurface1*)src->data[3]; >>> - va_surface = *(VASurfaceID*)mfx_surface->Data.MemId; >>> + mfxHDLPair *pair = (mfxHDLPair*)mfx_surface->Data.MemId; >>> + va_surface = *(VASurfaceID*)pair->first; >>> } else >>> #endif >>> if (src->format == AV_PIX_FMT_VAAPI) { >> >> Since these frames can be user-supplied, this implies that the user-facing >> API/ABI for AV_PIX_FMT_QSV has changed. >> >> It looks like this was broken by using HDLPairs when D3D11 was introduced, >> which silently changed the existing API for DXVA2 and VAAPI as well. >> >> Could someone related to that please document it properly (clearly not all >> possible valid mfxFrameSurface1s are allowed), and note in APIchanges when >> the API change happened? > > Hi Mark, > > QSV contexts always need to be backed by a child context, which can be DXVA2, > D3D11VA or VAAPI. You can create a QSV context either by deriving from one of > those contexts or when create a new QSV context, it automatically creates an > appropriate child context - either implicitly (auto mode) or explicitly, like > the ffmpeg implementation does in most cases. ... or by using the one the user supplies when they create it. > When working with "user-supplied" frames on Linux, you need to create a VAAPI > context with those frames and derive a QSV context from that context. > > There is no way to create or supply QSV frames directly. ??? The ability for the user to set up their own version of these things is literally the whole point of the split alloc/init API. // Some user stuff involving libmfx - has a D3D or VAAPI backing, but this code doesn't need to care about it. // It has a session and creates some surfaces to use with MemId filled compatible with ffmpeg. user_session = ...; user_surfaces = ...; // No ffmpeg involved before this, now we want to pass these surfaces we've got into ffmpeg. // Create a device context using the existing session. mfx_ctx = av_hwdevice_ctx_alloc(MFX); dc = mfx_ctx->data; mfx_dc = dc->hwctx; mfx_dc->session = user_session; av_hwdevice_ctx_init(mfx_ctx); // Create a frames context out of the surfaces we've got. mfx_frames = av_hwframes_ctx_alloc(mfx_ctx); fc = mfx_frames->data; fc.pool = user_surfaces.allocator; fc.width = user_surfaces.width; // etc. mfx_fc = fc->hwctx; mfx_fc.surfaces = user_surfaces.array; mfx_fc.nb_surfaces = user_surfaces.count; mfx_fc.frame_type = user_surfaces.memtype; av_hwframe_ctx_init(frames); // Do stuff with frames. > Looking at the code: > >> *mfx_surface = (mfxFrameSurface1*)src->data[3]; > > A QSV frames context is using the mfxFrameSurface1 structure for describing > the individual frames and mfxFrameSurface1 can only come from the MSDK runtime, > it cannot be user-supplied. > > I don't think that there's something that needs to be documented because > whatever user-side manipulation an API consumer would want to perform, it would > always need to derive the context either from QSV to D3D/VAAPI or from D3D to > VAAPI in order to access and manipulate individual frames. - Mark _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".