From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 2D05240BD6 for ; Tue, 28 Dec 2021 19:04:17 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9D93D68AF86; Tue, 28 Dec 2021 21:04:14 +0200 (EET) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11olkn2031.outbound.protection.outlook.com [40.92.19.31]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6173568A630 for ; Tue, 28 Dec 2021 21:04:07 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kmoBXNV8J+vEugUFHh+R24eLmDsSCPwfOFrkfesxPfzxzQjZHiLqhcWH5G+q3DjtkhtZh/AN2dZqoRjPOLgQyYYPitSj9OCLPTFBmSnmpD6CXV4QvXoe06g6ZzrA/5SXk5VQ7NGWdxosNDNc0KWV388q72yDx1brCG+hpVnNhKgzbMaNlPUSrRHXTmb8dw6ek60TuV9zbURmvFG481QYlPdJonebFEImPp5xl6nyEcYnMYyXg6owKt6YLI7TgrloqQR5kyU+DDgseSZbbJ5Cj+zZ+PcYaSCS914R5evjGa62MfHD4UcmeWbt4Ir7XMvZtRShzmYv0cV7hqppLXWpSw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ain1MT3lvRJhX/CZVI/NCfol/dLtRDsAk3HfVIK/duA=; b=NdOziIf09QzBZE615fqLHoa7Uetugl3Alt0at6gZgA2QQe66HWN3lvGjfdwvfvMN2wvZbWOT37dbRYzCM/6t8XYmqNql8qXgZUyorXQNdo4c7j9P5Ri9KqensRGN26nZjqcAV+jtKhlGx7zkw6GEAFqjzTq9KQoKfsH+jRxkkPHpfYlpo6VVAtodYc5C1Bz42XVfU0ojW/eWp4JD5bso747UVBCNYQUnXT2YPTkomf2SxVm3RtjqTONjXVw53HkJyP+tFj5kB1OPuAyV1o0/mfq0Noa9DpvCeKDypzFaf8zUvSK4uB4rzAXKQ7QdtYVpfRaQfKTUTKi26oVHeQD+aQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ain1MT3lvRJhX/CZVI/NCfol/dLtRDsAk3HfVIK/duA=; b=gMNBrJzmaNnOyeOgbTHFxDI/qrDkUjlB8OPKNK+XnK9m8PE3pt4SPc+sb7zXVo0dZMR3YO5Rm41oNDeA1jr2qXbJLnT1hIWZbza3vkLv77uAbiUlNWuLfD6gGx5Uw6BJuJAw2gygye/GT5j4SNN7NmZLx/84bTjdDbAoG25htTxu28bqNomD1aVoD+bw5D2gQXzxps9sPD4YooefLxdTFN7ZdyYMbuLzkR4YHww3DX94Xftxxr9EuWEI7yZkdZIL48Eb7P167LfZwvm2naBk8rQpi3QMLVWPHZOtYpdzk4rQa1xmuMTrXMvN9WxMloIqgALZV+CyduLN1dpJGT1svQ== Received: from BN0P223MB0358.NAMP223.PROD.OUTLOOK.COM (2603:10b6:408:145::16) by BN0P223MB0359.NAMP223.PROD.OUTLOOK.COM (2603:10b6:408:146::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4823.19; Tue, 28 Dec 2021 19:04:02 +0000 Received: from BN0P223MB0358.NAMP223.PROD.OUTLOOK.COM ([fe80::684f:3b62:947a:8492]) by BN0P223MB0358.NAMP223.PROD.OUTLOOK.COM ([fe80::684f:3b62:947a:8492%4]) with mapi id 15.20.4823.023; Tue, 28 Dec 2021 19:04:02 +0000 From: Soft Works To: FFmpeg development discussions and patches Thread-Topic: [FFmpeg-devel] [PATCH 3/3] libavutil/hwcontext_opencl: fix a bug for mapping qsv frame to opencl Thread-Index: AQHX2sLbRTyIEmFJc0+hIglvJzmE7KxG7+iAgAAOPbCAAEQcgIAAFGUggADHqgCAAGLWYA== Date: Tue, 28 Dec 2021 19:04:02 +0000 Message-ID: References: <20211116081623.3081766-1-wenbin.chen@intel.com> <20211116081623.3081766-3-wenbin.chen@intel.com> <92735cd0-179b-cbf6-b191-e440be779b66@jkqxz.net> <96dc30bc-2ff0-009b-713c-b41c248d6b6b@jkqxz.net> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tmn: [+5oV/1DDcGkLgJQrp2Gg493f8eytjqKw] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 57b991cf-04b3-4c2d-6e55-08d9ca34d019 x-ms-traffictypediagnostic: BN0P223MB0359:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: F13Yfz7OFCh3H1fxd6Xq/EijyCU3AecTCfAcPxg2akPQefNrO429XC5bzveTqTaw34ihcMpKYNwtMDSLMhf9QW1X9bwSufYArmQSv4pwyp+M8JC6vxQPK2MYj9JKhxTJ/YGgP8UiQUu7SELY2bYZTy5KMctpEI3lVe2W5iz0vKPxZrRhsNWz42L7sh9lcxqwPrzFU1Dq8WEID4JGyq03O+5FLK3h/YbPSmz9fzyc2jdxEau1RRWRRU/qDmApAHPgYOgAUiXFUfdLW6QToWcMKwe+/FBjYy71deRD7vL/D5crVizXHOqbELRJvXXGc0Yl7D5MFJS+fSG7ZP33n4lRVbkAUfdg2C9tgLECUMwjypmZZxU9Kxle4BH9R/73b8KlgG9LthxnorvUe93N0x/D3DqyPbcnboo/dUf++suhtv0lk+7Q2781I94JIl3dkn5AENO3PvNeAz70TCgchn+6xxVMdR7Ct54GY1hAAQpcCOsn9oXRV8pdAxEYliiUFS7bb1HycJvR2viF7ejISyWXf7RQO4oAicQYyxPscyHb1p0mOSJV7Qq0ErjoWXzpkHVYAxAmbkYmk91WZVtLVr2yeg== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: vB2Rok6jQdz4N9rBpm8915DZLCpAliWqu/+goGk/5VhIFkYfj6oeTx0F4/kME8eJdBq+380myMJkOEa1gCq/ak66g/j847vr0rio6t/y7MNVjvv2j9UwHTiIEYGVgTTgVKyzes3u6QuRHti2VB2htSOUfoZrXXtDgAGWS+m0i1y+q7hIRLfvldYcKExfcTHr0pqVYRkYUEE0DFoPcwgs5s8jiymg8WVzvvsPdH196BJ9cyId1SeiDQFiGDMP6TnWVrQm1necbkmz3r3TrcPHRoMhkWL+m//a0+BduQZzgz6LXx7b26yqAOZlsMlwPERcsmCdQnplXYLw33Lda7RwXEGKj3x5MK42bwsV9F+OHTH4Uqklq1wYZhYm9HVHhRSwpgAz1lPkQRgp2gblYbBIbNyPu0PfBVBZdNSYS+mR4WkHBc0F7ketT45ofajWIaMI3pHoZPPwjXaYuMTcdpwFVZ1Oi2uuJsQej9UUUmK+eFgxUYG2N5jDy4B8w60+R7lch7AnS/LlAKdi6Py1Jk5/uMxabIK+PlOoV+5Uphg7QIicjum4QvhJvAj1MGGI4iBKm99e+WqoB8x3JR+S7MXxKA7alSMu7r8QnglR+kgeGsIkOzkr6vNNuxcBT6Wyawf9xc3b6Cx+GVLHWAxghcQRbtgUIgxE5ngWJTnPLYFcJ7E0BTwG3Cm1LQfI3SMouK4yz2IQD4zccaATr0ZUYL7vjMCvIc/9u0rHF7E2OmVCvhxVkZAJJI761WZWdBZLVPhgfsbeYv9SUOTub+XQCFI6pNKUYwW3B/kECd7d8nMZSMdhJ4PergXLd4b3UAnK9c+ARWZhr1cz4nk6CZ0dWJdl4mniblonpR8+4kGt1JsmeIdKRj8Ib8/GCTTFrhfUajYeCNa4TeJp3MXpmMVkW3J77c7OX1tsgPlq+4Bjz+4oMABtGGqi3WSNSArKXqGMuXIS MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN0P223MB0358.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 57b991cf-04b3-4c2d-6e55-08d9ca34d019 X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Dec 2021 19:04:02.6199 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN0P223MB0359 Subject: Re: [FFmpeg-devel] [PATCH 3/3] libavutil/hwcontext_opencl: fix a bug for mapping qsv frame to opencl X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: > -----Original Message----- > From: ffmpeg-devel On Behalf Of Mark > Thompson > Sent: Tuesday, December 28, 2021 1:54 PM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH 3/3] libavutil/hwcontext_opencl: fix a bug > for mapping qsv frame to opencl > > On 28/12/2021 01:17, Soft Works wrote: > >> -----Original Message----- > >> From: ffmpeg-devel On Behalf Of Mark > >> Thompson > >> Sent: Tuesday, December 28, 2021 12:46 AM > >> To: ffmpeg-devel@ffmpeg.org > >> Subject: Re: [FFmpeg-devel] [PATCH 3/3] libavutil/hwcontext_opencl: fix a > bug > >> for mapping qsv frame to opencl > >> > >> On 27/12/2021 20:31, Soft Works wrote:>> -----Original Message----- > >>>> From: ffmpeg-devel On Behalf Of Mark > >>>> Thompson > >>>> Sent: Monday, December 27, 2021 7:51 PM > >>>> To: ffmpeg-devel@ffmpeg.org > >>>> Subject: Re: [FFmpeg-devel] [PATCH 3/3] libavutil/hwcontext_opencl: fix > a > >> bug > >>>> for mapping qsv frame to opencl > >>>> > >>>> On 16/11/2021 08:16, Wenbin Chen wrote: > >>>>> From: nyanmisaka > >>>>> > >>>>> mfxHDLPair was added to qsv, so modify qsv->opencl map function as > well. > >>>>> Now the following commandline works: > >>>>> > >>>>> ffmpeg -v verbose -init_hw_device vaapi=va:/dev/dri/renderD128 \ > >>>>> -init_hw_device qsv=qs@va -init_hw_device opencl=ocl@va - > filter_hw_device > >>>> ocl \ > >>>>> -hwaccel qsv -hwaccel_output_format qsv -hwaccel_device qs -c:v > h264_qsv > >> \ > >>>>> -i input.264 -vf > >> "hwmap=derive_device=opencl,format=opencl,avgblur_opencl, > >>>> \ > >>>>> hwmap=derive_device=qsv:reverse=1:extra_hw_frames=32,format=qsv" \ > >>>>> -c:v h264_qsv output.264 > >>>>> > >>>>> Signed-off-by: nyanmisaka > >>>>> Signed-off-by: Wenbin Chen > >>>>> --- > >>>>> libavutil/hwcontext_opencl.c | 3 ++- > >>>>> 1 file changed, 2 insertions(+), 1 deletion(-) > >>>>> > >>>>> diff --git a/libavutil/hwcontext_opencl.c > b/libavutil/hwcontext_opencl.c > >>>>> index 26a3a24593..4b6e74ff6f 100644 > >>>>> --- a/libavutil/hwcontext_opencl.c > >>>>> +++ b/libavutil/hwcontext_opencl.c > >>>>> @@ -2249,7 +2249,8 @@ static int opencl_map_from_qsv(AVHWFramesContext > >>>> *dst_fc, AVFrame *dst, > >>>>> #if CONFIG_LIBMFX > >>>>> if (src->format == AV_PIX_FMT_QSV) { > >>>>> mfxFrameSurface1 *mfx_surface = (mfxFrameSurface1*)src- > >>> data[3]; > >>>>> - va_surface = *(VASurfaceID*)mfx_surface->Data.MemId; > >>>>> + mfxHDLPair *pair = (mfxHDLPair*)mfx_surface->Data.MemId; > >>>>> + va_surface = *(VASurfaceID*)pair->first; > >>>>> } else > >>>>> #endif > >>>>> if (src->format == AV_PIX_FMT_VAAPI) { > >>>> > >>>> Since these frames can be user-supplied, this implies that the user- > facing > >>>> API/ABI for AV_PIX_FMT_QSV has changed. > >>>> > >>>> It looks like this was broken by using HDLPairs when D3D11 was > introduced, > >>>> which silently changed the existing API for DXVA2 and VAAPI as well. > >>>> > >>>> Could someone related to that please document it properly (clearly not > all > >>>> possible valid mfxFrameSurface1s are allowed), and note in APIchanges > when > >>>> the API change happened? > >>> > >>> Hi Mark, > >>> > >>> QSV contexts always need to be backed by a child context, which can be > >> DXVA2, > >>> D3D11VA or VAAPI. You can create a QSV context either by deriving from > one > >> of > >>> those contexts or when create a new QSV context, it automatically creates > >> an > >>> appropriate child context - either implicitly (auto mode) or explicitly, > >> like > >>> the ffmpeg implementation does in most cases. > >> > >> ... or by using the one the user supplies when they create it. > >> > >>> When working with "user-supplied" frames on Linux, you need to create a > >> VAAPI > >>> context with those frames and derive a QSV context from that context. > >>> > >>> There is no way to create or supply QSV frames directly. > >> > >> ??? The ability for the user to set up their own version of these things > is > >> literally the whole point of the split alloc/init API. > >> > >> > >> // Some user stuff involving libmfx - has a D3D or VAAPI backing, but this > >> code doesn't need to care about it. > >> > >> // It has a session and creates some surfaces to use with MemId filled > >> compatible with ffmpeg. > >> user_session = ...; > >> user_surfaces = ...; > >> > >> // No ffmpeg involved before this, now we want to pass these surfaces > we've > >> got into ffmpeg. > >> > >> // Create a device context using the existing session. > >> > >> mfx_ctx = av_hwdevice_ctx_alloc(MFX); > >> > >> dc = mfx_ctx->data; > >> mfx_dc = dc->hwctx; > >> mfx_dc->session = user_session; > >> > >> av_hwdevice_ctx_init(mfx_ctx); > >> > >> // Create a frames context out of the surfaces we've got. > >> > >> mfx_frames = av_hwframes_ctx_alloc(mfx_ctx); > >> > >> fc = mfx_frames->data; > >> fc.pool = user_surfaces.allocator; > >> fc.width = user_surfaces.width; > >> // etc. > >> > >> mfx_fc = fc->hwctx; > >> mfx_fc.surfaces = user_surfaces.array; > >> mfx_fc.nb_surfaces = user_surfaces.count; > >> mfx_fc.frame_type = user_surfaces.memtype; > >> > >> av_hwframe_ctx_init(frames); > >> > >> // Do stuff with frames. > > > > I wouldn't consider an mfxSession as an entity that could or should be > > shared between implementations. IMO, this is not a valid use case. > > A consumer of the mfx API needs to make certain choices regarding > > the usage of the API, one of which is the way how frames are allocated > > and managed. > > This is not something that is meant to be shared between implementations. > > Even inside ffmpeg, we don't use a single mfx session. We use separate > > sessions for decoding, encoding and filtering that are joined together > > via MFXJoinSession. > > When an (ffmpeg-)API consumer is creating its own MFX session and its > > own frame allocator implementation, it shouldn't be and allowed > > scenario to create an ffmpeg hw context using this session. > > The user is also aware of the thread safety rules. I was giving the example > above entirely serially to avoid that, but if they were using libmfx > elsewhere in parallel with the above then indeed they would need a bit more > care (create another session, some sort of locking to ensure serialisation) > when passing it to ffmpeg. > > > This shouldn't be considered a public API of ffmpeg because it doesn't > > make sense to share an mfx session like this. > > So, to clarify, your opinion is that none of hwcontext_qsv.h should be public > API? If it were actually removed (not visible in installed headers) then I > agree that would fix the problem. I think that exposing the mfx session handle is OK in order to allow an API user to interop in a way that you create (and manage) your own session and your own surface allocation, and use the exposes mfx session handle to join the ffmpeg-created (and managed) session. > > Besides that, is there any ffmpeg documentation indicating that > > the memId field of mfxFrameSurface1 could be casted to VASurfaceId? > > It is user-visible API, and it matched the behaviour of the libmfx examples > for D3D9 (identical pointer to IDirect3DSurface9) and VAAPI (a compatible > pointer to a structure with the VASurfaceID as the first member) so doing the > obvious thing worked. Also I think it's OK to expose the array of mfxSurface1 pointers allowing to use them in a opaque way (without making use of and any assumptions about the mfxMemId field). This would allow an API user to interoperate with a joined session, e.g. taking frames from a joined ffmpeg session as input to a user-created session doing custom VPP operations where the output frames are from (and managed by) the user-created session. On the other side, when and API user wants to access and interop with frames directly, then the proper way would be to use derive-from or derive-to (VAAPI, D3D9, D3D11) to get an appropriate frames context to work with. That's how I would see it. What do you think? Thanks, softworkz _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".