From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 297D242C76 for ; Tue, 3 May 2022 00:11:11 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D5B7468B1AC; Tue, 3 May 2022 03:11:09 +0300 (EEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11olkn2045.outbound.protection.outlook.com [40.92.19.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8570968B26E for ; Tue, 3 May 2022 03:11:02 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=a/frbZ5e7WtB6Y5N2ZFv97hhDB927sb2UVC8yKo4yMyR/XG5V7Z/HWK3GBTtqKxApkyu7XQK6XOKlIlEzTRLDKIF4HkpTKYhVdR1Caswbbsj1u0cKzpXrNfU8bdTuxnYjBq8Bhsw9ezg6fxmKRhjgxVMpCldyNZZrh9QF12tJONxUKPdHESA9tFV18Kvucz4BP0wX3Ko79SVxSN3OgCaRsJVxt6ud7pre8eT6tJ6m3Wbff18os7GwRSGEgHxTHhs3henU73XzVUW7h29ZhW+L6Sduf3uooBmH1UAULsHqMbaQ4G8t73OZFZpNZJTEXHjy+d5JpCiTw0PulBBNPGOlg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ui5PprEoHgTolmcrPx8xkp4jpDeGXl8qr2NyM/O6CWw=; b=GtjlMYTOlyPgO0V2HLAdypwE0GILWhutUHPHGgcPwkpHB4FY48yAUGqp1RKt2UiVgD7SJ3t9kS7Srp2oFg7nJ9RcOYQvEGH4Q+yxm8BGPdp2GeZvxUKhsb7De+Z/mcYMAZJZjKR1InFbeM5mALQehZx0AyO9dc6AH8FEHDxwgMn2HWbrt2/NIdglWjuC+9G9TV9qV4W9MRl4juhlpeVa0UVaENpz2JiYoyotNjiOkmA+pe8HC6Y11y1M7gv+9mRiE7hPnkwD5bxlxtbyaBeJ6Y3LquLyNDRRkgt4NUqtWLyenvqwNbTRI9f6TmwbRWWybTmjCTx7KwoAJiASghtRGg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ui5PprEoHgTolmcrPx8xkp4jpDeGXl8qr2NyM/O6CWw=; b=pCTSwfUpCmT6fhUdX0Cx/novh10W3EIn/a2RVxMcVYZ4+z8bSU/2AWs2d/g90lGOF9oy5Kt9IYrLzv1udKkdAk9wEdWyJA//DepJxzcYo/C8OIRrgjtY5x7VbqaNXrAMWAGbhUvi2pxT/rGOu5T0cXI/e4KCiJRfuc5NjoBJ+I+oLsW45VsYdsO86ndVTSkej2YsTC12SYMQbhqQKU6oHcyco9W34+gF6l7px07aOVDHvFWV3gOxofnv06oaVxV+pj5qjr0uv9FbQUJxpEmPu62BTfvFviAMms6aDj9px/8T7dRH44f826p2uNzFHPXXukCi2Bf+arVVLkjDsyoZSg== Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM (2603:10b6:8:b::20) by DM8P223MB0157.NAMP223.PROD.OUTLOOK.COM (2603:10b6:5:316::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.13; Tue, 3 May 2022 00:11:00 +0000 Received: from DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::7472:6f83:eeb:45e3]) by DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM ([fe80::7472:6f83:eeb:45e3%9]) with mapi id 15.20.5206.013; Tue, 3 May 2022 00:11:00 +0000 From: Soft Works To: FFmpeg development discussions and patches Thread-Topic: [FFmpeg-devel] [PATCH 1/3] avutils/hwcontext: add derive-device function which searches for existing devices in both directions Thread-Index: AQHYXM31okUFSHUnd0mZd8hUGLzrfq0I+66AgAABwGCAAZbXgIAAlKpggAEArACAAAVXcIAAGBsAgAAAnpA= Date: Tue, 3 May 2022 00:11:00 +0000 Message-ID: References: <85ff784b6ff80d4f2e0f724d0b93472e50d2c256.1651349262.git.ffmpegagent@gmail.com> <994ee7a9-ab91-4d13-9538-ed4502c73bd6@jkqxz.net> <33e93072-cde1-437b-bf60-e78250f26692@jkqxz.net> <99c7fc4a-dcc3-91f7-615e-36218564a65b@jkqxz.net> In-Reply-To: <99c7fc4a-dcc3-91f7-615e-36218564a65b@jkqxz.net> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tmn: [DDTtd6m8KhbxXCc6aIrHVijgW9JZDFHf] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e85110b9-726b-4ae1-b102-08da2c996753 x-ms-traffictypediagnostic: DM8P223MB0157:EE_ x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 7B080wjKR9dglOzmrvtX65k0H0frE3UUr18BFbS7L8c4othb1GrjpmxhBzSWXHbqcceX67pe9mizl0ONCqEDcJKhnPTbmkpFSOqAwrDN+ZzRMxFEP+HLgk157ZdaHBz+yx59KdaDqaIbhnsCUswOmv4FXquO5i5uaREig1d50fZ95wjBeqNmXffmqDC9f7QTLFnKMxj9YGGFHKqQE7/FHxJJSQikZILzSlzdh21htzaQrzYgsx9lSkLbcpBA2HV+BBNEwuACjjLJQhMF8zK8eo2M8BHQH2qMMI16AimON/axwAfQgcpvQ5Wf0gOTBNFTR6p9l4RDJcUSJsEMUs1xoSy1l+8uQ6ro4tbxhNd/1dhrmbFqeDOSyAq+p7mALXYySsHUDZ/tRSYfkNQKx8IKvcja9rls1BQwPu+1owB78Au2jT/q9J7//WaOLMf0OqmGTZSdJyx/eEOVNRJ6nQWUlXbGiyoCvi+gGYN5qjmie5Ci8w/rIH8zZLIGow2JzjdPInJg2Q/V44q5J6Z3uKUbxPFzHF+WNZc2O72zzwS3kzg5OxtxxyzJkBQKHzBvq06GS5sg5hOSkZkdAVavG1Vrkw== x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?B?enBsQTdBWjdRZ1UzWDhYdUp6WEhrTmpkWmVKS0x3eDZaQnZoZmc4U0Y2Yzlw?= =?utf-8?B?QXZHSzJMN0FSZGl2WUV2dU9pTFpidTZGbHhmSS9WSzhna25DbGxqNEFocGtJ?= =?utf-8?B?WFdHSFltMmxKUVBsOVVpbFFrbXYxaWMxc3h3VEFUK1BBT1FzSmZ3WC84bEpi?= =?utf-8?B?SzU1MGt1dmVNZGlQcW9aOFNDMXhTcGRWc2E4S2t3OERMM0t2Qlg2enRZZ2dI?= =?utf-8?B?UEFETG9abXB3cjNIRUF4Mk1pc3o1cTN1czZDRFJydGFFMkxlRjNyQW5hTnp1?= =?utf-8?B?cDFJRjFtdzdqNnlqTzFOdFZNeDVuNmdFcHlQN1hoU3BJVGpDQ1N2TGV5V0g1?= =?utf-8?B?cEQ2NHFlN2dQVWMvZmEvZmtLc205c3R2TzkzOXpHQmF0aEU3ZzZPTURGVFFC?= =?utf-8?B?ekdHM1g1N2R5eWROU2FOL3FHT2FsNlRhQzFzQm9CV1FmZkpyT1d2amZuRGxW?= =?utf-8?B?eUl5SDlYN2pxZlhWVHMxbTI2eDZqbHJrNTgzeG5BS3d3eWs0cXllZTM1d0Nx?= =?utf-8?B?N01CUURYN1BmTGJaOEpkSXFhbHBqSGxKL3JiU1lHbC93dlBnejMraVBHQWc3?= =?utf-8?B?NE8yQUhSSzhoelA2YVB5Z3lmS0ZZekNBMHFVa1p4M1V2WDJscUR6ekI2bFBS?= =?utf-8?B?cDJaVnZhWlVudXMvVGd1YnRIUVp3d1FZU3pPTkVIQXk3OUNpd1kwYkcwTWFa?= =?utf-8?B?ZG9NclNsRUFPZjR0aDdXelpncitsYlVkL3JSMTZsODJIbzcrVm5zcEZOU3NW?= =?utf-8?B?TXdKQlhQaVlycW03U3kxRHpvVk9CR1dsam8xMUZRUjgvMWx6Qkkzc1Z1bUpB?= =?utf-8?B?OWYvWG1EQ2FwQVkySDhzNTNMb1NBUWFTWVFMUExjM1lpSWhkKzBxUHBZcTRU?= =?utf-8?B?SC9oc0RXZFBRMSt6RzlmYmJOQzJUS1hJYkloS1FtVUlBblB6eGQ0Rko5T05l?= =?utf-8?B?dFA1VXU3cm1JbWNnMlk4TDJmQ3Y2VzBnSUdaSVBsMFNyYVlDbnA0OUR4bFFk?= =?utf-8?B?dUgySVd2MWloYjRGa0hJZ3NGcmwyTHVQaTU5bkh1NkkxM0FmQUtTYmozTFBQ?= =?utf-8?B?WHhzUy9FbXcrSVdUdXAzMjdsazVTUmt5VFJRSEJKekdLVlIram1sckFRa0xB?= =?utf-8?B?RUV2RXFyUmJVRmVwZHFRRWlUR2Vxakx3cVRHaytqMUhKWHJwSk82cThHL3BW?= =?utf-8?B?SFpKeHRRODFkanhBd1YzSHZKaEM0TDdwS2QzRkVNTE1lTnpZNko4KzFNSVEz?= =?utf-8?B?NGNEMXd2Zk5IM2xYdkxSMzFTeFpCNWtZcWV5Vk8zZEo3ZVhWcHZIS3VrTU82?= =?utf-8?B?UHhqc3c3eFhnRVQzQi9KN3RyR3lpTCtxcDFUSGdrSGVBMGMrdy9qNnRIUkRU?= =?utf-8?B?SjNuVkR0bkhNWjd1K01EMkJtWko3bitGYVJtYmdUVEwwMnJydXpxTWcrc3N6?= =?utf-8?B?eG5DeFE0N2Q3TXhWUUk2ZTZlMFJRNVNjclptbytMd09TVWdNakZlUmxVTGJN?= =?utf-8?B?OXFCREZ4K2VPWkppaTh0NDc5WjJzb1hob0tLeS9Na3NLWEpoY2NTQVRzQ3Nz?= =?utf-8?B?QTE2SU5vamM4Z1VmaXY2OXNRRVN5U1JCYjdMUDFGam44Zmh0UlM4YU54OUdi?= =?utf-8?B?bTh2K1VtSFNYYk5TMkUwTXp3N0d3ekt6WnNTYzJ3bkRzWmh4bEZhdTlneFlH?= =?utf-8?B?R0xYdFFmNHIrYkVmS3RJbUdwdnlFRXR3cW9nQWJWYXI3OExKc093MUREQlJa?= =?utf-8?B?ZVBvR3pXdVZNSHlDUU9zeUFrOHlFcnlQQUJyTVdNaGNlWkw0blB0TE9VT0xZ?= =?utf-8?B?ZVBSTHc3N1BpZW1vNmVYNTJwZjArcU5kR2cwNStVVStxMldodGtXWDM0TlYw?= =?utf-8?B?UkExNWYxU0VCZWVIeWlhemdRL2R3WXQ4SWF6UlVqVkR6dUNKL0J5K3daR0Fn?= =?utf-8?Q?/PQUPIBNU9U=3D?= MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-1ff67.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8P223MB0365.NAMP223.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: e85110b9-726b-4ae1-b102-08da2c996753 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 May 2022 00:11:00.1034 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8P223MB0157 Subject: Re: [FFmpeg-devel] [PATCH 1/3] avutils/hwcontext: add derive-device function which searches for existing devices in both directions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: > -----Original Message----- > From: ffmpeg-devel On Behalf Of Mark > Thompson > Sent: Tuesday, May 3, 2022 1:57 AM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH 1/3] avutils/hwcontext: add derive- > device function which searches for existing devices in both directions > > On 02/05/2022 23:59, Soft Works wrote: > >> -----Original Message----- > >> From: ffmpeg-devel On Behalf Of > Mark > >> Thompson > >> Sent: Tuesday, May 3, 2022 12:12 AM > >> To: ffmpeg-devel@ffmpeg.org > >> Subject: Re: [FFmpeg-devel] [PATCH 1/3] avutils/hwcontext: add > derive- > >> device function which searches for existing devices in both > directions > >> > >> On 02/05/2022 09:14, Soft Works wrote: > >>>> -----Original Message----- > >>>> From: ffmpeg-devel On Behalf Of > >> Mark > >>>> Thompson > >>>> Sent: Monday, May 2, 2022 12:01 AM > >>>> To: ffmpeg-devel@ffmpeg.org > >>>> Subject: Re: [FFmpeg-devel] [PATCH 1/3] avutils/hwcontext: add > >> derive- > >>>> device function which searches for existing devices in both > >> directions > >>> > >>> [..] > >>> > >>>>>> * The thread-safety properties of the hwcontext API have been > >> lost > >>>> - > >>>>>> you can no longer operate on devices independently across > threads > >>>>>> (insofar as the underlying API allows that). > >>>>>> Maybe there is an argument that derivation is something > >> which > >>>>>> should happen early on and therefore documenting it as thread- > >>>> unsafe > >>>>>> is ok, but when hwupload/hwmap can use it inside filtergraphs > >> that > >>>>>> just isn't going to happen (and will be violated in the FFmpeg > >>>> utility > >>>>>> if filters get threaded, as is being worked on). > >>>>> > >>>>> From my understanding there will be a single separate thread > >> which > >>>>> handles all filtergraph operations. > >>>>> I don't think it would even be possible (without massive > changes) > >>>>> to arbitrate filter processing in parallel. > >>>>> But even if this would be implemented: the filtergraph setup > >> (init, > >>>>> uninit, query_formats, etc.) would surely happen on a single > >> thread. > >>>> > >>>> The ffmpeg utility creates filtergraphs dynamically when the > first > >>>> frame is available from their inputs, so I don't see why you > >> wouldn't > >>>> allow multiple of them to be created in parallel in that case. > >>>> > >>>> If you create all devices at the beginning and then give > references > >> to > >>>> them to the various filters which need them (so never manipulate > >>>> devices dynamically within the graph) then it would be ok, but I > >> think > >>>> you've already rejected that approach. > >>> > >>> Please let's not break out of the scope of this patchset. > >>> This patchset is not about re-doing device derivation. The only > >>> small change that it does is that it returns an existing device > >>> instead of creating a new one when such device already exists > >>> in the same(!) chain. > >>> > >>> The change it makes has really nothing to do with threading or > >>> anything around it. > >> > >> The change makes existing thread-safe hwcontext APIs unsafe. That > is > >> definitely not "nothing to do with threading", and has to be > resolved > >> since they can currently be called from contexts which expect > thread- > >> safety (such as making filtergraphs). > > > > As I said, I'm not aware that filtergraph creation would be a > context > > which requires thread-safety, even when considering frames which get > > pushed into a filtergraph (like from a decoder). > > User programs can do whatever they like within the API constraints. > > > Could you please show a command line which causes a situation where > > device_derive is being called from multiple threads? > > Given that the ffmpeg utility is currently entirely single-threaded, > this will only happen once the intended plans for adding threading to > it are complete. As mentioned, I don't think that would be possible easily, and from what I have understood, the plan is to have separate threads for decoding, encoding and filtering but not multiple threads for filtering. > With that, it will be true for something which makes two filtergraphs > and uses derivation in both of them. For example:, this currently > makes two independent VAAPI devices, but equally could reuse the same > device: > > # ffmpeg -y -f kmsgrab -i /dev/dri/card0 -vf > hwmap=derive_device=vaapi,scale_vaapi=format=nv12 -c:v h264_vaapi > out1.mp4 -vf > hwmap=derive_device=vaapi,scale_vaapi=w=1280:h=720:format=nv12 -c:v > h264_vaapi out2.mp4 Well, multi-threading is not an issue right now, and I don't expect it to be then. But on a more practical take: what do you want me to do? Guard that function with a lock? I can do this, no problem. But none of any of the device control function does any synchronization, so why would exactly this - and only this function need synchronization? > >>>>>> * I'm not sure that it is reasonable to ignore options. If an > >>>>>> unrelated component derived a device before you with special > >>>> options, > >>>>>> you might get that device even if you have incompatible > different > >>>>>> options. > >>>>> > >>>>> I understand what you mean, but this is outside the scope of > >>>>> this patchset, because when you would want to do this, it > >>>>> would need to be implemented for derivation in general, not > >>>>> in this patchset which only adds reverse-search to the > >>>>> existing derivation functionality. > >>>> > >>>> I'm not sure what you mean by that? The feature already exists; > >> here > >>>> is a concrete example of where you would get the wrong result: > >>>> > >>>> Start with VAAPI device A. > >>>> > >>>> Component P derives Vulkan device B with some extension options > X. > >>>> > >>>> Component Q wants the same device as P, so it derives again with > >>>> extension options X and gets B. > >>>> > >>>> Everything works fine for a while. > >>>> > >>>> Later, unrelated component R is inserted before P and Q. It > wants > >> a > >>>> Vulkan device C with extension options Y, so it derives that. > >>>> > >>>> Now component Q is broken because it gets C instead of B and has > >> the > >>>> wrong extensions enabled. > >>> > >>> As per your request, this patchset's changes are now limited to > >>> use ffmpeg cli. And there is currently no support for "extension > >>> options" when deriving a device. > >> > >> Yes there is - see the "instance_extensions" and > "device_extensions" > >> options to Vulkan device derivation. (Hence the VAAPI+Vulkan > >> example.) > > > > OK, but when deriving a device via command line, it doesn't consider > > such parameters in the current device_derive function, and hence > it's > > not something that can be burdened onto this patchset. > > Then it sounds like you want this function to be part of the ffmpeg > utility, not inside one of the libraries which have other users. No. I say, matching device parameters when searching for an existing device isn't done right now, and it's outside the scope of this patchset. > >>> What I meant above is this: > >>> > >>> Assume this patchset wouldn't be applied, but a patchset would > >>> be applied that allows to specify "extension options". > >>> Then, even without this patchset, I could construct a similar > >>> example, where you would get the same device when deriving > >>> two times from the same device but with different extension > >>> options. > >> > >> How? > > > > VAAPI >> QSV(Param1) >> OpenCL > > > > Now, from OpenCL, you want to derive QSV but with different > parameters > > (Param2). You won't get a new device, you get the existing QSV > device. > > This doesn't make sense - remember that device derivation is > unidirectional, and OpenCL is a leaf API which can only derive from > other things. The "derive" operation there has to be interpreted as > "return the device this was derived from", in which case options don't > make sense. > > (Indeed, it seems like a good idea to disallow options in that case. > I will prepare a patch to that effect.) OK. > >> No it doesn't? Your new function find_derived_hwdevice_ctx() is > >> called only for the original source device, and recurses into its > >> (transitive) children. It can't return any of X, Y, V or W when > >> starting from C. > > > > OK, that's my mistake. It's been a while... > > What I described is the original behavior. There were reasons > > to limit it this way. > > Well, which one is is actually intended then? This patchset. It is used by us and by Intel for quite a while already. Thanks, softworkz _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".