From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id D7CE448A05 for ; Thu, 22 Feb 2024 20:20:24 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A144168BFE8; Thu, 22 Feb 2024 22:20:21 +0200 (EET) Received: from mr85p00im-hyfv06021401.me.com (mr85p00im-hyfv06021401.me.com [17.58.23.190]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CC97068BD47 for ; Thu, 22 Feb 2024 22:20:14 +0200 (EET) Received: from smtpclient.apple (mr38p00im-dlb-asmtp-mailmevip.me.com [17.57.152.18]) by mr85p00im-hyfv06021401.me.com (Postfix) with ESMTPSA id A4C3D3038770 for ; Thu, 22 Feb 2024 20:20:11 +0000 (UTC) Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Date: Fri, 23 Feb 2024 04:19:40 +0800 References: <20240222184603.13074-1-gnattuoc@me.com> <175469a3-87ad-4dd7-905a-f5e0d169b2c2@jkqxz.net> To: FFmpeg development discussions and patches In-Reply-To: Message-Id: <7069DAE1-AC50-4785-8A6F-951D79E589A5@me.com> X-Mailer: Apple Mail (2.3774.400.31) X-Proofpoint-ORIG-GUID: 7iGwoAueiumkiUA0tscAhGr_lIcyd1hF X-Proofpoint-GUID: 7iGwoAueiumkiUA0tscAhGr_lIcyd1hF X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-22_15,2024-02-22_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 clxscore=1015 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2308100000 definitions=main-2402220158 Subject: Re: [FFmpeg-devel] [PATCH] hwcontext_videotoolbox: add vt_device_derive X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Gnattu OC via ffmpeg-devel Reply-To: FFmpeg development discussions and patches Cc: Gnattu OC Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Actually, if you examine the `hwcontext_videotoolbox.c` file, you will find that the current function for `vt_device_create` is essentially a no-op: it only returns an error if a user attempts to select a device, otherwise it just returns. This is generally how VideoToolbox works. If the `create_device` function is already a no-op, then the derive operation will certainly also be a no-op. All other hardware filters can utilize the derive API, and there are use cases that depend on this API. Thus, it seems inappropriate to me that VideoToolbox users cannot use the derive API just because VideoToolbox does not require any device creation. > A libavfilter API user will set the correct hw_device_ctx on each filter and your use-case will work properly Can't this just be the modification of the `hwupload` filter? Instead of derive, the user just set the which initialized hardware to use, as `-init_hw_device` can already be called multiple times and creating alias for each hardware. > On Feb 23, 2024, at 03:50, Mark Thompson wrote: > > On 22/02/2024 19:39, ChenLiucheng via ffmpeg-devel wrote: >>> On Feb 23, 2024, at 03:28, Mark Thompson wrote: >>> >>> On 22/02/2024 18:46, gnattu via ffmpeg-devel wrote: >>>> There is no device context to be setup, nor devices to be >>>> selected with VideoToolbox. Just a simple return would allow >>>> us to use derived device in filters like >>>> `hwupload=derive_device=videotoolbox` >>>> Signed-off-by: Gnattu OC >>>> --- >>>> libavutil/hwcontext_videotoolbox.c | 9 +++++++++ >>>> 1 file changed, 9 insertions(+) >>>> diff --git a/libavutil/hwcontext_videotoolbox.c b/libavutil/hwcontext_videotoolbox.c >>>> index fe469dc161..d13199eca7 100644 >>>> --- a/libavutil/hwcontext_videotoolbox.c >>>> +++ b/libavutil/hwcontext_videotoolbox.c >>>> @@ -759,6 +759,14 @@ static int vt_device_create(AVHWDeviceContext *ctx, const char *device, >>>> return 0; >>>> } >>>> +static int vt_device_derive(AVHWDeviceContext *device_ctx, >>>> + AVHWDeviceContext *src_ctx, AVDictionary *opts, >>>> + int flags) >>>> +{ >>>> + // There is no context to be setup with VT, just return. >>>> + return 0; >>>> +} >>>> + >>>> const HWContextType ff_hwcontext_type_videotoolbox = { >>>> .type = AV_HWDEVICE_TYPE_VIDEOTOOLBOX, >>>> .name = "videotoolbox", >>>> @@ -766,6 +774,7 @@ const HWContextType ff_hwcontext_type_videotoolbox = { >>>> .frames_priv_size = sizeof(VTFramesContext), >>>> .device_create = vt_device_create, >>>> + .device_derive = vt_device_derive, >>>> .frames_hwctx_size = sizeof(AVVTFramesContext), >>>> .frames_init = vt_frames_init, >>>> .frames_get_buffer = vt_get_buffer, >>> >>> This derivation behaviour doesn't make any sense inside libavutil. Features which are only for the ffmpeg utility should be implemented inside the ffmpeg utility. >>> >>> (Also, try -init_hw_device.) >>> > > Well this does make sense. > > > > If only one type of hardware device is in use, then `-init_hw_device` will suffice. However, if we use hardware filters with different device types, such as OpenCL, and we want to switch to a VideoToolbox filter later in the chain, we will have a issue. Since there is no `hwmap` between OpenCL and VideoToolbox, we need to perform a `hwdownload` followed by a `hwupload`. If the `derive_device` option cannot be used with `hwupload` due to `device_derive` not being implemented, the filter will struggle to find the correct device for uploading. It may attempt to upload to the OpenCL device without `device_derive` set if it comes after an OpenCL filter. > > > > Yes, it is unfortunate that the ffmpeg utility is missing this feature. > > A libavfilter API user will set the correct hw_device_ctx on each filter and your use-case will work properly, but the ffmpeg utility does not fully expose the functionality of libavfilter to be able to do this. > > This is mostly for syntax reasons; implementation would be straightforward. If you can think of a good way to represent this in options to the ffmpeg utility (which doesn't become too horrible with escaping) then I would very much welcome suggestions. > > Thanks, > > - Mark > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".