From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 548DB48C5F for ; Thu, 22 Feb 2024 19:50:18 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0F73068BDBB; Thu, 22 Feb 2024 21:50:16 +0200 (EET) Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 666066801E2 for ; Thu, 22 Feb 2024 21:50:09 +0200 (EET) Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-412767742a0so673555e9.3 for ; Thu, 22 Feb 2024 11:50:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20230601.gappssmtp.com; s=20230601; t=1708631409; x=1709236209; darn=ffmpeg.org; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=kn0Xc6aIVVOf3TFEEqC4CzgqQFKeyUbWmXkQQVIOEK4=; b=TwOgQDO8f7675Ko2X2XRGaizJyNACzjiaGBgEyGyLfhG8DRG1h5OLuxo4DROszXrQM y4e3iaMMCzyEY0up5jjKds2J6KiGjfxnEpsLh0RVgkccrs859X0HqHIKw+xVbF6RNLKp fFgG58taqEJqwTcAUHa3IRUAaebTxgiRtFKUelco+z+NmEXFBZ9az6W7efIrGN05ZnoO eRR5BZJebTm1GvUCqsh8opujECA1IkHxm6jeu4WQQmKsjPiVt0NVHhoK0DHZFSHKwrcL QOdooN55zqaXocrWvM3f4HHaccIAl4F1sXQyC7QWjOhcXeNF23SCTAoh7JiewNB7p0kX Migw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708631409; x=1709236209; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=kn0Xc6aIVVOf3TFEEqC4CzgqQFKeyUbWmXkQQVIOEK4=; b=pSbw4NH31+ac4u/nFl3eVOwqBJybPbIWvaQf5uuGWc1ft5BBNjb6ZBVjWOyzAtPoVN AXZFwkB+Oy/nTIrW7WVrPfNUU2lLQnLV6927X54dMxYZYutZE+pvpYp25+/Wzq466U4v JJkENiYY8Yzg47DdyOoqVG+UvLBri3v5JxhS/GCczJFJ+l5BIOLWjijwHlRLJDN0Ganu 6MrlTdWZhELGMLRNdDJJKeuc9cMBHdaQO3zP5x40pA3DO8sDuTpOZJnpZjqNuzM7gJa0 YR4avuYpPu6epfgMEjTJrm71GwbRbGs0zrKN7ZjNFPAn1pfwUFjtP+p2noWx44tF0CMH axQg== X-Gm-Message-State: AOJu0YyIrZPZhDUsBHqcoidCH0qQzZ8tiWLKZl5YMT7QaudduNZY7SWO I3pKMMyztglxqgQ2Xcdk174XsDgYIhnU+mWkZlswx33ZfiXeUtmqMorj9NlcHKfsMKhzAcQRJ/7 p X-Google-Smtp-Source: AGHT+IEq/tF70REceszeqOPfe2NP+XMVZHsjdloxtapNZHU8S3nAszJyZzR/XGu2rpkogd8fBdxK2g== X-Received: by 2002:adf:e50c:0:b0:33d:fb3:9021 with SMTP id j12-20020adfe50c000000b0033d0fb39021mr109646wrm.54.1708631408597; Thu, 22 Feb 2024 11:50:08 -0800 (PST) Received: from [192.168.0.15] (cpc92302-cmbg19-2-0-cust1183.5-4.cable.virginm.net. [82.1.212.160]) by smtp.gmail.com with ESMTPSA id bv14-20020a0560001f0e00b0033d70dd0e04sm61271wrb.8.2024.02.22.11.50.08 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 22 Feb 2024 11:50:08 -0800 (PST) Message-ID: Date: Thu, 22 Feb 2024 19:50:36 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: ffmpeg-devel@ffmpeg.org References: <20240222184603.13074-1-gnattuoc@me.com> <175469a3-87ad-4dd7-905a-f5e0d169b2c2@jkqxz.net> From: Mark Thompson In-Reply-To: Subject: Re: [FFmpeg-devel] [PATCH] hwcontext_videotoolbox: add vt_device_derive X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On 22/02/2024 19:39, ChenLiucheng via ffmpeg-devel wrote: >> On Feb 23, 2024, at 03:28, Mark Thompson wrote: >> >> On 22/02/2024 18:46, gnattu via ffmpeg-devel wrote: >>> There is no device context to be setup, nor devices to be >>> selected with VideoToolbox. Just a simple return would allow >>> us to use derived device in filters like >>> `hwupload=derive_device=videotoolbox` >>> Signed-off-by: Gnattu OC >>> --- >>> libavutil/hwcontext_videotoolbox.c | 9 +++++++++ >>> 1 file changed, 9 insertions(+) >>> diff --git a/libavutil/hwcontext_videotoolbox.c b/libavutil/hwcontext_videotoolbox.c >>> index fe469dc161..d13199eca7 100644 >>> --- a/libavutil/hwcontext_videotoolbox.c >>> +++ b/libavutil/hwcontext_videotoolbox.c >>> @@ -759,6 +759,14 @@ static int vt_device_create(AVHWDeviceContext *ctx, const char *device, >>> return 0; >>> } >>> +static int vt_device_derive(AVHWDeviceContext *device_ctx, >>> + AVHWDeviceContext *src_ctx, AVDictionary *opts, >>> + int flags) >>> +{ >>> + // There is no context to be setup with VT, just return. >>> + return 0; >>> +} >>> + >>> const HWContextType ff_hwcontext_type_videotoolbox = { >>> .type = AV_HWDEVICE_TYPE_VIDEOTOOLBOX, >>> .name = "videotoolbox", >>> @@ -766,6 +774,7 @@ const HWContextType ff_hwcontext_type_videotoolbox = { >>> .frames_priv_size = sizeof(VTFramesContext), >>> .device_create = vt_device_create, >>> + .device_derive = vt_device_derive, >>> .frames_hwctx_size = sizeof(AVVTFramesContext), >>> .frames_init = vt_frames_init, >>> .frames_get_buffer = vt_get_buffer, >> >> This derivation behaviour doesn't make any sense inside libavutil. Features which are only for the ffmpeg utility should be implemented inside the ffmpeg utility. >> >> (Also, try -init_hw_device.) >> > Well this does make sense. > > If only one type of hardware device is in use, then `-init_hw_device` will suffice. However, if we use hardware filters with different device types, such as OpenCL, and we want to switch to a VideoToolbox filter later in the chain, we will have a issue. Since there is no `hwmap` between OpenCL and VideoToolbox, we need to perform a `hwdownload` followed by a `hwupload`. If the `derive_device` option cannot be used with `hwupload` due to `device_derive` not being implemented, the filter will struggle to find the correct device for uploading. It may attempt to upload to the OpenCL device without `device_derive` set if it comes after an OpenCL filter. > Yes, it is unfortunate that the ffmpeg utility is missing this feature. A libavfilter API user will set the correct hw_device_ctx on each filter and your use-case will work properly, but the ffmpeg utility does not fully expose the functionality of libavfilter to be able to do this. This is mostly for syntax reasons; implementation would be straightforward. If you can think of a good way to represent this in options to the ffmpeg utility (which doesn't become too horrible with escaping) then I would very much welcome suggestions. Thanks, - Mark _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".