From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 12DB5479C8 for ; Sun, 29 Oct 2023 02:31:32 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0235868CB31; Sun, 29 Oct 2023 04:31:29 +0200 (EET) Received: from out162-62-57-137.mail.qq.com (out162-62-57-137.mail.qq.com [162.62.57.137]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8692868C5BA for ; Sun, 29 Oct 2023 04:31:21 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1698546678; bh=d2UBZiOBbkACYUxUGCbPrP8Wzf8xQmH6H+Om6LLbon0=; h=Date:Subject:To:References:From:In-Reply-To; b=OhIfV+nq5X4G5dkmH6x48yzW8fqV3fmgz+eRjByMdJRqRLMUZrBwgoLyJ6uP0ZeJH eBDEFb7iGg5rdRI74jMxqDLeDesJEX0YRKQ5K8gNuuE/AjsMIS/LA+86saa7izijM3 CyqEyt2zVgWK1nsYQSg9iAVNBmnclB5YZs5QAV2A= Received: from [192.168.2.12] ([113.118.105.197]) by newxmesmtplogicsvrszb1-0.qq.com (NewEsmtp) with SMTP id 7D110217; Sun, 29 Oct 2023 10:31:17 +0800 X-QQ-mid: xmsmtpt1698546677tr3pnfd1c Message-ID: X-QQ-XMAILINFO: NiDupExshEc7sg/RwLzegGOEImZiyYP1fkyXv+Z7G0ZmVpjdS6hdihuosw0ud7 ca25tqWOsrPgrl2ekv7OFNupRNMPVUI0LWCgW9SMjOo4Ey6t9SDayA9Iq1emF+aOAd4S2yFxOypc klz7Ikp1eD65P4/Kic19k8UhgsnlG+aXwlADjghsbdkOtBYtqK7u6dJ/Y08R0lsuc+JAKwtpGpyd QaWjE44LI/1DoO/7Uf+xDcrzD+frlKQDi3CLr0jTJGq4RTMUfvHZySaTSc+PQfZ7LzEO1JiC6CIa Ffw2vqYOjZVzdCwSbbYCC0P5/Pb/KVdv8dARWcMHHMSVl4GhlizcU1hrmuMixnIcbisizXRrEc0I ZHP3i7fjWTnQfQcGiyy8461zCjnLn7BAyjGQqmtJltzqGWYosHSle6obFq04vEGMnIArT7J3h4E7 x4wTce1Fk0uOxE/Y4nwADNx0o8siwkAjtz2fMB/c3CuysnoucaZD67l0QAfFnc5Lr6fGfj2lVX1n SsewO8JferYHkf0H5T+/uxFK9todPVwUWD7I2S2wpg6Eu9xl+h3BRMfHPj3ZK1Zt0i6+V0PuEj+O HT82En2gDtSAVuUsV7krS3iwSWlBeH0K490wHy7kbv8Eo3B84MGP67ad2A4A3a8Rln2I0Bfg8NJp ZxOr3HOPaBQtN8mzsAZuAuYNHcL3kg0qt/JS+cyYS85fG/lFmaEoUkPZHCDhOMCWR5FrXIH58kt7 oYHffwq+YU9knxGX3jMpFmlPdho8GhKavGlnoD+E2tCks7JeXt6RCbX2kFnvhx56yXeXeK38luQl Z6eW2E9DBhLZUUZfdhi3aPkklxjPsdyf4FktD0YB9rnca9LINxjHKMbeMEbj+NUXYjUS16s3jmZ3 Y7xiVyWDfadXmh6fz1wG02JKpdyigEtzptDplTcuEOFcjwcAhd7Kfkue1VglamJbiOQc4TonIw6r IIGm5fs205GtQfn6fgQCXM0zQHLiLvL5WL6CHvrbMEQ5sEQfYltfs5qyJQPr7DZ+YV9v3epo8Ufg nJ5iy9rNy/QSF2EVps5l1Are8YGq8= X-QQ-XMRINFO: NI4Ajvh11aEj8Xl/2s1/T8w= X-OQ-MSGID: Date: Sun, 29 Oct 2023 10:31:17 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: ffmpeg-devel@ffmpeg.org References: <20231028181341.13090-1-quinkblack@foxmail.com> Content-Language: en-US From: Zhao Zhili In-Reply-To: X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [FFmpeg-devel] [PATCH v3 2/2] fftools/ffplay: add hwaccel decoding support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On 2023/10/29 03:32, Lynne wrote: > Oct 28, 2023, 20:14 byquinkblack@foxmail.com: > >> From: Zhao Zhili >> >> --- >> v3: shared vulkan instance between libplacebo and hwcontext >> >> fftools/ffplay.c | 43 ++++++ >> fftools/ffplay_renderer.c | 316 +++++++++++++++++++++++++++++++++++++- >> fftools/ffplay_renderer.h | 2 + >> 3 files changed, 355 insertions(+), 6 deletions(-) >> >> diff --git a/fftools/ffplay.c b/fftools/ffplay.c >> index 305d72d8b8..29b37a5e46 100644 >> --- a/fftools/ffplay.c >> +++ b/fftools/ffplay.c >> @@ -352,6 +352,7 @@ static int autorotate = 1; >> static int find_stream_info = 1; >> static int filter_nbthreads = 0; >> static int enable_vulkan = 0; >> +static const char *hwaccel = NULL; >> >> /* current context */ >> static int is_full_screen; >> @@ -2557,6 +2558,37 @@ static int audio_open(void *opaque, AVChannelLayout *wanted_channel_layout, int >> return spec.size; >> } >> >> +static int create_hwaccel(AVBufferRef **device_ctx) >> +{ >> + enum AVHWDeviceType type; >> + int ret; >> + AVBufferRef *vk_dev; >> + >> + *device_ctx = NULL; >> + >> + if (!hwaccel) >> + return 0; >> + >> + type = av_hwdevice_find_type_by_name(hwaccel); >> + if (type == AV_HWDEVICE_TYPE_NONE) >> + return AVERROR(ENOTSUP); >> + >> + ret = vk_renderer_get_hw_dev(vk_renderer, &vk_dev); >> + if (ret < 0) >> + return ret; >> + >> + ret = av_hwdevice_ctx_create_derived(device_ctx, type, vk_dev, 0); >> + if (!ret) >> + return 0; >> + >> + if (ret != AVERROR(ENOSYS)) >> + return ret; >> + >> + av_log(NULL, AV_LOG_WARNING, "Derive %s from vulkan not supported.\n", hwaccel); >> + ret = av_hwdevice_ctx_create(device_ctx, type, NULL, NULL, 0); >> + return ret; >> +} >> + >> /* open a given stream. Return 0 if OK */ >> static int stream_component_open(VideoState *is, int stream_index) >> { >> @@ -2624,6 +2656,12 @@ static int stream_component_open(VideoState *is, int stream_index) >> >> av_dict_set(&opts, "flags", "+copy_opaque", AV_DICT_MULTIKEY); >> >> + if (avctx->codec_type == AVMEDIA_TYPE_VIDEO) { >> + ret = create_hwaccel(&avctx->hw_device_ctx); >> + if (ret < 0) >> + goto fail; >> + } >> + >> if ((ret = avcodec_open2(avctx, codec, &opts)) < 0) { >> goto fail; >> } >> @@ -3625,6 +3663,7 @@ static const OptionDef options[] = { >> "read and decode the streams to fill missing information with heuristics" }, >> { "filter_threads", HAS_ARG | OPT_INT | OPT_EXPERT, { &filter_nbthreads }, "number of filter threads per graph" }, >> { "enable_vulkan", OPT_BOOL, { &enable_vulkan }, "enable vulkan render" }, >> + { "hwaccel", HAS_ARG | OPT_STRING | OPT_EXPERT, { &hwaccel }, "use HW accelerated decoding" }, >> { NULL, }, >> }; >> >> @@ -3739,6 +3778,10 @@ int main(int argc, char **argv) >> #ifdef SDL_HINT_VIDEO_X11_NET_WM_BYPASS_COMPOSITOR >> SDL_SetHint(SDL_HINT_VIDEO_X11_NET_WM_BYPASS_COMPOSITOR, "0"); >> #endif >> + if (hwaccel && !enable_vulkan) { >> + av_log(NULL, AV_LOG_INFO, "Enable vulkan renderer to support hwaccel %s\n", hwaccel); >> + enable_vulkan = 1; >> + } >> if (enable_vulkan) { >> vk_renderer = vk_get_renderer(); >> if (vk_renderer) { >> diff --git a/fftools/ffplay_renderer.c b/fftools/ffplay_renderer.c >> index 49e5516d5d..497d758ec7 100644 >> --- a/fftools/ffplay_renderer.c >> +++ b/fftools/ffplay_renderer.c >> @@ -35,6 +35,8 @@ struct VkRenderer { >> >> int (*create)(VkRenderer *renderer, SDL_Window *window); >> >> + int (*get_hw_dev)(VkRenderer *renderer, AVBufferRef **dev); >> + >> int (*display)(VkRenderer *renderer, AVFrame *frame); >> >> int (*resize)(VkRenderer *renderer, int width, int height); >> @@ -54,6 +56,13 @@ typedef struct RendererContext { >> pl_tex tex[4]; >> >> pl_log vk_log; >> + >> + AVBufferRef *hw_device_ref; >> + AVBufferRef *hw_frame_ref; >> + enum AVPixelFormat *transfer_formats; >> + AVHWFramesConstraints *constraints; >> + >> + AVFrame *vk_frame; >> } RendererContext; >> >> static void vk_log_cb(void *log_priv, enum pl_log_level level, const char *msg) { >> @@ -74,35 +83,46 @@ static void vk_log_cb(void *log_priv, enum pl_log_level level, const char *msg) >> static int create(VkRenderer *renderer, SDL_Window *window) >> { >> int ret = 0; >> - unsigned ext_num = 0; >> + unsigned num_ext = 0; >> const char **ext = NULL; >> int w, h; >> struct pl_log_params vk_log_params = { >> .log_cb = vk_log_cb, >> - .log_level = PL_LOG_WARN, >> + .log_level = PL_LOG_DEBUG, >> .log_priv = renderer, >> }; >> RendererContext *ctx = (RendererContext *)renderer; >> >> - if (!SDL_Vulkan_GetInstanceExtensions(window, &ext_num, NULL)) { >> + static const char *opt_exts[] = { >> + VK_EXT_DESCRIPTOR_BUFFER_EXTENSION_NAME, >> + VK_EXT_SHADER_ATOMIC_FLOAT_EXTENSION_NAME, >> + VK_KHR_VIDEO_DECODE_QUEUE_EXTENSION_NAME, >> + VK_KHR_VIDEO_DECODE_H264_EXTENSION_NAME, >> + VK_KHR_VIDEO_DECODE_H265_EXTENSION_NAME, >> + VK_KHR_VIDEO_QUEUE_EXTENSION_NAME, >> + "VK_MESA_video_decode_av1", >> + }; >> + const int num_opt_exts = FF_ARRAY_ELEMS(opt_exts); >> + >> + if (!SDL_Vulkan_GetInstanceExtensions(window, &num_ext, NULL)) { >> av_log(NULL, AV_LOG_FATAL, "Failed to get vulkan extensions: %s\n", SDL_GetError()); >> return AVERROR_EXTERNAL; >> > That's not a complete list. Look at optional_device_exts. > There are also device features that both we and libplacebo would like > to be enabled. Look into vulkan_device_create_internal(), the list there > is up to date. FFmpeg requires timeline semaphores, and descriptor_buffer > for lavfi filters to work. Fixed in patch v4. Maybe we should export optional_device_exts in API. > > Is there no option to let libavutil create the hwdevice for both SDL and libplacebo? > That way, that code path can also be tested. libplacebo doesn't support that: // Creates a new vulkan device based on the given parameters and initializes // a new GPU. This function will internally initialize a VkDevice. There is // currently no way to share a vulkan device with the caller. If `params` is // left as NULL, it defaults to &pl_vulkan_default_params. // // Thread-safety: Safe PL_API pl_vulkan pl_vulkan_create(pl_log log, const struct pl_vulkan_params *params); I think share instance or vkImage without support from vulkan spec is hard to implementation, if not impossible. https://www.yosoygames.com.ar/wp/2022/07/vulkan-integration-between-multiple-libraries-e-g-qt-ogrenext/ > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".