From: Wenbin Chen <wenbin.chen-at-intel.com@ffmpeg.org> To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 1/2] libavutil/hwcontext_vulkan: Fix VK_FORMAT_R8G8_UNORM and VK_FORMAT_R16G16_UNORM map problem on Vulkan Date: Wed, 30 Mar 2022 11:41:50 +0800 Message-ID: <20220330034151.834450-1-wenbin.chen@intel.com> (raw) Vulkan map both DRM_FORMAT_RG88 and DRM_FORMAT_GR88 to VK_FORMAT_R8G8_UNORM. This cannot distinguish nv12/nv24 and nv21/nv42. Vulkan also map both DRM_FORMAT_RG1616 and DRM_FORMAT_GR1616 to VK_FORMAT_R16G16_UNORM. This causes issue when map back. VK_FORMAT_R16G16 will be mapped to DRM_FORMAT_GR1616, while p010 need DRM_FORMAT_RG1616. Add sw_format check to vulkan_fmt_to_drm() to fix this problem. Signed-off-by: Wenbin Chen <wenbin.chen@intel.com> --- libavutil/hwcontext_vulkan.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 237caa4bc0..791649001f 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -3290,8 +3290,17 @@ static void vulkan_unmap_to_drm(AVHWFramesContext *hwfc, HWMapDescriptor *hwmap) av_free(drm_desc); } -static inline uint32_t vulkan_fmt_to_drm(VkFormat vkfmt) +static inline uint32_t vulkan_fmt_to_drm(VkFormat vkfmt, + enum AVPixelFormat sw_format) { + if (vkfmt == VK_FORMAT_R8G8_UNORM && + (sw_format == AV_PIX_FMT_NV12 || sw_format == AV_PIX_FMT_NV24)) + return DRM_FORMAT_RG88; + + if (vkfmt == VK_FORMAT_R16G16_UNORM && + (sw_format == AV_PIX_FMT_P010 || sw_format == AV_PIX_FMT_P016)) + return DRM_FORMAT_RG1616; + for (int i = 0; i < FF_ARRAY_ELEMS(vulkan_drm_format_map); i++) if (vulkan_drm_format_map[i].vk_format == vkfmt) return vulkan_drm_format_map[i].drm_fourcc; @@ -3373,7 +3382,7 @@ static int vulkan_map_to_drm(AVHWFramesContext *hwfc, AVFrame *dst, }; VkFormat plane_vkfmt = av_vkfmt_from_pixfmt(hwfc->sw_format)[i]; - drm_desc->layers[i].format = vulkan_fmt_to_drm(plane_vkfmt); + drm_desc->layers[i].format = vulkan_fmt_to_drm(plane_vkfmt, hwfc->sw_format); drm_desc->layers[i].nb_planes = 1; if (drm_desc->layers[i].format == DRM_FORMAT_INVALID) { -- 2.32.0 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
next reply other threads:[~2022-03-30 3:42 UTC|newest] Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-03-30 3:41 Wenbin Chen [this message] 2022-03-30 3:41 ` [FFmpeg-devel] [PATCH 2/2] libavutil/hwcontext_vaapi: fix nv12 map error Wenbin Chen 2022-04-13 6:05 ` [FFmpeg-devel] [PATCH 1/2] libavutil/hwcontext_vulkan: Fix VK_FORMAT_R8G8_UNORM and VK_FORMAT_R16G16_UNORM map problem on Vulkan Xiang, Haihao 2022-04-14 10:39 ` Lynne
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220330034151.834450-1-wenbin.chen@intel.com \ --to=wenbin.chen-at-intel.com@ffmpeg.org \ --cc=ffmpeg-devel@ffmpeg.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel This inbox may be cloned and mirrored by anyone: git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \ ffmpegdev@gitmailbox.com public-inbox-index ffmpegdev Example config snippet for mirrors. AGPL code for this site: git clone https://public-inbox.org/public-inbox.git