From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 2E0E04274E for ; Wed, 30 Mar 2022 03:42:27 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0FF7368B18C; Wed, 30 Mar 2022 06:42:25 +0300 (EEST) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D98BA68AA67 for ; Wed, 30 Mar 2022 06:42:18 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648611744; x=1680147744; h=from:to:subject:date:message-id:mime-version: content-transfer-encoding; bh=jnSYSevQyaugORbQj+jtAbEI1MzoY27TV7svsIZbeic=; b=BwpaI4aTUrMI+hg+IGlk5LzYlccnbEelR6T/icJmHEHT7uj13XXZr6Z9 2lGE0v3eJD3ceG+tGPsTiT3iaxBleKNoxI1Pkb92k+BChwz9Ax2wKJdSx jk3Yv5rMZhrKKJ5IhNk/XMQqdQva/qAy7HOlZkbBwdtjwBN2qyA17FqQE g8HCsQUf8VULFe9m/JaAagYqH4/s/5Wfl6F8FQTDtNYvCMcLrsNxvEul+ 4ziXmYlmXUJdnyNa6K9w2QJsDzhjfXTTLkm28gnBJ/UU3eJEfYWruEjb/ yPts+WnvO1pDOqEmTjLMM1gcP7Y+9u/XOOnqI3JenkacIdgd6eTDcIKJE A==; X-IronPort-AV: E=McAfee;i="6200,9189,10301"; a="241595152" X-IronPort-AV: E=Sophos;i="5.90,221,1643702400"; d="scan'208";a="241595152" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Mar 2022 20:42:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,221,1643702400"; d="scan'208";a="639581197" Received: from wenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.4]) by FMSMGA003.fm.intel.com with ESMTP; 29 Mar 2022 20:42:15 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Wed, 30 Mar 2022 11:41:50 +0800 Message-Id: <20220330034151.834450-1-wenbin.chen@intel.com> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] libavutil/hwcontext_vulkan: Fix VK_FORMAT_R8G8_UNORM and VK_FORMAT_R16G16_UNORM map problem on Vulkan X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Vulkan map both DRM_FORMAT_RG88 and DRM_FORMAT_GR88 to VK_FORMAT_R8G8_UNORM. This cannot distinguish nv12/nv24 and nv21/nv42. Vulkan also map both DRM_FORMAT_RG1616 and DRM_FORMAT_GR1616 to VK_FORMAT_R16G16_UNORM. This causes issue when map back. VK_FORMAT_R16G16 will be mapped to DRM_FORMAT_GR1616, while p010 need DRM_FORMAT_RG1616. Add sw_format check to vulkan_fmt_to_drm() to fix this problem. Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 237caa4bc0..791649001f 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -3290,8 +3290,17 @@ static void vulkan_unmap_to_drm(AVHWFramesContext *hwfc, HWMapDescriptor *hwmap) av_free(drm_desc); } -static inline uint32_t vulkan_fmt_to_drm(VkFormat vkfmt) +static inline uint32_t vulkan_fmt_to_drm(VkFormat vkfmt, + enum AVPixelFormat sw_format) { + if (vkfmt == VK_FORMAT_R8G8_UNORM && + (sw_format == AV_PIX_FMT_NV12 || sw_format == AV_PIX_FMT_NV24)) + return DRM_FORMAT_RG88; + + if (vkfmt == VK_FORMAT_R16G16_UNORM && + (sw_format == AV_PIX_FMT_P010 || sw_format == AV_PIX_FMT_P016)) + return DRM_FORMAT_RG1616; + for (int i = 0; i < FF_ARRAY_ELEMS(vulkan_drm_format_map); i++) if (vulkan_drm_format_map[i].vk_format == vkfmt) return vulkan_drm_format_map[i].drm_fourcc; @@ -3373,7 +3382,7 @@ static int vulkan_map_to_drm(AVHWFramesContext *hwfc, AVFrame *dst, }; VkFormat plane_vkfmt = av_vkfmt_from_pixfmt(hwfc->sw_format)[i]; - drm_desc->layers[i].format = vulkan_fmt_to_drm(plane_vkfmt); + drm_desc->layers[i].format = vulkan_fmt_to_drm(plane_vkfmt, hwfc->sw_format); drm_desc->layers[i].nb_planes = 1; if (drm_desc->layers[i].format == DRM_FORMAT_INVALID) { -- 2.32.0 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".