From patchwork Wed Mar 30 03:41:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Wenbin" X-Patchwork-Id: 35050 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c05:b0:7a:e998:b410 with SMTP id bw5csp610782pzb; Tue, 29 Mar 2022 20:42:30 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw32WXcqTB6E2Yc/wUHkiwO7vAYk4VSRJ3cEamztQ7JsUZNM3RgZoz5lYTFMZd9GP1BTVmJ X-Received: by 2002:aa7:c04e:0:b0:400:4daf:bab1 with SMTP id k14-20020aa7c04e000000b004004dafbab1mr8100618edo.101.1648611749951; Tue, 29 Mar 2022 20:42:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1648611749; cv=none; d=google.com; s=arc-20160816; b=YEKNPDqu/4AP2G5lv5LkYzyQY8RzNXZGtZktgmKduayGO/XlehAopDh6+126E5zYUl olKPHvTmvtm2l72SuWlSJyk/YEys1m2i3vCoRxtGQg7jaxDP30SSWj3k4eDvjfcf/FcE ypORootOTm4LGQ2b9CnuZM7IGAD/f1miAhAaV/WW2ayyDaBM84kA4GDxdeBSB1tnakcL 9hWYJYIyc/iRowlH9zuMA9zYTPO7egyVzIZh+9zmREkctmdU7m8vElP9E+1MO7/EoUn/ 8UA2rw9PPWi5SOxzB+2XEa5QzM21CHtTwvaljsJNDphZMvDX83JylwP1+R4PywReQ2P/ yCZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=57+ygdQXQsYkJrSwsHIcGjOmc3BiHOUA3f67C4V8y7E=; b=BI2uzrGR8bTO8emd67VITg57CMBVCZgUjGDlNKkfS0ruhGbnvz6nFQW4iNXZ3KSsJj fAVEaAVxFChweteAZcz3DSZMD+Nvq41+uwoB7MsgIe69G9rsty+dilbe/8Jd0WkspPrH /MY9WmNqJR3mWqVn/+3OC0drAAMiAhA3dxXneO/hT9yWAiYaKLE7Ak3iVpU7POsIKdsa S1moRlNZnMtYXYWi4jyiB/QD9mKInm/jdFS1VZGCk1Owf3HTuSScR/xHqgWUZmZ5c297 Cu4UmOk9BiClwcCL7d6LdqhTGba78KaszUlxPItQZ17yqVXG+jwCet+gSI6yzDoex7DN +iVQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=BwpaI4aT; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id n25-20020a1709065e1900b006df76385df9si20488922eju.665.2022.03.29.20.42.29; Tue, 29 Mar 2022 20:42:29 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=BwpaI4aT; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 69EF568B1C6; Wed, 30 Mar 2022 06:42:25 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D98BA68AA67 for ; Wed, 30 Mar 2022 06:42:18 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648611744; x=1680147744; h=from:to:subject:date:message-id:mime-version: content-transfer-encoding; bh=jnSYSevQyaugORbQj+jtAbEI1MzoY27TV7svsIZbeic=; b=BwpaI4aTUrMI+hg+IGlk5LzYlccnbEelR6T/icJmHEHT7uj13XXZr6Z9 2lGE0v3eJD3ceG+tGPsTiT3iaxBleKNoxI1Pkb92k+BChwz9Ax2wKJdSx jk3Yv5rMZhrKKJ5IhNk/XMQqdQva/qAy7HOlZkbBwdtjwBN2qyA17FqQE g8HCsQUf8VULFe9m/JaAagYqH4/s/5Wfl6F8FQTDtNYvCMcLrsNxvEul+ 4ziXmYlmXUJdnyNa6K9w2QJsDzhjfXTTLkm28gnBJ/UU3eJEfYWruEjb/ yPts+WnvO1pDOqEmTjLMM1gcP7Y+9u/XOOnqI3JenkacIdgd6eTDcIKJE A==; X-IronPort-AV: E=McAfee;i="6200,9189,10301"; a="241595152" X-IronPort-AV: E=Sophos;i="5.90,221,1643702400"; d="scan'208";a="241595152" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Mar 2022 20:42:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,221,1643702400"; d="scan'208";a="639581197" Received: from wenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.4]) by FMSMGA003.fm.intel.com with ESMTP; 29 Mar 2022 20:42:15 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Wed, 30 Mar 2022 11:41:50 +0800 Message-Id: <20220330034151.834450-1-wenbin.chen@intel.com> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] libavutil/hwcontext_vulkan: Fix VK_FORMAT_R8G8_UNORM and VK_FORMAT_R16G16_UNORM map problem on Vulkan X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: a44FO9puvWeG Vulkan map both DRM_FORMAT_RG88 and DRM_FORMAT_GR88 to VK_FORMAT_R8G8_UNORM. This cannot distinguish nv12/nv24 and nv21/nv42. Vulkan also map both DRM_FORMAT_RG1616 and DRM_FORMAT_GR1616 to VK_FORMAT_R16G16_UNORM. This causes issue when map back. VK_FORMAT_R16G16 will be mapped to DRM_FORMAT_GR1616, while p010 need DRM_FORMAT_RG1616. Add sw_format check to vulkan_fmt_to_drm() to fix this problem. Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 237caa4bc0..791649001f 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -3290,8 +3290,17 @@ static void vulkan_unmap_to_drm(AVHWFramesContext *hwfc, HWMapDescriptor *hwmap) av_free(drm_desc); } -static inline uint32_t vulkan_fmt_to_drm(VkFormat vkfmt) +static inline uint32_t vulkan_fmt_to_drm(VkFormat vkfmt, + enum AVPixelFormat sw_format) { + if (vkfmt == VK_FORMAT_R8G8_UNORM && + (sw_format == AV_PIX_FMT_NV12 || sw_format == AV_PIX_FMT_NV24)) + return DRM_FORMAT_RG88; + + if (vkfmt == VK_FORMAT_R16G16_UNORM && + (sw_format == AV_PIX_FMT_P010 || sw_format == AV_PIX_FMT_P016)) + return DRM_FORMAT_RG1616; + for (int i = 0; i < FF_ARRAY_ELEMS(vulkan_drm_format_map); i++) if (vulkan_drm_format_map[i].vk_format == vkfmt) return vulkan_drm_format_map[i].drm_fourcc; @@ -3373,7 +3382,7 @@ static int vulkan_map_to_drm(AVHWFramesContext *hwfc, AVFrame *dst, }; VkFormat plane_vkfmt = av_vkfmt_from_pixfmt(hwfc->sw_format)[i]; - drm_desc->layers[i].format = vulkan_fmt_to_drm(plane_vkfmt); + drm_desc->layers[i].format = vulkan_fmt_to_drm(plane_vkfmt, hwfc->sw_format); drm_desc->layers[i].nb_planes = 1; if (drm_desc->layers[i].format == DRM_FORMAT_INVALID) { From patchwork Wed Mar 30 03:41:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Wenbin" X-Patchwork-Id: 35051 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c05:b0:7a:e998:b410 with SMTP id bw5csp610853pzb; Tue, 29 Mar 2022 20:42:40 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwVV3LSxDE0JHtidUr5jDW+LZCzOFMtXjk89S/7lo1rlYtfa8uCeMFw8xUtkWMVlzJb9pv4 X-Received: by 2002:a17:907:94c6:b0:6da:9561:ce0 with SMTP id dn6-20020a17090794c600b006da95610ce0mr37372444ejc.342.1648611760315; Tue, 29 Mar 2022 20:42:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1648611760; cv=none; d=google.com; s=arc-20160816; b=ucjrwKYdwHuw5rljPFmocKwPXZvm6Kht7fQSpQEdg9/L1N7Ep1Q3/hypqezeKBtXZC +YL3Iw01a2T3LRaNN/atftQU6O29Svf2AcYOBpjmPCIa1reGTwyE70BsG9a3QAQO07sp Q0Cv0xk6f8JKvNg8HrdNREBx12rbt51H67xzlslQkFSkzkGuS86QuHzPnah0rCS1D/Wg j9rFpk9UxwUnJsFLjK3QBv/V3kAoWC3FrnkT6aAVaVVBGCb9TdaIXPrrZ2egvRy4oJ8D aZY8dqTv6tpmd5Vm/7kOlBh96AagqjwT9/76rbMmqLVZcHyeA1NEsPgzysWYEuwWR54o /JMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=OaIwcsSOTrbW1ACs8VE+k/WJcUnzMt8PVPZ5MN37zU8=; b=anGU0XhIrbwKDzruICkYzNmgqCpAnXI80nM9+dVjpFk3/Q5avuR0R70Ssg36o54G21 MHDt4G7ZhlRMNKLK1IqXIiW2JCpHs6vpzwAsTYGlDsMSC6ltJ7nNKvaCGUDjmQPTjHcI kovyBS7AA3finnn8dI/Yo3wcs+CpKv14GsExWISTNcbOT7oGf0YeFbrOhEsJ53alRoje GK+GwvETk4Gaf4jHBx0iR8OhfWXc5nLkOvU5ipzLoJKs2RtBXaCaN9G2n3tUMsugUR8c kKAWyVjV/vaNTvWtILFp9A6VzUMQroSGKnM9CRDjDWAiqdMD6Birkfu2pE2bRE8iRpGz QR3g== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Osgo6cWP; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id t17-20020a170906065100b006dfa8b9bc20si20003603ejb.894.2022.03.29.20.42.39; Tue, 29 Mar 2022 20:42:40 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Osgo6cWP; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7992468B1F8; Wed, 30 Mar 2022 06:42:31 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C56C568AA67 for ; Wed, 30 Mar 2022 06:42:24 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648611750; x=1680147750; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=0KB8gIrXg5d6XdU7Be48bmxlnsAvANp25hlgcj1HSE0=; b=Osgo6cWPoeiuqddGfyXelkIA8G3vsbDtEs0w1exKI9Z20Uu1Hpxz8heS Mf05hTM+iqIyW4U1PGHzY5Lh3tsixL3zX3+uhDXzI4q71H3kmOpY3s36j z8PrLhTbgNW3BFsawLqhByATQkPtrx8+/01PrIu75xCxnBXQW+eegb2v3 V4HATxckNEmjYPZdTSJ/1lFc+36yMQEc3sqk0qFUWH6KRPcr0HhF1ocPi n5Yhm24CALpJDEZOuFkdvRzseknQT1IEFEoH0hbvkIDnjktx+zcIJdEOu UfwaHXB6tcdrplh6NsdEMYZu4e1qfXPpK6FFX3sCvsQHPBcXDxo6lUI3K A==; X-IronPort-AV: E=McAfee;i="6200,9189,10301"; a="241595153" X-IronPort-AV: E=Sophos;i="5.90,221,1643702400"; d="scan'208";a="241595153" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Mar 2022 20:42:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,221,1643702400"; d="scan'208";a="639581200" Received: from wenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.4]) by FMSMGA003.fm.intel.com with ESMTP; 29 Mar 2022 20:42:16 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Wed, 30 Mar 2022 11:41:51 +0800 Message-Id: <20220330034151.834450-2-wenbin.chen@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220330034151.834450-1-wenbin.chen@intel.com> References: <20220330034151.834450-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] libavutil/hwcontext_vaapi: fix nv12 map error X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 3cDw4y+nCAay The UV plane of NV12 should not be mapped to DRM_FORMAT_GR88. Remove this map after the problem is fixed on ffmpeg-vulkan side. Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vaapi.c | 1 - 1 file changed, 1 deletion(-) diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index 994b744e4d..75acc851d6 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -992,7 +992,6 @@ static const struct { } vaapi_drm_format_map[] = { #ifdef DRM_FORMAT_R8 DRM_MAP(NV12, 2, DRM_FORMAT_R8, DRM_FORMAT_RG88), - DRM_MAP(NV12, 2, DRM_FORMAT_R8, DRM_FORMAT_GR88), #endif DRM_MAP(NV12, 1, DRM_FORMAT_NV12), #if defined(VA_FOURCC_P010) && defined(DRM_FORMAT_R16)