From patchwork Tue Aug 31 07:24:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29892 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4685219iov; Tue, 31 Aug 2021 00:26:59 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzVUibZ9D/TjZYfJW74dJ8ein/MLWkuc+vYT99fuGGS07qt0tllfFY9t+PjdogscJcWZB87 X-Received: by 2002:a17:907:7704:: with SMTP id kw4mr28668975ejc.23.1630394818871; Tue, 31 Aug 2021 00:26:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630394818; cv=none; d=google.com; s=arc-20160816; b=oMHJD1/NZAG3namQSi9WINLbTQOyEDtkp+cLZhTTnAx4NmAtWAdzezInyrUgrxxRVy UEITV24U0aRRQiXsKQpmqAnft2KLHe3dCwzBazRnJCjr3X79VWlAXh9XhhhT8Nn9unZN OiPJ6jC+9DFGOvX/cklgREzU6ogm8UxBB94zbQ0WZvKLe9A9WZWsDQJ/5Qm/d3Pi//tF o0JNb9Sq8DTPn51Rw0AiEpU1BEaIjSTvXGnHcdDNVGvkgxt70M3ChwsO1vBJ18qVwVFn VBi+Txr9dN++mWgyigvPX521TzCBECHVViaodqGCmXqIj0q/YXBUx7/a8OBaWVPUNYWx sw2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=affs3WuOkBcNV05wxV/DtuErCL2uXsBwCe5hvVkfLqo=; b=IdtTnDMwU1N6FWBhH0wwJGTNt17diiE4A0GR2sf7r0UbCUcfar21n8X25TIsAdIM/h FR5Dlt0qJMX6GW6zn7pV/6IxgEwMPWCMs8ejTRUIrhRLUaMye4E8myBWpqCmIHMSwEPx alTunOtNfCX2baNHGQOv4T/diSdc+Q14yDydqSkcZqSaVTzw/i0/VpDf2lRiaShXULlQ ykFtGKfaA1CJCdqPcFK5Kfp3xKixMoHXLjVTnYDS05ksNKLprHKgRHzeZxwa8g0c024C KiExiJ5SycNOd/b5+lzcQEykRwy3ATRskez/Lr6UfWYJE0bZQ6GtwyeZx4jkaq6JwmQR xoHA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id aq21si17512617ejc.637.2021.08.31.00.26.57; Tue, 31 Aug 2021 00:26:58 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5CD036899DA; Tue, 31 Aug 2021 10:26:53 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E8686680749 for ; Tue, 31 Aug 2021 10:26:45 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="282133617" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="282133617" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Aug 2021 00:26:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="540905078" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by fmsmga002.fm.intel.com with ESMTP; 31 Aug 2021 00:26:42 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 15:24:21 +0800 Message-Id: <20210831072430.303090-1-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [V2 01/10] libavfilter/vulkan: Fix problem when device have queue_count greater than 1 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: IlLUQlgFnQI4 If the descriptorSetCount is greater than the number of setLayouts, vkAllocateDescriptorSets will report error. Now fix it. Now the following command can run on the device that has queue_count greater than one: ffmpeg -v verbose -init_hw_device vulkan=vul:0 -filter_hw_device vul -i input1080p.264 -vf "hwupload=extra_hw_frames=16,scale_vulkan=1920:1080, hwdownload,format=yuv420p" -f rawvideo output.yuv Signed-off-by: Wenbin Chen --- libavfilter/vulkan.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavfilter/vulkan.c b/libavfilter/vulkan.c index 337c8d7d5a..e5b070b3e6 100644 --- a/libavfilter/vulkan.c +++ b/libavfilter/vulkan.c @@ -1160,7 +1160,7 @@ void ff_vk_update_descriptor_set(AVFilterContext *avctx, VulkanPipeline *pl, VulkanFilterContext *s = avctx->priv; vkUpdateDescriptorSetWithTemplate(s->hwctx->act_dev, - pl->desc_set[s->cur_queue_idx * pl->desc_layout_num + set_id], + pl->desc_set[set_id], pl->desc_template[set_id], s); } @@ -1179,7 +1179,7 @@ int ff_vk_init_pipeline_layout(AVFilterContext *avctx, VulkanPipeline *pl) VkResult ret; VulkanFilterContext *s = avctx->priv; - pl->descriptor_sets_num = pl->desc_layout_num * s->queue_count; + pl->descriptor_sets_num = pl->desc_layout_num; { /* Init descriptor set pool */ VkDescriptorPoolCreateInfo pool_create_info = { From patchwork Tue Aug 31 07:24:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29901 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4685323iov; Tue, 31 Aug 2021 00:27:09 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxzrt7eIzdWrN/GTeW2PEhtjUnaqFnp9cNZ21HJf4ik2j8kYiWRlp3/u2yM9JEnDyTaMtZB X-Received: by 2002:a17:906:4cc1:: with SMTP id q1mr30479451ejt.415.1630394829237; Tue, 31 Aug 2021 00:27:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630394829; cv=none; d=google.com; s=arc-20160816; b=M+XAouBDLhMZKqAxvXs8uBsOfe+t81Kh9TZ1nZZ5A9YyR/rOiMcBoYU+Luav9rHrsK 3qAZVgSgj6vy+ebzcuKGhz6AOQgo3+f7FofVtFLqqtE3NqGZX5FKAzpa2aJOlsybIEm/ Zqxe7qsyXj/p0+kDQjwrXPWTfAf8bFxXBCUnw30Rmyknbp25wQiOIwWLi+q8O+7ermZc +01+ThkedGn8MblNH0N+4BL1tQFx+vVIIoB6g51OSo2AdjCUFw/As1BkrTqs/uBE+t/a LiqSLh6w50k+9FzEETYc+QJJID7Kq9f1QKRN916MJIq9UUl40X+TLKychR/4FSPZZljh uyPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=KTcIXLozmHy9eAyvxWmQIIiex77TS01nEMNIk0d4XEM=; b=E00OyajeVgkhWiG7e8wZcR3grcbPvJkHjbQdXMg9Ly93q9BRcuohqQ3a1scvB+4qYD yDc37MIznPg08MQdwwIxQH37sQdu2srzSi72mHUCBVpFIy3eEVZxRZAEMprlZEPPijTi sWWQnx7tlveAEuorQ4yJJZZyVcHtoweALUIlFWsrGW04Gd2cKt5KkDoa1BnyPFzK/Wgs 72QLeexWHQS0VmY4broMRtu4tnU+wcXFPDamNR0ZbvPzC5N1l1BPU+KGKivMLDhtEt1y o+2mkjuppx2QHytiUlf/AhCVpdZI+TAZQ61gxv3Ma9EWL+pCaXaoaw1xOWU5BvRa0giB Nt3A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id fl21si1229948ejc.273.2021.08.31.00.27.08; Tue, 31 Aug 2021 00:27:09 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6C09C689F59; Tue, 31 Aug 2021 10:27:02 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 34593689E61 for ; Tue, 31 Aug 2021 10:26:55 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="215291843" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="215291843" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Aug 2021 00:26:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="540905108" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by fmsmga002.fm.intel.com with ESMTP; 31 Aug 2021 00:26:52 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 15:24:22 +0800 Message-Id: <20210831072430.303090-2-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210831072430.303090-1-wenbin.chen@intel.com> References: <20210831072430.303090-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [V2 02/10] libavutil/hwcontext_vulkan: fix a tile mismatch problem X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 2agv8F/un1m3 We should configure VkImageSubresource according to tiling rather than extension. We use extension to set tiling only when we map from drm. Normally the output VkImages are not created in this way, and it will report error when we map these VkImage to drm, so we should configure VkImageSubresource according to tiling rather than extension. Now fix it. Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 94fdad7f06..88db5b8b70 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -2888,7 +2888,7 @@ static int vulkan_map_to_drm(AVHWFramesContext *hwfc, AVFrame *dst, for (int i = 0; i < drm_desc->nb_layers; i++) { VkSubresourceLayout layout; VkImageSubresource sub = { - .aspectMask = p->extensions & EXT_DRM_MODIFIER_FLAGS ? + .aspectMask = f->tiling == VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT ? VK_IMAGE_ASPECT_MEMORY_PLANE_0_BIT_EXT : VK_IMAGE_ASPECT_COLOR_BIT, }; From patchwork Tue Aug 31 07:24:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29897 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4685429iov; Tue, 31 Aug 2021 00:27:20 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzJfkgorvbH9UPn8aDDWCM5ANpN6oaZo0nwZW/yPYVKxYH8qvAEGpMfhSuTtap3VekP/mXY X-Received: by 2002:aa7:c952:: with SMTP id h18mr28512954edt.18.1630394840239; Tue, 31 Aug 2021 00:27:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630394840; cv=none; d=google.com; s=arc-20160816; b=VcyqcFrvekuC54U/UAS5r+pCNRM7dzUrVNgt61U2y2cbhrbbOwAAY4oLeM0hxEkkuh Trk/WezSMCKFatl7wtbdWBp61tO6o9fsP6m2tWSwN91JHJCbbt6VbtiDk0oIu9nYnmGd CwIk88tUWV3RgkNdGY7oqLnugpzyemHd8jkZoYuiY5JA1Hwb6uAAmCvDsbavotqK7Huo OTpygPriTqNEVmRwI2vcC/KC0C5hxS0S0FJYaBzNhSA7YhzRe0lPoSVVFUhzwq2iCmkL gLNlj/0g2/wYecoRZAnmziQ6c7sL+7HZZeAxHH0VgbjwQbBxfao6BlpYZ9oatVc0Oy5P B1fQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=B86/XdSeQqUMkLOCsCAZJfQMh9oSK757btsw4+AP1Oc=; b=j9Td9xPY5OOfnEePhYlWDi04+Bn2zV3Nse/HRQdRUs96nNV0gp3RN0FJUIizPpkY+o PzWA7EDtTBViiPyqZjoycsjdaY9OxPWQy6IskcyPwAZ8uodx7XMQxWkjP/8AqheqpNpP Qv72/NlImpuMNacL9X3DNeHZ1nvFrIVEyDG38jO5vu/ElJ6tsDk5EIJnSxPHi0ApJQDo KIZdMcMT7DqBZkvtQfF5zV22F6CSfhd6rVgJJ1ttNr+2SHe4ijmHtdwUDMp+RK+umrPb aa9h/AF9TfdszqvE0rfMp2ttSctKgwsRcPKwsOQIVAsGStlMOAoWcZF/ZpAh2Q7XwBSM VZyA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id dm13si17267295edb.484.2021.08.31.00.27.19; Tue, 31 Aug 2021 00:27:20 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 89B2E68A0B6; Tue, 31 Aug 2021 10:27:10 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A290F689966 for ; Tue, 31 Aug 2021 10:27:02 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="282133652" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="282133652" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Aug 2021 00:27:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="540905132" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by fmsmga002.fm.intel.com with ESMTP; 31 Aug 2021 00:26:59 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 15:24:23 +0800 Message-Id: <20210831072430.303090-3-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210831072430.303090-1-wenbin.chen@intel.com> References: <20210831072430.303090-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [V2 03/10] libavfilter/vulkan: Fix the way to use sem X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ZmMnwJameXvi We chould set waitSem and signalSem differently. Current ffmpeg-vulkan uses the same sem to set waitSem and signalSem and it doesn't work on latest intel-vulkan-driver. The commit: a193060221c4df123e26a562949cae5df3e73cde on mesa causes this problem. This commit add code to resets the signalSem. This will reset waitSem too on current ffmpeg-vulkan. Now set waitSem and signalSem separetely. Now the following command can run on the latest mesa on intel platform: ffmpeg -v verbose -init_hw_device vulkan=vul:0,linear_images=1 -filter_hw_device vul -i input1080p.264 -vf "hwupload=extra_hw_frames=16,scale_vulkan=1920:1080, hwdownload,format=yuv420p" -f rawvideo output.yuv Signed-off-by: Wenbin Chen --- libavfilter/vf_avgblur_vulkan.c | 4 +-- libavfilter/vf_chromaber_vulkan.c | 4 +-- libavfilter/vf_overlay_vulkan.c | 6 ++-- libavfilter/vf_scale_vulkan.c | 4 +-- libavfilter/vulkan.c | 55 +++++++++++++++++-------------- libavfilter/vulkan.h | 3 +- libavutil/hwcontext_vulkan.c | 14 ++++---- 7 files changed, 50 insertions(+), 40 deletions(-) diff --git a/libavfilter/vf_avgblur_vulkan.c b/libavfilter/vf_avgblur_vulkan.c index 5ae487fc8c..d2104c191e 100644 --- a/libavfilter/vf_avgblur_vulkan.c +++ b/libavfilter/vf_avgblur_vulkan.c @@ -304,8 +304,8 @@ static int process_frames(AVFilterContext *avctx, AVFrame *out_f, AVFrame *tmp_f vkCmdDispatch(cmd_buf, s->vkctx.output_width, FFALIGN(s->vkctx.output_height, CGS)/CGS, 1); - ff_vk_add_exec_dep(avctx, s->exec, in_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT); - ff_vk_add_exec_dep(avctx, s->exec, out_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT); + ff_vk_add_exec_dep(avctx, s->exec, in_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, 1); + ff_vk_add_exec_dep(avctx, s->exec, out_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, 0); err = ff_vk_submit_exec_queue(avctx, s->exec); if (err) diff --git a/libavfilter/vf_chromaber_vulkan.c b/libavfilter/vf_chromaber_vulkan.c index 96fdd7bd9c..fe66a31cea 100644 --- a/libavfilter/vf_chromaber_vulkan.c +++ b/libavfilter/vf_chromaber_vulkan.c @@ -249,8 +249,8 @@ static int process_frames(AVFilterContext *avctx, AVFrame *out_f, AVFrame *in_f) FFALIGN(s->vkctx.output_width, CGROUPS[0])/CGROUPS[0], FFALIGN(s->vkctx.output_height, CGROUPS[1])/CGROUPS[1], 1); - ff_vk_add_exec_dep(avctx, s->exec, in_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT); - ff_vk_add_exec_dep(avctx, s->exec, out_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT); + ff_vk_add_exec_dep(avctx, s->exec, in_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, 1); + ff_vk_add_exec_dep(avctx, s->exec, out_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, 0); err = ff_vk_submit_exec_queue(avctx, s->exec); if (err) diff --git a/libavfilter/vf_overlay_vulkan.c b/libavfilter/vf_overlay_vulkan.c index 1815709d82..2e5bef5be5 100644 --- a/libavfilter/vf_overlay_vulkan.c +++ b/libavfilter/vf_overlay_vulkan.c @@ -331,9 +331,9 @@ static int process_frames(AVFilterContext *avctx, AVFrame *out_f, FFALIGN(s->vkctx.output_width, CGROUPS[0])/CGROUPS[0], FFALIGN(s->vkctx.output_height, CGROUPS[1])/CGROUPS[1], 1); - ff_vk_add_exec_dep(avctx, s->exec, main_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT); - ff_vk_add_exec_dep(avctx, s->exec, overlay_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT); - ff_vk_add_exec_dep(avctx, s->exec, out_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT); + ff_vk_add_exec_dep(avctx, s->exec, main_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, 1); + ff_vk_add_exec_dep(avctx, s->exec, overlay_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, 1); + ff_vk_add_exec_dep(avctx, s->exec, out_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, 0); err = ff_vk_submit_exec_queue(avctx, s->exec); if (err) diff --git a/libavfilter/vf_scale_vulkan.c b/libavfilter/vf_scale_vulkan.c index 4eb4fe5664..0d946e0416 100644 --- a/libavfilter/vf_scale_vulkan.c +++ b/libavfilter/vf_scale_vulkan.c @@ -377,8 +377,8 @@ static int process_frames(AVFilterContext *avctx, AVFrame *out_f, AVFrame *in_f) FFALIGN(s->vkctx.output_width, CGROUPS[0])/CGROUPS[0], FFALIGN(s->vkctx.output_height, CGROUPS[1])/CGROUPS[1], 1); - ff_vk_add_exec_dep(avctx, s->exec, in_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT); - ff_vk_add_exec_dep(avctx, s->exec, out_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT); + ff_vk_add_exec_dep(avctx, s->exec, in_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, 1); + ff_vk_add_exec_dep(avctx, s->exec, out_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, 0); err = ff_vk_submit_exec_queue(avctx, s->exec); if (err) diff --git a/libavfilter/vulkan.c b/libavfilter/vulkan.c index e5b070b3e6..e8cbf66b2b 100644 --- a/libavfilter/vulkan.c +++ b/libavfilter/vulkan.c @@ -462,9 +462,10 @@ VkCommandBuffer ff_vk_get_exec_buf(AVFilterContext *avctx, FFVkExecContext *e) } int ff_vk_add_exec_dep(AVFilterContext *avctx, FFVkExecContext *e, - AVFrame *frame, VkPipelineStageFlagBits in_wait_dst_flag) + AVFrame *frame, VkPipelineStageFlagBits in_wait_dst_flag, int input_frame) { AVFrame **dst; + VkSemaphore *sem_temp; VulkanFilterContext *s = avctx->priv; AVVkFrame *f = (AVVkFrame *)frame->data[0]; FFVkQueueCtx *q = &e->queues[s->cur_queue_idx]; @@ -472,33 +473,39 @@ int ff_vk_add_exec_dep(AVFilterContext *avctx, FFVkExecContext *e, int planes = av_pix_fmt_count_planes(fc->sw_format); for (int i = 0; i < planes; i++) { - e->sem_wait = av_fast_realloc(e->sem_wait, &e->sem_wait_alloc, - (e->sem_wait_cnt + 1)*sizeof(*e->sem_wait)); - if (!e->sem_wait) { - ff_vk_discard_exec_deps(avctx, e); - return AVERROR(ENOMEM); - } + if (input_frame) { + sem_temp = av_fast_realloc(e->sem_wait, &e->sem_wait_alloc, + (e->sem_wait_cnt + 1)*sizeof(*e->sem_wait)); + if (!sem_temp) { + ff_vk_discard_exec_deps(avctx, e); + return AVERROR(ENOMEM); + } + e->sem_wait = sem_temp; - e->sem_wait_dst = av_fast_realloc(e->sem_wait_dst, &e->sem_wait_dst_alloc, - (e->sem_wait_cnt + 1)*sizeof(*e->sem_wait_dst)); - if (!e->sem_wait_dst) { - ff_vk_discard_exec_deps(avctx, e); - return AVERROR(ENOMEM); - } + sem_temp = av_fast_realloc(e->sem_wait_dst, &e->sem_wait_dst_alloc, + (e->sem_wait_cnt + 1)*sizeof(*e->sem_wait_dst)); + if (!sem_temp) { + ff_vk_discard_exec_deps(avctx, e); + return AVERROR(ENOMEM); + } + e->sem_wait_dst = sem_temp; - e->sem_sig = av_fast_realloc(e->sem_sig, &e->sem_sig_alloc, - (e->sem_sig_cnt + 1)*sizeof(*e->sem_sig)); - if (!e->sem_sig) { - ff_vk_discard_exec_deps(avctx, e); - return AVERROR(ENOMEM); - } + e->sem_wait[e->sem_wait_cnt] = f->sem[i]; + e->sem_wait_dst[e->sem_wait_cnt] = in_wait_dst_flag; + e->sem_wait_cnt++; + } else { - e->sem_wait[e->sem_wait_cnt] = f->sem[i]; - e->sem_wait_dst[e->sem_wait_cnt] = in_wait_dst_flag; - e->sem_wait_cnt++; + sem_temp = av_fast_realloc(e->sem_sig, &e->sem_sig_alloc, + (e->sem_sig_cnt + 1)*sizeof(*e->sem_sig)); + if (!sem_temp) { + ff_vk_discard_exec_deps(avctx, e); + return AVERROR(ENOMEM); + } + e->sem_sig = sem_temp; - e->sem_sig[e->sem_sig_cnt] = f->sem[i]; - e->sem_sig_cnt++; + e->sem_sig[e->sem_sig_cnt] = f->sem[i]; + e->sem_sig_cnt++; + } } dst = av_fast_realloc(q->frame_deps, &q->frame_deps_alloc_size, diff --git a/libavfilter/vulkan.h b/libavfilter/vulkan.h index f9a4dc5839..2fdc0e1368 100644 --- a/libavfilter/vulkan.h +++ b/libavfilter/vulkan.h @@ -340,7 +340,8 @@ void ff_vk_discard_exec_deps(AVFilterContext *avctx, FFVkExecContext *e); * Must be called before submission. */ int ff_vk_add_exec_dep(AVFilterContext *avctx, FFVkExecContext *e, - AVFrame *frame, VkPipelineStageFlagBits in_wait_dst_flag); + AVFrame *frame, VkPipelineStageFlagBits in_wait_dst_flag, + int input_frame); /** * Submits a command buffer to the queue for execution. diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 88db5b8b70..9a29267aed 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -1737,8 +1737,6 @@ static int prepare_frame(AVHWFramesContext *hwfc, VulkanExecCtx *ectx, VkSubmitInfo s_info = { .sType = VK_STRUCTURE_TYPE_SUBMIT_INFO, - .pSignalSemaphores = frame->sem, - .signalSemaphoreCount = planes, }; VkPipelineStageFlagBits wait_st[AV_NUM_DATA_POINTERS]; @@ -1750,11 +1748,15 @@ static int prepare_frame(AVHWFramesContext *hwfc, VulkanExecCtx *ectx, new_layout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL; new_access = VK_ACCESS_TRANSFER_WRITE_BIT; dst_qf = VK_QUEUE_FAMILY_IGNORED; + s_info.pSignalSemaphores = frame->sem; + s_info.signalSemaphoreCount = planes; break; case PREP_MODE_RO_SHADER: new_layout = VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL; new_access = VK_ACCESS_TRANSFER_READ_BIT; dst_qf = VK_QUEUE_FAMILY_IGNORED; + s_info.pSignalSemaphores = frame->sem; + s_info.signalSemaphoreCount = planes; break; case PREP_MODE_EXTERNAL_EXPORT: new_layout = VK_IMAGE_LAYOUT_GENERAL; @@ -3226,11 +3228,11 @@ static int transfer_image_buf(AVHWFramesContext *hwfc, const AVFrame *f, VkSubmitInfo s_info = { .sType = VK_STRUCTURE_TYPE_SUBMIT_INFO, - .pSignalSemaphores = frame->sem, - .pWaitSemaphores = frame->sem, + .pSignalSemaphores = to_buf ? NULL: frame->sem, + .pWaitSemaphores = to_buf ? frame->sem : NULL, .pWaitDstStageMask = sem_wait_dst, - .signalSemaphoreCount = planes, - .waitSemaphoreCount = planes, + .signalSemaphoreCount = to_buf ? 0 : planes, + .waitSemaphoreCount = to_buf ? planes : 0, }; if ((err = wait_start_exec_ctx(hwfc, ectx))) From patchwork Tue Aug 31 07:24:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29898 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4685519iov; Tue, 31 Aug 2021 00:27:30 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx1Iut10d7epzx057U/Yopvv19k5LtIHzQgQNJmgKy5KlilevMIUku2XiRbNhiLpmT8gb+O X-Received: by 2002:a17:906:63ca:: with SMTP id u10mr28869342ejk.411.1630394850583; Tue, 31 Aug 2021 00:27:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630394850; cv=none; d=google.com; s=arc-20160816; b=OOtXqA3kJVLTlTL4BSJlsXyDpq3EyNlOM8kObvbgEXyW4cS1gg6MLu6P+Ls0kkDKU/ CTJvSVvLo617LFCkUQURQB+pOkXnPsaUFLNyIYOkNVnVW3nfwTBq/hdtkq5OBxsMqTKZ UdQGF3yK95H6nkc3iLf4wIlP1nocXBMFVaz7nOuESxrgaadO7xuu/PmcpjpPb5nbD03M bgCoAsBJbN/Cofft+wcxwgV3I7l/8NyLokWYInRAfyX21NqU6u7lIgX/W4V2jYL8lvSF ct2PM9nPn0u7EuVeG54MHma2cRQIAozoutylJeHgMAACj+tTQY47m8jqjlw9RM8hUdP4 cAtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=B9ZS4Q5STUEXuN7xPBnLS0OWXjWnqqhCX4dg195bg3Y=; b=xCfbjbWEzl+vpPlJRmTD9bMu2GS05OncBttcPqASA1XOa9Ku0/nIGyOTONR/yn8xFg xRqvIwdDt6EKjJVMKGppO0gPmP6Ftt6XUrFk/Y/EXUlA7jV3oA0H6lwvDCN4oHDhtGgo Y35TRRazUeLocb/MIiJprcmp+v9Tn4Ry2+aANge8IQhIRz/rKiqIwUf2s8/8W7zmXjuP BxjDemFFzVNKVZP3017EkEiJojMTqaOgKxYqkihcAK6lH8QU4PHaPM9ii25BYnk7B/fL /fcCVAcFcGGIvXNzDlenssRRdmNQqz4Ty2RX1iGBtTnAV76AU3bg8n2jesW0W4LyAJWT LRvw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id jg33si14183899ejc.234.2021.08.31.00.27.30; Tue, 31 Aug 2021 00:27:30 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 955A0689FFE; Tue, 31 Aug 2021 10:27:13 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 590D5689FFE for ; Tue, 31 Aug 2021 10:27:06 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="282133664" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="282133664" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Aug 2021 00:27:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="540905144" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by fmsmga002.fm.intel.com with ESMTP; 31 Aug 2021 00:27:03 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 15:24:24 +0800 Message-Id: <20210831072430.303090-4-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210831072430.303090-1-wenbin.chen@intel.com> References: <20210831072430.303090-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [V2 04/10] hwcontext_vaapi: Use PRIME_2 memory type for modifiers. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Bas Nieuwenhuizen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: /og9UARdDoHk From: Bas Nieuwenhuizen This way we can pass explicit modifiers in. Sometimes the modifier matters for the number of memory planes that libva accepts, in particular when dealing with driver-compressed textures. Furthermore the driver might not actually be able to determine the implicit modifier if all the buffer-passing has used explicit modifier. All these issues should be resolved by passing in the modifier, and for that we switch to using the PRIME_2 memory type. Tested with experimental radeonsi patches for modifiers and kmsgrab. Also tested with radeonsi without the patches to double-check it works without PRIME_2 support. v2: Cache PRIME_2 support to avoid doing two calls every time on libva drivers that do not support it. v3: Remove prime2_vas usage. Signed-off-by: Bas Nieuwenhuizen --- libavutil/hwcontext_vaapi.c | 158 ++++++++++++++++++++++++++---------- 1 file changed, 114 insertions(+), 44 deletions(-) diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index 83e542876d..75acc851d6 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -79,6 +79,9 @@ typedef struct VAAPIFramesContext { unsigned int rt_format; // Whether vaDeriveImage works. int derive_works; + // Caches whether VA_SURFACE_ATTRIB_MEM_TYPE_DRM_PRIME_2 is unsupported for + // surface imports. + int prime_2_import_unsupported; } VAAPIFramesContext; typedef struct VAAPIMapping { @@ -1022,32 +1025,17 @@ static void vaapi_unmap_from_drm(AVHWFramesContext *dst_fc, static int vaapi_map_from_drm(AVHWFramesContext *src_fc, AVFrame *dst, const AVFrame *src, int flags) { + VAAPIFramesContext *src_vafc = src_fc->internal->priv; AVHWFramesContext *dst_fc = (AVHWFramesContext*)dst->hw_frames_ctx->data; AVVAAPIDeviceContext *dst_dev = dst_fc->device_ctx->hwctx; const AVDRMFrameDescriptor *desc; const VAAPIFormatDescriptor *format_desc; VASurfaceID surface_id; - VAStatus vas; + VAStatus vas = VA_STATUS_SUCCESS; + int use_prime2; uint32_t va_fourcc; - int err, i, j, k; - - unsigned long buffer_handle; - VASurfaceAttribExternalBuffers buffer_desc; - VASurfaceAttrib attrs[2] = { - { - .type = VASurfaceAttribMemoryType, - .flags = VA_SURFACE_ATTRIB_SETTABLE, - .value.type = VAGenericValueTypeInteger, - .value.value.i = VA_SURFACE_ATTRIB_MEM_TYPE_DRM_PRIME, - }, - { - .type = VASurfaceAttribExternalBufferDescriptor, - .flags = VA_SURFACE_ATTRIB_SETTABLE, - .value.type = VAGenericValueTypePointer, - .value.value.p = &buffer_desc, - } - }; + int err, i, j; desc = (AVDRMFrameDescriptor*)src->data[0]; @@ -1083,35 +1071,117 @@ static int vaapi_map_from_drm(AVHWFramesContext *src_fc, AVFrame *dst, format_desc = vaapi_format_from_fourcc(va_fourcc); av_assert0(format_desc); - buffer_handle = desc->objects[0].fd; - buffer_desc.pixel_format = va_fourcc; - buffer_desc.width = src_fc->width; - buffer_desc.height = src_fc->height; - buffer_desc.data_size = desc->objects[0].size; - buffer_desc.buffers = &buffer_handle; - buffer_desc.num_buffers = 1; - buffer_desc.flags = 0; - - k = 0; - for (i = 0; i < desc->nb_layers; i++) { - for (j = 0; j < desc->layers[i].nb_planes; j++) { - buffer_desc.pitches[k] = desc->layers[i].planes[j].pitch; - buffer_desc.offsets[k] = desc->layers[i].planes[j].offset; - ++k; + use_prime2 = !src_vafc->prime_2_import_unsupported && + desc->objects[0].format_modifier != DRM_FORMAT_MOD_INVALID; + if (use_prime2) { + VADRMPRIMESurfaceDescriptor prime_desc; + VASurfaceAttrib prime_attrs[2] = { + { + .type = VASurfaceAttribMemoryType, + .flags = VA_SURFACE_ATTRIB_SETTABLE, + .value.type = VAGenericValueTypeInteger, + .value.value.i = VA_SURFACE_ATTRIB_MEM_TYPE_DRM_PRIME_2, + }, + { + .type = VASurfaceAttribExternalBufferDescriptor, + .flags = VA_SURFACE_ATTRIB_SETTABLE, + .value.type = VAGenericValueTypePointer, + .value.value.p = &prime_desc, + } + }; + prime_desc.fourcc = va_fourcc; + prime_desc.width = src_fc->width; + prime_desc.height = src_fc->height; + prime_desc.num_objects = desc->nb_objects; + for (i = 0; i < desc->nb_objects; ++i) { + prime_desc.objects[i].fd = desc->objects[i].fd; + prime_desc.objects[i].size = desc->objects[i].size; + prime_desc.objects[i].drm_format_modifier = + desc->objects[i].format_modifier; } - } - buffer_desc.num_planes = k; - if (format_desc->chroma_planes_swapped && - buffer_desc.num_planes == 3) { - FFSWAP(uint32_t, buffer_desc.pitches[1], buffer_desc.pitches[2]); - FFSWAP(uint32_t, buffer_desc.offsets[1], buffer_desc.offsets[2]); + prime_desc.num_layers = desc->nb_layers; + for (i = 0; i < desc->nb_layers; ++i) { + prime_desc.layers[i].drm_format = desc->layers[i].format; + prime_desc.layers[i].num_planes = desc->layers[i].nb_planes; + for (j = 0; j < desc->layers[i].nb_planes; ++j) { + prime_desc.layers[i].object_index[j] = + desc->layers[i].planes[j].object_index; + prime_desc.layers[i].offset[j] = desc->layers[i].planes[j].offset; + prime_desc.layers[i].pitch[j] = desc->layers[i].planes[j].pitch; + } + + if (format_desc->chroma_planes_swapped && + desc->layers[i].nb_planes == 3) { + FFSWAP(uint32_t, prime_desc.layers[i].pitch[1], + prime_desc.layers[i].pitch[2]); + FFSWAP(uint32_t, prime_desc.layers[i].offset[1], + prime_desc.layers[i].offset[2]); + } + } + + /* + * We can query for PRIME_2 support with vaQuerySurfaceAttributes, but that + * that needs the config_id which we don't have here . Both Intel and + * Gallium seem to do the correct error checks, so lets just try the + * PRIME_2 import first. + */ + vas = vaCreateSurfaces(dst_dev->display, format_desc->rt_format, + src->width, src->height, &surface_id, 1, + prime_attrs, FF_ARRAY_ELEMS(prime_attrs)); + if (vas != VA_STATUS_SUCCESS) + src_vafc->prime_2_import_unsupported = 1; } - vas = vaCreateSurfaces(dst_dev->display, format_desc->rt_format, - src->width, src->height, - &surface_id, 1, - attrs, FF_ARRAY_ELEMS(attrs)); + if (!use_prime2 || vas != VA_STATUS_SUCCESS) { + int k; + unsigned long buffer_handle; + VASurfaceAttribExternalBuffers buffer_desc; + VASurfaceAttrib buffer_attrs[2] = { + { + .type = VASurfaceAttribMemoryType, + .flags = VA_SURFACE_ATTRIB_SETTABLE, + .value.type = VAGenericValueTypeInteger, + .value.value.i = VA_SURFACE_ATTRIB_MEM_TYPE_DRM_PRIME, + }, + { + .type = VASurfaceAttribExternalBufferDescriptor, + .flags = VA_SURFACE_ATTRIB_SETTABLE, + .value.type = VAGenericValueTypePointer, + .value.value.p = &buffer_desc, + } + }; + + buffer_handle = desc->objects[0].fd; + buffer_desc.pixel_format = va_fourcc; + buffer_desc.width = src_fc->width; + buffer_desc.height = src_fc->height; + buffer_desc.data_size = desc->objects[0].size; + buffer_desc.buffers = &buffer_handle; + buffer_desc.num_buffers = 1; + buffer_desc.flags = 0; + + k = 0; + for (i = 0; i < desc->nb_layers; i++) { + for (j = 0; j < desc->layers[i].nb_planes; j++) { + buffer_desc.pitches[k] = desc->layers[i].planes[j].pitch; + buffer_desc.offsets[k] = desc->layers[i].planes[j].offset; + ++k; + } + } + buffer_desc.num_planes = k; + + if (format_desc->chroma_planes_swapped && + buffer_desc.num_planes == 3) { + FFSWAP(uint32_t, buffer_desc.pitches[1], buffer_desc.pitches[2]); + FFSWAP(uint32_t, buffer_desc.offsets[1], buffer_desc.offsets[2]); + } + + vas = vaCreateSurfaces(dst_dev->display, format_desc->rt_format, + src->width, src->height, + &surface_id, 1, + buffer_attrs, FF_ARRAY_ELEMS(buffer_attrs)); + } if (vas != VA_STATUS_SUCCESS) { av_log(dst_fc, AV_LOG_ERROR, "Failed to create surface from DRM " "object: %d (%s).\n", vas, vaErrorStr(vas)); From patchwork Tue Aug 31 07:24:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29893 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4685612iov; Tue, 31 Aug 2021 00:27:43 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzLFYfUWDsmt+KVuldoYmqKkJjCVh0vzZwhMbjWn4ALezpy1g/wa8hqth2pHg3uB53wipv0 X-Received: by 2002:a17:906:2541:: with SMTP id j1mr30797627ejb.128.1630394863169; Tue, 31 Aug 2021 00:27:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630394863; cv=none; d=google.com; s=arc-20160816; b=fU8eBpDrjc1iHgURRY3nNgWZyVhj+MFUHPHYK1wHW6dXGC6mw/bnW/yjPwXdeVx4hk IWexhGquWGmnH0xKU9lFIJSNRuzc8DoUZdwRLGAF1fHD3XxshMKjHNCIMkQqGmj60ZKB BZFMlShfO5whvyCChAr2SA9HKgnB6nga40g0+GIbQD9zsNyZxNx3VWGII+r5l5Wcne7B u1rJ3Rv9e4fYv3hJ2GwURBBJ7FaXhKzhvAuE6aI9IbOvNedaoRs4vdAVRcyfPYhRrkYu Ufe3lU10l5TFDpYDa27Xt8x1VYy8sUViTfsfhEq27u/FvoLIklUM/aSehnNBEPHqsb/T eHwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=JrvPegIXt3WVsBm98MngVNyOCmhchkjbCrtfuvo3WjU=; b=iOL2D8MEVhtAFBzlrpdyD7LEZgDxRM0xk1pHOctGkdv9+pV7BNx21E6GQSEk//Xcm6 LUNYwiXNPSO1WHtSKrLa2m9qYub+kgrvw7CXwy0QdSoR3aTzMLHnUhYRCIRTJ4fECF1I CFdYsQBiDCnkNOlgdrqfMitZFr5NNGEnp/f9avzZECnTHMBdSnSu/J+t+hOotwPc2Yt5 ldgVx9/Aya3ln3e2np0y8kph6rFm63FWaHzucTNEhrQOXiYWAW1eSJejy3PUPtSlYVHu xVHwo4otErGxUPWjVl3PFNQwM0p5BCKLcvE36sO1KXJByul00S6aXF/D3DFVtNrmbC7Q ddUw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b10si2202709ejj.523.2021.08.31.00.27.42; Tue, 31 Aug 2021 00:27:43 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 00D5668A3AF; Tue, 31 Aug 2021 10:27:32 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9791A68A2CE for ; Tue, 31 Aug 2021 10:27:28 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="279429262" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="279429262" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Aug 2021 00:27:23 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="540905206" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by fmsmga002.fm.intel.com with ESMTP; 31 Aug 2021 00:27:22 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 15:24:25 +0800 Message-Id: <20210831072430.303090-5-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210831072430.303090-1-wenbin.chen@intel.com> References: <20210831072430.303090-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [V2 05/10] libavutil/hwcontext_vaapi: Add a new nv12 format map to support vulkan frame X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 4Ci7n4ZoQBq7 Vulkan will map nv12 to R8 and GR88, so add this map to vaapi to support vulkan frame. Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vaapi.c | 1 + 1 file changed, 1 insertion(+) diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index 75acc851d6..994b744e4d 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -992,6 +992,7 @@ static const struct { } vaapi_drm_format_map[] = { #ifdef DRM_FORMAT_R8 DRM_MAP(NV12, 2, DRM_FORMAT_R8, DRM_FORMAT_RG88), + DRM_MAP(NV12, 2, DRM_FORMAT_R8, DRM_FORMAT_GR88), #endif DRM_MAP(NV12, 1, DRM_FORMAT_NV12), #if defined(VA_FOURCC_P010) && defined(DRM_FORMAT_R16) From patchwork Tue Aug 31 07:24:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29894 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4685739iov; Tue, 31 Aug 2021 00:27:57 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwoHcivRNocFyvVPb3atdIncNY3v0IkSqLr6yoApOPfkHf1n+Gd4VzdzJRuSD70NFWF2q8P X-Received: by 2002:a17:906:af9a:: with SMTP id mj26mr28735753ejb.96.1630394877021; Tue, 31 Aug 2021 00:27:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630394877; cv=none; d=google.com; s=arc-20160816; b=Sm17Y07RwTd+7zE7Y992qY7B2ogUjkN2XCGFYO+Oum+iJBvj1GS54eXgpobWPXeDp3 QrpVFlFGVVoclYIWM5A6fkh0h0/cB/jaOsMc4RZQHR36CyDtmVX8/zpywlt+OTiq8+gP 2KGM8Gq6LtnZW/R+OBsEpzeOqhk0vQ2ioiogJcCRbES4FxS3sSsuRiXQPLiwTKbTFuOP GSgLfU3Ws7QA1F1bEQ6W4Pyw94tEvYZ3frcHD1msFGpvBeXq5qpJcjx0SeGY6DjAgwVk DAOpmY2Ep0Xz3SMVBJbV1Lk+EJfmBEr99OyZwpwXm3RCQTviJrcpsuYZnIt8DPIrwBnO 0rlg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=IKDFRUS7vANt+ba5x26qWqGDK7i07WdTao5oR/WEBak=; b=PV8WkyG3QCggFY+NUkZn5l36p5rRGOHj5ep3xtnmceH4zUCJ/j7TfYTKJBXlH95Fdj KJREwA0YomDFKqWzcYlYvcvfORB+JTZO0khm196Fp9XogegFqrbkdcctnS0lUUNM0XRv tLYDL3PmLBMvFX94OoYb99693WVifQ8+nURU8WARBWhj531+RyBNJkkJ3IdhS7yylieh oEHg9ty6fwgSMAL5/EvOch1JUp5pSphJ3wlDjbG4ykGHd1E6uD+F+yE0z/qBktDFxmwP qWLfHZBgu2t/lUQuSBkNoL/XUA4v2LT3NN0IjvGjEasT/s14NFLdf5LmCuQErDdgfD52 SZTQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hr37si2376699ejc.264.2021.08.31.00.27.56; Tue, 31 Aug 2021 00:27:57 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 615FA68A42A; Tue, 31 Aug 2021 10:27:38 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AD1BB68A336 for ; Tue, 31 Aug 2021 10:27:29 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="279429271" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="279429271" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Aug 2021 00:27:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="540905217" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by fmsmga002.fm.intel.com with ESMTP; 31 Aug 2021 00:27:25 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 15:24:26 +0800 Message-Id: <20210831072430.303090-6-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210831072430.303090-1-wenbin.chen@intel.com> References: <20210831072430.303090-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [V2 06/10] libavutil/hwcontext_vulkan: Add one_memory flag to make vulkan compatible with vaapi device. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: sVVdcV5QKfz0 Vaapi can import external surface, but all the planes of the external frames should be in the same drm object. A new flag is introduced and vulkan can choose to allocate planes in one memory according this flag. This flag will be enabled when the vulkan device is derived from vaapi device, so that this change will not affect current vulkan behaviour. Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 9a29267aed..6417f59d4a 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -211,6 +211,9 @@ typedef struct VulkanDevicePriv { /* Settings */ int use_linear_images; + /* map all planes to one memory */ + int use_one_memory; + /* Nvidia */ int dev_is_nvidia; } VulkanDevicePriv; @@ -1321,6 +1324,11 @@ static int vulkan_device_create_internal(AVHWDeviceContext *ctx, if (opt_d) p->use_linear_images = strtol(opt_d->value, NULL, 10); + opt_d = av_dict_get(opts, "one_memory", NULL, 0); + if (opt_d) + p->use_one_memory = strtol(opt_d->value, NULL, 10); + + hwctx->enabled_dev_extensions = dev_info.ppEnabledExtensionNames; hwctx->nb_enabled_dev_extensions = dev_info.enabledExtensionCount; @@ -1443,8 +1451,10 @@ static int vulkan_device_derive(AVHWDeviceContext *ctx, return AVERROR_EXTERNAL; } - if (strstr(vendor, "Intel")) + if (strstr(vendor, "Intel")) { + av_dict_set_int(&opts, "one_memory", 1, 0); dev_select.vendor_id = 0x8086; + } if (strstr(vendor, "AMD")) dev_select.vendor_id = 0x1002; From patchwork Tue Aug 31 07:24:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29900 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4685850iov; Tue, 31 Aug 2021 00:28:08 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxVCOYGWCGPCmBfL3yBv7RbC3wA/mFkCTSbrwwIxPyxg/YkwI2hvawQew9OpbOESpQ58aaG X-Received: by 2002:a05:6402:344f:: with SMTP id l15mr28074108edc.56.1630394888731; Tue, 31 Aug 2021 00:28:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630394888; cv=none; d=google.com; s=arc-20160816; b=X8Vs7qzP+4ifuaHeqvybKVPW88gffmpR11wZjuyWENO9egaSOwvaZSTPSJEeD0iBz5 Y8rirlyNr6eU8CNu1GqSk9Lq2zrn9Uu8hyanRQ4e9gRNdyKHBcHv/0duUj1EiY6dUINs m2+Z5oMwTILSfGkpEWHEJ/CBD4SW9wiYK6C0UwicA35uOcUM4mkEAT4ckeeArwFCKS0i oZ/MjIAm9JAt79T97MOUZzJukT0/2z9PnKHP4Et26kOwzZTOKd7X05HaQXZ54mLhx1a3 iQdjIODl7nWrIiN5+wQbE/BiEf3moIbfjm7ajrj2hu95va6//vTQyajbfchNULhUWk51 y9GQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=OTpn5UCgJJRvcjUJZYHrvNtdmUpqnZNXKbajFAJNkHo=; b=Np6gZzmmG1RQmWTXEcF+alszpqXghPQ4FjB7uwLr/yvDPQSxHkTukKXMnj3a+hJ2AP qu2SVm1IvkLjKICLShSW88v83ganPtFkEBH2LySS/CeSuPssTk0OWFOqB5RnAtc9JwJu ll7CZiIk3UUUpKGv/4HkgmwwQkIOwqBEH5hqsFa9IS8g68EsjLUpYtehfRLs0fuRodNK di1GPsKlpZwvSbeMrF0p4DjvtHTT33OVJzBpUOmGssbLxVISJXn++cqdsLB6VAu67WU7 JkOOuGof2NfbSFiwKtqhKPE/EHvUmkvUfWSK/6UOV23zGjgRXDVo3qjN/kwfQD5gxL/d UlVw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id u6si18576348eju.564.2021.08.31.00.28.08; Tue, 31 Aug 2021 00:28:08 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 45CED68A450; Tue, 31 Aug 2021 10:27:39 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2B35E68A3BF for ; Tue, 31 Aug 2021 10:27:29 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="279429277" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="279429277" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Aug 2021 00:27:28 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="540905234" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by fmsmga002.fm.intel.com with ESMTP; 31 Aug 2021 00:27:27 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 15:24:27 +0800 Message-Id: <20210831072430.303090-7-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210831072430.303090-1-wenbin.chen@intel.com> References: <20210831072430.303090-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [V2 07/10] libavutil/hwcontext_vulkan: Allocate vkFrame in one memory X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: /toX6BMCE2KJ The vaapi can import external frame, but the planes of the external frames should be in the same drm object. I add a new function to allocate vkFrame in one memory and vulkan device will choose a way to allocate memory according to one_memory flag. A new variable is added to AVVKFrame to store the offset of each plane. Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 46 +++++++++++++++++++++++++++++++++++- libavutil/hwcontext_vulkan.h | 1 + 2 files changed, 46 insertions(+), 1 deletion(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 6417f59d4a..4983518a77 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -1667,6 +1667,9 @@ static int alloc_bind_mem(AVHWFramesContext *hwfc, AVVkFrame *f, VulkanFunctions *vk = &p->vkfn; const int planes = av_pix_fmt_count_planes(hwfc->sw_format); VkBindImageMemoryInfo bind_info[AV_NUM_DATA_POINTERS] = { { 0 } }; + VkMemoryRequirements memory_requirements = { 0 }; + int mem_size = 0; + int mem_size_list[AV_NUM_DATA_POINTERS] = { 0 }; AVVulkanDeviceContext *hwctx = ctx->hwctx; @@ -1694,6 +1697,23 @@ static int alloc_bind_mem(AVHWFramesContext *hwfc, AVVkFrame *f, req.memoryRequirements.size = FFALIGN(req.memoryRequirements.size, p->props.properties.limits.minMemoryMapAlignment); + if (p->use_one_memory) { + if (ded_req.prefersDedicatedAllocation | ded_req.requiresDedicatedAllocation) { + av_log(hwfc, AV_LOG_ERROR, "Cannot use dedicated allocation for intel vaapi\n"); + return AVERROR(EINVAL); + } + if (memory_requirements.size == 0) { + memory_requirements = req.memoryRequirements; + } else if (memory_requirements.memoryTypeBits != req.memoryRequirements.memoryTypeBits) { + av_log(hwfc, AV_LOG_ERROR, "the param for each planes are not the same\n"); + return AVERROR(EINVAL); + } + + mem_size_list[i] = req.memoryRequirements.size; + mem_size += mem_size_list[i]; + continue; + } + /* In case the implementation prefers/requires dedicated allocation */ use_ded_mem = ded_req.prefersDedicatedAllocation | ded_req.requiresDedicatedAllocation; @@ -1715,6 +1735,29 @@ static int alloc_bind_mem(AVHWFramesContext *hwfc, AVVkFrame *f, bind_info[i].memory = f->mem[i]; } + if (p->use_one_memory) { + memory_requirements.size = mem_size; + + /* Allocate memory */ + if ((err = alloc_mem(ctx, &memory_requirements, + f->tiling == VK_IMAGE_TILING_LINEAR ? + VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT : + VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT, + (void *)(((uint8_t *)alloc_pnext)), + &f->flags, &f->mem[0]))) + return err; + + f->size[0] = memory_requirements.size; + + for (int i = 0; i < planes; i++) { + bind_info[i].sType = VK_STRUCTURE_TYPE_BIND_IMAGE_MEMORY_INFO; + bind_info[i].image = f->img[i]; + bind_info[i].memory = f->mem[0]; + bind_info[i].memoryOffset = i == 0 ? 0 : mem_size_list[i-1]; + f->offset[i] = bind_info[i].memoryOffset; + } + } + /* Bind the allocated memory to the images */ ret = vk->BindImageMemory2(hwctx->act_dev, planes, bind_info); if (ret != VK_SUCCESS) { @@ -2921,7 +2964,8 @@ static int vulkan_map_to_drm(AVHWFramesContext *hwfc, AVFrame *dst, continue; vk->GetImageSubresourceLayout(hwctx->act_dev, f->img[i], &sub, &layout); - drm_desc->layers[i].planes[0].offset = layout.offset; + drm_desc->layers[i].planes[0].offset = p->use_one_memory ? + f->offset[i] : layout.offset; drm_desc->layers[i].planes[0].pitch = layout.rowPitch; } diff --git a/libavutil/hwcontext_vulkan.h b/libavutil/hwcontext_vulkan.h index e4645527d7..8fb25d1485 100644 --- a/libavutil/hwcontext_vulkan.h +++ b/libavutil/hwcontext_vulkan.h @@ -182,6 +182,7 @@ typedef struct AVVkFrame { */ VkDeviceMemory mem[AV_NUM_DATA_POINTERS]; size_t size[AV_NUM_DATA_POINTERS]; + size_t offset[AV_NUM_DATA_POINTERS]; /** * OR'd flags for all memory allocated From patchwork Tue Aug 31 07:24:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29899 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4686134iov; Tue, 31 Aug 2021 00:28:38 -0700 (PDT) X-Google-Smtp-Source: ABdhPJww1+hKKQfioUwyFqQDxCQOa/tNhQX1Ez3WsghM/08jPXXsonnGVuxNftfm6S1V3X18WowI X-Received: by 2002:a17:907:1184:: with SMTP id uz4mr28363217ejb.363.1630394918017; Tue, 31 Aug 2021 00:28:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630394918; cv=none; d=google.com; s=arc-20160816; b=n2iN08VJfkTLR1NJJj0h5TSrmySpzr09GdSfRmqhBR9b7h4AF/fyDvXBG9rv1tZRm9 J2lh/Ip/Kifhs3YIDZO1Ral4eoI/QetnSHSS0C1UBvku9OmFg9aR7Qno15OAIGismDFE v4NPkWYjbuvrE3Lvjyg+tohKc7QL+qstygDwWvMuELUGZUDXbr4Ay1ilxH7iGO9vaWPk Ux8V/4Y1D1xkKDCDNnuAIU7fJztLow2O8jydKrsMK8a/+fRbxx4OcrAOYlsgiC3zpGGL C6tf6SsIKcrzEuAv1qcTVCTwzn/X8bouagfJ+rzdITiJ5camJ2+txUc+I0Z7t60DLJVg COpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=4f7js9w3HHnryEJrUc9dx/RMZPRbfWnVVhtAIcrC+kY=; b=C7aKUHlcxKWxQdhkkl2E/qqI+xhbyjYUsKRNvJqxriCPqFjfer1miIUTPK2N4W6wwT B5T0yxHSTj1rwA2iRSmI3lbBRkXav1HqNlKqTMQXyDWA2ryb7tAseAtR3bzuYTDTaUiP KSnCzl2YlmO0jaNoQ8CXyWYOV3eoPQQq9FAwR1ZA7w8g+gIU7S4iMPNZxFmIVibgDlPg iD9Eivazz4a853CuO1WmyBTx6o21GdVXMKF1mdVkJtpmtQw1bC35uVZwV4o/VpxxuiOf 2ku/xNFPihXUzBvIXovCxPdBrCi9sgIYtDvOoeOzOZ/tpGAaA11xLAlJxpd3zNv5gjvd q7fg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 10si16932369ejw.91.2021.08.31.00.28.37; Tue, 31 Aug 2021 00:28:38 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4A08168A4D2; Tue, 31 Aug 2021 10:27:43 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7800A68A43A for ; Tue, 31 Aug 2021 10:27:33 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="279429282" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="279429282" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Aug 2021 00:27:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="540905242" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by fmsmga002.fm.intel.com with ESMTP; 31 Aug 2021 00:27:29 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 15:24:28 +0800 Message-Id: <20210831072430.303090-8-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210831072430.303090-1-wenbin.chen@intel.com> References: <20210831072430.303090-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [V2 08/10] libavutil/hwcontext_vulkan: fix wrong offset of plane X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: FrhzHv2HbDoa According to spec, if we use VkBindImagePlaneMemoryInfo to bind image we mush create image with disjoint flag. The offset in subresourcelayout is relative to the base address of the plane, but the offset in drm is relative to the drm objectis so I think this offset should be 0. Also, when I import vaapi frame to vulkan I got broken frame, and setting plane_data->offset to 0 makes command works. Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 4983518a77..3a639c997b 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -2382,7 +2382,9 @@ static int vulkan_map_from_drm_frame_desc(AVHWFramesContext *hwfc, AVVkFrame **f .extent.depth = 1, .mipLevels = 1, .arrayLayers = 1, - .flags = VK_IMAGE_CREATE_ALIAS_BIT, + .flags = VK_IMAGE_CREATE_ALIAS_BIT | + (has_modifiers && planes > 1) ? VK_IMAGE_CREATE_DISJOINT_BIT : + 0, .tiling = f->tiling, .initialLayout = VK_IMAGE_LAYOUT_UNDEFINED, /* specs say so */ .usage = frames_hwctx->usage, @@ -2397,7 +2399,7 @@ static int vulkan_map_from_drm_frame_desc(AVHWFramesContext *hwfc, AVVkFrame **f hwfc->sw_format, src->width, src->height, i); for (int j = 0; j < planes; j++) { - plane_data[j].offset = desc->layers[i].planes[j].offset; + plane_data[j].offset = 0; plane_data[j].rowPitch = desc->layers[i].planes[j].pitch; plane_data[j].size = 0; /* The specs say so for all 3 */ plane_data[j].arrayPitch = 0; From patchwork Tue Aug 31 07:24:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29896 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4685948iov; Tue, 31 Aug 2021 00:28:19 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxmXjiHAyn0jZHQJB4DqPA2HB+FpDRun04XOEghQ79/dFYDFV88pfxfTdiyLJzyyneR/sd4 X-Received: by 2002:a17:906:1f54:: with SMTP id d20mr29443076ejk.48.1630394898936; Tue, 31 Aug 2021 00:28:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630394898; cv=none; d=google.com; s=arc-20160816; b=dDKtNNPukM0974scndGXnBnJ+YQRdFrUxStAh1EG7EXrVAsemcCxgd7WnGK3g88hJG 5F+lja0Pe72kdmaFKSaB7plL59ooZjIRG1nTiijpwQZns3eyUcx3D6PIv2l1YQ0kAwQs 3oBSMD5U4IBMwKdHvLUZc5hCJvTVunrSsARJtL54sO5p+gUvJajpbXSWQ1fMncQ0hip6 DZB8evUrha1zk7OGrQPxr0vERURsR4/otooya5tngp+jIov0N38etLQySKrWRnTpOlFF vzp64jL2bPSf1GzCGPTVtGUX5KiIlj/CMOB0JBxAT/Ys+5jCYc9VXbt1vV2GRI+SP/Xs wMjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=c+/BEtC4CXQjc5oKmxZr7sStgBhbiLjjFrZ9cUtEtCc=; b=cBKpouIS1jkT8Nbrgq64wV8qAgPktYICvLIgaetF+lFDqZrZaYBCa9wDe4oM+GADwD EKBWaHkfqxIfOa5RDxSZ7QQAEhRiMIEsiNKbbLoCHOPOrR5SJQ8+jUblgVR5ovPk31Z0 yTbmLVCgcta3lzgRLWCH24Gv7FlwFHTe3uHtHeFzSyaE+/7aYcrCyYLZvms66sRWtZWf vvPM26iK76FW48bZOeRvkRYXJAL5OdqzuWRr5Dt1RJoQzyhTPYQ9ot02AYDTYBQ24i04 91B3JZVRt30wbXHHuF9XknIyESZEP18Sw3IhhidZ/vmcBe2P4/O+TaAPI1KGC3ah08nC V3vA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ar21si15921329ejc.460.2021.08.31.00.28.18; Tue, 31 Aug 2021 00:28:18 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4372868A4A3; Tue, 31 Aug 2021 10:27:41 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0A03D68A3BF for ; Tue, 31 Aug 2021 10:27:36 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="279429285" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="279429285" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Aug 2021 00:27:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="540905247" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by fmsmga002.fm.intel.com with ESMTP; 31 Aug 2021 00:27:31 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 15:24:29 +0800 Message-Id: <20210831072430.303090-9-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210831072430.303090-1-wenbin.chen@intel.com> References: <20210831072430.303090-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [V2 09/10] libavutil/hwcontext_vulkan: specify the modifier to create VKImage X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: FrVuUYy6IUPC On the lastset intel-vulkan-driver the VK_EXT_image_drm_format_modifier flags is enabled. As what driver log recommand, we need to use VK_IMAGE_TILING_LINEAR or VK_IMAGE_DRM_FORMAT_MODIFIER_EXT to create VKImage. Add code to get supported modifier for sw_format and use these modifier to create VKImage. Now the following command line works: ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i input_1080p.264 -vf "hwmap=derive_device=vulkan,format=vulkan, scale_vulkan=1920:1080,hwmap=derive_device=vaapi,format=vaapi" -c:v h264_vaapi output.264 Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 77 +++++++++++++++++++++++++++++++++--- libavutil/hwcontext_vulkan.h | 5 +++ 2 files changed, 76 insertions(+), 6 deletions(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 3a639c997b..99b2190dc3 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -1967,6 +1967,8 @@ static void try_export_flags(AVHWFramesContext *hwfc, AVVulkanDeviceContext *dev_hwctx = hwfc->device_ctx->hwctx; VulkanDevicePriv *p = hwfc->device_ctx->internal->priv; VulkanFunctions *vk = &p->vkfn; + const int has_modifiers = hwctx->tiling == VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT; + VkExternalImageFormatProperties eprops = { .sType = VK_STRUCTURE_TYPE_EXTERNAL_IMAGE_FORMAT_PROPERTIES_KHR, }; @@ -1974,9 +1976,18 @@ static void try_export_flags(AVHWFramesContext *hwfc, .sType = VK_STRUCTURE_TYPE_IMAGE_FORMAT_PROPERTIES_2, .pNext = &eprops, }; + VkPhysicalDeviceImageDrmFormatModifierInfoEXT phy_dev_mod_info = { + .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_IMAGE_DRM_FORMAT_MODIFIER_INFO_EXT, + .pNext = NULL, + .pQueueFamilyIndices = p->qfs, + .queueFamilyIndexCount = p->num_qfs, + .sharingMode = p->num_qfs > 1 ? VK_SHARING_MODE_CONCURRENT : + VK_SHARING_MODE_EXCLUSIVE, + }; VkPhysicalDeviceExternalImageFormatInfo enext = { .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_EXTERNAL_IMAGE_FORMAT_INFO, .handleType = exp, + .pNext = has_modifiers ? &phy_dev_mod_info : NULL, }; VkPhysicalDeviceImageFormatInfo2 pinfo = { .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_IMAGE_FORMAT_INFO_2, @@ -1988,11 +1999,15 @@ static void try_export_flags(AVHWFramesContext *hwfc, .flags = VK_IMAGE_CREATE_ALIAS_BIT, }; - ret = vk->GetPhysicalDeviceImageFormatProperties2(dev_hwctx->phys_dev, - &pinfo, &props); - if (ret == VK_SUCCESS) { - *iexp |= exp; - *comp_handle_types |= eprops.externalMemoryProperties.compatibleHandleTypes; + for (int i = 0; i < (has_modifiers ? hwctx->modifier_count : 1); i++) { + if (has_modifiers && hwctx->modifier_count) + phy_dev_mod_info.drmFormatModifier = hwctx->modifiers[i]; + ret = vk->GetPhysicalDeviceImageFormatProperties2(dev_hwctx->phys_dev, + &pinfo, &props); + if (ret == VK_SUCCESS) { + *iexp |= exp; + *comp_handle_types |= eprops.externalMemoryProperties.compatibleHandleTypes; + } } } @@ -2055,6 +2070,7 @@ fail: static void vulkan_frames_uninit(AVHWFramesContext *hwfc) { VulkanFramesPriv *fp = hwfc->internal->priv; + AVVulkanFramesContext *hwctx = hwfc->hwctx; free_exec_ctx(hwfc, &fp->conv_ctx); free_exec_ctx(hwfc, &fp->upload_ctx); @@ -2069,11 +2085,60 @@ static int vulkan_frames_init(AVHWFramesContext *hwfc) VulkanFramesPriv *fp = hwfc->internal->priv; AVVulkanDeviceContext *dev_hwctx = hwfc->device_ctx->hwctx; VulkanDevicePriv *p = hwfc->device_ctx->internal->priv; + const int has_modifiers = !!(p->extensions & EXT_DRM_MODIFIER_FLAGS); /* Default pool flags */ - hwctx->tiling = hwctx->tiling ? hwctx->tiling : p->use_linear_images ? + hwctx->tiling = hwctx->tiling ? hwctx->tiling : has_modifiers ? + VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT : p->use_linear_images ? VK_IMAGE_TILING_LINEAR : VK_IMAGE_TILING_OPTIMAL; + /* get the supported modifier */ + if (has_modifiers) { + const VkFormat *fmt = av_vkfmt_from_pixfmt(hwfc->sw_format); + VulkanFunctions *vk = &p->vkfn; + VkDrmFormatModifierPropertiesEXT mod_props[MAX_VULKAN_MODIFIERS]; + + VkDrmFormatModifierPropertiesListEXT mod_props_list = { + .sType = VK_STRUCTURE_TYPE_DRM_FORMAT_MODIFIER_PROPERTIES_LIST_EXT, + .pNext = NULL, + .drmFormatModifierCount = 0, + .pDrmFormatModifierProperties = NULL, + }; + VkFormatProperties2 prop = { + .sType = VK_STRUCTURE_TYPE_FORMAT_PROPERTIES_2, + .pNext = &mod_props_list, + }; + + vk->GetPhysicalDeviceFormatProperties2(dev_hwctx->phys_dev, fmt[0], &prop); + if (!mod_props_list.drmFormatModifierCount) { + av_log(hwfc, AV_LOG_ERROR, "There are not supported modifiers for sw_format\n"); + return AVERROR(EINVAL); + } + mod_props_list.drmFormatModifierCount = + FFMIN(mod_props_list.drmFormatModifierCount, MAX_VULKAN_MODIFIERS); + + mod_props_list.pDrmFormatModifierProperties = mod_props; + vk->GetPhysicalDeviceFormatProperties2(dev_hwctx->phys_dev, fmt[0], &prop); + + hwctx->modifier_count = 0; + for (int i = 0; i < mod_props_list.drmFormatModifierCount; i++) { + if (!(mod_props[i].drmFormatModifierTilingFeatures & DEFAULT_USAGE_FLAGS)) + continue; + hwctx->modifiers[hwctx->modifier_count++] = mod_props[i].drmFormatModifier; + } + if (!hwctx->modifier_count) { + av_log(hwfc, AV_LOG_ERROR, "The supported modifiers doesn't support" + "default usage\n"); + return AVERROR(EINVAL); + } + + hwctx->create_pnext = &hwctx->modifier_info; + hwctx->modifier_info.pNext = NULL; + hwctx->modifier_info.sType = VK_STRUCTURE_TYPE_IMAGE_DRM_FORMAT_MODIFIER_LIST_CREATE_INFO_EXT; + hwctx->modifier_info.drmFormatModifierCount = hwctx->modifier_count; + hwctx->modifier_info.pDrmFormatModifiers = hwctx->modifiers; + } + if (!hwctx->usage) hwctx->usage = DEFAULT_USAGE_FLAGS; diff --git a/libavutil/hwcontext_vulkan.h b/libavutil/hwcontext_vulkan.h index 8fb25d1485..95fd767d1b 100644 --- a/libavutil/hwcontext_vulkan.h +++ b/libavutil/hwcontext_vulkan.h @@ -32,6 +32,7 @@ * with the data pointer set to an AVVkFrame. */ +#define MAX_VULKAN_MODIFIERS 16 /** * Main Vulkan context, allocated as AVHWDeviceContext.hwctx. * All of these can be set before init to change what the context uses @@ -142,6 +143,10 @@ typedef struct AVVulkanFramesContext { */ void *create_pnext; + VkImageDrmFormatModifierListCreateInfoEXT modifier_info; + uint64_t modifiers[MAX_VULKAN_MODIFIERS]; + int modifier_count; + /** * Extension data for memory allocation. Must have as many entries as * the number of planes of the sw_format. From patchwork Tue Aug 31 07:24:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29895 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4686054iov; Tue, 31 Aug 2021 00:28:28 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyl0Eqe+16/0e+za9Eo80vAWbSYqzl2eHr70QEMgG/y5i3G5xE0CmnN9GntuiJTVaJEeGik X-Received: by 2002:a17:906:2bc2:: with SMTP id n2mr29072475ejg.455.1630394908610; Tue, 31 Aug 2021 00:28:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630394908; cv=none; d=google.com; s=arc-20160816; b=IxCyzDApn7qCCUKSLuCv2ssKyu361h/94rP2mjYz7euF8/pNHEjOqw+sjosG5ZZWi4 5IZAV7ES+D6xPxVgK24HZb+ywAsXnf4kug/kUxlPU1l1bKZaZVUByeuL0EDPkwlWVEKm Ao6SrryjImvOghVlka93oX3I4pQQsooiCaQHCC2rqAcjw5C14bHqr6jmFQxL0oQvAdQ7 FoOJEREVJbXPuhZ1FEzD4A3zvWiHFXF1+8NgvnyxgV+x9goBY9SHSyClWHTPOKr365Oi dzqPUADedBqaPRHzKA5LRKNB0S7CULMzEHtWLtFJW/DY8ZNLcpb6tuCV4plnmeyNpizu Wevg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=tWDNRD994RG+N+QFv8oiVU3udzISD5V7CTXnHlSP0XY=; b=U2EMqE+93FhTpqqadd3z9giqVIDn/obbJNp6B+5V5k4JXxeyOzFsoavAqcGuAV2nIb qmENUHcBhGKfpnb8k9PmM8a5kUqZV2hxkul97gk3Jbbq/BrJqJYCDeyvOi2LKRIstNzj L60PhsoL7CAWNs0U1SV6HwrvnH0YSy78ifUUJvb3i4V12SeLb8EwIH3DjdQxV4zHbTw/ riVIxu8hDYvihlP527musT/8qQ2seWBjCayEk/dVy+hRl0LWI3sf5ICStgRzA45lJadI jhgAecZene01LvFPTjVgrbyouKIaCsFpCFUqV+au1HJJD+ZuJSnCtiEwSSRJ272EVUYR T3qg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id l7si10707559edb.409.2021.08.31.00.28.28; Tue, 31 Aug 2021 00:28:28 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 482BF68A4C2; Tue, 31 Aug 2021 10:27:42 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 299FC68A408 for ; Tue, 31 Aug 2021 10:27:36 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="279429290" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="279429290" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Aug 2021 00:27:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="540905267" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by fmsmga002.fm.intel.com with ESMTP; 31 Aug 2021 00:27:33 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 15:24:30 +0800 Message-Id: <20210831072430.303090-10-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210831072430.303090-1-wenbin.chen@intel.com> References: <20210831072430.303090-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [V2 10/10] libavutil/hwcontext_vulkan: Add hwupload and hwdownload support when using one_memory flag. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: vK2M+FEHmLdE Add hwupload and hwdownload support to vulkan when frames are allocated in one memory Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 99b2190dc3..5dfa7adc6b 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -2252,7 +2252,7 @@ static int vulkan_map_frame_to_mem(AVHWFramesContext *hwfc, AVFrame *dst, const AVFrame *src, int flags) { VkResult ret; - int err, mapped_mem_count = 0; + int err, mapped_mem_count = 0, loop = 0; AVVkFrame *f = (AVVkFrame *)src->data[0]; AVVulkanDeviceContext *hwctx = hwfc->device_ctx->hwctx; const int planes = av_pix_fmt_count_planes(hwfc->sw_format); @@ -2281,7 +2281,8 @@ static int vulkan_map_frame_to_mem(AVHWFramesContext *hwfc, AVFrame *dst, dst->width = src->width; dst->height = src->height; - for (int i = 0; i < planes; i++) { + loop = p->use_one_memory ? 1 : planes; + for (int i = 0; i < loop; i++) { ret = vk->MapMemory(hwctx->act_dev, f->mem[i], 0, VK_WHOLE_SIZE, 0, (void **)&dst->data[i]); if (ret != VK_SUCCESS) { @@ -2292,6 +2293,11 @@ static int vulkan_map_frame_to_mem(AVHWFramesContext *hwfc, AVFrame *dst, } mapped_mem_count++; } + if (p->use_one_memory) { + for (int i = 0; i < planes; i++) { + dst->data[i] = dst->data[0] + f->offset[i]; + } + } /* Check if the memory contents matter */ if (((flags & AV_HWFRAME_MAP_READ) || !(flags & AV_HWFRAME_MAP_OVERWRITE)) &&