From patchwork Tue Aug 31 01:43:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29883 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4497223iov; Mon, 30 Aug 2021 18:46:04 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyJ41oUTg3/YaBIRln8jrkriAJTBQhpG9dUu4nlE7ssraeGYLatuEcaGOJZuur0Pq84ImN8 X-Received: by 2002:a17:906:3146:: with SMTP id e6mr27863453eje.296.1630374364156; Mon, 30 Aug 2021 18:46:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630374364; cv=none; d=google.com; s=arc-20160816; b=yvhWrgnYnVpNcFI4l0gtYT8P8fVkt5ERwyQKCaY429xP0gnsUPuT48Hz4oBFYKYDVb 50wB3dFL92wS0LAB/cp21te6j5D201HpNVFNJE7+pLuMgKThl8ZkYRkGntwAMZrQmZfy 54InQKsSKR2Tm+DhO2fiDc49c1qF4EVPXt36Sb8txyyhJl00JDmaqnckqDvEklar7vcW n4XUdUu+PIoKxXqN3TnX6R2sLkch8WPq+qzBrdYYVlEiVR2jmL5Q0U6H5fwxUxBtMvzU wtOSAAt1u77sfY6ytKlsxEXJVmNIRIddBt6DOCH/cvuW4gt2agSlMCCWS/CkVK6Fd2Qv qkRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=njcsD77SC/Dionl0xjX0KHezo+vMzyW0V1KIJOwBnHc=; b=Q/AUPxF95sb8aHTiIu/gGYxq/nZlT6RUGMDoj/xRSyF3xtOYOiXKcEX4RvhXuwAwuB V7onFQzNlc4pX+B/nwyFXhNCqP2DIodR2EsAJ3EsoF50oTTJXPlU7CdxC/PU48SLQbVP xtkPyMibvHzShmXJudTqW7O0FI3ZAxUbOLrTsorx3ISfuLwUG+LDn4r+JEOWHt5kfZQ+ BgG6RcJaFF5ewtNlyoAMBjNofd8SXyGPt0Hr0VlMET0USpi73DVhsWAxghCKFT7Q9qpC o8PAfP7YXVtPyzWVFxcOBiZBJoPY77SRReJp9PJBhdtnmrpfGEvPe/KqLi3bMF9x/CiB aNBw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id la20si19279201ejc.3.2021.08.30.18.45.59; Mon, 30 Aug 2021 18:46:04 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EA75468A066; Tue, 31 Aug 2021 04:45:55 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 16EC3689738 for ; Tue, 31 Aug 2021 04:45:47 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="216531693" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="216531693" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Aug 2021 18:45:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="530827015" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by FMSMGA003.fm.intel.com with ESMTP; 30 Aug 2021 18:45:40 -0700 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 09:43:29 +0800 Message-Id: <20210831014338.134086-1-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 01/10] libavfilter/vulkan: Fix problem when device have queue_count greater than 1 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: "Chen,Wenbin" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: dlcl/fKmAsJc From: "Chen,Wenbin" If the descriptorSetCount is greater than the number of setLayouts, vkAllocateDescriptorSets will report error. Now fix it. Now the following command can run on the device that has queue_count greater than one: ffmpeg -v verbose -init_hw_device vulkan=vul:0 -filter_hw_device vul -i input1080p.264 -vf "hwupload=extra_hw_frames=16,scale_vulkan=1920:1080, hwdownload,format=yuv420p" -f rawvideo output.yuv Signed-off-by: Wenbin Chen --- libavfilter/vulkan.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavfilter/vulkan.c b/libavfilter/vulkan.c index 337c8d7d5a..e5b070b3e6 100644 --- a/libavfilter/vulkan.c +++ b/libavfilter/vulkan.c @@ -1160,7 +1160,7 @@ void ff_vk_update_descriptor_set(AVFilterContext *avctx, VulkanPipeline *pl, VulkanFilterContext *s = avctx->priv; vkUpdateDescriptorSetWithTemplate(s->hwctx->act_dev, - pl->desc_set[s->cur_queue_idx * pl->desc_layout_num + set_id], + pl->desc_set[set_id], pl->desc_template[set_id], s); } @@ -1179,7 +1179,7 @@ int ff_vk_init_pipeline_layout(AVFilterContext *avctx, VulkanPipeline *pl) VkResult ret; VulkanFilterContext *s = avctx->priv; - pl->descriptor_sets_num = pl->desc_layout_num * s->queue_count; + pl->descriptor_sets_num = pl->desc_layout_num; { /* Init descriptor set pool */ VkDescriptorPoolCreateInfo pool_create_info = { From patchwork Tue Aug 31 01:43:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29887 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4497280iov; Mon, 30 Aug 2021 18:46:10 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxhe789w0vK2dzJrXSS3FanBFk2XPnLan1kEojzrQ9gw5fPyB8bUyHVr7GxO+U3ZHTqOGUu X-Received: by 2002:a05:6402:705:: with SMTP id w5mr27247367edx.344.1630374370680; Mon, 30 Aug 2021 18:46:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630374370; cv=none; d=google.com; s=arc-20160816; b=MDYk0AKAVIqX+ghCDvBcU5bhPUPjzTQohKvEnRPKfB3Q9TMvM4F6NcL+EpEcwOQOq1 rCnxSv0N3DXkwB47ZvX3zCaeF2R4/u7Rl9GUXQg8neJ/NMo715eu70ptq3C7Ii2Wu7vx pkqLTCs5uJulq/HjSUV9Vh/hXHZAbIBy0FEGfH4e8pq8BkbMuQAELsbjNfBsJPeNkE/5 EcuuV2/XgFhDMYhaDvEFVs1pCFPmDql/ow/u/dl9bdvrK08cV7w/zUkcXLbq88cM3L7+ Sc6Is3a+6OxK0YXYQq/ullKM4LkwZl0lJdBNHDcaJHtdlqJrYOh1WRA94/ttg75eqF9x CYww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=kVgwr5ZeHboqzR8FCVefUaNEBf2O/bez/rC42kUuE/8=; b=YHcgK85CPm/3H1yigd4bqgTnTf2asmLOyj/wXgS+I7z0qK7I5pr5bpw//h+zDPUBZA HEgjgqjRAeKyMJ00nB3P7qK2vvQeR+kiN82szMhgHvC7KVNC/4SerMMo0f02ZPkxZCkr P6fcC6S0It3Mxg0Yp/9sSsYo9+R46q6wHiwBGvxAPisdlnW5RUVCe+YwL38GwMb73jDN 3nLXOvvRU21F+903q9LKqUlPPzQrHmuGRHa6Msx3Io7KZUrcJOHcBLpPAuyk94eM4Kge x9hRNZkmNpRAn6zoDHH7GWJybwFtlY4DYML/aJGYFzsrADR+6bYQbRekpEw6waSjktMP efhA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y18si15660117ejh.298.2021.08.30.18.46.10; Mon, 30 Aug 2021 18:46:10 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1F5E568A2BD; Tue, 31 Aug 2021 04:45:58 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B98F5689FB2 for ; Tue, 31 Aug 2021 04:45:50 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="216531701" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="216531701" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Aug 2021 18:45:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="530827029" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by FMSMGA003.fm.intel.com with ESMTP; 30 Aug 2021 18:45:42 -0700 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 09:43:30 +0800 Message-Id: <20210831014338.134086-2-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210831014338.134086-1-wenbin.chen@intel.com> References: <20210831014338.134086-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 02/10] libavutil/hwcontext_vulkan: fix a tile mismatch problem X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: "Chen,Wenbin" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: DpZ0OJFYzS1e From: "Chen,Wenbin" We should configure VkImageSubresource according to tiling rather than extension. We use extension to set tiling only when we map from drm. Normally the output VkImages are not created in this way, and it will report error when we map these VkImage to drm, so we should configure VkImageSubresource according to tiling rather than extension. Now fix it. Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 94fdad7f06..88db5b8b70 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -2888,7 +2888,7 @@ static int vulkan_map_to_drm(AVHWFramesContext *hwfc, AVFrame *dst, for (int i = 0; i < drm_desc->nb_layers; i++) { VkSubresourceLayout layout; VkImageSubresource sub = { - .aspectMask = p->extensions & EXT_DRM_MODIFIER_FLAGS ? + .aspectMask = f->tiling == VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT ? VK_IMAGE_ASPECT_MEMORY_PLANE_0_BIT_EXT : VK_IMAGE_ASPECT_COLOR_BIT, }; From patchwork Tue Aug 31 01:43:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29886 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4497390iov; Mon, 30 Aug 2021 18:46:21 -0700 (PDT) X-Google-Smtp-Source: ABdhPJztvlCEC8SxwjiiV2Cr3+nD0j6sSMF+mrkXLZZ839K7Qwzcrdf5QWmCj8mL6/64yluaFoeG X-Received: by 2002:a17:906:32d6:: with SMTP id k22mr27695936ejk.228.1630374380978; Mon, 30 Aug 2021 18:46:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630374380; cv=none; d=google.com; s=arc-20160816; b=K78l7+zrZ79K6D8rUDw4YOVIryRTyyjDwY4feGhIy3jDdiorziI8D8FyYvqaGUR+80 tAxBTGKUcUSoUYDgqKSuBwmDWc/FsDktNyg/UvPJxaKQjgbvwEvCdF/7r/sgSqGkk/iS +C5fcJgeIaFQzszDZVCrEz/IomWxyGhSXsyV1qAUjNI+UD4Qm5f/PZD4b+DQU3B5BsG8 lHmajtqFBtgmrr8K+9/LCDIVN0GukN62cqYmOXr/lq3BNGiR471JUjsGVTTiYh8f9yCJ cE5tIn6OQSROaPHnc8W/SOkgW5HoXLcWfnyH6ji2U/2aWwBmbLaxB0DDzYh28TjngZkt +bBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=+bqJiFyQj69iaUrv26zESv0td4ek5hIvRFVIPUHYNao=; b=B5qFPxLRU0V8HoxkVhRs2pIM+X8Qc+9Tp9uOr8F3oqBb411E7JGHrKrIeVs+HiL5Bc razb+tDHyl3ca90DA8D8g6IIo6dutRYW7T8vcTSHDFPkz/wFCj1aTFc0Hw5Ux+fXm+A8 eQsBT9SKZCHReJH1+khBCsPbBo7rDCARmF5BYFA6WHHF/TdSxcGe3xX3O0UVPJ59ZmY0 Wtx8XmBwlz02yUGAW/w7Qmai9BN6VuvW0Xwtk0eb4ohxd34jZJf7QSiSoBHK3g2X4tXT x3n3aeWGOz8mBJ5cb1ZbhPRtPvwJySBqx9Bv/pCxIFKA0l+rDBB1JEtuWnEA7hVvbR4C yamg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a1si16687046edr.277.2021.08.30.18.46.20; Mon, 30 Aug 2021 18:46:20 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2B0A668A336; Tue, 31 Aug 2021 04:46:01 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5F6A3689E61 for ; Tue, 31 Aug 2021 04:45:52 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="216531709" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="216531709" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Aug 2021 18:45:45 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="530827040" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by FMSMGA003.fm.intel.com with ESMTP; 30 Aug 2021 18:45:44 -0700 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 09:43:31 +0800 Message-Id: <20210831014338.134086-3-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210831014338.134086-1-wenbin.chen@intel.com> References: <20210831014338.134086-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 03/10] libavfilter/vulkan: Fix the way to use sem X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: "Chen,Wenbin" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 14SnOm43nacW From: "Chen,Wenbin" We chould set waitSem and signalSem differently. Current ffmpeg-vulkan uses the same sem to set waitSem and signalSem and it doesn't work on latest intel-vulkan-driver. The commit: a193060221c4df123e26a562949cae5df3e73cde on mesa causes this problem. This commit add code to resets the signalSem. This will reset waitSem too on current ffmpeg-vulkan. Now set waitSem and signalSem separetely. Now the following command can run on the latest mesa on intel platform: ffmpeg -v verbose -init_hw_device vulkan=vul:0,linear_images=1 -filter_hw_device vul -i input1080p.264 -vf "hwupload=extra_hw_frames=16,scale_vulkan=1920:1080, hwdownload,format=yuv420p" -f rawvideo output.yuv Signed-off-by: Wenbin Chen --- libavfilter/vf_avgblur_vulkan.c | 4 +-- libavfilter/vf_chromaber_vulkan.c | 4 +-- libavfilter/vf_overlay_vulkan.c | 6 ++-- libavfilter/vf_scale_vulkan.c | 4 +-- libavfilter/vulkan.c | 55 +++++++++++++++++-------------- libavfilter/vulkan.h | 3 +- libavutil/hwcontext_vulkan.c | 14 ++++---- 7 files changed, 50 insertions(+), 40 deletions(-) diff --git a/libavfilter/vf_avgblur_vulkan.c b/libavfilter/vf_avgblur_vulkan.c index 5ae487fc8c..d2104c191e 100644 --- a/libavfilter/vf_avgblur_vulkan.c +++ b/libavfilter/vf_avgblur_vulkan.c @@ -304,8 +304,8 @@ static int process_frames(AVFilterContext *avctx, AVFrame *out_f, AVFrame *tmp_f vkCmdDispatch(cmd_buf, s->vkctx.output_width, FFALIGN(s->vkctx.output_height, CGS)/CGS, 1); - ff_vk_add_exec_dep(avctx, s->exec, in_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT); - ff_vk_add_exec_dep(avctx, s->exec, out_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT); + ff_vk_add_exec_dep(avctx, s->exec, in_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, 1); + ff_vk_add_exec_dep(avctx, s->exec, out_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, 0); err = ff_vk_submit_exec_queue(avctx, s->exec); if (err) diff --git a/libavfilter/vf_chromaber_vulkan.c b/libavfilter/vf_chromaber_vulkan.c index 96fdd7bd9c..fe66a31cea 100644 --- a/libavfilter/vf_chromaber_vulkan.c +++ b/libavfilter/vf_chromaber_vulkan.c @@ -249,8 +249,8 @@ static int process_frames(AVFilterContext *avctx, AVFrame *out_f, AVFrame *in_f) FFALIGN(s->vkctx.output_width, CGROUPS[0])/CGROUPS[0], FFALIGN(s->vkctx.output_height, CGROUPS[1])/CGROUPS[1], 1); - ff_vk_add_exec_dep(avctx, s->exec, in_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT); - ff_vk_add_exec_dep(avctx, s->exec, out_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT); + ff_vk_add_exec_dep(avctx, s->exec, in_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, 1); + ff_vk_add_exec_dep(avctx, s->exec, out_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, 0); err = ff_vk_submit_exec_queue(avctx, s->exec); if (err) diff --git a/libavfilter/vf_overlay_vulkan.c b/libavfilter/vf_overlay_vulkan.c index 1815709d82..2e5bef5be5 100644 --- a/libavfilter/vf_overlay_vulkan.c +++ b/libavfilter/vf_overlay_vulkan.c @@ -331,9 +331,9 @@ static int process_frames(AVFilterContext *avctx, AVFrame *out_f, FFALIGN(s->vkctx.output_width, CGROUPS[0])/CGROUPS[0], FFALIGN(s->vkctx.output_height, CGROUPS[1])/CGROUPS[1], 1); - ff_vk_add_exec_dep(avctx, s->exec, main_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT); - ff_vk_add_exec_dep(avctx, s->exec, overlay_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT); - ff_vk_add_exec_dep(avctx, s->exec, out_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT); + ff_vk_add_exec_dep(avctx, s->exec, main_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, 1); + ff_vk_add_exec_dep(avctx, s->exec, overlay_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, 1); + ff_vk_add_exec_dep(avctx, s->exec, out_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, 0); err = ff_vk_submit_exec_queue(avctx, s->exec); if (err) diff --git a/libavfilter/vf_scale_vulkan.c b/libavfilter/vf_scale_vulkan.c index 4eb4fe5664..0d946e0416 100644 --- a/libavfilter/vf_scale_vulkan.c +++ b/libavfilter/vf_scale_vulkan.c @@ -377,8 +377,8 @@ static int process_frames(AVFilterContext *avctx, AVFrame *out_f, AVFrame *in_f) FFALIGN(s->vkctx.output_width, CGROUPS[0])/CGROUPS[0], FFALIGN(s->vkctx.output_height, CGROUPS[1])/CGROUPS[1], 1); - ff_vk_add_exec_dep(avctx, s->exec, in_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT); - ff_vk_add_exec_dep(avctx, s->exec, out_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT); + ff_vk_add_exec_dep(avctx, s->exec, in_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, 1); + ff_vk_add_exec_dep(avctx, s->exec, out_f, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, 0); err = ff_vk_submit_exec_queue(avctx, s->exec); if (err) diff --git a/libavfilter/vulkan.c b/libavfilter/vulkan.c index e5b070b3e6..e8cbf66b2b 100644 --- a/libavfilter/vulkan.c +++ b/libavfilter/vulkan.c @@ -462,9 +462,10 @@ VkCommandBuffer ff_vk_get_exec_buf(AVFilterContext *avctx, FFVkExecContext *e) } int ff_vk_add_exec_dep(AVFilterContext *avctx, FFVkExecContext *e, - AVFrame *frame, VkPipelineStageFlagBits in_wait_dst_flag) + AVFrame *frame, VkPipelineStageFlagBits in_wait_dst_flag, int input_frame) { AVFrame **dst; + VkSemaphore *sem_temp; VulkanFilterContext *s = avctx->priv; AVVkFrame *f = (AVVkFrame *)frame->data[0]; FFVkQueueCtx *q = &e->queues[s->cur_queue_idx]; @@ -472,33 +473,39 @@ int ff_vk_add_exec_dep(AVFilterContext *avctx, FFVkExecContext *e, int planes = av_pix_fmt_count_planes(fc->sw_format); for (int i = 0; i < planes; i++) { - e->sem_wait = av_fast_realloc(e->sem_wait, &e->sem_wait_alloc, - (e->sem_wait_cnt + 1)*sizeof(*e->sem_wait)); - if (!e->sem_wait) { - ff_vk_discard_exec_deps(avctx, e); - return AVERROR(ENOMEM); - } + if (input_frame) { + sem_temp = av_fast_realloc(e->sem_wait, &e->sem_wait_alloc, + (e->sem_wait_cnt + 1)*sizeof(*e->sem_wait)); + if (!sem_temp) { + ff_vk_discard_exec_deps(avctx, e); + return AVERROR(ENOMEM); + } + e->sem_wait = sem_temp; - e->sem_wait_dst = av_fast_realloc(e->sem_wait_dst, &e->sem_wait_dst_alloc, - (e->sem_wait_cnt + 1)*sizeof(*e->sem_wait_dst)); - if (!e->sem_wait_dst) { - ff_vk_discard_exec_deps(avctx, e); - return AVERROR(ENOMEM); - } + sem_temp = av_fast_realloc(e->sem_wait_dst, &e->sem_wait_dst_alloc, + (e->sem_wait_cnt + 1)*sizeof(*e->sem_wait_dst)); + if (!sem_temp) { + ff_vk_discard_exec_deps(avctx, e); + return AVERROR(ENOMEM); + } + e->sem_wait_dst = sem_temp; - e->sem_sig = av_fast_realloc(e->sem_sig, &e->sem_sig_alloc, - (e->sem_sig_cnt + 1)*sizeof(*e->sem_sig)); - if (!e->sem_sig) { - ff_vk_discard_exec_deps(avctx, e); - return AVERROR(ENOMEM); - } + e->sem_wait[e->sem_wait_cnt] = f->sem[i]; + e->sem_wait_dst[e->sem_wait_cnt] = in_wait_dst_flag; + e->sem_wait_cnt++; + } else { - e->sem_wait[e->sem_wait_cnt] = f->sem[i]; - e->sem_wait_dst[e->sem_wait_cnt] = in_wait_dst_flag; - e->sem_wait_cnt++; + sem_temp = av_fast_realloc(e->sem_sig, &e->sem_sig_alloc, + (e->sem_sig_cnt + 1)*sizeof(*e->sem_sig)); + if (!sem_temp) { + ff_vk_discard_exec_deps(avctx, e); + return AVERROR(ENOMEM); + } + e->sem_sig = sem_temp; - e->sem_sig[e->sem_sig_cnt] = f->sem[i]; - e->sem_sig_cnt++; + e->sem_sig[e->sem_sig_cnt] = f->sem[i]; + e->sem_sig_cnt++; + } } dst = av_fast_realloc(q->frame_deps, &q->frame_deps_alloc_size, diff --git a/libavfilter/vulkan.h b/libavfilter/vulkan.h index f9a4dc5839..2fdc0e1368 100644 --- a/libavfilter/vulkan.h +++ b/libavfilter/vulkan.h @@ -340,7 +340,8 @@ void ff_vk_discard_exec_deps(AVFilterContext *avctx, FFVkExecContext *e); * Must be called before submission. */ int ff_vk_add_exec_dep(AVFilterContext *avctx, FFVkExecContext *e, - AVFrame *frame, VkPipelineStageFlagBits in_wait_dst_flag); + AVFrame *frame, VkPipelineStageFlagBits in_wait_dst_flag, + int input_frame); /** * Submits a command buffer to the queue for execution. diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 88db5b8b70..9a29267aed 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -1737,8 +1737,6 @@ static int prepare_frame(AVHWFramesContext *hwfc, VulkanExecCtx *ectx, VkSubmitInfo s_info = { .sType = VK_STRUCTURE_TYPE_SUBMIT_INFO, - .pSignalSemaphores = frame->sem, - .signalSemaphoreCount = planes, }; VkPipelineStageFlagBits wait_st[AV_NUM_DATA_POINTERS]; @@ -1750,11 +1748,15 @@ static int prepare_frame(AVHWFramesContext *hwfc, VulkanExecCtx *ectx, new_layout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL; new_access = VK_ACCESS_TRANSFER_WRITE_BIT; dst_qf = VK_QUEUE_FAMILY_IGNORED; + s_info.pSignalSemaphores = frame->sem; + s_info.signalSemaphoreCount = planes; break; case PREP_MODE_RO_SHADER: new_layout = VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL; new_access = VK_ACCESS_TRANSFER_READ_BIT; dst_qf = VK_QUEUE_FAMILY_IGNORED; + s_info.pSignalSemaphores = frame->sem; + s_info.signalSemaphoreCount = planes; break; case PREP_MODE_EXTERNAL_EXPORT: new_layout = VK_IMAGE_LAYOUT_GENERAL; @@ -3226,11 +3228,11 @@ static int transfer_image_buf(AVHWFramesContext *hwfc, const AVFrame *f, VkSubmitInfo s_info = { .sType = VK_STRUCTURE_TYPE_SUBMIT_INFO, - .pSignalSemaphores = frame->sem, - .pWaitSemaphores = frame->sem, + .pSignalSemaphores = to_buf ? NULL: frame->sem, + .pWaitSemaphores = to_buf ? frame->sem : NULL, .pWaitDstStageMask = sem_wait_dst, - .signalSemaphoreCount = planes, - .waitSemaphoreCount = planes, + .signalSemaphoreCount = to_buf ? 0 : planes, + .waitSemaphoreCount = to_buf ? planes : 0, }; if ((err = wait_start_exec_ctx(hwfc, ectx))) From patchwork Tue Aug 31 01:43:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29889 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4497480iov; Mon, 30 Aug 2021 18:46:31 -0700 (PDT) X-Google-Smtp-Source: ABdhPJybwRMG+FVw+4eP77+EKuElJP5VjbUw/j6q3a+wBtQxrKIxLST7HlaaSDUmIf9eQ4114tZB X-Received: by 2002:a17:906:fb19:: with SMTP id lz25mr28586479ejb.162.1630374391264; Mon, 30 Aug 2021 18:46:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630374391; cv=none; d=google.com; s=arc-20160816; b=XmGVGk3g1lJEgJrM2qCgR3zaf/fiOBWxy6Sga7qr0TMWRlC91id8BDUSEHNkIHYIyz 7LOzsSijyVpHMSXKRF3qEA8mlOJcfNPpdmgo/z3/SFG1+5C6sH/u73MkaA3fC1DmJHiG twiK+shRahcLhPr0FB5VeDo3OvnsBP4JaXAZOvHZB49Yh5/UbSp1YPm2MEoCfRultcuS w29uP+jQMKuFM/rLISk+1kCIPSjlKAPByIaO8DMnjqR0OM3REvH02ceZnSV1QPLis6Rn TOsrvK0K00XhNaPbBGnX+HYFYyxR0+d6S6TfZ+8ZsraoX3ceYZBx9OhZHkkxT8ImhASN Jt8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=YcAFXgXsIkriAoQMaLawuiiM00sa/5vQJoDX0HfEBxI=; b=VIqEyeDOzK6EkzqXuRICG0dO+YBaYUZlJpKdiJigP3DtbEK8JfxR+OSeCzXbO6t+o8 Y6pVvpIHS0h5APzi62depS6DkVxwriLPE9dBPBqu4yAgTTCcRafHJicKgoJKce7dvbzf TBaXQf3TGVZkTNndGRQjylMa13xnXuNLKfoqTsvArrN4t0SMPqJvXDAiDddF8N58r7LQ hJs0ZBjwpnHAjIJ946SzbW4DgxEmQqRKn2mkuz85hTc0pNRG0gShhNYFbwTJg9lTPdby 1JcRIsPCOU2eSprMySzfuHNCXmRj7LKcjoVs97ZH03DidbVsIXdyYuhkXArg/9UlIPlJ mKzw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id u2si16068108edy.66.2021.08.30.18.46.30; Mon, 30 Aug 2021 18:46:31 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3252368A3BF; Tue, 31 Aug 2021 04:46:02 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 276B3689E61 for ; Tue, 31 Aug 2021 04:45:53 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="216531714" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="216531714" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Aug 2021 18:45:47 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="530827043" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by FMSMGA003.fm.intel.com with ESMTP; 30 Aug 2021 18:45:46 -0700 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 09:43:32 +0800 Message-Id: <20210831014338.134086-4-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210831014338.134086-1-wenbin.chen@intel.com> References: <20210831014338.134086-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 04/10] hwcontext_vaapi: Use PRIME_2 memory type for modifiers. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Bas Nieuwenhuizen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: boSgtpcYuAln From: Bas Nieuwenhuizen This way we can pass explicit modifiers in. Sometimes the modifier matters for the number of memory planes that libva accepts, in particular when dealing with driver-compressed textures. Furthermore the driver might not actually be able to determine the implicit modifier if all the buffer-passing has used explicit modifier. All these issues should be resolved by passing in the modifier, and for that we switch to using the PRIME_2 memory type. Tested with experimental radeonsi patches for modifiers and kmsgrab. Also tested with radeonsi without the patches to double-check it works without PRIME_2 support. v2: Cache PRIME_2 support to avoid doing two calls every time on libva drivers that do not support it. v3: Remove prime2_vas usage. --- libavutil/hwcontext_vaapi.c | 158 ++++++++++++++++++++++++++---------- 1 file changed, 114 insertions(+), 44 deletions(-) diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index 83e542876d..75acc851d6 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -79,6 +79,9 @@ typedef struct VAAPIFramesContext { unsigned int rt_format; // Whether vaDeriveImage works. int derive_works; + // Caches whether VA_SURFACE_ATTRIB_MEM_TYPE_DRM_PRIME_2 is unsupported for + // surface imports. + int prime_2_import_unsupported; } VAAPIFramesContext; typedef struct VAAPIMapping { @@ -1022,32 +1025,17 @@ static void vaapi_unmap_from_drm(AVHWFramesContext *dst_fc, static int vaapi_map_from_drm(AVHWFramesContext *src_fc, AVFrame *dst, const AVFrame *src, int flags) { + VAAPIFramesContext *src_vafc = src_fc->internal->priv; AVHWFramesContext *dst_fc = (AVHWFramesContext*)dst->hw_frames_ctx->data; AVVAAPIDeviceContext *dst_dev = dst_fc->device_ctx->hwctx; const AVDRMFrameDescriptor *desc; const VAAPIFormatDescriptor *format_desc; VASurfaceID surface_id; - VAStatus vas; + VAStatus vas = VA_STATUS_SUCCESS; + int use_prime2; uint32_t va_fourcc; - int err, i, j, k; - - unsigned long buffer_handle; - VASurfaceAttribExternalBuffers buffer_desc; - VASurfaceAttrib attrs[2] = { - { - .type = VASurfaceAttribMemoryType, - .flags = VA_SURFACE_ATTRIB_SETTABLE, - .value.type = VAGenericValueTypeInteger, - .value.value.i = VA_SURFACE_ATTRIB_MEM_TYPE_DRM_PRIME, - }, - { - .type = VASurfaceAttribExternalBufferDescriptor, - .flags = VA_SURFACE_ATTRIB_SETTABLE, - .value.type = VAGenericValueTypePointer, - .value.value.p = &buffer_desc, - } - }; + int err, i, j; desc = (AVDRMFrameDescriptor*)src->data[0]; @@ -1083,35 +1071,117 @@ static int vaapi_map_from_drm(AVHWFramesContext *src_fc, AVFrame *dst, format_desc = vaapi_format_from_fourcc(va_fourcc); av_assert0(format_desc); - buffer_handle = desc->objects[0].fd; - buffer_desc.pixel_format = va_fourcc; - buffer_desc.width = src_fc->width; - buffer_desc.height = src_fc->height; - buffer_desc.data_size = desc->objects[0].size; - buffer_desc.buffers = &buffer_handle; - buffer_desc.num_buffers = 1; - buffer_desc.flags = 0; - - k = 0; - for (i = 0; i < desc->nb_layers; i++) { - for (j = 0; j < desc->layers[i].nb_planes; j++) { - buffer_desc.pitches[k] = desc->layers[i].planes[j].pitch; - buffer_desc.offsets[k] = desc->layers[i].planes[j].offset; - ++k; + use_prime2 = !src_vafc->prime_2_import_unsupported && + desc->objects[0].format_modifier != DRM_FORMAT_MOD_INVALID; + if (use_prime2) { + VADRMPRIMESurfaceDescriptor prime_desc; + VASurfaceAttrib prime_attrs[2] = { + { + .type = VASurfaceAttribMemoryType, + .flags = VA_SURFACE_ATTRIB_SETTABLE, + .value.type = VAGenericValueTypeInteger, + .value.value.i = VA_SURFACE_ATTRIB_MEM_TYPE_DRM_PRIME_2, + }, + { + .type = VASurfaceAttribExternalBufferDescriptor, + .flags = VA_SURFACE_ATTRIB_SETTABLE, + .value.type = VAGenericValueTypePointer, + .value.value.p = &prime_desc, + } + }; + prime_desc.fourcc = va_fourcc; + prime_desc.width = src_fc->width; + prime_desc.height = src_fc->height; + prime_desc.num_objects = desc->nb_objects; + for (i = 0; i < desc->nb_objects; ++i) { + prime_desc.objects[i].fd = desc->objects[i].fd; + prime_desc.objects[i].size = desc->objects[i].size; + prime_desc.objects[i].drm_format_modifier = + desc->objects[i].format_modifier; } - } - buffer_desc.num_planes = k; - if (format_desc->chroma_planes_swapped && - buffer_desc.num_planes == 3) { - FFSWAP(uint32_t, buffer_desc.pitches[1], buffer_desc.pitches[2]); - FFSWAP(uint32_t, buffer_desc.offsets[1], buffer_desc.offsets[2]); + prime_desc.num_layers = desc->nb_layers; + for (i = 0; i < desc->nb_layers; ++i) { + prime_desc.layers[i].drm_format = desc->layers[i].format; + prime_desc.layers[i].num_planes = desc->layers[i].nb_planes; + for (j = 0; j < desc->layers[i].nb_planes; ++j) { + prime_desc.layers[i].object_index[j] = + desc->layers[i].planes[j].object_index; + prime_desc.layers[i].offset[j] = desc->layers[i].planes[j].offset; + prime_desc.layers[i].pitch[j] = desc->layers[i].planes[j].pitch; + } + + if (format_desc->chroma_planes_swapped && + desc->layers[i].nb_planes == 3) { + FFSWAP(uint32_t, prime_desc.layers[i].pitch[1], + prime_desc.layers[i].pitch[2]); + FFSWAP(uint32_t, prime_desc.layers[i].offset[1], + prime_desc.layers[i].offset[2]); + } + } + + /* + * We can query for PRIME_2 support with vaQuerySurfaceAttributes, but that + * that needs the config_id which we don't have here . Both Intel and + * Gallium seem to do the correct error checks, so lets just try the + * PRIME_2 import first. + */ + vas = vaCreateSurfaces(dst_dev->display, format_desc->rt_format, + src->width, src->height, &surface_id, 1, + prime_attrs, FF_ARRAY_ELEMS(prime_attrs)); + if (vas != VA_STATUS_SUCCESS) + src_vafc->prime_2_import_unsupported = 1; } - vas = vaCreateSurfaces(dst_dev->display, format_desc->rt_format, - src->width, src->height, - &surface_id, 1, - attrs, FF_ARRAY_ELEMS(attrs)); + if (!use_prime2 || vas != VA_STATUS_SUCCESS) { + int k; + unsigned long buffer_handle; + VASurfaceAttribExternalBuffers buffer_desc; + VASurfaceAttrib buffer_attrs[2] = { + { + .type = VASurfaceAttribMemoryType, + .flags = VA_SURFACE_ATTRIB_SETTABLE, + .value.type = VAGenericValueTypeInteger, + .value.value.i = VA_SURFACE_ATTRIB_MEM_TYPE_DRM_PRIME, + }, + { + .type = VASurfaceAttribExternalBufferDescriptor, + .flags = VA_SURFACE_ATTRIB_SETTABLE, + .value.type = VAGenericValueTypePointer, + .value.value.p = &buffer_desc, + } + }; + + buffer_handle = desc->objects[0].fd; + buffer_desc.pixel_format = va_fourcc; + buffer_desc.width = src_fc->width; + buffer_desc.height = src_fc->height; + buffer_desc.data_size = desc->objects[0].size; + buffer_desc.buffers = &buffer_handle; + buffer_desc.num_buffers = 1; + buffer_desc.flags = 0; + + k = 0; + for (i = 0; i < desc->nb_layers; i++) { + for (j = 0; j < desc->layers[i].nb_planes; j++) { + buffer_desc.pitches[k] = desc->layers[i].planes[j].pitch; + buffer_desc.offsets[k] = desc->layers[i].planes[j].offset; + ++k; + } + } + buffer_desc.num_planes = k; + + if (format_desc->chroma_planes_swapped && + buffer_desc.num_planes == 3) { + FFSWAP(uint32_t, buffer_desc.pitches[1], buffer_desc.pitches[2]); + FFSWAP(uint32_t, buffer_desc.offsets[1], buffer_desc.offsets[2]); + } + + vas = vaCreateSurfaces(dst_dev->display, format_desc->rt_format, + src->width, src->height, + &surface_id, 1, + buffer_attrs, FF_ARRAY_ELEMS(buffer_attrs)); + } if (vas != VA_STATUS_SUCCESS) { av_log(dst_fc, AV_LOG_ERROR, "Failed to create surface from DRM " "object: %d (%s).\n", vas, vaErrorStr(vas)); From patchwork Tue Aug 31 01:43:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29882 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4497564iov; Mon, 30 Aug 2021 18:46:42 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwsorWIapAMfQE47dw+5B9XJUBwtmUjhXkN//bYFYK1XYjc6zBFsWbjxz92t8XJbkUCXTZI X-Received: by 2002:a17:906:ec9:: with SMTP id u9mr27041959eji.243.1630374402234; Mon, 30 Aug 2021 18:46:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630374402; cv=none; d=google.com; s=arc-20160816; b=GngbeulV6ImEy5j37daZJXaiPzw7uSxQ5sXUBS5oG4HMc7bdMy+rtSYI4IukUnmf2V o5+YsGVZx0H23nUfFLTz4pX8kAgy2ors3qDTaeWZis2MpxdXJKfuC58Pj1H0xu9hbEWL Y7Jvz7BJM2st+c+XJJjVZ0R4ygRRV1YyY8ibokecKH7SFCFKAT4vBuTVwRN8GIqHM8kc 4a0mA5TQbpvLQ5fzvnu6zu4/LOTHz2T/f2a3f9hE6+s+sNpyXXr17Db8NCFtsBHvO5dJ 1mh4+RbAi++/mp/vMcFokSgLAZLGilXDe2mEMRBl9voxcNN2TUNGHtIKbqQfBTs8y3wY Y5MQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=q5VAcmArfhVwYhtEPTLycYPUnXgeHuyG70tanvBTUvs=; b=pUfvsj7l1AAgcRO8KN7W1cCCnz0fuyKKht4kyZE8mWzs69y053lkHIWPEOBba8Olhl /YyrWY0R4CCMfe5400sYiYgYwi9VbIcdM3nacQ9PabSj8Ghnxtioh8Eol/SU5XFqsNGZ qytfd4jL59rE8GNupDnwJ2YiEL3jsJSgU9j0X6i/H9OSLlIOaURbvF5rsAPUOkigphKt mztJ2mu7Smridvqq886H1vwrz1SFCiEbeiC6d2XpJ1oVWW1aaSQJH9XSHIAa/uglsCHs wVoJy9r/r2iPSIYdcIL/cKiDaU8Qx+yAUXkqqg4b01E/qMzazLjbTTBzIB1nuiP+8Yaz qduA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f12si6474323edx.586.2021.08.30.18.46.41; Mon, 30 Aug 2021 18:46:42 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 31C1E68A473; Tue, 31 Aug 2021 04:46:06 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 648ED68A2CE for ; Tue, 31 Aug 2021 04:45:58 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="216531724" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="216531724" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Aug 2021 18:45:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="530827058" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by FMSMGA003.fm.intel.com with ESMTP; 30 Aug 2021 18:45:48 -0700 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 09:43:33 +0800 Message-Id: <20210831014338.134086-5-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210831014338.134086-1-wenbin.chen@intel.com> References: <20210831014338.134086-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 05/10] libavutil/hwcontext_vaapi: Add a new nv12 format map to support vulkan frame X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: "Chen,Wenbin" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: JvzNsT9M6Ew0 From: "Chen,Wenbin" Vulkan will map nv12 to R8 and GR88, so add this map to vaapi to support vulkan frame. Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vaapi.c | 1 + 1 file changed, 1 insertion(+) diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index 75acc851d6..994b744e4d 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -992,6 +992,7 @@ static const struct { } vaapi_drm_format_map[] = { #ifdef DRM_FORMAT_R8 DRM_MAP(NV12, 2, DRM_FORMAT_R8, DRM_FORMAT_RG88), + DRM_MAP(NV12, 2, DRM_FORMAT_R8, DRM_FORMAT_GR88), #endif DRM_MAP(NV12, 1, DRM_FORMAT_NV12), #if defined(VA_FOURCC_P010) && defined(DRM_FORMAT_R16) From patchwork Tue Aug 31 01:43:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29888 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4497666iov; Mon, 30 Aug 2021 18:46:53 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzAwb2Mfp0vAIIv8A+lelSElvxJbFn2DupmA/f/j2a7GtDeW29QjVACH/37AXzcpIiYPcrg X-Received: by 2002:a50:bb68:: with SMTP id y95mr8660349ede.231.1630374413131; Mon, 30 Aug 2021 18:46:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630374413; cv=none; d=google.com; s=arc-20160816; b=ddFyKDT6Klu8bH3SmTOg6xLCvw/iTnp4Uy2/OiJEL4s3u3rsQVTrCL4wwOpfQVv8Ut /cBfDSybdfVfI96AUPmP6VDTwGF9rtEmn/HpDzz1OFwKqHO+hOBMnYfdbRtmInGTWPcP xduIHR7r1UFZNFsfAyOZV2vZgfxJ9Bq3mObQuFPz5obVn0GBbWlWUzex6Wi256WV3Yo8 Wf0xS4fuxvLWshJ6wjyWHaSTiRCMgPBaG0MRHfNk0XaAoTOew/mmeA0ju2I+Hpva3Eab XQTUYPlxxlmdXqht4Q6+JW0kZjPpI6VrO8VFC3r61um9MEvwA4SlcOT0erFc6L6V7B4j /nlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=iN59ZpIpOfCPTZHbjVi/+CzsjHYN4fZToIyI6sPAtRM=; b=AptAcLvNbhTRinYBf+m4DFLGaCMNSzqWd2CLuuU84xD17rXDIUsQVedqQHu9BwEUe9 ntYf8i7YXLZuBXhjaHmfkHsIBrNZifJW+rPbuP2cJ4JcUdIjRyekIYIb7UwCbpcH4CdT FaQqMsPvx3kdeB+FuQyqbpki+DlQLRjws4MOuMMAvXh6DlNvVIV4Od5dc4JmuZmNuj8o wWOUFsBbTlW8NTa+l9LyJZFIrUpBih/IuLTKx56++EUREIr2oj3wdAw5clEb3+Q8KsLY v7LK9Qf6IqTB5lOhEhTQeKuH8En7y+awF6CqeFgUAVCtDlMdwz6iehIc7gHedPluPAiT k2og== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id dk23si16563499edb.444.2021.08.30.18.46.52; Mon, 30 Aug 2021 18:46:53 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0537968A497; Tue, 31 Aug 2021 04:46:07 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4897868A2CE for ; Tue, 31 Aug 2021 04:45:59 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="216531729" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="216531729" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Aug 2021 18:45:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="530827068" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by FMSMGA003.fm.intel.com with ESMTP; 30 Aug 2021 18:45:49 -0700 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 09:43:34 +0800 Message-Id: <20210831014338.134086-6-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210831014338.134086-1-wenbin.chen@intel.com> References: <20210831014338.134086-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 06/10] libavutil/hwcontext_vulkan: Add one_memory flag to make vulkan compatible with vaapi device. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: "Chen,Wenbin" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: dtVTpTZyKvpW From: "Chen,Wenbin" Vaapi can import external surface, but all the planes of the external frames should be in the same drm object. A new flag is introduced and vulkan can choose to allocate planes in one memory according this flag. This flag will be enabled when the vulkan device is derived from vaapi device, so that this change will not affect current vulkan behaviour. Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 9a29267aed..6417f59d4a 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -211,6 +211,9 @@ typedef struct VulkanDevicePriv { /* Settings */ int use_linear_images; + /* map all planes to one memory */ + int use_one_memory; + /* Nvidia */ int dev_is_nvidia; } VulkanDevicePriv; @@ -1321,6 +1324,11 @@ static int vulkan_device_create_internal(AVHWDeviceContext *ctx, if (opt_d) p->use_linear_images = strtol(opt_d->value, NULL, 10); + opt_d = av_dict_get(opts, "one_memory", NULL, 0); + if (opt_d) + p->use_one_memory = strtol(opt_d->value, NULL, 10); + + hwctx->enabled_dev_extensions = dev_info.ppEnabledExtensionNames; hwctx->nb_enabled_dev_extensions = dev_info.enabledExtensionCount; @@ -1443,8 +1451,10 @@ static int vulkan_device_derive(AVHWDeviceContext *ctx, return AVERROR_EXTERNAL; } - if (strstr(vendor, "Intel")) + if (strstr(vendor, "Intel")) { + av_dict_set_int(&opts, "one_memory", 1, 0); dev_select.vendor_id = 0x8086; + } if (strstr(vendor, "AMD")) dev_select.vendor_id = 0x1002; From patchwork Tue Aug 31 01:43:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29890 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4497833iov; Mon, 30 Aug 2021 18:47:12 -0700 (PDT) X-Google-Smtp-Source: ABdhPJylK+3tB6Zs4kGJuw1o5ZSBGqmnORbIynQSvZuluapmQMHFa2Fnym9btmhJy6bs7vLryl5G X-Received: by 2002:a17:906:4a82:: with SMTP id x2mr24405508eju.111.1630374432405; Mon, 30 Aug 2021 18:47:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630374432; cv=none; d=google.com; s=arc-20160816; b=om7WZkMGPRNajgJXYQCG5EcwhvX1zJGbEdnNRPdhIewjp7dyaiGcVUQA+MMusX97w+ VzzLFxYtwyNWCr6YQYEIaSX8plxpOe67rrUcmKMI/YzHuo7ZNnxVU8zMmnFOshP4D1bF uEPdK761a35PgaVyPDishlcBKAxHrvbpikslMKCqstH1AFbJDyRI/K314Q6iAusC/U9a fCmYOD0JPQSrtO2mj/MdgVuiqJyX52vMsi11HnjYsVA5lhycjjK8VVsTwV08A5sjYk/K VM1GR4JNTzIzg0oLfD+kXNoW9rm0vu75WFmRVHQb0hmaB8D3pQXTLEYTT7sV2mEjgNhG OLHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=q7+fUHnw8sGeF3vfs88Xi/8psULaZoINmPkphzWCeDY=; b=V1iHA5x/KfU1IqmOgGqm3Ir7ADKW6voIkS5F9IclMMRH9KyjwYpjwHgr0jsWql/lqO h5tRTRVcAjPujTh5epy4smt+aSY6aQ/hgaZl141aRy8aV50S0zgCU3Yea2BI/ycDJk4M Hk6RkzgPsM5LjjEtzxesaZYozN5Hr4sFRZjeVfUil1A3BVJJg6O9W3+eYJzUvK4CIS9A MDqkjRa3L70/hmqpY/FdfG60a0Kasi/1b7NJLV9ghV+BH+DGYyHoU66pLxO9ZVfqOKQe yAkJ856rwRukXF6a+X3Q+PwQE4F1fVjpyc4JTryHcml/Ea215571m6xA11tFsd2i6lkm rReA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id aq12si7372758ejc.417.2021.08.30.18.47.12; Mon, 30 Aug 2021 18:47:12 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BA0D068A4D1; Tue, 31 Aug 2021 04:46:08 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2A6FD68A408 for ; Tue, 31 Aug 2021 04:45:59 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="216531735" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="216531735" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Aug 2021 18:45:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="530827076" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by FMSMGA003.fm.intel.com with ESMTP; 30 Aug 2021 18:45:51 -0700 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 09:43:35 +0800 Message-Id: <20210831014338.134086-7-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210831014338.134086-1-wenbin.chen@intel.com> References: <20210831014338.134086-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 07/10] libavutil/hwcontext_vulkan: Allocate vkFrame in one memory X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: "Chen,Wenbin" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: hEe/9IVeJ4xp From: "Chen,Wenbin" The vaapi can import external frame, but the planes of the external frames should be in the same drm object. I add a new function to allocate vkFrame in one memory and vulkan device will choose a way to allocate memory according to one_memory flag. A new variable is added to AVVKFrame to store the offset of each plane. Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 46 +++++++++++++++++++++++++++++++++++- libavutil/hwcontext_vulkan.h | 1 + 2 files changed, 46 insertions(+), 1 deletion(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 6417f59d4a..4983518a77 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -1667,6 +1667,9 @@ static int alloc_bind_mem(AVHWFramesContext *hwfc, AVVkFrame *f, VulkanFunctions *vk = &p->vkfn; const int planes = av_pix_fmt_count_planes(hwfc->sw_format); VkBindImageMemoryInfo bind_info[AV_NUM_DATA_POINTERS] = { { 0 } }; + VkMemoryRequirements memory_requirements = { 0 }; + int mem_size = 0; + int mem_size_list[AV_NUM_DATA_POINTERS] = { 0 }; AVVulkanDeviceContext *hwctx = ctx->hwctx; @@ -1694,6 +1697,23 @@ static int alloc_bind_mem(AVHWFramesContext *hwfc, AVVkFrame *f, req.memoryRequirements.size = FFALIGN(req.memoryRequirements.size, p->props.properties.limits.minMemoryMapAlignment); + if (p->use_one_memory) { + if (ded_req.prefersDedicatedAllocation | ded_req.requiresDedicatedAllocation) { + av_log(hwfc, AV_LOG_ERROR, "Cannot use dedicated allocation for intel vaapi\n"); + return AVERROR(EINVAL); + } + if (memory_requirements.size == 0) { + memory_requirements = req.memoryRequirements; + } else if (memory_requirements.memoryTypeBits != req.memoryRequirements.memoryTypeBits) { + av_log(hwfc, AV_LOG_ERROR, "the param for each planes are not the same\n"); + return AVERROR(EINVAL); + } + + mem_size_list[i] = req.memoryRequirements.size; + mem_size += mem_size_list[i]; + continue; + } + /* In case the implementation prefers/requires dedicated allocation */ use_ded_mem = ded_req.prefersDedicatedAllocation | ded_req.requiresDedicatedAllocation; @@ -1715,6 +1735,29 @@ static int alloc_bind_mem(AVHWFramesContext *hwfc, AVVkFrame *f, bind_info[i].memory = f->mem[i]; } + if (p->use_one_memory) { + memory_requirements.size = mem_size; + + /* Allocate memory */ + if ((err = alloc_mem(ctx, &memory_requirements, + f->tiling == VK_IMAGE_TILING_LINEAR ? + VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT : + VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT, + (void *)(((uint8_t *)alloc_pnext)), + &f->flags, &f->mem[0]))) + return err; + + f->size[0] = memory_requirements.size; + + for (int i = 0; i < planes; i++) { + bind_info[i].sType = VK_STRUCTURE_TYPE_BIND_IMAGE_MEMORY_INFO; + bind_info[i].image = f->img[i]; + bind_info[i].memory = f->mem[0]; + bind_info[i].memoryOffset = i == 0 ? 0 : mem_size_list[i-1]; + f->offset[i] = bind_info[i].memoryOffset; + } + } + /* Bind the allocated memory to the images */ ret = vk->BindImageMemory2(hwctx->act_dev, planes, bind_info); if (ret != VK_SUCCESS) { @@ -2921,7 +2964,8 @@ static int vulkan_map_to_drm(AVHWFramesContext *hwfc, AVFrame *dst, continue; vk->GetImageSubresourceLayout(hwctx->act_dev, f->img[i], &sub, &layout); - drm_desc->layers[i].planes[0].offset = layout.offset; + drm_desc->layers[i].planes[0].offset = p->use_one_memory ? + f->offset[i] : layout.offset; drm_desc->layers[i].planes[0].pitch = layout.rowPitch; } diff --git a/libavutil/hwcontext_vulkan.h b/libavutil/hwcontext_vulkan.h index e4645527d7..8fb25d1485 100644 --- a/libavutil/hwcontext_vulkan.h +++ b/libavutil/hwcontext_vulkan.h @@ -182,6 +182,7 @@ typedef struct AVVkFrame { */ VkDeviceMemory mem[AV_NUM_DATA_POINTERS]; size_t size[AV_NUM_DATA_POINTERS]; + size_t offset[AV_NUM_DATA_POINTERS]; /** * OR'd flags for all memory allocated From patchwork Tue Aug 31 01:43:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29881 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4497754iov; Mon, 30 Aug 2021 18:47:02 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyXUmx3cUsUcaHCjOSCvf8v922vnBHBeY+EQtZ6IazWksepINZ/AV/3HZh/CpSWxmNYUZe4 X-Received: by 2002:a17:906:404:: with SMTP id d4mr27984880eja.449.1630374422602; Mon, 30 Aug 2021 18:47:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630374422; cv=none; d=google.com; s=arc-20160816; b=vvUoryQvF8KLwitCkWKvGWvGPqqwkGbKAxefPbt8IiFdo3xN+q0b0sX2K9/7wgE3B6 hlmG/RgPAuV5nJM8TR3SKhouwzTeGUpVK3jWArP+s14fttnNCKkdU8prqsp2YC5q7Nvv yrCrPllb7TJ7+dcAncCsfQvoFOxLH/FkZb+pCFnZMY+sCPkiSCo4A2oRI7/TahV3Qpjf 7sCpS3BDIaGxCJ9Dh6DpwEWrcKZoKOKVK3SBip4sOUUkWFXRFoKX+YkKAg7TmFrZaK5f a+fP0Ree8ieOYYe+XbT9ORr01Qsu2o0LiS1uDPK5wDjIgSSHUIc38jjIwtIr6yxO2W1d ws1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=3GPF92gy1WxzmZKaIJLiJOa/zUEuBkZEFzfW+QMhcWU=; b=rnwCzR9sac2nmCo4uAfrJOJjq/9Koov+CFhumMzeR301YeQWD4funzVlMDI87AhwSu uxRs39mRp/hPRscfL2XGvxBbYuq/jp5LBUFDmTeaZozHefij5EzJgaA+kb0VkFCZKZWJ At05V5qyWbHPDYgMI4YDdgqYnRptZrgITg2w/r+OPknJ4HZQdf4NE19ce0aGCKxtrre8 2vSpevCzjaSw+0KZu5+5Qeh9rEFKX+1JfGzhiFSw3Mdw/t6yYeV6GZ0uS7XnKalASf45 U65d6l+3Pi37geveinBF2wBd3XtcCWisirEQb6t6whc+/g3+Hy4k4n8ezxaBS8JZjgG0 Q02Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id j17si16731030edr.153.2021.08.30.18.47.02; Mon, 30 Aug 2021 18:47:02 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C765B68A4A4; Tue, 31 Aug 2021 04:46:07 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5695568A407 for ; Tue, 31 Aug 2021 04:46:04 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="216531744" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="216531744" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Aug 2021 18:45:53 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="530827088" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by FMSMGA003.fm.intel.com with ESMTP; 30 Aug 2021 18:45:53 -0700 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 09:43:36 +0800 Message-Id: <20210831014338.134086-8-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210831014338.134086-1-wenbin.chen@intel.com> References: <20210831014338.134086-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 08/10] libavutil/hwcontext_vulkan: fix wrong offset of plane X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: "Chen,Wenbin" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: AGS56ihs/gKk From: "Chen,Wenbin" According to spec, if we use VkBindImagePlaneMemoryInfo to bind image we mush create image with disjoint flag. The offset in subresourcelayout is relative to the base address of the plane, but the offset in drm is relative to the drm objectis so I think this offset should be 0. Also, when I import vaapi frame to vulkan I got broken frame, and setting plane_data->offset to 0 makes command works. Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 4983518a77..3a639c997b 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -2382,7 +2382,9 @@ static int vulkan_map_from_drm_frame_desc(AVHWFramesContext *hwfc, AVVkFrame **f .extent.depth = 1, .mipLevels = 1, .arrayLayers = 1, - .flags = VK_IMAGE_CREATE_ALIAS_BIT, + .flags = VK_IMAGE_CREATE_ALIAS_BIT | + (has_modifiers && planes > 1) ? VK_IMAGE_CREATE_DISJOINT_BIT : + 0, .tiling = f->tiling, .initialLayout = VK_IMAGE_LAYOUT_UNDEFINED, /* specs say so */ .usage = frames_hwctx->usage, @@ -2397,7 +2399,7 @@ static int vulkan_map_from_drm_frame_desc(AVHWFramesContext *hwfc, AVVkFrame **f hwfc->sw_format, src->width, src->height, i); for (int j = 0; j < planes; j++) { - plane_data[j].offset = desc->layers[i].planes[j].offset; + plane_data[j].offset = 0; plane_data[j].rowPitch = desc->layers[i].planes[j].pitch; plane_data[j].size = 0; /* The specs say so for all 3 */ plane_data[j].arrayPitch = 0; From patchwork Tue Aug 31 01:43:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29885 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4497925iov; Mon, 30 Aug 2021 18:47:21 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwdjJy8JUUNUiWfNlKwizP5dF+2UMysa1kB/rY7fp6dLlZrcJbnpDb3ki1RVRk+UDkGnp8x X-Received: by 2002:a17:906:1615:: with SMTP id m21mr28375180ejd.279.1630374441695; Mon, 30 Aug 2021 18:47:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630374441; cv=none; d=google.com; s=arc-20160816; b=V2ITnfF6Lfp3djQwJuGXnG+72GmqSN3wUQgyQV4L3ZYZNxW5HGtabvS6ita8AuXbU5 i/4b5FLuZNmN5C12Zd6Z0EyOVYVh5eUyi9JHM2WkagHeR8IOCU1n3vGMaw3A5Xrg349h HY+pY3Uo3yGf5EsmvV9Q2L902hUV7mFbK1NLq949LP8iSOb+4096NNDpEIKWRx+VbMrg RqUevObEogq/P6as7R+0U+WYtfIO6IIanseCeABIE+k6XXlXpHmwNEmltn0KXEZTiGFI L/m7SfB8YEmlp+x0bnJhJrby5Kp2WOWbjb0AOxDkKASUSYWuBDT/Mdkqh7MmzFe/D7lb ZwWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=vzy98ilcdP3qXjGbi5aZOsXej3unZOL3hVDilUdSNLk=; b=vzcD5QqEGs/JbYCtDYCY5CeQ04UnnAVyZ6FHoOhJikL80NQlDaNz2+LBuSmlD/a+rV MxVeCYqxkBDdx4lIhp1X/DMLaQigboBU8KxSZFYWULXpL38lavpTcuCj56dkWkr/CELZ YXGs2B3qj9wSMRg4d91j8lImQWl7767nNIbgTql8JBMj/FkGMsh+QOHtfl2SoZrtigpY Y9E7wWrYpUoKG6SdCJPNyRL6mxWnH4nk140z3uBuW2mPP09c0IWPSX4SorFoTvY4b/Tr y2X3X7iDgRk1acgUWvE7gnUMW73tSQWruw/w8QHlVbUbz/c6bNwLOul3JGxuHQ+x0vmy RO9Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id la20si19282799ejc.3.2021.08.30.18.47.21; Mon, 30 Aug 2021 18:47:21 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 81B5468A4E5; Tue, 31 Aug 2021 04:46:09 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3A86368A41C for ; Tue, 31 Aug 2021 04:46:05 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="216531746" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="216531746" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Aug 2021 18:45:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="530827101" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by FMSMGA003.fm.intel.com with ESMTP; 30 Aug 2021 18:45:54 -0700 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 09:43:37 +0800 Message-Id: <20210831014338.134086-9-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210831014338.134086-1-wenbin.chen@intel.com> References: <20210831014338.134086-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 09/10] libavutil/hwcontext_vulkan: specify the modifier to create VKImage X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Eni+pDJEwtvO From: Wenbin Chen On the lastset intel-vulkan-driver the VK_EXT_image_drm_format_modifier flags is enabled. As what driver log recommand, we need to use VK_IMAGE_TILING_LINEAR or VK_IMAGE_DRM_FORMAT_MODIFIER_EXT to create VKImage. Add code to get supported modifier for sw_format and use these modifier to create VKImage. Now the following command line works: ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i input_1080p.264 -vf "hwmap=derive_device=vulkan,format=vulkan, scale_vulkan=1920:1080,hwmap=derive_device=vaapi,format=vaapi" -c:v h264_vaapi output.264 Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 77 +++++++++++++++++++++++++++++++++--- libavutil/hwcontext_vulkan.h | 5 +++ 2 files changed, 76 insertions(+), 6 deletions(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 3a639c997b..99b2190dc3 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -1967,6 +1967,8 @@ static void try_export_flags(AVHWFramesContext *hwfc, AVVulkanDeviceContext *dev_hwctx = hwfc->device_ctx->hwctx; VulkanDevicePriv *p = hwfc->device_ctx->internal->priv; VulkanFunctions *vk = &p->vkfn; + const int has_modifiers = hwctx->tiling == VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT; + VkExternalImageFormatProperties eprops = { .sType = VK_STRUCTURE_TYPE_EXTERNAL_IMAGE_FORMAT_PROPERTIES_KHR, }; @@ -1974,9 +1976,18 @@ static void try_export_flags(AVHWFramesContext *hwfc, .sType = VK_STRUCTURE_TYPE_IMAGE_FORMAT_PROPERTIES_2, .pNext = &eprops, }; + VkPhysicalDeviceImageDrmFormatModifierInfoEXT phy_dev_mod_info = { + .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_IMAGE_DRM_FORMAT_MODIFIER_INFO_EXT, + .pNext = NULL, + .pQueueFamilyIndices = p->qfs, + .queueFamilyIndexCount = p->num_qfs, + .sharingMode = p->num_qfs > 1 ? VK_SHARING_MODE_CONCURRENT : + VK_SHARING_MODE_EXCLUSIVE, + }; VkPhysicalDeviceExternalImageFormatInfo enext = { .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_EXTERNAL_IMAGE_FORMAT_INFO, .handleType = exp, + .pNext = has_modifiers ? &phy_dev_mod_info : NULL, }; VkPhysicalDeviceImageFormatInfo2 pinfo = { .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_IMAGE_FORMAT_INFO_2, @@ -1988,11 +1999,15 @@ static void try_export_flags(AVHWFramesContext *hwfc, .flags = VK_IMAGE_CREATE_ALIAS_BIT, }; - ret = vk->GetPhysicalDeviceImageFormatProperties2(dev_hwctx->phys_dev, - &pinfo, &props); - if (ret == VK_SUCCESS) { - *iexp |= exp; - *comp_handle_types |= eprops.externalMemoryProperties.compatibleHandleTypes; + for (int i = 0; i < (has_modifiers ? hwctx->modifier_count : 1); i++) { + if (has_modifiers && hwctx->modifier_count) + phy_dev_mod_info.drmFormatModifier = hwctx->modifiers[i]; + ret = vk->GetPhysicalDeviceImageFormatProperties2(dev_hwctx->phys_dev, + &pinfo, &props); + if (ret == VK_SUCCESS) { + *iexp |= exp; + *comp_handle_types |= eprops.externalMemoryProperties.compatibleHandleTypes; + } } } @@ -2055,6 +2070,7 @@ fail: static void vulkan_frames_uninit(AVHWFramesContext *hwfc) { VulkanFramesPriv *fp = hwfc->internal->priv; + AVVulkanFramesContext *hwctx = hwfc->hwctx; free_exec_ctx(hwfc, &fp->conv_ctx); free_exec_ctx(hwfc, &fp->upload_ctx); @@ -2069,11 +2085,60 @@ static int vulkan_frames_init(AVHWFramesContext *hwfc) VulkanFramesPriv *fp = hwfc->internal->priv; AVVulkanDeviceContext *dev_hwctx = hwfc->device_ctx->hwctx; VulkanDevicePriv *p = hwfc->device_ctx->internal->priv; + const int has_modifiers = !!(p->extensions & EXT_DRM_MODIFIER_FLAGS); /* Default pool flags */ - hwctx->tiling = hwctx->tiling ? hwctx->tiling : p->use_linear_images ? + hwctx->tiling = hwctx->tiling ? hwctx->tiling : has_modifiers ? + VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT : p->use_linear_images ? VK_IMAGE_TILING_LINEAR : VK_IMAGE_TILING_OPTIMAL; + /* get the supported modifier */ + if (has_modifiers) { + const VkFormat *fmt = av_vkfmt_from_pixfmt(hwfc->sw_format); + VulkanFunctions *vk = &p->vkfn; + VkDrmFormatModifierPropertiesEXT mod_props[MAX_VULKAN_MODIFIERS]; + + VkDrmFormatModifierPropertiesListEXT mod_props_list = { + .sType = VK_STRUCTURE_TYPE_DRM_FORMAT_MODIFIER_PROPERTIES_LIST_EXT, + .pNext = NULL, + .drmFormatModifierCount = 0, + .pDrmFormatModifierProperties = NULL, + }; + VkFormatProperties2 prop = { + .sType = VK_STRUCTURE_TYPE_FORMAT_PROPERTIES_2, + .pNext = &mod_props_list, + }; + + vk->GetPhysicalDeviceFormatProperties2(dev_hwctx->phys_dev, fmt[0], &prop); + if (!mod_props_list.drmFormatModifierCount) { + av_log(hwfc, AV_LOG_ERROR, "There are not supported modifiers for sw_format\n"); + return AVERROR(EINVAL); + } + mod_props_list.drmFormatModifierCount = + FFMIN(mod_props_list.drmFormatModifierCount, MAX_VULKAN_MODIFIERS); + + mod_props_list.pDrmFormatModifierProperties = mod_props; + vk->GetPhysicalDeviceFormatProperties2(dev_hwctx->phys_dev, fmt[0], &prop); + + hwctx->modifier_count = 0; + for (int i = 0; i < mod_props_list.drmFormatModifierCount; i++) { + if (!(mod_props[i].drmFormatModifierTilingFeatures & DEFAULT_USAGE_FLAGS)) + continue; + hwctx->modifiers[hwctx->modifier_count++] = mod_props[i].drmFormatModifier; + } + if (!hwctx->modifier_count) { + av_log(hwfc, AV_LOG_ERROR, "The supported modifiers doesn't support" + "default usage\n"); + return AVERROR(EINVAL); + } + + hwctx->create_pnext = &hwctx->modifier_info; + hwctx->modifier_info.pNext = NULL; + hwctx->modifier_info.sType = VK_STRUCTURE_TYPE_IMAGE_DRM_FORMAT_MODIFIER_LIST_CREATE_INFO_EXT; + hwctx->modifier_info.drmFormatModifierCount = hwctx->modifier_count; + hwctx->modifier_info.pDrmFormatModifiers = hwctx->modifiers; + } + if (!hwctx->usage) hwctx->usage = DEFAULT_USAGE_FLAGS; diff --git a/libavutil/hwcontext_vulkan.h b/libavutil/hwcontext_vulkan.h index 8fb25d1485..95fd767d1b 100644 --- a/libavutil/hwcontext_vulkan.h +++ b/libavutil/hwcontext_vulkan.h @@ -32,6 +32,7 @@ * with the data pointer set to an AVVkFrame. */ +#define MAX_VULKAN_MODIFIERS 16 /** * Main Vulkan context, allocated as AVHWDeviceContext.hwctx. * All of these can be set before init to change what the context uses @@ -142,6 +143,10 @@ typedef struct AVVulkanFramesContext { */ void *create_pnext; + VkImageDrmFormatModifierListCreateInfoEXT modifier_info; + uint64_t modifiers[MAX_VULKAN_MODIFIERS]; + int modifier_count; + /** * Extension data for memory allocation. Must have as many entries as * the number of planes of the sw_format. From patchwork Tue Aug 31 01:43:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29884 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4497993iov; Mon, 30 Aug 2021 18:47:31 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzN/uV1B4EcWKhoOouEee+EkwPfilBMAkgTsC5ztXrpQAeQ8CNBzdbqy5fvs5KEtTA4hy79 X-Received: by 2002:a05:6402:350d:: with SMTP id b13mr17225555edd.1.1630374451707; Mon, 30 Aug 2021 18:47:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630374451; cv=none; d=google.com; s=arc-20160816; b=jcnO1sE/uZRoxvvLD6ndJyZ0PNlTzsA3DshyoSm0y0KUfahxYyNPJCxFK3/c20VpuO 6GAWp4tzQHD48BQ7+YPsX3fKC7EEvjuWSCfbbZKV/pC6FkGxOdSdGSKHbbnKDa1tZm15 1TwRazluv40V+/yPaVZ5x8O7LdXMYeEj1TObtFZmrf4HqSvqmsd9PBAFteZZ1LJZoNQF wWc3LACObq0mbmppJTO6/0FUWwKCv401C0ltpLr45WkVYqK4kwz43svOvdqc161U7QNS zzsGrXKXEWVTRbY23pvnBxI/NqXXTMoprEI6gnCvl6z0wwVBfDNqlRu4+xNJtKa60Awv t8Vw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=PqePHfBzHIvmz76N5pnkIVk/N0cU6FZE2I60parq5YU=; b=ThrOLEqN6pAY3zULQlTg8ze0YcRK17E1TDOYsFv1Ci8W1ZnXNIVtnHEGKcisqD41Oq hro9jrJ747+HMECDIBcEcFf0b09BGlbMGVVLPDYmceEwp9cy1eyhUqpBgmw25Sp+kcVU PezU3ZErkuxe5hbznr/7pOWRB/zCvZ7ROK5HBbwstv10RXnKuXSli7M7gb5++ign964V CQZa8R5f530gjkvqMsublySIA1eQrYBfEhazzhImFcrD2RZKiyfBY6xjbXecowwc2qLz dVn0aVrjzXEBmp1XF1em+1ued28ijTuvF0/WAOMj7sbzJ2GRdQpLl2HmK/M800d1P2ss QR4Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i18si21613586edc.336.2021.08.30.18.47.31; Mon, 30 Aug 2021 18:47:31 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5A6A568A498; Tue, 31 Aug 2021 04:46:10 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3FB5D68A42A for ; Tue, 31 Aug 2021 04:46:05 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="216531747" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="216531747" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Aug 2021 18:45:57 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="530827108" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by FMSMGA003.fm.intel.com with ESMTP; 30 Aug 2021 18:45:56 -0700 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 09:43:38 +0800 Message-Id: <20210831014338.134086-10-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210831014338.134086-1-wenbin.chen@intel.com> References: <20210831014338.134086-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 10/10] libavutil/hwcontext_vulkan: Add hwupload and hwdownload support when using one_memory flag. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: "Chen,Wenbin" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: mcfsXbojl5Kl From: "Chen,Wenbin" Add hwupload and hwdownload support to vulkan when frames are allocated in one memory Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 99b2190dc3..5dfa7adc6b 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -2252,7 +2252,7 @@ static int vulkan_map_frame_to_mem(AVHWFramesContext *hwfc, AVFrame *dst, const AVFrame *src, int flags) { VkResult ret; - int err, mapped_mem_count = 0; + int err, mapped_mem_count = 0, loop = 0; AVVkFrame *f = (AVVkFrame *)src->data[0]; AVVulkanDeviceContext *hwctx = hwfc->device_ctx->hwctx; const int planes = av_pix_fmt_count_planes(hwfc->sw_format); @@ -2281,7 +2281,8 @@ static int vulkan_map_frame_to_mem(AVHWFramesContext *hwfc, AVFrame *dst, dst->width = src->width; dst->height = src->height; - for (int i = 0; i < planes; i++) { + loop = p->use_one_memory ? 1 : planes; + for (int i = 0; i < loop; i++) { ret = vk->MapMemory(hwctx->act_dev, f->mem[i], 0, VK_WHOLE_SIZE, 0, (void **)&dst->data[i]); if (ret != VK_SUCCESS) { @@ -2292,6 +2293,11 @@ static int vulkan_map_frame_to_mem(AVHWFramesContext *hwfc, AVFrame *dst, } mapped_mem_count++; } + if (p->use_one_memory) { + for (int i = 0; i < planes; i++) { + dst->data[i] = dst->data[0] + f->offset[i]; + } + } /* Check if the memory contents matter */ if (((flags & AV_HWFRAME_MAP_READ) || !(flags & AV_HWFRAME_MAP_OVERWRITE)) &&