From patchwork Tue Aug 31 01:43:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 29890 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp4497833iov; Mon, 30 Aug 2021 18:47:12 -0700 (PDT) X-Google-Smtp-Source: ABdhPJylK+3tB6Zs4kGJuw1o5ZSBGqmnORbIynQSvZuluapmQMHFa2Fnym9btmhJy6bs7vLryl5G X-Received: by 2002:a17:906:4a82:: with SMTP id x2mr24405508eju.111.1630374432405; Mon, 30 Aug 2021 18:47:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630374432; cv=none; d=google.com; s=arc-20160816; b=om7WZkMGPRNajgJXYQCG5EcwhvX1zJGbEdnNRPdhIewjp7dyaiGcVUQA+MMusX97w+ VzzLFxYtwyNWCr6YQYEIaSX8plxpOe67rrUcmKMI/YzHuo7ZNnxVU8zMmnFOshP4D1bF uEPdK761a35PgaVyPDishlcBKAxHrvbpikslMKCqstH1AFbJDyRI/K314Q6iAusC/U9a fCmYOD0JPQSrtO2mj/MdgVuiqJyX52vMsi11HnjYsVA5lhycjjK8VVsTwV08A5sjYk/K VM1GR4JNTzIzg0oLfD+kXNoW9rm0vu75WFmRVHQb0hmaB8D3pQXTLEYTT7sV2mEjgNhG OLHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=q7+fUHnw8sGeF3vfs88Xi/8psULaZoINmPkphzWCeDY=; b=V1iHA5x/KfU1IqmOgGqm3Ir7ADKW6voIkS5F9IclMMRH9KyjwYpjwHgr0jsWql/lqO h5tRTRVcAjPujTh5epy4smt+aSY6aQ/hgaZl141aRy8aV50S0zgCU3Yea2BI/ycDJk4M Hk6RkzgPsM5LjjEtzxesaZYozN5Hr4sFRZjeVfUil1A3BVJJg6O9W3+eYJzUvK4CIS9A MDqkjRa3L70/hmqpY/FdfG60a0Kasi/1b7NJLV9ghV+BH+DGYyHoU66pLxO9ZVfqOKQe yAkJ856rwRukXF6a+X3Q+PwQE4F1fVjpyc4JTryHcml/Ea215571m6xA11tFsd2i6lkm rReA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id aq12si7372758ejc.417.2021.08.30.18.47.12; Mon, 30 Aug 2021 18:47:12 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BA0D068A4D1; Tue, 31 Aug 2021 04:46:08 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2A6FD68A408 for ; Tue, 31 Aug 2021 04:45:59 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10092"; a="216531735" X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="216531735" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Aug 2021 18:45:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,365,1620716400"; d="scan'208";a="530827076" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.5]) by FMSMGA003.fm.intel.com with ESMTP; 30 Aug 2021 18:45:51 -0700 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Aug 2021 09:43:35 +0800 Message-Id: <20210831014338.134086-7-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210831014338.134086-1-wenbin.chen@intel.com> References: <20210831014338.134086-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 07/10] libavutil/hwcontext_vulkan: Allocate vkFrame in one memory X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: "Chen,Wenbin" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: hEe/9IVeJ4xp From: "Chen,Wenbin" The vaapi can import external frame, but the planes of the external frames should be in the same drm object. I add a new function to allocate vkFrame in one memory and vulkan device will choose a way to allocate memory according to one_memory flag. A new variable is added to AVVKFrame to store the offset of each plane. Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 46 +++++++++++++++++++++++++++++++++++- libavutil/hwcontext_vulkan.h | 1 + 2 files changed, 46 insertions(+), 1 deletion(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 6417f59d4a..4983518a77 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -1667,6 +1667,9 @@ static int alloc_bind_mem(AVHWFramesContext *hwfc, AVVkFrame *f, VulkanFunctions *vk = &p->vkfn; const int planes = av_pix_fmt_count_planes(hwfc->sw_format); VkBindImageMemoryInfo bind_info[AV_NUM_DATA_POINTERS] = { { 0 } }; + VkMemoryRequirements memory_requirements = { 0 }; + int mem_size = 0; + int mem_size_list[AV_NUM_DATA_POINTERS] = { 0 }; AVVulkanDeviceContext *hwctx = ctx->hwctx; @@ -1694,6 +1697,23 @@ static int alloc_bind_mem(AVHWFramesContext *hwfc, AVVkFrame *f, req.memoryRequirements.size = FFALIGN(req.memoryRequirements.size, p->props.properties.limits.minMemoryMapAlignment); + if (p->use_one_memory) { + if (ded_req.prefersDedicatedAllocation | ded_req.requiresDedicatedAllocation) { + av_log(hwfc, AV_LOG_ERROR, "Cannot use dedicated allocation for intel vaapi\n"); + return AVERROR(EINVAL); + } + if (memory_requirements.size == 0) { + memory_requirements = req.memoryRequirements; + } else if (memory_requirements.memoryTypeBits != req.memoryRequirements.memoryTypeBits) { + av_log(hwfc, AV_LOG_ERROR, "the param for each planes are not the same\n"); + return AVERROR(EINVAL); + } + + mem_size_list[i] = req.memoryRequirements.size; + mem_size += mem_size_list[i]; + continue; + } + /* In case the implementation prefers/requires dedicated allocation */ use_ded_mem = ded_req.prefersDedicatedAllocation | ded_req.requiresDedicatedAllocation; @@ -1715,6 +1735,29 @@ static int alloc_bind_mem(AVHWFramesContext *hwfc, AVVkFrame *f, bind_info[i].memory = f->mem[i]; } + if (p->use_one_memory) { + memory_requirements.size = mem_size; + + /* Allocate memory */ + if ((err = alloc_mem(ctx, &memory_requirements, + f->tiling == VK_IMAGE_TILING_LINEAR ? + VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT : + VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT, + (void *)(((uint8_t *)alloc_pnext)), + &f->flags, &f->mem[0]))) + return err; + + f->size[0] = memory_requirements.size; + + for (int i = 0; i < planes; i++) { + bind_info[i].sType = VK_STRUCTURE_TYPE_BIND_IMAGE_MEMORY_INFO; + bind_info[i].image = f->img[i]; + bind_info[i].memory = f->mem[0]; + bind_info[i].memoryOffset = i == 0 ? 0 : mem_size_list[i-1]; + f->offset[i] = bind_info[i].memoryOffset; + } + } + /* Bind the allocated memory to the images */ ret = vk->BindImageMemory2(hwctx->act_dev, planes, bind_info); if (ret != VK_SUCCESS) { @@ -2921,7 +2964,8 @@ static int vulkan_map_to_drm(AVHWFramesContext *hwfc, AVFrame *dst, continue; vk->GetImageSubresourceLayout(hwctx->act_dev, f->img[i], &sub, &layout); - drm_desc->layers[i].planes[0].offset = layout.offset; + drm_desc->layers[i].planes[0].offset = p->use_one_memory ? + f->offset[i] : layout.offset; drm_desc->layers[i].planes[0].pitch = layout.rowPitch; } diff --git a/libavutil/hwcontext_vulkan.h b/libavutil/hwcontext_vulkan.h index e4645527d7..8fb25d1485 100644 --- a/libavutil/hwcontext_vulkan.h +++ b/libavutil/hwcontext_vulkan.h @@ -182,6 +182,7 @@ typedef struct AVVkFrame { */ VkDeviceMemory mem[AV_NUM_DATA_POINTERS]; size_t size[AV_NUM_DATA_POINTERS]; + size_t offset[AV_NUM_DATA_POINTERS]; /** * OR'd flags for all memory allocated