From patchwork Tue Nov 9 09:18:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 31334 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:d206:0:0:0:0:0 with SMTP id q6csp885153iob; Tue, 9 Nov 2021 01:22:51 -0800 (PST) X-Google-Smtp-Source: ABdhPJzuNnNm7SJomN3c/VsdXgcelQkzjbr+0QquNbLfr+Vp6N3Ji8cB7V6zqIx9swxpnEM+V8xl X-Received: by 2002:a17:907:7e83:: with SMTP id qb3mr7768790ejc.469.1636449771524; Tue, 09 Nov 2021 01:22:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1636449771; cv=none; d=google.com; s=arc-20160816; b=DVuyxM25N3gP5cUjy6+o4DnFrHihNepEngGu/pGT04ap6LGxzruZTKMDXaqQFbXGPL l65LD9g8mlWc3a6AMl1cHIkMtGX5yGRKbplfHVp377BwHbgn3XPyI89XeIBTka0N1ABV SfaBhG5atKzuXeImnvWMG+b8KySU1cdda3hageEUVzi8djsq7Byw+F1WYAxjxMpYwwn2 Qs5eXov75TRi25xRlFqVt4Kw0eip/JUCBG+Qcu0DQUxF2u27As6GJvIFccDy4+M1roXa BRo35mnIBa/oaLt9f6ceDGQM3LP4lLAE8l5XHkjBsC3i5wVqlT6Ky5oetuqmgTVjy3Jv AKhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=B9ZS4Q5STUEXuN7xPBnLS0OWXjWnqqhCX4dg195bg3Y=; b=bK6BPEntPtNc/TCJXBgIHxeD3EcQ3zBBOSKjDqK60KtRO38qnRasVtS1MwUQ+Zycq9 in6elshqaC6NncjBlTuT/azTQ+zWveM76CHH8/+tvEn1MahJ2VzYspvus2ml0k9AUBzJ E6LMN+Jgy/8vYCOvWy5+VYfAbHWtd5R3dyhxmUvmcNSOZClWt5QBkT0QXibBkiqemOE4 5aR9gUvh8gALYlB3XITAo3DdGcQnO70u9AGn44VAJLpK0sxzfQGV7908KPXSYnZc1euT 1QaAar3NzcypJLmTLkK7zTBHw5WXKCgm0+aXCCYoNg6boVX6+8dBOV5IIaW1i/Q6y6BI rheg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id o16si40427544edc.344.2021.11.09.01.22.50; Tue, 09 Nov 2021 01:22:51 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8978F68A990; Tue, 9 Nov 2021 11:22:47 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6B928689D0A for ; Tue, 9 Nov 2021 11:22:40 +0200 (EET) X-IronPort-AV: E=McAfee;i="6200,9189,10162"; a="231131809" X-IronPort-AV: E=Sophos;i="5.87,219,1631602800"; d="scan'208";a="231131809" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2021 01:22:38 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,219,1631602800"; d="scan'208";a="491593894" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.110]) by orsmga007.jf.intel.com with ESMTP; 09 Nov 2021 01:22:36 -0800 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Nov 2021 17:18:37 +0800 Message-Id: <20211109091843.2024927-1-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/7] hwcontext_vaapi: Use PRIME_2 memory type for modifiers. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Bas Nieuwenhuizen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: d3z7dde0QcD+ From: Bas Nieuwenhuizen This way we can pass explicit modifiers in. Sometimes the modifier matters for the number of memory planes that libva accepts, in particular when dealing with driver-compressed textures. Furthermore the driver might not actually be able to determine the implicit modifier if all the buffer-passing has used explicit modifier. All these issues should be resolved by passing in the modifier, and for that we switch to using the PRIME_2 memory type. Tested with experimental radeonsi patches for modifiers and kmsgrab. Also tested with radeonsi without the patches to double-check it works without PRIME_2 support. v2: Cache PRIME_2 support to avoid doing two calls every time on libva drivers that do not support it. v3: Remove prime2_vas usage. Signed-off-by: Bas Nieuwenhuizen --- libavutil/hwcontext_vaapi.c | 158 ++++++++++++++++++++++++++---------- 1 file changed, 114 insertions(+), 44 deletions(-) diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index 83e542876d..75acc851d6 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -79,6 +79,9 @@ typedef struct VAAPIFramesContext { unsigned int rt_format; // Whether vaDeriveImage works. int derive_works; + // Caches whether VA_SURFACE_ATTRIB_MEM_TYPE_DRM_PRIME_2 is unsupported for + // surface imports. + int prime_2_import_unsupported; } VAAPIFramesContext; typedef struct VAAPIMapping { @@ -1022,32 +1025,17 @@ static void vaapi_unmap_from_drm(AVHWFramesContext *dst_fc, static int vaapi_map_from_drm(AVHWFramesContext *src_fc, AVFrame *dst, const AVFrame *src, int flags) { + VAAPIFramesContext *src_vafc = src_fc->internal->priv; AVHWFramesContext *dst_fc = (AVHWFramesContext*)dst->hw_frames_ctx->data; AVVAAPIDeviceContext *dst_dev = dst_fc->device_ctx->hwctx; const AVDRMFrameDescriptor *desc; const VAAPIFormatDescriptor *format_desc; VASurfaceID surface_id; - VAStatus vas; + VAStatus vas = VA_STATUS_SUCCESS; + int use_prime2; uint32_t va_fourcc; - int err, i, j, k; - - unsigned long buffer_handle; - VASurfaceAttribExternalBuffers buffer_desc; - VASurfaceAttrib attrs[2] = { - { - .type = VASurfaceAttribMemoryType, - .flags = VA_SURFACE_ATTRIB_SETTABLE, - .value.type = VAGenericValueTypeInteger, - .value.value.i = VA_SURFACE_ATTRIB_MEM_TYPE_DRM_PRIME, - }, - { - .type = VASurfaceAttribExternalBufferDescriptor, - .flags = VA_SURFACE_ATTRIB_SETTABLE, - .value.type = VAGenericValueTypePointer, - .value.value.p = &buffer_desc, - } - }; + int err, i, j; desc = (AVDRMFrameDescriptor*)src->data[0]; @@ -1083,35 +1071,117 @@ static int vaapi_map_from_drm(AVHWFramesContext *src_fc, AVFrame *dst, format_desc = vaapi_format_from_fourcc(va_fourcc); av_assert0(format_desc); - buffer_handle = desc->objects[0].fd; - buffer_desc.pixel_format = va_fourcc; - buffer_desc.width = src_fc->width; - buffer_desc.height = src_fc->height; - buffer_desc.data_size = desc->objects[0].size; - buffer_desc.buffers = &buffer_handle; - buffer_desc.num_buffers = 1; - buffer_desc.flags = 0; - - k = 0; - for (i = 0; i < desc->nb_layers; i++) { - for (j = 0; j < desc->layers[i].nb_planes; j++) { - buffer_desc.pitches[k] = desc->layers[i].planes[j].pitch; - buffer_desc.offsets[k] = desc->layers[i].planes[j].offset; - ++k; + use_prime2 = !src_vafc->prime_2_import_unsupported && + desc->objects[0].format_modifier != DRM_FORMAT_MOD_INVALID; + if (use_prime2) { + VADRMPRIMESurfaceDescriptor prime_desc; + VASurfaceAttrib prime_attrs[2] = { + { + .type = VASurfaceAttribMemoryType, + .flags = VA_SURFACE_ATTRIB_SETTABLE, + .value.type = VAGenericValueTypeInteger, + .value.value.i = VA_SURFACE_ATTRIB_MEM_TYPE_DRM_PRIME_2, + }, + { + .type = VASurfaceAttribExternalBufferDescriptor, + .flags = VA_SURFACE_ATTRIB_SETTABLE, + .value.type = VAGenericValueTypePointer, + .value.value.p = &prime_desc, + } + }; + prime_desc.fourcc = va_fourcc; + prime_desc.width = src_fc->width; + prime_desc.height = src_fc->height; + prime_desc.num_objects = desc->nb_objects; + for (i = 0; i < desc->nb_objects; ++i) { + prime_desc.objects[i].fd = desc->objects[i].fd; + prime_desc.objects[i].size = desc->objects[i].size; + prime_desc.objects[i].drm_format_modifier = + desc->objects[i].format_modifier; } - } - buffer_desc.num_planes = k; - if (format_desc->chroma_planes_swapped && - buffer_desc.num_planes == 3) { - FFSWAP(uint32_t, buffer_desc.pitches[1], buffer_desc.pitches[2]); - FFSWAP(uint32_t, buffer_desc.offsets[1], buffer_desc.offsets[2]); + prime_desc.num_layers = desc->nb_layers; + for (i = 0; i < desc->nb_layers; ++i) { + prime_desc.layers[i].drm_format = desc->layers[i].format; + prime_desc.layers[i].num_planes = desc->layers[i].nb_planes; + for (j = 0; j < desc->layers[i].nb_planes; ++j) { + prime_desc.layers[i].object_index[j] = + desc->layers[i].planes[j].object_index; + prime_desc.layers[i].offset[j] = desc->layers[i].planes[j].offset; + prime_desc.layers[i].pitch[j] = desc->layers[i].planes[j].pitch; + } + + if (format_desc->chroma_planes_swapped && + desc->layers[i].nb_planes == 3) { + FFSWAP(uint32_t, prime_desc.layers[i].pitch[1], + prime_desc.layers[i].pitch[2]); + FFSWAP(uint32_t, prime_desc.layers[i].offset[1], + prime_desc.layers[i].offset[2]); + } + } + + /* + * We can query for PRIME_2 support with vaQuerySurfaceAttributes, but that + * that needs the config_id which we don't have here . Both Intel and + * Gallium seem to do the correct error checks, so lets just try the + * PRIME_2 import first. + */ + vas = vaCreateSurfaces(dst_dev->display, format_desc->rt_format, + src->width, src->height, &surface_id, 1, + prime_attrs, FF_ARRAY_ELEMS(prime_attrs)); + if (vas != VA_STATUS_SUCCESS) + src_vafc->prime_2_import_unsupported = 1; } - vas = vaCreateSurfaces(dst_dev->display, format_desc->rt_format, - src->width, src->height, - &surface_id, 1, - attrs, FF_ARRAY_ELEMS(attrs)); + if (!use_prime2 || vas != VA_STATUS_SUCCESS) { + int k; + unsigned long buffer_handle; + VASurfaceAttribExternalBuffers buffer_desc; + VASurfaceAttrib buffer_attrs[2] = { + { + .type = VASurfaceAttribMemoryType, + .flags = VA_SURFACE_ATTRIB_SETTABLE, + .value.type = VAGenericValueTypeInteger, + .value.value.i = VA_SURFACE_ATTRIB_MEM_TYPE_DRM_PRIME, + }, + { + .type = VASurfaceAttribExternalBufferDescriptor, + .flags = VA_SURFACE_ATTRIB_SETTABLE, + .value.type = VAGenericValueTypePointer, + .value.value.p = &buffer_desc, + } + }; + + buffer_handle = desc->objects[0].fd; + buffer_desc.pixel_format = va_fourcc; + buffer_desc.width = src_fc->width; + buffer_desc.height = src_fc->height; + buffer_desc.data_size = desc->objects[0].size; + buffer_desc.buffers = &buffer_handle; + buffer_desc.num_buffers = 1; + buffer_desc.flags = 0; + + k = 0; + for (i = 0; i < desc->nb_layers; i++) { + for (j = 0; j < desc->layers[i].nb_planes; j++) { + buffer_desc.pitches[k] = desc->layers[i].planes[j].pitch; + buffer_desc.offsets[k] = desc->layers[i].planes[j].offset; + ++k; + } + } + buffer_desc.num_planes = k; + + if (format_desc->chroma_planes_swapped && + buffer_desc.num_planes == 3) { + FFSWAP(uint32_t, buffer_desc.pitches[1], buffer_desc.pitches[2]); + FFSWAP(uint32_t, buffer_desc.offsets[1], buffer_desc.offsets[2]); + } + + vas = vaCreateSurfaces(dst_dev->display, format_desc->rt_format, + src->width, src->height, + &surface_id, 1, + buffer_attrs, FF_ARRAY_ELEMS(buffer_attrs)); + } if (vas != VA_STATUS_SUCCESS) { av_log(dst_fc, AV_LOG_ERROR, "Failed to create surface from DRM " "object: %d (%s).\n", vas, vaErrorStr(vas)); From patchwork Tue Nov 9 09:18:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 31333 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:d206:0:0:0:0:0 with SMTP id q6csp885505iob; Tue, 9 Nov 2021 01:23:14 -0800 (PST) X-Google-Smtp-Source: ABdhPJxEeplTN45X9dZBUhjlTzj+WsSKm5Q05B7WiTszqQNoAO0qXDmNHTl1tj7qcVvsHHFUG69k X-Received: by 2002:a17:907:3ea7:: with SMTP id hs39mr8197212ejc.164.1636449794683; Tue, 09 Nov 2021 01:23:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1636449794; cv=none; d=google.com; s=arc-20160816; b=AjK1AvzdKVo/TAnogVEL+O4EnIGUACKeml9Rjc2UHh/DIzOzo5xi04uGLpZ1CUd9UJ frPl865qK+KSA3x7gnjjcs1lp1vdKnLLm4ObBxwHJcxjFanebdmKIZfC1iRBRomlHX1a FjitsP+wWN1uicRu7c7cnDI29EJFMrO65/L1xjIHF42pUMNSiti7mRY8qLZdeIIhrVyt 0dZSWkYTAV0ynkIil0nq/IFWZCl1E+796jn/ymVZT/dZ8lGtG68K1sbMwQQ7uSN2M/Se fvEENqDJxuW0xa+qVRtgtcNbQfcLzVMuqgAGgcI+URK8GMlMzROwFYqO6LI92N3wVtab EhLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=JrvPegIXt3WVsBm98MngVNyOCmhchkjbCrtfuvo3WjU=; b=jpx8Y5ULn1MHMfHiNgKGvlRiVoujiH3IkkQT/3vajqCq75ec7/N/Y4X2D8xAE6gK8L 3EIHF1cKUTq8Hk5pz5zV56ZamQaRVSJpE6U1r/bAb9SjtAqOqATqo3ewV8YtYUDg6pAE C92bm5QO+7ZJ/C1LyntfGRFo7qfTV15mWQskENfE0BTSQxlXECifwa4uDmleRPg8ae4g 5tkOnuSD41Ais+mLIQzeA3YFgV9sERp2/F6gxX4AeLNWuwm3h7p6d4qCfdBWn8+lGiCQ 2y2yrWB748NAoApkrJXuNzw87nSQn5AlLSXzN7tg9DSlZwotSIvsEAxiH4P6hh6LofR1 Zsgw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b6si38662670ede.78.2021.11.09.01.23.14; Tue, 09 Nov 2021 01:23:14 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9181468AB2E; Tue, 9 Nov 2021 11:22:56 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E16B268A98A for ; Tue, 9 Nov 2021 11:22:43 +0200 (EET) X-IronPort-AV: E=McAfee;i="6200,9189,10162"; a="231131819" X-IronPort-AV: E=Sophos;i="5.87,219,1631602800"; d="scan'208";a="231131819" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2021 01:22:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,219,1631602800"; d="scan'208";a="491593920" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.110]) by orsmga007.jf.intel.com with ESMTP; 09 Nov 2021 01:22:41 -0800 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Nov 2021 17:18:38 +0800 Message-Id: <20211109091843.2024927-2-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211109091843.2024927-1-wenbin.chen@intel.com> References: <20211109091843.2024927-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/7] libavutil/hwcontext_vaapi: Add a new nv12 format map to support vulkan frame X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: cbLoZi18mdyQ Vulkan will map nv12 to R8 and GR88, so add this map to vaapi to support vulkan frame. Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vaapi.c | 1 + 1 file changed, 1 insertion(+) diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index 75acc851d6..994b744e4d 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -992,6 +992,7 @@ static const struct { } vaapi_drm_format_map[] = { #ifdef DRM_FORMAT_R8 DRM_MAP(NV12, 2, DRM_FORMAT_R8, DRM_FORMAT_RG88), + DRM_MAP(NV12, 2, DRM_FORMAT_R8, DRM_FORMAT_GR88), #endif DRM_MAP(NV12, 1, DRM_FORMAT_NV12), #if defined(VA_FOURCC_P010) && defined(DRM_FORMAT_R16) From patchwork Tue Nov 9 09:18:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 31335 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:d206:0:0:0:0:0 with SMTP id q6csp885290iob; Tue, 9 Nov 2021 01:23:03 -0800 (PST) X-Google-Smtp-Source: ABdhPJwvuICdTpGj4mhwSk0S8QOgWmBeoSPyR1Md3nH4wH1jOCh6ua23JolYFtuQHCXZuGn4jVCH X-Received: by 2002:a17:907:9690:: with SMTP id hd16mr7512953ejc.297.1636449783093; Tue, 09 Nov 2021 01:23:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1636449783; cv=none; d=google.com; s=arc-20160816; b=BwMmkC9LsQwZPMwqMSJDN35NsRVjpceEXZUKzW2PvtLJO7MD1dqBTnJPeoHZrz5MVY D3KX6JPjr1CF4DjW2aZaHZVbzMebrSGB7T36jS8k8MkIGmRBKh0eOzC/xkpgvvZnmUUD moBhgnh0p3cdAJOqMS9dSiGmLOGYTWclR/dOtakIcYJjc5wCd1IEvSNsYsVmu8ApzQi+ 6GEMgd6ww0UtSIJKb/jgg49LXF2+jHzPL1iii9MhaQ4/XP1FcEhzdlkzM4LuvADdrA8Y 0Y23aCOXYticw8v2kBHvkkl7W2aIBYCq/DWHCDmU/OQwxMOFBZm1YXYzTgU+yt2vsgAE QcGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=u2Yfrp8uP4C0/wrJqd9V696Z0tFyx5Ms69KtWp4J/Xw=; b=RY5BnKMHOhOCoc2WN4ialXFcXV4dFiPEaqeYQq/oadJ+IPJI91RfPqlXx/w69sv6oV T7kfwS9iTB+tpddB67iGpM1/1CYQHRQu5yE1jAb9zvHoQVY4fOr8YdJQddaE72tLKANj +HTTsLZJROz/CRS8CHWSimi7EAd4yzruuc3rFZqMqVhBUv0TB8dJgstKke78JgGbv8Fp 2O3wuhPDLwORfK2cIYNOaiTBhudhAcGfgeIhSYqsbZIwa4JpQXNT6AyCakJgJiZfTUaL lEYcDb/mhnOjiA0gKzDWeHb0fLL2AlIB2bASx8TOPDUBYRocpTaN2U3LiL3cP16UG3cV 69mg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 5si29240379eji.628.2021.11.09.01.23.02; Tue, 09 Nov 2021 01:23:03 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 99FC468A98D; Tue, 9 Nov 2021 11:22:55 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 11E6C68A9A8 for ; Tue, 9 Nov 2021 11:22:45 +0200 (EET) X-IronPort-AV: E=McAfee;i="6200,9189,10162"; a="231131823" X-IronPort-AV: E=Sophos;i="5.87,219,1631602800"; d="scan'208";a="231131823" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2021 01:22:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,219,1631602800"; d="scan'208";a="491593929" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.110]) by orsmga007.jf.intel.com with ESMTP; 09 Nov 2021 01:22:43 -0800 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Nov 2021 17:18:39 +0800 Message-Id: <20211109091843.2024927-3-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211109091843.2024927-1-wenbin.chen@intel.com> References: <20211109091843.2024927-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/7] libavutil/hwcontext_vulkan: Add one_memory flag to make vulkan compatible with vaapi device. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: U2Ea/Oh9CGgY Vaapi can import external surface, but all the planes of the external frames should be in the same drm object. A new flag is introduced and vulkan can choose to allocate planes in one memory according this flag. This flag will be enabled when the vulkan device is derived from vaapi device, so that this change will not affect current vulkan behaviour. Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 6041580117..ccf3e58f49 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -100,6 +100,9 @@ typedef struct VulkanDevicePriv { /* Settings */ int use_linear_images; + /* map all planes to one memory */ + int use_one_memory; + /* Nvidia */ int dev_is_nvidia; } VulkanDevicePriv; @@ -1245,6 +1248,11 @@ static int vulkan_device_create_internal(AVHWDeviceContext *ctx, if (opt_d) p->use_linear_images = strtol(opt_d->value, NULL, 10); + opt_d = av_dict_get(opts, "one_memory", NULL, 0); + if (opt_d) + p->use_one_memory = strtol(opt_d->value, NULL, 10); + + hwctx->enabled_dev_extensions = dev_info.ppEnabledExtensionNames; hwctx->nb_enabled_dev_extensions = dev_info.enabledExtensionCount; @@ -1365,8 +1373,10 @@ static int vulkan_device_derive(AVHWDeviceContext *ctx, return AVERROR_EXTERNAL; } - if (strstr(vendor, "Intel")) + if (strstr(vendor, "Intel")) { + av_dict_set_int(&opts, "one_memory", 1, 0); dev_select.vendor_id = 0x8086; + } if (strstr(vendor, "AMD")) dev_select.vendor_id = 0x1002; From patchwork Tue Nov 9 09:18:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 31337 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:d206:0:0:0:0:0 with SMTP id q6csp885940iob; Tue, 9 Nov 2021 01:23:40 -0800 (PST) X-Google-Smtp-Source: ABdhPJzzCyQRD5G1VVyQM9ZmDBMdZejcxR0faRbnC2HEf0LlV+ElzyvCGSvu9ENxMfe4GC79gPZH X-Received: by 2002:a17:906:a1da:: with SMTP id bx26mr7732345ejb.558.1636449820239; Tue, 09 Nov 2021 01:23:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1636449820; cv=none; d=google.com; s=arc-20160816; b=XXVJ/4JF8bBSXitChxW1ZiNj8Iok3uezkaOtePrFs2HcUBi66gYDe9O5AHGV/QdQbZ VghMay8eRyc/IabKticEBzWioDOtfbjqh2M5a6KQPugmA9yeIRhNY8/mwr/Jfc52uhlI CBdS/s3IvM0mmFBeaQuAbFWful9vQARhM6h7qrldxkeV42+dgepWgSvZEk6oMYjGfMuJ sFtekorXc6o4FxH/bfZ/NTID+771I6gIlhwPi7gzd4+SzfOUiUVX4Bl88MwPtVnL8dTr g0Dnnjeqv7RRH4zdFrTNWC3t1bEN5v7tj2BWj2WruMZZnd5Swmb/S8AwJfmPpr9jV7J7 tIEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=t99f1KJjWTAZhDmB0FXRVfJB12GCcxmr2337O8Y5XCA=; b=DrQludPqi/IiEP2slmHLNeEkeOzBgBCrPzHDq67WoI4cimnnKqzAdU0WCBBZ3QoTGy HAJJNJtUAFQFsuwi7In+2WlLA39Iz+InaysrWauwYdwgvJkbP6UbXKJYRJTlfTAjihui Lm0e6OWMMKihuJP5/SVxhOZkposUHq3Ue7JpkFs2uGfc4t2CD26uM7PSZGG3XLyRq09k VoMhoXeHhSgSMkVH8ohGia/0Irox2nkWTTYLhpYK+QrpBOnAKkiKT5HlY0Hx9rsQIXQ6 h6jsW3Mc8pvQEEru9Ag2R5FWiKCEvZCKrXbyzQwJw/v+/LgLv1tPuvWRU8Zj/I6XuYSd 7BVg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h24si33104289ejt.408.2021.11.09.01.23.39; Tue, 09 Nov 2021 01:23:40 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C34EC68AA35; Tue, 9 Nov 2021 11:23:02 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 795D268AB43 for ; Tue, 9 Nov 2021 11:22:54 +0200 (EET) X-IronPort-AV: E=McAfee;i="6200,9189,10162"; a="231131827" X-IronPort-AV: E=Sophos;i="5.87,219,1631602800"; d="scan'208";a="231131827" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2021 01:22:48 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,219,1631602800"; d="scan'208";a="491593945" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.110]) by orsmga007.jf.intel.com with ESMTP; 09 Nov 2021 01:22:47 -0800 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Nov 2021 17:18:40 +0800 Message-Id: <20211109091843.2024927-4-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211109091843.2024927-1-wenbin.chen@intel.com> References: <20211109091843.2024927-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/7] libavutil/hwcontext_vulkan: Allocate vkFrame in one memory X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ndE7NvH0hJ6N The vaapi can import external frame, but the planes of the external frames should be in the same drm object. I add a new function to allocate vkFrame in one memory and vulkan device will choose a way to allocate memory according to one_memory flag. A new variable is added to AVVKFrame to store the offset of each plane. Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 46 +++++++++++++++++++++++++++++++++++- libavutil/hwcontext_vulkan.h | 1 + 2 files changed, 46 insertions(+), 1 deletion(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index ccf3e58f49..f7878ed9c3 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -1600,6 +1600,9 @@ static int alloc_bind_mem(AVHWFramesContext *hwfc, AVVkFrame *f, FFVulkanFunctions *vk = &p->vkfn; const int planes = av_pix_fmt_count_planes(hwfc->sw_format); VkBindImageMemoryInfo bind_info[AV_NUM_DATA_POINTERS] = { { 0 } }; + VkMemoryRequirements memory_requirements = { 0 }; + int mem_size = 0; + int mem_size_list[AV_NUM_DATA_POINTERS] = { 0 }; AVVulkanDeviceContext *hwctx = ctx->hwctx; @@ -1627,6 +1630,23 @@ static int alloc_bind_mem(AVHWFramesContext *hwfc, AVVkFrame *f, req.memoryRequirements.size = FFALIGN(req.memoryRequirements.size, p->props.properties.limits.minMemoryMapAlignment); + if (p->use_one_memory) { + if (ded_req.prefersDedicatedAllocation | ded_req.requiresDedicatedAllocation) { + av_log(hwfc, AV_LOG_ERROR, "Cannot use dedicated allocation for intel vaapi\n"); + return AVERROR(EINVAL); + } + if (memory_requirements.size == 0) { + memory_requirements = req.memoryRequirements; + } else if (memory_requirements.memoryTypeBits != req.memoryRequirements.memoryTypeBits) { + av_log(hwfc, AV_LOG_ERROR, "the param for each planes are not the same\n"); + return AVERROR(EINVAL); + } + + mem_size_list[i] = req.memoryRequirements.size; + mem_size += mem_size_list[i]; + continue; + } + /* In case the implementation prefers/requires dedicated allocation */ use_ded_mem = ded_req.prefersDedicatedAllocation | ded_req.requiresDedicatedAllocation; @@ -1648,6 +1668,29 @@ static int alloc_bind_mem(AVHWFramesContext *hwfc, AVVkFrame *f, bind_info[i].memory = f->mem[i]; } + if (p->use_one_memory) { + memory_requirements.size = mem_size; + + /* Allocate memory */ + if ((err = alloc_mem(ctx, &memory_requirements, + f->tiling == VK_IMAGE_TILING_LINEAR ? + VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT : + VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT, + (void *)(((uint8_t *)alloc_pnext)), + &f->flags, &f->mem[0]))) + return err; + + f->size[0] = memory_requirements.size; + + for (int i = 0; i < planes; i++) { + bind_info[i].sType = VK_STRUCTURE_TYPE_BIND_IMAGE_MEMORY_INFO; + bind_info[i].image = f->img[i]; + bind_info[i].memory = f->mem[0]; + bind_info[i].memoryOffset = i == 0 ? 0 : mem_size_list[i-1]; + f->offset[i] = bind_info[i].memoryOffset; + } + } + /* Bind the allocated memory to the images */ ret = vk->BindImageMemory2(hwctx->act_dev, planes, bind_info); if (ret != VK_SUCCESS) { @@ -2924,7 +2967,8 @@ static int vulkan_map_to_drm(AVHWFramesContext *hwfc, AVFrame *dst, continue; vk->GetImageSubresourceLayout(hwctx->act_dev, f->img[i], &sub, &layout); - drm_desc->layers[i].planes[0].offset = layout.offset; + drm_desc->layers[i].planes[0].offset = p->use_one_memory ? + f->offset[i] : layout.offset; drm_desc->layers[i].planes[0].pitch = layout.rowPitch; } diff --git a/libavutil/hwcontext_vulkan.h b/libavutil/hwcontext_vulkan.h index 9264f70dbf..efb602ef27 100644 --- a/libavutil/hwcontext_vulkan.h +++ b/libavutil/hwcontext_vulkan.h @@ -189,6 +189,7 @@ typedef struct AVVkFrame { */ VkDeviceMemory mem[AV_NUM_DATA_POINTERS]; size_t size[AV_NUM_DATA_POINTERS]; + size_t offset[AV_NUM_DATA_POINTERS]; /** * OR'd flags for all memory allocated From patchwork Tue Nov 9 09:18:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 31336 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:d206:0:0:0:0:0 with SMTP id q6csp886151iob; Tue, 9 Nov 2021 01:23:53 -0800 (PST) X-Google-Smtp-Source: ABdhPJxBbPkL2JFfBXZ7oDqFRhfyrft3WVhYQ7ZlpXbzOqGITL+WbKcRTG5nrYFKObodpKYTSN7y X-Received: by 2002:a17:907:6287:: with SMTP id nd7mr7537280ejc.152.1636449833371; Tue, 09 Nov 2021 01:23:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1636449833; cv=none; d=google.com; s=arc-20160816; b=RhvMl1WP92cnn7QGCgwG8ksGTI+DJLbC3FCpDgUldZtE6lgiVm3XiOYbWUigOYtIdv uUp5+jNGX6Yx+a0lx5qab84ZtWERsmR6YumiFcdOVZpDBFSm7YC0xskEqeLkRA8zBM+C 0R+aSztlAYLP9GtimQJewJJ4d1iBqb/B+B6O/2HRHf5UNdHCXbMz7G5u6fjmXcBijwtw oIiaBP8BzN6ga92qMI/hUK8e3iG11VE/ZNnJe14eV6u8bDuV+f620eYKORtrEobMpqZ4 g6EITI7JK2MLDTOlD09NFLf/nTIfX2lZmgVweLiA3KkNH3COTLcExTQ1KtYPZWBDn5Eb lpXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=cJEqR1JaQePbeVamxzv71AjlzQ+8c9WVN17f/W4pP1s=; b=uS8pjIne1D38ST1J3tujRVbM1PnmxK7BDTMOXrCDK6A85yyqYqoDf3yfRXe8oziVlc cKhsASL+hkCDXe1SWO3Cf52SQ2O2ae6nU4hBrfCJWmFW2v9lQC07ncarID5GEHnFIOp0 wbN0Q1fmlORFMBzWuQ59CdAoqECZPYDKM5JTgw1aTV3A/MlPomK6a7+dFzDDrmQ+fjo2 OqS3NhiZHBV8TpzaOjWKf2Pg8GSJR6uCfPcQlLvKCiY45va1gqwUX32E4KFO1qTuMp5r 8PLmcUogKRFLtNA+qJwhxpsKNM+4fZ+w1nQDbWvv7TPgLmusU2SlirCC2k9KprsS315X MlEg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 29si27104927ejf.281.2021.11.09.01.23.53; Tue, 09 Nov 2021 01:23:53 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C735568AC06; Tue, 9 Nov 2021 11:23:03 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 759D968AA35 for ; Tue, 9 Nov 2021 11:22:54 +0200 (EET) X-IronPort-AV: E=McAfee;i="6200,9189,10162"; a="231131828" X-IronPort-AV: E=Sophos;i="5.87,219,1631602800"; d="scan'208";a="231131828" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2021 01:22:53 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,219,1631602800"; d="scan'208";a="491593956" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.110]) by orsmga007.jf.intel.com with ESMTP; 09 Nov 2021 01:22:49 -0800 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Nov 2021 17:18:41 +0800 Message-Id: <20211109091843.2024927-5-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211109091843.2024927-1-wenbin.chen@intel.com> References: <20211109091843.2024927-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 5/7] libavutil/hwcontext_vulkan: Add hwupload and hwdownload support when using one_memory flag. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: slG+JgGJUB9Y Add hwupload and hwdownload support to vulkan when frames are allocated in one memory Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index f7878ed9c3..b857d1a9ed 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -2138,7 +2138,7 @@ static int vulkan_map_frame_to_mem(AVHWFramesContext *hwfc, AVFrame *dst, const AVFrame *src, int flags) { VkResult ret; - int err, mapped_mem_count = 0; + int err, mapped_mem_count = 0, loop = 0; AVVkFrame *f = (AVVkFrame *)src->data[0]; AVVulkanDeviceContext *hwctx = hwfc->device_ctx->hwctx; const int planes = av_pix_fmt_count_planes(hwfc->sw_format); @@ -2167,7 +2167,8 @@ static int vulkan_map_frame_to_mem(AVHWFramesContext *hwfc, AVFrame *dst, dst->width = src->width; dst->height = src->height; - for (int i = 0; i < planes; i++) { + loop = p->use_one_memory ? 1 : planes; + for (int i = 0; i < loop; i++) { ret = vk->MapMemory(hwctx->act_dev, f->mem[i], 0, VK_WHOLE_SIZE, 0, (void **)&dst->data[i]); if (ret != VK_SUCCESS) { @@ -2178,6 +2179,11 @@ static int vulkan_map_frame_to_mem(AVHWFramesContext *hwfc, AVFrame *dst, } mapped_mem_count++; } + if (p->use_one_memory) { + for (int i = 0; i < planes; i++) { + dst->data[i] = dst->data[0] + f->offset[i]; + } + } /* Check if the memory contents matter */ if (((flags & AV_HWFRAME_MAP_READ) || !(flags & AV_HWFRAME_MAP_OVERWRITE)) && From patchwork Tue Nov 9 09:18:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 31338 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:d206:0:0:0:0:0 with SMTP id q6csp885749iob; Tue, 9 Nov 2021 01:23:27 -0800 (PST) X-Google-Smtp-Source: ABdhPJyOUCrhLlE1Xm9PpsVIyqNG+syudzJyWB2LSsUxehI40WWkbLuN1s9SFjLrGB5HElck8T4q X-Received: by 2002:a05:6402:35c2:: with SMTP id z2mr8256440edc.135.1636449807234; Tue, 09 Nov 2021 01:23:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1636449807; cv=none; d=google.com; s=arc-20160816; b=loYefhGPKWgk8RL9aXTmbwkoMxqtUvbKxj0Pvmekrnwthqds2Okaa90Eg9ZbwKWnZS XUkat6F37xvQ8PI9MhWG4gFrUnV9AqjRm+MxieoBO4WcussfEjHwgIkYJc6y1RRNtaK5 Ojdc8wtnRgwW5lST+TT4r0FcoS2hNTNkbHXkpGHeZdtjxxOKrc58zfD3cmm2ZJEELh9c o2AXDG9u1oRqhubb4dfD0agjZjIUJyAqwe1Pm+OC3P9VDUTcRTP2cbMDu/TJZ3zBcThf o7Z2JdrbV/6vEWPrM62fkurX/XfAyHjq3R4pR0aD6WV4rfE3P151wvCf9ghZPZMGY0ZZ zcFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=ZZdyHKXWSSd13pedTT65xfETDWp0/Yps9QE5OeR6H+A=; b=sGfd/w+3yBfpsp7MP1MjQyQWT2m4zIhswXVxyldcOapZ5YrFIIxEf9ZV/xUDjnrVbN Fvus/89LM4BOky7V+zmgWzl9YNLaijhNw4qnAgYohJXNG+2+RjCxT1lFlM1i3DYUiXM0 L78PhaakiUmHJJ/o8X5CQeqMFL1gJWIl7+9bzkBRBrP+Btps2dkG0AaJiNPrhLSQFExD yiJ5fPnEZRjq5AKLEYq+dEDI/jpydeVrVDjlqcQqjSlktt0knT3aRtIlRiaWzI3igRND xNZqRkU5xuD/qK520f+lEKTuZrbxRTrX/MOyrPi9P5ktqvVXVVBF8gkIm7k8cYbJMrfP vofQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h20si31092393edw.297.2021.11.09.01.23.26; Tue, 09 Nov 2021 01:23:27 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9EEC668AB71; Tue, 9 Nov 2021 11:23:01 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 46E8A68A98A for ; Tue, 9 Nov 2021 11:22:54 +0200 (EET) X-IronPort-AV: E=McAfee;i="6200,9189,10162"; a="293245483" X-IronPort-AV: E=Sophos;i="5.87,219,1631602800"; d="scan'208";a="293245483" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2021 01:22:52 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,219,1631602800"; d="scan'208";a="491593963" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.110]) by orsmga007.jf.intel.com with ESMTP; 09 Nov 2021 01:22:51 -0800 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Nov 2021 17:18:42 +0800 Message-Id: <20211109091843.2024927-6-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211109091843.2024927-1-wenbin.chen@intel.com> References: <20211109091843.2024927-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 6/7] libavutil/hwcontext_vulkan: fix a sem_wait bug when export drm X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: KwSDM6/dm+uz sem_sig_val is wrongly assigned to pWaitSemaphoreValues when export drm. Now fix it. Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index b857d1a9ed..29ade94b7f 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -1718,7 +1718,7 @@ static int prepare_frame(AVHWFramesContext *hwfc, VulkanExecCtx *ectx, const int planes = av_pix_fmt_count_planes(hwfc->sw_format); VulkanDevicePriv *p = hwfc->device_ctx->internal->priv; FFVulkanFunctions *vk = &p->vkfn; - uint64_t sem_sig_val[AV_NUM_DATA_POINTERS]; + uint64_t sem_sig_val[AV_NUM_DATA_POINTERS], sem_wait_val[AV_NUM_DATA_POINTERS]; VkImageMemoryBarrier img_bar[AV_NUM_DATA_POINTERS] = { 0 }; @@ -1738,6 +1738,7 @@ static int prepare_frame(AVHWFramesContext *hwfc, VulkanExecCtx *ectx, VkPipelineStageFlagBits wait_st[AV_NUM_DATA_POINTERS]; for (int i = 0; i < planes; i++) { wait_st[i] = VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT; + sem_wait_val[i] = frame->sem_value[i]; sem_sig_val[i] = frame->sem_value[i] + 1; } @@ -1756,7 +1757,7 @@ static int prepare_frame(AVHWFramesContext *hwfc, VulkanExecCtx *ectx, new_layout = VK_IMAGE_LAYOUT_GENERAL; new_access = VK_ACCESS_MEMORY_READ_BIT | VK_ACCESS_MEMORY_WRITE_BIT; dst_qf = VK_QUEUE_FAMILY_EXTERNAL_KHR; - s_timeline_sem_info.pWaitSemaphoreValues = sem_sig_val; + s_timeline_sem_info.pWaitSemaphoreValues = sem_wait_val; s_timeline_sem_info.waitSemaphoreValueCount = planes; s_info.pWaitSemaphores = frame->sem; s_info.pWaitDstStageMask = wait_st; From patchwork Tue Nov 9 09:18:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 31332 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:d206:0:0:0:0:0 with SMTP id q6csp886404iob; Tue, 9 Nov 2021 01:24:09 -0800 (PST) X-Google-Smtp-Source: ABdhPJzjORvC4UrdG48zksCAudgH9VmOEDHRcixSUCBslybHb7VUuWLKBfDpdFzs7rHOmo2kY5PE X-Received: by 2002:a50:9993:: with SMTP id m19mr7975350edb.72.1636449846236; Tue, 09 Nov 2021 01:24:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1636449846; cv=none; d=google.com; s=arc-20160816; b=L3WWpQT2e9ag458MNgbiQrYjHpHaaWi8ZJI/M/wVKXRgRqtYWJyhxR2LCi/R1comfp B9ZItuTgdIzqf/XDwKGuGjPrx6EEZlOCZWxj6jGvS5zyx6sHoPm7ebSzeB0WV7/5CJ12 9qmcJJkjxD6gR4rY1heI6uzsO0joBDLJMprRLjCeBQNkmziTGO58yKXzoScPl4XLyd7X 9HlY1Q7nCl/OgMgBt+LtAeXhupi1DuqZmo0oGjTW2GNixXUfWzWBiEKwCXmvp6IBHMsN HAZtPvRTMgbjQpuCpimigtZpD8pI9HBwQEYu9PpFJyL/DURmSsm/5/C0e4YbSRxPvGl8 KXBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=9GGp5m8g4UcciloSxU6oSK33DpBpFXM83nDprQbjB90=; b=zGTd1zy1iLde+irxHlPJvjE5FpUS0Aq1yNrrQpcXl5kqcL5GTEYPqwHign+l3gW3u1 sWHlNCQB7hoHwnNfL3M2LNE3SruH5X6O4U/tSK46sjR0mELDwMN00zH3ZZaYJ1ZWldSg J5UCeUp0cfhlLU60iSb78tah+/w1wMKwA1aA5OgM2UsQ0JmgP2O0QCK7SG+VUglVjVcf Z92BZvfaQFSEEnH113Ila7ztq4XuRPhD1QJOx/zOyYgShRong56XOXIC6y+e+FJojkwZ zwHRSSMV1s0PhYzQtB0/ExmwG7Dj4y1SRNIBn07izaFva6u4BOn7+AaOz6NHcocbJgdX e2nA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id t13si17212502ejf.616.2021.11.09.01.24.05; Tue, 09 Nov 2021 01:24:06 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D51A768AC4D; Tue, 9 Nov 2021 11:23:04 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AD15168AB7A for ; Tue, 9 Nov 2021 11:22:56 +0200 (EET) X-IronPort-AV: E=McAfee;i="6200,9189,10162"; a="295849401" X-IronPort-AV: E=Sophos;i="5.87,219,1631602800"; d="scan'208";a="295849401" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2021 01:22:54 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,219,1631602800"; d="scan'208";a="491593978" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.110]) by orsmga007.jf.intel.com with ESMTP; 09 Nov 2021 01:22:53 -0800 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Nov 2021 17:18:43 +0800 Message-Id: <20211109091843.2024927-7-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211109091843.2024927-1-wenbin.chen@intel.com> References: <20211109091843.2024927-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 7/7] libavutil/hwcontext_vulkan: specify the modifier to create VKImage X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: uO/sc//7mqJG When vulkan image exports to drm, the tilling need to be VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT. Now add code to create vulkan image using this format. Now the following command line works: ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format \ vaapi -i input_1080p.264 -vf "hwmap=derive_device=vulkan,format=vulkan, \ scale_vulkan=1920:1080,hwmap=derive_device=vaapi,format=vaapi" -c:v h264_vaapi output.264 Signed-off-by: Wenbin Chen --- libavutil/hwcontext_vulkan.c | 76 +++++++++++++++++++++++++++++++++--- libavutil/hwcontext_vulkan.h | 5 +++ 2 files changed, 75 insertions(+), 6 deletions(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 29ade94b7f..e252c2177e 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -1919,6 +1919,7 @@ static void try_export_flags(AVHWFramesContext *hwfc, AVVulkanDeviceContext *dev_hwctx = hwfc->device_ctx->hwctx; VulkanDevicePriv *p = hwfc->device_ctx->internal->priv; FFVulkanFunctions *vk = &p->vkfn; + const int has_modifiers = hwctx->tiling == VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT; VkExternalImageFormatProperties eprops = { .sType = VK_STRUCTURE_TYPE_EXTERNAL_IMAGE_FORMAT_PROPERTIES_KHR, }; @@ -1926,9 +1927,18 @@ static void try_export_flags(AVHWFramesContext *hwfc, .sType = VK_STRUCTURE_TYPE_IMAGE_FORMAT_PROPERTIES_2, .pNext = &eprops, }; + VkPhysicalDeviceImageDrmFormatModifierInfoEXT phy_dev_mod_info = { + .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_IMAGE_DRM_FORMAT_MODIFIER_INFO_EXT, + .pNext = NULL, + .pQueueFamilyIndices = p->qfs, + .queueFamilyIndexCount = p->num_qfs, + .sharingMode = p->num_qfs > 1 ? VK_SHARING_MODE_CONCURRENT : + VK_SHARING_MODE_EXCLUSIVE, + }; VkPhysicalDeviceExternalImageFormatInfo enext = { .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_EXTERNAL_IMAGE_FORMAT_INFO, .handleType = exp, + .pNext = has_modifiers ? &phy_dev_mod_info : NULL, }; VkPhysicalDeviceImageFormatInfo2 pinfo = { .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_IMAGE_FORMAT_INFO_2, @@ -1940,11 +1950,15 @@ static void try_export_flags(AVHWFramesContext *hwfc, .flags = VK_IMAGE_CREATE_ALIAS_BIT, }; - ret = vk->GetPhysicalDeviceImageFormatProperties2(dev_hwctx->phys_dev, - &pinfo, &props); - if (ret == VK_SUCCESS) { - *iexp |= exp; - *comp_handle_types |= eprops.externalMemoryProperties.compatibleHandleTypes; + for (int i = 0; i < (has_modifiers ? hwctx->modifier_count : 1); i++) { + if (has_modifiers && hwctx->modifier_count) + phy_dev_mod_info.drmFormatModifier = hwctx->modifiers[i]; + ret = vk->GetPhysicalDeviceImageFormatProperties2(dev_hwctx->phys_dev, + &pinfo, &props); + if (ret == VK_SUCCESS) { + *iexp |= exp; + *comp_handle_types |= eprops.externalMemoryProperties.compatibleHandleTypes; + } } } @@ -2007,6 +2021,7 @@ fail: static void vulkan_frames_uninit(AVHWFramesContext *hwfc) { VulkanFramesPriv *fp = hwfc->internal->priv; + AVVulkanFramesContext *hwctx = hwfc->hwctx; free_exec_ctx(hwfc, &fp->conv_ctx); free_exec_ctx(hwfc, &fp->upload_ctx); @@ -2021,11 +2036,60 @@ static int vulkan_frames_init(AVHWFramesContext *hwfc) VulkanFramesPriv *fp = hwfc->internal->priv; AVVulkanDeviceContext *dev_hwctx = hwfc->device_ctx->hwctx; VulkanDevicePriv *p = hwfc->device_ctx->internal->priv; + const int has_modifiers = !!(p->extensions & FF_VK_EXT_DRM_MODIFIER_FLAGS); /* Default pool flags */ - hwctx->tiling = hwctx->tiling ? hwctx->tiling : p->use_linear_images ? + hwctx->tiling = hwctx->tiling ? hwctx->tiling : has_modifiers ? + VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT : p->use_linear_images ? VK_IMAGE_TILING_LINEAR : VK_IMAGE_TILING_OPTIMAL; + /* get the supported modifier */ + if (has_modifiers) { + const VkFormat *fmt = av_vkfmt_from_pixfmt(hwfc->sw_format); + FFVulkanFunctions *vk = &p->vkfn; + VkDrmFormatModifierPropertiesEXT mod_props[MAX_VULKAN_MODIFIERS]; + + VkDrmFormatModifierPropertiesListEXT mod_props_list = { + .sType = VK_STRUCTURE_TYPE_DRM_FORMAT_MODIFIER_PROPERTIES_LIST_EXT, + .pNext = NULL, + .drmFormatModifierCount = 0, + .pDrmFormatModifierProperties = NULL, + }; + VkFormatProperties2 prop = { + .sType = VK_STRUCTURE_TYPE_FORMAT_PROPERTIES_2, + .pNext = &mod_props_list, + }; + + vk->GetPhysicalDeviceFormatProperties2(dev_hwctx->phys_dev, fmt[0], &prop); + if (!mod_props_list.drmFormatModifierCount) { + av_log(hwfc, AV_LOG_ERROR, "There are not supported modifiers for sw_format\n"); + return AVERROR(EINVAL); + } + mod_props_list.drmFormatModifierCount = + FFMIN(mod_props_list.drmFormatModifierCount, MAX_VULKAN_MODIFIERS); + + mod_props_list.pDrmFormatModifierProperties = mod_props; + vk->GetPhysicalDeviceFormatProperties2(dev_hwctx->phys_dev, fmt[0], &prop); + + hwctx->modifier_count = 0; + for (int i = 0; i < mod_props_list.drmFormatModifierCount; i++) { + if (!(mod_props[i].drmFormatModifierTilingFeatures & DEFAULT_USAGE_FLAGS)) + continue; + hwctx->modifiers[hwctx->modifier_count++] = mod_props[i].drmFormatModifier; + } + if (!hwctx->modifier_count) { + av_log(hwfc, AV_LOG_ERROR, "The supported modifiers doesn't support" + "default usage\n"); + return AVERROR(EINVAL); + } + + hwctx->create_pnext = &hwctx->modifier_info; + hwctx->modifier_info.pNext = NULL; + hwctx->modifier_info.sType = VK_STRUCTURE_TYPE_IMAGE_DRM_FORMAT_MODIFIER_LIST_CREATE_INFO_EXT; + hwctx->modifier_info.drmFormatModifierCount = hwctx->modifier_count; + hwctx->modifier_info.pDrmFormatModifiers = hwctx->modifiers; + } + if (!hwctx->usage) hwctx->usage = DEFAULT_USAGE_FLAGS; diff --git a/libavutil/hwcontext_vulkan.h b/libavutil/hwcontext_vulkan.h index efb602ef27..113448bb51 100644 --- a/libavutil/hwcontext_vulkan.h +++ b/libavutil/hwcontext_vulkan.h @@ -32,6 +32,7 @@ * with the data pointer set to an AVVkFrame. */ +#define MAX_VULKAN_MODIFIERS 16 /** * Main Vulkan context, allocated as AVHWDeviceContext.hwctx. * All of these can be set before init to change what the context uses @@ -149,6 +150,10 @@ typedef struct AVVulkanFramesContext { */ void *create_pnext; + VkImageDrmFormatModifierListCreateInfoEXT modifier_info; + uint64_t modifiers[MAX_VULKAN_MODIFIERS]; + int modifier_count; + /** * Extension data for memory allocation. Must have as many entries as * the number of planes of the sw_format.