From patchwork Thu Oct 5 21:26:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lynne X-Patchwork-Id: 44166 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:7a24:b0:15d:8365:d4b8 with SMTP id t36csp304967pzh; Thu, 5 Oct 2023 14:27:01 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFq7eNTRpGnZyEGLxtDsE1YkpM67LalLyS41amyBpTXRev9ah7VJRbyCICS3/BMzuYeex04 X-Received: by 2002:a17:906:da:b0:9ae:5253:175b with SMTP id 26-20020a17090600da00b009ae5253175bmr5342452eji.34.1696541220827; Thu, 05 Oct 2023 14:27:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696541220; cv=none; d=google.com; s=arc-20160816; b=lMDn+4OC9myX6LlDNnGjM/0FSMEpA3I8Pheuvci6evuoGKCdxRfm6zmUiYML8vZlp0 5zMqdguVcrWMyFGE7Tyt10kun5m28zMhR6bytilzJWyzBMV2Kqavyb4tJyuhuqcIYr5t e3DI6N7zMXB8vwaGUDGY0mm0Y26/hXqv92MZTdHGmrKH+yhcVMs53Wi9s8EF8Nelj1+5 k3RgYViBKkfNh1iMMKWC6ioRi9ulUIEu07lRJHkgpNEpkTyg7RqDEVJC5A6qEIe/bLP/ 99NwVyB+flZBs3bIGdzdvHx34URK7fQSuG6kjP2SLDDCsCYeP8lUcIyICSW8nNMaRjLR OIiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:message-id:to:from:date:dkim-signature:delivered-to; bh=MBWpBAw8eOZOtGOleWIyfyXqz2uMPhSnf4q1HnrZyDs=; fh=Q46kXK7oI5D1Jhi90JBr53c7NIaTxGaU4KPeRZyM/hI=; b=VhgKmkP1zLA5Yj0oo01deF4I1M5RUcYaS2HTmG/8Wzpsu8i1stMW9KEvYhCbY0AUKy WJ8gtVjMYs+8WJcqIaFPhynbGDtNL5jayeJJ00iGrVtyxfiu4uc2tBWeEzcqDjM+jpvL 0YI5RCwbTghveGXSqK7IdOoXiQKF0ETeqfQkrO4TsmRpW4jepocYsYRpIUyb9OHKorD+ zE2DBRqKA61mUOkXfJEkXU+pr9kaXoKRBrNd3TtDG6Ucx3ajgCeVHjGV7+ag/uCiKaNX dfy2IM3Arzq04KoYru4ipuAvQbVLpxygaa/qClymTt0735JBwudwsevIVUmY0hXJfpIX 4t6A== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@lynne.ee header.s=s1 header.b=r3EwOZLp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=lynne.ee Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s23-20020a170906bc5700b009b27d493c7asi1024799ejv.1055.2023.10.05.14.26.50; Thu, 05 Oct 2023 14:27:00 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@lynne.ee header.s=s1 header.b=r3EwOZLp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=lynne.ee Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5BD0C68CB53; Fri, 6 Oct 2023 00:26:46 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from w4.tutanota.de (w4.tutanota.de [81.3.6.165]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4327F68CAED for ; Fri, 6 Oct 2023 00:26:39 +0300 (EEST) Received: from tutadb.w10.tutanota.de (unknown [192.168.1.10]) by w4.tutanota.de (Postfix) with ESMTP id 2546B10600E8 for ; Thu, 5 Oct 2023 21:26:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1696541198; s=s1; d=lynne.ee; h=From:From:To:To:Subject:Subject:Content-Description:Content-ID:Content-Type:Content-Type:Content-Transfer-Encoding:Cc:Date:Date:In-Reply-To:MIME-Version:MIME-Version:Message-ID:Message-ID:Reply-To:References:Sender; bh=WO3WiQWiQK7r3OX9T93fHeU7CsH8XSBYBfzSVkmZ2HU=; b=r3EwOZLpSb7ZkLNoMZ+kg3LEXwSUu0U6mDXsmM7v1uKwfPXBuBgZurrY8DOmmn26 Iag4v1o/6MxnjalKQEpOxmOLMf9x/gnMMguZqSXLa9SOaQLj+3hljLmJnKtKeH1D4MM NNtFzTucEMJjURUcnGG0TxMYGgvCknM6tQGfo17YRbEaasUHgF3+cK64iBeVVXEPAMo ZHecok0aALE+N5Lm45cc3KGfMeTmz5+kYjK3Lp8UvVjbceqiGLJp2KVcmuCPByGUEVd KuIlbEtOTVWYFv0AzHgpkCxFRYhc9HVh5TGn4rGVBpykH/ybzE+Pzuczp29HD5xdF7O j9L5OXuBcw== Date: Thu, 5 Oct 2023 23:26:38 +0200 (CEST) From: Lynne To: Ffmpeg Devel Message-ID: MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] hwcontext_vulkan: add the VK_IMAGE_USAGE_SAMPLED_BIT bit to new images X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: L3Zh35UH64wQ This fixes filtering on Nvidia. Patch attached. From 2ea0d50049bae792477c53780b3cfd0e3863d914 Mon Sep 17 00:00:00 2001 From: Lynne Date: Thu, 5 Oct 2023 20:25:33 +0200 Subject: [PATCH] hwcontext_vulkan: properly support STORAGE usage for mutliplane images Fixes multiplane support on Nvidia. Also, remove the ENCODE usage, even if the driver signals it as supported. Currently, it's not used, and when it is used, it'll be gated behind two extension checks. --- libavfilter/vulkan_filter.c | 24 +++++++++++++++--------- libavutil/hwcontext_vulkan.c | 36 ++++++++++++++++++++++++------------ 2 files changed, 39 insertions(+), 21 deletions(-) diff --git a/libavfilter/vulkan_filter.c b/libavfilter/vulkan_filter.c index b4d8f952b5..f3f40465be 100644 --- a/libavfilter/vulkan_filter.c +++ b/libavfilter/vulkan_filter.c @@ -42,18 +42,23 @@ int ff_vk_filter_init_context(AVFilterContext *avctx, FFVulkanContext *s, vk_frames = frames_ctx->hwctx; vk_dev = device_ctx->hwctx; - /* Basic format validation */ + /* Width and height mismatch */ if (width != frames_ctx->width || - height != frames_ctx->height || - sw_format != frames_ctx->sw_format || - (vk_frames->tiling != VK_IMAGE_TILING_LINEAR && - vk_frames->tiling != VK_IMAGE_TILING_OPTIMAL) || - !(vk_frames->usage & VK_IMAGE_USAGE_SAMPLED_BIT)) { + height != frames_ctx->height) + goto skip; + + /* Format mismatch */ + if (sw_format != frames_ctx->sw_format) goto skip; - } - if (vk_frames->usage & VK_IMAGE_USAGE_STORAGE_BIT) - goto accept; + /* Unusual tiling mismatch. Don't let linear through either. */ + if (vk_frames->tiling != VK_IMAGE_TILING_OPTIMAL) + goto skip; + + /* Usage mismatch */ + if ((vk_frames->usage & (VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_STORAGE_BIT)) != + (VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_STORAGE_BIT)) + goto skip; s->extensions = ff_vk_extensions_to_mask(vk_dev->enabled_dev_extensions, vk_dev->nb_enabled_dev_extensions); @@ -110,6 +115,7 @@ accept: vk_frames = frames_ctx->hwctx; vk_frames->tiling = VK_IMAGE_TILING_OPTIMAL; vk_frames->usage = VK_IMAGE_USAGE_SAMPLED_BIT | + VK_IMAGE_USAGE_STORAGE_BIT | VK_IMAGE_USAGE_TRANSFER_SRC_BIT | VK_IMAGE_USAGE_TRANSFER_DST_BIT; diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index c676f4fc57..b955ec483e 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -282,9 +282,11 @@ FN_MAP_TO(VkImageUsageFlags, usage, VkFormatFeatureFlagBits2, feats) static int vkfmt_from_pixfmt2(AVHWDeviceContext *dev_ctx, enum AVPixelFormat p, VkImageTiling tiling, - VkFormat fmts[AV_NUM_DATA_POINTERS], - int *nb_images, VkImageAspectFlags *aspect, - VkImageUsageFlags *supported_usage, int disable_multiplane) + VkFormat fmts[AV_NUM_DATA_POINTERS], /* Output format list */ + int *nb_images, /* Output number of images */ + VkImageAspectFlags *aspect, /* Output aspect */ + VkImageUsageFlags *supported_usage, /* Output supported usage */ + int disable_multiplane, int need_storage) { AVVulkanDeviceContext *hwctx = dev_ctx->hwctx; VulkanDevicePriv *priv = dev_ctx->internal->priv; @@ -301,6 +303,7 @@ static int vkfmt_from_pixfmt2(AVHWDeviceContext *dev_ctx, enum AVPixelFormat p, }; VkFormatFeatureFlagBits2 feats_primary, feats_secondary; int basics_primary = 0, basics_secondary = 0; + int storage_primary = 0, storage_secondary = 0; vk->GetPhysicalDeviceFormatProperties2(hwctx->phys_dev, vk_formats_list[i].vkf, @@ -310,6 +313,7 @@ static int vkfmt_from_pixfmt2(AVHWDeviceContext *dev_ctx, enum AVPixelFormat p, prop.formatProperties.linearTilingFeatures : prop.formatProperties.optimalTilingFeatures; basics_primary = (feats_primary & basic_flags) == basic_flags; + storage_primary = !!(feats_primary & VK_FORMAT_FEATURE_2_STORAGE_IMAGE_BIT); if (vk_formats_list[i].vkf != vk_formats_list[i].fallback[0]) { vk->GetPhysicalDeviceFormatProperties2(hwctx->phys_dev, @@ -319,11 +323,15 @@ static int vkfmt_from_pixfmt2(AVHWDeviceContext *dev_ctx, enum AVPixelFormat p, prop.formatProperties.linearTilingFeatures : prop.formatProperties.optimalTilingFeatures; basics_secondary = (feats_secondary & basic_flags) == basic_flags; + storage_secondary = !!(feats_secondary & VK_FORMAT_FEATURE_2_STORAGE_IMAGE_BIT); } else { basics_secondary = basics_primary; + storage_secondary = storage_primary; } - if (basics_primary && !(disable_multiplane && vk_formats_list[i].vk_planes > 1)) { + if (basics_primary && + !(disable_multiplane && vk_formats_list[i].vk_planes > 1) && + (!need_storage || (need_storage && (storage_primary | storage_secondary)))) { if (fmts) fmts[0] = vk_formats_list[i].vkf; if (nb_images) @@ -331,9 +339,12 @@ static int vkfmt_from_pixfmt2(AVHWDeviceContext *dev_ctx, enum AVPixelFormat p, if (aspect) *aspect = vk_formats_list[i].aspect; if (supported_usage) - *supported_usage = map_feats_to_usage(feats_primary); + *supported_usage = map_feats_to_usage(feats_primary) | + ((need_storage && (storage_primary | storage_secondary)) ? + VK_IMAGE_USAGE_STORAGE_BIT : 0); return 0; - } else if (basics_secondary) { + } else if (basics_secondary && + (!need_storage || (need_storage && storage_secondary))) { if (fmts) { for (int j = 0; j < vk_formats_list[i].nb_images_fallback; j++) fmts[j] = vk_formats_list[i].fallback[j]; @@ -1640,7 +1651,7 @@ static int vulkan_frames_get_constraints(AVHWDeviceContext *ctx, count += vkfmt_from_pixfmt2(ctx, vk_formats_list[i].pixfmt, p->use_linear_images ? VK_IMAGE_TILING_LINEAR : VK_IMAGE_TILING_OPTIMAL, - NULL, NULL, NULL, NULL, 0) >= 0; + NULL, NULL, NULL, NULL, 0, 0) >= 0; } #if CONFIG_CUDA @@ -1658,7 +1669,7 @@ static int vulkan_frames_get_constraints(AVHWDeviceContext *ctx, if (vkfmt_from_pixfmt2(ctx, vk_formats_list[i].pixfmt, p->use_linear_images ? VK_IMAGE_TILING_LINEAR : VK_IMAGE_TILING_OPTIMAL, - NULL, NULL, NULL, NULL, 0) >= 0) { + NULL, NULL, NULL, NULL, 0, 0) >= 0) { constraints->valid_sw_formats[count++] = vk_formats_list[i].pixfmt; } } @@ -2294,7 +2305,8 @@ static int vulkan_frames_init(AVHWFramesContext *hwfc) /* Check if the sw_format itself is supported */ err = vkfmt_from_pixfmt2(hwfc->device_ctx, hwfc->sw_format, hwctx->tiling, NULL, - NULL, NULL, &supported_usage, 0); + NULL, NULL, &supported_usage, 0, + hwctx->usage & VK_IMAGE_USAGE_STORAGE_BIT); if (err < 0) { av_log(hwfc, AV_LOG_ERROR, "Unsupported sw format: %s!\n", av_get_pix_fmt_name(hwfc->sw_format)); @@ -2304,7 +2316,8 @@ static int vulkan_frames_init(AVHWFramesContext *hwfc) err = vkfmt_from_pixfmt2(hwfc->device_ctx, hwfc->sw_format, hwctx->tiling, hwctx->format, NULL, NULL, &supported_usage, - disable_multiplane); + disable_multiplane, + hwctx->usage & VK_IMAGE_USAGE_STORAGE_BIT); if (err < 0) return err; } @@ -2314,8 +2327,7 @@ static int vulkan_frames_init(AVHWFramesContext *hwfc) hwctx->usage = supported_usage & (VK_BUFFER_USAGE_TRANSFER_DST_BIT | VK_BUFFER_USAGE_TRANSFER_SRC_BIT | VK_IMAGE_USAGE_STORAGE_BIT | - VK_IMAGE_USAGE_SAMPLED_BIT | - VK_IMAGE_USAGE_VIDEO_ENCODE_SRC_BIT_KHR); + VK_IMAGE_USAGE_SAMPLED_BIT); } /* Image creation flags. -- 2.42.0