From patchwork Sun Jun 18 12:19:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lynne X-Patchwork-Id: 42205 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:be15:b0:121:b37c:e101 with SMTP id ge21csp82505pzb; Sun, 18 Jun 2023 05:20:03 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4/aAWumL3VaDq08x5K3nYze8vbMsZ5eEpgJibXTtZEqcYF2wZLbtZzLQSux8522TAZtud+ X-Received: by 2002:a17:907:d93:b0:982:a579:4a1e with SMTP id go19-20020a1709070d9300b00982a5794a1emr8397739ejc.14.1687090803047; Sun, 18 Jun 2023 05:20:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687090803; cv=none; d=google.com; s=arc-20160816; b=XJFiSyXNDCR8GOBfrck1+I10oCNcbwLkoOB1JJpF521lWuxU7CHesozyo/XB4WeSyq 41QDXXZkMoqsHyp7D+F1zecrmw2pPvg89raBX/WHYPZzuDLKlM0fHbzEiTGQ4lgZ6ylX e/6Eeh60x3zEv7hYeMdH0979osGRom132uydXSZuuSVuxFFGvHFjxvBSbyuEh6JuyDF2 pM/E4bjpGYkr12dwm1v+oa+HDqxVyf9koAzmyC3Zi2K+jfuQHcp2KQnB9mRPm+Qj0F5t Brkx5/TGlU1+hPR+//CWLZC+VBz+aTW4MrJpo/eJu+Lt3jH415WpFgwSopo7fjsnsSlg hKAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:message-id:to:from:date:dkim-signature:delivered-to; bh=xiN6ePAoIbjKg7pKegl+430zR9mfXYqhXv/Y0UWcoyE=; b=T8T0WU9GddD44kRwKgUK51Lo4tWPhs7I/aqEwj86LcRl3hbglSVenEoGmZ3Ng5FV06 NL5FePQJlG4yMge+AijOI7fO2TP5dg+oB1oPjM9kBKjYDUSeSCbNJMkaJfGmKVEGzeO0 2gMIICbVRknoJnj32MwK+OcccUcwdkvULAezO7EfP4Ww+pQx5gmaeVyg8wFuKLWT0U4R dnh+38Oqr7PPZCz8zI9UXNaAbCLSr8xMtVTwTzZcFn84tGW2g3+kbft3F/b3vSEcfpVv Lx44qOyt1J7zH+4FJeJeNGCwvJvVPt71EYMgBBhe1D7cqpg7iV3GUzK+myhHbsU9kWkc hTQA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@lynne.ee header.s=s1 header.b=18YvKMWT; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=lynne.ee Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f5-20020a1709062c4500b00987caa55379si1389193ejh.547.2023.06.18.05.20.02; Sun, 18 Jun 2023 05:20:03 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@lynne.ee header.s=s1 header.b=18YvKMWT; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=lynne.ee Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 415E368C025; Sun, 18 Jun 2023 15:20:00 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from w4.tutanota.de (w4.tutanota.de [81.3.6.165]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 09DD868C025 for ; Sun, 18 Jun 2023 15:19:54 +0300 (EEST) Received: from tutadb.w10.tutanota.de (unknown [192.168.1.10]) by w4.tutanota.de (Postfix) with ESMTP id 142061060136 for ; Sun, 18 Jun 2023 12:19:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1687090793; s=s1; d=lynne.ee; h=From:From:To:To:Subject:Subject:Content-Description:Content-ID:Content-Type:Content-Type:Content-Transfer-Encoding:Cc:Date:Date:In-Reply-To:MIME-Version:MIME-Version:Message-ID:Message-ID:Reply-To:References:Sender; bh=53pPdsEZ5ju/KQfab0dKE2C1J6EyAY+Me01NGtivCY4=; b=18YvKMWT3JVwuiQ+6RKB5STHn/1mNf/PPaCZMjaCX81v/8CEk54D+iBG9Vl0T6j+ ZkIRURDygjwgkrIvkaL+J1iPJd/38YqssMlPO+sBHJcVCZwTQ+gcno5NQ7J9tFkzUo9 liNdR0cbD9K2s5SDZ8gocuWuwX5Md0e5b91lZxPJvirWqPhHZIKu8UHGzgee/ZO5TFi RdA8j3p2BnQKqbi/T9EEBOCS86vn8MXw2SlEcQGYdWP12V41PjUNYjcQhu++p+E+pb3 7oSeHZW9vwlIe5MspPNgyTexq7/sbP66do37CLHE1kOjbiygWZmVMPq6Im837F4TPoE 2UE/6TBJsw== Date: Sun, 18 Jun 2023 14:19:53 +0200 (CEST) From: Lynne To: Ffmpeg Devel Message-ID: MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] vulkan_decode: use the hwfc->user_opaque field to store the profile X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: dXL0W4ETI9ip Patch attached. From fe6516e22880a067420a6e21a6928ad176f66b45 Mon Sep 17 00:00:00 2001 From: Lynne Date: Tue, 13 Jun 2023 06:10:20 +0200 Subject: [PATCH] vulkan_decode: use the hwfc->user_opaque field to store the profile --- libavcodec/vulkan_decode.c | 60 ++++++++++++++++++++++++-------------- libavcodec/vulkan_decode.h | 6 ++-- 2 files changed, 41 insertions(+), 25 deletions(-) diff --git a/libavcodec/vulkan_decode.c b/libavcodec/vulkan_decode.c index 6ba939da46..7ccee94b01 100644 --- a/libavcodec/vulkan_decode.c +++ b/libavcodec/vulkan_decode.c @@ -196,7 +196,6 @@ int ff_vk_decode_add_slice(AVCodecContext *avctx, FFVulkanDecodePicture *vp, { FFVulkanDecodeContext *dec = avctx->internal->hwaccel_priv_data; FFVulkanDecodeShared *ctx = (FFVulkanDecodeShared *)dec->shared_ref->data; - FFVulkanDecodeProfileData *prof = &ctx->profile_data; static const uint8_t startcode_prefix[3] = { 0x0, 0x0, 0x1 }; const size_t startcode_len = add_startcode ? sizeof(startcode_prefix) : 0; @@ -206,8 +205,8 @@ int ff_vk_decode_add_slice(AVCodecContext *avctx, FFVulkanDecodePicture *vp, FFVkVideoBuffer *vkbuf; size_t new_size = vp->slices_size + startcode_len + size + - prof->caps.minBitstreamBufferSizeAlignment; - new_size = FFALIGN(new_size, prof->caps.minBitstreamBufferSizeAlignment); + ctx->caps.minBitstreamBufferSizeAlignment; + new_size = FFALIGN(new_size, ctx->caps.minBitstreamBufferSizeAlignment); slice_off = av_fast_realloc(vp->slice_off, &vp->slice_off_max, (nb + 1)*sizeof(slice_off)); @@ -295,7 +294,6 @@ int ff_vk_decode_frame(AVCodecContext *avctx, FFVulkanDecodeContext *dec = avctx->internal->hwaccel_priv_data; FFVulkanDecodeShared *ctx = (FFVulkanDecodeShared *)dec->shared_ref->data; - FFVulkanDecodeProfileData *prof = &ctx->profile_data; FFVulkanFunctions *vk = &ctx->s.vkfn; /* Output */ @@ -319,7 +317,7 @@ int ff_vk_decode_frame(AVCodecContext *avctx, VkImageMemoryBarrier2 img_bar[37]; int nb_img_bar = 0; size_t data_size = FFALIGN(vp->slices_size, - prof->caps.minBitstreamBufferSizeAlignment); + ctx->caps.minBitstreamBufferSizeAlignment); FFVkExecContext *exec = ff_vk_exec_get(&ctx->exec_pool); @@ -640,10 +638,10 @@ static VkResult vulkan_setup_profile(AVCodecContext *avctx, VkVideoDecodeH264CapabilitiesKHR *h264_caps, VkVideoDecodeH265CapabilitiesKHR *h265_caps, VkVideoDecodeAV1CapabilitiesMESA *av1_caps, + VkVideoCapabilitiesKHR *caps, + VkVideoDecodeCapabilitiesKHR *dec_caps, int cur_profile) { - VkVideoCapabilitiesKHR *caps = &prof->caps; - VkVideoDecodeCapabilitiesKHR *dec_caps = &prof->dec_caps; VkVideoDecodeUsageInfoKHR *usage = &prof->usage; VkVideoProfileInfoKHR *profile = &prof->profile; VkVideoProfileListInfoKHR *profile_list = &prof->profile_list; @@ -708,6 +706,7 @@ static VkResult vulkan_setup_profile(AVCodecContext *avctx, static int vulkan_decode_get_profile(AVCodecContext *avctx, AVBufferRef *frames_ref, enum AVPixelFormat *pix_fmt, VkFormat *vk_fmt, + FFVulkanDecodeProfileData *prof, int *dpb_dedicate) { VkResult ret; @@ -724,9 +723,8 @@ static int vulkan_decode_get_profile(AVCodecContext *avctx, AVBufferRef *frames_ FFVulkanDecodeShared *ctx = (FFVulkanDecodeShared *)dec->shared_ref->data; FFVulkanFunctions *vk = &ctx->s.vkfn; - FFVulkanDecodeProfileData *prof = &ctx->profile_data; - VkVideoCapabilitiesKHR *caps = &prof->caps; - VkVideoDecodeCapabilitiesKHR *dec_caps = &prof->dec_caps; + VkVideoCapabilitiesKHR *caps = &ctx->caps; + VkVideoDecodeCapabilitiesKHR *dec_caps = &ctx->dec_caps; VkVideoDecodeH264CapabilitiesKHR h264_caps = { .sType = VK_STRUCTURE_TYPE_VIDEO_DECODE_H264_CAPABILITIES_KHR, @@ -765,6 +763,8 @@ static int vulkan_decode_get_profile(AVCodecContext *avctx, AVBufferRef *frames_ &h264_caps, &h265_caps, &av1_caps, + caps, + dec_caps, cur_profile); if (ret == VK_ERROR_VIDEO_PROFILE_OPERATION_NOT_SUPPORTED_KHR && avctx->flags & AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH && @@ -779,6 +779,8 @@ static int vulkan_decode_get_profile(AVCodecContext *avctx, AVBufferRef *frames_ &h264_caps, &h265_caps, &av1_caps, + caps, + dec_caps, cur_profile); } @@ -965,6 +967,11 @@ static int vulkan_decode_get_profile(AVCodecContext *avctx, AVBufferRef *frames_ return 0; } +static void free_profile_data(AVHWFramesContext *hwfc) +{ + av_free(hwfc->user_opaque); +} + int ff_vk_frame_params(AVCodecContext *avctx, AVBufferRef *hw_frames_ctx) { VkFormat vkfmt; @@ -972,7 +979,6 @@ int ff_vk_frame_params(AVCodecContext *avctx, AVBufferRef *hw_frames_ctx) AVHWFramesContext *frames_ctx = (AVHWFramesContext*)hw_frames_ctx->data; AVVulkanFramesContext *hwfc = frames_ctx->hwctx; FFVulkanDecodeContext *dec = avctx->internal->hwaccel_priv_data; - FFVulkanDecodeShared *ctx; FFVulkanDecodeProfileData *prof; frames_ctx->sw_format = AV_PIX_FMT_NONE; @@ -981,15 +987,19 @@ int ff_vk_frame_params(AVCodecContext *avctx, AVBufferRef *hw_frames_ctx) if (err < 0) return err; - ctx = (FFVulkanDecodeShared *)dec->shared_ref->data; - prof = &ctx->profile_data; + prof = av_mallocz(sizeof(FFVulkanDecodeProfileData)); + if (!prof) + return AVERROR(ENOMEM); err = vulkan_decode_get_profile(avctx, hw_frames_ctx, &frames_ctx->sw_format, &vkfmt, - &dedicated_dpb); + prof, &dedicated_dpb); if (err < 0) return err; + frames_ctx->user_opaque = prof; + frames_ctx->free = free_profile_data; + frames_ctx->width = avctx->width; frames_ctx->height = avctx->height; frames_ctx->format = AV_PIX_FMT_VULKAN; @@ -1032,10 +1042,10 @@ int ff_vk_decode_init(AVCodecContext *avctx) VkResult ret; FFVulkanDecodeContext *dec = avctx->internal->hwaccel_priv_data; FFVulkanDecodeShared *ctx; - FFVulkanDecodeProfileData *prof; FFVulkanContext *s; FFVulkanFunctions *vk; FFVkQueueFamilyCtx qf_dec; + const VkVideoProfileListInfoKHR *profile_list; VkVideoDecodeH264SessionParametersCreateInfoKHR h264_params = { .sType = VK_STRUCTURE_TYPE_VIDEO_DECODE_H264_SESSION_PARAMETERS_CREATE_INFO_KHR, @@ -1069,7 +1079,6 @@ int ff_vk_decode_init(AVCodecContext *avctx) /* Initialize contexts */ ctx = (FFVulkanDecodeShared *)dec->shared_ref->data; - prof = &ctx->profile_data; s = &ctx->s; vk = &ctx->s.vkfn; @@ -1080,6 +1089,13 @@ int ff_vk_decode_init(AVCodecContext *avctx) s->device = (AVHWDeviceContext *)s->frames->device_ref->data; s->hwctx = s->device->hwctx; + profile_list = ff_vk_find_struct(s->hwfc->create_pnext, + VK_STRUCTURE_TYPE_VIDEO_PROFILE_LIST_INFO_KHR); + if (!profile_list) { + av_log(avctx, AV_LOG_ERROR, "Profile list missing from frames context!"); + return AVERROR(EINVAL); + } + err = ff_vk_load_props(s); if (err < 0) goto fail; @@ -1101,13 +1117,13 @@ int ff_vk_decode_init(AVCodecContext *avctx) session_create.flags = 0x0; session_create.queueFamilyIndex = s->hwctx->queue_family_decode_index; - session_create.maxCodedExtent = prof->caps.maxCodedExtent; - session_create.maxDpbSlots = prof->caps.maxDpbSlots; - session_create.maxActiveReferencePictures = prof->caps.maxActiveReferencePictures; + session_create.maxCodedExtent = ctx->caps.maxCodedExtent; + session_create.maxDpbSlots = ctx->caps.maxDpbSlots; + session_create.maxActiveReferencePictures = ctx->caps.maxActiveReferencePictures; session_create.pictureFormat = s->hwfc->format[0]; session_create.referencePictureFormat = session_create.pictureFormat; session_create.pStdHeaderVersion = dec_ext[avctx->codec_id]; - session_create.pVideoProfile = &prof->profile_list.pProfiles[0]; + session_create.pVideoProfile = &profile_list->pProfiles[0]; /* Create decode exec context. * 2 async contexts per thread was experimentally determined to be optimal @@ -1152,14 +1168,14 @@ int ff_vk_decode_init(AVCodecContext *avctx) dpb_frames->height = s->frames->height; dpb_hwfc = dpb_frames->hwctx; - dpb_hwfc->create_pnext = (void *)&prof->profile_list; + dpb_hwfc->create_pnext = (void *)profile_list; dpb_hwfc->format[0] = s->hwfc->format[0]; dpb_hwfc->tiling = VK_IMAGE_TILING_OPTIMAL; dpb_hwfc->usage = VK_IMAGE_USAGE_VIDEO_DECODE_DPB_BIT_KHR | VK_IMAGE_USAGE_SAMPLED_BIT; /* Shuts validator up. */ if (dec->layered_dpb) - dpb_hwfc->nb_layers = prof->caps.maxDpbSlots; + dpb_hwfc->nb_layers = ctx->caps.maxDpbSlots; err = av_hwframe_ctx_init(ctx->dpb_hwfc_ref); if (err < 0) diff --git a/libavcodec/vulkan_decode.h b/libavcodec/vulkan_decode.h index 681d2476cd..3ac103f6b4 100644 --- a/libavcodec/vulkan_decode.h +++ b/libavcodec/vulkan_decode.h @@ -26,8 +26,6 @@ #include "vulkan_video.h" typedef struct FFVulkanDecodeProfileData { - VkVideoCapabilitiesKHR caps; - VkVideoDecodeCapabilitiesKHR dec_caps; VkVideoDecodeH264ProfileInfoKHR h264_profile; VkVideoDecodeH264ProfileInfoKHR h265_profile; VkVideoDecodeAV1ProfileInfoMESA av1_profile; @@ -40,7 +38,9 @@ typedef struct FFVulkanDecodeShared { FFVulkanContext s; FFVkVideoCommon common; FFVkExecPool exec_pool; - FFVulkanDecodeProfileData profile_data; + + VkVideoCapabilitiesKHR caps; + VkVideoDecodeCapabilitiesKHR dec_caps; AVBufferRef *dpb_hwfc_ref; /* Only used for dedicated_dpb */ -- 2.40.1