From patchwork Tue Jun 13 04:19:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lynne X-Patchwork-Id: 42054 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c526:b0:117:ac03:c9de with SMTP id gm38csp3424943pzb; Mon, 12 Jun 2023 21:19:45 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6SBOVRUyq72XoEt2lxB1napDWsiJtqZMkTKnmBoMaWC4KVw0LgJR9G7wz2Bm6KHjhIAD0l X-Received: by 2002:a17:907:7b97:b0:982:3e4f:ed6e with SMTP id ne23-20020a1709077b9700b009823e4fed6emr433966ejc.47.1686629984579; Mon, 12 Jun 2023 21:19:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686629984; cv=none; d=google.com; s=arc-20160816; b=o1WrF4CGPizcgsIsVr1a+WSzkf6K0f6tiz6f3Fpt1hlcOKClejgFyKmKWUkCoKzXe2 +2LTejZyD12UR1REO9M2UkqbL61FlLVfRvg+I2Uh9+gSSMH2Toc4t+92rpPOe0w6zmqu 1uVg8/xPUe8q1hcaf0uNCFLPaP2HVUyuga21P614ThAE1avUw6QbkZAktmXxxcd0DAk5 IbQk7OtZO6m1xeftOpRwML34O0XzD6OmNavNFL26Ogvc67dzKZNR66iXM9tyhDdYk80l +yLUJQp3j2Y9uw/rXiMUYGw6gCbOBB6aO+K9XXBocfSNbdYbHV6E8PcDyqjr+/g6pS6x fIyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:to:from:date :dkim-signature:delivered-to; bh=SmkJ5dAzUD//nx2+yqnvGUZU4SX0+FagelAm8hrCVSw=; b=eqbi+B2TOVqM6Xd1Mb291st5P/3AItbFtCXu3snt1fEf0OtNWz7C52gaeWqT6cJT6z w2NSQ1He2A9u8qG76yYNo8TYgi+2lMWS45sHcsfbTQRkPPtSkEeWQdKE0pgwDWNbI2NS aWpu/cOiPfZ36ZI4NH/HUvJWS34s4sp1lYr6ocTTsTKpMzEGywyuPd049FJCWRBV4RJc 6tP2qbL9WxnmEQKMf4vve1htUeyPFTgZ6A0mHwGg9YE/vCJmpJJxiDPkdslt2JkSo1x+ Z6iKFGR3DFrsT4WmQ97S/TjWB2sOjJiv4KVxESZJapnldDA1wdo/5TG+furX1Z7qxXGD +x/Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@lynne.ee header.s=s1 header.b=qAcPBrJa; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=lynne.ee Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s15-20020a056402164f00b0050bd4957988si6590881edx.455.2023.06.12.21.19.44; Mon, 12 Jun 2023 21:19:44 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@lynne.ee header.s=s1 header.b=qAcPBrJa; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=lynne.ee Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DF32768C362; Tue, 13 Jun 2023 07:19:41 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from w4.tutanota.de (w4.tutanota.de [81.3.6.165]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 05D1168C1E2 for ; Tue, 13 Jun 2023 07:19:35 +0300 (EEST) Received: from tutadb.w10.tutanota.de (unknown [192.168.1.10]) by w4.tutanota.de (Postfix) with ESMTP id 136A21060148 for ; Tue, 13 Jun 2023 04:19:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1686629974; s=s1; d=lynne.ee; h=From:From:To:To:Subject:Subject:Content-Description:Content-ID:Content-Type:Content-Type:Content-Transfer-Encoding:Cc:Date:Date:In-Reply-To:In-Reply-To:MIME-Version:MIME-Version:Message-ID:Message-ID:Reply-To:References:References:Sender; bh=FBAjH4oA1FZS2+kUO9G410nohikmRKptH59bPs7Bgr0=; b=qAcPBrJaxgaAxXbxCV1t+DDDDFEeNzUC7SQmwc9nG9xCmxGxL5GMsmQAwB3SioNS zv24OLAptNim4SGR4zAllnHKpxnXgg/9JbGqzjBqCE7NRGEUh360zOlwRdrJFzbadhn msnnUo9jYhz7Lr94XYqCpPBuFOXFojmd9t/kshYHkK4HWKW8gyaeZkDjuasv4TLvKy2 Ez38nfWWxitqY19pB+CTlAhrjWXorpB0HQY2axNcJTCQILNiAa2CN2yoKdB1ZuHC8Qn XCzoeD9aIu1doFoelM/CMW+i971ZwCznFG+0MxDrW1vYTHIYOfi5HkuzZuPvgVdhLPw pJsytdRMIQ== Date: Tue, 13 Jun 2023 06:19:34 +0200 (CEST) From: Lynne To: FFmpeg development discussions and patches Message-ID: In-Reply-To: References: MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/5] vulkan_decode: use the new AVHWFramesContext.opaque field X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: PNKi0NzApZzJ This depends on the previous patch, and allows moving the codec profile to the new AVHWFramesContext.opaque field. Patch attached. From f992905250062711fab7522906a573ff8ab5f716 Mon Sep 17 00:00:00 2001 From: Lynne Date: Tue, 13 Jun 2023 06:10:20 +0200 Subject: [PATCH 2/5] vulkan_decode: use the new AVHWFramesContext.opaque field --- libavcodec/vulkan_decode.c | 56 +++++++++++++++++++++++--------------- libavcodec/vulkan_decode.h | 6 ++-- 2 files changed, 37 insertions(+), 25 deletions(-) diff --git a/libavcodec/vulkan_decode.c b/libavcodec/vulkan_decode.c index 35e265a5b1..81085273d8 100644 --- a/libavcodec/vulkan_decode.c +++ b/libavcodec/vulkan_decode.c @@ -196,7 +196,6 @@ int ff_vk_decode_add_slice(AVCodecContext *avctx, FFVulkanDecodePicture *vp, { FFVulkanDecodeContext *dec = avctx->internal->hwaccel_priv_data; FFVulkanDecodeShared *ctx = (FFVulkanDecodeShared *)dec->shared_ref->data; - FFVulkanDecodeProfileData *prof = &ctx->profile_data; static const uint8_t startcode_prefix[3] = { 0x0, 0x0, 0x1 }; const size_t startcode_len = add_startcode ? sizeof(startcode_prefix) : 0; @@ -206,8 +205,8 @@ int ff_vk_decode_add_slice(AVCodecContext *avctx, FFVulkanDecodePicture *vp, FFVkVideoBuffer *vkbuf; size_t new_size = vp->slices_size + startcode_len + size + - prof->caps.minBitstreamBufferSizeAlignment; - new_size = FFALIGN(new_size, prof->caps.minBitstreamBufferSizeAlignment); + ctx->caps.minBitstreamBufferSizeAlignment; + new_size = FFALIGN(new_size, ctx->caps.minBitstreamBufferSizeAlignment); slice_off = av_fast_realloc(vp->slice_off, &vp->slice_off_max, (nb + 1)*sizeof(slice_off)); @@ -295,7 +294,6 @@ int ff_vk_decode_frame(AVCodecContext *avctx, FFVulkanDecodeContext *dec = avctx->internal->hwaccel_priv_data; FFVulkanDecodeShared *ctx = (FFVulkanDecodeShared *)dec->shared_ref->data; - FFVulkanDecodeProfileData *prof = &ctx->profile_data; FFVulkanFunctions *vk = &ctx->s.vkfn; /* Output */ @@ -319,7 +317,7 @@ int ff_vk_decode_frame(AVCodecContext *avctx, VkImageMemoryBarrier2 img_bar[37]; int nb_img_bar = 0; size_t data_size = FFALIGN(vp->slices_size, - prof->caps.minBitstreamBufferSizeAlignment); + ctx->caps.minBitstreamBufferSizeAlignment); FFVkExecContext *exec = ff_vk_exec_get(&ctx->exec_pool); @@ -640,10 +638,10 @@ static VkResult vulkan_setup_profile(AVCodecContext *avctx, VkVideoDecodeH264CapabilitiesKHR *h264_caps, VkVideoDecodeH265CapabilitiesKHR *h265_caps, VkVideoDecodeAV1CapabilitiesMESA *av1_caps, + VkVideoCapabilitiesKHR *caps, + VkVideoDecodeCapabilitiesKHR *dec_caps, int cur_profile) { - VkVideoCapabilitiesKHR *caps = &prof->caps; - VkVideoDecodeCapabilitiesKHR *dec_caps = &prof->dec_caps; VkVideoDecodeUsageInfoKHR *usage = &prof->usage; VkVideoProfileInfoKHR *profile = &prof->profile; VkVideoProfileListInfoKHR *profile_list = &prof->profile_list; @@ -703,6 +701,7 @@ static VkResult vulkan_setup_profile(AVCodecContext *avctx, static int vulkan_decode_get_profile(AVCodecContext *avctx, AVBufferRef *frames_ref, enum AVPixelFormat *pix_fmt, VkFormat *vk_fmt, + FFVulkanDecodeProfileData *prof, int *dpb_dedicate) { VkResult ret; @@ -719,9 +718,8 @@ static int vulkan_decode_get_profile(AVCodecContext *avctx, AVBufferRef *frames_ FFVulkanDecodeShared *ctx = (FFVulkanDecodeShared *)dec->shared_ref->data; FFVulkanFunctions *vk = &ctx->s.vkfn; - FFVulkanDecodeProfileData *prof = &ctx->profile_data; - VkVideoCapabilitiesKHR *caps = &prof->caps; - VkVideoDecodeCapabilitiesKHR *dec_caps = &prof->dec_caps; + VkVideoCapabilitiesKHR *caps = &ctx->caps; + VkVideoDecodeCapabilitiesKHR *dec_caps = &ctx->dec_caps; VkVideoDecodeH264CapabilitiesKHR h264_caps = { .sType = VK_STRUCTURE_TYPE_VIDEO_DECODE_H264_CAPABILITIES_KHR, @@ -760,6 +758,8 @@ static int vulkan_decode_get_profile(AVCodecContext *avctx, AVBufferRef *frames_ &h264_caps, &h265_caps, &av1_caps, + caps, + dec_caps, cur_profile); if (ret == VK_ERROR_VIDEO_PROFILE_OPERATION_NOT_SUPPORTED_KHR && avctx->flags & AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH && @@ -774,6 +774,8 @@ static int vulkan_decode_get_profile(AVCodecContext *avctx, AVBufferRef *frames_ &h264_caps, &h265_caps, &av1_caps, + caps, + dec_caps, cur_profile); } @@ -967,8 +969,8 @@ int ff_vk_frame_params(AVCodecContext *avctx, AVBufferRef *hw_frames_ctx) AVHWFramesContext *frames_ctx = (AVHWFramesContext*)hw_frames_ctx->data; AVVulkanFramesContext *hwfc = frames_ctx->hwctx; FFVulkanDecodeContext *dec = avctx->internal->hwaccel_priv_data; - FFVulkanDecodeShared *ctx; FFVulkanDecodeProfileData *prof; + AVBufferRef *prof_ref; frames_ctx->sw_format = AV_PIX_FMT_NONE; @@ -976,15 +978,19 @@ int ff_vk_frame_params(AVCodecContext *avctx, AVBufferRef *hw_frames_ctx) if (err < 0) return err; - ctx = (FFVulkanDecodeShared *)dec->shared_ref->data; - prof = &ctx->profile_data; + prof_ref = av_buffer_allocz(sizeof(*prof)); + if (!prof_ref) + return AVERROR(ENOMEM); + + prof = (FFVulkanDecodeProfileData *)prof_ref->data; err = vulkan_decode_get_profile(avctx, hw_frames_ctx, &frames_ctx->sw_format, &vkfmt, - &dedicated_dpb); + prof, &dedicated_dpb); if (err < 0) return err; + frames_ctx->opaque = prof_ref; frames_ctx->width = avctx->width; frames_ctx->height = avctx->height; frames_ctx->format = AV_PIX_FMT_VULKAN; @@ -1027,10 +1033,10 @@ int ff_vk_decode_init(AVCodecContext *avctx) VkResult ret; FFVulkanDecodeContext *dec = avctx->internal->hwaccel_priv_data; FFVulkanDecodeShared *ctx; - FFVulkanDecodeProfileData *prof; FFVulkanContext *s; FFVulkanFunctions *vk; FFVkQueueFamilyCtx qf_dec; + const VkVideoProfileListInfoKHR *profile_list; VkVideoDecodeH264SessionParametersCreateInfoKHR h264_params = { .sType = VK_STRUCTURE_TYPE_VIDEO_DECODE_H264_SESSION_PARAMETERS_CREATE_INFO_KHR, @@ -1064,7 +1070,6 @@ int ff_vk_decode_init(AVCodecContext *avctx) /* Initialize contexts */ ctx = (FFVulkanDecodeShared *)dec->shared_ref->data; - prof = &ctx->profile_data; s = &ctx->s; vk = &ctx->s.vkfn; @@ -1075,6 +1080,13 @@ int ff_vk_decode_init(AVCodecContext *avctx) s->device = (AVHWDeviceContext *)s->frames->device_ref->data; s->hwctx = s->device->hwctx; + profile_list = ff_vk_find_struct(s->hwfc->create_pnext, + VK_STRUCTURE_TYPE_VIDEO_PROFILE_LIST_INFO_KHR); + if (!profile_list) { + av_log(avctx, AV_LOG_ERROR, "Profile list missing from frames context!"); + return AVERROR(EINVAL); + } + err = ff_vk_load_props(s); if (err < 0) goto fail; @@ -1096,13 +1108,13 @@ int ff_vk_decode_init(AVCodecContext *avctx) session_create.flags = 0x0; session_create.queueFamilyIndex = s->hwctx->queue_family_decode_index; - session_create.maxCodedExtent = prof->caps.maxCodedExtent; - session_create.maxDpbSlots = prof->caps.maxDpbSlots; - session_create.maxActiveReferencePictures = prof->caps.maxActiveReferencePictures; + session_create.maxCodedExtent = ctx->caps.maxCodedExtent; + session_create.maxDpbSlots = ctx->caps.maxDpbSlots; + session_create.maxActiveReferencePictures = ctx->caps.maxActiveReferencePictures; session_create.pictureFormat = s->hwfc->format[0]; session_create.referencePictureFormat = session_create.pictureFormat; session_create.pStdHeaderVersion = dec_ext[avctx->codec_id]; - session_create.pVideoProfile = &prof->profile_list.pProfiles[0]; + session_create.pVideoProfile = &profile_list->pProfiles[0]; /* Create decode exec context. * 2 async contexts per thread was experimentally determined to be optimal @@ -1147,14 +1159,14 @@ int ff_vk_decode_init(AVCodecContext *avctx) dpb_frames->height = s->frames->height; dpb_hwfc = dpb_frames->hwctx; - dpb_hwfc->create_pnext = (void *)&prof->profile_list; + dpb_hwfc->create_pnext = (void *)profile_list; dpb_hwfc->format[0] = s->hwfc->format[0]; dpb_hwfc->tiling = VK_IMAGE_TILING_OPTIMAL; dpb_hwfc->usage = VK_IMAGE_USAGE_VIDEO_DECODE_DPB_BIT_KHR | VK_IMAGE_USAGE_SAMPLED_BIT; /* Shuts validator up. */ if (dec->layered_dpb) - dpb_hwfc->nb_layers = prof->caps.maxDpbSlots; + dpb_hwfc->nb_layers = ctx->caps.maxDpbSlots; err = av_hwframe_ctx_init(ctx->dpb_hwfc_ref); if (err < 0) diff --git a/libavcodec/vulkan_decode.h b/libavcodec/vulkan_decode.h index 681d2476cd..3ac103f6b4 100644 --- a/libavcodec/vulkan_decode.h +++ b/libavcodec/vulkan_decode.h @@ -26,8 +26,6 @@ #include "vulkan_video.h" typedef struct FFVulkanDecodeProfileData { - VkVideoCapabilitiesKHR caps; - VkVideoDecodeCapabilitiesKHR dec_caps; VkVideoDecodeH264ProfileInfoKHR h264_profile; VkVideoDecodeH264ProfileInfoKHR h265_profile; VkVideoDecodeAV1ProfileInfoMESA av1_profile; @@ -40,7 +38,9 @@ typedef struct FFVulkanDecodeShared { FFVulkanContext s; FFVkVideoCommon common; FFVkExecPool exec_pool; - FFVulkanDecodeProfileData profile_data; + + VkVideoCapabilitiesKHR caps; + VkVideoDecodeCapabilitiesKHR dec_caps; AVBufferRef *dpb_hwfc_ref; /* Only used for dedicated_dpb */ -- 2.40.1