From patchwork Wed Sep 6 06:00:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 43579 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4f16:b0:149:dfde:5c0a with SMTP id gi22csp2310594pzb; Tue, 5 Sep 2023 23:01:36 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHcHCVMGOHIjkE+joaTId2rBIS/XuPruE5N8K4tXgUwVOeTMo/oqvqSmfGazEil8gNzgVVh X-Received: by 2002:a17:907:2c57:b0:99b:c830:cf23 with SMTP id hf23-20020a1709072c5700b0099bc830cf23mr1419150ejc.27.1693980096437; Tue, 05 Sep 2023 23:01:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1693980096; cv=none; d=google.com; s=arc-20160816; b=rakjUS3toRRR+NevehrCTtN//WiY4HdIKi5kXz9dnj8NcrxaqtJu+nyUKZPFgDo0yW H4I/5/SYC2uHAmHDZ2vzY0796j14TjDHNzLlRqAJgI6BNq9SeEi36aK8gDHIP1BFwUyT E/XlhR7Oy+k++6Ob22osUIQnKE/RXsp3eCAgaGd3CQyNLJ8aB5W5Z4Noz1irdZjigyXp f2yVsGyPKV4NEDuZTbZ2YPfByAcvQaDludqGmh9kjqe0Wi2qWvFOwKdSl1RKZf+uAi4v rix9VqBciq0ubl4dfWZ1Shxf67KzSRPYpFP7pXISuIQ8dtajYvDNepTAZkIDIyybG/eK KWHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=EK+dfeZPs5jUX6s+4l+xztxe9K4+FeQgMgCJ1Dnj8mY=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=Cn0ktKyQJvCJilK0o28a2wW92E1bMcB3kce9PnBkGhrH1azdKxRSAzvDoZktZSCTHp l/7wUuu9k4/InhloGTZ5wAroOZK0Q8BQq1aPV3MXSf6vRAZSPLSZFlCQOsao0VcR972u U2ElL9HbmNQZ8hrF6GuGPt/otd4d+Dp0qfHBXf8bqlkBBSRRIo2UfvSocqy5Z2it8Yab 93SqLZCKBj0J9rxIt9ahhf3AHOlsGfoc1ORhitzTLDMteF3fJ8OP/1SVKy7lKItpf1Qr Xk5/W99xFJTIbFI7uXjn4PsuP54dl8mxp5O+IXioGNBVM9sajmoTMeTFH43g+wWnwKzL j83A== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=JlNmdVRN; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h22-20020a17090634d600b0099bcb54fae7si8903968ejb.512.2023.09.05.23.01.35; Tue, 05 Sep 2023 23:01:36 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=JlNmdVRN; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7EF4E68C79E; Wed, 6 Sep 2023 09:01:23 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.93]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0BE2E6800AC for ; Wed, 6 Sep 2023 09:01:13 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1693980079; x=1725516079; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=BiXF0RkK/EG9rUhHRobRm+EX7dpPkr2xi5ddGM9aZO4=; b=JlNmdVRNmQs17KVP5aaG+3bUPGyjV9X4EREvVuZNSao5StFDgAxGXZtg rOSkRktURM57o7F2jPruMwCPmgeHmUAlJd5kjmtGwDG0+2I5YNl04/1jN +C/BuNqCm9hFlpCHFetYT00RABLTzI3ypv+xad1/PD1sJ1sOBGnvdEBd/ I4rp1HFhpxku8IDarIMMXmOiW3eVt6VqCi/eQovGu1RffWvoi1jUkwoot KeArumyIWeh2O+SrIkT0PJbZKsvuJ+QOVhhhAptXwRKN+cNDqkLN9APtw 4mYvYu0hp6JAP3AR1FrsCDW+aGnvC1cBIaDR2Z1QVntvG7Jve2OqQmEEq A==; X-IronPort-AV: E=McAfee;i="6600,9927,10824"; a="374372401" X-IronPort-AV: E=Sophos;i="6.02,231,1688454000"; d="scan'208";a="374372401" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Sep 2023 23:01:10 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10824"; a="734919011" X-IronPort-AV: E=Sophos;i="6.02,231,1688454000"; d="scan'208";a="734919011" Received: from xhh-dg264.sh.intel.com ([10.238.2.76]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Sep 2023 23:01:09 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 6 Sep 2023 14:00:41 +0800 Message-Id: <20230906060052.698620-2-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230906060052.698620-1-haihao.xiang@intel.com> References: <20230906060052.698620-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 02/13] lavu/hwcontext_qsv: create dynamic frame pool if required X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: /7PJ9WKFOUH2 From: Haihao Xiang When AVHWFramesContext.initial_pool_size is 0, a dynamic frame pool is required. We may support this under certain conditions, e.g. oneVPL 2.9+ support dynamic frame allocation, we needn't provide a fixed frame pool in the mfxFrameAllocator.Alloc callback. Signed-off-by: Haihao Xiang --- libavutil/hwcontext_qsv.c | 157 +++++++++++++++++++++++++++++++++++++- 1 file changed, 154 insertions(+), 3 deletions(-) diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index b76a89d13a..21ce4225bd 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -104,8 +104,15 @@ typedef struct QSVFramesContext { #endif AVFrame realigned_upload_frame; AVFrame realigned_download_frame; + + mfxFrameInfo frame_info; } QSVFramesContext; +typedef struct QSVSurface { + mfxFrameSurface1 mfx_surface; + AVFrame *child_frame; +} QSVSurface; + static const struct { enum AVPixelFormat pix_fmt; uint32_t fourcc; @@ -151,6 +158,8 @@ extern int ff_qsv_get_surface_base_handle(mfxFrameSurface1 *surf, enum AVHWDeviceType base_dev_type, void **base_handle); +static int qsv_init_surface(AVHWFramesContext *ctx, mfxFrameSurface1 *surf); + /** * Caller needs to allocate enough space for base_handle pointer. **/ @@ -359,7 +368,32 @@ static void qsv_pool_release_dummy(void *opaque, uint8_t *data) { } -static AVBufferRef *qsv_pool_alloc(void *opaque, size_t size) +static void qsv_pool_release(void *opaque, uint8_t *data) +{ + AVHWFramesContext *ctx = (AVHWFramesContext*)opaque; + QSVFramesContext *s = ctx->internal->priv; + QSVSurface *qsv_surface = (QSVSurface *)data; + mfxHDLPair *hdl_pair = (mfxHDLPair *)qsv_surface->mfx_surface.Data.MemId; + AVHWFramesContext *child_frames_ctx; + + if (!s->child_frames_ref) + return; + + child_frames_ctx = (AVHWFramesContext*)s->child_frames_ref->data; + if (!child_frames_ctx->device_ctx) + return; + +#if CONFIG_VAAPI + if (child_frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_VAAPI) + av_freep(&hdl_pair->first); +#endif + + av_freep(&hdl_pair); + av_frame_free(&qsv_surface->child_frame); + av_freep(&qsv_surface); +} + +static AVBufferRef *qsv_fixed_pool_alloc(void *opaque, size_t size) { AVHWFramesContext *ctx = (AVHWFramesContext*)opaque; QSVFramesContext *s = ctx->internal->priv; @@ -374,6 +408,104 @@ static AVBufferRef *qsv_pool_alloc(void *opaque, size_t size) return NULL; } +static AVBufferRef *qsv_dynamic_pool_alloc(void *opaque, size_t size) +{ + AVHWFramesContext *ctx = (AVHWFramesContext*)opaque; + QSVFramesContext *s = ctx->internal->priv; + AVHWFramesContext *child_frames_ctx; + QSVSurface *qsv_surface = NULL; + mfxHDLPair *handle_pairs_internal = NULL; + int ret; + + if (!s->child_frames_ref) + goto fail; + + child_frames_ctx = (AVHWFramesContext*)s->child_frames_ref->data; + if (!child_frames_ctx->device_ctx) + goto fail; + +#if CONFIG_DXVA2 + if (child_frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_DXVA2) { + av_log(ctx, AV_LOG_ERROR, + "QSV on dxva2 requires a fixed frame pool size\n"); + goto fail; + } +#endif + + qsv_surface = av_calloc(1, sizeof(*qsv_surface)); + if (!qsv_surface) + goto fail; + + qsv_surface->child_frame = av_frame_alloc(); + if (!qsv_surface->child_frame) + goto fail; + + ret = av_hwframe_get_buffer(s->child_frames_ref, qsv_surface->child_frame, 0); + if (ret < 0) + goto fail; + + handle_pairs_internal = av_calloc(1, sizeof(*handle_pairs_internal)); + if (!handle_pairs_internal) + goto fail; + + ret = qsv_init_surface(ctx, &qsv_surface->mfx_surface); + if (ret < 0) + goto fail; + +#if CONFIG_VAAPI + if (child_frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_VAAPI) { + VASurfaceID *surface_id_internal; + + surface_id_internal = av_calloc(1, sizeof(*surface_id_internal)); + if (!surface_id_internal) + return NULL; + + *surface_id_internal = (VASurfaceID)(uintptr_t)qsv_surface->child_frame->data[3]; + handle_pairs_internal->first = (mfxHDL)surface_id_internal; + handle_pairs_internal->second = (mfxMemId)MFX_INFINITE; + } +#endif + +#if CONFIG_D3D11VA + if (child_frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) { + AVD3D11VAFramesContext *child_frames_hwctx = child_frames_ctx->hwctx; + handle_pairs_internal->first = (mfxMemId)qsv_surface->child_frame->data[0]; + + if (child_frames_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) + handle_pairs_internal->second = (mfxMemId)MFX_INFINITE; + else + handle_pairs_internal->second = (mfxMemId)qsv_surface->child_frame->data[1]; + + } +#endif + + qsv_surface->mfx_surface.Data.MemId = (mfxMemId)handle_pairs_internal; + return av_buffer_create((uint8_t *)qsv_surface, sizeof(*qsv_surface), + qsv_pool_release, ctx, 0); + +fail: + if (qsv_surface) { + av_frame_free(&qsv_surface->child_frame); + } + + av_freep(&qsv_surface); + av_freep(&handle_pairs_internal); + + return NULL; +} + +static AVBufferRef *qsv_pool_alloc(void *opaque, size_t size) +{ + AVHWFramesContext *ctx = (AVHWFramesContext*)opaque; + AVQSVFramesContext *hwctx = ctx->hwctx; + + if (hwctx->nb_surfaces == 0) { + return qsv_dynamic_pool_alloc(opaque, size); + } else { + return qsv_fixed_pool_alloc(opaque, size); + } +} + static int qsv_init_child_ctx(AVHWFramesContext *ctx) { AVQSVFramesContext *hwctx = ctx->hwctx; @@ -562,9 +694,28 @@ static int qsv_init_pool(AVHWFramesContext *ctx, uint32_t fourcc) int i, ret = 0; - if (ctx->initial_pool_size <= 0) { - av_log(ctx, AV_LOG_ERROR, "QSV requires a fixed frame pool size\n"); + if (ctx->initial_pool_size < 0) { + av_log(ctx, AV_LOG_ERROR, "Invalid frame pool size\n"); return AVERROR(EINVAL); + } else if (ctx->initial_pool_size == 0) { + mfxFrameSurface1 mfx_surf1; + + ret = qsv_init_child_ctx(ctx); + if (ret < 0) + return ret; + + ctx->internal->pool_internal = av_buffer_pool_init2(sizeof(mfxFrameSurface1), + ctx, qsv_pool_alloc, NULL); + if (!ctx->internal->pool_internal) + return AVERROR(ENOMEM); + + memset(&mfx_surf1, 0, sizeof(mfx_surf1)); + qsv_init_surface(ctx, &mfx_surf1); + s->frame_info = mfx_surf1.Info; + frames_hwctx->info = &s->frame_info; + frames_hwctx->nb_surfaces = 0; + + return 0; } s->handle_pairs_internal = av_calloc(ctx->initial_pool_size,