From patchwork Sun Apr 28 07:39:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 48330 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:1509:b0:1a9:af23:56c1 with SMTP id nq9csp1338199pzb; Sun, 28 Apr 2024 00:39:38 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXfbhR8HrtSjmmTFr7FKpBcBypUknG8sQx2MoOZZ/8szrXGKD6sRhn17fjYq5pJMH7ZtYg1IvJOwyX0DX2wqVino2qpPelZJ/DHcQ== X-Google-Smtp-Source: AGHT+IGVrVYGhnspby3bf/wZvHGt4WkJtrHZvaIRzkmxGdfWRoWrAddSxOYQWFJkfwnByLg1TJN4 X-Received: by 2002:a50:d54b:0:b0:568:d55c:1bb3 with SMTP id f11-20020a50d54b000000b00568d55c1bb3mr5956541edj.31.1714289978191; Sun, 28 Apr 2024 00:39:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714289978; cv=none; d=google.com; s=arc-20160816; b=dJWm/FZFeQ9rAqFjBNFFIcFNtSecFA6+28pxAlSRHVykahp/uOotub53PB8s6r5Qj8 QsaQX7XZmssi/+/raPCmj6zAzEB+xoPAjAOaVZz/9JXlSyFYc8IksLhjUuhF4pj7IZpQ xfnub4/siV78IiMlnUfSRGsr6Rj/ePwgM4iZ+lkBE14CUWvgdBAVknT9TYRD4EzSZi+7 dgFidi7nhQr0/eY1QTkMDv32FTznRPZR4mtsd4yjBJPRn7/bRM9FZlAFz7zeOolII0mf qeC5jvdvkOscYZwlF/3K9sNjVjT9UuL3mHYqPbS4e7t5E9Hl66dDqne4ByIXz42IPDDy +cTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=y4UbbIS0qHg9XBnTO+qRnme4jUT1pN3lgboueeuOnD0=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=futhprlX3VQw6AAtDzKcag/VEXfidbjETM+9aAsJoBL8nPth1uvLspgAm/+jr+AQeG pANjvNeFO3OXPgZDxaG1vRZp+nHgRmlNtNPX9cUDAGFUqjgr1Qh/iVy1xZV5H3viwQ8d DAaMViIXiDX6zgibamkgDtSmUCPWiAMwFKNeFekLc55gg/kT+9QvbvqodvORbXwRltDR haJVvaj5368ZXJ7U7zP/kazw2k+9CqXgBLAYaLqb2xxboJr67e62Yc9z1dp6e9C9GcWv iQUEcDIn+SOCBkRu6aM7ZNs3Dp3ouCBG3z+IqyGM3GMFojKFfU7ZpbPwfUixPLCu0lZV ZZdg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Ssg6tRFC; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v10-20020aa7d80a000000b0056bb595c538si12505491edq.462.2024.04.28.00.39.37; Sun, 28 Apr 2024 00:39:38 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Ssg6tRFC; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8E19568D377; Sun, 28 Apr 2024 10:39:34 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0834568CAFB for ; Sun, 28 Apr 2024 10:39:26 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1714289972; x=1745825972; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=gWtcdV8wg8IHBSL9viRb6RE/sKmwDL3vZcLmDXwef4Q=; b=Ssg6tRFCAExroVvB8XtjUIO0ixy0IsWTr96Q5N99IAPhMkZ5LzN8q+ST yyU9nb7SbKTiMdXgbZOcDfIz6hzs/AbmXu/Z3NW8mIbtdktKM+QaqLrFM s77alnjlG73iakaV/Z0GAIpRvcPvBOxNoHZ4YQ0mtFe3iz+BQ54fzEgLd 0I4J4N0Y+j25MCIvH4p93c25kkzjpRXgaxofe6yy8+EFULnSgxMFtjFfS SV/Cwr/u9wlGbcqHvQnWi94FTK6M41grvBlaXqQrpVmqAwaHqPvypj7Pr ubV/YMEc6suFU/XC8a4Tng96y5gD9xVGRQLhyfWzgVttPLihvFu7ejVXu w==; X-CSE-ConnectionGUID: ishrYgtyRYO4c8clIYIYqw== X-CSE-MsgGUID: Pagt1Xm6Tbaimz2fJav9tQ== X-IronPort-AV: E=McAfee;i="6600,9927,11057"; a="20662189" X-IronPort-AV: E=Sophos;i="6.07,237,1708416000"; d="scan'208";a="20662189" Received: from orviesa003.jf.intel.com ([10.64.159.143]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2024 00:39:24 -0700 X-CSE-ConnectionGUID: wX4+Xgg2RDqH5R1XJNpyQQ== X-CSE-MsgGUID: mRcGtlBvSBy2Wd163flu3Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,237,1708416000"; d="scan'208";a="30491620" Received: from unknown (HELO xhh-dg264.sh.intel.com) ([10.238.2.76]) by ORVIESA003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2024 00:39:23 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Sun, 28 Apr 2024 15:39:03 +0800 Message-Id: <20240428073911.423287-1-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/9] lavu/hwcontext_qsv: update AVQSVFramesContext to support dynamic frame pool X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: uq0qzauu8R8O From: Haihao Xiang Add AVQSVFramesContext.info and update the description. Signed-off-by: Haihao Xiang --- doc/APIchanges | 3 +++ libavutil/hwcontext_qsv.c | 4 ++-- libavutil/hwcontext_qsv.h | 28 +++++++++++++++++++++++++--- libavutil/version.h | 4 ++-- 4 files changed, 32 insertions(+), 7 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index 0566fcdcc5..4a434b2877 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -2,6 +2,9 @@ The last version increases of all libraries were on 2024-03-07 API changes, most recent first: +2024-04-xx - xxxxxxxxxx - lavu 59.17.100 - hwcontext_qsv.h + Add AVQSVFramesContext.info + 2024-04-24 - 8616cfe0890 - lavu 59.16.100 - opt.h Add AV_OPT_SERIALIZE_SEARCH_CHILDREN. diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index c7c7878644..f552811346 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -627,7 +627,7 @@ static mfxStatus frame_alloc(mfxHDL pthis, mfxFrameAllocRequest *req, QSVFramesContext *s = ctx->hwctx; AVQSVFramesContext *hwctx = &s->p; mfxFrameInfo *i = &req->Info; - mfxFrameInfo *i1 = &hwctx->surfaces[0].Info; + mfxFrameInfo *i1 = hwctx->nb_surfaces ? &hwctx->surfaces[0].Info : hwctx->info; if (!(req->Type & MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET) || !(req->Type & (MFX_MEMTYPE_FROM_VPPIN | MFX_MEMTYPE_FROM_VPPOUT)) || @@ -1207,7 +1207,7 @@ static int qsv_init_internal_session(AVHWFramesContext *ctx, MFX_IOPATTERN_OUT_SYSTEM_MEMORY; par.AsyncDepth = 1; - par.vpp.In = frames_hwctx->surfaces[0].Info; + par.vpp.In = frames_hwctx->nb_surfaces ? frames_hwctx->surfaces[0].Info : *frames_hwctx->info; /* Apparently VPP requires the frame rate to be set to some value, otherwise * init will fail (probably for the framerate conversion filter). Since we diff --git a/libavutil/hwcontext_qsv.h b/libavutil/hwcontext_qsv.h index e2dba8ad83..9e43a237d4 100644 --- a/libavutil/hwcontext_qsv.h +++ b/libavutil/hwcontext_qsv.h @@ -25,8 +25,8 @@ * @file * An API-specific header for AV_HWDEVICE_TYPE_QSV. * - * This API does not support dynamic frame pools. AVHWFramesContext.pool must - * contain AVBufferRefs whose data pointer points to an mfxFrameSurface1 struct. + * AVHWFramesContext.pool must contain AVBufferRefs whose data pointer points + * to a mfxFrameSurface1 struct. */ /** @@ -51,7 +51,29 @@ typedef struct AVQSVDeviceContext { * This struct is allocated as AVHWFramesContext.hwctx */ typedef struct AVQSVFramesContext { - mfxFrameSurface1 *surfaces; + /** + * A pointer to a mfxFrameSurface1 or mfxFrameInfo struct + * + * When nb_surfaces is non-zero, it is a pointer to a mfxFrameSurface1 + * struct. + * + * When nb_surfaces is 0, it is a pointer to a mfxFrameInfo struct, all + * buffers allocated from the pool have the same mfxFrameInfo. + */ + union { + mfxFrameSurface1 *surfaces; + mfxFrameInfo *info; + }; + + /** + * Number of frames in the pool + * + * It is 0 for dynamic frame pools or AVHWFramesContext.initial_pool_size + * for fixed frame pools. + * + * Note only oneVPL GPU runtime 2.9+ can support dynamic frame pools + * on d3d11va or vaapi + */ int nb_surfaces; /** diff --git a/libavutil/version.h b/libavutil/version.h index 735f6832e3..3b5a2e7aaa 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,8 +79,8 @@ */ #define LIBAVUTIL_VERSION_MAJOR 59 -#define LIBAVUTIL_VERSION_MINOR 16 -#define LIBAVUTIL_VERSION_MICRO 101 +#define LIBAVUTIL_VERSION_MINOR 17 +#define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ LIBAVUTIL_VERSION_MINOR, \ From patchwork Sun Apr 28 07:39:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 48331 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:1509:b0:1a9:af23:56c1 with SMTP id nq9csp1338245pzb; Sun, 28 Apr 2024 00:39:47 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWPdntdJk63edehM7ja/XJL8qClh2nSqW7YkC0zbnIe2qv9+26OFxMfajagsh4OrMQ6Q4KAlIsDtXFZnREdrhDvcuRym78B5zk+jQ== X-Google-Smtp-Source: AGHT+IGEFF/rfrwB1QOLrrdxr2vzlWiguadMffyx1t0lk7RVQlidvbvuUmk/vPFMKiFCk+7YlBr1 X-Received: by 2002:a50:9997:0:b0:56d:ed86:c6f4 with SMTP id m23-20020a509997000000b0056ded86c6f4mr4635406edb.11.1714289987022; Sun, 28 Apr 2024 00:39:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714289987; cv=none; d=google.com; s=arc-20160816; b=ibBIQoYdGlkHr/osIicWkZ6kl8+F0p1a0sYKT60KZl+bAC5cp5lzdMYMTtdbfn1/vx rlGFQ1lzNExPKlzp1oMUhZpVJDj2iaaxdJWM9MqlAV4AHkHGTH87yRmRO8TD67g+3lzV xejhQiu3Ov+dDItTRco0lEFdVTT6FHgvsP9z7tYNMaJI96p1dPTczmdhUXTxpNbDCwt6 H4ud4QLUuqVv/AJMIqomxE/pbjFu82PI6dVxLKTxZQhCQc4fqkJARx4AliUhgSU+6IF8 XhLEgkuYoX/YhcY7l6bL32rnUTtejDPGeFlpy5w6IJ5VrgffsHvbpvBLnsOjmOD//XUT Gr6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=bSx8rAh2Dm/VfI1QVVYTUM9+FNKbsHyjDW72iFcFh6k=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=SzdnBKl89DmSdHc+8Sxu94nGknOgoApOFVRAS7vOiFaK+dcUtNKfCTTJ+grLNJ8h5c 0XycLYl29a0+oqDwklDe+bFOXRtOtLcuKTQrJQWntE6kdxsX5WiButZCPXUQnxoglr/A Yf3mGloFr57WwtlIzJB/d2E0Egr8fG8BBMQkWDZU8qaT7JW9VFu9++YQpiyGaIRYDWWW rq1RR1LpsO4ohQZNzzA4d3VRsSuF+PE62MyvsmHIkvWjfl2AwwjH9RszcOQRGh0Uo0et DStpF1HNDCsI+Rs7INwaQIr1oKDbn0tQHuYyHptuhkeQsgw9ENHku4zb0AkhKU9mHSOJ RBLw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=n1FfKP7m; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d14-20020a50cd4e000000b00570be756e15si13245208edj.118.2024.04.28.00.39.46; Sun, 28 Apr 2024 00:39:47 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=n1FfKP7m; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A9E2268D380; Sun, 28 Apr 2024 10:39:36 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A922268D377 for ; Sun, 28 Apr 2024 10:39:28 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1714289973; x=1745825973; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7zkqFIVv7WBfpJtltwqfzQuciGanvsxhBvRUfsY/F5w=; b=n1FfKP7mqZ9Mll1Uv0C7TooDxUw1oo7VMdMVk2W5lfYT+T/VT8PfR5sC 1ZRw+9+9t2cxJYuhjdd8OCrBrCePJsaVeBHHf+jKy0i1wzIKP7Ow+rLZU cUmd+gJCAGG8yqDYkjhJE8bCW880B0Wr8wxOresISOxF2yJ80YfH+QTCu m81NDHWGtb3g4xpeXd1fgoJOgBMiSYOda3lo2a8zCXLXKDc5fTsD3pgX4 FdxCUqL0uU8eaAj9N3P6252VfHx+2OkIV27SqVgpZYFhqcEeOVr9atJGf /A+mksU6QyBDtZ1g892X4szye5eq0Z1Ko8uqDKKbMAVyVDp6l5HcR0Q9B w==; X-CSE-ConnectionGUID: 0Ggys4c2R5GsLsI1dxd0Aw== X-CSE-MsgGUID: LbQDVBlUT/uwkR8J/e1rxw== X-IronPort-AV: E=McAfee;i="6600,9927,11057"; a="20662190" X-IronPort-AV: E=Sophos;i="6.07,237,1708416000"; d="scan'208";a="20662190" Received: from orviesa003.jf.intel.com ([10.64.159.143]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2024 00:39:25 -0700 X-CSE-ConnectionGUID: 1/oVsYv9QRaB3FPkSiXc9g== X-CSE-MsgGUID: Fc11ZghxQlStwhIAshNUhg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,237,1708416000"; d="scan'208";a="30491623" Received: from unknown (HELO xhh-dg264.sh.intel.com) ([10.238.2.76]) by ORVIESA003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2024 00:39:24 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Sun, 28 Apr 2024 15:39:04 +0800 Message-Id: <20240428073911.423287-2-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240428073911.423287-1-haihao.xiang@intel.com> References: <20240428073911.423287-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/9] lavu/hwcontext_qsv: create dynamic frame pool if required X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: E9dolaUIfQUB From: Haihao Xiang When AVHWFramesContext.initial_pool_size is 0, a dynamic frame pool is required. We may support this under certain conditions, e.g. oneVPL 2.9+ support dynamic frame allocation, we needn't provide a fixed frame pool in the mfxFrameAllocator.Alloc callback. Signed-off-by: Haihao Xiang --- libavutil/hwcontext_qsv.c | 157 +++++++++++++++++++++++++++++++++++++- 1 file changed, 154 insertions(+), 3 deletions(-) diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index f552811346..01bd273a08 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -118,8 +118,15 @@ typedef struct QSVFramesContext { #endif AVFrame realigned_upload_frame; AVFrame realigned_download_frame; + + mfxFrameInfo frame_info; } QSVFramesContext; +typedef struct QSVSurface { + mfxFrameSurface1 mfx_surface; + AVFrame *child_frame; +} QSVSurface; + static const struct { enum AVPixelFormat pix_fmt; uint32_t fourcc; @@ -165,6 +172,8 @@ extern int ff_qsv_get_surface_base_handle(mfxFrameSurface1 *surf, enum AVHWDeviceType base_dev_type, void **base_handle); +static int qsv_init_surface(AVHWFramesContext *ctx, mfxFrameSurface1 *surf); + /** * Caller needs to allocate enough space for base_handle pointer. **/ @@ -373,7 +382,32 @@ static void qsv_pool_release_dummy(void *opaque, uint8_t *data) { } -static AVBufferRef *qsv_pool_alloc(void *opaque, size_t size) +static void qsv_pool_release(void *opaque, uint8_t *data) +{ + AVHWFramesContext *ctx = (AVHWFramesContext*)opaque; + QSVFramesContext *s = ctx->hwctx; + QSVSurface *qsv_surface = (QSVSurface *)data; + mfxHDLPair *hdl_pair = (mfxHDLPair *)qsv_surface->mfx_surface.Data.MemId; + AVHWFramesContext *child_frames_ctx; + + if (!s->child_frames_ref) + return; + + child_frames_ctx = (AVHWFramesContext*)s->child_frames_ref->data; + if (!child_frames_ctx->device_ctx) + return; + +#if CONFIG_VAAPI + if (child_frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_VAAPI) + av_freep(&hdl_pair->first); +#endif + + av_freep(&hdl_pair); + av_frame_free(&qsv_surface->child_frame); + av_freep(&qsv_surface); +} + +static AVBufferRef *qsv_fixed_pool_alloc(void *opaque, size_t size) { AVHWFramesContext *ctx = (AVHWFramesContext*)opaque; QSVFramesContext *s = ctx->hwctx; @@ -388,6 +422,104 @@ static AVBufferRef *qsv_pool_alloc(void *opaque, size_t size) return NULL; } +static AVBufferRef *qsv_dynamic_pool_alloc(void *opaque, size_t size) +{ + AVHWFramesContext *ctx = (AVHWFramesContext*)opaque; + QSVFramesContext *s = ctx->hwctx; + AVHWFramesContext *child_frames_ctx; + QSVSurface *qsv_surface = NULL; + mfxHDLPair *handle_pairs_internal = NULL; + int ret; + + if (!s->child_frames_ref) + goto fail; + + child_frames_ctx = (AVHWFramesContext*)s->child_frames_ref->data; + if (!child_frames_ctx->device_ctx) + goto fail; + +#if CONFIG_DXVA2 + if (child_frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_DXVA2) { + av_log(ctx, AV_LOG_ERROR, + "QSV on dxva2 requires a fixed frame pool size\n"); + goto fail; + } +#endif + + qsv_surface = av_calloc(1, sizeof(*qsv_surface)); + if (!qsv_surface) + goto fail; + + qsv_surface->child_frame = av_frame_alloc(); + if (!qsv_surface->child_frame) + goto fail; + + ret = av_hwframe_get_buffer(s->child_frames_ref, qsv_surface->child_frame, 0); + if (ret < 0) + goto fail; + + handle_pairs_internal = av_calloc(1, sizeof(*handle_pairs_internal)); + if (!handle_pairs_internal) + goto fail; + + ret = qsv_init_surface(ctx, &qsv_surface->mfx_surface); + if (ret < 0) + goto fail; + +#if CONFIG_VAAPI + if (child_frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_VAAPI) { + VASurfaceID *surface_id_internal; + + surface_id_internal = av_calloc(1, sizeof(*surface_id_internal)); + if (!surface_id_internal) + goto fail; + + *surface_id_internal = (VASurfaceID)(uintptr_t)qsv_surface->child_frame->data[3]; + handle_pairs_internal->first = (mfxHDL)surface_id_internal; + handle_pairs_internal->second = (mfxMemId)MFX_INFINITE; + } +#endif + +#if CONFIG_D3D11VA + if (child_frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) { + AVD3D11VAFramesContext *child_frames_hwctx = child_frames_ctx->hwctx; + handle_pairs_internal->first = (mfxMemId)qsv_surface->child_frame->data[0]; + + if (child_frames_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) + handle_pairs_internal->second = (mfxMemId)MFX_INFINITE; + else + handle_pairs_internal->second = (mfxMemId)qsv_surface->child_frame->data[1]; + + } +#endif + + qsv_surface->mfx_surface.Data.MemId = (mfxMemId)handle_pairs_internal; + return av_buffer_create((uint8_t *)qsv_surface, sizeof(*qsv_surface), + qsv_pool_release, ctx, 0); + +fail: + if (qsv_surface) { + av_frame_free(&qsv_surface->child_frame); + } + + av_freep(&qsv_surface); + av_freep(&handle_pairs_internal); + + return NULL; +} + +static AVBufferRef *qsv_pool_alloc(void *opaque, size_t size) +{ + AVHWFramesContext *ctx = (AVHWFramesContext*)opaque; + AVQSVFramesContext *hwctx = ctx->hwctx; + + if (hwctx->nb_surfaces == 0) { + return qsv_dynamic_pool_alloc(opaque, size); + } else { + return qsv_fixed_pool_alloc(opaque, size); + } +} + static int qsv_init_child_ctx(AVHWFramesContext *ctx) { QSVDeviceContext *device_priv = ctx->device_ctx->hwctx; @@ -576,9 +708,28 @@ static int qsv_init_pool(AVHWFramesContext *ctx, uint32_t fourcc) int i, ret = 0; - if (ctx->initial_pool_size <= 0) { - av_log(ctx, AV_LOG_ERROR, "QSV requires a fixed frame pool size\n"); + if (ctx->initial_pool_size < 0) { + av_log(ctx, AV_LOG_ERROR, "Invalid frame pool size\n"); return AVERROR(EINVAL); + } else if (ctx->initial_pool_size == 0) { + mfxFrameSurface1 mfx_surf1; + + ret = qsv_init_child_ctx(ctx); + if (ret < 0) + return ret; + + ffhwframesctx(ctx)->pool_internal = av_buffer_pool_init2(sizeof(mfxFrameSurface1), + ctx, qsv_pool_alloc, NULL); + if (!ffhwframesctx(ctx)->pool_internal) + return AVERROR(ENOMEM); + + memset(&mfx_surf1, 0, sizeof(mfx_surf1)); + qsv_init_surface(ctx, &mfx_surf1); + s->frame_info = mfx_surf1.Info; + frames_hwctx->info = &s->frame_info; + frames_hwctx->nb_surfaces = 0; + + return 0; } s->handle_pairs_internal = av_calloc(ctx->initial_pool_size, From patchwork Sun Apr 28 07:39:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 48332 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:1509:b0:1a9:af23:56c1 with SMTP id nq9csp1338264pzb; Sun, 28 Apr 2024 00:39:56 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWQe3xhrw1GPbmGsa4XhMYG2Pz981UMtbpvRZAc3rHb9TAajHLvE/PSz9fXSoLqlHCeIsSL13ZokBgx0LIxT9bHNQCFv1KenqOl9g== X-Google-Smtp-Source: AGHT+IHMjV7TCK2m1MHCHqmgnNvsqajWWIU5ahzHHt5tL4YCCnirzI7nGSmGeA2FlczyaO7OwpMv X-Received: by 2002:a05:6402:3609:b0:572:7dc7:4e36 with SMTP id el9-20020a056402360900b005727dc74e36mr446786edb.3.1714289995945; Sun, 28 Apr 2024 00:39:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714289995; cv=none; d=google.com; s=arc-20160816; b=zrGccSWSqiTki1vKIhfScWslvrcDHfz3L9qCXnUXmy0fLUWMne/ZhxC/MW57z6B+Nv 7hdzkzY0YKYhX/7++x1KmeI0x4Hg3O2mm2BxJOSb/1E5cwJiqCppnNCUxoBAYKCl1lev LZeEuJP049BDw7x+LzduSH8NBCPJv0e7E9EyHe7kNEABqoI3QZzoOgITCLEeRyW31lfS tNasvQv++6M1/Vh45PuOGbMKD5koSIOFyQaVXX+GGqDhImYTw/MHiAzABrd4ElpM1yJh 3Lz4pfxnPThdV6/P9ZKIQd0L72cVtFuZtD5fmixFPH2Y79peNsoBUf7qkMfFFMSiI85V SycQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=N2zT9Kyn/COFW6JAMEqMIhAnreELSs7uYC3Jj5pM3AU=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=avBQ6Fl1fQsOVe+Zg7W9xOI4Vaxno4gHYg6Ih7rfj1opTlPeMTXljCiXoLLIH7aiW5 O51a23Hsxf9ufytqWGimZcE9cDw6qGdknsMLk5WZnFx83QFY4YH0H2kxTRyUqS3WFTif Xhtr/E1azDa2jNZ/DKQtSZUnhOIvjH0UaRYzTYfOCtV5qF9OlC8rpo63TBpRU/r6xb0U hIV21Hf5EUZnMrbLWEc1n71rf7PWv5juzKlAetOrWlDokBb6u3yTeqfDlgRIaJwC+Ac4 awUmDyNssQazZIjRNtD1jXfNOSmJyEM3t1p/UxzcDLQig2hD52VAfMT3m7ZXu5S5zFL1 pO1g==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=bvZ2X+lM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g8-20020a50d5c8000000b00572798ce5a5si839449edj.209.2024.04.28.00.39.55; Sun, 28 Apr 2024 00:39:55 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=bvZ2X+lM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C896B68D3C9; Sun, 28 Apr 2024 10:39:37 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D144C68CB36 for ; Sun, 28 Apr 2024 10:39:29 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1714289975; x=1745825975; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Z4X7U6jAyvkSyqMXJ9ZiYTH49FSr7HfQ1qhJpTjyemg=; b=bvZ2X+lM1giNyM7cPIO12OQH2ZVGKp23iB1iqHVTPrxcQca7KaTmKZ4A TmWZhz2LBupA639kcEu4weqJGkSOntYbokitrMrMOIO8dW357nDgr1Bzc PFlvjDFaZ1RAROhvSwc91JWH7OKGcVVlzM9mRTnQqvNbLEyewiVLDYtaJ X/hq+YBMDctfvwYoplLwPxsbjBGRB8rXWy3dwXhsG+bpPMTZceX9D6vEo i025XMK41kkfiZieCX7QjZJn7ZdfH0F5zqKXe0dZF2VTf/keu0tmgCP/p RBl2/EKZXOIlPRo1HIGfo0BXcQuwL1crXJTfQIKPPlE9gAqQgE2PgBqd2 Q==; X-CSE-ConnectionGUID: 12cY77s1S/iFcz9BMKVzNg== X-CSE-MsgGUID: qvpDumr5RVOxk/3E1VRzGg== X-IronPort-AV: E=McAfee;i="6600,9927,11057"; a="20662191" X-IronPort-AV: E=Sophos;i="6.07,237,1708416000"; d="scan'208";a="20662191" Received: from orviesa003.jf.intel.com ([10.64.159.143]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2024 00:39:26 -0700 X-CSE-ConnectionGUID: Ckv3T3VbRFqG9FvzP0t/eQ== X-CSE-MsgGUID: bqdMCQwARKesOSa79TThQg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,237,1708416000"; d="scan'208";a="30491626" Received: from unknown (HELO xhh-dg264.sh.intel.com) ([10.238.2.76]) by ORVIESA003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2024 00:39:25 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Sun, 28 Apr 2024 15:39:05 +0800 Message-Id: <20240428073911.423287-3-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240428073911.423287-1-haihao.xiang@intel.com> References: <20240428073911.423287-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/9] lavu/hwcontext_qsv: add support for dynamic frame pool in qsv_frames_derive_to X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: WtHltUMJVxKx From: Haihao Xiang Make it work with the source which has a dynamic frame pool. Signed-off-by: Haihao Xiang --- libavutil/hwcontext_qsv.c | 61 ++++++++++++++++++++++++++++++++++----- 1 file changed, 54 insertions(+), 7 deletions(-) diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index 01bd273a08..1d3091e6f8 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -1972,18 +1972,52 @@ static int qsv_transfer_data_to(AVHWFramesContext *ctx, AVFrame *dst, return 0; } -static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, - AVHWFramesContext *src_ctx, int flags) +static int qsv_dynamic_frames_derive_to(AVHWFramesContext *dst_ctx, + AVHWFramesContext *src_ctx, int flags) { QSVFramesContext *s = dst_ctx->hwctx; AVQSVFramesContext *dst_hwctx = &s->p; - int i; + mfxFrameSurface1 mfx_surf1; - if (src_ctx->initial_pool_size == 0) { - av_log(dst_ctx, AV_LOG_ERROR, "Only fixed-size pools can be " - "mapped to QSV frames.\n"); - return AVERROR(EINVAL); + switch (src_ctx->device_ctx->type) { +#if CONFIG_VAAPI + case AV_HWDEVICE_TYPE_VAAPI: + dst_hwctx->frame_type = MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET; + break; +#endif + +#if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: + { + AVD3D11VAFramesContext *src_hwctx = src_ctx->hwctx; + + if (src_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { + dst_hwctx->frame_type |= MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET; + } else { + dst_hwctx->frame_type |= MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET; + } } + break; +#endif + + default: + return AVERROR(ENOSYS); + } + + memset(&mfx_surf1, 0, sizeof(mfx_surf1)); + qsv_init_surface(dst_ctx, &mfx_surf1); + s->frame_info = mfx_surf1.Info; + dst_hwctx->info = &s->frame_info; + dst_hwctx->nb_surfaces = 0; + return 0; +} + +static int qsv_fixed_frames_derive_to(AVHWFramesContext *dst_ctx, + AVHWFramesContext *src_ctx, int flags) +{ + QSVFramesContext *s = dst_ctx->hwctx; + AVQSVFramesContext *dst_hwctx = &s->p; + int i; switch (src_ctx->device_ctx->type) { #if CONFIG_VAAPI @@ -2075,6 +2109,19 @@ static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, return 0; } +static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, + AVHWFramesContext *src_ctx, int flags) +{ + if (src_ctx->initial_pool_size < 0) { + av_log(dst_ctx, AV_LOG_ERROR, "Invalid src frame pool. \n"); + return AVERROR(EINVAL); + } else if (src_ctx->initial_pool_size == 0) { + return qsv_dynamic_frames_derive_to(dst_ctx, src_ctx, flags); + } else { + return qsv_fixed_frames_derive_to(dst_ctx, src_ctx, flags); + } +} + static int qsv_map_to(AVHWFramesContext *dst_ctx, AVFrame *dst, const AVFrame *src, int flags) { From patchwork Sun Apr 28 07:39:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 48333 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:1509:b0:1a9:af23:56c1 with SMTP id nq9csp1338317pzb; Sun, 28 Apr 2024 00:40:04 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWNV04sl0mc4tbRAtsPuuzybIxrrEL9bUdJQ6JjPOaE0Vr/AOPYo+pjPKr+cEanzIT+tMI2xWHX38LIq5z9KaAt6s/iasUxBbhUuw== X-Google-Smtp-Source: AGHT+IEWaB61wxEEQdx1b1Getqow+KohrjRivTmR0biAfuVkh8GCxrPM7Rk+ojPD+a0P2rNDArbK X-Received: by 2002:a05:6512:702:b0:51b:af8f:f7c4 with SMTP id b2-20020a056512070200b0051baf8ff7c4mr5345580lfs.52.1714290004469; Sun, 28 Apr 2024 00:40:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714290004; cv=none; d=google.com; s=arc-20160816; b=W17jaXQo1YxMfp9f16mmI5GiJsNXX50Zu6GHz2Gg/0oBhbd1iNkSHj2tXTC6ZanMSl qOH2SdziiT5TIltRdaWEIRTjhlFnpUdIIWL1XIBUh7NYXhJkJk0cUnx5LnRjd1xmtLt9 R89DjDlfaZgBFptmI+SOhF2vqyHk9GmmtQ99UOjiZn2GaV1cp7T6c2IYoRYrByNGpl5I gvI92y0vKpjmKWhUU4RarCdVetHAxm3AI/f7ZjV0o+dOVDRZOyv2j/xzFWX/R8S04fgD 02yCxneNNUM6sTCZOg3qZHvJ7Zk6P8TsPvVJHodnBzovhdkxbnG+Es4k1l5qeGxwgdP/ PnJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=0U1fY/0ErSRnQsr+vJ+kSIKvj4gkaRP4LcStxZISXqU=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=s2BvHPDNNmY3AsREv1jdtEFeprmYEe9sLIwSaaDLtwqoq19eWbltouB7t6uwG65ddd 2O1dQFDQFjx9xAIfgpUso4JTFnT4OYnrIeDEOPc90qRoq5Qj7BW+OXF3+4BFfKFjqD+v Iozc8mvRCAX0b9yWo7VwAzTntHf0TJg9a1VDsLWOGi2vxgYYA3CNgGc+8/J6CT4XRvwM TXSo8F+oewjnhKhg/C9sivfm5v7/GVy3jYQp6NHk5Sn7WAI3x3j+FSMvmEXPL3aD21h+ /cq8DhqXM2zPTAdixYL+ZnOmql58EWTwdPVQyYKWhmU+fDuHGrqIUJHAUoO3Rmtkc9a2 nlgg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=a5e9yos2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id o17-20020a056512231100b0051d92eff121si196539lfu.246.2024.04.28.00.40.04; Sun, 28 Apr 2024 00:40:04 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=a5e9yos2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4AAB068D416; Sun, 28 Apr 2024 10:39:40 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E94EA68D3FE for ; Sun, 28 Apr 2024 10:39:32 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1714289978; x=1745825978; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JAbUPVhVvNnDS2VE18VPBZCQd42K1q9Tn48yU98U36Y=; b=a5e9yos2UlRTlK3EvE4LBvSHl8eAELknyUeydPEJVMNK744of3kiyjtl BpJG4yvKdJ3Cznw4PM9jvYv4pMYCqXxXFmSsz/NblqPKVD7XA/djmCBvF alyTabNwPmHUZlvkHdKmGQiBl3eLEhDnl8HfRBqzIVyjMXSuinss9+K/A Om9n4wkNT1fVG85uOBQ9/Db+COjppOucYlvNQKtjxSahc/W2BrYeSZKWh GWDOywy29hvR4DedkG7TDYEXk4uAZX+aqZidkfK26GckbZwgZGUSRwLMH YC7m63KHNQvgb11saEWZ7kiE87T6GNFKQ/rFu52UBfG+N9Duq7XivyjYG Q==; X-CSE-ConnectionGUID: LP14BMnrTamosuI7Gv3DVg== X-CSE-MsgGUID: I89qTj4ATIqgj3dG2pyEhw== X-IronPort-AV: E=McAfee;i="6600,9927,11057"; a="20662194" X-IronPort-AV: E=Sophos;i="6.07,237,1708416000"; d="scan'208";a="20662194" Received: from orviesa003.jf.intel.com ([10.64.159.143]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2024 00:39:28 -0700 X-CSE-ConnectionGUID: rjp3kockQqOxMwv0/S2arw== X-CSE-MsgGUID: 8fCOthx7R+yiM/Gf50CE6Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,237,1708416000"; d="scan'208";a="30491632" Received: from unknown (HELO xhh-dg264.sh.intel.com) ([10.238.2.76]) by ORVIESA003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2024 00:39:27 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Sun, 28 Apr 2024 15:39:06 +0800 Message-Id: <20240428073911.423287-4-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240428073911.423287-1-haihao.xiang@intel.com> References: <20240428073911.423287-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/9] lavu/hwcontext_qsv: add support for dynamic frame pool in qsv_map_to X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Iw9dEfkckeJy From: Haihao Xiang Make it work with the source which has a dynamic frame pool. Signed-off-by: Haihao Xiang --- libavutil/hwcontext_qsv.c | 131 +++++++++++++++++++++++++++++++++++++- 1 file changed, 129 insertions(+), 2 deletions(-) diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index 1d3091e6f8..8b7b4dc501 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -2122,8 +2122,8 @@ static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, } } -static int qsv_map_to(AVHWFramesContext *dst_ctx, - AVFrame *dst, const AVFrame *src, int flags) +static int qsv_fixed_pool_map_to(AVHWFramesContext *dst_ctx, + AVFrame *dst, const AVFrame *src, int flags) { AVQSVFramesContext *hwctx = dst_ctx->hwctx; int i, err, index = -1; @@ -2182,6 +2182,133 @@ static int qsv_map_to(AVHWFramesContext *dst_ctx, return 0; } +static void qsv_dynamic_pool_unmap(AVHWFramesContext *ctx, HWMapDescriptor *hwmap) +{ + mfxFrameSurface1 *surfaces_internal = (mfxFrameSurface1 *)hwmap->priv; + mfxHDLPair *handle_pairs_internal = (mfxHDLPair *)surfaces_internal->Data.MemId; + AVHWFramesContext *src_ctx = (AVHWFramesContext *)ffhwframesctx(ctx)->source_frames->data; + + switch (src_ctx->format) { +#if CONFIG_VAAPI + case AV_PIX_FMT_VAAPI: + { + av_freep(&handle_pairs_internal->first); + + break; + } +#endif + +#if CONFIG_D3D11VA + case AV_PIX_FMT_D3D11: + { + /* Do nothing */ + break; + } +#endif + default: + av_log(ctx, AV_LOG_ERROR, "Should not reach here. \n"); + break; + } + + av_freep(&handle_pairs_internal); + av_freep(&surfaces_internal); +} + +static int qsv_dynamic_pool_map_to(AVHWFramesContext *dst_ctx, + AVFrame *dst, const AVFrame *src, int flags) +{ + mfxFrameSurface1 *surfaces_internal = NULL; + mfxHDLPair *handle_pairs_internal = NULL; + int ret = 0; + + surfaces_internal = av_calloc(1, sizeof(*surfaces_internal)); + if (!surfaces_internal) { + ret = AVERROR(ENOMEM); + goto fail; + } + + handle_pairs_internal = av_calloc(1, sizeof(*handle_pairs_internal)); + if (!handle_pairs_internal) { + ret = AVERROR(ENOMEM); + goto fail; + } + + ret = qsv_init_surface(dst_ctx, surfaces_internal); + if (ret < 0) + goto fail; + + switch (src->format) { +#if CONFIG_VAAPI + case AV_PIX_FMT_VAAPI: + { + VASurfaceID *surface_id_internal; + + surface_id_internal = av_calloc(1, sizeof(*surface_id_internal)); + if (!surface_id_internal) { + ret =AVERROR(ENOMEM); + goto fail; + } + + *surface_id_internal = (VASurfaceID)(uintptr_t)src->data[3]; + handle_pairs_internal->first = (mfxHDL)surface_id_internal; + handle_pairs_internal->second = (mfxMemId)MFX_INFINITE; + + break; + } +#endif + +#if CONFIG_D3D11VA + case AV_PIX_FMT_D3D11: + { + AVHWFramesContext *src_ctx = (AVHWFramesContext*)src->hw_frames_ctx->data; + AVD3D11VAFramesContext *src_hwctx = src_ctx->hwctx; + + handle_pairs_internal->first = (mfxMemId)src->data[0]; + + if (src_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { + handle_pairs_internal->second = (mfxMemId)MFX_INFINITE; + } else { + handle_pairs_internal->second = (mfxMemId)src->data[1]; + } + + break; + } +#endif + default: + ret = AVERROR(ENOSYS); + goto fail; + } + + surfaces_internal->Data.MemId = (mfxMemId)handle_pairs_internal; + + ret = ff_hwframe_map_create(dst->hw_frames_ctx, + dst, src, qsv_dynamic_pool_unmap, surfaces_internal); + if (ret) + goto fail; + + dst->width = src->width; + dst->height = src->height; + dst->data[3] = (uint8_t*)surfaces_internal; + + return 0; + +fail: + av_freep(&handle_pairs_internal); + av_freep(&surfaces_internal); + return ret; +} + +static int qsv_map_to(AVHWFramesContext *dst_ctx, + AVFrame *dst, const AVFrame *src, int flags) +{ + AVQSVFramesContext *hwctx = dst_ctx->hwctx; + + if (hwctx->nb_surfaces) + return qsv_fixed_pool_map_to(dst_ctx, dst, src, flags); + else + return qsv_dynamic_pool_map_to(dst_ctx, dst, src, flags); +} + static int qsv_frames_get_constraints(AVHWDeviceContext *ctx, const void *hwconfig, AVHWFramesConstraints *constraints) From patchwork Sun Apr 28 07:39:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 48334 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:1509:b0:1a9:af23:56c1 with SMTP id nq9csp1338358pzb; Sun, 28 Apr 2024 00:40:12 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWHZY26Jaq9K9uxNl+CmlGE7oPgI9BYCAvF4Vqdmh+LQ4BWAWW0NAcUYHab4S0Wa3KlepcJQkUlEBkDTR/OYdThyfhsSbo9rplrFQ== X-Google-Smtp-Source: AGHT+IF1kVtMT2+zN77U6F8Q9ylFNunZcq2GtMc+YvTE2hQUAJCbAomyxrj9TitkmCRBjeFhNEST X-Received: by 2002:a50:d711:0:b0:572:1625:ae99 with SMTP id t17-20020a50d711000000b005721625ae99mr3049729edi.31.1714290012627; Sun, 28 Apr 2024 00:40:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714290012; cv=none; d=google.com; s=arc-20160816; b=d6jAkBKZ3TwyzuTP1Y65PQNb2scWVJAXV4l70ZjGW5z/kTCSR3Bk4jtzLHCZCPGZfn 5LV+9GP8ETsed5K7grHqUiSRYpd+iXQEIC7Sc0fbdDPt45vg++fFOecRhAnFryngz3wS Egcp6JIfxAl9zfOTsWKddKofylnfNAjtWewrkzXoB66Nxbp3I5dQPaR6EqPZUC9R9Qsi ZmKfNQdCiozzvmdCkN2iAVWJPCKFnPlWMixyoqzbwuF4bMfbnax5Ufc2i0RsGcfDuUAc Y+ActwMClvfh4GEEDUTeYypQH0RY+9HPvw1mTX8JgVlRA50IxlH0ofrfT4zpNDvrAfeW c5Kw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=e0l4zFT9NEzOXbJrPn2d7Yn4Q8phI07NXJhQ9UzHu/g=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=eR2uAAabB8g3FX2dqrFM/fy7LRZJCCi7suW3bPGo9Xi0F0TmOdulZwtqljM/1qhT+9 uMOOB3sLITXgXuLlrImV506FIBZDlkpdcC8RVHw7aA1AJjIFJV6hjEdx38Gy+FN4trQE tDUVu+JvdeP+lQNl1I15tG3ILCpWNsfQp31xIavQgMl4h0JrC9w7lPUERXnvUq41D2iH 9QcdTfjCebydDqEyStFiP8xe+HJ+5j8kFGbYKjefmsK0Q5pozgGsmBXneyehAaV/ngZW z9+68Bzt6/H98vtxTG1z+QkUjzKRGL89os/0VW15N6JykY6s76hOoo+X/4cY/voVvWvc E8+g==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=dWyYm7iu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f17-20020a056402005100b0056ffbff2694si12847356edu.515.2024.04.28.00.40.12; Sun, 28 Apr 2024 00:40:12 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=dWyYm7iu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BE03068D41A; Sun, 28 Apr 2024 10:39:44 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 962BE68D390 for ; Sun, 28 Apr 2024 10:39:34 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1714289979; x=1745825979; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=IJ2QGG8LcZ1a/eOeFESBQWjekb/c9M7ce/nkW3lvR+s=; b=dWyYm7iuwtCdYmuI6ZcnGYStJAAlyZSOkraVqg/nqN12lcqAWrqvn7XP zGUOwUZ8BDX+CvQbt2RZIhy3/jptPRjiqFoeAD1ptESP4hw8VJ/CWWKdV LnaVFL1CgBbG0yv3XllpAibwGQP1RdWgiyd8IFHtixYpFo4Q3qydHSpco qbqn5L9Opm+rswCYypbpQ/za/I8l1aUDpLDDzW/OQS4zswVmlUKiKF+QH oeo88idFWJq2DMoWBSry/Gm3SOzw2+b7hen1/7N2uqyzCzXsOIw1q+f06 kXka7I9duw58YX6Oz46VS/63t5lbSbsmG1BHzRaWmP63a3ryk/1T0bKwJ A==; X-CSE-ConnectionGUID: 5EBgTKBJSGCmojae0128YA== X-CSE-MsgGUID: WU26/+12REu1lguH5KQROQ== X-IronPort-AV: E=McAfee;i="6600,9927,11057"; a="20662195" X-IronPort-AV: E=Sophos;i="6.07,237,1708416000"; d="scan'208";a="20662195" Received: from orviesa003.jf.intel.com ([10.64.159.143]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2024 00:39:29 -0700 X-CSE-ConnectionGUID: sUao9iMcQoWR/I5lwd0ylQ== X-CSE-MsgGUID: fj/rvwZ4S2Wx0D/gI1z3YA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,237,1708416000"; d="scan'208";a="30491635" Received: from unknown (HELO xhh-dg264.sh.intel.com) ([10.238.2.76]) by ORVIESA003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2024 00:39:28 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Sun, 28 Apr 2024 15:39:07 +0800 Message-Id: <20240428073911.423287-5-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240428073911.423287-1-haihao.xiang@intel.com> References: <20240428073911.423287-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 5/9] lavc/qsv: fix the mfx allocator to support dynamic frame pool X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 8HJMrL+eoGFz From: Haihao Xiang When the external allocator is used for dynamic frame allocation, only video memory is supported, the SDK doesn't lock/unlock the memory block via Lock()/Unlock() calls. Signed-off-by: Haihao Xiang --- libavcodec/qsv.c | 68 +++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 56 insertions(+), 12 deletions(-) diff --git a/libavcodec/qsv.c b/libavcodec/qsv.c index b07302cdf6..6bbfe2a5a9 100644 --- a/libavcodec/qsv.c +++ b/libavcodec/qsv.c @@ -825,8 +825,16 @@ static mfxStatus qsv_frame_alloc(mfxHDL pthis, mfxFrameAllocRequest *req, AVHWFramesContext *frames_ctx = (AVHWFramesContext*)ctx->hw_frames_ctx->data; AVQSVFramesContext *frames_hwctx = frames_ctx->hwctx; mfxFrameInfo *i = &req->Info; - mfxFrameInfo *i1 = &frames_hwctx->surfaces[0].Info; + mfxFrameInfo *i1; + if (!frames_hwctx->nb_surfaces) { + av_log(ctx->logctx, AV_LOG_DEBUG, + "Dynamic frame pools, no frame is pre-allocated\n"); + + return MFX_ERR_NONE; + } + + i1 = &frames_hwctx->surfaces[0].Info; if (i->Width > i1->Width || i->Height > i1->Height || i->FourCC != i1->FourCC || i->ChromaFormat != i1->ChromaFormat) { av_log(ctx->logctx, AV_LOG_ERROR, "Mismatching surface properties in an " @@ -845,6 +853,7 @@ static mfxStatus qsv_frame_alloc(mfxHDL pthis, mfxFrameAllocRequest *req, } else if (req->Type & MFX_MEMTYPE_INTERNAL_FRAME) { /* internal frames -- allocate a new hw frames context */ AVHWFramesContext *ext_frames_ctx = (AVHWFramesContext*)ctx->hw_frames_ctx->data; + AVQSVFramesContext *ext_frames_hwctx = ext_frames_ctx->hwctx; mfxFrameInfo *i = &req->Info; AVBufferRef *frames_ref; @@ -852,6 +861,9 @@ static mfxStatus qsv_frame_alloc(mfxHDL pthis, mfxFrameAllocRequest *req, AVHWFramesContext *frames_ctx; AVQSVFramesContext *frames_hwctx; + if (!ext_frames_hwctx->nb_surfaces) + return MFX_ERR_UNSUPPORTED; + frames_ref = av_hwframe_ctx_alloc(ext_frames_ctx->device_ref); if (!frames_ref) return MFX_ERR_MEMORY_ALLOC; @@ -899,6 +911,9 @@ static mfxStatus qsv_frame_alloc(mfxHDL pthis, mfxFrameAllocRequest *req, static mfxStatus qsv_frame_free(mfxHDL pthis, mfxFrameAllocResponse *resp) { + if (!resp->mids) + return MFX_ERR_NONE; + av_buffer_unref((AVBufferRef**)&resp->mids[resp->NumFrameActual]); ff_refstruct_unref(&resp->mids[resp->NumFrameActual + 1]); av_freep(&resp->mids); @@ -907,11 +922,20 @@ static mfxStatus qsv_frame_free(mfxHDL pthis, mfxFrameAllocResponse *resp) static mfxStatus qsv_frame_lock(mfxHDL pthis, mfxMemId mid, mfxFrameData *ptr) { - QSVMid *qsv_mid = mid; - AVHWFramesContext *hw_frames_ctx = (AVHWFramesContext*)qsv_mid->hw_frames_ref->data; - AVQSVFramesContext *hw_frames_hwctx = hw_frames_ctx->hwctx; + QSVFramesContext *ctx = (QSVFramesContext *)pthis; + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)ctx->hw_frames_ctx->data; + AVQSVFramesContext *frames_hwctx = frames_ctx->hwctx; + QSVMid *qsv_mid; + AVHWFramesContext *hw_frames_ctx; + AVQSVFramesContext *hw_frames_hwctx; int ret; + if (!frames_hwctx->nb_surfaces) + return MFX_ERR_UNSUPPORTED; + + qsv_mid = mid; + hw_frames_ctx = (AVHWFramesContext*)qsv_mid->hw_frames_ref->data; + hw_frames_hwctx = hw_frames_ctx->hwctx; if (qsv_mid->locked_frame) return MFX_ERR_UNDEFINED_BEHAVIOR; @@ -964,8 +988,15 @@ fail: static mfxStatus qsv_frame_unlock(mfxHDL pthis, mfxMemId mid, mfxFrameData *ptr) { - QSVMid *qsv_mid = mid; + QSVFramesContext *ctx = (QSVFramesContext *)pthis; + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)ctx->hw_frames_ctx->data; + AVQSVFramesContext *frames_hwctx = frames_ctx->hwctx; + QSVMid *qsv_mid; + + if (!frames_hwctx->nb_surfaces) + return MFX_ERR_UNSUPPORTED; + qsv_mid = mid; av_frame_free(&qsv_mid->locked_frame); av_frame_free(&qsv_mid->hw_frame); @@ -974,9 +1005,18 @@ static mfxStatus qsv_frame_unlock(mfxHDL pthis, mfxMemId mid, mfxFrameData *ptr) static mfxStatus qsv_frame_get_hdl(mfxHDL pthis, mfxMemId mid, mfxHDL *hdl) { - QSVMid *qsv_mid = (QSVMid*)mid; + QSVFramesContext *ctx = (QSVFramesContext *)pthis; + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)ctx->hw_frames_ctx->data; + AVQSVFramesContext *frames_hwctx = frames_ctx->hwctx; mfxHDLPair *pair_dst = (mfxHDLPair*)hdl; - mfxHDLPair *pair_src = (mfxHDLPair*)qsv_mid->handle_pair; + mfxHDLPair *pair_src; + + if (frames_hwctx->nb_surfaces) { + QSVMid *qsv_mid = (QSVMid*)mid; + pair_src = (mfxHDLPair*)qsv_mid->handle_pair; + } else { + pair_src = (mfxHDLPair*)mid; + } pair_dst->first = pair_src->first; @@ -1090,13 +1130,17 @@ int ff_qsv_init_session_frames(AVCodecContext *avctx, mfxSession *psession, if (!opaque) { qsv_frames_ctx->logctx = avctx; + qsv_frames_ctx->mids = NULL; + qsv_frames_ctx->nb_mids = 0; /* allocate the memory ids for the external frames */ - ff_refstruct_unref(&qsv_frames_ctx->mids); - qsv_frames_ctx->mids = qsv_create_mids(qsv_frames_ctx->hw_frames_ctx); - if (!qsv_frames_ctx->mids) - return AVERROR(ENOMEM); - qsv_frames_ctx->nb_mids = frames_hwctx->nb_surfaces; + if (frames_hwctx->nb_surfaces) { + ff_refstruct_unref(&qsv_frames_ctx->mids); + qsv_frames_ctx->mids = qsv_create_mids(qsv_frames_ctx->hw_frames_ctx); + if (!qsv_frames_ctx->mids) + return AVERROR(ENOMEM); + qsv_frames_ctx->nb_mids = frames_hwctx->nb_surfaces; + } err = MFXVideoCORE_SetFrameAllocator(session, &frame_allocator); if (err != MFX_ERR_NONE) From patchwork Sun Apr 28 07:39:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 48335 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:1509:b0:1a9:af23:56c1 with SMTP id nq9csp1338403pzb; Sun, 28 Apr 2024 00:40:21 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXy50PA5FRgC56dKg6VTOyR6EG9lbh8u6hxjP1K+WPxF6Yi3b22AHmSQQQa3fAIGLDyGsco8/3JWRreSGgRnXlcmLT4XO2nv8AdHg== X-Google-Smtp-Source: AGHT+IF54jmbhl/QukRxpV2/kMVgD4J96tih9T4FTCiEnbwK6czSF/r6TiMZ0Ip+XeYHAZLnqb5t X-Received: by 2002:a17:907:7e97:b0:a58:ed29:332e with SMTP id qb23-20020a1709077e9700b00a58ed29332emr1825379ejc.4.1714290021341; Sun, 28 Apr 2024 00:40:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714290021; cv=none; d=google.com; s=arc-20160816; b=Yg2ORGfY/5nWUHuhADMCvVVS98ik8VicxSdJlqOjZwinchUfRelQKPC21hoWg+SPWg Cvsxgex8USVPiNtshcO2vcA1X4lDC+ydngXd2FOZS4Nd1kqA2uGpCMqRXnk1XnNnpInv mvuoBtQfeoORsfE4ZKIlq99az+o80BAfvauegNnOMKVV3EbOtFuApOfQnvJkLkIeo+Z5 1rBGJ8PXoRuAJwmDkB/19vgsoRLMXDmATAv6Cy6KwLiMh0JDTmIUsEe6n98Ec7HQ/0bw NCpC+2fldSZx5iTLIS8Setvnxf5TMjj/9/ONMj/RjJoN/Jpo4vJb1T/VDOCcRlMVu2qh jC8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=vXQADTe/YUXqywv7rCAy5cAW/i0Ka++pWiOC2RPj9AA=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=nw+5y3SNlPJcwnIhKX+xwtLeiqzgbuH3YfcU9XK+D6RLHC4dXIuNKctdUsfWiUfrFv fCJq+jS0qwvzhTJJbLuPvcZqfCMIjOyoYjeO+UmU3Fmp37MTckuU2zIlnJFtFllzBZ6a e2vxrFSps6S2IiyPrGvnUFGEBZ/KULSR2fpXPqjfQMyARPQiasSFAo2MOF+zQYXoTRlZ j06OcsbNLwOp64AzFkL2xtCRYxFC/hdf5Z9cJMezaq/CCj6JWic1JKEJy3jClWwPr0jF twiyIl5CRnVWAl0Fzd4BrPtl7Tn9t+znvxWM+de7KDD+aFQQv8aEA4llnKGaRSoYnvwA GTmg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=ZFpuRYYg; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b7-20020a17090630c700b00a58df77d59dsi1880657ejb.1050.2024.04.28.00.40.21; Sun, 28 Apr 2024 00:40:21 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=ZFpuRYYg; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E383368D439; Sun, 28 Apr 2024 10:39:45 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BE24C68D41A for ; Sun, 28 Apr 2024 10:39:35 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1714289981; x=1745825981; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9utUtQXk8J352q+yR8ZI9MZfBLkNMfldk1YRYTuwOnI=; b=ZFpuRYYgO1dN3uDOSkRy71iRZ2utAXTKQOnH4WmADI68hCyChrM67PN4 P7d5lIcTlBJkmjNOiAmWtn56gIerifbcdKwSB+aY6WiemXHFiQHMVaIqh E7/3yH7SNeib5TWql+vfDXtbbw1Pcmqgyt+Z5IAt9oqg5v6zZRdmGcCJM 6+Eu5z+X4VAXviquRMYav64OAc2tm9Jua/WerYX7+iz2nNAHB8rapw+z3 ThEOosMMu8xraPihK7+VpUvo45q5FTxUISJkp4kyZhf+SQKdKZm9ZYMtK 64X0SOhy2A+Gnrs2e22OqNx7JLZ/3hC32L8joteiaL+quZSK5tY90hfgi g==; X-CSE-ConnectionGUID: MaRaLnEGTmuvI0guJLnx0w== X-CSE-MsgGUID: ovG6Y2YxTFqsGSZn6bfF+A== X-IronPort-AV: E=McAfee;i="6600,9927,11057"; a="20662198" X-IronPort-AV: E=Sophos;i="6.07,237,1708416000"; d="scan'208";a="20662198" Received: from orviesa003.jf.intel.com ([10.64.159.143]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2024 00:39:30 -0700 X-CSE-ConnectionGUID: C24MChKATs60wRRYpJBL/g== X-CSE-MsgGUID: ycgY6a5KR6ONgRa5kVmdcQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,237,1708416000"; d="scan'208";a="30491640" Received: from unknown (HELO xhh-dg264.sh.intel.com) ([10.238.2.76]) by ORVIESA003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2024 00:39:29 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Sun, 28 Apr 2024 15:39:08 +0800 Message-Id: <20240428073911.423287-6-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240428073911.423287-1-haihao.xiang@intel.com> References: <20240428073911.423287-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 6/9] lavc/qsvenc: use the right info for encoding X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: BkAY0DmtdSYO From: Haihao Xiang Signed-off-by: Haihao Xiang --- libavcodec/qsvenc.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 018d193495..ca1a88e49f 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -744,8 +744,9 @@ static int init_video_param_jpeg(AVCodecContext *avctx, QSVEncContext *q) if (avctx->hw_frames_ctx) { AVHWFramesContext *frames_ctx = (AVHWFramesContext *)avctx->hw_frames_ctx->data; AVQSVFramesContext *frames_hwctx = frames_ctx->hwctx; - q->param.mfx.FrameInfo.Width = frames_hwctx->surfaces[0].Info.Width; - q->param.mfx.FrameInfo.Height = frames_hwctx->surfaces[0].Info.Height; + mfxFrameInfo *info = frames_hwctx->nb_surfaces ? &frames_hwctx->surfaces[0].Info : frames_hwctx->info; + q->param.mfx.FrameInfo.Width = info->Width; + q->param.mfx.FrameInfo.Height = info->Height; } if (avctx->framerate.den > 0 && avctx->framerate.num > 0) { @@ -868,8 +869,9 @@ static int init_video_param(AVCodecContext *avctx, QSVEncContext *q) if (avctx->hw_frames_ctx) { AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; AVQSVFramesContext *frames_hwctx = frames_ctx->hwctx; - q->param.mfx.FrameInfo.Width = frames_hwctx->surfaces[0].Info.Width; - q->param.mfx.FrameInfo.Height = frames_hwctx->surfaces[0].Info.Height; + mfxFrameInfo *info = frames_hwctx->nb_surfaces ? &frames_hwctx->surfaces[0].Info : frames_hwctx->info; + q->param.mfx.FrameInfo.Width = info->Width; + q->param.mfx.FrameInfo.Height = info->Height; } if (avctx->framerate.den > 0 && avctx->framerate.num > 0) { From patchwork Sun Apr 28 07:39:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 48337 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:1509:b0:1a9:af23:56c1 with SMTP id nq9csp1338492pzb; Sun, 28 Apr 2024 00:40:39 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXtuNqtv9ryEWFX31r/o2zdx6fQvj0zgtcjPwMVqD490RRSX6um+gZLLNW5R2aGDAVlOrTkUwGO55hlY0yqcHjtzN5Zuc3T48/ELQ== X-Google-Smtp-Source: AGHT+IF8k1ARt4vCK8I4WrUWIk6cfuGz3cZdFw/QJBBgBiUNJ4622Dq0g1YJy0nhQDVQ14jvMseY X-Received: by 2002:a50:a68e:0:b0:56e:23db:3e87 with SMTP id e14-20020a50a68e000000b0056e23db3e87mr5229405edc.11.1714290039051; Sun, 28 Apr 2024 00:40:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714290039; cv=none; d=google.com; s=arc-20160816; b=V7Zz50XJ0D9puB94AazsfB0acN9pKV6uDVq57m9BhrARSdVMGdlfrOHOYC8CDbLdon BZa4BQdz2BiBUEfedZS6vjE23Af/uHDDlITTdKSp2PVzvs2KHxrhzLRQiEvnS/cEEisR u9HJTg1rQ7JSmtVpTQAIAd8R/Mmm9D+bYsuiF6iHWTmEDpRPv+JgbF45XB/7cwxB2Cv/ X9Pd5FTPEY+WX9C3sjwSjpW7CqtwKUZMs1MdMyOsMFShXc5+Q3cS8Qqv9FqX7gobICZ5 5ZsS7sKH0kvAg9UXMPssi+495rTQGK+lcoPxz5AK5HGHXslBZjguq3Eg6p/7RPudMhV7 uBLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=BJH+0386Kqvxcgv63i3poE1MIRFr47BLeuWZZfJVc4U=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=IalmmXBYJRieXJ9b+UqiLN0Jsb/r//ysDu4Y3Q6AmaJu96d0IfSLA4NiyA672bN+YK ltr/CVeYcJwkl1V6uJMO3lhMmOezBmcdV75onKnKSxVK5YJKGgy3NBheFtQSQ9ltNVdG QGs1PpMJvC5iF9Se2oLO3n8+0yfqC8yTFw5y6xYe8Cpd36V02U2w1mMrfEzPPetyz4A2 3Qp6tnhoqlyeRIfGi6RmRNLWihRXRrhC0M6tvX8VPEfit1ABUGaCjhpjaVEMQ2rvcQUv JO4MDB417vUiSAn+o+gmS30nqVMGi0ZDqtGWNvAxCgxx6GL0kSfUBWbsMcTb0ja6oNb6 8E1g==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=fFLtd8fV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id t15-20020a50d70f000000b005727d5b1bc1si418392edi.637.2024.04.28.00.40.38; Sun, 28 Apr 2024 00:40:39 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=fFLtd8fV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 10DFB68D44C; Sun, 28 Apr 2024 10:39:49 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E026068D41A for ; Sun, 28 Apr 2024 10:39:38 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1714289984; x=1745825984; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZElBHfsq0steEF61ojznKLBF82oiH47z4nxoa8CSgXU=; b=fFLtd8fVGNA9uCm8QGHj5Vpckvh8dQSgKaeaHe9Nb3XUcEzXVm3aIJQb 5SmiyhDC8nX39hkISPpFyP66ccFefklwaI7fbmrS4F5W6UG1JjorfWBN5 jdsXT/XyzX4NcnHWQmxqDmPNjyqrbLkiSOQVYPB3j3SWazjRrdaWhS7I4 juONBXQPszkuXV/tsksfWguvsjzFGncEr7J+Fm79NDBUQN1LCCLLJeYBc LZGzzYbIo+znDxm7Q9xIKTWZMBC8B/uVfDTnzylqcaWw4sRWH9fU9lHKd My5t3wmqShaJoVzILq/ac/a0y8Sm89ZSYyaeN0iWVWLhxCI/bSlvCCYKf Q==; X-CSE-ConnectionGUID: kF3PCt+GQ4y/jjPuBkdf8w== X-CSE-MsgGUID: CZxAbwzhTYGJtJDRNvD8KA== X-IronPort-AV: E=McAfee;i="6600,9927,11057"; a="20662199" X-IronPort-AV: E=Sophos;i="6.07,237,1708416000"; d="scan'208";a="20662199" Received: from orviesa003.jf.intel.com ([10.64.159.143]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2024 00:39:31 -0700 X-CSE-ConnectionGUID: c6eIj9iYQF6rI6WofgveYw== X-CSE-MsgGUID: Ogg3aDVqSkeRnQFGZV6ozw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,237,1708416000"; d="scan'208";a="30491645" Received: from unknown (HELO xhh-dg264.sh.intel.com) ([10.238.2.76]) by ORVIESA003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2024 00:39:30 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Sun, 28 Apr 2024 15:39:09 +0800 Message-Id: <20240428073911.423287-7-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240428073911.423287-1-haihao.xiang@intel.com> References: <20240428073911.423287-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 7/9] lavc/qsvdec: require a dynamic frame pool if possible X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: rd+5XKV+pF9D From: Haihao Xiang This allows a downstream element stores more frames from qsv decoders and fixes error in get_buffer(). $ ffmpeg -hwaccel qsv -hwaccel_output_format qsv -i input.mp4 -vf reverse -f null - [vist#0:0/h264 @ 0x562248f12c50] Decoding error: Cannot allocate memory [h264_qsv @ 0x562248f66b10] get_buffer() failed Signed-off-by: Haihao Xiang --- libavcodec/qsvdec.c | 57 +++++++++++++++++++++++++++++++++++---------- 1 file changed, 45 insertions(+), 12 deletions(-) diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index 5528bcdc8c..f7eaf88e50 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -42,6 +42,7 @@ #include "libavutil/imgutils.h" #include "libavutil/film_grain_params.h" #include "libavutil/mastering_display_metadata.h" +#include "libavutil/avassert.h" #include "avcodec.h" #include "codec_internal.h" @@ -68,6 +69,8 @@ static const AVRational mfx_tb = { 1, 90000 }; AV_NOPTS_VALUE : pts_tb.num ? \ av_rescale_q(mfx_pts, mfx_tb, pts_tb) : mfx_pts) +#define MFX_IMPL_VIA_MASK(impl) (0x0f00 & (impl)) + typedef struct QSVAsyncFrame { mfxSyncPoint *sync; QSVFrame *frame; @@ -77,6 +80,7 @@ typedef struct QSVContext { // the session used for decoding mfxSession session; mfxVersion ver; + mfxHandleType handle_type; // the session we allocated internally, in case the caller did not provide // one @@ -183,6 +187,7 @@ static int qsv_init_session(AVCodecContext *avctx, QSVContext *q, mfxSession ses AVBufferRef *hw_frames_ref, AVBufferRef *hw_device_ref) { int ret; + mfxIMPL impl; if (q->gpu_copy == MFX_GPUCOPY_ON && !(q->iopattern & MFX_IOPATTERN_OUT_SYSTEM_MEMORY)) { @@ -240,27 +245,52 @@ static int qsv_init_session(AVCodecContext *avctx, QSVContext *q, mfxSession ses q->session = q->internal_qs.session; } - if (MFXQueryVersion(q->session, &q->ver) != MFX_ERR_NONE) { - av_log(avctx, AV_LOG_ERROR, "Error querying the session version. \n"); - q->session = NULL; + if (MFXQueryIMPL(q->session, &impl) == MFX_ERR_NONE) { + switch (MFX_IMPL_VIA_MASK(impl)) { + case MFX_IMPL_VIA_VAAPI: + q->handle_type = MFX_HANDLE_VA_DISPLAY; + break; - if (q->internal_qs.session) { - MFXClose(q->internal_qs.session); - q->internal_qs.session = NULL; - } + case MFX_IMPL_VIA_D3D11: + q->handle_type = MFX_HANDLE_D3D11_DEVICE; + break; + + case MFX_IMPL_VIA_D3D9: + q->handle_type = MFX_HANDLE_D3D9_DEVICE_MANAGER; + break; - if (q->internal_qs.loader) { - MFXUnload(q->internal_qs.loader); - q->internal_qs.loader = NULL; + default: + av_assert0(!"should not reach here"); } + } else { + av_log(avctx, AV_LOG_ERROR, "Error querying the implementation. \n"); + goto fail; + } - return AVERROR_EXTERNAL; + if (MFXQueryVersion(q->session, &q->ver) != MFX_ERR_NONE) { + av_log(avctx, AV_LOG_ERROR, "Error querying the session version. \n"); + goto fail; } /* make sure the decoder is uninitialized */ MFXVideoDECODE_Close(q->session); return 0; + +fail: + q->session = NULL; + + if (q->internal_qs.session) { + MFXClose(q->internal_qs.session); + q->internal_qs.session = NULL; + } + + if (q->internal_qs.loader) { + MFXUnload(q->internal_qs.loader); + q->internal_qs.loader = NULL; + } + + return AVERROR_EXTERNAL; } static int qsv_decode_preinit(AVCodecContext *avctx, QSVContext *q, enum AVPixelFormat pix_fmt, mfxVideoParam *param) @@ -310,7 +340,10 @@ static int qsv_decode_preinit(AVCodecContext *avctx, QSVContext *q, enum AVPixel hwframes_ctx->height = FFALIGN(avctx->coded_height, 32); hwframes_ctx->format = AV_PIX_FMT_QSV; hwframes_ctx->sw_format = avctx->sw_pix_fmt; - hwframes_ctx->initial_pool_size = q->suggest_pool_size + 16 + avctx->extra_hw_frames; + if (QSV_RUNTIME_VERSION_ATLEAST(q->ver, 2, 9) && q->handle_type != MFX_HANDLE_D3D9_DEVICE_MANAGER) + hwframes_ctx->initial_pool_size = 0; + else + hwframes_ctx->initial_pool_size = q->suggest_pool_size + 16 + avctx->extra_hw_frames; frames_hwctx->frame_type = MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET; ret = av_hwframe_ctx_init(avctx->hw_frames_ctx); From patchwork Sun Apr 28 07:39:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 48338 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:1509:b0:1a9:af23:56c1 with SMTP id nq9csp1338531pzb; Sun, 28 Apr 2024 00:40:48 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVz5LI3lfQJT1I+7OIhwKJ9y+SsJgy7Nus+Y1i2sXdvTZVFHftugAZN88S6mvv6OEUnweYSSauCAROR2drX5fjyi8Em851XBgURCw== X-Google-Smtp-Source: AGHT+IGdMibosy/B4XuGV4jTWn8xv7K0ZHFCrPPpHacQAtaS989gRMttQDB9l98Rsd8OXaGMM2g4 X-Received: by 2002:aa7:c952:0:b0:572:512a:ecb6 with SMTP id h18-20020aa7c952000000b00572512aecb6mr3983939edt.4.1714290048068; Sun, 28 Apr 2024 00:40:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714290048; cv=none; d=google.com; s=arc-20160816; b=kNx9Qngn6rvlS88k7dB4+i6Tbf7FcWrcKmBdR9lcM0gTxoVleEektu+iyrTOLBWro/ 8PgUL499Luzy0cbp5p9n+qe6i/5W6l18PYVgP0S973nBrIfOyM9GSppcP10/1Iui2vjd qIl0qG39TluxckWS8gOAaZL/3BYYwpkkBbfyIB4kwEn86gDFY8nnQv3PdYxhMjxUSKPH 1duUZ/4fGJfQYsdFasWKKsvoMLiNx2YF9UD8Qt+dFBglcIKQhCMwawya1TpOWrr/Yn72 tqR1zXzPemwJhELJEdrPEEwgO4NmTXwjHwYVzjb7cMo1GNOMklh6CZeLHO6geB6MVkTA oU/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=vD533FF3uWcCeZ5DCcAIm9iN3R7A4SgkR2bPv7M32UU=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=GrTMbY6XevSM4pBxIZ4X8raQSi645Acd8MRZYb4V2NWq1M+cQw1HB4BBEGX2aIBDka 5CKmzKmhft3jp6ElrZhISCd5hDG18Ritk4z7ugOZrd5+BwHqK1utB8gQyLQJEkI4OCxM Qh/mjIPcK141/zQgs/lHWo7Gf9UDsMHjHO01pq+IcXiweEIVlxVmAFPNQLBxJsz4Z9FJ ++Poruw+fximPmi0xQ9DbvaxQsimbjgJ2T+VUBgEBap2yuhEjBVdkIVFhx2NUDA6Dh/C cW/X5TeX7EiIoA2BmmrEWqoyOv+b01nr0e6RnDQ4esshGr25FdOQA/JRoHDJI2ksyaEc vPaw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=K9qRm5Zl; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id z3-20020a05640235c300b005726a3e8bb4si1721208edc.603.2024.04.28.00.40.47; Sun, 28 Apr 2024 00:40:48 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=K9qRm5Zl; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1C01968CAFB; Sun, 28 Apr 2024 10:39:50 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 995F568D436 for ; Sun, 28 Apr 2024 10:39:40 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1714289985; x=1745825985; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vobbuksIXQFGvr2uf1uD24g1pb7m2wNNUpAdFsh1PDk=; b=K9qRm5ZlmBRSv4bn6JiPDdCDX3+wjGTALeXwMY3EuvLIWlkUbYcBc/kn Y3+ZWA8eI2R9ZtCqxmZvk7eYBEKSbcPoo9VovzOBJTA7A2tVwsaO40kXz +c5Zbk07HfBZFS7NYhEx7Ej6XUcZzHJDzvERKYNY1S9SgKmO1MZq35j3A 9V1SE1EpoqGNzqqv6dnQaRUnny6yvOojzWo8JFUn6qZbSgXOxA3xB3eBi oircW7WXp1kotVdBk3yN8/GyTa5b2a5WmCDpmQk3B0ujuW9M13kzm2Gsm 894UE2W/VoKprH+5k7DGnQ3MyfeyjLrP86C+u0797pPiz3UK+Fjb7tE+M A==; X-CSE-ConnectionGUID: 8HYQRaXDTS26VaIil398qA== X-CSE-MsgGUID: 9nLnjjyvSKiz8u5l0Ie7Dg== X-IronPort-AV: E=McAfee;i="6600,9927,11057"; a="20662200" X-IronPort-AV: E=Sophos;i="6.07,237,1708416000"; d="scan'208";a="20662200" Received: from orviesa003.jf.intel.com ([10.64.159.143]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2024 00:39:33 -0700 X-CSE-ConnectionGUID: xxMKA7jsQiui0KtnUvmeEw== X-CSE-MsgGUID: pbS0DgkdRlSkTyViLebvTA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,237,1708416000"; d="scan'208";a="30491653" Received: from unknown (HELO xhh-dg264.sh.intel.com) ([10.238.2.76]) by ORVIESA003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2024 00:39:32 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Sun, 28 Apr 2024 15:39:10 +0800 Message-Id: <20240428073911.423287-8-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240428073911.423287-1-haihao.xiang@intel.com> References: <20240428073911.423287-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 8/9] lavfi/qsvvpp: use the right mfxFrameInfo when dynamic frame pool is used X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: lzh9+QnrT5vc From: Haihao Xiang Signed-off-by: Haihao Xiang --- libavfilter/qsvvpp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavfilter/qsvvpp.c b/libavfilter/qsvvpp.c index 8c92fec0c1..6814d8add8 100644 --- a/libavfilter/qsvvpp.c +++ b/libavfilter/qsvvpp.c @@ -308,7 +308,7 @@ static int fill_frameinfo_by_link(mfxFrameInfo *frameinfo, AVFilterLink *link) frames_ctx = (AVHWFramesContext *)link->hw_frames_ctx->data; frames_hwctx = frames_ctx->hwctx; - *frameinfo = frames_hwctx->surfaces[0].Info; + *frameinfo = frames_hwctx->nb_surfaces ? frames_hwctx->surfaces[0].Info : *frames_hwctx->info; } else { pix_fmt = link->format; desc = av_pix_fmt_desc_get(pix_fmt); From patchwork Sun Apr 28 07:39:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 48336 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:1509:b0:1a9:af23:56c1 with SMTP id nq9csp1338454pzb; Sun, 28 Apr 2024 00:40:30 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUKBg/8zDRLEkbahzhxkMyJLLZdauooVksamlN8YYHbVpAN5rXOB7havl3AylI4gs9bd3fDaWXjuwpSia5o06UbnGMhB3WyIU0EWA== X-Google-Smtp-Source: AGHT+IF78Dq5dyeZMwDSq+l7gCftvrqd1GvYPBcOWR3q0LWhwbCmSWYsn/uEG2QwzkQ8LVWpxbwS X-Received: by 2002:a50:a687:0:b0:568:1882:651f with SMTP id e7-20020a50a687000000b005681882651fmr4612992edc.25.1714290030010; Sun, 28 Apr 2024 00:40:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714290029; cv=none; d=google.com; s=arc-20160816; b=j5mnH0a9od385955FIs3BS7MnSvLXzrugKcBM4Gg3hFPOXvgFTqxXme0KrGlN0wZQA UYYIyDBeDx4w0bDTXslnEeV2r+Pavh5xSh9fwP0wztXjccZS7DydQ/rhIjJIRP9PBzhW ifElrvMcmyE5rHOPNHGsSRPWRJTN6VXdBUtXFKmbs4kNhY0N3d95bZp0LXirbA3C9ANC xpCzvzv4esiFhx3Ho6xHmX8kAEeCaZ5ynIKO7pMpvHYYQT4dIUQdSmuqaHW48E+O9IcS /WHyZNIXyC5tW3XKBnaprj3XpVoCp2P/wpcOfQfXKvcIQzapkHMvBhestjLjSeHw164V f5ow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=srVeswSGWM0yo5ePsneBjgqrsYcddYd6cFiYFrQpb9M=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=ja/RR3Dbj3SX7qXBeEo9Lyirvo6ilmiGG7ESLIUzInUOOpLXHuILk8OyfaUyQPVtp3 A7tyXwbfUzmKIyuHs22dhP2l/yc0uqvZMDK0u31D8i5OtskUXgY29zljdj/2hAsA3Q6t 8fql6nFTNDITct1ls95CsEkslTiXJOaqs5/y165/ALcO21AskK0WU8r6TZ/41XDCs/kF mWBhmGybbzlHMbuZi4YhHtALj5PkCtHErZ6Uc816kzXB5DoDoZNpV6u1e0ntlO0GcAXd 0vW6JxdcK3DU1+5C0zoKXGSDJrsCx/9GZSIaEcJSUpDrYD2xBXBYT+vvFXaKAQbsaCO4 BnVA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=baupX+y5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f10-20020a056402150a00b005727a2245ebsi764194edw.97.2024.04.28.00.40.29; Sun, 28 Apr 2024 00:40:29 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=baupX+y5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1306868D3A3; Sun, 28 Apr 2024 10:39:47 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CFF0568D3FE for ; Sun, 28 Apr 2024 10:39:41 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1714289982; x=1745825982; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=qi9q3WqqaBKtnu4/jfiBFOb5LvFPAn5ARt4eDJx9diQ=; b=baupX+y5dinFWTUyiX80wIbb+1X2hc29XsdJ1gZMR079y/2ZfmJ48JR+ mBwBZ/SM2oDt+Aia1jvl8SubaKsXkhQexhjeYmEM8+XxyiCUYQohAqHAO rNQfmu5b1gSW8kjuztLtY/7XdLE+uYo6Eop2+rwdp5uwSfZ5APyQCSerT CFtpWmkFqmkNug7rUbxZ7I2gzAxDbUe/Ilf2rM8bfRBarOxA5IWYE4uKH vsqIXC6kOZTvs7jO1/PorypATMmDQ1Q8O8iPbqSQwtGhIfq/zvrQB0LRF XQftLX5SBIt9ifBpXVq9MVZsMSzR2JvnF0LT3pACrpBz91Vi+FbJMPEE6 Q==; X-CSE-ConnectionGUID: 0Mh9jqY5RouNxvZtpfXGdA== X-CSE-MsgGUID: su3nOfvIThaOsjCymprkOA== X-IronPort-AV: E=McAfee;i="6600,9927,11057"; a="20662202" X-IronPort-AV: E=Sophos;i="6.07,237,1708416000"; d="scan'208";a="20662202" Received: from orviesa003.jf.intel.com ([10.64.159.143]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2024 00:39:34 -0700 X-CSE-ConnectionGUID: 6ueWlciNTQ6jV5aswsG6dQ== X-CSE-MsgGUID: us4THYQeR0qwRj1BQEgG7Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,237,1708416000"; d="scan'208";a="30491661" Received: from unknown (HELO xhh-dg264.sh.intel.com) ([10.238.2.76]) by ORVIESA003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2024 00:39:33 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Sun, 28 Apr 2024 15:39:11 +0800 Message-Id: <20240428073911.423287-9-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240428073911.423287-1-haihao.xiang@intel.com> References: <20240428073911.423287-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 9/9] lavfi/qsvvpp: require a dynamic frame pool for output if possible X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 1hVmC8HW1SW6 From: Haihao Xiang Signed-off-by: Haihao Xiang --- libavfilter/qsvvpp.c | 52 ++++++++++++++++++++++++-------------------- 1 file changed, 29 insertions(+), 23 deletions(-) diff --git a/libavfilter/qsvvpp.c b/libavfilter/qsvvpp.c index 6814d8add8..f3897823db 100644 --- a/libavfilter/qsvvpp.c +++ b/libavfilter/qsvvpp.c @@ -604,6 +604,26 @@ static int init_vpp_session(AVFilterContext *avctx, QSVVPPContext *s) device_ctx = (AVHWDeviceContext *)device_ref->data; device_hwctx = device_ctx->hwctx; + /* extract the properties of the "master" session given to us */ + ret = MFXQueryIMPL(device_hwctx->session, &impl); + if (ret == MFX_ERR_NONE) + ret = MFXQueryVersion(device_hwctx->session, &ver); + if (ret != MFX_ERR_NONE) { + av_log(avctx, AV_LOG_ERROR, "Error querying the session attributes\n"); + return AVERROR_UNKNOWN; + } + + if (MFX_IMPL_VIA_VAAPI == MFX_IMPL_VIA_MASK(impl)) { + handle_type = MFX_HANDLE_VA_DISPLAY; + } else if (MFX_IMPL_VIA_D3D11 == MFX_IMPL_VIA_MASK(impl)) { + handle_type = MFX_HANDLE_D3D11_DEVICE; + } else if (MFX_IMPL_VIA_D3D9 == MFX_IMPL_VIA_MASK(impl)) { + handle_type = MFX_HANDLE_D3D9_DEVICE_MANAGER; + } else { + av_log(avctx, AV_LOG_ERROR, "Error unsupported handle type\n"); + return AVERROR_UNKNOWN; + } + if (outlink->format == AV_PIX_FMT_QSV) { AVHWFramesContext *out_frames_ctx; AVBufferRef *out_frames_ref = av_hwframe_ctx_alloc(device_ref); @@ -625,9 +645,15 @@ static int init_vpp_session(AVFilterContext *avctx, QSVVPPContext *s) out_frames_ctx->width = FFALIGN(outlink->w, 32); out_frames_ctx->height = FFALIGN(outlink->h, 32); out_frames_ctx->sw_format = s->out_sw_format; - out_frames_ctx->initial_pool_size = 64; - if (avctx->extra_hw_frames > 0) - out_frames_ctx->initial_pool_size += avctx->extra_hw_frames; + + if (QSV_RUNTIME_VERSION_ATLEAST(ver, 2, 9) && handle_type != MFX_HANDLE_D3D9_DEVICE_MANAGER) + out_frames_ctx->initial_pool_size = 0; + else { + out_frames_ctx->initial_pool_size = 64; + if (avctx->extra_hw_frames > 0) + out_frames_ctx->initial_pool_size += avctx->extra_hw_frames; + } + out_frames_hwctx->frame_type = s->out_mem_mode; ret = av_hwframe_ctx_init(out_frames_ref); @@ -653,26 +679,6 @@ static int init_vpp_session(AVFilterContext *avctx, QSVVPPContext *s) } else s->out_mem_mode = MFX_MEMTYPE_SYSTEM_MEMORY; - /* extract the properties of the "master" session given to us */ - ret = MFXQueryIMPL(device_hwctx->session, &impl); - if (ret == MFX_ERR_NONE) - ret = MFXQueryVersion(device_hwctx->session, &ver); - if (ret != MFX_ERR_NONE) { - av_log(avctx, AV_LOG_ERROR, "Error querying the session attributes\n"); - return AVERROR_UNKNOWN; - } - - if (MFX_IMPL_VIA_VAAPI == MFX_IMPL_VIA_MASK(impl)) { - handle_type = MFX_HANDLE_VA_DISPLAY; - } else if (MFX_IMPL_VIA_D3D11 == MFX_IMPL_VIA_MASK(impl)) { - handle_type = MFX_HANDLE_D3D11_DEVICE; - } else if (MFX_IMPL_VIA_D3D9 == MFX_IMPL_VIA_MASK(impl)) { - handle_type = MFX_HANDLE_D3D9_DEVICE_MANAGER; - } else { - av_log(avctx, AV_LOG_ERROR, "Error unsupported handle type\n"); - return AVERROR_UNKNOWN; - } - ret = MFXVideoCORE_GetHandle(device_hwctx->session, handle_type, &handle); if (ret < 0) return ff_qsvvpp_print_error(avctx, ret, "Error getting the session handle");