From patchwork Wed Dec 20 07:10:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 45246 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5184:b0:194:e134:edd4 with SMTP id j4csp43539pzf; Tue, 19 Dec 2023 23:11:23 -0800 (PST) X-Google-Smtp-Source: AGHT+IH2y1Xt4uVmxpqME+U3zkMmB5JqTT0dJsYDFRUtc8lrzm+dc7Rq7dyWcSyYlCsSJi6dJNyy X-Received: by 2002:a17:906:74d0:b0:a26:988f:b9aa with SMTP id z16-20020a17090674d000b00a26988fb9aamr39051ejl.206.1703056283188; Tue, 19 Dec 2023 23:11:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703056283; cv=none; d=google.com; s=arc-20160816; b=gR6y6hXwVEK4AcX+V8Y5OvBqvs37BgN5QJKHz3gVm8kI7GO8WTERdrWrPc3RRk3Y1o FZFsd9ttMqGzXJ2O/8MBaluD3WxAggG2cU+oMsfoIMxRFxYKWK7dR9zBegFVweAMDWpS +Ge1w69H4GGUHSaXAZNuv2dXXjdyBBfWFGXTo2T2qbmnLbg01dz1v9kRu7f1kVeL2ZGW knPKDw8Z9So7q7968jtboP3djjFzb2KQwVmh8mkjw+853hdEujRdNFN6F7pbqgzd9CUq m7tMnMP2ehsocYbTzTMhgnDYxcl4QQM4RvNQgSruC6inkIyO5WlOqoE0ozqnZAU2f0Pg gZ8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=rRqLuYiVhgM/Un00McJ7V+tspQY/i3q/bBOj31LJRGs=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=A/yVx9yNEFPQP/tk5pcYvTA8Dgshrq6PotH6+0YK+mnDBPDoAPxl49jzgLRK3y+wLX ApAaBKRp0VLlgLOPDsLfn8Z5gdqAuxKlUsVA0XbHriUh6NWjbQ3TpCAR4HEErnEX1C9s V0vUrzv9IOkEcsFcv1/LQvYX+2IDLRqy7tTIQYD8mzXf3Y3Fh2Ict1nWYls+VCbMlsxO WZMeNAu5Jl8V22G+YCzCtcZAY8BXbc1s9rUHgx636HjGBfp6KtK9y8iP/zBOqSzw9Rr8 M13UNOCmWmefXZj8PwmcjhkkGtt+q9dkUJKVm/t2imNmRhD2JmqShBTHNDOPNuqK2gyo +skg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=W+qkNoi2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f16-20020a170906139000b00a2369e99556si1763305ejc.230.2023.12.19.23.11.22; Tue, 19 Dec 2023 23:11:23 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=W+qkNoi2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5BD9468D0AD; Wed, 20 Dec 2023 09:11:18 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1F61368CF6B for ; Wed, 20 Dec 2023 09:11:10 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703056276; x=1734592276; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=SFFhjkPsWHXv29vaSfbOIZ3+Aiiu9EQELCwgDjw0ObQ=; b=W+qkNoi2sYtbXhNKO5SfqLKuLZOeWFisS8PBz523od36KJkBGDl/Eq7B 2W3O+tzI5gnYEoqvTsV4r5C4LKAaVFIX5kwjH7zwYSYYwC8Heytf2KSGY 9Z8m1NOyA4+HYEVeGDYQO0ThFeW3CAG8faw5L9BqJAEjpUN9DOMQCEXOy /goMPjX61KnLZSq/Rj2hw5OX6jn7ru9zaN6cQKkhxW1N8oo4I8IXvo0d1 TqFWKnH7YGdXPakWQuRKMqpu67E4V8gyOKkxJLSl2U6rK0Z5oBM2YBmLU fi3mGGX5FnIxOZ6R4RIOHjyVwEiWrJ9eVYALO8+pP6kGDXbYIzfNc1dmh g==; X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="426921027" X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="426921027" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:08 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="18191691" Received: from xhh-dg264.sh.intel.com ([10.238.2.76]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:07 -0800 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 20 Dec 2023 15:10:39 +0800 Message-Id: <20231220071050.3175819-1-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 01/12] lavu/hwcontext_qsv: update AVQSVFramesContext to support dynamic frame pools X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Qd4MMFbkQmJ8 From: Haihao Xiang Add AVQSVFramesContext.info and update the description Signed-off-by: Haihao Xiang --- Rebased https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=9639 against the latest master with a few changes. doc/APIchanges | 3 +++ libavutil/hwcontext_qsv.c | 4 ++-- libavutil/hwcontext_qsv.h | 31 +++++++++++++++++++++++++++---- libavutil/version.h | 2 +- 4 files changed, 33 insertions(+), 7 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index a3e4ebbccd..b68156ee94 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -2,6 +2,9 @@ The last version increases of all libraries were on 2023-02-09 API changes, most recent first: +2023-12-xx - xxxxxxxxxx - lavu 58.36.100 - hwcontext_qsv.h + Add AVQSVFramesContext.info + 2023-12-18 - 74279227dd2 - lavc 60.36.100 - packet.h Add AV_PKT_DATA_IAMF_MIX_GAIN_PARAM, AV_PKT_DATA_IAMF_DEMIXING_INFO_PARAM and AV_PKT_DATA_IAMF_RECON_GAIN_INFO_PARAM. diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index a67552b5ac..071489f070 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -613,7 +613,7 @@ static mfxStatus frame_alloc(mfxHDL pthis, mfxFrameAllocRequest *req, QSVFramesContext *s = ctx->internal->priv; AVQSVFramesContext *hwctx = ctx->hwctx; mfxFrameInfo *i = &req->Info; - mfxFrameInfo *i1 = &hwctx->surfaces[0].Info; + mfxFrameInfo *i1 = hwctx->nb_surfaces ? &hwctx->surfaces[0].Info : hwctx->info; if (!(req->Type & MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET) || !(req->Type & (MFX_MEMTYPE_FROM_VPPIN | MFX_MEMTYPE_FROM_VPPOUT)) || @@ -1173,7 +1173,7 @@ static int qsv_init_internal_session(AVHWFramesContext *ctx, MFX_IOPATTERN_OUT_SYSTEM_MEMORY; par.AsyncDepth = 1; - par.vpp.In = frames_hwctx->surfaces[0].Info; + par.vpp.In = frames_hwctx->nb_surfaces ? frames_hwctx->surfaces[0].Info : *frames_hwctx->info; /* Apparently VPP requires the frame rate to be set to some value, otherwise * init will fail (probably for the framerate conversion filter). Since we diff --git a/libavutil/hwcontext_qsv.h b/libavutil/hwcontext_qsv.h index e2dba8ad83..5950e38500 100644 --- a/libavutil/hwcontext_qsv.h +++ b/libavutil/hwcontext_qsv.h @@ -25,8 +25,8 @@ * @file * An API-specific header for AV_HWDEVICE_TYPE_QSV. * - * This API does not support dynamic frame pools. AVHWFramesContext.pool must - * contain AVBufferRefs whose data pointer points to an mfxFrameSurface1 struct. + * AVHWFramesContext.pool must contain AVBufferRefs whose data pointer points + * to a mfxFrameSurface1 struct. */ /** @@ -51,11 +51,34 @@ typedef struct AVQSVDeviceContext { * This struct is allocated as AVHWFramesContext.hwctx */ typedef struct AVQSVFramesContext { - mfxFrameSurface1 *surfaces; + /** + * A pointer to a mfxFrameSurface1 or mfxFrameInfo struct + * + * When nb_surfaces is non-zero, it is a pointer to a mfxFrameSurface1 + * struct. + * + * When nb_surfaces is 0, it is a pointer to a mfxFrameInfo struct, all + * buffers allocated from the pool have the same mfxFrameInfo. + */ + union { + mfxFrameSurface1 *surfaces; + mfxFrameInfo *info; + }; + + /** + * Number of frames in the pool + * + * It is 0 for dynamic frame pools or AVHWFramesContext.initial_pool_size + * for fixed frame pools. + * + * Note only oneVPL GPU runtime 2.9+ can support dynamic frame pools + * on d3d11va or vaapi + */ int nb_surfaces; /** - * A combination of MFX_MEMTYPE_* describing the frame pool. + * Set by user. It is a combination of MFX_MEMTYPE_* describing the frame + * pool. */ int frame_type; } AVQSVFramesContext; diff --git a/libavutil/version.h b/libavutil/version.h index 7664cd97bd..3b4c50e9a9 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 58 -#define LIBAVUTIL_VERSION_MINOR 35 +#define LIBAVUTIL_VERSION_MINOR 36 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ From patchwork Wed Dec 20 07:10:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 45247 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5184:b0:194:e134:edd4 with SMTP id j4csp43603pzf; Tue, 19 Dec 2023 23:11:35 -0800 (PST) X-Google-Smtp-Source: AGHT+IHVdYbRWEGKLDyDFNKYGJdJ9NcSs/MIWXSFGTiwdDH5orSyZ7dqcZRzmcrCa1Xfw5vm4SHm X-Received: by 2002:a17:906:1086:b0:a26:8d17:9f29 with SMTP id u6-20020a170906108600b00a268d179f29mr373959eju.112.1703056295187; Tue, 19 Dec 2023 23:11:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703056295; cv=none; d=google.com; s=arc-20160816; b=R/xePKuILgF/GPDg6J4ueDMw7/aXqRsDmQIiqYJF54I9d5DG2MI3LUMV7RPKGyKMlh m8O5XOZpSJQrQ6VhFX+Nt1qjunaQvP0hEQcEmBRE6Ae4HvSQEv3yHvdYzVdXr+CZDkb6 MOzDgwSJBgWLTf9JKr/7JmmsI4e0xiH2gvNL26vCO7rqDIxp3UOie6BxVD6wxSnj+DPR lSi1VLqE/gUtiYu5FtCG/o4yw8AdujFwOb4uZU2SdbOOPch57k+8I6XLBZSNe99NNrgH A/fWGVDc44XkpZRW9HNq4+PT6XdtUJ23t9K8NCm95u2EsP/4bg1V+G46P7rmAIBTEg1x 23xA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=jYKYiieJbVddpRQny9rBaIuYYT2n0jiyo6SqB4z+mdE=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=gvFBTJjACfsAwuQqOUYk6dF+jfSlyTuUtPjCveCrzO6b5L1w0hgHxZ7cShCY0EFaci CdBSdDHB/QDIcmaeANsdQ74hxjPHSIOoii0seKxf2ZtuUdgZZFmxBnZQHV3Fdg1bRPh8 KrqZhWHSD4aKHPndihgRBo/ojw5Zq+XU6kQMCVTwHyTsUSQtCnlvZIIYxrOmpWASL47B rowf93MY7cehU18FRGU9HWMXuOlNMMTxwjpHKwqE2WLsKbq1wkBN9ZRN8OHvRh2wMWPt JSNs8/jljQejoEDV/p+LZyuxR1NCV7FmuwpsFSXJgvvGtr4+Kxs+4pdKbsV139MGh/Uz GAJQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=HtgAANpV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id lx19-20020a170906af1300b00a23506e9a80si2605610ejb.914.2023.12.19.23.11.34; Tue, 19 Dec 2023 23:11:35 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=HtgAANpV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 817F668D137; Wed, 20 Dec 2023 09:11:19 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5392B68CF6C for ; Wed, 20 Dec 2023 09:11:12 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703056277; x=1734592277; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rymGb3r+qaFTY/7idSTvB8IwYZW3a3m/u/9Gd7v2JAk=; b=HtgAANpVOVf6Hlm+cMnMEYROBZLYCBLCBhKrKAtIGHHLetoMNBQz32kj S4Eywu+cbKZH+kRqq7IfGcpuz+YWgvoK+onhsybo+J6oosyby9LLS0LCL JQn2hxokSQxnFbVqzP4C/O6B605Rqy/lkzHgBZmSV27GyM2JAUyF8Feg4 VendRlgncMMQgZuO76pNJThYF54I0OED1eZJsf6HK5l57ZgOFsZTxxTLs BBvtg3/c0TMMMENlZQO6zgpGAMh8chPYGU8+YDI1h9EIEW7RaHuG3+Epn 2ddUXu3s//D8gEZf4XTKurx+Ofm4Si2d5rEWSnWtSzybh6TCZmtT5NkmJ w==; X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="426921030" X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="426921030" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:09 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="18191710" Received: from xhh-dg264.sh.intel.com ([10.238.2.76]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:08 -0800 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 20 Dec 2023 15:10:40 +0800 Message-Id: <20231220071050.3175819-2-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231220071050.3175819-1-haihao.xiang@intel.com> References: <20231220071050.3175819-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 02/12] lavu/hwcontext_qsv: create dynamic frame pool if required X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: pva+EVb6X6u2 From: Haihao Xiang When AVHWFramesContext.initial_pool_size is 0, a dynamic frame pool is required. We may support this under certain conditions, e.g. oneVPL 2.9+ support dynamic frame allocation, we needn't provide a fixed frame pool in the mfxFrameAllocator.Alloc callback. Signed-off-by: Haihao Xiang --- libavutil/hwcontext_qsv.c | 157 +++++++++++++++++++++++++++++++++++++- 1 file changed, 154 insertions(+), 3 deletions(-) diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index 071489f070..3922ba3ec0 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -104,8 +104,15 @@ typedef struct QSVFramesContext { #endif AVFrame realigned_upload_frame; AVFrame realigned_download_frame; + + mfxFrameInfo frame_info; } QSVFramesContext; +typedef struct QSVSurface { + mfxFrameSurface1 mfx_surface; + AVFrame *child_frame; +} QSVSurface; + static const struct { enum AVPixelFormat pix_fmt; uint32_t fourcc; @@ -151,6 +158,8 @@ extern int ff_qsv_get_surface_base_handle(mfxFrameSurface1 *surf, enum AVHWDeviceType base_dev_type, void **base_handle); +static int qsv_init_surface(AVHWFramesContext *ctx, mfxFrameSurface1 *surf); + /** * Caller needs to allocate enough space for base_handle pointer. **/ @@ -359,7 +368,32 @@ static void qsv_pool_release_dummy(void *opaque, uint8_t *data) { } -static AVBufferRef *qsv_pool_alloc(void *opaque, size_t size) +static void qsv_pool_release(void *opaque, uint8_t *data) +{ + AVHWFramesContext *ctx = (AVHWFramesContext*)opaque; + QSVFramesContext *s = ctx->internal->priv; + QSVSurface *qsv_surface = (QSVSurface *)data; + mfxHDLPair *hdl_pair = (mfxHDLPair *)qsv_surface->mfx_surface.Data.MemId; + AVHWFramesContext *child_frames_ctx; + + if (!s->child_frames_ref) + return; + + child_frames_ctx = (AVHWFramesContext*)s->child_frames_ref->data; + if (!child_frames_ctx->device_ctx) + return; + +#if CONFIG_VAAPI + if (child_frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_VAAPI) + av_freep(&hdl_pair->first); +#endif + + av_freep(&hdl_pair); + av_frame_free(&qsv_surface->child_frame); + av_freep(&qsv_surface); +} + +static AVBufferRef *qsv_fixed_pool_alloc(void *opaque, size_t size) { AVHWFramesContext *ctx = (AVHWFramesContext*)opaque; QSVFramesContext *s = ctx->internal->priv; @@ -374,6 +408,104 @@ static AVBufferRef *qsv_pool_alloc(void *opaque, size_t size) return NULL; } +static AVBufferRef *qsv_dynamic_pool_alloc(void *opaque, size_t size) +{ + AVHWFramesContext *ctx = (AVHWFramesContext*)opaque; + QSVFramesContext *s = ctx->internal->priv; + AVHWFramesContext *child_frames_ctx; + QSVSurface *qsv_surface = NULL; + mfxHDLPair *handle_pairs_internal = NULL; + int ret; + + if (!s->child_frames_ref) + goto fail; + + child_frames_ctx = (AVHWFramesContext*)s->child_frames_ref->data; + if (!child_frames_ctx->device_ctx) + goto fail; + +#if CONFIG_DXVA2 + if (child_frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_DXVA2) { + av_log(ctx, AV_LOG_ERROR, + "QSV on dxva2 requires a fixed frame pool size\n"); + goto fail; + } +#endif + + qsv_surface = av_calloc(1, sizeof(*qsv_surface)); + if (!qsv_surface) + goto fail; + + qsv_surface->child_frame = av_frame_alloc(); + if (!qsv_surface->child_frame) + goto fail; + + ret = av_hwframe_get_buffer(s->child_frames_ref, qsv_surface->child_frame, 0); + if (ret < 0) + goto fail; + + handle_pairs_internal = av_calloc(1, sizeof(*handle_pairs_internal)); + if (!handle_pairs_internal) + goto fail; + + ret = qsv_init_surface(ctx, &qsv_surface->mfx_surface); + if (ret < 0) + goto fail; + +#if CONFIG_VAAPI + if (child_frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_VAAPI) { + VASurfaceID *surface_id_internal; + + surface_id_internal = av_calloc(1, sizeof(*surface_id_internal)); + if (!surface_id_internal) + goto fail; + + *surface_id_internal = (VASurfaceID)(uintptr_t)qsv_surface->child_frame->data[3]; + handle_pairs_internal->first = (mfxHDL)surface_id_internal; + handle_pairs_internal->second = (mfxMemId)MFX_INFINITE; + } +#endif + +#if CONFIG_D3D11VA + if (child_frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) { + AVD3D11VAFramesContext *child_frames_hwctx = child_frames_ctx->hwctx; + handle_pairs_internal->first = (mfxMemId)qsv_surface->child_frame->data[0]; + + if (child_frames_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) + handle_pairs_internal->second = (mfxMemId)MFX_INFINITE; + else + handle_pairs_internal->second = (mfxMemId)qsv_surface->child_frame->data[1]; + + } +#endif + + qsv_surface->mfx_surface.Data.MemId = (mfxMemId)handle_pairs_internal; + return av_buffer_create((uint8_t *)qsv_surface, sizeof(*qsv_surface), + qsv_pool_release, ctx, 0); + +fail: + if (qsv_surface) { + av_frame_free(&qsv_surface->child_frame); + } + + av_freep(&qsv_surface); + av_freep(&handle_pairs_internal); + + return NULL; +} + +static AVBufferRef *qsv_pool_alloc(void *opaque, size_t size) +{ + AVHWFramesContext *ctx = (AVHWFramesContext*)opaque; + AVQSVFramesContext *hwctx = ctx->hwctx; + + if (hwctx->nb_surfaces == 0) { + return qsv_dynamic_pool_alloc(opaque, size); + } else { + return qsv_fixed_pool_alloc(opaque, size); + } +} + static int qsv_init_child_ctx(AVHWFramesContext *ctx) { AVQSVFramesContext *hwctx = ctx->hwctx; @@ -562,9 +694,28 @@ static int qsv_init_pool(AVHWFramesContext *ctx, uint32_t fourcc) int i, ret = 0; - if (ctx->initial_pool_size <= 0) { - av_log(ctx, AV_LOG_ERROR, "QSV requires a fixed frame pool size\n"); + if (ctx->initial_pool_size < 0) { + av_log(ctx, AV_LOG_ERROR, "Invalid frame pool size\n"); return AVERROR(EINVAL); + } else if (ctx->initial_pool_size == 0) { + mfxFrameSurface1 mfx_surf1; + + ret = qsv_init_child_ctx(ctx); + if (ret < 0) + return ret; + + ctx->internal->pool_internal = av_buffer_pool_init2(sizeof(mfxFrameSurface1), + ctx, qsv_pool_alloc, NULL); + if (!ctx->internal->pool_internal) + return AVERROR(ENOMEM); + + memset(&mfx_surf1, 0, sizeof(mfx_surf1)); + qsv_init_surface(ctx, &mfx_surf1); + s->frame_info = mfx_surf1.Info; + frames_hwctx->info = &s->frame_info; + frames_hwctx->nb_surfaces = 0; + + return 0; } s->handle_pairs_internal = av_calloc(ctx->initial_pool_size, From patchwork Wed Dec 20 07:10:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 45248 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5184:b0:194:e134:edd4 with SMTP id j4csp43660pzf; Tue, 19 Dec 2023 23:11:44 -0800 (PST) X-Google-Smtp-Source: AGHT+IF4cpN1LIwMvSMteovjGACA0OOWqQWuAWyHSlSxW95h970TRTyJCd1EXlZh4qc07GV95859 X-Received: by 2002:a17:906:d6:b0:a23:ff8:cda6 with SMTP id 22-20020a17090600d600b00a230ff8cda6mr5811162eji.75.1703056304161; Tue, 19 Dec 2023 23:11:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703056304; cv=none; d=google.com; s=arc-20160816; b=RqHnjWS54lcE7p6wnxExxBxvtNGjVQcOGyhsSog4RyriMcz4jecG5BC2/6c/kYLuZ0 97Qb+8y1Gse75V0EShonixrWC8zeUoyEQJgAGpV+2vgrInwkWNbmOolomu8ZzLYPTkR5 G3LtaDRoAKSxl/ifGfnsW9xT1/C0Pa5ltUohcdMBUtz9ffEyXs2BZifys1Wb/Q658eVr OL0CSePq08Kknmve7P9Tkx/B70OapyBbOjtUzpxrqggQGH7PINCrzXeHy9ZDh5GrXwzT x0gpaqpjpRaP7wW+Lj34l3nPfs4gVdhxTvBUWqmRYNb1+oHBChp7Rl2Wpxd4iZS6q50A 86ww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=WWnc44Yhhy8CUjh2XeT8Ki5kMWBuXm0/Fu2rZhD+lQI=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=h8iiIgrzAea5M3P6O/8xwRcItlEFNspm6HvmFTMfjuYvqTpqeXtiPzwCg9m46lIu5D KZxQQgSE5wTOSuL1sK5a0/T+fWI3Rs1lFiqGdx/UbACimxWZNk5by77MicnGLg3m8kAQ VcCLv1tAQyTAVtcXPxFDxAtqVwuZTss9Qbe8legxdOtBUBJEKxj7JgXebgscz7ucbym+ hlHLvpLbFBsqpApmAEucsRz/BZ+ces/o4NDs6A2XsdgjGSg7bp9Kq1TVsDBNhKJGiRbN 6ireNdi6Dtl9pSPyP0bp4c+8m4/99tPkv7acoPUdVaiSWrBode6pJdtAHh8Mvdnd9bMZ l35g== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=F5VB25I0; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id u3-20020a17090657c300b00a1d991c3a0esi11454780ejr.361.2023.12.19.23.11.43; Tue, 19 Dec 2023 23:11:44 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=F5VB25I0; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B8ECE68D17D; Wed, 20 Dec 2023 09:11:22 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7691168D0C2 for ; Wed, 20 Dec 2023 09:11:13 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703056278; x=1734592278; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9myP+V3DjPIpvtllQ726pZtNK4hpCsuO/L2qf1kB6so=; b=F5VB25I0uitHZoYYBLDy61T0iS8m41EiWws5+G8wQueRSMqbtxg8PMbb IrrNg1s+wKHRyo1QNdFgZBjVT35mnA0wqOab2AoXeSuuSgY3nGFM8iDr3 o3xA4I7JAxLqFO5wZz0vyP8WF+iM5bVUATsj3DaHiwXmKPIWB3+A/Xuqt 2/EQ0vHk+Y0VDi1YzD+S0NzyPUqj1OaEECFg9ngo9GcnHQV7/8H7pcIZs YLEIZzU4vUrM7VVL8R2VpqjIzB5qiuXREF84prJ9/5VG3V6cqN8ijZuQY PQp4jWo2SSq6S4BGzfL9VPwlPVtvBCTBrUr8nujKOLzXeqwzS7aCcIzfP A==; X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="426921032" X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="426921032" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:10 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="18191733" Received: from xhh-dg264.sh.intel.com ([10.238.2.76]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:09 -0800 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 20 Dec 2023 15:10:41 +0800 Message-Id: <20231220071050.3175819-3-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231220071050.3175819-1-haihao.xiang@intel.com> References: <20231220071050.3175819-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 03/12] lavu/hwcontext_qsv: add support for dynamic frame pool in qsv_frames_derive_to X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: j6CV9UYpPh9w From: Haihao Xiang Allow the source is a dynamic frame pool Signed-off-by: Haihao Xiang --- libavutil/hwcontext_qsv.c | 61 ++++++++++++++++++++++++++++++++++----- 1 file changed, 54 insertions(+), 7 deletions(-) diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index 3922ba3ec0..3259cd38ba 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -1938,18 +1938,52 @@ static int qsv_transfer_data_to(AVHWFramesContext *ctx, AVFrame *dst, return 0; } -static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, - AVHWFramesContext *src_ctx, int flags) +static int qsv_dynamic_frames_derive_to(AVHWFramesContext *dst_ctx, + AVHWFramesContext *src_ctx, int flags) { QSVFramesContext *s = dst_ctx->internal->priv; AVQSVFramesContext *dst_hwctx = dst_ctx->hwctx; - int i; + mfxFrameSurface1 mfx_surf1; - if (src_ctx->initial_pool_size == 0) { - av_log(dst_ctx, AV_LOG_ERROR, "Only fixed-size pools can be " - "mapped to QSV frames.\n"); - return AVERROR(EINVAL); + switch (src_ctx->device_ctx->type) { +#if CONFIG_VAAPI + case AV_HWDEVICE_TYPE_VAAPI: + dst_hwctx->frame_type = MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET; + break; +#endif + +#if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: + { + AVD3D11VAFramesContext *src_hwctx = src_ctx->hwctx; + + if (src_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { + dst_hwctx->frame_type |= MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET; + } else { + dst_hwctx->frame_type |= MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET; + } } + break; +#endif + + default: + return AVERROR(ENOSYS); + } + + memset(&mfx_surf1, 0, sizeof(mfx_surf1)); + qsv_init_surface(dst_ctx, &mfx_surf1); + s->frame_info = mfx_surf1.Info; + dst_hwctx->info = &s->frame_info; + dst_hwctx->nb_surfaces = 0; + return 0; +} + +static int qsv_fixed_frames_derive_to(AVHWFramesContext *dst_ctx, + AVHWFramesContext *src_ctx, int flags) +{ + QSVFramesContext *s = dst_ctx->internal->priv; + AVQSVFramesContext *dst_hwctx = dst_ctx->hwctx; + int i; switch (src_ctx->device_ctx->type) { #if CONFIG_VAAPI @@ -2041,6 +2075,19 @@ static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, return 0; } +static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, + AVHWFramesContext *src_ctx, int flags) +{ + if (src_ctx->initial_pool_size < 0) { + av_log(dst_ctx, AV_LOG_ERROR, "Invalid src frame pool. \n"); + return AVERROR(EINVAL); + } else if (src_ctx->initial_pool_size == 0) { + return qsv_dynamic_frames_derive_to(dst_ctx, src_ctx, flags); + } else { + return qsv_fixed_frames_derive_to(dst_ctx, src_ctx, flags); + } +} + static int qsv_map_to(AVHWFramesContext *dst_ctx, AVFrame *dst, const AVFrame *src, int flags) { From patchwork Wed Dec 20 07:10:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 45249 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5184:b0:194:e134:edd4 with SMTP id j4csp43720pzf; Tue, 19 Dec 2023 23:11:53 -0800 (PST) X-Google-Smtp-Source: AGHT+IF+mtCI6tRQVEuoeSgNglFrnRM+tX4UTylanOJ1knfusMwB4b3gfXucjgff1mDlGjsyzd3A X-Received: by 2002:a50:c88d:0:b0:551:e1ea:9f1a with SMTP id d13-20020a50c88d000000b00551e1ea9f1amr6583919edh.78.1703056313549; Tue, 19 Dec 2023 23:11:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703056313; cv=none; d=google.com; s=arc-20160816; b=EPl9eM8Z3MJA8XBKgr7ans45c+uhM1SdF1UtlYn9+6qu5O8AWJxz4Ap1o7jKY+Pj/G Pg5d8+cdpe3bEBOkE8/Xhnc/YPQ6haI+0TZth0Uw4DOdaCml+yZ4x2JfCJWC2ZavZ1kj dbB94ZHZAoTNM5ZcZwKoEhtFCOSSxt8eCERZFkOnGre64oQZ/FskWNXTEtOjtxU87Ye5 U1AjhRAH52qniBKmReAOI5QJctjHFQbhp2IwfZMSWvtIYvJn4DsczpIgilARQ7xokll3 JqWWqyR+bDSIZ9mzhyIwt6yl+/Uzy/VtlRVbAZ1oAGArbN9XDI0fDRmp6oZ8BZtJLtaY H+sg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=WReOdBeKmRb8s58Cb6faK9NGF5fOiK/8rIe7ihZDRts=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=pDtNhZW67AGnOzT2JVPo1MiP5x6ZOVE52w43MI4sQrUIqukpGEkR70dpqwBAXkByst xPDESLDhfcv4UZT0LO+Ipp5gQKGC81K2wnJ+X+99XVOpRo7Zze3raMRFFwqR00U/CEfa XxENl53PyCA/c1r1snjBYTWXAz9gL+lDq3kvPwyiiDQzxKvDCLq1SQS9rs+PGciwZxK4 AlR4EWTG/6TVCgumAdDIe8Xuapm91FnIBCedZ7gM1WpAxU/KHvhHnVHo9Z65ECdGcBXE HPfZxUvy+mnCgHGpENQDHMJyr8IfRZV/vqxAJoGorx0/ryWfthyWvj6TLDDnINE1Q+pm e5zQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=gaif6mFU; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id cz22-20020a0564021cb600b00553efa7b201si45121edb.106.2023.12.19.23.11.53; Tue, 19 Dec 2023 23:11:53 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=gaif6mFU; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D834668D188; Wed, 20 Dec 2023 09:11:23 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 09C9C68D0C3 for ; Wed, 20 Dec 2023 09:11:16 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703056282; x=1734592282; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=YhCBSYUHFIkIO2G6ta9BoIsAAjSXpn9TM6UZvwVIVAo=; b=gaif6mFUkJ/0Y9O7+tr9CiNnGIJcd+rCIAYeecp8rdXJKcN7RsPEVtGH cGCo9zE6wXSHDejcHbh+rAoLBvyWxo9TL8RtJeyP0btS7+hjVyENAxc47 5Vf5zYQYuIZJHMrd1DzeLT/4bYwpMhfOcRiBJvb+l9OWYGaaB48NpLiHT Yy2TeU+LqzBrLYPWHF5nGVtobvfx6ioAgI6sKZKtW+k6s1Rk7tJQqx0xz gPLvdx05oldmPhaV+d45zmpC43Uod9MpZg9EsDB9fU9KqSZ3jrYNvvzGY nbXo2AseL4zAcfg+os9W/gWCHTfp1BeAWFvRhYouTlEbhIUrLMT/LbW0j A==; X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="426921034" X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="426921034" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="18191763" Received: from xhh-dg264.sh.intel.com ([10.238.2.76]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:10 -0800 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 20 Dec 2023 15:10:42 +0800 Message-Id: <20231220071050.3175819-4-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231220071050.3175819-1-haihao.xiang@intel.com> References: <20231220071050.3175819-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 04/12] lavu/hwcontext_qsv: add support for dynamic frame pool in qsv_map_to X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 9KdOTh72oxKZ From: Haihao Xiang Signed-off-by: Haihao Xiang --- libavutil/hwcontext_qsv.c | 131 +++++++++++++++++++++++++++++++++++++- 1 file changed, 129 insertions(+), 2 deletions(-) diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index 3259cd38ba..2e75f5ca0b 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -2088,8 +2088,8 @@ static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, } } -static int qsv_map_to(AVHWFramesContext *dst_ctx, - AVFrame *dst, const AVFrame *src, int flags) +static int qsv_fixed_pool_map_to(AVHWFramesContext *dst_ctx, + AVFrame *dst, const AVFrame *src, int flags) { AVQSVFramesContext *hwctx = dst_ctx->hwctx; int i, err, index = -1; @@ -2148,6 +2148,133 @@ static int qsv_map_to(AVHWFramesContext *dst_ctx, return 0; } +static void qsv_dynamic_pool_unmap(AVHWFramesContext *ctx, HWMapDescriptor *hwmap) +{ + mfxFrameSurface1 *surfaces_internal = (mfxFrameSurface1 *)hwmap->priv; + mfxHDLPair *handle_pairs_internal = (mfxHDLPair *)surfaces_internal->Data.MemId; + AVHWFramesContext *src_ctx = (AVHWFramesContext *)ctx->internal->source_frames->data; + + switch (src_ctx->format) { +#if CONFIG_VAAPI + case AV_PIX_FMT_VAAPI: + { + av_freep(&handle_pairs_internal->first); + + break; + } +#endif + +#if CONFIG_D3D11VA + case AV_PIX_FMT_D3D11: + { + /* Do nothing */ + break; + } +#endif + default: + av_log(ctx, AV_LOG_ERROR, "Should not reach here. \n"); + break; + } + + av_freep(&handle_pairs_internal); + av_freep(&surfaces_internal); +} + +static int qsv_dynamic_pool_map_to(AVHWFramesContext *dst_ctx, + AVFrame *dst, const AVFrame *src, int flags) +{ + mfxFrameSurface1 *surfaces_internal = NULL; + mfxHDLPair *handle_pairs_internal = NULL; + int ret = 0; + + surfaces_internal = av_calloc(1, sizeof(*surfaces_internal)); + if (!surfaces_internal) { + ret = AVERROR(ENOMEM); + goto fail; + } + + handle_pairs_internal = av_calloc(1, sizeof(*handle_pairs_internal)); + if (!handle_pairs_internal) { + ret = AVERROR(ENOMEM); + goto fail; + } + + ret = qsv_init_surface(dst_ctx, surfaces_internal); + if (ret < 0) + goto fail; + + switch (src->format) { +#if CONFIG_VAAPI + case AV_PIX_FMT_VAAPI: + { + VASurfaceID *surface_id_internal; + + surface_id_internal = av_calloc(1, sizeof(*surface_id_internal)); + if (!surface_id_internal) { + ret =AVERROR(ENOMEM); + goto fail; + } + + *surface_id_internal = (VASurfaceID)(uintptr_t)src->data[3]; + handle_pairs_internal->first = (mfxHDL)surface_id_internal; + handle_pairs_internal->second = (mfxMemId)MFX_INFINITE; + + break; + } +#endif + +#if CONFIG_D3D11VA + case AV_PIX_FMT_D3D11: + { + AVHWFramesContext *src_ctx = (AVHWFramesContext*)src->hw_frames_ctx->data; + AVD3D11VAFramesContext *src_hwctx = src_ctx->hwctx; + + handle_pairs_internal->first = (mfxMemId)src->data[0]; + + if (src_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { + handle_pairs_internal->second = (mfxMemId)MFX_INFINITE; + } else { + handle_pairs_internal->second = (mfxMemId)src->data[1]; + } + + break; + } +#endif + default: + ret = AVERROR(ENOSYS); + goto fail; + } + + surfaces_internal->Data.MemId = (mfxMemId)handle_pairs_internal; + + ret = ff_hwframe_map_create(dst->hw_frames_ctx, + dst, src, qsv_dynamic_pool_unmap, surfaces_internal); + if (ret) + goto fail; + + dst->width = src->width; + dst->height = src->height; + dst->data[3] = (uint8_t*)surfaces_internal; + + return 0; + +fail: + av_freep(&handle_pairs_internal); + av_freep(&surfaces_internal); + return ret; +} + +static int qsv_map_to(AVHWFramesContext *dst_ctx, + AVFrame *dst, const AVFrame *src, int flags) +{ + AVQSVFramesContext *hwctx = dst_ctx->hwctx; + + if (hwctx->nb_surfaces) + return qsv_fixed_pool_map_to(dst_ctx, dst, src, flags); + else + return qsv_dynamic_pool_map_to(dst_ctx, dst, src, flags); +} + static int qsv_frames_get_constraints(AVHWDeviceContext *ctx, const void *hwconfig, AVHWFramesConstraints *constraints) From patchwork Wed Dec 20 07:10:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 45250 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5184:b0:194:e134:edd4 with SMTP id j4csp43784pzf; Tue, 19 Dec 2023 23:12:03 -0800 (PST) X-Google-Smtp-Source: AGHT+IHeZwZjEw9Atx+bBj9nVk4PsRm9LE0LoHjXAur1wF8JgU3PKybncchgdHaRu6LtVgJRx8Ip X-Received: by 2002:a17:907:a1a4:b0:a23:4abd:95ee with SMTP id om36-20020a170907a1a400b00a234abd95eemr1078198ejc.105.1703056322801; Tue, 19 Dec 2023 23:12:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703056322; cv=none; d=google.com; s=arc-20160816; b=yaf2+I6ImC2et2piOPlJLsBPC2Hh8UkNLigbxh2xsDdCwGgBw4BWSNJMRtB9ZTQsqJ kkH3SsyWElNptfgfQ4ntvS8C58q3HGv7mb/QMWpX2d8bsDLzU4zw6n0HHd9voNcNft99 ASZuaT4aicIMlCjs4Hzjc+ErA1GsXqMQPLGmQhsN1LrbqXA1eEpcjCFx+3ApOqPLQOP/ 7yQ2gezRg4Zg51ySCkDXzVCaPmEO47/wC7EpJGP+wKIgX1SN0/TXwoah4/QbIY0fFFMz 4yai/BTpocKPIM5VYnlfMG/yvTKDI4bnakl5WwHeOFyroUI+zDJXN1w4+gn5BPI+GG26 MU+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=SQLeA1L/1XR7ri+FG5C8t11O2RtckrOlPASDfasULZM=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=CTtBlE7X/emt+HTulpTadGiqyHiqY8TmzYPQbHRYua67FiR6lbytAd84q0TJgC6uIA Di/ikVmF1iq8CCKpG+aKbxnNuBdhHvOLp6C9elteGyOVdyPSxtUKyCQpiwaSepvJisIk bUsgSeabIMrAcykTNcLGVRDe6eTNsz1Q4E9a/8GDorvs/KpkbQHxBBBWMKZ7SDpCqdPh Q37QYNPxyJPu6vlJjIFWSxvB2ADdLKUrj2BXITwLpMoj2xER3mCg+HXLPvPoE6/43tu0 SsCysUInFu3CaIjzZsTJT6mDbl5LfF9WgRsandLKby4K4b0G5nQ2tqZtTw88c7aUf14a uhKA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=TO6SES67; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id oy29-20020a170907105d00b00a232e088c69si3412620ejb.290.2023.12.19.23.12.02; Tue, 19 Dec 2023 23:12:02 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=TO6SES67; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1CC8668D195; Wed, 20 Dec 2023 09:11:26 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3858C68D16D for ; Wed, 20 Dec 2023 09:11:18 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703056283; x=1734592283; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8QroD+KwVO5RuGQ1D5M3rDOiOMPkxumfwEV9j6SDEQM=; b=TO6SES67lmRLs9pRrQsffmvdkqhltuXHrYAYhRVOcNdTClsbmpXyQG/s eUlh+Gx64WikoFnvL8PusnsCW/40YFb1upZuRVBvp3rQdwn/3EnjqM7mo HwUq3mOAC+6NH4XO2Pl11NQcBQM7VgY8MWkEo6ADTawfuxObsekeMjdSw /a3rXuxcpmzigMfki0SASEmJy88zg38KwvJiG1Usb9pviK6AwmB60btbB 7f0LDKbQ/QBtRiGVymK6KPkMS1GoQMyHW3og6HREQPqCuycBS85mtXaus nUgwI+kXF24FOJ1WxXft3A0X/B3HkNXwrTHbU2V4/jFfr0+wPmRTirWk+ g==; X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="426921035" X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="426921035" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="18191779" Received: from xhh-dg264.sh.intel.com ([10.238.2.76]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:12 -0800 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 20 Dec 2023 15:10:43 +0800 Message-Id: <20231220071050.3175819-5-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231220071050.3175819-1-haihao.xiang@intel.com> References: <20231220071050.3175819-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 05/12] lavc/qsv: fix the mfx allocator to support dynamic frame pools X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 0GnfHJ2z0tRU From: Haihao Xiang Lock()/Unlock() aren't called to map/unmap data to system memory, so the array of QSVMid is not needed. Signed-off-by: Haihao Xiang --- libavcodec/qsv.c | 70 +++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 57 insertions(+), 13 deletions(-) diff --git a/libavcodec/qsv.c b/libavcodec/qsv.c index 7563625627..e9410d417e 100644 --- a/libavcodec/qsv.c +++ b/libavcodec/qsv.c @@ -839,8 +839,16 @@ static mfxStatus qsv_frame_alloc(mfxHDL pthis, mfxFrameAllocRequest *req, AVHWFramesContext *frames_ctx = (AVHWFramesContext*)ctx->hw_frames_ctx->data; AVQSVFramesContext *frames_hwctx = frames_ctx->hwctx; mfxFrameInfo *i = &req->Info; - mfxFrameInfo *i1 = &frames_hwctx->surfaces[0].Info; + mfxFrameInfo *i1; + if (!frames_hwctx->nb_surfaces) { + av_log(ctx->logctx, AV_LOG_DEBUG, + "Dynamic frame pools, no frame is pre-allocated\n"); + + return MFX_ERR_NONE; + } + + i1 = &frames_hwctx->surfaces[0].Info; if (i->Width > i1->Width || i->Height > i1->Height || i->FourCC != i1->FourCC || i->ChromaFormat != i1->ChromaFormat) { av_log(ctx->logctx, AV_LOG_ERROR, "Mismatching surface properties in an " @@ -859,12 +867,16 @@ static mfxStatus qsv_frame_alloc(mfxHDL pthis, mfxFrameAllocRequest *req, } else if (req->Type & MFX_MEMTYPE_INTERNAL_FRAME) { /* internal frames -- allocate a new hw frames context */ AVHWFramesContext *ext_frames_ctx = (AVHWFramesContext*)ctx->hw_frames_ctx->data; + AVQSVFramesContext *ext_frames_hwctx = ext_frames_ctx->hwctx; mfxFrameInfo *i = &req->Info; AVBufferRef *frames_ref, *mids_buf; AVHWFramesContext *frames_ctx; AVQSVFramesContext *frames_hwctx; + if (!ext_frames_hwctx->nb_surfaces) + return MFX_ERR_UNSUPPORTED; + frames_ref = av_hwframe_ctx_alloc(ext_frames_ctx->device_ref); if (!frames_ref) return MFX_ERR_MEMORY_ALLOC; @@ -912,6 +924,9 @@ static mfxStatus qsv_frame_alloc(mfxHDL pthis, mfxFrameAllocRequest *req, static mfxStatus qsv_frame_free(mfxHDL pthis, mfxFrameAllocResponse *resp) { + if (!resp->mids) + return MFX_ERR_NONE; + av_buffer_unref((AVBufferRef**)&resp->mids[resp->NumFrameActual]); av_buffer_unref((AVBufferRef**)&resp->mids[resp->NumFrameActual + 1]); av_freep(&resp->mids); @@ -920,11 +935,20 @@ static mfxStatus qsv_frame_free(mfxHDL pthis, mfxFrameAllocResponse *resp) static mfxStatus qsv_frame_lock(mfxHDL pthis, mfxMemId mid, mfxFrameData *ptr) { - QSVMid *qsv_mid = mid; - AVHWFramesContext *hw_frames_ctx = (AVHWFramesContext*)qsv_mid->hw_frames_ref->data; - AVQSVFramesContext *hw_frames_hwctx = hw_frames_ctx->hwctx; + QSVFramesContext *ctx = (QSVFramesContext *)pthis; + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)ctx->hw_frames_ctx->data; + AVQSVFramesContext *frames_hwctx = frames_ctx->hwctx; + QSVMid *qsv_mid; + AVHWFramesContext *hw_frames_ctx; + AVQSVFramesContext *hw_frames_hwctx; int ret; + if (!frames_hwctx->nb_surfaces) + return MFX_ERR_UNSUPPORTED; + + qsv_mid = mid; + hw_frames_ctx = (AVHWFramesContext*)qsv_mid->hw_frames_ref->data; + hw_frames_hwctx = hw_frames_ctx->hwctx; if (qsv_mid->locked_frame) return MFX_ERR_UNDEFINED_BEHAVIOR; @@ -977,8 +1001,15 @@ fail: static mfxStatus qsv_frame_unlock(mfxHDL pthis, mfxMemId mid, mfxFrameData *ptr) { - QSVMid *qsv_mid = mid; + QSVFramesContext *ctx = (QSVFramesContext *)pthis; + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)ctx->hw_frames_ctx->data; + AVQSVFramesContext *frames_hwctx = frames_ctx->hwctx; + QSVMid *qsv_mid; + if (!frames_hwctx->nb_surfaces) + return MFX_ERR_UNSUPPORTED; + + qsv_mid = mid; av_frame_free(&qsv_mid->locked_frame); av_frame_free(&qsv_mid->hw_frame); @@ -987,9 +1018,18 @@ static mfxStatus qsv_frame_unlock(mfxHDL pthis, mfxMemId mid, mfxFrameData *ptr) static mfxStatus qsv_frame_get_hdl(mfxHDL pthis, mfxMemId mid, mfxHDL *hdl) { - QSVMid *qsv_mid = (QSVMid*)mid; + QSVFramesContext *ctx = (QSVFramesContext *)pthis; + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)ctx->hw_frames_ctx->data; + AVQSVFramesContext *frames_hwctx = frames_ctx->hwctx; mfxHDLPair *pair_dst = (mfxHDLPair*)hdl; - mfxHDLPair *pair_src = (mfxHDLPair*)qsv_mid->handle_pair; + mfxHDLPair *pair_src; + + if (frames_hwctx->nb_surfaces) { + QSVMid *qsv_mid = (QSVMid*)mid; + pair_src = (mfxHDLPair*)qsv_mid->handle_pair; + } else { + pair_src = (mfxHDLPair*)mid; + } pair_dst->first = pair_src->first; @@ -1103,14 +1143,18 @@ int ff_qsv_init_session_frames(AVCodecContext *avctx, mfxSession *psession, if (!opaque) { qsv_frames_ctx->logctx = avctx; + av_buffer_unref(&qsv_frames_ctx->mids_buf); + qsv_frames_ctx->mids = NULL; + qsv_frames_ctx->nb_mids = 0; /* allocate the memory ids for the external frames */ - av_buffer_unref(&qsv_frames_ctx->mids_buf); - qsv_frames_ctx->mids_buf = qsv_create_mids(qsv_frames_ctx->hw_frames_ctx); - if (!qsv_frames_ctx->mids_buf) - return AVERROR(ENOMEM); - qsv_frames_ctx->mids = (QSVMid*)qsv_frames_ctx->mids_buf->data; - qsv_frames_ctx->nb_mids = frames_hwctx->nb_surfaces; + if (frames_hwctx->nb_surfaces) { + qsv_frames_ctx->mids_buf = qsv_create_mids(qsv_frames_ctx->hw_frames_ctx); + if (!qsv_frames_ctx->mids_buf) + return AVERROR(ENOMEM); + qsv_frames_ctx->mids = (QSVMid*)qsv_frames_ctx->mids_buf->data; + qsv_frames_ctx->nb_mids = frames_hwctx->nb_surfaces; + } err = MFXVideoCORE_SetFrameAllocator(session, &frame_allocator); if (err != MFX_ERR_NONE) From patchwork Wed Dec 20 07:10:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 45251 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5184:b0:194:e134:edd4 with SMTP id j4csp43842pzf; Tue, 19 Dec 2023 23:12:11 -0800 (PST) X-Google-Smtp-Source: AGHT+IGc5IhhJ8qj1CVZgMlf4HHSMZYkgIbn7I6wvq/UiYBXqJ5PFlUjcEBRtonvzTUGnpVtTxLW X-Received: by 2002:a17:907:d384:b0:a1a:541c:561b with SMTP id vh4-20020a170907d38400b00a1a541c561bmr21661899ejc.6.1703056331594; Tue, 19 Dec 2023 23:12:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703056331; cv=none; d=google.com; s=arc-20160816; b=WGdFemdJKZ4jn02X0gavJ6U9lU+jcmBN2u6gx5Tii0f0ottCglLh8SMhNUYFEtcfRT kX9E7WY5WJaxBlccPMifi1amknb7/vSfFmXisdVb6STEQbBeA4Pvvy39jMiJ70Da6yco rIrtvf/bRRC3mjSIF6DYkMPzoYaAH/8xcSbta+VIPQzLZqd2iAUiKZKORWlXlF/whJMI sqNBSpWNLr9ZfZ+Pacm8bs8gVmjBgoAXxN21XXXxyx6ZDgBIJeWSDJ1n4XHCJEKDrK9z YW16NbeJnF7S0S9W7EqJQppEx4/ohafyL72hlAfPvA5tf9UYkg43mvxQHki7Bbqhp0GE 7+MA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=O7sz0kmkfTCXDVgR2w3VMXv3cIuH7no/Uoq55sfsr0k=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=ht4nsLINd7+SL8V3fDDlR/opdHny1YzSM7hAcS9jfqwCQgjxUiEri+l6ZfNQxs3fXF lM/FfUZnQRyzJBX2UIM8oTcWgTJgxFW2FP2VO/JUVX696dpJWOKAo367eYlGHGLglkoe wdZVKKwjl99kRenDJpxACtYN9CNjpCL9DHkv/g1g0m3chqW4fQxdUcW2ySQh1bQOEAsL OAxDQqwQ0Jw2T+5JFSpQHAY+NXig9KeTfsVpbCHRgPDs0EVJYTsZXSeuEpk6WeaRDyCS 8H/stCtcPSGDQ/E66nHkvRG4fIwAQxUbbA6Oj4zMjLi/IwY8aRUuPFJ9RpDMJbOZNiZu Ew/A== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=G2cxVd4F; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a16-20020a170906369000b00a1d94472b0dsi11784143ejc.169.2023.12.19.23.12.11; Tue, 19 Dec 2023 23:12:11 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=G2cxVd4F; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5A30568D1AE; Wed, 20 Dec 2023 09:11:27 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5DC2C68D18B for ; Wed, 20 Dec 2023 09:11:19 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703056284; x=1734592284; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=u7/p1x9avUVU0e1C+yFAQjjuF/QhbqlNxP45OVHGfjI=; b=G2cxVd4F2eaZWHFf4RpCBygd514dIuqaxbz5/G0mHH2vIz53w5hCwwoI 2i/xMsn2SG/WvXaSOQjXoOsjZYSlNFEw0I8OMODBwoWUugv3bCYIKRs9i e6J5eMG9XgKyoN3QhaWqtAO0+d8yr++T5jGDJT3caiSjhIWSqeJ+2g6FC yCff1PByOQBH66q2NtzqZh45syMaNKtqmVFjv47ZYH3vwrtvq6MfhWuPE X8WbDxLGvKvAiO7ovgZ+DqoKiF3MOlfyxJMqpDnExMgUrjjv4OiAFgkO7 HlzGgjW2B2+3ivh7q/sVais7b/S6MDhXcBLA9NIAwphwKgiNf8WeaexdZ A==; X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="426921040" X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="426921040" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:14 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="18191800" Received: from xhh-dg264.sh.intel.com ([10.238.2.76]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:13 -0800 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 20 Dec 2023 15:10:44 +0800 Message-Id: <20231220071050.3175819-6-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231220071050.3175819-1-haihao.xiang@intel.com> References: <20231220071050.3175819-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 06/12] lavc/qsvenc: use the right info to config encoding parameters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: OhgdvVkpqdOP From: Haihao Xiang Signed-off-by: Haihao Xiang --- libavcodec/qsvenc.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index a0144b0760..0f7e7eb85a 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -718,8 +718,9 @@ static int init_video_param_jpeg(AVCodecContext *avctx, QSVEncContext *q) if (avctx->hw_frames_ctx) { AVHWFramesContext *frames_ctx = (AVHWFramesContext *)avctx->hw_frames_ctx->data; AVQSVFramesContext *frames_hwctx = frames_ctx->hwctx; - q->param.mfx.FrameInfo.Width = frames_hwctx->surfaces[0].Info.Width; - q->param.mfx.FrameInfo.Height = frames_hwctx->surfaces[0].Info.Height; + mfxFrameInfo *info = frames_hwctx->nb_surfaces ? &frames_hwctx->surfaces[0].Info : frames_hwctx->info; + q->param.mfx.FrameInfo.Width = info->Width; + q->param.mfx.FrameInfo.Height = info->Height; } if (avctx->framerate.den > 0 && avctx->framerate.num > 0) { @@ -842,8 +843,9 @@ static int init_video_param(AVCodecContext *avctx, QSVEncContext *q) if (avctx->hw_frames_ctx) { AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; AVQSVFramesContext *frames_hwctx = frames_ctx->hwctx; - q->param.mfx.FrameInfo.Width = frames_hwctx->surfaces[0].Info.Width; - q->param.mfx.FrameInfo.Height = frames_hwctx->surfaces[0].Info.Height; + mfxFrameInfo *info = frames_hwctx->nb_surfaces ? &frames_hwctx->surfaces[0].Info : frames_hwctx->info; + q->param.mfx.FrameInfo.Width = info->Width; + q->param.mfx.FrameInfo.Height = info->Height; } if (avctx->framerate.den > 0 && avctx->framerate.num > 0) { From patchwork Wed Dec 20 07:10:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 45257 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5184:b0:194:e134:edd4 with SMTP id j4csp44158pzf; Tue, 19 Dec 2023 23:13:02 -0800 (PST) X-Google-Smtp-Source: AGHT+IENYGK24i/SRNKjQgI0tzB/uFh/zbepLhSIZPItBXTyxVOx5W+dPgb+dzX1XPCQY6fdoGv3 X-Received: by 2002:a05:651c:504:b0:2cc:76c4:e7ce with SMTP id o4-20020a05651c050400b002cc76c4e7cemr2206855ljp.67.1703056382476; Tue, 19 Dec 2023 23:13:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703056382; cv=none; d=google.com; s=arc-20160816; b=CSZub4NtRROPY4t5ErxB5c2jlab5cd6uNPxDjN4YEcB413AqfAE/jDUdv04pcEHYLi 1l5L5VDr4E5Iuawlte7xz00JjxZaKnEBqwjid1i8wUYDOfQ1ZH5sbhjFSrV9nyrB4LYo jVJeWgqWipBSrlqzc0czvGZrRf7Yan9C7jBMUUr4A1aoi5YvoWWfHgGNWVl4EFD0mHNm h3uUvptr3v70VCoR5BngECXdY23D6UK+4SUJB1RDoZg21IayjnzFGS2e+hEnuPnU+DyA bBzM8o/+uNOp2lpqwBs93ehj1ip0tjJzsG562IwITeAvbbOOKRbyTg/yyX78WzV6vqHe mj6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=GePHf14e/T9F9WQxHvVPhWlPeY+4g6mlhy6/BR6fHEs=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=PW7ly5Aenuwu+aZNxEZCrXaWh+/q5AMzw6e/S1YCwnkKScYfIAqNhmqHrckA/kX6HN IaSpiZjCwNdCXyClJ3peCJFe5a+7K/9SMtwDVgpXixmnldoI/K/McIiJ7xuPPCwz2hQq +4ZcTnkd6L7Dz1qZD2RWU14HIEZxNuCilNW8s0ntQTsB+gIvuP0C5LG9ppP5VlLQbkdy FTTJPXfdFVykIXC859rydq+z1+UbYoMK/3i2QUEppiq3O/jS+Yyf8YkzMep1pfU05qTC +Mqf/IQEiJwcwKfDE/phA6lrRxVNDYyM8Ej7A4/YCqO8KNfS1dMXpY5h8UEExMQhFYbd L1yw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=LMyD5iII; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id cq8-20020a056402220800b00553add301f5si804178edb.145.2023.12.19.23.13.01; Tue, 19 Dec 2023 23:13:02 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=LMyD5iII; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6E91E68D1EB; Wed, 20 Dec 2023 09:11:34 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2F84368D1A8 for ; Wed, 20 Dec 2023 09:11:22 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703056288; x=1734592288; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5cQjLBAlSvLI9IyT6Iof7dgntDZXl8uWjbasQDB4RaQ=; b=LMyD5iII0BZPAPXScElHUShcUQYrjnNneu6C7iPojl6XA9+56YrpsAtZ O1UhVD7ZH17a7QV141ZoxHFnKgNBFFEuDyuUSwUw0XPFj8CB+yQOgWhIk QjELVOszqc5tlOgDAry7nLIeVIYDltzrW/gX3oho8vv5T/IqrCDFSDXUc OXzRaBSkvoBGtOhXVHrgsX/pgFrCG44RKMdWqiZIMiZNPBAxI44nnnxRq JlTssNT0PUCNQVclAgRlHhTuKYyNc6arM8feMFc3n+Bs1UtDAwsVAq+AL NmkgcKSVW3QPBNGyKh+8WiCZrpa1Tc2ySOdFVTG/xFQeWUrSVNJq3xO/L g==; X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="426921044" X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="426921044" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:15 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="18191830" Received: from xhh-dg264.sh.intel.com ([10.238.2.76]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:14 -0800 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 20 Dec 2023 15:10:45 +0800 Message-Id: <20231220071050.3175819-7-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231220071050.3175819-1-haihao.xiang@intel.com> References: <20231220071050.3175819-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 07/12] lavc/qsvdec: require a dynamic frame pool if possible X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: SuOOHYqndyJY From: Haihao Xiang This allows a downstream element stores more frames from qsv decoders and fixes error in get_buffer() $ ffmpeg -hwaccel qsv -hwaccel_output_format qsv -i input.mp4 -vf reverse -f null - [vist#0:0/h264 @ 0x562248f12c50] Decoding error: Cannot allocate memory [h264_qsv @ 0x562248f66b10] get_buffer() failed Signed-off-by: Haihao Xiang --- libavcodec/qsvdec.c | 57 +++++++++++++++++++++++++++++++++++---------- 1 file changed, 45 insertions(+), 12 deletions(-) diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index 559f63698a..3895b3a168 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -42,6 +42,7 @@ #include "libavutil/imgutils.h" #include "libavutil/film_grain_params.h" #include "libavutil/mastering_display_metadata.h" +#include "libavutil/avassert.h" #include "avcodec.h" #include "codec_internal.h" @@ -67,6 +68,8 @@ static const AVRational mfx_tb = { 1, 90000 }; AV_NOPTS_VALUE : pts_tb.num ? \ av_rescale_q(mfx_pts, mfx_tb, pts_tb) : mfx_pts) +#define MFX_IMPL_VIA_MASK(impl) (0x0f00 & (impl)) + typedef struct QSVAsyncFrame { mfxSyncPoint *sync; QSVFrame *frame; @@ -76,6 +79,7 @@ typedef struct QSVContext { // the session used for decoding mfxSession session; mfxVersion ver; + mfxHandleType handle_type; // the session we allocated internally, in case the caller did not provide // one @@ -182,6 +186,7 @@ static int qsv_init_session(AVCodecContext *avctx, QSVContext *q, mfxSession ses AVBufferRef *hw_frames_ref, AVBufferRef *hw_device_ref) { int ret; + mfxIMPL impl; if (q->gpu_copy == MFX_GPUCOPY_ON && !(q->iopattern & MFX_IOPATTERN_OUT_SYSTEM_MEMORY)) { @@ -239,27 +244,52 @@ static int qsv_init_session(AVCodecContext *avctx, QSVContext *q, mfxSession ses q->session = q->internal_qs.session; } - if (MFXQueryVersion(q->session, &q->ver) != MFX_ERR_NONE) { - av_log(avctx, AV_LOG_ERROR, "Error querying the session version. \n"); - q->session = NULL; + if (MFXQueryIMPL(q->session, &impl) == MFX_ERR_NONE) { + switch (MFX_IMPL_VIA_MASK(impl)) { + case MFX_IMPL_VIA_VAAPI: + q->handle_type = MFX_HANDLE_VA_DISPLAY; + break; - if (q->internal_qs.session) { - MFXClose(q->internal_qs.session); - q->internal_qs.session = NULL; - } + case MFX_IMPL_VIA_D3D11: + q->handle_type = MFX_HANDLE_D3D11_DEVICE; + break; + + case MFX_IMPL_VIA_D3D9: + q->handle_type = MFX_HANDLE_D3D9_DEVICE_MANAGER; + break; - if (q->internal_qs.loader) { - MFXUnload(q->internal_qs.loader); - q->internal_qs.loader = NULL; + default: + av_assert0(!"should not reach here"); } + } else { + av_log(avctx, AV_LOG_ERROR, "Error querying the implementation. \n"); + goto fail; + } - return AVERROR_EXTERNAL; + if (MFXQueryVersion(q->session, &q->ver) != MFX_ERR_NONE) { + av_log(avctx, AV_LOG_ERROR, "Error querying the session version. \n"); + goto fail; } /* make sure the decoder is uninitialized */ MFXVideoDECODE_Close(q->session); return 0; + +fail: + q->session = NULL; + + if (q->internal_qs.session) { + MFXClose(q->internal_qs.session); + q->internal_qs.session = NULL; + } + + if (q->internal_qs.loader) { + MFXUnload(q->internal_qs.loader); + q->internal_qs.loader = NULL; + } + + return AVERROR_EXTERNAL; } static int qsv_decode_preinit(AVCodecContext *avctx, QSVContext *q, enum AVPixelFormat pix_fmt, mfxVideoParam *param) @@ -309,7 +339,10 @@ static int qsv_decode_preinit(AVCodecContext *avctx, QSVContext *q, enum AVPixel hwframes_ctx->height = FFALIGN(avctx->coded_height, 32); hwframes_ctx->format = AV_PIX_FMT_QSV; hwframes_ctx->sw_format = avctx->sw_pix_fmt; - hwframes_ctx->initial_pool_size = q->suggest_pool_size + 16 + avctx->extra_hw_frames; + if (QSV_RUNTIME_VERSION_ATLEAST(q->ver, 2, 9) && q->handle_type != MFX_HANDLE_D3D9_DEVICE_MANAGER) + hwframes_ctx->initial_pool_size = 0; + else + hwframes_ctx->initial_pool_size = q->suggest_pool_size + 16 + avctx->extra_hw_frames; frames_hwctx->frame_type = MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET; ret = av_hwframe_ctx_init(avctx->hw_frames_ctx); From patchwork Wed Dec 20 07:10:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 45252 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5184:b0:194:e134:edd4 with SMTP id j4csp43894pzf; Tue, 19 Dec 2023 23:12:20 -0800 (PST) X-Google-Smtp-Source: AGHT+IE8HiRr/fFJuAXJZLpATTTCSXMu7iaZPZxwifisPAwGw9acRvqsEHJOu5gk7AEfPtq83B5g X-Received: by 2002:a05:6402:222b:b0:54f:51cc:6570 with SMTP id cr11-20020a056402222b00b0054f51cc6570mr6613565edb.63.1703056340240; Tue, 19 Dec 2023 23:12:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703056340; cv=none; d=google.com; s=arc-20160816; b=n24m46ufPm2xriZ3KWNBAYX/ko8JXD45wEEr0zNqc6wVe/iuEMNOIFSwpdL7PHYHfQ WWViJAfkl6PM5r9DYBt9rqyN1GhqVonWgCCuFHGSng3Fn/KU51tiScLozNDzxs3Vu45j 1gU+wBzk7JYjnpHyWZl3G82BR4M6IzhZrirfqo/1CKiv2aDwBGDH9wYWp3BJYpwrHpaL 1I7Ily3un8BoHHdbG+/Ectq194rCsI99q4TiOQVeVHuzurseVnWPMW2pt5dNITaxC/qL VKia0P3fa8iFcMyTbwr1I+knYe1TAN9XuP1I1l8naJn1hJs+lrXQWtSRTQk+wtlA6YHm vGbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=3o2pU8FjfnLgRTfo3qcrCWKXMK9HXFBnzU74cJgyt3A=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=fdyIcshflnx5xw9KnhyKx3Y8Ct/OdW8KM67xJPvf6OWiLdej/oOBy0OkYstKHtXJIb A6vtwFYI/Ek5pSiS7tT6UM7AiVOJ7vEVbLSNgNeuUXnu5xBNmcCxp4CgaCnMqXmyrnoJ jDrEvqKBAC38KxgTwDrkw5tEICn7QPhUcgPPSiz1hyc/lxSOnfDMt7Z1H2aXSoRK0nXF 5dfIKsrAyu8CNklR+xyoyLUngsX2tu5jRpT3EVA3NW7kHU9L89C3+c7T3JkzbRQnsBsD oHYeVLM1LSbOJO0nC6ZhAICn34ShW0+Z2vr8ai6oNUoppMEyjvXQ6cc4UgxamCAAG3XA gdSw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=HUwJjZ54; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id l11-20020a056402254b00b0054bf6c62e5asi11589065edb.419.2023.12.19.23.12.19; Tue, 19 Dec 2023 23:12:20 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=HUwJjZ54; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6BB9768D1B7; Wed, 20 Dec 2023 09:11:28 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7582468D0C3 for ; Wed, 20 Dec 2023 09:11:24 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703056284; x=1734592284; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=EjjhjehYIg/paTUgQquVzN6CR+0HvE7iW72CZlbyo9c=; b=HUwJjZ54Hzomw8i2s1mFRF/A+0XjVnfQvobUWjrB+yyDTZ7EhnMqXvrO 0iOLZ6FyRpTYdXsFTq/Q8OxIjWl2f8xf7zS+cZN5gCvYSVIZJf68z6Bl+ MoUa5QgDfhoPcetlATzzrmoMNmAAzOaKUkLdszHVBroWYEJvDRRxAtOQd UdC6LtuJMS0D6orss53hQr0G+U5DJSbW9vWvqf/0HpXcJzQXfLlISHScj bkw7IysJFMkuwKgaHZglaJzJOX8Ik4c1Gj3BDdDcTACstzhZH/uNxLlkO 5U7T6+oY2wb7YiteukMbzebSOO1hK1Gd9BLjb+Q9WC+7XPUFRL7ucgYZz g==; X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="426921048" X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="426921048" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:16 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="18191855" Received: from xhh-dg264.sh.intel.com ([10.238.2.76]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:15 -0800 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 20 Dec 2023 15:10:46 +0800 Message-Id: <20231220071050.3175819-8-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231220071050.3175819-1-haihao.xiang@intel.com> References: <20231220071050.3175819-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 08/12] lavfi/qsvvpp: set right mfxFrameInfo for frames in dynamic frame pools X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 9Dk4hmWdL633 From: Haihao Xiang Signed-off-by: Haihao Xiang --- libavfilter/qsvvpp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavfilter/qsvvpp.c b/libavfilter/qsvvpp.c index 5cdba7d54a..d09140d89b 100644 --- a/libavfilter/qsvvpp.c +++ b/libavfilter/qsvvpp.c @@ -307,7 +307,7 @@ static int fill_frameinfo_by_link(mfxFrameInfo *frameinfo, AVFilterLink *link) frames_ctx = (AVHWFramesContext *)link->hw_frames_ctx->data; frames_hwctx = frames_ctx->hwctx; - *frameinfo = frames_hwctx->surfaces[0].Info; + *frameinfo = frames_hwctx->nb_surfaces ? frames_hwctx->surfaces[0].Info : *frames_hwctx->info; } else { pix_fmt = link->format; desc = av_pix_fmt_desc_get(pix_fmt); From patchwork Wed Dec 20 07:10:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 45253 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5184:b0:194:e134:edd4 with SMTP id j4csp43949pzf; Tue, 19 Dec 2023 23:12:29 -0800 (PST) X-Google-Smtp-Source: AGHT+IGL6pDHONMIpi+sZd3FgZs8s/zmtq09SP43MhHLc2xvGgvh4gZfsGfpBSxvcxZ9qZ4NV7Fo X-Received: by 2002:a17:907:9518:b0:a23:5814:4b76 with SMTP id ew24-20020a170907951800b00a2358144b76mr2145510ejc.35.1703056348664; Tue, 19 Dec 2023 23:12:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703056348; cv=none; d=google.com; s=arc-20160816; b=n7kLHTwOzMi1gS2/QEm/t8GEmYDVnnfkgDSg1ghrvkxXqryaO4LNvALnwtlYmjNmge JDNR+uWYqRawvBxkBAjff1bh4xFUy2tG/OAuRZnJt9RhVCKt4bBh+jutonmz94zkAHLP qIc1ExkRw+a4So/SwKGsVntlAD8QQqNpU8fdMJ3zvny6VMrz1z5c9oLTjoE80aJ+mT05 3+xN++inEOvKNeO5uZpmwfHgaNTH02cHHYR9BOHJac/O74TF52hWzgfN8JOyPuHOVrbs OQFvpKccEsSAmEN6U/egCFgAP3DsY9mYfX9VbeUfihDfXioyDDij2JyDAPOfJWAqOqPr hc6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=WBybTZRFqoLQcv9bLk7lLcjwdmfTcAkhG1AqUnMbOh4=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=GqQkD/4Ug/P4MbGqDUH/Yeam2Ou45GjVJRIXrYC6OovCBYTr1Mw9k2ynvYKQrMaCDp Ke+SUnWJuyWDIIP8KN4qRNQ+x2TQOg5VThuQVgg6gouQuZdvo1QTiOb9CFEE1cj5NZ0P 8IvCJPSRQHeimkoei9t5ZuhiZYIJWK8TLZ9L1e06Rs6bUYES61VVplCN5A/jKRizGBGg 3QLY9gci598cvvTL/tWIsF5JHUgLAt2Vbnz4oIqzmjSSU6ztSqzIZEV75b9YFtmyZ0RK NNQd79s8vyjGMZartfYsm5Oo7xdCEVd/mZq3KETjCiJ2WplLHLd1+jquSLyeGzYe/kf4 9kDA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=ah2VWfMS; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id p27-20020a1709060e9b00b00a26968ceee5si208721ejf.328.2023.12.19.23.12.28; Tue, 19 Dec 2023 23:12:28 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=ah2VWfMS; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6C2C068D1D1; Wed, 20 Dec 2023 09:11:30 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 75C9B68D0C3 for ; Wed, 20 Dec 2023 09:11:25 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703056285; x=1734592285; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2g24ZbPUjpNpUIM43E6+1/ShO9eJe8ljY2VhpL17F+E=; b=ah2VWfMSKxqEV5nUity3KVbzt/lfje5Y/BfoO/jVsOatTGUvBKKEIMue oTHfWtZqi8YExDHJGNk/QmxrDi5sHhiFyqjvHZBu4IM4qCyUhlo0lJitc 97H8kRAmckHBg9qNgKsdKYEhvhDNvKkGHunZInRhTceKNgX5iXSKcjVJH WwLrxm/ISEXpL1JnWyr9YbTCOeaJ5JQ0VkXP+JvXhJwWKSvkQ+CgF0IVy SPCDv5ODui6CC6s546h8qiA7J1r/7gm/IE1mvOBivttPNQPr1s3UBLXFq M24arjY4/BQg9XslvJPjpqFp2ICK9uXg1+jupOPnCmvMPGn/QEzMOum54 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="426921053" X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="426921053" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:17 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="18191882" Received: from xhh-dg264.sh.intel.com ([10.238.2.76]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:17 -0800 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 20 Dec 2023 15:10:47 +0800 Message-Id: <20231220071050.3175819-9-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231220071050.3175819-1-haihao.xiang@intel.com> References: <20231220071050.3175819-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 09/12] lavfi/qsvvpp: require a dynamic frame pool for output if possible X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: bni07MFo3D6j From: Haihao Xiang Signed-off-by: Haihao Xiang --- libavfilter/qsvvpp.c | 52 ++++++++++++++++++++++++-------------------- 1 file changed, 29 insertions(+), 23 deletions(-) diff --git a/libavfilter/qsvvpp.c b/libavfilter/qsvvpp.c index d09140d89b..2c8e73e87d 100644 --- a/libavfilter/qsvvpp.c +++ b/libavfilter/qsvvpp.c @@ -603,6 +603,26 @@ static int init_vpp_session(AVFilterContext *avctx, QSVVPPContext *s) device_ctx = (AVHWDeviceContext *)device_ref->data; device_hwctx = device_ctx->hwctx; + /* extract the properties of the "master" session given to us */ + ret = MFXQueryIMPL(device_hwctx->session, &impl); + if (ret == MFX_ERR_NONE) + ret = MFXQueryVersion(device_hwctx->session, &ver); + if (ret != MFX_ERR_NONE) { + av_log(avctx, AV_LOG_ERROR, "Error querying the session attributes\n"); + return AVERROR_UNKNOWN; + } + + if (MFX_IMPL_VIA_VAAPI == MFX_IMPL_VIA_MASK(impl)) { + handle_type = MFX_HANDLE_VA_DISPLAY; + } else if (MFX_IMPL_VIA_D3D11 == MFX_IMPL_VIA_MASK(impl)) { + handle_type = MFX_HANDLE_D3D11_DEVICE; + } else if (MFX_IMPL_VIA_D3D9 == MFX_IMPL_VIA_MASK(impl)) { + handle_type = MFX_HANDLE_D3D9_DEVICE_MANAGER; + } else { + av_log(avctx, AV_LOG_ERROR, "Error unsupported handle type\n"); + return AVERROR_UNKNOWN; + } + if (outlink->format == AV_PIX_FMT_QSV) { AVHWFramesContext *out_frames_ctx; AVBufferRef *out_frames_ref = av_hwframe_ctx_alloc(device_ref); @@ -624,9 +644,15 @@ static int init_vpp_session(AVFilterContext *avctx, QSVVPPContext *s) out_frames_ctx->width = FFALIGN(outlink->w, 32); out_frames_ctx->height = FFALIGN(outlink->h, 32); out_frames_ctx->sw_format = s->out_sw_format; - out_frames_ctx->initial_pool_size = 64; - if (avctx->extra_hw_frames > 0) - out_frames_ctx->initial_pool_size += avctx->extra_hw_frames; + + if (QSV_RUNTIME_VERSION_ATLEAST(ver, 2, 9) && handle_type != MFX_HANDLE_D3D9_DEVICE_MANAGER) + out_frames_ctx->initial_pool_size = 0; + else { + out_frames_ctx->initial_pool_size = 64; + if (avctx->extra_hw_frames > 0) + out_frames_ctx->initial_pool_size += avctx->extra_hw_frames; + } + out_frames_hwctx->frame_type = s->out_mem_mode; ret = av_hwframe_ctx_init(out_frames_ref); @@ -652,26 +678,6 @@ static int init_vpp_session(AVFilterContext *avctx, QSVVPPContext *s) } else s->out_mem_mode = MFX_MEMTYPE_SYSTEM_MEMORY; - /* extract the properties of the "master" session given to us */ - ret = MFXQueryIMPL(device_hwctx->session, &impl); - if (ret == MFX_ERR_NONE) - ret = MFXQueryVersion(device_hwctx->session, &ver); - if (ret != MFX_ERR_NONE) { - av_log(avctx, AV_LOG_ERROR, "Error querying the session attributes\n"); - return AVERROR_UNKNOWN; - } - - if (MFX_IMPL_VIA_VAAPI == MFX_IMPL_VIA_MASK(impl)) { - handle_type = MFX_HANDLE_VA_DISPLAY; - } else if (MFX_IMPL_VIA_D3D11 == MFX_IMPL_VIA_MASK(impl)) { - handle_type = MFX_HANDLE_D3D11_DEVICE; - } else if (MFX_IMPL_VIA_D3D9 == MFX_IMPL_VIA_MASK(impl)) { - handle_type = MFX_HANDLE_D3D9_DEVICE_MANAGER; - } else { - av_log(avctx, AV_LOG_ERROR, "Error unsupported handle type\n"); - return AVERROR_UNKNOWN; - } - ret = MFXVideoCORE_GetHandle(device_hwctx->session, handle_type, &handle); if (ret < 0) return ff_qsvvpp_print_error(avctx, ret, "Error getting the session handle"); From patchwork Wed Dec 20 07:10:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 45254 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5184:b0:194:e134:edd4 with SMTP id j4csp43997pzf; Tue, 19 Dec 2023 23:12:37 -0800 (PST) X-Google-Smtp-Source: AGHT+IGgjHmFvYsy2v2rV/uOF8jTABzA7dSe7M1epZwvwz2jdS0OgUcv5PZCUfWBavnnFQYXyikP X-Received: by 2002:a50:cdca:0:b0:553:e00f:5604 with SMTP id h10-20020a50cdca000000b00553e00f5604mr451616edj.2.1703056357201; Tue, 19 Dec 2023 23:12:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703056357; cv=none; d=google.com; s=arc-20160816; b=KkhnFWmLy53NDwxVYsSnr0KW6p5J2lu6x+cpZR621U0wDWcIMiop64TYg23ufZWdqT 5HDa1OthH3gHXiFi1DdB6iqrzgS7nUQOXSEBZBgWvPcPKjK/W6e6D5mXVqaxqqX1QKFC BiYDECw0lGRQq91Y+1appG1lM4RtVIacCccmhenYWPeKKAbgZgDCsZwygMvnvf0p9cit agj9f48TmDdIcPJ46Jg74XmonX+dtzNYjZlWZ6nnjFZaWm0m5FqdL9TSstcjTjnYWbWX B+dZZhmoFwoXjsq9xx2FffOlaNmfP+4Ryp/GLa5WQWUfFlqjPqzLGzP+TXM4rZF+qZ7T cgag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=8RJPPj4CKedbbhECLdyv7F8sqWyBLiq5EA9NBpCnwbI=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=LYouWx5HcxZmvAUwMtQpzXRvCrLK1cUXhSNMofG+lqtL44OOK+HG8xk2q564JddTZ4 urYOFINmry6JHEBLg4VZXwukcL31Ly0xo1ThnU8/kqz2yDUKhew+XP7go+/gwrDd6ipH xPawX/OAyt96mi2jYz4YHPCOkyZSgZTqk0pL5k19SCIN9O0qVkOXPs67YmpYaqZiMLY4 tAOad1dIJcUNQSXkbIgOAYLKmUlFKeKG4FKwGsrTQI0DcEDyvs8OEnqdjU/9O25WRX9y WX6jI3GfVDynBo50nuhMLnwuRQDasZeIERiV6daGeSBj7n1tA4+uqTLiMNlE5AZM4J7f qX8w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=gFtUOLjY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id t17-20020a50c251000000b0055223d7990asi6674242edf.218.2023.12.19.23.12.36; Tue, 19 Dec 2023 23:12:37 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=gFtUOLjY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6802C68D19F; Wed, 20 Dec 2023 09:11:31 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 85FB468D18F for ; Wed, 20 Dec 2023 09:11:25 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703056285; x=1734592285; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WPeh8Z/bgpKCIsYR5/LYJFJnm17HJZBtIZlZ8LGmrG8=; b=gFtUOLjYXhXR14OKxQsl7NJK/HHnWterHIh7hHTce2lMwwNGLyP4lhUc pC86ruPRA52QgsLcZC5HZhqgNPX8Yh3ds34U/w99APKozhICrEcqSO84m 2YXVQvA5Kkoq+/TDriraG0QWgoTbkARZkyBveoP+hz0Jc0G11PLcvJa3y VOh7Qcin4XqlJ3gXn9A5lpjUsKfSbHM6rxE6OBw019c47hRFf1PRWv/qq ub8vOgquGKA5SP0NuuaL8exncPuP82+Au/E6FIp3spuL9RZTwceltdBFq cakMy7hrN1VhcYVhyG4V6RlPD2LiuKj1ytD2Sk9iepbLPXS8agJ3pVM1G w==; X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="426921055" X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="426921055" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:19 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="18191906" Received: from xhh-dg264.sh.intel.com ([10.238.2.76]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:18 -0800 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 20 Dec 2023 15:10:48 +0800 Message-Id: <20231220071050.3175819-10-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231220071050.3175819-1-haihao.xiang@intel.com> References: <20231220071050.3175819-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 10/12] lavu/hwcontext_vaapi: relax the requirement when using libva2 (VAAPI 1) X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Al8bSQT28eMx From: Haihao Xiang With libva2, the argument for render target list to vaCreateContext() is a hint, so we may use a dynamic frame pool. Signed-off-by: Haihao Xiang --- doc/APIchanges | 4 ++++ libavutil/hwcontext_vaapi.c | 2 +- libavutil/hwcontext_vaapi.h | 5 +++-- libavutil/version.h | 2 +- 4 files changed, 9 insertions(+), 4 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index b68156ee94..0e36f43a91 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -2,6 +2,10 @@ The last version increases of all libraries were on 2023-02-09 API changes, most recent first: +2023-12-xx - xxxxxxxxxx - lavu 58.37.100 - hwcontext_vaapi.h + Modify the documentation to relax the constraint for dynamic + frames pool + 2023-12-xx - xxxxxxxxxx - lavu 58.36.100 - hwcontext_qsv.h Add AVQSVFramesContext.info diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index 29fc8bd648..628232e2b8 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -618,7 +618,7 @@ static int vaapi_frames_init(AVHWFramesContext *hwfc) } } else { // This pool allows dynamic sizing, and will not be usable as a - // render target. + // render target with libva. It can be used with libva2 avfc->nb_surfaces = 0; avfc->surface_ids = NULL; } diff --git a/libavutil/hwcontext_vaapi.h b/libavutil/hwcontext_vaapi.h index 0b2e071cb3..cff88ad237 100644 --- a/libavutil/hwcontext_vaapi.h +++ b/libavutil/hwcontext_vaapi.h @@ -25,9 +25,10 @@ * @file * API-specific header for AV_HWDEVICE_TYPE_VAAPI. * - * Dynamic frame pools are supported, but note that any pool used as a render + * Dynamic frame pools are supported. Note that any pool used as a render * target is required to be of fixed size in order to be be usable as an - * argument to vaCreateContext(). + * argument to vaCreateContext() when libva is used. When libva2 (VAAPI 1) + * is used, a pool used as a render target can be dynamic. * * For user-allocated pools, AVHWFramesContext.pool must return AVBufferRefs * with the data pointer set to a VASurfaceID. diff --git a/libavutil/version.h b/libavutil/version.h index 3b4c50e9a9..3b38f8f5da 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 58 -#define LIBAVUTIL_VERSION_MINOR 36 +#define LIBAVUTIL_VERSION_MINOR 37 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ From patchwork Wed Dec 20 07:10:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 45255 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5184:b0:194:e134:edd4 with SMTP id j4csp44052pzf; Tue, 19 Dec 2023 23:12:45 -0800 (PST) X-Google-Smtp-Source: AGHT+IEmwu52KMt5KxOr5/C3+fh09wDlxFmoL9icbY7TElNt086eQZs0juNsUaT0gHuYj7cYEIhn X-Received: by 2002:a17:906:18b1:b0:a23:2801:4c54 with SMTP id c17-20020a17090618b100b00a2328014c54mr1416343ejf.298.1703056365400; Tue, 19 Dec 2023 23:12:45 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703056365; cv=none; d=google.com; s=arc-20160816; b=vPrFI9zEaOIinE3DN7BlnE0Lb8EabTJtgDQ3C8bn730UOjuSa13HfqKm/BFq4Z9GVS 2STRxEeBCQfXobZiTam8Hr6DC8Rm6uyCeJegWlUVX9bqaiRRcjFv+XRM0NEmwbdiu4zs DLZFX7d2QUZ/aS94Ghy+CNbLlw9RflTS7NnJVxaiUfMIhFfbDPbRFuer/M0NPFSSznPG cZdwh9aZsq2tAezrxMA1IRVstgDMBb5Uc4jFO9WpAEddjj5/1Uw7Bb/5rd4e1VV6Murq kULAlJICGIoPlIPsVwx3MdokveXev8GoLQKlLCYeKXY4z8+yfuOhiy079K9SwhotrA7j BekQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=ZQOBcItivTFzxa6oqarbqtHgEieAPdyD6npX4CsOwHM=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=gZb+EcevTYEWCJxr5ttJcRjq8LWPB0Nw5aTrfJ7NZPvG84+JzbZCkocFvR6TouFFyQ 5AalYXKsk9A2F8LESereQY4lR9fzVRYVdmYcsOo77O5PMBnEpvjSGbxEXPCyJNYxboQt 3N7Xv2yIHWC3Q9rI5Cqopl0D5ZmbWJRiysD31D+A3LJaOp41drk2bXtn1ViXsnGTyqcR 19y1To79UZWU+uk5kf3zqMtlbPS3SwZgDuDeYjFmeDe1mrUtl+hyRv+0DdNEQDvZaMPL 8rfz5Od0Ny8KlPwbTW2zDRZ17ad811QkqiJBHEb7OCpW5PKaDe0uKS+CizaR/kWvf+hl LahA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=gjxX0RV0; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id r2-20020a170906550200b00a233cdf0627si3140857ejp.527.2023.12.19.23.12.45; Tue, 19 Dec 2023 23:12:45 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=gjxX0RV0; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7FF3C68D1DA; Wed, 20 Dec 2023 09:11:32 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BC66068D1A3 for ; Wed, 20 Dec 2023 09:11:26 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703056286; x=1734592286; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=z5pc/UPRhf5r4nOWIynzt9pzs0wHPvsZ1KOKtOFdI1E=; b=gjxX0RV0JsGztHTK+H3P8slNpFzUQZ7H4zIo7NVpU/SFhvvLpSw3T3SI zZkSOckHepCLpPdphwYE1TiNn52y+gUDwKcRal/DYlfbWYmhK4VYwwKhk aDGdTP+QH8XKG3NH372xmq/HHxCjqS3eVK32+sYMbXLQEewn9JVi563KW eZjQdevGdWF1+NpnB48XBYofgNz8fp8kf/DIqSppcqLozUl+jNyl5w4WV bdcMkg/N+U2QMu0kUETQmgV9WOqSNxJ9WBj0hwQs3aIbwfFfB8jk3F2YQ QGVBOC1qChgmqxbtYktsJSQDuHBOf2G6srfwnljnPvmOD4ufgDA8kSFEf A==; X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="426921057" X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="426921057" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:20 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="18191923" Received: from xhh-dg264.sh.intel.com ([10.238.2.76]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:19 -0800 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 20 Dec 2023 15:10:49 +0800 Message-Id: <20231220071050.3175819-11-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231220071050.3175819-1-haihao.xiang@intel.com> References: <20231220071050.3175819-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 11/12] lavc/vaapi_decode: use dynamic frame pool for output frames with libva2 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 2GBu+bHl/xdJ From: Haihao Xiang This allows a downstream element stores more frames from VAAPI decoders and fixes error in get_buffer() $ ffmpeg -hwaccel vaapi -hwaccel_output_format vaapi -i input_100frames.mp4 \ -vf reverse -an -f null - ... [h264 @ 0x557a075a1400] get_buffer() failed [h264 @ 0x557a075a1400] thread_get_buffer() failed [h264 @ 0x557a075a1400] decode_slice_header error [h264 @ 0x557a075a1400] no frame! Signed-off-by: Haihao Xiang --- libavcodec/vaapi_decode.c | 36 ++++++++++++++++++++---------------- 1 file changed, 20 insertions(+), 16 deletions(-) diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c index ceac769c52..8cc29e96f9 100644 --- a/libavcodec/vaapi_decode.c +++ b/libavcodec/vaapi_decode.c @@ -601,22 +601,26 @@ static int vaapi_decode_make_config(AVCodecContext *avctx, if (err < 0) goto fail; - frames->initial_pool_size = 1; - // Add per-codec number of surfaces used for storing reference frames. - switch (avctx->codec_id) { - case AV_CODEC_ID_H264: - case AV_CODEC_ID_HEVC: - case AV_CODEC_ID_AV1: - frames->initial_pool_size += 16; - break; - case AV_CODEC_ID_VP9: - frames->initial_pool_size += 8; - break; - case AV_CODEC_ID_VP8: - frames->initial_pool_size += 3; - break; - default: - frames->initial_pool_size += 2; + if (CONFIG_VAAPI_1) + frames->initial_pool_size = 0; + else { + frames->initial_pool_size = 1; + // Add per-codec number of surfaces used for storing reference frames. + switch (avctx->codec_id) { + case AV_CODEC_ID_H264: + case AV_CODEC_ID_HEVC: + case AV_CODEC_ID_AV1: + frames->initial_pool_size += 16; + break; + case AV_CODEC_ID_VP9: + frames->initial_pool_size += 8; + break; + case AV_CODEC_ID_VP8: + frames->initial_pool_size += 3; + break; + default: + frames->initial_pool_size += 2; + } } } From patchwork Wed Dec 20 07:10:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 45256 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5184:b0:194:e134:edd4 with SMTP id j4csp44102pzf; Tue, 19 Dec 2023 23:12:54 -0800 (PST) X-Google-Smtp-Source: AGHT+IGCf3vta015S2IenDgSsDgqkvvwq9PV5iky1hLbEAsu22STGdOCm6LGzdDedm+hb1rrrFAv X-Received: by 2002:a17:906:2b56:b0:a25:d0fc:9501 with SMTP id b22-20020a1709062b5600b00a25d0fc9501mr462485ejg.31.1703056373971; Tue, 19 Dec 2023 23:12:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703056373; cv=none; d=google.com; s=arc-20160816; b=zneAVOwE9d+KyyzOC/z/4PL2FDD7KC8TZJ6JX99PIcVyIJ0FgIimUZLzweoxQ3h8Jn NwbJFjs9aGCu7QmSvgxh/vERTBbfZOT/H+8+jet/NS2QYCziVwOgPwX6eh7uxpmxzIAe QuExyExHPiF3QgxKPEl4ojNdVxBy2yURRcK6xcrocfZqVyOHxsIGzV9mytVId1FJJeJY Peq4bhsmsxT0LakzLoNMHP1i0WGjo2vZVmtWomsOShuUPskK5CzXd0rQ47gN5IADwt1n XnXs02oGulTPuUCeHynPYWVHc4DMUcAjtNmIT/lcOG7HRL31TYotGxbryIoRmaPlcE5g PZhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=KE3Rj1Mszv/k8k3mWVJPCIWMGJvzdBrMFarVD8oii+M=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=xNwxp5oOF/o7shlQuUAoAI6sDEw6jWBAC8q4FvQOaUrdXiyxRhgsABmiFXT7IbKWX6 KFdiZsg9TbLDclJNruFXmS75VrsT3+gykhYMrZ9xjIf7kCVJk8OpfwII1wvcJBIQ+5cf hf4/MYf1l2bsHfm9XXoYQun1T/VF40AIIaMDcZO+jDwB70G27CGUp/05xNmTuNskTwQs ohwt7GBOvQ8JLFMLG0MjClHjY/wMKzZtUUWc05u1pZTDlk2f7MuIbJYKxN7+yOBl/R7P kI9pr+ywModj6+99b+HPOPTixHHRNsiunuUVnyds6spMWm+MN6xrJNV/rM/NFEsQ584h sLMw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=PnT0Hk9X; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id w5-20020a170906d20500b00a265d4f7650si949673ejz.736.2023.12.19.23.12.53; Tue, 19 Dec 2023 23:12:53 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=PnT0Hk9X; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7129A68D1DF; Wed, 20 Dec 2023 09:11:33 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BFF9368D1A4 for ; Wed, 20 Dec 2023 09:11:26 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703056287; x=1734592287; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=NCz/L3vPyl/PUHk6L+BF/dU09uLWZJwB8Xaro+H2jNw=; b=PnT0Hk9X0HEBQUSPWg0e3D/pmoDgiIu2LCQyIocmnQ24l8XNDfHVGOhG kn6mllwMn70+DKV3bN+HxKW9kc0BvtIyE6/md105DX/FYSjqE6JUXSEFt qu4K0AkTROSXbRWbAG9boGAMTUbx6AWLcZS20zC4t+3NLB/HMGoqYQHUT Ga9Yf3D57CRx42gimNp7Kbq3dNGGuaxnCako3n/eZFgMCFd4CB40T3Tqu XlJy1AKjvPp1KLBNiK8cVETxwQhbcE2cMo192v2JcP33tQFn/WWkN6nZU g6U1YQFr2i0ZUMDvdBkLEIKbx0HOLm3bPS4txItfL+R8n3/tGSf/3n4hG Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10929"; a="426921060" X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="426921060" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:21 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.04,290,1695711600"; d="scan'208";a="18191953" Received: from xhh-dg264.sh.intel.com ([10.238.2.76]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2023 23:11:20 -0800 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 20 Dec 2023 15:10:50 +0800 Message-Id: <20231220071050.3175819-12-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231220071050.3175819-1-haihao.xiang@intel.com> References: <20231220071050.3175819-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 12/12] lavfi/vaapi_vpp: use dynamic frame pool for output link with libva2 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: XQNyN731nBxR From: Haihao Xiang Signed-off-by: Haihao Xiang --- libavfilter/vaapi_vpp.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/libavfilter/vaapi_vpp.c b/libavfilter/vaapi_vpp.c index cf2592e068..818bc7d58a 100644 --- a/libavfilter/vaapi_vpp.c +++ b/libavfilter/vaapi_vpp.c @@ -199,7 +199,10 @@ int ff_vaapi_vpp_config_output(AVFilterLink *outlink) output_frames->width = ctx->output_width; output_frames->height = ctx->output_height; - output_frames->initial_pool_size = 4; + if (CONFIG_VAAPI_1) + output_frames->initial_pool_size = 0; + else + output_frames->initial_pool_size = 4; err = ff_filter_init_hw_frames(avctx, outlink, 10); if (err < 0)