From patchwork Wed Jul 20 08:40:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 36865 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:1649:b0:8b:613a:194d with SMTP id no9csp2255555pzb; Wed, 20 Jul 2022 01:42:11 -0700 (PDT) X-Google-Smtp-Source: AGRyM1sPxrOiS5NJ2R9HsROD7UCRiURLgP852YTXMbt3q95nMPQwTgqcUWGF8Ca2yMTNBVW4SqD3 X-Received: by 2002:a17:907:7dac:b0:72e:e75e:62d9 with SMTP id oz44-20020a1709077dac00b0072ee75e62d9mr25605413ejc.627.1658306531624; Wed, 20 Jul 2022 01:42:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1658306531; cv=none; d=google.com; s=arc-20160816; b=VIeaBbcC5+xG+5glqh3rj7GoZBvTY3TYmCdonGtgG6OqP3OTDg/6uFNBf+xLxeXryL cq1GbLYlIGcpXnMQSxAIBcB4CyoRCEEzQjeSG7hOjGYG8w42B0nlfXPyniPUoG2Bi81n hWJiNWWcvuz6Vx+yyhM6AtYPZnJgiFwoNHXwJDqsrY2a8RI6DXgpC7bFvrL/XvURXmxR tmMCmL2pthm3DtSF9rpvLjyoWwJ1lXn+/rJVIkB9tYl4zpTjmcGszVo3Q6L4TWBX1Bh3 UCjNf1ygBQoDt3OQgHAr4NMh2bwq+VzCKnZoXhyGT4ywaOdb6la9C8LXrtulCIKD/Jmj a0eA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=dTRbBqmzwyKjIY8K0eeWHGKrt1a3Y83wyNsc0JhltVg=; b=zM/iWLY8YQn+bVfaksXbDfItzxDVkwvDgJECMaMQWNazJEo4SVjs9BYc+TTmC+PJr6 sChZ4cwmXtncvQnTn0SxO4kqBVPr7/5HEP2JGAjDJCrs9Mqr7xHquI3KfMGQi156+bVn UMZ5E1vZfUHkUOZ8ODo2pcp5M/FhGcoK/gPRQK38Ee6N27NqcbojIvHPrNxs9mqSnqat B/3JMsSRtMYlMaJ/m9c5RN24FOoKiWJBwmQpWZSEewYbbUJa93mrlTC3uzXox5aDv4d3 3wYzCtNzZ5kTId4A3iz/e1jHkHFKadgVbsDoRllgSvz70mV7gpdjAfMpGH8Xn1MhuQ1T ON6Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=UxMWssKH; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g13-20020a056402090d00b0043b753b1e70si8679400edz.514.2022.07.20.01.42.11; Wed, 20 Jul 2022 01:42:11 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=UxMWssKH; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5FAC668B823; Wed, 20 Jul 2022 11:40:47 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BE73568B856 for ; Wed, 20 Jul 2022 11:40:42 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1658306443; x=1689842443; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=oSyHAQZC2n3eLSkDGGRrcvnLojmpjfFMPs4NZd15J2Q=; b=UxMWssKH+81VYeahNpO4selBSpRM1e77VsIfDQEx9dGdgtSxbjmRuARw PLL16XZ9ZTXDk+4QKiEb64oCgjG69ag3QbSgaHR/eIRLcZIQjeA3Tkr2Z ibEvf19TWh/yMDmWARB72svTNSOWbz1JdCBrl15jCJ/EpMjO4BkdVAfGj WT5Gv7wqq2EyNhT231NRTd6UmGszdjs9cKBj4MvM39QUAlAUh7WEtcOif CbfvpM3Hr9FHubkkWKvA/o7U1fL09AdFtwP0KItoRUASS+A0uICWXNXkb gvQt3fv+NbwEsjJkw2D5vQslT2PtWDJI/ieHN38XdAeZNqi5g9MB7ST4E Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10413"; a="348412558" X-IronPort-AV: E=Sophos;i="5.92,286,1650956400"; d="scan'208";a="348412558" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Jul 2022 01:40:41 -0700 X-IronPort-AV: E=Sophos;i="5.92,286,1650956400"; d="scan'208";a="625574717" Received: from xhh-dg164.sh.intel.com ([10.238.5.169]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Jul 2022 01:40:39 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 20 Jul 2022 16:40:04 +0800 Message-Id: <20220720084006.5033-12-haihao.xiang@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220720084006.5033-1-haihao.xiang@intel.com> References: <20220720084006.5033-1-haihao.xiang@intel.com> Subject: [FFmpeg-devel] [PATCH v11 11/13] lavc/qsv: create mfx session using oneVPL for decoding/encoding X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: h59itQvFBHfA From: Haihao Xiang If qsv hwdevice is available, use the mfxLoader handle in qsv hwdevice to create mfx session. Otherwise create mfx session with a new mfxLoader handle. This is in preparation for oneVPL support --- libavcodec/qsv.c | 226 +++++++++++++++++++++++++++++++++++--- libavcodec/qsv_internal.h | 1 + libavcodec/qsvdec.c | 11 ++ libavcodec/qsvenc.h | 3 + libavcodec/qsvenc_h264.c | 1 - libavcodec/qsvenc_hevc.c | 1 - libavcodec/qsvenc_jpeg.c | 1 - libavcodec/qsvenc_mpeg2.c | 1 - libavcodec/qsvenc_vp9.c | 1 - 9 files changed, 223 insertions(+), 23 deletions(-) diff --git a/libavcodec/qsv.c b/libavcodec/qsv.c index 432675bccf..fe998c9649 100644 --- a/libavcodec/qsv.c +++ b/libavcodec/qsv.c @@ -45,6 +45,12 @@ #include #endif +#if QSV_ONEVPL +#include +#else +#define MFXUnload(a) do { } while(0) +#endif + int ff_qsv_codec_id_to_mfx(enum AVCodecID codec_id) { switch (codec_id) { @@ -419,6 +425,193 @@ static int ff_qsv_set_display_handle(AVCodecContext *avctx, QSVSession *qs) } #endif //AVCODEC_QSV_LINUX_SESSION_HANDLE +#if QSV_ONEVPL +static int qsv_new_mfx_loader(AVCodecContext *avctx, + mfxIMPL implementation, + mfxVersion *pver, + void **ploader) +{ + mfxStatus sts; + mfxLoader loader = NULL; + mfxConfig cfg; + mfxVariant impl_value; + + loader = MFXLoad(); + if (!loader) { + av_log(avctx, AV_LOG_ERROR, "Error creating a MFX loader\n"); + goto fail; + } + + /* Create configurations for implementation */ + cfg = MFXCreateConfig(loader); + if (!cfg) { + av_log(avctx, AV_LOG_ERROR, "Error creating a MFX configurations\n"); + goto fail; + } + + impl_value.Type = MFX_VARIANT_TYPE_U32; + impl_value.Data.U32 = (implementation == MFX_IMPL_SOFTWARE) ? + MFX_IMPL_TYPE_SOFTWARE : MFX_IMPL_TYPE_HARDWARE; + sts = MFXSetConfigFilterProperty(cfg, + (const mfxU8 *)"mfxImplDescription.Impl", impl_value); + if (sts != MFX_ERR_NONE) { + av_log(avctx, AV_LOG_ERROR, "Error adding a MFX configuration " + "property: %d\n", sts); + goto fail; + } + + impl_value.Type = MFX_VARIANT_TYPE_U32; + impl_value.Data.U32 = pver->Version; + sts = MFXSetConfigFilterProperty(cfg, + (const mfxU8 *)"mfxImplDescription.ApiVersion.Version", + impl_value); + if (sts != MFX_ERR_NONE) { + av_log(avctx, AV_LOG_ERROR, "Error adding a MFX configuration " + "property: %d\n", sts); + goto fail; + } + + *ploader = loader; + + return 0; + +fail: + if (loader) + MFXUnload(loader); + + *ploader = NULL; + return AVERROR_UNKNOWN; +} + +static int qsv_create_mfx_session_from_loader(void *ctx, mfxLoader loader, mfxSession *psession) +{ + mfxStatus sts; + mfxSession session = NULL; + uint32_t impl_idx = 0; + + while (1) { + /* Enumerate all implementations */ + mfxImplDescription *impl_desc; + + sts = MFXEnumImplementations(loader, impl_idx, + MFX_IMPLCAPS_IMPLDESCSTRUCTURE, + (mfxHDL *)&impl_desc); + /* Failed to find an available implementation */ + if (sts == MFX_ERR_NOT_FOUND) + break; + else if (sts != MFX_ERR_NONE) { + impl_idx++; + continue; + } + + sts = MFXCreateSession(loader, impl_idx, &session); + MFXDispReleaseImplDescription(loader, impl_desc); + if (sts == MFX_ERR_NONE) + break; + + impl_idx++; + } + + if (sts != MFX_ERR_NONE) { + av_log(ctx, AV_LOG_ERROR, "Error creating a MFX session: %d.\n", sts); + goto fail; + } + + *psession = session; + + return 0; + +fail: + if (session) + MFXClose(session); + + *psession = NULL; + return AVERROR_UNKNOWN; +} + +static int qsv_create_mfx_session(AVCodecContext *avctx, + mfxIMPL implementation, + mfxVersion *pver, + int gpu_copy, + mfxSession *psession, + void **ploader) +{ + mfxLoader loader = NULL; + + /* Don't create a new MFX loader if the input loader is valid */ + if (*ploader == NULL) { + av_log(avctx, AV_LOG_VERBOSE, + "Use Intel(R) oneVPL to create MFX session, the required " + "implementation version is %d.%d\n", + pver->Major, pver->Minor); + + if (qsv_new_mfx_loader(avctx, implementation, pver, (void **)&loader)) + goto fail; + + av_assert0(loader); + } else { + av_log(avctx, AV_LOG_VERBOSE, + "Use Intel(R) oneVPL to create MFX session with the specified MFX loader\n"); + + loader = *ploader; + } + + if (qsv_create_mfx_session_from_loader(avctx, loader, psession)) + goto fail; + + if (!*ploader) + *ploader = loader; + + return 0; + +fail: + if (!*ploader && loader) + MFXUnload(loader); + + return AVERROR_UNKNOWN; +} + +#else + +static int qsv_create_mfx_session(AVCodecContext *avctx, + mfxIMPL implementation, + mfxVersion *pver, + int gpu_copy, + mfxSession *psession, + void **ploader) +{ + mfxInitParam init_par = { MFX_IMPL_AUTO_ANY }; + mfxSession session = NULL; + mfxStatus sts; + + av_log(avctx, AV_LOG_VERBOSE, + "Use Intel(R) Media SDK to create MFX session, the required " + "implementation version is %d.%d\n", + pver->Major, pver->Minor); + + *psession = NULL; + *ploader = NULL; + + init_par.GPUCopy = gpu_copy; + init_par.Implementation = implementation; + init_par.Version = *pver; + sts = MFXInitEx(init_par, &session); + if (sts < 0) + return ff_qsv_print_error(avctx, sts, + "Error initializing a MFX session"); + else if (sts > 0) { + ff_qsv_print_warning(avctx, sts, + "Warning in MFX initialization"); + return AVERROR_UNKNOWN; + } + + *psession = session; + + return 0; +} + +#endif + int ff_qsv_init_internal_session(AVCodecContext *avctx, QSVSession *qs, const char *load_plugins, int gpu_copy) { @@ -428,18 +621,12 @@ int ff_qsv_init_internal_session(AVCodecContext *avctx, QSVSession *qs, mfxIMPL impl = MFX_IMPL_AUTO_ANY; #endif mfxVersion ver = { { QSV_VERSION_MINOR, QSV_VERSION_MAJOR } }; - mfxInitParam init_par = { MFX_IMPL_AUTO_ANY }; const char *desc; - int ret; - - init_par.GPUCopy = gpu_copy; - init_par.Implementation = impl; - init_par.Version = ver; - ret = MFXInitEx(init_par, &qs->session); - if (ret < 0) - return ff_qsv_print_error(avctx, ret, - "Error initializing an internal MFX session"); + int ret = qsv_create_mfx_session(avctx, impl, &ver, gpu_copy, &qs->session, + &qs->loader); + if (ret) + return ret; #ifdef AVCODEC_QSV_LINUX_SESSION_HANDLE ret = ff_qsv_set_display_handle(avctx, qs); @@ -743,7 +930,7 @@ int ff_qsv_init_session_device(AVCodecContext *avctx, mfxSession *psession, AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)device_ref->data; AVQSVDeviceContext *device_hwctx = device_ctx->hwctx; mfxSession parent_session = device_hwctx->session; - mfxInitParam init_par = { MFX_IMPL_AUTO_ANY }; + void *loader = device_hwctx->loader; mfxHDL handle = NULL; int hw_handle_supported = 0; @@ -784,13 +971,10 @@ int ff_qsv_init_session_device(AVCodecContext *avctx, mfxSession *psession, "from the session\n"); } - init_par.GPUCopy = gpu_copy; - init_par.Implementation = impl; - init_par.Version = ver; - err = MFXInitEx(init_par, &session); - if (err != MFX_ERR_NONE) - return ff_qsv_print_error(avctx, err, - "Error initializing a child MFX session"); + ret = qsv_create_mfx_session(avctx, impl, &ver, gpu_copy, &session, + &loader); + if (ret) + return ret; if (handle) { err = MFXVideoCORE_SetHandle(session, handle_type, handle); @@ -869,6 +1053,12 @@ int ff_qsv_close_internal_session(QSVSession *qs) MFXClose(qs->session); qs->session = NULL; } + + if (qs->loader) { + MFXUnload(qs->loader); + qs->loader = NULL; + } + #ifdef AVCODEC_QSV_LINUX_SESSION_HANDLE av_buffer_unref(&qs->va_device_ref); #endif diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h index d52f0dcd15..f914956596 100644 --- a/libavcodec/qsv_internal.h +++ b/libavcodec/qsv_internal.h @@ -103,6 +103,7 @@ typedef struct QSVSession { AVBufferRef *va_device_ref; AVHWDeviceContext *va_device_ctx; #endif + void *loader; } QSVSession; typedef struct QSVFramesContext { diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index 8b718a08ff..89ec5dcee8 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -50,6 +50,12 @@ #include "qsv.h" #include "qsv_internal.h" +#if QSV_ONEVPL +#include +#else +#define MFXUnload(a) do { } while(0) +#endif + static const AVRational mfx_tb = { 1, 90000 }; #define PTS_TO_MFX_PTS(pts, pts_tb) ((pts) == AV_NOPTS_VALUE ? \ @@ -231,6 +237,11 @@ static int qsv_init_session(AVCodecContext *avctx, QSVContext *q, mfxSession ses q->internal_qs.session = NULL; } + if (q->internal_qs.loader) { + MFXUnload(q->internal_qs.loader); + q->internal_qs.loader = NULL; + } + return AVERROR_EXTERNAL; } diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h index c2def19658..d35a1318f2 100644 --- a/libavcodec/qsvenc.h +++ b/libavcodec/qsvenc.h @@ -28,6 +28,9 @@ #include +#include "libavutil/common.h" +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_qsv.h" #include "libavutil/avutil.h" #include "libavutil/fifo.h" diff --git a/libavcodec/qsvenc_h264.c b/libavcodec/qsvenc_h264.c index 64c22852d2..ad9cf629df 100644 --- a/libavcodec/qsvenc_h264.c +++ b/libavcodec/qsvenc_h264.c @@ -32,7 +32,6 @@ #include "avcodec.h" #include "codec_internal.h" #include "qsv.h" -#include "qsv_internal.h" #include "qsvenc.h" #include "atsc_a53.h" diff --git a/libavcodec/qsvenc_hevc.c b/libavcodec/qsvenc_hevc.c index 1d92b8ec13..e11f5dec4a 100644 --- a/libavcodec/qsvenc_hevc.c +++ b/libavcodec/qsvenc_hevc.c @@ -35,7 +35,6 @@ #include "hevcdec.h" #include "h2645_parse.h" #include "qsv.h" -#include "qsv_internal.h" #include "qsvenc.h" enum LoadPlugin { diff --git a/libavcodec/qsvenc_jpeg.c b/libavcodec/qsvenc_jpeg.c index 582886c7c0..6d17d896eb 100644 --- a/libavcodec/qsvenc_jpeg.c +++ b/libavcodec/qsvenc_jpeg.c @@ -30,7 +30,6 @@ #include "avcodec.h" #include "codec_internal.h" #include "qsv.h" -#include "qsv_internal.h" #include "qsvenc.h" typedef struct QSVMJPEGEncContext { diff --git a/libavcodec/qsvenc_mpeg2.c b/libavcodec/qsvenc_mpeg2.c index 8ed578c396..5666b3e6e2 100644 --- a/libavcodec/qsvenc_mpeg2.c +++ b/libavcodec/qsvenc_mpeg2.c @@ -30,7 +30,6 @@ #include "avcodec.h" #include "codec_internal.h" #include "qsv.h" -#include "qsv_internal.h" #include "qsvenc.h" typedef struct QSVMpeg2EncContext { diff --git a/libavcodec/qsvenc_vp9.c b/libavcodec/qsvenc_vp9.c index 7ae392da75..c738da3904 100644 --- a/libavcodec/qsvenc_vp9.c +++ b/libavcodec/qsvenc_vp9.c @@ -30,7 +30,6 @@ #include "avcodec.h" #include "codec_internal.h" #include "qsv.h" -#include "qsv_internal.h" #include "qsvenc.h" typedef struct QSVVP9EncContext {