From patchwork Tue Jul 12 06:27:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 36749 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:6da0:b0:8b:e47:9dbf with SMTP id wl32csp1812003pzb; Mon, 11 Jul 2022 23:30:10 -0700 (PDT) X-Google-Smtp-Source: AGRyM1u4dLm3hpXwnqOovvnQzJnWIB05/cXX+MDDlGIjxltDUcwsUSkWipmkkemUX+ZOy+uLgsDZ X-Received: by 2002:aa7:db9a:0:b0:43a:76bf:5401 with SMTP id u26-20020aa7db9a000000b0043a76bf5401mr29344829edt.244.1657607410454; Mon, 11 Jul 2022 23:30:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1657607410; cv=none; d=google.com; s=arc-20160816; b=Ej3dvd6Przt2P7gm7yvwxWGVNLaSNOgTDpyRFFLwDzNHcBvVRKUkwJaTrsdvW0fJdS pZSHjZGRrqGOnRcPD9s5YWowQuUSpBIAzUX2vCM5YKP+eWhV7eM5WhA8uzaq3Wg2N83a qWmAiy3rsrAuSf3sK8cya08/MuyNl6wwcrzadneoeK281Boy2xxQpJdJzNw0lcsSynW1 x2lrYNuqOMpbW+9msDVX1R8OqG6QDG+B+AMCLcEOuiUyJ4pcQ6QBy/X4rOMhNuM5FdRU omMj7PjCEB/ojKt4DJ1NhoHZ+5PnZcl1vWw3s/HNxDOc+/TtnUjwkGT+eE5q6KG34qu0 DYKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=rMxxovo2Zu1di2MFj1204APzHtOpqHefu9TIhhUzp6A=; b=BClh8qkhyJRFbPntJJEcWPZc6PvbOpY9/disJI3aLcVZxHOho3pLKrSkzlvYxW+zPw f49MCUBb9jHN1kgbDiACMIzIqVvkQcfWY6obsG9EQ8hrYsaZ1ZvcatBOCn/2ixLkzjuj /YkrVHm6aSZcN6ZGG08cYeksXGCGwZwhMAYAOj2+XKpbZl5s6HpsMOnZzNqenq2ssaU8 C0lOrThd2aEliVWGJFAu8Caj3S/hDUVVvgRjPf7zUPbFPhBVP8OEwcC1OYF2/nwHzAPE yaAFE2bmZjhE/VDw0KWLuQrfTwa55B8xuNScZQ7WTYfea4O1j7wZsMWSPGSN9cEZxDac 0jjw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b="NgCP/pD1"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y8-20020a50e608000000b0043a7e704d2asi12523132edm.365.2022.07.11.23.30.10; Mon, 11 Jul 2022 23:30:10 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b="NgCP/pD1"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B6E3D68B8F0; Tue, 12 Jul 2022 09:28:50 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8C03B68B84A for ; Tue, 12 Jul 2022 09:28:41 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657607322; x=1689143322; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=kpqO4jhrOTZZfRSB7oGqNCEO/XOAalym9oERWnoBVC4=; b=NgCP/pD1cJJGPgmPcHd3xHwJkdK/Q06D4UKFMK9XQ5KLUkUBPWFendFP lb9/Zie+FTZeDYvavuSx7jQKVfwq3DcrqCB+hJ5MWmiZJoLEq0PULwJN2 OzO2iiL5Snxf1cpR9lKw+XEHSDvRCzexSvlelx188iGvlKrdhAqGfRoEz ViUKEho0YOBxNXf7J4F4hvAv8h7mfgBy2CrSjz2MrBueVzoffT27y5Pd9 /hrCLgxMUyYIIJ+Jkgy+j34TNp11rwW/FUZpWH+fW9GZ6CjoE7Ln5AWj+ Ut69p3dDhuT9nYB9zVUxmV3i/ceHWQFiAlpHrvOSJZiTM8oAUDbo6i0EI g==; X-IronPort-AV: E=McAfee;i="6400,9594,10405"; a="346534454" X-IronPort-AV: E=Sophos;i="5.92,265,1650956400"; d="scan'208";a="346534454" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jul 2022 23:28:21 -0700 X-IronPort-AV: E=Sophos;i="5.92,265,1650956400"; d="scan'208";a="622382056" Received: from xhh-dg164.sh.intel.com ([10.239.159.146]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jul 2022 23:28:19 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Tue, 12 Jul 2022 14:27:33 +0800 Message-Id: <20220712062735.20339-12-haihao.xiang@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220712062735.20339-1-haihao.xiang@intel.com> References: <20220712062735.20339-1-haihao.xiang@intel.com> Subject: [FFmpeg-devel] [PATCH v10 11/13] lavc/qsv: create mfx session using oneVPL for decoding/encoding X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: g/7MSyD+hmQV From: Haihao Xiang If qsv hwdevice is available, use the mfxLoader handle in qsv hwdevice to create mfx session. Otherwise create mfx session with a new mfxLoader handle. This is in preparation for oneVPL support --- libavcodec/qsv.c | 226 +++++++++++++++++++++++++++++++++++--- libavcodec/qsv_internal.h | 1 + libavcodec/qsvdec.c | 11 ++ libavcodec/qsvenc.h | 3 + libavcodec/qsvenc_h264.c | 1 - libavcodec/qsvenc_hevc.c | 1 - libavcodec/qsvenc_jpeg.c | 1 - libavcodec/qsvenc_mpeg2.c | 1 - libavcodec/qsvenc_vp9.c | 1 - 9 files changed, 223 insertions(+), 23 deletions(-) diff --git a/libavcodec/qsv.c b/libavcodec/qsv.c index 432675bccf..fe998c9649 100644 --- a/libavcodec/qsv.c +++ b/libavcodec/qsv.c @@ -45,6 +45,12 @@ #include #endif +#if QSV_ONEVPL +#include +#else +#define MFXUnload(a) do { } while(0) +#endif + int ff_qsv_codec_id_to_mfx(enum AVCodecID codec_id) { switch (codec_id) { @@ -419,6 +425,193 @@ static int ff_qsv_set_display_handle(AVCodecContext *avctx, QSVSession *qs) } #endif //AVCODEC_QSV_LINUX_SESSION_HANDLE +#if QSV_ONEVPL +static int qsv_new_mfx_loader(AVCodecContext *avctx, + mfxIMPL implementation, + mfxVersion *pver, + void **ploader) +{ + mfxStatus sts; + mfxLoader loader = NULL; + mfxConfig cfg; + mfxVariant impl_value; + + loader = MFXLoad(); + if (!loader) { + av_log(avctx, AV_LOG_ERROR, "Error creating a MFX loader\n"); + goto fail; + } + + /* Create configurations for implementation */ + cfg = MFXCreateConfig(loader); + if (!cfg) { + av_log(avctx, AV_LOG_ERROR, "Error creating a MFX configurations\n"); + goto fail; + } + + impl_value.Type = MFX_VARIANT_TYPE_U32; + impl_value.Data.U32 = (implementation == MFX_IMPL_SOFTWARE) ? + MFX_IMPL_TYPE_SOFTWARE : MFX_IMPL_TYPE_HARDWARE; + sts = MFXSetConfigFilterProperty(cfg, + (const mfxU8 *)"mfxImplDescription.Impl", impl_value); + if (sts != MFX_ERR_NONE) { + av_log(avctx, AV_LOG_ERROR, "Error adding a MFX configuration " + "property: %d\n", sts); + goto fail; + } + + impl_value.Type = MFX_VARIANT_TYPE_U32; + impl_value.Data.U32 = pver->Version; + sts = MFXSetConfigFilterProperty(cfg, + (const mfxU8 *)"mfxImplDescription.ApiVersion.Version", + impl_value); + if (sts != MFX_ERR_NONE) { + av_log(avctx, AV_LOG_ERROR, "Error adding a MFX configuration " + "property: %d\n", sts); + goto fail; + } + + *ploader = loader; + + return 0; + +fail: + if (loader) + MFXUnload(loader); + + *ploader = NULL; + return AVERROR_UNKNOWN; +} + +static int qsv_create_mfx_session_from_loader(void *ctx, mfxLoader loader, mfxSession *psession) +{ + mfxStatus sts; + mfxSession session = NULL; + uint32_t impl_idx = 0; + + while (1) { + /* Enumerate all implementations */ + mfxImplDescription *impl_desc; + + sts = MFXEnumImplementations(loader, impl_idx, + MFX_IMPLCAPS_IMPLDESCSTRUCTURE, + (mfxHDL *)&impl_desc); + /* Failed to find an available implementation */ + if (sts == MFX_ERR_NOT_FOUND) + break; + else if (sts != MFX_ERR_NONE) { + impl_idx++; + continue; + } + + sts = MFXCreateSession(loader, impl_idx, &session); + MFXDispReleaseImplDescription(loader, impl_desc); + if (sts == MFX_ERR_NONE) + break; + + impl_idx++; + } + + if (sts != MFX_ERR_NONE) { + av_log(ctx, AV_LOG_ERROR, "Error creating a MFX session: %d.\n", sts); + goto fail; + } + + *psession = session; + + return 0; + +fail: + if (session) + MFXClose(session); + + *psession = NULL; + return AVERROR_UNKNOWN; +} + +static int qsv_create_mfx_session(AVCodecContext *avctx, + mfxIMPL implementation, + mfxVersion *pver, + int gpu_copy, + mfxSession *psession, + void **ploader) +{ + mfxLoader loader = NULL; + + /* Don't create a new MFX loader if the input loader is valid */ + if (*ploader == NULL) { + av_log(avctx, AV_LOG_VERBOSE, + "Use Intel(R) oneVPL to create MFX session, the required " + "implementation version is %d.%d\n", + pver->Major, pver->Minor); + + if (qsv_new_mfx_loader(avctx, implementation, pver, (void **)&loader)) + goto fail; + + av_assert0(loader); + } else { + av_log(avctx, AV_LOG_VERBOSE, + "Use Intel(R) oneVPL to create MFX session with the specified MFX loader\n"); + + loader = *ploader; + } + + if (qsv_create_mfx_session_from_loader(avctx, loader, psession)) + goto fail; + + if (!*ploader) + *ploader = loader; + + return 0; + +fail: + if (!*ploader && loader) + MFXUnload(loader); + + return AVERROR_UNKNOWN; +} + +#else + +static int qsv_create_mfx_session(AVCodecContext *avctx, + mfxIMPL implementation, + mfxVersion *pver, + int gpu_copy, + mfxSession *psession, + void **ploader) +{ + mfxInitParam init_par = { MFX_IMPL_AUTO_ANY }; + mfxSession session = NULL; + mfxStatus sts; + + av_log(avctx, AV_LOG_VERBOSE, + "Use Intel(R) Media SDK to create MFX session, the required " + "implementation version is %d.%d\n", + pver->Major, pver->Minor); + + *psession = NULL; + *ploader = NULL; + + init_par.GPUCopy = gpu_copy; + init_par.Implementation = implementation; + init_par.Version = *pver; + sts = MFXInitEx(init_par, &session); + if (sts < 0) + return ff_qsv_print_error(avctx, sts, + "Error initializing a MFX session"); + else if (sts > 0) { + ff_qsv_print_warning(avctx, sts, + "Warning in MFX initialization"); + return AVERROR_UNKNOWN; + } + + *psession = session; + + return 0; +} + +#endif + int ff_qsv_init_internal_session(AVCodecContext *avctx, QSVSession *qs, const char *load_plugins, int gpu_copy) { @@ -428,18 +621,12 @@ int ff_qsv_init_internal_session(AVCodecContext *avctx, QSVSession *qs, mfxIMPL impl = MFX_IMPL_AUTO_ANY; #endif mfxVersion ver = { { QSV_VERSION_MINOR, QSV_VERSION_MAJOR } }; - mfxInitParam init_par = { MFX_IMPL_AUTO_ANY }; const char *desc; - int ret; - - init_par.GPUCopy = gpu_copy; - init_par.Implementation = impl; - init_par.Version = ver; - ret = MFXInitEx(init_par, &qs->session); - if (ret < 0) - return ff_qsv_print_error(avctx, ret, - "Error initializing an internal MFX session"); + int ret = qsv_create_mfx_session(avctx, impl, &ver, gpu_copy, &qs->session, + &qs->loader); + if (ret) + return ret; #ifdef AVCODEC_QSV_LINUX_SESSION_HANDLE ret = ff_qsv_set_display_handle(avctx, qs); @@ -743,7 +930,7 @@ int ff_qsv_init_session_device(AVCodecContext *avctx, mfxSession *psession, AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)device_ref->data; AVQSVDeviceContext *device_hwctx = device_ctx->hwctx; mfxSession parent_session = device_hwctx->session; - mfxInitParam init_par = { MFX_IMPL_AUTO_ANY }; + void *loader = device_hwctx->loader; mfxHDL handle = NULL; int hw_handle_supported = 0; @@ -784,13 +971,10 @@ int ff_qsv_init_session_device(AVCodecContext *avctx, mfxSession *psession, "from the session\n"); } - init_par.GPUCopy = gpu_copy; - init_par.Implementation = impl; - init_par.Version = ver; - err = MFXInitEx(init_par, &session); - if (err != MFX_ERR_NONE) - return ff_qsv_print_error(avctx, err, - "Error initializing a child MFX session"); + ret = qsv_create_mfx_session(avctx, impl, &ver, gpu_copy, &session, + &loader); + if (ret) + return ret; if (handle) { err = MFXVideoCORE_SetHandle(session, handle_type, handle); @@ -869,6 +1053,12 @@ int ff_qsv_close_internal_session(QSVSession *qs) MFXClose(qs->session); qs->session = NULL; } + + if (qs->loader) { + MFXUnload(qs->loader); + qs->loader = NULL; + } + #ifdef AVCODEC_QSV_LINUX_SESSION_HANDLE av_buffer_unref(&qs->va_device_ref); #endif diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h index d52f0dcd15..f914956596 100644 --- a/libavcodec/qsv_internal.h +++ b/libavcodec/qsv_internal.h @@ -103,6 +103,7 @@ typedef struct QSVSession { AVBufferRef *va_device_ref; AVHWDeviceContext *va_device_ctx; #endif + void *loader; } QSVSession; typedef struct QSVFramesContext { diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index cd0a0f9cc1..bce7c903b7 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -50,6 +50,12 @@ #include "qsv.h" #include "qsv_internal.h" +#if QSV_ONEVPL +#include +#else +#define MFXUnload(a) do { } while(0) +#endif + static const AVRational mfx_tb = { 1, 90000 }; #define PTS_TO_MFX_PTS(pts, pts_tb) ((pts) == AV_NOPTS_VALUE ? \ @@ -231,6 +237,11 @@ static int qsv_init_session(AVCodecContext *avctx, QSVContext *q, mfxSession ses q->internal_qs.session = NULL; } + if (q->internal_qs.loader) { + MFXUnload(q->internal_qs.loader); + q->internal_qs.loader = NULL; + } + return AVERROR_EXTERNAL; } diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h index 2d7a4fff89..b36b89499e 100644 --- a/libavcodec/qsvenc.h +++ b/libavcodec/qsvenc.h @@ -28,6 +28,9 @@ #include +#include "libavutil/common.h" +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_qsv.h" #include "libavutil/avutil.h" #include "libavutil/fifo.h" diff --git a/libavcodec/qsvenc_h264.c b/libavcodec/qsvenc_h264.c index bc28589d10..53d754a473 100644 --- a/libavcodec/qsvenc_h264.c +++ b/libavcodec/qsvenc_h264.c @@ -32,7 +32,6 @@ #include "avcodec.h" #include "codec_internal.h" #include "qsv.h" -#include "qsv_internal.h" #include "qsvenc.h" #include "atsc_a53.h" diff --git a/libavcodec/qsvenc_hevc.c b/libavcodec/qsvenc_hevc.c index 6df4a1a136..b1a82104ff 100644 --- a/libavcodec/qsvenc_hevc.c +++ b/libavcodec/qsvenc_hevc.c @@ -35,7 +35,6 @@ #include "hevcdec.h" #include "h2645_parse.h" #include "qsv.h" -#include "qsv_internal.h" #include "qsvenc.h" enum LoadPlugin { diff --git a/libavcodec/qsvenc_jpeg.c b/libavcodec/qsvenc_jpeg.c index 030b358d75..ca56989d23 100644 --- a/libavcodec/qsvenc_jpeg.c +++ b/libavcodec/qsvenc_jpeg.c @@ -30,7 +30,6 @@ #include "avcodec.h" #include "codec_internal.h" #include "qsv.h" -#include "qsv_internal.h" #include "qsvenc.h" typedef struct QSVMJPEGEncContext { diff --git a/libavcodec/qsvenc_mpeg2.c b/libavcodec/qsvenc_mpeg2.c index c68bf64231..6b7677148c 100644 --- a/libavcodec/qsvenc_mpeg2.c +++ b/libavcodec/qsvenc_mpeg2.c @@ -30,7 +30,6 @@ #include "avcodec.h" #include "codec_internal.h" #include "qsv.h" -#include "qsv_internal.h" #include "qsvenc.h" typedef struct QSVMpeg2EncContext { diff --git a/libavcodec/qsvenc_vp9.c b/libavcodec/qsvenc_vp9.c index 9e96751828..81f5e69e24 100644 --- a/libavcodec/qsvenc_vp9.c +++ b/libavcodec/qsvenc_vp9.c @@ -30,7 +30,6 @@ #include "avcodec.h" #include "codec_internal.h" #include "qsv.h" -#include "qsv_internal.h" #include "qsvenc.h" typedef struct QSVVP9EncContext {