From patchwork Wed Jul 20 08:40:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 36861 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:1649:b0:8b:613a:194d with SMTP id no9csp2255322pzb; Wed, 20 Jul 2022 01:41:38 -0700 (PDT) X-Google-Smtp-Source: AGRyM1tRAaz/QanYTSaoysxGLkIcboBOvEVuisNaU7l0LsOo97+dRIg2wyMHlhprxRngGSAwXJks X-Received: by 2002:a17:906:8501:b0:711:bf65:2a47 with SMTP id i1-20020a170906850100b00711bf652a47mr35184097ejx.150.1658306497924; Wed, 20 Jul 2022 01:41:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1658306497; cv=none; d=google.com; s=arc-20160816; b=QfjrpvcTywnjng/dMjsFYeh6XlWrdkWRE8X/ytmjmZRS5mBwVGpcOW15wRVpNFIe14 hB9cQOyC47xexCQsR9oBqYEaErF2doqL0w8aMAaQGlcQPXxjAxtdGXCulatEGaGo8H71 hCRgLB2dVBpv/QtOSHNpGmndEixE/MJksAoMjaDe3trl6SOsR7VDq7YbmVpJ+k1HXAdK tKE00KqutnNgwiprFVXNHrGaTuknwUFlp/VxU4WS7jX2SaAMmbjnXgEXXaWODb6hpjlS ji4m1mpOGXVkOvnPYV/De1Cxz+EgEkI9EfPu6nX/YC54kqNGxNB2bmC2bYhRD/NAgKZK HHYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=crSmFePEV+cpxUqq8DxGNquAYAgNUlUrgzU3PJenO3o=; b=ICujMrbqjEV38T8u47zBImON1EKwjBe3LEVJ7uIVErfQq/Ph99zLB5dGQBbop1uRyd EAH+UlA9IrG5yALcH8iTy1uO+tF0i5G9kzRgjce9O8CwP72c/2OPkSeDpvruYurbsZvX GN7XcFndA0jKC+MVPxJB6ufK7AJZd8ChVGhsjv8dZtXKmQtmNOLrQ4OFoq3nBDzmVTDu biRdiQAR8dtnt1Dmg0JjStKO5L8jaS4jsLnhmsTF77V6B12ScsNYpwPgsV9SZhT8QqkW hGDOizkTRNPwg2TF8ghKemKqeuDpTx6F7N6VrTaANtokGuDimug0QbsN0E9wlU7euagi hQyw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Z1aOXayd; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id dr15-20020a170907720f00b0072b9be1e354si2474878ejc.18.2022.07.20.01.41.33; Wed, 20 Jul 2022 01:41:37 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Z1aOXayd; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CA06868B641; Wed, 20 Jul 2022 11:40:43 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A70CF68B671 for ; Wed, 20 Jul 2022 11:40:40 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1658306440; x=1689842440; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=BCghDp/DS5NOhmcfwjvrI+6F5Aln62sWCWKYxuXAdV4=; b=Z1aOXaydSIJLJBiGKj5iU7Ol7Pj1p5anKAKdi6LlUfrQrn+wlEzik17I dOILMGVI4ruxDaX3VKCfqHGyX5PmILKLoVvs2Y2/BO9MvVE1vOsUS32hV +eDNJtP8h9cxAvAQqsN0+FVDfFUBRU6VhFGhp28nGxfIFVDjT6jArDh+f CxJztB2Y9sJ1z+xFs1Ii2amDRRhE4hufsACh95U7h/QJhfPQw45xbaQmv +dTuU/4O0tm8X7jx3nXd/i9u+uyyUV5IZlnKa85/LdfMLioXt5Iu4/Cww 16XJR0WYe36ldC6F9BVK4e+i35T0Mwh66SYgVSDg60V6cIfpvvWe5xr+O A==; X-IronPort-AV: E=McAfee;i="6400,9594,10413"; a="348412531" X-IronPort-AV: E=Sophos;i="5.92,286,1650956400"; d="scan'208";a="348412531" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Jul 2022 01:40:36 -0700 X-IronPort-AV: E=Sophos;i="5.92,286,1650956400"; d="scan'208";a="625574692" Received: from xhh-dg164.sh.intel.com ([10.238.5.169]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Jul 2022 01:40:34 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 20 Jul 2022 16:40:01 +0800 Message-Id: <20220720084006.5033-9-haihao.xiang@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220720084006.5033-1-haihao.xiang@intel.com> References: <20220720084006.5033-1-haihao.xiang@intel.com> Subject: [FFmpeg-devel] [PATCH v11 08/13] qsv: support OPAQUE memory when MFX_VERSION < 2.0 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: nEGdLwN9cP9L From: Haihao Xiang OPAQUE memory isn't supported for MFX_VERSION >= 2.0[1][2]. This is in preparation for oneVPL support [1] https://spec.oneapi.io/versions/latest/elements/oneVPL/source/VPL_intel_media_sdk.html#msdk-full-name-feature-removals [2] https://github.com/oneapi-src/oneVPL --- libavcodec/qsv.c | 4 ++ libavcodec/qsv.h | 2 + libavcodec/qsv_internal.h | 1 + libavcodec/qsvdec.c | 9 ++++ libavcodec/qsvenc.c | 21 +++++++++ libavcodec/qsvenc.h | 2 + libavfilter/qsvvpp.c | 26 ++++++++++- libavfilter/qsvvpp.h | 3 ++ libavfilter/vf_deinterlace_qsv.c | 57 +++++++++++++----------- libavfilter/vf_scale_qsv.c | 74 ++++++++++++++++++-------------- libavutil/hwcontext_qsv.c | 56 +++++++++++++++++------- 11 files changed, 181 insertions(+), 74 deletions(-) diff --git a/libavcodec/qsv.c b/libavcodec/qsv.c index cc4b6cfd5d..432675bccf 100644 --- a/libavcodec/qsv.c +++ b/libavcodec/qsv.c @@ -81,10 +81,14 @@ static const struct { } qsv_iopatterns[] = { {MFX_IOPATTERN_IN_VIDEO_MEMORY, "input is video memory surface" }, {MFX_IOPATTERN_IN_SYSTEM_MEMORY, "input is system memory surface" }, +#if QSV_HAVE_OPAQUE {MFX_IOPATTERN_IN_OPAQUE_MEMORY, "input is opaque memory surface" }, +#endif {MFX_IOPATTERN_OUT_VIDEO_MEMORY, "output is video memory surface" }, {MFX_IOPATTERN_OUT_SYSTEM_MEMORY, "output is system memory surface" }, +#if QSV_HAVE_OPAQUE {MFX_IOPATTERN_OUT_OPAQUE_MEMORY, "output is opaque memory surface" }, +#endif }; int ff_qsv_print_iopattern(void *log_ctx, int mfx_iopattern, diff --git a/libavcodec/qsv.h b/libavcodec/qsv.h index 04ae0d6f34..c156b08d07 100644 --- a/libavcodec/qsv.h +++ b/libavcodec/qsv.h @@ -61,6 +61,8 @@ typedef struct AVQSVContext { * required by the encoder and the user-provided value nb_opaque_surfaces. * The array of the opaque surfaces will be exported to the caller through * the opaque_surfaces field. + * + * The caller must set this field to zero for oneVPL (MFX_VERSION >= 2.0) */ int opaque_alloc; diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h index 949955bda0..d52f0dcd15 100644 --- a/libavcodec/qsv_internal.h +++ b/libavcodec/qsv_internal.h @@ -66,6 +66,7 @@ (MFX_VERSION.Major == (MAJOR) && MFX_VERSION.Minor >= (MINOR))) #define QSV_ONEVPL QSV_VERSION_ATLEAST(2, 0) +#define QSV_HAVE_OPAQUE !QSV_ONEVPL typedef struct QSVMid { AVBufferRef *hw_frames_ref; diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index edbe7bcdee..8b718a08ff 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -187,7 +187,11 @@ static int qsv_init_session(AVCodecContext *avctx, QSVContext *q, mfxSession ses ret = ff_qsv_init_session_frames(avctx, &q->internal_qs.session, &q->frames_ctx, q->load_plugins, +#if QSV_HAVE_OPAQUE q->iopattern == MFX_IOPATTERN_OUT_OPAQUE_MEMORY, +#else + 0, +#endif q->gpu_copy); if (ret < 0) { av_buffer_unref(&q->frames_ctx.hw_frames_ctx); @@ -300,10 +304,15 @@ static int qsv_decode_preinit(AVCodecContext *avctx, QSVContext *q, enum AVPixel AVQSVFramesContext *frames_hwctx = frames_ctx->hwctx; if (!iopattern) { +#if QSV_HAVE_OPAQUE if (frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME) iopattern = MFX_IOPATTERN_OUT_OPAQUE_MEMORY; else if (frames_hwctx->frame_type & MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET) iopattern = MFX_IOPATTERN_OUT_VIDEO_MEMORY; +#else + if (frames_hwctx->frame_type & MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET) + iopattern = MFX_IOPATTERN_OUT_VIDEO_MEMORY; +#endif } } diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index d3a9a2118e..4831640868 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -1176,6 +1176,7 @@ static int qsv_retrieve_enc_params(AVCodecContext *avctx, QSVEncContext *q) return 0; } +#if QSV_HAVE_OPAQUE static int qsv_init_opaque_alloc(AVCodecContext *avctx, QSVEncContext *q) { AVQSVContext *qsv = avctx->hwaccel_context; @@ -1212,6 +1213,7 @@ static int qsv_init_opaque_alloc(AVCodecContext *avctx, QSVEncContext *q) return 0; } +#endif static int qsvenc_init_session(AVCodecContext *avctx, QSVEncContext *q) { @@ -1227,7 +1229,11 @@ static int qsvenc_init_session(AVCodecContext *avctx, QSVEncContext *q) ret = ff_qsv_init_session_frames(avctx, &q->internal_qs.session, &q->frames_ctx, q->load_plugins, +#if QSV_HAVE_OPAQUE q->param.IOPattern == MFX_IOPATTERN_IN_OPAQUE_MEMORY, +#else + 0, +#endif MFX_GPUCOPY_OFF); if (ret < 0) { av_buffer_unref(&q->frames_ctx.hw_frames_ctx); @@ -1279,11 +1285,17 @@ int ff_qsv_enc_init(AVCodecContext *avctx, QSVEncContext *q) AVQSVFramesContext *frames_hwctx = frames_ctx->hwctx; if (!iopattern) { +#if QSV_HAVE_OPAQUE if (frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME) iopattern = MFX_IOPATTERN_IN_OPAQUE_MEMORY; else if (frames_hwctx->frame_type & (MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET | MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET)) iopattern = MFX_IOPATTERN_IN_VIDEO_MEMORY; +#else + if (frames_hwctx->frame_type & + (MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET | MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET)) + iopattern = MFX_IOPATTERN_IN_VIDEO_MEMORY; +#endif } } @@ -1357,9 +1369,16 @@ int ff_qsv_enc_init(AVCodecContext *avctx, QSVEncContext *q) "Error querying (IOSurf) the encoding parameters"); if (opaque_alloc) { +#if QSV_HAVE_OPAQUE ret = qsv_init_opaque_alloc(avctx, q); if (ret < 0) return ret; +#else + av_log(avctx, AV_LOG_ERROR, "User is requesting to allocate OPAQUE surface, " + "however libmfx %d.%d doesn't support OPAQUE memory.\n", + q->ver.Major, q->ver.Minor); + return AVERROR_UNKNOWN; +#endif } ret = MFXVideoENCODE_Init(q->session, &q->param); @@ -1913,8 +1932,10 @@ int ff_qsv_enc_close(AVCodecContext *avctx, QSVEncContext *q) av_fifo_freep2(&q->async_fifo); } +#if QSV_HAVE_OPAQUE av_freep(&q->opaque_surfaces); av_buffer_unref(&q->opaque_alloc_buf); +#endif av_freep(&q->extparam); diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h index 40e2f65eff..c2def19658 100644 --- a/libavcodec/qsvenc.h +++ b/libavcodec/qsvenc.h @@ -137,9 +137,11 @@ typedef struct QSVEncContext { mfxExtHEVCTiles exthevctiles; mfxExtVP9Param extvp9param; +#if QSV_HAVE_OPAQUE mfxExtOpaqueSurfaceAlloc opaque_alloc; mfxFrameSurface1 **opaque_surfaces; AVBufferRef *opaque_alloc_buf; +#endif mfxExtVideoSignalInfo extvsi; diff --git a/libavfilter/qsvvpp.c b/libavfilter/qsvvpp.c index 3647891d13..3f984fd5f9 100644 --- a/libavfilter/qsvvpp.c +++ b/libavfilter/qsvvpp.c @@ -34,7 +34,9 @@ #define IS_VIDEO_MEMORY(mode) (mode & (MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET | \ MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET)) +#if QSV_HAVE_OPAQUE #define IS_OPAQUE_MEMORY(mode) (mode & MFX_MEMTYPE_OPAQUE_FRAME) +#endif #define IS_SYSTEM_MEMORY(mode) (mode & MFX_MEMTYPE_SYSTEM_MEMORY) #define MFX_IMPL_VIA_MASK(impl) (0x0f00 & (impl)) @@ -53,10 +55,14 @@ static const struct { } qsv_iopatterns[] = { {MFX_IOPATTERN_IN_VIDEO_MEMORY, "input is video memory surface" }, {MFX_IOPATTERN_IN_SYSTEM_MEMORY, "input is system memory surface" }, +#if QSV_HAVE_OPAQUE {MFX_IOPATTERN_IN_OPAQUE_MEMORY, "input is opaque memory surface" }, +#endif {MFX_IOPATTERN_OUT_VIDEO_MEMORY, "output is video memory surface" }, {MFX_IOPATTERN_OUT_SYSTEM_MEMORY, "output is system memory surface" }, +#if QSV_HAVE_OPAQUE {MFX_IOPATTERN_OUT_OPAQUE_MEMORY, "output is opaque memory surface" }, +#endif }; int ff_qsvvpp_print_iopattern(void *log_ctx, int mfx_iopattern, @@ -536,9 +542,13 @@ static int init_vpp_session(AVFilterContext *avctx, QSVVPPContext *s) if (!out_frames_ref) return AVERROR(ENOMEM); +#if QSV_HAVE_OPAQUE s->out_mem_mode = IS_OPAQUE_MEMORY(s->in_mem_mode) ? MFX_MEMTYPE_OPAQUE_FRAME : MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET | MFX_MEMTYPE_FROM_VPPOUT; +#else + s->out_mem_mode = MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET | MFX_MEMTYPE_FROM_VPPOUT; +#endif out_frames_ctx = (AVHWFramesContext *)out_frames_ref->data; out_frames_hwctx = out_frames_ctx->hwctx; @@ -624,6 +634,7 @@ static int init_vpp_session(AVFilterContext *avctx, QSVVPPContext *s) return AVERROR_UNKNOWN; } +#if QSV_HAVE_OPAQUE if (IS_OPAQUE_MEMORY(s->in_mem_mode) || IS_OPAQUE_MEMORY(s->out_mem_mode)) { s->opaque_alloc.In.Surfaces = s->surface_ptrs_in; s->opaque_alloc.In.NumSurface = s->nb_surface_ptrs_in; @@ -635,7 +646,9 @@ static int init_vpp_session(AVFilterContext *avctx, QSVVPPContext *s) s->opaque_alloc.Header.BufferId = MFX_EXTBUFF_OPAQUE_SURFACE_ALLOCATION; s->opaque_alloc.Header.BufferSz = sizeof(s->opaque_alloc); - } else if (IS_VIDEO_MEMORY(s->in_mem_mode) || IS_VIDEO_MEMORY(s->out_mem_mode)) { + } else +#endif + if (IS_VIDEO_MEMORY(s->in_mem_mode) || IS_VIDEO_MEMORY(s->out_mem_mode)) { mfxFrameAllocator frame_allocator = { .pthis = s, .Alloc = frame_alloc, @@ -707,6 +720,7 @@ int ff_qsvvpp_create(AVFilterContext *avctx, QSVVPPContext **vpp, QSVVPPParam *p goto failed; } +#if QSV_HAVE_OPAQUE if (IS_OPAQUE_MEMORY(s->in_mem_mode) || IS_OPAQUE_MEMORY(s->out_mem_mode)) { s->nb_ext_buffers = param->num_ext_buf + 1; s->ext_buffers = av_calloc(s->nb_ext_buffers, sizeof(*s->ext_buffers)); @@ -724,6 +738,10 @@ int ff_qsvvpp_create(AVFilterContext *avctx, QSVVPPContext **vpp, QSVVPPParam *p s->vpp_param.NumExtParam = param->num_ext_buf; s->vpp_param.ExtParam = param->ext_buf; } +#else + s->vpp_param.NumExtParam = param->num_ext_buf; + s->vpp_param.ExtParam = param->ext_buf; +#endif s->got_frame = 0; @@ -741,15 +759,19 @@ int ff_qsvvpp_create(AVFilterContext *avctx, QSVVPPContext **vpp, QSVVPPParam *p s->vpp_param.IOPattern |= MFX_IOPATTERN_IN_SYSTEM_MEMORY; else if (IS_VIDEO_MEMORY(s->in_mem_mode)) s->vpp_param.IOPattern |= MFX_IOPATTERN_IN_VIDEO_MEMORY; +#if QSV_HAVE_OPAQUE else if (IS_OPAQUE_MEMORY(s->in_mem_mode)) s->vpp_param.IOPattern |= MFX_IOPATTERN_IN_OPAQUE_MEMORY; +#endif if (IS_SYSTEM_MEMORY(s->out_mem_mode)) s->vpp_param.IOPattern |= MFX_IOPATTERN_OUT_SYSTEM_MEMORY; else if (IS_VIDEO_MEMORY(s->out_mem_mode)) s->vpp_param.IOPattern |= MFX_IOPATTERN_OUT_VIDEO_MEMORY; +#if QSV_HAVE_OPAQUE else if (IS_OPAQUE_MEMORY(s->out_mem_mode)) s->vpp_param.IOPattern |= MFX_IOPATTERN_OUT_OPAQUE_MEMORY; +#endif /* Print input memory mode */ ff_qsvvpp_print_iopattern(avctx, s->vpp_param.IOPattern & 0x0F, "VPP"); @@ -788,7 +810,9 @@ int ff_qsvvpp_free(QSVVPPContext **vpp) clear_frame_list(&s->out_frame_list); av_freep(&s->surface_ptrs_in); av_freep(&s->surface_ptrs_out); +#if QSV_HAVE_OPAQUE av_freep(&s->ext_buffers); +#endif av_freep(&s->frame_infos); av_fifo_freep2(&s->async_fifo); av_freep(vpp); diff --git a/libavfilter/qsvvpp.h b/libavfilter/qsvvpp.h index 802abd987d..3e7d56021b 100644 --- a/libavfilter/qsvvpp.h +++ b/libavfilter/qsvvpp.h @@ -41,6 +41,7 @@ (MFX_VERSION.Major == (MAJOR) && MFX_VERSION.Minor >= (MINOR))) #define QSV_ONEVPL QSV_VERSION_ATLEAST(2, 0) +#define QSV_HAVE_OPAQUE !QSV_ONEVPL typedef struct QSVFrame { AVFrame *frame; @@ -66,10 +67,12 @@ typedef struct QSVVPPContext { mfxFrameSurface1 **surface_ptrs_in; mfxFrameSurface1 **surface_ptrs_out; +#if QSV_HAVE_OPAQUE /** MFXVPP extern parameters */ mfxExtOpaqueSurfaceAlloc opaque_alloc; mfxExtBuffer **ext_buffers; int nb_ext_buffers; +#endif int got_frame; int async_depth; diff --git a/libavfilter/vf_deinterlace_qsv.c b/libavfilter/vf_deinterlace_qsv.c index b8ff3e8339..50f9156d14 100644 --- a/libavfilter/vf_deinterlace_qsv.c +++ b/libavfilter/vf_deinterlace_qsv.c @@ -62,7 +62,9 @@ typedef struct QSVDeintContext { mfxFrameSurface1 **surface_ptrs; int nb_surface_ptrs; +#if QSV_HAVE_OPAQUE mfxExtOpaqueSurfaceAlloc opaque_alloc; +#endif mfxExtVPPDeinterlacing deint_conf; mfxExtBuffer *ext_buffers[2]; int num_ext_buffers; @@ -154,9 +156,7 @@ static int init_out_session(AVFilterContext *ctx) AVHWFramesContext *hw_frames_ctx = (AVHWFramesContext*)s->hw_frames_ctx->data; AVQSVFramesContext *hw_frames_hwctx = hw_frames_ctx->hwctx; AVQSVDeviceContext *device_hwctx = hw_frames_ctx->device_ctx->hwctx; - - int opaque = !!(hw_frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME); - + int opaque = 0; mfxHDL handle = NULL; mfxHandleType handle_type; mfxVersion ver; @@ -165,6 +165,9 @@ static int init_out_session(AVFilterContext *ctx) mfxStatus err; int i; +#if QSV_HAVE_OPAQUE + opaque = !!(hw_frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME); +#endif /* extract the properties of the "master" session given to us */ err = MFXQueryIMPL(device_hwctx->session, &impl); if (err == MFX_ERR_NONE) @@ -223,28 +226,7 @@ static int init_out_session(AVFilterContext *ctx) s->ext_buffers[s->num_ext_buffers++] = (mfxExtBuffer *)&s->deint_conf; - if (opaque) { - s->surface_ptrs = av_calloc(hw_frames_hwctx->nb_surfaces, - sizeof(*s->surface_ptrs)); - if (!s->surface_ptrs) - return AVERROR(ENOMEM); - for (i = 0; i < hw_frames_hwctx->nb_surfaces; i++) - s->surface_ptrs[i] = hw_frames_hwctx->surfaces + i; - s->nb_surface_ptrs = hw_frames_hwctx->nb_surfaces; - - s->opaque_alloc.In.Surfaces = s->surface_ptrs; - s->opaque_alloc.In.NumSurface = s->nb_surface_ptrs; - s->opaque_alloc.In.Type = hw_frames_hwctx->frame_type; - - s->opaque_alloc.Out = s->opaque_alloc.In; - - s->opaque_alloc.Header.BufferId = MFX_EXTBUFF_OPAQUE_SURFACE_ALLOCATION; - s->opaque_alloc.Header.BufferSz = sizeof(s->opaque_alloc); - - s->ext_buffers[s->num_ext_buffers++] = (mfxExtBuffer *)&s->opaque_alloc; - - par.IOPattern = MFX_IOPATTERN_IN_OPAQUE_MEMORY | MFX_IOPATTERN_OUT_OPAQUE_MEMORY; - } else { + if (!opaque) { mfxFrameAllocator frame_allocator = { .pthis = ctx, .Alloc = frame_alloc, @@ -268,6 +250,31 @@ static int init_out_session(AVFilterContext *ctx) par.IOPattern = MFX_IOPATTERN_IN_VIDEO_MEMORY | MFX_IOPATTERN_OUT_VIDEO_MEMORY; } +#if QSV_HAVE_OPAQUE + else { + s->surface_ptrs = av_calloc(hw_frames_hwctx->nb_surfaces, + sizeof(*s->surface_ptrs)); + + if (!s->surface_ptrs) + return AVERROR(ENOMEM); + for (i = 0; i < hw_frames_hwctx->nb_surfaces; i++) + s->surface_ptrs[i] = hw_frames_hwctx->surfaces + i; + s->nb_surface_ptrs = hw_frames_hwctx->nb_surfaces; + + s->opaque_alloc.In.Surfaces = s->surface_ptrs; + s->opaque_alloc.In.NumSurface = s->nb_surface_ptrs; + s->opaque_alloc.In.Type = hw_frames_hwctx->frame_type; + + s->opaque_alloc.Out = s->opaque_alloc.In; + + s->opaque_alloc.Header.BufferId = MFX_EXTBUFF_OPAQUE_SURFACE_ALLOCATION; + s->opaque_alloc.Header.BufferSz = sizeof(s->opaque_alloc); + + s->ext_buffers[s->num_ext_buffers++] = (mfxExtBuffer *)&s->opaque_alloc; + + par.IOPattern = MFX_IOPATTERN_IN_OPAQUE_MEMORY | MFX_IOPATTERN_OUT_OPAQUE_MEMORY; + } +#endif par.ExtParam = s->ext_buffers; par.NumExtParam = s->num_ext_buffers; diff --git a/libavfilter/vf_scale_qsv.c b/libavfilter/vf_scale_qsv.c index af0e85bd03..fa0152d785 100644 --- a/libavfilter/vf_scale_qsv.c +++ b/libavfilter/vf_scale_qsv.c @@ -89,7 +89,9 @@ typedef struct QSVScaleContext { mfxFrameSurface1 **surface_ptrs_out; int nb_surface_ptrs_out; +#if QSV_HAVE_OPAQUE mfxExtOpaqueSurfaceAlloc opaque_alloc; +#endif mfxExtVPPScaling scale_conf; int mode; @@ -268,7 +270,7 @@ static int init_out_session(AVFilterContext *ctx) AVQSVFramesContext *out_frames_hwctx = out_frames_ctx->hwctx; AVQSVDeviceContext *device_hwctx = in_frames_ctx->device_ctx->hwctx; - int opaque = !!(in_frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME); + int opaque = 0; mfxHDL handle = NULL; mfxHandleType handle_type; @@ -278,6 +280,9 @@ static int init_out_session(AVFilterContext *ctx) mfxStatus err; int i; +#if QSV_HAVE_OPAQUE + opaque = !!(in_frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME); +#endif s->num_ext_buf = 0; /* extract the properties of the "master" session given to us */ @@ -330,38 +335,7 @@ static int init_out_session(AVFilterContext *ctx) memset(&par, 0, sizeof(par)); - if (opaque) { - s->surface_ptrs_in = av_calloc(in_frames_hwctx->nb_surfaces, - sizeof(*s->surface_ptrs_in)); - if (!s->surface_ptrs_in) - return AVERROR(ENOMEM); - for (i = 0; i < in_frames_hwctx->nb_surfaces; i++) - s->surface_ptrs_in[i] = in_frames_hwctx->surfaces + i; - s->nb_surface_ptrs_in = in_frames_hwctx->nb_surfaces; - - s->surface_ptrs_out = av_calloc(out_frames_hwctx->nb_surfaces, - sizeof(*s->surface_ptrs_out)); - if (!s->surface_ptrs_out) - return AVERROR(ENOMEM); - for (i = 0; i < out_frames_hwctx->nb_surfaces; i++) - s->surface_ptrs_out[i] = out_frames_hwctx->surfaces + i; - s->nb_surface_ptrs_out = out_frames_hwctx->nb_surfaces; - - s->opaque_alloc.In.Surfaces = s->surface_ptrs_in; - s->opaque_alloc.In.NumSurface = s->nb_surface_ptrs_in; - s->opaque_alloc.In.Type = in_frames_hwctx->frame_type; - - s->opaque_alloc.Out.Surfaces = s->surface_ptrs_out; - s->opaque_alloc.Out.NumSurface = s->nb_surface_ptrs_out; - s->opaque_alloc.Out.Type = out_frames_hwctx->frame_type; - - s->opaque_alloc.Header.BufferId = MFX_EXTBUFF_OPAQUE_SURFACE_ALLOCATION; - s->opaque_alloc.Header.BufferSz = sizeof(s->opaque_alloc); - - s->ext_buffers[s->num_ext_buf++] = (mfxExtBuffer*)&s->opaque_alloc; - - par.IOPattern = MFX_IOPATTERN_IN_OPAQUE_MEMORY | MFX_IOPATTERN_OUT_OPAQUE_MEMORY; - } else { + if (!opaque) { mfxFrameAllocator frame_allocator = { .pthis = ctx, .Alloc = frame_alloc, @@ -393,6 +367,40 @@ static int init_out_session(AVFilterContext *ctx) par.IOPattern = MFX_IOPATTERN_IN_VIDEO_MEMORY | MFX_IOPATTERN_OUT_VIDEO_MEMORY; } +#if QSV_HAVE_OPAQUE + else { + s->surface_ptrs_in = av_calloc(in_frames_hwctx->nb_surfaces, + sizeof(*s->surface_ptrs_in)); + if (!s->surface_ptrs_in) + return AVERROR(ENOMEM); + for (i = 0; i < in_frames_hwctx->nb_surfaces; i++) + s->surface_ptrs_in[i] = in_frames_hwctx->surfaces + i; + s->nb_surface_ptrs_in = in_frames_hwctx->nb_surfaces; + + s->surface_ptrs_out = av_calloc(out_frames_hwctx->nb_surfaces, + sizeof(*s->surface_ptrs_out)); + if (!s->surface_ptrs_out) + return AVERROR(ENOMEM); + for (i = 0; i < out_frames_hwctx->nb_surfaces; i++) + s->surface_ptrs_out[i] = out_frames_hwctx->surfaces + i; + s->nb_surface_ptrs_out = out_frames_hwctx->nb_surfaces; + + s->opaque_alloc.In.Surfaces = s->surface_ptrs_in; + s->opaque_alloc.In.NumSurface = s->nb_surface_ptrs_in; + s->opaque_alloc.In.Type = in_frames_hwctx->frame_type; + + s->opaque_alloc.Out.Surfaces = s->surface_ptrs_out; + s->opaque_alloc.Out.NumSurface = s->nb_surface_ptrs_out; + s->opaque_alloc.Out.Type = out_frames_hwctx->frame_type; + + s->opaque_alloc.Header.BufferId = MFX_EXTBUFF_OPAQUE_SURFACE_ALLOCATION; + s->opaque_alloc.Header.BufferSz = sizeof(s->opaque_alloc); + + s->ext_buffers[s->num_ext_buf++] = (mfxExtBuffer*)&s->opaque_alloc; + + par.IOPattern = MFX_IOPATTERN_IN_OPAQUE_MEMORY | MFX_IOPATTERN_OUT_OPAQUE_MEMORY; + } +#endif memset(&s->scale_conf, 0, sizeof(mfxExtVPPScaling)); s->scale_conf.Header.BufferId = MFX_EXTBUFF_VPP_SCALING; diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index 99735f5557..21a2a805f8 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -55,6 +55,8 @@ MFX_VERSION_MAJOR == (MAJOR) && MFX_VERSION_MINOR >= (MINOR)) #define MFX_IMPL_VIA_MASK(impl) (0x0f00 & (impl)) +#define QSV_ONEVPL QSV_VERSION_ATLEAST(2, 0) +#define QSV_HAVE_OPAQUE !QSV_ONEVPL typedef struct QSVDevicePriv { AVBufferRef *child_device_ctx; @@ -86,11 +88,13 @@ typedef struct QSVFramesContext { // used in the frame allocator for non-opaque surfaces mfxMemId *mem_ids; +#if QSV_HAVE_OPAQUE // used in the opaque alloc request for opaque surfaces mfxFrameSurface1 **surface_ptrs; mfxExtOpaqueSurfaceAlloc opaque_alloc; mfxExtBuffer *ext_buffers[1]; +#endif AVFrame realigned_upload_frame; AVFrame realigned_download_frame; } QSVFramesContext; @@ -299,7 +303,9 @@ static void qsv_frames_uninit(AVHWFramesContext *ctx) #endif av_freep(&s->mem_ids); +#if QSV_HAVE_OPAQUE av_freep(&s->surface_ptrs); +#endif av_freep(&s->surfaces_internal); av_freep(&s->handle_pairs_internal); av_frame_unref(&s->realigned_upload_frame); @@ -535,11 +541,17 @@ static int qsv_init_pool(AVHWFramesContext *ctx, uint32_t fourcc) return ret; } +#if QSV_HAVE_OPAQUE if (!(frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME)) { ret = qsv_init_child_ctx(ctx); if (ret < 0) return ret; } +#else + ret = qsv_init_child_ctx(ctx); + if (ret < 0) + return ret; +#endif ctx->internal->pool_internal = av_buffer_pool_init2(sizeof(mfxFrameSurface1), ctx, qsv_pool_alloc, NULL); @@ -610,10 +622,9 @@ static mfxStatus frame_get_hdl(mfxHDL pthis, mfxMemId mid, mfxHDL *hdl) static int qsv_init_internal_session(AVHWFramesContext *ctx, mfxSession *session, int upload) { - QSVFramesContext *s = ctx->internal->priv; AVQSVFramesContext *frames_hwctx = ctx->hwctx; QSVDeviceContext *device_priv = ctx->device_ctx->internal->priv; - int opaque = !!(frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME); + int opaque = 0; mfxFrameAllocator frame_allocator = { .pthis = ctx, @@ -627,6 +638,11 @@ static int qsv_init_internal_session(AVHWFramesContext *ctx, mfxVideoParam par; mfxStatus err; +#if QSV_HAVE_OPAQUE + QSVFramesContext *s = ctx->internal->priv; + opaque = !!(frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME); +#endif + err = MFXInit(device_priv->impl, &device_priv->ver, session); if (err != MFX_ERR_NONE) { av_log(ctx, AV_LOG_ERROR, "Error initializing an internal session\n"); @@ -648,15 +664,18 @@ static int qsv_init_internal_session(AVHWFramesContext *ctx, memset(&par, 0, sizeof(par)); - if (opaque) { + if (!opaque) { + par.IOPattern = upload ? MFX_IOPATTERN_OUT_VIDEO_MEMORY : + MFX_IOPATTERN_IN_VIDEO_MEMORY; + } +#if QSV_HAVE_OPAQUE + else { par.ExtParam = s->ext_buffers; par.NumExtParam = FF_ARRAY_ELEMS(s->ext_buffers); par.IOPattern = upload ? MFX_IOPATTERN_OUT_OPAQUE_MEMORY : MFX_IOPATTERN_IN_OPAQUE_MEMORY; - } else { - par.IOPattern = upload ? MFX_IOPATTERN_OUT_VIDEO_MEMORY : - MFX_IOPATTERN_IN_VIDEO_MEMORY; } +#endif par.IOPattern |= upload ? MFX_IOPATTERN_IN_SYSTEM_MEMORY : MFX_IOPATTERN_OUT_SYSTEM_MEMORY; @@ -688,11 +707,15 @@ static int qsv_frames_init(AVHWFramesContext *ctx) QSVFramesContext *s = ctx->internal->priv; AVQSVFramesContext *frames_hwctx = ctx->hwctx; - int opaque = !!(frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME); + int opaque = 0; uint32_t fourcc; int i, ret; +#if QSV_HAVE_OPAQUE + opaque = !!(frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME); +#endif + fourcc = qsv_fourcc_from_pix_fmt(ctx->sw_format); if (!fourcc) { av_log(ctx, AV_LOG_ERROR, "Unsupported pixel format\n"); @@ -707,7 +730,16 @@ static int qsv_frames_init(AVHWFramesContext *ctx) } } - if (opaque) { + if (!opaque) { + s->mem_ids = av_calloc(frames_hwctx->nb_surfaces, sizeof(*s->mem_ids)); + if (!s->mem_ids) + return AVERROR(ENOMEM); + + for (i = 0; i < frames_hwctx->nb_surfaces; i++) + s->mem_ids[i] = frames_hwctx->surfaces[i].Data.MemId; + } +#if QSV_HAVE_OPAQUE + else { s->surface_ptrs = av_calloc(frames_hwctx->nb_surfaces, sizeof(*s->surface_ptrs)); if (!s->surface_ptrs) @@ -726,14 +758,8 @@ static int qsv_frames_init(AVHWFramesContext *ctx) s->opaque_alloc.Header.BufferSz = sizeof(s->opaque_alloc); s->ext_buffers[0] = (mfxExtBuffer*)&s->opaque_alloc; - } else { - s->mem_ids = av_calloc(frames_hwctx->nb_surfaces, sizeof(*s->mem_ids)); - if (!s->mem_ids) - return AVERROR(ENOMEM); - - for (i = 0; i < frames_hwctx->nb_surfaces; i++) - s->mem_ids[i] = frames_hwctx->surfaces[i].Data.MemId; } +#endif s->session_download = NULL; s->session_upload = NULL;