From patchwork Fri May 14 04:53:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 27744 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:b214:0:0:0:0:0 with SMTP id b20csp119842iof; Thu, 13 May 2021 21:54:50 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxQywYqICtADIdnLI+peY2ufSg5RMiUK04IZqt/8IbtqSXtENk24PoV/EXWMIkzXGw4AX8V X-Received: by 2002:a17:906:998d:: with SMTP id af13mr2893785ejc.540.1620968090637; Thu, 13 May 2021 21:54:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1620968090; cv=none; d=google.com; s=arc-20160816; b=b2cu5eHPUdo2OAwpU/NqO9km7WQGe8NtYvMby4sJ69+XeVbUP0aMo5fJXOGaRymOmX GZ6ikcL2/6qZWnyI3nlUSJwBtKTcEarUQTVbt/5icKiRSIKjcWn8cfHIq14jj4jcZebw d6swJ5E+4KUANrehzgITIV+DEegl9HzwdHzh3UrDteelZ4avzZ96ZObiqXNt+m1v+ONg MdgLcs3NphXv7+xxP73hkXWWh6bBjU49+Ho7HkT+Y8udx3QJfPvIMPuVIex4aS9ajxoI YHzWHBwvjyOnu5Iucw4GScpUj0/isbq5398fgqQlzZdiRpOzOz1TAVrBjqFfzo/d5HZw LR5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:ironport-sdr:ironport-sdr:delivered-to; bh=XIAGT9gJ9RqsqgtMt49my9CWlN0RdF/kDMJDHKX1ItA=; b=uy2S2sN56BXzHd+rd/AHWwXgvOJRLAadwScHyUoUTKdK/cyuVx+SmtrymBUrwmTc39 LP73wgQdLyKPuzZRgvsC0vTeWWSubSb04ywqlxA8TzIpej6wcOgIiHmEHdRiJW2h+CKp Jxz+tBTPuHKbeUkdc6fceUI3DU3jr4ZGwGfD4OGkx7Bu/DXLfF5jkAJh66z31FOa4CtT zKa1JFvbJTOVG63iyWI82W9hWAFCPOFiCZHUGknN5vrKCUtoAes3pM2RHvJrWHK8Swvc rg1xWIV/0SSRaU3uoWuEYLERjRn715KznA9v5e2HiKXvZaPkrd9Q0yOvvbJEMc1IRycM 65ww== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i20si1829673edc.88.2021.05.13.21.54.50; Thu, 13 May 2021 21:54:50 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 22D586880FA; Fri, 14 May 2021 07:54:37 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CE2FE687F3D for ; Fri, 14 May 2021 07:54:28 +0300 (EEST) IronPort-SDR: zsUvY7QdFljxQ7SHKoolxFWWVziB0K5wcI+2jJ6nSenAmHP0b9V4BXzY7Fk2PV1gsIVg2Nq2rf r6Aiagz3CsDA== X-IronPort-AV: E=McAfee;i="6200,9189,9983"; a="200165335" X-IronPort-AV: E=Sophos;i="5.82,299,1613462400"; d="scan'208";a="200165335" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2021 21:54:25 -0700 IronPort-SDR: AkyPWb31dVFFLp38iBvppz5ArfD60snaiEWKFje1I0OXiB7MM1PCoCn8kFc7ALNJOULAiyk6os itblyLjAA7Pw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.82,299,1613462400"; d="scan'208";a="626673052" Received: from xhh-tgl64.sh.intel.com ([10.239.159.130]) by fmsmga005.fm.intel.com with ESMTP; 13 May 2021 21:54:24 -0700 From: Haihao Xiang To: ffmpeg-devel@ffmpeg.org Date: Fri, 14 May 2021 12:53:39 +0800 Message-Id: <20210514045400.3277256-2-haihao.xiang@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210514045400.3277256-1-haihao.xiang@intel.com> References: <20210514045400.3277256-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 01/22] lavf/qsv: use QSVVPPContext as base context in vf_vpp_qsv/vf_overlay_qsv X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: eL5H9kAybjCT The same members between QSVVPPContext and VPPContext are removed from VPPContext, and async_depth is moved from QSVVPPParam to QSVVPPContext so that all QSV filters using QSVVPPContext may support async depth. In addition we may use QSVVPPContext as base context in other QSV filters in the future. --- libavfilter/qsvvpp.c | 25 ++++++++----------------- libavfilter/qsvvpp.h | 8 ++++---- libavfilter/vf_overlay_qsv.c | 11 +++++------ libavfilter/vf_vpp_qsv.c | 34 +++++++++++++--------------------- 4 files changed, 30 insertions(+), 48 deletions(-) diff --git a/libavfilter/qsvvpp.c b/libavfilter/qsvvpp.c index 4768f6208b..5b0b30e23c 100644 --- a/libavfilter/qsvvpp.c +++ b/libavfilter/qsvvpp.c @@ -647,15 +647,11 @@ static unsigned int qsv_fifo_size(const AVFifoBuffer* fifo) return av_fifo_size(fifo)/qsv_fifo_item_size(); } -int ff_qsvvpp_create(AVFilterContext *avctx, QSVVPPContext **vpp, QSVVPPParam *param) +int ff_qsvvpp_init(AVFilterContext *avctx, QSVVPPParam *param) { int i; int ret; - QSVVPPContext *s; - - s = av_mallocz(sizeof(*s)); - if (!s) - return AVERROR(ENOMEM); + QSVVPPContext *s = avctx->priv; s->filter_frame = param->filter_frame; if (!s->filter_frame) @@ -722,14 +718,13 @@ int ff_qsvvpp_create(AVFilterContext *avctx, QSVVPPContext **vpp, QSVVPPParam *p s->got_frame = 0; /** keep fifo size at least 1. Even when async_depth is 0, fifo is used. */ - s->async_fifo = av_fifo_alloc((param->async_depth + 1) * qsv_fifo_item_size()); - s->async_depth = param->async_depth; + s->async_fifo = av_fifo_alloc((s->async_depth + 1) * qsv_fifo_item_size()); if (!s->async_fifo) { ret = AVERROR(ENOMEM); goto failed; } - s->vpp_param.AsyncDepth = param->async_depth; + s->vpp_param.AsyncDepth = s->async_depth; if (IS_SYSTEM_MEMORY(s->in_mem_mode)) s->vpp_param.IOPattern |= MFX_IOPATTERN_IN_SYSTEM_MEMORY; @@ -756,25 +751,22 @@ int ff_qsvvpp_create(AVFilterContext *avctx, QSVVPPContext **vpp, QSVVPPParam *p } else if (ret > 0) ff_qsvvpp_print_warning(avctx, ret, "Warning When creating qsvvpp"); - *vpp = s; return 0; failed: - ff_qsvvpp_free(&s); + ff_qsvvpp_close(avctx); return ret; } -int ff_qsvvpp_free(QSVVPPContext **vpp) +int ff_qsvvpp_close(AVFilterContext *avctx) { - QSVVPPContext *s = *vpp; - - if (!s) - return 0; + QSVVPPContext *s = avctx->priv; if (s->session) { MFXVideoVPP_Close(s->session); MFXClose(s->session); + s->session = NULL; } /* release all the resources */ @@ -785,7 +777,6 @@ int ff_qsvvpp_free(QSVVPPContext **vpp) av_freep(&s->ext_buffers); av_freep(&s->frame_infos); av_fifo_free(s->async_fifo); - av_freep(vpp); return 0; } diff --git a/libavfilter/qsvvpp.h b/libavfilter/qsvvpp.h index e0f4c8f5bb..b6fe0d3fa7 100644 --- a/libavfilter/qsvvpp.h +++ b/libavfilter/qsvvpp.h @@ -48,6 +48,8 @@ typedef struct QSVFrame { } QSVFrame; typedef struct QSVVPPContext { + const AVClass *class; + mfxSession session; int (*filter_frame) (AVFilterLink *outlink, AVFrame *frame); /**< callback */ enum AVPixelFormat out_sw_format; /**< Real output format */ @@ -95,15 +97,13 @@ typedef struct QSVVPPParam { /* Crop information for each input, if needed */ int num_crop; QSVVPPCrop *crop; - - int async_depth; } QSVVPPParam; /* create and initialize the QSV session */ -int ff_qsvvpp_create(AVFilterContext *avctx, QSVVPPContext **vpp, QSVVPPParam *param); +int ff_qsvvpp_init(AVFilterContext *avctx, QSVVPPParam *param); /* release the resources (eg.surfaces) */ -int ff_qsvvpp_free(QSVVPPContext **vpp); +int ff_qsvvpp_close(AVFilterContext *avctx); /* vpp filter frame and call the cb if needed */ int ff_qsvvpp_filter_frame(QSVVPPContext *vpp, AVFilterLink *inlink, AVFrame *frame); diff --git a/libavfilter/vf_overlay_qsv.c b/libavfilter/vf_overlay_qsv.c index 7a4afd77d4..0b978d6528 100644 --- a/libavfilter/vf_overlay_qsv.c +++ b/libavfilter/vf_overlay_qsv.c @@ -58,10 +58,9 @@ enum var_name { }; typedef struct QSVOverlayContext { - const AVClass *class; + QSVVPPContext qsv; FFFrameSync fs; - QSVVPPContext *qsv; QSVVPPParam qsv_param; mfxExtVPPComposite comp_conf; double var_values[VAR_VARS_NB]; @@ -231,14 +230,14 @@ static int config_overlay_input(AVFilterLink *inlink) static int process_frame(FFFrameSync *fs) { AVFilterContext *ctx = fs->parent; - QSVOverlayContext *s = fs->opaque; + QSVVPPContext *qsv = fs->opaque; AVFrame *frame = NULL; int ret = 0, i; for (i = 0; i < ctx->nb_inputs; i++) { ret = ff_framesync_get_frame(fs, i, &frame, 0); if (ret == 0) - ret = ff_qsvvpp_filter_frame(s->qsv, ctx->inputs[i], frame); + ret = ff_qsvvpp_filter_frame(qsv, ctx->inputs[i], frame); if (ret < 0 && ret != AVERROR(EAGAIN)) break; } @@ -300,7 +299,7 @@ static int config_output(AVFilterLink *outlink) if (ret < 0) return ret; - return ff_qsvvpp_create(ctx, &vpp->qsv, &vpp->qsv_param); + return ff_qsvvpp_init(ctx, &vpp->qsv_param); } /* @@ -349,7 +348,7 @@ static av_cold void overlay_qsv_uninit(AVFilterContext *ctx) { QSVOverlayContext *vpp = ctx->priv; - ff_qsvvpp_free(&vpp->qsv); + ff_qsvvpp_close(ctx); ff_framesync_uninit(&vpp->fs); av_freep(&vpp->comp_conf.InputStream); av_freep(&vpp->qsv_param.ext_buf); diff --git a/libavfilter/vf_vpp_qsv.c b/libavfilter/vf_vpp_qsv.c index b9ab5c6490..72df8a8373 100644 --- a/libavfilter/vf_vpp_qsv.c +++ b/libavfilter/vf_vpp_qsv.c @@ -48,9 +48,7 @@ #define QSV_HAVE_MIRRORING QSV_VERSION_ATLEAST(1, 19) typedef struct VPPContext{ - const AVClass *class; - - QSVVPPContext *qsv; + QSVVPPContext qsv; /* Video Enhancement Algorithms */ mfxExtVPPDeinterlacing deinterlace_conf; @@ -94,9 +92,6 @@ typedef struct VPPContext{ char *cx, *cy, *cw, *ch; char *ow, *oh; char *output_format_str; - - int async_depth; - int eof; } VPPContext; static const AVOption options[] = { @@ -132,7 +127,7 @@ static const AVOption options[] = { { "h", "Output video height", OFFSET(oh), AV_OPT_TYPE_STRING, { .str="w*ch/cw" }, 0, 255, .flags = FLAGS }, { "height", "Output video height", OFFSET(oh), AV_OPT_TYPE_STRING, { .str="w*ch/cw" }, 0, 255, .flags = FLAGS }, { "format", "Output pixel format", OFFSET(output_format_str), AV_OPT_TYPE_STRING, { .str = "same" }, .flags = FLAGS }, - { "async_depth", "Internal parallelization depth, the higher the value the higher the latency.", OFFSET(async_depth), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, .flags = FLAGS }, + { "async_depth", "Internal parallelization depth, the higher the value the higher the latency.", OFFSET(qsv.async_depth), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, .flags = FLAGS }, { NULL } }; @@ -308,7 +303,6 @@ static int config_output(AVFilterLink *outlink) param.filter_frame = NULL; param.num_ext_buf = 0; param.ext_buf = ext_buf; - param.async_depth = vpp->async_depth; if (inlink->format == AV_PIX_FMT_QSV) { if (!inlink->hw_frames_ctx || !inlink->hw_frames_ctx->data) @@ -463,8 +457,9 @@ static int config_output(AVFilterLink *outlink) if (vpp->use_frc || vpp->use_crop || vpp->deinterlace || vpp->denoise || vpp->detail || vpp->procamp || vpp->rotate || vpp->hflip || inlink->w != outlink->w || inlink->h != outlink->h || in_format != vpp->out_format) - return ff_qsvvpp_create(ctx, &vpp->qsv, ¶m); + return ff_qsvvpp_init(ctx, ¶m); else { + /* No MFX session is created in this case */ av_log(ctx, AV_LOG_VERBOSE, "qsv vpp pass through mode.\n"); if (inlink->hw_frames_ctx) outlink->hw_frames_ctx = av_buffer_ref(inlink->hw_frames_ctx); @@ -477,33 +472,31 @@ static int activate(AVFilterContext *ctx) { AVFilterLink *inlink = ctx->inputs[0]; AVFilterLink *outlink = ctx->outputs[0]; - VPPContext *s =ctx->priv; - QSVVPPContext *qsv = s->qsv; + QSVVPPContext *qsv = ctx->priv; AVFrame *in = NULL; int ret, status; int64_t pts; FF_FILTER_FORWARD_STATUS_BACK(outlink, inlink); - if (!s->eof) { + if (!qsv->eof) { ret = ff_inlink_consume_frame(inlink, &in); if (ret < 0) return ret; if (ff_inlink_acknowledge_status(inlink, &status, &pts)) { if (status == AVERROR_EOF) { - s->eof = 1; + qsv->eof = 1; } } } - if (qsv) { - if (in || s->eof) { - qsv->eof = s->eof; + if (qsv->session) { + if (in || qsv->eof) { ret = ff_qsvvpp_filter_frame(qsv, inlink, in); av_frame_free(&in); - if (s->eof) { + if (qsv->eof) { ff_outlink_set_status(outlink, status, pts); return 0; } @@ -514,6 +507,7 @@ static int activate(AVFilterContext *ctx) } } } else { + /* No MFX session is created in pass-through mode */ if (in) { if (in->pts != AV_NOPTS_VALUE) in->pts = av_rescale_q(in->pts, inlink->time_base, outlink->time_base); @@ -523,7 +517,7 @@ static int activate(AVFilterContext *ctx) } } - if (s->eof) { + if (qsv->eof) { ff_outlink_set_status(outlink, status, pts); return 0; } else { @@ -561,9 +555,7 @@ static int query_formats(AVFilterContext *ctx) static av_cold void vpp_uninit(AVFilterContext *ctx) { - VPPContext *vpp = ctx->priv; - - ff_qsvvpp_free(&vpp->qsv); + ff_qsvvpp_close(ctx); } static const AVClass vpp_class = {