From patchwork Fri May 14 04:53:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 27754 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:b214:0:0:0:0:0 with SMTP id b20csp121421iof; Thu, 13 May 2021 21:57:40 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwAxPs8vkWzn2ONRIrR0+6NXX7W0pndgeQqTto5ml+ueqAV+34M5TU7H0hdfOBF2fUC3Q9f X-Received: by 2002:a50:fd0f:: with SMTP id i15mr43590607eds.278.1620968260004; Thu, 13 May 2021 21:57:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1620968260; cv=none; d=google.com; s=arc-20160816; b=jOeu6ZTncXU8maO4l3tmqFYl5Ukpr6XRsRMbN/47pgHCm5FwB7o1JCFnrm1xPEbNHu nGBaSUv73s0GD7k7+QJ9aiWCFsO/r9LiHm3h2On+89r5dGBer6ZvH0dQzqEdR7Xoz4rg 0uDwuO1+k6z5OyuEhfGN2k1Uy9PPOhH5+JnyoSDnkHy4yvXd2FTbwuvd0sF7S180LTBx 8I8W6J3UIdVlvFHVUTRykffYOVkdGHWAIU9Cr1K25KHNIml9RAlq3ffuLPSn/BduN/a7 W50qgflBXLogcllW84P1CnkMi8laKw52XS9nmi9WdM/oi7Rm4Hw4LoH8/qvTDQfkJMp/ EVJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:ironport-sdr:ironport-sdr:delivered-to; bh=dV9m04+79fEEebdrWtIWwkJKlvAa5CIViLI1V79pWNg=; b=saHv45iho8cT2W4Lp+CavlvlPD7qmZS1uIzbUU5M86lv7D73AWzet/dQnh5zHHhoXZ mRQwGkIQc+EkLuXgvloI0EcL+djjyfvTUoAIEWQSmEtZWE/k9QlAijhEzaKv0cnsGbsT RH6G5XBs/IiTWo7eN4dKd1rsHTnaIcsK8y1N3NhZ6G7Kv8wNZm2043Lq8eq2r8lfHYkR WXwvmjYs4GWtmRt2BKerSJKTdiMkgAGZg0ojfdtJMCf7Ydoq1HbErXmPIQ8i5QHImHyx ZfhNwTTIuTl70WagIxrYcPL+zM691QW0SlAS+g6WBMDu4Qc0W6Id1G6qM5FxyEMHjmgv 0wNw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id bo20si4923933edb.524.2021.05.13.21.57.39; Thu, 13 May 2021 21:57:39 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 341AA6898FB; Fri, 14 May 2021 07:54:59 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 195A9687FED for ; Fri, 14 May 2021 07:54:47 +0300 (EEST) IronPort-SDR: HW3rJrMd4irEBVbKwiDQwZbx/JDIhwq6Z5SKIi/DjaK87TXOsYlQ1hc1FxGiAUVEUBdHRnGTP7 bTaROcV3ADEw== X-IronPort-AV: E=McAfee;i="6200,9189,9983"; a="179711911" X-IronPort-AV: E=Sophos;i="5.82,299,1613462400"; d="scan'208";a="179711911" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2021 21:54:40 -0700 IronPort-SDR: A1Fp1ZxqUWdvUxEEVB/8D5iLlQzc9fu6HftXb537BFkAM+Q3h49S4OqfrN0gTW8Z5pwfsUMKO6 knaHz7WCoFwg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.82,299,1613462400"; d="scan'208";a="626673119" Received: from xhh-tgl64.sh.intel.com ([10.239.159.130]) by fmsmga005.fm.intel.com with ESMTP; 13 May 2021 21:54:39 -0700 From: Haihao Xiang To: ffmpeg-devel@ffmpeg.org Date: Fri, 14 May 2021 12:53:54 +0800 Message-Id: <20210514045400.3277256-17-haihao.xiang@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210514045400.3277256-1-haihao.xiang@intel.com> References: <20210514045400.3277256-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 16/22] lavf/qsvvpp: set PTS for output frame X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ptMQqOkWWcyE When the SDK returns MFX_ERR_MORE_SURFACE, the PTS is not set for the output frame. We assign a PTS calculated from the input frame to the output frame. After applying this patch, we may avoid the error below: [null @ 0x56395cab4ae0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 456 >= 0 Note this patch only fixes PTS issue when deinterlacing is enabled --- libavfilter/qsvvpp.c | 21 +++++++++++++++++++-- libavfilter/qsvvpp.h | 3 +++ libavfilter/vf_vpp_qsv.c | 2 ++ 3 files changed, 24 insertions(+), 2 deletions(-) diff --git a/libavfilter/qsvvpp.c b/libavfilter/qsvvpp.c index 82a8e29387..01d9d754d3 100644 --- a/libavfilter/qsvvpp.c +++ b/libavfilter/qsvvpp.c @@ -653,6 +653,7 @@ int ff_qsvvpp_init(AVFilterContext *avctx, QSVVPPParam *param) int ret; QSVVPPContext *s = avctx->priv; + s->last_in_pts = AV_NOPTS_VALUE; s->filter_frame = param->filter_frame; if (!s->filter_frame) s->filter_frame = ff_filter_frame; @@ -769,6 +770,8 @@ int ff_qsvvpp_close(AVFilterContext *avctx) s->session = NULL; } + s->last_in_pts = AV_NOPTS_VALUE; + /* release all the resources */ clear_frame_list(&s->in_frame_list); clear_frame_list(&s->out_frame_list); @@ -788,6 +791,7 @@ int ff_qsvvpp_filter_frame(QSVVPPContext *s, AVFilterLink *inlink, AVFrame *picr mfxSyncPoint sync; QSVFrame *in_frame, *out_frame, *tmp; int ret, ret1, filter_ret; + int64_t dpts = 0; while (s->eof && qsv_fifo_size(s->async_fifo)) { av_fifo_generic_read(s->async_fifo, &tmp, sizeof(tmp), NULL); @@ -836,8 +840,19 @@ int ff_qsvvpp_filter_frame(QSVVPPContext *s, AVFilterLink *inlink, AVFrame *picr ret = AVERROR(EAGAIN); break; } - out_frame->frame->pts = av_rescale_q(out_frame->surface.Data.TimeStamp, - default_tb, outlink->time_base); + + /* TODO: calculate the PTS for other cases */ + if (s->deinterlace_enabled && + s->last_in_pts != AV_NOPTS_VALUE && + ret == MFX_ERR_MORE_SURFACE && + out_frame->surface.Data.TimeStamp == MFX_TIMESTAMP_UNKNOWN) + dpts = (in_frame->frame->pts - s->last_in_pts) / 2; + else + dpts = 0; + + out_frame->frame->pts = av_rescale_q(in_frame->frame->pts - dpts, + inlink->time_base, + outlink->time_base); out_frame->queued++; av_fifo_generic_write(s->async_fifo, &out_frame, sizeof(out_frame), NULL); @@ -870,5 +885,7 @@ int ff_qsvvpp_filter_frame(QSVVPPContext *s, AVFilterLink *inlink, AVFrame *picr } } while(ret == MFX_ERR_MORE_SURFACE); + s->last_in_pts = in_frame->frame->pts; + return ret; } diff --git a/libavfilter/qsvvpp.h b/libavfilter/qsvvpp.h index b6fe0d3fa7..8627c8c868 100644 --- a/libavfilter/qsvvpp.h +++ b/libavfilter/qsvvpp.h @@ -74,8 +74,11 @@ typedef struct QSVVPPContext { int got_frame; int async_depth; int eof; + int deinterlace_enabled; /** order with frame_out, sync */ AVFifoBuffer *async_fifo; + + int64_t last_in_pts; } QSVVPPContext; typedef struct QSVVPPCrop { diff --git a/libavfilter/vf_vpp_qsv.c b/libavfilter/vf_vpp_qsv.c index ec35f85b04..018b0e8689 100644 --- a/libavfilter/vf_vpp_qsv.c +++ b/libavfilter/vf_vpp_qsv.c @@ -369,6 +369,8 @@ static int config_output(AVFilterLink *outlink) vpp->extbuf.field = value; \ } while (0) + vpp->qsv.deinterlace_enabled = !!vpp->deinterlace; + if (vpp->deinterlace) { INIT_MFX_EXTBUF(deinterlace_conf, MFX_EXTBUFF_VPP_DEINTERLACING); SET_MFX_PARAM_FIELD(deinterlace_conf, Mode, (vpp->deinterlace == 1 ?