From patchwork Wed Jun 9 04:18:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 28181 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5e:c91a:0:0:0:0:0 with SMTP id z26csp479421iol; Tue, 8 Jun 2021 21:18:59 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzGtHyW0+VuybX2WGy2Cz1toE2N1AlsDYRYr9grDL2nzFVwWSfv6fpPnQHpHk/DPfaa+vKW X-Received: by 2002:a50:ff13:: with SMTP id a19mr28764275edu.300.1623212339176; Tue, 08 Jun 2021 21:18:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1623212339; cv=none; d=google.com; s=arc-20160816; b=Tcd6GJaBCwnpTJXe1Bqf/FB3xFnjG3RDFJwQiyWcX+45V+5mgs8bRYYcMKDrjZjQis ddDG0E+jM83QM9bc8WatOhhCNRLLJxjiOiUqzLGel7CULOqXdXMkeV+D/aKH9JoyboF4 3Ns7N6ODuoy1U+rdeNNv1W+UFkD+BMLzsdgXx/YF2ibaOP5mbgl57F0bEc+iq45wB3Qr JKu9W+79sC0H61NKTJT2yEkDxHuEfjgtQKphmUHbdCCareHz59V1/s6PskvmTHeDXoZM q9GnVBtq/JMdOVuSDtDkDO5iNSJaB3JKYPQ/F76EU1tXLpGCkGMS6TDW9ePjIYd55UVq zU2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :ironport-sdr:ironport-sdr:delivered-to; bh=YUrUGSID96OlmUAdPpkPvJyiHrDDFdXVufJ4j/OF8ZU=; b=AK88Hk4nKkmU5qTotCvdaa0oqh5UsQ0745JxgmzmhQUKZobrQXLqVEVeYVDB6cK/C6 x8TaJZLEoffKHiVn6fy7sJ/hHauPBOq5yjQsJN3giuW/pwCoUwF2g/Ypp7XcfRg3iEvS SjEvqO+3Dp7DP0FnjNHugCxQ4Ymxu9+sgJchDqcPuuma13LIB0iZjbD/CyKPM7LCHJtR KBG9dYam3Sco1rbCfBaVwRjZ3zsyFez1nsT40EPb2rTQshGbkCtGltipuMHjYXq2rVu7 tU1whBIPxgVOj0jt+0NKnQ294Ny5XdQJNaM2ONXQc2OvyLku1l6Kx5qFXgUj4fAmUqnH EpWA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id k24si1385112edq.487.2021.06.08.21.18.58; Tue, 08 Jun 2021 21:18:59 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 33BB9687F34; Wed, 9 Jun 2021 07:18:54 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D84C468069C for ; Wed, 9 Jun 2021 07:18:46 +0300 (EEST) IronPort-SDR: fv+fx9sYDflDFaOnYkAkAtK0FX8BHHd8LJU72qs5AqbVHRdRooPsA9W1s11H4IK+fw+oExt44Y peKqAqyu5vmg== X-IronPort-AV: E=McAfee;i="6200,9189,10009"; a="192108160" X-IronPort-AV: E=Sophos;i="5.83,260,1616482800"; d="scan'208";a="192108160" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2021 21:18:40 -0700 IronPort-SDR: vOS2WFKokIpsTGtrd4Kwgj0G7VxbVO8UzoSWAzomZWFGnVoGrSuMc2VqFOUyO9F/sCc/PSKygU g5YanGGkPnOA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.83,260,1616482800"; d="scan'208";a="485580284" Received: from xhh-tgl64.sh.intel.com ([10.239.159.53]) by fmsmga002.fm.intel.com with ESMTP; 08 Jun 2021 21:18:37 -0700 From: Haihao Xiang To: ffmpeg-devel@ffmpeg.org Date: Wed, 9 Jun 2021 12:18:29 +0800 Message-Id: <20210609041830.1302044-1-haihao.xiang@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] lavc/qsvdec: fix pts X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: bMUxkRocL/lx The time base used for compressed bitstream and video frame in the SDK is { 1, 90000 }. [1][2] This can avoid the error message below from the muxer. $> ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.h265 -f null - ... [null @ 0x561c24f6f2f0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 2 >= 2 [1]https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.md#mfxbitstream [2]https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.md#mfxframedata --- libavcodec/qsvdec.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index f543defb18..622750927c 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -46,6 +46,16 @@ #include "qsv.h" #include "qsv_internal.h" +static const AVRational mfx_tb = { 1, 90000 }; + +#define PTS_TO_MFX_PTS(pts, pts_tb) ((pts) == AV_NOPTS_VALUE ? \ + MFX_TIMESTAMP_UNKNOWN : pts_tb.num ? \ + av_rescale_q(pts, pts_tb, mfx_tb) : pts) + +#define MFX_PTS_TO_PTS(mfx_pts, pts_tb) ((mfx_pts) == MFX_TIMESTAMP_UNKNOWN ? \ + AV_NOPTS_VALUE : pts_tb.num ? \ + av_rescale_q(mfx_pts, mfx_tb, pts_tb) : mfx_pts) + typedef struct QSVContext { // the session used for decoding mfxSession session; @@ -308,7 +318,7 @@ static int qsv_decode_header(AVCodecContext *avctx, QSVContext *q, bs.Data = avpkt->data; bs.DataLength = avpkt->size; bs.MaxLength = bs.DataLength; - bs.TimeStamp = avpkt->pts; + bs.TimeStamp = PTS_TO_MFX_PTS(avpkt->pts, avctx->pkt_timebase); if (avctx->field_order == AV_FIELD_PROGRESSIVE) bs.DataFlag |= MFX_BITSTREAM_COMPLETE_FRAME; } else @@ -456,7 +466,7 @@ static int qsv_decode(AVCodecContext *avctx, QSVContext *q, bs.Data = avpkt->data; bs.DataLength = avpkt->size; bs.MaxLength = bs.DataLength; - bs.TimeStamp = avpkt->pts; + bs.TimeStamp = PTS_TO_MFX_PTS(avpkt->pts, avctx->pkt_timebase); if (avctx->field_order == AV_FIELD_PROGRESSIVE) bs.DataFlag |= MFX_BITSTREAM_COMPLETE_FRAME; } @@ -544,7 +554,7 @@ static int qsv_decode(AVCodecContext *avctx, QSVContext *q, outsurf = &out_frame->surface; - frame->pts = outsurf->Data.TimeStamp; + frame->pts = MFX_PTS_TO_PTS(outsurf->Data.TimeStamp, avctx->pkt_timebase); frame->repeat_pict = outsurf->Info.PicStruct & MFX_PICSTRUCT_FRAME_TRIPLING ? 4 : @@ -748,6 +758,9 @@ static av_cold int qsv_decode_init(AVCodecContext *avctx) goto fail; } + if (!avctx->pkt_timebase.num) + av_log(avctx, AV_LOG_WARNING, "Invalid pkt_timebase, passing timestamps as-is.\n"); + return 0; fail: qsv_decode_close(avctx); From patchwork Wed Jun 9 04:18:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 28182 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5e:c91a:0:0:0:0:0 with SMTP id z26csp479486iol; Tue, 8 Jun 2021 21:19:08 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy9xvDwBN5LtxkMHb+RJ/Siy4bqT93YMqGRKFcMRCGIiym1r9FrmQ0DLKpNbx1sMfXI8dCW X-Received: by 2002:a50:fb8f:: with SMTP id e15mr27823205edq.46.1623212348652; Tue, 08 Jun 2021 21:19:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1623212348; cv=none; d=google.com; s=arc-20160816; b=XuZgHTdebgG1d84f7PGM6x/aWnjqXB0JuZ7XPFk0FMK/XvaPY/F9ZXDaSumDSsyMRA 5PN/N+ijZX73pT2yqr/UayGw13S0n+P7yZowE5+BCAtJA2Dip/2D4eFqXIQx4z43Xrzf 1fzDlcnREqI16MCHUpeMLUdhuY5XiCVosW6EIi4txkLWkpxZ1jFD5y9xPtiofMRKzxQN uf9EAezu7qpXGsX5vFSrNdTNsterkF2D8RZKF8vJ0DR9JgWhBhUPUnbIVE2orEwbqEpd bpR4j/fSbvgDefgwUVR1y+xPmh3LSCBNEk2XSwUXmTC/kq/rusz6RL6QEqVC7kTBEFfO U8QQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:ironport-sdr:ironport-sdr:delivered-to; bh=NVKX1m5zExQqVqOp+jwC206+vrBCcRjvV4pA77Cy7Kg=; b=s5GkihIAtdMZy+az0yR2gIPThTMIfMjR/Wpa2idY8DNKo6p9DZt3z8g95scCjtefkB hYlrcBXdmJnT7sVRc8QDanO2+8mc3TxW7Htq7DHjjj/Zj3YIQGT1wyS+9+NuxTQO8fOr NqDeClQP3hvi8yr5ADB1IYmxG1mQsSp2yg4abjqzhl+C8br/qzvsGu3OMd5HbNuJazj0 AbE3bslfwZPBvAWFY1y+IV+PSc7n3Ng1zCKNvUijJ08nD3WbCBTORjq+KBm5kZuG7EAp tF0BlYm7avmpvIBcSHdMhSZvUTZQIaaFMOrdwfA0S2et0Mt7Gi8VLh7nPX2Lnj/X28iu oBVg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id m1si1375976ejj.214.2021.06.08.21.19.08; Tue, 08 Jun 2021 21:19:08 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5F7D4687FB2; Wed, 9 Jun 2021 07:19:00 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 82A7A680BBD for ; Wed, 9 Jun 2021 07:18:53 +0300 (EEST) IronPort-SDR: cblCq1VlSsUBrmeBTzNC5gnsKddmHmffokljy/aeLXHxS4PJrTAuJUr4gV/DANZd2tupXzp7iB s256Swq03OeA== X-IronPort-AV: E=McAfee;i="6200,9189,10009"; a="192108173" X-IronPort-AV: E=Sophos;i="5.83,260,1616482800"; d="scan'208";a="192108173" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2021 21:18:44 -0700 IronPort-SDR: hG6LXsr5j+Lg1PzdCZyYsLKVvZArTqspTF2rOEJLj5dXTtF711vBtJ+0bvauV2NbsXQ65lT9Wc 2m/DjZiR3Omw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.83,260,1616482800"; d="scan'208";a="485580305" Received: from xhh-tgl64.sh.intel.com ([10.239.159.53]) by fmsmga002.fm.intel.com with ESMTP; 08 Jun 2021 21:18:43 -0700 From: Haihao Xiang To: ffmpeg-devel@ffmpeg.org Date: Wed, 9 Jun 2021 12:18:30 +0800 Message-Id: <20210609041830.1302044-2-haihao.xiang@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210609041830.1302044-1-haihao.xiang@intel.com> References: <20210609041830.1302044-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] lavfi/vf_vpp_qsv: fix the time_base for outlink X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: U1U9emTLuq+G Since commit 89ffcd1, the pts on output pad is in the time base of the input link, not the time base of the output link when EOF is reached, so a filter after vpp_qsv might output some unexpected frames. In order to avoid this issue, use the same time base for input and ouput links The issue can be triggered with the command below: $> ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.h265 -vf "vpp_qsv=w=1920:h=1080,fps=fps=60" -f null - [out_0_0 @ 0x55eb017b5060] 100 buffers queued in out_0_0, something may be wrong. [out_0_0 @ 0x55eb017b5060] 1000 buffers queued in out_0_0, something may be wrong. [out_0_0 @ 0x55eb017b5060] 10000 buffers queued in out_0_0, something may be wrong. --- libavfilter/vf_vpp_qsv.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavfilter/vf_vpp_qsv.c b/libavfilter/vf_vpp_qsv.c index b9ab5c6490..74d1d51e7c 100644 --- a/libavfilter/vf_vpp_qsv.c +++ b/libavfilter/vf_vpp_qsv.c @@ -303,7 +303,7 @@ static int config_output(AVFilterLink *outlink) outlink->w = vpp->out_width; outlink->h = vpp->out_height; outlink->frame_rate = vpp->framerate; - outlink->time_base = av_inv_q(vpp->framerate); + outlink->time_base = inlink->time_base; param.filter_frame = NULL; param.num_ext_buf = 0;