From patchwork Fri Jun 11 02:19:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 28200 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5e:c91a:0:0:0:0:0 with SMTP id z26csp38546iol; Thu, 10 Jun 2021 19:20:10 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyNhjYXv+sL5Dw99v7fcHfoTi8TcFyfSTPYUYaKvzHk3yPEAwPvGStt/HcV7miv5hUgobfo X-Received: by 2002:a17:907:2165:: with SMTP id rl5mr1300673ejb.98.1623378010116; Thu, 10 Jun 2021 19:20:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1623378010; cv=none; d=google.com; s=arc-20160816; b=DRo8NFv+pAea4zImk5lcNx+HXOOe1ewhUcB7KHA/h7JBCyRQuj1RVewV5Glbr38DP1 2LE8XhVHDnfOTi+Mxo98idgKC3xFexJefZqBwxDzO3DHpPXN5VtdeldYttkNL1Al7RSZ i53wNHCVVlqycHtrOljBiJi6FAXkiozrTqwqfHTsiekvPqlymSnUPsNNR8RaRRV2a5Nc P7ZglDtaawMuLvH4uhhSrmFJSoGPPQAk4DnNrpONDwPk13M9Vsbc1aHTlbNL0p4LBq08 WcxaA7oMpNwZ4s1GySTES1hJOMEmuiuqxEz4cZBfe8JxhALB730/Bw+0G0hG9uf5QZ98 53sA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :ironport-sdr:ironport-sdr:delivered-to; bh=YUrUGSID96OlmUAdPpkPvJyiHrDDFdXVufJ4j/OF8ZU=; b=AxNSWO+X0uh6jpP+tVORlIeRn1zWMp8pS8cquH8M1SUSN9DesnFUzce6SXey54gauc qVkQK/By9b86Z6cxvZtut7cb77ugtR9s0Ysds+piDCAcYB3oCi8d5DDx+Tjm6+XNOPXc Hq94YWWVU8AGOspLrh7PCs7G6tM09DPG6Ixv/Wo5Y8BNfUpEwU+ofSuHqGJ7SSzDS7M3 fRVJQNIetyJnB82xVcVVdxbH0cVIt/yDNKMXR+XaypdQ5ELWXuBLDTaPcrJxLeiYgez+ Et5amfsROlFUQOyow8FBgDVUuwSFSizJQalICoKAb6N1Il/28wig5A604WLueB5Dn/oi cg8A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id c12si4033423edx.372.2021.06.10.19.20.08; Thu, 10 Jun 2021 19:20:10 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 037DA6806FE; Fri, 11 Jun 2021 05:20:04 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 95C6E68049A for ; Fri, 11 Jun 2021 05:19:55 +0300 (EEST) IronPort-SDR: d4It2+JceV+FcM3KYFmPiAizOBbRuj2g0eDzdhBJubZVs2isBR8EdqX8+A1AIn8muIWwAjwtLM LucIVzlo7rSw== X-IronPort-AV: E=McAfee;i="6200,9189,10011"; a="205407445" X-IronPort-AV: E=Sophos;i="5.83,265,1616482800"; d="scan'208";a="205407445" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Jun 2021 19:19:51 -0700 IronPort-SDR: kk8LG3PpdP/dP/JmXWldsbyFrUcFEpRhb7ubnzjnTAH0YhBlm6+S4FiLmN7XP834zr/qdKz3+m AJGp7uPE6t1Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.83,265,1616482800"; d="scan'208";a="638551338" Received: from xhh-tgl64.sh.intel.com ([10.239.159.53]) by fmsmga005.fm.intel.com with ESMTP; 10 Jun 2021 19:19:50 -0700 From: Haihao Xiang To: ffmpeg-devel@ffmpeg.org Date: Fri, 11 Jun 2021 10:19:45 +0800 Message-Id: <20210611021946.1437868-1-haihao.xiang@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 1/2] lavc/qsvdec: fix pts X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: jZGYNADebrLY The time base used for compressed bitstream and video frame in the SDK is { 1, 90000 }. [1][2] This can avoid the error message below from the muxer. $> ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.h265 -f null - ... [null @ 0x561c24f6f2f0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 2 >= 2 [1]https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.md#mfxbitstream [2]https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.md#mfxframedata --- libavcodec/qsvdec.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index f543defb18..622750927c 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -46,6 +46,16 @@ #include "qsv.h" #include "qsv_internal.h" +static const AVRational mfx_tb = { 1, 90000 }; + +#define PTS_TO_MFX_PTS(pts, pts_tb) ((pts) == AV_NOPTS_VALUE ? \ + MFX_TIMESTAMP_UNKNOWN : pts_tb.num ? \ + av_rescale_q(pts, pts_tb, mfx_tb) : pts) + +#define MFX_PTS_TO_PTS(mfx_pts, pts_tb) ((mfx_pts) == MFX_TIMESTAMP_UNKNOWN ? \ + AV_NOPTS_VALUE : pts_tb.num ? \ + av_rescale_q(mfx_pts, mfx_tb, pts_tb) : mfx_pts) + typedef struct QSVContext { // the session used for decoding mfxSession session; @@ -308,7 +318,7 @@ static int qsv_decode_header(AVCodecContext *avctx, QSVContext *q, bs.Data = avpkt->data; bs.DataLength = avpkt->size; bs.MaxLength = bs.DataLength; - bs.TimeStamp = avpkt->pts; + bs.TimeStamp = PTS_TO_MFX_PTS(avpkt->pts, avctx->pkt_timebase); if (avctx->field_order == AV_FIELD_PROGRESSIVE) bs.DataFlag |= MFX_BITSTREAM_COMPLETE_FRAME; } else @@ -456,7 +466,7 @@ static int qsv_decode(AVCodecContext *avctx, QSVContext *q, bs.Data = avpkt->data; bs.DataLength = avpkt->size; bs.MaxLength = bs.DataLength; - bs.TimeStamp = avpkt->pts; + bs.TimeStamp = PTS_TO_MFX_PTS(avpkt->pts, avctx->pkt_timebase); if (avctx->field_order == AV_FIELD_PROGRESSIVE) bs.DataFlag |= MFX_BITSTREAM_COMPLETE_FRAME; } @@ -544,7 +554,7 @@ static int qsv_decode(AVCodecContext *avctx, QSVContext *q, outsurf = &out_frame->surface; - frame->pts = outsurf->Data.TimeStamp; + frame->pts = MFX_PTS_TO_PTS(outsurf->Data.TimeStamp, avctx->pkt_timebase); frame->repeat_pict = outsurf->Info.PicStruct & MFX_PICSTRUCT_FRAME_TRIPLING ? 4 : @@ -748,6 +758,9 @@ static av_cold int qsv_decode_init(AVCodecContext *avctx) goto fail; } + if (!avctx->pkt_timebase.num) + av_log(avctx, AV_LOG_WARNING, "Invalid pkt_timebase, passing timestamps as-is.\n"); + return 0; fail: qsv_decode_close(avctx); From patchwork Fri Jun 11 02:19:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 28201 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5e:c91a:0:0:0:0:0 with SMTP id z26csp38670iol; Thu, 10 Jun 2021 19:20:21 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwg9Cy+64AszCtLY6R2aY8AlWyqrcGsX1RwmbZ2CCJztizhj/1fhP+6L8cJfBygB1kWGAgw X-Received: by 2002:a17:906:4109:: with SMTP id j9mr1354186ejk.250.1623378021739; Thu, 10 Jun 2021 19:20:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1623378021; cv=none; d=google.com; s=arc-20160816; b=ERPQt6xnyFeM6PLAhERt3iHnUo4+QlR1ZCBy6sifkLwWGFaxxl7anqCs58cw9TUy6o g71Lh5blx3IN78V76PHItN4u5+6dCAk+hhdNqorGZ/1sPfhMNtvDrXuq+irFDXQCZpo2 jXYc1Ikz5un8Y/WibWDqBe4cp75ZXQ3B9wmrgC2bwgNqgd1M7poQDfVqVxtX84OGWIvo Gj4DlP+IhyUf0m22L/3uT2aX94WuOghNqLRPgbYAcswJUfVmvNdMZn5RdEkrIxrf/Cmf vC21ZY3sztk2Ngk8O1n6WR1/vrZ5TuZLP3kmGgI1Df1HGdZh619M3EF3Pr/H0/wHVwMM MYyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:ironport-sdr:ironport-sdr:delivered-to; bh=aFodLmlA6MB9v6150p7FXFqbtc2V0zgm7AQ5QAB4SBs=; b=n7NU/cGhrDphv5xw5ytTU5hkb3Lg60GT4IF1Q58cr43zuyZ+wqnkG8xKYxe7QGzphT JREUmFeYI5c6PYnWldcYhJse+l2MZJcWbh0bRI0SNkquAc9yD2XVaeCOMK/fDR8o2pH7 SK9kVPZHbNs7Uf3L6wdv0TTAB7y2fn2i/nPtwWVU9dgoipqFVUHEff3mekyYaLv+6a6G 6HjMrHJJ5p7ZsaTVPzWiUzRAfC9WLD/C/M5KseaqEleOH499VpyVtCc15f0L2EGvsAsN /c4ZoZ6X3gvEI/rg+rewFPPEv0pIjkjoiBIt8GmtDldyivedzSA3tMKQOocG9Jeer056 MpSg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id j10si3629793ejv.206.2021.06.10.19.20.19; Thu, 10 Jun 2021 19:20:21 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 30C7B687FB2; Fri, 11 Jun 2021 05:20:05 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 60E7B68049A for ; Fri, 11 Jun 2021 05:19:57 +0300 (EEST) IronPort-SDR: v7HElIWlrdcqiHSbIix/cnf5Dxq3iLYidrbcSZUB91Cs+Bpndxka7g04scYIHQsC2nSSbqG5fE YltvWGZGFmEQ== X-IronPort-AV: E=McAfee;i="6200,9189,10011"; a="205407455" X-IronPort-AV: E=Sophos;i="5.83,265,1616482800"; d="scan'208";a="205407455" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Jun 2021 19:19:55 -0700 IronPort-SDR: 953mZ2h59SG/mUk5HJcYOUUB8qzKZBOSYz7kwR5fCbcwbVMuxAgm5oLN+hOnoHGPW8e66Sg1tW 9Dvefh7EQJ9g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.83,265,1616482800"; d="scan'208";a="638551354" Received: from xhh-tgl64.sh.intel.com ([10.239.159.53]) by fmsmga005.fm.intel.com with ESMTP; 10 Jun 2021 19:19:54 -0700 From: Haihao Xiang To: ffmpeg-devel@ffmpeg.org Date: Fri, 11 Jun 2021 10:19:46 +0800 Message-Id: <20210611021946.1437868-2-haihao.xiang@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210611021946.1437868-1-haihao.xiang@intel.com> References: <20210611021946.1437868-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 2/2] lavfi/vf_vpp_qsv: fix the time_base for outlink X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: R3J9q0x2c69B Since commit 89ffcd1, the status pts of the output link is set to a value in the input link time base, not in the output link time base when EOF is reached. Usually this pst value is larger than the required one because the output link time base is more greater than the input link time base. When "-vf vpp_qsv,fps" is used, user has to wait a long time for the ending of the pipeline because fps filter output a huge number of frames until the wrong status pts is hit. The issue can be triggered with the command below (use a clip with 1000 frames in this case): $> time ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.h265 -vf "vpp_qsv=w=1920:h=1080,fps=fps=30" -f null - ... [out_0_0 @ 0x564ccd27e020] 10000000 buffers queued in out_0_0, something may be wrong. frame=40119596 fps=88080 q=-0.0 Lsize=N/A time=371:28:39.96 bitrate=N/A speed=2.94e+03x video:17238889kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown real 9m7.451s user 2m34.102s sys 0m39.734s In order to avoid the above issue, the same time base for input and ouput links is used in this patch. Fixes ticket #9286 --- v2: update the commit log libavfilter/vf_vpp_qsv.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavfilter/vf_vpp_qsv.c b/libavfilter/vf_vpp_qsv.c index b9ab5c6490..74d1d51e7c 100644 --- a/libavfilter/vf_vpp_qsv.c +++ b/libavfilter/vf_vpp_qsv.c @@ -303,7 +303,7 @@ static int config_output(AVFilterLink *outlink) outlink->w = vpp->out_width; outlink->h = vpp->out_height; outlink->frame_rate = vpp->framerate; - outlink->time_base = av_inv_q(vpp->framerate); + outlink->time_base = inlink->time_base; param.filter_frame = NULL; param.num_ext_buf = 0;