From patchwork Tue Jan 3 10:22:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gyan Doshi X-Patchwork-Id: 39836 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:bc95:b0:ad:ade2:bfd2 with SMTP id fx21csp7962942pzb; Tue, 3 Jan 2023 02:23:00 -0800 (PST) X-Google-Smtp-Source: AMrXdXuIFBqig0Y20+Ah4sDSKY1oOsjupTXBAayGZa+APkbNPvu5Tp7kZXiFpKek7FtfyVL0xvt6 X-Received: by 2002:a17:906:2509:b0:7c0:f2c5:ac3f with SMTP id i9-20020a170906250900b007c0f2c5ac3fmr35000717ejb.58.1672741380720; Tue, 03 Jan 2023 02:23:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1672741380; cv=none; d=google.com; s=arc-20160816; b=EuQZndt91mYG6QFGbynDK8fklig1jZxEe5lLkZ/QHbtXb9i4LqYUOh1jLqcCcRNt9z 0VVgilkPeEH2EXNt09R56XbUcKOzxbDfw7/5zNWQIb7YHGls9RMwFw/FQbnI/SkKVA4v oOAWNJ8eQKkzxShNabtn3o/HnljVB69VIoL2msQ+H9FiGuIQZ3L2xfS1xNwrbBxB9VTl qg4HLC2pyHaSvpTS9zLRmUd51Q6MIraSzUALhmzQVwDjcTlisykB6noYUa5N5TOJHr9p RgCMso4IwUCta3D3U85MOZpCr+Dic69ETthmeNI4eg1nMmU59nPHughkd12PnUGFDhCu fcZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=3c8/sAnz6UttAe5kZKvt5SabgClUQVhqUMPvig4Sjf0=; b=GNujgyRcDheC+NBfTmoGA6g0yGySh+Qkk7k+Gm/i7+hzrI4EHI4pBQbQNXun9LCejz UQ4IY61cnWxtCFX2GbD7RahQ+RLqEJjCs1Z1TUSTDacPEYEg1An215u5+WRCtIhTF1of wfgLRUSuRPoEa39nvfflQIctgF2k0c8fdU/usvrEGMChgdoTYWHE4ewBQfKe2nlnaSgz 2VCLzvOVuee2JcuQEwj5R0EPS1UVAIA8TQmeI8tt6JbRDUuYBkB4/9dbOmZ8tdJ6XgT0 i8FqiYKhZqQjmhyXY187rxJJXKNIwNhKlc8XA1LFpC9DTVmDUpsC4IgxDaHwgFfWCTKa xrNw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id go31-20020a1709070d9f00b0078266dc4b8csi29126694ejc.719.2023.01.03.02.23.00; Tue, 03 Jan 2023 02:23:00 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 841F068BD1D; Tue, 3 Jan 2023 12:22:57 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mout-p-201.mailbox.org (mout-p-201.mailbox.org [80.241.56.171]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3E0ED68BCE8 for ; Tue, 3 Jan 2023 12:22:50 +0200 (EET) Received: from smtp1.mailbox.org (smtp1.mailbox.org [10.196.197.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-p-201.mailbox.org (Postfix) with ESMTPS id 4NmTMB6GXkz9sTd for ; Tue, 3 Jan 2023 11:22:46 +0100 (CET) From: Gyan Doshi To: ffmpeg-devel@ffmpeg.org Date: Tue, 3 Jan 2023 15:52:17 +0530 Message-Id: <20230103102217.1074-1-ffmpeg@gyani.pro> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] ffmpeg: apply discontinuity adjustment per-stream X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: eGsGHdN1lqrQ At present, the offset for discontinuity adjustment is applied per-file but the check for discontinuity is intra-stream so the same discontinuity when seen in multiple streams with copyts, leads to compounded adjustment of the discontinuity offset. This introduces gaps in streams, leading to loss of sync or even de facto loss of stream. The ts_offset_discont parameter is transferred to InputStream and adjusted based on intra-stream gaps. --- Here's a simulated example to demonstrate the issue: 1) generate input ffmpeg -f lavfi -i nullsrc=s=32x32:r=1,format=gray -f lavfi -i anullsrc=cl=mono:r=8000 -c:v libx264 -preset superfast -bf 0 -crf 50 -c:a aac -b:a 64k -muxpreload 0 -muxdelay 0 -output_ts_offset 95000 -t 100000 src-100000s.ts 2) demo ffmpeg -copyts -i "src-100000.ts" -vf "showinfo=checksum=0" -af "ashowinfo" -f null - -report ==Before patch== timestamp discontinuity for stream #0:1 (id=257, type=audio): -95443717689, new offset= 95443717689 timestamp discontinuity for stream #0:0 (id=256, type=video): -95443717689, new offset= 190887435378 [Parsed_ashowinfo_0 @ 000002700af98800] n:0 pts:759998976 pts_time:94999.9 [Parsed_ashowinfo_0 @ 000002700af98800] n:1 pts:760000000 pts_time:95000 ... [Parsed_ashowinfo_0 @ 000002700af98800] n:745202 pts:1523085824 pts_time:190386 [Parsed_ashowinfo_0 @ 000002700af98800] n:745203 pts:1523086848 pts_time:190386 [Parsed_ashowinfo_0 @ 000002700af98800] n:745204 pts:2286637614 pts_time:285830 [Parsed_ashowinfo_0 @ 000002700af98800] n:745205 pts:2286638638 pts_time:285830 [Parsed_showinfo_0 @ 000002700ec34700] n: 0 pts:8550000000 pts_time:95000 [Parsed_showinfo_0 @ 000002700ec34700] n: 1 pts:8550090000 pts_time:95001 ... [Parsed_showinfo_0 @ 000002700ec34700] n:95380 pts:17134200000 pts_time:190380 [Parsed_showinfo_0 @ 000002700ec34700] n:95381 pts:17134290000 pts_time:190381 [Parsed_showinfo_0 @ 000002700ec34700] n:95382 pts:25724314592 pts_time:285826 [Parsed_showinfo_0 @ 000002700ec34700] n:95383 pts:25724404592 pts_time:285827 ==After patch== timestamp discontinuity for stream #0:1 (id=257, type=audio): -95443717689, new stream offset= 95443717689 timestamp discontinuity for stream #0:0 (id=256, type=video): -95443717689, new stream offset= 95443717689 [Parsed_ashowinfo_0 @ 0000023c773c8880] n:0 pts:759998976 pts_time:94999.9 [Parsed_ashowinfo_0 @ 0000023c773c8880] n:1 pts:760000000 pts_time:95000 ... [Parsed_ashowinfo_0 @ 0000023c773c8880] n:745202 pts:1523085824 pts_time:190386 [Parsed_ashowinfo_0 @ 0000023c773c8880] n:745203 pts:1523086848 pts_time:190386 [Parsed_ashowinfo_0 @ 0000023c773c8880] n:745204 pts:1523087872 pts_time:190386 [Parsed_ashowinfo_0 @ 0000023c773c8880] n:745205 pts:1523088896 pts_time:190386 [Parsed_showinfo_0 @ 0000023c795d07c0] n: 0 pts:8550000000 pts_time:95000 [Parsed_showinfo_0 @ 0000023c795d07c0] n: 1 pts:8550090000 pts_time:95001 ... [Parsed_showinfo_0 @ 0000023c795d07c0] n:95380 pts:17134200000 pts_time:190380 [Parsed_showinfo_0 @ 0000023c795d07c0] n:95381 pts:17134290000 pts_time:190381 [Parsed_showinfo_0 @ 0000023c795d07c0] n:95382 pts:17134380000 pts_time:190382 [Parsed_showinfo_0 @ 0000023c795d07c0] n:95383 pts:17134470000 pts_time:190383 fftools/ffmpeg.c | 12 ++++++------ fftools/ffmpeg.h | 8 ++++---- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 881d6f0af2..b967055059 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -3418,13 +3418,13 @@ static void ts_discontinuity_detect(InputFile *ifile, InputStream *ist, if (fmt_is_discont) { if (FFABS(delta) > 1LL * dts_delta_threshold * AV_TIME_BASE || pkt_dts + AV_TIME_BASE/10 < FFMAX(ist->pts, ist->dts)) { - ifile->ts_offset_discont -= delta; + ist->ts_offset_discont -= delta; av_log(NULL, AV_LOG_DEBUG, "timestamp discontinuity for stream #%d:%d " - "(id=%d, type=%s): %"PRId64", new offset= %"PRId64"\n", + "(id=%d, type=%s): %"PRId64", new stream offset= %"PRId64"\n", ist->file_index, ist->st->index, ist->st->id, av_get_media_type_string(ist->par->codec_type), - delta, ifile->ts_offset_discont); + delta, ist->ts_offset_discont); pkt->dts -= av_rescale_q(delta, AV_TIME_BASE_Q, ist->st->time_base); if (pkt->pts != AV_NOPTS_VALUE) pkt->pts -= av_rescale_q(delta, AV_TIME_BASE_Q, ist->st->time_base); @@ -3447,10 +3447,10 @@ static void ts_discontinuity_detect(InputFile *ifile, InputStream *ist, fmt_is_discont && ifile->last_ts != AV_NOPTS_VALUE) { int64_t delta = pkt_dts - ifile->last_ts; if (FFABS(delta) > 1LL * dts_delta_threshold * AV_TIME_BASE) { - ifile->ts_offset_discont -= delta; + ist->ts_offset_discont -= delta; av_log(NULL, AV_LOG_DEBUG, "Inter stream timestamp discontinuity %"PRId64", new offset= %"PRId64"\n", - delta, ifile->ts_offset_discont); + delta, ist->ts_offset_discont); pkt->dts -= av_rescale_q(delta, AV_TIME_BASE_Q, ist->st->time_base); if (pkt->pts != AV_NOPTS_VALUE) pkt->pts -= av_rescale_q(delta, AV_TIME_BASE_Q, ist->st->time_base); @@ -3463,7 +3463,7 @@ static void ts_discontinuity_detect(InputFile *ifile, InputStream *ist, static void ts_discontinuity_process(InputFile *ifile, InputStream *ist, AVPacket *pkt) { - int64_t offset = av_rescale_q(ifile->ts_offset_discont, AV_TIME_BASE_Q, + int64_t offset = av_rescale_q(ist->ts_offset_discont, AV_TIME_BASE_Q, ist->st->time_base); // apply previously-detected timestamp-discontinuity offset diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 5527dbe49b..d7bba8a190 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -352,6 +352,10 @@ typedef struct InputStream { int64_t next_pts; ///< synthetic pts for the next decode frame (in AV_TIME_BASE units) int64_t pts; ///< current pts of the decoded frame (in AV_TIME_BASE units) int wrap_correction_done; + /** + * Extra timestamp offset added by discontinuity handling. + */ + int64_t ts_offset_discont; // the value of AVCodecParserContext.repeat_pict from the AVStream parser // for the last packet returned from ifile_get_packet() @@ -444,10 +448,6 @@ typedef struct InputFile { */ int64_t start_time_effective; int64_t ts_offset; - /** - * Extra timestamp offset added by discontinuity handling. - */ - int64_t ts_offset_discont; int64_t last_ts; int64_t start_time; /* user-specified start time in AV_TIME_BASE or AV_NOPTS_VALUE */ int64_t recording_time;