From patchwork Mon Jul 8 10:34:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lance Wang X-Patchwork-Id: 13854 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 19AAE4489B8 for ; Mon, 8 Jul 2019 13:34:49 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E62CE68AF91; Mon, 8 Jul 2019 13:34:48 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f193.google.com (mail-pf1-f193.google.com [209.85.210.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1E23568AF8F for ; Mon, 8 Jul 2019 13:34:42 +0300 (EEST) Received: by mail-pf1-f193.google.com with SMTP id g2so2601472pfq.0 for ; Mon, 08 Jul 2019 03:34:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=ttROf+IW0efRThG9e3jtx3Phr8tbCiF7ZQ4vnoyP9kE=; b=GB6XZhtf28tGK4kXbj2guWcXRDoivUX26aU8uBUxeOV96ip+IFgHn8arUcNEiVgoZX wSICGMoHxT69SPDcUsgGUtftyDtoRHeZmymB2DjnoaYsPQeB32KxHEf02jJN4Rm7kM1j HeBj2xTDJe2h9R97PPHMsgn/fvSwuHi4C2snxtovOnmgjHzhfHbOLsJ/l/kvPdC7WUcW IYrjoxJxodhypYz6+9xhMo7ulTQidjNDyi0cdh2rjgK+yHuijygIYrgO13/tzl2dK8Xu jL4ySabeolw1v26XpjXGKqVBYDNvIcYBjb6EPb5rqVhpmKoW1vOm2QMzhcHArkJthcJx Z/LQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=ttROf+IW0efRThG9e3jtx3Phr8tbCiF7ZQ4vnoyP9kE=; b=PpcEHcS1rcGrNM4RqrhK0BwkQ47jc89JTasqiI8dRd4je3Uia3Oi8RQZXkpNey+C0C S1xnzAVrb0FXAsHUSQUn9IYCZ8989+sB8BQ18DVwPx4qzz/KFg1rDm/YNxlgAHoV4KQZ NdjGgKP++kWMDW6gHsI+HEQbDCvLQWZgAxDj3pTUTZBtElxPRSdGm88/2uWFElomNPEn wscibvzUWjEpaW/ade/6VsGN/TNDkfw86jAWATJXqNp8tVo4ysfyWqzu0FH1GrTlWIwT E5ej6ZzA+WlWm+b6IpN+OwegV1zl2dUdBEQrtELC172ucTVd9hORpzeMTrCSxYzW6jAD Cxuw== X-Gm-Message-State: APjAAAUWCy/W+OnOAzp5AmbrDWxW9oqWgMqzVf6wvlaI0ExWvfU/jYkk te5XGIeXBpWrAk5kDigMmHPuw88B X-Google-Smtp-Source: APXvYqwSrphr85FRD+VHPej7ALccWnrOwEPxPBdtEntAeeR0V05qfQIkE8hCzCI2N/bZ7EU5hGz88A== X-Received: by 2002:a17:90a:8985:: with SMTP id v5mr24105376pjn.136.1562582079885; Mon, 08 Jul 2019 03:34:39 -0700 (PDT) Received: from izj6c97garwn1o4y2zidq0z.localdomain ([47.90.99.151]) by smtp.gmail.com with ESMTPSA id r15sm17225727pfh.121.2019.07.08.03.34.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 08 Jul 2019 03:34:39 -0700 (PDT) From: lance.lmwang@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Mon, 8 Jul 2019 18:34:12 +0800 Message-Id: <1562582052-27161-1-git-send-email-lance.lmwang@gmail.com> X-Mailer: git-send-email 1.8.3.1 Subject: [FFmpeg-devel] [PATCH] fftools/ffmpeg: fix video frame lost sync for the dual input ts files with audio X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Limin Wang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Limin Wang How to reproduce the problem(use two ts files with audio stream): ffmpeg -i left_w_a.ts -i right_w_a.ts -filter_complex "hstack=inputs=2" -f null - With above command, the audio stream of the second input will be discarded default, however the start_time is initalized from the audio stream, so the old code will try to correct the ts offset in the condition which will cause the video frame out of sync and report some dts error like below: [null @ 0x7fa78c000c00] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 43 >= 43 [null @ 0x7fa78c000c00] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 44 >= 44 [null @ 0x7fa78c000c00] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 45 >= 45 [null @ 0x7fa78c000c00] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 46 >= 46 [null @ 0x7fa78c000c00] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 47 >= 47 Or you can try with below command to reproduce the issue: ffmpeg -i input_w_a.ts -i input_w_a.ts -filter_complex "ssim=f=ssim.log" -f null - You'll get below ssim result which isn't expected result for the two input is same, the SSIM should be 1.00000 SSIM Y:0.777353 (6.523822) U:0.964087 (14.447539) V:0.966739 (14.780661) All:0.840039 (7.959872) With the patch applied, the above testing command can get expected result. other solution are: use the setpts=PTS-STARTPTS filter or skip all audio stream to avoid the timestamp adjustment. Signed-off-by: Limin Wang --- fftools/ffmpeg.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 01f04103cf..3fed92639a 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -4378,13 +4378,16 @@ static int process_input(int file_index) && ifile->ts_offset == -is->start_time && (is->iformat->flags & AVFMT_TS_DISCONT)) { int64_t new_start_time = INT64_MAX; + int nb_used_streams = 0; + for (i=0; inb_streams; i++) { AVStream *st = is->streams[i]; if(st->discard == AVDISCARD_ALL || st->start_time == AV_NOPTS_VALUE) continue; + nb_used_streams++; new_start_time = FFMIN(new_start_time, av_rescale_q(st->start_time, st->time_base, AV_TIME_BASE_Q)); } - if (new_start_time > is->start_time) { + if (nb_used_streams > 1 && new_start_time > is->start_time) { av_log(is, AV_LOG_VERBOSE, "Correcting start time by %"PRId64"\n", new_start_time - is->start_time); ifile->ts_offset = -new_start_time; }