From patchwork Sun Jul 18 10:22:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hu Weiwen X-Patchwork-Id: 28966 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5d:965a:0:0:0:0:0 with SMTP id d26csp3441551ios; Sun, 18 Jul 2021 03:23:05 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxEW1R0O2zHpwRHpMwzQWu2vSDPqIZ+Z0X/rePJcCUnrxXjU9VpU1PRBDnDKtERBMBWL58Y X-Received: by 2002:a05:6402:49a:: with SMTP id k26mr27648340edv.279.1626603785380; Sun, 18 Jul 2021 03:23:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1626603785; cv=none; d=google.com; s=arc-20160816; b=ZiSjmO1/k5gFfFZeiPnC9LOWRbalkfPX/CJhcpiCVSyi0bGqdqeJHnhKsotGUR0u9o ogl++jJ1sI6W/WGKtYiljYFIhzi22GdyXS9p7C2QQ6rBnnBzGPkaFbXfVJ8yEqg59iG7 vMYOBP1P1G3oAYxQB5SuE5ffcAo3WaZpqrs+0Fwg/wu1OOE8VgD6JjYw+Jzy7sZJ8Idm AujUhVub82V0zNdI9nCRBsG7BM758/IQII/EgfNRNYVvAAK1ty0rW+8E5hyO//cgfP3v hssEbVB+uFcawZZLgNqjOH2B0mEZYpkCo/nd6TmPHtccHJ/qkl3EY+DgIqlbmR6bqfI3 rUyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=hPctjA0dJJ7aO/29571f56no0tAjS+USVAnhpOZ2q3k=; b=BfhjPN99cwQ5NXy1BayYgnLp/HzZuiAvn3Pdsh4mWeKsxovvnkQC4j0KYqyz6sSHrR s6x9v4nxDXx802oHjQpx08yZeQHPbxpU8mTGF6m5W4UQGYS9Rv0dJyHw2CPypnlqz2e5 0pWsGXwDAU1wE18dlq9rhPQZ5yggkPww/wFRjFLS74ha0LMfekX5nN6pVesEBMwD5UK1 F4PmliSFSdJfhimiYMyI0wZsVRAIt3Bo7Ga/42pdDjF1+wXplYZU/F6tYOHStyebQSWr lryvlLe9+n7n8q7ikjG8mKI+YRLJXIB+LmbC5VjcpRsMecBEqoW5EgcpZNHaukdTzMrs S02A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=scut.edu.cn Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id w25si16733311eja.725.2021.07.18.03.23.04; Sun, 18 Jul 2021 03:23:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=scut.edu.cn Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2089668AA6E; Sun, 18 Jul 2021 13:23:00 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.scut.edu.cn (stumail1.scut.edu.cn [202.38.213.20]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 38F20689AF5 for ; Sun, 18 Jul 2021 13:22:51 +0300 (EEST) Received: from dorm.huww98.cn (unknown [125.216.246.30]) by front (Coremail) with SMTP id AWSowACXn__gAPRgpPesAQ--.27124S4; Sun, 18 Jul 2021 18:22:25 +0800 (CST) From: Hu Weiwen To: ffmpeg-devel@ffmpeg.org Date: Sun, 18 Jul 2021 18:22:32 +0800 Message-Id: <20210718102232.1382376-1-sehuww@mail.scut.edu.cn> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-CM-TRANSID: AWSowACXn__gAPRgpPesAQ--.27124S4 X-Coremail-Antispam: 1UD129KBjvJXoW3Jr17KFWfCw13Jw1fWr4rKrg_yoW7trykpw srtF4Iy3W5tFWvvr1DA3s8XFW8GrWfJFsFqw1UWa4ftr9Ykrn5tryUKFWxtry5CryUGr4f ZFnYka4j9r42krDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUyG14x267AKxVWUJVW8JwAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2ocxC64kIII0Yj41l84x0c7CEw4AK67xGY2AK02 1l84ACjcxK6xIIjxv20xvE14v26r1j6r1xM28EF7xvwVC0I7IYx2IY6xkF7I0E14v26r1j 6r4UM28EF7xvwVC2z280aVAFwI0_Cr0_Gr1UM28EF7xvwVC2z280aVCY1x0267AKxVW8JV W8Jr1le2I262IYc4CY6c8Ij28IcVAaY2xG8wAqx4xG64xvF2IEw4CE5I8CrVC2j2WlYx0E 2Ix0cI8IcVAFwI0_Jr0_Jr4lYx0Ex4A2jsIE14v26r1j6r4UMcvjeVCFs4IE7xkEbVWUJV W8JwACjcxG0xvY0x0EwIxGrwACjI8F5VA0II8E6IAqYI8I648v4I1l42xK82IYc2Ij64vI r41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8Gjc xK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r1Y6r17MIIYrxkI7VAKI48JMIIF0xvE2Ix0 cI8IcVAFwI0_Jr0_JF4lIxAIcVC0I7IYx2IY6xkF7I0E14v26r1j6r4UMIIF0xvE42xK8V AvwI8IcIk0rVWrZr1j6s0DMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7Cj xVAFwI0_Jr0_GrUvcSsGvfC2KfnxnUUI43ZEXa7VUbrMaUUUUUU== X-CM-SenderInfo: qsqrljqqwxllyrt6zt1loo2ulxwovvfxof0/1tbiAQAEBlepTBR2qAAWst Subject: [FFmpeg-devel] [PATCH] avformat/movenc: Fix tfdt out of sync X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Hu Weiwen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 5cQk9Ffj0sFB Fix an edge case when auto flushing an empty fragment, and the previous fragment has inaccurate duration, the track->frag_start would be out of sync from input dts, and all subsequent tfdt box will have out of sync base_media_decode_time. This error can accumulate to quite large over long-running streaming. This can be easily reproduced by remux a variable frame rate source with dash muxer and set streaming to 1. When flushing a fragment, we may guess the duration if we have not got the next package. Then when we get the first packet of the next fragment, we intend to set the trk->cluster[0].dts to match the previously guessed duration. However, at this time trk->track_duration may already be updated from mov_write_single_packet() when auto flushing the empty fragment (e.g. if FF_MOV_FLAG_FRAG_EVERY_FRAME is set), but no moof was written to reflect this change. So the trk->cluster[0].dts do NOT actually match the previously written duration. Then, when flushing the next non-empty fragment, we will get a wrong "duration", and track->frag_start will be out of sync. And it cannot get back on track again. This error is expected to accumulate over time. trk->cluster[0].dts will only get too large. it will not get too small because we have check_pkt(), so track->frag_start will fall more and more behind from input dts. To fix this, we can directly remove the track->frag_start. It can always be replaced by "track->cluster[0].dts - track->start_dts". Signed-off-by: Hu Weiwen --- libavformat/movenc.c | 24 ++++-------------------- libavformat/movenc.h | 1 - 2 files changed, 4 insertions(+), 21 deletions(-) diff --git a/libavformat/movenc.c b/libavformat/movenc.c index c85efe87489..48736acab65 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -4482,8 +4482,7 @@ static int mov_write_tfxd_tag(AVIOContext *pb, MOVTrack *track) avio_write(pb, uuid, sizeof(uuid)); avio_w8(pb, 1); avio_wb24(pb, 0); - avio_wb64(pb, track->start_dts + track->frag_start + - track->cluster[0].cts); + avio_wb64(pb, track->cluster[0].dts + track->cluster[0].cts); avio_wb64(pb, track->end_pts - (track->cluster[0].dts + track->cluster[0].cts)); @@ -4562,8 +4561,7 @@ static int mov_add_tfra_entries(AVIOContext *pb, MOVMuxContext *mov, int tracks, info->size = size; // Try to recreate the original pts for the first packet // from the fields we have stored - info->time = track->start_dts + track->frag_start + - track->cluster[0].cts; + info->time = track->cluster[0].dts + track->cluster[0].cts; info->duration = track->end_pts - (track->cluster[0].dts + track->cluster[0].cts); // If the pts is less than zero, we will have trimmed @@ -4601,7 +4599,7 @@ static int mov_write_tfdt_tag(AVIOContext *pb, MOVTrack *track) ffio_wfourcc(pb, "tfdt"); avio_w8(pb, 1); /* version */ avio_wb24(pb, 0); - avio_wb64(pb, track->frag_start); + avio_wb64(pb, track->cluster[0].dts - track->start_dts); return update_size(pb, pos); } @@ -4678,8 +4676,7 @@ static int mov_write_sidx_tag(AVIOContext *pb, if (track->entry) { entries = 1; - presentation_time = track->start_dts + track->frag_start + - track->cluster[0].cts; + presentation_time = track->cluster[0].dts + track->cluster[0].cts; duration = track->end_pts - (track->cluster[0].dts + track->cluster[0].cts); starts_with_SAP = track->cluster[0].flags & MOV_SYNC_SAMPLE; @@ -5358,10 +5355,6 @@ static int mov_flush_fragment(AVFormatContext *s, int force) mov->moov_written = 1; mov->mdat_size = 0; for (i = 0; i < mov->nb_streams; i++) { - if (mov->tracks[i].entry) - mov->tracks[i].frag_start += mov->tracks[i].start_dts + - mov->tracks[i].track_duration - - mov->tracks[i].cluster[0].dts; mov->tracks[i].entry = 0; mov->tracks[i].end_reliable = 0; } @@ -5415,11 +5408,7 @@ static int mov_flush_fragment(AVFormatContext *s, int force) MOVTrack *track = &mov->tracks[i]; int buf_size, write_moof = 1, moof_tracks = -1; uint8_t *buf; - int64_t duration = 0; - if (track->entry) - duration = track->start_dts + track->track_duration - - track->cluster[0].dts; if (mov->flags & FF_MOV_FLAG_SEPARATE_MOOF) { if (!track->mdat_buf) continue; @@ -5439,8 +5428,6 @@ static int mov_flush_fragment(AVFormatContext *s, int force) ffio_wfourcc(s->pb, "mdat"); } - if (track->entry) - track->frag_start += duration; track->entry = 0; track->entries_flushed = 0; track->end_reliable = 0; @@ -5759,7 +5746,6 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) /* New fragment, but discontinuous from previous fragments. * Pretend the duration sum of the earlier fragments is * pkt->dts - trk->start_dts. */ - trk->frag_start = pkt->dts - trk->start_dts; trk->end_pts = AV_NOPTS_VALUE; trk->frag_discont = 0; } @@ -5781,12 +5767,10 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) /* Pretend the whole stream started at pts=0, with earlier fragments * already written. If the stream started at pts=0, the duration sum * of earlier fragments would have been pkt->pts. */ - trk->frag_start = pkt->pts; trk->start_dts = pkt->dts - pkt->pts; } else { /* Pretend the whole stream started at dts=0, with earlier fragments * already written, with a duration summing up to pkt->dts. */ - trk->frag_start = pkt->dts; trk->start_dts = 0; } trk->frag_discont = 0; diff --git a/libavformat/movenc.h b/libavformat/movenc.h index af1ea0bce64..daeaad1cc68 100644 --- a/libavformat/movenc.h +++ b/libavformat/movenc.h @@ -138,7 +138,6 @@ typedef struct MOVTrack { AVIOContext *mdat_buf; int64_t data_offset; - int64_t frag_start; int frag_discont; int entries_flushed;