From patchwork Tue Jun 6 18:16:01 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sasi Inguva X-Patchwork-Id: 3859 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.10.2 with SMTP id 2csp1979711vsk; Tue, 6 Jun 2017 11:17:22 -0700 (PDT) X-Received: by 10.223.163.198 with SMTP id m6mr18835922wrb.58.1496773042539; Tue, 06 Jun 2017 11:17:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1496773042; cv=none; d=google.com; s=arc-20160816; b=PAxA0o3+X8/XDiCCKqSe1TtD4MwzUjoGSNCDOekbincAHXHjFK1mqREkswx3hXCBZE eLASVh5OFQlCW9fIe9vUPUfpbSvEKdXqJ4m8bF9hiuxgculV+YMUY+NcHJpWyvGTW5op 6pVQJn5mbNHNOu0kZlstOBYgOPhXvuxoEg4ctZ75n5cy8F4uACnpzaGHn/iSDEx1HSdW fgTAAyxLmV3Av2cmNFvd6Mi0VSOLYA860jfXSiGhbjqe3w1N9pfms/8iGLzXpz1SzSL5 IRWzpnGMAaFX48xfpSlSZ40FLhohJY8Qf/asd+C9tCXrtIJGpv3eiFxmr/ceCpYkRbLg pzcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=tMnL403m8S1TGeyeEpbG8a0AFoCxDsxGv9ShQE52tPI=; b=IlX94GTjrj5EZ6ereY4v8WmJywc1x9/IY6fnkbMqaD2WyXultWWujbupGtRHRNjlck NQ0GWaGJvseHEbIk9cIYfSKSErLqlKHGDvtOUANNMfMjPWQn5CDT8ymWpUDTa7YVKhzR zN1e/RIUwo2Ly9hg8WjoGqpvfIlap8lowyzd0hlinU/EXjP47uyLabdOUKzmWCZsXRzG J/uUGoRPpzZljU8mU72F0rzZW2VHhE/N0D7hC/xQde3PfB9fjbbt90XcGhthvEZcJnsU q72LgsjHkTg7cyw9JYa6DJaKK9fvbg+8e6gLEyxZpZ00E4OBLkNu66xwG6z3IwtDy29j J6MA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id k20si19193164wmc.3.2017.06.06.11.17.21; Tue, 06 Jun 2017 11:17:22 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 00F6B689E4D; Tue, 6 Jun 2017 21:17:11 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg0-f46.google.com (mail-pg0-f46.google.com [74.125.83.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0F1AE689AF8 for ; Tue, 6 Jun 2017 21:17:05 +0300 (EEST) Received: by mail-pg0-f46.google.com with SMTP id k71so4518247pgd.2 for ; Tue, 06 Jun 2017 11:17:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=eCM2MkCYmzSA/L8Cr9YKJzNo10NOho9SkxhuH9nD24k=; b=OsHS8oiMkeq8jZGifYkcVssprD52kgzlPmZLCJvCgZkurF9WwHi6vdpgHw7u3hjU1Y fAWqcVIKLxCjRilfpkPOU+FYnHykYF0JDx+KGvqJUYwdhcAgBQA8ZfROmadWxt7NTPpi RodpQYg1CRroKnZIxe9KozuxXb5QgqX/4vxyLxr1C28FNZzKqDAMrT8CrkWGRqmG45Fw ddiLbeuGfSF6WWXhdvxU25MDOaULfMje9yxJdFHAqou1MPWG0n9rGpJLt8GpDy4tj2Pu qcu++9R0rPzhchyv4s2X1dw4X+BMqx7kQpDF9b/iJCjoa4xtqetMpe7RaHhKkPuG8Tl7 hjuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=eCM2MkCYmzSA/L8Cr9YKJzNo10NOho9SkxhuH9nD24k=; b=H07idNnRifLgtVKMw3HlvG1AYSOZOgtmCuLe5/Mj2Whn0CtkKPAOaN4juqMejTU2E+ AgK/QSaOka4OigYNA9mhesYON+Jy114adoH2o56OYHVEE42zQBmDX6WguPUbuM07+X6x AlZHn/Cj8GbHyLmLsvzv1ND9Jv2WlHrL/9TaGJ8PkjkTLitLLgePoVh44m7w9MJn42HD seUMoKEoKFideKKQ9kPHvlth4lQVBs0vQpt/PWfyQQPI6M8BS1m4vubJx5KdTUb2WeXW FC8dMvZHH0DhfsmGmboFEvtDSCj/3xtL+PAiBfVB0Auxk8gTkyghkN0FIjAJ2KnFmNlY h3aQ== X-Gm-Message-State: AODbwcA9mozlgOQPc+ZFgYHJfxU4UzsP0VSfjGQNSz++pK5O6Atd1+nY /MhP+E08IqvXsS/h2dsjj5hs X-Received: by 10.84.128.1 with SMTP id 1mr23125735pla.244.1496773029659; Tue, 06 Jun 2017 11:17:09 -0700 (PDT) Received: from isasi.mtv.corp.google.com ([172.27.196.44]) by smtp.gmail.com with ESMTPSA id h84sm68018553pfh.45.2017.06.06.11.17.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 06 Jun 2017 11:17:08 -0700 (PDT) From: Sasi Inguva To: ffmpeg-devel@ffmpeg.org Date: Tue, 6 Jun 2017 11:16:01 -0700 Message-Id: <20170606181601.25187-1-isasi@google.com> X-Mailer: git-send-email 2.13.0.506.g27d5fe0cd-goog In-Reply-To: <20170606155155.7079721e@debian> References: <20170606155155.7079721e@debian> Subject: [FFmpeg-devel] [PATCH] lavf/mov.c: Offset index timestamps by the minimum pts to make first pts zero. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Sasi Inguva MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Fixes t/6421. If the videos starts with B frame, then the minimum composition time as computed by stts + ctts will be non-zero. Hence we need to shift the DTS, so that the first pts is zero. This was the intention of that code-block. However it was subtracting by the wrong amount. For example, for one of the videos in the bug nonFormatted.mp4 we have stts: sample_count duration 960 1001 ctts: sample_count duration 1 3003 2 0 1 3003 .... The resulting composition times are : 3003, 1001, 2002, 6006, ... The minimum composition time or PTS is 1001, which should be used to offset DTS. However the code block was wrongly using ctts[0] which is 3003. Hence the PTS was negative. This change computes the minimum pts encountered while fixing the index, and then subtracts it from all the timestamps after the edit list fixes are applied. fate-suite/h264/twofields_packet.mp4 is a similar file starting with 2 Bframes. Before this change the PTS of first two B-frames was -6006 and -3003, and I am guessing one of them got dropped when being decoded and remuxed to the framecrc before, and now it is not being dropped. Signed-off-by: Sasi Inguva --- libavformat/mov.c | 57 ++++++++++++++++++++++++---------- tests/ref/fate/h264-twofields-packet | 60 ++++++++++++++++++------------------ 2 files changed, 70 insertions(+), 47 deletions(-) diff --git a/libavformat/mov.c b/libavformat/mov.c index 3845e63b53..d7d64c3361 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -3039,6 +3039,9 @@ static void mov_fix_index(MOVContext *mov, AVStream *st) int64_t edit_list_dts_entry_end = 0; int64_t edit_list_start_ctts_sample = 0; int64_t curr_cts; + int64_t curr_ctts = 0; + int64_t min_corrected_pts = -1; + int64_t empty_edits_sum_duration = 0; int64_t edit_list_index = 0; int64_t index; int64_t index_ctts_count; @@ -3053,6 +3056,7 @@ static void mov_fix_index(MOVContext *mov, AVStream *st) int first_non_zero_audio_edit = -1; int packet_skip_samples = 0; MOVIndexRange *current_index_range; + int i; if (!msc->elst_data || msc->elst_count <= 0 || nb_old <= 0) { return; @@ -3080,13 +3084,9 @@ static void mov_fix_index(MOVContext *mov, AVStream *st) // If the dts_shift is positive (in case of negative ctts values in mov), // then negate the DTS by dts_shift - if (msc->dts_shift > 0) + if (msc->dts_shift > 0) { edit_list_dts_entry_end -= msc->dts_shift; - - // Offset the DTS by ctts[0] to make the PTS of the first frame 0 - if (ctts_data_old && ctts_count_old > 0) { - edit_list_dts_entry_end -= ctts_data_old[0].duration; - av_log(mov->fc, AV_LOG_DEBUG, "Offset DTS by ctts[%d].duration: %d\n", 0, ctts_data_old[0].duration); + av_log(mov->fc, AV_LOG_DEBUG, "Shifting DTS by %d because of negative CTTS.\n", msc->dts_shift); } start_dts = edit_list_dts_entry_end; @@ -3100,6 +3100,7 @@ static void mov_fix_index(MOVContext *mov, AVStream *st) edit_list_dts_entry_end += edit_list_duration; num_discarded_begin = 0; if (edit_list_media_time == -1) { + empty_edits_sum_duration += edit_list_duration; continue; } @@ -3179,11 +3180,13 @@ static void mov_fix_index(MOVContext *mov, AVStream *st) // frames (pts) before or after edit list curr_cts = current->timestamp + msc->dts_shift; + curr_ctts = 0; if (ctts_data_old && ctts_index_old < ctts_count_old) { - av_log(mov->fc, AV_LOG_DEBUG, "shifted frame pts, curr_cts: %"PRId64" @ %"PRId64", ctts: %d, ctts_count: %"PRId64"\n", - curr_cts, ctts_index_old, ctts_data_old[ctts_index_old].duration, ctts_count_old); - curr_cts += ctts_data_old[ctts_index_old].duration; + curr_ctts = ctts_data_old[ctts_index_old].duration; + av_log(mov->fc, AV_LOG_DEBUG, "stts: %"PRId64" ctts: %"PRId64", ctts_index: %"PRId64", ctts_count: %"PRId64"\n", + curr_cts, curr_ctts, ctts_index_old, ctts_count_old); + curr_cts += curr_ctts; ctts_sample_old++; if (ctts_sample_old == ctts_data_old[ctts_index_old].count) { if (add_ctts_entry(&msc->ctts_data, &msc->ctts_count, @@ -3244,14 +3247,21 @@ static void mov_fix_index(MOVContext *mov, AVStream *st) } } } - } else if (edit_list_start_encountered == 0) { - edit_list_start_encountered = 1; - // Make timestamps strictly monotonically increasing for audio, by rewriting timestamps for - // discarded packets. - if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO && frame_duration_buffer) { - fix_index_entry_timestamps(st, st->nb_index_entries, edit_list_dts_counter, - frame_duration_buffer, num_discarded_begin); - av_freep(&frame_duration_buffer); + } else { + if (min_corrected_pts < 0) { + min_corrected_pts = edit_list_dts_counter + curr_ctts + msc->dts_shift; + } else { + min_corrected_pts = FFMIN(min_corrected_pts, edit_list_dts_counter + curr_ctts + msc->dts_shift); + } + if (edit_list_start_encountered == 0) { + edit_list_start_encountered = 1; + // Make timestamps strictly monotonically increasing for audio, by rewriting timestamps for + // discarded packets. + if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO && frame_duration_buffer) { + fix_index_entry_timestamps(st, st->nb_index_entries, edit_list_dts_counter, + frame_duration_buffer, num_discarded_begin); + av_freep(&frame_duration_buffer); + } } } @@ -3292,6 +3302,19 @@ static void mov_fix_index(MOVContext *mov, AVStream *st) } } } + // If there are empty edits, then min_corrected_pts might be positive intentionally. So we subtract the + // sum duration of emtpy edits here. + min_corrected_pts -= empty_edits_sum_duration; + + // If the minimum pts turns out to be greater than zero after fixing the index, then we subtract the + // dts by that amount to make the first pts zero. + if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO && min_corrected_pts > 0) { + av_log(mov->fc, AV_LOG_DEBUG, "Offset DTS by %"PRId64" to make first pts zero.\n", min_corrected_pts); + for (i = 0; i < st->nb_index_entries; ++i) { + st->index_entries[i].timestamp -= min_corrected_pts; + } + } + // Update av stream length st->duration = edit_list_dts_entry_end - start_dts; diff --git a/tests/ref/fate/h264-twofields-packet b/tests/ref/fate/h264-twofields-packet index 4cff0a15bd..db12498f15 100644 --- a/tests/ref/fate/h264-twofields-packet +++ b/tests/ref/fate/h264-twofields-packet @@ -3,33 +3,33 @@ #codec_id 0: rawvideo #dimensions 0: 1920x1080 #sar 0: 1/1 -0, 0, 0, 1, 3110400, 0x40d65f69 -0, 1, 1, 1, 3110400, 0xdcbc50bf -0, 2, 2, 1, 3110400, 0x73a2276a -0, 3, 3, 1, 3110400, 0x84a2b3c6 -0, 4, 4, 1, 3110400, 0x7cf3b570 -0, 5, 5, 1, 3110400, 0xa2d1e03a -0, 6, 6, 1, 3110400, 0x03220fb1 -0, 7, 7, 1, 3110400, 0x89cd526a -0, 8, 8, 1, 3110400, 0xbb4b7531 -0, 9, 9, 1, 3110400, 0x0a69f053 -0, 10, 10, 1, 3110400, 0x0187994b -0, 11, 11, 1, 3110400, 0x26ed49fa -0, 12, 12, 1, 3110400, 0xbe8966d4 -0, 13, 13, 1, 3110400, 0x248d203c -0, 14, 14, 1, 3110400, 0x3139c754 -0, 15, 15, 1, 3110400, 0xf22380c4 -0, 16, 16, 1, 3110400, 0x3e00dcc1 -0, 17, 17, 1, 3110400, 0x8cbe2483 -0, 18, 18, 1, 3110400, 0x6951cd63 -0, 19, 19, 1, 3110400, 0x36aca4c5 -0, 20, 20, 1, 3110400, 0x4d4f6fbe -0, 21, 21, 1, 3110400, 0x997247aa -0, 22, 22, 1, 3110400, 0x0fd40e06 -0, 23, 23, 1, 3110400, 0xa10d2d67 -0, 24, 24, 1, 3110400, 0x87c481da -0, 25, 25, 1, 3110400, 0xe3dca3cd -0, 26, 26, 1, 3110400, 0x5f77b078 -0, 27, 27, 1, 3110400, 0xf1ddd098 -0, 28, 28, 1, 3110400, 0xedcd1754 -0, 29, 29, 1, 3110400, 0x14ac7153 +0, 0, 0, 1, 3110400, 0x48e39acd +0, 1, 1, 1, 3110400, 0x40d65f69 +0, 2, 2, 1, 3110400, 0xdcbc50bf +0, 3, 3, 1, 3110400, 0x73a2276a +0, 4, 4, 1, 3110400, 0x84a2b3c6 +0, 5, 5, 1, 3110400, 0x7cf3b570 +0, 6, 6, 1, 3110400, 0xa2d1e03a +0, 7, 7, 1, 3110400, 0x03220fb1 +0, 8, 8, 1, 3110400, 0x89cd526a +0, 9, 9, 1, 3110400, 0xbb4b7531 +0, 10, 10, 1, 3110400, 0x0a69f053 +0, 11, 11, 1, 3110400, 0x0187994b +0, 12, 12, 1, 3110400, 0x26ed49fa +0, 13, 13, 1, 3110400, 0xbe8966d4 +0, 14, 14, 1, 3110400, 0x248d203c +0, 15, 15, 1, 3110400, 0x3139c754 +0, 16, 16, 1, 3110400, 0xf22380c4 +0, 17, 17, 1, 3110400, 0x3e00dcc1 +0, 18, 18, 1, 3110400, 0x8cbe2483 +0, 19, 19, 1, 3110400, 0x6951cd63 +0, 20, 20, 1, 3110400, 0x36aca4c5 +0, 21, 21, 1, 3110400, 0x4d4f6fbe +0, 22, 22, 1, 3110400, 0x997247aa +0, 23, 23, 1, 3110400, 0x0fd40e06 +0, 24, 24, 1, 3110400, 0xa10d2d67 +0, 25, 25, 1, 3110400, 0x87c481da +0, 26, 26, 1, 3110400, 0xe3dca3cd +0, 27, 27, 1, 3110400, 0x5f77b078 +0, 28, 28, 1, 3110400, 0xf1ddd098 +0, 29, 29, 1, 3110400, 0xedcd1754