From patchwork Tue Sep 27 23:12:22 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sasi Inguva X-Patchwork-Id: 759 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.140.66 with SMTP id o63csp118993vsd; Tue, 27 Sep 2016 16:12:35 -0700 (PDT) X-Received: by 10.28.196.14 with SMTP id u14mr5276483wmf.17.1475017955785; Tue, 27 Sep 2016 16:12:35 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id t129si5093698wmb.79.2016.09.27.16.12.34; Tue, 27 Sep 2016 16:12:35 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D6C62689F4B; Wed, 28 Sep 2016 02:12:20 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-vk0-f45.google.com (mail-vk0-f45.google.com [209.85.213.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A1824689C0E for ; Wed, 28 Sep 2016 02:12:13 +0300 (EEST) Received: by mail-vk0-f45.google.com with SMTP id v205so28203498vke.1 for ; Tue, 27 Sep 2016 16:12:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=/bARhyZ2mKUvYzif7ua9GK7DvUkPn/bWQmOTmo9j5Zg=; b=Gml4B4jxGGq8cefLLpG+szh6EtD5EgEFxcmgAbrjEs2AOHTjGT/xdyqQ9bkxg08M7z RHjqciAn11CYdTe/ZiNzSR1OUXWcvEP33G4ubpG//GFyrIAID9q23adHDEJ8HrwfkZNv Dh1vrB6E5IHlxxCitx1iYtSF6htzxwNRB+RIt7N3AsRgTYtQjSLKiMxjAooYEHhTjv5y hwbPmekRlBrlGcXhiOOT6tRMBO9MkZneYKmVUwlvlskyJQgJ/gPvpBt4UyeAFT5sLThi te7idczZuFKG4ESPtBVuQ5y+ImMHADOHqTZurpMHvLIHJsSlPN+kpewZ5gyyRCdjjAYG NvBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=/bARhyZ2mKUvYzif7ua9GK7DvUkPn/bWQmOTmo9j5Zg=; b=TSptEDP8xCksE5mn+l2F8VwVR8B9PU4Z5Gjr8evfeA0vBAH/WXvdZVh+lx03qKFJYp tNeJD4g/HLQKS2w/YPEn8HUeMDVDtWpMwJPHVxaxe9F3LOfn4vEjVaTu5onZQzHz70Rd SvRvczvO9F3n3eW1ZwOx7sd3EBJyRRU/HSUc2OVtgiXVvJDLxv4pRzSG+O+OFFwIqzj4 23RcAgjOIYXUI3yjdCnpf2ut+6jR8xZoz+OHZPEufi+uIYvwvdN9lgmjYfEg4oZckPid qcsesEI8OennsqeulCEOX9LtPKynvnRH1o4HS6tf54UMjQJ4FMayqNzd45vr0sX64HJS cqEA== X-Gm-Message-State: AA6/9RlX5fpnpkyUrp0tHAdwYIiYrAWg07uunl7+Nc/iePpmoWXKBdOwCdtn5zZT6P8GC7PSNH1kW/Mq8z1ikfSK X-Received: by 10.31.63.86 with SMTP id m83mr12258020vka.103.1475017944075; Tue, 27 Sep 2016 16:12:24 -0700 (PDT) MIME-Version: 1.0 Received: by 10.31.153.207 with HTTP; Tue, 27 Sep 2016 16:12:22 -0700 (PDT) In-Reply-To: <20160927190248.GA26091@sunshine.barsnick.net> References: <20160927132825.GD22530@sunshine.barsnick.net> <1474993693-2758-1-git-send-email-isasi@google.com> <20160927190248.GA26091@sunshine.barsnick.net> From: Sasi Inguva Date: Tue, 27 Sep 2016 16:12:22 -0700 Message-ID: To: FFmpeg development discussions and patches X-Content-Filtered-By: Mailman/MimeDel 2.1.20 Subject: Re: [FFmpeg-devel] [PATCH 2/3] lavf/mov.c: Make audio timestamps strictly monotonically increasing inside an edit list. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Corrected indentation. Don't know why the patch would not apply. I am attaching it as a file, if it helps. On Tue, Sep 27, 2016 at 12:02 PM, Moritz Barsnick wrote: > On Tue, Sep 27, 2016 at 09:28:13 -0700, Sasi Inguva wrote: > > if (curr_cts < edit_list_media_time || curr_cts >= > (edit_list_duration + edit_list_media_time)) { > > - if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO && > curr_cts < edit_list_media_time && > > - curr_cts + frame_duration > edit_list_media_time && > > - st->skip_samples == 0 && msc->start_pad == 0) { > > - st->skip_samples = msc->start_pad = > edit_list_media_time - curr_cts; > > - > > - // Shift the index entry timestamp by skip_samples > to be correct. > > - edit_list_dts_counter -= st->skip_samples; > > + if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO && > st->codecpar->codec_id != AV_CODEC_ID_VORBIS && > > + curr_cts < edit_list_media_time && curr_cts + > frame_duration > edit_list_media_time && > > + first_non_zero_audio_edit > 0) { > > + packet_skip_samples = edit_list_media_time - > curr_cts; > > + st->skip_samples += packet_skip_samples; > > + > > + // Shift the index entry timestamp by > packet_skip_samples to be correct. > > + edit_list_dts_counter -= packet_skip_samples; > > if (edit_list_start_encountered == 0) { > > - edit_list_start_encountered = 1; > > + edit_list_start_encountered = 1; > > + // Make timestamps strictly monotonically > increasing for audio, by rewriting timestamps for > > + // discarded packets. > > + if (frame_duration_buffer) { > > + fix_index_entry_timestamps(st, > st->nb_index_entries, edit_list_dts_counter, > > + > frame_duration_buffer, num_discarded_begin); > > + av_freep(&frame_duration_buffer); > > + } > > Something's fishy with your indentation here. > > > + if (!frame_duration_buffer) { > > + av_log(mov->fc, AV_LOG_ERROR, "Cannot > reallocate frame duration buffer\n"); > > + break; > > + } > > I fail to see the whole code block (and I have problems applying the > patch). What is this breaking from? > > Moritz > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > From cd2fa9c14a068165c8480b8ca72eccdbb8edd54b Mon Sep 17 00:00:00 2001 From: Sasi Inguva Date: Mon, 26 Sep 2016 09:56:26 -0700 Subject: [PATCH 2/3] lavf/mov.c: Make audio timestamps strictly monotonically increasing inside an edit list. Fixes gapless decoding. Adjust skip_samples field correctly in case of DISCARDed audio frames. Signed-off-by: Sasi Inguva --- libavformat/mov.c | 80 ++++++++++++++++++++++++---- tests/ref/fate/gaplessenc-itunes-to-ipod-aac | 2 +- tests/ref/fate/gaplessenc-pcm-to-mov-aac | 2 +- 3 files changed, 71 insertions(+), 13 deletions(-) diff --git a/libavformat/mov.c b/libavformat/mov.c index b84d9c0..8b7bbf1 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -2856,6 +2856,21 @@ static int64_t add_index_entry(AVStream *st, int64_t pos, int64_t timestamp, } /** + * Rewrite timestamps of index entries in the range [end_index - frame_duration_buffer_size, end_index) + * by subtracting end_ts successively by the amounts given in frame_duration_buffer. + */ +static void fix_index_entry_timestamps(AVStream* st, int end_index, int64_t end_ts, + int64_t* frame_duration_buffer, + int frame_duration_buffer_size) { + int i = 0; + av_assert0(end_index >= 0 && end_index <= st->nb_index_entries); + for (i = 0; i < frame_duration_buffer_size; i++) { + end_ts -= frame_duration_buffer[frame_duration_buffer_size - 1 - i]; + st->index_entries[end_index - 1 - i].timestamp = end_ts; + } +} + +/** * Append a new ctts entry to ctts_data. * Returns the new ctts_count if successful, else returns -1. */ @@ -2919,7 +2934,10 @@ static void mov_fix_index(MOVContext *mov, AVStream *st) int64_t edit_list_media_time_dts = 0; int64_t edit_list_start_encountered = 0; int64_t search_timestamp = 0; - + int64_t* frame_duration_buffer = NULL; + int num_discarded_begin = 0; + int first_non_zero_audio_edit = -1; + int packet_skip_samples = 0; if (!msc->elst_data || msc->elst_count <= 0) { return; @@ -2955,6 +2973,7 @@ static void mov_fix_index(MOVContext *mov, AVStream *st) edit_list_index++; edit_list_dts_counter = edit_list_dts_entry_end; edit_list_dts_entry_end += edit_list_duration; + num_discarded_begin = 0; if (edit_list_media_time == -1) { continue; } @@ -2962,7 +2981,14 @@ static void mov_fix_index(MOVContext *mov, AVStream *st) // If we encounter a non-negative edit list reset the skip_samples/start_pad fields and set them // according to the edit list below. if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) { - st->skip_samples = msc->start_pad = 0; + if (first_non_zero_audio_edit < 0) { + first_non_zero_audio_edit = 1; + } else { + first_non_zero_audio_edit = 0; + } + + if (first_non_zero_audio_edit > 0) + st->skip_samples = msc->start_pad = 0; } //find closest previous key frame @@ -3041,24 +3067,56 @@ static void mov_fix_index(MOVContext *mov, AVStream *st) } if (curr_cts < edit_list_media_time || curr_cts >= (edit_list_duration + edit_list_media_time)) { - if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO && curr_cts < edit_list_media_time && - curr_cts + frame_duration > edit_list_media_time && - st->skip_samples == 0 && msc->start_pad == 0) { - st->skip_samples = msc->start_pad = edit_list_media_time - curr_cts; - - // Shift the index entry timestamp by skip_samples to be correct. - edit_list_dts_counter -= st->skip_samples; + if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO && st->codecpar->codec_id != AV_CODEC_ID_VORBIS && + curr_cts < edit_list_media_time && curr_cts + frame_duration > edit_list_media_time && + first_non_zero_audio_edit > 0) { + packet_skip_samples = edit_list_media_time - curr_cts; + st->skip_samples += packet_skip_samples; + + // Shift the index entry timestamp by packet_skip_samples to be correct. + edit_list_dts_counter -= packet_skip_samples; if (edit_list_start_encountered == 0) { - edit_list_start_encountered = 1; + edit_list_start_encountered = 1; + // Make timestamps strictly monotonically increasing for audio, by rewriting timestamps for + // discarded packets. + if (frame_duration_buffer) { + fix_index_entry_timestamps(st, st->nb_index_entries, edit_list_dts_counter, + frame_duration_buffer, num_discarded_begin); + av_freep(&frame_duration_buffer); + } } - av_log(mov->fc, AV_LOG_DEBUG, "skip %d audio samples from curr_cts: %"PRId64"\n", st->skip_samples, curr_cts); + av_log(mov->fc, AV_LOG_DEBUG, "skip %d audio samples from curr_cts: %"PRId64"\n", packet_skip_samples, curr_cts); } else { flags |= AVINDEX_DISCARD_FRAME; av_log(mov->fc, AV_LOG_DEBUG, "drop a frame at curr_cts: %"PRId64" @ %"PRId64"\n", curr_cts, index); + + if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO && edit_list_start_encountered == 0) { + num_discarded_begin++; + frame_duration_buffer = av_realloc(frame_duration_buffer, + num_discarded_begin * sizeof(int64_t)); + if (!frame_duration_buffer) { + av_log(mov->fc, AV_LOG_ERROR, "Cannot reallocate frame duration buffer\n"); + break; + } + frame_duration_buffer[num_discarded_begin - 1] = frame_duration; + + // Increment skip_samples for the first non-zero audio edit list + if (first_non_zero_audio_edit > 0 && st->codecpar->codec_id != AV_CODEC_ID_VORBIS) { + st->skip_samples += frame_duration; + msc->start_pad = st->skip_samples; + } + } } } else if (edit_list_start_encountered == 0) { edit_list_start_encountered = 1; + // Make timestamps strictly monotonically increasing for audio, by rewriting timestamps for + // discarded packets. + if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO && frame_duration_buffer) { + fix_index_entry_timestamps(st, st->nb_index_entries, edit_list_dts_counter, + frame_duration_buffer, num_discarded_begin); + av_freep(&frame_duration_buffer); + } } if (add_index_entry(st, current->pos, edit_list_dts_counter, current->size, diff --git a/tests/ref/fate/gaplessenc-itunes-to-ipod-aac b/tests/ref/fate/gaplessenc-itunes-to-ipod-aac index 043c085..789681f 100644 --- a/tests/ref/fate/gaplessenc-itunes-to-ipod-aac +++ b/tests/ref/fate/gaplessenc-itunes-to-ipod-aac @@ -7,7 +7,7 @@ duration_ts=103326 start_time=0.000000 duration=2.367000 [/FORMAT] -packet|pts=0|dts=0|duration=N/A +packet|pts=-1024|dts=-1024|duration=1024 packet|pts=0|dts=0|duration=1024 packet|pts=1024|dts=1024|duration=1024 packet|pts=2048|dts=2048|duration=1024 diff --git a/tests/ref/fate/gaplessenc-pcm-to-mov-aac b/tests/ref/fate/gaplessenc-pcm-to-mov-aac index 8b7e3f6..8702611 100644 --- a/tests/ref/fate/gaplessenc-pcm-to-mov-aac +++ b/tests/ref/fate/gaplessenc-pcm-to-mov-aac @@ -7,7 +7,7 @@ duration_ts=529200 start_time=0.000000 duration=12.024000 [/FORMAT] -packet|pts=0|dts=0|duration=N/A +packet|pts=-1024|dts=-1024|duration=1024 packet|pts=0|dts=0|duration=1024 packet|pts=1024|dts=1024|duration=1024 packet|pts=2048|dts=2048|duration=1024 -- 2.8.0.rc3.226.g39d4020