From patchwork Tue Oct 31 23:42:56 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sasi Inguva X-Patchwork-Id: 5800 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.90 with SMTP id m26csp91818jah; Tue, 31 Oct 2017 16:50:20 -0700 (PDT) X-Google-Smtp-Source: ABhQp+Qy0Oid252b//XTN0ozk7fUi2bhaSrRj44vsXRaK1CoHR3fl6u9MeCfBS1C0vFbo25hlyO5 X-Received: by 10.223.135.231 with SMTP id c36mr3362043wrc.248.1509493820133; Tue, 31 Oct 2017 16:50:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1509493820; cv=none; d=google.com; s=arc-20160816; b=KVoQRqIC080YykH34KA+LJF+eKYv8Yk+SsCaO4Ji/GOej1arJo1shC39jZAVrP7Wee CxkIY4gEvPwqGIdVbZ2ZlFjtTEb5prYZcPS+uS3lmCfYohIUVVuVH6rKO04prWQOw/9w m7rCdjMt/pCyUGMFwiYsnLg/mDWIpj+MrNGFfBk5FJVSGJwZ0oc8D0S0bYfHQ5wZvIN7 cnW9BQ7UBSevSiY94NpE2XDQCYtaZMM8SbCLpKbJV527Lsz0mNBCClE8+MG+jKW8DCdp mvu1ghOZc2FZzhRdq7SXgsJcBvRTPcH6VlYW2Wp0OlI3lRZP96G6Kub18s/EeBn+Alcv 0ZHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=Kp8ui0Zi2MoqGjkELRT0+47t2ze0jtZZMxGiKUlY4iM=; b=i4GHwr7zPivNVqR04ALWlHR1WPbRHoN4n0B7DCENgw9BegwZ5hOAvGfbHzBtSwOIkV y9mMvlAkC6ETMz/7S+lV9eRQQ6KvYbcUtxI/lMdHaC7aqsEH8DxINQvtmgbhXZCIix0C bDor9S272Gkm2kDPkLZ+Voge/JspsooxO1TdzxollnSyvZ7fJpEgoHGWq4PqQefE2iin mPJjgv71DHwQzE+dzsDfq53+CzTrqs94Zq6Y0JG7csNauN567cZ4pry+GfnFw3t8xmZX qdm6wGpKgAsZvTKQhydh7z7BZpbhRsxyGw0uG8PnTCm72pZKtWO2MI/e2i+KhWTrgvph yjMQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20161025 header.b=SzLKXSaa; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id t110si2453798wrc.507.2017.10.31.16.50.19; Tue, 31 Oct 2017 16:50:20 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20161025 header.b=SzLKXSaa; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 804FE689DE7; Wed, 1 Nov 2017 01:50:08 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-io0-f180.google.com (mail-io0-f180.google.com [209.85.223.180]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6B1D9689756 for ; Wed, 1 Nov 2017 01:50:07 +0200 (EET) Received: by mail-io0-f180.google.com with SMTP id b186so2537316iof.8 for ; Tue, 31 Oct 2017 16:50:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=1e3BvlNjPm+cVCYpOA9wo+MeihLCmDncScqdc9yy+xo=; b=SzLKXSaaflNsgxJUpH/UhwwjVaqj+rLhXeorSRJQTv8IHdMmmKdFZuu9aHS659bApF 6b9bu7gGDxmwp3IDbv0UTKiAbZPGM+twvzWes0y8cEtXnJ7xZvUqATnDukZ8pYY/661r cpNqKQVyJtUgqmEOpRcgbzJ9M/lCotf8WEI5on9tr2bFCFtrx8YUwDJchy8pp6AN6D3s Ghar3RMqHNjLDpadmHwIBwhHOI0rb+jstOdtJuvFEraJNqGAeusA7Kj+MnT6PvZNmwMN Bo+RF3N4YNol1Lq49mPU/4uoiKRxyqelpWDGa5eOZIJ38riuyqPxhnZmWVQzp8+RT2f1 f5eA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=1e3BvlNjPm+cVCYpOA9wo+MeihLCmDncScqdc9yy+xo=; b=KbZcbIEBFfQnLSxv5TWS6nFTOOY9bbhyrLtmPmgKAIEKSaDud07UPTWFqvUWq2F8nk uy/K1n48jm28NzXVqKICmKvaUt2zd0fkBI0WlL+//qvxf3250V38yYMt5M744r8CuYng YFrmrdOR4BZoDRoQR7iRHFpmPUh1HgShdqKO5C3xTCAE399q1nK7GFSlWm6xli2rSog0 qw57FpaNB++uexGx3EX76wBaA/BOFN0x+svHmjVl3xVLreNdsm4bcJq7W+a2PVb73tth MM3fUOwYFcwCDcnT0a8yglVq/fWv3zeVnjGesEDSXk/nYMsWZQee/yT3eGAC6VG8oeiS Nsyg== X-Gm-Message-State: AMCzsaVDTSJOPW4XLgB1O7axMYRJaHZJpZqDZdLhMWsRqcTvcPb2+GMi yW+0cu+wGfNQsZyR9SZQacXbm6+aCs4= X-Received: by 10.36.76.1 with SMTP id a1mr5591478itb.94.1509493441493; Tue, 31 Oct 2017 16:44:01 -0700 (PDT) Received: from isasi.mtv.corp.google.com ([172.27.82.28]) by smtp.gmail.com with ESMTPSA id e34sm1235905ioj.83.2017.10.31.16.44.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 31 Oct 2017 16:44:00 -0700 (PDT) From: Sasi Inguva To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Oct 2017 16:42:56 -0700 Message-Id: <20171031234256.29076-1-isasi@google.com> X-Mailer: git-send-email 2.15.0.403.gc27cc4dac6-goog Subject: [FFmpeg-devel] [PATCH] lavf/mov.c: Refine edit list start seek, based on PTS computed from CTTS. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Sasi Inguva MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Partially fixes t/6699. --- libavformat/mov.c | 125 +++++++++++++++++++++++++++++++++++------------------ tests/fate/mov.mak | 8 ++++ 2 files changed, 90 insertions(+), 43 deletions(-) diff --git a/libavformat/mov.c b/libavformat/mov.c index 60f0228e2d..a295445651 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -3014,34 +3014,95 @@ static int get_edit_list_entry(MOVContext *mov, } /** - * Find the closest previous frame to the timestamp, in e_old index + * Find the closest previous frame to the timestamp_pts, in e_old index * entries. Searching for just any frame / just key frames can be controlled by * last argument 'flag'. - * Returns the index of the entry in st->index_entries if successful, - * else returns -1. + * Here the timestamp_pts is considered to be a presentation timestamp and + * the timestamp of index entries are considered to be decoding timestamps. + * + * Returns 0 if successful in finding a frame, else returns -1. + * Places the found index corresponding output arg. + * + * If ctts_old is not NULL, then refines the searched entry by searching + * backwards from the found timestamp, to find the frame with correct PTS. + * + * Places the found ctts_index and ctts_sample in corresponding output args. */ -static int64_t find_prev_closest_index(AVStream *st, - AVIndexEntry *e_old, - int nb_old, - int64_t timestamp, - int flag) +static int find_prev_closest_index(AVStream *st, + AVIndexEntry *e_old, + int nb_old, + MOVStts* ctts_data, + int64_t ctts_count, + int64_t timestamp_pts, + int flag, + int64_t* index, + int64_t* ctts_index, + int64_t* ctts_sample) { + MOVStreamContext *msc = st->priv_data; AVIndexEntry *e_keep = st->index_entries; int nb_keep = st->nb_index_entries; - int64_t found = -1; int64_t i = 0; + int64_t index_ctts_count; + + av_assert0(index); + + // If dts_shift > 0, then all the index timestamps will have to be offset by + // at least dts_shift amount to obtain PTS. + // Hence we decrement the searched timestamp_pts by dts_shift to find the closest index element. + if (msc->dts_shift > 0) { + timestamp_pts -= msc->dts_shift; + } st->index_entries = e_old; st->nb_index_entries = nb_old; - found = av_index_search_timestamp(st, timestamp, flag | AVSEEK_FLAG_BACKWARD); + *index = av_index_search_timestamp(st, timestamp_pts, flag | AVSEEK_FLAG_BACKWARD); // Keep going backwards in the index entries until the timestamp is the same. - if (found >= 0) { - for (i = found; i > 0 && e_old[i].timestamp == e_old[i - 1].timestamp; + if (*index >= 0) { + for (i = *index; i > 0 && e_old[i].timestamp == e_old[i - 1].timestamp; i--) { if ((flag & AVSEEK_FLAG_ANY) || (e_old[i - 1].flags & AVINDEX_KEYFRAME)) { - found = i - 1; + *index = i - 1; + } + } + } + + // If we have CTTS then refine the search, by searching backwards over PTS + // computed by adding corresponding CTTS durations to index timestamps. + if (ctts_data && *index >= 0) { + av_assert0(ctts_index); + av_assert0(ctts_sample); + // Find out the ctts_index for the found frame. + *ctts_index = 0; + *ctts_sample = 0; + for (index_ctts_count = 0; index_ctts_count < *index; index_ctts_count++) { + if (*ctts_index < ctts_count) { + (*ctts_sample)++; + if (ctts_data[*ctts_index].count == *ctts_sample) { + (*ctts_index)++; + *ctts_sample = 0; + } + } + } + + while (*index >= 0 && (*ctts_index) >= 0) { + // Find a "key frame" with PTS <= timestamp_pts (So that we can decode B-frames correctly). + // No need to add dts_shift to the timestamp here becase timestamp_pts has already been + // compensated by dts_shift above. + if ((e_old[*index].timestamp + ctts_data[*ctts_index].duration) <= timestamp_pts && + (e_old[*index].flags & AVINDEX_KEYFRAME)) { + break; + } + + (*index)--; + if (*ctts_sample == 0) { + (*ctts_index)--; + if (*ctts_index >= 0) + *ctts_sample = ctts_data[*ctts_index].count - 1; + } else { + (*ctts_sample)--; } } } @@ -3049,7 +3110,7 @@ static int64_t find_prev_closest_index(AVStream *st, /* restore AVStream state*/ st->index_entries = e_keep; st->nb_index_entries = nb_keep; - return found; + return *index >= 0 ? 0 : -1; } /** @@ -3220,10 +3281,8 @@ static void mov_fix_index(MOVContext *mov, AVStream *st) int64_t empty_edits_sum_duration = 0; int64_t edit_list_index = 0; int64_t index; - int64_t index_ctts_count; int flags; int64_t start_dts = 0; - int64_t edit_list_media_time_dts = 0; int64_t edit_list_start_encountered = 0; int64_t search_timestamp = 0; int64_t* frame_duration_buffer = NULL; @@ -3293,17 +3352,11 @@ static void mov_fix_index(MOVContext *mov, AVStream *st) st->skip_samples = msc->start_pad = 0; } - //find closest previous key frame - edit_list_media_time_dts = edit_list_media_time; - if (msc->dts_shift > 0) { - edit_list_media_time_dts -= msc->dts_shift; - } - // While reordering frame index according to edit list we must handle properly // the scenario when edit list entry starts from none key frame. // We find closest previous key frame and preserve it and consequent frames in index. // All frames which are outside edit list entry time boundaries will be dropped after decoding. - search_timestamp = edit_list_media_time_dts; + search_timestamp = edit_list_media_time; if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) { // Audio decoders like AAC need need a decoder delay samples previous to the current sample, // to correctly decode this frame. Hence for audio we seek to a frame 1 sec. before the @@ -3311,38 +3364,24 @@ static void mov_fix_index(MOVContext *mov, AVStream *st) search_timestamp = FFMAX(search_timestamp - msc->time_scale, e_old[0].timestamp); } - index = find_prev_closest_index(st, e_old, nb_old, search_timestamp, 0); - if (index == -1) { + if (find_prev_closest_index(st, e_old, nb_old, ctts_data_old, ctts_count_old, search_timestamp, 0, + &index, &ctts_index_old, &ctts_sample_old) < 0) { av_log(mov->fc, AV_LOG_WARNING, "st: %d edit list: %"PRId64" Missing key frame while searching for timestamp: %"PRId64"\n", st->index, edit_list_index, search_timestamp); - index = find_prev_closest_index(st, e_old, nb_old, search_timestamp, AVSEEK_FLAG_ANY); - - if (index == -1) { + if (find_prev_closest_index(st, e_old, nb_old, ctts_data_old, ctts_count_old, search_timestamp, AVSEEK_FLAG_ANY, + &index, &ctts_index_old, &ctts_sample_old) < 0) { av_log(mov->fc, AV_LOG_WARNING, "st: %d edit list %"PRId64" Cannot find an index entry before timestamp: %"PRId64".\n" "Rounding edit list media time to zero.\n", st->index, edit_list_index, search_timestamp); index = 0; + ctts_index_old = 0; + ctts_sample_old = 0; edit_list_media_time = 0; } } current = e_old + index; - - ctts_index_old = 0; - ctts_sample_old = 0; - - // set ctts_index properly for the found key frame - for (index_ctts_count = 0; index_ctts_count < index; index_ctts_count++) { - if (ctts_data_old && ctts_index_old < ctts_count_old) { - ctts_sample_old++; - if (ctts_data_old[ctts_index_old].count == ctts_sample_old) { - ctts_index_old++; - ctts_sample_old = 0; - } - } - } - edit_list_start_ctts_sample = ctts_sample_old; // Iterate over index and arrange it according to edit list diff --git a/tests/fate/mov.mak b/tests/fate/mov.mak index 6815e4feca..01893a0767 100644 --- a/tests/fate/mov.mak +++ b/tests/fate/mov.mak @@ -9,6 +9,7 @@ FATE_MOV = fate-mov-3elist \ fate-mov-invalid-elst-entry-count \ fate-mov-gpmf-remux \ fate-mov-440hz-10ms \ + fate-mov-ibi-elst-starts-b \ FATE_MOV_FFPROBE = fate-mov-aac-2048-priming \ fate-mov-zombie \ @@ -47,6 +48,13 @@ fate-mov-440hz-10ms: CMD = framemd5 -i $(TARGET_SAMPLES)/mov/440hz-10ms.m4a # Makes sure that we handle invalid edit list entry count correctly. fate-mov-invalid-elst-entry-count: CMD = framemd5 -flags +bitexact -i $(TARGET_SAMPLES)/mov/invalid_elst_entry_count.mov +# Makes sure that 1st key-frame is picked when, +# i) One B-frame between 2 key-frames +# ii) Edit list starts on B-frame. +# iii) Both key-frames have their DTS < edit list start +# i.e. Pts Order: I-B-I +fate-mov-ibi-elst-starts-b: CMD = framemd5 -flags +bitexact -i $(TARGET_SAMPLES)/mov/mov_ibi_elst_starts_b.mov + fate-mov-aac-2048-priming: CMD = run ffprobe$(PROGSSUF)$(EXESUF) -show_packets -print_format compact $(TARGET_SAMPLES)/mov/aac-2048-priming.mov fate-mov-zombie: CMD = run ffprobe$(PROGSSUF)$(EXESUF) -show_streams -show_packets -show_frames -bitexact -print_format compact $(TARGET_SAMPLES)/mov/white_zombie_scrunch-part.mov