From patchwork Wed Aug 16 12:11:16 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: zhangjiejun1992@gmail.com X-Patchwork-Id: 4721 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.46.211 with SMTP id u202csp712645vsu; Wed, 16 Aug 2017 05:43:36 -0700 (PDT) X-Received: by 10.28.111.200 with SMTP id c69mr1266020wmi.115.1502887416421; Wed, 16 Aug 2017 05:43:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1502887416; cv=none; d=google.com; s=arc-20160816; b=XCB+WQ/hTXHmcNg0RphukH0n0C7jqRunW31NXnb10xQ+areAqOGztss0LQucbWprsJ wYvwypLG4OqSs0RhfIQ1slGetRY/F4EcEcWhDztNlZzASE9neJlbB5255aUr6PK2UCLr zJnez/HUlkghRexjh5iCdK/xDQxoT6vwr+JpeLgnTaiFgOrtvk+aF4wCNEQNjzWZQCg3 PLgl2d3RqWD8WVzHscGyRnQCTKz1QeeL84CfD6eR49ZjM/ry9D14IOvZEoYaQ5g8EiZk CRFyCRTN24He7iITp4JV1YIb5aM1y4sxg7UN/lVqGauHE+NNHp7poiVAViYm7W8sfzfA mV5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=T8JeBUzIhFjEgCfTKK0q87ziiVHbg3Pt0/B2l07TOpg=; b=uy8RbNAeStwB84R4VP8btLZZz51bOVsBHA7Zv/ImOOtwdrDe8G3fegEFnYiMrm3GGu 11rKyBXv2mBlOr0GTAr0/Mx9z9ryTW4iwPF3iBHtHH5oH3ZIPapnSbCkzvyq7EXHjmjf rDD3NYMGL5pKtBf4TAWw9FuNZfYdiytP2tlAzLmdHeE74PBi9kx8/G4njx0DyLlooPsk N2f6/PwAdw/zd+cO++BSAGNijbciGJtrV9rZfTXDeiRE6PDQce1Jba4hL4djznIAcmuJ rTa+yvPCHX3NhUIuktX3YA8hLbaNqGQ69NQDHAueUWSD+AVwSNGcT/OK4IwhvO5XYxHF 4aJw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=JVlDwtpa; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id k16si642459wrk.226.2017.08.16.05.43.36; Wed, 16 Aug 2017 05:43:36 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=JVlDwtpa; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EB14F6898EF; Wed, 16 Aug 2017 15:43:28 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf0-f193.google.com (mail-pf0-f193.google.com [209.85.192.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4CE0B6898D1 for ; Wed, 16 Aug 2017 15:43:22 +0300 (EEST) Received: by mail-pf0-f193.google.com with SMTP id p13so1287324pfd.4 for ; Wed, 16 Aug 2017 05:43:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=PdiIaFgvUtnl3IRlzwobn3kAQ0RN8CRPrfDbRyhbkyQ=; b=JVlDwtpae0vYEAhOVVLfj6J4VJ4NvBfA8IJc2RsF4BQR8tzr+4Ub9jqT6t9eXk4ILy 7JNKFwqsJMq+c7qBmPzBTKGNDCusI5eqofGdZCK4DcRTB/GuOTIkQ5lEPK+/tsZrjuAc Q6imr8i8wDZjRUQsoFGp7WNhK+o2Jl/QA5CpeaS6J/aPt1ZK9z89gtIryPgd/IXKnsG1 UFTymHtXyc1VvSmbHiKI2Rpje3DA5SU6iERxwIyij3EhQ/gKokCqLo5sIbGFxdNQjug2 VUrv014N3bU+t0yFgDdfFpquVIUp24NSCPb87rOwSJj0HvE+vzodQNu1VYfX/XQQqw8y lPqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=PdiIaFgvUtnl3IRlzwobn3kAQ0RN8CRPrfDbRyhbkyQ=; b=MsyE58o78e/aw3p5jByzFH1AevPHNqGCXwDGNkD4w5dk8xDQLbVEbQJlYdsZGHgMZz vGMG2CW9BMFc+FbwwGz6CVFcWyJIpSmH0Ig61zM0F5MeIEsk2g5jSBIQNvY3YqGm4ebc epHbtYA/F3BbKSxYiRpfMT6bYzBzrmBsQTqRiZy00AwyGNQtqpxHwQ2OGo2QCTbQyKZ5 nFJAgXKIcUbNeQh3mlVbPdGHpCmchnBeEzAIF40JUvNsNuGJqukWahunKv5tvaygZ8Qg t3Z7SFoJFy387LpBEK71uzmHyhc6QdTQIrm+r9t2HhNsgBGGi3pjH172BLEOLWBtjaNO 1UaQ== X-Gm-Message-State: AHYfb5jVaBmTtR9wbfUUHdbLc3xf2zTWEKORSf309AfGUw7+aNCXWmdW 9WyeMy+Z3vc8jMAm X-Received: by 10.99.96.74 with SMTP id u71mr1384144pgb.129.1502885521279; Wed, 16 Aug 2017 05:12:01 -0700 (PDT) Received: from localhost.localdomain (ec2-52-220-245-83.ap-southeast-1.compute.amazonaws.com. [52.220.245.83]) by smtp.gmail.com with ESMTPSA id s3sm3332437pgn.70.2017.08.16.05.11.59 (version=TLS1 cipher=AES128-SHA bits=128/128); Wed, 16 Aug 2017 05:12:00 -0700 (PDT) From: zhangjiejun1992@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 16 Aug 2017 20:11:16 +0800 Message-Id: <20170816121116.39917-1-zhangjiejun1992@gmail.com> X-Mailer: git-send-email 2.11.0 (Apple Git-81) In-Reply-To: References: Subject: [FFmpeg-devel] [PATCH v2] libavformat/mov: fix multiple stsd handling of files with edit list, fix #6584 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Jiejun Zhang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Jiejun Zhang When an edit list exists in a MOV file, counting by stscs no longer works because stscs' order is different from the actual timeline. This commit adds stsd-change markers to the actual timeline and changes stsd according to these markers. --- libavformat/isom.h | 14 ++++-- libavformat/mov.c | 122 +++++++++++++++++++++++++++++++++++++++++------------ 2 files changed, 105 insertions(+), 31 deletions(-) diff --git a/libavformat/isom.h b/libavformat/isom.h index ff009b0896..51cd333a8c 100644 --- a/libavformat/isom.h +++ b/libavformat/isom.h @@ -127,6 +127,11 @@ typedef struct MOVIndexRange { int64_t end; } MOVIndexRange; +typedef struct MOVStsdChangeEntry { + int64_t sample_index; ///< stsd changes at this sample index + int stsd_id; ///< stsd changes to stsd_id +} MOVStsdChangeEntry; + typedef struct MOVStreamContext { AVIOContext *pb; int pb_is_copied; @@ -140,16 +145,17 @@ typedef struct MOVStreamContext { MOVStts *ctts_data; unsigned int stsc_count; MOVStsc *stsc_data; - int stsc_index; - int stsc_sample; + MOVStsdChangeEntry *stsd_change_data; + int stsd_change_count; + int stsd_change_index; unsigned int stps_count; unsigned *stps_data; ///< partial sync sample for mpeg-2 open gop MOVElst *elst_data; unsigned int elst_count; int ctts_index; int ctts_sample; - unsigned int sample_size; ///< may contain value calculated from stsd or value from stsz atom - unsigned int stsz_sample_size; ///< always contains sample size from stsz atom + unsigned int sample_size; ///< may contain value calculated from stsd or value from stsz atom unsigned + int stsz_sample_size; ///< always contains sample size from stsz atom unsigned int sample_count; int *sample_sizes; int keyframe_absent; diff --git a/libavformat/mov.c b/libavformat/mov.c index c02caf6719..628cf31cec 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -2926,6 +2926,35 @@ static int64_t add_index_entry(AVStream *st, int64_t pos, int64_t timestamp, return index; } +static int add_stsd_change_entry(MOVStreamContext *msc, int64_t sample_index, int stsd_id, + unsigned int* allocated_size) +{ + MOVStsdChangeEntry *entries; + const size_t min_size_needed = (msc->stsd_change_count + 1) * sizeof(MOVStsdChangeEntry); + + const size_t requested_size = + min_size_needed > *allocated_size ? + FFMAX(min_size_needed, 2 * (*allocated_size)) : + min_size_needed; + + if ((unsigned)msc->stsd_change_count + 1 >= UINT_MAX / sizeof(MOVStsdChangeEntry)) + return -1; + + entries = av_fast_realloc(msc->stsd_change_data, + allocated_size, + requested_size); + + if (!entries) + return -1; + + msc->stsd_change_data = entries; + entries[msc->stsd_change_count].sample_index = sample_index; + entries[msc->stsd_change_count].stsd_id = stsd_id; + msc->stsd_change_count++; + + return 0; +} + /** * Rewrite timestamps of index entries in the range [end_index - frame_duration_buffer_size, end_index) * by subtracting end_ts successively by the amounts given in frame_duration_buffer. @@ -3058,6 +3087,8 @@ static void mov_fix_index(MOVContext *mov, AVStream *st) int num_discarded_begin = 0; int first_non_zero_audio_edit = -1; int packet_skip_samples = 0; + int stsc_index = 0; + int stsc_sample = 0; MOVIndexRange *current_index_range; int i; @@ -3094,6 +3125,11 @@ static void mov_fix_index(MOVContext *mov, AVStream *st) start_dts = edit_list_dts_entry_end; + msc->last_stsd_index = -1; + unsigned int stsd_change_data_allocated_size = 0; + msc->stsd_change_data = NULL; + msc->stsd_change_count = 0; + while (get_edit_list_entry(mov, msc, edit_list_index, &edit_list_media_time, &edit_list_duration, mov->time_scale)) { av_log(mov->fc, AV_LOG_DEBUG, "Processing st: %d, edit list %"PRId64" - media time: %"PRId64", duration: %"PRId64"\n", @@ -3156,6 +3192,20 @@ static void mov_fix_index(MOVContext *mov, AVStream *st) } current = e_old + index; + // adjust stsd index + int time_sample = 0; + for (i = 0; i < msc->stsc_count; i++) { + int next = time_sample + mov_get_stsc_samples(msc, i); + if (next > index) { + stsc_index = i; + stsc_sample = index - time_sample; + break; + } + time_sample = next; + } + + av_log(mov->fc, AV_LOG_INFO, "stream %d, adjusted stsc_index: %d, stsc_sample: %d\n", st->index, stsc_index, stsc_sample); + ctts_index_old = 0; ctts_sample_old = 0; @@ -3268,12 +3318,35 @@ static void mov_fix_index(MOVContext *mov, AVStream *st) } } - if (add_index_entry(st, current->pos, edit_list_dts_counter, current->size, - current->min_distance, flags) == -1) { + int index_entry_index = add_index_entry(st, current->pos, edit_list_dts_counter, current->size, + current->min_distance, flags); + if (index_entry_index == -1) { av_log(mov->fc, AV_LOG_ERROR, "Cannot add index entry\n"); break; } + if (msc->stsc_data) { + if (msc->stsc_data[stsc_index].id > 0 && + msc->stsc_data[stsc_index].id - 1 < msc->stsd_count && + msc->stsc_data[stsc_index].id - 1 != msc->last_stsd_index) { + msc->last_stsd_index = msc->stsc_data[stsc_index].id - 1; + int ret = add_stsd_change_entry(msc, index_entry_index, msc->last_stsd_index, &stsd_change_data_allocated_size); + if (ret < 0) { + av_log(mov->fc, AV_LOG_ERROR, "Cannot add stsd change entry\n"); + break; + } + av_log(mov->fc, AV_LOG_INFO, "added a stsd change entry sample_index: %d, stsd_index:%d\n", + index_entry_index, + msc->last_stsd_index); + } + stsc_sample++; + if (stsc_index < msc->stsc_count - 1 && + mov_get_stsc_samples(msc, stsc_index) == stsc_sample) { + stsc_index++; + stsc_sample = 0; + } + } + // Update the index ranges array if (current_index_range < msc->index_ranges || index != current_index_range->end) { current_index_range++; @@ -3330,6 +3403,7 @@ static void mov_fix_index(MOVContext *mov, AVStream *st) current_index_range->start = 0; current_index_range->end = 0; msc->current_index = msc->index_ranges[0].start; + msc->last_stsd_index = 0; } static void mov_build_index(MOVContext *mov, AVStream *st) @@ -6435,9 +6509,6 @@ static int mov_change_extradata(MOVStreamContext *sc, AVPacket *pkt) uint8_t *side, *extradata; int extradata_size; - /* Save the current index. */ - sc->last_stsd_index = sc->stsc_data[sc->stsc_index].id - 1; - /* Notify the decoder that extradata changed. */ extradata_size = sc->extradata_size[sc->last_stsd_index]; extradata = sc->extradata[sc->last_stsd_index]; @@ -6557,18 +6628,18 @@ static int mov_read_packet(AVFormatContext *s, AVPacket *pkt) pkt->pos = sample->pos; /* Multiple stsd handling. */ - if (sc->stsc_data) { - /* Keep track of the stsc index for the given sample, then check - * if the stsd index is different from the last used one. */ - sc->stsc_sample++; - if (mov_stsc_index_valid(sc->stsc_index, sc->stsc_count) && - mov_get_stsc_samples(sc, sc->stsc_index) == sc->stsc_sample) { - sc->stsc_index++; - sc->stsc_sample = 0; - /* Do not check indexes after a switch. */ - } else if (sc->stsc_data[sc->stsc_index].id > 0 && - sc->stsc_data[sc->stsc_index].id - 1 < sc->stsd_count && - sc->stsc_data[sc->stsc_index].id - 1 != sc->last_stsd_index) { + if (sc->stsd_change_data) { + if (sc->stsd_change_index + 1 < sc->stsd_change_count && + // sc->current_sample is already the next sample, so subtract 1 + sc->stsd_change_data[sc->stsd_change_index + 1].sample_index == sc->current_sample - 1) { + ++sc->stsd_change_index; + } + if (sc->last_stsd_index != sc->stsd_change_data[sc->stsd_change_index].stsd_id) { + av_log(mov->fc, AV_LOG_INFO, "stsd id changed from %d to %d, at sample %d\n", + sc->last_stsd_index, + sc->stsd_change_data[sc->stsd_change_index].stsd_id, + sc->current_sample - 1); + sc->last_stsd_index = sc->stsd_change_data[sc->stsd_change_index].stsd_id; ret = mov_change_extradata(sc, pkt); if (ret < 0) return ret; @@ -6646,18 +6717,15 @@ static int mov_seek_stream(AVFormatContext *s, AVStream *st, int64_t timestamp, } } - /* adjust stsd index */ - time_sample = 0; - for (i = 0; i < sc->stsc_count; i++) { - int next = time_sample + mov_get_stsc_samples(sc, i); - if (next > sc->current_sample) { - sc->stsc_index = i; - sc->stsc_sample = sc->current_sample - time_sample; - break; + /* adjust stsd change index */ + for (i = 0; i < sc->stsd_change_count; i++) { + if (sc->current_sample >= sc->stsd_change_data[i].sample_index && + (i + 1 == sc->stsd_change_count || + sc->current_sample < sc->stsd_change_data[i + 1].sample_index)) { + sc->stsd_change_index = i; } - time_sample = next; } - + return sample; }