From patchwork Fri Mar 11 17:16:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre-Anthony Lemieux X-Patchwork-Id: 34706 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6838:3486:0:0:0:0 with SMTP id ek6csp1887097nkb; Fri, 11 Mar 2022 09:17:30 -0800 (PST) X-Google-Smtp-Source: ABdhPJzQgYctDj7A3YjtjhZ8cGuDDlKtI01l3LWtaZpcU+KQb8tz5BcBuGQS/O12vY2YN7cSMcAg X-Received: by 2002:a17:907:7704:b0:6cf:48ac:b4a8 with SMTP id kw4-20020a170907770400b006cf48acb4a8mr9038084ejc.305.1647019050611; Fri, 11 Mar 2022 09:17:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1647019050; cv=none; d=google.com; s=arc-20160816; b=jHvbunBC7to9nbO2xBztoaV8jzGZhko1B/XJgVF+/s+Fz6Onvj1waXjTeJGh9DxONG x2V8PxrIveZVJeQ6+/Vtq1xFGjFGLPM1zJOp10jH7jAJtmSJ3zS3xVV7pTcH5HbGHkf6 9sP4iufPhJtO0Uvwsx2Zp+2CrVpYgfakgAq8unc0Ff/AY830fg43DwqLcLpcwZw1hFDn 3pvTWYetYpcJlQN8iBLGw4v1hQRpnSeUK83kdTnJ50z3Um6mjZ6ZijrU/DGQORzhD03n f132J7V5KX+B0sTkCpDPA5Dr4Dr/T7z6vwRciwhgTp4CDKctotcy2uhjGoaAB1nrnI5k p4/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=bXHQy9vIq/HqVtEpjblDk6YpwIF6biMFinWMgnqEAwA=; b=eHiguOCpWGcGQ2bFeORzxdie4gNqhpRWVVlVJGz8DSWnzS9MqKPL9roApK+gK36TdO mgNsp7+X1L9rsdnp5nly+aRctemc2iB75gWLV4ue5PiyaQUDK/vQgzRwgW1mLzUOaVAt 4zgaTtaSv4e2/16Cy/eKit/JM6ZYe2cQNlDfX10AcmK9UfISAyJXKmopu/necoqHOf7B nFnLcI2MC4NaBpW88Wrfy/KvvjvriIBLA085NW3QEou56s2zwr0FmBPhPcALcraGB1Ai gI6atMpelEkkZU0H2kBwXQBfzgwFRxzx4T1iuih6lpUrvQlAV98SLRC8dyOByow7MmOr Pl3g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g22-20020a170906521600b006b1c4eaf5a0si5132787ejm.756.2022.03.11.09.17.30; Fri, 11 Mar 2022 09:17:30 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 32A9C68B167; Fri, 11 Mar 2022 19:17:27 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f170.google.com (mail-pg1-f170.google.com [209.85.215.170]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B713F68B159 for ; Fri, 11 Mar 2022 19:17:19 +0200 (EET) Received: by mail-pg1-f170.google.com with SMTP id o8so7958934pgf.9 for ; Fri, 11 Mar 2022 09:17:19 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vy2ZRdL08jBVJtXwaLOABsPw163H74v2c1Goe67Dnkc=; b=YeUdpaC6wiAs6nckkkfwE27o+1st3ib+VcX3mkSxJOy2SdnRbzWI9VT/KVHRTvQwY+ U4CULiaW7GoERY7Wd8vBUe2FNEYTY3dYq5AiRnSvwukAVqJ8k+5t839xThp5SigKqm7V YhjXk/H9V0I8ciTk4lpjQ3FfAYDnFk1jZEJ27ovlHp4okX+7QpN+d7EW+n0pcu7WWrSv fHfVESEZSL3qoehbwTGgx9ct5jF1NGRzpB3038WitQVtnngvgDQoHG1CjvKuBd+N/nNv ZiYJxNfGCZ5iClMc/y8oo41YgeWm30t1eStdnFMwQ3GMR8JzlEC7Av3lO6qI0qoXxsMZ a7Rw== X-Gm-Message-State: AOAM533uCZ9d2dEUyqgIOLI5+ne0Zzg84M3utUBocDLPLhBeb1qnstEu FDTs6yfQhDq2+BfxJVxoHfWsLnO7/pI= X-Received: by 2002:a05:6a00:1354:b0:4c9:1e96:d15c with SMTP id k20-20020a056a00135400b004c91e96d15cmr11013994pfu.30.1647019036333; Fri, 11 Mar 2022 09:17:16 -0800 (PST) Received: from localhost (76-14-89-2.sf-cable.astound.net. [76.14.89.2]) by smtp.gmail.com with ESMTPSA id i192-20020a636dc9000000b0037c7149fb0asm9334802pgc.89.2022.03.11.09.17.14 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 11 Mar 2022 09:17:16 -0800 (PST) Received: by localhost (sSMTP sendmail emulation); Fri, 11 Mar 2022 09:17:12 -0800 From: pal@sandflow.com To: ffmpeg-devel@ffmpeg.org Date: Fri, 11 Mar 2022 09:16:48 -0800 Message-Id: <20220311171653.17420-2-pal@sandflow.com> X-Mailer: git-send-email 2.35.1.windows.2 In-Reply-To: <20220311171653.17420-1-pal@sandflow.com> References: <20220311171653.17420-1-pal@sandflow.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 2/7] avformat/imf: add support for input seeking X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Pierre-Anthony Lemieux Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: fB3oO6pG/M3z From: Pierre-Anthony Lemieux The IMF demuxer did not implement AVInputFormat::read_seek2(), resulting in inefficient input seeking. Addresses https://trac.ffmpeg.org/ticket/9648 Byte- and frame-seeking are not supported. --- libavformat/imfdec.c | 129 ++++++++++++++++++++++++++++++++++--------- 1 file changed, 102 insertions(+), 27 deletions(-) diff --git a/libavformat/imfdec.c b/libavformat/imfdec.c index b98af020d2..f208b262c3 100644 --- a/libavformat/imfdec.c +++ b/libavformat/imfdec.c @@ -359,13 +359,15 @@ static IMFAssetLocator *find_asset_map_locator(IMFAssetLocatorMap *asset_map, FF } static int open_track_resource_context(AVFormatContext *s, - IMFVirtualTrackResourcePlaybackCtx *track_resource) + IMFVirtualTrackPlaybackCtx *track, + int32_t resource_index) { IMFContext *c = s->priv_data; int ret = 0; - int64_t entry_point; + int64_t seek_offset = 0; AVDictionary *opts = NULL; AVStream *st; + IMFVirtualTrackResourcePlaybackCtx *track_resource = track->resources + resource_index; if (track_resource->ctx) { av_log(s, @@ -416,32 +418,27 @@ static int open_track_resource_context(AVFormatContext *s, st = track_resource->ctx->streams[0]; - /* Warn if the resource time base does not match the file time base */ - if (av_cmp_q(st->time_base, av_inv_q(track_resource->resource->base.edit_rate))) - av_log(s, - AV_LOG_WARNING, - "Incoherent source stream timebase " AVRATIONAL_FORMAT - "regarding resource edit rate: " AVRATIONAL_FORMAT, - st->time_base.num, - st->time_base.den, - track_resource->resource->base.edit_rate.den, - track_resource->resource->base.edit_rate.num); - - entry_point = av_rescale_q(track_resource->resource->base.entry_point, st->time_base, - av_inv_q(track_resource->resource->base.edit_rate)); - - if (entry_point) { - av_log(s, - AV_LOG_DEBUG, - "Seek at resource %s entry point: %" PRIu32 "\n", - track_resource->locator->absolute_uri, - track_resource->resource->base.entry_point); - ret = avformat_seek_file(track_resource->ctx, 0, entry_point, entry_point, entry_point, 0); + /* Determine the seek offset into the Track File, taking into account: + * - the current timestamp within the virtual track + * - the entry point of the resource + */ + if (imf_time_to_ts(&seek_offset, + av_sub_q(track->current_timestamp, track_resource->ts_offset), + st->time_base)) + av_log(s, AV_LOG_WARNING, "Incoherent stream timebase " AVRATIONAL_FORMAT + "and composition timeline position: " AVRATIONAL_FORMAT "\n", + st->time_base.num, st->time_base.den, + track->current_timestamp.den, track->current_timestamp.num); + + if (seek_offset) { + av_log(s, AV_LOG_DEBUG, "Seek at resource %s entry point: %" PRIi64 "\n", + track_resource->locator->absolute_uri, seek_offset); + ret = avformat_seek_file(track_resource->ctx, 0, seek_offset, seek_offset, seek_offset, 0); if (ret < 0) { av_log(s, AV_LOG_ERROR, "Could not seek at %" PRId64 "on %s: %s\n", - entry_point, + seek_offset, track_resource->locator->absolute_uri, av_err2str(ret)); avformat_close_input(&track_resource->ctx); @@ -584,7 +581,7 @@ static int set_context_streams_from_tracks(AVFormatContext *s) AVStream *first_resource_stream; /* Open the first resource of the track to get stream information */ - ret = open_track_resource_context(s, &c->tracks[i]->resources[0]); + ret = open_track_resource_context(s, c->tracks[i], 0); if (ret) return ret; first_resource_stream = c->tracks[i]->resources[0].ctx->streams[0]; @@ -774,7 +771,7 @@ static int get_resource_context_for_timestamp(AVFormatContext *s, IMFVirtualTrac "Switch resource on track %d: re-open context\n", track->index); - ret = open_track_resource_context(s, track->resources + i); + ret = open_track_resource_context(s, track, i); if (ret != 0) return ret; if (track->current_resource_index > 0) @@ -942,6 +939,83 @@ static int imf_probe(const AVProbeData *p) return AVPROBE_SCORE_MAX; } +static void rescale_interval(AVRational tb_in, AVRational tb_out, + int64_t *min_ts, int64_t *ts, int64_t *max_ts) +{ + *ts = av_rescale_q(*ts, tb_in, tb_out); + *min_ts = av_rescale_q_rnd(*min_ts, tb_in, tb_out, AV_ROUND_UP | AV_ROUND_PASS_MINMAX); + *max_ts = av_rescale_q_rnd(*max_ts, tb_in, tb_out, AV_ROUND_DOWN | AV_ROUND_PASS_MINMAX); +} + +static int coherent_ts(int64_t ts, AVRational in_tb, AVRational out_tb) +{ + int dst_num; + int dst_den; + int ret; + + ret = av_reduce(&dst_num, &dst_den, ts * in_tb.num * out_tb.den, + in_tb.den * out_tb.num, INT64_MAX); + if (!ret || dst_den != 1) + return 0; + + return 1; +} + +static int imf_seek(AVFormatContext *s, int stream_index, int64_t min_ts, + int64_t ts, int64_t max_ts, int flags) +{ + IMFContext *c = s->priv_data; + uint32_t i; + + if (flags & (AVSEEK_FLAG_BYTE | AVSEEK_FLAG_FRAME)) + return AVERROR(ENOSYS); + + /* rescale timestamps to Composition edit units */ + if (stream_index < 0) + rescale_interval(AV_TIME_BASE_Q, + av_make_q(c->cpl->edit_rate.den, c->cpl->edit_rate.num), + &min_ts, &ts, &max_ts); + else + rescale_interval(s->streams[stream_index]->time_base, + av_make_q(c->cpl->edit_rate.den, c->cpl->edit_rate.num), + &min_ts, &ts, &max_ts); + + /* requested timestamp bounds are too close */ + if (max_ts < min_ts) + return -1; + + /* clamp requested timestamp to provided bounds */ + ts = FFMAX(FFMIN(ts, max_ts), min_ts); + + av_log(s, AV_LOG_DEBUG, "Seeking to Composition Playlist edit unit %" PRIi64 "\n", ts); + + /* set the dts of each stream and temporal offset of each track */ + for (i = 0; i < c->track_count; i++) { + AVStream *st = s->streams[i]; + IMFVirtualTrackPlaybackCtx *t = c->tracks[i]; + int64_t dts; + + if (!coherent_ts(ts, av_make_q(c->cpl->edit_rate.den, c->cpl->edit_rate.num), + st->time_base)) + av_log(s, AV_LOG_WARNING, "Seek position is not coherent across tracks\n"); + + dts = av_rescale(ts, + st->time_base.den * c->cpl->edit_rate.den, + st->time_base.num * c->cpl->edit_rate.num); + + av_log(s, AV_LOG_DEBUG, "Seeking to dts=%" PRId64 " on stream_index=%d\n", + dts, i); + + t->current_timestamp = av_mul_q(av_make_q(dts, 1), st->time_base); + if (t->current_resource_index >= 0) { + avformat_close_input(&t->resources[t->current_resource_index].ctx); + t->current_resource_index = -1; + } + } + + return 0; +} + static const AVOption imf_options[] = { { .name = "assetmaps", @@ -966,7 +1040,7 @@ static const AVClass imf_class = { const AVInputFormat ff_imf_demuxer = { .name = "imf", .long_name = NULL_IF_CONFIG_SMALL("IMF (Interoperable Master Format)"), - .flags = AVFMT_EXPERIMENTAL, + .flags = AVFMT_EXPERIMENTAL | AVFMT_NO_BYTE_SEEK, .flags_internal = FF_FMT_INIT_CLEANUP, .priv_class = &imf_class, .priv_data_size = sizeof(IMFContext), @@ -974,4 +1048,5 @@ const AVInputFormat ff_imf_demuxer = { .read_header = imf_read_header, .read_packet = imf_read_packet, .read_close = imf_close, + .read_seek2 = imf_seek, };