From patchwork Sun Jan 30 22:00:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre-Anthony Lemieux X-Patchwork-Id: 33948 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2c4e:0:0:0:0 with SMTP id x14csp2244958iov; Sun, 30 Jan 2022 14:01:24 -0800 (PST) X-Google-Smtp-Source: ABdhPJzmtV5pKsJFmedGGAQIYlRn8dfK2BY9k5DWO5IzepYC1syMpAwl45dwQxkbjPSmRtjJtUll X-Received: by 2002:a17:907:ea8:: with SMTP id ho40mr14697047ejc.654.1643580084707; Sun, 30 Jan 2022 14:01:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1643580084; cv=none; d=google.com; s=arc-20160816; b=XzqooiQjTQIRzxV3u5FQxU8bqDyFBS2CSM+R8AR6BlqTvs4jLzSBqL5vem2SKI5afy K5GN3T+1S4smlyQh0vQWG51E18vNY2+oQn39rAeyoVkvl46uqt+VopC5NUPFU5oDhGps rUIyB5RXR1cUKY9TJqeAf5bZhd4xh3xRielhhaWo3ClOGOtumASItZTxpWMA+YwTQPy5 zVrqaY17Qyh5IC7a4Nwwp+OXpnivBmMyy63oNR5egUxQXE32GEr8v0XwvEF7Yp13sAUl xilZAXuclxhCCUq8IQu6kOokr7SClJJExUPZi2JMGLY/SWsnPrPlWrEUIg4y6Ra9PSbH d99w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=C7bMrBAHQhwQkhXC7hi7khv6QSTNLl7TyBBuS6qrjmU=; b=wAxcDkEEeS5upPCqX3PRlqewcXz3NKwJ16LteBvHyr77agGv33N0vxe5qtThOcNsSi yf5baVxWNbDYQDHD0D6IE44YXE1U2eg002OHv6scF7lL11P+tcS5r5/+byVWgjd++Skr rodEXrSE+OE6XEzluryIbwojQTyX7yOQ46DRyn1XRQf98IM3Gp+6SP/nO8rkBp+JEuZV /y65PqNmZyeNo+ZvEqRgW1o1EMTw3h5l2Bsw7KkFQHKFlpm0VeJ4/A/z2gh4d4vBWnMN QhvGkzkTNcmMRe5sF+QJEvcF0MNfAyuBmtyNxAheM1Yt2T8YEP85YY2okGdyzEdHSIR1 ZU/A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gt12si7272607ejc.886.2022.01.30.14.01.23; Sun, 30 Jan 2022 14:01:24 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EED1E68B13F; Mon, 31 Jan 2022 00:01:18 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 341DD6898FB for ; Mon, 31 Jan 2022 00:01:12 +0200 (EET) Received: by mail-pj1-f47.google.com with SMTP id qe6-20020a17090b4f8600b001b7aaad65b9so5801353pjb.2 for ; Sun, 30 Jan 2022 14:01:12 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=LdO5+6smjp3rrf39YXVUveH6nAQlNFOxquHn9ZGqyL4=; b=4UrfZISndeW5WImZB4qgqAFzdTQS231LZXkURfUXokUSC3JVZTbUYdF3bVvc6IISGb 38KVi13D7fY6Wh5tjvmB7g0KttiwP9/40mjO9/BEseVBrBmwDRdUWvssggr4Az+8JYEr 0kk1GrRdSB64OzHAtcvD0Q9ATdmlGtSqbPsOgjUpurzZPjyP8v9vka1db2F8e3d0r+p0 K174qGsKE9P3NxJsTnYm1tJDl5dXx0MBML6l6RpLimHZ3qGO/cGaMs0mCKKDomi4X8p6 Xo7S9YVZW7OXQZEdLq+IzfnWuw7H2LYEOkb9GVWQWDHq5EyGWYFMsAfWBFlsul4GUZq+ Hfbw== X-Gm-Message-State: AOAM530YWui9RtGpd8vaZsnRk+Z4s7/esJlvfnVhdm1vDdtKktTW9gn0 3wFFGwrWUBYpBNoIBVZ/WyjMT246JS8= X-Received: by 2002:a17:903:18b:: with SMTP id z11mr18458325plg.47.1643580069905; Sun, 30 Jan 2022 14:01:09 -0800 (PST) Received: from localhost (76-14-89-2.sf-cable.astound.net. [76.14.89.2]) by smtp.gmail.com with ESMTPSA id lp17sm9383497pjb.25.2022.01.30.14.01.07 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sun, 30 Jan 2022 14:01:09 -0800 (PST) Received: by localhost (sSMTP sendmail emulation); Sun, 30 Jan 2022 14:00:58 -0800 From: pal@sandflow.com To: ffmpeg-devel@ffmpeg.org Date: Sun, 30 Jan 2022 14:00:52 -0800 Message-Id: <20220130220055.2595-1-pal@sandflow.com> X-Mailer: git-send-email 2.35.0.windows.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v1 1/4] avformat/imf: open resources only when first needed X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Pierre-Anthony Lemieux Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Cj7tljXxCpsq From: Pierre-Anthony Lemieux IMF CPLs can reference thousands of files, which can result in system limits for the number of open files to be exceeded. The following patch opens and closes files as needed. --- libavformat/imfdec.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/libavformat/imfdec.c b/libavformat/imfdec.c index 566a0fb792..d03dcd623d 100644 --- a/libavformat/imfdec.c +++ b/libavformat/imfdec.c @@ -103,10 +103,11 @@ typedef struct IMFVirtualTrackPlaybackCtx { int32_t index; /**< Track index in playlist */ AVRational current_timestamp; /**< Current temporal position */ AVRational duration; /**< Overall duration */ - uint32_t resource_count; /**< Number of resources */ + uint32_t resource_count; /**< Number of resources (<= INT32_MAX) */ unsigned int resources_alloc_sz; /**< Size of the buffer holding the resource */ IMFVirtualTrackResourcePlaybackCtx *resources; /**< Buffer holding the resources */ - uint32_t current_resource_index; /**< Current resource */ + int32_t current_resource_index; /**< Index of the current resource in resources, + or < 0 if a current resource has yet to be selected */ int64_t last_pts; /**< Last timestamp */ } IMFVirtualTrackPlaybackCtx; @@ -435,7 +436,6 @@ static int open_track_file_resource(AVFormatContext *s, IMFContext *c = s->priv_data; IMFAssetLocator *asset_locator; void *tmp; - int ret; asset_locator = find_asset_map_locator(&c->asset_locator_map, track_file_resource->track_file_uuid); if (!asset_locator) { @@ -452,7 +452,7 @@ static int open_track_file_resource(AVFormatContext *s, UID_ARG(asset_locator->uuid), asset_locator->absolute_uri); - if (track->resource_count > UINT32_MAX - track_file_resource->base.repeat_count + if (track->resource_count > INT32_MAX - track_file_resource->base.repeat_count || (track->resource_count + track_file_resource->base.repeat_count) > INT_MAX / sizeof(IMFVirtualTrackResourcePlaybackCtx)) return AVERROR(ENOMEM); @@ -470,8 +470,6 @@ static int open_track_file_resource(AVFormatContext *s, vt_ctx.locator = asset_locator; vt_ctx.resource = track_file_resource; vt_ctx.ctx = NULL; - if ((ret = open_track_resource_context(s, &vt_ctx)) != 0) - return ret; track->resources[track->resource_count++] = vt_ctx; track->duration = av_add_q(track->duration, av_make_q((int)track_file_resource->base.duration @@ -501,6 +499,7 @@ static int open_virtual_track(AVFormatContext *s, if (!(track = av_mallocz(sizeof(IMFVirtualTrackPlaybackCtx)))) return AVERROR(ENOMEM); + track->current_resource_index = -1; track->index = track_index; track->duration = av_make_q(0, 1); @@ -551,6 +550,7 @@ static int set_context_streams_from_tracks(AVFormatContext *s) AVStream *first_resource_stream; /* Open the first resource of the track to get stream information */ + open_track_resource_context(s, &c->tracks[i]->resources[0]); first_resource_stream = c->tracks[i]->resources[0].ctx->streams[0]; av_log(s, AV_LOG_DEBUG, "Open the first resource of track %d\n", c->tracks[i]->index); @@ -741,7 +741,8 @@ static IMFVirtualTrackResourcePlaybackCtx *get_resource_context_for_timestamp(AV track->index); if (open_track_resource_context(s, &(track->resources[i])) != 0) return NULL; - avformat_close_input(&(track->resources[track->current_resource_index].ctx)); + if (track->current_resource_index > 0) + avformat_close_input(&track->resources[track->current_resource_index].ctx); track->current_resource_index = i; }