From patchwork Wed Feb 2 00:12:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre-Anthony Lemieux X-Patchwork-Id: 34060 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a10:6681:0:0:0:0 with SMTP id b1csp1023656pxu; Tue, 1 Feb 2022 16:13:38 -0800 (PST) X-Google-Smtp-Source: ABdhPJzrBfmEwu+1BvemLeVEcfkXr4Ws3sTHc9t4jDOjzAAgdz/99N/tysvnGPQgZmZUp6oT1D9j X-Received: by 2002:a17:907:8a24:: with SMTP id sc36mr22660721ejc.318.1643760818414; Tue, 01 Feb 2022 16:13:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1643760818; cv=none; d=google.com; s=arc-20160816; b=yB0NX5ZFa+5ZeBqVetHQ1gqBl2Wk/QuJHcteOwmOclv31/aRwv6sup0ay9QstoPX9T eUZiYAA8RBuwoolq7JYAkG1EoNZR4CHkZyI2SfSzXh+DJrbtdEI//06n9coDNHYhcbvZ EJF69gQX3aY3ZXvS141fMdiuH149HDtWPh9K8k0mv117/C+++uSKMv7/wml4BciWogJ2 jt9rlHulIHMD2hhAGCRYmxrS3k/NUZHkj15US4cLrwv1/sppqlXMSDK9hUSmgx3pYzYk nyO/iR8rLjFWX3y1wfEG5gLLZvsm9fFGPBT+HAltCA+MTCCGdPw5EixUu2dKWRQNc6iG r6XQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=L/30UYK5QYPHwycTdb7UthJcUng1IU+gFTNdxzb4AZg=; b=JXfBqQRynHg0nOws7YFRaQEwJ0JPzF3NVRV5uMKPbnIZEZpOv+tsi7ajieb7wyJOo1 HWm5g9sgvKHtVVwXNtJr4o6mkqkYTwAIB7Zqb1BTYzKuM6pUdtmuKfTwvqTge061WV8l VTVKbZJdYMdfFgIkrEz01Z/YDEDkx8bOGZt7n+3j8ygwYPrc1M+sw4EQuekwyU/2TX7W eDZQez+wfoxUdQOG/SWQv0wxBqSbrb+DftmrSNN9MyFLipEYwwNC5WEDBug0Ho21m1KM 3BUXR1/ypNbVZcJyzx2bKOHsf2bLifLxBxZNai8Fiy8tcO94EBBAhNgAHFg2yuDLTNyC oEsQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 18si9481790ejm.511.2022.02.01.16.13.37; Tue, 01 Feb 2022 16:13:38 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A759668B289; Wed, 2 Feb 2022 02:13:33 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E447668B193 for ; Wed, 2 Feb 2022 02:13:26 +0200 (EET) Received: by mail-pj1-f45.google.com with SMTP id z10-20020a17090acb0a00b001b520826011so5071927pjt.5 for ; Tue, 01 Feb 2022 16:13:26 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=4qFv0nR0JHxaZsxcgoA98ClQ2aF77HI5wEMVVlzBiq0=; b=I+tESetKXoud/NuUZOYJNQwJJ5gPssxyuNbDBEYCq8DA+57s++s/X1sFNFezmnSlh6 1G2Hlcycdn2xq4SR/EKvSuNyFgHeHhh94s+J1Moz2BsjX9/uvhsDeyFA+mBijJLeli1K xKlISBOx7I4++L+DjEqcqlUjggY0sXpZnSQsDxITv4szwc6l+t8eRbBo2dXYxDB/Jwq8 VyX69GnzRop82P8YT65y2B8iKBkfyCep3qERrULQJujMwNNf297P5LO71FYvplME+NA/ hP9p700ERBc520fkKE/MLBqwTFHGZi4yPE2hqqFnCn+/XQ3mCMt2pQuHUIUWKbHmfwWN XdZA== X-Gm-Message-State: AOAM531yeAW1r3d9O6HTmavVnHO8itLC6KCx3pxK9Q3x+KljnstC4L7J EF8G8+l3z0pm/H6Ms8+07B30XjJhn2k= X-Received: by 2002:a17:902:ea06:: with SMTP id s6mr28471994plg.68.1643760804354; Tue, 01 Feb 2022 16:13:24 -0800 (PST) Received: from localhost (76-14-89-2.sf-cable.astound.net. [76.14.89.2]) by smtp.gmail.com with ESMTPSA id a125sm15103074pfa.205.2022.02.01.16.13.22 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 01 Feb 2022 16:13:23 -0800 (PST) Received: by localhost (sSMTP sendmail emulation); Tue, 01 Feb 2022 16:13:11 -0800 From: pal@sandflow.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 1 Feb 2022 16:12:59 -0800 Message-Id: <20220202001302.4430-1-pal@sandflow.com> X-Mailer: git-send-email 2.35.0.windows.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 1/4] avformat/imf: open resources only when first needed X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Pierre-Anthony Lemieux Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: zv86Iis9HQKZ From: Pierre-Anthony Lemieux IMF CPLs can reference thousands of files, which can result in system limits for the number of open files to be exceeded. The following patch opens and closes files as needed. Addresses https://trac.ffmpeg.org/ticket/9623 --- libavformat/imfdec.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/libavformat/imfdec.c b/libavformat/imfdec.c index d67c9b8898..e6a1020ecc 100644 --- a/libavformat/imfdec.c +++ b/libavformat/imfdec.c @@ -103,10 +103,11 @@ typedef struct IMFVirtualTrackPlaybackCtx { int32_t index; /**< Track index in playlist */ AVRational current_timestamp; /**< Current temporal position */ AVRational duration; /**< Overall duration */ - uint32_t resource_count; /**< Number of resources */ + uint32_t resource_count; /**< Number of resources (<= INT32_MAX) */ unsigned int resources_alloc_sz; /**< Size of the buffer holding the resource */ IMFVirtualTrackResourcePlaybackCtx *resources; /**< Buffer holding the resources */ - uint32_t current_resource_index; /**< Current resource */ + int32_t current_resource_index; /**< Index of the current resource in resources, + or < 0 if a current resource has yet to be selected */ int64_t last_pts; /**< Last timestamp */ } IMFVirtualTrackPlaybackCtx; @@ -435,7 +436,6 @@ static int open_track_file_resource(AVFormatContext *s, IMFContext *c = s->priv_data; IMFAssetLocator *asset_locator; void *tmp; - int ret; asset_locator = find_asset_map_locator(&c->asset_locator_map, track_file_resource->track_file_uuid); if (!asset_locator) { @@ -452,7 +452,7 @@ static int open_track_file_resource(AVFormatContext *s, UID_ARG(asset_locator->uuid), asset_locator->absolute_uri); - if (track->resource_count > UINT32_MAX - track_file_resource->base.repeat_count + if (track->resource_count > INT32_MAX - track_file_resource->base.repeat_count || (track->resource_count + track_file_resource->base.repeat_count) > INT_MAX / sizeof(IMFVirtualTrackResourcePlaybackCtx)) return AVERROR(ENOMEM); @@ -470,8 +470,6 @@ static int open_track_file_resource(AVFormatContext *s, vt_ctx.locator = asset_locator; vt_ctx.resource = track_file_resource; vt_ctx.ctx = NULL; - if ((ret = open_track_resource_context(s, &vt_ctx)) != 0) - return ret; track->resources[track->resource_count++] = vt_ctx; track->duration = av_add_q(track->duration, av_make_q((int)track_file_resource->base.duration @@ -501,6 +499,7 @@ static int open_virtual_track(AVFormatContext *s, if (!(track = av_mallocz(sizeof(IMFVirtualTrackPlaybackCtx)))) return AVERROR(ENOMEM); + track->current_resource_index = -1; track->index = track_index; track->duration = av_make_q(0, 1); @@ -551,6 +550,7 @@ static int set_context_streams_from_tracks(AVFormatContext *s) AVStream *first_resource_stream; /* Open the first resource of the track to get stream information */ + open_track_resource_context(s, &c->tracks[i]->resources[0]); first_resource_stream = c->tracks[i]->resources[0].ctx->streams[0]; av_log(s, AV_LOG_DEBUG, "Open the first resource of track %d\n", c->tracks[i]->index); @@ -741,7 +741,8 @@ static IMFVirtualTrackResourcePlaybackCtx *get_resource_context_for_timestamp(AV track->index); if (open_track_resource_context(s, &(track->resources[i])) != 0) return NULL; - avformat_close_input(&(track->resources[track->current_resource_index].ctx)); + if (track->current_resource_index > 0) + avformat_close_input(&track->resources[track->current_resource_index].ctx); track->current_resource_index = i; }