From patchwork Wed Feb 16 16:54:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre-Anthony Lemieux X-Patchwork-Id: 34343 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6838:90eb:0:0:0:0 with SMTP id a11csp1391738nkf; Wed, 16 Feb 2022 08:54:34 -0800 (PST) X-Google-Smtp-Source: ABdhPJwdKcnuIWr6sFqPTyG07p7H9qcPiJjtyPDSjNaeLSmFkdUsjVkoMVLhSw9OJ8l18+5fZ/AH X-Received: by 2002:a05:6402:1d8d:b0:408:4a31:9790 with SMTP id dk13-20020a0564021d8d00b004084a319790mr4084086edb.92.1645030474051; Wed, 16 Feb 2022 08:54:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1645030474; cv=none; d=google.com; s=arc-20160816; b=KFLFdhILRonpTHUuGZIAEGBPz3sV0CC1pSwmj5sf9k66LQKYVXgaYqqq6+T8poPKLp e6tHyPvRTRQbrF4jdaqrb60EqOBAW03UIO3bEtcqnumLYKKX3CsKxbSx/fV0l6TqVVIr agtmlZz/yNbbcLCc1zNK2SY8M+/oht73BDc8RNMM4CQXoqaDcq8ZFCQGmsQJnxRLUuas EKQ8zyVxpxNAVhVFkR3aavzuFYsWmthabpR3DWZRl/Qt99imq2g1oFfxLpuVhcJ1f0tX NpzjlmhVSdEdwJjltBDkTT3v/vUBBqRvwvWZHW2Ir0QDd1p0wD5t+B7UBHHvRk83SqmS Bszw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=I5T5Z/6znMFuaLVgp9Hfr30bTMK9KGbwxUzkeHRRgvs=; b=JdoQUjRbRvoQM0s98qxkgm6qVvz9AYBURfaP6R2O0pbEbHbOXVKA38EU9xrwxLDXqm 0oIoTxG1snqBEoyWiy8y7H+FKDl1fJYYQJFIAyvTiumILph308HK/SdxmZCgM9h4GVw1 Bcpbs9JIfP3Et5fPk5mpONX13zCf5CmxAMoBWs/UYymPkeNWRW1M7TsqkVN2W8T2xwbg UkrzQttTPjE/PVKH1Uma65QIvneTSbKn09EvSCQXTs89K8kisdVs+1+sWzKGOl4iv23u wUiyUzy68iKVaCePV73W5Ae5EG5P+OakvMPd5aLIX6B+f7mpE6TYDgj11h3eoFNjwdi1 ArSA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id cw16si184664ejc.700.2022.02.16.08.54.33; Wed, 16 Feb 2022 08:54:34 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DB57B68B09C; Wed, 16 Feb 2022 18:54:30 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f177.google.com (mail-pg1-f177.google.com [209.85.215.177]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2507768B09C for ; Wed, 16 Feb 2022 18:54:24 +0200 (EET) Received: by mail-pg1-f177.google.com with SMTP id r76so2639995pgr.10 for ; Wed, 16 Feb 2022 08:54:24 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=cxtqvGl+p2ZE9tNaWEe2D/86ewtXEJH53t7bH34X75k=; b=Gb8pCEACiSdCz0ibGs6BBAddRCR3pO1kmrYJDMEE3+ll9loe59NDza9hmTNMj2AnHz zAhLF1d8mTqCYjumS1HdJ3GBYAPfJvSNTqUpxrGv40UypkN2TiTUr1XldBJUeEcZknsM hPxcwwu2Mr4shf7PDNf+t9TA7HkfhbdxNMi1JAaWqoxuN3tVy0G7vN3bZUzpbqN8bdn8 IraEpRghE7OJQOXs4RoBbEJnsVa689xszSDc0CDoAxn/JCLLQFFc8cgBZ+q9EMmHypOn wzz/ujrsf5Ak2EwH5rZqvHJ+0RdAJVWBS1vLt0lQ6nIKWwQJWLTphMdVV8IUFZ/SR7h+ 3hDQ== X-Gm-Message-State: AOAM530qqeaLgARcZqdh3S01jCUo+wsYFDTB3LNs16wBmklOi5Oh6cBr SWkusuiRvmHMXQ3HcNQSGcvgfgVsiNQ= X-Received: by 2002:a63:1844:0:b0:36c:6a88:ad72 with SMTP id 4-20020a631844000000b0036c6a88ad72mr2928812pgy.329.1645030461945; Wed, 16 Feb 2022 08:54:21 -0800 (PST) Received: from localhost (76-14-89-2.sf-cable.astound.net. [76.14.89.2]) by smtp.gmail.com with ESMTPSA id gb8sm5920329pjb.21.2022.02.16.08.54.20 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 16 Feb 2022 08:54:21 -0800 (PST) Received: by localhost (sSMTP sendmail emulation); Wed, 16 Feb 2022 08:54:12 -0800 From: pal@sandflow.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 16 Feb 2022 08:54:08 -0800 Message-Id: <20220216165410.17063-1-pal@sandflow.com> X-Mailer: git-send-email 2.35.1.windows.2 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 1/3] avformat/imf: open resources only when first needed X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Pierre-Anthony Lemieux Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: NLUvxczI38zc From: Pierre-Anthony Lemieux IMF CPLs can reference thousands of files, which can result in system limits for the number of open files to be exceeded. The following patch opens and closes files as needed. Addresses https://trac.ffmpeg.org/ticket/9623 --- libavformat/imfdec.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/libavformat/imfdec.c b/libavformat/imfdec.c index 847dead7f0..48bd5c78e8 100644 --- a/libavformat/imfdec.c +++ b/libavformat/imfdec.c @@ -103,10 +103,11 @@ typedef struct IMFVirtualTrackPlaybackCtx { int32_t index; /**< Track index in playlist */ AVRational current_timestamp; /**< Current temporal position */ AVRational duration; /**< Overall duration */ - uint32_t resource_count; /**< Number of resources */ + uint32_t resource_count; /**< Number of resources (<= INT32_MAX) */ unsigned int resources_alloc_sz; /**< Size of the buffer holding the resource */ IMFVirtualTrackResourcePlaybackCtx *resources; /**< Buffer holding the resources */ - uint32_t current_resource_index; /**< Current resource */ + int32_t current_resource_index; /**< Index of the current resource in resources, + or < 0 if a current resource has yet to be selected */ int64_t last_pts; /**< Last timestamp */ } IMFVirtualTrackPlaybackCtx; @@ -435,7 +436,6 @@ static int open_track_file_resource(AVFormatContext *s, IMFContext *c = s->priv_data; IMFAssetLocator *asset_locator; void *tmp; - int ret; asset_locator = find_asset_map_locator(&c->asset_locator_map, track_file_resource->track_file_uuid); if (!asset_locator) { @@ -452,7 +452,7 @@ static int open_track_file_resource(AVFormatContext *s, UID_ARG(asset_locator->uuid), asset_locator->absolute_uri); - if (track->resource_count > UINT32_MAX - track_file_resource->base.repeat_count + if (track->resource_count > INT32_MAX - track_file_resource->base.repeat_count || (track->resource_count + track_file_resource->base.repeat_count) > INT_MAX / sizeof(IMFVirtualTrackResourcePlaybackCtx)) return AVERROR(ENOMEM); @@ -470,8 +470,6 @@ static int open_track_file_resource(AVFormatContext *s, vt_ctx.locator = asset_locator; vt_ctx.resource = track_file_resource; vt_ctx.ctx = NULL; - if ((ret = open_track_resource_context(s, &vt_ctx)) != 0) - return ret; track->resources[track->resource_count++] = vt_ctx; track->duration = av_add_q(track->duration, av_make_q((int)track_file_resource->base.duration @@ -501,6 +499,7 @@ static int open_virtual_track(AVFormatContext *s, if (!(track = av_mallocz(sizeof(IMFVirtualTrackPlaybackCtx)))) return AVERROR(ENOMEM); + track->current_resource_index = -1; track->index = track_index; track->duration = av_make_q(0, 1); @@ -551,6 +550,9 @@ static int set_context_streams_from_tracks(AVFormatContext *s) AVStream *first_resource_stream; /* Open the first resource of the track to get stream information */ + ret = open_track_resource_context(s, &c->tracks[i]->resources[0]); + if (ret) + return ret; first_resource_stream = c->tracks[i]->resources[0].ctx->streams[0]; av_log(s, AV_LOG_DEBUG, "Open the first resource of track %d\n", c->tracks[i]->index); @@ -741,7 +743,8 @@ static IMFVirtualTrackResourcePlaybackCtx *get_resource_context_for_timestamp(AV track->index); if (open_track_resource_context(s, &(track->resources[i])) != 0) return NULL; - avformat_close_input(&(track->resources[track->current_resource_index].ctx)); + if (track->current_resource_index > 0) + avformat_close_input(&track->resources[track->current_resource_index].ctx); track->current_resource_index = i; }