From patchwork Thu Feb 3 04:07:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre-Anthony Lemieux X-Patchwork-Id: 34088 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2c4e:0:0:0:0 with SMTP id x14csp2133264iov; Wed, 2 Feb 2022 20:08:15 -0800 (PST) X-Google-Smtp-Source: ABdhPJwwP4yxwuxwjttJ5/neeI9A36oclKDJRp1W6HhMUKK8D7VI2wWj04tHS557JFg+6U3e1iwB X-Received: by 2002:a17:907:1c92:: with SMTP id nb18mr27257775ejc.309.1643861294984; Wed, 02 Feb 2022 20:08:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1643861294; cv=none; d=google.com; s=arc-20160816; b=c2Zeg9N7MrhdVhkIhcDuk34etGwL/6B/LNgOK6PMWU+GKmR/YoIdvgw4c/TzNCVk0O LxvIdG/s3H7Yb5erwwnjT1z8AV9rbpZkDYZfaeFAlIkxn8+W7UEWAe1pu52zPxBoNodn T1YqgdoB5nVIB+d5gN090fcE1FouyxPUH1CBBtuSx4RsKRNL94CriZJNnNcEEFQRGZp1 7Andft7R0qgWiS8imzynFQ1e5yZ/O6NWU1kuqLJp9NkGd5OkQj+tO2+L4IYdvTaHq0E0 4qcNtzRk+C4jM2tOOGazaH+7xKazND5W4u2853fQzt9mMNmYwS5IjunqXTcrRIqKnw/8 9NEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=L/30UYK5QYPHwycTdb7UthJcUng1IU+gFTNdxzb4AZg=; b=d1kd87qKDaRQleChObyN873teImZlauOnb6YL6DY2xEH3SoCysfT+pHoJ6Nm994Ul8 wLP3kJu8txpdQbyuk3Uxm+9h422s3vkD8c0OsURKlGlBppmZ6TnT6QVH/taRQYFh2wnL b2V2KGy2VH3kiQelFd633yFbPWJehQuCnEMA2I4zje2wQhcnJHBIdR2K8amAbAOtGGby 0fQ+4Glj/9LWIE23yOVyqLX5qfzlT8tJ8H6FCs1kLaXk1t0kGsCb4vl9qTlWopllspbp IztZ6M4SMmn47e6ij5Xpa7HAld/KxFVKWAMtmGhaMYHqPlKhGnkoj0yBkt7ozGx/0o1b f9OQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id m3si7457174edq.548.2022.02.02.20.08.14; Wed, 02 Feb 2022 20:08:14 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 68AEF68AB96; Thu, 3 Feb 2022 06:08:10 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9022468AB96 for ; Thu, 3 Feb 2022 06:08:03 +0200 (EET) Received: by mail-pl1-f170.google.com with SMTP id k17so1153456plk.0 for ; Wed, 02 Feb 2022 20:08:03 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=4qFv0nR0JHxaZsxcgoA98ClQ2aF77HI5wEMVVlzBiq0=; b=G2zcgdQpa+apaY6XKLwRdGuZ+l9x2yZkX8haeGoUiInTVwS8/kntgEdpDRwtETII5O xUpnaWpXneC5OfSqW8yeVL4CthZUlL3Q4UpCTv7vD5im0bTL/VvB3YuMQsKSZipnXmT6 AeyzuwHjjPamxCQpd46hGGsv6Fasx1T+feqfWSBHyg6aJ1PAtXAraPl2PNinsV/eis49 9GJQO+NgmaNkmVMl8bZIeglCFoPTxu//MkI7JmgPO3qE8ZQV5dERCuFOZYhfT7wf7ifl /huO/XvoG+vGruThUdenUujUVyc1PF04RuK10N+Ww6mcMXKPVwVeCp+rcjwRukFAZJDA AJlg== X-Gm-Message-State: AOAM5313+bu5QNtz/aOjJryHilX9r43LIDPtLsoReTA7K2+6wN2k0gGj aN9z1MCW2s/vvoSc3C5h07fX6mH7qZk= X-Received: by 2002:a17:903:11c3:: with SMTP id q3mr34004497plh.97.1643861281317; Wed, 02 Feb 2022 20:08:01 -0800 (PST) Received: from localhost (76-14-89-2.sf-cable.astound.net. [76.14.89.2]) by smtp.gmail.com with ESMTPSA id nu7sm8868129pjb.30.2022.02.02.20.07.59 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 02 Feb 2022 20:08:00 -0800 (PST) Received: by localhost (sSMTP sendmail emulation); Wed, 02 Feb 2022 20:07:47 -0800 From: pal@sandflow.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 2 Feb 2022 20:07:42 -0800 Message-Id: <20220203040745.10983-1-pal@sandflow.com> X-Mailer: git-send-email 2.35.0.windows.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 1/4] avformat/imf: open resources only when first needed X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Pierre-Anthony Lemieux Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: wN9Zp+IK6vz0 From: Pierre-Anthony Lemieux IMF CPLs can reference thousands of files, which can result in system limits for the number of open files to be exceeded. The following patch opens and closes files as needed. Addresses https://trac.ffmpeg.org/ticket/9623 --- libavformat/imfdec.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/libavformat/imfdec.c b/libavformat/imfdec.c index d67c9b8898..e6a1020ecc 100644 --- a/libavformat/imfdec.c +++ b/libavformat/imfdec.c @@ -103,10 +103,11 @@ typedef struct IMFVirtualTrackPlaybackCtx { int32_t index; /**< Track index in playlist */ AVRational current_timestamp; /**< Current temporal position */ AVRational duration; /**< Overall duration */ - uint32_t resource_count; /**< Number of resources */ + uint32_t resource_count; /**< Number of resources (<= INT32_MAX) */ unsigned int resources_alloc_sz; /**< Size of the buffer holding the resource */ IMFVirtualTrackResourcePlaybackCtx *resources; /**< Buffer holding the resources */ - uint32_t current_resource_index; /**< Current resource */ + int32_t current_resource_index; /**< Index of the current resource in resources, + or < 0 if a current resource has yet to be selected */ int64_t last_pts; /**< Last timestamp */ } IMFVirtualTrackPlaybackCtx; @@ -435,7 +436,6 @@ static int open_track_file_resource(AVFormatContext *s, IMFContext *c = s->priv_data; IMFAssetLocator *asset_locator; void *tmp; - int ret; asset_locator = find_asset_map_locator(&c->asset_locator_map, track_file_resource->track_file_uuid); if (!asset_locator) { @@ -452,7 +452,7 @@ static int open_track_file_resource(AVFormatContext *s, UID_ARG(asset_locator->uuid), asset_locator->absolute_uri); - if (track->resource_count > UINT32_MAX - track_file_resource->base.repeat_count + if (track->resource_count > INT32_MAX - track_file_resource->base.repeat_count || (track->resource_count + track_file_resource->base.repeat_count) > INT_MAX / sizeof(IMFVirtualTrackResourcePlaybackCtx)) return AVERROR(ENOMEM); @@ -470,8 +470,6 @@ static int open_track_file_resource(AVFormatContext *s, vt_ctx.locator = asset_locator; vt_ctx.resource = track_file_resource; vt_ctx.ctx = NULL; - if ((ret = open_track_resource_context(s, &vt_ctx)) != 0) - return ret; track->resources[track->resource_count++] = vt_ctx; track->duration = av_add_q(track->duration, av_make_q((int)track_file_resource->base.duration @@ -501,6 +499,7 @@ static int open_virtual_track(AVFormatContext *s, if (!(track = av_mallocz(sizeof(IMFVirtualTrackPlaybackCtx)))) return AVERROR(ENOMEM); + track->current_resource_index = -1; track->index = track_index; track->duration = av_make_q(0, 1); @@ -551,6 +550,7 @@ static int set_context_streams_from_tracks(AVFormatContext *s) AVStream *first_resource_stream; /* Open the first resource of the track to get stream information */ + open_track_resource_context(s, &c->tracks[i]->resources[0]); first_resource_stream = c->tracks[i]->resources[0].ctx->streams[0]; av_log(s, AV_LOG_DEBUG, "Open the first resource of track %d\n", c->tracks[i]->index); @@ -741,7 +741,8 @@ static IMFVirtualTrackResourcePlaybackCtx *get_resource_context_for_timestamp(AV track->index); if (open_track_resource_context(s, &(track->resources[i])) != 0) return NULL; - avformat_close_input(&(track->resources[track->current_resource_index].ctx)); + if (track->current_resource_index > 0) + avformat_close_input(&track->resources[track->current_resource_index].ctx); track->current_resource_index = i; }