From patchwork Tue Mar 22 17:23:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Gaullier X-Patchwork-Id: 34903 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:ab0:5fda:0:0:0:0:0 with SMTP id g26csp714201uaj; Tue, 22 Mar 2022 10:23:53 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy9+JWclcZfE2R3v6BZSrX1pk5WZCPq4mnL/mEqe7LLqnzQ9TrIbUGMEgZYpFDOT4gPV2hT X-Received: by 2002:a17:907:7815:b0:6ce:5242:1280 with SMTP id la21-20020a170907781500b006ce52421280mr27117534ejc.217.1647969833389; Tue, 22 Mar 2022 10:23:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1647969833; cv=none; d=google.com; s=arc-20160816; b=jZ0OG7Z5KX0vkxzXQaGylrDpwM8MCEjGhsmQHTWcd35Gtl77fWNIZjc/WXXildVDyH xcxr8mZ9mPoe8tEt94mBvUj1OjOV8BBvRLO/xMA77WcraUmdgpnRK3LHgQPYnV86mVcz RFuPee+qJ+wTY3ZaWAf6F+vBZDw/TagxJPzhwk7p4hKP8AyDPx8xJZW6qy91OWWoxG3X JpkCNUyyRyOnSe1k6v5dBVKZg85tzWJmMrJ7IxPgjUTHBzy8GRk7u36WYTbX5LGSyoU0 vcPLXvRFiQXiIXDuxJ5Qj1MzjtfauNIiYKhcnlwmAL3+ed4sCCKBUbMYvGNneqMz9kpc b8Rg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=xZIWOiifMaYW0rp94dqpzjnMGE93hxX6Vwcyd+gvC+g=; b=ORwxiySMXy4UOcfTruRR8itDlGGXbq/9u83aDP5OODFml7uoNVYHsAW9X/rCzoLzHb 9+KVA2Ma3siHGCvA/b+R2SSDoJTB9jA2ZLm/z6oIkOZC1OMw+yQ1Amept6FO0UPtX2xh grJR2HwYvJ6B25gd9lW+JlQpqQl6nO8h0W7kLG+AmPFOMeZwCI2WZaYD0+wJMZ+5KfSN Wuu+dg2hRuMB0EMom8tmZBQWulsBjuqGIyBp0ZPymKC5COoZDCrLe2UaGSO7LSIGyvEz 2xhI44I49aDeTVFGuOn0Qq/E7ibDH4C0bwE+MwtJAOebA1Nxq4/tMAmCULR/9b68PFca UOdQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id fx5-20020a170906b74500b006df76385d99si9366664ejb.569.2022.03.22.10.23.52; Tue, 22 Mar 2022 10:23:53 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 45C1A68B10B; Tue, 22 Mar 2022 19:23:49 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from srv-infra-1.infra.inf.glb.tvvideoms.com (www.inf.tvvideoms.com [213.205.126.156]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 752A5680574 for ; Tue, 22 Mar 2022 19:23:42 +0200 (EET) Received: from cji.paris (unknown [172.16.3.159]) by srv-infra-1.infra.inf.glb.tvvideoms.com (Postfix) with ESMTP id EB02D2141B; Tue, 22 Mar 2022 17:23:41 +0000 (UTC) From: Nicolas Gaullier To: ffmpeg-devel@ffmpeg.org Date: Tue, 22 Mar 2022 18:23:36 +0100 Message-Id: <20220322172337.65005-1-nicolas.gaullier@cji.paris> X-Mailer: git-send-email 2.34.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 1/2] lavf/mpegenc: fix ever-growing fifo size since the new API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nicolas Gaullier Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: xU6aqmF6MP87 The older av_fifo_realloc2 implemented an auto grow that should be ported as such in the new API. This patch introduces a limitation in the fifo buffer size. The default is set to 128MB and may be overriden by a new user option. The amount of memory allocated depends on multiple factors, including the number of audio streams. A worst case scenario is where an out-of-spec high video bitrate is combined with numerous low bitrate audios. A fatal error mentions the availability of fifo_size_limit option. Fix regressing since ea511196a6c85eb433e10cdbecb0b2c722faf20d Signed-off-by: Nicolas Gaullier --- libavformat/mpegenc.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/libavformat/mpegenc.c b/libavformat/mpegenc.c index cc47a43288..e0955a7d33 100644 --- a/libavformat/mpegenc.c +++ b/libavformat/mpegenc.c @@ -84,6 +84,7 @@ typedef struct MpegMuxContext { int64_t vcd_padding_bytes_written; int preload; + int fifo_size_limit; } MpegMuxContext; extern const AVOutputFormat ff_mpeg1vcd_muxer; @@ -461,9 +462,10 @@ static av_cold int mpeg_mux_init(AVFormatContext *ctx) av_get_media_type_string(st->codecpar->codec_type), i); return AVERROR(EINVAL); } - stream->fifo = av_fifo_alloc2(16, 1, 0); + stream->fifo = av_fifo_alloc2(16, 1, AV_FIFO_FLAG_AUTO_GROW); if (!stream->fifo) return AVERROR(ENOMEM); + av_fifo_auto_grow_limit(stream->fifo, s->fifo_size_limit); } bitrate = 0; audio_bitrate = 0; @@ -1207,10 +1209,6 @@ static int mpeg_mux_write_packet(AVFormatContext *ctx, AVPacket *pkt) pkt_desc->unwritten_size = pkt_desc->size = size; - ret = av_fifo_grow2(stream->fifo, size); - if (ret < 0) - return ret; - if (s->is_dvd) { // min VOBU length 0.4 seconds (mpucoder) if (is_iframe && @@ -1222,10 +1220,15 @@ static int mpeg_mux_write_packet(AVFormatContext *ctx, AVPacket *pkt) } } - av_fifo_write(stream->fifo, buf, size); + ret = av_fifo_write(stream->fifo, buf, size); + if (ret == AVERROR(ENOSPC)) + av_log(s, AV_LOG_FATAL, "Input stream buffer overrun. " + "To avoid, increase fifo_size_limit option.\n"); + if (ret < 0) + return ret; for (;;) { - int ret = output_packet(ctx, 0); + ret = output_packet(ctx, 0); if (ret <= 0) return ret; } @@ -1277,6 +1280,7 @@ static void mpeg_mux_deinit(AVFormatContext *ctx) static const AVOption options[] = { { "muxrate", NULL, OFFSET(user_mux_rate), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, ((1<<22) - 1) * (8 * 50), E }, { "preload", "Initial demux-decode delay in microseconds.", OFFSET(preload), AV_OPT_TYPE_INT, { .i64 = 500000 }, 0, INT_MAX, E }, + { "fifo_size_limit", "Maximum allowed memory for buffering an input stream in bytes", OFFSET(fifo_size_limit), AV_OPT_TYPE_INT, {.i64 = 128 * 1024 * 1024 }, 16, FFMIN(INT_MAX, SIZE_MAX), E}, { NULL }, }; From patchwork Tue Mar 22 17:23:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Gaullier X-Patchwork-Id: 34904 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:ab0:5fda:0:0:0:0:0 with SMTP id g26csp714244uaj; Tue, 22 Mar 2022 10:24:02 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwDbudTk8gqEf/r5TdgmuIVX7fYo1vLl/FhewuZVpbNFCjsJtYv5TUYYRK7GE14ppZUkFLZ X-Received: by 2002:a17:906:5ad6:b0:6e0:1799:4a15 with SMTP id x22-20020a1709065ad600b006e017994a15mr9654531ejs.594.1647969842578; Tue, 22 Mar 2022 10:24:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1647969842; cv=none; d=google.com; s=arc-20160816; b=cEplPbTICUuXj3SPvdjM3ponyJncjfTQ6Fx/sFRwd1XKzRnX+fQGXPQIWf1eI27Um5 O+IGNhl9Xjp/LLIBAZrAvmMqepRKjKDaFZf1nB+gUVnXIQt+AUwhs3U3SsKO6+ZGBcFz LO/wzPD7gLk4nzmCNo1LTIkKrgnQgi0j8qo/MAbLCUh7s1imjgU/oLvSI0ZGx5R2X21A alhTp2qakZViCdGHCfCV1yiuzv8Miw6z7pdB9+S/gP55puuAiOgAlTMbcK226CWGcZD1 WUZi0o4TRX/oUnwXSaLPTT7vK9kQW5AJHaqDFIlZkBxYufUPVnYDZubdKGxy+KpqJRes QwJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=bDY4awBxMXKg/BGRTYd5x8WGsVCaowWS/habhkVk0rI=; b=qqmJcjrILPMR99UDdQo9nirgYkLctxl9WdHjkcakqoFQO2re1qXEnlCMag8glM03e3 Y+BmHeJUEGMj/n0re7JR/71ThgXGcPd6wvAl/n61PcPsqiBwbtIjrFFYxYA5jDOWDXsQ X4dlnldK7iZtHILqjdqJSh58M0W6NdVPDfVXTmtdGg1zy6tncBS4OdD3FridizE7GAPU 44HZY0c+2CbJOY2MSq2neTqb05j8Ky7TriICQA7Z6Qs886B7lzjBsUZxO7FClbAJCibf 8XHjME+k7AYSM5UdeaiTdG43UYVu/cZ21I1+W1Ee7eYnqiFmNMw/oqXa0w7fMfLBpTMW W0Jw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id z68-20020a509e4a000000b00418c2b5bdf7si13125212ede.217.2022.03.22.10.24.02; Tue, 22 Mar 2022 10:24:02 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2CA6868B15A; Tue, 22 Mar 2022 19:23:50 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from srv-infra-1.infra.inf.glb.tvvideoms.com (www.inf.tvvideoms.com [213.205.126.156]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8F87C68AC00 for ; Tue, 22 Mar 2022 19:23:42 +0200 (EET) Received: from cji.paris (unknown [172.16.3.159]) by srv-infra-1.infra.inf.glb.tvvideoms.com (Postfix) with ESMTP id 233E82158C; Tue, 22 Mar 2022 17:23:42 +0000 (UTC) From: Nicolas Gaullier To: ffmpeg-devel@ffmpeg.org Date: Tue, 22 Mar 2022 18:23:37 +0100 Message-Id: <20220322172337.65005-2-nicolas.gaullier@cji.paris> X-Mailer: git-send-email 2.34.0 In-Reply-To: <20220322172337.65005-1-nicolas.gaullier@cji.paris> References: <20220322172337.65005-1-nicolas.gaullier@cji.paris> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 2/2] lavf/mpegenc: fix termination following a fifo overrun X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nicolas Gaullier Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 8f0vIZOTi8PW Avoid an infinite 'retry' loop in output_packet when flushing. Signed-off-by: Nicolas Gaullier --- libavformat/mpegenc.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/libavformat/mpegenc.c b/libavformat/mpegenc.c index e0955a7d33..91574ca95a 100644 --- a/libavformat/mpegenc.c +++ b/libavformat/mpegenc.c @@ -85,6 +85,7 @@ typedef struct MpegMuxContext { int preload; int fifo_size_limit; + int fifo_size_exceeded; } MpegMuxContext; extern const AVOutputFormat ff_mpeg1vcd_muxer; @@ -1221,9 +1222,11 @@ static int mpeg_mux_write_packet(AVFormatContext *ctx, AVPacket *pkt) } ret = av_fifo_write(stream->fifo, buf, size); - if (ret == AVERROR(ENOSPC)) + if (ret == AVERROR(ENOSPC)) { + s->fifo_size_exceeded = 1; av_log(s, AV_LOG_FATAL, "Input stream buffer overrun. " "To avoid, increase fifo_size_limit option.\n"); + } if (ret < 0) return ret; @@ -1236,9 +1239,13 @@ static int mpeg_mux_write_packet(AVFormatContext *ctx, AVPacket *pkt) static int mpeg_mux_end(AVFormatContext *ctx) { + MpegMuxContext *s = ctx->priv_data; StreamInfo *stream; int i; + if (s->fifo_size_exceeded) + return 0; + for (;;) { int ret = output_packet(ctx, 1); if (ret < 0)