From patchwork Tue Mar 22 16:39:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Gaullier X-Patchwork-Id: 34900 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:ab0:5fda:0:0:0:0:0 with SMTP id g26csp706972uaj; Tue, 22 Mar 2022 09:39:24 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyQfMq8/6xpVB5OTdIDj2FFLaOm0l15/CsrR3za0C+VWn4NlNNMcD8p9+auZ+mTecegegIu X-Received: by 2002:a17:906:948:b0:6d6:e479:1fe4 with SMTP id j8-20020a170906094800b006d6e4791fe4mr26441431ejd.240.1647967164635; Tue, 22 Mar 2022 09:39:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1647967164; cv=none; d=google.com; s=arc-20160816; b=H2xXzF1rSzGWMo79FCo7s/Z1RNdvlOq5dkIayaTLQf/1gSr82z8ko5Z2+6u4lW9PfU S8o0Q4mjOQybcKiiuIkXBKtvEuHZBGzTXGaphF4ZGQzD2zW6qv5zvHytPWpGUQJ4/XaP lh58ZF1qzBrhTrxH/QdQpUqUAO2DHI/upFoM8GsxDASerIFF7namWX1Vu3xKvHBulAx4 7hY0FScY5ukN3G0O8gyqKXNFqotrMz00lq32GtSBmhnc+H4DDxQQplzTcODzda0/cyxi Ge6/M9cOlp/eKV6pQZtfuVCA7lbhj2avsis9l/J3MEpaarUxCUOWSRKXaqq7lO7RLFPd PQfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=Kt+P+YEjxsRmUsMFYYqcj4Akfm/Ny3ksA/IRuoo5udw=; b=l7bFzoEXeMJgElLx66+/2tGGSpzxnmumvjlt90T6InryIENDA8tG7DgRVFsmXepb5+ A6d6CfMskcgV8j+dVGZKCRxzH7E+ZlC/ciuUuiDmvF4Hv0R4KaferoOumsHXP+1PKAjN +0gWWnbxEfRpqaQUFa0hHrZA+KIzS4zgni8weJ5R5zK59Oh1FCdZYZy1aGr0KA7vQEa5 71DlwpKStPcqZY1ELzWG2gqQ1xSUv01yuvuZC3MAqdDaLZ63S5ZVt3AlzvE4Loe4mhgw d2MwxpFHxQtebnKNjxo3hrj+7sTgl+47Z3dh8SQr227bfnR7ySR6td69OfbsBIDmyaEO LMWg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g21-20020a50d0d5000000b00418c2b5bf26si12329596edf.520.2022.03.22.09.39.23; Tue, 22 Mar 2022 09:39:24 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4819B68B151; Tue, 22 Mar 2022 18:39:20 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from srv-infra-2.infra.inf.glb.tvvideoms.com (www.inf.tvvideoms.com [213.205.126.156]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3025868A6F6 for ; Tue, 22 Mar 2022 18:39:14 +0200 (EET) Received: from cji.paris (unknown [172.16.3.159]) by srv-infra-2.infra.inf.glb.tvvideoms.com (Postfix) with ESMTP id 2D96442A62; Tue, 22 Mar 2022 16:39:13 +0000 (UTC) From: Nicolas Gaullier To: ffmpeg-devel@ffmpeg.org Date: Tue, 22 Mar 2022 17:39:10 +0100 Message-Id: <20220322163911.64772-1-nicolas.gaullier@cji.paris> X-Mailer: git-send-email 2.34.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 1/2] lavf/mpegenc: fix ever-growing fifo size since the new API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nicolas Gaullier Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: TaqNURp+SIT4 The older av_fifo_realloc2 implemented an auto grow that should be ported as such in the new API. This patch introduces a limitation in the fifo buffer size. The default is set to 128MB and may be overriden by a new user option. The amount of memory allocated depends on multiple factors, including the number of audio streams. A worst case scenario is where an out-of-spec high video bitrate is combined with numerous low bitrate audios. Fix regressing since ea511196a6c85eb433e10cdbecb0b2c722faf20d Signed-off-by: Nicolas Gaullier --- libavformat/mpegenc.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/libavformat/mpegenc.c b/libavformat/mpegenc.c index cc47a43288..5d755e3bdd 100644 --- a/libavformat/mpegenc.c +++ b/libavformat/mpegenc.c @@ -84,6 +84,7 @@ typedef struct MpegMuxContext { int64_t vcd_padding_bytes_written; int preload; + uint32_t fifo_size_limit; } MpegMuxContext; extern const AVOutputFormat ff_mpeg1vcd_muxer; @@ -461,9 +462,10 @@ static av_cold int mpeg_mux_init(AVFormatContext *ctx) av_get_media_type_string(st->codecpar->codec_type), i); return AVERROR(EINVAL); } - stream->fifo = av_fifo_alloc2(16, 1, 0); + stream->fifo = av_fifo_alloc2(16, 1, AV_FIFO_FLAG_AUTO_GROW); if (!stream->fifo) return AVERROR(ENOMEM); + av_fifo_auto_grow_limit(stream->fifo, s->fifo_size_limit); } bitrate = 0; audio_bitrate = 0; @@ -1151,7 +1153,7 @@ static int mpeg_mux_write_packet(AVFormatContext *ctx, AVPacket *pkt) StreamInfo *stream = st->priv_data; int64_t pts, dts; PacketDesc *pkt_desc; - int preload, ret; + int preload; const int is_iframe = st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO && (pkt->flags & AV_PKT_FLAG_KEY); @@ -1207,10 +1209,6 @@ static int mpeg_mux_write_packet(AVFormatContext *ctx, AVPacket *pkt) pkt_desc->unwritten_size = pkt_desc->size = size; - ret = av_fifo_grow2(stream->fifo, size); - if (ret < 0) - return ret; - if (s->is_dvd) { // min VOBU length 0.4 seconds (mpucoder) if (is_iframe && @@ -1277,6 +1275,7 @@ static void mpeg_mux_deinit(AVFormatContext *ctx) static const AVOption options[] = { { "muxrate", NULL, OFFSET(user_mux_rate), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, ((1<<22) - 1) * (8 * 50), E }, { "preload", "Initial demux-decode delay in microseconds.", OFFSET(preload), AV_OPT_TYPE_INT, { .i64 = 500000 }, 0, INT_MAX, E }, + { "fifo_size_limit", "Maximum allowed memory for buffering an input stream in bytes", OFFSET(fifo_size_limit), AV_OPT_TYPE_INT, {.i64 = 128 * 1024 * 1024 }, 0, UINT_MAX, E}, { NULL }, }; From patchwork Tue Mar 22 16:39:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Gaullier X-Patchwork-Id: 34901 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:ab0:5fda:0:0:0:0:0 with SMTP id g26csp706993uaj; Tue, 22 Mar 2022 09:39:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzeaSXYup29BMSMqZYr0Uv2nkz0Gg4qVc8IzWf4uwxu9YsDucR3hIX0HSBOf5vjbBoZwe7R X-Received: by 2002:a17:907:70cc:b0:6e0:1646:9123 with SMTP id yk12-20020a17090770cc00b006e016469123mr9173912ejb.329.1647967174437; Tue, 22 Mar 2022 09:39:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1647967174; cv=none; d=google.com; s=arc-20160816; b=qlC2WvQ+cf6b25MNKDtoeV6eGDkwV0zA0rjpKbMKMgQj+UIH8E8KY7N/8IY9IB0z6c 7qxub1lvlvXIjk3H4pyAbAeCMS6Gddz+pn42xJKmK673ugRfKKwO613R70VEgP1fciro bZsZOA4r3qaYgrsgr8Z3VZHnKmVbE4ozkz/euouUTHcFK0haqkNvXfcF3aFennx1IEZP eMaelvFYkWzNWrYe9+3u7FlayCLS1UXlTOc3cIHbBwbO9d+42/JeCHdIzjRkTFsq5ITA tATMQjWtdUhTopLvAFGNnEmCJeIoy36NmqJ25pS2xTVYMgVHam2M5x9h1Hjq5P4u8GHK H6Zg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=WcTnFp5z/1LeK5QiROAH6CIPdhpcmBOaCFUtcG3hGOE=; b=QuFaDlyVxhZa1ioCSa8xCNcEe+ykLhqZgjfJ2CWFZ2sAAGNxmsDXG9I+VKywXJa8ox Jf/PFKaxfAgTp07H+FwmhD2sF1aR1sN7c12X3Y30d7+OJj9rFHRW5Q8Z8gDyDrvxgiv4 ddxWRJg007Lk507nLE8EK36mrTCP1T5rdBIv+qz9PwbqUyCduhXu/um/QQy5lF5ISY3b +4mRX6AZg6Nz3Ujx49qTFaN4vXan2tc+EFNhHhLbgYsAxpJ1BoenCbiU1vzUmnVbTg5G oMmoH3XSD2KrY06Jthf6MK/nm3VaaBpqC2zBAKH8QYzFS+Jf8KW0nA6FAE3Zov6M4rKN qP/g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id r14-20020a50c00e000000b00418d7a006e3si12458828edb.263.2022.03.22.09.39.33; Tue, 22 Mar 2022 09:39:34 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7D86568B022; Tue, 22 Mar 2022 18:39:21 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from srv-infra-2.infra.inf.glb.tvvideoms.com (www.inf.tvvideoms.com [213.205.126.156]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2D58368A066 for ; Tue, 22 Mar 2022 18:39:14 +0200 (EET) Received: from cji.paris (unknown [172.16.3.159]) by srv-infra-2.infra.inf.glb.tvvideoms.com (Postfix) with ESMTP id 59C6E42A93; Tue, 22 Mar 2022 16:39:13 +0000 (UTC) From: Nicolas Gaullier To: ffmpeg-devel@ffmpeg.org Date: Tue, 22 Mar 2022 17:39:11 +0100 Message-Id: <20220322163911.64772-2-nicolas.gaullier@cji.paris> X-Mailer: git-send-email 2.34.0 In-Reply-To: <20220322163911.64772-1-nicolas.gaullier@cji.paris> References: <20220322163911.64772-1-nicolas.gaullier@cji.paris> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 2/2] lavf/mpegenc: fix termination following a fifo overrun X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nicolas Gaullier Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: udAvd4lePNat Avoid an infinite 'retry' loop in output_packet when flushing. A fatal error mentions the availability of fifo_size_limit option. Signed-off-by: Nicolas Gaullier --- libavformat/mpegenc.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/libavformat/mpegenc.c b/libavformat/mpegenc.c index 5d755e3bdd..eff4531037 100644 --- a/libavformat/mpegenc.c +++ b/libavformat/mpegenc.c @@ -85,6 +85,7 @@ typedef struct MpegMuxContext { int preload; uint32_t fifo_size_limit; + int fifo_size_exceeded; } MpegMuxContext; extern const AVOutputFormat ff_mpeg1vcd_muxer; @@ -1153,7 +1154,7 @@ static int mpeg_mux_write_packet(AVFormatContext *ctx, AVPacket *pkt) StreamInfo *stream = st->priv_data; int64_t pts, dts; PacketDesc *pkt_desc; - int preload; + int preload, ret; const int is_iframe = st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO && (pkt->flags & AV_PKT_FLAG_KEY); @@ -1220,10 +1221,17 @@ static int mpeg_mux_write_packet(AVFormatContext *ctx, AVPacket *pkt) } } - av_fifo_write(stream->fifo, buf, size); + ret = av_fifo_write(stream->fifo, buf, size); + if (ret == AVERROR(ENOSPC)) { + s->fifo_size_exceeded = 1; + av_log(s, AV_LOG_FATAL, "Input stream buffer overrun. " + "To avoid, increase fifo_size_limit option.\n"); + } + if (ret < 0) + return ret; for (;;) { - int ret = output_packet(ctx, 0); + ret = output_packet(ctx, 0); if (ret <= 0) return ret; } @@ -1231,9 +1239,13 @@ static int mpeg_mux_write_packet(AVFormatContext *ctx, AVPacket *pkt) static int mpeg_mux_end(AVFormatContext *ctx) { + MpegMuxContext *s = ctx->priv_data; StreamInfo *stream; int i; + if (s->fifo_size_exceeded) + return 0; + for (;;) { int ret = output_packet(ctx, 1); if (ret < 0)