From patchwork Fri Apr 1 09:23:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 35125 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c05:b0:7a:e998:b410 with SMTP id bw5csp532266pzb; Fri, 1 Apr 2022 02:23:30 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzWEVjx8u2VbNRByZJoNFZtnDiEK/SUJIT+ATKwrtdlIrM2H6p5BDCldZyCwtNBM7dIyxmY X-Received: by 2002:a17:907:2053:b0:6e0:2ad8:e10e with SMTP id pg19-20020a170907205300b006e02ad8e10emr8299777ejb.167.1648805010719; Fri, 01 Apr 2022 02:23:30 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h17-20020a1709063c1100b006e4de0d1509si267508ejg.303.2022.04.01.02.23.29; Fri, 01 Apr 2022 02:23:30 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=nqRdFms2; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4E3B468B20C; Fri, 1 Apr 2022 12:23:26 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-oln040092067081.outbound.protection.outlook.com [40.92.67.81]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 87A6168A905 for ; Fri, 1 Apr 2022 12:23:19 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WinR8tCIHG+mXhLj3Yb+BifzrH1+XC+dk97phY/gPcm4a79lsjfUgsfVqpLBEVJ0+Mllfd7f7O8OFlF1Tzk+R/g3r7QcAhryQuI8Of2rw+4aGfsKpd19yXf02S1/zRrsGA5AchwQjIpxVJk9/oD77x/nu27Ueh6S4OIt3jJi9m5NrEZ5vlYdWK3ddV5WFB/iU9So11JmYo+l4cFroYUS4RGsjxY4B+DkvonX1G/GgfeD6ItTn0BOZX1I5gSPfoPvQN+nnaIrJBIQkWSRSvdHH2CWWIl/EbcuAZhm25gEJ+7WeBgLOBxeVREpF3lWhS6SaKcpe6z9UNTI9fBnnp/JZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9iSPVnu/QqBMedZi8ufL8aJdjZRNf5oDZLc2qF2jTRg=; b=fmcRUfdBPQAzqqktXFT7WPdQa+jKUZnh1QRCUL6Sl65lh2w8oMOlHASAbK28mmPIkQiV/y/f+ZrhBhvCaG4VRaf5KhvL7hD7hlt4PpP95gBXZKMBP6ZaDA1JeZYbeBWnY4MZJgaOeeyEwBFxklXyAbcNyhC0T3zd6lfEqWi7a1XTkNgwtRvOS7r/nXQJ6lBRl6Ujw5n6CRaNVdJzxngVOfvqCip1W6qeQvU+MrxrI7MrAiuf+EDCINiU4VOnoHQPafb2AbyCEERS6nH49gyxic5Pb/zlSOLUTm5i1FNkYE5m4215M9naCwC61S9bu5TdzALOiz9hGt8b8FxnUcvzMg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9iSPVnu/QqBMedZi8ufL8aJdjZRNf5oDZLc2qF2jTRg=; b=nqRdFms2I0PtGHLaMGbZQGglNHUCQRBctII/jD97/Pe8Q+4vszanrSsSFsZB1Ul6N2DbvaBm4CR1s9UB1+uK4EsxkHeYsD+WRfDTkxL2SHKCQuyLUIfsveg3Pr6ujBlL5QR4kVTwFXg1UONYQCfMdAxsj8+09AXdz+XY0dX5zRrH5xGP2teqyTmUTyKOmKu6s/IdrdjlR5fTIapZBMFNUnaW2lg1HOdZ8OWfmuxMs7gOF4EZkOSrVmPX83DqJY0f8AtvBrf+61/Ls3tn+9316JBnPwTcCmCjkLA5KCM9/RtA56oP3AWxUpxoasBcZzqrytT0t+oNI/lAoFZ1eDIZVw== Received: from AS1PR01MB9564.eurprd01.prod.exchangelabs.com (2603:10a6:20b:4d1::16) by HE1PR01MB3690.eurprd01.prod.exchangelabs.com (2603:10a6:7:9f::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5123.25; Fri, 1 Apr 2022 09:23:16 +0000 Received: from AS1PR01MB9564.eurprd01.prod.exchangelabs.com ([fe80::9070:a5fd:e532:bdf8]) by AS1PR01MB9564.eurprd01.prod.exchangelabs.com ([fe80::9070:a5fd:e532:bdf8%3]) with mapi id 15.20.5102.022; Fri, 1 Apr 2022 09:23:16 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Apr 2022 11:23:08 +0200 Message-ID: X-Mailer: git-send-email 2.32.0 X-TMN: [7Gjl6dDa6C0dhmYTJj3m6+lkY+7A11jH] X-ClientProxiedBy: ZR0P278CA0065.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:21::16) To AS1PR01MB9564.eurprd01.prod.exchangelabs.com (2603:10a6:20b:4d1::16) X-Microsoft-Original-Message-ID: <20220401092308.763375-1-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3efb1b65-8635-44b2-3c0f-08da13c140ab X-MS-Exchange-SLBlob-MailProps: zswcL9HXbeUsOhYymtKbuEK5lbDcH3V/CuVlhZcGEGrtMr+Vbp6PtYs/HV82R4PvB64c0DXgLOqFOhKQlcjjW73FC1PC0/cg78fHIForZmuha1qYt9F6fHglBp5tKpUISWlFpM9aGWpfcmlRwNF915WAc73fuTSWRSlgy2Kqht9tPgFkEiWnM6AiTeAmcBSpNMfyqWW3TacUQSJBPuQUfLCwvcqpy9hpEw1uJc9JmNRY3OTsALRYir26CAyJtqXSjRG3f7ZNSGrSY1TMvtoGPZt8vbb3TxioXyD2g0ouHatcVBb9edvlAcM1i4TAKn0joRvK6lOij0aQNi99hlRZ98LS+30hviHLpB2p3Yu4oxtSlkox08wDHhPxXc7QnoWyAxk9Zss6FKb0ev92K1vsfQxJLrXr2RqovQa5JgtAabHUwUw6a00YegzwcIclgiFqNymITRrLio24YO+/E+5gRugo4inFxH2wtZ55oomUSF5w7l3MReOIiqXcyQZDvDEl7w42d26DI8q3ZPHp+2679qIMobrK7KZ2mqiFBBJOV2VxfFra9gYcyXBpJRICXPkJ3vg1We2td9irHso4jvkdlW8S716ZVs2F4VQM1GsrbsuF7229scBnZmmxQE6xLDA9skqktCJrXCBfMIbIIuzryH/fX1cVMyyyeyjG/epJ87zUC5pdjc/V/U7QN+mUq7LSUuGRJUjfoYAYLJLuqPyNZzRk1x24NbU7fz63LewZt+L4zMfl4kt4odrXNe9ihi0i X-MS-TrafficTypeDiagnostic: HE1PR01MB3690:EE_ X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZO7BeDiFzENXOgNo9UDTq2ktWzVOjf8UykNJE6spsr/+eCw2Nf2HRnivLjsFqvFzRmwzyrT/L38mI7xuT1C4hmpQ2HREFZcq0mmVdhr63/aw2iGdi8LKtZJBGGstQulvbZZJ3aSwyrF+mtBBlKueFcCX/m1SgHq3yW3QjYxWpKZgy92hCyT9qKihohspx2bbnPPlXIp56u66UNxgQlpEFkKnRruwMf6ydEwhuwn8u9/4Fis7du7kcW/JGs5QqfAoUT3dgyyNOBTyiL22w1y9Gnnr38pw6DGaDXtgQMUlIpG9cRb7+awCoF1Er6a1QbP4NM5Qk7NXKMPPlqQrkdqPamQPa0e9yb8v8HX7GePb0P3Y+f5vPvRNIXwv7fb9NwkdvSf6ebDMXWZFmZeGm4D6uHxj3uaoERe92RIGOtt1K4pqYi8UOo99xXQsI8c4mijToKQNDqkYzDGUvLzGINitmFLSOoadb/I4xK3uGbVOEJ3TSrOokzMH9D2hej+ebx7conYfOWRplybtC9gHjT4P/x1XTmz3nDa+UUVM7MbavfDJLN2IlBzzptVCr9wGps0V+kg6uReSo0k2Gauuyy2yjA== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 7iuEraEYYLqB5yqvY7DjmSADcq2eiGtMx317OcBv/ObUFDQq4Rii6qrwDheE9Vf6vWQ8aopvp/4DtTS1uVYzIZoG27IseXlto4vgD6LaiRiIzrLqYvo+a0SW91bVJ+rS0+QTWh46Q4JgFl90xg3UknTTJnqPDhU5CcQM1zgQijpqnCH8brKTFS2RbxVLjITBWQtokqe+56YhftqG9Pbxu92RWKsNlcd1NC2It3ilKmMW3SSen5vYxaeC4wxAiwoA0aHJGO4liNvx5RIcwfPXGygmxwo0p25AkUTtugvtBtJ0dz9R5hMWRCeyomV83YQHfX/HvW/1OvcKF9BzRRUqKMHC3nTrMlkxEXc8SckqWgTYx9soKIcQXQMCfjS8gvxG1wuXZiMVuf4OO/lYPv58HCEPi7dsgUIuF5JAeKWYYSbXyhYr3G1JPJTycoY/8tXmFPdxpxBMbQf2F3BbBBggQMIPJ0HIsjGjJ1HNR9DTtjS5EdPnWXjS++r/bzekYMbBadpVt0NnyrmOrh67KrybFNqcpLE2SDEFG4VC2bwQCExvfmAKWHzFt8oSpglEdGxwsVkOJvRRoIHm2K/vn6YZth9BO7+soVMpxs2Qvw4vcPKNbRBGjIq1Ua172/LeZiJT/dJ6lTK5xfJ/dftfk6hodkjF40v8gbl60N5CFRsrldE85x++wFfkpqGTjYHkgXE3RTq4fNJZ2SiWfAUpVXT2EPHnmI2rPPfmrF3H0so6XIYHILtDpGnxLgNuxYyccSBn1k5pNCMIZ2zfimsUsKO9BHv9bfahC7BiHf73ZdL3x0Hw6lVdL2Dvep3nWAcZs1FAQOUg31vKBvGDy2S2U8fi2h15l99FPtM3pGvAZhA8mXUQXRTRepxWRGe5q1i9j7z1KrfKPf4rSZw1Ka/miHKrvc0x12DJkHPjq+h+ZAylwQMnmSqPbeFL/l8yQJKnLNDXGI7mraSIWt314+3am+pNpbawQLn/azlkRP3OI7cYd3uYsU5W2pnezobWZDMNsFeRVrGUtvAZpqTeoh2KJ8zCiP3mkouV3uI0KA5wxaRhLpYQg9rvTmTJHQU4PKIVXbhUzWAFD0BCOwS10HPwzz/OOElvPnPHCEZ506rEVUFXC9ZO722RrfhOS7/KKIEvPkmc5uFp+zEZxjVmAsyMCCfLosHvJn+Xe8uNDa0ADlI3u/h/1cYbmNVYgOaQuLn4FpSkp4LsE3ikZlSxhb+vIBMdLsnkZrOS1cL9iOv/ANBrcUcUDavwE1ZdPQZGzV+xnHLAbzXw+JS6yU11vuPYRMPUKjQMGpOW+sKMhu5+5TSiKTTzGeON5QRM/JyyF5UIBL9vC/GucicSeka57IYCXK7bgcRwdlmnr1AFDhpJDlFoRE/lv/X4dijOJEQefc/hPmcg+UCzfzq2PU0K9B5rPQWqSY6+GpFEPkvHP/R1pUOHC1XiWyp/E5WJVkcNBShpgOHb X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3efb1b65-8635-44b2-3c0f-08da13c140ab X-MS-Exchange-CrossTenant-AuthSource: AS1PR01MB9564.eurprd01.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Apr 2022 09:23:16.4363 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR01MB3690 Subject: [FFmpeg-devel] [PATCH] avformat/mpegenc: Fix ever growing FIFO and infinite loop on error X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nicolas Gaullier , Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 7tlJIKJ7btFW Since the switch to the new FIFO API in commit ea511196a6c85eb433e10cdbecb0b2c722faf20d, the FIFO is always grown by the amount of data intended to be written into it even in case the FIFO has enough free space. Fix this by only growing the FIFO if needed and then only by the amount that is actually needed. The allocation errors that resulted from this uncovered another bug: The context is left in an inconsistent state in case the FIFO can't be grown, because the FIFO does not contain as much data as the sizes contained in the PacketDesc list claim. This led to an infinite loop in output_packet() (called from mpeg_mux_end()). Fix this by growing the FIFO before adding a new PacketDesc element, thereby preventing the context from becoming inconsistent. Reported-by: Nicolas Gaullier Signed-off-by: Andreas Rheinhardt --- libavformat/mpegenc.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/libavformat/mpegenc.c b/libavformat/mpegenc.c index cc47a43288..62692bfcd1 100644 --- a/libavformat/mpegenc.c +++ b/libavformat/mpegenc.c @@ -1152,6 +1152,7 @@ static int mpeg_mux_write_packet(AVFormatContext *ctx, AVPacket *pkt) int64_t pts, dts; PacketDesc *pkt_desc; int preload, ret; + size_t can_write; const int is_iframe = st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO && (pkt->flags & AV_PKT_FLAG_KEY); @@ -1192,6 +1193,14 @@ static int mpeg_mux_write_packet(AVFormatContext *ctx, AVPacket *pkt) size -= 3; } + /* Enlarge the FIFO before adding a new PacketDesc + * in order to avoid inconsistencies on failure. */ + can_write = av_fifo_can_write(stream->fifo); + if (can_write < size) { + ret = av_fifo_grow2(stream->fifo, size - can_write); + if (ret < 0) + return ret; + } pkt_desc = av_mallocz(sizeof(PacketDesc)); if (!pkt_desc) return AVERROR(ENOMEM); @@ -1207,10 +1216,6 @@ static int mpeg_mux_write_packet(AVFormatContext *ctx, AVPacket *pkt) pkt_desc->unwritten_size = pkt_desc->size = size; - ret = av_fifo_grow2(stream->fifo, size); - if (ret < 0) - return ret; - if (s->is_dvd) { // min VOBU length 0.4 seconds (mpucoder) if (is_iframe &&