From patchwork Thu Sep 9 16:07:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 30087 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp658122iov; Thu, 9 Sep 2021 09:08:08 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxNltB+pkufCVEb1LH6u7lqGTI9aFYZdZX/7tA0GRdrLFMRz1P/3fj66P9FtFB2vN4Pi8C+ X-Received: by 2002:aa7:dc19:: with SMTP id b25mr4093016edu.23.1631203688374; Thu, 09 Sep 2021 09:08:08 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i2si2826383ejp.581.2021.09.09.09.08.07; Thu, 09 Sep 2021 09:08:08 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=rIyKU6mv; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B3CD868A9BE; Thu, 9 Sep 2021 19:08:03 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05olkn2046.outbound.protection.outlook.com [40.92.89.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4A84A68A7FD for ; Thu, 9 Sep 2021 19:07:57 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XfZyChm1G07JXgIohq03+xSpB1MZ6VbqU87gktxxXrK7/FuE2zsp6Zg6lLY5P/KinRdrFrLblt256WxIk8Jo6k0UX0FWNkjcbDXOsld+XlXHCer5s5Nn4YHvOwZBDigCfwbVI1NkH1NC17yu5xtUYGQHyG5jBV6/tjfgmZcIFhJLP6XUfC6HSL1GKc0NsBbXVXdKXlQnx7PYPEiAdkdqGiBJpqaYo7HkNRfMpDRdJu2TTsh0+rLyH7dRyP2hPYOawbYkDMbfW7ByAFAsxQWbeLgOsvdFnaLGjzKb+RNdQPVzPF18juZm8jh09VbY8ybxW/jqbF6azmM0pHmR/MNrIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=4sr4BI0XJzWkfKoMc80aK/kFRrjk5ufp6ekTlPhbMIc=; b=J3bYJyC7qFGtiT8+KWbY12vyolDjSuZOOxLaA7HHhHOSAos7RbFXkd5xs1Mbai7KsGZ2wulUoAkH64FX4m6S09fY3+i65x85aGhRyxPo5wmXJHIp9s5hgOn7zlNveGZUF5LzBfCVU8Ce3rJggzJRHuWNsRBBuwWxPeY2XFSDUz0vOXlzB+JV3yZgd4qtanNMaHDFR4GRw9NiyFtEgxKD9+H/XbvNd9LO3PIeGgbT07mrs1sxQggZv818bAVe9B8QGTI511VWJir5MMZf7IGFzfjMj0P0nc7mBcSsxcOPkKds6vdmnBRhf1s9KRltRXXM1XY2cWMXzcjzxUQ0wU6LOQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4sr4BI0XJzWkfKoMc80aK/kFRrjk5ufp6ekTlPhbMIc=; b=rIyKU6mv3pJK2japVVgMugst4l2WBnr+2vBh3+aYgO/eJbnMSdBpWpKGnD1PbsiAed8giIczGlV8R0cgJ2USPGS8rOe/HeU9GNQ3atQJdeVw85hz40fjytR+DZCmiwLBeo+8t2kFn1MOIyzTksV4Vhk+hTWhrIohPY6ZTjAWSfHGwv2z9Nqvvy6Y6+UUrgnqnP0m/qKBsygskvbKsO9mA0x/m6Bbp7/EJwWTpjR08R7qfG4y+Ku+sMvwqzvNXIOnnT6f4GmLsC8hIhhHBD1N8oZdHU9QlmvNLRzgUxpqr5ZmFtACgICY5EBRskxXPBzg/E7xIu6WSHC7l1fGDG90mA== Received: from PR3PR03MB6665.eurprd03.prod.outlook.com (2603:10a6:102:7d::6) by PR3PR03MB6556.eurprd03.prod.outlook.com (2603:10a6:102:70::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Thu, 9 Sep 2021 16:07:55 +0000 Received: from PR3PR03MB6665.eurprd03.prod.outlook.com ([fe80::2924:e83a:49ed:92f0]) by PR3PR03MB6665.eurprd03.prod.outlook.com ([fe80::2924:e83a:49ed:92f0%5]) with mapi id 15.20.4478.025; Thu, 9 Sep 2021 16:07:55 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Thu, 9 Sep 2021 18:07:46 +0200 Message-ID: X-Mailer: git-send-email 2.30.2 In-Reply-To: References: X-TMN: [czGm2eLQHPSwWPoWot/gV3TAOojtcwLC] X-ClientProxiedBy: PR0P264CA0283.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1::31) To PR3PR03MB6665.eurprd03.prod.outlook.com (2603:10a6:102:7d::6) X-Microsoft-Original-Message-ID: <20210909160747.1841811-1-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from sblaptop.fritz.box (188.192.142.38) by PR0P264CA0283.FRAP264.PROD.OUTLOOK.COM (2603:10a6:100:1::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Thu, 9 Sep 2021 16:07:55 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: eb0c0a0a-a24a-4600-631d-08d973abfbeb X-MS-Exchange-SLBlob-MailProps: f36zkjAOy3XPOJFAXlWDV2wrQ9tjfmBZ7QIQozD/Xr9xtDymICrE2/a+CETH6h0xJGix1e4NvRVjfDIf2rhnUvLB0M+oJYQPDFXqHljRqSRtSO4W4GNurIMCKgb1R3gwH89ad4MvYV3kcx/4Bd6gm+u27FpdNeVgFNeERt2Q4RtV5CVU2e+OW10uEtruQ2XQE4NPMaqyfo/l0liS1dpqkCqZKWN8Zd2WvAXZys/cxZkyTe00WxKaUImmR9xoZiHkeMvsk4sWGk1+d5pNQUbmauSf9JXrriTIRGNv2kh5v2GxqU0V/bUH9rPO5OxXAin84hENpcxLREceVVVjW/DmZIOemUOMAdLeDqPWm37ff2Izry+P9220SJ6752jXldvatfpPwMyxsCBQ0BSddUio+9E85FmtWSjcAKjoIaDz8Q9H6y6tkcUvr+jLkuYO0HTHPzPKAnoc9bp5pAna4+g36YLfJxUyO1TGCL0KKbbx4pP4tE4omrnUyLxk0+cUj/eAtgJWehb04ZykMfOAcrNwlmGJboq+9sSDwwgPtpfgO4hS3zGcv6nNaqqU4trE1DpakEqMfVD88duB66Vp49gyLwX63EFbAsp3RkWTiW6G5WKwHaidhuuVywaX1WVszBpJoFlhACBMCVAwuwxopp9D13S4XWhUz8j4qq5eiOM9ge65eEDhwLxXbnMXHJSWopN6u0EkLH3MR+pJ+6UXiBlybKeA1/m4TwZ5IsWaHlQvJoI= X-MS-TrafficTypeDiagnostic: PR3PR03MB6556: X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KwwqnjHdI2eACy+lv4+Plazep78V94OIyBppjb6Pw5YjhqTQT9lF5oyTxXaDIPS7CnJMqpkHeQu15OyVA11QwNEo66wKCurpjqbYdx3lT34tZnSheSX1I4VLDCBgTuriS+VwRxRrsHBhTqoya5fjONHQQonES0VCMtvIcc/O9dbZu2QTmNiZvPjaBYwf0SJ08rniL1NYLYZeodwcvMYmxvxDby83TKTwR5hwgNQDQLH1I0W66sgItvSWLtBc8i1CfOMGKtUQmJ+4YKB/XbIlQtYgbpgFmPDNyY0I11LCvkRmeJfe+L3SWal2+2Jpi91Gf296UHjvxYZgDdHJgc/UO6rPJ4tKCgGrlU+AlV4AiUNOeZ2+X71tPheTjE4UQxal+v1U2Ant1NuZEBsGeBbvjt9SoZHDATeSPEMEZjlPeJzD1rpFNEyc6j0k1zB1mCn8 X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 8zi93VImolcHbeoISkoNK/FleYOxqPEYswZt4a4jcUN2TGPL3HsHcYp+UImo8TnVWmmPAy+0cyskhO5r6TytRjmpah53eEp5TzVmXAKc+0rvbr4bStWSk1wLxsYknj6LItMBSYuAm8XnQxDARlCQRg== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: eb0c0a0a-a24a-4600-631d-08d973abfbeb X-MS-Exchange-CrossTenant-AuthSource: PR3PR03MB6665.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 16:07:55.5562 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR03MB6556 Subject: [FFmpeg-devel] [PATCH 15/16] avcodec/qsvenc: Fix memleaks upon allocation errors X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: dvnlJhZnEIx3 Fixes leaks in case the allocation of the H.264-specific stuff fails. Fixes Coverity issues #1442911 and #1442914. Signed-off-by: Andreas Rheinhardt --- Sending this as part of the other patchset so that fate doesn't fail. It is of course completely separate. libavcodec/qsvenc.c | 49 +++++++++++++++++---------------------------- 1 file changed, 18 insertions(+), 31 deletions(-) diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 090fdbe75e..bd4e8fd8cd 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -1417,10 +1417,10 @@ static int encode_frame(AVCodecContext *avctx, QSVEncContext *q, const AVFrame *frame) { AVPacket new_pkt = { 0 }; - mfxBitstream *bs; + mfxBitstream *bs = NULL; #if QSV_VERSION_ATLEAST(1, 26) - mfxExtAVCEncodedFrameInfo *enc_info; - mfxExtBuffer **enc_buf; + mfxExtAVCEncodedFrameInfo *enc_info = NULL; + mfxExtBuffer **enc_buf = NULL; #endif mfxFrameSurface1 *surf = NULL; @@ -1454,10 +1454,8 @@ static int encode_frame(AVCodecContext *avctx, QSVEncContext *q, } bs = av_mallocz(sizeof(*bs)); - if (!bs) { - av_packet_unref(&new_pkt); - return AVERROR(ENOMEM); - } + if (!bs) + goto nomem; bs->Data = new_pkt.data; bs->MaxLength = new_pkt.size; @@ -1465,14 +1463,14 @@ static int encode_frame(AVCodecContext *avctx, QSVEncContext *q, if (avctx->codec_id == AV_CODEC_ID_H264) { enc_info = av_mallocz(sizeof(*enc_info)); if (!enc_info) - return AVERROR(ENOMEM); + goto nomem; enc_info->Header.BufferId = MFX_EXTBUFF_ENCODED_FRAME_INFO; enc_info->Header.BufferSz = sizeof (*enc_info); bs->NumExtParam = 1; enc_buf = av_mallocz(sizeof(mfxExtBuffer *)); if (!enc_buf) - return AVERROR(ENOMEM); + goto nomem; enc_buf[0] = (mfxExtBuffer *)enc_info; bs->ExtParam = enc_buf; @@ -1484,17 +1482,8 @@ static int encode_frame(AVCodecContext *avctx, QSVEncContext *q, } sync = av_mallocz(sizeof(*sync)); - if (!sync) { - av_freep(&bs); - #if QSV_VERSION_ATLEAST(1, 26) - if (avctx->codec_id == AV_CODEC_ID_H264) { - av_freep(&enc_info); - av_freep(&enc_buf); - } - #endif - av_packet_unref(&new_pkt); - return AVERROR(ENOMEM); - } + if (!sync) + goto nomem; do { ret = MFXVideoENCODE_EncodeFrameAsync(q->session, enc_ctrl, surf, bs, sync); @@ -1506,27 +1495,22 @@ static int encode_frame(AVCodecContext *avctx, QSVEncContext *q, ff_qsv_print_warning(avctx, ret, "Warning during encoding"); if (ret < 0) { - av_packet_unref(&new_pkt); - av_freep(&bs); -#if QSV_VERSION_ATLEAST(1, 26) - if (avctx->codec_id == AV_CODEC_ID_H264) { - av_freep(&enc_info); - av_freep(&enc_buf); - } -#endif - av_freep(&sync); - return (ret == MFX_ERR_MORE_DATA) ? + ret = (ret == MFX_ERR_MORE_DATA) ? 0 : ff_qsv_print_error(avctx, ret, "Error during encoding"); + goto free; } if (ret == MFX_WRN_INCOMPATIBLE_VIDEO_PARAM && frame->interlaced_frame) print_interlace_msg(avctx, q); + ret = 0; + if (*sync) { av_fifo_generic_write(q->async_fifo, &new_pkt, sizeof(new_pkt), NULL); av_fifo_generic_write(q->async_fifo, &sync, sizeof(sync), NULL); av_fifo_generic_write(q->async_fifo, &bs, sizeof(bs), NULL); } else { +free: av_freep(&sync); av_packet_unref(&new_pkt); av_freep(&bs); @@ -1538,7 +1522,10 @@ static int encode_frame(AVCodecContext *avctx, QSVEncContext *q, #endif } - return 0; + return ret; +nomem: + ret = AVERROR(ENOMEM); + goto free; } int ff_qsv_encode(AVCodecContext *avctx, QSVEncContext *q,