From patchwork Sat Aug 6 17:28:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 37162 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:92a9:b0:8d:7f3b:94fc with SMTP id q41csp896031pzg; Sat, 6 Aug 2022 10:28:32 -0700 (PDT) X-Google-Smtp-Source: AA6agR5EHKm45ggAmynJ4wSuMWmd2Gb8xdffGFQeL0CzqNBJJ/X/AolrdT+vqzGjZCEnb8XZRlHR X-Received: by 2002:a05:6402:400e:b0:440:253e:961a with SMTP id d14-20020a056402400e00b00440253e961amr6749876eda.285.1659806911865; Sat, 06 Aug 2022 10:28:31 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id u6-20020aa7d0c6000000b0043a6ddb2941si2093841edo.253.2022.08.06.10.28.31; Sat, 06 Aug 2022 10:28:31 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=oOaYEBQ1; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 65DE768B658; Sat, 6 Aug 2022 20:28:27 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05olkn2069.outbound.protection.outlook.com [40.92.90.69]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7311F68AD28 for ; Sat, 6 Aug 2022 20:28:20 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SXt02RFKfriRf8HB91HbACxTExOuLiHglliymoS55i0+4zMUY45kFR+HvCqm1DZAhrjYYsIwwObzE3SzAKLINyvVxKecrA90yxYh7nCfyxFcd2zboJ9foNzKJIg0mRHKKcRJz+KwnHxwXIRqaSqX25HGWtQlkY427a3b1H2vCaKnYw0W8I4Xl8yqfnd+M43e/FQW2t6WQftP05LTjDIPcecNLPiiEbEsUwBc6qnDdTjMgHdTHYKjTNkp1IU0qGj5sJvr/0Km8i6itg5g6PjEPMavuh1QYyOyrW5fpgrsGrdGBnWAcsdashaSHuIvH+J3NEWxdpMV7/6BIthAZHJOIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5oEGnzPgWoNkInHiBMndreY7/h7EoaApqcCAeNEa3os=; b=jF/sXCoUpzHCD4GqYiA8+yoKBVJ+XdnLiDYap9Fj8Ntm9gILn5HMIQVac+cu+wk1FY6QFTIfWx0jqUyniZAuL22WdTUzDvj3R1XG9VKIok7zU+FFEvT2GX6fJNKquy24YohlnqmrEYWbXDRI6w9SX0aIrUD/MMilTR3/Y4qPwk8RPRjITQ5QU4OADbzrAhNPRrGTjrTG6E6Ui1Kn70SaXzIITrrqb8nFDPwKeIqpUldJVTERrBbHsZkQ5Ijd4U1zme7NRx4pHrPv7dothavCGy3wgGlHcNVOw6ZreHZldSlJQf47VlO6lrXfW/MJoG5bYd5nhiXgZQx3oz0roWxm7A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5oEGnzPgWoNkInHiBMndreY7/h7EoaApqcCAeNEa3os=; b=oOaYEBQ1IQkXbKUEQj/XHyOMPuLImVbQ1n0YSwS5SkcKXJ1vIxyFjShfImTICeuhW7rbcFSlbhtg44lwCLF2LkBZNly3/YbuQU6uIjs+DOg2yXf9EbXSnB3FU4vuCQFw602q+PzY+zxKKY9rpL6TIdr+gG4r0rI/DuVOsYF/OuSDlLsO6wv9shh4rNVWcNjT91VDX7IHTnuanFjT8ZKDKfIx0ZtkqAIE+VbMy19hyPBA6jiTaE0z9XlO/IE92SKl3PZed/ZJO52mK926T6g4WAL4CqsU6Cw1fu9I/mxecrfeiZzFoC2VDwBpS7+qsi56I1Mlo/Gaa8cgVH6d/9iang== Received: from DB6PR0101MB2214.eurprd01.prod.exchangelabs.com (2603:10a6:4:42::27) by VI1PR01MB4400.eurprd01.prod.exchangelabs.com (2603:10a6:803:6e::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.16; Sat, 6 Aug 2022 17:28:18 +0000 Received: from DB6PR0101MB2214.eurprd01.prod.exchangelabs.com ([fe80::210e:b627:bcc9:8c46]) by DB6PR0101MB2214.eurprd01.prod.exchangelabs.com ([fe80::210e:b627:bcc9:8c46%11]) with mapi id 15.20.5504.016; Sat, 6 Aug 2022 17:28:18 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Sat, 6 Aug 2022 19:28:08 +0200 Message-ID: X-Mailer: git-send-email 2.34.1 X-TMN: [WA01cySsEGZLxB2bpGaYKFQsEFxvpKl6] X-ClientProxiedBy: AS9PR05CA0009.eurprd05.prod.outlook.com (2603:10a6:20b:488::10) To DB6PR0101MB2214.eurprd01.prod.exchangelabs.com (2603:10a6:4:42::27) X-Microsoft-Original-Message-ID: <20220806172810.294485-1-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c8c3726d-1781-4652-1a46-08da77d10d36 X-MS-Exchange-SLBlob-MailProps: q+fD6XS3/ULkXTfjO87bVJ5/VVNh3R154/YYEY2hcVZ27eeUCXex3FJiK2HECbGhsJufi03wVzzr2v5arheBobxmuCzjuABlUNROMT+XoSAoE1/txWMXk3MlC8mn5p17Rn/1ODkFuCN1pnzHV9USDGYYAAFHEYl6d0b4evYfTYZOFAh7wuqM+wR69ZD2F1l1fk1RhTkg4v7dY6Hqi2+5QmHQBtIstYubc2OtyaIQF3Lg863f1wnone3u4g86USB48mzoUngciZyGQhIxMpqy8oS10BGkjcbnZkXlwjUPkRn7iV0r74FdGSXMZss12Cmg92ROkeQMvxSnfBZ17p2ON609hgLiyIgl0ofMmUvRg8skhBcPRDtmH5BhQcXMjXrZaWplxAO/HMca5YZeYHpXtv2QROw+Cp9CVX1C4Don7TsWhMu1NlPCKZeA+uMDnS9HFOsAGUNQMqoDp5/XE/wumY8VNlOFgbE0gYOSka5VF+R7l+GEQrE7XI75cFAis/CZTwBBeSs/ntX298AGLVUxU44qOq0iaWHcP/hcWGVAW0vgsXZQjCt2kZNyaKzcGH4rrh3/Fa32+5bitVeohvV71COfj8oYoFGZyb/Wo1LnpVylocs9UxztiZVqa65C+mBsWfcXpHK1RHE4diEgWXo+RHeZ1dPGdCFvHd//w5ZgH7ir7RrBnmoGgix7dH8MiOde3cdIyMkYU3YVWNANjZ44zw== X-MS-TrafficTypeDiagnostic: VI1PR01MB4400:EE_ X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: rkzlx6xn8gzKw8HMT4JsRTAoCZ/48ZE9GcLKrBleThRWzRCdMw9bX6O6Ol+3R9S2JF1kP42tUb39+WwJQNsV6I8FdYtg3wfot+deVSVa1tiLveERIOCOCriNGVvK3GcZbDdZSfxzdNc72vpFhMHM2xsyux6dXbKb8sfKkOsf+KvtwQx12xTZPtLKTVxNsuW9+ud7OJd0MvbzWuFEjEmYHeQUk1FeYeIZV4JuqNScswuwM1OIx75phqMkP0uqJaQlT5/DpHHWkAiEZ14CLKNP2Dn7aFZs+y8bAiovNuuF/nsK6jTPA6Unt5eXhDgDGDE3+HRJpE4JS/r32YDrR6Rrsvo8urcQP61Zy2GrA2B2F7wA3T4Kduvd5LXxjY2dC291eLEwcorLCtVwr2YFDz478aCXN0Vlenhj3t5Zx6oKzJlaz6cmd6KhjCmDKne+bdy5sqRFyJNkbX4gc8KpSisuykP2qDCtjbjOJPTTUCuyqwoExVMrLZcqDAXbGQZVHcSyo4hGCVcbv4DZ4oOy+v2oeKWXwJQLtf5hP27fX1sKJs4ARZKSuTfm1e9E+O/IpS/+bW+wnYWjENTyykFA9FsKPiNSn3MS03nNKDyhTrmjlbcoWtcNDcoOVGmNV8Y40tZdgQUxxGl3mPV349e92UbMeg== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Ps/2r1ibfEuGqbD1PBKNEx/QQ/TRqkv7ChFHYgXgOqtRlLtGmxcTVxxehrMhI6dWPdFJeGDufSLZtEyKjv3rZVTfrj1VPrtZGfkRnrfYqLlkbD1PmR2SnH0sfq4gzZhiA5wFBWrlL3wsyPVM+OquzUDWHQy9+PfPVlhIYoP3DG1JQyVo4EB/bjiSnsxn9FB9lvKAAO6tW3HMqoSsg3fwWYHWSvth9zUXzO45+LS/AOEBbYZofJlTQMNflDmkfjVTVU8qoXaQhQuzwz5QLSgOQ8KK4SoSPocXUcNZwx8WOyRduoeTkNQ021onDr6T2qhWSn2S65ldUEYYS3FbJA9FaX2bx4qLSLAZ2/Qr/566qRGFWl30ZgOjsLQqyZUYTGypBXYbXl2EHPUo4OP2hh0Wjdbsv7g8xW89IgdZR+75poT8k5e3wc33R4owW5cCmyRlEyNzAdRV77D2XCsMqjv+Ku3H5u1IUWd5mdeOqHSlycIOUwyOi6mdAsxkIpkpURpybA/SVpdU37MLGehvq7P/0hukidP0tGEJzoHQJVCloGRVlo2D4dHmR1FoWx8KBEiaSqxOPAJq4ak2foxCJzZ4zbsOvdKkb1jnBVaJpXX0eDAOOyLxQV2HgJjVwpDpy7aqS97fANJxdRHFsK3wI0dFEga9xTrp/GBS1YqiKcyT0Tgs8BtL3rwzrepIU2iTnWG8cyGnmleFwJhysbGks/SSlJel1p95YRpBL/xEJZwd9KxU2aw73iwsVuhd9qEdh9VqeuW1z7lJSZG7U70/oigmJtPZbsSk0abaDK3G86u8xvQp8UoGYcVV+KaBMCatbVMn0yjBQKEE1HOjxcYjbE4s12Rn0Qb1JikWNeq6yEie6LQEgvji/Eq31MEnu3UcOGhGVhCwf5FrRl9gV0vhKXkLjWmWZL5chwVCoBM6ri6SQbw1yZxi0CPDzXjGlGdLArEqkHe70QM1CB9LChQTnrTnytDm2TPec3pdZ4FI3e3yJoQtJ4ysiU2TAokI7tKOELVAVyVjDeCc4xdu/Z+NFUpFAGBXZFmtLtFeavgQRyFx4Aq6yd6KQQ43EK8ZVKO9s385TmATzmADp9BaaQIW2CfJs0MLLU6vjPNqdVAnIXhgqoZuApDDAaBm+QdeTXFjh/mqr02Bxc0/lS2Z3C672BS0IQaxBcPZRXq8z0M0/KrHE76w6G2bhLa692IMRCf0AkvN6WSNNBsgWFKyPe/gFAgsqGQdlqq3cCkkut9O1RRmTMmNz2mKvRSbS908/sxgbANBeO/077JQmXK9eroW0Orf5rg0QgaD6+MG9hhEHcf9CRY= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: c8c3726d-1781-4652-1a46-08da77d10d36 X-MS-Exchange-CrossTenant-AuthSource: DB6PR0101MB2214.eurprd01.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Aug 2022 17:28:18.4553 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR01MB4400 Subject: [FFmpeg-devel] [PATCH 1/3] avcodec/nvdec: Check av_buffer_ref() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: H6TWi543H/SY It (unfortunately) involves an allocation and can therefore fail. Signed-off-by: Andreas Rheinhardt --- I don't have any nvidia hardware, so all these patches are untested (apart from ensuring that they compile without creating new warnings). libavcodec/nvdec.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/libavcodec/nvdec.c b/libavcodec/nvdec.c index edff46d310..15665b83bb 100644 --- a/libavcodec/nvdec.c +++ b/libavcodec/nvdec.c @@ -532,8 +532,11 @@ static int nvdec_retrieve_data(void *logctx, AVFrame *frame) } unmap_data->idx = cf->idx; - unmap_data->idx_ref = av_buffer_ref(cf->idx_ref); - unmap_data->decoder_ref = av_buffer_ref(cf->decoder_ref); + if (!(unmap_data->idx_ref = av_buffer_ref(cf->idx_ref)) || + !(unmap_data->decoder_ref = av_buffer_ref(cf->decoder_ref))) { + ret = AVERROR(ENOMEM); + goto copy_fail; + } av_pix_fmt_get_chroma_sub_sample(hwctx->sw_format, &shift_h, &shift_v); for (i = 0; frame->linesize[i]; i++) {