From patchwork Sun Apr 24 04:42:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 35420 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b9e:b0:7d:cfb5:dc7c with SMTP id b30csp1237747pzh; Sat, 23 Apr 2022 21:43:35 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxbBgQG+pz1vdFlwUK5qkk01+VIkDBhpEltLrqlPTSmHIHfMs//Vl9ybjiHSebLMWzkl8ZG X-Received: by 2002:a05:6402:5189:b0:423:f342:e0ce with SMTP id q9-20020a056402518900b00423f342e0cemr12828597edd.120.1650775415541; Sat, 23 Apr 2022 21:43:35 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id dm19-20020a170907949300b006dfb1342cedsi12577560ejc.618.2022.04.23.21.43.35; Sat, 23 Apr 2022 21:43:35 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=OBSw+juJ; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 617FA68B42A; Sun, 24 Apr 2022 07:43:27 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03olkn2093.outbound.protection.outlook.com [40.92.58.93]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5AA4B68B26C for ; Sun, 24 Apr 2022 07:43:20 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TdtIv1i5uhXNVDCO3WI+1GNJ87UoscVVDsbLAccgqjYkl9wVosMWyDJUdXFcOtQuN0Vmne8neXDdCrXIxdf+KjRXHfzaEBee/C8dxOEhsd4rRtIHQ2wCQGr/b/nFL4GPYlrAu1WojhhLIKWfDY1nkJwLcklrdtp053zsiAjGsOKLYtnOK/g8V0V04TH5HQrAjVnOoCrc0rGS3LSdAVXwM367sXh4BVXkKdpsjznz1CxJUYf+Ec/CmXodMaO4gL4YF88Fx7fTkcM2SfwoT0wNDLxHKNXo0gq3xp5wcfSiRR7HaZfX7JWaYcP4mHdA/6xQ6J+0Z0Dt1a/VDV9nmgU/6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ty8BjIjGJ3Ralx/Jok+44cpPSFAL/4FRt+ru6GrTRBs=; b=V/TKYSVMpPpMAxRyxS1MikI1Zc+5HnnRi62y9mxFnuwNR9LSYou+XnbFqjVZzr73f+4b5IVNqm65cNyCP5EkUuF4ZJ40OiZtkdIpz9TbPZWk+KedzRIGzFF2kWxWHVqI8FsytPPdP3WtQs1VT9bu+65F2M5JdU96SsUYec3ADtcSTHB13XFQ+zkBLfaykxLB8lRE3cwpjJWPVU17oL+eMGyKE0OUKzqVpPp/f0MqQkcSTIFXmK4N2M6tRwhrFqTCua3DG5VaHHl/Qqp/oGWCa9ln/nml6Br7pf+7EOOOybAIwkedT1X8CbfaQlA7ef82oGEc4I/LqEwOTuCCZAEq1A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ty8BjIjGJ3Ralx/Jok+44cpPSFAL/4FRt+ru6GrTRBs=; b=OBSw+juJUqNh+c9rP1nJsOvfXbmkoBxoPpySFeszJ1HNRml152jV/p+Te197VV9/VywofzIZgE+GdVtaTjpbyXg7GcvdwrOqcI2Dj/hFWW/2nyC4U2ACoTjSmCCOZ3IYrrXEfemmS+npiyr7yPYoxemiF6fnCyCcjp5E5OmVyYOJu0uOnH/fh1WcLSv6xanh1Pzmc5Dh26fmZuS4FxEkldWDjUSGoZhPqBMHz/OJvpMRfZJVYGmEVaCCoRbWDIEe79J9IRPjspVCr+5NCPlr7CgtqULOszLLuj18UeO4T9kxCPyaTIDV1AG+kcFlXQncVRvPt9S1HUzwQv9a93GniQ== Received: from AS8PR01MB7944.eurprd01.prod.exchangelabs.com (2603:10a6:20b:373::5) by AM4PR0101MB2210.eurprd01.prod.exchangelabs.com (2603:10a6:200:53::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.13; Sun, 24 Apr 2022 04:43:11 +0000 Received: from AS8PR01MB7944.eurprd01.prod.exchangelabs.com ([fe80::1854:2c30:7ba1:c431]) by AS8PR01MB7944.eurprd01.prod.exchangelabs.com ([fe80::1854:2c30:7ba1:c431%6]) with mapi id 15.20.5186.020; Sun, 24 Apr 2022 04:43:11 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Sun, 24 Apr 2022 06:42:55 +0200 Message-ID: X-Mailer: git-send-email 2.32.0 In-Reply-To: References: X-TMN: [DSOqVD1+Kgmp1kD3HcJ7PIXQ5GjVJbFZ] X-ClientProxiedBy: ZR0P278CA0069.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:21::20) To AS8PR01MB7944.eurprd01.prod.exchangelabs.com (2603:10a6:20b:373::5) X-Microsoft-Original-Message-ID: <20220424044258.888081-3-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b7ac8566-aacd-406e-f660-08da25acef74 X-MS-Exchange-SLBlob-MailProps: S/btQ8cKWiT4+P6zBac8z1eXEhCqSjjDnXLdqFQwCJIlneOBnZ47gzUBdsyOhAY6Qg0KPZ2HKr8z79ijV19mmzzyIyyT8rYx39/YIUdjiWgRI11n3Qy6LQTH2ACDPN55j17qj/5xY+Eo+Yi8KBMAC+kRT/LbXc3DrvXk0Mr7CNP7QJ1DxK1BAXA06TZvAedFLuuH2iHXJNIbt3vDtE14hM3leGyImguTitg5Xa9p3cjQdGLiC+rYmTJ68Qr9QM92fMN+NFTW+Xcq1cVYMA+tbLU0smlJPdZmdw4cWPJXzidoLNi9u7xQQik4gTZ9nA8QG4labyQeq+uCpP9VJx1KywKjdSgZXqsDR6l0F+MdCXa5WBniGO8/Gn28CTRsTpHF3eesOBFEE5Zb69rV1excGqSKGqxoTAj2QShAICMqAQcDRRgouWeXpUpJYTeesV4eLS/0UEy6Jrr+XFrhCv3XPidli9DfFedc59GR1QF5EZE5urrNxLH9iNAj6FhA6ehzLLC7rt5dI6fg4fn2bk4+yHnJx54FtXJ4C5v+EHZ0w3AhB6/jYAFmobVNklDrYFr9bmCTfYcqWCNNk5MGPdhsEGFWZumXnZ30ehIPXPQeShunxVhuRGfVRjQiO3PmV8Q8DkW2kPW/r0723wGLrQonGZfiRp/ZM4UtIpZkkSEMMgkVgjGIIGk0857CJcPPOxAWbcQe5Jytx6ZO7G/BK5hqFsK2i6kfCVYoOlfz5dH7LKTnAsf1UGYf1kCpsClSKbr6scGlimjpWNI= X-MS-TrafficTypeDiagnostic: AM4PR0101MB2210:EE_ X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cdbQSY/X3hCiKLPGOHTJ8AChuMdmSERRLfseQPveJB/ND+1fy3E9OboE7XerqAPNAwcuRRo2gCCtJlfnB+NketRP9TFP/tVvSE41Y4+DHeLX2gq2uXEfwZY5z+OCvEtiafRYHT8eRi6xi9+UbO++0MVEnu0/YgK3GvkKUrjaVRDW5XEOCIhfE/S9Ve/kmMR5PfIJyg0w8VEjCkzUdXZqDx3aaNmMYtcuKMjJjtEm2cniDrWB6trWSH7aNqWX97FOfm/H1X/NbML5Q+8uq6PCxfYiJdQEy5PJgmWCPu9OJBNw22BbPk7QDNtoHTFDhz3qLnqi27dGKCcbLIRS4bSX91bSgCoEP0pffxF1VPTXy0QLYjqyt69D6ZVPC3mKxbo17NHb3RbcbDQqGqbrXPO8CdBjM9H0qjqQL+EwLMfhJY97i5bMIRLcJ2T4C6VIZP92z6HbpGh0QN/Y34eG25nsq7whVcEstBKCbLdiE9xlEkWU3IPSj/mhM1Qm1AVa9WPUnoRmz4oxc8WbV8TwoC3q9wxtJx1lfYcUCKzheK7oyeexcdf06ht5PbdEbREPwBF9U3c9r/7hTcq/6N0fPMZF4w== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: cWGJ6b3Q6ots0rfRF+xfIzioN009+9WPob0HxZKjhzyZpKHE08oRu5a4fCQtgWCe8Gm+iAxPyGZNgPRkjGwQDwhmYIuJtrYT9hp8mtkhLsk0CyIQmCnz7wyqVWfBRVnNF4qND8RpLxoF3nhKdpspvFrQO8kU6mgjoRBxStEhI0GLyufym349z7PyeLwQW1eKT0C44ZBIf1gHhlEvxKKLSSo6Mh1A5r+aAz3Egp/Nae2y+t60by+rl9z2RGZ3RRgbCaEPKQEEAdBH7MRW+Aog6TaK6mryBlHMo5ZnvRsoGtCpJMo16GRvNUwRoBxD7yBaxIQ6Nvtu2KQNEtHouM7ObIkCPUmrIoJsmZ+hmiUJAwdKSwLg0ypB6teooA7xb+nonmruXx1rOMOpgPVRb5UInpa03eCSaJYJOSHpTlfU/p6yTGHWK/MR/IElGVtT0ZUi/KP+kg16pEDMJ34d2XTI+y2h+QFApp4izRvWcwoDq8/SwJx7pPX9auTFwhb+dhvaJ3ZQMBPYrM1FwHaC1aOPQb4oO6hWJIamk8LhncgVeqgEqGnfqIRKpAQayeedGMCL83ORYtWS0O9sKuCfd0mP17rNANN7BcyZO7tfJFCCcJW6IhKRmFyMAobIbRt9odfhe6ToaTs6dbYPZMFPopLY2T5yaoHGj4t8N0ceElyEL9/VeCihI0uld58SsFy3IPkUtISA5gY+AFqXBD8RjWPWpQFaOQvAAyJKTxQlwhd+7YDDJxOWDuPRlz67+48oE1dVkrkQ7bb7l7K+UVVI49NoFlsszpFeF9tGheXRjyOKhm7ZaicGW7IeiLQ9Z4nHs2c9gq/G38YDhF+lz9hffncQCdtF4EZAI1FH7EZokT5Qt8gLFV/oEIfKnFHV6Es+5s9I3bqzqxCnBWvBg+9JOCbih9MzZtleZ8qDU6cKmv7+eocC7vYGg4yX1A6XSTK+GOKPelH80Akdhl8fidIiQckkU1CvuhAldEqMejGYWZRsJPwdpLS04s4cciiNhanQ5tbDanRw4qXSucP2Eu+z/qCAwpVDD2iaItQ9zwVL3JvTZUObrSDlf0h4L3k6/7g9vhXJ35ej3MSM5//b0j3RH+BweGp73VYVhw1+m6GhoAqDmCcHr3BIsCnylKFwqZ9Jk11/pl+F/a+zy122Ydv/leiG2lBfP2mnQaCzocGfx7xF9ExsRcpC1Qp72grK/6M6zMqix9sXPy5WHgJUZmL/w8gzYjPcbaskNhdxPs1KqmJHBH3PiQMzL2Q2oE/7iNqxfnvWPxkb3wH3ruqmVY3NyvZSBYZkZ3Xr/X/VnrFzeQs+OS8XA2w/v1fUZV2YqmtnObtTamBTa6wnbKmHkUv75uxDpeTY6QY3C8Bn5W1A40IttIGjO2ygd1D5dkoovvEorAY1oORqqGUJa3TIvnSRjjS7eLdNdgXJjfj75HHCf2I+/wHXyRqMNvBnvBgQi7JIwcDiq07RYkCL6Kge7Ep+gQZ46g== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: b7ac8566-aacd-406e-f660-08da25acef74 X-MS-Exchange-CrossTenant-AuthSource: AS8PR01MB7944.eurprd01.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2022 04:43:11.0399 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR0101MB2210 Subject: [FFmpeg-devel] [PATCH 4/7] avcodec/pgxdec: Fix issue with negative linesizes X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: j/rf6Fm1y2Cg The PGX decoder accesses the lines via code like (PIXEL*)frame->data[0] + i*frame->linesize[0]/sizeof(PIXEL) where PIXEL is a macro parameter. This code has issues with negative linesizes, because the type of sizeof(PIXEL) is size_t, so that on common systems i*linesize/sizeof(PIXEL) will always be an unsigned type that is very large in case linesize is negative. This happens to work*, but it is undefined behaviour and e.g. leads to "src/libavcodec/pgxdec.c:114:1: runtime error: addition of unsigned offset to 0x7efe9c2b7040 overflowed to 0x7efe9c2b6040" errors from UBSAN. Fix this by using (PIXEL*)(frame->data[0] + i*frame->linesize[0]). This is allowed because linesize has to be suitably aligned. *: Converting a negative int to size_t works by adding SIZE_MAX + 1 to the number, so that the result is off by (SIZE_MAX + 1) / sizeof(PIXEL). Converting the pointer arithmetic (performed on PIXELs) back to ordinary pointers is tantamount to multiplying by sizeof(PIXEL), so that the result is off by SIZE_MAX + 1; but SIZE_MAX + 1 == 0 for the underlying pointers. Signed-off-by: Andreas Rheinhardt --- libavcodec/pgxdec.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/pgxdec.c b/libavcodec/pgxdec.c index c9ada5afb5..30895b51ee 100644 --- a/libavcodec/pgxdec.c +++ b/libavcodec/pgxdec.c @@ -97,7 +97,7 @@ error: { \ int i, j; \ for (i = 0; i < height; i++) { \ - PIXEL *line = (PIXEL*)frame->data[0] + i*frame->linesize[0]/sizeof(PIXEL); \ + PIXEL *line = (PIXEL*)(frame->data[0] + i * frame->linesize[0]); \ for (j = 0; j < width; j++) { \ unsigned val; \ if (sign) \