From patchwork Wed Sep 28 18:58:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 38455 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b1c:b0:96:9ee8:5cfd with SMTP id c28csp444332pzh; Wed, 28 Sep 2022 11:58:46 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7mILATiECGik5Z43+YrZ9fvQWgdinuuojKgquEPbM+JrvnQuLcYRE7cU0YXSbmnG9A2usL X-Received: by 2002:a17:907:7d8d:b0:781:fd6b:4545 with SMTP id oz13-20020a1709077d8d00b00781fd6b4545mr29722584ejc.117.1664391526523; Wed, 28 Sep 2022 11:58:46 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hr38-20020a1709073fa600b007867dcd3ee8si6489618ejc.810.2022.09.28.11.58.46; Wed, 28 Sep 2022 11:58:46 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=tWc8ct5P; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CF47768B357; Wed, 28 Sep 2022 21:58:38 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05olkn2096.outbound.protection.outlook.com [40.92.91.96]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2217968BBBD for ; Wed, 28 Sep 2022 21:58:32 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=l0CBbPZYiBf1ZYYo3NwrHc+KD53Ecus+HOTNpxCUAOSUOfvNC8UcgSqwtsXQeuSk8poZjlmR5+3CqINvWVS6UrmiXjf6iwVtdwQJ9wt/RoZoa0nwP6M/y7NAO7TgwR0Ybo/m3j2iCh0wsUTIrg/pQz5G+1Kg3rYLrXGVtkaN2Hg3ZIzzIGKq/YvIdc8GB0QYFvxArOnIrkViugPZqSi7aZsH2HTd9Cf/WDwIxX5sxqbRmAZiss9f19J4v9G0gr/L47A8MeegmFWQlIMMzkIWoM0jr9AyHyS/J57rhMiE/mnvq/zqYwImhV7fV/EDzKYhjvDiEsuBcwBaocvUAVtdYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8hEu5/r4wuNIxhoUNESat05UH7x6EisL1QE+F+oOYeY=; b=inSvYaDgWkmSFfdoJm1ljD2yPh1mXjA7uk/5k4tAvrtMBDhR3STcnHDZgqPZq+0oMgvySx4JSnjS8eH7ejpWweVKG2Qs2sz+pv3i1YPB6Jxf4Qz9ajcH7BOR3S+RKPxXipZ0LdLli/oYL1FRqZiGH9YQ48t5Rkp7JMsOVIf2zCfnk9JcZ3nQcslt36Jp6jpnNCkmEpfmvjWfHIZvdXMtZODCHAFaiz5AAncdS4pRP6ncpRnttBfWrgorCzkSJ1TaT8s6OZvwaMf5AOwKPAhzP7hqBDnzeDWHyhzPk/LNYAcyKvdi5vQGXTqW5fGdqWeAh2UWtIkXnWrNJXtbDvBkVA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8hEu5/r4wuNIxhoUNESat05UH7x6EisL1QE+F+oOYeY=; b=tWc8ct5PpipG44QvbXlXuiGfRU5YjN/uG2xvVskO7FUHuD3Wvc4p3SUnoPGZ9JptUo1o7P4FsvvauU5ybpkaDMuYY/BSYxfneU1SDJx4xzQ3u0oE4RcXWMkgwHvWNPU4YyDfQTFmz3zX+6i/F63IYiwFTQMDh8/9naJeDqaRKEZO1ObOjIe64gnOnDJsVdsuZvPdeqH5QTPq+G+h8btyp8Tmi79kcruC0BLEobFZUBvx5KN5Z+KA72mkZrmcRFaCrdFBG9PVVPlV8Wc/i+x5Q68xk4TZRqWh3gasTcfXy8ylTXlDgJqwss8xjpq1FQEMXWq3wVgI86Q3rIt8ClLiCw== Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) by DU2P250MB0014.EURP250.PROD.OUTLOOK.COM (2603:10a6:10:2b7::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.17; Wed, 28 Sep 2022 18:58:26 +0000 Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::68bd:2fc7:ac52:38f8]) by GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::68bd:2fc7:ac52:38f8%9]) with mapi id 15.20.5654.025; Wed, 28 Sep 2022 18:58:26 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Wed, 28 Sep 2022 20:58:16 +0200 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: X-TMN: [YVoguzegjZuWD9F72+GU0GGKYBF9aQg60/YvdDvA+50=] X-ClientProxiedBy: FR0P281CA0051.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:48::12) To GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) X-Microsoft-Original-Message-ID: <20220928185818.1074689-3-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: GV1P250MB0737:EE_|DU2P250MB0014:EE_ X-MS-Office365-Filtering-Correlation-Id: d52fe29a-3498-4916-761d-08daa1836c78 X-MS-Exchange-SLBlob-MailProps: EgT5Wr3QDKyaZg3Nqsb2Hykb5+EITt3rPgoizaLs1HqW8ByCzlics+Jad1NwaVPmXMCOvo9u1Q7AjdXdF2H0RpCNesnBH0e63Zg51aUMumq+rqnZm09WPuGnjmnKnZ6DkU9+WO3ZFzPBTu8vq5df0IkihvdhOf9Kv/hbnm01EHF1TbJqJcZ5Mcc8FNFFjrPmXkQwbaEJDVEzzs+m6oBc5JTtPvhKAMGa4hhiVBSG46+tWO9df7BUtGUlBx2mZTLkswZB8tX92bZ4v3ZZHaY4dJwQmcZhCSS2IDz3zJETZNGKV4/6On6JMl7ZYvToQ1TI1pa0Pf4vvQShEuk2tqCpY0llex9LkFR4ko/4J5fp/GdYk5b45rQwf5hV/n86Q+fbOPH0Bz0Iix4t5yCnzLPwmUB573r22NATD0bULwtbUlIZKOITTQGkGQiG6Y7GyjcWVfcr15Z8pigBiyKZYNIW8LqttiOMHQxakcsGKJwXskWbrWbrppk4JuMjbx6SP51uPybOcmZylRmXJObrdFc5WAfHsO7UybT3B9xGwsb64EwVRH3NCsss/naWlsYMLH4FIWvegijx1CUEEOBLnVnDuC0eC/dqoS/DC9T4aKtUtRq7z/yOj5kW2n3OKiXCnWvbRoX6SrXu9pFyTJiItgYDrUUD9DCui7CS/e9f5/ZodeAzCEKjHgHMLdJ/c+DW2runWVajM8j/JFWeuuMYjW4iW0InEDRVzgE5bKNCRds/Lxw= X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ARWpaucfBdYWMLwB1zS1ZzcRDsI2v73R0vBv+ohP9dnfETdV4jO4trLH1W5xrUJwjZfZaQfbocAPzyac2I4m3Qtvjcz1GZ9Nh1W8A3JYvWCvJcpvCW5GcMCiFn7yJ5T5ZkuLOGiIWQ34WiOIwLTklQzy0sATm/AMNPtMW9nlVMubskHrgFNFlqFasDAeUl1v0wrdWRjl9bLkRSB/B3/JZJqnu344lFgddZjhFPf+9t+HguGpKrWCAR39nKyj6fpdecAgbtC+mX5GMDYi1eCyXS8WboeZZs3FbHzrT27kdvbFo90PqSywws+AZOON8gcvJLBwtTpLXfM3BcFT/WHKz1t9lONTgR9M8zWsr3aRGb4eT567UtD25bhmMX+PVdfx/i8cDHSsC8wL768WsvSD2mzqdLIExZPLo8xFIQroDY54Y5kUYOAdKQEfAecNNkzUM4oERCkMCC1jfKNldBPChEifaGq69D5h2dwSBuH4JGvWkyeMJHSelwndXL7rh8JVv4R5Gw55NTNEqk27PJ90lCwuzEK78MsxVDclzNFfYMXR0SwiBljryCXMPy1s2J0ru1zKUIMG9Rb14XT/LkEgY+hz6RUnUUcUe7xykxcORve+gOoSZZ4KlAHZGC9lPDMBaj0U/V7NIHReAtxnaV0CMw== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: U0W9l5YRpdS7W9rleogYcgeIC9JxgLT244AoUzbLwyRpibZaHI9xLD+NDCVoKCkbf3/551LlXJWriFFrI+6I9rw5ftkdtTahrt3Y3g+UTDrnpXgw2ijrtBqu2nVhVsWX7278rIEVeM0LwG7Abphst/rrgHJ3IpylldfDA6xfhltCzVrkc6LVG5iZgYI2CDMBd0oJHD4oVVSA5eR+fkVmmBbeDiFsCvXbQOvH3F2CK3l59TTFsJOAca2N2soe0cyGyE627j11s7D9w92rXsfvNpvHBymERw+gPRzrjjXutRHwUIKzP4eSnGsgl4nb5YKGwVLUeM4tA9xsv1cBnrblNAwTUjsgGlc8Nl8Tl8QZea3NUrhuy8t2EbJx4bvMUSBPIOPnlyXN4VuHdc7OfNQVFSIWNyJ77o/lbh4ulEMVxOOqc1Qkvshqrc/uqAu3eB8L7Ngs9S0/WNhV32jUnZDdVQfi98EXgg4vdIOmwURtQdLU8oKNlNSnGk6Jn5mQ3/Mc9KdsqXht/9KAg4HgCNRUepGoTcMhAPxyqSs6svSQj8zlbyeCo2cc70z9yAFDZ5kiVEhJDXj5SEto3/lmZPebdYq93bFWH4GnNt0jcRaBPOin15y6rA9BdhJ7KNyIilXBXvgiXi1NeV8sjv94Uzevtpvl7JdUa9hKThDmf/NgYuJ2dlV9ucNrTJw1d7uJ9PcOYaMaIFJenA4L9tHdpREohjmHC2OrLVw2ZsG2zFeWBrK3pBCFw4S2hUqqy+tUv1uWyQsSiuyALQW+gQs+geWMnR7bKlfxe+Jtx7PAkEaz7xkAHB7wBp9hu1PSo0KhMxdR/DYK0a1d9/PwH+/HQc6NvWwz0rbwo1OIUPeGr33XIf6wVUNRLSQe2AMoTenzKZRoZC0dGAoXBADgwU8SLSAc8PgLMiljXUjEFFA3Xxra/9Qa+HhOLirWfo12bJHDvy1ZGOVDl2+5vEMmmYWGAgdpL03iQZafZVogyRPjavh0/Jhkrdh63bO0lHP/fvelf7jsNu/Agj0nPYN1ZkPqyts73YU8v1lPJ/K9qHt+/kt5AT5j+79wgLMbEe364AS9tOdwJyXZEUjTsgwCOzR2MhH3DjccZ7b29tdKxESuUPqRWh0in72gv+KITYo1Hi3r0Y+MNzeb3x4zy5w38xmKJMYy5d0GxL/15SnD+ywRRQ6ceEBiQHJpXymLgTqB7sH9zhHnczuC8RHDG4h/Zbi1Hd9AAedH8rUGEoD52gbMvejJcTsF22wViN2/PNyQNG7OgFd/nz4hLDBcS53gtKGjYDNwQgT7HYm90cDtL9qg2etv54CTRXiG1nMNZBdYCuulZ9Ow X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: d52fe29a-3498-4916-761d-08daa1836c78 X-MS-Exchange-CrossTenant-AuthSource: GV1P250MB0737.EURP250.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2022 18:58:26.0530 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2P250MB0014 Subject: [FFmpeg-devel] [PATCH 4/6] avcodec/jrevdct: Fix UB left shifts of negative numbers X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: cgbGOixwDZb/ Affected the rv20-1239 FATE test. Signed-off-by: Andreas Rheinhardt --- libavcodec/jrevdct.c | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/libavcodec/jrevdct.c b/libavcodec/jrevdct.c index 36160cb663..7f1863515f 100644 --- a/libavcodec/jrevdct.c +++ b/libavcodec/jrevdct.c @@ -255,7 +255,7 @@ void ff_j_rev_dct(DCTBLOCK data) if (d0) { /* Compute a 32 bit value to assign. */ int16_t dcval = (int16_t) (d0 * (1 << PASS1_BITS)); - register int v = (dcval & 0xffff) | ((dcval * (1 << 16)) & 0xffff0000); + register unsigned v = (dcval & 0xffff) | ((uint32_t)dcval << 16); AV_WN32A(&idataptr[ 0], v); AV_WN32A(&idataptr[ 4], v); @@ -988,8 +988,8 @@ void ff_j_rev_dct4(DCTBLOCK data) /* AC terms all zero */ if (d0) { /* Compute a 32 bit value to assign. */ - int16_t dcval = (int16_t) (d0 << PASS1_BITS); - register int v = (dcval & 0xffff) | ((dcval << 16) & 0xffff0000); + int16_t dcval = (int16_t) (d0 * (1 << PASS1_BITS)); + register unsigned v = (dcval & 0xffff) | ((uint32_t)dcval << 16); AV_WN32A(&idataptr[0], v); AV_WN32A(&idataptr[4], v); @@ -1008,8 +1008,8 @@ void ff_j_rev_dct4(DCTBLOCK data) tmp2 = z1 + MULTIPLY(-d6, FIX_1_847759065); tmp3 = z1 + MULTIPLY(d2, FIX_0_765366865); - tmp0 = (d0 + d4) << CONST_BITS; - tmp1 = (d0 - d4) << CONST_BITS; + tmp0 = (d0 + d4) * (1 << CONST_BITS); + tmp1 = (d0 - d4) * (1 << CONST_BITS); tmp10 = tmp0 + tmp3; tmp13 = tmp0 - tmp3; @@ -1020,8 +1020,8 @@ void ff_j_rev_dct4(DCTBLOCK data) tmp2 = MULTIPLY(-d6, FIX_1_306562965); tmp3 = MULTIPLY(d6, FIX_0_541196100); - tmp0 = (d0 + d4) << CONST_BITS; - tmp1 = (d0 - d4) << CONST_BITS; + tmp0 = (d0 + d4) * (1 << CONST_BITS); + tmp1 = (d0 - d4) * (1 << CONST_BITS); tmp10 = tmp0 + tmp3; tmp13 = tmp0 - tmp3; @@ -1034,8 +1034,8 @@ void ff_j_rev_dct4(DCTBLOCK data) tmp2 = MULTIPLY(d2, FIX_0_541196100); tmp3 = MULTIPLY(d2, FIX_1_306562965); - tmp0 = (d0 + d4) << CONST_BITS; - tmp1 = (d0 - d4) << CONST_BITS; + tmp0 = (d0 + d4) * (1 << CONST_BITS); + tmp1 = (d0 - d4) * (1 << CONST_BITS); tmp10 = tmp0 + tmp3; tmp13 = tmp0 - tmp3; @@ -1043,8 +1043,8 @@ void ff_j_rev_dct4(DCTBLOCK data) tmp12 = tmp1 - tmp2; } else { /* d0 != 0, d2 == 0, d4 != 0, d6 == 0 */ - tmp10 = tmp13 = (d0 + d4) << CONST_BITS; - tmp11 = tmp12 = (d0 - d4) << CONST_BITS; + tmp10 = tmp13 = (d0 + d4) * (1 << CONST_BITS); + tmp11 = tmp12 = (d0 - d4) * (1 << CONST_BITS); } } @@ -1086,8 +1086,8 @@ void ff_j_rev_dct4(DCTBLOCK data) tmp2 = z1 + MULTIPLY(-d6, FIX_1_847759065); tmp3 = z1 + MULTIPLY(d2, FIX_0_765366865); - tmp0 = (d0 + d4) << CONST_BITS; - tmp1 = (d0 - d4) << CONST_BITS; + tmp0 = (d0 + d4) * (1 << CONST_BITS); + tmp1 = (d0 - d4) * (1 << CONST_BITS); tmp10 = tmp0 + tmp3; tmp13 = tmp0 - tmp3; @@ -1098,8 +1098,8 @@ void ff_j_rev_dct4(DCTBLOCK data) tmp2 = MULTIPLY(-d6, FIX_1_306562965); tmp3 = MULTIPLY(d6, FIX_0_541196100); - tmp0 = (d0 + d4) << CONST_BITS; - tmp1 = (d0 - d4) << CONST_BITS; + tmp0 = (d0 + d4) * (1 << CONST_BITS); + tmp1 = (d0 - d4) * (1 << CONST_BITS); tmp10 = tmp0 + tmp3; tmp13 = tmp0 - tmp3; @@ -1112,8 +1112,8 @@ void ff_j_rev_dct4(DCTBLOCK data) tmp2 = MULTIPLY(d2, FIX_0_541196100); tmp3 = MULTIPLY(d2, FIX_1_306562965); - tmp0 = (d0 + d4) << CONST_BITS; - tmp1 = (d0 - d4) << CONST_BITS; + tmp0 = (d0 + d4) * (1 << CONST_BITS); + tmp1 = (d0 - d4) * (1 << CONST_BITS); tmp10 = tmp0 + tmp3; tmp13 = tmp0 - tmp3; @@ -1121,8 +1121,8 @@ void ff_j_rev_dct4(DCTBLOCK data) tmp12 = tmp1 - tmp2; } else { /* d0 != 0, d2 == 0, d4 != 0, d6 == 0 */ - tmp10 = tmp13 = (d0 + d4) << CONST_BITS; - tmp11 = tmp12 = (d0 - d4) << CONST_BITS; + tmp10 = tmp13 = (d0 + d4) * (1 << CONST_BITS); + tmp11 = tmp12 = (d0 - d4) * (1 << CONST_BITS); } }