From patchwork Fri Sep 30 17:02:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 38479 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b1c:b0:96:9ee8:5cfd with SMTP id c28csp1837050pzh; Fri, 30 Sep 2022 10:02:51 -0700 (PDT) X-Google-Smtp-Source: AMsMyM57/FYJkjqmcaRELJ9FXWidgnDC2OnLH4nIOFCtRykl00qUNj8XC+F/AdBOoFrGS+/jnZ2W X-Received: by 2002:a17:906:ef8c:b0:77c:7b35:8b04 with SMTP id ze12-20020a170906ef8c00b0077c7b358b04mr6785626ejb.573.1664557370975; Fri, 30 Sep 2022 10:02:50 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id o14-20020a170906974e00b0077e94a4867csi2388900ejy.796.2022.09.30.10.02.40; Fri, 30 Sep 2022 10:02:50 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=DRYaryVb; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0530F68BBD5; Fri, 30 Sep 2022 20:02:37 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-oln040092065043.outbound.protection.outlook.com [40.92.65.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A9AEE68BB2E for ; Fri, 30 Sep 2022 20:02:29 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ixxhcKTenLNgzlfOeZsREfGnzhwZSlulW7CYIhKblNoFVGAOKJME2HiGpIXZL/EJ0rZmG/EAbLXrNBhNdokA5SjO/MiT5aVIziLaMkgqw1Lr+3ILvkdhblqTzJYS5hVvEeZusf3h/hKw27PECoODgdHq4MPinRfXYriNjnKKU5WnnLVRxoDkwqc4OsnvPYe8YypnFpjs59aa6v3OqwfkFOA/qlu382mY7ujkeeDxjqRGgw3USbFBnQbZynNS9tb1atFCJe1FC0SHhQxIOQBkfNFVEU/zlAiQvXNCUH92F//mvhaiuHBqnu/3bPq4bfaPNhngVjRcSPWvdKPfCMYGyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=v6xZbkkE/DO6FfVzE9v7XsdNUHlDx9RlBWfO3FkcU5s=; b=Ql+jkmqS5U62/BLdEay14VikV1U5ucmvH21n3XGQx6e9l6v66CLdHMCD8Mqwb9RGzOgYeCD0b+dtc+V6As6c1ogPRl3ebCf6X+dlOSeKH2ovT0EgFsMs8O48LL9Dp2wbEZ02LKeQoPmIG1kIyspHoyoC0YPwdDkqWSTXE3bysUS3d5lEoG8qAuG4mWcKvLoaxbK8kMZfNl9LIFT1gTtKSqrIjlsYEM1l4XRGvtyNDKbVbrGfSuHxbzrbOZTDvGq/TSfNpj5rNJeT4AtxS8VPuY7QHSa4waLSWX1TUnrujlh76lZVUHwtECTyPyw3pgX+3Iv3Kr8OM4zVYLdDpsy97g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=v6xZbkkE/DO6FfVzE9v7XsdNUHlDx9RlBWfO3FkcU5s=; b=DRYaryVbaW8c6njwiGMIPPOOlsJh/TWUroWhS9gjLLl9+grK4wxMUjgO4q3faMrdcIE57KNXHAotb77Fp4OfCpquj1aNVptl7cFnvF1IJQ4Pz+cGIm5WB7USz5j2GXNbkSGAp0vZ6t/4PkBAiNywze3Q0QQI1tdlDMInHl1AD9Tvv8eqDvtqQsS27l/G8dI3MBVnXdMARLGtec5FqiHjlakglarAgMbkCCdcCDvVGIqDcnm/Y4L4FqUez5W++tQoD1tv8rsmzNQwpH4J+ND80O2Ogsz7Aba1cKN4UqvB+ObewgZRvVsHR++0xLevsh/d29aMbNq+zCLv0TtVtMazCg== Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) by DU2P250MB0175.EURP250.PROD.OUTLOOK.COM (2603:10a6:10:274::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.23; Fri, 30 Sep 2022 17:02:27 +0000 Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::68bd:2fc7:ac52:38f8]) by GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::68bd:2fc7:ac52:38f8%9]) with mapi id 15.20.5654.025; Fri, 30 Sep 2022 17:02:27 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 30 Sep 2022 19:02:21 +0200 Message-ID: X-Mailer: git-send-email 2.34.1 X-TMN: [KB0tGc0I8ZEnHqHqTuYGDknGwquX+P91rB3OcIwCQMY=] X-ClientProxiedBy: FR3P281CA0178.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:a0::19) To GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) X-Microsoft-Original-Message-ID: <20220930170227.3648162-1-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: GV1P250MB0737:EE_|DU2P250MB0175:EE_ X-MS-Office365-Filtering-Correlation-Id: 6f495efe-3b98-458d-019a-08daa3058db2 X-MS-Exchange-SLBlob-MailProps: EgT5Wr3QDKzwu8YPsqYDy2dEXP85Yujlm5jI7azyUJzTyHFgJhCKt0OhlHlVFiqJXi0RtMwv2Dg0sacHUaWCO5vgTSAKrhd5R1pKqk0QXbrfbMvztjt4t6tSapdvWZxGnonsPKo6FK2ngBwGxEZFft14uJYpZmV9NuhynUj3hEbn/bIbQzoUmQ/FzxEqHoj2mQ367d6raF8cN+wnMMUApgyltTliYbL/uX92Q5LaNrY1xEVaJf36/+Lq0llZeb31ntvraqlFwqIQ9MxjgdfQdo5zfbwPRcVHeEIp3vwWWSqhJzDWDUak0GsNrec/HxCLJua85gDuGiv4LI0CsUdS6vZgpHRITCkRxLXdcYsv2XwdcdyjL+Yd5OnDuBKqu5ZxdYCVkwxO3sQwfBIaR/Wn6JQ5BKUvSpkHqPMpzjBXT9vFPFkPB99i2QTBapc2Wf1+1ILl9zz/nnwbd/DsYBse9PHXpBNz24IQujIQz6RZntW+/faxwL4LUYjgCc77YgeXaXKv+CzX/TntKolmlr7i7zBBgAG/BbjAEoJaNvqRSR0zGorFvqx/PhqIPxGnzpFI+NAQsSXiepmEhqotSCR4k+Onscu0OvVJdu5bMQJoPrNbuWqWyNs3hjBWCBjMBH0t7O9bcyaZfgS8eXASSHBVbm8j2W21XPSasTGHMC7CklyfjNBi4k//B6iT3k+AJdc9aSobJ8L+k7SXx6a7QB/yROwBrbpZ46KATgQUFCa9YeQ= X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3zWcTGrr1uhQiEPWizoh8Z2lBVHEkAuXH2GHIjtquVkG9OBnI6bNr//bd2ZoDWlOMucOmmSsHfP06UG3pEGvQedhyLd92j2PrMwjwW2FcAqS/lMRPwkOTkGmde90KIoc9ungmqP6wHf11XfGr1vRv6IgfMyyZbsn5D6m+/3FzzwkKXauaNxwU8vOrGHZAvedQ6Qy+E6ukBmfHsuTbD9VVwXQkkZJ3eBgSUNXGj/pCxQh7meiNy5Q8MkTcr55cqtRb7ZF30NTLi9IfPpQpcP8O0b7ky73dcl4EaPh6cM6fN0HyVK6gl2keR+lAkV14ShbzBGCwHcmYPCh+xnEumeIXjyGeVqEeDr7mhz/OVSLiT66Q3ox1rR4WXY1ZvDlDeaIoVtwv/gQ40RZ8R8X8T8rNpBaGKczZ84W0VBnkTq5490eVBvCO1JkvA70y6O9DorlsdefS7mZCSKOCSN/umwL8TRYQA2oFqsz8gxdq1NErn7eX9R0RZKxZakjRaTDzqB0xAHnlzl5o4TXiOSrVXvJCCN62Uzn/J1dCR6OcwfAyPdNoxB4+tdnmYGXcSpZEWOhDlJOLZubEDdD1pNrAQPgo20RrI9kmfbiZmJ4VZc1J6FOCg7jW+I5AswjuDiIsgEY X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ViIjPUdXHmvnyrIhpyS/vrXDP8Bbo/oCyXqMI3NwizzjobblE1WEbZpuOWQfhW0f8jZgRyaK1aFjKOsWANNw8Wcts6Ku8DzDTxgLPwE2AxUfUlX1wdirQ3sAjphU8mY0XmXKTLEAbfWkcsLA7NHhMKuLJb0ojbXWSjTgo9Y+DTFk/AYWkCe6045lnRrDP1TAcanaaq3FUEK6rZcuQE0RQjQj0GBHxXlqveCuIdYkKwH6Z2lW+JJ1yF4JvAx6o+aeyyG9DgcF4BJya/KW8gDVYkVTaQzImm4FoV80PW8LbIRwtuYpN3lQUV/HgBkSGzZEImwQ2scACoMRoLqMR8EXciyxweETHRFAjh3J1bvI3pTUnX6wvnf0T8IQJcVF0ZLwVqOf0KKp2OcSLHSgrwEiqaMhi8Z8gABikLAJ2mXfOkngA5Sm9ZGWCSLnprgSqGC7gY7Hj3EqID0a64am1OOQipZxi95f+oZqEbhZTDrJf4dN486RqS2QZqLTSFljG5WFe9wOfC2XRxwCnTiaQV6QWv3rp3hZNOqi8UijF8aNDOrMX17ugkSs8zARPywnpvzR0SV5Ixp4TkVHVKnjHJ+2utqPQjQ7wyvzs0h9QyRhTh5CfB4XLnv0uDk/tEsHvbBL+eAHGtzFVVA/f4j84VReo2xRREKIkq4X3c7LV/PP/C5zqtF3M6njm12HMlPyY6O3rgMc8K8tn6fXanCLsGAQdEvnejELDYqASAWGE4DZK1Mh04ll/BDVMXJE/E0YgfjOMzBDaioInzA8UcD62+PXA4NSAa9LMtlyPgwp2elZYPbhed5mjosIlYeDLFZMV/DUyh+c07j1e7otSGC9MIItjWeekgcFZLxAJPlkueUDMS8aqbothjciehWiQ2PlDyvDWwEj8Tx96regX+DFkO8/aQCCCHywhzGTX1N8NGND73GneGLiGPzlZzezKeFZ+NmySqzmT5pGiTKD+l3RrUFZhWaNVXZZHaNpk1Jl+Z7CmbJFWqDnzaJPZujQ8QY7XpdiG6WiII2qNN6640cFPKp8m3aWY7XipimgbdQ/Iz3bCos2IagUT7OfS6yrCqBYMit5i/KS6BlUzuQ7j5eDNlm15D0qR57JF7jz1tnljOqTeVcEJUeyGYq42ozCE2z+2jljn+Gl0rJPr8EJ6rjCYEvs8L2agxAZsMDoyqwrldJ7Pgo+HiWBMBUY4iu1GY97gZbbtKpMNN2Jy73/Tn2j0nOgxZ8PAxVj9WvzB80QQp1TcCv++GmBBjy9AKjBVyeJM2l3aNAFJl9WVyF2uDzxQqxxGKNfftI1zDLqckm+WXi3ga4= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6f495efe-3b98-458d-019a-08daa3058db2 X-MS-Exchange-CrossTenant-AuthSource: GV1P250MB0737.EURP250.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2022 17:02:27.6732 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2P250MB0175 Subject: [FFmpeg-devel] [PATCH 1/7] avcodec/sunrast: Use ptrdiff_t for stride X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ViNt4SQaDXju Fixes segfaults with negative linesizes; in particular, this affected the sunraster-(1|8|24)bit-(raw|rle) and sunraster-8bit_gray-raw FATE tests. Signed-off-by: Andreas Rheinhardt --- libavcodec/sunrast.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/libavcodec/sunrast.c b/libavcodec/sunrast.c index 77feef06e1..45b29e4d72 100644 --- a/libavcodec/sunrast.c +++ b/libavcodec/sunrast.c @@ -31,7 +31,8 @@ static int sunrast_decode_frame(AVCodecContext *avctx, AVFrame *p, { const uint8_t *buf = avpkt->data; const uint8_t *buf_end = avpkt->data + avpkt->size; - unsigned int w, h, depth, type, maptype, maplength, stride, x, y, len, alen; + unsigned int w, h, depth, type, maptype, maplength, x, y, len, alen; + ptrdiff_t stride; uint8_t *ptr, *ptr2 = NULL; const uint8_t *bufstart = buf; int ret; @@ -141,7 +142,7 @@ static int sunrast_decode_frame(AVCodecContext *avctx, AVFrame *p, if (type == RT_BYTE_ENCODED) { int value, run; - uint8_t *end = ptr + h * stride; + uint8_t *end = ptr + (ptrdiff_t)h * stride; x = 0; while (ptr != end && buf < buf_end) { From patchwork Fri Sep 30 17:05:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 38481 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b1c:b0:96:9ee8:5cfd with SMTP id c28csp1839111pzh; Fri, 30 Sep 2022 10:05:28 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4x7x2N65B6XPnae9wTdpZMvPDLQJqZOarpJSO8/aiwskD5CZUUXVkbtdnhT1DqOTd4YpFj X-Received: by 2002:a17:907:3188:b0:741:4bf7:ec1a with SMTP id xe8-20020a170907318800b007414bf7ec1amr7346077ejb.448.1664557528154; Fri, 30 Sep 2022 10:05:28 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id r16-20020a056402035000b0045898c7b85fsi646167edw.178.2022.09.30.10.05.27; Fri, 30 Sep 2022 10:05:28 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=h5SEmfl4; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 72B5268BBE1; Fri, 30 Sep 2022 20:05:25 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-oln040092065044.outbound.protection.outlook.com [40.92.65.44]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1382068BB41 for ; Fri, 30 Sep 2022 20:05:18 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=A5J3oWNj5IRzZSXfaRQzp5UZe4Z5KgVM3BI6cqKOWo24LNFdyVYQvhxU3ILJ5n9sdUmYO6M5OlIPp+OubcmSepvDpE9qI7MTUmlXm8zhQggONQ8p23Ust6XLEyh/58z5OKXczgQVO9r4oIim0sfAy+JmYIBp6srPfZpEtJvVgQGXq7LQH1qt+6geUwrjpu/am1qIMX/6pSCgTbzX7K/2vCCMPpKHQT/S8eR37z5bqYAFB6hQpTHW1JFyol/NUnef4FQ/d26GyoMrTV5odm1VcjyjAoFjiaL10jNLEfDwIDbEuEu1AVnzYrzUU5ibAG+MaqOVYMQwlrnuOgaVjR46nQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Tlva94tVC9SUWD7Mjm97Z2lMMvMKbGb6K2+kPurn/IA=; b=FQ3F7N0jxBgKlNjWt+OsmWuPyUxP03pQg+/hnc6EUYRIgDAgOyw2yT+ss4yU+N6+c1mrg+6sXdxxjotS/Tv86n9qqSaR8v8Q6CVyIJhukdTuvfLQ8KOU3EJdz3SZZrIxPSpxOKvttsW217Lk9ypOaAd2Ft4t0IT7AWKNOeNc94y7Su2GcGK/0Kmdn7Zd7lYADhrLURykzpkf/91Rbdp4EZ6jzOht67rlaPpXZRudn0u7dN9MHcTqk/xPSSNu7f+Yu+fp0if2I0Cq7QlzLsCHVCMKKG5U5eitKgE6ROyx7W7bwMByPy5vt0UJbF7wkMeKkN0X6rBxvKZzOUdEzLf2Xg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Tlva94tVC9SUWD7Mjm97Z2lMMvMKbGb6K2+kPurn/IA=; b=h5SEmfl4VDWatIo2+w/Sg/4AF25jKDwp/MtbG1phHNDiuHSchULK35/4wJkgmldutmAcgh+dxmj9aKPTyNQTYB3uBph+mIc6RIQX/6b5anfF5PkEHrBT6Q73o7oCsMzu1E+Em47ZHRWgwrDO5NT0Jj7IEgs8lVrFEjTqy2XSFIbPbR0016Ok2d3o00KDEZsMQDovJv/Dh1flwLc+TolgsKdLDz+HHkm1QjxRdnkB6ncEBvp5n3hcl9JhOZeMNl6OPof7xFRhAwiDNwcaN2oiMN13/iWao5dMkxlYi7eQNevQqj34/CwZS/06FnfP5w8RiJreTNQ1dKF4/Q/EthgsiQ== Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) by DU2P250MB0175.EURP250.PROD.OUTLOOK.COM (2603:10a6:10:274::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.23; Fri, 30 Sep 2022 17:05:16 +0000 Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::68bd:2fc7:ac52:38f8]) by GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::68bd:2fc7:ac52:38f8%9]) with mapi id 15.20.5654.025; Fri, 30 Sep 2022 17:05:16 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 30 Sep 2022 19:05:10 +0200 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: X-TMN: [tF7nHvOlYexJQClH35+RdAY3ZmWDfZcsAIBZeGFqY0U=] X-ClientProxiedBy: FR3P281CA0040.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::8) To GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) X-Microsoft-Original-Message-ID: <20220930170515.3648239-1-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: GV1P250MB0737:EE_|DU2P250MB0175:EE_ X-MS-Office365-Filtering-Correlation-Id: 80c4a5df-6b40-4cc5-4a9b-08daa305f234 X-MS-Exchange-SLBlob-MailProps: AZnQBsB9XmrST/aUJU0/QUaXtXS9lKwz3e081neeoua6ykP/oFTdZu9H2pO0elfSP2DnWRODcsTvxQAlNXXZm2DJvgT2cBz/gz83tASRt8l+oOd3PdqDQQdpPOCzWtQtXYZ9lGjvfcpi9js2EpydptKcjDu6qiNtH6U/qvztt44cmnm+sPGdKCuY0dAW08s4WFlxJJvIXD/lVAVVLN7D+oYewlD0fxxEzWe+gzZFX1kiT1vwI2ccCf30MCu1gF+nzHCboUrdXenuo9bsE8o5JTEgHwh0vo49wVjH7N2XJNw/pg/xDmIJkk2VmwB5h1mll7WTElCfk+qdGQ4kVj+5R9U49ya1agl59L0XhCyxyxZbsfwXoW/HlTv+ydSkNz3c4O7FQLDT0qIIp2QhrLAquswjEUyCbv4CAhXJZb4e25GBcANIhHtbj4V2olTVEPo1M+sVHVOTPsc+YYL3XYfG8rwsWef7UcAqN68N6/ZFQ3tNu3M7dWVj/wcIPqJKZSy/Kes7pf8RrvxbwpPwAp9jQNg8TfU54w0q5xg9PrEwPXbf2r4bN6cW1GcvTEG1AXn7kgnt3jwot/l3DNhNUfiNYPGuoSgb4ycjeyOhphpa/iAx/OAQXpTPNy0v2ObDQkdun/+sLx6sAxJjlGhyBbYsvis0EFPfNXU/bx2pi73MqTlCOTTUMyFRvA3I7Q74vjbx+dLXGmI8jsocCnDRXh/X0o+WsBVjLZESmQ+bWkWv/VhAyl/rAf3hkp89SaEcPqzSx3k6yw2ArpM= X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zGaYXPh9PxuAAZ9Q7C5skA18iS0GMLGZCxN6OVXJ8MFRCVIUlY3AelZKD7vF4lo42jNainJc4SW4KavO0YmgHnhzfvPie2SpIRs0zsOjrmyqxDr2ysuaZaOWyBr+LQShLDTwOjSZMuHHWAnNL1KiiwfJRtGIkVdRvhfAzQq2FYGhSig5UmJq1FRky7AY9AIAv8TofP7i2UeHW568gYWFpn4Sqp+gleS02R3D/w02JpjFg2QwEsn27lxO67jbLWfg2N97H6N5Nt62l0O8QkqNRq8uVYWEyAISxiXbE8lSi8OvHw6m1PKe01KRwnUcbMxm3srL6BErtyxhldsCwKedsxtw/c0CqTwZ94CidB/iX3r5Qrii8DZ13TX4getMkWwqBzmM0E+u1r1mTPA9VjZvcm5ztrMBrvGW0CPoplMlg2gkznRgzNZ3Q2pLMMUCArlnTDbwEI/cqylufUJZ8fMOBKmkH4tEsytDermfYGtOKDgK1io7o36uWbry7DGHaS0ox94mH+lAqzWPLuok71SW2RWxBu0+i9fXWg5uGL+gy+aFuCDZxloi7Z31do9WHdgpunwuJ+AsfO6Jxoul6sO0XS1zi++QN5kDVkRjeJ0MpsNv5JZ16/eewhMzAsM3rGhFMMoiLkpv7S1JcPXYtV8WLQ== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 4tQtZvqYGb4ewPmUB4bfvXxaC5YvatyLn+2oPS2I2NnqeSaagYzbOtj/2ytKqfhOvRz5IWAPKY2BqBEzb4Kdrwr3UPfXEQZxRJiI362zd5Ds2gePPoLkdZLLy+W6b2enhVyw+pbVDYDKPe+QGrJCbKSFRHmHqbUiJrwUeIIAt7ngYwLhT1MCsv+Ow7uX/u0Srh2pcxTnCCyafeKIhsmH+xDvI8d2gQOPIq3ina89DVEwjXrfxd1o29xyXyz0hu0+dXk9OM35Z1xFxOS+4oqMvAbj6SJZAOm8y+aipeXagHz112Rp/vSdIc1DobRc7ptz+XbJE/cd+7R5j1j+644V3zug0b+QMHerxUUFtkLNBTKLFV2Z5PwHhmJQehrM66W3O+IK6BcbLYbHBbkQUe8qQOJAJa1/JMXWuAAyinK54JxsXoIzhMx+cV2rL/4m4NkeX/RDGFa/K24F4pCb/MPsctyu7yY3nYsGI3o2GeOkfdWCZ60lWL2PFBkJH8B6IsRR1sUw32cnoo3XiDg2lFLx0KDu3iG0qN7BqgQFo7ickyuhKXLuAQE/vSs3DHlQgfRH0yINN7/sXmYjke66Zijb7T4qB6zOaURzu1g68zjB6mAYGA8EzoOOhQXztDzB5VnCXMyC6ThcxA7EVljghI1Bx2OBkWfnjDh0ZPqGDjoxVeon6G41KG4yqoipFpRohml//+Rbxrf2hRJFwm7OQc+QOK6P+yOZYxvVNXhDNf3Fy/K+qYS2O6IkgCutF4D7guD2sdTHbghtNE0LOiMMstmDDL687imHHz8GWmzr4XqfXZi8PRc32HvIEaPlHK+/OBC2msXAYBDtMlRl9/zONGUzw+hiUpSoYehl0T94JQ6QNjEcgPX1izibh1j/dehJTCBqAy1Ynr77imxbxWDeiyp8CMK+jlJaU0fnseICtyuluhfY8qNrVzyqbG6/9AfZoj3iBor6xISRWhnMuwoyIJ20EAXnGJZjhKjXU7bC8cGeGTNO9JzDeNLRR9rQxHp7FHknEu77/p2KD4L9nI+u+6TWFx6GrzIgTa+3DkjgKDf6QBenGnKVCc+q6fJb9w+PDIQpA8+3pIOH53ReHjCVEXRicgl/95THPHl5x8AAO8BdwRLyydZ8Ci9OrO9zofp/SM8FG9ykoZr/A78t9/rr4lEp4/rlvbhuO66OJa0eQD/73xzvjgQ3cAl1W4ZrXCwW7GwCfqM+fmHpcQeFeKES/2dsOXUD12yVFZqlPyZkX770NETV/YhZ2LnuCK9AW7uJI6GntKKVdBT9/KzHf0focZ77qncBpPeIt5CX5q7/HjX/BoM= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 80c4a5df-6b40-4cc5-4a9b-08daa305f234 X-MS-Exchange-CrossTenant-AuthSource: GV1P250MB0737.EURP250.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2022 17:05:16.1742 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2P250MB0175 Subject: [FFmpeg-devel] [PATCH 2/7] avcodec/sgidec: Support negative linesizes X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: VGYvXYewhvYI The earlier code used "p->data[0] + p->linesize[0] * s->height" with the latter being unsigned, which gives the wrong value for negative linesizes. There is also a not so obvious problem with this: In case of negative linesizes, the last line is the start of the allocated buffer, so using the line after the last line would involve undefined pointer arithmetic. So don't do it. Signed-off-by: Andreas Rheinhardt --- libavcodec/sgidec.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/libavcodec/sgidec.c b/libavcodec/sgidec.c index dd3dc46b48..a449859bf8 100644 --- a/libavcodec/sgidec.c +++ b/libavcodec/sgidec.c @@ -28,7 +28,7 @@ typedef struct SgiState { AVCodecContext *avctx; unsigned int width; - unsigned int height; + int height; unsigned int depth; unsigned int bytes_per_channel; int linesize; @@ -127,12 +127,11 @@ static int expand_rle_row16(SgiState *s, uint16_t *out_buf, * @param s the current image state * @return 0 if no error, else return error code. */ -static int read_rle_sgi(uint8_t *out_buf, SgiState *s) +static int read_rle_sgi(uint8_t *last_line, SgiState *s) { uint8_t *dest_row; unsigned int len = s->height * s->depth * 4; GetByteContext g_table = s->g; - unsigned int y, z; unsigned int start_offset; int linesize, ret; @@ -141,11 +140,10 @@ static int read_rle_sgi(uint8_t *out_buf, SgiState *s) return AVERROR_INVALIDDATA; } - for (z = 0; z < s->depth; z++) { - dest_row = out_buf; - for (y = 0; y < s->height; y++) { + for (unsigned z = 0; z < s->depth; z++) { + dest_row = last_line; + for (int remaining_lines = s->height;;) { linesize = s->width * s->depth; - dest_row -= s->linesize; start_offset = bytestream2_get_be32(&g_table); bytestream2_seek(&s->g, start_offset, SEEK_SET); if (s->bytes_per_channel == 1) @@ -154,6 +152,9 @@ static int read_rle_sgi(uint8_t *out_buf, SgiState *s) ret = expand_rle_row16(s, (uint16_t *)dest_row + z, linesize, s->depth); if (ret != s->width) return AVERROR_INVALIDDATA; + if (--remaining_lines == 0) + break; + dest_row -= s->linesize; } } return 0; @@ -204,7 +205,7 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *p, SgiState *s = avctx->priv_data; unsigned int dimension, rle; int ret = 0; - uint8_t *out_buf, *out_end; + uint8_t *out_buf, *last_line; bytestream2_init(&s->g, avpkt->data, avpkt->size); if (bytestream2_get_bytes_left(&s->g) < SGI_HEADER_SIZE) { @@ -258,14 +259,14 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *p, p->key_frame = 1; out_buf = p->data[0]; - out_end = out_buf + p->linesize[0] * s->height; + last_line = out_buf + p->linesize[0] * (s->height - 1); s->linesize = p->linesize[0]; /* Skip header. */ bytestream2_seek(&s->g, SGI_HEADER_SIZE, SEEK_SET); if (rle) { - ret = read_rle_sgi(out_end, s); + ret = read_rle_sgi(last_line, s); } else { ret = read_uncompressed_sgi(out_buf, s); } From patchwork Fri Sep 30 17:05:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 38482 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b1c:b0:96:9ee8:5cfd with SMTP id c28csp1839652pzh; Fri, 30 Sep 2022 10:06:05 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7L3zjVI6Th6eZ5pL6TXcXOV9UmT9ZBWs3FxImf965Bh8qWXTNdazRlog1TBLdKvnEDe4xF X-Received: by 2002:a05:6402:538f:b0:444:c17b:1665 with SMTP id ew15-20020a056402538f00b00444c17b1665mr8713018edb.98.1664557565208; Fri, 30 Sep 2022 10:06:05 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id k17-20020a1709060cb100b007316843d58bsi1758387ejh.925.2022.09.30.10.06.04; Fri, 30 Sep 2022 10:06:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b="TL/3xzZZ"; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 685C268BBF0; Fri, 30 Sep 2022 20:05:58 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03olkn2035.outbound.protection.outlook.com [40.92.58.35]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 28A0268BA02 for ; Fri, 30 Sep 2022 20:05:52 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AX0Gqs7tLrKIfl01QDWWTEzVAuvdCbatVzq8aPBuA9N+i9DoAlgFSRJoIpVYNr6As7MWenazqtPyj4n3a38lTIMMKD0RgkrMkggJcYLnDbNiUUvP9ipkqO3EOypHhhYTzW0FbSUD/bEUy7B6hsNC8LB7npT8qeYfneDIKVDtQbkAfYJ69bEV8w/6sAH3aHKICv0uhRXAFAtRUZ3B6lD+cYFrPOuwzMWrNA718Gm5aW8xDmhF5Zs/UvjHpdd7zqZARz5r4DgkhlGhygiVXRoa1q+VY4NfHy5BRvBKZXWKKu8IThgVJtWd2ICgnvTBoguIQn03KZeC5mMJTakZ/RaMHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ooabRpLAZ9079T3bPi8aA1KilBEljaU/khs1lq1gxds=; b=W29k1/UliDq2LrhcDaomaiuzCti7iUgKWd0pnoVPK8IgE1V5yDVNu0zN3s0Ccs1Ey0m/4WiWv0SHPorMgK7xVKQi55WfY225HBFS9K7CGw1YFhcjl3GX6etzMwgS1qjsXF6scrp5gKb0dWOHMp5oxFHMlspX25bAIyF3D/hNg9fZeCbldAjdEIeJadJLY+4JLZ5SPG+bIiQhF5m3Y+NLLj3W5bHf+9wWCCfcONrLXR6p84DGl72yK1o3sxCMyoqAjVi0oPVnP4A9UH98+GZqqn1AcTy+b29WqEbbUhL5Uj8Fe18b7KPWDGWafhdL5HHsI7zAgv5dCaAj2Gu8Z/HqbQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ooabRpLAZ9079T3bPi8aA1KilBEljaU/khs1lq1gxds=; b=TL/3xzZZpHL4nlDWcPEd1DUeTeHSnyh8wEb3/U6JSXh3KAvqL/3MCUycsqCAGHq5zV6DJS1xrvSZaxETckohmZZg4fiO/UFn0BMdbhaZziCnMxCxc5QEWWB35ykJOsipIbMe81JiawHlWo5SA22Ud3h86KYbrqtxsTZ/WkXTlfU30QQ7uF9iI8IVJZf1ZI+XE0TcZLcPV/Aww+KKNUXa5C99rw4EbjAs2ADj+uwcsQbRcVmlaGySY/zeE6t+Y1vEA5FzDkIfl3GXxRV9+Z5gkh1otv3gs+qrlnZrNEzJtGvQqqSXybYgdZiGqVD2aBxziMOywppJxsqnE2Widm03Zg== Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) by DU2P250MB0175.EURP250.PROD.OUTLOOK.COM (2603:10a6:10:274::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.23; Fri, 30 Sep 2022 17:05:45 +0000 Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::68bd:2fc7:ac52:38f8]) by GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::68bd:2fc7:ac52:38f8%9]) with mapi id 15.20.5654.025; Fri, 30 Sep 2022 17:05:45 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 30 Sep 2022 19:05:12 +0200 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: X-TMN: [QT7znFn8a8eUInof+hE9RTO0+zWZ2qYR] X-ClientProxiedBy: FR3P281CA0040.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::8) To GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) X-Microsoft-Original-Message-ID: <20220930170515.3648239-3-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: GV1P250MB0737:EE_|DU2P250MB0175:EE_ X-MS-Office365-Filtering-Correlation-Id: b9f175da-e6f2-44d6-a909-08daa30603a1 X-MS-Exchange-SLBlob-MailProps: feAVlmA1hHWt4DLVZWJFH70LntyHTKMV6NQiUfVccTvF4F1I8Bor5HjCZrOj6TPvAYkF/UAv0QU/xbI9UevWwQFgRMrGUC1lYkAwmHZA3A2+7P+u5DWT0SHbH0vML3L097zUvr5hw7TUcjZm7UibhySzKYvaOQ+i12WturvK4+4kM6wSmM/KDG8AJ7T9/paYh4EnSLyQq3L+Tt4kNQuiJUeZOP43u3MIvvYuDocea9a19nx9r5mFTLEhIl1laHEQkmglsdo4PUtimpEppgZIK58GnZ0ZS8mdPCERWk71Llh+PZTRlX1BLZOa8tgDBvK7roDwZicgYPK2u1o9MEBrbefaxKgT6c/ymgNoIyErLfbdh4NYA7NMAWNmz0wV10lj3U7wQTadl0Cbu/5/k+nG5LuObUTzHX8ONA0CNhtmXtID5ibbpBl93e+WRhuVadJvEKFojqohWN1SkTmkPvh7X0ZIppRDJYuBH/Ec8Mhl8dPu89CZRUTjILoSwS51EFocAcKcCxqGHTMSrxfgPdOu3mdi7BlyOMusnnfEH1VJoKWSuPTi4ul5Lu10iSR6V0xoYAnCGb/XGOjF/fhX//MmBD50voEUY/LGKAa9uVjz93AlKlDzPDedMWcRKONa7/KA5K26wIgT2c1RosjNfvamtiljrZmvIuO5OLfVdebk7HhVyedqFv+BAPUyhD6hsKTbDM66Mvm+moByrkcFX2KRnA== X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VKzf+PtRaoesIRIF5krmrXEvvifwYeXFGQwk3kjkUbTtBPK0+mEPqfhP526hdCKfDawUKi5supp/SiEah1/t6L2HSNG+DyRHTjgUC5xlT0mAuNCqmSLLC1w0aH1o5L4c3cpCqZQqT/wV7fT8ChA0IYVtg7P3+N6aQIOJIuGNZRsCVynpnXONBXNEMudkPWtygzcRX+sl/kHWQIZV4kUPje/5bEMiuRPuniQfAMKd8lOdborlofOpdSMCUsQjQdPaeeP8xgEM3VrScMkUKNSv2jFPRWh/ar3mpTDIt2jHnQgNl5vlCeNFJDwJ/kCc9oF25i24V4/qJOLMKPrKgCTrb7PqOKs5UVMNxf4P7S7U+3pI0etGWaBj56ZKnFU4XDCKBZFELxRaGhKrmGpOF62vy9T9XlfwkvemXdoUtWh+9XOjr5tIKWebboyXZiSc6Gd2qh0zDtTqmf0ieAuTheAhN23x82tBYDZ0TgmlqZwvNa2D7eV82hST4qGnkULaNsoTssYPzPp5HxJ7dKRhPmjfCPTFzFIn+1SXk+5Y8cs2INXwOat+73s1YNjow4QAcs57o+sk8m8leNPQa1OlMhhSaDPFH9OZ9nqwTl6Cr+6IQC5HN9Rvb+qUhOJZpeYT1HFN7koeFh1w1q44r4mfECimCQ== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: mtE+VL5Yf/nB2+cpJ9zOAi8sXjYyP8X6z6KAMbRswtuy36y+/jPW9rA/2LafY63OSwWgTgM/YQjV7qYsYx/Uatm76r3kkqu9Nm3muY+P+Wl9pKkApV+FMip2wHL+FwOp2m5OdDEHq4IlP2JmhzdXcBwkPE7gqkdVnBsFDdsDIco/QxbM6qdyyn2hnrk5GMkl83eAiNtTX9ny4rViUSgyrAvHgCVgd37HdvKCs58jNfrepc6OKiGloC6YJz8oOtDcA9N56HQ8+Nawitn9Bygn30KvGlF8MpBcbQyk2cfqKY/Sv7M/NJ7pWW3wkIHJ6y0ECYKvuUwLYttLCE4csaMxLbzZJmLJ9Ng0a4aN/y5J0B6Grc/zC0TAo682LMFMHfw5A2aZq4XhnKvGTjPQP0fu2is+LYnKwuovt2w7afV5dDW3+kQzvDVWuKS6isWA4xoEsEmbfiwMPPOcQjF7znVMhTro1a3Nal/xGKFhOXAgswQoy1GIOPNrzrPBaQIC+YgSrT64YcZJycpdNeRD59YE7IJ+5D3ZhUUF1IXcdWcoj9YjxSA3Osrn0a64/In0UBtl04jnqhH6pGxLrS5nzOXWA109TKL3IAvbAezAjvUFSWHPfCmS3VtXt4yaWVBJE7Ga64EszcZwYCOXN+MGjn+FbHj63eHmWjiuq/k2tcAcN2PL6rNLUoVLI5iyx7G2V0gH7mrAxvChiFS1xWb2xKblQmW4urIFAZakxWoxjTaClHKNsQlm0eQ6XR5weaFDAn5qQZiCdoEMQ6WPDrv3Y0P1htN1qSUUvcqL+MYNf8Y7JoNt48Ig6QdOlw2OrviG+2/XUPB7AD4NgWn+Wwgx+zyX+zo8Baf0UK/rgHXlnHpyrlnbihCAQmpkCR1RkPQ4x8KrvmZ8r6LDssAL1ERjQ7eEQscI/GuRglAPl22ePtqQzFlRaa6MRnqW9GWQTC317dODlgkArt6XwFzXmhHOOQ2+jTJz6WSz5eAUT2/QrErAj9ZJr1lsxA3tqoaQI+hNaLN1fgXIdXXeIdroHtRiFZfsK9jItLfxEjVFEpnWU2TUpHL49+H5LHYeH33iM3OyLaoO9TUGa5HAGvkgqoQgwR5E8xz2yJJzB2C7IruIGyZ3JThrOtVm7XhsQuaEM4zfTXYMTEamlJTJe7qSux9I8fvyOfggV6fgJhsQjdu3HomoS6D0soUkzLd8pPsG2FVBVt4ECsgNz0Jkv83YMQAmoJaY7G+YF0cFRVIxjaOG1zkG7W0xd3CGtE9k9jtcAX5uYnq3/RgmuhAPD01McJXHfHI/f9yinUEF4uU3vs/6cR47yZc= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: b9f175da-e6f2-44d6-a909-08daa30603a1 X-MS-Exchange-CrossTenant-AuthSource: GV1P250MB0737.EURP250.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2022 17:05:45.4446 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2P250MB0175 Subject: [FFmpeg-devel] [PATCH 4/7] avcodec/sgidec: Use planar pixel formats X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: PiIRAxOWc1Jk The data in SGI images is stored planar, so exporting it via planar pixel formats is natural. Signed-off-by: Andreas Rheinhardt --- libavcodec/sgidec.c | 123 ++++++++++++++++------------------ tests/ref/fate/sgi-rgb24 | 2 +- tests/ref/fate/sgi-rgb24-rle | 2 +- tests/ref/fate/sgi-rgb48 | 2 +- tests/ref/fate/sgi-rgb48-rle | 2 +- tests/ref/fate/sgi-rgba | 2 +- tests/ref/fate/sgi-rgba-rle | 2 +- tests/ref/fate/sgi-rgba64 | 2 +- tests/ref/fate/sgi-rgba64-rle | 2 +- tests/ref/lavf/sgi | 2 +- 10 files changed, 66 insertions(+), 75 deletions(-) diff --git a/libavcodec/sgidec.c b/libavcodec/sgidec.c index bd49a3510d..6ff2ee97f6 100644 --- a/libavcodec/sgidec.c +++ b/libavcodec/sgidec.c @@ -30,17 +30,15 @@ * @param logctx a logcontext * @param out_buf Points to one line after the output buffer. * @param g GetByteContext used to read input from - * @param len length of out_buf in bytes - * @param pixelstride pixel stride of input buffer - * @return size of output in bytes, else return error code. + * @param width length of out_buf in nb of elements + * @return nb of elements written, else return error code. */ static int expand_rle_row8(void *logctx, uint8_t *out_buf, - GetByteContext *g, - int len, int pixelstride) + GetByteContext *g, unsigned width) { unsigned char pixel, count; unsigned char *orig = out_buf; - uint8_t *out_end = out_buf + len; + uint8_t *out_end = out_buf + width; while (out_buf < out_end) { if (bytestream2_get_bytes_left(g) < 1) @@ -51,36 +49,31 @@ static int expand_rle_row8(void *logctx, uint8_t *out_buf, } /* Check for buffer overflow. */ - if (out_end - out_buf <= pixelstride * (count - 1)) { + if (out_end - out_buf < count) { av_log(logctx, AV_LOG_ERROR, "Invalid pixel count.\n"); return AVERROR_INVALIDDATA; } if (pixel & 0x80) { - while (count--) { - *out_buf = bytestream2_get_byte(g); - out_buf += pixelstride; - } + while (count--) + *out_buf++ = bytestream2_get_byte(g); } else { pixel = bytestream2_get_byte(g); - while (count--) { - *out_buf = pixel; - out_buf += pixelstride; - } + while (count--) + *out_buf++ = pixel; } } - return (out_buf - orig) / pixelstride; + return out_buf - orig; } static int expand_rle_row16(void *logctx, uint16_t *out_buf, - GetByteContext *g, - int len, int pixelstride) + GetByteContext *g, unsigned width) { unsigned short pixel; unsigned char count; unsigned short *orig = out_buf; - uint16_t *out_end = out_buf + len; + uint16_t *out_end = out_buf + width; while (out_buf < out_end) { if (bytestream2_get_bytes_left(g) < 2) @@ -90,7 +83,7 @@ static int expand_rle_row16(void *logctx, uint16_t *out_buf, break; /* Check for buffer overflow. */ - if (out_end - out_buf <= pixelstride * (count - 1)) { + if (out_end - out_buf < count) { av_log(logctx, AV_LOG_ERROR, "Invalid pixel count.\n"); return AVERROR_INVALIDDATA; } @@ -99,18 +92,18 @@ static int expand_rle_row16(void *logctx, uint16_t *out_buf, while (count--) { pixel = bytestream2_get_ne16(g); AV_WN16A(out_buf, pixel); - out_buf += pixelstride; + out_buf++; } } else { pixel = bytestream2_get_ne16(g); while (count--) { AV_WN16A(out_buf, pixel); - out_buf += pixelstride; + out_buf++; } } } - return (out_buf - orig) / pixelstride; + return out_buf - orig; } @@ -120,15 +113,14 @@ static int expand_rle_row16(void *logctx, uint16_t *out_buf, * @param s the current image state * @return 0 if no error, else return error code. */ -static int read_rle_sgi(void *logctx, uint8_t *last_line, GetByteContext *g, - ptrdiff_t stride, unsigned width, unsigned height, +static int read_rle_sgi(void *logctx, uint8_t *out[4], ptrdiff_t stride[4], + GetByteContext *g, unsigned width, int height, unsigned nb_components, unsigned bytes_per_channel) { - uint8_t *dest_row; unsigned int len = height * nb_components * 4; GetByteContext g_table = *g; unsigned int start_offset; - int linesize, ret; + int ret; /* size of RLE offset and length tables */ if (len * 2 > bytestream2_get_bytes_left(g)) { @@ -136,22 +128,19 @@ static int read_rle_sgi(void *logctx, uint8_t *last_line, GetByteContext *g, } for (unsigned z = 0; z < nb_components; z++) { - dest_row = last_line; - for (unsigned remaining_lines = height;;) { - linesize = width * nb_components; + uint8_t *dest_row = out[z] + (height - 1) * stride[z]; + while (1) { start_offset = bytestream2_get_be32(&g_table); bytestream2_seek(g, start_offset, SEEK_SET); if (bytes_per_channel == 1) - ret = expand_rle_row8(logctx, dest_row + z, g, - linesize, nb_components); + ret = expand_rle_row8(logctx, dest_row, g, width); else - ret = expand_rle_row16(logctx, (uint16_t *)dest_row + z, g, - linesize, nb_components); + ret = expand_rle_row16(logctx, (uint16_t *)dest_row, g, width); if (ret != width) return AVERROR_INVALIDDATA; - if (--remaining_lines == 0) + if (dest_row == out[z]) break; - dest_row -= stride; + dest_row -= stride[z]; } } return 0; @@ -163,35 +152,23 @@ static int read_rle_sgi(void *logctx, uint8_t *last_line, GetByteContext *g, * @param s the current image state * @return 0 if read success, else return error code. */ -static int read_uncompressed_sgi(unsigned char *out_buf, GetByteContext *g, - ptrdiff_t stride, unsigned width, unsigned height, +static int read_uncompressed_sgi(uint8_t *const out[4], const ptrdiff_t stride[4], + GetByteContext *g, unsigned width, int height, unsigned nb_components, unsigned bytes_per_channel) { - unsigned int offset = height * width * bytes_per_channel; - GetByteContext gp[4]; - uint8_t *out_end; + unsigned rowsize = width * bytes_per_channel; /* Test buffer size. */ - if (offset * nb_components > bytestream2_get_bytes_left(g)) + if (rowsize * (int64_t)height > bytestream2_get_bytes_left(g)) return AVERROR_INVALIDDATA; - /* Create a reader for each plane */ for (unsigned z = 0; z < nb_components; z++) { - gp[z] = *g; - bytestream2_skip(&gp[z], z * offset); - } - - for (int y = height - 1; y >= 0; y--) { - out_end = out_buf + y * stride; - if (bytes_per_channel == 1) { - for (unsigned x = width; x > 0; x--) - for (unsigned z = 0; z < nb_components; z++) - *out_end++ = bytestream2_get_byteu(&gp[z]); - } else { - uint16_t *out16 = (uint16_t *)out_end; - for (unsigned x = width; x > 0; x--) - for (unsigned z = 0; z < nb_components; z++) - *out16++ = bytestream2_get_ne16u(&gp[z]); + uint8_t *cur_row = out[z] + (height - 1) * stride[z]; + while (1) { + bytestream2_get_bufferu(g, cur_row, rowsize); + if (cur_row == out[z]) + break; + cur_row -= stride[z]; } } return 0; @@ -202,9 +179,10 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *p, { GetByteContext g; unsigned int bytes_per_channel, nb_components, dimension, rle, width; + uint8_t *out[4]; + ptrdiff_t linesize[4]; int height; int ret = 0; - uint8_t *out_buf, *last_line; bytestream2_init(&g, avpkt->data, avpkt->size); if (bytestream2_get_bytes_left(&g) < SGI_HEADER_SIZE) { @@ -239,9 +217,9 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *p, if (nb_components == SGI_GRAYSCALE) { avctx->pix_fmt = bytes_per_channel == 2 ? AV_PIX_FMT_GRAY16BE : AV_PIX_FMT_GRAY8; } else if (nb_components == SGI_RGB) { - avctx->pix_fmt = bytes_per_channel == 2 ? AV_PIX_FMT_RGB48BE : AV_PIX_FMT_RGB24; + avctx->pix_fmt = bytes_per_channel == 2 ? AV_PIX_FMT_GBRP16BE : AV_PIX_FMT_GBRP; } else if (nb_components == SGI_RGBA) { - avctx->pix_fmt = bytes_per_channel == 2 ? AV_PIX_FMT_RGBA64BE : AV_PIX_FMT_RGBA; + avctx->pix_fmt = bytes_per_channel == 2 ? AV_PIX_FMT_GBRAP16BE : AV_PIX_FMT_GBRAP; } else { av_log(avctx, AV_LOG_ERROR, "wrong picture format\n"); return AVERROR_INVALIDDATA; @@ -254,19 +232,32 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *p, if ((ret = ff_get_buffer(avctx, p, 0)) < 0) return ret; + switch (nb_components) { +#define MAP(in_idx, out_idx) \ + out[(in_idx)] = p->data[(out_idx)]; \ + linesize[(in_idx)] = p->linesize[(out_idx)] + case SGI_GRAYSCALE: + MAP(0, 0); + break; + case SGI_RGBA: + MAP(3, 3); + /* fallthrough */ + case SGI_RGB: + MAP(0, 2); + MAP(1, 0); + MAP(2, 1); + break; + } p->pict_type = AV_PICTURE_TYPE_I; p->key_frame = 1; - out_buf = p->data[0]; - - last_line = out_buf + p->linesize[0] * (height - 1); /* Skip header. */ bytestream2_seek(&g, SGI_HEADER_SIZE, SEEK_SET); if (rle) { - ret = read_rle_sgi(avctx, last_line, &g, p->linesize[0], + ret = read_rle_sgi(avctx, out, linesize, &g, width, height, nb_components, bytes_per_channel); } else { - ret = read_uncompressed_sgi(out_buf, &g, p->linesize[0], + ret = read_uncompressed_sgi(out, linesize, &g, width, height, nb_components, bytes_per_channel); } if (ret) diff --git a/tests/ref/fate/sgi-rgb24 b/tests/ref/fate/sgi-rgb24 index 4326cabe00..238e3a1f26 100644 --- a/tests/ref/fate/sgi-rgb24 +++ b/tests/ref/fate/sgi-rgb24 @@ -3,4 +3,4 @@ #codec_id 0: rawvideo #dimensions 0: 512x256 #sar 0: 0/1 -0, 0, 0, 1, 393216, 0xa9b28fd9 +0, 0, 0, 1, 393216, 0xc8478fd9 diff --git a/tests/ref/fate/sgi-rgb24-rle b/tests/ref/fate/sgi-rgb24-rle index d21bde15ba..25cc5f4737 100644 --- a/tests/ref/fate/sgi-rgb24-rle +++ b/tests/ref/fate/sgi-rgb24-rle @@ -3,4 +3,4 @@ #codec_id 0: rawvideo #dimensions 0: 512x256 #sar 0: 0/1 -0, 0, 0, 1, 393216, 0xe96e1de2 +0, 0, 0, 1, 393216, 0x7e231de2 diff --git a/tests/ref/fate/sgi-rgb48 b/tests/ref/fate/sgi-rgb48 index 29fe302514..3c59c782e5 100644 --- a/tests/ref/fate/sgi-rgb48 +++ b/tests/ref/fate/sgi-rgb48 @@ -3,4 +3,4 @@ #codec_id 0: rawvideo #dimensions 0: 512x256 #sar 0: 0/1 -0, 0, 0, 1, 786432, 0xee4aa667 +0, 0, 0, 1, 786432, 0xf9a2a667 diff --git a/tests/ref/fate/sgi-rgb48-rle b/tests/ref/fate/sgi-rgb48-rle index 49fc973017..377f4577f1 100644 --- a/tests/ref/fate/sgi-rgb48-rle +++ b/tests/ref/fate/sgi-rgb48-rle @@ -3,4 +3,4 @@ #codec_id 0: rawvideo #dimensions 0: 512x256 #sar 0: 0/1 -0, 0, 0, 1, 786432, 0xbc743bc4 +0, 0, 0, 1, 786432, 0xec343bc4 diff --git a/tests/ref/fate/sgi-rgba b/tests/ref/fate/sgi-rgba index 6a2d176582..dcdaf84ab2 100644 --- a/tests/ref/fate/sgi-rgba +++ b/tests/ref/fate/sgi-rgba @@ -3,4 +3,4 @@ #codec_id 0: rawvideo #dimensions 0: 512x256 #sar 0: 0/1 -0, 0, 0, 1, 524288, 0x4ee5adbb +0, 0, 0, 1, 524288, 0x7401adbb diff --git a/tests/ref/fate/sgi-rgba-rle b/tests/ref/fate/sgi-rgba-rle index 6a2d176582..dcdaf84ab2 100644 --- a/tests/ref/fate/sgi-rgba-rle +++ b/tests/ref/fate/sgi-rgba-rle @@ -3,4 +3,4 @@ #codec_id 0: rawvideo #dimensions 0: 512x256 #sar 0: 0/1 -0, 0, 0, 1, 524288, 0x4ee5adbb +0, 0, 0, 1, 524288, 0x7401adbb diff --git a/tests/ref/fate/sgi-rgba64 b/tests/ref/fate/sgi-rgba64 index 00181dcb3b..4a28a29169 100644 --- a/tests/ref/fate/sgi-rgba64 +++ b/tests/ref/fate/sgi-rgba64 @@ -3,4 +3,4 @@ #codec_id 0: rawvideo #dimensions 0: 512x256 #sar 0: 0/1 -0, 0, 0, 1, 1048576, 0xc657e22b +0, 0, 0, 1, 1048576, 0x1b61e22b diff --git a/tests/ref/fate/sgi-rgba64-rle b/tests/ref/fate/sgi-rgba64-rle index 354d391826..5610fc7fe6 100644 --- a/tests/ref/fate/sgi-rgba64-rle +++ b/tests/ref/fate/sgi-rgba64-rle @@ -3,4 +3,4 @@ #codec_id 0: rawvideo #dimensions 0: 512x256 #sar 0: 0/1 -0, 0, 0, 1, 1048576, 0xb619d0f1 +0, 0, 0, 1, 1048576, 0x6122d0f1 diff --git a/tests/ref/lavf/sgi b/tests/ref/lavf/sgi index ad27b805f0..bfab92c8a8 100644 --- a/tests/ref/lavf/sgi +++ b/tests/ref/lavf/sgi @@ -1,3 +1,3 @@ d446e540a7c18da5fd3cc0e9942cd46f *tests/data/images/sgi/02.sgi 307287 tests/data/images/sgi/02.sgi -tests/data/images/sgi/%02d.sgi CRC=0x6da01946 +tests/data/images/sgi/%02d.sgi CRC=0x36e71946 From patchwork Fri Sep 30 17:05:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 38485 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b1c:b0:96:9ee8:5cfd with SMTP id c28csp1840060pzh; Fri, 30 Sep 2022 10:06:38 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7qLdD9qlJ/p7Q6HMK6KObkUm3iXSFcAqHnj+pyPztgV/QsTRkbI7Q/zbUjpAYHHVpoh3sw X-Received: by 2002:a17:907:a0c7:b0:787:f13b:5533 with SMTP id hw7-20020a170907a0c700b00787f13b5533mr4247767ejc.50.1664557598477; Fri, 30 Sep 2022 10:06:38 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id o7-20020a056402038700b00458250c3da7si1102447edv.588.2022.09.30.10.06.14; Fri, 30 Sep 2022 10:06:38 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=mO3cUIve; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8372068BBFB; Fri, 30 Sep 2022 20:06:03 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03olkn2035.outbound.protection.outlook.com [40.92.58.35]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 42E9E68BBEC for ; Fri, 30 Sep 2022 20:05:57 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eiwRj8V6GvQ1WGmuPJoqKbV2uDGhSuCmlH8GxTIMJdag/MrUhWJvJwCB2l8DpvZirzGT7thV9NkMxb+hbjvECrNPHsT51ht82sepko3LN9KVR5O2R4WuzN01qTZBb7A5pOg6w/16dpfLzZ1I6m5HEE6EMJ1b/kBmCMxrrhHPWqadHSf1EHCW1XIjR1mXQuFEqOJSqwVOMPPfzzyhoQpWykJfVbHhgFPw79xeHANWG16YZZJ/Wjr63VSQSPhkpIapVhYNRSgiKA8MwAHjLtj95Xr5jSKj1US23ERI/b/lZRXw4G2Yp1FwYRvjygY2n7tqMudX74sh+aE2ty7V6e9o0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0dbA8Z1CyO/Q7j2K7Wq3yeH3+Ak1iazz4eX1zk9khxU=; b=DxkbDFF+igJBLykPU8qkMhBcq7srEeNCzsI14LmFpUCwR0UQlt916gZ/5wGIssiupkaRgEpyAUrsF2r8OZRkXPGANmf/YMQKW/w5xn4f7Z0A7JTsW2tvN4H6xhG4L7wUeurrmy3f3e0/2MEXRikuMoW9GNXfOBM+sJ9iNeqUDEngWzAVgxv7NrqmtpnZBoiajbHu0642jXLr+MBx8xVifJlzecf9fZXPDzcb7hhxlTQRu9iYFbYP9B2LNpdRo9yUY2j4shVEssw0binvvtukejwVYZg/RKdRlvcR2FhupZyEnci80XGX1yXt5Y3QSn7aKkeDmDrUgl4TFGy8QZvI4A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0dbA8Z1CyO/Q7j2K7Wq3yeH3+Ak1iazz4eX1zk9khxU=; b=mO3cUIve0QNV+4oA3nY+q7S0bU/QTlO2OjukMB0iJd13CTbnWRcX5fSrPh9qwlj1IwLn+Rmz66LTG2zSaMwqDbertaWh7apvg5XjOYvULLQLRoqPZbUnV1Uk71GN+RP9kWmIQtfIZiXs/uLcMEqgK4lrqkPUaIBzoWSK9uAotF9cickOVQUGGUsA6lXItWJ76V4JvHLnlF8bu3F3zANag4GWt91EUlgvW1m6fOi9faLGKf5tgnYD2akV2+mhrTZ6C3ycVBsXI6sTvYtmm7dkUF4jTxWq00hXHFluknALKTJOqOHxZq4RVJu5I8xOXQamA2M3mfrFe2XT4MWDK5ZWFw== Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) by DU2P250MB0175.EURP250.PROD.OUTLOOK.COM (2603:10a6:10:274::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.23; Fri, 30 Sep 2022 17:05:47 +0000 Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::68bd:2fc7:ac52:38f8]) by GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::68bd:2fc7:ac52:38f8%9]) with mapi id 15.20.5654.025; Fri, 30 Sep 2022 17:05:47 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 30 Sep 2022 19:05:13 +0200 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: X-TMN: [Cm/uBlAhZxQtEXAv/2Py+DyvDvKa6CXi] X-ClientProxiedBy: FR3P281CA0040.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::8) To GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) X-Microsoft-Original-Message-ID: <20220930170515.3648239-4-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: GV1P250MB0737:EE_|DU2P250MB0175:EE_ X-MS-Office365-Filtering-Correlation-Id: 9b0e97f5-e2ae-4a1b-5545-08daa3060427 X-MS-Exchange-SLBlob-MailProps: AZnQBsB9XmrST/aUJU0/QUaXtXS9lKwztdLlpkzskV6QyBdAWfeLkSDgKau/Gcc3gnNv6zg2YXuZyORcxrgU6E3W7tbVHGupbIpvDAH/y/QOUJMvkFWHE4O69zD409+KCMx4kCmXXN4QfZHy9tLilJOCBktA4gMak/51WGvjzk9OSJXCEOOV5ucC5lSSCDNGsoK8LpJbarZL5RHTlrNLmkdw/7TXwL3o/vOripLVVLIY/CNpmrK3UKc8yKEPcWhtOmiZfmcT/iAtuqx9+UjZOdpB9a84yh6xiQM9dNEOnx3MwwsF//PDZyXv89VJQ0OquHLRXtYbeS2aq/WY47vYR1VSCLz8yAjW+15yy4zYc6u01x77+M1WFNb/x9mvTrnYFeWznWBg06p15i2XjMyAAi0h6GhCYQtHBxQZA+h65ZiU6pc5D2p1My068UiD32RpMgzU9lpiXgdHaEDtmL/YjDIsZieASjP3SEEzDsb5p984tRVHtj+iTEVHCZJEjrwiwW4T0+Se/VT688Ynzdlqh+jWwUXf7OMMllTJlZ1w5yHDDLRq/dN+QtmHEE7pAY78Tn7/Fp/3Vq68ELwkcuZCHfNhgsUae4gTVwcfTL08hM6xugPa6ZfvKZ9YtbsNBEE8yfve/RouK63xgSi9AYzyhH6fgALgl8RMhcoGSAD195zWVcSboyKLmWgkZVbOAYBNMnAY0C1ksgfpUPqDkwLNL5nAIqJmUXktGwinq+6/CR91NjKOXLwI4Eg9pH1D8wSeKOgx1O6mor8= X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: U65lwUdJjJRgd/LNx2P7Bo3BFqQtwcglkj53zb00MDGJYWkadSSbdWh/o8/t1mWWh11rPwloCpVFkh332CKVfGlYjwGQrGmowUltPuWqVd4HmxHHLcBVEu9P0W5DxHMduaNBAuABJL222XO3XwilBP+WWUFXOXH41Sh6xoC3keauhW/BxyQmAAHRBrReqnnsnwVE2T02CiHbyfF60XvNgRxudEeK1eYZrBSz9Oe+8Sdng0tFVw8VJtUrux0gY+X3kaXdhAkRUFHMmQPmSPlDQC87T2qRWzDZdJ8aZKHC+ZuhutjEP6r8gIWJh4dv8jbstI8nyu68b+v24plo1s3YJ4eveKnoZOU8dGQsyBVAY5xiFVVzQlca9/C6lMVyFUZE1+5YG6IvlEfMo1fRlO8Z19bZeRpn0h0YTZHFgDZ7Z+8qBOHSQgpcDY5sYjWMlYKw4xQeE6GbABTCIIueYLwQRqQjrM8w3iipR+IwlJFjl1i6zgC3dkaYi6WE9BLIGlyRABaEKafCjfGtzOeRDXyBZDqdySaaw+6yXCnTVUW/xOxkoIUIwjawyCUw1tNhUGXTfiWbEBBTgUyzS0NMbhkbE8ietj5/+tu9gm5Ex9z3TzEoWlggv8T4UgpcrMFkMKFAZBlnApxxja21llaZeoIwMg== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: dAHKrzWfFQq+HGZcjeoylOqgEcqzy4gUDqmu9RD8ecEn7pyscMur+FuxN3dplKDRIBJyLwcN5uLxY7tDAH30j92CmsFIhPJXAjMdD1fzI+SGAOXVRxNdIteX72EK2rJ2J0FUy1nx7K7Ystn6OErCCLOrMH0bNut1HFG7UZNZetsv3xSnhH6IO0tbrmo9PiycWnd4qeVSRkFLrQ35gYtj91iay0LfIKFKOGmay7xobBSSSfCZbKpXDd7i1oXC7x32PJP3leqgonbzHZh0lVyBKQopRcGigr66clL8OZkPg5Tr8zqoVAZdrlzFe9u5zatd0krfYpoprSkvXYbdWBCv2TeaBE8P0vGrzxGrYPX+dgyqE4af75mn1GqYx/RFGlLsujdjVwo1pbbQ+aRtQho4VywR2sgzOJlIu48a2qQ9z+bYHr/0pBMuLRNCqmuKikJvRRHptq+L/+lWmUl/6blmb2/iKWpVtykbjpcw2TdrQda2q7ZF9WFBnq5YWNrSgjgnLfOUw/7332NEUbFJ1d2l0Ur9N17UKibmHsCeXbzU1s5LJOeYE3JNDWqDh1qYNQlShIoc3RxD3f6SZm/l3sAP+90b2wpdmWG1S8AHKekLSsbeTvFSaD3eQpVkAXixY+wroh46etR349FyyGPOiljIqEu6Qkht0kqJ4GtEFQQkuzag8YM7OlfLm6Y0AvtIHtkemydtKqj+QqlwFVrOoMtCU+wigsKYJjRptBmD6juxzCXrckqE6D8ekCqY/AEdGiubYM0DYQfKH2Nnqdry2iiXFq8dLzh0MuSWVnUkYYRgQKU1be18w/jKKE86EDDwRHSv4wZ2rSbgn6MJ3qviBgq4+fLjrXhHUZC+1A5UYDwCKeCJSu0ThgQo2XNzkguN+jK0YjTKjczYE77dbCI3vjtPGLKCeZPQ+byFqRDx7M4nB+FNbPPsTD9Pk824YEo+AaTJYlDG0fPaNlGhRPAcU0HaJQruyef9+XPQ8QMS44+KEPk3vRDYHSWx5xkkDYR67i8nPwBNKnIEo2YIFWPXFkXxn6tuFSCh8B5mU9JzZFld5TzDQeXMSK+Qdl9Bjbv5MiGS0zNK30DfxenEVs5e2BnWgumJrQ8flHXZPqzukwNXRqpbFl0S0s1iEL3MdGm5hMi7WmOhNHJeJWw00ty7eU9DDGN8ku/UVELqTn16RMkaMCDZBmI/M2oV+Z1DZ6mNzzJZcScT4o+ro1YyvnEZB2mM4OpkfjPyqaPly1YGCF/tfOEZvo9HaRQmWPJ20Jwq+CqggMh01xruX3z3+i/katxS9IdT6DuBcDq8K4Vcy3csxDw= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9b0e97f5-e2ae-4a1b-5545-08daa3060427 X-MS-Exchange-CrossTenant-AuthSource: GV1P250MB0737.EURP250.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2022 17:05:46.2883 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2P250MB0175 Subject: [FFmpeg-devel] [PATCH 5/7] avcodec/c93: Fix segfault when using negative linesizes X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: p3TSGprEKaLh c93.c used an int for the stride and an unsigned for the current linenumber. This does not work when using negative linesizes. So use ptrdiff_t for stride and int for linenumber. This fixes the cyberia-c93 FATE test when using negative linesizes. Signed-off-by: Andreas Rheinhardt --- libavcodec/c93.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/libavcodec/c93.c b/libavcodec/c93.c index 66b551a5d6..bfcbc7c150 100644 --- a/libavcodec/c93.c +++ b/libavcodec/c93.c @@ -130,7 +130,8 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *rframe, AVFrame * const oldpic = c93->pictures[c93->currentpic^1]; GetByteContext gb; uint8_t *out; - int stride, ret, i, x, y, b, bt = 0; + int ret, i, x, y, b, bt = 0; + ptrdiff_t stride; if ((ret = ff_set_dimensions(avctx, WIDTH, HEIGHT)) < 0) return ret; @@ -156,7 +157,6 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *rframe, out = newpic->data[0] + y * stride; for (x = 0; x < WIDTH; x += 8) { uint8_t *copy_from = oldpic->data[0]; - unsigned int offset, j; uint8_t cols[4], grps[4]; C93BlockType block_type; @@ -165,16 +165,17 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *rframe, block_type= bt & 0x0F; switch (block_type) { - case C93_8X8_FROM_PREV: - offset = bytestream2_get_le16(&gb); + case C93_8X8_FROM_PREV: { + int offset = bytestream2_get_le16(&gb); if ((ret = copy_block(avctx, out, copy_from, offset, 8, stride)) < 0) return ret; break; + } case C93_4X4_FROM_CURR: copy_from = newpic->data[0]; case C93_4X4_FROM_PREV: - for (j = 0; j < 8; j += 4) { + for (int j = 0; j < 8; j += 4) { for (i = 0; i < 8; i += 4) { int offset = bytestream2_get_le16(&gb); int from_x = offset % WIDTH; @@ -203,7 +204,7 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *rframe, case C93_4X4_2COLOR: case C93_4X4_4COLOR: case C93_4X4_4COLOR_GRP: - for (j = 0; j < 8; j += 4) { + for (int j = 0; j < 8; j += 4) { for (i = 0; i < 8; i += 4) { if (block_type == C93_4X4_2COLOR) { bytestream2_get_buffer(&gb, cols, 2); @@ -226,7 +227,7 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *rframe, break; case C93_8X8_INTRA: - for (j = 0; j < 8; j++) + for (int j = 0; j < 8; j++) bytestream2_get_buffer(&gb, out + j*stride, 8); break; From patchwork Fri Sep 30 17:05:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 38483 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b1c:b0:96:9ee8:5cfd with SMTP id c28csp1839882pzh; Fri, 30 Sep 2022 10:06:24 -0700 (PDT) X-Google-Smtp-Source: AMsMyM41z2+al7RzfExRVvVSBMjKNHJFW6xUdJW3yTz/uXDFGYXj5vY7YO9C1taDUJXARBM1c5ti X-Received: by 2002:a17:906:db02:b0:780:24e:cf9 with SMTP id xj2-20020a170906db0200b00780024e0cf9mr7137109ejb.460.1664557584201; Fri, 30 Sep 2022 10:06:24 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id w15-20020a170906384f00b00718d0985aa0si1758178ejc.247.2022.09.30.10.06.23; Fri, 30 Sep 2022 10:06:24 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=rQCq+2CA; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7A57668BC01; Fri, 30 Sep 2022 20:06:08 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03olkn2035.outbound.protection.outlook.com [40.92.58.35]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5D8F668BBDB for ; Fri, 30 Sep 2022 20:06:02 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mYMLsal7tlG0HwL6LrK4T59ZpUaDjPkDLbf2SNy4GfEp3bx5z2J1Mv/ndWA8koF54iv5ufwHILVmV6bRWgB+LksPthanqOd2Ugup631AszrqNeXIZkiHPTHAv3vmEqhkI6gdOrVfAHaClNG2IMkhnauLHBG3CbPNO60QD0eePwKBXmGkm0EciXVKEvYPn/fXkqK3aE988iwon6FxL/dTfGV26DEHkjZt0oeXJXsJWKNozU6p7jo2aq0+rQNmZfLPwHBm6wgEhXv/f7oN3Lee2tp0cUbOkU8oNb8JNTNDaAGOFT6oTLKo7O797HYJ72SRItGK+jnp92VxVgLhabXy+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=90ZDJ8Ks0MdQIfnXmikWtVcW/N7Gasd9UyVLBmdV+Mo=; b=ClezvI+182KE0I04cfeFMBTwKoQ4B78sZK6WMyE6gfsP7sNUaq3rWlLF1bbJGhKhuQLMkt5YknBizs9aiQ9JY8tENQVvsdRpMpsh6YlN68H/V1oS6d4KPDSmo/OE/2vEep3IqiZcRUov3bQJKuY7AXEPaS5oBAq8CT6jt6a4upzne0o7lEciTGeXrdiKY5s4GuRnsepoZsiy1h4xyPTiRTzmmdK+OVbETshXXDgn4EUOlyOPc5vhQP+HG+VlYKa8x1+q+G0DBF4esHS5Whzn5UyaPBKFFKfGgySjWLNj9buPhUddPqWWdxNSR+CSOy29YKsvEDYLF6J4Co8zV9HZlw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=90ZDJ8Ks0MdQIfnXmikWtVcW/N7Gasd9UyVLBmdV+Mo=; b=rQCq+2CAXNdwWZhndVbr0XTOVkComtt8BEVQf5+sXJjU3tSQi2uDvxO7U+Qb41rZeDtgiiO5YO3lm0BHWt0Lt9Idj691RiytkZ4AWLDHL83wvfm/eMsMQMeYVzG56JrGYJqoQfKYjRrOKAPOKYRpmpIJI6XfxCN7rQDdkt/tVnq4X64eErUhQ2LeFVjfUEvveoqN3PPWMmi7XJ2VXNHPaWGW0TTJcrx0AMYHRpkq1Fu9kxVEQ1efDuWsPIsKGgGnGl+t72dKOMa3eWGMRaD/2XpbpmDeAEVAf7yHEKt8v7PIVTrZpTUTXhblXR92w9BnYMC1PVn332ZM3A2Rac7G/Q== Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) by DU2P250MB0175.EURP250.PROD.OUTLOOK.COM (2603:10a6:10:274::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.23; Fri, 30 Sep 2022 17:05:47 +0000 Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::68bd:2fc7:ac52:38f8]) by GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::68bd:2fc7:ac52:38f8%9]) with mapi id 15.20.5654.025; Fri, 30 Sep 2022 17:05:47 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 30 Sep 2022 19:05:14 +0200 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: X-TMN: [GKLmSkSq0TT08qhs9YtpZAugtV9OhRCx] X-ClientProxiedBy: FR3P281CA0040.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::8) To GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) X-Microsoft-Original-Message-ID: <20220930170515.3648239-5-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: GV1P250MB0737:EE_|DU2P250MB0175:EE_ X-MS-Office365-Filtering-Correlation-Id: 20ad623a-8207-4200-7c95-08daa30604ab X-MS-Exchange-SLBlob-MailProps: AZnQBsB9XmrST/aUJU0/QUaXtXS9lKwzaWl64MCTP4+YBx8yKufmryrhPNGv3KdE4deaS7tW4nquZYSB1hOesOLueFRiykuowY2v0QM15cRJDzJBvIrZW1Ecs/XKF8a5Yzf7Nv+4RZhr0DF5iuBfSC3juTIU/Rl/fcBOxyRV7g4Jk8zl1h7AcZ/rrUtEMKIqpEgoQDdwdsZJa6TRf1xOwzAhKHg8rv4q4fx/qk1mN0Z9/5dDj1Uj/tCsw3W3wKOvjdTSeGHNi1r19eD6hv4bOQf6HI70la46XDNuUajR2rLboFa4S4Kc4Vbvj5PYydNrN7vhEBqF+XhTmhBMz1E5JcZdwc6QpWuf50mSTOGWbj8FqTNuE7aRg/JY3PoEHT5W4gIv9vIlL4+QdW4TPy4impN9W6ow9W9s6LbGjugljX2je+6DjUtDMT4sIuI7jLCi7lAzOVGTE22/zfLViDITM+Va/GzOZgx9yyCInO9ElAYI+DgwlJj8WuuqQ7ZMWjo20uDgHFPy13XegWiw/8zDT9L1rNt75waD3KZ5qvs5hNjWBop0xVHzgngqDgtpFzhW0yPGx0gwAN0Y9bKgGU2w0K2yrcNtNMJPPlqib7YxYyLQXxk9NB+PDUUnmSVUUOfSTN2vdrEpA9cAOgqJheRe/vYq/kMe3o0ooFDAPAPQkhErndZEiGYKfR3eqb+3WiJ05OaZDyBgWo4reFWUSKcoduI52t5zRkCFJ3HULjO5S91s7T/g26OVv4fCZtjoPD86PjN/xdCPLyU= X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /jFVd8lerLXeFZb/8fONG1imCA5xXaJh8uED8XQ5RWaMb63LcLAdj/u+/JFUKf78DiIyFfdZTH0M4ek7Kd7yunLqfweqZapNR8XVXTeyEVUZQ2cy8p4hQpyH94ZuLsh7WkoEo4Mzdc18UMbNaNIpfGmA340RV1mjU4jk2c2ybeMW0dYyJWqOh1OLh3PPNvHDQMMxZnTk8mNDB7qUisM0vHy+oX0kc+bUkyTp35zj+UJR7zI/GGKsA88juneQMwgyprhDx6TKkRWSNUmUHvbSMySxHh7z3gtgb/0J/OEnfbUPNZoovhH1QXuLCNxAFXWqGLJca/nOo0pyRMDr4NSTgPTg4RdqSwjT0Nu1+kf0H5IdrTzAtUkRMeHeXm80iesEAjuQbgm6xAXcrXQbx0xsH0XUwIPvWp5p1t0xjJtmPMNNNjLjQ5eel3NmBgS+DOi3OcbFDVLFJmVcYaOS4rvhPsKK/J96s52k1MXYhk8sjnMaLRz4zaZGDBATVcmez3Qpqm0apqIv/WZO3Ws9gApO50zbrtxhBQotiGCaqnd9Zv8g+X3wO7pKW1hQDHNlpEp+zRC9ANNA8oph7SADoFBwrfBeGijfptX0xIGmCb30to7VxyWtcZ79BAFIWdqQ6KnnhEzmajSxBz2l+mOi4fnSaQ== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: yQCl3deN4q8g9De//fTO4St47RymhVrbHzRSd7R1ERFVw5piLgcb6QSxXrkOsVJQR9mM7w1QYWOSc3fQ1Sw6k43B2tP2ljQ6KR/TwJLPbKB6ieWelnT4FTu1894AfTQwmhprxTCpufg5tkyZzTSqXjKTqXP0wAFpDcqKjvavIDkIvZRxzGTQxSv/MUHdJelY1YXA+LCr1fOR2m9s+6nqaqJxk/WZGs/Sj/066mfgPA3P9MKfhTDIGFjLX70XQtKVJ4SwCetCrj8WOZ6QJCDwAVjUhIgMjUfMOfqv9In3yFXVYLcz4IfJlmo46CLraqZ7qTSWF9I0Tk3HATD26Wlc2yX8fbjwLlDvPuRQD9swSetofbyVyIURNbeEk5rDV5Xvu5HchiE5IxAyzeKMgwWv4U5F3sBo/UZUDMCpZjph3IXtSejF/zhPsMLpoTdx+ghDo3bdgZeJf9lzjT//uVKnAcYpzkEQi1CD2MndLeJ4LCEOrUXneMCJ1Vdw97kULSsC+6qPUyLEHpCaUlsHpbpZcUXGiaWuVXutetTJF2NZ4RPyUvfgeokXpoprm5mEDBkDX2mKqy7mOjhTQ8zfAIW4cCDhKlL8A5ZZlcAmeriuipum1g/7MM0bPxwWkQKnnMgtgUTLrs8U9L4YtJ/SeraVn7KQmVPEu+aV9NCFrIs/WZRX/q4g+Qzyy1zt0kC5+GmX5QejogzjD0JlEiFya22Dg9QGEQF6PvLHbszUAVchhftKJdwygAYGl4Od7zqmt5nXwiQ+iRZGidwMi9RgCXE6xVkDCQFrn+weWCFdrgm9hnjo8HJ/SiGsxCRYKaJHQVeumBEmY9J0tZan93r6Qpefi6xO2ZN8A/Oc+CdM0VSZ7ZybibJifCDn7/D49MwucPqfy8MnVSWOJ2sDtYm7eHeFS9VvhSGOYMg0za7McIWmqbD81zHa0Ks4hfff8EjPVpVlXuFjg3rR2/RPhS3+aULIjSTNkAybci0ILm/vKeZ3oVl0iFHYlQfqzAAAoIpENNIIldZIHJocBQ65IqeLg2kLwl8r9rpPpJ2h9ZSflADHoCcj0QMuxcAbfpAVHST4oV1GO8cBtAECl1V5WG1EuTs9QaMhP71Pj87wXmRWdHSqbP7Jv4tEr7UPpy06KVeXsn+JR9tgvpZOivBOf9Fgsl+14UX0Fvsa66LBA0n427ug8o+ZEHuirrfCwtuamCozSKN5R0DYfZIfa8U/zaTY8z9dz3oFHPvsRsiHj/qvpiZO1qgitNOrLZzTHOuhNlzRCEWf0s2E69Q5mU7qULte1hAxvfaGXf5v9HS3RcynIudPG88= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 20ad623a-8207-4200-7c95-08daa30604ab X-MS-Exchange-CrossTenant-AuthSource: GV1P250MB0737.EURP250.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2022 17:05:47.2259 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2P250MB0175 Subject: [FFmpeg-devel] [PATCH 6/7] avcodec/escape124: Fix segfault with negative linesizes X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: JJjwPs4WnBlm Use ptrdiff_t instead of unsigned for linesizes. Fixes the armovie-escape124 FATE test with negative linesizes. Signed-off-by: Andreas Rheinhardt --- libavcodec/escape124.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/libavcodec/escape124.c b/libavcodec/escape124.c index eeeb9bb0f7..024eec59ce 100644 --- a/libavcodec/escape124.c +++ b/libavcodec/escape124.c @@ -178,8 +178,8 @@ static void insert_mb_into_sb(SuperBlock* sb, MacroBlock mb, unsigned index) { dst[4] = mb.pixels32[1]; } -static void copy_superblock(uint16_t* dest, unsigned dest_stride, - uint16_t* src, unsigned src_stride) +static void copy_superblock(uint16_t* dest, ptrdiff_t dest_stride, + uint16_t* src, ptrdiff_t src_stride) { unsigned y; if (src) @@ -211,7 +211,7 @@ static int escape124_decode_frame(AVCodecContext *avctx, AVFrame *frame, superblocks_per_row = avctx->width / 8, skip = -1; uint16_t* old_frame_data, *new_frame_data; - unsigned old_stride, new_stride; + ptrdiff_t old_stride, new_stride; int ret; From patchwork Fri Sep 30 17:05:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 38484 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b1c:b0:96:9ee8:5cfd with SMTP id c28csp1839986pzh; Fri, 30 Sep 2022 10:06:33 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7oun/gQbq2kjvpqIbHeQnM+VuICegijjCOnWoH4FC53yFwUwSj4zSMXL7gIGSa5B1EMh35 X-Received: by 2002:a17:907:3201:b0:741:94f2:aeaf with SMTP id xg1-20020a170907320100b0074194f2aeafmr6973251ejb.505.1664557593769; Fri, 30 Sep 2022 10:06:33 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x69-20020a50bacb000000b00450c4d9a097si2212169ede.54.2022.09.30.10.06.33; Fri, 30 Sep 2022 10:06:33 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=sT19tyMH; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 67BFE68BC07; Fri, 30 Sep 2022 20:06:14 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03olkn2035.outbound.protection.outlook.com [40.92.58.35]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 77E1C68BBDB for ; Fri, 30 Sep 2022 20:06:07 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jj+0e/RKDIQlhtMfEfRnLsAHBaaGCGe2QoN2efm4cAZrK9IEEZg4zs9Wvdmf/xC3r8+1StZKkDND5KtRfulpeyBG11WIZRU37mJflsoWjgNQ6E9mCQphAejThceTBp+5x3hZyhvrzHYodNnlh3Z6NxHlLTACkwYrx+IKgw8P7LPKkMP6//rjfkvoskTPwQMQjOwIF5PYkJFX63SxDFJVwRw53W4zeZyN0dFZQGJbk0OCrongcKVECL7USwQxW+jONEH5muTt1J1rCkAogKaKZA9cxFL4w2J9sOyl34KNFQ0YZ8VMCnfzG4nxq4Xd2IQvI/6UaaZjOwWlUwrkqniiuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=t6HNqMpEevv2ywwUZLn0ra7PRs4EssxLCZlSo8JgV6U=; b=D0+mOpQYnQC4YFNw+a4PpeAVSgPwbcRqDb9afQAtX+McGGNDNI/0emV77j7f3R8CzLTqpgpvNUD95SwLDhcFQf7SpzAddlqWIA+pV17/HoqiQBhPMb6jZnQMTsSPvwgKRRby7sHMD13ruVq+H6GeGtXGbmGi931WFojZo8TJAmEaz+dAfAMTTemCJtAtNszsBxiwdqQiqwnIwRc7LPBrTShxFyj/3IwkmqYsFSVyDmK1xvQnAbX10caG/jHZ7iCBZhw8Wr4ejlB6R44PV9SP8YtubvhP+BCLN2lQyvHyXaTbAeV1xZ12cY7ScoqUE6s1VoL9Flgs400JwMpmCQGl2A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=t6HNqMpEevv2ywwUZLn0ra7PRs4EssxLCZlSo8JgV6U=; b=sT19tyMHGchb8GrNt9r0/pz1KBtCYnANKr1w+cXodORTqcAyA3wVwd+Qayq808dSkgYaTSIeHV44Vf+SAcUkM9M4RZkvnU5Xa410aO6i28jQv+L0qadZ5zuIdLjfMhHIkTb4hK2lLNKkOJDbrntV5Hd4lMvpYfftIHynJjvxtdQ2C0qah7+BSk2OCNMGja5lW+duCSviUEFDl00se+kS0y/V5gATmFnEJeuQMpE19XYpjYZcn+BQsoHFW7aSrHo9V2LJMX4FQxvVcDtBit/NQBZs1CFp7puvh/HRsVsBU4B8WCPrDiqkuOZPeTjoPYSPlMmIkWHtn7fwYD5cAxBPjQ== Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) by DU2P250MB0175.EURP250.PROD.OUTLOOK.COM (2603:10a6:10:274::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.23; Fri, 30 Sep 2022 17:05:48 +0000 Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::68bd:2fc7:ac52:38f8]) by GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::68bd:2fc7:ac52:38f8%9]) with mapi id 15.20.5654.025; Fri, 30 Sep 2022 17:05:48 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 30 Sep 2022 19:05:15 +0200 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: X-TMN: [/axKKcZSRe9GJVlsSmTUAm9TuF3URLMR] X-ClientProxiedBy: FR3P281CA0040.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::8) To GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) X-Microsoft-Original-Message-ID: <20220930170515.3648239-6-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: GV1P250MB0737:EE_|DU2P250MB0175:EE_ X-MS-Office365-Filtering-Correlation-Id: e9743632-38ac-4010-ff01-08daa3060547 X-MS-Exchange-SLBlob-MailProps: AZnQBsB9XmrST/aUJU0/QUaXtXS9lKwzqrcIgu5FoPSW6N2PQGZygfKmdIrAAu+GUgtEFt9pPjqSiAYYiFO+IuCBsrJxN0b8/SBDtrw8pAa91lscQrWzdQO/W27pkqZL3X2xa5V2IYXMRDTYhr/CWVUEXqigZ9mByW5akTXx/4IPqpXidZd+rUcpTFxzFiKlUj/sBhUSHye0kdseZfMZ/mUaZCnTFOUKDvDyhK6Nx1qzL2hOBkNGlqCn5gQ6a/bS61GoG+FC9N6s4QvDBKJfHEVNInqGALCrOeogp7CsuY/Z742ex2Y2TfTIGbMJB5i/tE5kOty873y1cBGwkApqZhZ8w50HtgBrzNl5LA50IY+yMHsan76h/n8LQ3wGp4yWz9LvdvTnOhiz7I44RRPbT08/IuCUPZs3EsOnAH8c3gXPTVpKk821zPFrAbp+8FtzmDTsHoJKnloanhCzfmpOa7mnr+sVcuncXLF4BbVju9zcsk9Mk+iC6faEDdAJaTsBOfMkb1b4lDB7sxSLaSRHx168tB1fDkBj+Um+acHQ3BWX6/WShpgAUfLYBjwLiVecRMFpp3JrFFGUPAbBXV+K5tvH7Dd6zj9M/cBobNsRof3OfflBLz/sRNowaO2bo9KNzJHHv51Ztc/1+/c6h1j9vfoG7evomKqHWBOkz1nkql89fvCm1g7E25Byvwi23ZBO6fv1Vvj8M0O1JWYfg3UpuCNcQb2snzl4sJw53S4OEwkom5S8NfKbmX6HNJ9JmzO77HbP44euFig= X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: u62J77W9t/VWQCjEeibqG/afeANm6TUwOryfO2AkaRKswshKqU9M1/3l/IiTNQ51Q7ymkYdWeMeWATR6l3o+M6mR4a/nyVRiD2NRbnQ76N0yx0xMaXJ3qSOyV3sCH/HIpqhX2soUTrEQ0ZLBtvHctHgHcDWa94CdNZ7pa+FhQIhaiMg9dw7alb7Uysxp6V6dzjcc9PTW0S2hbh4TGM9SiBneo6N86TzVFrp6GcBuE6Q0ocqN8Qnihe9VmA+tEI7G79I8Vv5OMzk/VAW96bhFb6Ks3FFuaP1DND5lHd3i99cxnoZfRONH/iytLmy/0FQ6h42Xt1Umn0mBMcNMvvvF9T0mYaj9iCj5kx5jfzfZhBSBzQtdeOgS5cEoMwSwWMLjpI+qSdTyT1ttaiS62bdsan2XteZKmVbgGMVBbinCheBtMEVuZJSDW6ubxFMsjyffDx+L7IrKe6WPDKfnLpnAh2wxGtq3aWWciMxRCFz14S0xdP8LXMAud9/EzQfqEl7Kpo9LEiYMtC+yPh4gUV2ClqF4QvGL7IaklkgLm5M/PwBEewJ0X4cqE9JiNys/UXZgz06DUtqP/xSCdiP09bOBW6b+YL5XBsrNSKq4KJEsbCxM0Q5cQ36TuedWIlaoVxPpE+5+oMywJy/oxAVw/qLQ3Q== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 4VZ5+DKfuW6QGZ/srz82v2tF/KZ9PCaMshMD1eSaWTmx27+i1Zb/qIrwDxL8HEqHwSX58IZTPAQEMx1Q8Hcj189Ky6dR2e7+Kn59IToN29xyDTRhz6qDB9uwxRLG9QRfcKihxdYn5PIWseIafRxyOPuaVwuJC8/I9vWhLjZZHDGlbfk0QZBgs8Jc+aB75rlZKavrsW1Anu+QxK+G61RSVYtGhB0DlIxTucgxmtUOjV5/CEPKgANneS19JaYdG23BcGl8CPhym6dNHgsEMcrQunGf6BVYYNf0+LMUhTJfFtYqFGGVCWAl7lu1egvVfgK6rL5y9JzlVeSStlp1IOF6C7e1RCp1Jc8pOQTs5ZW9Z9GEb4lV7G63xR6HGSoSq19LhL8uCexEzOYBU4sXiWfgXDjNGSgVQMr3xCvRfcc1AVyWPf0iHG1CA01s4qThzW8nvnSDxusDuLMD6pnR9yFiwzAYJhvFGESCIvNuLhCr1nD6nnatwFg7AGsWURJ0NBRvPtrcSoEo9LPeF4ldfzCwsVVQbNsi9KTwAsW6g4C90VB6LiZfTPyWkRLanzm28ekv6h+JniSTYypwcO2b9VPnA5R1ioIitRutq5pnnE07eOHwhGul1uE/MrrsYUPlOBvnb8nhKeCMI/zcgl+qOV0Dh+uhckAucSk45WTUuAkxbI7CBmYvLXr72Tf9A7DH5XW4Jdr2bnsa+eOG94zofb2/Dte3HwsLcQSB6BW//caz9Tt+MIt5f75ZCUhw0MyHMXL2ZktXpsjL3/X7VgxxknIKVdA4iYlPcI9eojjy5RvlhH1PknYqfsWFppJhMMsW7UD328k7wMusrNR0tz0QAX1uMq99+TO7NbudaquItB2ULxjMLFqGvC4wVG/tyw3AxVmLus2fWW+9da4C0luiVDNiVKHMks0nhN7W9Bio+VwDCbhTOIlZ6TfAIFX4O0uauMhc3Xy7jC3ndN8e1QrN3uAmF4F1zNszp8gShx9whRiiy5w8+Hn8IUXcYxfQQUQsaQQqt+bCy58hnuVDtq9PY1fjxwbeXv7VdfVCGNW1K+hMUh1p1o6S9qn6p+JNA7WfusiVKefB09utOF3aQR6wBxMMlS+YJ3sdbb4klM7aPnaR0s+X++fxV1QwPWz0n8BSWmGBxwseLgU7lEtl4ot5qH6Q/PTt/O7czzWcC/ZdFAjWiborTpqQ2u8+RHDkHZ4ggGQRNMx9jSBVgKiNblYwoo5D/ByO5+5mL0Gd/n6Dxir80Go+gmqZimDiYm28Ut5DAKkt5pRf6a1MjN5GKD69unLIU+67pxLAZIAk2O8jZO8FfIw= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: e9743632-38ac-4010-ff01-08daa3060547 X-MS-Exchange-CrossTenant-AuthSource: GV1P250MB0737.EURP250.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2022 17:05:48.1957 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2P250MB0175 Subject: [FFmpeg-devel] [PATCH 7/7] avcodec/fraps: Fix segfault with negative linesizes X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: oNj1shln39rh Using unsigned and negative linesizes doesn't really work. Use ptrdiff_t instead. This fixes the fraps-v0 and fraps-v1 FATE tests with negative linesizes. Signed-off-by: Andreas Rheinhardt --- libavcodec/fraps.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/libavcodec/fraps.c b/libavcodec/fraps.c index 9c8cbf7323..4c4c46b602 100644 --- a/libavcodec/fraps.c +++ b/libavcodec/fraps.c @@ -141,7 +141,6 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *f, int buf_size = avpkt->size; uint32_t header; unsigned int version,header_size; - unsigned int x, y; const uint32_t *buf32; uint32_t *luma1,*luma2,*cb,*cr; uint32_t offs[4]; @@ -238,12 +237,12 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *f, } buf32 = (const uint32_t*)buf; - for (y = 0; y < avctx->height / 2; y++) { + for (ptrdiff_t y = 0; y < avctx->height / 2; y++) { luma1 = (uint32_t*)&f->data[0][ y * 2 * f->linesize[0] ]; luma2 = (uint32_t*)&f->data[0][ (y * 2 + 1) * f->linesize[0] ]; cr = (uint32_t*)&f->data[1][ y * f->linesize[1] ]; cb = (uint32_t*)&f->data[2][ y * f->linesize[2] ]; - for (x = 0; x < avctx->width; x += 8) { + for (ptrdiff_t x = 0; x < avctx->width; x += 8) { *luma1++ = *buf32++; *luma1++ = *buf32++; *luma2++ = *buf32++; @@ -258,18 +257,18 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *f, if (is_pal) { uint32_t *pal = (uint32_t *)f->data[1]; - for (y = 0; y < 256; y++) { + for (unsigned y = 0; y < 256; y++) { pal[y] = AV_RL32(buf) | 0xFF000000; buf += 4; } - for (y = 0; y height; y++) + for (ptrdiff_t y = 0; y < avctx->height; y++) memcpy(&f->data[0][y * f->linesize[0]], &buf[y * avctx->width], avctx->width); } else { /* Fraps v1 is an upside-down BGR24 */ - for (y = 0; yheight; y++) + for (ptrdiff_t y = 0; y < avctx->height; y++) memcpy(&f->data[0][(avctx->height - y - 1) * f->linesize[0]], &buf[y * avctx->width * 3], 3 * avctx->width);