From patchwork Fri Sep 30 17:05:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 38484 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b1c:b0:96:9ee8:5cfd with SMTP id c28csp1839986pzh; Fri, 30 Sep 2022 10:06:33 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7oun/gQbq2kjvpqIbHeQnM+VuICegijjCOnWoH4FC53yFwUwSj4zSMXL7gIGSa5B1EMh35 X-Received: by 2002:a17:907:3201:b0:741:94f2:aeaf with SMTP id xg1-20020a170907320100b0074194f2aeafmr6973251ejb.505.1664557593769; Fri, 30 Sep 2022 10:06:33 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x69-20020a50bacb000000b00450c4d9a097si2212169ede.54.2022.09.30.10.06.33; Fri, 30 Sep 2022 10:06:33 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=sT19tyMH; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 67BFE68BC07; Fri, 30 Sep 2022 20:06:14 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03olkn2035.outbound.protection.outlook.com [40.92.58.35]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 77E1C68BBDB for ; Fri, 30 Sep 2022 20:06:07 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jj+0e/RKDIQlhtMfEfRnLsAHBaaGCGe2QoN2efm4cAZrK9IEEZg4zs9Wvdmf/xC3r8+1StZKkDND5KtRfulpeyBG11WIZRU37mJflsoWjgNQ6E9mCQphAejThceTBp+5x3hZyhvrzHYodNnlh3Z6NxHlLTACkwYrx+IKgw8P7LPKkMP6//rjfkvoskTPwQMQjOwIF5PYkJFX63SxDFJVwRw53W4zeZyN0dFZQGJbk0OCrongcKVECL7USwQxW+jONEH5muTt1J1rCkAogKaKZA9cxFL4w2J9sOyl34KNFQ0YZ8VMCnfzG4nxq4Xd2IQvI/6UaaZjOwWlUwrkqniiuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=t6HNqMpEevv2ywwUZLn0ra7PRs4EssxLCZlSo8JgV6U=; b=D0+mOpQYnQC4YFNw+a4PpeAVSgPwbcRqDb9afQAtX+McGGNDNI/0emV77j7f3R8CzLTqpgpvNUD95SwLDhcFQf7SpzAddlqWIA+pV17/HoqiQBhPMb6jZnQMTsSPvwgKRRby7sHMD13ruVq+H6GeGtXGbmGi931WFojZo8TJAmEaz+dAfAMTTemCJtAtNszsBxiwdqQiqwnIwRc7LPBrTShxFyj/3IwkmqYsFSVyDmK1xvQnAbX10caG/jHZ7iCBZhw8Wr4ejlB6R44PV9SP8YtubvhP+BCLN2lQyvHyXaTbAeV1xZ12cY7ScoqUE6s1VoL9Flgs400JwMpmCQGl2A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=t6HNqMpEevv2ywwUZLn0ra7PRs4EssxLCZlSo8JgV6U=; b=sT19tyMHGchb8GrNt9r0/pz1KBtCYnANKr1w+cXodORTqcAyA3wVwd+Qayq808dSkgYaTSIeHV44Vf+SAcUkM9M4RZkvnU5Xa410aO6i28jQv+L0qadZ5zuIdLjfMhHIkTb4hK2lLNKkOJDbrntV5Hd4lMvpYfftIHynJjvxtdQ2C0qah7+BSk2OCNMGja5lW+duCSviUEFDl00se+kS0y/V5gATmFnEJeuQMpE19XYpjYZcn+BQsoHFW7aSrHo9V2LJMX4FQxvVcDtBit/NQBZs1CFp7puvh/HRsVsBU4B8WCPrDiqkuOZPeTjoPYSPlMmIkWHtn7fwYD5cAxBPjQ== Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) by DU2P250MB0175.EURP250.PROD.OUTLOOK.COM (2603:10a6:10:274::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.23; Fri, 30 Sep 2022 17:05:48 +0000 Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::68bd:2fc7:ac52:38f8]) by GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::68bd:2fc7:ac52:38f8%9]) with mapi id 15.20.5654.025; Fri, 30 Sep 2022 17:05:48 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 30 Sep 2022 19:05:15 +0200 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: X-TMN: [/axKKcZSRe9GJVlsSmTUAm9TuF3URLMR] X-ClientProxiedBy: FR3P281CA0040.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4a::8) To GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) X-Microsoft-Original-Message-ID: <20220930170515.3648239-6-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: GV1P250MB0737:EE_|DU2P250MB0175:EE_ X-MS-Office365-Filtering-Correlation-Id: e9743632-38ac-4010-ff01-08daa3060547 X-MS-Exchange-SLBlob-MailProps: AZnQBsB9XmrST/aUJU0/QUaXtXS9lKwzqrcIgu5FoPSW6N2PQGZygfKmdIrAAu+GUgtEFt9pPjqSiAYYiFO+IuCBsrJxN0b8/SBDtrw8pAa91lscQrWzdQO/W27pkqZL3X2xa5V2IYXMRDTYhr/CWVUEXqigZ9mByW5akTXx/4IPqpXidZd+rUcpTFxzFiKlUj/sBhUSHye0kdseZfMZ/mUaZCnTFOUKDvDyhK6Nx1qzL2hOBkNGlqCn5gQ6a/bS61GoG+FC9N6s4QvDBKJfHEVNInqGALCrOeogp7CsuY/Z742ex2Y2TfTIGbMJB5i/tE5kOty873y1cBGwkApqZhZ8w50HtgBrzNl5LA50IY+yMHsan76h/n8LQ3wGp4yWz9LvdvTnOhiz7I44RRPbT08/IuCUPZs3EsOnAH8c3gXPTVpKk821zPFrAbp+8FtzmDTsHoJKnloanhCzfmpOa7mnr+sVcuncXLF4BbVju9zcsk9Mk+iC6faEDdAJaTsBOfMkb1b4lDB7sxSLaSRHx168tB1fDkBj+Um+acHQ3BWX6/WShpgAUfLYBjwLiVecRMFpp3JrFFGUPAbBXV+K5tvH7Dd6zj9M/cBobNsRof3OfflBLz/sRNowaO2bo9KNzJHHv51Ztc/1+/c6h1j9vfoG7evomKqHWBOkz1nkql89fvCm1g7E25Byvwi23ZBO6fv1Vvj8M0O1JWYfg3UpuCNcQb2snzl4sJw53S4OEwkom5S8NfKbmX6HNJ9JmzO77HbP44euFig= X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: u62J77W9t/VWQCjEeibqG/afeANm6TUwOryfO2AkaRKswshKqU9M1/3l/IiTNQ51Q7ymkYdWeMeWATR6l3o+M6mR4a/nyVRiD2NRbnQ76N0yx0xMaXJ3qSOyV3sCH/HIpqhX2soUTrEQ0ZLBtvHctHgHcDWa94CdNZ7pa+FhQIhaiMg9dw7alb7Uysxp6V6dzjcc9PTW0S2hbh4TGM9SiBneo6N86TzVFrp6GcBuE6Q0ocqN8Qnihe9VmA+tEI7G79I8Vv5OMzk/VAW96bhFb6Ks3FFuaP1DND5lHd3i99cxnoZfRONH/iytLmy/0FQ6h42Xt1Umn0mBMcNMvvvF9T0mYaj9iCj5kx5jfzfZhBSBzQtdeOgS5cEoMwSwWMLjpI+qSdTyT1ttaiS62bdsan2XteZKmVbgGMVBbinCheBtMEVuZJSDW6ubxFMsjyffDx+L7IrKe6WPDKfnLpnAh2wxGtq3aWWciMxRCFz14S0xdP8LXMAud9/EzQfqEl7Kpo9LEiYMtC+yPh4gUV2ClqF4QvGL7IaklkgLm5M/PwBEewJ0X4cqE9JiNys/UXZgz06DUtqP/xSCdiP09bOBW6b+YL5XBsrNSKq4KJEsbCxM0Q5cQ36TuedWIlaoVxPpE+5+oMywJy/oxAVw/qLQ3Q== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 4VZ5+DKfuW6QGZ/srz82v2tF/KZ9PCaMshMD1eSaWTmx27+i1Zb/qIrwDxL8HEqHwSX58IZTPAQEMx1Q8Hcj189Ky6dR2e7+Kn59IToN29xyDTRhz6qDB9uwxRLG9QRfcKihxdYn5PIWseIafRxyOPuaVwuJC8/I9vWhLjZZHDGlbfk0QZBgs8Jc+aB75rlZKavrsW1Anu+QxK+G61RSVYtGhB0DlIxTucgxmtUOjV5/CEPKgANneS19JaYdG23BcGl8CPhym6dNHgsEMcrQunGf6BVYYNf0+LMUhTJfFtYqFGGVCWAl7lu1egvVfgK6rL5y9JzlVeSStlp1IOF6C7e1RCp1Jc8pOQTs5ZW9Z9GEb4lV7G63xR6HGSoSq19LhL8uCexEzOYBU4sXiWfgXDjNGSgVQMr3xCvRfcc1AVyWPf0iHG1CA01s4qThzW8nvnSDxusDuLMD6pnR9yFiwzAYJhvFGESCIvNuLhCr1nD6nnatwFg7AGsWURJ0NBRvPtrcSoEo9LPeF4ldfzCwsVVQbNsi9KTwAsW6g4C90VB6LiZfTPyWkRLanzm28ekv6h+JniSTYypwcO2b9VPnA5R1ioIitRutq5pnnE07eOHwhGul1uE/MrrsYUPlOBvnb8nhKeCMI/zcgl+qOV0Dh+uhckAucSk45WTUuAkxbI7CBmYvLXr72Tf9A7DH5XW4Jdr2bnsa+eOG94zofb2/Dte3HwsLcQSB6BW//caz9Tt+MIt5f75ZCUhw0MyHMXL2ZktXpsjL3/X7VgxxknIKVdA4iYlPcI9eojjy5RvlhH1PknYqfsWFppJhMMsW7UD328k7wMusrNR0tz0QAX1uMq99+TO7NbudaquItB2ULxjMLFqGvC4wVG/tyw3AxVmLus2fWW+9da4C0luiVDNiVKHMks0nhN7W9Bio+VwDCbhTOIlZ6TfAIFX4O0uauMhc3Xy7jC3ndN8e1QrN3uAmF4F1zNszp8gShx9whRiiy5w8+Hn8IUXcYxfQQUQsaQQqt+bCy58hnuVDtq9PY1fjxwbeXv7VdfVCGNW1K+hMUh1p1o6S9qn6p+JNA7WfusiVKefB09utOF3aQR6wBxMMlS+YJ3sdbb4klM7aPnaR0s+X++fxV1QwPWz0n8BSWmGBxwseLgU7lEtl4ot5qH6Q/PTt/O7czzWcC/ZdFAjWiborTpqQ2u8+RHDkHZ4ggGQRNMx9jSBVgKiNblYwoo5D/ByO5+5mL0Gd/n6Dxir80Go+gmqZimDiYm28Ut5DAKkt5pRf6a1MjN5GKD69unLIU+67pxLAZIAk2O8jZO8FfIw= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: e9743632-38ac-4010-ff01-08daa3060547 X-MS-Exchange-CrossTenant-AuthSource: GV1P250MB0737.EURP250.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2022 17:05:48.1957 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2P250MB0175 Subject: [FFmpeg-devel] [PATCH 7/7] avcodec/fraps: Fix segfault with negative linesizes X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: oNj1shln39rh Using unsigned and negative linesizes doesn't really work. Use ptrdiff_t instead. This fixes the fraps-v0 and fraps-v1 FATE tests with negative linesizes. Signed-off-by: Andreas Rheinhardt --- libavcodec/fraps.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/libavcodec/fraps.c b/libavcodec/fraps.c index 9c8cbf7323..4c4c46b602 100644 --- a/libavcodec/fraps.c +++ b/libavcodec/fraps.c @@ -141,7 +141,6 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *f, int buf_size = avpkt->size; uint32_t header; unsigned int version,header_size; - unsigned int x, y; const uint32_t *buf32; uint32_t *luma1,*luma2,*cb,*cr; uint32_t offs[4]; @@ -238,12 +237,12 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *f, } buf32 = (const uint32_t*)buf; - for (y = 0; y < avctx->height / 2; y++) { + for (ptrdiff_t y = 0; y < avctx->height / 2; y++) { luma1 = (uint32_t*)&f->data[0][ y * 2 * f->linesize[0] ]; luma2 = (uint32_t*)&f->data[0][ (y * 2 + 1) * f->linesize[0] ]; cr = (uint32_t*)&f->data[1][ y * f->linesize[1] ]; cb = (uint32_t*)&f->data[2][ y * f->linesize[2] ]; - for (x = 0; x < avctx->width; x += 8) { + for (ptrdiff_t x = 0; x < avctx->width; x += 8) { *luma1++ = *buf32++; *luma1++ = *buf32++; *luma2++ = *buf32++; @@ -258,18 +257,18 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *f, if (is_pal) { uint32_t *pal = (uint32_t *)f->data[1]; - for (y = 0; y < 256; y++) { + for (unsigned y = 0; y < 256; y++) { pal[y] = AV_RL32(buf) | 0xFF000000; buf += 4; } - for (y = 0; y height; y++) + for (ptrdiff_t y = 0; y < avctx->height; y++) memcpy(&f->data[0][y * f->linesize[0]], &buf[y * avctx->width], avctx->width); } else { /* Fraps v1 is an upside-down BGR24 */ - for (y = 0; yheight; y++) + for (ptrdiff_t y = 0; y < avctx->height; y++) memcpy(&f->data[0][(avctx->height - y - 1) * f->linesize[0]], &buf[y * avctx->width * 3], 3 * avctx->width);