From patchwork Mon Nov 21 23:58:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 39381 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp2622254pzb; Mon, 21 Nov 2022 15:58:36 -0800 (PST) X-Google-Smtp-Source: AA0mqf5eh+chJGJMbElpv8XAdoPf6Py+39ZRzj4x/a7VvK6kDTDV30XUco6XKzEC9JwMbxToZCpn X-Received: by 2002:a17:906:4e4e:b0:7ae:e6ac:2427 with SMTP id g14-20020a1709064e4e00b007aee6ac2427mr17514409ejw.345.1669075115746; Mon, 21 Nov 2022 15:58:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669075115; cv=none; d=google.com; s=arc-20160816; b=lX03/vIHW8q4Zq0AmWEoUQnDsc5hRMbUtTEQOJzym0DTITvoVS3F9h2oeTakx0u40l NA8p4rz/cBLafRcicGkogo2WPQr2hEs5xaQ1FvARXpiVPEThUjgZi08HSS85hL3JzVM0 c10rT4GUA80fU4oyG67FKSWxsB9qM3HH9wrsNr1grHv9cAPqjpYCZ9WxAE8jiUT2uci+ H5LRjGwmp/+/FnFBbsATvYrXviZLAu0EfDsjtyij5yUn1doKw+LX2MXjQ6SyqmXJlZin vaysebE7oBnZnDBtlIsjwthxhifSpLFv6OA0SLx6hPgKQ9nih57FljG2ypGG4gOAITxI o7Nw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:delivered-to; bh=EUG4SbD57pBdmSNgwQ8AxYI05+sf/92TaXRUo3Wjsxc=; b=QXov1PeoFNDhFHmW69kJYnQ4y2qRNEJrTk0AShTNGfNj9EWOhI3dSjBgZ3DJP98oDC yyiCUXjUM13F65uHqss/MPkLVi32Q+IOA3js6/oa8jZmE03zmKTJPAw20/P6zOVHDI+g nDwyXWFWCM+AH7Pkx3nColxE94eEXFaoIaWab19ACNaGuqwuM77GGxuCt3+gMkzSdf0X MDfQRENx7uFHeRID3ImtcOQe6J3yVOdEkz1EEEly/GcKVjySWTkKs1XC7EhXbMVtLBV/ CZPiiPvC+Y1z7DplfmjWBmx5htvZ9luuI/fYVPXLQc49nKFb2TFMr03FiVvV74ouECno 34TA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id cs13-20020a0564020c4d00b004615c5728e8si9136125edb.494.2022.11.21.15.58.35; Mon, 21 Nov 2022 15:58:35 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6458068BA43; Tue, 22 Nov 2022 01:58:31 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5607F68B1A1 for ; Tue, 22 Nov 2022 01:58:24 +0200 (EET) Received: (Authenticated sender: michael@niedermayer.cc) by mail.gandi.net (Postfix) with ESMTPSA id EF5DE60002 for ; Mon, 21 Nov 2022 23:58:23 +0000 (UTC) From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Tue, 22 Nov 2022 00:58:21 +0100 Message-Id: <20221121235823.28468-1-michael@niedermayer.cc> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH 1/3] avutil/tx: Use unsigned in ff_tx_fft_sr_combine() to avoid undefined behavior X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: vx3TkYqJTRDH Fixes: signed integer overflow: -1284837070 - 982101618 cannot be represented in type 'int' Fixes: 53105/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AC3_FIXED_fuzzer-4848015827664896 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavutil/tx_priv.h | 3 +++ libavutil/tx_template.c | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/libavutil/tx_priv.h b/libavutil/tx_priv.h index fb61119009..6f01f305fc 100644 --- a/libavutil/tx_priv.h +++ b/libavutil/tx_priv.h @@ -34,6 +34,7 @@ #define MULT(x, m) ((x) * (m)) #define SCALE_TYPE float typedef float TXSample; +typedef float TXUSample; typedef AVComplexFloat TXComplex; #elif defined(TX_DOUBLE) #define TX_TAB(x) x ## _double @@ -45,6 +46,7 @@ typedef AVComplexFloat TXComplex; #define MULT(x, m) ((x) * (m)) #define SCALE_TYPE double typedef double TXSample; +typedef double TXUSample; typedef AVComplexDouble TXComplex; #elif defined(TX_INT32) #define TX_TAB(x) x ## _int32 @@ -56,6 +58,7 @@ typedef AVComplexDouble TXComplex; #define MULT(x, m) (((((int64_t)(x)) * (int64_t)(m)) + 0x40000000) >> 31) #define SCALE_TYPE float typedef int32_t TXSample; +typedef uint32_t TXUSample; typedef AVComplexInt32 TXComplex; #else typedef void TXComplex; diff --git a/libavutil/tx_template.c b/libavutil/tx_template.c index 666af5e496..b624869010 100644 --- a/libavutil/tx_template.c +++ b/libavutil/tx_template.c @@ -497,7 +497,7 @@ static inline void TX_NAME(ff_tx_fft_sr_combine)(TXComplex *z, int o2 = 4*len; int o3 = 6*len; const TXSample *wim = cos + o1 - 7; - TXSample t1, t2, t3, t4, t5, t6, r0, i0, r1, i1; + TXUSample t1, t2, t3, t4, t5, t6, r0, i0, r1, i1; for (int i = 0; i < len; i += 4) { TRANSFORM(z[0], z[o1 + 0], z[o2 + 0], z[o3 + 0], cos[0], wim[7]);