From patchwork Wed Sep 13 23:47:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 43727 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4b2f:b0:149:dfde:5c0a with SMTP id fp47csp58383pzb; Wed, 13 Sep 2023 16:48:10 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFdch3QBqFvqopISnsF30yoX9gU6TWOaZEZv8YxU1xmfKnG2sUgx1t7VBw/sN/931mGkhLI X-Received: by 2002:a05:6402:8c5:b0:52e:d2a4:45bf with SMTP id d5-20020a05640208c500b0052ed2a445bfmr3326908edz.30.1694648890311; Wed, 13 Sep 2023 16:48:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694648890; cv=none; d=google.com; s=arc-20160816; b=Z3kZ6tQgYpwTCfYb5gxyUaftu3ZPx1T7cie/N1RYHZ/9raDIn3Fpa/b3l7Bt1QnZi0 Z8Bc79VCJszcm1OprpRS9l3tmcPDvoTarhdmQ/TIBbJOJ/iPd951Wt2bec7tl7Rc8eCd 3STQna+m4h0A9x46ZPHybLpPXu1BxLH/khqbixLpj3I39yanSo8UgL84xzeXGj+B+LjZ bqjjeI5fiLeIlCrQt/D+CR1aUDfJ1qLttL+aaGHh3SAy5pxn5Wa67jAuPs0mnQG8WEkd 5juNgkmFRbNuWOklhJYKVv0jS1WCLwebyXjoplfikFmahKvZYafdEWTEKgh5hpV9zwbA CDdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to; bh=FOATPzIkg9K2y96+ccSYTlPUgOEimQ6fxamu1Y4p/UU=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=z+ga0JHfuWIDn9hqnyY5Bm3747Tsqovv0/40f44KEYi5eegZ/Sfaln+QrKQVwddCOz SKD7SLx0cwk1zui4zKS8h9og2OXu8fjn0LIT6SmtbFA+bW9z4xllNwJwYzX785HDDVvu PtTlB7SXHidqDe7l9FwTNtSCeMhB9hn8m8UsJEL8Vgb6WSL/O0s0QbeQDY0ByntBj6gT zejMHcbk42n7OQlakpFAuA1CWlg0Of2cYFDngoPShLsGVOQnrlb+b0aDlXPcOclt6KpE wyRVM1sAn0L5080QQG9QE0s6AwCBAOVCTLYcIYEDlfj4pHGA2CPvggGmaCwKNbNdhglK 1Stw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y6-20020a50e606000000b0052fcb647657si235775edm.420.2023.09.13.16.48.09; Wed, 13 Sep 2023 16:48:10 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E27C568CA0A; Thu, 14 Sep 2023 02:47:45 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CCC1668C948 for ; Thu, 14 Sep 2023 02:47:38 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id EC452240003 for ; Wed, 13 Sep 2023 23:47:37 +0000 (UTC) From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Thu, 14 Sep 2023 01:47:34 +0200 Message-Id: <20230913234734.22402-3-michael@niedermayer.cc> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230913234734.22402-1-michael@niedermayer.cc> References: <20230913234734.22402-1-michael@niedermayer.cc> X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 3/3] avcodec/aacdec_template: Better avoidance of signed integer overflow in imdct_and_windowing_eld() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: dZF2m7uF+T76 Fixes: 62171/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AAC_FIXED_fuzzer-5644657180409856 Fixes: signed integer overflow: 2 * 1079352273 cannot be represented in type 'int' Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/aacdec_template.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavcodec/aacdec_template.c b/libavcodec/aacdec_template.c index 51a4cb2b66f..954399f86bb 100644 --- a/libavcodec/aacdec_template.c +++ b/libavcodec/aacdec_template.c @@ -2856,8 +2856,8 @@ static void imdct_and_windowing_eld(AACContext *ac, SingleChannelElement *sce) ac->mdct512_fn(ac->mdct512, buf, in, sizeof(INTFLOAT)); for (i = 0; i < n; i+=2) { - buf[i + 0] = -(int)(USE_FIXED + 1U)*buf[i + 0]; - buf[i + 1] = (int)(USE_FIXED + 1U)*buf[i + 1]; + buf[i + 0] = -(UINTFLOAT)(USE_FIXED + 1)*buf[i + 0]; + buf[i + 1] = (UINTFLOAT)(USE_FIXED + 1)*buf[i + 1]; } // Like with the regular IMDCT at this point we still have the middle half // of a transform but with even symmetry on the left and odd symmetry on