From patchwork Sun Nov 5 20:20:08 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 5873 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.90 with SMTP id m26csp2195548jah; Sun, 5 Nov 2017 12:20:22 -0800 (PST) X-Google-Smtp-Source: ABhQp+QSFpNe1RNkqxXvctea2GJw0+3Uqk3fac4kKGkCgiIlLK5T9DYCIMOkxSnb0588DusRCZ+p X-Received: by 10.223.200.134 with SMTP id k6mr10123955wrh.159.1509913222150; Sun, 05 Nov 2017 12:20:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1509913222; cv=none; d=google.com; s=arc-20160816; b=ovus8zB5I9PukD1nk11GEcFVeVJ0RjbeKY6cy9c+B6ZKTJrP0Dg2HqLepwjq521W+v UIOFQnsuBE/SZRS71W/LKLmeuYCJORMegV2aHnhGJyerQdue9wr2cgyp24CYSLIfkKHs QfK5eH7LgPkYB1DvhDvVYH5gNuFba27BmyC9pqsWS5e5tOx52b0J2bkQsP7FfvDXun4S d+z5LiWVq8pHOIPLHrPyfHYij02l7u7BvDcj1+uEGTntfVIA5NFxZZgT4to7wsSZ1Lvt 5Z3YAZQEeTtPlqgBoBpE+hKVioUWis4nrA+WNFaOYhMiEad+PLbuYeBMsyZgiWDj8jPc l8Fg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to:arc-authentication-results; bh=2mi8KukoXHbaJqDggazI09cyOG8iH5rbMTfndAKSIKI=; b=CzWZbxqR5QTucDWSZnSwM7ahSWtcQmaog4b+FNUruVFXOByMzr4zTz3hQbSWj1UdYU IgSFohJYD1RctBqehYEXGdhVeqDJ2JQEqoOXUOvdS6UD8rLtC2kmshyEiLs8DHbeyQFR JkL7HS/He0Xs077Yen5PX0bifFcCx2ToSHNa2nrVdrTNDb50Hq+FWj91fzi8LkM3vJ39 Iw5IyMvh5HZCK/wColGqAPm6lBVj1n0OUnDHJaE6zQxAvHq0P72ZlvZM+Cg/7wGF+mrW Rl93iRoYPxRnKIIu2TIHPX56VXbcJlBUSnBMPDFwzKOrMGS6yKfG74bJDbvvO8f1y6W3 aX+w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 31si9283756wrm.180.2017.11.05.12.20.21; Sun, 05 Nov 2017 12:20:22 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 03951689C26; Sun, 5 Nov 2017 22:20:09 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vie01a-dmta-pe03-3.mx.upcmail.net (vie01a-dmta-pe03-3.mx.upcmail.net [62.179.121.162]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3FB5D689E5F for ; Sun, 5 Nov 2017 22:20:03 +0200 (EET) Received: from [172.31.216.43] (helo=vie01a-pemc-psmtp-pe01) by vie01a-dmta-pe03.mx.upcmail.net with esmtp (Exim 4.88) (envelope-from ) id 1eBROw-0004Fd-IO for ffmpeg-devel@ffmpeg.org; Sun, 05 Nov 2017 21:20:14 +0100 Received: from localhost ([213.47.41.20]) by vie01a-pemc-psmtp-pe01 with SMTP @ mailcloud.upcmail.net id WLLC1w0190S5wYM01LLDoM; Sun, 05 Nov 2017 21:20:13 +0100 X-SourceIP: 213.47.41.20 From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Sun, 5 Nov 2017 21:20:08 +0100 Message-Id: <20171105202008.29966-4-michael@niedermayer.cc> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20171105202008.29966-1-michael@niedermayer.cc> References: <20171105202008.29966-1-michael@niedermayer.cc> Subject: [FFmpeg-devel] [PATCH 4/4] avcodec/aacpsdsp_template: Fix integer overflows in ps_decorrelate_c() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Fixes: runtime error: signed integer overflow: 1939661764 - -454942263 cannot be represented in type 'int' Fixes: 3191/clusterfuzz-testcase-minimized-5688798451073024 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/aacpsdsp_template.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/libavcodec/aacpsdsp_template.c b/libavcodec/aacpsdsp_template.c index e35e9699b0..19be200653 100644 --- a/libavcodec/aacpsdsp_template.c +++ b/libavcodec/aacpsdsp_template.c @@ -130,12 +130,12 @@ static void ps_decorrelate_c(INTFLOAT (*out)[2], INTFLOAT (*delay)[2], INTFLOAT apd_im = in_im; in_re = AAC_MSUB30(link_delay_re, fractional_delay_re, link_delay_im, fractional_delay_im); - in_re -= a_re; + in_re -= (UINTFLOAT)a_re; in_im = AAC_MADD30(link_delay_re, fractional_delay_im, link_delay_im, fractional_delay_re); - in_im -= a_im; - ap_delay[m][n+5][0] = apd_re + AAC_MUL31(ag[m], in_re); - ap_delay[m][n+5][1] = apd_im + AAC_MUL31(ag[m], in_im); + in_im -= (UINTFLOAT)a_im; + ap_delay[m][n+5][0] = apd_re + (UINTFLOAT)AAC_MUL31(ag[m], in_re); + ap_delay[m][n+5][1] = apd_im + (UINTFLOAT)AAC_MUL31(ag[m], in_im); } out[n][0] = AAC_MUL16(transient_gain[n], in_re); out[n][1] = AAC_MUL16(transient_gain[n], in_im);