From patchwork Sun Jul 9 16:27:58 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 4275 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.1.76 with SMTP id 73csp2552954vsb; Sun, 9 Jul 2017 09:28:21 -0700 (PDT) X-Received: by 10.223.152.20 with SMTP id v20mr5808733wrb.8.1499617701310; Sun, 09 Jul 2017 09:28:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1499617701; cv=none; d=google.com; s=arc-20160816; b=LSZQe1k9Ioip09vhA7zqCuMsL2nT+1GQxUXi0kcFhCaM8NNCMSuxH/ZELcLElQi1cc FDIvRQNVOlEM2Yitmk3v7/RgVM8FyxSeXZYsYmXSufqkmS3VNDsfYLZve3/C794JnELK 50G+dV3q31YCwKNzsT1QzPIXLEGISPud+92hCAkLZ/rDW1YLN8jDkxtOgggn5RnQw76a QCoNrOY/a+PNSNtvT8s162ZZ5fwjSLsm7nWQj/r3L56heEXo6PSgayGaDfHWLHj1o6Qa AdgOl+nZwDOp8WeoIY0DijtnJV+UDmtk4mpwdgUWgCdWYq3ptmW13nbDkfz/n28k6BTK d1GQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:delivered-to :arc-authentication-results; bh=mrvjYaphBGgQShOZzAj845H9n2kpJb0IJymFfOI8srM=; b=GjPQ7iXixHTVHsMnl+L0Q/N3P34wKI1APXjXyNg+Gt2iXotMWVfGCwT7TuTusQzmFS 23mSgMMWEvAxIRNkUiOEhPaX0wuG5vfPyrP+3cde6N/A/MlinZkeEUkg+OtI7Oi211jk I0q+P2uDQsERyaNPvWJYB8uOZzCx7/ga8Ry0gwCIRyoLdCUM9QevYblaPoNiHV10DhfU B96UZcST10Oey3XGHCYKCWq7ZR40YNkJU4xb6DmhAMDb++AC9/2Ho9wBczYEnO4P4CJu YwCF8ovUJc+EjcTjYAQOr2gfpQ12vW/f/F5p0PhUMZmJslqdDQi5qMPPShvFrtx9ZgGN SEmQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id p3si4260654wmi.108.2017.07.09.09.28.20; Sun, 09 Jul 2017 09:28:21 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1D3D5689CED; Sun, 9 Jul 2017 19:28:15 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vie01a-qmta-pe01-3.mx.upcmail.net (vie01a-qmta-pe01-3.mx.upcmail.net [62.179.121.180]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9FED4689C64 for ; Sun, 9 Jul 2017 19:28:08 +0300 (EEST) Received: from [172.31.218.33] (helo=vie01a-dmta-pe01-3.mx.upcmail.net) by vie01a-pqmta-pe01.mx.upcmail.net with esmtp (Exim 4.88) (envelope-from ) id 1dUF47-0006o6-W1 for ffmpeg-devel@ffmpeg.org; Sun, 09 Jul 2017 18:28:12 +0200 Received: from [172.31.216.43] (helo=vie01a-pemc-psmtp-pe01) by vie01a-dmta-pe01.mx.upcmail.net with esmtp (Exim 4.88) (envelope-from ) id 1dUF42-0007I1-53 for ffmpeg-devel@ffmpeg.org; Sun, 09 Jul 2017 18:28:06 +0200 Received: from localhost ([213.47.41.20]) by vie01a-pemc-psmtp-pe01 with SMTP @ mailcloud.upcmail.net id igTy1v01j0S5wYM01gTz3z; Sun, 09 Jul 2017 18:28:00 +0200 X-SourceIP: 213.47.41.20 From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Sun, 9 Jul 2017 18:27:58 +0200 Message-Id: <20170709162758.1033-1-michael@niedermayer.cc> X-Mailer: git-send-email 2.13.0 Subject: [FFmpeg-devel] [PATCH] avcodec/aacps (fixed point): Fix multiple signed integer overflows X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Fixes: runtime error: signed integer overflow: 1421978265 - -1810326882 cannot be represented in type 'int' Fixes: 2527/clusterfuzz-testcase-minimized-5260915396050944 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/aacps.c | 25 ++++++++----------------- 1 file changed, 8 insertions(+), 17 deletions(-) diff --git a/libavcodec/aacps.c b/libavcodec/aacps.c index 473da7bd43..5758b919a1 100644 --- a/libavcodec/aacps.c +++ b/libavcodec/aacps.c @@ -697,26 +697,17 @@ static void decorrelation(PSContext *ps, INTFLOAT (*out)[32][2], const INTFLOAT for (i = 0; i < NR_PAR_BANDS[is34]; i++) { for (n = n0; n < nL; n++) { int decayed_peak; - int denom; - decayed_peak = (int)(((int64_t)peak_decay_factor * \ peak_decay_nrg[i] + 0x40000000) >> 31); peak_decay_nrg[i] = FFMAX(decayed_peak, power[i][n]); - power_smooth[i] += (power[i][n] - power_smooth[i] + 2) >> 2; - peak_decay_diff_smooth[i] += (peak_decay_nrg[i] - power[i][n] - \ - peak_decay_diff_smooth[i] + 2) >> 2; - denom = peak_decay_diff_smooth[i] + (peak_decay_diff_smooth[i] >> 1); - if (denom > power_smooth[i]) { - int p = power_smooth[i]; - while (denom < 0x40000000) { - denom <<= 1; - p <<= 1; - } - transient_gain[i][n] = p / (denom >> 16); - } - else { - transient_gain[i][n] = 1 << 16; - } + power_smooth[i] += (power[i][n] + 2LL - power_smooth[i]) >> 2; + peak_decay_diff_smooth[i] += (peak_decay_nrg[i] + 2LL - power[i][n] - \ + peak_decay_diff_smooth[i]) >> 2; + + if (peak_decay_diff_smooth[i]) { + transient_gain[i][n] = FFMIN(power_smooth[i]*43691LL / peak_decay_diff_smooth[i], 1<<16); + } else + transient_gain[i][n] = 1 << 16; } } #else