From patchwork Wed Oct 10 23:45:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 10612 Delivered-To: ffmpegpatchwork@gmail.com Received: by 2002:ab0:73d2:0:0:0:0:0 with SMTP id m18csp1462432uaq; Wed, 10 Oct 2018 16:49:05 -0700 (PDT) X-Google-Smtp-Source: ACcGV63oRtdJgavA+RtbVoOZbC0Ymy74l1BGqE77ltb+cwLIsVjb9qu6u1vVhFcqC/Ya0Ox1CTqx X-Received: by 2002:a1c:2384:: with SMTP id j126-v6mr2448632wmj.124.1539215345263; Wed, 10 Oct 2018 16:49:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539215345; cv=none; d=google.com; s=arc-20160816; b=bPtTxlwRIVa/j3gP4s1C/52sBaSwljcZh6OSyzAi91gVnDWaNNgWgV7MWBKRlnMZPQ Y7jyYwY0xbTUwsNDU1T3dDlHWTqpOyYQyBxPwbL/+0/wUcNn8Q5UMgFSwjw4Th12SRcr O70aWoSeT/s4h6xe+pmV6jQndTT9iNUA8JNsqkqoRw4fFS5i994q6armmOhcDaIqrfxI SY7t848OV4s4ei2bmp3RZPlhD1ouz7OOyJ6vYCc2oYghq7Z/3tbnQx3eNf/OpvyNIhb9 pXs6AUSuM3TLUo5JufLDS5/2oc32J6+HkxjcYH4Jcx7KGsw7o014fGNeHbZ1cfl0OldH BMPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=Y94p482kFbkqTY6P4qWljs7l8UnCI1R5G++FSTv/uLs=; b=jF7PU1OL9j9WwyP5NdFlRzJjWHKLExzl/+NQHQUsmMz1hYB+/mEBqYnw9en9ThSy+X GfYgLLMv5KrLDGrmKIBXrH9d6D3q0nSATG2h5bZaZmPVXcxEV/qftfnz7KNLz6LgVtO0 2xlKZttBMU0L24fl+7DOngzQXd5lrDFFfSu5s98dBdFwBMHcAnqXannBR4ZWqE1/fW8/ pGyPDOeT5IrNTQjsHkJYPIq1wy3/9MqqutdIDmktf7xJ9rQy6Y5rN8L5sZ7KoyDCLjAh asFIFwuDc1hAbsAxFjoojcLLLmKSlilzWr5W9LRzA4ALpMaWx7KmXWVjTOxUnxzRBcre ongQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v25-v6si16756737wra.457.2018.10.10.16.49.04; Wed, 10 Oct 2018 16:49:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B21AA689FC6; Thu, 11 Oct 2018 02:48:39 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vie01a-qmta-pe02-1.mx.upcmail.net (vie01a-qmta-pe02-1.mx.upcmail.net [62.179.121.181]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 44E8968829D for ; Thu, 11 Oct 2018 02:48:33 +0300 (EEST) Received: from [172.31.218.51] (helo=vie01a-dmta-pe07-3.mx.upcmail.net) by vie01a-pqmta-pe02.mx.upcmail.net with esmtp (Exim 4.88) (envelope-from ) id 1gAODn-00018f-0T for ffmpeg-devel@ffmpeg.org; Thu, 11 Oct 2018 01:48:55 +0200 Received: from [172.31.216.43] (helo=vie01a-pemc-psmtp-pe01) by vie01a-dmta-pe07.mx.upcmail.net with esmtp (Exim 4.88) (envelope-from ) id 1gAODh-0001Pq-B9 for ffmpeg-devel@ffmpeg.org; Thu, 11 Oct 2018 01:48:49 +0200 Received: from localhost ([213.47.41.20]) by vie01a-pemc-psmtp-pe01 with SMTP @ mailcloud.upcmail.net id lzon1y0040S5wYM01zooUv; Thu, 11 Oct 2018 01:48:48 +0200 X-SourceIP: 213.47.41.20 From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Thu, 11 Oct 2018 01:45:31 +0200 Message-Id: <20181010234532.5857-3-michael@niedermayer.cc> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181010234532.5857-1-michael@niedermayer.cc> References: <20181010234532.5857-1-michael@niedermayer.cc> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/4] avcodec/ilbcdec: Fix multiple invalid left shifts X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Fixes: 10651/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ILBC_fuzzer-5202341540659200 Fixes: left shift of negative value -512 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/ilbcdec.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/libavcodec/ilbcdec.c b/libavcodec/ilbcdec.c index 76ecdf0e18..58044f4ba8 100644 --- a/libavcodec/ilbcdec.c +++ b/libavcodec/ilbcdec.c @@ -376,15 +376,15 @@ static void get_lsp_poly(int16_t *lsp, int32_t *f) for (j = i; j > 1; j--, l--) { high = f[l - 1] >> 16; - low = (f[l - 1] - (high << 16)) >> 1; + low = (f[l - 1] - (high * (1 << 16))) >> 1; - tmp = ((high * lsp[k]) << 2) + (((low * lsp[k]) >> 15) << 2); + tmp = ((high * lsp[k]) * 4) + (((low * lsp[k]) >> 15) * 4); f[l] += f[l - 2]; f[l] -= tmp; } - f[l] -= lsp[k] << 10; + f[l] -= lsp[k] * (1 << 10); l += i; } } @@ -1269,7 +1269,7 @@ static int xcorr_coeff(int16_t *target, int16_t *regressor, /* Calculate the total number of (dynamic) right shifts that have been performed on (cross_corr*cross_corr)/energy */ - totscale = energy_scale - (cross_corr_scale << 1); + totscale = energy_scale - (cross_corr_scale * 2); /* Calculate the shift difference in order to be able to compare the two (cross_corr*cross_corr)/energy in the same domain @@ -1322,7 +1322,7 @@ static void hp_output(int16_t *signal, const int16_t *ba, int16_t *y, tmp = (tmp >> 15); tmp += SPL_MUL_16_16(y[0], ba[3]); /* (-a[1])*y[i-1] (high part) */ tmp += SPL_MUL_16_16(y[2], ba[4]); /* (-a[2])*y[i-2] (high part) */ - tmp = (tmp << 1); + tmp = (tmp * 2); tmp += SPL_MUL_16_16(signal[i], ba[0]); /* b[0]*x[0] */ tmp += SPL_MUL_16_16(x[0], ba[1]); /* b[1]*x[i-1] */ @@ -1345,11 +1345,11 @@ static void hp_output(int16_t *signal, const int16_t *ba, int16_t *y, } else if (tmp < -268435456) { tmp = INT32_MIN; } else { - tmp = tmp << 3; + tmp = tmp * 8; } y[0] = tmp >> 16; - y[1] = (tmp - (y[0] << 16)) >> 1; + y[1] = (tmp - (y[0] * (1 << 16))) >> 1; } }