From patchwork Fri Sep 8 21:29:12 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 5058 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.36.26 with SMTP id f26csp671696jaa; Fri, 8 Sep 2017 14:29:39 -0700 (PDT) X-Google-Smtp-Source: AOwi7QDKNWAbjTDtEilulPFy905on0fjYP1W+TmISnsQwc31YlWJEMNuajDDNFN7GEVS3hasTJEH X-Received: by 10.28.30.13 with SMTP id e13mr2468064wme.72.1504906179256; Fri, 08 Sep 2017 14:29:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1504906179; cv=none; d=google.com; s=arc-20160816; b=Y3YnWt9cPRpdccruB0nc3LtErJfPMXoSvK+025hB30ZfZXE47lzNlCACAHsdbVGPVx DVUfJPDCZCe6rxz9Y37TrRfQ/zgo0UjV2ogzCHTnr5upjcMa20L8/UUj1QfhCdm4768B sJo6v0ITEocVXdfJhBiCfP4129wl38EiAWKBSbKd40Sr85iRNqSpMTdKcicITqWF7zR/ bQIUI4wgVFqk0MHN25iP+RMUgDCx59pvG+6j6BV4W2g5JzYNEaDVwnY9aTaUzPab7yKS V5dtJddqoXyAx/ltM2OdrI0+H8P+mmQje61ZTDyxRX8XM9EVNQfJXRuGXbcB1nvakCSY mZJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to:arc-authentication-results; bh=unYG2aRo7uUJSLHgGr195M0po1dPA9pDO3lo9+kGMEg=; b=JqVW+lADpum8DJAqCdDKb+9KJ1Ymejgyj+Z9eCIj+VIJz9CX5dsVxLUEB0NhynXd2w DTEnsA2pv/IR2UeFsXm3hoSGCqwstJ1x6ycfkmAuv9596e+PKD+ks/19yrUqPIjDmnQ4 s+ANFZUukXacNoG9No5mms+TBtLgyVGVxmToMcC50T1m03OOjvtxHqdFjWYdnDfhtgf4 aVvBjYM00MS6hOnsXWfem3b0v2zCjrdiu19EyGRxA/xWZ9ZWSFq1HQJVJrflKDZ8DAxY RtIm/UyDjP1+DQB2hnlquM58YXXiSIrmo2HY+GALhehtVKuo87jyknLbLxH1M1sAAMcL lntw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 2si2041761wms.238.2017.09.08.14.29.37; Fri, 08 Sep 2017 14:29:39 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6D897689CEA; Sat, 9 Sep 2017 00:29:29 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vie01a-dmta-pe04-3.mx.upcmail.net (vie01a-dmta-pe04-3.mx.upcmail.net [62.179.121.165]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 95AF0689C63 for ; Sat, 9 Sep 2017 00:29:23 +0300 (EEST) Received: from [172.31.216.43] (helo=vie01a-pemc-psmtp-pe01) by vie01a-dmta-pe04.mx.upcmail.net with esmtp (Exim 4.88) (envelope-from ) id 1dqQq7-0000ar-2x for ffmpeg-devel@ffmpeg.org; Fri, 08 Sep 2017 23:29:27 +0200 Received: from localhost ([213.47.41.20]) by vie01a-pemc-psmtp-pe01 with SMTP @ mailcloud.upcmail.net id 79VF1w00L0S5wYM019VGkd; Fri, 08 Sep 2017 23:29:16 +0200 X-SourceIP: 213.47.41.20 From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Fri, 8 Sep 2017 23:29:12 +0200 Message-Id: <20170908212913.20478-2-michael@niedermayer.cc> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20170908212913.20478-1-michael@niedermayer.cc> References: <20170908212913.20478-1-michael@niedermayer.cc> Subject: [FFmpeg-devel] [PATCH 2/3] avcodec/jpeg2000dsp: Fix multiple integer overflows in ict_int() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Fixes: runtime error: signed integer overflow: 22553 * -188962 cannot be represented in type 'int' Fixes: 3042/clusterfuzz-testcase-minimized-5174210131394560 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/jpeg2000dsp.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavcodec/jpeg2000dsp.c b/libavcodec/jpeg2000dsp.c index c746aed924..cae73b5b61 100644 --- a/libavcodec/jpeg2000dsp.c +++ b/libavcodec/jpeg2000dsp.c @@ -65,9 +65,9 @@ static void ict_int(void *_src0, void *_src1, void *_src2, int csize) for (i = 0; i < csize; i++) { i0 = *src0 + *src2 + (((26345 * *src2) + (1 << 15)) >> 16); - i1 = *src0 - (((i_ict_params[1] * *src1) + (1 << 15)) >> 16) + i1 = *src0 - ((int)((22553U * *src1) + (1 << 15)) >> 16) - (((i_ict_params[2] * *src2) + (1 << 15)) >> 16); - i2 = *src0 + (2 * *src1) + (((-14942 * *src1) + (1 << 15)) >> 16); + i2 = *src0 + (2 * *src1) + ((int)((-14942U * *src1) + (1 << 15)) >> 16); *src0++ = i0; *src1++ = i1; *src2++ = i2;