From patchwork Sat Nov 25 02:43:29 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 6347 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.94 with SMTP id m30csp2979634jah; Fri, 24 Nov 2017 18:43:48 -0800 (PST) X-Google-Smtp-Source: AGs4zMYUOJwSytJh2oM8CKSsqe6879Tq7ekdNvUwB8Uuld3RGH00AcAXu3+9vXsfZBv7wOcO3cHL X-Received: by 10.28.234.80 with SMTP id i77mr10640544wmh.76.1511577828395; Fri, 24 Nov 2017 18:43:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1511577828; cv=none; d=google.com; s=arc-20160816; b=mBV0u4imUGN4P0VxH/WbSB238ih3Guuophqk4VvZfzyMvu6rM6Zbe6qxuFvvhSggVr CPuk0dJDJ6uNa/TGTv55/XpUGRVEZ754hNPbk6AoGB7Njc5IfPhJZNh7ECxQjRllWxlw hLncrD+wVYeI2EvZV3x6lACzzo1GVZ3vy42QIy+P+NpF2RvVSWEjCSUQRCpjRTZbyyZw M7ri08P4D5qvTNJCoze5W/6UH4ymA87hRl9/tXU4IXtzCktnpW2uhHjBrbISLcVsCZ/D LM5RgVuBlfVykiLCFpAeS0CH/lhHGA8Yoff3yCom729NGo/CNblFREz3F2SAKVRlBeVJ FrXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:delivered-to :arc-authentication-results; bh=+D+Ku8qPrWoEdDOgFsxGLv530O+ELhyKTo3JKJRdOUo=; b=NnsAYQlDY+hhviEqWpoqsvtDMvkiTn/oY0XhQA5VpSEgBCpLYMkvvBfyHc1uo40HCN FtxaHeBJ90SWaWWagPkaN6G/iFYYb6oRwdW58J5FANZFNvtaHZIOTt52FOp/GYiWsyfc Zo0jaFScydb0SSLO7yRzWmBfw8NJ1fWKkUB94Irb7YmzTZvJb/dLQ5lwZDnELTjq/3y2 p1jqxculmJOLBfCZgrOy4E7EyoQNrd4Cf1wyOUUYuhE0VH3RqqLqQ6sbxymFDfBlunUN /Sj4qUO4Q7WTnBgWjVub3/zyWHkxUR9lT+yntESU0ZJCsSfm42NWSjvIlARXu+dZUWDZ sZpw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id l44si12058523wre.375.2017.11.24.18.43.47; Fri, 24 Nov 2017 18:43:48 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E41C068A29A; Sat, 25 Nov 2017 04:43:45 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vie01a-dmta-pe04-3.mx.upcmail.net (vie01a-dmta-pe04-3.mx.upcmail.net [62.179.121.165]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 741CB68A28B for ; Sat, 25 Nov 2017 04:43:39 +0200 (EET) Received: from [172.31.216.43] (helo=vie01a-pemc-psmtp-pe01) by vie01a-dmta-pe04.mx.upcmail.net with esmtp (Exim 4.88) (envelope-from ) id 1eIQRP-0000gN-Ot for ffmpeg-devel@ffmpeg.org; Sat, 25 Nov 2017 03:43:39 +0100 Received: from localhost ([213.47.41.20]) by vie01a-pemc-psmtp-pe01 with SMTP @ mailcloud.upcmail.net id e2jW1w00W0S5wYM012jXfW; Sat, 25 Nov 2017 03:43:31 +0100 X-SourceIP: 213.47.41.20 From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Sat, 25 Nov 2017 03:43:29 +0100 Message-Id: <20171125024329.24991-1-michael@niedermayer.cc> X-Mailer: git-send-email 2.15.0 Subject: [FFmpeg-devel] [PATCH] avcodec/dirac_dwt: Fix integer overflow in COMPOSE_FIDELITYi* X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Fixes: runtime error: signed integer overflow: -2143827186 - 7404944 cannot be represented in type 'int' Fixes: 4354/clusterfuzz-testcase-minimized-4671122764201984 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/dirac_dwt.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavcodec/dirac_dwt.h b/libavcodec/dirac_dwt.h index f9a9e9e1b3..eb5aebc878 100644 --- a/libavcodec/dirac_dwt.h +++ b/libavcodec/dirac_dwt.h @@ -111,10 +111,10 @@ void ff_spatial_idwt_slice2(DWTContext *d, int y); (b0 + b1) #define COMPOSE_FIDELITYiL0(b0, b1, b2, b3, b4, b5, b6, b7, b8)\ - (b4 - ((int)(-8*(b0+(unsigned)b8) + 21*(b1+(unsigned)b7) - 46*(b2+(unsigned)b6) + 161*(b3+(unsigned)b5) + 128) >> 8)) + ((unsigned)b4 - ((int)(-8*(b0+(unsigned)b8) + 21*(b1+(unsigned)b7) - 46*(b2+(unsigned)b6) + 161*(b3+(unsigned)b5) + 128) >> 8)) #define COMPOSE_FIDELITYiH0(b0, b1, b2, b3, b4, b5, b6, b7, b8)\ - (b4 + ((int)(-2*(b0+(unsigned)b8) + 10*(b1+(unsigned)b7) - 25*(b2+(unsigned)b6) + 81*(b3+(unsigned)b5) + 128) >> 8)) + ((unsigned)b4 + ((int)(-2*(b0+(unsigned)b8) + 10*(b1+(unsigned)b7) - 25*(b2+(unsigned)b6) + 81*(b3+(unsigned)b5) + 128) >> 8)) #define COMPOSE_DAUB97iL1(b0, b1, b2)\ (b1 - ((int)(1817*(b0 + (unsigned)b2) + 2048) >> 12))