From patchwork Sun Jan 7 22:13:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 7187 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.78.2 with SMTP id r2csp1956102jaa; Sun, 7 Jan 2018 14:13:56 -0800 (PST) X-Google-Smtp-Source: ACJfBounKiTK72QPExJDrvg7kUEAY7XrWyYn8tRVlJNpIXzUXBLBcHoqeiqPfVJjMiUDhLgaixgo X-Received: by 10.223.188.78 with SMTP id a14mr9320233wrh.267.1515363236484; Sun, 07 Jan 2018 14:13:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1515363236; cv=none; d=google.com; s=arc-20160816; b=qz8FnROpMfPNMVI5cEFSdyBANuBw1/ZEchWW3Ongk91vFljK9hjbMCX0jNwNWacGTH y6PjrHr1wKSPz2bWOJH+AjwI86WtH92P/RJ9be8iyD4qNVOhRe2vOsa4xaSmG3ZTvjFw iLPXssrbpoUyKQb0HQEGzHmuRmH5GgZf7MmpGtkjwUidThyUFCaPVKtKsXjBEuwGcJsb l7GnuJpxxIuqjl6Fo5T/N7wJXyGqvJfpJbOSL5UmD2OBvVaFOxe6xfiOIb1VSTAAve4w XwdH/wfivqUe7YPARt6hldIFxnm5lyFKUrF8FzMtUfBD88Hwpx03r6RDcKbkMeizs/hy t5Tg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to:arc-authentication-results; bh=3tiGL7RSdzgR8ZYxXbrE/i8209taznEqGXtS9fidlPE=; b=WwT2F5RvX2HFkcKLN6PP1RGhQwA3mT+OJPm7p2tW7C74+vdjTasH6wYOMG0oXWGI7+ tPH31pHcX7c3o7xZCOqLrog6ASXKB5XLCiesPJI8ONuwi0aSPdSReLukFAemp6lp55Vq nwooK880GaFy8Gulkc20lcMu4KtB8QYb9pyUB5NYyexiKrGD/NPU2EOP3yc27j0Q2cdb AawvM0XspZpJVGPm5uhLXEJJ5j9i/lOWZsKU4BWGq4QVQnrECn1/YpbZYw2wf0bWcY4/ jAlYlYKzxHtXCVS9SrE8o5P1kvE/ERgIwaQoARyeoTMp5QOwvbv3nG9wKvjZcaaPTJAQ xeWA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id n18si7008878wmh.43.2018.01.07.14.13.56; Sun, 07 Jan 2018 14:13:56 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 24C6C689E86; Mon, 8 Jan 2018 00:13:51 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vie01a-dmta-pe05-1.mx.upcmail.net (vie01a-dmta-pe05-1.mx.upcmail.net [84.116.36.11]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C0632689AB3 for ; Mon, 8 Jan 2018 00:13:44 +0200 (EET) Received: from [172.31.216.43] (helo=vie01a-pemc-psmtp-pe01) by vie01a-dmta-pe05.mx.upcmail.net with esmtp (Exim 4.88) (envelope-from ) id 1eYJCK-0007MO-VK for ffmpeg-devel@ffmpeg.org; Sun, 07 Jan 2018 23:13:44 +0100 Received: from localhost ([213.47.41.20]) by vie01a-pemc-psmtp-pe01 with SMTP @ mailcloud.upcmail.net id vaDf1w01J0S5wYM01aDgCU; Sun, 07 Jan 2018 23:13:40 +0100 X-SourceIP: 213.47.41.20 From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Sun, 7 Jan 2018 23:13:37 +0100 Message-Id: <20180107221337.23431-2-michael@niedermayer.cc> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180107221337.23431-1-michael@niedermayer.cc> References: <20180107221337.23431-1-michael@niedermayer.cc> Subject: [FFmpeg-devel] [PATCH 2/2] avcodec/dirac_dwt: Fix overflows in COMPOSE_HAARiH0/COMPOSE_HAARiL0 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Fixes: 4830/clusterfuzz-testcase-minimized-5255392054476800 Fixes: signed integer overflow: 2147483646 - -7 cannot be represented in type 'int' Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/dirac_dwt.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavcodec/dirac_dwt.h b/libavcodec/dirac_dwt.h index f9828d95a4..1af41e0702 100644 --- a/libavcodec/dirac_dwt.h +++ b/libavcodec/dirac_dwt.h @@ -105,10 +105,10 @@ void ff_spatial_idwt_slice2(DWTContext *d, int y); (int)(((unsigned)(b2) - ((int)(-b0 + 9U*b1 + 9U*b3 - b4 + 16) >> 5))) #define COMPOSE_HAARiL0(b0, b1)\ - (b0 - ((b1 + 1) >> 1)) + ((int)(b0 - (unsigned)((int)(b1 + 1U) >> 1))) #define COMPOSE_HAARiH0(b0, b1)\ - (b0 + b1) + ((int)(b0 + (unsigned)(b1))) #define COMPOSE_FIDELITYiL0(b0, b1, b2, b3, b4, b5, b6, b7, b8)\ ((unsigned)b4 - ((int)(-8*(b0+(unsigned)b8) + 21*(b1+(unsigned)b7) - 46*(b2+(unsigned)b6) + 161*(b3+(unsigned)b5) + 128) >> 8))