From patchwork Mon Nov 20 13:58:15 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 6227 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.94 with SMTP id m30csp3939929jah; Mon, 20 Nov 2017 05:58:34 -0800 (PST) X-Google-Smtp-Source: AGs4zMYMFzGx9zbAYBOoh7PA/qFW6KNSkoSBu6DjrwHwzb3OfWnkaJfgycRDNcv8+WhNrzktXMgP X-Received: by 10.28.232.129 with SMTP id f1mr9640965wmi.32.1511186314344; Mon, 20 Nov 2017 05:58:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1511186314; cv=none; d=google.com; s=arc-20160816; b=goPUq5OFPwv0lDEvwmqFmjsBCJf8Qw58Jt8zhkgcgBDxlcLoTtT0tiRZ/j9h5NRpzb bJIGo6Zzsvz2GDTsdMSFIWyjWjo88ayeRx6E6DDf7EW1t7cfQqOWzKHJ2HsuyGRJ7z1m oksbRTRgT5mheg0NcqZkY0mBDmIKFmdZ32ZksLMcVGLht6sbseVOTWconbOZ0JScGpJJ kGFLm6cS1yYiN6y+0muAfDSxp8jEri5LT4UsDyvzafKpXOrnKB5GRKU3duhmpdzc3rxw NekXhcaXFiNq8hecxhgcskmPaBKdMPIVc/99J4R8ohS9eHTBcgDW35h14VJEe9g1h/Og g5ng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:delivered-to :arc-authentication-results; bh=xr8MIQ7ZF57K+XFZ+a5LLg+GFvG14UTlSGA1h8kwJvw=; b=gzXxYCl3jj8eLvHfGKquR9SKLouLDBF8kjvBq7E1XESypnJD5jvYO0Q+JuGK2I8sKc K2VUb6ulc+cujMg4fLDzwv6aDjrPftkkVqsvqLts0ulGJ0YfuTlINi5GJjCSzT8nyaxK M6BSi4Dy8Wk9h1FhriBM5yKVRxpLeZs/f06R1BsGMnb/qdGR0LH2YEl0QXqwoaWV7khT M24Btq91XWJ9gAgzhmlmCX5sSa0e6UCYvKHGI/GYu7cjuAJ5EEwfqNxKZt27WQ7yNej/ yIseEm6VxNKmmhD3onFgKkCCOHb7b5Nu1fSYpjfEFqIvrXc3DvyWnWQEgb5UiCxDFa4+ Uwng== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x1si7622571wrd.29.2017.11.20.05.58.33; Mon, 20 Nov 2017 05:58:34 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1BE1B689C89; Mon, 20 Nov 2017 15:58:15 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vie01a-dmta-pe05-2.mx.upcmail.net (vie01a-dmta-pe05-2.mx.upcmail.net [84.116.36.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0DACA689C89 for ; Mon, 20 Nov 2017 15:58:08 +0200 (EET) Received: from [172.31.216.43] (helo=vie01a-pemc-psmtp-pe01) by vie01a-dmta-pe05.mx.upcmail.net with esmtp (Exim 4.88) (envelope-from ) id 1eGmae-0000z7-FI for ffmpeg-devel@ffmpeg.org; Mon, 20 Nov 2017 14:58:24 +0100 Received: from localhost ([213.47.41.20]) by vie01a-pemc-psmtp-pe01 with SMTP @ mailcloud.upcmail.net id cDyF1w00d0S5wYM01DyGh2; Mon, 20 Nov 2017 14:58:16 +0100 X-SourceIP: 213.47.41.20 From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Mon, 20 Nov 2017 14:58:15 +0100 Message-Id: <20171120135815.20251-1-michael@niedermayer.cc> X-Mailer: git-send-email 2.15.0 Subject: [FFmpeg-devel] [PATCH] avcodec/h264idct_template: Fix integer overflow in ff_h264_idct8_add X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Fixes: signed integer overflow: 452986184 - -2113885312 cannot be represented in type 'int' Fixes: 4196/clusterfuzz-testcase-minimized-5580648594014208 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/h264idct_template.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/libavcodec/h264idct_template.c b/libavcodec/h264idct_template.c index 7526bdd812..5993ae2e6e 100644 --- a/libavcodec/h264idct_template.c +++ b/libavcodec/h264idct_template.c @@ -76,10 +76,10 @@ void FUNCC(ff_h264_idct8_add)(uint8_t *_dst, int16_t *_block, int stride){ for( i = 0; i < 8; i++ ) { - const unsigned int a0 = block[i+0*8] + block[i+4*8]; - const unsigned int a2 = block[i+0*8] - block[i+4*8]; - const unsigned int a4 = (block[i+2*8]>>1) - block[i+6*8]; - const unsigned int a6 = (block[i+6*8]>>1) + block[i+2*8]; + const unsigned int a0 = block[i+0*8] + (unsigned)block[i+4*8]; + const unsigned int a2 = block[i+0*8] - (unsigned)block[i+4*8]; + const unsigned int a4 = (block[i+2*8]>>1) - (unsigned)block[i+6*8]; + const unsigned int a6 = (block[i+6*8]>>1) + (unsigned)block[i+2*8]; const unsigned int b0 = a0 + a6; const unsigned int b2 = a2 + a4;