From patchwork Mon Dec 12 23:50:19 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Cadhalpun X-Patchwork-Id: 1769 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.65.86 with SMTP id o83csp1928899vsa; Mon, 12 Dec 2016 15:50:29 -0800 (PST) X-Received: by 10.28.36.193 with SMTP id k184mr23023wmk.40.1481586629606; Mon, 12 Dec 2016 15:50:29 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id sd16si46817027wjb.290.2016.12.12.15.50.28; Mon, 12 Dec 2016 15:50:29 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@googlemail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE dis=NONE) header.from=googlemail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0929D689F7C; Tue, 13 Dec 2016 01:50:22 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 69910689EC8 for ; Tue, 13 Dec 2016 01:50:16 +0200 (EET) Received: by mail-wm0-f67.google.com with SMTP id a20so14662785wme.2 for ; Mon, 12 Dec 2016 15:50:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=from:subject:to:references:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding; bh=2tuktaZuCQDHiMGM0SfSW1K8QXiZ5XcPBVPK2ceU6l8=; b=hUZWM9+fQLeEXV9HzyPqkLBbNMeHpVGBF75GWFjMV9kms/AWt6f/zcmMxS5444RRt6 swG9+bvw/RounhCpLfsVdXm3s9f2uixqM6d9PoSUSqcyAvt+IsdlV5zTJL9lCkqERpx6 jRZhez2sorhgHF4P9ruCaPlTyCLVsTp2x7wgTD2mT6gsqE2c66CLz04D4vY9wimWmIug ay+l7TLXlZIOF2TVejZm7rSfyon84KgoIQynuMkMB2nQ2CzkWTDGZXKO77vvEtm38LuQ ryiMglobweWLFa+FwR3OPLjtWAbNxGfEuX8BtQsgpP+yNpEdnOTSBqVhpj3t7xRVhox/ Exfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:subject:to:references:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=2tuktaZuCQDHiMGM0SfSW1K8QXiZ5XcPBVPK2ceU6l8=; b=K0bEnqFZj+uNwhkEkPVVPYangA/iS5p6kUZr4YYCvCd8XUTuKmXu0t7DsPttqX3p+J 86XLP38j2KxhHMFxRV73K7Q8IdQJyZQzn3KUGl1ywAols2RcjPtA5+p2adEchCD9SlG1 Op34OXA4QBvrT+JdsuNlj93xkJVFgdCvoC+FjyggMy2C5HRlDgjaFRb/12Y43XsA+9NB E7mK/HKURaVHYryvQdyXSGPI8MPQeAGD//HMh/S/v+jw04s5zZuyQn9LahS6uPak85KJ 5aRwjyaboxXkqo25q7SdKSJiLBblIQgDZiz0SxGGkGoj9P3J8mBJhY5kTX0EEDEKseKi fe5w== X-Gm-Message-State: AKaTC035HSgMjD8t1xYjZNFOPS+SyobryEbJiPYzD7EnaS9FWfkYH96XYyd/DCqUNlDvIg== X-Received: by 10.28.126.146 with SMTP id z140mr20724wmc.84.1481586620953; Mon, 12 Dec 2016 15:50:20 -0800 (PST) Received: from [192.168.2.21] (p5B095619.dip0.t-ipconnect.de. [91.9.86.25]) by smtp.googlemail.com with ESMTPSA id c187sm37666440wmd.13.2016.12.12.15.50.20 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Dec 2016 15:50:20 -0800 (PST) From: Andreas Cadhalpun X-Google-Original-From: Andreas Cadhalpun To: FFmpeg development discussions and patches References: <18a6d792-ed50-8963-ccfd-8c585824682f@googlemail.com> Message-ID: Date: Tue, 13 Dec 2016 00:50:19 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Icedove/45.4.0 MIME-Version: 1.0 In-Reply-To: <18a6d792-ed50-8963-ccfd-8c585824682f@googlemail.com> Subject: [FFmpeg-devel] [PATCH 3/3] tiff: fix overflows when calling av_readuce X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The arguments of av_reduce are signed, so the cast to uint64_t is misleading. Signed-off-by: Andreas Cadhalpun --- libavcodec/tiff.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/libavcodec/tiff.c b/libavcodec/tiff.c index 4721e94..12ef419 100644 --- a/libavcodec/tiff.c +++ b/libavcodec/tiff.c @@ -772,9 +772,16 @@ static void set_sar(TiffContext *s, unsigned tag, unsigned num, unsigned den) int offset = tag == TIFF_YRES ? 2 : 0; s->res[offset++] = num; s->res[offset] = den; - if (s->res[0] && s->res[1] && s->res[2] && s->res[3]) + if (s->res[0] && s->res[1] && s->res[2] && s->res[3]) { + uint64_t num = s->res[2] * (uint64_t)s->res[1]; + uint64_t den = s->res[0] * (uint64_t)s->res[3]; + if (num > INT64_MAX || den > INT64_MAX) { + num = num >> 1; + den = den >> 1; + } av_reduce(&s->avctx->sample_aspect_ratio.num, &s->avctx->sample_aspect_ratio.den, - s->res[2] * (uint64_t)s->res[1], s->res[0] * (uint64_t)s->res[3], INT32_MAX); + num, den, INT32_MAX); + } } static int tiff_decode_tag(TiffContext *s, AVFrame *frame)