From patchwork Tue Dec 13 23:57:12 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Cadhalpun X-Patchwork-Id: 1775 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.65.86 with SMTP id o83csp2481825vsa; Tue, 13 Dec 2016 15:57:25 -0800 (PST) X-Received: by 10.194.149.143 with SMTP id ua15mr102882310wjb.48.1481673445162; Tue, 13 Dec 2016 15:57:25 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id jv4si51687633wjb.64.2016.12.13.15.57.24; Tue, 13 Dec 2016 15:57:25 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@googlemail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE dis=NONE) header.from=googlemail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5B8B6689978; Wed, 14 Dec 2016 01:57:16 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wj0-f196.google.com (mail-wj0-f196.google.com [209.85.210.196]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5EA846891DF for ; Wed, 14 Dec 2016 01:57:10 +0200 (EET) Received: by mail-wj0-f196.google.com with SMTP id xy5so714919wjc.1 for ; Tue, 13 Dec 2016 15:57:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=from:subject:to:references:message-id:date:user-agent:mime-version :in-reply-to; bh=Dm40xM53IVbGKk8Ho2HboRaxL8c5QIzE3Of+bm1exuE=; b=BkP8jTQ8jQi39GHvl10iSOLvbE4b7TJ3b/QyCSIRkxo6Ia4OaDLEIuL89MU2PyC3OV zDzoHib4wnCNM7vDZdAXoLPAUT4GHiOrF/3TGzhFU4+QYsCOR4U9kj/HB2SdKocPBNUx yxkX7IhYjTxqZpkTRxCGX1Fcq3piQdX2Z0pfHQb14iCB4xYHsMJOwcGikevKAnVt3P42 o/6pZD+pWknrQFRpvGw380DIYIeJpzmT3BGuJxZcL+d7qMMT9PBBXBAXqtYtvo0Wpl1U NfcztSUpObpphdSlNiW7BBkiJmmbeeG4qc7sdhvL1hnm59QQb2XzuTQe+3DYDyHvjDex OZPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:subject:to:references:message-id:date :user-agent:mime-version:in-reply-to; bh=Dm40xM53IVbGKk8Ho2HboRaxL8c5QIzE3Of+bm1exuE=; b=ixQH7DyZXpcaAwczh46NijLdqhulNr3jLRcYu0FjnsVVRHtM4Vq9uWRs2roIazHc1O tr/OW2TRJaBzWG3tZ1Wx2IDBiYfDem4h7u0sxk/1dAvtD3b7qOaB+1sQ/JWmmVQHA+cq fG555zhfvshdMSrQPBET8zO8Nu2H4XQc1ORxMf9k3xLt/KxvDEh8py82V64kjZ25W0NY iGu1dDLlFwRCM0ohr0Rx6fSl65krJ3P6ob/vJd9/5hKyFdAVZzOJweKyugOwmggkikU6 hOG0GNdzQ/jieeWdtGc9nUWgUzxc2rQTq/viu12qo/V5BEQISn8iWzCNI5LoRf0+JPHW VrZg== X-Gm-Message-State: AKaTC00IwRhHIm6woeGaulB8z+l69Xqv8TSJMqkMhxmD/KkJbvsk8hsGeH3l4Gtb7hPSlA== X-Received: by 10.28.113.197 with SMTP id d66mr4663658wmi.77.1481673435179; Tue, 13 Dec 2016 15:57:15 -0800 (PST) Received: from [192.168.2.21] (p5B095BC2.dip0.t-ipconnect.de. [91.9.91.194]) by smtp.googlemail.com with ESMTPSA id v3sm64445310wjp.13.2016.12.13.15.57.14 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Dec 2016 15:57:14 -0800 (PST) From: Andreas Cadhalpun X-Google-Original-From: Andreas Cadhalpun To: ffmpeg-devel@ffmpeg.org References: <18a6d792-ed50-8963-ccfd-8c585824682f@googlemail.com> <20161213003228.GA4749@nb4> Message-ID: Date: Wed, 14 Dec 2016 00:57:12 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Icedove/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161213003228.GA4749@nb4> Subject: Re: [FFmpeg-devel] [PATCH 3/3] tiff: fix overflows when calling av_readuce X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" On 13.12.2016 01:32, Michael Niedermayer wrote: > On Tue, Dec 13, 2016 at 12:50:19AM +0100, Andreas Cadhalpun wrote: >> The arguments of av_reduce are signed, so the cast to uint64_t is misleading. >> >> Signed-off-by: Andreas Cadhalpun >> --- >> libavcodec/tiff.c | 11 +++++++++-- >> 1 file changed, 9 insertions(+), 2 deletions(-) >> >> diff --git a/libavcodec/tiff.c b/libavcodec/tiff.c >> index 4721e94..12ef419 100644 >> --- a/libavcodec/tiff.c >> +++ b/libavcodec/tiff.c >> @@ -772,9 +772,16 @@ static void set_sar(TiffContext *s, unsigned tag, unsigned num, unsigned den) >> int offset = tag == TIFF_YRES ? 2 : 0; >> s->res[offset++] = num; >> s->res[offset] = den; >> - if (s->res[0] && s->res[1] && s->res[2] && s->res[3]) >> + if (s->res[0] && s->res[1] && s->res[2] && s->res[3]) { >> + uint64_t num = s->res[2] * (uint64_t)s->res[1]; >> + uint64_t den = s->res[0] * (uint64_t)s->res[3]; >> + if (num > INT64_MAX || den > INT64_MAX) { >> + num = num >> 1; >> + den = den >> 1; >> + } > > this can make one of them 0, in fact i think even if they arent 0 > the sample_aspect_ratio can be after reduce > should they be checked after all that instead of before ? I've added a check for !s->avctx->sample_aspect_ratio.den after av_reduce. The check before is still necessary to prevent sample_aspect_ratio from becoming negative. Best regards, Andreas From 3cd8cb663d762bc15694e285ea48cdb8e9abfd4b Mon Sep 17 00:00:00 2001 From: Andreas Cadhalpun Date: Tue, 13 Dec 2016 00:43:21 +0100 Subject: [PATCH] tiff: fix overflows when calling av_reduce The arguments of av_reduce are signed, so the cast to uint64_t is misleading. Signed-off-by: Andreas Cadhalpun --- libavcodec/tiff.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/libavcodec/tiff.c b/libavcodec/tiff.c index 4721e94..7ccda51 100644 --- a/libavcodec/tiff.c +++ b/libavcodec/tiff.c @@ -772,9 +772,18 @@ static void set_sar(TiffContext *s, unsigned tag, unsigned num, unsigned den) int offset = tag == TIFF_YRES ? 2 : 0; s->res[offset++] = num; s->res[offset] = den; - if (s->res[0] && s->res[1] && s->res[2] && s->res[3]) + if (s->res[0] && s->res[1] && s->res[2] && s->res[3]) { + uint64_t num = s->res[2] * (uint64_t)s->res[1]; + uint64_t den = s->res[0] * (uint64_t)s->res[3]; + if (num > INT64_MAX || den > INT64_MAX) { + num = num >> 1; + den = den >> 1; + } av_reduce(&s->avctx->sample_aspect_ratio.num, &s->avctx->sample_aspect_ratio.den, - s->res[2] * (uint64_t)s->res[1], s->res[0] * (uint64_t)s->res[3], INT32_MAX); + num, den, INT32_MAX); + if (!s->avctx->sample_aspect_ratio.den) + s->avctx->sample_aspect_ratio = (AVRational) {0, 1}; + } } static int tiff_decode_tag(TiffContext *s, AVFrame *frame) -- 2.10.2