From patchwork Sat Nov 19 16:27:19 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Cadhalpun X-Patchwork-Id: 1487 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.90.1 with SMTP id o1csp709940vsb; Sat, 19 Nov 2016 08:27:33 -0800 (PST) X-Received: by 10.25.37.134 with SMTP id l128mr1117350lfl.120.1479572853653; Sat, 19 Nov 2016 08:27:33 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id n199si889680lfa.82.2016.11.19.08.27.33; Sat, 19 Nov 2016 08:27:33 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@googlemail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE dis=NONE) header.from=googlemail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EB53E689F12; Sat, 19 Nov 2016 18:27:27 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wj0-f194.google.com (mail-wj0-f194.google.com [209.85.210.194]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5F95B689C35 for ; Sat, 19 Nov 2016 18:27:21 +0200 (EET) Received: by mail-wj0-f194.google.com with SMTP id jb2so820329wjb.3 for ; Sat, 19 Nov 2016 08:27:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=from:subject:to:references:message-id:date:user-agent:mime-version :in-reply-to; bh=Jx5zRmLbQRMfrX9QgvMjV08Y0+Z3hw7o53Yhjhina3c=; b=UBuiq2b/SAfcSzClM9y0GBJklts65gfsBD+KSTUYCTSmORfA6CRR2+bRpWZ3qDZ9xH EwTg9VDLIDBqgMmWeFrUZPpitvz0CNvq3gvDyokurFvZum2q3OLWcRpZDNk7LitWFsbC x8sCCZ0yGV0kfETAzH2Ki3JrygwjSroLj13NAHTfuegU5G2M0nn7E9Bm38o4ifBdOQ/V 3UIz2rBRH2N++CRJp2vLXymgCtxXuaFRBlWZneyQ9UkGRvXTAWVq2wU3e2oj9cQszzj7 Rqu8rQn3wSVdaGR9mp/bDtP0DhS3AeXGPDGGRQIEDowCy2DHbLcfY8HkDAHJx5F/nFww zrig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:subject:to:references:message-id:date :user-agent:mime-version:in-reply-to; bh=Jx5zRmLbQRMfrX9QgvMjV08Y0+Z3hw7o53Yhjhina3c=; b=ULi+cmo50ogmpBoUlRg2UiS9XE8J3/fZK/bA6RDCCnX2e5iuCelgstvjKa/j/kCsuO kcAWurPQ2KfZGIw8EQKLSK8eHi/SjPnnXlj/vlq5/VrVEzjBHyUOy8BjyBUHKUdb875J OkjT6o7lu6OSDB0AfS3AIWnNGQslOH23Kz6QWpvSMPxXb/EHrWWqV8BKITikO6Qabd67 KT4l2WZGedsg+JPN3AC+MYMHNrU7eR9Kd8FrsIfvLz0b410HP7Eb4PsEzObPhz9jedxM DFymYsy3ikNnh97qfcANZsGJAVLb0MLgOV9Un1VSFIcZOscZj40ikAZNGj/ExGcLxaOw db/Q== X-Gm-Message-State: AKaTC00Z9NF7l3gV3qBPh1q+GLaI2UEcQSec8j6l5cOqLgyc9xzMO99NubaNouNt79UFrA== X-Received: by 10.194.222.132 with SMTP id qm4mr3928486wjc.150.1479572842879; Sat, 19 Nov 2016 08:27:22 -0800 (PST) Received: from [192.168.2.21] (p5B095CF1.dip0.t-ipconnect.de. [91.9.92.241]) by smtp.googlemail.com with ESMTPSA id g73sm9512855wme.16.2016.11.19.08.27.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 19 Nov 2016 08:27:22 -0800 (PST) From: Andreas Cadhalpun X-Google-Original-From: Andreas Cadhalpun To: ffmpeg-devel@ffmpeg.org, libav development References: <451af605-69ea-c944-9e1b-0fb67ae1abcf@googlemail.com> <20161119151321.GP4824@nb4> Message-ID: Date: Sat, 19 Nov 2016 17:27:19 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Icedove/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161119151321.GP4824@nb4> Subject: Re: [FFmpeg-devel] [PATCH] smacker: limit recursion depth of smacker_decode_bigtree X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" On 19.11.2016 16:13, Michael Niedermayer wrote: > On Sat, Nov 19, 2016 at 02:29:35PM +0100, Andreas Cadhalpun wrote: >> This fixes segmentation faults due to stack-overflow caused by too deep >> recursion. >> >> Signed-off-by: Andreas Cadhalpun >> --- >> libavcodec/smacker.c | 4 ++-- >> 1 file changed, 2 insertions(+), 2 deletions(-) > > breaks fate > > [smackvid @ 0x3586b80] size 31232 too large > Input #0, smk, from 'fate/fate-suite//smacker/wetlogo.smk': > Duration: 00:00:07.10, start: 0.000000, bitrate: 815 kb/s > Stream #0:0: Video: smackvideo (SMK2 / 0x324B4D53), pal8, 320x200, 14.08 tbr, 14.08 tbn, 14.08 tbc > Stream #0:1: Audio: smackaudio (SMKA / 0x414B4D53), 22050 Hz, mono, u8 > [smackvid @ 0x358b860] size 31232 too large > Stream mapping: > Stream #0:0 -> #0:0 (smackvideo (smackvid) -> rawvideo (native)) > Error while opening decoder for input stream #0:0 : Invalid data found when processing input > make: *** [fate-smacker-video] Error 1 Then this has to be checked more directly, see attached patch. Best regards, Andreas From bcb4a860ff5ca987b5bcd99797704faaba6d23a2 Mon Sep 17 00:00:00 2001 From: Andreas Cadhalpun Date: Sat, 19 Nov 2016 14:21:11 +0100 Subject: [PATCH] smacker: limit recursion depth of smacker_decode_bigtree This fixes segmentation faults due to stack-overflow caused by too deep recursion. Signed-off-by: Andreas Cadhalpun --- libavcodec/smacker.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/libavcodec/smacker.c b/libavcodec/smacker.c index b8a0c55..be3914b 100644 --- a/libavcodec/smacker.c +++ b/libavcodec/smacker.c @@ -129,8 +129,12 @@ static int smacker_decode_tree(GetBitContext *gb, HuffContext *hc, uint32_t pref /** * Decode header tree */ -static int smacker_decode_bigtree(GetBitContext *gb, HuffContext *hc, DBCtx *ctx) +static int smacker_decode_bigtree(GetBitContext *gb, HuffContext *hc, DBCtx *ctx, int length) { + if(length > 5000) { // Larger length can cause segmentation faults due to too deep recursion. + av_log(NULL, AV_LOG_ERROR, "length too long\n"); + return AVERROR_INVALIDDATA; + } if (hc->current + 1 >= hc->length) { av_log(NULL, AV_LOG_ERROR, "Tree size exceeded!\n"); return AVERROR_INVALIDDATA; @@ -159,12 +163,12 @@ static int smacker_decode_bigtree(GetBitContext *gb, HuffContext *hc, DBCtx *ctx int r = 0, r_new, t; t = hc->current++; - r = smacker_decode_bigtree(gb, hc, ctx); + r = smacker_decode_bigtree(gb, hc, ctx, length + 1); if(r < 0) return r; hc->values[t] = SMK_NODE | r; r++; - r_new = smacker_decode_bigtree(gb, hc, ctx); + r_new = smacker_decode_bigtree(gb, hc, ctx, length + 1); if (r_new < 0) return r_new; return r + r_new; @@ -275,7 +279,7 @@ static int smacker_decode_header_tree(SmackVContext *smk, GetBitContext *gb, int goto error; } - if (smacker_decode_bigtree(gb, &huff, &ctx) < 0) + if (smacker_decode_bigtree(gb, &huff, &ctx, 0) < 0) err = -1; skip_bits1(gb); if(ctx.last[0] == -1) ctx.last[0] = huff.current++; -- 2.10.2