From patchwork Tue Jan 31 00:59:38 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Cadhalpun X-Patchwork-Id: 2376 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.89.21 with SMTP id n21csp1713192vsb; Mon, 30 Jan 2017 16:59:48 -0800 (PST) X-Received: by 10.28.220.135 with SMTP id t129mr17535786wmg.97.1485824388475; Mon, 30 Jan 2017 16:59:48 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s131si15380667wmf.117.2017.01.30.16.59.48; Mon, 30 Jan 2017 16:59:48 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@googlemail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1580568A4B8; Tue, 31 Jan 2017 02:59:44 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f65.google.com (mail-wm0-f65.google.com [74.125.82.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 421AB689F13 for ; Tue, 31 Jan 2017 02:59:37 +0200 (EET) Received: by mail-wm0-f65.google.com with SMTP id r18so9351027wmd.3 for ; Mon, 30 Jan 2017 16:59:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=from:to:subject:message-id:date:user-agent:mime-version :content-transfer-encoding; bh=PgXEUp2DT2DPVZ72uRtEKT725zbi0cz9TIkGFe0+ZuU=; b=effDmnB4FYe8mV5Vys2XAB4U7pK2C/nEXZ0v3uZkDyHdOm2FLjDH1o+eumhYvT89Oc BmPjUlmK/31pmmhTyHxGglp3puddxWg2XLTlXYny9JWffbzL6vrnA/HycimRDh87Xq1t m87qJCnCrq7BsOVdUNMkZD4tXDqwfF1rua5AA6FSNV8PPpYCF2YL+bTiC4vAWL9TVOnj mNU9v/BAv9J6dICLiMkdLzmlAkef8iuxiOXB7Vzeg/NurZ/wPPRPWBzZLyL5h7fzszff pHuoTPfdsD61dx9MYhAwkQegibZueXOjwGmTPQH7azjqmde+XxT8kLJaU4xqAika58ad brNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:message-id:date:user-agent :mime-version:content-transfer-encoding; bh=PgXEUp2DT2DPVZ72uRtEKT725zbi0cz9TIkGFe0+ZuU=; b=nfoBI7JXUz/959TXQWXcfcxiUW2l9c7mNW5WE9RFwr3Lp5mTV0ro4Z4kFsjNOHsbGA S9veGx8Kt7wI19TA+lEC4wOzJIIUSW8fHAavbQ7ANa/2b9H/A6l8Wllps+BfEavkeyKN GSk6WYWhIrT1rgdqQ6izDajp9Zdd4fPLFFJexr35aL9pKN25JriUP0m+hmna3XbMok4E KiWNycJb5L9wZsJnEo2xwEfwIj2jO8UoxZ0iNgFbkCtBVxTkr0xcKDVs3cg8/sL2Vd21 95q0JME9k7A73BeJNW6SVhqEptcHlkHewMJkVhCiqvMwwHCfCFerqser8YJ/cvHyMFwe YMdA== X-Gm-Message-State: AIkVDXLoO4BGG/rHZSIOR6ok5jA3XobETMz/bxFUDF9WR+CVNc1e0VSQO8Z09XmaLPmIIg== X-Received: by 10.223.155.221 with SMTP id e29mr21239743wrc.107.1485824379396; Mon, 30 Jan 2017 16:59:39 -0800 (PST) Received: from [192.168.2.21] (p5B095744.dip0.t-ipconnect.de. [91.9.87.68]) by smtp.googlemail.com with ESMTPSA id 36sm25305911wrz.8.2017.01.30.16.59.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 30 Jan 2017 16:59:38 -0800 (PST) From: Andreas Cadhalpun X-Google-Original-From: Andreas Cadhalpun To: FFmpeg development discussions and patches , libav development Message-ID: <3335367d-e47e-6ac3-2f9e-6a46266b2ce6@googlemail.com> Date: Tue, 31 Jan 2017 01:59:38 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] pgssubdec: reset rle_data_len/rle_remaining_len on allocation error X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The code relies on their validity and otherwise can try to access a NULL object->rle pointer, causing segmentation faults. Signed-off-by: Andreas Cadhalpun --- libavcodec/pgssubdec.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/libavcodec/pgssubdec.c b/libavcodec/pgssubdec.c index b50b37b206..b897d72aab 100644 --- a/libavcodec/pgssubdec.c +++ b/libavcodec/pgssubdec.c @@ -300,8 +300,11 @@ static int parse_object_segment(AVCodecContext *avctx, av_fast_padded_malloc(&object->rle, &object->rle_buffer_size, rle_bitmap_len); - if (!object->rle) + if (!object->rle) { + object->rle_data_len = 0; + object->rle_remaining_len = 0; return AVERROR(ENOMEM); + } memcpy(object->rle, buf, buf_size); object->rle_data_len = buf_size;