From patchwork Tue Nov 15 21:18:08 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Cadhalpun X-Patchwork-Id: 1439 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.90.1 with SMTP id o1csp1821651vsb; Tue, 15 Nov 2016 13:18:21 -0800 (PST) X-Received: by 10.28.29.23 with SMTP id d23mr5767332wmd.91.1479244700966; Tue, 15 Nov 2016 13:18:20 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id k186si4493727wma.76.2016.11.15.13.18.20; Tue, 15 Nov 2016 13:18:20 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@googlemail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE dis=QUARANTINE) header.from=googlemail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A383C689C72; Tue, 15 Nov 2016 23:18:17 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f66.google.com (mail-wm0-f66.google.com [74.125.82.66]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9A449689BD1 for ; Tue, 15 Nov 2016 23:18:10 +0200 (EET) Received: by mail-wm0-f66.google.com with SMTP id u144so4277170wmu.0 for ; Tue, 15 Nov 2016 13:18:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=from:to:subject:message-id:date:user-agent:mime-version :content-transfer-encoding; bh=WW8HMtNKc3gaJtZFbZ7fJ/P1I9mj5DbYA9OeBylfiEQ=; b=Ct1uKEggRnE0B9fTFQXGXLzAgkS4Km9NCqSWipb6uYlvwq/GE8MofqC7Pni+gUSZxj rY85T9YQV5JpewlH0HnIAeg/ORNHa618YdZe5+DJM73BiutUoEGq3eF5cw91PkLWTRvc xvLcbyS4KKZnG2XCINeafQ4eR6a93R4MEs7ekPaEnIupfGnxhIBvXXtMroykYw9a9aEA j/BYRoiV0RR/yv/a8HzMZDC3D9XpeUwRykHWxEzYuXdjiQ0biq9DR1GsukIQAKRvUVeC 8aa595FVtH/Zw8qALSVtAcvG3AAg+SM+KutXwqUz4TaHZmKsbhEidWoObNIIpblGPBz0 YPMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:message-id:date:user-agent :mime-version:content-transfer-encoding; bh=WW8HMtNKc3gaJtZFbZ7fJ/P1I9mj5DbYA9OeBylfiEQ=; b=Zw312R/ufrthRpxbdTumXPt3TUuuCHwCwIYllUxf7W9+aVNNIq/U3Pub3s1kh/Z8R7 6ZpSAMSWH1yTtNMKffE+JFTdXOMopfHgHKcB05Lb8Ptf5lDahmM+jnZEQzr1fh42M2QI SZ5Dw1xL+6gEPJoljo4oXwqglz6KpZDZOpOE8Bie4uoZ6E/b1gyJTY6uq+7u+kjxGnPi ODwMmxyWpldDFZHw04j0vyrMfvGjGu9yPllcUH5nqdGwT7AF7KIS6c2J1xfpYaInKXja i7rfh7DErf+vLlnMFRK2KAfFb1ugVQvrrgNoH5qlo3WkWp+61BUpb+c9UQSRYanOCX43 +rWA== X-Gm-Message-State: ABUngvdbyRKW+73SNWpfffUWiNPpKEIkTIKO+sD4Na9DRGVvX0jcIYkFRwnuWZFBSuE8AA== X-Received: by 10.194.21.201 with SMTP id x9mr13418231wje.153.1479244690640; Tue, 15 Nov 2016 13:18:10 -0800 (PST) Received: from [192.168.2.21] (p5B095CE1.dip0.t-ipconnect.de. [91.9.92.225]) by smtp.googlemail.com with ESMTPSA id kq7sm22645683wjb.30.2016.11.15.13.18.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Nov 2016 13:18:10 -0800 (PST) From: Andreas Cadhalpun X-Google-Original-From: Andreas Cadhalpun To: FFmpeg development discussions and patches , Paul B Mahol Message-ID: <6f222917-50fa-0b17-7255-49af2e0fa7a2@googlemail.com> Date: Tue, 15 Nov 2016 22:18:08 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Icedove/45.4.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] dds: limit 4 bpp handling to AV_PIX_FMT_PAL8 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This fixes NULL pointer dereferencing for formats, where frame->data[1] is not allocated. The problem was introduced in commit 257fbc3af4cba08ac471dab68924182160bde6fd. Signed-off-by: Andreas Cadhalpun --- libavcodec/dds.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/dds.c b/libavcodec/dds.c index 43319a8..fa0f34d 100644 --- a/libavcodec/dds.c +++ b/libavcodec/dds.c @@ -680,7 +680,7 @@ static int dds_decode(AVCodecContext *avctx, void *data, /* Use the decompress function on the texture, one block per thread. */ ctx->tex_data = gbc->buffer; avctx->execute2(avctx, decompress_texture_thread, frame, NULL, ctx->slice_count); - } else if (!ctx->paletted && ctx->bpp == 4) { + } else if (!ctx->paletted && ctx->bpp == 4 && avctx->pix_fmt == AV_PIX_FMT_PAL8) { uint8_t *dst = frame->data[0]; int x, y, i;