From patchwork Mon Nov 7 21:32:29 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Cadhalpun X-Patchwork-Id: 1325 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.90.1 with SMTP id o1csp1353474vsb; Mon, 7 Nov 2016 13:32:46 -0800 (PST) X-Received: by 10.194.80.99 with SMTP id q3mr9439977wjx.13.1478554366195; Mon, 07 Nov 2016 13:32:46 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id bp10si31532551wjb.135.2016.11.07.13.32.45; Mon, 07 Nov 2016 13:32:46 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@googlemail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE dis=NONE) header.from=googlemail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9200F689734; Mon, 7 Nov 2016 23:32:39 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f68.google.com (mail-wm0-f68.google.com [74.125.82.68]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7E0F0687EF1 for ; Mon, 7 Nov 2016 23:32:33 +0200 (EET) Received: by mail-wm0-f68.google.com with SMTP id p190so18781920wmp.1 for ; Mon, 07 Nov 2016 13:32:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=from:to:subject:message-id:date:user-agent:mime-version :content-transfer-encoding; bh=36+AKMZwMBwx1eTeJzzsKuFxTlrzM3b0c5eCSCP7+GM=; b=rSVbmS/oGpZL1xyXuIx6y/XuP99pbrvMxJuukEjvv1OZxWRsVIK4428KwUQrvFvFTo wLfcbcmnXrHhyBXmyouUgFedwIbDpaqmLBIWcr/y0c/8X7lxzHfXogbcQ0y/azJopGgC XcVC7settFrID2UHjLFvAiWVThV1oMxb+eHa5RnzmUkJBFsSjEKIm1IhHT9KPmIaOpG5 IEd/tipZ66agtlrB17woNKVrZggZYdv2jCMpKhdhp6foD2pbRJ8XrOh0mL7W9vtXiSO4 IczBRTte+ydLdO1jW2bbmLbnIgt144vKTxsrj0r0+qIpJm74Okuw6cNaiHwIO24ucAX9 IHlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:message-id:date:user-agent :mime-version:content-transfer-encoding; bh=36+AKMZwMBwx1eTeJzzsKuFxTlrzM3b0c5eCSCP7+GM=; b=Qgj33fEPUwl1Oc4nT4IODEalXd7L24qp4OIpPx0RQLPmSGaSmLmWMVur8sDBCJgQEb JkxFtAZ2a2n0kQPjJyfvTajvApFK3vT25AuSK+FHiBQMH5d1Z/7osTLPUIIKpVTLjc/i tu228XiGtUZQZCI7FLQYgDftg/eP4P0mq81cvSIpPbJ/P12InqYc8i/m4HVDzwr78ACD Rt53OMEz0vQZ4oRQiNiShEbHEZ9IjPkQgISryT1YQxBCISKhWjryXUTi8+DZC51bwzry XDcJhECTQAqg9pH61sxSbzVfhA5iGon18z8CVLQaMHxXuO8TMTlRoL5TJqRL48gs22Gs tIAA== X-Gm-Message-State: ABUngveNf8MjjlRH+Aa76za+sfzvM3n67M1oO4JbwAH0oTg+GRfWX1ym2rZI+4dIBjnsEg== X-Received: by 10.28.169.74 with SMTP id s71mr7058627wme.1.1478554356390; Mon, 07 Nov 2016 13:32:36 -0800 (PST) Received: from [192.168.2.21] (p5B072203.dip0.t-ipconnect.de. [91.7.34.3]) by smtp.googlemail.com with ESMTPSA id a1sm33323460wjl.28.2016.11.07.13.32.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Nov 2016 13:32:35 -0800 (PST) From: Andreas Cadhalpun X-Google-Original-From: Andreas Cadhalpun To: FFmpeg development discussions and patches , libav development Message-ID: <75b43580-088b-3d28-8552-f5b64386f83d@googlemail.com> Date: Mon, 7 Nov 2016 22:32:29 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Icedove/45.4.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] mpegpicture: use coded_width/coded_height to allocate frame X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This fixes a heap-buffer-overflow in ff_er_frame_end when decoding mss2 with coded_width/coded_height larger than width/height. Signed-off-by: Andreas Cadhalpun --- libavcodec/mpegpicture.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/libavcodec/mpegpicture.c b/libavcodec/mpegpicture.c index 6748fc2..70b4d3c 100644 --- a/libavcodec/mpegpicture.c +++ b/libavcodec/mpegpicture.c @@ -108,15 +108,15 @@ static int alloc_frame_buffer(AVCodecContext *avctx, Picture *pic, avctx->codec_id != AV_CODEC_ID_VC1IMAGE && avctx->codec_id != AV_CODEC_ID_MSS2) { if (edges_needed) { - pic->f->width = avctx->width + 2 * EDGE_WIDTH; - pic->f->height = avctx->height + 2 * EDGE_WIDTH; + pic->f->width = avctx->coded_width + 2 * EDGE_WIDTH; + pic->f->height = avctx->coded_height + 2 * EDGE_WIDTH; } r = ff_thread_get_buffer(avctx, &pic->tf, pic->reference ? AV_GET_BUFFER_FLAG_REF : 0); } else { - pic->f->width = avctx->width; - pic->f->height = avctx->height; + pic->f->width = avctx->coded_width; + pic->f->height = avctx->coded_height; pic->f->format = avctx->pix_fmt; r = avcodec_default_get_buffer2(avctx, pic->f, 0); } @@ -135,8 +135,8 @@ static int alloc_frame_buffer(AVCodecContext *avctx, Picture *pic, (EDGE_WIDTH >> (i ? chroma_x_shift : 0)); pic->f->data[i] += offset; } - pic->f->width = avctx->width; - pic->f->height = avctx->height; + pic->f->width = avctx->coded_width; + pic->f->height = avctx->coded_height; } if (avctx->hwaccel) {