From patchwork Mon Feb 26 02:29:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 7736 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.181.170 with SMTP id m39csp888561jaj; Sun, 25 Feb 2018 18:30:02 -0800 (PST) X-Google-Smtp-Source: AH8x2253KItIiRueATz3z7rmEjxDagLWib7gvclpIwhbanHPWWYOQx0D6i7HJoMudXGPjLtixEnX X-Received: by 10.223.175.50 with SMTP id z47mr7501222wrc.198.1519612201938; Sun, 25 Feb 2018 18:30:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519612201; cv=none; d=google.com; s=arc-20160816; b=a4atryzf9oO+A/95pJrLPyW8InveuS6St6skNreN0t5/D0qHUA2koy5SVvKGhj+YE1 vyyBwrh4+4O/sfGc0Yz78NvbqHoOnZzesC2sJ50PjsKf8dhyoP4cH7c0ibPwzbfHZsob PcQbHhfXT0HPCPFG4ZfKt6bkKTUcHrcaJ5JbIJzd36HPi7rSDDesUVH1Zcixamktlz63 I4HRxDHTFFdY3LAVZ+bQtYLGLdRw20ChPr/dmDiRObCA2Gauw/DQ9qeYdJRDNWOpyTJo UmbeWGUlTn3oED/+NrXHWJRYfEbnZhy5gG+O806z+s5cgKmdFglU6UVSTud1Bu+5VsrM +OsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to:arc-authentication-results; bh=w5QLG6XurBY8F23iEdm/GdhTXWkiJQ4Vl7St0KBhbco=; b=nioQId19k9sRRneZEgjSMOmeRATT/22COdKEoD/ATBIA/sEIWCfFRwSZJY6DpIMAVN BVmEpMjZSIzKxTZhlr0LyrnyGRBa5eYwMBIn5Z/pWIQZp3J/3w5fCbBpnZgICZIneYXN Dhl6GCPGvdcQCbos3sHFtsLCW3KzVTFgOEl3NxnB5kLd39wSXEdIWREHZjMKARhzKyKW E04V2WvcrysHrPyKxjC4ONDZZd+186BzncI+yZUZH/xu1K6jeWZ+/XgO0MUEp2J3rl+G cLmVEorYTd+1hQbSyGQLLvgEbmp+8r+AlHj1L3LRUlvHWdRkteKaaOjoWQ/niGaTaEbG bw5A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b76si3105112wrd.152.2018.02.25.18.30.00; Sun, 25 Feb 2018 18:30:01 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 90F1B68A28D; Mon, 26 Feb 2018 04:29:54 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vie01a-dmta-pe01-1.mx.upcmail.net (vie01a-dmta-pe01-1.mx.upcmail.net [62.179.121.154]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CB43468A13F for ; Mon, 26 Feb 2018 04:29:47 +0200 (EET) Received: from [172.31.216.43] (helo=vie01a-pemc-psmtp-pe01) by vie01a-dmta-pe01.mx.upcmail.net with esmtp (Exim 4.88) (envelope-from ) id 1eq8Y3-0005JL-5X for ffmpeg-devel@ffmpeg.org; Mon, 26 Feb 2018 03:29:51 +0100 Received: from localhost ([213.47.41.20]) by vie01a-pemc-psmtp-pe01 with SMTP @ mailcloud.upcmail.net id FEVp1x00E0S5wYM01EVq51; Mon, 26 Feb 2018 03:29:50 +0100 X-SourceIP: 213.47.41.20 From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Mon, 26 Feb 2018 03:29:47 +0100 Message-Id: <20180226022947.3129-2-michael@niedermayer.cc> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180226022947.3129-1-michael@niedermayer.cc> References: <20180226022947.3129-1-michael@niedermayer.cc> Subject: [FFmpeg-devel] [PATCH 2/2] avcodec/utvideodec: Check subsample factors X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Fixes: Out of array read Fixes: heap_poc Found-by: GwanYeong Kim Signed-off-by: Michael Niedermayer --- libavcodec/utvideodec.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/libavcodec/utvideodec.c b/libavcodec/utvideodec.c index c5f5534964..086129d094 100644 --- a/libavcodec/utvideodec.c +++ b/libavcodec/utvideodec.c @@ -30,6 +30,7 @@ #define UNCHECKED_BITSTREAM_READER 1 #include "libavutil/intreadwrite.h" +#include "libavutil/pixdesc.h" #include "avcodec.h" #include "bswapdsp.h" #include "bytestream.h" @@ -912,6 +913,7 @@ static int decode_frame(AVCodecContext *avctx, void *data, int *got_frame, static av_cold int decode_init(AVCodecContext *avctx) { UtvideoContext * const c = avctx->priv_data; + int h_shift, v_shift; c->avctx = avctx; @@ -1012,6 +1014,13 @@ static av_cold int decode_init(AVCodecContext *avctx) return AVERROR_INVALIDDATA; } + av_pix_fmt_get_chroma_sub_sample(avctx->pix_fmt, &h_shift, &v_shift); + if ((avctx->width & ((1<height & ((1<pack && avctx->extradata_size >= 16) { av_log(avctx, AV_LOG_DEBUG, "Encoder version %d.%d.%d.%d\n", avctx->extradata[3], avctx->extradata[2],