From patchwork Tue Apr 10 00:28:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 8366 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.1.70 with SMTP id c67csp454877jad; Mon, 9 Apr 2018 17:29:16 -0700 (PDT) X-Google-Smtp-Source: AIpwx49seSP6iRmBV2DYp4/s8ga+KRGtNF/u3lRU4QVi/sR/zPmpivP7GeMaN8A4d8QY5Ju5V9UT X-Received: by 10.28.160.9 with SMTP id j9mr19508wme.80.1523320156661; Mon, 09 Apr 2018 17:29:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1523320156; cv=none; d=google.com; s=arc-20160816; b=iSanRa/UGCqmo8VVPHpX8sauT6TNWqLpyvQd/qaSv7yiYFf+Y+fbvXVAow1Tum/AJq hgriRWN29TrNUndXqKiJAEoXRgHriHjNrVbMECwOGpBHad1hX98EKbZC7w7vNBOkIA+F +FkftaRB3Ym3aXy6jlInqeHF9G5V1MBBqVjBQ4U9xAx2iSCt2wK/X42P+DZ9ljq8pgel SgdMxeponJGkXAC0F02Ca7zON+QeDcegpJiB614tpnESOr1Xik5pZTRpNmO4c4hKtNEO pq/yjb6TA8A7dkLXrhzgLEW6a8XMKOhqULCX3UtjR/aD9+S4r5pn7bV9c/urHipV8wr4 HGcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to:arc-authentication-results; bh=TOx+KKNpnIHuOzNgve45PCNYXKz4blMyEoofDL8yWlY=; b=tnH7juvLnhTuCSRlsEhvFRc5l4zy2xRJ4AueGaiP4aILvJ4RAEutasnu3w0Kw+xyKO HOnmY1h37HZLO5c9KG6mU7qXZe9iFNwSKuT3UFvtXMA+D/ZlTzxX9x+vAl/hUMlY8var M4QrqIYzFBu3K/nAlFD6XGr+sw4HylNTBDE18wsLkCoWLWyFM+OFErxcYDdvUoXUudxL jaIx4jHhDVXL+c9f/qgJ1tGwwa8xbti9sOYeWIVaZsdPQA7QIYb9HsAVVi+PTumDMzzg srQA08q+mZ5miHY7NcrzG3ttPVfrNY/x6aasx2L88GJVTpwmp20gxBG+hSQu//gP19+K 0IyA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id r78si352867wme.29.2018.04.09.17.29.15; Mon, 09 Apr 2018 17:29:16 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B5300689F62; Tue, 10 Apr 2018 03:28:50 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vie01a-qmta-pe02-3.mx.upcmail.net (vie01a-qmta-pe02-3.mx.upcmail.net [62.179.121.183]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DFC75689ED3 for ; Tue, 10 Apr 2018 03:28:49 +0300 (EEST) Received: from [172.31.218.40] (helo=vie01a-dmta-pe04-1.mx.upcmail.net) by vie01a-pqmta-pe02.mx.upcmail.net with esmtp (Exim 4.88) (envelope-from ) id 1f5h9s-0005zo-SR for ffmpeg-devel@ffmpeg.org; Tue, 10 Apr 2018 02:29:12 +0200 Received: from [172.31.216.43] (helo=vie01a-pemc-psmtp-pe01) by vie01a-dmta-pe04.mx.upcmail.net with esmtp (Exim 4.88) (envelope-from ) id 1f5h9s-00020K-10 for ffmpeg-devel@ffmpeg.org; Tue, 10 Apr 2018 02:29:12 +0200 Received: from localhost ([213.47.41.20]) by vie01a-pemc-psmtp-pe01 with SMTP @ mailcloud.upcmail.net id YQV61x02A0S5wYM01QV7Qs; Tue, 10 Apr 2018 02:29:07 +0200 X-SourceIP: 213.47.41.20 From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Tue, 10 Apr 2018 02:28:20 +0200 Message-Id: <20180410002820.15374-2-michael@niedermayer.cc> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180410002820.15374-1-michael@niedermayer.cc> References: <20180410002820.15374-1-michael@niedermayer.cc> Subject: [FFmpeg-devel] [PATCH 2/2] avcodec/h264_slice: Fix integer overflow with last_poc X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Fixes: signed integer overflow: 2147483646 - -2816 cannot be represented in type 'int' Fixes: crbug 823145 Reported-by: Matt Wolenetz Signed-off-by: Michael Niedermayer --- libavcodec/h264_slice.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c index 90e05ed8f1..d71ddbe9ba 100644 --- a/libavcodec/h264_slice.c +++ b/libavcodec/h264_slice.c @@ -1316,7 +1316,7 @@ static int h264_select_output_frame(H264Context *h) } out_of_order = MAX_DELAYED_PIC_COUNT - i; if( cur->f->pict_type == AV_PICTURE_TYPE_B - || (h->last_pocs[MAX_DELAYED_PIC_COUNT-2] > INT_MIN && h->last_pocs[MAX_DELAYED_PIC_COUNT-1] - h->last_pocs[MAX_DELAYED_PIC_COUNT-2] > 2)) + || (h->last_pocs[MAX_DELAYED_PIC_COUNT-2] > INT_MIN && h->last_pocs[MAX_DELAYED_PIC_COUNT-1] - (int64_t)h->last_pocs[MAX_DELAYED_PIC_COUNT-2] > 2)) out_of_order = FFMAX(out_of_order, 1); if (out_of_order == MAX_DELAYED_PIC_COUNT) { av_log(h->avctx, AV_LOG_VERBOSE, "Invalid POC %d<%d\n", cur->poc, h->last_pocs[0]);