diff mbox

[FFmpeg-devel,avformat] Prevent undefined shift with wrap_bits > 63.

Message ID CAPUDrwewsAZk_uodJktoGnWcD1e29PG6NO6pt_9_a=Tx8hfc_w@mail.gmail.com
State Superseded
Headers show

Commit Message

Dale Curtis Nov. 21, 2017, 1:42 a.m. UTC 
On Mon, Nov 20, 2017 at 2:24 PM, Michael Niedermayer <michael@niedermayer.cc
> wrote:

>
> I think that could end with the correct result
>
>
Thanks for the review. Done.

- dale
diff mbox

Patch

From fc7fb3511aa40810e64d9dacbd33d1e9336d0c52 Mon Sep 17 00:00:00 2001
From: Dale Curtis <dalecurtis@chromium.org>
Date: Fri, 17 Nov 2017 13:35:56 -0800
Subject: [PATCH] [avformat] Prevent undefined shift with wrap_bits > 63.

2LL << (wrap_bits=63 - 1) does not fit in int64_t; change the
code to use a uint64_t (2ULL) and apply the check used in other
places to ensure wrap_bits <= 63.

Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
---
 libavformat/utils.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libavformat/utils.c b/libavformat/utils.c
index ff5e14df6c..08d2f9b2e1 100644
--- a/libavformat/utils.c
+++ b/libavformat/utils.c
@@ -1738,9 +1738,9 @@  int av_read_frame(AVFormatContext *s, AVPacket *pkt)
                 // current one had no dts, we will set this to AV_NOPTS_VALUE.
                 int64_t last_dts = next_pkt->dts;
                 while (pktl && next_pkt->pts == AV_NOPTS_VALUE) {
-                    if (pktl->pkt.stream_index == next_pkt->stream_index &&
-                        (av_compare_mod(next_pkt->dts, pktl->pkt.dts, 2LL << (wrap_bits - 1)) < 0)) {
-                        if (av_compare_mod(pktl->pkt.pts, pktl->pkt.dts, 2LL << (wrap_bits - 1))) {
+                    if (pktl->pkt.stream_index == next_pkt->stream_index && wrap_bits < 64 &&
+                        av_compare_mod(next_pkt->dts, pktl->pkt.dts, 2ULL << (wrap_bits - 1)) < 0) {
+                        if (av_compare_mod(pktl->pkt.pts, pktl->pkt.dts, 2ULL << (wrap_bits - 1))) {
                             // not B-frame
                             next_pkt->pts = pktl->pkt.dts;
                         }
-- 
2.15.0.448.gf294e3d99a-goog