diff mbox

[FFmpeg-devel,avformat] Prevent undefined shift with wrap_bits > 63.

Message ID CAPUDrwd8aJunG3s47cgrAKTWNwSPygE830rmx1oR4Vq2BvJEOg@mail.gmail.com
State New
Headers show

Commit Message

Dale Curtis Nov. 21, 2017, 11:19 p.m. UTC
Ah, realized this approach can work for wrap_bits == 64 too. Updated the
patch.

On Mon, Nov 20, 2017 at 5:42 PM, Dale Curtis <dalecurtis@chromium.org>
wrote:

> On Mon, Nov 20, 2017 at 2:24 PM, Michael Niedermayer <
> michael@niedermayer.cc> wrote:
>
>>
>> I think that could end with the correct result
>>
>>
> Thanks for the review. Done.
>
> - dale
>

Comments

Michael Niedermayer Nov. 22, 2017, 3:05 a.m. UTC | #1
On Tue, Nov 21, 2017 at 03:19:38PM -0800, Dale Curtis wrote:
> Ah, realized this approach can work for wrap_bits == 64 too. Updated the
> patch.
> 
> On Mon, Nov 20, 2017 at 5:42 PM, Dale Curtis <dalecurtis@chromium.org>
> wrote:
> 
> > On Mon, Nov 20, 2017 at 2:24 PM, Michael Niedermayer <
> > michael@niedermayer.cc> wrote:
> >
> >>
> >> I think that could end with the correct result
> >>
> >>
> > Thanks for the review. Done.
> >
> > - dale
> >

>  utils.c |    6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 37722f8edea291bc79742519d06fbea906031074  wrap_bits_v4.patch
> From 6f087bbdb6499dc21a53fcb838348ea271d4ca5a Mon Sep 17 00:00:00 2001
> From: Dale Curtis <dalecurtis@chromium.org>
> Date: Fri, 17 Nov 2017 13:35:56 -0800
> Subject: [PATCH] [avformat] Prevent undefined shift with wrap_bits > 64.
> 
> 2LL << (wrap_bits=64 - 1) does not fit in int64_t; change the
> code to use a uint64_t (2ULL) and apply the check used in other
> places to ensure wrap_bits <= 64.
> 
> Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
> ---
>  libavformat/utils.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/libavformat/utils.c b/libavformat/utils.c
> index ff5e14df6c..2cf8d61e82 100644
> --- a/libavformat/utils.c
> +++ b/libavformat/utils.c
> @@ -1738,9 +1738,9 @@ int av_read_frame(AVFormatContext *s, AVPacket *pkt)
>                  // current one had no dts, we will set this to AV_NOPTS_VALUE.
>                  int64_t last_dts = next_pkt->dts;
>                  while (pktl && next_pkt->pts == AV_NOPTS_VALUE) {
> -                    if (pktl->pkt.stream_index == next_pkt->stream_index &&
> -                        (av_compare_mod(next_pkt->dts, pktl->pkt.dts, 2LL << (wrap_bits - 1)) < 0)) {
> -                        if (av_compare_mod(pktl->pkt.pts, pktl->pkt.dts, 2LL << (wrap_bits - 1))) {

> +                    if (pktl->pkt.stream_index == next_pkt->stream_index && wrap_bits <= 64 &&

I dont think wrap_bits can/should be > 64 or do i miss something ?

maybe a av_assert* for that would be better.

Static analyzers like coverity love to assume that a check implies
the possibility of a field having some value. That could lead to
strange things and false positves if its not actually possible


[...]
Dale Curtis Nov. 22, 2017, 6:56 p.m. UTC | #2
On Tue, Nov 21, 2017 at 7:05 PM, Michael Niedermayer <michael@niedermayer.cc
> wrote:
>
> I dont think wrap_bits can/should be > 64 or do i miss something ?
>

Good point, this seems true with the current code.


>
> maybe a av_assert* for that would be better.
>

Done; used av_assert2().
Michael Niedermayer Nov. 24, 2017, 4:18 p.m. UTC | #3
On Wed, Nov 22, 2017 at 10:56:07AM -0800, Dale Curtis wrote:
> On Tue, Nov 21, 2017 at 7:05 PM, Michael Niedermayer <michael@niedermayer.cc
> > wrote:
> >
> > I dont think wrap_bits can/should be > 64 or do i miss something ?
> >
> 
> Good point, this seems true with the current code.
> 
> 
> >
> > maybe a av_assert* for that would be better.
> >
> 
> Done; used av_assert2().

>  utils.c |    5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> e49bb30e3ddb00bc9f6780c924bfe97e710d7e99  wrap_bits_v5.patch
> From 0297fdc52f043592235d93076f91d38c3df40a14 Mon Sep 17 00:00:00 2001
> From: Dale Curtis <dalecurtis@chromium.org>
> Date: Fri, 17 Nov 2017 13:35:56 -0800
> Subject: [PATCH] [avformat] Prevent undefined shift with wrap_bits > 64.
> 
> 2LL << (wrap_bits=64 - 1) does not fit in int64_t; change the
> code to use a uint64_t (2ULL) and add an av_assert2() to
> ensure wrap_bits <= 64.
> 
> Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
> ---
>  libavformat/utils.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)

will apply

thx

[...]
diff mbox

Patch

From 6f087bbdb6499dc21a53fcb838348ea271d4ca5a Mon Sep 17 00:00:00 2001
From: Dale Curtis <dalecurtis@chromium.org>
Date: Fri, 17 Nov 2017 13:35:56 -0800
Subject: [PATCH] [avformat] Prevent undefined shift with wrap_bits > 64.

2LL << (wrap_bits=64 - 1) does not fit in int64_t; change the
code to use a uint64_t (2ULL) and apply the check used in other
places to ensure wrap_bits <= 64.

Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
---
 libavformat/utils.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libavformat/utils.c b/libavformat/utils.c
index ff5e14df6c..2cf8d61e82 100644
--- a/libavformat/utils.c
+++ b/libavformat/utils.c
@@ -1738,9 +1738,9 @@  int av_read_frame(AVFormatContext *s, AVPacket *pkt)
                 // current one had no dts, we will set this to AV_NOPTS_VALUE.
                 int64_t last_dts = next_pkt->dts;
                 while (pktl && next_pkt->pts == AV_NOPTS_VALUE) {
-                    if (pktl->pkt.stream_index == next_pkt->stream_index &&
-                        (av_compare_mod(next_pkt->dts, pktl->pkt.dts, 2LL << (wrap_bits - 1)) < 0)) {
-                        if (av_compare_mod(pktl->pkt.pts, pktl->pkt.dts, 2LL << (wrap_bits - 1))) {
+                    if (pktl->pkt.stream_index == next_pkt->stream_index && wrap_bits <= 64 &&
+                        av_compare_mod(next_pkt->dts, pktl->pkt.dts, 2ULL << (wrap_bits - 1)) < 0) {
+                        if (av_compare_mod(pktl->pkt.pts, pktl->pkt.dts, 2ULL << (wrap_bits - 1))) {
                             // not B-frame
                             next_pkt->pts = pktl->pkt.dts;
                         }
-- 
2.15.0.448.gf294e3d99a-goog