diff mbox

[FFmpeg-devel,5/5] avcodec/flacdsp: Fix: runtime error: signed integer overflow: -1027555328 + -1226681270 cannot be represented in type 'int'

Message ID 20170225200726.7928-5-michael@niedermayer.cc
State Accepted
Commit 7e9ba78f6bd10edae77d1126ada109709e981e9f
Headers show

Commit Message

Michael Niedermayer Feb. 25, 2017, 8:07 p.m. UTC
Fixes: 673/clusterfuzz-testcase-5948736536576000

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavcodec/flacdsp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Paul B Mahol Feb. 25, 2017, 9 p.m. UTC | #1
On 2/25/17, Michael Niedermayer <michael@niedermayer.cc> wrote:
> Fixes: 673/clusterfuzz-testcase-5948736536576000
>
> Found-by: continuous fuzzing process
> https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>  libavcodec/flacdsp.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/libavcodec/flacdsp.c b/libavcodec/flacdsp.c
> index 560091f73a..bc9a5dbed9 100644
> --- a/libavcodec/flacdsp.c
> +++ b/libavcodec/flacdsp.c
> @@ -67,7 +67,7 @@ static void flac_lpc_16_c(int32_t *decoded, const int
> coeffs[32],
>          int sum = 0;
>          for (j = 0; j < pred_order; j++)
>              sum += coeffs[j] * (SUINT)decoded[j];
> -        decoded[j] += sum >> qlevel;
> +        decoded[j] = decoded[j] + (unsigned)(sum >> qlevel);
>      }
>  }
>

What about making sum unsigned instead?
Michael Niedermayer Feb. 25, 2017, 9:25 p.m. UTC | #2
On Sat, Feb 25, 2017 at 10:00:36PM +0100, Paul B Mahol wrote:
> On 2/25/17, Michael Niedermayer <michael@niedermayer.cc> wrote:
> > Fixes: 673/clusterfuzz-testcase-5948736536576000
> >
> > Found-by: continuous fuzzing process
> > https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
> > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> > ---
> >  libavcodec/flacdsp.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/libavcodec/flacdsp.c b/libavcodec/flacdsp.c
> > index 560091f73a..bc9a5dbed9 100644
> > --- a/libavcodec/flacdsp.c
> > +++ b/libavcodec/flacdsp.c
> > @@ -67,7 +67,7 @@ static void flac_lpc_16_c(int32_t *decoded, const int
> > coeffs[32],
> >          int sum = 0;
> >          for (j = 0; j < pred_order; j++)
> >              sum += coeffs[j] * (SUINT)decoded[j];
> > -        decoded[j] += sum >> qlevel;
> > +        decoded[j] = decoded[j] + (unsigned)(sum >> qlevel);
> >      }
> >  }
> >
> 
> What about making sum unsigned instead?

in "sum >> qlevel" sum needs to be signed to get the correct
result

I can also add a check for the overflow and return an error but thats
more code

[...]
Paul B Mahol Feb. 26, 2017, 9:06 a.m. UTC | #3
On 2/25/17, Michael Niedermayer <michael@niedermayer.cc> wrote:
> On Sat, Feb 25, 2017 at 10:00:36PM +0100, Paul B Mahol wrote:
>> On 2/25/17, Michael Niedermayer <michael@niedermayer.cc> wrote:
>> > Fixes: 673/clusterfuzz-testcase-5948736536576000
>> >
>> > Found-by: continuous fuzzing process
>> > https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
>> > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
>> > ---
>> >  libavcodec/flacdsp.c | 2 +-
>> >  1 file changed, 1 insertion(+), 1 deletion(-)
>> >
>> > diff --git a/libavcodec/flacdsp.c b/libavcodec/flacdsp.c
>> > index 560091f73a..bc9a5dbed9 100644
>> > --- a/libavcodec/flacdsp.c
>> > +++ b/libavcodec/flacdsp.c
>> > @@ -67,7 +67,7 @@ static void flac_lpc_16_c(int32_t *decoded, const int
>> > coeffs[32],
>> >          int sum = 0;
>> >          for (j = 0; j < pred_order; j++)
>> >              sum += coeffs[j] * (SUINT)decoded[j];
>> > -        decoded[j] += sum >> qlevel;
>> > +        decoded[j] = decoded[j] + (unsigned)(sum >> qlevel);
>> >      }
>> >  }
>> >
>>
>> What about making sum unsigned instead?
>
> in "sum >> qlevel" sum needs to be signed to get the correct
> result
>
> I can also add a check for the overflow and return an error but thats
> more code

no need, this is ok
Michael Niedermayer Feb. 26, 2017, 3:32 p.m. UTC | #4
On Sun, Feb 26, 2017 at 10:06:57AM +0100, Paul B Mahol wrote:
> On 2/25/17, Michael Niedermayer <michael@niedermayer.cc> wrote:
> > On Sat, Feb 25, 2017 at 10:00:36PM +0100, Paul B Mahol wrote:
> >> On 2/25/17, Michael Niedermayer <michael@niedermayer.cc> wrote:
> >> > Fixes: 673/clusterfuzz-testcase-5948736536576000
> >> >
> >> > Found-by: continuous fuzzing process
> >> > https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
> >> > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> >> > ---
> >> >  libavcodec/flacdsp.c | 2 +-
> >> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >> >
> >> > diff --git a/libavcodec/flacdsp.c b/libavcodec/flacdsp.c
> >> > index 560091f73a..bc9a5dbed9 100644
> >> > --- a/libavcodec/flacdsp.c
> >> > +++ b/libavcodec/flacdsp.c
> >> > @@ -67,7 +67,7 @@ static void flac_lpc_16_c(int32_t *decoded, const int
> >> > coeffs[32],
> >> >          int sum = 0;
> >> >          for (j = 0; j < pred_order; j++)
> >> >              sum += coeffs[j] * (SUINT)decoded[j];
> >> > -        decoded[j] += sum >> qlevel;
> >> > +        decoded[j] = decoded[j] + (unsigned)(sum >> qlevel);
> >> >      }
> >> >  }
> >> >
> >>
> >> What about making sum unsigned instead?
> >
> > in "sum >> qlevel" sum needs to be signed to get the correct
> > result
> >
> > I can also add a check for the overflow and return an error but thats
> > more code
> 
> no need, this is ok

applied

thx

[...]
diff mbox

Patch

diff --git a/libavcodec/flacdsp.c b/libavcodec/flacdsp.c
index 560091f73a..bc9a5dbed9 100644
--- a/libavcodec/flacdsp.c
+++ b/libavcodec/flacdsp.c
@@ -67,7 +67,7 @@  static void flac_lpc_16_c(int32_t *decoded, const int coeffs[32],
         int sum = 0;
         for (j = 0; j < pred_order; j++)
             sum += coeffs[j] * (SUINT)decoded[j];
-        decoded[j] += sum >> qlevel;
+        decoded[j] = decoded[j] + (unsigned)(sum >> qlevel);
     }
 }