| Message ID | 20190903001426.8957-1-michael@niedermayer.cc |
|---|---|
| State | Accepted |
| Commit | 9d3ddef519e88c40c05be8cb94cd9e71c0957ec7 |
| Headers | show |
tis 2019-09-03 klockan 02:14 +0200 skrev Michael Niedermayer: > Fixes: signed integer overflow: 2147480546 + 4096 cannot be represented in type 'int' > Fixes: 16280/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5123442566758400 > > Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc> > --- > libavcodec/apedec.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c > index 490b11b94e..ed22f0f019 100644 > --- a/libavcodec/apedec.c > +++ b/libavcodec/apedec.c > @@ -1266,7 +1266,7 @@ static void do_apply_filter(APEContext *ctx, int version, APEFilter *f, > f->delay - order, > f->adaptcoeffs - order, > order, APESIGN(*data)); > - res = (res + (1 << (fracbits - 1))) >> fracbits; > + res = (int)(res + (1U << (fracbits - 1))) >> fracbits; Does this mean the old code was decoding incorrectly, or is this just an UB fix? (fixing UBs is good of course) /Tomas
On Tue, Sep 03, 2019 at 12:16:47PM +0200, Tomas Härdin wrote: > tis 2019-09-03 klockan 02:14 +0200 skrev Michael Niedermayer: > > Fixes: signed integer overflow: 2147480546 + 4096 cannot be represented in type 'int' > > Fixes: 16280/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5123442566758400 > > > > Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg > > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc> > > --- > > libavcodec/apedec.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c > > index 490b11b94e..ed22f0f019 100644 > > --- a/libavcodec/apedec.c > > +++ b/libavcodec/apedec.c > > @@ -1266,7 +1266,7 @@ static void do_apply_filter(APEContext *ctx, int version, APEFilter *f, > > f->delay - order, > > f->adaptcoeffs - order, > > order, APESIGN(*data)); > > - res = (res + (1 << (fracbits - 1))) >> fracbits; > > + res = (int)(res + (1U << (fracbits - 1))) >> fracbits; > > Does this mean the old code was decoding incorrectly, or is this just > an UB fix? (fixing UBs is good of course) it is just a UB fix. If iam not misreading the reference source then it has the same UB in it so there is no correct way in case this ever triggers on a real file. The specific patch here was done so that the result should hopefully match what a common machine with 32bit int (most platforms) would likely produce. thanks [...]
tis 2019-09-03 klockan 13:25 +0200 skrev Michael Niedermayer: > On Tue, Sep 03, 2019 at 12:16:47PM +0200, Tomas Härdin wrote: > > tis 2019-09-03 klockan 02:14 +0200 skrev Michael Niedermayer: > > > Fixes: signed integer overflow: 2147480546 + 4096 cannot be represented in type 'int' > > > Fixes: 16280/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5123442566758400 > > > > > > Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg > > > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc> > > > --- > > > libavcodec/apedec.c | 2 +- > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c > > > index 490b11b94e..ed22f0f019 100644 > > > --- a/libavcodec/apedec.c > > > +++ b/libavcodec/apedec.c > > > @@ -1266,7 +1266,7 @@ static void do_apply_filter(APEContext *ctx, int version, APEFilter *f, > > > f->delay - order, > > > f->adaptcoeffs - order, > > > order, APESIGN(*data)); > > > - res = (res + (1 << (fracbits - 1))) >> fracbits; > > > + res = (int)(res + (1U << (fracbits - 1))) >> fracbits; > > > > Does this mean the old code was decoding incorrectly, or is this just > > an UB fix? (fixing UBs is good of course) > > it is just a UB fix. > If iam not misreading the reference source then it has the same UB in it > so there is no correct way in case this ever triggers on a real file. > > The specific patch here was done so that the result should hopefully > match what a common machine with 32bit int (most platforms) would > likely produce. Right, and the cast back to int is required for proper sign extension in the >> fracbits. Looks OK to me then. Kind of a weird codec feature IMO /Tomas
On Tue, Sep 03, 2019 at 06:58:03PM +0200, Tomas Härdin wrote: > tis 2019-09-03 klockan 13:25 +0200 skrev Michael Niedermayer: > > On Tue, Sep 03, 2019 at 12:16:47PM +0200, Tomas Härdin wrote: > > > tis 2019-09-03 klockan 02:14 +0200 skrev Michael Niedermayer: > > > > Fixes: signed integer overflow: 2147480546 + 4096 cannot be represented in type 'int' > > > > Fixes: 16280/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5123442566758400 > > > > > > > > Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg > > > > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc> > > > > --- > > > > libavcodec/apedec.c | 2 +- > > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > > > diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c > > > > index 490b11b94e..ed22f0f019 100644 > > > > --- a/libavcodec/apedec.c > > > > +++ b/libavcodec/apedec.c > > > > @@ -1266,7 +1266,7 @@ static void do_apply_filter(APEContext *ctx, int version, APEFilter *f, > > > > f->delay - order, > > > > f->adaptcoeffs - order, > > > > order, APESIGN(*data)); > > > > - res = (res + (1 << (fracbits - 1))) >> fracbits; > > > > + res = (int)(res + (1U << (fracbits - 1))) >> fracbits; > > > > > > Does this mean the old code was decoding incorrectly, or is this just > > > an UB fix? (fixing UBs is good of course) > > > > it is just a UB fix. > > If iam not misreading the reference source then it has the same UB in it > > so there is no correct way in case this ever triggers on a real file. > > > > The specific patch here was done so that the result should hopefully > > match what a common machine with 32bit int (most platforms) would > > likely produce. > > Right, and the cast back to int is required for proper sign extension > in the >> fracbits. Looks OK to me then. Kind of a weird codec feature > IMO ok, will apply thanks [...]
diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c index 490b11b94e..ed22f0f019 100644 --- a/libavcodec/apedec.c +++ b/libavcodec/apedec.c @@ -1266,7 +1266,7 @@ static void do_apply_filter(APEContext *ctx, int version, APEFilter *f, f->delay - order, f->adaptcoeffs - order, order, APESIGN(*data)); - res = (res + (1 << (fracbits - 1))) >> fracbits; + res = (int)(res + (1U << (fracbits - 1))) >> fracbits; res += *data; *data++ = res;
Fixes: signed integer overflow: 2147480546 + 4096 cannot be represented in type 'int' Fixes: 16280/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5123442566758400 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc> --- libavcodec/apedec.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)