| Message ID | 20170521001221.30906-1-michael@niedermayer.cc |
|---|---|
| State | Accepted |
| Commit | d4ee76780869c659a5d3b0815c56024ab260a81d |
| Headers | show |
Hi, On Sat, May 20, 2017 at 8:12 PM, Michael Niedermayer <michael@niedermayer.cc > wrote: > Fixes: 1710/clusterfuzz-testcase-minimized-4837032931098624 > > Found-by: continuous fuzzing process https://github.com/google/oss- > fuzz/tree/master/projects/ffmpeg > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc> > --- > libavcodec/vp9block.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/libavcodec/vp9block.c b/libavcodec/vp9block.c > index ae2f0e4c6f..a16ccdccdb 100644 > --- a/libavcodec/vp9block.c > +++ b/libavcodec/vp9block.c > @@ -915,9 +915,9 @@ skip_eob: > if (!--band_left) > band_left = band_counts[++band]; > if (is_tx32x32) > - STORE_COEF(coef, rc, ((vp8_rac_get(c) ? -val : val) * > qmul[!!i]) / 2); > + STORE_COEF(coef, rc, (int)((vp8_rac_get(c) ? -val : val) * > (unsigned)qmul[!!i]) / 2); > else > - STORE_COEF(coef, rc, (vp8_rac_get(c) ? -val : val) * > qmul[!!i]); > + STORE_COEF(coef, rc, (vp8_rac_get(c) ? -val : val) * > (unsigned)qmul[!!i]); > nnz = (1 + cache[nb[i][0]] + cache[nb[i][1]]) >> 1; > tp = p[band][nnz]; > } while (++i < n_coeffs); > -- > 2.13.0 Since this is the only use of qmul[], why don't you make the array unsigned instead? That saves a cast. Ronald
On Sat, May 20, 2017 at 10:35:34PM -0400, Ronald S. Bultje wrote: > Hi, > > On Sat, May 20, 2017 at 8:12 PM, Michael Niedermayer <michael@niedermayer.cc > > wrote: > > > Fixes: 1710/clusterfuzz-testcase-minimized-4837032931098624 > > > > Found-by: continuous fuzzing process https://github.com/google/oss- > > fuzz/tree/master/projects/ffmpeg > > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc> > > --- > > libavcodec/vp9block.c | 4 ++-- > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > diff --git a/libavcodec/vp9block.c b/libavcodec/vp9block.c > > index ae2f0e4c6f..a16ccdccdb 100644 > > --- a/libavcodec/vp9block.c > > +++ b/libavcodec/vp9block.c > > @@ -915,9 +915,9 @@ skip_eob: > > if (!--band_left) > > band_left = band_counts[++band]; > > if (is_tx32x32) > > - STORE_COEF(coef, rc, ((vp8_rac_get(c) ? -val : val) * > > qmul[!!i]) / 2); > > + STORE_COEF(coef, rc, (int)((vp8_rac_get(c) ? -val : val) * > > (unsigned)qmul[!!i]) / 2); > > else > > - STORE_COEF(coef, rc, (vp8_rac_get(c) ? -val : val) * > > qmul[!!i]); > > + STORE_COEF(coef, rc, (vp8_rac_get(c) ? -val : val) * > > (unsigned)qmul[!!i]); > > nnz = (1 + cache[nb[i][0]] + cache[nb[i][1]]) >> 1; > > tp = p[band][nnz]; > > } while (++i < n_coeffs); > > -- > > 2.13.0 > > > Since this is the only use of qmul[], why don't you make the array unsigned > instead? That saves a cast. will change and apply thx [...]
On Sun, May 21, 2017 at 12:51:56PM +0200, Michael Niedermayer wrote: > On Sat, May 20, 2017 at 10:35:34PM -0400, Ronald S. Bultje wrote: > > Hi, > > > > On Sat, May 20, 2017 at 8:12 PM, Michael Niedermayer <michael@niedermayer.cc > > > wrote: > > > > > Fixes: 1710/clusterfuzz-testcase-minimized-4837032931098624 > > > > > > Found-by: continuous fuzzing process https://github.com/google/oss- > > > fuzz/tree/master/projects/ffmpeg > > > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc> > > > --- > > > libavcodec/vp9block.c | 4 ++-- > > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > > > diff --git a/libavcodec/vp9block.c b/libavcodec/vp9block.c > > > index ae2f0e4c6f..a16ccdccdb 100644 > > > --- a/libavcodec/vp9block.c > > > +++ b/libavcodec/vp9block.c > > > @@ -915,9 +915,9 @@ skip_eob: > > > if (!--band_left) > > > band_left = band_counts[++band]; > > > if (is_tx32x32) > > > - STORE_COEF(coef, rc, ((vp8_rac_get(c) ? -val : val) * > > > qmul[!!i]) / 2); > > > + STORE_COEF(coef, rc, (int)((vp8_rac_get(c) ? -val : val) * > > > (unsigned)qmul[!!i]) / 2); > > > else > > > - STORE_COEF(coef, rc, (vp8_rac_get(c) ? -val : val) * > > > qmul[!!i]); > > > + STORE_COEF(coef, rc, (vp8_rac_get(c) ? -val : val) * > > > (unsigned)qmul[!!i]); > > > nnz = (1 + cache[nb[i][0]] + cache[nb[i][1]]) >> 1; > > > tp = p[band][nnz]; > > > } while (++i < n_coeffs); > > > -- > > > 2.13.0 > > > > > > Since this is the only use of qmul[], why don't you make the array unsigned > > instead? That saves a cast. > > will change and apply actually, no i cant do that qmul is int16_t *, uint16_t * will be "promoted" to signed int do you see another way ? thx [...]
Hi, On Sun, May 21, 2017 at 6:57 AM, Michael Niedermayer <michael@niedermayer.cc > wrote: > On Sun, May 21, 2017 at 12:51:56PM +0200, Michael Niedermayer wrote: > > On Sat, May 20, 2017 at 10:35:34PM -0400, Ronald S. Bultje wrote: > > > Hi, > > > > > > On Sat, May 20, 2017 at 8:12 PM, Michael Niedermayer > <michael@niedermayer.cc > > > > wrote: > > > > > > > Fixes: 1710/clusterfuzz-testcase-minimized-4837032931098624 > > > > > > > > Found-by: continuous fuzzing process https://github.com/google/oss- > > > > fuzz/tree/master/projects/ffmpeg > > > > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc> > > > > --- > > > > libavcodec/vp9block.c | 4 ++-- > > > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > > > > > diff --git a/libavcodec/vp9block.c b/libavcodec/vp9block.c > > > > index ae2f0e4c6f..a16ccdccdb 100644 > > > > --- a/libavcodec/vp9block.c > > > > +++ b/libavcodec/vp9block.c > > > > @@ -915,9 +915,9 @@ skip_eob: > > > > if (!--band_left) > > > > band_left = band_counts[++band]; > > > > if (is_tx32x32) > > > > - STORE_COEF(coef, rc, ((vp8_rac_get(c) ? -val : val) * > > > > qmul[!!i]) / 2); > > > > + STORE_COEF(coef, rc, (int)((vp8_rac_get(c) ? -val : > val) * > > > > (unsigned)qmul[!!i]) / 2); > > > > else > > > > - STORE_COEF(coef, rc, (vp8_rac_get(c) ? -val : val) * > > > > qmul[!!i]); > > > > + STORE_COEF(coef, rc, (vp8_rac_get(c) ? -val : val) * > > > > (unsigned)qmul[!!i]); > > > > nnz = (1 + cache[nb[i][0]] + cache[nb[i][1]]) >> 1; > > > > tp = p[band][nnz]; > > > > } while (++i < n_coeffs); > > > > -- > > > > 2.13.0 > > > > > > > > > Since this is the only use of qmul[], why don't you make the array > unsigned > > > instead? That saves a cast. > > > > will change and apply > > actually, no i cant do that > qmul is int16_t *, uint16_t * will be "promoted" to signed int > > do you see another way ? Hm, crap, you're right. OK, existing patch is fine. (Changing type to unsigned is also fine, but I probably slightly prefer the cast so the in-memory representation is smaller.) Ronald
On Sun, May 21, 2017 at 07:25:10AM -0400, Ronald S. Bultje wrote: > Hi, > > On Sun, May 21, 2017 at 6:57 AM, Michael Niedermayer <michael@niedermayer.cc > > wrote: > > > On Sun, May 21, 2017 at 12:51:56PM +0200, Michael Niedermayer wrote: > > > On Sat, May 20, 2017 at 10:35:34PM -0400, Ronald S. Bultje wrote: > > > > Hi, > > > > > > > > On Sat, May 20, 2017 at 8:12 PM, Michael Niedermayer > > <michael@niedermayer.cc > > > > > wrote: > > > > > > > > > Fixes: 1710/clusterfuzz-testcase-minimized-4837032931098624 > > > > > > > > > > Found-by: continuous fuzzing process https://github.com/google/oss- > > > > > fuzz/tree/master/projects/ffmpeg > > > > > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc> > > > > > --- > > > > > libavcodec/vp9block.c | 4 ++-- > > > > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > > > > > > > diff --git a/libavcodec/vp9block.c b/libavcodec/vp9block.c > > > > > index ae2f0e4c6f..a16ccdccdb 100644 > > > > > --- a/libavcodec/vp9block.c > > > > > +++ b/libavcodec/vp9block.c > > > > > @@ -915,9 +915,9 @@ skip_eob: > > > > > if (!--band_left) > > > > > band_left = band_counts[++band]; > > > > > if (is_tx32x32) > > > > > - STORE_COEF(coef, rc, ((vp8_rac_get(c) ? -val : val) * > > > > > qmul[!!i]) / 2); > > > > > + STORE_COEF(coef, rc, (int)((vp8_rac_get(c) ? -val : > > val) * > > > > > (unsigned)qmul[!!i]) / 2); > > > > > else > > > > > - STORE_COEF(coef, rc, (vp8_rac_get(c) ? -val : val) * > > > > > qmul[!!i]); > > > > > + STORE_COEF(coef, rc, (vp8_rac_get(c) ? -val : val) * > > > > > (unsigned)qmul[!!i]); > > > > > nnz = (1 + cache[nb[i][0]] + cache[nb[i][1]]) >> 1; > > > > > tp = p[band][nnz]; > > > > > } while (++i < n_coeffs); > > > > > -- > > > > > 2.13.0 > > > > > > > > > > > > Since this is the only use of qmul[], why don't you make the array > > unsigned > > > > instead? That saves a cast. > > > > > > will change and apply > > > > actually, no i cant do that > > qmul is int16_t *, uint16_t * will be "promoted" to signed int > > > > do you see another way ? > > > Hm, crap, you're right. OK, existing patch is fine. (Changing type to > unsigned is also fine, but I probably slightly prefer the cast so the > in-memory representation is smaller.) ok, will apply thx [...]
diff --git a/libavcodec/vp9block.c b/libavcodec/vp9block.c index ae2f0e4c6f..a16ccdccdb 100644 --- a/libavcodec/vp9block.c +++ b/libavcodec/vp9block.c @@ -915,9 +915,9 @@ skip_eob: if (!--band_left) band_left = band_counts[++band]; if (is_tx32x32) - STORE_COEF(coef, rc, ((vp8_rac_get(c) ? -val : val) * qmul[!!i]) / 2); + STORE_COEF(coef, rc, (int)((vp8_rac_get(c) ? -val : val) * (unsigned)qmul[!!i]) / 2); else - STORE_COEF(coef, rc, (vp8_rac_get(c) ? -val : val) * qmul[!!i]); + STORE_COEF(coef, rc, (vp8_rac_get(c) ? -val : val) * (unsigned)qmul[!!i]); nnz = (1 + cache[nb[i][0]] + cache[nb[i][1]]) >> 1; tp = p[band][nnz]; } while (++i < n_coeffs);
Fixes: 1710/clusterfuzz-testcase-minimized-4837032931098624 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc> --- libavcodec/vp9block.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)