| Message ID | 20230428044248.42988-1-etemesicaleb@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | [FFmpeg-devel] avcodec/jpeg2000dec: Ensure calculation of buf_size cannot overflow. | expand |
| Context | Check | Description |
|---|---|---|
| andriy/make_x86 | success | Make finished |
| andriy/make_fate_x86 | success | Make fate finished |
On Fri, Apr 28, 2023 at 07:42:48AM +0300, etemesicaleb@gmail.com wrote: > From: caleb <etemesicaleb@gmail.com> > > --- > libavcodec/jpeg2000htdec.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/libavcodec/jpeg2000htdec.c b/libavcodec/jpeg2000htdec.c > index 51cd96e0f1..d77293ddd8 100644 > --- a/libavcodec/jpeg2000htdec.c > +++ b/libavcodec/jpeg2000htdec.c > @@ -595,7 +595,7 @@ static int jpeg2000_decode_ht_cleanup_segment(const Jpeg2000DecoderContext *s, > const uint16_t quad_width = ff_jpeg2000_ceildivpow2(width, 1); > const uint16_t quad_height = ff_jpeg2000_ceildivpow2(height, 1); > > - size_t buf_size = 4 * quad_width * quad_height; > + size_t buf_size = 4UL * quad_width * quad_height; The only caller asserts that width and height are <= 1024 so this change should not be needed 4 is an int and int is by POSIX 32bit at least if you want to make the code "nicer", i would suggest changing the quad_width and quad_height to int. uint16_t should generally only be used if theres a reason. (space for an array can be a reason so can it be to intentionally truncate to 16bit) But needing only 16bit is a bad reason. uint16_t is generally not faster, in fact C treats it like a int anyway thx [...]
diff --git a/libavcodec/jpeg2000htdec.c b/libavcodec/jpeg2000htdec.c index 51cd96e0f1..d77293ddd8 100644 --- a/libavcodec/jpeg2000htdec.c +++ b/libavcodec/jpeg2000htdec.c @@ -595,7 +595,7 @@ static int jpeg2000_decode_ht_cleanup_segment(const Jpeg2000DecoderContext *s, const uint16_t quad_width = ff_jpeg2000_ceildivpow2(width, 1); const uint16_t quad_height = ff_jpeg2000_ceildivpow2(height, 1); - size_t buf_size = 4 * quad_width * quad_height; + size_t buf_size = 4UL * quad_width * quad_height; uint8_t *sigma_n = av_calloc(buf_size, sizeof(uint8_t)); uint8_t *E = av_calloc(buf_size, sizeof(uint8_t));
From: caleb <etemesicaleb@gmail.com> --- libavcodec/jpeg2000htdec.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)