diff mbox

[FFmpeg-devel,1/2] avcodec/svq3: Fix overflow in svq3_add_idct_c()

Message ID 20170918230453.19774-1-michael@niedermayer.cc
State Accepted
Commit 2c933c51687db958d8045d25ed87848342e869f6
Headers show

Commit Message

Michael Niedermayer Sept. 18, 2017, 11:04 p.m. UTC
Fixes: runtime error: signed integer overflow: 2147392585 + 524288 cannot be represented in type 'int'
Fixes: 3348/clusterfuzz-testcase-minimized-4809500517203968

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavcodec/svq3.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Michael Niedermayer Sept. 22, 2017, 9:31 a.m. UTC | #1
On Tue, Sep 19, 2017 at 01:04:52AM +0200, Michael Niedermayer wrote:
> Fixes: runtime error: signed integer overflow: 2147392585 + 524288 cannot be represented in type 'int'
> Fixes: 3348/clusterfuzz-testcase-minimized-4809500517203968
> 
> Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>  libavcodec/svq3.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

patchset applied

[...]
diff mbox

Patch

diff --git a/libavcodec/svq3.c b/libavcodec/svq3.c
index a766fa49ad..5cb5bd45b7 100644
--- a/libavcodec/svq3.c
+++ b/libavcodec/svq3.c
@@ -285,7 +285,7 @@  static void svq3_add_idct_c(uint8_t *dst, int16_t *block,
         const unsigned z1 = 13 * (block[i + 4 * 0] -      block[i + 4 * 2]);
         const unsigned z2 =  7 *  block[i + 4 * 1] - 17 * block[i + 4 * 3];
         const unsigned z3 = 17 *  block[i + 4 * 1] +  7 * block[i + 4 * 3];
-        const int rr = (dc + 0x80000);
+        const int rr = (dc + 0x80000u);
 
         dst[i + stride * 0] = av_clip_uint8(dst[i + stride * 0] + ((int)((z0 + z3) * qmul + rr) >> 20));
         dst[i + stride * 1] = av_clip_uint8(dst[i + stride * 1] + ((int)((z1 + z2) * qmul + rr) >> 20));