diff mbox series

[FFmpeg-devel,3/4] avcodec/pixlet: Avoid signed integer overflow in scaling in filterfn()

Message ID 20201027162119.19025-3-michael@niedermayer.cc
State Accepted
Commit 0c1f20c6c858b753effda274b58ef635d1924915
Headers show
Series [FFmpeg-devel,1/4] avformat/aiffdec: Check packet size | expand

Checks

Context Check Description
andriy/x86_make success Make finished
andriy/x86_make_fate success Make fate finished

Commit Message

Michael Niedermayer Oct. 27, 2020, 4:21 p.m. UTC 
Fixes: signed integer overflow: 11494 * 1073741824000000 cannot be represented in type 'long'
Fixes: 26586/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PIXLET_fuzzer-5752633970917376

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavcodec/pixlet.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Comments

Michael Niedermayer March 20, 2022, 11:49 p.m. UTC | #1 
On Tue, Oct 27, 2020 at 05:21:18PM +0100, Michael Niedermayer wrote:
> Fixes: signed integer overflow: 11494 * 1073741824000000 cannot be represented in type 'long'
> Fixes: 26586/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PIXLET_fuzzer-5752633970917376
> 
> Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>  libavcodec/pixlet.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)

will apply

[...]
diff mbox series

Patch

diff --git a/libavcodec/pixlet.c b/libavcodec/pixlet.c
index 42acd683fa..fd5e2ef804 100644
--- a/libavcodec/pixlet.c
+++ b/libavcodec/pixlet.c
@@ -405,7 +405,7 @@  static void filterfn(int16_t *dest, int16_t *tmp, unsigned size, int64_t scale)
                 (int64_t) low [i - 1] * -INT64_C(325392907)  +
                 (int64_t) high[i + 0] *  INT64_C(1518500249) +
                 (int64_t) high[i - 1] *  INT64_C(1518500249);
-        dest[i * 2] = av_clip_int16(((value >> 32) * scale) >> 32);
+        dest[i * 2] = av_clip_int16(((value >> 32) * (uint64_t)scale) >> 32);
     }
 
     for (i = 0; i < hsize; i++) {
@@ -416,7 +416,7 @@  static void filterfn(int16_t *dest, int16_t *tmp, unsigned size, int64_t scale)
                 (int64_t) high[i + 1] *  INT64_C(303700064)  +
                 (int64_t) high[i + 0] * -INT64_C(3644400640) +
                 (int64_t) high[i - 1] *  INT64_C(303700064);
-        dest[i * 2 + 1] = av_clip_int16(((value >> 32) * scale) >> 32);
+        dest[i * 2 + 1] = av_clip_int16(((value >> 32) * (uint64_t)scale) >> 32);
     }
 }