diff mbox series

[FFmpeg-devel,1/2] avcodec/diracdsp: Fix integer anomaly in dequant_subband_*

Message ID 20200718204727.29121-1-michael@niedermayer.cc
State Accepted
Commit ca3c6c981aa5b0af8a5576020b79fdd3cdf9ae9e
Headers show
Series [FFmpeg-devel,1/2] avcodec/diracdsp: Fix integer anomaly in dequant_subband_*
Related show

Checks

Context Check Description
andriy/default pending
andriy/make_warn warning New warnings during build
andriy/make success Make finished
andriy/make_fate success Make fate finished

Commit Message

Michael Niedermayer July 18, 2020, 8:47 p.m. UTC
Fixes: negation of -2147483648 cannot be represented in type 'int32_t' (aka 'int'); cast to an unsigned type to negate this value to itself
Fixes: 23760/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-604209011412172

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavcodec/diracdsp.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

Comments

Michael Niedermayer Sept. 18, 2020, 10:42 p.m. UTC | #1
On Sat, Jul 18, 2020 at 10:47:26PM +0200, Michael Niedermayer wrote:
> Fixes: negation of -2147483648 cannot be represented in type 'int32_t' (aka 'int'); cast to an unsigned type to negate this value to itself
> Fixes: 23760/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-604209011412172
> 
> Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>  libavcodec/diracdsp.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)

will apply patchset

[...]
diff mbox series

Patch

diff --git a/libavcodec/diracdsp.c b/libavcodec/diracdsp.c
index 2dd56f83f3..4e08d3817e 100644
--- a/libavcodec/diracdsp.c
+++ b/libavcodec/diracdsp.c
@@ -198,9 +198,9 @@  static void dequant_subband_ ## PX ## _c(uint8_t *src, uint8_t *dst, ptrdiff_t s
         PX c, sign, *src_r = (PX *)src, *dst_r = (PX *)dst;                                \
         for (i = 0; i < tot_h; i++) {                                                      \
             c = *src_r++;                                                                  \
-            sign = FFSIGN(c)*(!!c);                                                        \
-            c = (FFABS(c)*(unsigned)qf + qs) >> 2;                                                   \
-            *dst_r++ = c*sign;                                                             \
+            if     (c < 0) c = -((-(unsigned)c*qf + qs) >> 2);                             \
+            else if(c > 0) c =  (( (unsigned)c*qf + qs) >> 2);                             \
+            *dst_r++ = c;                                                                  \
         }                                                                                  \
         src += tot_h << (sizeof(PX) >> 1);                                                 \
         dst += stride;                                                                     \