[FFmpeg-devel] avcodec/dirac_vlc: limit res_bits in APPEND_RESIDUE()

Submitted by Michael Niedermayer on Sept. 9, 2017, 1:51 p.m.

Details

Message ID 20170909135145.13358-1-michael@niedermayer.cc
State New
Headers show

Commit Message

Michael Niedermayer Sept. 9, 2017, 1:51 p.m.
Fixes: runtime error: left shift of 1073741838 by 1 places cannot be represented in type 'int32_t' (aka 'int')
Fixes: 3279/clusterfuzz-testcase-minimized-4564805744590848

Suggested-by: <atomnuker>
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavcodec/dirac_vlc.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)

Comments

Michael Niedermayer Sept. 9, 2017, 5:06 p.m.
On Sat, Sep 09, 2017 at 03:51:45PM +0200, Michael Niedermayer wrote:
> Fixes: runtime error: left shift of 1073741838 by 1 places cannot be represented in type 'int32_t' (aka 'int')
> Fixes: 3279/clusterfuzz-testcase-minimized-4564805744590848
> 
> Suggested-by: <atomnuker>
> Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>  libavcodec/dirac_vlc.c | 8 +-------
>  1 file changed, 1 insertion(+), 7 deletions(-)

applied with changes suggested by atomnuker

[...]

Patch hide | download patch | download mbox

diff --git a/libavcodec/dirac_vlc.c b/libavcodec/dirac_vlc.c
index bd37f31f46..bb525f89d8 100644
--- a/libavcodec/dirac_vlc.c
+++ b/libavcodec/dirac_vlc.c
@@ -37,7 +37,7 @@ 
 
 #define APPEND_RESIDUE(N, M)                                                   \
     N          |= M >> (N ## _bits);                                           \
-    N ## _bits +=      (M ## _bits)
+    N ## _bits  = (N ## _bits + (M ## _bits)) & 63
 
 int ff_dirac_golomb_read_32bit(DiracGolombLUT *lut_ctx, const uint8_t *buf,
                                int bytes, uint8_t *_dst, int coeffs)
@@ -56,9 +56,6 @@  int ff_dirac_golomb_read_32bit(DiracGolombLUT *lut_ctx, const uint8_t *buf,
         if ((c_idx + 1) > coeffs)
             return c_idx;
 
-        if (res_bits >= RSIZE_BITS)
-            res_bits = res = 0;
-
         /* res_bits is a hint for better branch prediction */
         if (res_bits && l->sign) {
             int32_t coeff = 1;
@@ -99,9 +96,6 @@  int ff_dirac_golomb_read_16bit(DiracGolombLUT *lut_ctx, const uint8_t *buf,
         if ((c_idx + 1) > coeffs)
             return c_idx;
 
-        if (res_bits >= RSIZE_BITS)
-            res_bits = res = 0;
-
         if (res_bits && l->sign) {
             int32_t coeff = 1;
             APPEND_RESIDUE(res, l->preamble);