diff mbox

[FFmpeg-devel,10/10] avutil/common: Fix undefined shift

Message ID 20190918032607.11774-10-andreas.rheinhardt@gmail.com
State Accepted
Commit ebd25a5ba532633326d3944eda2e982380ef6626
Headers show

Commit Message

Andreas Rheinhardt Sept. 18, 2019, 3:26 a.m. UTC
av_mod_uintp2_c uses a bitwise AND with (1 << p) - 1 to clear the high
bits of an unsigned int. But this is undefined if p == 31, because 1 is
an int and 2^31 is not representable in an int. So make 1 unsigned.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
---
 libavutil/common.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Michael Niedermayer Sept. 18, 2019, 9:03 p.m. UTC | #1
On Wed, Sep 18, 2019 at 05:26:07AM +0200, Andreas Rheinhardt wrote:
> av_mod_uintp2_c uses a bitwise AND with (1 << p) - 1 to clear the high
> bits of an unsigned int. But this is undefined if p == 31, because 1 is
> an int and 2^31 is not representable in an int. So make 1 unsigned.
> 
> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
> ---
>  libavutil/common.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

will apply

thx

[...]
diff mbox

Patch

diff --git a/libavutil/common.h b/libavutil/common.h
index 8db0291170..af35397eb9 100644
--- a/libavutil/common.h
+++ b/libavutil/common.h
@@ -240,7 +240,7 @@  static av_always_inline av_const unsigned av_clip_uintp2_c(int a, int p)
  */
 static av_always_inline av_const unsigned av_mod_uintp2_c(unsigned a, unsigned p)
 {
-    return a & ((1 << p) - 1);
+    return a & ((1U << p) - 1);
 }
 
 /**