diff mbox series

[FFmpeg-devel,2/4] avfilter/vf_paletteuse: Fix left shift outside of range of int

Message ID 20200122145210.6898-2-andreas.rheinhardt@gmail.com
State Accepted
Commit 797c2ecc8fca65721b79637073df0a31fe31c463
Headers show
Series [FFmpeg-devel,1/4] avcodec/startcode: Use AV_RN due to UBSan warning about unaligned access | expand

Checks

Context Check Description
andriy/ffmpeg-patchwork success Make fate finished

Commit Message

Andreas Rheinhardt Jan. 22, 2020, 2:52 p.m. UTC
by keeping the variable uint32_t which in this situation is the natural
type anyway. This affected the FATE-test filter-paletteuse-sierra2_4a.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
---
 libavfilter/vf_paletteuse.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

Comments

Andreas Rheinhardt March 26, 2021, 5:50 p.m. UTC | #1
Andreas Rheinhardt:
> by keeping the variable uint32_t which in this situation is the natural
> type anyway. This affected the FATE-test filter-paletteuse-sierra2_4a.
> 
> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
> ---
>  libavfilter/vf_paletteuse.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/libavfilter/vf_paletteuse.c b/libavfilter/vf_paletteuse.c
> index ed128813d6..c9603601aa 100644
> --- a/libavfilter/vf_paletteuse.c
> +++ b/libavfilter/vf_paletteuse.c
> @@ -158,9 +158,10 @@ static int query_formats(AVFilterContext *ctx)
>      return 0;
>  }
>  
> -static av_always_inline int dither_color(uint32_t px, int er, int eg, int eb, int scale, int shift)
> +static av_always_inline uint32_t dither_color(uint32_t px, int er, int eg,
> +                                              int eb, int scale, int shift)
>  {
> -    return av_clip_uint8( px >> 24                                      ) << 24
> +    return                px >> 24                                        << 24
>           | av_clip_uint8((px >> 16 & 0xff) + ((er * scale) / (1<<shift))) << 16
>           | av_clip_uint8((px >>  8 & 0xff) + ((eg * scale) / (1<<shift))) <<  8
>           | av_clip_uint8((px       & 0xff) + ((eb * scale) / (1<<shift)));
> 
Will apply.

- Andreas
diff mbox series

Patch

diff --git a/libavfilter/vf_paletteuse.c b/libavfilter/vf_paletteuse.c
index ed128813d6..c9603601aa 100644
--- a/libavfilter/vf_paletteuse.c
+++ b/libavfilter/vf_paletteuse.c
@@ -158,9 +158,10 @@  static int query_formats(AVFilterContext *ctx)
     return 0;
 }
 
-static av_always_inline int dither_color(uint32_t px, int er, int eg, int eb, int scale, int shift)
+static av_always_inline uint32_t dither_color(uint32_t px, int er, int eg,
+                                              int eb, int scale, int shift)
 {
-    return av_clip_uint8( px >> 24                                      ) << 24
+    return                px >> 24                                        << 24
          | av_clip_uint8((px >> 16 & 0xff) + ((er * scale) / (1<<shift))) << 16
          | av_clip_uint8((px >>  8 & 0xff) + ((eg * scale) / (1<<shift))) <<  8
          | av_clip_uint8((px       & 0xff) + ((eb * scale) / (1<<shift)));