diff mbox series

[FFmpeg-devel,3/4] swscale/output: Fix integer overflow in yuv2rgb_write_full() with out of range input

Message ID 20200216194303.819-3-michael@niedermayer.cc
State Accepted
Headers show
Series [FFmpeg-devel,1/4] avformat/swfenc: Fix integer overflow in frame rate handling | expand

Checks

Context Check Description
andriy/ffmpeg-patchwork success Make fate finished

Commit Message

Michael Niedermayer Feb. 16, 2020, 7:43 p.m. UTC 
Fixes: signed integer overflow: 1169365504 + 981452800 cannot be represented in type 'int'
Fixes: ticket8293

Found-by: Suhwan
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libswscale/output.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

Comments

Michael Niedermayer April 4, 2020, 8:20 p.m. UTC | #1 
On Sun, Feb 16, 2020 at 08:43:02PM +0100, Michael Niedermayer wrote:
> Fixes: signed integer overflow: 1169365504 + 981452800 cannot be represented in type 'int'
> Fixes: ticket8293
> 
> Found-by: Suhwan
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>  libswscale/output.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)

will apply

[...]
diff mbox series

Patch

diff --git a/libswscale/output.c b/libswscale/output.c
index a793a89443..04595d2d01 100644
--- a/libswscale/output.c
+++ b/libswscale/output.c
@@ -1850,9 +1850,9 @@  static av_always_inline void yuv2rgb_write_full(SwsContext *c,
     Y -= c->yuv2rgb_y_offset;
     Y *= c->yuv2rgb_y_coeff;
     Y += 1 << 21;
-    R = Y + V*c->yuv2rgb_v2r_coeff;
-    G = Y + V*c->yuv2rgb_v2g_coeff + U*c->yuv2rgb_u2g_coeff;
-    B = Y +                          U*c->yuv2rgb_u2b_coeff;
+    R = (unsigned)Y + V*c->yuv2rgb_v2r_coeff;
+    G = (unsigned)Y + V*c->yuv2rgb_v2g_coeff + U*c->yuv2rgb_u2g_coeff;
+    B = (unsigned)Y +                          U*c->yuv2rgb_u2b_coeff;
     if ((R | G | B) & 0xC0000000) {
         R = av_clip_uintp2(R, 30);
         G = av_clip_uintp2(G, 30);