diff mbox series

[FFmpeg-devel,2/2] avfilter/vf_signature: Fix integer overflow in filter_frame()

Message ID 20220518155505.4912-2-michael@niedermayer.cc
State New
Headers show
Series [FFmpeg-devel,1/2] avformat/http: Remove always true non NULL check | expand

Checks

Context Check Description
yinshiyou/make_loongarch64 success Make finished
yinshiyou/make_fate_loongarch64 success Make fate finished
andriy/make_x86 success Make finished
andriy/make_fate_x86 success Make fate finished

Commit Message

Michael Niedermayer May 18, 2022, 3:55 p.m. UTC
Fixes: CID1403233

The second of the 2 changes may be unneeded but will help coverity

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavfilter/vf_signature.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
diff mbox series

Patch

diff --git a/libavfilter/vf_signature.c b/libavfilter/vf_signature.c
index 4ca57ebf1d..66149dcc01 100644
--- a/libavfilter/vf_signature.c
+++ b/libavfilter/vf_signature.c
@@ -219,7 +219,7 @@  static int filter_frame(AVFilterLink *inlink, AVFrame *picref)
     dw1 = inlink->w / 32;
     if (inlink->w % 32)
         dw2 = dw1 + 1;
-    denom = (sc->divide) ? dh1 * dh2 * dw1 * dw2 : 1;
+    denom = (sc->divide) ? dh1 * (int64_t)dh2 * dw1 * dw2 : 1;
 
     for (i = 0; i < 32; i++) {
         rowcount = 0;
@@ -245,7 +245,7 @@  static int filter_frame(AVFilterLink *inlink, AVFrame *picref)
         }
     }
 
-    denom = (sc->divide) ? 1 : dh1 * dh2 * dw1 * dw2;
+    denom = (sc->divide) ? 1 : dh1 * (int64_t)dh2 * dw1 * dw2;
 
     for (i = 0; i < ELEMENT_COUNT; i++) {
         const ElemCat* elemcat = elements[i];