diff mbox

[FFmpeg-devel,v2,07/15] swscale/utils: Fix invalid left shifts of negative numbers

Message ID 20190928022610.5903-7-andreas.rheinhardt@gmail.com
State Accepted
Commit e2646e23be69bdef1e41d4decee1a4298701b8d1
Headers show

Commit Message

Andreas Rheinhardt Sept. 28, 2019, 2:26 a.m. UTC 
Affected the FATE-tests vsynth_lena-dv-411, vsynth1-dv-411,
vsynth2-dv-411 and hevc-paramchange-yuv420p.yuv420p10.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
---
 libswscale/utils.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Michael Niedermayer Sept. 28, 2019, 3:35 p.m. UTC | #1 
On Sat, Sep 28, 2019 at 04:26:02AM +0200, Andreas Rheinhardt wrote:
> Affected the FATE-tests vsynth_lena-dv-411, vsynth1-dv-411,
> vsynth2-dv-411 and hevc-paramchange-yuv420p.yuv420p10.
> 
> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
> ---
>  libswscale/utils.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

will apply

thx

[...]
diff mbox

Patch

diff --git a/libswscale/utils.c b/libswscale/utils.c
index 1b1f779532..57c4fd2b0f 100644
--- a/libswscale/utils.c
+++ b/libswscale/utils.c
@@ -390,7 +390,7 @@  static av_cold int initFilter(int16_t **outFilter, int32_t **filterPos,
             (*filterPos)[i] = xx;
             // bilinear upscale / linear interpolate / area averaging
             for (j = 0; j < filterSize; j++) {
-                int64_t coeff= fone - FFABS(((int64_t)xx<<16) - xDstInSrc)*(fone>>16);
+                int64_t coeff = fone - FFABS((int64_t)xx * (1 << 16) - xDstInSrc) * (fone >> 16);
                 if (coeff < 0)
                     coeff = 0;
                 filter[i * filterSize + j] = coeff;