diff mbox

[FFmpeg-devel,1/4] avcodec/dirac_dwt_template: Fix signedness regression in interleave()

Message ID 20180713172640.19675-1-michael@niedermayer.cc
State Accepted
Commit 181435a4de6e38e0a15ddaf16de9a157ef41cb18
Headers show

Commit Message

Michael Niedermayer July 13, 2018, 5:26 p.m. UTC
Found-by: <jdarnley>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavcodec/dirac_dwt_template.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Comments

James Darnley July 13, 2018, 5:59 p.m. UTC | #1
On 2018-07-13 19:26, Michael Niedermayer wrote:
> Found-by: <jdarnley>
> 
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>  libavcodec/dirac_dwt_template.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/dirac_dwt_template.c b/libavcodec/dirac_dwt_template.c
> index 528fc7e9e7..2369c8d15b 100644
> --- a/libavcodec/dirac_dwt_template.c
> +++ b/libavcodec/dirac_dwt_template.c
> @@ -57,8 +57,8 @@ static av_always_inline void RENAME(interleave)(TYPE *dst, TYPE *src0, TYPE *src
>  {
>      int i;
>      for (i = 0; i < w2; i++) {
> -        dst[2*i  ] = (src0[i] + (unsigned)add) >> shift;
> -        dst[2*i+1] = (src1[i] + (unsigned)add) >> shift;
> +        dst[2*i  ] = ((int)(src0[i] + (unsigned)add)) >> shift;
> +        dst[2*i+1] = ((int)(src1[i] + (unsigned)add)) >> shift;
>      }
>  }
>  
> 

This does fix the failure I saw.  I will submit a patch for more fate
tests which happen to cover this piece of code.
Michael Niedermayer July 13, 2018, 11:16 p.m. UTC | #2
On Fri, Jul 13, 2018 at 07:59:43PM +0200, James Darnley wrote:
> On 2018-07-13 19:26, Michael Niedermayer wrote:
> > Found-by: <jdarnley>
> > 
> > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> > ---
> >  libavcodec/dirac_dwt_template.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/libavcodec/dirac_dwt_template.c b/libavcodec/dirac_dwt_template.c
> > index 528fc7e9e7..2369c8d15b 100644
> > --- a/libavcodec/dirac_dwt_template.c
> > +++ b/libavcodec/dirac_dwt_template.c
> > @@ -57,8 +57,8 @@ static av_always_inline void RENAME(interleave)(TYPE *dst, TYPE *src0, TYPE *src
> >  {
> >      int i;
> >      for (i = 0; i < w2; i++) {
> > -        dst[2*i  ] = (src0[i] + (unsigned)add) >> shift;
> > -        dst[2*i+1] = (src1[i] + (unsigned)add) >> shift;
> > +        dst[2*i  ] = ((int)(src0[i] + (unsigned)add)) >> shift;
> > +        dst[2*i+1] = ((int)(src1[i] + (unsigned)add)) >> shift;
> >      }
> >  }
> >  
> > 
> 
> This does fix the failure I saw.  

will apply


> I will submit a patch for more fate
> tests which happen to cover this piece of code.

thanks


[...]
diff mbox

Patch

diff --git a/libavcodec/dirac_dwt_template.c b/libavcodec/dirac_dwt_template.c
index 528fc7e9e7..2369c8d15b 100644
--- a/libavcodec/dirac_dwt_template.c
+++ b/libavcodec/dirac_dwt_template.c
@@ -57,8 +57,8 @@  static av_always_inline void RENAME(interleave)(TYPE *dst, TYPE *src0, TYPE *src
 {
     int i;
     for (i = 0; i < w2; i++) {
-        dst[2*i  ] = (src0[i] + (unsigned)add) >> shift;
-        dst[2*i+1] = (src1[i] + (unsigned)add) >> shift;
+        dst[2*i  ] = ((int)(src0[i] + (unsigned)add)) >> shift;
+        dst[2*i+1] = ((int)(src1[i] + (unsigned)add)) >> shift;
     }
 }