diff mbox series

[FFmpeg-devel,2/4] avcodec/apedec: properly calculate and store absolute value

Message ID 20201006001714.19577-2-onemda@gmail.com
State Accepted
Commit ea0972f6dd3ad51451b8acb9797364eddfecada5
Headers show
Series [FFmpeg-devel,1/4] avcodec/apedec: fix decoding insane files with recent versions
Related show

Checks

Context Check Description
andriy/default pending
andriy/make success Make finished
andriy/make_fate success Make fate finished

Commit Message

Paul B Mahol Oct. 6, 2020, 12:17 a.m. UTC
Signed-off-by: Paul B Mahol <onemda@gmail.com>
---
 libavcodec/apedec.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Comments

Anton Khirnov Oct. 6, 2020, 7:53 a.m. UTC | #1
Quoting Paul B Mahol (2020-10-06 02:17:12)
> Signed-off-by: Paul B Mahol <onemda@gmail.com>
> ---
>  libavcodec/apedec.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c
> index 273abe2490..aa4d8fa524 100644
> --- a/libavcodec/apedec.c
> +++ b/libavcodec/apedec.c
> @@ -1311,7 +1311,7 @@ static void do_apply_filter(APEContext *ctx, int version, APEFilter *f,
>                              int32_t *data, int count, int order, int fracbits)
>  {
>      int res;
> -    int absres;
> +    unsigned absres;

Does anything other than the type change?
Paul B Mahol Oct. 6, 2020, 8:23 a.m. UTC | #2
On Tue, Oct 06, 2020 at 09:53:44AM +0200, Anton Khirnov wrote:
> Quoting Paul B Mahol (2020-10-06 02:17:12)
> > Signed-off-by: Paul B Mahol <onemda@gmail.com>
> > ---
> >  libavcodec/apedec.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c
> > index 273abe2490..aa4d8fa524 100644
> > --- a/libavcodec/apedec.c
> > +++ b/libavcodec/apedec.c
> > @@ -1311,7 +1311,7 @@ static void do_apply_filter(APEContext *ctx, int version, APEFilter *f,
> >                              int32_t *data, int count, int order, int fracbits)
> >  {
> >      int res;
> > -    int absres;
> > +    unsigned absres;
> 
> Does anything other than the type change?

absres value should change in single case when -INT32_MIN is stored again in int.
Reference implementation use int64_t type instead.
Anton Khirnov Oct. 6, 2020, 9:57 a.m. UTC | #3
Quoting Paul B Mahol (2020-10-06 10:23:13)
> On Tue, Oct 06, 2020 at 09:53:44AM +0200, Anton Khirnov wrote:
> > Quoting Paul B Mahol (2020-10-06 02:17:12)
> > > Signed-off-by: Paul B Mahol <onemda@gmail.com>
> > > ---
> > >  libavcodec/apedec.c | 4 ++--
> > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c
> > > index 273abe2490..aa4d8fa524 100644
> > > --- a/libavcodec/apedec.c
> > > +++ b/libavcodec/apedec.c
> > > @@ -1311,7 +1311,7 @@ static void do_apply_filter(APEContext *ctx, int version, APEFilter *f,
> > >                              int32_t *data, int count, int order, int fracbits)
> > >  {
> > >      int res;
> > > -    int absres;
> > > +    unsigned absres;
> > 
> > Does anything other than the type change?
> 
> absres value should change in single case when -INT32_MIN is stored again in int.
> Reference implementation use int64_t type instead.

ok then
diff mbox series

Patch

diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c
index 273abe2490..aa4d8fa524 100644
--- a/libavcodec/apedec.c
+++ b/libavcodec/apedec.c
@@ -1311,7 +1311,7 @@  static void do_apply_filter(APEContext *ctx, int version, APEFilter *f,
                             int32_t *data, int count, int order, int fracbits)
 {
     int res;
-    int absres;
+    unsigned absres;
 
     while (count--) {
         /* round fixedpoint scalar product */
@@ -1335,7 +1335,7 @@  static void do_apply_filter(APEContext *ctx, int version, APEFilter *f,
             /* Version 3.98 and later files */
 
             /* Update the adaption coefficients */
-            absres = res < 0 ? -(unsigned)res : res;
+            absres = FFABS(res);
             if (absres)
                 *f->adaptcoeffs = APESIGN(res) *
                                   (8 << ((absres > f->avg * 3) + (absres > f->avg * 4 / 3)));