diff mbox series

[FFmpeg-devel] avcodec/motion_est: Fix xy indexing on range violation in ff_get_best_fcode()

Message ID 20220208205032.2013-1-michael@niedermayer.cc
State New
Headers show
Series [FFmpeg-devel] avcodec/motion_est: Fix xy indexing on range violation in ff_get_best_fcode() | expand

Checks

Context Check Description
andriy/make_x86 success Make finished
andriy/make_fate_x86 success Make fate finished
andriy/make_ppc success Make finished
andriy/make_fate_ppc success Make fate finished

Commit Message

Michael Niedermayer Feb. 8, 2022, 8:50 p.m. UTC
This codepath seems untested, no testcases change

Found-by: <mkver>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavcodec/motion_est.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

Comments

Andreas Rheinhardt Feb. 9, 2022, 2:39 a.m. UTC | #1
Michael Niedermayer:
> This codepath seems untested, no testcases change
> 
> Found-by: <mkver>
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>  libavcodec/motion_est.c | 8 +++++---
>  1 file changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/libavcodec/motion_est.c b/libavcodec/motion_est.c
> index 9c548c1567..4cf1afe888 100644
> --- a/libavcodec/motion_est.c
> +++ b/libavcodec/motion_est.c
> @@ -1621,9 +1621,11 @@ int ff_get_best_fcode(MpegEncContext * s, int16_t (*mv_table)[2], int type)
>                                       fcode_tab[my + MAX_MV]);
>                      int j;
>  
> -                        if(mx >= range || mx < -range ||
> -                           my >= range || my < -range)
> -                            continue;
> +                    if (mx >= range || mx < -range ||
> +                        my >= range || my < -range) {
> +                        xy++;
> +                        continue;
> +                    }
>  
>                      for(j=0; j<fcode && j<8; j++){
>                          if(s->pict_type==AV_PICTURE_TYPE_B || s->current_picture.mc_mb_var[xy] < s->current_picture.mb_var[xy])

It would be simpler if you added the xy++ to the outer-for-loop and
removed the other xy++.

- Andreas
Michael Niedermayer Feb. 9, 2022, 9:36 a.m. UTC | #2
On Wed, Feb 09, 2022 at 03:39:20AM +0100, Andreas Rheinhardt wrote:
> Michael Niedermayer:
> > This codepath seems untested, no testcases change
> > 
> > Found-by: <mkver>
> > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> > ---
> >  libavcodec/motion_est.c | 8 +++++---
> >  1 file changed, 5 insertions(+), 3 deletions(-)
> > 
> > diff --git a/libavcodec/motion_est.c b/libavcodec/motion_est.c
> > index 9c548c1567..4cf1afe888 100644
> > --- a/libavcodec/motion_est.c
> > +++ b/libavcodec/motion_est.c
> > @@ -1621,9 +1621,11 @@ int ff_get_best_fcode(MpegEncContext * s, int16_t (*mv_table)[2], int type)
> >                                       fcode_tab[my + MAX_MV]);
> >                      int j;
> >  
> > -                        if(mx >= range || mx < -range ||
> > -                           my >= range || my < -range)
> > -                            continue;
> > +                    if (mx >= range || mx < -range ||
> > +                        my >= range || my < -range) {
> > +                        xy++;
> > +                        continue;
> > +                    }
> >  
> >                      for(j=0; j<fcode && j<8; j++){
> >                          if(s->pict_type==AV_PICTURE_TYPE_B || s->current_picture.mc_mb_var[xy] < s->current_picture.mb_var[xy])
> 
> It would be simpler if you added the xy++ to the outer-for-loop and
> removed the other xy++.

that leaves the indention wrong. Will move the xy in the for loop and
fix the indention in a separate commit

thx

[...]
diff mbox series

Patch

diff --git a/libavcodec/motion_est.c b/libavcodec/motion_est.c
index 9c548c1567..4cf1afe888 100644
--- a/libavcodec/motion_est.c
+++ b/libavcodec/motion_est.c
@@ -1621,9 +1621,11 @@  int ff_get_best_fcode(MpegEncContext * s, int16_t (*mv_table)[2], int type)
                                      fcode_tab[my + MAX_MV]);
                     int j;
 
-                        if(mx >= range || mx < -range ||
-                           my >= range || my < -range)
-                            continue;
+                    if (mx >= range || mx < -range ||
+                        my >= range || my < -range) {
+                        xy++;
+                        continue;
+                    }
 
                     for(j=0; j<fcode && j<8; j++){
                         if(s->pict_type==AV_PICTURE_TYPE_B || s->current_picture.mc_mb_var[xy] < s->current_picture.mb_var[xy])