[FFmpeg-devel] [PATCH] avcodec/motion_est: Fix xy indexing on range violation in ff_get_best_fcode()

[FFmpeg-devel] [PATCH] avcodec/motion_est: Fix xy indexing on range violation in ff_get_best_fcode()

[Michael Niedermayer]

This codepath seems untested, no testcases change

Found-by: <mkver>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavcodec/motion_est.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/libavcodec/motion_est.c b/libavcodec/motion_est.c
index 9c548c1567..4cf1afe888 100644
--- a/libavcodec/motion_est.c
+++ b/libavcodec/motion_est.c
@@ -1621,9 +1621,11 @@ int ff_get_best_fcode(MpegEncContext * s, int16_t (*mv_table)[2], int type)
                                      fcode_tab[my + MAX_MV]);
                     int j;
 
-                        if(mx >= range || mx < -range ||
-                           my >= range || my < -range)
-                            continue;
+                    if (mx >= range || mx < -range ||
+                        my >= range || my < -range) {
+                        xy++;
+                        continue;
+                    }
 
                     for(j=0; j<fcode && j<8; j++){
                         if(s->pict_type==AV_PICTURE_TYPE_B || s->current_picture.mc_mb_var[xy] < s->current_picture.mb_var[xy])
-- 
2.17.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/motion_est: Fix xy indexing on range violation in ff_get_best_fcode()

[Andreas Rheinhardt]

Michael Niedermayer:
> This codepath seems untested, no testcases change
> 
> Found-by: <mkver>
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>  libavcodec/motion_est.c | 8 +++++---
>  1 file changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/libavcodec/motion_est.c b/libavcodec/motion_est.c
> index 9c548c1567..4cf1afe888 100644
> --- a/libavcodec/motion_est.c
> +++ b/libavcodec/motion_est.c
> @@ -1621,9 +1621,11 @@ int ff_get_best_fcode(MpegEncContext * s, int16_t (*mv_table)[2], int type)
>                                       fcode_tab[my + MAX_MV]);
>                      int j;
>  
> -                        if(mx >= range || mx < -range ||
> -                           my >= range || my < -range)
> -                            continue;
> +                    if (mx >= range || mx < -range ||
> +                        my >= range || my < -range) {
> +                        xy++;
> +                        continue;
> +                    }
>  
>                      for(j=0; j<fcode && j<8; j++){
>                          if(s->pict_type==AV_PICTURE_TYPE_B || s->current_picture.mc_mb_var[xy] < s->current_picture.mb_var[xy])

It would be simpler if you added the xy++ to the outer-for-loop and
removed the other xy++.

- Andreas
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/motion_est: Fix xy indexing on range violation in ff_get_best_fcode()

[Michael Niedermayer]

[-- Attachment #1.1: Type: text/plain, Size: 1784 bytes --]

On Wed, Feb 09, 2022 at 03:39:20AM +0100, Andreas Rheinhardt wrote:
> Michael Niedermayer:
> > This codepath seems untested, no testcases change
> > 
> > Found-by: <mkver>
> > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> > ---
> >  libavcodec/motion_est.c | 8 +++++---
> >  1 file changed, 5 insertions(+), 3 deletions(-)
> > 
> > diff --git a/libavcodec/motion_est.c b/libavcodec/motion_est.c
> > index 9c548c1567..4cf1afe888 100644
> > --- a/libavcodec/motion_est.c
> > +++ b/libavcodec/motion_est.c
> > @@ -1621,9 +1621,11 @@ int ff_get_best_fcode(MpegEncContext * s, int16_t (*mv_table)[2], int type)
> >                                       fcode_tab[my + MAX_MV]);
> >                      int j;
> >  
> > -                        if(mx >= range || mx < -range ||
> > -                           my >= range || my < -range)
> > -                            continue;
> > +                    if (mx >= range || mx < -range ||
> > +                        my >= range || my < -range) {
> > +                        xy++;
> > +                        continue;
> > +                    }
> >  
> >                      for(j=0; j<fcode && j<8; j++){
> >                          if(s->pict_type==AV_PICTURE_TYPE_B || s->current_picture.mc_mb_var[xy] < s->current_picture.mb_var[xy])
> 
> It would be simpler if you added the xy++ to the outer-for-loop and
> removed the other xy++.

that leaves the indention wrong. Will move the xy in the for loop and
fix the indention in a separate commit

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

When you are offended at any man's fault, turn to yourself and study your
own failings. Then you will forget your anger. -- Epictetus

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".