diff mbox series

[FFmpeg-devel,1/6] avcodec/dxtory: Fix get_raw_size() for YUV

Message ID 20200925222638.23675-1-michael@niedermayer.cc
State Accepted
Commit 497debb171b866500ef428479139a112ee7e50d8
Headers show
Series [FFmpeg-devel,1/6] avcodec/dxtory: Fix get_raw_size() for YUV | expand

Checks

Context Check Description
andriy/default pending
andriy/make success Make finished
andriy/make_fate success Make fate finished

Commit Message

Michael Niedermayer Sept. 25, 2020, 10:26 p.m. UTC
Fixes: out of array read
Fixes: 25455/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DXTORY_fuzzer-6327985731534848

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavcodec/dxtory.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Comments

Paul B Mahol Sept. 26, 2020, 7:39 a.m. UTC | #1
On Sat, Sep 26, 2020 at 12:26:33AM +0200, Michael Niedermayer wrote:
> Fixes: out of array read
> Fixes: 25455/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DXTORY_fuzzer-6327985731534848
> 
> Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>  libavcodec/dxtory.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)

lgtm

> 
> diff --git a/libavcodec/dxtory.c b/libavcodec/dxtory.c
> index 3f3c23ff2a..157e4b3ed2 100644
> --- a/libavcodec/dxtory.c
> +++ b/libavcodec/dxtory.c
> @@ -44,9 +44,9 @@ static int64_t get_raw_size(enum AVPixelFormat fmt, int width, int height)
>      case AV_PIX_FMT_YUV444P:
>          return width * height * 3LL;
>      case AV_PIX_FMT_YUV420P:
> -        return (int64_t)(width * height) + AV_CEIL_RSHIFT(width, 1) * AV_CEIL_RSHIFT(height, 1);
> +        return (int64_t)(width * height) + 2 * AV_CEIL_RSHIFT(width, 1) * AV_CEIL_RSHIFT(height, 1);
>      case AV_PIX_FMT_YUV410P:
> -        return (int64_t)(width * height) + AV_CEIL_RSHIFT(width, 2) * AV_CEIL_RSHIFT(height, 2);
> +        return (int64_t)(width * height) + 2 * AV_CEIL_RSHIFT(width, 2) * AV_CEIL_RSHIFT(height, 2);
>      }
>  
>      return 0;
> -- 
> 2.17.1
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Michael Niedermayer Sept. 26, 2020, 5:31 p.m. UTC | #2
On Sat, Sep 26, 2020 at 09:39:45AM +0200, Paul B Mahol wrote:
> On Sat, Sep 26, 2020 at 12:26:33AM +0200, Michael Niedermayer wrote:
> > Fixes: out of array read
> > Fixes: 25455/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DXTORY_fuzzer-6327985731534848
> > 
> > Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> > ---
> >  libavcodec/dxtory.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> lgtm

will apply

thx

[...]
diff mbox series

Patch

diff --git a/libavcodec/dxtory.c b/libavcodec/dxtory.c
index 3f3c23ff2a..157e4b3ed2 100644
--- a/libavcodec/dxtory.c
+++ b/libavcodec/dxtory.c
@@ -44,9 +44,9 @@  static int64_t get_raw_size(enum AVPixelFormat fmt, int width, int height)
     case AV_PIX_FMT_YUV444P:
         return width * height * 3LL;
     case AV_PIX_FMT_YUV420P:
-        return (int64_t)(width * height) + AV_CEIL_RSHIFT(width, 1) * AV_CEIL_RSHIFT(height, 1);
+        return (int64_t)(width * height) + 2 * AV_CEIL_RSHIFT(width, 1) * AV_CEIL_RSHIFT(height, 1);
     case AV_PIX_FMT_YUV410P:
-        return (int64_t)(width * height) + AV_CEIL_RSHIFT(width, 2) * AV_CEIL_RSHIFT(height, 2);
+        return (int64_t)(width * height) + 2 * AV_CEIL_RSHIFT(width, 2) * AV_CEIL_RSHIFT(height, 2);
     }
 
     return 0;