diff mbox series

[FFmpeg-devel,2/4] avformat/rpl: Use 64bit in bitrate computation and check it

Message ID 20210427192135.17286-2-michael@niedermayer.cc
State New
Headers show
Series [FFmpeg-devel,1/4] avformat/rmdec: Check old_format len for overflow
Related show

Checks

Context Check Description
andriy/x86_make success Make finished
andriy/x86_make_fate success Make fate finished
andriy/PPC64_make success Make finished
andriy/PPC64_make_fate success Make fate finished

Commit Message

Michael Niedermayer April 27, 2021, 7:21 p.m. UTC
Fixes: signed integer overflow: 777777776 * 4 cannot be represented in type 'int'
Fixes: 29102/clusterfuzz-testcase-minimized-ffmpeg_dem_RPL_fuzzer-6726188921913344

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavformat/rpl.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

Comments

Andreas Rheinhardt April 27, 2021, 7:31 p.m. UTC | #1
Michael Niedermayer:
> Fixes: signed integer overflow: 777777776 * 4 cannot be represented in type 'int'
> Fixes: 29102/clusterfuzz-testcase-minimized-ffmpeg_dem_RPL_fuzzer-6726188921913344
> 
> Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>  libavformat/rpl.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/libavformat/rpl.c b/libavformat/rpl.c
> index b98488c7b1..558e5718ba 100644
> --- a/libavformat/rpl.c
> +++ b/libavformat/rpl.c
> @@ -207,8 +207,10 @@ static int rpl_read_header(AVFormatContext *s)
>              ast->codecpar->bits_per_coded_sample = 4;
>  
>          ast->codecpar->bit_rate = ast->codecpar->sample_rate *
> -                                  ast->codecpar->bits_per_coded_sample *
> -                                  ast->codecpar->channels;
> +                                  (int64_t)ast->codecpar->bits_per_coded_sample;
> +        if (ast->codecpar->bit_rate > INT64_MAX / ast->codecpar->channels)
> +            return AVERROR_INVALIDDATA;
> +        ast->codecpar->bit_rate *= ast->codecpar->channels;
>  
>          ast->codecpar->codec_id = AV_CODEC_ID_NONE;
>          switch (audio_format) {
> 
read_line_and_int() can return zero and so you must not divide by
channels. (Something like __builtin_smulll_overflow() might be
advantageous here.)

- Andreas
Michael Niedermayer April 28, 2021, 8:22 a.m. UTC | #2
On Tue, Apr 27, 2021 at 09:31:56PM +0200, Andreas Rheinhardt wrote:
> Michael Niedermayer:
> > Fixes: signed integer overflow: 777777776 * 4 cannot be represented in type 'int'
> > Fixes: 29102/clusterfuzz-testcase-minimized-ffmpeg_dem_RPL_fuzzer-6726188921913344
> > 
> > Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> > ---
> >  libavformat/rpl.c | 6 ++++--
> >  1 file changed, 4 insertions(+), 2 deletions(-)
> > 
> > diff --git a/libavformat/rpl.c b/libavformat/rpl.c
> > index b98488c7b1..558e5718ba 100644
> > --- a/libavformat/rpl.c
> > +++ b/libavformat/rpl.c
> > @@ -207,8 +207,10 @@ static int rpl_read_header(AVFormatContext *s)
> >              ast->codecpar->bits_per_coded_sample = 4;
> >  
> >          ast->codecpar->bit_rate = ast->codecpar->sample_rate *
> > -                                  ast->codecpar->bits_per_coded_sample *
> > -                                  ast->codecpar->channels;
> > +                                  (int64_t)ast->codecpar->bits_per_coded_sample;
> > +        if (ast->codecpar->bit_rate > INT64_MAX / ast->codecpar->channels)
> > +            return AVERROR_INVALIDDATA;
> > +        ast->codecpar->bit_rate *= ast->codecpar->channels;
> >  
> >          ast->codecpar->codec_id = AV_CODEC_ID_NONE;
> >          switch (audio_format) {
> > 
> read_line_and_int() can return zero and so you must not divide by
> channels. (Something like __builtin_smulll_overflow() might be
> advantageous here.)

hmm, i saw bits_per_coded_sample cant be 0 and wanted to use that
but clearly i didnt, fascinating.
will switch channels out with bits_per_coded_sample

thx

[...]
diff mbox series

Patch

diff --git a/libavformat/rpl.c b/libavformat/rpl.c
index b98488c7b1..558e5718ba 100644
--- a/libavformat/rpl.c
+++ b/libavformat/rpl.c
@@ -207,8 +207,10 @@  static int rpl_read_header(AVFormatContext *s)
             ast->codecpar->bits_per_coded_sample = 4;
 
         ast->codecpar->bit_rate = ast->codecpar->sample_rate *
-                                  ast->codecpar->bits_per_coded_sample *
-                                  ast->codecpar->channels;
+                                  (int64_t)ast->codecpar->bits_per_coded_sample;
+        if (ast->codecpar->bit_rate > INT64_MAX / ast->codecpar->channels)
+            return AVERROR_INVALIDDATA;
+        ast->codecpar->bit_rate *= ast->codecpar->channels;
 
         ast->codecpar->codec_id = AV_CODEC_ID_NONE;
         switch (audio_format) {