diff mbox series

[FFmpeg-devel,6/7] avformat/lxfdec: Fix multiple integer overflows related to track_size

Message ID 20210114225116.13486-6-michael@niedermayer.cc
State Accepted
Commit 7819412f4468514a2bab924291d79806a569388c
Headers show
Series [FFmpeg-devel,1/7] avformat/ads: Check size | expand

Checks

Context Check Description
andriy/x86_make success Make finished
andriy/x86_make_fate success Make fate finished
andriy/PPC64_make success Make finished
andriy/PPC64_make_fate success Make fate finished

Commit Message

Michael Niedermayer Jan. 14, 2021, 10:51 p.m. UTC
Fixes: signed integer overflow: 538976288 * 8 cannot be represented in type 'int'
Fixes: 26910/clusterfuzz-testcase-minimized-ffmpeg_dem_LXF_fuzzer-6634030636335104

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
---
 libavformat/lxfdec.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

Comments

Tomas Härdin Jan. 15, 2021, 8:48 a.m. UTC | #1
tor 2021-01-14 klockan 23:51 +0100 skrev Michael Niedermayer:
> Fixes: signed integer overflow: 538976288 * 8 cannot be represented in type 'int'
> Fixes: 26910/clusterfuzz-testcase-minimized-ffmpeg_dem_LXF_fuzzer-6634030636335104
> 
> Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> ---
>  libavformat/lxfdec.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/libavformat/lxfdec.c b/libavformat/lxfdec.c
> index fa84ceea78..509d19fe7f 100644
> --- a/libavformat/lxfdec.c
> +++ b/libavformat/lxfdec.c
> @@ -195,7 +195,7 @@ static int get_packet_header(AVFormatContext *s)
>              return AVERROR_PATCHWELCOME;
>          }
>  
> -        samples = track_size * 8 / st->codecpar->bits_per_coded_sample;
> +        samples = track_size * 8LL / st->codecpar->bits_per_coded_sample;
>  
>          //use audio packet size to determine video standard
>          //for NTSC we have one 8008-sample audio frame per five video frames
> @@ -210,6 +210,8 @@ static int get_packet_header(AVFormatContext *s)
>              avpriv_set_pts_info(s->streams[0], 64, 1, 25);
>          }
>  
> +        if (av_popcount(channels) * (uint64_t)track_size > INT_MAX)
> +            return AVERROR_INVALIDDATA;
>          //TODO: warning if track mask != (1 << channels) - 1?
>          ret = av_popcount(channels) * track_size;

What happens if channels == 0 ? Probably gets caught in
av_new_packet(), just making sure

/Tomas
Michael Niedermayer Jan. 15, 2021, 11:22 p.m. UTC | #2
On Fri, Jan 15, 2021 at 09:48:38AM +0100, Tomas Härdin wrote:
> tor 2021-01-14 klockan 23:51 +0100 skrev Michael Niedermayer:
> > Fixes: signed integer overflow: 538976288 * 8 cannot be represented in type 'int'
> > Fixes: 26910/clusterfuzz-testcase-minimized-ffmpeg_dem_LXF_fuzzer-6634030636335104
> > 
> > Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> > ---
> >  libavformat/lxfdec.c | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> > 
> > diff --git a/libavformat/lxfdec.c b/libavformat/lxfdec.c
> > index fa84ceea78..509d19fe7f 100644
> > --- a/libavformat/lxfdec.c
> > +++ b/libavformat/lxfdec.c
> > @@ -195,7 +195,7 @@ static int get_packet_header(AVFormatContext *s)
> >              return AVERROR_PATCHWELCOME;
> >          }
> >  
> > -        samples = track_size * 8 / st->codecpar->bits_per_coded_sample;
> > +        samples = track_size * 8LL / st->codecpar->bits_per_coded_sample;
> >  
> >          //use audio packet size to determine video standard
> >          //for NTSC we have one 8008-sample audio frame per five video frames
> > @@ -210,6 +210,8 @@ static int get_packet_header(AVFormatContext *s)
> >              avpriv_set_pts_info(s->streams[0], 64, 1, 25);
> >          }
> >  
> > +        if (av_popcount(channels) * (uint64_t)track_size > INT_MAX)
> > +            return AVERROR_INVALIDDATA;
> >          //TODO: warning if track mask != (1 << channels) - 1?
> >          ret = av_popcount(channels) * track_size;
> 
> What happens if channels == 0 ? Probably gets caught in
> av_new_packet(), just making sure

Probably would produce a 0 sized packet.
That could be checked for too but seems unrelated to the integer overflow
are there any other constraints i should check for ?
The code suggests  mask != (1 << channels) - 1
what level of warning vs. error do you prefer for the different odd
possibilities?

thx


[...]
Tomas Härdin Jan. 20, 2021, 3:54 p.m. UTC | #3
lör 2021-01-16 klockan 00:22 +0100 skrev Michael Niedermayer:
> On Fri, Jan 15, 2021 at 09:48:38AM +0100, Tomas Härdin wrote:
> > tor 2021-01-14 klockan 23:51 +0100 skrev Michael Niedermayer:
> > > Fixes: signed integer overflow: 538976288 * 8 cannot be represented in type 'int'
> > > Fixes: 26910/clusterfuzz-testcase-minimized-ffmpeg_dem_LXF_fuzzer-6634030636335104
> > > 
> > > Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> > > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> > > ---
> > >  libavformat/lxfdec.c | 4 +++-
> > >  1 file changed, 3 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/libavformat/lxfdec.c b/libavformat/lxfdec.c
> > > index fa84ceea78..509d19fe7f 100644
> > > --- a/libavformat/lxfdec.c
> > > +++ b/libavformat/lxfdec.c
> > > @@ -195,7 +195,7 @@ static int get_packet_header(AVFormatContext *s)
> > >              return AVERROR_PATCHWELCOME;
> > >          }
> > >  
> > > -        samples = track_size * 8 / st->codecpar->bits_per_coded_sample;
> > > +        samples = track_size * 8LL / st->codecpar->bits_per_coded_sample;
> > >  
> > >          //use audio packet size to determine video standard
> > >          //for NTSC we have one 8008-sample audio frame per five video frames
> > > @@ -210,6 +210,8 @@ static int get_packet_header(AVFormatContext *s)
> > >              avpriv_set_pts_info(s->streams[0], 64, 1, 25);
> > >          }
> > >  
> > > +        if (av_popcount(channels) * (uint64_t)track_size > INT_MAX)
> > > +            return AVERROR_INVALIDDATA;
> > >          //TODO: warning if track mask != (1 << channels) - 1?
> > >          ret = av_popcount(channels) * track_size;
> > 
> > What happens if channels == 0 ? Probably gets caught in
> > av_new_packet(), just making sure
> 
> Probably would produce a 0 sized packet.
> That could be checked for too but seems unrelated to the integer overflow
> are there any other constraints i should check for ?
> The code suggests  mask != (1 << channels) - 1
> what level of warning vs. error do you prefer for the different odd
> possibilities?

I haven't seen LXF files in quite a while, so I can't really say
whether we should warn or not. Probably best not to increase nag in
this case.

/Tomas
Michael Niedermayer March 3, 2021, 3:44 p.m. UTC | #4
On Wed, Jan 20, 2021 at 04:54:56PM +0100, Tomas Härdin wrote:
> lör 2021-01-16 klockan 00:22 +0100 skrev Michael Niedermayer:
> > On Fri, Jan 15, 2021 at 09:48:38AM +0100, Tomas Härdin wrote:
> > > tor 2021-01-14 klockan 23:51 +0100 skrev Michael Niedermayer:
> > > > Fixes: signed integer overflow: 538976288 * 8 cannot be represented in type 'int'
> > > > Fixes: 26910/clusterfuzz-testcase-minimized-ffmpeg_dem_LXF_fuzzer-6634030636335104
> > > > 
> > > > Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> > > > Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
> > > > ---
> > > >  libavformat/lxfdec.c | 4 +++-
> > > >  1 file changed, 3 insertions(+), 1 deletion(-)
> > > > 
> > > > diff --git a/libavformat/lxfdec.c b/libavformat/lxfdec.c
> > > > index fa84ceea78..509d19fe7f 100644
> > > > --- a/libavformat/lxfdec.c
> > > > +++ b/libavformat/lxfdec.c
> > > > @@ -195,7 +195,7 @@ static int get_packet_header(AVFormatContext *s)
> > > >              return AVERROR_PATCHWELCOME;
> > > >          }
> > > >  
> > > > -        samples = track_size * 8 / st->codecpar->bits_per_coded_sample;
> > > > +        samples = track_size * 8LL / st->codecpar->bits_per_coded_sample;
> > > >  
> > > >          //use audio packet size to determine video standard
> > > >          //for NTSC we have one 8008-sample audio frame per five video frames
> > > > @@ -210,6 +210,8 @@ static int get_packet_header(AVFormatContext *s)
> > > >              avpriv_set_pts_info(s->streams[0], 64, 1, 25);
> > > >          }
> > > >  
> > > > +        if (av_popcount(channels) * (uint64_t)track_size > INT_MAX)
> > > > +            return AVERROR_INVALIDDATA;
> > > >          //TODO: warning if track mask != (1 << channels) - 1?
> > > >          ret = av_popcount(channels) * track_size;
> > > 
> > > What happens if channels == 0 ? Probably gets caught in
> > > av_new_packet(), just making sure
> > 
> > Probably would produce a 0 sized packet.
> > That could be checked for too but seems unrelated to the integer overflow
> > are there any other constraints i should check for ?
> > The code suggests  mask != (1 << channels) - 1
> > what level of warning vs. error do you prefer for the different odd
> > possibilities?
> 
> I haven't seen LXF files in quite a while, so I can't really say
> whether we should warn or not. Probably best not to increase nag in
> this case.

ok then i guess i should just apply this, will do so

thanks

[...]
diff mbox series

Patch

diff --git a/libavformat/lxfdec.c b/libavformat/lxfdec.c
index fa84ceea78..509d19fe7f 100644
--- a/libavformat/lxfdec.c
+++ b/libavformat/lxfdec.c
@@ -195,7 +195,7 @@  static int get_packet_header(AVFormatContext *s)
             return AVERROR_PATCHWELCOME;
         }
 
-        samples = track_size * 8 / st->codecpar->bits_per_coded_sample;
+        samples = track_size * 8LL / st->codecpar->bits_per_coded_sample;
 
         //use audio packet size to determine video standard
         //for NTSC we have one 8008-sample audio frame per five video frames
@@ -210,6 +210,8 @@  static int get_packet_header(AVFormatContext *s)
             avpriv_set_pts_info(s->streams[0], 64, 1, 25);
         }
 
+        if (av_popcount(channels) * (uint64_t)track_size > INT_MAX)
+            return AVERROR_INVALIDDATA;
         //TODO: warning if track mask != (1 << channels) - 1?
         ret = av_popcount(channels) * track_size;