| Message ID | 20210214193616.111213-1-andreas.rheinhardt@gmail.com |
|---|---|
| State | Accepted |
| Commit | 39df279c741928c6adf223890ff19b457f96b9bf |
| Headers | show |
| Series | [FFmpeg-devel,1/2] avutil/video_enc_params: Combine overflow checks | expand |
| Context | Check | Description |
|---|---|---|
| andriy/x86_make | success | Make finished |
| andriy/x86_make_fate | success | Make fate finished |
| andriy/PPC64_make | success | Make finished |
| andriy/PPC64_make_fate | success | Make fate finished |
Andreas Rheinhardt: > This patch also fixes a -Wtautological-constant-out-of-range-compare > warning from Clang and a -Wtype-limits warning from GCC on systems > where size_t is 64bits and unsigned 32bits. The reason for this seems > to be that variable (whose value derives from sizeof() and can therefore > be known at compile-time) is used instead of using sizeof() directly in > the comparison. > > Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com> > --- > Using (UINT_MAX > SIZE_MAX / sizeof(AVVideoBlockParams) && nb_blocks > > SIZE_MAX / sizeof(AVVideoBlockParams)) would have fixed the warning with > Clang, but not GCC. > > libavutil/video_enc_params.c | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/libavutil/video_enc_params.c b/libavutil/video_enc_params.c > index c46c0f1dc6..b9cdafddbb 100644 > --- a/libavutil/video_enc_params.c > +++ b/libavutil/video_enc_params.c > @@ -33,8 +33,7 @@ AVVideoEncParams *av_video_enc_params_alloc(enum AVVideoEncParamsType type, > size_t size; > > size = sizeof(*par); > - if (nb_blocks > SIZE_MAX / sizeof(AVVideoBlockParams) || > - nb_blocks * sizeof(AVVideoBlockParams) > SIZE_MAX - size) > + if (nb_blocks > (SIZE_MAX - size) / sizeof(AVVideoBlockParams)) > return NULL; > size += sizeof(AVVideoBlockParams) * nb_blocks; > > Will apply this patchset tomorrow unless there are objections. - Andreas
diff --git a/libavutil/video_enc_params.c b/libavutil/video_enc_params.c index c46c0f1dc6..b9cdafddbb 100644 --- a/libavutil/video_enc_params.c +++ b/libavutil/video_enc_params.c @@ -33,8 +33,7 @@ AVVideoEncParams *av_video_enc_params_alloc(enum AVVideoEncParamsType type, size_t size; size = sizeof(*par); - if (nb_blocks > SIZE_MAX / sizeof(AVVideoBlockParams) || - nb_blocks * sizeof(AVVideoBlockParams) > SIZE_MAX - size) + if (nb_blocks > (SIZE_MAX - size) / sizeof(AVVideoBlockParams)) return NULL; size += sizeof(AVVideoBlockParams) * nb_blocks;
This patch also fixes a -Wtautological-constant-out-of-range-compare warning from Clang and a -Wtype-limits warning from GCC on systems where size_t is 64bits and unsigned 32bits. The reason for this seems to be that variable (whose value derives from sizeof() and can therefore be known at compile-time) is used instead of using sizeof() directly in the comparison. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com> --- Using (UINT_MAX > SIZE_MAX / sizeof(AVVideoBlockParams) && nb_blocks > SIZE_MAX / sizeof(AVVideoBlockParams)) would have fixed the warning with Clang, but not GCC. libavutil/video_enc_params.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)