| Message ID | 20180312053810.3679-1-Pengfei.Qu@intel.com |
|---|---|
| State | Superseded |
| Headers | show |
On 12/03/18 05:38, Pengfei Qu wrote: > this fix the overflow during the caculation before value assignment. > > Signed-off-by: Pengfei Qu <Pengfei.Qu@intel.com> > --- > libavcodec/vaapi_encode.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c > index 36c85a3..78347d4 100644 > --- a/libavcodec/vaapi_encode.c > +++ b/libavcodec/vaapi_encode.c > @@ -1168,9 +1168,9 @@ static av_cold int vaapi_encode_init_rate_control(AVCodecContext *avctx) > rc_target_percentage = 100; > } else { > rc_bits_per_second = avctx->rc_max_rate; > - rc_target_percentage = (avctx->bit_rate * 100) / rc_bits_per_second; > + rc_target_percentage = (unsigned long)(avctx->bit_rate * 100) / rc_bits_per_second; What is the problem here? All this will do is truncate the 64-bit value on platforms with 32-bit long. > } > - rc_window_size = (hrd_buffer_size * 1000) / avctx->bit_rate; > + rc_window_size = (unsigned long)(hrd_buffer_size * 1000) / avctx->bit_rate; Perhaps you want to ensure that the multiplication doesn't overflow rather than truncating the result after it has overflowed. > } > > ctx->rc_params.misc.type = VAEncMiscParameterTypeRateControl; >
-----Original Message----- From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf Of Mark Thompson Sent: Tuesday, March 13, 2018 6:54 AM To: ffmpeg-devel@ffmpeg.org Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavc/vaapi_encode: fix the caculation overflow On 12/03/18 05:38, Pengfei Qu wrote: > this fix the overflow during the caculation before value assignment. > > Signed-off-by: Pengfei Qu <Pengfei.Qu@intel.com> > --- > libavcodec/vaapi_encode.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c > index 36c85a3..78347d4 100644 > --- a/libavcodec/vaapi_encode.c > +++ b/libavcodec/vaapi_encode.c > @@ -1168,9 +1168,9 @@ static av_cold int vaapi_encode_init_rate_control(AVCodecContext *avctx) > rc_target_percentage = 100; > } else { > rc_bits_per_second = avctx->rc_max_rate; > - rc_target_percentage = (avctx->bit_rate * 100) / rc_bits_per_second; > + rc_target_percentage = (unsigned long)(avctx->bit_rate * > + 100) / rc_bits_per_second; What is the problem here? All this will do is truncate the 64-bit value on platforms with 32-bit long. [Pengfei] same as the below. > } > - rc_window_size = (hrd_buffer_size * 1000) / avctx->bit_rate; > + rc_window_size = (unsigned long)(hrd_buffer_size * 1000) / > + avctx->bit_rate; Perhaps you want to ensure that the multiplication doesn't overflow rather than truncating the result after it has overflowed. [Pengfei] Yes. > } > > ctx->rc_params.misc.type = VAEncMiscParameterTypeRateControl; > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 36c85a3..78347d4 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -1168,9 +1168,9 @@ static av_cold int vaapi_encode_init_rate_control(AVCodecContext *avctx) rc_target_percentage = 100; } else { rc_bits_per_second = avctx->rc_max_rate; - rc_target_percentage = (avctx->bit_rate * 100) / rc_bits_per_second; + rc_target_percentage = (unsigned long)(avctx->bit_rate * 100) / rc_bits_per_second; } - rc_window_size = (hrd_buffer_size * 1000) / avctx->bit_rate; + rc_window_size = (unsigned long)(hrd_buffer_size * 1000) / avctx->bit_rate; } ctx->rc_params.misc.type = VAEncMiscParameterTypeRateControl;
this fix the overflow during the caculation before value assignment. Signed-off-by: Pengfei Qu <Pengfei.Qu@intel.com> --- libavcodec/vaapi_encode.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)