[FFmpeg-devel,1/2] lavc/vaapi_encode: fix the caculation overflow

Submitted by Pengfei Qu on March 12, 2018, 5:38 a.m.

Details

Message ID 20180312053810.3679-1-Pengfei.Qu@intel.com
State New
Headers show

Commit Message

Pengfei Qu March 12, 2018, 5:38 a.m.
this fix the overflow during the caculation before value assignment.

Signed-off-by: Pengfei Qu <Pengfei.Qu@intel.com>
---
 libavcodec/vaapi_encode.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Comments

Mark Thompson March 12, 2018, 10:54 p.m.
On 12/03/18 05:38, Pengfei Qu wrote:
> this fix the overflow during the caculation before value assignment.
> 
> Signed-off-by: Pengfei Qu <Pengfei.Qu@intel.com>
> ---
>  libavcodec/vaapi_encode.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
> index 36c85a3..78347d4 100644
> --- a/libavcodec/vaapi_encode.c
> +++ b/libavcodec/vaapi_encode.c
> @@ -1168,9 +1168,9 @@ static av_cold int vaapi_encode_init_rate_control(AVCodecContext *avctx)
>              rc_target_percentage = 100;
>          } else {
>              rc_bits_per_second   = avctx->rc_max_rate;
> -            rc_target_percentage = (avctx->bit_rate * 100) / rc_bits_per_second;
> +            rc_target_percentage = (unsigned long)(avctx->bit_rate * 100) / rc_bits_per_second;

What is the problem here?  All this will do is truncate the 64-bit value on platforms with 32-bit long.

>          }
> -        rc_window_size = (hrd_buffer_size * 1000) / avctx->bit_rate;
> +        rc_window_size = (unsigned long)(hrd_buffer_size * 1000) / avctx->bit_rate;

Perhaps you want to ensure that the multiplication doesn't overflow rather than truncating the result after it has overflowed.

>      }
>  
>      ctx->rc_params.misc.type = VAEncMiscParameterTypeRateControl;
>
Pengfei Qu March 13, 2018, 1:56 a.m.
-----Original Message-----
From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf Of Mark Thompson

Sent: Tuesday, March 13, 2018 6:54 AM
To: ffmpeg-devel@ffmpeg.org
Subject: Re: [FFmpeg-devel] [PATCH 1/2] lavc/vaapi_encode: fix the caculation overflow

On 12/03/18 05:38, Pengfei Qu wrote:
> this fix the overflow during the caculation before value assignment.

> 

> Signed-off-by: Pengfei Qu <Pengfei.Qu@intel.com>

> ---

>  libavcodec/vaapi_encode.c | 4 ++--

>  1 file changed, 2 insertions(+), 2 deletions(-)

> 

> diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c 

> index 36c85a3..78347d4 100644

> --- a/libavcodec/vaapi_encode.c

> +++ b/libavcodec/vaapi_encode.c

> @@ -1168,9 +1168,9 @@ static av_cold int vaapi_encode_init_rate_control(AVCodecContext *avctx)

>              rc_target_percentage = 100;

>          } else {

>              rc_bits_per_second   = avctx->rc_max_rate;

> -            rc_target_percentage = (avctx->bit_rate * 100) / rc_bits_per_second;

> +            rc_target_percentage = (unsigned long)(avctx->bit_rate * 

> + 100) / rc_bits_per_second;


What is the problem here?  All this will do is truncate the 64-bit value on platforms with 32-bit long.
[Pengfei] same as the below.
>          }

> -        rc_window_size = (hrd_buffer_size * 1000) / avctx->bit_rate;

> +        rc_window_size = (unsigned long)(hrd_buffer_size * 1000) / 

> + avctx->bit_rate;


Perhaps you want to ensure that the multiplication doesn't overflow rather than truncating the result after it has overflowed.
[Pengfei] Yes.
>      }

>  

>      ctx->rc_params.misc.type = VAEncMiscParameterTypeRateControl;

> 

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Patch hide | download patch | download mbox

diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
index 36c85a3..78347d4 100644
--- a/libavcodec/vaapi_encode.c
+++ b/libavcodec/vaapi_encode.c
@@ -1168,9 +1168,9 @@  static av_cold int vaapi_encode_init_rate_control(AVCodecContext *avctx)
             rc_target_percentage = 100;
         } else {
             rc_bits_per_second   = avctx->rc_max_rate;
-            rc_target_percentage = (avctx->bit_rate * 100) / rc_bits_per_second;
+            rc_target_percentage = (unsigned long)(avctx->bit_rate * 100) / rc_bits_per_second;
         }
-        rc_window_size = (hrd_buffer_size * 1000) / avctx->bit_rate;
+        rc_window_size = (unsigned long)(hrd_buffer_size * 1000) / avctx->bit_rate;
     }
 
     ctx->rc_params.misc.type = VAEncMiscParameterTypeRateControl;