From patchwork Mon Mar 26 05:51:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pengfei Qu X-Patchwork-Id: 8174 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.1.70 with SMTP id c67csp3159297jad; Sun, 25 Mar 2018 22:51:31 -0700 (PDT) X-Google-Smtp-Source: AG47ELudaMtsgmLmTyX71laSEvc2COQWmawgbbr7nypsj2Opt/AHk+WxnQNBG1vWU/T+3IxhVJLp X-Received: by 10.28.35.205 with SMTP id j196mr13556474wmj.108.1522043491660; Sun, 25 Mar 2018 22:51:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1522043491; cv=none; d=google.com; s=arc-20160816; b=U2nBu/SoyCPTtB1jd08ePbmDK/d0EO7QNdXnp0WmMrjoRjjpaGOKmBdRWNT5tlBeoW ABUoZCLt5HEfvGzqGaHxORU1St5FXYzylrUXeaNiz31lDHoRow4y7hv2IgtQY3KCsyMj zXcH7Sxpk6WDV5++ZdLbboxQh2S+W6cR9YiR7cL1+YqaB/9HLgeGcfI+VnyanUCbi+PH jk9pi+aJ1bp4O9XQ6M9LNL/o7rZa6ykJkrOYiEocce7+yyJByU2ryR6X+5b47jWO4XXT xK11DW8FEK3ZQ092DSYhKcGWM0QULPXwnr1iw4p1/dh1VjPg8PYkDsiD5Do78s7k42cZ SFyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:delivered-to :arc-authentication-results; bh=ZZWhMT29ezWeSY19laDNaFF9qPUgqoyezhKtzoZMFy0=; b=Tvus5be9EN8+ikgGuvV7eE7YFgHuIvpOcsbmoeGMXJiNxrVeHJVqmvjF4iwPPXXaTf lNs1PucqAWgFJOHRGhF01yagGR5n1UlL3PxpeIACqPlLtOBYB2wtrpS+ikaWaNH8gutK VhC4SFZEFfHFUGDLrgNksSG7QxgjTAWmm8AZaXObSlh/R4U/6W8l5P0AOcYYnz+yXoff RX/qFz3xt9uIavNUTDI9DMRrt5/YDleYG47tFrGIYow0utY+USLNsbRvpbk1xoLrh3eP 98hUn+JYl+jl0QWOsN3i93BOdMydyIe+G66b6RDuzv/KDl3fPR0Ony2yWFh4EybqRHrf P0Aw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x18si10347751wrd.85.2018.03.25.22.51.30; Sun, 25 Mar 2018 22:51:31 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B0246680504; Mon, 26 Mar 2018 08:51:11 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7E08F680570 for ; Mon, 26 Mar 2018 08:51:04 +0300 (EEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 25 Mar 2018 22:51:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.48,363,1517904000"; d="scan'208";a="185976482" Received: from skl-bj-01.bj.intel.com ([10.238.144.117]) by orsmga004.jf.intel.com with ESMTP; 25 Mar 2018 22:51:17 -0700 From: Pengfei Qu To: ffmpeg-devel@ffmpeg.org Date: Mon, 26 Mar 2018 13:51:17 +0800 Message-Id: <20180326055118.32367-1-Pengfei.Qu@intel.com> X-Mailer: git-send-email 2.9.3 Subject: [FFmpeg-devel] [V2 1/2] lavc/vaapi_encode: fix the caculation overflow X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Pengfei Qu MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" this fix the overflow during the caculation before value assignment. Signed-off-by: Pengfei Qu --- libavcodec/vaapi_encode.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 36c85a3..78347d4 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -1168,9 +1168,9 @@ static av_cold int vaapi_encode_init_rate_control(AVCodecContext *avctx) rc_target_percentage = 100; } else { rc_bits_per_second = avctx->rc_max_rate; - rc_target_percentage = (avctx->bit_rate * 100) / rc_bits_per_second; + rc_target_percentage = (unsigned long)(avctx->bit_rate * 100) / rc_bits_per_second; } - rc_window_size = (hrd_buffer_size * 1000) / avctx->bit_rate; + rc_window_size = (unsigned long)(hrd_buffer_size * 1000) / avctx->bit_rate; } ctx->rc_params.misc.type = VAEncMiscParameterTypeRateControl;