From patchwork Sun May 12 00:03:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 48798 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:1706:b0:1af:cdee:28c5 with SMTP id nv6csp348101pzb; Sat, 11 May 2024 17:04:31 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCU1tAxkAzWKmADUPtEjAE4W/GvQ8lqEMpSjMASHIF188ukfbmNVq9CPWMiYieZbaSo31DlgYYq2BJLHLa7vCaWzz5t0bnjkukuh3Q== X-Google-Smtp-Source: AGHT+IHCo4tFvm9UlbYT5VdKFwiLPJQjeU6wDtqfHxMoGbtMop4PZLWGhORyMQ0tMagWAolHf40/ X-Received: by 2002:a17:906:a0c7:b0:a5a:580f:8e74 with SMTP id a640c23a62f3a-a5a580f9031mr65851766b.5.1715472270944; Sat, 11 May 2024 17:04:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1715472270; cv=none; d=google.com; s=arc-20160816; b=z5bRmxFglL7cu4wAwH9KeYlfhgEv9eUVXjmiXUdB0gvgKNEiZYIIyZC7siMQC3eccd EiFeR8H7I6ecAuylc0VQbRkanYDt9OagQa+AQiuJceUOfvbNRA/fIaQf9fllzVVALwk5 fhpAOXqsvKlUcopqixLOz9ieypeir6JlyzYxLMRsvk1sNH5F/jd5uSqZMOQ6CJDuNWZF gctgq3Zu0T4EuEtH8fIVkJob9E1nuEvAavyFb1hlWbmtFxBCXwE2EETZLCOnGpLgLBZU WcvQ4c5Q5eloZQHlZfLNCRc5pc13pXD9Rl2yKQndjgJp1AFM1sMQtQdo0d58x/aWm9IK oWkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=MWyQJXDox4R2jFGi1p1R0UNaYxxHQWChrppUZ5IsJHA=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=p62aZ/YZV6TEQE7MeYVojzIAgN+tOJ3PHuielyx6ljufC9OQyP/0tNk3jiraSa+Kww 5TeHOHZipaK0esi2Vv7ppX1SUXpWs/R0xIVQz9Xa7vHDA6060e1MlIyjmRcjUPtcKY+V 1E0/nMybKrFuc2YRg3aXE+6uFEu3Af4yAXT79hvNNjPrA/i/lMsdzTMrfxlAaBzsX/Na UuBljdagXqlFIRaEnacs6XieiaX3v3yikrasDAu4OvI7wcQ0cLz2pe6r3hPR4RjPCzD+ ltl8G16Y9NorqqMo0Xm0K0x8BJ+e1Y6n7d/blZpz61D+VUtuKHWwsCAKwlsOglbSCjNV zK2Q==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=lwa6Y33+; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a5a17c2c728si328710566b.970.2024.05.11.17.04.30; Sat, 11 May 2024 17:04:30 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=lwa6Y33+; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 247BB68D692; Sun, 12 May 2024 03:04:02 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay7-d.mail.gandi.net (relay7-d.mail.gandi.net [217.70.183.200]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 15E6768D3DF for ; Sun, 12 May 2024 03:03:54 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id 70F8B20002 for ; Sun, 12 May 2024 00:03:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niedermayer.cc; s=gm1; t=1715472233; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qRMaV1KD5O3d4jiclNc6sn95WaPU2TQiOK3XK/1cQow=; b=lwa6Y33+DbsJouIYTshyH9l1aQeJ4TPoyJS/f+Q4fyXwLM9a/1nKoeUFnvt0cKiCUcYKoJ Xtk1fg2gSKavJBLl6Mz5UJLht0DKd4gVBs4ShXH0CJyq5gXiJD+ZwgXlTXhcssK4xveZT9 7bLCdZI4rrF9VpJ+tWy9eocRCBvEnejVAvMOXnEBQB91Cc62WGBp/jqx3BQXJ+HjzzuT8g 8Tc8lUrZkQTq5Cjys6prkgefpbAayEOZtI/IwiOlLjOAFUj7ToMLVSZs1k40MzoZ6Jqom7 SeHfPV4j77Gij3D2NJIUBvdzmMrFB+hxpTlrovwJEm97HqxUD4Q3q6TllQ34uw== From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Sun, 12 May 2024 02:03:48 +0200 Message-ID: <20240512000349.3381912-4-michael@niedermayer.cc> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240512000349.3381912-1-michael@niedermayer.cc> References: <20240512000349.3381912-1-michael@niedermayer.cc> MIME-Version: 1.0 X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 4/5] avcodec/mpegvideo_enc: Fix potential overflow in RD X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: B79oOqDXWHEy Fixes: CID1500285 Unintentional integer overflow Sponsored-by: Sovereign Tech Fund Signed-off-by: Michael Niedermayer --- libavcodec/mpegvideo_enc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavcodec/mpegvideo_enc.c b/libavcodec/mpegvideo_enc.c index b601a1a9e40..73a9082265b 100644 --- a/libavcodec/mpegvideo_enc.c +++ b/libavcodec/mpegvideo_enc.c @@ -1433,7 +1433,7 @@ static int estimate_best_b_count(MpegEncContext *s) goto fail; } - rd += (out_size * lambda2) >> (FF_LAMBDA_SHIFT - 3); + rd += (out_size * (uint64_t)lambda2) >> (FF_LAMBDA_SHIFT - 3); } /* get the delayed frames */ @@ -1442,7 +1442,7 @@ static int estimate_best_b_count(MpegEncContext *s) ret = out_size; goto fail; } - rd += (out_size * lambda2) >> (FF_LAMBDA_SHIFT - 3); + rd += (out_size * (uint64_t)lambda2) >> (FF_LAMBDA_SHIFT - 3); rd += c->error[0] + c->error[1] + c->error[2];