From patchwork Tue Jun 18 13:48:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 49993 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:9196:0:b0:460:55fa:d5ed with SMTP id s22csp2569469vqg; Tue, 18 Jun 2024 06:49:00 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCX/eNfYhbQVF8Gx4DXrlhXmoOl9NWLs7p9SKsex5KfPrsn76n69/UOeRUOvY6yI+On1qU79/ySfTybhf5M+NYSB/TvXkWMQTvJA2w== X-Google-Smtp-Source: AGHT+IHKMksDGIVMR5TxUKPUxFnp+RWfpehJ/+iVFZPOfwWlSob6FzeohPFPkDpn19fpmUI8DRJ+ X-Received: by 2002:a2e:8757:0:b0:2ec:3ca1:e54b with SMTP id 38308e7fff4ca-2ec3ca1e74emr1386011fa.49.1718718540355; Tue, 18 Jun 2024 06:49:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718718540; cv=none; d=google.com; s=arc-20160816; b=hHqPCcJgPGkEiUJxuM1W7sDcTalLJ9V1j2eyJkpLy8G0+sdXEaugX28xdq/X/v0YbG aQmRJEuaFr7m4cQ5QGRygYTyTwhblPVdyHDfSEvZYDTFWbx631+QNMsoB3A8B3syVJTt EugwQRF4LQsY4x2Nw2vVFRZbnk0mW+67f1OVfnZBo9vmNBXqFGSDAWlB6Aifjy66d9Ec 4PmC11+w5OO53Ys1e9bTjKGEEVXIndMdwn9voIUV7SA3piIpDahzqPoFvuD6rjME6ZqR QAJ3Prqe0QX8+UtjJSIrMJjIDA9cZk7WLnCg/HlJJ2w0ZPu8T/dcMENNdPaJlb8GSPD0 9Q5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=f6S3AhDPXJ9s2WbwPpdJnwyocYkQ+zYSf/NHJd/+hmM=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=kcyEs+Be1dGXeYdwz6nyUSTZTM/xLcz1WVUbfQgqJUsK7Oru+gfYqnNurV9tKKm1Ku 2RrytScPfP7LJuOSBVMkLJv//WmnHjAzfcuPSWJw1nPAfkqTkJII4D53Ps47TKPcA7+/ ZPP1O5IIBFqlSgLUO9uyhRxwvwcoBZc863gwW3Oywp+rY2+9swysEEZBfzOcKEfDHOGV CTIGV87+S/rK02A3LXSh4xmlZV4228dqZc6CCuBYXVdZavXNfq/viHnUaM/PAZjhe6DA PkBqXv5DLtCDcG8qYuvoBT2WdsCT3MzpQZFRnhfKJrgKMuw2NDM7FxSfQNvdZPDwZ9iU jmZQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=RGyYWRe3; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 38308e7fff4ca-2ec3c8e6dcfsi428701fa.216.2024.06.18.06.48.59; Tue, 18 Jun 2024 06:49:00 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=RGyYWRe3; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 82DB968D7C3; Tue, 18 Jun 2024 16:48:39 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7F16268D7C2 for ; Tue, 18 Jun 2024 16:48:29 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id DBDCD1C000C for ; Tue, 18 Jun 2024 13:48:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niedermayer.cc; s=gm1; t=1718718509; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=86ji+/sv6yrK37+gkIIKO29+y9QrmG19UywP5HPapPc=; b=RGyYWRe3XQdi1FWXQkeMFXL3eE2B2wa7EpBLgDjUEK4gWmpTeI5TeeGBnvAErc2+2g4DUR 7U6MrEa1EfzEmb2o5AvswQdGv7y7FFvwMV7aYOeACJRZscaQUvf1Apn/QIQDEALJIqOBrt KkrWS8sYHU6mOnMltI5kIwyd52YNxNTWxQXusIggWurQ3XlH+lsy4iyi4DzwtszkkLpGdq 8BTFAbOY+Qjs1pVDS+ugc4jJonWtCEwYVl7EcESx7V7Nk7X1ZzvpZXcGu2W2gTIZHDgY2/ xuG5AzRErxZ0/jw4TkXAkGr3meCdmmlsIVQtMBJu83WFa04B3BQFB+Kqh8AYXw== From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Tue, 18 Jun 2024 15:48:22 +0200 Message-ID: <20240618134826.2189719-3-michael@niedermayer.cc> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240618134826.2189719-1-michael@niedermayer.cc> References: <20240618134826.2189719-1-michael@niedermayer.cc> MIME-Version: 1.0 X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 3/7] avcodec/ratecontrol: Handle wanted bits overflow X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: cK/6GKxod68n Fixes: 5.92611e+20 is outside the range of representable values of type 'unsigned long' Fixes: 68984/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SNOW_fuzzer-5155755073273856 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/ratecontrol.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/libavcodec/ratecontrol.c b/libavcodec/ratecontrol.c index df27639ca73..86ec7a3443e 100644 --- a/libavcodec/ratecontrol.c +++ b/libavcodec/ratecontrol.c @@ -936,6 +936,7 @@ float ff_rate_estimate_qscale(MpegEncContext *s, int dry_run) wanted_bits = rce->expected_bits; } else { const MPVPicture *dts_pic; + double wanted_bits_double; rce = &local_rce; /* FIXME add a dts field to AVFrame and ensure it is set and use it @@ -947,9 +948,14 @@ float ff_rate_estimate_qscale(MpegEncContext *s, int dry_run) dts_pic = s->last_pic.ptr; if (!dts_pic || dts_pic->f->pts == AV_NOPTS_VALUE) - wanted_bits = (uint64_t)(s->bit_rate * (double)picture_number / fps); + wanted_bits_double = s->bit_rate * (double)picture_number / fps; else - wanted_bits = (uint64_t)(s->bit_rate * (double)dts_pic->f->pts / fps); + wanted_bits_double = s->bit_rate * (double)dts_pic->f->pts / fps; + if (wanted_bits_double > INT64_MAX) { + av_log(s, AV_LOG_WARNING, "Bits exceed 64bit range\n"); + wanted_bits = INT64_MAX; + } else + wanted_bits = (int64_t)wanted_bits_double; } diff = s->total_bits - wanted_bits;