From patchwork Tue Jun 18 13:48:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 49991 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:9196:0:b0:460:55fa:d5ed with SMTP id s22csp2569256vqg; Tue, 18 Jun 2024 06:48:40 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVl6WbV3qLeOSjzySr1rVuzOto2a1VaHlAeM5D4D7mFunQnA2kwUYj94lv9cYZEwolDfm22pzGqbaSQfLW8xfW9ANmbH0OMZzfyiQ== X-Google-Smtp-Source: AGHT+IF8xHoW16bT1fFcQ5zEcE2hJozE3/MJfF9w0KqpqyWZn3DWMPif2PZbMFkQtljaB5JW58IM X-Received: by 2002:a17:906:280f:b0:a6f:5765:671f with SMTP id a640c23a62f3a-a6f60de2700mr850294366b.68.1718718519693; Tue, 18 Jun 2024 06:48:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718718519; cv=none; d=google.com; s=arc-20160816; b=rh0ugiMONG+QLUZAL19Az6FuTiSRnwb5eUf2BvIpXyCK9KXX3rZvToKS4beNpxfyVH ndH3kt0f98FLRQpsEjJXGKat1s8J8kETZTbMKwLyo7dCXohhtJWwMjlhbmbr1BfXcZj+ zJYJ1Z6/23vEatyrCc45ZbKFaO0g0kkdp/wgPV+wg52DnmXn4MlaGZ85b8peZ7oK9AZV S6KS0jnB5wt4Rjtw5TyYeNR+9+qr35DQsEDcXzNueL/Crdp12iwtxDjV90JFkhqZx86r OMmHNeNRcjqyZ8blBOOOiH/EEFkRwHCFRKPAVxuYO7p0PMrDIj9Ta52+2s+BRrcPBwbZ BeAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=aJdTe9wPgSgyixfI+F+yH9C65HVhDloHAcL5X5EdqsY=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=dowcJpv7rFgrlRrsM796XuiABEyQdKHXmdnc/Dce01mVpn7ltMrjbx3SyDRYqRXgFC n6VCTV9/YK7TtA06Xz9L0WjyVVUAu1P3NAdmx6G6V7lQWi6E0c90ZM/5363MDPWt4NmI 4qG4CXJ33aWLv7uR4w4Cs2jsjGMHhF1HEmxtqlgBD9k/JdIg7hQ/BtdsrEqjSS3bJCev mXEW/jPVtpL2D1tbK1cTP92LCRoU4f5BJ+6Bn8D5r3nsohuJ0nQf9JpVrGGG6Q9CglUc 5ghAmyots72R+qTtogZFmW++X07hQD/KKcc9ZD8sslX7/9PVuf3OTBIt9s9ij6PDr/24 DJVg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=hTvuUuVV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a6f56d4b5f6si550409766b.217.2024.06.18.06.48.38; Tue, 18 Jun 2024 06:48:39 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=hTvuUuVV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1137B68D7AD; Tue, 18 Jun 2024 16:48:36 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [217.70.183.198]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id ED76B68D28D for ; Tue, 18 Jun 2024 16:48:27 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id 2DA03C0003 for ; Tue, 18 Jun 2024 13:48:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niedermayer.cc; s=gm1; t=1718718507; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=Lx5SWFzOG1k34znJjvh07r7VAhOXTa782NUcZRdRuVM=; b=hTvuUuVVYIfd/D+Fwk1ur28OC7W8DyItBZYs12oVOM1qc5gGHaGwvrB1bA0+6RpfjcCWep DyEgqGellzqD9vZpI5vMvV24sgwMs7ki2L8NpswPOe4kwyMFfDgZxqFMAC1TLSD3NM1mLV +824JnuoVhOumkGTltWGQEjYn09j7htaJLrs4yT5nofjWhvxkoLnA8V3YBqLmxuzV/eGDf jmky22uvWICMMZUDRbF+aHvM/lHq5Rbg81Ie6W6FL+Dqd5wPQ2Zzv0OnOEH8xPaeYsGetQ Pgjx7pmcVJIUG9Nv3aMQfUopd25oAM/GmY5qlOe8BYAzBI98mydza8yOQJlkRg== From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Tue, 18 Jun 2024 15:48:20 +0200 Message-ID: <20240618134826.2189719-1-michael@niedermayer.cc> X-Mailer: git-send-email 2.45.2 MIME-Version: 1.0 X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 1/7] avcodec/utils: apply the same alignment to YUV410 as we do to YUV420 when motion estimation is used X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: b8+G45FMlb5c The snow encoder uses block based motion estimation which can read out of array if insufficient alignment is used Fixes: out of array access Fixes: 68963/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SNOW_fuzzer-4979988435632128 Fixes: 68969/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SNOW_fuzzer-6239933667803136.fuzz Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/utils.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/libavcodec/utils.c b/libavcodec/utils.c index 337c00e789a..7914f799041 100644 --- a/libavcodec/utils.c +++ b/libavcodec/utils.c @@ -259,6 +259,9 @@ void avcodec_align_dimensions2(AVCodecContext *s, int *width, int *height, if (s->codec_id == AV_CODEC_ID_SVQ1) { w_align = 64; h_align = 64; + } else if (s->codec_id == AV_CODEC_ID_SNOW) { + w_align = 16; + h_align = 16; } break; case AV_PIX_FMT_RGB555: From patchwork Tue Jun 18 13:48:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 49992 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:9196:0:b0:460:55fa:d5ed with SMTP id s22csp2569347vqg; Tue, 18 Jun 2024 06:48:49 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUNKcn3GK9LJ1cbjELhgzfNdDdZWGzAy/moL44px+0JYYvP6uYflAqRwB7Yjho3iZVCzMa9dxDoM48qol0ewzYVGaikaa8Uf6h8YA== X-Google-Smtp-Source: AGHT+IHHpoVGd0M2SvAigEdjRlk6cJYwXGcuJrFkCpiGpTpw7WDyNrmF4nQUy6N0wCHk7Xm9LgBv X-Received: by 2002:a50:8e52:0:b0:57c:fc75:408c with SMTP id 4fb4d7f45d1cf-57cfc7541ebmr1236633a12.19.1718718529496; Tue, 18 Jun 2024 06:48:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718718529; cv=none; d=google.com; s=arc-20160816; b=W+oK7GRBawxdonnyostIIcgKDvv2s8iXvsaenmtGxxS9Nlg7vO73Vo9T4bHhTEFHQJ sRgY0wFXjfv0rrlQs7hqrKTLT69sfE5bgal0Za2fUkQjI8b7EW4mWrsF3jjzUmRCKTKp 9CK1UMzBqCJfg6eOhHm5mmfaBf5EKPfkGhDlP9dZZwszuaVXydp9E+ErBpMeodNltyVE HbDJwB+jHrzpAhI7R1yac6UvtnB37Eq5ARoMl6dN5cdV3v1mSnZ+GB5KmByMskEAJQjo hHlmfomNvwSC+/rpVxi80j1qE7GVFZK6rP1KqQ2vE5KSCsMSjrtbbKjMThKPlfV7eBsz QvhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=gytfDHr4URqLhTa7mAcRRSafUyOaZ+Czn1cKEi8KmyU=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=hD2QHYp9diCmDR4OprTR5CMXNOAO9CnGYJG3WYFfPF/wUrkbB8VZKiJ9L231SL2T60 uB3Rd376TzwDTLHns8s4zS1YzmdP3crPua9XL/uTFTChirHrxL7Z9PBOiMgsPkk8sru5 KeFZhIeDNvU0QS8W82fZArYH4Dqz3skxmI4X6dhP3jysYeG8OEAoklGshSP/a+ikB+He ng5pPukjnwSkIlK2zRezDr7HIIli6DXexudDr8WD4wg/mCLUmL8rgrzem+zr6ZZMPTjN k4rGKduj3aOWHRtqBOnWEzGmN1ZceavU3FqtodJ3vqfEwBz/ECnNA1NX5PDJGggnzHGa PqSw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b="nx//uD68"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-57cb72d6222si5751922a12.113.2024.06.18.06.48.49; Tue, 18 Jun 2024 06:48:49 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b="nx//uD68"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D0B2368D7CE; Tue, 18 Jun 2024 16:48:37 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B5CC268D28D for ; Tue, 18 Jun 2024 16:48:28 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id 1D05E1C000A for ; Tue, 18 Jun 2024 13:48:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niedermayer.cc; s=gm1; t=1718718508; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xTuFf14dgCjPlI6/ymzXb7WnSoKEp4JSfXdufJkY24Q=; b=nx//uD68Qc9tNmyb3AwJ1d3nWf0X/OyUus9lrQENwrqFdToGzYLClB2bD5r+YGt3Gebls2 dJ6Rc0GmHcXzukYGjJ/JmDAFZFrHdnmOjby4s1EI0LIa0JXahJEdprRtRV9aokkd+4lFhm fmX0HcVQBp6EcXrt8eQ++9AK2z1STpqM3Soull5y2EZZvyfWpgBhtqxDLe4k1hUEebGofo AiNR2HDmVT3fPzpSIy2L1b/i9Y3sbST7oU4XmsrYwcKm+xYXvLSZBxleDmn9i7AzN+fZUT FujRbYA5Grw0gV2HBDdp31AMcPnI5eEzqS1+fFhROJjxNynKQdI1d7frrGw2JA== From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Tue, 18 Jun 2024 15:48:21 +0200 Message-ID: <20240618134826.2189719-2-michael@niedermayer.cc> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240618134826.2189719-1-michael@niedermayer.cc> References: <20240618134826.2189719-1-michael@niedermayer.cc> MIME-Version: 1.0 X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 2/7] avcodec/ratecontrol: Try to keep fps as a rational X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Ow8k9Q/R3jD1 Signed-off-by: Michael Niedermayer --- libavcodec/ratecontrol.c | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/libavcodec/ratecontrol.c b/libavcodec/ratecontrol.c index 609d47faeb4..df27639ca73 100644 --- a/libavcodec/ratecontrol.c +++ b/libavcodec/ratecontrol.c @@ -56,20 +56,25 @@ void ff_write_pass1_stats(MpegEncContext *s) s->header_bits); } -static double get_fps(AVCodecContext *avctx) +static AVRational get_fpsQ(AVCodecContext *avctx) { if (avctx->framerate.num > 0 && avctx->framerate.den > 0) - return av_q2d(avctx->framerate); + return avctx->framerate; FF_DISABLE_DEPRECATION_WARNINGS - return 1.0 / av_q2d(avctx->time_base) #if FF_API_TICKS_PER_FRAME - / FFMAX(avctx->ticks_per_frame, 1) + return av_div_q((AVRational){1, FFMAX(avctx->ticks_per_frame, 1)}, avctx->time_base); +#else + return av_inv_q(avctx->time_base); #endif - ; FF_ENABLE_DEPRECATION_WARNINGS } +static double get_fps(AVCodecContext *avctx) +{ + return av_q2d(get_fpsQ(avctx)); +} + static inline double qp2bits(const RateControlEntry *rce, double qp) { if (qp <= 0.0) { @@ -332,12 +337,13 @@ static int init_pass2(MpegEncContext *s) RateControlContext *rcc = &s->rc_context; AVCodecContext *a = s->avctx; int i, toobig; - double fps = get_fps(s->avctx); + AVRational fps = get_fpsQ(s->avctx); double complexity[5] = { 0 }; // approximate bits at quant=1 uint64_t const_bits[5] = { 0 }; // quantizer independent bits uint64_t all_const_bits; - uint64_t all_available_bits = (uint64_t)(s->bit_rate * - (double)rcc->num_entries / fps); + uint64_t all_available_bits = av_rescale_q(s->bit_rate, + (AVRational){rcc->num_entries,1}, + fps); double rate_factor = 0; double step; const int filter_size = (int)(a->qblur * 4) | 1; From patchwork Tue Jun 18 13:48:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 49993 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:9196:0:b0:460:55fa:d5ed with SMTP id s22csp2569469vqg; Tue, 18 Jun 2024 06:49:00 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCX/eNfYhbQVF8Gx4DXrlhXmoOl9NWLs7p9SKsex5KfPrsn76n69/UOeRUOvY6yI+On1qU79/ySfTybhf5M+NYSB/TvXkWMQTvJA2w== X-Google-Smtp-Source: AGHT+IHKMksDGIVMR5TxUKPUxFnp+RWfpehJ/+iVFZPOfwWlSob6FzeohPFPkDpn19fpmUI8DRJ+ X-Received: by 2002:a2e:8757:0:b0:2ec:3ca1:e54b with SMTP id 38308e7fff4ca-2ec3ca1e74emr1386011fa.49.1718718540355; Tue, 18 Jun 2024 06:49:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718718540; cv=none; d=google.com; s=arc-20160816; b=hHqPCcJgPGkEiUJxuM1W7sDcTalLJ9V1j2eyJkpLy8G0+sdXEaugX28xdq/X/v0YbG aQmRJEuaFr7m4cQ5QGRygYTyTwhblPVdyHDfSEvZYDTFWbx631+QNMsoB3A8B3syVJTt EugwQRF4LQsY4x2Nw2vVFRZbnk0mW+67f1OVfnZBo9vmNBXqFGSDAWlB6Aifjy66d9Ec 4PmC11+w5OO53Ys1e9bTjKGEEVXIndMdwn9voIUV7SA3piIpDahzqPoFvuD6rjME6ZqR QAJ3Prqe0QX8+UtjJSIrMJjIDA9cZk7WLnCg/HlJJ2w0ZPu8T/dcMENNdPaJlb8GSPD0 9Q5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=f6S3AhDPXJ9s2WbwPpdJnwyocYkQ+zYSf/NHJd/+hmM=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=kcyEs+Be1dGXeYdwz6nyUSTZTM/xLcz1WVUbfQgqJUsK7Oru+gfYqnNurV9tKKm1Ku 2RrytScPfP7LJuOSBVMkLJv//WmnHjAzfcuPSWJw1nPAfkqTkJII4D53Ps47TKPcA7+/ ZPP1O5IIBFqlSgLUO9uyhRxwvwcoBZc863gwW3Oywp+rY2+9swysEEZBfzOcKEfDHOGV CTIGV87+S/rK02A3LXSh4xmlZV4228dqZc6CCuBYXVdZavXNfq/viHnUaM/PAZjhe6DA PkBqXv5DLtCDcG8qYuvoBT2WdsCT3MzpQZFRnhfKJrgKMuw2NDM7FxSfQNvdZPDwZ9iU jmZQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=RGyYWRe3; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 38308e7fff4ca-2ec3c8e6dcfsi428701fa.216.2024.06.18.06.48.59; Tue, 18 Jun 2024 06:49:00 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=RGyYWRe3; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 82DB968D7C3; Tue, 18 Jun 2024 16:48:39 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7F16268D7C2 for ; Tue, 18 Jun 2024 16:48:29 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id DBDCD1C000C for ; Tue, 18 Jun 2024 13:48:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niedermayer.cc; s=gm1; t=1718718509; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=86ji+/sv6yrK37+gkIIKO29+y9QrmG19UywP5HPapPc=; b=RGyYWRe3XQdi1FWXQkeMFXL3eE2B2wa7EpBLgDjUEK4gWmpTeI5TeeGBnvAErc2+2g4DUR 7U6MrEa1EfzEmb2o5AvswQdGv7y7FFvwMV7aYOeACJRZscaQUvf1Apn/QIQDEALJIqOBrt KkrWS8sYHU6mOnMltI5kIwyd52YNxNTWxQXusIggWurQ3XlH+lsy4iyi4DzwtszkkLpGdq 8BTFAbOY+Qjs1pVDS+ugc4jJonWtCEwYVl7EcESx7V7Nk7X1ZzvpZXcGu2W2gTIZHDgY2/ xuG5AzRErxZ0/jw4TkXAkGr3meCdmmlsIVQtMBJu83WFa04B3BQFB+Kqh8AYXw== From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Tue, 18 Jun 2024 15:48:22 +0200 Message-ID: <20240618134826.2189719-3-michael@niedermayer.cc> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240618134826.2189719-1-michael@niedermayer.cc> References: <20240618134826.2189719-1-michael@niedermayer.cc> MIME-Version: 1.0 X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 3/7] avcodec/ratecontrol: Handle wanted bits overflow X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: cK/6GKxod68n Fixes: 5.92611e+20 is outside the range of representable values of type 'unsigned long' Fixes: 68984/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SNOW_fuzzer-5155755073273856 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/ratecontrol.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/libavcodec/ratecontrol.c b/libavcodec/ratecontrol.c index df27639ca73..86ec7a3443e 100644 --- a/libavcodec/ratecontrol.c +++ b/libavcodec/ratecontrol.c @@ -936,6 +936,7 @@ float ff_rate_estimate_qscale(MpegEncContext *s, int dry_run) wanted_bits = rce->expected_bits; } else { const MPVPicture *dts_pic; + double wanted_bits_double; rce = &local_rce; /* FIXME add a dts field to AVFrame and ensure it is set and use it @@ -947,9 +948,14 @@ float ff_rate_estimate_qscale(MpegEncContext *s, int dry_run) dts_pic = s->last_pic.ptr; if (!dts_pic || dts_pic->f->pts == AV_NOPTS_VALUE) - wanted_bits = (uint64_t)(s->bit_rate * (double)picture_number / fps); + wanted_bits_double = s->bit_rate * (double)picture_number / fps; else - wanted_bits = (uint64_t)(s->bit_rate * (double)dts_pic->f->pts / fps); + wanted_bits_double = s->bit_rate * (double)dts_pic->f->pts / fps; + if (wanted_bits_double > INT64_MAX) { + av_log(s, AV_LOG_WARNING, "Bits exceed 64bit range\n"); + wanted_bits = INT64_MAX; + } else + wanted_bits = (int64_t)wanted_bits_double; } diff = s->total_bits - wanted_bits; From patchwork Tue Jun 18 13:48:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 49994 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:9196:0:b0:460:55fa:d5ed with SMTP id s22csp2569545vqg; Tue, 18 Jun 2024 06:49:09 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVlt9VNE2NyqCPtt8C42Uln7bKfEJYYM1Y3tntzGM8sY6Pr0+yInY5pJqIC/RdcDLqpzsWGBNTXes/xe1UPd3KD0QlpnRpmT+HdJg== X-Google-Smtp-Source: AGHT+IH7cYZPIxiL5zXW9pT2KT3iX7KRUr/BJ6/N4N0MOstL9rRKTUcHEPVBNAq7f8rUkH5Gp3vO X-Received: by 2002:a17:907:940c:b0:a6f:5ef5:2f63 with SMTP id a640c23a62f3a-a6f60d20f0fmr957829966b.18.1718718549461; Tue, 18 Jun 2024 06:49:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718718549; cv=none; d=google.com; s=arc-20160816; b=OQsaUKGZf+orxbrdccce3/J1g3eWRvv5PjmVyWH4nWFSnvvLkEBVDC0Fx2syUNQv5r bhhMBJOdszcr7LTXqHCrsLSy+4j4Kg4TmDmikCJpZbVYdnRrYlr+LS4D31TYZFKvEwnJ bgdB2AONyAazesAFD2apJExdRa6t9auvIsp2F57CrRTHRCZReVZc4OlbnJMBQRyudVNd 8jumplNsUo3n56H9rAaSkxocveL6ZbvY5komP9L2P37DBHDF1A+WDP8uMemxxnWv2Orq jBoMSzgY0Dmne2iCS23pWpiXEvzEa/lq1omueG6KTSZD7hBDqGntRIuX1O8OjjnrDkVD hoyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=fGVp5bqbRVB5rHmW/5Yv8kJxKECZKKkIW3sGrOC+tdM=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=tw3HMVmbfdmkA/bZlOT5tcaJpD9FgdWRsdcUWOZwCMs0O9J1hNHAZJa9FC3iyI4DYj 4oNrUghIpM9K4/yZvMHlo6AnPbn3xAsGFxMWiF/7cFGXBdyNLfZUGyj4NixjxJJisqSY ufSPAJykbZec6mlUGyifOjU+JGJRGUDXvEPa4PUJSZsNvRaIpGYgh+rIMx8PPrRHYSyj 13GhCXle4YbaUCWRytgJoFFmbZ0TCUk5w5jjWmbEZesHXU71HkAgdYQCPJY45HBfHfpw mSi3LxVlQxHUOwbQ28u8sjwZFv1seBIqCFVG3hQxU06YLpwr1Wz0yPaYXiKlfbdVQ6xv eCIA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=bGN1ukip; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a6f56e26910si540917866b.804.2024.06.18.06.49.08; Tue, 18 Jun 2024 06:49:09 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=bGN1ukip; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6C31368D7DA; Tue, 18 Jun 2024 16:48:40 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D22E168D28D for ; Tue, 18 Jun 2024 16:48:30 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id 06DA2240009 for ; Tue, 18 Jun 2024 13:48:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niedermayer.cc; s=gm1; t=1718718510; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CBGAXaT9TMY+GjcOlTRLYak4UIJGt7Ze3OyM4h930E0=; b=bGN1ukipFbyeq7d7CGXIHaRQMrSm7ppcZQYu/MwrDrQeImm3rwYwe6IrS1RXB4y5LCmnxU WQigIOvsd3+bqYihYwtW99Sygu5Jn4EBQ3423nx5+zsBd2sN5a+IoAaU7hXqVV/GKR0RIL tqINEO7UP736ZUI6Qssx4vlJLD/Ul9DA+d9fb5lUJR41+c8yWF25FL9yjRVJ4Ss0Dh5a54 NncD4fuKKq6Jxq/lP0AGaphnxeD/nQNEyZxIjr0N/ByWpbwy7nC7fsVhai1ZCCy5SLqdW5 h/ftZdXw8nsk5nutm9TtbLFviSu1kEq40owGHJhOifI8pjO5remKkHIP8tjd6w== From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Tue, 18 Jun 2024 15:48:23 +0200 Message-ID: <20240618134826.2189719-4-michael@niedermayer.cc> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240618134826.2189719-1-michael@niedermayer.cc> References: <20240618134826.2189719-1-michael@niedermayer.cc> MIME-Version: 1.0 X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 4/7] avcodec/snowenc: MV limits due to mv_penalty table size X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: euaHF7OFqbL8 Fixes: out of array read Fixes: 69673/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SNOW_fuzzer-5476592894148608 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/snowenc.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/libavcodec/snowenc.c b/libavcodec/snowenc.c index 8d6dabae658..dd6ce36aa54 100644 --- a/libavcodec/snowenc.c +++ b/libavcodec/snowenc.c @@ -413,6 +413,7 @@ static int encode_q_branch(SnowEncContext *enc, int level, int x, int y) int my_context= av_log2(2*FFABS(left->my - top->my)); int s_context= 2*left->level + 2*top->level + tl->level + tr->level; int ref, best_ref, ref_score, ref_mx, ref_my; + int range = MAX_MV >> (1 + qpel); av_assert0(sizeof(s->block_state) >= 256); if(s->keyframe){ @@ -454,6 +455,11 @@ static int encode_q_branch(SnowEncContext *enc, int level, int x, int y) c->xmax = - (x+1)*block_w + (w<<(LOG2_MB_SIZE - s->block_max_depth)) + 16-3; c->ymax = - (y+1)*block_w + (h<<(LOG2_MB_SIZE - s->block_max_depth)) + 16-3; + c->xmin = FFMAX(c->xmin,-range); + c->xmax = FFMIN(c->xmax, range); + c->ymin = FFMAX(c->ymin,-range); + c->ymax = FFMIN(c->ymax, range); + if(P_LEFT[0] > (c->xmax<xmax< (c->ymax<ymax< (c->xmax<xmax< X-Patchwork-Id: 49995 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:9196:0:b0:460:55fa:d5ed with SMTP id s22csp2569645vqg; Tue, 18 Jun 2024 06:49:19 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCV1MpsSjXBLTzqcwbpadE5dcXf1Lf2rc/zQ6sAEiPS0k7t1oWQGgw2M8f9JGm+FnXDUSLRkTJZ0AR9u2PmJQHPXsAGgMnOHgHUg8w== X-Google-Smtp-Source: AGHT+IHUHbRYC6wETztTw6Kp82X27XB4x7vbfwRmAd+KNMqBP21C2xIuNfHIi+3cJXABx0SDd6n9 X-Received: by 2002:a2e:9785:0:b0:2ec:2038:925d with SMTP id 38308e7fff4ca-2ec203894admr69287171fa.1.1718718559619; Tue, 18 Jun 2024 06:49:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718718559; cv=none; d=google.com; s=arc-20160816; b=mgZm3QANcAAdmRBod5HtJrRK0NdbbTHpW+DjSD6YvkcZKzTuJ5+hpo4uqWkRYVmtqw 65Cpbj2cacBrf980XLVMSrrngsUvfE0JnNQWTX9DQoTj/yi79INbOspyj7OSeIp+kMoH aiqoMEo4kW6myS4pIgPnflEixXX0zPhcPS3kexnzNtVyQSu26Fu7/zAF7BVxDlkhS0Ce v2/0bQtsiq136/P6m0FvFA9cMlnqGjI7Xh6aA7wq4rTrmlAx8zSrhZ5aF0gG5W8y0gNb nXr698jdvn+LIposek+6GUdreDVMbaeZsjCy14/dAuF9SZBdNe6DSv44HPmmp//WC37X Oddg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=5Xwu4MfEuLaPvwvKyR8xl8kOF72qRv3C3RvgF+XDD7w=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=HZCH44K7dSU3SJQ1KjZFofCeF/MpevG863uQkyOk2jhRmf8pTpKGGmTPiKE4PZ+CKw OlgdQ7yx95blpXpc2HQ52ciQfjnoWBD6UVdfH0T4D/nyGtydeHlI4I7cWslB1GwIZbNF 90DdiF/+ehZrS45VATol1uRD0rWIn+910s1U11S0GwU3y7TnugSEZaQDg1CDTfja20Si tTMSQd7pOQMDkTrT7N5vOB2ucxkpSn6+erwB/DNulqS9dhmPGfbcMtg4B2YXczXdoo05 Hosg5zmaQikUV6aIXv51WJ5XTgUhCA6s03LfOVPUygnBXRuSGf1zbSoXXBYOWOTXC8YI aTGA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=BFYRBXh2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a6f8e53ebbfsi127205266b.805.2024.06.18.06.49.18; Tue, 18 Jun 2024 06:49:19 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=BFYRBXh2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7720E68D7E3; Tue, 18 Jun 2024 16:48:41 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 34F7B68D7BE for ; Tue, 18 Jun 2024 16:48:32 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id 68DAAFF80C for ; Tue, 18 Jun 2024 13:48:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niedermayer.cc; s=gm1; t=1718718511; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XBT1n/0Ja9mevrJkxAIK2LG1Vo8zdStvg8tcbUBKhNI=; b=BFYRBXh20SRE/DTrCpIegaatqQsp5AktFL/RPEqtVnKLAl0I8ade/AezIOCqIsN5KyFF+P L3zJUVeVS4qctboGt5hClmV8ZwTpeidcoVr3GFS3PB2SawrS0bXFaBG8a21KIWGpFxRsBe /JcJrq9eUHlOFQqTBKg/UFMdpMAWHi5YMWPaul2lrDi6CbaVp8DdFVIHel9kssGpniQ5U8 t4j2aj01vExjvjtQSeBzhdfQh4/zIiLJf4ytV1qvevPqfVQPcoMyhyqBAnGZODolZr0abx 44OVHitb8ZYYu5SzteyBAr5ijf5EeVjO0+PY35wZs5HQTXWmwWI9utGHGtpxNQ== From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Tue, 18 Jun 2024 15:48:24 +0200 Message-ID: <20240618134826.2189719-5-michael@niedermayer.cc> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240618134826.2189719-1-michael@niedermayer.cc> References: <20240618134826.2189719-1-michael@niedermayer.cc> MIME-Version: 1.0 X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 5/7] avcodec/jfdctint_template: Fewer integer anomalies X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: PiVy3DMC7EkW Fixes: signed integer overflow: 105788 * -20995 cannot be represented in type 'int' Fixes: signed integer overflow: 923211729 + 2073948236 cannot be represented in type 'int' Fixes: signed integer overflow: 1281179284 + 2073948236 cannot be represented in type 'int' Fixes: 68975/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PRORES_fuzzer-6266769177116672 Fixes: 68997/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PRORES_KS_fuzzer-6284237161431040 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/jfdctint_template.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/libavcodec/jfdctint_template.c b/libavcodec/jfdctint_template.c index ca17300c324..aa2680132ee 100644 --- a/libavcodec/jfdctint_template.c +++ b/libavcodec/jfdctint_template.c @@ -69,7 +69,7 @@ #define GLOBAL(x) x #define RIGHT_SHIFT(x, n) ((x) >> (n)) #define MULTIPLY16C16(var,const) ((var)*(const)) -#define DESCALE(x,n) RIGHT_SHIFT((x) + (1 << ((n) - 1)), n) +#define DESCALE(x,n) RIGHT_SHIFT((int)(x) + (1 << ((n) - 1)), n) /* @@ -175,7 +175,7 @@ #if BITS_IN_JSAMPLE == 8 && CONST_BITS<=13 && PASS1_BITS<=2 #define MULTIPLY(var,const) MULTIPLY16C16(var,const) #else -#define MULTIPLY(var,const) ((var) * (const)) +#define MULTIPLY(var,const) (int)((var) * (unsigned)(const)) #endif @@ -261,7 +261,7 @@ FUNC(ff_jpeg_fdct_islow)(int16_t *data) { int tmp0, tmp1, tmp2, tmp3, tmp4, tmp5, tmp6, tmp7; int tmp10, tmp11, tmp12, tmp13; - int z1, z2, z3, z4, z5; + unsigned z1, z2, z3, z4, z5; int16_t *dataptr; int ctr; From patchwork Tue Jun 18 13:48:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 49996 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:9196:0:b0:460:55fa:d5ed with SMTP id s22csp2569754vqg; Tue, 18 Jun 2024 06:49:30 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUBa2WlnbD5rNI0k8RLWDkfPV2TASBExUkSk3i8x9tfJ9wzm7WDmJVM9ixyFW5JZKjxYuAGb1bMsrigXv4YXNbsna5LQOn6nCskvg== X-Google-Smtp-Source: AGHT+IEoSAjoLTBb1WmqU/7r0xcun4k5hUqjTJUZKdrDzoCmqoe5Xfmm9um61+m2GP9HbSl+WOqS X-Received: by 2002:a17:906:35da:b0:a6e:f6bd:edd9 with SMTP id a640c23a62f3a-a6f60dc51ddmr797768966b.59.1718718570018; Tue, 18 Jun 2024 06:49:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718718570; cv=none; d=google.com; s=arc-20160816; b=WnTwonPvDNAuTkp8iO4e7E5patT6Bh0KnbQybaCVXn3eSlBPVaF2K10AZQjh3Lze2r qPdPvngyX65/H+izNpkJNqHc0nBWjHT623/swJE+ZsaTZSpGLS9fcnDQyPkV0VbAn2q2 tDoAkPzfNEs/SeTy+SaHLhq9s6DjhYjAQe0sd/K5H4DfptEYh+t5XkzuwDCTVryLgC+8 xqeQYXs07UUE361DVBLUAL5Z6XIhbUCqec7qsxSOf70SLlopF+ncREps/iOYqzDD4v5w Jbi8nqDSS4X/BTVdAVzbxX3BzBu4odIQbK8tzZTn3uCn+aMgb/wXpWabNPNrSrq8oy64 EtnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=foiwmHhhOVVBUD5hRUjmKzEkNG0h7ozl8y//0Dzrhmg=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=P8uwV+TqJL9sd6cYJC+UELIfaFokisdu416vsPpEol0XH1gnwBypmvnflBNOg7Tkby LWHZiC0aRwcMy6wl/X+IIDOKqBjQuAckXMFNivCKm/qfQe/f7mI2/RJrKGJBeJ4ofSCR p09XYWgj67I2HtVONrUJNVNFK/dU1uLS5YlVJl1nfPeA2+IysNp7ijrWcirWVxBzzeZN NownwQeugVV/dfRwE2FOvkRYS+R7L8LfYGKG1M5V8E8MfkWjNNq6Aoyfhi5nd35bIZj3 EwfDklNfwVIfZcGaXlNxue2NV2RknxGGUFazoed5kGVSZcbgR2852Y/lf0ncRSaqG5YL ssXQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b="S/Ci9lvh"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a6f56df7496si524862166b.528.2024.06.18.06.49.28; Tue, 18 Jun 2024 06:49:30 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b="S/Ci9lvh"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5BF3C68D7C5; Tue, 18 Jun 2024 16:48:43 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 620E868D7CF for ; Tue, 18 Jun 2024 16:48:33 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id AF0901BF20B for ; Tue, 18 Jun 2024 13:48:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niedermayer.cc; s=gm1; t=1718718512; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HN9q2Xui9vPvLA49HQKIJI0zR0DQ5JwqRRKQhaXVNL0=; b=S/Ci9lvhcXFwG/qEN0BkirGq3svVhM+Ob0M31i6r24TOlNTXDRFNvsOQsbZSs+brHdIBDB jF1fpf8CK3DRytusjxMo31XDLEtkvslHM69qhzpHzrgK509a2zGwCwKDGrxlBIghqd5ERQ aQbbC/UmjdrcXvPw7BJK5zDBi155JnLBxTLHu9KAeM7jGvUakMbnAfdWP4tJ78C4b8hNuX 2Ss1vdND8ZmKpxNisGH3joS+xYjx/WVY3NOW+jluaZXpYMCam0BqieQnT6sQDEjO6wrbfS J//AUt8PWKv32guW71QRV9l3od0H/PWEbCjrDSTtp1CqliILEvBXMwVGhsSt8w== From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Tue, 18 Jun 2024 15:48:25 +0200 Message-ID: <20240618134826.2189719-6-michael@niedermayer.cc> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240618134826.2189719-1-michael@niedermayer.cc> References: <20240618134826.2189719-1-michael@niedermayer.cc> MIME-Version: 1.0 X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 6/7] avcodec/vc2enc: Fix overflows with storing large values X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: IAhAJt2CiQmf Fixes: left shift of 1431634944 by 2 places cannot be represented in type 'int' Fixes: left shift of 1073741824 by 1 places cannot be represented in type 'int' Fixes: 69061/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC2_fuzzer-6325700826038272 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/vc2enc.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/libavcodec/vc2enc.c b/libavcodec/vc2enc.c index 4ea836d9c9a..7fa6ddc4ca0 100644 --- a/libavcodec/vc2enc.c +++ b/libavcodec/vc2enc.c @@ -189,7 +189,9 @@ typedef struct VC2EncContext { static av_always_inline void put_vc2_ue_uint(PutBitContext *pb, uint32_t val) { int i; - int pbits = 0, bits = 0, topbit = 1, maxval = 1; + int bits = 0; + unsigned topbit = 1, maxval = 1; + uint64_t pbits = 0; if (!val++) { put_bits(pb, 1, 1); @@ -206,12 +208,13 @@ static av_always_inline void put_vc2_ue_uint(PutBitContext *pb, uint32_t val) for (i = 0; i < bits; i++) { topbit >>= 1; + av_assert2(pbits <= UINT64_MAX>>3); pbits <<= 2; if (val & topbit) pbits |= 0x1; } - put_bits(pb, bits*2 + 1, (pbits << 1) | 1); + put_bits64(pb, bits*2 + 1, (pbits << 1) | 1); } static av_always_inline int count_vc2_ue_uint(uint32_t val) From patchwork Tue Jun 18 13:48:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 49997 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:9196:0:b0:460:55fa:d5ed with SMTP id s22csp2569843vqg; Tue, 18 Jun 2024 06:49:39 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUBoRF9+LSMbTqNjxi90OHQj9o0z+BWY8QJiyXFslu2tGCVQW/j7PcjD8qCXe7nsCNayFhddVvL27epxMUP2Re9EIvfAIdPUI3DyQ== X-Google-Smtp-Source: AGHT+IHFnd2PoMdLLepVRLFUVaegfsMX+S1QgOlUsogHOFtM6wYZ1+abxko/6JF4wu/XVRGRKvJQ X-Received: by 2002:a17:906:259a:b0:a6f:4b7d:5994 with SMTP id a640c23a62f3a-a6f60df13admr685634466b.7.1718718579233; Tue, 18 Jun 2024 06:49:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718718579; cv=none; d=google.com; s=arc-20160816; b=lrQGV8vVKY87p+Lh1nQkWaB8uIVfCqJ0ZCvEiNAkN8vwcxAJpy+EbJKTe6iZHuTkyv 6MLuPC0jqLtCVp6WyRLUOEbZCDMRczJDRcPsJIbX9HlJ7zb053GOr1gVbO/ALtzUofHq 0htRogOAm7pBE1D2U2H2snEOzqNvg/n47QAQyzFM/VZJgGw3/StHMZScE9Zw7AeHVMSE OUfBZbeGFNgL6fAIvRPDJL6pF1qBCR0Kv+glWXPOjzmVYzzx931dGiiQrHTZC8Hu0nbJ 7vrvmjtbUxTw+lm++FBNDl3303AZlQWLXxHMBfmMt2zb/NAp+1Np2n8jWTX13Tf+OB8T uHMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=oAevSvDTDFREte9hsPHZRUXqvbfHGiHwGPVVJIKuu2E=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=FGLi1tLd49SMnxIwJCXO7x6Ik+hPXR7e7g5DGUHG1kfXYrKlrDc2KtsA3emzjI9aRW QgWjJMLBuM4hM8Duve7CzyI/UPKPd3lBnzHXTqqCPChKo/FGK8kysyBM3P74ZKE6F+ob HWkiEBcOFt7WE3prKsaz6aAM9rePlgiia0e84ZTlp2jvVj0o1dN3CPN2RFUQfH8Sq4KL 7y3RJdodAxoyID7W3AQ3L+sesSkbf5eX0q48M12dJWn91u0Dv0S+RsgX5Ml2p4g1cC3O yBOC0WO6XMCDNyHxRPma+WCzy/tEMBnqtFTT4tBcLiFXv6GaKaLNpByzxuhZG4cm6BSz Fr7Q==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=BfGgjz2J; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a6f56dd63b2si511114466b.443.2024.06.18.06.49.38; Tue, 18 Jun 2024 06:49:39 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=BfGgjz2J; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E0F2868D7ED; Tue, 18 Jun 2024 16:48:44 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0ED2D68D7CA for ; Tue, 18 Jun 2024 16:48:34 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id 7369E240008 for ; Tue, 18 Jun 2024 13:48:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niedermayer.cc; s=gm1; t=1718718513; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=T369jEnMEzDITyJsoKyE0c/ZWEF+Ut2Zy/vvT2Y07zI=; b=BfGgjz2JxPg7d5Ld6vmZ51rUTGNdDSjbqxZihfzKf/VsLhL+RRLDHNhYeN1Yh1pVYQBQzr OROq9pQXDIQsbO8+FhOqKn6W9id9WQXaoeSxVCxVq/Gn641GE4a9uiRTJ9YHYedDOHhLiS btFbbE/C3M2hH6re6TsIrTTA4qdUPkeIuNru1kltJId5JRneVMEv0lrkbXXyfy4+03Zh/3 bykpEIdJgduZka853MDdh0DFnpsRl9xWZPmjfDHslc1ngbdr03UedCCQyykLOvu46Tv04u F41i/vhJqvPuCw0A0f+IoRJnEy5hWH066+BuDCmjfqR6/eC+sfvMH3733Eq47g== From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Tue, 18 Jun 2024 15:48:26 +0200 Message-ID: <20240618134826.2189719-7-michael@niedermayer.cc> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240618134826.2189719-1-michael@niedermayer.cc> References: <20240618134826.2189719-1-michael@niedermayer.cc> MIME-Version: 1.0 X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 7/7] avcodec/proresenc_kostya: use unsigned alpha for rotation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: SQOBpGV0/pwt Fixes: left shift of negative value -208 Fixes: 69073/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PRORES_KS_fuzzer-4745020002336768 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/proresenc_kostya.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/proresenc_kostya.c b/libavcodec/proresenc_kostya.c index 8b91ca1a98a..fe8cc5f0fda 100644 --- a/libavcodec/proresenc_kostya.c +++ b/libavcodec/proresenc_kostya.c @@ -343,7 +343,7 @@ static void get_slice_data(ProresContext *ctx, const uint16_t *src, static void get_alpha_data(ProresContext *ctx, const uint16_t *src, ptrdiff_t linesize, int x, int y, int w, int h, - int16_t *blocks, int mbs_per_slice, int abits) + uint16_t *blocks, int mbs_per_slice, int abits) { const int slice_width = 16 * mbs_per_slice; int i, j, copy_w, copy_h;