From patchwork Mon Sep 18 22:35:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 43768 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:a886:b0:149:dfde:5c0a with SMTP id ca6csp136080pzb; Mon, 18 Sep 2023 15:35:57 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGqtQgg7L3I89mpRERsFOHE0sg4sUVUo/GJ9KjKa1hqwqWN+5U+EacYGJJRjvV5zvBVfbyC X-Received: by 2002:aa7:dd51:0:b0:523:4bfa:b450 with SMTP id o17-20020aa7dd51000000b005234bfab450mr8570186edw.27.1695076556984; Mon, 18 Sep 2023 15:35:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695076556; cv=none; d=google.com; s=arc-20160816; b=mVyOZUyR5T6N2LOWrj2EPhXMX1vLqK/2IEDpSWBoGHYvbD8Wd0uCdQzAHPljy9DZ1D Og2LnXz5AgvbgWbUgeKIozEK5ixGRgGMd//zhA9t7FinElLXQAeRnc8fmvaSDPuXdTy+ EQvRaktwgf+lUvPgz2798u26nXaYOg/PZxOzdxpFvwieYCB9GXgqi4P8w8vnn+PGbq2l i/fi1iPtvMelrRDt0wQP7NmBqBAiCLeITiTOUj/CCBKxBmHnwZVtfeXRsPqs8g+wa4Tb PS1eatQ9wB1wZlw4OGhTIIZjl7+644OLrpSdUi4j1OuD3rVmirII0SFnXkjQqZ1K1JCE g5WQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to; bh=40FvYijq3mCSTGUkjXcV6CV4XVvWFL4YRwXnz0sCDvc=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=FpJuvZFz+5a9Ew//Xc+nOKyEtHDeRk4WPxC4lkje6Obw4nJVdoXB510+2V5LSiZhAe PfQ96r2JnR3vqH/qGDfdc679PLhD08rgC7UacjV+K9SbJc13g+ABS1c1go8Ay6Jl/kVT CP9Q2UGf/ZHWfEIF3HANEsWu3u7SJalN8DiUotEB9mM9NPmXw/z0EgJ0eJ+E+zx53yro CHGF4K6u9z6/xmicVAIqx3UWqna6/TtIG5gxrs/YmT7+nwRr4xaR/oG6j/RldIGeBDes B+mwa2fBrurvKyQh+4Itwnvkt+8H+HDPqhRijhUHp/R200eMAV46/5fF+RVnFr3dbDLW OA3A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id t3-20020aa7d703000000b0052892174844si8750158edq.549.2023.09.18.15.35.56; Mon, 18 Sep 2023 15:35:56 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F2B0468C88D; Tue, 19 Sep 2023 01:35:45 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [217.70.183.196]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 32E0D68C751 for ; Tue, 19 Sep 2023 01:35:37 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id 64B6CE0003 for ; Mon, 18 Sep 2023 22:35:36 +0000 (UTC) From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Tue, 19 Sep 2023 00:35:31 +0200 Message-Id: <20230918223534.23567-2-michael@niedermayer.cc> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230918223534.23567-1-michael@niedermayer.cc> References: <20230918223534.23567-1-michael@niedermayer.cc> X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 2/5] avcodec/apedec: Fix an integer overflow in predictor_update_filter() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: fw6hDTtiDO9q Fixes: signed integer overflow: -2147483506 + -801380 cannot be represented in type 'int' Fixes: 62164/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-6578985923117056 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/apedec.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c index 62cb397490d..8bfbb75b41e 100644 --- a/libavcodec/apedec.c +++ b/libavcodec/apedec.c @@ -1220,7 +1220,7 @@ static av_always_inline int predictor_update_filter(APEPredictor64 *p, if (interim_mode < 1) { predictionA = (int32_t)predictionA; predictionB = (int32_t)predictionB; - p->lastA[filter] = decoded + ((int32_t)(predictionA + (predictionB >> 1)) >> 10); + p->lastA[filter] = (int32_t)(decoded + (unsigned)((int32_t)(predictionA + (predictionB >> 1)) >> 10)); } else { p->lastA[filter] = decoded + ((int64_t)((uint64_t)predictionA + (predictionB >> 1)) >> 10); }