From patchwork Sat Nov 5 20:16:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 39183 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp1198848pzb; Sat, 5 Nov 2022 13:17:06 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4MIVGrZV53wkBNXXOefZ6yKi0AGJl4gNg8F4GhNXc+YHQP5wtXn0KTWAqfIwfJkzZTkxgL X-Received: by 2002:a17:907:78b:b0:741:3d29:33d2 with SMTP id xd11-20020a170907078b00b007413d2933d2mr42116511ejb.103.1667679426705; Sat, 05 Nov 2022 13:17:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667679426; cv=none; d=google.com; s=arc-20160816; b=0JM3+JE1jz1iijJGXWSzjDHRALGuohCDICVoAQbfWg8HGIxXOhHEjn+4toYumC1HTP HLl5qRjKGt0z79jI+czK042cxM1Ov1OGj6QJ0gXo7wz6+D1D0frsJW5JZUjGEtfNp4Y9 cuygRbbiwCfr7PpRqOpDQbaoWenpcX/O3R/gqMrmdcTF2PzZ4KLYiF5GIQAa/bBQYCmI G6xdx2WkWnEzB2FNC5lVcuh/LGfqsWfI+tObk/5mXHjDLkZJZfxFHfMnIkq54Cn9dAHL N+kvvaSRU0nEZ0XOA20ehKiQe3rrjEM+v0IVBUK7wqsnsCnCZUiA7na6mLGy9rb21d/K zRcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to; bh=xO7cBE8cBWu1B92WjR321qulp5km92I/UH1Ne4+NMuU=; b=VbXxG28DLJMmtPgVWDX/HGqGnLiBKgVJbKgZ6ChEyqQWBTUlJV2MvtZC+1cQanST4t FZ9CfvvJLX7ugBwKxYq2tFAwqVDeFdDP8DG+ILEiB0aFBO3pAV/Pd1SJqUDLb+QQQacE A/VtJ2PdrhH7+45E1d353A3NGmz3d/LcqXrV7303f9G7Fruouyqek2eq7amC/rmV4phQ La/SphmQ0OFqvD7WciTX4sw+5lUctHvJqQVoe2del1JBmVoea7URsX6B+U5w+9Aud8FW Y3jHQ7VNPA8Yg6RB7Xjm21BD6JXj0TIc6DwdzU/psSG/GoRyiAhB4FTi6Nh6JUYMRJr+ WYpA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x16-20020a05640226d000b0045a5bd66302si4240777edd.571.2022.11.05.13.17.06; Sat, 05 Nov 2022 13:17:06 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 19DFD68B296; Sat, 5 Nov 2022 22:16:44 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [217.70.183.196]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B790568B244 for ; Sat, 5 Nov 2022 22:16:37 +0200 (EET) Received: (Authenticated sender: michael@niedermayer.cc) by mail.gandi.net (Postfix) with ESMTPSA id E9F67E0008 for ; Sat, 5 Nov 2022 20:16:36 +0000 (UTC) From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Sat, 5 Nov 2022 21:16:29 +0100 Message-Id: <20221105201629.1980-4-michael@niedermayer.cc> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221105201629.1980-1-michael@niedermayer.cc> References: <20221105201629.1980-1-michael@niedermayer.cc> Subject: [FFmpeg-devel] [PATCH 4/4] avcodec/bonk: Check for undefined overflow in predictor_calc_error() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: PGiYIurYs1Xk Fixes: signed integer overflow: 22 * -2107998208 cannot be represented in type 'int' Fixes: 51363/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_BONK_fuzzer-5660734784143360 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/bonk.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/libavcodec/bonk.c b/libavcodec/bonk.c index 1695229dbd..40963aa7c6 100644 --- a/libavcodec/bonk.c +++ b/libavcodec/bonk.c @@ -278,10 +278,13 @@ static int predictor_calc_error(int *k, int *state, int order, int error) *state_ptr = &(state[order-2]); for (i = order-2; i >= 0; i--, k_ptr--, state_ptr--) { - int k_value = *k_ptr, state_value = *state_ptr; + int64_t k_value = *k_ptr, state_value = *state_ptr; x -= shift_down(k_value * state_value, LATTICE_SHIFT); - state_ptr[1] = state_value + shift_down(k_value * x, LATTICE_SHIFT); + k_value *= x; + if ((int32_t)k_value != k_value) + return AVERROR_INVALIDDATA; + state_ptr[1] = state_value + shift_down(k_value, LATTICE_SHIFT); } // don't drift too far, to avoid overflows @@ -366,6 +369,8 @@ static int bonk_decode(AVCodecContext *avctx, AVFrame *frame, int64_t t64; for (int j = 0; j < s->down_sampling - 1; j++) { sample[0] = predictor_calc_error(s->k, state, s->n_taps, 0); + if (sample[0] == AVERROR_INVALIDDATA) + return sample[0]; sample++; } @@ -374,6 +379,8 @@ static int bonk_decode(AVCodecContext *avctx, AVFrame *frame, return AVERROR_INVALIDDATA; sample[0] = predictor_calc_error(s->k, state, s->n_taps, t64); + if (sample[0] == AVERROR_INVALIDDATA) + return sample[0]; sample++; }