From patchwork Thu Jun 8 17:18:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 42016 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c526:b0:117:ac03:c9de with SMTP id gm38csp564298pzb; Thu, 8 Jun 2023 10:18:40 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7Apmyaq4nGWYmeDSh7T2ByAa6ygv3pk1RSUp3KmjdybHL/WaswRMDnKsLOJVbpqeEwLt2d X-Received: by 2002:a05:6402:1616:b0:514:224b:a48c with SMTP id f22-20020a056402161600b00514224ba48cmr3353454edv.0.1686244720157; Thu, 08 Jun 2023 10:18:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686244720; cv=none; d=google.com; s=arc-20160816; b=t9vMvEWN1fPaK5w0i0oIPPGX8QmodSkxAWkbY0ICfHcoWSNrAXUyP9min4UygGMWxr mKX9WMQPoDEX3Y0VD0/TlzWuVc6bSMEq8RTNa42ZO5DjHCvprNEnUSNtSb8KImQE2fds H6eINR5KWp017CF1rrIRpRXVw/BIKTOjV7HcTm98MCDrMW6iHv2+iyhyMFIms+EvQfub D3gBwtSjHhJukSZkFnkLSOky2P0VsIB+7jDrhjsRl7uFgUAA51eLGR6gMHrMwGBrqqoW ylCAcSopHdtv6AqcxEXBYcWLPfPolVFGha0S9pmUAboLL+GGhq1mnM3yphYHcZf8Sv5x /IOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:delivered-to; bh=6kDwBQjAVbHGgqQEiwSgJwQGmj+rQ/NpNK3lWa9gIOs=; b=iBI+QAFY3534HlCHYZMVACxT6S81g2RZqgLVvfMOx4LJyPX7TySYPL52D6asdDv2On 90uPDbnQtmKa9JKbZ8NeSb012Wvn2oEDldnJWhfAf/bpebSScVmvQtpusR64xwOI355F uCXh3u7VBOYJUi4+GXcn4O77DlDEUATmdaC/a6r4PeDcLb9aR3HKcWuOjA32zZu/39f0 eljMduKnPN5UaIiaiMyHv/zieGPUSbBwaDHG8izLrD5EQFhmwegqJ0UpOw73jMLOi6xg lX0V3jsVKtI99cCPltMbxM8NMPmJiTuRgz1/aIrSBKc/mpQS9uQDZ9PO+l843NH5ORZS gIDQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id l6-20020aa7d946000000b00514b957c9dfsi1005320eds.571.2023.06.08.10.18.38; Thu, 08 Jun 2023 10:18:40 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id AFC42689D6F; Thu, 8 Jun 2023 20:18:34 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-01-1.mymagenta.at (mail-01-1.mymagenta.at [80.109.253.246]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 18D22689D6F for ; Thu, 8 Jun 2023 20:18:28 +0300 (EEST) Received: from [192.168.232.136] (helo=ren-mail-psmtp-mg02.) by mail-01.mymagenta.at with esmtp (Exim 4.93) (envelope-from ) id 1q7JHG-003tgd-EE for ffmpeg-devel@ffmpeg.org; Thu, 08 Jun 2023 19:18:26 +0200 Received: from localhost ([84.115.40.24]) by ren-mail-psmtp-mg02. with ESMTP id 7JHHqT0jobZLD7JHHqqQSa; Thu, 08 Jun 2023 19:18:27 +0200 X-Env-Mailfrom: michael@niedermayer.cc X-Env-Rcptto: ffmpeg-devel@ffmpeg.org X-SourceIP: 84.115.40.24 X-CNFS-Analysis: v=2.4 cv=Ufwy9IeN c=1 sm=1 tr=0 ts=64820d63 a=4thelYDX6rwh+ygQwvsI+Q==:117 a=4thelYDX6rwh+ygQwvsI+Q==:17 a=MKtGQD3n3ToA:10 a=1oJP67jkp3AA:10 a=GEAsPZ9sns4A:10 a=NEAV23lmAAAA:8 a=oW1aMXZbcGzkY8OzWQ0A:9 From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Thu, 8 Jun 2023 19:18:23 +0200 Message-Id: <20230608171825.6575-1-michael@niedermayer.cc> X-Mailer: git-send-email 2.17.1 X-CMAE-Envelope: MS4xfBuTGCQv6SrJKgXaiKifUTMah0QrL6FmLZS4OkmWSzKcWcvQQTDi2XzsDiPkko3PNhdwflK5zkrBwUXppJIsi9MPsXe1lOO3Lgp58b1s93FQy4+ygvYp yr3iRVxWdpLRrOABdKYTLSH6sJFXvwMKc/FCOXlafi++ErNYDAt3HjxIt8DkoA3tR3TX46x3hotULA== Subject: [FFmpeg-devel] [PATCH 1/3] avcodec/rka: use unsigned for buf0 additions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 8FV6pxdAAN4Z Fixes: signed integer overflow: -38912000 + -2109276160 cannot be represented in type 'int' Fixes: 59670/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RKA_fuzzer-4987563245699072 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/rka.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavcodec/rka.c b/libavcodec/rka.c index 76dca52602..d56f4faee4 100644 --- a/libavcodec/rka.c +++ b/libavcodec/rka.c @@ -735,10 +735,10 @@ static int decode_filter(RKAContext *s, ChContext *ctx, ACoder *ac, int off, uns ctx->buf1[off] = (val + (sum >> bits)) * (1U << bits) + (((1U << bits) - 1U) & ctx->buf1[off + -1]); } - ctx->buf0[off] = ctx->buf1[off] + ctx->buf0[off + -1]; + ctx->buf0[off] = ctx->buf1[off] + (unsigned)ctx->buf0[off + -1]; } else { val *= 1U << ctx->cmode; - sum += ctx->buf0[off + -1] + val; + sum += ctx->buf0[off + -1] + (unsigned)val; switch (s->bps) { case 16: sum = av_clip_int16(sum); break; case 8: sum = av_clip_int8(sum); break;