From patchwork Fri Mar 29 19:32:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 47662 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9f96:b0:1a3:b6bb:3029 with SMTP id mm22csp2173983pzb; Fri, 29 Mar 2024 12:32:35 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXgGN45CRWXJ8Ms09hO1SptYguVVRMVV3tjolYeBMdZSiI8jt2RB7AOay2je18oWwFLp0CHIyYBWK0bMcY3a78+40jKQ8m7XJO8Ew== X-Google-Smtp-Source: AGHT+IGxYWyICqRS6PLiCWSPKF3E3wJhX+YRXQOseKMMAxfhjv2ZrI2wC/e+W5LXAhF6Xj6JLLhx X-Received: by 2002:a17:906:49a:b0:a4d:f85a:c458 with SMTP id f26-20020a170906049a00b00a4df85ac458mr2001168eja.4.1711740754993; Fri, 29 Mar 2024 12:32:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1711740754; cv=none; d=google.com; s=arc-20160816; b=J993dIWG7TBeDAmEaDB8PQInHDIDtcDuvt2cw6uhUeSMRaFz8/r4t04pBeloW0iaEg cUwIwMG53Nx34nEa15o9wDnzXFBbqSywnAVFnXMkZC8IfGRgwblOH0foIK4655WlleDk nl84bg8ds3I37PCE9TxCNXpaLGnstvZMxUhna9Ro6wsV3+acwLfbWVc/L3Xv4JsC1KWD sWN59RnfqU5p9MgleqNeSuX1yfmXY1sgde9Jni8ZMHLE3n/3CEdU+HcR9iJcKW8WlRKK gRGK7NdRLJpWA9KNEnurB3P+Qi3cnTop3mDWwf5ldkjIX2f2DGPZMIXnsXM5/6uF9aQQ aVaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to; bh=8hVJaqaazRKqaNwnfw2oT4B28CgSB+Ov6QwwZVywWcc=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=fFzuJYSMXESV8mkhEQv6jleFR7dfs43GtOOZOndaepG9c9hrH4sT9kCFXXYLgkbSjq GN19w0vH/bCVwli0JKingnlZp8jCEvj0M4TU8hLnugK0tS7M0361T5hb9wIgQ89/RZq/ d56ynCl7M7kzGEMAzfxAhwsn6v4d6LW60LYxdYPP7ZA5BMamvPqywQcMxgfrkTDjfFm7 AF8tW9n1Xbv1gfi7mDrrVRn6lbIG3n+S7DJ5BgLq8JkQeWaAdVgQHVLpi+XHohyD9Yhw 9jwhDHgyOyzJL5ByrUZ51mYy+NUTe9cafyZOOcY7rcnZUz9GbpxceDO4iuTBRPkbobm7 IAsA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=B9Msmq6m; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ca2-20020a170906a3c200b00a4633a910desi2057470ejb.867.2024.03.29.12.32.34; Fri, 29 Mar 2024 12:32:34 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@niedermayer.cc header.s=gm1 header.b=B9Msmq6m; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 29D4A68D21E; Fri, 29 Mar 2024 21:32:30 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay7-d.mail.gandi.net (relay7-d.mail.gandi.net [217.70.183.200]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1E0ED68D1FD for ; Fri, 29 Mar 2024 21:32:23 +0200 (EET) Received: by mail.gandi.net (Postfix) with ESMTPSA id 3258F20005 for ; Fri, 29 Mar 2024 19:32:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=niedermayer.cc; s=gm1; t=1711740742; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc; bh=Ey+NhAGFsuFXG4ZkrFBACPZXz24vv7m8y/1sK5TtJA0=; b=B9Msmq6mDfK8i0g6oYZOtp3Rz0optXqsSb8sADfPOwMu5M81YGkU/4i5hWJ6YlfagyAqe+ FtT102LyKCNdbi1/Ck8XUN+LLkODM4UtdpGz8dRmaLAAX61MQCTGS/2GGwdK4Um5RwK1Dy l/LS8Nn35VhDNS4osABu5EQwKW16oO/G8xv7oDdXdC7D6O9AXHaqiw7cgy8fUnIMyE2f2n 4PEl6WVsRw5D3mRHBWK69MmGiBUcNTzmELJVQdH6EqukO+8fxPIYHavm+DpjpJNiQalKf1 xsSXiwLiMsmR5EHG09wrFgOKAturWgNIulTLSDgb0/9ZuEDG+xd5E3NSKZjf/A== From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Fri, 29 Mar 2024 20:32:19 +0100 Message-Id: <20240329193221.11522-1-michael@niedermayer.cc> X-Mailer: git-send-email 2.17.1 X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 1/3] avcodec/jpeg2000htdec: Check magp before using it in a shift X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: IFAzbBsB7cTg Fixes: shift exponent -1 is negative Fixes: 65378/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-5457678193197056 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/jpeg2000dec.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/libavcodec/jpeg2000dec.c b/libavcodec/jpeg2000dec.c index 1afc6b1e2dd..fe2afb05057 100644 --- a/libavcodec/jpeg2000dec.c +++ b/libavcodec/jpeg2000dec.c @@ -1910,6 +1910,8 @@ static inline void tile_codeblocks(const Jpeg2000DecoderContext *s, Jpeg2000Tile int nb_precincts, precno; Jpeg2000Band *band = rlevel->band + bandno; int cblkno = 0, bandpos; + /* See Rec. ITU-T T.800, Equation E-2 */ + int magp = quantsty->expn[subbandno] + quantsty->nguardbits - 1; bandpos = bandno + (reslevelno > 0); @@ -1917,6 +1919,9 @@ static inline void tile_codeblocks(const Jpeg2000DecoderContext *s, Jpeg2000Tile band->coord[1][0] == band->coord[1][1]) continue; + if ((codsty->cblk_style & JPEG2000_CTSY_HTJ2K_F) && magp >= 31) + return; + nb_precincts = rlevel->num_precincts_x * rlevel->num_precincts_y; /* Loop on precincts */ for (precno = 0; precno < nb_precincts; precno++) { @@ -1927,8 +1932,6 @@ static inline void tile_codeblocks(const Jpeg2000DecoderContext *s, Jpeg2000Tile cblkno < prec->nb_codeblocks_width * prec->nb_codeblocks_height; cblkno++) { int x, y, ret; - /* See Rec. ITU-T T.800, Equation E-2 */ - int magp = quantsty->expn[subbandno] + quantsty->nguardbits - 1; Jpeg2000Cblk *cblk = prec->cblk + cblkno;