From patchwork Sun Oct 15 00:49:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leo Izen X-Patchwork-Id: 44263 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4b15:b0:15d:8365:d4b8 with SMTP id fp21csp1820294pzb; Sat, 14 Oct 2023 17:49:49 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHSeXIbiYdJi9bzC0h7qiz9i28mm6ZFGz3PQyVRD+S8jVRwfH4fNBiPEYcfrCHfGcjhWts/ X-Received: by 2002:a17:907:970a:b0:9be:e153:3f24 with SMTP id jg10-20020a170907970a00b009bee1533f24mr2411374ejc.63.1697330989389; Sat, 14 Oct 2023 17:49:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697330989; cv=none; d=google.com; s=arc-20160816; b=efbynFxUicPMa24SNyVC0iUCQtLXT/oKmmdwVOmD0RozCvF8RDTy/xIq3WVOOkUnkC 8xfemuKnj3cyRJKHK76m1EOBGdQTnGT/Fn/b58t5XDp9B++k/QPdtQobpqeWLHPbssnT c2c5zW2rdIq4rbizmi6qzv7ALfr27jKilX+/QXg6H5Al/70PSyWhzP+4omI2z36ZFYOJ 8JaUgfVx+wb5WwiWiDtKSJrqUJ2nSKqZbqJhUAqyyRPpsc/9MZsfJMgRaejdO+xnSbFx LdHaO1Oo+l2DPb553+Sq0OZnisXRPbFZW6pEFtOuCUpZzwgcF6dNfzZPwsSfHQlfwVLp 5NAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=gXY1gDMzHbANY9qfdDCdAVPrsJdKEfJJN9N/eNLrW6E=; fh=VDN5+TNMvI4QcA450Y5zHrxzCeKSduY8ULDi31AwbZ8=; b=eLB136mgSFeymCyRfOUZSsv7jeS1AL5TfE2/XcIZyGvb+h3YShJWmvtSWaKlpmiYIE AvN4xaMy9Ds/QuP38fOqrgk407cyws1kMmA02NBPd2Aqol1Shz2G0Gqv6bi+P2MfVBCO wZhJsAQMU1ZnwYalyvt91HhPe5O45/OQAnChhS5Wy69JN4VrpS8WvoIEuPVyNFO4uHsc KfmXe/N5CfRZQUqtIk7OR5qQmKIzzQZAQrb8plHjZMQ6LIIZ9iXwZyQRGKOHQU9SPwVY 2Qo1Q29g/akphTEKiPNMor0r6VKJRW3KBAULcjW5LPjOXB7Jsq62vGI05taA6qs3v7Bm nC2Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=B8et2WDt; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id lr13-20020a170906fb8d00b0099bd73cbd7dsi1218793ejb.429.2023.10.14.17.49.48; Sat, 14 Oct 2023 17:49:49 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=B8et2WDt; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B951D68C734; Sun, 15 Oct 2023 03:49:45 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f51.google.com (mail-qv1-f51.google.com [209.85.219.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1CC5468ADCB for ; Sun, 15 Oct 2023 03:49:39 +0300 (EEST) Received: by mail-qv1-f51.google.com with SMTP id 6a1803df08f44-66d31e8d198so3501756d6.1 for ; Sat, 14 Oct 2023 17:49:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697330977; x=1697935777; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8a9f1q7SfC+IXA/qGeWrXQC10ygNtzOjydFihc0jZ+U=; b=B8et2WDtvIC9VLVu0Wsz8H3PmfI4rfD7H4rIVszki64/05gh9BRfEtgkhsSI4I8jre Obm0yAmAc5It+XXLMKynQ0mNMc+JAybNJKJiuMFtM4bkOzOYCGJ02EDrSNJWy0rhnfQP 6kpah7eRhfjrMXAeUKfL2jDS1jpf0DDRP+lQfGNGMZa4s6/hOe2n973T3jwwaj3VaSoT J1I7omJvJIBjt/T48Fz012ddeV85sqAcV9W3Mfz0phk4G4c7FCRmtrzFnocvoL/tvx6l NBBoVWiGIa9lW6P7/ULuEzp08yRWTJyySBLwmWBJRCapiQ0OlIwm5tRk1EKZLYIgmdDE yt7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697330977; x=1697935777; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8a9f1q7SfC+IXA/qGeWrXQC10ygNtzOjydFihc0jZ+U=; b=vh/KQRgwDmQgQgFWh3rcxRK4QaToLUMdsBiHmljendt9lguujY1z6jKKVEOZzccyG8 6nKhLPIBr+VfqjYBhjRq/SOYo5gE+vrr1r/fUE4iHJid5ntBWT2ERYYeXI+/XVujXkC6 WZJ1hym1xytwK4TttlMzfDdKsHzYb/3cA6GIOUjRZHAFcImcUIeqX67ar3huHMKgwjVN qteIwayunBgSLyG6Ann6sR00Xwyk5cpL8o0hMPV3/FdiQonCnqjJO5JcVasPjB5/HnEs uQ2EFftFWW/PUv9PuYhVodMMSWnhJpdrUFgsJrnhLDwtBZn8GbavBOPYLSqYfHt4G377 bhwg== X-Gm-Message-State: AOJu0Yy97n1txBQWlA23qg1UVYiAy+jOwwSowIGc91i/IbpO85vAcY3h 0EACPn6zTnjLH2alWOND1KEpeboMvkKY24cF X-Received: by 2002:a0c:ef90:0:b0:66d:1b9b:1964 with SMTP id w16-20020a0cef90000000b0066d1b9b1964mr8260422qvr.2.1697330976959; Sat, 14 Oct 2023 17:49:36 -0700 (PDT) Received: from gauss.local (c-68-56-149-176.hsd1.mi.comcast.net. [68.56.149.176]) by smtp.gmail.com with ESMTPSA id i17-20020ad44bb1000000b0065b22afe53csm2126872qvw.94.2023.10.14.17.49.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 14 Oct 2023 17:49:36 -0700 (PDT) From: Leo Izen To: ffmpeg-devel@ffmpeg.org Date: Sat, 14 Oct 2023 20:49:24 -0400 Message-ID: <20231015004924.597746-1-leo.izen@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231013014959.536776-1-leo.izen@gmail.com> References: <20231013014959.536776-1-leo.izen@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2] avcodec/jpegxl_parser: fix OOB read regression X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Cole Dilorenzo , Leo Izen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: qND0o1HXGcXJ In f7ac3512f5b5cb8eb149f37300b43461d8e93af3 the size of the dynamically allocated buffer was shrunk, but it was made too small for very small alphabet sizes. This patch restores the size to prevent an OOB read. Reported-by: Cole Dilorenzo Signed-off-by: Leo Izen --- libavcodec/jpegxl_parser.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/libavcodec/jpegxl_parser.c b/libavcodec/jpegxl_parser.c index dde36b0d6e..630fc8a60b 100644 --- a/libavcodec/jpegxl_parser.c +++ b/libavcodec/jpegxl_parser.c @@ -683,7 +683,7 @@ static int read_vlc_prefix(GetBitContext *gb, JXLEntropyDecoder *dec, JXLSymbolD int repeat_count_prev = 0, repeat_count_zero = 0, prev = 8; int total_code = 0, len, hskip, num_codes = 0, ret; - VLC level1_vlc; + VLC level1_vlc = { 0 }; if (dist->alphabet_size == 1) { dist->vlc.bits = 0; @@ -709,8 +709,10 @@ static int read_vlc_prefix(GetBitContext *gb, JXLEntropyDecoder *dec, JXLSymbolD } } - if (total_code != 32 && num_codes >= 2 || num_codes < 1) - return AVERROR_INVALIDDATA; + if (total_code != 32 && num_codes >= 2 || num_codes < 1) { + ret = AVERROR_INVALIDDATA; + goto end; + } for (int i = 1; i < 19; i++) level1_codecounts[i] += level1_codecounts[i - 1]; @@ -726,7 +728,7 @@ static int read_vlc_prefix(GetBitContext *gb, JXLEntropyDecoder *dec, JXLSymbolD if (ret < 0) goto end; - buf = av_mallocz(dist->alphabet_size * (2 * sizeof(int8_t) + sizeof(int16_t) + sizeof(uint32_t)) + buf = av_mallocz(MAX_PREFIX_ALPHABET_SIZE * (2 * sizeof(int8_t) + sizeof(int16_t) + sizeof(uint32_t)) + sizeof(uint32_t)); if (!buf) { ret = AVERROR(ENOMEM); @@ -734,21 +736,22 @@ static int read_vlc_prefix(GetBitContext *gb, JXLEntropyDecoder *dec, JXLSymbolD } level2_lens = (int8_t *)buf; - level2_lens_s = (int8_t *)(buf + dist->alphabet_size * sizeof(int8_t)); - level2_syms = (int16_t *)(buf + dist->alphabet_size * (2 * sizeof(int8_t))); - level2_codecounts = (uint32_t *)(buf + dist->alphabet_size * (2 * sizeof(int8_t) + sizeof(int16_t))); + level2_lens_s = (int8_t *)(buf + MAX_PREFIX_ALPHABET_SIZE * sizeof(int8_t)); + level2_syms = (int16_t *)(buf + MAX_PREFIX_ALPHABET_SIZE * (2 * sizeof(int8_t))); + level2_codecounts = (uint32_t *)(buf + MAX_PREFIX_ALPHABET_SIZE * (2 * sizeof(int8_t) + sizeof(int16_t))); total_code = 0; for (int i = 0; i < dist->alphabet_size; i++) { len = get_vlc2(gb, level1_vlc.table, 5, 1); + if (get_bits_left(gb) < 0) { + ret = AVERROR_BUFFER_TOO_SMALL; + goto end; + } if (len == 16) { int extra = 3 + get_bits(gb, 2); if (repeat_count_prev) - extra = 4 * (repeat_count_prev - 2) - repeat_count_prev + extra; - if (i + extra > dist->alphabet_size) { - ret = AVERROR_INVALIDDATA; - goto end; - } + extra += 4 * (repeat_count_prev - 2) - repeat_count_prev; + extra = FFMIN(extra, dist->alphabet_size - i); for (int j = 0; j < extra; j++) level2_lens[i + j] = prev; total_code += (32768 >> prev) * extra; @@ -759,7 +762,8 @@ static int read_vlc_prefix(GetBitContext *gb, JXLEntropyDecoder *dec, JXLSymbolD } else if (len == 17) { int extra = 3 + get_bits(gb, 3); if (repeat_count_zero > 0) - extra = 8 * (repeat_count_zero - 2) - repeat_count_zero + extra; + extra += 8 * (repeat_count_zero - 2) - repeat_count_zero; + extra = FFMIN(extra, dist->alphabet_size - i); i += extra - 1; repeat_count_prev = 0; repeat_count_zero += extra;