From patchwork Sun Oct 22 21:51:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Niedermayer X-Patchwork-Id: 44322 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1b28:b0:15d:8365:d4b8 with SMTP id ch40csp1069869pzb; Sun, 22 Oct 2023 14:51:45 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFKWg8q9Z1frebjvgbxm68zGeJir5cqoSKOoKcszlZpeSeG/PRKio5vNG04eYrlsu19V5TV X-Received: by 2002:a17:907:5cb:b0:9ae:4eb9:e09b with SMTP id wg11-20020a17090705cb00b009ae4eb9e09bmr6405231ejb.27.1698011505684; Sun, 22 Oct 2023 14:51:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698011505; cv=none; d=google.com; s=arc-20160816; b=RRcBPFHCfEFF3xP/L07/Z4wZflfja6UjsJK6CeCFLK80+RLTRpeEKxemokLixwrsvh f50N4tMenfQ7j0YKn0H2r73DUtndQNbiPvZSzUkLIFx9lwSIeHZDZYc4MlS9txFyfUE4 sKF14IGx9nGfgP3DQZi5tAjxqCyWew19kWcXE2s1Z33D23HCLfjyQ35Ol/m9eR3nSdQA 1Mb57PEDP44laWCEpn3S8Mi0C7F136iLthRyIHeFP4h+Nr9x5ssCFkbmuj9tpUVKYzJw tDRdLOeKqOxM++d7SxwkIQ6ltesbX2i2hFLGTI/E1rmStoHY8u8HHC93iRF+g6Q/fmK9 4eUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to; bh=5BY7mzyz7+FA5JHJfI3QJxs6nQYbwrAJm70BYpTmUHU=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=ZJ+fAT6DSxd1qGsQ8WK3OPTBQT0LE70tN6YFgwo6Y1JQV1wZES2S/NxITbgs9J+5p5 Bq7apyY/D9BQJEDJC+Ntw2rvrnSxL8XbIc9lW3u2sz2BbaW3mb27liiOXsPDS8jPFpbw F04slHKwx6n43vReBzFsv/rNTbybhTbAwksJSPzkh5ioDpgk/7y4SqSKgb1UlJdBX/J7 q0tKj8Iv5QmV5Mmmwut72FaclrfT+SIo/+qZpFFNmCjDuecmZAzQ7YFv4e31hiutTwuN wtS/5dVPh+Fffj3h8IiTYXAq4cnIsznJvgj8cux+2YCjOHNugh5GtlixxTywxas/8cxZ jdRg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gs4-20020a170906f18400b009b28c286b38si2631842ejb.789.2023.10.22.14.51.45; Sun, 22 Oct 2023 14:51:45 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A672B68CA22; Mon, 23 Oct 2023 00:51:24 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [217.70.183.198]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 91C9968CA20 for ; Mon, 23 Oct 2023 00:51:16 +0300 (EEST) Received: by mail.gandi.net (Postfix) with ESMTPSA id DEF33C0006 for ; Sun, 22 Oct 2023 21:51:15 +0000 (UTC) From: Michael Niedermayer To: FFmpeg development discussions and patches Date: Sun, 22 Oct 2023 23:51:07 +0200 Message-Id: <20231022215113.3469-3-michael@niedermayer.cc> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20231022215113.3469-1-michael@niedermayer.cc> References: <20231022215113.3469-1-michael@niedermayer.cc> X-GND-Sasl: michael@niedermayer.cc Subject: [FFmpeg-devel] [PATCH 3/9] avcodec/vlc: Skip subtable entries in multi VLC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Lh/OKUz6EHRC These entries do not correspond to VLC symbols that can be used they do corrupt various variables like min/max bits This also no longer assumes that there is a single non subtable entry Probably fixes some infinite loops too Signed-off-by: Michael Niedermayer --- libavcodec/vlc.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/libavcodec/vlc.c b/libavcodec/vlc.c index 77860430861..65883a506ff 100644 --- a/libavcodec/vlc.c +++ b/libavcodec/vlc.c @@ -401,15 +401,23 @@ static int vlc_multi_gen(VLC_MULTI_ELEM *table, const VLC *single, int minbits, maxbits, max = nb_codes-1; unsigned count[VLC_MULTI_MAX_SYMBOLS-1] = { 0, }; VLC_MULTI_ELEM info = { { 0, }, 0, 0, }; + int count0 = 0; - minbits = buf[0].bits; - maxbits = buf[0].bits; + for (int j = 0; j < 1<table[j].len > 0) { + count0 ++; + j += (1 << (numbits - single->table[j].len)) - 1; + } + } + + minbits = 32; + maxbits = 0; - for (int n = 1; n < nb_codes; n++) { + for (int n = nb_codes - count0; n < nb_codes; n++) { minbits = FFMIN(minbits, buf[n].bits); maxbits = FFMAX(maxbits, buf[n].bits); } - maxbits = FFMIN(maxbits, numbits); + av_assert0(maxbits <= numbits); while (max >= nb_codes/2) { if (buf[max].bits+minbits > maxbits)