From patchwork Tue Apr 9 09:11:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank Plowman X-Patchwork-Id: 47968 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9c8d:b0:1a7:a0dc:8de5 with SMTP id mj13csp229065pzb; Tue, 9 Apr 2024 02:11:51 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCX7WYuEer2zwsHABvUzTkt1/Jv+qDfSEQT3egWemfuai6EF8JG40kcbsxeG47iWK6Scb1LX15lz51tYpq1P+0+C/6+AtnAG501dKw== X-Google-Smtp-Source: AGHT+IGOyN9+qbSGNWAbuNtX9qFnLLvEeGXZS9zQmQYmUVtYrlP8n7sOwtbzXkOC8/zOWmwwAksF X-Received: by 2002:aa7:d046:0:b0:56e:7666:501 with SMTP id n6-20020aa7d046000000b0056e76660501mr490418edo.4.1712653911062; Tue, 09 Apr 2024 02:11:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712653911; cv=none; d=google.com; s=arc-20160816; b=IDurIJFqGDs4qzQ+ukkgX8K/jhGGy3AY96a0TzA1Y0S6c3MSYBZWYqn47ey4pgBHaF S9BshibO2fVBnEGCBLKED0q4XkB0cO9JSvBk62zxdRduQ2Hh7T77Uzrnfn7Bl01qsVS2 IKnxWCA2t4W6M0I3lIuNjdmzXLy0yzdGPQa8QG1pF1pBvWgJSgpolWjJay7iJOE3YuxW XHtpbqFrNPwgFauYGW7/P+6xZf6iABpE9zMHPm6meZw8ztPwAHbW7iPUWXKrtBaKmkW6 YdxQzrshjP5CeecvVMAtUPKx/7V5XWweDirSYRgXcPbkxnfgkbHGu0S5NjkckSavY/nZ l54A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:message-id:date:from :dkim-signature:delivered-to; bh=iv0rPyKBSR5CsQSJ3sUK7YG3bGle5xEXGltNGj5etxY=; fh=GABYHefZpsCOOVRn8a1IgmYuOvaCu1oFlwEOjvaxWTE=; b=ESOkrXLWG4aEYg1suvWwFofjlnap38lggDpJOX5nVKT7dhOHCqpM+ulml3urltBvq6 dQ8t/VQFbl7UNJM3Ob/c6XfJLKp8qJBuxIuJyhXXpw4Q3kePf+GuU5zi6UzDG43XbBK0 sHjC9P/7tm0YRE6KdwO+aGQbFdQp14fr7vYpBik4JYXjtKDmOfFm+UgQM5f5RiLGwnJi 6S7byj/oNptyJua9IVmUHdiQdzcDIdhuXdRtYD2wHemGfOMc/aO5BySJTjjMOUv9G+sX cX2gmtRzPqrMOb2Q1hpsQl5CHgfHH6QkwyYLfer/HjLlDnq/WvVWW+3Banpf363AzLry obsw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@frankplowman.com header.s=s1 header.b=IqcBMkVe; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b15-20020a50b40f000000b0056dfb2fe573si4685136edh.394.2024.04.09.02.11.50; Tue, 09 Apr 2024 02:11:51 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@frankplowman.com header.s=s1 header.b=IqcBMkVe; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A153E68D02A; Tue, 9 Apr 2024 12:11:47 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from wrqvffvh.outbound-mail.sendgrid.net (wrqvffvh.outbound-mail.sendgrid.net [149.72.255.128]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 01D4668CFEC for ; Tue, 9 Apr 2024 12:11:40 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=frankplowman.com; h=from:subject:mime-version:to:cc:content-transfer-encoding: content-type:cc:content-type:from:subject:to; s=s1; bh=W07hcFXWXytNJMQugfhew1Ja6age3GwB+5YCIU3RCXs=; b=IqcBMkVe2Qdth2XZcILQU4SqgC1PyacZ9Cg2sQUCxDCqBI4MT8LPSOAWKIe+c/r+31xi vkUjpvDQtyVc4RvCgiwyTiX66UGLtt+oRbxBFeC0A1dsiQYj+lwowIbr3GLCOGfCfV+xHV S5oWKIgURM8/wqt3JMBfVpmRy9Ofei4yaoyqGnNCGRpkSeMwBuOlVHw/+zRthUEW8OujM5 pAiHt2WB1hv3AOlA1H8s5GM0lEtY5Kc3whl52Jt5ml8uGFaH60zFfwzB2SOMs383aN/KFU T56+rsdEMEPpW65RuhaSdDZTR8t9jqhxPjT7off1t6vJJtaIDdsv0zKLMaMK4bPg== Received: by recvd-6fc499775c-sfjtq with SMTP id recvd-6fc499775c-sfjtq-1-6615064B-5 2024-04-09 09:11:39.428253824 +0000 UTC m=+1858355.942891699 Received: from localhost.localdomain (unknown) by geopod-ismtpd-11 (SG) with ESMTP id jrwB5JmvTfG2JzYOc5_f_A Tue, 09 Apr 2024 09:11:39.222 +0000 (UTC) From: Frank Plowman Date: Tue, 09 Apr 2024 09:11:39 +0000 (UTC) Message-ID: <20240409091134.57665-1-post@frankplowman.com> X-Mailer: git-send-email 2.44.0 MIME-Version: 1.0 X-SG-EID: u001.Z0KJCHpts8tvDq7PHgz5cpqJ+vJcSmdTtST/g91WT3qAbNxUpEMGIDQq91mXy3Yjvepp8fYsFUJzGsyhT1EwBbgeUjhjA1fv5eTi5z/n/e08mTcl6IukSC3G8jTfkCU4Ube8uWqNwDbcRdudyUiruFoSnxd1XrHNELhNwWO+ZWISMxqo8f4bPIX4BOJCzNmhkJpw1EU+kEveqk2ik3mMefs/KTZsTT2zB0J9kmxqcYuvLMSqulWPDTs5C37+Zw5l To: ffmpeg-devel@ffmpeg.org X-Entity-ID: u001.qzljkbu34TNIX4NwfTiKWA== Subject: [FFmpeg-devel] [PATCH] lavc/vvc: Fix out-of-bounds array access X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Frank Plowman Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: SyU/9ZoGZHWp The 2 which has been changed to an 8 in the array length expression is the maximum value of sps_bitdepth_minus8. This was missed when updating to VVCv2, which increased this maximum from 2 to 8. Signed-off-by: Frank Plowman --- libavcodec/vvc/intra.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/libavcodec/vvc/intra.c b/libavcodec/vvc/intra.c index 5ac7d02c80..def113239b 100644 --- a/libavcodec/vvc/intra.c +++ b/libavcodec/vvc/intra.c @@ -339,18 +339,22 @@ static void derive_qp(const VVCLocalContext *lc, const TransformUnit *tu, Transf //8.7.3 Scaling process for transform coefficients static av_always_inline int derive_scale(const TransformBlock *tb, const int sh_dep_quant_used_flag) { - static const uint8_t rem6[63 + 2 * 6 + 1] = { + static const uint8_t rem6[63 + 8 * 6 + 1] = { 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, - 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3 + 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, + 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, + 0, 1, 2, 3, }; - static const uint8_t div6[63 + 2 * 6 + 1] = { + static const uint8_t div6[63 + 8 * 6 + 1] = { 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8, 8, 9, 9, 9, 9, 9, 9, 10, 10, 10, 10, - 10, 10, 11, 11, 11, 11, 11, 11, 12, 12, 12, 12 + 10, 10, 11, 11, 11, 11, 11, 11, 12, 12, 12, 12, 12, 12, 13, 13, 13, 13, + 13, 13, 14, 14, 14, 14, 14, 14, 15, 15, 15, 15, 15, 15, 16, 16, 16, 16, + 16, 16, 17, 17, 17, 17, 17, 17, 18, 18, 18, 18, }; const static int level_scale[2][6] = {