From patchwork Tue Dec 27 23:17:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?Q2zDqW1lbnQgQsWTc2No?= X-Patchwork-Id: 39795 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:bc95:b0:ad:ade2:bfd2 with SMTP id fx21csp3762823pzb; Tue, 27 Dec 2022 15:22:40 -0800 (PST) X-Google-Smtp-Source: AMrXdXtvaBkU9BJbBdAGxpq2kw350Yq7iKuUVYEcHsJGB/nlfZhgzw92UMEIB5hoYOyBW2Q5tHGD X-Received: by 2002:a17:906:6693:b0:7c1:8ba7:3182 with SMTP id z19-20020a170906669300b007c18ba73182mr24129861ejo.69.1672183360302; Tue, 27 Dec 2022 15:22:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1672183360; cv=none; d=google.com; s=arc-20160816; b=fguN6RQEQHLmp1Hqk9NAGXuJHMGyBt7uCPU/MDXFv6sjYpjdJ3lqXiYf83ZaeE++7z 70nUG7wJSqgoSG66yQb763fxsm10/+uP3KZAvuDxJ+e6oWZVHWp01Kq4rymetgwfiUns 7Fit3Wh1Lhgn7q8TueTk/HpFoomE+M4iKWNP1WEgPegu6bx0q5+VLX9kHUqf29nkjJoy X8Y0582UzHAvnwZudCsIdNHx1PhJXgzP+SjNlysrPzuqWhtJz0/Fhp9oA/ujOXPHCZi9 Ni8uMjCFeVla2iPmhuDVJWWIkxdRroytj9VmtM3b6qOlhLwhKmR/h459w0Ru3oJ3NY7p G4KQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=rar6+Kx829HYvY367a5c2TsMgWjZp8nWgiFcoYcuGrs=; b=D2gdEcnWLzCYuYEs4jSxGgvWmmRGB+UkuFyIj+y6SmM/yWQ5QPxoY9tbhzN2rbiANs eewMWgzal0iSwu/zRoR8XLiVuJdWgArCEnQfZVr6XpUCqCfcyHmPp32LhMJ/IugO53Lx q0d32oYe3b7XE749dgyMhnv3QX6wNlaTIQUgvOi9PXaiYu42bM0UOegJezuS8yqui6O9 FntgjKHV78oFStqWdGwQUUWlJZd50pgaMrv9onvte4cFcAaN9z/FSaDWZqCkmjuj1YyG gLj8IsPBA3EELvQzXhivLaeLWpenJoG/92jfC6mTBl9NVj6P0nQyyVNixNcMhxBcWE67 9ZAA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@pkh.me header.s=selector1 header.b=o01wV9PM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=pkh.me Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id sh39-20020a1709076ea700b00782ff2649a7si11293239ejc.346.2022.12.27.15.22.39; Tue, 27 Dec 2022 15:22:40 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@pkh.me header.s=selector1 header.b=o01wV9PM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=pkh.me Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id ECBC668BC9F; Wed, 28 Dec 2022 01:18:57 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from ssq0.pkh.me (laubervilliers-656-1-228-164.w92-154.abo.wanadoo.fr [92.154.28.164]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B0DF368BCAA for ; Wed, 28 Dec 2022 01:18:31 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pkh.me; s=selector1; t=1672183097; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Gw8qnUlaZAyt0LgcetFYrBzl10bG++umd1kmFuGXSY4=; b=o01wV9PMMxOc3Dwlu9SV7BTEXIxXjm0zB9daD9MXjJBrGCuTwwyzNQJYI97SMcrjvZ7/lu Nr62G/VQmCIXiTbh/U4kpoOkwswtdkukHuoY3P9U5S6R+4DdahY6P7ImTxZh1VfOWAtZaD 8KZp2fp9ca70oTDmLI+pykodhLshPfI= Received: from localhost (ssq0.pkh.me [local]) by ssq0.pkh.me (OpenSMTPD) with ESMTPA id b095be1a; Tue, 27 Dec 2022 23:18:17 +0000 (UTC) From: =?utf-8?b?Q2zDqW1lbnQgQsWTc2No?= To: ffmpeg-devel@ffmpeg.org Date: Wed, 28 Dec 2022 00:17:49 +0100 Message-Id: <20221227231814.2520181-8-u@pkh.me> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221227231814.2520181-1-u@pkh.me> References: <20221105152617.1809282-1-u@pkh.me> <20221227231814.2520181-1-u@pkh.me> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 07/32] avfilter/paletteuse: remove unused alpha split dimension X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: =?utf-8?b?Q2zDqW1lbnQgQsWTc2No?= Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 1neZ0rBA2bzZ The equalities in the w{r,g,b} range checks make sure longest is never 0. Even if the alpha ended up being selected in get_next_color() it would cause underread memory accesses in its caller (colormap_insert). --- libavfilter/vf_paletteuse.c | 33 ++++++++++++++++----------------- 1 file changed, 16 insertions(+), 17 deletions(-) diff --git a/libavfilter/vf_paletteuse.c b/libavfilter/vf_paletteuse.c index f43f077454..8954a02524 100644 --- a/libavfilter/vf_paletteuse.c +++ b/libavfilter/vf_paletteuse.c @@ -211,7 +211,7 @@ static void colormap_nearest_node(const struct color_node *map, struct nearest_color *nearest) { const struct color_node *kd = map + node_pos; - const int shift = (3 - kd->split) * 8; + const int shift = (2 - kd->split) * 8; int dx, nearer_kd_id, further_kd_id; const uint32_t current = kd->val; const int current_to_target = diff(target, current, trans_thresh); @@ -270,7 +270,7 @@ static av_always_inline uint8_t colormap_nearest_iterative(const struct color_no /* Check if it's not a leaf */ if (kd->left_id != -1 || kd->right_id != -1) { - const int shift = (3 - kd->split) * 8; + const int shift = (2 - kd->split) * 8; const int dx = (target>>shift & 0xff) - (current>>shift & 0xff); int nearer_kd_id, further_kd_id; @@ -497,7 +497,7 @@ static void disp_node(AVBPrint *buf, const uint32_t fontcolor = (node->val>>16 & 0xff) > 0x50 && (node->val>> 8 & 0xff) > 0x50 && (node->val & 0xff) > 0x50 ? 0 : 0xffffff; - const int rgb_comp = node->split - 1; + const int rgb_comp = node->split; av_bprintf(buf, "%*cnode%d [" "label=\"%c%02X%c%02X%c%02X%c\" " "fillcolor=\"#%06"PRIX32"\" " @@ -588,16 +588,15 @@ static int cmp_##name(const void *pa, const void *pb) \ { \ const struct color *a = pa; \ const struct color *b = pb; \ - return (int)(a->value >> (8 * (3 - (pos))) & 0xff) \ - - (int)(b->value >> (8 * (3 - (pos))) & 0xff); \ + return (int)(a->value >> (8 * (2 - (pos))) & 0xff) \ + - (int)(b->value >> (8 * (2 - (pos))) & 0xff); \ } -DECLARE_CMP_FUNC(a, 0) -DECLARE_CMP_FUNC(r, 1) -DECLARE_CMP_FUNC(g, 2) -DECLARE_CMP_FUNC(b, 3) +DECLARE_CMP_FUNC(r, 0) +DECLARE_CMP_FUNC(g, 1) +DECLARE_CMP_FUNC(b, 2) -static const cmp_func cmp_funcs[] = {cmp_a, cmp_r, cmp_g, cmp_b}; +static const cmp_func cmp_funcs[] = {cmp_r, cmp_g, cmp_b}; static int get_next_color(const uint8_t *color_used, const uint32_t *palette, const int trans_thresh, @@ -650,9 +649,9 @@ static int get_next_color(const uint8_t *color_used, const uint32_t *palette, wr = ranges.max[0] - ranges.min[0]; wg = ranges.max[1] - ranges.min[1]; wb = ranges.max[2] - ranges.min[2]; - if (wr >= wg && wr >= wb) longest = 1; - if (wg >= wr && wg >= wb) longest = 2; - if (wb >= wr && wb >= wg) longest = 3; + if (wr >= wg && wr >= wb) longest = 0; + if (wg >= wr && wg >= wb) longest = 1; + if (wb >= wr && wb >= wg) longest = 2; cmpf = cmp_funcs[longest]; *component = longest; @@ -692,13 +691,13 @@ static int colormap_insert(struct color_node *map, /* get the two boxes this node creates */ box1 = box2 = *box; - comp_value = node->val >> ((3 - component) * 8) & 0xff; - box1.max[component-1] = comp_value; - box2.min[component-1] = FFMIN(comp_value + 1, 255); + comp_value = node->val >> ((2 - component) * 8) & 0xff; + box1.max[component] = comp_value; + box2.min[component] = FFMIN(comp_value + 1, 255); node_left_id = colormap_insert(map, color_used, nb_used, palette, trans_thresh, &box1); - if (box2.min[component-1] <= box2.max[component-1]) + if (box2.min[component] <= box2.max[component]) node_right_id = colormap_insert(map, color_used, nb_used, palette, trans_thresh, &box2); node->left_id = node_left_id;