From patchwork Sat Nov 5 15:26:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?Q2zDqW1lbnQgQsWTc2No?= X-Patchwork-Id: 39170 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp1085421pzb; Sat, 5 Nov 2022 08:27:38 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7QdFLSjZdsHcB7nHGPYja7J9TwsQrincyjyhRgYN5Nfn0lGbMzGGUNDqE7/NWGBors2Gch X-Received: by 2002:a17:907:c0e:b0:7ae:1134:37c5 with SMTP id ga14-20020a1709070c0e00b007ae113437c5mr15319472ejc.648.1667662058417; Sat, 05 Nov 2022 08:27:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667662058; cv=none; d=google.com; s=arc-20160816; b=Af5ThCuNQPj2lUFIkctxHE3FLN8e1x8PB+B4ZXM7U51FykveWdXCi2b8v2UnjZmMo2 4xC/6QZ75k9HJIekNk3KuZXX1x/dqCqF1+eZsn/ypkhoInz6+9nbpbMuXTJld+IA5oxT JuQKAI7/mVSWAu/ARjhRPUK1QRWUzlNbMAu+clBR0IKaNjYWWg4gX1iIkesl4D47DgNM 9Rb4oYCd2PM5zrBt4UUrKV6Gt4dBF216de5LvBtYvAj9x0y5msIVkjOpTC9Pd3hdv5fs TdvdI/Tto8+HTJ/dngNXB5anoNjg5Hzm2qf1BZ5b/qgIM0tGQa/3CblrwItVJyjyZIJq KY9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=iaUkVoeFlYLUbRuHPZmJWY3abbzBx0f1DJ7t6YhUL8A=; b=iJdYuMpDhH14oh8RFHa3im3887dmyUYdqJsggBmL2OTdLOZTvyAft/ZoO51b6WnmQm YrW26sqfJgNcdFFejMNqSPTu0UMbxqVVfChsazmqeGQp41djLMOW4/TSXs0iwSM5Ipa3 jVAEIlkaHyZswE7RBQx2GWYtlw5uAiOWoN4gU67EmwylRXg9lqJkuBXtfOm6gRE3vFCB DhfzHzrkNTXUitGoPCN7sFU/P+OZ8b1kimfTA42jr9ECecEL7AX12NPI45bIZ5OJlnGA mRm2feF1eoioxWVxZjQnkcefkgLzxWMBI1BxdYV7Tq0TKqlARbY7zaXNWU1tT2hq5ZHL Me9Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@pkh.me header.s=selector1 header.b=RBsuO6y0; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=pkh.me Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hb13-20020a170907160d00b007707f83375bsi2798261ejc.945.2022.11.05.08.27.38; Sat, 05 Nov 2022 08:27:38 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@pkh.me header.s=selector1 header.b=RBsuO6y0; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=pkh.me Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BD86868B59B; Sat, 5 Nov 2022 17:26:42 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from ssq0.pkh.me (laubervilliers-656-1-228-164.w92-154.abo.wanadoo.fr [92.154.28.164]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DBEFE68B32D for ; Sat, 5 Nov 2022 17:26:35 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pkh.me; s=selector1; t=1667661981; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D4PDdhEnXAEyPvTekJZOpB54XHjADLiSKJctDu5YJys=; b=RBsuO6y08qO9FKjwRVoOaIBgfyIym4227q4RXHOhn6XMTFNq/bSqANqBNJTxwFOfKjdkPz yWWbFn2F9607cnMiDuPo2FMRXsQRw7yldVyGTxT5WnSaMwAzCN7E8AsVh6rCTyMAoVGH0p /xT5jtQ9EKeF0Sb3sIZjF7QKoiZP6jE= Received: from localhost (ssq0.pkh.me [local]) by ssq0.pkh.me (OpenSMTPD) with ESMTPA id bcb4c9a5; Sat, 5 Nov 2022 15:26:21 +0000 (UTC) From: =?utf-8?b?Q2zDqW1lbnQgQsWTc2No?= To: ffmpeg-devel@ffmpeg.org Date: Sat, 5 Nov 2022 16:26:08 +0100 Message-Id: <20221105152617.1809282-7-u@pkh.me> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221105152617.1809282-1-u@pkh.me> References: <20221105152617.1809282-1-u@pkh.me> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 06/15] avfilter/paletteuse: remove unused alpha split dimension X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: =?utf-8?b?Q2zDqW1lbnQgQsWTc2No?= Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: jeg3f6DIxUx8 The equalities in the w{r,g,b} range checks make sure longest is never 0. Even if the alpha ended up being selected in get_next_color() it would cause underread memory accesses in its caller (colormap_insert). --- libavfilter/vf_paletteuse.c | 33 ++++++++++++++++----------------- 1 file changed, 16 insertions(+), 17 deletions(-) diff --git a/libavfilter/vf_paletteuse.c b/libavfilter/vf_paletteuse.c index f43f077454..8954a02524 100644 --- a/libavfilter/vf_paletteuse.c +++ b/libavfilter/vf_paletteuse.c @@ -211,7 +211,7 @@ static void colormap_nearest_node(const struct color_node *map, struct nearest_color *nearest) { const struct color_node *kd = map + node_pos; - const int shift = (3 - kd->split) * 8; + const int shift = (2 - kd->split) * 8; int dx, nearer_kd_id, further_kd_id; const uint32_t current = kd->val; const int current_to_target = diff(target, current, trans_thresh); @@ -270,7 +270,7 @@ static av_always_inline uint8_t colormap_nearest_iterative(const struct color_no /* Check if it's not a leaf */ if (kd->left_id != -1 || kd->right_id != -1) { - const int shift = (3 - kd->split) * 8; + const int shift = (2 - kd->split) * 8; const int dx = (target>>shift & 0xff) - (current>>shift & 0xff); int nearer_kd_id, further_kd_id; @@ -497,7 +497,7 @@ static void disp_node(AVBPrint *buf, const uint32_t fontcolor = (node->val>>16 & 0xff) > 0x50 && (node->val>> 8 & 0xff) > 0x50 && (node->val & 0xff) > 0x50 ? 0 : 0xffffff; - const int rgb_comp = node->split - 1; + const int rgb_comp = node->split; av_bprintf(buf, "%*cnode%d [" "label=\"%c%02X%c%02X%c%02X%c\" " "fillcolor=\"#%06"PRIX32"\" " @@ -588,16 +588,15 @@ static int cmp_##name(const void *pa, const void *pb) \ { \ const struct color *a = pa; \ const struct color *b = pb; \ - return (int)(a->value >> (8 * (3 - (pos))) & 0xff) \ - - (int)(b->value >> (8 * (3 - (pos))) & 0xff); \ + return (int)(a->value >> (8 * (2 - (pos))) & 0xff) \ + - (int)(b->value >> (8 * (2 - (pos))) & 0xff); \ } -DECLARE_CMP_FUNC(a, 0) -DECLARE_CMP_FUNC(r, 1) -DECLARE_CMP_FUNC(g, 2) -DECLARE_CMP_FUNC(b, 3) +DECLARE_CMP_FUNC(r, 0) +DECLARE_CMP_FUNC(g, 1) +DECLARE_CMP_FUNC(b, 2) -static const cmp_func cmp_funcs[] = {cmp_a, cmp_r, cmp_g, cmp_b}; +static const cmp_func cmp_funcs[] = {cmp_r, cmp_g, cmp_b}; static int get_next_color(const uint8_t *color_used, const uint32_t *palette, const int trans_thresh, @@ -650,9 +649,9 @@ static int get_next_color(const uint8_t *color_used, const uint32_t *palette, wr = ranges.max[0] - ranges.min[0]; wg = ranges.max[1] - ranges.min[1]; wb = ranges.max[2] - ranges.min[2]; - if (wr >= wg && wr >= wb) longest = 1; - if (wg >= wr && wg >= wb) longest = 2; - if (wb >= wr && wb >= wg) longest = 3; + if (wr >= wg && wr >= wb) longest = 0; + if (wg >= wr && wg >= wb) longest = 1; + if (wb >= wr && wb >= wg) longest = 2; cmpf = cmp_funcs[longest]; *component = longest; @@ -692,13 +691,13 @@ static int colormap_insert(struct color_node *map, /* get the two boxes this node creates */ box1 = box2 = *box; - comp_value = node->val >> ((3 - component) * 8) & 0xff; - box1.max[component-1] = comp_value; - box2.min[component-1] = FFMIN(comp_value + 1, 255); + comp_value = node->val >> ((2 - component) * 8) & 0xff; + box1.max[component] = comp_value; + box2.min[component] = FFMIN(comp_value + 1, 255); node_left_id = colormap_insert(map, color_used, nb_used, palette, trans_thresh, &box1); - if (box2.min[component-1] <= box2.max[component-1]) + if (box2.min[component] <= box2.max[component]) node_right_id = colormap_insert(map, color_used, nb_used, palette, trans_thresh, &box2); node->left_id = node_left_id;