From patchwork Mon Oct 2 17:24:33 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bjorn Roche X-Patchwork-Id: 5380 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.61.8 with SMTP id n8csp1752324jan; Mon, 2 Oct 2017 10:25:08 -0700 (PDT) X-Received: by 10.223.185.74 with SMTP id b10mr1555329wrg.100.1506965107957; Mon, 02 Oct 2017 10:25:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1506965107; cv=none; d=google.com; s=arc-20160816; b=lwRHZmTwudp7+Pd5vQD4Ecr7AD8KDX9V+fiBpthcK/Wn+xd+BXD5i+PWQ9mH82oVIJ 34UJp445zyxYbCuOVynjw6/NXobWwoM7SOOq/7ZN+HfZrKJyBEzU2Lj5tYAQYiXA8YQD 5A7/b7oWIAp7AKcw0ISwK2EHh29R2g+T6F9rOsh29ewpmC71l4Key4PMD+ZTGMx9AVbT G80EWImhUEYVZcDeSOL6Q1lx7OuuwWhRSOz1/T7NmY00fyGMN/Y7JZi/M51T7wkX266I gTOGGX6pyvd+Smm92fnvdsyd0O+MfIDTQCCssJ6WFlr25euXggimmVyvKfAVG0IHRRZw vuCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to :arc-authentication-results; bh=SjFRrv/74/tHKSCnK2qGJUDMlYulXRImkSkVvDbZtXg=; b=ypC5EWIWFa2CPGy4XlKFC1JyZyRC1F2UVb3n+l5k5lS0tUFtcpWaCFpo02SC28C6yb uxjCS5FBnun4bP3lle506SRbgofl6NKVq5AJDLEImY1hE+n+eRbumPetamwEzFEG03RC dyW4iAP3QB3uwPIV5A0QYlUwHgDHN62NzJ7WjbglUpL24Q833JOTEfDW4wtIM9Ef8cZ8 aFGIjzbfRTrT3pJRmQMy6EjJjWOLaZ/srRPeYR2tbpzNb5vsdcVdBjqk6vcmiBWTsgae yVyDHZfuJ0TWmF+vinZzmrA+yu+/GfdEMogGt4a3a6WGeka94U0fpdcEFAp60dMmrk/v UYfw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@giphy.com header.s=google header.b=q9a/koZx; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id k10si9348266wrg.550.2017.10.02.10.25.07; Mon, 02 Oct 2017 10:25:07 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@giphy.com header.s=google header.b=q9a/koZx; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1FB65689ED3; Mon, 2 Oct 2017 20:24:52 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk0-f170.google.com (mail-qk0-f170.google.com [209.85.220.170]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 68024689EAE for ; Mon, 2 Oct 2017 20:24:45 +0300 (EEST) Received: by mail-qk0-f170.google.com with SMTP id o187so3690484qke.7 for ; Mon, 02 Oct 2017 10:25:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=giphy.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+yxdfUD5ugYs2sEjiKmfPUZufgwwYycte0AeOW4qyAE=; b=q9a/koZxtPXEogUaKDj1oT5Ouw7pVvSASoiwqROmb+H+Bt67cMFSQaFr2oOOU1JLDK XJyr4/zbZKvM28F0pvvcai3IJH8KeM4qFa5cjrNKFWRwRSknsVtHdtsvp9DaxQ9JGEaV SS40K/CBFa2P/W5mLJpm7AIX4R3kRclDlwKSQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+yxdfUD5ugYs2sEjiKmfPUZufgwwYycte0AeOW4qyAE=; b=jdPiLoJuXtBV0X1I8n/IIFi8hQ5g1c5v2LDV1AWGT0/GjscKqpzHwxMpugvULHZLZ5 OBx2csERBGhYp+lTe4opemWaP/A+R8HfEYgpQLazymYYMxBtNkaClDpa04kuTT/PJV3R ljqb2dK34S+cc1+mSIiIqHeoHOBr9d1/YGT+gddrjhsua+ZrE6L6Edznm1cKI15rHUjx jeboall0msm4AQDjH0hjV99xXHh9vPwZktPvHhm8BZgRzu2LdQBiVtsev5GTOzPOCpAH j9wH8KFEoROoI9xwNQEYXuDsE9JHvTp8tFGnmtPy1GlPto3Y8zGr9JzjiFfR+yCETLjw TrlA== X-Gm-Message-State: AMCzsaWPflavsEfrXQ4kmTaizHRwqjyWTMAf/gueX8rPkSHFpKOrgjan uhnnfRQJj58tz7NAWRahdDx/KAYWMQg= X-Google-Smtp-Source: AOwi7QD7C473sRBAnaa37qKtAfHHrw793o9C6v33ef6xDeAez+sva4EVoYZLsghBzNKosrAhrZ+wxQ== X-Received: by 10.233.244.72 with SMTP id z8mr15154968qkl.148.1506965098463; Mon, 02 Oct 2017 10:24:58 -0700 (PDT) Received: from localhost.localdomain ([208.184.100.82]) by smtp.gmail.com with ESMTPSA id p25sm7339001qte.28.2017.10.02.10.24.57 (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 02 Oct 2017 10:24:58 -0700 (PDT) From: Bjorn Roche To: ffmpeg-devel@ffmpeg.org Date: Mon, 2 Oct 2017 13:24:33 -0400 Message-Id: <20171002172439.65628-2-bjorn@giphy.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20171002172439.65628-1-bjorn@giphy.com> References: <20171002172439.65628-1-bjorn@giphy.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/7] First pass at fixing paletteuse X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Bjorn Roche Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Bjorn Roche Dithering doesn’t work, and only iterative and brute force color mapping work. --- libavfilter/vf_paletteuse.c | 265 +++++++++++++++++++++++++++++++------------- 1 file changed, 185 insertions(+), 80 deletions(-) diff --git a/libavfilter/vf_paletteuse.c b/libavfilter/vf_paletteuse.c index ffd37bf1da..e62503c8ea 100644 --- a/libavfilter/vf_paletteuse.c +++ b/libavfilter/vf_paletteuse.c @@ -56,7 +56,7 @@ enum diff_mode { }; struct color_node { - uint8_t val[3]; + uint8_t val[4]; uint8_t palette_id; int split; int left_id, right_id; @@ -86,6 +86,7 @@ typedef struct PaletteUseContext { struct cache_node cache[CACHE_SIZE]; /* lookup cache */ struct color_node map[AVPALETTE_COUNT]; /* 3D-Tree (KD-Tree with K=3) for reverse colormap */ uint32_t palette[AVPALETTE_COUNT]; + int transparency_index; /* index in the palette of transparency. -1 if there isn't a transparency. */ int palette_loaded; int dither; int new; @@ -162,29 +163,44 @@ static av_always_inline int dither_color(uint32_t px, int er, int eg, int eb, in | av_clip_uint8((px & 0xff) + ((eb * scale) / (1<>24 & 0xff, palette[i]>>16 & 0xff, palette[i]>> 8 & 0xff, palette[i] & 0xff, }; - const int d = diff(palrgb, rgb); + const int d = diffAlpha(palargb, rgb); if (d < min_dist) { pal_id = i; min_dist = d; @@ -195,6 +211,7 @@ static av_always_inline uint8_t colormap_nearest_bruteforce(const uint32_t *pale } /* Recursive form, simpler but a bit slower. Kept for reference. */ +/* may not work with transparency */ struct nearest_color { int node_pos; int dist_sqd; @@ -205,11 +222,12 @@ static void colormap_nearest_node(const struct color_node *map, const uint8_t *target, struct nearest_color *nearest) { + printf( "colormap_nearest_node\n" ); const struct color_node *kd = map + node_pos; - const int s = kd->split; + const int s = kd->split; //FIXME: spliut is one more than it was before int dx, nearer_kd_id, further_kd_id; const uint8_t *current = kd->val; - const int current_to_target = diff(target, current); + const int current_to_target = diffAlpha(target, current); if (current_to_target < nearest->dist_sqd) { nearest->node_pos = node_pos; @@ -232,6 +250,7 @@ static void colormap_nearest_node(const struct color_node *map, static av_always_inline uint8_t colormap_nearest_recursive(const struct color_node *node, const uint8_t *rgb) { + printf( "colormap_nearest_recursive\n" ); struct nearest_color res = {.dist_sqd = INT_MAX, .node_pos = -1}; colormap_nearest_node(node, 0, rgb, &res); return node[res.node_pos].palette_id; @@ -244,6 +263,7 @@ struct stack_node { static av_always_inline uint8_t colormap_nearest_iterative(const struct color_node *root, const uint8_t *target) { + printf( "colormap_nearest_iterative\n" ); int pos = 0, best_node_id = -1, best_dist = INT_MAX, cur_color_id = 0; struct stack_node nodes[16]; struct stack_node *node = &nodes[0]; @@ -252,21 +272,26 @@ static av_always_inline uint8_t colormap_nearest_iterative(const struct color_no const struct color_node *kd = &root[cur_color_id]; const uint8_t *current = kd->val; - const int current_to_target = diff(target, current); + const int current_to_target = diffAlpha(target, current); + + printf( "%d-%d-%d-%d, %d, %d-%d-%d-%d\n", target[0], target[1], target[2], target[3], current_to_target, current[0], current[1], current[2], current[3] ); /* Compare current color node to the target and update our best node if * it's actually better. */ if (current_to_target < best_dist) { best_node_id = cur_color_id; - if (!current_to_target) + if (!current_to_target) { + printf( "exact\n"); goto end; // exact match, we can return immediately + } best_dist = current_to_target; } /* Check if it's not a leaf */ if (kd->left_id != -1 || kd->right_id != -1) { const int split = kd->split; - const int dx = target[split] - current[split]; + printf( "split %d\n", split ); + const int dx = target[split-1] - current[split-1]; int nearer_kd_id, further_kd_id; /* Define which side is the most interesting. */ @@ -300,8 +325,10 @@ static av_always_inline uint8_t colormap_nearest_iterative(const struct color_no /* Unstack as much as we can, typically as long as the least probable * branch aren't actually probable. */ do { - if (--pos < 0) + if (--pos < 0) { + printf( "pos < 0\n"); goto end; + } node--; } while (node->dx2 >= best_dist); @@ -325,46 +352,62 @@ end: * Note: r, g, and b are the component of c but are passed as well to avoid * recomputing them (they are generally computed by the caller for other uses). */ -static av_always_inline int color_get(struct cache_node *cache, uint32_t color, - uint8_t r, uint8_t g, uint8_t b, +static av_always_inline int color_get(struct cache_node *cache, uint32_t argb, + uint8_t a, uint8_t r, uint8_t g, uint8_t b, + int transparency_index, const struct color_node *map, const uint32_t *palette, const enum color_search_method search_method) { - int i; - const uint8_t rgb[] = {r, g, b}; + printf("color_get\n"); + //int i; + const uint8_t argb_elts[] = {a, r, g, b}; const uint8_t rhash = r & ((1<nb_entries; i++) { - e = &node->entries[i]; - if (e->color == color) - return e->pal_entry; + // is this transparent? + printf( "a: %d, ti: %d\n", a, transparency_index ); + if( a == 0 && transparency_index >= 0 ) { + printf( "trans %d %d %d %d\n", a, r, g, b ); + return transparency_index; } + printf( "opaque %d %d %d %d\n", a, r, g, b ); + + // FIXME + // for (i = 0; i < node->nb_entries; i++) { + // e = &node->entries[i]; + // if (e->color == rgb) + // return e->pal_entry; + // } e = av_dynarray2_add((void**)&node->entries, &node->nb_entries, sizeof(*node->entries), NULL); if (!e) return AVERROR(ENOMEM); - e->color = color; - e->pal_entry = COLORMAP_NEAREST(search_method, palette, map, rgb); + e->color = argb; + // FIXME + e->pal_entry = COLORMAP_NEAREST(search_method, palette, map, argb_elts); return e->pal_entry; } static av_always_inline int get_dst_color_err(struct cache_node *cache, - uint32_t c, const struct color_node *map, + uint32_t argb, const struct color_node *map, const uint32_t *palette, + int transparency_index, int *er, int *eg, int *eb, const enum color_search_method search_method) { - const uint8_t r = c >> 16 & 0xff; - const uint8_t g = c >> 8 & 0xff; - const uint8_t b = c & 0xff; - const int dstx = color_get(cache, c, r, g, b, map, palette, search_method); + const uint8_t a = argb >> 24 & 0xff; + const uint8_t r = argb >> 16 & 0xff; + const uint8_t g = argb >> 8 & 0xff; + const uint8_t b = argb & 0xff; + printf( "get_dst_color_err %d %d %d %d %d\n", argb, a, r, g, b ); + const int dstx = color_get(cache, argb, a, r, g, b, transparency_index, map, palette, search_method); const uint32_t dstc = palette[dstx]; *er = r - (dstc >> 16 & 0xff); *eg = g - (dstc >> 8 & 0xff); @@ -378,6 +421,7 @@ static av_always_inline int set_frame(PaletteUseContext *s, AVFrame *out, AVFram const enum color_search_method search_method) { int x, y; + printf( "set_frame\n" ); const struct color_node *map = s->map; struct cache_node *cache = s->cache; const uint32_t *palette = s->palette; @@ -385,6 +429,7 @@ static av_always_inline int set_frame(PaletteUseContext *s, AVFrame *out, AVFram const int dst_linesize = out->linesize[0]; uint32_t *src = ((uint32_t *)in ->data[0]) + y_start*src_linesize; uint8_t *dst = out->data[0] + y_start*dst_linesize; + int transparency_index = s->transparency_index; w += x_start; h += y_start; @@ -393,24 +438,34 @@ static av_always_inline int set_frame(PaletteUseContext *s, AVFrame *out, AVFram for (x = x_start; x < w; x++) { int er, eg, eb; + // const uint8_t asss = src[x] >> 24 & 0xff; + // const uint8_t rsss = src[x] >> 16 & 0xff; + // const uint8_t gsss = src[x] >> 8 & 0xff; + // const uint8_t bsss = src[x] & 0xff; + + //printf( "converted %d: %d %d %d %d\n", src[x], asss, rsss, gsss, bsss ); + if (dither == DITHERING_BAYER) { + printf( "bayer\n" ); const int d = s->ordered_dither[(y & 7)<<3 | (x & 7)]; + const uint8_t a8 = src[x] >> 24 & 0xff; const uint8_t r8 = src[x] >> 16 & 0xff; const uint8_t g8 = src[x] >> 8 & 0xff; const uint8_t b8 = src[x] & 0xff; const uint8_t r = av_clip_uint8(r8 + d); const uint8_t g = av_clip_uint8(g8 + d); const uint8_t b = av_clip_uint8(b8 + d); - const uint32_t c = r<<16 | g<<8 | b; - const int color = color_get(cache, c, r, g, b, map, palette, search_method); + //const uint32_t argb = a8 << 24 | r<<16 | g<<8 | b; + const int color = color_get(cache, src[x], a8, r, g, b, transparency_index, map, palette, search_method); if (color < 0) return color; dst[x] = color; } else if (dither == DITHERING_HECKBERT) { + printf( "heckbert\n" ); const int right = x < w - 1, down = y < h - 1; - const int color = get_dst_color_err(cache, src[x], map, palette, &er, &eg, &eb, search_method); + const int color = get_dst_color_err(cache, src[x], map, palette, transparency_index, &er, &eg, &eb, search_method); if (color < 0) return color; @@ -421,8 +476,9 @@ static av_always_inline int set_frame(PaletteUseContext *s, AVFrame *out, AVFram if (right && down) src[src_linesize + x + 1] = dither_color(src[src_linesize + x + 1], er, eg, eb, 2, 3); } else if (dither == DITHERING_FLOYD_STEINBERG) { + printf( "fs\n" ); const int right = x < w - 1, down = y < h - 1, left = x > x_start; - const int color = get_dst_color_err(cache, src[x], map, palette, &er, &eg, &eb, search_method); + const int color = get_dst_color_err(cache, src[x], map, palette, transparency_index, &er, &eg, &eb, search_method); if (color < 0) return color; @@ -434,9 +490,10 @@ static av_always_inline int set_frame(PaletteUseContext *s, AVFrame *out, AVFram if (right && down) src[src_linesize + x + 1] = dither_color(src[src_linesize + x + 1], er, eg, eb, 1, 4); } else if (dither == DITHERING_SIERRA2) { + printf( "s2\n" ); const int right = x < w - 1, down = y < h - 1, left = x > x_start; const int right2 = x < w - 2, left2 = x > x_start + 1; - const int color = get_dst_color_err(cache, src[x], map, palette, &er, &eg, &eb, search_method); + const int color = get_dst_color_err(cache, src[x], map, palette, transparency_index, &er, &eg, &eb, search_method); if (color < 0) return color; @@ -454,8 +511,9 @@ static av_always_inline int set_frame(PaletteUseContext *s, AVFrame *out, AVFram } } else if (dither == DITHERING_SIERRA2_4A) { + printf( "s24a\n" ); const int right = x < w - 1, down = y < h - 1, left = x > x_start; - const int color = get_dst_color_err(cache, src[x], map, palette, &er, &eg, &eb, search_method); + const int color = get_dst_color_err(cache, src[x], map, palette, transparency_index, &er, &eg, &eb, search_method); if (color < 0) return color; @@ -466,10 +524,23 @@ static av_always_inline int set_frame(PaletteUseContext *s, AVFrame *out, AVFram if ( down) src[src_linesize + x ] = dither_color(src[src_linesize + x ], er, eg, eb, 1, 2); } else { + printf( "none\n" ); + const uint8_t a = src[x] >> 24 & 0xff; const uint8_t r = src[x] >> 16 & 0xff; const uint8_t g = src[x] >> 8 & 0xff; const uint8_t b = src[x] & 0xff; - const int color = color_get(cache, src[x] & 0xffffff, r, g, b, map, palette, search_method); + const int color = color_get(cache, src[x], a, r, g, b, transparency_index, map, palette, search_method); + + printf( "%3d,%3d,%3d,%3d -> %3d -> %3d,%3d,%3d,%3d\n", + src[x] >> 24 & 0xff, + src[x] >> 16 & 0xff, + src[x] >> 8 & 0xff, + src[x] >> 0 & 0xff, + color, + s->palette[color] >> 24 & 0xff, + s->palette[color] >> 16 & 0xff, + s->palette[color] >> 8 & 0xff, + s->palette[color] >> 0 & 0xff ); if (color < 0) return color; @@ -489,19 +560,19 @@ static void disp_node(AVBPrint *buf, int depth) { const struct color_node *node = &map[node_id]; - const uint32_t fontcolor = node->val[0] > 0x50 && - node->val[1] > 0x50 && - node->val[2] > 0x50 ? 0 : 0xffffff; + const uint32_t fontcolor = node->val[1] > 0x50 && + node->val[2] > 0x50 && + node->val[3] > 0x50 ? 0 : 0xffffff; av_bprintf(buf, "%*cnode%d [" "label=\"%c%02X%c%02X%c%02X%c\" " "fillcolor=\"#%02x%02x%02x\" " "fontcolor=\"#%06"PRIX32"\"]\n", depth*INDENT, ' ', node->palette_id, - "[ "[node->split], node->val[0], - "][ "[node->split], node->val[1], - " ]["[node->split], node->val[2], + "[ "[node->split], node->val[1], + "][ "[node->split], node->val[2], + " ]["[node->split], node->val[3], " ]"[node->split], - node->val[0], node->val[1], node->val[2], + node->val[1], node->val[2], node->val[3], fontcolor); if (parent_id != -1) av_bprintf(buf, "%*cnode%d -> node%d\n", depth*INDENT, ' ', @@ -552,8 +623,8 @@ static int debug_accuracy(const struct color_node *node, const uint32_t *palette const uint32_t c2 = palette[r2]; const uint8_t palrgb1[] = { c1>>16 & 0xff, c1>> 8 & 0xff, c1 & 0xff }; const uint8_t palrgb2[] = { c2>>16 & 0xff, c2>> 8 & 0xff, c2 & 0xff }; - const int d1 = diff(palrgb1, rgb); - const int d2 = diff(palrgb2, rgb); + const int d1 = diffAlpha(palrgb1, rgb); + const int d2 = diffAlpha(palrgb2, rgb); if (d1 != d2) { av_log(NULL, AV_LOG_ERROR, "/!\\ %02X%02X%02X: %d ! %d (%06"PRIX32" ! %06"PRIX32") / dist: %d ! %d\n", @@ -584,17 +655,18 @@ static int cmp_##name(const void *pa, const void *pb) \ { \ const struct color *a = pa; \ const struct color *b = pb; \ - return (a->value >> (8 * (2 - (pos))) & 0xff) \ - - (b->value >> (8 * (2 - (pos))) & 0xff); \ + return (a->value >> (8 * (3 - (pos))) & 0xff) \ + - (b->value >> (8 * (3 - (pos))) & 0xff); \ } -DECLARE_CMP_FUNC(r, 0) -DECLARE_CMP_FUNC(g, 1) -DECLARE_CMP_FUNC(b, 2) +DECLARE_CMP_FUNC(a, 0) +DECLARE_CMP_FUNC(r, 1) +DECLARE_CMP_FUNC(g, 2) +DECLARE_CMP_FUNC(b, 3) -static const cmp_func cmp_funcs[] = {cmp_r, cmp_g, cmp_b}; +static const cmp_func cmp_funcs[] = {cmp_a, cmp_r, cmp_g, cmp_b}; -static int get_next_color(const uint8_t *color_used, const uint32_t *palette, +static int get_next_color(const uint8_t *color_used, const uint32_t *palette, const int nb_colors, int *component, const struct color_rect *box) { int wr, wg, wb; @@ -607,7 +679,7 @@ static int get_next_color(const uint8_t *color_used, const uint32_t *palette, ranges.min[0] = ranges.min[1] = ranges.min[2] = 0xff; ranges.max[0] = ranges.max[1] = ranges.max[2] = 0x00; - for (i = 0; i < AVPALETTE_COUNT; i++) { + for (i = 0; i < nb_colors; i++) { const uint32_t c = palette[i]; const uint8_t r = c >> 16 & 0xff; const uint8_t g = c >> 8 & 0xff; @@ -639,9 +711,9 @@ static int get_next_color(const uint8_t *color_used, const uint32_t *palette, wr = ranges.max[0] - ranges.min[0]; wg = ranges.max[1] - ranges.min[1]; wb = ranges.max[2] - ranges.min[2]; - if (wr >= wg && wr >= wb) longest = 0; - if (wg >= wr && wg >= wb) longest = 1; - if (wb >= wr && wb >= wg) longest = 2; + if (wr >= wg && wr >= wb) longest = 1; + if (wg >= wr && wg >= wb) longest = 2; + if (wb >= wr && wb >= wg) longest = 3; cmpf = cmp_funcs[longest]; *component = longest; @@ -655,6 +727,7 @@ static int colormap_insert(struct color_node *map, uint8_t *color_used, int *nb_used, const uint32_t *palette, + const int nb_colors, const struct color_rect *box) { uint32_t c; @@ -662,7 +735,7 @@ static int colormap_insert(struct color_node *map, int node_left_id = -1, node_right_id = -1; struct color_node *node; struct color_rect box1, box2; - const int pal_id = get_next_color(color_used, palette, &component, box); + const int pal_id = get_next_color(color_used, palette, nb_colors, &component, box); if (pal_id < 0) return -1; @@ -673,21 +746,22 @@ static int colormap_insert(struct color_node *map, node = &map[cur_id]; node->split = component; node->palette_id = pal_id; - node->val[0] = c>>16 & 0xff; - node->val[1] = c>> 8 & 0xff; - node->val[2] = c & 0xff; + node->val[0] = c>>24 & 0xff; + node->val[1] = c>>16 & 0xff; + node->val[2] = c>> 8 & 0xff; + node->val[3] = c & 0xff; color_used[pal_id] = 1; /* get the two boxes this node creates */ box1 = box2 = *box; - box1.max[component] = node->val[component]; - box2.min[component] = node->val[component] + 1; + box1.max[component-1] = node->val[component-1]; + box2.min[component-1] = node->val[component-1] + 1; - node_left_id = colormap_insert(map, color_used, nb_used, palette, &box1); + node_left_id = colormap_insert(map, color_used, nb_used, palette, nb_colors, &box1); - if (box2.min[component] <= box2.max[component]) - node_right_id = colormap_insert(map, color_used, nb_used, palette, &box2); + if (box2.min[component-1] <= box2.max[component-1]) + node_right_id = colormap_insert(map, color_used, nb_used, palette, nb_colors, &box2); node->left_id = node_left_id; node->right_id = node_right_id; @@ -700,9 +774,10 @@ static int cmp_pal_entry(const void *a, const void *b) const int c1 = *(const uint32_t *)a & 0xffffff; const int c2 = *(const uint32_t *)b & 0xffffff; return c1 - c2; + // return *(const uint32_t *)a - *(const uint32_t *)b ; } -static void load_colormap(PaletteUseContext *s) +static void load_colormap(PaletteUseContext *s, int nb_colors) { int i, nb_used = 0; uint8_t color_used[AVPALETTE_COUNT] = {0}; @@ -710,24 +785,31 @@ static void load_colormap(PaletteUseContext *s) struct color_rect box; /* disable transparent colors and dups */ - qsort(s->palette, AVPALETTE_COUNT, sizeof(*s->palette), cmp_pal_entry); - for (i = 0; i < AVPALETTE_COUNT; i++) { + qsort(s->palette, nb_colors, sizeof(*s->palette), cmp_pal_entry); + for (i = 0; i < nb_colors; i++) { const uint32_t c = s->palette[i]; if (i != 0 && c == last_color) { color_used[i] = 1; continue; } last_color = c; - if ((c & 0xff000000) != 0xff000000) { - color_used[i] = 1; // ignore transparent color(s) - continue; - } + // if ((c & 0xff000000) == 0x00000000) { // ignore totally transparent + // color_used[i] = 1; // ignore transparent color(s) + // continue; + // } } box.min[0] = box.min[1] = box.min[2] = 0x00; box.max[0] = box.max[1] = box.max[2] = 0xff; - colormap_insert(s->map, color_used, &nb_used, s->palette, &box); + //printf( "-0- %d\n", nb_colors ); + + colormap_insert(s->map, color_used, &nb_used, s->palette, nb_colors, &box); + + for (i = 0; i < nb_used; i++) { + //printf( "-p-: %d\n", s->palette[i] ); + printf( "map: %3d %3d %3d %3d -> %3d\n", s->map[i].val[0], s->map[i].val[1], s->map[i].val[2], s->map[i].val[3], s->map[i].palette_id ); + } if (s->dot_filename) disp_tree(s->map, s->dot_filename); @@ -756,7 +838,7 @@ static void debug_mean_error(PaletteUseContext *s, const AVFrame *in1, const uint32_t c2 = palette[src2[x]]; const uint8_t rgb1[] = {c1 >> 16 & 0xff, c1 >> 8 & 0xff, c1 & 0xff}; const uint8_t rgb2[] = {c2 >> 16 & 0xff, c2 >> 8 & 0xff, c2 & 0xff}; - mean_err += diff(rgb1, rgb2); + mean_err += diffAlpha(rgb1, rgb2); } src1 += src1_linesize; src2 += src2_linesize; @@ -937,10 +1019,13 @@ static int config_input_palette(AVFilterLink *inlink) static void load_palette(PaletteUseContext *s, const AVFrame *palette_frame) { - int i, x, y; + int pal_index, i, has_transparency, x, y; const uint32_t *p = (const uint32_t *)palette_frame->data[0]; const int p_linesize = palette_frame->linesize[0] >> 2; + s->transparency_index = -1; + has_transparency = 0; + if (s->new) { memset(s->palette, 0, sizeof(s->palette)); memset(s->map, 0, sizeof(s->map)); @@ -949,14 +1034,34 @@ static void load_palette(PaletteUseContext *s, const AVFrame *palette_frame) memset(s->cache, 0, sizeof(s->cache)); } - i = 0; + pal_index = 0; for (y = 0; y < palette_frame->height; y++) { - for (x = 0; x < palette_frame->width; x++) - s->palette[i++] = p[x]; + for (x = 0; x < palette_frame->width; x++) { + if( (p[x]>>24 & 0xff) > 0 ) { + s->palette[pal_index] = p[x]; + ++pal_index; + } else { + has_transparency = 1; + printf( "ti set %d\n", pal_index ); + } + } p += p_linesize; } - load_colormap(s); + load_colormap(s, pal_index); + + if( has_transparency ) { + s->transparency_index = pal_index; + printf( "ti set\n" ); + } + printf( "%d\n", s->transparency_index ); + + + for (int j=0; jheight * palette_frame->width; ++j ) { + printf( "palette: %3i: %3d, %3d, %3d, %3d\n", j, ( s->palette[j] >> 24 ) & 0xff, ( s->palette[j] >> 16 ) & 0xff, ( s->palette[j] >> 8 ) & 0xff, ( s->palette[j] >> 0 ) & 0xff ); + } + + printf( "::palette loaded with transparency index :: %d\n", s->transparency_index ); if (!s->new) s->palette_loaded = 1;