From patchwork Fri Oct 6 20:59:55 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Roche X-Patchwork-Id: 5447 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.90 with SMTP id m26csp25707jah; Fri, 6 Oct 2017 14:08:27 -0700 (PDT) X-Received: by 10.28.178.81 with SMTP id b78mr1049061wmf.157.1507324107726; Fri, 06 Oct 2017 14:08:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1507324107; cv=none; d=google.com; s=arc-20160816; b=rn53rLYvnl0zYn/9z4jQvsMj6n43IMotsPC6c4CciQDGlPBirN1nPkwODXPUnreyfL Ze2alsIiS3R/hnudRvCMloEfxE3RYrjTZytLnlPsmpwOASpevlanw6PBLV/O8x+4In4f 6zqeIkDfa8z0v9OqU8tlzGWv6/34/wUnNv/CGNhx/2ijGsvQv4mgxGjSqSEmToRYJQ5t qarEFTDlt5TQvfej3g+JCkFf2t5e7o9t9gC3qwY3b+n1fRktKaf/ocW9EWYSB+iBqxHf cbSOF2MWo3ip0MCTEPxdZQzBz7n8d+dWtiO50uuNY6vByfB6NywNju2sPfWM7oo3cNu0 ZFXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=DW83r1t+z7i6CE/RzhoxwBhLNqCa9BnrlOj90sGDe3I=; b=dM2uXNUFp9fsoCEWqE0xYP1UPNMYnGcPT7U3q7N73R7u5IQWa2uoHgr4c+xBqUAkQa 43zWFcOH/f78Ujp33RYWAQGnKo06PPmWPFpHrks7JVitTaC0EENKMDliMtAtHn7ulGOP zq267gtjfRciLs30MEUu3xVlFMmSx4mm7Uqp17K/BQk9u6Yal+y38TnG7gc5fqn2BnsI cQZcwJdi4rKiqfXEtwjBeNrZ86vC2ykRLaA4m/wj8ziOtx2zw9xf/2QTGKB324hQ/wM6 AMSQZQh2bMRRKDV5FMy30S0jrx+XtzUvnCfQLIOjhHfWeevHBIehSr1ekiVrB489CCp/ Lfnw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@giphy.com header.s=google header.b=kwfrdlna; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i1si1961790wmf.240.2017.10.06.14.08.26; Fri, 06 Oct 2017 14:08:27 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@giphy.com header.s=google header.b=kwfrdlna; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 497AE68A534; Sat, 7 Oct 2017 00:08:25 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk0-f178.google.com (mail-qk0-f178.google.com [209.85.220.178]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DC2D068A4DC for ; Sat, 7 Oct 2017 00:08:18 +0300 (EEST) Received: by mail-qk0-f178.google.com with SMTP id r64so18884251qkc.1 for ; Fri, 06 Oct 2017 14:08:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=giphy.com; s=google; h=from:to:cc:subject:date:message-id; bh=0tYbq6yLPHCVv18p/osME3aKVOBjX+tQxE1G+JfvJh8=; b=kwfrdlnauDYkC/in/dbbNIGbej/mlI6x2j5engik3AJWJDoTqyE/B5s1A0oC3uj6h0 HXI4qxmDOy9LYLHWon9+Nosg7s52nawsmXLGuIgM1TFr0oXAHxKAWyOntIIujfJP3tBp TVgGG5JF7L1nyxM4TSpfANXExZ3SMS/HMZIzA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=0tYbq6yLPHCVv18p/osME3aKVOBjX+tQxE1G+JfvJh8=; b=mq82bEDW3brYRTZKtPXmN8m3rw273rY0vHaztScKblb1o8hPeuGGPJ3nn8gx7LaslI 3Mu5ipCf5gkC9QnGM/5Rwk0szpPFcjxZzfSJM0e5U8YL8JqrJ2nRJ/Bkow6qslZvV+Ei RBE6e2B89XihfYyW0VlUAgWb4f7M0AZmnbD/Hq3RN6abtH5a3TtyioKD4uxNS0nLcvJi Qfo+15aNxirqbRqAUnHy7sGzfH+y9OY4Hl5AmrOg8/s3OFItIsP7C6j7mpxIZP0HBl34 54xnuyus12rDQhdWevRL9qq5gtUnh1a1qmtHSXsXnNAZ3FnCETZBVcutysBWwQOSRKmu 4xUw== X-Gm-Message-State: AMCzsaX4UOOrtJRJKQrrGYmjC5+YR6py84KvpkGI86q1Rhz/3gN8djQs SALkd2LswJenfk3f983CjPdrlDn4MJA= X-Google-Smtp-Source: AOwi7QA8pPtDvpBYQBhAUrB2MqcqdJKM7d9aAAqdlHvvOAxxHFmI4M179BNaU0Cj452fJ/L+VThQSQ== X-Received: by 10.55.40.229 with SMTP id o98mr180188qko.73.1507323609879; Fri, 06 Oct 2017 14:00:09 -0700 (PDT) Received: from localhost.localdomain ([208.184.100.82]) by smtp.gmail.com with ESMTPSA id m31sm1512610qtf.55.2017.10.06.14.00.09 (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 06 Oct 2017 14:00:09 -0700 (PDT) From: Bjorn Roche To: ffmpeg-devel@ffmpeg.org Date: Fri, 6 Oct 2017 16:59:55 -0400 Message-Id: <20171006205955.93052-1-bjorn@giphy.com> X-Mailer: git-send-email 2.14.1 Subject: [FFmpeg-devel] [PATCH] Fix for paletteuse to support transparency X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Bjorn Roche MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavfilter/vf_paletteuse.c | 175 ++++++++++++++++++++++++++++---------------- 1 file changed, 112 insertions(+), 63 deletions(-) diff --git a/libavfilter/vf_paletteuse.c b/libavfilter/vf_paletteuse.c index ffd37bf1da..4203543843 100644 --- a/libavfilter/vf_paletteuse.c +++ b/libavfilter/vf_paletteuse.c @@ -56,7 +56,7 @@ enum diff_mode { }; struct color_node { - uint8_t val[3]; + uint8_t val[4]; uint8_t palette_id; int split; int left_id, right_id; @@ -86,6 +86,7 @@ typedef struct PaletteUseContext { struct cache_node cache[CACHE_SIZE]; /* lookup cache */ struct color_node map[AVPALETTE_COUNT]; /* 3D-Tree (KD-Tree with K=3) for reverse colormap */ uint32_t palette[AVPALETTE_COUNT]; + int transparency_index; /* index in the palette of transparency. -1 if there isn't a transparency. */ int palette_loaded; int dither; int new; @@ -108,6 +109,7 @@ typedef struct PaletteUseContext { #define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM static const AVOption paletteuse_options[] = { { "dither", "select dithering mode", OFFSET(dither), AV_OPT_TYPE_INT, {.i64=DITHERING_SIERRA2_4A}, 0, NB_DITHERING-1, FLAGS, "dithering_mode" }, + { "none", "no dither", 0, AV_OPT_TYPE_CONST, {.i64=DITHERING_NONE}, INT_MIN, INT_MAX, FLAGS, "dithering_mode" }, { "bayer", "ordered 8x8 bayer dithering (deterministic)", 0, AV_OPT_TYPE_CONST, {.i64=DITHERING_BAYER}, INT_MIN, INT_MAX, FLAGS, "dithering_mode" }, { "heckbert", "dithering as defined by Paul Heckbert in 1982 (simple error diffusion)", 0, AV_OPT_TYPE_CONST, {.i64=DITHERING_HECKBERT}, INT_MIN, INT_MAX, FLAGS, "dithering_mode" }, { "floyd_steinberg", "Floyd and Steingberg dithering (error diffusion)", 0, AV_OPT_TYPE_CONST, {.i64=DITHERING_FLOYD_STEINBERG}, INT_MIN, INT_MAX, FLAGS, "dithering_mode" }, @@ -157,7 +159,8 @@ static int query_formats(AVFilterContext *ctx) static av_always_inline int dither_color(uint32_t px, int er, int eg, int eb, int scale, int shift) { - return av_clip_uint8((px >> 16 & 0xff) + ((er * scale) / (1<> 24 & 0xff) ) << 24 + | av_clip_uint8((px >> 16 & 0xff) + ((er * scale) / (1<> 8 & 0xff) + ((eg * scale) / (1<>24 & 0xff, palette[i]>>16 & 0xff, palette[i]>> 8 & 0xff, palette[i] & 0xff, }; - const int d = diff(palrgb, rgb); + const int d = diff(palargb, rgb); if (d < min_dist) { pal_id = i; min_dist = d; } } } + return pal_id; } @@ -325,14 +338,15 @@ end: * Note: r, g, and b are the component of c but are passed as well to avoid * recomputing them (they are generally computed by the caller for other uses). */ -static av_always_inline int color_get(struct cache_node *cache, uint32_t color, - uint8_t r, uint8_t g, uint8_t b, +static av_always_inline int color_get(struct cache_node *cache, uint32_t argb, + uint8_t a, uint8_t r, uint8_t g, uint8_t b, + int transparency_index, const struct color_node *map, const uint32_t *palette, const enum color_search_method search_method) { int i; - const uint8_t rgb[] = {r, g, b}; + const uint8_t argb_elts[] = {a, r, g, b}; const uint8_t rhash = r & ((1<= 0) { + return transparency_index; + } + for (i = 0; i < node->nb_entries; i++) { e = &node->entries[i]; - if (e->color == color) + if (e->color == argb) return e->pal_entry; } @@ -350,21 +369,24 @@ static av_always_inline int color_get(struct cache_node *cache, uint32_t color, sizeof(*node->entries), NULL); if (!e) return AVERROR(ENOMEM); - e->color = color; - e->pal_entry = COLORMAP_NEAREST(search_method, palette, map, rgb); + e->color = argb; + e->pal_entry = COLORMAP_NEAREST(search_method, palette, map, argb_elts); + return e->pal_entry; } static av_always_inline int get_dst_color_err(struct cache_node *cache, - uint32_t c, const struct color_node *map, + uint32_t argb, const struct color_node *map, const uint32_t *palette, + int transparency_index, int *er, int *eg, int *eb, const enum color_search_method search_method) { - const uint8_t r = c >> 16 & 0xff; - const uint8_t g = c >> 8 & 0xff; - const uint8_t b = c & 0xff; - const int dstx = color_get(cache, c, r, g, b, map, palette, search_method); + const uint8_t a = argb >> 24 & 0xff; + const uint8_t r = argb >> 16 & 0xff; + const uint8_t g = argb >> 8 & 0xff; + const uint8_t b = argb & 0xff; + const int dstx = color_get(cache, argb, a, r, g, b, transparency_index, map, palette, search_method); const uint32_t dstc = palette[dstx]; *er = r - (dstc >> 16 & 0xff); *eg = g - (dstc >> 8 & 0xff); @@ -385,6 +407,7 @@ static av_always_inline int set_frame(PaletteUseContext *s, AVFrame *out, AVFram const int dst_linesize = out->linesize[0]; uint32_t *src = ((uint32_t *)in ->data[0]) + y_start*src_linesize; uint8_t *dst = out->data[0] + y_start*dst_linesize; + int transparency_index = s->transparency_index; w += x_start; h += y_start; @@ -395,14 +418,14 @@ static av_always_inline int set_frame(PaletteUseContext *s, AVFrame *out, AVFram if (dither == DITHERING_BAYER) { const int d = s->ordered_dither[(y & 7)<<3 | (x & 7)]; + const uint8_t a8 = src[x] >> 24 & 0xff; const uint8_t r8 = src[x] >> 16 & 0xff; const uint8_t g8 = src[x] >> 8 & 0xff; const uint8_t b8 = src[x] & 0xff; const uint8_t r = av_clip_uint8(r8 + d); const uint8_t g = av_clip_uint8(g8 + d); const uint8_t b = av_clip_uint8(b8 + d); - const uint32_t c = r<<16 | g<<8 | b; - const int color = color_get(cache, c, r, g, b, map, palette, search_method); + const int color = color_get(cache, src[x], a8, r, g, b, transparency_index, map, palette, search_method); if (color < 0) return color; @@ -410,7 +433,7 @@ static av_always_inline int set_frame(PaletteUseContext *s, AVFrame *out, AVFram } else if (dither == DITHERING_HECKBERT) { const int right = x < w - 1, down = y < h - 1; - const int color = get_dst_color_err(cache, src[x], map, palette, &er, &eg, &eb, search_method); + const int color = get_dst_color_err(cache, src[x], map, palette, transparency_index, &er, &eg, &eb, search_method); if (color < 0) return color; @@ -422,7 +445,7 @@ static av_always_inline int set_frame(PaletteUseContext *s, AVFrame *out, AVFram } else if (dither == DITHERING_FLOYD_STEINBERG) { const int right = x < w - 1, down = y < h - 1, left = x > x_start; - const int color = get_dst_color_err(cache, src[x], map, palette, &er, &eg, &eb, search_method); + const int color = get_dst_color_err(cache, src[x], map, palette, transparency_index, &er, &eg, &eb, search_method); if (color < 0) return color; @@ -436,7 +459,7 @@ static av_always_inline int set_frame(PaletteUseContext *s, AVFrame *out, AVFram } else if (dither == DITHERING_SIERRA2) { const int right = x < w - 1, down = y < h - 1, left = x > x_start; const int right2 = x < w - 2, left2 = x > x_start + 1; - const int color = get_dst_color_err(cache, src[x], map, palette, &er, &eg, &eb, search_method); + const int color = get_dst_color_err(cache, src[x], map, palette, transparency_index, &er, &eg, &eb, search_method); if (color < 0) return color; @@ -455,7 +478,7 @@ static av_always_inline int set_frame(PaletteUseContext *s, AVFrame *out, AVFram } else if (dither == DITHERING_SIERRA2_4A) { const int right = x < w - 1, down = y < h - 1, left = x > x_start; - const int color = get_dst_color_err(cache, src[x], map, palette, &er, &eg, &eb, search_method); + const int color = get_dst_color_err(cache, src[x], map, palette, transparency_index, &er, &eg, &eb, search_method); if (color < 0) return color; @@ -466,10 +489,11 @@ static av_always_inline int set_frame(PaletteUseContext *s, AVFrame *out, AVFram if ( down) src[src_linesize + x ] = dither_color(src[src_linesize + x ], er, eg, eb, 1, 2); } else { + const uint8_t a = src[x] >> 24 & 0xff; const uint8_t r = src[x] >> 16 & 0xff; const uint8_t g = src[x] >> 8 & 0xff; const uint8_t b = src[x] & 0xff; - const int color = color_get(cache, src[x] & 0xffffff, r, g, b, map, palette, search_method); + const int color = color_get(cache, src[x], a, r, g, b, transparency_index, map, palette, search_method); if (color < 0) return color; @@ -489,19 +513,19 @@ static void disp_node(AVBPrint *buf, int depth) { const struct color_node *node = &map[node_id]; - const uint32_t fontcolor = node->val[0] > 0x50 && - node->val[1] > 0x50 && - node->val[2] > 0x50 ? 0 : 0xffffff; + const uint32_t fontcolor = node->val[1] > 0x50 && + node->val[2] > 0x50 && + node->val[3] > 0x50 ? 0 : 0xffffff; av_bprintf(buf, "%*cnode%d [" "label=\"%c%02X%c%02X%c%02X%c\" " "fillcolor=\"#%02x%02x%02x\" " "fontcolor=\"#%06"PRIX32"\"]\n", depth*INDENT, ' ', node->palette_id, - "[ "[node->split], node->val[0], - "][ "[node->split], node->val[1], - " ]["[node->split], node->val[2], + "[ "[node->split], node->val[1], + "][ "[node->split], node->val[2], + " ]["[node->split], node->val[3], " ]"[node->split], - node->val[0], node->val[1], node->val[2], + node->val[1], node->val[2], node->val[3], fontcolor); if (parent_id != -1) av_bprintf(buf, "%*cnode%d -> node%d\n", depth*INDENT, ' ', @@ -584,17 +608,18 @@ static int cmp_##name(const void *pa, const void *pb) \ { \ const struct color *a = pa; \ const struct color *b = pb; \ - return (a->value >> (8 * (2 - (pos))) & 0xff) \ - - (b->value >> (8 * (2 - (pos))) & 0xff); \ + return (a->value >> (8 * (3 - (pos))) & 0xff) \ + - (b->value >> (8 * (3 - (pos))) & 0xff); \ } -DECLARE_CMP_FUNC(r, 0) -DECLARE_CMP_FUNC(g, 1) -DECLARE_CMP_FUNC(b, 2) +DECLARE_CMP_FUNC(a, 0) +DECLARE_CMP_FUNC(r, 1) +DECLARE_CMP_FUNC(g, 2) +DECLARE_CMP_FUNC(b, 3) -static const cmp_func cmp_funcs[] = {cmp_r, cmp_g, cmp_b}; +static const cmp_func cmp_funcs[] = {cmp_a, cmp_r, cmp_g, cmp_b}; -static int get_next_color(const uint8_t *color_used, const uint32_t *palette, +static int get_next_color(const uint8_t *color_used, const uint32_t *palette, const int nb_colors, int *component, const struct color_rect *box) { int wr, wg, wb; @@ -607,13 +632,18 @@ static int get_next_color(const uint8_t *color_used, const uint32_t *palette, ranges.min[0] = ranges.min[1] = ranges.min[2] = 0xff; ranges.max[0] = ranges.max[1] = ranges.max[2] = 0x00; - for (i = 0; i < AVPALETTE_COUNT; i++) { + for (i = 0; i < nb_colors; i++) { const uint32_t c = palette[i]; + const uint8_t a = c >> 24 & 0xff; const uint8_t r = c >> 16 & 0xff; const uint8_t g = c >> 8 & 0xff; const uint8_t b = c & 0xff; - if (color_used[i] || + if (a != 0xff) { + continue; + } + + if (color_used[i] || (a != 0xff) || r < box->min[0] || g < box->min[1] || b < box->min[2] || r > box->max[0] || g > box->max[1] || b > box->max[2]) continue; @@ -639,9 +669,9 @@ static int get_next_color(const uint8_t *color_used, const uint32_t *palette, wr = ranges.max[0] - ranges.min[0]; wg = ranges.max[1] - ranges.min[1]; wb = ranges.max[2] - ranges.min[2]; - if (wr >= wg && wr >= wb) longest = 0; - if (wg >= wr && wg >= wb) longest = 1; - if (wb >= wr && wb >= wg) longest = 2; + if (wr >= wg && wr >= wb) longest = 1; + if (wg >= wr && wg >= wb) longest = 2; + if (wb >= wr && wb >= wg) longest = 3; cmpf = cmp_funcs[longest]; *component = longest; @@ -655,6 +685,7 @@ static int colormap_insert(struct color_node *map, uint8_t *color_used, int *nb_used, const uint32_t *palette, + const int nb_colors, const struct color_rect *box) { uint32_t c; @@ -662,7 +693,7 @@ static int colormap_insert(struct color_node *map, int node_left_id = -1, node_right_id = -1; struct color_node *node; struct color_rect box1, box2; - const int pal_id = get_next_color(color_used, palette, &component, box); + const int pal_id = get_next_color(color_used, palette, nb_colors, &component, box); if (pal_id < 0) return -1; @@ -673,21 +704,22 @@ static int colormap_insert(struct color_node *map, node = &map[cur_id]; node->split = component; node->palette_id = pal_id; - node->val[0] = c>>16 & 0xff; - node->val[1] = c>> 8 & 0xff; - node->val[2] = c & 0xff; + node->val[0] = c>>24 & 0xff; + node->val[1] = c>>16 & 0xff; + node->val[2] = c>> 8 & 0xff; + node->val[3] = c & 0xff; color_used[pal_id] = 1; /* get the two boxes this node creates */ box1 = box2 = *box; - box1.max[component] = node->val[component]; - box2.min[component] = node->val[component] + 1; + box1.max[component-1] = node->val[component]; + box2.min[component-1] = node->val[component] + 1; - node_left_id = colormap_insert(map, color_used, nb_used, palette, &box1); + node_left_id = colormap_insert(map, color_used, nb_used, palette, nb_colors, &box1); - if (box2.min[component] <= box2.max[component]) - node_right_id = colormap_insert(map, color_used, nb_used, palette, &box2); + if (box2.min[component-1] <= box2.max[component-1]) + node_right_id = colormap_insert(map, color_used, nb_used, palette, nb_colors, &box2); node->left_id = node_left_id; node->right_id = node_right_id; @@ -702,7 +734,7 @@ static int cmp_pal_entry(const void *a, const void *b) return c1 - c2; } -static void load_colormap(PaletteUseContext *s) +static void load_colormap(PaletteUseContext *s, int nb_colors) { int i, nb_used = 0; uint8_t color_used[AVPALETTE_COUNT] = {0}; @@ -710,8 +742,18 @@ static void load_colormap(PaletteUseContext *s) struct color_rect box; /* disable transparent colors and dups */ - qsort(s->palette, AVPALETTE_COUNT, sizeof(*s->palette), cmp_pal_entry); - for (i = 0; i < AVPALETTE_COUNT; i++) { + qsort(s->palette, nb_colors, sizeof(*s->palette), cmp_pal_entry); + // update transparency index: + if (s->transparency_index >= 0) { + for (i = 0; i < nb_colors; i++) { + if ((s->palette[i]>>24 & 0xff) == 0) { + s->transparency_index = i; // we are assuming at most one transparent color + break; + } + } + } + + for (i = 0; i < nb_colors; i++) { const uint32_t c = s->palette[i]; if (i != 0 && c == last_color) { color_used[i] = 1; @@ -727,7 +769,7 @@ static void load_colormap(PaletteUseContext *s) box.min[0] = box.min[1] = box.min[2] = 0x00; box.max[0] = box.max[1] = box.max[2] = 0xff; - colormap_insert(s->map, color_used, &nb_used, s->palette, &box); + colormap_insert(s->map, color_used, &nb_used, s->palette, nb_colors, &box); if (s->dot_filename) disp_tree(s->map, s->dot_filename); @@ -937,10 +979,12 @@ static int config_input_palette(AVFilterLink *inlink) static void load_palette(PaletteUseContext *s, const AVFrame *palette_frame) { - int i, x, y; + int pal_index, i, x, y; const uint32_t *p = (const uint32_t *)palette_frame->data[0]; const int p_linesize = palette_frame->linesize[0] >> 2; + s->transparency_index = -1; + if (s->new) { memset(s->palette, 0, sizeof(s->palette)); memset(s->map, 0, sizeof(s->map)); @@ -949,14 +993,19 @@ static void load_palette(PaletteUseContext *s, const AVFrame *palette_frame) memset(s->cache, 0, sizeof(s->cache)); } - i = 0; + pal_index = 0; for (y = 0; y < palette_frame->height; y++) { - for (x = 0; x < palette_frame->width; x++) - s->palette[i++] = p[x]; + for (x = 0; x < palette_frame->width; x++) { + s->palette[pal_index] = p[x]; + if ((p[x]>>24 & 0xff) == 0) { + s->transparency_index = pal_index; // we are assuming at most one transparent color + } + ++pal_index; + } p += p_linesize; } - load_colormap(s); + load_colormap(s, pal_index); if (!s->new) s->palette_loaded = 1;