From patchwork Sun Apr 10 18:47:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 35246 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:671c:b0:7c:62c8:b2d1 with SMTP id q28csp696820pzh; Sun, 10 Apr 2022 11:45:07 -0700 (PDT) X-Google-Smtp-Source: ABdhPJylvbfFV8ftwuVK3g4+06XE5M+9+0rdlMl6YhbP7KsDhnOY0mFl8cCKilYvBm7snWwPcsOL X-Received: by 2002:a50:9e61:0:b0:419:d8e5:6236 with SMTP id z88-20020a509e61000000b00419d8e56236mr29495862ede.327.1649616307445; Sun, 10 Apr 2022 11:45:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1649616307; cv=none; d=google.com; s=arc-20160816; b=u07a4YXiTdwLl3Ok1TLNDCYakKUQaUhqFW4HYlyhuea7X/Qt9HvBG4RPgT5f62yXDw 2FSLdH0gP9RP3XzGVO1VnhglTCy7hpIW60UFnV7Kf5kDt1Q5rDIErctF1/oOkhWYpBNa ikDlgFgUoFMgrkLi+9Z1sPVikz1uQ4E/bfhmz+m8z17sWKIJIWZk22Jrzf1PzRQnjFN/ KWiATowk6YNZlsyArgueptbULCr5QgIiHGzZIzQpew0lRjE3UAHbx4cMENbqAnBMN/Nt 5YgWksAMcVF0pezZ+COQrO4Dvr3c7mMYOdPJ0TOBJiSJ4ht584Y8KNM/23CZKImMlZt3 AEtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=IZxPNnPBF4Ej1kXt36hNipm9axMsKm9l2XlkjS/oPAk=; b=vAloOczZT84w56leQF3JPZE0JG6vg97MMNMawZTHr96ZQiXXTc9v1S/Y0F6amUgHIv 0dHZSc9Q14lcS3dkN9/pi5gjN4l2ZrRs9CsXTNDqhazvJWrK4IzSwA56lqRsPM46K/rI MTvmwa813fwwudPWzzMHnwenflbfJi+4PlpyG3N/zblI0SgXVXGu+rT4DwVR0Zm4ChsE yl8whyH6qazuPuq7vLiM9GWdzHLpTPLfcupD+3+MtGoIz7M4rMZHiVh0DZ5EGIzntIxs 9vnO9BJBN58H7nIiL8BXFFmCQfzuSEuRXPGdpiGyXTOSxvN39x1qS/Exks6RluCW/HXs DD7w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=lqBmayeh; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id qt9-20020a170906ece900b006e84ce927dasi5331921ejb.831.2022.04.10.11.45.06; Sun, 10 Apr 2022 11:45:07 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=lqBmayeh; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A1FBF68B34D; Sun, 10 Apr 2022 21:45:02 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CD18668B25F for ; Sun, 10 Apr 2022 21:44:56 +0300 (EEST) Received: by mail-wr1-f47.google.com with SMTP id w4so19992115wrg.12 for ; Sun, 10 Apr 2022 11:44:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=JTmgYs+pLw35c/280j1zRlSjfhuTXIlpsp3begL639c=; b=lqBmayehJnX7oYIjaSRRyP0xrBFCOc3oWTqY4YsaSByLr600AT51rG0BIHbSURoNF3 21eASoLOLugN/HqDg+YGAMi82BAv9GlTrv2FqNtXeQceUGlmTD6ZMegajd87hQc1ymwt CTCOBnz4RhHQUBLl8tW7jMHa5c7ID3fnCZB5Dxa1kC3mY9DHbTQj78sz8Op63Vkw0zao l8XgIyZ7NuMLbO+R24lGM117itfPz3vcvqI1sukmrXm5YNDQDp2oqzKTFlp3S6nZb1mA HgBYnBmCkq5Ke6xYTZyqNXqGuoJvTr4CsagRYdPUCNEWiIBWQwUMPb0a2RrvzM+jSEdf AC/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=JTmgYs+pLw35c/280j1zRlSjfhuTXIlpsp3begL639c=; b=0yRZVBoyhFDIhHc2wIKtLA/DrXAc5cylXTKIHmb4KvMWfFfHtr1LQ0YjsPmD1zO1S/ x7UK58fOpnmTpqTsfrmVJhACcfThJkM0GaINVEU0CJ/PHBXFOrZCJc06j38onHRkd/s4 uyoVxdTxIRmfgHOdujtV8q1mq2kjZoNoxlSlal+Xd/7YvWuVIBUniI5ZGVu1CUe7V4Ce TLSVUe1drdCOKIQhzanLW6g1U2Xdf6k9tLne9vXY3CqJSKRz4FcV1zvK4Szx2GwikOqS SAmjsiCqinKWLKchHtc4hShK69L5Xbkdh0sYM8wKvC6QApIw9M7Trux7i/uuucdn6lMx RssA== X-Gm-Message-State: AOAM531ccL+eA88+aa+6RxF9AuO08/uNR+184cjJuBb6pWmlWyDGz6Dd PCw6mFlI7Ml2M6Z8PFWhWtWOc9oSGHs= X-Received: by 2002:a5d:6d81:0:b0:207:9100:2863 with SMTP id l1-20020a5d6d81000000b0020791002863mr11002794wrs.397.1649616295778; Sun, 10 Apr 2022 11:44:55 -0700 (PDT) Received: from localhost.localdomain ([212.15.177.33]) by smtp.gmail.com with ESMTPSA id b15-20020a05600018af00b002057c72d45fsm29079409wri.77.2022.04.10.11.44.54 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 10 Apr 2022 11:44:55 -0700 (PDT) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Sun, 10 Apr 2022 20:47:13 +0200 Message-Id: <20220410184714.60045-1-onemda@gmail.com> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] avfilter: add colormap video filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Ta8NZzd6+f8P Signed-off-by: Paul B Mahol --- doc/filters.texi | 33 +++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_colormap.c | 579 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 614 insertions(+) create mode 100644 libavfilter/vf_colormap.c diff --git a/doc/filters.texi b/doc/filters.texi index a5a817ea0a..ac49092743 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -8941,6 +8941,39 @@ colorlevels=romin=0.5:gomin=0.5:bomin=0.5 This filter supports the all above options as @ref{commands}. +@section colormap +Apply custom color maps to video stream. + +This filter needs three input video streams. +First stream is video stream that is going to be filtered out. +Second and third video stream specify color patches for source +color to target color mapping. + +The filter accepts the following options: + +@table @option +@item patch_size +Set the source and target video stream patch size in pixels. + +@item nb_patches +Set the max number of used patches from source and target video stream. + +@item type +Set the adjustments used for target colors. Can be @code{relative} or @code{absolute}. +Defaults is @code{absolute}. + +@item kernel +Set the kernel used to measure color differences between mapped colors. + +The accepted values are: +@table @samp +@item euclidean +@item weuclidean +@end table + +Default is @code{euclidean}. +@end table + @section colormatrix Convert color matrix. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 3f905bced4..d9865ef25e 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -219,6 +219,7 @@ OBJS-$(CONFIG_COLORKEY_OPENCL_FILTER) += vf_colorkey_opencl.o opencl.o \ opencl/colorkey.o OBJS-$(CONFIG_COLORHOLD_FILTER) += vf_colorkey.o OBJS-$(CONFIG_COLORLEVELS_FILTER) += vf_colorlevels.o +OBJS-$(CONFIG_COLORMAP_FILTER) += vf_colormap.o OBJS-$(CONFIG_COLORMATRIX_FILTER) += vf_colormatrix.o OBJS-$(CONFIG_COLORSPACE_FILTER) += vf_colorspace.o colorspace.o colorspacedsp.o OBJS-$(CONFIG_COLORTEMPERATURE_FILTER) += vf_colortemperature.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index b278423884..8574d62e85 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -205,6 +205,7 @@ extern const AVFilter ff_vf_colorkey; extern const AVFilter ff_vf_colorkey_opencl; extern const AVFilter ff_vf_colorhold; extern const AVFilter ff_vf_colorlevels; +extern const AVFilter ff_vf_colormap; extern const AVFilter ff_vf_colormatrix; extern const AVFilter ff_vf_colorspace; extern const AVFilter ff_vf_colortemperature; diff --git a/libavfilter/vf_colormap.c b/libavfilter/vf_colormap.c new file mode 100644 index 0000000000..30e89aaad4 --- /dev/null +++ b/libavfilter/vf_colormap.c @@ -0,0 +1,579 @@ +/* + * Copyright (c) 2022 Paul B Mahol + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * Compute a look-up table from map of colors. + */ + +#include "config_components.h" + +#include "libavutil/attributes.h" +#include "libavutil/avstring.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "avfilter.h" +#include "formats.h" +#include "internal.h" +#include "framesync.h" +#include "video.h" + +#define MAX_SIZE 64 + +enum KernelType { + EUCLIDEAN, + WEUCLIDEAN, + NB_KERNELS, +}; + +typedef struct ColorMapContext { + const AVClass *class; + int w, h; + int size; + int nb_maps; + int changed[2]; + + float source[MAX_SIZE][4]; + float ttarget[MAX_SIZE][4]; + float target[MAX_SIZE][4]; + float icoeff[4][4]; + float coeff[MAX_SIZE][4]; + + int target_type; + int kernel_type; + float (*kernel)(const float *x, const float *y); + + FFFrameSync fs; +} ColorMapContext; + +#define OFFSET(x) offsetof(ColorMapContext, x) +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_RUNTIME_PARAM + +static const AVOption colormap_options[] = { + { "patch_size", "set patch size", OFFSET(w), AV_OPT_TYPE_IMAGE_SIZE, {.str = "64x64"}, 0, 0, FLAGS }, + { "nb_patches", "set number of patches", OFFSET(size), AV_OPT_TYPE_INT, {.i64 = 8}, 1, MAX_SIZE, FLAGS }, + { "type", "set the target type used", OFFSET(target_type), AV_OPT_TYPE_INT, {.i64=1}, 0, 1, FLAGS, "type" }, + { "relative", "the target colors are relative", 0, AV_OPT_TYPE_CONST, {.i64=0}, 0, 1, FLAGS, "type" }, + { "absolute", "the target colors are absolute", 0, AV_OPT_TYPE_CONST, {.i64=1}, 0, 1, FLAGS, "type" }, + { "kernel", "set the kernel used for measuring color difference", OFFSET(kernel_type), AV_OPT_TYPE_INT, {.i64=0}, 0, NB_KERNELS-1, FLAGS, "kernel" }, + { "euclidean", "square root of sum of squared differences", 0, AV_OPT_TYPE_CONST, {.i64=EUCLIDEAN}, 0, 0, FLAGS, "kernel" }, + { "weuclidean", "weighted square root of sum of squared differences",0, AV_OPT_TYPE_CONST, {.i64=WEUCLIDEAN}, 0, 0, FLAGS, "kernel" }, + { NULL } +}; + +static int gauss_make_triangular(double *A, int *p, int n) +{ + p[n - 1] = n - 1; + for (int k = 0; k < n; k++) { + double t1; + int m = k; + + for (int i = k + 1; i < n; i++) + if (fabs(A[k + n * i]) > fabs(A[k + n * m])) + m = i; + p[k] = m; + t1 = A[k + n * m]; + A[k + n * m] = A[k + n * k]; + A[k + n * k] = t1; + if (t1 != 0) { + for (int i = k + 1; i < n; i++) + A[k + n * i] /= -t1; + if (k != m) + for (int i = k + 1; i < n; i++) { + double t2 = A[i + n * m]; + A[i + n * m] = A[i + n * k]; + A[i + n * k] = t2; + } + for (int j = k + 1; j < n; j++) + for (int i = k + 1; i < n; i++) + A[i + n * j] += A[k + j * n] * A[i + k * n]; + } else { + return 0; + } + } + + return 1; +} + +static void gauss_solve_triangular(const double *A, const int *p, double *b, int n) +{ + for(int k = 0; k < n - 1; k++) { + int m = p[k]; + double t = b[m]; + b[m] = b[k]; + b[k] = t; + for (int i = k + 1; i < n; i++) + b[i] += A[k + n * i] * t; + } + + for(int k = n - 1; k > 0; k--) { + b[k] /= A[k + n * k]; + double t = b[k]; + for (int i = 0; i < k; i++) + b[i] -= A[k + n * i] * t; + } + + b[0] /= A[0 + 0 * n]; +} + +static int gauss_solve(double *A, double *b, int n) +{ + int *p = av_calloc(n, sizeof(*p)); + + if (!p) + return 1; + + if (!gauss_make_triangular(A, p, n)) { + av_freep(&p); + return 1; + } + + gauss_solve_triangular(A, p, b, n); + + av_freep(&p); + + return 0; +} + +#define P2(x) ((x)*(x)) + +static float euclidean_kernel(const float *x, const float *y) +{ + const float d2 = P2(x[0]-y[0]) + + P2(x[1]-y[1]) + + P2(x[2]-y[2]); + return sqrtf(d2); +} + +static float weuclidean_kernel(const float *x, const float *y) +{ + const float rm = (x[0] + y[0]) * 0.5f; + const float d2 = P2(x[0]-y[0]) * (2.f + rm) + + P2(x[1]-y[1]) * 4.f + + P2(x[2]-y[2]) * (3.f - rm); + return sqrtf(d2); +} + +static void build_map(AVFilterContext *ctx) +{ + ColorMapContext *s = ctx->priv; + + for (int j = 0; j < s->nb_maps; j++) { + s->target[j][0] = s->target_type == 0 ? s->source[j][0] + s->ttarget[j][0] : s->ttarget[j][0]; + s->target[j][1] = s->target_type == 0 ? s->source[j][1] + s->ttarget[j][1] : s->ttarget[j][1]; + s->target[j][2] = s->target_type == 0 ? s->source[j][2] + s->ttarget[j][2] : s->ttarget[j][2]; + } + + for (int c = 0; c < 3; c++) { + for (int j = 0; j < s->nb_maps; j++) + s->coeff[j][c] = 0.f; + + for (int j = 0; j < 4; j++) { + s->icoeff[j][c] = 0; + s->icoeff[j][c] = 0; + s->icoeff[j][c] = 0; + } + + s->icoeff[c+1][c] = 1.f; + + switch (s->nb_maps) { + case 1: + { + float div = fabsf(s->source[0][c]) < 1e-6f ? 1e-6f : s->source[0][c]; + s->icoeff[c][1+c] = s->target[0][c] / div; + } + break; + case 2: + { + double A[2 * 2] = { 1, s->source[0][c], + 1, s->source[1][c] }; + double b[2] = { s->target[0][c], s->target[1][c] }; + + if (gauss_solve(A, b, 2)) + continue; + + s->icoeff[0 ][c] = b[0]; + s->icoeff[1+c][c] = b[1]; + } + break; + case 3: + { + const uint8_t idx[3][3] = {{ 0, 1, 2 }, + { 1, 0, 2 }, + { 2, 0, 1 }}; + const uint8_t didx[3][4] = {{ 0, 1, 2, 2 }, + { 0, 2, 1, 2 }, + { 0, 2, 2, 1 }}; + const int C0 = idx[c][0]; + const int C1 = idx[c][1]; + const int C2 = idx[c][2]; + double A[3 * 3] = { 1, s->source[0][C0], s->source[0][C1] + s->source[0][C2], + 1, s->source[1][C0], s->source[1][C1] + s->source[1][C2], + 1, s->source[2][C0], s->source[2][C1] + s->source[2][C2] }; + double b[3] = { s->target[0][c], s->target[1][c], s->target[2][c] }; + + if (gauss_solve(A, b, 3)) + continue; + + s->icoeff[0][c] = b[didx[c][0]]; + s->icoeff[1][c] = b[didx[c][1]]; + s->icoeff[2][c] = b[didx[c][2]]; + s->icoeff[3][c] = b[didx[c][3]]; + } + break; + case 4: + { + double A[4 * 4] = { 1, s->source[0][0], s->source[0][1], s->source[0][2], + 1, s->source[1][0], s->source[1][1], s->source[1][2], + 1, s->source[2][0], s->source[2][1], s->source[2][2], + 1, s->source[3][0], s->source[3][1], s->source[3][2] }; + double b[4] = { s->target[0][c], s->target[1][c], s->target[2][c], s->target[3][c] }; + int pivot[4]; + + if (!gauss_make_triangular(A, pivot, 4)) + continue; + gauss_solve_triangular(A, pivot, b, 4); + + s->icoeff[0][c] = b[0]; + s->icoeff[1][c] = b[1]; + s->icoeff[2][c] = b[2]; + s->icoeff[3][c] = b[3]; + } + break; + default: + { + const int N = s->nb_maps; + const int N4 = N + 4; + double *A = av_calloc(sizeof(*A), N4 * N4); + double *b = av_calloc(sizeof(*b), N4); + int *pivot = NULL; + + if (!A || !b) + goto error; + + for (int j = 0; j < N; j++) + for (int i = j; i < N; i++) + A[j*N4+i] = A[i*N4+j] = s->kernel(s->source[i], s->source[j]); + + for (int i = 0; i < N; i++) + A[i*N4+N+0] = A[(N+0)*N4+i] = 1; + for (int i = 0; i < N; i++) + A[i*N4+N+1] = A[(N+1)*N4+i] = s->source[i][0]; + for (int i = 0; i < N; i++) + A[i*N4+N+2] = A[(N+2)*N4+i] = s->source[i][1]; + for (int i = 0; i < N; i++) + A[i*N4+N+3] = A[(N+3)*N4+i] = s->source[i][2]; + + for (int j = N; j < N4; j++) + for (int i = N;i < N4; i++) + A[j * N4 + i] = 0.; + + pivot = av_calloc(N4, sizeof(*pivot)); + if (!pivot) + goto error; + + if (gauss_make_triangular(A, pivot, N4)) { + for (int i = 0; i < N; i++) + b[i] = s->target[i][c]; + for (int i = N; i < N + 4; i++) + b[i] = 0; + + gauss_solve_triangular(A, pivot, b, N4); + + for (int i = 0; i < N; i++) + s->coeff[i][c] = b[i]; + + for (int i = 0; i < 4; i++) + s->icoeff[i][c] = b[N + i]; + } +error: + av_free(pivot); + av_free(b); + av_free(A); + } + } + } +} + +typedef struct ThreadData { + AVFrame *in, *out; +} ThreadData; + +static int colormap_slice(AVFilterContext *ctx, void *arg, int jobnr, int nb_jobs) +{ + ColorMapContext *s = ctx->priv; + ThreadData *td = arg; + AVFrame *in = td->in; + AVFrame *out = td->out; + const int maps = s->nb_maps; + const int width = out->width; + const int height = out->height; + const int slice_start = (height * jobnr) / nb_jobs; + const int slice_end = (height * (jobnr + 1)) / nb_jobs; + const int sr_linesize = in->linesize[2] / 4; + const int dr_linesize = out->linesize[2] / 4; + const int sg_linesize = in->linesize[0] / 4; + const int dg_linesize = out->linesize[0] / 4; + const int sb_linesize = in->linesize[1] / 4; + const int db_linesize = out->linesize[1] / 4; + const float *sr = (float *)in->data[2] + slice_start * sr_linesize; + const float *sg = (float *)in->data[0] + slice_start * sg_linesize; + const float *sb = (float *)in->data[1] + slice_start * sb_linesize; + float *r = (float *)out->data[2] + slice_start * dr_linesize; + float *g = (float *)out->data[0] + slice_start * dg_linesize; + float *b = (float *)out->data[1] + slice_start * db_linesize; + + for (int y = slice_start; y < slice_end; y++) { + for (int x = 0; x < width; x++) { + const float input[3] = { sr[x], sg[x], sb[x] }; + float srv, sgv, sbv; + float rv, gv, bv; + + srv = sr[x]; + sgv = sg[x]; + sbv = sb[x]; + + rv = s->icoeff[0][0]; + gv = s->icoeff[0][1]; + bv = s->icoeff[0][2]; + + rv += s->icoeff[1][0] * srv + s->icoeff[2][0] * sgv + s->icoeff[3][0] * sbv; + gv += s->icoeff[1][1] * srv + s->icoeff[2][1] * sgv + s->icoeff[3][1] * sbv; + bv += s->icoeff[1][2] * srv + s->icoeff[2][2] * sgv + s->icoeff[3][2] * sbv; + + for (int z = 0; z < maps && maps > 4; z++) { + const float cr = s->coeff[z][0]; + const float cg = s->coeff[z][1]; + const float cb = s->coeff[z][2]; + const float f = s->kernel(input, s->source[z]); + + rv += f * cr; + gv += f * cg; + bv += f * cb; + } + + r[x] = rv; + g[x] = gv; + b[x] = bv; + } + + sg += sg_linesize; + g += dg_linesize; + sb += sb_linesize; + b += db_linesize; + sr += sr_linesize; + r += dr_linesize; + } + + return 0; +} + +static int import_map(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx = inlink->dst; + ColorMapContext *s = ctx->priv; + const int is_target = FF_INLINK_IDX(inlink) > 1; + const int pw = s->w; + const int pw2 = s->w / 2; + const int ph = s->h; + const int ph2 = s->h / 2; + int changed = 0; + int idx; + + for (int plane = 0; plane < 3; plane++) { + const int c = plane == 0 ? 1 : plane == 1 ? 2 : 0; + + idx = 0; + for (int y = ph2; y < in->height && idx < MAX_SIZE; y += ph) { + const float *src = (const float *)(in->data[plane] + y * in->linesize[plane]); + + for (int x = pw2; x < in->width && idx < MAX_SIZE; x += pw) { + float value = src[x]; + + if (is_target) { + if (s->ttarget[idx][c] != value) + changed = 1; + s->ttarget[idx][c] = value; + } else { + if (s->source[idx][c] != value) + changed = 1; + s->source[idx][c] = value; + } + + idx++; + } + } + } + + if (changed) + s->changed[is_target] = 1; + if (!is_target) + s->nb_maps = FFMIN(idx, s->size); + + return 0; +} + +static int process_frame(FFFrameSync *fs) +{ + AVFilterContext *ctx = fs->parent; + ColorMapContext *s = fs->opaque; + AVFilterLink *outlink = ctx->outputs[0]; + AVFrame *in, *out, *source, *target; + ThreadData td; + int ret; + + switch (s->kernel_type) { + case EUCLIDEAN: + s->kernel = euclidean_kernel; + break; + case WEUCLIDEAN: + s->kernel = weuclidean_kernel; + break; + default: + return AVERROR_BUG; + } + + if ((ret = ff_framesync_get_frame(&s->fs, 0, &in, 1)) < 0 || + (ret = ff_framesync_get_frame(&s->fs, 1, &source, 0)) < 0 || + (ret = ff_framesync_get_frame(&s->fs, 2, &target, 0)) < 0) + return ret; + + import_map(ctx->inputs[1], source); + import_map(ctx->inputs[2], target); + + if (s->changed[0] || s->changed[1]) { + build_map(ctx); + s->changed[0] = s->changed[1] = 0; + } + + if (!ctx->is_disabled) { + if (av_frame_is_writable(in)) { + out = in; + } else { + out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!out) { + av_frame_free(&in); + return AVERROR(ENOMEM); + } + av_frame_copy_props(out, in); + } + + td.in = in; + td.out = out; + ff_filter_execute(ctx, colormap_slice, &td, NULL, + FFMIN(in->height, ff_filter_get_nb_threads(ctx))); + + if (out != in) + av_frame_free(&in); + } else { + out = in; + } + + out->pts = av_rescale_q(s->fs.pts, s->fs.time_base, outlink->time_base); + + return ff_filter_frame(outlink, out); +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + ColorMapContext *s = ctx->priv; + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *source = ctx->inputs[1]; + AVFilterLink *target = ctx->inputs[2]; + FFFrameSyncIn *in; + int ret; + + outlink->time_base = inlink->time_base; + outlink->frame_rate = inlink->frame_rate; + outlink->sample_aspect_ratio = inlink->sample_aspect_ratio; + outlink->w = inlink->w; + outlink->h = inlink->h; + + if ((ret = ff_framesync_init(&s->fs, ctx, 3)) < 0) + return ret; + + in = s->fs.in; + in[0].time_base = inlink->time_base; + in[1].time_base = source->time_base; + in[2].time_base = target->time_base; + in[0].sync = 1; + in[0].before = EXT_STOP; + in[0].after = EXT_INFINITY; + in[1].sync = 1; + in[1].before = EXT_STOP; + in[1].after = EXT_INFINITY; + in[2].sync = 1; + in[2].before = EXT_STOP; + in[2].after = EXT_INFINITY; + s->fs.opaque = s; + s->fs.on_event = process_frame; + + ret = ff_framesync_configure(&s->fs); + outlink->time_base = s->fs.time_base; + + return ret; +} + +static int activate(AVFilterContext *ctx) +{ + ColorMapContext *s = ctx->priv; + return ff_framesync_activate(&s->fs); +} + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, + { + .name = "source", + .type = AVMEDIA_TYPE_VIDEO, + }, + { + .name = "target", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_output, + }, +}; + +AVFILTER_DEFINE_CLASS(colormap); + +const AVFilter ff_vf_colormap = { + .name = "colormap", + .description = NULL_IF_CONFIG_SMALL("Apply custom Color Maps to video stream."), + .priv_class = &colormap_class, + .priv_size = sizeof(ColorMapContext), + .activate = activate, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_PIXFMTS(AV_PIX_FMT_GBRPF32, AV_PIX_FMT_GBRAPF32), + .flags = AVFILTER_FLAG_SUPPORT_TIMELINE_INTERNAL | + AVFILTER_FLAG_SLICE_THREADS, + .process_command = ff_filter_process_command, +};