From patchwork Mon Aug 28 13:25:54 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 4865 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.15.201 with SMTP id 70csp2244447jao; Mon, 28 Aug 2017 06:34:22 -0700 (PDT) X-Received: by 10.223.145.5 with SMTP id j5mr457783wrj.148.1503927261956; Mon, 28 Aug 2017 06:34:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1503927261; cv=none; d=google.com; s=arc-20160816; b=vc156EGV8o4Jmfs0bT5N+jZt7eR/j2q1Su2pU2YsDLIVxHDrxk6TP3RM+9TcLMAMFU 5iyEMEIRVWbCINofSQIkxgiCRK6wO7K5Z6zbaYiRuW9fS6kU4S7EJbCEoK0LrZDLq5Yp PJ1TMCJc+7/pfwHs2urQcnKu0nn/tEPgzMT1H7ENwG28JWXBt9DArVNhaH1h3th2zWYH mkwLsBytEpf6F6PHi22Y9mMZPVfom+F1eKULSglJNibCrek7PPaJYJi0eQVMCN1NQxJh krFBhxtJaksKvMVH4m6DlO8R1UH1NfwkIwIg3x4KQ6RS0JrwkVZUTIdDfZNJhkbr2bLE BjiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=A30oE7mObbW8b5qnMgHo4hXMz5ZucK8tp7hBFN/KhvI=; b=t2hiGHmpyBcCNO7MsvRFFzwm1+lWQtIiHplwYjhPL3vb4So7rKQSQBDHoVODVpODd1 qSeA+1YprZC4m+HkDZDGOfKTVMqn5Ruf4kNdYuE/SFe41CmVvrKcFd26mC3wEQO661dl ZuCm7fyVGQfz6pvWX7Qom7L8BR99tU4ts5uYmJWIvnkB2fBgwiY/O8e5cw8CrcYgcr56 EgliFGclLbmA0ATWCnS5cZ/tRR/U6JT5zOa77wpoB/4EjKdIBajBK+5IyvVGhtR9Sw+b pLKPY/KPb1c9yudykq5MHk+G0NNHJj/uJ3soUS7WvwQGYtaqBPudv3uPqCXSz52juD9o msmA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=Ofty/LYu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 194si283526wmg.147.2017.08.28.06.34.21; Mon, 28 Aug 2017 06:34:21 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=Ofty/LYu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D6662689EB0; Mon, 28 Aug 2017 16:34:07 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr0-f196.google.com (mail-wr0-f196.google.com [209.85.128.196]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6532F689DD8 for ; Mon, 28 Aug 2017 16:34:01 +0300 (EEST) Received: by mail-wr0-f196.google.com with SMTP id z91so394757wrc.1 for ; Mon, 28 Aug 2017 06:34:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=CZaQpwKs9etKZuVE6tdyrwKA6K6afJRNlKpYl6uf92s=; b=Ofty/LYuCLuo5jPJ389AujIoGc3GgLoThKqGQnuvEEVkrASwhuu+XBXgP0EOCfHOtW 9qGOuoUEUkl97CbwHHLHi9+4BoJOgynGXXo00/j6gcZy2Ngv2eD/gMZriDOwbQFFu/kH bQGdkIT8X5nnxHwZVg6CHDFbx7ZPvmSStyiGdkv3YyIuN2VgatmckG9Q6F3n/eg9A6kl NvgPjKG+a+hpSR0UnHcwDgpNvyD5mkVoJW4rvKJlMot/oqmsk22BnJZOnUx7HGJlwYNR 06kS+CHuV9kyG4UMADhIGz4X2OzlRtA0n7UK7/MeI+5uzxYGPheARJsFVLNjfT21lfst cARQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=CZaQpwKs9etKZuVE6tdyrwKA6K6afJRNlKpYl6uf92s=; b=JeaE9HkwZubSrrfq1PVKAwPCBKd6y1OLDaSP5tpxqtInbPFf21XgRYGKFZR8WS9bM4 Nh3BYK+10cWsUyU91qIP9A+f/0byQL2oXUQwQmAVAOz5U6/SVI1TFFSBlD0gCaNhazov GI+Rifdk2B62R1N4/HsFcjvqEiBTa4lX4aqlo7Ep8FVOZzX1niDwlqKdn8HVi/nPYbJm FiiHsJHmI3lofJ56a8aNWK3UITsYeCnmCvzBSkp/2fvcxCFjNO8vUgPAy+5PelpF4TCo uaCL9VdB4t1QAhHTjPRfrOCaHutmhKiDOhfIFPstT3WA0Pzr0jpTRdutu/gU9Da2tl50 pxPw== X-Gm-Message-State: AHYfb5gomyS/LEkT6/pJMFUWS1HYQtQAh6Ppw3S1mXMbiIR5pW3dtA7D 6kODUjw1OR8A3Uwd X-Received: by 10.223.187.10 with SMTP id r10mr492889wrg.8.1503926766300; Mon, 28 Aug 2017 06:26:06 -0700 (PDT) Received: from localhost.localdomain ([94.250.174.60]) by smtp.gmail.com with ESMTPSA id 136sm336501wmo.48.2017.08.28.06.26.04 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Aug 2017 06:26:05 -0700 (PDT) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Mon, 28 Aug 2017 15:25:54 +0200 Message-Id: <20170828132554.585-1-onemda@gmail.com> X-Mailer: git-send-email 2.9.3 Subject: [FFmpeg-devel] [PATCH] avfilter: add despill filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Paul B Mahol --- doc/filters.texi | 35 +++++++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_despill.c | 183 +++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 220 insertions(+) create mode 100644 libavfilter/vf_despill.c diff --git a/doc/filters.texi b/doc/filters.texi index 779fd53..1d8b46d 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -6780,6 +6780,41 @@ FFmpeg was configured with @code{--enable-opencl}. Default value is 0. @end table +@section despill + +Remove unwanted contamination of foreground colors, caused by reflected color of +greenscreen or bluescreen. + +This filter accepts the following options: + +@table @option +@item type +Set what type of despill to use. + +@item mix +Set how spillmap will be generated. + +@item expand +Set how much to get rid of still remaining spill. + +@item red +Controls ammount of red in spill area. + +@item green +Controls ammount of green in spill area. +Should be -1 for greenscreen. + +@item blue +Controls ammount of blue in spill area. +Should be -1 for bluescreen. + +@item brightness +Controls brightness of spill area, preserving colors. + +@item alpha +Modify alpha from generated spillmap. +@end table + @section detelecine Apply an exact inverse of the telecine operation. It requires a predefined diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 1d92dc1..23d20cc 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -166,6 +166,7 @@ OBJS-$(CONFIG_DEINTERLACE_VAAPI_FILTER) += vf_deinterlace_vaapi.o OBJS-$(CONFIG_DEJUDDER_FILTER) += vf_dejudder.o OBJS-$(CONFIG_DELOGO_FILTER) += vf_delogo.o OBJS-$(CONFIG_DESHAKE_FILTER) += vf_deshake.o +OBJS-$(CONFIG_DESPILL_FILTER) += vf_despill.o OBJS-$(CONFIG_DETELECINE_FILTER) += vf_detelecine.o OBJS-$(CONFIG_DILATION_FILTER) += vf_neighbor.o OBJS-$(CONFIG_DISPLACE_FILTER) += vf_displace.o framesync2.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 8b9b9a4..9a2cfea 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -178,6 +178,7 @@ static void register_all(void) REGISTER_FILTER(DEJUDDER, dejudder, vf); REGISTER_FILTER(DELOGO, delogo, vf); REGISTER_FILTER(DESHAKE, deshake, vf); + REGISTER_FILTER(DESPILL, despill, vf); REGISTER_FILTER(DETELECINE, detelecine, vf); REGISTER_FILTER(DILATION, dilation, vf); REGISTER_FILTER(DISPLACE, displace, vf); diff --git a/libavfilter/vf_despill.c b/libavfilter/vf_despill.c new file mode 100644 index 0000000..eff9877 --- /dev/null +++ b/libavfilter/vf_despill.c @@ -0,0 +1,183 @@ +/* + * Copyright (c) 2017 Paul B Mahol + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/opt.h" +#include "libavutil/imgutils.h" +#include "avfilter.h" +#include "formats.h" +#include "internal.h" +#include "video.h" + +typedef struct DespillContext { + const AVClass *class; + + int co[4]; /* color offsets rgba */ + + int alpha; + int type; + float spillmix; + float spillexpand; + float redscale; + float greenscale; + float bluescale; + float brightness; +} DespillContext; + +static int do_despill_slice(AVFilterContext *ctx, void *arg, int jobnr, int nb_jobs) +{ + DespillContext *s = ctx->priv; + AVFrame *frame = arg; + const int ro = s->co[0], go = s->co[1], bo = s->co[2], ao = s->co[3]; + const int slice_start = (frame->height * jobnr) / nb_jobs; + const int slice_end = (frame->height * (jobnr + 1)) / nb_jobs; + const float brightness = s->brightness; + const float redscale = s->redscale; + const float greenscale = s->greenscale; + const float bluescale = s->bluescale; + const float spillmix = s->spillmix; + const float factor = (1.f - spillmix) * (1.f - s->spillexpand); + float red, green, blue; + int x, y; + + for (y = slice_start; y < slice_end; y++) { + uint8_t *dst = frame->data[0] + y * frame->linesize[0]; + + for (x = 0; x < frame->width; x++) { + float spillmap; + + red = dst[x * 4 + ro] / 255.f; + green = dst[x * 4 + go] / 255.f; + blue = dst[x * 4 + bo] / 255.f; + + if (s->type) { + spillmap = FFMAX(blue - (red * spillmix + green * factor), 0.f); + } else { + spillmap = FFMAX(green - (red * spillmix + blue * factor), 0.f); + } + + red = FFMAX(red + spillmap * redscale + brightness * spillmap, 0.f); + green = FFMAX(green + spillmap * greenscale + brightness * spillmap, 0.f); + blue = FFMAX(blue + spillmap * bluescale + brightness * spillmap, 0.f); + + dst[x * 4 + ro] = av_clip_uint8(red * 255); + dst[x * 4 + go] = av_clip_uint8(green * 255); + dst[x * 4 + bo] = av_clip_uint8(blue * 255); + if (s->alpha) { + spillmap = 1.f - spillmap; + dst[x * 4 + ao] = av_clip_uint8(spillmap * 255); + } + } + } + + return 0; +} + +static int filter_frame(AVFilterLink *link, AVFrame *frame) +{ + AVFilterContext *ctx = link->dst; + int ret; + + if (ret = av_frame_make_writable(frame)) + return ret; + + if (ret = ctx->internal->execute(ctx, do_despill_slice, frame, NULL, FFMIN(frame->height, ff_filter_get_nb_threads(ctx)))) + return ret; + + return ff_filter_frame(ctx->outputs[0], frame); +} + +static av_cold int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + DespillContext *s = ctx->priv; + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(outlink->format); + int i; + + for (i = 0; i < 4; ++i) + s->co[i] = desc->comp[i].offset; + + return 0; +} + +static av_cold int query_formats(AVFilterContext *ctx) +{ + static const enum AVPixelFormat pixel_fmts[] = { + AV_PIX_FMT_ARGB, + AV_PIX_FMT_RGBA, + AV_PIX_FMT_ABGR, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_NONE + }; + AVFilterFormats *formats = NULL; + + formats = ff_make_format_list(pixel_fmts); + if (!formats) + return AVERROR(ENOMEM); + + return ff_set_common_formats(ctx, formats); +} + +static const AVFilterPad despill_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + }, + { NULL } +}; + +static const AVFilterPad despill_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_output, + }, + { NULL } +}; + +#define OFFSET(x) offsetof(DespillContext, x) +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM + +static const AVOption despill_options[] = { + { "type", "set the screen type", OFFSET(type), AV_OPT_TYPE_INT, {.i64=0}, 0, 1, FLAGS, "type" }, + { "green", "greenscreen", 0, AV_OPT_TYPE_CONST, {.i64=0}, 0, 0, FLAGS, "type" }, + { "blue", "bluescreen", 0, AV_OPT_TYPE_CONST, {.i64=1}, 0, 0, FLAGS, "type" }, + { "mix", "set the spillmap mix", OFFSET(spillmix), AV_OPT_TYPE_FLOAT, {.dbl=0.5}, 0, 1, FLAGS }, + { "expand", "set the spillmap expand", OFFSET(spillexpand), AV_OPT_TYPE_FLOAT, {.dbl=0}, 0, 1, FLAGS }, + { "red", "set red scale", OFFSET(bluescale), AV_OPT_TYPE_FLOAT, {.dbl=0}, -100, 100, FLAGS }, + { "green", "set green scale", OFFSET(greenscale), AV_OPT_TYPE_FLOAT, {.dbl=-1}, -100, 100, FLAGS }, + { "blue", "set blue scale", OFFSET(bluescale), AV_OPT_TYPE_FLOAT, {.dbl=0}, -100, 100, FLAGS }, + { "brightness", "set brightness", OFFSET(brightness), AV_OPT_TYPE_FLOAT, {.dbl=0}, -10, 10, FLAGS }, + { "alpha", "change alpha component", OFFSET(alpha), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(despill); + +AVFilter ff_vf_despill = { + .name = "despill", + .description = NULL_IF_CONFIG_SMALL("Despill video."), + .priv_size = sizeof(DespillContext), + .priv_class = &despill_class, + .query_formats = query_formats, + .inputs = despill_inputs, + .outputs = despill_outputs, + .flags = AVFILTER_FLAG_SUPPORT_TIMELINE_GENERIC | AVFILTER_FLAG_SLICE_THREADS, +};