From patchwork Tue Nov 13 19:35:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 11012 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id E909944D9BE for ; Tue, 13 Nov 2018 21:35:23 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5A93A689FC6; Tue, 13 Nov 2018 21:34:55 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4EE18689D0F for ; Tue, 13 Nov 2018 21:34:49 +0200 (EET) Received: by mail-wr1-f68.google.com with SMTP id z13-v6so14656477wrs.3 for ; Tue, 13 Nov 2018 11:35:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=ti8SAgIq8Pz7Ht7VV8Ri9J5Zup2wMzrVlv3c14zC0Tw=; b=QSVK/Cwz+tCWsnLhnueP+0FF9CHYujANOYqOIXZiv8dn775E0AKhcQju2wA6IjKhil xG5jrbPPvM4uT/nZjMywmLhGwaIEagIgipi4gXpdnTNmaloL5PmyJv6+HQLyY3cTtQBa PMe3JPe9XXsATi01FjDgAp+pnXuhOI0M5m7WN4PE3tbPytyYGqkAWXX2oitYuVq5Z6fm qXffUXXYV09OY/J+UDB3LMFiF/GPh972KQ2KOnTd4aRwiDxL+OfbBKiJWLH2u1gmKX8f 1qV7OpGs5UVADLnV59cQIZlZWSFJCa19DDytQ+JlqG01GADv3PRferiglR31LW0L3n3D zAgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=ti8SAgIq8Pz7Ht7VV8Ri9J5Zup2wMzrVlv3c14zC0Tw=; b=QNb3QvTwkbWel6KzLHumRZYjE8k9Keg7CggExmkoka0UEJq0TGKrp9gtE7Gn7+C04Y Yfpq5ZnIADPo5L7jhzGR84XoIqm7ormeuK1VAl3FGOv4GZMUBmP+sABB4lSX+PJ/ZB5D UlQB1Lj6VhxVKs8I5f5R1gOxYbTeVnqezLd514bVTGI6dE0MoJaMsowmarShAWnMjlEl 49FON5PpX5yT86HRyGfgV4XggHRdIDkLGwqNNmfsd0gPB/pL5NvqkAApLZMnF9hksHSN E8A80JUATCO3Sx5gp1JKFENJAo+d6+pBXTS8XyMV72opfBn/25kSYbjSWY1Ex5Bv0Qho MoIA== X-Gm-Message-State: AGRZ1gJ+V4jFdlH8sfkTlVL9fbuzdqj/Vu8eEFGoyMHyr/0XLTZZOBMD jXI7qo/BCoT6J7CRzm9NRnf2ua/5Eog= X-Google-Smtp-Source: AJdET5eT4P57WjQhLStj30G44i9urBclFQ6dMxoOs3xB2XJAqmkZGCgfiV1P769PvRev4us3I4Dtrw== X-Received: by 2002:adf:b50a:: with SMTP id a10-v6mr5891199wrd.273.1542137725876; Tue, 13 Nov 2018 11:35:25 -0800 (PST) Received: from localhost.localdomain ([94.250.174.60]) by smtp.gmail.com with ESMTPSA id x6-v6sm22375214wrq.52.2018.11.13.11.35.24 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Nov 2018 11:35:25 -0800 (PST) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Tue, 13 Nov 2018 20:35:17 +0100 Message-Id: <20181113193517.31788-1-onemda@gmail.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH] avfilter: add dedot video filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Paul B Mahol --- doc/filters.texi | 24 +++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_dedot.c | 330 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 356 insertions(+) create mode 100644 libavfilter/vf_dedot.c diff --git a/doc/filters.texi b/doc/filters.texi index 0d9ff43ef0..42cb3f54d6 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -7761,6 +7761,30 @@ had noise. The @code{deconvolve} filter also supports the @ref{framesync} options. +@section dedot + +Reduce cross-luminance (dot-crawl) and cross-color (rainbows) from video. + +It accepts the following options: + +@table @option +@item m +Set mode of operation. Can be combination of @var{dot-crawl} for cross-luminance reduction and/or +@var{rainbows} for cross-color reduction. + +@item lt +Set spatial luma threshold. Lower values increases reduction of cross-luminance. + +@item tl +Set tolerance for temporal luma. Higher values increases reduction of cross-luminance. + +@item tc +Set tolerance for chroma temporal variation. Higher values increases reduction of cross-color. + +@item ct +Set temporal chroma threshold. Lower values increases reduction of cross-color. +@end table + @section deflate Apply deflate effect to the video. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 7c6fc836e5..eb5abee6b9 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -194,6 +194,7 @@ OBJS-$(CONFIG_DEBAND_FILTER) += vf_deband.o OBJS-$(CONFIG_DEBLOCK_FILTER) += vf_deblock.o OBJS-$(CONFIG_DECIMATE_FILTER) += vf_decimate.o OBJS-$(CONFIG_DECONVOLVE_FILTER) += vf_convolve.o framesync.o +OBJS-$(CONFIG_DEDOT_FILTER) += vf_dedot.o OBJS-$(CONFIG_DEFLATE_FILTER) += vf_neighbor.o OBJS-$(CONFIG_DEFLICKER_FILTER) += vf_deflicker.o OBJS-$(CONFIG_DEINTERLACE_QSV_FILTER) += vf_deinterlace_qsv.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 484b080dea..8dfa572fde 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -182,6 +182,7 @@ extern AVFilter ff_vf_deband; extern AVFilter ff_vf_deblock; extern AVFilter ff_vf_decimate; extern AVFilter ff_vf_deconvolve; +extern AVFilter ff_vf_dedot; extern AVFilter ff_vf_deflate; extern AVFilter ff_vf_deflicker; extern AVFilter ff_vf_deinterlace_qsv; diff --git a/libavfilter/vf_dedot.c b/libavfilter/vf_dedot.c new file mode 100644 index 0000000000..ee6456be29 --- /dev/null +++ b/libavfilter/vf_dedot.c @@ -0,0 +1,330 @@ +/* + * Copyright (c) 2018 Paul B Mahol + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" + +#include "avfilter.h" +#include "filters.h" +#include "formats.h" +#include "internal.h" +#include "video.h" + +typedef struct DedotContext { + const AVClass *class; + int m; + float lt; + float tl; + float tc; + float ct; + + const AVPixFmtDescriptor *desc; + int depth; + int max; + int luma2d; + int lumaT; + int chromaT1; + int chromaT2; + + int nb_planes; + int planewidth[4]; + int planeheight[4]; + + AVFrame *frames[5]; + + void (*dedotcrawl)(AVFilterContext *ctx, AVFrame *out); + void (*derainbow)(AVFilterContext *ctx, AVFrame *out, int plane); +} DedotContext; + +static int query_formats(AVFilterContext *ctx) +{ + static const enum AVPixelFormat pixel_fmts[] = { + AV_PIX_FMT_YUVA444P, AV_PIX_FMT_YUV444P, AV_PIX_FMT_YUV440P, + AV_PIX_FMT_YUVJ444P, AV_PIX_FMT_YUVJ440P, + AV_PIX_FMT_YUVA422P, AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUVA420P, AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUVJ422P, AV_PIX_FMT_YUVJ420P, + AV_PIX_FMT_YUVJ411P, AV_PIX_FMT_YUV411P, AV_PIX_FMT_YUV410P, + AV_PIX_FMT_YUV420P9, AV_PIX_FMT_YUV422P9, AV_PIX_FMT_YUV444P9, + AV_PIX_FMT_YUV420P10, AV_PIX_FMT_YUV422P10, AV_PIX_FMT_YUV444P10, + AV_PIX_FMT_YUV420P12, AV_PIX_FMT_YUV422P12, AV_PIX_FMT_YUV444P12, AV_PIX_FMT_YUV440P12, + AV_PIX_FMT_YUV420P14, AV_PIX_FMT_YUV422P14, AV_PIX_FMT_YUV444P14, + AV_PIX_FMT_YUV420P16, AV_PIX_FMT_YUV422P16, AV_PIX_FMT_YUV444P16, + AV_PIX_FMT_YUVA420P9, AV_PIX_FMT_YUVA422P9, AV_PIX_FMT_YUVA444P9, + AV_PIX_FMT_YUVA420P10, AV_PIX_FMT_YUVA422P10, AV_PIX_FMT_YUVA444P10, + AV_PIX_FMT_YUVA420P16, AV_PIX_FMT_YUVA422P16, AV_PIX_FMT_YUVA444P16, + AV_PIX_FMT_NONE + }; + AVFilterFormats *formats = ff_make_format_list(pixel_fmts); + if (!formats) + return AVERROR(ENOMEM); + return ff_set_common_formats(ctx, formats); +} + +#define DEFINE_DEDOTCRAWL(name, type, div) \ +static void dedotcrawl##name(AVFilterContext *ctx, AVFrame *out) \ +{ \ + DedotContext *s = ctx->priv; \ + int src_linesize = s->frames[2]->linesize[0] / div; \ + int dst_linesize = out->linesize[0] / div; \ + int p0_linesize = s->frames[0]->linesize[0] / div; \ + int p1_linesize = s->frames[1]->linesize[0] / div; \ + int p3_linesize = s->frames[3]->linesize[0] / div; \ + int p4_linesize = s->frames[4]->linesize[0] / div; \ + type *p0 = (type *)s->frames[0]->data[0] + p0_linesize; \ + type *p1 = (type *)s->frames[1]->data[0] + p1_linesize; \ + type *p3 = (type *)s->frames[3]->data[0] + p3_linesize; \ + type *p4 = (type *)s->frames[4]->data[0] + p4_linesize; \ + type *src = (type *)s->frames[2]->data[0] + src_linesize; \ + type *dst = (type *)out->data[0] + dst_linesize; \ + const int luma2d = s->luma2d; \ + const int lumaT = s->lumaT; \ + \ + for (int y = 1; y < s->planeheight[0] - 1; y++) { \ + for (int x = 1; x < s->planewidth[0] - 1; x++) { \ + int above = src[x - src_linesize]; \ + int bellow = src[x + src_linesize]; \ + int cur = src[x]; \ + int left = src[x - 1]; \ + int right = src[x + 1]; \ + \ + if (FFABS(above + bellow - 2 * cur) <= luma2d && \ + FFABS(left + right - 2 * cur) <= luma2d) \ + continue; \ + \ + if (FFABS(cur - p0[x]) <= lumaT && \ + FFABS(cur - p4[x]) <= lumaT && \ + FFABS(p1[x] - p3[x]) <= lumaT) { \ + int diff1 = FFABS(cur - p1[x]); \ + int diff2 = FFABS(cur - p3[x]); \ + \ + if (diff1 < diff2) \ + dst[x] = (src[x] + p1[x] + 1) >> 1; \ + else \ + dst[x] = (src[x] + p3[x] + 1) >> 1; \ + } \ + } \ + \ + dst += dst_linesize; \ + src += src_linesize; \ + p0 += p0_linesize; \ + p1 += p1_linesize; \ + p3 += p3_linesize; \ + p4 += p4_linesize; \ + } \ +} + +DEFINE_DEDOTCRAWL(8, uint8_t, 1) +DEFINE_DEDOTCRAWL(16, uint16_t, 2) + +#define DEFINE_DERAINBOW(name, type, div) \ +static void derainbow##name(AVFilterContext *ctx, AVFrame *out, int plane) \ +{ \ + DedotContext *s = ctx->priv; \ + int src_linesize = s->frames[2]->linesize[plane] / div; \ + int dst_linesize = out->linesize[plane] / div; \ + int p0_linesize = s->frames[0]->linesize[plane] / div; \ + int p1_linesize = s->frames[1]->linesize[plane] / div; \ + int p3_linesize = s->frames[3]->linesize[plane] / div; \ + int p4_linesize = s->frames[4]->linesize[plane] / div; \ + type *p0 = (type *)s->frames[0]->data[plane]; \ + type *p1 = (type *)s->frames[1]->data[plane]; \ + type *p3 = (type *)s->frames[3]->data[plane]; \ + type *p4 = (type *)s->frames[4]->data[plane]; \ + type *src = (type *)s->frames[2]->data[plane]; \ + type *dst = (type *)out->data[plane]; \ + const int chromaT1 = s->chromaT1; \ + const int chromaT2 = s->chromaT2; \ + \ + for (int y = 0; y < s->planeheight[plane]; y++) { \ + for (int x = 0; x < s->planewidth[plane]; x++) { \ + int cur = src[x]; \ + \ + if (FFABS(cur - p0[x]) <= chromaT1 && \ + FFABS(cur - p4[x]) <= chromaT1 && \ + FFABS(p1[x] - p3[x]) <= chromaT1 && \ + FFABS(cur - p1[x]) > chromaT2 && \ + FFABS(cur - p3[x]) > chromaT2) { \ + int diff1 = FFABS(cur - p1[x]); \ + int diff2 = FFABS(cur - p3[x]); \ + \ + if (diff1 < diff2) \ + dst[x] = (src[x] + p1[x] + 1) >> 1; \ + else \ + dst[x] = (src[x] + p3[x] + 1) >> 1; \ + } \ + } \ + \ + dst += dst_linesize; \ + src += src_linesize; \ + p0 += p0_linesize; \ + p1 += p1_linesize; \ + p3 += p3_linesize; \ + p4 += p4_linesize; \ + } \ +} + +DEFINE_DERAINBOW(8, uint8_t, 1) +DEFINE_DERAINBOW(16, uint16_t, 2) + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + DedotContext *s = ctx->priv; + AVFilterLink *inlink = ctx->inputs[0]; + + s->desc = av_pix_fmt_desc_get(outlink->format); + if (!s->desc) + return AVERROR_BUG; + s->nb_planes = av_pix_fmt_count_planes(outlink->format); + s->depth = s->desc->comp[0].depth; + s->max = (1 << s->depth) - 1; + s->luma2d = s->lt * s->max; + s->lumaT = s->tl * s->max; + s->chromaT1 = s->tc * s->max; + s->chromaT2 = s->ct * s->max; + + s->planewidth[1] = s->planewidth[2] = AV_CEIL_RSHIFT(inlink->w, s->desc->log2_chroma_w); + s->planewidth[0] = s->planewidth[3] = inlink->w; + + s->planeheight[1] = s->planeheight[2] = AV_CEIL_RSHIFT(inlink->h, s->desc->log2_chroma_h); + s->planeheight[0] = s->planeheight[3] = inlink->h; + + if (s->depth <= 8) { + s->dedotcrawl = dedotcrawl8; + s->derainbow = derainbow8; + } else { + s->dedotcrawl = dedotcrawl16; + s->derainbow = derainbow16; + } + + return 0; +} + +static int activate(AVFilterContext *ctx) +{ + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + DedotContext *s = ctx->priv; + AVFrame *frame; + int ret; + + FF_FILTER_FORWARD_STATUS_BACK(outlink, inlink); + + ret = ff_inlink_consume_frame(inlink, &frame); + if (ret < 0) + return ret; + if (ret > 0) { + AVFrame *out; + + for (int i = 0; i < 5; i++) { + if (!s->frames[i]) + s->frames[i] = av_frame_clone(frame); + } + av_frame_free(&frame); + out = av_frame_clone(s->frames[2]); + if (out) { + if (!ctx->is_disabled) { + ret = av_frame_make_writable(out); + if (ret >= 0) { + if (s->m & 1) + s->dedotcrawl(ctx, out); + if (s->m & 2) { + s->derainbow(ctx, out, 1); + s->derainbow(ctx, out, 2); + } + } + } + av_frame_free(&s->frames[0]); + s->frames[0] = s->frames[1]; + s->frames[1] = s->frames[2]; + s->frames[2] = s->frames[3]; + s->frames[3] = s->frames[4]; + s->frames[4] = NULL; + } else { + ret = AVERROR(ENOMEM); + } + + if (ret < 0) + return ret; + return ff_filter_frame(outlink, out); + } + + FF_FILTER_FORWARD_STATUS(inlink, outlink); + FF_FILTER_FORWARD_WANTED(outlink, inlink); + + return FFERROR_NOT_READY; +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + DedotContext *s = ctx->priv; + + for (int i = 0; i < 5; i++) + av_frame_free(&s->frames[i]); +} + +#define OFFSET(x) offsetof(DedotContext, x) +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_FILTERING_PARAM + +static const AVOption dedot_options[] = { + { "m", "set filtering mode", OFFSET( m), AV_OPT_TYPE_FLAGS, {.i64=3}, 0, 3, FLAGS, "m" }, + { "dot-crawl", 0, 0, AV_OPT_TYPE_CONST, {.i64=1}, 0, 0, FLAGS, "m" }, + { "cross-luminance", 0, 0, AV_OPT_TYPE_CONST, {.i64=1}, 0, 0, FLAGS, "m" }, + { "rainbows", 0, 0, AV_OPT_TYPE_CONST, {.i64=2}, 0, 0, FLAGS, "m" }, + { "cross-color", 0, 0, AV_OPT_TYPE_CONST, {.i64=2}, 0, 0, FLAGS, "m" }, + { "lt", "set spatial luma threshold", OFFSET(lt), AV_OPT_TYPE_FLOAT, {.dbl=.079}, 0, 1, FLAGS }, + { "tl", "set tolerance for temporal luma", OFFSET(tl), AV_OPT_TYPE_FLOAT, {.dbl=.079}, 0, 1, FLAGS }, + { "tc", "set tolerance for chroma temporal variation", OFFSET(tc), AV_OPT_TYPE_FLOAT, {.dbl=.058}, 0, 1, FLAGS }, + { "ct", "set temporal chroma threshold", OFFSET(ct), AV_OPT_TYPE_FLOAT, {.dbl=.019}, 0, 1, FLAGS }, + { NULL }, +}; + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, + { NULL } +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_output, + }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(dedot); + +AVFilter ff_vf_dedot = { + .name = "dedot", + .description = NULL_IF_CONFIG_SMALL("Reduce cross-luminance and cross-color."), + .priv_size = sizeof(DedotContext), + .priv_class = &dedot_class, + .query_formats = query_formats, + .activate = activate, + .uninit = uninit, + .inputs = inputs, + .outputs = outputs, + .flags = AVFILTER_FLAG_SUPPORT_TIMELINE_INTERNAL, +};