From patchwork Sun Apr 28 14:42:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 12936 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 1F2B3446D3C for ; Sun, 28 Apr 2019 17:51:13 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0246068A5DF; Sun, 28 Apr 2019 17:51:13 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id F2FE36805D0 for ; Sun, 28 Apr 2019 17:51:05 +0300 (EEST) Received: by mail-wr1-f67.google.com with SMTP id r6so11916537wrm.4 for ; Sun, 28 Apr 2019 07:51:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=9mU6W0eUGLkZHedaELXLhD2lslujcuCqdkvvh2jdIPg=; b=QqmXVndmYeWVYQtAu9zm8B3E9RBbaiCwR6d4ftdIv4TG0vV7veBPBiFNmkVZr2DIfL UqJPYcT+6vZzUAMeLSGU5UeVGurp73IPOZqYLSi6jLRhtd9fCN7HnkV0aAICPsxhf04+ tNDivGcr4jEh2qytsMv5w5D6eSD1xb1G0kVHMuZXFyk/uiKhuY4Er81lQ4V8kYnnwtE7 D2Fu29XGqRUWFyN2D283yj3eytobRFb9f9Vr0FL46geJy7ZIlvJIreT3vYeu0eFL8o+J KxDjVjuAGZvwhmVyMd++epf3xw4L5C//V2jcsE/WsUEmMBbgPIIZOc8yZDNHBWEcFSZn viNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=9mU6W0eUGLkZHedaELXLhD2lslujcuCqdkvvh2jdIPg=; b=NuDjbUXISls9tf7V+sj9/MPdc8PV81qsnHnsTs06pqI2Ery+h3A1G9+ybmLALPulF8 2rHPB5Hulc/oDDu5wAnK0KQ8T/rKif8SwviUNO9nV7lcWjZGUmhuY41N5+RO3/PnLT2V ze4Z9JollKsA95gcP3RjG6qnFK4DFGAXYTqyQxgBXusnJEHa+ORftpad7Xn6Z9QjwW3+ yqSoQMukUNqnUdzLB3qad7o4jbNIIV4KBKN9TMxozwNy4O9t9oQbMwJk7BU+d7/Uq3ES FCW4hTszke9z/+vldyZb5u3Bu34FLY/lcP7XKwMWnDQe81mbEef8hX5t3TqHr9K0m5w0 /slw== X-Gm-Message-State: APjAAAUPIcJi0pCpaoAmjiktALV21E7AIAhyZB0cokk5xUlNjM0tIA/s 1wdJDB/dwO2dTGdr47dV/M2UpwmN X-Google-Smtp-Source: APXvYqwG8kj0sloPZgY1fcZZnRD0s11RXjUI33+B2XTlXlT+EE9/1zFzIoXhDraPhGXeJiZrRQ1Clg== X-Received: by 2002:adf:f7d0:: with SMTP id a16mr11512879wrq.211.1556462577758; Sun, 28 Apr 2019 07:42:57 -0700 (PDT) Received: from localhost.localdomain ([37.244.241.103]) by smtp.gmail.com with ESMTPSA id v25sm16282195wrv.7.2019.04.28.07.42.56 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 28 Apr 2019 07:42:56 -0700 (PDT) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Sun, 28 Apr 2019 16:42:45 +0200 Message-Id: <20190428144245.18024-1-onemda@gmail.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH] avfilter: add bilateral filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Paul B Mahol --- doc/filters.texi | 25 ++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_gblur.c | 257 ++++++++++++++++++++++++++++++++++++++- 4 files changed, 280 insertions(+), 4 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 14c33ecf90..04ca946d25 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -5953,6 +5953,31 @@ The filter accepts the following option: Set the minimal luminance value. Default is @code{16}. @end table +@section bilateral +Apply bilateral filter, spatial smoothing while preserving edges. + +The filter accepts the following options: +@table @option +@item sigmaS +Set sigma of gaussian function to calculate spatial weight. +Allowed range is 0 to 1024. Default is 3. + +@item sigmaR +Set sigma of gaussian function to calculate range weight. +Allowed range is 0 to 1024. Default is 0.5 + +@item steps +Set number of gaussian steps. Default is 1. +Alowed range is from 1 to 6. + +@item bsteps +Set number of bilateral steps. Default is 4. +Alowed range is from 2 to 256. + +@item planes +Set planes to filter. Default is first only. +@end table + @section bitplanenoise Show and measure bit plane noise. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 59d12ce069..a2e1477313 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -160,6 +160,7 @@ OBJS-$(CONFIG_AVGBLUR_OPENCL_FILTER) += vf_avgblur_opencl.o opencl.o \ opencl/avgblur.o boxblur.o OBJS-$(CONFIG_BBOX_FILTER) += bbox.o vf_bbox.o OBJS-$(CONFIG_BENCH_FILTER) += f_bench.o +OBJS-$(CONFIG_BILATERAL_FILTER) += vf_gblur.o OBJS-$(CONFIG_BITPLANENOISE_FILTER) += vf_bitplanenoise.o OBJS-$(CONFIG_BLACKDETECT_FILTER) += vf_blackdetect.o OBJS-$(CONFIG_BLACKFRAME_FILTER) += vf_blackframe.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index ae725cb0e0..931ff9d6c5 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -150,6 +150,7 @@ extern AVFilter ff_vf_avgblur; extern AVFilter ff_vf_avgblur_opencl; extern AVFilter ff_vf_bbox; extern AVFilter ff_vf_bench; +extern AVFilter ff_vf_bilateral; extern AVFilter ff_vf_bitplanenoise; extern AVFilter ff_vf_blackdetect; extern AVFilter ff_vf_blackframe; diff --git a/libavfilter/vf_gblur.c b/libavfilter/vf_gblur.c index f77a3fffc3..778163f53e 100644 --- a/libavfilter/vf_gblur.c +++ b/libavfilter/vf_gblur.c @@ -38,6 +38,8 @@ typedef struct GBlurContext { float sigma; float sigmaV; + float sigmaS; + float sigmaR; int steps; int planes; @@ -45,6 +47,8 @@ typedef struct GBlurContext { int planewidth[4]; int planeheight[4]; float *buffer; + float *den; + float *num; float boundaryscale; float boundaryscaleV; float postscale; @@ -52,6 +56,11 @@ typedef struct GBlurContext { float nu; float nuV; int nb_planes; + + float lut[65536]; + int blut[256]; + float *bbufers[256]; + int bsteps; } GBlurContext; #define OFFSET(x) offsetof(GBlurContext, x) @@ -212,7 +221,8 @@ static int query_formats(AVFilterContext *ctx) static int config_input(AVFilterLink *inlink) { const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(inlink->format); - GBlurContext *s = inlink->dst->priv; + AVFilterContext *ctx = inlink->dst; + GBlurContext *s = ctx->priv; s->depth = desc->comp[0].depth; s->planewidth[1] = s->planewidth[2] = AV_CEIL_RSHIFT(inlink->w, desc->log2_chroma_w); @@ -222,9 +232,11 @@ static int config_input(AVFilterLink *inlink) s->nb_planes = av_pix_fmt_count_planes(inlink->format); - s->buffer = av_malloc_array(inlink->w, inlink->h * sizeof(*s->buffer)); - if (!s->buffer) - return AVERROR(ENOMEM); + if (strcmp(ctx->filter->name, "bilateral")) { + s->buffer = av_malloc_array(inlink->w, inlink->h * sizeof(*s->buffer)); + if (!s->buffer) + return AVERROR(ENOMEM); + } if (s->sigmaV < 0) { s->sigmaV = s->sigma; @@ -332,10 +344,19 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) static av_cold void uninit(AVFilterContext *ctx) { GBlurContext *s = ctx->priv; + int i; + av_freep(&s->den); + av_freep(&s->num); av_freep(&s->buffer); + + for (i = 0; i < s->bsteps; i++) { + av_freep(&s->bbufers[i]); + } } +#if CONFIG_GBLUR_FILTER + static const AVFilterPad gblur_inputs[] = { { .name = "default", @@ -365,3 +386,231 @@ AVFilter ff_vf_gblur = { .outputs = gblur_outputs, .flags = AVFILTER_FLAG_SUPPORT_TIMELINE_GENERIC | AVFILTER_FLAG_SLICE_THREADS, }; +#endif /* CONFIG_GBLUR_FILTER */ + +#if CONFIG_BILATERAL_FILTER + +static const AVOption bilateral_options[] = { + { "sigmaS", "set sigma S", OFFSET(sigmaS), AV_OPT_TYPE_FLOAT, {.dbl=3}, 0.0, 1024, FLAGS }, + { "sigmaR", "set sigma R", OFFSET(sigmaR), AV_OPT_TYPE_FLOAT, {.dbl=.5},0.0, 1024, FLAGS }, + { "steps", "set number of gaussian steps", OFFSET(steps), AV_OPT_TYPE_INT, {.i64=1}, 1, 6, FLAGS }, + { "bsteps", "set number of bilateral steps", OFFSET(bsteps), AV_OPT_TYPE_INT, {.i64=4}, 2, 256, FLAGS }, + { "planes", "set planes to filter", OFFSET(planes), AV_OPT_TYPE_INT, {.i64=1}, 0, 0xF, FLAGS }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(bilateral); + +static float lut_lookup(const float *lut, const int val1, const int val2) +{ + return lut[val1 > val2 ? val1 - val2 : val2 - val1]; +} + +static int bilateral_filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx = inlink->dst; + GBlurContext *s = ctx->priv; + AVFilterLink *outlink = ctx->outputs[0]; + AVFrame *out; + int plane; + + s->sigma = s->sigmaS; + set_params(s->sigmaS, s->steps, &s->postscale, &s->boundaryscale, &s->nu); + set_params(s->sigmaS, s->steps, &s->postscaleV, &s->boundaryscaleV, &s->nuV); + + if (av_frame_is_writable(in)) { + out = in; + } else { + out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!out) { + av_frame_free(&in); + return AVERROR(ENOMEM); + } + av_frame_copy_props(out, in); + } + + for (plane = 0; plane < s->nb_planes; plane++) { + const int height = s->planeheight[plane]; + const int width = s->planewidth[plane]; + const uint8_t *src = in->data[plane]; + const uint16_t *src16 = (const uint16_t *)in->data[plane]; + const int src_linesize = in->linesize[plane]; + const int src16_linesize = in->linesize[plane] / 2; + uint8_t *dst = out->data[plane]; + uint16_t *dst16 = (uint16_t *)out->data[plane]; + const int dst_linesize = out->linesize[plane]; + const int dst16_linesize = out->linesize[plane] / 2; + int k, y, x; + + if (!s->sigmaS || !(s->planes & (1 << plane))) { + if (out != in) + av_image_copy_plane(out->data[plane], out->linesize[plane], + in->data[plane], in->linesize[plane], + width * ((s->depth + 7) / 8), height); + continue; + } + + for (k = 0; k < s->bsteps; k++) { + float *num = s->num; + float *den = s->den; + + src = in->data[plane]; + src16 = (uint16_t *)in->data[plane]; + if (s->depth == 8) { + for (y = 0; y < height; y++) { + for (x = 0; x < width; x++) { + den[x] = lut_lookup(s->lut, s->blut[k], src[x]); + num[x] = den[x] * src[x]; + } + den += width; + num += width; + src += src_linesize; + } + } else { + for (y = 0; y < height; y++) { + for (x = 0; x < width; x++) { + den[x] = lut_lookup(s->lut, s->blut[k], src16[x]); + num[x] = den[x] * src16[x]; + } + den += width; + num += width; + src16 += src16_linesize; + } + } + + s->buffer = s->den; + gaussianiir2d(ctx, plane); + s->buffer = s->num; + gaussianiir2d(ctx, plane); + s->buffer = NULL; + + num = s->num; + den = s->den; + for (y = 0; y < height; y++) { + for (x = 0; x < width; x++) { + s->bbufers[k][y * width + x] = den[x] == 0 ? 0 : num[x] / den[x]; + } + num += width; + den += width; + } + } + + src = in->data[plane]; + src16 = (uint16_t *)in->data[plane]; + for (y = 0; y < height; y++) { + for (x = 0; x < width; x++) { + for (k = 0; k < s->bsteps - 2; k++) { + if (s->depth == 8) { + if (src[x] < s->blut[k + 1] && src[x] >= s->blut[k]) + break; + } else { + if (src16[x] < s->blut[k + 1] && src16[x] >= s->blut[k]) + break; + } + } + + if (s->depth == 8) { + dst[x] = av_clip_uint8(((s->blut[k + 1] - src[x]) * s->bbufers[k][x + y * width] + + (src[x] - s->blut[k]) * s->bbufers[k + 1][x + y * width]) / + (s->blut[k + 1] - s->blut[k]) + 0.5f); + } else { + dst16[x] = av_clip_uintp2_c(((s->blut[k + 1] - src16[x]) * s->bbufers[k][x + y * width] + + (src16[x] - s->blut[k]) * s->bbufers[k + 1][x + y * width]) / + (s->blut[k + 1] - s->blut[k]) + 0.5f, s->depth); + } + } + + dst += dst_linesize; + src += src_linesize; + src16 += src16_linesize; + dst16 += dst16_linesize; + } + } + + if (out != in) + av_frame_free(&in); + return ff_filter_frame(outlink, out); +} + +static inline float normalized_gaussian(float x, float sigma) +{ + x /= sigma; + return expf(x*x/-2.f) / (sqrtf(M_PI * 2.f) * sigma); +} + +static void create_lut(const int range, float sigmaR, float *lut) +{ + const int upper = FFMIN(range, (int)(sigmaR * 8.f * range + 0.5f)); + int max = range + 1; + int i; + + for (i = 0; i <= upper; i++) { + lut[i] = normalized_gaussian(i / (float)range, sigmaR); + } + if (i < max) { + const float value = lut[upper]; + + for (; i < max; i++) { + lut[i] = value; + } + } +} + +static int bilateral_config_input(AVFilterLink *inlink) +{ + GBlurContext *s = inlink->dst->priv; + int ret, i; + float max; + + ret = config_input(inlink); + if (ret < 0) + return ret; + + max = (1 << s->depth) - 1; + for (i = 0; i < s->bsteps; i++) { + s->blut[i] = max * i / (s->bsteps - 1) + 0.5; + s->bbufers[i] = av_malloc_array(inlink->w, inlink->h * sizeof(float)); + if (!s->bbufers[i]) + return AVERROR(ENOMEM); + } + + s->num = av_malloc_array(inlink->w, inlink->h * sizeof(*s->num)); + s->den = av_malloc_array(inlink->w, inlink->h * sizeof(*s->den)); + if (!s->den || !s->num) + return AVERROR(ENOMEM); + + create_lut(max, s->sigmaR, s->lut); + + return 0; +} + +static const AVFilterPad bilateral_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = bilateral_config_input, + .filter_frame = bilateral_filter_frame, + }, + { NULL } +}; + +static const AVFilterPad bilateral_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, + { NULL } +}; + +AVFilter ff_vf_bilateral = { + .name = "bilateral", + .description = NULL_IF_CONFIG_SMALL("Apply Bilateral filter."), + .priv_size = sizeof(GBlurContext), + .priv_class = &bilateral_class, + .uninit = uninit, + .query_formats = query_formats, + .inputs = bilateral_inputs, + .outputs = bilateral_outputs, + .flags = AVFILTER_FLAG_SUPPORT_TIMELINE_GENERIC | AVFILTER_FLAG_SLICE_THREADS, +}; +#endif /* CONFIG_BILATERAL_FILTER */