From patchwork Tue May 8 11:08:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 8865 Delivered-To: ffmpegpatchwork@gmail.com Received: by 2002:a02:155:0:0:0:0:0 with SMTP id c82-v6csp3880738jad; Tue, 8 May 2018 04:08:51 -0700 (PDT) X-Google-Smtp-Source: AB8JxZoRkfItZf3pJg1+7gabPPIOXlZBooOj4I/36pnBMcQSL8JDHlPRm4IXrxnvVJjVIOv8x7v9 X-Received: by 2002:adf:9607:: with SMTP id b7-v6mr30539232wra.129.1525777731770; Tue, 08 May 2018 04:08:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1525777731; cv=none; d=google.com; s=arc-20160816; b=kmNIfLduNNyKiPHPEUZEff5xdNooQaJ1rzkxR969ffXIwCzgRlSs+iBcPTUb0LhFV9 CD7QafBoF40c9VJoye8qoIVi+a1n6IaYXoE85xEg3s8hsdTosLBRCYu2+IkuTkFZA/yK +lkRa7UlU+6jhWPzgyzF1NIlHZfw+Mo/WvhrS2khBUrbYFAwyBkY05L8N1VIvne0auiP Ko6gk0toOetqYMYnuNKxSYlHR93TEPjGWeZA1Pc0ueCLrg91G3pQky/9QUU6gm/Q3oZM suq5rEdK+kwUdnaTcpZfZC6y55Hcmm5CjzG3fcKlHt/scJHCfR+6CGB3hwxU/+GuPWoO wlwQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=YkKWzAw4HDxzYOdLtjOQSonfiySKJUDKHOHXllMWphU=; b=V4b21fRa50lNF1xWxaYJiLAik2mwI4Hwnse5hKFEGqQefA/0rkUOcG0Zee8Ya66MrT Y3ILf7mK50Xoh4RuNO0q4CIXAH8nu6wuBIfwKYGyiOFTGjDcqLAUCHcZ7/QaanTLx+9G kfz5UGBZWhap1qwI23aRBiU5qbzwWnlc46VUQT3CQZymcsbwSy41OE1ABheAPZgR9uQo 6zjoiKvQQq9QJxo7Jm4QnmBzVtWtMPIMDuGM+7UFZQrRt4xXDCAhXyPR9X0IYZvoaF94 Dt89bfpODt95q6Co/tpKxf8IsSdlmEpXAv5LP/SJCz4tS2Cp2E9+tVdawtUvjPOeZPj8 yWWQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=qV7CPheu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id e196si5877863wmg.184.2018.05.08.04.08.51; Tue, 08 May 2018 04:08:51 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=qV7CPheu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8463F68A700; Tue, 8 May 2018 14:08:14 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f43.google.com (mail-wm0-f43.google.com [74.125.82.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 85F3F68A6F2 for ; Tue, 8 May 2018 14:08:08 +0300 (EEST) Received: by mail-wm0-f43.google.com with SMTP id j5-v6so20763359wme.5 for ; Tue, 08 May 2018 04:08:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=mxAxyFg1FSrnl+X03tiMX3x3Ne5Yc+1wrAyk3vEYW7s=; b=qV7CPheusRCLY88UqPuqE1CRZscOcoxGO7R8pC3sGosW7lLY3KfXjVU3BRgmxXtE3F lPdd2pfrf+E93dp9ADRg2mrLCAVPD1ncab17iggkGBrGTzF/77Zro+zowwyjtaFOYt7K tEhu1xCDxcHZN9Aadt2uWyhkm0TD4V+FX/3JrXVlqmAk+l5yX8kHyk9hNjF76tUsRpaW IQW/DSIi8RBDZtvbsP7VEbuUSVwazrz6/CuPqi4gV6nSBf+ZxUmT9uaeN4s89mcej4cK bBId88Vji9l2OoDQSYYjpZDarCeBpVcxI5ypv3QK7cpEjqO8kqbuSrWGGoWLs6TusnSf /wrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=mxAxyFg1FSrnl+X03tiMX3x3Ne5Yc+1wrAyk3vEYW7s=; b=Mx1OqBcvG/UmZwF07lvIwaDU2Q4xqCdRnIw/TLNIKHsKcj02QgBHI1RNuCrE3PNcL5 Bsqh23D9vUM8SKYTkopOvpiPMGXH/qPUOFMPXIqYEOLdxEg7PXU3Z0kuXWzNfDy53NDn sqe6/duYh327UyHk/S6CaJjSOub1bEF24k5yl8d5H1vL5yIWukStkbevREd/t0V1w+Aj eXkeN6j+NHLYCNdQ9XMF9/9VnwIrAGWQfac9zwx5EUjyFJW5QGbZhO369VivGVkW6R0b VVXCnP8stKId1r7gtcVFW4Il2agkGJKAzEmxS7E2wXzHEy7GGgyE8k961mtzegKv2jAe xwWw== X-Gm-Message-State: ALKqPwffGMVHJrg/7fajCfsjsxqW2TN0xiWnDEJIDhm/owOaQpRBoYFe ZoiDD3x9eZsjHSFihLqgg4f7Tg== X-Received: by 2002:a1c:8b88:: with SMTP id n130-v6mr2954450wmd.8.1525777722767; Tue, 08 May 2018 04:08:42 -0700 (PDT) Received: from localhost.localdomain ([94.250.174.60]) by smtp.gmail.com with ESMTPSA id d54-v6sm34843189wrd.94.2018.05.08.04.08.41 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 08 May 2018 04:08:42 -0700 (PDT) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Tue, 8 May 2018 13:08:32 +0200 Message-Id: <20180508110832.22270-1-onemda@gmail.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180507135929.22485-1-onemda@gmail.com> References: <20180507135929.22485-1-onemda@gmail.com> Subject: [FFmpeg-devel] [PATCH] avfilter: add fftdnoiz filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Paul B Mahol --- configure | 2 + doc/filters.texi | 32 +++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_fftdnoiz.c | 691 ++++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 727 insertions(+) create mode 100644 libavfilter/vf_fftdnoiz.c diff --git a/configure b/configure index 7c143238a8..093d854ef1 100755 --- a/configure +++ b/configure @@ -3325,6 +3325,8 @@ elbg_filter_deps="avcodec" eq_filter_deps="gpl" fftfilt_filter_deps="avcodec" fftfilt_filter_select="rdft" +fftdnoiz_filter_deps="avcodec" +fftdnoiz_filter_select="fft" find_rect_filter_deps="avcodec avformat gpl" firequalizer_filter_deps="avcodec" firequalizer_filter_select="rdft" diff --git a/doc/filters.texi b/doc/filters.texi index 33e27e10f3..8f48971e57 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -8696,6 +8696,38 @@ fftfilt=dc_Y=0:weight_Y='exp(-4 * ((Y+X)/(W+H)))' @end itemize +@section fftdnoiz +Denoise frames using 3D FFT (frequency domain filtering). + +The filter accepts the following options: + +@table @option +@item sigma +Set the noise sigma constant. This sets denoising strength. +Default value is 1. Allowed range is from 0 to 7. + +@item amount +Set amount of denoising. By default all detected noise is removed. +Default value is 1. Allowed range is from 0 to 1. + +@item block +Set size of block, Default is 4, can be 4, 5 or 6. +Actual size of block in pixels is 2 to power of @var{block}, so by default +block size in pixels is 2^4 which is 16. + +@item overlap +Set block overlap. Default is 0.5. Allowed range is from 0.2 to 0.8. + +@item prev +Set number of previous frames to use for denoising. By default is set to 0. + +@item next +Set number of next frames to to use for denoising. By default is set to 0. + +@item planes +Set planes which will be filtered, by default are all available except alpha. +@end table + @section field Extract a single field from an interlaced image using stride diff --git a/libavfilter/Makefile b/libavfilter/Makefile index a416fa1fae..92f526a275 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -201,6 +201,7 @@ OBJS-$(CONFIG_EQ_FILTER) += vf_eq.o OBJS-$(CONFIG_EROSION_FILTER) += vf_neighbor.o OBJS-$(CONFIG_EXTRACTPLANES_FILTER) += vf_extractplanes.o OBJS-$(CONFIG_FADE_FILTER) += vf_fade.o +OBJS-$(CONFIG_FFTDNOIZ_FILTER) += vf_fftdnoiz.o OBJS-$(CONFIG_FFTFILT_FILTER) += vf_fftfilt.o OBJS-$(CONFIG_FIELD_FILTER) += vf_field.o OBJS-$(CONFIG_FIELDHINT_FILTER) += vf_fieldhint.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 42f956b3bf..f3a76c4652 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -193,6 +193,7 @@ extern AVFilter ff_vf_eq; extern AVFilter ff_vf_erosion; extern AVFilter ff_vf_extractplanes; extern AVFilter ff_vf_fade; +extern AVFilter ff_vf_fftdnoiz; extern AVFilter ff_vf_fftfilt; extern AVFilter ff_vf_field; extern AVFilter ff_vf_fieldhint; diff --git a/libavfilter/vf_fftdnoiz.c b/libavfilter/vf_fftdnoiz.c new file mode 100644 index 0000000000..272a37cd40 --- /dev/null +++ b/libavfilter/vf_fftdnoiz.c @@ -0,0 +1,691 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include + +#include "libavutil/avassert.h" +#include "libavutil/common.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "internal.h" +#include "libavcodec/avfft.h" + +enum BufferTypes { + CURRENT, + PREV, + NEXT, + BSIZE +}; + +typedef struct PlaneContext { + int planewidth, planeheight; + int nox, noy; + int b; + int o; + float n; + + float *buffer[BSIZE]; + FFTComplex *hdata, *vdata; + int data_linesize; + int buffer_linesize; + + FFTContext *fft, *ifft; +} PlaneContext; + +typedef struct FFTdnoizContext { + const AVClass *class; + + float sigma; + float amount; + int block_bits; + float overlap; + int nb_prev; + int nb_next; + int planesf; + + AVFrame *prev, *cur, *next; + + int depth; + int nb_planes; + PlaneContext planes[4]; + + void (*import_row)(FFTComplex *dst, uint8_t *src, int rw); + void (*export_row)(FFTComplex *src, uint8_t *dst, int rw, float scale, int depth); +} FFTdnoizContext; + +#define OFFSET(x) offsetof(FFTdnoizContext, x) +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM +static const AVOption fftdnoiz_options[] = { + { "sigma", "set noise sigma", + OFFSET(sigma), AV_OPT_TYPE_FLOAT, {.dbl=1}, 0, 7, .flags = FLAGS }, + { "amount", "set amount of denoising", + OFFSET(amount), AV_OPT_TYPE_FLOAT, {.dbl=1}, 0.01, 1, .flags = FLAGS }, + { "block", "set block log2(size)", + OFFSET(block_bits), AV_OPT_TYPE_INT, {.i64=4}, 4, 6, .flags = FLAGS }, + { "overlap", "set block overlap", + OFFSET(overlap), AV_OPT_TYPE_FLOAT, {.dbl=0.5}, 0.2, 0.8, .flags = FLAGS }, + { "prev", "set number of previous frames for temporal denoising", + OFFSET(nb_prev), AV_OPT_TYPE_INT, {.i64=0}, 0, 1, .flags = FLAGS }, + { "next", "set number of next frames for temporal denoising", + OFFSET(nb_next), AV_OPT_TYPE_INT, {.i64=0}, 0, 1, .flags = FLAGS }, + { "planes", "set planes to filter", + OFFSET(planesf), AV_OPT_TYPE_INT, {.i64=7}, 0, 15, .flags = FLAGS }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(fftdnoiz); + +static av_cold int init(AVFilterContext *ctx) +{ + FFTdnoizContext *s = ctx->priv; + int i; + + for (i = 0; i < 4; i++) { + PlaneContext *p = &s->planes[i]; + + p->fft = av_fft_init(s->block_bits, 0); + p->ifft = av_fft_init(s->block_bits, 1); + if (!p->fft || !p->ifft) + return AVERROR(ENOMEM); + } + + return 0; +} + +static int query_formats(AVFilterContext *ctx) +{ + static const enum AVPixelFormat pix_fmts[] = { + AV_PIX_FMT_GRAY8, AV_PIX_FMT_GRAY9, + AV_PIX_FMT_GRAY10, AV_PIX_FMT_GRAY12, + AV_PIX_FMT_GRAY14, AV_PIX_FMT_GRAY16, + AV_PIX_FMT_YUV410P, AV_PIX_FMT_YUV411P, + AV_PIX_FMT_YUV420P, AV_PIX_FMT_YUV422P, + AV_PIX_FMT_YUV440P, AV_PIX_FMT_YUV444P, + AV_PIX_FMT_YUVJ420P, AV_PIX_FMT_YUVJ422P, + AV_PIX_FMT_YUVJ440P, AV_PIX_FMT_YUVJ444P, + AV_PIX_FMT_YUVJ411P, + AV_PIX_FMT_YUV420P9, AV_PIX_FMT_YUV422P9, AV_PIX_FMT_YUV444P9, + AV_PIX_FMT_YUV420P10, AV_PIX_FMT_YUV422P10, AV_PIX_FMT_YUV444P10, + AV_PIX_FMT_YUV440P10, + AV_PIX_FMT_YUV444P12, AV_PIX_FMT_YUV422P12, AV_PIX_FMT_YUV420P12, + AV_PIX_FMT_YUV440P12, + AV_PIX_FMT_YUV444P14, AV_PIX_FMT_YUV422P14, AV_PIX_FMT_YUV420P14, + AV_PIX_FMT_YUV420P16, AV_PIX_FMT_YUV422P16, AV_PIX_FMT_YUV444P16, + AV_PIX_FMT_GBRP, AV_PIX_FMT_GBRP9, AV_PIX_FMT_GBRP10, + AV_PIX_FMT_GBRP12, AV_PIX_FMT_GBRP14, AV_PIX_FMT_GBRP16, + AV_PIX_FMT_NONE + }; + AVFilterFormats *fmts_list = ff_make_format_list(pix_fmts); + if (!fmts_list) + return AVERROR(ENOMEM); + return ff_set_common_formats(ctx, fmts_list); +} + +typedef struct ThreadData { + float *src, *dst; +} ThreadData; + +static void import_row8(FFTComplex *dst, uint8_t *src, int rw) +{ + int j; + + for (j = 0; j < rw; j++) { + dst[j].re = src[j]; + dst[j].im = 0; + } +} + +static void export_row8(FFTComplex *src, uint8_t *dst, int rw, float scale, int depth) +{ + int j; + + for (j = 0; j < rw; j++) + dst[j] = av_clip_uint8(src[j].re * scale); +} + +static void import_row16(FFTComplex *dst, uint8_t *srcp, int rw) +{ + uint16_t *src = (uint16_t *)srcp; + int j; + + for (j = 0; j < rw; j++) { + dst[j].re = src[j]; + dst[j].im = 0; + } +} + +static void export_row16(FFTComplex *src, uint8_t *dstp, int rw, float scale, int depth) +{ + uint16_t *dst = (uint16_t *)dstp; + int j; + + for (j = 0; j < rw; j++) + dst[j] = av_clip_uintp2(src[j].re * scale, depth); +} + +static int config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + const AVPixFmtDescriptor *desc; + FFTdnoizContext *s = ctx->priv; + int i; + + desc = av_pix_fmt_desc_get(inlink->format); + s->depth = desc->comp[0].depth; + + if (s->depth <= 8) { + s->import_row = import_row8; + s->export_row = export_row8; + } else { + s->import_row = import_row16; + s->export_row = export_row16; + s->sigma *= 1 << (s->depth - 8); + } + + s->planes[1].planewidth = s->planes[2].planewidth = AV_CEIL_RSHIFT(inlink->w, desc->log2_chroma_w); + s->planes[0].planewidth = s->planes[3].planewidth = inlink->w; + s->planes[1].planeheight = s->planes[2].planeheight = AV_CEIL_RSHIFT(inlink->h, desc->log2_chroma_h); + s->planes[0].planeheight = s->planes[3].planeheight = inlink->h; + + s->nb_planes = av_pix_fmt_count_planes(inlink->format); + + for (i = 0; i < s->nb_planes; i++) { + PlaneContext *p = &s->planes[i]; + int size; + + p->b = 1 << s->block_bits; + p->n = 1.f / (p->b * p->b); + p->o = p->b * s->overlap; + size = p->b - p->o; + p->nox = (p->planewidth + (size - 1)) / size; + p->noy = (p->planeheight + (size - 1)) / size; + + av_log(ctx, AV_LOG_DEBUG, "nox:%d noy:%d size:%d\n", p->nox, p->noy, size); + + p->buffer_linesize = p->b * p->nox * sizeof(FFTComplex); + p->buffer[CURRENT] = av_calloc(p->b * p->noy, p->buffer_linesize); + if (!p->buffer[CURRENT]) + return AVERROR(ENOMEM); + if (s->nb_prev > 0) { + p->buffer[PREV] = av_calloc(p->b * p->noy, p->buffer_linesize); + if (!p->buffer[PREV]) + return AVERROR(ENOMEM); + } + if (s->nb_next > 0) { + p->buffer[NEXT] = av_calloc(p->b * p->noy, p->buffer_linesize); + if (!p->buffer[NEXT]) + return AVERROR(ENOMEM); + } + p->data_linesize = 2 * p->b * sizeof(float); + p->hdata = av_calloc(p->b, p->data_linesize); + p->vdata = av_calloc(p->b, p->data_linesize); + if (!p->hdata || !p->vdata) + return AVERROR(ENOMEM); + } + + return 0; +} + +static void import_plane(FFTdnoizContext *s, + uint8_t *srcp, int src_linesize, + float *buffer, int buffer_linesize, int plane) +{ + PlaneContext *p = &s->planes[plane]; + const int width = p->planewidth; + const int height = p->planeheight; + const int block = p->b; + const int overlap = p->o; + const int size = block - overlap; + const int nox = p->nox; + const int noy = p->noy; + const int bpp = (s->depth + 7) / 8; + const int data_linesize = p->data_linesize / sizeof(FFTComplex); + FFTComplex *hdata = p->hdata; + FFTComplex *vdata = p->vdata; + int x, y, i, j; + + buffer_linesize /= sizeof(float); + for (y = 0; y < noy; y++) { + for (x = 0; x < nox; x++) { + const int rh = FFMIN(block, height - y * size); + const int rw = FFMIN(block, width - x * size); + uint8_t *src = srcp + src_linesize * y * size + x * size * bpp; + float *bdst = buffer + buffer_linesize * y * block + x * block * 2; + FFTComplex *ssrc, *dst = hdata; + + for (i = 0; i < rh; i++) { + s->import_row(dst, src, rw); + for (; j < block; j++) { + dst[j].re = dst[block - j - 1].re; + dst[j].im = 0; + } + av_fft_permute(p->fft, dst); + av_fft_calc(p->fft, dst); + + src += src_linesize; + dst += data_linesize; + } + + dst = hdata; + for (; i < block; i++) { + for (j = 0; j < block; j++) { + dst[j].re = dst[(block - i - 1) * data_linesize + j].re; + dst[j].im = dst[(block - i - 1) * data_linesize + j].im; + } + } + + ssrc = hdata; + dst = vdata; + for (i = 0; i < block; i++) { + for (j = 0; j < block; j++) + dst[j] = ssrc[j * data_linesize + i]; + av_fft_permute(p->fft, dst); + av_fft_calc(p->fft, dst); + memcpy(bdst, dst, block * sizeof(FFTComplex)); + + dst += data_linesize; + bdst += buffer_linesize; + } + } + } +} + +static void export_plane(FFTdnoizContext *s, + uint8_t *dstp, int dst_linesize, + float *buffer, int buffer_linesize, int plane) +{ + PlaneContext *p = &s->planes[plane]; + const int depth = s->depth; + const int bpp = (depth + 7) / 8; + const int width = p->planewidth; + const int height = p->planeheight; + const int block = p->b; + const int overlap = p->o; + const int hoverlap = overlap / 2; + const int size = block - overlap; + const int nox = p->nox; + const int noy = p->noy; + const int data_linesize = p->data_linesize / sizeof(FFTComplex); + const float scale = 1.f / (block * block); + FFTComplex *hdata = p->hdata; + FFTComplex *vdata = p->vdata; + int x, y, i, j; + + buffer_linesize /= sizeof(float); + for (y = 0; y < noy; y++) { + for (x = 0; x < nox; x++) { + const int woff = x == 0 ? 0 : hoverlap; + const int hoff = y == 0 ? 0 : hoverlap; + const int rw = x == 0 ? block : FFMIN(size, width - x * size - woff); + const int rh = y == 0 ? block : FFMIN(size, height - y * size - hoff); + float *bsrc = buffer + buffer_linesize * y * block + x * block * 2; + uint8_t *dst = dstp + dst_linesize * (y * size + hoff) + (x * size + woff) * bpp; + FFTComplex *hdst, *ddst = vdata; + + hdst = hdata; + for (i = 0; i < block; i++) { + memcpy(ddst, bsrc, block * sizeof(FFTComplex)); + av_fft_permute(p->ifft, ddst); + av_fft_calc(p->ifft, ddst); + for (j = 0; j < block; j++) { + hdst[j * data_linesize + i] = ddst[j]; + } + + ddst += data_linesize; + bsrc += buffer_linesize; + } + + hdst = hdata + hoff * data_linesize; + for (i = 0; i < rh; i++) { + av_fft_permute(p->ifft, hdst); + av_fft_calc(p->ifft, hdst); + s->export_row(hdst + woff, dst, rw, scale, depth); + + hdst += data_linesize; + dst += dst_linesize; + } + } + } +} + +static void filter_plane3d2(FFTdnoizContext *s, int plane, float *pbuffer, float *nbuffer) +{ + PlaneContext *p = &s->planes[plane]; + const int block = p->b; + const int nox = p->nox; + const int noy = p->noy; + const int buffer_linesize = p->buffer_linesize / sizeof(float); + const float sigma = s->sigma * s->sigma * block * block; + const float limit = 1.f - s->amount; + float *cbuffer = p->buffer[CURRENT]; + const float cfactor = sqrtf(3.f) * 0.5f; + const float scale = 1.f / 3.f; + int y, x, i, j; + + for (y = 0; y < noy; y++) { + for (x = 0; x < nox; x++) { + float *cbuff = cbuffer + buffer_linesize * y * block + x * block * 2; + float *pbuff = pbuffer + buffer_linesize * y * block + x * block * 2; + float *nbuff = nbuffer + buffer_linesize * y * block + x * block * 2; + + for (i = 0; i < block; i++) { + for (j = 0; j < block; j++) { + float sumr, sumi, difr, difi, mpr, mpi, mnr, mni; + float factor, power, sumpnr, sumpni; + + sumpnr = pbuff[2 * j ] + nbuff[2 * j ]; + sumpni = pbuff[2 * j + 1] + nbuff[2 * j + 1]; + sumr = cbuff[2 * j ] + sumpnr; + sumi = cbuff[2 * j + 1] + sumpni; + difr = cfactor * (nbuff[2 * j ] - pbuff[2 * j ]); + difi = cfactor * (pbuff[2 * j + 1] - nbuff[2 * j + 1]); + mpr = cbuff[2 * j ] - 0.5f * sumpnr + difi; + mnr = mpr - difi - difi; + mpi = cbuff[2 * j + 1] - 0.5f * sumpni + difr; + mni = mpi - difr - difr; + power = sumr * sumr + sumi * sumi + 1e-15f; + factor = FFMAX((power - sigma) / power, limit); + sumr *= factor; + sumi *= factor; + power = mpr * mpr + mpi * mpi + 1e-15f; + factor = FFMAX((power - sigma) / power, limit); + mpr *= factor; + mpi *= factor; + power = mnr * mnr + mni * mni + 1e-15f; + factor = FFMAX((power - sigma) / power, limit); + mnr *= factor; + mni *= factor; + cbuff[2 * j ] = (sumr + mpr + mnr) * scale; + cbuff[2 * j + 1] = (sumi + mpi + mni) * scale; + + } + + cbuff += buffer_linesize; + pbuff += buffer_linesize; + nbuff += buffer_linesize; + } + } + } +} + +static void filter_plane3d1(FFTdnoizContext *s, int plane, float *pbuffer) +{ + PlaneContext *p = &s->planes[plane]; + const int block = p->b; + const int nox = p->nox; + const int noy = p->noy; + const int buffer_linesize = p->buffer_linesize / sizeof(float); + const float sigma = s->sigma * s->sigma * block * block; + const float limit = 1.f - s->amount; + float *cbuffer = p->buffer[CURRENT]; + int y, x, i, j; + + for (y = 0; y < noy; y++) { + for (x = 0; x < nox; x++) { + float *cbuff = cbuffer + buffer_linesize * y * block + x * block * 2; + float *pbuff = pbuffer + buffer_linesize * y * block + x * block * 2; + + for (i = 0; i < block; i++) { + for (j = 0; j < block; j++) { + float factor, power, re, im, pre, pim; + float sumr, sumi, difr, difi; + + re = cbuff[j * 2 ]; + pre = pbuff[j * 2 ]; + im = cbuff[j * 2 + 1]; + pim = pbuff[j * 2 + 1]; + + sumr = re + pre; + sumi = im + pim; + difr = re - pre; + difi = im - pim; + + power = sumr * sumr + sumi * sumi + 1e-15f; + factor = FFMAX(limit, (power - sigma) / power); + sumr *= factor; + sumi *= factor; + power = difr * difr + difi * difi + 1e-15f; + factor = FFMAX(limit, (power - sigma) / power); + difr *= factor; + difi *= factor; + + cbuff[j * 2 ] = (sumr + difr) * 0.5f; + cbuff[j * 2 + 1] = (sumi + difi) * 0.5f; + } + + cbuff += buffer_linesize; + pbuff += buffer_linesize; + } + } + } +} + +static void filter_plane2d(FFTdnoizContext *s, int plane) +{ + PlaneContext *p = &s->planes[plane]; + const int block = p->b; + const int nox = p->nox; + const int noy = p->noy; + const int buffer_linesize = p->buffer_linesize / 4; + const float sigma = s->sigma * s->sigma * block * block; + const float limit = 1.f - s->amount; + float *buffer = p->buffer[CURRENT]; + int y, x, i, j; + + for (y = 0; y < noy; y++) { + for (x = 0; x < nox; x++) { + float *buff = buffer + buffer_linesize * y * block + x * block * 2; + + for (i = 0; i < block; i++) { + for (j = 0; j < block; j++) { + float factor, power, re, im; + + re = buff[j * 2 ]; + im = buff[j * 2 + 1]; + power = re * re + im * im + 1e-15f; + factor = FFMAX(limit, (power - sigma) / power); + buff[j * 2 ] *= factor; + buff[j * 2 + 1] *= factor; + } + + buff += buffer_linesize; + } + } + } +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx = inlink->dst; + FFTdnoizContext *s = ctx->priv; + AVFilterLink *outlink = ctx->outputs[0]; + int direct, plane; + AVFrame *out; + + if (s->nb_next > 0 && s->nb_prev > 0) { + av_frame_free(&s->prev); + s->prev = s->cur; + s->cur = s->next; + s->next = in; + + if (!s->cur || !s->prev) + return 0; + } else if (s->nb_next > 0) { + av_frame_free(&s->cur); + s->cur = s->next; + s->next = in; + + if (!s->cur) + return 0; + } else if (s->nb_prev > 0) { + av_frame_free(&s->prev); + s->prev = s->cur; + s->cur = in; + + if (!s->prev) + return 0; + } else { + s->cur = in; + } + + if (av_frame_is_writable(in) && s->nb_next == 0 && s->nb_prev == 0) { + direct = 1; + out = in; + } else { + direct = 0; + out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!out) + return AVERROR(ENOMEM); + av_frame_copy_props(out, s->prev ? s->prev : s->cur); + } + + for (plane = 0; plane < s->nb_planes; plane++) { + PlaneContext *p = &s->planes[plane]; + + if (!((1 << plane) & s->planesf) || ctx->is_disabled) { + if (!direct) + av_image_copy_plane(out->data[plane], out->linesize[plane], + s->cur->data[plane], s->cur->linesize[plane], + p->planewidth, p->planeheight); + continue; + } + + if (s->next) { + import_plane(s, s->next->data[plane], s->next->linesize[plane], + p->buffer[NEXT], p->buffer_linesize, plane); + } + + if (s->prev) { + import_plane(s, s->prev->data[plane], s->prev->linesize[plane], + p->buffer[PREV], p->buffer_linesize, plane); + } + + import_plane(s, s->cur->data[plane], s->cur->linesize[plane], + p->buffer[CURRENT], p->buffer_linesize, plane); + + if (s->next && s->prev) { + filter_plane3d2(s, plane, p->buffer[PREV], p->buffer[NEXT]); + } else if (s->next) { + filter_plane3d1(s, plane, p->buffer[NEXT]); + } else if (s->prev) { + filter_plane3d1(s, plane, p->buffer[PREV]); + } else { + filter_plane2d(s, plane); + } + + export_plane(s, out->data[plane], out->linesize[plane], + p->buffer[CURRENT], p->buffer_linesize, plane); + } + + if (s->nb_next == 0 && s->nb_prev == 0) { + if (direct) { + s->cur = NULL; + } else { + av_frame_free(&s->cur); + } + } + return ff_filter_frame(outlink, out); +} + +static int request_frame(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + FFTdnoizContext *s = ctx->priv; + int ret = 0; + + ret = ff_request_frame(ctx->inputs[0]); + + if (ret == AVERROR_EOF && (s->nb_next > 0 || s->nb_prev > 0)) { + int repeat = s->nb_next + s->nb_prev; + AVFrame *buf; + + while (repeat) { + if (s->next && s->nb_next > 0) + buf = av_frame_clone(s->next); + else if (s->cur) + buf = av_frame_clone(s->cur); + else + buf = av_frame_clone(s->prev); + if (!buf) + return AVERROR(ENOMEM); + + ret = filter_frame(ctx->inputs[0], buf); + if (ret < 0) + return ret; + repeat--; + } + ret = AVERROR_EOF; + } + + return ret; +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + FFTdnoizContext *s = ctx->priv; + int i; + + for (i = 0; i < 4; i++) { + PlaneContext *p = &s->planes[i]; + + av_freep(&p->hdata); + av_freep(&p->vdata); + av_freep(&p->buffer[PREV]); + av_freep(&p->buffer[CURRENT]); + av_freep(&p->buffer[NEXT]); + av_fft_end(p->fft); + av_fft_end(p->ifft); + } + + av_frame_free(&s->prev); + av_frame_free(&s->cur); + av_frame_free(&s->next); +} + +static const AVFilterPad fftdnoiz_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + .config_props = config_input, + }, + { NULL } +}; + +static const AVFilterPad fftdnoiz_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .request_frame = request_frame, + }, + { NULL } +}; + +AVFilter ff_vf_fftdnoiz = { + .name = "fftdnoiz", + .description = NULL_IF_CONFIG_SMALL("Denoise frames using 3D FFT."), + .priv_size = sizeof(FFTdnoizContext), + .init = init, + .uninit = uninit, + .query_formats = query_formats, + .inputs = fftdnoiz_inputs, + .outputs = fftdnoiz_outputs, + .priv_class = &fftdnoiz_class, + .flags = AVFILTER_FLAG_SUPPORT_TIMELINE_INTERNAL, +};