From patchwork Tue Jun 2 13:32:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 20069 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 7279B44A48F for ; Tue, 2 Jun 2020 16:33:12 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5684B68A9DB; Tue, 2 Jun 2020 16:33:12 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 851A06804E8 for ; Tue, 2 Jun 2020 16:33:05 +0300 (EEST) Received: by mail-ej1-f54.google.com with SMTP id f7so12763375ejq.6 for ; Tue, 02 Jun 2020 06:33:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=+N/YRIaYgUp2azSLv7SdBAKoXUOW4NLlF0FKYTd5S/U=; b=isXgWuWYR6lLPOiFCz5hX4Tj3FOXkJd2DEsbTdOVgV84hghrpXhKGAQHWa+FiTmtzh OBpj8KSgnfdwpaEBQeMnC6mgS3cGqEyU2a1VOZGQV3dWBR0XfltBZ0EDzRkkbBwQguky 962Ucsz/gQJhmAprXgg//4MOlzgJ+dAhJ9Jivv3Wr8dMgI/thIglvt5Ci7jS+ayotpWZ GH3CX9aBqjjdZcs3sNVL26Ha8opE1QfT4JdDz5YhzUVGyou1/ICvMz7knUofU8DsjXKe jAsX5hXyLGGPaLPeivT5U1J+cMbCEKg4Qo8Wlev3+vwB5XZKVvJI/HNp1G7cn1bRxbDD lTWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=+N/YRIaYgUp2azSLv7SdBAKoXUOW4NLlF0FKYTd5S/U=; b=VuBlKZ0SZvYILOq6NV8djKTESctv6X1vjQpDm2JItssBjBOtJEL5JsTSFKUDJO7vd+ zSEJjxnlscvXF3WHd8IouHSmV43fTLCCRQXFY0lFA3jJcNqMPOUosSC51aF0xdL4Wp0I wpbe22xomCuK8SNY7yH9pEPTP7w0kiMaaiXEIzUmboavF63qcvTpLrZcklfH4B5L1hZw Ma+mfVy3ppsYPwcXcDz7yd3TElccpCxX5fuZLleQ3anOH/as4Vocq2T+SZQNegX7oFxW q7+PREcogmNrirD3Qv43iqmd+BJLFLtwn96HjbJOKAZBfN1/oJ73U4ySOzK9ln6YGxk/ d3CQ== X-Gm-Message-State: AOAM5336IskMVoRJPZB6CnfmeARohIZQUkPbHLTWN3+zIViZwz4NKgHZ isp7HrjuFbQa2IAb/zW4d4zlDFSbgV8= X-Google-Smtp-Source: ABdhPJxXNY6crf/uVZS0F5WB1xQQWOGDt+eUw3iSYrEXgG4MTjgeuQEpblHjU5Tws/xQ5RD2Prl/Jw== X-Received: by 2002:a17:906:f891:: with SMTP id lg17mr23036665ejb.405.1591104783897; Tue, 02 Jun 2020 06:33:03 -0700 (PDT) Received: from localhost.localdomain ([37.244.239.188]) by smtp.gmail.com with ESMTPSA id n35sm85411edc.11.2020.06.02.06.33.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 06:33:03 -0700 (PDT) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Tue, 2 Jun 2020 15:32:54 +0200 Message-Id: <20200602133254.5732-1-onemda@gmail.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH] avfilter: add afwtdn filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Paul B Mahol --- doc/filters.texi | 41 +++ libavfilter/Makefile | 1 + libavfilter/af_afwtdn.c | 653 +++++++++++++++++++++++++++++++++++++++ libavfilter/allfilters.c | 1 + 4 files changed, 696 insertions(+) create mode 100644 libavfilter/af_afwtdn.c diff --git a/doc/filters.texi b/doc/filters.texi index a0b4ab2228..15a8636398 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -1314,6 +1314,47 @@ Force the output to either unsigned 8-bit or signed 16-bit stereo aformat=sample_fmts=u8|s16:channel_layouts=stereo @end example +@section afwtdn +Reduce broadband noise from input samples using Fast Wavelet Transform. + +A description of the accepted options follows. + +@table @option +@item sigma +Set the noise sigma, allowed range is from 0 to 100. +Default value is 0. +This option controls strength of denoising applied to input samples. + +@item levels +Set the number of wavelet levels of decomposition. +Allowed range is from 1 to 19. +Default value is 10. +Setting this too low make denoising performance very poor. + +@item wavet +Set wavelet type for decomposition of input frame. +Available wavelets are: + +@table @samp +@item deb4 +@item deb10 +@item coif5 +@item rbior68 +@end table + +@item percent +Set percent of full denoising. Allowed range is from 0 to 100 percent. +Default value is 85 percent or partial denoising. + +@item profile +If set enabled, first input frame will be used as noise profile. +If first frame samples contain non-noise performance will be very poor. + +@item frames +Set size of single frame in number of samples. Allowed range is from 1024 to +262144. Default frame size is 8192 samples. +@end table + @section agate A gate is mainly used to reduce lower parts of a signal. This kind of signal diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 5123540653..191826a622 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -50,6 +50,7 @@ OBJS-$(CONFIG_AFFTDN_FILTER) += af_afftdn.o OBJS-$(CONFIG_AFFTFILT_FILTER) += af_afftfilt.o OBJS-$(CONFIG_AFIR_FILTER) += af_afir.o OBJS-$(CONFIG_AFORMAT_FILTER) += af_aformat.o +OBJS-$(CONFIG_AFWTDN_FILTER) += af_afwtdn.o OBJS-$(CONFIG_AGATE_FILTER) += af_agate.o OBJS-$(CONFIG_AIIR_FILTER) += af_aiir.o OBJS-$(CONFIG_AINTEGRAL_FILTER) += af_aderivative.o diff --git a/libavfilter/af_afwtdn.c b/libavfilter/af_afwtdn.c new file mode 100644 index 0000000000..2d51c7bf2d --- /dev/null +++ b/libavfilter/af_afwtdn.c @@ -0,0 +1,653 @@ +/* + * Copyright (c) 2020 Paul B Mahol + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include + +#include "libavutil/avassert.h" +#include "libavutil/audio_fifo.h" +#include "libavutil/avstring.h" +#include "libavutil/opt.h" +#include "avfilter.h" +#include "audio.h" +#include "filters.h" +#include "formats.h" + +static const double rbior68_lp[18] = { + 0.0, 0.0, 0.0, 0.0, + 0.014426282505624435, 0.014467504896790148, + -0.07872200106262882, -0.04036797903033992, + 0.41784910915027457, 0.7589077294536541, + 0.41784910915027457, -0.04036797903033992, + -0.07872200106262882, 0.014467504896790148, + 0.014426282505624435, 0.0, 0.0, 0.0, +}; + +static const double rbior68_hp[18] = { + -0.0019088317364812906, -0.0019142861290887667, + 0.016990639867602342, 0.01193456527972926, + -0.04973290349094079, -0.07726317316720414, + 0.09405920349573646, 0.4207962846098268, + -0.8259229974584023, 0.4207962846098268, + 0.09405920349573646, -0.07726317316720414, + -0.04973290349094079, 0.01193456527972926, + 0.016990639867602342, -0.0019142861290887667, + -0.0019088317364812906, 0.0, +}; + +static const double rbior68_ilp[18] = { + 0.0019088317364812906, -0.0019142861290887667, + -0.016990639867602342, 0.01193456527972926, + 0.04973290349094079, -0.07726317316720414, + -0.09405920349573646, 0.4207962846098268, + 0.8259229974584023, 0.4207962846098268, + -0.09405920349573646, -0.07726317316720414, + 0.04973290349094079, 0.01193456527972926, + -0.016990639867602342, -0.0019142861290887667, + 0.0019088317364812906, 0.0, +}; + +static const double rbior68_ihp[18] = { + 0.0, 0.0, 0.0, 0.0, + 0.014426282505624435, -0.014467504896790148, + -0.07872200106262882, 0.04036797903033992, + 0.41784910915027457, -0.7589077294536541, + 0.41784910915027457, 0.04036797903033992, + -0.07872200106262882, -0.014467504896790148, + 0.014426282505624435, 0.0, 0.0, 0.0, +}; + +static const double coif5_lp[30] = { + -9.517657273819165e-08, -1.6744288576823017e-07, + 2.0637618513646814e-06, 3.7346551751414047e-06, + -2.1315026809955787e-05, -4.134043227251251e-05, + 0.00014054114970203437, 0.00030225958181306315, + -0.0006381313430451114, -0.0016628637020130838, + 0.0024333732126576722, 0.006764185448053083, + -0.009164231162481846, -0.01976177894257264, + 0.03268357426711183, 0.0412892087501817, + -0.10557420870333893, -0.06203596396290357, + 0.4379916261718371, 0.7742896036529562, + 0.4215662066908515, -0.05204316317624377, + -0.09192001055969624, 0.02816802897093635, + 0.023408156785839195, -0.010131117519849788, + -0.004159358781386048, 0.0021782363581090178, + 0.00035858968789573785, -0.00021208083980379827, +}; + +static const double coif5_hp[30] = { + 0.00021208083980379827, 0.00035858968789573785, + -0.0021782363581090178, -0.004159358781386048, + 0.010131117519849788, 0.023408156785839195, + -0.02816802897093635, -0.09192001055969624, + 0.05204316317624377, 0.4215662066908515, + -0.7742896036529562, 0.4379916261718371, + 0.06203596396290357, -0.10557420870333893, + -0.0412892087501817, 0.03268357426711183, + 0.01976177894257264, -0.009164231162481846, + -0.006764185448053083, 0.0024333732126576722, + 0.0016628637020130838, -0.0006381313430451114, + -0.00030225958181306315, 0.00014054114970203437, + 4.134043227251251e-05, -2.1315026809955787e-05, + -3.7346551751414047e-06, 2.0637618513646814e-06, + 1.6744288576823017e-07, -9.517657273819165e-08, +}; + +static const double coif5_ilp[30] = { + -0.00021208083980379827, 0.00035858968789573785, + 0.0021782363581090178, -0.004159358781386048, + -0.010131117519849788, 0.023408156785839195, + 0.02816802897093635, -0.09192001055969624, + -0.05204316317624377, 0.4215662066908515, + 0.7742896036529562, 0.4379916261718371, + -0.06203596396290357, -0.10557420870333893, + 0.0412892087501817, 0.03268357426711183, + -0.01976177894257264, -0.009164231162481846, + 0.006764185448053083, 0.0024333732126576722, + -0.0016628637020130838, -0.0006381313430451114, + 0.00030225958181306315, 0.00014054114970203437, + -4.134043227251251e-05, -2.1315026809955787e-05, + 3.7346551751414047e-06, 2.0637618513646814e-06, + -1.6744288576823017e-07, -9.517657273819165e-08, +}; + +static const double coif5_ihp[30] = { + -9.517657273819165e-08, 1.6744288576823017e-07, + 2.0637618513646814e-06, -3.7346551751414047e-06, + -2.1315026809955787e-05, 4.134043227251251e-05, + 0.00014054114970203437, -0.00030225958181306315, + -0.0006381313430451114, 0.0016628637020130838, + 0.0024333732126576722, -0.006764185448053083, + -0.009164231162481846, 0.01976177894257264, + 0.03268357426711183, -0.0412892087501817, + -0.10557420870333893, 0.06203596396290357, + 0.4379916261718371, -0.7742896036529562, + 0.4215662066908515, 0.05204316317624377, + -0.09192001055969624, -0.02816802897093635, + 0.023408156785839195, 0.010131117519849788, + -0.004159358781386048, -0.0021782363581090178, + 0.00035858968789573785, 0.00021208083980379827, +}; + +static const double deb10_lp[20] = { + -1.326420300235487e-05, 9.358867000108985e-05, + -0.0001164668549943862, -0.0006858566950046825, + 0.00199240529499085, 0.0013953517469940798, + -0.010733175482979604, 0.0036065535669883944, + 0.03321267405893324, -0.02945753682194567, + -0.07139414716586077, 0.09305736460380659, + 0.12736934033574265, -0.19594627437659665, + -0.24984642432648865, 0.2811723436604265, + 0.6884590394525921, 0.5272011889309198, + 0.18817680007762133, 0.026670057900950818, +}; + +static const double deb10_hp[20] = { + -0.026670057900950818, 0.18817680007762133, + -0.5272011889309198, 0.6884590394525921, + -0.2811723436604265, -0.24984642432648865, + 0.19594627437659665, 0.12736934033574265, + -0.09305736460380659, -0.07139414716586077, + 0.02945753682194567, 0.03321267405893324, + -0.0036065535669883944, -0.010733175482979604, + -0.0013953517469940798, 0.00199240529499085, + 0.0006858566950046825, -0.0001164668549943862, + -9.358867000108985e-05, -1.326420300235487e-05, +}; + +static const double deb10_ilp[20] = { + 0.026670057900950818, 0.18817680007762133, + 0.5272011889309198, 0.6884590394525921, + 0.2811723436604265, -0.24984642432648865, + -0.19594627437659665, 0.12736934033574265, + 0.09305736460380659, -0.07139414716586077, + -0.02945753682194567, 0.03321267405893324, + 0.0036065535669883944, -0.010733175482979604, + 0.0013953517469940798, 0.00199240529499085, + -0.0006858566950046825, -0.0001164668549943862, + 9.358867000108985e-05, -1.326420300235487e-05, +}; + +static const double deb10_ihp[20] = { + -1.326420300235487e-05, -9.358867000108985e-05, + -0.0001164668549943862, 0.0006858566950046825, + 0.00199240529499085, -0.0013953517469940798, + -0.010733175482979604, -0.0036065535669883944, + 0.03321267405893324, 0.02945753682194567, + -0.07139414716586077, -0.09305736460380659, + 0.12736934033574265, 0.19594627437659665, + -0.24984642432648865, -0.2811723436604265, + 0.6884590394525921, -0.5272011889309198, + 0.18817680007762133, -0.026670057900950818, +}; + +static const double deb4_lp[8] = { + -0.0105974018, 0.0328830117, + 0.0308413818, -0.1870348117, + -0.0279837694, 0.6308807679, + 0.7148465706, 0.2303778133, +}; + +static const double deb4_hp[8] = { + -0.2303778133, 0.7148465706, + -0.6308807679, -0.0279837694, + 0.1870348117, 0.0308413818, + -0.0328830117, -0.0105974018, +}; + +static const double deb4_ilp[8] = { + 0.23037781330885523, 0.7148465705525415, + 0.6308807679295904, -0.02798376941698385, + -0.18703481171888114, 0.030841381835986965, + 0.032883011666982945, -0.010597401784997278, +}; + +static const double deb4_ihp[8] = { + -0.010597401784997278, -0.032883011666982945, + 0.030841381835986965, 0.18703481171888114, + -0.02798376941698385, -0.6308807679295904, + 0.7148465705525415, -0.23037781330885523, +}; + +#define MAX_LEVELS 20 + +typedef struct AudioFWTDNContext { + const AVClass *class; + + double sigma; + double percent; + + int wavelet_type; + int thresholding_function; + int nb_samples; + int levels; + int wavelet_length; + int length[MAX_LEVELS]; + int out_length; + int need_profile; + int got_profile; + + const double *lp, *hp; + const double *ilp, *ihp; + + AVFrame *temp0, *temp1; + AVFrame *wtc, *power, *profile, *filter, *signal; +} AudioFWTDNContext; + +#define OFFSET(x) offsetof(AudioFWTDNContext, x) +#define AF AV_OPT_FLAG_AUDIO_PARAM|AV_OPT_FLAG_FILTERING_PARAM + +static const AVOption afwtdn_options[] = { + { "sigma", "set noise sigma", OFFSET(sigma), AV_OPT_TYPE_DOUBLE, {.dbl=0}, 0, 100, AF }, + { "levels", "set number of wavelet levels", OFFSET(levels), AV_OPT_TYPE_INT, {.i64=10}, 1, MAX_LEVELS-1, AF }, + { "wavet", "set wavelet type", OFFSET(wavelet_type), AV_OPT_TYPE_INT, {.i64=0}, 0, 3, AF, "wavet" }, + { "deb4", "deb4", 0, AV_OPT_TYPE_CONST, {.i64=0}, 0, 0, AF, "wavet" }, + { "deb10", "deb10", 0, AV_OPT_TYPE_CONST, {.i64=1}, 0, 0, AF, "wavet" }, + { "coif5", "coif5", 0, AV_OPT_TYPE_CONST, {.i64=2}, 0, 0, AF, "wavet" }, + { "rbior68", "rbior68", 0, AV_OPT_TYPE_CONST, {.i64=3}, 0, 0, AF, "wavet" }, + { "percent", "set percent of full denoising", OFFSET(percent),AV_OPT_TYPE_DOUBLE, {.dbl=85}, 0, 100, AF }, + { "profile", "profile noise", OFFSET(need_profile), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, + { "frames", "set frame size", OFFSET(nb_samples), AV_OPT_TYPE_INT, {.i64=8192}, 1024, 262144, AF }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(afwtdn); + +static int query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats = NULL; + AVFilterChannelLayouts *layouts = NULL; + static const enum AVSampleFormat sample_fmts[] = { + AV_SAMPLE_FMT_DBLP, + AV_SAMPLE_FMT_NONE + }; + int ret; + + formats = ff_make_format_list(sample_fmts); + if (!formats) + return AVERROR(ENOMEM); + ret = ff_set_common_formats(ctx, formats); + if (ret < 0) + return ret; + + layouts = ff_all_channel_counts(); + if (!layouts) + return AVERROR(ENOMEM); + + ret = ff_set_common_channel_layouts(ctx, layouts); + if (ret < 0) + return ret; + + formats = ff_all_samplerates(); + return ff_set_common_samplerates(ctx, formats); +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + AudioFWTDNContext *s = ctx->priv; + int N, i; + + switch (s->wavelet_type) { + case 0: + s->wavelet_length = 8; + s->lp = deb4_lp; + s->hp = deb4_hp; + s->ilp = deb4_ilp; + s->ihp = deb4_ihp; + break; + case 1: + s->wavelet_length = 20; + s->lp = deb10_lp; + s->hp = deb10_hp; + s->ilp = deb10_ilp; + s->ihp = deb10_ihp; + break; + case 2: + s->wavelet_length = 30; + s->lp = coif5_lp; + s->hp = coif5_hp; + s->ilp = coif5_ilp; + s->ihp = coif5_ihp; + break; + case 3: + s->wavelet_length = 18; + s->lp = rbior68_lp; + s->hp = rbior68_hp; + s->ilp = rbior68_ilp; + s->ihp = rbior68_ihp; + break; + default: + av_assert0(0); + } + + s->levels = FFMIN(s->levels, lrint(log(s->nb_samples / (s->wavelet_length - 1.0)) / M_LN2)); + av_log(ctx, AV_LOG_VERBOSE, "levels:%d\n", s->levels); + N = s->nb_samples; + i = s->levels; + + while (i > 0) { + N = N + s->wavelet_length - 2; + N = ceil(N / 2.0); + s->length[i] = N; + s->out_length += N; + i--; + } + + s->length[0] = s->length[1]; + s->out_length += s->length[0]; + + s->temp0 = ff_get_audio_buffer(outlink, s->out_length); + s->temp1 = ff_get_audio_buffer(outlink, s->out_length); + s->wtc = ff_get_audio_buffer(outlink, s->out_length); + s->power = ff_get_audio_buffer(outlink, s->out_length); + s->filter = ff_get_audio_buffer(outlink, s->out_length); + s->signal = ff_get_audio_buffer(outlink, s->out_length); + s->profile = ff_get_audio_buffer(outlink, MAX_LEVELS); + if (!s->temp0 || !s->temp1 || !s->wtc || + !s->power || !s->profile || !s->filter || + !s->signal) + return AVERROR(ENOMEM); + + return 0; +} + +static void dwt(double *src, int N, + const double *lf, const double *hf, int flen, + double *low, double *high, int lhlen) +{ + for (int i = 0; i < lhlen; i++) { + const int t = 2 * i + 1; + + low[i] = 0.0; + high[i] = 0.0; + + for (int l = 0; l < flen; l++) { + if (t - l >= 0 && t - l < N) { + const int is = t - l; + low[i] += lf[l] * src[is]; + high[i] += hf[l] * src[is]; + } else if (t - l < 0) { + const int is = -t + l - 1; + low[i] += lf[l] * src[is]; + high[i] += hf[l] * src[is]; + } else if (t - l >= N) { + const int is = 2 * N - t + l - 1; + low[i] += lf[l] * src[is]; + high[i] += hf[l] * src[is]; + } + } + } +} + +static void idwt(double *low, int lhlen, double *high, + const double *lf, const double *hf, + int flen, double *dst) +{ + int m = -2, n = -1; + + for (int v = 0; v < lhlen; v++) { + m += 2; + n += 2; + dst[m] = 0.0; + dst[n] = 0.0; + for (int l = 0; l < flen / 2; l++) { + const int t = 2 * l; + + if (v - l >= 0 && v - l < lhlen) { + const int is = v - l; + + dst[m] += lf[t] * low[is] + hf[t] * high[is]; + dst[n] += lf[t + 1] * low[is] + hf[t + 1] * high[is]; + } + } + } +} + +static void dwt_levels(AudioFWTDNContext *s, int levels, int inlength, int out_length, + const double *in, double *temp0, double *temp1, double *wtc) +{ + int N = out_length; + int temp_len = s->nb_samples; + + for (int i = 0; i < inlength; i++) + temp0[i] = in[i]; + for (int i = inlength; i < s->nb_samples; i++) + temp0[i] = 0.; + + for (int level = 0; level < levels; level++) { + const int level_length = s->length[levels - level]; + + N -= level_length; + dwt(temp0, temp_len, s->lp, s->hp, s->wavelet_length, temp1, wtc + N, level_length); + temp_len = s->length[levels - level]; + + if (level == levels - 1) { + for (int i = 0; i < level_length; i++) + wtc[i] = temp1[i]; + } else { + for (int i = 0; i < level_length; i++) + temp0[i] = temp1[i]; + } + } +} + +static void idwt_levels(AudioFWTDNContext *s, int levels, double *out, + double *temp, double *wtc) +{ + const int app_len = s->length[0]; + const int lf = s->wavelet_length; + int iter = app_len; + + for (int i = 0; i < app_len; i++) + out[i] = wtc[i]; + + for (int i = 0; i < levels; i++) { + const int det_len = s->length[i + 1]; + + idwt(out, det_len, wtc + iter, s->ilp, s->ihp, s->wavelet_length, temp); + for (int k = lf - 2; k < 2 * det_len; k++) + out[k - lf + 2] = temp[k]; + + iter += det_len; + } +} + +static void denoise_level(double *out, const double *in, double *filter, double *signal, double percent, int length) +{ + const double x = percent * 0.01; + const double y = 1.0 - x; + + for (int i = 0; i < length; i++) + out[i] = in[i] * (x * filter[i] * signal[i] + y); +} + +static double sqr(double in) +{ + return in * in; +} + +static void power_average(const double *in, double *out, int length) +{ + out[0] = sqr(in[0]); + for (int i = 1; i < length; i++) + out[i] = 0.1 * sqr(in[i]) + 0.9 * out[i - 1]; +} + +static void noise_filter(const double *in, double *out, double profile, double ak, int length) +{ + for (int i = 0; i < length; i++) + out[i] = 1.0 - profile * ak / in[i]; +} + +static void signal_filter(const double *in, double *out, double profile, double ak, int length) +{ + for (int i = 0; i < length; i++) + out[i] = in[i] >= profile * ak; +} + +static void measure_level_profile(AVFilterContext *ctx, const double *wtc, double *profile, int length) +{ + double sum = 0.; + + for (int i = 0; i < length; i++) + sum += sqr(wtc[i]); + + *profile = sum / length; +} + +typedef struct ThreadData { + AVFrame *in, *out; +} ThreadData; + +static int filter_channel(AVFilterContext *ctx, void *arg, int ch, int nb_jobs) +{ + AudioFWTDNContext *s = ctx->priv; + ThreadData *td = arg; + AVFrame *in = td->in; + AVFrame *out = td->out; + const double *src = (const double *)(in->extended_data[ch]); + double *temp0 = (double *)s->temp0->extended_data[ch]; + double *temp1 = (double *)s->temp1->extended_data[ch]; + double *wtc = (double *)s->wtc->extended_data[ch]; + double *dst = (double *)out->extended_data[ch]; + double *profile = (double *)s->profile->extended_data[ch]; + double *power = (double *)s->power->extended_data[ch]; + double *filter = (double *)s->filter->extended_data[ch]; + double *signal = (double *)s->signal->extended_data[ch]; + int offset = 0; + + dwt_levels(s, s->levels, in->nb_samples, s->out_length, src, temp0, temp1, wtc); + + if (!s->got_profile && s->need_profile) { + for (int level = 0; level <= s->levels; level++) { + measure_level_profile(ctx, wtc + offset, &profile[level], s->length[level]); + offset += s->length[level]; + } + offset = 0; + } else if (!s->got_profile && !s->need_profile) { + for (int level = 0; level <= s->levels; level++) + profile[level] = s->sigma / 1000000.; + } + + for (int level = 0; level <= s->levels; level++) { + const double ak = 2.0 + ((4.0 * level) / s->levels); + const int length = s->length[level]; + + power_average(wtc + offset, power + offset, length); + noise_filter(power + offset, filter + offset, profile[level], ak, length); + signal_filter(power + offset, signal + offset, profile[level], ak, length); + denoise_level(wtc + offset, wtc + offset, filter + offset, signal + offset, s->percent, length); + offset += length; + } + + idwt_levels(s, s->levels, dst, temp0, wtc); + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx = inlink->dst; + AudioFWTDNContext *s = ctx->priv; + AVFilterLink *outlink = ctx->outputs[0]; + ThreadData td; + AVFrame *out = NULL; + + out = ff_get_audio_buffer(outlink, in->nb_samples); + if (!out) { + av_frame_free(&in); + return AVERROR(ENOMEM); + } + out->pts = in->pts; + + td.in = in; + td.out = out; + ctx->internal->execute(ctx, filter_channel, &td, NULL, inlink->channels); + if (s->need_profile) + s->got_profile = 1; + + av_frame_free(&in); + return ff_filter_frame(outlink, out); +} + +static int activate(AVFilterContext *ctx) +{ + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + AudioFWTDNContext *s = ctx->priv; + AVFrame *in = NULL; + int ret; + + FF_FILTER_FORWARD_STATUS_BACK(outlink, inlink); + + ret = ff_inlink_consume_samples(inlink, s->nb_samples, s->nb_samples, &in); + if (ret < 0) + return ret; + if (ret > 0) + return filter_frame(inlink, in); + + FF_FILTER_FORWARD_STATUS(inlink, outlink); + FF_FILTER_FORWARD_WANTED(outlink, inlink); + + return FFERROR_NOT_READY; +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + AudioFWTDNContext *s = ctx->priv; + + av_frame_free(&s->filter); + av_frame_free(&s->power); + av_frame_free(&s->profile); + av_frame_free(&s->signal); + av_frame_free(&s->temp0); + av_frame_free(&s->temp1); + av_frame_free(&s->wtc); +} + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + }, + { NULL } +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + .config_props = config_output, + }, + { NULL } +}; + +AVFilter ff_af_afwtdn = { + .name = "afwtdn", + .description = NULL_IF_CONFIG_SMALL("Reduce broadband noise from stream using Fast Wavelet Transform."), + .query_formats = query_formats, + .priv_size = sizeof(AudioFWTDNContext), + .priv_class = &afwtdn_class, + .activate = activate, + .uninit = uninit, + .inputs = inputs, + .outputs = outputs, + .flags = AVFILTER_FLAG_SLICE_THREADS, +}; diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 1183e40267..de5884529c 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -43,6 +43,7 @@ extern AVFilter ff_af_afftdn; extern AVFilter ff_af_afftfilt; extern AVFilter ff_af_afir; extern AVFilter ff_af_aformat; +extern AVFilter ff_af_afwtdn; extern AVFilter ff_af_agate; extern AVFilter ff_af_aiir; extern AVFilter ff_af_aintegral;