From patchwork Sun Nov 28 00:51:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 31785 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:d206:0:0:0:0:0 with SMTP id q6csp4270285iob; Sat, 27 Nov 2021 16:51:48 -0800 (PST) X-Google-Smtp-Source: ABdhPJyYyNP6Zg2qncKGLIKqAZ1kMWdYb9zhxBMrzeZBVzVPuOxbcouysLCKxxVRYlgf5SoBt0cG X-Received: by 2002:a17:906:1396:: with SMTP id f22mr49686503ejc.228.1638060707859; Sat, 27 Nov 2021 16:51:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1638060707; cv=none; d=google.com; s=arc-20160816; b=d62Xv+R1UDD6i8fO3BObhHkNh92uDLL2Lyso3G+vGBMp6WI7d9wBi9kbGhLBFE1CP6 MYC3F/DtOWEOcv4fqbhpOuPOmGD8SDIfBmEieDfyfOlImqZH5iqkLL1JqO15XVAikx4p wG9FSMFZY8Bg0Wc5rn/4Ld84DhPDj0JOJsGI/5pThjTO09EfwgNxA50oKU9zpn7eP7PV 98WyKG16OGZ6Qo3raO8Jq/vtBujwamNnh3X5VAcC2OfJMX9OOj7I6oul6Fez8En8y+Ld SRJ/SzFTiLzh4z1qsJxUgZozvcr47e/5GzgFqrgny4I60Wvbi63xLOaNgmZ1bp8WjIFZ FR0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=2WSHuhVC/AFTIqAfacZYmUBZBR8UYlRAXVrbgOkD4qU=; b=qvr5IPEw0pA9fh0QrcG6yoPNa8YaSru6PV8goBbrdB+Yj2r3HJ8cVsoTvOAjtUnLYS 53QUXtiU5rr8bzpnfwjelpaJayzHFkXzvA5Wutf+CsjiHZMmO6eABSiBd3yNRu17l3JY DhMeXuhBBhsE3n3pO7T5H4pKJi7yyPLP63z8w3Vw5CvAn/1Jk2zAZ43MLCl/849pp3b/ muImCJzLGdv4Z93iSoov9U54JVik3EuF9VftXWbbB32KlT7Vn4hTDdW/RAtO44aKka+q D2S51GZru+5AHsmke+6nRvg6e+opqUpAFrt2WaZXYRlk4Dbo0onmFiqpyPgj8QiRH5dM /oLw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=fprNKvK+; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s2si23195271ejn.51.2021.11.27.16.51.46; Sat, 27 Nov 2021 16:51:47 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=fprNKvK+; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D1005689831; Sun, 28 Nov 2021 02:51:42 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f51.google.com (mail-ed1-f51.google.com [209.85.208.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9645668834D for ; Sun, 28 Nov 2021 02:51:36 +0200 (EET) Received: by mail-ed1-f51.google.com with SMTP id g14so55380668edb.8 for ; Sat, 27 Nov 2021 16:51:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=yuZvXe6a+ydgdgT7Jh3zNK167XW7sDyAlrzh1yHTmpo=; b=fprNKvK+f0IBeSsyAdqWPDyw7qEQNp+AHEbjFSX0CtmbNBrwfa0GDa8Jlmdf85lQbF eE7IGV80GOkZ8Gf4y395U83KaPXYGSc8rsRONLJXOgHM6gaQBCBWNkQuDtGx1d8fxORQ J15yV1Bm71HXLXfXoQaPCx7wbd7dGz+tEnchpWfyN3HEarCWigRK9L0/Q4sDxOUYVTH+ F+lDRvEfH4hE/0mvMynHB+hWFuLruVH37ceOgC2pwv5H069rzPv/S5lWPtWE5jKcWno1 PEnhDdxJYbbxuMx4B5phUjCoWwO0LXoaddnpqxcUiODBgaH6SX3SjKsTLCpvPi7fPuQb XguQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=yuZvXe6a+ydgdgT7Jh3zNK167XW7sDyAlrzh1yHTmpo=; b=jHQLlG07YysyV3OvCMPV1G6dquFS15ffEpB09Z2nynpXEveqqO1/UIod3i3IbOMOM4 4FtyjS0K9lTU1NfFmY7P7EEKgSM/96V/vP3VXZzpMugx+Ccq7wQcF2bdqUHh8h6yfhjX 191ks+8MBRDFJSO9kmuo1hAPIxdJ9hbyccYK6Xbf62rwRgqDWhHCN8ziujvosCev/IgF OoWivEyfj5/RiSorPEoltKroQCS97ntxYkle5VjCT4QscmuPyDV1g+m8G0S+UuqC5pr1 YwpOFefRNYHGAWjeKpRo/QwZwocVgTaEVSJqvbv9pDxHVOzZDgoIBAiHBVSulSqVRMmS DHKw== X-Gm-Message-State: AOAM530sx7tLMwFb5P/G9wUGN20+soIK/VIo8Zn+lva2JmvP+9EPAlD0 HKSxv7r9s0xIqP2nvbkK/I0O2BpGDuw= X-Received: by 2002:a17:906:4fc4:: with SMTP id i4mr52237626ejw.81.1638060695849; Sat, 27 Nov 2021 16:51:35 -0800 (PST) Received: from localhost.localdomain ([95.168.120.85]) by smtp.gmail.com with ESMTPSA id ga37sm5164280ejc.65.2021.11.27.16.51.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 27 Nov 2021 16:51:35 -0800 (PST) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Sun, 28 Nov 2021 01:51:38 +0100 Message-Id: <20211128005138.660051-1-onemda@gmail.com> X-Mailer: git-send-email 2.33.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avfilter: add audio dynamic equalizer filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: mU8QP4vy81nX Signed-off-by: Paul B Mahol --- doc/filters.texi | 67 +++++++ libavfilter/Makefile | 1 + libavfilter/af_adynamicequalizer.c | 287 +++++++++++++++++++++++++++++ libavfilter/allfilters.c | 1 + 4 files changed, 356 insertions(+) create mode 100644 libavfilter/af_adynamicequalizer.c diff --git a/doc/filters.texi b/doc/filters.texi index 7852948d2f..60e72896ae 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -843,6 +843,73 @@ Compute derivative/integral of audio stream. Applying both filters one after another produces original audio. +@section adynamicequalizer + +Apply dynamic equalization to input audio stream. + +A description of the accepted options follows. + +@table @option +@item threshold +Set the detection threshold used to trigger equalization. +Threshold detection is using bandpass filter. +Default value is 0. Allowed range is from 0 to 50. + +@item dfrequency +Set the detection frequency in Hz used for bandpass filter used to trigger equalization. +Default value is 1000 Hz. Allowed range is between 2 and 1000000 Hz. + +@item dqfactor +Set the detection resonance factor for bandpass filter used to trigger equalization. +Default value is 1. Allowed range is from 0.001 to 1000. + +@item tfrequency +Set target frequency of equalization filter. +Default value is 1000 Hz. Allowed range is between 2 and 1000000 Hz. + +@item tqfactor +Set target resonance factor for target equalization filter. +Default value is 1. Allowed range is from 0.001 to 1000. + +@item attack +Amount of milliseconds the signal from detection has to rise above +the detection threshold before equalization starts. +Default is 10. Allowed range is between 1 and 2000. + +@item release +Amount of milliseconds the signal from detection has to fall below the +detection threshold before equalization ends. +Default is 80. Allowed range is between 1 and 2000. + +@item knee +Curve the sharp knee around the detection threshold to calculate +equalization gain more softly. +Default is 2. Allowed range is between 1 and 8. + +@item ratio +Set the ratio by which the equalization gain is raised. +Default is 1. Range is between 1 and 20. + +@item range +Set max allowed cut/boost amount. Default is 0.06125. +Allowed range is from 0.00000001 to 1. + +@item mode +Set mode of filter operation, can be one of the following: + +@table @samp +@item cut +Cut frequencies above detection threshold. +@item boost +Boost frequencies bellow detection threshold. +@end table +Default mode is @samp{cut}. +@end table + +@subsection Commands + +This filter supports the all above options as @ref{commands}. + @section adynamicsmooth Apply dynamic smoothing to input audio stream. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index c8082c4a2f..d40be4b252 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -44,6 +44,7 @@ OBJS-$(CONFIG_ADECORRELATE_FILTER) += af_adecorrelate.o OBJS-$(CONFIG_ADELAY_FILTER) += af_adelay.o OBJS-$(CONFIG_ADENORM_FILTER) += af_adenorm.o OBJS-$(CONFIG_ADERIVATIVE_FILTER) += af_aderivative.o +OBJS-$(CONFIG_ADYNAMICEQUALIZER_FILTER) += af_adynamicequalizer.o OBJS-$(CONFIG_ADYNAMICSMOOTH_FILTER) += af_adynamicsmooth.o OBJS-$(CONFIG_AECHO_FILTER) += af_aecho.o OBJS-$(CONFIG_AEMPHASIS_FILTER) += af_aemphasis.o diff --git a/libavfilter/af_adynamicequalizer.c b/libavfilter/af_adynamicequalizer.c new file mode 100644 index 0000000000..c56f610579 --- /dev/null +++ b/libavfilter/af_adynamicequalizer.c @@ -0,0 +1,287 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/ffmath.h" +#include "libavutil/opt.h" +#include "avfilter.h" +#include "audio.h" +#include "formats.h" +#include "hermite.h" + +typedef struct AudioDynamicEqualizerContext { + const AVClass *class; + + double threshold; + double lin_threshold; + double dfrequency; + double dqfactor; + double tfrequency; + double tqfactor; + double attack; + double release; + double knee; + double ratio; + double range; + double knee_sqrt; + double attack_coeff; + double release_coeff; + double lin_knee_start; + double lin_knee_stop; + double knee_start; + double knee_stop; + double compressed_knee_start; + int mode; + + AVFrame *state; +} AudioDynamicEqualizerContext; + +static int config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + AudioDynamicEqualizerContext *s = ctx->priv; + + s->state = ff_get_audio_buffer(inlink, 5); + if (!s->state) + return AVERROR(ENOMEM); + + return 0; +} + +static double get_gain(AVFilterContext *ctx, double in, + double sample_rate, int ch) +{ + AudioDynamicEqualizerContext *s = ctx->priv; + double *state = (double *)s->state->extended_data[ch]; + const double ratio = s->ratio; + const double knee = s->knee; + const double attack_coeff = s->attack_coeff; + const double release_coeff = s->release_coeff; + const double lin_knee_start = s->lin_knee_start; + const double threshold = s->threshold; + const double knee_start = s->knee_start; + const double knee_stop = s->knee_stop; + const double compressed_knee_start = s->compressed_knee_start; + const double abs_sample = fabs(in); + const int mode = s->mode; + double lin_slope = state[2]; + double tratio = ratio; + double range = s->range; + double delta = 0.; + double gain = 0.; + double slope; + int detected; + + lin_slope += (abs_sample - lin_slope) * (abs_sample > lin_slope ? attack_coeff : release_coeff); + + detected = lin_slope > lin_knee_start; + + state[2] = lin_slope; + if (lin_slope <= 0.0 || !detected) + return 1.; + + slope = log(lin_slope); + gain = (slope - threshold) * tratio + threshold; + delta = tratio; + + if (knee >= 1.0) + gain = hermite_interpolation(slope, knee_stop, knee_start, + knee_stop, compressed_knee_start, + 1.0, delta); + if (!mode) + gain = -gain; + + gain = exp(gain - slope); + gain = mode ? av_clipd(gain, 1., 1. / range) : av_clipd(gain, range, 1.); + + return gain; +} + +static double get_svf(double in, double *m, double *a, double *b) +{ + const double v0 = in; + const double v3 = v0 - b[1]; + const double v1 = a[0] * b[0] + a[1] * v3; + const double v2 = b[1] + a[1] * b[0] + a[2] * v3; + + b[0] = 2. * v1 - b[0]; + b[1] = 2. * v2 - b[1]; + + return m[0] * v0 + m[1] * v1 + m[2] * v2; +} + +typedef struct ThreadData { + AVFrame *in, *out; +} ThreadData; + +static int filter_channels(AVFilterContext *ctx, void *arg, int jobnr, int nb_jobs) +{ + AudioDynamicEqualizerContext *s = ctx->priv; + ThreadData *td = arg; + AVFrame *in = td->in; + AVFrame *out = td->out; + const double sample_rate = in->sample_rate; + const double dfrequency = fmin(s->dfrequency, sample_rate * 0.5); + const double tfrequency = fmin(s->tfrequency, sample_rate * 0.5); + const double dqfactor = s->dqfactor; + const double tqfactor = s->tqfactor; + const double fg = tan(M_PI * tfrequency / sample_rate); + const double dg = tan(M_PI * dfrequency / sample_rate); + const int start = (in->channels * jobnr) / nb_jobs; + const int end = (in->channels * (jobnr+1)) / nb_jobs; + double da[3], dm[3]; + + { + double k = 1. / dqfactor; + + da[0] = 1. / (1. + dg * (dg + k)); + da[1] = dg * da[0]; + da[2] = dg * da[1]; + + dm[0] = 0.; + dm[1] = 1.; + dm[2] = 0.; + } + + for (int ch = start; ch < end; ch++) { + const double *src = (const double *)in->extended_data[ch]; + double *dst = (double *)out->extended_data[ch]; + double *state = (double *)s->state->extended_data[ch]; + + for (int n = 0; n < out->nb_samples; n++) { + double detect, gain, v; + double fa[3], fm[3]; + + detect = get_svf(src[n], dm, da, state); + gain = get_gain(ctx, detect, sample_rate, ch); + + { + double k = 1. / (tqfactor * gain); + + fa[0] = 1. / (1. + fg * (fg + k)); + fa[1] = fg * fa[0]; + fa[2] = fg * fa[1]; + fm[0] = 1.; + fm[1] = k * (gain * gain - 1.); + fm[2] = 0.; + } + + v = get_svf(src[n], fm, fa, &state[3]); + dst[n] = ctx->is_disabled ? src[n] : v; + } + } + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx = inlink->dst; + AudioDynamicEqualizerContext *s = ctx->priv; + AVFilterLink *outlink = ctx->outputs[0]; + ThreadData td; + AVFrame *out; + + if (av_frame_is_writable(in)) { + out = in; + } else { + out = ff_get_audio_buffer(outlink, in->nb_samples); + if (!out) { + av_frame_free(&in); + return AVERROR(ENOMEM); + } + av_frame_copy_props(out, in); + } + + s->attack_coeff = FFMIN(1., 1. / (s->attack * in->sample_rate / 4000.)); + s->release_coeff = FFMIN(1., 1. / (s->release * in->sample_rate / 4000.)); + s->knee_sqrt = sqrt(s->knee); + s->lin_knee_stop = s->lin_threshold * s->knee_sqrt; + s->lin_knee_start = s->lin_threshold / s->knee_sqrt; + s->knee_start = log(s->lin_knee_start); + s->knee_stop = log(s->lin_knee_stop); + s->threshold = log(s->lin_threshold); + s->compressed_knee_start = (s->knee_start - s->threshold) / s->ratio + s->threshold; + + td.in = in; + td.out = out; + ff_filter_execute(ctx, filter_channels, &td, NULL, + FFMIN(outlink->channels, ff_filter_get_nb_threads(ctx))); + + if (out != in) + av_frame_free(&in); + return ff_filter_frame(outlink, out); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + AudioDynamicEqualizerContext *s = ctx->priv; + + av_frame_free(&s->state); +} + +#define OFFSET(x) offsetof(AudioDynamicEqualizerContext, x) +#define FLAGS AV_OPT_FLAG_AUDIO_PARAM|AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_RUNTIME_PARAM + +static const AVOption adynamicequalizer_options[] = { + { "threshold", "set detection threshold", OFFSET(lin_threshold), AV_OPT_TYPE_DOUBLE, {.dbl=0}, 0, 50, FLAGS }, + { "dfrequency", "set detection frequency", OFFSET(dfrequency), AV_OPT_TYPE_DOUBLE, {.dbl=1000}, 2, 1000000, FLAGS }, + { "dqfactor", "set detection Q factor", OFFSET(dqfactor), AV_OPT_TYPE_DOUBLE, {.dbl=1}, 0.001, 1000, FLAGS }, + { "tfrequency", "set target frequency", OFFSET(tfrequency), AV_OPT_TYPE_DOUBLE, {.dbl=1000}, 2, 1000000, FLAGS }, + { "tqfactor", "set target Q factor", OFFSET(tqfactor), AV_OPT_TYPE_DOUBLE, {.dbl=1}, 0.001, 1000, FLAGS }, + { "attack", "set attack", OFFSET(attack), AV_OPT_TYPE_DOUBLE, {.dbl=10}, 1, 2000, FLAGS }, + { "release", "set release", OFFSET(release), AV_OPT_TYPE_DOUBLE, {.dbl=80}, 1, 2000, FLAGS }, + { "knee", "set knee factor", OFFSET(knee), AV_OPT_TYPE_DOUBLE, {.dbl=2}, 1, 8, FLAGS }, + { "ratio", "set ratio factor", OFFSET(ratio), AV_OPT_TYPE_DOUBLE, {.dbl=1}, 1, 20, FLAGS }, + { "range", "set max gain", OFFSET(range), AV_OPT_TYPE_DOUBLE, {.dbl=0.06125},0.00000001, 1,FLAGS }, + { "mode", "set mode", OFFSET(mode), AV_OPT_TYPE_INT, {.i64=0}, 0, 1, FLAGS, "mode" }, + { "cut", 0, 0, AV_OPT_TYPE_CONST, {.i64=0}, 0, 0, FLAGS, "mode" }, + { "boost", 0, 0, AV_OPT_TYPE_CONST, {.i64=1}, 0, 0, FLAGS, "mode" }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(adynamicequalizer); + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + .filter_frame = filter_frame, + .config_props = config_input, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + }, +}; + +const AVFilter ff_af_adynamicequalizer = { + .name = "adynamicequalizer", + .description = NULL_IF_CONFIG_SMALL("Apply Dynamic Equalization of input audio."), + .priv_size = sizeof(AudioDynamicEqualizerContext), + .priv_class = &adynamicequalizer_class, + .uninit = uninit, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), + FILTER_SINGLE_SAMPLEFMT(AV_SAMPLE_FMT_DBLP), + .flags = AVFILTER_FLAG_SUPPORT_TIMELINE_INTERNAL | + AVFILTER_FLAG_SLICE_THREADS, + .process_command = ff_filter_process_command, +}; diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index c5c0e9b28b..7018337c85 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -37,6 +37,7 @@ extern const AVFilter ff_af_adecorrelate; extern const AVFilter ff_af_adelay; extern const AVFilter ff_af_adenorm; extern const AVFilter ff_af_aderivative; +extern const AVFilter ff_af_adynamicequalizer; extern const AVFilter ff_af_adynamicsmooth; extern const AVFilter ff_af_aecho; extern const AVFilter ff_af_aemphasis;