From patchwork Sat May 2 10:51:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 19431 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 04CDF44A99F for ; Sat, 2 May 2020 13:51:31 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D212A68C3E3; Sat, 2 May 2020 13:51:30 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B219168C197 for ; Sat, 2 May 2020 13:51:24 +0300 (EEST) Received: by mail-wm1-f44.google.com with SMTP id g12so2951287wmh.3 for ; Sat, 02 May 2020 03:51:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=2lBxfNzNF4gITsVPp4yxOhu0Pa4rXRe5a7jH09wuUv4=; b=dLhqsTUHfV7J7ml+zKtccvjA8BVV/CyiZza+vzC9J4TnlpGRHR0gELP0vmzSbXcR4w XhGycUlO88ff4rFAgSnTOuoDcfxq/eL700BHbrsU6hu0lpm0rspW/HJakHLH3z5mJ0zR jmvVT2rbo/bUDPxu31OmIRnX/imS+yXQDJd/W33A0lsIrCUD7+OAwnF4hzkZJV9ystCY 68K6FcySSl4Sh7BXF8qw6ACKkobRB+ZTWZXV6bebHpunfD08KQMlGTw9fRcNBV8vzYZ8 mBZRAomoTvLq6O8Di0tp9Fkn4RAzx8ELrNvTmSWWoqdBFgdRBBFMg5ZeNv9huXHeWnoj zXRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=2lBxfNzNF4gITsVPp4yxOhu0Pa4rXRe5a7jH09wuUv4=; b=IrPwZAOxbYUkneAu5aAdW6+I2WEyc/y3HGOe3FfU8qGht051SbjD0NP96Hbi0vxKjg NeiU0I2ueRwPbjFccAEDQTBfBsE2GKHZAAhT13ArtZsXAqirwypFYVFh0EUZuZJoccKC 8Q6yMVLbnJ9qZiovWqitktQsyygfBU9ZVUWxYjKqOTs9cDKgbkXMquImocjzcuNo5pGi rMRvYZqOQT01e5hdoReUzjgiQA+36YW9EcJuj2FRnOo2HRTDnE/vsZIP5PaYumLhFzIg 9TiL4QrivDoYA3zIXsWXpTkX/HpK56/jaZ6zkhLv+lEpQFsv3NIJCKw0jOkSbT4KlVZR cgpg== X-Gm-Message-State: AGi0PubxmgTTy2rVuPZGO166lJDkLBYl00YHogFc8W01ob9Vy1eV43t5 SZgWEZZObgI0jS5qJXxCkv7KrOCQ X-Google-Smtp-Source: APiQypJxL3DbWD7TrNC/ObRIyJ1IsNnIzmfzRyCnJKiJZuFXc7HzIGLEmNnOVsk8lDmBIo2wAVp3+g== X-Received: by 2002:a1c:7ed7:: with SMTP id z206mr3836879wmc.64.1588416683511; Sat, 02 May 2020 03:51:23 -0700 (PDT) Received: from localhost.localdomain ([94.250.167.28]) by smtp.gmail.com with ESMTPSA id w6sm1973402wrt.39.2020.05.02.03.51.22 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 02 May 2020 03:51:23 -0700 (PDT) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Sat, 2 May 2020 12:51:15 +0200 Message-Id: <20200502105115.20564-1-onemda@gmail.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH] avfilter: add aringmodulator filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Paul B Mahol --- doc/filters.texi | 78 +++++++++++++ libavfilter/Makefile | 1 + libavfilter/af_apulsator.c | 224 +++++++++++++++++++++++++++++++++++++ libavfilter/allfilters.c | 1 + 4 files changed, 304 insertions(+) diff --git a/doc/filters.texi b/doc/filters.texi index d19fd346ae..a8987a56c7 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -2141,6 +2141,84 @@ atrim=end=5,areverse @end example @end itemize +@section aringmodulator +Apply ring modulator to input audio stream. + +Ring Modulator distorts the original signal with a LFO-generated signal - +the so called "modulator". The modulator can be altered in frequency even +by another LFO which adds some kind of modulating distortion to the signal. +A third LFO is used to modulate different aspects of the ring modulator to +create even more strange effects. + +This filter accepts the following options: +@table @option +@item level_in +Set the input gain before signal is processed. + +@item level_out +Set the output gain after signal is processed. + +@item mod_mode +Set waveform of the modulator. + +@item mod_freq +Set frequency of the modulator. + +@item mod_amount +Set amount by how much original signal is modulated. + +@item mod_phase +Shift the phase of the right channels modulator against the left channel. + +@item mod_detune +Detune the modulators frequency between left and right channel. + +@item mod_listen +Only output modulator signal. By default is disabled. + +@item lfo1_mode +Set waveform of the LFO1 modulator. + +@item lfo1_freq +Set frequency of the LFO1 modulator. + +@item lfo1_mod_freq_lo +Set the minimal modulators frequency to reach at lowest point. + +@item lfo1_mod_freq_hi +Set the maximal modulators frequency to reach at highest point. + +@item lfo1_mod_freq +Enable modulating modulators frequency by LFO1. By defauls is disabled. + +@item lfo1_mod_detune_lo +Set the minimal modulators frequency to reach at lowest point. + +@item lfo1_mod_detune_hi +Set the maximal modulators frequency to reach at highest point. + +@item lfo1_mod_detune +Enable modulating modulators frequency by LFO1. By default is disabled. + +@item lfo2_mode +Set waveform of the LFO2 modulator. + +@item lfo2_freq +Set frequency of the LFO2 modulator. + +@item lfo2_lfo1_freq +Enable modulating LFO1's frequency by LFO2. By defauls is disabled. + +@item lfo2_mod_amount +Enable modulating modulators amount by LFO2. By defauls is disabled. + +@item lfo2_mod_amount_lo +Set the minimal modulators amount to reach at LFO2 lowest point. + +@item lfo2_mod_amount_hi +Set the maximal modulators amount to reach at LFO2 highest point. +@end table + @section arnndn Reduce noise from speech using Recurrent Neural Networks. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index f982afe15f..dc512fe4b2 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -72,6 +72,7 @@ OBJS-$(CONFIG_APULSATOR_FILTER) += af_apulsator.o OBJS-$(CONFIG_AREALTIME_FILTER) += f_realtime.o OBJS-$(CONFIG_ARESAMPLE_FILTER) += af_aresample.o OBJS-$(CONFIG_AREVERSE_FILTER) += f_reverse.o +OBJS-$(CONFIG_ARINGMODULATOR_FILTER) += af_apulsator.o OBJS-$(CONFIG_ARNNDN_FILTER) += af_arnndn.o OBJS-$(CONFIG_ASELECT_FILTER) += f_select.o OBJS-$(CONFIG_ASENDCMD_FILTER) += f_sendcmd.o diff --git a/libavfilter/af_apulsator.c b/libavfilter/af_apulsator.c index 67711a28ce..9ba1d48341 100644 --- a/libavfilter/af_apulsator.c +++ b/libavfilter/af_apulsator.c @@ -255,3 +255,227 @@ AVFilter ff_af_apulsator = { .inputs = inputs, .outputs = outputs, }; + +#if CONFIG_ARINGMODULATOR_FILTER +typedef struct AudioRingModulatorContext { + const AVClass *class; + double level_in; + double level_out; + int mod_mode; + double mod_freq; + double mod_amount; + double mod_phase; + double mod_detune; + int mod_listen; + int lfo1_mode; + double lfo1_freq; + double lfo1_mod_freq_lo; + double lfo1_mod_freq_hi; + int lfo1_mod_freq_active; + int lfo2_lfo1_freq_active; + int lfo1_mod_detune_active; + double lfo1_mod_detune_lo; + double lfo1_mod_detune_hi; + int lfo2_mode; + double lfo2_freq; + double lfo2_lfo1_freq_lo; + double lfo2_lfo1_freq_hi; + int lfo2_mod_amount_active; + double lfo2_mod_amount_lo; + double lfo2_mod_amount_hi; + + SimpleLFO lfo1, lfo2, modL, modR; +} AudioRingModulatorContext; + +static int ringmodulator_config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + AudioRingModulatorContext *s = ctx->priv; + + s->modL.freq = s->mod_freq * pow(pow(2, 1.0 / 1200.0), s->mod_detune / 2); + s->modR.freq = s->mod_freq * pow(pow(2, 1.0 / 1200.0), s->mod_detune / 2); + s->modL.mode = s->mod_mode; + s->modR.mode = s->mod_mode; + s->modL.offset = 0; + s->modR.offset = s->mod_phase; + s->modL.srate = inlink->sample_rate; + s->modR.srate = inlink->sample_rate; + s->modL.amount = s->mod_amount; + s->modR.amount = s->mod_amount; + s->modL.pwidth = 1; + s->modR.pwidth = 1; + s->lfo1.freq = s->lfo1_freq; + s->lfo2.freq = s->lfo2_freq; + s->lfo1.mode = s->lfo1_mode; + s->lfo2.mode = s->lfo2_mode; + s->lfo1.srate = inlink->sample_rate; + s->lfo2.srate = inlink->sample_rate; + s->lfo1.amount = 1; + s->lfo2.amount = 1; + s->lfo1.pwidth = 1; + s->lfo2.pwidth = 1; + + return 0; +} + +static int ringmodulator_filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx = inlink->dst; + AVFilterLink *outlink = ctx->outputs[0]; + AudioRingModulatorContext *s = ctx->priv; + const double *src = (const double *)in->data[0]; + const int nb_samples = in->nb_samples; + const double level_out = s->level_out; + const double level_in = s->level_in; + AVFrame *out; + double *dst; + int n; + + if (av_frame_is_writable(in)) { + out = in; + } else { + out = ff_get_audio_buffer(inlink, in->nb_samples); + if (!out) { + av_frame_free(&in); + return AVERROR(ENOMEM); + } + av_frame_copy_props(out, in); + } + dst = (double *)out->data[0]; + + for (n = 0; n < nb_samples; n++) { + double mod_amount = s->mod_amount; + double freq = 0.; + double outL = 0.; + double outR = 0.; + double inL = src[0] * level_in; + double inR = src[1] * level_in; + double procL; + double procR; + double modulL; + double modulR; + + if (s->lfo1_mod_freq_active) { + freq = (s->lfo1_mod_freq_hi - s->lfo1_mod_freq_lo) * + ((lfo_get_value(&s->lfo1) + 1) / 2.) + + s->lfo1_mod_freq_lo; + s->modL.freq = freq; + s->modR.freq = freq; + } + + if (s->lfo1_mod_detune_active) { + double detune = (s->lfo1_mod_detune_hi - s->lfo1_mod_detune_lo) * + ((lfo_get_value(&s->lfo1) + 1) / 2.) + s->lfo1_mod_detune_lo; + + s->modL.freq = freq ? freq : s->mod_freq * pow(pow(2, 1.0 / 1200.0), detune / 2); + s->modR.freq = freq ? freq : s->mod_freq * pow(pow(2, 1.0 / 1200.0), detune / -2); + } + + if (s->lfo2_lfo1_freq_active) { + s->lfo1.freq = (s->lfo2_lfo1_freq_hi - s->lfo2_lfo1_freq_lo) * + ((lfo_get_value(&s->lfo2) + 1) / 2.) + + s->lfo2_lfo1_freq_lo; + } + + if (s->lfo2_mod_amount_active) { + mod_amount = (s->lfo2_mod_amount_hi - s->lfo2_mod_amount_lo) * + ((lfo_get_value(&s->lfo2) + 1) / 2.) + + s->lfo2_mod_amount_lo; + } + + modulL = lfo_get_value(&s->modL) * mod_amount; + modulR = lfo_get_value(&s->modR) * mod_amount; + + procL = inL * modulL; + procR = inR * modulR; + + outL = s->mod_listen ? modulL : procL + inL * (1 - mod_amount); + outR = s->mod_listen ? modulR : procR + inR * (1 - mod_amount); + + outL *= level_out; + outR *= level_out; + + dst[0] = outL; + dst[1] = outR; + + lfo_advance(&s->lfo1, 1); + lfo_advance(&s->lfo2, 1); + lfo_advance(&s->modL, 1); + lfo_advance(&s->modR, 1); + + dst += 2; + src += 2; + } + + if (in != out) + av_frame_free(&in); + + return ff_filter_frame(outlink, out); +} + +#undef OFFSET +#undef FLAGS +#define OFFSET(x) offsetof(AudioRingModulatorContext, x) +#define FLAGS AV_OPT_FLAG_AUDIO_PARAM|AV_OPT_FLAG_FILTERING_PARAM + +static const AVOption aringmodulator_options[] = { + { "level_in", "set input gain", OFFSET(level_in), AV_OPT_TYPE_DOUBLE, {.dbl=1}, 0.015625, 64, FLAGS, }, + { "level_out", "set output gain", OFFSET(level_out), AV_OPT_TYPE_DOUBLE, {.dbl=1}, 0.015625, 64, FLAGS, }, + { "mod_mode", "set modulation mode", OFFSET(mod_mode), AV_OPT_TYPE_INT, {.i64=SINE}, SINE, NB_MODES-1, FLAGS, "mode" }, + { "sine", NULL, 0, AV_OPT_TYPE_CONST, {.i64=SINE}, 0, 0, FLAGS, "mode" }, + { "triangle", NULL, 0, AV_OPT_TYPE_CONST, {.i64=TRIANGLE}, 0, 0, FLAGS, "mode" }, + { "square", NULL, 0, AV_OPT_TYPE_CONST, {.i64=SQUARE}, 0, 0, FLAGS, "mode" }, + { "sawup", NULL, 0, AV_OPT_TYPE_CONST, {.i64=SAWUP}, 0, 0, FLAGS, "mode" }, + { "sawdown", NULL, 0, AV_OPT_TYPE_CONST, {.i64=SAWDOWN}, 0, 0, FLAGS, "mode" }, + { "mod_freq", NULL, OFFSET(mod_freq), AV_OPT_TYPE_DOUBLE, {.dbl=1000}, 1, 20000, FLAGS }, + { "mod_amount", NULL, OFFSET(mod_amount), AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 1, FLAGS }, + { "mod_phase", NULL, OFFSET(mod_phase), AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 1, FLAGS }, + { "mod_detune", NULL, OFFSET(mod_detune), AV_OPT_TYPE_DOUBLE, {.dbl=0}, -200, 200, FLAGS }, + { "mod_listen", NULL, OFFSET(mod_listen), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS }, + { "lfo1_mode", NULL, OFFSET(lfo1_mode), AV_OPT_TYPE_INT, {.i64=SINE}, SINE, NB_MODES-1, FLAGS, "mode" }, + { "lfo1_freq", NULL, OFFSET(lfo1_freq), AV_OPT_TYPE_DOUBLE, {.dbl=0.1}, 0.01, 10, FLAGS }, + { "lfo1_mod_freq_lo", NULL, OFFSET(lfo1_mod_freq_lo), AV_OPT_TYPE_DOUBLE, {.dbl=100}, 1,20000, FLAGS }, + { "lfo1_mod_freq_hi", NULL, OFFSET(lfo1_mod_freq_hi), AV_OPT_TYPE_DOUBLE, {.dbl=1000}, 1,20000, FLAGS }, + { "lfo1_mod_freq", NULL, OFFSET(lfo1_mod_freq_active), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS }, + { "lfo1_mod_detune_lo", NULL, OFFSET(lfo1_mod_detune_lo), AV_OPT_TYPE_DOUBLE, {.dbl=-100}, -200, 200, FLAGS }, + { "lfo1_mod_detune_hi", NULL, OFFSET(lfo1_mod_detune_hi), AV_OPT_TYPE_DOUBLE, {.dbl=100}, -200, 200, FLAGS }, + { "lfo1_mod_detune", NULL, OFFSET(lfo1_mod_detune_active), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS }, + { "lfo2_mode", NULL, OFFSET(lfo2_mode), AV_OPT_TYPE_INT, {.i64=SINE}, SINE, NB_MODES-1, FLAGS, "mode" }, + { "lfo2_freq", NULL, OFFSET(lfo2_freq), AV_OPT_TYPE_DOUBLE, {.dbl=0.2}, 0.01, 10, FLAGS }, + { "lfo2_lfo1_freq", NULL, OFFSET(lfo2_lfo1_freq_active), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS }, + { "lfo2_mod_amount", NULL, OFFSET(lfo2_mod_amount_active), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS }, + { "lfo2_mod_amount_lo", NULL, OFFSET(lfo2_mod_amount_lo), AV_OPT_TYPE_DOUBLE, {.dbl=0.3}, 0, 1, FLAGS }, + { "lfo2_mod_amount_hi", NULL, OFFSET(lfo2_mod_amount_hi), AV_OPT_TYPE_DOUBLE, {.dbl=0.6}, 0, 1, FLAGS }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(aringmodulator); + +static const AVFilterPad ringmodulator_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + .config_props = ringmodulator_config_input, + .filter_frame = ringmodulator_filter_frame, + }, + { NULL } +}; + +static const AVFilterPad ringmodulator_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + }, + { NULL } +}; + +AVFilter ff_af_aringmodulator = { + .name = "aringmodulator", + .description = NULL_IF_CONFIG_SMALL("Audio Ring Modulator."), + .priv_size = sizeof(AudioRingModulatorContext), + .priv_class = åmodulator_class, + .query_formats = query_formats, + .inputs = ringmodulator_inputs, + .outputs = ringmodulator_outputs, +}; +#endif /* CONFIG_ARINGMODULATOR_FILTER */ diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 1b94501da0..6ec92181f3 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -65,6 +65,7 @@ extern AVFilter ff_af_apulsator; extern AVFilter ff_af_arealtime; extern AVFilter ff_af_aresample; extern AVFilter ff_af_areverse; +extern AVFilter ff_af_aringmodulator; extern AVFilter ff_af_arnndn; extern AVFilter ff_af_aselect; extern AVFilter ff_af_asendcmd;