From patchwork Sun Jun 30 16:05:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 13773 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id D1BC3449AAC for ; Sun, 30 Jun 2019 19:05:53 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id AC24E689A5E; Sun, 30 Jun 2019 19:05:53 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DBAED689909 for ; Sun, 30 Jun 2019 19:05:47 +0300 (EEST) Received: by mail-wm1-f44.google.com with SMTP id 207so13424596wma.1 for ; Sun, 30 Jun 2019 09:05:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=c6BTttyUYkS932w3oD+ep58cBKpgRm//gqz40UKmG8k=; b=Y6kuieCMOj3zL7RgtzMRWKo+SzUDCupZtCcpCVcAG919JqXtaSRIecfxS1apyxowzw e7FxKoItKsqC8GZo4ur9xsbT0Edc7+X7ljMT9ZnrmeukS14l9kQFWL76ysKxdeb+Qk3G JWYpq90uizH1ZnxB3mdpfdAtb6GLmPBv58p11ZvYLElq+ITjunWOtynu3a7lzrq3vdOf qYPMN4CthSf8gP13eALEnhU0zCXzl3v/JdSOm15sMnrmTtQKglOeKWZSs7bObLPUuZlI vklYPO01i3WRntq36Pvx+CUf5sWd2gh8IQ/2nenEluNkqRW+CStxPeRXnuQ67C6gk5Ez CAhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=c6BTttyUYkS932w3oD+ep58cBKpgRm//gqz40UKmG8k=; b=uK05czSy1z9vI0Qf4VAU9uwVQrabLh4gphCjX0CFhF0LC2cdiOQGyFpZD9PGGF84Lh 0zsu5CoKi9SuEoeEA0U3HnnRrIymy8AAQG1YJcntNGhzGDeRxgT+JQOFo7TtxqCHCSvm I2lW0iS09Tg9yA8GUXcrgvY5s3e/UYPnXLYBY+LdXUhgcgdrzaqZcN0RjLhplsKOPYgy j3ndHW9g2f+B0vINVv0B/iV7C1zCEpEBb7x+cDe29oixeY3rpBsEyIZVQXl1IYFdZRdi N+zFjagKk0YNoFq61+FBNyFxqx0N81plSGY1/i0gT3GvvLbQM1p39PBy22IYqDHV/K0C BhVA== X-Gm-Message-State: APjAAAUiyhn8OnwnGBnVrVJWetRm8ceDrC0xyTTiQbdmKfJLA8SMYB5q k8lQiWf1ayZ0CaBHBnkLviaeitlr X-Google-Smtp-Source: APXvYqwoR/l9XRMD+bi9NFERiCraRmamTP9tYNXRolh8geQKsWtgA0HYhZmiWKCrmw64eWdEF5YuJQ== X-Received: by 2002:a1c:b457:: with SMTP id d84mr15171937wmf.153.1561910746933; Sun, 30 Jun 2019 09:05:46 -0700 (PDT) Received: from localhost.localdomain ([77.237.101.48]) by smtp.gmail.com with ESMTPSA id l124sm10796493wmf.36.2019.06.30.09.05.45 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 30 Jun 2019 09:05:46 -0700 (PDT) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Sun, 30 Jun 2019 18:05:36 +0200 Message-Id: <20190630160536.2685-1-onemda@gmail.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH] avfilter: add deesser audio filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Paul B Mahol --- doc/filters.texi | 36 ++++++ libavfilter/Makefile | 1 + libavfilter/af_deesser.c | 240 +++++++++++++++++++++++++++++++++++++++ libavfilter/allfilters.c | 1 + 4 files changed, 278 insertions(+) create mode 100644 libavfilter/af_deesser.c diff --git a/doc/filters.texi b/doc/filters.texi index 2d9af46a6b..8cca5d008a 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -3045,6 +3045,42 @@ Optional. It should have a value much less than 1 (e.g. 0.05 or 0.02) and is used to prevent clipping. @end table +@section deesser + +Apply de-essing to the audio samples. + +@table @option +@item i +Set intensity for triggering de-essing. Allowed range is from 0 to 1. +Default is 0. + +@item m +Set amount of ducking on treble part of sound. Allowed range is from 0 to 1. +Default is 0.5. + +@item f +How much of original frequency content to keep when de-essing. Allowed range is from 0 to 1. +Default is 0.5. + +@item s +Set the output mode. + +It accepts the following values: +@table @option +@item i +Pass input unchanged. + +@item o +Pass ess filtered out. + +@item e +Pass only ess. + +Default value is @var{o}. +@end table + +@end table + @section drmeter Measure audio dynamic range. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 07ea8d7edc..455c809b15 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -101,6 +101,7 @@ OBJS-$(CONFIG_COMPENSATIONDELAY_FILTER) += af_compensationdelay.o OBJS-$(CONFIG_CROSSFEED_FILTER) += af_crossfeed.o OBJS-$(CONFIG_CRYSTALIZER_FILTER) += af_crystalizer.o OBJS-$(CONFIG_DCSHIFT_FILTER) += af_dcshift.o +OBJS-$(CONFIG_DEESSER_FILTER) += af_deesser.o OBJS-$(CONFIG_DRMETER_FILTER) += af_drmeter.o OBJS-$(CONFIG_DYNAUDNORM_FILTER) += af_dynaudnorm.o OBJS-$(CONFIG_EARWAX_FILTER) += af_earwax.o diff --git a/libavfilter/af_deesser.c b/libavfilter/af_deesser.c new file mode 100644 index 0000000000..1775dc7d4c --- /dev/null +++ b/libavfilter/af_deesser.c @@ -0,0 +1,240 @@ +/* + * Copyright (c) 2018 Chris Johnson + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +#include "libavutil/channel_layout.h" +#include "libavutil/opt.h" +#include "avfilter.h" +#include "audio.h" +#include "formats.h" + +typedef struct DeesserChannel { + double s1, s2, s3; + double m1, m2; + double ratioA, ratioB; + double iirSampleA, iirSampleB; + int flip; +} DeesserChannel; + +typedef struct DeesserContext { + const AVClass *class; + + double intensity; + double max; + double frequency; + int mode; + + DeesserChannel *chan; +} DeesserContext; + +enum OutModes { + IN_MODE, + OUT_MODE, + ESS_MODE, + NB_MODES +}; + +#define OFFSET(x) offsetof(DeesserContext, x) +#define A AV_OPT_FLAG_AUDIO_PARAM|AV_OPT_FLAG_FILTERING_PARAM + +static const AVOption deesser_options[] = { + { "i", "set intensity", OFFSET(intensity), AV_OPT_TYPE_DOUBLE, {.dbl=0.0}, 0.0, 1.0, A }, + { "m", "set max deessing", OFFSET(max), AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0.0, 1.0, A }, + { "f", "set frequency", OFFSET(frequency), AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0.0, 1.0, A }, + { "s", "set output mode", OFFSET(mode), AV_OPT_TYPE_INT, {.i64=OUT_MODE}, 0, NB_MODES-1, A, "mode" }, + { "i", "input", 0, AV_OPT_TYPE_CONST, {.i64=IN_MODE}, 0, 0, A, "mode" }, + { "o", "output", 0, AV_OPT_TYPE_CONST, {.i64=OUT_MODE}, 0, 0, A, "mode" }, + { "e", "ess", 0, AV_OPT_TYPE_CONST, {.i64=ESS_MODE}, 0, 0, A, "mode" }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(deesser); + +static int query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats = NULL; + AVFilterChannelLayouts *layouts = NULL; + static const enum AVSampleFormat sample_fmts[] = { + AV_SAMPLE_FMT_DBLP, + AV_SAMPLE_FMT_NONE + }; + int ret; + + formats = ff_make_format_list(sample_fmts); + if (!formats) + return AVERROR(ENOMEM); + ret = ff_set_common_formats(ctx, formats); + if (ret < 0) + return ret; + + layouts = ff_all_channel_counts(); + if (!layouts) + return AVERROR(ENOMEM); + + ret = ff_set_common_channel_layouts(ctx, layouts); + if (ret < 0) + return ret; + + formats = ff_all_samplerates(); + return ff_set_common_samplerates(ctx, formats); +} + +static int config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + DeesserContext *s = ctx->priv; + + s->chan = av_calloc(inlink->channels, sizeof(*s->chan)); + if (!s->chan) + return AVERROR(ENOMEM); + + for (int i = 0; i < inlink->channels; i++) { + DeesserChannel *chan = &s->chan[i];; + + chan->ratioA = chan->ratioB = 1.0; + } + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx = inlink->dst; + AVFilterLink *outlink = ctx->outputs[0]; + DeesserContext *s = ctx->priv; + AVFrame *out; + + if (av_frame_is_writable(in)) { + out = in; + } else { + out = ff_get_audio_buffer(outlink, in->nb_samples); + if (!out) { + av_frame_free(&in); + return AVERROR(ENOMEM); + } + av_frame_copy_props(out, in); + } + + for (int ch = 0; ch < inlink->channels; ch++) { + DeesserChannel *dec = &s->chan[ch]; + double *src = (double *)in->extended_data[ch]; + double *dst = (double *)out->extended_data[ch]; + double overallscale = (1.0 / 44100.0) * inlink->sample_rate; + double intensity = pow(s->intensity, 5) * (8192 / overallscale); + double maxdess = 1.0 / pow(10.0,((s->max-1.0)*48.0) / 20); + double iirAmount = pow(s->frequency, 2) / overallscale; + double offset; + double sense; + double recovery; + double attackspeed; + + for (int i = 0; i < in->nb_samples; i++) { + double sample = src[i]; + + dec->s3 = dec->s2; + dec->s2 = dec->s1; + dec->s1 = sample; + dec->m1 = (dec->s1 - dec->s2) * ((dec->s1 - dec->s2) / 1.3); + dec->m2 = (dec->s2 - dec->s3) * ((dec->s1 - dec->s2) / 1.3); + sense = fabs((dec->m1 - dec->m2) * ((dec->m1 - dec->m2) / 1.3)); + attackspeed = 7.0 + sense * 1024; + + sense = 1.0 + intensity * intensity * sense; + sense = FFMIN(sense, intensity); + recovery = 1.0 + (0.01 / sense); + + offset = 1.0 - fabs(sample); + + if (dec->flip) { + dec->iirSampleA = (dec->iirSampleA * (1.0 - (offset * iirAmount))) + + (sample * (offset * iirAmount)); + if (dec->ratioA < sense) { + dec->ratioA = ((dec->ratioA * attackspeed) + sense) / (attackspeed + 1.0); + } else { + dec->ratioA = 1.0 + ((dec->ratioA - 1.0) / recovery); + } + + dec->ratioA = FFMIN(dec->ratioA, maxdess); + sample = dec->iirSampleA + ((sample - dec->iirSampleA) / dec->ratioA); + } else { + dec->iirSampleB = (dec->iirSampleB * (1.0 - (offset * iirAmount))) + + (sample * (offset * iirAmount)); + if (dec->ratioB < sense) { + dec->ratioB = ((dec->ratioB * attackspeed) + sense) / (attackspeed + 1.0); + } else { + dec->ratioB = 1.0 + ((dec->ratioB - 1.0) / recovery); + } + + dec->ratioB = FFMIN(dec->ratioB, maxdess); + sample = dec->iirSampleB + ((sample - dec->iirSampleB) / dec->ratioB); + } + + dec->flip = !dec->flip; + + switch (s->mode) { + case IN_MODE: dst[i] = src[i]; break; + case OUT_MODE: dst[i] = sample; break; + case ESS_MODE: dst[i] = src[i] - sample; break; + } + } + } + + if (out != in) + av_frame_free(&in); + + return ff_filter_frame(outlink, out); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + DeesserContext *s = ctx->priv; + + av_freep(&s->chan); +} + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + .filter_frame = filter_frame, + .config_props = config_input, + }, + { NULL } +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + }, + { NULL } +}; + +AVFilter ff_af_deesser = { + .name = "deesser", + .description = NULL_IF_CONFIG_SMALL("Apply de-essing to the audio."), + .query_formats = query_formats, + .priv_size = sizeof(DeesserContext), + .priv_class = &deesser_class, + .uninit = uninit, + .inputs = inputs, + .outputs = outputs, +}; diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 9c846b1ddd..04a3df7d56 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -93,6 +93,7 @@ extern AVFilter ff_af_compensationdelay; extern AVFilter ff_af_crossfeed; extern AVFilter ff_af_crystalizer; extern AVFilter ff_af_dcshift; +extern AVFilter ff_af_deesser; extern AVFilter ff_af_drmeter; extern AVFilter ff_af_dynaudnorm; extern AVFilter ff_af_earwax;