From patchwork Wed Aug 9 21:36:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 43195 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8b1b:b0:130:ccc6:6c4b with SMTP id l27csp239376pzh; Wed, 9 Aug 2023 14:29:18 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH64ex8aFstikrf+RXOMQ9qKAKs6GjLh8rJ/+U4gm6u8tgrbyqBz/0NDp9zDqqV1XuP5W9p X-Received: by 2002:a05:6402:1856:b0:523:9c4:544f with SMTP id v22-20020a056402185600b0052309c4544fmr300825edy.31.1691616557977; Wed, 09 Aug 2023 14:29:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691616557; cv=none; d=google.com; s=arc-20160816; b=CsP8Y1fLDWdOENKs8oov6gCQOPn5tzddyGfe7pgVBfS90522JlRP7ONG64V4yRAA21 FibTDQQNmbnzpeGWGKyTPTopWz791eVshrtfAzbYSCErY9rNrtUGa7rqN7PcXqbJgk9f gF8+rd/BQNa18KcdidJ49OHRXV5OYJfbDFAI1IxLYgGfYRYB7lwxVls5MsngfFPro8tf GvAYoFlZnX9QT+xHSaoyuUB8ETtzT6euN2xcw3j0wFHzRDxyGxMXNNk4xyFs9KmTtze/ JiQEANZlcvbjiVAhyttkirIdDMVvYzcFBeR85x6hjVUByvL+grSZLVtzeyHrVtiJYzgQ U1SQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject:to :message-id:date:from:mime-version:dkim-signature:delivered-to; bh=9fE3VLxd9/5HGZmyJX3hQEnB3LbfCsg0Mg6naJtlEyI=; fh=ghmNPjFTSM6oKC/7k5yo9hnu8V2TFPrY7GkS/gBuc+4=; b=RI3OfFNQ27HgYkmtahZmzLfm/GTC0HTT4hdQsu6zDjJe0V/CQr0kZfBHjcp8bBtnGm 3YqWPWs9hewLOurgPY1kyPAMJXSSxYfYU/Cw7UXS9k0xs3OJaScaNEfvx371FtxCk/Ad hHHmqfubOqZ503GHWwAY63raYlGoZbH8z1Kt0yMAF/wyztqARudGzPH5RxHxY/tnV00Y T1mHs+jeJVH5sq4GOUDEIx/VaQqM1LWzYHj5WIahUlql6wuIPftff5A6PondB2BTiKj0 WFOWg6baVayPXfm0Hz/MPYW/i8BGH2SxEnWUngYw964oQuB1QbxpSdrI1ZjiCh0wgciD 4wsw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20221208 header.b=DZ55v8FW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id l11-20020aa7c30b000000b005233fd1111fsi3288988edq.398.2023.08.09.14.29.15; Wed, 09 Aug 2023 14:29:17 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20221208 header.b=DZ55v8FW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 838BB68C8A0; Thu, 10 Aug 2023 00:29:12 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-vk1-f178.google.com (mail-vk1-f178.google.com [209.85.221.178]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2351568C807 for ; Thu, 10 Aug 2023 00:29:06 +0300 (EEST) Received: by mail-vk1-f178.google.com with SMTP id 71dfb90a1353d-48711283853so130102e0c.0 for ; Wed, 09 Aug 2023 14:29:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691616544; x=1692221344; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=QddAQLSr+ppXAsbABNKwQu59kCd4lvJ81ZvsQ2Kmtx0=; b=DZ55v8FWrIwqRpJEXrZHDxYjtQmJju/kw0f7M26SQxSzy/3moRbKFMWoqT1Pk0rULx QT19pbZAwFC8fcsKY9h/SuFxJYyg6qZikta0eAAo1eG3fzXO+GSlDGJ8SXWF4OtvT3B+ 8aG20OGv4pJGR5kevAMrgMZ2c/b/czA/SHZabr4Lcamp6UWx22hrG+WZO6OyNy9/g/W/ Yz6V/9AaciHKB6DyHrgzTw+9KNAX/QXkeMvUl82PoGzsrPQyIqTKXC4aifaOT63eIRBC ekstrkFPYYnQvr4aUAoZURRkUNUiyeCjL2vwWM0qhVmrIyCGcFzqNlqVuqN6W6PWEbwb fpVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691616544; x=1692221344; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=QddAQLSr+ppXAsbABNKwQu59kCd4lvJ81ZvsQ2Kmtx0=; b=Es4J0a1HD9fRVX9RrlbA9xj1wLpQZmGufRcFPCwPaF8CVjHH1XkQkivHE1T8WDeW4w c9r5mvPe6dT0xYNIy8qeoFoE5cd45CD8R19n66ls8wWSgkRSOLOg5dKcKyyfJ/FV3n/K mBIq6ltN5DcWVME6j+KmZt/VTlU34AIOt38N7xRWm9ZAhq6QNL9vuPT/g9lr35CXTzxg 5kL3i9uWmfDmz6TdRAO9pkg376pQsS4lW6Mt9WmRc5ZXAY/l3O+XVoQkZw3R96T5T3yF T99EvMPw/pszNdkAJLYH0C5ywy61G+rK4Gh1F5SLobA5BGKQfR/B0gJ8xqKaY9Nm23OZ iSaA== X-Gm-Message-State: AOJu0YyYcAHRuqm5oCh4LpS2LJjvryv4HHozfVf4egpmcjHvgspTD1AV Vt9+ETBgHAEC2A4RlhvZpeOzWWfwyol0ZkS5V0M3JhZYrUg= X-Received: by 2002:a1f:eb42:0:b0:487:179c:25e1 with SMTP id j63-20020a1feb42000000b00487179c25e1mr714157vkh.4.1691616544585; Wed, 09 Aug 2023 14:29:04 -0700 (PDT) MIME-Version: 1.0 From: Paul B Mahol Date: Wed, 9 Aug 2023 23:36:16 +0200 Message-ID: To: FFmpeg development discussions and patches X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: [FFmpeg-devel] [PATCH] new audio filter and misc improvements X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: CS9aEDFDAfcg Patches attached. From af565f57f733af327edc1e1724e31a3c5f1fe44f Mon Sep 17 00:00:00 2001 From: Paul B Mahol Date: Wed, 9 Aug 2023 18:21:25 +0200 Subject: [PATCH 1/3] avfilter: add adesurge filter Signed-off-by: Paul B Mahol --- doc/filters.texi | 51 ++++++++++++++++++++++ libavfilter/Makefile | 1 + libavfilter/af_adeclick.c | 91 +++++++++++++++++++++++++++++++++++++-- libavfilter/allfilters.c | 1 + 4 files changed, 140 insertions(+), 4 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index b30ff8240b..e041adc40f 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -913,6 +913,57 @@ Compute derivative/integral of audio stream. Applying both filters one after another produces original audio. +@section adesurge +Remove big surge in audio samples from input audio. + +Samples detected as instant increase or decrease of volume are replaced by +interpolated samples using autoregressive modelling. + +@table @option +@item window, w +Set window size, in milliseconds. Allowed range is from @code{10} to +@code{100}. Default value is @code{85} milliseconds. +This sets size of window which will be processed at once. + +@item overlap, o +Set window overlap, in percentage of window size. Allowed range is from +@code{50} to @code{95}. Default value is @code{75} percent. +Setting this to a very high value may increases surges removal but makes +whole process much slower. + +@item arorder, a +Set autoregression order, in percentage of window size. Allowed range is from +@code{0} to @code{25}. Default value is @code{0.5} percent. This option also +controls quality of interpolated samples using neighbour good samples. + +@item threshold, t +Set threshold value. Allowed range is from @code{1} to @code{100}. +Default value is @code{20}. +This controls the strength of surges which is going to be removed. +The lower value, the more samples will be detected as surges. + +@item surges, s +Set surges size, in number of samples. Allowed range is @code{1} to +@code{50}. Default value is @code{5}. +For any sample detected as surge then also its surrounding samples of this size +will be interpolated also. + +@item method, m +Set overlap method. + +It accepts the following values: +@table @option +@item add, a +Select overlap-add method. Even not interpolated samples are slightly +changed with this method. + +@item save, s +Select overlap-save method. Not interpolated samples remain unchanged. +@end table + +Default value is @code{s}. +@end table + @section adrc Apply spectral dynamic range controller filter to input audio stream. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 30a0e22ef8..b26a85a7d2 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -49,6 +49,7 @@ OBJS-$(CONFIG_ADECORRELATE_FILTER) += af_adecorrelate.o OBJS-$(CONFIG_ADELAY_FILTER) += af_adelay.o OBJS-$(CONFIG_ADENORM_FILTER) += af_adenorm.o OBJS-$(CONFIG_ADERIVATIVE_FILTER) += af_aderivative.o +OBJS-$(CONFIG_ADESURGE_FILTER) += af_adesurge.o OBJS-$(CONFIG_ADRC_FILTER) += af_adrc.o OBJS-$(CONFIG_ADYNAMICEQUALIZER_FILTER) += af_adynamicequalizer.o OBJS-$(CONFIG_ADYNAMICSMOOTH_FILTER) += af_adynamicsmooth.o diff --git a/libavfilter/af_adeclick.c b/libavfilter/af_adeclick.c index 822f3065b8..1137dbe25b 100644 --- a/libavfilter/af_adeclick.c +++ b/libavfilter/af_adeclick.c @@ -25,6 +25,8 @@ #include "filters.h" #include "internal.h" +static const char *filter_modes[] = {"clicks", "clips", "surges" }; + typedef struct DeclickChannel { double *auxiliary; double *detection; @@ -55,8 +57,9 @@ typedef struct AudioDeclickContext { int method; int nb_hbins; - int is_declip; + int mode; int ar_order; + int nb_surge_samples; int nb_burst_samples; int window_size; int hop_size; @@ -468,6 +471,46 @@ static int detect_clicks(AudioDeclickContext *s, DeclickChannel *c, return nb_clicks; } +static int detect_surges(AudioDeclickContext *s, DeclickChannel *c, + double sigmae, + double *detection, double *acoefficients, + uint8_t *surge, int *index, + const double *src, double *dst) +{ + const double threshold = s->threshold; + const int size = s->nb_surge_samples * 2; + int i, j, nb_surges = 0; + + memset(detection, 0, s->window_size * sizeof(*detection)); + + for (i = s->ar_order; i < s->window_size; i++) { + for (j = 0; j <= s->ar_order; j++) { + detection[i] += acoefficients[j] * src[i - j]; + } + } + + for (i = 0; i < s->window_size; i++) { + surge[i] = fabs(detection[i]) > sigmae * threshold; + dst[i] = src[i]; + } + + for (i = 0; i < s->window_size;) { + if (!surge[i++]) + continue; + memset(surge + FFMAX(i - size/2, 0), 1, FFMIN(size, s->window_size - i)); + i += size/2; + } + + memset(surge, 0, s->ar_order * sizeof(*surge)); + memset(surge + (s->window_size - s->ar_order), 0, s->ar_order * sizeof(*surge)); + + for (i = s->ar_order; i < s->window_size - s->ar_order; i++) + if (surge[i]) + index[nb_surges++] = i; + + return nb_surges; +} + typedef struct ThreadData { AVFrame *out; } ThreadData; @@ -661,10 +704,14 @@ static av_cold int init(AVFilterContext *ctx) { AudioDeclickContext *s = ctx->priv; - s->is_declip = !strcmp(ctx->filter->name, "adeclip"); - if (s->is_declip) { + if (!strcmp(ctx->filter->name, "adesurge")) { + s->mode = 2; + s->detector = detect_surges; + } else if (!strcmp(ctx->filter->name, "adeclip")) { + s->mode = 1; s->detector = detect_clips; } else { + s->mode = 0; s->detector = detect_clicks; } @@ -677,7 +724,7 @@ static av_cold void uninit(AVFilterContext *ctx) int i; av_log(ctx, AV_LOG_INFO, "Detected %s in %"PRId64" of %"PRId64" samples (%g%%).\n", - s->is_declip ? "clips" : "clicks", s->detected_errors, + filter_modes[s->mode], s->detected_errors, s->nb_samples, 100. * s->detected_errors / s->nb_samples); av_audio_fifo_free(s->fifo); @@ -772,3 +819,39 @@ const AVFilter ff_af_adeclip = { FILTER_SINGLE_SAMPLEFMT(AV_SAMPLE_FMT_DBLP), .flags = AVFILTER_FLAG_SLICE_THREADS | AVFILTER_FLAG_SUPPORT_TIMELINE_INTERNAL, }; + +static const AVOption adesurge_options[] = { + { "window", "set window size", OFFSET(w), AV_OPT_TYPE_DOUBLE, {.dbl=85}, 10, 100, AF }, + { "w", "set window size", OFFSET(w), AV_OPT_TYPE_DOUBLE, {.dbl=85}, 10, 100, AF }, + { "overlap", "set window overlap", OFFSET(overlap), AV_OPT_TYPE_DOUBLE, {.dbl=75}, 50, 95, AF }, + { "o", "set window overlap", OFFSET(overlap), AV_OPT_TYPE_DOUBLE, {.dbl=75}, 50, 95, AF }, + { "arorder", "set autoregression order", OFFSET(ar), AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 25, AF }, + { "a", "set autoregression order", OFFSET(ar), AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 25, AF }, + { "threshold", "set threshold", OFFSET(threshold), AV_OPT_TYPE_DOUBLE, {.dbl=20}, 1, 100, AF }, + { "t", "set threshold", OFFSET(threshold), AV_OPT_TYPE_DOUBLE, {.dbl=20}, 1, 100, AF }, + { "surges", "set surge size", OFFSET(nb_surge_samples), AV_OPT_TYPE_INT, {.i64=5}, 1, 50, AF }, + { "s", "set surge size", OFFSET(nb_surge_samples), AV_OPT_TYPE_INT, {.i64=5}, 1, 50, AF }, + { "method", "set overlap method", OFFSET(method), AV_OPT_TYPE_INT, {.i64=1}, 0, 1, AF, "m" }, + { "m", "set overlap method", OFFSET(method), AV_OPT_TYPE_INT, {.i64=1}, 0, 1, AF, "m" }, + { "add", "overlap-add", 0, AV_OPT_TYPE_CONST, {.i64=0}, 0, 0, AF, "m" }, + { "a", "overlap-add", 0, AV_OPT_TYPE_CONST, {.i64=0}, 0, 0, AF, "m" }, + { "save", "overlap-save", 0, AV_OPT_TYPE_CONST, {.i64=1}, 0, 0, AF, "m" }, + { "s", "overlap-save", 0, AV_OPT_TYPE_CONST, {.i64=1}, 0, 0, AF, "m" }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(adesurge); + +const AVFilter ff_af_adesurge = { + .name = "adesurge", + .description = NULL_IF_CONFIG_SMALL("Remove surges from input audio."), + .priv_size = sizeof(AudioDeclickContext), + .priv_class = &adesurge_class, + .init = init, + .activate = activate, + .uninit = uninit, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(ff_audio_default_filterpad), + FILTER_SINGLE_SAMPLEFMT(AV_SAMPLE_FMT_DBLP), + .flags = AVFILTER_FLAG_SLICE_THREADS | AVFILTER_FLAG_SUPPORT_TIMELINE_INTERNAL, +}; diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 089ad3a0ed..286e601799 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -35,6 +35,7 @@ extern const AVFilter ff_af_adecorrelate; extern const AVFilter ff_af_adelay; extern const AVFilter ff_af_adenorm; extern const AVFilter ff_af_aderivative; +extern const AVFilter ff_af_adesurge; extern const AVFilter ff_af_adrc; extern const AVFilter ff_af_adynamicequalizer; extern const AVFilter ff_af_adynamicsmooth; -- 2.39.1