From patchwork Sun Mar 25 16:41:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 8157 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.1.70 with SMTP id c67csp2662024jad; Sun, 25 Mar 2018 09:42:19 -0700 (PDT) X-Google-Smtp-Source: AG47ELsIzYLgjlWKWkR1p7EprNNu7IGVZXQMWUbR01jYXol0YmqTkgVertFt1FDthwNVBdr4yUWg X-Received: by 10.223.132.103 with SMTP id 94mr27249707wrf.275.1521996138925; Sun, 25 Mar 2018 09:42:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1521996138; cv=none; d=google.com; s=arc-20160816; b=XerJ81kIduTILwJnc2l1MobBQV27l/cTud6M5JqZ7/aUV6/8Gh34L8NXedzwjcLgOe BwmFRzrBDrn1avKt8S51q8w++HbJYwlNhUwi4hjeBPoo+k3hma2BgUS37puIo+lbsGUy wxMKDM7xT5LxivR5FV58RG9KLsWC5lMscvFFGnQH0Berp5Lq/Ld7cdaDYklTN1l0kgLf t+nB07GymrG1hMyoFsCaR4RJW0nmga1tg0VV8a+yMm/Uq0+3l3ZCeS2WS/tbLIgOrYQe aPBdYfGBLwzZoiXyEXCj7QRRI22doGk27wxVArvHWRj/r18IB5tHG/CfnPD047Stysve vS7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=nmJH82CVjcVpPs/HjxUSjHebWLcdtaA8NBi1AnvjPsk=; b=hnIPI3qBsQaUK5D9E+PBOPV7ivWyyJ2W4PHSvoL0n2vcSg5sw+CSJN5fdgUh2D+d+5 Dw55SMCe297jes93rQE6uoq9oAJ0XEH9sgFF+vqBSVvUIKU9LeyT/nPQIwov6CkMZJzn xYik0WOlIxiJyZar5kXxSpregItevfeBoPl1xRk3ho1GfLuz/VjiY52ruqmLSeGhjC+2 4vGd9CPlkrGtD3T8GRL+DOHeD9Lh66QiNnOi9X1pyBnU2dVHcFQYF10TuGn99HsTYqMN ylKwLb6/KHlG1EVCHNjLH5xf4I1ShETegnV1umh2t/qqLeRQN3iiljyLHOdidE1F3dVO sHHw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=rvBWVpHv; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s1si11231879wre.235.2018.03.25.09.42.17; Sun, 25 Mar 2018 09:42:18 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=rvBWVpHv; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C0C16689A66; Sun, 25 Mar 2018 19:41:59 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f66.google.com (mail-wm0-f66.google.com [74.125.82.66]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4CB176804BC for ; Sun, 25 Mar 2018 19:41:53 +0300 (EEST) Received: by mail-wm0-f66.google.com with SMTP id l9so11308210wmh.2 for ; Sun, 25 Mar 2018 09:42:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=R4OHx1nZypwJmbTvDU93Kkmev5ikINcl23e211EuMWQ=; b=rvBWVpHvFAsO+fhzglOOic2UPwIpkqA8tCVQlvqz/CoTnHbrVonkz0+uCEHw0ycmiM pN2Apu/uM13CaHEUZ/DaDt9p8ikw9xDQY9+otuWHB7narihFb+WpACMP8OyDCoRkAEuY qiunLt0aYAfbPC5eKG+MWr4A1uCtI9y5imyiXTc3tmVda8RXgSCjQiIgNfOj3P4epMOR VOHTvydf7ouJYm8ePhTtZfXUVihEaA6TaR6O3nQVfY9Fx2eSXPyzwDA4hfV/Qe6fLJ9b lxq1VlDHk0xu1yRlZvzJmU2LC0+GvEbgCpD7+wLqgNtfkkLgRz7AGn5hMOz74duCoLdg /kDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=R4OHx1nZypwJmbTvDU93Kkmev5ikINcl23e211EuMWQ=; b=CBW9ISkaSDa5O+Gwfr54hc4zzA3yvmFbi8aEEeMyxUy4egk2WkKd0RlfYswxr28/Qh mvXRUu3Vv6H6WkDEivrHMNrH33H8XxiHaPNP+jlW0lYJF8GGWYu2+OmrCnacbuek7xBC trCJot2m+LeHdii3sLSeNKSGAk/4u3mzbCdQlzgWxZVFysKmF4BS5oc7TGdMWFOU0UaE 7mi80L7wzeLjrjwa9uYCrgwyxf+8ggQ7TaWATZAgEb4eGR1izhtkPIZqJL/zOCQ24yyt t5QyknVJ4XVxOfng0/J3Tplio4tmqmjmBCLuylJsxNkMnYhBW0+SIbcQbl60wYN2GR61 YnqQ== X-Gm-Message-State: AElRT7Fsgm8CAkvIu0nMB2yrzZjQc6oLc0ZJ0FvKAB35iR4/4LLP2d5b W70H6MrGFOD+6niOxs0yzjEQcg== X-Received: by 10.28.24.66 with SMTP id 63mr13004159wmy.115.1521996128299; Sun, 25 Mar 2018 09:42:08 -0700 (PDT) Received: from localhost.localdomain ([94.250.174.60]) by smtp.gmail.com with ESMTPSA id 142sm16093678wmq.47.2018.03.25.09.42.07 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 25 Mar 2018 09:42:07 -0700 (PDT) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Sun, 25 Mar 2018 18:41:25 +0200 Message-Id: <20180325164125.22940-1-onemda@gmail.com> X-Mailer: git-send-email 2.11.0 Subject: [FFmpeg-devel] [PATCH 2/2] avfilter: add declip audio filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Paul B Mahol --- Depends on declick filter. --- doc/filters.texi | 28 +++++++++++++++++++ libavfilter/Makefile | 1 + libavfilter/af_declick.c | 73 +++++++++++++++++++++++++++++++++++++++++++++--- libavfilter/allfilters.c | 1 + 4 files changed, 99 insertions(+), 4 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 9a067ba9ea..86cf570ab9 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -2597,6 +2597,34 @@ This controls between how much samples, which are detected as impulsive noise, any sample between 2 detected noise samples is considered also as noise sample. @end table +@section declip +Remove clipped samples from input audio. + +Samples detected as clipped are replaced by interpolated samples using +autoregressive modeling. + +@table @option +@item w +Set window size, in milliseconds. Allowed range is from @code{10} to @code{100}. +Default value is @code{50} milliseconds. +This sets size of window which will be processed at once. + +@item o +Set window overlap, in percentage of window size. Allowed range is from @code{50} +to @code{95}. Default value is @code{75} percent. + +@item a +Set autoregression order, in percentage of window size. Allowed range is from +@code{1} to @code{50}. Default value is @code{2} percent. This option also controls +quality of interpolated samples using neighbour good samples. + +@item t +Set threshold value. Allowed range is from @code{0.2} to @code{1.0}. +Default value is @code{0.98}. +Any sample which absolute value is equal or higher of this value will be +detected as clipped and be replaced with interpolated value. +@end table + @section drmeter Measure audio dynamic range. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 978751d2a0..505287b5b4 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -88,6 +88,7 @@ OBJS-$(CONFIG_CROSSFEED_FILTER) += af_crossfeed.o OBJS-$(CONFIG_CRYSTALIZER_FILTER) += af_crystalizer.o OBJS-$(CONFIG_DCSHIFT_FILTER) += af_dcshift.o OBJS-$(CONFIG_DECLICK_FILTER) += af_declick.o +OBJS-$(CONFIG_DECLIP_FILTER) += af_declick.o OBJS-$(CONFIG_DRMETER_FILTER) += af_drmeter.o OBJS-$(CONFIG_DYNAUDNORM_FILTER) += af_dynaudnorm.o OBJS-$(CONFIG_EARWAX_FILTER) += af_earwax.o diff --git a/libavfilter/af_declick.c b/libavfilter/af_declick.c index 0de4c35c95..658dae420b 100644 --- a/libavfilter/af_declick.c +++ b/libavfilter/af_declick.c @@ -29,9 +29,11 @@ typedef struct DeclickContext { double w; double overlap; double threshold; + double clip_threshold; double ar; double burst; + int is_declip; int ar_order; int nb_burst_samples; int window_size; @@ -68,6 +70,10 @@ typedef struct DeclickContext { AVAudioFifo *fifo; double *window_func_lut; + + int (*detector)(struct DeclickContext *s, double sigmae, double *detection, + double *acoefficients, uint8_t *click, int *index, + const double *src, double *dst); } DeclickContext; #define OFFSET(x) offsetof(DeclickContext, x) @@ -354,6 +360,28 @@ static int interpolation(DeclickContext *s, const double *src, int ar_order, return cholesky_decomposition(s, matrix, vector, nb_clicks, interpolated); } +static int detect_clips(DeclickContext *s, double unused0, double *unused1, double *unused2, + uint8_t *clips, int *index, + const double *src, double *dst) +{ + const double threshold = s->clip_threshold; + int i, nb_clips = 0; + + for (i = 0; i < s->window_size; i++) { + clips[i] = fabs(src[i]) >= threshold; + dst[i] = src[i]; + } + + memset(clips, 0, s->ar_order * sizeof(*clips)); + memset(clips + (s->window_size - s->ar_order), 0, s->ar_order * sizeof(*clips)); + + for (i = s->ar_order; i < s->window_size - s->ar_order; i++) + if (clips[i]) + index[nb_clips++] = i; + + return nb_clips; +} + static int detect_clicks(DeclickContext *s, double sigmae, double *detection, double *acoefficients, uint8_t *click, int *index, const double *src, double *dst) @@ -441,8 +469,8 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) int *index = s->index; int nb_clicks; - nb_clicks = detect_clicks(s, sigmae, s->detection, s->acoefficients, - s->click, index, src, dst); + nb_clicks = s->detector(s, sigmae, s->detection, s->acoefficients, + s->click, index, src, dst); if (nb_clicks > 0) { ret = interpolation(s, src, s->ar_order, s->acoefficients, index, nb_clicks, s->auxiliary, interpolated); @@ -521,13 +549,27 @@ static int request_frame(AVFilterLink *outlink) return ret; } +static av_cold int init(AVFilterContext *ctx) +{ + DeclickContext *s = ctx->priv; + + s->is_declip = !strcmp(ctx->filter->name, "declip"); + if (s->is_declip) { + s->detector = detect_clips; + } else { + s->detector = detect_clicks; + } + + return 0; +} static av_cold void uninit(AVFilterContext *ctx) { DeclickContext *s = ctx->priv; - av_log(ctx, AV_LOG_INFO, "Detected clicks in %"PRId64" of %"PRId64" samples (%g%%).\n", - s->detected_clicks, s->nb_samples, 100. * s->detected_clicks / s->nb_samples); + av_log(ctx, AV_LOG_INFO, "Detected %s in %"PRId64" of %"PRId64" samples (%g%%).\n", + s->is_declip ? "clips" : "clicks", s->detected_clicks, + s->nb_samples, 100. * s->detected_clicks / s->nb_samples); av_audio_fifo_free(s->fifo); av_freep(&s->window_func_lut); @@ -580,6 +622,29 @@ AVFilter ff_af_declick = { .query_formats = query_formats, .priv_size = sizeof(DeclickContext), .priv_class = &declick_class, + .init = init, + .uninit = uninit, + .inputs = inputs, + .outputs = outputs, +}; + +static const AVOption declip_options[] = { + { "w", "set window size", OFFSET(w), AV_OPT_TYPE_DOUBLE, {.dbl=50}, 10, 100, AF }, + { "o", "set window overlap", OFFSET(overlap), AV_OPT_TYPE_DOUBLE, {.dbl=75}, 50, 95, AF }, + { "a", "set autoregression order", OFFSET(ar), AV_OPT_TYPE_DOUBLE, {.dbl=8}, 1, 50, AF }, + { "t", "set threshold", OFFSET(clip_threshold), AV_OPT_TYPE_DOUBLE, {.dbl=0.98}, 0.2, 1., AF }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(declip); + +AVFilter ff_af_declip = { + .name = "declip", + .description = NULL_IF_CONFIG_SMALL("Remove clipping from input audio."), + .query_formats = query_formats, + .priv_size = sizeof(DeclickContext), + .priv_class = &declip_class, + .init = init, .uninit = uninit, .inputs = inputs, .outputs = outputs, diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index cf5016f2c1..4d18e243ab 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -99,6 +99,7 @@ static void register_all(void) REGISTER_FILTER(CRYSTALIZER, crystalizer, af); REGISTER_FILTER(DCSHIFT, dcshift, af); REGISTER_FILTER(DECLICK, declick, af); + REGISTER_FILTER(DECLIP, declip, af); REGISTER_FILTER(DRMETER, drmeter, af); REGISTER_FILTER(DYNAUDNORM, dynaudnorm, af); REGISTER_FILTER(EARWAX, earwax, af);