From patchwork Sun Mar 27 06:08:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wang Cao X-Patchwork-Id: 35000 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:6d9d:b0:79:214c:b845 with SMTP id gl29csp3688494pzb; Sat, 26 Mar 2022 23:08:29 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxb8U3HS9tAkmwuXcLXWm12gxVHXEwqMSmjE8pJIyYT/W/+t9BbXLMhq4kxAHFJKjSdKMsm X-Received: by 2002:aa7:d98f:0:b0:419:1200:91e0 with SMTP id u15-20020aa7d98f000000b00419120091e0mr8832716eds.145.1648361309395; Sat, 26 Mar 2022 23:08:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1648361309; cv=none; d=google.com; s=arc-20160816; b=isRxVu1twRcyjB1lBOS/tjkZy/YkQS8VVWB4j/SSAYSsCL2lfiwdurm8QjeJxHzwS8 JIsksfvJENzWYiWzyBISYka4W0WpBGDp7z0ZURxt9U4f8UyOHXAaU/vNDXe0USMkIEK4 H1WaXJ6iMY5FCrGAhIbSq9y/lXrdyNxPlCXDKJ10vFcEAyJgTj/FrV4DQx0hf5K1yZOX LFarmKNtbge3UjU6xmDCfyletb+u25wJnZX3Y9u5Uk0ZExbjiraeoT36yWbg+RnBH6tM QxnGF9U01xkkTptH7mWbxRSozfkADlddZj8ydMPNK7OV+2Ug6XDRaoNXbwSrBWjHtV8E WSjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:from:mime-version:message-id:date :dkim-signature:delivered-to; bh=AFNUZsVopN9SeSagtzgg9d8nPBCgT/GDgX3Xr/3IdK8=; b=D6aHPFEKUmG402BpK7CfD9qWAp3IGz61SMvU2KoD7pN3mrDr5pOK/b4op/ZOjRHryn 4hMJslZRwg6RYZXVg4RwKhxYwQtokSr1zSXXLi4x6xyb4u0wS5qCfJwMso9iBppPJ3xD AUo0eCVuaiYNzwkwfVCzwTzr3C4yW7YYQ4Jhacma6Oy9isyDynWF2oJK2nJAqkMWbzxf J1BuuyMX2NFeDjnKWUQSZd/tNb0yaF1K1VsGivLr83Wi+hPj/wYaiCA3jW+3S1nymGRw ujHvkMo6Oq7PikjpPldZbyavP02lAhLG0oDfntFK6/NklFyY/uNkL+iM+PwXrrwt6pFY TBCA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=nVSFCwKB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id zk16-20020a17090733d000b006dfa3f0e6f9si9045295ejb.545.2022.03.26.23.08.28; Sat, 26 Mar 2022 23:08:29 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=nVSFCwKB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0A7B068A320; Sun, 27 Mar 2022 09:08:25 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A639168A320 for ; Sun, 27 Mar 2022 09:08:18 +0300 (EEST) Received: by mail-yb1-f201.google.com with SMTP id w1-20020a5b08c1000000b00633be107f28so9047490ybq.4 for ; Sat, 26 Mar 2022 23:08:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:message-id:mime-version:subject:from:to:cc; bh=wlxIflX2A/2CmvGvcJpdEXPTWxEO5wohfTQoAuUBk58=; b=nVSFCwKBqJhQ52mBZMEiqaDBKc2cQQ9Gfk+03bB5iWRl8e2eTlJoSyc2ToCbr8jOKz Pp8iWuoQD00gMK4F1QwSH7FIG87tA8weJE+hvwP948fCJp8K5Tt//tMCmUqjkF9rtc9t HdkQQI8a+B3PBLaCToxAEJ4vPQwF/iirPu/MkNE0lbQpXm5ZyMEaNNAscsl36Zj9LT0x hwSMD5aCRBFFZj31zrjOADwIoRoPWvRUkRHULkEx4tjqT8YVA7s6Otz4GVowyRxdoAd3 2Uv9maoaBrgm2reXTtzSwAQJkf9O5/crBA2MvLVrwWqv+n8HyV+b3Dr4b1F8hou/Ylj6 pn3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=wlxIflX2A/2CmvGvcJpdEXPTWxEO5wohfTQoAuUBk58=; b=0i7up4dN09fq5NuORf4MefPwcyiB7ucZssto1YOKlkbbl0En4+hZCI1NXlq8HAVYZw styCQRUyfZ/sHv1V8S6KhumxVb1Soq/SbUfkKXOF+5wQ8nkoQ6OrCQ6MfnrNtNbas6Fy 87udAuE2xcgrf1q6oDWz8mgVcKyJcBjAc18Q5EFVm3sIYVgQopLlVLjIe+Nqs0nzI03E nM3wgZxJp5YsNVeNqpkvH3/yBtLg4BoNayfW76woND+7oYgDu/He5JAb5s/JbIk4lyVT TsRrEV5Sz5YaAEqBbo2Y6RuA3x9z4uUJjKebojjHpW3VIwzIscOMNNwjaOEAnZXFnZ9D xxhA== X-Gm-Message-State: AOAM531f2YyK5cY0ZKKA2rKKFIixf94U3WoO3KZVPaJgbF5LL6i4xqgH oV/Q6GDKXND773rpygZYBYL2tKHAUuy0Zjen5EJStSXZg8T7WTaGalEIhnmZoHVgvl6t9Q25P3V LX2arPtEX9djU2XSk6GufMKzLlCl0oypaXn2BrOcaJH2qd/R4ONBZ+0kqbyJCVblbiA== X-Received: from wangcao.mtv.corp.google.com ([2620:0:1000:4011:8e1b:338b:a60c:e834]) (user=wangcao job=sendgmr) by 2002:a0d:ebc4:0:b0:2e5:c810:f0e0 with SMTP id u187-20020a0debc4000000b002e5c810f0e0mr19261913ywe.155.1648361296432; Sat, 26 Mar 2022 23:08:16 -0700 (PDT) Date: Sat, 26 Mar 2022 23:08:01 -0700 Message-Id: <20220327060800.3732289-1-wangcao@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.35.1.1021.g381101b075-goog From: Wang Cao To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wang Cao Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ugCVmnBZG0gx The change in the commit will add some samples to the end of the audio stream. The intention is to add a "zero_delay" option eventually to not have the delay in the begining the output from alimiter due to lookahead. Signed-off-by: Wang Cao --- doc/filters.texi | 5 ++++ libavfilter/af_alimiter.c | 57 ++++++++++++++++++++++++++++++++------- 2 files changed, 53 insertions(+), 9 deletions(-) If the intention is clear to you, do you prefer us add the "zero_delay" option to the same patch or it needs to go in a separate patch? Thanks! diff --git a/doc/filters.texi b/doc/filters.texi index d70ac3e237..bb8f7c1a0b 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -1978,6 +1978,11 @@ in release time while 1 produces higher release times. @item level Auto level output signal. Default is enabled. This normalizes audio back to 0dB if enabled. + +@item flush_buffer +Flushes the internal buffer so that all the input audio samples to the limiter +will appear to the output. Currently due to lookahead buffer, the total number +of output samples will be larger than the input. @end table Depending on picked setting it is recommended to upsample input 2x or 4x times diff --git a/libavfilter/af_alimiter.c b/libavfilter/af_alimiter.c index 133f98f165..ba0a1361ac 100644 --- a/libavfilter/af_alimiter.c +++ b/libavfilter/af_alimiter.c @@ -55,6 +55,9 @@ typedef struct AudioLimiterContext { int *nextpos; double *nextdelta; + int flush_buffer; + int total_samples_to_flush; + double delta; int nextiter; int nextlen; @@ -65,14 +68,15 @@ typedef struct AudioLimiterContext { #define AF AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_FILTERING_PARAM | AV_OPT_FLAG_RUNTIME_PARAM static const AVOption alimiter_options[] = { - { "level_in", "set input level", OFFSET(level_in), AV_OPT_TYPE_DOUBLE, {.dbl=1},.015625, 64, AF }, - { "level_out", "set output level", OFFSET(level_out), AV_OPT_TYPE_DOUBLE, {.dbl=1},.015625, 64, AF }, - { "limit", "set limit", OFFSET(limit), AV_OPT_TYPE_DOUBLE, {.dbl=1}, 0.0625, 1, AF }, - { "attack", "set attack", OFFSET(attack), AV_OPT_TYPE_DOUBLE, {.dbl=5}, 0.1, 80, AF }, - { "release", "set release", OFFSET(release), AV_OPT_TYPE_DOUBLE, {.dbl=50}, 1, 8000, AF }, - { "asc", "enable asc", OFFSET(auto_release), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, - { "asc_level", "set asc level", OFFSET(asc_coeff), AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 1, AF }, - { "level", "auto level", OFFSET(auto_level), AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, AF }, + { "level_in", "set input level", OFFSET(level_in), AV_OPT_TYPE_DOUBLE, {.dbl=1},.015625, 64, AF }, + { "level_out", "set output level", OFFSET(level_out), AV_OPT_TYPE_DOUBLE, {.dbl=1},.015625, 64, AF }, + { "limit", "set limit", OFFSET(limit), AV_OPT_TYPE_DOUBLE, {.dbl=1}, 0.0625, 1, AF }, + { "attack", "set attack", OFFSET(attack), AV_OPT_TYPE_DOUBLE, {.dbl=5}, 0.1, 80, AF }, + { "release", "set release", OFFSET(release), AV_OPT_TYPE_DOUBLE, {.dbl=50}, 1, 8000, AF }, + { "asc", "enable asc", OFFSET(auto_release), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, + { "asc_level", "set asc level", OFFSET(asc_coeff), AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 1, AF }, + { "level", "auto level", OFFSET(auto_level), AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, AF }, + { "flush_buffer","flush the samples in the lookahead buffer", OFFSET(flush_buffer), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, { NULL } }; @@ -275,6 +279,39 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) return ff_filter_frame(outlink, out); } + +static int request_frame(AVFilterLink* outlink) { + AVFilterContext* ctx = outlink->src; + AudioLimiterContext* s = (AudioLimiterContext*)ctx->priv; + int ret; + AVFilterLink *inlink; + AVFrame *silence_frame; + + ret = ff_request_frame(ctx->inputs[0]); + + if (!s->flush_buffer) { + return 0; + } + + if (ret != AVERROR_EOF || s->total_samples_to_flush) { + // Not necessarily an error, just not EOF.s + return ret; + } + + // We reach here when input filters have finished producing data (i.e. EOF), + // but because of the attac param, s->buffer still has meaningful + // audio content that needs flushing. The amount of remaining audio to flush + // is the same as the amount of lookahead that was trimmed from the beginning. + inlink = ctx->inputs[0]; + // Pushes silence frame to flush valid audio in the s->buffer + silence_frame = ff_get_audio_buffer(inlink, s->total_samples_to_flush); + ret = filter_frame(inlink, silence_frame); + if (ret < 0) { + return ret; + } + return AVERROR_EOF; +} + static int config_input(AVFilterLink *inlink) { AVFilterContext *ctx = inlink->dst; @@ -292,7 +329,8 @@ static int config_input(AVFilterLink *inlink) return AVERROR(ENOMEM); memset(s->nextpos, -1, obuffer_size * sizeof(*s->nextpos)); - s->buffer_size = inlink->sample_rate * s->attack * inlink->ch_layout.nb_channels; + s->total_samples_to_flush = inlink->sample_rate * s->attack; + s->buffer_size = s->total_samples_to_flush * inlink->ch_layout.nb_channels; s->buffer_size -= s->buffer_size % inlink->ch_layout.nb_channels; if (s->buffer_size <= 0) { @@ -325,6 +363,7 @@ static const AVFilterPad alimiter_outputs[] = { { .name = "default", .type = AVMEDIA_TYPE_AUDIO, + .request_frame = request_frame, }, };