From patchwork Mon May 1 08:30:20 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 3530 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.3.129 with SMTP id 123csp1314660vsd; Mon, 1 May 2017 01:30:52 -0700 (PDT) X-Received: by 10.28.146.11 with SMTP id u11mr120881wmd.53.1493627451930; Mon, 01 May 2017 01:30:51 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g11si16151466wra.216.2017.05.01.01.30.51; Mon, 01 May 2017 01:30:51 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A5EDB6883A9; Mon, 1 May 2017 11:30:45 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr0-f174.google.com (mail-wr0-f174.google.com [209.85.128.174]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id F12B0681866 for ; Mon, 1 May 2017 11:30:38 +0300 (EEST) Received: by mail-wr0-f174.google.com with SMTP id l50so60903149wrc.3 for ; Mon, 01 May 2017 01:30:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=HCumcETFjnThWgOFHka69bm7gP5p6xSUnO27OWEdPmM=; b=sL83nICWcRhU9Up26jJxUqGpsrIZphLCxmDOZZTCRYMzZnShyRfsyQkERfvUk6bTOg +6QkyS4HU7IGrWok1cnSURhviEeHUk2qEXKGQ551irz3421a+eaIyoc9RVqLQ5dClYKb CQ2rJRez/VXs6ZxDlPBF8Di8gB565QFDeqCrJClWnlva3pyyVsQ2GCnB/LcVHxqZ2V2E Cvq+wrNKP/kVgbsLnWWOfy5/5e8yV/hvTY+kxOWoWkPNDBpk8bd67KMDIKa7QjiPylxF cxY+O1M/UAccreccMPx2mxJnYqIXiFqwx90uTfi4xnsZpFdzTpGnx3nKPAKfUYakbz6L /XQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=HCumcETFjnThWgOFHka69bm7gP5p6xSUnO27OWEdPmM=; b=RCxZKniLw5Nz9CA9NBECGdOxnDGy+izYizkHLUAu7z0b6h4mJEgfK6QWjDaaaWVFpi jXtwxw94GYdbEma75EKUyvISMAJUI6fpBN4kwHVu/WLMXQgswry5cTlhsQNZIShGCFgk xKT9litax59lP5BehHMeh8BNm97OwSc8xlpTSzTQ00fCKA4K9DyHJrFiEnQSfprEws5E J62xaJivv4aEtw1pSkFkFNOyYSPDAM3A/DNX+CAgC+554/DCH56NnmUN4VtZ4YWV9LS/ hdQUnvF/Erx5lBogpoLLkToXxcMgBg7GvSLVnffScZ1F00rPvlXEFNAr1oytaozU/w6i Amzw== X-Gm-Message-State: AN3rC/4N9I+q734nFhiNfTbSqopi5DJYYI5S35vPyJ+ilpnmrm0sLzBu jTr53X/GthEwxdX7 X-Received: by 10.223.183.12 with SMTP id l12mr14749840wre.191.1493627441546; Mon, 01 May 2017 01:30:41 -0700 (PDT) Received: from localhost.localdomain ([94.250.174.60]) by smtp.gmail.com with ESMTPSA id l29sm15567051wmi.8.2017.05.01.01.30.40 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 01 May 2017 01:30:40 -0700 (PDT) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Mon, 1 May 2017 10:30:20 +0200 Message-Id: <20170501083020.5250-1-onemda@gmail.com> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20170430220217.19998-1-onemda@gmail.com> References: <20170430220217.19998-1-onemda@gmail.com> Subject: [FFmpeg-devel] [PATCH] avfilter: add arbitrary audio FIR filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Paul B Mahol --- configure | 2 + doc/filters.texi | 10 ++ libavfilter/Makefile | 1 + libavfilter/af_afirfilter.c | 409 ++++++++++++++++++++++++++++++++++++++++++++ libavfilter/allfilters.c | 1 + 5 files changed, 423 insertions(+) create mode 100644 libavfilter/af_afirfilter.c diff --git a/configure b/configure index b3cb5b0..7fc7af4 100755 --- a/configure +++ b/configure @@ -3078,6 +3078,8 @@ unix_protocol_select="network" # filters afftfilt_filter_deps="avcodec" afftfilt_filter_select="fft" +afirfilter_filter_deps="avcodec" +afirfilter_filter_select="fft" amovie_filter_deps="avcodec avformat" aresample_filter_deps="swresample" ass_filter_deps="libass" diff --git a/doc/filters.texi b/doc/filters.texi index 119e747..ea343d1 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -878,6 +878,16 @@ afftfilt="1-clip((b/nb)*b,0,1)" @end example @end itemize +@section afirfilter + +Apply an Arbitary Frequency Impulse Response filter. + +This filter uses second stream as FIR coefficients. +If second stream holds single channel, it will be used +for all input channels in first stream, otherwise +number of channels in second stream must be same as +number of channels in first stream. + @anchor{aformat} @section aformat diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 66c36e4..1a0f24b 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -38,6 +38,7 @@ OBJS-$(CONFIG_AEMPHASIS_FILTER) += af_aemphasis.o OBJS-$(CONFIG_AEVAL_FILTER) += aeval.o OBJS-$(CONFIG_AFADE_FILTER) += af_afade.o OBJS-$(CONFIG_AFFTFILT_FILTER) += af_afftfilt.o window_func.o +OBJS-$(CONFIG_AFIRFILTER_FILTER) += af_afirfilter.o OBJS-$(CONFIG_AFORMAT_FILTER) += af_aformat.o OBJS-$(CONFIG_AGATE_FILTER) += af_agate.o OBJS-$(CONFIG_AINTERLEAVE_FILTER) += f_interleave.o diff --git a/libavfilter/af_afirfilter.c b/libavfilter/af_afirfilter.c new file mode 100644 index 0000000..ef2488a --- /dev/null +++ b/libavfilter/af_afirfilter.c @@ -0,0 +1,409 @@ +/* + * Copyright (c) 2017 Paul B Mahol + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * An arbitrary audio FIR filter + */ + +#include "libavutil/audio_fifo.h" +#include "libavutil/avassert.h" +#include "libavutil/channel_layout.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavcodec/avfft.h" + +#include "audio.h" +#include "avfilter.h" +#include "formats.h" +#include "internal.h" + +typedef struct FIRContext { + const AVClass *class; + + int n; + int eof_coeffs; + int have_coeffs; + int nb_taps; + int fft_length; + int nb_channels; + int one2many; + + FFTContext *fft, *ifft; + FFTComplex **fft_data; + FFTComplex **fft_coef; + + AVAudioFifo *fifo[2]; + AVFrame *in[2]; + AVFrame *buffer; + int64_t pts; + int hop_size; + int start, end; +} FIRContext; + +static int fir_filter(FIRContext *s, AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + int start = s->start, end = s->end; + int ret = 0, n, ch, j, k; + int nb_samples; + AVFrame *out; + + nb_samples = FFMIN(s->fft_length, av_audio_fifo_size(s->fifo[0])); + + s->in[0] = ff_get_audio_buffer(ctx->inputs[0], nb_samples); + if (!s->in[0]) + return AVERROR(ENOMEM); + + av_audio_fifo_peek(s->fifo[0], (void **)s->in[0]->extended_data, nb_samples); + + for (ch = 0; ch < outlink->channels; ch++) { + const float *src = (float *)s->in[0]->extended_data[ch]; + float *buf = (float *)s->buffer->extended_data[ch]; + FFTComplex *fft_data = s->fft_data[ch]; + FFTComplex *fft_coef = s->fft_coef[ch]; + + memset(fft_data, 0, sizeof(*fft_data) * s->fft_length); + for (n = 0; n < nb_samples; n++) { + fft_data[n].re = src[n]; + fft_data[n].im = 0; + } + + av_fft_permute(s->fft, fft_data); + av_fft_calc(s->fft, fft_data); + + fft_data[0].re *= fft_coef[0].re; + fft_data[0].im *= fft_coef[0].im; + for (n = 1; n < s->fft_length; n++) { + const float re = fft_data[n].re; + const float im = fft_data[n].im; + + fft_data[n].re = re * fft_coef[n].re - im * fft_coef[n].im; + fft_data[n].im = re * fft_coef[n].im + im * fft_coef[n].re; + } + + av_fft_permute(s->ifft, fft_data); + av_fft_calc(s->ifft, fft_data); + + start = s->start; + end = s->end; + k = end; + + for (n = 0, j = start; j < k && n < s->fft_length; n++, j++) { + buf[j] = fft_data[n].re; + } + + for (; n < s->fft_length; n++, j++) { + buf[j] = fft_data[n].re; + } + + start += s->hop_size; + end = j; + } + + s->start = start; + s->end = end; + + if (start >= nb_samples) { + float *dst, *buf; + + start -= nb_samples; + end -= nb_samples; + + s->start = start; + s->end = end; + + out = ff_get_audio_buffer(outlink, nb_samples); + if (!out) + return AVERROR(ENOMEM); + + out->pts = s->pts; + s->pts += nb_samples; + + for (ch = 0; ch < s->nb_channels; ch++) { + dst = (float *)out->extended_data[ch]; + buf = (float *)s->buffer->extended_data[ch]; + + for (n = 0; n < nb_samples; n++) + dst[n] = buf[n]; + memmove(buf, buf + nb_samples, nb_samples * 4); + } + + ret = ff_filter_frame(outlink, out); + } + + av_audio_fifo_drain(s->fifo[0], FFMIN(nb_samples, s->hop_size)); + av_frame_free(&s->in[0]); + + return ret; +} + +static int convert_coeffs(AVFilterContext *ctx) +{ + FIRContext *s = ctx->priv; + int ch, n; + + s->nb_taps = av_audio_fifo_size(s->fifo[1]); + if (s->nb_taps > 32768) { + av_log(ctx, AV_LOG_ERROR, "Too big number of taps: %d > 32768.\n", s->nb_taps); + return AVERROR(EINVAL); + } + + for (n = 1; (1 << n) < s->nb_taps; n++); + s->n = n + 2; + s->fft_length = 1 << s->n; + + for (ch = 0; ch < ctx->inputs[0]->channels; ch++) { + s->fft_data[ch] = av_calloc(s->fft_length, sizeof(**s->fft_data)); + if (!s->fft_data[ch]) + return AVERROR(ENOMEM); + } + + for (ch = 0; ch < ctx->inputs[1]->channels; ch++) { + s->fft_coef[ch] = av_calloc(s->fft_length, sizeof(**s->fft_coef)); + if (!s->fft_coef[ch]) + return AVERROR(ENOMEM); + } + + s->hop_size = s->fft_length - s->nb_taps + 1; + if (s->hop_size <= 0) { + av_log(ctx, AV_LOG_ERROR, "Too big number of taps.\n"); + return AVERROR(EINVAL); + } + + s->buffer = ff_get_audio_buffer(ctx->inputs[0], s->fft_length * 2); + if (!s->buffer) + return AVERROR(ENOMEM); + + s->fft = av_fft_init(s->n, 0); + s->ifft = av_fft_init(s->n, 1); + if (!s->fft || !s->ifft) + return AVERROR(ENOMEM); + + s->in[1] = ff_get_audio_buffer(ctx->inputs[1], s->nb_taps); + if (!s->in[1]) + return AVERROR(ENOMEM); + + av_audio_fifo_read(s->fifo[1], (void **)s->in[1]->extended_data, s->nb_taps); + for (ch = 0; ch < ctx->inputs[1]->channels; ch++) { + FFTComplex *fft_coef = s->fft_coef[ch]; + const float *re = (const float *)s->in[1]->extended_data[!s->one2many * ch]; + const float scale = 1.f / s->fft_length; + const int offset = (s->fft_length - s->nb_taps); + + memset(fft_coef, 0, sizeof(*fft_coef) * s->fft_length); + for (n = 0; n < s->nb_taps; n++) { + fft_coef[n + offset].re = re[n] * scale; + } + av_fft_permute(s->fft, fft_coef); + av_fft_calc(s->fft, fft_coef); + } + + av_frame_free(&s->in[1]); + s->have_coeffs = 1; + + return 0; +} + +static int read_coeffs(AVFilterLink *link, AVFrame *frame) +{ + AVFilterContext *ctx = link->dst; + FIRContext *s = ctx->priv; + + av_audio_fifo_write(s->fifo[1], (void **)frame->extended_data, + frame->nb_samples); + av_frame_free(&frame); + + return 0; +} + +static int filter_frame(AVFilterLink *link, AVFrame *frame) +{ + AVFilterContext *ctx = link->dst; + FIRContext *s = ctx->priv; + AVFilterLink *outlink = ctx->outputs[0]; + int ret = 0; + + av_audio_fifo_write(s->fifo[0], (void **)frame->extended_data, + frame->nb_samples); + av_frame_free(&frame); + + if (!s->have_coeffs && s->eof_coeffs) { + ret = convert_coeffs(ctx); + if (ret < 0) + return ret; + } + + if (s->have_coeffs) { + while (av_audio_fifo_size(s->fifo[0]) >= s->fft_length) { + ret = fir_filter(s, outlink); + if (ret < 0) + break; + } + } + return ret; +} + +static int request_frame(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + FIRContext *s = ctx->priv; + int ret; + + if (!s->eof_coeffs) { + ret = ff_request_frame(ctx->inputs[1]); + if (ret == AVERROR_EOF) { + s->eof_coeffs = 1; + ret = 0; + } + return ret; + } + ret = ff_request_frame(ctx->inputs[0]); + if (ret == AVERROR_EOF && s->have_coeffs) { + while (av_audio_fifo_size(s->fifo[0]) > 0) { + ret = fir_filter(s, outlink); + if (ret < 0) + return ret; + } + ret = AVERROR_EOF; + } + return ret; +} + +static int query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats; + AVFilterChannelLayouts *layouts = NULL; + static const enum AVSampleFormat sample_fmts[] = { + AV_SAMPLE_FMT_FLTP, + AV_SAMPLE_FMT_NONE + }; + int ret, i; + + layouts = ff_all_channel_counts(); + if ((ret = ff_channel_layouts_ref(layouts, &ctx->outputs[0]->in_channel_layouts)) < 0) + return ret; + + for (i = 0; i < 2; i++) { + layouts = ff_all_channel_counts(); + if ((ret = ff_channel_layouts_ref(layouts, &ctx->inputs[i]->out_channel_layouts)) < 0) + return ret; + } + + formats = ff_make_format_list(sample_fmts); + if ((ret = ff_set_common_formats(ctx, formats)) < 0) + return ret; + + formats = ff_all_samplerates(); + return ff_set_common_samplerates(ctx, formats); +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + FIRContext *s = ctx->priv; + + if (ctx->inputs[0]->channels != ctx->inputs[1]->channels && + ctx->inputs[1]->channels != 1) { + av_log(ctx, AV_LOG_ERROR, + "Second input must have same number of channels as first input or " + "exactly 1 channel.\n"); + return AVERROR(EINVAL); + } + + s->one2many = ctx->inputs[1]->channels == 1; + outlink->sample_rate = ctx->inputs[0]->sample_rate; + outlink->time_base = ctx->inputs[0]->time_base; + outlink->channel_layout = ctx->inputs[0]->channel_layout; + outlink->channels = ctx->inputs[0]->channels; + + s->fifo[0] = av_audio_fifo_alloc(ctx->inputs[0]->format, ctx->inputs[0]->channels, 1024); + s->fifo[1] = av_audio_fifo_alloc(ctx->inputs[1]->format, ctx->inputs[1]->channels, 1024); + if (!s->fifo[0] || !s->fifo[1]) + return AVERROR(ENOMEM); + + s->fft_data = av_calloc(outlink->channels, sizeof(*s->fft_data)); + s->fft_coef = av_calloc(ctx->inputs[1]->channels, sizeof(*s->fft_coef)); + if (!s->fft_data || !s->fft_coef) + return AVERROR(ENOMEM); + s->nb_channels = outlink->channels; + + return 0; +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + FIRContext *s = ctx->priv; + int ch; + + for (ch = 0; ch < s->nb_channels; ch++) { + if (s->fft_data) + av_freep(&s->fft_data[ch]); + } + av_freep(&s->fft_data); + + for (ch = 0; ch < s->nb_channels; ch++) { + if (s->fft_coef) + av_freep(&s->fft_coef[ch]); + } + av_freep(&s->fft_coef); + + av_fft_end(s->fft); + av_fft_end(s->ifft); + + av_frame_free(&s->in[0]); + av_frame_free(&s->in[1]); + + av_audio_fifo_free(s->fifo[0]); + av_audio_fifo_free(s->fifo[1]); +} + +static const AVFilterPad afirfilter_inputs[] = { + { + .name = "main", + .type = AVMEDIA_TYPE_AUDIO, + .filter_frame = filter_frame, + },{ + .name = "coefficients", + .type = AVMEDIA_TYPE_AUDIO, + .filter_frame = read_coeffs, + }, + { NULL } +}; + +static const AVFilterPad afirfilter_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + .config_props = config_output, + .request_frame = request_frame, + }, + { NULL } +}; + +AVFilter ff_af_afirfilter = { + .name = "afirfilter", + .description = NULL_IF_CONFIG_SMALL("Apply Finite Impulse Response filter with supplied coefficients in 2nd stream."), + .priv_size = sizeof(FIRContext), + .query_formats = query_formats, + .uninit = uninit, + .inputs = afirfilter_inputs, + .outputs = afirfilter_outputs, +}; diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 8fb87eb..8bfe1ae 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -50,6 +50,7 @@ static void register_all(void) REGISTER_FILTER(AEVAL, aeval, af); REGISTER_FILTER(AFADE, afade, af); REGISTER_FILTER(AFFTFILT, afftfilt, af); + REGISTER_FILTER(AFIRFILTER, afirfilter, af); REGISTER_FILTER(AFORMAT, aformat, af); REGISTER_FILTER(AGATE, agate, af); REGISTER_FILTER(AINTERLEAVE, ainterleave, af);