From patchwork Tue Mar 20 09:20:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 8056 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.1.70 with SMTP id c67csp3649090jad; Tue, 20 Mar 2018 02:21:44 -0700 (PDT) X-Google-Smtp-Source: AG47ELsx8fg31T8iYd+yz52Kfm/iagpiBKftJccLvehwyoDDFQcvmklR3TTOvE/QWdoV5ZquepIY X-Received: by 10.223.225.142 with SMTP id k14mr11935769wri.38.1521537704217; Tue, 20 Mar 2018 02:21:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1521537704; cv=none; d=google.com; s=arc-20160816; b=gK5Gcccrnr11b1AACd8wgYScLuLVs02y2Ae5L+zDVY+bvzE1VBvUgXjMgfCka0Tla7 jFu6dLbtGlhvJSTM8f8GTyRn4S9ooUb5R7o2CGFQiv8prTsIImoXvKPbXMmTMQ9VAt5U tfQQGXrSk8P9KCYyOJLzDfBMt0atFTmfAf1gmhJ0ojFgvRZtr75o59GfTnQKTRQzk317 9CWYBgdtkOIDzQPyS7+p/irBlp+a++w2vRZqJfn4Y5Zr4DHiGp1ZC+0KsnHGIiOwz/oX 2oNB0QO7tYZvoLPo7wpavCAuAJu/CUTaptnRw4uMp1G/0Sy3a0QSYH0Sui4ilW8pCsSQ kE7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=AN47R7hbGazUcaSwUHAiWHGLXXjV1nM07BQaow3ebJ8=; b=ppv/abiAus1jPOpHas07YiiWhXZ0nOaQAvwdOdx+1sqLR3AnJWhhGBjtZynaculHfk IG3P0kSIN+5u8zP5OO28mF5+FncsTFPbmsC8uBvCd2YIp74WiEHW0H9z17ZT9i4W/ONn 29cRhNLttzwz0qO+jJLtyKDoPPOZFeq8TvWHu3I7m7LhpShiJlG3vQWOp4o7CqwL5hAk vjVtwkL36VY+G01cvk3fH1i2FkwbEknIfC6qWYC+hNV4gzj/RMXttV7bh8g9554c7OHQ p72dWCVh4+W8w5bpY8UzJrxDj5DkOevLfnTW33AfkPgn1Z9In28Xr5G9LCMFtpoSU5Wd DgrA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=afDrze4w; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id k9si900542wmk.228.2018.03.20.02.21.41; Tue, 20 Mar 2018 02:21:44 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=afDrze4w; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B1D4E68A143; Tue, 20 Mar 2018 11:21:25 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f52.google.com (mail-wm0-f52.google.com [74.125.82.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8B3FA689F72 for ; Tue, 20 Mar 2018 11:21:19 +0200 (EET) Received: by mail-wm0-f52.google.com with SMTP id l16so2005713wmh.3 for ; Tue, 20 Mar 2018 02:21:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=IjvMTsKlgKPC7D9jYOgSJrFbKlgbw1Hg2UO1UiBxSv0=; b=afDrze4w8lCoy1K4BgUEUTOqYjaruIk0yrD1K/6mXvmnRxOcAsjRO6J+A8ihcK968o PoMGVqig2Or3EAwWa933mPn17Zk5xGOQ4be6FMA74l3iwqGwoGMVaEvVa0fVns+mNxjS 46uREP5zo9BrKSJyryyU2olASTP8Wwd9SjE/lSP/qhf2iRUGW+KBcA65IJuaF86PmSEe x5E3OLcSVEXmSgTVKPNdfrwM1A6zmw/m+ZJjkgHwKuLs/cYgRQ7WTidue58WYOBrb8QW wmJ9nksB4m7N8OM9vEJVnQIqyozSH6PSR1lhCVSnJqOwv9SwPFBjLJNRIj9exkvVokev BZww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=IjvMTsKlgKPC7D9jYOgSJrFbKlgbw1Hg2UO1UiBxSv0=; b=G1BbR4+nxSGeIHM87YepswPogK5QIeWw0iIZFj/OYonKkeAXcNF/yBUVhC8eIg9JSj uCUJhgNqw9GNhpyjU0JQzVr5xcmVBM1Sm+QOWF3JjmDjWa5i7KIY5jyeS8UHH4z7xP+0 XcDb24p2XJ6eaCnckfbMn4rv+crtFx5QNRXhxCXo6KCA4760a1x5sTgm/Oq6niWNBNid MmUMFAuNpfJJ+6WlnW9b2JV5X/l4X6DYJtSxIr7wm8GS/dptTIWiD/1ixpkBOrnE/0XT B/85TT2ME5l9rlg5x3LIAnVek3FefFEH9iuWMtkri7Ksn5IXC7gb0iAKey+m95NduhSn I3vw== X-Gm-Message-State: AElRT7HXceDJ4AI2szTGz/YOMrC0tGDpSrQOFDplUI8Nx6yShD4mJP9S t4VpLJqTe42B0BmJmgsUBici2w== X-Received: by 10.80.173.50 with SMTP id y47mr16340918edc.247.1521537692637; Tue, 20 Mar 2018 02:21:32 -0700 (PDT) Received: from localhost.localdomain ([94.250.174.60]) by smtp.gmail.com with ESMTPSA id 93sm1503273edi.19.2018.03.20.02.21.31 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Mar 2018 02:21:32 -0700 (PDT) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Tue, 20 Mar 2018 10:20:47 +0100 Message-Id: <20180320092047.18323-1-onemda@gmail.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180315175400.7993-1-onemda@gmail.com> References: <20180315175400.7993-1-onemda@gmail.com> Subject: [FFmpeg-devel] [PATCH] avfilter: add hrtfm filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Paul B Mahol --- doc/filters.texi | 66 ++++++ libavfilter/Makefile | 1 + libavfilter/af_hrtfm.c | 557 +++++++++++++++++++++++++++++++++++++++++++++++ libavfilter/allfilters.c | 1 + 4 files changed, 625 insertions(+) create mode 100644 libavfilter/af_hrtfm.c diff --git a/doc/filters.texi b/doc/filters.texi index bd43a7ac6e..95950f7666 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -3218,6 +3218,72 @@ Change highpass width. Syntax for the command is : "@var{width}" @end table +@section hrtfm + +Apply simple Head Related Transfer Function Model to audio stream. + +hrtfm filter creates virtual loudspeakers around the user for binaural +listening via headphones (audio formats up to 9 channels supported). + +This is very simple implementation which does not use any HRIRs. + +It accepts the following parameters: + +@table @option +@item hradius +Set head radius of listener. In centimeters. Default value is @code{8.91}. + +@item sspeed +Set sound speed in meters per second. Default value is @code{334}. +Allowed range is from @code{300} to @code{400}. + +@item amin +Set minimum alfa. Default value is @code{0.05}. +Allowed range is from @code{0.001} to @code{1}. + +@item gain +Set output gain in dB. Default value is @code{0}. +Allowed range is from @code{-40} to @code{40}. + +@item rotation +Set rotation of virtual loudspeakers in degrees. Default is @code{0}. +Allowed range is from @code{-180} to @code{180}. + +@item elevation +Set elevation of virtual speakers in degrees. Default is @code{0}. +Allowed range is from @code{-90} to @code{90}. + +@item speakers +Set custom positions of virtual loudspeakers. Syntax for this option is: + [| |...]. +Each virtual loudspeaker is described with short channel name following with +azimuth and elevation in degrees. +Each virtual loudspeaker description is separated by '|'. +For example to override front left and front right channel positions use: +'speakers=FL 45 15|FR 345 15'. +Descriptions with unrecognised channel names are ignored. + +@item lfegain +Set LFE gain in dB. Default value is @code{0}. +Allowed range is from @code{-40} to @code{40}. +@end table + +@subsection Examples + +@itemize +@item +Apply filter with custom head radius, speed of sound and minimum alpha: +@example +hrtfm=hradius=0.09:sspeed=334:amin=0.001 +@end example + +@item +Similar as above but also with custom speaker positions: +@example +"hrtfm=0.1:334:0.001:speakers=FL -15|FR 15|BL -160|BR 160|SL -80|SR 80" +@end example +@end itemize + @section join Join multiple input streams into one multi-channel stream. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index fc16512e2c..65783a8443 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -99,6 +99,7 @@ OBJS-$(CONFIG_HAAS_FILTER) += af_haas.o OBJS-$(CONFIG_HDCD_FILTER) += af_hdcd.o OBJS-$(CONFIG_HEADPHONE_FILTER) += af_headphone.o OBJS-$(CONFIG_HIGHPASS_FILTER) += af_biquads.o +OBJS-$(CONFIG_HRTFM_FILTER) += af_hrtfm.o OBJS-$(CONFIG_JOIN_FILTER) += af_join.o OBJS-$(CONFIG_LADSPA_FILTER) += af_ladspa.o OBJS-$(CONFIG_LOUDNORM_FILTER) += af_loudnorm.o ebur128.o diff --git a/libavfilter/af_hrtfm.c b/libavfilter/af_hrtfm.c new file mode 100644 index 0000000000..9f35b1cb9b --- /dev/null +++ b/libavfilter/af_hrtfm.c @@ -0,0 +1,557 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/avstring.h" +#include "libavutil/ffmath.h" +#include "libavutil/opt.h" + +#include "audio.h" +#include "avfilter.h" +#include "internal.h" + +#define CHPARAMS(name, type) \ +typedef struct ChParams_## name { \ + type azim; \ + type elev; \ + \ + type Al[2], Bl[2], al[2], bl[2]; \ + type Ar[2], Br[2], ar[2], br[2]; \ + \ + type gain[2]; \ + \ + type *delayed_samples[2]; \ + int delayed_index[2]; \ + \ + type cache1_in[2]; \ + type cache1_out[2]; \ + \ + type cache2_in[2]; \ + type cache2_out[2]; \ + \ + int nb_samples[2]; \ + int lfe; \ +} ChParams_## name; + +CHPARAMS(flt, float) +CHPARAMS(dbl, double) + +typedef struct VirtualSpeaker { + uint8_t set; + float azim; + float elev; +} VirtualSpeaker; + +typedef struct HRTFMContext { + const AVClass *class; + + double sspeed; + double hradius; + double alpha_min; + double gain; /* filter gain (in dB) */ + double lfe_gain; + double rotation; /* rotation of virtual loudspeakers (in degrees) */ + double elevation; /* elevation of virtual loudspeakers (in deg.) */ + char *speakers_pos; /* custom positions of the virtual loudspeakers */ + + ChParams_flt *params_flt; + ChParams_dbl *params_dbl; + + int n_conv; + double gain_lfe; + double scale; + + VirtualSpeaker vspkrpos[64]; +} HRTFMContext; + +static int query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats = NULL; + AVFilterChannelLayouts *layouts = NULL; + static const enum AVSampleFormat sample_fmts[] = { + AV_SAMPLE_FMT_FLT, + AV_SAMPLE_FMT_DBL, + AV_SAMPLE_FMT_NONE + }; + int ret; + + formats = ff_make_format_list(sample_fmts); + if (!formats) + return AVERROR(ENOMEM); + ret = ff_set_common_formats(ctx, formats); + if (ret) + return ret; + + layouts = ff_all_channel_layouts(); + if (!layouts) + return AVERROR(ENOMEM); + + ret = ff_channel_layouts_ref(layouts, &ctx->inputs[0]->out_channel_layouts); + if (ret) + return ret; + + layouts = NULL; + ret = ff_add_channel_layout(&layouts, AV_CH_LAYOUT_STEREO); + if (ret) + return ret; + + ret = ff_channel_layouts_ref(layouts, &ctx->outputs[0]->in_channel_layouts); + if (ret) + return ret; + + formats = ff_all_samplerates(); + if (!formats) + return AVERROR(ENOMEM); + return ff_set_common_samplerates(ctx, formats); +} + +static int parse_channel_name(char **arg, int *rchannel, char *buf) +{ + int len, i, channel_id = 0; + int64_t layout, layout0; + + /* try to parse a channel name, e.g. "FL" */ + if (sscanf(*arg, "%7[A-Z]%n", buf, &len)) { + layout0 = layout = av_get_channel_layout(buf); + /* channel_id <- first set bit in layout */ + for (i = 32; i > 0; i >>= 1) { + if (layout >= 1LL << i) { + channel_id += i; + layout >>= i; + } + } + /* reject layouts that are not a single channel */ + if (channel_id >= 64 || layout0 != 1LL << channel_id) + return AVERROR(EINVAL); + *rchannel = channel_id; + *arg += len; + return 0; + } + return AVERROR(EINVAL); +} + +static void parse_speaker_pos(AVFilterContext *ctx, int64_t in_channel_layout) +{ + HRTFMContext *s = ctx->priv; + char *arg, *tokenizer, *p, *args = av_strdup(s->speakers_pos); + + if (!args) + return; + p = args; + + while ((arg = av_strtok(p, "|", &tokenizer))) { + char buf[8]; + float azim, elev; + int out_ch_id; + + p = NULL; + if (parse_channel_name(&arg, &out_ch_id, buf)) { + av_log(ctx, AV_LOG_WARNING, "Failed to parse \'%s\' as channel name.\n", buf); + continue; + } + if (sscanf(arg, "%f %f", &azim, &elev) == 2) { + s->vspkrpos[out_ch_id].set = 1; + s->vspkrpos[out_ch_id].azim = azim; + s->vspkrpos[out_ch_id].elev = elev; + } else if (sscanf(arg, "%f", &azim) == 1) { + s->vspkrpos[out_ch_id].set = 1; + s->vspkrpos[out_ch_id].azim = azim; + s->vspkrpos[out_ch_id].elev = 0; + } + } + + av_free(args); +} + +static int get_speaker_pos(AVFilterContext *ctx) +{ + HRTFMContext *s = ctx->priv; + uint64_t channels_layout = ctx->inputs[0]->channel_layout; + int m, ch, n_conv = ctx->inputs[0]->channels; /* get no. input channels */ + + if (n_conv > 16) + return AVERROR(EINVAL); + + if (s->speakers_pos) + parse_speaker_pos(ctx, channels_layout); + + /* set speaker positions according to input channel configuration: */ + for (m = 0, ch = 0; ch < n_conv && m < 64; m++) { + ChParams_flt *pf = &s->params_flt[ch]; + ChParams_dbl *pd = &s->params_dbl[ch]; + double azim = 0, elev = 0; + uint64_t mask = channels_layout & (1ULL << m); + + switch (mask) { + case AV_CH_FRONT_LEFT: azim = -30; break; + case AV_CH_FRONT_RIGHT: azim = 30; break; + case AV_CH_FRONT_CENTER: azim = 0; break; + case AV_CH_LOW_FREQUENCY: + case AV_CH_LOW_FREQUENCY_2: pd->lfe = 1; + pf->lfe = 1; break; + case AV_CH_BACK_LEFT: azim =-140; break; + case AV_CH_BACK_RIGHT: azim = 140; break; + case AV_CH_BACK_CENTER: azim = 180; break; + case AV_CH_SIDE_LEFT: azim = -90; break; + case AV_CH_SIDE_RIGHT: azim = 90; break; + case AV_CH_FRONT_LEFT_OF_CENTER: azim = -15; break; + case AV_CH_FRONT_RIGHT_OF_CENTER: azim = 15; break; + case AV_CH_TOP_CENTER: azim = 0; + elev = 90; break; + case AV_CH_TOP_FRONT_LEFT: azim = -30; + elev = 45; break; + case AV_CH_TOP_FRONT_CENTER: azim = 0; + elev = 45; break; + case AV_CH_TOP_FRONT_RIGHT: azim = 30; + elev = 45; break; + case AV_CH_TOP_BACK_LEFT: azim =-140; + elev = 45; break; + case AV_CH_TOP_BACK_RIGHT: azim = 140; + elev = 45; break; + case AV_CH_TOP_BACK_CENTER: azim = 180; + elev = 45; break; + case AV_CH_WIDE_LEFT: azim = -90; break; + case AV_CH_WIDE_RIGHT: azim = 90; break; + case AV_CH_SURROUND_DIRECT_LEFT: azim = -90; break; + case AV_CH_SURROUND_DIRECT_RIGHT: azim = 90; break; + case AV_CH_STEREO_LEFT: azim = -90; break; + case AV_CH_STEREO_RIGHT: azim = 90; break; + case 0: break; + default: + return AVERROR(EINVAL); + } + + if (s->vspkrpos[m].set) { + azim = s->vspkrpos[m].azim; + elev = s->vspkrpos[m].elev; + } + + azim += s->rotation; + elev += s->elevation; + + pf->azim = pd->azim = fabs(azim) > 180 ? fmod(azim + 180., 360.) - 180. : azim; + pf->elev = pd->elev = fabs(elev) > 90 ? fmod(elev + 90., 180.) - 90. : elev; + + if (mask) + ch++; + } + + return 0; +} + +static void hsfilter(double angle, double Fs, double hradius, double sspeed, double alpha_min, + double *B0, double *B1, double *A0, double *A1, + double *b0, double *b1, double *a0, double *a1) +{ + double theta = angle + 90.; + double theta0 = 180.; + double w0 = sspeed / (hradius / 100.); + double alpha = 1. + alpha_min / 2. + (1. - alpha_min / 2.) * cos(theta / theta0 * M_PI); + double beta = w0 / Fs; + double gdelay, ac; + + *B0 = ( alpha + beta) / (1. + beta); + *B1 = (-alpha + beta) / (1. + beta); + + *A0 = 1.; + *A1 = -(1. - beta) / (1. + beta); + + if (fabs(theta) < 90.) + gdelay = -beta * (cos(theta * M_PI / 180.) - 1.); + else + gdelay = beta * ((fabs(theta) - 90.) * M_PI / 180. + 1.); + + ac = (1. - gdelay) / (1. + gdelay); + + *b0 = *a1 = ac; + *b1 = *a0 = 1.; +} + +static void shoulder(double angle, double elevation, int Fs, double dBgain, + double *gain, int *nb_samples) +{ + double theta = angle < -179. ? 180. : angle; + double phi = elevation; + double delay = (1.2 * (180 - theta) / 180) * pow(1. - 0.00004 * ((phi - 80) * (180 / (180 + theta))), 2); + + *gain = ff_exp10(dBgain / 20); + *nb_samples = round(delay / 1000 * Fs); +} + +static int config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + HRTFMContext *s = ctx->priv; + int ret, ch; + + s->n_conv = inlink->channels; + s->gain_lfe = exp((s->gain - 3 * s->n_conv + s->lfe_gain) / 20 * M_LN10); + s->scale = exp((s->gain - 3 * s->n_conv) / 20 * M_LN10); + + s->params_flt = av_calloc(s->n_conv, sizeof(*s->params_flt)); + s->params_dbl = av_calloc(s->n_conv, sizeof(*s->params_dbl)); + if (!s->params_flt || !s->params_dbl) + return AVERROR(ENOMEM); + + /* get speaker positions */ + if ((ret = get_speaker_pos(ctx)) < 0) { + av_log(ctx, AV_LOG_ERROR, "Couldn't get speaker positions. Input channel configuration not supported.\n"); + return ret; + } + + for (ch = 0; ch < inlink->channels; ch++) { + ChParams_dbl *p = &s->params_dbl[ch]; + ChParams_flt *pf = &s->params_flt[ch]; + + hsfilter( p->azim, inlink->sample_rate, s->hradius, s->sspeed, s->alpha_min, + &p->Bl[0], &p->Bl[1], &p->Al[0], &p->Al[1], + &p->bl[0], &p->bl[1], &p->al[0], &p->al[1]); + hsfilter(-p->azim, inlink->sample_rate, s->hradius, s->sspeed, s->alpha_min, + &p->Br[0], &p->Br[1], &p->Ar[0], &p->Ar[1], + &p->br[0], &p->br[1], &p->ar[0], &p->ar[1]); + + shoulder( p->azim, p->elev, inlink->sample_rate, 0, &p->gain[0], &p->nb_samples[0]); + shoulder(-p->azim, p->elev, inlink->sample_rate, 0, &p->gain[1], &p->nb_samples[1]); + + if (inlink->format == AV_SAMPLE_FMT_DBL) { + p->delayed_samples[0] = av_calloc(p->nb_samples[0], sizeof(double)); + p->delayed_samples[1] = av_calloc(p->nb_samples[1], sizeof(double)); + if (!p->delayed_samples[0] || !p->delayed_samples[1]) + return AVERROR(ENOMEM); + } else { + pf->Bl[0] = p->Bl[0]; pf->Bl[1] = p->Bl[1]; + pf->bl[0] = p->bl[0]; pf->bl[1] = p->bl[1]; + pf->Al[0] = p->Al[0]; pf->Al[1] = p->Al[1]; + pf->al[0] = p->al[0]; pf->al[1] = p->al[1]; + pf->Br[0] = p->Br[0]; pf->Br[1] = p->Br[1]; + pf->br[0] = p->br[0]; pf->br[1] = p->br[1]; + pf->Ar[0] = p->Ar[0]; pf->Ar[1] = p->Ar[1]; + pf->ar[0] = p->ar[0]; pf->ar[1] = p->ar[1]; + pf->gain[0] = p->gain[0]; pf->gain[1] = p->gain[1]; + pf->nb_samples[0] = p->nb_samples[0]; + pf->nb_samples[1] = p->nb_samples[1]; + + pf->delayed_samples[0] = av_calloc(p->nb_samples[0], sizeof(float)); + pf->delayed_samples[1] = av_calloc(p->nb_samples[1], sizeof(float)); + if (!pf->delayed_samples[0] || !pf->delayed_samples[1]) + return AVERROR(ENOMEM); + } + } + + return 0; +} + +#define HRTFM_BFILTER(name, type) \ +static av_always_inline type bfilter_## name (type input, type *icache, type *ocache, \ + type b0, type b1, type a1) \ +{ \ + type output = input * b0 + *icache * b1 - *ocache * a1; \ + \ + *icache = input; \ + *ocache = output; \ + \ + return output; \ +} + +HRTFM_BFILTER(flt, float) +HRTFM_BFILTER(dbl, double) + +#define HRTFM_DFILTER(name, type) \ +static av_always_inline type dfilter_## name (type input, type gain, int nb_samples, \ + type *delayed_samples, int *delayed_index) \ +{ \ + type output = gain * delayed_samples[*delayed_index]; \ + \ + delayed_samples[*delayed_index] = input; \ + (*delayed_index)++; \ + if (*delayed_index >= nb_samples) \ + *delayed_index = 0; \ + \ + return input + output; \ +} + +HRTFM_DFILTER(flt, float) +HRTFM_DFILTER(dbl, double) + +#define HRTFM_FILTER(name, type) \ +static void filter_samples_## name (AVFilterContext *ctx, AVFrame *in, AVFrame *out) \ +{ \ + HRTFMContext *s = ctx->priv; \ + unsigned n_clippings[2] = { 0 }; \ + const type scale = s->scale; \ + const type *src = (type *)in->data[0]; \ + type *dst; \ + int ch, n; \ + \ + dst = (type *)out->data[0]; \ + \ + for (n = 0; n < in->nb_samples; n++, dst+=2) { \ + dst[0] = dst[1] = 0; \ + \ + for (ch = 0; ch < in->channels; ch++, src++) { \ + ChParams_##name *p = &s->params_## name [ch]; \ + type left, right; \ + \ + if (p->lfe) { \ + left = right = src[0]; \ + dst[0] += s->gain_lfe * left; \ + dst[1] += s->gain_lfe * right; \ + } else { \ + left = bfilter_## name (src[0], \ + &p->cache1_in[0], \ + &p->cache1_out[0], \ + p->Bl[0], p->Bl[1], p->Al[1]); \ + \ + left = bfilter_## name (left, \ + &p->cache2_in[0], \ + &p->cache2_out[0], \ + p->bl[0], p->bl[1], p->al[1]); \ + \ + right = bfilter_## name (src[0], \ + &p->cache1_in[1], \ + &p->cache1_out[1], \ + p->Br[0], p->Br[1], p->Ar[1]); \ + \ + right = bfilter_## name (right, \ + &p->cache2_in[1], \ + &p->cache2_out[1], \ + p->br[0], p->br[1], p->ar[1]); \ + \ + left += dfilter_## name (left, p->gain[0], p->nb_samples[0], \ + p->delayed_samples[0], &p->delayed_index[0]); \ + \ + right += dfilter_## name (right, p->gain[1], p->nb_samples[1], \ + p->delayed_samples[1], &p->delayed_index[1]); \ + \ + dst[0] += scale * left; \ + dst[1] += scale * right; \ + } \ + } \ + \ + if (fabs(dst[0]) > 1) \ + n_clippings[0]++; \ + if (fabs(dst[1]) > 1) \ + n_clippings[1]++; \ + } \ + \ + /* display warning message if clipping occurred */ \ + if (n_clippings[0] + n_clippings[1] > 0) { \ + av_log(ctx, AV_LOG_WARNING, "%d of %d samples clipped. Please reduce gain.\n",\ + n_clippings[0] + n_clippings[1], out->nb_samples * 2); \ + } \ +} + +HRTFM_FILTER(flt, float) +HRTFM_FILTER(dbl, double) + +static int filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx = inlink->dst; + AVFilterLink *outlink = ctx->outputs[0]; + AVFrame *out = ff_get_audio_buffer(outlink, in->nb_samples); + + if (!out) { + av_frame_free(&in); + return AVERROR(ENOMEM); + } + av_frame_copy_props(out, in); + + if (inlink->format == AV_SAMPLE_FMT_FLT) + filter_samples_flt(ctx, in, out); + else + filter_samples_dbl(ctx, in, out); + + av_frame_free(&in); + return ff_filter_frame(outlink, out); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + HRTFMContext *s = ctx->priv; + + if (s->params_flt) { + int ch; + + for (ch = 0; ch < s->n_conv; ch++) { + ChParams_flt *p = &s->params_flt[ch]; + + av_freep(&p->delayed_samples[0]); + av_freep(&p->delayed_samples[1]); + } + + av_freep(&s->params_flt); + } + + if (s->params_dbl) { + int ch; + + for (ch = 0; ch < s->n_conv; ch++) { + ChParams_dbl *p = &s->params_dbl[ch]; + + av_freep(&p->delayed_samples[0]); + av_freep(&p->delayed_samples[1]); + } + + av_freep(&s->params_dbl); + } +} + +#define OFFSET(x) offsetof(HRTFMContext, x) +#define FLAGS AV_OPT_FLAG_AUDIO_PARAM|AV_OPT_FLAG_FILTERING_PARAM + +static const AVOption hrtfm_options[] = { + { "hradius", "set head radius", OFFSET(hradius), AV_OPT_TYPE_DOUBLE, {.dbl=8.91}, 5, 25, .flags = FLAGS }, + { "sspeed", "set sound speed", OFFSET(sspeed), AV_OPT_TYPE_DOUBLE, {.dbl=334}, 300, 400, .flags = FLAGS }, + { "amin", "set alpha min", OFFSET(alpha_min), AV_OPT_TYPE_DOUBLE, {.dbl=0.05},0.001, 1, .flags = FLAGS }, + { "gain", "set gain in dB", OFFSET(gain), AV_OPT_TYPE_DOUBLE, {.dbl=0}, -40, 40, .flags = FLAGS }, + { "rotation", "set rotation" , OFFSET(rotation), AV_OPT_TYPE_DOUBLE, {.dbl=0}, -180, 180, .flags = FLAGS }, + { "elevation", "set elevation", OFFSET(elevation), AV_OPT_TYPE_DOUBLE, {.dbl=0}, -90, 90, .flags = FLAGS }, + { "speakers", "set speaker custom positions", OFFSET(speakers_pos), AV_OPT_TYPE_STRING, {.str=0}, 0, 0, .flags = FLAGS }, + { "lfegain", "set LFE gain in dB", OFFSET(lfe_gain), AV_OPT_TYPE_DOUBLE, {.dbl=0}, -40, 40, .flags = FLAGS }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(hrtfm); + +static const AVFilterPad hrtfm_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + .config_props = config_input, + .filter_frame = filter_frame, + }, + { NULL } +}; + +static const AVFilterPad hrtfm_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + }, + { NULL } +}; + +AVFilter ff_af_hrtfm = { + .name = "hrtfm", + .description = NULL_IF_CONFIG_SMALL("Apply Head Related Transfer Function Model filter."), + .priv_size = sizeof(HRTFMContext), + .priv_class = &hrtfm_class, + .query_formats = query_formats, + .uninit = uninit, + .inputs = hrtfm_inputs, + .outputs = hrtfm_outputs, +}; diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index cc423af738..cf795febc1 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -110,6 +110,7 @@ static void register_all(void) REGISTER_FILTER(HDCD, hdcd, af); REGISTER_FILTER(HEADPHONE, headphone, af); REGISTER_FILTER(HIGHPASS, highpass, af); + REGISTER_FILTER(HRTFM, hrtfm, af); REGISTER_FILTER(JOIN, join, af); REGISTER_FILTER(LADSPA, ladspa, af); REGISTER_FILTER(LOUDNORM, loudnorm, af);