From patchwork Tue Mar 17 16:19:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marshall Murmu X-Patchwork-Id: 18271 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 6362544A924 for ; Tue, 17 Mar 2020 18:20:49 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4E15368B657; Tue, 17 Mar 2020 18:20:49 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D22F668B260 for ; Tue, 17 Mar 2020 18:20:42 +0200 (EET) Received: by mail-pl1-f175.google.com with SMTP id f16so9839699plj.4 for ; Tue, 17 Mar 2020 09:20:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=RcUZG19bYog7B6hapZh4ZYEo5gdqSFBBdC8RCOesTk0=; b=biS/Zt553Stp2xAuln5iSgwIwG8jIcqWRutfLtmZB4DKm1qsv/rrIOpqHspVMhqFfu Q/YbTxFnokmM/KIVP1X2vCyCF9k17RQBRMhLJ/MbjWhJ2/ByS4AYZ4J+Q0UWGJZm0+Et Aa4GAEVBhphGrYy9CV8v1MfUHPYleIab1Gf080I7SlnQvf2LfpcTithbkoCEMFgSAZI4 Z4smz46lGjSgpgHMJpCjzPkSPXSEAS/G7QQUPKRy6DAAIIR9DQrV6LGmMde1AqFfdKMA tltRpF+q3ylPxZuHUOl7YzwpUjxZ4r/rioUXHI4Wdmb04UBD3ca6P9WNEAZtfomjoNp2 /7Sw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RcUZG19bYog7B6hapZh4ZYEo5gdqSFBBdC8RCOesTk0=; b=DlERChFK778t8/PxGRZA2fZZwuCI5mqVbFhirvdkibuoxz8Qv+CJwpfHoOxCPtxUOh DDxM4zqYgHC9IVhryPG/qNja/LHfpUuR2znOgwrxAheVKIiUhA0eKAALC37LNeLqe35F Qi+vusc2TfSQhyKWxrES72PB2JfJ5KU2nEM0S7Idd/N1c2WBO6cSAIzRI7SHuOI36YH6 FuClT8NGGOzrE57vOutDkJrOjBauturIEdFJvJ8/9AFYVpaauKYYpO15Rv9g0YYkIkou wO1kjJCGx0cEt4D4pep9lMbVqwE3QUSj/Pg7PqvHIpUxzcjIjHVU6lHRTavkV4WWEHpB fXRQ== X-Gm-Message-State: ANhLgQ3lxkZuddOYPyvmmX39/eIUOo4aLqfY4eQ0jY3uizv/13MXcC4u FFgXY9YfxwFereeCOxYEijL0C85KpVY= X-Google-Smtp-Source: ADFU+vvqblc1QWzj1bQWiclEHpDWGqq6og8Gy5PPdDNNwXDmu7CdvG+8C2xYLaLLaRofcrt89B3bzQ== X-Received: by 2002:a17:902:a612:: with SMTP id u18mr5144044plq.1.1584462040225; Tue, 17 Mar 2020 09:20:40 -0700 (PDT) Received: from localhost.localdomain ([2402:3a80:a96:c224:b124:5d9b:8f6b:f05d]) by smtp.gmail.com with ESMTPSA id y9sm3098759pgo.80.2020.03.17.09.20.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Mar 2020 09:20:39 -0700 (PDT) From: Marshall Murmu To: ffmpeg-devel@ffmpeg.org Date: Tue, 17 Mar 2020 21:49:26 +0530 Message-Id: <20200317161927.18980-1-marshallmax1991@gmail.com> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200314102832.31962-1-marshallmax1991@gmail.com> References: <20200314102832.31962-1-marshallmax1991@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3][GSOC] avfilter: add atone filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" I hope this one's simple enough --- Changelog | 1 + configure | 4 + doc/filters.texi | 29 ++++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/asrc_atone.c | 188 +++++++++++++++++++++++++++++++++++++++ libavfilter/version.h | 2 +- 7 files changed, 225 insertions(+), 1 deletion(-) create mode 100644 libavfilter/asrc_atone.c diff --git a/Changelog b/Changelog index 711861bda9..7888ffc7fb 100644 --- a/Changelog +++ b/Changelog @@ -54,6 +54,7 @@ version : - DERF demuxer - CRI HCA decoder - CRI HCA demuxer +- atone filter version 4.2: diff --git a/configure b/configure index 18f2841765..b083ac6453 100755 --- a/configure +++ b/configure @@ -233,6 +233,7 @@ External library support: and libraw1394 [no] --enable-libfdk-aac enable AAC de/encoding via libfdk-aac [no] --enable-libflite enable flite (voice synthesis) support via libflite [no] + --enable-libfluidsynth enable fluidsynth support via libfluidsynth [no] --enable-libfontconfig enable libfontconfig, useful for drawtext filter [no] --enable-libfreetype enable libfreetype, needed for drawtext filter [no] --enable-libfribidi enable libfribidi, improves drawtext filter [no] @@ -1770,6 +1771,7 @@ EXTERNAL_LIBRARY_LIST=" libdc1394 libdrm libflite + libfluidsynth libfontconfig libfreetype libfribidi @@ -3465,6 +3467,7 @@ asr_filter_deps="pocketsphinx" ass_filter_deps="libass" atempo_filter_deps="avcodec" atempo_filter_select="rdft" +atone_filter_deps="libfluidsynth" avgblur_opencl_filter_deps="opencl" avgblur_vulkan_filter_deps="vulkan libglslang" azmq_filter_deps="libzmq" @@ -6270,6 +6273,7 @@ enabled libfdk_aac && { check_pkg_config libfdk_aac fdk-aac "fdk-aac/aace warn "using libfdk without pkg-config"; } } flite_extralibs="-lflite_cmu_time_awb -lflite_cmu_us_awb -lflite_cmu_us_kal -lflite_cmu_us_kal16 -lflite_cmu_us_rms -lflite_cmu_us_slt -lflite_usenglish -lflite_cmulex -lflite" enabled libflite && require libflite "flite/flite.h" flite_init $flite_extralibs +enabled libfluidsynth && require_pkg_config libfluidsynth fluidsynth "fluidsynth.h" fluid_log enabled fontconfig && enable libfontconfig enabled libfontconfig && require_pkg_config libfontconfig fontconfig "fontconfig/fontconfig.h" FcInit enabled libfreetype && require_pkg_config libfreetype freetype2 "ft2build.h FT_FREETYPE_H" FT_Init_FreeType diff --git a/doc/filters.texi b/doc/filters.texi index 328e984e92..918b1703f8 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -5946,6 +5946,35 @@ anullsrc=r=48000:cl=mono All the parameters need to be explicitly defined. +@section atone + +Synthesize random notes using libfluidsynth library. + +To compile this filter you need to configure FFmpeg with +@code{--enable-libfluidsynth}. + +The filter accepts the following options: + +@table @option +@item sample_rate, r +Set the sample rate of the synthesizer. Default value is 44100. + +@item nb_samples, n +Set the number of samples per frame. Default value is 1024. + +@item duration, d +Set the duration of sound generation. Default value is 10 sec. + +@item soundfont +Enter the location of the soundfont. Without loading the soundfont fluidsynth won't be able to synthesize. + +@item mchan +Set the MIDI channel. Default value is 0. + +@item seed +Set the seed value for the PRNG +@end table + @section flite Synthesize a voice utterance using the libflite library. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 750412da6b..020c4553cb 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -147,6 +147,7 @@ OBJS-$(CONFIG_AEVALSRC_FILTER) += aeval.o OBJS-$(CONFIG_AFIRSRC_FILTER) += asrc_afirsrc.o OBJS-$(CONFIG_ANOISESRC_FILTER) += asrc_anoisesrc.o OBJS-$(CONFIG_ANULLSRC_FILTER) += asrc_anullsrc.o +OBJS-$(CONFIG_ATONE_FILTER) += asrc_atone.o OBJS-$(CONFIG_FLITE_FILTER) += asrc_flite.o OBJS-$(CONFIG_HILBERT_FILTER) += asrc_hilbert.o OBJS-$(CONFIG_SINC_FILTER) += asrc_sinc.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 501e5d041b..d167499cf1 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -141,6 +141,7 @@ extern AVFilter ff_asrc_aevalsrc; extern AVFilter ff_asrc_afirsrc; extern AVFilter ff_asrc_anoisesrc; extern AVFilter ff_asrc_anullsrc; +extern AVFilter ff_asrc_atone; extern AVFilter ff_asrc_flite; extern AVFilter ff_asrc_hilbert; extern AVFilter ff_asrc_sinc; diff --git a/libavfilter/asrc_atone.c b/libavfilter/asrc_atone.c new file mode 100644 index 0000000000..25490424e0 --- /dev/null +++ b/libavfilter/asrc_atone.c @@ -0,0 +1,188 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with FFmpeg; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include "libavutil/avassert.h" +#include "libavutil/opt.h" +#include "libavutil/lfg.h" +#include "libavutil/random_seed.h" +#include "avfilter.h" +#include "audio.h" +#include "formats.h" +#include "internal.h" + +typedef struct AToneContext { + const AVClass *class; + fluid_settings_t *settings; + fluid_synth_t *synth; + int soundfont_id; + int nb_samples; + int sample_rate; + int mchan; + char *soundfont; + int64_t pts; + int64_t duration; + int64_t interval; + int64_t seed; + AVLFG c; +} AToneContext; + +#define OFFSET(x) offsetof(AToneContext, x) +#define FLAGS AV_OPT_FLAG_AUDIO_PARAM|AV_OPT_FLAG_FILTERING_PARAM + +static const AVOption atone_options[] = { + {"sample_rate", "set sample rate", OFFSET(sample_rate), AV_OPT_TYPE_INT, {.i64=44100}, 1, INT_MAX, FLAGS}, + {"r", "set sample rate", OFFSET(sample_rate), AV_OPT_TYPE_INT, {.i64=44100}, 1, INT_MAX, FLAGS}, + {"nb_samples", "set number of samples per requested frame", OFFSET(nb_samples), AV_OPT_TYPE_INT, {.i64=1024}, 1, INT64_MAX, FLAGS}, + {"n", "set number of samples per requested frame", OFFSET(nb_samples), AV_OPT_TYPE_INT, {.i64=1024}, 1, INT64_MAX, FLAGS}, + {"duration", "set duration", OFFSET(duration), AV_OPT_TYPE_DURATION, {.i64=10000000}, 0, INT64_MAX, FLAGS}, + {"d", "set duration", OFFSET(duration), AV_OPT_TYPE_DURATION, {.i64=10000000}, 0, INT64_MAX, FLAGS}, + {"soundfont", "location of soundfont", OFFSET(soundfont), AV_OPT_TYPE_STRING, {.str=NULL}, FLAGS}, + {"mchan", "set MIDI channel", OFFSET(mchan), AV_OPT_TYPE_INT, {.i64=0}, 0, 15, FLAGS}, + {"seed", "set random seed", OFFSET(seed), AV_OPT_TYPE_INT64, {.i64=-1}, -1, INT64_MAX, FLAGS}, + {"t", "set interval between notes", OFFSET(interval), AV_OPT_TYPE_DURATION, {.i64=0}, 0, INT64_MAX, FLAGS}, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(atone); + +static av_cold int init(AVFilterContext *ctx) +{ + AToneContext *fluidsynth = ctx->priv; + fluidsynth->soundfont_id = -1; + fluidsynth->settings = new_fluid_settings(); + if (!fluidsynth->settings) { + av_log(ctx, AV_LOG_ERROR, "Failed to create fluidsynth settings\n"); + return AVERROR_EXTERNAL; + } + fluidsynth->synth = new_fluid_synth(fluidsynth->settings); + if (!fluidsynth->synth) { + av_log(ctx, AV_LOG_ERROR, "Failed to create fluidsynth synthesizer\n"); + return AVERROR_EXTERNAL; + } + fluidsynth->soundfont_id = fluid_synth_sfload(fluidsynth->synth, fluidsynth->soundfont, 1); + if (fluidsynth->soundfont_id < 0) { + av_log(ctx, AV_LOG_ERROR, "Failed to load soundfont\n"); + return AVERROR_EXTERNAL; + } + + return 0; +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + AToneContext *fluidsynth = ctx->priv; + delete_fluid_synth(fluidsynth->synth); + delete_fluid_settings(fluidsynth->settings); +} + +static int query_formats(AVFilterContext *ctx) +{ + AToneContext *fluidsynth = ctx->priv; + AVFilterChannelLayouts *chanlayout = NULL; + int64_t chanlayouts = av_get_default_channel_layout(2*fluid_synth_count_audio_channels(fluidsynth->synth)); + AVFilterFormats *formats = NULL; + AVFilterFormats *sample_rate = NULL; + int ret; + + if ((ret = ff_add_format (&formats , AV_SAMPLE_FMT_FLT )) < 0 || + (ret = ff_set_common_formats (ctx , formats )) < 0 || + (ret = ff_add_channel_layout (&chanlayout , chanlayouts )) < 0 || + (ret = ff_set_common_channel_layouts (ctx , chanlayout )) < 0 || + (ret = ff_add_format (&sample_rate, fluidsynth->sample_rate)) < 0 || + (ret = ff_set_common_samplerates (ctx , sample_rate )) < 0) + return ret; + return 0; +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + AToneContext *fluidsynth = ctx->priv; + + if (fluidsynth->seed == -1) + fluidsynth->seed = av_get_random_seed(); + av_lfg_init(&fluidsynth->c, fluidsynth->seed); + + outlink->sample_rate = fluidsynth->sample_rate; + fluidsynth->duration = av_rescale(fluidsynth->duration, fluidsynth->sample_rate, AV_TIME_BASE); + fluidsynth->interval = av_rescale(fluidsynth->interval, fluidsynth->sample_rate, AV_TIME_BASE); + return 0; +} + +static int request_frame(AVFilterLink *outlink) +{ + AVFrame *frame; + AToneContext *fluidsynth = outlink->src->priv; + int nb_samples, key; + int64_t off; + + if (fluidsynth->duration) { + nb_samples = FFMIN(fluidsynth->nb_samples, fluidsynth->duration - fluidsynth->pts); + av_assert1(nb_samples >= 0); + if (!nb_samples) + return AVERROR_EOF; + } + + if (!(frame = ff_get_audio_buffer(outlink, nb_samples))) + return AVERROR(ENOMEM); + + key = av_lfg_get(&fluidsynth->c) % 128 ; + if (fluidsynth->interval <= nb_samples) { + fluid_synth_noteon(fluidsynth->synth, fluidsynth->mchan, key, 100); + fluid_synth_write_float(fluidsynth->synth, nb_samples, frame->data[0], 0, 2, frame->data[0], 1, 2); + } + if (nb_samples < fluidsynth->interval) { + off = fluidsynth->interval - (fluidsynth->pts % fluidsynth->interval); + if (fluidsynth->pts % fluidsynth->interval == 0) { + fluid_synth_noteon(fluidsynth->synth, fluidsynth->mchan, key, 100); + fluid_synth_write_float(fluidsynth->synth, nb_samples, frame->data[0], 0, 2, frame->data[0], 1, 2); + } else if (off < nb_samples) { + fluid_synth_write_float(fluidsynth->synth, off, frame->data[0], 0, 2, frame->data[0], 1, 2); + fluid_synth_noteon(fluidsynth->synth, fluidsynth->mchan, key, 100); + fluid_synth_write_float(fluidsynth->synth, nb_samples-off, frame->data[0], off*2, 2, frame->data[0], (off*2)+1, 2); + } else + fluid_synth_write_float(fluidsynth->synth, nb_samples, frame->data[0], 0, 2, frame->data[0], 1, 2); + } + + frame->pts = fluidsynth->pts; + fluidsynth->pts += nb_samples; + return ff_filter_frame(outlink, frame); +} + +static const AVFilterPad atone_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + .request_frame = request_frame, + .config_props = config_output, + }, + { NULL } +}; + +AVFilter ff_asrc_atone = { + .name = "atone", + .description = NULL_IF_CONFIG_SMALL("Synthesize tones using libfluidsynth."), + .query_formats = query_formats, + .init = init, + .uninit = uninit, + .priv_size = sizeof(AToneContext), + .inputs = NULL, + .outputs = atone_outputs, + .priv_class = &atone_class, +}; diff --git a/libavfilter/version.h b/libavfilter/version.h index 7b41018be7..4c4e8afe2d 100644 --- a/libavfilter/version.h +++ b/libavfilter/version.h @@ -30,7 +30,7 @@ #include "libavutil/version.h" #define LIBAVFILTER_VERSION_MAJOR 7 -#define LIBAVFILTER_VERSION_MINOR 77 +#define LIBAVFILTER_VERSION_MINOR 78 #define LIBAVFILTER_VERSION_MICRO 100