From patchwork Thu Mar 12 21:29:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marshall Murmu X-Patchwork-Id: 18141 Delivered-To: andriy.gelman@gmail.com Received: by 2002:a0c:9128:0:0:0:0:0 with SMTP id q37csp39360qvq; Thu, 12 Mar 2020 14:31:04 -0700 (PDT) X-Google-Smtp-Source: ADFU+vtQZifALJnChHXVkmOmN7K+OemDk8h88BcChkWqXmbuM5AuxNgqdp87EIILQsV9sGZsnYou X-Received: by 2002:a17:906:af6d:: with SMTP id os13mr8793588ejb.23.1584048663940; Thu, 12 Mar 2020 14:31:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1584048663; cv=none; d=google.com; s=arc-20160816; b=sazP42FFxaYAP0WsKKZRTgtLkageiftqahnL4v1K+D+1V23ZD9GCnwZUTaII2ZC88h UXrLXC2wp/0PDQ5oArEMGxgOVmtUtdKNrRdMeuBhy0wrDQ6zgn3phjkjsBCgXsYeEzMZ gzb7iZDu+SFbZAzzwmVgouF1V3oiZhzFnXtXB1GNKB8AkZwiFEbN3/NBvOKBpSwCEWdf jJ1IAUx/154Rl2qKCag31I1akSkAlH8VIz4IXmLYtHo+SbfhP449V4ZWPI8FKPu7vBJU jfx08eF2DTTceMnhgXchb0Rwc6ggKmkbSPZ23h2pZt+tVAnhO8BaUZ6kks1kpaVl4wj9 fERw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=WCTKkmRtR1rlMSwNWrPsrdjRQAbxgd7BLS3+/kL0Z40=; b=ZJPvFuFYS1eaUgXU9jGmk07lk/pEgmgKFlwEG+0fnhoRUdYzTGr7zrYDHMC37fFVLB alL65nO6AMIDJWbmY+cTKfQ5wqME3DW+H2fZfrx8+RE8y7T2CcS4sNMvqO2jKjGDkKae Ymt/BRgIfJddUQPWwXKjFERnttnwD7af/9+uUVLzLc8uP0ht1VAIvqIxQ3yHPKgwS0RJ 6zNYgDuyFRA6Vlx29JXTPSEApSjxK9oZF0xqmWmo2YDNOiFh27LNHQYKpxmIaprdm8HF oP4uTguK7cnjou9k9kIg8xBy8xV0CoHQXeXcOCn6AFwQuoo6F+sjFAbvE7gX33SFwLcV yY3g== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=Rw2f4tmx; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id p15si1803552edq.403.2020.03.12.14.31.03; Thu, 12 Mar 2020 14:31:03 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=Rw2f4tmx; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7AD6968AED5; Thu, 12 Mar 2020 23:31:01 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4107F68AF94 for ; Thu, 12 Mar 2020 23:30:54 +0200 (EET) Received: by mail-pl1-f175.google.com with SMTP id ay11so3196433plb.0 for ; Thu, 12 Mar 2020 14:30:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=h1eBP6KMnSAMhNWNtaMigLlU/f6MCSGevhQ2QuWVLjY=; b=Rw2f4tmxRSiELwEgh70iSYLS5M0yg0XdK0ppqSb3Yjq6BoSd2W/xUZuzKrKOZ9+b8v TDipsgaHSh0KxvbAN1YFvHX8kZKQDV+LW7Ggf5EZA3QRxweQa2F0b+xfFQ7yf34QABWW /7ecU/Hs/5Pk5PekkLY+xXE6Ace4Mvbj9x0WMEiYxTxeyv4MqyAE8Nc4+l9SzaeOIihE neQj/3S7NXsidWgnbxBbN7eX22uOHI5guOf3aoB51ppxeMlrG0H8wDfMeAVEDjgePH2k h+OE/xZ1L03+2TXtO9P664x7H0MYTelO9e63Yg0zumcwy65GfEbc6Qj7KHEi/QpfxMzi O5Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=h1eBP6KMnSAMhNWNtaMigLlU/f6MCSGevhQ2QuWVLjY=; b=TpgVKZkLvSwBodKMRPK2qvEo18hEgRsvjI3zKyx/AtXhlU+xAsjrfXtcl0/3C8Qts+ SZMds2aWsv1TAIdTpDLPfXI6h9CVGfcVb7GYPnavbZEXLsxbl7VQP0byrN27ZBzJMOZE 7TuF2hvL706M/iw1BOP7WgEwskkHy9KIDJXOEwM4lM11Xjl2GedEOeCQBGqgH6SSHxqd M/qg03PFBnFqaZAlwHDeKdmmOqAxkeL7CRl2luwSv8NRqNlaTr16F4DcUOaPTZto9+8D Tg5KsgAm4117xQS3F73Z6a2XmAlUTsAakgPaUc7XKB3QuL9PYAI58KSa4V+5NXsq4Rc8 asaQ== X-Gm-Message-State: ANhLgQ3RZY6bzPRBF+QMeDU5oUxkVCLyiUeqFPZMitJhr3Q/R1KZfOI1 +koUlZOYzIJrDaRI8ElIvqVOB+/P X-Received: by 2002:a17:902:9308:: with SMTP id bc8mr10289315plb.268.1584048652085; Thu, 12 Mar 2020 14:30:52 -0700 (PDT) Received: from localhost.localdomain ([2402:3a80:abe:9fcb:a63a:e4a1:8f3:baa9]) by smtp.gmail.com with ESMTPSA id j12sm39383029pga.78.2020.03.12.14.30.49 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Mar 2020 14:30:51 -0700 (PDT) From: Marshall Murmu To: ffmpeg-devel@ffmpeg.org Date: Fri, 13 Mar 2020 02:59:52 +0530 Message-Id: <20200312212952.4623-1-marshallmax1991@gmail.com> X-Mailer: git-send-email 2.25.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH][GSOC] avfilter: add fluidsynth filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Ep0/bCkUUI2D Content-Length: 16940 This patch is part of the qualification task for Audio Tones Source Filter. --- Changelog | 1 + configure | 4 + doc/filters.texi | 26 +++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/asrc_fluidsynth.c | 177 ++++++++++++++++++++++++++++++++++ libavfilter/version.h | 2 +- 7 files changed, 211 insertions(+), 1 deletion(-) create mode 100644 libavfilter/asrc_fluidsynth.c diff --git a/Changelog b/Changelog index db2ca92e8a..2a22599969 100644 --- a/Changelog +++ b/Changelog @@ -46,6 +46,7 @@ version : - High Voltage Software ADPCM decoder - LEGO Racers ALP (.tun & .pcm) demuxer - AMQP 0-9-1 protocol (RabbitMQ) +- fluidsynth filter version 4.2: diff --git a/configure b/configure index 8b17134944..b22a09fac1 100755 --- a/configure +++ b/configure @@ -233,6 +233,7 @@ External library support: and libraw1394 [no] --enable-libfdk-aac enable AAC de/encoding via libfdk-aac [no] --enable-libflite enable flite (voice synthesis) support via libflite [no] + --enable-libfluidsynth enable fluidsynth support via libfluidsynth [no] --enable-libfontconfig enable libfontconfig, useful for drawtext filter [no] --enable-libfreetype enable libfreetype, needed for drawtext filter [no] --enable-libfribidi enable libfribidi, improves drawtext filter [no] @@ -1770,6 +1771,7 @@ EXTERNAL_LIBRARY_LIST=" libdc1394 libdrm libflite + libfluidsynth libfontconfig libfreetype libfribidi @@ -3509,6 +3511,7 @@ find_rect_filter_deps="avcodec avformat gpl" firequalizer_filter_deps="avcodec" firequalizer_filter_select="rdft" flite_filter_deps="libflite" +fluidsynth_filter_deps="libfluidsynth" framerate_filter_select="scene_sad" freezedetect_filter_select="scene_sad" frei0r_filter_deps="frei0r libdl" @@ -6269,6 +6272,7 @@ enabled libfdk_aac && { check_pkg_config libfdk_aac fdk-aac "fdk-aac/aace warn "using libfdk without pkg-config"; } } flite_extralibs="-lflite_cmu_time_awb -lflite_cmu_us_awb -lflite_cmu_us_kal -lflite_cmu_us_kal16 -lflite_cmu_us_rms -lflite_cmu_us_slt -lflite_usenglish -lflite_cmulex -lflite" enabled libflite && require libflite "flite/flite.h" flite_init $flite_extralibs +enabled libfluidsynth && require_pkg_config libfluidsynth fluidsynth "fluidsynth.h" fluid_log enabled fontconfig && enable libfontconfig enabled libfontconfig && require_pkg_config libfontconfig fontconfig "fontconfig/fontconfig.h" FcInit enabled libfreetype && require_pkg_config libfreetype freetype2 "ft2build.h FT_FREETYPE_H" FT_Init_FreeType diff --git a/doc/filters.texi b/doc/filters.texi index ff008b119f..cdf96d1ba1 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -6010,6 +6010,32 @@ ffplay -f lavfi flite=text='No more be grieved for which that thou hast done.' For more information about libflite, check: @url{http://www.festvox.org/flite/} +@section fluidsynth + +Synthesize random notes using libfluidsynth library. + +To compile this filter you need to configure FFmpeg with +@code{--enable-libfluidsynth}. + +The filter accepts the following options: + +@table @option +@item sample_rate, r +Set the sample rate of the synthesizer. Default value is 44100. + +@item nb_samples, n +Set the number of samples per frame. Default value is 1024. + +@item duration, d +Set the duration of sound generation. Default value is 10 sec. + +@item soundfont +Enter the location of the soundfont. Without loading the soundfont fluidsynth won't be able to synthesize. + +@item chan +Set the MIDI channel. Default value is 0. +@end table + @section anoisesrc Generate a noise audio signal. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 750412da6b..626d7c0f09 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -148,6 +148,7 @@ OBJS-$(CONFIG_AFIRSRC_FILTER) += asrc_afirsrc.o OBJS-$(CONFIG_ANOISESRC_FILTER) += asrc_anoisesrc.o OBJS-$(CONFIG_ANULLSRC_FILTER) += asrc_anullsrc.o OBJS-$(CONFIG_FLITE_FILTER) += asrc_flite.o +OBJS-$(CONFIG_FLUIDSYNTH_FILTER) += asrc_fluidsynth.o OBJS-$(CONFIG_HILBERT_FILTER) += asrc_hilbert.o OBJS-$(CONFIG_SINC_FILTER) += asrc_sinc.o OBJS-$(CONFIG_SINE_FILTER) += asrc_sine.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 501e5d041b..a1e812093c 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -142,6 +142,7 @@ extern AVFilter ff_asrc_afirsrc; extern AVFilter ff_asrc_anoisesrc; extern AVFilter ff_asrc_anullsrc; extern AVFilter ff_asrc_flite; +extern AVFilter ff_asrc_fluidsynth; extern AVFilter ff_asrc_hilbert; extern AVFilter ff_asrc_sinc; extern AVFilter ff_asrc_sine; diff --git a/libavfilter/asrc_fluidsynth.c b/libavfilter/asrc_fluidsynth.c new file mode 100644 index 0000000000..be98815c56 --- /dev/null +++ b/libavfilter/asrc_fluidsynth.c @@ -0,0 +1,177 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with FFmpeg; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include "libavutil/avassert.h" +#include "libavutil/opt.h" +#include "libavutil/lfg.h" +#include "libavutil/random_seed.h" +#include "avfilter.h" +#include "audio.h" +#include "formats.h" +#include "internal.h" + +typedef struct FluidSynthContext { + const AVClass *class; + fluid_settings_t *settings; + fluid_synth_t *synth; + int soundfont_id; + int nb_samples; + int sample_rate; + int chan; + char *soundfont; + int64_t pts; + int64_t duration; + AVLFG c; +} FluidSynthContext; + +#define OFFSET(x) offsetof(FluidSynthContext, x) +#define FLAGS AV_OPT_FLAG_AUDIO_PARAM|AV_OPT_FLAG_FILTERING_PARAM + +static const AVOption fluidsynth_options[] = { + {"sample_rate", "set sample rate", OFFSET(sample_rate), AV_OPT_TYPE_INT, {.i64=44100}, 1, INT_MAX, FLAGS}, + {"r", "set sample rate", OFFSET(sample_rate), AV_OPT_TYPE_INT, {.i64=44100}, 1, INT_MAX, FLAGS}, + {"nb_samples", "set number of samples per requested frame", OFFSET(nb_samples), AV_OPT_TYPE_INT, {.i64=1024}, 1, INT64_MAX, FLAGS}, + {"n", "set number of samples per requested frame", OFFSET(nb_samples), AV_OPT_TYPE_INT, {.i64=1024}, 1, INT64_MAX, FLAGS}, + {"duration", "set duration", OFFSET(duration), AV_OPT_TYPE_DURATION, {.i64=10000000}, 0, INT64_MAX, FLAGS}, + {"d", "set duration", OFFSET(duration), AV_OPT_TYPE_DURATION, {.i64=10000000}, 0, INT64_MAX, FLAGS}, + {"soundfont", "location of soundfont", OFFSET(soundfont), AV_OPT_TYPE_STRING, {.str=NULL}, CHAR_MIN, CHAR_MAX, FLAGS}, + {"chan", "set MIDI channel", OFFSET(chan), AV_OPT_TYPE_INT, {.str=0}, 0, 15, FLAGS}, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(fluidsynth); + +static av_cold int init(AVFilterContext *ctx) +{ + FluidSynthContext *fluidsynth = ctx->priv; + fluidsynth->soundfont_id = -1; + /// Initialize the fluidsynth setting + fluidsynth->settings = new_fluid_settings(); + if(fluidsynth->settings == NULL) { + av_log(ctx, AV_LOG_ERROR, "Failed to create fluidsynth settings\n"); + return AVERROR_UNKNOWN; + } + /// Initialize the synthesizer + fluidsynth->synth = new_fluid_synth(fluidsynth->settings); + if(fluidsynth->synth == NULL) { + av_log(ctx, AV_LOG_ERROR, "Failed to create fluidsynth synthesizer\n"); + return AVERROR_UNKNOWN; + } + + fluidsynth->soundfont_id = fluid_synth_sfload(fluidsynth->synth, fluidsynth->soundfont, 1); + if(fluidsynth->soundfont_id < 0) { + av_log(ctx, AV_LOG_ERROR, "Failed to load soundfont\n"); + } + + return 0; +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + FluidSynthContext *fluidsynth = ctx->priv; + + delete_fluid_synth(fluidsynth->synth); + delete_fluid_settings(fluidsynth->settings); + +} + +static int query_formats(AVFilterContext *ctx) +{ + FluidSynthContext *fluidsynth = ctx->priv; + + AVFilterChannelLayouts *chanlayout = NULL; + int64_t chanlayouts = av_get_default_channel_layout(2*fluid_synth_count_audio_channels(fluidsynth->synth)); + + AVFilterFormats *formats = NULL; + AVFilterFormats *sample_rate = NULL; + int ret; + + if ((ret = ff_add_format (&formats , AV_SAMPLE_FMT_FLT )) < 0 || + (ret = ff_set_common_formats (ctx , formats )) < 0 || + (ret = ff_add_channel_layout (&chanlayout , chanlayouts )) < 0 || + (ret = ff_set_common_channel_layouts (ctx , chanlayout )) < 0 || + (ret = ff_add_format (&sample_rate, fluidsynth->sample_rate)) < 0 || + (ret = ff_set_common_samplerates (ctx , sample_rate )) < 0) + return ret; + return 0; +} + + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + FluidSynthContext *fluidsynth = ctx->priv; + + av_lfg_init(&fluidsynth->c, av_get_random_seed()); + + outlink->sample_rate = fluidsynth->sample_rate; + fluidsynth->duration = av_rescale(fluidsynth->duration, fluidsynth->sample_rate, AV_TIME_BASE); + + return 0; +} + +static int request_frame(AVFilterLink *outlink) +{ + AVFrame *frame; + FluidSynthContext *fluidsynth = outlink->src->priv; + int nb_samples, key; + + if (fluidsynth->duration) { + nb_samples = FFMIN(fluidsynth->nb_samples, fluidsynth->duration - fluidsynth->pts); + av_assert1(nb_samples >= 0); + if (!nb_samples) + return AVERROR_EOF; + } + + if(!(frame = ff_get_audio_buffer(outlink, nb_samples))) + return AVERROR(ENOMEM); + + key = av_lfg_get(&fluidsynth->c) % 128 ; + + fluid_synth_noteon(fluidsynth->synth, fluidsynth->chan, key, 100); + fluid_synth_write_float(fluidsynth->synth, nb_samples, frame->data[0], 0, 2, frame->data[0], 1, 2); + fluid_synth_noteoff(fluidsynth->synth, fluidsynth->chan, key); + + frame->pts = fluidsynth->pts; + fluidsynth->pts += nb_samples; + + return ff_filter_frame(outlink, frame); +} + +static const AVFilterPad fluidsynth_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + .request_frame = request_frame, + .config_props = config_output, + }, + { NULL } +}; + +AVFilter ff_asrc_fluidsynth = { + .name = "fluidsynth", + .description = NULL_IF_CONFIG_SMALL("Synthesize tones using libfluidsynth."), + .query_formats = query_formats, + .init = init, + .uninit = uninit, + .priv_size = sizeof(FluidSynthContext), + .inputs = NULL, + .outputs = fluidsynth_outputs, + .priv_class = &fluidsynth_class, +}; diff --git a/libavfilter/version.h b/libavfilter/version.h index 7b41018be7..4c4e8afe2d 100644 --- a/libavfilter/version.h +++ b/libavfilter/version.h @@ -30,7 +30,7 @@ #include "libavutil/version.h" #define LIBAVFILTER_VERSION_MAJOR 7 -#define LIBAVFILTER_VERSION_MINOR 77 +#define LIBAVFILTER_VERSION_MINOR 78 #define LIBAVFILTER_VERSION_MICRO 100