From patchwork Wed Mar 18 13:42:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashutosh Pradhan X-Patchwork-Id: 18286 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id D245344A6EA for ; Wed, 18 Mar 2020 15:44:08 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B961068B6D6; Wed, 18 Mar 2020 15:44:08 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7DEBC68B6A7 for ; Wed, 18 Mar 2020 15:44:02 +0200 (EET) Received: by mail-pj1-f47.google.com with SMTP id bo3so1228107pjb.5 for ; Wed, 18 Mar 2020 06:44:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Lgu5iMo77t9/VbUfuww1DMqT38x7s7xYSqSOot2lbTs=; b=fGua7Yn2foF3URrJ3kK9DJQSGw+kzB+4E6NuThIVF0Jy9dLEH7XbfUxid1xsut3uSG zvlx38DWlwV4XoVvIHq1rKk3X4kSPdvyNaHmit4ChaEYzzFVbhFZoUmV8ppLFnFRqB9q z1xWHgxPvodw/Ge12kUKrsTw3eGTW0FpMLZUM4AVGbQeKK2KUB1seql9Te1Qc+ixb1mA 9DjAn/J8y0qS0pzqa2ibiCVS+oZXXEucVse82xmAsK5Kny7rpXdzuB3TwUVHIeEd4q+9 ZIF1/Xl6QhMMlweE4+ltrVQuFsdBN23F7rCcAhT8gKmjGBImsfRiHA0TGolqwbJ8c6QP hnaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Lgu5iMo77t9/VbUfuww1DMqT38x7s7xYSqSOot2lbTs=; b=p1+UfV1h0gcvJ4r6Bp++2HQ406GHGVzYn53f1PZvozEZqQOOrIo0zTaTwLcijLdo0l 1nrP17Ak0K8gqBa9YnbHm/vJrS8dY2VWGfNHFOuehJAiErDpkQj257MNN79OQ3TeMNji ias/vFmVAFxNA5ypInPTksPZWpKi1My1C1iBEmVh2BsDpViUxPxwS2xJZlbeFFUNxARo tQnI8/+La483HFaQj1yi6n11p5JLvxmHRvNzfAt4bfhrzmrlUQG2KyH2FHd2dpPz6PVW gpMwpMmxLl8lt2g0rR6CMGmTTRGG/6cqliLTh2vRGgnSoiZFBSorgiShqTDdn5++H5vm /avA== X-Gm-Message-State: ANhLgQ2Bb2JCjFy7hdto7voO/CFHw+rZkev6+aPPxqE/ad/Bu90Reqs6 OEytZHzm3KKxIFcq4pPEcl+ZY1vB X-Google-Smtp-Source: ADFU+vsgdd52brcZrfiCGKQhqZg1gEX7HUeX02MSdNHDwLXGrYKixYCGsrJ9fAd0CM0Z0ZCDBVeIow== X-Received: by 2002:a17:902:5a44:: with SMTP id f4mr3943073plm.306.1584539040133; Wed, 18 Mar 2020 06:44:00 -0700 (PDT) Received: from localhost.localdomain ([106.211.151.167]) by smtp.gmail.com with ESMTPSA id r186sm7012786pfc.181.2020.03.18.06.43.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Mar 2020 06:43:59 -0700 (PDT) From: Ashutosh Pradhan To: ffmpeg-devel@ffmpeg.org Date: Wed, 18 Mar 2020 19:12:27 +0530 Message-Id: <20200318134227.31994-1-ashutoshp012345@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: Subject: [FFmpeg-devel] [PATCH][GSoC]audio filter-use cellular automata to generate tones X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: ashutoshp012345@gmail.com MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Use cellular automata to generate tones diff --git a/Changelog b/Changelog index d1572553a5..5ddd2484b0 100644 --- a/Changelog +++ b/Changelog @@ -48,6 +48,7 @@ version : - AMQP 0-9-1 protocol (RabbitMQ) - Vulkan support - avgblur_vulkan, overlay_vulkan, scale_vulkan and chromaber_vulkan filters +- atone filter version 4.2: diff --git a/configure b/configure index 6ceb0c7af4..2ec7e377f3 100755 --- a/configure +++ b/configure @@ -233,6 +233,7 @@ External library support: and libraw1394 [no] --enable-libfdk-aac enable AAC de/encoding via libfdk-aac [no] --enable-libflite enable flite (voice synthesis) support via libflite [no] + --enable-libfluidsynth enable libfluidsynth support for fluidsynth [no] --enable-libfontconfig enable libfontconfig, useful for drawtext filter [no] --enable-libfreetype enable libfreetype, needed for drawtext filter [no] --enable-libfribidi enable libfribidi, improves drawtext filter [no] @@ -1770,6 +1771,7 @@ EXTERNAL_LIBRARY_LIST=" libdc1394 libdrm libflite + libfluidsynth libfontconfig libfreetype libfribidi @@ -3465,6 +3467,7 @@ asr_filter_deps="pocketsphinx" ass_filter_deps="libass" atempo_filter_deps="avcodec" atempo_filter_select="rdft" +atone_filter_deps="libfluidsynth" avgblur_opencl_filter_deps="opencl" avgblur_vulkan_filter_deps="vulkan libglslang" azmq_filter_deps="libzmq" @@ -6270,6 +6273,7 @@ enabled libfdk_aac && { check_pkg_config libfdk_aac fdk-aac "fdk-aac/aace warn "using libfdk without pkg-config"; } } flite_extralibs="-lflite_cmu_time_awb -lflite_cmu_us_awb -lflite_cmu_us_kal -lflite_cmu_us_kal16 -lflite_cmu_us_rms -lflite_cmu_us_slt -lflite_usenglish -lflite_cmulex -lflite" enabled libflite && require libflite "flite/flite.h" flite_init $flite_extralibs +enabled libfluidsynth && require_pkg_config libfluidsynth fluidsynth "fluidsynth.h" fluid_log enabled fontconfig && enable libfontconfig enabled libfontconfig && require_pkg_config libfontconfig fontconfig "fontconfig/fontconfig.h" FcInit enabled libfreetype && require_pkg_config libfreetype freetype2 "ft2build.h FT_FREETYPE_H" FT_Init_FreeType diff --git a/doc/filters.texi b/doc/filters.texi index 328e984e92..0eda8a4c6e 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -6050,6 +6050,57 @@ anoisesrc=d=60:c=pink:r=44100:a=0.5 @end example @end itemize +@section atone + +Generate legato tones using cellular automata. +To compile filter configure ffmpeg with @code{--enable-libfluidsynth} + +The filter accepts the following options: + +@table @option +@item sample_rate, r +Specify the sample rate. Default value is 44100 Hz. + +@item sfont +Specify the location of soundfont file. Default value is +"/usr/share/sounds/sf2/FluidR3_GM.sf2"(for linux). + +@item duration, d +Specify the duration of the generated audio stream. Not specifying this option +results in playing tones for infinite length. + +@item velocity, v +Specify the velocity of key press. Default value is 80. + +@item MIDI_channel +Specify the MIDI channel number between(0 to 16) to play tones. Default is 0. + +@item tone_change_interval, t +Specify the time interval between successive tones in seconds. Default is 0.2s. + +@item rule +Specify the rule between 0 to 255 to get the rule set. Default is 30. + +@item width +Specify the width of the cells array. Default is 64. + +@item samples_per_frame +Set the number of samples per each output frame. Default is 1024. +@end table + +@subsection Examples + +@itemize + +@item +Generate 10 seconds of random tones, with a key velocity of 100, midi channel 1 +and an tone change interval of 0.1s: +@example +atone=d=10:MIDI_channel=1:v=100:t=0.1:sfont="example.sf2":rule=193 +atone=duration=10:MIDI_channel=1:velocity=100:tone_change_interval=0.1:sfont="example.sf2":rule=193 +@end example +@end itemize + @section hilbert Generate odd-tap Hilbert transform FIR coefficients. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 750412da6b..b1f0c4be35 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -151,6 +151,7 @@ OBJS-$(CONFIG_FLITE_FILTER) += asrc_flite.o OBJS-$(CONFIG_HILBERT_FILTER) += asrc_hilbert.o OBJS-$(CONFIG_SINC_FILTER) += asrc_sinc.o OBJS-$(CONFIG_SINE_FILTER) += asrc_sine.o +OBJS-$(CONFIG_ATONE_FILTER) += asrc_atone.o OBJS-$(CONFIG_ANULLSINK_FILTER) += asink_anullsink.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 501e5d041b..4d3efc7e15 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -145,6 +145,7 @@ extern AVFilter ff_asrc_flite; extern AVFilter ff_asrc_hilbert; extern AVFilter ff_asrc_sinc; extern AVFilter ff_asrc_sine; +extern AVFilter ff_asrc_atone; extern AVFilter ff_asink_anullsink; diff --git a/libavfilter/asrc_atone.c b/libavfilter/asrc_atone.c new file mode 100644 index 0000000000..04465ec4e0 --- /dev/null +++ b/libavfilter/asrc_atone.c @@ -0,0 +1,275 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with FFmpeg; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ +#include +#include +#include +#include +#include +#include + +#include "libavutil/avassert.h" +#include "libavutil/channel_layout.h" +#include "libavutil/eval.h" +#include "libavutil/opt.h" +#include "libavutil/lfg.h" +#include "libavutil/random_seed.h" +#include "audio.h" +#include "avfilter.h" +#include "internal.h" + +typedef struct AtoneContext +{ + const AVClass* class; + int64_t duration; + int nb_samples; + int sample_rate; + int64_t pts; + int infinite; + + fluid_settings_t* settings; + fluid_synth_t* synth; + char* sfont; ///< soundfont file + int sfont_id; + int midi_chan; ///< midi channel number + int velocity; ///< velocity of key + double changerate; ///< get the time interval of changing tones + int key[7]; ///< play tones within one octave + int *cells; + int *nextgen; + int rule; ///< Rule for changing value in cell(taking 3 cells at time hence 2^8 rules possible) + int width; ///< width of the cell array + int ruleset[8]; ///< taking 3 cells at a time +}AtoneContext; + +#define CONTEXT AtoneContext +#define FLAGS AV_OPT_FLAG_AUDIO_PARAM|AV_OPT_FLAG_FILTERING_PARAM + +#define OPT_GENERIC(name, field, def, min, max, descr, type, deffield, ...) \ + { name, descr, offsetof(CONTEXT, field), AV_OPT_TYPE_ ## type, \ + { .deffield = def }, min, max, FLAGS, __VA_ARGS__ } + +#define OPT_INT(name, field, def, min, max, descr, ...) \ + OPT_GENERIC(name, field, def, min, max, descr, INT, i64, __VA_ARGS__) + +#define OPT_DBL(name, field, def, min, max, descr, ...) \ + OPT_GENERIC(name, field, def, min, max, descr, DOUBLE, dbl, __VA_ARGS__) + +#define OPT_DUR(name, field, def, min, max, descr, ...) \ + OPT_GENERIC(name, field, def, min, max, descr, DURATION, str, __VA_ARGS__) + +#define OPT_STR(name, field, def, min, max, descr, ...) \ + OPT_GENERIC(name, field, def, min, max, descr, STRING, str, __VA_ARGS__) + +static const AVOption atone_options[] = { + OPT_INT("velocity", velocity, 80, 0, INT_MAX, "set the velocity of key press",), + OPT_INT("v", velocity, 80, 0, INT_MAX, "set the velocity of key press",), + OPT_INT("sample_rate", sample_rate, 44100, 1, INT_MAX, "set the sample rate",), + OPT_INT("r", sample_rate, 44100, 1, INT_MAX, "set the sample rate",), + OPT_DUR("duration", duration, 0, 0, INT64_MAX, "set the audio duration",), + OPT_DUR("d", duration, 0, 0, INT64_MAX, "set the audio duration",), + OPT_STR("sfont", sfont, "/usr/share/sounds/sf2/FluidR3_GM.sf2", 0, 0, "set the soundfont file",), + OPT_INT("samples_per_frame", nb_samples, 1024, 0, INT_MAX, "set the number of samples per frame",), + OPT_INT("MIDI_channel", midi_chan, 0, 0, 16, "set the MIDI Channel",), + OPT_DBL("tone_change_interval",changerate, 0.2, 0, DBL_MAX, "set the random tone change time in seconds",), + OPT_DBL("t", changerate, 0.2, 0, DBL_MAX, "set the random tone change time in seconds",), + OPT_INT("rule", rule, 30, 0, 256, "set the rule for changing value in cell",), + OPT_INT("width", width, 64, 0, INT_MAX, "set the width of the cell array",), + {NULL} +}; + +AVFILTER_DEFINE_CLASS(atone); + +static av_cold int init(AVFilterContext *ctx) +{ + AtoneContext *s = ctx->priv; + int val; + /*Initialise the fluidsynth settings object followed by synthesizer*/ + s->settings = new_fluid_settings(); + if (s->settings== NULL){ + av_log(s, AV_LOG_ERROR, "Failed to create the fluidsynth settings"); + return AVERROR_EXTERNAL; + } + + s->synth = new_fluid_synth(s->settings); + if (s->synth== NULL){ + av_log(s, AV_LOG_ERROR, "Failed to create the fluidsynth synth"); + return AVERROR_EXTERNAL; + } + + s->sfont_id= fluid_synth_sfload(s->synth, s->sfont, 1); + if(s->sfont_id== FLUID_FAILED){ + av_log(s, AV_LOG_ERROR, "Loading the Soundfont Failed"); + return AVERROR_EXTERNAL; + } + if (!(s->cells = av_malloc(sizeof(int)*(s->width)))) + return AVERROR(ENOMEM); + + if (!(s->nextgen = av_malloc(sizeof(int)*(s->width)))) + return AVERROR(ENOMEM); + + for (int i=0; i<7; i++) + s->key[i]=0; + + s->changerate= s->changerate*s->sample_rate/s->nb_samples; + if (s->changerate<1.0) + s->changerate= 1.0; + + for (int i=0; i< s->width; i++) + s->cells[i]=0; + s->cells[s->width/2]=1; + + val= s->rule; + for (int i=0; i<8; i++){ + s->ruleset[i]= val%2; + val= val>>1; + } + return 0; +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + AtoneContext *s = ctx->priv; + + delete_fluid_synth(s->synth); + delete_fluid_settings(s->settings); + av_freep(&s->cells); + av_freep(&s->nextgen); +} + +static av_cold int config_props(AVFilterLink *outlink) +{ + AtoneContext *s = outlink->src->priv; + + if (s->duration==0) + s->infinite=1; + s->duration = av_rescale(s->duration, s->sample_rate, AV_TIME_BASE); + return 0; +} +/* +Generate keys using cellular automata. Reference taken from http://tones.wolfram.com/about/how-it-works +to generate the cells. Rule 30(default) gives random numbers. Here middle part(+-3) of the cell generation +is taken to get the keys. All the keys generated in the octave are played at the same time. +*/ +static void generate(AtoneContext* s){ + for (int i=0; i< s->width; i++){ + int c=0; + for (int j=0; j<3; j++){ + c+= s->cells[(i+j-1)%s->width]<<(j); + } + s->nextgen[i]= s->ruleset[c]; + } + memcpy(s->cells, s->nextgen, s->width*sizeof(int)); + memcpy(s->key, &s->cells[s->width/2-3], 7*sizeof(int)); +} + +static void gen_random_tone_samples(AtoneContext* s, int buf_size, float* data){ + + for(int i=0; i<7; i++) + if (s->key[i]==1) + fluid_synth_noteon(s->synth, s->midi_chan, (64+i), s->velocity);//pp range octave + + fluid_synth_write_float(s->synth, buf_size, data, 0, 2, data, 1, 2); + fluid_synth_all_notes_off(s->synth, s->midi_chan); +} + +static int request_frame(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + AtoneContext *s = ctx->priv; + AVFrame *frame; + int nb_samples; + if (!s->infinite && s->duration <= 0) { + return AVERROR_EOF; + } else if (!s->infinite && s->duration < s->nb_samples) { + nb_samples = s->duration; + } else { + nb_samples = s->nb_samples; + } + + if (!(frame = ff_get_audio_buffer(outlink, nb_samples))) + return AVERROR(ENOMEM); + + if (s->pts % ((int)(s->changerate)*nb_samples) == 0) + generate(s); + + gen_random_tone_samples(s, nb_samples, (float*)frame->data[0]); + + if (!s->infinite) + s->duration -= nb_samples; + + frame->pts = s->pts; + s->pts += nb_samples; + return ff_filter_frame(outlink, frame); +} + +static av_cold int query_formats(AVFilterContext *ctx) +{ + AtoneContext *s = ctx->priv; + static const int64_t chlayouts[] = { AV_CH_LAYOUT_STEREO, -1 }; + int sample_rates[] = { s->sample_rate, -1 }; + static const enum AVSampleFormat sample_fmts[] = { AV_SAMPLE_FMT_FLT, AV_SAMPLE_FMT_NONE}; + AVFilterFormats *formats; + AVFilterChannelLayouts *layouts; + int ret; + + formats = ff_make_format_list(sample_fmts); + if (!formats) + return AVERROR(ENOMEM); + ret = ff_set_common_formats (ctx, formats); + if (ret < 0) + return ret; + + layouts = avfilter_make_format64_list(chlayouts); + if (!layouts) + return AVERROR(ENOMEM); + ret = ff_set_common_channel_layouts(ctx, layouts); + if (ret < 0) + return ret; + + formats = ff_make_format_list(sample_rates); + if (!formats) + return AVERROR(ENOMEM); + return ff_set_common_samplerates(ctx, formats); +} + + +static const AVFilterPad atone_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + .request_frame = request_frame, + .config_props = config_props, + }, + { NULL } +}; + +AVFilter ff_asrc_atone = { + .name = "atone", + .description = NULL_IF_CONFIG_SMALL("Generate tones using cellular automata."), + .query_formats = query_formats, + .init = init, + .uninit = uninit, + .priv_size = sizeof(AtoneContext), + .inputs = NULL, + .outputs = atone_outputs, + .priv_class = &atone_class, +}; + + + + diff --git a/libavfilter/version.h b/libavfilter/version.h index 7b41018be7..4c4e8afe2d 100644 --- a/libavfilter/version.h +++ b/libavfilter/version.h @@ -30,7 +30,7 @@ #include "libavutil/version.h" #define LIBAVFILTER_VERSION_MAJOR 7 -#define LIBAVFILTER_VERSION_MINOR 77 +#define LIBAVFILTER_VERSION_MINOR 78 #define LIBAVFILTER_VERSION_MICRO 100