From patchwork Sat Jan 11 09:30:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 17302 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B72D144B85D for ; Sat, 11 Jan 2020 11:38:22 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9649968AE8B; Sat, 11 Jan 2020 11:38:22 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f46.google.com (mail-wm1-f46.google.com [209.85.128.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C55B668062B for ; Sat, 11 Jan 2020 11:38:15 +0200 (EET) Received: by mail-wm1-f46.google.com with SMTP id f129so4454551wmf.2 for ; Sat, 11 Jan 2020 01:38:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=Z8I+XkXd8/gnVdk+rjRfuEUoj1AqiIfKIKUWk1xc97Q=; b=ibTMYORpoH2CcmAJJFkLXsCvFWaogFuBP1l/oKuBiBjkdIUWb8lWSPgUFF/Ku2S7To 0KDY9UyhuyzP8nFsuXtr5mozQtnz7sRNiIb6tsmJT9Q1PECVaXt53oahIH+aU7GdBYng vipXtM33Kz4Bjvq3LDMw1qlj/xGikp/9zZOVK4husYHz0p8X9hPJ5bfgYJXusXYeihtV DSvi3MVVfYeopx6zJrRlH4wRMLpZc79aFQjxTrY47y62RajrOZxBttTk6n58YNYllgMG OUMVvSeddcZTMstMwLcwfxaQsJKEotVNJyuRH1/4futwXNe5S8+a7RxoZGM6RYRSxSXf s6mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=Z8I+XkXd8/gnVdk+rjRfuEUoj1AqiIfKIKUWk1xc97Q=; b=TfFGhQqonHRAqD5AqQ7Ea0HfoorxEKysxNqPueJHy7gFvn9w8DjZQDMq98henmUxLK aJtn9WCKGNT4SdMy3H/LXGg1Vx4Tzal4UTUqPMlMuH3j70T0Bi7OneoazK3ixRvGfOXT T4jA//IswjBpMyfQAFOrU7mf2Z89ut4qjRq3B5Y+DWFed/738NzDGzxuED2dk6n4pfsf meLlQi+FTUMnDirzQ326RyKNJKVNzol6o6O6nB+a/HaBYTgMGpT9Z6B+N1dWnmiJ/5PL 4OWNZrADEp8bLPrhS5mAn/Yn6DwcP2KbLXx8VsT9wOZHS2PlHXwfc3S+gp1orq6ws/8v AUig== X-Gm-Message-State: APjAAAVt013aYWUQ1UHEovSwf7UiGqFOclvSwzRqrOGZ+yX2Q0oqBW4R VwPYOABdvQZ0fHXtfpokWg2CdGyAv0Q= X-Google-Smtp-Source: APXvYqyRFEbkZWp9RPTZE7kShbx96i6FvwVuwscrrEcwVaGLJB9jGQwTV2xQtPB6s0UwYPM0MmZp/w== X-Received: by 2002:a05:600c:210e:: with SMTP id u14mr9154723wml.28.1578735033124; Sat, 11 Jan 2020 01:30:33 -0800 (PST) Received: from localhost.localdomain ([212.15.166.117]) by smtp.gmail.com with ESMTPSA id m7sm5735149wrr.40.2020.01.11.01.30.31 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 11 Jan 2020 01:30:32 -0800 (PST) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Sat, 11 Jan 2020 10:30:20 +0100 Message-Id: <20200111093020.22537-1-onemda@gmail.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH] avfilter: add csound audio filter wrapper X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Paul B Mahol --- configure | 4 + doc/filters.texi | 16 +++ libavfilter/Makefile | 1 + libavfilter/af_csound.c | 267 +++++++++++++++++++++++++++++++++++++++ libavfilter/allfilters.c | 1 + 5 files changed, 289 insertions(+) create mode 100644 libavfilter/af_csound.c diff --git a/configure b/configure index 46f2038627..da5bd9a485 100755 --- a/configure +++ b/configure @@ -207,6 +207,7 @@ External library support: --disable-bzlib disable bzlib [autodetect] --disable-coreimage disable Apple CoreImage framework [autodetect] --enable-chromaprint enable audio fingerprinting with chromaprint [no] + --enable-csound enable Csound audio filtering [no] --enable-frei0r enable frei0r video filtering [no] --enable-gcrypt enable gcrypt, needed for rtmp(t)e support if openssl, librtmp or gmp is not used [no] @@ -1752,6 +1753,7 @@ EXTERNAL_LIBRARY_LIST=" $EXTERNAL_LIBRARY_VERSION3_LIST $EXTERNAL_LIBRARY_GPLV3_LIST chromaprint + csound gcrypt gnutls jni @@ -3474,6 +3476,7 @@ coreimagesrc_filter_deps="coreimage appkit" coreimagesrc_filter_extralibs="-framework OpenGL" cover_rect_filter_deps="avcodec avformat gpl" cropdetect_filter_deps="gpl" +csound_filter_deps="csound" deconvolve_filter_deps="avcodec" deconvolve_filter_select="fft" deinterlace_qsv_filter_deps="libmfx" @@ -6222,6 +6225,7 @@ done # these are off by default, so fail if requested and not available enabled cuda_nvcc && { check_nvcc cuda_nvcc || die "ERROR: failed checking for nvcc."; } enabled chromaprint && require chromaprint chromaprint.h chromaprint_get_version -lchromaprint +enabled csound && require csound csound/csound.h csoundCreate -lcsound64 enabled decklink && { require_headers DeckLinkAPI.h && { test_cpp_condition DeckLinkAPIVersion.h "BLACKMAGIC_DECKLINK_API_VERSION >= 0x0a090500" || die "ERROR: Decklink API version must be >= 10.9.5."; } } enabled frei0r && require_headers "frei0r.h dlfcn.h" diff --git a/doc/filters.texi b/doc/filters.texi index 6fb660b05a..41c90e6819 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -3230,6 +3230,22 @@ Enable clipping. By default is enabled. This filter supports the all above options as @ref{commands}. +@section csound + +Load a Csound plugin. + +Csound is a unit generator-based, user-programmable computer music system. + +To enable compilation of this filter you need to configure FFmpeg with +@code{--enable-csound}. + +@table @option +@item csd +Give name or full path to CSD script file. +CSD file holds unified orchestra and score file. +This option must always be set. +@end table + @section dcshift Apply a DC shift to the audio. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 8b8a5bd535..18885950ec 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -102,6 +102,7 @@ OBJS-$(CONFIG_COMPAND_FILTER) += af_compand.o OBJS-$(CONFIG_COMPENSATIONDELAY_FILTER) += af_compensationdelay.o OBJS-$(CONFIG_CROSSFEED_FILTER) += af_crossfeed.o OBJS-$(CONFIG_CRYSTALIZER_FILTER) += af_crystalizer.o +OBJS-$(CONFIG_CSOUND_FILTER) += af_csound.o OBJS-$(CONFIG_DCSHIFT_FILTER) += af_dcshift.o OBJS-$(CONFIG_DEESSER_FILTER) += af_deesser.o OBJS-$(CONFIG_DRMETER_FILTER) += af_drmeter.o diff --git a/libavfilter/af_csound.c b/libavfilter/af_csound.c new file mode 100644 index 0000000000..9404a42f58 --- /dev/null +++ b/libavfilter/af_csound.c @@ -0,0 +1,267 @@ +/* + * Copyright (c) 2020 Paul B Mahol + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * Csound wrapper + */ + +#include +#include "libavutil/avassert.h" +#include "libavutil/avstring.h" +#include "libavutil/channel_layout.h" +#include "libavutil/opt.h" +#include "audio.h" +#include "avfilter.h" +#include "filters.h" +#include "internal.h" + +typedef struct CsoundContext { + const AVClass *class; + int sample_rate; + int nchnls, nchnls_input; + char *csd_filename; + + uint32_t ksmps; + int format; + int64_t pts; + CSOUND *csound; + controlChannelInfo_t *devs; +} CsoundContext; + +#define OFFSET(x) offsetof(CsoundContext, x) +#define FLAGS AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_FILTERING_PARAM + +static const AVOption csound_options[] = { + { "csd", "set CSD name or full path", OFFSET(csd_filename), AV_OPT_TYPE_STRING, .flags = FLAGS }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(csound); + +static int activate(AVFilterContext *ctx) +{ + CsoundContext *s = ctx->priv; + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + int ret, nb_samples, nb_in_samples; + MYFLT *buffer; + AVFrame *out, *in; + + FF_FILTER_FORWARD_STATUS_BACK_ALL(outlink, ctx); + + nb_in_samples = csoundGetInputBufferSize(s->csound) / s->nchnls; + + if (nb_in_samples > 0) { + ret = ff_inlink_consume_samples(inlink, nb_in_samples, nb_in_samples, &in); + if (ret > 0) { + buffer = csoundGetInputBuffer(s->csound); + if (buffer) { + const int nch = s->nchnls; + MYFLT *dst = buffer; + MYFLT *src = (MYFLT *)in->data[0]; + + for (int f = 0; f < nb_in_samples / s->ksmps; f++) { + for (int ch = 0; ch < nch; ch++) { + for (int n = 0; n < s->ksmps; n++) { + dst[n] = *src++; + } + dst += s->ksmps; + } + } + } + av_frame_free(&in); + } else if (!ret) { + FF_FILTER_FORWARD_WANTED(outlink, inlink); + return 0; + } else { + return ret; + } + } + ret = csoundPerformBuffer(s->csound); + nb_samples = csoundGetOutputBufferSize(s->csound) / s->nchnls; + if (ret || !nb_samples) { + ff_outlink_set_status(outlink, AVERROR_EOF, 0); + return 0; + } + + buffer = csoundGetOutputBuffer(s->csound); + if (buffer) { + const int nch = s->nchnls; + MYFLT *src = buffer; + MYFLT *dst; + + out = ff_get_audio_buffer(outlink, nb_samples); + if (!out) + return AVERROR(ENOMEM); + dst = (MYFLT *)out->data[0]; + out->pts = s->pts; + s->pts += nb_samples; + for (int f = 0; f < nb_samples / s->ksmps; f++) { + for (int ch = 0; ch < nch; ch++) { + for (int n = 0; n < s->ksmps; n++) { + *dst++ = src[n]; + } + src += s->ksmps; + } + } + + return ff_filter_frame(outlink, out); + } + + FF_FILTER_FORWARD_STATUS(inlink, outlink); + + return 0; +} + +static void callback(CSOUND *csound, int attr, const char *format, va_list valist) +{ + int level; + + switch (attr & CSOUNDMSG_TYPE_MASK) { + case CSOUNDMSG_ERROR: level = AV_LOG_ERROR; break; + case CSOUNDMSG_WARNING: level = AV_LOG_WARNING; break; + default: level = AV_LOG_VERBOSE; break; + } + + av_vlog(csoundGetHostData(csound), level, format, valist); +} + +static av_cold int init(AVFilterContext *ctx) +{ + CsoundContext *s = ctx->priv; + int ret, size; + + csoundSetDefaultMessageCallback(callback); + s->csound = csoundCreate(NULL); + if (!s->csound) + return AVERROR(ENOMEM); + + if (!s->csd_filename) + return AVERROR(EINVAL); + + csoundSetHostData(s->csound, s); + csoundSetHostImplementedAudioIO(s->csound, 1, 1024); + csoundSetMessageCallback(s->csound, callback); + ret = csoundCompileCsd(s->csound, s->csd_filename); + if (ret != 0) + return AVERROR_EXTERNAL; + csoundStart(s->csound); + s->sample_rate = csoundGetSr(s->csound); + s->ksmps = csoundGetKsmps(s->csound); + s->nchnls = csoundGetNchnls(s->csound); + s->nchnls_input = csoundGetNchnlsInput(s->csound); + + size = csoundGetSizeOfMYFLT(); + switch (size) { + case 4: s->format = AV_SAMPLE_FMT_FLT; break; + case 8: s->format = AV_SAMPLE_FMT_DBL; break; + default: return AVERROR_BUG; + } + + return 0; +} + +static int query_formats(AVFilterContext *ctx) +{ + CsoundContext *s = ctx->priv; + AVFilterFormats *formats; + AVFilterChannelLayouts *layouts; + int sample_rates[] = { s->sample_rate, -1 }; + enum AVSampleFormat sample_fmts[] = { + s->format, AV_SAMPLE_FMT_NONE }; + uint64_t inlayout = FF_COUNT2LAYOUT(s->nchnls_input); + uint64_t outlayout = FF_COUNT2LAYOUT(s->nchnls); + AVFilterLink *outlink = ctx->outputs[0]; + AVFilterLink *inlink = ctx->inputs[0]; + int ret; + + formats = ff_make_format_list(sample_fmts); + if (!formats) + return AVERROR(ENOMEM); + + ret = ff_set_common_formats(ctx, formats); + if (ret < 0) + return ret; + + formats = ff_make_format_list(sample_rates); + if (!formats) + return AVERROR(ENOMEM); + + ret = ff_set_common_samplerates(ctx, formats); + if (ret < 0) + return ret; + + + layouts = NULL; + ret = ff_add_channel_layout(&layouts, inlayout); + if (ret < 0) + return ret; + ret = ff_channel_layouts_ref(layouts, &inlink->out_channel_layouts); + if (ret < 0) + return ret; + + layouts = NULL; + ret = ff_add_channel_layout(&layouts, outlayout); + if (ret < 0) + return ret; + ret = ff_channel_layouts_ref(layouts, &outlink->in_channel_layouts); + if (ret < 0) + return ret; + + return 0; +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + CsoundContext *s = ctx->priv; + + csoundReset(s->csound); + csoundDestroy(s->csound); +} + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + }, + { NULL } +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + }, + { NULL } +}; + +AVFilter ff_af_csound = { + .name = "csound", + .description = NULL_IF_CONFIG_SMALL("Apply Csound script."), + .priv_size = sizeof(CsoundContext), + .priv_class = &csound_class, + .init = init, + .activate = activate, + .uninit = uninit, + .query_formats = query_formats, + .inputs = inputs, + .outputs = outputs, +}; diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 9f2080f857..12a591ff5d 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -95,6 +95,7 @@ extern AVFilter ff_af_compand; extern AVFilter ff_af_compensationdelay; extern AVFilter ff_af_crossfeed; extern AVFilter ff_af_crystalizer; +extern AVFilter ff_af_csound; extern AVFilter ff_af_dcshift; extern AVFilter ff_af_deesser; extern AVFilter ff_af_drmeter;