From patchwork Sat Jan 21 18:28:52 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 2280 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.89.21 with SMTP id n21csp593423vsb; Sat, 21 Jan 2017 10:29:15 -0800 (PST) X-Received: by 10.28.130.212 with SMTP id e203mr7400410wmd.104.1485023355645; Sat, 21 Jan 2017 10:29:15 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h35si12395686wrh.233.2017.01.21.10.29.15; Sat, 21 Jan 2017 10:29:15 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 23A8868A850; Sat, 21 Jan 2017 20:29:01 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f68.google.com (mail-wm0-f68.google.com [74.125.82.68]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CF2D668A821 for ; Sat, 21 Jan 2017 20:28:54 +0200 (EET) Received: by mail-wm0-f68.google.com with SMTP id c85so15061113wmi.1 for ; Sat, 21 Jan 2017 10:29:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=zUXAQ4fAZm15rgsicUQXnBk/LFnEJjOJuya5DqjZZUo=; b=AL0V2/ggGMfYza4Ff8liLW4JYZF+jMe15j7COSdRdLGpsbCFjOhOzWLSH4Z19c47PR s+W3IWzfB1mXLdpRkHT4I3BZQeRTnnnFyCHojTg9a4II9xNWd4B/tu9so6v+2t90U8Qi h6W1QlJDuNlFrYUttivm0ePQrCeifebWqHVdi3ah8IOxkSHlhadr7LMF3wiAwdsjKorV e1uY+5PG/XxboLrNX2bNAWkPvly5voDNUGti9YY31XWtpyQOKtOoMK2LZXtQVlce70+l KZXuoKo3a62J5caVJqWcUIqExXU5xQZ4fmRHhVPvMtLNhY80h0cGpc8HKwEY0NosZC2r fLOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=zUXAQ4fAZm15rgsicUQXnBk/LFnEJjOJuya5DqjZZUo=; b=JIcPjI/8DH8nX6DyYWw9zAU+WYf1IFZLTezdYnxHaXLPyF2RHNhSGDzA1jH8WGGaMw eLv6vySvSgdpagnEAKA4S7UcKdqxWeQwdsHsersnaHNUnqIWXMrYawIf/6/vNmX+4+II PD3qeP37gIUZeQGYQpv+VbNP07o1KurTbw/Q0ue0Xoqw5jFAhP7yXt1GfzuIxYzLXVeK kMvWUa3axTvbJCHIL/TDlCdfBcEHq+L3jEOIu5sv0J7MBQuBTqz6f3jZo81wyPVS1xOl BkB3Dzwwg7uDQblpdxB5moyVCfQ+l2bQ6obDljMxOuz+KDZaTrcgCmU6zhs6W00xbOcO LDbA== X-Gm-Message-State: AIkVDXLPCGyebzT74QpsRk3gNBVZrHF/SsnG9jTrupyweYqbRRgDuFyhn+3NlaVVifhl5A== X-Received: by 10.28.64.213 with SMTP id n204mr8512693wma.12.1485023345839; Sat, 21 Jan 2017 10:29:05 -0800 (PST) Received: from localhost.localdomain ([37.244.225.202]) by smtp.gmail.com with ESMTPSA id w70sm5578454wrc.47.2017.01.21.10.29.03 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 Jan 2017 10:29:04 -0800 (PST) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Sat, 21 Jan 2017 19:28:52 +0100 Message-Id: <20170121182852.1611-1-onemda@gmail.com> X-Mailer: git-send-email 2.9.3 Subject: [FFmpeg-devel] [PATCH] avfilter: add abitscope multimedia filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Paul B Mahol --- doc/filters.texi | 22 ++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/avf_abitscope.c | 250 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 274 insertions(+) create mode 100644 libavfilter/avf_abitscope.c diff --git a/doc/filters.texi b/doc/filters.texi index a0c2980..888b2eb 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -15557,6 +15557,28 @@ tools. Below is a description of the currently available multimedia filters. +@section abitscope + +Convert input audio to a video output, displaying the audio bit scope. + +The filter accepts the following options: + +@table @option +@item rate, r +Set frame rate, expressed as number of frames per second. Default +value is "25". + +@item size, s +Specify the video size for the output. For the syntax of this option, check the +@ref{video size syntax,,"Video size" section in the ffmpeg-utils manual,ffmpeg-utils}. +Default value is @code{1024x256}. + +@item colors +Specify list of colors separated by space or by '|' which will be used to +draw channel frequencies. Unrecognized or missing colors will be replaced +by white color. +@end table + @section ahistogram Convert input audio to a video output, displaying the volume histogram. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 9ab65eb..683d52b 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -332,6 +332,7 @@ OBJS-$(CONFIG_YUVTESTSRC_FILTER) += vsrc_testsrc.o OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o # multimedia filters +OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o OBJS-$(CONFIG_ADRAWGRAPH_FILTER) += f_drawgraph.o OBJS-$(CONFIG_AHISTOGRAM_FILTER) += avf_ahistogram.o OBJS-$(CONFIG_APHASEMETER_FILTER) += avf_aphasemeter.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 2c37818..1a224c0 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -347,6 +347,7 @@ void avfilter_register_all(void) REGISTER_FILTER(NULLSINK, nullsink, vsink); /* multimedia filters */ + REGISTER_FILTER(ABITSCOPE, abitscope, avf); REGISTER_FILTER(ADRAWGRAPH, adrawgraph, avf); REGISTER_FILTER(AHISTOGRAM, ahistogram, avf); REGISTER_FILTER(APHASEMETER, aphasemeter, avf); diff --git a/libavfilter/avf_abitscope.c b/libavfilter/avf_abitscope.c new file mode 100644 index 0000000..4f5d4c7 --- /dev/null +++ b/libavfilter/avf_abitscope.c @@ -0,0 +1,250 @@ +/* + * Copyright (c) 2016 Paul B Mahol + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/avassert.h" +#include "libavutil/avstring.h" +#include "libavutil/intreadwrite.h" +#include "libavutil/opt.h" +#include "libavutil/parseutils.h" +#include "avfilter.h" +#include "formats.h" +#include "audio.h" +#include "video.h" +#include "internal.h" + +typedef struct AudioBitScopeContext { + const AVClass *class; + int w, h; + AVRational frame_rate; + char *colors; + + int nb_channels; + int depth; + uint8_t *fg; + + uint64_t counter[64]; +} AudioBitScopeContext; + +#define OFFSET(x) offsetof(AudioBitScopeContext, x) +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM + +static const AVOption abitscope_options[] = { + { "rate", "set video rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, {.str="25"}, 0, 0, FLAGS }, + { "r", "set video rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, {.str="25"}, 0, 0, FLAGS }, + { "size", "set video size", OFFSET(w), AV_OPT_TYPE_IMAGE_SIZE, {.str="1024x256"}, 0, 0, FLAGS }, + { "s", "set video size", OFFSET(w), AV_OPT_TYPE_IMAGE_SIZE, {.str="1024x256"}, 0, 0, FLAGS }, + { "colors", "set channels colors", OFFSET(colors), AV_OPT_TYPE_STRING, {.str = "red|green|blue|yellow|orange|lime|pink|magenta|brown" }, 0, 0, FLAGS }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(abitscope); + +static int query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats = NULL; + AVFilterChannelLayouts *layouts; + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + static const enum AVSampleFormat sample_fmts[] = { AV_SAMPLE_FMT_S16P, AV_SAMPLE_FMT_S32P, AV_SAMPLE_FMT_NONE }; + static const enum AVPixelFormat pix_fmts[] = { AV_PIX_FMT_RGBA, AV_PIX_FMT_NONE }; + int ret; + + formats = ff_make_format_list(sample_fmts); + if ((ret = ff_formats_ref(formats, &inlink->out_formats)) < 0) + return ret; + + layouts = ff_all_channel_counts(); + if (!layouts) + return AVERROR(ENOMEM); + if ((ret = ff_channel_layouts_ref(layouts, &inlink->out_channel_layouts)) < 0) + return ret; + + formats = ff_all_samplerates(); + if ((ret = ff_formats_ref(formats, &inlink->out_samplerates)) < 0) + return ret; + + formats = ff_make_format_list(pix_fmts); + if ((ret = ff_formats_ref(formats, &outlink->in_formats)) < 0) + return ret; + + return 0; +} + +static int config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + AudioBitScopeContext *s = ctx->priv; + int ch, nb_samples; + char *colors, *saveptr = NULL; + + nb_samples = FFMAX(1024, ((double)inlink->sample_rate / av_q2d(s->frame_rate)) + 0.5); + inlink->partial_buf_size = + inlink->min_samples = + inlink->max_samples = nb_samples; + s->nb_channels = inlink->channels; + s->depth = inlink->format == AV_SAMPLE_FMT_S16P ? 16 : 32; + + s->fg = av_malloc_array(s->nb_channels, 4 * sizeof(*s->fg)); + if (!s->fg) + return AVERROR(ENOMEM); + + colors = av_strdup(s->colors); + if (!colors) + return AVERROR(ENOMEM); + + for (ch = 0; ch < s->nb_channels; ch++) { + uint8_t fg[4] = { 0xff, 0xff, 0xff, 0xff }; + char *color; + + color = av_strtok(ch == 0 ? colors : NULL, " |", &saveptr); + if (color) + av_parse_color(fg, color, -1, ctx); + s->fg[4 * ch + 0] = fg[0]; + s->fg[4 * ch + 1] = fg[1]; + s->fg[4 * ch + 2] = fg[2]; + s->fg[4 * ch + 3] = fg[3]; + } + av_free(colors); + + return 0; +} + +static int config_output(AVFilterLink *outlink) +{ + AudioBitScopeContext *s = outlink->src->priv; + + outlink->w = s->w; + outlink->h = s->h; + outlink->sample_aspect_ratio = (AVRational){1,1}; + outlink->frame_rate = s->frame_rate; + + return 0; +} + +static void count_bits(AudioBitScopeContext *s, uint32_t sample, int max) +{ + int i; + + for (i = 0; i < max; i++) { + if (sample & (1 << i)) + s->counter[i]++; + } +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *insamples) +{ + AVFilterContext *ctx = inlink->dst; + AVFilterLink *outlink = ctx->outputs[0]; + AudioBitScopeContext *s = ctx->priv; + AVFrame *outpicref; + int ch, i, j, b; + + outpicref = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!outpicref) { + av_frame_free(&insamples); + return AVERROR(ENOMEM); + } + + for (i = 0; i < outlink->h; i++) + memset(outpicref->data[0] + i * outpicref->linesize[0], 0, outlink->w * 4); + + outpicref->pts = insamples->pts; + switch (insamples->format) { + case AV_SAMPLE_FMT_S16P: + for (ch = 0; ch < inlink->channels; ch++) { + uint16_t *in = (uint16_t *)insamples->extended_data[ch]; + int w = outpicref->width / inlink->channels; + int h = outpicref->height / 16; + uint32_t color = AV_RN32(&s->fg[4 * ch]); + + memset(s->counter, 0, sizeof(s->counter)); + for (i = 0; i < insamples->nb_samples; i++) + count_bits(s, in[i], 16); + + for (b = 0; b < 16; b++) { + for (j = 1; j < h - 1; j++) { + uint8_t *dst = outpicref->data[0] + (b * h + j) * outpicref->linesize[0] + w * ch * 4; + int ww = (s->counter[16 - b - 1] / (float)insamples->nb_samples) * (w - 1); + + for (i = 0; i < ww; i++) { + AV_WN32(&dst[i * 4], color); + } + } + } + } + break; + case AV_SAMPLE_FMT_S32P: + for (ch = 0; ch < inlink->channels; ch++) { + uint32_t *in = (uint32_t *)insamples->extended_data[ch]; + int w = outpicref->width / inlink->channels; + int h = outpicref->height / 32; + uint32_t color = AV_RN32(&s->fg[4 * ch]); + + memset(s->counter, 0, sizeof(s->counter)); + for (i = 0; i < insamples->nb_samples; i++) + count_bits(s, in[i], 32); + + for (b = 0; b < 32; b++) { + for (j = 1; j < h - 1; j++) { + uint8_t *dst = outpicref->data[0] + (b * h + j) * outpicref->linesize[0] + w * ch * 4; + int ww = (s->counter[32 - b - 1] / (float)insamples->nb_samples) * (w - 1); + + for (i = 0; i < ww; i++) { + AV_WN32(&dst[i * 4], color); + } + } + } + } + break; + } + + av_frame_free(&insamples); + + return ff_filter_frame(outlink, outpicref); +} + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + .config_props = config_input, + .filter_frame = filter_frame, + }, + { NULL } +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_output, + }, + { NULL } +}; + +AVFilter ff_avf_abitscope = { + .name = "abitscope", + .description = NULL_IF_CONFIG_SMALL("Convert input audio to audio bit scope video output."), + .query_formats = query_formats, + .priv_size = sizeof(AudioBitScopeContext), + .inputs = inputs, + .outputs = outputs, + .priv_class = &abitscope_class, +};