From patchwork Wed Jul 3 13:33:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Romane Lafon X-Patchwork-Id: 13805 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 23116448079 for ; Wed, 3 Jul 2019 16:34:07 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F0FD368A9B4; Wed, 3 Jul 2019 16:34:06 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oi1-f177.google.com (mail-oi1-f177.google.com [209.85.167.177]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D1FB968A08B for ; Wed, 3 Jul 2019 16:33:59 +0300 (EEST) Received: by mail-oi1-f177.google.com with SMTP id t76so2079166oih.4 for ; Wed, 03 Jul 2019 06:33:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nomalab-com.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=nN1F2Xm6W8dys0HpZgMZnLGOLOl0SlsMudskuJGYyNA=; b=WV3CfGqJqTVqhZkGGQZdjjm01k1V8tBPnoN11b/5LgIaS/MKoe5zEgEPRnsXr1iZd+ ENtBQnniDpBThPLVBhu3/FvkkWd1R/rZ+/NsmW4yKt5tFuJBiKADDh/+D+WEBr07d9aI kavgIAmjHFRh58FuIYURrX9DNWB3unrAnzKzTSclw5Rvd1JubvIN19DFgIbSW2Pue8ei tVE8LoAj4lbCQIuJp8W2pL75KAMt9CJ/SQB+pIgHcSl0IW0qrfIDLlQc9CvbcCz3+0vo qeYKk5HKaTG8alSIhNvnfXFlzf2wZWZ4yv3CFmNKpAZbuVIVbyfz4C9qGmOPHemytrR0 cn3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=nN1F2Xm6W8dys0HpZgMZnLGOLOl0SlsMudskuJGYyNA=; b=qVNQMW07AfgJTKQTlarXHIdfSPIkfvKm2B/aKQzH0rCPWkYjjklFlp2+euKU9ss4/R WDu8WqBli8yVhnJQMAcjhFnGobeNl0WI2uwJvBDXZ59ntPE5Gj1zabwqd8hVxp2l3f6g /oLFnvpqauExd7rZ7i6udUVQu1+Js1cWdSDLgDfCm14oCdbU1GCL9BF7yOVS6JefwXXQ b9GW0kWw4Zdn+S38WuF3KT1+6r2KeCPS3v48K1XlmuhbvH+EEvJS2OCB6YLZuWMDsO1f PTzoF2Tn2yANy0HOWiPYQmxh4Ba3TXWVKnhhpG7CdxT8ih+mdrFRd9RkumXrV86EzTF5 Usdw== X-Gm-Message-State: APjAAAWHhx/0QQVEdVFQ4hJ4sqXXgv3HDU1VKVHY+md7VzWI2AGLyLT3 z4qOKSbgo8CUzltuXe+F7LvmnofHV2FP1CNup9R8bqD7 X-Google-Smtp-Source: APXvYqwkbLcGHx6UR/+drSAXF4b2uPTTgsR8rkMfAfoZKCHLCaUrHDBxGiaXO2h6GY8O06I/5qVLUlmsV7evdfVj8N8= X-Received: by 2002:aca:da56:: with SMTP id r83mr6298099oig.84.1562160837927; Wed, 03 Jul 2019 06:33:57 -0700 (PDT) MIME-Version: 1.0 From: Romane Lafon Date: Wed, 3 Jul 2019 15:33:46 +0200 Message-ID: To: FFmpeg development discussions and patches X-Content-Filtered-By: Mailman/MimeDel 2.1.20 Subject: [FFmpeg-devel] [PATCH v4] avfilter/avf_aphasemeter: Add out-of-phase and mono detection X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" I've added documentation for the extension of aphasemeter filter. Also, I'm not sure that "phasing" is the right word to describe the detection. From 1e356929e878a2081add102b77a9560647232ef8 Mon Sep 17 00:00:00 2001 From: Romane Lafon Date: Wed, 3 Jul 2019 15:15:16 +0200 Subject: [PATCH] avfilter/avf_aphasemeter: Add out of phase and mono detection Signed-off-by: Romane Lafon --- doc/filters.texi | 32 +++++++++++ libavfilter/avf_aphasemeter.c | 127 ++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 155 insertions(+), 4 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 700a76f239..ec8c73d558 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -20656,6 +20656,38 @@ Set color which will be used for drawing median phase. If color is Enable video output. Default is enabled. @end table +@subsection phasing detection + +The filter also detects out of phase and mono sequences in stereo streams. +It logs the sequence start, end and duration when it lasts longer or as long as the minimum set. + +The filter accepts the following options for this detection: + +@table @option +@item phasing +Enable mono and out of phase detection. Default is disabled. + +@item tolerance +Set phase tolerance for mono detection, in amplitude ratio. Default is @code{0}. +Allowed range is @code{[0, 1]}. + +@item angle +Set angle threshold for out of phase detection, in degree. Default is @code{170}. +Allowed range is @code{[0, 180]}. + +@item duration +Set mono or out of phase duration until notification, expressed in seconds. Default is @code{2}. + +@subsection Examples + +@itemize +@item +Complete example with @command{ffmpeg} to detect 1 second of mono with 0.001 phase tolerance: +@example +ffmpeg -i stereo.wav -af aphasemeter=video=0:phasing=1:duration=1:tolerance=0.001 -f null - +@end example +@end itemize + @section avectorscope Convert input audio to a video output, representing the audio vector diff --git a/libavfilter/avf_aphasemeter.c b/libavfilter/avf_aphasemeter.c index f497bc9969..77701e5cde 100644 --- a/libavfilter/avf_aphasemeter.c +++ b/libavfilter/avf_aphasemeter.c @@ -28,26 +28,41 @@ #include "libavutil/intreadwrite.h" #include "libavutil/opt.h" #include "libavutil/parseutils.h" +#include "libavutil/timestamp.h" #include "avfilter.h" #include "formats.h" #include "audio.h" #include "video.h" #include "internal.h" +#include "stdbool.h" +#include "float.h" typedef struct AudioPhaseMeterContext { const AVClass *class; AVFrame *out; int do_video; + int do_phasing_detection; int w, h; AVRational frame_rate; int contrast[4]; uint8_t *mpc_str; uint8_t mpc[4]; int draw_median_phase; + int is_mono; + int is_out_phase; + int start_mono_presence; + int start_out_phase_presence; + float tolerance; + float angle; + float phase; + float mono_idx[2]; + float out_phase_idx[2]; + double duration; } AudioPhaseMeterContext; #define OFFSET(x) offsetof(AudioPhaseMeterContext, x) #define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM +#define get_duration(index) (index[1] - index[0]) static const AVOption aphasemeter_options[] = { { "rate", "set video rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, {.str="25"}, 0, INT_MAX, FLAGS }, @@ -59,6 +74,10 @@ static const AVOption aphasemeter_options[] = { { "bc", "set blue contrast", OFFSET(contrast[2]), AV_OPT_TYPE_INT, {.i64=1}, 0, 255, FLAGS }, { "mpc", "set median phase color", OFFSET(mpc_str), AV_OPT_TYPE_STRING, {.str = "none"}, 0, 0, FLAGS }, { "video", "set video output", OFFSET(do_video), AV_OPT_TYPE_BOOL, {.i64 = 1}, 0, 1, FLAGS }, + { "phasing", "set mono and out-of-phase detection output", OFFSET(do_phasing_detection), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, FLAGS }, + { "tolerance", "set phase tolerance for mono detection", OFFSET(tolerance), AV_OPT_TYPE_FLOAT, {.dbl = 0.}, 0, 1, FLAGS }, + { "angle", "set angle threshold for out-of-phase detection", OFFSET(angle), AV_OPT_TYPE_FLOAT, {.dbl = 170.}, 90, 180, FLAGS }, + { "duration", "set minimum mono or out-of-phase duration in seconds", OFFSET(duration), AV_OPT_TYPE_DOUBLE, {.dbl=2.}, 0, 24*60*60, FLAGS }, { NULL } }; @@ -140,6 +159,22 @@ static inline int get_x(float phase, int w) return (phase + 1.) / 2. * (w - 1); } +static inline float get_index(AVFilterLink *inlink, AVFrame *in) +{ + char *index_str = av_ts2timestr(in->pts, &inlink->time_base); + return atof(index_str); +} + +static inline void add_metadata(AVFrame *insamples, const char *key, float value) +{ + char buf[128]; + char str[128]; + + snprintf(str, sizeof(str), "%f", value); + snprintf(buf, sizeof(buf), "lavfi.aphasemeter.%s", key); + av_dict_set(&insamples->metadata, buf, str, 0); +} + static int filter_frame(AVFilterLink *inlink, AVFrame *in) { AVFilterContext *ctx = inlink->dst; @@ -154,6 +189,10 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) AVFrame *out; uint8_t *dst; int i; + int mono_measurement; + int out_phase_measurement; + float tolerance = 1.0f - s->tolerance; + float angle = cosf(s->angle/180.0f*M_PI); if (s->do_video && (!s->out || s->out->width != outlink->w || s->out->height != outlink->h)) { @@ -193,7 +232,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) fphase += phase; } fphase /= in->nb_samples; - + s->phase = fphase; if (s->do_video) { if (s->draw_median_phase) { dst = out->data[0] + get_x(fphase, s->w) * 4; @@ -206,10 +245,64 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) metadata = &in->metadata; if (metadata) { - uint8_t value[128]; + add_metadata(in, "phase", fphase); + } - snprintf(value, sizeof(value), "%f", fphase); - av_dict_set(metadata, "lavfi.aphasemeter.phase", value, 0); + if (s->do_phasing_detection) { + mono_measurement = (tolerance - fphase) < FLT_EPSILON; + out_phase_measurement = (angle - fphase) > FLT_EPSILON; + if (!s->is_mono && mono_measurement) { + s->is_mono = 1; + s->start_mono_presence = 1; + s->mono_idx[0] = get_index(inlink, in); + } + if (s->is_mono && mono_measurement && s->start_mono_presence) { + float mono_duration; + s->mono_idx[1] = get_index(inlink, in); + mono_duration = get_duration(s->mono_idx); + if (mono_duration >= s->duration) { + add_metadata(in, "mono_start", s->mono_idx[0]); + av_log(s, AV_LOG_INFO, "mono_start: %f\n", s->mono_idx[0]); + s->start_mono_presence = 0; + } + } + if (s->is_mono && !mono_measurement) { + float mono_duration; + s->mono_idx[1] = get_index(inlink, in); + mono_duration = get_duration(s->mono_idx); + if (mono_duration > s->duration) { + add_metadata(in, "mono_end", s->mono_idx[1]); + add_metadata(in, "mono_duration", mono_duration); + av_log(s, AV_LOG_INFO, "mono_end: %f | mono_duration: %f\n", s->mono_idx[1], mono_duration); + } + s->is_mono = 0; + } + if (!s->is_out_phase && out_phase_measurement) { + s->out_phase_idx[0] = get_index(inlink, in); + s->is_out_phase = 1; + s->start_out_phase_presence = 1; + } + if (s->is_out_phase && out_phase_measurement && s->start_out_phase_presence) { + float out_phase_duration; + s->out_phase_idx[1] = get_index(inlink, in); + out_phase_duration = get_duration(s->out_phase_idx); + if (out_phase_duration >= s->duration) { + add_metadata(in, "out_phase_start", s->out_phase_idx[0]); + av_log(s, AV_LOG_INFO, "out_phase_start: %f\n", s->out_phase_idx[0]); + s->start_out_phase_presence = 0; + } + } + if (s->is_out_phase && !out_phase_measurement) { + float out_phase_duration; + s->out_phase_idx[1] = get_index(inlink, in); + out_phase_duration = get_duration(s->out_phase_idx); + if (out_phase_duration > s->duration) { + add_metadata(in, "out_phase_end", s->out_phase_idx[1]); + add_metadata(in, "out_phase_duration", out_phase_duration); + av_log(s, AV_LOG_INFO, "out_phase_end: %f | out_phase_duration: %f\n", s->out_phase_idx[1], out_phase_duration); + } + s->is_out_phase = 0; + } } if (s->do_video) { @@ -224,6 +317,32 @@ static av_cold void uninit(AVFilterContext *ctx) AudioPhaseMeterContext *s = ctx->priv; int i; + if (s->do_phasing_detection) { + float tolerance = 1.0f - s->tolerance; + float angle = cosf(s->angle/180.0f*M_PI); + float mono_duration; + float out_phase_duration; + AVFilterLink *inlink = ctx->inputs[0]; + + if (s->is_mono) { + char *index_str = av_ts2timestr(inlink->current_pts, &inlink->time_base); + s->mono_idx[1] = atof(index_str); + mono_duration = get_duration(s->mono_idx); + if (mono_duration > s->duration) { + av_log(s, AV_LOG_INFO, "mono_end: %f | mono_duration: %f\n", s->mono_idx[1], mono_duration); + } + s->is_mono = 0; + } + if (s->is_out_phase) { + char *index_str = av_ts2timestr(inlink->current_pts, &inlink->time_base); + s->out_phase_idx[1] = atof(index_str); + out_phase_duration = get_duration(s->out_phase_idx); + if (out_phase_duration > s->duration) { + av_log(s, AV_LOG_INFO, "out_phase_end: %f | out_phase_duration: %f\n", s->out_phase_idx[1], out_phase_duration); + } + s->is_out_phase = 0; + } + } av_frame_free(&s->out); for (i = 0; i < ctx->nb_outputs; i++) av_freep(&ctx->output_pads[i].name); -- 2.11.0