From patchwork Mon Jul 8 12:00:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Romane Lafon X-Patchwork-Id: 13856 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B0A9B449C5F for ; Mon, 8 Jul 2019 15:01:10 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8620A68B02F; Mon, 8 Jul 2019 15:01:10 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oi1-f170.google.com (mail-oi1-f170.google.com [209.85.167.170]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 411D268B02A for ; Mon, 8 Jul 2019 15:01:04 +0300 (EEST) Received: by mail-oi1-f170.google.com with SMTP id e189so12303706oib.11 for ; Mon, 08 Jul 2019 05:01:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nomalab-com.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=DJgCwSMNT+LYD9zv9pfEsCiV4JdAd1vELk1aYJNa+AA=; b=NOe/lnNIPLbs7EhctZSXCrwW+CBhRi6YW3gRoisRuBkPS+HBR/aThmcdVHa5Raw1KN d9JI5/Htj2HuHAOAIVS9lYu0zl7IcyKZ6gJaHIHeY+4JlNgWbZfWFLZ/4V3H3mImrIH1 ZvGQVrVAYPtCwYVWYoFWizQPFBd7vVFC65vNlpadvKXXZwKXNUQa+e47hEwf3S3ejerq t4PwJ6Zdr1syblxdCw0eU6t8AIqYR14XMCPaTqg8MAlOxkBR/bVQKaTgaP/fqZAYhGEv pXQC2x59aEmbU/Tk89uovC5QO+S9kuzNCwH1wy8BvY+ksop16NDB4xGZ/q0WsnR8VER9 yFzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=DJgCwSMNT+LYD9zv9pfEsCiV4JdAd1vELk1aYJNa+AA=; b=f9/inxzWcHu6xT+cHQeZDsC5ABkpUJhYnifrpbWIwR0ueSvRfmOOTmNCQhIvjRNX16 Ns7T6Q7uwYkgXJQMxRyNz3HFAaBIhOmrJeE6MkUIerSaYtWx6PF3qv3q+pbUNvjQUaza OqPyfZzd4y7Wpep/dQ82fZDIb2QlKZuYH+/Gxi6Ot+38q7vWQlKrhaoaGF6+K91UGX6q MmjScyx/+s8L0zfoJROdz/ki484n8r1iMkFsycZ58dofnvUFUBv7poHc3moCShwRt8G7 YRCiqq1PSfTziw9JRmb84Ef/2vykPW36FUJDUSVCRB5Uaiqi6Z0M7zAyVMADDbI17Q4F PwEw== X-Gm-Message-State: APjAAAWIGLILGCMMyBa4YjZOmTW5J9aXTMjHcquYu/0TGOtoGog2k5YX R6ADv6CK6FEl/GJRqjEez3k/+scVO5DnBvrItueTG+2V X-Google-Smtp-Source: APXvYqzdOjiVLgJyeD7NV4uc9Y3sP+cI6GwKL6kr6g0NeK4a5kZ0we0KeoE8ATtWVQzleS7tnz/9WAFVUVbzHLJaXms= X-Received: by 2002:aca:1304:: with SMTP id e4mr9049603oii.149.1562587262444; Mon, 08 Jul 2019 05:01:02 -0700 (PDT) MIME-Version: 1.0 From: Romane Lafon Date: Mon, 8 Jul 2019 14:00:51 +0200 Message-ID: To: FFmpeg development discussions and patches X-Content-Filtered-By: Mailman/MimeDel 2.1.20 Subject: [FFmpeg-devel] [PATCH v5] avfilter/avf_aphasemeter: Add out-of-phase and mono detection X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This patch extends aphasemeter to detect out of phase or mono sequences in stereo streams, with its associated documentation. From 7e2846ac3b4b79a2e9beca845d0f2be7ae6abcdb Mon Sep 17 00:00:00 2001 From: Romane Lafon Date: Mon, 8 Jul 2019 13:51:05 +0200 Subject: [PATCH] avfilter/avf_aphasemeter: Add out of phase and mono detection Signed-off-by: Romane Lafon --- doc/filters.texi | 33 +++++++++++ libavfilter/avf_aphasemeter.c | 127 ++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 156 insertions(+), 4 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index ee6a93ffbf..46d19826d0 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -20672,6 +20672,39 @@ Set color which will be used for drawing median phase. If color is Enable video output. Default is enabled. @end table +@subsection phasing detection + +The filter also detects out of phase and mono sequences in stereo streams. +It logs the sequence start, end and duration when it lasts longer or as long as the minimum set. + +The filter accepts the following options for this detection: + +@table @option +@item phasing +Enable mono and out of phase detection. Default is disabled. + +@item tolerance +Set phase tolerance for mono detection, in amplitude ratio. Default is @code{0}. +Allowed range is @code{[0, 1]}. + +@item angle +Set angle threshold for out of phase detection, in degree. Default is @code{170}. +Allowed range is @code{[0, 180]}. + +@item duration +Set mono or out of phase duration until notification, expressed in seconds. Default is @code{2}. +@end table + +@subsection Examples + +@itemize +@item +Complete example with @command{ffmpeg} to detect 1 second of mono with 0.001 phase tolerance: +@example +ffmpeg -i stereo.wav -af aphasemeter=video=0:phasing=1:duration=1:tolerance=0.001 -f null - +@end example +@end itemize + @section avectorscope Convert input audio to a video output, representing the audio vector diff --git a/libavfilter/avf_aphasemeter.c b/libavfilter/avf_aphasemeter.c index f497bc9969..77701e5cde 100644 --- a/libavfilter/avf_aphasemeter.c +++ b/libavfilter/avf_aphasemeter.c @@ -28,26 +28,41 @@ #include "libavutil/intreadwrite.h" #include "libavutil/opt.h" #include "libavutil/parseutils.h" +#include "libavutil/timestamp.h" #include "avfilter.h" #include "formats.h" #include "audio.h" #include "video.h" #include "internal.h" +#include "stdbool.h" +#include "float.h" typedef struct AudioPhaseMeterContext { const AVClass *class; AVFrame *out; int do_video; + int do_phasing_detection; int w, h; AVRational frame_rate; int contrast[4]; uint8_t *mpc_str; uint8_t mpc[4]; int draw_median_phase; + int is_mono; + int is_out_phase; + int start_mono_presence; + int start_out_phase_presence; + float tolerance; + float angle; + float phase; + float mono_idx[2]; + float out_phase_idx[2]; + double duration; } AudioPhaseMeterContext; #define OFFSET(x) offsetof(AudioPhaseMeterContext, x) #define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM +#define get_duration(index) (index[1] - index[0]) static const AVOption aphasemeter_options[] = { { "rate", "set video rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, {.str="25"}, 0, INT_MAX, FLAGS }, @@ -59,6 +74,10 @@ static const AVOption aphasemeter_options[] = { { "bc", "set blue contrast", OFFSET(contrast[2]), AV_OPT_TYPE_INT, {.i64=1}, 0, 255, FLAGS }, { "mpc", "set median phase color", OFFSET(mpc_str), AV_OPT_TYPE_STRING, {.str = "none"}, 0, 0, FLAGS }, { "video", "set video output", OFFSET(do_video), AV_OPT_TYPE_BOOL, {.i64 = 1}, 0, 1, FLAGS }, + { "phasing", "set mono and out-of-phase detection output", OFFSET(do_phasing_detection), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, FLAGS }, + { "tolerance", "set phase tolerance for mono detection", OFFSET(tolerance), AV_OPT_TYPE_FLOAT, {.dbl = 0.}, 0, 1, FLAGS }, + { "angle", "set angle threshold for out-of-phase detection", OFFSET(angle), AV_OPT_TYPE_FLOAT, {.dbl = 170.}, 90, 180, FLAGS }, + { "duration", "set minimum mono or out-of-phase duration in seconds", OFFSET(duration), AV_OPT_TYPE_DOUBLE, {.dbl=2.}, 0, 24*60*60, FLAGS }, { NULL } }; @@ -140,6 +159,22 @@ static inline int get_x(float phase, int w) return (phase + 1.) / 2. * (w - 1); } +static inline float get_index(AVFilterLink *inlink, AVFrame *in) +{ + char *index_str = av_ts2timestr(in->pts, &inlink->time_base); + return atof(index_str); +} + +static inline void add_metadata(AVFrame *insamples, const char *key, float value) +{ + char buf[128]; + char str[128]; + + snprintf(str, sizeof(str), "%f", value); + snprintf(buf, sizeof(buf), "lavfi.aphasemeter.%s", key); + av_dict_set(&insamples->metadata, buf, str, 0); +} + static int filter_frame(AVFilterLink *inlink, AVFrame *in) { AVFilterContext *ctx = inlink->dst; @@ -154,6 +189,10 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) AVFrame *out; uint8_t *dst; int i; + int mono_measurement; + int out_phase_measurement; + float tolerance = 1.0f - s->tolerance; + float angle = cosf(s->angle/180.0f*M_PI); if (s->do_video && (!s->out || s->out->width != outlink->w || s->out->height != outlink->h)) { @@ -193,7 +232,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) fphase += phase; } fphase /= in->nb_samples; - + s->phase = fphase; if (s->do_video) { if (s->draw_median_phase) { dst = out->data[0] + get_x(fphase, s->w) * 4; @@ -206,10 +245,64 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) metadata = &in->metadata; if (metadata) { - uint8_t value[128]; + add_metadata(in, "phase", fphase); + } - snprintf(value, sizeof(value), "%f", fphase); - av_dict_set(metadata, "lavfi.aphasemeter.phase", value, 0); + if (s->do_phasing_detection) { + mono_measurement = (tolerance - fphase) < FLT_EPSILON; + out_phase_measurement = (angle - fphase) > FLT_EPSILON; + if (!s->is_mono && mono_measurement) { + s->is_mono = 1; + s->start_mono_presence = 1; + s->mono_idx[0] = get_index(inlink, in); + } + if (s->is_mono && mono_measurement && s->start_mono_presence) { + float mono_duration; + s->mono_idx[1] = get_index(inlink, in); + mono_duration = get_duration(s->mono_idx); + if (mono_duration >= s->duration) { + add_metadata(in, "mono_start", s->mono_idx[0]); + av_log(s, AV_LOG_INFO, "mono_start: %f\n", s->mono_idx[0]); + s->start_mono_presence = 0; + } + } + if (s->is_mono && !mono_measurement) { + float mono_duration; + s->mono_idx[1] = get_index(inlink, in); + mono_duration = get_duration(s->mono_idx); + if (mono_duration > s->duration) { + add_metadata(in, "mono_end", s->mono_idx[1]); + add_metadata(in, "mono_duration", mono_duration); + av_log(s, AV_LOG_INFO, "mono_end: %f | mono_duration: %f\n", s->mono_idx[1], mono_duration); + } + s->is_mono = 0; + } + if (!s->is_out_phase && out_phase_measurement) { + s->out_phase_idx[0] = get_index(inlink, in); + s->is_out_phase = 1; + s->start_out_phase_presence = 1; + } + if (s->is_out_phase && out_phase_measurement && s->start_out_phase_presence) { + float out_phase_duration; + s->out_phase_idx[1] = get_index(inlink, in); + out_phase_duration = get_duration(s->out_phase_idx); + if (out_phase_duration >= s->duration) { + add_metadata(in, "out_phase_start", s->out_phase_idx[0]); + av_log(s, AV_LOG_INFO, "out_phase_start: %f\n", s->out_phase_idx[0]); + s->start_out_phase_presence = 0; + } + } + if (s->is_out_phase && !out_phase_measurement) { + float out_phase_duration; + s->out_phase_idx[1] = get_index(inlink, in); + out_phase_duration = get_duration(s->out_phase_idx); + if (out_phase_duration > s->duration) { + add_metadata(in, "out_phase_end", s->out_phase_idx[1]); + add_metadata(in, "out_phase_duration", out_phase_duration); + av_log(s, AV_LOG_INFO, "out_phase_end: %f | out_phase_duration: %f\n", s->out_phase_idx[1], out_phase_duration); + } + s->is_out_phase = 0; + } } if (s->do_video) { @@ -224,6 +317,32 @@ static av_cold void uninit(AVFilterContext *ctx) AudioPhaseMeterContext *s = ctx->priv; int i; + if (s->do_phasing_detection) { + float tolerance = 1.0f - s->tolerance; + float angle = cosf(s->angle/180.0f*M_PI); + float mono_duration; + float out_phase_duration; + AVFilterLink *inlink = ctx->inputs[0]; + + if (s->is_mono) { + char *index_str = av_ts2timestr(inlink->current_pts, &inlink->time_base); + s->mono_idx[1] = atof(index_str); + mono_duration = get_duration(s->mono_idx); + if (mono_duration > s->duration) { + av_log(s, AV_LOG_INFO, "mono_end: %f | mono_duration: %f\n", s->mono_idx[1], mono_duration); + } + s->is_mono = 0; + } + if (s->is_out_phase) { + char *index_str = av_ts2timestr(inlink->current_pts, &inlink->time_base); + s->out_phase_idx[1] = atof(index_str); + out_phase_duration = get_duration(s->out_phase_idx); + if (out_phase_duration > s->duration) { + av_log(s, AV_LOG_INFO, "out_phase_end: %f | out_phase_duration: %f\n", s->out_phase_idx[1], out_phase_duration); + } + s->is_out_phase = 0; + } + } av_frame_free(&s->out); for (i = 0; i < ctx->nb_outputs; i++) av_freep(&ctx->output_pads[i].name); -- 2.11.0