From patchwork Tue Aug 13 16:01:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Romane Lafon X-Patchwork-Id: 14487 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B5CB3449B12 for ; Tue, 13 Aug 2019 19:07:25 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8930868ABFA; Tue, 13 Aug 2019 19:07:25 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ot1-f41.google.com (mail-ot1-f41.google.com [209.85.210.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A767168A830 for ; Tue, 13 Aug 2019 19:07:18 +0300 (EEST) Received: by mail-ot1-f41.google.com with SMTP id g17so23559651otl.2 for ; Tue, 13 Aug 2019 09:07:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nomalab-com.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=7qNeo7DfcA7F/nawraZEDoHEoJm5zxGpPmTzvhwOTDA=; b=K7cY3ASMfzYdedD97vqA20EurOQPlSvliunk6B/aD5th5IvcQXXljhJEP0+3MCSegI 4/6fhY6rvvOI9Udc4Obrl0PWglRs5U4sOCxgll4fsBk9Lu8n7qRVQG0guB6JaQFoJ2yC /CQHj/hhOircmGvSld8P69C0XIZtBU3mlctDs0rlW2Vrk2Nw1TwNhZZJO3e5h6D53snw feLE+5EqnHICA4CaN9nrsrMUd0H6DdrF0YXjcRck4LJOp4kHCeWbg3jS4K4+MgStDNAd Oevi1oTLQiI8vLtAOAjVSDIcbzqmrMUdJWIe6Icl+po5bVxWawqtDTcCoQNSxc3PKxmI EbGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=7qNeo7DfcA7F/nawraZEDoHEoJm5zxGpPmTzvhwOTDA=; b=VBkjRiIsNCcThASfPtrKEk10SfjS03zwEFw9h4RSw3ASrVgnKUFes9ACqXWO8P8TAx upGfuvGgPCP5uL4p9SKz2QFGOa1hCPsKiY9HKwDTWZ624cXp9mOnB0GcYRHGNGlhbffh uFOzMAqOC0t2m6mfezci2z6jO3hOeO/8tB24eWmnuY6MVu5sAqosPBc4m3dmB4JZFfyR LSmM/qcY4FHS/WmMarL8ERIHgosSOZ09JAX/V2C/ForhspwJXgKaqIsQu5CTUJRx5z89 1Tw/EILO0ZH2xkznmER0NlOL4Smn8z7rw3BFtmD7mGv4PcW/UztOELEfC85JV5irulSO Weiw== X-Gm-Message-State: APjAAAXPGsczKYWskr4vSAg1tjWgzE/XoiZXBbafDbmlHPQGpmgcM83U P12Pm0JtraHAtNSUHYVt4Kp6CR5NK3vMKu5sfE5C4avoyMU= X-Google-Smtp-Source: APXvYqwhLhNIDo+JILOvr2BY4we5AxxeIKgcR06DVeAF5sOdsAIirmY2r8u8lypFnFkki8ZawQhiPSOdRoZVenjD1mw= X-Received: by 2002:a9d:76da:: with SMTP id p26mr36265742otl.311.1565712100255; Tue, 13 Aug 2019 09:01:40 -0700 (PDT) MIME-Version: 1.0 From: Romane Lafon Date: Tue, 13 Aug 2019 18:01:29 +0200 Message-ID: To: FFmpeg development discussions and patches X-Content-Filtered-By: Mailman/MimeDel 2.1.20 Subject: [FFmpeg-devel] [PATCH v6] avfilter/avf_aphasemeter: Add out-of-phase and mono detection X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This patch extends aphasemeter to detect out of phase or mono sequences in stereo streams, with its associated documentation. From f7f627e48282c4aefa5229925454f97747822deb Mon Sep 17 00:00:00 2001 From: Romane Lafon Date: Tue, 13 Aug 2019 17:39:36 +0200 Subject: [PATCH] avfilter/avf_aphasemeter: Add out of phase and mono detection Signed-off-by: Romane Lafon --- doc/filters.texi | 33 +++++++++++ libavfilter/avf_aphasemeter.c | 124 ++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 153 insertions(+), 4 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index e081cdc7bc..d1f272c091 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -20810,6 +20810,39 @@ Set color which will be used for drawing median phase. If color is Enable video output. Default is enabled. @end table +@subsection phasing detection + +The filter also detects out of phase and mono sequences in stereo streams. +It logs the sequence start, end and duration when it lasts longer or as long as the minimum set. + +The filter accepts the following options for this detection: + +@table @option +@item phasing +Enable mono and out of phase detection. Default is disabled. + +@item tolerance +Set phase tolerance for mono detection, in amplitude ratio. Default is @code{0}. +Allowed range is @code{[0, 1]}. + +@item angle +Set angle threshold for out of phase detection, in degree. Default is @code{170}. +Allowed range is @code{[0, 180]}. + +@item duration +Set mono or out of phase duration until notification, expressed in seconds. Default is @code{2}. +@end table + +@subsection Examples + +@itemize +@item +Complete example with @command{ffmpeg} to detect 1 second of mono with 0.001 phase tolerance: +@example +ffmpeg -i stereo.wav -af aphasemeter=video=0:phasing=1:duration=1:tolerance=0.001 -f null - +@end example +@end itemize + @section avectorscope Convert input audio to a video output, representing the audio vector diff --git a/libavfilter/avf_aphasemeter.c b/libavfilter/avf_aphasemeter.c index f497bc9969..60cc153e61 100644 --- a/libavfilter/avf_aphasemeter.c +++ b/libavfilter/avf_aphasemeter.c @@ -28,26 +28,40 @@ #include "libavutil/intreadwrite.h" #include "libavutil/opt.h" #include "libavutil/parseutils.h" +#include "libavutil/timestamp.h" #include "avfilter.h" #include "formats.h" #include "audio.h" #include "video.h" #include "internal.h" +#include "float.h" typedef struct AudioPhaseMeterContext { const AVClass *class; AVFrame *out; int do_video; + int do_phasing_detection; int w, h; AVRational frame_rate; int contrast[4]; uint8_t *mpc_str; uint8_t mpc[4]; int draw_median_phase; + int is_mono; + int is_out_phase; + int start_mono_presence; + int start_out_phase_presence; + float tolerance; + float angle; + float phase; + float mono_idx[2]; + float out_phase_idx[2]; + double duration; } AudioPhaseMeterContext; #define OFFSET(x) offsetof(AudioPhaseMeterContext, x) #define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM +#define get_duration(index) (index[1] - index[0]) static const AVOption aphasemeter_options[] = { { "rate", "set video rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, {.str="25"}, 0, INT_MAX, FLAGS }, @@ -59,6 +73,10 @@ static const AVOption aphasemeter_options[] = { { "bc", "set blue contrast", OFFSET(contrast[2]), AV_OPT_TYPE_INT, {.i64=1}, 0, 255, FLAGS }, { "mpc", "set median phase color", OFFSET(mpc_str), AV_OPT_TYPE_STRING, {.str = "none"}, 0, 0, FLAGS }, { "video", "set video output", OFFSET(do_video), AV_OPT_TYPE_BOOL, {.i64 = 1}, 0, 1, FLAGS }, + { "phasing", "set mono and out-of-phase detection output", OFFSET(do_phasing_detection), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, FLAGS }, + { "tolerance", "set phase tolerance for mono detection", OFFSET(tolerance), AV_OPT_TYPE_FLOAT, {.dbl = 0.}, 0, 1, FLAGS }, + { "angle", "set angle threshold for out-of-phase detection", OFFSET(angle), AV_OPT_TYPE_FLOAT, {.dbl = 170.}, 90, 180, FLAGS }, + { "duration", "set minimum mono or out-of-phase duration in seconds", OFFSET(duration), AV_OPT_TYPE_DOUBLE, {.dbl=2.}, 0, 24*60*60, FLAGS }, { NULL } }; @@ -140,6 +158,22 @@ static inline int get_x(float phase, int w) return (phase + 1.) / 2. * (w - 1); } +static inline float get_index(AVFilterLink *inlink, AVFrame *in) +{ + char *index_str = av_ts2timestr(in->pts, &inlink->time_base); + return atof(index_str); +} + +static inline void add_metadata(AVFrame *insamples, const char *key, float value) +{ + char buf[128]; + char str[128]; + + snprintf(str, sizeof(str), "%f", value); + snprintf(buf, sizeof(buf), "lavfi.aphasemeter.%s", key); + av_dict_set(&insamples->metadata, buf, str, 0); +} + static int filter_frame(AVFilterLink *inlink, AVFrame *in) { AVFilterContext *ctx = inlink->dst; @@ -154,6 +188,10 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) AVFrame *out; uint8_t *dst; int i; + int mono_measurement; + int out_phase_measurement; + float tolerance = 1.0f - s->tolerance; + float angle = cosf(s->angle/180.0f*M_PI); if (s->do_video && (!s->out || s->out->width != outlink->w || s->out->height != outlink->h)) { @@ -193,7 +231,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) fphase += phase; } fphase /= in->nb_samples; - + s->phase = fphase; if (s->do_video) { if (s->draw_median_phase) { dst = out->data[0] + get_x(fphase, s->w) * 4; @@ -206,10 +244,64 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) metadata = &in->metadata; if (metadata) { - uint8_t value[128]; + add_metadata(in, "phase", fphase); + } - snprintf(value, sizeof(value), "%f", fphase); - av_dict_set(metadata, "lavfi.aphasemeter.phase", value, 0); + if (s->do_phasing_detection) { + mono_measurement = (tolerance - fphase) < FLT_EPSILON; + out_phase_measurement = (angle - fphase) > FLT_EPSILON; + if (!s->is_mono && mono_measurement) { + s->is_mono = 1; + s->start_mono_presence = 1; + s->mono_idx[0] = get_index(inlink, in); + } + if (s->is_mono && mono_measurement && s->start_mono_presence) { + float mono_duration; + s->mono_idx[1] = get_index(inlink, in); + mono_duration = get_duration(s->mono_idx); + if (mono_duration >= s->duration) { + add_metadata(in, "mono_start", s->mono_idx[0]); + av_log(s, AV_LOG_INFO, "mono_start: %f\n", s->mono_idx[0]); + s->start_mono_presence = 0; + } + } + if (s->is_mono && !mono_measurement) { + float mono_duration; + s->mono_idx[1] = get_index(inlink, in); + mono_duration = get_duration(s->mono_idx); + if (mono_duration > s->duration) { + add_metadata(in, "mono_end", s->mono_idx[1]); + add_metadata(in, "mono_duration", mono_duration); + av_log(s, AV_LOG_INFO, "mono_end: %f | mono_duration: %f\n", s->mono_idx[1], mono_duration); + } + s->is_mono = 0; + } + if (!s->is_out_phase && out_phase_measurement) { + s->out_phase_idx[0] = get_index(inlink, in); + s->is_out_phase = 1; + s->start_out_phase_presence = 1; + } + if (s->is_out_phase && out_phase_measurement && s->start_out_phase_presence) { + float out_phase_duration; + s->out_phase_idx[1] = get_index(inlink, in); + out_phase_duration = get_duration(s->out_phase_idx); + if (out_phase_duration >= s->duration) { + add_metadata(in, "out_phase_start", s->out_phase_idx[0]); + av_log(s, AV_LOG_INFO, "out_phase_start: %f\n", s->out_phase_idx[0]); + s->start_out_phase_presence = 0; + } + } + if (s->is_out_phase && !out_phase_measurement) { + float out_phase_duration; + s->out_phase_idx[1] = get_index(inlink, in); + out_phase_duration = get_duration(s->out_phase_idx); + if (out_phase_duration > s->duration) { + add_metadata(in, "out_phase_end", s->out_phase_idx[1]); + add_metadata(in, "out_phase_duration", out_phase_duration); + av_log(s, AV_LOG_INFO, "out_phase_end: %f | out_phase_duration: %f\n", s->out_phase_idx[1], out_phase_duration); + } + s->is_out_phase = 0; + } } if (s->do_video) { @@ -224,6 +316,30 @@ static av_cold void uninit(AVFilterContext *ctx) AudioPhaseMeterContext *s = ctx->priv; int i; + if (s->do_phasing_detection) { + float mono_duration; + float out_phase_duration; + AVFilterLink *inlink = ctx->inputs[0]; + + if (s->is_mono) { + char *index_str = av_ts2timestr(inlink->current_pts, &inlink->time_base); + s->mono_idx[1] = atof(index_str); + mono_duration = get_duration(s->mono_idx); + if (mono_duration > s->duration) { + av_log(s, AV_LOG_INFO, "mono_end: %f | mono_duration: %f\n", s->mono_idx[1], mono_duration); + } + s->is_mono = 0; + } + if (s->is_out_phase) { + char *index_str = av_ts2timestr(inlink->current_pts, &inlink->time_base); + s->out_phase_idx[1] = atof(index_str); + out_phase_duration = get_duration(s->out_phase_idx); + if (out_phase_duration > s->duration) { + av_log(s, AV_LOG_INFO, "out_phase_end: %f | out_phase_duration: %f\n", s->out_phase_idx[1], out_phase_duration); + } + s->is_out_phase = 0; + } + } av_frame_free(&s->out); for (i = 0; i < ctx->nb_outputs; i++) av_freep(&ctx->output_pads[i].name); -- 2.11.0