From patchwork Mon Aug 19 15:37:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Romane Lafon X-Patchwork-Id: 14587 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id EC3DA44886D for ; Mon, 19 Aug 2019 18:38:07 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C13B06899FD; Mon, 19 Aug 2019 18:38:07 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ot1-f48.google.com (mail-ot1-f48.google.com [209.85.210.48]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 449E9689989 for ; Mon, 19 Aug 2019 18:38:01 +0300 (EEST) Received: by mail-ot1-f48.google.com with SMTP id w4so2045839ote.11 for ; Mon, 19 Aug 2019 08:38:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nomalab-com.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=XOuvsB1vxVlvrJC1/tMC++qdl7myXt0OGybtMAAkxjI=; b=Yq9n/00zXXXYbxAuwc+0v2AEwhjfesR174+0mOlbpD0LLBcmDuCqiK9WXSvowpdYh7 ynpCKeVsAdKXJZOo/glACEQ7PrhFkZqPZKdZprDwppPO4L/uTcPl/GVx5HUF0Q2k8EG1 KmRU86oLAKT4+zZjKh/bkhlEOrS3mjXxKow46lSzDlmWKgmnDIw+oFmoI7ERrh+kjJAy a8MPdxf+DV5e4belrGsS7fnre063ocZiZ3gZIWBHh3rasZJFAE4XyQ6aVmRb+IDdYura np7XRYWWavi++5q+8QLwBH6fOzQ7zu6HQFQPYUKHC3Nk5FmVVAmz3OU6zXn/nldKaDz3 r9+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=XOuvsB1vxVlvrJC1/tMC++qdl7myXt0OGybtMAAkxjI=; b=FDC87j3aTYhE4m7EZ0KOFcp4JEPg87dSR9wE2phscXDiLA1O6DNS1+rjlPN3oRisg5 +D2zoji4tOcYOEa7JK8njJx9ov3SlPV9i5JIsq3sZafBqc7C+en0V1ezNRahtChlwMom a7myGp2rzhwJvyaISXPNqVwXpMfzNoHhixW7ZV5kqGZKdNIbLrfFTAkpauSDB3+3gIpd XsmuLbFlgL8MNo1zi4IisifDsKitSPD7+b6RGSChvYdlh7WA9Q0Nu6dajhEQzR723uzh 5B/+MZw56CDoO/mkpGEmMHMHPOr6Vc++20tYJXBQNdBi//avMnq4yo8h7GIwjFcbVBov SNOg== X-Gm-Message-State: APjAAAUvJQOgWxe7Qu1NxGXzCIcXXEqic0yKF4N5EpxHjzSOUN1km2OK 3NEht6Egm7LPbX+br5UzIi0iXBA3FxjxrQfIj22W69mASIA= X-Google-Smtp-Source: APXvYqwT/jHhkTejMAk9ic83OyJPDRQ3dho3vKcPCX8gDNwA+M4ntCoTX9BMuQaaIhHsQQkkQmpUi6jwrTA2q+FV+Do= X-Received: by 2002:a05:6830:158:: with SMTP id j24mr17968458otp.236.1566229079370; Mon, 19 Aug 2019 08:37:59 -0700 (PDT) MIME-Version: 1.0 From: Romane Lafon Date: Mon, 19 Aug 2019 17:37:48 +0200 Message-ID: To: FFmpeg development discussions and patches X-Content-Filtered-By: Mailman/MimeDel 2.1.20 Subject: [FFmpeg-devel] [PATCH v7] avfilter/avf_aphasemeter: Add out-of-phase and mono detection X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This patch extends aphasemeter to detect out of phase or mono sequences in stereo streams, with its associated documentation. From dce30e5a978ff5296fa1c0013bbd78fd27ecec2a Mon Sep 17 00:00:00 2001 From: Romane Lafon Date: Fri, 16 Aug 2019 14:14:11 +0200 Subject: [PATCH] avfilter/avf_aphasemeter: Add out-of-phase and mono detection Signed-off-by: Romane Lafon --- doc/filters.texi | 33 ++++++++++ libavfilter/avf_aphasemeter.c | 121 ++++++++++++++++++++++++++++++++-- 2 files changed, 150 insertions(+), 4 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 01262d845e..5acd36a98c 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -20822,6 +20822,39 @@ Set color which will be used for drawing median phase. If color is Enable video output. Default is enabled. @end table +@subsection phasing detection + +The filter also detects out of phase and mono sequences in stereo streams. +It logs the sequence start, end and duration when it lasts longer or as long as the minimum set. + +The filter accepts the following options for this detection: + +@table @option +@item phasing +Enable mono and out of phase detection. Default is disabled. + +@item tolerance +Set phase tolerance for mono detection, in amplitude ratio. Default is @code{0}. +Allowed range is @code{[0, 1]}. + +@item angle +Set angle threshold for out of phase detection, in degree. Default is @code{170}. +Allowed range is @code{[0, 180]}. + +@item duration +Set mono or out of phase duration until notification, expressed in seconds. Default is @code{2}. +@end table + +@subsection Examples + +@itemize +@item +Complete example with @command{ffmpeg} to detect 1 second of mono with 0.001 phase tolerance: +@example +ffmpeg -i stereo.wav -af aphasemeter=video=0:phasing=1:duration=1:tolerance=0.001 -f null - +@end example +@end itemize + @section avectorscope Convert input audio to a video output, representing the audio vector diff --git a/libavfilter/avf_aphasemeter.c b/libavfilter/avf_aphasemeter.c index f497bc9969..b94cbbade6 100644 --- a/libavfilter/avf_aphasemeter.c +++ b/libavfilter/avf_aphasemeter.c @@ -28,26 +28,40 @@ #include "libavutil/intreadwrite.h" #include "libavutil/opt.h" #include "libavutil/parseutils.h" +#include "libavutil/timestamp.h" #include "avfilter.h" #include "formats.h" #include "audio.h" #include "video.h" #include "internal.h" +#include "float.h" typedef struct AudioPhaseMeterContext { const AVClass *class; AVFrame *out; int do_video; + int do_phasing_detection; int w, h; AVRational frame_rate; int contrast[4]; uint8_t *mpc_str; uint8_t mpc[4]; int draw_median_phase; + int is_mono; + int is_out_phase; + int start_mono_presence; + int start_out_phase_presence; + float tolerance; + float angle; + float phase; + float mono_idx[2]; + float out_phase_idx[2]; + double duration; } AudioPhaseMeterContext; #define OFFSET(x) offsetof(AudioPhaseMeterContext, x) #define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM +#define get_duration(index) (index[1] - index[0]) static const AVOption aphasemeter_options[] = { { "rate", "set video rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, {.str="25"}, 0, INT_MAX, FLAGS }, @@ -59,6 +73,10 @@ static const AVOption aphasemeter_options[] = { { "bc", "set blue contrast", OFFSET(contrast[2]), AV_OPT_TYPE_INT, {.i64=1}, 0, 255, FLAGS }, { "mpc", "set median phase color", OFFSET(mpc_str), AV_OPT_TYPE_STRING, {.str = "none"}, 0, 0, FLAGS }, { "video", "set video output", OFFSET(do_video), AV_OPT_TYPE_BOOL, {.i64 = 1}, 0, 1, FLAGS }, + { "phasing", "set mono and out-of-phase detection output", OFFSET(do_phasing_detection), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, FLAGS }, + { "tolerance", "set phase tolerance for mono detection", OFFSET(tolerance), AV_OPT_TYPE_FLOAT, {.dbl = 0.}, 0, 1, FLAGS }, + { "angle", "set angle threshold for out-of-phase detection", OFFSET(angle), AV_OPT_TYPE_FLOAT, {.dbl = 170.}, 90, 180, FLAGS }, + { "duration", "set minimum mono or out-of-phase duration in seconds", OFFSET(duration), AV_OPT_TYPE_DOUBLE, {.dbl=2.}, 0, 24*60*60, FLAGS }, { NULL } }; @@ -140,6 +158,21 @@ static inline int get_x(float phase, int w) return (phase + 1.) / 2. * (w - 1); } +static inline float get_index(AVFilterLink *inlink, int64_t pts) +{ + return atof(av_ts2timestr(pts, &inlink->time_base)); +} + +static inline void add_metadata(AVFrame *insamples, const char *key, float value) +{ + char buf[128]; + char str[128]; + + snprintf(str, sizeof(str), "%f", value); + snprintf(buf, sizeof(buf), "lavfi.aphasemeter.%s", key); + av_dict_set(&insamples->metadata, buf, str, 0); +} + static int filter_frame(AVFilterLink *inlink, AVFrame *in) { AVFilterContext *ctx = inlink->dst; @@ -154,6 +187,10 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) AVFrame *out; uint8_t *dst; int i; + int mono_measurement; + int out_phase_measurement; + float tolerance = 1.0f - s->tolerance; + float angle = cosf(s->angle/180.0f*M_PI); if (s->do_video && (!s->out || s->out->width != outlink->w || s->out->height != outlink->h)) { @@ -193,7 +230,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) fphase += phase; } fphase /= in->nb_samples; - + s->phase = fphase; if (s->do_video) { if (s->draw_median_phase) { dst = out->data[0] + get_x(fphase, s->w) * 4; @@ -206,10 +243,64 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) metadata = &in->metadata; if (metadata) { - uint8_t value[128]; + add_metadata(in, "phase", fphase); + } - snprintf(value, sizeof(value), "%f", fphase); - av_dict_set(metadata, "lavfi.aphasemeter.phase", value, 0); + if (s->do_phasing_detection) { + mono_measurement = (tolerance - fphase) < FLT_EPSILON; + out_phase_measurement = (angle - fphase) > FLT_EPSILON; + if (!s->is_mono && mono_measurement) { + s->is_mono = 1; + s->start_mono_presence = 1; + s->mono_idx[0] = get_index(inlink, in->pts); + } + if (s->is_mono && mono_measurement && s->start_mono_presence) { + float mono_duration; + s->mono_idx[1] = get_index(inlink, in->pts); + mono_duration = get_duration(s->mono_idx); + if (mono_duration >= s->duration) { + add_metadata(in, "mono_start", s->mono_idx[0]); + av_log(s, AV_LOG_INFO, "mono_start: %f\n", s->mono_idx[0]); + s->start_mono_presence = 0; + } + } + if (s->is_mono && !mono_measurement) { + float mono_duration; + s->mono_idx[1] = get_index(inlink, in->pts); + mono_duration = get_duration(s->mono_idx); + if (mono_duration > s->duration) { + add_metadata(in, "mono_end", s->mono_idx[1]); + add_metadata(in, "mono_duration", mono_duration); + av_log(s, AV_LOG_INFO, "mono_end: %f | mono_duration: %f\n", s->mono_idx[1], mono_duration); + } + s->is_mono = 0; + } + if (!s->is_out_phase && out_phase_measurement) { + s->out_phase_idx[0] = get_index(inlink, in->pts); + s->is_out_phase = 1; + s->start_out_phase_presence = 1; + } + if (s->is_out_phase && out_phase_measurement && s->start_out_phase_presence) { + float out_phase_duration; + s->out_phase_idx[1] = get_index(inlink, in->pts); + out_phase_duration = get_duration(s->out_phase_idx); + if (out_phase_duration >= s->duration) { + add_metadata(in, "out_phase_start", s->out_phase_idx[0]); + av_log(s, AV_LOG_INFO, "out_phase_start: %f\n", s->out_phase_idx[0]); + s->start_out_phase_presence = 0; + } + } + if (s->is_out_phase && !out_phase_measurement) { + float out_phase_duration; + s->out_phase_idx[1] = get_index(inlink, in->pts); + out_phase_duration = get_duration(s->out_phase_idx); + if (out_phase_duration > s->duration) { + add_metadata(in, "out_phase_end", s->out_phase_idx[1]); + add_metadata(in, "out_phase_duration", out_phase_duration); + av_log(s, AV_LOG_INFO, "out_phase_end: %f | out_phase_duration: %f\n", s->out_phase_idx[1], out_phase_duration); + } + s->is_out_phase = 0; + } } if (s->do_video) { @@ -224,6 +315,28 @@ static av_cold void uninit(AVFilterContext *ctx) AudioPhaseMeterContext *s = ctx->priv; int i; + if (s->do_phasing_detection) { + float mono_duration; + float out_phase_duration; + AVFilterLink *inlink = ctx->inputs[0]; + + if (s->is_mono) { + s->mono_idx[1] = get_index(inlink, inlink->current_pts); + mono_duration = get_duration(s->mono_idx); + if (mono_duration > s->duration) { + av_log(s, AV_LOG_INFO, "mono_end: %f | mono_duration: %f\n", s->mono_idx[1], mono_duration); + } + s->is_mono = 0; + } + if (s->is_out_phase) { + s->out_phase_idx[1] = get_index(inlink, inlink->current_pts); + out_phase_duration = get_duration(s->out_phase_idx); + if (out_phase_duration > s->duration) { + av_log(s, AV_LOG_INFO, "out_phase_end: %f | out_phase_duration: %f\n", s->out_phase_idx[1], out_phase_duration); + } + s->is_out_phase = 0; + } + } av_frame_free(&s->out); for (i = 0; i < ctx->nb_outputs; i++) av_freep(&ctx->output_pads[i].name); -- 2.20.1