From patchwork Mon May 20 12:40:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Romane Lafon X-Patchwork-Id: 13200 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id AD4EC447B47 for ; Mon, 20 May 2019 15:40:43 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 896BA68A831; Mon, 20 May 2019 15:40:43 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ot1-f54.google.com (mail-ot1-f54.google.com [209.85.210.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DE09A68A7CD for ; Mon, 20 May 2019 15:40:36 +0300 (EEST) Received: by mail-ot1-f54.google.com with SMTP id s19so12805790otq.5 for ; Mon, 20 May 2019 05:40:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nomalab-com.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=LI7l6+jBCfVBTN3y1lyNiupHQO52dZcmqZZiVqzsXCg=; b=ifDR+fdYx0ThWUo4HwWM2w8ByIYNOVUC6CcXCV4QbK2WqloR0vaUr042ko19a7Mwut GELOSrraAQASNGNS1nEdwjcv6LlQqjYQ2cslLTO524bs7sDhCLsXrhsi0cFyPrWvrK1G Iv/fa3feJ9hPPwTadKPa27FouHVuTayKvkPMPUGmZDEhCyik/IY3gQO9qZISQ6Rmhj6D Gd792rm479EnGqJpxRARucx85kqIgoBZ417ly55zYC+vA3Q6cz2aUC1SLs5cZXHrYnYh sHnDvlYcGz8WgoeTzspgBXSUZk92kjF3byAniNZClYuuHzKmTreIl//FOwGK8I75wvVk PDaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=LI7l6+jBCfVBTN3y1lyNiupHQO52dZcmqZZiVqzsXCg=; b=UK5+QAawZomRkoE0IQxtnDEqAgQ1nsx9jDH5vwNk21MnZa1+brgKkMCR6LMpcWt4/W Iv9Gm9R9mrXGjEgS2eMf9aZQ4iA8oLsL8AovuQe24cdYZb/yjz/ixnrpp5WDMTWHXYJA LZJ+OjuL0bvPPDqrgv4jaY+188r3tjwJ8uAF76UcN1sQxDXjzK8vK1NsLL45ASnxocq0 jR4vHFyMobusyX2eh1+GwhGrEmoJmyPiLvIeKyxUSqfjOmu4nqBiU5fulddGt9chTTW5 fbYDK7jI8Hhy2437GunrS7tGccJvIbKNF5KvAfHqkl3qzbrnvzyNJbBaxmDS52lFEx2h aT0g== X-Gm-Message-State: APjAAAUUj0LI8g0CO4tOgrFyDpbJZ8Rz/GFOleBUxY6o/uNH7yYfWLgx B9AFkul0mXLgVwZLePAq162PRkELOhO22Yih6RydVnv9 X-Google-Smtp-Source: APXvYqzcGjOcer+0iVYnNQmQHeGwNS8vpA+n2WHnxT2MhoffWmmMjv8Ryh+O3KhSu/Wmy/Lv2xI+kLwaBjGjeo91jYU= X-Received: by 2002:a9d:469e:: with SMTP id z30mr23887233ote.311.1558356035277; Mon, 20 May 2019 05:40:35 -0700 (PDT) MIME-Version: 1.0 From: Romane Lafon Date: Mon, 20 May 2019 14:40:24 +0200 Message-ID: To: ffmpeg-devel@ffmpeg.org X-Content-Filtered-By: Mailman/MimeDel 2.1.20 Subject: [FFmpeg-devel] [PATCH v2] avfilter/avf_aphasemeter: Add out-of-phase and mono detection X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" I have updated the patch that extends aphasemeter filter. It allows to get metadata for out-of-phase or mono sequences of stereo streams. It displays start, end and duration as for silencedetect filter. From 08daf8a0394da0d482ee09f29d7cefa14b5c5440 Mon Sep 17 00:00:00 2001 From: Romane Lafon Date: Mon, 20 May 2019 13:28:21 +0200 Subject: [PATCH] avfilter/avf_aphasemeter: Add out of phase and mono detection Signed-off-by: Romane Lafon --- libavfilter/avf_aphasemeter.c | 124 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 120 insertions(+), 4 deletions(-) diff --git a/libavfilter/avf_aphasemeter.c b/libavfilter/avf_aphasemeter.c index ed837059ea..daa15b46a9 100644 --- a/libavfilter/avf_aphasemeter.c +++ b/libavfilter/avf_aphasemeter.c @@ -28,26 +28,42 @@ #include "libavutil/intreadwrite.h" #include "libavutil/opt.h" #include "libavutil/parseutils.h" +#include "libavutil/timestamp.h" #include "avfilter.h" #include "formats.h" #include "audio.h" #include "video.h" #include "internal.h" +#include "stdbool.h" +#include "float.h" typedef struct AudioPhaseMeterContext { const AVClass *class; AVFrame *out; int do_video; + int do_phasing_detection; int w, h; AVRational frame_rate; int contrast[4]; uint8_t *mpc_str; uint8_t mpc[4]; int draw_median_phase; + int is_mono; + int is_out_phase; + int start_mono_presence; + int start_out_phase_presence; + float tolerance; + float angle; + float phase; + float mono_idx[2]; + float out_phase_idx[2]; + double duration; } AudioPhaseMeterContext; #define OFFSET(x) offsetof(AudioPhaseMeterContext, x) #define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM +#define PI 3.14159265359 +#define get_duration(index) (index[1] - index[0]) static const AVOption aphasemeter_options[] = { { "rate", "set video rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, {.str="25"}, 0, INT_MAX, FLAGS }, @@ -59,6 +75,10 @@ static const AVOption aphasemeter_options[] = { { "bc", "set blue contrast", OFFSET(contrast[2]), AV_OPT_TYPE_INT, {.i64=1}, 0, 255, FLAGS }, { "mpc", "set median phase color", OFFSET(mpc_str), AV_OPT_TYPE_STRING, {.str = "none"}, 0, 0, FLAGS }, { "video", "set video output", OFFSET(do_video), AV_OPT_TYPE_BOOL, {.i64 = 1}, 0, 1, FLAGS }, + { "phasing", "set mono and out-of-phase detection output", OFFSET(do_phasing_detection), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, FLAGS }, + { "tolerance", "set phase tolerance for mono detection", OFFSET(tolerance), AV_OPT_TYPE_FLOAT, {.dbl = 0.}, 0, 1, FLAGS }, + { "angle", "set angle threshold for out-of-phase detection", OFFSET(angle), AV_OPT_TYPE_FLOAT, {.dbl = 170.}, 90, 180, FLAGS }, + { "duration", "set minimum mono or out-of-phase duration in seconds", OFFSET(duration), AV_OPT_TYPE_DOUBLE, {.dbl=2.}, 0, 24*60*60, FLAGS }, { NULL } }; @@ -140,6 +160,22 @@ static inline int get_x(float phase, int w) return (phase + 1.) / 2. * (w - 1); } +static inline float get_index(AVFilterLink *inlink, AVFrame *in) +{ + char *index_str = av_ts2timestr(in->pts, &inlink->time_base); + return atof(index_str); +} + +static inline void add_metadata(AVFrame *insamples, const char *key, float value) +{ + char buf[128]; + char str[128]; + + snprintf(str, sizeof(str), "%f", value); + snprintf(buf, sizeof(buf), "lavfi.aphasemeter.%s", key); + av_dict_set(&insamples->metadata, buf, str, 0); +} + static int filter_frame(AVFilterLink *inlink, AVFrame *in) { AVFilterContext *ctx = inlink->dst; @@ -154,6 +190,8 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) AVFrame *out; uint8_t *dst; int i; + float tolerance = 1.0 - s->tolerance; + float angle = cos(s->angle/180.0*PI); if (s->do_video && (!s->out || s->out->width != outlink->w || s->out->height != outlink->h)) { @@ -193,7 +231,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) fphase += phase; } fphase /= in->nb_samples; - + s->phase = fphase; if (s->do_video) { if (s->draw_median_phase) { dst = out->data[0] + get_x(fphase, s->w) * 4; @@ -206,10 +244,62 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) metadata = &in->metadata; if (metadata) { - uint8_t value[128]; + add_metadata(in, "phase", fphase); + } - snprintf(value, sizeof(value), "%f", fphase); - av_dict_set(metadata, "lavfi.aphasemeter.phase", value, 0); + if (s->do_phasing_detection) { + if (!s->is_mono && ((tolerance - fphase) < FLT_EPSILON)) { + s->is_mono = 1; + s->start_mono_presence = 1; + s->mono_idx[0] = get_index(inlink, in); + } + if (s->is_mono && ((tolerance - fphase) < FLT_EPSILON) && s->start_mono_presence) { + float mono_duration; + s->mono_idx[1] = get_index(inlink, in); + mono_duration = get_duration(s->mono_idx); + if (mono_duration >= s->duration) { + add_metadata(in, "mono_start", s->mono_idx[0]); + av_log(s, AV_LOG_INFO, "mono_start: %f\n", s->mono_idx[0]); + s->start_mono_presence = 0; + } + } + if (s->is_mono && ((tolerance - fphase) > FLT_EPSILON)) { + float mono_duration; + s->mono_idx[1] = get_index(inlink, in); + mono_duration = get_duration(s->mono_idx); + if (mono_duration > s->duration) { + add_metadata(in, "mono_end", s->mono_idx[1]); + add_metadata(in, "mono_duration", mono_duration); + av_log(s, AV_LOG_INFO, "mono_end: %f | mono_duration: %f\n", s->mono_idx[1], mono_duration); + } + s->is_mono = 0; + } + if (!s->is_out_phase && (angle - fphase) > FLT_EPSILON) { + s->out_phase_idx[0] = get_index(inlink, in); + s->is_out_phase = 1; + s->start_out_phase_presence = 1; + } + if (s->is_out_phase && ((angle - fphase) > FLT_EPSILON) && s->start_out_phase_presence) { + float out_phase_duration; + s->out_phase_idx[1] = get_index(inlink, in); + out_phase_duration = get_duration(s->out_phase_idx); + if (out_phase_duration >= s->duration) { + add_metadata(in, "out_phase_start", s->out_phase_idx[0]); + av_log(s, AV_LOG_INFO, "out_phase_start: %f\n", s->out_phase_idx[0]); + s->start_out_phase_presence = 0; + } + } + if (s->is_out_phase && (angle - fphase) < FLT_EPSILON) { + float out_phase_duration; + s->out_phase_idx[1] = get_index(inlink, in); + out_phase_duration = get_duration(s->out_phase_idx); + if (out_phase_duration > s->duration) { + add_metadata(in, "out_phase_end", s->out_phase_idx[1]); + add_metadata(in, "out_phase_duration", out_phase_duration); + av_log(s, AV_LOG_INFO, "out_phase_end: %f | out_phase_duration: %f\n", s->out_phase_idx[1], out_phase_duration); + } + s->is_out_phase = 0; + } } if (s->do_video) { @@ -224,6 +314,32 @@ static av_cold void uninit(AVFilterContext *ctx) AudioPhaseMeterContext *s = ctx->priv; int i; + if (s->do_phasing_detection) { + float tolerance = 1.0 - s->tolerance; + float angle = cos(s->angle/180.0*PI); + float mono_duration; + float out_phase_duration; + AVFilterLink *inlink = ctx->inputs[0]; + + if ((s->is_mono) && ((tolerance - s->phase) < FLT_EPSILON)) { + char *index_str = av_ts2timestr(inlink->current_pts, &inlink->time_base); + s->mono_idx[1] = atof(index_str); + mono_duration = get_duration(s->mono_idx); + if (mono_duration > s->duration) { + av_log(s, AV_LOG_INFO, "mono_end: %f | mono_duration: %f\n", s->mono_idx[1], mono_duration); + } + s->is_mono = 0; + } + if ((s->is_out_phase) && ((angle - s->phase) > FLT_EPSILON)) { + char *index_str = av_ts2timestr(inlink->current_pts, &inlink->time_base); + s->out_phase_idx[1] = atof(index_str); + out_phase_duration = get_duration(s->out_phase_idx); + if (out_phase_duration > s->duration) { + av_log(s, AV_LOG_INFO, "out_phase_end: %f | out_phase_duration: %f\n", s->out_phase_idx[1], out_phase_duration); + } + s->is_out_phase = 0; + } + } av_frame_free(&s->out); for (i = 0; i < ctx->nb_outputs; i++) av_freep(&ctx->output_pads[i].name); -- 2.11.0