From patchwork Sun Sep 12 18:44:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 30196 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp3373978iov; Sun, 12 Sep 2021 11:44:21 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyFvTZ8fmoM5ZhrZhiJmDTGSOtRcjUamvRe5frRTK/IL+LuiiydUpMrHeo5aeis4OYr1DXj X-Received: by 2002:aa7:c806:: with SMTP id a6mr9150703edt.354.1631472261022; Sun, 12 Sep 2021 11:44:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1631472261; cv=none; d=google.com; s=arc-20160816; b=MuAqFB3Kv3mTT8I3inBjaS348o94ZZBW0Ft1xqo9gS4p6PXTouGLHt0ZhSqF2Ger3R yegDbcpsMjLHx93AJnQjslKmLUG2tFdyHj2QF4Kb3/HRavxWIh6iRK0IUng3ixW6nGGh A3kCdvak5QpVe9MhzVXC08r7VdWaax9qPMRoiQm9TYE1NvFAI25uKaRJRH3LVq0a8j6B hA++6oU7p0yu2U7d0Z9eYelo+4U6JQujzCoBbXFt1rDBVRKaMa+TMGgmsq+qKl6BC1Pq mEal6utpXwUAuTVCH755cBNoMHnfJd8oukWuCWjAdEH6zIlSmXiwqfi11brAgJEfStu4 aJfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to; bh=rj2FnGrDRk/80zaEvBvLqt7CfViMxllnZx0NXGVmZHY=; b=Ekg4z4XI/hG713zFrijaElkGbV/5L44viZLmXj4WR2nqoXIj7WuwXjjVcHO17Vewci rDCFhk8JSJDJ9MIFreKyJmpFCWZmWsvdqn+NxY8DWZkNnGP1gCLpsp4iz3OlOTXJwMMp tDQwMBjIGk3tu0E7FKHK/YtrUieEYbjzdhz8PNt/Q3Y41yzcTEtFHYTidsY8xIBvBFLa 8oEZbmUhj0R5RjKQoVdrR1b4Ei51FwKvIAvjIHR8/Nxzb/OYrkC7npqLZuhxc3Ud/prU mV0a8qO/VTUSAuoAkMda2kJQ3jlpylFSfZXQo/0tHABc7sNB0vIxiiCPUivmbHBYBgLQ msJw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=BVdja9x1; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id kq12si5463840ejb.759.2021.09.12.11.44.20; Sun, 12 Sep 2021 11:44:21 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=BVdja9x1; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E6DDD68A8DE; Sun, 12 Sep 2021 21:44:16 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 66F3668A16C for ; Sun, 12 Sep 2021 21:44:10 +0300 (EEST) Received: by mail-wr1-f51.google.com with SMTP id i23so11113482wrb.2 for ; Sun, 12 Sep 2021 11:44:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id; bh=cgjeBfHPLL3rPNxEq73k7dCFM7tQOtllnrCdPugVptA=; b=BVdja9x1BU+/bIzof258ZITQd2SIHAURuRrdg2wb1iIB8Jo6PKpOWlJOOBuUkUV9T7 bYs/Medbs6YnkNRM9TRJnL/H5zEYG/KcDhXTzqg5GPW5K4FDG4npelKPL399X+Csa2XZ lwHrtRoHUJgbXvdAUKgy3Prs0GqjwNT3Gd8u8FB3yW7VJJtqqqZoExbTcd/uIQZRkpq8 TDsuyIspc0wXRwp3ypzOXIPtNM8xSGUJ99GJPJ+z90xjH+q3YrghST2+2gL2UHhZR3U2 t8UI/14fOUxYyVNrjAeKQik4BQZCq3VJ+2HkDU/NvxM36vkvGMnkk0tj1bttGeJpk2rt V3QA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id; bh=cgjeBfHPLL3rPNxEq73k7dCFM7tQOtllnrCdPugVptA=; b=qRr/adcQIE3jPrWPRUFD+0jyiNkVoJ3aeKuqHx1Zd9Y4wHNpG5O2qCRYDAOpGQqkZ3 zCzg96X7aj4WkpM7NtTQ9dz18fDaFUJ9ej9F6ca1vbdy9k1W0UpgMPpaCimo78b0GgU5 72N5YQWYIGf4Ev4Zen7hGucEp+2IZh/u1IZSEF1f5H3ba7laPVpjaSj2gsPI3QH+98/j DTqVkAOUeHzHkQNmALWunIN1mI14hy3YJtf1g1FOBg9RYFm5l2Hba4EM3YXBPiJIkDOQ oxw/BtsfQuAR69P3EQUO+BeP/HIQWO7mn7hecVu3Gf2zO5otGfRFvTltw+57bzqTuKGM piOg== X-Gm-Message-State: AOAM531i0xph+M+JPirxNPxvl/dlVAoCH0LoiIWh7E5iCZNpj+qGFEUz 0QmEk4z7YqkGN+wJoSgUiUTb92IiwTE= X-Received: by 2002:a5d:6944:: with SMTP id r4mr8600677wrw.186.1631472249651; Sun, 12 Sep 2021 11:44:09 -0700 (PDT) Received: from localhost.localdomain ([95.168.121.30]) by smtp.gmail.com with ESMTPSA id r129sm4723445wmr.7.2021.09.12.11.44.08 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Sep 2021 11:44:09 -0700 (PDT) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Sun, 12 Sep 2021 20:44:01 +0200 Message-Id: <20210912184401.18782-1-onemda@gmail.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH] avfilter: add audio signal to distortion ratio filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: KyELMcVofyII Signed-off-by: Paul B Mahol --- doc/filters.texi | 7 ++ libavfilter/Makefile | 1 + libavfilter/af_asdr.c | 237 +++++++++++++++++++++++++++++++++++++++ libavfilter/allfilters.c | 1 + 4 files changed, 246 insertions(+) create mode 100644 libavfilter/af_asdr.c diff --git a/doc/filters.texi b/doc/filters.texi index 8f20ccf8c6..6af7344820 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -2531,6 +2531,13 @@ noise removed from input signal. This filter supports the all above options as @ref{commands}. +@section asdr +Measure Audio Signal-to-Distortion Ratio. + +This filter takes two audio streams for input, and outputs first +audio stream. +Results are in dB per channel at end of either input. + @section asetnsamples Set the number of samples per each output audio frame. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 76c65c3f42..865252ef3f 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -82,6 +82,7 @@ OBJS-$(CONFIG_AREALTIME_FILTER) += f_realtime.o OBJS-$(CONFIG_ARESAMPLE_FILTER) += af_aresample.o OBJS-$(CONFIG_AREVERSE_FILTER) += f_reverse.o OBJS-$(CONFIG_ARNNDN_FILTER) += af_arnndn.o +OBJS-$(CONFIG_ASDR_FILTER) += af_asdr.o OBJS-$(CONFIG_ASEGMENT_FILTER) += f_segment.o OBJS-$(CONFIG_ASELECT_FILTER) += f_select.o OBJS-$(CONFIG_ASENDCMD_FILTER) += f_sendcmd.o diff --git a/libavfilter/af_asdr.c b/libavfilter/af_asdr.c new file mode 100644 index 0000000000..220da74a85 --- /dev/null +++ b/libavfilter/af_asdr.c @@ -0,0 +1,237 @@ +/* + * Copyright (c) 2021 Paul B Mahol + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/audio_fifo.h" +#include "libavutil/channel_layout.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" + +#include "audio.h" +#include "avfilter.h" +#include "formats.h" +#include "filters.h" +#include "internal.h" + +typedef struct AudioSDRContext { + int channels; + int64_t pts; + double *sum_u; + double *sum_uv; + + AVFrame *cache[2]; + + AVAudioFifo *fifo[2]; +} AudioSDRContext; + +static int query_formats(AVFilterContext *ctx) +{ + static const enum AVSampleFormat sample_fmts[] = { + AV_SAMPLE_FMT_DBLP, + AV_SAMPLE_FMT_NONE + }; + int ret = ff_set_common_all_channel_counts(ctx); + if (ret < 0) + return ret; + + ret = ff_set_common_formats_from_list(ctx, sample_fmts); + if (ret < 0) + return ret; + + return ff_set_common_all_samplerates(ctx); +} + +static void sdr(AVFilterContext *ctx, const AVFrame *u, const AVFrame *v) +{ + AudioSDRContext *s = ctx->priv; + + for (int ch = 0; ch < u->channels; ch++) { + const double *const us = (double *)u->extended_data[ch]; + const double *const vs = (double *)v->extended_data[ch]; + double sum_uv = s->sum_uv[ch]; + double sum_u = s->sum_u[ch]; + + for (int n = 0; n < u->nb_samples; n++) { + sum_u += us[n] * us[n]; + sum_uv += (us[n] - vs[n]) * (us[n] - vs[n]); + } + + s->sum_uv[ch] = sum_uv; + s->sum_u[ch] = sum_u; + } +} + +static int activate(AVFilterContext *ctx) +{ + AudioSDRContext *s = ctx->priv; + AVFrame *frame = NULL; + int ret, status; + int available; + int64_t pts; + + FF_FILTER_FORWARD_STATUS_BACK_ALL(ctx->outputs[0], ctx); + + for (int i = 0; i < 2; i++) { + ret = ff_inlink_consume_frame(ctx->inputs[i], &frame); + if (ret > 0) { + if (s->pts == AV_NOPTS_VALUE) + s->pts = frame->pts; + ret = av_audio_fifo_write(s->fifo[i], (void **)frame->extended_data, + frame->nb_samples); + av_frame_free(&frame); + if (ret < 0) + return ret; + } + } + + available = FFMIN(av_audio_fifo_size(s->fifo[0]), av_audio_fifo_size(s->fifo[1])); + if (available > 0) { + AVFrame *out; + + if (!s->cache[0] || s->cache[0]->nb_samples < available) { + av_frame_free(&s->cache[0]); + s->cache[0] = ff_get_audio_buffer(ctx->outputs[0], available); + if (!s->cache[0]) + return AVERROR(ENOMEM); + } + + if (!s->cache[1] || s->cache[1]->nb_samples < available) { + av_frame_free(&s->cache[1]); + s->cache[1] = ff_get_audio_buffer(ctx->outputs[0], available); + if (!s->cache[1]) + return AVERROR(ENOMEM); + } + + ret = av_audio_fifo_peek(s->fifo[0], (void **)s->cache[0]->extended_data, available); + if (ret < 0) + return ret; + + ret = av_audio_fifo_peek(s->fifo[1], (void **)s->cache[1]->extended_data, available); + if (ret < 0) + return ret; + + sdr(ctx, s->cache[0], s->cache[1]); + + out = av_frame_clone(s->cache[0]); + out->nb_samples = available; + out->pts = s->pts; + s->pts += available; + + av_audio_fifo_drain(s->fifo[0], available); + av_audio_fifo_drain(s->fifo[1], available); + + return ff_filter_frame(ctx->outputs[0], out); + } + + for (int i = 0; i < 2; i++) { + if (ff_inlink_acknowledge_status(ctx->inputs[i], &status, &pts)) { + ff_outlink_set_status(ctx->outputs[0], status, pts); + return 0; + } + } + + if (av_audio_fifo_size(s->fifo[0]) > 0 && + av_audio_fifo_size(s->fifo[1]) > 0) { + ff_filter_set_ready(ctx, 10); + return 0; + } + + if (ff_outlink_frame_wanted(ctx->outputs[0])) { + for (int i = 0; i < 2; i++) { + if (av_audio_fifo_size(s->fifo[i]) > 0) + continue; + ff_inlink_request_frame(ctx->inputs[i]); + return 0; + } + } + + return FFERROR_NOT_READY; +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + AVFilterLink *inlink = ctx->inputs[0]; + AudioSDRContext *s = ctx->priv; + + s->pts = AV_NOPTS_VALUE; + + s->channels = inlink->channels; + outlink->format = inlink->format; + outlink->channels = inlink->channels; + + s->fifo[0] = av_audio_fifo_alloc(outlink->format, outlink->channels, 1024); + s->fifo[1] = av_audio_fifo_alloc(outlink->format, outlink->channels, 1024); + if (!s->fifo[0] || !s->fifo[1]) + return AVERROR(ENOMEM); + + s->sum_u = av_calloc(outlink->channels, sizeof(*s->sum_u)); + s->sum_uv = av_calloc(outlink->channels, sizeof(*s->sum_uv)); + if (!s->sum_u || !s->sum_uv) + return AVERROR(ENOMEM); + + return 0; +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + AudioSDRContext *s = ctx->priv; + + for (int ch = 0; ch < s->channels; ch++) + av_log(ctx, AV_LOG_INFO, "SDR ch%d: %g dB\n", ch, 20. * log10(s->sum_u[ch] / s->sum_uv[ch])); + + av_audio_fifo_free(s->fifo[0]); + av_audio_fifo_free(s->fifo[1]); + + av_frame_free(&s->cache[0]); + av_frame_free(&s->cache[1]); + + av_freep(&s->sum_u); + av_freep(&s->sum_uv); +} + +static const AVFilterPad inputs[] = { + { + .name = "input0", + .type = AVMEDIA_TYPE_AUDIO, + }, + { + .name = "input1", + .type = AVMEDIA_TYPE_AUDIO, + }, +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + .config_props = config_output, + }, +}; + +const AVFilter ff_af_asdr = { + .name = "asdr", + .description = NULL_IF_CONFIG_SMALL("Measure Audio Signal-to-Distortion Ratio."), + .priv_size = sizeof(AudioSDRContext), + .query_formats = query_formats, + .activate = activate, + .uninit = uninit, + FILTER_INPUTS(inputs), + FILTER_OUTPUTS(outputs), +}; diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 73a0bf9c44..7234ca6dbe 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -75,6 +75,7 @@ extern const AVFilter ff_af_arealtime; extern const AVFilter ff_af_aresample; extern const AVFilter ff_af_areverse; extern const AVFilter ff_af_arnndn; +extern const AVFilter ff_af_asdr; extern const AVFilter ff_af_asegment; extern const AVFilter ff_af_aselect; extern const AVFilter ff_af_asendcmd;