From patchwork Fri Aug 5 21:54:44 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Burt P X-Patchwork-Id: 101 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.140.67 with SMTP id o64csp1927565vsd; Fri, 5 Aug 2016 15:09:18 -0700 (PDT) X-Received: by 10.194.97.73 with SMTP id dy9mr73973681wjb.132.1470434958261; Fri, 05 Aug 2016 15:09:18 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x8si10422029wme.6.2016.08.05.15.09.17; Fri, 05 Aug 2016 15:09:18 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2C02F68A71F; Sat, 6 Aug 2016 01:09:06 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-io0-f193.google.com (mail-io0-f193.google.com [209.85.223.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4F0B368A5C7 for ; Sat, 6 Aug 2016 01:08:57 +0300 (EEST) Received: by mail-io0-f193.google.com with SMTP id y195so25076572iod.0 for ; Fri, 05 Aug 2016 15:09:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:subject:date:message-id:in-reply-to:references; bh=37ogiZtXBOcUuX+Eb9sBXPQV440FSZoyaS5I5RYJE/o=; b=BapAoBGcuWOzFonqe7TC9bXQ9bUyd5OqTo79DvfiJ4NJe4ALH7gPBUKQBQQzZKEVOr GkWK7OFu5YeWolJvsjfu1ykFPElPQMjgsgBGbbccG3hc3bh6wT2B2wZgcO2fq+7Sk5pq zskKmfQTpo0hXZYZ4/Ok2gu2HRmJcFk/sP5xNoF+2k3lvUP4CoeZAKV3gIZwezPlP1v6 tI0gDWekv+qPP+kpDOpqBPI2Luwu0irEoEA0A4B/TeEHwftA2z5tgFK81Y7eb4G4j2QN KtPtz1aLFMtqXMYblDI2vHj0hfsvXNRZFDGmp/cxYnKqLuIUsnqZNSCqOIbS874HGmbZ Rusg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=37ogiZtXBOcUuX+Eb9sBXPQV440FSZoyaS5I5RYJE/o=; b=IH9HsnGX6Tf2qZe05kzp4zHfTjQqRLRtAG4fsdAhosXixgZONV9ZUfytdNM4mniVvZ 5PMwkqdn476Q54OteO3dqVwdWx9F1e1OCmt+9otPJkCoOi6LrMDhrccp3dHn8PfLBkiO UCb3NwhzNl+XUe4K64Uz5m5lt0L+HgYivt3VeBfOjDP2YDy9FoZksRoRh7+zq2ZdvvLc qHlD9UGE8Qu1iEuh8S7F+5DUHhpDBkU1Wv16uLMchG1XdeISq4k3/i+BzYd0F27AC/iM M8PlZOxczZK7oFgtIeUS6SARHw2azkrmufj1PYvAeztZZ/u77IubBeP44869hPGRQFm1 c0pw== X-Gm-Message-State: AEkoouvNMrH8IEcgTZFw5HZYhRudH+nGgUdO0ek7+OFNBOeM1roAS6sCdZ4SVDC2MHszSw== X-Received: by 10.107.128.210 with SMTP id k79mr83127180ioi.3.1470434106584; Fri, 05 Aug 2016 14:55:06 -0700 (PDT) Received: from localhost.localdomain ([216.16.66.181]) by smtp.gmail.com with ESMTPSA id c192sm4722503ith.11.2016.08.05.14.55.04 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 05 Aug 2016 14:55:06 -0700 (PDT) From: Burt P To: ffmpeg-devel@ffmpeg.org Date: Fri, 5 Aug 2016 16:54:44 -0500 Message-Id: <1470434084-1323-3-git-send-email-pburt0@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1470434084-1323-1-git-send-email-pburt0@gmail.com> References: <1470434084-1323-1-git-send-email-pburt0@gmail.com> Subject: [FFmpeg-devel] [PATCH 2/2] af_hdcd: Add analyze mode X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" A new mode, selected by filter option, to aid in analysis of HDCD encoded audio. In this mode the audio is replaced by a solid tone and the amplitude is adjusted to signal some specified aspect of the process. The output file can be loaded in an audio editor alongside the original, where the user can see where different features or states are present. Signed-off-by: Burt P --- doc/filters.texi | 32 ++++++++++ libavfilter/af_hdcd.c | 164 +++++++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 188 insertions(+), 8 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 7e042e4..e8df2ee 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -8454,6 +8454,38 @@ ffmpeg -i HDCD16.wav -af hdcd OUT16.wav ffmpeg -i HDCD16.wav -af hdcd -acodec pcm_s24le OUT24.wav @end example +The filter accepts the following options: + +@table @option +@item process_mode +Process the stereo channels together. If target_gain does not match between +channels, consider it invalid and use the last valid target_gain. + +@item force_pe +Always extend peaks above -3dBFS even if PE isn't signaled. + +@item analyze_mode +Replace audio with a solid tone and adjust the amplitude to signal some +specific aspect of the decoding process. The output file can be loaded in +an audio editor alongside the original to aid analysis. + +@code{analyze_mode=pe:force_pe=1} can be used to see all samples above the PE level. + +Modes are: +@table @samp +@item 0, off +Disabled +@item 1, lle +Gain adjustment level at each sample +@item 2, pe +Samples where peak extend occurs +@item 3, cdt +Samples where the code detect timer is active +@item 4, tgm +Samples where the target gain does not match between channels +@end table +@end table + @section hflip Flip the input video horizontally. diff --git a/libavfilter/af_hdcd.c b/libavfilter/af_hdcd.c index 610dd9e..edf3138 100644 --- a/libavfilter/af_hdcd.c +++ b/libavfilter/af_hdcd.c @@ -870,6 +870,27 @@ static const char * const pe_str[] = { * the always-negative value is stored positive, so make it negative */ #define GAINTOFLOAT(g) (g) ? -(float)(g>>1) - ((g & 1) ? 0.5 : 0.0) : 0.0 +#define HDCD_ANA_OFF 0 +#define HDCD_ANA_OFF_DESC "disabled" +#define HDCD_ANA_LLE 1 +#define HDCD_ANA_LLE_DESC "gain adjustment level at each sample" +#define HDCD_ANA_PE 2 +#define HDCD_ANA_PE_DESC "samples where peak extend occurs" +#define HDCD_ANA_CDT 3 +#define HDCD_ANA_CDT_DESC "samples where the code detect timer is active" +#define HDCD_ANA_TGM 4 +#define HDCD_ANA_TGM_DESC "samples where the target gain does not match between channels" +#define HDCD_ANA_TOP 5 /* used in max value of AVOption */ +typedef int hdcd_ana_mode_t; /* formerly enum, but that didn't work for AVOption initialization */ + +static const char * const ana_mode_str[] = { + HDCD_ANA_OFF_DESC, + HDCD_ANA_LLE_DESC, + HDCD_ANA_PE_DESC, + HDCD_ANA_CDT_DESC, + HDCD_ANA_TGM_DESC, +}; + typedef struct HDCDContext { const AVClass *class; hdcd_state_t state[HDCD_MAX_CHANNELS]; @@ -885,6 +906,12 @@ typedef struct HDCDContext { * default is off */ int force_pe; + /* analyze mode replaces the audio with a solid tone and adjusts + * the amplitude to signal some specific aspect of the decoding + * process. See docs or hdcd_ana_mode_t */ + hdcd_ana_mode_t analyze_mode; + int ana_snb; /* used in tone generation */ + /* config_input() and config_output() scan links for any resampling * or format changes. If found, warnings are issued and bad_config * is set. */ @@ -909,6 +936,13 @@ static const AVOption hdcd_options[] = { OFFSET(process_stereo), AV_OPT_TYPE_BOOL, { .i64 = HDCD_PROCESS_STEREO_DEFAULT }, 0, 1, A }, { "force_pe", "Always extend peaks above -3dBFS even when PE is not signaled.", OFFSET(force_pe), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, A }, + { "analyze_mode", "Replace audio with solid tone and signal some processing aspect in the amplitude.", + OFFSET(analyze_mode), AV_OPT_TYPE_INT, { .i64=HDCD_ANA_OFF }, 0, HDCD_ANA_TOP-1, A, "analyze_mode"}, + { "off", HDCD_ANA_OFF_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_OFF}, 0, 0, A, "analyze_mode" }, + { "lle", HDCD_ANA_LLE_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_LLE}, 0, 0, A, "analyze_mode" }, + { "pe", HDCD_ANA_PE_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_PE}, 0, 0, A, "analyze_mode" }, + { "cdt", HDCD_ANA_CDT_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_CDT}, 0, 0, A, "analyze_mode" }, + { "tgm", HDCD_ANA_TGM_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_TGM}, 0, 0, A, "analyze_mode" }, {NULL} }; @@ -1209,6 +1243,77 @@ static int hdcd_scan_stereo(HDCDContext *ctx, const int32_t *samples, int max) return result; } +/* encode a value in the given sample by adjusting the amplitude */ +static int32_t hdcd_analyze_gen(int32_t sample, unsigned int v, unsigned int maxv) +{ + float sflt = sample, vv = v; + vv /= maxv; + if (vv > 1.0) vv = 1.0; + sflt *= 1.0 + (vv * 18); + return (int32_t)sflt; +} + +/* behaves like hdcd_envelope(), but encodes processing information in + * a way that is audible (and visible in an audio editor) to aid analysis. */ +static int hdcd_analyze(int32_t *samples, int count, int stride, int gain, int target_gain, int extend, hdcd_ana_mode_t mode, int cdt_active, int tg_mismatch) +{ + static const int maxg = 0xf << 7; + int i; + int32_t *samples_end = samples + stride * count; + + for (i = 0; i < count; i++) { + samples[i * stride] <<= 15; + if (mode == HDCD_ANA_PE) { + int pel = (samples[i * stride] >> 16) & 1; + int32_t sample = samples[i * stride]; + samples[i * stride] = hdcd_analyze_gen(sample, !!(pel && extend), 1); + } else if (mode == HDCD_ANA_TGM && tg_mismatch > 0) + samples[i * stride] = hdcd_analyze_gen(samples[i * stride], 1, 1); + else if (mode == HDCD_ANA_CDT && cdt_active) + samples[i * stride] = hdcd_analyze_gen(samples[i * stride], 1, 1); + } + + if (gain <= target_gain) { + int len = FFMIN(count, target_gain - gain); + /* attenuate slowly */ + for (i = 0; i < len; i++) { + ++gain; + if (mode == HDCD_ANA_LLE) + *samples = hdcd_analyze_gen(*samples, gain, maxg); + samples += stride; + } + count -= len; + } else { + int len = FFMIN(count, (gain - target_gain) >> 3); + /* amplify quickly */ + for (i = 0; i < len; i++) { + gain -= 8; + if (mode == HDCD_ANA_LLE) + *samples = hdcd_analyze_gen(*samples, gain, maxg); + samples += stride; + } + if (gain - 8 < target_gain) + gain = target_gain; + count -= len; + } + + /* hold a steady level */ + if (gain == 0) { + if (count > 0) + samples += count * stride; + } else { + while (--count >= 0) { + if (mode == HDCD_ANA_LLE) + *samples = hdcd_analyze_gen(*samples, gain, maxg); + samples += stride; + } + } + + av_assert0(samples == samples_end); + + return gain; +} + static int hdcd_envelope(int32_t *samples, int count, int stride, int gain, int target_gain, int extend) { int i; @@ -1316,7 +1421,10 @@ static void hdcd_process(HDCDContext *ctx, hdcd_state_t *state, int32_t *samples envelope_run = run - 1; av_assert0(samples + envelope_run * stride <= samples_end); - gain = hdcd_envelope(samples, envelope_run, stride, gain, target_gain, peak_extend); + if (ctx->analyze_mode) + gain = hdcd_analyze(samples, envelope_run, stride, gain, target_gain, peak_extend, ctx->analyze_mode, state->sustain, -1); + else + gain = hdcd_envelope(samples, envelope_run, stride, gain, target_gain, peak_extend); samples += envelope_run * stride; count -= envelope_run; @@ -1325,7 +1433,10 @@ static void hdcd_process(HDCDContext *ctx, hdcd_state_t *state, int32_t *samples } if (lead > 0) { av_assert0(samples + lead * stride <= samples_end); - gain = hdcd_envelope(samples, lead, stride, gain, target_gain, peak_extend); + if (ctx->analyze_mode) + gain = hdcd_analyze(samples, lead, stride, gain, target_gain, peak_extend, ctx->analyze_mode, state->sustain, -1); + else + gain = hdcd_envelope(samples, lead, stride, gain, target_gain, peak_extend); } state->running_gain = gain; @@ -1339,7 +1450,7 @@ static void hdcd_process_stereo(HDCDContext *ctx, int32_t *samples, int count) int peak_extend[2]; int lead = 0; - hdcd_control_stereo(ctx, &peak_extend[0], &peak_extend[1]); + int ctlret = hdcd_control_stereo(ctx, &peak_extend[0], &peak_extend[1]); while (count > lead) { int envelope_run, run; @@ -1349,7 +1460,16 @@ static void hdcd_process_stereo(HDCDContext *ctx, int32_t *samples, int count) av_assert0(samples + envelope_run * stride <= samples_end); - if (envelope_run) { + if (ctx->analyze_mode) { + gain[0] = hdcd_analyze(samples, envelope_run, stride, gain[0], ctx->val_target_gain, peak_extend[0], + ctx->analyze_mode, + ctx->state[0].sustain, + !!(ctlret == HDCD_TG_MISMATCH) ); + gain[1] = hdcd_analyze(samples + 1, envelope_run, stride, gain[1], ctx->val_target_gain, peak_extend[1], + ctx->analyze_mode, + ctx->state[1].sustain, + !!(ctlret == HDCD_TG_MISMATCH) ); + } else { gain[0] = hdcd_envelope(samples, envelope_run, stride, gain[0], ctx->val_target_gain, peak_extend[0]); gain[1] = hdcd_envelope(samples + 1, envelope_run, stride, gain[1], ctx->val_target_gain, peak_extend[1]); } @@ -1358,18 +1478,32 @@ static void hdcd_process_stereo(HDCDContext *ctx, int32_t *samples, int count) count -= envelope_run; lead = run - envelope_run; - hdcd_control_stereo(ctx, &peak_extend[0], &peak_extend[1]); + ctlret = hdcd_control_stereo(ctx, &peak_extend[0], &peak_extend[1]); } if (lead > 0) { av_assert0(samples + lead * stride <= samples_end); - gain[0] = hdcd_envelope(samples, lead, stride, gain[0], ctx->val_target_gain, peak_extend[0]); - gain[1] = hdcd_envelope(samples + 1, lead, stride, gain[1], ctx->val_target_gain, peak_extend[1]); + if (ctx->analyze_mode) { + gain[0] = hdcd_analyze(samples, lead, stride, gain[0], ctx->val_target_gain, peak_extend[0], + ctx->analyze_mode, + ctx->state[0].sustain, + !!(ctlret == HDCD_TG_MISMATCH) ); + gain[1] = hdcd_analyze(samples + 1, lead, stride, gain[1], ctx->val_target_gain, peak_extend[1], + ctx->analyze_mode, + ctx->state[1].sustain, + !!(ctlret == HDCD_TG_MISMATCH) ); + } else { + gain[0] = hdcd_envelope(samples, lead, stride, gain[0], ctx->val_target_gain, peak_extend[0]); + gain[1] = hdcd_envelope(samples + 1, lead, stride, gain[1], ctx->val_target_gain, peak_extend[1]); + } } ctx->state[0].running_gain = gain[0]; ctx->state[1].running_gain = gain[1]; } +/* tone generator: sample_number, frequency, sample_rate, amplitude */ +#define TONEGEN16(sn, f, sr, a) (int16_t)(sin((6.28318530718 * sn * f) /sr) * a * 0x7fff) + static int filter_frame(AVFilterLink *inlink, AVFrame *in) { AVFilterContext *ctx = inlink->dst; @@ -1390,8 +1524,21 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) in_data = (int16_t*)in->data[0]; out_data = (int32_t*)out->data[0]; - for (n = 0; n < in->nb_samples * in->channels; n++) { + for (c = n = 0; n < in->nb_samples * in->channels; n++) { out_data[n] = in_data[n]; + if (s->analyze_mode) { + /* in analyze mode, the audio is replaced by a solid tone, and + * amplitude is changed to signal when the specified feature is + * used. + * bit 0: HDCD signal preserved + * bit 1: Original sample was above PE level */ + int32_t save = (abs(in_data[n]) - 0x5981 >= 0) ? 2 : 0; /* above PE level */ + save |= in_data[n] & 1; /* save LSB for HDCD packets */ + out_data[n] = TONEGEN16(s->ana_snb, 277.18, 44100, 0.1); + out_data[n] = (out_data[n] | 3) ^ ((~save) & 3); + if (++c == in->channels) { s->ana_snb++; c = 0; } + if (s->ana_snb > 0x3fffffff) s->ana_snb = 0; + } } s->det_errors = 0; /* re-sum every pass */ @@ -1557,6 +1704,7 @@ static av_cold int init(AVFilterContext *ctx) (s->process_stereo) ? "process stereo channels together" : "process each channel separately"); av_log(ctx, AV_LOG_VERBOSE, "Force PE: %s\n", (s->force_pe) ? "on" : "off"); + av_log(ctx, AV_LOG_VERBOSE, "Analyze mode: [%d] %s\n", s->analyze_mode, ana_mode_str[s->analyze_mode] ); return 0; }