From patchwork Sun Aug 7 07:41:26 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Burt P X-Patchwork-Id: 113 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.140.67 with SMTP id o64csp2478695vsd; Sun, 7 Aug 2016 00:41:57 -0700 (PDT) X-Received: by 10.194.77.97 with SMTP id r1mr84006043wjw.83.1470555717008; Sun, 07 Aug 2016 00:41:57 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b189si15722650wmd.92.2016.08.07.00.41.56; Sun, 07 Aug 2016 00:41:56 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 62C91689F7C; Sun, 7 Aug 2016 10:41:39 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-io0-f194.google.com (mail-io0-f194.google.com [209.85.223.194]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AA2A6689F16 for ; Sun, 7 Aug 2016 10:41:25 +0300 (EEST) Received: by mail-io0-f194.google.com with SMTP id q83so26952459iod.2 for ; Sun, 07 Aug 2016 00:41:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:subject:date:message-id:in-reply-to:references; bh=owYS5MddDYwH0LZN9p/D1qGkA2yJFZalpEmV7LGX4OU=; b=C4XGuE7mYb6FgoRcVK1f4auv8MTF5s5DdEZQIOA05hi2u1IDzTFCixm5O6LRUNWvBg uFAob0RbFAM2hUf1U3IQigzFnsZ0BMIrtzu6bOqPh2Qh+N7PP/v9hjCFoVrqL1kLopHa LL+Ln3OGuExiUjQsHEVqTl/24QstUC1AJwvbpPzLvg4gQYE8Te+vpXXya4BPk/0OaCcS YMLBite1ZFBEVyIu9XeuVxBKeq/t6IMFAGXacP70PvnTwWkwhjGxuAA2wTb1nF+rlsc6 1HQU9X6z5+z5RSZlcIl7opjTM56qYEK0eRAEFxn3vwlFKFFf3rgK1zQoGsstDVvqtoz0 ixLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=owYS5MddDYwH0LZN9p/D1qGkA2yJFZalpEmV7LGX4OU=; b=P9aanbGja7WHsTefVP32fwxJqcBxmtwVP9/TLHY0upTGh0DB7nhGdjhfWLxBrYbfNT cRW/4Px0DgKoAUvnqoq/pSXrmnM4CkcmyfnsJNHb6Fh617v8RGn76S7zhgnXSR04VxH1 rgoTIIceJ1tZWouC+u1Q1Cp5i0mSt3Y5dYh/aRXWdnrdmbja/DmoYaRoKmfm/W9Zponk uBVNPIJ4QYv4Lstklzul23i5SY6SX2uj0q3dHgh4AmSiU7mBCavlAwncWbRqUq2vul5i W8odX9oZB9LKrM71RnlVgd5K4TMDDBfL6LAqTmcKzmJdtT2ejJ6r90gJQN9SHpnoOq+D 1yDw== X-Gm-Message-State: AEkooutigG1MQ8QdVdH70ouI1SO35ChvK1Z6+RJBllIzboVM18kB7m7YDIT0cT0U8bbb3A== X-Received: by 10.107.192.69 with SMTP id q66mr89349555iof.5.1470555691370; Sun, 07 Aug 2016 00:41:31 -0700 (PDT) Received: from localhost.localdomain ([216.16.66.181]) by smtp.gmail.com with ESMTPSA id q204sm7629722itc.19.2016.08.07.00.41.30 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 07 Aug 2016 00:41:30 -0700 (PDT) From: Burt P To: ffmpeg-devel@ffmpeg.org Date: Sun, 7 Aug 2016 02:41:26 -0500 Message-Id: <1470555686-12654-1-git-send-email-pburt0@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1470487638-2709-1-git-send-email-pburt0@gmail.com> References: <1470487638-2709-1-git-send-email-pburt0@gmail.com> Subject: [FFmpeg-devel] [PATCHv4] af_hdcd: Add analyze mode (v4) X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" A new mode, selected by filter option, to aid in analysis of HDCD encoded audio. In this mode the audio is replaced by a solid tone and the amplitude is adjusted to signal some specified aspect of the process. The output file can be loaded in an audio editor alongside the original, where the user can see where different features or states are present. Signed-off-by: Burt P --- doc/filters.texi | 32 ++++++++++ libavfilter/af_hdcd.c | 163 +++++++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 187 insertions(+), 8 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 7e042e4..0d0684f 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -8454,6 +8454,38 @@ ffmpeg -i HDCD16.wav -af hdcd OUT16.wav ffmpeg -i HDCD16.wav -af hdcd -acodec pcm_s24le OUT24.wav @end example +The filter accepts the following options: + +@table @option +@item process_stereo +Process the stereo channels together. If target_gain does not match between +channels, consider it invalid and use the last valid target_gain. + +@item force_pe +Always extend peaks above -3dBFS even if PE isn't signaled. + +@item analyze_mode +Replace audio with a solid tone and adjust the amplitude to signal some +specific aspect of the decoding process. The output file can be loaded in +an audio editor alongside the original to aid analysis. + +@code{analyze_mode=pe:force_pe=1} can be used to see all samples above the PE level. + +Modes are: +@table @samp +@item 0, off +Disabled +@item 1, lle +Gain adjustment level at each sample +@item 2, pe +Samples where peak extend occurs +@item 3, cdt +Samples where the code detect timer is active +@item 4, tgm +Samples where the target gain does not match between channels +@end table +@end table + @section hflip Flip the input video horizontally. diff --git a/libavfilter/af_hdcd.c b/libavfilter/af_hdcd.c index 610dd9e..5532528 100644 --- a/libavfilter/af_hdcd.c +++ b/libavfilter/af_hdcd.c @@ -870,6 +870,26 @@ static const char * const pe_str[] = { * the always-negative value is stored positive, so make it negative */ #define GAINTOFLOAT(g) (g) ? -(float)(g>>1) - ((g & 1) ? 0.5 : 0.0) : 0.0 +#define HDCD_ANA_OFF 0 +#define HDCD_ANA_OFF_DESC "disabled" +#define HDCD_ANA_LLE 1 +#define HDCD_ANA_LLE_DESC "gain adjustment level at each sample" +#define HDCD_ANA_PE 2 +#define HDCD_ANA_PE_DESC "samples where peak extend occurs" +#define HDCD_ANA_CDT 3 +#define HDCD_ANA_CDT_DESC "samples where the code detect timer is active" +#define HDCD_ANA_TGM 4 +#define HDCD_ANA_TGM_DESC "samples where the target gain does not match between channels" +#define HDCD_ANA_TOP 5 /* used in max value of AVOption */ + +static const char * const ana_mode_str[] = { + HDCD_ANA_OFF_DESC, + HDCD_ANA_LLE_DESC, + HDCD_ANA_PE_DESC, + HDCD_ANA_CDT_DESC, + HDCD_ANA_TGM_DESC, +}; + typedef struct HDCDContext { const AVClass *class; hdcd_state_t state[HDCD_MAX_CHANNELS]; @@ -885,6 +905,12 @@ typedef struct HDCDContext { * default is off */ int force_pe; + /* analyze mode replaces the audio with a solid tone and adjusts + * the amplitude to signal some specific aspect of the decoding + * process. See docs or HDCD_ANA_* defines. */ + int analyze_mode; + int ana_snb; /* used in tone generation */ + /* config_input() and config_output() scan links for any resampling * or format changes. If found, warnings are issued and bad_config * is set. */ @@ -909,6 +935,13 @@ static const AVOption hdcd_options[] = { OFFSET(process_stereo), AV_OPT_TYPE_BOOL, { .i64 = HDCD_PROCESS_STEREO_DEFAULT }, 0, 1, A }, { "force_pe", "Always extend peaks above -3dBFS even when PE is not signaled.", OFFSET(force_pe), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, A }, + { "analyze_mode", "Replace audio with solid tone and signal some processing aspect in the amplitude.", + OFFSET(analyze_mode), AV_OPT_TYPE_INT, { .i64=HDCD_ANA_OFF }, 0, HDCD_ANA_TOP-1, A, "analyze_mode"}, + { "off", HDCD_ANA_OFF_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_OFF}, 0, 0, A, "analyze_mode" }, + { "lle", HDCD_ANA_LLE_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_LLE}, 0, 0, A, "analyze_mode" }, + { "pe", HDCD_ANA_PE_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_PE}, 0, 0, A, "analyze_mode" }, + { "cdt", HDCD_ANA_CDT_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_CDT}, 0, 0, A, "analyze_mode" }, + { "tgm", HDCD_ANA_TGM_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_TGM}, 0, 0, A, "analyze_mode" }, {NULL} }; @@ -1209,6 +1242,77 @@ static int hdcd_scan_stereo(HDCDContext *ctx, const int32_t *samples, int max) return result; } +/* encode a value in the given sample by adjusting the amplitude */ +static int32_t hdcd_analyze_gen(int32_t sample, unsigned int v, unsigned int maxv) +{ + float sflt = sample, vv = v; + vv /= maxv; + if (vv > 1.0) vv = 1.0; + sflt *= 1.0 + (vv * 18); + return (int32_t)sflt; +} + +/* behaves like hdcd_envelope(), but encodes processing information in + * a way that is audible (and visible in an audio editor) to aid analysis. */ +static int hdcd_analyze(int32_t *samples, int count, int stride, int gain, int target_gain, int extend, int mode, int cdt_active, int tg_mismatch) +{ + static const int maxg = 0xf << 7; + int i; + int32_t *samples_end = samples + stride * count; + + for (i = 0; i < count; i++) { + samples[i * stride] <<= 15; + if (mode == HDCD_ANA_PE) { + int pel = (samples[i * stride] >> 16) & 1; + int32_t sample = samples[i * stride]; + samples[i * stride] = hdcd_analyze_gen(sample, !!(pel && extend), 1); + } else if (mode == HDCD_ANA_TGM && tg_mismatch > 0) + samples[i * stride] = hdcd_analyze_gen(samples[i * stride], 1, 1); + else if (mode == HDCD_ANA_CDT && cdt_active) + samples[i * stride] = hdcd_analyze_gen(samples[i * stride], 1, 1); + } + + if (gain <= target_gain) { + int len = FFMIN(count, target_gain - gain); + /* attenuate slowly */ + for (i = 0; i < len; i++) { + ++gain; + if (mode == HDCD_ANA_LLE) + *samples = hdcd_analyze_gen(*samples, gain, maxg); + samples += stride; + } + count -= len; + } else { + int len = FFMIN(count, (gain - target_gain) >> 3); + /* amplify quickly */ + for (i = 0; i < len; i++) { + gain -= 8; + if (mode == HDCD_ANA_LLE) + *samples = hdcd_analyze_gen(*samples, gain, maxg); + samples += stride; + } + if (gain - 8 < target_gain) + gain = target_gain; + count -= len; + } + + /* hold a steady level */ + if (gain == 0) { + if (count > 0) + samples += count * stride; + } else { + while (--count >= 0) { + if (mode == HDCD_ANA_LLE) + *samples = hdcd_analyze_gen(*samples, gain, maxg); + samples += stride; + } + } + + av_assert0(samples == samples_end); + + return gain; +} + static int hdcd_envelope(int32_t *samples, int count, int stride, int gain, int target_gain, int extend) { int i; @@ -1316,7 +1420,10 @@ static void hdcd_process(HDCDContext *ctx, hdcd_state_t *state, int32_t *samples envelope_run = run - 1; av_assert0(samples + envelope_run * stride <= samples_end); - gain = hdcd_envelope(samples, envelope_run, stride, gain, target_gain, peak_extend); + if (ctx->analyze_mode) + gain = hdcd_analyze(samples, envelope_run, stride, gain, target_gain, peak_extend, ctx->analyze_mode, state->sustain, -1); + else + gain = hdcd_envelope(samples, envelope_run, stride, gain, target_gain, peak_extend); samples += envelope_run * stride; count -= envelope_run; @@ -1325,7 +1432,10 @@ static void hdcd_process(HDCDContext *ctx, hdcd_state_t *state, int32_t *samples } if (lead > 0) { av_assert0(samples + lead * stride <= samples_end); - gain = hdcd_envelope(samples, lead, stride, gain, target_gain, peak_extend); + if (ctx->analyze_mode) + gain = hdcd_analyze(samples, lead, stride, gain, target_gain, peak_extend, ctx->analyze_mode, state->sustain, -1); + else + gain = hdcd_envelope(samples, lead, stride, gain, target_gain, peak_extend); } state->running_gain = gain; @@ -1339,7 +1449,7 @@ static void hdcd_process_stereo(HDCDContext *ctx, int32_t *samples, int count) int peak_extend[2]; int lead = 0; - hdcd_control_stereo(ctx, &peak_extend[0], &peak_extend[1]); + int ctlret = hdcd_control_stereo(ctx, &peak_extend[0], &peak_extend[1]); while (count > lead) { int envelope_run, run; @@ -1349,7 +1459,16 @@ static void hdcd_process_stereo(HDCDContext *ctx, int32_t *samples, int count) av_assert0(samples + envelope_run * stride <= samples_end); - if (envelope_run) { + if (ctx->analyze_mode) { + gain[0] = hdcd_analyze(samples, envelope_run, stride, gain[0], ctx->val_target_gain, peak_extend[0], + ctx->analyze_mode, + ctx->state[0].sustain, + (ctlret == HDCD_TG_MISMATCH) ); + gain[1] = hdcd_analyze(samples + 1, envelope_run, stride, gain[1], ctx->val_target_gain, peak_extend[1], + ctx->analyze_mode, + ctx->state[1].sustain, + (ctlret == HDCD_TG_MISMATCH) ); + } else { gain[0] = hdcd_envelope(samples, envelope_run, stride, gain[0], ctx->val_target_gain, peak_extend[0]); gain[1] = hdcd_envelope(samples + 1, envelope_run, stride, gain[1], ctx->val_target_gain, peak_extend[1]); } @@ -1358,18 +1477,32 @@ static void hdcd_process_stereo(HDCDContext *ctx, int32_t *samples, int count) count -= envelope_run; lead = run - envelope_run; - hdcd_control_stereo(ctx, &peak_extend[0], &peak_extend[1]); + ctlret = hdcd_control_stereo(ctx, &peak_extend[0], &peak_extend[1]); } if (lead > 0) { av_assert0(samples + lead * stride <= samples_end); - gain[0] = hdcd_envelope(samples, lead, stride, gain[0], ctx->val_target_gain, peak_extend[0]); - gain[1] = hdcd_envelope(samples + 1, lead, stride, gain[1], ctx->val_target_gain, peak_extend[1]); + if (ctx->analyze_mode) { + gain[0] = hdcd_analyze(samples, lead, stride, gain[0], ctx->val_target_gain, peak_extend[0], + ctx->analyze_mode, + ctx->state[0].sustain, + !!(ctlret == HDCD_TG_MISMATCH) ); + gain[1] = hdcd_analyze(samples + 1, lead, stride, gain[1], ctx->val_target_gain, peak_extend[1], + ctx->analyze_mode, + ctx->state[1].sustain, + !!(ctlret == HDCD_TG_MISMATCH) ); + } else { + gain[0] = hdcd_envelope(samples, lead, stride, gain[0], ctx->val_target_gain, peak_extend[0]); + gain[1] = hdcd_envelope(samples + 1, lead, stride, gain[1], ctx->val_target_gain, peak_extend[1]); + } } ctx->state[0].running_gain = gain[0]; ctx->state[1].running_gain = gain[1]; } +/* tone generator: sample_number, frequency, sample_rate, amplitude */ +#define TONEGEN16(sn, f, sr, a) (int16_t)(sin((6.28318530718 * (sn) * (f)) /(sr)) * (a) * 0x7fff) + static int filter_frame(AVFilterLink *inlink, AVFrame *in) { AVFilterContext *ctx = inlink->dst; @@ -1390,8 +1523,21 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) in_data = (int16_t*)in->data[0]; out_data = (int32_t*)out->data[0]; - for (n = 0; n < in->nb_samples * in->channels; n++) { + for (c = n = 0; n < in->nb_samples * in->channels; n++) { out_data[n] = in_data[n]; + if (s->analyze_mode) { + /* in analyze mode, the audio is replaced by a solid tone, and + * amplitude is changed to signal when the specified feature is + * used. + * bit 0: HDCD signal preserved + * bit 1: Original sample was above PE level */ + int32_t save = (abs(in_data[n]) - 0x5981 >= 0) ? 2 : 0; /* above PE level */ + save |= in_data[n] & 1; /* save LSB for HDCD packets */ + out_data[n] = TONEGEN16(s->ana_snb, 277.18, 44100, 0.1); + out_data[n] = (out_data[n] | 3) ^ ((~save) & 3); + if (++c == in->channels) { s->ana_snb++; c = 0; } + if (s->ana_snb > 0x3fffffff) s->ana_snb = 0; + } } s->det_errors = 0; /* re-sum every pass */ @@ -1557,6 +1703,7 @@ static av_cold int init(AVFilterContext *ctx) (s->process_stereo) ? "process stereo channels together" : "process each channel separately"); av_log(ctx, AV_LOG_VERBOSE, "Force PE: %s\n", (s->force_pe) ? "on" : "off"); + av_log(ctx, AV_LOG_VERBOSE, "Analyze mode: [%d] %s\n", s->analyze_mode, ana_mode_str[s->analyze_mode] ); return 0; }