From patchwork Sat Aug 6 12:47:18 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Burt P X-Patchwork-Id: 110 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.140.67 with SMTP id o64csp2187754vsd; Sat, 6 Aug 2016 06:00:17 -0700 (PDT) X-Received: by 10.194.39.3 with SMTP id l3mr87149329wjk.133.1470488417417; Sat, 06 Aug 2016 06:00:17 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x25si12774315wma.24.2016.08.06.06.00.16; Sat, 06 Aug 2016 06:00:17 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 464A868A7B5; Sat, 6 Aug 2016 16:00:00 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-it0-f68.google.com (mail-it0-f68.google.com [209.85.214.68]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 87E6B68A54A for ; Sat, 6 Aug 2016 15:59:46 +0300 (EEST) Received: by mail-it0-f68.google.com with SMTP id j124so3373759ith.3 for ; Sat, 06 Aug 2016 05:59:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:subject:date:message-id:in-reply-to:references; bh=U7vR38DAjPn5wJlrs13CLd7/dEqxjjnCoSMsOfoTlJA=; b=SVOKW76cWREnQ4E6j/a7vM2onCcocLLYMfVQyqi51HymM05bNrUNOdVZCOcN2r8N1w YL0Fqwwj1ethXMvtvFkLYm2tc41jIBfKV0HLt0zJPeXKjeGeycr1iTymOWU1h5RjzaI+ NM+RV4ZByWtjlN+IjBGn48oOBeXTznk4noafq+ScdfXR0DhbmPG9cKS8PhQ03ADGcAnn 0vSQoP1PIMc6uqkY2D6zbMdIVFN8inBQwFX5QLwX+/SXJdhEYrzjf7npwjA6KKF5DH0T eqGa5bm0cCqYf6q6soYkUr6ZtYtGyHeMJFm8hldgzZtWUfwleZoqf3abUjPGF5Y8u+0z GWjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=U7vR38DAjPn5wJlrs13CLd7/dEqxjjnCoSMsOfoTlJA=; b=IvUcFXfU4bdy9J/9366pJkM4PhdWMHtbx6wO027rboLVySNOdLzvwiPmyu7zTozc98 EzJRLXqScsUf3V5Reve/t1aPSgK3DjkQrD7wC7x+PBNP6E2PAOthwybOeld6SnyPed1K WRfA6RUL8bln/sUj6YJWUqFaFf+yw7cMM4oeHOkrB6xbRI8pmAEZZl9FocVkEXKTK7j6 tv4K3/WTGGh2zWUOotBWtBKZ8XG+CswDKLL1q0veilkadDl60S5mU+dHYNyS4MnLtgNT 9iMT+6hPXHPg6Aw15oHMmgeJzBl5XCb7EoTQqxBxg7trBF5T0FZnNzgeJ1mYmxAfGAoN ljUg== X-Gm-Message-State: AEkooutqYdbZWkV2yNAkwimQjKqAkDccxE3PduqfrhrsfmUZYiiGWwoQDqv7A8AT+873sQ== X-Received: by 10.36.150.70 with SMTP id z67mr8864627itd.80.1470487644093; Sat, 06 Aug 2016 05:47:24 -0700 (PDT) Received: from localhost.localdomain ([216.16.66.181]) by smtp.gmail.com with ESMTPSA id c40sm10138282iod.6.2016.08.06.05.47.23 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 06 Aug 2016 05:47:23 -0700 (PDT) From: Burt P To: ffmpeg-devel@ffmpeg.org Date: Sat, 6 Aug 2016 07:47:18 -0500 Message-Id: <1470487638-2709-1-git-send-email-pburt0@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1470435784-1945-1-git-send-email-pburt0@gmail.com> References: <1470435784-1945-1-git-send-email-pburt0@gmail.com> Subject: [FFmpeg-devel] [PATCHv3] af_hdcd: Add analyze mode (v3) X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" A new mode, selected by filter option, to aid in analysis of HDCD encoded audio. In this mode the audio is replaced by a solid tone and the amplitude is adjusted to signal some specified aspect of the process. The output file can be loaded in an audio editor alongside the original, where the user can see where different features or states are present. Signed-off-by: Burt P --- doc/filters.texi | 32 ++++++++++ libavfilter/af_hdcd.c | 164 +++++++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 188 insertions(+), 8 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 7e042e4..0d0684f 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -8454,6 +8454,38 @@ ffmpeg -i HDCD16.wav -af hdcd OUT16.wav ffmpeg -i HDCD16.wav -af hdcd -acodec pcm_s24le OUT24.wav @end example +The filter accepts the following options: + +@table @option +@item process_stereo +Process the stereo channels together. If target_gain does not match between +channels, consider it invalid and use the last valid target_gain. + +@item force_pe +Always extend peaks above -3dBFS even if PE isn't signaled. + +@item analyze_mode +Replace audio with a solid tone and adjust the amplitude to signal some +specific aspect of the decoding process. The output file can be loaded in +an audio editor alongside the original to aid analysis. + +@code{analyze_mode=pe:force_pe=1} can be used to see all samples above the PE level. + +Modes are: +@table @samp +@item 0, off +Disabled +@item 1, lle +Gain adjustment level at each sample +@item 2, pe +Samples where peak extend occurs +@item 3, cdt +Samples where the code detect timer is active +@item 4, tgm +Samples where the target gain does not match between channels +@end table +@end table + @section hflip Flip the input video horizontally. diff --git a/libavfilter/af_hdcd.c b/libavfilter/af_hdcd.c index 610dd9e..ada30f4 100644 --- a/libavfilter/af_hdcd.c +++ b/libavfilter/af_hdcd.c @@ -870,6 +870,27 @@ static const char * const pe_str[] = { * the always-negative value is stored positive, so make it negative */ #define GAINTOFLOAT(g) (g) ? -(float)(g>>1) - ((g & 1) ? 0.5 : 0.0) : 0.0 +#define HDCD_ANA_OFF 0 +#define HDCD_ANA_OFF_DESC "disabled" +#define HDCD_ANA_LLE 1 +#define HDCD_ANA_LLE_DESC "gain adjustment level at each sample" +#define HDCD_ANA_PE 2 +#define HDCD_ANA_PE_DESC "samples where peak extend occurs" +#define HDCD_ANA_CDT 3 +#define HDCD_ANA_CDT_DESC "samples where the code detect timer is active" +#define HDCD_ANA_TGM 4 +#define HDCD_ANA_TGM_DESC "samples where the target gain does not match between channels" +#define HDCD_ANA_TOP 5 /* used in max value of AVOption */ +typedef int hdcd_ana_mode_t; /* formerly enum, but that didn't work for AVOption initialization */ + +static const char * const ana_mode_str[] = { + HDCD_ANA_OFF_DESC, + HDCD_ANA_LLE_DESC, + HDCD_ANA_PE_DESC, + HDCD_ANA_CDT_DESC, + HDCD_ANA_TGM_DESC, +}; + typedef struct HDCDContext { const AVClass *class; hdcd_state_t state[HDCD_MAX_CHANNELS]; @@ -885,6 +906,12 @@ typedef struct HDCDContext { * default is off */ int force_pe; + /* analyze mode replaces the audio with a solid tone and adjusts + * the amplitude to signal some specific aspect of the decoding + * process. See docs or hdcd_ana_mode_t */ + hdcd_ana_mode_t analyze_mode; + int ana_snb; /* used in tone generation */ + /* config_input() and config_output() scan links for any resampling * or format changes. If found, warnings are issued and bad_config * is set. */ @@ -909,6 +936,13 @@ static const AVOption hdcd_options[] = { OFFSET(process_stereo), AV_OPT_TYPE_BOOL, { .i64 = HDCD_PROCESS_STEREO_DEFAULT }, 0, 1, A }, { "force_pe", "Always extend peaks above -3dBFS even when PE is not signaled.", OFFSET(force_pe), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, A }, + { "analyze_mode", "Replace audio with solid tone and signal some processing aspect in the amplitude.", + OFFSET(analyze_mode), AV_OPT_TYPE_INT, { .i64=HDCD_ANA_OFF }, 0, HDCD_ANA_TOP-1, A, "analyze_mode"}, + { "off", HDCD_ANA_OFF_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_OFF}, 0, 0, A, "analyze_mode" }, + { "lle", HDCD_ANA_LLE_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_LLE}, 0, 0, A, "analyze_mode" }, + { "pe", HDCD_ANA_PE_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_PE}, 0, 0, A, "analyze_mode" }, + { "cdt", HDCD_ANA_CDT_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_CDT}, 0, 0, A, "analyze_mode" }, + { "tgm", HDCD_ANA_TGM_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_TGM}, 0, 0, A, "analyze_mode" }, {NULL} }; @@ -1209,6 +1243,77 @@ static int hdcd_scan_stereo(HDCDContext *ctx, const int32_t *samples, int max) return result; } +/* encode a value in the given sample by adjusting the amplitude */ +static int32_t hdcd_analyze_gen(int32_t sample, unsigned int v, unsigned int maxv) +{ + float sflt = sample, vv = v; + vv /= maxv; + if (vv > 1.0) vv = 1.0; + sflt *= 1.0 + (vv * 18); + return (int32_t)sflt; +} + +/* behaves like hdcd_envelope(), but encodes processing information in + * a way that is audible (and visible in an audio editor) to aid analysis. */ +static int hdcd_analyze(int32_t *samples, int count, int stride, int gain, int target_gain, int extend, hdcd_ana_mode_t mode, int cdt_active, int tg_mismatch) +{ + static const int maxg = 0xf << 7; + int i; + int32_t *samples_end = samples + stride * count; + + for (i = 0; i < count; i++) { + samples[i * stride] <<= 15; + if (mode == HDCD_ANA_PE) { + int pel = (samples[i * stride] >> 16) & 1; + int32_t sample = samples[i * stride]; + samples[i * stride] = hdcd_analyze_gen(sample, !!(pel && extend), 1); + } else if (mode == HDCD_ANA_TGM && tg_mismatch > 0) + samples[i * stride] = hdcd_analyze_gen(samples[i * stride], 1, 1); + else if (mode == HDCD_ANA_CDT && cdt_active) + samples[i * stride] = hdcd_analyze_gen(samples[i * stride], 1, 1); + } + + if (gain <= target_gain) { + int len = FFMIN(count, target_gain - gain); + /* attenuate slowly */ + for (i = 0; i < len; i++) { + ++gain; + if (mode == HDCD_ANA_LLE) + *samples = hdcd_analyze_gen(*samples, gain, maxg); + samples += stride; + } + count -= len; + } else { + int len = FFMIN(count, (gain - target_gain) >> 3); + /* amplify quickly */ + for (i = 0; i < len; i++) { + gain -= 8; + if (mode == HDCD_ANA_LLE) + *samples = hdcd_analyze_gen(*samples, gain, maxg); + samples += stride; + } + if (gain - 8 < target_gain) + gain = target_gain; + count -= len; + } + + /* hold a steady level */ + if (gain == 0) { + if (count > 0) + samples += count * stride; + } else { + while (--count >= 0) { + if (mode == HDCD_ANA_LLE) + *samples = hdcd_analyze_gen(*samples, gain, maxg); + samples += stride; + } + } + + av_assert0(samples == samples_end); + + return gain; +} + static int hdcd_envelope(int32_t *samples, int count, int stride, int gain, int target_gain, int extend) { int i; @@ -1316,7 +1421,10 @@ static void hdcd_process(HDCDContext *ctx, hdcd_state_t *state, int32_t *samples envelope_run = run - 1; av_assert0(samples + envelope_run * stride <= samples_end); - gain = hdcd_envelope(samples, envelope_run, stride, gain, target_gain, peak_extend); + if (ctx->analyze_mode) + gain = hdcd_analyze(samples, envelope_run, stride, gain, target_gain, peak_extend, ctx->analyze_mode, state->sustain, -1); + else + gain = hdcd_envelope(samples, envelope_run, stride, gain, target_gain, peak_extend); samples += envelope_run * stride; count -= envelope_run; @@ -1325,7 +1433,10 @@ static void hdcd_process(HDCDContext *ctx, hdcd_state_t *state, int32_t *samples } if (lead > 0) { av_assert0(samples + lead * stride <= samples_end); - gain = hdcd_envelope(samples, lead, stride, gain, target_gain, peak_extend); + if (ctx->analyze_mode) + gain = hdcd_analyze(samples, lead, stride, gain, target_gain, peak_extend, ctx->analyze_mode, state->sustain, -1); + else + gain = hdcd_envelope(samples, lead, stride, gain, target_gain, peak_extend); } state->running_gain = gain; @@ -1339,7 +1450,7 @@ static void hdcd_process_stereo(HDCDContext *ctx, int32_t *samples, int count) int peak_extend[2]; int lead = 0; - hdcd_control_stereo(ctx, &peak_extend[0], &peak_extend[1]); + int ctlret = hdcd_control_stereo(ctx, &peak_extend[0], &peak_extend[1]); while (count > lead) { int envelope_run, run; @@ -1349,7 +1460,16 @@ static void hdcd_process_stereo(HDCDContext *ctx, int32_t *samples, int count) av_assert0(samples + envelope_run * stride <= samples_end); - if (envelope_run) { + if (ctx->analyze_mode) { + gain[0] = hdcd_analyze(samples, envelope_run, stride, gain[0], ctx->val_target_gain, peak_extend[0], + ctx->analyze_mode, + ctx->state[0].sustain, + !!(ctlret == HDCD_TG_MISMATCH) ); + gain[1] = hdcd_analyze(samples + 1, envelope_run, stride, gain[1], ctx->val_target_gain, peak_extend[1], + ctx->analyze_mode, + ctx->state[1].sustain, + !!(ctlret == HDCD_TG_MISMATCH) ); + } else { gain[0] = hdcd_envelope(samples, envelope_run, stride, gain[0], ctx->val_target_gain, peak_extend[0]); gain[1] = hdcd_envelope(samples + 1, envelope_run, stride, gain[1], ctx->val_target_gain, peak_extend[1]); } @@ -1358,18 +1478,32 @@ static void hdcd_process_stereo(HDCDContext *ctx, int32_t *samples, int count) count -= envelope_run; lead = run - envelope_run; - hdcd_control_stereo(ctx, &peak_extend[0], &peak_extend[1]); + ctlret = hdcd_control_stereo(ctx, &peak_extend[0], &peak_extend[1]); } if (lead > 0) { av_assert0(samples + lead * stride <= samples_end); - gain[0] = hdcd_envelope(samples, lead, stride, gain[0], ctx->val_target_gain, peak_extend[0]); - gain[1] = hdcd_envelope(samples + 1, lead, stride, gain[1], ctx->val_target_gain, peak_extend[1]); + if (ctx->analyze_mode) { + gain[0] = hdcd_analyze(samples, lead, stride, gain[0], ctx->val_target_gain, peak_extend[0], + ctx->analyze_mode, + ctx->state[0].sustain, + !!(ctlret == HDCD_TG_MISMATCH) ); + gain[1] = hdcd_analyze(samples + 1, lead, stride, gain[1], ctx->val_target_gain, peak_extend[1], + ctx->analyze_mode, + ctx->state[1].sustain, + !!(ctlret == HDCD_TG_MISMATCH) ); + } else { + gain[0] = hdcd_envelope(samples, lead, stride, gain[0], ctx->val_target_gain, peak_extend[0]); + gain[1] = hdcd_envelope(samples + 1, lead, stride, gain[1], ctx->val_target_gain, peak_extend[1]); + } } ctx->state[0].running_gain = gain[0]; ctx->state[1].running_gain = gain[1]; } +/* tone generator: sample_number, frequency, sample_rate, amplitude */ +#define TONEGEN16(sn, f, sr, a) (int16_t)(sin((6.28318530718 * (sn) * (f)) /(sr)) * (a) * 0x7fff) + static int filter_frame(AVFilterLink *inlink, AVFrame *in) { AVFilterContext *ctx = inlink->dst; @@ -1390,8 +1524,21 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) in_data = (int16_t*)in->data[0]; out_data = (int32_t*)out->data[0]; - for (n = 0; n < in->nb_samples * in->channels; n++) { + for (c = n = 0; n < in->nb_samples * in->channels; n++) { out_data[n] = in_data[n]; + if (s->analyze_mode) { + /* in analyze mode, the audio is replaced by a solid tone, and + * amplitude is changed to signal when the specified feature is + * used. + * bit 0: HDCD signal preserved + * bit 1: Original sample was above PE level */ + int32_t save = (abs(in_data[n]) - 0x5981 >= 0) ? 2 : 0; /* above PE level */ + save |= in_data[n] & 1; /* save LSB for HDCD packets */ + out_data[n] = TONEGEN16(s->ana_snb, 277.18, 44100, 0.1); + out_data[n] = (out_data[n] | 3) ^ ((~save) & 3); + if (++c == in->channels) { s->ana_snb++; c = 0; } + if (s->ana_snb > 0x3fffffff) s->ana_snb = 0; + } } s->det_errors = 0; /* re-sum every pass */ @@ -1557,6 +1704,7 @@ static av_cold int init(AVFilterContext *ctx) (s->process_stereo) ? "process stereo channels together" : "process each channel separately"); av_log(ctx, AV_LOG_VERBOSE, "Force PE: %s\n", (s->force_pe) ? "on" : "off"); + av_log(ctx, AV_LOG_VERBOSE, "Analyze mode: [%d] %s\n", s->analyze_mode, ana_mode_str[s->analyze_mode] ); return 0; }