From patchwork Fri Aug 5 22:23:04 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Burt P X-Patchwork-Id: 103 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.140.67 with SMTP id o64csp1931896vsd; Fri, 5 Aug 2016 15:23:21 -0700 (PDT) X-Received: by 10.194.97.73 with SMTP id dy9mr74005443wjb.132.1470435801621; Fri, 05 Aug 2016 15:23:21 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id n187si10495844wmf.6.2016.08.05.15.23.21; Fri, 05 Aug 2016 15:23:21 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7C34568A743; Sat, 6 Aug 2016 01:23:12 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-io0-f193.google.com (mail-io0-f193.google.com [209.85.223.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 583D268A718 for ; Sat, 6 Aug 2016 01:23:04 +0300 (EEST) Received: by mail-io0-f193.google.com with SMTP id y195so25093616iod.0 for ; Fri, 05 Aug 2016 15:23:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:subject:date:message-id:in-reply-to:references; bh=3cNAS5+ViZ9atBUGZ6YYbt6iMYPraOROlPEWPee6AeM=; b=NM9PIeqftT2/SSsJbap4avhhWliKy0+fkJNRJaDrzcytBdZHu0US8dMps594gSAckY kDeMhQITt7ITcNaJa1t4LIq/GXVi1UtC9SzvtgqnyLjwDYFESsP6XvfsSR1Zu6GhNcON fLRT7y/Um/wDMHKW1ol3ozgy1ams+Rw0LBMPfEKZfv4vXhUmIba1w+ZToezKDEWWUnKg eLpl3ZtQ7KEYFiUJMoYb4ulw5wDyDRnb8fbA7Qs23415S+qkLDwDXjUuOv0W/vQlPIm8 CGe1QSjBslyKkHntV6dclogmzLEfQqaFK1ZA2BJjCzcthcaletu7ZTQ9KwrOazUhjE4e FQhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=3cNAS5+ViZ9atBUGZ6YYbt6iMYPraOROlPEWPee6AeM=; b=Lp0jLd4+iiHeKo0aR9JPp2XnMMn/ml2QwDNKg7+NaqfrWAHUNgkapEE/hBa+JD+tsP ToIa9C7Q6LOgRJXyyHoICjy7pDQg335b0yV9N0VSwLvdROmvmKN9sxg+/6Cq+t3ZAiEN a1wZXWyfmUUj5TzcUfon5w1sQV8XoRX1XMFeE4rlFuQ0rJUuyqIuJbgMq7qK+1B7lF54 Xt3wkVnxdx3dKbcTsFWkT67wlXWDcZbZUgr0elt9go/av/LOn1mDLUf/BC7k2/vccOj/ mw0a03lSj3GO49oEBpPbOoofN51dhaS5DMncJZdRQalDp+8OtRKrL2qY2kRlCoDxTLqD 56TQ== X-Gm-Message-State: AEkoouufu2aAK2AW4rxIoKOMilTwXYHGR4qwBbAETqU3HNWRRIjJDnBMkSNDHh6ZujyiKQ== X-Received: by 10.107.55.70 with SMTP id e67mr84422160ioa.51.1470435790189; Fri, 05 Aug 2016 15:23:10 -0700 (PDT) Received: from localhost.localdomain ([216.16.66.181]) by smtp.gmail.com with ESMTPSA id n23sm8911914ioe.18.2016.08.05.15.23.09 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 05 Aug 2016 15:23:09 -0700 (PDT) From: Burt P To: ffmpeg-devel@ffmpeg.org Date: Fri, 5 Aug 2016 17:23:04 -0500 Message-Id: <1470435784-1945-1-git-send-email-pburt0@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1470434084-1323-3-git-send-email-pburt0@gmail.com> References: <1470434084-1323-3-git-send-email-pburt0@gmail.com> Subject: [FFmpeg-devel] [PATCHv2] af_hdcd: Add analyze mode X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" A new mode, selected by filter option, to aid in analysis of HDCD encoded audio. In this mode the audio is replaced by a solid tone and the amplitude is adjusted to signal some specified aspect of the process. The output file can be loaded in an audio editor alongside the original, where the user can see where different features or states are present. Signed-off-by: Burt P --- doc/filters.texi | 32 ++++++++++ libavfilter/af_hdcd.c | 164 +++++++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 188 insertions(+), 8 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 7e042e4..0d0684f 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -8454,6 +8454,38 @@ ffmpeg -i HDCD16.wav -af hdcd OUT16.wav ffmpeg -i HDCD16.wav -af hdcd -acodec pcm_s24le OUT24.wav @end example +The filter accepts the following options: + +@table @option +@item process_stereo +Process the stereo channels together. If target_gain does not match between +channels, consider it invalid and use the last valid target_gain. + +@item force_pe +Always extend peaks above -3dBFS even if PE isn't signaled. + +@item analyze_mode +Replace audio with a solid tone and adjust the amplitude to signal some +specific aspect of the decoding process. The output file can be loaded in +an audio editor alongside the original to aid analysis. + +@code{analyze_mode=pe:force_pe=1} can be used to see all samples above the PE level. + +Modes are: +@table @samp +@item 0, off +Disabled +@item 1, lle +Gain adjustment level at each sample +@item 2, pe +Samples where peak extend occurs +@item 3, cdt +Samples where the code detect timer is active +@item 4, tgm +Samples where the target gain does not match between channels +@end table +@end table + @section hflip Flip the input video horizontally. diff --git a/libavfilter/af_hdcd.c b/libavfilter/af_hdcd.c index 610dd9e..edf3138 100644 --- a/libavfilter/af_hdcd.c +++ b/libavfilter/af_hdcd.c @@ -870,6 +870,27 @@ static const char * const pe_str[] = { * the always-negative value is stored positive, so make it negative */ #define GAINTOFLOAT(g) (g) ? -(float)(g>>1) - ((g & 1) ? 0.5 : 0.0) : 0.0 +#define HDCD_ANA_OFF 0 +#define HDCD_ANA_OFF_DESC "disabled" +#define HDCD_ANA_LLE 1 +#define HDCD_ANA_LLE_DESC "gain adjustment level at each sample" +#define HDCD_ANA_PE 2 +#define HDCD_ANA_PE_DESC "samples where peak extend occurs" +#define HDCD_ANA_CDT 3 +#define HDCD_ANA_CDT_DESC "samples where the code detect timer is active" +#define HDCD_ANA_TGM 4 +#define HDCD_ANA_TGM_DESC "samples where the target gain does not match between channels" +#define HDCD_ANA_TOP 5 /* used in max value of AVOption */ +typedef int hdcd_ana_mode_t; /* formerly enum, but that didn't work for AVOption initialization */ + +static const char * const ana_mode_str[] = { + HDCD_ANA_OFF_DESC, + HDCD_ANA_LLE_DESC, + HDCD_ANA_PE_DESC, + HDCD_ANA_CDT_DESC, + HDCD_ANA_TGM_DESC, +}; + typedef struct HDCDContext { const AVClass *class; hdcd_state_t state[HDCD_MAX_CHANNELS]; @@ -885,6 +906,12 @@ typedef struct HDCDContext { * default is off */ int force_pe; + /* analyze mode replaces the audio with a solid tone and adjusts + * the amplitude to signal some specific aspect of the decoding + * process. See docs or hdcd_ana_mode_t */ + hdcd_ana_mode_t analyze_mode; + int ana_snb; /* used in tone generation */ + /* config_input() and config_output() scan links for any resampling * or format changes. If found, warnings are issued and bad_config * is set. */ @@ -909,6 +936,13 @@ static const AVOption hdcd_options[] = { OFFSET(process_stereo), AV_OPT_TYPE_BOOL, { .i64 = HDCD_PROCESS_STEREO_DEFAULT }, 0, 1, A }, { "force_pe", "Always extend peaks above -3dBFS even when PE is not signaled.", OFFSET(force_pe), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, A }, + { "analyze_mode", "Replace audio with solid tone and signal some processing aspect in the amplitude.", + OFFSET(analyze_mode), AV_OPT_TYPE_INT, { .i64=HDCD_ANA_OFF }, 0, HDCD_ANA_TOP-1, A, "analyze_mode"}, + { "off", HDCD_ANA_OFF_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_OFF}, 0, 0, A, "analyze_mode" }, + { "lle", HDCD_ANA_LLE_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_LLE}, 0, 0, A, "analyze_mode" }, + { "pe", HDCD_ANA_PE_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_PE}, 0, 0, A, "analyze_mode" }, + { "cdt", HDCD_ANA_CDT_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_CDT}, 0, 0, A, "analyze_mode" }, + { "tgm", HDCD_ANA_TGM_DESC, 0, AV_OPT_TYPE_CONST, {.i64=HDCD_ANA_TGM}, 0, 0, A, "analyze_mode" }, {NULL} }; @@ -1209,6 +1243,77 @@ static int hdcd_scan_stereo(HDCDContext *ctx, const int32_t *samples, int max) return result; } +/* encode a value in the given sample by adjusting the amplitude */ +static int32_t hdcd_analyze_gen(int32_t sample, unsigned int v, unsigned int maxv) +{ + float sflt = sample, vv = v; + vv /= maxv; + if (vv > 1.0) vv = 1.0; + sflt *= 1.0 + (vv * 18); + return (int32_t)sflt; +} + +/* behaves like hdcd_envelope(), but encodes processing information in + * a way that is audible (and visible in an audio editor) to aid analysis. */ +static int hdcd_analyze(int32_t *samples, int count, int stride, int gain, int target_gain, int extend, hdcd_ana_mode_t mode, int cdt_active, int tg_mismatch) +{ + static const int maxg = 0xf << 7; + int i; + int32_t *samples_end = samples + stride * count; + + for (i = 0; i < count; i++) { + samples[i * stride] <<= 15; + if (mode == HDCD_ANA_PE) { + int pel = (samples[i * stride] >> 16) & 1; + int32_t sample = samples[i * stride]; + samples[i * stride] = hdcd_analyze_gen(sample, !!(pel && extend), 1); + } else if (mode == HDCD_ANA_TGM && tg_mismatch > 0) + samples[i * stride] = hdcd_analyze_gen(samples[i * stride], 1, 1); + else if (mode == HDCD_ANA_CDT && cdt_active) + samples[i * stride] = hdcd_analyze_gen(samples[i * stride], 1, 1); + } + + if (gain <= target_gain) { + int len = FFMIN(count, target_gain - gain); + /* attenuate slowly */ + for (i = 0; i < len; i++) { + ++gain; + if (mode == HDCD_ANA_LLE) + *samples = hdcd_analyze_gen(*samples, gain, maxg); + samples += stride; + } + count -= len; + } else { + int len = FFMIN(count, (gain - target_gain) >> 3); + /* amplify quickly */ + for (i = 0; i < len; i++) { + gain -= 8; + if (mode == HDCD_ANA_LLE) + *samples = hdcd_analyze_gen(*samples, gain, maxg); + samples += stride; + } + if (gain - 8 < target_gain) + gain = target_gain; + count -= len; + } + + /* hold a steady level */ + if (gain == 0) { + if (count > 0) + samples += count * stride; + } else { + while (--count >= 0) { + if (mode == HDCD_ANA_LLE) + *samples = hdcd_analyze_gen(*samples, gain, maxg); + samples += stride; + } + } + + av_assert0(samples == samples_end); + + return gain; +} + static int hdcd_envelope(int32_t *samples, int count, int stride, int gain, int target_gain, int extend) { int i; @@ -1316,7 +1421,10 @@ static void hdcd_process(HDCDContext *ctx, hdcd_state_t *state, int32_t *samples envelope_run = run - 1; av_assert0(samples + envelope_run * stride <= samples_end); - gain = hdcd_envelope(samples, envelope_run, stride, gain, target_gain, peak_extend); + if (ctx->analyze_mode) + gain = hdcd_analyze(samples, envelope_run, stride, gain, target_gain, peak_extend, ctx->analyze_mode, state->sustain, -1); + else + gain = hdcd_envelope(samples, envelope_run, stride, gain, target_gain, peak_extend); samples += envelope_run * stride; count -= envelope_run; @@ -1325,7 +1433,10 @@ static void hdcd_process(HDCDContext *ctx, hdcd_state_t *state, int32_t *samples } if (lead > 0) { av_assert0(samples + lead * stride <= samples_end); - gain = hdcd_envelope(samples, lead, stride, gain, target_gain, peak_extend); + if (ctx->analyze_mode) + gain = hdcd_analyze(samples, lead, stride, gain, target_gain, peak_extend, ctx->analyze_mode, state->sustain, -1); + else + gain = hdcd_envelope(samples, lead, stride, gain, target_gain, peak_extend); } state->running_gain = gain; @@ -1339,7 +1450,7 @@ static void hdcd_process_stereo(HDCDContext *ctx, int32_t *samples, int count) int peak_extend[2]; int lead = 0; - hdcd_control_stereo(ctx, &peak_extend[0], &peak_extend[1]); + int ctlret = hdcd_control_stereo(ctx, &peak_extend[0], &peak_extend[1]); while (count > lead) { int envelope_run, run; @@ -1349,7 +1460,16 @@ static void hdcd_process_stereo(HDCDContext *ctx, int32_t *samples, int count) av_assert0(samples + envelope_run * stride <= samples_end); - if (envelope_run) { + if (ctx->analyze_mode) { + gain[0] = hdcd_analyze(samples, envelope_run, stride, gain[0], ctx->val_target_gain, peak_extend[0], + ctx->analyze_mode, + ctx->state[0].sustain, + !!(ctlret == HDCD_TG_MISMATCH) ); + gain[1] = hdcd_analyze(samples + 1, envelope_run, stride, gain[1], ctx->val_target_gain, peak_extend[1], + ctx->analyze_mode, + ctx->state[1].sustain, + !!(ctlret == HDCD_TG_MISMATCH) ); + } else { gain[0] = hdcd_envelope(samples, envelope_run, stride, gain[0], ctx->val_target_gain, peak_extend[0]); gain[1] = hdcd_envelope(samples + 1, envelope_run, stride, gain[1], ctx->val_target_gain, peak_extend[1]); } @@ -1358,18 +1478,32 @@ static void hdcd_process_stereo(HDCDContext *ctx, int32_t *samples, int count) count -= envelope_run; lead = run - envelope_run; - hdcd_control_stereo(ctx, &peak_extend[0], &peak_extend[1]); + ctlret = hdcd_control_stereo(ctx, &peak_extend[0], &peak_extend[1]); } if (lead > 0) { av_assert0(samples + lead * stride <= samples_end); - gain[0] = hdcd_envelope(samples, lead, stride, gain[0], ctx->val_target_gain, peak_extend[0]); - gain[1] = hdcd_envelope(samples + 1, lead, stride, gain[1], ctx->val_target_gain, peak_extend[1]); + if (ctx->analyze_mode) { + gain[0] = hdcd_analyze(samples, lead, stride, gain[0], ctx->val_target_gain, peak_extend[0], + ctx->analyze_mode, + ctx->state[0].sustain, + !!(ctlret == HDCD_TG_MISMATCH) ); + gain[1] = hdcd_analyze(samples + 1, lead, stride, gain[1], ctx->val_target_gain, peak_extend[1], + ctx->analyze_mode, + ctx->state[1].sustain, + !!(ctlret == HDCD_TG_MISMATCH) ); + } else { + gain[0] = hdcd_envelope(samples, lead, stride, gain[0], ctx->val_target_gain, peak_extend[0]); + gain[1] = hdcd_envelope(samples + 1, lead, stride, gain[1], ctx->val_target_gain, peak_extend[1]); + } } ctx->state[0].running_gain = gain[0]; ctx->state[1].running_gain = gain[1]; } +/* tone generator: sample_number, frequency, sample_rate, amplitude */ +#define TONEGEN16(sn, f, sr, a) (int16_t)(sin((6.28318530718 * sn * f) /sr) * a * 0x7fff) + static int filter_frame(AVFilterLink *inlink, AVFrame *in) { AVFilterContext *ctx = inlink->dst; @@ -1390,8 +1524,21 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) in_data = (int16_t*)in->data[0]; out_data = (int32_t*)out->data[0]; - for (n = 0; n < in->nb_samples * in->channels; n++) { + for (c = n = 0; n < in->nb_samples * in->channels; n++) { out_data[n] = in_data[n]; + if (s->analyze_mode) { + /* in analyze mode, the audio is replaced by a solid tone, and + * amplitude is changed to signal when the specified feature is + * used. + * bit 0: HDCD signal preserved + * bit 1: Original sample was above PE level */ + int32_t save = (abs(in_data[n]) - 0x5981 >= 0) ? 2 : 0; /* above PE level */ + save |= in_data[n] & 1; /* save LSB for HDCD packets */ + out_data[n] = TONEGEN16(s->ana_snb, 277.18, 44100, 0.1); + out_data[n] = (out_data[n] | 3) ^ ((~save) & 3); + if (++c == in->channels) { s->ana_snb++; c = 0; } + if (s->ana_snb > 0x3fffffff) s->ana_snb = 0; + } } s->det_errors = 0; /* re-sum every pass */ @@ -1557,6 +1704,7 @@ static av_cold int init(AVFilterContext *ctx) (s->process_stereo) ? "process stereo channels together" : "process each channel separately"); av_log(ctx, AV_LOG_VERBOSE, "Force PE: %s\n", (s->force_pe) ? "on" : "off"); + av_log(ctx, AV_LOG_VERBOSE, "Analyze mode: [%d] %s\n", s->analyze_mode, ana_mode_str[s->analyze_mode] ); return 0; }