From patchwork Sat May 20 20:17:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthias Walliczek X-Patchwork-Id: 41750 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:ba91:b0:105:feb:71f2 with SMTP id fb17csp738853pzb; Sat, 20 May 2023 13:17:39 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4KMBtYYR8GewRPZUxzNq5ZNHp0NziBK6P17WmcdYgwBnnqCCEjpuRmKGwDhodqTiROl90D X-Received: by 2002:a05:6402:8d0:b0:50d:9b59:4336 with SMTP id d16-20020a05640208d000b0050d9b594336mr4871097edz.37.1684613858884; Sat, 20 May 2023 13:17:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684613858; cv=none; d=google.com; s=arc-20160816; b=NrZqyylZpGFGrmgPeC/93vFOQtNFqT4eICvfJ/bWoiOKyW3iqxRHpe/MIuUp+X3ZlH FuMkldvpzz1/ElnuylXrdB4TBi+xahsAETKFY3ybLYg3JuOeENmnt5RRkVHJCSmKcKUi a7L0gLqvJYOeRKVgiOuUu8mxTx3iEJasDtPxG/AiXWViijR6wL0N0Da04IXdG8L3XLZF aqLZxlWuE+o8FvAC3l/WX+z6+p+huOERIp49/GstRb5mxK413MmAVfxst94qW0+feV6F WM3jgGlTIU+wIoY0T0uruzzgnjkdq4x5QQlGleDPlEfWrnii9P6MHSoeXzhnk8NRCQtn BwsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:content-language:to:from:user-agent:mime-version :date:message-id:dkim-signature:delivered-to; bh=FnhJ7sD/EnalEg0KH1duKnz+1HsrWTB9jHb69E6BHJ4=; b=X139Gq7EE0aP2fEgM6T03z+lkl2YM4QNvBt8undObnkyb3LuHZgKH9Y956AzGkcVHk fKbtOFMUBWgnh8a7/NHztc3eSf7mG1DtLXyj7uo/0uRzH+Np+G5481Q0HhQM4bKIZHlS qOpZ1W3plf8y5M5pMSlgmHqkuNNyRNjXnuLWAeVzYy+XeRqLZRQe+KqRqtvKMQmq/llZ H7mC2md242PtyFox9SxMNYJgK+Nua/P39GFziflKNchX9LsMgGjq7vlG24kNABmnfZ/3 uJouBb8BfrL+yM6i+8SrVaadKYp/dsVqd70OwV9OS1ZwITOUqLKo68W3UrOax8EmZKhc IsjQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@walliczek.de header.s=mail header.b=lzLW3NSB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=NONE dis=NONE) header.from=walliczek.de Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id w8-20020a056402070800b0050bf820398asi1827902edx.371.2023.05.20.13.17.38; Sat, 20 May 2023 13:17:38 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@walliczek.de header.s=mail header.b=lzLW3NSB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=NONE dis=NONE) header.from=walliczek.de Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CBB6268C145; Sat, 20 May 2023 23:17:34 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.walliczek.de (mail.walliczek.de [78.47.132.121]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 43DA268C0AE for ; Sat, 20 May 2023 23:17:28 +0300 (EEST) Received: from [192.168.178.91] (ip-176-199-209-061.um44.pools.vodafone-ip.de [176.199.209.61]) by mail.walliczek.de (Postfix) with ESMTPSA id CFB941FFAF for ; Sat, 20 May 2023 22:17:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=walliczek.de; s=mail; t=1684613846; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=xIgkCoeU2JCjCbDt6vGQ2sS23WIqLu3JLfL7CBxZkJ0=; b=lzLW3NSBw8EXcj/qkIlu4IOe8L7ZzyIf7ynDgs8ClyGbIqd14lrOIzi0ZkNdqI9vywmBor l67dYiN6nil1a8ncDM2H9+SdkmzPo4BT9E7BfnpApIBsURyB1iDikBOvbzSCgDMsExzQAt gUvpKXZIk97bQWAW8hX32jMFiLNRi/E= Message-ID: <5fd9b52d-9b88-30f0-0f8e-92e229026636@walliczek.de> Date: Sat, 20 May 2023 22:17:24 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 From: Matthias Walliczek To: ffmpeg-devel@ffmpeg.org Content-Language: en-US Subject: [FFmpeg-devel] [PATCH v2] avcodec/aacdec: add support for AAC SBR with 960 frame length X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 0KZrj4mmb9fw Supports the european DAB+ digital radio coding Fixes: https://trac.ffmpeg.org/ticket/1407 "HE-AAC (v2): 960/120 MDCT window is not implemented" Co-developed-by: Lynne Signed-off-by: Matthias Walliczek --- libavcodec/aacdec_template.c | 17 ++------------ libavcodec/aacps.c | 8 ++++--- libavcodec/aacps.h | 7 +++--- libavcodec/aacps_common.c | 3 ++- libavcodec/aacsbr_template.c | 43 +++++++++++++++++++++--------------- libavcodec/sbr.h | 2 +- 6 files changed, 38 insertions(+), 42 deletions(-) INTFLOAT (*alpha0)[2], INTFLOAT (*alpha1)[2], const INTFLOAT X_low[32][40][2], int k0); diff --git a/libavcodec/aacdec_template.c b/libavcodec/aacdec_template.c index 444dc4fa9d..72446537f1 100644 --- a/libavcodec/aacdec_template.c +++ b/libavcodec/aacdec_template.c @@ -849,12 +849,6 @@ static int decode_ga_specific_config(AACContext *ac, AVCodecContext *avctx, int tags = 0; m4ac->frame_length_short = get_bits1(gb); - if (m4ac->frame_length_short && m4ac->sbr == 1) { - avpriv_report_missing_feature(avctx, "SBR with 960 frame length"); - if (ac) ac->warned_960_sbr = 1; - m4ac->sbr = 0; - m4ac->ps = 0; - } if (get_bits1(gb)) // dependsOnCoreCoder skip_bits(gb, 14); // coreCoderDelay @@ -1047,10 +1041,10 @@ static int decode_audio_specific_config_gb(AACContext *ac, } ff_dlog(avctx, - "AOT %d chan config %d sampling index %d (%d) SBR %d PS %d\n", + "AOT %d chan config %d sampling index %d (%d) SBR %d PS %d frame length %d\n", m4ac->object_type, m4ac->chan_config, m4ac->sampling_index, m4ac->sample_rate, m4ac->sbr, - m4ac->ps); + m4ac->ps, m4ac->frame_length_short ? 960 : 1024); return get_bits_count(gb); } @@ -2467,13 +2461,6 @@ static int decode_extension_payload(AACContext *ac, GetBitContext *gb, int cnt, if (!che) { av_log(ac->avctx, AV_LOG_ERROR, "SBR was found before the first channel element.\n"); return res; - } else if (ac->oc[1].m4ac.frame_length_short) { - if (!ac->warned_960_sbr) - avpriv_report_missing_feature(ac->avctx, - "SBR with 960 frame length"); - ac->warned_960_sbr = 1; - skip_bits_long(gb, 8 * cnt - 4); - return res; } else if (!ac->oc[1].m4ac.sbr) { av_log(ac->avctx, AV_LOG_ERROR, "SBR signaled to be not-present but was found in the bitstream.\n"); skip_bits_long(gb, 8 * cnt - 4); diff --git a/libavcodec/aacps.c b/libavcodec/aacps.c index 655e8fe5b4..4804722e59 100644 --- a/libavcodec/aacps.c +++ b/libavcodec/aacps.c @@ -397,7 +397,7 @@ static void map_val_20_to_34(INTFLOAT par[PS_MAX_NR_IIDICC]) par[ 1] = AAC_HALF_SUM(par[ 0], par[ 1]); } -static void decorrelation(PSContext *ps, INTFLOAT (*out)[32][2], const INTFLOAT (*s)[32][2], int is34) +static void decorrelation(PSContext *ps, INTFLOAT (*out)[32][2], const INTFLOAT (*s)[32][2], int is34, int frame_length_short) { LOCAL_ALIGNED_16(INTFLOAT, power, [34], [PS_QMF_TIME_SLOTS]); LOCAL_ALIGNED_16(INTFLOAT, transient_gain, [34], [PS_QMF_TIME_SLOTS]); @@ -414,6 +414,7 @@ static void decorrelation(PSContext *ps, INTFLOAT (*out)[32][2], const INTFLOAT int i, k, m, n; int n0 = 0, nL = 32; const INTFLOAT peak_decay_factor = Q31(0.76592833836465f); + const int numQMFSlots = frame_length_short ? 30 : 32; memset(power, 0, 34 * sizeof(*power)); @@ -717,7 +718,8 @@ static void stereo_processing(PSContext *ps, INTFLOAT (*l)[32][2], INTFLOAT (*r) } } -int AAC_RENAME(ff_ps_apply)(AVCodecContext *avctx, PSContext *ps, INTFLOAT L[2][38][64], INTFLOAT R[2][38][64], int top) +int AAC_RENAME(ff_ps_apply)(AVCodecContext *avctx, PSContext *ps, INTFLOAT L[2][38][64], INTFLOAT R[2][38][64], int top, + int frame_length_short) { INTFLOAT (*Lbuf)[32][2] = ps->Lbuf; INTFLOAT (*Rbuf)[32][2] = ps->Rbuf; @@ -730,7 +732,7 @@ int AAC_RENAME(ff_ps_apply)(AVCodecContext *avctx, PSContext *ps, INTFLOAT L[2][ memset(ps->ap_delay + top, 0, (NR_ALLPASS_BANDS[is34] - top)*sizeof(ps->ap_delay[0])); hybrid_analysis(&ps->dsp, Lbuf, ps->in_buf, L, is34, len); - decorrelation(ps, Rbuf, (const INTFLOAT (*)[32][2]) Lbuf, is34); + decorrelation(ps, Rbuf, (const INTFLOAT (*)[32][2]) Lbuf, is34, frame_length_short); stereo_processing(ps, Lbuf, Rbuf, is34); hybrid_synthesis(&ps->dsp, L, Lbuf, is34, len); hybrid_synthesis(&ps->dsp, R, Rbuf, is34, len); diff --git a/libavcodec/aacps.h b/libavcodec/aacps.h index 3efa38ad88..c771bfb270 100644 --- a/libavcodec/aacps.h +++ b/libavcodec/aacps.h @@ -43,8 +43,6 @@ ///< Baseline implies 10 or 20 stereo bands, ///< mixing mode A, and no ipd/opd -#define numQMFSlots 32 //numTimeSlots * RATE - typedef struct PSCommonContext { int start; int enable_iid; @@ -96,7 +94,8 @@ void ff_ps_init_common(void); void AAC_RENAME(ff_ps_init)(void); void AAC_RENAME(ff_ps_ctx_init)(PSContext *ps); int ff_ps_read_data(AVCodecContext *avctx, GetBitContext *gb, - PSCommonContext *ps, int bits_left); -int AAC_RENAME(ff_ps_apply)(AVCodecContext *avctx, PSContext *ps, INTFLOAT L[2][38][64], INTFLOAT R[2][38][64], int top); + PSCommonContext *ps, int bits_left, int frame_length_short); +int AAC_RENAME(ff_ps_apply)(AVCodecContext *avctx, PSContext *ps, INTFLOAT L[2][38][64], INTFLOAT R[2][38][64], int top, + int frame_length_short); #endif /* AVCODEC_AACPS_H */ diff --git a/libavcodec/aacps_common.c b/libavcodec/aacps_common.c index c388d5b9bc..87521c52e3 100644 --- a/libavcodec/aacps_common.c +++ b/libavcodec/aacps_common.c @@ -132,13 +132,14 @@ static int ps_read_extension_data(GetBitContext *gb, PSCommonContext *ps, } int ff_ps_read_data(AVCodecContext *avctx, GetBitContext *gb_host, - PSCommonContext *ps, int bits_left) + PSCommonContext *ps, int bits_left, int frame_length_short) { int e; int bit_count_start = get_bits_count(gb_host); int header; int bits_consumed; GetBitContext gbc = *gb_host, *gb = &gbc; + int numQMFSlots = frame_length_short ? 30 : 32; header = get_bits1(gb); if (header) { //enable_ps_header diff --git a/libavcodec/aacsbr_template.c b/libavcodec/aacsbr_template.c index cdca402f04..fc64b1de4a 100644 --- a/libavcodec/aacsbr_template.c +++ b/libavcodec/aacsbr_template.c @@ -647,8 +647,7 @@ static int read_sbr_grid(AACContext *ac, SpectralBandReplication *sbr, { int i; int bs_pointer = 0; - // frameLengthFlag ? 15 : 16; 960 sample length frames unsupported; this value is numTimeSlots - int abs_bord_trail = 16; + int abs_bord_trail = ac->oc[1].m4ac.frame_length_short ? 15 : 16; int num_rel_lead, num_rel_trail; unsigned bs_num_env_old = ch_data->bs_num_env; int bs_frame_class, bs_num_env; @@ -972,7 +971,8 @@ static void read_sbr_extension(AACContext *ac, SpectralBandReplication *sbr, skip_bits_long(gb, *num_bits_left); // bs_fill_bits *num_bits_left = 0; } else { - *num_bits_left -= ff_ps_read_data(ac->avctx, gb, &sbr->ps.common, *num_bits_left); + *num_bits_left -= ff_ps_read_data(ac->avctx, gb, &sbr->ps.common, *num_bits_left, + ac->oc[1].m4ac.frame_length_short); ac->avctx->profile = FF_PROFILE_AAC_HE_V2; // ensure the warning is not printed if PS extension is present ac->warned_he_aac_mono = 1; @@ -1190,16 +1190,20 @@ static void sbr_qmf_analysis(AVFloatDSPContext *dsp, AVTXContext *mdct, av_tx_fn mdct_fn, #endif /* USE_FIXED */ SBRDSPContext *sbrdsp, const INTFLOAT *in, INTFLOAT *x, - INTFLOAT z[320], INTFLOAT W[2][32][32][2], int buf_idx) + INTFLOAT z[320], INTFLOAT W[2][32][32][2], int buf_idx, + int frame_length_short) { int i; + const int frameLength = frame_length_short ? 960 : 1024; + const int lp = frame_length_short ? 15*2 : 16*2; #if USE_FIXED int j; #endif - memcpy(x , x+1024, (320-32)*sizeof(x[0])); - memcpy(x+288, in, 1024*sizeof(x[0])); - for (i = 0; i < 32; i++) { // numTimeSlots*RATE = 16*2 as 960 sample frames - // are not supported + + memcpy(x , x+frameLength, (320-32)*sizeof(x[0])); + memcpy(x+288, in, frameLength*sizeof(x[0])); + + for (i = 0; i < lp; i++) { dsp->vector_fmul_reverse(z, sbr_qmf_window_ds, x, 320); sbrdsp->sum64x5(z); sbrdsp->qmf_pre_shuffle(z); @@ -1238,13 +1242,15 @@ static void sbr_qmf_synthesis(AVTXContext *mdct, av_tx_fn mdct_fn, #endif /* USE_FIXED */ INTFLOAT *out, INTFLOAT X[2][38][64], INTFLOAT mdct_buf[2][64], - INTFLOAT *v0, int *v_off, const unsigned int div) + INTFLOAT *v0, int *v_off, const unsigned int div, + int frame_length_short) { int i, n; const INTFLOAT *sbr_qmf_window = div ? sbr_qmf_window_ds : sbr_qmf_window_us; const int step = 128 >> div; + const int lp = frame_length_short ? 15*2 : 16*2; INTFLOAT *v; - for (i = 0; i < 32; i++) { + for (i = 0; i < lp; i++) { if (*v_off < step) { int saved_samples = (1280 - 128) >> div; memcpy(&v0[SBR_SYNTHESIS_BUF_SIZE - saved_samples], v0, saved_samples * sizeof(INTFLOAT)); @@ -1288,7 +1294,7 @@ static int sbr_lf_gen(AACContext *ac, SpectralBandReplication *sbr, { int i, k; const int t_HFGen = 8; - const int i_f = 32; + const int i_f = ac->oc[1].m4ac.frame_length_short ? 15*2 : 16*2; memset(X_low, 0, 32*sizeof(*X_low)); for (k = 0; k < sbr->kx[1]; k++) { for (i = t_HFGen; i < i_f + t_HFGen; i++) { @@ -1344,10 +1350,10 @@ static int sbr_hf_gen(AACContext *ac, SpectralBandReplication *sbr, /// Generate the subband filtered lowband static int sbr_x_gen(SpectralBandReplication *sbr, INTFLOAT X[2][38][64], const INTFLOAT Y0[38][64][2], const INTFLOAT Y1[38][64][2], - const INTFLOAT X_low[32][40][2], int ch) + const INTFLOAT X_low[32][40][2], int ch, int frame_length_short) { int k, i; - const int i_f = 32; + const int i_f = frame_length_short ? 15*2 : 16*2; const int i_Temp = FFMAX(2*sbr->data[ch].t_env_num_env_old - i_f, 0); memset(X, 0, 2*sizeof(*X)); for (k = 0; k < sbr->kx[0]; k++) { @@ -1531,7 +1537,7 @@ void AAC_RENAME(ff_sbr_apply)(AACContext *ac, SpectralBandReplication *sbr, int sbr_qmf_analysis(ac->fdsp, sbr->mdct_ana, sbr->mdct_ana_fn, &sbr->dsp, ch ? R : L, sbr->data[ch].analysis_filterbank_samples, (INTFLOAT*)sbr->qmf_filter_scratch, - sbr->data[ch].W, sbr->data[ch].Ypos); + sbr->data[ch].W, sbr->data[ch].Ypos, ac->oc[1].m4ac.frame_length_short); sbr->c.sbr_lf_gen(ac, sbr, sbr->X_low, (const INTFLOAT (*)[32][32][2]) sbr->data[ch].W, sbr->data[ch].Ypos); @@ -1564,12 +1570,13 @@ void AAC_RENAME(ff_sbr_apply)(AACContext *ac, SpectralBandReplication *sbr, int sbr->c.sbr_x_gen(sbr, sbr->X[ch], (const INTFLOAT (*)[64][2]) sbr->data[ch].Y[1-sbr->data[ch].Ypos], (const INTFLOAT (*)[64][2]) sbr->data[ch].Y[ sbr->data[ch].Ypos], - (const INTFLOAT (*)[40][2]) sbr->X_low, ch); + (const INTFLOAT (*)[40][2]) sbr->X_low, ch, ac->oc[1].m4ac.frame_length_short); } if (ac->oc[1].m4ac.ps == 1) { if (sbr->ps.common.start) { - AAC_RENAME(ff_ps_apply)(ac->avctx, &sbr->ps, sbr->X[0], sbr->X[1], sbr->kx[1] + sbr->m[1]); + AAC_RENAME(ff_ps_apply)(ac->avctx, &sbr->ps, sbr->X[0], sbr->X[1], sbr->kx[1] + sbr->m[1], + ac->oc[1].m4ac.frame_length_short); } else { memcpy(sbr->X[1], sbr->X[0], sizeof(sbr->X[0])); } @@ -1580,13 +1587,13 @@ void AAC_RENAME(ff_sbr_apply)(AACContext *ac, SpectralBandReplication *sbr, int L, sbr->X[0], sbr->qmf_filter_scratch, sbr->data[0].synthesis_filterbank_samples, &sbr->data[0].synthesis_filterbank_samples_offset, - downsampled); + downsampled, ac->oc[1].m4ac.frame_length_short); if (nch == 2) sbr_qmf_synthesis(sbr->mdct, sbr->mdct_fn, &sbr->dsp, ac->fdsp, R, sbr->X[1], sbr->qmf_filter_scratch, sbr->data[1].synthesis_filterbank_samples, &sbr->data[1].synthesis_filterbank_samples_offset, - downsampled); + downsampled, ac->oc[1].m4ac.frame_length_short); } static void aacsbr_func_ptr_init(AACSBRContext *c) diff --git a/libavcodec/sbr.h b/libavcodec/sbr.h index f949465ef5..5a03b1597f 100644 --- a/libavcodec/sbr.h +++ b/libavcodec/sbr.h @@ -130,7 +130,7 @@ typedef struct AACSBRContext { const int e_a[2]); int (*sbr_x_gen)(SpectralBandReplication *sbr, INTFLOAT X[2][38][64], const INTFLOAT Y0[38][64][2], const INTFLOAT Y1[38][64][2], - const INTFLOAT X_low[32][40][2], int ch); + const INTFLOAT X_low[32][40][2], int ch, int frame_length_short); void (*sbr_hf_inverse_filter)(SBRDSPContext *dsp,