From patchwork Fri Oct 7 20:25:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 38613 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:789c:b0:9c:c2cd:5584 with SMTP id bf28csp206187pzc; Fri, 7 Oct 2022 13:26:17 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6sm6sObklaBRBce77tIvQZPbDaWLKw6IKB8PJp+kMOkgvoe8WaQdQ13MpnP65hLliSSHJT X-Received: by 2002:a05:6402:3887:b0:458:289e:c9cc with SMTP id fd7-20020a056402388700b00458289ec9ccmr6257023edb.101.1665174377103; Fri, 07 Oct 2022 13:26:17 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4-20020a508744000000b004571c47b13csi2627546edv.418.2022.10.07.13.26.16; Fri, 07 Oct 2022 13:26:17 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=IbxxdPzj; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B305268BBCE; Fri, 7 Oct 2022 23:25:44 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-oln040092072056.outbound.protection.outlook.com [40.92.72.56]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7D34068BA9F for ; Fri, 7 Oct 2022 23:25:43 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=S377TG0O4KYjJScvlLbmyrwnN5uLQd1iAAw0p9LbA1aBzyrGOr1aqODqZosqD3G+xXjgkpdcaDOQrLZMhh/kII9xLeMBd+qESswev585d9FaR+VYFlPfTXO2YC7eIvA1Py3c/W21O5AWgDr3XzanqsgxN0vQtBiN27sEXd7wsHBovuLf0mjQ2lsW51Pg092F0i7j0p1fV0TF8SqzEj7dyRnpQoVC3MHw+wUWB9meo+jDmkRaXRqnEkhv5jpsDpqyOA03fkwY09khcpFA3eADM9S+00LseSQuFEQ+MK72g6hG/IT9h7D28fiX1D8suXVbeWKSG+K3WpCcLYjFSgFVUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8qN1m4cKh9loVoZ7a4qMx+GlIT7RatyBJWxgM8yrIzw=; b=lxNORFiBJQ/7OQ3BlObdwx9C2QKQgMZSGOWPitH5JwX07SiaXveczHZoFBYmdio3SXWtme8yuX3CsHd1niJSpuGO83FzhsBw3sv7B8gE/S3vFzbaLhxfMO9qUPvA+q3oFD1ADlpNsHkV6MGnKuVmf6HoD3VRwERqZHwo9aG754jdMSglSzbAePyJ6OkXorKLRWo6/XBZKey1PQinm9z79UOw5/6I9dYB9bAdjRp2ICwY/1laTm4Vd+Q0EM05rXAOi3EYiVSAqbzcQrg/hdKGpj4dq3OxxeKChk+Sg5H7I7yeIT6K4RTevW6sBM1qiYlT/ZRvUe1E5unCh614HW+qag== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8qN1m4cKh9loVoZ7a4qMx+GlIT7RatyBJWxgM8yrIzw=; b=IbxxdPzjw9N9co1tNl6nZTm5SgjvpALNWHqpzvRspfZT6HUJxu1fXsOsGEcH7DVFpMoWvyWEduDhJT6hfarb2S7KD7hfu/tw2tAki2GLjdAs8AjxdU5h2LXe2BTVB4jX3FutIMCkG3xBEWU6q0Ird57+j5ysbYKBNrbS+Aw8NFVhzV/lHuhg/I3e2JQU+1RH2YmkzOeIDH9SU1US45jUlXHwnzYG5ignHEt45FnxS1mdXiZIiYgF0e4zTB7TJQlpTOXNs7HrpNcggbXOuV0w+jDmKSOiHAT9rQydRegIhtw56bmtt2EInKK64yWp0/DvFb7y2PEbIW/1cUlQDLKqSg== Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) by DU2P250MB0285.EURP250.PROD.OUTLOOK.COM (2603:10a6:10:27a::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5709.15; Fri, 7 Oct 2022 20:25:41 +0000 Received: from GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::68bd:2fc7:ac52:38f8]) by GV1P250MB0737.EURP250.PROD.OUTLOOK.COM ([fe80::68bd:2fc7:ac52:38f8%9]) with mapi id 15.20.5676.028; Fri, 7 Oct 2022 20:25:41 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 7 Oct 2022 22:25:07 +0200 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: X-TMN: [6shd98M9tX3BIFyQKLzGZt7bU5dDT8069mr63NVay38=] X-ClientProxiedBy: ZR0P278CA0106.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:23::21) To GV1P250MB0737.EURP250.PROD.OUTLOOK.COM (2603:10a6:150:8e::17) X-Microsoft-Original-Message-ID: <20221007202508.4188148-7-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: GV1P250MB0737:EE_|DU2P250MB0285:EE_ X-MS-Office365-Filtering-Correlation-Id: a05e542a-cc1f-4a5c-62e8-08daa8a21a73 X-MS-Exchange-SLBlob-MailProps: EgT5Wr3QDKyO3QjhF2EUZd1To6DGOTfGc/7qxJv1aOW5TRWxszAFO8OKUyowSimZ4lhts6vald+WTW5m3n1v8nxESHy0TawUW9o6p/0OdiEAbj/F5OKDoXpJqufcBz2RbNhdz6WpH3FHFpmpXjvt8VCVhoKe2oBhyvVUnGeqPIyqvS5zh1PEJXBAtnGLL8yL3L+vGWa/wjffDHceSllQk/cK0DJnQVLL6Ic5F6Hi4tJ2QlyORFQY0bapSYh5UEnKTTPLgNLRSCRFhpHqZMVLp6QvoKLGETfOi3MgIa7iEHhYE9u0CEKBZ0df9hHEi49gxDFOuN+D6Ox4kzFn7w6bagH1VwTgCKmSm3dxPrqwYcAamQtcRcnf8swD3DQcIzu8wGP5GPjjX15FV8si2idCzV/Gnb7FI45sZ1mTNvIci2xMEKXl4YD13LyM7OC0u7q877bcMhuW2iXomF+n7SpfYw6Fe5wdSFVWX6+8ywO9xG4h9Q53xhmc1w5DRGPmrd3J2yl5pLZNsQcWRxX8aCYybJ9nxsy06kGmhFp7IbD/I9Dq+BGI6CWj0obkPTDJ17G8gWnQaG0e6vt9AAt/sf/qq6ZhUIxvvwOf0hxG7bSvhNjGzdtkvllXDvG/Y46ZWO5dc7LXUMIFY7VwkyVaTgVoWeS1GWRwIXfQsyeuCjnFYGxbyIZKTs14oxqHeo8Vvvgw/JgMyRXf0+LGMTlw46TKFZGMhfPXSeLQLbrUSzNs9HU= X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FpyDhShTR0zxLJYAo40b18kvg4eDYXN+PYovtorCtWAqiCrbdmT8Nnr70PIk5YFVeeQJJ3YdHSzmX0U0T5DQc3NZF3i1YkC5hwGSMec/Ch70vOGW+X226KM4Za1aC0AStcQH2CLKtV727s2u0bjz4WzuiOOMpk5PqM8eCNshR+2PcRQFElRZV2K67+KivPc+eZQ10kN/7lbvny//cUHZi1MCjP4vermrqnvAj3Uo9Q+7JFzm2zqmQgIdzwUOMVNEdo+fIsxzov0cUf+Z/vk4jVIntaKGysMR+SwsHiaFPU8EASo+v6SLVQbaqK98sjwD96xgMF3fz+kNIsjx2VZju7KM4BOr1LtXj6YJq09dwdaEvVfE7+5fLsVep3KYLSvKkrqCZKENFPwxuQlyGsdqQB2jLmLESl3klM2ZVrLXItJ/fZZEKfVkwqoSRmljET1AsnjIBsYKjlIFxW9krMUvz31Wu27jiy+JTL8GEwSpPtyloLp2ldijSzOu/HF21AZ6hxfstiz6HjWp8prMMv1VIhIouhXdHesB6/2Iv/8XMti5WUnZ2mwIAgP7Yqp9cXQXfRpVMuzN3Xbz/Dvbg8oGe7VdiLQHX/P243QSUnpIfiPXxKsyzm2ajvz40mg5tNdEk9Mz3YNosazX+ppJT+cbsN+1s97YBRP/K8p/jamB7ncj3du/4ZPMTn2H1bvA6wy4 X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: eBV7DeFfu1NtQGLxkr1N/7wLh4vNcSEkYH7uTojWe5G4SENuyjARqAzWDWUevnxop+WkJKzdf9qsnFOB5LRlFLdwDHchOEdrcdhf7tR6ByImE5m4gstDy1Nqd2t8SaSvI8/OiZAq3IZzN3WqEnwsOVropoh+gjf4XXXlSPp8vKwwgrlXd4UHw+PwzzdHgDUPumPBaI1NIhZUDfrafgyqQYpVjXbdM4i2J3vLEj1ZMADun67h7PxdnxFMDbtaJFtcRz0ftnrl2y7pW3BBbn66hkMUfJx7k6nDVFAHo73eUFyzDQ4lVke1mRSg60N1T/pPHBN6ickQ63C3q2/fobnVRefTBbnWsuZMvsR8pGuO/wDHTFLF6bbL4ZJb/eQaR4UZ6873UzoO60WU9gT3lVOGWJpfCsn1ddk6FKsRH5oqfRy6t4Vmhi2vNtwCyRhdj8VOFg8hqammIUrv9O0JAWw+fiuPaP6NsVrUGuvPR9PZ3anAVzn28ClHCgq6aoxXqAlbiqnTh7kCTuBGAcTnqlHn/UgeYABE6j2CdZNaFziTJBeTqp85V9cN+VcY5jUer2rDKLjwrGpISdh0PaFvqBBVpaFMbYGz2vTuLvVso1w5wKd1fj09QAQYhRcLPPxUSdBbE97MwL3HNyEcwm2VizvyLWbpse9isrtx07Sb34FG4KAd+z+BVaoBmzaJ/fo5m9pLJaFijqvd7LRtBw+roLTwoGlPlxRNWMkNBvVkxHw5GBNjxmwsv16KHRz3ZKqoc5USc5lM3+VLESY9psrQuSJhgkzjhVm0I4CXvPXh1UBHCRonpHB5G6X+7K2ACOcd3cdFr0hDPn0PgSOUlXipzRZM4hS5MBbO7ny71Fe00bkZVGXLJcFeXUSLW/26e0hEeU2d0bVC7rFTNI3HQswOfzLvepnpAxegqaV1mtjTfkfUMv0K9DxVK8Ooc+TpYgFZPykcl+TGRDl4A/Ylid29b7oXQhA5pQif32B+KblEQw2C5x3/p00VxtGTagw5zGPCgXzz4T5cxEssIttWJ3iy0KK/+jJbmBgO1iD36PgHbXhICSwgZzruIvCSj735Ri1fUJ8SHBTUmRViE4WcV389QUpD1biqmhOkBoEXYnvuiCLzvp1u57HjOKiygJgD0rUtjB8xeConLL53ocTzR/T4Mmen88l8ZwUAGIvNiNq3LhcMyTVgIfi+9ML1mqa0uybFFvajdx5HJkzXpvYZUpr3Z79BDX2aExIcfGV8gJ2S/8bIxD8/7UzQXvJ+dlZMwhZSVFdGXIie9GaKB9tE1w6uE5gKFYLyi9f/p5l5nz2rrKBshg8= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: a05e542a-cc1f-4a5c-62e8-08daa8a21a73 X-MS-Exchange-CrossTenant-AuthSource: GV1P250MB0737.EURP250.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Oct 2022 20:25:41.0783 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2P250MB0285 Subject: [FFmpeg-devel] [PATCH 8/9] avcodec/opus: Rename opus.c->opus_celt.c, opus_celt.c->opusdec_celt.c X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: I5JzJGBqFgXV Since commit 4fc2531fff112836026aad2bdaf128c9d15a72e3 opus.c contains only the celt stuff shared between decoder and encoder. meanwhile, opus_celt.c is decoder-only. So the new names reflect the actual content better than the current ones. Signed-off-by: Andreas Rheinhardt --- libavcodec/Makefile | 4 +- libavcodec/opus.c | 522 ---------------------- libavcodec/opus_celt.c | 899 ++++++++++++++++++-------------------- libavcodec/opusdec_celt.c | 587 +++++++++++++++++++++++++ 4 files changed, 1006 insertions(+), 1006 deletions(-) delete mode 100644 libavcodec/opus.c create mode 100644 libavcodec/opusdec_celt.c diff --git a/libavcodec/Makefile b/libavcodec/Makefile index b7eb3b1e48..949c65a0e3 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -554,10 +554,10 @@ OBJS-$(CONFIG_NELLYMOSER_ENCODER) += nellymoserenc.o nellymoser.o OBJS-$(CONFIG_NOTCHLC_DECODER) += notchlc.o OBJS-$(CONFIG_NUV_DECODER) += nuv.o rtjpeg.o OBJS-$(CONFIG_ON2AVC_DECODER) += on2avc.o on2avcdata.o -OBJS-$(CONFIG_OPUS_DECODER) += opusdec.o opus.o opus_celt.o \ +OBJS-$(CONFIG_OPUS_DECODER) += opusdec.o opusdec_celt.o opus_celt.o \ opus_pvq.o opus_silk.o opustab.o vorbis_data.o \ opusdec_rc.o opusdsp.o opus_parse.o -OBJS-$(CONFIG_OPUS_ENCODER) += opusenc.o opus.o opusenc_psy.o \ +OBJS-$(CONFIG_OPUS_ENCODER) += opusenc.o opus_celt.o opusenc_psy.o \ opusenc_rc.o opustab.o opus_pvq.o OBJS-$(CONFIG_PAF_AUDIO_DECODER) += pafaudio.o OBJS-$(CONFIG_PAF_VIDEO_DECODER) += pafvideo.o diff --git a/libavcodec/opus.c b/libavcodec/opus.c deleted file mode 100644 index 8def5e6e34..0000000000 --- a/libavcodec/opus.c +++ /dev/null @@ -1,522 +0,0 @@ -/* - * Copyright (c) 2012 Andrew D'Addesio - * Copyright (c) 2013-2014 Mozilla Corporation - * - * This file is part of FFmpeg. - * - * FFmpeg is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public - * License as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * FFmpeg is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with FFmpeg; if not, write to the Free Software - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA - */ - -#include - -#include "config_components.h" -#include "opus_celt.h" -#include "opus_pvq.h" -#include "opustab.h" -#include "opus_rc.h" -#include "opusdec_rc.h" -#include "opusenc_rc.h" - -#if !CONFIG_OPUS_ENCODER -#define ff_opus_rc_enc_log(...) -#define ff_opus_rc_enc_cdf(...) -#define ff_opus_rc_enc_uint(...) -#endif - -#if !CONFIG_OPUS_DECODER -#define ff_opus_rc_dec_log(...) 0 -#define ff_opus_rc_dec_cdf(...) 0 -#define ff_opus_rc_dec_uint(...) 0 -#endif - -static inline void opus_rc_enc_log(OpusRangeCoder *rc, int val, uint32_t bits) -{ - ff_opus_rc_enc_log((OpusEncRangeCoder*)rc, val, bits); -} - -static inline uint32_t opus_rc_dec_log(OpusRangeCoder *rc, uint32_t bits) -{ - return ff_opus_rc_dec_log((OpusDecRangeCoder*)rc, bits); -} - -static inline void opus_rc_enc_cdf(OpusRangeCoder *rc, int val, const uint16_t *cdf) -{ - ff_opus_rc_enc_cdf((OpusEncRangeCoder*)rc, val, cdf); -} - -static inline uint32_t opus_rc_dec_cdf(OpusRangeCoder *rc, const uint16_t *cdf) -{ - return ff_opus_rc_dec_cdf((OpusDecRangeCoder*)rc, cdf); -} - -void ff_celt_quant_bands(CeltFrame *f, OpusRangeCoder *rc) -{ - float lowband_scratch[8 * 22]; - float norm1[2 * 8 * 100]; - float *norm2 = norm1 + 8 * 100; - - int totalbits = (f->framebits << 3) - f->anticollapse_needed; - - int update_lowband = 1; - int lowband_offset = 0; - - int i, j; - - for (i = f->start_band; i < f->end_band; i++) { - uint32_t cm[2] = { (1 << f->blocks) - 1, (1 << f->blocks) - 1 }; - int band_offset = ff_celt_freq_bands[i] << f->size; - int band_size = ff_celt_freq_range[i] << f->size; - float *X = f->block[0].coeffs + band_offset; - float *Y = (f->channels == 2) ? f->block[1].coeffs + band_offset : NULL; - float *norm_loc1, *norm_loc2; - - int consumed = opus_rc_tell_frac(rc); - int effective_lowband = -1; - int b = 0; - - /* Compute how many bits we want to allocate to this band */ - if (i != f->start_band) - f->remaining -= consumed; - f->remaining2 = totalbits - consumed - 1; - if (i <= f->coded_bands - 1) { - int curr_balance = f->remaining / FFMIN(3, f->coded_bands-i); - b = av_clip_uintp2(FFMIN(f->remaining2 + 1, f->pulses[i] + curr_balance), 14); - } - - if ((ff_celt_freq_bands[i] - ff_celt_freq_range[i] >= ff_celt_freq_bands[f->start_band] || - i == f->start_band + 1) && (update_lowband || lowband_offset == 0)) - lowband_offset = i; - - if (i == f->start_band + 1) { - /* Special Hybrid Folding (RFC 8251 section 9). Copy the first band into - the second to ensure the second band never has to use the LCG. */ - int count = (ff_celt_freq_range[i] - ff_celt_freq_range[i-1]) << f->size; - - memcpy(&norm1[band_offset], &norm1[band_offset - count], count * sizeof(float)); - - if (f->channels == 2) - memcpy(&norm2[band_offset], &norm2[band_offset - count], count * sizeof(float)); - } - - /* Get a conservative estimate of the collapse_mask's for the bands we're - going to be folding from. */ - if (lowband_offset != 0 && (f->spread != CELT_SPREAD_AGGRESSIVE || - f->blocks > 1 || f->tf_change[i] < 0)) { - int foldstart, foldend; - - /* This ensures we never repeat spectral content within one band */ - effective_lowband = FFMAX(ff_celt_freq_bands[f->start_band], - ff_celt_freq_bands[lowband_offset] - ff_celt_freq_range[i]); - foldstart = lowband_offset; - while (ff_celt_freq_bands[--foldstart] > effective_lowband); - foldend = lowband_offset - 1; - while (++foldend < i && ff_celt_freq_bands[foldend] < effective_lowband + ff_celt_freq_range[i]); - - cm[0] = cm[1] = 0; - for (j = foldstart; j < foldend; j++) { - cm[0] |= f->block[0].collapse_masks[j]; - cm[1] |= f->block[f->channels - 1].collapse_masks[j]; - } - } - - if (f->dual_stereo && i == f->intensity_stereo) { - /* Switch off dual stereo to do intensity */ - f->dual_stereo = 0; - for (j = ff_celt_freq_bands[f->start_band] << f->size; j < band_offset; j++) - norm1[j] = (norm1[j] + norm2[j]) / 2; - } - - norm_loc1 = effective_lowband != -1 ? norm1 + (effective_lowband << f->size) : NULL; - norm_loc2 = effective_lowband != -1 ? norm2 + (effective_lowband << f->size) : NULL; - - if (f->dual_stereo) { - cm[0] = f->pvq->quant_band(f->pvq, f, rc, i, X, NULL, band_size, b >> 1, - f->blocks, norm_loc1, f->size, - norm1 + band_offset, 0, 1.0f, - lowband_scratch, cm[0]); - - cm[1] = f->pvq->quant_band(f->pvq, f, rc, i, Y, NULL, band_size, b >> 1, - f->blocks, norm_loc2, f->size, - norm2 + band_offset, 0, 1.0f, - lowband_scratch, cm[1]); - } else { - cm[0] = f->pvq->quant_band(f->pvq, f, rc, i, X, Y, band_size, b >> 0, - f->blocks, norm_loc1, f->size, - norm1 + band_offset, 0, 1.0f, - lowband_scratch, cm[0] | cm[1]); - cm[1] = cm[0]; - } - - f->block[0].collapse_masks[i] = (uint8_t)cm[0]; - f->block[f->channels - 1].collapse_masks[i] = (uint8_t)cm[1]; - f->remaining += f->pulses[i] + consumed; - - /* Update the folding position only as long as we have 1 bit/sample depth */ - update_lowband = (b > band_size << 3); - } -} - -#define NORMC(bits) ((bits) << (f->channels - 1) << f->size >> 2) - -void ff_celt_bitalloc(CeltFrame *f, OpusRangeCoder *rc, int encode) -{ - int i, j, low, high, total, done, bandbits, remaining, tbits_8ths; - int skip_startband = f->start_band; - int skip_bit = 0; - int intensitystereo_bit = 0; - int dualstereo_bit = 0; - int dynalloc = 6; - int extrabits = 0; - - int boost[CELT_MAX_BANDS] = { 0 }; - int trim_offset[CELT_MAX_BANDS]; - int threshold[CELT_MAX_BANDS]; - int bits1[CELT_MAX_BANDS]; - int bits2[CELT_MAX_BANDS]; - - if (!CONFIG_OPUS_DECODER || !CONFIG_OPUS_ENCODER) - encode = CONFIG_OPUS_ENCODER; - /* Spread */ - if (opus_rc_tell(rc) + 4 <= f->framebits) { - if (encode) - opus_rc_enc_cdf(rc, f->spread, ff_celt_model_spread); - else - f->spread = opus_rc_dec_cdf(rc, ff_celt_model_spread); - } else { - f->spread = CELT_SPREAD_NORMAL; - } - - /* Initialize static allocation caps */ - for (i = 0; i < CELT_MAX_BANDS; i++) - f->caps[i] = NORMC((ff_celt_static_caps[f->size][f->channels - 1][i] + 64) * ff_celt_freq_range[i]); - - /* Band boosts */ - tbits_8ths = f->framebits << 3; - for (i = f->start_band; i < f->end_band; i++) { - int quanta = ff_celt_freq_range[i] << (f->channels - 1) << f->size; - int b_dynalloc = dynalloc; - int boost_amount = f->alloc_boost[i]; - quanta = FFMIN(quanta << 3, FFMAX(6 << 3, quanta)); - - while (opus_rc_tell_frac(rc) + (b_dynalloc << 3) < tbits_8ths && boost[i] < f->caps[i]) { - int is_boost; - if (encode) { - is_boost = boost_amount--; - opus_rc_enc_log(rc, is_boost, b_dynalloc); - } else { - is_boost = opus_rc_dec_log(rc, b_dynalloc); - } - - if (!is_boost) - break; - - boost[i] += quanta; - tbits_8ths -= quanta; - - b_dynalloc = 1; - } - - if (boost[i]) - dynalloc = FFMAX(dynalloc - 1, 2); - } - - /* Allocation trim */ - if (!encode) - f->alloc_trim = 5; - if (opus_rc_tell_frac(rc) + (6 << 3) <= tbits_8ths) - if (encode) - opus_rc_enc_cdf(rc, f->alloc_trim, ff_celt_model_alloc_trim); - else - f->alloc_trim = opus_rc_dec_cdf(rc, ff_celt_model_alloc_trim); - - /* Anti-collapse bit reservation */ - tbits_8ths = (f->framebits << 3) - opus_rc_tell_frac(rc) - 1; - f->anticollapse_needed = 0; - if (f->transient && f->size >= 2 && tbits_8ths >= ((f->size + 2) << 3)) - f->anticollapse_needed = 1 << 3; - tbits_8ths -= f->anticollapse_needed; - - /* Band skip bit reservation */ - if (tbits_8ths >= 1 << 3) - skip_bit = 1 << 3; - tbits_8ths -= skip_bit; - - /* Intensity/dual stereo bit reservation */ - if (f->channels == 2) { - intensitystereo_bit = ff_celt_log2_frac[f->end_band - f->start_band]; - if (intensitystereo_bit <= tbits_8ths) { - tbits_8ths -= intensitystereo_bit; - if (tbits_8ths >= 1 << 3) { - dualstereo_bit = 1 << 3; - tbits_8ths -= 1 << 3; - } - } else { - intensitystereo_bit = 0; - } - } - - /* Trim offsets */ - for (i = f->start_band; i < f->end_band; i++) { - int trim = f->alloc_trim - 5 - f->size; - int band = ff_celt_freq_range[i] * (f->end_band - i - 1); - int duration = f->size + 3; - int scale = duration + f->channels - 1; - - /* PVQ minimum allocation threshold, below this value the band is - * skipped */ - threshold[i] = FFMAX(3 * ff_celt_freq_range[i] << duration >> 4, - f->channels << 3); - - trim_offset[i] = trim * (band << scale) >> 6; - - if (ff_celt_freq_range[i] << f->size == 1) - trim_offset[i] -= f->channels << 3; - } - - /* Bisection */ - low = 1; - high = CELT_VECTORS - 1; - while (low <= high) { - int center = (low + high) >> 1; - done = total = 0; - - for (i = f->end_band - 1; i >= f->start_band; i--) { - bandbits = NORMC(ff_celt_freq_range[i] * ff_celt_static_alloc[center][i]); - - if (bandbits) - bandbits = FFMAX(bandbits + trim_offset[i], 0); - bandbits += boost[i]; - - if (bandbits >= threshold[i] || done) { - done = 1; - total += FFMIN(bandbits, f->caps[i]); - } else if (bandbits >= f->channels << 3) { - total += f->channels << 3; - } - } - - if (total > tbits_8ths) - high = center - 1; - else - low = center + 1; - } - high = low--; - - /* Bisection */ - for (i = f->start_band; i < f->end_band; i++) { - bits1[i] = NORMC(ff_celt_freq_range[i] * ff_celt_static_alloc[low][i]); - bits2[i] = high >= CELT_VECTORS ? f->caps[i] : - NORMC(ff_celt_freq_range[i] * ff_celt_static_alloc[high][i]); - - if (bits1[i]) - bits1[i] = FFMAX(bits1[i] + trim_offset[i], 0); - if (bits2[i]) - bits2[i] = FFMAX(bits2[i] + trim_offset[i], 0); - - if (low) - bits1[i] += boost[i]; - bits2[i] += boost[i]; - - if (boost[i]) - skip_startband = i; - bits2[i] = FFMAX(bits2[i] - bits1[i], 0); - } - - /* Bisection */ - low = 0; - high = 1 << CELT_ALLOC_STEPS; - for (i = 0; i < CELT_ALLOC_STEPS; i++) { - int center = (low + high) >> 1; - done = total = 0; - - for (j = f->end_band - 1; j >= f->start_band; j--) { - bandbits = bits1[j] + (center * bits2[j] >> CELT_ALLOC_STEPS); - - if (bandbits >= threshold[j] || done) { - done = 1; - total += FFMIN(bandbits, f->caps[j]); - } else if (bandbits >= f->channels << 3) - total += f->channels << 3; - } - if (total > tbits_8ths) - high = center; - else - low = center; - } - - /* Bisection */ - done = total = 0; - for (i = f->end_band - 1; i >= f->start_band; i--) { - bandbits = bits1[i] + (low * bits2[i] >> CELT_ALLOC_STEPS); - - if (bandbits >= threshold[i] || done) - done = 1; - else - bandbits = (bandbits >= f->channels << 3) ? - f->channels << 3 : 0; - - bandbits = FFMIN(bandbits, f->caps[i]); - f->pulses[i] = bandbits; - total += bandbits; - } - - /* Band skipping */ - for (f->coded_bands = f->end_band; ; f->coded_bands--) { - int allocation; - j = f->coded_bands - 1; - - if (j == skip_startband) { - /* all remaining bands are not skipped */ - tbits_8ths += skip_bit; - break; - } - - /* determine the number of bits available for coding "do not skip" markers */ - remaining = tbits_8ths - total; - bandbits = remaining / (ff_celt_freq_bands[j+1] - ff_celt_freq_bands[f->start_band]); - remaining -= bandbits * (ff_celt_freq_bands[j+1] - ff_celt_freq_bands[f->start_band]); - allocation = f->pulses[j] + bandbits * ff_celt_freq_range[j]; - allocation += FFMAX(remaining - (ff_celt_freq_bands[j] - ff_celt_freq_bands[f->start_band]), 0); - - /* a "do not skip" marker is only coded if the allocation is - * above the chosen threshold */ - if (allocation >= FFMAX(threshold[j], (f->channels + 1) << 3)) { - int do_not_skip; - if (encode) { - do_not_skip = f->coded_bands <= f->skip_band_floor; - opus_rc_enc_log(rc, do_not_skip, 1); - } else { - do_not_skip = opus_rc_dec_log(rc, 1); - } - - if (do_not_skip) - break; - - total += 1 << 3; - allocation -= 1 << 3; - } - - /* the band is skipped, so reclaim its bits */ - total -= f->pulses[j]; - if (intensitystereo_bit) { - total -= intensitystereo_bit; - intensitystereo_bit = ff_celt_log2_frac[j - f->start_band]; - total += intensitystereo_bit; - } - - total += f->pulses[j] = (allocation >= f->channels << 3) ? f->channels << 3 : 0; - } - - /* IS start band */ - if (encode) { - if (intensitystereo_bit) { - f->intensity_stereo = FFMIN(f->intensity_stereo, f->coded_bands); - ff_opus_rc_enc_uint((OpusEncRangeCoder*)rc, f->intensity_stereo, f->coded_bands + 1 - f->start_band); - } - } else { - f->intensity_stereo = f->dual_stereo = 0; - if (intensitystereo_bit) - f->intensity_stereo = f->start_band + ff_opus_rc_dec_uint((OpusDecRangeCoder*)rc, f->coded_bands + 1 - f->start_band); - } - - /* DS flag */ - if (f->intensity_stereo <= f->start_band) - tbits_8ths += dualstereo_bit; /* no intensity stereo means no dual stereo */ - else if (dualstereo_bit) - if (encode) - opus_rc_enc_log(rc, f->dual_stereo, 1); - else - f->dual_stereo = opus_rc_dec_log(rc, 1); - - /* Supply the remaining bits in this frame to lower bands */ - remaining = tbits_8ths - total; - bandbits = remaining / (ff_celt_freq_bands[f->coded_bands] - ff_celt_freq_bands[f->start_band]); - remaining -= bandbits * (ff_celt_freq_bands[f->coded_bands] - ff_celt_freq_bands[f->start_band]); - for (i = f->start_band; i < f->coded_bands; i++) { - const int bits = FFMIN(remaining, ff_celt_freq_range[i]); - f->pulses[i] += bits + bandbits * ff_celt_freq_range[i]; - remaining -= bits; - } - - /* Finally determine the allocation */ - for (i = f->start_band; i < f->coded_bands; i++) { - int N = ff_celt_freq_range[i] << f->size; - int prev_extra = extrabits; - f->pulses[i] += extrabits; - - if (N > 1) { - int dof; /* degrees of freedom */ - int temp; /* dof * channels * log(dof) */ - int fine_bits; - int max_bits; - int offset; /* fine energy quantization offset, i.e. - * extra bits assigned over the standard - * totalbits/dof */ - - extrabits = FFMAX(f->pulses[i] - f->caps[i], 0); - f->pulses[i] -= extrabits; - - /* intensity stereo makes use of an extra degree of freedom */ - dof = N * f->channels + (f->channels == 2 && N > 2 && !f->dual_stereo && i < f->intensity_stereo); - temp = dof * (ff_celt_log_freq_range[i] + (f->size << 3)); - offset = (temp >> 1) - dof * CELT_FINE_OFFSET; - if (N == 2) /* dof=2 is the only case that doesn't fit the model */ - offset += dof << 1; - - /* grant an additional bias for the first and second pulses */ - if (f->pulses[i] + offset < 2 * (dof << 3)) - offset += temp >> 2; - else if (f->pulses[i] + offset < 3 * (dof << 3)) - offset += temp >> 3; - - fine_bits = (f->pulses[i] + offset + (dof << 2)) / (dof << 3); - max_bits = FFMIN((f->pulses[i] >> 3) >> (f->channels - 1), CELT_MAX_FINE_BITS); - max_bits = FFMAX(max_bits, 0); - f->fine_bits[i] = av_clip(fine_bits, 0, max_bits); - - /* If fine_bits was rounded down or capped, - * give priority for the final fine energy pass */ - f->fine_priority[i] = (f->fine_bits[i] * (dof << 3) >= f->pulses[i] + offset); - - /* the remaining bits are assigned to PVQ */ - f->pulses[i] -= f->fine_bits[i] << (f->channels - 1) << 3; - } else { - /* all bits go to fine energy except for the sign bit */ - extrabits = FFMAX(f->pulses[i] - (f->channels << 3), 0); - f->pulses[i] -= extrabits; - f->fine_bits[i] = 0; - f->fine_priority[i] = 1; - } - - /* hand back a limited number of extra fine energy bits to this band */ - if (extrabits > 0) { - int fineextra = FFMIN(extrabits >> (f->channels + 2), - CELT_MAX_FINE_BITS - f->fine_bits[i]); - f->fine_bits[i] += fineextra; - - fineextra <<= f->channels + 2; - f->fine_priority[i] = (fineextra >= extrabits - prev_extra); - extrabits -= fineextra; - } - } - f->remaining = extrabits; - - /* skipped bands dedicate all of their bits for fine energy */ - for (; i < f->end_band; i++) { - f->fine_bits[i] = f->pulses[i] >> (f->channels - 1) >> 3; - f->pulses[i] = 0; - f->fine_priority[i] = f->fine_bits[i] < 1; - } -} diff --git a/libavcodec/opus_celt.c b/libavcodec/opus_celt.c index a14764ec18..8def5e6e34 100644 --- a/libavcodec/opus_celt.c +++ b/libavcodec/opus_celt.c @@ -1,7 +1,6 @@ /* * Copyright (c) 2012 Andrew D'Addesio * Copyright (c) 2013-2014 Mozilla Corporation - * Copyright (c) 2016 Rostislav Pehlivanov * * This file is part of FFmpeg. * @@ -20,568 +19,504 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ -/** - * @file - * Opus CELT decoder - */ - -#include +#include +#include "config_components.h" #include "opus_celt.h" -#include "opusdec_rc.h" -#include "opustab.h" #include "opus_pvq.h" +#include "opustab.h" +#include "opus_rc.h" +#include "opusdec_rc.h" +#include "opusenc_rc.h" -/* Use the 2D z-transform to apply prediction in both the time domain (alpha) - * and the frequency domain (beta) */ -static void celt_decode_coarse_energy(CeltFrame *f, OpusDecRangeCoder *rc) -{ - int i, j; - float prev[2] = { 0 }; - float alpha = ff_celt_alpha_coef[f->size]; - float beta = ff_celt_beta_coef[f->size]; - const uint8_t *model = ff_celt_coarse_energy_dist[f->size][0]; - - /* intra frame */ - if (opus_rc_tell(&rc->c) + 3 <= f->framebits && ff_opus_rc_dec_log(rc, 3)) { - alpha = 0.0f; - beta = 1.0f - (4915.0f/32768.0f); - model = ff_celt_coarse_energy_dist[f->size][1]; - } - - for (i = 0; i < CELT_MAX_BANDS; i++) { - for (j = 0; j < f->channels; j++) { - CeltBlock *block = &f->block[j]; - float value; - int available; - - if (i < f->start_band || i >= f->end_band) { - block->energy[i] = 0.0; - continue; - } +#if !CONFIG_OPUS_ENCODER +#define ff_opus_rc_enc_log(...) +#define ff_opus_rc_enc_cdf(...) +#define ff_opus_rc_enc_uint(...) +#endif - available = f->framebits - opus_rc_tell(&rc->c); - if (available >= 15) { - /* decode using a Laplace distribution */ - int k = FFMIN(i, 20) << 1; - value = ff_opus_rc_dec_laplace(rc, model[k] << 7, model[k+1] << 6); - } else if (available >= 2) { - int x = ff_opus_rc_dec_cdf(rc, ff_celt_model_energy_small); - value = (x>>1) ^ -(x&1); - } else if (available >= 1) { - value = -(float)ff_opus_rc_dec_log(rc, 1); - } else value = -1; - - block->energy[i] = FFMAX(-9.0f, block->energy[i]) * alpha + prev[j] + value; - prev[j] += beta * value; - } - } -} +#if !CONFIG_OPUS_DECODER +#define ff_opus_rc_dec_log(...) 0 +#define ff_opus_rc_dec_cdf(...) 0 +#define ff_opus_rc_dec_uint(...) 0 +#endif -static void celt_decode_fine_energy(CeltFrame *f, OpusDecRangeCoder *rc) +static inline void opus_rc_enc_log(OpusRangeCoder *rc, int val, uint32_t bits) { - int i; - for (i = f->start_band; i < f->end_band; i++) { - int j; - if (!f->fine_bits[i]) - continue; - - for (j = 0; j < f->channels; j++) { - CeltBlock *block = &f->block[j]; - int q2; - float offset; - q2 = ff_opus_rc_get_raw(rc, f->fine_bits[i]); - offset = (q2 + 0.5f) * (1 << (14 - f->fine_bits[i])) / 16384.0f - 0.5f; - block->energy[i] += offset; - } - } + ff_opus_rc_enc_log((OpusEncRangeCoder*)rc, val, bits); } -static void celt_decode_final_energy(CeltFrame *f, OpusDecRangeCoder *rc) +static inline uint32_t opus_rc_dec_log(OpusRangeCoder *rc, uint32_t bits) { - int priority, i, j; - int bits_left = f->framebits - opus_rc_tell(&rc->c); - - for (priority = 0; priority < 2; priority++) { - for (i = f->start_band; i < f->end_band && bits_left >= f->channels; i++) { - if (f->fine_priority[i] != priority || f->fine_bits[i] >= CELT_MAX_FINE_BITS) - continue; - - for (j = 0; j < f->channels; j++) { - int q2; - float offset; - q2 = ff_opus_rc_get_raw(rc, 1); - offset = (q2 - 0.5f) * (1 << (14 - f->fine_bits[i] - 1)) / 16384.0f; - f->block[j].energy[i] += offset; - bits_left--; - } - } - } + return ff_opus_rc_dec_log((OpusDecRangeCoder*)rc, bits); } -static void celt_decode_tf_changes(CeltFrame *f, OpusDecRangeCoder *rc) +static inline void opus_rc_enc_cdf(OpusRangeCoder *rc, int val, const uint16_t *cdf) { - int i, diff = 0, tf_select = 0, tf_changed = 0, tf_select_bit; - int consumed, bits = f->transient ? 2 : 4; - - consumed = opus_rc_tell(&rc->c); - tf_select_bit = (f->size != 0 && consumed+bits+1 <= f->framebits); - - for (i = f->start_band; i < f->end_band; i++) { - if (consumed+bits+tf_select_bit <= f->framebits) { - diff ^= ff_opus_rc_dec_log(rc, bits); - consumed = opus_rc_tell(&rc->c); - tf_changed |= diff; - } - f->tf_change[i] = diff; - bits = f->transient ? 4 : 5; - } - - if (tf_select_bit && ff_celt_tf_select[f->size][f->transient][0][tf_changed] != - ff_celt_tf_select[f->size][f->transient][1][tf_changed]) - tf_select = ff_opus_rc_dec_log(rc, 1); - - for (i = f->start_band; i < f->end_band; i++) { - f->tf_change[i] = ff_celt_tf_select[f->size][f->transient][tf_select][f->tf_change[i]]; - } + ff_opus_rc_enc_cdf((OpusEncRangeCoder*)rc, val, cdf); } -static void celt_denormalize(CeltFrame *f, CeltBlock *block, float *data) +static inline uint32_t opus_rc_dec_cdf(OpusRangeCoder *rc, const uint16_t *cdf) { - int i, j; - - for (i = f->start_band; i < f->end_band; i++) { - float *dst = data + (ff_celt_freq_bands[i] << f->size); - float log_norm = block->energy[i] + ff_celt_mean_energy[i]; - float norm = exp2f(FFMIN(log_norm, 32.0f)); - - for (j = 0; j < ff_celt_freq_range[i] << f->size; j++) - dst[j] *= norm; - } + return ff_opus_rc_dec_cdf((OpusDecRangeCoder*)rc, cdf); } -static void celt_postfilter_apply_transition(CeltBlock *block, float *data) +void ff_celt_quant_bands(CeltFrame *f, OpusRangeCoder *rc) { - const int T0 = block->pf_period_old; - const int T1 = block->pf_period; - - float g00, g01, g02; - float g10, g11, g12; - - float x0, x1, x2, x3, x4; - - int i; - - if (block->pf_gains[0] == 0.0 && - block->pf_gains_old[0] == 0.0) - return; - - g00 = block->pf_gains_old[0]; - g01 = block->pf_gains_old[1]; - g02 = block->pf_gains_old[2]; - g10 = block->pf_gains[0]; - g11 = block->pf_gains[1]; - g12 = block->pf_gains[2]; - - x1 = data[-T1 + 1]; - x2 = data[-T1]; - x3 = data[-T1 - 1]; - x4 = data[-T1 - 2]; - - for (i = 0; i < CELT_OVERLAP; i++) { - float w = ff_celt_window2[i]; - x0 = data[i - T1 + 2]; - - data[i] += (1.0 - w) * g00 * data[i - T0] + - (1.0 - w) * g01 * (data[i - T0 - 1] + data[i - T0 + 1]) + - (1.0 - w) * g02 * (data[i - T0 - 2] + data[i - T0 + 2]) + - w * g10 * x2 + - w * g11 * (x1 + x3) + - w * g12 * (x0 + x4); - x4 = x3; - x3 = x2; - x2 = x1; - x1 = x0; - } -} + float lowband_scratch[8 * 22]; + float norm1[2 * 8 * 100]; + float *norm2 = norm1 + 8 * 100; -static void celt_postfilter(CeltFrame *f, CeltBlock *block) -{ - int len = f->blocksize * f->blocks; - const int filter_len = len - 2 * CELT_OVERLAP; + int totalbits = (f->framebits << 3) - f->anticollapse_needed; - celt_postfilter_apply_transition(block, block->buf + 1024); + int update_lowband = 1; + int lowband_offset = 0; - block->pf_period_old = block->pf_period; - memcpy(block->pf_gains_old, block->pf_gains, sizeof(block->pf_gains)); + int i, j; - block->pf_period = block->pf_period_new; - memcpy(block->pf_gains, block->pf_gains_new, sizeof(block->pf_gains)); + for (i = f->start_band; i < f->end_band; i++) { + uint32_t cm[2] = { (1 << f->blocks) - 1, (1 << f->blocks) - 1 }; + int band_offset = ff_celt_freq_bands[i] << f->size; + int band_size = ff_celt_freq_range[i] << f->size; + float *X = f->block[0].coeffs + band_offset; + float *Y = (f->channels == 2) ? f->block[1].coeffs + band_offset : NULL; + float *norm_loc1, *norm_loc2; + + int consumed = opus_rc_tell_frac(rc); + int effective_lowband = -1; + int b = 0; + + /* Compute how many bits we want to allocate to this band */ + if (i != f->start_band) + f->remaining -= consumed; + f->remaining2 = totalbits - consumed - 1; + if (i <= f->coded_bands - 1) { + int curr_balance = f->remaining / FFMIN(3, f->coded_bands-i); + b = av_clip_uintp2(FFMIN(f->remaining2 + 1, f->pulses[i] + curr_balance), 14); + } - if (len > CELT_OVERLAP) { - celt_postfilter_apply_transition(block, block->buf + 1024 + CELT_OVERLAP); + if ((ff_celt_freq_bands[i] - ff_celt_freq_range[i] >= ff_celt_freq_bands[f->start_band] || + i == f->start_band + 1) && (update_lowband || lowband_offset == 0)) + lowband_offset = i; - if (block->pf_gains[0] > FLT_EPSILON && filter_len > 0) - f->opusdsp.postfilter(block->buf + 1024 + 2 * CELT_OVERLAP, - block->pf_period, block->pf_gains, - filter_len); + if (i == f->start_band + 1) { + /* Special Hybrid Folding (RFC 8251 section 9). Copy the first band into + the second to ensure the second band never has to use the LCG. */ + int count = (ff_celt_freq_range[i] - ff_celt_freq_range[i-1]) << f->size; - block->pf_period_old = block->pf_period; - memcpy(block->pf_gains_old, block->pf_gains, sizeof(block->pf_gains)); - } + memcpy(&norm1[band_offset], &norm1[band_offset - count], count * sizeof(float)); - memmove(block->buf, block->buf + len, (1024 + CELT_OVERLAP / 2) * sizeof(float)); -} + if (f->channels == 2) + memcpy(&norm2[band_offset], &norm2[band_offset - count], count * sizeof(float)); + } -static int parse_postfilter(CeltFrame *f, OpusDecRangeCoder *rc, int consumed) -{ - int i; - - memset(f->block[0].pf_gains_new, 0, sizeof(f->block[0].pf_gains_new)); - memset(f->block[1].pf_gains_new, 0, sizeof(f->block[1].pf_gains_new)); - - if (f->start_band == 0 && consumed + 16 <= f->framebits) { - int has_postfilter = ff_opus_rc_dec_log(rc, 1); - if (has_postfilter) { - float gain; - int tapset, octave, period; - - octave = ff_opus_rc_dec_uint(rc, 6); - period = (16 << octave) + ff_opus_rc_get_raw(rc, 4 + octave) - 1; - gain = 0.09375f * (ff_opus_rc_get_raw(rc, 3) + 1); - tapset = (opus_rc_tell(&rc->c) + 2 <= f->framebits) ? - ff_opus_rc_dec_cdf(rc, ff_celt_model_tapset) : 0; - - for (i = 0; i < 2; i++) { - CeltBlock *block = &f->block[i]; - - block->pf_period_new = FFMAX(period, CELT_POSTFILTER_MINPERIOD); - block->pf_gains_new[0] = gain * ff_celt_postfilter_taps[tapset][0]; - block->pf_gains_new[1] = gain * ff_celt_postfilter_taps[tapset][1]; - block->pf_gains_new[2] = gain * ff_celt_postfilter_taps[tapset][2]; + /* Get a conservative estimate of the collapse_mask's for the bands we're + going to be folding from. */ + if (lowband_offset != 0 && (f->spread != CELT_SPREAD_AGGRESSIVE || + f->blocks > 1 || f->tf_change[i] < 0)) { + int foldstart, foldend; + + /* This ensures we never repeat spectral content within one band */ + effective_lowband = FFMAX(ff_celt_freq_bands[f->start_band], + ff_celt_freq_bands[lowband_offset] - ff_celt_freq_range[i]); + foldstart = lowband_offset; + while (ff_celt_freq_bands[--foldstart] > effective_lowband); + foldend = lowband_offset - 1; + while (++foldend < i && ff_celt_freq_bands[foldend] < effective_lowband + ff_celt_freq_range[i]); + + cm[0] = cm[1] = 0; + for (j = foldstart; j < foldend; j++) { + cm[0] |= f->block[0].collapse_masks[j]; + cm[1] |= f->block[f->channels - 1].collapse_masks[j]; } } - consumed = opus_rc_tell(&rc->c); - } + if (f->dual_stereo && i == f->intensity_stereo) { + /* Switch off dual stereo to do intensity */ + f->dual_stereo = 0; + for (j = ff_celt_freq_bands[f->start_band] << f->size; j < band_offset; j++) + norm1[j] = (norm1[j] + norm2[j]) / 2; + } - return consumed; -} + norm_loc1 = effective_lowband != -1 ? norm1 + (effective_lowband << f->size) : NULL; + norm_loc2 = effective_lowband != -1 ? norm2 + (effective_lowband << f->size) : NULL; -static void process_anticollapse(CeltFrame *f, CeltBlock *block, float *X) -{ - int i, j, k; + if (f->dual_stereo) { + cm[0] = f->pvq->quant_band(f->pvq, f, rc, i, X, NULL, band_size, b >> 1, + f->blocks, norm_loc1, f->size, + norm1 + band_offset, 0, 1.0f, + lowband_scratch, cm[0]); - for (i = f->start_band; i < f->end_band; i++) { - int renormalize = 0; - float *xptr; - float prev[2]; - float Ediff, r; - float thresh, sqrt_1; - int depth; - - /* depth in 1/8 bits */ - depth = (1 + f->pulses[i]) / (ff_celt_freq_range[i] << f->size); - thresh = exp2f(-1.0 - 0.125f * depth); - sqrt_1 = 1.0f / sqrtf(ff_celt_freq_range[i] << f->size); - - xptr = X + (ff_celt_freq_bands[i] << f->size); - - prev[0] = block->prev_energy[0][i]; - prev[1] = block->prev_energy[1][i]; - if (f->channels == 1) { - CeltBlock *block1 = &f->block[1]; - - prev[0] = FFMAX(prev[0], block1->prev_energy[0][i]); - prev[1] = FFMAX(prev[1], block1->prev_energy[1][i]); - } - Ediff = block->energy[i] - FFMIN(prev[0], prev[1]); - Ediff = FFMAX(0, Ediff); - - /* r needs to be multiplied by 2 or 2*sqrt(2) depending on LM because - short blocks don't have the same energy as long */ - r = exp2f(1 - Ediff); - if (f->size == 3) - r *= M_SQRT2; - r = FFMIN(thresh, r) * sqrt_1; - for (k = 0; k < 1 << f->size; k++) { - /* Detect collapse */ - if (!(block->collapse_masks[i] & 1 << k)) { - /* Fill with noise */ - for (j = 0; j < ff_celt_freq_range[i]; j++) - xptr[(j << f->size) + k] = (celt_rng(f) & 0x8000) ? r : -r; - renormalize = 1; - } + cm[1] = f->pvq->quant_band(f->pvq, f, rc, i, Y, NULL, band_size, b >> 1, + f->blocks, norm_loc2, f->size, + norm2 + band_offset, 0, 1.0f, + lowband_scratch, cm[1]); + } else { + cm[0] = f->pvq->quant_band(f->pvq, f, rc, i, X, Y, band_size, b >> 0, + f->blocks, norm_loc1, f->size, + norm1 + band_offset, 0, 1.0f, + lowband_scratch, cm[0] | cm[1]); + cm[1] = cm[0]; } - /* We just added some energy, so we need to renormalize */ - if (renormalize) - celt_renormalize_vector(xptr, ff_celt_freq_range[i] << f->size, 1.0f); + f->block[0].collapse_masks[i] = (uint8_t)cm[0]; + f->block[f->channels - 1].collapse_masks[i] = (uint8_t)cm[1]; + f->remaining += f->pulses[i] + consumed; + + /* Update the folding position only as long as we have 1 bit/sample depth */ + update_lowband = (b > band_size << 3); } } -int ff_celt_decode_frame(CeltFrame *f, OpusDecRangeCoder *rc, - float **output, int channels, int frame_size, - int start_band, int end_band) -{ - int i, j, downmix = 0; - int consumed; // bits of entropy consumed thus far for this frame - AVTXContext *imdct; - av_tx_fn imdct_fn; - - if (channels != 1 && channels != 2) { - av_log(f->avctx, AV_LOG_ERROR, "Invalid number of coded channels: %d\n", - channels); - return AVERROR_INVALIDDATA; - } - if (start_band < 0 || start_band > end_band || end_band > CELT_MAX_BANDS) { - av_log(f->avctx, AV_LOG_ERROR, "Invalid start/end band: %d %d\n", - start_band, end_band); - return AVERROR_INVALIDDATA; - } +#define NORMC(bits) ((bits) << (f->channels - 1) << f->size >> 2) - f->silence = 0; - f->transient = 0; - f->anticollapse = 0; - f->flushed = 0; - f->channels = channels; - f->start_band = start_band; - f->end_band = end_band; - f->framebits = rc->c.rb.bytes * 8; - - f->size = av_log2(frame_size / CELT_SHORT_BLOCKSIZE); - if (f->size > CELT_MAX_LOG_BLOCKS || - frame_size != CELT_SHORT_BLOCKSIZE * (1 << f->size)) { - av_log(f->avctx, AV_LOG_ERROR, "Invalid CELT frame size: %d\n", - frame_size); - return AVERROR_INVALIDDATA; +void ff_celt_bitalloc(CeltFrame *f, OpusRangeCoder *rc, int encode) +{ + int i, j, low, high, total, done, bandbits, remaining, tbits_8ths; + int skip_startband = f->start_band; + int skip_bit = 0; + int intensitystereo_bit = 0; + int dualstereo_bit = 0; + int dynalloc = 6; + int extrabits = 0; + + int boost[CELT_MAX_BANDS] = { 0 }; + int trim_offset[CELT_MAX_BANDS]; + int threshold[CELT_MAX_BANDS]; + int bits1[CELT_MAX_BANDS]; + int bits2[CELT_MAX_BANDS]; + + if (!CONFIG_OPUS_DECODER || !CONFIG_OPUS_ENCODER) + encode = CONFIG_OPUS_ENCODER; + /* Spread */ + if (opus_rc_tell(rc) + 4 <= f->framebits) { + if (encode) + opus_rc_enc_cdf(rc, f->spread, ff_celt_model_spread); + else + f->spread = opus_rc_dec_cdf(rc, ff_celt_model_spread); + } else { + f->spread = CELT_SPREAD_NORMAL; } - if (!f->output_channels) - f->output_channels = channels; + /* Initialize static allocation caps */ + for (i = 0; i < CELT_MAX_BANDS; i++) + f->caps[i] = NORMC((ff_celt_static_caps[f->size][f->channels - 1][i] + 64) * ff_celt_freq_range[i]); - for (i = 0; i < f->channels; i++) { - memset(f->block[i].coeffs, 0, sizeof(f->block[i].coeffs)); - memset(f->block[i].collapse_masks, 0, sizeof(f->block[i].collapse_masks)); - } + /* Band boosts */ + tbits_8ths = f->framebits << 3; + for (i = f->start_band; i < f->end_band; i++) { + int quanta = ff_celt_freq_range[i] << (f->channels - 1) << f->size; + int b_dynalloc = dynalloc; + int boost_amount = f->alloc_boost[i]; + quanta = FFMIN(quanta << 3, FFMAX(6 << 3, quanta)); + + while (opus_rc_tell_frac(rc) + (b_dynalloc << 3) < tbits_8ths && boost[i] < f->caps[i]) { + int is_boost; + if (encode) { + is_boost = boost_amount--; + opus_rc_enc_log(rc, is_boost, b_dynalloc); + } else { + is_boost = opus_rc_dec_log(rc, b_dynalloc); + } - consumed = opus_rc_tell(&rc->c); + if (!is_boost) + break; - /* obtain silence flag */ - if (consumed >= f->framebits) - f->silence = 1; - else if (consumed == 1) - f->silence = ff_opus_rc_dec_log(rc, 15); + boost[i] += quanta; + tbits_8ths -= quanta; + b_dynalloc = 1; + } - if (f->silence) { - consumed = f->framebits; - rc->c.total_bits += f->framebits - opus_rc_tell(&rc->c); + if (boost[i]) + dynalloc = FFMAX(dynalloc - 1, 2); } - /* obtain post-filter options */ - consumed = parse_postfilter(f, rc, consumed); - - /* obtain transient flag */ - if (f->size != 0 && consumed+3 <= f->framebits) - f->transient = ff_opus_rc_dec_log(rc, 3); - - f->blocks = f->transient ? 1 << f->size : 1; - f->blocksize = frame_size / f->blocks; - - imdct = f->tx[f->transient ? 0 : f->size]; - imdct_fn = f->tx_fn[f->transient ? 0 : f->size]; - - if (channels == 1) { - for (i = 0; i < CELT_MAX_BANDS; i++) - f->block[0].energy[i] = FFMAX(f->block[0].energy[i], f->block[1].energy[i]); + /* Allocation trim */ + if (!encode) + f->alloc_trim = 5; + if (opus_rc_tell_frac(rc) + (6 << 3) <= tbits_8ths) + if (encode) + opus_rc_enc_cdf(rc, f->alloc_trim, ff_celt_model_alloc_trim); + else + f->alloc_trim = opus_rc_dec_cdf(rc, ff_celt_model_alloc_trim); + + /* Anti-collapse bit reservation */ + tbits_8ths = (f->framebits << 3) - opus_rc_tell_frac(rc) - 1; + f->anticollapse_needed = 0; + if (f->transient && f->size >= 2 && tbits_8ths >= ((f->size + 2) << 3)) + f->anticollapse_needed = 1 << 3; + tbits_8ths -= f->anticollapse_needed; + + /* Band skip bit reservation */ + if (tbits_8ths >= 1 << 3) + skip_bit = 1 << 3; + tbits_8ths -= skip_bit; + + /* Intensity/dual stereo bit reservation */ + if (f->channels == 2) { + intensitystereo_bit = ff_celt_log2_frac[f->end_band - f->start_band]; + if (intensitystereo_bit <= tbits_8ths) { + tbits_8ths -= intensitystereo_bit; + if (tbits_8ths >= 1 << 3) { + dualstereo_bit = 1 << 3; + tbits_8ths -= 1 << 3; + } + } else { + intensitystereo_bit = 0; + } } - celt_decode_coarse_energy(f, rc); - celt_decode_tf_changes (f, rc); - ff_celt_bitalloc (f, &rc->c, 0); - celt_decode_fine_energy (f, rc); - ff_celt_quant_bands (f, &rc->c); - - if (f->anticollapse_needed) - f->anticollapse = ff_opus_rc_get_raw(rc, 1); - - celt_decode_final_energy(f, rc); + /* Trim offsets */ + for (i = f->start_band; i < f->end_band; i++) { + int trim = f->alloc_trim - 5 - f->size; + int band = ff_celt_freq_range[i] * (f->end_band - i - 1); + int duration = f->size + 3; + int scale = duration + f->channels - 1; - /* apply anti-collapse processing and denormalization to - * each coded channel */ - for (i = 0; i < f->channels; i++) { - CeltBlock *block = &f->block[i]; + /* PVQ minimum allocation threshold, below this value the band is + * skipped */ + threshold[i] = FFMAX(3 * ff_celt_freq_range[i] << duration >> 4, + f->channels << 3); - if (f->anticollapse) - process_anticollapse(f, block, f->block[i].coeffs); + trim_offset[i] = trim * (band << scale) >> 6; - celt_denormalize(f, block, f->block[i].coeffs); + if (ff_celt_freq_range[i] << f->size == 1) + trim_offset[i] -= f->channels << 3; } - /* stereo -> mono downmix */ - if (f->output_channels < f->channels) { - f->dsp->vector_fmac_scalar(f->block[0].coeffs, f->block[1].coeffs, 1.0, FFALIGN(frame_size, 16)); - downmix = 1; - } else if (f->output_channels > f->channels) - memcpy(f->block[1].coeffs, f->block[0].coeffs, frame_size * sizeof(float)); - - if (f->silence) { - for (i = 0; i < 2; i++) { - CeltBlock *block = &f->block[i]; - - for (j = 0; j < FF_ARRAY_ELEMS(block->energy); j++) - block->energy[j] = CELT_ENERGY_SILENCE; + /* Bisection */ + low = 1; + high = CELT_VECTORS - 1; + while (low <= high) { + int center = (low + high) >> 1; + done = total = 0; + + for (i = f->end_band - 1; i >= f->start_band; i--) { + bandbits = NORMC(ff_celt_freq_range[i] * ff_celt_static_alloc[center][i]); + + if (bandbits) + bandbits = FFMAX(bandbits + trim_offset[i], 0); + bandbits += boost[i]; + + if (bandbits >= threshold[i] || done) { + done = 1; + total += FFMIN(bandbits, f->caps[i]); + } else if (bandbits >= f->channels << 3) { + total += f->channels << 3; + } } - memset(f->block[0].coeffs, 0, sizeof(f->block[0].coeffs)); - memset(f->block[1].coeffs, 0, sizeof(f->block[1].coeffs)); - } - /* transform and output for each output channel */ - for (i = 0; i < f->output_channels; i++) { - CeltBlock *block = &f->block[i]; + if (total > tbits_8ths) + high = center - 1; + else + low = center + 1; + } + high = low--; - /* iMDCT and overlap-add */ - for (j = 0; j < f->blocks; j++) { - float *dst = block->buf + 1024 + j * f->blocksize; + /* Bisection */ + for (i = f->start_band; i < f->end_band; i++) { + bits1[i] = NORMC(ff_celt_freq_range[i] * ff_celt_static_alloc[low][i]); + bits2[i] = high >= CELT_VECTORS ? f->caps[i] : + NORMC(ff_celt_freq_range[i] * ff_celt_static_alloc[high][i]); + + if (bits1[i]) + bits1[i] = FFMAX(bits1[i] + trim_offset[i], 0); + if (bits2[i]) + bits2[i] = FFMAX(bits2[i] + trim_offset[i], 0); + + if (low) + bits1[i] += boost[i]; + bits2[i] += boost[i]; + + if (boost[i]) + skip_startband = i; + bits2[i] = FFMAX(bits2[i] - bits1[i], 0); + } - imdct_fn(imdct, dst + CELT_OVERLAP / 2, f->block[i].coeffs + j, - sizeof(float)*f->blocks); - f->dsp->vector_fmul_window(dst, dst, dst + CELT_OVERLAP / 2, - ff_celt_window, CELT_OVERLAP / 2); + /* Bisection */ + low = 0; + high = 1 << CELT_ALLOC_STEPS; + for (i = 0; i < CELT_ALLOC_STEPS; i++) { + int center = (low + high) >> 1; + done = total = 0; + + for (j = f->end_band - 1; j >= f->start_band; j--) { + bandbits = bits1[j] + (center * bits2[j] >> CELT_ALLOC_STEPS); + + if (bandbits >= threshold[j] || done) { + done = 1; + total += FFMIN(bandbits, f->caps[j]); + } else if (bandbits >= f->channels << 3) + total += f->channels << 3; } - - if (downmix) - f->dsp->vector_fmul_scalar(&block->buf[1024], &block->buf[1024], 0.5f, frame_size); - - /* postfilter */ - celt_postfilter(f, block); - - /* deemphasis */ - block->emph_coeff = f->opusdsp.deemphasis(output[i], - &block->buf[1024 - frame_size], - block->emph_coeff, frame_size); + if (total > tbits_8ths) + high = center; + else + low = center; } - if (channels == 1) - memcpy(f->block[1].energy, f->block[0].energy, sizeof(f->block[0].energy)); + /* Bisection */ + done = total = 0; + for (i = f->end_band - 1; i >= f->start_band; i--) { + bandbits = bits1[i] + (low * bits2[i] >> CELT_ALLOC_STEPS); - for (i = 0; i < 2; i++ ) { - CeltBlock *block = &f->block[i]; + if (bandbits >= threshold[i] || done) + done = 1; + else + bandbits = (bandbits >= f->channels << 3) ? + f->channels << 3 : 0; - if (!f->transient) { - memcpy(block->prev_energy[1], block->prev_energy[0], sizeof(block->prev_energy[0])); - memcpy(block->prev_energy[0], block->energy, sizeof(block->prev_energy[0])); - } else { - for (j = 0; j < CELT_MAX_BANDS; j++) - block->prev_energy[0][j] = FFMIN(block->prev_energy[0][j], block->energy[j]); - } - - for (j = 0; j < f->start_band; j++) { - block->prev_energy[0][j] = CELT_ENERGY_SILENCE; - block->energy[j] = 0.0; - } - for (j = f->end_band; j < CELT_MAX_BANDS; j++) { - block->prev_energy[0][j] = CELT_ENERGY_SILENCE; - block->energy[j] = 0.0; - } + bandbits = FFMIN(bandbits, f->caps[i]); + f->pulses[i] = bandbits; + total += bandbits; } - f->seed = rc->c.range; + /* Band skipping */ + for (f->coded_bands = f->end_band; ; f->coded_bands--) { + int allocation; + j = f->coded_bands - 1; - return 0; -} - -void ff_celt_flush(CeltFrame *f) -{ - int i, j; - - if (f->flushed) - return; + if (j == skip_startband) { + /* all remaining bands are not skipped */ + tbits_8ths += skip_bit; + break; + } - for (i = 0; i < 2; i++) { - CeltBlock *block = &f->block[i]; + /* determine the number of bits available for coding "do not skip" markers */ + remaining = tbits_8ths - total; + bandbits = remaining / (ff_celt_freq_bands[j+1] - ff_celt_freq_bands[f->start_band]); + remaining -= bandbits * (ff_celt_freq_bands[j+1] - ff_celt_freq_bands[f->start_band]); + allocation = f->pulses[j] + bandbits * ff_celt_freq_range[j]; + allocation += FFMAX(remaining - (ff_celt_freq_bands[j] - ff_celt_freq_bands[f->start_band]), 0); + + /* a "do not skip" marker is only coded if the allocation is + * above the chosen threshold */ + if (allocation >= FFMAX(threshold[j], (f->channels + 1) << 3)) { + int do_not_skip; + if (encode) { + do_not_skip = f->coded_bands <= f->skip_band_floor; + opus_rc_enc_log(rc, do_not_skip, 1); + } else { + do_not_skip = opus_rc_dec_log(rc, 1); + } - for (j = 0; j < CELT_MAX_BANDS; j++) - block->prev_energy[0][j] = block->prev_energy[1][j] = CELT_ENERGY_SILENCE; + if (do_not_skip) + break; - memset(block->energy, 0, sizeof(block->energy)); - memset(block->buf, 0, sizeof(block->buf)); + total += 1 << 3; + allocation -= 1 << 3; + } - memset(block->pf_gains, 0, sizeof(block->pf_gains)); - memset(block->pf_gains_old, 0, sizeof(block->pf_gains_old)); - memset(block->pf_gains_new, 0, sizeof(block->pf_gains_new)); + /* the band is skipped, so reclaim its bits */ + total -= f->pulses[j]; + if (intensitystereo_bit) { + total -= intensitystereo_bit; + intensitystereo_bit = ff_celt_log2_frac[j - f->start_band]; + total += intensitystereo_bit; + } - /* libopus uses CELT_EMPH_COEFF on init, but 0 is better since there's - * a lesser discontinuity when seeking. - * The deemphasis functions differ from libopus in that they require - * an initial state divided by the coefficient. */ - block->emph_coeff = 0.0f / CELT_EMPH_COEFF; + total += f->pulses[j] = (allocation >= f->channels << 3) ? f->channels << 3 : 0; } - f->seed = 0; - - f->flushed = 1; -} - -void ff_celt_free(CeltFrame **f) -{ - CeltFrame *frm = *f; - int i; - - if (!frm) - return; - for (i = 0; i < FF_ARRAY_ELEMS(frm->tx); i++) - av_tx_uninit(&frm->tx[i]); - - ff_celt_pvq_uninit(&frm->pvq); - - av_freep(&frm->dsp); - av_freep(f); -} - -int ff_celt_init(AVCodecContext *avctx, CeltFrame **f, int output_channels, - int apply_phase_inv) -{ - CeltFrame *frm; - int i, ret; + /* IS start band */ + if (encode) { + if (intensitystereo_bit) { + f->intensity_stereo = FFMIN(f->intensity_stereo, f->coded_bands); + ff_opus_rc_enc_uint((OpusEncRangeCoder*)rc, f->intensity_stereo, f->coded_bands + 1 - f->start_band); + } + } else { + f->intensity_stereo = f->dual_stereo = 0; + if (intensitystereo_bit) + f->intensity_stereo = f->start_band + ff_opus_rc_dec_uint((OpusDecRangeCoder*)rc, f->coded_bands + 1 - f->start_band); + } - if (output_channels != 1 && output_channels != 2) { - av_log(avctx, AV_LOG_ERROR, "Invalid number of output channels: %d\n", - output_channels); - return AVERROR(EINVAL); + /* DS flag */ + if (f->intensity_stereo <= f->start_band) + tbits_8ths += dualstereo_bit; /* no intensity stereo means no dual stereo */ + else if (dualstereo_bit) + if (encode) + opus_rc_enc_log(rc, f->dual_stereo, 1); + else + f->dual_stereo = opus_rc_dec_log(rc, 1); + + /* Supply the remaining bits in this frame to lower bands */ + remaining = tbits_8ths - total; + bandbits = remaining / (ff_celt_freq_bands[f->coded_bands] - ff_celt_freq_bands[f->start_band]); + remaining -= bandbits * (ff_celt_freq_bands[f->coded_bands] - ff_celt_freq_bands[f->start_band]); + for (i = f->start_band; i < f->coded_bands; i++) { + const int bits = FFMIN(remaining, ff_celt_freq_range[i]); + f->pulses[i] += bits + bandbits * ff_celt_freq_range[i]; + remaining -= bits; } - frm = av_mallocz(sizeof(*frm)); - if (!frm) - return AVERROR(ENOMEM); + /* Finally determine the allocation */ + for (i = f->start_band; i < f->coded_bands; i++) { + int N = ff_celt_freq_range[i] << f->size; + int prev_extra = extrabits; + f->pulses[i] += extrabits; + + if (N > 1) { + int dof; /* degrees of freedom */ + int temp; /* dof * channels * log(dof) */ + int fine_bits; + int max_bits; + int offset; /* fine energy quantization offset, i.e. + * extra bits assigned over the standard + * totalbits/dof */ + + extrabits = FFMAX(f->pulses[i] - f->caps[i], 0); + f->pulses[i] -= extrabits; + + /* intensity stereo makes use of an extra degree of freedom */ + dof = N * f->channels + (f->channels == 2 && N > 2 && !f->dual_stereo && i < f->intensity_stereo); + temp = dof * (ff_celt_log_freq_range[i] + (f->size << 3)); + offset = (temp >> 1) - dof * CELT_FINE_OFFSET; + if (N == 2) /* dof=2 is the only case that doesn't fit the model */ + offset += dof << 1; + + /* grant an additional bias for the first and second pulses */ + if (f->pulses[i] + offset < 2 * (dof << 3)) + offset += temp >> 2; + else if (f->pulses[i] + offset < 3 * (dof << 3)) + offset += temp >> 3; + + fine_bits = (f->pulses[i] + offset + (dof << 2)) / (dof << 3); + max_bits = FFMIN((f->pulses[i] >> 3) >> (f->channels - 1), CELT_MAX_FINE_BITS); + max_bits = FFMAX(max_bits, 0); + f->fine_bits[i] = av_clip(fine_bits, 0, max_bits); + + /* If fine_bits was rounded down or capped, + * give priority for the final fine energy pass */ + f->fine_priority[i] = (f->fine_bits[i] * (dof << 3) >= f->pulses[i] + offset); + + /* the remaining bits are assigned to PVQ */ + f->pulses[i] -= f->fine_bits[i] << (f->channels - 1) << 3; + } else { + /* all bits go to fine energy except for the sign bit */ + extrabits = FFMAX(f->pulses[i] - (f->channels << 3), 0); + f->pulses[i] -= extrabits; + f->fine_bits[i] = 0; + f->fine_priority[i] = 1; + } - frm->avctx = avctx; - frm->output_channels = output_channels; - frm->apply_phase_inv = apply_phase_inv; + /* hand back a limited number of extra fine energy bits to this band */ + if (extrabits > 0) { + int fineextra = FFMIN(extrabits >> (f->channels + 2), + CELT_MAX_FINE_BITS - f->fine_bits[i]); + f->fine_bits[i] += fineextra; - for (i = 0; i < FF_ARRAY_ELEMS(frm->tx); i++) { - const float scale = -1.0f/32768; - if ((ret = av_tx_init(&frm->tx[i], &frm->tx_fn[i], AV_TX_FLOAT_MDCT, 1, 15 << (i + 3), &scale, 0)) < 0) - goto fail; + fineextra <<= f->channels + 2; + f->fine_priority[i] = (fineextra >= extrabits - prev_extra); + extrabits -= fineextra; + } } + f->remaining = extrabits; - if ((ret = ff_celt_pvq_init(&frm->pvq, 0)) < 0) - goto fail; - - frm->dsp = avpriv_float_dsp_alloc(avctx->flags & AV_CODEC_FLAG_BITEXACT); - if (!frm->dsp) { - ret = AVERROR(ENOMEM); - goto fail; + /* skipped bands dedicate all of their bits for fine energy */ + for (; i < f->end_band; i++) { + f->fine_bits[i] = f->pulses[i] >> (f->channels - 1) >> 3; + f->pulses[i] = 0; + f->fine_priority[i] = f->fine_bits[i] < 1; } - - ff_opus_dsp_init(&frm->opusdsp); - ff_celt_flush(frm); - - *f = frm; - - return 0; -fail: - ff_celt_free(&frm); - return ret; } diff --git a/libavcodec/opusdec_celt.c b/libavcodec/opusdec_celt.c new file mode 100644 index 0000000000..a14764ec18 --- /dev/null +++ b/libavcodec/opusdec_celt.c @@ -0,0 +1,587 @@ +/* + * Copyright (c) 2012 Andrew D'Addesio + * Copyright (c) 2013-2014 Mozilla Corporation + * Copyright (c) 2016 Rostislav Pehlivanov + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * Opus CELT decoder + */ + +#include + +#include "opus_celt.h" +#include "opusdec_rc.h" +#include "opustab.h" +#include "opus_pvq.h" + +/* Use the 2D z-transform to apply prediction in both the time domain (alpha) + * and the frequency domain (beta) */ +static void celt_decode_coarse_energy(CeltFrame *f, OpusDecRangeCoder *rc) +{ + int i, j; + float prev[2] = { 0 }; + float alpha = ff_celt_alpha_coef[f->size]; + float beta = ff_celt_beta_coef[f->size]; + const uint8_t *model = ff_celt_coarse_energy_dist[f->size][0]; + + /* intra frame */ + if (opus_rc_tell(&rc->c) + 3 <= f->framebits && ff_opus_rc_dec_log(rc, 3)) { + alpha = 0.0f; + beta = 1.0f - (4915.0f/32768.0f); + model = ff_celt_coarse_energy_dist[f->size][1]; + } + + for (i = 0; i < CELT_MAX_BANDS; i++) { + for (j = 0; j < f->channels; j++) { + CeltBlock *block = &f->block[j]; + float value; + int available; + + if (i < f->start_band || i >= f->end_band) { + block->energy[i] = 0.0; + continue; + } + + available = f->framebits - opus_rc_tell(&rc->c); + if (available >= 15) { + /* decode using a Laplace distribution */ + int k = FFMIN(i, 20) << 1; + value = ff_opus_rc_dec_laplace(rc, model[k] << 7, model[k+1] << 6); + } else if (available >= 2) { + int x = ff_opus_rc_dec_cdf(rc, ff_celt_model_energy_small); + value = (x>>1) ^ -(x&1); + } else if (available >= 1) { + value = -(float)ff_opus_rc_dec_log(rc, 1); + } else value = -1; + + block->energy[i] = FFMAX(-9.0f, block->energy[i]) * alpha + prev[j] + value; + prev[j] += beta * value; + } + } +} + +static void celt_decode_fine_energy(CeltFrame *f, OpusDecRangeCoder *rc) +{ + int i; + for (i = f->start_band; i < f->end_band; i++) { + int j; + if (!f->fine_bits[i]) + continue; + + for (j = 0; j < f->channels; j++) { + CeltBlock *block = &f->block[j]; + int q2; + float offset; + q2 = ff_opus_rc_get_raw(rc, f->fine_bits[i]); + offset = (q2 + 0.5f) * (1 << (14 - f->fine_bits[i])) / 16384.0f - 0.5f; + block->energy[i] += offset; + } + } +} + +static void celt_decode_final_energy(CeltFrame *f, OpusDecRangeCoder *rc) +{ + int priority, i, j; + int bits_left = f->framebits - opus_rc_tell(&rc->c); + + for (priority = 0; priority < 2; priority++) { + for (i = f->start_band; i < f->end_band && bits_left >= f->channels; i++) { + if (f->fine_priority[i] != priority || f->fine_bits[i] >= CELT_MAX_FINE_BITS) + continue; + + for (j = 0; j < f->channels; j++) { + int q2; + float offset; + q2 = ff_opus_rc_get_raw(rc, 1); + offset = (q2 - 0.5f) * (1 << (14 - f->fine_bits[i] - 1)) / 16384.0f; + f->block[j].energy[i] += offset; + bits_left--; + } + } + } +} + +static void celt_decode_tf_changes(CeltFrame *f, OpusDecRangeCoder *rc) +{ + int i, diff = 0, tf_select = 0, tf_changed = 0, tf_select_bit; + int consumed, bits = f->transient ? 2 : 4; + + consumed = opus_rc_tell(&rc->c); + tf_select_bit = (f->size != 0 && consumed+bits+1 <= f->framebits); + + for (i = f->start_band; i < f->end_band; i++) { + if (consumed+bits+tf_select_bit <= f->framebits) { + diff ^= ff_opus_rc_dec_log(rc, bits); + consumed = opus_rc_tell(&rc->c); + tf_changed |= diff; + } + f->tf_change[i] = diff; + bits = f->transient ? 4 : 5; + } + + if (tf_select_bit && ff_celt_tf_select[f->size][f->transient][0][tf_changed] != + ff_celt_tf_select[f->size][f->transient][1][tf_changed]) + tf_select = ff_opus_rc_dec_log(rc, 1); + + for (i = f->start_band; i < f->end_band; i++) { + f->tf_change[i] = ff_celt_tf_select[f->size][f->transient][tf_select][f->tf_change[i]]; + } +} + +static void celt_denormalize(CeltFrame *f, CeltBlock *block, float *data) +{ + int i, j; + + for (i = f->start_band; i < f->end_band; i++) { + float *dst = data + (ff_celt_freq_bands[i] << f->size); + float log_norm = block->energy[i] + ff_celt_mean_energy[i]; + float norm = exp2f(FFMIN(log_norm, 32.0f)); + + for (j = 0; j < ff_celt_freq_range[i] << f->size; j++) + dst[j] *= norm; + } +} + +static void celt_postfilter_apply_transition(CeltBlock *block, float *data) +{ + const int T0 = block->pf_period_old; + const int T1 = block->pf_period; + + float g00, g01, g02; + float g10, g11, g12; + + float x0, x1, x2, x3, x4; + + int i; + + if (block->pf_gains[0] == 0.0 && + block->pf_gains_old[0] == 0.0) + return; + + g00 = block->pf_gains_old[0]; + g01 = block->pf_gains_old[1]; + g02 = block->pf_gains_old[2]; + g10 = block->pf_gains[0]; + g11 = block->pf_gains[1]; + g12 = block->pf_gains[2]; + + x1 = data[-T1 + 1]; + x2 = data[-T1]; + x3 = data[-T1 - 1]; + x4 = data[-T1 - 2]; + + for (i = 0; i < CELT_OVERLAP; i++) { + float w = ff_celt_window2[i]; + x0 = data[i - T1 + 2]; + + data[i] += (1.0 - w) * g00 * data[i - T0] + + (1.0 - w) * g01 * (data[i - T0 - 1] + data[i - T0 + 1]) + + (1.0 - w) * g02 * (data[i - T0 - 2] + data[i - T0 + 2]) + + w * g10 * x2 + + w * g11 * (x1 + x3) + + w * g12 * (x0 + x4); + x4 = x3; + x3 = x2; + x2 = x1; + x1 = x0; + } +} + +static void celt_postfilter(CeltFrame *f, CeltBlock *block) +{ + int len = f->blocksize * f->blocks; + const int filter_len = len - 2 * CELT_OVERLAP; + + celt_postfilter_apply_transition(block, block->buf + 1024); + + block->pf_period_old = block->pf_period; + memcpy(block->pf_gains_old, block->pf_gains, sizeof(block->pf_gains)); + + block->pf_period = block->pf_period_new; + memcpy(block->pf_gains, block->pf_gains_new, sizeof(block->pf_gains)); + + if (len > CELT_OVERLAP) { + celt_postfilter_apply_transition(block, block->buf + 1024 + CELT_OVERLAP); + + if (block->pf_gains[0] > FLT_EPSILON && filter_len > 0) + f->opusdsp.postfilter(block->buf + 1024 + 2 * CELT_OVERLAP, + block->pf_period, block->pf_gains, + filter_len); + + block->pf_period_old = block->pf_period; + memcpy(block->pf_gains_old, block->pf_gains, sizeof(block->pf_gains)); + } + + memmove(block->buf, block->buf + len, (1024 + CELT_OVERLAP / 2) * sizeof(float)); +} + +static int parse_postfilter(CeltFrame *f, OpusDecRangeCoder *rc, int consumed) +{ + int i; + + memset(f->block[0].pf_gains_new, 0, sizeof(f->block[0].pf_gains_new)); + memset(f->block[1].pf_gains_new, 0, sizeof(f->block[1].pf_gains_new)); + + if (f->start_band == 0 && consumed + 16 <= f->framebits) { + int has_postfilter = ff_opus_rc_dec_log(rc, 1); + if (has_postfilter) { + float gain; + int tapset, octave, period; + + octave = ff_opus_rc_dec_uint(rc, 6); + period = (16 << octave) + ff_opus_rc_get_raw(rc, 4 + octave) - 1; + gain = 0.09375f * (ff_opus_rc_get_raw(rc, 3) + 1); + tapset = (opus_rc_tell(&rc->c) + 2 <= f->framebits) ? + ff_opus_rc_dec_cdf(rc, ff_celt_model_tapset) : 0; + + for (i = 0; i < 2; i++) { + CeltBlock *block = &f->block[i]; + + block->pf_period_new = FFMAX(period, CELT_POSTFILTER_MINPERIOD); + block->pf_gains_new[0] = gain * ff_celt_postfilter_taps[tapset][0]; + block->pf_gains_new[1] = gain * ff_celt_postfilter_taps[tapset][1]; + block->pf_gains_new[2] = gain * ff_celt_postfilter_taps[tapset][2]; + } + } + + consumed = opus_rc_tell(&rc->c); + } + + return consumed; +} + +static void process_anticollapse(CeltFrame *f, CeltBlock *block, float *X) +{ + int i, j, k; + + for (i = f->start_band; i < f->end_band; i++) { + int renormalize = 0; + float *xptr; + float prev[2]; + float Ediff, r; + float thresh, sqrt_1; + int depth; + + /* depth in 1/8 bits */ + depth = (1 + f->pulses[i]) / (ff_celt_freq_range[i] << f->size); + thresh = exp2f(-1.0 - 0.125f * depth); + sqrt_1 = 1.0f / sqrtf(ff_celt_freq_range[i] << f->size); + + xptr = X + (ff_celt_freq_bands[i] << f->size); + + prev[0] = block->prev_energy[0][i]; + prev[1] = block->prev_energy[1][i]; + if (f->channels == 1) { + CeltBlock *block1 = &f->block[1]; + + prev[0] = FFMAX(prev[0], block1->prev_energy[0][i]); + prev[1] = FFMAX(prev[1], block1->prev_energy[1][i]); + } + Ediff = block->energy[i] - FFMIN(prev[0], prev[1]); + Ediff = FFMAX(0, Ediff); + + /* r needs to be multiplied by 2 or 2*sqrt(2) depending on LM because + short blocks don't have the same energy as long */ + r = exp2f(1 - Ediff); + if (f->size == 3) + r *= M_SQRT2; + r = FFMIN(thresh, r) * sqrt_1; + for (k = 0; k < 1 << f->size; k++) { + /* Detect collapse */ + if (!(block->collapse_masks[i] & 1 << k)) { + /* Fill with noise */ + for (j = 0; j < ff_celt_freq_range[i]; j++) + xptr[(j << f->size) + k] = (celt_rng(f) & 0x8000) ? r : -r; + renormalize = 1; + } + } + + /* We just added some energy, so we need to renormalize */ + if (renormalize) + celt_renormalize_vector(xptr, ff_celt_freq_range[i] << f->size, 1.0f); + } +} + +int ff_celt_decode_frame(CeltFrame *f, OpusDecRangeCoder *rc, + float **output, int channels, int frame_size, + int start_band, int end_band) +{ + int i, j, downmix = 0; + int consumed; // bits of entropy consumed thus far for this frame + AVTXContext *imdct; + av_tx_fn imdct_fn; + + if (channels != 1 && channels != 2) { + av_log(f->avctx, AV_LOG_ERROR, "Invalid number of coded channels: %d\n", + channels); + return AVERROR_INVALIDDATA; + } + if (start_band < 0 || start_band > end_band || end_band > CELT_MAX_BANDS) { + av_log(f->avctx, AV_LOG_ERROR, "Invalid start/end band: %d %d\n", + start_band, end_band); + return AVERROR_INVALIDDATA; + } + + f->silence = 0; + f->transient = 0; + f->anticollapse = 0; + f->flushed = 0; + f->channels = channels; + f->start_band = start_band; + f->end_band = end_band; + f->framebits = rc->c.rb.bytes * 8; + + f->size = av_log2(frame_size / CELT_SHORT_BLOCKSIZE); + if (f->size > CELT_MAX_LOG_BLOCKS || + frame_size != CELT_SHORT_BLOCKSIZE * (1 << f->size)) { + av_log(f->avctx, AV_LOG_ERROR, "Invalid CELT frame size: %d\n", + frame_size); + return AVERROR_INVALIDDATA; + } + + if (!f->output_channels) + f->output_channels = channels; + + for (i = 0; i < f->channels; i++) { + memset(f->block[i].coeffs, 0, sizeof(f->block[i].coeffs)); + memset(f->block[i].collapse_masks, 0, sizeof(f->block[i].collapse_masks)); + } + + consumed = opus_rc_tell(&rc->c); + + /* obtain silence flag */ + if (consumed >= f->framebits) + f->silence = 1; + else if (consumed == 1) + f->silence = ff_opus_rc_dec_log(rc, 15); + + + if (f->silence) { + consumed = f->framebits; + rc->c.total_bits += f->framebits - opus_rc_tell(&rc->c); + } + + /* obtain post-filter options */ + consumed = parse_postfilter(f, rc, consumed); + + /* obtain transient flag */ + if (f->size != 0 && consumed+3 <= f->framebits) + f->transient = ff_opus_rc_dec_log(rc, 3); + + f->blocks = f->transient ? 1 << f->size : 1; + f->blocksize = frame_size / f->blocks; + + imdct = f->tx[f->transient ? 0 : f->size]; + imdct_fn = f->tx_fn[f->transient ? 0 : f->size]; + + if (channels == 1) { + for (i = 0; i < CELT_MAX_BANDS; i++) + f->block[0].energy[i] = FFMAX(f->block[0].energy[i], f->block[1].energy[i]); + } + + celt_decode_coarse_energy(f, rc); + celt_decode_tf_changes (f, rc); + ff_celt_bitalloc (f, &rc->c, 0); + celt_decode_fine_energy (f, rc); + ff_celt_quant_bands (f, &rc->c); + + if (f->anticollapse_needed) + f->anticollapse = ff_opus_rc_get_raw(rc, 1); + + celt_decode_final_energy(f, rc); + + /* apply anti-collapse processing and denormalization to + * each coded channel */ + for (i = 0; i < f->channels; i++) { + CeltBlock *block = &f->block[i]; + + if (f->anticollapse) + process_anticollapse(f, block, f->block[i].coeffs); + + celt_denormalize(f, block, f->block[i].coeffs); + } + + /* stereo -> mono downmix */ + if (f->output_channels < f->channels) { + f->dsp->vector_fmac_scalar(f->block[0].coeffs, f->block[1].coeffs, 1.0, FFALIGN(frame_size, 16)); + downmix = 1; + } else if (f->output_channels > f->channels) + memcpy(f->block[1].coeffs, f->block[0].coeffs, frame_size * sizeof(float)); + + if (f->silence) { + for (i = 0; i < 2; i++) { + CeltBlock *block = &f->block[i]; + + for (j = 0; j < FF_ARRAY_ELEMS(block->energy); j++) + block->energy[j] = CELT_ENERGY_SILENCE; + } + memset(f->block[0].coeffs, 0, sizeof(f->block[0].coeffs)); + memset(f->block[1].coeffs, 0, sizeof(f->block[1].coeffs)); + } + + /* transform and output for each output channel */ + for (i = 0; i < f->output_channels; i++) { + CeltBlock *block = &f->block[i]; + + /* iMDCT and overlap-add */ + for (j = 0; j < f->blocks; j++) { + float *dst = block->buf + 1024 + j * f->blocksize; + + imdct_fn(imdct, dst + CELT_OVERLAP / 2, f->block[i].coeffs + j, + sizeof(float)*f->blocks); + f->dsp->vector_fmul_window(dst, dst, dst + CELT_OVERLAP / 2, + ff_celt_window, CELT_OVERLAP / 2); + } + + if (downmix) + f->dsp->vector_fmul_scalar(&block->buf[1024], &block->buf[1024], 0.5f, frame_size); + + /* postfilter */ + celt_postfilter(f, block); + + /* deemphasis */ + block->emph_coeff = f->opusdsp.deemphasis(output[i], + &block->buf[1024 - frame_size], + block->emph_coeff, frame_size); + } + + if (channels == 1) + memcpy(f->block[1].energy, f->block[0].energy, sizeof(f->block[0].energy)); + + for (i = 0; i < 2; i++ ) { + CeltBlock *block = &f->block[i]; + + if (!f->transient) { + memcpy(block->prev_energy[1], block->prev_energy[0], sizeof(block->prev_energy[0])); + memcpy(block->prev_energy[0], block->energy, sizeof(block->prev_energy[0])); + } else { + for (j = 0; j < CELT_MAX_BANDS; j++) + block->prev_energy[0][j] = FFMIN(block->prev_energy[0][j], block->energy[j]); + } + + for (j = 0; j < f->start_band; j++) { + block->prev_energy[0][j] = CELT_ENERGY_SILENCE; + block->energy[j] = 0.0; + } + for (j = f->end_band; j < CELT_MAX_BANDS; j++) { + block->prev_energy[0][j] = CELT_ENERGY_SILENCE; + block->energy[j] = 0.0; + } + } + + f->seed = rc->c.range; + + return 0; +} + +void ff_celt_flush(CeltFrame *f) +{ + int i, j; + + if (f->flushed) + return; + + for (i = 0; i < 2; i++) { + CeltBlock *block = &f->block[i]; + + for (j = 0; j < CELT_MAX_BANDS; j++) + block->prev_energy[0][j] = block->prev_energy[1][j] = CELT_ENERGY_SILENCE; + + memset(block->energy, 0, sizeof(block->energy)); + memset(block->buf, 0, sizeof(block->buf)); + + memset(block->pf_gains, 0, sizeof(block->pf_gains)); + memset(block->pf_gains_old, 0, sizeof(block->pf_gains_old)); + memset(block->pf_gains_new, 0, sizeof(block->pf_gains_new)); + + /* libopus uses CELT_EMPH_COEFF on init, but 0 is better since there's + * a lesser discontinuity when seeking. + * The deemphasis functions differ from libopus in that they require + * an initial state divided by the coefficient. */ + block->emph_coeff = 0.0f / CELT_EMPH_COEFF; + } + f->seed = 0; + + f->flushed = 1; +} + +void ff_celt_free(CeltFrame **f) +{ + CeltFrame *frm = *f; + int i; + + if (!frm) + return; + + for (i = 0; i < FF_ARRAY_ELEMS(frm->tx); i++) + av_tx_uninit(&frm->tx[i]); + + ff_celt_pvq_uninit(&frm->pvq); + + av_freep(&frm->dsp); + av_freep(f); +} + +int ff_celt_init(AVCodecContext *avctx, CeltFrame **f, int output_channels, + int apply_phase_inv) +{ + CeltFrame *frm; + int i, ret; + + if (output_channels != 1 && output_channels != 2) { + av_log(avctx, AV_LOG_ERROR, "Invalid number of output channels: %d\n", + output_channels); + return AVERROR(EINVAL); + } + + frm = av_mallocz(sizeof(*frm)); + if (!frm) + return AVERROR(ENOMEM); + + frm->avctx = avctx; + frm->output_channels = output_channels; + frm->apply_phase_inv = apply_phase_inv; + + for (i = 0; i < FF_ARRAY_ELEMS(frm->tx); i++) { + const float scale = -1.0f/32768; + if ((ret = av_tx_init(&frm->tx[i], &frm->tx_fn[i], AV_TX_FLOAT_MDCT, 1, 15 << (i + 3), &scale, 0)) < 0) + goto fail; + } + + if ((ret = ff_celt_pvq_init(&frm->pvq, 0)) < 0) + goto fail; + + frm->dsp = avpriv_float_dsp_alloc(avctx->flags & AV_CODEC_FLAG_BITEXACT); + if (!frm->dsp) { + ret = AVERROR(ENOMEM); + goto fail; + } + + ff_opus_dsp_init(&frm->opusdsp); + ff_celt_flush(frm); + + *f = frm; + + return 0; +fail: + ff_celt_free(&frm); + return ret; +}