From patchwork Fri Apr 20 17:23:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Drew Allen X-Patchwork-Id: 8536 Delivered-To: ffmpegpatchwork@gmail.com Received: by 2002:a02:155:0:0:0:0:0 with SMTP id c82-v6csp932067jad; Fri, 20 Apr 2018 10:30:45 -0700 (PDT) X-Google-Smtp-Source: AIpwx49LQ8WPHTJkJUcfQXx8kQ9RzwLygNvo0KjZVjAYxMR2prKjHIk5y9Nth49MLgQ6RBaML8bH X-Received: by 2002:adf:8861:: with SMTP id e30-v6mr8452074wre.252.1524245445714; Fri, 20 Apr 2018 10:30:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1524245445; cv=none; d=google.com; s=arc-20160816; b=AHYtReD0aA7PjaRqhLeKDPu9ovyXvNubPSBM/58PIkmMaeE5pgftcLTTQfJMrcIPYU cVCabjqvNKv7qpQgLgwcDfUcXaJuNc9TF6ibSHYG74UvjDfyxZ/vTbHZATc7ZXPqvVQt RNJZflsd4bgesIrzMSCyfX3uOZMCruMeYRe17vpDLpP0QpczPccy4Hzv0BB/+l2qx7nf kKQun20XDgWD0TJhlRbbYiT4quACWW2igxmlZ8kPmzAQUsFvMlWdi/KMVLnbCcQq78LP zx+m7DLbFKPuNADUptNoEp6p3/m1Qq3Vv4MYXf4h7Y65Vi6zCTJPtIUgi7CYysaUQPsP y90Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject:to :message-id:date:from:in-reply-to:references:mime-version :dkim-signature:delivered-to:arc-authentication-results; bh=MCFqztklNjQHT9wX3po4m3TqUmueh2PG5GpYdCZ1cGY=; b=TFsmLOBImPO/Ca3P/6rhl2bhQyuuBO03IB4rBGt72F8+vEUuhToqPet6rZOm2sRcvU ok1EiMnH3T91BC69PDpORcwu9tF82ZfsW2r6yTtKXumKUmf4QLO/xN1YX+csmlquwAoT /hEJmsuuhPoyrrr3jIwKnP7ozc07NCfLpJXcJk+eGetH+kK2ipUosvJA+8qHrbb148bK MqDTkuw9swrm1LwXnC20GeJXrW1bq3yFwsTqf8QTnhniPGiHCtwiaEgryt8XWC2IQaZE 7C0343vmh+T9TVE7b12epUtcexYmu1QzXiDqN2kvUfsz6aMW73h64O1POB+rOTrCzgft SfiQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20161025 header.b=AMPsmg7w; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y21-v6si5232527wrb.347.2018.04.20.10.30.44; Fri, 20 Apr 2018 10:30:45 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20161025 header.b=AMPsmg7w; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id AF89E6899F7; Fri, 20 Apr 2018 20:30:14 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-it0-f49.google.com (mail-it0-f49.google.com [209.85.214.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4DA4E6898CA for ; Fri, 20 Apr 2018 20:30:07 +0300 (EEST) Received: by mail-it0-f49.google.com with SMTP id h143-v6so3479678ita.4 for ; Fri, 20 Apr 2018 10:30:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ZA0PnydaHG2CPIBQqGY5JjzpvIuRsRY9pbWf2i/ncL0=; b=AMPsmg7wlT62dMjnMxi1z2sK5rnNijB2xp8tQhfn+JXTyBG/dyqgaYDW4pXQAudfvt IFkHBCFiII4/jraihmqI/l/5HTsh7Ov8QzNQN2yT0xh2AP5PoX9pDF4NdLz46FwTz7S7 O7ZRAxqBgY9Qo+v704Jnd5nvezrBWzOKmMEVwK8So8VfezaPLcMdhVdIYV4xlPtfjDXk yCXzqKWl1mIrf52v0O2xvufFQEnheT0vxD9zEwLWIiwoC2+jmZZabeUSxBlCHJRFNfb5 agn/2kZYT83FQ542buz6pkv6rZ00GK20GEce+85dtNoZIut4bjAf5cm+Jt2Bf1FxRpSY 8L5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ZA0PnydaHG2CPIBQqGY5JjzpvIuRsRY9pbWf2i/ncL0=; b=JVCsW6fwNv3V1S9Whx8FrmrOKcT15WbfLmQqSoepNcm7/Ta73UYobL1wMmcHtVDxwD QKYt69BUV71UPZEXEhLlD0Fai9TPI/Pb/R5u6F7nu4lq8o0D8xNNcMnCuunezFdhAfqV ox7JoAYK+mkiKFSCKU8tgiZyDkaUVWAJ8DrWR6YuEx6FL7lSfjYJJQghCYELQcjx7kjj X0iSq9YGealb6McD1Uzj2Z1/luoX/E4qHiSp4+2cwral3FhwZMxrzJYbBFRZKogMfwHz LYDCkKs5I/3qihHA4ZsoLgD/ytfwHvJafs3KagMatdLLXO0iHPAd+dLXgyRmH0VnnT1F nFpQ== X-Gm-Message-State: ALQs6tDTY2wuogB6O8iuEm0t6PeSqsqb+0ag5zXouw12N4VSo6wYIZiV 2+IoTf9hQKGE6QHEGPbLcP7xN2Erc2T3A3Qy1MovCZZv X-Received: by 2002:a24:5494:: with SMTP id t142-v6mr4044670ita.132.1524245019162; Fri, 20 Apr 2018 10:23:39 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Drew Allen Date: Fri, 20 Apr 2018 17:23:28 +0000 Message-ID: To: ffmpeg-devel@ffmpeg.org X-Content-Filtered-By: Mailman/MimeDel 2.1.20 Subject: Re: [FFmpeg-devel] [PATCH] Support for Ambisonics and OpusProjection* API. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Damien Kelly Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Hello all, Attached is a revised patch based on your suggestions. Rostislav Pehlivanov - I'm not sure how to address your concerns. My patch provides support for the ambisonics features already approved and present in Opus 1.3. If there are ways I can change my patch, please let me know. Cheers, Drew On Mon, Apr 9, 2018 at 10:57 AM Drew Allen wrote: > Friendly weekly ping about this patch. > > Can someone please let me know if I need to do anything more to submit > this patch? Thanks! > > On Mon, Apr 2, 2018 at 9:13 AM Drew Allen wrote: > >> Friendly ping to allow this to push to the member list, please. >> >> On Wed, Mar 28, 2018 at 2:58 PM Drew Allen wrote: >> >>> Hello all, >>> >>> My name is Andrew Allen and I'm a contributor to Opus, supporting new >>> channel mappings 2 and 3 for ambisonics compression. I've attached a patch >>> to support the new OpusProjectionEncoder and OpusProjectionDecoder APIs for >>> handling the new channel mapping 3 in OPUS. >>> >>> Please let me know of any changes I should make or if there are any >>> questions I can help answer. >>> >>> Cheers, >>> Drew >>> >> From 35f9b25fbe5dd086b6fab0276d00491b4783cd64 Mon Sep 17 00:00:00 2001 From: Andrew Allen Date: Wed, 28 Mar 2018 14:48:46 -0700 Subject: [PATCH] Support for Ambisonics and OpusProjection* API. --- libavcodec/libopusdec.c | 159 ++++++++++++++++++++----- libavcodec/libopusenc.c | 253 ++++++++++++++++++++++++++++++++++------ libavcodec/opus.c | 18 ++- 3 files changed, 350 insertions(+), 80 deletions(-) diff --git a/libavcodec/libopusdec.c b/libavcodec/libopusdec.c index 2a97811d18..5caf8f3f53 100644 --- a/libavcodec/libopusdec.c +++ b/libavcodec/libopusdec.c @@ -21,6 +21,9 @@ #include #include +#ifdef OPUS_HAVE_OPUS_PROJECTION_H +#include +#endif #include "libavutil/internal.h" #include "libavutil/intreadwrite.h" @@ -33,9 +36,89 @@ #include "mathops.h" #include "libopus.h" +typedef struct OpusGenericDecoder { +#ifdef OPUS_HAVE_OPUS_PROJECTION_H + OpusProjectionDecoder *pr; +#endif + OpusMSDecoder *ms; +} OpusGenericDecoder; + +static int libopus_generic_decoder_init(OpusGenericDecoder *enc, int Fs, + int channels, int nb_streams, + int nb_coupled, uint8_t *mapping, + uint8_t *dmatrix) { + int err; + if (dmatrix != NULL) { +#ifdef OPUS_HAVE_OPUS_PROJECTION_H + opus_int32 size; + size = 2 * channels * (nb_streams + nb_coupled); + enc->pr = opus_projection_decoder_create(Fs, channels, nb_streams, + nb_coupled, dmatrix, size, &err); +#else + err = OPUS_UNIMPLEMENTED; +#endif + return err; + } + enc->ms = opus_multistream_decoder_create(Fs, channels, nb_streams, + nb_coupled, mapping, &err); + return err; +} + +static void libopus_generic_decoder_cleanup(OpusGenericDecoder *enc) +{ +#ifdef OPUS_HAVE_OPUS_PROJECTION_H + if (enc->pr) opus_projection_decoder_destroy(enc->pr); +#endif + if (enc->ms) opus_multistream_decoder_destroy(enc->ms); +} + +static int libopus_generic_decode(OpusGenericDecoder *enc, + const unsigned char *data, opus_int32 len, opus_int16 *pcm, + int frame_size, int decode_fec) { + int ret; + +#ifdef OPUS_HAVE_OPUS_PROJECTION_H + if (enc->pr){ + ret=opus_projection_decode(enc->pr, data, len, pcm, frame_size, + decode_fec); + return ret; + } +#endif + ret=opus_multistream_decode(enc->ms, data, len, pcm, frame_size, + decode_fec); + return ret; +} + +static int libopus_generic_decode_float(OpusGenericDecoder *enc, + const unsigned char *data, opus_int32 len, float *pcm, int frame_size, + int decode_fec) { + int ret; + +#ifdef OPUS_HAVE_OPUS_PROJECTION_H + if (enc->pr){ + ret=opus_projection_decode_float(enc->pr, data, len, pcm, frame_size, + decode_fec); + return ret; + } +#endif + ret=opus_multistream_decode_float(enc->ms, data, len, pcm, frame_size, + decode_fec); + return ret; +} + +#ifdef OPUS_HAVE_OPUS_PROJECTION_H +# define libopus_generic_decoder_ctl(enc, request) \ + ((enc)->pr != NULL ? \ + opus_projection_decoder_ctl((enc)->pr, request) : \ + opus_multistream_decoder_ctl((enc)->ms, request)) +#else +# define libopus_generic_decoder_ctl(enc, request) \ + opus_multistream_decoder_ctl((enc)->ms, request) +#endif + struct libopus_context { AVClass *class; - OpusMSDecoder *dec; + OpusGenericDecoder dec; int pre_skip; #ifndef OPUS_SET_GAIN union { int i; double d; } gain; @@ -46,12 +129,17 @@ struct libopus_context { }; #define OPUS_HEAD_SIZE 19 +#ifdef OPUS_HAVE_OPUS_PROJECTION_H +# define OPUS_MAX_CHANNELS 18 +#else +# define OPUS_MAX_CHANNELS 8 +#endif static av_cold int libopus_decode_init(AVCodecContext *avc) { struct libopus_context *opus = avc->priv_data; int ret, channel_map = 0, gain_db = 0, nb_streams, nb_coupled; - uint8_t mapping_arr[8] = { 0, 1 }, *mapping; + uint8_t mapping_arr[OPUS_MAX_CHANNELS] = { 0, 1 }, *mapping, *dmatrix = NULL; avc->channels = avc->extradata_size >= 10 ? avc->extradata[9] : (avc->channels == 1) ? 1 : 2; if (avc->channels <= 0) { @@ -74,7 +162,21 @@ static av_cold int libopus_decode_init(AVCodecContext *avc) nb_coupled = avc->extradata[OPUS_HEAD_SIZE + 1]; if (nb_streams + nb_coupled != avc->channels) av_log(avc, AV_LOG_WARNING, "Inconsistent channel mapping.\n"); - mapping = avc->extradata + OPUS_HEAD_SIZE + 2; + if (channel_map == 3) { + int ch; + if (avc->extradata_size >= OPUS_HEAD_SIZE + 2 + 2 * avc->channels * (nb_streams + nb_coupled)) { + dmatrix =avc->extradata + OPUS_HEAD_SIZE + 2; + } else { + av_log(avc, AV_LOG_ERROR, + "Demixing matrix not present.\n"); + return AVERROR_INVALIDDATA; + } + for (ch = 0; ch < avc->channels; ch++) + mapping_arr[ch] = ch; + mapping = mapping_arr; + } else { + mapping = avc->extradata + OPUS_HEAD_SIZE + 2; + } } else { if (avc->channels > 2 || channel_map) { av_log(avc, AV_LOG_ERROR, @@ -98,18 +200,15 @@ static av_cold int libopus_decode_init(AVCodecContext *avc) mapping_arr[ch] = mapping[vorbis_offset[ch]]; mapping = mapping_arr; } - } else if (channel_map == 2) { - int ambisonic_order = ff_sqrt(avc->channels) - 1; - if (avc->channels != (ambisonic_order + 1) * (ambisonic_order + 1) && - avc->channels != (ambisonic_order + 1) * (ambisonic_order + 1) + 2) { + } else if (channel_map == 2 || channel_map == 3) { + int order_plus_one = ff_sqrt(avc->channels); + int nondiegetic_channels = avc->channels - order_plus_one * order_plus_one; + if (order_plus_one < 1 || order_plus_one > 15 || + (nondiegetic_channels != 0 && nondiegetic_channels != 2)) { av_log(avc, AV_LOG_ERROR, - "Channel mapping 2 is only specified for channel counts" - " which can be written as (n + 1)^2 or (n + 2)^2 + 2" - " for nonnegative integer n\n"); - return AVERROR_INVALIDDATA; - } - if (avc->channels > 227) { - av_log(avc, AV_LOG_ERROR, "Too many channels\n"); + "This channel mapping is only specified for channel counts" + " which can be written as (n + 1)^2 + 2j, where n is a" + " non-negative integar from 0 to 14 and j is either 0 or 1.\n"); return AVERROR_INVALIDDATA; } avc->channel_layout = 0; @@ -117,17 +216,16 @@ static av_cold int libopus_decode_init(AVCodecContext *avc) avc->channel_layout = 0; } - opus->dec = opus_multistream_decoder_create(avc->sample_rate, avc->channels, - nb_streams, nb_coupled, - mapping, &ret); - if (!opus->dec) { + ret = libopus_generic_decoder_init(&opus->dec, avc->sample_rate, avc->channels, + nb_streams, nb_coupled, mapping, dmatrix); + if (ret != OPUS_OK) { av_log(avc, AV_LOG_ERROR, "Unable to create decoder: %s\n", opus_strerror(ret)); return ff_opus_error_to_averror(ret); } #ifdef OPUS_SET_GAIN - ret = opus_multistream_decoder_ctl(opus->dec, OPUS_SET_GAIN(gain_db)); + ret = libopus_generic_decoder_ctl(&opus->dec, OPUS_SET_GAIN(gain_db)); if (ret != OPUS_OK) av_log(avc, AV_LOG_WARNING, "Failed to set gain: %s\n", opus_strerror(ret)); @@ -142,8 +240,8 @@ static av_cold int libopus_decode_init(AVCodecContext *avc) #endif #ifdef OPUS_SET_PHASE_INVERSION_DISABLED_REQUEST - ret = opus_multistream_decoder_ctl(opus->dec, - OPUS_SET_PHASE_INVERSION_DISABLED(!opus->apply_phase_inv)); + ret = libopus_generic_decoder_ctl(&opus->dec, + OPUS_SET_PHASE_INVERSION_DISABLED(!opus->apply_phase_inv)); if (ret != OPUS_OK) av_log(avc, AV_LOG_WARNING, "Unable to set phase inversion: %s\n", @@ -160,10 +258,7 @@ static av_cold int libopus_decode_close(AVCodecContext *avc) { struct libopus_context *opus = avc->priv_data; - if (opus->dec) { - opus_multistream_decoder_destroy(opus->dec); - opus->dec = NULL; - } + libopus_generic_decoder_cleanup(&opus->dec); return 0; } @@ -181,13 +276,13 @@ static int libopus_decode(AVCodecContext *avc, void *data, return ret; if (avc->sample_fmt == AV_SAMPLE_FMT_S16) - nb_samples = opus_multistream_decode(opus->dec, pkt->data, pkt->size, - (opus_int16 *)frame->data[0], - frame->nb_samples, 0); + nb_samples = libopus_generic_decode(&opus->dec, pkt->data, pkt->size, + (opus_int16 *)frame->data[0], + frame->nb_samples, 0); else - nb_samples = opus_multistream_decode_float(opus->dec, pkt->data, pkt->size, - (float *)frame->data[0], - frame->nb_samples, 0); + nb_samples = libopus_generic_decode_float(&opus->dec, pkt->data, pkt->size, + (float *)frame->data[0], + frame->nb_samples, 0); if (nb_samples < 0) { av_log(avc, AV_LOG_ERROR, "Decoding error: %s\n", @@ -220,7 +315,7 @@ static void libopus_flush(AVCodecContext *avc) { struct libopus_context *opus = avc->priv_data; - opus_multistream_decoder_ctl(opus->dec, OPUS_RESET_STATE); + libopus_generic_decoder_ctl(&opus->dec, OPUS_RESET_STATE); /* The stream can have been extracted by a tool that is not Opus-aware. Therefore, any packet can become the first of the stream. */ avc->internal->skip_samples = opus->pre_skip; diff --git a/libavcodec/libopusenc.c b/libavcodec/libopusenc.c index 4ae81b0bb2..d3305a1459 100644 --- a/libavcodec/libopusenc.c +++ b/libavcodec/libopusenc.c @@ -21,15 +21,135 @@ #include #include +#ifdef OPUS_HAVE_OPUS_PROJECTION_H +#include +#endif #include "libavutil/opt.h" #include "avcodec.h" #include "bytestream.h" #include "internal.h" #include "libopus.h" +#include "mathops.h" #include "vorbis.h" #include "audio_frame_queue.h" +typedef struct OpusGenericEncoder { +#ifdef OPUS_HAVE_OPUS_PROJECTION_H + OpusProjectionEncoder *pr; +#endif + OpusMSEncoder *ms; +} OpusGenericEncoder; + +static int libopus_generic_encoder_surround_init(OpusGenericEncoder *enc, + int Fs, + int channels, + int mapping_family, + int *nb_streams, + int *nb_coupled, + unsigned char *stream_map, + int application) +{ + int ret; + if (mapping_family == 3) { +#ifdef OPUS_HAVE_OPUS_PROJECTION_H + int ci; + enc->pr = opus_projection_ambisonics_encoder_create( + Fs, channels, mapping_family, nb_streams, nb_coupled, + application, &ret); + for (ci = 0; ci < channels; ci++) + stream_map[ci] = ci; +#else + ret = OPUS_UNIMPLEMENTED; +#endif + return ret; + } + enc->ms = opus_multistream_surround_encoder_create( + Fs, channels, mapping_family, nb_streams, nb_coupled, stream_map, + application, &ret); + return ret; +} + +static int libopus_generic_encoder_init(OpusGenericEncoder *enc, int Fs, + int channels, int streams, + int coupled_streams, + const unsigned char *mapping, + int application) +{ + int ret; + enc->ms = opus_multistream_encoder_create(Fs, channels, streams, + coupled_streams, mapping, application, &ret); + return ret; +} + +static int libopus_generic_encode(OpusGenericEncoder *enc, + const opus_int16 *pcm, + int frame_size, unsigned char *data, + opus_int32 max_data_bytes) +{ + int ret; +#ifdef OPUS_HAVE_OPUS_PROJECTION_H + if (enc->pr) { + ret = opus_projection_encode(enc->pr, pcm, frame_size, data, + max_data_bytes); + return ret; + } +#endif + ret = opus_multistream_encode(enc->ms, pcm, frame_size, data, + max_data_bytes); + return ret; +} + +static int libopus_generic_encode_float(OpusGenericEncoder *enc, + const float *pcm, + int frame_size, unsigned char *data, + opus_int32 max_data_bytes) +{ + int ret; +#ifdef OPUS_HAVE_OPUS_PROJECTION_H + if (enc->pr) { + ret = opus_projection_encode_float(enc->pr, pcm, frame_size, data, + max_data_bytes); + return ret; + } +#endif + ret = opus_multistream_encode_float(enc->ms, pcm, frame_size, data, + max_data_bytes); + return ret; +} + +static void libous_generic_encoder_cleanup(OpusGenericEncoder *enc) +{ +#ifdef OPUS_HAVE_OPUS_PROJECTION_H + if (enc->pr) opus_projection_encoder_destroy(enc->pr); +#endif + if (enc->ms) opus_multistream_encoder_destroy(enc->ms); +} + +#ifdef OPUS_HAVE_OPUS_PROJECTION_H +# define libopus_generic_encoder_ctl(enc, request) \ + ((enc)->pr != NULL ? \ + opus_projection_encoder_ctl((enc)->pr, request) : \ + opus_multistream_encoder_ctl((enc)->ms, request)) +#else +# define libopus_generic_encoder_ctl(enc, request) \ + opus_multistream_encoder_ctl((enc)->ms, request) +#endif + +static int libopus_generic_encoder_get_header_size(int mapping_family, + int channels, int streams, + int coupled_streams) +{ + int size = 19; + if (mapping_family == 1 || mapping_family == 2 || mapping_family == 255) { + return size + 2 + channels; + } + else if (mapping_family == 3) { + return size + 2 + 2 * channels * (streams + coupled_streams); + } + return size; +} + typedef struct LibopusEncOpts { int vbr; int application; @@ -46,7 +166,7 @@ typedef struct LibopusEncOpts { typedef struct LibopusEncContext { AVClass *class; - OpusMSEncoder *enc; + OpusGenericEncoder enc; int stream_count; uint8_t *samples; LibopusEncOpts opts; @@ -85,28 +205,68 @@ static const uint8_t libavcodec_libopus_channel_map[8][8] = { static void libopus_write_header(AVCodecContext *avctx, int stream_count, int coupled_stream_count, int mapping_family, - const uint8_t *channel_mapping) + const uint8_t *channel_mapping, + OpusGenericEncoder *enc) { uint8_t *p = avctx->extradata; int channels = avctx->channels; + int gain = 0; bytestream_put_buffer(&p, "OpusHead", 8); bytestream_put_byte(&p, 1); /* Version */ bytestream_put_byte(&p, channels); bytestream_put_le16(&p, avctx->initial_padding); /* Lookahead samples at 48kHz */ bytestream_put_le32(&p, avctx->sample_rate); /* Original sample rate */ - bytestream_put_le16(&p, 0); /* Gain of 0dB is recommended. */ + if (mapping_family == 3) { +#ifdef OPUS_HAVE_OPUS_PROJECTION_H + int ret; + ret = libopus_generic_encoder_ctl(enc, + OPUS_PROJECTION_GET_DEMIXING_MATRIX_GAIN(&gain)); + if (ret != OPUS_OK) { + av_log(avctx, AV_LOG_ERROR, + "Unable to write header, demixing matrix gain not found.\n"); + return; + } +#endif + } + bytestream_put_le16(&p, gain); /* Gain of 0dB is recommended. */ /* Channel mapping */ bytestream_put_byte(&p, mapping_family); - if (mapping_family != 0) { + if (mapping_family == 3) { + int ret; + int32_t size; + size = 2 * channels * (stream_count + coupled_stream_count); + bytestream_put_byte(&p, stream_count); + bytestream_put_byte(&p, coupled_stream_count); + bytestream_put_byte(&p, stream_count); +#ifdef OPUS_HAVE_OPUS_PROJECTION_H + ret = libopus_generic_encoder_ctl(enc, + OPUS_PROJECTION_GET_DEMIXING_MATRIX_SIZE(&size)); + if (ret != OPUS_OK) { + av_log(avctx, AV_LOG_ERROR, + "Unable to write header, demixing matrix size not found.\n"); + return; + } + ret = libopus_generic_encoder_ctl(enc, + OPUS_PROJECTION_GET_DEMIXING_MATRIX(p, size)); + if (ret != OPUS_OK) { + av_log(avctx, AV_LOG_ERROR, + "Unable to write header, demixing matrix not found.\n"); + return; + } + (*(&p)) += size; +#endif + } + else if (mapping_family != 0) { bytestream_put_byte(&p, stream_count); bytestream_put_byte(&p, coupled_stream_count); bytestream_put_buffer(&p, channel_mapping, channels); } } -static int libopus_configure_encoder(AVCodecContext *avctx, OpusMSEncoder *enc, +static int libopus_configure_encoder(AVCodecContext *avctx, + OpusGenericEncoder *enc, LibopusEncOpts *opts) { int ret; @@ -118,48 +278,48 @@ static int libopus_configure_encoder(AVCodecContext *avctx, OpusMSEncoder *enc, return AVERROR(EINVAL); } - ret = opus_multistream_encoder_ctl(enc, OPUS_SET_BITRATE(avctx->bit_rate)); + ret = libopus_generic_encoder_ctl(enc, OPUS_SET_BITRATE(avctx->bit_rate)); if (ret != OPUS_OK) { av_log(avctx, AV_LOG_ERROR, "Failed to set bitrate: %s\n", opus_strerror(ret)); return ret; } - ret = opus_multistream_encoder_ctl(enc, - OPUS_SET_COMPLEXITY(opts->complexity)); + ret = libopus_generic_encoder_ctl(enc, + OPUS_SET_COMPLEXITY(opts->complexity)); if (ret != OPUS_OK) av_log(avctx, AV_LOG_WARNING, "Unable to set complexity: %s\n", opus_strerror(ret)); - ret = opus_multistream_encoder_ctl(enc, OPUS_SET_VBR(!!opts->vbr)); + ret = libopus_generic_encoder_ctl(enc, OPUS_SET_VBR(!!opts->vbr)); if (ret != OPUS_OK) av_log(avctx, AV_LOG_WARNING, "Unable to set VBR: %s\n", opus_strerror(ret)); - ret = opus_multistream_encoder_ctl(enc, - OPUS_SET_VBR_CONSTRAINT(opts->vbr == 2)); + ret = libopus_generic_encoder_ctl(enc, + OPUS_SET_VBR_CONSTRAINT(opts->vbr == 2)); if (ret != OPUS_OK) av_log(avctx, AV_LOG_WARNING, "Unable to set constrained VBR: %s\n", opus_strerror(ret)); - ret = opus_multistream_encoder_ctl(enc, - OPUS_SET_PACKET_LOSS_PERC(opts->packet_loss)); + ret = libopus_generic_encoder_ctl(enc, + OPUS_SET_PACKET_LOSS_PERC(opts->packet_loss)); if (ret != OPUS_OK) av_log(avctx, AV_LOG_WARNING, "Unable to set expected packet loss percentage: %s\n", opus_strerror(ret)); if (avctx->cutoff) { - ret = opus_multistream_encoder_ctl(enc, - OPUS_SET_MAX_BANDWIDTH(opts->max_bandwidth)); + ret = libopus_generic_encoder_ctl(enc, + OPUS_SET_MAX_BANDWIDTH(opts->max_bandwidth)); if (ret != OPUS_OK) av_log(avctx, AV_LOG_WARNING, "Unable to set maximum bandwidth: %s\n", opus_strerror(ret)); } #ifdef OPUS_SET_PHASE_INVERSION_DISABLED_REQUEST - ret = opus_multistream_encoder_ctl(enc, - OPUS_SET_PHASE_INVERSION_DISABLED(!opts->apply_phase_inv)); + ret = libopus_generic_encoder_ctl(enc, + OPUS_SET_PHASE_INVERSION_DISABLED(!opts->apply_phase_inv)); if (ret != OPUS_OK) av_log(avctx, AV_LOG_WARNING, "Unable to set phase inversion: %s\n", @@ -207,6 +367,8 @@ static int libopus_validate_layout_and_get_channel_map( { const uint8_t * channel_map = NULL; int ret; + int order_plus_one; + int nondiegetic_channels; switch (mapping_family) { case -1: @@ -231,6 +393,23 @@ static int libopus_validate_layout_and_get_channel_map( channel_map = ff_vorbis_channel_layout_offsets[avctx->channels - 1]; } break; + case 2: + case 3: + order_plus_one = ff_sqrt(avctx->channels); + nondiegetic_channels = avctx->channels - order_plus_one * order_plus_one; + if (order_plus_one < 1 || order_plus_one > 15 || + (nondiegetic_channels != 0 && nondiegetic_channels != 2)) { + av_log(avctx, AV_LOG_ERROR, + "This channel mapping is only specified for channel counts" + " which can be written as (n + 1)^2 + 2j, where n is a" + " non-negative integar from 0 to 14 and j is either 0 or 1.\n"); + ret = AVERROR_INVALIDDATA; + } else { + ret = 0; + } + + /* Channels do not need to be reordered. */ + break; case 255: ret = libopus_check_max_channels(avctx, 254); break; @@ -248,7 +427,7 @@ static int libopus_validate_layout_and_get_channel_map( static av_cold int libopus_encode_init(AVCodecContext *avctx) { LibopusEncContext *opus = avctx->priv_data; - OpusMSEncoder *enc; + OpusGenericEncoder *enc = &opus->enc; uint8_t libopus_channel_mapping[255]; int ret = OPUS_OK; int av_ret; @@ -335,20 +514,20 @@ static av_cold int libopus_encode_init(AVCodecContext *avctx) opus_vorbis_channel_map[avctx->channels - 1], avctx->channels * sizeof(*libopus_channel_mapping)); - enc = opus_multistream_encoder_create( - avctx->sample_rate, avctx->channels, opus->stream_count, + ret = libopus_generic_encoder_init( + enc, avctx->sample_rate, avctx->channels, opus->stream_count, coupled_stream_count, libavcodec_libopus_channel_map[avctx->channels - 1], - opus->opts.application, &ret); + opus->opts.application); } else { /* Use the newer multistream API. The encoder will set the channel * mapping and coupled stream counts to its internal defaults and will * use surround masking analysis to save bits. */ mapping_family = opus->opts.mapping_family; - enc = opus_multistream_surround_encoder_create( - avctx->sample_rate, avctx->channels, mapping_family, + ret = libopus_generic_encoder_surround_init( + enc, avctx->sample_rate, avctx->channels, mapping_family, &opus->stream_count, &coupled_stream_count, libopus_channel_mapping, - opus->opts.application, &ret); + opus->opts.application); } if (ret != OPUS_OK) { @@ -380,7 +559,9 @@ static av_cold int libopus_encode_init(AVCodecContext *avctx) } /* Header includes channel mapping table if and only if mapping family is NOT 0 */ - header_size = 19 + (mapping_family == 0 ? 0 : 2 + avctx->channels); + header_size = libopus_generic_encoder_get_header_size( + mapping_family, avctx->channels, opus->stream_count, + coupled_stream_count); avctx->extradata = av_malloc(header_size + AV_INPUT_BUFFER_PADDING_SIZE); if (!avctx->extradata) { av_log(avctx, AV_LOG_ERROR, "Failed to allocate extradata.\n"); @@ -397,23 +578,21 @@ static av_cold int libopus_encode_init(AVCodecContext *avctx) goto fail; } - ret = opus_multistream_encoder_ctl(enc, OPUS_GET_LOOKAHEAD(&avctx->initial_padding)); + ret = libopus_generic_encoder_ctl(enc, OPUS_GET_LOOKAHEAD(&avctx->initial_padding)); if (ret != OPUS_OK) av_log(avctx, AV_LOG_WARNING, "Unable to get number of lookahead samples: %s\n", opus_strerror(ret)); libopus_write_header(avctx, opus->stream_count, coupled_stream_count, - mapping_family, libopus_channel_mapping); + mapping_family, libopus_channel_mapping, enc); ff_af_queue_init(avctx, &opus->afq); - opus->enc = enc; - return 0; fail: - opus_multistream_encoder_destroy(enc); + libous_generic_encoder_cleanup(enc); av_freep(&avctx->extradata); return ret; } @@ -470,13 +649,13 @@ static int libopus_encode(AVCodecContext *avctx, AVPacket *avpkt, return ret; if (avctx->sample_fmt == AV_SAMPLE_FMT_FLT) - ret = opus_multistream_encode_float(opus->enc, (float *)audio, - opus->opts.packet_size, - avpkt->data, avpkt->size); + ret = libopus_generic_encode_float(&opus->enc, (float *)audio, + opus->opts.packet_size, + avpkt->data, avpkt->size); else - ret = opus_multistream_encode(opus->enc, (opus_int16 *)audio, - opus->opts.packet_size, - avpkt->data, avpkt->size); + ret = libopus_generic_encode(&opus->enc, (opus_int16 *)audio, + opus->opts.packet_size, + avpkt->data, avpkt->size); if (ret < 0) { av_log(avctx, AV_LOG_ERROR, @@ -517,7 +696,7 @@ static av_cold int libopus_encode_close(AVCodecContext *avctx) { LibopusEncContext *opus = avctx->priv_data; - opus_multistream_encoder_destroy(opus->enc); + libous_generic_encoder_cleanup(&opus->enc); ff_af_queue_close(&opus->afq); diff --git a/libavcodec/opus.c b/libavcodec/opus.c index aa827b604c..f8877233af 100644 --- a/libavcodec/opus.c +++ b/libavcodec/opus.c @@ -373,18 +373,14 @@ av_cold int ff_opus_parse_extradata(AVCodecContext *avctx, layout = ff_vorbis_channel_layouts[channels - 1]; channel_reorder = channel_reorder_vorbis; } else if (map_type == 2) { - int ambisonic_order = ff_sqrt(channels) - 1; - if (channels != ((ambisonic_order + 1) * (ambisonic_order + 1)) && - channels != ((ambisonic_order + 1) * (ambisonic_order + 1) + 2)) { + int order_plus_one = ff_sqrt(channels); + int nondiegetic_channels = channels - order_plus_one * order_plus_one; + if (order_plus_one < 1 || order_plus_one > 15 || + (nondiegetic_channels != 0 && nondiegetic_channels != 2)) { av_log(avctx, AV_LOG_ERROR, - "Channel mapping 2 is only specified for channel counts" - " which can be written as (n + 1)^2 or (n + 1)^2 + 2" - " for nonnegative integer n\n"); - return AVERROR_INVALIDDATA; - } - if (channels > 227) { - av_log(avctx, AV_LOG_ERROR, "Too many channels\n"); - return AVERROR_INVALIDDATA; + "This channel mapping is only specified for channel counts" + " which can be written as (n + 1)^2 + 2j, where n is a" + " non-negative integar from 0 to 14 and j is either 0 or 1.\n"); } layout = 0; } else -- 2.17.0.484.g0c8726318c-goog