From patchwork Tue Dec 5 22:43:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 44934 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9153:b0:181:818d:5e7f with SMTP id x19csp658809pzc; Tue, 5 Dec 2023 14:44:09 -0800 (PST) X-Google-Smtp-Source: AGHT+IHru9dCW7PzeVk3JvgZRMm4ShPnxfD3HwWwPbRmoc52VMBeLh47Dl7EO8tvtkBwRL+06QeE X-Received: by 2002:a05:6402:3595:b0:54c:6fc0:483a with SMTP id y21-20020a056402359500b0054c6fc0483amr39622edc.2.1701816249350; Tue, 05 Dec 2023 14:44:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701816249; cv=none; d=google.com; s=arc-20160816; b=uARSmmI1Dy2JU1JD46sR5QlwFiS3p37tH9Wdn682trYzKERMysieaNG57n97ldFME1 Eg0fz9zo6v8IgofsFsfBRxjExE7lrHwCpkMUCyVTjBJBxhUUXsi9wTVTP1/uvKmXKXm+ cFoqJeePJX7AHE7MyZF1wYYJ+OQRJg3EKmK5npb9fJzFDkSjpckqyZhdwZIfe4JOVGee 8rZUFvimdQpNz4vD9XbXhkAsm/4jCOU5N3uBKQmaHxyNkVhxO39NpCZ4GyVl8uSVB3TP 5RMKDVgYAvz9n6/JhaaNH7cJpnl7afvQzwm99Tkj9R9zIc/orlgahkMuiDAe6dT11d9c RqDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=kxGltdGlDPFdTUPC2FoJC3/pxybbqGu08+bhb/67LI4=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=FKJAV5bk46Lkjn0QWvLbOjliYinbZcEZB0z9r+RUIlEY0G+StpmXM55PTLeAdDXqnv Fe4JEZxExcOm9fpvU+6BhmS/HFXXXOVkTeiIwJM5GBvKb7RdW5x8h0H3RwYa+AuKDRU+ wY0i8utkKo5bUWnTDh+ryx267xwfN5l6Qv6ZDqX70nGU9jA660H2cwi3R+gq6IMg9Z3n NFpFzDREc+0j5Q2Wczwpu90Vkd6X2aAnDDFnt6QMiTZD5FBlKeh30U5K2QhkA5x9apKR cBVjP85FqquBWjMFpQwI2O8SYvrw3o88xgJmMpNiBLuiQNekQMAW6LrsL/gTuteK1kRQ N6Aw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=gnVvtrgm; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 25-20020a508759000000b0054ca4c38763si1391794edv.38.2023.12.05.14.44.07; Tue, 05 Dec 2023 14:44:09 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=gnVvtrgm; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EE21E68CDF6; Wed, 6 Dec 2023 00:43:52 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oo1-f47.google.com (mail-oo1-f47.google.com [209.85.161.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B1C7F68CCED for ; Wed, 6 Dec 2023 00:43:45 +0200 (EET) Received: by mail-oo1-f47.google.com with SMTP id 006d021491bc7-58e1ddc68b2so2818860eaf.2 for ; Tue, 05 Dec 2023 14:43:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701816223; x=1702421023; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=fwpRsb5hWG682ruOsy0ZVmI2G/8LE0na4QCpxaw5yO0=; b=gnVvtrgmPU6AVelyCKDrFCiHsl0Wr1ek2Sasad5Tct4kChUPUkFQEITovWJE3qBzVI OHLtpNtsBr1Yz7Q81LPkSK863ExsujZMQ7MrQP4EIwxXZpWUFEOB4PLONKIorlYmiF76 c5E9qiBz2+xfdSBZGwXiXSmWS5Zt8OOSB8AfBkMtEyxZ29cwwuWdagwMc7+ScicF5HAX AsfmaPB6E8UoDf8dbUH/i+xzh25rGwUOaNTmJZB4WxLQlG252H7db6e8CV6CJYoyJtYU 8TP1NiJ8jYBaYhpma6oUUg3A8U1naV6vBOxO6WCaPn7uiHa8qLFkss23dcATpH06B5km yWAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701816223; x=1702421023; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fwpRsb5hWG682ruOsy0ZVmI2G/8LE0na4QCpxaw5yO0=; b=FZELP6TVB7LXQlnAwrg7wC+lEkOLJW6fr9ozO7blc7QaPsW2ES2kMe8/bzxTlNYdY3 uQOu6IeJwOvTBDnygJU8c9yvXuLwSndGep4/ijxSc5AWVwTAVrwhpW88DlTcWnz+jO27 B6SxGfMLn/yI8lqTtHQtNaT2ts9uIngvoYaeHLOW2izhmR5YkBOdKjWdInFS0xE3DJjm YVIheNkhOMm8BabWalzPqgqqGBU54n0ztX7bl8f4xeyMFNif7x1U5J4mzgmlbVtoFo5i /3dw35Ja9stZPHQh0wlkmJfR8dcIx8IfV9CELsGOlZ0kJQRFChCn5pmTXnyvUREj1yRi Jp5g== X-Gm-Message-State: AOJu0YwS15Z6CLcGQTwhS67+/Y/NYVRmH0arLQkjgBC5IZqUHu3L/cTe DqZhnC7C/vt13/EypewVGcBNxzSKH/0= X-Received: by 2002:a05:6358:9101:b0:170:5522:597b with SMTP id q1-20020a056358910100b001705522597bmr18353rwq.56.1701816222693; Tue, 05 Dec 2023 14:43:42 -0800 (PST) Received: from localhost.localdomain (host197.190-225-105.telecom.net.ar. [190.225.105.197]) by smtp.gmail.com with ESMTPSA id hq25-20020a056a00681900b0064fd4a6b306sm2037688pfb.76.2023.12.05.14.43.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Dec 2023 14:43:42 -0800 (PST) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Tue, 5 Dec 2023 19:43:55 -0300 Message-ID: <20231205224402.14540-2-jamrial@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20231205224402.14540-1-jamrial@gmail.com> References: <20231205224402.14540-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/8] avutil: introduce an Immersive Audio Model and Formats API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 3VL50/r7zGVF Signed-off-by: James Almer --- libavutil/Makefile | 2 + libavutil/iamf.c | 564 ++++++++++++++++++++++++++++++++++++++++++++ libavutil/iamf.h | 573 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 1139 insertions(+) create mode 100644 libavutil/iamf.c create mode 100644 libavutil/iamf.h diff --git a/libavutil/Makefile b/libavutil/Makefile index 4711f8cde8..62cc1a1831 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -51,6 +51,7 @@ HEADERS = adler32.h \ hwcontext_videotoolbox.h \ hwcontext_vdpau.h \ hwcontext_vulkan.h \ + iamf.h \ imgutils.h \ intfloat.h \ intreadwrite.h \ @@ -140,6 +141,7 @@ OBJS = adler32.o \ hdr_dynamic_vivid_metadata.o \ hmac.o \ hwcontext.o \ + iamf.o \ imgutils.o \ integer.o \ intmath.o \ diff --git a/libavutil/iamf.c b/libavutil/iamf.c new file mode 100644 index 0000000000..5f646f2d65 --- /dev/null +++ b/libavutil/iamf.c @@ -0,0 +1,564 @@ +/* + * Immersive Audio Model and Formats helper functions and defines + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include +#include + +#include "avassert.h" +#include "error.h" +#include "iamf.h" +#include "log.h" +#include "mem.h" +#include "opt.h" + +#define IAMF_ADD_FUNC_TEMPLATE(parent_type, parent_name, child_type, child_name, suffix) \ +child_type *av_iamf_ ## parent_name ## _add_ ## child_name(parent_type *parent_name) \ +{ \ + child_type **child_name ## suffix, *child_name; \ + \ + if (parent_name->nb_## child_name ## suffix == UINT_MAX) \ + return NULL; \ + \ + child_name ## suffix = av_realloc_array(parent_name->child_name ## suffix, \ + parent_name->nb_## child_name ## suffix + 1, \ + sizeof(*parent_name->child_name ## suffix)); \ + if (!child_name ## suffix) \ + return NULL; \ + \ + parent_name->child_name ## suffix = child_name ## suffix; \ + \ + child_name = parent_name->child_name ## suffix[parent_name->nb_## child_name ## suffix] \ + = av_mallocz(sizeof(*child_name)); \ + if (!child_name) \ + return NULL; \ + \ + child_name->av_class = &child_name ## _class; \ + av_opt_set_defaults(child_name); \ + parent_name->nb_## child_name ## suffix++; \ + \ + return child_name; \ +} + +#define FLAGS AV_OPT_FLAG_ENCODING_PARAM + +// +// Param Definition +// +#define OFFSET(x) offsetof(AVIAMFMixGain, x) +static const AVOption mix_gain_options[] = { + { "subblock_duration", "set subblock_duration", OFFSET(subblock_duration), AV_OPT_TYPE_INT64, {.i64 = 1 }, 1, UINT_MAX, FLAGS }, + { "animation_type", "set animation_type", OFFSET(animation_type), AV_OPT_TYPE_INT, {.i64 = 0 }, 0, 2, FLAGS }, + { "start_point_value", "set start_point_value", OFFSET(animation_type), AV_OPT_TYPE_RATIONAL, {.dbl = 0 }, -128.0, 128.0, FLAGS }, + { "end_point_value", "set end_point_value", OFFSET(animation_type), AV_OPT_TYPE_RATIONAL, {.dbl = 0 }, -128.0, 128.0, FLAGS }, + { "control_point_value", "set control_point_value", OFFSET(animation_type), AV_OPT_TYPE_RATIONAL, {.dbl = 0 }, -128.0, 128.0, FLAGS }, + { "control_point_relative_time", "set control_point_relative_time", OFFSET(animation_type), AV_OPT_TYPE_RATIONAL, {.dbl = 0 }, 0.0, 1.0, FLAGS }, + { NULL }, +}; + +static const AVClass mix_gain_class = { + .class_name = "AVIAMFSubmixElement", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, + .option = mix_gain_options, +}; + +#undef OFFSET +#define OFFSET(x) offsetof(AVIAMFDemixingInfo, x) +static const AVOption demixing_info_options[] = { + { "subblock_duration", "set subblock_duration", OFFSET(subblock_duration), AV_OPT_TYPE_INT64, {.i64 = 1 }, 1, UINT_MAX, FLAGS }, + { "dmixp_mode", "set dmixp_mode", OFFSET(dmixp_mode), AV_OPT_TYPE_INT, {.i64 = 0 }, 0, 6, FLAGS }, + { NULL }, +}; + +static const AVClass demixing_info_class = { + .class_name = "AVIAMFDemixingInfo", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, + .option = demixing_info_options, +}; + +#undef OFFSET +#define OFFSET(x) offsetof(AVIAMFReconGain, x) +static const AVOption recon_gain_options[] = { + { "subblock_duration", "set subblock_duration", OFFSET(subblock_duration), AV_OPT_TYPE_INT64, {.i64 = 1 }, 1, UINT_MAX, FLAGS }, + { NULL }, +}; + +static const AVClass recon_gain_class = { + .class_name = "AVIAMFReconGain", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, + .option = recon_gain_options, +}; + +#undef OFFSET +#define OFFSET(x) offsetof(AVIAMFParamDefinition, x) +static const AVOption param_definition_options[] = { + { "parameter_id", "set parameter_id", OFFSET(parameter_id), AV_OPT_TYPE_INT64, {.i64 = 0 }, 0, UINT_MAX, FLAGS }, + { "parameter_rate", "set parameter_rate", OFFSET(parameter_rate), AV_OPT_TYPE_INT64, {.i64 = 0 }, 0, UINT_MAX, FLAGS }, + { "param_definition_mode", "set param_definition_mode", OFFSET(param_definition_mode), AV_OPT_TYPE_INT, {.i64 = 1 }, 0, 1, FLAGS }, + { "duration", "set duration", OFFSET(duration), AV_OPT_TYPE_INT64, {.i64 = 0 }, 0, UINT_MAX, FLAGS }, + { "constant_subblock_duration", "set constant_subblock_duration", OFFSET(constant_subblock_duration), AV_OPT_TYPE_INT64, {.i64 = 0 }, 0, UINT_MAX, FLAGS }, + { NULL }, +}; + +static const AVClass *param_definition_child_iterate(void **opaque) +{ + uintptr_t i = (uintptr_t)*opaque; + const AVClass *ret = NULL; + + switch(i) { + case AV_IAMF_PARAMETER_DEFINITION_MIX_GAIN: + ret = &mix_gain_class; + break; + case AV_IAMF_PARAMETER_DEFINITION_DEMIXING: + ret = &demixing_info_class; + break; + case AV_IAMF_PARAMETER_DEFINITION_RECON_GAIN: + ret = &recon_gain_class; + break; + default: + break; + } + + if (ret) + *opaque = (void*)(i + 1); + return ret; +} + +static const AVClass param_definition_class = { + .class_name = "AVIAMFParamDefinition", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, + .option = param_definition_options, + .child_class_iterate = param_definition_child_iterate, +}; + +const AVClass *av_iamf_param_definition_get_class(void) +{ + return ¶m_definition_class; +} + +AVIAMFParamDefinition *av_iamf_param_definition_alloc(enum AVIAMFParamDefinitionType type, + unsigned int nb_subblocks, size_t *out_size) +{ + + struct MixGainStruct { + AVIAMFParamDefinition p; + AVIAMFMixGain m; + }; + struct DemixStruct { + AVIAMFParamDefinition p; + AVIAMFDemixingInfo d; + }; + struct ReconGainStruct { + AVIAMFParamDefinition p; + AVIAMFReconGain r; + }; + size_t subblocks_offset, subblock_size; + size_t size; + AVIAMFParamDefinition *par; + + switch (type) { + case AV_IAMF_PARAMETER_DEFINITION_MIX_GAIN: + subblocks_offset = offsetof(struct MixGainStruct, m); + subblock_size = sizeof(AVIAMFMixGain); + break; + case AV_IAMF_PARAMETER_DEFINITION_DEMIXING: + subblocks_offset = offsetof(struct DemixStruct, d); + subblock_size = sizeof(AVIAMFDemixingInfo); + break; + case AV_IAMF_PARAMETER_DEFINITION_RECON_GAIN: + subblocks_offset = offsetof(struct ReconGainStruct, r); + subblock_size = sizeof(AVIAMFReconGain); + break; + default: + return NULL; + } + + size = subblocks_offset; + if (nb_subblocks > (SIZE_MAX - size) / subblock_size) + return NULL; + size += subblock_size * nb_subblocks; + + par = av_mallocz(size); + if (!par) + return NULL; + + par->av_class = ¶m_definition_class; + av_opt_set_defaults(par); + + par->param_definition_type = type; + par->nb_subblocks = nb_subblocks; + par->subblock_size = subblock_size; + par->subblocks_offset = subblocks_offset; + + for (int i = 0; i < nb_subblocks; i++) { + void *subblock = av_iamf_param_definition_get_subblock(par, i); + + switch (type) { + case AV_IAMF_PARAMETER_DEFINITION_MIX_GAIN: + ((AVIAMFMixGain *)subblock)->av_class = &mix_gain_class; + break; + case AV_IAMF_PARAMETER_DEFINITION_DEMIXING: + ((AVIAMFDemixingInfo *)subblock)->av_class = &demixing_info_class; + break; + case AV_IAMF_PARAMETER_DEFINITION_RECON_GAIN: + ((AVIAMFReconGain *)subblock)->av_class = &recon_gain_class; + break; + default: + av_assert0(0); + } + + av_opt_set_defaults(subblock); + } + + if (out_size) + *out_size = size; + + return par; +} + +// +// Audio Element +// +#undef OFFSET +#define OFFSET(x) offsetof(AVIAMFLayer, x) +static const AVOption layer_options[] = { + { "ch_layout", "set ch_layout", OFFSET(ch_layout), AV_OPT_TYPE_CHLAYOUT, {.str = NULL }, 0, 0, FLAGS }, + { "flags", "set flags", OFFSET(flags), AV_OPT_TYPE_FLAGS, + {.i64 = 0 }, 0, AV_IAMF_LAYER_FLAG_RECON_GAIN, FLAGS, "flags" }, + {"recon_gain", "Recon gain is present", 0, AV_OPT_TYPE_CONST, + {.i64 = AV_IAMF_LAYER_FLAG_RECON_GAIN }, INT_MIN, INT_MAX, FLAGS, "flags"}, + { "output_gain_flags", "set output_gain_flags", OFFSET(output_gain_flags), AV_OPT_TYPE_FLAGS, + {.i64 = 0 }, 0, (1 << 6) - 1, FLAGS, "output_gain_flags" }, + {"FL", "Left channel", 0, AV_OPT_TYPE_CONST, + {.i64 = 1 << 5 }, INT_MIN, INT_MAX, FLAGS, "output_gain_flags"}, + {"FR", "Right channel", 0, AV_OPT_TYPE_CONST, + {.i64 = 1 << 4 }, INT_MIN, INT_MAX, FLAGS, "output_gain_flags"}, + {"BL", "Left surround channel", 0, AV_OPT_TYPE_CONST, + {.i64 = 1 << 3 }, INT_MIN, INT_MAX, FLAGS, "output_gain_flags"}, + {"BR", "Right surround channel", 0, AV_OPT_TYPE_CONST, + {.i64 = 1 << 2 }, INT_MIN, INT_MAX, FLAGS, "output_gain_flags"}, + {"TFL", "Left top front channel", 0, AV_OPT_TYPE_CONST, + {.i64 = 1 << 1 }, INT_MIN, INT_MAX, FLAGS, "output_gain_flags"}, + {"TFR", "Right top front channel", 0, AV_OPT_TYPE_CONST, + {.i64 = 1 << 0 }, INT_MIN, INT_MAX, FLAGS, "output_gain_flags"}, + { "output_gain", "set output_gain", OFFSET(output_gain), AV_OPT_TYPE_RATIONAL, { .dbl = 0 }, -128.0, 128.0, FLAGS }, + { "ambisonics_mode", "set ambisonics_mode", OFFSET(ambisonics_mode), AV_OPT_TYPE_INT, + { .i64 = AV_IAMF_AMBISONICS_MODE_MONO }, + AV_IAMF_AMBISONICS_MODE_MONO, AV_IAMF_AMBISONICS_MODE_PROJECTION, FLAGS, "ambisonics_mode" }, + { "mono", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = AV_IAMF_AMBISONICS_MODE_MONO }, .unit = "ambisonics_mode" }, + { "projection", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = AV_IAMF_AMBISONICS_MODE_PROJECTION }, .unit = "ambisonics_mode" }, + { NULL }, +}; + +static const AVClass layer_class = { + .class_name = "AVIAMFLayer", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, + .option = layer_options, +}; + +#undef OFFSET +#define OFFSET(x) offsetof(AVIAMFAudioElement, x) +static const AVOption audio_element_options[] = { + { "audio_element_type", "set audio_element_type", OFFSET(audio_element_type), AV_OPT_TYPE_INT, + {.i64 = AV_IAMF_AUDIO_ELEMENT_TYPE_CHANNEL }, + AV_IAMF_AUDIO_ELEMENT_TYPE_CHANNEL, AV_IAMF_AUDIO_ELEMENT_TYPE_SCENE, FLAGS, "audio_element_type" }, + { "channel", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = AV_IAMF_AUDIO_ELEMENT_TYPE_CHANNEL }, .unit = "audio_element_type" }, + { "scene", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = AV_IAMF_AUDIO_ELEMENT_TYPE_SCENE }, .unit = "audio_element_type" }, + { "default_w", "set default_w", OFFSET(default_w), AV_OPT_TYPE_INT, {.i64 = 0 }, 0, 10, FLAGS }, + { NULL }, +}; + +static const AVClass *audio_element_child_iterate(void **opaque) +{ + uintptr_t i = (uintptr_t)*opaque; + const AVClass *ret = NULL; + + if (i) + ret = &layer_class; + + if (ret) + *opaque = (void*)(i + 1); + return ret; +} + +static const AVClass audio_element_class = { + .class_name = "AVIAMFAudioElement", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, + .option = audio_element_options, + .child_class_iterate = audio_element_child_iterate, +}; + +const AVClass *av_iamf_audio_element_get_class(void) +{ + return &audio_element_class; +} + +AVIAMFAudioElement *av_iamf_audio_element_alloc(void) +{ + AVIAMFAudioElement *audio_element = av_mallocz(sizeof(*audio_element)); + + if (audio_element) { + audio_element->av_class = &audio_element_class; + av_opt_set_defaults(audio_element); + } + + return audio_element; +} + +IAMF_ADD_FUNC_TEMPLATE(AVIAMFAudioElement, audio_element, AVIAMFLayer, layer, s) + +void av_iamf_audio_element_free(AVIAMFAudioElement **paudio_element) +{ + AVIAMFAudioElement *audio_element = *paudio_element; + + if (!audio_element) + return; + + for (int i = 0; i < audio_element->nb_layers; i++) { + AVIAMFLayer *layer = audio_element->layers[i]; + av_opt_free(layer); + av_free(layer->demixing_matrix); + av_free(layer); + } + av_free(audio_element->layers); + + av_free(audio_element->demixing_info); + av_free(audio_element->recon_gain_info); + av_freep(paudio_element); +} + +// +// Mix Presentation +// +#undef OFFSET +#define OFFSET(x) offsetof(AVIAMFSubmixElement, x) +static const AVOption submix_element_options[] = { + { "headphones_rendering_mode", "Headphones rendering mode", OFFSET(headphones_rendering_mode), AV_OPT_TYPE_INT, + { .i64 = AV_IAMF_HEADPHONES_MODE_STEREO }, + AV_IAMF_HEADPHONES_MODE_STEREO, AV_IAMF_HEADPHONES_MODE_BINAURAL, FLAGS, "headphones_rendering_mode" }, + { "stereo", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = AV_IAMF_HEADPHONES_MODE_STEREO }, .unit = "headphones_rendering_mode" }, + { "binaural", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = AV_IAMF_HEADPHONES_MODE_BINAURAL }, .unit = "headphones_rendering_mode" }, + { "default_mix_gain", "Default mix gain", OFFSET(default_mix_gain), AV_OPT_TYPE_RATIONAL, { .dbl = 0 }, -128.0, 128.0, FLAGS }, + { "annotations", "Annotations", OFFSET(annotations), AV_OPT_TYPE_DICT, { .str = NULL }, 0, 0, FLAGS }, + { NULL }, +}; + +static void *submix_element_child_next(void *obj, void *prev) +{ + AVIAMFSubmixElement *submix_element = obj; + if (!prev) + return submix_element->element_mix_config; + + return NULL; +} + +static const AVClass *submix_element_child_iterate(void **opaque) +{ + uintptr_t i = (uintptr_t)*opaque; + const AVClass *ret = NULL; + + if (i) + ret = ¶m_definition_class; + + if (ret) + *opaque = (void*)(i + 1); + return ret; +} + +static const AVClass element_class = { + .class_name = "AVIAMFSubmixElement", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, + .option = submix_element_options, + .child_next = submix_element_child_next, + .child_class_iterate = submix_element_child_iterate, +}; + +IAMF_ADD_FUNC_TEMPLATE(AVIAMFSubmix, submix, AVIAMFSubmixElement, element, s) + +#undef OFFSET +#define OFFSET(x) offsetof(AVIAMFSubmixLayout, x) +static const AVOption submix_layout_options[] = { + { "layout_type", "Layout type", OFFSET(layout_type), AV_OPT_TYPE_INT, + { .i64 = AV_IAMF_SUBMIX_LAYOUT_TYPE_LOUDSPEAKERS }, + AV_IAMF_SUBMIX_LAYOUT_TYPE_LOUDSPEAKERS, AV_IAMF_SUBMIX_LAYOUT_TYPE_BINAURAL, FLAGS, "layout_type" }, + { "loudspeakers", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = AV_IAMF_SUBMIX_LAYOUT_TYPE_LOUDSPEAKERS }, .unit = "layout_type" }, + { "binaural", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = AV_IAMF_SUBMIX_LAYOUT_TYPE_BINAURAL }, .unit = "layout_type" }, + { "sound_system", "Sound System", OFFSET(sound_system), AV_OPT_TYPE_CHLAYOUT, { .str = NULL }, 0, 0, FLAGS }, + { "integrated_loudness", "Integrated loudness", OFFSET(integrated_loudness), AV_OPT_TYPE_RATIONAL, { .dbl = 0 }, -128.0, 128.0, FLAGS }, + { "digital_peak", "Digital peak", OFFSET(digital_peak), AV_OPT_TYPE_RATIONAL, { .dbl = 0 }, -128.0, 128.0, FLAGS }, + { "true_peak", "True peak", OFFSET(true_peak), AV_OPT_TYPE_RATIONAL, { .dbl = 0 }, -128.0, 128.0, FLAGS }, + { "dialog_anchored_loudness", "Anchored loudness (Dialog)", OFFSET(dialogue_anchored_loudness), AV_OPT_TYPE_RATIONAL, { .dbl = 0 }, -128.0, 128.0, FLAGS }, + { "album_anchored_loudness", "Anchored loudness (Album)", OFFSET(album_anchored_loudness), AV_OPT_TYPE_RATIONAL, { .dbl = 0 }, -128.0, 128.0, FLAGS }, + { NULL }, +}; + +static const AVClass layout_class = { + .class_name = "AVIAMFSubmixLayout", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, + .option = submix_layout_options, +}; + +IAMF_ADD_FUNC_TEMPLATE(AVIAMFSubmix, submix, AVIAMFSubmixLayout, layout, s) + +#undef OFFSET +#define OFFSET(x) offsetof(AVIAMFSubmix, x) +static const AVOption submix_presentation_options[] = { + { "default_mix_gain", "Default mix gain", OFFSET(default_mix_gain), AV_OPT_TYPE_RATIONAL, { .dbl = 0 }, -128.0, 128.0, FLAGS }, + { NULL }, +}; + +static void *submix_presentation_child_next(void *obj, void *prev) +{ + AVIAMFSubmix *sub_mix = obj; + if (!prev) + return sub_mix->output_mix_config; + + return NULL; +} + +static const AVClass *submix_presentation_child_iterate(void **opaque) +{ + uintptr_t i = (uintptr_t)*opaque; + const AVClass *ret = NULL; + + switch(i) { + case 0: + ret = &element_class; + break; + case 1: + ret = &layout_class; + break; + case 2: + ret = ¶m_definition_class; + break; + default: + break; + } + + if (ret) + *opaque = (void*)(i + 1); + return ret; +} + +static const AVClass submix_class = { + .class_name = "AVIAMFSubmix", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, + .option = submix_presentation_options, + .child_next = submix_presentation_child_next, + .child_class_iterate = submix_presentation_child_iterate, +}; + +#undef OFFSET +#define OFFSET(x) offsetof(AVIAMFMixPresentation, x) +static const AVOption mix_presentation_options[] = { + { "annotations", "set annotations", OFFSET(annotations), AV_OPT_TYPE_DICT, {.str = NULL }, 0, 0, FLAGS }, + { NULL }, +}; + +#undef OFFSET +#undef FLAGS + +static const AVClass *mix_presentation_child_iterate(void **opaque) +{ + uintptr_t i = (uintptr_t)*opaque; + const AVClass *ret = NULL; + + if (i) + ret = &submix_class; + + if (ret) + *opaque = (void*)(i + 1); + return ret; +} + +static const AVClass mix_presentation_class = { + .class_name = "AVIAMFMixPresentation", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, + .option = mix_presentation_options, + .child_class_iterate = mix_presentation_child_iterate, +}; + +const AVClass *av_iamf_mix_presentation_get_class(void) +{ + return &mix_presentation_class; +} + +AVIAMFMixPresentation *av_iamf_mix_presentation_alloc(void) +{ + AVIAMFMixPresentation *mix_presentation = av_mallocz(sizeof(*mix_presentation)); + + if (mix_presentation) { + mix_presentation->av_class = &mix_presentation_class; + av_opt_set_defaults(mix_presentation); + } + + return mix_presentation; +} + +IAMF_ADD_FUNC_TEMPLATE(AVIAMFMixPresentation, mix_presentation, AVIAMFSubmix, submix, es) + +void av_iamf_mix_presentation_free(AVIAMFMixPresentation **pmix_presentation) +{ + AVIAMFMixPresentation *mix_presentation = *pmix_presentation; + + if (!mix_presentation) + return; + + for (int i = 0; i < mix_presentation->nb_submixes; i++) { + AVIAMFSubmix *sub_mix = mix_presentation->submixes[i]; + for (int j = 0; j < sub_mix->nb_elements; j++) { + AVIAMFSubmixElement *submix_element = sub_mix->elements[j]; + av_opt_free(submix_element); + av_free(submix_element->element_mix_config); + av_free(submix_element); + } + av_free(sub_mix->elements); + for (int j = 0; j < sub_mix->nb_layouts; j++) { + AVIAMFSubmixLayout *submix_layout = sub_mix->layouts[j]; + av_opt_free(submix_layout); + av_free(submix_layout); + } + av_free(sub_mix->layouts); + av_free(sub_mix->output_mix_config); + av_free(sub_mix); + } + av_opt_free(mix_presentation); + av_free(mix_presentation->submixes); + + av_freep(pmix_presentation); +} diff --git a/libavutil/iamf.h b/libavutil/iamf.h new file mode 100644 index 0000000000..bc0363153d --- /dev/null +++ b/libavutil/iamf.h @@ -0,0 +1,573 @@ +/* + * Immersive Audio Model and Formats helper functions and defines + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_IAMF_H +#define AVUTIL_IAMF_H + +/** + * @file + * Immersive Audio Model and Formats API header + * @see Immersive Audio Model and Formats + */ + +#include +#include + +#include "attributes.h" +#include "avassert.h" +#include "channel_layout.h" +#include "dict.h" +#include "rational.h" + +/** + * @defgroup lavf_iamf_params Parameter Definition + * @{ + * Parameters as defined in section 3.6.1 and 3.8 of IAMF. + * @} + * @defgroup lavf_iamf_audio Audio Element + * @{ + * Audio Elements as defined in section 3.6 of IAMF. + * @} + * @defgroup lavf_iamf_mix Mix Presentation + * @{ + * Mix Presentations as defined in section 3.7 of IAMF. + * @} + * + * @} + * @addtogroup lavf_iamf_params + * @{ + */ +enum AVIAMFAnimationType { + AV_IAMF_ANIMATION_TYPE_STEP, + AV_IAMF_ANIMATION_TYPE_LINEAR, + AV_IAMF_ANIMATION_TYPE_BEZIER, +}; + +/** + * Mix Gain Parameter Data as defined in section 3.8.1 of IAMF. + * + * Subblocks in AVIAMFParamDefinition use this struct when the value or + * @ref AVIAMFParamDefinition.param_definition_type param_definition_type is + * AV_IAMF_PARAMETER_DEFINITION_MIX_GAIN. + */ +typedef struct AVIAMFMixGain { + const AVClass *av_class; + + unsigned int subblock_duration; + /** + * The type of animation applied to the parameter values. + */ + enum AVIAMFAnimationType animation_type; + /** + * Parameter value that is applied at the start of the subblock. + * Applies to all defined Animation Types. + * + * Valid range of values is -128.0 to 128.0 + */ + AVRational start_point_value; + /** + * Parameter value that is applied at the end of the subblock. + * Applies only to AV_IAMF_ANIMATION_TYPE_LINEAR and + * AV_IAMF_ANIMATION_TYPE_BEZIER Animation Types. + * + * Valid range of values is -128.0 to 128.0 + */ + AVRational end_point_value; + /** + * Parameter value of the middle control point of a quadratic Bezier + * curve, i.e., its y-axis value. + * Applies only to AV_IAMF_ANIMATION_TYPE_BEZIER Animation Type. + * + * Valid range of values is -128.0 to 128.0 + */ + AVRational control_point_value; + /** + * Parameter value of the time of the middle control point of a + * quadratic Bezier curve, i.e., its x-axis value. + * Applies only to AV_IAMF_ANIMATION_TYPE_BEZIER Animation Type. + * + * Valid range of values is 0.0 to 1.0 + */ + AVRational control_point_relative_time; +} AVIAMFMixGain; + +/** + * Demixing Info Parameter Data as defined in section 3.8.2 of IAMF.. + * + * Subblocks in AVIAMFParamDefinition use this struct when the value or + * @ref AVIAMFParamDefinition.param_definition_type param_definition_type is + * AV_IAMF_PARAMETER_DEFINITION_DEMIXING. + */ +typedef struct AVIAMFDemixingInfo { + const AVClass *av_class; + + unsigned int subblock_duration; + /** + * Pre-defined combination of demixing parameters. + */ + unsigned int dmixp_mode; +} AVIAMFDemixingInfo; + +/** + * Recon Gain Info Parameter Data as defined in section 3.8.3 of IAMF. + * + * Subblocks in AVIAMFParamDefinition use this struct when the value or + * @ref AVIAMFParamDefinition.param_definition_type param_definition_type is + * AV_IAMF_PARAMETER_DEFINITION_RECON_GAIN. + */ +typedef struct AVIAMFReconGain { + const AVClass *av_class; + + unsigned int subblock_duration; + + /** + * Array of gain values to be applied to each channel for each layer + * defined in the Audio Element referencing the parent Parameter Definition. + * Values for layers where the AV_IAMF_LAYER_FLAG_RECON_GAIN flag is not set + * are undefined. + * + * Channel order is: FL, C, FR, SL, SR, TFL, TFR, BL, BR, TBL, TBR, LFE + */ + uint8_t recon_gain[6][12]; +} AVIAMFReconGain; + +enum AVIAMFParamDefinitionType { + AV_IAMF_PARAMETER_DEFINITION_MIX_GAIN, + AV_IAMF_PARAMETER_DEFINITION_DEMIXING, + AV_IAMF_PARAMETER_DEFINITION_RECON_GAIN, +}; + +/** + * Parameters as defined in section 3.6.1 of IAMF. + */ +typedef struct AVIAMFParamDefinition { + const AVClass *av_class; + + size_t subblocks_offset; + size_t subblock_size; + + enum AVIAMFParamDefinitionType param_definition_type; + unsigned int nb_subblocks; + + unsigned int parameter_id; + unsigned int parameter_rate; + unsigned int param_definition_mode; + unsigned int duration; + unsigned int constant_subblock_duration; +} AVIAMFParamDefinition; + +const AVClass *av_iamf_param_definition_get_class(void); + +/** + * Allocates memory for AVIAMFParamDefinition, plus an array of {@code nb_subblocks} + * amount of subblocks of the given type and initializes the variables. Can be + * freed with a normal av_free() call. + * + * @param size if non-NULL, the size in bytes of the resulting data array is written here. + */ +AVIAMFParamDefinition *av_iamf_param_definition_alloc(enum AVIAMFParamDefinitionType param_definition_type, + unsigned int nb_subblocks, size_t *size); + +/** + * Get the subblock at the specified {@code idx}. Must be between 0 and nb_subblocks - 1. + * + * The @ref AVIAMFParamDefinition.param_definition_type "param definition type" defines + * the struct type of the returned pointer. + */ +static av_always_inline void* +av_iamf_param_definition_get_subblock(const AVIAMFParamDefinition *par, unsigned int idx) +{ + av_assert0(idx < par->nb_subblocks); + return (void *)((uint8_t *)par + par->subblocks_offset + idx * par->subblock_size); +} + +/** + * @} + * @addtogroup lavf_iamf_audio + * @{ + */ + +enum AVIAMFAmbisonicsMode { + AV_IAMF_AMBISONICS_MODE_MONO, + AV_IAMF_AMBISONICS_MODE_PROJECTION, +}; + +/** + * Recon gain information for the layer is present in AVIAMFReconGain + */ +#define AV_IAMF_LAYER_FLAG_RECON_GAIN (1 << 0) + +/** + * A layer defining a Channel Layout in the Audio Element. + * + * When @ref AVIAMFAudioElement.audio_element_type "the parent's Audio Element type" + * is AV_IAMF_AUDIO_ELEMENT_TYPE_CHANNEL, this corresponds to an Scalable Channel + * Layout layer as defined in section 3.6.2 of IAMF. + * For AV_IAMF_AUDIO_ELEMENT_TYPE_SCENE, it is an Ambisonics channel + * layout as defined in section 3.6.3 of IAMF. + */ +typedef struct AVIAMFLayer { + const AVClass *av_class; + + AVChannelLayout ch_layout; + + /** + * A bitmask which may contain a combination of AV_IAMF_LAYER_FLAG_* flags. + */ + unsigned int flags; + /** + * Output gain channel flags as defined in section 3.6.2 of IAMF. + * + * This field is defined only if @ref AVIAMFAudioElement.audio_element_type + * "the parent's Audio Element type" is AV_IAMF_AUDIO_ELEMENT_TYPE_CHANNEL, + * must be 0 otherwise. + */ + unsigned int output_gain_flags; + /** + * Output gain as defined in section 3.6.2 of IAMF. + * + * Must be 0 if @ref output_gain_flags is 0. + */ + AVRational output_gain; + /** + * Ambisonics mode as defined in section 3.6.3 of IAMF. + * + * This field is defined only if @ref AVIAMFAudioElement.audio_element_type + * "the parent's Audio Element type" is AV_IAMF_AUDIO_ELEMENT_TYPE_SCENE. + * + * If AV_IAMF_AMBISONICS_MODE_MONO, channel_mapping is defined implicitly + * (Ambisonic Order) or explicitly (Custom Order with ambi channels) in + * @ref ch_layout. + * If AV_IAMF_AMBISONICS_MODE_PROJECTION, @ref demixing_matrix must be set. + */ + enum AVIAMFAmbisonicsMode ambisonics_mode; + + /** + * Demixing matrix as defined in section 3.6.3 of IAMF. + * + * May be set only if @ref ambisonics_mode == AV_IAMF_AMBISONICS_MODE_PROJECTION, + * must be NULL otherwise. + */ + AVRational *demixing_matrix; +} AVIAMFLayer; + + +enum AVIAMFAudioElementType { + AV_IAMF_AUDIO_ELEMENT_TYPE_CHANNEL, + AV_IAMF_AUDIO_ELEMENT_TYPE_SCENE, +}; + +typedef struct AVIAMFAudioElement { + const AVClass *av_class; + + AVIAMFLayer **layers; + /** + * Number of layers, or channel groups, in the Audio Element. + * For @ref audio_element_type AV_IAMF_AUDIO_ELEMENT_TYPE_SCENE, there + * may be exactly 1. + * + * Set by av_iamf_audio_element_add_layer(), must not be + * modified by any other code. + */ + unsigned int nb_layers; + + /** + * Demixing information used to reconstruct a scalable channel audio + * representation. + * The @ref AVIAMFParamDefinition.param_definition_type "type" must be + * AV_IAMF_PARAMETER_DEFINITION_DEMIXING. + */ + AVIAMFParamDefinition *demixing_info; + /** + * Recon gain information used to reconstruct a scalable channel audio + * representation. + * The @ref AVIAMFParamDefinition.param_definition_type "type" must be + * AV_IAMF_PARAMETER_DEFINITION_RECON_GAIN. + */ + AVIAMFParamDefinition *recon_gain_info; + + /** + * Audio element type as defined in section 3.6 of IAMF. + */ + enum AVIAMFAudioElementType audio_element_type; + + /** + * Default weight value as defined in section 3.6 of IAMF. + */ + unsigned int default_w; +} AVIAMFAudioElement; + +const AVClass *av_iamf_audio_element_get_class(void); + +/** + * Allocates a AVIAMFAudioElement, and initializes its fields with default values. + * No layers are allocated. Must be freed with av_iamf_audio_element_free(). + * + * @see av_iamf_audio_element_add_layer() + */ +AVIAMFAudioElement *av_iamf_audio_element_alloc(void); + +/** + * Allocate a layer and add it to a given AVIAMFAudioElement. + * It is freed by av_iamf_audio_element_free() alongside the rest of the parent + * AVIAMFAudioElement. + * + * @return a pointer to the allocated layer. + */ +AVIAMFLayer *av_iamf_audio_element_add_layer(AVIAMFAudioElement *audio_element); + +void av_iamf_audio_element_free(AVIAMFAudioElement **audio_element); + +/** + * @} + * @addtogroup lavf_iamf_mix + * @{ + */ + +enum AVIAMFHeadphonesMode { + /** + * The referenced Audio Element shall be rendered to stereo loudspeakers. + */ + AV_IAMF_HEADPHONES_MODE_STEREO, + /** + * The referenced Audio Element shall be rendered with a binaural renderer. + */ + AV_IAMF_HEADPHONES_MODE_BINAURAL, +}; + +typedef struct AVIAMFSubmixElement { + const AVClass *av_class; + + /** + * The id of the Audio Element this submix element references. + */ + unsigned int audio_element_id; + + /** + * Information required required for applying any processing to the + * referenced and rendered Audio Element before being summed with other + * processed Audio Elements. + * The @ref AVIAMFParamDefinition.param_definition_type "type" must be + * AV_IAMF_PARAMETER_DEFINITION_MIX_GAIN. + */ + AVIAMFParamDefinition *element_mix_config; + + /** + * Default mix gain value to apply when there are no AVIAMFParamDefinition + * with @ref element_mix_config "element_mix_config's" + * @ref AVIAMFParamDefinition.parameter_id "parameter_id" available for a + * given audio frame. + */ + AVRational default_mix_gain; + + /** + * A value that indicates whether the referenced channel-based Audio Element + * shall be rendered to stereo loudspeakers or spatialized with a binaural + * renderer when played back on headphones. + * If the Audio Element is not of @ref AVIAMFAudioElement.audio_element_type + * "type" AV_IAMF_AUDIO_ELEMENT_TYPE_CHANNEL, then this field is undefined. + */ + enum AVIAMFHeadphonesMode headphones_rendering_mode; + + /** + * A dictionary of strings describing the submix in different languages. + * Must have the same amount of entries as + * @ref AVIAMFMixPresentation.annotations "the mix's annotations", stored + * in the same order, and with the same key strings. + * + * @ref AVDictionaryEntry.key "key" is a string conforming to BCP-47 that + * specifies the language for the string stored in + * @ref AVDictionaryEntry.value "value". + */ + AVDictionary *annotations; +} AVIAMFSubmixElement; + +enum AVIAMFSubmixLayoutType { + /** + * The layout follows the loudspeaker sound system convention of ITU-2051-3. + */ + AV_IAMF_SUBMIX_LAYOUT_TYPE_LOUDSPEAKERS = 2, + /** + * The layout is binaural. + */ + AV_IAMF_SUBMIX_LAYOUT_TYPE_BINAURAL = 3, +}; + +typedef struct AVIAMFSubmixLayout { + const AVClass *av_class; + + enum AVIAMFSubmixLayoutType layout_type; + + /** + * Channel layout matching one of Sound Systems A to J of ITU-2051-3, plus + * 7.1.2ch and 3.1.2ch + * If layout_type is not AV_IAMF_SUBMIX_LAYOUT_TYPE_LOUDSPEAKERS, this field + * is undefined. + */ + AVChannelLayout sound_system; + /** + * The program integrated loudness information, as defined in + * ITU-1770-4. + */ + AVRational integrated_loudness; + /** + * The digital (sampled) peak value of the audio signal, as defined + * in ITU-1770-4. + */ + AVRational digital_peak; + /** + * The true peak of the audio signal, as defined in ITU-1770-4. + */ + AVRational true_peak; + /** + * The Dialogue loudness information, as defined in ITU-1770-4. + */ + AVRational dialogue_anchored_loudness; + /** + * The Album loudness information, as defined in ITU-1770-4. + */ + AVRational album_anchored_loudness; +} AVIAMFSubmixLayout; + +typedef struct AVIAMFSubmix { + const AVClass *av_class; + + /** + * Array of submix elements. + * + * Set by av_iamf_submix_add_element(), must not be modified by any + * other code. + */ + AVIAMFSubmixElement **elements; + /** + * Number of elements in the submix. + * + * Set by av_iamf_submix_add_element(), must not be modified by any + * other code. + */ + unsigned int nb_elements; + + /** + * Array of submix layouts. + * + * Set by av_iamf_submix_add_layout(), must not be modified by any + * other code. + */ + AVIAMFSubmixLayout **layouts; + /** + * Number of layouts in the submix. + * + * Set by av_iamf_submix_add_layout(), must not be modified by any + * other code. + */ + unsigned int nb_layouts; + + /** + * Information required for post-processing the mixed audio signal to + * generate the audio signal for playback. + * The @ref AVIAMFParamDefinition.param_definition_type "type" must be + * AV_IAMF_PARAMETER_DEFINITION_MIX_GAIN. + */ + AVIAMFParamDefinition *output_mix_config; + + /** + * Default mix gain value to apply when there are no AVIAMFParamDefinition + * with @ref output_mix_config "output_mix_config's" + * @ref AVIAMFParamDefinition.parameter_id "parameter_id" available for a + * given audio frame. + */ + AVRational default_mix_gain; +} AVIAMFSubmix; + +typedef struct AVIAMFMixPresentation { + const AVClass *av_class; + + /** + * Array of submixes. + * + * Set by av_iamf_mix_presentation_add_submix(), must not be modified + * by any other code. + */ + AVIAMFSubmix **submixes; + /** + * Number of submixes in the presentation. + * + * Set by av_iamf_mix_presentation_add_submix(), must not be modified + * by any other code. + */ + unsigned int nb_submixes; + + /** + * A dictionary of strings describing the mix in different languages. + * Must have the same amount of entries as every + * @ref AVIAMFSubmixElement.annotations "Submix element annotations", + * stored in the same order, and with the same key strings. + * + * @ref AVDictionaryEntry.key "key" is a string conforming to BCP-47 + * that specifies the language for the string stored in + * @ref AVDictionaryEntry.value "value". + */ + AVDictionary *annotations; +} AVIAMFMixPresentation; + +const AVClass *av_iamf_mix_presentation_get_class(void); + +/** + * Allocates a AVIAMFMixPresentation, and initializes its fields with default + * values. No submixes are allocated. + * Must be freed with av_iamf_mix_presentation_free(). + * + * @see av_iamf_mix_presentation_add_submix() + */ +AVIAMFMixPresentation *av_iamf_mix_presentation_alloc(void); + +/** + * Allocate a submix and add it to a given AVIAMFMixPresentation. + * It is freed by av_iamf_mix_presentation_free() alongside the rest of the + * parent AVIAMFMixPresentation. + * + * @return a pointer to the allocated submix. + */ +AVIAMFSubmix *av_iamf_mix_presentation_add_submix(AVIAMFMixPresentation *mix_presentation); + +/** + * Allocate a submix element and add it to a given AVIAMFSubmix. + * It is freed by av_iamf_mix_presentation_free() alongside the rest of the + * parent AVIAMFSubmix. + * + * @return a pointer to the allocated submix. + */ +AVIAMFSubmixElement *av_iamf_submix_add_element(AVIAMFSubmix *submix); + +/** + * Allocate a submix layout and add it to a given AVIAMFSubmix. + * It is freed by av_iamf_mix_presentation_free() alongside the rest of the + * parent AVIAMFSubmix. + * + * @return a pointer to the allocated submix. + */ +AVIAMFSubmixLayout *av_iamf_submix_add_layout(AVIAMFSubmix *submix); + +void av_iamf_mix_presentation_free(AVIAMFMixPresentation **mix_presentation); +/** + * @} + */ + +#endif /* AVUTIL_IAMF_H */