From patchwork Sun Nov 26 01:28:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 44800 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:bca6:b0:181:818d:5e7f with SMTP id fx38csp2452117pzb; Sat, 25 Nov 2023 17:30:32 -0800 (PST) X-Google-Smtp-Source: AGHT+IFQSDGLbYWzd9IbYE+2udZUd1BHcXxaxHfGdTTRcvQcWwvgg2bUANYlARvhiMjq/v95B3Ve X-Received: by 2002:ac2:596b:0:b0:50b:a823:6217 with SMTP id h11-20020ac2596b000000b0050ba8236217mr2333225lfp.45.1700962232033; Sat, 25 Nov 2023 17:30:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700962232; cv=none; d=google.com; s=arc-20160816; b=anZI5DgyL8UrWsbdmTjNxgnmRR9RQX5gP+z6jhGDTWdvb+bWUQP6iHt597+WnVIDx4 oQo9WiMzA4WDbLM9TX8ydXqQPEXC6mKvzqnL91otF/4cS2bjkik0VOECPqkmLMdwMvYP UjuQfqtQrcZYM+qKiw/TMRGgTganhy+y3RGjJKNyZG/VEjqyYuLdbeKPqmPFCW8cRIcB RRFCYwOdBdnCq02hfIdD7yMQTjGzaK3taFUI3dL6DAWkbhvUsOunCwzGjBmBIA/59pnS /+mlUcEKYRmYvbqiVbgFs+Lp/kk7WvSZQJBSL43ACiVlEoZyauUUUGfZAV9ymBJar5IZ vxLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=v0BEO6LTAxBM2lsgwiB+HSAKd2VGDd45dBF9BejKWEA=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=HHHXrJTaxipxInzusMpNCFdr+J0RlrnQoHvHcrpxCfSJVw/ZbnbsJx/P8+oyLlb1nM jYi74BY/5Tkb2AxBxzNCYE02Mls5mPI/AtXhZCHx3LtiQeCg2sLRgYUBMSaa3VdvfZUQ rnlV1RWLUfScNtTJ7oLrFOoCshHNBEToMcNtvFJRiiLcF1LnqsStVXW5FFFr5DAAWaiM 67Pu/KNeXV5dKpFZDsIPw+7MEnBgAvmLU5NpBtCmk8uOAmjvacU5tQIjIqUxy4h/vMsW EjszJm+jdXHagTN9SMnoAj0H74EnV0FEV/VxM3jLHIopNsp991gFxwrKEQ5VF8Yyqbgk yQ4w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=ZC6iZ520; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s17-20020a50d491000000b00548851486c8si3402973edi.214.2023.11.25.17.30.30; Sat, 25 Nov 2023 17:30:31 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=ZC6iZ520; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 04DF668CF76; Sun, 26 Nov 2023 03:29:39 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f179.google.com (mail-pg1-f179.google.com [209.85.215.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 11F2D68CF5A for ; Sun, 26 Nov 2023 03:29:29 +0200 (EET) Received: by mail-pg1-f179.google.com with SMTP id 41be03b00d2f7-517ab9a4a13so2344769a12.1 for ; Sat, 25 Nov 2023 17:29:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700962166; x=1701566966; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=AdLS/Ay8IIRtgS3D3umO0gTJipQtgrKE9U6qprtF5fs=; b=ZC6iZ520XWDOCg6KAbpBMHCtsPg57WdZ3kKDuYYUWg5roNv37vNJSBTBb/JZjTN1V6 matfgn7UVeLSCu4c//sbZlICiYFjL4dMtHEjF8815W/3B/LdJzb+/JLKrTjw4wx9uyYt 4uC0YDYPy5qGhicKZxtbK/EKvWs83CrQSVKrvfv5meKrLvru57iaQmIZ4sQtUe0MQzY2 ZDDb56oxgA95Oita+DTVGrEohqSdomsmaXyT/eBgdneTEPo2vZDAZ6xPRBq4+iFoC4jH CevoJAzVSV1Zk+zgshoRJrZ4cMc263HYuIk1ATOLseUgPO6RIyn1av7l33BsOba4esbV NJaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700962166; x=1701566966; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AdLS/Ay8IIRtgS3D3umO0gTJipQtgrKE9U6qprtF5fs=; b=edde2kiVVKTm25Za+FmJKotFZgnLa7dhKn2x0tBdGeuO0SnCGG10IHTgMqVvT3k4CG +HGhGve724TM+4qN7u7L6qXxsApScb1ONl6pD0am2rKUEAijwGPi32gqzlhfDA9GqUq+ DMCgddDjFEliaieFaoS3x4s5YGFqkOJ64yaHd683KPirac8ChB2iJVMXlSlmYbt/7A+r yuF+WYYPCwwHfY/s0DNbQOoqQQYsRekQLWk0XWeENJRk5/lTPOjsmH+1d54o472kcDDt LMPcJbPn0LVSSi4N6KtRwIJ/Z1J9iQd5bFAXedmQcf9NoW2u3ox6FDvfu741IQp0/HVr 5DDw== X-Gm-Message-State: AOJu0YxUR8EHLfm6fRuMJfzpJPuXSfc8OT1onfWg54iR4gDfCkjBjAH9 /RzdMQEqoPTEJiKQyTd45QDh6eVAFEQ= X-Received: by 2002:a05:6a20:144b:b0:18c:548d:3d0f with SMTP id a11-20020a056a20144b00b0018c548d3d0fmr3855270pzi.5.1700962166052; Sat, 25 Nov 2023 17:29:26 -0800 (PST) Received: from localhost.localdomain (host197.190-225-105.telecom.net.ar. [190.225.105.197]) by smtp.gmail.com with ESMTPSA id g3-20020a62e303000000b0068a13b0b300sm5049519pfh.11.2023.11.25.17.29.24 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 25 Nov 2023 17:29:25 -0800 (PST) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sat, 25 Nov 2023 22:28:54 -0300 Message-ID: <20231126012858.40388-6-jamrial@gmail.com> X-Mailer: git-send-email 2.42.1 In-Reply-To: <20231126012858.40388-1-jamrial@gmail.com> References: <20231126012858.40388-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 5/9] avformat: introduce AVStreamGroup X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: VdOFBoNLd7gs Signed-off-by: James Almer --- doc/fftools-common-opts.texi | 17 +++- libavformat/avformat.c | 185 ++++++++++++++++++++++++++++++++++- libavformat/avformat.h | 169 ++++++++++++++++++++++++++++++++ libavformat/dump.c | 147 +++++++++++++++++++++++----- libavformat/internal.h | 33 +++++++ libavformat/options.c | 139 ++++++++++++++++++++++++++ 6 files changed, 656 insertions(+), 34 deletions(-) diff --git a/doc/fftools-common-opts.texi b/doc/fftools-common-opts.texi index d9145704d6..f459bfdc1d 100644 --- a/doc/fftools-common-opts.texi +++ b/doc/fftools-common-opts.texi @@ -37,9 +37,9 @@ Matches the stream with this index. E.g. @code{-threads:1 4} would set the thread count for the second stream to 4. If @var{stream_index} is used as an additional stream specifier (see below), then it selects stream number @var{stream_index} from the matching streams. Stream numbering is based on the -order of the streams as detected by libavformat except when a program ID is -also specified. In this case it is based on the ordering of the streams in the -program. +order of the streams as detected by libavformat except when a stream group +specifier or program ID is also specified. In this case it is based on the +ordering of the streams in the group or program. @item @var{stream_type}[:@var{additional_stream_specifier}] @var{stream_type} is one of following: 'v' or 'V' for video, 'a' for audio, 's' for subtitle, 'd' for data, and 't' for attachments. 'v' matches all video @@ -48,6 +48,17 @@ thumbnails or cover arts. If @var{additional_stream_specifier} is used, then it matches streams which both have this type and match the @var{additional_stream_specifier}. Otherwise, it matches all streams of the specified type. +@item g:@var{group_specifier}[:@var{additional_stream_specifier}] +Matches streams which are in the group with the specifier @var{group_specifier}. +if @var{additional_stream_specifier} is used, then it matches streams which both +are part of the group and match the @var{additional_stream_specifier}. +@var{group_specifier} may be one of the following: +@table @option +@item @var{group_index} +Match the stream with this group index. +@item #@var{group_id} or i:@var{group_id} +Match the stream with this group id. +@end table @item p:@var{program_id}[:@var{additional_stream_specifier}] Matches streams which are in the program with the id @var{program_id}. If @var{additional_stream_specifier} is used, then it matches streams which both diff --git a/libavformat/avformat.c b/libavformat/avformat.c index 5b8bb7879e..863fbdd7b8 100644 --- a/libavformat/avformat.c +++ b/libavformat/avformat.c @@ -24,6 +24,7 @@ #include "libavutil/avstring.h" #include "libavutil/channel_layout.h" #include "libavutil/frame.h" +#include "libavutil/iamf.h" #include "libavutil/intreadwrite.h" #include "libavutil/mem.h" #include "libavutil/opt.h" @@ -80,6 +81,32 @@ FF_ENABLE_DEPRECATION_WARNINGS av_freep(pst); } +void ff_free_stream_group(AVStreamGroup **pstg) +{ + AVStreamGroup *stg = *pstg; + + if (!stg) + return; + + av_freep(&stg->streams); + av_dict_free(&stg->metadata); + av_freep(&stg->priv_data); + switch (stg->type) { + case AV_STREAM_GROUP_PARAMS_IAMF_AUDIO_ELEMENT: { + av_iamf_audio_element_free(&stg->params.iamf_audio_element); + break; + } + case AV_STREAM_GROUP_PARAMS_IAMF_MIX_PRESENTATION: { + av_iamf_mix_presentation_free(&stg->params.iamf_mix_presentation); + break; + } + default: + break; + } + + av_freep(pstg); +} + void ff_remove_stream(AVFormatContext *s, AVStream *st) { av_assert0(s->nb_streams>0); @@ -88,6 +115,14 @@ void ff_remove_stream(AVFormatContext *s, AVStream *st) ff_free_stream(&s->streams[ --s->nb_streams ]); } +void ff_remove_stream_group(AVFormatContext *s, AVStreamGroup *stg) +{ + av_assert0(s->nb_stream_groups > 0); + av_assert0(s->stream_groups[ s->nb_stream_groups - 1 ] == stg); + + ff_free_stream_group(&s->stream_groups[ --s->nb_stream_groups ]); +} + /* XXX: suppress the packet queue */ void ff_flush_packet_queue(AVFormatContext *s) { @@ -118,6 +153,9 @@ void avformat_free_context(AVFormatContext *s) for (unsigned i = 0; i < s->nb_streams; i++) ff_free_stream(&s->streams[i]); + for (unsigned i = 0; i < s->nb_stream_groups; i++) + ff_free_stream_group(&s->stream_groups[i]); + s->nb_stream_groups = 0; s->nb_streams = 0; for (unsigned i = 0; i < s->nb_programs; i++) { @@ -139,6 +177,7 @@ void avformat_free_context(AVFormatContext *s) av_packet_free(&si->pkt); av_packet_free(&si->parse_pkt); av_freep(&s->streams); + av_freep(&s->stream_groups); ff_flush_packet_queue(s); av_freep(&s->url); av_free(s); @@ -464,7 +503,7 @@ int av_find_best_stream(AVFormatContext *ic, enum AVMediaType type, */ static int match_stream_specifier(const AVFormatContext *s, const AVStream *st, const char *spec, const char **indexptr, - const AVProgram **p) + const AVStreamGroup **g, const AVProgram **p) { int match = 1; /* Stores if the specifier matches so far. */ while (*spec) { @@ -493,6 +532,46 @@ static int match_stream_specifier(const AVFormatContext *s, const AVStream *st, match = 0; if (nopic && (st->disposition & AV_DISPOSITION_ATTACHED_PIC)) match = 0; + } else if (*spec == 'g' && *(spec + 1) == ':') { + int64_t group_idx = -1, group_id = -1; + int found = 0; + char *endptr; + spec += 2; + if (*spec == '#' || (*spec == 'i' && *(spec + 1) == ':')) { + spec += 1 + (*spec == 'i'); + group_id = strtol(spec, &endptr, 0); + if (spec == endptr || (*endptr && *endptr++ != ':')) + return AVERROR(EINVAL); + spec = endptr; + } else { + group_idx = strtol(spec, &endptr, 0); + /* Disallow empty id and make sure that if we are not at the end, then another specifier must follow. */ + if (spec == endptr || (*endptr && *endptr++ != ':')) + return AVERROR(EINVAL); + spec = endptr; + } + if (match) { + if (group_id > 0) { + for (unsigned i = 0; i < s->nb_stream_groups; i++) { + if (group_id == s->stream_groups[i]->id) { + group_idx = i; + break; + } + } + } + if (group_idx < 0 || group_idx > s->nb_stream_groups) + return AVERROR(EINVAL); + for (unsigned j = 0; j < s->stream_groups[group_idx]->nb_streams; j++) { + if (st->index == s->stream_groups[group_idx]->streams[j]->index) { + found = 1; + if (g) + *g = s->stream_groups[group_idx]; + break; + } + } + } + if (!found) + match = 0; } else if (*spec == 'p' && *(spec + 1) == ':') { int prog_id; int found = 0; @@ -591,10 +670,11 @@ int avformat_match_stream_specifier(AVFormatContext *s, AVStream *st, int ret, index; char *endptr; const char *indexptr = NULL; + const AVStreamGroup *g = NULL; const AVProgram *p = NULL; int nb_streams; - ret = match_stream_specifier(s, st, spec, &indexptr, &p); + ret = match_stream_specifier(s, st, spec, &indexptr, &g, &p); if (ret < 0) goto error; @@ -612,10 +692,11 @@ int avformat_match_stream_specifier(AVFormatContext *s, AVStream *st, return (index == st->index); /* If we requested a matching stream index, we have to ensure st is that. */ - nb_streams = p ? p->nb_stream_indexes : s->nb_streams; + nb_streams = g ? g->nb_streams : (p ? p->nb_stream_indexes : s->nb_streams); for (int i = 0; i < nb_streams && index >= 0; i++) { - const AVStream *candidate = s->streams[p ? p->stream_index[i] : i]; - ret = match_stream_specifier(s, candidate, spec, NULL, NULL); + unsigned idx = g ? g->streams[i]->index : (p ? p->stream_index[i] : i); + const AVStream *candidate = s->streams[idx]; + ret = match_stream_specifier(s, candidate, spec, NULL, NULL, NULL); if (ret < 0) goto error; if (ret > 0 && index-- == 0 && st == candidate) @@ -629,6 +710,100 @@ error: return ret; } +/** + * Matches a stream specifier (but ignores requested index). + * + * @param indexptr set to point to the requested stream index if there is one + * + * @return <0 on error + * 0 if st is NOT a matching stream + * >0 if st is a matching stream + */ +static int match_stream_group_specifier(const AVFormatContext *s, const AVStreamGroup *stg, + const char *spec, const char **indexptr) +{ + int match = 1; /* Stores if the specifier matches so far. */ + while (*spec) { + if (*spec <= '9' && *spec >= '0') { /* opt:index */ + if (indexptr) + *indexptr = spec; + return match; + } else if (*spec == 't' && *(spec + 1) == ':') { + int64_t group_type = -1; + int found = 0; + char *endptr; + spec += 2; + group_type = strtol(spec, &endptr, 0); + /* Disallow empty type and make sure that if we are not at the end, then another specifier must follow. */ + if (spec == endptr || (*endptr && *endptr++ != ':')) + return AVERROR(EINVAL); + spec = endptr; + if (match && group_type > 0) { + for (unsigned i = 0; i < s->nb_stream_groups; i++) { + if (group_type == s->stream_groups[i]->type) { + found = 1; + break; + } + } + } + if (!found) + match = 0; + } else if (*spec == '#' || + (*spec == 'i' && *(spec + 1) == ':')) { + int group_id; + char *endptr; + spec += 1 + (*spec == 'i'); + group_id = strtol(spec, &endptr, 0); + if (spec == endptr || *endptr) /* Disallow empty id and make sure we are at the end. */ + return AVERROR(EINVAL); + return match && (group_id == stg->id); + } + } + + return match; +} + +int avformat_match_stream_group_specifier(AVFormatContext *s, AVStreamGroup *stg, + const char *spec) +{ + int ret, index; + char *endptr; + const char *indexptr = NULL; + + ret = match_stream_group_specifier(s, stg, spec, &indexptr); + if (ret < 0) + goto error; + + if (!indexptr) + return ret; + + index = strtol(indexptr, &endptr, 0); + if (*endptr) { /* We can't have anything after the requested index. */ + ret = AVERROR(EINVAL); + goto error; + } + + /* This is not really needed but saves us a loop for simple stream index specifiers. */ + if (spec == indexptr) + return (index == stg->index); + + /* If we requested a matching stream index, we have to ensure stg is that. */ + for (int i = 0; i < s->nb_stream_groups && index >= 0; i++) { + const AVStreamGroup *candidate = s->stream_groups[i]; + ret = match_stream_group_specifier(s, candidate, spec, NULL); + if (ret < 0) + goto error; + if (ret > 0 && index-- == 0 && stg == candidate) + return 1; + } + return 0; + +error: + if (ret == AVERROR(EINVAL)) + av_log(s, AV_LOG_ERROR, "Invalid stream group specifier: %s.\n", spec); + return ret; +} + AVRational av_guess_sample_aspect_ratio(AVFormatContext *format, AVStream *stream, AVFrame *frame) { AVRational undef = {0, 1}; diff --git a/libavformat/avformat.h b/libavformat/avformat.h index 9e7eca007e..9e428ee843 100644 --- a/libavformat/avformat.h +++ b/libavformat/avformat.h @@ -1018,6 +1018,83 @@ typedef struct AVStream { int pts_wrap_bits; } AVStream; +enum AVStreamGroupParamsType { + AV_STREAM_GROUP_PARAMS_NONE, + AV_STREAM_GROUP_PARAMS_IAMF_AUDIO_ELEMENT, + AV_STREAM_GROUP_PARAMS_IAMF_MIX_PRESENTATION, +}; + +struct AVIAMFAudioElement; +struct AVIAMFMixPresentation; + +typedef struct AVStreamGroup { + /** + * A class for @ref avoptions. Set by avformat_stream_group_create(). + */ + const AVClass *av_class; + + void *priv_data; + + /** + * Group index in AVFormatContext. + */ + unsigned int index; + + /** + * Group type-specific group ID. + * + * decoding: set by libavformat + * encoding: may set by the user + */ + int64_t id; + + /** + * Group type + * + * decoding: set by libavformat on group creation + * encoding: set by avformat_stream_group_create() + */ + enum AVStreamGroupParamsType type; + + /** + * Group type-specific parameters + */ + union { + struct AVIAMFAudioElement *iamf_audio_element; + struct AVIAMFMixPresentation *iamf_mix_presentation; + } params; + + /** + * Metadata that applies to the whole group. + * + * - demuxing: set by libavformat on group creation + * - muxing: may be set by the caller before avformat_write_header() + * + * Freed by libavformat in avformat_free_context(). + */ + AVDictionary *metadata; + + /** + * Number of elements in AVStreamGroup.streams. + * + * Set by avformat_stream_group_add_stream() must not be modified by any other code. + */ + unsigned int nb_streams; + + /** + * A list of streams in the group. New entries are created with + * avformat_stream_group_add_stream(). + * + * - demuxing: entries are created by libavformat on group creation. + * If AVFMTCTX_NOHEADER is set in ctx_flags, then new entries may also + * appear in av_read_frame(). + * - muxing: entries are created by the user before avformat_write_header(). + * + * Freed by libavformat in avformat_free_context(). + */ + AVStream **streams; +} AVStreamGroup; + struct AVCodecParserContext *av_stream_get_parser(const AVStream *s); #if FF_API_GET_END_PTS @@ -1726,6 +1803,26 @@ typedef struct AVFormatContext { * @return 0 on success, a negative AVERROR code on failure */ int (*io_close2)(struct AVFormatContext *s, AVIOContext *pb); + + /** + * Number of elements in AVFormatContext.stream_groups. + * + * Set by avformat_stream_group_create(), must not be modified by any other code. + */ + unsigned int nb_stream_groups; + + /** + * A list of all stream groups in the file. New groups are created with + * avformat_stream_group_create(), and filled with avformat_stream_group_add_stream(). + * + * - demuxing: groups may be created by libavformat in avformat_open_input(). + * If AVFMTCTX_NOHEADER is set in ctx_flags, then new groups may also + * appear in av_read_frame(). + * - muxing: groups may be created by the user before avformat_write_header(). + * + * Freed by libavformat in avformat_free_context(). + */ + AVStreamGroup **stream_groups; } AVFormatContext; /** @@ -1844,6 +1941,37 @@ const AVClass *avformat_get_class(void); */ const AVClass *av_stream_get_class(void); +/** + * Get the AVClass for AVStreamGroup. It can be used in combination with + * AV_OPT_SEARCH_FAKE_OBJ for examining options. + * + * @see av_opt_find(). + */ +const AVClass *av_stream_group_get_class(void); + +/** + * Add a new empty stream group to a media file. + * + * When demuxing, it may be called by the demuxer in read_header(). If the + * flag AVFMTCTX_NOHEADER is set in s.ctx_flags, then it may also + * be called in read_packet(). + * + * When muxing, may be called by the user before avformat_write_header(). + * + * User is required to call avformat_free_context() to clean up the allocation + * by avformat_stream_group_create(). + * + * New streams can be added to the group with avformat_stream_group_add_stream(). + * + * @param s media file handle + * + * @return newly created group or NULL on error. + * @see avformat_new_stream, avformat_stream_group_add_stream. + */ +AVStreamGroup *avformat_stream_group_create(AVFormatContext *s, + enum AVStreamGroupParamsType type, + AVDictionary **options); + /** * Add a new stream to a media file. * @@ -1863,6 +1991,31 @@ const AVClass *av_stream_get_class(void); */ AVStream *avformat_new_stream(AVFormatContext *s, const struct AVCodec *c); +/** + * Add an already allocated stream to a stream group. + * + * When demuxing, it may be called by the demuxer in read_header(). If the + * flag AVFMTCTX_NOHEADER is set in s.ctx_flags, then it may also + * be called in read_packet(). + * + * When muxing, may be called by the user before avformat_write_header() after + * having allocated a new group with avformat_stream_group_create() and stream with + * avformat_new_stream(). + * + * User is required to call avformat_free_context() to clean up the allocation + * by avformat_stream_group_add_stream(). + * + * @param stg stream group belonging to a media file. + * @param st stream in the media file to add to the group. + * + * @retval 0 success + * @retval AVERROR(EEXIST) the stream was already in the group + * @retval "another negative error code" legitimate errors + * + * @see avformat_new_stream, avformat_stream_group_create. + */ +int avformat_stream_group_add_stream(AVStreamGroup *stg, AVStream *st); + #if FF_API_AVSTREAM_SIDE_DATA /** * Wrap an existing array as stream side data. @@ -2819,6 +2972,22 @@ AVRational av_guess_frame_rate(AVFormatContext *ctx, AVStream *stream, int avformat_match_stream_specifier(AVFormatContext *s, AVStream *st, const char *spec); +/** + * Check if the group stg contained in s is matched by the stream group + * specifier spec. + * + * See the "stream group specifiers" chapter in the documentation for the + * syntax of spec. + * + * @return >0 if stg is matched by spec; + * 0 if stg is not matched by spec; + * AVERROR code if spec is invalid + * + * @note A stream group specifier can match several groups in the format. + */ +int avformat_match_stream_group_specifier(AVFormatContext *s, AVStreamGroup *stg, + const char *spec); + int avformat_queue_attached_pictures(AVFormatContext *s); enum AVTimebaseSource { diff --git a/libavformat/dump.c b/libavformat/dump.c index c0868a1bb3..81e5f13004 100644 --- a/libavformat/dump.c +++ b/libavformat/dump.c @@ -24,6 +24,7 @@ #include "libavutil/channel_layout.h" #include "libavutil/display.h" +#include "libavutil/iamf.h" #include "libavutil/intreadwrite.h" #include "libavutil/log.h" #include "libavutil/mastering_display_metadata.h" @@ -134,28 +135,36 @@ static void print_fps(double d, const char *postfix) av_log(NULL, AV_LOG_INFO, "%1.0fk %s", d / 1000, postfix); } -static void dump_metadata(void *ctx, const AVDictionary *m, const char *indent) +static void dump_dictionary(void *ctx, const AVDictionary *m, + const char *name, const char *indent) { - if (m && !(av_dict_count(m) == 1 && av_dict_get(m, "language", NULL, 0))) { - const AVDictionaryEntry *tag = NULL; - - av_log(ctx, AV_LOG_INFO, "%sMetadata:\n", indent); - while ((tag = av_dict_iterate(m, tag))) - if (strcmp("language", tag->key)) { - const char *p = tag->value; - av_log(ctx, AV_LOG_INFO, - "%s %-16s: ", indent, tag->key); - while (*p) { - size_t len = strcspn(p, "\x8\xa\xb\xc\xd"); - av_log(ctx, AV_LOG_INFO, "%.*s", (int)(FFMIN(255, len)), p); - p += len; - if (*p == 0xd) av_log(ctx, AV_LOG_INFO, " "); - if (*p == 0xa) av_log(ctx, AV_LOG_INFO, "\n%s %-16s: ", indent, ""); - if (*p) p++; - } - av_log(ctx, AV_LOG_INFO, "\n"); + const AVDictionaryEntry *tag = NULL; + + if (!m) + return; + + av_log(ctx, AV_LOG_INFO, "%s%s:\n", indent, name); + while ((tag = av_dict_iterate(m, tag))) + if (strcmp("language", tag->key)) { + const char *p = tag->value; + av_log(ctx, AV_LOG_INFO, + "%s %-16s: ", indent, tag->key); + while (*p) { + size_t len = strcspn(p, "\x8\xa\xb\xc\xd"); + av_log(ctx, AV_LOG_INFO, "%.*s", (int)(FFMIN(255, len)), p); + p += len; + if (*p == 0xd) av_log(ctx, AV_LOG_INFO, " "); + if (*p == 0xa) av_log(ctx, AV_LOG_INFO, "\n%s %-16s: ", indent, ""); + if (*p) p++; } - } + av_log(ctx, AV_LOG_INFO, "\n"); + } +} + +static void dump_metadata(void *ctx, const AVDictionary *m, const char *indent) +{ + if (m && !(av_dict_count(m) == 1 && av_dict_get(m, "language", NULL, 0))) + dump_dictionary(ctx, m, "Metadata", indent); } /* param change side data*/ @@ -509,7 +518,7 @@ static void dump_sidedata(void *ctx, const AVStream *st, const char *indent) /* "user interface" functions */ static void dump_stream_format(const AVFormatContext *ic, int i, - int index, int is_output) + int group_index, int index, int is_output) { char buf[256]; int flags = (is_output ? ic->oformat->flags : ic->iformat->flags); @@ -517,6 +526,8 @@ static void dump_stream_format(const AVFormatContext *ic, int i, const FFStream *const sti = cffstream(st); const AVDictionaryEntry *lang = av_dict_get(st->metadata, "language", NULL, 0); const char *separator = ic->dump_separator; + const char *group_indent = group_index >= 0 ? " " : ""; + const char *extra_indent = group_index >= 0 ? " " : " "; AVCodecContext *avctx; int ret; @@ -543,7 +554,8 @@ static void dump_stream_format(const AVFormatContext *ic, int i, avcodec_string(buf, sizeof(buf), avctx, is_output); avcodec_free_context(&avctx); - av_log(NULL, AV_LOG_INFO, " Stream #%d:%d", index, i); + av_log(NULL, AV_LOG_INFO, "%s Stream #%d", group_indent, index); + av_log(NULL, AV_LOG_INFO, ":%d", i); /* the pid is an important information, so we display it */ /* XXX: add a generic system */ @@ -621,9 +633,89 @@ static void dump_stream_format(const AVFormatContext *ic, int i, av_log(NULL, AV_LOG_INFO, " (non-diegetic)"); av_log(NULL, AV_LOG_INFO, "\n"); - dump_metadata(NULL, st->metadata, " "); + dump_metadata(NULL, st->metadata, extra_indent); + + dump_sidedata(NULL, st, extra_indent); +} + +static void dump_stream_group(const AVFormatContext *ic, uint8_t *printed, + int i, int index, int is_output) +{ + const AVStreamGroup *stg = ic->stream_groups[i]; + int flags = (is_output ? ic->oformat->flags : ic->iformat->flags); + char buf[512]; + int ret; - dump_sidedata(NULL, st, " "); + av_log(NULL, AV_LOG_INFO, " Stream group #%d:%d", index, i); + if (flags & AVFMT_SHOW_IDS) + av_log(NULL, AV_LOG_INFO, "[0x%"PRIx64"]", stg->id); + av_log(NULL, AV_LOG_INFO, ":"); + + switch (stg->type) { + case AV_STREAM_GROUP_PARAMS_IAMF_AUDIO_ELEMENT: { + const AVIAMFAudioElement *audio_element = stg->params.iamf_audio_element; + av_log(NULL, AV_LOG_INFO, " IAMF Audio Element\n"); + dump_metadata(NULL, stg->metadata, " "); + for (int j = 0; j < audio_element->num_layers; j++) { + const AVIAMFLayer *layer = audio_element->layers[j]; + int channel_count = layer->ch_layout.nb_channels; + av_log(NULL, AV_LOG_INFO, " Layer %d:", j); + ret = av_channel_layout_describe(&layer->ch_layout, buf, sizeof(buf)); + if (ret >= 0) + av_log(NULL, AV_LOG_INFO, " %s", buf); + av_log(NULL, AV_LOG_INFO, "\n"); + for (int k = 0; channel_count > 0 && k < stg->nb_streams; k++) { + AVStream *st = stg->streams[k]; + dump_stream_format(ic, st->index, i, index, is_output); + printed[st->index] = 1; + channel_count -= st->codecpar->ch_layout.nb_channels; + } + } + break; + } + case AV_STREAM_GROUP_PARAMS_IAMF_MIX_PRESENTATION: { + const AVIAMFMixPresentation *mix_presentation = stg->params.iamf_mix_presentation; + av_log(NULL, AV_LOG_INFO, " IAMF Mix Presentation\n"); + dump_metadata(NULL, stg->metadata, " "); + dump_dictionary(NULL, mix_presentation->annotations, "Annotations", " "); + for (int j = 0; j < mix_presentation->num_submixes; j++) { + AVIAMFSubmix *sub_mix = mix_presentation->submixes[j]; + av_log(NULL, AV_LOG_INFO, " Submix %d:\n", j); + for (int k = 0; k < sub_mix->num_elements; k++) { + const AVIAMFSubmixElement *submix_element = sub_mix->elements[k]; + const AVStreamGroup *audio_element = NULL; + for (int l = 0; l < ic->nb_stream_groups; l++) + if (ic->stream_groups[l]->type == AV_STREAM_GROUP_PARAMS_IAMF_AUDIO_ELEMENT && + ic->stream_groups[l]->id == submix_element->audio_element_id) { + audio_element = ic->stream_groups[l]; + break; + } + if (audio_element) { + av_log(NULL, AV_LOG_INFO, " IAMF Audio Element #%d:%d", + index, audio_element->index); + if (flags & AVFMT_SHOW_IDS) + av_log(NULL, AV_LOG_INFO, "[0x%"PRIx64"]", audio_element->id); + av_log(NULL, AV_LOG_INFO, "\n"); + dump_dictionary(NULL, submix_element->annotations, "Annotations", " "); + } + } + for (int k = 0; k < sub_mix->num_layouts; k++) { + const AVIAMFSubmixLayout *submix_layout = sub_mix->layouts[k]; + av_log(NULL, AV_LOG_INFO, " Layout #%d:", k); + if (submix_layout->layout_type == 2) { + ret = av_channel_layout_describe(&submix_layout->sound_system, buf, sizeof(buf)); + if (ret >= 0) + av_log(NULL, AV_LOG_INFO, " %s", buf); + } else if (submix_layout->layout_type == 3) + av_log(NULL, AV_LOG_INFO, " Binaural"); + av_log(NULL, AV_LOG_INFO, "\n"); + } + } + break; + } + default: + break; + } } void av_dump_format(AVFormatContext *ic, int index, @@ -699,7 +791,7 @@ void av_dump_format(AVFormatContext *ic, int index, dump_metadata(NULL, program->metadata, " "); for (k = 0; k < program->nb_stream_indexes; k++) { dump_stream_format(ic, program->stream_index[k], - index, is_output); + -1, index, is_output); printed[program->stream_index[k]] = 1; } total += program->nb_stream_indexes; @@ -708,9 +800,12 @@ void av_dump_format(AVFormatContext *ic, int index, av_log(NULL, AV_LOG_INFO, " No Program\n"); } + for (i = 0; i < ic->nb_stream_groups; i++) + dump_stream_group(ic, printed, i, index, is_output); + for (i = 0; i < ic->nb_streams; i++) if (!printed[i]) - dump_stream_format(ic, i, index, is_output); + dump_stream_format(ic, i, -1, index, is_output); av_free(printed); } diff --git a/libavformat/internal.h b/libavformat/internal.h index 7702986c9c..c6181683ef 100644 --- a/libavformat/internal.h +++ b/libavformat/internal.h @@ -202,6 +202,7 @@ typedef struct FFStream { */ AVStream pub; + AVFormatContext *fmtctx; /** * Set to 1 if the codec allows reordering, so pts can be different * from dts. @@ -427,6 +428,26 @@ static av_always_inline const FFStream *cffstream(const AVStream *st) return (const FFStream*)st; } +typedef struct FFStreamGroup { + /** + * The public context. + */ + AVStreamGroup pub; + + AVFormatContext *fmtctx; +} FFStreamGroup; + + +static av_always_inline FFStreamGroup *ffstreamgroup(AVStreamGroup *stg) +{ + return (FFStreamGroup*)stg; +} + +static av_always_inline const FFStreamGroup *cffstreamgroup(const AVStreamGroup *stg) +{ + return (const FFStreamGroup*)stg; +} + #ifdef __GNUC__ #define dynarray_add(tab, nb_ptr, elem)\ do {\ @@ -608,6 +629,18 @@ void ff_free_stream(AVStream **st); */ void ff_remove_stream(AVFormatContext *s, AVStream *st); +/** + * Frees a stream group without modifying the corresponding AVFormatContext. + * Must only be called if the latter doesn't matter or if the stream + * is not yet attached to an AVFormatContext. + */ +void ff_free_stream_group(AVStreamGroup **pstg); +/** + * Remove a stream group from its AVFormatContext and free it. + * The group must be the last stream of the AVFormatContext. + */ +void ff_remove_stream_group(AVFormatContext *s, AVStreamGroup *stg); + unsigned int ff_codec_get_tag(const AVCodecTag *tags, enum AVCodecID id); enum AVCodecID ff_codec_get_id(const AVCodecTag *tags, unsigned int tag); diff --git a/libavformat/options.c b/libavformat/options.c index 1d8c52246b..bf6113ca95 100644 --- a/libavformat/options.c +++ b/libavformat/options.c @@ -26,6 +26,7 @@ #include "libavcodec/codec_par.h" #include "libavutil/avassert.h" +#include "libavutil/iamf.h" #include "libavutil/internal.h" #include "libavutil/intmath.h" #include "libavutil/opt.h" @@ -271,6 +272,7 @@ AVStream *avformat_new_stream(AVFormatContext *s, const AVCodec *c) if (!st->codecpar) goto fail; + sti->fmtctx = s; sti->avctx = avcodec_alloc_context3(NULL); if (!sti->avctx) goto fail; @@ -325,6 +327,143 @@ fail: return NULL; } +static void *stream_group_child_next(void *obj, void *prev) +{ + AVStreamGroup *stg = obj; + if (!prev) { + switch(stg->type) { + case AV_STREAM_GROUP_PARAMS_IAMF_AUDIO_ELEMENT: + return stg->params.iamf_audio_element; + case AV_STREAM_GROUP_PARAMS_IAMF_MIX_PRESENTATION: + return stg->params.iamf_mix_presentation; + default: + break; + } + } + return NULL; +} + +static const AVClass *stream_group_child_iterate(void **opaque) +{ + uintptr_t i = (uintptr_t)*opaque; + const AVClass *ret = NULL; + + switch(i) { + case AV_STREAM_GROUP_PARAMS_IAMF_AUDIO_ELEMENT: + ret = av_iamf_audio_element_get_class(); + break; + case AV_STREAM_GROUP_PARAMS_IAMF_MIX_PRESENTATION: + ret = av_iamf_mix_presentation_get_class(); + break; + default: + break; + } + + if (ret) + *opaque = (void*)(i + 1); + return ret; +} + +static const AVOption stream_group_options[] = { + {"id", "Set group id", offsetof(AVStreamGroup, id), AV_OPT_TYPE_INT64, {.i64 = 0}, 0, INT64_MAX, AV_OPT_FLAG_ENCODING_PARAM }, + { NULL } +}; + +static const AVClass stream_group_class = { + .class_name = "AVStreamGroup", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, + .option = stream_group_options, + .child_next = stream_group_child_next, + .child_class_iterate = stream_group_child_iterate, +}; + +const AVClass *av_stream_group_get_class(void) +{ + return &stream_group_class; +} + +AVStreamGroup *avformat_stream_group_create(AVFormatContext *s, + enum AVStreamGroupParamsType type, + AVDictionary **options) +{ + AVStreamGroup **stream_groups; + AVStreamGroup *stg; + FFStreamGroup *stgi; + + stream_groups = av_realloc_array(s->stream_groups, s->nb_stream_groups + 1, + sizeof(*stream_groups)); + if (!stream_groups) + return NULL; + s->stream_groups = stream_groups; + + stgi = av_mallocz(sizeof(*stgi)); + if (!stgi) + return NULL; + stg = &stgi->pub; + + stg->av_class = &stream_group_class; + av_opt_set_defaults(stg); + stg->type = type; + switch (type) { + case AV_STREAM_GROUP_PARAMS_IAMF_AUDIO_ELEMENT: + stg->params.iamf_audio_element = av_iamf_audio_element_alloc(); + if (!stg->params.iamf_audio_element) + goto fail; + break; + case AV_STREAM_GROUP_PARAMS_IAMF_MIX_PRESENTATION: + stg->params.iamf_mix_presentation = av_iamf_mix_presentation_alloc(); + if (!stg->params.iamf_mix_presentation) + goto fail; + break; + default: + goto fail; + } + + if (options) { + if (av_opt_set_dict2(stg, options, AV_OPT_SEARCH_CHILDREN)) + goto fail; + } + + stgi->fmtctx = s; + stg->index = s->nb_stream_groups; + + s->stream_groups[s->nb_stream_groups++] = stg; + + return stg; +fail: + ff_free_stream_group(&stg); + return NULL; +} + +static int stream_group_add_stream(AVStreamGroup *stg, AVStream *st) +{ + AVStream **streams = av_realloc_array(stg->streams, stg->nb_streams + 1, + sizeof(*stg->streams)); + if (!streams) + return AVERROR(ENOMEM); + + stg->streams = streams; + stg->streams[stg->nb_streams++] = st; + + return 0; +} + +int avformat_stream_group_add_stream(AVStreamGroup *stg, AVStream *st) +{ + const FFStreamGroup *stgi = cffstreamgroup(stg); + const FFStream *sti = cffstream(st); + + if (stgi->fmtctx != sti->fmtctx) + return AVERROR(EINVAL); + + for (int i = 0; i < stg->nb_streams; i++) + if (stg->streams[i]->index == st->index) + return AVERROR(EEXIST); + + return stream_group_add_stream(stg, st); +} + static int option_is_disposition(const AVOption *opt) { return opt->type == AV_OPT_TYPE_CONST &&