From patchwork Thu Sep 12 00:59:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 51532 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:14c:b0:48e:c0f8:d0de with SMTP id h12csp684953vqi; Wed, 11 Sep 2024 21:04:14 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCV9+vLYXZq87PJxXumsHCsfZxMYwRPln98+Wq1ky+31Az0hCEMFNUfIHCrdwMEHdGlEohE1mdtBWrJub3bKQB8X@gmail.com X-Google-Smtp-Source: AGHT+IGUcnL+JOX9Y3C1vDzjXO7wEdaQTmmY29zj3IRQOjtbgVKnpr0uOJsbqhRjDl4AlWI0evjN X-Received: by 2002:a5d:6743:0:b0:371:6fba:d555 with SMTP id ffacd0b85a97d-378c2cf40bamr728282f8f.18.1726113854246; Wed, 11 Sep 2024 21:04:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1726113854; cv=none; d=google.com; s=arc-20240605; b=WHmi0CwIOUOEZJb5EmarLJpI4L7BqHtAM6ovNE9utfkJ5WUU47RsNCrngbl9vc3DrI Sup/5sz1501H0DljpMPeLmSDaiwFNFEcXt1gDBiix87sa6BO3/OJxovSQGNtu0lCAtw9 cd4sZFYWzN0LIQ20k6LKDocFmhwDGPI31rbhF9OLYOCJUNY9cSQ94+8W7Xs3rReE8f18 FNQMAhGDTK0yXk+5Dr+3yeI/67MXgDMIeuYwQ0D/iyjP9syiIGDo8mHz+JEyAth2ibtu YA8PnB5QjRFRRbed7m2sIiUvjNAQsC71lgWmnjpI4+NWfANFXbGeE9zQUMxPRjrp4pzi iemA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=Z8N4PSzIS4OU3Au+6TpmoPhgf4IrJ6muKhCea17DU1o=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=Vi+DZiyZwwQNqR6BKDl0tNEc6h2dhVAUNPT4xpyZJ6iwYAzUWtdhXlz3vyn4KK5rJA Z6VlVfcYpZ+2y1r0H/7F26FDhZbLowV9XcJSo5H3tsV1UMTLM9diMQ7JfJNOhjgyVB6v lTgIg5nYB7srLYmeAvHvfVwVY1Kv+N6NL/kct5c7PlcX6Dyz0SzkOMrQgQg/LMpxlnVN t5pKyoru303+MX9prbmHF/ypmG3gk1SPD1KKhd4FzKn6Qeg2fhbuzAmPjoiwe/KwfJLy P3pjvyXRURnuFk60E3ajiF2lp5BfP+vekA5aXFJzb31ElTe8/jkU2k36XyuoLeEqo7YV aWAg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=XTi72sbe; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a8d2585ecdbsi259837666b.201.2024.09.11.21.04.13; Wed, 11 Sep 2024 21:04:14 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=XTi72sbe; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8D1E268D842; Thu, 12 Sep 2024 03:59:49 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f170.google.com (mail-pg1-f170.google.com [209.85.215.170]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1CB2E68D7D8 for ; Thu, 12 Sep 2024 03:59:43 +0300 (EEST) Received: by mail-pg1-f170.google.com with SMTP id 41be03b00d2f7-7d82b0ebd51so192900a12.2 for ; Wed, 11 Sep 2024 17:59:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726102781; x=1726707581; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=GjewA5gHMmltCXgdFUNIEmIUePl0aBneI+XD9A9cQmM=; b=XTi72sbeDT50PnASwfkxDB46x1XfGpiaqdvzX7pEWiDmocfeGRqcPuwy6XuX4uvMA2 1y36qKUDChTpL5WgF3XlX/1Ia3DevW5SsD2UsspeOgr0Bad6hfL8lfvpBb8tO8CoAjNq I9D88nFYMhmBEwIesZ3Vz+OrcF6Fmj5EPjGYtFSj+7ace2JhAzQr+LLnX0HcnTP/NEpb YEWdg4RKhHki+5X9dhs145O5HxEZuYVNM60BjWEzUB1R11A2dPMqLfLhkGTk6zR8RLH8 nO9WSLrhg+BrcTjcfRhup8OO6TgANpPk7n+BBZa/L6UZ1Vfx2/6t2LNZv5AH4kHe1ua+ IPdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726102781; x=1726707581; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GjewA5gHMmltCXgdFUNIEmIUePl0aBneI+XD9A9cQmM=; b=maSKBcjTcANFqAtJauG3Jx32hKCDZhukaLk9wI45E1BozgSnOs8P+UbJJY8LIknias AgWAQMiPACvq2XH2ujwUwiQCU4KZJwut5G4aVRcoSK/uYeV5TtMmgWPEoz0PzufsFUrW 3YD+OXbDXRp4wz592nfeSZ5nbvjHMa5RR3Lp1HryYNH9X3F72KhRyGu+6r3NxwkQscQA kkrmkvkQ46U6+3z45ufcTJNsgy/CEPphXeuuiElLmEiHEC6uAULfde4NQjDyCCcIpZko EvhzKYbAL8r8B5YTMC//yMjhv3QIQlHYrJb0H68gQS25SMspkSm2UXhB1yONw8MQt9NU JVtA== X-Gm-Message-State: AOJu0YxoQLOiqSIYd/MAYDUScaZ5HsizoCKX4ay+GvS7R/o5sM3E7dOt 6JzS1CLbN+C+UGJ2y1yKIX6ZJQle2mhP8FxDUqCa7LS45AuBFFE4JW+JpA== X-Received: by 2002:a05:6300:404c:b0:1cf:1db2:da13 with SMTP id adf61e73a8af0-1cf75ea2425mr1512750637.1.1726102780431; Wed, 11 Sep 2024 17:59:40 -0700 (PDT) Received: from localhost.localdomain ([181.92.233.116]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71908fc8210sm3557769b3a.36.2024.09.11.17.59.38 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Sep 2024 17:59:39 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Wed, 11 Sep 2024 21:59:19 -0300 Message-ID: <20240912005925.10151-5-jamrial@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240912005925.10151-1-jamrial@gmail.com> References: <20240912005925.10151-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 05/10 v2] avformat: add an LCEVC stream group X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: vSwLrWGE/Q9N Signed-off-by: James Almer --- Added some documentation and an index field to easily find the enhancement layer stream. doc/APIchanges | 5 +++++ libavformat/avformat.c | 5 +++++ libavformat/avformat.h | 27 +++++++++++++++++++++++++++ libavformat/dump.c | 27 +++++++++++++++++++++++++++ libavformat/options.c | 29 ++++++++++++++++++++++++++++- libavformat/version.h | 4 ++-- 6 files changed, 94 insertions(+), 3 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index 9d7480e5ee..c151041da1 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -2,6 +2,11 @@ The last version increases of all libraries were on 2024-03-07 API changes, most recent first: +2024-09-08 - xxxxxxxxxx - lavf 61.5.100 - avformat.h + Add AVStreamGroupLCEVC + Add AV_STREAM_GROUP_PARAMS_LCEVC + Add AVStreamGroup.params.lcevc + 2024-09-08 - xxxxxxxxxx - lavc 61.14.100 - avcodec.h Add AV_CODEC_ID_LCEVC. diff --git a/libavformat/avformat.c b/libavformat/avformat.c index b4f20502fb..06dcde0565 100644 --- a/libavformat/avformat.c +++ b/libavformat/avformat.c @@ -104,6 +104,10 @@ void ff_free_stream_group(AVStreamGroup **pstg) av_freep(&stg->params.tile_grid->offsets); av_freep(&stg->params.tile_grid); break; + case AV_STREAM_GROUP_PARAMS_LCEVC: + av_opt_free(stg->params.lcevc); + av_freep(&stg->params.lcevc); + break; default: break; } @@ -327,6 +331,7 @@ const char *avformat_stream_group_name(enum AVStreamGroupParamsType type) case AV_STREAM_GROUP_PARAMS_IAMF_AUDIO_ELEMENT: return "IAMF Audio Element"; case AV_STREAM_GROUP_PARAMS_IAMF_MIX_PRESENTATION: return "IAMF Mix Presentation"; case AV_STREAM_GROUP_PARAMS_TILE_GRID: return "Tile Grid"; + case AV_STREAM_GROUP_PARAMS_LCEVC: return "LCEVC (Split video and enhancement)"; } return NULL; } diff --git a/libavformat/avformat.h b/libavformat/avformat.h index 4a3fb00529..56c1c80289 100644 --- a/libavformat/avformat.h +++ b/libavformat/avformat.h @@ -1084,11 +1084,37 @@ typedef struct AVStreamGroupTileGrid { int height; } AVStreamGroupTileGrid; +/** + * AVStreamGroupLCEVC is meant to define the relation between video streams + * and a data stream containing LCEVC enhancement layer NALUs. + * + * No more than one stream of @ref AVCodecParameters.codec_type "codec_type" + * AVMEDIA_TYPE_DATA shall be present, and it must be of + * @ref AVCodecParameters.codec_id "codec_id" AV_CODEC_ID_LCEVC. + */ +typedef struct AVStreamGroupLCEVC { + const AVClass *av_class; + + /** + * Index of the LCEVC data stream in AVStreamGroup. + */ + unsigned int lcevc_index; + /** + * Width of the final stream for presentation. + */ + int width; + /** + * Height of the final image for presentation. + */ + int height; +} AVStreamGroupLCEVC; + enum AVStreamGroupParamsType { AV_STREAM_GROUP_PARAMS_NONE, AV_STREAM_GROUP_PARAMS_IAMF_AUDIO_ELEMENT, AV_STREAM_GROUP_PARAMS_IAMF_MIX_PRESENTATION, AV_STREAM_GROUP_PARAMS_TILE_GRID, + AV_STREAM_GROUP_PARAMS_LCEVC, }; struct AVIAMFAudioElement; @@ -1130,6 +1156,7 @@ typedef struct AVStreamGroup { struct AVIAMFAudioElement *iamf_audio_element; struct AVIAMFMixPresentation *iamf_mix_presentation; struct AVStreamGroupTileGrid *tile_grid; + struct AVStreamGroupLCEVC *lcevc; } params; /** diff --git a/libavformat/dump.c b/libavformat/dump.c index 5e1f367742..f20c2c4953 100644 --- a/libavformat/dump.c +++ b/libavformat/dump.c @@ -789,6 +789,33 @@ static void dump_stream_group(const AVFormatContext *ic, uint8_t *printed, } break; } + case AV_STREAM_GROUP_PARAMS_LCEVC: { + const AVStreamGroupLCEVC *lcevc = stg->params.lcevc; + AVCodecContext *avctx = avcodec_alloc_context3(NULL); + const char *ptr = NULL; + av_log(NULL, AV_LOG_INFO, " LCEVC:"); + if (avctx && stg->nb_streams && !avcodec_parameters_to_context(avctx, stg->streams[0]->codecpar)) { + avctx->width = lcevc->width; + avctx->height = lcevc->height; + avctx->coded_width = lcevc->width; + avctx->coded_height = lcevc->height; + if (ic->dump_separator) + av_opt_set(avctx, "dump_separator", ic->dump_separator, 0); + buf[0] = 0; + avcodec_string(buf, sizeof(buf), avctx, is_output); + ptr = av_stristr(buf, " "); + } + avcodec_free_context(&avctx); + if (ptr) + av_log(NULL, AV_LOG_INFO, "%s", ptr); + av_log(NULL, AV_LOG_INFO, "\n"); + for (int i = 0; i < stg->nb_streams; i++) { + const AVStream *st = stg->streams[i]; + dump_stream_format(ic, st->index, i, index, is_output, AV_LOG_VERBOSE); + printed[st->index] = 1; + } + break; + } default: break; } diff --git a/libavformat/options.c b/libavformat/options.c index 485265df52..039f1eea42 100644 --- a/libavformat/options.c +++ b/libavformat/options.c @@ -348,7 +348,6 @@ static const AVOption tile_grid_options[] = { { "vertical_offset", NULL, OFFSET(vertical_offset), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, FLAGS }, { NULL }, }; -#undef FLAGS #undef OFFSET static const AVClass tile_grid_class = { @@ -357,6 +356,20 @@ static const AVClass tile_grid_class = { .option = tile_grid_options, }; +#define OFFSET(x) offsetof(AVStreamGroupLCEVC, x) +static const AVOption lcevc_options[] = { + { "video_size", "size of video after LCEVC enhancement has been applied", OFFSET(width), + AV_OPT_TYPE_IMAGE_SIZE, { .str = NULL }, 0, INT_MAX, FLAGS }, + { NULL }, +}; +#undef OFFSET + +static const AVClass lcevc_class = { + .class_name = "AVStreamGroupLCEVC", + .version = LIBAVUTIL_VERSION_INT, + .option = lcevc_options, +}; + static void *stream_group_child_next(void *obj, void *prev) { AVStreamGroup *stg = obj; @@ -368,6 +381,8 @@ static void *stream_group_child_next(void *obj, void *prev) return stg->params.iamf_mix_presentation; case AV_STREAM_GROUP_PARAMS_TILE_GRID: return stg->params.tile_grid; + case AV_STREAM_GROUP_PARAMS_LCEVC: + return stg->params.lcevc; default: break; } @@ -375,6 +390,8 @@ static void *stream_group_child_next(void *obj, void *prev) return NULL; } +#undef FLAGS + static const AVClass *stream_group_child_iterate(void **opaque) { uintptr_t i = (uintptr_t)*opaque; @@ -393,6 +410,9 @@ static const AVClass *stream_group_child_iterate(void **opaque) case AV_STREAM_GROUP_PARAMS_TILE_GRID: ret = &tile_grid_class; break; + case AV_STREAM_GROUP_PARAMS_LCEVC: + ret = &lcevc_class; + break; default: break; } @@ -462,6 +482,13 @@ AVStreamGroup *avformat_stream_group_create(AVFormatContext *s, stg->params.tile_grid->av_class = &tile_grid_class; av_opt_set_defaults(stg->params.tile_grid); break; + case AV_STREAM_GROUP_PARAMS_LCEVC: + stg->params.lcevc = av_mallocz(sizeof(*stg->params.lcevc)); + if (!stg->params.lcevc) + goto fail; + stg->params.lcevc->av_class = &lcevc_class; + av_opt_set_defaults(stg->params.lcevc); + break; default: goto fail; } diff --git a/libavformat/version.h b/libavformat/version.h index 384cbd49cc..4bde82abb4 100644 --- a/libavformat/version.h +++ b/libavformat/version.h @@ -31,8 +31,8 @@ #include "version_major.h" -#define LIBAVFORMAT_VERSION_MINOR 5 -#define LIBAVFORMAT_VERSION_MICRO 101 +#define LIBAVFORMAT_VERSION_MINOR 6 +#define LIBAVFORMAT_VERSION_MICRO 100 #define LIBAVFORMAT_VERSION_INT AV_VERSION_INT(LIBAVFORMAT_VERSION_MAJOR, \ LIBAVFORMAT_VERSION_MINOR, \