From patchwork Wed Jan 24 14:54:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 45793 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:120f:b0:199:de12:6fa6 with SMTP id v15csp1324025pzf; Wed, 24 Jan 2024 06:54:45 -0800 (PST) X-Google-Smtp-Source: AGHT+IG+mrGGGCa+XSa7p3LCvHjPbPOc7Pk9CDt09GZcYAhKvdtbg8udUbOCKPRRNMPeqUPPftbC X-Received: by 2002:a05:600c:1d2a:b0:40e:676c:33e5 with SMTP id l42-20020a05600c1d2a00b0040e676c33e5mr883930wms.79.1706108084843; Wed, 24 Jan 2024 06:54:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1706108084; cv=none; d=google.com; s=arc-20160816; b=NUrSqogTab8CMFc2iiVxX/lL9r8MsymOIo+w267Dkk5V9bApRxJD2p7qKcrbl+rpQD AKoj5wEfQ33ywPyl5WhocGHVZt8r1RsRWR4HCvfe3uOfehyiCHF78rNVs6aAu/0IXmIh B6K4CDIr83to2tm7NW1ushkxfO+p6SPmTcUc3A9bzBiZfO0DOADy1LV4ibJu7TM4BhEQ GYCuR1r4AYuqbh0i3qaRkiTvBU6/NcXYWGjYcFoRMJhMRR+zYRYzJGhLYCRzUElgU0WQ lnSk1ZLStFkl8G3rKDeKuB6Nl7hxx/GrFlm920GJojwOHuzErv2ZQCwWxWc1fUvfRtL1 4tVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=LFNxx8XokPl32Fhdl7F5aCSY8QEwbbvBicT7FK+Lyus=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=GCYagJaqutlidtrMrcrG8X9QzLg5XofuftOzfImzU2N55kPCDbyqMIPGyQq6hAy4Hq /uGKfHDhMfGMDcsb5PC1703wnt/Qf+MvNBFkV+0rF3zrNwAEZBwzlMCgyCa2RJOqo/gL GlaPM9GDBy8tkcqecw0krZryCbYZZHsm1AWw2UXzNcD0PwZztnzuW0CnlXNQv9AZduCD UjpFgjTOLgOW3PoQ62EpAKidKS1mfHr1Nq4NMG/za8L5xKn/XQw2CZtOUzyX2y4hlbBS aoZA//9PGDIoK5TJm/uCqslFxl1AVqqQxRTnXe3nx+9WKT7tiAS5XIgztOIeF3P3Kuhf dhTw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=dla8w2sQ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ij14-20020a056402158e00b0055cc7f11b3fsi727015edb.672.2024.01.24.06.54.44; Wed, 24 Jan 2024 06:54:44 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=dla8w2sQ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 107BF68D0EE; Wed, 24 Jan 2024 16:54:35 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 59CE868CD0C for ; Wed, 24 Jan 2024 16:54:26 +0200 (EET) Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-6ddc1fad6ddso188076b3a.0 for ; Wed, 24 Jan 2024 06:54:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706108063; x=1706712863; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=oYfrk22LfGGes5ye51Hq3PCr7wmHNu/ut98ZTZmKbtc=; b=dla8w2sQ1thVuNNYG7V5aooooUrPI91l9X23SjHTBrMnWm/dXN9nDb2IrLjH/bvXwz VwD3wbbTz1pY12NMwvGVx4D+X1Z+jVnFU906QWGSlAAC/MyoJDdURTe1hyIR0LUNaafa WXOSd84lOH0WJSK+8/u7PrtZRoGdqyxwYIz6AK6xrUANSLiIbL9dK46N2YzX3gmpFahV t81JTM7+nwSmJxnsSa5FFR9PVSgIfoKuJGj4y/0O29gFxKYGAQc109I9O4dYE8hFpR3r rXpXAcuNYp92tSMJrbB8wEsBuGXLf4N7zEP938io5ojF1oIYv3jbnOYoE6qzzxk9PrPm GtsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706108063; x=1706712863; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oYfrk22LfGGes5ye51Hq3PCr7wmHNu/ut98ZTZmKbtc=; b=LicSfPxJahDSqMc1Ch4pOxuXdQl+/SmD9Q5PDuAQLTH72tEdWPh6oHissnHG/Erx63 bIrrTlJsLIFMoyj3INAo5RTifWC2Rk2URQANcYiiStkxe5/ecftyDvhBUbD/ZoppJxJw 6cAVgfqePDxDYcey+QzBsgJiZT6SiUNhua82rFVzcQDh2D7mPrvpFhp0VZndSDYeQ9fR 4bXKTptIwkVfiLnXtUkJWqLTkw0mAGX5v9AOabKnuLRV4TMENAGcp2xei7i6dmpVFATJ 1+gnzwYh5QgFMGMCGREKc4XLIDQXb5tEANkAinbGJkiJ2OdpB3Pt6wa5EYc69xSrLIbq 4vwA== X-Gm-Message-State: AOJu0YyyonzOZRKZfX97SJyksaEJMxR/kCSCTmTlZbZkJinKeDB2CEgx LPAtxw8fDScZBQfx/4INs8imTfU9K+DBlXpH81D3Xcpv1o9XTinf54o2NyKb X-Received: by 2002:a05:6a21:3941:b0:19c:6392:bd04 with SMTP id ac1-20020a056a21394100b0019c6392bd04mr998136pzc.12.1706108063473; Wed, 24 Jan 2024 06:54:23 -0800 (PST) Received: from localhost.localdomain (host197.190-225-105.telecom.net.ar. [190.225.105.197]) by smtp.gmail.com with ESMTPSA id s23-20020a62e717000000b006dda22e1c2csm1144598pfh.115.2024.01.24.06.54.22 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 06:54:22 -0800 (PST) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Wed, 24 Jan 2024 11:54:17 -0300 Message-ID: <20240124145419.7980-2-jamrial@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240124145419.7980-1-jamrial@gmail.com> References: <20240124145419.7980-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/4 v4] avformat: add a Tile Grid stream group type X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 7pTfhkZXUiLH This will be used to support tiled image formats like HEIF. Signed-off-by: James Almer --- libavformat/avformat.c | 8 ++++ libavformat/avformat.h | 86 ++++++++++++++++++++++++++++++++++++++++++ libavformat/dump.c | 36 ++++++++++++++++++ libavformat/options.c | 32 ++++++++++++++++ 4 files changed, 162 insertions(+) diff --git a/libavformat/avformat.c b/libavformat/avformat.c index 882927f7b1..58dfa0746a 100644 --- a/libavformat/avformat.c +++ b/libavformat/avformat.c @@ -100,6 +100,14 @@ void ff_free_stream_group(AVStreamGroup **pstg) av_iamf_mix_presentation_free(&stg->params.iamf_mix_presentation); break; } + case AV_STREAM_GROUP_PARAMS_TILE_GRID: { + if (stg->params.tile_grid) { + av_freep(&stg->params.tile_grid->tile_width); + av_freep(&stg->params.tile_grid->tile_height); + } + av_freep(&stg->params.tile_grid); + break; + } default: break; } diff --git a/libavformat/avformat.h b/libavformat/avformat.h index 5d0fe82250..ab9a3fc6be 100644 --- a/libavformat/avformat.h +++ b/libavformat/avformat.h @@ -1022,11 +1022,96 @@ enum AVStreamGroupParamsType { AV_STREAM_GROUP_PARAMS_NONE, AV_STREAM_GROUP_PARAMS_IAMF_AUDIO_ELEMENT, AV_STREAM_GROUP_PARAMS_IAMF_MIX_PRESENTATION, + AV_STREAM_GROUP_PARAMS_TILE_GRID, }; struct AVIAMFAudioElement; struct AVIAMFMixPresentation; +/** + * AVTileGrid holds information on how to combine several independent images in + * a single grid for presentation. + * + * Its size is not a part of the ABI. No new fields may be added to this struct + * without a major version bump. + */ +typedef struct AVTileGrid { + const AVClass *av_class; + + /** + * Amount of rows in the grid. + * + * Must be > 0. + */ + int tile_rows; + /** + * Amount of columns in the grid. + * + * Must be > 0. + */ + int tile_cols; + + /** + * A @ref tile_rows * @ref tile_cols sized array of width values for each + * tile in the grid, in row major order. + * The sum of tile width values must be consistent across all rows. + * + * Must be allocated with the av_malloc() family of functions, and will be + * freed by av_tile_grid_free(). + */ + int *tile_width; + + /** + * A @ref tile_rows * @ref tile_cols sized array of height values for each + * tile in the grid, in row major order. + * The sum of tile height values must be consistent across all columns. + * + * Must be allocated with the av_malloc() family of functions, and will be + * freed by av_tile_grid_free(). + */ + int *tile_height; + + /** + * Offset in pixels from the left edge of the grid where the actual image + * meant for presentation starts. + * + * This field must be >= 0 and <= the sum of values for a row in the + * tile_width array minus @ref output_width. + */ + int horizontal_offset; + /** + * Offset in pixels from the top edge of the grid where the actual image meant + * for presentation starts. + * + * This field must be >= 0 and <= the sum of values for a column in the + * tile_height array minus @ref output_height. + */ + int vertical_offset; + + /** + * Width of the final image for presentation. + * + * Must be > 0 and <= the sum of values for a row in the tile_width array + * minus @ref horizontal_offset. + * When it's not equal to the sum of values for a row in the tile_width array, + * the result of said sum minus output_width minus @ref horizontal_offset is + * the amount of pixels to be cropped from the right edge of the final image + * before presentation. + */ + int output_width; + /** + * Height of the final image for presentation. + * + * Must be > 0 and <= the sum of values for a column in the tile_height array + * minus @ref vertical_offset. + * When it's not equal to the sum of values for a column in the tile_height, + * the result of said sum minus output_height minus @ref vertical_offset is + * the amount of pixels to be cropped from the bottom edge of the final image + * before presentation. + */ + int output_height; +} AVTileGrid; + typedef struct AVStreamGroup { /** * A class for @ref avoptions. Set by avformat_stream_group_create(). @@ -1062,6 +1147,7 @@ typedef struct AVStreamGroup { union { struct AVIAMFAudioElement *iamf_audio_element; struct AVIAMFMixPresentation *iamf_mix_presentation; + struct AVTileGrid *tile_grid; } params; /** diff --git a/libavformat/dump.c b/libavformat/dump.c index aff51b43f6..33932f39ae 100644 --- a/libavformat/dump.c +++ b/libavformat/dump.c @@ -22,6 +22,7 @@ #include #include +#include "libavutil/avstring.h" #include "libavutil/channel_layout.h" #include "libavutil/display.h" #include "libavutil/iamf.h" @@ -720,6 +721,41 @@ static void dump_stream_group(const AVFormatContext *ic, uint8_t *printed, } } break; + case AV_STREAM_GROUP_PARAMS_TILE_GRID: { + const AVTileGrid *tile_grid = stg->params.tile_grid; + AVCodecContext *avctx = avcodec_alloc_context3(NULL); + const char *ptr = NULL; + av_log(NULL, AV_LOG_INFO, " Tile Grid:"); + av_log(NULL, AV_LOG_VERBOSE, " %d rows, %d columns,", tile_grid->tile_rows, tile_grid->tile_cols); + if (avctx && stg->nb_streams && !avcodec_parameters_to_context(avctx, stg->streams[0]->codecpar)) { + int size = tile_grid->tile_rows * tile_grid->tile_cols; + int coded_width = 0, coded_height = 0; + avctx->width = tile_grid->output_width; + avctx->height = tile_grid->output_height; + for (int j = 0; j < tile_grid->tile_cols; j++) + coded_width += tile_grid->tile_width[j]; + for (int j = 0; j < size; j += tile_grid->tile_cols) + coded_height += tile_grid->tile_height[j]; + avctx->coded_width = coded_width; + avctx->coded_height = coded_height; + if (ic->dump_separator) + av_opt_set(avctx, "dump_separator", ic->dump_separator, 0); + buf[0] = 0; + avcodec_string(buf, sizeof(buf), avctx, is_output); + ptr = av_stristr(buf, " "); + } + avcodec_free_context(&avctx); + if (ptr) + av_log(NULL, AV_LOG_INFO, "%s", ptr); + av_log(NULL, AV_LOG_INFO, "\n"); + dump_metadata(NULL, stg->metadata, " ", AV_LOG_INFO); + for (int i = 0; i < stg->nb_streams; i++) { + const AVStream *st = stg->streams[i]; + dump_stream_format(ic, st->index, i, index, is_output, AV_LOG_VERBOSE); + printed[st->index] = 1; + } + break; + } } default: break; diff --git a/libavformat/options.c b/libavformat/options.c index 75ec86ce05..31620013c6 100644 --- a/libavformat/options.c +++ b/libavformat/options.c @@ -337,6 +337,26 @@ fail: return NULL; } +#define FLAGS AV_OPT_FLAG_ENCODING_PARAM | AV_OPT_FLAG_VIDEO_PARAM +#define OFFSET(x) offsetof(AVTileGrid, x) +static const AVOption tile_grid_options[] = { + { "tile_rows", NULL, OFFSET(tile_rows), AV_OPT_TYPE_INT, { .i64 = 1 }, 1, INT_MAX, FLAGS }, + { "tile_cols", NULL, OFFSET(tile_cols), AV_OPT_TYPE_INT, { .i64 = 1 }, 1, INT_MAX, FLAGS }, + { "horizontal_offset", NULL, OFFSET(horizontal_offset), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, FLAGS }, + { "vertical_offset", NULL, OFFSET(vertical_offset), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, FLAGS }, + { "output_size", "size of the output image", OFFSET(output_width), AV_OPT_TYPE_IMAGE_SIZE, + { .str = NULL }, 0, INT_MAX, FLAGS }, + { NULL }, +}; +#undef FLAGS +#undef OFFSET + +static const AVClass tile_grid_class = { + .class_name = "AVTileGrid", + .version = LIBAVUTIL_VERSION_INT, + .option = tile_grid_options, +}; + static void *stream_group_child_next(void *obj, void *prev) { AVStreamGroup *stg = obj; @@ -346,6 +366,8 @@ static void *stream_group_child_next(void *obj, void *prev) return stg->params.iamf_audio_element; case AV_STREAM_GROUP_PARAMS_IAMF_MIX_PRESENTATION: return stg->params.iamf_mix_presentation; + case AV_STREAM_GROUP_PARAMS_TILE_GRID: + return stg->params.tile_grid; default: break; } @@ -368,6 +390,9 @@ static const AVClass *stream_group_child_iterate(void **opaque) case AV_STREAM_GROUP_PARAMS_IAMF_MIX_PRESENTATION: ret = av_iamf_mix_presentation_get_class(); break; + case AV_STREAM_GROUP_PARAMS_TILE_GRID: + ret = &tile_grid_class; + break; default: break; } @@ -429,6 +454,13 @@ AVStreamGroup *avformat_stream_group_create(AVFormatContext *s, if (!stg->params.iamf_mix_presentation) goto fail; break; + case AV_STREAM_GROUP_PARAMS_TILE_GRID: + stg->params.tile_grid = av_mallocz(sizeof(*stg->params.tile_grid)); + if (!stg->params.tile_grid) + goto fail; + stg->params.tile_grid->av_class = &tile_grid_class; + av_opt_set_defaults(stg->params.tile_grid); + break; default: goto fail; }