From patchwork Sat Dec 11 12:17:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 32285 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp3033004iog; Sat, 11 Dec 2021 04:18:03 -0800 (PST) X-Google-Smtp-Source: ABdhPJwOp5rWXvUo9a1G+RDgWmMdrh0EAMXPMkkndWl9AY5BiiVUXi4hw7mtyr5yChBQ6Sa1IraJ X-Received: by 2002:aa7:c846:: with SMTP id g6mr46055002edt.75.1639225083038; Sat, 11 Dec 2021 04:18:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639225083; cv=none; d=google.com; s=arc-20160816; b=mggCSB4ZG5RDynLE8zGQtu5TrVgtQFoyyEQ7hGsCyGMgpgzRpGd5veNZ+/PhUZ3eyK P3VDFbuuTR/P/t+gRX0YgwJhsB/P5QmAw0qa3IXQe6G2z1TRubmCqchrlcq/K398BR3S VJ8FtLqk4QNpA+vkQGF94c2ietX6IziEWymeszDPPGS69b/3D+pD90evW9u2HHZCDJNV aKCe1Lc6FAxuM8LryfTjVxSqxjX9zafmQV8/c+p/Gn8dlyrx4ln9x/2iqClQeCbobObO 1S24xU3284ck+++RIOCswfeieSxgAL2CuvEIfz/C4nn02FDu/3q5fUvew8rQ4pVf6Usf 2orQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=2O27cpaxWhkgCrg3chD579kvAJrCgNbgemiQKG2FEsI=; b=FVDCsHTNybeQsfch7W82Nwqgb5QUUy9BOTCcz/IZaT2u/fay3uty15a/IYSCPx22Sa SiXX+PPVoOqiu4/rMtT1y96Fu1HQlE5u7PUhz1pdUSlrqdXJdqpIb6BZQVW5Fxd03X6r BCenW0VXfZkSkIgfIiW+xdM531hjsh2QMD7RwjmsflWw3ndAxDpU/aNmCB157iLRP1SK X/j/VG2qzmlwxVKIV7quu+y25aMN3p7b5dyqfaRT4QsR97lqCSfoJ8tqQYqie942nFtU WaZts6K4bF+iSZgSLvJmI2k6IqvXh1izLtD7uskCe3MStmxiUPPYZ6Ea0i+/Ec/h7xX4 obUg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=CGWRh8PN; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hv12si7977500ejc.372.2021.12.11.04.18.02; Sat, 11 Dec 2021 04:18:03 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=CGWRh8PN; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2CED068AECA; Sat, 11 Dec 2021 14:17:58 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 640BC68A8A0 for ; Sat, 11 Dec 2021 14:17:51 +0200 (EET) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 1559D49251; Sat, 11 Dec 2021 13:17:51 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1639225071; bh=OjVaoSxVR+Kvo4Ybx8zW3g8uaXiW/wqA+JjDgdr1mzU=; h=From:To:Cc:Subject:Date:From; b=CGWRh8PNKWzkODQz6tEEZU0x3oxaeaCQjmURYCOwRGvA9TzOZWuJVvMx62FI5zl9X 7JRs/Ze9DT98FOgAL6wpt83MQY/S+H1dSShz+lL3l819+wPWOIzctMt0ldzej26wC5 IoFx1/pCepq+sOw+YlkEjSSW9VBDA9MYl7u3Ckj8= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Sat, 11 Dec 2021 13:17:43 +0100 Message-Id: <20211211121747.117143-1-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 1/5] lavu/frame: Add Dolby Vision metadata side data type X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: oVGwY3H0ZBXr From: Niklas Haas Signed-off-by: Niklas Haas --- doc/APIchanges | 3 ++ libavutil/dovi_meta.h | 116 ++++++++++++++++++++++++++++++++++++++++++ libavutil/frame.c | 1 + libavutil/frame.h | 9 +++- libavutil/version.h | 2 +- 5 files changed, 129 insertions(+), 2 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index 2914ad6734..422874e3b9 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -14,6 +14,9 @@ libavutil: 2021-04-27 API changes, most recent first: +2021-12-06 - xxxxxxxxxx - lavu 57.11.100 - frame.h + Add AV_FRAME_DATA_DOVI_RESHAPING. + 2021-11-xx - xxxxxxxxxx - lavfi 8.19.100 - avfilter.h Add AVFILTER_FLAG_METADATA_ONLY. diff --git a/libavutil/dovi_meta.h b/libavutil/dovi_meta.h index 299911d434..432c48b53d 100644 --- a/libavutil/dovi_meta.h +++ b/libavutil/dovi_meta.h @@ -29,6 +29,7 @@ #include #include +#include "rational.h" /* * DOVI configuration @@ -67,4 +68,119 @@ typedef struct AVDOVIDecoderConfigurationRecord { */ AVDOVIDecoderConfigurationRecord *av_dovi_alloc(size_t *size); +/** + * Dolby Vision RPU data header. + */ +typedef struct AVDOVIRpuDataHeader { + uint8_t rpu_type; + uint16_t rpu_format; + uint8_t vdr_rpu_profile; + uint8_t vdr_rpu_level; + int chroma_resampling_explicit_filter_flag; + uint8_t coef_data_type; /* informative, lavc always converts to fixed */ + uint8_t coef_log2_denom; + uint8_t vdr_rpu_normalized_idc; + int bl_video_full_range_flag; + uint8_t bl_bit_depth; /* [8, 16] */ + uint8_t el_bit_depth; /* [8, 16] */ + uint8_t vdr_bit_depth; /* [8, 16] */ + int spatial_resampling_filter_flag; + int el_spatial_resampling_filter_flag; + int disable_residual_flag; +} AVDOVIRpuDataHeader; + +enum AVDOVIMappingMethod { + AV_DOVI_MAPPING_POLYNOMIAL = 0, + AV_DOVI_MAPPING_MMR = 1, +}; + +/** + * Coefficients of a piece-wise function. The pieces of the function span the + * value ranges between two adjacent pivot values. + */ +#define FF_DOVI_MAX_PIECES 8 +typedef struct AVDOVIReshapingCurve { + uint8_t num_pivots; /* [2, 9] */ + uint16_t pivots[FF_DOVI_MAX_PIECES + 1]; /* sorted ascending */ + enum AVDOVIMappingMethod mapping_idc[FF_DOVI_MAX_PIECES]; + /* AV_DOVI_MAPPING_POLYNOMIAL */ + uint8_t poly_order[FF_DOVI_MAX_PIECES]; /* [1, 2] */ + int64_t poly_coef[FF_DOVI_MAX_PIECES][3]; /* x^0, x^1, x^2 */ + /* AV_DOVI_MAPPING_MMR */ + uint8_t mmr_order[FF_DOVI_MAX_PIECES]; /* [1, 3] */ + int64_t mmr_constant[FF_DOVI_MAX_PIECES]; + int64_t mmr_coef[FF_DOVI_MAX_PIECES][3/* order - 1 */][7]; +} AVDOVIReshapingCurve; + +enum AVDOVINLQMethod { + AV_DOVI_NLQ_NONE = -1, + AV_DOVI_NLQ_LINEAR_DZ = 0, +}; + +/** + * Coefficients of the non-linear inverse quantization. For the interpretation + * of these, see ETSI GS CCM 001. + */ +typedef struct AVDOVINLQParams { + uint64_t nlq_offset; + uint64_t vdr_in_max; + /* AV_DOVI_NLQ_LINEAR_DZ */ + uint64_t linear_deadzone_slope; + uint64_t linear_deadzone_threshold; +} AVDOVINLQParams; + +/** + * Dolby Vision RPU data mapping parameters. + */ +typedef struct AVDOVIDataMapping { + uint8_t vdr_rpu_id; + uint8_t mapping_color_space; + uint8_t mapping_chroma_format_idc; + AVDOVIReshapingCurve curves[3]; /* per component */ + + /* Non-linear inverse quantization */ + enum AVDOVINLQMethod nlq_method_idc; + uint32_t num_x_partitions; + uint32_t num_y_partitions; + AVDOVINLQParams nlq[3]; /* per component */ +} AVDOVIDataMapping; + +typedef struct AVDOVIColorMetadata { + uint8_t dm_metadata_id; + int scene_refresh_flag; + + /** + * Coefficients of the custom Dolby Vision IPT-PQ matrices. These are to be + * used instead of the matrices indicated by the frame's colorspace tags. + * The output of rgb_to_lms_matrix is to be fed into a BT.2020 LMS->RGB + * matrix based on a Hunt-Pointer-Estevez transform, but without any + * crosstalk. (See the definition of the ICtCp colorspace for more + * information.) + */ + AVRational ycc_to_rgb_matrix[9]; /* before PQ linearization */ + AVRational ycc_to_rgb_offset[3]; /* input offset of neutral value */ + AVRational rgb_to_lms_matrix[9]; /* after PQ linearization */ + + /** + * Extra signal metadata (see Dolby patents for more info). + */ + uint16_t signal_eotf; + uint16_t signal_eotf_param0; + uint16_t signal_eotf_param1; + uint32_t signal_eotf_param2; + uint8_t signal_bit_depth; + uint8_t signal_color_space; + uint8_t signal_chroma_format; + uint8_t signal_full_range_flag; /* [0, 3] */ + uint16_t source_min_pq; + uint16_t source_max_pq; + uint16_t source_diagonal; +} AVDOVIColorMetadata; + +typedef struct AVDOVIMetadata { + AVDOVIRpuDataHeader header; + AVDOVIDataMapping mapping; + AVDOVIColorMetadata color; +} AVDOVIMetadata; + #endif /* AVUTIL_DOVI_META_H */ diff --git a/libavutil/frame.c b/libavutil/frame.c index 0912ad9131..8997c85e35 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -729,6 +729,7 @@ const char *av_frame_side_data_name(enum AVFrameSideDataType type) case AV_FRAME_DATA_FILM_GRAIN_PARAMS: return "Film grain parameters"; case AV_FRAME_DATA_DETECTION_BBOXES: return "Bounding boxes for object detection and classification"; case AV_FRAME_DATA_DOVI_RPU_BUFFER: return "Dolby Vision RPU Data"; + case AV_FRAME_DATA_DOVI_METADATA: return "Dolby Vision Metadata"; } return NULL; } diff --git a/libavutil/frame.h b/libavutil/frame.h index 3f295f6b9e..18e239f870 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -189,11 +189,18 @@ enum AVFrameSideDataType { AV_FRAME_DATA_DETECTION_BBOXES, /** - * Dolby Vision RPU data, suitable for passing to x265 + * Dolby Vision RPU raw data, suitable for passing to x265 * or other libraries. Array of uint8_t, with NAL emulation * bytes intact. */ AV_FRAME_DATA_DOVI_RPU_BUFFER, + + /** + * Parsed Dolby Vision metadata, suitable for passing to a software + * implementation. The payload is the AVDOVIMetadata struct defined in + * libavutil/dovi_meta.h. + */ + AV_FRAME_DATA_DOVI_METADATA, }; enum AVActiveFormatDescription { diff --git a/libavutil/version.h b/libavutil/version.h index 017fc277a6..678401fcf5 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 57 -#define LIBAVUTIL_VERSION_MINOR 10 +#define LIBAVUTIL_VERSION_MINOR 11 #define LIBAVUTIL_VERSION_MICRO 101 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \