From patchwork Tue Dec 7 19:58:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 32112 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6738341iog; Tue, 7 Dec 2021 11:59:24 -0800 (PST) X-Google-Smtp-Source: ABdhPJzFKaqA4gZqJ0t05gZ3RYVgn5lWEjXiZCjPiyiBuJPtrQ9tnYObFIGoiOcntsVOtvrIFQzY X-Received: by 2002:a50:c38c:: with SMTP id h12mr12678793edf.72.1638907163857; Tue, 07 Dec 2021 11:59:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1638907163; cv=none; d=google.com; s=arc-20160816; b=Qd5TyYYX/EODjzcgEstk8nvxmaqTBNYa3qLpq3Mi56/spCz0UY5Vgi/QKpzJP+8OUp bH5RSL9EUQ+Ze1QpbVyHZ2jqgLNXbg36OKOlI0TH/Xmuwb7AlBhYIgQ3fVSL7h9d3Emy Az+POV7eQkv6RpXLgCn/QGFJ+jT1g/h9YQQcmp/I+CE/LwQHMTHNsrvf3AAq++aySsa1 Yq0wl8T0i7JV3Wk+bDsBTPVCTllPM/mixghcA0f5AGgzjhpwKyeBXmdtqZXmmTAeS8y5 VHcThkgq1pDPkaww2jJj/FKpSZVe/HNT7niAe2AjpNNBTvmPYj0B1RJBlDk27f5LBcbW 0olA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=ehee6v+IHLawd+fOK2PXYyG11zDo56XyR8EyxavbpL0=; b=UB7ZbeimmsEMWMfPhyLdOZDxIVjffsh2vb1KJKUxOvmDjaqIWQk8xnZXP7VFZq9BEm WOlmws40hRw/5PJKeIDlVOZEWZWradpe5ROtw0YzbsA1YtM1geSk0ofLRTociQqxO3dh 2gC0Ty1gxZkQbw2BtQyr5aPMMDMm+tDU+Z7wG7yFCyiTrJj7Jta8Jd+dgDMbTDsMI73V funj60Iqymn7GFjWMuAUWLSV0WgvdX5bpwiTZYsXIYi7MKRe08TgUDSkuBEx2oSfPZwu KxorDg1T4YGzmj+2Ok88FUZQKvz9r7eAcjXgDyXt6i5dDqWdUtGzxvMb9URGSmr1Jub1 NKCA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=c8jJekSk; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id j23si698730ejs.4.2021.12.07.11.59.18; Tue, 07 Dec 2021 11:59:23 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=c8jJekSk; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2F6F668AF1F; Tue, 7 Dec 2021 21:59:14 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BC69468AD15 for ; Tue, 7 Dec 2021 21:59:06 +0200 (EET) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id BD49E4910E; Tue, 7 Dec 2021 20:59:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1638907145; bh=yc+kbTae7DNgVUEqmdiPIBvrcAEKREkzbLICm2wqYHg=; h=From:To:Cc:Subject:Date:From; b=c8jJekSkaj9STPTE4zoQihTn1NSG5Ep8XNAsne05AnKvT4ac1ZG++6Zbv7EM1A4ce AenJ7+P4ehj7TUTNDBEYuBpzBBir9CMwy47vD5a8RP7Pgt9BXeYL/4rHhbZPE4UCJF qdgdzJlfamTgUO0pdsceJ7mOQfMIpsSgq9GORc9U= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 7 Dec 2021 20:58:59 +0100 Message-Id: <20211207195902.29995-1-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/4] lavu/frame: Add Dolby Vision metadata side data type X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: KG8wj+ibgrzz From: Niklas Haas Signed-off-by: Niklas Haas --- doc/APIchanges | 3 ++ libavutil/dovi_meta.c | 23 ++++++++ libavutil/dovi_meta.h | 121 ++++++++++++++++++++++++++++++++++++++++++ libavutil/frame.c | 1 + libavutil/frame.h | 9 +++- libavutil/version.h | 2 +- 6 files changed, 157 insertions(+), 2 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index 2914ad6734..422874e3b9 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -14,6 +14,9 @@ libavutil: 2021-04-27 API changes, most recent first: +2021-12-06 - xxxxxxxxxx - lavu 57.11.100 - frame.h + Add AV_FRAME_DATA_DOVI_RESHAPING. + 2021-11-xx - xxxxxxxxxx - lavfi 8.19.100 - avfilter.h Add AVFILTER_FLAG_METADATA_ONLY. diff --git a/libavutil/dovi_meta.c b/libavutil/dovi_meta.c index 7bd08f6c54..e2ef8ac3a4 100644 --- a/libavutil/dovi_meta.c +++ b/libavutil/dovi_meta.c @@ -33,3 +33,26 @@ AVDOVIDecoderConfigurationRecord *av_dovi_alloc(size_t *size) return dovi; } + +/* based on guesswork, see mkvtoolnix and dovi_tool */ +int av_dovi_profile(const AVDOVIRpuDataHeader *hdr) +{ + switch (hdr->vdr_rpu_profile) { + case 0: + if (hdr->bl_video_full_range_flag) + return 5; + break; + case 1: + if (hdr->el_spatial_resampling_filter_flag && !hdr->disable_residual_flag) { + if (hdr->vdr_bit_depth == 12) { + return 7; + } else { + return 4; + } + } else { + return 8; + } + } + + return 0; /* unknown */ +} diff --git a/libavutil/dovi_meta.h b/libavutil/dovi_meta.h index 299911d434..02c414752c 100644 --- a/libavutil/dovi_meta.h +++ b/libavutil/dovi_meta.h @@ -29,6 +29,7 @@ #include #include +#include "rational.h" /* * DOVI configuration @@ -67,4 +68,124 @@ typedef struct AVDOVIDecoderConfigurationRecord { */ AVDOVIDecoderConfigurationRecord *av_dovi_alloc(size_t *size); +/** + * Dolby Vision RPU data header. + */ +typedef struct AVDOVIRpuDataHeader { + uint8_t rpu_type; + uint16_t rpu_format; + uint8_t vdr_rpu_profile; + uint8_t vdr_rpu_level; + int chroma_resampling_explicit_filter_flag; + uint8_t coef_data_type; /* informative, lavc always converts to fixed */ + uint8_t coef_log2_denom; + uint8_t vdr_rpu_normalized_idc; + int bl_video_full_range_flag; + uint8_t bl_bit_depth; /* [8, 16] */ + uint8_t el_bit_depth; /* [8, 16] */ + uint8_t vdr_bit_depth; /* [8, 16] */ + int spatial_resampling_filter_flag; + int el_spatial_resampling_filter_flag; + int disable_residual_flag; +} AVDOVIRpuDataHeader; + +/** + * Return the Dolby Vision profile number derived from a given RPU data header, + * or 0 for unknown/unrecognized profiles. + */ +int av_dovi_profile(const AVDOVIRpuDataHeader *hdr); + +enum AVDOVIMappingMethod { + AV_DOVI_MAPPING_POLYNOMIAL = 0, + AV_DOVI_MAPPING_MMR = 1, +}; + +/** + * Coefficients of a piece-wise function. The pieces of the function span the + * value ranges between two adjacent pivot values. + */ +#define FF_DOVI_MAX_PIECES 8 +typedef struct AVDOVIReshapingCurve { + uint8_t num_pivots; /* [2, 9], or 0 to skip this comp */ + uint16_t pivots[FF_DOVI_MAX_PIECES + 1]; /* sorted ascending */ + enum AVDOVIMappingMethod mapping_idc[FF_DOVI_MAX_PIECES]; + /* AV_DOVI_MAPPING_POLYNOMIAL */ + uint8_t poly_order[FF_DOVI_MAX_PIECES]; /* [1, 2] */ + int64_t poly_coef[FF_DOVI_MAX_PIECES][3]; /* x^0, x^1, x^2 */ + /* AV_DOVI_MAPPING_MMR */ + uint8_t mmr_order[FF_DOVI_MAX_PIECES]; /* [1, 3] */ + int64_t mmr_constant[FF_DOVI_MAX_PIECES]; + int64_t mmr_coef[FF_DOVI_MAX_PIECES][3/* order - 1 */][7]; +} AVDOVIReshapingCurve; + +enum AVDOVINLQMethod { + AV_DOVI_NLQ_LINEAR_DZ = 0, +}; + +/** + * Coefficients of the non-linear inverse quantization. For the interpretation + * of these, see ETSI GS CCM 001. + */ +typedef struct AVDOVINLQParams { + uint64_t nlq_offset; + uint64_t vdr_in_max; + /* AV_DOVI_NLQ_LINEAR_DZ */ + uint64_t linear_deadzone_slope; + uint64_t linear_deadzone_threshold; +} AVDOVINLQParams; + +/** + * Dolby Vision RPU data mapping parameters. + */ +typedef struct AVDOVIDataMapping { + uint8_t vdr_rpu_id; + uint8_t mapping_color_space; + uint8_t mapping_chroma_format_idc; + AVDOVIReshapingCurve curves[3]; /* per component */ + + /* Non-linear inverse quantization */ + enum AVDOVINLQMethod nlq_method_idc; + uint32_t num_x_partitions; + uint32_t num_y_partitions; + AVDOVINLQParams nlq[3]; /* per component */ +} AVDOVIDataMapping; + +typedef struct AVDOVIColorMetadata { + uint8_t dm_metadata_id; + int scene_refresh_flag; + + /** + * Coefficients of the custom Dolby Vision IPT-PQ matrices. These are to be + * used instead of the matrices indicated by the frame's colorspace tags. + * The output of rgb_to_lms_matrix is to be fed into a BT.2020 LMS->RGB + * matrix based on a Hunt-Pointer-Estevez transform, but without any + * crosstalk. (See the definition of the ICtCp colorspace for more + * information.) + */ + AVRational ycc_to_rgb_matrix[9]; /* before PQ linearization */ + AVRational ycc_to_rgb_offset[3]; /* input offset of neutral value */ + AVRational rgb_to_lms_matrix[9]; /* after PQ linearization */ + + /** + * Extra signal metadata (see Dolby patents for more info). + */ + uint16_t signal_eotf; + uint16_t signal_eotf_param0; + uint16_t signal_eotf_param1; + uint32_t signal_eotf_param2; + uint8_t signal_bit_depth; + uint8_t signal_color_space; + uint8_t signal_chroma_format; + uint8_t signal_full_range_flag; /* [0, 3] */ + uint16_t source_min_pq; + uint16_t source_max_pq; + uint16_t source_diagonal; +} AVDOVIColorMetadata; + +typedef struct AVDOVIMetadata { + AVDOVIRpuDataHeader header; + AVDOVIDataMapping mapping; + AVDOVIColorMetadata color; +} AVDOVIMetadata; + #endif /* AVUTIL_DOVI_META_H */ diff --git a/libavutil/frame.c b/libavutil/frame.c index 0912ad9131..8997c85e35 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -729,6 +729,7 @@ const char *av_frame_side_data_name(enum AVFrameSideDataType type) case AV_FRAME_DATA_FILM_GRAIN_PARAMS: return "Film grain parameters"; case AV_FRAME_DATA_DETECTION_BBOXES: return "Bounding boxes for object detection and classification"; case AV_FRAME_DATA_DOVI_RPU_BUFFER: return "Dolby Vision RPU Data"; + case AV_FRAME_DATA_DOVI_METADATA: return "Dolby Vision Metadata"; } return NULL; } diff --git a/libavutil/frame.h b/libavutil/frame.h index 3f295f6b9e..18e239f870 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -189,11 +189,18 @@ enum AVFrameSideDataType { AV_FRAME_DATA_DETECTION_BBOXES, /** - * Dolby Vision RPU data, suitable for passing to x265 + * Dolby Vision RPU raw data, suitable for passing to x265 * or other libraries. Array of uint8_t, with NAL emulation * bytes intact. */ AV_FRAME_DATA_DOVI_RPU_BUFFER, + + /** + * Parsed Dolby Vision metadata, suitable for passing to a software + * implementation. The payload is the AVDOVIMetadata struct defined in + * libavutil/dovi_meta.h. + */ + AV_FRAME_DATA_DOVI_METADATA, }; enum AVActiveFormatDescription { diff --git a/libavutil/version.h b/libavutil/version.h index 017fc277a6..678401fcf5 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 57 -#define LIBAVUTIL_VERSION_MINOR 10 +#define LIBAVUTIL_VERSION_MINOR 11 #define LIBAVUTIL_VERSION_MICRO 101 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \