From patchwork Mon Feb 14 23:44:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lance Wang X-Patchwork-Id: 34297 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6838:14aa:0:0:0:0 with SMTP id bz10csp5960319nkb; Mon, 14 Feb 2022 15:45:02 -0800 (PST) X-Google-Smtp-Source: ABdhPJxlkFS8KT5Pl0/XtiYrrojr22VgQzdsN/D4lx8nNfMkb1b02uhO+1PtUPSrdxahamLBbuCI X-Received: by 2002:aa7:ccce:: with SMTP id y14mr1275338edt.89.1644882302639; Mon, 14 Feb 2022 15:45:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1644882302; cv=none; d=google.com; s=arc-20160816; b=MeBhtNCVaobsW0vNs2RRD3hyto3/7GmqBDnfV0db176vZsBPOfnF2MIHd5ThI5/HZe hDK3adF42bRaqxZvt9b5UhJPlGPQtS3+GVMXj8USmAUrzCbMidlvt/xwETxMN2M+1Txe ARo9cWhWV/3O5WY4xW8+OGFouJhvGL6EhfuajI0Sh+FGM4EkULhIposxFREfzZM7ujqq afOQOLmPA+VbBx+xMViXvK3zUrP0Ha3sZDM6Rm0oG2GiIdXIPQ6zybBaH9nz1xGdswAu YfazcjeuFjSsyBF/kQZRq5FUR0meSbVqnsoxpBxc1NBO9H6m/AEoysceXh/7uh/Xruvk TjDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to; bh=rRTS6cNCP3ZiQdkGK9AloDG5mHyAOWNCloLmkDIqTbo=; b=anwj6P+KrbarvugbEzKiMdKJGRG/4L36pwZx9KgfFIIdkoJeE+Q9NIeKrsVzMoL/d2 jUE0+FeNJ1Rq66xsjfEcMJNJ9/9dh/51fBwQr8/3TXnYS8dW1fxvt7cFE2pGMV4NTDLs juqre6RRp2yfKvfu2etY8H8EuuMHEVbT51f1bKF44ilxHQQPbVvpPcMXcHvI6DVSkYua FFr2/p2gddGb8bSks6+1soTg+lrE/joubeS6TohU2TwsgtWX9pdqXlhcFiiOxzM0/awf SqrybKzVPnq7E0KDqQmnvNGTKb/M0DxbmQfObXd8U3Y0S3lwL5op23CM87ou4R9COPtL q/BQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=pRwrv+5s; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h23si3474434eja.3.2022.02.14.15.44.38; Mon, 14 Feb 2022 15:45:02 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=pRwrv+5s; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3F19568B193; Tue, 15 Feb 2022 01:44:34 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7BC0168AE70 for ; Tue, 15 Feb 2022 01:44:27 +0200 (EET) Received: by mail-pl1-f173.google.com with SMTP id l8so5742786pls.7 for ; Mon, 14 Feb 2022 15:44:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id; bh=/dF0pKmyDccJf0+HFvG3blglC43HpyDRZv4EmOL0QUI=; b=pRwrv+5s6l4EhB+DTjvIB/hSKKwAv1MZKoS8XKn2K3DfalzTqfi2Dv7dAbzegSNfQd FMw6wp8c6gANlJ+gkmZVDupQpPf2cJtwz1AiJBMPAw4GQvrNYWScPr83K59anIpFTXKk 9GzcorlBag39uwWxN6EiBlnm83OhwF/fat1swIc8iKO4gY1NiVoF1/cYe63o1r2pkVvy sX+dCoV+skB+CiCUjOCQrzwd0O5u3dIs3oZWlYp1DAvr6z/cl8hfBhYeZ39eeG1SkBj2 zgznV3Pz0eWT1iNbxaEcQn1Yc7ykrTldVxfoELp1QAIIcx5qQdpUudpm2swsZYCldw9O 8fWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=/dF0pKmyDccJf0+HFvG3blglC43HpyDRZv4EmOL0QUI=; b=7FEb5UhQWLdajVtHYCRau1QE7IiLDu4znAMm9jHag1Ab6qfeQWnuQsWR/k63KYP3Z9 kwbKdBfInynDKSgvh66h7LurMwzchliUzO8mUZKOz/xh2AjybK4aoAjpSBUOj+snwIVA jt4tBDLu1yQ1BmTSv5I+wCcvky4b5NPPyVm3mVL8cs5BLgtf72tXimofm/55XAC9i5Hy Iiecv4zgm7lMIGFTLi/+ZGnwP/BOsj0w+tVAPiZ7cV6ek3lo+zvDAJMQdjvuF2Ib35LV BHwxABbKS5VbpoBYV6UUy2ZBCTZMIUvfQ/J/NlFCgiwy50kZtBgw2QwdMrFEfpvH80li ezhg== X-Gm-Message-State: AOAM533+4mAPFyDM3j1FRC86c+ewoG2WM3DO/GDoLjtKLPGgmjCbwrQP KQr8CeLUiG3nakiIwjdgnKTYv2Prsx4= X-Received: by 2002:a17:902:bd81:: with SMTP id q1mr1493578pls.100.1644882265298; Mon, 14 Feb 2022 15:44:25 -0800 (PST) Received: from vpn2.localdomain ([161.117.202.209]) by smtp.gmail.com with ESMTPSA id mi7sm492304pjb.21.2022.02.14.15.44.24 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 14 Feb 2022 15:44:24 -0800 (PST) From: lance.lmwang@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 15 Feb 2022 07:44:17 +0800 Message-Id: <1644882260-8559-1-git-send-email-lance.lmwang@gmail.com> X-Mailer: git-send-email 1.8.3.1 Subject: [FFmpeg-devel] [PATCH 1/4] avutil: add support for CUVA HDR Vivid metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Limin Wang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: rP8wamaBpRWc From: Limin Wang Signed-off-by: Limin Wang --- libavutil/Makefile | 2 + libavutil/frame.c | 1 + libavutil/frame.h | 7 + libavutil/hdr_dynamic_vivid_metadata.c | 47 ++++++ libavutil/hdr_dynamic_vivid_metadata.h | 285 +++++++++++++++++++++++++++++++++ libavutil/version.h | 2 +- 6 files changed, 343 insertions(+), 1 deletion(-) create mode 100644 libavutil/hdr_dynamic_vivid_metadata.c create mode 100644 libavutil/hdr_dynamic_vivid_metadata.h diff --git a/libavutil/Makefile b/libavutil/Makefile index d17876d..a8d7587 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -34,6 +34,7 @@ HEADERS = adler32.h \ frame.h \ hash.h \ hdr_dynamic_metadata.h \ + hdr_dynamic_vivid_metadata.h \ hmac.h \ hwcontext.h \ hwcontext_cuda.h \ @@ -130,6 +131,7 @@ OBJS = adler32.o \ frame.o \ hash.o \ hdr_dynamic_metadata.o \ + hdr_dynamic_vivid_metadata.o \ hmac.o \ hwcontext.o \ imgutils.o \ diff --git a/libavutil/frame.c b/libavutil/frame.c index 8997c85..b035e28 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -723,6 +723,7 @@ const char *av_frame_side_data_name(enum AVFrameSideDataType type) case AV_FRAME_DATA_SPHERICAL: return "Spherical Mapping"; case AV_FRAME_DATA_ICC_PROFILE: return "ICC profile"; case AV_FRAME_DATA_DYNAMIC_HDR_PLUS: return "HDR Dynamic Metadata SMPTE2094-40 (HDR10+)"; + case AV_FRAME_DATA_DYNAMIC_HDR_VIVID: return "HDR Dynamic Metadata CUVA 005.1 2021 (Vivid)"; case AV_FRAME_DATA_REGIONS_OF_INTEREST: return "Regions Of Interest"; case AV_FRAME_DATA_VIDEO_ENC_PARAMS: return "Video encoding parameters"; case AV_FRAME_DATA_SEI_UNREGISTERED: return "H.26[45] User Data Unregistered SEI message"; diff --git a/libavutil/frame.h b/libavutil/frame.h index 18e239f..32cde3c 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -158,6 +158,13 @@ enum AVFrameSideDataType { AV_FRAME_DATA_DYNAMIC_HDR_PLUS, /** + * HDR Vivid dynamic metadata associated with a video frame. The payload is + * an AVDynamicHDRVivid type and contains information for color + * volume transform - CUVA 005.1-2021. + */ + AV_FRAME_DATA_DYNAMIC_HDR_VIVID, + + /** * Regions Of Interest, the data is an array of AVRegionOfInterest type, the number of * array element is implied by AVFrameSideData.size / AVRegionOfInterest.self_size. */ diff --git a/libavutil/hdr_dynamic_vivid_metadata.c b/libavutil/hdr_dynamic_vivid_metadata.c new file mode 100644 index 0000000..32da01f --- /dev/null +++ b/libavutil/hdr_dynamic_vivid_metadata.c @@ -0,0 +1,47 @@ +/** + * Copyright (c) 2021 Limin Wang + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "hdr_dynamic_vivid_metadata.h" +#include "mem.h" + +AVDynamicHDRVivid *av_dynamic_hdr_vivid_alloc(size_t *size) +{ + AVDynamicHDRVivid *hdr_vivid = av_mallocz(sizeof(AVDynamicHDRVivid)); + if (!hdr_vivid) + return NULL; + + if (size) + *size = sizeof(*hdr_vivid); + + return hdr_vivid; +} + +AVDynamicHDRVivid *av_dynamic_hdr_vivid_create_side_data(AVFrame *frame) +{ + AVFrameSideData *side_data = av_frame_new_side_data(frame, + AV_FRAME_DATA_DYNAMIC_HDR_VIVID, + sizeof(AVDynamicHDRVivid)); + if (!side_data) + return NULL; + + memset(side_data->data, 0, sizeof(AVDynamicHDRVivid)); + + return (AVDynamicHDRVivid *)side_data->data; +} diff --git a/libavutil/hdr_dynamic_vivid_metadata.h b/libavutil/hdr_dynamic_vivid_metadata.h new file mode 100644 index 0000000..832f7f9 --- /dev/null +++ b/libavutil/hdr_dynamic_vivid_metadata.h @@ -0,0 +1,285 @@ +/* + * Copyright (c) 2021 Limin Wang + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_HDR_DYNAMIC_VIVID_METADATA_H +#define AVUTIL_HDR_DYNAMIC_VIVID_METADATA_H + +#include "frame.h" +#include "rational.h" + +/** + * Color tone mapping parameters at a processing window in a dynamic metadata for + * CUVA 005.1:2021. + */ +typedef struct AVHDRVividColorToneMappingParams { + /** + * The nominal maximum display luminance of the targeted system display, + * in multiples of 1.0/4095 candelas per square metre. The value shall be in + * the range of 0.0 to 1.0, inclusive. + */ + AVRational targeted_system_display_maximum_luminance; + + /** + * This flag indicates that transfer the base paramter(for value of 1) + */ + int base_enable_flag; + + /** + * base_param_m_p in the base parameter, + * in multiples of 1.0/16383. The value shall be in + * the range of 0.0 to 1.0, inclusive. + */ + AVRational base_param_m_p; + + /** + * base_param_m_m in the base parameter, + * in multiples of 1.0/10. The value shall be in + * the range of 0.0 to 6.3, inclusive. + */ + AVRational base_param_m_m; + + /** + * base_param_m_a in the base parameter, + * in multiples of 1.0/1023. The value shall be in + * the range of 0.0 to 1.0 inclusive. + */ + AVRational base_param_m_a; + + /** + * base_param_m_b in the base parameter, + * in multiples of 1/1023. The value shall be in + * the range of 0.0 to 1.0, inclusive. + */ + AVRational base_param_m_b; + + /** + * base_param_m_n in the base parameter, + * in multiples of 1.0/10. The value shall be in + * the range of 0.0 to 6.3, inclusive. + */ + AVRational base_param_m_n; + + /** + * indicates k1_0 in the base parameter, + * base_param_k1 <= 1: k1_0 = base_param_k1 + * base_param_k1 > 1: reserved + */ + int base_param_k1; + + /** + * indicates k2_0 in the base parameter, + * base_param_k2 <= 1: k2_0 = base_param_k2 + * base_param_k2 > 1: reserved + */ + int base_param_k2; + + /** + * indicates k3_0 in the base parameter, + * base_param_k3 == 1: k3_0 = base_param_k3 + * base_param_k3 == 2: k3_0 = maximum_maxrgb + * base_param_k3 > 2: reserved + */ + int base_param_k3; + + /** + * This flag indicates that delta mode of base paramter(for value of 1) + */ + int base_param_Delta_enable_mode; + + /** + * base_param_Delta in the base parameter, + * in multiples of 1.0/127. The value shall be in + * the range of 0.0 to 1.0, inclusive. + */ + AVRational base_param_Delta; + + /** + * indicates 3Spline_flag in the base parameter, + * This flag indicates that transfer three Spline of base paramter(for value of 1) + */ + int three_Spline_flag; + + /** + * The number of three Spline. The value shall be in the range + * of 1 to 2, inclusive. + */ + int three_Spline_num; + + /** + * The mode of three Spline. the value shall be in the range + * of 0 to 3, inclusive. + */ + int three_Spline_TH_mode; + + /** + * three_Spline_TH_enable MB is in the range of 0.0 to 1.0, inclusive + * and in multiples of 1.0/255. + * + */ + AVRational three_Spline_TH_enable_MB; + + /** + * 3Spline_TH_enable of three Spline. + * The value shall be in the range of 0.0 to 1.0, inclusive. + * and in multiples of 1.0/4095. + */ + AVRational three_Spline_TH; + + /** + * 3Spline_TH_Delta1 of three Spline. + * The value shall be in the range of 0.0 to 0.25, inclusive, + * and in multiples of 0.25/1023. + */ + AVRational three_Spline_TH_Delta1; + + /** + * 3Spline_TH_Delta2 of three Spline. + * The value shall be in the range of 0.0 to 0.25, inclusive, + * and in multiples of 0.25/1023. + */ + AVRational three_Spline_TH_Delta2; + + /** + * 3Spline_enable_Strength of three Spline. + * The value shall be in the range of 0.0 to 1, inclusive, + * and in multiples of 1.0/255. + */ + AVRational three_Spline_enable_Strength; +} AVHDRVividColorToneMappingParams; + + +/** + * Color transform parameters at a processing window in a dynamic metadata for + * CUVA 005.1:2021. + */ +typedef struct AVHDRVividColorTransformParams { + /** + * Indicates the minimum brightness of the displayed content. + * The values should be in the range of 0.0 to 1.0, + * inclusive and in multiples of 1/4095. + */ + AVRational minimum_maxrgb; + + /** + * Indicates the average brightness of the displayed content. + * The values should be in the range of 0.0 to 1.0, + * inclusive and in multiples of 1/4095. + */ + AVRational average_maxrgb; + + /** + * Indicates the variance brightness of the displayed content. + * The values should be in the range of 0.0 to 1.0, + * inclusive and in multiples of 1/4095. + */ + AVRational variance_maxrgb; + + /** + * Indicates the maximum brightness of the displayed content. + * The values should be in the range of 0.0 to 1.0, inclusive + * and in multiples of 1/4095. + */ + AVRational maximum_maxrgb; + + /** + * This flag indicates that the metadata for the tone mapping function in + * the processing window is present (for value of 1). + */ + int tone_mapping_mode_flag; + + /** + * The number of tone mapping param. The value shall be in the range + * of 1 to 2, inclusive. + */ + int tone_mapping_param_num; + + /** + * The color tone mapping parameters. + */ + AVHDRVividColorToneMappingParams tm_params[2]; + + /** + * This flag indicates that the metadata for the color saturation mapping in + * the processing window is present (for value of 1). + */ + int color_saturation_mapping_flag; + + /** + * The number of color saturation param. The value shall be in the range + * of 0 to 7, inclusive. + */ + int color_saturation_num; + + /** + * Indicates the color correction strength parameter. + * The values should be in the range of 0.0 to 2.0, inclusive + * and in multiples of 1/128. + */ + AVRational color_saturation_gain[8]; +} AVHDRVividColorTransformParams; + +/** + * This struct represents dynamic metadata for color volume transform - + * CUVA 005.1:2021 standard + * + * To be used as payload of a AVFrameSideData or AVPacketSideData with the + * appropriate type. + * + * @note The struct should be allocated with + * av_dynamic_hdr_vivid_alloc() and its size is not a part of + * the public ABI. + */ +typedef struct AVDynamicHDRVivid { + /** + * The system start code. The value shall be set to 0x01. + */ + uint8_t system_start_code; + + /** + * The number of processing windows. The value shall be set to 0x01 + * if the system_start_code is 0x01. + */ + uint8_t num_windows; + + /** + * The color transform parameters for every processing window. + */ + AVHDRVividColorTransformParams params[3]; +} AVDynamicHDRVivid; + +/** + * Allocate an AVDynamicHDRVivid structure and set its fields to + * default values. The resulting struct can be freed using av_freep(). + * + * @return An AVDynamicHDRVivid filled with default values or NULL + * on failure. + */ +AVDynamicHDRVivid *av_dynamic_hdr_vivid_alloc(size_t *size); + +/** + * Allocate a complete AVDynamicHDRVivid and add it to the frame. + * @param frame The frame which side data is added to. + * + * @return The AVDynamicHDRVivid structure to be filled by caller or NULL + * on failure. + */ +AVDynamicHDRVivid *av_dynamic_hdr_vivid_create_side_data(AVFrame *frame); + +#endif /* AVUTIL_HDR_DYNAMIC_VIVID_METADATA_H */ diff --git a/libavutil/version.h b/libavutil/version.h index a9c9bc6..c80e9d7 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 57 -#define LIBAVUTIL_VERSION_MINOR 21 +#define LIBAVUTIL_VERSION_MINOR 22 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \