From patchwork Thu Dec 20 19:07:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mohammad Izadi X-Patchwork-Id: 11491 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 106C044E2F6 for ; Thu, 20 Dec 2018 21:14:48 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 39AF968A49E; Thu, 20 Dec 2018 21:14:48 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f195.google.com (mail-pf1-f195.google.com [209.85.210.195]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A6BCB68A26E for ; Thu, 20 Dec 2018 21:14:41 +0200 (EET) Received: by mail-pf1-f195.google.com with SMTP id q1so1370488pfi.5 for ; Thu, 20 Dec 2018 11:14:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nRt6BflaJuuYw5ro6V/3jLjT0Do+9tr6sSwJgsWuX5M=; b=QQzBwFTNpLFGNKkj2dk9nzuT8K4B+nFN184fwVajLcDwRj14i40zKTj+renb0QQOOC GvdK4fB2fOLdnS8OtK6KoW5tLWwLkmHyg9KP8IEeSiMOAf2Rup2F9twRJ06D4lhEsEmN 0+4P3GckCx7dUdoL1Fo7DBPu8AlpZ4dGCs3ZoLpdzwSfTCLwD94i/3oEo1tRma3spZQQ Eo7qTF5dBIErIU063SWZbK1rOxNOY22Qt4RFNkYfcI2U6dtJRzHgzT/5OfrzQ1RHnu4Z Pg4Zsq0lsIxaa2SyMaF3AFqFU36VSb3empRl8x26mteDQK04TdBHo0bImxxrwNJgtc9d HkMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nRt6BflaJuuYw5ro6V/3jLjT0Do+9tr6sSwJgsWuX5M=; b=qeYSn2BNFXNFPM9buXcWCuItCL9682iBlRu1MachnCWTbhcyzDGzVVzHCOD9fIPWoF TwfCbx1EtqGgGg91b5ZXiUMEewuZSXzN5xrea+wU8/0BOvhwmzeHiv+RQ53uQkPy64bn guou45LLSAwzjPhbSlji3kdVrKnZlOagJX2Q2m1CCKLkEL76KssXBBfeYZUD0UIyPeJH hGkFgw5smuiGvnEn/G9nZIUrSnP7HQvedVrRqrn87Os1u4oz1nBOj1ZNI3UVnPCywYdN BvHEYyqXY4Mkv6xGENl7ghNCTOSpYxvh1XEXw9i7pFQeZ2jQgDj+KVUcdXOyyOaWeEip 0W8g== X-Gm-Message-State: AA+aEWYCluYjcfn7xjpG0faONKMYJoP9RJrLQHGZV8eqmyAWUCsBLlsq GLsstqQifkuHNinz08cBfMVDuYLR0Q== X-Google-Smtp-Source: AFSGD/XMyi8GEKXjfw0EOhBhpFIeWZz/o2DRc9dnOjTe/VVkoQzT7SVlqU0/K0JLvi9J00vIsrOF5g== X-Received: by 2002:a63:6cc:: with SMTP id 195mr24257441pgg.401.1545332873951; Thu, 20 Dec 2018 11:07:53 -0800 (PST) Received: from izadi.mtv.corp.google.com ([2620:0:1000:4011:3bcf:9bbf:ad2c:87b2]) by smtp.gmail.com with ESMTPSA id u87sm47497588pfi.2.2018.12.20.11.07.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Dec 2018 11:07:52 -0800 (PST) From: Mohammad Izadi To: ffmpeg-devel@ffmpeg.org Date: Thu, 20 Dec 2018 11:07:43 -0800 Message-Id: <20181220190743.251513-1-moh.izadi@gmail.com> X-Mailer: git-send-email 2.20.1.415.g653613c723-goog In-Reply-To: References: MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] Add HDR dynamic metadata struct (for SMPTE 2094-40) to libavutil. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Mohammad Izadi Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Mohammad Izadi The dynamic metadata contains data for color volume transform - application 4 of SMPTE 2094-40:2016 standard. The data comes from HEVC in the SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35. I'll add support to HEVC in a follow-up. --- libavutil/Makefile | 2 + libavutil/frame.c | 1 + libavutil/frame.h | 7 + libavutil/hdr_dynamic_metadata.c | 47 +++++ libavutil/hdr_dynamic_metadata.h | 343 +++++++++++++++++++++++++++++++ libavutil/version.h | 4 +- 6 files changed, 402 insertions(+), 2 deletions(-) create mode 100644 libavutil/hdr_dynamic_metadata.c create mode 100644 libavutil/hdr_dynamic_metadata.h diff --git a/libavutil/Makefile b/libavutil/Makefile index caddc7e155..7dcb92b06b 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -31,6 +31,7 @@ HEADERS = adler32.h \ file.h \ frame.h \ hash.h \ + hdr_dynamic_metadata.h \ hmac.h \ hwcontext.h \ hwcontext_cuda.h \ @@ -119,6 +120,7 @@ OBJS = adler32.o \ fixed_dsp.o \ frame.o \ hash.o \ + hdr_dynamic_metadata.o \ hmac.o \ hwcontext.o \ imgutils.o \ diff --git a/libavutil/frame.c b/libavutil/frame.c index 9b3fb13e68..34a6210d9e 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -840,6 +840,7 @@ const char *av_frame_side_data_name(enum AVFrameSideDataType type) case AV_FRAME_DATA_QP_TABLE_PROPERTIES: return "QP table properties"; case AV_FRAME_DATA_QP_TABLE_DATA: return "QP table data"; #endif + case AV_FRAME_DATA_DYNAMIC_HDR_PLUS: return "HDR Dynamic Metadata SMPTE2094-40 (HDR10+)"; } return NULL; } diff --git a/libavutil/frame.h b/libavutil/frame.h index 66f27f44bd..582ac470b2 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -166,6 +166,13 @@ enum AVFrameSideDataType { * function in libavutil/timecode.c. */ AV_FRAME_DATA_S12M_TIMECODE, + + /** + * HDR dynamic metadata associated with a video frame. The payload is + * an AVDynamicHDRPlus type and contains information for color + * volume transform - application 4 of SMPTE 2094-40:2016 standard. + */ + AV_FRAME_DATA_DYNAMIC_HDR_PLUS, }; enum AVActiveFormatDescription { diff --git a/libavutil/hdr_dynamic_metadata.c b/libavutil/hdr_dynamic_metadata.c new file mode 100644 index 0000000000..7e4d7dec10 --- /dev/null +++ b/libavutil/hdr_dynamic_metadata.c @@ -0,0 +1,47 @@ +/** + * Copyright (c) 2018 Mohammad Izadi + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "hdr_dynamic_metadata.h" +#include "mem.h" + +AVDynamicHDRPlus *av_dynamic_hdr_plus_alloc(size_t *size) +{ + AVDynamicHDRPlus *hdr_plus = av_mallocz(sizeof(AVDynamicHDRPlus)); + if (!hdr_plus) + return NULL; + + if (size) + *size = sizeof(*hdr_plus); + + return hdr_plus; +} + +AVDynamicHDRPlus *av_dynamic_hdr_plus_create_side_data(AVFrame *frame) +{ + AVFrameSideData *side_data = av_frame_new_side_data(frame, + AV_FRAME_DATA_DYNAMIC_HDR_PLUS, + sizeof(AVDynamicHDRPlus)); + if (!side_data) + return NULL; + + memset(side_data->data, 0, sizeof(AVDynamicHDRPlus)); + + return (AVDynamicHDRPlus *)side_data->data; +} diff --git a/libavutil/hdr_dynamic_metadata.h b/libavutil/hdr_dynamic_metadata.h new file mode 100644 index 0000000000..9ca6321c76 --- /dev/null +++ b/libavutil/hdr_dynamic_metadata.h @@ -0,0 +1,343 @@ +/* + * Copyright (c) 2018 Mohammad Izadi + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_HDR_DYNAMIC_METADATA_H +#define AVUTIL_HDR_DYNAMIC_METADATA_H + +#include "frame.h" +#include "rational.h" + +/** + * Option for overlapping elliptical pixel selectors in an image. + */ +enum AVHDRPlusOverlapProcessOption { + AV_HDR_PLUS_OVERLAP_PROCESS_WEIGHTED_AVERAGING = 0, + AV_HDR_PLUS_OVERLAP_PROCESS_LAYERING = 1, +}; + +/** + * Represents the percentile at a specific percentage in + * a distribution. + */ +typedef struct AVHDRPlusPercentile { + /** + * The percentage value corresponding to a specific percentile linearized + * RGB value in the processing window in the scene. The value shall be in + * the range of 0 to100, inclusive. + */ + uint8_t percentage; + /** + * The linearized maxRGB value at a specific percentile in the processing + * window in the scene. The value shall be in the range of 0 to 1, inclusive + * and in multiples of 0.00001. + */ + AVRational percentile; +} AVHDRPlusPercentile; + +/** + * Color transform parameters at a processing window in a dynamic metadata for + * SMPTE 2094-40. + */ +typedef struct AVHDRPlusColorTransformParams { + /** + * The relative x coordinate of the top left pixel of the processing + * window. The value shall be in the range of 0 and 1, inclusive and + * in multiples of 1/(width of Picture – 1). The value 1 corresponds + * to the absolute coordinate of width of Picture – 1. The value for + * first processing window shall be 0. + */ + AVRational window_upper_left_corner_x; + + /** + * The relative y coordinate of the top left pixel of the processing + * window. The value shall be in the range of 0 and 1, inclusive and + * in multiples of 1/(height of Picture – 1). The value 1 corresponds + * to the absolute coordinate of height of Picture – 1. The value for + * first processing window shall be 0. + */ + AVRational window_upper_left_corner_y; + + /** + * The relative x coordinate of the bottom right pixel of the processing + * window. The value shall be in the range of 0 and 1, inclusive and + * in multiples of 1/(width of Picture – 1). The value 1 corresponds + * to the absolute coordinate of width of Picture – 1. The value for + * first processing window shall be 1. + */ + AVRational window_lower_right_corner_x; + + /** + * The relative y coordinate of the bottom right pixel of the processing + * window. The value shall be in the range of 0 and 1, inclusive and + * in multiples of 1/(height of Picture – 1). The value 1 corresponds + * to the absolute coordinate of height of Picture – 1. The value for + * first processing window shall be 1. + */ + AVRational window_lower_right_corner_y; + + /** + * The x coordinate of the center position of the concentric internal and + * external ellipses of the elliptical pixel selector in the processing + * window. The value shall be in the range of 0 to (width of Picture – 1), + * inclusive and in multiples of 1 pixel. + */ + uint16_t center_of_ellipse_x; + + /** + * The y coordinate of the center position of the concentric internal and + * external ellipses of the elliptical pixel selector in the processing + * window. The value shall be in the range of 0 to (height of Picture – 1), + * inclusive and in multiples of 1 pixel. + */ + uint16_t center_of_ellipse_y; + + /** + * The clockwise rotation angle in degree of arc with respect to the + * positive direction of the x-axis of the concentric internal and external + * ellipses of the elliptical pixel selector in the processing window. The + * value shall be in the range of 0 to 180, inclusive and in multiples of 1. + */ + uint8_t rotation_angle; + + /** + * The semi-major axis value of the internal ellipse of the elliptical pixel + * selector in amount of pixels in the processing window. The value shall be + * in the range of 1 to 65535, inclusive and in multiples of 1 pixel. + */ + uint16_t semimajor_axis_internal_ellipse; + + /** + * The semi-major axis value of the external ellipse of the elliptical pixel + * selector in amount of pixels in the processing window. The value + * shall not be less than semimajor_axis_internal_ellipse of the current + * processing window. The value shall be in the range of 1 to 65535, + * inclusive and in multiples of 1 pixel. + */ + uint16_t semimajor_axis_external_ellipse; + + /** + * The semi-minor axis value of the external ellipse of the elliptical pixel + * selector in amount of pixels in the processing window. The value shall be + * in the range of 1 to 65535, inclusive and in multiples of 1 pixel. + */ + uint16_t semiminor_axis_external_ellipse; + + /** + * Overlap process option indicates one of the two methods of combining + * rendered pixels in the processing window in an image with at least one + * elliptical pixel selector. For overlapping elliptical pixel selectors + * in an image, overlap_process_option shall have the same value. + */ + enum AVHDRPlusOverlapProcessOption overlap_process_option; + + /** + * The maximum of the color components of linearized RGB values in the + * processing window in the scene. The values should be in the range of 0 to + * 1, inclusive and in multiples of 0.00001. maxscl[ 0 ], maxscl[ 1 ], and + * maxscl[ 2 ] are corresponding to R, G, B color components respectively. + */ + AVRational maxscl[3]; + + /** + * The average of linearized maxRGB values in the processing window in the + * scene. The value should be in the range of 0 to 1, inclusive and in + * multiples of 0.00001. + */ + AVRational average_maxrgb; + + /** + * The number of linearized maxRGB values at given percentiles in the + * processing window in the scene. The maximum value shall be 15. + */ + uint8_t num_distribution_maxrgb_percentiles; + + /** + * The linearized maxRGB values at given percentiles in the + * processing window in the scene. + */ + AVHDRPlusPercentile distribution_maxrgb[15]; + + /** + * The fraction of selected pixels in the image that contains the brightest + * pixel in the scene. The value shall be in the range of 0 to 1, inclusive + * and in multiples of 0.001. + */ + AVRational fraction_bright_pixels; + + /** + * This flag indicates that the metadata for the tone mapping function in + * the processing window is present (for value of 1). + */ + uint8_t tone_mapping_flag; + + /** + * The x coordinate of the separation point between the linear part and the + * curved part of the tone mapping function. The value shall be in the range + * of 0 to 1, excluding 0 and in multiples of 1/4095. + */ + AVRational knee_point_x; + + /** + * The y coordinate of the separation point between the linear part and the + * curved part of the tone mapping function. The value shall be in the range + * of 0 to 1, excluding 0 and in multiples of 1/4095. + */ + AVRational knee_point_y; + + /** + * The number of the intermediate anchor parameters of the tone mapping + * function in the processing window. The maximum value shall be 15. + */ + uint8_t num_bezier_curve_anchors; + + /** + * The intermediate anchor parameters of the tone mapping function in the + * processing window in the scene. The values should be in the range of 0 + * to 1, inclusive and in multiples of 1/1023. + */ + AVRational bezier_curve_anchors[15]; + + /** + * This flag shall be equal to 0 in bitstreams conforming to this version of + * this Specification. Other values are reserved for future use. + */ + uint8_t color_saturation_mapping_flag; + + /** + * The color saturation gain in the processing window in the scene. The + * value shall be in the range of 0 to 63/8, inclusive and in multiples of + * 1/8. The default value shall be 1. + */ + AVRational color_saturation_weight; +} AVHDRPlusColorTransformParams; + +/** + * This struct represents dynamic metadata for color volume transform - + * application 4 of SMPTE 2094-40:2016 standard. + * + * To be used as payload of a AVFrameSideData or AVPacketSideData with the + * appropriate type. + * + * @note The struct should be allocated with + * av_dynamic_hdr_plus_alloc() and its size is not a part of + * the public ABI. + */ +typedef struct AVDynamicHDRPlus { + /** + * Country code by Rec. ITU-T T.35 Annex A. The value shall be 0xB5. + */ + uint8_t itu_t_t35_country_code; + + /** + * Application version in the application defining document in ST-2094 + * suite. The value shall be set to 0. + */ + uint8_t application_version; + + /** + * The number of processing windows. The value shall be in the range + * of 1 to 3, inclusive. + */ + uint8_t num_windows; + + /** + * The color transform parameters for every processing window. + */ + AVHDRPlusColorTransformParams params[3]; + + /** + * The nominal maximum display luminance of the targeted system display, + * in units of 0.0001 candelas per square metre. The value shall be in + * the range of 0 to 10000, inclusive. + */ + AVRational targeted_system_display_maximum_luminance; + + /** + * This flag shall be equal to 0 in bit streams conforming to this version + * of this Specification. The value 1 is reserved for future use. + */ + uint8_t targeted_system_display_actual_peak_luminance_flag; + + /** + * The number of rows in the targeted system_display_actual_peak_luminance + * array. The value shall be in the range of 2 to 25, inclusive. + */ + uint8_t num_rows_targeted_system_display_actual_peak_luminance; + + /** + * The number of columns in the + * targeted_system_display_actual_peak_luminance array. The value shall be + * in the range of 2 to 25, inclusive. + */ + uint8_t num_cols_targeted_system_display_actual_peak_luminance; + + /** + * The normalized actual peak luminance of the targeted system display. The + * values should be in the range of 0 to 1, inclusive and in multiples of + * 1/15. + */ + AVRational targeted_system_display_actual_peak_luminance[25][25]; + + /** + * This flag shall be equal to 0 in bitstreams conforming to this version of + * this Specification. The value 1 is reserved for future use. + */ + uint8_t mastering_display_actual_peak_luminance_flag; + + /** + * The number of rows in the mastering_display_actual_peak_luminance array. + * The value shall be in the range of 2 to 25, inclusive. + */ + uint8_t num_rows_mastering_display_actual_peak_luminance; + + /** + * The number of columns in the mastering_display_actual_peak_luminance + * array. The value shall be in the range of 2 to 25, inclusive. + */ + uint8_t num_cols_mastering_display_actual_peak_luminance; + + /** + * The normalized actual peak luminance of the mastering display used for + * mastering the image essence. The values should be in the range of 0 to 1, + * inclusive and in multiples of 1/15. + */ + AVRational mastering_display_actual_peak_luminance[25][25]; + +} AVDynamicHDRPlus; + +/** + * Allocate an AVDynamicHDRPlus structure and set its fields to + * default values. The resulting struct can be freed using av_freep(). + * + * @return An AVDynamicHDRPlus filled with default values or NULL + * on failure. + */ +AVDynamicHDRPlus *av_dynamic_hdr_plus_alloc(size_t *size); + +/** + * Allocate a complete AVDynamicHDRPlus and add it to the frame. + * @param frame The frame which side data is added to. + * + * @return The AVDynamicHDRPlus structure to be filled by caller or NULL on + * failure. + */ +AVDynamicHDRPlus *av_dynamic_hdr_plus_create_side_data(AVFrame *frame); + +#endif /* AVUTIL_HDR_DYNAMIC_METADATA_H */ diff --git a/libavutil/version.h b/libavutil/version.h index 62112a6049..f9976151a7 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,8 +79,8 @@ */ #define LIBAVUTIL_VERSION_MAJOR 56 -#define LIBAVUTIL_VERSION_MINOR 24 -#define LIBAVUTIL_VERSION_MICRO 101 +#define LIBAVUTIL_VERSION_MINOR 25 +#define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ LIBAVUTIL_VERSION_MINOR, \