From patchwork Mon Dec 10 19:16:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mohammad Izadi X-Patchwork-Id: 11366 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 39D1F44C9E0 for ; Mon, 10 Dec 2018 21:49:57 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7BFD468AA4F; Mon, 10 Dec 2018 21:49:47 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f195.google.com (mail-pg1-f195.google.com [209.85.215.195]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4C39B68AA4F for ; Mon, 10 Dec 2018 21:49:40 +0200 (EET) Received: by mail-pg1-f195.google.com with SMTP id s198so5456913pgs.2 for ; Mon, 10 Dec 2018 11:49:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=hXp7Qfan3F2i+qqo4de9x2s/+m3WGX0F8PuNiw/a6tQ=; b=XHdL71J5I42+BocLWXkNBE8IphZQyOlqTczQVNKGV+N/GrjkYEuyBvX3vgZFio2iDz ZR7ntEJ0C+Bzxg5t++6K80AGnwSafgW96jx3KyW8PJPPeys19I4GnjrjDshaNy6j/gY5 rw6r6pmW5T3qFyQw7DW7hHPgnvLnQdXelyMl0Kail8FBAeIZEQIeu+JkOe0PnxvJH0qk 7ZF1ZXbnbf8WmA+OYxGFA6LyXkPUa/WmOW5j8c3msdH5QKJrYPcWVo/1Ex+buXNzWGht VOv5czeWSVkLeHMcJj+CxJb/ugaABTGEMsPEtSxvmnB+xclZZmuTxzs78XzII/aGduwM giMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=hXp7Qfan3F2i+qqo4de9x2s/+m3WGX0F8PuNiw/a6tQ=; b=o505/OlH3Z23+mNprrfoJyaKlPumu8XdMu+mw4oDcbMksmCi8yuBn8YHTkdS0B3y63 sk8O0J75F0nMxQVrTzXwMQ+4Iy+GnwPCbNQBsjrX0pX/88Aax28tMf6oL5Fv4QRcGkWe V5QW6G5BoloGUK3RXI0EkefFnhK7IwrMx1gaEXfBswMO1im6XyKPfm8SbHM7e/7HNNLb 7+CCl3RK+x/9SV1+eIRgLr9G4swp0mJhwg6BtRAaK5Shdnsmigq3BFuEqqNBcbopFDrP job8C8wZE5ntuTp4P8Qp1DZUJbfewwi02mGifX+3tMdr4YQxxfY2hvqtjWYzAgQulFVn fhDQ== X-Gm-Message-State: AA+aEWZtq0kTne22VO2fMCxNM2rMHOKs+v9/myPBFfKyYObKNzsC1Duz dP2JK7xPr0dsemvRW20W8RdzaAB5jA== X-Google-Smtp-Source: AFSGD/XqzZIsWiYTI+10rkRy3hni4tRpu+hyX8wl0FRwmMW1xowSl9GKz+dSjC46eznhAnIuWVaVDg== X-Received: by 2002:a63:594d:: with SMTP id j13mr11890393pgm.210.1544469437457; Mon, 10 Dec 2018 11:17:17 -0800 (PST) Received: from izadi.mtv.corp.google.com ([2620:0:1000:4011:3bcf:9bbf:ad2c:87b2]) by smtp.gmail.com with ESMTPSA id w11sm14154014pgk.16.2018.12.10.11.17.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 10 Dec 2018 11:17:16 -0800 (PST) From: Mohammad Izadi To: ffmpeg-devel@ffmpeg.org Date: Mon, 10 Dec 2018 11:16:59 -0800 Message-Id: <20181210191659.11884-1-moh.izadi@gmail.com> X-Mailer: git-send-email 2.20.0.rc2.403.gdbc3b29805-goog MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] Add HDR dynamic metadata struct (for SPMTE 2094-40) to libavutil. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Mohammad Izadi Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Mohammad Izadi The dynamic metadata contains data for color volume transform - application 4 of SPMTE 2094-40:2016 standard. The data comes from HEVC in the SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35. I'll add support to HEVC in a follow-up. --- libavutil/Makefile | 2 + libavutil/frame.c | 1 + libavutil/frame.h | 7 + libavutil/hdr_dynamic_metadata.c | 40 ++++ libavutil/hdr_dynamic_metadata.h | 344 +++++++++++++++++++++++++++++++ libavutil/version.h | 2 +- 6 files changed, 395 insertions(+), 1 deletion(-) create mode 100644 libavutil/hdr_dynamic_metadata.c create mode 100644 libavutil/hdr_dynamic_metadata.h diff --git a/libavutil/Makefile b/libavutil/Makefile index caddc7e155..7dcb92b06b 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -31,6 +31,7 @@ HEADERS = adler32.h \ file.h \ frame.h \ hash.h \ + hdr_dynamic_metadata.h \ hmac.h \ hwcontext.h \ hwcontext_cuda.h \ @@ -119,6 +120,7 @@ OBJS = adler32.o \ fixed_dsp.o \ frame.o \ hash.o \ + hdr_dynamic_metadata.o \ hmac.o \ hwcontext.o \ imgutils.o \ diff --git a/libavutil/frame.c b/libavutil/frame.c index 9b3fb13e68..c5f30b6847 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -840,6 +840,7 @@ const char *av_frame_side_data_name(enum AVFrameSideDataType type) case AV_FRAME_DATA_QP_TABLE_PROPERTIES: return "QP table properties"; case AV_FRAME_DATA_QP_TABLE_DATA: return "QP table data"; #endif + case AV_FRAME_DATA_HDR_DYNAMIC_METADATA_SMPTE2094_40: return "HDR Dynamic Metadata SMPTE2094-40"; } return NULL; } diff --git a/libavutil/frame.h b/libavutil/frame.h index 66f27f44bd..1b65a34ab3 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -166,6 +166,13 @@ enum AVFrameSideDataType { * function in libavutil/timecode.c. */ AV_FRAME_DATA_S12M_TIMECODE, + + /** + * HDR dynamic metadata associated with a video frame. The payload is + * an AVDynamicMetadataSMPTE2094_40 type and contains information for color + * volume transform - application 4 of SPMTE 2094-40:2016 standard. + */ + AV_FRAME_DATA_HDR_DYNAMIC_METADATA_SMPTE2094_40, }; enum AVActiveFormatDescription { diff --git a/libavutil/hdr_dynamic_metadata.c b/libavutil/hdr_dynamic_metadata.c new file mode 100644 index 0000000000..59dfcc84e4 --- /dev/null +++ b/libavutil/hdr_dynamic_metadata.c @@ -0,0 +1,40 @@ +/** + * Copyright (c) 2018 Mohammad Izadi + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "hdr_dynamic_metadata.h" +#include "mem.h" + +AVDynamicMetadataSMPTE2094_40 *av_dynamic_metadata_smpte2094_40_alloc(void) +{ + return av_mallocz(sizeof(AVDynamicMetadataSMPTE2094_40)); +} + +AVDynamicMetadataSMPTE2094_40 *av_dynamic_metadata_smpte2094_40_create_side_data(AVFrame *frame) +{ + AVFrameSideData *side_data = av_frame_new_side_data(frame, + AV_FRAME_DATA_HDR_DYNAMIC_METADATA_SMPTE2094_40, + sizeof(AVDynamicMetadataSMPTE2094_40)); + if (!side_data) + return NULL; + + memset(side_data->data, 0, sizeof(AVDynamicMetadataSMPTE2094_40)); + + return (AVDynamicMetadataSMPTE2094_40 *)side_data->data; +} diff --git a/libavutil/hdr_dynamic_metadata.h b/libavutil/hdr_dynamic_metadata.h new file mode 100644 index 0000000000..35c38454c3 --- /dev/null +++ b/libavutil/hdr_dynamic_metadata.h @@ -0,0 +1,344 @@ +/* + * Copyright (c) 2018 Mohammad Izadi + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_HDR_DYNAMIC_METADATA_H +#define AVUTIL_HDR_DYNAMIC_METADATA_H + +#include "frame.h" +#include "rational.h" + +/** + * Option for overlapping elliptical pixel selectors in an image. + */ +enum AVOverlapProcessOption { + AV_OVERLAP_PROCESS_WEIGHTED_AVERAGING = 0, + AV_OVERLAP_PROCESS_LAYERING = 1, +}; + +/** + * Percentile represents the percentile at a specific percentage in + * a distribution. + */ +typedef struct AVPercentile { + /** + * The percentage value corresponding to a spesific percentile linearized + * RGB value in the processing window in the scene. The value shall be in + * the range of 0 to100, inclusive. + */ + uint8_t percentage; + /** + * The linearized maxRGB value at a specific percentile in the processing + * window in the scene. The value shall be in the range of 0 to 1, inclusive + * and in multiples of 0.00001. + */ + AVRational percentile; +} AVPercentile; + +/** + * Color transform parameters at a processing window in a dynamic metadata for + * SPMTE 2094-40. + */ +typedef struct AVSMPTE2094_40ColorTransformParams { + /** + * The relative x coordinate of the top left pixel of the processing + * window. The value shall be in the range of 0 and 1, inclusive and + * in multiples of 1/(width of Picture – 1). The value 1 corresponds + * to the absolute coordinate of width of Picture – 1. The value for + * first processing window shall be 0. + */ + AVRational window_upper_left_corner_x; + + /** + * The relative y coordinate of the top left pixel of the processing + * window. The value shall be in the range of 0 and 1, inclusive and + * in multiples of 1/(height of Picture – 1). The value 1 corresponds + * to the absolute coordinate of height of Picture – 1. The value for + * first processing window shall be 0. + */ + AVRational window_upper_left_corner_y; + + /** + * The relative x coordinate of the bottom right pixel of the processing + * window. The value shall be in the range of 0 and 1, inclusive and + * in multiples of 1/(width of Picture – 1). The value 1 corresponds + * to the absolute coordinate of width of Picture – 1. The value for + * first processing window shall be 1. + */ + AVRational window_lower_right_corner_x; + + /** + * The relative y coordinate of the bottom right pixel of the processing + * window. The value shall be in the range of 0 and 1, inclusive and + * in multiples of 1/(height of Picture – 1). The value 1 corresponds + * to the absolute coordinate of height of Picture – 1. The value for + * first processing window shall be 1. + */ + AVRational window_lower_right_corner_y; + + /** + * The x coordinate of the center position of the concentric internal and + * external ellipses of the elliptical pixel selector in the processing + * window. The value shall be in the range of 0 to (width of Picture – 1), + * inclusive and in multiples of 1 pixel. + */ + uint16_t center_of_ellipse_x; + + /** + * The y coordinate of the center position of the concentric internal and + * external ellipses of the elliptical pixel selector in the processing + * window. The value shall be in the range of 0 to (height of Picture – 1), + * inclusive and in multiples of 1 pixel. + */ + uint16_t center_of_ellipse_y; + + /** + * The clockwise rotation angle in degree of arc with respect to the + * positive direction of the x-axis of the concentric internal and external + * ellipses of the elliptical pixel selector in the processing window. The + * value shall be in the range of 0 to 180, inclusive and in multiples of 1. + */ + uint8_t rotation_angle; + + /** + * The semi-major axis value of the internal ellipse of the elliptical pixel + * selector in amount of pixels in the processing window. The value shall be + * in the range of 1 to 65535, inclusive and in multiples of 1 pixel. + */ + uint16_t semimajor_axis_internal_ellipse; + + /** + * The semi-major axis value of the external ellipse of the elliptical pixel + * selector in amount of pixels in the processing window. The value + * shall not be less than semimajor_axis_internal_ellipse of the current + * processing window. The value shall be in the range of 1 to 65535, + * inclusive and in multiples of 1 pixel. + */ + uint16_t semimajor_axis_external_ellipse; + + /** + * The semi-minor axis value of the external ellipse of the elliptical pixel + * selector in amount of pixels in the processing window. The value shall be + * in the range of 1 to 65535, inclusive and in multiples of 1 pixel. + */ + uint16_t semiminor_axis_external_ellipse; + + /** + * Overlap process option indicates one of the two methods of combining + * rendered pixels in the processing window in an image with at least one + * elliptical pixel selector. For overlapping elliptical pixel selectors + * in an image, overlap_process_option shall have the same value. + */ + enum AVOverlapProcessOption overlap_process_option; + + /** + * The maximum of the color components of linearized RGB values in the + * processing window in the scene. The values should be in the range of 0 to + * 1, inclusive and in multiples of 0.00001. maxscl[ 0 ], maxscl[ 1 ], and + * maxscl[ 2 ] are corresponding to R, G, B color components respectively. + */ + AVRational maxscl[3]; + + /** + * The average of linearized maxRGB values in the processing window in the + * scene. The value should be in the range of 0 to 1, inclusive and in + * multiples of 0.00001. + */ + AVRational average_maxrgb; + + /** + * The number of linearized maxRGB values at given percentiles in the + * processing window in the scene. The maximum value shall be 15. + */ + uint8_t num_distribution_maxrgb_percentiles; + + /** + * The linearized maxRGB values at given percentiles in the + * processing window in the scene. + */ + AVPercentile distribution_maxrgb[15]; + + /** + * The fraction of selected pixels in the image that contains the brightest + * pixel in the scene. The value shall be in the range of 0 to 1, inclusive + * and in multiples of 0.001. + */ + AVRational fraction_bright_pixels; + + /** + * This flag indicates that the metadata for the tone mapping function in + * the processing window is present (for value of 1). + */ + uint8_t tone_mapping_flag; + + /** + * The x coordinate of the separation point between the linear part and the + * curved part of the tone mapping function. The value shall be in the range + * of 0 to 1, excluding 0 and in multiples of 1/4095. + */ + AVRational knee_point_x; + + /** + * The y coordinate of the separation point between the linear part and the + * curved part of the tone mapping function. The value shall be in the range + * of 0 to 1, excluding 0 and in multiples of 1/4095. + */ + AVRational knee_point_y; + + /** + * The number of the intermediate anchor parameters of the tone mapping + * function in the processing window. The maximum value shall be 15. + */ + uint8_t num_bezier_curve_anchors; + + /** + * The intermediate anchor parameters of the tone mapping function in the + * processing window in the scene. The values should be in the range of 0 + * to 1, inclusive and in multiples of 1/1023. + */ + AVRational bezier_curve_anchors[15]; + + /** + * This flag shall be equal to 0 in bitstreams conforming to this version of + * this Specification. Other values are reserved for future use. + */ + uint8_t color_saturation_mapping_flag; + + /** + * The color saturation gain in the processing window in the scene. The + * value shall be in the range of 0 to 63/8, inclusive and in multiples of + * 1/8. The default value shall be 1. + */ + AVRational color_saturation_weight; +} AVSMPTE2094_40ColorTransformParams; + +/** + * This struct represents dynamic metadata for color volume transform - + * application 4 of SPMTE 2094-40:2016 standard. + * + * To be used as payload of a AVFrameSideData or AVPacketSideData with the + * appropriate type. + * + * @note The struct should be allocated with + * av_dynamic_metadata_smpte2094_40_alloc() and its size is not a part of + * the public ABI. + */ +typedef struct AVDynamicMetadataSMPTE2094_40 { + /** + * Country code by Rec. ITU-T T.35 Annex A. The value shall be 0xB5. + */ + uint8_t itu_t_t35_country_code; + + /** + * Application version in the application defining document in ST-2094 + * suite. The value shall be set to 0. + */ + uint8_t application_version; + + /** + * The number of processing windows. The value shall be in the range + * of 1 to 3, inclusive. + */ + uint8_t num_windows; + + /** + * The color transform parameters for every processing window. + */ + AVSMPTE2094_40ColorTransformParams params[3]; + + /** + * The nominal maximum display luminance of the targeted system display, + * in units of 0.0001 candelas per square metre. The value shall be in + * the range of 0 to 10000, inclusive. + */ + AVRational targeted_system_display_maximum_luminance; + + /** + * This flag shall be equal to 0 in bit streams conforming to this version + * of this Specification. The value 1 is reserved for future use. + */ + uint8_t targeted_system_display_actual_peak_luminance_flag; + + /** + * The number of rows in the targeted system_display_actual_peak_luminance + * array. The value shall be in the range of 2 to 25, inclusive. + */ + uint8_t num_rows_targeted_system_display_actual_peak_luminance; + + /** + * The number of columns in the + * targeted_system_display_actual_peak_luminance array. The value shall be + * in the range of 2 to 25, inclusive. + */ + uint8_t num_cols_targeted_system_display_actual_peak_luminance; + + /** + * The normalized actual peak luminance of the targeted system display. The + * values should be in the range of 0 to 1, inclusive and in multiples of + * 1/15. + */ + AVRational targeted_system_display_actual_peak_luminance[25][25]; + + /** + * This flag shall be equal to 0 in bitstreams conforming to this version of + * this Specification. The value 1 is reserved for future use. + */ + uint8_t mastering_display_actual_peak_luminance_flag; + + /** + * The number of rows in the mastering_display_actual_peak_luminance array. + * The value shall be in the range of 2 to 25, inclusive. + */ + uint8_t num_rows_mastering_display_actual_peak_luminance; + + /** + * The number of columns in the mastering_display_actual_peak_luminance + * array. The value shall be in the range of 2 to 25, inclusive. + */ + uint8_t num_cols_mastering_display_actual_peak_luminance; + + /** + * The normalized actual peak luminance of the mastering display used for + * mastering the image essence. The values should be in the range of 0 to 1, + * inclusive and in multiples of 1/15. + */ + AVRational mastering_display_actual_peak_luminance[25][25]; + +} AVDynamicMetadataSMPTE2094_40; + +/** + * Allocate an AVDynamicMetadataSMPTE2094_40 structure and set its fields to + * default values. The resulting struct can be freed using av_freep(). + * + * @return An AVDynamicMetadataSMPTE2094_40 filled with default values or NULL + * on failure. + */ +AVDynamicMetadataSMPTE2094_40 *av_dynamic_metadata_smpte2094_40_alloc(void); + +/** + * Allocate a complete AVDynamicMetadataSMPTE2094_40 and add it to the frame. + * + * @param frame The frame which side data is added to. + * + * @return The AVDynamicMetadataSMPTE2094_40 structure to be filled by caller. + */ +AVDynamicMetadataSMPTE2094_40 * +av_dynamic_metadata_smpte2094_40_create_side_data(AVFrame *frame); + +#endif /* AVUTIL_HDR_DYNAMIC_METADATA_H */ diff --git a/libavutil/version.h b/libavutil/version.h index 62112a6049..6aea82c609 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 56 -#define LIBAVUTIL_VERSION_MINOR 24 +#define LIBAVUTIL_VERSION_MINOR 25 #define LIBAVUTIL_VERSION_MICRO 101 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \