From patchwork Wed Feb 5 02:44:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mohammad Izadi X-Patchwork-Id: 17689 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 7488144B75C for ; Wed, 5 Feb 2020 05:14:12 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4E65B689D6C; Wed, 5 Feb 2020 05:14:12 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f173.google.com (mail-pg1-f173.google.com [209.85.215.173]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E7D316881CD for ; Wed, 5 Feb 2020 05:14:04 +0200 (EET) Received: by mail-pg1-f173.google.com with SMTP id j15so228637pgm.6 for ; Tue, 04 Feb 2020 19:14:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ClfJ9EhWZTs+0UOvjKN+XTwEkHhdoPAnTC54lZh1fYI=; b=s3BJOhJW7LIU6jAGaK6413R2EDOa9ad0WXk6Yum+B0f9MgAo63JmF06/fbl9I/udca s0p/6qBgRL2t3slWfft+NIz+kzHRyFO6GmT2SD2sZUfb9Nk2hlk6tHA7yau6fWC0Cc1U KPP4Cr8rx6B13c6VKofu5tnibAAMxiGIY2xDRYRKSlnQ/QrVX0Ksvd6zhHELW91xyBVy 7MkHW+/Jc1CIlK3qhuVQH/NZNm9aEJz/1JcgJCtVE2v22BdNF2P1urNBwO0ByLxdUPxp 9ULOj+43CPMKP8hrkGcTpTY5pSfNMX68bVpX699tLMfe/4ZKlTNOiJHbpagRaU9qbkYx b/cA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ClfJ9EhWZTs+0UOvjKN+XTwEkHhdoPAnTC54lZh1fYI=; b=PElJbuinTaQyuwsO/48Y2J/5hYgc8r3JpoVogUDeBO7imXXJ8jg3M0OwzpCxgDmFAW TJB0Ubff1calcKa+X8bTCljBXpGDYPRkzEbgG3AaLLG00V6/OE0jRHkNQDhyZVek4gx9 L3RiYN6t/b/AijgDOO+Zz+p9TAUD4I8/OBZfMdZ7jwCZ/hVU+CDECG4hNx20nAYUwCoX UkXMTD931edcKvINJgQs0YE85MUpiPfBHSeU5PfEMohn/fCNn14dkWtNbug0olHvFx1E 0Fw1c53yFEfjcxdjwp7KipkHQ2ctrgeXC1eH+n3ZTPDHvJhTqwvhcavgbKO7DB2BbE1l 2ruA== X-Gm-Message-State: APjAAAVgq1wHIcYV8P607dlCtudWDbisUMyKgLVuFx3pyD7K9jQ3bWWr ZFUQoE9Dx+rt78PoOID7VfvJn0Q= X-Google-Smtp-Source: APXvYqxj/P3s9kmv5iOtr7I735M288EZ7NeJ+Tron0KXPsgD+PCrN9ps0IcsrJnKSF7XXnFM38gVOA== X-Received: by 2002:a62:f247:: with SMTP id y7mr33901253pfl.5.1580870654114; Tue, 04 Feb 2020 18:44:14 -0800 (PST) Received: from izadi.mtv.corp.google.com ([2620:0:1000:4001:366c:3b00:ff02:a7ea]) by smtp.gmail.com with ESMTPSA id f9sm25465756pfd.141.2020.02.04.18.44.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Feb 2020 18:44:13 -0800 (PST) From: Mohammad Izadi To: ffmpeg-devel@ffmpeg.org Date: Tue, 4 Feb 2020 18:44:00 -0800 Message-Id: <20200205024400.46883-1-moh.izadi@gmail.com> X-Mailer: git-send-email 2.25.0.341.g760bfbb309-goog In-Reply-To: References: MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] Update HDR10+ metadata structure. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Mohammad Izadi Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Mohammad Izadi Trying to read HDR10+ metadata from HEVC/SEI and pass it to packet side data in the follow-up CLs. --- libavutil/hdr_dynamic_metadata.c | 167 ++++++++++++++++++++++++++++++- libavutil/hdr_dynamic_metadata.h | 14 ++- libavutil/version.h | 2 +- 3 files changed, 180 insertions(+), 3 deletions(-) diff --git a/libavutil/hdr_dynamic_metadata.c b/libavutil/hdr_dynamic_metadata.c index 0fa1ee82de..fbab192c62 100644 --- a/libavutil/hdr_dynamic_metadata.c +++ b/libavutil/hdr_dynamic_metadata.c @@ -1,4 +1,4 @@ -/** + /** * Copyright (c) 2018 Mohammad Izadi * * This file is part of FFmpeg. @@ -21,6 +21,14 @@ #include "hdr_dynamic_metadata.h" #include "mem.h" +static const int64_t luminance_den = 1; +static const int32_t peak_luminance_den = 15; +static const int64_t rgb_den = 100000; +static const int32_t fraction_pixel_den = 1000; +static const int32_t knee_point_den = 4095; +static const int32_t bezier_anchor_den = 1023; +static const int32_t saturation_weight_den = 8; + AVDynamicHDRPlus *av_dynamic_hdr_plus_alloc(size_t *size) { AVDynamicHDRPlus *hdr_plus = av_mallocz(sizeof(AVDynamicHDRPlus)); @@ -45,3 +53,160 @@ AVDynamicHDRPlus *av_dynamic_hdr_plus_create_side_data(AVFrame *frame) return (AVDynamicHDRPlus *)side_data->data; } + +int av_dynamic_hdr_plus_decode(const uint8_t *data, size_t size, + AVDynamicHDRPlus *s) +{ + int w, i, j; + GetBitContext gb; + if(!data) + return AVERROR_INVALIDDATA; + + int ret = init_get_bits8(&gb, data, size); + if(ret < 0) + return AVERROR_INVALIDDATA; + + if (get_bits_left(&gb) < 2) + return AVERROR_INVALIDDATA; + s->num_windows = get_bits(&gb, 2); + + if (s->num_windows < 1 || s->num_windows > 3) { + return AVERROR_INVALIDDATA; + } + + if (get_bits_left(&gb) < ((19 * 8 + 1) * (s->num_windows - 1))) + return AVERROR_INVALIDDATA; + for (w = 1; w < s->num_windows; w++) { + s->params[w].window_upper_left_corner_x.num = get_bits(&gb, 16); + s->params[w].window_upper_left_corner_y.num = get_bits(&gb, 16); + s->params[w].window_lower_right_corner_x.num = get_bits(&gb, 16); + s->params[w].window_lower_right_corner_y.num = get_bits(&gb, 16); + // The corners are set to absolute coordinates here. They should be + // converted to the relative coordinates (in [0, 1]) in the decoder. + s->params[w].window_upper_left_corner_x.den = 1; + s->params[w].window_upper_left_corner_y.den = 1; + s->params[w].window_lower_right_corner_x.den = 1; + s->params[w].window_lower_right_corner_y.den = 1; + + s->params[w].center_of_ellipse_x = get_bits(&gb, 16); + s->params[w].center_of_ellipse_y = get_bits(&gb, 16); + s->params[w].rotation_angle = get_bits(&gb, 8); + s->params[w].semimajor_axis_internal_ellipse = get_bits(&gb, 16); + s->params[w].semimajor_axis_external_ellipse = get_bits(&gb, 16); + s->params[w].semiminor_axis_external_ellipse = get_bits(&gb, 16); + s->params[w].overlap_process_option = get_bits1(&gb); + } + + if (get_bits_left(&gb) < 28) + return AVERROR(EINVAL); + s->targeted_system_display_maximum_luminance.num = get_bits(&gb, 27); + s->targeted_system_display_maximum_luminance.den = luminance_den; + s->targeted_system_display_actual_peak_luminance_flag = get_bits1(&gb); + + if (s->targeted_system_display_actual_peak_luminance_flag) { + int rows, cols; + if (get_bits_left(&gb) < 10) + return AVERROR(EINVAL); + rows = get_bits(&gb, 5); + cols = get_bits(&gb, 5); + if (((rows < 2) || (rows > 25)) || ((cols < 2) || (cols > 25))) { + return AVERROR_INVALIDDATA; + } + s->num_rows_targeted_system_display_actual_peak_luminance = rows; + s->num_cols_targeted_system_display_actual_peak_luminance = cols; + + if (get_bits_left(&gb) < (rows * cols * 4)) + return AVERROR(EINVAL); + + for (i = 0; i < rows; i++) { + for (j = 0; j < cols; j++) { + s->targeted_system_display_actual_peak_luminance[i][j].num = get_bits(&gb, 4); + s->targeted_system_display_actual_peak_luminance[i][j].den = peak_luminance_den; + } + } + } + for (w = 0; w < s->num_windows; w++) { + if (get_bits_left(&gb) < (3 * 17 + 17 + 4)) + return AVERROR(EINVAL); + for (i = 0; i < 3; i++) { + s->params[w].maxscl[i].num = get_bits(&gb, 17); + s->params[w].maxscl[i].den = rgb_den; + } + s->params[w].average_maxrgb.num = get_bits(&gb, 17); + s->params[w].average_maxrgb.den = rgb_den; + s->params[w].num_distribution_maxrgb_percentiles = get_bits(&gb, 4); + + if (get_bits_left(&gb) < + (s->params[w].num_distribution_maxrgb_percentiles * 24)) + return AVERROR(EINVAL); + for (i = 0; i < s->params[w].num_distribution_maxrgb_percentiles; i++) { + s->params[w].distribution_maxrgb[i].percentage = get_bits(&gb, 7); + s->params[w].distribution_maxrgb[i].percentile.num = get_bits(&gb, 17); + s->params[w].distribution_maxrgb[i].percentile.den = rgb_den; + } + + if (get_bits_left(&gb) < 10) + return AVERROR(EINVAL); + s->params[w].fraction_bright_pixels.num = get_bits(&gb, 10); + s->params[w].fraction_bright_pixels.den = fraction_pixel_den; + } + if (get_bits_left(&gb) < 1) + return AVERROR(EINVAL); + s->mastering_display_actual_peak_luminance_flag = get_bits1(&gb); + if (s->mastering_display_actual_peak_luminance_flag) { + int rows, cols; + if (get_bits_left(&gb) < 10) + return AVERROR(EINVAL); + rows = get_bits(&gb, 5); + cols = get_bits(&gb, 5); + if (((rows < 2) || (rows > 25)) || ((cols < 2) || (cols > 25))) { + return AVERROR_INVALIDDATA; + } + s->num_rows_mastering_display_actual_peak_luminance = rows; + s->num_cols_mastering_display_actual_peak_luminance = cols; + + if (get_bits_left(&gb) < (rows * cols * 4)) + return AVERROR(EINVAL); + + for (i = 0; i < rows; i++) { + for (j = 0; j < cols; j++) { + s->mastering_display_actual_peak_luminance[i][j].num = get_bits(&gb, 4); + s->mastering_display_actual_peak_luminance[i][j].den = peak_luminance_den; + } + } + } + + for (w = 0; w < s->num_windows; w++) { + if (get_bits_left(&gb) < 1) + return AVERROR(EINVAL); + s->params[w].tone_mapping_flag = get_bits1(&gb); + if (s->params[w].tone_mapping_flag) { + if (get_bits_left(&gb) < 28) + return AVERROR(EINVAL); + s->params[w].knee_point_x.num = get_bits(&gb, 12); + s->params[w].knee_point_x.den = knee_point_den; + s->params[w].knee_point_y.num = get_bits(&gb, 12); + s->params[w].knee_point_y.den = knee_point_den; + s->params[w].num_bezier_curve_anchors = get_bits(&gb, 4); + + if (get_bits_left(&gb) < (s->params[w].num_bezier_curve_anchors * 10)) + return AVERROR(EINVAL); + for (i = 0; i < s->params[w].num_bezier_curve_anchors; i++) { + s->params[w].bezier_curve_anchors[i].num = get_bits(&gb, 10); + s->params[w].bezier_curve_anchors[i].den = bezier_anchor_den; + } + } + + if (get_bits_left(&gb) < 1) + return AVERROR(EINVAL); + s->params[w].color_saturation_mapping_flag = get_bits1(&gb); + if (s->params[w].color_saturation_mapping_flag) { + if (get_bits_left(&gb) < 6) + return AVERROR(EINVAL); + s->params[w].color_saturation_weight.num = get_bits(&gb, 6); + s->params[w].color_saturation_weight.den = saturation_weight_den; + } + } + + return 0; +} diff --git a/libavutil/hdr_dynamic_metadata.h b/libavutil/hdr_dynamic_metadata.h index 2d72de56ae..0fc070b078 100644 --- a/libavutil/hdr_dynamic_metadata.h +++ b/libavutil/hdr_dynamic_metadata.h @@ -23,6 +23,8 @@ #include "frame.h" #include "rational.h" +#include "libavcodec/get_bits.h" +#include "libavcodec/put_bits.h" /** * Option for overlapping elliptical pixel selectors in an image. @@ -265,7 +267,7 @@ typedef struct AVDynamicHDRPlus { /** * The nominal maximum display luminance of the targeted system display, - * in units of 0.0001 candelas per square metre. The value shall be in + * in units of 1 candelas per square metre. The value shall be in * the range of 0 to 10000, inclusive. */ AVRational targeted_system_display_maximum_luminance; @@ -340,4 +342,14 @@ AVDynamicHDRPlus *av_dynamic_hdr_plus_alloc(size_t *size); */ AVDynamicHDRPlus *av_dynamic_hdr_plus_create_side_data(AVFrame *frame); +/** + * Decode SMPTE ST 2094-40 (the user data registered ITU-T T.35) to AVDynamicHDRPlus. + * @param data The user data registered ITU-T T.35 (SMPTE ST 2094-40). + * @param size The size of the user data registered ITU-T T.35 (SMPTE ST 2094-40). + * @param s The decoded AVDynamicHDRPlus structure. + * + * @return 0 if succeed. Otherwise, returns the appropriate AVERROR. + */ +int av_dynamic_hdr_plus_decode(const uint8_t *data, size_t size, AVDynamicHDRPlus *s); + #endif /* AVUTIL_HDR_DYNAMIC_METADATA_H */ diff --git a/libavutil/version.h b/libavutil/version.h index af8f614aff..2bc1b98615 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 56 -#define LIBAVUTIL_VERSION_MINOR 38 +#define LIBAVUTIL_VERSION_MINOR 39 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \