From patchwork Fri Dec 30 14:27:14 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Nikola_Kolarovi=C4=87?= X-Patchwork-Id: 1991 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.89.21 with SMTP id n21csp2934099vsb; Fri, 30 Dec 2016 09:06:25 -0800 (PST) X-Received: by 10.28.178.205 with SMTP id b196mr43155366wmf.8.1483117585461; Fri, 30 Dec 2016 09:06:25 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id pb5si62977419wjb.189.2016.12.30.09.06.24; Fri, 30 Dec 2016 09:06:25 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6B162689DA8; Fri, 30 Dec 2016 19:06:18 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.rt-rk.com (mx2.rt-rk.com [89.216.37.149]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id ABBDF680AD5 for ; Fri, 30 Dec 2016 16:33:08 +0200 (EET) Received: from localhost (localhost [127.0.0.1]) by mail.rt-rk.com (Postfix) with ESMTP id 66ABC1A202B; Fri, 30 Dec 2016 15:27:57 +0100 (CET) X-Virus-Scanned: amavisd-new at rt-rk.com Received: from [192.168.236.91] (rtrkn087-lin.domain.local [192.168.236.91]) by mail.rt-rk.com (Postfix) with ESMTPSA id 513381A1183; Fri, 30 Dec 2016 15:27:57 +0100 (CET) To: Nicolas George , FFmpeg development discussions and patches References: <20161228064805.31028-1-nikola.kolarovic@rt-rk.com> <20161229184629.GA1087110@phare.normalesup.org> From: =?UTF-8?Q?Nikola_Kolarovi=c4=87?= Message-ID: <79ffc23d-1631-5e0c-239d-210439d9fd16@rt-rk.com> Date: Fri, 30 Dec 2016 15:27:14 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <20161229184629.GA1087110@phare.normalesup.org> X-Mailman-Approved-At: Fri, 30 Dec 2016 19:06:17 +0200 Subject: Re: [FFmpeg-devel] [PATCH] avcodec/hevc: Extract Content light level from HEVC SEI prefix message X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" On 12/29/2016 07:46 PM, Nicolas George wrote: > Thanks for the patch. > > L'octidi 8 nivôse, an CCXXV, Nikola Kolarović a écrit : >> From: Nikola Kolarović >> >> Extract max_content_light_level and max_frame_average_light_level >> which are used in HEVC Main 10 (HDR10) for luminosity adjustment. >> >> Based on ISO/IEC 23008-2:2015 section D.2.35. >> --- >> libavcodec/hevc.h | 4 ++++ >> libavcodec/hevc_sei.c | 12 ++++++++++++ >> 2 files changed, 16 insertions(+) > The fields you set in this patch are not used anywhere else. Is there a > second patch coming to make use of them? > > Regards, Hi Nicolas, No problem, we can expose parsed values in frame/packet side_data. Follow-up patch is below, and I've run FATE regression tests on x86_64 linux. Thanks, Nikola From 3caeb8dcd2b68a966c521ffc686c2a326563c7fe Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nikola=20Kolarovi=C4=87?= Date: Fri, 30 Dec 2016 14:12:09 +0100 Subject: [PATCH] avcodec/hevc: Expose light level data using AVContentLightLevelMetadata Set HEVC side data (frame and packet) based on values extracted from Content light level HEVC SEI prefix message. --- libavcodec/avcodec.h | 7 +++++++ libavcodec/hevc.c | 17 +++++++++++++++++ libavcodec/hevc.h | 1 + libavcodec/hevc_sei.c | 1 + libavcodec/utils.c | 1 + libavfilter/f_sidedata.c | 1 + libavutil/frame.c | 7 ++++--- libavutil/frame.h | 6 ++++++ libavutil/mastering_display_metadata.h | 24 ++++++++++++++++++++++++ 9 files changed, 62 insertions(+), 3 deletions(-) diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index ca8b786077..5dc4556684 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -1543,6 +1543,13 @@ enum AVPacketSideDataType { AV_PKT_DATA_MASTERING_DISPLAY_METADATA, /** + * Content light level metadata (based on SMPTE 2086:2014). The payload + * is an AVContentLightLevelMetadata type and it contains upper bounds for + * the nominal target brightness light level. + */ + AV_PKT_DATA_CONTENT_LIGHT_LEVEL_METADATA, + + /** * This side data should be associated with a video stream and corresponds * to the AVSphericalMapping structure. */ diff --git a/libavcodec/hevc.c b/libavcodec/hevc.c index 7c563a34ba..1a3ccc51a6 100644 --- a/libavcodec/hevc.c +++ b/libavcodec/hevc.c @@ -2645,7 +2645,24 @@ static int set_side_data(HEVCContext *s) "min_luminance=%f, max_luminance=%f\n", av_q2d(metadata->min_luminance), av_q2d(metadata->max_luminance)); } + if (s->sei_content_light_level_info_present) { + AVFrameSideData* sd = av_frame_new_side_data(out, + AV_FRAME_DATA_CONTENT_LIGHT_LEVEL_METADATA, + sizeof(AVContentLightLevelMetadata)); + if (sd) { + AVContentLightLevelMetadata* metadata = (AVContentLightLevelMetadata*)sd->data; + const int luma_den = 10000; + + metadata->max_content_light_level.num = s->max_content_light_level; + metadata->max_content_light_level.den = luma_den; + metadata->max_frame_average_light_level.num = s->max_frame_average_light_level; + metadata->max_frame_average_light_level.den = luma_den; + av_log(s->avctx, AV_LOG_DEBUG, "Content Light Level: MaxCLL=%f, MaxFALL=%f\n", + av_q2d(metadata->max_content_light_level), + av_q2d(metadata->max_frame_average_light_level)); + } + } if (s->a53_caption) { AVFrameSideData* sd = av_frame_new_side_data(out, AV_FRAME_DATA_A53_CC, diff --git a/libavcodec/hevc.h b/libavcodec/hevc.h index d5970b61b6..0af633f52e 100644 --- a/libavcodec/hevc.h +++ b/libavcodec/hevc.h @@ -932,6 +932,7 @@ typedef struct HEVCContext { uint32_t min_mastering_luminance; /** content light level */ + int sei_content_light_level_info_present; uint16_t max_content_light_level; uint16_t max_frame_average_light_level; diff --git a/libavcodec/hevc_sei.c b/libavcodec/hevc_sei.c index 13ecc67173..ec75e66668 100644 --- a/libavcodec/hevc_sei.c +++ b/libavcodec/hevc_sei.c @@ -109,6 +109,7 @@ static int decode_nal_sei_content_light_level_info(HEVCContext *s) s->max_content_light_level = get_bits(gb, 16); s->max_frame_average_light_level = get_bits(gb, 16); + s->sei_content_light_level_info_present = 1; return 0; } diff --git a/libavcodec/utils.c b/libavcodec/utils.c index 1263306f63..3393ca34c9 100644 --- a/libavcodec/utils.c +++ b/libavcodec/utils.c @@ -766,6 +766,7 @@ int ff_init_buffer_info(AVCodecContext *avctx, AVFrame *frame) { AV_PKT_DATA_STEREO3D, AV_FRAME_DATA_STEREO3D }, { AV_PKT_DATA_AUDIO_SERVICE_TYPE, AV_FRAME_DATA_AUDIO_SERVICE_TYPE }, { AV_PKT_DATA_MASTERING_DISPLAY_METADATA, AV_FRAME_DATA_MASTERING_DISPLAY_METADATA }, + { AV_PKT_DATA_CONTENT_LIGHT_LEVEL_METADATA, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL_METADATA }, }; if (pkt) { diff --git a/libavfilter/f_sidedata.c b/libavfilter/f_sidedata.c index 45d246b732..1a30a8014d 100644 --- a/libavfilter/f_sidedata.c +++ b/libavfilter/f_sidedata.c @@ -61,6 +61,7 @@ static const AVOption filt_name##_options[] = { \ { "SKIP_SAMPLES", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_SKIP_SAMPLES }, 0, 0, FLAGS, "type" }, \ { "AUDIO_SERVICE_TYPE", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_AUDIO_SERVICE_TYPE }, 0, 0, FLAGS, "type" }, \ { "MASTERING_DISPLAY_METADATA", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_MASTERING_DISPLAY_METADATA }, 0, 0, FLAGS, "type" }, \ + { "CONTENT_LIGHT_LEVEL_METADATA","",0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_CONTENT_LIGHT_LEVEL_METADATA},0, 0, FLAGS, "type" }, \ { "GOP_TIMECODE", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_GOP_TIMECODE }, 0, 0, FLAGS, "type" }, \ { NULL } \ } diff --git a/libavutil/frame.c b/libavutil/frame.c index c2f55098c8..2efc227920 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -758,9 +758,10 @@ const char *av_frame_side_data_name(enum AVFrameSideDataType type) case AV_FRAME_DATA_AFD: return "Active format description"; case AV_FRAME_DATA_MOTION_VECTORS: return "Motion vectors"; case AV_FRAME_DATA_SKIP_SAMPLES: return "Skip samples"; - case AV_FRAME_DATA_AUDIO_SERVICE_TYPE: return "Audio service type"; - case AV_FRAME_DATA_MASTERING_DISPLAY_METADATA: return "Mastering display metadata"; - case AV_FRAME_DATA_GOP_TIMECODE: return "GOP timecode"; + case AV_FRAME_DATA_AUDIO_SERVICE_TYPE: return "Audio service type"; + case AV_FRAME_DATA_MASTERING_DISPLAY_METADATA: return "Mastering display metadata"; + case AV_FRAME_DATA_CONTENT_LIGHT_LEVEL_METADATA: return "Content light level"; + case AV_FRAME_DATA_GOP_TIMECODE: return "GOP timecode"; } return NULL; } diff --git a/libavutil/frame.h b/libavutil/frame.h index b4500923af..a80269a59b 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -117,6 +117,12 @@ enum AVFrameSideDataType { */ AV_FRAME_DATA_MASTERING_DISPLAY_METADATA, /** + * Content light level metadata associated with a video frame. The payload + * is an AVContentLightLevelMetadata type and it contains upper bounds for + * the nominal target brightness light level. + */ + AV_FRAME_DATA_CONTENT_LIGHT_LEVEL_METADATA, + /** * The GOP timecode in 25 bit timecode format. Data format is 64-bit integer. * This is set on the first frame of a GOP that has a temporal reference of 0. */ diff --git a/libavutil/mastering_display_metadata.h b/libavutil/mastering_display_metadata.h index 936533fec4..45249a7502 100644 --- a/libavutil/mastering_display_metadata.h +++ b/libavutil/mastering_display_metadata.h @@ -68,6 +68,30 @@ typedef struct AVMasteringDisplayMetadata { } AVMasteringDisplayMetadata; + +/** + * Luminance attributes of the mastered content, calculated in linear light + * domain, defined by SMPTE 2086:2014 and ISO/IEC 23008-2:2015. + * + * To be used as payload of a AVFrameSideData or AVPacketSideData with the + * appropriate type. + * + * @note The struct should be allocated with av_frame_new_side_data() + */ +typedef struct AVContentLightLevelMetadata { + /** + * MaxCLL: Maximum Content Light Level (cd/m^2). + */ + AVRational max_content_light_level; + + /** + * MaxFALL: Maximum Frame-Average Light Level (cd/m^2). + */ + AVRational max_frame_average_light_level; + +} AVContentLightLevelMetadata; + + /** * Allocate an AVMasteringDisplayMetadata structure and set its fields to * default values. The resulting struct can be freed using av_freep().