From patchwork Sun Aug 23 22:33:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 21863 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 36C2644B129 for ; Mon, 24 Aug 2020 01:35:40 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 225A168AA85; Mon, 24 Aug 2020 01:35:40 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f68.google.com (mail-wm1-f68.google.com [209.85.128.68]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DE42468044F for ; Mon, 24 Aug 2020 01:35:30 +0300 (EEST) Received: by mail-wm1-f68.google.com with SMTP id s20so1577209wmj.1 for ; Sun, 23 Aug 2020 15:35:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=3rJqc7kGbhrKHM/SU6DakDJWbVBXOarr/SZ8jcdrPzo=; b=T6hcjzQQSXTdvwziZcwy2/Lx7pZ1mIcOnmQjAcBFE2S+BU8lZdDz+Hs1gfJCLy6bFn oN0PiWirq8ns5TjuBaH94lt8b6Ipe3Rb8Cx6cswwNDCMcoIJ2r+hovAv/jFkUn10s+Hd DbqoK/jn09nrzLKSRF3YF9ty5SROpvO6XIi25BT5PvPNnmTUygOYJZcT1MFsOkBquW2s AfotXhLvhZ+Ktq01bj0PMEt+ksR88Zz5lBHB6HbNORwvQAH3BudUh3PHXWF3q3DYUBSP FpniWny62ZdFQUG9uh6qKBQ7ijU26YdO8+JixH2pvwuOvIvmmbx6t1zN8WJ2O54A8+Zj Kv5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3rJqc7kGbhrKHM/SU6DakDJWbVBXOarr/SZ8jcdrPzo=; b=AqhyiKfyLn2LOfPszFsJxV0AsbFEOlI4YA8bG8RtVXjhT4PHE/z4cfgfLSmmChyWbx m6yxVF73XZGZYNx8h/1VZs4c9lrl4ngME7YEiVTWbkEua2q3EoPbwUzQ4gyK/ZGx2+nV y0Rbf6j+gAl5gLuPxYiFjpB0+RxyaSSx7qyV/lFg8OcJkqVs4hUMwXI3jLBEuj7pZgcR EvhhhPAy2I5wCI2TDfk61zZlMUuhs121sUmyjd5xy2j8Cy2K76phz4eOHz2LAy4j0j6E Sdcu8/ant72sE4srSv9cbTvaSN9JwpJpVfI32QqpQv628qAePpfua93bmTGmGdzgLwiV 4BIg== X-Gm-Message-State: AOAM531tCmszoHnozMKe5VMtbEaMzaMgDbrB7t61FtprsVC8L9ePCgUJ zpFuyWH/QKjk4/LbsYQ8ivz+vbjZU1MOgA== X-Google-Smtp-Source: ABdhPJxkwbBdTlxQrtzsvTQBHi2Dud+/fWHJITKTNn/gNsdqf2zzLbh2Sc9yml/TgXBanxX5CSTBxQ== X-Received: by 2002:a7b:c5c1:: with SMTP id n1mr2764933wmk.125.1598222129661; Sun, 23 Aug 2020 15:35:29 -0700 (PDT) Received: from localhost.localdomain (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id f10sm11216852wmj.37.2020.08.23.15.35.28 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Aug 2020 15:35:28 -0700 (PDT) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Aug 2020 23:33:10 +0100 Message-Id: <20200823223310.233061-8-sw@jkqxz.net> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200823223310.233061-1-sw@jkqxz.net> References: <20200823223310.233061-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 8/8] RFC: editing HDR properties in H.265 metadata BSF X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- Setting HDR properties is a useful feature, but it's very unclear what we want it to actually look like to the user. Not all encoders and decoders support it, so it's essentially required that the implementation happen at the bitstream filter level so that we can support all codecs in the same way. This is several patches mashed together to invite comments on a bitstream filter approach. It modifies hevc_metadata to allow inserting both explicit values as well as ones derived from existing side-data. Example: hevc_metadata=mastering_display_colour_volume=insert:x265_master_display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1):content_light_level_information=insert:x265_max_cll=1000,400 * I don't see a nice answer as to what form explicit values should take. This copies the options from x265 precisely and gives them names including "x265_" to indicate that, but it seems like we should allow a more flexible format (in particular: the units are from the H.265 SEI message and are therefore a bit silly for humans, and also you might want to edit /only some/ of the values from the source). Maybe each value could be specified as a separate argument, but given the interdependencies between them that would somewhat more complex to deal with. It would be helpful to be able to set multiple values together as well, like colour_primaries=BT709 or white_point=D65 rather than setting each coordinate individually. Obviously if we go this single-codec route then it would need to be duplicated (possibly with some parts suitably abstracted) in other codecs - H.264 and AV1 both want it too. To avoid this, rather than passing explicit values to codec-specific filters we could instead have a generic bitstream filter which only edits side-data. Then the codec-specific filters would insert side-data themselves. Comibining those, an exmaple to edit some fields would be something like: hevc_metadata=mastering_display_colour_volume=extract,side_data=max_luminance=9100:min_luminance=300,hevc_metadata=mastering_display_colour_volume=insert Or maybe some combination of those, so you have both the generic filter and the codec-specific filters accept the value options too? Thoughts invited on any of this. Thanks, - Mark * This doesn't actually work on the ffmpeg command-line because there is no way to escape the commas in the -bsf:v argument. Splicing in the code works, but that should probably be fixed. libavcodec/cbs_h2645.c | 4 +- libavcodec/cbs_h265.c | 190 +++++++++++++++++++++++++ libavcodec/cbs_h265.h | 50 +++++++ libavcodec/h265_metadata_bsf.c | 248 ++++++++++++++++++++++++++++++++- 4 files changed, 489 insertions(+), 3 deletions(-) diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index 1677731db9..b9f219cdf7 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -1386,7 +1386,7 @@ static const CodedBitstreamUnitTypeDescriptor cbs_h264_unit_types[] = { CBS_UNIT_TYPE_END_OF_LIST }; -static void cbs_h265_free_sei_payload(H265RawSEIPayload *payload) +void ff_cbs_h265_free_sei_payload(H265RawSEIPayload *payload) { switch (payload->payload_type) { case HEVC_SEI_TYPE_BUFFERING_PERIOD: @@ -1420,7 +1420,7 @@ static void cbs_h265_free_sei(void *opaque, uint8_t *content) H265RawSEI *sei = (H265RawSEI*)content; int i; for (i = 0; i < sei->payload_count; i++) - cbs_h265_free_sei_payload(&sei->payload[i]); + ff_cbs_h265_free_sei_payload(&sei->payload[i]); av_freep(&content); } diff --git a/libavcodec/cbs_h265.c b/libavcodec/cbs_h265.c index f6226d8743..3d2c99af03 100644 --- a/libavcodec/cbs_h265.c +++ b/libavcodec/cbs_h265.c @@ -20,8 +20,145 @@ #include "libavutil/mastering_display_metadata.h" #include "cbs_h265.h" +#include "cbs_internal.h" +int ff_cbs_h265_find_sei_message(CodedBitstreamFragment *au, + int type, + H265RawSEIPayload **message) +{ + H265RawSEI *sei; + int i, j, found_message; + + found_message = !*message; + for (i = 0; i < au->nb_units; i++) { + if (au->units[i].type != HEVC_NAL_SEI_PREFIX && + au->units[i].type != HEVC_NAL_SEI_SUFFIX) + continue; + sei = au->units[i].content; + + for (j = 0; j < sei->payload_count; j++) { + if (sei->payload[j].payload_type == type) { + if (found_message) { + *message = &sei->payload[j]; + return 0; + } else { + if (&sei->payload[j] == *message) + found_message = 1; + } + } + } + } + + *message = NULL; + return AVERROR(ENOENT); +} + +int ff_cbs_h265_add_sei_message(CodedBitstreamFragment *au, + H265RawSEIPayload *payload, int prefix) +{ + H265RawSEI *sei = NULL; + int sei_type = prefix ? HEVC_NAL_SEI_PREFIX + : HEVC_NAL_SEI_SUFFIX; + int err, i; + + // Find an existing SEI NAL unit to add to. + for (i = 0; i < au->nb_units; i++) { + if (au->units[i].type == sei_type) { + sei = au->units[i].content; + if (sei->payload_count < H265_MAX_SEI_PAYLOADS) + break; + + sei = NULL; + } + } + + if (!sei) { + // Need to make a new SEI NAL unit. Insert it before the VCL if + // prefix, or after if suffix. + CodedBitstreamUnit *unit; + int position = -1; + + for (i = 0; i < au->nb_units; i++) { + if (au->units[i].type <= HEVC_NAL_RSV_VCL31) { + if (prefix) { + position = i; + break; + } else { + position = i + 1; + } + } + } + + err = ff_cbs_insert_unit(au, position); + if (err < 0) + goto fail; + unit = &au->units[position]; + unit->type = sei_type; + + // Needs a context for the type information but we don't have + // one, so forge one containing only the type information. + err = ff_cbs_alloc_unit_content2(&(CodedBitstreamContext) { + .codec = &ff_cbs_type_h265 }, unit); + if (err < 0) { + ff_cbs_delete_unit(au, i); + goto fail; + } + sei = unit->content; + + *sei = (H265RawSEI) { + .nal_unit_header = { + .nal_unit_type = sei_type, + }, + }; + } + + memcpy(&sei->payload[sei->payload_count], payload, sizeof(*payload)); + ++sei->payload_count; + + return 0; +fail: + ff_cbs_h265_free_sei_payload(payload); + return err; +} + +void ff_cbs_h265_delete_sei_message(CodedBitstreamFragment *au, + H265RawSEIPayload *message) +{ + H265RawSEI *sei; + int i, j, position; + + position = -1; + for (i = 0; i < au->nb_units; i++) { + if (au->units[i].type != HEVC_NAL_SEI_PREFIX && + au->units[i].type != HEVC_NAL_SEI_SUFFIX) + continue; + sei = au->units[i].content; + for (j = 0; j < sei->payload_count; j++) { + if (&sei->payload[j] == message) { + position = j; + break; + } + } + if (position >= 0) + break; + } + // Message not found in this access unit. + av_assert0(position >= 0); + + if (position == 0 && sei->payload_count == 1) { + // Deleting NAL unit entirely. + ff_cbs_delete_unit(au, i); + } else { + ff_cbs_h265_free_sei_payload(&sei->payload[position]); + + --sei->payload_count; + memmove(sei->payload + position, + sei->payload + position + 1, + (sei->payload_count - position) * sizeof(*sei->payload)); + } +} + static uint32_t rescale_clip(AVRational value, uint32_t scale, uint32_t min, uint32_t max) { @@ -93,3 +230,56 @@ void ff_cbs_h265_fill_sei_content_light_level(H265RawSEIContentLightLevelInfo *c cll->max_content_light_level = av_clip_uintp2(clm->MaxCLL, 16); cll->max_pic_average_light_level = av_clip_uintp2(clm->MaxFALL, 16); } + + +void ff_cbs_h265_extract_sei_mastering_display(AVMasteringDisplayMetadata *mdm, + const H265RawSEIMasteringDisplayColourVolume *mdcv) +{ +#define IN_RANGE(v, min, max) ((v) >= (min) && (v) <= (max)) +#define IS_VALID_COORD(x, y) (IN_RANGE(x, 5, 37000) && IN_RANGE(y, 5, 42000)) + int valid_chromaticity = 1; + for (int a = 0; a < 3; a++) { + if (!IS_VALID_COORD(mdcv->display_primaries_x[a], + mdcv->display_primaries_y[a])) + valid_chromaticity = 0; + } + if (!IS_VALID_COORD(mdcv->white_point_x, mdcv->white_point_y)) + valid_chromaticity = 0; + + memset(mdm, 0, sizeof(*mdm)); + + if (valid_chromaticity) { + for (int a = 0; a < 3; a++) { + // SEI message in GBR order, but metadata structure in RGB order. + static const uint8_t mapping[] = { 2, 0, 1 }; + int b = mapping[a]; + + mdm->display_primaries[a][0] = + av_make_q(mdcv->display_primaries_x[b], 50000); + mdm->display_primaries[a][1] = + av_make_q(mdcv->display_primaries_y[b], 50000); + } + + mdm->white_point[0] = av_make_q(mdcv->white_point_x, 50000); + mdm->white_point[1] = av_make_q(mdcv->white_point_y, 50000); + + mdm->has_primaries = 1; + } + + if (IN_RANGE(mdcv->min_display_mastering_luminance, 1, 50000) && + IN_RANGE(mdcv->max_display_mastering_luminance, 50000, 100000000)) { + mdm->min_luminance = av_make_q(mdcv->min_display_mastering_luminance, 10000); + mdm->max_luminance = av_make_q(mdcv->max_display_mastering_luminance, 10000); + + mdm->has_luminance = 1; + } +#undef IN_RANGE +#undef IS_VALID_COORD +} + +void ff_cbs_h265_extract_sei_content_light_level(AVContentLightMetadata *clm, + const H265RawSEIContentLightLevelInfo *cll) +{ + clm->MaxCLL = cll->max_content_light_level; + clm->MaxFALL = cll->max_pic_average_light_level; +} diff --git a/libavcodec/cbs_h265.h b/libavcodec/cbs_h265.h index 999353f607..eb6d4b0702 100644 --- a/libavcodec/cbs_h265.h +++ b/libavcodec/cbs_h265.h @@ -22,6 +22,7 @@ #include #include +#include "cbs.h" #include "cbs_h2645.h" #include "hevc.h" @@ -747,6 +748,42 @@ typedef struct CodedBitstreamH265Context { } CodedBitstreamH265Context; +/** + * Free an SEI payload structure. + */ +void ff_cbs_h265_free_sei_payload(H265RawSEIPayload *payload); + +/** + * Find an SEI message of the given type within an access unit. + * + * If message is already set, only looks after it - this can be used to + * iterate over all messages of a given type. + */ +int ff_cbs_h265_find_sei_message(CodedBitstreamFragment *au, + int type, + H265RawSEIPayload **message); + +/** + * Add an SEI message to an access unit. + * + * Adds at the end of an existing SEI NAL unit of the appropriate type + * (PREFIX_SEI if prefix is set, otherwise SUFFIX_SEI), otherwise creates + * a new NAL unit to contain the message. + */ +int ff_cbs_h265_add_sei_message(CodedBitstreamFragment *au, + H265RawSEIPayload *payload, + int prefix); + +/** + * Delete an SEI message from an access unit. + * + * Deletes the given message from the NAL Unit containing it. If it is + * the last message in the NAL unit, also deletes it from the access unit. + */ +void ff_cbs_h265_delete_sei_message(CodedBitstreamFragment *au, + H265RawSEIPayload *message); + + struct AVMasteringDisplayMetadata; struct AVContentLightMetadata; @@ -764,5 +801,18 @@ void ff_cbs_h265_fill_sei_mastering_display(H265RawSEIMasteringDisplayColourVolu void ff_cbs_h265_fill_sei_content_light_level(H265RawSEIContentLightLevelInfo *cll, const struct AVContentLightMetadata *clm); +/** + * Fill an AVMasteringDisplayMetadata side-data structure with values + * derived from the SEI Mastering Display Colour Volume structure. + */ +void ff_cbs_h265_extract_sei_mastering_display(struct AVMasteringDisplayMetadata *mdm, + const H265RawSEIMasteringDisplayColourVolume *mdcv); + +/** + * Fill an AVContentLightMetadata structure with values derived from + * the SEI Content Light Level Information structure. + */ +void ff_cbs_h265_extract_sei_content_light_level(struct AVContentLightMetadata *clm, + const H265RawSEIContentLightLevelInfo *cll); #endif /* AVCODEC_CBS_H265_H */ diff --git a/libavcodec/h265_metadata_bsf.c b/libavcodec/h265_metadata_bsf.c index c3eadee92b..8e18e88a3d 100644 --- a/libavcodec/h265_metadata_bsf.c +++ b/libavcodec/h265_metadata_bsf.c @@ -17,6 +17,7 @@ */ #include "libavutil/common.h" +#include "libavutil/mastering_display_metadata.h" #include "libavutil/opt.h" #include "bsf.h" @@ -24,6 +25,7 @@ #include "cbs_bsf.h" #include "cbs_h265.h" #include "hevc.h" +#include "hevc_sei.h" #include "h265_profile_level.h" enum { @@ -34,6 +36,8 @@ enum { typedef struct H265MetadataContext { CBSBSFContext common; + int done_first_au; + H265RawAUD aud_nal; int aud; @@ -60,6 +64,18 @@ typedef struct H265MetadataContext { int level; int level_guess; int level_warned; + + int mdcv_sei; + int mdcv_multiple_warned; + AVMasteringDisplayMetadata mdcv_side_data; + H265RawSEIMasteringDisplayColourVolume mdcv_x265; + const char *mdcv_x265_str; + + int cll_sei; + int cll_multiple_warned; + AVContentLightMetadata cll_side_data; + H265RawSEIContentLightLevelInfo cll_x265; + const char *cll_x265_str; } H265MetadataContext; @@ -328,11 +344,157 @@ static int h265_metadata_update_sps(AVBSFContext *bsf, return 0; } +static int h265_metadata_update_mdcv(AVBSFContext *bsf, AVPacket *pkt, + CodedBitstreamFragment *au, + int seek_point) +{ + H265MetadataContext *ctx = bsf->priv_data; + H265RawSEIPayload *message = NULL; + AVMasteringDisplayMetadata *mdm; + uint8_t *data; + int err, first; + + first = 1; + while (1) { + err = ff_cbs_h265_find_sei_message(au, + HEVC_SEI_TYPE_MASTERING_DISPLAY_INFO, &message); + if (err < 0) + break; + + if (ctx->mdcv_sei == BSF_ELEMENT_REMOVE || + ctx->mdcv_sei == BSF_ELEMENT_INSERT) { + ff_cbs_h265_delete_sei_message(au, message); + + } else if (ctx->mdcv_sei == BSF_ELEMENT_EXTRACT && pkt) { + if (first) { + data = av_packet_new_side_data(pkt, + AV_PKT_DATA_MASTERING_DISPLAY_METADATA, sizeof(*mdm)); + if (!data) + return AVERROR(ENOMEM); + + mdm = (AVMasteringDisplayMetadata*)data; + ff_cbs_h265_extract_sei_mastering_display(mdm, + &message->payload.mastering_display); + } else if (!ctx->mdcv_multiple_warned) { + av_log(bsf, AV_LOG_WARNING, "Multiple MDCV SEI " + "messages found in access unit: " + "all but the first will be ignored.\n"); + ctx->mdcv_multiple_warned = 1; + } + } + + first = 0; + } + + if (seek_point && ctx->mdcv_sei == BSF_ELEMENT_INSERT) { + H265RawSEIPayload payload = { + .payload_type = HEVC_SEI_TYPE_MASTERING_DISPLAY_INFO, + }; + H265RawSEIMasteringDisplayColourVolume *mdcv = + &payload.payload.mastering_display; + int size; + + data = av_packet_get_side_data(pkt, + AV_PKT_DATA_MASTERING_DISPLAY_METADATA, &size); + if (data) { + av_assert0(size >= sizeof(*mdm)); + mdm = (AVMasteringDisplayMetadata*)data; + ctx->mdcv_side_data = *mdm; + } + + if (ctx->mdcv_x265_str) { + *mdcv = ctx->mdcv_x265; + } else { + ff_cbs_h265_fill_sei_mastering_display(mdcv, + &ctx->mdcv_side_data); + } + + err = ff_cbs_h265_add_sei_message(au, &payload, 1); + if (err < 0) + return err; + } + + return 0; +} + +static int h265_metadata_update_cll(AVBSFContext *bsf, AVPacket *pkt, + CodedBitstreamFragment *au, + int seek_point) +{ + H265MetadataContext *ctx = bsf->priv_data; + H265RawSEIPayload *message = NULL; + AVContentLightMetadata *clm; + uint8_t *data; + int err, first; + + first = 1; + while (1) { + err = ff_cbs_h265_find_sei_message(au, + HEVC_SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO, &message); + if (err < 0) + break; + + if (ctx->cll_sei == BSF_ELEMENT_REMOVE || + ctx->cll_sei == BSF_ELEMENT_INSERT) { + ff_cbs_h265_delete_sei_message(au, message); + + } else if (ctx->cll_sei == BSF_ELEMENT_EXTRACT && pkt) { + if (first) { + data = av_packet_new_side_data(pkt, + AV_PKT_DATA_CONTENT_LIGHT_LEVEL, sizeof(*clm)); + if (!data) + return AVERROR(ENOMEM); + + clm = (AVContentLightMetadata*)data; + ff_cbs_h265_extract_sei_content_light_level(clm, + &message->payload.content_light_level); + } else if (!ctx->cll_multiple_warned) { + av_log(bsf, AV_LOG_WARNING, "Multiple CLL SEI " + "messages found in access unit: " + "all but the first will be ignored.\n"); + ctx->cll_multiple_warned = 1; + } + } + + first = 0; + } + + if (seek_point && ctx->cll_sei == BSF_ELEMENT_INSERT) { + H265RawSEIPayload payload = { + .payload_type = HEVC_SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO, + }; + H265RawSEIContentLightLevelInfo *cll = + &payload.payload.content_light_level; + int size; + + data = av_packet_get_side_data(pkt, + AV_PKT_DATA_MASTERING_DISPLAY_METADATA, &size); + if (data) { + av_assert0(size >= sizeof(*clm)); + clm = (AVContentLightMetadata*)data; + ctx->cll_side_data = *clm; + } + + if (ctx->cll_x265_str) { + *cll = ctx->cll_x265; + } else { + ff_cbs_h265_fill_sei_content_light_level(cll, + &ctx->cll_side_data); + } + + err = ff_cbs_h265_add_sei_message(au, &payload, 1); + if (err < 0) + return err; + } + + return 0; +} + static int h265_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, CodedBitstreamFragment *au) { H265MetadataContext *ctx = bsf->priv_data; - int err, i; + int err, i, has_vps, has_sps, seek_point; // If an AUD is present, it must be the first NAL unit. if (au->units[0].type == HEVC_NAL_AUD) { @@ -380,19 +542,48 @@ static int h265_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, if (ctx->level == LEVEL_AUTO && !ctx->level_guess) h265_metadata_guess_level(bsf, au); + has_vps = has_sps = 0; for (i = 0; i < au->nb_units; i++) { if (au->units[i].type == HEVC_NAL_VPS) { + has_vps = 1; err = h265_metadata_update_vps(bsf, au->units[i].content); if (err < 0) return err; } if (au->units[i].type == HEVC_NAL_SPS) { + has_sps = 1; err = h265_metadata_update_sps(bsf, au->units[i].content); if (err < 0) return err; } } + if (pkt) { + // The current packet should be treated as a seek point for + // metadata insertion if any of: + // - It is the first packet in the stream. + // - It contains a VPS or SPS, indicating that a sequence might + // start here. + // - It is marked as containing a key frame. + seek_point = !ctx->done_first_au || has_vps || has_sps || + (pkt->flags & AV_PKT_FLAG_KEY); + } else { + seek_point = 0; + } + + if (ctx->mdcv_sei != BSF_ELEMENT_PASS) { + err = h265_metadata_update_mdcv(bsf, pkt, au, seek_point); + if (err < 0) + return err; + } + if (ctx->cll_sei != BSF_ELEMENT_PASS) { + err = h265_metadata_update_cll(bsf, pkt, au, seek_point); + if (err < 0) + return err; + } + + if (pkt) + ctx->done_first_au = 1; return 0; } @@ -405,6 +596,44 @@ static const CBSBSFType h265_metadata_type = { static int h265_metadata_init(AVBSFContext *bsf) { + H265MetadataContext *ctx = bsf->priv_data; + int ret; + + if (ctx->mdcv_x265_str) { + ret = sscanf(ctx->mdcv_x265_str, + "G(%"SCNu16",%"SCNu16")" + "B(%"SCNu16",%"SCNu16")" + "R(%"SCNu16",%"SCNu16")" + "WP(%"SCNu16",%"SCNu16")" + "L(%"SCNu32",%"SCNu32")", + &ctx->mdcv_x265.display_primaries_x[0], + &ctx->mdcv_x265.display_primaries_y[0], + &ctx->mdcv_x265.display_primaries_x[1], + &ctx->mdcv_x265.display_primaries_y[1], + &ctx->mdcv_x265.display_primaries_x[2], + &ctx->mdcv_x265.display_primaries_y[2], + &ctx->mdcv_x265.white_point_x, + &ctx->mdcv_x265.white_point_y, + &ctx->mdcv_x265.max_display_mastering_luminance, + &ctx->mdcv_x265.min_display_mastering_luminance); + if (ret != 10) { + av_log(bsf, AV_LOG_ERROR, "Failed to parse " + "x265_master_display string: check formatting.\n"); + return AVERROR(EINVAL); + } + } + + if (ctx->cll_x265_str) { + ret = sscanf(ctx->cll_x265_str, "%"SCNu16",%"SCNu16, + &ctx->cll_x265.max_content_light_level, + &ctx->cll_x265.max_pic_average_light_level); + if (ret != 2) { + av_log(bsf, AV_LOG_ERROR, "Failed to parse " + "x265_max_cll string: check formatting.\n"); + return AVERROR(EINVAL); + } + } + return ff_cbs_bsf_init(bsf, &h265_metadata_type); } @@ -484,6 +713,23 @@ static const AVOption h265_metadata_options[] = { { LEVEL("8.5", 255) }, #undef LEVEL + BSF_ELEMENT_OPTIONS_PIRE("mastering_display_colour_volume", + "Mastering display colour volume SEI", + mdcv_sei, "mdcv_sei"), + { "x265_master_display", + "Mastering display colour volume values in x265 format", + OFFSET(mdcv_x265_str), AV_OPT_TYPE_STRING, + { .str = NULL }, .flags = FLAGS }, + + BSF_ELEMENT_OPTIONS_PIRE("content_light_level_information", + "Content light level information SEI", + cll_sei, "cll_sei"), + + { "x265_max_cll", + "Content light level values in x265 format", + OFFSET(cll_x265_str), AV_OPT_TYPE_STRING, + { .str = NULL }, .flags = FLAGS }, + { NULL } };