From patchwork Tue Dec 29 03:56:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: klobliu X-Patchwork-Id: 24673 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 2AC1B449F97 for ; Tue, 29 Dec 2020 06:04:39 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 056DB68A2C0; Tue, 29 Dec 2020 06:04:39 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D892868A2C0 for ; Tue, 29 Dec 2020 06:04:31 +0200 (EET) Received: by mail-pj1-f45.google.com with SMTP id l23so859084pjg.1 for ; Mon, 28 Dec 2020 20:04:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=ZhAboAXzSn73vgaPZBe4DwdjAheRHQzDsmPktss+zHY=; b=F7/+0PSUeyQydS4jLTsTiVRF6hBma0+Mz7arAIZs+V9kGOffJYRKk2NiWJuvzYhZSJ I5WTFAxU2MJ9oxcCN3lEWiZyjJFCqzXdIpdPFyYQPkKmNFDYaCmEBIWpQV5j/8GOurEI lIizlKS6UKT4u9ry/NQzFm8rjEFim7USPBqShVpA6PcfzLF3SuAYfwe06ms9byGaEc7K Qxpz6xGjn3OTNXrVsxwEk3ZkWQq0RUCtyGaCmhtKVkiF/bdjnhMvUbbQXMoHr5V5CUEl aympdFjIZ1VjueC1YnX4ZCObisQedwVE4FGfNp7HaPia2vJ5NKQj3/XBjUqsqIivbxkP J4jA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=ZhAboAXzSn73vgaPZBe4DwdjAheRHQzDsmPktss+zHY=; b=k1uM9JtDqCW6v6niWRBbmlXrZlkSQnzAK6ddQ/YbA86xKPC2zcMJ7OshbOcM10jnqn +qlFikfr6qPBuSDdQzDYoRYkkKxvG2mm9X5W0wCoi6cq9ISkZuuiZ6y2QlHZ/6L8/VJS d30ZOQQnyg0TfM5a0X9ncZZ3Gfr8e/9fAY0mRPo5hqU25dC4FJjQj4k6sOqclMKaTMHJ eZViq40Z4ziROeBy5dgu3RhEEaZ/6D2pcg86TzkzhiiirGXYhpPX7ttJipkV/5xQ/AkZ gz47dE6n12cNzBI1a/3SPkmlHg+MLknteGpTK/M2Nxv0bt2r9X+nyWu4y/DpBM/sN/Z+ 4MVQ== X-Gm-Message-State: AOAM532/AzDlm5m/qMfFRA/Or/OPCfeeHQWysz41ioSSYBajOfEHzKk3 +KG3ymBhcmsjVOpjVChoiVvpDqJo07FryQ== X-Google-Smtp-Source: ABdhPJyBC7RejKYTlPD8o3lwEjnTQKgBt8V4niUP1Maty+tQ6h3qbUNp8SyBwTP82A5Cq/CinM+yTQ== X-Received: by 2002:a17:902:9681:b029:db:fd65:d10e with SMTP id n1-20020a1709029681b02900dbfd65d10emr48007385plp.6.1609214214445; Mon, 28 Dec 2020 19:56:54 -0800 (PST) Received: from klobliu-PC2.tencent.com ([203.205.141.53]) by smtp.gmail.com with ESMTPSA id y16sm36806239pfb.83.2020.12.28.19.56.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 28 Dec 2020 19:56:54 -0800 (PST) From: klobliu To: ffmpeg-devel@ffmpeg.org Date: Tue, 29 Dec 2020 11:56:49 +0800 Message-Id: <20201229035649.5612-1-kloblic@gmail.com> X-Mailer: git-send-email 2.13.3.windows.1 Subject: [FFmpeg-devel] [PATCH v1] avcodec: add hdr sei support for hevc_nvenc X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: klobliu MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: klobliu Signed-off-by: klobliu --- libavcodec/cbs.c | 1 + libavcodec/cbs_h2645.c | 60 +++++++++++++++++++++++++++++++++++++++ libavcodec/cbs_h265.h | 2 ++ libavcodec/h265_metadata_bsf.c | 64 +++++++++++++++++++++++++++++++++++++++++- 4 files changed, 126 insertions(+), 1 deletion(-) diff --git a/libavcodec/cbs.c b/libavcodec/cbs.c index f98531e131..b3db4749bf 100644 --- a/libavcodec/cbs.c +++ b/libavcodec/cbs.c @@ -26,6 +26,7 @@ #include "cbs.h" #include "cbs_internal.h" +#include "cbs_h265.h" static const CodedBitstreamType *cbs_type_table[] = { diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index 434322492c..ca55c4f298 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -1538,6 +1538,66 @@ const CodedBitstreamType ff_cbs_type_h265 = { .close = &cbs_h265_close, }; +int ff_cbs_h265_add_sei_message(CodedBitstreamFragment *au, + H265RawSEIPayload *payload) +{ + H265RawSEI *sei = NULL; + int err, i, k; + + // Use single SEI NAL unit to store HDR info. + for (i = 0; i < au->nb_units; i++) { + if (au->units[i].type == HEVC_NAL_SEI_PREFIX || au->units[i].type == HEVC_NAL_SEI_SUFFIX) { + H265RawSEI * raw_sei = (H265RawSEI *)au->units[i].content; + if (raw_sei) { + for (k = 0; k < raw_sei->payload_count; k++) { + if (raw_sei->payload[k].payload_type == payload->payload_type) { + sei = au->units[i].content; + if (sei->payload_count < H265_MAX_SEI_PAYLOADS) + break; + sei = NULL; + } + } + } + } + } + + if (!sei) { + AVBufferRef *sei_ref; + + sei = av_mallocz(sizeof(*sei)); + if (!sei) { + err = AVERROR(ENOMEM); + goto fail; + } + + sei->nal_unit_header.nal_unit_type = HEVC_NAL_SEI_PREFIX; + sei->nal_unit_header.nuh_layer_id = 0; + sei->nal_unit_header.nuh_temporal_id_plus1 = 1; + sei_ref = av_buffer_create((uint8_t*)sei, sizeof(*sei), + &cbs_h265_free_sei, NULL, 0); + if (!sei_ref) { + av_freep(&sei); + err = AVERROR(ENOMEM); + goto fail; + } + + err = ff_cbs_insert_unit_content(au, i, HEVC_NAL_SEI_PREFIX, + sei, sei_ref); + av_buffer_unref(&sei_ref); + if (err < 0) + goto fail; + } + + memcpy(&sei->payload[sei->payload_count], payload, sizeof(*payload)); + ++sei->payload_count; + + return 0; +fail: + cbs_h265_free_sei_payload(payload); + return err; +} + + int ff_cbs_h264_add_sei_message(CodedBitstreamFragment *au, H264RawSEIPayload *payload) { diff --git a/libavcodec/cbs_h265.h b/libavcodec/cbs_h265.h index 15b22bbfd4..cd3824c7fa 100644 --- a/libavcodec/cbs_h265.h +++ b/libavcodec/cbs_h265.h @@ -24,6 +24,7 @@ #include "cbs_h2645.h" #include "hevc.h" +#include "cbs.h" enum { // This limit is arbitrary - it is sufficient for one message of each @@ -746,5 +747,6 @@ typedef struct CodedBitstreamH265Context { const H265RawPPS *active_pps; } CodedBitstreamH265Context; +int ff_cbs_h265_add_sei_message(CodedBitstreamFragment *au, H265RawSEIPayload *payload); #endif /* AVCODEC_CBS_H265_H */ diff --git a/libavcodec/h265_metadata_bsf.c b/libavcodec/h265_metadata_bsf.c index 504a75dac2..91e5df05d2 100644 --- a/libavcodec/h265_metadata_bsf.c +++ b/libavcodec/h265_metadata_bsf.c @@ -25,6 +25,7 @@ #include "cbs_h265.h" #include "hevc.h" #include "h265_profile_level.h" +#include "hevc_sei.h" enum { PASS, @@ -70,6 +71,11 @@ typedef struct H265MetadataContext { int level; int level_guess; int level_warned; + + char * max_cll; + char * mastering_display_color_volume; + int done_first_au; + } H265MetadataContext; @@ -393,7 +399,7 @@ static int h265_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) { H265MetadataContext *ctx = bsf->priv_data; CodedBitstreamFragment *au = &ctx->access_unit; - int err, i; + int err, i, has_sps; err = ff_bsf_get_packet_ref(bsf, pkt); if (err < 0) @@ -471,15 +477,64 @@ static int h265_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) err = h265_metadata_update_sps(bsf, au->units[i].content); if (err < 0) goto fail; + has_sps = 1; } } + if (has_sps || !ctx->done_first_au) { + if (ctx->max_cll) { + H265RawSEIPayload payload = { + .payload_type = HEVC_SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO, + }; + payload.payload_size = 4; + + if (sscanf(ctx->max_cll, "%hu|%hu", + &payload.payload.content_light_level.max_content_light_level, + &payload.payload.content_light_level.max_pic_average_light_level) == 2) { + err = ff_cbs_h265_add_sei_message(au, &payload); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to add HEVC_SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO SEI " + "message to access unit.\n"); + goto fail; + } + } + } + + if (ctx->mastering_display_color_volume) { + H265RawSEIPayload payload = { + .payload_type = HEVC_SEI_TYPE_MASTERING_DISPLAY_INFO, + }; + payload.payload_size = 24; + + if (sscanf(ctx->mastering_display_color_volume, + "G(%hu|%hu)B(%hu|%hu)R(%hu|%hu)WP(%hu|%hu)L(%u|%u)", + &payload.payload.mastering_display.display_primaries_x[0], + &payload.payload.mastering_display.display_primaries_y[0], + &payload.payload.mastering_display.display_primaries_x[1], + &payload.payload.mastering_display.display_primaries_y[1], + &payload.payload.mastering_display.display_primaries_x[2], + &payload.payload.mastering_display.display_primaries_y[2], + &payload.payload.mastering_display.white_point_x, + &payload.payload.mastering_display.white_point_y, + &payload.payload.mastering_display.max_display_mastering_luminance, + &payload.payload.mastering_display.min_display_mastering_luminance) == 10) { + err = ff_cbs_h265_add_sei_message(au, &payload); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to add HEVC_SEI_TYPE_MASTERING_DISPLAY_INFO SEI " + "message to access unit.\n"); + goto fail; + } + } + } + } + err = ff_cbs_write_packet(ctx->output, pkt, au); if (err < 0) { av_log(bsf, AV_LOG_ERROR, "Failed to write packet.\n"); goto fail; } + ctx->done_first_au = 1; err = 0; fail: ff_cbs_fragment_reset(au); @@ -607,6 +662,13 @@ static const AVOption h265_metadata_options[] = { OFFSET(crop_bottom), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, HEVC_MAX_HEIGHT, FLAGS }, + { "max_cll", "Content light level for hdr. example 1000|800", + OFFSET(max_cll), AV_OPT_TYPE_STRING, + { .str = NULL }, .flags = FLAGS }, + { "master_display", "Mastering display color volume for hdr.", + OFFSET(mastering_display_color_volume), AV_OPT_TYPE_STRING, + { .str = NULL }, .flags = FLAGS }, + { "level", "Set level (tables A.6 and A.7)", OFFSET(level), AV_OPT_TYPE_INT, { .i64 = LEVEL_UNSET }, LEVEL_UNSET, 0xff, FLAGS, "level" },