From patchwork Thu May 3 03:07:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 8756 Delivered-To: ffmpegpatchwork@gmail.com Received: by 2002:a02:155:0:0:0:0:0 with SMTP id c82-v6csp1318525jad; Wed, 2 May 2018 20:08:25 -0700 (PDT) X-Google-Smtp-Source: AB8JxZrXPCqaOMijEhkErgFkK/1oRIRTxH6Jgea/mC0+AeGr2uMc2bQXRdmsmHpyvOBH48MRm+We X-Received: by 2002:adf:aa94:: with SMTP id h20-v6mr16339356wrc.149.1525316905846; Wed, 02 May 2018 20:08:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1525316905; cv=none; d=google.com; s=arc-20160816; b=pziUFzTnJ+tJoIL/oxpzaLr0ESstJ2cQ5hAj1PIxHjLsyMOl7/kKSTsgV4fkbw8f4D ZRJRlSmzXMxvLvOijYp8BPyXX5NhIWmJekxmqfz9z0JD0prj4pqfyiOeDXvUv+nDm/7R K/McAPPEAYHMIKvMwhiTQQNIomEAZLGdFASf9vqmkxhbbyixbDtK1MAarGs1syNgXpdR bAsC/6UGh2OUYqvLe/+tHBvyFLo8P9pcaJMXu0Te9Ds3OzgrOAIwywW2q1ASuAoibjt8 Mc+Cc6EINDbihbSG5UbtiYMPsfGAwNLVEwBjO2f3SzFs1/1NhPh5uNiV+Tv8/W+IouYO avnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to:arc-authentication-results; bh=Hoy/tiRTcsOVonzTOM5wB05qkD/Ge71ZO1SNIHdg36M=; b=j/GgqsDR35+jx2qNRFTv8bJdZGR7HFXtcymcdHqYOWGnH6UwyFIEn1osxAELA06cwt f29PtVLSIM2AK3e4F3VCZvHa+GPYXyc2NiMV9CBJLI/O0Gb64ZU++1x/3dcjJ46J5c3O M6OeTrdWwA6cYPQzRtpZ8CJVl79LSYn8ODL1514av+Kqu3D9U6HSNBTveiuU+qMIp+gn zVBmVJmYfo25P0KbzfoYkOLLE+EH6okcHjafhFZoQ5TcY7qOjJAqHb1sCET0Xs01deHc OKTgohfblMtZhaGAI5TALqcRMYpBLsthQjmvcp5bpuu/Z2sh/bq/PXmaEMup/4di4sug x+Cw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id o6si1564582wmi.18.2018.05.02.20.08.25; Wed, 02 May 2018 20:08:25 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D097168A53B; Thu, 3 May 2018 06:07:41 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2FBC768A52F for ; Thu, 3 May 2018 06:07:34 +0300 (EEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 02 May 2018 20:08:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,356,1520924400"; d="scan'208";a="36461699" Received: from xhh-cfl64.sh.intel.com ([10.239.13.24]) by fmsmga007.fm.intel.com with ESMTP; 02 May 2018 20:08:05 -0700 From: Haihao Xiang To: ffmpeg-devel@ffmpeg.org Date: Thu, 3 May 2018 11:07:28 +0800 Message-Id: <20180503030730.8504-2-haihao.xiang@intel.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20180503030730.8504-1-haihao.xiang@intel.com> References: <20180503030730.8504-1-haihao.xiang@intel.com> Subject: [FFmpeg-devel] [PATCH 2/4] vaapi_encode_h265: Insert mastering display colour colume if needed X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang , Mark Thompson MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" '-sei xxx' is added to control SEI insertion, so far only mastering display colour colume is available for testing. v2: use the mastering display parameters from AVMasteringDisplayMetadata, set SEI_MASTERING_DISPLAY to 8 to match the H.264 part and take VAAPIEncodeH265Context::sei_needed as a ORed value so that we needn't check the correspoding SEI message again when writting the header. Signed-off-by: Haihao Xiang --- libavcodec/vaapi_encode_h265.c | 128 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 127 insertions(+), 1 deletion(-) diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 5203c6871d..326fe4fe66 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -24,15 +24,20 @@ #include "libavutil/avassert.h" #include "libavutil/common.h" #include "libavutil/opt.h" +#include "libavutil/mastering_display_metadata.h" #include "avcodec.h" #include "cbs.h" #include "cbs_h265.h" #include "hevc.h" +#include "hevc_sei.h" #include "internal.h" #include "put_bits.h" #include "vaapi_encode.h" +enum { + SEI_MASTERING_DISPLAY = 0x08, +}; typedef struct VAAPIEncodeH265Context { unsigned int ctu_width; @@ -47,6 +52,9 @@ typedef struct VAAPIEncodeH265Context { H265RawSPS sps; H265RawPPS pps; H265RawSlice slice; + H265RawSEI sei; + + H265RawSEIMasteringDiplayColourVolume mastering_display; int64_t last_idr_frame; int pic_order_cnt; @@ -58,6 +66,7 @@ typedef struct VAAPIEncodeH265Context { CodedBitstreamContext *cbc; CodedBitstreamFragment current_access_unit; int aud_needed; + int sei_needed; } VAAPIEncodeH265Context; typedef struct VAAPIEncodeH265Options { @@ -65,6 +74,7 @@ typedef struct VAAPIEncodeH265Options { int aud; int profile; int level; + int sei; } VAAPIEncodeH265Options; @@ -175,6 +185,64 @@ fail: return err; } +static int vaapi_encode_h265_write_extra_header(AVCodecContext *avctx, + VAAPIEncodePicture *pic, + int index, int *type, + char *data, size_t *data_len) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + VAAPIEncodeH265Context *priv = ctx->priv_data; + CodedBitstreamFragment *au = &priv->current_access_unit; + int err, i; + + if (priv->sei_needed) { + if (priv->aud_needed) { + err = vaapi_encode_h265_add_nal(avctx, au, &priv->aud); + if (err < 0) + goto fail; + priv->aud_needed = 0; + } + + memset(&priv->sei, 0, sizeof(priv->sei)); + priv->sei.nal_unit_header = (H265RawNALUnitHeader) { + .nal_unit_type = HEVC_NAL_SEI_PREFIX, + .nuh_layer_id = 0, + .nuh_temporal_id_plus1 = 1, + }; + + i = 0; + + if (priv->sei_needed & SEI_MASTERING_DISPLAY) { + priv->sei.payload[i].payload_type = HEVC_SEI_TYPE_MASTERING_DISPLAY_INFO; + priv->sei.payload[i].payload.mastering_display = priv->mastering_display; + ++i; + } + + priv->sei.payload_count = i; + av_assert0(priv->sei.payload_count > 0); + + err = vaapi_encode_h265_add_nal(avctx, au, &priv->sei); + if (err < 0) + goto fail; + priv->sei_needed = 0; + + err = vaapi_encode_h265_write_access_unit(avctx, data, data_len, au); + if (err < 0) + goto fail; + + ff_cbs_fragment_uninit(priv->cbc, au); + + *type = VAEncPackedHeaderRawData; + return 0; + } else { + return AVERROR_EOF; + } + +fail: + ff_cbs_fragment_uninit(priv->cbc, au); + return err; +} + static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) { VAAPIEncodeContext *ctx = avctx->priv_data; @@ -587,6 +655,53 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, priv->aud_needed = 0; } + priv->sei_needed = 0; + + if ((opt->sei & SEI_MASTERING_DISPLAY) && + (pic->type == PICTURE_TYPE_I || pic->type == PICTURE_TYPE_IDR)) { + AVFrameSideData *sd = + av_frame_get_side_data(pic->input_image, + AV_FRAME_DATA_MASTERING_DISPLAY_METADATA); + + if (sd) { + AVMasteringDisplayMetadata *mdm = + (AVMasteringDisplayMetadata *)sd->data; + + if (mdm->has_primaries && mdm->has_luminance) { + const int mapping[3] = {1, 2, 0}; + const int chroma_den = 50000; + const int luma_den = 10000; + + for (i = 0; i < 3; i++) { + const int j = mapping[i]; + priv->mastering_display.display_primaries_x[i] = + FFMIN(lrint(chroma_den * + av_q2d(mdm->display_primaries[j][0])), + 50000); + priv->mastering_display.display_primaries_y[i] = + FFMIN(lrint(chroma_den * + av_q2d(mdm->display_primaries[j][1])), + 50000); + } + + priv->mastering_display.white_point_x = + FFMIN(lrint(chroma_den * av_q2d(mdm->white_point[0])), + 50000); + priv->mastering_display.white_point_y = + FFMIN(lrint(chroma_den * av_q2d(mdm->white_point[1])), + 50000); + + priv->mastering_display.max_display_mastering_luminance = + lrint(luma_den * av_q2d(mdm->max_luminance)); + priv->mastering_display.min_display_mastering_luminance = + FFMIN(lrint(luma_den * av_q2d(mdm->min_luminance)), + priv->mastering_display.max_display_mastering_luminance); + + priv->sei_needed |= SEI_MASTERING_DISPLAY; + } + } + } + vpic->decoded_curr_pic = (VAPictureHEVC) { .picture_id = pic->recon_surface, .pic_order_cnt = priv->pic_order_cnt, @@ -895,6 +1010,8 @@ static const VAAPIEncodeType vaapi_encode_type_h265 = { .slice_header_type = VAEncPackedHeaderHEVC_Slice, .write_slice_header = &vaapi_encode_h265_write_slice_header, + + .write_extra_header = &vaapi_encode_h265_write_extra_header, }; static av_cold int vaapi_encode_h265_init(AVCodecContext *avctx) @@ -943,7 +1060,8 @@ static av_cold int vaapi_encode_h265_init(AVCodecContext *avctx) ctx->va_packed_headers = VA_ENC_PACKED_HEADER_SEQUENCE | // VPS, SPS and PPS. - VA_ENC_PACKED_HEADER_SLICE; // Slice headers. + VA_ENC_PACKED_HEADER_SLICE | // Slice headers. + VA_ENC_PACKED_HEADER_MISC; // SEI ctx->surface_width = FFALIGN(avctx->width, 16); ctx->surface_height = FFALIGN(avctx->height, 16); @@ -1003,6 +1121,14 @@ static const AVOption vaapi_encode_h265_options[] = { { LEVEL("6.2", 186) }, #undef LEVEL + { "sei", "Set SEI to include", + OFFSET(sei), AV_OPT_TYPE_FLAGS, + { .i64 = SEI_MASTERING_DISPLAY }, + 0, INT_MAX, FLAGS, "sei" }, + { "mastering_display", "Include mastering display colour volume", + 0, AV_OPT_TYPE_CONST, { .i64 = SEI_MASTERING_DISPLAY }, + INT_MIN, INT_MAX, FLAGS, "sei" }, + { NULL }, };