From patchwork Sun Feb 21 19:51:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25876 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 03491449DB5 for ; Sun, 21 Feb 2021 21:53:52 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DC4DE68A9BE; Sun, 21 Feb 2021 21:53:51 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 62FD068A857 for ; Sun, 21 Feb 2021 21:53:41 +0200 (EET) Received: by mail-wr1-f53.google.com with SMTP id 7so16925892wrz.0 for ; Sun, 21 Feb 2021 11:53:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=nffE39tD0t8oYzOgkDi/f3pMhV/FkjvYbdYEM2ppLR4=; b=sbbYOHkoClnR9XVKqTHCd9yXiEig4zWWlw/PhKlEzdrPjt3WN5EzGSBmzsyI/EQ0Vi 0/Tr3YGu5uLfixvBrNRnHqs5QKDQSdEHA5LOdqEpBCTccDvX3oh/CNXXRFBywP2YDNUj qVkO4qYrFa0ZyPmad0RGfayRo9QnwthEqczK1VCDECNpsfaBicwKA8P/D7yA92+zXF6g oIP86lMBNmGAM22eYUPbSMZF4LrapTL8rwoU7CYvCtXFh+KX5+KLKpYmrmS1tFuOB5XO cVdWfFYtJAGMQTxGMQ+pynQ4353VqHsti+pwWQySO1mbG8NCBL1FyiIddxstBBzpLVMM yzxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nffE39tD0t8oYzOgkDi/f3pMhV/FkjvYbdYEM2ppLR4=; b=qoXV337Go/Dz2w0a/7SCXu8coSzDsGmNwSW7ufAaDDJgn5UZ/FLt8Wm5PBUWc1EyaI EiocMYgYJfhJdzMmb1cRXITTOP95Bt7+Ud7GHPlTN/hQbXWG/JNGOr5VHimD2i3OZyzN 2lgv3r3+m3thm+be5EHYKVbWq8yG8EIMCrq/LVO05d5lgDQiHCE8gvnh4LP041nbpBGS ot/QGQ63jUnGhTLS9n0Mkx1cUbOz3jP3FoJZAXvcCu+th2Iq41XY0KzBdZlWD72BQFne qgTnXllP2JMqJ3BEAlwQppX/+Jnsw6KhbPyu1PG67c/Ss2zCS/zEEOJuwMY/gpj+d+y9 1OSg== X-Gm-Message-State: AOAM533J6mrgwY3WSDnKroQmX32MhiOUDbhgQTQx/JHnRkiqB0Hj6tap wxJ84J77nQrYnM2HMsdwI0IXnIHpWttvwA== X-Google-Smtp-Source: ABdhPJyuvYlUDqgcIXkncbzly88yR448rq4SXOJIvG6XLJfTSnnSJ0/p45wJuCXhMPuD5FVgVwKNOw== X-Received: by 2002:adf:e444:: with SMTP id t4mr7326214wrm.97.1613937221094; Sun, 21 Feb 2021 11:53:41 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:40 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:23 +0000 Message-Id: <20210221195125.1901683-16-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210221195125.1901683-1-sw@jkqxz.net> References: <20210221195125.1901683-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 16/18] vaapi_encode_h265: Ensure that HDR metadata is included with seek points X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Save the most recent HDR side data structures and use them when generating an IDR or CRA frame. --- libavcodec/vaapi_encode_h265.c | 93 ++++++++++++---------------------- 1 file changed, 32 insertions(+), 61 deletions(-) diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 33acbf23f2..75ebb63e74 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -30,6 +30,7 @@ #include "avcodec.h" #include "cbs.h" #include "cbs_h265.h" +#include "cbs_metadata.h" #include "h265_profile_level.h" #include "hevc.h" #include "hevc_sei.h" @@ -70,13 +71,15 @@ typedef struct VAAPIEncodeH265Context { H265RawPPS raw_pps; H265RawSlice raw_slice; - SEIRawMasteringDisplayColourVolume sei_mastering_display; - SEIRawContentLightLevelInfo sei_content_light_level; - CodedBitstreamContext *cbc; CodedBitstreamFragment current_access_unit; int aud_needed; int sei_needed; + + int have_mastering_display; + AVMasteringDisplayMetadata mastering_display; + int have_content_light_level; + AVContentLightMetadata content_light_level; } VAAPIEncodeH265Context; @@ -200,17 +203,17 @@ static int vaapi_encode_h265_write_extra_header(AVCodecContext *avctx, } if (priv->sei_needed & SEI_MASTERING_DISPLAY) { - err = ff_cbs_sei_add_message(priv->cbc, au, 1, - SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME, - &priv->sei_mastering_display, NULL); + err = ff_cbs_insert_metadata(priv->cbc, au, + CBS_METADATA_MASTERING_DISPLAY, + &priv->mastering_display); if (err < 0) goto fail; } if (priv->sei_needed & SEI_CONTENT_LIGHT_LEVEL) { - err = ff_cbs_sei_add_message(priv->cbc, au, 1, - SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO, - &priv->sei_content_light_level, NULL); + err = ff_cbs_insert_metadata(priv->cbc, au, + CBS_METADATA_CONTENT_LIGHT_LEVEL, + &priv->content_light_level); if (err < 0) goto fail; } @@ -746,72 +749,40 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, priv->sei_needed = 0; - // Only look for the metadata on I/IDR frame on the output. We - // may force an IDR frame on the output where the medadata gets - // changed on the input frame. - if ((priv->sei & SEI_MASTERING_DISPLAY) && - (pic->type == PICTURE_TYPE_I || pic->type == PICTURE_TYPE_IDR)) { + // We check for new metadata on every frame, but only include the + // SEI with IRAP frames. + if (priv->sei & SEI_MASTERING_DISPLAY) { AVFrameSideData *sd = av_frame_get_side_data(pic->input_image, AV_FRAME_DATA_MASTERING_DISPLAY_METADATA); if (sd) { - AVMasteringDisplayMetadata *mdm = - (AVMasteringDisplayMetadata *)sd->data; - - // SEI is needed when both the primaries and luminance are set - if (mdm->has_primaries && mdm->has_luminance) { - SEIRawMasteringDisplayColourVolume *mdcv = - &priv->sei_mastering_display; - const int mapping[3] = {1, 2, 0}; - const int chroma_den = 50000; - const int luma_den = 10000; - - for (i = 0; i < 3; i++) { - const int j = mapping[i]; - mdcv->display_primaries_x[i] = - FFMIN(lrint(chroma_den * - av_q2d(mdm->display_primaries[j][0])), - chroma_den); - mdcv->display_primaries_y[i] = - FFMIN(lrint(chroma_den * - av_q2d(mdm->display_primaries[j][1])), - chroma_den); - } - - mdcv->white_point_x = - FFMIN(lrint(chroma_den * av_q2d(mdm->white_point[0])), - chroma_den); - mdcv->white_point_y = - FFMIN(lrint(chroma_den * av_q2d(mdm->white_point[1])), - chroma_den); - - mdcv->max_display_mastering_luminance = - lrint(luma_den * av_q2d(mdm->max_luminance)); - mdcv->min_display_mastering_luminance = - FFMIN(lrint(luma_den * av_q2d(mdm->min_luminance)), - mdcv->max_display_mastering_luminance); - - priv->sei_needed |= SEI_MASTERING_DISPLAY; - } + av_assert0(sd->size >= sizeof(priv->mastering_display)); + priv->have_mastering_display = 1; + memcpy(&priv->mastering_display, sd->data, + sizeof(priv->mastering_display)); + } + + if (priv->have_mastering_display && + (pic->type == PICTURE_TYPE_IDR || pic->type == PICTURE_TYPE_I)) { + priv->sei_needed |= SEI_MASTERING_DISPLAY; } } - if ((priv->sei & SEI_CONTENT_LIGHT_LEVEL) && - (pic->type == PICTURE_TYPE_I || pic->type == PICTURE_TYPE_IDR)) { + if (priv->sei & SEI_CONTENT_LIGHT_LEVEL) { AVFrameSideData *sd = av_frame_get_side_data(pic->input_image, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL); if (sd) { - AVContentLightMetadata *clm = - (AVContentLightMetadata *)sd->data; - SEIRawContentLightLevelInfo *clli = - &priv->sei_content_light_level; - - clli->max_content_light_level = FFMIN(clm->MaxCLL, 65535); - clli->max_pic_average_light_level = FFMIN(clm->MaxFALL, 65535); + av_assert0(sd->size >= sizeof(priv->content_light_level)); + priv->have_mastering_display = 1; + memcpy(&priv->content_light_level, sd->data, + sizeof(priv->content_light_level)); + } + if (priv->have_content_light_level && + (pic->type == PICTURE_TYPE_IDR || pic->type == PICTURE_TYPE_I)) { priv->sei_needed |= SEI_CONTENT_LIGHT_LEVEL; } }