From patchwork Fri Jan 1 21:35:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24745 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id EAACC44AE1C for ; Fri, 1 Jan 2021 23:40:22 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6C1DC68AA35; Fri, 1 Jan 2021 23:38:19 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E698368A995 for ; Fri, 1 Jan 2021 23:38:14 +0200 (EET) Received: by mail-wr1-f43.google.com with SMTP id c5so22915389wrp.6 for ; Fri, 01 Jan 2021 13:38:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=8tMn8FLzLIfoLCdUD+tONTzl97sZtu7BjyIOxs4KRkM=; b=HJERVPLqubgdwzXFa+jKRD2DEc42ay3gtBWaJM4jCQMjC1hR+0SeiHmv6BZv8zfecs TbQ3i/zDDVD6wZLQ5CYn6ymoAO9KOOxmYy7gOZ32B4ffpLbbErS3nriQn5L8Ds/CitLI yQVm7YIunyBokErDpNCq5FoKhmvVAdzg9EbyOxd+bsrzWGDR07yfAEuy74qCFqzZLOU2 J5RhTtBnxUEzwl44fw7aN+Wm6EOswbIr9PqrxToPOPWYfvyaSCIlr2hXEItrKcTAgIVR YAR30xz6MiGncDFrYdEnS5M8fys5akbtpLc1EIviQ3gVEFvMveikbiJtFkPbt6uZ3Cka cz6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8tMn8FLzLIfoLCdUD+tONTzl97sZtu7BjyIOxs4KRkM=; b=WJeuOzZEaEUb20f/K3EmrYY2Ar5d6Yd+P1AfNo0VK7+bS5ByvcdJNfcvntTQo2+zEG 9wo5Xyxm13LTEBoTyWdvAF7NyhzPF/z1MhcxzSRz6WBwMobN8C5PhtTGdhMyBBH2y46q wcQyG8SRgvQF3Geq4ZNOc6zr8kJuuDU2Aqnla0FEf8pA3XHw5sWBjeQOvYxhjDvDWKbY ZlVEpgPyKcyAqYsSoRjiLXq2RTX2v28oErxgD881fCN3y59EAFi4T/wVED+TABDF0CnH wUB4xjPyOHFdEptG20Mw3qGnXXub4zN7mmfyMAWZdHJmKRKABX01cxA1miBPhX1ZVvA1 vgLw== X-Gm-Message-State: AOAM532oMylXUGeFmFTVMIAx5nwaESYdd8Vt3RqiIPhclhBxb3iveAi/ URNZNeFmGIm6JLvqDV/Cyz5OPrCISy8YMw== X-Google-Smtp-Source: ABdhPJw7S+AZZxmZcRKjPLGsS04rFBX3n3jNK6oq50tRxgiIyyjryAkzjFVxK3+mllXs66yJDFvBpQ== X-Received: by 2002:a5d:5146:: with SMTP id u6mr69720924wrt.46.1609537094237; Fri, 01 Jan 2021 13:38:14 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.13 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:13 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:36 +0000 Message-Id: <20210101213537.169546-27-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 26/27] vaapi_encode_h265: Ensure that HDR metadata is included with seek points X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Save the most recent HDR side data structures and use them when generating an IDR or CRA frame. --- libavcodec/vaapi_encode_h265.c | 93 ++++++++++++---------------------- 1 file changed, 32 insertions(+), 61 deletions(-) diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 09d5a8b5cb..75ebb63e74 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -30,6 +30,7 @@ #include "avcodec.h" #include "cbs.h" #include "cbs_h265.h" +#include "cbs_metadata.h" #include "h265_profile_level.h" #include "hevc.h" #include "hevc_sei.h" @@ -70,13 +71,15 @@ typedef struct VAAPIEncodeH265Context { H265RawPPS raw_pps; H265RawSlice raw_slice; - SEIRawMasteringDisplayColourVolume sei_mastering_display; - SEIRawContentLightLevelInfo sei_content_light_level; - CodedBitstreamContext *cbc; CodedBitstreamFragment current_access_unit; int aud_needed; int sei_needed; + + int have_mastering_display; + AVMasteringDisplayMetadata mastering_display; + int have_content_light_level; + AVContentLightMetadata content_light_level; } VAAPIEncodeH265Context; @@ -200,17 +203,17 @@ static int vaapi_encode_h265_write_extra_header(AVCodecContext *avctx, } if (priv->sei_needed & SEI_MASTERING_DISPLAY) { - err = ff_cbs_sei_add_message(priv->cbc, au, 1, - HEVC_SEI_TYPE_MASTERING_DISPLAY_INFO, - &priv->sei_mastering_display, NULL); + err = ff_cbs_insert_metadata(priv->cbc, au, + CBS_METADATA_MASTERING_DISPLAY, + &priv->mastering_display); if (err < 0) goto fail; } if (priv->sei_needed & SEI_CONTENT_LIGHT_LEVEL) { - err = ff_cbs_sei_add_message(priv->cbc, au, 1, - HEVC_SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO, - &priv->sei_content_light_level, NULL); + err = ff_cbs_insert_metadata(priv->cbc, au, + CBS_METADATA_CONTENT_LIGHT_LEVEL, + &priv->content_light_level); if (err < 0) goto fail; } @@ -746,72 +749,40 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, priv->sei_needed = 0; - // Only look for the metadata on I/IDR frame on the output. We - // may force an IDR frame on the output where the medadata gets - // changed on the input frame. - if ((priv->sei & SEI_MASTERING_DISPLAY) && - (pic->type == PICTURE_TYPE_I || pic->type == PICTURE_TYPE_IDR)) { + // We check for new metadata on every frame, but only include the + // SEI with IRAP frames. + if (priv->sei & SEI_MASTERING_DISPLAY) { AVFrameSideData *sd = av_frame_get_side_data(pic->input_image, AV_FRAME_DATA_MASTERING_DISPLAY_METADATA); if (sd) { - AVMasteringDisplayMetadata *mdm = - (AVMasteringDisplayMetadata *)sd->data; - - // SEI is needed when both the primaries and luminance are set - if (mdm->has_primaries && mdm->has_luminance) { - SEIRawMasteringDisplayColourVolume *mdcv = - &priv->sei_mastering_display; - const int mapping[3] = {1, 2, 0}; - const int chroma_den = 50000; - const int luma_den = 10000; - - for (i = 0; i < 3; i++) { - const int j = mapping[i]; - mdcv->display_primaries_x[i] = - FFMIN(lrint(chroma_den * - av_q2d(mdm->display_primaries[j][0])), - chroma_den); - mdcv->display_primaries_y[i] = - FFMIN(lrint(chroma_den * - av_q2d(mdm->display_primaries[j][1])), - chroma_den); - } - - mdcv->white_point_x = - FFMIN(lrint(chroma_den * av_q2d(mdm->white_point[0])), - chroma_den); - mdcv->white_point_y = - FFMIN(lrint(chroma_den * av_q2d(mdm->white_point[1])), - chroma_den); - - mdcv->max_display_mastering_luminance = - lrint(luma_den * av_q2d(mdm->max_luminance)); - mdcv->min_display_mastering_luminance = - FFMIN(lrint(luma_den * av_q2d(mdm->min_luminance)), - mdcv->max_display_mastering_luminance); - - priv->sei_needed |= SEI_MASTERING_DISPLAY; - } + av_assert0(sd->size >= sizeof(priv->mastering_display)); + priv->have_mastering_display = 1; + memcpy(&priv->mastering_display, sd->data, + sizeof(priv->mastering_display)); + } + + if (priv->have_mastering_display && + (pic->type == PICTURE_TYPE_IDR || pic->type == PICTURE_TYPE_I)) { + priv->sei_needed |= SEI_MASTERING_DISPLAY; } } - if ((priv->sei & SEI_CONTENT_LIGHT_LEVEL) && - (pic->type == PICTURE_TYPE_I || pic->type == PICTURE_TYPE_IDR)) { + if (priv->sei & SEI_CONTENT_LIGHT_LEVEL) { AVFrameSideData *sd = av_frame_get_side_data(pic->input_image, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL); if (sd) { - AVContentLightMetadata *clm = - (AVContentLightMetadata *)sd->data; - SEIRawContentLightLevelInfo *clli = - &priv->sei_content_light_level; - - clli->max_content_light_level = FFMIN(clm->MaxCLL, 65535); - clli->max_pic_average_light_level = FFMIN(clm->MaxFALL, 65535); + av_assert0(sd->size >= sizeof(priv->content_light_level)); + priv->have_mastering_display = 1; + memcpy(&priv->content_light_level, sd->data, + sizeof(priv->content_light_level)); + } + if (priv->have_content_light_level && + (pic->type == PICTURE_TYPE_IDR || pic->type == PICTURE_TYPE_I)) { priv->sei_needed |= SEI_CONTENT_LIGHT_LEVEL; } }