From patchwork Wed Jul 26 07:15:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 42986 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:7528:b0:130:ccc6:6c4b with SMTP id r40csp341108pzd; Wed, 26 Jul 2023 00:16:32 -0700 (PDT) X-Google-Smtp-Source: APBJJlFbgbRz1bI+BFpiXX8bZvtxEcT6r467B5KO6FLdd35xPAypYxGES5okEQLdGwecp9UFaSus X-Received: by 2002:a50:ef0e:0:b0:51e:1a51:d414 with SMTP id m14-20020a50ef0e000000b0051e1a51d414mr796496eds.32.1690355792308; Wed, 26 Jul 2023 00:16:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690355792; cv=none; d=google.com; s=arc-20160816; b=SI+svNvmbrLkJzlwcW4bArIkAiNzk2lWw9O9A4RkYD5V/BCTbsQ2ATgerapBSdD9A5 TviNA0xYqfqftbeqUb4duPD6oKJclzsM5TnwwyNK9h4QXFH/+qaSne4yI4uO6D9Iasak u3J75dKslnR8PNSBNlDg/kmylISzB/ruSZg8D0DyidhX34+OmhdLBAGCBwrfMGVlFYyQ UyosA510p12VQFWVoQKElAxmxdlJqmOl+G/Wivxnf8RYeQ1JQXkrAV7jn+/4j8NHN5uy cysV44VTnEAj5iH8CzKHK5W1tPztpwNA3hBd0x9Ifk5e1O6vvRQKo1tudjIih/6nLycm wZIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=S5TcFHp+EeUR+JsmQeNj8kxmxjrcKPxSzj81Zb80KPA=; fh=Tt4aWcAPplwJLYowQh9cIyedJjsIqBA8GlwfHZgTuYo=; b=gZEsa8XiQIqoTDLCTN4S47u0G5MYlF/UPSJHvZ5m5kuuzvxx/cdt5izLgEKF3sFIjn Cr+jORI5dtNVzlidyHm1ueHjL4cX+gTnvOwMUyIKsCR594kvVRgmLUYWjlKKN2f9ERfS ZiY76tlbRLHzS97tAszITzgY9mu3rPjYVViOr8ou+kqGeFFJ3xxrzKgqawxOQmARxOdk TJwe59d3/9LgqKlkiwcYtXIWOiWfPW9uh/C04lra5lpFBY176lo+4Ax28wWpNNINIa9/ orQGx+G3WS3fDZ6FAKPVI+6fzx0I7vze66u5Oj9qxjs8r7lz+4/RJ9f8N/CS+DLgPNRj riAA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=I6ULo7TR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s26-20020aa7d79a000000b005224ce45f0esi2208474edq.396.2023.07.26.00.16.31; Wed, 26 Jul 2023 00:16:32 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=I6ULo7TR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 102DD68C918; Wed, 26 Jul 2023 10:16:14 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1BEDE68C912 for ; Wed, 26 Jul 2023 10:16:05 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690355772; x=1721891772; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+YZIzP/ZtdCJP/Txjq2onFXGU+Fl6UtftA24IUnF81A=; b=I6ULo7TRpWA8yNu1XLu17UcVXSr9ykvSpFI+xAOxrNVDfMnTiO75eyEN h5iiTbnHZpGFw+uVIr2DKpgXBGvt3ZPj9vMKUmMiLATRDoB8Z4+YX8rzc klCt3XKc5EhZkCezPsxxFGEwN0xvucMWbkXdSbjVWIxPkHuUxL07bA3L0 SLWpF95ADlxqoaeq+e8W3D+WnljlIG/oIoG5eUiwqZfSUJYr9Gp2T992u jyJFuXBkBhL2W87BqRAr4vc8CI4Yu0gP5gqDen3zrqYcBYXF0k2LmN0dB rkrGpeSTT8WNpm/xZuKOJHtYPIxmMgTY098bdpfsnuQUaA0WNLwWs+Izi A==; X-IronPort-AV: E=McAfee;i="6600,9927,10782"; a="347551677" X-IronPort-AV: E=Sophos;i="6.01,231,1684825200"; d="scan'208";a="347551677" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jul 2023 00:15:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10782"; a="756106898" X-IronPort-AV: E=Sophos;i="6.01,231,1684825200"; d="scan'208";a="756106898" Received: from xhh-tgl64.sh.intel.com ([10.238.2.19]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jul 2023 00:15:58 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 26 Jul 2023 15:15:22 +0800 Message-Id: <20230726071522.38149-4-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230726071522.38149-1-haihao.xiang@intel.com> References: <20230726071522.38149-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/4] lavfi/vf_vpp_qsv: perform conversion from HDR to SDR X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Wyty5bIb85Y9 From: Haihao Xiang option tonemap is added to disable / enable tonemapping. By default tonemapping is not performed. Signed-off-by: Haihao Xiang --- libavfilter/vf_vpp_qsv.c | 88 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) diff --git a/libavfilter/vf_vpp_qsv.c b/libavfilter/vf_vpp_qsv.c index 795b859de1..615ff5b0d4 100644 --- a/libavfilter/vf_vpp_qsv.c +++ b/libavfilter/vf_vpp_qsv.c @@ -31,6 +31,7 @@ #include "libavutil/hwcontext_qsv.h" #include "libavutil/pixdesc.h" #include "libavutil/mathematics.h" +#include "libavutil/mastering_display_metadata.h" #include "formats.h" #include "internal.h" @@ -63,6 +64,9 @@ typedef struct VPPContext{ mfxExtVideoSignalInfo invsi_conf; /** Video signal info attached on the output frame */ mfxExtVideoSignalInfo outvsi_conf; + /** HDR parameters attached on the input frame */ + mfxExtMasteringDisplayColourVolume mdcv_conf; + mfxExtContentLightLevelInfo clli_conf; #endif /** @@ -118,6 +122,7 @@ typedef struct VPPContext{ int has_passthrough; /* apply pass through mode if possible */ int field_rate; /* Generate output at frame rate or field rate for deinterlace mode, 0: frame, 1: field */ + int tonemap; /* 1: perform tonemapping if the input has HDR metadata, 0: always disable tonemapping */ } VPPContext; static const char *const var_names[] = { @@ -389,6 +394,10 @@ static int vpp_set_frame_ext_params(AVFilterContext *ctx, const AVFrame *in, AVF VPPContext *vpp = ctx->priv; QSVVPPContext *qsvvpp = &vpp->qsv; mfxExtVideoSignalInfo invsi_conf, outvsi_conf; + mfxExtMasteringDisplayColourVolume mdcv_conf; + mfxExtContentLightLevelInfo clli_conf; + AVFrameSideData *sd; + int tm = 0; fp->num_ext_buf = 0; @@ -405,6 +414,73 @@ static int vpp_set_frame_ext_params(AVFilterContext *ctx, const AVFrame *in, AVF invsi_conf.MatrixCoefficients = (in->colorspace == AVCOL_SPC_UNSPECIFIED) ? AVCOL_SPC_BT709 : in->colorspace; invsi_conf.ColourDescriptionPresent = 1; + memset(&mdcv_conf, 0, sizeof(mfxExtMasteringDisplayColourVolume)); + sd = av_frame_get_side_data(in, AV_FRAME_DATA_MASTERING_DISPLAY_METADATA); + if (vpp->tonemap && sd) { + AVMasteringDisplayMetadata *mdm = (AVMasteringDisplayMetadata *)sd->data; + + if (mdm->has_primaries && mdm->has_luminance) { + const int mapping[3] = {1, 2, 0}; + const int chroma_den = 50000; + const int luma_den = 10000; + int i; + + mdcv_conf.Header.BufferId = MFX_EXTBUFF_MASTERING_DISPLAY_COLOUR_VOLUME_IN; + mdcv_conf.Header.BufferSz = sizeof(mfxExtMasteringDisplayColourVolume); + + for (i = 0; i < 3; i++) { + const int j = mapping[i]; + + mdcv_conf.DisplayPrimariesX[i] = + FFMIN(lrint(chroma_den * + av_q2d(mdm->display_primaries[j][0])), + chroma_den); + mdcv_conf.DisplayPrimariesY[i] = + FFMIN(lrint(chroma_den * + av_q2d(mdm->display_primaries[j][1])), + chroma_den); + } + + mdcv_conf.WhitePointX = + FFMIN(lrint(chroma_den * av_q2d(mdm->white_point[0])), + chroma_den); + mdcv_conf.WhitePointY = + FFMIN(lrint(chroma_den * av_q2d(mdm->white_point[1])), + chroma_den); + + /* MaxDisplayMasteringLuminance is in the unit of 1 nits however + * MinDisplayMasteringLuminance is in the unit of 0.0001 nits + */ + mdcv_conf.MaxDisplayMasteringLuminance = + lrint(av_q2d(mdm->max_luminance)); + mdcv_conf.MinDisplayMasteringLuminance = + lrint(luma_den * av_q2d(mdm->min_luminance)); + tm = 1; + } + } + + memset(&clli_conf, 0, sizeof(mfxExtContentLightLevelInfo)); + sd = av_frame_get_side_data(in, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL); + if (vpp->tonemap && sd) { + AVContentLightMetadata *clm = (AVContentLightMetadata *)sd->data; + + clli_conf.Header.BufferId = MFX_EXTBUFF_CONTENT_LIGHT_LEVEL_INFO; + clli_conf.Header.BufferSz = sizeof(mfxExtContentLightLevelInfo); + clli_conf.MaxContentLightLevel = FFMIN(clm->MaxCLL, 65535); + clli_conf.MaxPicAverageLightLevel = FFMIN(clm->MaxFALL, 65535); + tm = 1; + } + + if (tm) { + av_frame_remove_side_data(out, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL); + av_frame_remove_side_data(out, AV_FRAME_DATA_MASTERING_DISPLAY_METADATA); + + out->color_primaries = AVCOL_PRI_BT709; + out->color_trc = AVCOL_TRC_BT709; + out->colorspace = AVCOL_SPC_BT709; + out->color_range = AVCOL_RANGE_MPEG; + } + if (vpp->color_range != AVCOL_RANGE_UNSPECIFIED) out->color_range = vpp->color_range; if (vpp->color_primaries != AVCOL_PRI_UNSPECIFIED) @@ -424,6 +500,8 @@ static int vpp_set_frame_ext_params(AVFilterContext *ctx, const AVFrame *in, AVF outvsi_conf.ColourDescriptionPresent = 1; if (memcmp(&vpp->invsi_conf, &invsi_conf, sizeof(mfxExtVideoSignalInfo)) || + memcmp(&vpp->mdcv_conf, &mdcv_conf, sizeof(mfxExtMasteringDisplayColourVolume)) || + memcmp(&vpp->clli_conf, &clli_conf, sizeof(mfxExtContentLightLevelInfo)) || memcmp(&vpp->outvsi_conf, &outvsi_conf, sizeof(mfxExtVideoSignalInfo))) { vpp->invsi_conf = invsi_conf; fp->ext_buf[fp->num_ext_buf++] = (mfxExtBuffer*)&vpp->invsi_conf; @@ -431,6 +509,13 @@ static int vpp_set_frame_ext_params(AVFilterContext *ctx, const AVFrame *in, AVF vpp->outvsi_conf = outvsi_conf; fp->ext_buf[fp->num_ext_buf++] = (mfxExtBuffer*)&vpp->outvsi_conf; + vpp->mdcv_conf = mdcv_conf; + if (mdcv_conf.Header.BufferId) + fp->ext_buf[fp->num_ext_buf++] = (mfxExtBuffer*)&vpp->mdcv_conf; + + vpp->clli_conf = clli_conf; + if (clli_conf.Header.BufferId) + fp->ext_buf[fp->num_ext_buf++] = (mfxExtBuffer*)&vpp->clli_conf; } #endif @@ -622,6 +707,7 @@ static int config_output(AVFilterLink *outlink) vpp->color_primaries != AVCOL_PRI_UNSPECIFIED || vpp->color_transfer != AVCOL_TRC_UNSPECIFIED || vpp->color_matrix != AVCOL_SPC_UNSPECIFIED || + vpp->tonemap || !vpp->has_passthrough) return ff_qsvvpp_init(ctx, ¶m); else { @@ -829,6 +915,8 @@ static const AVOption vpp_options[] = { { "out_color_transfer", "Output color transfer characteristics", OFFSET(color_transfer_str), AV_OPT_TYPE_STRING, { .str = NULL }, .flags = FLAGS }, + {"tonemap", "Perform tonemapping (0=disable tonemapping, 1=perform tonemapping if the input has HDR metadata)", OFFSET(tonemap), AV_OPT_TYPE_INT, {.i64 = 0 }, 0, 1, .flags = FLAGS}, + { NULL } };