From patchwork Wed Jul 26 07:15:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 42983 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:7528:b0:130:ccc6:6c4b with SMTP id r40csp340918pzd; Wed, 26 Jul 2023 00:16:07 -0700 (PDT) X-Google-Smtp-Source: APBJJlFAfKCmmqcMbUzTZLCCUP5U9MKs+mQFPIcZK+7wGBwpXlDWp7nZyoq4M3G/ZlfLbA1jg2GX X-Received: by 2002:a05:6402:386:b0:521:ad49:8493 with SMTP id o6-20020a056402038600b00521ad498493mr1344313edv.6.1690355767349; Wed, 26 Jul 2023 00:16:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690355767; cv=none; d=google.com; s=arc-20160816; b=sB/s30Qx6QraqU+U2AJgSlh/fy8HUtC0G4MLczawiM0J8QcH+p8mnbajksNY75XxNs hOUJGxiU5kx/72Wlk8VrAbj1p+FND8LfpDzvaatnr7EakSBlDcCBBmZMqUHcLIfaNGjC wx8J6SpgvRDFpwwoVTzd0maVq7Xltn7BiN05Vu+M4BpKB//+By8tVhXfV8HTnZdGQKAm la54Y1l+KdftAm/ctlPj3Qs0rbWemBC2xttV8aV95U8Sv2nGUilIhkdw3ibfeqv0KTcz B1xcvHrhv2q3ZDLeLI3hbNvDzES9B+IzbehRHDVGERBsf9+NTLh4nnbw+KVM2fPB9kYL abTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=z52cJxS2qG7AGqZsuUJN6pSR/ldbhjUiDu7oCKhDDrk=; fh=Tt4aWcAPplwJLYowQh9cIyedJjsIqBA8GlwfHZgTuYo=; b=HdwQwf5BQCpYLJypwv26KOrmM3n6WN6ZdIEtvlxFBHJLLLC2UWwMWNjL3+cufqsRhf bWB/Nq92gSP2PB8zH57DCovmeA4paJEziHzRN+cAPbsDQklL9T7a0jI5R8iFNVzihZLt MCON+yttgCSZEA/d2XM183fsKya/khUwA7CDqQ0o8N480Du9fnsyyxVLdf453TyVoOY/ DhcjECcIH1zsNBVC6kkyMkUr0VlqGcwhD8lFwDPfviC8o0FC4TWrHeCIhP3Uu5vfw2iG fIpE+cnvHkoaEKxLY/PZfec5uTX7mPnDTU4zMZ2ZM8IGzb7X88sP6P4KIBR8KS4qekMC RhXQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=UWOYoKgw; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f5-20020a056402150500b00522307b6ff7si3612972edw.428.2023.07.26.00.16.07; Wed, 26 Jul 2023 00:16:07 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=UWOYoKgw; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 06B9568C8FD; Wed, 26 Jul 2023 10:16:05 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 665C468C6CD for ; Wed, 26 Jul 2023 10:15:57 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690355763; x=1721891763; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=ocCstuf3YRBZPZHf3zSwSPeHin/p4CNl+w1QwXquLWA=; b=UWOYoKgwvHMn74dwdS8fJh7sU+sSbkaVRFbGyrqTXJmrF4TaAGpkwdBW O0GXhRW3eWZPlncXvxeucPN3JbkoX5V06Lj9zMiWkWW/RvyB0lV7Ki4XJ hDCBqpKiEAxQyicQC2szWQNZtj7UISwcb95/AzpJ5YzFp6+q3nMur2U3F +hkq//QMlrLmAcDHZ9foGDt8docwGQVLkK7aOoQnP8hNjTDbBkAqK0I4v WmQhn/LS/Bs1ZaTpm8rM95zgkRzq00io+zkBYRCIfqi5eAWwAd92IjbeP J2MfU3Sw3h0DRo8ttarxS1lxUy03YlDoit4sU3V/Q1kU/uNi1GcJWoHfx g==; X-IronPort-AV: E=McAfee;i="6600,9927,10782"; a="347551658" X-IronPort-AV: E=Sophos;i="6.01,231,1684825200"; d="scan'208";a="347551658" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jul 2023 00:15:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10782"; a="756106873" X-IronPort-AV: E=Sophos;i="6.01,231,1684825200"; d="scan'208";a="756106873" Received: from xhh-tgl64.sh.intel.com ([10.238.2.19]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jul 2023 00:15:52 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 26 Jul 2023 15:15:19 +0800 Message-Id: <20230726071522.38149-1-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/4] lavfi/qsvvpp: add set_frame_ext_params callback X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: CvLW9kc2rO1c From: Haihao Xiang This allows we add mfxExtBuffer per frame later. Signed-off-by: Haihao Xiang --- libavfilter/qsvvpp.c | 67 +++++++++++++++++++++++++++++++++----------- libavfilter/qsvvpp.h | 10 +++++++ 2 files changed, 61 insertions(+), 16 deletions(-) diff --git a/libavfilter/qsvvpp.c b/libavfilter/qsvvpp.c index a03de05d9c..3c8dfea16a 100644 --- a/libavfilter/qsvvpp.c +++ b/libavfilter/qsvvpp.c @@ -731,6 +731,11 @@ static int init_vpp_session(AVFilterContext *avctx, QSVVPPContext *s) return 0; } +static int set_frame_ext_params_null(AVFilterContext *ctx, const AVFrame *in, AVFrame *out, QSVVPPFrameParam *fp) +{ + return 0; +} + int ff_qsvvpp_init(AVFilterContext *avctx, QSVVPPParam *param) { int i; @@ -742,6 +747,10 @@ int ff_qsvvpp_init(AVFilterContext *avctx, QSVVPPParam *param) s->filter_frame = ff_filter_frame; s->out_sw_format = param->out_sw_format; + s->set_frame_ext_params = param->set_frame_ext_params; + if (!s->set_frame_ext_params) + s->set_frame_ext_params = set_frame_ext_params_null; + /* create the vpp session */ ret = init_vpp_session(avctx, s); if (ret < 0) @@ -868,27 +877,53 @@ failed: static int qsvvpp_init_vpp_session(AVFilterContext *avctx, QSVVPPContext *s, const QSVFrame *in, QSVFrame *out) { int ret; + mfxExtBuffer *ext_param[QSVVPP_MAX_FRAME_EXTBUFS]; + QSVVPPFrameParam fp = { 0, ext_param }; - if (s->vpp_initted) - return 0; + ret = s->set_frame_ext_params(avctx, in->frame, out->frame, &fp); + if (ret) + return ret; - s->vpp_param.vpp.In.PicStruct = in->surface.Info.PicStruct; - s->vpp_param.vpp.Out.PicStruct = out->surface.Info.PicStruct; + if (fp.num_ext_buf) { + av_freep(&s->ext_buffers); + s->nb_ext_buffers = s->nb_seq_buffers + fp.num_ext_buf; - /* Query VPP params again, including params for frame */ - ret = MFXVideoVPP_Query(s->session, &s->vpp_param, &s->vpp_param); - if (ret < 0) - return ff_qsvvpp_print_error(avctx, ret, "Error querying VPP params"); - else if (ret > 0) - ff_qsvvpp_print_warning(avctx, ret, "Warning When querying VPP params"); + s->ext_buffers = av_calloc(s->nb_ext_buffers, sizeof(*s->ext_buffers)); + if (!s->ext_buffers) + return AVERROR(ENOMEM); - ret = MFXVideoVPP_Init(s->session, &s->vpp_param); - if (ret < 0) - return ff_qsvvpp_print_error(avctx, ret, "Failed to create a qsvvpp"); - else if (ret > 0) - ff_qsvvpp_print_warning(avctx, ret, "Warning When creating qsvvpp"); + memcpy(&s->ext_buffers[0], s->seq_buffers, s->nb_seq_buffers * sizeof(*s->seq_buffers)); + memcpy(&s->ext_buffers[s->nb_seq_buffers], fp.ext_buf, fp.num_ext_buf * sizeof(*fp.ext_buf)); + s->vpp_param.ExtParam = s->ext_buffers; + s->vpp_param.NumExtParam = s->nb_ext_buffers; + } + + if (!s->vpp_initted) { + s->vpp_param.vpp.In.PicStruct = in->surface.Info.PicStruct; + s->vpp_param.vpp.Out.PicStruct = out->surface.Info.PicStruct; + + /* Query VPP params again, including params for frame */ + ret = MFXVideoVPP_Query(s->session, &s->vpp_param, &s->vpp_param); + if (ret < 0) + return ff_qsvvpp_print_error(avctx, ret, "Error querying VPP params"); + else if (ret > 0) + ff_qsvvpp_print_warning(avctx, ret, "Warning When querying VPP params"); + + ret = MFXVideoVPP_Init(s->session, &s->vpp_param); + if (ret < 0) + return ff_qsvvpp_print_error(avctx, ret, "Failed to create a qsvvpp"); + else if (ret > 0) + ff_qsvvpp_print_warning(avctx, ret, "Warning When creating qsvvpp"); - s->vpp_initted = 1; + s->vpp_initted = 1; + } else if (fp.num_ext_buf) { + ret = MFXVideoVPP_Reset(s->session, &s->vpp_param); + if (ret < 0) { + ret = ff_qsvvpp_print_error(avctx, ret, "Failed to reset session for qsvvpp"); + return ret; + } else if (ret > 0) + ff_qsvvpp_print_warning(avctx, ret, "Warning When resetting session for qsvvpp"); + } return 0; } diff --git a/libavfilter/qsvvpp.h b/libavfilter/qsvvpp.h index fba5f037d4..4eea7a46c7 100644 --- a/libavfilter/qsvvpp.h +++ b/libavfilter/qsvvpp.h @@ -52,11 +52,20 @@ typedef struct QSVFrame { int queued; } QSVFrame; +#define QSVVPP_MAX_FRAME_EXTBUFS 8 + +typedef struct QSVVPPFrameParam { + /* To fill with MFX enhanced filter configurations */ + int num_ext_buf; + mfxExtBuffer **ext_buf; +} QSVVPPFrameParam; + typedef struct QSVVPPContext { const AVClass *class; mfxSession session; int (*filter_frame) (AVFilterLink *outlink, AVFrame *frame); /**< callback */ + int (*set_frame_ext_params)(AVFilterContext *ctx, const AVFrame *in, AVFrame *out, QSVVPPFrameParam *fp); /**< callbak */ enum AVPixelFormat out_sw_format; /**< Real output format */ mfxVideoParam vpp_param; mfxFrameInfo *frame_infos; /**< frame info for each input */ @@ -101,6 +110,7 @@ typedef struct QSVVPPCrop { typedef struct QSVVPPParam { /* default is ff_filter_frame */ int (*filter_frame)(AVFilterLink *outlink, AVFrame *frame); + int (*set_frame_ext_params)(AVFilterContext *ctx, const AVFrame *in, AVFrame *out, QSVVPPFrameParam *fp); /**< callbak */ /* To fill with MFX enhanced filter configurations */ int num_ext_buf; From patchwork Wed Jul 26 07:15:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 42984 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:7528:b0:130:ccc6:6c4b with SMTP id r40csp340983pzd; Wed, 26 Jul 2023 00:16:16 -0700 (PDT) X-Google-Smtp-Source: APBJJlGv56sf0F+hWZPHT87sDE8dy/UtluEyH67Aw3nZeYAVdFBGHgdJ0DcQXBTHFbBoIcK7krKs X-Received: by 2002:a17:907:763c:b0:991:37d2:c9f0 with SMTP id jy28-20020a170907763c00b0099137d2c9f0mr985325ejc.68.1690355776156; Wed, 26 Jul 2023 00:16:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690355776; cv=none; d=google.com; s=arc-20160816; b=Eyy6LD+FASZG9nZeQFYIn5iQgC/bRncXRNQ8MDpsc5ERUVAmSvh3b3ha3vuG0BW/H4 z+IjaISnh6BXM0QibqreYM6xU13jNSb8UocSSEZd3FDgZyViTTcy9wUJFy5TAEkm/vGQ cKw1QOmsMvx68DF6UUYicScxCCh5mesv9KbW8CMn0jytkygqipnO++pyP8KYtv3mz17y ziC79Wf/mduamVFWiZrDK2X4UFy1Fxrx6PZkfYtosVaN+jRjQB4jB1XHczNPawnxcy7x 1A1U3VJI1kjsTq9wUP+bFWikMUTRYptN2YC07kZ8c8mVbL6OCft8j1w/sSPvyhyOYSpP STIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=+TBVjr+Z6DEeWRljgKg4IkYGdLP2zb933lBpw2xfIc4=; fh=Tt4aWcAPplwJLYowQh9cIyedJjsIqBA8GlwfHZgTuYo=; b=awZ6Ohkp8/94csmwUeN3K41QcYG0bgTViEGAyjX/Eix1SRRT7YU9hjrXwCOwgr1yub dIfN1ov4r7Jst1yvWTnqVw1WOD4cxSVvW6KhM4fHDRdDAI0LiBOvCNBYrzR1aoyUHbs2 CyI/7quBWWcvoQRdE3kfsuwKOzvIl3bsxJfcpVpZGgUOb+MpqC0Pk8sBGq2DDaQmBD+A aoepgP45SMI5mYbZ/e3z06zlshVwb6ScLFi+RWxHQ1uW7K0kOy2ahuKc278xEC2xIIck ncE1JZqY8Otr5yfAjPORvR0iPAM/uCVOs3+sTqNdoesYzdLLmciSI6MISEOZEC5eYl4V JNXQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=DCLcLz2E; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id r16-20020a170906a21000b0099b49483268si9423591ejy.301.2023.07.26.00.16.15; Wed, 26 Jul 2023 00:16:16 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=DCLcLz2E; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0937C68C90A; Wed, 26 Jul 2023 10:16:09 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 04A7768C6CD for ; Wed, 26 Jul 2023 10:16:00 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690355767; x=1721891767; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=R8U144JWrZxFWzIPY/NkXTtg67HQZdTei4yWy6+2r1I=; b=DCLcLz2EW9QyvIc+GpaO1sUtViT3UXtdwT1tBbIFZLAYsHkGB6zCdV1S txEHyPcij7ZvecHa652mdALYvDeT0la6ZwsDVD+1RZg3+8/z7+E3s6E1R 9uTt7TKNgVDN7et0qvmm/pNfQ7RvHPKPomr6Ws5ezTaglYkgO6f9AIMdx EGCDHFFvv4CVZWR5nswLoJmQNfWE6DXEc+6sH4hrPDlr/kjH0TcUphD40 ELiN2ObLwoqXs1XsGPUtA+vvky43rKYFIqBq331ZKzWdFO1c8ZWZeTXxZ EnripI4P9hjIWiWnn4KkQo5Va9OHCuaTA/vc+G5sV+m798chV3FjImVX2 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10782"; a="347551668" X-IronPort-AV: E=Sophos;i="6.01,231,1684825200"; d="scan'208";a="347551668" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jul 2023 00:15:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10782"; a="756106885" X-IronPort-AV: E=Sophos;i="6.01,231,1684825200"; d="scan'208";a="756106885" Received: from xhh-tgl64.sh.intel.com ([10.238.2.19]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jul 2023 00:15:55 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 26 Jul 2023 15:15:20 +0800 Message-Id: <20230726071522.38149-2-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230726071522.38149-1-haihao.xiang@intel.com> References: <20230726071522.38149-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/4] lavfi/vf_vpp_qsv: take input color properties into account X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: m/yjf8UkYkjW From: Haihao Xiang Signed-off-by: Haihao Xiang --- libavfilter/vf_vpp_qsv.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/libavfilter/vf_vpp_qsv.c b/libavfilter/vf_vpp_qsv.c index 334a86551b..bf4264efc5 100644 --- a/libavfilter/vf_vpp_qsv.c +++ b/libavfilter/vf_vpp_qsv.c @@ -58,6 +58,10 @@ typedef struct VPPContext{ mfxExtVPPRotation rotation_conf; mfxExtVPPMirroring mirroring_conf; mfxExtVPPScaling scale_conf; +#if QSV_ONEVPL + /** Video signal info attached on the input frame */ + mfxExtVideoSignalInfo invsi_conf; +#endif /** * New dimensions. Special values are: @@ -344,6 +348,37 @@ static mfxStatus get_mfx_version(const AVFilterContext *ctx, mfxVersion *mfx_ver return MFXQueryVersion(device_hwctx->session, mfx_version); } +static int vpp_set_frame_ext_params(AVFilterContext *ctx, const AVFrame *in, AVFrame *out, QSVVPPFrameParam *fp) +{ +#if QSV_ONEVPL + VPPContext *vpp = ctx->priv; + QSVVPPContext *qsvvpp = &vpp->qsv; + mfxExtVideoSignalInfo invsi_conf; + + fp->num_ext_buf = 0; + + if (!in || + !QSV_RUNTIME_VERSION_ATLEAST(qsvvpp->ver, 2, 0)) + return 0; + + memset(&invsi_conf, 0, sizeof(mfxExtVideoSignalInfo)); + invsi_conf.Header.BufferId = MFX_EXTBUFF_VIDEO_SIGNAL_INFO_IN; + invsi_conf.Header.BufferSz = sizeof(mfxExtVideoSignalInfo); + invsi_conf.VideoFullRange = (in->color_range == AVCOL_RANGE_JPEG); + invsi_conf.ColourPrimaries = (in->color_primaries == AVCOL_PRI_UNSPECIFIED) ? AVCOL_PRI_BT709 : in->color_primaries; + invsi_conf.TransferCharacteristics = (in->color_trc == AVCOL_TRC_UNSPECIFIED) ? AVCOL_TRC_BT709 : in->color_trc; + invsi_conf.MatrixCoefficients = (in->colorspace == AVCOL_SPC_UNSPECIFIED) ? AVCOL_SPC_BT709 : in->colorspace; + invsi_conf.ColourDescriptionPresent = 1; + + if (memcmp(&vpp->invsi_conf, &invsi_conf, sizeof(mfxExtVideoSignalInfo))) { + vpp->invsi_conf = invsi_conf; + fp->ext_buf[fp->num_ext_buf++] = (mfxExtBuffer*)&vpp->invsi_conf; + } +#endif + + return 0; +} + static int config_output(AVFilterLink *outlink) { AVFilterContext *ctx = outlink->src; @@ -361,6 +396,7 @@ static int config_output(AVFilterLink *outlink) outlink->time_base = av_inv_q(vpp->framerate); param.filter_frame = NULL; + param.set_frame_ext_params = vpp_set_frame_ext_params; param.num_ext_buf = 0; param.ext_buf = ext_buf; From patchwork Wed Jul 26 07:15:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 42985 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:7528:b0:130:ccc6:6c4b with SMTP id r40csp341033pzd; Wed, 26 Jul 2023 00:16:24 -0700 (PDT) X-Google-Smtp-Source: APBJJlFxze0b97NugGt8pZDQho42HIwoCkxR2N8gka4xOeBZFNJSxd5hkh7amp2dQNIvhKYta3Wx X-Received: by 2002:a17:906:5594:b0:957:1df0:9cbf with SMTP id y20-20020a170906559400b009571df09cbfmr907548ejp.19.1690355784031; Wed, 26 Jul 2023 00:16:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690355784; cv=none; d=google.com; s=arc-20160816; b=r4CQCt1D2Jez66TpTgYVBY5WxbyMPv5xjttbl7pWuMS/SIp1qft1L2RCWzFoaD+Nja QSSZ2mvrxAFW8PD44qlzCDtnKGmGXcwYMadofgNFRpNKCVJM6uW21GHUyGNw7CiBXBzE AkzMXnSVBkb0sqgvyIsMwE6UUp17Q+flMJMV5PPG8IwyB6jadFBljcGbMszIYR3wsxS6 d3cWQWVcar36Pv8XaQp83uj8Sn5pB0xqtpYiSWzhJF/+eyi/b7TQeDxXTUz2VD2TGk5O rZy3DVKqijghYe3ETZV/Uz22O0ysPSoZK30OJMkxaNsD9AHioTveYgdMOQD58vNjd2kK 0KpA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=UwTpAOd+FoufH22JHprSfoblHHrIiuL72Iw58fdmEf0=; fh=Tt4aWcAPplwJLYowQh9cIyedJjsIqBA8GlwfHZgTuYo=; b=q9/J35ZOrRvd+b/skdA8wfBkvI5msOFMCbnswgHcpRz1YgOz0YgA1JIp/fJxOovLjR 3sewYc1SPppc3uUKoFxl1d6Z2rZ8ikgB984KwLch1+YLgqPnf0MO0NhZ0ugKnP/W1F4t 2G3lXKzJ1b1vKCPlqselLINXB8RCOXnaOx/mC+a28U3eZYoRCA/1ULOpvRbm5p9x9fTK k90NRL2GJaOFlPEo+AqbP8zFWWk/kz7oL3cK5xwcAzab5S6Gbrzn4rwBCNQmd0nGo9pu PouC+gjpcZCkKZeK/kiYE1nk0F5WQl5ZAelOUHb9FCwtT2AT27w3pFXzUz+Zn7xz/Zym kU1Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=F2mqz+pn; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id n21-20020a170906119500b00992e21b04aasi9093425eja.720.2023.07.26.00.16.23; Wed, 26 Jul 2023 00:16:24 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=F2mqz+pn; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 06F4768C911; Wed, 26 Jul 2023 10:16:11 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9D31568C90D for ; Wed, 26 Jul 2023 10:16:04 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690355769; x=1721891769; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1DOb6GUn18ILppVTE7VbC4wMneuXlondYxVBHBLHNqQ=; b=F2mqz+pn3ETntm23JCjes0mGLX9vQBD3K02HF18U6tjcBe4TQQBILMnv 7Mr3ejvofeaMqcbNpEdQnhF5ys3abxsyI5pg5lHXM0m5NdBwgXcE9J6RD /jRooeKfI3oQXpuS6p3kKQwWjLYWNgtR/jVswGIETIy2LPZj3tenyd9Sw uMzHhFs8sq+6+/IfWrg7Ug8uhOsafhrVtcvmG9EAUcZpw0bG+dkDREryc ySIuQYGnv66nUStNOOXrbP6v3ZYG1Z/vEdrW+mPPwnPkY7xlsWsZBpqN5 edpK/fgKk8BrJ4lnxKqoOKeMtYXIH0wYF9Bmwyyw6RzVMgiMBrkkBwWiO Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10782"; a="347551669" X-IronPort-AV: E=Sophos;i="6.01,231,1684825200"; d="scan'208";a="347551669" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jul 2023 00:15:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10782"; a="756106891" X-IronPort-AV: E=Sophos;i="6.01,231,1684825200"; d="scan'208";a="756106891" Received: from xhh-tgl64.sh.intel.com ([10.238.2.19]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jul 2023 00:15:57 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 26 Jul 2023 15:15:21 +0800 Message-Id: <20230726071522.38149-3-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230726071522.38149-1-haihao.xiang@intel.com> References: <20230726071522.38149-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/4] lavfi/vf_vpp_qsv: set color properties for output X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: o2Uh2Mcm6i1K From: Haihao Xiang User may set color range / matrix coefficient set / primaries / transfer characteristics for output. Signed-off-by: Haihao Xiang --- libavfilter/vf_vpp_qsv.c | 90 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 87 insertions(+), 3 deletions(-) diff --git a/libavfilter/vf_vpp_qsv.c b/libavfilter/vf_vpp_qsv.c index bf4264efc5..795b859de1 100644 --- a/libavfilter/vf_vpp_qsv.c +++ b/libavfilter/vf_vpp_qsv.c @@ -61,6 +61,8 @@ typedef struct VPPContext{ #if QSV_ONEVPL /** Video signal info attached on the input frame */ mfxExtVideoSignalInfo invsi_conf; + /** Video signal info attached on the output frame */ + mfxExtVideoSignalInfo outvsi_conf; #endif /** @@ -104,6 +106,16 @@ typedef struct VPPContext{ char *ow, *oh; char *output_format_str; + /** The color properties for output */ + char *color_primaries_str; + char *color_transfer_str; + char *color_matrix_str; + + int color_range; + enum AVColorPrimaries color_primaries; + enum AVColorTransferCharacteristic color_transfer; + enum AVColorSpace color_matrix; + int has_passthrough; /* apply pass through mode if possible */ int field_rate; /* Generate output at frame rate or field rate for deinterlace mode, 0: frame, 1: field */ } VPPContext; @@ -231,6 +243,11 @@ static av_cold int vpp_preinit(AVFilterContext *ctx) vpp->contrast = 1.0; vpp->transpose = -1; + vpp->color_range = AVCOL_RANGE_UNSPECIFIED; + vpp->color_primaries = AVCOL_PRI_UNSPECIFIED; + vpp->color_transfer = AVCOL_TRC_UNSPECIFIED; + vpp->color_matrix = AVCOL_SPC_UNSPECIFIED; + vpp->has_passthrough = 1; return 0; @@ -250,6 +267,24 @@ static av_cold int vpp_init(AVFilterContext *ctx) } } +#define STRING_OPTION(var_name, func_name, default_value) do { \ + if (vpp->var_name ## _str) { \ + int var = av_ ## func_name ## _from_name(vpp->var_name ## _str); \ + if (var < 0) { \ + av_log(ctx, AV_LOG_ERROR, "Invalid %s.\n", #var_name); \ + return AVERROR(EINVAL); \ + } \ + vpp->var_name = var; \ + } else { \ + vpp->var_name = default_value; \ + } \ + } while (0) + + STRING_OPTION(color_primaries, color_primaries, AVCOL_PRI_UNSPECIFIED); + STRING_OPTION(color_transfer, color_transfer, AVCOL_TRC_UNSPECIFIED); + STRING_OPTION(color_matrix, color_space, AVCOL_SPC_UNSPECIFIED); + +#undef STRING_OPTION return 0; } @@ -353,11 +388,11 @@ static int vpp_set_frame_ext_params(AVFilterContext *ctx, const AVFrame *in, AVF #if QSV_ONEVPL VPPContext *vpp = ctx->priv; QSVVPPContext *qsvvpp = &vpp->qsv; - mfxExtVideoSignalInfo invsi_conf; + mfxExtVideoSignalInfo invsi_conf, outvsi_conf; fp->num_ext_buf = 0; - if (!in || + if (!in || !out || !QSV_RUNTIME_VERSION_ATLEAST(qsvvpp->ver, 2, 0)) return 0; @@ -370,9 +405,32 @@ static int vpp_set_frame_ext_params(AVFilterContext *ctx, const AVFrame *in, AVF invsi_conf.MatrixCoefficients = (in->colorspace == AVCOL_SPC_UNSPECIFIED) ? AVCOL_SPC_BT709 : in->colorspace; invsi_conf.ColourDescriptionPresent = 1; - if (memcmp(&vpp->invsi_conf, &invsi_conf, sizeof(mfxExtVideoSignalInfo))) { + if (vpp->color_range != AVCOL_RANGE_UNSPECIFIED) + out->color_range = vpp->color_range; + if (vpp->color_primaries != AVCOL_PRI_UNSPECIFIED) + out->color_primaries = vpp->color_primaries; + if (vpp->color_transfer != AVCOL_TRC_UNSPECIFIED) + out->color_trc = vpp->color_transfer; + if (vpp->color_matrix != AVCOL_SPC_UNSPECIFIED) + out->colorspace = vpp->color_matrix; + + memset(&outvsi_conf, 0, sizeof(mfxExtVideoSignalInfo)); + outvsi_conf.Header.BufferId = MFX_EXTBUFF_VIDEO_SIGNAL_INFO_OUT; + outvsi_conf.Header.BufferSz = sizeof(mfxExtVideoSignalInfo); + outvsi_conf.VideoFullRange = (out->color_range == AVCOL_RANGE_JPEG); + outvsi_conf.ColourPrimaries = (out->color_primaries == AVCOL_PRI_UNSPECIFIED) ? AVCOL_PRI_BT709 : out->color_primaries; + outvsi_conf.TransferCharacteristics = (out->color_trc == AVCOL_TRC_UNSPECIFIED) ? AVCOL_TRC_BT709 : out->color_trc; + outvsi_conf.MatrixCoefficients = (out->colorspace == AVCOL_SPC_UNSPECIFIED) ? AVCOL_SPC_BT709 : out->colorspace; + outvsi_conf.ColourDescriptionPresent = 1; + + if (memcmp(&vpp->invsi_conf, &invsi_conf, sizeof(mfxExtVideoSignalInfo)) || + memcmp(&vpp->outvsi_conf, &outvsi_conf, sizeof(mfxExtVideoSignalInfo))) { vpp->invsi_conf = invsi_conf; fp->ext_buf[fp->num_ext_buf++] = (mfxExtBuffer*)&vpp->invsi_conf; + + vpp->outvsi_conf = outvsi_conf; + fp->ext_buf[fp->num_ext_buf++] = (mfxExtBuffer*)&vpp->outvsi_conf; + } #endif @@ -560,6 +618,10 @@ static int config_output(AVFilterLink *outlink) if (vpp->use_frc || vpp->use_crop || vpp->deinterlace || vpp->denoise || vpp->detail || vpp->procamp || vpp->rotate || vpp->hflip || inlink->w != outlink->w || inlink->h != outlink->h || in_format != vpp->out_format || + vpp->color_range != AVCOL_RANGE_UNSPECIFIED || + vpp->color_primaries != AVCOL_PRI_UNSPECIFIED || + vpp->color_transfer != AVCOL_TRC_UNSPECIFIED || + vpp->color_matrix != AVCOL_SPC_UNSPECIFIED || !vpp->has_passthrough) return ff_qsvvpp_init(ctx, ¶m); else { @@ -745,6 +807,28 @@ static const AVOption vpp_options[] = { { "field", "Output at field rate (one frame of output for each field)", 0, AV_OPT_TYPE_CONST, { .i64 = 1 }, 0, 0, FLAGS, "rate" }, + { "out_range", "Output color range", + OFFSET(color_range), AV_OPT_TYPE_INT, { .i64 = AVCOL_RANGE_UNSPECIFIED }, + AVCOL_RANGE_UNSPECIFIED, AVCOL_RANGE_JPEG, FLAGS, "range" }, + { "full", "Full range", + 0, AV_OPT_TYPE_CONST, { .i64 = AVCOL_RANGE_JPEG }, 0, 0, FLAGS, "range" }, + { "limited", "Limited range", + 0, AV_OPT_TYPE_CONST, { .i64 = AVCOL_RANGE_MPEG }, 0, 0, FLAGS, "range" }, + { "jpeg", "Full range", + 0, AV_OPT_TYPE_CONST, { .i64 = AVCOL_RANGE_JPEG }, 0, 0, FLAGS, "range" }, + { "mpeg", "Limited range", + 0, AV_OPT_TYPE_CONST, { .i64 = AVCOL_RANGE_MPEG }, 0, 0, FLAGS, "range" }, + { "tv", "Limited range", + 0, AV_OPT_TYPE_CONST, { .i64 = AVCOL_RANGE_MPEG }, 0, 0, FLAGS, "range" }, + { "pc", "Full range", + 0, AV_OPT_TYPE_CONST, { .i64 = AVCOL_RANGE_JPEG }, 0, 0, FLAGS, "range" }, + { "out_color_matrix", "Output color matrix coefficient set", + OFFSET(color_matrix_str), AV_OPT_TYPE_STRING, { .str = NULL }, .flags = FLAGS }, + { "out_color_primaries", "Output color primaries", + OFFSET(color_primaries_str), AV_OPT_TYPE_STRING, { .str = NULL }, .flags = FLAGS }, + { "out_color_transfer", "Output color transfer characteristics", + OFFSET(color_transfer_str), AV_OPT_TYPE_STRING, { .str = NULL }, .flags = FLAGS }, + { NULL } }; From patchwork Wed Jul 26 07:15:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 42986 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:7528:b0:130:ccc6:6c4b with SMTP id r40csp341108pzd; Wed, 26 Jul 2023 00:16:32 -0700 (PDT) X-Google-Smtp-Source: APBJJlFbgbRz1bI+BFpiXX8bZvtxEcT6r467B5KO6FLdd35xPAypYxGES5okEQLdGwecp9UFaSus X-Received: by 2002:a50:ef0e:0:b0:51e:1a51:d414 with SMTP id m14-20020a50ef0e000000b0051e1a51d414mr796496eds.32.1690355792308; Wed, 26 Jul 2023 00:16:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690355792; cv=none; d=google.com; s=arc-20160816; b=SI+svNvmbrLkJzlwcW4bArIkAiNzk2lWw9O9A4RkYD5V/BCTbsQ2ATgerapBSdD9A5 TviNA0xYqfqftbeqUb4duPD6oKJclzsM5TnwwyNK9h4QXFH/+qaSne4yI4uO6D9Iasak u3J75dKslnR8PNSBNlDg/kmylISzB/ruSZg8D0DyidhX34+OmhdLBAGCBwrfMGVlFYyQ UyosA510p12VQFWVoQKElAxmxdlJqmOl+G/Wivxnf8RYeQ1JQXkrAV7jn+/4j8NHN5uy cysV44VTnEAj5iH8CzKHK5W1tPztpwNA3hBd0x9Ifk5e1O6vvRQKo1tudjIih/6nLycm wZIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=S5TcFHp+EeUR+JsmQeNj8kxmxjrcKPxSzj81Zb80KPA=; fh=Tt4aWcAPplwJLYowQh9cIyedJjsIqBA8GlwfHZgTuYo=; b=gZEsa8XiQIqoTDLCTN4S47u0G5MYlF/UPSJHvZ5m5kuuzvxx/cdt5izLgEKF3sFIjn Cr+jORI5dtNVzlidyHm1ueHjL4cX+gTnvOwMUyIKsCR594kvVRgmLUYWjlKKN2f9ERfS ZiY76tlbRLHzS97tAszITzgY9mu3rPjYVViOr8ou+kqGeFFJ3xxrzKgqawxOQmARxOdk TJwe59d3/9LgqKlkiwcYtXIWOiWfPW9uh/C04lra5lpFBY176lo+4Ax28wWpNNINIa9/ orQGx+G3WS3fDZ6FAKPVI+6fzx0I7vze66u5Oj9qxjs8r7lz+4/RJ9f8N/CS+DLgPNRj riAA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=I6ULo7TR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s26-20020aa7d79a000000b005224ce45f0esi2208474edq.396.2023.07.26.00.16.31; Wed, 26 Jul 2023 00:16:32 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=I6ULo7TR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 102DD68C918; Wed, 26 Jul 2023 10:16:14 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1BEDE68C912 for ; Wed, 26 Jul 2023 10:16:05 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690355772; x=1721891772; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+YZIzP/ZtdCJP/Txjq2onFXGU+Fl6UtftA24IUnF81A=; b=I6ULo7TRpWA8yNu1XLu17UcVXSr9ykvSpFI+xAOxrNVDfMnTiO75eyEN h5iiTbnHZpGFw+uVIr2DKpgXBGvt3ZPj9vMKUmMiLATRDoB8Z4+YX8rzc klCt3XKc5EhZkCezPsxxFGEwN0xvucMWbkXdSbjVWIxPkHuUxL07bA3L0 SLWpF95ADlxqoaeq+e8W3D+WnljlIG/oIoG5eUiwqZfSUJYr9Gp2T992u jyJFuXBkBhL2W87BqRAr4vc8CI4Yu0gP5gqDen3zrqYcBYXF0k2LmN0dB rkrGpeSTT8WNpm/xZuKOJHtYPIxmMgTY098bdpfsnuQUaA0WNLwWs+Izi A==; X-IronPort-AV: E=McAfee;i="6600,9927,10782"; a="347551677" X-IronPort-AV: E=Sophos;i="6.01,231,1684825200"; d="scan'208";a="347551677" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jul 2023 00:15:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10782"; a="756106898" X-IronPort-AV: E=Sophos;i="6.01,231,1684825200"; d="scan'208";a="756106898" Received: from xhh-tgl64.sh.intel.com ([10.238.2.19]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jul 2023 00:15:58 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 26 Jul 2023 15:15:22 +0800 Message-Id: <20230726071522.38149-4-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230726071522.38149-1-haihao.xiang@intel.com> References: <20230726071522.38149-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/4] lavfi/vf_vpp_qsv: perform conversion from HDR to SDR X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Wyty5bIb85Y9 From: Haihao Xiang option tonemap is added to disable / enable tonemapping. By default tonemapping is not performed. Signed-off-by: Haihao Xiang --- libavfilter/vf_vpp_qsv.c | 88 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) diff --git a/libavfilter/vf_vpp_qsv.c b/libavfilter/vf_vpp_qsv.c index 795b859de1..615ff5b0d4 100644 --- a/libavfilter/vf_vpp_qsv.c +++ b/libavfilter/vf_vpp_qsv.c @@ -31,6 +31,7 @@ #include "libavutil/hwcontext_qsv.h" #include "libavutil/pixdesc.h" #include "libavutil/mathematics.h" +#include "libavutil/mastering_display_metadata.h" #include "formats.h" #include "internal.h" @@ -63,6 +64,9 @@ typedef struct VPPContext{ mfxExtVideoSignalInfo invsi_conf; /** Video signal info attached on the output frame */ mfxExtVideoSignalInfo outvsi_conf; + /** HDR parameters attached on the input frame */ + mfxExtMasteringDisplayColourVolume mdcv_conf; + mfxExtContentLightLevelInfo clli_conf; #endif /** @@ -118,6 +122,7 @@ typedef struct VPPContext{ int has_passthrough; /* apply pass through mode if possible */ int field_rate; /* Generate output at frame rate or field rate for deinterlace mode, 0: frame, 1: field */ + int tonemap; /* 1: perform tonemapping if the input has HDR metadata, 0: always disable tonemapping */ } VPPContext; static const char *const var_names[] = { @@ -389,6 +394,10 @@ static int vpp_set_frame_ext_params(AVFilterContext *ctx, const AVFrame *in, AVF VPPContext *vpp = ctx->priv; QSVVPPContext *qsvvpp = &vpp->qsv; mfxExtVideoSignalInfo invsi_conf, outvsi_conf; + mfxExtMasteringDisplayColourVolume mdcv_conf; + mfxExtContentLightLevelInfo clli_conf; + AVFrameSideData *sd; + int tm = 0; fp->num_ext_buf = 0; @@ -405,6 +414,73 @@ static int vpp_set_frame_ext_params(AVFilterContext *ctx, const AVFrame *in, AVF invsi_conf.MatrixCoefficients = (in->colorspace == AVCOL_SPC_UNSPECIFIED) ? AVCOL_SPC_BT709 : in->colorspace; invsi_conf.ColourDescriptionPresent = 1; + memset(&mdcv_conf, 0, sizeof(mfxExtMasteringDisplayColourVolume)); + sd = av_frame_get_side_data(in, AV_FRAME_DATA_MASTERING_DISPLAY_METADATA); + if (vpp->tonemap && sd) { + AVMasteringDisplayMetadata *mdm = (AVMasteringDisplayMetadata *)sd->data; + + if (mdm->has_primaries && mdm->has_luminance) { + const int mapping[3] = {1, 2, 0}; + const int chroma_den = 50000; + const int luma_den = 10000; + int i; + + mdcv_conf.Header.BufferId = MFX_EXTBUFF_MASTERING_DISPLAY_COLOUR_VOLUME_IN; + mdcv_conf.Header.BufferSz = sizeof(mfxExtMasteringDisplayColourVolume); + + for (i = 0; i < 3; i++) { + const int j = mapping[i]; + + mdcv_conf.DisplayPrimariesX[i] = + FFMIN(lrint(chroma_den * + av_q2d(mdm->display_primaries[j][0])), + chroma_den); + mdcv_conf.DisplayPrimariesY[i] = + FFMIN(lrint(chroma_den * + av_q2d(mdm->display_primaries[j][1])), + chroma_den); + } + + mdcv_conf.WhitePointX = + FFMIN(lrint(chroma_den * av_q2d(mdm->white_point[0])), + chroma_den); + mdcv_conf.WhitePointY = + FFMIN(lrint(chroma_den * av_q2d(mdm->white_point[1])), + chroma_den); + + /* MaxDisplayMasteringLuminance is in the unit of 1 nits however + * MinDisplayMasteringLuminance is in the unit of 0.0001 nits + */ + mdcv_conf.MaxDisplayMasteringLuminance = + lrint(av_q2d(mdm->max_luminance)); + mdcv_conf.MinDisplayMasteringLuminance = + lrint(luma_den * av_q2d(mdm->min_luminance)); + tm = 1; + } + } + + memset(&clli_conf, 0, sizeof(mfxExtContentLightLevelInfo)); + sd = av_frame_get_side_data(in, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL); + if (vpp->tonemap && sd) { + AVContentLightMetadata *clm = (AVContentLightMetadata *)sd->data; + + clli_conf.Header.BufferId = MFX_EXTBUFF_CONTENT_LIGHT_LEVEL_INFO; + clli_conf.Header.BufferSz = sizeof(mfxExtContentLightLevelInfo); + clli_conf.MaxContentLightLevel = FFMIN(clm->MaxCLL, 65535); + clli_conf.MaxPicAverageLightLevel = FFMIN(clm->MaxFALL, 65535); + tm = 1; + } + + if (tm) { + av_frame_remove_side_data(out, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL); + av_frame_remove_side_data(out, AV_FRAME_DATA_MASTERING_DISPLAY_METADATA); + + out->color_primaries = AVCOL_PRI_BT709; + out->color_trc = AVCOL_TRC_BT709; + out->colorspace = AVCOL_SPC_BT709; + out->color_range = AVCOL_RANGE_MPEG; + } + if (vpp->color_range != AVCOL_RANGE_UNSPECIFIED) out->color_range = vpp->color_range; if (vpp->color_primaries != AVCOL_PRI_UNSPECIFIED) @@ -424,6 +500,8 @@ static int vpp_set_frame_ext_params(AVFilterContext *ctx, const AVFrame *in, AVF outvsi_conf.ColourDescriptionPresent = 1; if (memcmp(&vpp->invsi_conf, &invsi_conf, sizeof(mfxExtVideoSignalInfo)) || + memcmp(&vpp->mdcv_conf, &mdcv_conf, sizeof(mfxExtMasteringDisplayColourVolume)) || + memcmp(&vpp->clli_conf, &clli_conf, sizeof(mfxExtContentLightLevelInfo)) || memcmp(&vpp->outvsi_conf, &outvsi_conf, sizeof(mfxExtVideoSignalInfo))) { vpp->invsi_conf = invsi_conf; fp->ext_buf[fp->num_ext_buf++] = (mfxExtBuffer*)&vpp->invsi_conf; @@ -431,6 +509,13 @@ static int vpp_set_frame_ext_params(AVFilterContext *ctx, const AVFrame *in, AVF vpp->outvsi_conf = outvsi_conf; fp->ext_buf[fp->num_ext_buf++] = (mfxExtBuffer*)&vpp->outvsi_conf; + vpp->mdcv_conf = mdcv_conf; + if (mdcv_conf.Header.BufferId) + fp->ext_buf[fp->num_ext_buf++] = (mfxExtBuffer*)&vpp->mdcv_conf; + + vpp->clli_conf = clli_conf; + if (clli_conf.Header.BufferId) + fp->ext_buf[fp->num_ext_buf++] = (mfxExtBuffer*)&vpp->clli_conf; } #endif @@ -622,6 +707,7 @@ static int config_output(AVFilterLink *outlink) vpp->color_primaries != AVCOL_PRI_UNSPECIFIED || vpp->color_transfer != AVCOL_TRC_UNSPECIFIED || vpp->color_matrix != AVCOL_SPC_UNSPECIFIED || + vpp->tonemap || !vpp->has_passthrough) return ff_qsvvpp_init(ctx, ¶m); else { @@ -829,6 +915,8 @@ static const AVOption vpp_options[] = { { "out_color_transfer", "Output color transfer characteristics", OFFSET(color_transfer_str), AV_OPT_TYPE_STRING, { .str = NULL }, .flags = FLAGS }, + {"tonemap", "Perform tonemapping (0=disable tonemapping, 1=perform tonemapping if the input has HDR metadata)", OFFSET(tonemap), AV_OPT_TYPE_INT, {.i64 = 0 }, 0, 1, .flags = FLAGS}, + { NULL } };