From patchwork Mon Jan 24 08:24:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 33761 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:e71a:0:0:0:0:0 with SMTP id b26csp130532ioh; Mon, 24 Jan 2022 00:26:40 -0800 (PST) X-Google-Smtp-Source: ABdhPJw+C1cDF/frepGhhkAlT82H4C08cRwkCQ7CbdLZH8w8mqtRwM8DnVCljbLNj4tt0ZbVgCdu X-Received: by 2002:a17:906:29cd:: with SMTP id y13mr12330529eje.755.1643012800067; Mon, 24 Jan 2022 00:26:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1643012800; cv=none; d=google.com; s=arc-20160816; b=txSfw2rO94bTzBW7dTN2COTsc+BJCiK5sB6oabpfFYzk7T+Q4sc70JDybPHRlqa/xN 4MpYa6DzgTsoECtf9eVHzWCrIQPjIzF+6epeoK/wRe8fJGJhZuCnkcxYlOqL1VlEctqQ kU/VwJY0qFtGZEQmEG6tOX6CKnhmA0g8oDt6OEpcly2CBYBia88TWlM7WVCRFj4VQLoJ Ed9Akh71Pke4FslZTSzLhmTQhkhWJxQYcNJ5JwgykUJYp26rVKeB8kYOSveCwq12DtWq /yu4KSwb9e6CITPqpd5eJJw+3OAZg0830HGy/ULyjfiWMlGEujYDoTUtFCMAW6jGpiTO kUjg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=xe2v9lzPLrGDBLIS1CBoFe/5qf3WEYFWbqLsX4/Rqs8=; b=eKPa8AKP/GYT81UQbHW1U1Mz89OTn57PnL/fitvgagGAQ6KA6/qZUXH+fPu4E9x83x kaw2Az5KzqPimQgAn/Ioo56NmDDgvyZ8GFG5Icf8szdJsVX0M4E+EA7J50/0xzP/tEXs nJDeBQvAhEppYTNuof4N5I10UnsdtnydiXR8H7MyOqxkEJ5QnRrCXm5jAr4dBE+wTK0R JtI8Aua0dRIj6iaA/4Y1+Fr47/HUZ7LyU5vbcF3HTgjnTg93kV1yqS7ROJGGgKYFhB9m 9ay9fHGsCMhjscKWbbv64lTDTylUH7INcNlVjyj0gumWBbqstP/PDkOayq3eFAw3AGmq q5Iw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=UepUUMDH; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id sh31si1004840ejc.857.2022.01.24.00.26.39; Mon, 24 Jan 2022 00:26:40 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=UepUUMDH; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BE4D268B06E; Mon, 24 Jan 2022 10:26:19 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 55A3868B05B for ; Mon, 24 Jan 2022 10:26:13 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643012778; x=1674548778; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=ySND3NSYh/Ydn9AclsN74THOnZp/DnG4t8ecgOIf6FI=; b=UepUUMDHh8sPxg7yL1mrTDn3mTnuoogh54Q9Xg5FNaUmvXdfLVy8/rCH cxX1Aev/9N3GHyeakiMFqI1dT3EAl8Xs0ZgrrxpKDDN4cOAHxxnPc8J6/ Q0ONvln0G/4yxzkI2ct3QvYokF8jpJNAerPKDq7fvia2gEEJnFaXznd0N cMrBadl4tLsorkccw6BIJaxP1onBNVBzYCupuExP5Rmrku6E0oP2/JqiJ o/Ms33GiKMedncdQzJw1Leu4eC6AXAGy6b/1S1IoHt0tQQv1M6CQlovt3 /NYgvwY2BzAWOj15SD5SHzloUtIjd4oKCEfuo0wFhJgNXdCcb/CBaOT2A A==; X-IronPort-AV: E=McAfee;i="6200,9189,10236"; a="245940008" X-IronPort-AV: E=Sophos;i="5.88,311,1635231600"; d="scan'208";a="245940008" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Jan 2022 00:26:06 -0800 X-IronPort-AV: E=Sophos;i="5.88,311,1635231600"; d="scan'208";a="534128386" Received: from xhh-dg164.sh.intel.com ([10.239.159.167]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Jan 2022 00:26:05 -0800 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Jan 2022 16:24:55 +0800 Message-Id: <20220124082455.30931-3-haihao.xiang@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220124082455.30931-1-haihao.xiang@intel.com> References: <20220124082455.30931-1-haihao.xiang@intel.com> Subject: [FFmpeg-devel] [PATCH v2 3/3] lavc/qsvdec: export AVFilmGrainParams side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: gEm/t4NY7Xrf From: Haihao Xiang When AV_CODEC_EXPORT_DATA_FILM_GRAIN is present, AV1 decoder should disable film grain application and export the corresponding side data Signed-off-by: Haihao Xiang --- libavcodec/qsv_internal.h | 3 ++ libavcodec/qsvdec.c | 91 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 94 insertions(+) diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h index 6a38e87d23..58186ea7ca 100644 --- a/libavcodec/qsv_internal.h +++ b/libavcodec/qsv_internal.h @@ -76,6 +76,9 @@ typedef struct QSVFrame { mfxFrameSurface1 surface; mfxEncodeCtrl enc_ctrl; mfxExtDecodedFrameInfo dec_info; +#if QSV_VERSION_ATLEAST(1, 34) + mfxExtAV1FilmGrainParam av1_film_grain_param; +#endif mfxExtBuffer *ext_param[QSV_MAX_FRAME_EXT_PARAMS]; int num_ext_params; diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index 8b83d5695f..32077ab31a 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -38,6 +38,7 @@ #include "libavutil/pixfmt.h" #include "libavutil/time.h" #include "libavutil/imgutils.h" +#include "libavutil/film_grain_params.h" #include "avcodec.h" #include "internal.h" @@ -404,6 +405,11 @@ static int qsv_decode_header(AVCodecContext *avctx, QSVContext *q, param->ExtParam = q->ext_buffers; param->NumExtParam = q->nb_ext_buffers; +#if QSV_VERSION_ATLEAST(1, 34) + if (QSV_RUNTIME_VERSION_ATLEAST(q->ver, 1, 34) && avctx->codec_id == AV_CODEC_ID_AV1) + param->mfx.FilmGrain = (avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN) ? 0 : param->mfx.FilmGrain; +#endif + return 0; } @@ -443,6 +449,14 @@ static int alloc_frame(AVCodecContext *avctx, QSVContext *q, QSVFrame *frame) frame->dec_info.Header.BufferId = MFX_EXTBUFF_DECODED_FRAME_INFO; frame->dec_info.Header.BufferSz = sizeof(frame->dec_info); ff_qsv_frame_add_ext_param(avctx, frame, (mfxExtBuffer *)&frame->dec_info); +#if QSV_VERSION_ATLEAST(1, 34) + if (QSV_RUNTIME_VERSION_ATLEAST(q->ver, 1, 34) && avctx->codec_id == AV_CODEC_ID_AV1) { + frame->av1_film_grain_param.Header.BufferId = MFX_EXTBUFF_AV1_FILM_GRAIN_PARAM; + frame->av1_film_grain_param.Header.BufferSz = sizeof(frame->av1_film_grain_param); + frame->av1_film_grain_param.FilmGrainFlags = 0; + ff_qsv_frame_add_ext_param(avctx, frame, (mfxExtBuffer *)&frame->av1_film_grain_param); + } +#endif frame->used = 1; @@ -513,6 +527,73 @@ static QSVFrame *find_frame(QSVContext *q, mfxFrameSurface1 *surf) return NULL; } +#if QSV_VERSION_ATLEAST(1, 34) +static int qsv_export_film_grain(AVCodecContext *avctx, mfxExtAV1FilmGrainParam *ext_param, AVFrame *frame) +{ + AVFilmGrainParams *fgp; + AVFilmGrainAOMParams *aom; + int i; + + if (!(ext_param->FilmGrainFlags & MFX_FILM_GRAIN_APPLY)) + return 0; + + fgp = av_film_grain_params_create_side_data(frame); + + if (!fgp) + return AVERROR(ENOMEM); + + fgp->type = AV_FILM_GRAIN_PARAMS_AV1; + fgp->seed = ext_param->GrainSeed; + aom = &fgp->codec.aom; + + aom->chroma_scaling_from_luma = !!(ext_param->FilmGrainFlags & MFX_FILM_GRAIN_CHROMA_SCALING_FROM_LUMA); + aom->scaling_shift = ext_param->GrainScalingMinus8 + 8; + aom->ar_coeff_lag = ext_param->ArCoeffLag; + aom->ar_coeff_shift = ext_param->ArCoeffShiftMinus6 + 6; + aom->grain_scale_shift = ext_param->GrainScaleShift; + aom->overlap_flag = !!(ext_param->FilmGrainFlags & MFX_FILM_GRAIN_OVERLAP); + aom->limit_output_range = !!(ext_param->FilmGrainFlags & MFX_FILM_GRAIN_CLIP_TO_RESTRICTED_RANGE); + + aom->num_y_points = ext_param->NumYPoints; + + for (i = 0; i < aom->num_y_points; i++) { + aom->y_points[i][0] = ext_param->PointY[i].Value; + aom->y_points[i][1] = ext_param->PointY[i].Scaling; + } + + aom->num_uv_points[0] = ext_param->NumCbPoints; + + for (i = 0; i < aom->num_uv_points[0]; i++) { + aom->uv_points[0][i][0] = ext_param->PointCb[i].Value; + aom->uv_points[0][i][1] = ext_param->PointCb[i].Scaling; + } + + aom->num_uv_points[1] = ext_param->NumCrPoints; + + for (i = 0; i < aom->num_uv_points[1]; i++) { + aom->uv_points[1][i][0] = ext_param->PointCr[i].Value; + aom->uv_points[1][i][1] = ext_param->PointCr[i].Scaling; + } + + for (i = 0; i < 24; i++) + aom->ar_coeffs_y[i] = ext_param->ArCoeffsYPlus128[i] - 128; + + for (i = 0; i < 25; i++) { + aom->ar_coeffs_uv[0][i] = ext_param->ArCoeffsCbPlus128[i] - 128; + aom->ar_coeffs_uv[1][i] = ext_param->ArCoeffsCrPlus128[i] - 128; + } + + aom->uv_mult[0] = ext_param->CbMult; + aom->uv_mult[1] = ext_param->CrMult; + aom->uv_mult_luma[0] = ext_param->CbLumaMult; + aom->uv_mult_luma[1] = ext_param->CrLumaMult; + aom->uv_offset[0] = ext_param->CbOffset; + aom->uv_offset[1] = ext_param->CrOffset; + + return 0; +} +#endif + static int qsv_decode(AVCodecContext *avctx, QSVContext *q, AVFrame *frame, int *got_frame, const AVPacket *avpkt) @@ -617,6 +698,16 @@ static int qsv_decode(AVCodecContext *avctx, QSVContext *q, outsurf = &out_frame->surface; frame->pts = MFX_PTS_TO_PTS(outsurf->Data.TimeStamp, avctx->pkt_timebase); +#if QSV_VERSION_ATLEAST(1, 34) + if ((avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN) && + QSV_RUNTIME_VERSION_ATLEAST(q->ver, 1, 34) && + avctx->codec_id == AV_CODEC_ID_AV1) { + ret = qsv_export_film_grain(avctx, &out_frame->av1_film_grain_param, frame); + + if (ret < 0) + return ret; + } +#endif frame->repeat_pict = outsurf->Info.PicStruct & MFX_PICSTRUCT_FRAME_TRIPLING ? 4 :