From patchwork Tue Mar 23 03:00:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 26547 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id C68F144963D for ; Tue, 23 Mar 2021 05:02:36 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9879168AA62; Tue, 23 Mar 2021 05:02:36 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B021468A94D for ; Tue, 23 Mar 2021 05:02:29 +0200 (EET) IronPort-SDR: cIiy6qL8YJ2rEpgYtvqLz+wL1Q92yTEW6WnW9PCQqcif+C9YJtMVCNfKHv3/5+bVHCgrJreifB 9HXlAcgMewEw== X-IronPort-AV: E=McAfee;i="6000,8403,9931"; a="189792564" X-IronPort-AV: E=Sophos;i="5.81,270,1610438400"; d="scan'208";a="189792564" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Mar 2021 20:02:26 -0700 IronPort-SDR: n867T9wNAoxPBKbQGhJ2N7WzBIQaHhvkds/SPfAC3XklnIbmW2HcJ3VhAzazGr75HTfVlUHpjB F6dH4VJq9PAw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,270,1610438400"; d="scan'208";a="414760612" Received: from xhh-tgl64.sh.intel.com ([10.239.159.152]) by orsmga008.jf.intel.com with ESMTP; 22 Mar 2021 20:02:25 -0700 From: Haihao Xiang To: ffmpeg-devel@ffmpeg.org Date: Tue, 23 Mar 2021 11:00:06 +0800 Message-Id: <20210323030007.590540-1-haihao.xiang@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] lavc/qsv: allow to add more parameter buffers to QSV frame X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/qsv.c | 27 +++++++++++++++++++++++++++ libavcodec/qsv_internal.h | 8 +++++++- libavcodec/qsvdec.c | 8 +++++--- 3 files changed, 39 insertions(+), 4 deletions(-) diff --git a/libavcodec/qsv.c b/libavcodec/qsv.c index 6e3154e1a3..879e109092 100644 --- a/libavcodec/qsv.c +++ b/libavcodec/qsv.c @@ -821,3 +821,30 @@ int ff_qsv_close_internal_session(QSVSession *qs) #endif return 0; } + +void ff_qsv_frame_add_ext_param (AVCodecContext *avctx, QSVFrame *frame, + mfxExtBuffer * param) +{ + int i; + + for (i = 0; i < frame->num_ext_params; i++) { + mfxExtBuffer *ext_buffer = frame->ext_param[i]; + + if (ext_buffer->BufferId == param->BufferId) { + av_log(avctx, AV_LOG_WARNING, "A buffer with the same type has been " + "added\n"); + return; + } + } + + if (frame->num_ext_params < QSV_MAX_FRAME_EXT_PARAMS) { + frame->ext_param[frame->num_ext_params] = param; + frame->num_ext_params++; + frame->surface.Data.NumExtParam = frame->num_ext_params; + } else { + av_log(avctx, AV_LOG_WARNING, "Ignore this extra buffer because do not " + "have enough space\n"); + } + + +} diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h index 6b2fbbe252..1d94d429e8 100644 --- a/libavcodec/qsv_internal.h +++ b/libavcodec/qsv_internal.h @@ -52,6 +52,8 @@ #define QSV_MAX_ENC_PAYLOAD 2 // # of mfxEncodeCtrl payloads supported +#define QSV_MAX_FRAME_EXT_PARAMS 4 + #define QSV_VERSION_ATLEAST(MAJOR, MINOR) \ (MFX_VERSION_MAJOR > (MAJOR) || \ MFX_VERSION_MAJOR == (MAJOR) && MFX_VERSION_MINOR >= (MINOR)) @@ -74,7 +76,8 @@ typedef struct QSVFrame { mfxFrameSurface1 surface; mfxEncodeCtrl enc_ctrl; mfxExtDecodedFrameInfo dec_info; - mfxExtBuffer *ext_param; + mfxExtBuffer *ext_param[QSV_MAX_FRAME_EXT_PARAMS]; + int num_ext_params; int queued; int used; @@ -142,4 +145,7 @@ int ff_qsv_init_session_frames(AVCodecContext *avctx, mfxSession *session, int ff_qsv_find_surface_idx(QSVFramesContext *ctx, QSVFrame *frame); +void ff_qsv_frame_add_ext_param(AVCodecContext *avctx, QSVFrame *frame, + mfxExtBuffer *param); + #endif /* AVCODEC_QSV_INTERNAL_H */ diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index 5f2e641373..55cf9f35c5 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -366,11 +366,13 @@ static int alloc_frame(AVCodecContext *avctx, QSVContext *q, QSVFrame *frame) frame->surface.Data.MemId = &q->frames_ctx.mids[ret]; } - frame->surface.Data.ExtParam = &frame->ext_param; - frame->surface.Data.NumExtParam = 1; - frame->ext_param = (mfxExtBuffer*)&frame->dec_info; + + frame->surface.Data.ExtParam = frame->ext_param; + frame->surface.Data.NumExtParam = 0; + frame->num_ext_params = 0; frame->dec_info.Header.BufferId = MFX_EXTBUFF_DECODED_FRAME_INFO; frame->dec_info.Header.BufferSz = sizeof(frame->dec_info); + ff_qsv_frame_add_ext_param(avctx, frame, (mfxExtBuffer *)&frame->dec_info); frame->used = 1; From patchwork Tue Mar 23 03:00:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 26548 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id D583544963D for ; Tue, 23 Mar 2021 05:02:46 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C05BC68AAD5; Tue, 23 Mar 2021 05:02:46 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A96C268A94D for ; Tue, 23 Mar 2021 05:02:39 +0200 (EET) IronPort-SDR: hWiUCo7LdpRnminuNzgmRWLU/28e5mnEOap1g3am1ThQR4JniK+KbKTASv4EQ7YnzPPOQZS6NG quM3E9SxN63g== X-IronPort-AV: E=McAfee;i="6000,8403,9931"; a="170355511" X-IronPort-AV: E=Sophos;i="5.81,270,1610438400"; d="scan'208";a="170355511" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Mar 2021 20:02:37 -0700 IronPort-SDR: Wyp4g+g9fCwQs8l/rongWS1OWANMEdTNffj+P+YDuumfgXTqC9RJvRNxd7FobDap9Fe7GF1Uo0 umClCopbO9DA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,270,1610438400"; d="scan'208";a="414760673" Received: from xhh-tgl64.sh.intel.com ([10.239.159.152]) by orsmga008.jf.intel.com with ESMTP; 22 Mar 2021 20:02:36 -0700 From: Haihao Xiang To: ffmpeg-devel@ffmpeg.org Date: Tue, 23 Mar 2021 11:00:07 +0800 Message-Id: <20210323030007.590540-2-haihao.xiang@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210323030007.590540-1-haihao.xiang@intel.com> References: <20210323030007.590540-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] lavc/qsvdec: export AVFilmGrainParams side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" When AV_CODEC_EXPORT_DATA_FILM_GRAIN is present, AV1 decoder should disable film grain application and export the corresponding side data --- libavcodec/qsv_internal.h | 3 ++ libavcodec/qsvdec.c | 88 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 91 insertions(+) diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h index 1d94d429e8..754581087d 100644 --- a/libavcodec/qsv_internal.h +++ b/libavcodec/qsv_internal.h @@ -76,6 +76,9 @@ typedef struct QSVFrame { mfxFrameSurface1 surface; mfxEncodeCtrl enc_ctrl; mfxExtDecodedFrameInfo dec_info; +#if QSV_VERSION_ATLEAST(1, 34) + mfxExtAV1FilmGrainParam av1_film_grain_param; +#endif mfxExtBuffer *ext_param[QSV_MAX_FRAME_EXT_PARAMS]; int num_ext_params; diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index 55cf9f35c5..e34441fc0b 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -38,6 +38,7 @@ #include "libavutil/pixfmt.h" #include "libavutil/time.h" #include "libavutil/imgutils.h" +#include "libavutil/film_grain_params.h" #include "avcodec.h" #include "internal.h" @@ -334,6 +335,11 @@ static int qsv_decode_header(AVCodecContext *avctx, QSVContext *q, return ff_qsv_print_error(avctx, ret, "Error decoding stream header"); +#if QSV_VERSION_ATLEAST(1, 34) + if (avctx->codec_id == AV_CODEC_ID_AV1) + param->mfx.FilmGrain = (avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN) ? 0 : param->mfx.FilmGrain; +#endif + return 0; } @@ -373,6 +379,12 @@ static int alloc_frame(AVCodecContext *avctx, QSVContext *q, QSVFrame *frame) frame->dec_info.Header.BufferId = MFX_EXTBUFF_DECODED_FRAME_INFO; frame->dec_info.Header.BufferSz = sizeof(frame->dec_info); ff_qsv_frame_add_ext_param(avctx, frame, (mfxExtBuffer *)&frame->dec_info); +#if QSV_VERSION_ATLEAST(1, 34) + frame->av1_film_grain_param.Header.BufferId = MFX_EXTBUFF_AV1_FILM_GRAIN_PARAM; + frame->av1_film_grain_param.Header.BufferSz = sizeof(frame->av1_film_grain_param); + frame->av1_film_grain_param.FilmGrainFlags = 0; + ff_qsv_frame_add_ext_param(avctx, frame, (mfxExtBuffer *)&frame->av1_film_grain_param); +#endif frame->used = 1; @@ -443,6 +455,73 @@ static QSVFrame *find_frame(QSVContext *q, mfxFrameSurface1 *surf) return NULL; } +#if QSV_VERSION_ATLEAST(1, 34) +static int qsv_export_film_grain(AVCodecContext *avctx, mfxExtAV1FilmGrainParam *ext_param, AVFrame *frame) +{ + AVFilmGrainParams *fgp; + AVFilmGrainAOMParams *aom; + int i; + + if (!(ext_param->FilmGrainFlags & MFX_FILM_GRAIN_APPLY)) + return 0; + + fgp = av_film_grain_params_create_side_data(frame); + + if (!fgp) + return AVERROR(ENOMEM); + + fgp->type = AV_FILM_GRAIN_PARAMS_AV1; + fgp->seed = ext_param->GrainSeed; + aom = &fgp->codec.aom; + + aom->chroma_scaling_from_luma = !!(ext_param->FilmGrainFlags & MFX_FILM_GRAIN_CHROMA_SCALING_FROM_LUMA); + aom->scaling_shift = ext_param->GrainScalingMinus8 + 8; + aom->ar_coeff_lag = ext_param->ArCoeffLag; + aom->ar_coeff_shift = ext_param->ArCoeffShiftMinus6 + 6; + aom->grain_scale_shift = ext_param->GrainScaleShift; + aom->overlap_flag = !!(ext_param->FilmGrainFlags & MFX_FILM_GRAIN_OVERLAP); + aom->limit_output_range = !!(ext_param->FilmGrainFlags & MFX_FILM_GRAIN_CLIP_TO_RESTRICTED_RANGE); + + aom->num_y_points = ext_param->NumYPoints; + + for (i = 0; i < aom->num_y_points; i++) { + aom->y_points[i][0] = ext_param->PointY[i].Value; + aom->y_points[i][1] = ext_param->PointY[i].Scaling; + } + + aom->num_uv_points[0] = ext_param->NumCbPoints; + + for (i = 0; i < aom->num_uv_points[0]; i++) { + aom->uv_points[0][i][0] = ext_param->PointCb[i].Value; + aom->uv_points[0][i][1] = ext_param->PointCb[i].Scaling; + } + + aom->num_uv_points[1] = ext_param->NumCrPoints; + + for (i = 0; i < aom->num_uv_points[1]; i++) { + aom->uv_points[1][i][0] = ext_param->PointCr[i].Value; + aom->uv_points[1][i][1] = ext_param->PointCr[i].Scaling; + } + + for (i = 0; i < 24; i++) + aom->ar_coeffs_y[i] = ext_param->ArCoeffsYPlus128[i] - 128; + + for (i = 0; i < 25; i++) { + aom->ar_coeffs_uv[0][i] = ext_param->ArCoeffsCbPlus128[i] - 128; + aom->ar_coeffs_uv[1][i] = ext_param->ArCoeffsCrPlus128[i] - 128; + } + + aom->uv_mult[0] = ext_param->CbMult; + aom->uv_mult[1] = ext_param->CrMult; + aom->uv_mult_luma[0] = ext_param->CbLumaMult; + aom->uv_mult_luma[1] = ext_param->CrLumaMult; + aom->uv_offset[0] = ext_param->CbOffset; + aom->uv_offset[1] = ext_param->CrOffset; + + return 0; +} +#endif + static int qsv_decode(AVCodecContext *avctx, QSVContext *q, AVFrame *frame, int *got_frame, const AVPacket *avpkt) @@ -546,6 +625,15 @@ static int qsv_decode(AVCodecContext *avctx, QSVContext *q, outsurf = &out_frame->surface; +#if QSV_VERSION_ATLEAST(1, 34) + if (avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN) { + ret = qsv_export_film_grain(avctx, &out_frame->av1_film_grain_param, frame); + + if (ret < 0) + return ret; + } +#endif + #if FF_API_PKT_PTS FF_DISABLE_DEPRECATION_WARNINGS frame->pkt_pts = outsurf->Data.TimeStamp;