From patchwork Mon Jan 24 08:24:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 33759 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:e71a:0:0:0:0:0 with SMTP id b26csp130399ioh; Mon, 24 Jan 2022 00:26:20 -0800 (PST) X-Google-Smtp-Source: ABdhPJz9lgKtPTm2QiRLZ1eHrhxM96HTtg5WAW3LAUk11AdHjCuzwNZdbMGbZH6T/+nJ2sGio0hc X-Received: by 2002:a17:906:5048:: with SMTP id e8mr11487487ejk.651.1643012780014; Mon, 24 Jan 2022 00:26:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1643012780; cv=none; d=google.com; s=arc-20160816; b=iVL5kt8FIOkMnoRL/FlLCcWzP/+O2LOauD1c7tf69uhSXPlryGqw2Ef8KOEoHH0qbT w309+YJbMEaEvh33IRWTTNMi4lmY8IbWGtfGDnLI/1curOTsWC/WMyR91oop0WVu71R5 1WcFkwVCJGohWp4hWYB8MN9jkeA3SK0J74OOOJuhjsnFv6MldIz+WOD2nhnCrvT5rWxE Eu8LfpGeaMHqp9od+5+uWLwuxn+e91nkFQ3+fhl/6WuL+Ib5oX3UPCE2xM4Ae2tNCzld zODqBPYLywb69c6JucokZAhAWivJTYDKoKfufI7WzgCKlldJ/eOfeCtT1hppHi1Z4Als RMMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to; bh=0PJq6M9snmXi3rDWTuEcTgh9gG8Lvf4/YuXLSOOob+Y=; b=BebMv+iKdz2KdMFMJeE8gPtl96jXwECVdWd1KyKYMYN4JvwrxM3DDQGRE2337EYNR1 k9JGgdlHEVElKOivZPxlMrgkqqNH3/trm29NR0xcPPSFQJFQkiA4GR53ShEVWEbZdvlq gIBLUBTEz/LPeqZMbFMH2/CRxbLC+PCPBH4DUJ2DQqOmy1Z0d+IUS1TranZTsiHx6K+I W0K2nqJSM5zw09R7QgOw9Yc+yhkGboYSnX/4c1V3bkNChrnSpvd19/0MtpHldihYXj6i EBsWhidSonCPLnv0J4BTnNcYuB4Zq2XB88/0BlzlRbhwy5GCCteT6Q+F1kK1s7a51sQg 6l/Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=gb+0s9E2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id z67si3958271ede.475.2022.01.24.00.26.19; Mon, 24 Jan 2022 00:26:20 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=gb+0s9E2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9B7A168B04D; Mon, 24 Jan 2022 10:26:14 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5E90468AECE for ; Mon, 24 Jan 2022 10:26:07 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643012772; x=1674548772; h=from:to:cc:subject:date:message-id; bh=MWLUrpIIU92u23IDqGU+CkZ48FMWQwzutAJJH0H/vNk=; b=gb+0s9E2y3AnuKDf9FsWRB1ZqLYwPvkrPXj5efOMIkk+G7n1PgDnYGKy Ag2UGnAbMaE8KLGVddf3SM1uHkBcFDs7QkpAWS2l0XT6adlKzT7pJoXJ1 DzR+So/x/fPAoxgkq63OB/25Q+IA8+FacMTcwpMsEZIbUG/br+ctcCmkF 3dtfUFXlQiBGEevf/HmUd5CbrvJMS5vKgkc3uIIVPXUlmTkvSGgzB31nL ZVsQOZiwq+c1uFQH+8VnKDFUkhj5CXpFtAoSFbCgG194HtLFeKJ0uToXM otGgi0Nr+P2QVYJTt1k2nYMhVsX3F9BDvNNYqWK40N38lMkeTud7lYitf Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10236"; a="245940002" X-IronPort-AV: E=Sophos;i="5.88,311,1635231600"; d="scan'208";a="245940002" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Jan 2022 00:26:04 -0800 X-IronPort-AV: E=Sophos;i="5.88,311,1635231600"; d="scan'208";a="534128360" Received: from xhh-dg164.sh.intel.com ([10.239.159.167]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Jan 2022 00:26:03 -0800 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Jan 2022 16:24:53 +0800 Message-Id: <20220124082455.30931-1-haihao.xiang@intel.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH v2 1/3] lavc/qsv: allow to add more parameter buffers to QSV frame X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ZFM0xxoarsYk From: Haihao Xiang Signed-off-by: Haihao Xiang --- v2: rebased the patchset against the latest FFmpeg and added code to make sure the corresponding extra parameter buffer is added for AV1 only. libavcodec/qsv.c | 27 +++++++++++++++++++++++++++ libavcodec/qsv_internal.h | 8 +++++++- libavcodec/qsvdec.c | 8 +++++--- 3 files changed, 39 insertions(+), 4 deletions(-) diff --git a/libavcodec/qsv.c b/libavcodec/qsv.c index 9d08485c92..1a432dbd82 100644 --- a/libavcodec/qsv.c +++ b/libavcodec/qsv.c @@ -828,3 +828,30 @@ int ff_qsv_close_internal_session(QSVSession *qs) #endif return 0; } + +void ff_qsv_frame_add_ext_param (AVCodecContext *avctx, QSVFrame *frame, + mfxExtBuffer * param) +{ + int i; + + for (i = 0; i < frame->num_ext_params; i++) { + mfxExtBuffer *ext_buffer = frame->ext_param[i]; + + if (ext_buffer->BufferId == param->BufferId) { + av_log(avctx, AV_LOG_WARNING, "A buffer with the same type has been " + "added\n"); + return; + } + } + + if (frame->num_ext_params < QSV_MAX_FRAME_EXT_PARAMS) { + frame->ext_param[frame->num_ext_params] = param; + frame->num_ext_params++; + frame->surface.Data.NumExtParam = frame->num_ext_params; + } else { + av_log(avctx, AV_LOG_WARNING, "Ignore this extra buffer because do not " + "have enough space\n"); + } + + +} diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h index fe9d5319c4..6a38e87d23 100644 --- a/libavcodec/qsv_internal.h +++ b/libavcodec/qsv_internal.h @@ -52,6 +52,8 @@ #define QSV_MAX_ENC_PAYLOAD 2 // # of mfxEncodeCtrl payloads supported +#define QSV_MAX_FRAME_EXT_PARAMS 4 + #define QSV_VERSION_ATLEAST(MAJOR, MINOR) \ (MFX_VERSION_MAJOR > (MAJOR) || \ MFX_VERSION_MAJOR == (MAJOR) && MFX_VERSION_MINOR >= (MINOR)) @@ -74,7 +76,8 @@ typedef struct QSVFrame { mfxFrameSurface1 surface; mfxEncodeCtrl enc_ctrl; mfxExtDecodedFrameInfo dec_info; - mfxExtBuffer *ext_param; + mfxExtBuffer *ext_param[QSV_MAX_FRAME_EXT_PARAMS]; + int num_ext_params; mfxPayload *payloads[QSV_MAX_ENC_PAYLOAD]; ///< used for enc_ctrl.Payload @@ -138,4 +141,7 @@ int ff_qsv_init_session_frames(AVCodecContext *avctx, mfxSession *session, int ff_qsv_find_surface_idx(QSVFramesContext *ctx, QSVFrame *frame); +void ff_qsv_frame_add_ext_param(AVCodecContext *avctx, QSVFrame *frame, + mfxExtBuffer *param); + #endif /* AVCODEC_QSV_INTERNAL_H */ diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index d9e0fef1f1..783d252002 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -423,11 +423,13 @@ static int alloc_frame(AVCodecContext *avctx, QSVContext *q, QSVFrame *frame) frame->surface.Data.MemId = &q->frames_ctx.mids[ret]; } - frame->surface.Data.ExtParam = &frame->ext_param; - frame->surface.Data.NumExtParam = 1; - frame->ext_param = (mfxExtBuffer*)&frame->dec_info; + + frame->surface.Data.ExtParam = frame->ext_param; + frame->surface.Data.NumExtParam = 0; + frame->num_ext_params = 0; frame->dec_info.Header.BufferId = MFX_EXTBUFF_DECODED_FRAME_INFO; frame->dec_info.Header.BufferSz = sizeof(frame->dec_info); + ff_qsv_frame_add_ext_param(avctx, frame, (mfxExtBuffer *)&frame->dec_info); frame->used = 1; From patchwork Mon Jan 24 08:24:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 33760 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:e71a:0:0:0:0:0 with SMTP id b26csp130447ioh; Mon, 24 Jan 2022 00:26:30 -0800 (PST) X-Google-Smtp-Source: ABdhPJwKPtypiEdsX/OMp6YcrKeiXT4G5DXVm7y/0L4esykVA1S7h+e/UBFiNOhNuwJ3XDEEZDr1 X-Received: by 2002:a05:6402:1003:: with SMTP id c3mr15068996edu.405.1643012790721; Mon, 24 Jan 2022 00:26:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1643012790; cv=none; d=google.com; s=arc-20160816; b=dXDwZ2q/EJcVFJIaeAsnnL8CKNAq1Hwr+t5GMU+5fjwvuJTXGcH1JQqro8jwf+ZcXX wafHaumFQS5TAaP6wooIZv4jY4Xz4GJKoCHTTwoCIgbFoL22NtaPRT60UPHrpJPy17q1 ozmNbKQf4NYS3x6RW21eUzApHLR4h0XmJhq0OTK7sPLeArOvAPAkaxhSVp2HlxhiW2Iu uW+382pXCP3Vf7CAZB80zBwxYxl6rMQeNAEUPEMeutW3vDsePYxTEQHlW9lbYpVIsNnY OZCY7/NiA4RUIQ+EVj2kmGfPUrxltAjL+8/3NFcwf6D6X/8iRZyRoPCwguOzDY+pyUyD pkgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=FRZ9KoPnn6R06aEeCEJ9dTBOjgXufdFdrn13t98bHmk=; b=Q76HaBEd3iskaugjR3LzCsDKruPcvZFSneO3vQ8TypHTQkRjRFSqlg7dWxacQICL4g 1gxu065Isof7OMELwxgo4kGSjjF5OOIrzQggvRPklzamAdORwWQsTQMw8lg4l5kMTy0A VGPxQSE8Neguuadta2UojS1YnGdDKWEhH58wqiQm/4WBz1JoOBB2N0wDClAythApKbZb IaPx3tvfxQn+cmSo8D7xj/HKBDYo89T4FR9xwQbTRBr/4M2t4Hg9PVXrVOWJs6bitvMp CrjzGStcXoMwcnZCfvpz225vcE/G5wkRCroxx5tYQw98C/0NcRquxWerbILpc8xTVkpS Qi+w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=N3E6K57b; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id q4si7959949edb.472.2022.01.24.00.26.30; Mon, 24 Jan 2022 00:26:30 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=N3E6K57b; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B6F6368B058; Mon, 24 Jan 2022 10:26:17 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A296E68AECE for ; Mon, 24 Jan 2022 10:26:08 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643012773; x=1674548773; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=QfZBsoglk7P5zATZlQxTzazBpJYpgUPYYqFcpqhJakM=; b=N3E6K57bFUIGp5oUPM5Og3/4TGPv1rH4+Ht9HWlsjvmm9x1Bb1psL4gZ 3V0PdVt1tXLaYzrQZodijbp8RIN+UMdgz78AcLCaG5v62noNn0QfZPXQ4 VrLrg4EqKOP2jpou/lXvWTHmfMqyGKJhFoAyxnL/UTLWtfkysSz/EL8Pn gasszFk/Sb/GvS68lggiEyN5ZWvO4swUdhhtXphhqA/WiyRgnWOmZ3K9i rtdky/AX6NFOJjLY+jkAyHXq6zzfqknhU7/kx2Ynf0fCQ5G3U9zVD0is5 8p/P+XU1VMcPSQRtm9XXWr7c3BfGUiB8VKRhmPxBoZmUVbz/eH7pbJ8/o A==; X-IronPort-AV: E=McAfee;i="6200,9189,10236"; a="245940003" X-IronPort-AV: E=Sophos;i="5.88,311,1635231600"; d="scan'208";a="245940003" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Jan 2022 00:26:05 -0800 X-IronPort-AV: E=Sophos;i="5.88,311,1635231600"; d="scan'208";a="534128371" Received: from xhh-dg164.sh.intel.com ([10.239.159.167]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Jan 2022 00:26:04 -0800 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Jan 2022 16:24:54 +0800 Message-Id: <20220124082455.30931-2-haihao.xiang@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220124082455.30931-1-haihao.xiang@intel.com> References: <20220124082455.30931-1-haihao.xiang@intel.com> Subject: [FFmpeg-devel] [PATCH v2 2/3] lavc/qsvdec: track the runtime session version X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: FVp7ksdVFVyf From: Haihao Xiang We may check the runtime version for the given features Signed-off-by: Haihao Xiang --- libavcodec/qsvdec.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index 783d252002..8b83d5695f 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -59,6 +59,7 @@ static const AVRational mfx_tb = { 1, 90000 }; typedef struct QSVContext { // the session used for decoding mfxSession session; + mfxVersion ver; // the session we allocated internally, in case the caller did not provide // one @@ -202,6 +203,18 @@ static int qsv_init_session(AVCodecContext *avctx, QSVContext *q, mfxSession ses q->session = q->internal_qs.session; } + if (MFXQueryVersion(q->session, &q->ver) != MFX_ERR_NONE) { + av_log(avctx, AV_LOG_ERROR, "Error querying the session version. \n"); + q->session = NULL; + + if (q->internal_qs.session) { + MFXClose(q->internal_qs.session); + q->internal_qs.session = NULL; + } + + return AVERROR_EXTERNAL; + } + /* make sure the decoder is uninitialized */ MFXVideoDECODE_Close(q->session); From patchwork Mon Jan 24 08:24:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 33761 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:e71a:0:0:0:0:0 with SMTP id b26csp130532ioh; Mon, 24 Jan 2022 00:26:40 -0800 (PST) X-Google-Smtp-Source: ABdhPJw+C1cDF/frepGhhkAlT82H4C08cRwkCQ7CbdLZH8w8mqtRwM8DnVCljbLNj4tt0ZbVgCdu X-Received: by 2002:a17:906:29cd:: with SMTP id y13mr12330529eje.755.1643012800067; Mon, 24 Jan 2022 00:26:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1643012800; cv=none; d=google.com; s=arc-20160816; b=txSfw2rO94bTzBW7dTN2COTsc+BJCiK5sB6oabpfFYzk7T+Q4sc70JDybPHRlqa/xN 4MpYa6DzgTsoECtf9eVHzWCrIQPjIzF+6epeoK/wRe8fJGJhZuCnkcxYlOqL1VlEctqQ kU/VwJY0qFtGZEQmEG6tOX6CKnhmA0g8oDt6OEpcly2CBYBia88TWlM7WVCRFj4VQLoJ Ed9Akh71Pke4FslZTSzLhmTQhkhWJxQYcNJ5JwgykUJYp26rVKeB8kYOSveCwq12DtWq /yu4KSwb9e6CITPqpd5eJJw+3OAZg0830HGy/ULyjfiWMlGEujYDoTUtFCMAW6jGpiTO kUjg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=xe2v9lzPLrGDBLIS1CBoFe/5qf3WEYFWbqLsX4/Rqs8=; b=eKPa8AKP/GYT81UQbHW1U1Mz89OTn57PnL/fitvgagGAQ6KA6/qZUXH+fPu4E9x83x kaw2Az5KzqPimQgAn/Ioo56NmDDgvyZ8GFG5Icf8szdJsVX0M4E+EA7J50/0xzP/tEXs nJDeBQvAhEppYTNuof4N5I10UnsdtnydiXR8H7MyOqxkEJ5QnRrCXm5jAr4dBE+wTK0R JtI8Aua0dRIj6iaA/4Y1+Fr47/HUZ7LyU5vbcF3HTgjnTg93kV1yqS7ROJGGgKYFhB9m 9ay9fHGsCMhjscKWbbv64lTDTylUH7INcNlVjyj0gumWBbqstP/PDkOayq3eFAw3AGmq q5Iw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=UepUUMDH; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id sh31si1004840ejc.857.2022.01.24.00.26.39; Mon, 24 Jan 2022 00:26:40 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=UepUUMDH; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BE4D268B06E; Mon, 24 Jan 2022 10:26:19 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 55A3868B05B for ; Mon, 24 Jan 2022 10:26:13 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643012778; x=1674548778; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=ySND3NSYh/Ydn9AclsN74THOnZp/DnG4t8ecgOIf6FI=; b=UepUUMDHh8sPxg7yL1mrTDn3mTnuoogh54Q9Xg5FNaUmvXdfLVy8/rCH cxX1Aev/9N3GHyeakiMFqI1dT3EAl8Xs0ZgrrxpKDDN4cOAHxxnPc8J6/ Q0ONvln0G/4yxzkI2ct3QvYokF8jpJNAerPKDq7fvia2gEEJnFaXznd0N cMrBadl4tLsorkccw6BIJaxP1onBNVBzYCupuExP5Rmrku6E0oP2/JqiJ o/Ms33GiKMedncdQzJw1Leu4eC6AXAGy6b/1S1IoHt0tQQv1M6CQlovt3 /NYgvwY2BzAWOj15SD5SHzloUtIjd4oKCEfuo0wFhJgNXdCcb/CBaOT2A A==; X-IronPort-AV: E=McAfee;i="6200,9189,10236"; a="245940008" X-IronPort-AV: E=Sophos;i="5.88,311,1635231600"; d="scan'208";a="245940008" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Jan 2022 00:26:06 -0800 X-IronPort-AV: E=Sophos;i="5.88,311,1635231600"; d="scan'208";a="534128386" Received: from xhh-dg164.sh.intel.com ([10.239.159.167]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Jan 2022 00:26:05 -0800 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Jan 2022 16:24:55 +0800 Message-Id: <20220124082455.30931-3-haihao.xiang@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220124082455.30931-1-haihao.xiang@intel.com> References: <20220124082455.30931-1-haihao.xiang@intel.com> Subject: [FFmpeg-devel] [PATCH v2 3/3] lavc/qsvdec: export AVFilmGrainParams side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: gEm/t4NY7Xrf From: Haihao Xiang When AV_CODEC_EXPORT_DATA_FILM_GRAIN is present, AV1 decoder should disable film grain application and export the corresponding side data Signed-off-by: Haihao Xiang --- libavcodec/qsv_internal.h | 3 ++ libavcodec/qsvdec.c | 91 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 94 insertions(+) diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h index 6a38e87d23..58186ea7ca 100644 --- a/libavcodec/qsv_internal.h +++ b/libavcodec/qsv_internal.h @@ -76,6 +76,9 @@ typedef struct QSVFrame { mfxFrameSurface1 surface; mfxEncodeCtrl enc_ctrl; mfxExtDecodedFrameInfo dec_info; +#if QSV_VERSION_ATLEAST(1, 34) + mfxExtAV1FilmGrainParam av1_film_grain_param; +#endif mfxExtBuffer *ext_param[QSV_MAX_FRAME_EXT_PARAMS]; int num_ext_params; diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index 8b83d5695f..32077ab31a 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -38,6 +38,7 @@ #include "libavutil/pixfmt.h" #include "libavutil/time.h" #include "libavutil/imgutils.h" +#include "libavutil/film_grain_params.h" #include "avcodec.h" #include "internal.h" @@ -404,6 +405,11 @@ static int qsv_decode_header(AVCodecContext *avctx, QSVContext *q, param->ExtParam = q->ext_buffers; param->NumExtParam = q->nb_ext_buffers; +#if QSV_VERSION_ATLEAST(1, 34) + if (QSV_RUNTIME_VERSION_ATLEAST(q->ver, 1, 34) && avctx->codec_id == AV_CODEC_ID_AV1) + param->mfx.FilmGrain = (avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN) ? 0 : param->mfx.FilmGrain; +#endif + return 0; } @@ -443,6 +449,14 @@ static int alloc_frame(AVCodecContext *avctx, QSVContext *q, QSVFrame *frame) frame->dec_info.Header.BufferId = MFX_EXTBUFF_DECODED_FRAME_INFO; frame->dec_info.Header.BufferSz = sizeof(frame->dec_info); ff_qsv_frame_add_ext_param(avctx, frame, (mfxExtBuffer *)&frame->dec_info); +#if QSV_VERSION_ATLEAST(1, 34) + if (QSV_RUNTIME_VERSION_ATLEAST(q->ver, 1, 34) && avctx->codec_id == AV_CODEC_ID_AV1) { + frame->av1_film_grain_param.Header.BufferId = MFX_EXTBUFF_AV1_FILM_GRAIN_PARAM; + frame->av1_film_grain_param.Header.BufferSz = sizeof(frame->av1_film_grain_param); + frame->av1_film_grain_param.FilmGrainFlags = 0; + ff_qsv_frame_add_ext_param(avctx, frame, (mfxExtBuffer *)&frame->av1_film_grain_param); + } +#endif frame->used = 1; @@ -513,6 +527,73 @@ static QSVFrame *find_frame(QSVContext *q, mfxFrameSurface1 *surf) return NULL; } +#if QSV_VERSION_ATLEAST(1, 34) +static int qsv_export_film_grain(AVCodecContext *avctx, mfxExtAV1FilmGrainParam *ext_param, AVFrame *frame) +{ + AVFilmGrainParams *fgp; + AVFilmGrainAOMParams *aom; + int i; + + if (!(ext_param->FilmGrainFlags & MFX_FILM_GRAIN_APPLY)) + return 0; + + fgp = av_film_grain_params_create_side_data(frame); + + if (!fgp) + return AVERROR(ENOMEM); + + fgp->type = AV_FILM_GRAIN_PARAMS_AV1; + fgp->seed = ext_param->GrainSeed; + aom = &fgp->codec.aom; + + aom->chroma_scaling_from_luma = !!(ext_param->FilmGrainFlags & MFX_FILM_GRAIN_CHROMA_SCALING_FROM_LUMA); + aom->scaling_shift = ext_param->GrainScalingMinus8 + 8; + aom->ar_coeff_lag = ext_param->ArCoeffLag; + aom->ar_coeff_shift = ext_param->ArCoeffShiftMinus6 + 6; + aom->grain_scale_shift = ext_param->GrainScaleShift; + aom->overlap_flag = !!(ext_param->FilmGrainFlags & MFX_FILM_GRAIN_OVERLAP); + aom->limit_output_range = !!(ext_param->FilmGrainFlags & MFX_FILM_GRAIN_CLIP_TO_RESTRICTED_RANGE); + + aom->num_y_points = ext_param->NumYPoints; + + for (i = 0; i < aom->num_y_points; i++) { + aom->y_points[i][0] = ext_param->PointY[i].Value; + aom->y_points[i][1] = ext_param->PointY[i].Scaling; + } + + aom->num_uv_points[0] = ext_param->NumCbPoints; + + for (i = 0; i < aom->num_uv_points[0]; i++) { + aom->uv_points[0][i][0] = ext_param->PointCb[i].Value; + aom->uv_points[0][i][1] = ext_param->PointCb[i].Scaling; + } + + aom->num_uv_points[1] = ext_param->NumCrPoints; + + for (i = 0; i < aom->num_uv_points[1]; i++) { + aom->uv_points[1][i][0] = ext_param->PointCr[i].Value; + aom->uv_points[1][i][1] = ext_param->PointCr[i].Scaling; + } + + for (i = 0; i < 24; i++) + aom->ar_coeffs_y[i] = ext_param->ArCoeffsYPlus128[i] - 128; + + for (i = 0; i < 25; i++) { + aom->ar_coeffs_uv[0][i] = ext_param->ArCoeffsCbPlus128[i] - 128; + aom->ar_coeffs_uv[1][i] = ext_param->ArCoeffsCrPlus128[i] - 128; + } + + aom->uv_mult[0] = ext_param->CbMult; + aom->uv_mult[1] = ext_param->CrMult; + aom->uv_mult_luma[0] = ext_param->CbLumaMult; + aom->uv_mult_luma[1] = ext_param->CrLumaMult; + aom->uv_offset[0] = ext_param->CbOffset; + aom->uv_offset[1] = ext_param->CrOffset; + + return 0; +} +#endif + static int qsv_decode(AVCodecContext *avctx, QSVContext *q, AVFrame *frame, int *got_frame, const AVPacket *avpkt) @@ -617,6 +698,16 @@ static int qsv_decode(AVCodecContext *avctx, QSVContext *q, outsurf = &out_frame->surface; frame->pts = MFX_PTS_TO_PTS(outsurf->Data.TimeStamp, avctx->pkt_timebase); +#if QSV_VERSION_ATLEAST(1, 34) + if ((avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN) && + QSV_RUNTIME_VERSION_ATLEAST(q->ver, 1, 34) && + avctx->codec_id == AV_CODEC_ID_AV1) { + ret = qsv_export_film_grain(avctx, &out_frame->av1_film_grain_param, frame); + + if (ret < 0) + return ret; + } +#endif frame->repeat_pict = outsurf->Info.PicStruct & MFX_PICSTRUCT_FRAME_TRIPLING ? 4 :