From patchwork Wed Jan 12 05:28:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Wenbin" X-Patchwork-Id: 33273 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp4338662iog; Tue, 11 Jan 2022 21:30:28 -0800 (PST) X-Google-Smtp-Source: ABdhPJxIxI54FRhU7xgZo//JIbcMyro5qRjF+QewxuIRGXC4G7K9EONFO+mTq3FRhV0UzgzWy/2u X-Received: by 2002:a17:906:4782:: with SMTP id cw2mr2780867ejc.337.1641965428654; Tue, 11 Jan 2022 21:30:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1641965428; cv=none; d=google.com; s=arc-20160816; b=lSyGmr6R+zEhLEr1++7tVJ3SbeddZB8A9bfDtT1Jlfc6xZTqHYFQdE9hSazLVXjsXz 3lWXljvPaSHzxUc8/aV/JIANtKgXHhSc0xVrY3alv9y2sI++RJsGYyCAs3bl6f5NZzTF O4JZnrnd+AYutKJpdtZuwAl3MmPVoP0Dph278mLX5JIx/8jcMkVrj6KTWA11yO0tVWQ2 1RpPFPeVXZcnBPmzRiq7AqEX7lE8AbDl9HWGzqPrzecLzP921vMHUJH8moyM5/2kbNnN rw8/Sa9h6u37k/55N/CPnBWrv4NJahYfDxqM9Wy7MUUFPOnxUP8ILj4tX88BA+p8zPmV UZxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=Gdi0jb4BKR8Labn2ktc0upw3GGcHYhd1PH98C+KGQrk=; b=nhiMnYmoUDQ4u2jU6COiSelpfRjTdjN4VAQpFS8Hj2ElH0+4hYgcduituz5B6+VaNj 9wITnYZFknogPWLSnKV8RuwSfROe52m4mZlwY46tY39gdJycuW7vObUzziUAx9h0v38V hyJkvKANSpY6ZZAJAClFZrAgDL1q15EgyzyoqRGkcxs2cgN0Sge4cQP7enZq/NmEhII7 1/GGv79WxXOg7TroGSVI9Tgwan2yUekJk70Z/r/5dfzrtuuhYadwhdRjFtfZ3hMOlyYy ZV4gI5FZvvRWMk11G6A3Q1JJdxQAn31zKOSPauZh5EK9/7QjIp35Rc/9JJiwp9nm9CrZ ofgg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=XfmOZbQx; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g25si6373639eds.562.2022.01.11.21.30.28; Tue, 11 Jan 2022 21:30:28 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=XfmOZbQx; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8377368AE34; Wed, 12 Jan 2022 07:30:17 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E9BF9680BA3 for ; Wed, 12 Jan 2022 07:30:07 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1641965413; x=1673501413; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=G2GlGyDMynX3sjSVpOLw9ORWAZ1wI45J4QWjx5nXPNA=; b=XfmOZbQxFjxgryRSVe0xhYm6Y6xMsfEkzbpEV1cMtoM+exHWOyQoJGO5 /lN47OD7lwIzMiNt9mxqgf6/B03j8/5gvVbMUkl++ZVB15Pcc2OquSfWB RcunWf9uBX0CL2mD+dVGdqheu64eIdFD3oTFnA6xhKX1qU7c238Hr2iHE UuHtmPJPQyy36ri+Jc87uIW4Phc6N6EoqhA+hyDvq4PIMIwyGR3owQYd6 gllBtnvLxFuXz+LKw+jg12Fif5MpXnP6JXLvtmDEl9pSoy/qSOiuj7blh JJAt3OsqMeOyQ58az5JgcWadDKx3K59cRZi24XlvF1dkTHgGaZFbel+3H g==; X-IronPort-AV: E=McAfee;i="6200,9189,10224"; a="241215384" X-IronPort-AV: E=Sophos;i="5.88,282,1635231600"; d="scan'208";a="241215384" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jan 2022 21:30:05 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,282,1635231600"; d="scan'208";a="515357864" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.110]) by orsmga007.jf.intel.com with ESMTP; 11 Jan 2022 21:30:04 -0800 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Wed, 12 Jan 2022 13:28:48 +0800 Message-Id: <20220112052850.1147160-2-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220112052850.1147160-1-wenbin.chen@intel.com> References: <20220112052850.1147160-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH V2] libavcodec/qsvenc: Add low latency P-pyramid support to qsv X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: v7aN2uhpz/0B Add low latency P-pyramid support to qsv. This feature relates to command line option "-p_strategy". To enable this flag, user also need to set "-bf" to 0. P-strategy has two modes "1-simple" and "2-pyramid". The details of the two models refer to https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.md#preftype Signed-off-by: Wenbin Chen --- doc/encoders.texi | 6 ++++++ libavcodec/qsvenc.c | 36 ++++++++++++++++++++++++++++++++++++ libavcodec/qsvenc.h | 3 +++ 3 files changed, 45 insertions(+) diff --git a/doc/encoders.texi b/doc/encoders.texi index 6fc94daa11..3d7c944fba 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -3296,6 +3296,9 @@ to allow changing of frame type from P and B to I. @item @var{adaptive_b} This flag controls changing of frame type from B to P. +@item @var{p_strategy} +Enable P-pyramid: 0-default 1-simple 2-pyramid(bf need to be set to 0). + @item @var{b_strategy} This option controls usage of B frames as reference. @@ -3394,6 +3397,9 @@ Enable rate distortion optimization. @item @var{max_frame_size} Maximum encoded frame size in bytes. +@item @var{p_strategy} +Enable P-pyramid: 0-default 1-simple 2-pyramid(bf need to be set to 0). + @item @var{dblk_idc} This option disable deblocking. It has value in range 0~2. diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 3a51d00ca9..5f405c3d3a 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -315,6 +315,14 @@ static void dump_video_param(AVCodecContext *avctx, QSVEncContext *q, case MFX_B_REF_PYRAMID: av_log(avctx, AV_LOG_VERBOSE, "pyramid"); break; default: av_log(avctx, AV_LOG_VERBOSE, "auto"); break; } + + av_log(avctx, AV_LOG_VERBOSE, "; PRefType: "); + switch (co3->PRefType) { + case MFX_P_REF_DEFAULT: av_log(avctx, AV_LOG_VERBOSE, "default"); break; + case MFX_P_REF_SIMPLE: av_log(avctx, AV_LOG_VERBOSE, "simple"); break; + case MFX_P_REF_PYRAMID: av_log(avctx, AV_LOG_VERBOSE, "pyramid"); break; + default: av_log(avctx, AV_LOG_VERBOSE, "unknown"); break; + } av_log(avctx, AV_LOG_VERBOSE, "\n"); #endif @@ -930,6 +938,34 @@ static int init_video_param(AVCodecContext *avctx, QSVEncContext *q) #if QSV_HAVE_CO3 q->extco3.Header.BufferId = MFX_EXTBUFF_CODING_OPTION3; q->extco3.Header.BufferSz = sizeof(q->extco3); + + if (avctx->codec_id == AV_CODEC_ID_HEVC || + avctx->codec_id == AV_CODEC_ID_H264) { +#if QSV_HAVE_PREF + switch (q->p_strategy) { + case 0: + q->extco3.PRefType = MFX_P_REF_DEFAULT; + break; + case 1: + q->extco3.PRefType = MFX_P_REF_SIMPLE; + break; + case 2: + q->extco3.PRefType = MFX_P_REF_PYRAMID; + break; + default: + q->extco3.PRefType = MFX_P_REF_DEFAULT; + av_log(avctx, AV_LOG_WARNING, + "invalid p_strategy, set to default\n"); + break; + } + if (q->extco3.PRefType == MFX_P_REF_PYRAMID && + avctx->max_b_frames != 0) { + av_log(avctx, AV_LOG_WARNING, + "Please set max_b_frames(-bf) to 0 to enable P-pyramid\n"); + } +#endif + } + #if QSV_HAVE_GPB if (avctx->codec_id == AV_CODEC_ID_HEVC) q->extco3.GPB = q->gpb ? MFX_CODINGOPTION_ON : MFX_CODINGOPTION_OFF; diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h index aa49b35f07..960604cb9a 100644 --- a/libavcodec/qsvenc.h +++ b/libavcodec/qsvenc.h @@ -51,6 +51,7 @@ #define QSV_HAVE_LA_DS QSV_VERSION_ATLEAST(1, 8) #define QSV_HAVE_LA_HRD QSV_VERSION_ATLEAST(1, 11) #define QSV_HAVE_VDENC QSV_VERSION_ATLEAST(1, 15) +#define QSV_HAVE_PREF QSV_VERSION_ATLEAST(1, 16) #define QSV_HAVE_GPB QSV_VERSION_ATLEAST(1, 18) @@ -95,6 +96,7 @@ { "extbrc", "Extended bitrate control", OFFSET(qsv.extbrc), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VE }, \ { "adaptive_i", "Adaptive I-frame placement", OFFSET(qsv.adaptive_i), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VE }, \ { "adaptive_b", "Adaptive B-frame placement", OFFSET(qsv.adaptive_b), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VE }, \ +{ "p_strategy", "Enable P-pyramid: 0-default 1-simple 2-pyramid(bf need to be set to 0).", OFFSET(qsv.p_strategy), AV_OPT_TYPE_INT, { .i64 = 0}, 0, 2, VE }, \ { "b_strategy", "Strategy to choose between I/P/B-frames", OFFSET(qsv.b_strategy), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VE }, \ { "forced_idr", "Forcing I frames as IDR frames", OFFSET(qsv.forced_idr), AV_OPT_TYPE_BOOL,{ .i64 = 0 }, 0, 1, VE }, \ { "low_power", "enable low power mode(experimental: many limitations by mfx version, BRC modes, etc.)", OFFSET(qsv.low_power), AV_OPT_TYPE_BOOL, { .i64 = -1}, -1, 1, VE},\ @@ -187,6 +189,7 @@ typedef struct QSVEncContext { int adaptive_i; int adaptive_b; int b_strategy; + int p_strategy; int cavlc; int int_ref_type;