From patchwork Tue Mar 22 14:11:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Fei W" X-Patchwork-Id: 34898 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:ab0:5fda:0:0:0:0:0 with SMTP id g26csp685559uaj; Tue, 22 Mar 2022 07:17:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwCQfq6tKQGyFtZopTJx5y8MFu8l6mtKzCjKldj+Y/+lkVm6H/oDA9pqc54GrmoyG0CPqKu X-Received: by 2002:a17:906:32d0:b0:6ce:e1cf:3f2e with SMTP id k16-20020a17090632d000b006cee1cf3f2emr25154783ejk.214.1647958672558; Tue, 22 Mar 2022 07:17:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1647958672; cv=none; d=google.com; s=arc-20160816; b=jWJh4uWLTC3w3o1lnyLuHQWxqIqALgXwxSLUvttaV9+4ExTYeUcRQgDzJoCRWxgIsd 1sWQreEQsfXBuxgo3KVkdE+bSfx/g5W4MWROysgH3pbddsT9xBYxUVtn80RIULyImqgG gSc8xqil/ZWhDGJ1Anv/NMIFOZOuKMTLWzSJJb55qMgqo3JvOP5VTBsIpuHBI1Hc9sLP nKXC8BMvmYo2zK2bsl3CsU/coMjcGVRI3RglAwhTnBBAUfTKMmupkBOv3F6vMzgpYuaG x69A57CL34vKHtzPYjC69c2m4yeupIlhFb6dl5l5e+xs8lN0ezj0AOs6zDitRR8pigrt kk9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=SWnA8IJchkQRiPZ/voko/rQ5DeAjBcaQmBTLt/iNOAk=; b=pd2FEgzn+EuCYCE5HBlDaU6pwhoSlS8OJK1+pExdK5FsGqOfjkCFwsi7pV0d4UY3WO 77caOf4WJi7w5hRdoxXD+JbAy5zlUDGNmV+nbpm87tzKWI5c+VRS+YxfbWSqpEhPpFUJ P6mfByrRA5kAJbYLSitjR4m7s5ne3f56/DU/R7egWd4/MgFfrHxYHeymMsPEvLR/f3me dPHS0HJXy8EEWtXcYFpYJFCKlNFlBaYg8b8boH8toVMRblKUXszxUS/W9dkbDSmnX/Wj rXzwwUWGC/nezjx9YynD9vKa7M5rtnxdQAN+0Y4zEl+lnJsE/mRFbci5cwyus6ucJWyT zdeg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=TeQolPnn; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i12-20020a17090639cc00b006df76385e0bsi9102721eje.683.2022.03.22.07.17.52; Tue, 22 Mar 2022 07:17:52 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=TeQolPnn; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3C69D687EBB; Tue, 22 Mar 2022 16:17:49 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 741AA687EBB for ; Tue, 22 Mar 2022 16:17:41 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1647958666; x=1679494666; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=HSRz9t0xhsS2BAWOtcASa8Mjd6ij8noNERc/FP1bKFc=; b=TeQolPnn2iuytsBc2y+qCEdtyzbB6LdZc/sztBOBapHNx+n1yqqLYIWh 9CW5izNcE7zq1uj6U8vtISjgcW8Y1FY0iP4xmK/gGWQRQzMu22QrjKr79 Nl97ZuXarGeg2IDHr8DR1q6iN21kY4wu0ojjeQcA9Vbiu894s59tDfRjs gzB/s4TexdLqBdsZRFjadXewrjFeQlhxNfN8XGvNpuTyB4HFWCvflRgoF T52QxedpQoy4AegEuA/FY865A6zXrpYKtRzp/vKeVbVDb88w1BKqbcyoc pwCADZ4MS8SaPXW9Ig1Tt8vDyhmfJXkBnrvxXgkTMkMcT6SMJke4KsVjz w==; X-IronPort-AV: E=McAfee;i="6200,9189,10293"; a="237769895" X-IronPort-AV: E=Sophos;i="5.90,201,1643702400"; d="scan'208";a="237769895" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Mar 2022 07:17:39 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,201,1643702400"; d="scan'208";a="716944007" Received: from t.sh.intel.com ([10.239.159.147]) by orsmga005.jf.intel.com with ESMTP; 22 Mar 2022 07:17:38 -0700 From: Fei Wang To: ffmpeg-devel@ffmpeg.org Date: Tue, 22 Mar 2022 22:11:18 +0800 Message-Id: <20220322141119.595627-1-fei.w.wang@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 1/2] lavc/vaapi_encode: add support for maxframesize X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Fei Wang , Linjie Fu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: LpjaovY615RC From: Linjie Fu Add support for max frame size: - max_frame_size (bytes) to indicate the max allowed size for frame. If the frame size exceeds the limitation, encoder will to control the frame size by adjusting QP value. - MFS_NUM_PASSES to indicate number of passes for QP adjust. - MFS_DELTA_QP to indicate adjust qp value per pass. To simplify the usage, default QP adjust is set to delta_qp[4] = {1, 1, 1, 1}. Use new_qp for encoder if frame size exceeds the limitation: new_qp = base_qp + delta_qp[0] + delta_qp[1] + ... ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -f rawvideo \ -v verbose -s:v 352x288 -i ./input.yuv -vf format=nv12,hwupload \ -c:v h264_vaapi -profile:v main -g 30 -bf 3 -max_frame_size 40000 \ -vframes 100 -y ./max_frame_size.h264 Max frame size was enabled since VA-API version (1, 3, 0), but query is available since (1, 5, 0). It will be passed as a parameter in picParam and should be set for each frame. Signed-off-by: Linjie Fu Signed-off-by: Fei Wang --- 1. re-send the 2 legacy patch: https://patchwork.ffmpeg.org/project/ffmpeg/patch/20190715105936.4860-1-linjie.fu@intel.com/ https://patchwork.ffmpeg.org/project/ffmpeg/patch/20190715110000.5069-1-linjie.fu@intel.com/ libavcodec/vaapi_encode.c | 67 +++++++++++++++++++++++++++++++++++++++ libavcodec/vaapi_encode.h | 19 +++++++++-- 2 files changed, 84 insertions(+), 2 deletions(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index ffd6cb1c25..b2782e6f9e 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -365,6 +365,17 @@ static int vaapi_encode_issue(AVCodecContext *avctx, goto fail; } +#if VA_CHECK_VERSION(1, 5, 0) + if (ctx->max_frame_size) { + err = vaapi_encode_make_misc_param_buffer(avctx, pic, + VAEncMiscParameterTypeMultiPassFrameSize, + &ctx->mfs_params, + sizeof(ctx->mfs_params)); + if (err < 0) + goto fail; + } +#endif + if (pic->type == PICTURE_TYPE_IDR) { if (ctx->va_packed_headers & VA_ENC_PACKED_HEADER_SEQUENCE && ctx->codec->write_sequence_header) { @@ -1869,6 +1880,54 @@ rc_mode_found: return 0; } +static av_cold int vaapi_encode_init_max_frame_size(AVCodecContext *avctx) +{ +#if VA_CHECK_VERSION(1, 5, 0) + VAAPIEncodeContext *ctx = avctx->priv_data; + VAConfigAttrib attr = { VAConfigAttribMaxFrameSize }; + VAStatus vas; + int i; + + vas = vaGetConfigAttributes(ctx->hwctx->display, + ctx->va_profile, + ctx->va_entrypoint, + &attr, 1); + if (vas != VA_STATUS_SUCCESS) { + ctx->max_frame_size = 0; + av_log(avctx, AV_LOG_ERROR, "Failed to query max frame size " + "config attribute: %d (%s).\n", vas, vaErrorStr(vas)); + return AVERROR_EXTERNAL; + } + + if (attr.value == VA_ATTRIB_NOT_SUPPORTED) { + ctx->max_frame_size = 0; + av_log(avctx, AV_LOG_WARNING, "Max frame size attribute " + "is not supported.\n"); + } else { + ctx->delta_qp = av_calloc(MFS_NUM_PASSES, sizeof(uint8_t)); + if (!ctx->delta_qp) { + return AVERROR(ENOMEM); + } + for (i = 0; i delta_qp[i] = MFS_DELTA_QP; + + ctx->mfs_params = (VAEncMiscParameterBufferMultiPassFrameSize){ + .max_frame_size = ctx->max_frame_size, + .num_passes = MFS_NUM_PASSES, + .delta_qp = ctx->delta_qp, + }; + + av_log(avctx, AV_LOG_VERBOSE, "Max Frame Size: %d bytes.\n ", + ctx->max_frame_size); + } +#else + av_log(avctx, AV_LOG_WARNING, "Max Frame Size is " + "not supported with this VA version.\n"); +#endif + + return 0; +} + static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) { VAAPIEncodeContext *ctx = avctx->priv_data; @@ -2475,6 +2534,12 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) goto fail; } + if (ctx->max_frame_size) { + err = vaapi_encode_init_max_frame_size(avctx); + if (err < 0) + goto fail; + } + vas = vaCreateConfig(ctx->hwctx->display, ctx->va_profile, ctx->va_entrypoint, ctx->config_attributes, ctx->nb_config_attributes, @@ -2618,6 +2683,8 @@ av_cold int ff_vaapi_encode_close(AVCodecContext *avctx) av_frame_free(&ctx->frame); + if (ctx->delta_qp) + av_freep(&ctx->delta_qp); av_freep(&ctx->codec_sequence_params); av_freep(&ctx->codec_picture_params); av_fifo_freep2(&ctx->encode_fifo); diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index 1b40819c69..72f9256a8e 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -60,6 +60,11 @@ enum { PICTURE_TYPE_B = 3, }; +enum { + MFS_NUM_PASSES = 4, + MFS_DELTA_QP = 1, +}; + typedef struct VAAPIEncodeSlice { int index; int row_start; @@ -191,6 +196,9 @@ typedef struct VAAPIEncodeContext { // Desired B frame reference depth. int desired_b_depth; + // Max Frame Size + int max_frame_size; + // Explicitly set RC mode (otherwise attempt to pick from // available modes). int explicit_rc_mode; @@ -271,7 +279,9 @@ typedef struct VAAPIEncodeContext { #if VA_CHECK_VERSION(0, 36, 0) VAEncMiscParameterBufferQualityLevel quality_params; #endif - +#if VA_CHECK_VERSION(1, 3, 0) + VAEncMiscParameterBufferMultiPassFrameSize mfs_params; +#endif // Per-sequence parameter structure (VAEncSequenceParameterBuffer*). void *codec_sequence_params; @@ -331,6 +341,7 @@ typedef struct VAAPIEncodeContext { int idr_counter; int gop_counter; int end_of_stream; + uint8_t *delta_qp; // Whether the driver supports ROI at all. int roi_allowed; @@ -470,7 +481,11 @@ int ff_vaapi_encode_close(AVCodecContext *avctx); "Increase this to improve single channel performance. This option " \ "doesn't work if driver doesn't implement vaSyncBuffer function.", \ OFFSET(common.async_depth), AV_OPT_TYPE_INT, \ - { .i64 = 2 }, 1, MAX_ASYNC_DEPTH, FLAGS } + { .i64 = 2 }, 1, MAX_ASYNC_DEPTH, FLAGS }, \ + { "max_frame_size", \ + "Maximum frame size (in bytes)",\ + OFFSET(common.max_frame_size), AV_OPT_TYPE_INT, \ + { .i64 = 0 }, 0, INT_MAX, FLAGS } #define VAAPI_ENCODE_RC_MODE(name, desc) \ { #name, desc, 0, AV_OPT_TYPE_CONST, { .i64 = RC_MODE_ ## name }, \