From patchwork Thu May 5 09:07:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Fei W" X-Patchwork-Id: 35595 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:a885:b0:7f:4be2:bd17 with SMTP id ca5csp946614pzb; Thu, 5 May 2022 02:13:51 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxDVpHExSnr7eUM2dmKSa2+X3FAa7KizFReUbkTnjMocysrAgbGxHpwSCK9bqW2oaVao9yY X-Received: by 2002:a05:6402:51cb:b0:428:2617:64ba with SMTP id r11-20020a05640251cb00b00428261764bamr5162340edd.135.1651742031188; Thu, 05 May 2022 02:13:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1651742031; cv=none; d=google.com; s=arc-20160816; b=dduc+bwHgtJ0pxPSWn0dkyruM5UAk3kg1gTUOPxii67wIv5tvhAtVDaXyf4PLLj3uE 61FDgdAdmZG32qJ5DcRNn4xXecNXeYC/oej5PEjp3aOl7DdSm3ZYiS6Q0nJ1WIOnI6u8 2nUqJ3yTOOvjb1wjL9lT+dazP+qQof76KNBGLcXaJEWwysJVCxe5DlpHsaZx0UCi/1Qo yK0qjYE30VrO22hd0ZDk8WIiDA6psd3OT0hmLTRU8NdnAswxn8pk+FIMbN+KqCoc/e51 iKD/296RbuWbrM8ETZSchKRWZEPthAnddq0f0oCdHbB97uZ5celXyhek1FuwXkQ1M+f0 SoHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=FKi4tpi9YzMvlkyZ4MmPbgObtwFFWdsD+DXYpBwkdZQ=; b=ts/Lb3BJzvENilFjTvnq0XQ1tHURGUTqSpRL2AQaG0AR6llzNnJbMFjpxELV49aaTd tvpJ6cYu0Duhw/duHGbmxdfXTTfLQQIXKfLBK6Cq8UGOjeQF6PXXXuoQ9fEIsazNmULS KOlPKHZXRQHvuE+bf10qYijaqata3rpSI9He2RXTW1sKCn1dzXJ5++q3G87Y1DX/WW2E M8hU3BbwGiaxJX41OWV9xBJOjToUYqo57zDAyKLNszfCaRvMFgwRexp78vzkn0c9Wccd KcoGyo648KQxAkiN90uk+Eor2zOUkLpaeRrr3Lznh4JTP+cDqCKpmoIpbSMbD5RHimTi VkAA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Cwr2WvIW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id sd22-20020a170906ce3600b006e07d5f6986si1676786ejb.933.2022.05.05.02.13.50; Thu, 05 May 2022 02:13:51 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Cwr2WvIW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 40241688093; Thu, 5 May 2022 12:13:48 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D6734688093 for ; Thu, 5 May 2022 12:13:40 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1651742026; x=1683278026; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=dF/zY6WAyNXyjlPSz+sv2OioBNdXh+l2Z1nZFbhwnNs=; b=Cwr2WvIWsyzlvnLN6hJ1b5OYb+a3D+tL+Ky6cSTB6WqTb7zZmmGHc5zl fqQ55iMcdzoeqadnzzAy2tGyDdgH0cDis0RD1Sy5xDV6zhIyRw6vy3YGz RwxZDRpiH+oz5L4ci81/d8CZzVQQD7N2BCKbHtHBQ2mqZZf4UWXec9qie 04MXP8vMh2+dcQsSgEOmp3T5dtOlfxjjaiPisQjTfpqFeob0ZeHu3LTTb MHFJTgTF4KyWQlA0HBY8WdW+6Tvgw3xWFFaTCV704MF7TOYuXd3NJ6J8m K4BUPmFJ7aEJ2LOZHF+6HaURuiZXOATpGtrKY3EC5eyO9KcX9SgS1RQxS g==; X-IronPort-AV: E=McAfee;i="6400,9594,10337"; a="266892137" X-IronPort-AV: E=Sophos;i="5.91,200,1647327600"; d="scan'208";a="266892137" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2022 02:13:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,200,1647327600"; d="scan'208";a="563152011" Received: from t.sh.intel.com ([10.239.159.147]) by orsmga007.jf.intel.com with ESMTP; 05 May 2022 02:13:37 -0700 From: Fei Wang To: ffmpeg-devel@ffmpeg.org Date: Thu, 5 May 2022 17:07:14 +0800 Message-Id: <20220505090715.2016246-1-fei.w.wang@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v7 1/2] lavc/vaapi_encode: add support for maxframesize X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Fei Wang , Linjie Fu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Qh4AAdrqNMZ1 From: Linjie Fu Add support for max frame size: - max_frame_size (bytes) to indicate the max allowed size for frame. Control each encoded frame size into target limitation size by adjusting whole frame's average QP value. The driver will use multi passes to adjust average QP setp by step to achieve the target, and the result may not strictly guaranteed. Frame size may exceed target alone with using the maximum average QP value. The failure always happens on the intra(especially the first intra frame of a new GOP) frames or set max_frame_size with a very small number. example cmdline: ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -f rawvideo \ -v verbose -s:v 352x288 -i ./input.yuv -vf format=nv12,hwupload \ -c:v h264_vaapi -profile:v main -g 30 -rc_mode VBR -b:v 500k \ -bf 3 -max_frame_size 40000 -vframes 100 -y ./max_frame_size.h264 Max frame size was enabled since VA-API version (0, 33, 0), but query is available since (1, 5, 0). It will be passed as a parameter in picParam and should be set for each frame. Signed-off-by: Linjie Fu Signed-off-by: Fei Wang --- update: 1. return error when fail to set max frame size. 2. refine commit and debug message. libavcodec/vaapi_encode.c | 74 +++++++++++++++++++++++++++++++++++++++ libavcodec/vaapi_encode.h | 10 +++++- 2 files changed, 83 insertions(+), 1 deletion(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 0e2f5ed447..284ce29888 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -365,6 +365,17 @@ static int vaapi_encode_issue(AVCodecContext *avctx, goto fail; } +#if VA_CHECK_VERSION(1, 5, 0) + if (ctx->max_frame_size) { + err = vaapi_encode_make_misc_param_buffer(avctx, pic, + VAEncMiscParameterTypeMaxFrameSize, + &ctx->mfs_params, + sizeof(ctx->mfs_params)); + if (err < 0) + goto fail; + } +#endif + if (pic->type == PICTURE_TYPE_IDR) { if (ctx->va_packed_headers & VA_ENC_PACKED_HEADER_SEQUENCE && ctx->codec->write_sequence_header) { @@ -1869,6 +1880,63 @@ rc_mode_found: return 0; } +static av_cold int vaapi_encode_init_max_frame_size(AVCodecContext *avctx) +{ +#if VA_CHECK_VERSION(1, 5, 0) + VAAPIEncodeContext *ctx = avctx->priv_data; + VAConfigAttrib attr = { VAConfigAttribMaxFrameSize }; + VAStatus vas; + + if (ctx->va_rc_mode == VA_RC_CQP) { + ctx->max_frame_size = 0; + av_log(avctx, AV_LOG_ERROR, "Max frame size is invalid in CQP rate " + "control mode.\n"); + return AVERROR(EINVAL); + } + + vas = vaGetConfigAttributes(ctx->hwctx->display, + ctx->va_profile, + ctx->va_entrypoint, + &attr, 1); + if (vas != VA_STATUS_SUCCESS) { + ctx->max_frame_size = 0; + av_log(avctx, AV_LOG_ERROR, "Failed to query max frame size " + "config attribute: %d (%s).\n", vas, vaErrorStr(vas)); + return AVERROR_EXTERNAL; + } + + if (attr.value == VA_ATTRIB_NOT_SUPPORTED) { + ctx->max_frame_size = 0; + av_log(avctx, AV_LOG_ERROR, "Max frame size attribute " + "is not supported.\n"); + return AVERROR(EINVAL); + } else { + VAConfigAttribValMaxFrameSize attr_mfs; + attr_mfs.value = attr.value; + // Prefer to use VAEncMiscParameterTypeMaxFrameSize for max frame size. + if (!attr_mfs.bits.max_frame_size && attr_mfs.bits.multiple_pass) { + ctx->max_frame_size = 0; + av_log(avctx, AV_LOG_ERROR, "Driver only supports multiple pass " + "max frame size which has not been implemented in FFmpeg.\n"); + return AVERROR(EINVAL); + } + + ctx->mfs_params = (VAEncMiscParameterBufferMaxFrameSize){ + .max_frame_size = ctx->max_frame_size * 8, + }; + + av_log(avctx, AV_LOG_VERBOSE, "Set max frame size: %d bytes.\n", + ctx->max_frame_size); + } +#else + av_log(avctx, AV_LOG_ERROR, "The max frame size option is not supported with " + "this VAAPI version.\n"); + return AVERROR(EINVAL); +#endif + + return 0; +} + static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) { VAAPIEncodeContext *ctx = avctx->priv_data; @@ -2548,6 +2616,12 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) goto fail; } + if (ctx->max_frame_size) { + err = vaapi_encode_init_max_frame_size(avctx); + if (err < 0) + goto fail; + } + vas = vaCreateConfig(ctx->hwctx->display, ctx->va_profile, ctx->va_entrypoint, ctx->config_attributes, ctx->nb_config_attributes, diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index af0588c30b..359f954fff 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -191,6 +191,9 @@ typedef struct VAAPIEncodeContext { // Desired B frame reference depth. int desired_b_depth; + // Max Frame Size + int max_frame_size; + // Explicitly set RC mode (otherwise attempt to pick from // available modes). int explicit_rc_mode; @@ -268,6 +271,7 @@ typedef struct VAAPIEncodeContext { VAEncMiscParameterRateControl rc_params; VAEncMiscParameterHRD hrd_params; VAEncMiscParameterFrameRate fr_params; + VAEncMiscParameterBufferMaxFrameSize mfs_params; #if VA_CHECK_VERSION(0, 36, 0) VAEncMiscParameterBufferQualityLevel quality_params; #endif @@ -478,7 +482,11 @@ int ff_vaapi_encode_close(AVCodecContext *avctx); "Increase this to improve single channel performance. This option " \ "doesn't work if driver doesn't implement vaSyncBuffer function.", \ OFFSET(common.async_depth), AV_OPT_TYPE_INT, \ - { .i64 = 2 }, 1, MAX_ASYNC_DEPTH, FLAGS } + { .i64 = 2 }, 1, MAX_ASYNC_DEPTH, FLAGS }, \ + { "max_frame_size", \ + "Maximum frame size (in bytes)",\ + OFFSET(common.max_frame_size), AV_OPT_TYPE_INT, \ + { .i64 = 0 }, 0, INT_MAX, FLAGS } #define VAAPI_ENCODE_RC_MODE(name, desc) \ { #name, desc, 0, AV_OPT_TYPE_CONST, { .i64 = RC_MODE_ ## name }, \ From patchwork Thu May 5 09:07:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Fei W" X-Patchwork-Id: 35596 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:a885:b0:7f:4be2:bd17 with SMTP id ca5csp946671pzb; Thu, 5 May 2022 02:14:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwe6PFSp7sty+hvtSq9g7h/5dsDmECKCxh95fsE4GN9FAXK9ieAN7E3yoicTZMu+reHE1OZ X-Received: by 2002:a17:906:fad6:b0:6f3:9ea7:1d49 with SMTP id lu22-20020a170906fad600b006f39ea71d49mr24933196ejb.635.1651742039712; Thu, 05 May 2022 02:13:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1651742039; cv=none; d=google.com; s=arc-20160816; b=S9K+3qM2oxmu3CKkd205qd9bm287sUdlUPwJNmqY6oLP46W2sN+uxVDH6lqCNQ+O4B 0rlSoxj293hU+huJHRmaEouPE3Xat43IFORAKEVwoqjnYRwde3y9wGRYMBKGA9W/G+GH aob8NJDKhSxnh6TCkbYKKUBOceT9RDF8sz8vCCZTJJeroORJRukepyHCsCnXMdbbhAe8 QNzGmnadLZJmgIp0JdRoHZ6+f2jKTswrA0qjq+sstAQ/GuiTQ9kb9EeMiydjCzqaLdjY URK/xUdpmpPOt+nKK9WwtveJO5Oc8+5/ZIjgzp6NxLXJx6fTTyk+E5V23XLyrFtTfpOa d/aw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=Str0fn9LEqNDEKzai8Sy3ze54FOxP8xaQQiXX8HpAH4=; b=i3fJUPpA97gBJlVfR7tbqTvWVlMNNjMcUIL1NhN7ZitVHbweM83CzZpcT4VrK1gcrA PvjeHcBrsRRHFa84DhQU8LKykcBUxp85+ai7MIsPFQVu+sZrXE0AamJ3gNFdFo/cofdZ 9Woxz1yvg0of/xni5B7vplFr8DCz9hEHUTlr5L/ga/5OKIi22vxPwnbQTQfYF23dXZAn NpcYASm2CETrnbSjc9PWzYmQSzYvV4ew/d55UTfVTH+OZyORxh7sAI2EV1U2TlH8va9Y AgrVbMW9eFModsCwQGnqJIddlGsUlqtmaTXT8foueTneWR5q47tPtfrZSr2IwyupDP5x +2ww== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=kaSH0pR5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id j13-20020a05640211cd00b0041d76c3b504si1374328edw.545.2022.05.05.02.13.59; Thu, 05 May 2022 02:13:59 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=kaSH0pR5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 60FAE68B3AA; Thu, 5 May 2022 12:13:55 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BC81868B3AA for ; Thu, 5 May 2022 12:13:46 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1651742032; x=1683278032; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Ei8JUkL2lyh3BF9/RxFfugMKNCQrFAz5Ehk4nqytHSY=; b=kaSH0pR5uIjw2YdxLzj6j8LdWrNqOdWK8IFU+jbpkCLgj+gls+N2MCXd v08Zkkb/I/gBgGNa+RD5eSuNqG9T/I5td1NBVTusuUfp+qtbiiDESzAZe aYDY9V4+ASNtKAzJQV2ZgrD1C3R2sSRHZU2aOAo95FxzDje3EwZ2OF9MO cKliIG/TBXY2qxTnt/e/pCSfGvKS8h9AlLFgXMg0M7SwWXNVBvUkjDI34 UrrATjU8wmIRM9PZSMXHQdOGlRu0r8KRTxmdU6vRkJHX9/wFRJ3EfDRS/ V2M5pVwLk0uJ4dctNGGgjlcV+7FoRO0MrEiPTpEb5GxY08fDPjq20RjVL w==; X-IronPort-AV: E=McAfee;i="6400,9594,10337"; a="266892140" X-IronPort-AV: E=Sophos;i="5.91,200,1647327600"; d="scan'208";a="266892140" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2022 02:13:39 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,200,1647327600"; d="scan'208";a="563152020" Received: from t.sh.intel.com ([10.239.159.147]) by orsmga007.jf.intel.com with ESMTP; 05 May 2022 02:13:38 -0700 From: Fei Wang To: ffmpeg-devel@ffmpeg.org Date: Thu, 5 May 2022 17:07:15 +0800 Message-Id: <20220505090715.2016246-2-fei.w.wang@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220505090715.2016246-1-fei.w.wang@intel.com> References: <20220505090715.2016246-1-fei.w.wang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v7 2/2] doc/vaapi_encode: add documentations for max_frame_size X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Fei Wang , Linjie Fu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 0daQZwgZu4zj From: Linjie Fu Add docs for max_frame_size option. Signed-off-by: Linjie Fu Signed-off-by: Fei Wang --- doc/encoders.texi | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/doc/encoders.texi b/doc/encoders.texi index 966032a720..43f6288e33 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -3665,6 +3665,11 @@ performance. This option doesn't work if driver doesn't implement vaSyncBuffer function. Please make sure there are enough hw_frames allocated if a large number of async_depth is used. +@item max_frame_size +Set the allowed max size in bytes for each frame. If the frame size exceeds +the limitation, encoder will adjust the QP value to control the frame size. +Invalid in CQP rate control mode. + @item rc_mode Set the rate control mode to use. A given driver may only support a subset of modes.