From patchwork Fri Apr 29 07:31:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Fei W" X-Patchwork-Id: 35497 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:a885:b0:7f:4be2:bd17 with SMTP id ca5csp143145pzb; Fri, 29 Apr 2022 00:38:15 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxKNvaUu19DfWMtlMJbcxDnFYKnsPErgWJJoTop17BCFw89k7M1HnmfvLcclt5Sonvht6f5 X-Received: by 2002:a05:6402:5211:b0:423:fcbe:cf06 with SMTP id s17-20020a056402521100b00423fcbecf06mr39068163edd.39.1651217895781; Fri, 29 Apr 2022 00:38:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1651217895; cv=none; d=google.com; s=arc-20160816; b=uGcLm/zPMyim87jhibAIkv91UtwaMyB5ybKHxlQ+OfE6JowHooTOA0qL9/XlaJhYs9 AboqVCipopwxSABPKXMoAsyL+FEqmhYV+QhS7S7H9CpFS94ViNnZhyW3TKeFfkaHwiVf TJ9RLfZK8VlN7qUUwU7EhFJpzN9RdiFKHdhwg9Cfqwg1qIa7rW3JYjNvc4+dJjgJciL4 N72/GoKc8luRqvxcUULQx6J/SqjyDQm1Rk7kkCFoCtm+4hPrc7e5EbyHO9UWVWOf1lyX xusnHHctOzuB0aBipnRLGBLKGP79mMSscENlPq4ZjmR26VKXKcfREALYz7kjophBB05H B3yw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=sfgll5Uf8Qs1yIq/d8IP6XQjqocGuJoW5YWjDcyTqB8=; b=ao7U0tvx0vzUFuU7aYfp6BNZHB3iHGyxNdulbg7njXuYzVQNkPNKryAaKNOSdq1Eq7 51sCjTv32bIvIklG12bK/DVPlNGW2z8b+ceeLezstgIHghL6fsx2+xwwnEkgPHuRhocZ ztS4BwYPmn+xDLIPZ9hhbx4J2C8lj37iIqpHCI8vdvx4+xzXgWRxMltRsmXxjY5TZbFD NhtB2Bz4TvfyDBeXyVTCeaCeP2FibohL9bakigzsidAKafVDAkcMVjE7wBCeCmnm+1Qn qVCVYxFV5DbvoDIkVvAGpjmsOVwzZzHa+lkZMoqpooGqybd/rZpi9+C9QLk2vj/HY9E1 /6OQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=T98RgAfI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b2-20020a170906150200b006ef7402d64asi5479537ejd.501.2022.04.29.00.38.14; Fri, 29 Apr 2022 00:38:15 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=T98RgAfI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 82A8368B2B6; Fri, 29 Apr 2022 10:38:11 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E646168B262 for ; Fri, 29 Apr 2022 10:38:03 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1651217889; x=1682753889; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=VGYa6cd8v36SC6tO+uJTalPEmN48kAZnMX3zI3QxKUw=; b=T98RgAfIl5jbe557SnyhtWGKJRYjND7tGDXtFCaRxBse6JSHBVKJ5Ixl VY1NGc8PQc0nzAedmSR4HhI7m42sc49mvVM9fxqlXLuHHcLugmqSVe7VQ iQG4n5AiFZj2qcAXLKa5XTyCvZ5yXq+h/FFb8ItZEtzPeM2e1LBJ84gzx z6Rvi9Hcub9u5BNE6sMM4erkCyHkVNcyRaIACCI8/PEFahLhf9I909sDk VfwegDnuhhlypqkKSJfMLw2cMNdE1cw+s9nfCTAi76kUven+zqnkWWXFx btDz01BbSeZZxPMksZ4VdlAdpvnO+tBN8scEKGC5UtpoHhKKEv3WxIBwI w==; X-IronPort-AV: E=McAfee;i="6400,9594,10331"; a="291732732" X-IronPort-AV: E=Sophos;i="5.91,297,1647327600"; d="scan'208";a="291732732" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Apr 2022 00:38:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,297,1647327600"; d="scan'208";a="581964560" Received: from t.sh.intel.com ([10.239.159.147]) by orsmga008.jf.intel.com with ESMTP; 29 Apr 2022 00:37:59 -0700 From: Fei Wang To: ffmpeg-devel@ffmpeg.org Date: Fri, 29 Apr 2022 15:31:41 +0800 Message-Id: <20220429073142.1843209-1-fei.w.wang@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v6 1/2] lavc/vaapi_encode: add support for maxframesize X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Fei Wang , Linjie Fu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: IPYZUkYKQ6l8 From: Linjie Fu Add support for max frame size: - max_frame_size (bytes) to indicate the max allowed size for frame. If the frame size exceeds the limitation, encoder will to control the frame size by adjusting QP value. ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -f rawvideo \ -v verbose -s:v 352x288 -i ./input.yuv -vf format=nv12,hwupload \ -c:v h264_vaapi -profile:v main -g 30 -rc_mode VBR -b:v 500k \ -bf 3 -max_frame_size 40000 -vframes 100 -y ./max_frame_size.h264 Max frame size was enabled since VA-API version (0, 33, 0), but query is available since (1, 5, 0). It will be passed as a parameter in picParam and should be set for each frame. Signed-off-by: Linjie Fu Signed-off-by: Fei Wang --- update: Set ctx->max_frame_size to 0 in CQP mode to avoid to pass VAEncMiscParameterTypeMaxFrameSize type VABuffer. libavcodec/vaapi_encode.c | 72 +++++++++++++++++++++++++++++++++++++++ libavcodec/vaapi_encode.h | 10 +++++- 2 files changed, 81 insertions(+), 1 deletion(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 0e2f5ed447..3b96d4c671 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -365,6 +365,17 @@ static int vaapi_encode_issue(AVCodecContext *avctx, goto fail; } +#if VA_CHECK_VERSION(1, 5, 0) + if (ctx->max_frame_size) { + err = vaapi_encode_make_misc_param_buffer(avctx, pic, + VAEncMiscParameterTypeMaxFrameSize, + &ctx->mfs_params, + sizeof(ctx->mfs_params)); + if (err < 0) + goto fail; + } +#endif + if (pic->type == PICTURE_TYPE_IDR) { if (ctx->va_packed_headers & VA_ENC_PACKED_HEADER_SEQUENCE && ctx->codec->write_sequence_header) { @@ -1869,6 +1880,61 @@ rc_mode_found: return 0; } +static av_cold int vaapi_encode_init_max_frame_size(AVCodecContext *avctx) +{ +#if VA_CHECK_VERSION(1, 5, 0) + VAAPIEncodeContext *ctx = avctx->priv_data; + VAConfigAttrib attr = { VAConfigAttribMaxFrameSize }; + VAStatus vas; + + if (ctx->va_rc_mode == VA_RC_CQP) { + ctx->max_frame_size = 0; + av_log(avctx, AV_LOG_WARNING, "Max frame size is invalid in CQP rate " + "control mode.\n"); + return 0; + } + + vas = vaGetConfigAttributes(ctx->hwctx->display, + ctx->va_profile, + ctx->va_entrypoint, + &attr, 1); + if (vas != VA_STATUS_SUCCESS) { + ctx->max_frame_size = 0; + av_log(avctx, AV_LOG_ERROR, "Failed to query max frame size " + "config attribute: %d (%s).\n", vas, vaErrorStr(vas)); + return AVERROR_EXTERNAL; + } + + if (attr.value == VA_ATTRIB_NOT_SUPPORTED) { + ctx->max_frame_size = 0; + av_log(avctx, AV_LOG_WARNING, "Max frame size attribute " + "is not supported.\n"); + } else { + VAConfigAttribValMaxFrameSize attr_mfs; + attr_mfs.value = attr.value; + // Prefer to use VAEncMiscParameterTypeMaxFrameSize for max frame size. + if (!attr_mfs.bits.max_frame_size && attr_mfs.bits.multiple_pass) { + ctx->max_frame_size = 0; + av_log(avctx, AV_LOG_WARNING, "Driver only support multiple pass " + "max frame size which have not been implemented in FFmpeg.\n"); + return 0; + } + + ctx->mfs_params = (VAEncMiscParameterBufferMaxFrameSize){ + .max_frame_size = ctx->max_frame_size * 8, + }; + + av_log(avctx, AV_LOG_VERBOSE, "Max Frame Size: %d bytes.\n", + ctx->max_frame_size); + } +#else + av_log(avctx, AV_LOG_WARNING, "Max Frame Size is not support with " + "this VA version.\n"); +#endif + + return 0; +} + static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) { VAAPIEncodeContext *ctx = avctx->priv_data; @@ -2548,6 +2614,12 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) goto fail; } + if (ctx->max_frame_size) { + err = vaapi_encode_init_max_frame_size(avctx); + if (err < 0) + goto fail; + } + vas = vaCreateConfig(ctx->hwctx->display, ctx->va_profile, ctx->va_entrypoint, ctx->config_attributes, ctx->nb_config_attributes, diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index af0588c30b..359f954fff 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -191,6 +191,9 @@ typedef struct VAAPIEncodeContext { // Desired B frame reference depth. int desired_b_depth; + // Max Frame Size + int max_frame_size; + // Explicitly set RC mode (otherwise attempt to pick from // available modes). int explicit_rc_mode; @@ -268,6 +271,7 @@ typedef struct VAAPIEncodeContext { VAEncMiscParameterRateControl rc_params; VAEncMiscParameterHRD hrd_params; VAEncMiscParameterFrameRate fr_params; + VAEncMiscParameterBufferMaxFrameSize mfs_params; #if VA_CHECK_VERSION(0, 36, 0) VAEncMiscParameterBufferQualityLevel quality_params; #endif @@ -478,7 +482,11 @@ int ff_vaapi_encode_close(AVCodecContext *avctx); "Increase this to improve single channel performance. This option " \ "doesn't work if driver doesn't implement vaSyncBuffer function.", \ OFFSET(common.async_depth), AV_OPT_TYPE_INT, \ - { .i64 = 2 }, 1, MAX_ASYNC_DEPTH, FLAGS } + { .i64 = 2 }, 1, MAX_ASYNC_DEPTH, FLAGS }, \ + { "max_frame_size", \ + "Maximum frame size (in bytes)",\ + OFFSET(common.max_frame_size), AV_OPT_TYPE_INT, \ + { .i64 = 0 }, 0, INT_MAX, FLAGS } #define VAAPI_ENCODE_RC_MODE(name, desc) \ { #name, desc, 0, AV_OPT_TYPE_CONST, { .i64 = RC_MODE_ ## name }, \ From patchwork Fri Apr 29 07:31:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Fei W" X-Patchwork-Id: 35498 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:a885:b0:7f:4be2:bd17 with SMTP id ca5csp143197pzb; Fri, 29 Apr 2022 00:38:25 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzXRsOaEPUDlLXF7shwhGTz9MJ4879h8H8zag2kUP0EAbNrL68gJP5rUPPziMsa5vDw5aLz X-Received: by 2002:a05:6402:518f:b0:426:25ed:4a2 with SMTP id q15-20020a056402518f00b0042625ed04a2mr8330557edd.9.1651217905670; Fri, 29 Apr 2022 00:38:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1651217905; cv=none; d=google.com; s=arc-20160816; b=sUq65mcaBsLAsQGT6kYhqNLZGueQiKrqY/PuO7DEB1ucu5KHVIiZWv/ENyJfmZNUJN esBlC4ugkFVBVQElpl+hVuzq2iFw2vK+tTNLklMILbM5lcnYEh6QrYN1uqotF4jXX4cS 9sJFS3n9vQWR5yJq0WPDf4Qm+SR6+lwOovygNziH/grdeMfU4m3YH9zBM1oyRWAj6UXt aNnU7Yjeo2lje1zjXMqjdxfEUG1BuItPbGVvuukIfYZrgBuComPVo/lAWjIQ4p7NlmvK 4mdaxypb2FwNOlwvi1CLdGw5nwSTYclb1nRw+lWOkzS5XCjy0ITs1A7JDqYb3Xc5nW7A b9WA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=Str0fn9LEqNDEKzai8Sy3ze54FOxP8xaQQiXX8HpAH4=; b=d1KaSVVAjttqoIlhjZ+83+PXTF3ka4IU14IMGcx3teDa77xzgPIEE4fcUwt7PWWzAV z62vK3sf55zXjQoVTI7WtM/6GzWc+7IseBYCKcmDQE+5fmLNypJJ9sB5FqOFic9ogQQD jSxk9EOFDgru6pGIfGWk3He4yoq3F52pvQHpsZ7HaFSdjeFUgvaSUKuQmcEGzcouitMe zgxWOPl/LOa7iMO3M4hmRWuwIrlNTjK6RKzF+u/0OXmXmHxpMq9QbNyRvqut+kCHSCDV +F7ozDgwHeRvBwof9vkL1dlMtSIieH5AeqzzQD3Z8r2DDx7A4bqPSCh8MKB4FQGDbphT KXdQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=UxyNMNaD; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id jr4-20020a170906514400b006f3f040eb34si1198531ejc.670.2022.04.29.00.38.24; Fri, 29 Apr 2022 00:38:25 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=UxyNMNaD; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A85BD68AF13; Fri, 29 Apr 2022 10:38:16 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DB4BE68AF13 for ; Fri, 29 Apr 2022 10:38:09 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1651217895; x=1682753895; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Ei8JUkL2lyh3BF9/RxFfugMKNCQrFAz5Ehk4nqytHSY=; b=UxyNMNaDz7j5Fn4TeoaTU0eHRBO1WY/mAq9QKl4VzUhf22UYT9bCGbHM STjJ/nt7CF1sYviTBKHeJh6rb2mHcXON0E+N5gJVFNmz/iAoY9NsX4X/S p5xYf5LqRNDPGLcO7fGnCgkHDJ/RhxRwjmXiFgiSOWhQCT39Z0eCayrrj iow4ZXODv1/VtMdf/B0l7nydzKU5yt4MCc7u3TTVr2wshBP5odebNs+xH HiFE3uYEpZLw6uylOAdh1Vff2NPM64usLLvVVYzesn+yjp1W+JW+AXoQQ gohLkbyij6W3nvbeDMTN1GmFK7qEobK5K1rKb4fO/7msKJe339GZdMrho g==; X-IronPort-AV: E=McAfee;i="6400,9594,10331"; a="291732733" X-IronPort-AV: E=Sophos;i="5.91,297,1647327600"; d="scan'208";a="291732733" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Apr 2022 00:38:02 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,297,1647327600"; d="scan'208";a="581964573" Received: from t.sh.intel.com ([10.239.159.147]) by orsmga008.jf.intel.com with ESMTP; 29 Apr 2022 00:38:01 -0700 From: Fei Wang To: ffmpeg-devel@ffmpeg.org Date: Fri, 29 Apr 2022 15:31:42 +0800 Message-Id: <20220429073142.1843209-2-fei.w.wang@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220429073142.1843209-1-fei.w.wang@intel.com> References: <20220429073142.1843209-1-fei.w.wang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v6 2/2] doc/vaapi_encode: add documentations for max_frame_size X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Fei Wang , Linjie Fu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: TDTZyTkerI1v From: Linjie Fu Add docs for max_frame_size option. Signed-off-by: Linjie Fu Signed-off-by: Fei Wang --- doc/encoders.texi | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/doc/encoders.texi b/doc/encoders.texi index 966032a720..43f6288e33 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -3665,6 +3665,11 @@ performance. This option doesn't work if driver doesn't implement vaSyncBuffer function. Please make sure there are enough hw_frames allocated if a large number of async_depth is used. +@item max_frame_size +Set the allowed max size in bytes for each frame. If the frame size exceeds +the limitation, encoder will adjust the QP value to control the frame size. +Invalid in CQP rate control mode. + @item rc_mode Set the rate control mode to use. A given driver may only support a subset of modes.