From patchwork Fri Apr 22 05:36:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Fei W" X-Patchwork-Id: 35372 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b9e:b0:7d:cfb5:dc7c with SMTP id b30csp195162pzh; Thu, 21 Apr 2022 22:43:20 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyQedMHY2ag6rmTazj+FlitdXvwN0843qBcA154JhyYOUxoeaoy1yFOa2pkSUUvrecqJj1z X-Received: by 2002:a17:906:e18:b0:6e8:69c2:528c with SMTP id l24-20020a1709060e1800b006e869c2528cmr2528574eji.439.1650606200183; Thu, 21 Apr 2022 22:43:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1650606200; cv=none; d=google.com; s=arc-20160816; b=RS+he6B9PcBU9foPIPU+eSs+W3afAgF3dRj/Wm8C60gp9RilkGWU8mh9q/wdIC26gk qRuq09yAPr6AyJlinLGe+vSfJP/y+xP1MM5xnnC0pWmLc3z9o82rW2p8g6/YhfrS7W53 eJkqEP1TbhPQAJ2Mna/Mc4sWBk73EkQnGgJYKjpxo+bYhTo/+z8swC+Ir4QFK6j3Vz1W Dh1DiRnZc1xUtdatPlllzVm8SUfYH/zyRNEbnxtrO80RCtaEmSSRPQItsi8mWEh/xrit NhwT7aFIlPbBKm19Zcnx5HeLqIIZigF2AqLGJtRthNzP9tP3w/xd+VmM9cz2KKorypOB OGzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=VEOAaW/sGC7UqbkWfSKOpIAQREOc5QWnySkZs43vJLQ=; b=A+0mIHE51CvvM9NkheSrUomhekBsX/qddz3LGZpFdXsh3kfR50W0DG1qXipujosSM9 UyDqE/nE6RoXEHT9Xx17CPkRMqtuOrLn5TACvHjk4Fv6mOUZ/OYb+TqUx5p4RR6TN3v/ uXSrJIY1agkXPppYMv5NNUqJtzlKhcwcStaUP4CqCBAEyJSpmkBEZakEnzwKn7WbHnra Qc/2K43NYBVo9UB11A51ouFx77cb2w6/AyxTbjqKEpb+9F7LJM4TwQJvBUTSwUrj22B3 eRSIWHnaGBX0cMhgYeZSKc9K8l+SN31F08OLjUsDfwlNEQLoqY60aoHe8VB1/gx4eW1S EBhQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=cs7x7Xbb; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id m13-20020a056402510d00b00424085b80c5si6136875edd.215.2022.04.21.22.42.50; Thu, 21 Apr 2022 22:43:20 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=cs7x7Xbb; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C874568B3CA; Fri, 22 Apr 2022 08:42:46 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7508568B3AB for ; Fri, 22 Apr 2022 08:42:39 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1650606164; x=1682142164; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=knFkuYmPysykYfbfTZ7zwJCGqLvh2lndtz5BfgLqxVU=; b=cs7x7XbbA74dRCoKsV15+44wLEqxP1u7s2e1acYtOFAWfRPsbXv6i+dK hSYq435niR6My4AYxhrYEhmawHk7WT7VD2OefhtGbOMXHIp0zWAehhlXj hr8nWj4lNse0dBcQLktevpwL3UojAyXlORfo2RBo5r/wGEHlkP6xaAc1G +awOm06Xk3cQVIs70DtdDncBjVNidVAwutmq8szkkqm6eOn3WT4XjFi3z WJplntaLCOltnDq6lL78vSc8laDQwSVthPCs2ILdd68bJb7T7XW/u3j8W LZxbSv9HHKqQjtyRrZEH0m9f86UXLqeLXcMg/ZEtekbR7qDPMBJJc9B2o w==; X-IronPort-AV: E=McAfee;i="6400,9594,10324"; a="325030586" X-IronPort-AV: E=Sophos;i="5.90,280,1643702400"; d="scan'208";a="325030586" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Apr 2022 22:42:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,280,1643702400"; d="scan'208";a="556182264" Received: from t.sh.intel.com ([10.239.159.147]) by orsmga007.jf.intel.com with ESMTP; 21 Apr 2022 22:42:36 -0700 From: Fei Wang To: ffmpeg-devel@ffmpeg.org Date: Fri, 22 Apr 2022 13:36:14 +0800 Message-Id: <20220422053615.1638095-1-fei.w.wang@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v5 1/2] lavc/vaapi_encode: add support for maxframesize X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Fei Wang , Linjie Fu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: gJSlUuBEZbH1 From: Linjie Fu Add support for max frame size: - max_frame_size (bytes) to indicate the max allowed size for frame. If the frame size exceeds the limitation, encoder will to control the frame size by adjusting QP value. ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -f rawvideo \ -v verbose -s:v 352x288 -i ./input.yuv -vf format=nv12,hwupload \ -c:v h264_vaapi -profile:v main -g 30 -rc_mode VBR -b:v 500k \ -bf 3 -max_frame_size 40000 -vframes 100 -y ./max_frame_size.h264 Max frame size was enabled since VA-API version (0, 33, 0), but query is available since (1, 5, 0). It will be passed as a parameter in picParam and should be set for each frame. Signed-off-by: Linjie Fu Signed-off-by: Fei Wang --- Change: 1. use VAEncMiscParameterTypeMaxFrameSize to set max frame parameters, which is a more generic way. libavcodec/vaapi_encode.c | 65 +++++++++++++++++++++++++++++++++++++++ libavcodec/vaapi_encode.h | 10 +++++- 2 files changed, 74 insertions(+), 1 deletion(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 0e2f5ed447..c2a39bc4ed 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -365,6 +365,17 @@ static int vaapi_encode_issue(AVCodecContext *avctx, goto fail; } +#if VA_CHECK_VERSION(1, 5, 0) + if (ctx->max_frame_size) { + err = vaapi_encode_make_misc_param_buffer(avctx, pic, + VAEncMiscParameterTypeMaxFrameSize, + &ctx->mfs_params, + sizeof(ctx->mfs_params)); + if (err < 0) + goto fail; + } +#endif + if (pic->type == PICTURE_TYPE_IDR) { if (ctx->va_packed_headers & VA_ENC_PACKED_HEADER_SEQUENCE && ctx->codec->write_sequence_header) { @@ -1869,6 +1880,54 @@ rc_mode_found: return 0; } +static av_cold int vaapi_encode_init_max_frame_size(AVCodecContext *avctx) +{ +#if VA_CHECK_VERSION(1, 5, 0) + VAAPIEncodeContext *ctx = avctx->priv_data; + VAConfigAttrib attr = { VAConfigAttribMaxFrameSize }; + VAStatus vas; + + vas = vaGetConfigAttributes(ctx->hwctx->display, + ctx->va_profile, + ctx->va_entrypoint, + &attr, 1); + if (vas != VA_STATUS_SUCCESS) { + ctx->max_frame_size = 0; + av_log(avctx, AV_LOG_ERROR, "Failed to query max frame size " + "config attribute: %d (%s).\n", vas, vaErrorStr(vas)); + return AVERROR_EXTERNAL; + } + + if (attr.value == VA_ATTRIB_NOT_SUPPORTED) { + ctx->max_frame_size = 0; + av_log(avctx, AV_LOG_WARNING, "Max frame size attribute " + "is not supported.\n"); + } else { + VAConfigAttribValMaxFrameSize attr_mfs; + attr_mfs.value = attr.value; + // Prefer to use VAEncMiscParameterTypeMaxFrameSize for max frame size. + if (!attr_mfs.bits.max_frame_size && attr_mfs.bits.multiple_pass) { + ctx->max_frame_size = 0; + av_log(avctx, AV_LOG_WARNING, "Driver only support multiple pass " + "max frame size which have not been implemented in FFmpeg.\n"); + return 0; + } + + ctx->mfs_params = (VAEncMiscParameterBufferMaxFrameSize){ + .max_frame_size = ctx->max_frame_size * 8, + }; + + av_log(avctx, AV_LOG_VERBOSE, "Max Frame Size: %d bytes.\n", + ctx->max_frame_size); + } +#else + av_log(avctx, AV_LOG_WARNING, "Max Frame Size is not support with " + "this VA version.\n"); +#endif + + return 0; +} + static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) { VAAPIEncodeContext *ctx = avctx->priv_data; @@ -2548,6 +2607,12 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) goto fail; } + if (ctx->va_rc_mode != VA_RC_CQP && ctx->max_frame_size) { + err = vaapi_encode_init_max_frame_size(avctx); + if (err < 0) + goto fail; + } + vas = vaCreateConfig(ctx->hwctx->display, ctx->va_profile, ctx->va_entrypoint, ctx->config_attributes, ctx->nb_config_attributes, diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index af0588c30b..359f954fff 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -191,6 +191,9 @@ typedef struct VAAPIEncodeContext { // Desired B frame reference depth. int desired_b_depth; + // Max Frame Size + int max_frame_size; + // Explicitly set RC mode (otherwise attempt to pick from // available modes). int explicit_rc_mode; @@ -268,6 +271,7 @@ typedef struct VAAPIEncodeContext { VAEncMiscParameterRateControl rc_params; VAEncMiscParameterHRD hrd_params; VAEncMiscParameterFrameRate fr_params; + VAEncMiscParameterBufferMaxFrameSize mfs_params; #if VA_CHECK_VERSION(0, 36, 0) VAEncMiscParameterBufferQualityLevel quality_params; #endif @@ -478,7 +482,11 @@ int ff_vaapi_encode_close(AVCodecContext *avctx); "Increase this to improve single channel performance. This option " \ "doesn't work if driver doesn't implement vaSyncBuffer function.", \ OFFSET(common.async_depth), AV_OPT_TYPE_INT, \ - { .i64 = 2 }, 1, MAX_ASYNC_DEPTH, FLAGS } + { .i64 = 2 }, 1, MAX_ASYNC_DEPTH, FLAGS }, \ + { "max_frame_size", \ + "Maximum frame size (in bytes)",\ + OFFSET(common.max_frame_size), AV_OPT_TYPE_INT, \ + { .i64 = 0 }, 0, INT_MAX, FLAGS } #define VAAPI_ENCODE_RC_MODE(name, desc) \ { #name, desc, 0, AV_OPT_TYPE_CONST, { .i64 = RC_MODE_ ## name }, \ From patchwork Fri Apr 22 05:36:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Fei W" X-Patchwork-Id: 35371 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b9e:b0:7d:cfb5:dc7c with SMTP id b30csp195075pzh; Thu, 21 Apr 2022 22:43:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy0T6RwWPub4jVrrvYYGJhIgCdNYLjqVzGnqnRCeZdTR1uQZrQZW2LBnUCHjURpp0LdjgBk X-Received: by 2002:a17:906:9b8f:b0:6e0:6bcb:fc59 with SMTP id dd15-20020a1709069b8f00b006e06bcbfc59mr2635922ejc.624.1650606180378; Thu, 21 Apr 2022 22:43:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1650606180; cv=none; d=google.com; s=arc-20160816; b=KCv0R0+bUG9QJLDK+3OhlDyY6VRNRiQ9xZ4DROLWsqcSHT26b31+ipbgo+5RJVmT/Z gxGMQyaoH3i9e3GeTXZOjKiJw4BU5fopRLoGvuki4OPxNaZyP7f6bBdiMf5VAplcD9ph dzPHyMAJwyQ1+TJMdMbH3mfWs2jeFviviSryQ32DibxfLxSq7HwenENJFPp4Gx+RiYkN KwKk6nOOLJumqOx1ot8Z7XYMcxbueMvUt0REZp5FhXNaZhSx5o1FoAEj/OUJYLKJrLO6 JpAc2XdEZpJChCkn0J79ExeFrrEBYufyKkYTv9SJpzNbWQpTAoMtbH+byVCy8AXPz0p2 odkw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=OCLX8mYQnLJRMdqQBmvEcm7O5XiCiLgXvJBsPZkF8NM=; b=SCyJcsf/H2dDNlrsWcvEFZdgNWmUBSorOSMCFt99JxV1iwp8/L+C5JoWiI9F4ut5Wf F//Uuy72cdoDfdJLseHpGguVGfZl9i3xdr8eJT3RrIFLsK57MXIQAN5H9hSx+0dxNZA+ xP6vVf2E2gxzBs1GRTcIv85NnQ2sHGspNdZfaSVL1uw0eCplXAKA46rigiPuKqrM6NOi xBf5Y16oxLHJUmNiaxXFD9/nysYUtnuitdsnm05D/uwsbUAGnX+mYa8mxx+tIsel4M7p inxSAFZn++tjkBavaCHiKGGRdeZfF9WjVJJk9XGr61ZoZchDiJZqHniZWJgt4qPpA4tQ vLqw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=YpqU1gUG; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b11-20020a056402084b00b0041d7943941asi5906616edz.238.2022.04.21.22.43.00; Thu, 21 Apr 2022 22:43:00 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=YpqU1gUG; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C2BDB68B3DE; Fri, 22 Apr 2022 08:42:52 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 635D968B3C5 for ; Fri, 22 Apr 2022 08:42:45 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1650606170; x=1682142170; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9bIXngKWuq/yoHAFWnVXi4d4F+WaMwLsRXYwk6pQq9Q=; b=YpqU1gUGNUg8ZjhCKnnCtiywsFLAOo3IRvJwX5dWbcgONNBHyf9wv7qS sdKW+Y8M0ObXijcUNkAQsN33Hy4CAzjEPnB7sCtCXwhLLTZJxCp60k5dq LQqBXKmf++PlueLcY7mfbOpQheG1r5owr7+EMcrjioofS7JRw2iUkad01 308lnV6Lkcd564O8UsxPkkYSHXUPciCwtbRcyayOYma/yrLUoPKYm35CL 0exsg/Us8i/Fs32o+l/mqVZnKSLZXAGswDOJ40sucEVtlwMwlTJnLE+XO f8FADmP1bJCGmiMmgA3Q8cplPPFAEeDwNpIzNf0Q0QZhBIJJVr+Zz59B+ Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10324"; a="325030589" X-IronPort-AV: E=Sophos;i="5.90,280,1643702400"; d="scan'208";a="325030589" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Apr 2022 22:42:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,280,1643702400"; d="scan'208";a="556182274" Received: from t.sh.intel.com ([10.239.159.147]) by orsmga007.jf.intel.com with ESMTP; 21 Apr 2022 22:42:37 -0700 From: Fei Wang To: ffmpeg-devel@ffmpeg.org Date: Fri, 22 Apr 2022 13:36:15 +0800 Message-Id: <20220422053615.1638095-2-fei.w.wang@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220422053615.1638095-1-fei.w.wang@intel.com> References: <20220422053615.1638095-1-fei.w.wang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v5 2/2] doc/vaapi_encode: add documentations for max_frame_size X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Fei Wang , Linjie Fu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: V/bUUdM+/shY From: Linjie Fu Add docs for max_frame_size option. Signed-off-by: Linjie Fu Signed-off-by: Fei Wang --- doc/encoders.texi | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/doc/encoders.texi b/doc/encoders.texi index aac9f25e55..d66713b69f 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -3630,6 +3630,11 @@ performance. This option doesn't work if driver doesn't implement vaSyncBuffer function. Please make sure there are enough hw_frames allocated if a large number of async_depth is used. +@item max_frame_size +Set the allowed max size in bytes for each frame. If the frame size exceeds +the limitation, encoder will adjust the QP value to control the frame size. +Invalid in CQP rate control mode. + @item rc_mode Set the rate control mode to use. A given driver may only support a subset of modes.