From patchwork Mon Jun 25 21:30:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 9507 Delivered-To: ffmpegpatchwork@gmail.com Received: by 2002:a02:141:0:0:0:0:0 with SMTP id c62-v6csp4584642jad; Mon, 25 Jun 2018 14:30:29 -0700 (PDT) X-Google-Smtp-Source: AAOMgpdnWkH5ZH5TCsStmKh60Tq1GsL1yLczBvHDwUFnBYUjT4LsNykCHSMHeidc8iZaSJVOPHHo X-Received: by 2002:adf:b8c7:: with SMTP id c7-v6mr11887934wrg.65.1529962229735; Mon, 25 Jun 2018 14:30:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1529962229; cv=none; d=google.com; s=arc-20160816; b=icRrOWzU75ytUCwpq7AZYOzuk1LImldn2O2N/k1diGUm3C7eFzgylCib8i5xHOxuWv J0BsJC/SGntOHMu7mOgc5157a4N8GYfYOl7LREcqV0iISG8BpwUxPWr5FhfYh7Arl96B 43Gmq4VsKB6xl8wvukans0u98nF0snt3Alc/eLFYciamJqY7krYcA5U0qDlmELNbDIf+ H5lSLgVEcDyZNfItLs65q7jQ9MSSlOT4Wz9N+Wcme+oIEIVAbD2qVpCUMbXXWY/FPY5L Q4Bx0/g7Pr48Ycz8vtQShM/XWr7vH9paKBBZJ/scN+9qr4+kJ0ZvWzrJ+pK5f8b/TfNV Mc5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:dkim-signature :delivered-to:arc-authentication-results; bh=WJzazkTnfqIlBQL7LxAxaiWLgki98oVnlUUIQrxTtvo=; b=jCFHJd64bYEediZ/1rNBHwUEqJLdXABhosGxy3tdu1xAazLOXMJQNDlkwCDjmo5Uck sreMY2rOI4kiE171Q7tzwdT9eXebZ+OaizKJgA9+Cf/V4iKlTWn67YqLFZ+7h+ZxcUkK gSxuCjA6TaPk5nMlnhXARwsFzLhkGN56rAG0SRUNU7nu4lymqxjr4Cb38/4hoBr0yfLI 6/VwsG7WgnLohQtofFWsRMv6pg1jyfh5FTiKY9gdF9owkqhBHMGjAD60z43nL+cC1qDu qVpyr77FImB791Y/kYkP9GTOBN4U0OlZ+MbdYs2dGzKtdpm2vg21EXvmMCmtzTyOK/c5 S9tA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=bmJoiRH2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id o23-v6si187393wmc.93.2018.06.25.14.30.29; Mon, 25 Jun 2018 14:30:29 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=bmJoiRH2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id AF08868A2FE; Tue, 26 Jun 2018 00:29:32 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6E212689CFA for ; Tue, 26 Jun 2018 00:29:31 +0300 (EEST) Received: by mail-wm0-f67.google.com with SMTP id 69-v6so11502592wmf.3 for ; Mon, 25 Jun 2018 14:30:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-language:content-transfer-encoding; bh=+/FENczNUsTImSFc8VrOYuNm1ydbHU5a0M3YjnqKG/8=; b=bmJoiRH2Uvm8cT9OBdlQX/k+mdK1E9r4F/UAiNNqhKE+i9gYh8a994u4eSUJOGybWs +8F9KrqKFDk34dAzw+ccMptpDCKcgxk4LWK0n5Jf0DqIp0d+i1xtzvbcaxuvF8HeexHe wVRZBPIcv2jD11gHlKKQBzZRAcnt87VsuQA0U3ej/O/p1Cp87oCaIVCs7jOf8Z8ogzia e0h4uRhb3sVVKYwTuTE/QPR0Kqc6i+l6dn1PZYEN4fxLziu9LbsJdke0I7zSUeCzBvT4 IWtKBmmzUIC2Eml7jE5A9N8unEXPpBqboyO02FPKhXKWZu2aiq+7NdL+HN2Jvu56PUJd NUjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=+/FENczNUsTImSFc8VrOYuNm1ydbHU5a0M3YjnqKG/8=; b=jhvZJ3mUyYFTOFoQ1NeJB4ZFz5u8qo5pE92/cWUJaT4aDc3c8KbByszRsvLAEa/YDU ikwUcr4QZqHb/tCf8b6Hbycxls7KqKT6LSwdtqDmJrVCLHOPl3wq99kJSfNsngX11+2O eqEIGpyqmqidMHfZs0c1fihVLhLlluyTFnCeXs0j6uMpeZPGp7IbVvtMTTxU1zDEhI0N tiGSN9uRa/1i7mYYxuy8rAVWXUdsm9J/Ymx0Pen+4ibH/oKtQE9/6GbU5dedqdQEZGE5 cY+9CKGyr6rFsjaHcsWzK2+mlcyZQdQOauKHYcfZ1q9QPj3Bn9ubL/2k11u19jrUEon9 xUsQ== X-Gm-Message-State: APt69E1KDMJskWspXBKhl2LM0nn6CoLiDDLft1tZ6+JbOt9vCpv45yvS jx0UEL8VU/UHrGsx+F8FFBPBgQH1 X-Received: by 2002:a1c:64d7:: with SMTP id y206-v6mr2110385wmb.43.1529962226134; Mon, 25 Jun 2018 14:30:26 -0700 (PDT) Received: from ?IPv6:2a00:23c5:418a:8b00:5afb:84ff:fe66:bcaf? ([2a00:23c5:418a:8b00:5afb:84ff:fe66:bcaf]) by smtp.gmail.com with ESMTPSA id 16-v6sm209904wmi.36.2018.06.25.14.30.25 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 25 Jun 2018 14:30:25 -0700 (PDT) To: ffmpeg-devel@ffmpeg.org References: <20180607234331.32139-1-sw@jkqxz.net> <20180607234331.32139-17-sw@jkqxz.net> <1528873411.12353.155.camel@intel.com> <79ebdb48-9fe4-9066-ff95-a930052da675@jkqxz.net> <1528956460.11470.59.camel@intel.com> <7D9B0A5171224A4A9A6308992BC4469B3BDABC4B@shsmsx102.ccr.corp.intel.com> <7D9B0A5171224A4A9A6308992BC4469B3BDAE01E@shsmsx102.ccr.corp.intel.com> <966454ce-630c-0426-57ac-7acdca94fe9b@jkqxz.net> <20180621170319.GQ4859@michaelspb> From: Mark Thompson Message-ID: <5540c08e-fd41-0138-fea0-c7a33da66621@jkqxz.net> Date: Mon, 25 Jun 2018 22:30:24 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <20180621170319.GQ4859@michaelspb> Content-Language: en-US Subject: Re: [FFmpeg-devel] [PATCH v2 16/36] vaapi_encode: Clean up rate control configuration X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" On 21/06/18 18:03, Michael Niedermayer wrote: > On Thu, Jun 21, 2018 at 12:10:04AM +0100, Mark Thompson wrote: >> On 20/06/18 10:44, Li, Zhong wrote: >>>> -----Original Message----- >>>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf >>>> Of Mark Thompson >>>> Sent: Sunday, June 17, 2018 9:51 PM >>>> To: ffmpeg-devel@ffmpeg.org >>>> Subject: Re: [FFmpeg-devel] [PATCH v2 16/36] vaapi_encode: Clean up rate >>>> control configuration >>>> >>>> On 14/06/18 08:22, Li, Zhong wrote: >>>>>> -----Original Message----- >>>>>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On >>>> Behalf >>>>>> Of Xiang, Haihao >>>>>> Sent: Thursday, June 14, 2018 2:08 PM >>>>>> To: ffmpeg-devel@ffmpeg.org >>>>>> Subject: Re: [FFmpeg-devel] [PATCH v2 16/36] vaapi_encode: Clean up >>>>>> rate control configuration >>>>>> >>>>>> On Wed, 2018-06-13 at 23:42 +0100, Mark Thompson wrote: >>>>>>> On 13/06/18 08:03, Xiang, Haihao wrote: >>>>>>>> On Fri, 2018-06-08 at 00:43 +0100, Mark Thompson wrote: >>>>>>>>> Query which modes are supported and select between VBR and CBR >>>>>>>>> based on that - this removes all of the codec-specific rate >>>>>>>>> control mode selection code. >>>>>>>>> --- >>>>>>>>> doc/encoders.texi | 2 - >>>>>>>>> libavcodec/vaapi_encode.c | 173 >>>>>> ++++++++++++++++++++++++++++------- >>>>>>>>> ---- >>>>>>>>> - >>>>>>>>> libavcodec/vaapi_encode.h | 6 +- >>>>>>>>> libavcodec/vaapi_encode_h264.c | 18 +---- >>>>>>>>> libavcodec/vaapi_encode_h265.c | 14 +--- >>>>>>>>> libavcodec/vaapi_encode_mjpeg.c | 3 +- >>>>>>>>> libavcodec/vaapi_encode_mpeg2.c | 9 +-- >>>>>>>>> libavcodec/vaapi_encode_vp8.c | 13 +-- >>>>>>>>> libavcodec/vaapi_encode_vp9.c | 13 +-- >>>>>>>>> 9 files changed, 137 insertions(+), 114 deletions(-) >>>>>>>>> >>>>>>>>> ... >>>>>>>>> diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c >>>>>>>>> index f4c063734c..5de5483454 100644 >>>>>>>>> --- a/libavcodec/vaapi_encode.c >>>>>>>>> +++ b/libavcodec/vaapi_encode.c >>>>>>>>> ... >>>>>>>>> + if (avctx->flags & AV_CODEC_FLAG_QSCALE || >>>>>>>>> + avctx->bit_rate <= 0) { >>>> >>>> This condition ^ >>>> >>>>>>>>> + if (rc_attr.value & VA_RC_CQP) { >>>>>>>>> + av_log(avctx, AV_LOG_VERBOSE, "Using >>>>>> constant-quality >>>>>>>>> mode.\n"); >>>>>>>>> + ctx->va_rc_mode = VA_RC_CQP; >>>>>>>>> + return 0; >>>>>>>>> + } else { >>>>>>>>> + av_log(avctx, AV_LOG_ERROR, "Driver does not >>>>>> support " >>>>>>>>> + "constant-quality mode (%#x).\n", >>>>>> rc_attr.value); >>>>>>>>> + return AVERROR(EINVAL); >>>>>>>>> + } >>>>>>>>> + } >>>>>>>>> ... >>>>>>>>> + } else if (avctx->rc_max_rate == avctx->bit_rate) { >>>>>>>>> + if (!(rc_attr.value & VA_RC_CBR)) { >>>>>>>>> + av_log(avctx, AV_LOG_WARNING, "Driver does not >>>>>> support " >>>>>>>>> + "CBR mode (%#x), using VBR mode >>>>>> instead.\n", >>>>>>>>> + rc_attr.value); >>>>>>>>> + ctx->va_rc_mode = VA_RC_VBR; >>>>>>>>> + } else { >>>>>>>>> + ctx->va_rc_mode = VA_RC_CBR; >>>>>>>>> + } >>>>>>>>> >>>>>>>>> - if (ctx->va_rc_mode == VA_RC_CBR) { >>>>>>>>> rc_bits_per_second = avctx->bit_rate; >>>>>>>>> rc_target_percentage = 100; >>>>>>>>> - rc_window_size = 1000; >>>>>>>>> + >>>>>>>>> } else { >>>>>>>>> - if (avctx->rc_max_rate < avctx->bit_rate) { >>>>>>>>> - // Max rate is unset or invalid, just use the normal >>>>>> bitrate. >>>>>>>>> + if (rc_attr.value & VA_RC_VBR) { >>>>>>>>> + ctx->va_rc_mode = VA_RC_VBR; >>>>>>>> >>>>>>>> Is it better to take it as CBR when avctx->rc_max_rate is 0 and CBR >>>>>>>> is supported by driver? >>>>>>> >>>>>>> I don't think so? VBR with the specified target is probably what >>>>>>> you want in most cases, and I think anyone with specific constraints >>>>>>> that want constant bitrate should expect to set maxrate to achieve that. >>>>>>> >>>>>> >>>>>> I agree VBR is probably what an user wants in most case, however >>>>>> target percent set to 50% is not suitable for most case. To get a >>>>>> specific target percent, user should set both target bitrate and max >>>>>> bitrate, so it is reasonable to ask user must set both target bitrate >>>>>> and max bitrate for VBR cases, and for CBR user may set target bitrate >>>> only. >>>>> >>>>> How about set the max_rate to be a very larger number such as INT_MAX >>>> if user hasn't set it? >>>>> User may don't set max_rate on purpose, expecting better quality with >>>> unlimited bitrate fluctuation (common requirement for local video files). >>>>> Double of target_bit_rate is too strict IMHO. And I haven't such a >>>> limitation in x264 ABR mode. >>>> >>>> This unconstrained setup you describe was my intent (as you say, it's usually >>>> what you want for local files), but unfortunately the API doesn't really let us >>>> do it - the target bitrate is specified as an integer percentage of the max >>>> bitrate. 50% was an arbitrary value picked to not have a strong constraint >>>> but also not be small enough that we need to think about rounding/overflow >>>> problems. >>>> >>>> We could try to pick a large value with the right properties (for example: if >>>> target < 2^32 / 100 then choose maxrate = target * 100, percentage = 1; >>>> otherwise choose percentage = 2^32 * 100 / bitrate, maxrate = bitrate * 100 >>>> / percentage), but that would be complex to test and I don't think the drivers >>>> handle this sort of setup very well. >>>> >>>>>> I just saw it is also VBR in QSV when max bitrate is not set so I'm >>>>>> fine to keep consistency with QSV for this case. >>>>> >>>>> What will happen if user set a max_rate but without a setting for >>>> target_bitrate? >>>>> The mode will be VBR (if driver support) with target_bitrate=0. >>>>> Tried this on qsv, MSDK returns an error of invalid video parameters. >>>>> Is it ok for vaapi? And also with iHD driver? >>>> >>>> If AVCodecContext.bit_rate isn't set then we use constant-quality mode >>>> instead - see the block I've pointed out above. >>>> >>>> There isn't currently any constant-quality with max-rate constraint mode in >>>> VAAPI. >>> >>> Then the problem I see is that -max_rate hasn't been respected well if user set it (he may don't care about the target bitrate except the peak value). >>> Maybe we can add a warning at least? >> >> Given that it's really CQP, I don't think anyone would ever expect this to work? Encoders generally don't warn about ignoring extra irrelevant options in AVCodecContext. >> >> (Is there any encoder at all which supports that combination? E.g. libx264 supports maxrate in CRF but not CQP mode.) > > if i understand correctly, then yes, see vbv_ignore_qmax. If max rate > cannot be achievied the mpegvideo encoders should attempt to encode the frame > again without qmax and at lower quality Ok, fair enough. I've added a warning as below so that it's clear this case isn't supported. Thanks, - Mark diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 53c3a2132a..25d89c65c9 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -1311,6 +1311,10 @@ static av_cold int vaapi_encode_init_rate_control(AVCodecContext *avctx) if (rc_attr.value & VA_RC_CQP) { av_log(avctx, AV_LOG_VERBOSE, "Using constant-quality mode.\n"); ctx->va_rc_mode = VA_RC_CQP; + if (avctx->bit_rate > 0 || avctx->rc_max_rate > 0) { + av_log(avctx, AV_LOG_WARNING, "Bitrate target parameters " + "ignored in constant-quality mode.\n"); + } return 0; } else { av_log(avctx, AV_LOG_ERROR, "Driver does not support "