From patchwork Sun Feb 10 19:47:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 12022 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 23BB744665E for ; Sun, 10 Feb 2019 21:47:24 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1367068A8AC; Sun, 10 Feb 2019 21:47:24 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f67.google.com (mail-wm1-f67.google.com [209.85.128.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 88F9D68A2B8 for ; Sun, 10 Feb 2019 21:47:21 +0200 (EET) Received: by mail-wm1-f67.google.com with SMTP id x10so7577737wmg.2 for ; Sun, 10 Feb 2019 11:47:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=4X5YJ+ESROIe59JLFzPefh/TaGld7zYmV8/qVyPq77I=; b=AcD6smDo+7e1Tq/NKwcKOs55YubZUpCSYFA+B8SGrAThCiwIPj9l2uGsuR7Yxr55V1 Bm6C70yo8gLBTzPzZ1nIdXBWM/vB+awtJlFIcGrxxNsyLAj9XikKGpyscTNIyny0igJ6 f2j0L5EwYnYp8LIS6HRNcGsJNQXzwPno4Ty7kFjcAWdlNtdNiNgag9SYnwVz1eK/K9k7 uaCSpwZwmu3pYEHgO9o05tO0za7+lqjskt7iLakvBjIKfs6twO/Ye1iyIYkXd0J4QrlK J2Et2UI1MEKvAphg+3KbQlV8K21fFOA/gZsCWZ36TfaNDOYE49a2ECkMmtccDsiDsFbu odZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4X5YJ+ESROIe59JLFzPefh/TaGld7zYmV8/qVyPq77I=; b=SHCkj8O0wsGJfViXG+zkkW+lSmUn6Kfgfn8cwpJjvCvgyj5950/vOp9Wj+1Vs7dYFt CCOI9QXrx25BqKqnNvC7js+7BEgYnTVOqAsWwLdOvyjrxseBDRJA4yosm4rfhasHEM3q EHIA/bcd/V+cwXBXey32/VNkzwbU2zpPCdtVOFDi9M0R/QkPMYvB4QnICdb+vaytLRka v1KGNAS7Npp/jtTeIUIw97QixBfn2mryLy4jMNJisMAIRg7lKId8afcQVNZ00kOK7b+v 3bmZmtJ+V21K2e1/y3HIpiH4HgQI4/PCpb8Ge9t2xadUneRanauCQVk1rAdgD4L+voYr NN4w== X-Gm-Message-State: AHQUAuZnRYcPx7i64gmJQe9t0JBXcPmI7bwFt2/C13qmRSVgxCLIoB0Z Pw2M7sFIRQ6NqDtU5ptpX/Fbfvb9/a5L6w== X-Google-Smtp-Source: AHgI3IZhSUFu6XHv1kJhuYaRK7StkNARp5jI83jERd8Yl7VPzv7C2JgaULA9adBVBJWWyMSPRre9ug== X-Received: by 2002:a1c:f207:: with SMTP id s7mr6489678wmc.87.1549828040629; Sun, 10 Feb 2019 11:47:20 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id u15sm24059889wrm.63.2019.02.10.11.47.19 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 10 Feb 2019 11:47:19 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 10 Feb 2019 19:47:03 +0000 Message-Id: <20190210194705.3598-10-sw@jkqxz.net> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20190210194705.3598-1-sw@jkqxz.net> References: <20190210194705.3598-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 10/12] vaapi_encode: Add support for VFR mode X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Use the frame-skip feature to maintain a specified framerate from the point of view of the driver. --- "max_fps" collides with an option used in lavf, so renamed to "vfr_max_fps" here. (That hopefully also makes it clearer what the use of the option is.) doc/encoders.texi | 7 +++ libavcodec/vaapi_encode.c | 116 +++++++++++++++++++++++++++++++++++--- libavcodec/vaapi_encode.h | 18 +++++- 3 files changed, 132 insertions(+), 9 deletions(-) diff --git a/doc/encoders.texi b/doc/encoders.texi index 29625ba07c..39b8fc1d37 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -2848,6 +2848,13 @@ Quality-defined variable-bitrate. Average variable bitrate. @end table +@item vfr_max_fps +Enable VFR encoding with this maximum framerate. This is implemented by +indicating to the driver that frames have been skipped at some locations in +the stream, and requires driver support. The minimum interval between frames +must not be smaller than this, and there may be problems if the maximum +interval is more than a small multiple of it. + @end table Each encoder also has its own specific options: diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 2dda451882..df8f65e431 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -412,6 +412,29 @@ static int vaapi_encode_issue(AVCodecContext *avctx, } } +#if VA_CHECK_VERSION(0, 40, 0) + if (ctx->vfr_mode && pic->frame_skips > 0) { + struct { + VAEncMiscParameterBuffer misc; + VAEncMiscParameterSkipFrame skip; + } param = { + .misc = { + .type = VAEncMiscParameterTypeSkipFrame, + }, + .skip = { + .skip_frame_flag = 1, + .num_skip_frames = pic->frame_skips, + }, + }; + + err = vaapi_encode_make_param_buffer(avctx, pic, + VAEncMiscParameterBufferType, + (char*)¶m, sizeof(param)); + if (err < 0) + goto fail; + } +#endif + vas = vaBeginPicture(ctx->hwctx->display, ctx->va_context, pic->input_surface); if (vas != VA_STATUS_SUCCESS) { @@ -942,6 +965,36 @@ int ff_vaapi_encode_send_frame(AVCodecContext *avctx, const AVFrame *frame) pic->input_surface = (VASurfaceID)(uintptr_t)frame->data[3]; pic->pts = frame->pts; + if (ctx->vfr_mode && ctx->input_order > 0) { + if (frame->pts < ctx->prev_pts) { + av_log(avctx, AV_LOG_WARNING, "Timestamp discontinuity " + "(backward step: %"PRId64" -> %"PRId64"): " + "VFR mode reset.\n", ctx->prev_pts, frame->pts); + ctx->ticks_outstanding = av_make_q(0, 1); + } else { + AVRational step_ticks, ticks; + int ticks_int; + step_ticks = av_div_q(av_make_q(frame->pts - ctx->prev_pts, 1), + ctx->ticks_per_frame); + ticks = av_add_q(ctx->ticks_outstanding, step_ticks); + ticks_int = ticks.num / ticks.den; + if (ticks_int < 1) { + av_log(avctx, AV_LOG_WARNING, "Max FPS exceeded!\n"); + } else if (ticks_int > 256) { + av_log(avctx, AV_LOG_WARNING, "Timestamp discontinuity " + "(forward step: %"PRId64" -> %"PRId64"): " + "VFR mode reset.\n", ctx->prev_pts, frame->pts); + } else { + av_log(avctx, AV_LOG_DEBUG, "Inserting %d frame skips before " + "frame %"PRId64".\n", ticks_int - 1, frame->pts); + pic->frame_skips = ticks_int - 1; + } + ctx->ticks_outstanding = + av_sub_q(ticks, av_make_q(ticks_int, 1)); + } + } + ctx->prev_pts = frame->pts; + if (ctx->input_order == 0) ctx->first_pts = pic->pts; if (ctx->input_order == ctx->decode_delay) @@ -1315,7 +1368,6 @@ static av_cold int vaapi_encode_init_rate_control(AVCodecContext *avctx) int rc_quality; int64_t hrd_buffer_size; int64_t hrd_initial_buffer_fullness; - int fr_num, fr_den; VAConfigAttrib rc_attr = { VAConfigAttribRateControl }; VAStatus vas; char supported_rc_modes_string[64]; @@ -1609,22 +1661,66 @@ rc_mode_found: sizeof(ctx->hrd_params)); } - if (avctx->framerate.num > 0 && avctx->framerate.den > 0) - av_reduce(&fr_num, &fr_den, - avctx->framerate.num, avctx->framerate.den, 65535); - else + return 0; +} + +static av_cold int vaapi_encode_init_framerate(AVCodecContext *avctx) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + +#if VA_CHECK_VERSION(0, 40, 0) + int fr_num, fr_den; + + ctx->vfr_mode = ctx->vfr_max_fps.num > 0 && ctx->vfr_max_fps.den > 0; + if (ctx->vfr_mode) { + VAConfigAttrib attr = { VAConfigAttribEncSkipFrame }; + VAStatus vas; + + vas = vaGetConfigAttributes(ctx->hwctx->display, + ctx->va_profile, ctx->va_entrypoint, + &attr, 1); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to query skip-frame " + "config attribute: %d (%s).\n", vas, vaErrorStr(vas)); + return AVERROR_EXTERNAL; + } + if (attr.value == VA_ATTRIB_NOT_SUPPORTED || + attr.value == 0) { + av_log(avctx, AV_LOG_ERROR, "Skip-frame attribute is not " + "supported by driver: VFR mode cannot be used.\n"); + return AVERROR(EINVAL); + } + av_reduce(&fr_num, &fr_den, - avctx->time_base.den, avctx->time_base.num, 65535); + ctx->vfr_max_fps.num, ctx->vfr_max_fps.den, 65535); + + ctx->ticks_per_frame = av_inv_q(av_mul_q(avctx->time_base, + av_make_q(fr_num, fr_den))); + ctx->ticks_outstanding = av_make_q(0, 1); + } else { + if (avctx->framerate.num > 0 && avctx->framerate.den > 0) + av_reduce(&fr_num, &fr_den, + avctx->framerate.num, avctx->framerate.den, 65535); + else + av_reduce(&fr_num, &fr_den, + avctx->time_base.den, avctx->time_base.num, 65535); + } - av_log(avctx, AV_LOG_VERBOSE, "RC framerate: %d/%d (%.2f fps).\n", + av_log(avctx, AV_LOG_VERBOSE, "RC framerate (%s mode): " + "%d/%d (%.2f fps).\n", ctx->vfr_mode ? "VFR" : "CFR", fr_num, fr_den, (double)fr_num / fr_den); ctx->fr_params.misc.type = VAEncMiscParameterTypeFrameRate; ctx->fr_params.fr.framerate = (unsigned int)fr_den << 16 | fr_num; -#if VA_CHECK_VERSION(0, 40, 0) vaapi_encode_add_global_param(avctx, &ctx->fr_params.misc, sizeof(ctx->fr_params)); + +#else + if (ctx->vfr_max_fps.num > 0 && ctx->vfr_max_fps.den > 0) { + av_log(avctx, AV_LOG_WARNING, "Variable framerate is " + "not supported with this VAAPI version.\n"); + } #endif return 0; @@ -2077,6 +2173,10 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) if (err < 0) goto fail; + err = vaapi_encode_init_framerate(avctx); + if (err < 0) + goto fail; + err = vaapi_encode_init_gop_structure(avctx); if (err < 0) goto fail; diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index 44a8db566e..860b67dbe2 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -69,6 +69,9 @@ typedef struct VAAPIEncodePicture { int64_t pts; int force_idr; + // Number of frame-skips to insert before this frame in VFR mode. + int frame_skips; + int type; int b_depth; int encode_issued; @@ -184,6 +187,10 @@ typedef struct VAAPIEncodeContext { // (Forces CQP mode when set, overriding everything else.) int explicit_qp; + // When set, enable skip-frame VFR mode with this maximum + // framerate. + AVRational vfr_max_fps; + // Desired packed headers. unsigned int desired_packed_headers; @@ -297,6 +304,12 @@ typedef struct VAAPIEncodeContext { int64_t dts_pts_diff; int64_t ts_ring[MAX_REORDER_DELAY * 3]; + // VFR state. + int vfr_mode; + AVRational ticks_per_frame; + AVRational ticks_outstanding; + int64_t prev_pts; + // Slice structure. int slice_block_rows; int slice_block_cols; @@ -445,7 +458,10 @@ int ff_vaapi_encode_close(AVCodecContext *avctx); VAAPI_ENCODE_RC_MODE(VBR, "Variable-bitrate"), \ VAAPI_ENCODE_RC_MODE(ICQ, "Intelligent constant-quality"), \ VAAPI_ENCODE_RC_MODE(QVBR, "Quality-defined variable-bitrate"), \ - VAAPI_ENCODE_RC_MODE(AVBR, "Average variable-bitrate") + VAAPI_ENCODE_RC_MODE(AVBR, "Average variable-bitrate"), \ + { "vfr_max_fps", "Enable VFR mode with this maximum framerate", \ + OFFSET(common.vfr_max_fps), AV_OPT_TYPE_RATIONAL, \ + { .dbl = 0.0 }, 0, INT_MAX, FLAGS } #endif /* AVCODEC_VAAPI_ENCODE_H */