From patchwork Thu Dec 20 20:40:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 11503 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 0C52544CDFA for ; Thu, 20 Dec 2018 22:48:53 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3800668AE79; Thu, 20 Dec 2018 22:48:53 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f68.google.com (mail-wm1-f68.google.com [209.85.128.68]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 88B5168AE41 for ; Thu, 20 Dec 2018 22:48:47 +0200 (EET) Received: by mail-wm1-f68.google.com with SMTP id f81so3647373wmd.4 for ; Thu, 20 Dec 2018 12:48:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=GZPCuSwl16oKaDZS0zWvcG8MdRnLBhOo/O0LAIXORpc=; b=acOJtBwKAeTdYWvAtlccq/dvwgqUbdwn/jLA7HnN5qUFdISjiHy1mcUWD0RdZqPxwt MgzQjUWdrSaoKbvWKoLusfwtN1z1pwumJ8sla41Zsik38wGbW3z9kKDb3GR00I0HzJqC uZSz+ZVrUt6YphCqwCaq2L9psZHB/DNMKgc0ZmHfAa2ufqjTa46XQQ/NUfZ4CcORyWI8 D+MwdEo0pI9UhuP+SOWw8SQ+DwGj6Zo7+ZtmssEE5DFSopZk4rXf/cHcc0XUASOh1CQ7 Jx3Y73m+1i1Niy4Pm+sfnUajyDA4x4cYQqJZEYTaL1Z8iUMYjWlwXmRSWY4pm+2B6XJr 3mZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GZPCuSwl16oKaDZS0zWvcG8MdRnLBhOo/O0LAIXORpc=; b=jAJ4P0doWkSIxTAo2AnKjBVOnfjcHDyqaN4fXZIi5al1ktIqf8crVv2yHfsC98BJ43 haRlB0y+/bt01Lxm8G1M+2lgG33wg6YybBDl3dTO5rCFotsmHB/43j/Gnr2JtViJ/mXJ SL+IDXJ1IitD0vb+HIVYU5P8CfFkIRyTe+dIyKh/RU97SEgHDGHkLcOeUHXrQo7HmHg1 D/T39zwO4ECPT9gGCOctQPMqO80ySMVuBAJxJ/WJEjmCvmw3XwIluY4aaYfrybjswOGr m0NG4m3fhffA4MiUsbTgs+CVgCxgTw+VWk1n7YLiX5dblbDt0HoxMnAWYoBwbUAChjw7 m5IA== X-Gm-Message-State: AJcUukeY1iAGa6oUcCzPxjvuAnS2piU2bnPBJt2vyJQlfijbXEUg7s98 HSmNBLV4YOl0KsAICRge7Ww/ksWmMV0= X-Google-Smtp-Source: ALg8bN5YITlBp080N3fAGTDNIFh9yvA3P8W4sb6nhn50EWt07HftaMHKleJ7+ZcCK4GtGVn4bviiKw== X-Received: by 2002:a1c:1a43:: with SMTP id a64mr140514wma.136.1545338428394; Thu, 20 Dec 2018 12:40:28 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id x10sm11956884wrn.29.2018.12.20.12.40.27 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Dec 2018 12:40:27 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Thu, 20 Dec 2018 20:40:08 +0000 Message-Id: <20181220204008.32766-14-sw@jkqxz.net> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20181220204008.32766-1-sw@jkqxz.net> References: <20181220204008.32766-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 14/14] vaapi_encode: Add support for VFR mode X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Use the frame-skip feature to maintain a specified framerate from the point of view of the driver. --- Not finished - mostly a POC for using timestamps rather than the suggestion of adding a new side-data type to trigger the VFR frame-skip feature. Kindof works, with the iHD driver only - triggers skips in the right place, though I'm not sure it's making the right thing happen in the driver because the output bitrate is somewhere in between the no-skip and the expected values. Not sure what's going on there, and more generally additional thought is needed on what's actually wanted here. libavcodec/vaapi_encode.c | 116 +++++++++++++++++++++++++++++++++++--- libavcodec/vaapi_encode.h | 18 +++++- 2 files changed, 125 insertions(+), 9 deletions(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 282faf041f..65419b34cd 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -412,6 +412,29 @@ static int vaapi_encode_issue(AVCodecContext *avctx, } } +#if VA_CHECK_VERSION(0, 40, 0) + if (ctx->vfr_mode && pic->frame_skips > 0) { + struct { + VAEncMiscParameterBuffer misc; + VAEncMiscParameterSkipFrame skip; + } param = { + .misc = { + .type = VAEncMiscParameterTypeSkipFrame, + }, + .skip = { + .skip_frame_flag = 1, + .num_skip_frames = pic->frame_skips, + }, + }; + + err = vaapi_encode_make_param_buffer(avctx, pic, + VAEncMiscParameterBufferType, + (char*)¶m, sizeof(¶m)); + if (err < 0) + goto fail; + } +#endif + vas = vaBeginPicture(ctx->hwctx->display, ctx->va_context, pic->input_surface); if (vas != VA_STATUS_SUCCESS) { @@ -942,6 +965,36 @@ int ff_vaapi_encode_send_frame(AVCodecContext *avctx, const AVFrame *frame) pic->input_surface = (VASurfaceID)(uintptr_t)frame->data[3]; pic->pts = frame->pts; + if (ctx->vfr_mode && ctx->input_order > 0) { + if (frame->pts < ctx->prev_pts) { + av_log(avctx, AV_LOG_WARNING, "Timestamp discontinuity " + "(backward step: %"PRId64" -> %"PRId64"): " + "VFR mode reset.\n", ctx->prev_pts, frame->pts); + ctx->ticks_outstanding = av_make_q(0, 1); + } else { + AVRational step_ticks, ticks; + int ticks_int; + step_ticks = av_div_q(av_make_q(frame->pts - ctx->prev_pts, 1), + ctx->ticks_per_frame); + ticks = av_add_q(ctx->ticks_outstanding, step_ticks); + ticks_int = ticks.num / ticks.den; + if (ticks_int < 1) { + av_log(avctx, AV_LOG_WARNING, "Max FPS exceeded!\n"); + } else if (ticks_int > 256) { + av_log(avctx, AV_LOG_WARNING, "Timestamp discontinuity " + "(forward step: %"PRId64" -> %"PRId64"): " + "VFR mode reset.\n", ctx->prev_pts, frame->pts); + } else { + av_log(avctx, AV_LOG_DEBUG, "Inserting %d frame skips before " + "frame %"PRId64".\n", ticks_int - 1, frame->pts); + pic->frame_skips = ticks_int - 1; + } + ctx->ticks_outstanding = + av_sub_q(ticks, av_make_q(ticks_int, 1)); + } + } + ctx->prev_pts = frame->pts; + if (ctx->input_order == 0) ctx->first_pts = pic->pts; if (ctx->input_order == ctx->decode_delay) @@ -1315,7 +1368,6 @@ static av_cold int vaapi_encode_init_rate_control(AVCodecContext *avctx) int rc_quality; int64_t hrd_buffer_size; int64_t hrd_initial_buffer_fullness; - int fr_num, fr_den; VAConfigAttrib rc_attr = { VAConfigAttribRateControl }; VAStatus vas; char supported_rc_modes_string[64]; @@ -1572,22 +1624,66 @@ rc_mode_found: sizeof(ctx->hrd_params)); } - if (avctx->framerate.num > 0 && avctx->framerate.den > 0) - av_reduce(&fr_num, &fr_den, - avctx->framerate.num, avctx->framerate.den, 65535); - else + return 0; +} + +static av_cold int vaapi_encode_init_framerate(AVCodecContext *avctx) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + +#if VA_CHECK_VERSION(0, 40, 0) + int fr_num, fr_den; + + ctx->vfr_mode = ctx->vfr_max_fps.num > 0 && ctx->vfr_max_fps.den > 0; + if (ctx->vfr_mode) { + VAConfigAttrib attr = { VAConfigAttribEncSkipFrame }; + VAStatus vas; + + vas = vaGetConfigAttributes(ctx->hwctx->display, + ctx->va_profile, ctx->va_entrypoint, + &attr, 1); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to query skip-frame " + "config attribute: %d (%s).\n", vas, vaErrorStr(vas)); + return AVERROR_EXTERNAL; + } + if (attr.value == VA_ATTRIB_NOT_SUPPORTED || + attr.value == 0) { + av_log(avctx, AV_LOG_ERROR, "Skip-frame attribute is not " + "supported by driver: VFR mode cannot be used.\n"); + return AVERROR(EINVAL); + } + av_reduce(&fr_num, &fr_den, - avctx->time_base.den, avctx->time_base.num, 65535); + ctx->vfr_max_fps.num, ctx->vfr_max_fps.den, 65535); + + ctx->ticks_per_frame = av_inv_q(av_mul_q(avctx->time_base, + av_make_q(fr_num, fr_den))); + ctx->ticks_outstanding = av_make_q(0, 1); + } else { + if (avctx->framerate.num > 0 && avctx->framerate.den > 0) + av_reduce(&fr_num, &fr_den, + avctx->framerate.num, avctx->framerate.den, 65535); + else + av_reduce(&fr_num, &fr_den, + avctx->time_base.den, avctx->time_base.num, 65535); + } - av_log(avctx, AV_LOG_VERBOSE, "RC framerate: %d/%d (%.2f fps).\n", + av_log(avctx, AV_LOG_VERBOSE, "RC framerate (%s mode): " + "%d/%d (%.2f fps).\n", ctx->vfr_mode ? "VFR" : "CFR", fr_num, fr_den, (double)fr_num / fr_den); ctx->fr_params.misc.type = VAEncMiscParameterTypeFrameRate; ctx->fr_params.fr.framerate = (unsigned int)fr_den << 16 | fr_num; -#if VA_CHECK_VERSION(0, 40, 0) vaapi_encode_add_global_param(avctx, &ctx->fr_params.misc, sizeof(ctx->fr_params)); + +#else + if (ctx->max_fps.num > 0 && ctx->max_fps.den > 0) { + av_log(avctx, AV_LOG_WARNING, "Variable framerate is " + "not supported with this VAAPI version.\n"); + } #endif return 0; @@ -2040,6 +2136,10 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) if (err < 0) goto fail; + err = vaapi_encode_init_framerate(avctx); + if (err < 0) + goto fail; + err = vaapi_encode_init_gop_structure(avctx); if (err < 0) goto fail; diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index 123a46efbb..aae62b386f 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -70,6 +70,9 @@ typedef struct VAAPIEncodePicture { int64_t pts; int force_idr; + // Number of frame-skips to insert before this frame in VFR mode. + int frame_skips; + int type; int b_depth; int encode_issued; @@ -183,6 +186,10 @@ typedef struct VAAPIEncodeContext { // (Forces CQP mode when set, overriding everything else.) int explicit_qp; + // When set, enable skip-frame VFR mode with this maximum + // framerate. + AVRational vfr_max_fps; + // Desired packed headers. unsigned int desired_packed_headers; @@ -296,6 +303,12 @@ typedef struct VAAPIEncodeContext { int64_t dts_pts_diff; int64_t ts_ring[MAX_REORDER_DELAY * 3]; + // VFR state. + int vfr_mode; + AVRational ticks_per_frame; + AVRational ticks_outstanding; + int64_t prev_pts; + // Slice structure. int slice_block_rows; int slice_block_cols; @@ -444,7 +457,10 @@ int ff_vaapi_encode_close(AVCodecContext *avctx); VAAPI_ENCODE_RC_MODE(VBR, "Variable-bitrate"), \ VAAPI_ENCODE_RC_MODE(ICQ, "Intelligent constant-quality"), \ VAAPI_ENCODE_RC_MODE(QVBR, "Quality-defined variable-bitrate"), \ - VAAPI_ENCODE_RC_MODE(AVBR, "Average variable-bitrate") + VAAPI_ENCODE_RC_MODE(AVBR, "Average variable-bitrate"), \ + { "max_fps", "Enable VFR mode with this maximum framerate", \ + OFFSET(common.vfr_max_fps), AV_OPT_TYPE_RATIONAL, \ + { .dbl = 0.0 }, 0, INT_MAX, FLAGS } #endif /* AVCODEC_VAAPI_ENCODE_H */