From patchwork Mon Jun 1 13:49:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Fu, Linjie" X-Patchwork-Id: 20056 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 5F453449A40 for ; Mon, 1 Jun 2020 16:53:42 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3AE73689AB3; Mon, 1 Jun 2020 16:53:42 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C3527680CF2 for ; Mon, 1 Jun 2020 16:53:35 +0300 (EEST) IronPort-SDR: Q0Twwl4S5l8QGPkCvEaXZFiJklXnnreNvGHvDEBYeXiRfXA5o7V5BALnucTFno5c2l/dpAEH4n SuM7b8KgFIsA== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jun 2020 06:53:33 -0700 IronPort-SDR: wtpZwiY32r3RxBmQrauS8qYDYAQVPoO0buSQs5hxDovVK3/VAPclSFDM0jqEEUuTU0+PcpTcUg +i15Qo5XZhTw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,461,1583222400"; d="scan'208";a="347043876" Received: from icl-dev.sh.intel.com ([10.239.158.73]) by orsmga001.jf.intel.com with ESMTP; 01 Jun 2020 06:53:29 -0700 From: Linjie Fu To: ffmpeg-devel@ffmpeg.org Date: Mon, 1 Jun 2020 21:49:44 +0800 Message-Id: <1591019384-21910-1-git-send-email-linjie.fu@intel.com> X-Mailer: git-send-email 2.7.4 Subject: [FFmpeg-devel] [PATCH] lavc/vaapi_encode: Convert P to B-frames if driver doesn't support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Linjie Fu MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Automatically convert P-frames to B-frames according to the query results of VAConfigAttribPredictionDirection. If P-frames is not supported by driver, convert P-frames to B-frames with 2 same ref_lists(forward-prediction only, low delay B-frames). If query is not supported with current VAAPI version, convert P to B by default for low power mode to avoid unexpected errors. Signed-off-by: Linjie Fu --- Since the query support is ready in LIBVA and media-driver, this patch is now ready for upstream: https://github.com/intel/libva/commit/0014ada0eb0fd09a0dd928e5a132ecf12ed85c2e https://github.com/intel/media-driver/commit/d7061b608ad4d55fccab00f8d80b852f20eab1d7 libavcodec/vaapi_encode.c | 61 +++++++++++++++++++++++++++++++++++------- libavcodec/vaapi_encode.h | 2 ++ libavcodec/vaapi_encode_h265.c | 7 +++++ 3 files changed, 60 insertions(+), 10 deletions(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index cb05ebd..0710be7 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -1768,26 +1768,57 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) { VAAPIEncodeContext *ctx = avctx->priv_data; VAStatus vas; - VAConfigAttrib attr = { VAConfigAttribEncMaxRefFrames }; - uint32_t ref_l0, ref_l1; + VAConfigAttrib attr[2] = { { VAConfigAttribEncMaxRefFrames }, +#if VA_CHECK_VERSION(1, 8, 0) + { VAConfigAttribPredictionDirection } +#else + { 0 } +#endif + }; + uint32_t ref_l0, ref_l1, b_predict_direction; vas = vaGetConfigAttributes(ctx->hwctx->display, ctx->va_profile, ctx->va_entrypoint, - &attr, 1); + attr, FF_ARRAY_ELEMS(attr)); if (vas != VA_STATUS_SUCCESS) { av_log(avctx, AV_LOG_ERROR, "Failed to query reference frames " "attribute: %d (%s).\n", vas, vaErrorStr(vas)); return AVERROR_EXTERNAL; } - if (attr.value == VA_ATTRIB_NOT_SUPPORTED) { + if (attr[0].value == VA_ATTRIB_NOT_SUPPORTED) { ref_l0 = ref_l1 = 0; } else { - ref_l0 = attr.value & 0xffff; - ref_l1 = attr.value >> 16 & 0xffff; + ref_l0 = attr[0].value & 0xffff; + ref_l1 = attr[0].value >> 16 & 0xffff; } +#if VA_CHECK_VERSION(1, 8, 0) + if (attr[1].value != VA_ATTRIB_NOT_SUPPORTED) + b_predict_direction = attr[1].value; + else + b_predict_direction = 0; + if (b_predict_direction & VA_PREDICTION_DIRECTION_BI_NOT_EMPTY) { + av_log(avctx, AV_LOG_WARNING, "Driver does not support P " + "reference frames.\n"); + if (!ref_l0 || !ref_l1) { + av_log(avctx, AV_LOG_ERROR, "Query result conflicts.\n"); + return AVERROR_EXTERNAL; + } + ctx->p_to_b = 1; + av_log(avctx, AV_LOG_WARNING, "Convert P-frames to low delay " + "B frames.\n"); + } +#else + av_log(avctx, AV_LOG_WARNING, "B-frame prediction direction query " + "is not supported with this VAAPI version.\n"); + // P frames is not supported in low power encoding for HEVC,hence + // we convert P-frames to low delay B-frames if query is not + // available with this VAAPI version. + if (ctx->low_power) + ctx->p_to_b = 1; +#endif if (ctx->codec->flags & FLAG_INTRA_ONLY || avctx->gop_size <= 1) { av_log(avctx, AV_LOG_VERBOSE, "Using intra frames only.\n"); @@ -1798,14 +1829,24 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) return AVERROR(EINVAL); } else if (!(ctx->codec->flags & FLAG_B_PICTURES) || ref_l1 < 1 || avctx->max_b_frames < 1) { - av_log(avctx, AV_LOG_VERBOSE, "Using intra and P-frames " - "(supported references: %d / %d).\n", ref_l0, ref_l1); + if (ctx->p_to_b) + av_log(avctx, AV_LOG_VERBOSE, "Using intra and low delay " + "B-frames (supported references: %d / %d).\n", + ref_l0, ref_l1); + else + av_log(avctx, AV_LOG_VERBOSE, "Using intra and P-frames " + "(supported references: %d / %d).\n", ref_l0, ref_l1); ctx->gop_size = avctx->gop_size; ctx->p_per_i = INT_MAX; ctx->b_per_p = 0; } else { - av_log(avctx, AV_LOG_VERBOSE, "Using intra, P- and B-frames " - "(supported references: %d / %d).\n", ref_l0, ref_l1); + if (ctx->p_to_b) + av_log(avctx, AV_LOG_VERBOSE, "Using intra, low delay B- and " + "B-frames (supported references: %d / %d).\n", + ref_l0, ref_l1); + else + av_log(avctx, AV_LOG_VERBOSE, "Using intra, P- and B-frames " + "(supported references: %d / %d).\n", ref_l0, ref_l1); ctx->gop_size = avctx->gop_size; ctx->p_per_i = INT_MAX; ctx->b_per_p = avctx->max_b_frames; diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index 1329f64..69dd9b3 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -313,6 +313,8 @@ typedef struct VAAPIEncodeContext { int idr_counter; int gop_counter; int end_of_stream; + // Convert P-frames to B-frames with forward reference only + int p_to_b; // Whether the driver supports ROI at all. int roi_allowed; diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 92e0510..bc7626b 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -872,6 +872,7 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, VAAPIEncodePicture *pic, VAAPIEncodeSlice *slice) { + VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodeH265Context *priv = avctx->priv_data; VAAPIEncodeH265Picture *hpic = pic->priv_data; const H265RawSPS *sps = &priv->raw_sps; @@ -894,6 +895,9 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, sh->slice_type = hpic->slice_type; + if (sh->slice_type == HEVC_SLICE_P && ctx->p_to_b) + sh->slice_type = HEVC_SLICE_B; + sh->slice_pic_order_cnt_lsb = hpic->pic_order_cnt & (1 << (sps->log2_max_pic_order_cnt_lsb_minus4 + 4)) - 1; @@ -1052,6 +1056,9 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, av_assert0(pic->type == PICTURE_TYPE_P || pic->type == PICTURE_TYPE_B); vslice->ref_pic_list0[0] = vpic->reference_frames[0]; + if (ctx->p_to_b && pic->type == PICTURE_TYPE_P) + // Reference for low delay B-frame, L0 == L1 + vslice->ref_pic_list1[0] = vpic->reference_frames[0]; } if (pic->nb_refs >= 2) { // Forward reference for B-frame.