From patchwork Tue Apr 14 08:44:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Fu, Linjie" X-Patchwork-Id: 18942 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 1F5A744BB03 for ; Tue, 14 Apr 2020 11:51:56 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EF4EE68B8F5; Tue, 14 Apr 2020 11:51:55 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CB0CD68B6A1 for ; Tue, 14 Apr 2020 11:51:48 +0300 (EEST) IronPort-SDR: EVoHJ4BPStMnoqnP0iE9EF97EjMFZq5M/VyVgQssTL9Jv3gxjZoiCWFGarWOYkB1uMF4NjPGs0 bTwIHCF1Jgfg== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Apr 2020 01:51:46 -0700 IronPort-SDR: rYjhdbVZGhfyodSU7ACU7iDIMmIO3dyJeq5BvAVb+jVyT/1zeMmnjrXBVZ5SExjEaVPEqEwoeL hEF92Or+M49A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,382,1580803200"; d="scan'208";a="243762277" Received: from icl-dev.sh.intel.com ([10.239.158.73]) by fmsmga007.fm.intel.com with ESMTP; 14 Apr 2020 01:51:45 -0700 From: Linjie Fu To: ffmpeg-devel@ffmpeg.org Date: Tue, 14 Apr 2020 16:44:21 +0800 Message-Id: <1586853861-21221-1-git-send-email-linjie.fu@intel.com> X-Mailer: git-send-email 2.7.4 Subject: [FFmpeg-devel] [PATCH, v2] lavc/vaapi_encode_h265: add private b_strategy option for hevc_vaapi X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Linjie Fu MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Allow user to choose between I/P/B frames: - normal Ordinary IB..BPB..B GOP structure. - low delay B-frames: Allows forward-predict only for all B frames, L0 == L1, supported on ICL+ platforms, required by VDENC(low_power). - reference B-frames: Convert P-frames to low delay B-frames, normal B frames still have 2 different ref_lists and allow bi-prediction. Low delay B: There is an on-going work in libva and media-driver to add querys support for low delay b, would add it once it's ready: https://github.com/intel/libva/pull/220 https://github.com/intel/libva/pull/364 https://github.com/intel/media-driver/issues/721 Signed-off-by: Linjie Fu --- Further step for low delay B would be refining the reference management to enable support for more forward-references in same ref_list. doc/encoders.texi | 16 ++++++++++++++++ libavcodec/vaapi_encode.c | 20 +++++++++++++++----- libavcodec/vaapi_encode.h | 1 + libavcodec/vaapi_encode_h265.c | 32 +++++++++++++++++++++++++++++++- 4 files changed, 63 insertions(+), 6 deletions(-) diff --git a/doc/encoders.texi b/doc/encoders.texi index e23b6b3..1fb5ecf 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -3089,6 +3089,22 @@ Some combination of the following values: Include HDR metadata if the input frames have it (@emph{mastering_display_colour_volume} and @emph{content_light_level} messages). + +@item b_strategy +Allow user to choose between I/P/B frames and specify the type of B-frames. +@table @samp +@item normal +Normal IBBPBB strategy. + +@item low_delay_b +Allows forward-predict only for all B frames, ref_list0 equals to ref_list1, +supported on ICL+ platforms, required by VDENC(low_power). + +@item ref_b +Convert P-frames to low delay B-frames as references, while normal B frames +still have 2 different ref_lists and allow bi-prediction. +@end table + @end table @end table diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 8ff720e..ea7efb6 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -1792,15 +1792,25 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) "reference frames.\n"); return AVERROR(EINVAL); } else if (!(ctx->codec->flags & FLAG_B_PICTURES) || - ref_l1 < 1 || avctx->max_b_frames < 1) { - av_log(avctx, AV_LOG_VERBOSE, "Using intra and P-frames " - "(supported references: %d / %d).\n", ref_l0, ref_l1); + ref_l1 < 1 || avctx->max_b_frames < 1 || + ctx->b_frame_strategy == 1) { + if (ctx->b_frame_strategy == 1) + av_log(avctx, AV_LOG_VERBOSE, "Using intra and B -frames " + "(low delay) (supported references: %d / %d).\n", + ref_l0, ref_l1); + else + av_log(avctx, AV_LOG_VERBOSE, "Using intra and P-frames " + "(supported references: %d / %d).\n", ref_l0, ref_l1); ctx->gop_size = avctx->gop_size; ctx->p_per_i = INT_MAX; ctx->b_per_p = 0; } else { - av_log(avctx, AV_LOG_VERBOSE, "Using intra, P- and B-frames " - "(supported references: %d / %d).\n", ref_l0, ref_l1); + if (ctx->b_frame_strategy == 2) + av_log(avctx, AV_LOG_VERBOSE, "Using intra, P- and B-frames " + "(supported references: %d / %d).\n", ref_l0, ref_l1); + else + av_log(avctx, AV_LOG_VERBOSE, "Using intra, P- and B-frames " + "(supported references: %d / %d).\n", ref_l0, ref_l1); ctx->gop_size = avctx->gop_size; ctx->p_per_i = INT_MAX; ctx->b_per_p = avctx->max_b_frames; diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index b9a3def..1feca6c 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -310,6 +310,7 @@ typedef struct VAAPIEncodeContext { int idr_counter; int gop_counter; int end_of_stream; + int b_frame_strategy; // Whether the driver supports ROI at all. int roi_allowed; diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 97dc5a7..cecde73 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -62,6 +62,7 @@ typedef struct VAAPIEncodeH265Context { int tier; int level; int sei; + int b_frame_strategy; // Derived settings. int fixed_qp_idr; @@ -894,6 +895,9 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, sh->slice_type = hpic->slice_type; + if (sh->slice_type == HEVC_SLICE_P && priv->b_frame_strategy) + sh->slice_type = HEVC_SLICE_B; + sh->slice_pic_order_cnt_lsb = hpic->pic_order_cnt & (1 << (sps->log2_max_pic_order_cnt_lsb_minus4 + 4)) - 1; @@ -1052,10 +1056,13 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, av_assert0(pic->type == PICTURE_TYPE_P || pic->type == PICTURE_TYPE_B); vslice->ref_pic_list0[0] = vpic->reference_frames[0]; + if (priv->b_frame_strategy == 1 && pic->type == PICTURE_TYPE_P) + // Reference for low delay B-frame, L0 == L1 + vslice->ref_pic_list1[0] = vpic->reference_frames[0]; } if (pic->nb_refs >= 2) { - // Forward reference for B-frame. av_assert0(pic->type == PICTURE_TYPE_B); + // Forward reference for B-frame. vslice->ref_pic_list1[0] = vpic->reference_frames[1]; } @@ -1181,6 +1188,21 @@ static av_cold int vaapi_encode_h265_init(AVCodecContext *avctx) if (priv->qp > 0) ctx->explicit_qp = priv->qp; + // Low delay B-frames is required for low power encoding. + if (ctx->low_power && priv->b_frame_strategy != 1) { + priv->b_frame_strategy = 1; + av_log(avctx, AV_LOG_WARNING, "Low delay B-frames required " + "for low power encoding.\n"); + } + + if (priv->b_frame_strategy) { + ctx->b_frame_strategy = priv->b_frame_strategy; + if (ctx->b_frame_strategy == 1) + av_log(avctx, AV_LOG_VERBOSE, "Low delay B-frames enabled.\n"); + else + av_log(avctx, AV_LOG_VERBOSE, "Reference B-frames enabled.\n"); + } + return ff_vaapi_encode_init(avctx); } @@ -1256,6 +1278,14 @@ static const AVOption vaapi_encode_h265_options[] = { 0, AV_OPT_TYPE_CONST, { .i64 = SEI_MASTERING_DISPLAY | SEI_CONTENT_LIGHT_LEVEL }, INT_MIN, INT_MAX, FLAGS, "sei" }, + { "b_strategy", "Strategy to choose between I/P/B-frames", + OFFSET(b_frame_strategy), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 2, FLAGS, "b_strategy" }, + { "normal", "Normal IB..BPB..B strategy", + 0, AV_OPT_TYPE_CONST, { .i64 = 0 }, INT_MIN, INT_MAX, FLAGS, "b_strategy" }, + { "low_delay_b", "Use low delay B-frames with forward-prediction only", + 0, AV_OPT_TYPE_CONST, { .i64 = 1 }, INT_MIN, INT_MAX, FLAGS, "b_strategy" }, + { "ref_b", "Only convert P-frames to low delay B-frames as references", + 0, AV_OPT_TYPE_CONST, { .i64 = 2 }, INT_MIN, INT_MAX, FLAGS, "b_strategy" }, { NULL }, };