From patchwork Tue May 12 13:47:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Fu, Linjie" X-Patchwork-Id: 19649 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 59B5144A35F for ; Tue, 12 May 2020 16:49:36 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 40F60689C58; Tue, 12 May 2020 16:49:36 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9F648680CAD for ; Tue, 12 May 2020 16:49:29 +0300 (EEST) IronPort-SDR: YW0JUDuNp0FlHNavXjpf37t3A2TaqPE2poB0Yqv6GgDx3M1ak1X0D7K0r85m6zG/YngbTgsgZr xM5hLUKjsc6w== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 May 2020 06:49:27 -0700 IronPort-SDR: ANAReqifM+/X6K7/XqdQUqOxzPegyApBjinFqRMHTIYoJw0b058qvthMpIEK9g3nh8600/du0F HR2RkMwR+MdQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,383,1583222400"; d="scan'208";a="265510469" Received: from icl-dev.sh.intel.com ([10.239.158.73]) by orsmga006.jf.intel.com with ESMTP; 12 May 2020 06:49:26 -0700 From: Linjie Fu To: ffmpeg-devel@ffmpeg.org Date: Tue, 12 May 2020 21:47:10 +0800 Message-Id: <1589291233-5342-1-git-send-email-linjie.fu@intel.com> X-Mailer: git-send-email 2.7.4 Subject: [FFmpeg-devel] [PATCH v2 1/4] lavc/vaapi_encode: add EQUAL_MULTI_ROWS support for slice structure X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Linjie Fu MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" VA_ENC_SLICE_STRUCTURE_EQUAL_MULTI_ROWS is added to in the latest libva (1.8.0) which matches the hardware behaviour: /** \brief Driver supports any number of rows per slice but they must be the same * for all slices except for the last one, which must be equal or smaller * to the previous slices. */ And VA_ENC_SLICE_STRUCTURE_EQUAL_ROWS is kind of deprecated for iHD since it's somehow introduced in [1] however misleading from what we actually handle, and one row per slice would not get a good quality. Caps query support in driver is WIP, and this would fix the multi slice encoding for VAEntrypointEncSliceLP. [1] Signed-off-by: Linjie Fu --- libavcodec/vaapi_encode.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index cb05ebd..234618a 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -1888,6 +1888,9 @@ static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx) req_slices = avctx->slices; } if (slice_structure & VA_ENC_SLICE_STRUCTURE_ARBITRARY_ROWS || +#if VA_CHECK_VERSION(1, 8, 0) + slice_structure & VA_ENC_SLICE_STRUCTURE_EQUAL_MULTI_ROWS || +#endif slice_structure & VA_ENC_SLICE_STRUCTURE_ARBITRARY_MACROBLOCKS) { ctx->nb_slices = req_slices; ctx->slice_size = ctx->slice_block_rows / ctx->nb_slices; From patchwork Tue May 12 13:47:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Fu, Linjie" X-Patchwork-Id: 19650 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 770E244A35F for ; Tue, 12 May 2020 16:49:42 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5D1E6689D53; Tue, 12 May 2020 16:49:42 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 82B5D680CAD for ; Tue, 12 May 2020 16:49:35 +0300 (EEST) IronPort-SDR: a/UWkhCCQePEofTOnmQnzFC71rwdl+FnI4ApBpywrvLhb2g6zWIkE9/4dEkx9zy3GiMOKPCXxi d/DcwzaUPfsg== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 May 2020 06:49:34 -0700 IronPort-SDR: QdC0RSQsB1daSw85euzJL8SQvk4Ph9vk9IwHeMqGKYx+0kOJfzvaL7UP51MqPLnoZvMZVh/WJr 2Ln72uHSUtTw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,383,1583222400"; d="scan'208";a="265510494" Received: from icl-dev.sh.intel.com ([10.239.158.73]) by orsmga006.jf.intel.com with ESMTP; 12 May 2020 06:49:32 -0700 From: Linjie Fu To: ffmpeg-devel@ffmpeg.org Date: Tue, 12 May 2020 21:47:11 +0800 Message-Id: <1589291233-5342-2-git-send-email-linjie.fu@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1589291233-5342-1-git-send-email-linjie.fu@intel.com> References: <1589291233-5342-1-git-send-email-linjie.fu@intel.com> Subject: [FFmpeg-devel] [PATCH v2 2/4] lavc/vaapi_encode: wrap slice codes into row slice functions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Linjie Fu MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Wrap current whole-row slice codes into following functions: - vaapi_encode_make_row_slice() - vaapi_encode_init_row_slice_structure() Signed-off-by: Linjie Fu --- libavcodec/vaapi_encode.c | 190 ++++++++++++++++++++++++++-------------------- 1 file changed, 109 insertions(+), 81 deletions(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 234618a..8deec02 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -162,6 +162,61 @@ static int vaapi_encode_wait(AVCodecContext *avctx, return 0; } +static int vaapi_encode_make_row_slice(AVCodecContext *avctx, + VAAPIEncodePicture *pic) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + VAAPIEncodeSlice *slice; + int i, rounding; + + for (i = 0; i < pic->nb_slices; i++) + pic->slices[i].row_size = ctx->slice_size; + + rounding = ctx->slice_block_rows - ctx->nb_slices * ctx->slice_size; + if (rounding > 0) { + // Place rounding error at top and bottom of frame. + av_assert0(rounding < pic->nb_slices); + // Some Intel drivers contain a bug where the encoder will fail + // if the last slice is smaller than the one before it. Since + // that's straightforward to avoid here, just do so. + if (rounding <= 2) { + for (i = 0; i < rounding; i++) + ++pic->slices[i].row_size; + } else { + for (i = 0; i < (rounding + 1) / 2; i++) + ++pic->slices[pic->nb_slices - i - 1].row_size; + for (i = 0; i < rounding / 2; i++) + ++pic->slices[i].row_size; + } + } else if (rounding < 0) { + // Remove rounding error from last slice only. + av_assert0(rounding < ctx->slice_size); + pic->slices[pic->nb_slices - 1].row_size += rounding; + } + + for (i = 0; i < pic->nb_slices; i++) { + slice = &pic->slices[i]; + slice->index = i; + if (i == 0) { + slice->row_start = 0; + slice->block_start = 0; + } else { + const VAAPIEncodeSlice *prev = &pic->slices[i - 1]; + slice->row_start = prev->row_start + prev->row_size; + slice->block_start = prev->block_start + prev->block_size; + } + slice->block_size = slice->row_size * ctx->slice_block_cols; + + av_log(avctx, AV_LOG_DEBUG, "Slice %d: %d-%d (%d rows), " + "%d-%d (%d blocks).\n", i, slice->row_start, + slice->row_start + slice->row_size - 1, slice->row_size, + slice->block_start, slice->block_start + slice->block_size - 1, + slice->block_size); + } + + return 0; +} + static int vaapi_encode_issue(AVCodecContext *avctx, VAAPIEncodePicture *pic) { @@ -345,57 +400,17 @@ static int vaapi_encode_issue(AVCodecContext *avctx, if (pic->nb_slices == 0) pic->nb_slices = ctx->nb_slices; if (pic->nb_slices > 0) { - int rounding; - pic->slices = av_mallocz_array(pic->nb_slices, sizeof(*pic->slices)); if (!pic->slices) { err = AVERROR(ENOMEM); goto fail; } - for (i = 0; i < pic->nb_slices; i++) - pic->slices[i].row_size = ctx->slice_size; - - rounding = ctx->slice_block_rows - ctx->nb_slices * ctx->slice_size; - if (rounding > 0) { - // Place rounding error at top and bottom of frame. - av_assert0(rounding < pic->nb_slices); - // Some Intel drivers contain a bug where the encoder will fail - // if the last slice is smaller than the one before it. Since - // that's straightforward to avoid here, just do so. - if (rounding <= 2) { - for (i = 0; i < rounding; i++) - ++pic->slices[i].row_size; - } else { - for (i = 0; i < (rounding + 1) / 2; i++) - ++pic->slices[pic->nb_slices - i - 1].row_size; - for (i = 0; i < rounding / 2; i++) - ++pic->slices[i].row_size; - } - } else if (rounding < 0) { - // Remove rounding error from last slice only. - av_assert0(rounding < ctx->slice_size); - pic->slices[pic->nb_slices - 1].row_size += rounding; - } + vaapi_encode_make_row_slice(avctx, pic); } + for (i = 0; i < pic->nb_slices; i++) { slice = &pic->slices[i]; - slice->index = i; - if (i == 0) { - slice->row_start = 0; - slice->block_start = 0; - } else { - const VAAPIEncodeSlice *prev = &pic->slices[i - 1]; - slice->row_start = prev->row_start + prev->row_size; - slice->block_start = prev->block_start + prev->block_size; - } - slice->block_size = slice->row_size * ctx->slice_block_cols; - - av_log(avctx, AV_LOG_DEBUG, "Slice %d: %d-%d (%d rows), " - "%d-%d (%d blocks).\n", i, slice->row_start, - slice->row_start + slice->row_size - 1, slice->row_size, - slice->block_start, slice->block_start + slice->block_size - 1, - slice->block_size); if (ctx->codec->slice_params_size > 0) { slice->codec_slice_params = av_mallocz(ctx->codec->slice_params_size); @@ -1828,6 +1843,54 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) return 0; } +static av_cold int vaapi_encode_init_row_slice_structure(AVCodecContext *avctx, + uint32_t slice_structure) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + int req_slices; + + // For fixed-size slices currently we only support whole rows, making + // rectangular slices. This could be extended to arbitrary runs of + // blocks, but since slices tend to be a conformance requirement and + // most cases (such as broadcast or bluray) want rectangular slices + // only it would need to be gated behind another option. + if (avctx->slices > ctx->slice_block_rows) { + av_log(avctx, AV_LOG_WARNING, "Not enough rows to use " + "configured number of slices (%d < %d); using " + "maximum.\n", ctx->slice_block_rows, avctx->slices); + req_slices = ctx->slice_block_rows; + } else { + req_slices = avctx->slices; + } + if (slice_structure & VA_ENC_SLICE_STRUCTURE_ARBITRARY_ROWS || +#if VA_CHECK_VERSION(1, 8, 0) + slice_structure & VA_ENC_SLICE_STRUCTURE_EQUAL_MULTI_ROWS || +#endif + slice_structure & VA_ENC_SLICE_STRUCTURE_ARBITRARY_MACROBLOCKS) { + ctx->nb_slices = req_slices; + ctx->slice_size = ctx->slice_block_rows / ctx->nb_slices; + } else if (slice_structure & VA_ENC_SLICE_STRUCTURE_POWER_OF_TWO_ROWS) { + int k; + for (k = 1;; k *= 2) { + if (2 * k * (req_slices - 1) + 1 >= ctx->slice_block_rows) + break; + } + ctx->nb_slices = (ctx->slice_block_rows + k - 1) / k; + ctx->slice_size = k; +#if VA_CHECK_VERSION(1, 0, 0) + } else if (slice_structure & VA_ENC_SLICE_STRUCTURE_EQUAL_ROWS) { + ctx->nb_slices = ctx->slice_block_rows; + ctx->slice_size = 1; +#endif + } else { + av_log(avctx, AV_LOG_ERROR, "Driver does not support any usable " + "slice structure modes (%#x).\n", slice_structure); + return AVERROR(EINVAL); + } + + return 0; +} + static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx) { VAAPIEncodeContext *ctx = avctx->priv_data; @@ -1835,7 +1898,7 @@ static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx) { VAConfigAttribEncSliceStructure } }; VAStatus vas; uint32_t max_slices, slice_structure; - int req_slices; + int ret; if (!(ctx->codec->flags & FLAG_SLICE_CONTROL)) { if (avctx->slices > 0) { @@ -1874,44 +1937,9 @@ static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx) return AVERROR(EINVAL); } - // For fixed-size slices currently we only support whole rows, making - // rectangular slices. This could be extended to arbitrary runs of - // blocks, but since slices tend to be a conformance requirement and - // most cases (such as broadcast or bluray) want rectangular slices - // only it would need to be gated behind another option. - if (avctx->slices > ctx->slice_block_rows) { - av_log(avctx, AV_LOG_WARNING, "Not enough rows to use " - "configured number of slices (%d < %d); using " - "maximum.\n", ctx->slice_block_rows, avctx->slices); - req_slices = ctx->slice_block_rows; - } else { - req_slices = avctx->slices; - } - if (slice_structure & VA_ENC_SLICE_STRUCTURE_ARBITRARY_ROWS || -#if VA_CHECK_VERSION(1, 8, 0) - slice_structure & VA_ENC_SLICE_STRUCTURE_EQUAL_MULTI_ROWS || -#endif - slice_structure & VA_ENC_SLICE_STRUCTURE_ARBITRARY_MACROBLOCKS) { - ctx->nb_slices = req_slices; - ctx->slice_size = ctx->slice_block_rows / ctx->nb_slices; - } else if (slice_structure & VA_ENC_SLICE_STRUCTURE_POWER_OF_TWO_ROWS) { - int k; - for (k = 1;; k *= 2) { - if (2 * k * (req_slices - 1) + 1 >= ctx->slice_block_rows) - break; - } - ctx->nb_slices = (ctx->slice_block_rows + k - 1) / k; - ctx->slice_size = k; -#if VA_CHECK_VERSION(1, 0, 0) - } else if (slice_structure & VA_ENC_SLICE_STRUCTURE_EQUAL_ROWS) { - ctx->nb_slices = ctx->slice_block_rows; - ctx->slice_size = 1; -#endif - } else { - av_log(avctx, AV_LOG_ERROR, "Driver does not support any usable " - "slice structure modes (%#x).\n", slice_structure); - return AVERROR(EINVAL); - } + ret = vaapi_encode_init_row_slice_structure(avctx, slice_structure); + if (ret < 0) + return ret; if (ctx->nb_slices > avctx->slices) { av_log(avctx, AV_LOG_WARNING, "Slice count rounded up to " From patchwork Tue May 12 13:47:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Fu, Linjie" X-Patchwork-Id: 19651 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B43AB44A35F for ; Tue, 12 May 2020 16:49:43 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9B56968A16F; Tue, 12 May 2020 16:49:43 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id EBE6B680CAD for ; Tue, 12 May 2020 16:49:36 +0300 (EEST) IronPort-SDR: mM95gQrxNcM+qWwYdCY6IOlWLR2cwVR2VkG/RQSlsRQS/VJnYE8BREEgk3S52jIbvNgXjzQpbO Uq2KiW9dmwvw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 May 2020 06:49:36 -0700 IronPort-SDR: 6pxw7RJmgXlHhPlmkU9hluT/5xG1Mzi6PsXqp+4ormhBk0yKnLlJNbnBpb6VstBkmqSLxNkiu/ Kun5N+P6bDAg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,383,1583222400"; d="scan'208";a="265510507" Received: from icl-dev.sh.intel.com ([10.239.158.73]) by orsmga006.jf.intel.com with ESMTP; 12 May 2020 06:49:35 -0700 From: Linjie Fu To: ffmpeg-devel@ffmpeg.org Date: Tue, 12 May 2020 21:47:12 +0800 Message-Id: <1589291233-5342-3-git-send-email-linjie.fu@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1589291233-5342-1-git-send-email-linjie.fu@intel.com> References: <1589291233-5342-1-git-send-email-linjie.fu@intel.com> Subject: [FFmpeg-devel] [PATCH v2 3/4] lavc/vaapi_encode: add tile slice encoding support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Linjie Fu MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Add functions to initialize tile slice structure and make tile slice: - vaapi_encode_init_tile_slice_structure - vaapi_encode_make_tile_slice Tile slice is not allowed to cross the boundary of a tile due to the constraints of media-driver. Currently adding support for one slice per tile. N x N tile encoding is supposed to be supported with the the capability of ARBITRARY_MACROBLOCKS slice structures. N X 1 tile encoding should also work in ARBITRARY_ROWS slice structure. Signed-off-by: Linjie Fu --- libavcodec/vaapi_encode.c | 128 +++++++++++++++++++++++++++++++++++++++++++--- libavcodec/vaapi_encode.h | 16 ++++++ 2 files changed, 136 insertions(+), 8 deletions(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 8deec02..2eb8b48 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -217,6 +217,33 @@ static int vaapi_encode_make_row_slice(AVCodecContext *avctx, return 0; } +static int vaapi_encode_make_tile_slice(AVCodecContext *avctx, + VAAPIEncodePicture *pic) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + VAAPIEncodeSlice *slice; + int i, j, index; + + for (i = 0; i < ctx->tile_cols; i++) { + for (j = 0; j < ctx->tile_rows; j++) { + index = j * ctx->tile_cols + i; + slice = &pic->slices[index]; + slice->index = index; + + pic->slices[index].block_start = ctx->col_bd[i] + + ctx->row_bd[j] * ctx->slice_block_cols; + pic->slices[index].block_size = ctx->row_height[j] * ctx->col_width[i]; + + av_log(avctx, AV_LOG_DEBUG, "Slice %2d: (%2d, %2d) start at: %4d " + "width:%2d height:%2d (%d blocks).\n", index, ctx->col_bd[i], + ctx->row_bd[j], slice->block_start, ctx->col_width[i], + ctx->row_height[j], slice->block_size); + } + } + + return 0; +} + static int vaapi_encode_issue(AVCodecContext *avctx, VAAPIEncodePicture *pic) { @@ -406,7 +433,10 @@ static int vaapi_encode_issue(AVCodecContext *avctx, goto fail; } - vaapi_encode_make_row_slice(avctx, pic); + if (ctx->tile_rows && ctx->tile_cols) + vaapi_encode_make_tile_slice(avctx, pic); + else + vaapi_encode_make_row_slice(avctx, pic); } for (i = 0; i < pic->nb_slices; i++) { @@ -1891,11 +1921,76 @@ static av_cold int vaapi_encode_init_row_slice_structure(AVCodecContext *avctx, return 0; } +static av_cold int vaapi_encode_init_tile_slice_structure(AVCodecContext *avctx, + uint32_t slice_structure) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + int i, req_tiles; + + if (!(slice_structure & VA_ENC_SLICE_STRUCTURE_ARBITRARY_MACROBLOCKS || + (slice_structure & VA_ENC_SLICE_STRUCTURE_ARBITRARY_ROWS && + ctx->tile_cols == 1))) { + av_log(avctx, AV_LOG_ERROR, "Supported slice structure (%#x) doesn't work for " + "current tile requirement.\n", slice_structure); + return AVERROR(EINVAL); + } + + if (ctx->tile_rows > ctx->slice_block_rows || + ctx->tile_cols > ctx->slice_block_cols) { + av_log(avctx, AV_LOG_WARNING, "Not enough block rows/cols (%d x %d) " + "for configured number of tile (%d x %d); ", + ctx->slice_block_rows, ctx->slice_block_cols, + ctx->tile_rows, ctx->tile_cols); + ctx->tile_rows = ctx->tile_rows > ctx->slice_block_rows ? + ctx->slice_block_rows : ctx->tile_rows; + ctx->tile_cols = ctx->tile_cols > ctx->slice_block_cols ? + ctx->slice_block_cols : ctx->tile_cols; + av_log(avctx, AV_LOG_WARNING, "using allowed maximum (%d x %d).\n", + ctx->tile_rows, ctx->tile_cols); + } + + req_tiles = ctx->tile_rows * ctx->tile_cols; + + // Tile slice is not allowed to cross the boundary of a tile due to + // the constraints of media-driver. Currently we support one slice + // per tile. This could be extended to multiple slices per tile. + if (avctx->slices != req_tiles) + av_log(avctx, AV_LOG_WARNING, "The number of requested slices " + "mismatches with configured number of tile (%d != %d); " + "using requested tile number for slice.\n", + avctx->slices, req_tiles); + + ctx->nb_slices = req_tiles; + + // Default in uniform spacing + // 6-3, 6-5 + for (i = 0; i < ctx->tile_cols; i++) { + ctx->col_width[i] = ( i + 1 ) * ctx->slice_block_cols / ctx->tile_cols - + i * ctx->slice_block_cols / ctx->tile_cols; + ctx->col_bd[i + 1] = ctx->col_bd[i] + ctx->col_width[i]; + } + // 6-4, 6-6 + for (i = 0; i < ctx->tile_rows; i++) { + ctx->row_height[i] = ( i + 1 ) * ctx->slice_block_rows / ctx->tile_rows - + i * ctx->slice_block_rows / ctx->tile_rows; + ctx->row_bd[i + 1] = ctx->row_bd[i] + ctx->row_height[i]; + } + + av_log(avctx, AV_LOG_VERBOSE, "Encoding pictures with %d x %d tile.\n", + ctx->tile_rows, ctx->tile_cols); + + return 0; +} + static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx) { VAAPIEncodeContext *ctx = avctx->priv_data; - VAConfigAttrib attr[2] = { { VAConfigAttribEncMaxSlices }, - { VAConfigAttribEncSliceStructure } }; + VAConfigAttrib attr[3] = { { VAConfigAttribEncMaxSlices }, + { VAConfigAttribEncSliceStructure }, +#if VA_CHECK_VERSION(1, 1, 0) + { VAConfigAttribEncTileSupport }, +#endif + }; VAStatus vas; uint32_t max_slices, slice_structure; int ret; @@ -1913,7 +2008,7 @@ static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx) ctx->slice_block_cols = (avctx->width + ctx->slice_block_width - 1) / ctx->slice_block_width; - if (avctx->slices <= 1) { + if (avctx->slices <= 1 && !ctx->tile_rows && !ctx->tile_cols) { ctx->nb_slices = 1; ctx->slice_size = ctx->slice_block_rows; return 0; @@ -1937,7 +2032,25 @@ static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx) return AVERROR(EINVAL); } - ret = vaapi_encode_init_row_slice_structure(avctx, slice_structure); + if (ctx->tile_rows && ctx->tile_cols) { +#if VA_CHECK_VERSION(1, 1, 0) + uint32_t tile_support = attr[2].value; + if (tile_support == VA_ATTRIB_NOT_SUPPORTED) { + av_log(avctx, AV_LOG_ERROR, "Driver does not support encoding " + "pictures as multiple tiles.\n."); + return AVERROR(EINVAL); + } +#else + av_log(avctx, AV_LOG_ERROR, "Tile encoding option is " + "not supported with this VAAPI version.\n"); + return AVERROR(EINVAL); +#endif + } + + if (ctx->tile_rows && ctx->tile_cols) + ret = vaapi_encode_init_tile_slice_structure(avctx, slice_structure); + else + ret = vaapi_encode_init_row_slice_structure(avctx, slice_structure); if (ret < 0) return ret; @@ -1953,9 +2066,8 @@ static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx) return AVERROR(EINVAL); } - av_log(avctx, AV_LOG_VERBOSE, "Encoding pictures with %d slices " - "(default size %d block rows).\n", - ctx->nb_slices, ctx->slice_size); + av_log(avctx, AV_LOG_VERBOSE, "Encoding pictures with %d slices.\n", + ctx->nb_slices); return 0; } diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index 1329f64..10cffce 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -43,6 +43,10 @@ enum { MAX_PICTURE_REFERENCES = 2, MAX_REORDER_DELAY = 16, MAX_PARAM_BUFFER_SIZE = 1024, + // A.4.1: table A.6 allows at most 22 tile rows for any level. + MAX_TILE_ROWS = 22, + // A.4.1: table A.6 allows at most 20 tile columns for any level. + MAX_TILE_COLS = 20, }; extern const AVCodecHWConfigInternal *ff_vaapi_encode_hw_configs[]; @@ -302,6 +306,18 @@ typedef struct VAAPIEncodeContext { int nb_slices; int slice_size; + // Tile encoding. + int tile_rows; + int tile_cols; + // Tile width of the i-th column. + int col_width[MAX_TILE_COLS]; + // Tile height of i-th row. + int row_height[MAX_TILE_ROWS]; + // Location of the i-th tile column boundary. + int col_bd[MAX_TILE_COLS + 1]; + // Location of the i-th tile row boundary. + int row_bd[MAX_TILE_ROWS + 1]; + // Frame type decision. int gop_size; int closed_gop; From patchwork Tue May 12 13:47:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Fu, Linjie" X-Patchwork-Id: 19652 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id CCA5544A35F for ; Tue, 12 May 2020 16:49:46 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B48D368A176; Tue, 12 May 2020 16:49:46 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 29D41689A91 for ; Tue, 12 May 2020 16:49:39 +0300 (EEST) IronPort-SDR: 2yQP2rIggoaVlTf5SEIG0CZI0L85VdLP1CecRWc0go2uOTKwk0puzKm6gHRf7LCs6sKG0zf/CY NydloPuVnlXw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 May 2020 06:49:38 -0700 IronPort-SDR: D4cJ9jw/wpQKW40HDyIjs6CPOjpmwMenFfF/ybcFI1dmgIdb6BZrhlTbNZG0whux3WVb0Ae0RD D3BWWLWvlMkg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,383,1583222400"; d="scan'208";a="265510515" Received: from icl-dev.sh.intel.com ([10.239.158.73]) by orsmga006.jf.intel.com with ESMTP; 12 May 2020 06:49:37 -0700 From: Linjie Fu To: ffmpeg-devel@ffmpeg.org Date: Tue, 12 May 2020 21:47:13 +0800 Message-Id: <1589291233-5342-4-git-send-email-linjie.fu@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1589291233-5342-1-git-send-email-linjie.fu@intel.com> References: <1589291233-5342-1-git-send-email-linjie.fu@intel.com> Subject: [FFmpeg-devel] [PATCH v2 4/4] lavc/vaapi_encode_h265: add h265 tile encoding support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Linjie Fu MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Default to enable uniform_spacing_flag. Guess level by the tile rows/cols. Supported for ICL+ platforms. Also add documentations. To encode with 4 rows 2 columns: ffmpeg ... -c:v hevc_vaapi -tiles 4x2 ... ffmpeg ... -c:v hevc_vaapi -tile_rows 4 -tile_cols 2 ... Suggested-by: Jun Zhao Signed-off-by: Linjie Fu --- doc/encoders.texi | 13 +++++++++++++ libavcodec/vaapi_encode_h265.c | 38 +++++++++++++++++++++++++++++++++++++- 2 files changed, 50 insertions(+), 1 deletion(-) diff --git a/doc/encoders.texi b/doc/encoders.texi index aa3a6ee..ea4ae28 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -3093,6 +3093,19 @@ Include HDR metadata if the input frames have it messages). @end table +@item tiles +Set the number of tiles to encode the input video with, as rows x columns. +Larger numbers allow greater parallelism in both encoding and decoding, but +may decrease coding efficiency. + +@item tile_rows +Selects how many rows of tiles to encode with. For example, 4 tile rows would +be requested by setting the tile_rows option to 4. + +@item tile_cols +Selects how many columns of tiles to encode with. For example, 5 tile columns +would be requested by setting the tile_cols option to 5. + @end table @item mjpeg_vaapi diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 92e0510..51e3847 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -63,6 +63,9 @@ typedef struct VAAPIEncodeH265Context { int level; int sei; + int trows; + int tcols; + // Derived settings. int fixed_qp_idr; int fixed_qp_p; @@ -345,7 +348,7 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) level = ff_h265_guess_level(ptl, avctx->bit_rate, ctx->surface_width, ctx->surface_height, - ctx->nb_slices, 1, 1, + ctx->nb_slices, ctx->tile_rows, ctx->tile_cols, (ctx->b_per_p > 0) + 1); if (level) { av_log(avctx, AV_LOG_VERBOSE, "Using level %s.\n", level->name); @@ -558,6 +561,20 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) pps->pps_loop_filter_across_slices_enabled_flag = 1; + if (ctx->tile_rows && ctx->tile_cols) { + pps->tiles_enabled_flag = 1; + pps->uniform_spacing_flag = 1; + + pps->num_tile_rows_minus1 = ctx->tile_rows - 1; + pps->num_tile_columns_minus1 = ctx->tile_cols - 1; + + pps->loop_filter_across_tiles_enabled_flag = 1; + + for (i = 0; i <= pps->num_tile_rows_minus1; i++) + pps->row_height_minus1[i] = ctx->row_height[i] - 1; + for (i = 0; i <= pps->num_tile_columns_minus1; i++) + pps->column_width_minus1[i] = ctx->col_width[i] - 1; + } // Fill VAAPI parameter buffers. @@ -666,6 +683,13 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) }, }; + if (pps->tiles_enabled_flag) { + for (i = 0; i <= vpic->num_tile_rows_minus1; i++) + vpic->row_height_minus1[i] = pps->row_height_minus1[i]; + for (i = 0; i <= vpic->num_tile_columns_minus1; i++) + vpic->column_width_minus1[i] = pps->column_width_minus1[i]; + } + return 0; } @@ -1181,6 +1205,11 @@ static av_cold int vaapi_encode_h265_init(AVCodecContext *avctx) if (priv->qp > 0) ctx->explicit_qp = priv->qp; + if (priv->trows && priv->tcols) { + ctx->tile_rows = priv->trows; + ctx->tile_cols = priv->tcols; + } + return ff_vaapi_encode_init(avctx); } @@ -1257,6 +1286,13 @@ static const AVOption vaapi_encode_h265_options[] = { { .i64 = SEI_MASTERING_DISPLAY | SEI_CONTENT_LIGHT_LEVEL }, INT_MIN, INT_MAX, FLAGS, "sei" }, + { "tiles", "Tile rows x cols", + OFFSET(trows), AV_OPT_TYPE_IMAGE_SIZE, { .str = NULL }, 0, 0, FLAGS }, + { "tile_rows", "Number of rows for tile encoding", + OFFSET(trows), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, FLAGS }, + { "tile_cols", "Number of cols for tile encoding", + OFFSET(tcols), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, FLAGS }, + { NULL }, };