From patchwork Mon Mar 23 14:24:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Fu, Linjie" X-Patchwork-Id: 18370 Delivered-To: andriy.gelman@gmail.com Received: by 2002:a0c:ab15:0:0:0:0:0 with SMTP id h21csp3126730qvb; Mon, 23 Mar 2020 07:30:03 -0700 (PDT) X-Google-Smtp-Source: ADFU+vufEX8IUQfPVKvJUF1fBKW3qQDDpaUo/7oPYAiVGCF7WSgJvnTuTDhYlUwbxqzn2XV5x305 X-Received: by 2002:aa7:c2cc:: with SMTP id m12mr21957375edp.11.1584973803155; Mon, 23 Mar 2020 07:30:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1584973803; cv=none; d=google.com; s=arc-20160816; b=Qg49NyPkEuwLpONRwEyIqSKDNWNUJA2Xi0Po7WJv42rvYSHNxHN5yLfZjRo2HycmJB h6Zflui98yXJHuKxlYvQwP6v0YIl62XfZjDFNTKsQVWbaWI9N+1+wuHsfr/Dqgm0HOx8 JJxJ5FbiocXqKeTZ29M+CvBGOk51RgqIfK0TJHNBRJj5Dwft3teVpbIx2Tk8urvlqOyE HqIoXjYvg7TxRhCOJeoJ0ByZW5mzwMk5YhWcvZpBLBiclOD21kHJA8iCZn/u9Onw32Rb X4uq3p0PeNBOgJ0Rzi2q/EnLL5c0xkbolOohHM+fN2KoGItvSrTTix5OXMRgA1KGllDe RUoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:ironport-sdr :ironport-sdr:delivered-to; bh=oXcOFjPwLehUPG3CoCJhbq9cuLUQQ+w2aulPfLi1pgA=; b=VDOUiuXRcTPxPuA/QBvLtsWOr70izafUHXhyRivgScT/hTuItnq/nGCfgqmIKyVJZf AqkefE0BdgPiG4JEvEl6zEqcQMaLvi9fcxBljNWHzLbBuMeh2PV1Kx8OAi3mkjwuFNvq 5w0JyKjDUzzS0F7uxTjUTydDPfc+UPbpQ4qA5lBsExRA3K7zWAeki4U0bCY7eFQzjOm4 kDpCTky2ghenyth/NyPFpSr0EqP+JjN5XEvX8iR6JnXcWA6aCLgRE+GSJt5VDSl+9+v3 O4cOs1h27JPiqoGrdNxYzy6izLW0WerTmuB6SRuErWrT2pGgj6/qwHgT2VDXpVnIAlVt DBvQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a9si9789704edk.466.2020.03.23.07.30.02; Mon, 23 Mar 2020 07:30:03 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DCBCA68B703; Mon, 23 Mar 2020 16:30:01 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CA36468B679 for ; Mon, 23 Mar 2020 16:29:54 +0200 (EET) IronPort-SDR: jV2lTbQIpO9bUdf9JDhf5zItqMl+vPfCUEJWAt3omXNUIqMJw9kRbe/3xd+UZu1SgtQg3rnJTL IOqv7iDVZ0HA== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Mar 2020 07:29:52 -0700 IronPort-SDR: cucs5H246pdPqS+8szWNixhU+eKd7HtjSWfMikPVtHoB6uH9tXMNq7QsuPPZ+55ACy8ETT1uQb 7C7MAvAN+Taw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,296,1580803200"; d="scan'208";a="445828950" Received: from icl-dev.sh.intel.com ([10.239.158.73]) by fmsmga005.fm.intel.com with ESMTP; 23 Mar 2020 07:29:51 -0700 From: Linjie Fu To: ffmpeg-devel@ffmpeg.org Date: Mon, 23 Mar 2020 22:24:16 +0800 Message-Id: <1584973456-32021-1-git-send-email-linjie.fu@intel.com> X-Mailer: git-send-email 2.7.4 Subject: [FFmpeg-devel] [PATCH 2/3] lavc/vaapi_encode: add tile slice encoding support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Linjie Fu MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: z9hveICHNaoU Content-Length: 12406 Add functions to initialize tile slice structure and make tile slice: - vaapi_encode_init_tile_slice_structure - vaapi_encode_make_tile_slice Tile slice is not allowed to cross the boundary of a tile due to the constraints of media-driver. Currently adding support for one slice per tile. N x N tile encoding is supposed to be supported with the the capability of ARBITRARY_MACROBLOCKS slice structures. N X 1 tile encoding should also work in ARBITRARY_ROWS slice structure. Signed-off-by: Linjie Fu --- libavcodec/vaapi_encode.c | 127 +++++++++++++++++++++++++++++++++++++++++++--- libavcodec/vaapi_encode.h | 16 ++++++ 2 files changed, 135 insertions(+), 8 deletions(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 8b48379..16b157d 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -212,6 +212,33 @@ static int vaapi_encode_make_row_slice(AVCodecContext *avctx, return 0; } +static int vaapi_encode_make_tile_slice(AVCodecContext *avctx, + VAAPIEncodePicture *pic) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + VAAPIEncodeSlice *slice; + int i, j, index; + + for (i = 0; i < ctx->tile_cols; i++) { + for (j = 0; j < ctx->tile_rows; j++) { + index = j * ctx->tile_cols + i; + slice = &pic->slices[index]; + slice->index = index; + + pic->slices[index].block_start = ctx->col_bd[i] + + ctx->row_bd[j] * ctx->slice_block_cols; + pic->slices[index].block_size = ctx->row_height[j] * ctx->col_width[i]; + + av_log(avctx, AV_LOG_DEBUG, "Slice %2d: (%2d, %2d) start at: %4d " + "width:%2d height:%2d (%d blocks).\n", index, ctx->col_bd[i], + ctx->row_bd[j], slice->block_start, ctx->col_width[i], + ctx->row_height[j], slice->block_size); + } + } + + return 0; +} + static int vaapi_encode_issue(AVCodecContext *avctx, VAAPIEncodePicture *pic) { @@ -401,7 +428,10 @@ static int vaapi_encode_issue(AVCodecContext *avctx, goto fail; } - vaapi_encode_make_row_slice(avctx, pic); + if (ctx->tile_rows && ctx->tile_cols) + vaapi_encode_make_tile_slice(avctx, pic); + else + vaapi_encode_make_row_slice(avctx, pic); } for (i = 0; i < pic->nb_slices; i++) { @@ -1883,11 +1913,75 @@ static av_cold int vaapi_encode_init_row_slice_structure(AVCodecContext *avctx, return 0; } +static av_cold int vaapi_encode_init_tile_slice_structure(AVCodecContext *avctx, + uint32_t slice_structure) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + int i, req_tiles; + + if (!(slice_structure & VA_ENC_SLICE_STRUCTURE_ARBITRARY_MACROBLOCKS || + (slice_structure & VA_ENC_SLICE_STRUCTURE_ARBITRARY_ROWS && ctx->tile_cols == 1))) { + av_log(avctx, AV_LOG_ERROR, "Supported slice structure (%#x) doesn't work for " + "current tile requirement.\n", slice_structure); + return AVERROR(EINVAL); + } + + if (ctx->tile_rows > ctx->slice_block_rows || + ctx->tile_cols > ctx->slice_block_cols) { + av_log(avctx, AV_LOG_WARNING, "Not enough block rows/cols (%d x %d) " + "for configured number of tile (%d x %d); ", + ctx->slice_block_rows, ctx->slice_block_cols, + ctx->tile_rows, ctx->tile_cols); + ctx->tile_rows = ctx->tile_rows > ctx->slice_block_rows ? + ctx->slice_block_rows : ctx->tile_rows; + ctx->tile_cols = ctx->tile_cols > ctx->slice_block_cols ? + ctx->slice_block_cols : ctx->tile_cols; + av_log(avctx, AV_LOG_WARNING, "using allowed maximum (%d x %d).\n", + ctx->tile_rows, ctx->tile_cols); + } + + req_tiles = ctx->tile_rows * ctx->tile_cols; + + // Tile slice is not allowed to cross the boundary of a tile due to + // the constraints of media-driver. Currently we support one slice + // per tile. This could be extended to multiple slices per tile. + if (avctx->slices != req_tiles) + av_log(avctx, AV_LOG_WARNING, "The number of requested slices " + "mismatches with configured number of tile (%d != %d); " + "using requested tile number for slice.\n", + avctx->slices, req_tiles); + + ctx->nb_slices = req_tiles; + + // Default in uniform spacing + // 6-3, 6-5 + for (i = 0; i < ctx->tile_cols; i++) { + ctx->col_width[i] = ( i + 1 ) * ctx->slice_block_cols / ctx->tile_cols - + i * ctx->slice_block_cols / ctx->tile_cols; + ctx->col_bd[i + 1] = ctx->col_bd[i] + ctx->col_width[i]; + } + // 6-4, 6-6 + for (i = 0; i < ctx->tile_rows; i++) { + ctx->row_height[i] = ( i + 1 ) * ctx->slice_block_rows / ctx->tile_rows - + i * ctx->slice_block_rows / ctx->tile_rows; + ctx->row_bd[i + 1] = ctx->row_bd[i] + ctx->row_height[i]; + } + + av_log(avctx, AV_LOG_VERBOSE, "Encoding pictures with %d x %d tile.\n", + ctx->tile_rows, ctx->tile_cols); + + return 0; +} + static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx) { VAAPIEncodeContext *ctx = avctx->priv_data; - VAConfigAttrib attr[2] = { { VAConfigAttribEncMaxSlices }, - { VAConfigAttribEncSliceStructure } }; + VAConfigAttrib attr[3] = { { VAConfigAttribEncMaxSlices }, + { VAConfigAttribEncSliceStructure }, +#if VA_CHECK_VERSION(1, 1, 0) + { VAConfigAttribEncTileSupport }, +#endif + }; VAStatus vas; uint32_t max_slices, slice_structure; int ret; @@ -1905,7 +1999,7 @@ static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx) ctx->slice_block_cols = (avctx->width + ctx->slice_block_width - 1) / ctx->slice_block_width; - if (avctx->slices <= 1) { + if (avctx->slices <= 1 && !ctx->tile_rows && !ctx->tile_cols) { ctx->nb_slices = 1; ctx->slice_size = ctx->slice_block_rows; return 0; @@ -1929,7 +2023,25 @@ static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx) return AVERROR(EINVAL); } - ret = vaapi_encode_init_row_slice_structure(avctx, slice_structure); + if (ctx->tile_rows && ctx->tile_cols) { +#if VA_CHECK_VERSION(1, 1, 0) + uint32_t tile_support = attr[2].value; + if (tile_support == VA_ATTRIB_NOT_SUPPORTED) { + av_log(avctx, AV_LOG_ERROR, "Driver does not support encoding " + "pictures as multiple tiles.\n."); + return AVERROR(EINVAL); + } +#else + av_log(avctx, AV_LOG_ERROR, "Tile encoding option is " + "not supported with this VAAPI version.\n"); + return AVERROR(EINVAL); +#endif + } + + if (ctx->tile_rows && ctx->tile_cols) + ret = vaapi_encode_init_tile_slice_structure(avctx, slice_structure); + else + ret = vaapi_encode_init_row_slice_structure(avctx, slice_structure); if (ret < 0) return ret; @@ -1945,9 +2057,8 @@ static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx) return AVERROR(EINVAL); } - av_log(avctx, AV_LOG_VERBOSE, "Encoding pictures with %d slices " - "(default size %d block rows).\n", - ctx->nb_slices, ctx->slice_size); + av_log(avctx, AV_LOG_VERBOSE, "Encoding pictures with %d slices.\n", + ctx->nb_slices); return 0; } diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index b9a3def..5da9a76 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -42,6 +42,10 @@ enum { MAX_PICTURE_REFERENCES = 2, MAX_REORDER_DELAY = 16, MAX_PARAM_BUFFER_SIZE = 1024, + // A.4.1: table A.6 allows at most 22 tile rows for any level. + MAX_TILE_ROWS = 22, + // A.4.1: table A.6 allows at most 20 tile columns for any level. + MAX_TILE_COLS = 20, }; enum { @@ -299,6 +303,18 @@ typedef struct VAAPIEncodeContext { int nb_slices; int slice_size; + // Tile encoding. + int tile_rows; + int tile_cols; + // Tile width of the i-th column. + int col_width[MAX_TILE_COLS]; + // Tile height of i-th row. + int row_height[MAX_TILE_ROWS]; + // Location of the i-th tile column boundary. + int col_bd[MAX_TILE_COLS + 1]; + // Location of the i-th tile row boundary. + int row_bd[MAX_TILE_ROWS + 1]; + // Frame type decision. int gop_size; int closed_gop;