From patchwork Wed Nov 8 01:05:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 44564 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:f392:b0:181:818d:5e7f with SMTP id qr18csp69312pzb; Tue, 7 Nov 2023 17:07:12 -0800 (PST) X-Google-Smtp-Source: AGHT+IGWcqRxMKbt0alxu7C4JxFMWa3HiCj4gMQ2ZDHqkxwRACmatJz/ax4xi4KuWI5FcO7WcEXE X-Received: by 2002:a17:907:3e9b:b0:9c5:2806:72e9 with SMTP id hs27-20020a1709073e9b00b009c5280672e9mr256201ejc.34.1699405631814; Tue, 07 Nov 2023 17:07:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1699405631; cv=none; d=google.com; s=arc-20160816; b=Of5Yphl83n3FBVemO4uQbwv95k9v/AgSyjvG5dxa3HLnxd6TRBkssMvOmSA/Wp9PQJ WqVpJpg84/TlxOZj7ddaC0lmw4wrV2WnIhWmbcp7MHQgW+s/GDIhv4VUZkA9ARdE3k+s AUSZi5txYjwKU/xIFFJ0+jqvlqEI9f5U4JGUdVgzXBaMohv6wJEg0ikCUT3kpvNzOxYf vA1iMJBWov14AUI75afqLaVaGTSmQ+/d7DwFt5HnWd1B6gR/p2gsisfIDAiqHZiTeRQj pflpyJMqKpaK5CJ1mVnys7sPzXPVqX7rRNXZYYasiviTy5S7gkGdiWHdh1h1qj4gGOEW HQUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=mJ9J03fAxXNpHLavJsGghIuY6plnFIuLX9+2wr10uLs=; fh=9GN8hc/WTnabikg2VxPLeDN+i3R37IqYi/LhxNANmV0=; b=ge2t+VjDvyTy+007aVaDTVZSAvVKk/VuAORNH6Gr6AXck6m8cf+6KjZ/CKB5mqd+cA oiDuROWMhueKUoZz+ZPjsQ5ROX7bCPV8clZ/q7ZiOuY2GNhD4F8/aTIpdoQPB3oCnhPw byyDjo+31cB9M8WNel+GRfOvOtGVwDykRtFI2jM+JXZjG5XPC96IaDw1K1porVVdJssZ 36yGl76yh7pXysmDrPmQ2kqcjk7uB2RVRZiW09pae8B883vSsP++RgIaKnBvR22UDvI6 QkMjys5mvaTDheSmQq8oHNHRlsRk+0JaUoQwA2IQfQk8pFFq1y9XnU19hxBcqTgJ3ZzG EQdg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=YnvM9M4G; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id mp40-20020a1709071b2800b009d3f3aa3639si1683974ejc.950.2023.11.07.17.07.05; Tue, 07 Nov 2023 17:07:11 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=YnvM9M4G; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9514868CCCB; Wed, 8 Nov 2023 03:06:35 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.115]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 96A9B68CCC2 for ; Wed, 8 Nov 2023 03:06:28 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1699405593; x=1730941593; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Il8FiQq4h/jdhEqjdLs3baKEZD7cY/e87xMnMCrc/x0=; b=YnvM9M4G34YfHi5hrBS3G8JdhlRIAXUX5QA+oDoRbxvWOO/yzKF1BtB2 aZaSuMQtNom15aUZSR5ie4Zg/710lueKDmqcECCveE6/HjhHHoVd1mi1W veixeeEH6Mbgp/znN7ANucp1b5o3OhS5Y19V4H7rriC+IXwAtUO6MSqpz vkGuhlywmN9pKFtTXOe8gzp9I6DF/2b1eeZcjYg5/W5p9hq3FkIdzL/bo DBFoM8rDYM3mm1CoZhizBMirzEKsdrdi+GosoNU8xMRhKEq5PwASsDXdH DB+51OwI+v4X/jJVTHvDh1qHsVbHCV2UCd37pYIl8+xv9tgtnay2agbhI Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10887"; a="389470359" X-IronPort-AV: E=Sophos;i="6.03,285,1694761200"; d="scan'208";a="389470359" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Nov 2023 17:06:22 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10887"; a="762894261" X-IronPort-AV: E=Sophos;i="6.03,285,1694761200"; d="scan'208";a="762894261" Received: from desktop-qn7n0nf.sh.intel.com (HELO localhost.localdomain) ([10.239.160.66]) by orsmga002.jf.intel.com with ESMTP; 07 Nov 2023 17:06:20 -0800 From: Tong Wu To: ffmpeg-devel@ffmpeg.org Date: Wed, 8 Nov 2023 09:05:46 +0800 Message-ID: <20231108010552.52-4-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20231108010552.52-1-tong1.wu@intel.com> References: <20231108010552.52-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v9 4/9] avcodec: add D3D12VA hardware accelerated VP9 decoding X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu , Wu Jianhua Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: EelzWggqAoam From: Wu Jianhua The command below is how to enable d3d12va: ffmpeg -hwaccel d3d12va -i input.mp4 output.mp4 Signed-off-by: Wu Jianhua Signed-off-by: Tong Wu --- configure | 2 + libavcodec/Makefile | 1 + libavcodec/d3d12va_vp9.c | 176 ++++++++++++++++++++++++++++++++++++ libavcodec/dxva2_internal.h | 2 + libavcodec/dxva2_vp9.c | 7 +- libavcodec/hwaccels.h | 1 + libavcodec/vp9.c | 7 ++ 7 files changed, 193 insertions(+), 3 deletions(-) create mode 100644 libavcodec/d3d12va_vp9.c diff --git a/configure b/configure index dc6ebd9768..99b6cdcb13 100755 --- a/configure +++ b/configure @@ -3161,6 +3161,8 @@ vp9_d3d11va_hwaccel_deps="d3d11va DXVA_PicParams_VP9" vp9_d3d11va_hwaccel_select="vp9_decoder" vp9_d3d11va2_hwaccel_deps="d3d11va DXVA_PicParams_VP9" vp9_d3d11va2_hwaccel_select="vp9_decoder" +vp9_d3d12va_hwaccel_deps="d3d12va DXVA_PicParams_VP9" +vp9_d3d12va_hwaccel_select="vp9_decoder" vp9_dxva2_hwaccel_deps="dxva2 DXVA_PicParams_VP9" vp9_dxva2_hwaccel_select="vp9_decoder" vp9_nvdec_hwaccel_deps="nvdec" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 7fdfc1801c..3d8df464ab 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1042,6 +1042,7 @@ OBJS-$(CONFIG_VP8_NVDEC_HWACCEL) += nvdec_vp8.o OBJS-$(CONFIG_VP8_VAAPI_HWACCEL) += vaapi_vp8.o OBJS-$(CONFIG_VP9_D3D11VA_HWACCEL) += dxva2_vp9.o OBJS-$(CONFIG_VP9_DXVA2_HWACCEL) += dxva2_vp9.o +OBJS-$(CONFIG_VP9_D3D12VA_HWACCEL) += dxva2_vp9.o d3d12va_vp9.o OBJS-$(CONFIG_VP9_NVDEC_HWACCEL) += nvdec_vp9.o OBJS-$(CONFIG_VP9_VAAPI_HWACCEL) += vaapi_vp9.o OBJS-$(CONFIG_VP9_VDPAU_HWACCEL) += vdpau_vp9.o diff --git a/libavcodec/d3d12va_vp9.c b/libavcodec/d3d12va_vp9.c new file mode 100644 index 0000000000..6ff0c5400d --- /dev/null +++ b/libavcodec/d3d12va_vp9.c @@ -0,0 +1,176 @@ +/* + * Direct3D 12 VP9 HW acceleration + * + * copyright (c) 2022-2023 Wu Jianhua + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "config_components.h" + +#include "libavutil/avassert.h" +#include "libavutil/pixdesc.h" +#include "libavutil/hwcontext_d3d12va_internal.h" + +#include "vp9shared.h" +#include "dxva2_internal.h" +#include "d3d12va_decode.h" + +typedef struct VP9DecodePictureContext { + DXVA_PicParams_VP9 pp; + DXVA_Slice_VPx_Short slice; + const uint8_t *bitstream; + unsigned bitstream_size; +} VP9DecodePictureContext; + +static void fill_slice_short(DXVA_Slice_VPx_Short *slice, unsigned position, unsigned size) +{ + memset(slice, 0, sizeof(*slice)); + slice->BSNALunitDataLocation = position; + slice->SliceBytesInBuffer = size; + slice->wBadSliceChopping = 0; +} + +static int d3d12va_vp9_start_frame(AVCodecContext *avctx, av_unused const uint8_t *buffer, av_unused uint32_t size) +{ + const VP9SharedContext *h = avctx->priv_data; + D3D12VADecodeContext *ctx = D3D12VA_DECODE_CONTEXT(avctx); + VP9DecodePictureContext *ctx_pic = h->frames[CUR_FRAME].hwaccel_picture_private; + + if (!ctx) + return -1; + + av_assert0(ctx_pic); + + if (ff_dxva2_vp9_fill_picture_parameters(avctx, (AVDXVAContext *)ctx, &ctx_pic->pp) < 0) + return -1; + + ctx_pic->bitstream_size = 0; + ctx_pic->bitstream = NULL; + + return 0; +} + +static int d3d12va_vp9_decode_slice(AVCodecContext *avctx, const uint8_t *buffer, uint32_t size) +{ + const VP9SharedContext *h = avctx->priv_data; + VP9DecodePictureContext *ctx_pic = h->frames[CUR_FRAME].hwaccel_picture_private; + unsigned position; + + if (!ctx_pic->bitstream) + ctx_pic->bitstream = buffer; + ctx_pic->bitstream_size += size; + + position = buffer - ctx_pic->bitstream; + fill_slice_short(&ctx_pic->slice, position, size); + + return 0; +} + +static int update_input_arguments(AVCodecContext *avctx, D3D12_VIDEO_DECODE_INPUT_STREAM_ARGUMENTS *input_args, ID3D12Resource *buffer) +{ + D3D12VADecodeContext *ctx = D3D12VA_DECODE_CONTEXT(avctx); + AVHWFramesContext *frames_ctx = D3D12VA_FRAMES_CONTEXT(avctx); + AVD3D12VAFramesContext *frames_hwctx = frames_ctx->hwctx; + + const VP9SharedContext *h = avctx->priv_data; + VP9DecodePictureContext *ctx_pic = h->frames[CUR_FRAME].hwaccel_picture_private; + + int index; + uint8_t *mapped_data; + D3D12_VIDEO_DECODE_FRAME_ARGUMENT *args; + + if (FAILED(ID3D12Resource_Map(buffer, 0, NULL, &mapped_data))) { + av_log(avctx, AV_LOG_ERROR, "Failed to map D3D12 Buffer resource!\n"); + return AVERROR(EINVAL); + } + + args = &input_args->FrameArguments[input_args->NumFrameArguments++]; + args->Type = D3D12_VIDEO_DECODE_ARGUMENT_TYPE_SLICE_CONTROL; + args->Size = sizeof(ctx_pic->slice); + args->pData = &ctx_pic->slice; + + memcpy(mapped_data, ctx_pic->bitstream, ctx_pic->slice.SliceBytesInBuffer); + + ID3D12Resource_Unmap(buffer, 0, NULL); + + input_args->CompressedBitstream = (D3D12_VIDEO_DECODE_COMPRESSED_BITSTREAM){ + .pBuffer = buffer, + .Offset = 0, + .Size = ctx_pic->slice.SliceBytesInBuffer, + }; + + index = ctx_pic->pp.CurrPic.Index7Bits; + ctx->ref_resources[index] = frames_hwctx->texture_infos[index].texture; + + for (int i = 0; i < FF_ARRAY_ELEMS(ctx_pic->pp.frame_refs); i++) { + index = ctx_pic->pp.frame_refs[i].Index7Bits; + if (index != 0x7f) + ctx->ref_resources[index] = frames_hwctx->texture_infos[index].texture; + } + + return 0; +} + +static int d3d12va_vp9_end_frame(AVCodecContext *avctx) +{ + VP9SharedContext *h = avctx->priv_data; + VP9DecodePictureContext *ctx_pic = h->frames[CUR_FRAME].hwaccel_picture_private; + + if (ctx_pic->bitstream_size <= 0) + return -1; + + return ff_d3d12va_common_end_frame(avctx, h->frames[CUR_FRAME].tf.f, + &ctx_pic->pp, sizeof(ctx_pic->pp), NULL, 0, update_input_arguments); +} + +static int d3d12va_vp9_decode_init(AVCodecContext *avctx) +{ + D3D12VADecodeContext *ctx = D3D12VA_DECODE_CONTEXT(avctx); + + switch (avctx->profile) { + case FF_PROFILE_VP9_2: + case FF_PROFILE_VP9_3: + ctx->cfg.DecodeProfile = D3D12_VIDEO_DECODE_PROFILE_VP9_10BIT_PROFILE2; + break; + + case FF_PROFILE_VP9_0: + case FF_PROFILE_VP9_1: + default: + ctx->cfg.DecodeProfile = D3D12_VIDEO_DECODE_PROFILE_VP9; + break; + }; + + return ff_d3d12va_decode_init(avctx); +} + +#if CONFIG_VP9_D3D12VA_HWACCEL +const FFHWAccel ff_vp9_d3d12va_hwaccel = { + .p.name = "vp9_d3d12va", + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_VP9, + .p.pix_fmt = AV_PIX_FMT_D3D12, + .init = d3d12va_vp9_decode_init, + .uninit = ff_d3d12va_decode_uninit, + .start_frame = d3d12va_vp9_start_frame, + .decode_slice = d3d12va_vp9_decode_slice, + .end_frame = d3d12va_vp9_end_frame, + .frame_params = ff_d3d12va_common_frame_params, + .frame_priv_data_size = sizeof(VP9DecodePictureContext), + .priv_data_size = sizeof(D3D12VADecodeContext), +}; +#endif diff --git a/libavcodec/dxva2_internal.h b/libavcodec/dxva2_internal.h index f06ea75c60..1c361929f1 100644 --- a/libavcodec/dxva2_internal.h +++ b/libavcodec/dxva2_internal.h @@ -170,4 +170,6 @@ void ff_dxva2_hevc_fill_picture_parameters(const AVCodecContext *avctx, AVDXVACo void ff_dxva2_hevc_fill_scaling_lists(const AVCodecContext *avctx, AVDXVAContext *ctx, DXVA_Qmatrix_HEVC *qm); +int ff_dxva2_vp9_fill_picture_parameters(const AVCodecContext *avctx, AVDXVAContext *ctx, DXVA_PicParams_VP9 *pp); + #endif /* AVCODEC_DXVA2_INTERNAL_H */ diff --git a/libavcodec/dxva2_vp9.c b/libavcodec/dxva2_vp9.c index eba4df9031..2fdb184cea 100644 --- a/libavcodec/dxva2_vp9.c +++ b/libavcodec/dxva2_vp9.c @@ -43,11 +43,12 @@ static void fill_picture_entry(DXVA_PicEntry_VPx *pic, pic->bPicEntry = index | (flag << 7); } -static int fill_picture_parameters(const AVCodecContext *avctx, AVDXVAContext *ctx, const VP9SharedContext *h, +int ff_dxva2_vp9_fill_picture_parameters(const AVCodecContext *avctx, AVDXVAContext *ctx, DXVA_PicParams_VP9 *pp) { + const VP9SharedContext *h = avctx->priv_data; + const AVPixFmtDescriptor *pixdesc = av_pix_fmt_desc_get(avctx->sw_pix_fmt); int i; - const AVPixFmtDescriptor * pixdesc = av_pix_fmt_desc_get(avctx->sw_pix_fmt); if (!pixdesc) return -1; @@ -265,7 +266,7 @@ static int dxva2_vp9_start_frame(AVCodecContext *avctx, av_assert0(ctx_pic); /* Fill up DXVA_PicParams_VP9 */ - if (fill_picture_parameters(avctx, ctx, h, &ctx_pic->pp) < 0) + if (ff_dxva2_vp9_fill_picture_parameters(avctx, ctx, &ctx_pic->pp) < 0) return -1; ctx_pic->bitstream_size = 0; diff --git a/libavcodec/hwaccels.h b/libavcodec/hwaccels.h index 6b71919b0a..1344b84326 100644 --- a/libavcodec/hwaccels.h +++ b/libavcodec/hwaccels.h @@ -73,6 +73,7 @@ extern const struct FFHWAccel ff_vp8_nvdec_hwaccel; extern const struct FFHWAccel ff_vp8_vaapi_hwaccel; extern const struct FFHWAccel ff_vp9_d3d11va_hwaccel; extern const struct FFHWAccel ff_vp9_d3d11va2_hwaccel; +extern const struct FFHWAccel ff_vp9_d3d12va_hwaccel; extern const struct FFHWAccel ff_vp9_dxva2_hwaccel; extern const struct FFHWAccel ff_vp9_nvdec_hwaccel; extern const struct FFHWAccel ff_vp9_vaapi_hwaccel; diff --git a/libavcodec/vp9.c b/libavcodec/vp9.c index 75e2e1d329..855936cdc1 100644 --- a/libavcodec/vp9.c +++ b/libavcodec/vp9.c @@ -170,6 +170,7 @@ static int update_size(AVCodecContext *avctx, int w, int h) { #define HWACCEL_MAX (CONFIG_VP9_DXVA2_HWACCEL + \ CONFIG_VP9_D3D11VA_HWACCEL * 2 + \ + CONFIG_VP9_D3D12VA_HWACCEL + \ CONFIG_VP9_NVDEC_HWACCEL + \ CONFIG_VP9_VAAPI_HWACCEL + \ CONFIG_VP9_VDPAU_HWACCEL + \ @@ -196,6 +197,9 @@ static int update_size(AVCodecContext *avctx, int w, int h) *fmtp++ = AV_PIX_FMT_D3D11VA_VLD; *fmtp++ = AV_PIX_FMT_D3D11; #endif +#if CONFIG_VP9_D3D12VA_HWACCEL + *fmtp++ = AV_PIX_FMT_D3D12; +#endif #if CONFIG_VP9_NVDEC_HWACCEL *fmtp++ = AV_PIX_FMT_CUDA; #endif @@ -1899,6 +1903,9 @@ const FFCodec ff_vp9_decoder = { #if CONFIG_VP9_D3D11VA2_HWACCEL HWACCEL_D3D11VA2(vp9), #endif +#if CONFIG_VP9_D3D12VA_HWACCEL + HWACCEL_D3D12VA(vp9), +#endif #if CONFIG_VP9_NVDEC_HWACCEL HWACCEL_NVDEC(vp9), #endif