From patchwork Fri Jun 2 08:06:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 41952 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c51c:b0:10c:5e6f:955f with SMTP id gm28csp1055255pzb; Fri, 2 Jun 2023 01:12:17 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ60XH3d1tAnbb+LtFtSe+e8lME6TctI9s066GOpGxXvsJaF9Fi5vPMJMN+lZ2jRt9HlJD5f X-Received: by 2002:a17:907:940e:b0:974:4a33:83a7 with SMTP id dk14-20020a170907940e00b009744a3383a7mr4026010ejc.12.1685693537331; Fri, 02 Jun 2023 01:12:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685693537; cv=none; d=google.com; s=arc-20160816; b=yIK5dvAHz0aBBiRm4RUBvzfWO+YvAnFjcPbEEakoQc2Q8s440Z3BajMNpHYmNoCdxH NeSqtItuUX4EYdiBWrsHFd2c2VMZ3uwUJZk7eXYwMOLeWB6jE/wfkk6h39U4bkaZn6y+ fPd2z46geOIYxQ5L4ZjlRVpyBC5W0riKLd4iBDUaCi/Mfy6FPMXQ+T+P+4VlEnpMKttv w5HMgp27ta66S2h/bksn0MGwXitLQm1Lrn31F7kCWKz5rfVtOtI5h3c/okSiLQL24Hz9 pf2BCeJ6HGcM4j3hBouyJ2fj7q25hIFCdkhIVpf9Xl/HTGy5zBhZSW70H1lpsDgxJm7p Y5wQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=vnzV4vsqKnhFMGXbIP1sdTpobHCQBZUmv3P8FeWZeUw=; b=lYBY6guCngdDMg8X1bTqthDm6Z3HklJtywxH30Dy8zbhj3OcVGP+KNO7P19Uvhroh5 +VlPSL8ntMTDR9fOwQ6Wwen4B57mX6sLAiFRzGw/P7sZ3QNWn2Fg4N+gfMsuvZ9CX5Mg cC8ln6xbyYqRrGGd6vvLI9a5+L2IEH6d97Zc0XV10FSbUPluhNB9cDgrmujRKLNevexF y0In7Sxo9hMgOrSir8FLQcAnIYO7BsSKy1BzyLfkYowBarlVnyYJ2ngx9ONwy9V6j341 LjxWoswy7rx8U/jL08Re9LNXJCwkfGsoSuSpuN72QNKHM1ABf0cXJmH3MQaJI5QeKWRU /gUg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=JD8L8tbq; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x8-20020a170906148800b0094edce87118si430275ejc.1042.2023.06.02.01.12.16; Fri, 02 Jun 2023 01:12:17 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=JD8L8tbq; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 93F1668C310; Fri, 2 Jun 2023 11:11:46 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B3C3068C30E for ; Fri, 2 Jun 2023 11:11:37 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1685693502; x=1717229502; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=B5Zc4VkJiA/22MQ5gqr6Vqoh7Z6WG9TIUOQGH/unK4w=; b=JD8L8tbq70pR8b2oOtd8ryI0yNo7ttRazIErsESyBYT71QB5n6D3XE6L D6q2vUF5hI4DT0yT6dkRDOguWl0UgHllqIWrdUnW6mQp/xAoJaCXjPlcZ QbAqLs7DCqu65a1yH0PkEBIISriGdBkUhMovq1PfZsz0nG6UqqXgNqMIH KTSoZMyNhKnBfftCPtf1pAG0wValp2RlcxMoX2lm3B8ZP17qrEG5i7DNX CJ/Zv70cdQNgCnPj7Aiyki2CnzJy6164ZoVAs4gQB899K/mUmDiOnYyRi gBH+q90OZFl04N/RX61cYBLP+NKOC2/zAcXN+bHDNAjLUSqrh3o2UBm9C A==; X-IronPort-AV: E=McAfee;i="6600,9927,10728"; a="421629870" X-IronPort-AV: E=Sophos;i="6.00,212,1681196400"; d="scan'208";a="421629870" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jun 2023 01:11:23 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10728"; a="852060674" X-IronPort-AV: E=Sophos;i="6.00,212,1681196400"; d="scan'208";a="852060674" Received: from desktop-qn7n0nf.sh.intel.com (HELO localhost.localdomain) ([10.239.160.59]) by fmsmga001.fm.intel.com with ESMTP; 02 Jun 2023 01:11:22 -0700 From: Tong Wu To: ffmpeg-devel@ffmpeg.org Date: Fri, 2 Jun 2023 16:06:56 +0800 Message-Id: <20230602080701.1754-4-tong1.wu@intel.com> X-Mailer: git-send-email 2.35.1.windows.2 In-Reply-To: <20230602080701.1754-1-tong1.wu@intel.com> References: <20230602080701.1754-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 4/9] avcodec: add D3D12VA hardware accelerated VP9 decoding X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu , Wu Jianhua Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: dz+6fMHd8vYI From: Wu Jianhua The command below is how to enable d3d12va: ffmpeg -hwaccel d3d12va -i input.mp4 output.mp4 Signed-off-by: Wu Jianhua Signed-off-by: Tong Wu --- configure | 2 + libavcodec/Makefile | 1 + libavcodec/d3d12va_vp9.c | 176 ++++++++++++++++++++++++++++++++++++ libavcodec/dxva2_internal.h | 2 + libavcodec/dxva2_vp9.c | 7 +- libavcodec/hwaccels.h | 1 + libavcodec/vp9.c | 7 ++ 7 files changed, 193 insertions(+), 3 deletions(-) create mode 100644 libavcodec/d3d12va_vp9.c diff --git a/configure b/configure index 3d25e5fdea..fa2548f3e2 100755 --- a/configure +++ b/configure @@ -3119,6 +3119,8 @@ vp9_d3d11va_hwaccel_deps="d3d11va DXVA_PicParams_VP9" vp9_d3d11va_hwaccel_select="vp9_decoder" vp9_d3d11va2_hwaccel_deps="d3d11va DXVA_PicParams_VP9" vp9_d3d11va2_hwaccel_select="vp9_decoder" +vp9_d3d12va_hwaccel_deps="d3d12va DXVA_PicParams_VP9" +vp9_d3d12va_hwaccel_select="vp9_decoder" vp9_dxva2_hwaccel_deps="dxva2 DXVA_PicParams_VP9" vp9_dxva2_hwaccel_select="vp9_decoder" vp9_nvdec_hwaccel_deps="nvdec" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 6cc28f2fd0..d5a1bfef7a 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1036,6 +1036,7 @@ OBJS-$(CONFIG_VP8_NVDEC_HWACCEL) += nvdec_vp8.o OBJS-$(CONFIG_VP8_VAAPI_HWACCEL) += vaapi_vp8.o OBJS-$(CONFIG_VP9_D3D11VA_HWACCEL) += dxva2_vp9.o OBJS-$(CONFIG_VP9_DXVA2_HWACCEL) += dxva2_vp9.o +OBJS-$(CONFIG_VP9_D3D12VA_HWACCEL) += dxva2_vp9.o d3d12va_vp9.o OBJS-$(CONFIG_VP9_NVDEC_HWACCEL) += nvdec_vp9.o OBJS-$(CONFIG_VP9_VAAPI_HWACCEL) += vaapi_vp9.o OBJS-$(CONFIG_VP9_VDPAU_HWACCEL) += vdpau_vp9.o diff --git a/libavcodec/d3d12va_vp9.c b/libavcodec/d3d12va_vp9.c new file mode 100644 index 0000000000..dc1c461f5c --- /dev/null +++ b/libavcodec/d3d12va_vp9.c @@ -0,0 +1,176 @@ +/* + * Direct3D 12 VP9 HW acceleration + * + * copyright (c) 2022-2023 Wu Jianhua + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "config_components.h" + +#include "libavutil/avassert.h" +#include "libavutil/pixdesc.h" +#include "libavutil/hwcontext_d3d12va_internal.h" + +#include "vp9shared.h" +#include "dxva2_internal.h" +#include "d3d12va.h" + +typedef struct VP9DecodePictureContext { + DXVA_PicParams_VP9 pp; + DXVA_Slice_VPx_Short slice; + const uint8_t *bitstream; + unsigned bitstream_size; +} VP9DecodePictureContext; + +static void fill_slice_short(DXVA_Slice_VPx_Short *slice, unsigned position, unsigned size) +{ + memset(slice, 0, sizeof(*slice)); + slice->BSNALunitDataLocation = position; + slice->SliceBytesInBuffer = size; + slice->wBadSliceChopping = 0; +} + +static int d3d12va_vp9_start_frame(AVCodecContext *avctx, av_unused const uint8_t *buffer, av_unused uint32_t size) +{ + const VP9SharedContext *h = avctx->priv_data; + D3D12VADecodeContext *ctx = D3D12VA_DECODE_CONTEXT(avctx); + VP9DecodePictureContext *ctx_pic = h->frames[CUR_FRAME].hwaccel_picture_private; + + if (!ctx) + return -1; + + av_assert0(ctx_pic); + + if (ff_dxva2_vp9_fill_picture_parameters(avctx, (AVDXVAContext *)ctx, &ctx_pic->pp) < 0) + return -1; + + ctx_pic->bitstream_size = 0; + ctx_pic->bitstream = NULL; + + return 0; +} + +static int d3d12va_vp9_decode_slice(AVCodecContext *avctx, const uint8_t *buffer, uint32_t size) +{ + const VP9SharedContext *h = avctx->priv_data; + VP9DecodePictureContext *ctx_pic = h->frames[CUR_FRAME].hwaccel_picture_private; + unsigned position; + + if (!ctx_pic->bitstream) + ctx_pic->bitstream = buffer; + ctx_pic->bitstream_size += size; + + position = buffer - ctx_pic->bitstream; + fill_slice_short(&ctx_pic->slice, position, size); + + return 0; +} + +static int update_input_arguments(AVCodecContext *avctx, D3D12_VIDEO_DECODE_INPUT_STREAM_ARGUMENTS *input_args, ID3D12Resource *buffer) +{ + D3D12VADecodeContext *ctx = D3D12VA_DECODE_CONTEXT(avctx); + AVHWFramesContext *frames_ctx = D3D12VA_FRAMES_CONTEXT(avctx); + AVD3D12VAFramesContext *frames_hwctx = frames_ctx->hwctx; + + const VP9SharedContext *h = avctx->priv_data; + VP9DecodePictureContext *ctx_pic = h->frames[CUR_FRAME].hwaccel_picture_private; + + int index; + uint8_t *mapped_data; + D3D12_VIDEO_DECODE_FRAME_ARGUMENT *args; + + if (FAILED(ID3D12Resource_Map(buffer, 0, NULL, &mapped_data))) { + av_log(avctx, AV_LOG_ERROR, "Failed to map D3D12 Buffer resource!\n"); + return AVERROR(EINVAL); + } + + args = &input_args->FrameArguments[input_args->NumFrameArguments++]; + args->Type = D3D12_VIDEO_DECODE_ARGUMENT_TYPE_SLICE_CONTROL; + args->Size = sizeof(ctx_pic->slice); + args->pData = &ctx_pic->slice; + + memcpy(mapped_data, ctx_pic->bitstream, ctx_pic->slice.SliceBytesInBuffer); + + ID3D12Resource_Unmap(buffer, 0, NULL); + + input_args->CompressedBitstream = (D3D12_VIDEO_DECODE_COMPRESSED_BITSTREAM){ + .pBuffer = buffer, + .Offset = 0, + .Size = ctx_pic->slice.SliceBytesInBuffer, + }; + + index = ctx_pic->pp.CurrPic.Index7Bits; + ctx->ref_resources[index] = frames_hwctx->texture_infos[index].texture; + + for (int i = 0; i < FF_ARRAY_ELEMS(ctx_pic->pp.frame_refs); i++) { + index = ctx_pic->pp.frame_refs[i].Index7Bits; + if (index != 0x7f) + ctx->ref_resources[index] = frames_hwctx->texture_infos[index].texture; + } + + return 0; +} + +static int d3d12va_vp9_end_frame(AVCodecContext *avctx) +{ + VP9SharedContext *h = avctx->priv_data; + VP9DecodePictureContext *ctx_pic = h->frames[CUR_FRAME].hwaccel_picture_private; + + if (ctx_pic->bitstream_size <= 0) + return -1; + + return ff_d3d12va_common_end_frame(avctx, h->frames[CUR_FRAME].tf.f, + &ctx_pic->pp, sizeof(ctx_pic->pp), NULL, 0, update_input_arguments); +} + +static int d3d12va_vp9_decode_init(AVCodecContext *avctx) +{ + D3D12VADecodeContext *ctx = D3D12VA_DECODE_CONTEXT(avctx); + + switch (avctx->profile) { + case FF_PROFILE_VP9_2: + case FF_PROFILE_VP9_3: + ctx->cfg.DecodeProfile = D3D12_VIDEO_DECODE_PROFILE_VP9_10BIT_PROFILE2; + break; + + case FF_PROFILE_VP9_0: + case FF_PROFILE_VP9_1: + default: + ctx->cfg.DecodeProfile = D3D12_VIDEO_DECODE_PROFILE_VP9; + break; + }; + + return ff_d3d12va_decode_init(avctx); +} + +#if CONFIG_VP9_D3D12VA_HWACCEL +const AVHWAccel ff_vp9_d3d12va_hwaccel = { + .name = "vp9_d3d12va", + .type = AVMEDIA_TYPE_VIDEO, + .id = AV_CODEC_ID_VP9, + .pix_fmt = AV_PIX_FMT_D3D12, + .init = d3d12va_vp9_decode_init, + .uninit = ff_d3d12va_decode_uninit, + .start_frame = d3d12va_vp9_start_frame, + .decode_slice = d3d12va_vp9_decode_slice, + .end_frame = d3d12va_vp9_end_frame, + .frame_params = ff_d3d12va_common_frame_params, + .frame_priv_data_size = sizeof(VP9DecodePictureContext), + .priv_data_size = sizeof(D3D12VADecodeContext), +}; +#endif diff --git a/libavcodec/dxva2_internal.h b/libavcodec/dxva2_internal.h index 08847aef22..1de749f83b 100644 --- a/libavcodec/dxva2_internal.h +++ b/libavcodec/dxva2_internal.h @@ -170,4 +170,6 @@ void ff_dxva2_hevc_fill_picture_parameters(const AVCodecContext *avctx, AVDXVACo void ff_dxva2_hevc_fill_scaling_lists(const AVCodecContext *avctx, AVDXVAContext *ctx, DXVA_Qmatrix_HEVC *qm); +int ff_dxva2_vp9_fill_picture_parameters(const AVCodecContext *avctx, AVDXVAContext *ctx, DXVA_PicParams_VP9 *pp); + #endif /* AVCODEC_DXVA2_INTERNAL_H */ diff --git a/libavcodec/dxva2_vp9.c b/libavcodec/dxva2_vp9.c index dbe6c08ad1..480d734acd 100644 --- a/libavcodec/dxva2_vp9.c +++ b/libavcodec/dxva2_vp9.c @@ -42,11 +42,12 @@ static void fill_picture_entry(DXVA_PicEntry_VPx *pic, pic->bPicEntry = index | (flag << 7); } -static int fill_picture_parameters(const AVCodecContext *avctx, AVDXVAContext *ctx, const VP9SharedContext *h, +int ff_dxva2_vp9_fill_picture_parameters(const AVCodecContext *avctx, AVDXVAContext *ctx, DXVA_PicParams_VP9 *pp) { + const VP9SharedContext *h = avctx->priv_data; + const AVPixFmtDescriptor *pixdesc = av_pix_fmt_desc_get(avctx->sw_pix_fmt); int i; - const AVPixFmtDescriptor * pixdesc = av_pix_fmt_desc_get(avctx->sw_pix_fmt); if (!pixdesc) return -1; @@ -264,7 +265,7 @@ static int dxva2_vp9_start_frame(AVCodecContext *avctx, av_assert0(ctx_pic); /* Fill up DXVA_PicParams_VP9 */ - if (fill_picture_parameters(avctx, ctx, h, &ctx_pic->pp) < 0) + if (ff_dxva2_vp9_fill_picture_parameters(avctx, ctx, &ctx_pic->pp) < 0) return -1; ctx_pic->bitstream_size = 0; diff --git a/libavcodec/hwaccels.h b/libavcodec/hwaccels.h index 70e115f78a..1059b6db70 100644 --- a/libavcodec/hwaccels.h +++ b/libavcodec/hwaccels.h @@ -75,6 +75,7 @@ extern const AVHWAccel ff_vp8_nvdec_hwaccel; extern const AVHWAccel ff_vp8_vaapi_hwaccel; extern const AVHWAccel ff_vp9_d3d11va_hwaccel; extern const AVHWAccel ff_vp9_d3d11va2_hwaccel; +extern const AVHWAccel ff_vp9_d3d12va_hwaccel; extern const AVHWAccel ff_vp9_dxva2_hwaccel; extern const AVHWAccel ff_vp9_nvdec_hwaccel; extern const AVHWAccel ff_vp9_vaapi_hwaccel; diff --git a/libavcodec/vp9.c b/libavcodec/vp9.c index 4f704ec0dd..9a34cd331e 100644 --- a/libavcodec/vp9.c +++ b/libavcodec/vp9.c @@ -184,6 +184,7 @@ static int update_size(AVCodecContext *avctx, int w, int h) { #define HWACCEL_MAX (CONFIG_VP9_DXVA2_HWACCEL + \ CONFIG_VP9_D3D11VA_HWACCEL * 2 + \ + CONFIG_VP9_D3D12VA_HWACCEL + \ CONFIG_VP9_NVDEC_HWACCEL + \ CONFIG_VP9_VAAPI_HWACCEL + \ CONFIG_VP9_VDPAU_HWACCEL + \ @@ -210,6 +211,9 @@ static int update_size(AVCodecContext *avctx, int w, int h) *fmtp++ = AV_PIX_FMT_D3D11VA_VLD; *fmtp++ = AV_PIX_FMT_D3D11; #endif +#if CONFIG_VP9_D3D12VA_HWACCEL + *fmtp++ = AV_PIX_FMT_D3D12; +#endif #if CONFIG_VP9_NVDEC_HWACCEL *fmtp++ = AV_PIX_FMT_CUDA; #endif @@ -1910,6 +1914,9 @@ const FFCodec ff_vp9_decoder = { #if CONFIG_VP9_D3D11VA2_HWACCEL HWACCEL_D3D11VA2(vp9), #endif +#if CONFIG_VP9_D3D12VA_HWACCEL + HWACCEL_D3D12VA(vp9), +#endif #if CONFIG_VP9_NVDEC_HWACCEL HWACCEL_NVDEC(vp9), #endif