From patchwork Tue Dec 5 06:46:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 44907 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9153:b0:181:818d:5e7f with SMTP id x19csp145774pzc; Mon, 4 Dec 2023 22:52:10 -0800 (PST) X-Google-Smtp-Source: AGHT+IEZfHbdArpR557vvCtkVEgq4fIzyJOzcOSLkqdltPJZvqIFDRrUDn/+QH+UfMOfvMp1RUh6 X-Received: by 2002:a50:cd97:0:b0:54c:81f4:2526 with SMTP id p23-20020a50cd97000000b0054c81f42526mr2336377edi.36.1701759130536; Mon, 04 Dec 2023 22:52:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701759130; cv=none; d=google.com; s=arc-20160816; b=HbByfEA+uticUm3ScFUBWJ7rCenxWDDwUSQtHl6zb7p5X7AEmgX44wNNJho0ogBQ3i jtBfnYjxksVozzlbNRohHo/Jp2PFzpLRT1Pe4ygNSY7WwnL33pMUScxt+xmgaQE1pGvn BUbR/QpGIaIVZyMWudL4LC1yCYmXbT3kkiSHyQncZD+3GIV37vkPZUTdDRO3qYJRrcfe chjIRBoLz4reeKWSWf5UQnE1H8QxDvD+5+yQ+dAQ2ss9Hn9g86+cuFRN0dNrchGnVorw lFOPJk9X8NM11pdsYisbgTBR6JNBtoupUGr01jvpxrV5sclCwb/WwvjI6YIlWBmT2KRG t+Zw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=6joVLQGpRh0Ed2zFfE8Q+qg4c5zOco+jGfovAkf/yDo=; fh=9GN8hc/WTnabikg2VxPLeDN+i3R37IqYi/LhxNANmV0=; b=tFItJorIp+3Z8+E/sJauMM+y5HxpJHzhoXg+XNnyFD4OlHnqzjA+Tz06lVeoOaTm9x ctn7Q8zc1DdZxrYIagubiAfZHl7Z1HUk9YyExz3Y1GXeXlFTRs1RPL5kCj2JmiXIl3SK K08cU+ggFcGMpEyeF+CNvQvgYHlfLjXkiswHqD6Sl2tid32rpHEIJYG4dyZrvpWgGuwV 2oBfUD+OFvHBdu4XSc1r4FcOUhUwAcdWX0/1IdnMHagjR6KS4GUzPXJyYYLWbO5gwZon Eq19b/UMn8S5t1XJlY7F8YiwFroicq1dkc9MkkwV6Lz0mp7FZONq8mhJOZRx48LQmRJg mpUQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=T+UPEYf6; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id u26-20020a50d51a000000b00542ecfc2aa2si595356edi.360.2023.12.04.22.52.10; Mon, 04 Dec 2023 22:52:10 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=T+UPEYf6; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6AEAF68CEF1; Tue, 5 Dec 2023 08:51:39 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.93]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1296768CE3C for ; Tue, 5 Dec 2023 08:51:31 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1701759097; x=1733295097; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=q4ZUTtmCq2zl8Yf43gnsiQclB3ZDhOc/ToEvf2STVkE=; b=T+UPEYf6IkRNmtRWZ4LD5E6u8WoDFrL0uLtW1pDnKauK0ZvFheNJrno2 VrsFveC47igiBeWtP9g1QBlFFykZESLRgfQD4z5iXB0TJc8mtULCP2U2y xviLZ6/3bkNta3scSWhP3n2GVSJbuaEESYZRC2htOXOS/G/3AfLOMCScl R/Ef70mnByosGbnHEPsfzp3tjNpZO0HZOmdYuIWYyeKM4MgZDYyDD8tie dkk2qtn9nbY96qZQpILt0o4nFNPNWH53LZuhmgQoYXmT7+Bru9eefNlG4 TC1uEI09y4dvfADz5rrwMjGDlV0YLzZO61pwlY1eEwOeaJlIcSY1dwIxt g==; X-IronPort-AV: E=McAfee;i="6600,9927,10914"; a="391012387" X-IronPort-AV: E=Sophos;i="6.04,251,1695711600"; d="scan'208";a="391012387" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Dec 2023 22:51:25 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10914"; a="799853605" X-IronPort-AV: E=Sophos;i="6.04,251,1695711600"; d="scan'208";a="799853605" Received: from desktop-qn7n0nf.sh.intel.com (HELO localhost.localdomain) ([10.239.160.66]) by orsmga008.jf.intel.com with ESMTP; 04 Dec 2023 22:51:24 -0800 From: Tong Wu To: ffmpeg-devel@ffmpeg.org Date: Tue, 5 Dec 2023 14:46:46 +0800 Message-ID: <20231205065034.1621-4-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20231205065034.1621-1-tong1.wu@intel.com> References: <20231205065034.1621-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v12 4/9] avcodec: add D3D12VA hardware accelerated VP9 decoding X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu , Wu Jianhua Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: XYK8VOFq2+/g From: Wu Jianhua The command below is how to enable d3d12va: ffmpeg -hwaccel d3d12va -i input.mp4 output.mp4 Signed-off-by: Wu Jianhua Signed-off-by: Tong Wu --- configure | 2 + libavcodec/Makefile | 1 + libavcodec/d3d12va_vp9.c | 171 ++++++++++++++++++++++++++++++++++++ libavcodec/dxva2_internal.h | 2 + libavcodec/dxva2_vp9.c | 11 +-- libavcodec/hwaccels.h | 1 + libavcodec/vp9.c | 7 ++ 7 files changed, 190 insertions(+), 5 deletions(-) create mode 100644 libavcodec/d3d12va_vp9.c diff --git a/configure b/configure index f1a9406123..cb4d74a321 100755 --- a/configure +++ b/configure @@ -3168,6 +3168,8 @@ vp9_d3d11va_hwaccel_deps="d3d11va DXVA_PicParams_VP9" vp9_d3d11va_hwaccel_select="vp9_decoder" vp9_d3d11va2_hwaccel_deps="d3d11va DXVA_PicParams_VP9" vp9_d3d11va2_hwaccel_select="vp9_decoder" +vp9_d3d12va_hwaccel_deps="d3d12va DXVA_PicParams_VP9" +vp9_d3d12va_hwaccel_select="vp9_decoder" vp9_dxva2_hwaccel_deps="dxva2 DXVA_PicParams_VP9" vp9_dxva2_hwaccel_select="vp9_decoder" vp9_nvdec_hwaccel_deps="nvdec" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 66284b3420..dd193115d4 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1045,6 +1045,7 @@ OBJS-$(CONFIG_VP8_NVDEC_HWACCEL) += nvdec_vp8.o OBJS-$(CONFIG_VP8_VAAPI_HWACCEL) += vaapi_vp8.o OBJS-$(CONFIG_VP9_D3D11VA_HWACCEL) += dxva2_vp9.o OBJS-$(CONFIG_VP9_DXVA2_HWACCEL) += dxva2_vp9.o +OBJS-$(CONFIG_VP9_D3D12VA_HWACCEL) += dxva2_vp9.o d3d12va_vp9.o OBJS-$(CONFIG_VP9_NVDEC_HWACCEL) += nvdec_vp9.o OBJS-$(CONFIG_VP9_VAAPI_HWACCEL) += vaapi_vp9.o OBJS-$(CONFIG_VP9_VDPAU_HWACCEL) += vdpau_vp9.o diff --git a/libavcodec/d3d12va_vp9.c b/libavcodec/d3d12va_vp9.c new file mode 100644 index 0000000000..bb94e18781 --- /dev/null +++ b/libavcodec/d3d12va_vp9.c @@ -0,0 +1,171 @@ +/* + * Direct3D 12 VP9 HW acceleration + * + * copyright (c) 2022-2023 Wu Jianhua + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "config_components.h" + +#include "libavutil/avassert.h" +#include "libavutil/pixdesc.h" +#include "libavutil/hwcontext_d3d12va_internal.h" + +#include "vp9shared.h" +#include "dxva2_internal.h" +#include "d3d12va_decode.h" + +typedef struct VP9DecodePictureContext { + DXVA_PicParams_VP9 pp; + DXVA_Slice_VPx_Short slice; + const uint8_t *bitstream; + unsigned bitstream_size; +} VP9DecodePictureContext; + +static void fill_slice_short(DXVA_Slice_VPx_Short *slice, unsigned position, unsigned size) +{ + memset(slice, 0, sizeof(*slice)); + slice->BSNALunitDataLocation = position; + slice->SliceBytesInBuffer = size; + slice->wBadSliceChopping = 0; +} + +static int d3d12va_vp9_start_frame(AVCodecContext *avctx, av_unused const uint8_t *buffer, av_unused uint32_t size) +{ + const VP9SharedContext *h = avctx->priv_data; + D3D12VADecodeContext *ctx = D3D12VA_DECODE_CONTEXT(avctx); + VP9DecodePictureContext *ctx_pic = h->frames[CUR_FRAME].hwaccel_picture_private; + + if (!ctx) + return -1; + + av_assert0(ctx_pic); + + ctx->used_mask = 0; + + if (ff_dxva2_vp9_fill_picture_parameters(avctx, (AVDXVAContext *)ctx, &ctx_pic->pp) < 0) + return -1; + + ctx_pic->bitstream_size = 0; + ctx_pic->bitstream = NULL; + + return 0; +} + +static int d3d12va_vp9_decode_slice(AVCodecContext *avctx, const uint8_t *buffer, uint32_t size) +{ + const VP9SharedContext *h = avctx->priv_data; + VP9DecodePictureContext *ctx_pic = h->frames[CUR_FRAME].hwaccel_picture_private; + unsigned position; + + if (!ctx_pic->bitstream) + ctx_pic->bitstream = buffer; + ctx_pic->bitstream_size += size; + + position = buffer - ctx_pic->bitstream; + fill_slice_short(&ctx_pic->slice, position, size); + + return 0; +} + +static int update_input_arguments(AVCodecContext *avctx, D3D12_VIDEO_DECODE_INPUT_STREAM_ARGUMENTS *input_args, ID3D12Resource *buffer) +{ + D3D12VADecodeContext *ctx = D3D12VA_DECODE_CONTEXT(avctx); + AVHWFramesContext *frames_ctx = D3D12VA_FRAMES_CONTEXT(avctx); + AVD3D12VAFramesContext *frames_hwctx = frames_ctx->hwctx; + + const VP9SharedContext *h = avctx->priv_data; + VP9DecodePictureContext *ctx_pic = h->frames[CUR_FRAME].hwaccel_picture_private; + + uint8_t *mapped_data; + D3D12_VIDEO_DECODE_FRAME_ARGUMENT *args; + + if (FAILED(ID3D12Resource_Map(buffer, 0, NULL, &mapped_data))) { + av_log(avctx, AV_LOG_ERROR, "Failed to map D3D12 Buffer resource!\n"); + return AVERROR(EINVAL); + } + + args = &input_args->FrameArguments[input_args->NumFrameArguments++]; + args->Type = D3D12_VIDEO_DECODE_ARGUMENT_TYPE_SLICE_CONTROL; + args->Size = sizeof(ctx_pic->slice); + args->pData = &ctx_pic->slice; + + memcpy(mapped_data, ctx_pic->bitstream, ctx_pic->slice.SliceBytesInBuffer); + + ID3D12Resource_Unmap(buffer, 0, NULL); + + input_args->CompressedBitstream = (D3D12_VIDEO_DECODE_COMPRESSED_BITSTREAM){ + .pBuffer = buffer, + .Offset = 0, + .Size = ctx_pic->slice.SliceBytesInBuffer, + }; + + return 0; +} + +static int d3d12va_vp9_end_frame(AVCodecContext *avctx) +{ + VP9SharedContext *h = avctx->priv_data; + VP9DecodePictureContext *ctx_pic = h->frames[CUR_FRAME].hwaccel_picture_private; + + if (ctx_pic->bitstream_size <= 0) + return -1; + + return ff_d3d12va_common_end_frame(avctx, h->frames[CUR_FRAME].tf.f, + &ctx_pic->pp, sizeof(ctx_pic->pp), NULL, 0, update_input_arguments); +} + +static int d3d12va_vp9_decode_init(AVCodecContext *avctx) +{ + D3D12VADecodeContext *ctx = D3D12VA_DECODE_CONTEXT(avctx); + DXVA_PicParams_VP9 pp; + + switch (avctx->profile) { + case FF_PROFILE_VP9_2: + case FF_PROFILE_VP9_3: + ctx->cfg.DecodeProfile = D3D12_VIDEO_DECODE_PROFILE_VP9_10BIT_PROFILE2; + break; + + case FF_PROFILE_VP9_0: + case FF_PROFILE_VP9_1: + default: + ctx->cfg.DecodeProfile = D3D12_VIDEO_DECODE_PROFILE_VP9; + break; + }; + + ctx->max_num_ref = FF_ARRAY_ELEMS(pp.frame_refs) + 1; + + return ff_d3d12va_decode_init(avctx); +} + +#if CONFIG_VP9_D3D12VA_HWACCEL +const FFHWAccel ff_vp9_d3d12va_hwaccel = { + .p.name = "vp9_d3d12va", + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_VP9, + .p.pix_fmt = AV_PIX_FMT_D3D12, + .init = d3d12va_vp9_decode_init, + .uninit = ff_d3d12va_decode_uninit, + .start_frame = d3d12va_vp9_start_frame, + .decode_slice = d3d12va_vp9_decode_slice, + .end_frame = d3d12va_vp9_end_frame, + .frame_params = ff_d3d12va_common_frame_params, + .frame_priv_data_size = sizeof(VP9DecodePictureContext), + .priv_data_size = sizeof(D3D12VADecodeContext), +}; +#endif diff --git a/libavcodec/dxva2_internal.h b/libavcodec/dxva2_internal.h index 49c696068a..7c712b31f3 100644 --- a/libavcodec/dxva2_internal.h +++ b/libavcodec/dxva2_internal.h @@ -175,4 +175,6 @@ void ff_dxva2_hevc_fill_picture_parameters(const AVCodecContext *avctx, AVDXVACo void ff_dxva2_hevc_fill_scaling_lists(const AVCodecContext *avctx, AVDXVAContext *ctx, DXVA_Qmatrix_HEVC *qm); +int ff_dxva2_vp9_fill_picture_parameters(const AVCodecContext *avctx, AVDXVAContext *ctx, DXVA_PicParams_VP9 *pp); + #endif /* AVCODEC_DXVA2_INTERNAL_H */ diff --git a/libavcodec/dxva2_vp9.c b/libavcodec/dxva2_vp9.c index 21699eb3f4..1498deb3c8 100644 --- a/libavcodec/dxva2_vp9.c +++ b/libavcodec/dxva2_vp9.c @@ -43,19 +43,18 @@ static void fill_picture_entry(DXVA_PicEntry_VPx *pic, pic->bPicEntry = index | (flag << 7); } -static int fill_picture_parameters(const AVCodecContext *avctx, AVDXVAContext *ctx, const VP9SharedContext *h, +int ff_dxva2_vp9_fill_picture_parameters(const AVCodecContext *avctx, AVDXVAContext *ctx, DXVA_PicParams_VP9 *pp) { + const VP9SharedContext *h = avctx->priv_data; + const AVPixFmtDescriptor *pixdesc = av_pix_fmt_desc_get(avctx->sw_pix_fmt); int i; - const AVPixFmtDescriptor * pixdesc = av_pix_fmt_desc_get(avctx->sw_pix_fmt); if (!pixdesc) return -1; memset(pp, 0, sizeof(*pp)); - fill_picture_entry(&pp->CurrPic, ff_dxva2_get_surface_index(avctx, ctx, h->frames[CUR_FRAME].tf.f, 1), 0); - pp->profile = h->h.profile; pp->wFormatAndPictureInfoFlags = ((h->h.keyframe == 0) << 0) | ((h->h.invisible == 0) << 1) | @@ -98,6 +97,8 @@ static int fill_picture_parameters(const AVCodecContext *avctx, AVDXVAContext *c pp->ref_frame_sign_bias[i + 1] = h->h.signbias[i]; } + fill_picture_entry(&pp->CurrPic, ff_dxva2_get_surface_index(avctx, ctx, h->frames[CUR_FRAME].tf.f, 1), 0); + pp->filter_level = h->h.filter.level; pp->sharpness_level = h->h.filter.sharpness; @@ -265,7 +266,7 @@ static int dxva2_vp9_start_frame(AVCodecContext *avctx, av_assert0(ctx_pic); /* Fill up DXVA_PicParams_VP9 */ - if (fill_picture_parameters(avctx, ctx, h, &ctx_pic->pp) < 0) + if (ff_dxva2_vp9_fill_picture_parameters(avctx, ctx, &ctx_pic->pp) < 0) return -1; ctx_pic->bitstream_size = 0; diff --git a/libavcodec/hwaccels.h b/libavcodec/hwaccels.h index 6b71919b0a..1344b84326 100644 --- a/libavcodec/hwaccels.h +++ b/libavcodec/hwaccels.h @@ -73,6 +73,7 @@ extern const struct FFHWAccel ff_vp8_nvdec_hwaccel; extern const struct FFHWAccel ff_vp8_vaapi_hwaccel; extern const struct FFHWAccel ff_vp9_d3d11va_hwaccel; extern const struct FFHWAccel ff_vp9_d3d11va2_hwaccel; +extern const struct FFHWAccel ff_vp9_d3d12va_hwaccel; extern const struct FFHWAccel ff_vp9_dxva2_hwaccel; extern const struct FFHWAccel ff_vp9_nvdec_hwaccel; extern const struct FFHWAccel ff_vp9_vaapi_hwaccel; diff --git a/libavcodec/vp9.c b/libavcodec/vp9.c index 75e2e1d329..855936cdc1 100644 --- a/libavcodec/vp9.c +++ b/libavcodec/vp9.c @@ -170,6 +170,7 @@ static int update_size(AVCodecContext *avctx, int w, int h) { #define HWACCEL_MAX (CONFIG_VP9_DXVA2_HWACCEL + \ CONFIG_VP9_D3D11VA_HWACCEL * 2 + \ + CONFIG_VP9_D3D12VA_HWACCEL + \ CONFIG_VP9_NVDEC_HWACCEL + \ CONFIG_VP9_VAAPI_HWACCEL + \ CONFIG_VP9_VDPAU_HWACCEL + \ @@ -196,6 +197,9 @@ static int update_size(AVCodecContext *avctx, int w, int h) *fmtp++ = AV_PIX_FMT_D3D11VA_VLD; *fmtp++ = AV_PIX_FMT_D3D11; #endif +#if CONFIG_VP9_D3D12VA_HWACCEL + *fmtp++ = AV_PIX_FMT_D3D12; +#endif #if CONFIG_VP9_NVDEC_HWACCEL *fmtp++ = AV_PIX_FMT_CUDA; #endif @@ -1899,6 +1903,9 @@ const FFCodec ff_vp9_decoder = { #if CONFIG_VP9_D3D11VA2_HWACCEL HWACCEL_D3D11VA2(vp9), #endif +#if CONFIG_VP9_D3D12VA_HWACCEL + HWACCEL_D3D12VA(vp9), +#endif #if CONFIG_VP9_NVDEC_HWACCEL HWACCEL_NVDEC(vp9), #endif