From patchwork Wed Aug 5 10:07:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Lhomme X-Patchwork-Id: 21493 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 7D1D644A788 for ; Wed, 5 Aug 2020 13:07:30 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 50F0668B664; Wed, 5 Aug 2020 13:07:30 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mout-p-102.mailbox.org (mout-p-102.mailbox.org [80.241.56.152]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 70D6768B638 for ; Wed, 5 Aug 2020 13:07:24 +0300 (EEST) Received: from smtp1.mailbox.org (smtp1.mailbox.org [80.241.60.240]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by mout-p-102.mailbox.org (Postfix) with ESMTPS id 4BM6k3564wzKmhn for ; Wed, 5 Aug 2020 12:07:23 +0200 (CEST) X-Virus-Scanned: amavisd-new at heinlein-support.de Received: from smtp1.mailbox.org ([80.241.60.240]) by spamfilter06.heinlein-hosting.de (spamfilter06.heinlein-hosting.de [80.241.56.125]) (amavisd-new, port 10030) with ESMTP id uqHCP5H076_P for ; Wed, 5 Aug 2020 12:07:19 +0200 (CEST) From: Steve Lhomme To: ffmpeg-devel@ffmpeg.org Date: Wed, 5 Aug 2020 12:07:11 +0200 Message-Id: <20200805100712.16834-1-robux4@ycbcr.xyz> MIME-Version: 1.0 X-MBO-SPAM-Probability: 1 X-Rspamd-Score: 0.27 / 15.00 / 15.00 X-Rspamd-Queue-Id: AD4A4183D X-Rspamd-UID: 948ac6 Subject: [FFmpeg-devel] [PATCH v3 1/2] dxva: wait until D3D11 buffer copies are done before submitting them X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" When used aggressively, calling SubmitDecoderBuffers() just after ReleaseDecoderBuffer() may have the buffers not used properly and creates decoding artifacts. It's likely due to the time to copy the submitted buffer in CPU mapped memory to GPU memory. SubmitDecoderBuffers() doesn't appear to wait for the state of the buffer submitted to become "ready". For now it's not supported in the legacy API using AVD3D11VAContext, we need to add a ID3D11DeviceContext in there as it cannot be derived from the other interfaces we provide (ID3D11VideoContext is not a kind of ID3D11DeviceContext). --- libavcodec/dxva2.c | 33 +++++++++++++++++++++++++++++++++ libavcodec/dxva2_internal.h | 2 ++ 2 files changed, 35 insertions(+) diff --git a/libavcodec/dxva2.c b/libavcodec/dxva2.c index 32416112bf..1a0e5b69b2 100644 --- a/libavcodec/dxva2.c +++ b/libavcodec/dxva2.c @@ -692,6 +692,12 @@ int ff_dxva2_decode_init(AVCodecContext *avctx) d3d11_ctx->surface = sctx->d3d11_views; d3d11_ctx->workaround = sctx->workaround; d3d11_ctx->context_mutex = INVALID_HANDLE_VALUE; + + D3D11_QUERY_DESC query = { 0 }; + query.Query = D3D11_QUERY_EVENT; + if (FAILED(ID3D11Device_CreateQuery(device_hwctx->device, &query, + (ID3D11Query**)&sctx->wait_copies))) + sctx->wait_copies = NULL; } #endif @@ -729,6 +735,8 @@ int ff_dxva2_decode_uninit(AVCodecContext *avctx) av_buffer_unref(&sctx->decoder_ref); #if CONFIG_D3D11VA + if (sctx->wait_copies) + ID3D11Asynchronous_Release(sctx->wait_copies); for (i = 0; i < sctx->nb_d3d11_views; i++) { if (sctx->d3d11_views[i]) ID3D11VideoDecoderOutputView_Release(sctx->d3d11_views[i]); @@ -932,6 +940,12 @@ int ff_dxva2_common_end_frame(AVCodecContext *avctx, AVFrame *frame, #if CONFIG_D3D11VA if (ff_dxva2_is_d3d11(avctx)) { + if (sctx->wait_copies) { + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + AVD3D11VADeviceContext *device_hwctx = frames_ctx->device_ctx->hwctx; + ID3D11DeviceContext_Begin(device_hwctx->device_context, sctx->wait_copies); + } + buffer = &buffer11[buffer_count]; type = D3D11_VIDEO_DECODER_BUFFER_PICTURE_PARAMETERS; } @@ -1005,9 +1019,28 @@ int ff_dxva2_common_end_frame(AVCodecContext *avctx, AVFrame *frame, #if CONFIG_D3D11VA if (ff_dxva2_is_d3d11(avctx)) + { + int maxWait = 10; + /* wait until all the buffer release is done copying data to the GPU + * before doing the submit command */ + if (sctx->wait_copies) { + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + AVD3D11VADeviceContext *device_hwctx = frames_ctx->device_ctx->hwctx; + ID3D11DeviceContext_End(device_hwctx->device_context, sctx->wait_copies); + + while (maxWait-- && S_FALSE == + ID3D11DeviceContext_GetData(device_hwctx->device_context, + sctx->wait_copies, NULL, 0, 0)) { + ff_dxva2_unlock(avctx); + SleepEx(2, TRUE); + ff_dxva2_lock(avctx); + } + } + hr = ID3D11VideoContext_SubmitDecoderBuffers(D3D11VA_CONTEXT(ctx)->video_context, D3D11VA_CONTEXT(ctx)->decoder, buffer_count, buffer11); + } #endif #if CONFIG_DXVA2 if (avctx->pix_fmt == AV_PIX_FMT_DXVA2_VLD) { diff --git a/libavcodec/dxva2_internal.h b/libavcodec/dxva2_internal.h index b822af59cd..c44e8e09b0 100644 --- a/libavcodec/dxva2_internal.h +++ b/libavcodec/dxva2_internal.h @@ -81,6 +81,8 @@ typedef struct FFDXVASharedContext { ID3D11VideoDecoderOutputView **d3d11_views; int nb_d3d11_views; ID3D11Texture2D *d3d11_texture; + + ID3D11Asynchronous *wait_copies; #endif #if CONFIG_DXVA2 From patchwork Wed Aug 5 10:07:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Lhomme X-Patchwork-Id: 21494 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 6F5CE44A788 for ; Wed, 5 Aug 2020 13:07:31 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5848268B794; Wed, 5 Aug 2020 13:07:31 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mout-p-202.mailbox.org (mout-p-202.mailbox.org [80.241.56.172]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7C40268B6C7 for ; Wed, 5 Aug 2020 13:07:24 +0300 (EEST) Received: from smtp1.mailbox.org (smtp1.mailbox.org [80.241.60.240]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by mout-p-202.mailbox.org (Postfix) with ESMTPS id 4BM6k40WNXzQlQT for ; Wed, 5 Aug 2020 12:07:24 +0200 (CEST) X-Virus-Scanned: amavisd-new at heinlein-support.de Received: from smtp1.mailbox.org ([80.241.60.240]) by spamfilter01.heinlein-hosting.de (spamfilter01.heinlein-hosting.de [80.241.56.115]) (amavisd-new, port 10030) with ESMTP id lSuALNYxNWbT for ; Wed, 5 Aug 2020 12:07:20 +0200 (CEST) From: Steve Lhomme To: ffmpeg-devel@ffmpeg.org Date: Wed, 5 Aug 2020 12:07:12 +0200 Message-Id: <20200805100712.16834-2-robux4@ycbcr.xyz> In-Reply-To: <20200805100712.16834-1-robux4@ycbcr.xyz> References: <20200805100712.16834-1-robux4@ycbcr.xyz> MIME-Version: 1.0 X-MBO-SPAM-Probability: 1 X-Rspamd-Score: 0.27 / 15.00 / 15.00 X-Rspamd-Queue-Id: E14C3181E X-Rspamd-UID: 640a91 Subject: [FFmpeg-devel] [PATCH v3 2/2] dxva: add a ID3D11DeviceContext to the old D3D11VA API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This way it's possible to use the wait_copy feature with the old API as well. If it's NULL, wait_copies remains NULL and is not used. --- doc/APIchanges | 3 +++ libavcodec/d3d11va.h | 6 ++++++ libavcodec/dxva2.c | 38 +++++++++++++++++++++++++++++++------- libavcodec/version.h | 2 +- 4 files changed, 41 insertions(+), 8 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index a9bf1afd4c..11a10ddb16 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -15,6 +15,9 @@ libavutil: 2017-10-21 API changes, most recent first: +2020-08-05 - xxxxxxxxxx - lavu 56.100.100 - d3d11va.h + Add a ID3D11VideoContext to AVD3D11VAContext matching the ID3D11VideoContext. + 2020-08-04 - xxxxxxxxxx - lavu 56.58.100 - channel_layout.h Add AV_CH_LAYOUT_22POINT2 together with its newly required pieces: AV_CH_TOP_SIDE_LEFT, AV_CH_TOP_SIDE_RIGHT, AV_CH_BOTTOM_FRONT_CENTER, diff --git a/libavcodec/d3d11va.h b/libavcodec/d3d11va.h index 6816b6c1e6..134dee388c 100644 --- a/libavcodec/d3d11va.h +++ b/libavcodec/d3d11va.h @@ -96,6 +96,12 @@ typedef struct AVD3D11VAContext { * Mutex to access video_context */ HANDLE context_mutex; + + /** + * Device Context on which to send the D3D11VA commands. + * It should be the same used to get the ID3D11VideoDecoder or NULL. + */ + ID3D11DeviceContext *device_context; } AVD3D11VAContext; /** diff --git a/libavcodec/dxva2.c b/libavcodec/dxva2.c index 1a0e5b69b2..fd4685305e 100644 --- a/libavcodec/dxva2.c +++ b/libavcodec/dxva2.c @@ -656,7 +656,27 @@ int ff_dxva2_decode_init(AVCodecContext *avctx) // Old API. if (avctx->hwaccel_context) + { +#if CONFIG_D3D11VA + if (avctx->hwaccel->pix_fmt == AV_PIX_FMT_D3D11VA_VLD) { + AVDXVAContext *ctx = DXVA_CONTEXT(avctx); + + if (D3D11VA_CONTEXT(ctx)->device_context) { + ID3D11Device *d3ddevice; + ID3D11DeviceContext_GetDevice(D3D11VA_CONTEXT(ctx)->device_context, &d3ddevice); + + D3D11_QUERY_DESC query = { 0 }; + query.Query = D3D11_QUERY_EVENT; + if (FAILED(ID3D11Device_CreateQuery(d3ddevice, &query, + (ID3D11Query**)&sctx->wait_copies))) + sctx->wait_copies = NULL; + + ID3D11Device_Release(d3ddevice); + } + } +#endif return 0; + } // (avctx->pix_fmt is not updated yet at this point) sctx->pix_fmt = avctx->hwaccel->pix_fmt; @@ -896,6 +916,7 @@ int ff_dxva2_common_end_frame(AVCodecContext *avctx, AVFrame *frame, unsigned buffer_count = 0; #if CONFIG_D3D11VA D3D11_VIDEO_DECODER_BUFFER_DESC buffer11[4]; + ID3D11DeviceContext *device_context; #endif #if CONFIG_DXVA2 DXVA2_DecodeBufferDesc buffer2[4]; @@ -941,9 +962,14 @@ int ff_dxva2_common_end_frame(AVCodecContext *avctx, AVFrame *frame, #if CONFIG_D3D11VA if (ff_dxva2_is_d3d11(avctx)) { if (sctx->wait_copies) { - AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; - AVD3D11VADeviceContext *device_hwctx = frames_ctx->device_ctx->hwctx; - ID3D11DeviceContext_Begin(device_hwctx->device_context, sctx->wait_copies); + if (avctx->hwaccel_context) + device_context = D3D11VA_CONTEXT(ctx)->device_context; + else { + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + AVD3D11VADeviceContext *device_hwctx = frames_ctx->device_ctx->hwctx; + device_context = device_hwctx->device_context; + } + ID3D11DeviceContext_Begin(device_context, sctx->wait_copies); } buffer = &buffer11[buffer_count]; @@ -1024,12 +1050,10 @@ int ff_dxva2_common_end_frame(AVCodecContext *avctx, AVFrame *frame, /* wait until all the buffer release is done copying data to the GPU * before doing the submit command */ if (sctx->wait_copies) { - AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; - AVD3D11VADeviceContext *device_hwctx = frames_ctx->device_ctx->hwctx; - ID3D11DeviceContext_End(device_hwctx->device_context, sctx->wait_copies); + ID3D11DeviceContext_End(device_context, sctx->wait_copies); while (maxWait-- && S_FALSE == - ID3D11DeviceContext_GetData(device_hwctx->device_context, + ID3D11DeviceContext_GetData(device_context, sctx->wait_copies, NULL, 0, 0)) { ff_dxva2_unlock(avctx); SleepEx(2, TRUE); diff --git a/libavcodec/version.h b/libavcodec/version.h index f66919617a..a3f9f828ee 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -28,7 +28,7 @@ #include "libavutil/version.h" #define LIBAVCODEC_VERSION_MAJOR 58 -#define LIBAVCODEC_VERSION_MINOR 99 +#define LIBAVCODEC_VERSION_MINOR 100 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \