From patchwork Fri Mar 6 13:09:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: galinart X-Patchwork-Id: 18071 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id D997744B76E for ; Fri, 6 Mar 2020 15:16:12 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B912468ADB0; Fri, 6 Mar 2020 15:16:12 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f66.google.com (mail-wm1-f66.google.com [209.85.128.66]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CAE3B68ADB0 for ; Fri, 6 Mar 2020 15:16:05 +0200 (EET) Received: by mail-wm1-f66.google.com with SMTP id 6so2363058wmi.5 for ; Fri, 06 Mar 2020 05:16:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=S2oSy4oF88mefKbPC5vL2TTQ/Zxe3jF9dTrI4t1lieI=; b=pL94yLzZWDj7CLmGmNlo+cMJNxrJBarFxTqI9qp8RWEqZqoxCL0ZJwWqYM5yku6hda DcdPONsm6hrsm2y58+zfBY4/ZnayNOoLkKF+gbW5SS8j7odYH5zlccOGIxi4E2f+Dldg 0BrnOzDocBDjWObIbKrpbtHw7wmqMr62qpJ08cYJrPRNZdG0Y7TbO9/O+Be9kQWnW51c mbciwgGHXtzPjm5UzktfVXJUIfrSoP30Fqo4GYayIbkranVkF6PohQyS/jfdlck3hihi KiAI+l/me5tp7+/fF6/YUw+Oo8XZRu5pg8+vXjZtVNLV3Qt9v1FuEFwMEyPeP5kJ7Bwf w4xA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=S2oSy4oF88mefKbPC5vL2TTQ/Zxe3jF9dTrI4t1lieI=; b=ifWxUlapRRhJuG1RdZ3CA3j5yzE37H0xDgVLY8Dmsq86QceSAa/sZoxZHGCfdOgyfD U9jGSPpBnZvVD0dSp7ytVRuk0z6duy5XatYOv6I3iz5JAVrmwu/YaMKUTzX2M7c/bVIh nW6+SpQToTXGOm+MPQnMeT2adr5tPnqUm0AwSjpIodbh9r9n6O/Wx3TbSqP9k6wwDF1O /txDubc+FU4ILt+sbaeOpuDv2OXVA3obYAfrNqpecb1y9aINlS/ep1AHwAW2R8vA/teZ lrMuz7YZ17H9QNN+Jq96M/NV1RS2RrPlUTkR7A8EOM9e6z+YMhDbAaq9wiuPSITm0goe st3g== X-Gm-Message-State: ANhLgQ2BdVmZXINXDUUEBvWZDYYI7OGWF3XBpO6S84VzMYsObLevFUVe 5UUOUvsSw2cNUW5tK41/8idZhZ16Ato= X-Google-Smtp-Source: ADFU+vtpYj+6/1VQ9I/ieuC8VaoiQrnmzDczAjm/Gpmo0PYhpZ/a4xhFAIya0RdpfnoWYWQ+Ma2cKQ== X-Received: by 2002:a05:600c:4151:: with SMTP id h17mr4196215wmm.189.1583500222660; Fri, 06 Mar 2020 05:10:22 -0800 (PST) Received: from localhost.localdomain ([134.191.244.0]) by smtp.gmail.com with ESMTPSA id k66sm10904298wmf.0.2020.03.06.05.10.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Mar 2020 05:10:21 -0800 (PST) From: Artem Galin To: ffmpeg-devel@ffmpeg.org Date: Fri, 6 Mar 2020 13:09:54 +0000 Message-Id: <20200306130954.8938-1-artem.galin@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200123151813.31739-1-artem.galin@gmail.com> References: <20200123151813.31739-1-artem.galin@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4] lavc/qsv: adding DX11 support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Artem Galin Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This enables DX11 support for QSV with higher priority than DX9. In case of multiple GPUs configuration, DX9 API does not allow to get access to QSV device in some cases - headless. Implementation based on DX11 resolves that restriction by enumerating list of available GPUs and finding device with QSV support. Signed-off-by: Artem Galin --- fftools/ffmpeg_opt.c | 12 +- libavcodec/qsv.c | 53 +++++++- libavcodec/qsv_internal.h | 2 + libavfilter/qsvvpp.c | 36 +++-- libavutil/hwcontext_d3d11va.c | 56 +++++++- libavutil/hwcontext_qsv.c | 240 ++++++++++++++++++++++++++++------ 6 files changed, 340 insertions(+), 59 deletions(-) diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 1b721c4954..da619b0043 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -504,7 +504,17 @@ static int opt_init_hw_device(void *optctx, const char *opt, const char *arg) printf("\n"); exit_program(0); } else { - return hw_device_init_from_string(arg, NULL); + HWDevice *dev; + int err; + if (!arg) + return AVERROR(ENOMEM); + err = hw_device_init_from_string(arg, &dev); + if (err < 0) + return err; + hw_device_ctx = av_buffer_ref(dev->device_ref); + if (!hw_device_ctx) + return AVERROR(ENOMEM); + return 0; } } diff --git a/libavcodec/qsv.c b/libavcodec/qsv.c index db98c75073..beea76896f 100644 --- a/libavcodec/qsv.c +++ b/libavcodec/qsv.c @@ -36,6 +36,8 @@ #include "avcodec.h" #include "qsv_internal.h" +#define MFX_IMPL_VIA_MASK(impl) (0x0f00 & (impl)) + #if QSV_VERSION_ATLEAST(1, 12) #include "mfx/mfxvp8.h" #endif @@ -221,8 +223,13 @@ int ff_qsv_find_surface_idx(QSVFramesContext *ctx, QSVFrame *frame) int i; for (i = 0; i < ctx->nb_mids; i++) { QSVMid *mid = &ctx->mids[i]; +#if CONFIG_D3D11VA + if (mid->handle_pair.second == frame->surface.Data.MemId) + return i; +#else if (mid->handle == frame->surface.Data.MemId) return i; +#endif } return AVERROR_BUG; } @@ -362,7 +369,11 @@ static int ff_qsv_set_display_handle(AVCodecContext *avctx, QSVSession *qs) int ff_qsv_init_internal_session(AVCodecContext *avctx, QSVSession *qs, const char *load_plugins, int gpu_copy) { +#ifdef AVCODEC_QSV_LINUX_SESSION_HANDLE mfxIMPL impl = MFX_IMPL_AUTO_ANY; +#else + mfxIMPL impl = MFX_IMPL_AUTO_ANY | MFX_IMPL_VIA_D3D11; +#endif mfxVersion ver = { { QSV_VERSION_MINOR, QSV_VERSION_MAJOR } }; mfxInitParam init_par = { MFX_IMPL_AUTO_ANY }; @@ -449,11 +460,21 @@ static AVBufferRef *qsv_create_mids(AVBufferRef *hw_frames_ref) return NULL; } +#if CONFIG_D3D11VA + for (i = 0; i < nb_surfaces; i++) { + QSVMid *mid = &mids[i]; + mid->handle_is_pair = 1; + mid->handle_pair.first = (mfxHDL)frames_hwctx->surfaces[i].Data.reserved2; + mid->handle_pair.second = frames_hwctx->surfaces[i].Data.MemId; + mid->hw_frames_ref = hw_frames_ref1; + } +#else for (i = 0; i < nb_surfaces; i++) { QSVMid *mid = &mids[i]; mid->handle = frames_hwctx->surfaces[i].Data.MemId; mid->hw_frames_ref = hw_frames_ref1; } +#endif return mids_buf; } @@ -661,7 +682,12 @@ static mfxStatus qsv_frame_unlock(mfxHDL pthis, mfxMemId mid, mfxFrameData *ptr) static mfxStatus qsv_frame_get_hdl(mfxHDL pthis, mfxMemId mid, mfxHDL *hdl) { QSVMid *qsv_mid = (QSVMid*)mid; - *hdl = qsv_mid->handle; + if (qsv_mid->handle_is_pair) { + mfxHDLPair *hdlpair = (mfxHDLPair*)hdl; + *hdlpair = qsv_mid->handle_pair; + } else { + *hdl = qsv_mid->handle; + } return MFX_ERR_NONE; } @@ -679,11 +705,11 @@ int ff_qsv_init_session_device(AVCodecContext *avctx, mfxSession *psession, mfxSession parent_session = device_hwctx->session; mfxInitParam init_par = { MFX_IMPL_AUTO_ANY }; mfxHDL handle = NULL; + mfxHandleType handle_type = MFX_HANDLE_D3D11_DEVICE; mfxSession session; mfxVersion ver; mfxIMPL impl; - mfxHandleType handle_type; mfxStatus err; int i, ret; @@ -695,11 +721,26 @@ int ff_qsv_init_session_device(AVCodecContext *avctx, mfxSession *psession, return ff_qsv_print_error(avctx, err, "Error querying the session attributes"); + if (MFX_IMPL_VIA_D3D11 == MFX_IMPL_VIA_MASK(impl)) + { + handle_type = MFX_HANDLE_D3D11_DEVICE; + } + else if (MFX_IMPL_VIA_D3D9 == MFX_IMPL_VIA_MASK(impl)) + { + handle_type = MFX_HANDLE_D3D9_DEVICE_MANAGER; + } + else if (MFX_IMPL_VIA_VAAPI == MFX_IMPL_VIA_MASK(impl)) + { + handle_type = MFX_HANDLE_VA_DISPLAY; + } + for (i = 0; i < FF_ARRAY_ELEMS(handle_types); i++) { - err = MFXVideoCORE_GetHandle(parent_session, handle_types[i], &handle); - if (err == MFX_ERR_NONE) { - handle_type = handle_types[i]; - break; + if (handle_types[i] == handle_type) + { + err = MFXVideoCORE_GetHandle(parent_session, handle_types[i], &handle); + if (err == MFX_ERR_NONE) { + break; + } } handle = NULL; } diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h index 6489836a67..99dfa8dcff 100644 --- a/libavcodec/qsv_internal.h +++ b/libavcodec/qsv_internal.h @@ -60,7 +60,9 @@ typedef struct QSVMid { AVBufferRef *hw_frames_ref; + int handle_is_pair; mfxHDL handle; + mfxHDLPair handle_pair; AVFrame *locked_frame; AVFrame *hw_frame; diff --git a/libavfilter/qsvvpp.c b/libavfilter/qsvvpp.c index 8d5ff2eb65..306bb2e0c3 100644 --- a/libavfilter/qsvvpp.c +++ b/libavfilter/qsvvpp.c @@ -32,10 +32,11 @@ #include "qsvvpp.h" #include "video.h" -#define IS_VIDEO_MEMORY(mode) (mode & (MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET | \ +#define IS_VIDEO_MEMORY(mode) (mode & (MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET | \ MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET)) -#define IS_OPAQUE_MEMORY(mode) (mode & MFX_MEMTYPE_OPAQUE_FRAME) -#define IS_SYSTEM_MEMORY(mode) (mode & MFX_MEMTYPE_SYSTEM_MEMORY) +#define IS_OPAQUE_MEMORY(mode) (mode & MFX_MEMTYPE_OPAQUE_FRAME) +#define IS_SYSTEM_MEMORY(mode) (mode & MFX_MEMTYPE_SYSTEM_MEMORY) +#define MFX_IMPL_VIA_MASK(impl) (0x0f00 & (impl)) typedef struct QSVFrame { AVFrame *frame; @@ -405,12 +406,12 @@ static int init_vpp_session(AVFilterContext *avctx, QSVVPPContext *s) AVFilterLink *outlink = avctx->outputs[0]; AVQSVFramesContext *in_frames_hwctx = NULL; AVQSVFramesContext *out_frames_hwctx = NULL; + mfxHandleType handle_type = MFX_HANDLE_D3D11_DEVICE; AVBufferRef *device_ref; AVHWDeviceContext *device_ctx; AVQSVDeviceContext *device_hwctx; mfxHDL handle; - mfxHandleType handle_type; mfxVersion ver; mfxIMPL impl; int ret, i; @@ -497,15 +498,30 @@ static int init_vpp_session(AVFilterContext *avctx, QSVVPPContext *s) return AVERROR_UNKNOWN; } + if (MFX_IMPL_VIA_D3D11 == MFX_IMPL_VIA_MASK(impl)) + { + handle_type = MFX_HANDLE_D3D11_DEVICE; + } + else if (MFX_IMPL_VIA_D3D9 == MFX_IMPL_VIA_MASK(impl)) + { + handle_type = MFX_HANDLE_D3D9_DEVICE_MANAGER; + } + else if (MFX_IMPL_VIA_VAAPI == MFX_IMPL_VIA_MASK(impl)) + { + handle_type = MFX_HANDLE_VA_DISPLAY; + } + for (i = 0; i < FF_ARRAY_ELEMS(handle_types); i++) { - ret = MFXVideoCORE_GetHandle(device_hwctx->session, handle_types[i], &handle); - if (ret == MFX_ERR_NONE) { - handle_type = handle_types[i]; - break; + if (handle_types[i] == handle_type) + { + ret = MFXVideoCORE_GetHandle(device_hwctx->session, handle_types[i], &handle); + if (ret == MFX_ERR_NONE) { + break; + } } + handle = NULL; } - - if (ret != MFX_ERR_NONE) { + if (!handle) { av_log(avctx, AV_LOG_ERROR, "Error getting the session handle\n"); return AVERROR_UNKNOWN; } diff --git a/libavutil/hwcontext_d3d11va.c b/libavutil/hwcontext_d3d11va.c index c8ae58f908..13279b7e6f 100644 --- a/libavutil/hwcontext_d3d11va.c +++ b/libavutil/hwcontext_d3d11va.c @@ -511,15 +511,57 @@ static void d3d11va_device_uninit(AVHWDeviceContext *hwdev) } } +static int d3d11va_device_find_adapter_by_vendor_id(AVHWDeviceContext *ctx, UINT creationFlags, char *vendor) +{ + HRESULT hr; + IDXGIAdapter *adapter = NULL; + int adapter_id = 0; + IDXGIFactory2 *factory; + long int vendor_id = strtol(vendor, NULL, 0); + hr = mCreateDXGIFactory(&IID_IDXGIFactory2, (void **)&factory); + while (IDXGIFactory2_EnumAdapters(factory, adapter_id++, &adapter) != DXGI_ERROR_NOT_FOUND) + { + ID3D11Device* device = NULL; + DXGI_ADAPTER_DESC adapter_desc; + + hr = mD3D11CreateDevice(adapter, D3D_DRIVER_TYPE_UNKNOWN, NULL, creationFlags, NULL, 0, D3D11_SDK_VERSION, &device, NULL, NULL); + if (FAILED(hr)) { + av_log(ctx, AV_LOG_ERROR, "D3D11CreateDevice returned error\n"); + continue; + } + + hr = IDXGIAdapter2_GetDesc(adapter, &adapter_desc); + if (FAILED(hr)) { + av_log(ctx, AV_LOG_ERROR, "IDXGIAdapter2_GetDesc returned error\n"); + continue; + } + + if (device) + ID3D11Device_Release(device); + + if (adapter) + IDXGIAdapter_Release(adapter); + + if (adapter_desc.VendorId == vendor_id) { + IDXGIFactory2_Release(factory); + return adapter_id - 1; + } + } + IDXGIFactory2_Release(factory); + return -1; +} + static int d3d11va_device_create(AVHWDeviceContext *ctx, const char *device, AVDictionary *opts, int flags) { AVD3D11VADeviceContext *device_hwctx = ctx->hwctx; HRESULT hr; + AVDictionaryEntry *e; IDXGIAdapter *pAdapter = NULL; ID3D10Multithread *pMultithread; UINT creationFlags = D3D11_CREATE_DEVICE_VIDEO_SUPPORT; + int adapter = -1; int is_debug = !!av_dict_get(opts, "debug", NULL, 0); int ret; @@ -539,11 +581,23 @@ static int d3d11va_device_create(AVHWDeviceContext *ctx, const char *device, return AVERROR_UNKNOWN; } + e = av_dict_get(opts, "vendor", NULL, 0); + if (e) { + adapter = d3d11va_device_find_adapter_by_vendor_id(ctx, creationFlags, e ? e->value : NULL); + if (adapter < 0) { + av_log(ctx, AV_LOG_ERROR, "Failed to find d3d11va adapter by vendor id %s\n", e ? e->value : NULL); + return AVERROR_UNKNOWN; + } + } + if (device) { + adapter = atoi(device); + } + + if (adapter >= 0) { IDXGIFactory2 *pDXGIFactory; hr = mCreateDXGIFactory(&IID_IDXGIFactory2, (void **)&pDXGIFactory); if (SUCCEEDED(hr)) { - int adapter = atoi(device); if (FAILED(IDXGIFactory2_EnumAdapters(pDXGIFactory, adapter, &pAdapter))) pAdapter = NULL; IDXGIFactory2_Release(pDXGIFactory); diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index b1b67400de..97a18088c0 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -27,9 +27,13 @@ #include #endif +#define COBJMACROS #if CONFIG_VAAPI #include "hwcontext_vaapi.h" #endif +#if CONFIG_D3D11VA +#include "hwcontext_d3d11va.h" +#endif #if CONFIG_DXVA2 #include "hwcontext_dxva2.h" #endif @@ -44,6 +48,8 @@ #include "pixdesc.h" #include "time.h" +#define MFX_IMPL_VIA_MASK(impl) (0x0f00 & (impl)) + typedef struct QSVDevicePriv { AVBufferRef *child_device_ctx; } QSVDevicePriv; @@ -89,6 +95,9 @@ static const struct { #if CONFIG_VAAPI { MFX_HANDLE_VA_DISPLAY, AV_HWDEVICE_TYPE_VAAPI, AV_PIX_FMT_VAAPI }, #endif +#if CONFIG_D3D11VA + { MFX_HANDLE_D3D11_DEVICE, AV_HWDEVICE_TYPE_D3D11VA, AV_PIX_FMT_D3D11 }, +#endif #if CONFIG_DXVA2 { MFX_HANDLE_D3D9_DEVICE_MANAGER, AV_HWDEVICE_TYPE_DXVA2, AV_PIX_FMT_DXVA2_VLD }, #endif @@ -119,25 +128,11 @@ static int qsv_device_init(AVHWDeviceContext *ctx) { AVQSVDeviceContext *hwctx = ctx->hwctx; QSVDeviceContext *s = ctx->internal->priv; + mfxHandleType handle_type = MFX_HANDLE_D3D11_DEVICE; mfxStatus err; int i; - for (i = 0; supported_handle_types[i].handle_type; i++) { - err = MFXVideoCORE_GetHandle(hwctx->session, supported_handle_types[i].handle_type, - &s->handle); - if (err == MFX_ERR_NONE) { - s->handle_type = supported_handle_types[i].handle_type; - s->child_device_type = supported_handle_types[i].device_type; - s->child_pix_fmt = supported_handle_types[i].pix_fmt; - break; - } - } - if (!s->handle) { - av_log(ctx, AV_LOG_VERBOSE, "No supported hw handle could be retrieved " - "from the session\n"); - } - err = MFXQueryIMPL(hwctx->session, &s->impl); if (err == MFX_ERR_NONE) err = MFXQueryVersion(hwctx->session, &s->ver); @@ -146,6 +141,36 @@ static int qsv_device_init(AVHWDeviceContext *ctx) return AVERROR_UNKNOWN; } + if (MFX_IMPL_VIA_D3D11 == MFX_IMPL_VIA_MASK(s->impl)) + { + handle_type = MFX_HANDLE_D3D11_DEVICE; + } + else if (MFX_IMPL_VIA_D3D9 == MFX_IMPL_VIA_MASK(s->impl)) + { + handle_type = MFX_HANDLE_D3D9_DEVICE_MANAGER; + } + else if (MFX_IMPL_VIA_VAAPI == MFX_IMPL_VIA_MASK(s->impl)) + { + handle_type = MFX_HANDLE_VA_DISPLAY; + } + + for (i = 0; supported_handle_types[i].handle_type; i++) { + if (supported_handle_types[i].handle_type == handle_type) { + err = MFXVideoCORE_GetHandle(hwctx->session, supported_handle_types[i].handle_type, + &s->handle); + if (err == MFX_ERR_NONE) { + s->handle_type = supported_handle_types[i].handle_type; + s->child_device_type = supported_handle_types[i].device_type; + s->child_pix_fmt = supported_handle_types[i].pix_fmt; + break; + } + } + } + if (!s->handle) { + av_log(ctx, AV_LOG_VERBOSE, "No supported hw handle could be retrieved " + "from the session\n"); + } + return 0; } @@ -229,6 +254,13 @@ static int qsv_init_child_ctx(AVHWFramesContext *ctx) child_device_hwctx->display = (VADisplay)device_priv->handle; } #endif +#if CONFIG_D3D11VA + if (child_device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) { + AVD3D11VADeviceContext *child_device_hwctx = child_device_ctx->hwctx; + ID3D11Device_AddRef((ID3D11Device*)device_priv->handle); + child_device_hwctx->device = (ID3D11Device*)device_priv->handle; + } +#endif #if CONFIG_DXVA2 if (child_device_ctx->type == AV_HWDEVICE_TYPE_DXVA2) { AVDXVA2DeviceContext *child_device_hwctx = child_device_ctx->hwctx; @@ -255,6 +287,16 @@ static int qsv_init_child_ctx(AVHWFramesContext *ctx) child_frames_ctx->width = FFALIGN(ctx->width, 16); child_frames_ctx->height = FFALIGN(ctx->height, 16); +#if CONFIG_D3D11VA + if (child_device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) { + AVD3D11VAFramesContext *child_frames_hwctx = child_frames_ctx->hwctx; + child_frames_hwctx->MiscFlags |= D3D11_RESOURCE_MISC_SHARED; + if (hwctx->frame_type & MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET) + child_frames_hwctx->BindFlags = D3D11_BIND_RENDER_TARGET ; + else + child_frames_hwctx->BindFlags = D3D11_BIND_DECODER; + } +#endif #if CONFIG_DXVA2 if (child_device_ctx->type == AV_HWDEVICE_TYPE_DXVA2) { AVDXVA2FramesContext *child_frames_hwctx = child_frames_ctx->hwctx; @@ -279,6 +321,19 @@ static int qsv_init_child_ctx(AVHWFramesContext *ctx) hwctx->frame_type = MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET; } #endif +#if CONFIG_D3D11VA + if (child_device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) { + AVD3D11VAFramesContext *child_frames_hwctx = child_frames_ctx->hwctx; + for (i = 0; i < ctx->initial_pool_size; i++) { + s->surfaces_internal[i].Data.MemId = (mfxMemId)(intptr_t)i; + s->surfaces_internal[i].Data.reserved2 = (intptr_t)child_frames_hwctx->texture; + } + if (child_frames_hwctx->BindFlags & D3D11_BIND_DECODER) + hwctx->frame_type = MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET; + else + hwctx->frame_type = MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET; + } +#endif #if CONFIG_DXVA2 if (child_device_ctx->type == AV_HWDEVICE_TYPE_DXVA2) { AVDXVA2FramesContext *child_frames_hwctx = child_frames_ctx->hwctx; @@ -421,6 +476,19 @@ static mfxStatus frame_unlock(mfxHDL pthis, mfxMemId mid, mfxFrameData *ptr) static mfxStatus frame_get_hdl(mfxHDL pthis, mfxMemId mid, mfxHDL *hdl) { +#if CONFIG_D3D11VA + AVHWFramesContext *ctx = pthis; + QSVFramesContext *s = ctx->internal->priv; + AVHWFramesContext *child_ctx = + (AVHWFramesContext*)s->child_frames_ref->data; + if (child_ctx->format == AV_PIX_FMT_D3D11) { + AVD3D11VAFramesContext *child_hwctx = child_ctx->hwctx; + mfxHDLPair *hdlpair = (mfxHDLPair*)hdl; + hdlpair->first = child_hwctx->texture; + hdlpair->second = mid; // (Texture array index.) + return MFX_ERR_NONE; + } +#endif *hdl = mid; return MFX_ERR_NONE; } @@ -668,6 +736,11 @@ static int qsv_map_from(AVHWFramesContext *ctx, child_data = (uint8_t*)(intptr_t)*(VASurfaceID*)surf->Data.MemId; break; #endif +#if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: + child_data = surf->Data.MemId; + break; +#endif #if CONFIG_DXVA2 case AV_HWDEVICE_TYPE_DXVA2: child_data = surf->Data.MemId; @@ -954,6 +1027,12 @@ static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, AVQSVFramesContext *dst_hwctx = dst_ctx->hwctx; int i; + if (src_ctx->initial_pool_size == 0) { + av_log(dst_ctx, AV_LOG_ERROR, "Only fixed-size pools can be " + "mapped to QSV frames.\n"); + return AVERROR(EINVAL); + } + switch (src_ctx->device_ctx->type) { #if CONFIG_VAAPI case AV_HWDEVICE_TYPE_VAAPI: @@ -972,6 +1051,27 @@ static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, } break; #endif +#if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: + { + AVD3D11VAFramesContext *src_hwctx = src_ctx->hwctx; + s->surfaces_internal = av_mallocz_array(src_ctx->initial_pool_size, + sizeof(*s->surfaces_internal)); + if (!s->surfaces_internal) + return AVERROR(ENOMEM); + for (i = 0; i < src_ctx->initial_pool_size; i++) { + qsv_init_surface(dst_ctx, &s->surfaces_internal[i]); + s->surfaces_internal[i].Data.MemId = (mfxMemId)(intptr_t)i; + s->surfaces_internal[i].Data.reserved2 = (intptr_t)src_hwctx->texture; + } + dst_hwctx->nb_surfaces = src_ctx->initial_pool_size; + if (src_hwctx->BindFlags & D3D11_BIND_DECODER) + dst_hwctx->frame_type = MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET; + else + dst_hwctx->frame_type = MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET; + } + break; +#endif #if CONFIG_DXVA2 case AV_HWDEVICE_TYPE_DXVA2: { @@ -1005,21 +1105,33 @@ static int qsv_map_to(AVHWFramesContext *dst_ctx, AVFrame *dst, const AVFrame *src, int flags) { AVQSVFramesContext *hwctx = dst_ctx->hwctx; - int i, err; + int i, err, index = -1; - for (i = 0; i < hwctx->nb_surfaces; i++) { + for (i = 0; i < hwctx->nb_surfaces && index < 0; i++) { #if CONFIG_VAAPI - if (*(VASurfaceID*)hwctx->surfaces[i].Data.MemId == - (VASurfaceID)(uintptr_t)src->data[3]) + if (AV_PIX_FMT_VAAPI == src->format) { + if (*(VASurfaceID*)hwctx->surfaces[i].Data.MemId == + (VASurfaceID)(uintptr_t)src->data[3]) + index = i break; + } #endif -#if CONFIG_DXVA2 - if ((IDirect3DSurface9*)hwctx->surfaces[i].Data.MemId == - (IDirect3DSurface9*)(uintptr_t)src->data[3]) +#if CONFIG_D3D11VA + if (AV_PIX_FMT_D3D11 == src->format) { + if (i == (intptr_t)src->data[1]) + index = i; break; + } +#endif +#if CONFIG_DXVA2 + if (AV_PIX_FMT_DXVA2_VLD == src->format) { + if ((IDirect3DSurface9*)hwctx->surfaces[i].Data.MemId == + (IDirect3DSurface9*)(uintptr_t)src->data[3]) + index = i; + } #endif } - if (i >= hwctx->nb_surfaces) { + if (index < 0) { av_log(dst_ctx, AV_LOG_ERROR, "Trying to map from a surface which " "is not in the mapped frames context.\n"); return AVERROR(EINVAL); @@ -1032,7 +1144,7 @@ static int qsv_map_to(AVHWFramesContext *dst_ctx, dst->width = src->width; dst->height = src->height; - dst->data[3] = (uint8_t*)&hwctx->surfaces[i]; + dst->data[3] = (uint8_t*)&hwctx->surfaces[index]; return 0; } @@ -1074,7 +1186,7 @@ static void qsv_device_free(AVHWDeviceContext *ctx) av_freep(&priv); } -static mfxIMPL choose_implementation(const char *device) +static mfxIMPL choose_implementation(const char *device, enum AVHWDeviceType child_device_type) { static const struct { const char *name; @@ -1103,6 +1215,10 @@ static mfxIMPL choose_implementation(const char *device) impl = strtol(device, NULL, 0); } + if ( (child_device_type == AV_HWDEVICE_TYPE_D3D11VA) && (impl != MFX_IMPL_SOFTWARE) ) { + impl |= MFX_IMPL_VIA_D3D11; + } + return impl; } @@ -1129,6 +1245,15 @@ static int qsv_device_derive_from_child(AVHWDeviceContext *ctx, } break; #endif +#if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: + { + AVD3D11VADeviceContext *child_device_hwctx = child_device_ctx->hwctx; + handle_type = MFX_HANDLE_D3D11_DEVICE; + handle = (mfxHDL)child_device_hwctx->device; + } + break; +#endif #if CONFIG_DXVA2 case AV_HWDEVICE_TYPE_DXVA2: { @@ -1191,7 +1316,9 @@ fail: static int qsv_device_derive(AVHWDeviceContext *ctx, AVHWDeviceContext *child_device_ctx, int flags) { - return qsv_device_derive_from_child(ctx, MFX_IMPL_HARDWARE_ANY, + mfxIMPL impl; + impl = choose_implementation("hw_any", child_device_ctx->type); + return qsv_device_derive_from_child(ctx, impl, child_device_ctx, flags); } @@ -1214,35 +1341,66 @@ static int qsv_device_create(AVHWDeviceContext *ctx, const char *device, ctx->user_opaque = priv; ctx->free = qsv_device_free; - e = av_dict_get(opts, "child_device", NULL, 0); - - child_device_opts = NULL; - if (CONFIG_VAAPI) { + e = av_dict_get(opts, "child_device_type", NULL, 0); + if (e) { + child_device_type = av_hwdevice_find_type_by_name(e ? e->value : NULL); + if (child_device_type == AV_HWDEVICE_TYPE_NONE) { + av_log(ctx, AV_LOG_ERROR, "Unknown child device type " + "\"%s\".\n", e ? e->value : NULL); + return AVERROR(EINVAL); + } + } else if (CONFIG_VAAPI) { child_device_type = AV_HWDEVICE_TYPE_VAAPI; - // libmfx does not actually implement VAAPI properly, rather it - // depends on the specific behaviour of a matching iHD driver when - // used on recent Intel hardware. Set options to the VAAPI device - // creation so that we should pick a usable setup by default if - // possible, even when multiple devices and drivers are available. - av_dict_set(&child_device_opts, "kernel_driver", "i915", 0); - av_dict_set(&child_device_opts, "driver", "iHD", 0); - } else if (CONFIG_DXVA2) + } else if (CONFIG_D3D11VA) { + child_device_type = AV_HWDEVICE_TYPE_D3D11VA; + } else if (CONFIG_DXVA2) { child_device_type = AV_HWDEVICE_TYPE_DXVA2; - else { + } else { av_log(ctx, AV_LOG_ERROR, "No supported child device type is enabled\n"); return AVERROR(ENOSYS); } + child_device_opts = NULL; + switch (child_device_type) { +#if CONFIG_VAAPI + case AV_HWDEVICE_TYPE_VAAPI: + { + // libmfx does not actually implement VAAPI properly, rather it + // depends on the specific behaviour of a matching iHD driver when + // used on recent Intel hardware. Set options to the VAAPI device + // creation so that we should pick a usable setup by default if + // possible, even when multiple devices and drivers are available. + av_dict_set(&child_device_opts, "kernel_driver", "i915", 0); + av_dict_set(&child_device_opts, "driver", "iHD", 0); + } + break; +#endif +#if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: + break; +#endif +#if CONFIG_DXVA2 + case AV_HWDEVICE_TYPE_DXVA2: + break; +#endif + default: + { + av_log(ctx, AV_LOG_ERROR, "No supported child device type is enabled\n"); + return AVERROR(ENOSYS); + } + break; + } + + e = av_dict_get(opts, "child_device", NULL, 0); ret = av_hwdevice_ctx_create(&priv->child_device_ctx, child_device_type, e ? e->value : NULL, child_device_opts, 0); - av_dict_free(&child_device_opts); if (ret < 0) return ret; child_device = (AVHWDeviceContext*)priv->child_device_ctx->data; - impl = choose_implementation(device); + impl = choose_implementation(device, child_device_type); return qsv_device_derive_from_child(ctx, impl, child_device, 0); }