From patchwork Thu Apr 9 17:50:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: galinart X-Patchwork-Id: 18799 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id D87EB44A892 for ; Thu, 9 Apr 2020 20:59:53 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id ADB4368B884; Thu, 9 Apr 2020 20:59:53 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f68.google.com (mail-wm1-f68.google.com [209.85.128.68]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id EB35068B636 for ; Thu, 9 Apr 2020 20:59:46 +0300 (EEST) Received: by mail-wm1-f68.google.com with SMTP id d77so668989wmd.3 for ; Thu, 09 Apr 2020 10:59:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=kcpQRpim++WFb4IEEF0vf1NYi4Uq9qxxo7ScS/yPuSU=; b=N29opSmoZos06NXZqRaIBCh9z5gbeDSd78wp4dweA9cgolhMuQc6lAGHpKLdsIGj77 bF2DmeXw1N1we4upu+wHZ2eYnfZA51sHYGH0RuGFA9ReM1ZuHSzNmXs+/RRMm7K2UUWR Egd5uRmylO6XVAZB7Z0pc8i1+PKzdWMFtD9kSUczNKJWIc58I+fMeUOJdfs+BRRNNGz3 HvXKYr4oo028PO6IpclPfec3biYkekRJ4M/SMCivVujuzQDe/+aZmQT2kVjTb6uxcLy2 y9oxywppLuaymmPgl5g4uAIAoX+QNwX7Dpz0XPrN51EpdDeltgjK86MYqP03UOZQTIbC pUzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=kcpQRpim++WFb4IEEF0vf1NYi4Uq9qxxo7ScS/yPuSU=; b=nVGKHNVY/iBpvWyHriRFND/9ysSeSu4Ek8gPM1nxXpoI8dGfkx+sY3Xaz06MdXPBUH Bqau8M567iG90vz7GB7EG8Dez5QQZkXxmIrAt7j91l9Xv/vDb23ckkEQiUewLi7C4Eni 0gRQmXe1in5yvEkIfld7b47Yb2rtmHY7nxlswT0+ejaEbLbfV/CSlQqiLUbM0h36L96l 5jiYm4IwrMtdurl6t0w4mni4WffXR6qFWNQ3A7d2/c2CAb5o7TCjZ4xVapf0GggV6oHe ZjjkpLiXVpP0V8VoBH4JGCqU9f4Ka8J/mxEzw+ZhYXUsUOhAjstWBN7VMsaKGNKmc/q3 E2XA== X-Gm-Message-State: AGi0PuaydF6D6lCuvC2Ff2Mw0KPN47zQMb3qCk8jBc2D4shXCdBYP48s +k6blT9wfMqRmdAdXoOCXWfzgCI9SGg= X-Google-Smtp-Source: APiQypIyk6Hb/CLp5KvAZ6ZyK4R022Mf78qRStsfUFvQiA2uwhmuTnfWoO/lSQYu/EHkQ5uvb0g3Sg== X-Received: by 2002:a05:600c:2284:: with SMTP id 4mr975629wmf.103.1586454680827; Thu, 09 Apr 2020 10:51:20 -0700 (PDT) Received: from galinart-MOBL.ger.corp.intel.com ([90.219.218.77]) by smtp.googlemail.com with ESMTPSA id z11sm25622574wrv.58.2020.04.09.10.51.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Apr 2020 10:51:20 -0700 (PDT) From: artem.galin@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Thu, 9 Apr 2020 18:50:51 +0100 Message-Id: <20200409175054.23468-1-artem.galin@gmail.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH 1/4] fftools/qsv: enabling d3d11va/dxva2 device selection X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Artem Galin MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Artem Galin child_device_type argument is responsible for selection. Usage examples: -init_hw_device qsv:hw,child_device_type=d3d11va -init_hw_device qsv:hw,child_device_type=dxva2 Signed-off-by: Artem Galin --- fftools/ffmpeg_opt.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 95001a963f..82232c60b3 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -568,7 +568,17 @@ static int opt_init_hw_device(void *optctx, const char *opt, const char *arg) printf("\n"); exit_program(0); } else { - return hw_device_init_from_string(arg, NULL); + HWDevice *dev; + int err; + if (!arg) + return AVERROR(ENOMEM); + err = hw_device_init_from_string(arg, &dev); + if (err < 0) + return err; + hw_device_ctx = av_buffer_ref(dev->device_ref); + if (!hw_device_ctx) + return AVERROR(ENOMEM); + return 0; } } From patchwork Thu Apr 9 17:50:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: galinart X-Patchwork-Id: 18797 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 814DD4497A5 for ; Thu, 9 Apr 2020 20:51:29 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 55F8268B897; Thu, 9 Apr 2020 20:51:29 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f66.google.com (mail-wm1-f66.google.com [209.85.128.66]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A4A6C68B6D8 for ; Thu, 9 Apr 2020 20:51:22 +0300 (EEST) Received: by mail-wm1-f66.google.com with SMTP id h2so637747wmb.4 for ; Thu, 09 Apr 2020 10:51:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=v9dNu3MStEWEsdnI/cd1b0QQW4TIEb7QPvyMSUKqjfs=; b=I7Qq7cZ/1W7YD/oT0gyp80WZAVtR5VtRqrW7GEWB4kbWu3hZVB1ctq6axyQwtNMz/x mJrWcVMs1E80SRla9JDb16bSaZb4hY5WRXXTyxCj1hrTsNlf8HInINlOxX9vzfSx4AJt aj+IsArm+XNyX7dRsSmzQTv22EELfhxXi1Gyv5Oa1hvUPLwC5ReZY0D//5frlchaoXeT hNSxpSD6khu5QldZUwLfWNOzUR09XfQzwZaTAQqBDUb4TI5DjVmbfvXw5KChmG4TLUEX 3mHvk9lCBQdwfSq6qPQ64rsG7ivaiB/t4P7D2pL+LChwsmDMhJsiy+CzccNSlhlf19wX vu/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=v9dNu3MStEWEsdnI/cd1b0QQW4TIEb7QPvyMSUKqjfs=; b=DX2tNWRa7UGpmNe8S/G/xYQwPGWXLv4xOIuLlGVaagn5fqIlgOfYNqdSRyPPX99//b tzWyouHB3xySnd0AzRDhXtjuf4K9sqB5gMv5wNk2yiyMV8+JHaL1sOd5UImgEV0y73qC EXbJOoXlTbUOR/7mMV7UCkwRHrSCh4Mwn8xrXG9Hx8Gtq7+3I6aPysxpGUNGHWG+VxO5 W46m8IFmj36lJAsRgmitP1jvXxdsUUYXJpuujolBZUCxQmtKOapIr/epEpynHVFOtmhE o+Vg9WcLuiDUm54aQppZfulvAS0D4Bn1So6pPz4lS9nj0kAzVMgMPSUzI59RYtW8RAKt zx/A== X-Gm-Message-State: AGi0PuZ7zTu33qfCivZ6R5tlQHfcazTrd0wEMbeml4KNcmlUW8lnHWni hnOvCdIaviLS8JrxjXgoD7u2MgPFO7A= X-Google-Smtp-Source: APiQypIzQfeBcdngdih+RMxwQiO9F1uuqqVGYRabOO20yim/dFWtcDEcr9bZoIMSFxacx3N6h1T0Vg== X-Received: by 2002:a1c:9ad7:: with SMTP id c206mr1005409wme.48.1586454681918; Thu, 09 Apr 2020 10:51:21 -0700 (PDT) Received: from galinart-MOBL.ger.corp.intel.com ([90.219.218.77]) by smtp.googlemail.com with ESMTPSA id z11sm25622574wrv.58.2020.04.09.10.51.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Apr 2020 10:51:21 -0700 (PDT) From: artem.galin@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Thu, 9 Apr 2020 18:50:52 +0100 Message-Id: <20200409175054.23468-2-artem.galin@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200409175054.23468-1-artem.galin@gmail.com> References: <20200409175054.23468-1-artem.galin@gmail.com> Subject: [FFmpeg-devel] [PATCH 2/4] libavfilter/qsvvpp: enabling d3d11va support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Artem Galin MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Artem Galin Adding DX11 relevant device type checks and adjusting callback with proper MediaSDK pair type support. Signed-off-by: Artem Galin --- libavfilter/qsvvpp.c | 40 ++++++++++++++++++++++++++++++---------- 1 file changed, 30 insertions(+), 10 deletions(-) diff --git a/libavfilter/qsvvpp.c b/libavfilter/qsvvpp.c index 8d5ff2eb65..8c6c878bac 100644 --- a/libavfilter/qsvvpp.c +++ b/libavfilter/qsvvpp.c @@ -32,10 +32,11 @@ #include "qsvvpp.h" #include "video.h" -#define IS_VIDEO_MEMORY(mode) (mode & (MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET | \ +#define IS_VIDEO_MEMORY(mode) (mode & (MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET | \ MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET)) -#define IS_OPAQUE_MEMORY(mode) (mode & MFX_MEMTYPE_OPAQUE_FRAME) -#define IS_SYSTEM_MEMORY(mode) (mode & MFX_MEMTYPE_SYSTEM_MEMORY) +#define IS_OPAQUE_MEMORY(mode) (mode & MFX_MEMTYPE_OPAQUE_FRAME) +#define IS_SYSTEM_MEMORY(mode) (mode & MFX_MEMTYPE_SYSTEM_MEMORY) +#define MFX_IMPL_VIA_MASK(impl) (0x0f00 & (impl)) typedef struct QSVFrame { AVFrame *frame; @@ -129,7 +130,17 @@ static mfxStatus frame_unlock(mfxHDL pthis, mfxMemId mid, mfxFrameData *ptr) static mfxStatus frame_get_hdl(mfxHDL pthis, mfxMemId mid, mfxHDL *hdl) { +#if CONFIG_VAAPI *hdl = mid; +#else + mfxHDLPair *pair_dst = (mfxHDLPair*)hdl; + mfxHDLPair *pair_src = (mfxHDLPair*)mid; + + pair_dst->first = pair_src->first; + + if (pair_src->second != (mfxMemId)MFX_INFINITE) + pair_dst->second = pair_src->second; +#endif return MFX_ERR_NONE; } @@ -451,7 +462,7 @@ static int init_vpp_session(AVFilterContext *avctx, QSVVPPContext *s) s->out_mem_mode = IS_OPAQUE_MEMORY(s->in_mem_mode) ? MFX_MEMTYPE_OPAQUE_FRAME : - MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET; + MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET | MFX_MEMTYPE_FROM_VPPOUT; out_frames_ctx = (AVHWFramesContext *)out_frames_ref->data; out_frames_hwctx = out_frames_ctx->hwctx; @@ -497,15 +508,24 @@ static int init_vpp_session(AVFilterContext *avctx, QSVVPPContext *s) return AVERROR_UNKNOWN; } + if (MFX_IMPL_VIA_D3D11 == MFX_IMPL_VIA_MASK(impl)) { + handle_type = MFX_HANDLE_D3D11_DEVICE; + } else if (MFX_IMPL_VIA_D3D9 == MFX_IMPL_VIA_MASK(impl)) { + handle_type = MFX_HANDLE_D3D9_DEVICE_MANAGER; + } else if (MFX_IMPL_VIA_VAAPI == MFX_IMPL_VIA_MASK(impl)) { + handle_type = MFX_HANDLE_VA_DISPLAY; + } + for (i = 0; i < FF_ARRAY_ELEMS(handle_types); i++) { - ret = MFXVideoCORE_GetHandle(device_hwctx->session, handle_types[i], &handle); - if (ret == MFX_ERR_NONE) { - handle_type = handle_types[i]; - break; + if (handle_types[i] == handle_type) { + ret = MFXVideoCORE_GetHandle(device_hwctx->session, handle_types[i], &handle); + if (ret == MFX_ERR_NONE) { + break; + } } + handle = NULL; } - - if (ret != MFX_ERR_NONE) { + if (!handle) { av_log(avctx, AV_LOG_ERROR, "Error getting the session handle\n"); return AVERROR_UNKNOWN; } From patchwork Thu Apr 9 17:50:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: galinart X-Patchwork-Id: 18798 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 81D3344A3C0 for ; Thu, 9 Apr 2020 20:57:03 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 67AEF68B89B; Thu, 9 Apr 2020 20:57:03 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f67.google.com (mail-wm1-f67.google.com [209.85.128.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1995268B814 for ; Thu, 9 Apr 2020 20:56:57 +0300 (EEST) Received: by mail-wm1-f67.google.com with SMTP id d77so660894wmd.3 for ; Thu, 09 Apr 2020 10:56:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Dd/b5mwI9ji3tnFq3VJrGalL7oPEdVNdHMqrS/7Mljs=; b=aR5vrhEjZ7RrKOJOk59YOHZu35/RBnRynlxsJUl8LTaoYfw8ax4cLtIXaIjDz+iz7F XLoIOMGAv0xIKgTOSFfwTm0AGECpmWPvHACiR8mXfJucecKvXoYd8Z3CYBxQ2DY5lkBX gXJ+lp7zRS1H5CM4+/9xK1YEYPObp4eGZpB1CRl3h8kPrEmlhOLu7MJfYf5Ok1HGZy2g IqkRhgpZBy8KU9grK3DP5GhY2q4m8UNCocB6NyuT157vttfT9nGcYKTLCTWAre7MXg73 yxxZ/Bv2+rCwjR0v4KFo/tg5wJv3y0vPsaTl0jQGb9Z2XuCnwvgqVemvuDp+xVWJqHOA wSWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Dd/b5mwI9ji3tnFq3VJrGalL7oPEdVNdHMqrS/7Mljs=; b=h/6ZtIrWVZ26xI5/X5GB16vblGAxAoPhQCXTioDiUKTLy+HNHTLM8+MyP35icbiXhV uzAcnS5Fo1Y6CASla6YeCE/Z7xWJvkrdvmknd1wwGj7UeBf+MDeK45y1cMnX16eOk811 a3/XW/znj9hp8Z96XW7x5JGvCUoD66y/+HpWh9oKhX/ELScaxh8P6t4k4OdWKh0s0J77 abP+1IABS0CguwImtFQ8sJfv5lJHvee+EZQYE5Q7lwkGidh/iilytvpau8pKXp/t7Ur+ M75fPfvFHwDicvYnyIgllrYF0PxYDerluYiq+O15L53OGaBkTx6stYiExPH5w3aUX8PH KRFQ== X-Gm-Message-State: AGi0PubCD377jGWTHkKT1P6LjfupzCn7dEcFwDcPPFzV9E0/oFzepSpI B6Tzmegi4Hwzw/ljg5pPD7z3SCn3dtg= X-Google-Smtp-Source: APiQypLvQZ/HGwQQP2OlVDsPCSl0TuCzRaMqJSyaOYKHwUHhiIGh2+h4nwfqKv7WLqLOlulEb69ccQ== X-Received: by 2002:a1c:668a:: with SMTP id a132mr1069138wmc.46.1586454682908; Thu, 09 Apr 2020 10:51:22 -0700 (PDT) Received: from galinart-MOBL.ger.corp.intel.com ([90.219.218.77]) by smtp.googlemail.com with ESMTPSA id z11sm25622574wrv.58.2020.04.09.10.51.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Apr 2020 10:51:22 -0700 (PDT) From: artem.galin@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Thu, 9 Apr 2020 18:50:53 +0100 Message-Id: <20200409175054.23468-3-artem.galin@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200409175054.23468-1-artem.galin@gmail.com> References: <20200409175054.23468-1-artem.galin@gmail.com> Subject: [FFmpeg-devel] [PATCH 3/4] libavcodec/qsv: enabling d3d11va support, added mfxhdlpair X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Artem Galin MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Artem Galin Adding DX11 relevant device type checks and adjusting callbacks with proper MediaSDK pair type support. Extending structure for proper MediaSDK pair type support. Signed-off-by: Artem Galin --- libavcodec/qsv.c | 52 ++++++++++++++++++++++++++++++++++++--- libavcodec/qsv_internal.h | 1 + 2 files changed, 49 insertions(+), 4 deletions(-) diff --git a/libavcodec/qsv.c b/libavcodec/qsv.c index db98c75073..ceef1b7fd9 100644 --- a/libavcodec/qsv.c +++ b/libavcodec/qsv.c @@ -36,6 +36,8 @@ #include "avcodec.h" #include "qsv_internal.h" +#define MFX_IMPL_VIA_MASK(impl) (0x0f00 & (impl)) + #if QSV_VERSION_ATLEAST(1, 12) #include "mfx/mfxvp8.h" #endif @@ -221,8 +223,15 @@ int ff_qsv_find_surface_idx(QSVFramesContext *ctx, QSVFrame *frame) int i; for (i = 0; i < ctx->nb_mids; i++) { QSVMid *mid = &ctx->mids[i]; +#if CONFIG_VAAPI if (mid->handle == frame->surface.Data.MemId) return i; +#else + mfxHDLPair *pair = (mfxHDLPair*)frame->surface.Data.MemId; + if ((mid->handle_pair.first == pair->first) && + (mid->handle_pair.second == pair->second)) + return i; +#endif } return AVERROR_BUG; } @@ -362,7 +371,11 @@ static int ff_qsv_set_display_handle(AVCodecContext *avctx, QSVSession *qs) int ff_qsv_init_internal_session(AVCodecContext *avctx, QSVSession *qs, const char *load_plugins, int gpu_copy) { +#ifdef AVCODEC_QSV_LINUX_SESSION_HANDLE mfxIMPL impl = MFX_IMPL_AUTO_ANY; +#else + mfxIMPL impl = MFX_IMPL_AUTO_ANY | MFX_IMPL_VIA_D3D11; +#endif mfxVersion ver = { { QSV_VERSION_MINOR, QSV_VERSION_MAJOR } }; mfxInitParam init_par = { MFX_IMPL_AUTO_ANY }; @@ -449,11 +462,19 @@ static AVBufferRef *qsv_create_mids(AVBufferRef *hw_frames_ref) return NULL; } +#if CONFIG_VAAPI for (i = 0; i < nb_surfaces; i++) { QSVMid *mid = &mids[i]; mid->handle = frames_hwctx->surfaces[i].Data.MemId; mid->hw_frames_ref = hw_frames_ref1; } +#else + for (i = 0; i < nb_surfaces; i++) { + QSVMid *mid = &mids[i]; + mid->handle_pair = *((mfxHDLPair*)frames_hwctx->surfaces[i].Data.MemId); + mid->hw_frames_ref = hw_frames_ref1; + } +#endif return mids_buf; } @@ -628,7 +649,11 @@ static mfxStatus qsv_frame_lock(mfxHDL pthis, mfxMemId mid, mfxFrameData *ptr) goto fail; qsv_mid->surf.Info = hw_frames_hwctx->surfaces[0].Info; +#if CONFIG_VAAPI qsv_mid->surf.Data.MemId = qsv_mid->handle; +#else + qsv_mid->surf.Data.MemId = &qsv_mid->handle_pair; +#endif /* map the data to the system memory */ ret = av_hwframe_map(qsv_mid->locked_frame, qsv_mid->hw_frame, @@ -661,7 +686,17 @@ static mfxStatus qsv_frame_unlock(mfxHDL pthis, mfxMemId mid, mfxFrameData *ptr) static mfxStatus qsv_frame_get_hdl(mfxHDL pthis, mfxMemId mid, mfxHDL *hdl) { QSVMid *qsv_mid = (QSVMid*)mid; +#if CONFIG_VAAPI *hdl = qsv_mid->handle; +#else + mfxHDLPair *pair_dst = (mfxHDLPair*)hdl; + mfxHDLPair *pair_src = (mfxHDLPair*)&qsv_mid->handle_pair; + + pair_dst->first = pair_src->first; + + if (pair_src->second != (mfxMemId)MFX_INFINITE) + pair_dst->second = pair_src->second; +#endif return MFX_ERR_NONE; } @@ -695,11 +730,20 @@ int ff_qsv_init_session_device(AVCodecContext *avctx, mfxSession *psession, return ff_qsv_print_error(avctx, err, "Error querying the session attributes"); + if (MFX_IMPL_VIA_D3D11 == MFX_IMPL_VIA_MASK(impl)) { + handle_type = MFX_HANDLE_D3D11_DEVICE; + } else if (MFX_IMPL_VIA_D3D9 == MFX_IMPL_VIA_MASK(impl)) { + handle_type = MFX_HANDLE_D3D9_DEVICE_MANAGER; + } else if (MFX_IMPL_VIA_VAAPI == MFX_IMPL_VIA_MASK(impl)) { + handle_type = MFX_HANDLE_VA_DISPLAY; + } + for (i = 0; i < FF_ARRAY_ELEMS(handle_types); i++) { - err = MFXVideoCORE_GetHandle(parent_session, handle_types[i], &handle); - if (err == MFX_ERR_NONE) { - handle_type = handle_types[i]; - break; + if (handle_types[i] == handle_type) { + err = MFXVideoCORE_GetHandle(parent_session, handle_types[i], &handle); + if (err == MFX_ERR_NONE) { + break; + } } handle = NULL; } diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h index 6489836a67..7a4a66e9d6 100644 --- a/libavcodec/qsv_internal.h +++ b/libavcodec/qsv_internal.h @@ -61,6 +61,7 @@ typedef struct QSVMid { AVBufferRef *hw_frames_ref; mfxHDL handle; + mfxHDLPair handle_pair; AVFrame *locked_frame; AVFrame *hw_frame; From patchwork Thu Apr 9 17:50:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: galinart X-Patchwork-Id: 18800 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 404E144B09F for ; Thu, 9 Apr 2020 21:14:10 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1B67168B867; Thu, 9 Apr 2020 21:14:10 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f50.google.com (mail-wr1-f50.google.com [209.85.221.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C2C9368B67E for ; Thu, 9 Apr 2020 21:14:02 +0300 (EEST) Received: by mail-wr1-f50.google.com with SMTP id g3so13025811wrx.2 for ; Thu, 09 Apr 2020 11:14:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=eqO7LjEB/M87Dwdx72H7vvNiFCD9LapR15G51QvP6rw=; b=Ngk0uLysXmmMKH0CMkItwTEaYMHFAHwSnkpSyRGIJ+05ychKX+PzzjY3jb2EZFiV0U I9G12i4vBo6ENjQ5AAXZlNKXOJyQffiWD80g4mnpLJvSEQiWJk9+h0HbbYKl8qbNdEbE zRE7Vqu7EWFqYVFk1u0W8YrGHgP0ARRL0lWDl4X2bZzyWJYOlWR5Xt/OPZ7DQcSF5FUu 9jgpE9Xbjoi2KxIyzqHW6rokJOOa//0+W9/Kr57XIcw1Ci0J/6qYZQoelyDmXYHzMUpc 0AUOiHBlfC1XRkAzAJ9sj0PJwx7ZFq1SquzmikiVg3tfByLg3ss3ao5J2hcakUaUiJCE bNmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=eqO7LjEB/M87Dwdx72H7vvNiFCD9LapR15G51QvP6rw=; b=luTdF/hURZiEVplG1jfaD1vC46nGjtkqZ+n2fWd7ipF/yI8kmrA7cpf5Q6CbEg47ip ULwR/nl9FflrusXrq9Q16Tv5M8cJHVCGjLZwR0/jawS0/hxHLfLZitvn5PpanoElHhvo rukIyuvlHkiDrSK+eV3uvL8FdpWyYdrZmG7nCbbMuwr5mdqhLEvty93/6UdGFo9CSTbZ ovzaVvdr3xZc6YpRjFvhpiehm6q4Aku7q86T/iYHXadRpiyvPdy5SSWx/RVHceAdC0Ae cuakpXqKB8iii3DnzaFg0E0BOlm3gmzSTZ9AFjX7SX4DKHEy8m3Vd2rgTWiU7xATwJUr 47NQ== X-Gm-Message-State: AGi0PuaSNuBGvmuhfX4Sm0iW4Pc8J69fSHSlzTaxBEYbl/RaAo28zrNV C8ZEcMHNQpN4x6k3/07V9FWXqAp+dOo= X-Google-Smtp-Source: APiQypKjiS9EcEL1KcjgRw9ehegrm7tQLSaeSDHOXzpCCAr6MjRFsgVRC/m+fgDD+DJyu4NZeAzC/g== X-Received: by 2002:adf:fa85:: with SMTP id h5mr292041wrr.63.1586454685100; Thu, 09 Apr 2020 10:51:25 -0700 (PDT) Received: from galinart-MOBL.ger.corp.intel.com ([90.219.218.77]) by smtp.googlemail.com with ESMTPSA id z11sm25622574wrv.58.2020.04.09.10.51.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Apr 2020 10:51:24 -0700 (PDT) From: artem.galin@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Thu, 9 Apr 2020 18:50:54 +0100 Message-Id: <20200409175054.23468-4-artem.galin@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200409175054.23468-1-artem.galin@gmail.com> References: <20200409175054.23468-1-artem.galin@gmail.com> Subject: [FFmpeg-devel] [PATCH 4/4] libavutil/qsv: enabling d3d11va support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Artem Galin MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Artem Galin Makes selection of d3d11va device type by default and over DirectX 9, which is still supported but requires explicit selection. This enables usage of non-powered/headless GPU, better HDR support. Pool of resources is allocated as one texture with array of slices. Added d3d11va device selection by vendor id. Example: --init_hw_device d3d11va:,vendor=0x8086 DirectX 9 usage. Example: --init_hw_device qsv:hw,child_device_type=0x8086 Signed-off-by: Artem Galin --- libavutil/hwcontext_d3d11va.c | 82 +++++++- libavutil/hwcontext_d3d11va.h | 8 + libavutil/hwcontext_qsv.c | 339 +++++++++++++++++++++++++++++----- 3 files changed, 371 insertions(+), 58 deletions(-) diff --git a/libavutil/hwcontext_d3d11va.c b/libavutil/hwcontext_d3d11va.c index c8ae58f908..9d7615eb55 100644 --- a/libavutil/hwcontext_d3d11va.c +++ b/libavutil/hwcontext_d3d11va.c @@ -72,7 +72,7 @@ static av_cold void load_functions(void) } typedef struct D3D11VAFramesContext { - int nb_surfaces_used; + int nb_surfaces; DXGI_FORMAT format; @@ -112,6 +112,8 @@ static void d3d11va_frames_uninit(AVHWFramesContext *ctx) if (s->staging_texture) ID3D11Texture2D_Release(s->staging_texture); s->staging_texture = NULL; + + av_freep(&frames_hwctx->texture_infos); } static int d3d11va_frames_get_constraints(AVHWDeviceContext *ctx, @@ -152,8 +154,9 @@ static void free_texture(void *opaque, uint8_t *data) av_free(data); } -static AVBufferRef *wrap_texture_buf(ID3D11Texture2D *tex, int index) +static AVBufferRef *wrap_texture_buf(AVHWFramesContext *ctx, ID3D11Texture2D *tex, int index) { + AVD3D11VAFramesContext *frames_hwctx = ctx->hwctx; AVBufferRef *buf; AVD3D11FrameDescriptor *desc = av_mallocz(sizeof(*desc)); if (!desc) { @@ -161,6 +164,10 @@ static AVBufferRef *wrap_texture_buf(ID3D11Texture2D *tex, int index) return NULL; } + frames_hwctx->texture_infos[frames_hwctx->nb_surfaces_used].texture = tex; + frames_hwctx->texture_infos[frames_hwctx->nb_surfaces_used].index = index; + frames_hwctx->nb_surfaces_used++; + desc->texture = tex; desc->index = index; @@ -199,13 +206,12 @@ static AVBufferRef *d3d11va_alloc_single(AVHWFramesContext *ctx) return NULL; } - return wrap_texture_buf(tex, 0); + return wrap_texture_buf(ctx, tex, 0); } static AVBufferRef *d3d11va_pool_alloc(void *opaque, int size) { AVHWFramesContext *ctx = (AVHWFramesContext*)opaque; - D3D11VAFramesContext *s = ctx->internal->priv; AVD3D11VAFramesContext *hwctx = ctx->hwctx; D3D11_TEXTURE2D_DESC texDesc; @@ -214,13 +220,13 @@ static AVBufferRef *d3d11va_pool_alloc(void *opaque, int size) ID3D11Texture2D_GetDesc(hwctx->texture, &texDesc); - if (s->nb_surfaces_used >= texDesc.ArraySize) { + if (hwctx->nb_surfaces_used >= texDesc.ArraySize) { av_log(ctx, AV_LOG_ERROR, "Static surface pool size exceeded.\n"); return NULL; } ID3D11Texture2D_AddRef(hwctx->texture); - return wrap_texture_buf(hwctx->texture, s->nb_surfaces_used++); + return wrap_texture_buf(ctx, hwctx->texture, hwctx->nb_surfaces_used); } static int d3d11va_frames_init(AVHWFramesContext *ctx) @@ -267,7 +273,7 @@ static int d3d11va_frames_init(AVHWFramesContext *ctx) av_log(ctx, AV_LOG_ERROR, "User-provided texture has mismatching parameters\n"); return AVERROR(EINVAL); } - } else if (texDesc.ArraySize > 0) { + } else if (!(texDesc.BindFlags & D3D11_BIND_RENDER_TARGET) && texDesc.ArraySize > 0) { hr = ID3D11Device_CreateTexture2D(device_hwctx->device, &texDesc, NULL, &hwctx->texture); if (FAILED(hr)) { av_log(ctx, AV_LOG_ERROR, "Could not create the texture (%lx)\n", (long)hr); @@ -275,6 +281,12 @@ static int d3d11va_frames_init(AVHWFramesContext *ctx) } } + hwctx->texture_infos = av_mallocz_array(ctx->initial_pool_size, sizeof(*hwctx->texture_infos)); + if (!hwctx->texture_infos) + return AVERROR(ENOMEM); + + s->nb_surfaces = ctx->initial_pool_size; + ctx->internal->pool_internal = av_buffer_pool_init2(sizeof(AVD3D11FrameDescriptor), ctx, d3d11va_pool_alloc, NULL); if (!ctx->internal->pool_internal) @@ -511,15 +523,56 @@ static void d3d11va_device_uninit(AVHWDeviceContext *hwdev) } } +static int d3d11va_device_find_adapter_by_vendor_id(AVHWDeviceContext *ctx, UINT creationFlags, char *vendor) +{ + HRESULT hr; + IDXGIAdapter *adapter = NULL; + int adapter_id = 0; + IDXGIFactory2 *factory; + long int vendor_id = strtol(vendor, NULL, 0); + hr = mCreateDXGIFactory(&IID_IDXGIFactory2, (void **)&factory); + while (IDXGIFactory2_EnumAdapters(factory, adapter_id++, &adapter) != DXGI_ERROR_NOT_FOUND) { + ID3D11Device* device = NULL; + DXGI_ADAPTER_DESC adapter_desc; + + hr = mD3D11CreateDevice(adapter, D3D_DRIVER_TYPE_UNKNOWN, NULL, creationFlags, NULL, 0, D3D11_SDK_VERSION, &device, NULL, NULL); + if (FAILED(hr)) { + av_log(ctx, AV_LOG_ERROR, "D3D11CreateDevice returned error\n"); + continue; + } + + hr = IDXGIAdapter2_GetDesc(adapter, &adapter_desc); + if (FAILED(hr)) { + av_log(ctx, AV_LOG_ERROR, "IDXGIAdapter2_GetDesc returned error\n"); + continue; + } + + if (device) + ID3D11Device_Release(device); + + if (adapter) + IDXGIAdapter_Release(adapter); + + if (adapter_desc.VendorId == vendor_id) { + IDXGIFactory2_Release(factory); + return adapter_id - 1; + } + } + IDXGIFactory2_Release(factory); + return -1; +} + static int d3d11va_device_create(AVHWDeviceContext *ctx, const char *device, AVDictionary *opts, int flags) { AVD3D11VADeviceContext *device_hwctx = ctx->hwctx; HRESULT hr; + AVDictionaryEntry *e; IDXGIAdapter *pAdapter = NULL; ID3D10Multithread *pMultithread; UINT creationFlags = D3D11_CREATE_DEVICE_VIDEO_SUPPORT; + int adapter = -1; int is_debug = !!av_dict_get(opts, "debug", NULL, 0); int ret; @@ -539,11 +592,23 @@ static int d3d11va_device_create(AVHWDeviceContext *ctx, const char *device, return AVERROR_UNKNOWN; } + e = av_dict_get(opts, "vendor", NULL, 0); + if (e) { + adapter = d3d11va_device_find_adapter_by_vendor_id(ctx, creationFlags, e ? e->value : NULL); + if (adapter < 0) { + av_log(ctx, AV_LOG_ERROR, "Failed to find d3d11va adapter by vendor id %s\n", e ? e->value : NULL); + return AVERROR_UNKNOWN; + } + } + if (device) { + adapter = atoi(device); + } + + if (adapter >= 0) { IDXGIFactory2 *pDXGIFactory; hr = mCreateDXGIFactory(&IID_IDXGIFactory2, (void **)&pDXGIFactory); if (SUCCEEDED(hr)) { - int adapter = atoi(device); if (FAILED(IDXGIFactory2_EnumAdapters(pDXGIFactory, adapter, &pAdapter))) pAdapter = NULL; IDXGIFactory2_Release(pDXGIFactory); @@ -568,6 +633,7 @@ static int d3d11va_device_create(AVHWDeviceContext *ctx, const char *device, return AVERROR_UNKNOWN; } + av_log(ctx, AV_LOG_VERBOSE, "Using D3D11 device.\n"); hr = ID3D11Device_QueryInterface(device_hwctx->device, &IID_ID3D10Multithread, (void **)&pMultithread); if (SUCCEEDED(hr)) { ID3D10Multithread_SetMultithreadProtected(pMultithread, TRUE); diff --git a/libavutil/hwcontext_d3d11va.h b/libavutil/hwcontext_d3d11va.h index 9f91e9b1b6..4312fe0b62 100644 --- a/libavutil/hwcontext_d3d11va.h +++ b/libavutil/hwcontext_d3d11va.h @@ -39,6 +39,11 @@ #include #include +typedef struct D3D11TextureInfo { + ID3D11Texture2D *texture; + int index; +} D3D11TextureInfo; + /** * This struct is allocated as AVHWDeviceContext.hwctx */ @@ -164,6 +169,9 @@ typedef struct AVD3D11VAFramesContext { * This field is ignored/invalid if a user-allocated texture is provided. */ UINT MiscFlags; + + D3D11TextureInfo *texture_infos; + int nb_surfaces_used; } AVD3D11VAFramesContext; #endif /* AVUTIL_HWCONTEXT_D3D11VA_H */ diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index b1b67400de..1bc02c41eb 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -27,9 +27,13 @@ #include #endif +#define COBJMACROS #if CONFIG_VAAPI #include "hwcontext_vaapi.h" #endif +#if CONFIG_D3D11VA +#include "hwcontext_d3d11va.h" +#endif #if CONFIG_DXVA2 #include "hwcontext_dxva2.h" #endif @@ -44,6 +48,8 @@ #include "pixdesc.h" #include "time.h" +#define MFX_IMPL_VIA_MASK(impl) (0x0f00 & (impl)) + typedef struct QSVDevicePriv { AVBufferRef *child_device_ctx; } QSVDevicePriv; @@ -70,6 +76,7 @@ typedef struct QSVFramesContext { AVBufferRef *child_frames_ref; mfxFrameSurface1 *surfaces_internal; + mfxHDLPair *handle_pairs_internal; int nb_surfaces_used; // used in the frame allocator for non-opaque surfaces @@ -89,6 +96,9 @@ static const struct { #if CONFIG_VAAPI { MFX_HANDLE_VA_DISPLAY, AV_HWDEVICE_TYPE_VAAPI, AV_PIX_FMT_VAAPI }, #endif +#if CONFIG_D3D11VA + { MFX_HANDLE_D3D11_DEVICE, AV_HWDEVICE_TYPE_D3D11VA, AV_PIX_FMT_D3D11 }, +#endif #if CONFIG_DXVA2 { MFX_HANDLE_D3D9_DEVICE_MANAGER, AV_HWDEVICE_TYPE_DXVA2, AV_PIX_FMT_DXVA2_VLD }, #endif @@ -115,29 +125,32 @@ static uint32_t qsv_fourcc_from_pix_fmt(enum AVPixelFormat pix_fmt) return 0; } +#if CONFIG_D3D11VA +static uint32_t qsv_get_d3d11va_bind_flags(int mem_type) +{ + uint32_t bind_flags = 0; + + if ((mem_type & MFX_MEMTYPE_VIDEO_MEMORY_ENCODER_TARGET) && (mem_type & MFX_MEMTYPE_INTERNAL_FRAME)) + bind_flags = D3D11_BIND_DECODER | D3D11_BIND_VIDEO_ENCODER; + else + bind_flags = D3D11_BIND_DECODER; + + if ((MFX_MEMTYPE_FROM_VPPOUT & mem_type) || (MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET & mem_type)) + bind_flags = D3D11_BIND_RENDER_TARGET; + + return bind_flags; +} +#endif + static int qsv_device_init(AVHWDeviceContext *ctx) { AVQSVDeviceContext *hwctx = ctx->hwctx; QSVDeviceContext *s = ctx->internal->priv; + mfxHandleType handle_type = MFX_HANDLE_D3D11_DEVICE; mfxStatus err; int i; - for (i = 0; supported_handle_types[i].handle_type; i++) { - err = MFXVideoCORE_GetHandle(hwctx->session, supported_handle_types[i].handle_type, - &s->handle); - if (err == MFX_ERR_NONE) { - s->handle_type = supported_handle_types[i].handle_type; - s->child_device_type = supported_handle_types[i].device_type; - s->child_pix_fmt = supported_handle_types[i].pix_fmt; - break; - } - } - if (!s->handle) { - av_log(ctx, AV_LOG_VERBOSE, "No supported hw handle could be retrieved " - "from the session\n"); - } - err = MFXQueryIMPL(hwctx->session, &s->impl); if (err == MFX_ERR_NONE) err = MFXQueryVersion(hwctx->session, &s->ver); @@ -146,6 +159,31 @@ static int qsv_device_init(AVHWDeviceContext *ctx) return AVERROR_UNKNOWN; } + if (MFX_IMPL_VIA_D3D11 == MFX_IMPL_VIA_MASK(s->impl)) { + handle_type = MFX_HANDLE_D3D11_DEVICE; + } else if (MFX_IMPL_VIA_D3D9 == MFX_IMPL_VIA_MASK(s->impl)) { + handle_type = MFX_HANDLE_D3D9_DEVICE_MANAGER; + } else if (MFX_IMPL_VIA_VAAPI == MFX_IMPL_VIA_MASK(s->impl)) { + handle_type = MFX_HANDLE_VA_DISPLAY; + } + + for (i = 0; supported_handle_types[i].handle_type; i++) { + if (supported_handle_types[i].handle_type == handle_type) { + err = MFXVideoCORE_GetHandle(hwctx->session, supported_handle_types[i].handle_type, + &s->handle); + if (err == MFX_ERR_NONE) { + s->handle_type = supported_handle_types[i].handle_type; + s->child_device_type = supported_handle_types[i].device_type; + s->child_pix_fmt = supported_handle_types[i].pix_fmt; + break; + } + } + } + if (!s->handle) { + av_log(ctx, AV_LOG_VERBOSE, "No supported hw handle could be retrieved " + "from the session\n"); + } + return 0; } @@ -175,6 +213,7 @@ static void qsv_frames_uninit(AVHWFramesContext *ctx) av_freep(&s->mem_ids); av_freep(&s->surface_ptrs); av_freep(&s->surfaces_internal); + av_freep(&s->handle_pairs_internal); av_buffer_unref(&s->child_frames_ref); } @@ -190,6 +229,8 @@ static AVBufferRef *qsv_pool_alloc(void *opaque, int size) if (s->nb_surfaces_used < hwctx->nb_surfaces) { s->nb_surfaces_used++; + av_buffer_create((uint8_t*)(s->handle_pairs_internal + s->nb_surfaces_used - 1), + sizeof(*s->handle_pairs_internal), qsv_pool_release_dummy, NULL, 0); return av_buffer_create((uint8_t*)(s->surfaces_internal + s->nb_surfaces_used - 1), sizeof(*hwctx->surfaces), qsv_pool_release_dummy, NULL, 0); } @@ -229,6 +270,13 @@ static int qsv_init_child_ctx(AVHWFramesContext *ctx) child_device_hwctx->display = (VADisplay)device_priv->handle; } #endif +#if CONFIG_D3D11VA + if (child_device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) { + AVD3D11VADeviceContext *child_device_hwctx = child_device_ctx->hwctx; + ID3D11Device_AddRef((ID3D11Device*)device_priv->handle); + child_device_hwctx->device = (ID3D11Device*)device_priv->handle; + } +#endif #if CONFIG_DXVA2 if (child_device_ctx->type == AV_HWDEVICE_TYPE_DXVA2) { AVDXVA2DeviceContext *child_device_hwctx = child_device_ctx->hwctx; @@ -255,6 +303,16 @@ static int qsv_init_child_ctx(AVHWFramesContext *ctx) child_frames_ctx->width = FFALIGN(ctx->width, 16); child_frames_ctx->height = FFALIGN(ctx->height, 16); +#if CONFIG_D3D11VA + if (child_device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) { + AVD3D11VAFramesContext *child_frames_hwctx = child_frames_ctx->hwctx; + if (hwctx->frame_type == 0) + hwctx->frame_type = MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET; + if (hwctx->frame_type & MFX_MEMTYPE_SHARED_RESOURCE) + child_frames_hwctx->MiscFlags = D3D11_RESOURCE_MISC_SHARED; + child_frames_hwctx->BindFlags = qsv_get_d3d11va_bind_flags(hwctx->frame_type); + } +#endif #if CONFIG_DXVA2 if (child_device_ctx->type == AV_HWDEVICE_TYPE_DXVA2) { AVDXVA2FramesContext *child_frames_hwctx = child_frames_ctx->hwctx; @@ -279,11 +337,33 @@ static int qsv_init_child_ctx(AVHWFramesContext *ctx) hwctx->frame_type = MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET; } #endif +#if CONFIG_D3D11VA + if (child_device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) { + AVD3D11VAFramesContext *child_frames_hwctx = child_frames_ctx->hwctx; + for (i = 0; i < ctx->initial_pool_size; i++) { + s->handle_pairs_internal[i].first = (mfxMemId)child_frames_hwctx->texture_infos[i].texture; + if(child_frames_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { + s->handle_pairs_internal[i].second = (mfxMemId)MFX_INFINITE; + } else { + s->handle_pairs_internal[i].second = (mfxMemId)child_frames_hwctx->texture_infos[i].index; + } + s->surfaces_internal[i].Data.MemId = (mfxMemId)&s->handle_pairs_internal[i]; + } + if (child_frames_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { + hwctx->frame_type |= MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET; + } else { + hwctx->frame_type |= MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET; + } + } +#endif #if CONFIG_DXVA2 if (child_device_ctx->type == AV_HWDEVICE_TYPE_DXVA2) { AVDXVA2FramesContext *child_frames_hwctx = child_frames_ctx->hwctx; - for (i = 0; i < ctx->initial_pool_size; i++) - s->surfaces_internal[i].Data.MemId = (mfxMemId)child_frames_hwctx->surfaces[i]; + for (i = 0; i < ctx->initial_pool_size; i++) { + s->handle_pairs_internal[i].first = (mfxMemId)child_frames_hwctx->surfaces[i]; + s->handle_pairs_internal[i].second = (mfxMemId)MFX_INFINITE; + s->surfaces_internal[i].Data.MemId = (mfxMemId)&s->handle_pairs_internal[i]; + } if (child_frames_hwctx->surface_type == DXVA2_VideoProcessorRenderTarget) hwctx->frame_type = MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET; else @@ -348,6 +428,11 @@ static int qsv_init_pool(AVHWFramesContext *ctx, uint32_t fourcc) return AVERROR(EINVAL); } + s->handle_pairs_internal = av_mallocz_array(ctx->initial_pool_size, + sizeof(*s->handle_pairs_internal)); + if (!s->handle_pairs_internal) + return AVERROR(ENOMEM); + s->surfaces_internal = av_mallocz_array(ctx->initial_pool_size, sizeof(*s->surfaces_internal)); if (!s->surfaces_internal) @@ -421,7 +506,17 @@ static mfxStatus frame_unlock(mfxHDL pthis, mfxMemId mid, mfxFrameData *ptr) static mfxStatus frame_get_hdl(mfxHDL pthis, mfxMemId mid, mfxHDL *hdl) { +#if CONFIG_VAAPI *hdl = mid; +#else + mfxHDLPair *pair_dst = (mfxHDLPair*)hdl; + mfxHDLPair *pair_src = (mfxHDLPair*)mid; + + pair_dst->first = pair_src->first; + + if (pair_src->second != (mfxMemId)MFX_INFINITE) + pair_dst->second = pair_src->second; +#endif return MFX_ERR_NONE; } @@ -621,6 +716,18 @@ static int qsv_frames_derive_from(AVHWFramesContext *dst_ctx, } break; #endif +#if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: + { + AVD3D11VAFramesContext *dst_hwctx = dst_ctx->hwctx; + mfxHDLPair *pair = (mfxHDLPair*)src_hwctx->surfaces[i].Data.MemId; + dst_hwctx->texture = (ID3D11Texture2D*)pair->first; + if (src_hwctx->frame_type & MFX_MEMTYPE_SHARED_RESOURCE) + dst_hwctx->MiscFlags = D3D11_RESOURCE_MISC_SHARED; + dst_hwctx->BindFlags = qsv_get_d3d11va_bind_flags(src_hwctx->frame_type); + } + break; +#endif #if CONFIG_DXVA2 case AV_HWDEVICE_TYPE_DXVA2: { @@ -629,9 +736,10 @@ static int qsv_frames_derive_from(AVHWFramesContext *dst_ctx, sizeof(*dst_hwctx->surfaces)); if (!dst_hwctx->surfaces) return AVERROR(ENOMEM); - for (i = 0; i < src_hwctx->nb_surfaces; i++) - dst_hwctx->surfaces[i] = - (IDirect3DSurface9*)src_hwctx->surfaces[i].Data.MemId; + for (i = 0; i < src_hwctx->nb_surfaces; i++) { + mfxHDLPair *pair = (mfxHDLPair*)src_hwctx->surfaces[i].Data.MemId; + dst_hwctx->surfaces[i] = (IDirect3DSurface9*)pair->first; + } dst_hwctx->nb_surfaces = src_hwctx->nb_surfaces; if (src_hwctx->frame_type == MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET) dst_hwctx->surface_type = DXVA2_VideoDecoderRenderTarget; @@ -668,10 +776,21 @@ static int qsv_map_from(AVHWFramesContext *ctx, child_data = (uint8_t*)(intptr_t)*(VASurfaceID*)surf->Data.MemId; break; #endif +#if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: + { + mfxHDLPair *pair = (mfxHDLPair*)surf->Data.MemId; + child_data = pair->first; + break; + } +#endif #if CONFIG_DXVA2 case AV_HWDEVICE_TYPE_DXVA2: - child_data = surf->Data.MemId; + { + mfxHDLPair *pair = (mfxHDLPair*)surf->Data.MemId; + child_data = pair->first; break; + } #endif default: return AVERROR(ENOSYS); @@ -685,7 +804,14 @@ static int qsv_map_from(AVHWFramesContext *ctx, dst->width = src->width; dst->height = src->height; - dst->data[3] = child_data; + + if (child_frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) { + mfxHDLPair *pair = (mfxHDLPair*)surf->Data.MemId; + dst->data[0] = pair->first; + dst->data[1] = pair->second; + } else { + dst->data[3] = child_data; + } return 0; } @@ -708,7 +834,14 @@ static int qsv_map_from(AVHWFramesContext *ctx, dummy->format = child_frames_ctx->format; dummy->width = src->width; dummy->height = src->height; - dummy->data[3] = child_data; + + if (child_frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) { + mfxHDLPair *pair = (mfxHDLPair*)surf->Data.MemId; + dummy->data[0] = pair->first; + dummy->data[1] = pair->second; + } else { + dummy->data[3] = child_data; + } ret = av_hwframe_map(dst, dummy, flags); @@ -954,6 +1087,12 @@ static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, AVQSVFramesContext *dst_hwctx = dst_ctx->hwctx; int i; + if (src_ctx->initial_pool_size == 0) { + av_log(dst_ctx, AV_LOG_ERROR, "Only fixed-size pools can be " + "mapped to QSV frames.\n"); + return AVERROR(EINVAL); + } + switch (src_ctx->device_ctx->type) { #if CONFIG_VAAPI case AV_HWDEVICE_TYPE_VAAPI: @@ -972,6 +1111,36 @@ static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, } break; #endif +#if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: + { + AVD3D11VAFramesContext *src_hwctx = src_ctx->hwctx; + s->handle_pairs_internal = av_mallocz_array(src_ctx->initial_pool_size, sizeof(*s->handle_pairs_internal)); + if (!s->handle_pairs_internal) + return AVERROR(ENOMEM); + s->surfaces_internal = av_mallocz_array(src_ctx->initial_pool_size, + sizeof(*s->surfaces_internal)); + if (!s->surfaces_internal) + return AVERROR(ENOMEM); + for (i = 0; i < src_ctx->initial_pool_size; i++) { + qsv_init_surface(dst_ctx, &s->surfaces_internal[i]); + s->handle_pairs_internal[i].first = (mfxMemId)src_hwctx->texture_infos[i].texture; + if (src_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { + s->handle_pairs_internal[i].second = (mfxMemId)MFX_INFINITE; + } else { + s->handle_pairs_internal[i].second = (mfxMemId)src_hwctx->texture_infos[i].index; + } + s->surfaces_internal[i].Data.MemId = (mfxMemId)&s->handle_pairs_internal[i]; + } + dst_hwctx->nb_surfaces = src_ctx->initial_pool_size; + if (src_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { + dst_hwctx->frame_type |= MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET; + } else { + dst_hwctx->frame_type |= MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET; + } + } + break; +#endif #if CONFIG_DXVA2 case AV_HWDEVICE_TYPE_DXVA2: { @@ -982,7 +1151,9 @@ static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, return AVERROR(ENOMEM); for (i = 0; i < src_hwctx->nb_surfaces; i++) { qsv_init_surface(dst_ctx, &s->surfaces_internal[i]); - s->surfaces_internal[i].Data.MemId = (mfxMemId)src_hwctx->surfaces[i]; + s->handle_pairs_internal[i].first = (mfxMemId)src_hwctx->surfaces[i]; + s->handle_pairs_internal[i].second = (mfxMemId)MFX_INFINITE; + s->surfaces_internal[i].Data.MemId = (mfxMemId)&s->handle_pairs_internal[i]; } dst_hwctx->nb_surfaces = src_hwctx->nb_surfaces; if (src_hwctx->surface_type == DXVA2_VideoProcessorRenderTarget) @@ -1005,21 +1176,43 @@ static int qsv_map_to(AVHWFramesContext *dst_ctx, AVFrame *dst, const AVFrame *src, int flags) { AVQSVFramesContext *hwctx = dst_ctx->hwctx; - int i, err; + int i, err, index = -1; - for (i = 0; i < hwctx->nb_surfaces; i++) { + for (i = 0; i < hwctx->nb_surfaces && index < 0; i++) { + switch(src->format) { #if CONFIG_VAAPI - if (*(VASurfaceID*)hwctx->surfaces[i].Data.MemId == - (VASurfaceID)(uintptr_t)src->data[3]) + case AV_PIX_FMT_VAAPI: + if (*(VASurfaceID*)hwctx->surfaces[i].Data.MemId == + (VASurfaceID)(uintptr_t)src->data[3]) + index = i; break; #endif +#if CONFIG_D3D11VA + case AV_PIX_FMT_D3D11: + { + mfxHDLPair *pair = (mfxHDLPair*)hwctx->surfaces[i].Data.MemId; + if ((ID3D11Texture2D*)pair->first == + (ID3D11Texture2D*)(uintptr_t)src->data[0] + && pair->second == (mfxMemId)src->data[1]) { + index = i; + break; + } + } +#endif #if CONFIG_DXVA2 - if ((IDirect3DSurface9*)hwctx->surfaces[i].Data.MemId == - (IDirect3DSurface9*)(uintptr_t)src->data[3]) - break; + case AV_PIX_FMT_DXVA2_VLD: + { + mfxHDLPair *pair = (mfxHDLPair*)hwctx->surfaces[i].Data.MemId; + if ((IDirect3DSurface9*)pair->first == + (IDirect3DSurface9*)(uintptr_t)src->data[3]) { + index = i; + break; + } + } #endif + } } - if (i >= hwctx->nb_surfaces) { + if (index < 0) { av_log(dst_ctx, AV_LOG_ERROR, "Trying to map from a surface which " "is not in the mapped frames context.\n"); return AVERROR(EINVAL); @@ -1032,7 +1225,7 @@ static int qsv_map_to(AVHWFramesContext *dst_ctx, dst->width = src->width; dst->height = src->height; - dst->data[3] = (uint8_t*)&hwctx->surfaces[i]; + dst->data[3] = (uint8_t*)&hwctx->surfaces[index]; return 0; } @@ -1074,7 +1267,7 @@ static void qsv_device_free(AVHWDeviceContext *ctx) av_freep(&priv); } -static mfxIMPL choose_implementation(const char *device) +static mfxIMPL choose_implementation(const char *device, enum AVHWDeviceType child_device_type) { static const struct { const char *name; @@ -1103,6 +1296,10 @@ static mfxIMPL choose_implementation(const char *device) impl = strtol(device, NULL, 0); } + if ( (child_device_type == AV_HWDEVICE_TYPE_D3D11VA) && (impl != MFX_IMPL_SOFTWARE) ) { + impl |= MFX_IMPL_VIA_D3D11; + } + return impl; } @@ -1129,6 +1326,15 @@ static int qsv_device_derive_from_child(AVHWDeviceContext *ctx, } break; #endif +#if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: + { + AVD3D11VADeviceContext *child_device_hwctx = child_device_ctx->hwctx; + handle_type = MFX_HANDLE_D3D11_DEVICE; + handle = (mfxHDL)child_device_hwctx->device; + } + break; +#endif #if CONFIG_DXVA2 case AV_HWDEVICE_TYPE_DXVA2: { @@ -1191,7 +1397,9 @@ fail: static int qsv_device_derive(AVHWDeviceContext *ctx, AVHWDeviceContext *child_device_ctx, int flags) { - return qsv_device_derive_from_child(ctx, MFX_IMPL_HARDWARE_ANY, + mfxIMPL impl; + impl = choose_implementation("hw_any", child_device_ctx->type); + return qsv_device_derive_from_child(ctx, impl, child_device_ctx, flags); } @@ -1214,35 +1422,66 @@ static int qsv_device_create(AVHWDeviceContext *ctx, const char *device, ctx->user_opaque = priv; ctx->free = qsv_device_free; - e = av_dict_get(opts, "child_device", NULL, 0); - - child_device_opts = NULL; - if (CONFIG_VAAPI) { + e = av_dict_get(opts, "child_device_type", NULL, 0); + if (e) { + child_device_type = av_hwdevice_find_type_by_name(e ? e->value : NULL); + if (child_device_type == AV_HWDEVICE_TYPE_NONE) { + av_log(ctx, AV_LOG_ERROR, "Unknown child device type " + "\"%s\".\n", e ? e->value : NULL); + return AVERROR(EINVAL); + } + } else if (CONFIG_VAAPI) { child_device_type = AV_HWDEVICE_TYPE_VAAPI; - // libmfx does not actually implement VAAPI properly, rather it - // depends on the specific behaviour of a matching iHD driver when - // used on recent Intel hardware. Set options to the VAAPI device - // creation so that we should pick a usable setup by default if - // possible, even when multiple devices and drivers are available. - av_dict_set(&child_device_opts, "kernel_driver", "i915", 0); - av_dict_set(&child_device_opts, "driver", "iHD", 0); - } else if (CONFIG_DXVA2) + } else if (CONFIG_D3D11VA) { + child_device_type = AV_HWDEVICE_TYPE_D3D11VA; + } else if (CONFIG_DXVA2) { child_device_type = AV_HWDEVICE_TYPE_DXVA2; - else { + } else { av_log(ctx, AV_LOG_ERROR, "No supported child device type is enabled\n"); return AVERROR(ENOSYS); } + child_device_opts = NULL; + switch (child_device_type) { +#if CONFIG_VAAPI + case AV_HWDEVICE_TYPE_VAAPI: + { + // libmfx does not actually implement VAAPI properly, rather it + // depends on the specific behaviour of a matching iHD driver when + // used on recent Intel hardware. Set options to the VAAPI device + // creation so that we should pick a usable setup by default if + // possible, even when multiple devices and drivers are available. + av_dict_set(&child_device_opts, "kernel_driver", "i915", 0); + av_dict_set(&child_device_opts, "driver", "iHD", 0); + } + break; +#endif +#if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: + break; +#endif +#if CONFIG_DXVA2 + case AV_HWDEVICE_TYPE_DXVA2: + break; +#endif + default: + { + av_log(ctx, AV_LOG_ERROR, "No supported child device type is enabled\n"); + return AVERROR(ENOSYS); + } + break; + } + + e = av_dict_get(opts, "child_device", NULL, 0); ret = av_hwdevice_ctx_create(&priv->child_device_ctx, child_device_type, e ? e->value : NULL, child_device_opts, 0); - av_dict_free(&child_device_opts); if (ret < 0) return ret; child_device = (AVHWDeviceContext*)priv->child_device_ctx->data; - impl = choose_implementation(device); + impl = choose_implementation(device, child_device_type); return qsv_device_derive_from_child(ctx, impl, child_device, 0); }