From patchwork Sun Sep 10 20:53:31 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 5084 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.36.26 with SMTP id f26csp2357362jaa; Sun, 10 Sep 2017 13:54:46 -0700 (PDT) X-Received: by 10.223.176.133 with SMTP id i5mr7321980wra.17.1505076886382; Sun, 10 Sep 2017 13:54:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1505076886; cv=none; d=google.com; s=arc-20160816; b=VCEZ1m6QXsnTirDKUW0l1lUOT8aWSSzjiUYBt1NQLM81Feg6MjtXb9rbZj15Q4CDHc EEOP7nFwFQxaXFSJlrw7kTKYyNbySWAU4xLzhrHP2EARALZgA9PPopPwFEnr6qKLslSA muIvUEyTCOEByC+K/NvrEZSPIu7LBlzZAZkab1ewquHfDRL/v9PFLZzfxhyl+VLqvJgP 5H09rZVgA5yYm7/v2X+bH/SvphZSbMoOFEMy2cHINf05fzxpJrrIK6WxiRptAkZSUySP BQzGqYSyP7wDNQofyvVF9yWt/lRH8ppdEcwqfBc4z/vyZSEGygBpgXfhgosHhaWS39o7 VdKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=eUwOTIcwk4uoCb9HnvApAymxkmZYw3foDQzuNUszc9s=; b=t4ufZhS6hnJANbjgAW6OF8NReSWRX2guJKms83oKvrpQlPqGuErpZNpkbD/5Yy0cXU WJa6OWyUSXhrXqRLA47eGf0135ePO/zsS0A56Yo4uarEkeHN5PzK7q6clH+S0HHLvpMV nD6IUXpxvmz0hnjZxmIsgDHDmirTdlRQn7z+baj3Zzjq3DQyro4qFSU/UsMsWXwFnBY9 khvP6E1yy5Zb5wznoCMVpEjSUpjqJ77PSV31w9Xolc7EwcsCq3YfMNj+f4EPhKZFkEIz 9yTTZ2xL7IRQ1F7SzQ9iV4d87hHeeZfF026v7SWkhTFOxXeGzhledka4Ujl/nXThELIf /TfA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=dyPdciAl; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id c42si5746536wra.412.2017.09.10.13.54.46; Sun, 10 Sep 2017 13:54:46 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=dyPdciAl; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 41492689D08; Sun, 10 Sep 2017 23:53:52 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr0-f169.google.com (mail-wr0-f169.google.com [209.85.128.169]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 87EAA689C4C for ; Sun, 10 Sep 2017 23:53:48 +0300 (EEST) Received: by mail-wr0-f169.google.com with SMTP id 108so11200229wra.5 for ; Sun, 10 Sep 2017 13:53:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=NWec1G1IW3LB3QMNE6GW55RSpM3+knkOQc13jj83rEY=; b=dyPdciAlAhV4g6pyur5HFMnU4LcOYILnuLKOdIuIPDvsBGDnZrAPqyf2rl/twglGl1 +0GqXb80hkTHikmbHWhXb94CUzruUHd5cIcrlOQTNcdcjnAr29+h2vO6+AQ1zUamfr4z bamt2mypvYlUdPl+OHeAk8orF9jwkXrEpwlsOnAtfShFv3ixWI87ztsYao+EqfBg973A VNXYyr8heWF8q7MLw7T+hgIz7+9DWO/Qt/l+JY5dMfeC0ElspZW8jtnBfsSyXrNy1mra hquheyoKL3WbQ0F+m2BUHcjxtBTO7/lpTWUcAY9P84vlhDCfCSt/ZT1Mc5kaz+5MopXs o1vQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=NWec1G1IW3LB3QMNE6GW55RSpM3+knkOQc13jj83rEY=; b=jzSI/1MrSAeY5Z4AmAq62bHS+6PaAbKb4yhycXl71SXcKDXHgBNybvRQwthuafNJj/ brC6gexTQoUY1jyWrvnoH1j2uzTCTdHt3MbfKbd4AWVdJDGI4YzAOaUKt0DX/7lj9sM2 uLLmHi1kpYCaDnX3MP6KBK6t0EpEpFDkH8Yd0n62KQH6MqyjRBpLqIVi8Bxoh9HEIwNT u7yABQe7eDqv/hxmciclFZ6iCEzprYvi1CR93+zfsl/yK7s/a7LJiNlShQ6WsXRFfKyM d30GVtLJD5izw2l6D/rf2ed3mjh7IZphmGhZ5Q41lrpTBnDGrQayEJf7LBEghx/7AWVc iIMw== X-Gm-Message-State: AHPjjUgtNFQc4/JJF6UMJ876j7ESh8PITURlSMZG+BQER2k6yxa4Xsnk gsf80j1amwIgnPyrl9g= X-Google-Smtp-Source: ADKCNb6siGbNagpWJL9vAIDhhnIbn4a/tyXvKPUQVQZsJw9+ajSR34E6aJk19uYsO2f69vhzT9yO1A== X-Received: by 10.223.195.87 with SMTP id e23mr7965099wrg.77.1505076833016; Sun, 10 Sep 2017 13:53:53 -0700 (PDT) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id z51sm8742868wrz.80.2017.09.10.13.53.51 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 10 Sep 2017 13:53:52 -0700 (PDT) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 10 Sep 2017 21:53:31 +0100 Message-Id: <20170910205338.29687-8-sw@jkqxz.net> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170910205338.29687-1-sw@jkqxz.net> References: <20170910205338.29687-1-sw@jkqxz.net> Subject: [FFmpeg-devel] [PATCH 07/14] hwcontext_opencl: D3D11 to OpenCL mapping X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Using cl_khr_d3d11_sharing and cl_intel_d3d11_nv12_media_sharing. --- configure | 6 + libavutil/hwcontext_opencl.c | 320 ++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 324 insertions(+), 2 deletions(-) diff --git a/configure b/configure index c00059067f..8376f2e3c9 100755 --- a/configure +++ b/configure @@ -2093,6 +2093,7 @@ HAVE_LIST=" makeinfo makeinfo_html MMAL_PARAMETER_VIDEO_MAX_NUM_CALLBACKS + opencl_d3d11 opencl_dxva2 opencl_vaapi_beignet opencl_vaapi_intel_media @@ -6204,6 +6205,11 @@ if enabled_all opencl dxva2 ; then enable opencl_dxva2 fi +if enabled_all opencl d3d11va ; then + check_type "CL/cl_d3d11.h" clGetDeviceIDsFromD3D11KHR_fn && + enable opencl_d3d11 +fi + enabled vdpau && check_cpp_condition vdpau/vdpau.h "defined VDP_DECODER_PROFILE_MPEG4_PART2_ASP" || disable vdpau diff --git a/libavutil/hwcontext_opencl.c b/libavutil/hwcontext_opencl.c index a440d0085a..7af5ae90dc 100644 --- a/libavutil/hwcontext_opencl.c +++ b/libavutil/hwcontext_opencl.c @@ -51,6 +51,11 @@ #include "hwcontext_dxva2.h" #endif +#if HAVE_OPENCL_D3D11 +#include +#include "hwcontext_d3d11va.h" +#endif + typedef struct OpenCLDeviceContext { // Default command queue to use for transfer/mapping operations on @@ -89,6 +94,16 @@ typedef struct OpenCLDeviceContext { clEnqueueReleaseDX9MediaSurfacesKHR_fn clEnqueueReleaseDX9MediaSurfacesKHR; #endif + +#if HAVE_OPENCL_D3D11 + int d3d11_mapping_usable; + clCreateFromD3D11Texture2DKHR_fn + clCreateFromD3D11Texture2DKHR; + clEnqueueAcquireD3D11ObjectsKHR_fn + clEnqueueAcquireD3D11ObjectsKHR; + clEnqueueReleaseD3D11ObjectsKHR_fn + clEnqueueReleaseD3D11ObjectsKHR; +#endif } OpenCLDeviceContext; typedef struct OpenCLFramesContext { @@ -98,7 +113,7 @@ typedef struct OpenCLFramesContext { // device. cl_command_queue command_queue; -#if HAVE_OPENCL_DXVA2 +#if HAVE_OPENCL_DXVA2 || HAVE_OPENCL_D3D11 // For mapping APIs which have separate creation and acquire/release // steps, this stores the OpenCL memory objects corresponding to each // frame. @@ -778,6 +793,39 @@ static int opencl_device_init(AVHWDeviceContext *hwdev) } #endif +#if HAVE_OPENCL_D3D11 + { + const char *d3d11_ext = "cl_khr_d3d11_sharing"; + const char *nv12_ext = "cl_intel_d3d11_nv12_media_sharing"; + int fail = 0; + + if (!opencl_check_extension(hwdev, d3d11_ext)) { + av_log(hwdev, AV_LOG_VERBOSE, "The %s extension is " + "required for D3D11 to OpenCL mapping.\n", d3d11_ext); + fail = 1; + } else if (!opencl_check_extension(hwdev, nv12_ext)) { + av_log(hwdev, AV_LOG_VERBOSE, "The %s extension may be " + "required for D3D11 to OpenCL mapping.\n", nv12_ext); + // Not fatal. + } + + CL_FUNC(clCreateFromD3D11Texture2DKHR, + "D3D11 to OpenCL mapping"); + CL_FUNC(clEnqueueAcquireD3D11ObjectsKHR, + "D3D11 in OpenCL acquire"); + CL_FUNC(clEnqueueReleaseD3D11ObjectsKHR, + "D3D11 in OpenCL release"); + + if (fail) { + av_log(hwdev, AV_LOG_WARNING, "D3D11 to OpenCL mapping " + "not usable.\n"); + priv->d3d11_mapping_usable = 0; + } else { + priv->d3d11_mapping_usable = 1; + } + } +#endif + #undef CL_FUNC return 0; @@ -957,7 +1005,80 @@ static int opencl_enumerate_dxva2_devices(AVHWDeviceContext *hwdev, return 0; } +#endif + +#if HAVE_OPENCL_D3D11 +static int opencl_filter_d3d11_platform(AVHWDeviceContext *hwdev, + cl_platform_id platform_id, + const char *platform_name, + void *context) +{ + const char *d3d11_ext = "cl_khr_d3d11_sharing"; + + if (opencl_check_platform_extension(platform_id, d3d11_ext)) { + return 0; + } else { + av_log(hwdev, AV_LOG_DEBUG, "Platform %s does not support the " + "%s extension.\n", platform_name, d3d11_ext); + return 1; + } +} + +static int opencl_enumerate_d3d11_devices(AVHWDeviceContext *hwdev, + cl_platform_id platform_id, + const char *platform_name, + cl_uint *nb_devices, + cl_device_id **devices, + void *context) +{ + ID3D11Device *device = context; + clGetDeviceIDsFromD3D11KHR_fn clGetDeviceIDsFromD3D11KHR; + cl_int cle; + + clGetDeviceIDsFromD3D11KHR = + clGetExtensionFunctionAddressForPlatform(platform_id, + "clGetDeviceIDsFromD3D11KHR"); + if (!clGetDeviceIDsFromD3D11KHR) { + av_log(hwdev, AV_LOG_ERROR, "Failed to get address of " + "clGetDeviceIDsFromD3D11KHR().\n"); + return AVERROR_UNKNOWN; + } + + cle = clGetDeviceIDsFromD3D11KHR(platform_id, + CL_D3D11_DEVICE_KHR, device, + CL_PREFERRED_DEVICES_FOR_D3D11_KHR, + 0, NULL, nb_devices); + if (cle == CL_DEVICE_NOT_FOUND) { + av_log(hwdev, AV_LOG_DEBUG, "No D3D11-supporting devices found " + "on platform \"%s\".\n", platform_name); + *nb_devices = 0; + return 0; + } else if (cle != CL_SUCCESS) { + av_log(hwdev, AV_LOG_ERROR, "Failed to get number of devices " + "on platform \"%s\": %d.\n", platform_name, cle); + return AVERROR_UNKNOWN; + } + + *devices = av_malloc_array(*nb_devices, sizeof(**devices)); + if (!*devices) + return AVERROR(ENOMEM); + + cle = clGetDeviceIDsFromD3D11KHR(platform_id, + CL_D3D11_DEVICE_KHR, device, + CL_PREFERRED_DEVICES_FOR_D3D11_KHR, + *nb_devices, *devices, NULL); + if (cle != CL_SUCCESS) { + av_log(hwdev, AV_LOG_ERROR, "Failed to get list of D3D11-supporting " + "devices on platform \"%s\": %d.\n", platform_name, cle); + av_freep(devices); + return AVERROR_UNKNOWN; + } + + return 0; +} +#endif +#if HAVE_OPENCL_DXVA2 || HAVE_OPENCL_D3D11 static int opencl_filter_gpu_device(AVHWDeviceContext *hwdev, cl_device_id device_id, const char *device_name, @@ -1103,6 +1224,32 @@ static int opencl_device_derive(AVHWDeviceContext *hwdev, break; #endif +#if HAVE_OPENCL_D3D11 + case AV_HWDEVICE_TYPE_D3D11VA: + { + AVD3D11VADeviceContext *src_hwctx = src_ctx->hwctx; + cl_context_properties props[5] = { + CL_CONTEXT_PLATFORM, + 0, + CL_CONTEXT_D3D11_DEVICE_KHR, + (intptr_t)src_hwctx->device, + 0, + }; + OpenCLDeviceSelector selector = { + .platform_index = -1, + .device_index = -1, + .context = src_hwctx->device, + .enumerate_platforms = &opencl_enumerate_platforms, + .filter_platform = &opencl_filter_d3d11_platform, + .enumerate_devices = &opencl_enumerate_d3d11_devices, + .filter_device = &opencl_filter_gpu_device, + }; + + err = opencl_device_create_internal(hwdev, &selector, props); + } + break; +#endif + default: err = AVERROR(ENOSYS); break; @@ -1461,7 +1608,7 @@ static void opencl_frames_uninit(AVHWFramesContext *hwfc) OpenCLFramesContext *priv = hwfc->internal->priv; cl_int cle; -#if HAVE_OPENCL_DXVA2 +#if HAVE_OPENCL_DXVA2 || HAVE_OPENCL_D3D11 int i, p; for (i = 0; i < priv->nb_mapped_frames; i++) { AVOpenCLFrameDescriptor *desc = &priv->mapped_frames[i]; @@ -2257,6 +2404,158 @@ fail: #endif +#if HAVE_OPENCL_D3D11 + +static void opencl_unmap_from_d3d11(AVHWFramesContext *dst_fc, + HWMapDescriptor *hwmap) +{ + AVOpenCLFrameDescriptor *desc = hwmap->priv; + OpenCLDeviceContext *device_priv = dst_fc->device_ctx->internal->priv; + OpenCLFramesContext *frames_priv = dst_fc->device_ctx->internal->priv; + cl_event event; + cl_int cle; + + cle = device_priv->clEnqueueReleaseD3D11ObjectsKHR( + frames_priv->command_queue, desc->nb_planes, desc->planes, + 0, NULL, &event); + if (cle != CL_SUCCESS) { + av_log(dst_fc, AV_LOG_ERROR, "Failed to release surface " + "handle: %d.\n", cle); + } + + opencl_wait_events(dst_fc, &event, 1); +} + +static int opencl_map_from_d3d11(AVHWFramesContext *dst_fc, AVFrame *dst, + const AVFrame *src, int flags) +{ + OpenCLDeviceContext *device_priv = dst_fc->device_ctx->internal->priv; + OpenCLFramesContext *frames_priv = dst_fc->internal->priv; + AVOpenCLFrameDescriptor *desc; + cl_event event; + cl_int cle; + int err, index, i; + + index = (intptr_t)src->data[1]; + if (index >= frames_priv->nb_mapped_frames) { + av_log(dst_fc, AV_LOG_ERROR, "Texture array index out of range for " + "mapping: %d >= %d.\n", index, frames_priv->nb_mapped_frames); + return AVERROR(EINVAL); + } + + av_log(dst_fc, AV_LOG_DEBUG, "Map D3D11 texture %d to OpenCL.\n", + index); + + desc = &frames_priv->mapped_frames[index]; + + cle = device_priv->clEnqueueAcquireD3D11ObjectsKHR( + frames_priv->command_queue, desc->nb_planes, desc->planes, + 0, NULL, &event); + if (cle != CL_SUCCESS) { + av_log(dst_fc, AV_LOG_ERROR, "Failed to acquire surface " + "handle: %d.\n", cle); + return AVERROR(EIO); + } + + err = opencl_wait_events(dst_fc, &event, 1); + if (err < 0) + goto fail; + + for (i = 0; i < desc->nb_planes; i++) + dst->data[i] = (uint8_t*)desc->planes[i]; + + err = ff_hwframe_map_create(dst->hw_frames_ctx, dst, src, + &opencl_unmap_from_d3d11, desc); + if (err < 0) + goto fail; + + dst->width = src->width; + dst->height = src->height; + + return 0; + +fail: + cle = device_priv->clEnqueueReleaseD3D11ObjectsKHR( + frames_priv->command_queue, desc->nb_planes, desc->planes, + 0, NULL, &event); + if (cle == CL_SUCCESS) + opencl_wait_events(dst_fc, &event, 1); + return err; +} + +static int opencl_frames_derive_from_d3d11(AVHWFramesContext *dst_fc, + AVHWFramesContext *src_fc, int flags) +{ + AVOpenCLDeviceContext *dst_dev = dst_fc->device_ctx->hwctx; + AVD3D11VAFramesContext *src_hwctx = src_fc->hwctx; + OpenCLDeviceContext *device_priv = dst_fc->device_ctx->internal->priv; + OpenCLFramesContext *frames_priv = dst_fc->internal->priv; + cl_mem_flags cl_flags; + cl_int cle; + int err, i, p, nb_planes; + + if (src_fc->sw_format != AV_PIX_FMT_NV12) { + av_log(dst_fc, AV_LOG_ERROR, "Only NV12 textures are supported " + "for D3D11 to OpenCL mapping.\n"); + return AVERROR(EINVAL); + } + nb_planes = 2; + + if (src_fc->initial_pool_size == 0) { + av_log(dst_fc, AV_LOG_ERROR, "Only fixed-size pools are supported " + "for D3D11 to OpenCL mapping.\n"); + return AVERROR(EINVAL); + } + + cl_flags = opencl_mem_flags_for_mapping(flags); + if (!cl_flags) + return AVERROR(EINVAL); + + frames_priv->nb_mapped_frames = src_fc->initial_pool_size; + + frames_priv->mapped_frames = + av_mallocz_array(frames_priv->nb_mapped_frames, + sizeof(*frames_priv->mapped_frames)); + if (!frames_priv->mapped_frames) + return AVERROR(ENOMEM); + + for (i = 0; i < frames_priv->nb_mapped_frames; i++) { + AVOpenCLFrameDescriptor *desc = &frames_priv->mapped_frames[i]; + desc->nb_planes = nb_planes; + for (p = 0; p < nb_planes; p++) { + UINT subresource = 2 * i + p; + + desc->planes[p] = + device_priv->clCreateFromD3D11Texture2DKHR( + dst_dev->context, cl_flags, src_hwctx->texture, + subresource, &cle); + if (!desc->planes[p]) { + av_log(dst_fc, AV_LOG_ERROR, "Failed to create CL " + "image from plane %d of D3D texture " + "index %d (subresource %u): %d.\n", + p, i, (unsigned int)subresource, cle); + err = AVERROR(EIO); + goto fail; + } + } + } + + return 0; + +fail: + for (i = 0; i < frames_priv->nb_mapped_frames; i++) { + AVOpenCLFrameDescriptor *desc = &frames_priv->mapped_frames[i]; + for (p = 0; p < desc->nb_planes; p++) { + if (desc->planes[p]) + clReleaseMemObject(desc->planes[p]); + } + } + av_freep(&frames_priv->mapped_frames); + return err; +} + +#endif + static int opencl_map_from(AVHWFramesContext *hwfc, AVFrame *dst, const AVFrame *src, int flags) { @@ -2288,6 +2587,11 @@ static int opencl_map_to(AVHWFramesContext *hwfc, AVFrame *dst, if (priv->dxva2_mapping_usable) return opencl_map_from_dxva2(hwfc, dst, src, flags); #endif +#if HAVE_OPENCL_D3D11 + case AV_PIX_FMT_D3D11: + if (priv->d3d11_mapping_usable) + return opencl_map_from_d3d11(hwfc, dst, src, flags); +#endif } return AVERROR(ENOSYS); } @@ -2322,6 +2626,18 @@ static int opencl_frames_derive_to(AVHWFramesContext *dst_fc, } break; #endif +#if HAVE_OPENCL_D3D11 + case AV_HWDEVICE_TYPE_D3D11VA: + if (!priv->d3d11_mapping_usable) + return AVERROR(ENOSYS); + { + int err; + err = opencl_frames_derive_from_d3d11(dst_fc, src_fc, flags); + if (err < 0) + return err; + } + break; +#endif default: return AVERROR(ENOSYS); }