From patchwork Sun Nov 25 19:34:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 11156 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id BA0F644CA2B for ; Sun, 25 Nov 2018 21:34:40 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3A0BD68A319; Sun, 25 Nov 2018 21:34:41 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f65.google.com (mail-wm1-f65.google.com [209.85.128.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0B62168A08B for ; Sun, 25 Nov 2018 21:34:35 +0200 (EET) Received: by mail-wm1-f65.google.com with SMTP id k198so16266459wmd.3 for ; Sun, 25 Nov 2018 11:34:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-language:content-transfer-encoding; bh=Tz4sjNIVLcdgqKcnWC6CwA3442LgCNwy/0BrSfomtEg=; b=gP7dYRJ/zlCT2K/h+esovTXn4JakLjxwISK+ntMgtQHmeZFCV1ipnRfUHOMj7lIM5K H6SKFt72rhhI3GcuTPS0UlhzlcPfCTU2PSSNvgPR94wOkhxIrDXx8jM3uUXXQXAK/OTp W5+xaOxjPBpcBQxUMTSZsCa2pzisrsAefAzM7ucx2EoiucBfrDMctNV4u/CP6a3+s2Cr MFETgxHEhKjxfnyQuQE5SgUexv4Fy+sgBYq0BuU/YvkX+I8MAz40e7fibeqFiw8BjdF9 BHk7u9p32bzHdGJhn9SArTs6p5sEea1jwrW7DnDHILdaowaaeFzmHCRnsKSbrlE5YsRk XoBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=Tz4sjNIVLcdgqKcnWC6CwA3442LgCNwy/0BrSfomtEg=; b=nMGO+JrwAmcP44W+gF7Tp2C1GNmOJY8d1R3elUym6K/UAbyUt/ma2yR00z/l0TpnNO ZzKu6PBY6top/UWTIFhiai2HV5YJI+2R/XcBOCN1FYCj6TNjIzGT06F56WWOfdcemHAg IpjQaDUPa8kXDPYjPxMEzpHs/G8AL4fyIeemamhNVMKQ8s3HAJ/wRSnCR8jXDNj1Lpfb VIM8WvpYSn+xlxT4QlIvyaWxIMkb5KHlspEJpfXGFlSYa47hinbHMxoJxRl9pFfAlLtB WwEClYLQbnSz2b4DWxqXQs34jkUMYLUHyWwhGFfCZUyQdXOiT22SOQV2LO9usxiC3SLH 4kcA== X-Gm-Message-State: AA+aEWbxLkk+2NNBsygQ4YGVHA2it0MO7Ib3HCf7R9Rfo0w9yasL94tW LevDotUrISlRpY2y52ZBXQKnRYKwGPU= X-Google-Smtp-Source: AFSGD/Wfb/BMKJQPFnRHtEPWRzeQ5bxmxS/FhrW/bN8HnQvYXcJDil+y3aUioDG1PL2X0yF1w2rP1A== X-Received: by 2002:a1c:8484:: with SMTP id g126mr9502422wmd.117.1543174478560; Sun, 25 Nov 2018 11:34:38 -0800 (PST) Received: from [192.168.0.4] (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id 198sm9779612wmj.38.2018.11.25.11.34.37 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 25 Nov 2018 11:34:37 -0800 (PST) To: ffmpeg-devel@ffmpeg.org References: From: Mark Thompson Message-ID: <0cbfcf22-2474-70a5-bd33-4c45ae396f7a@jkqxz.net> Date: Sun, 25 Nov 2018 19:34:37 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.3.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Subject: Re: [FFmpeg-devel] [INFO]AMD D3D11 to OpenCL interop extension for NV12 and P010 textures - split planes X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" On 24/05/2018 15:26, Mironov, Mikhail wrote: > AMD has published OpenCL extension which allows split D3D11 texture interoped as a single 2D image into two 2D images representing Y and UV planes. > https://www.khronos.org/registry/OpenCL/extensions/amd/cl_amd_planar_yuv.txt I had a go at implementing this now that it is actually visible in released drivers, but I can't get it to work. Patch trying to implement it is below. It finds the extension and the new function correctly, but I'm stuck on the creation of the whole-texture image with clCreateFromD3D11Texture2DKHR(). The error returned is CL_INVALID_D3D11_RESOURCE_KHR (-1007), but as far as I can tell none of the documented failure cases which would return that error code apply. $ ./ffmpeg_g.exe -report -v debug -y -hwaccel d3d11va -hwaccel_device 0 -hwaccel_output_format d3d11 -i input.mp4 -an -vf "hwmap=derive_device=opencl:mode=read,unsharp_opencl,hwdownload,format=nv12" -f null - ... [AVHWDeviceContext @ 0000000001c0de80] Using device 1002:665f (AMD Radeon (TM) R7 360 Series). ... [h264 @ 000000000284adc0] Format d3d11 chosen by get_format(). ... [Parsed_hwmap_0 @ 0000000002a2be00] Configure hwmap d3d11 -> opencl. [AVHWDeviceContext @ 000000000d328500] 2 OpenCL platforms found. [AVHWDeviceContext @ 000000000d328500] 1 OpenCL devices found on platform "Intel(R) OpenCL". [AVHWDeviceContext @ 000000000d328500] Device Intel(R) Core(TM) i3-6300 CPU @ 3.80GHz skipped (not GPU). [AVHWDeviceContext @ 000000000d328500] 1 OpenCL devices found on platform "AMD Accelerated Parallel Processing". [AVHWDeviceContext @ 000000000d328500] 1.0: AMD Accelerated Parallel Processing / Bonaire [AVHWDeviceContext @ 000000000d328500] DXVA2 to OpenCL mapping function found (clCreateFromDX9MediaSurfaceKHR). [AVHWDeviceContext @ 000000000d328500] DXVA2 in OpenCL acquire function found (clEnqueueAcquireDX9MediaSurfacesKHR). [AVHWDeviceContext @ 000000000d328500] DXVA2 in OpenCL release function found (clEnqueueReleaseDX9MediaSurfacesKHR). [AVHWDeviceContext @ 000000000d328500] cl_khr_d3d11_sharing found as platform extension. [AVHWDeviceContext @ 000000000d328500] cl_amd_planar_yuv found as device extension. [AVHWDeviceContext @ 000000000d328500] D3D11 to OpenCL mapping function found (clCreateFromD3D11Texture2DKHR). [AVHWDeviceContext @ 000000000d328500] D3D11 in OpenCL acquire function found (clEnqueueAcquireD3D11ObjectsKHR). [AVHWDeviceContext @ 000000000d328500] D3D11 in OpenCL release function found (clEnqueueReleaseD3D11ObjectsKHR). [AVHWDeviceContext @ 000000000d328500] D3D11 to OpenCL mapping on AMD function found (clGetPlaneFromImageAMD). [AVHWFramesContext @ 0000000002c13180] Failed to create CL image from D3D texture index 0: -1007. [Parsed_hwmap_0 @ 0000000002a2be00] Failed to create derived frames context: -5. [Parsed_hwmap_0 @ 0000000002a2be00] Failed to configure output pad on Parsed_hwmap_0 Are there any examples of using this extension that I could compare with? Alternatively, is the source code for the CL driver available somewhere so that I can work out what that error actually means? Thanks, - Mark From 25fb98f021b1347394d56ecf4781466096616542 Mon Sep 17 00:00:00 2001 From: Mark Thompson Date: Sun, 25 Nov 2018 16:59:24 +0000 Subject: [PATCH] hwcontext_opencl: Add support for D3D11 to OpenCL mapping on AMD Uses cl_amd_planar_yuv. --- libavutil/hwcontext_opencl.c | 106 ++++++++++++++++++++++++++++------- 1 file changed, 86 insertions(+), 20 deletions(-) diff --git a/libavutil/hwcontext_opencl.c b/libavutil/hwcontext_opencl.c index 728877553f..c745b91775 100644 --- a/libavutil/hwcontext_opencl.c +++ b/libavutil/hwcontext_opencl.c @@ -64,6 +64,12 @@ #if HAVE_OPENCL_D3D11 #include #include "hwcontext_d3d11va.h" + +// From cl_amd_planar_yuv; unfortunately no header is provided. +typedef cl_mem (*clGetPlaneFromImageAMD_fn)(cl_context context, + cl_mem mem, + cl_uint plane, + cl_int *errcode_ret); #endif #if HAVE_OPENCL_DRM_ARM @@ -113,12 +119,17 @@ typedef struct OpenCLDeviceContext { #if HAVE_OPENCL_D3D11 int d3d11_mapping_usable; + int d3d11_map_amd; + int d3d11_map_intel; + clCreateFromD3D11Texture2DKHR_fn clCreateFromD3D11Texture2DKHR; clEnqueueAcquireD3D11ObjectsKHR_fn clEnqueueAcquireD3D11ObjectsKHR; clEnqueueReleaseD3D11ObjectsKHR_fn clEnqueueReleaseD3D11ObjectsKHR; + clGetPlaneFromImageAMD_fn + clGetPlaneFromImageAMD; #endif #if HAVE_OPENCL_DRM_ARM @@ -817,17 +828,25 @@ static int opencl_device_init(AVHWDeviceContext *hwdev) #if HAVE_OPENCL_D3D11 { const char *d3d11_ext = "cl_khr_d3d11_sharing"; - const char *nv12_ext = "cl_intel_d3d11_nv12_media_sharing"; + const char *amd_ext = "cl_amd_planar_yuv"; + const char *intel_ext = "cl_intel_d3d11_nv12_media_sharing"; int fail = 0; if (!opencl_check_extension(hwdev, d3d11_ext)) { av_log(hwdev, AV_LOG_VERBOSE, "The %s extension is " "required for D3D11 to OpenCL mapping.\n", d3d11_ext); fail = 1; - } else if (!opencl_check_extension(hwdev, nv12_ext)) { - av_log(hwdev, AV_LOG_VERBOSE, "The %s extension may be " - "required for D3D11 to OpenCL mapping.\n", nv12_ext); - // Not fatal. + } else { + if (opencl_check_extension(hwdev, amd_ext)) { + priv->d3d11_map_amd = 1; + } else if (opencl_check_extension(hwdev, intel_ext)) { + priv->d3d11_map_intel = 1; + } else { + av_log(hwdev, AV_LOG_VERBOSE, "One of the %s or %s " + "extensions are required for D3D11 to OpenCL " + "mapping.\n", amd_ext, intel_ext); + fail = 1; + } } CL_FUNC(clCreateFromD3D11Texture2DKHR, @@ -837,6 +856,11 @@ static int opencl_device_init(AVHWDeviceContext *hwdev) CL_FUNC(clEnqueueReleaseD3D11ObjectsKHR, "D3D11 in OpenCL release"); + if (priv->d3d11_map_amd) { + CL_FUNC(clGetPlaneFromImageAMD, + "D3D11 to OpenCL mapping on AMD"); + } + if (fail) { av_log(hwdev, AV_LOG_WARNING, "D3D11 to OpenCL mapping " "not usable.\n"); @@ -2573,10 +2597,22 @@ static int opencl_frames_derive_from_d3d11(AVHWFramesContext *dst_fc, cl_int cle; int err, i, p, nb_planes; - if (src_fc->sw_format != AV_PIX_FMT_NV12) { - av_log(dst_fc, AV_LOG_ERROR, "Only NV12 textures are supported " - "for D3D11 to OpenCL mapping.\n"); - return AVERROR(EINVAL); + // AMD supports NV12 and P010, Intel only supports NV12. + if (device_priv->d3d11_map_amd) { + if (src_fc->sw_format != AV_PIX_FMT_NV12 && + src_fc->sw_format != AV_PIX_FMT_P010) { + av_log(dst_fc, AV_LOG_ERROR, "Only NV12 and P010 textures are " + "supported with AMD for D3D11 to OpenCL mapping.\n"); + return AVERROR(EINVAL); + } + } else if (device_priv->d3d11_map_intel) { + if (src_fc->sw_format != AV_PIX_FMT_NV12) { + av_log(dst_fc, AV_LOG_ERROR, "Only NV12 and P010 textures are " + "supported with Intel for D3D11 to OpenCL mapping.\n"); + return AVERROR(EINVAL); + } + } else { + av_assert0(0); } nb_planes = 2; @@ -2601,21 +2637,51 @@ static int opencl_frames_derive_from_d3d11(AVHWFramesContext *dst_fc, for (i = 0; i < frames_priv->nb_mapped_frames; i++) { AVOpenCLFrameDescriptor *desc = &frames_priv->mapped_frames[i]; desc->nb_planes = nb_planes; - for (p = 0; p < nb_planes; p++) { - UINT subresource = 2 * i + p; - desc->planes[p] = - device_priv->clCreateFromD3D11Texture2DKHR( - dst_dev->context, cl_flags, src_hwctx->texture, - subresource, &cle); - if (!desc->planes[p]) { - av_log(dst_fc, AV_LOG_ERROR, "Failed to create CL " - "image from plane %d of D3D texture " - "index %d (subresource %u): %d.\n", - p, i, (unsigned int)subresource, cle); + if (device_priv->d3d11_map_amd) { + cl_mem image; + + image = device_priv->clCreateFromD3D11Texture2DKHR( + dst_dev->context, cl_flags, src_hwctx->texture, i, &cle); + if (!image) { + av_log(dst_fc, AV_LOG_ERROR, "Failed to create CL image " + "from D3D texture index %d: %d.\n", i, cle); err = AVERROR(EIO); goto fail; } + + for (p = 0; p < nb_planes; p++) { + desc->planes[p] = device_priv->clGetPlaneFromImageAMD( + dst_dev->context, image, p, &cle); + if (!desc->planes[p]) { + av_log(dst_fc, AV_LOG_ERROR, "Failed to create CL image " + "from plane %d of image created from D3D11 " + "texture index %d: %d.\n", p, cle, i); + clReleaseMemObject(image); + err = AVERROR(EIO); + goto fail; + } + } + + clReleaseMemObject(image); + + } else { + for (p = 0; p < nb_planes; p++) { + UINT subresource = 2 * i + p; + + desc->planes[p] = + device_priv->clCreateFromD3D11Texture2DKHR( + dst_dev->context, cl_flags, src_hwctx->texture, + subresource, &cle); + if (!desc->planes[p]) { + av_log(dst_fc, AV_LOG_ERROR, "Failed to create CL " + "image from plane %d of D3D texture " + "index %d (subresource %u): %d.\n", + p, i, (unsigned int)subresource, cle); + err = AVERROR(EIO); + goto fail; + } + } } }