From patchwork Sun Sep 10 20:53:32 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 5086 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.36.26 with SMTP id f26csp2357522jaa; Sun, 10 Sep 2017 13:55:03 -0700 (PDT) X-Received: by 10.223.179.66 with SMTP id k2mr7801289wrd.238.1505076903478; Sun, 10 Sep 2017 13:55:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1505076903; cv=none; d=google.com; s=arc-20160816; b=hKkqUv/bR8+x4xd91fnRAavrNP4f62Fi48HtjYpW0JlChz+ovZZ8+2tcg1Z2Sn82nH 3e1uj9spdIXbheUZ6jpHYTL6pk+9yXv0zNyoVi67rDCnPSRP2w4Bm3WfHG+CjJR2x5EL fL4NUkMhoDnY302t1oHCNT2E02ks7Ct2nvjFDiMcwsu2Vtgb6vrzgaJqE1T1J2vcA33S EGb3mHPcQpeheN5sMSnkBVkTfdq6+xcQckDm0lKhsNxXIhHK7wVOi3AvuvjrhSpUhvpK p5H0+Sa7jSS/0ghon8RbzsskUrKrOTKvAuyfhp1IpcjRuGkqUcshHMh2ynMZYl6xLlT3 riBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=kztbRo6xmwLOrsTlt1NZKh+/mH2BMmom/mnZFewLt4Q=; b=DNXLUuVQjB2KDXzG0kLxVf2eQfw8q+iFEAYGwH/XRzDmYOpiDtD3NxNZ6lq7Vr0fiO jIGbpfuk3hjAH/R0RFJsUlwT8IDIPbM9zT8h4/EIIUjXmGqzv6lWgdCT5Zptn2xFN/ns 3e/yHyvhFiaknVCmdatAMpIujChpvRavPZu+pYnBoLBCKanyD4wsOJghOOwWqEIb98iM qIxso+4DiOdSWX0ouWB2VuGE4Dq3GZmKxxYvaFde8Jflulizgtn9Dde4wwzzwKCJIAL0 fgpUb1tkLIzCwjhycecKDQl465UCz+q+t/bbWCsghkZoPXWPvSMSJqpSNRUZzsKwBjM8 kb1w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=lzt1qduq; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b11si5407051wme.208.2017.09.10.13.55.03; Sun, 10 Sep 2017 13:55:03 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=lzt1qduq; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EAAB7689D3B; Sun, 10 Sep 2017 23:53:52 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr0-f171.google.com (mail-wr0-f171.google.com [209.85.128.171]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7A6A2689CF1 for ; Sun, 10 Sep 2017 23:53:49 +0300 (EEST) Received: by mail-wr0-f171.google.com with SMTP id k20so11186184wre.4 for ; Sun, 10 Sep 2017 13:53:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=OQ1em9aLjr1nYnm13FgVeQSbLjBWJNaQNTkZdISnCDk=; b=lzt1qduqb6eXiO7R7upmVPwHWzibuZDHTi5OcYl05e1mzcCGKgunIxHiF60alQPt9N g7jBQNaSPOq6K10shcdV3ts2kh8Yap7DUu4y3C4ZAiRhi0NYNwFug3nEFbTLw5qLvTE3 zAfsGSeH9f0Xl6RzzdKZPzE1MS/UJwXLLmtl9t9yQodCtbvZePeyC4y3cNVHxtwZXIQi yni3nU8HkEl5LYNc2azUhyFFTU9qY6mgIvp6R7cFjOJsymhMle27BOyuUW+yiHA0v7ZH tVdMXz2pcPW+zSn4AuhINo9RpEe+6MurKMInhHWlxRYCSC3Z6IxC8QoHivwf2H0AbgBK HhiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=OQ1em9aLjr1nYnm13FgVeQSbLjBWJNaQNTkZdISnCDk=; b=ZdnUj3S0PJZzL4oCDZGU4JHqRJYGYa8a06UlbMAcoWEH2XmempsUTWZVpxUstOeq85 g/tgUskdIFh4PQYVWgLxwfyqE/lQdxJ+Ah9axVrsbHqwJpImX3KwW8ccGeA9X/jvYg8C E43FQ9ir38ee1Iw+kwXC1SO4ilZ1AJjEDnO+mB/TUf1DFayoYXtndY2yttodOvA2ZwRE bhhE7kKQ+A8bXH9bDO3F79TRpA5G2C0kzMw7n2dO+VqQfxl8HeSxqMFKc+3v+QbAJ5Tn C5ERqtpnxzMQaLtEhf3009TTMpO+IDIQk1+fe3fJFNJ/JndzEdWwDNCzMwR2s7e+7ZXM L5yw== X-Gm-Message-State: AHPjjUhQ+DkJ5XI/dERM+piJgO9xrWLyyB6DVwC45eWtCQ0qrMogaJLy 6DN+CZhJ5PM6PM8tLto= X-Google-Smtp-Source: ADKCNb6uIKAbNoKl20DTRKsK5MrXQzBLd58/m1tIBH+klETSV/uYTvMdFfSoZEUuXVtekUD4ibC+GQ== X-Received: by 10.223.148.195 with SMTP id 61mr6591717wrr.36.1505076834016; Sun, 10 Sep 2017 13:53:54 -0700 (PDT) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id z51sm8742868wrz.80.2017.09.10.13.53.53 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 10 Sep 2017 13:53:53 -0700 (PDT) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 10 Sep 2017 21:53:32 +0100 Message-Id: <20170910205338.29687-9-sw@jkqxz.net> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170910205338.29687-1-sw@jkqxz.net> References: <20170910205338.29687-1-sw@jkqxz.net> Subject: [FFmpeg-devel] [PATCH 08/14] hwcontext_opencl: DRM to OpenCL mapping for ARM X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Using cl_arm_import_memory. Unfortunately, despite this not being a standard extension, the function clImportMemoryARM() is not accessible via clGetExtensionFunctionAddressForPlatform(). This means that it has to be linked directly to the ARM OpenCL binary, so making a portable binary is not possible as it is with all other mapping extensions. --- configure | 6 + libavutil/hwcontext_opencl.c | 263 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 269 insertions(+) diff --git a/configure b/configure index 8376f2e3c9..0ae71112cd 100755 --- a/configure +++ b/configure @@ -2094,6 +2094,7 @@ HAVE_LIST=" makeinfo_html MMAL_PARAMETER_VIDEO_MAX_NUM_CALLBACKS opencl_d3d11 + opencl_drm_arm opencl_dxva2 opencl_vaapi_beignet opencl_vaapi_intel_media @@ -6210,6 +6211,11 @@ if enabled_all opencl d3d11va ; then enable opencl_d3d11 fi +if enabled_all opencl libdrm ; then + check_func_headers "CL/cl_ext.h" clImportMemoryARM && + enable opencl_drm_arm +fi + enabled vdpau && check_cpp_condition vdpau/vdpau.h "defined VDP_DECODER_PROFILE_MPEG4_PART2_ASP" || disable vdpau diff --git a/libavutil/hwcontext_opencl.c b/libavutil/hwcontext_opencl.c index 7af5ae90dc..9ec3a455d2 100644 --- a/libavutil/hwcontext_opencl.c +++ b/libavutil/hwcontext_opencl.c @@ -56,6 +56,12 @@ #include "hwcontext_d3d11va.h" #endif +#if HAVE_OPENCL_DRM_ARM +#include +#include +#include "hwcontext_drm.h" +#endif + typedef struct OpenCLDeviceContext { // Default command queue to use for transfer/mapping operations on @@ -104,6 +110,10 @@ typedef struct OpenCLDeviceContext { clEnqueueReleaseD3D11ObjectsKHR_fn clEnqueueReleaseD3D11ObjectsKHR; #endif + +#if HAVE_OPENCL_DRM_ARM + int drm_arm_mapping_usable; +#endif } OpenCLDeviceContext; typedef struct OpenCLFramesContext { @@ -826,6 +836,37 @@ static int opencl_device_init(AVHWDeviceContext *hwdev) } #endif +#if HAVE_OPENCL_DRM_ARM + { + const char *drm_arm_ext = "cl_arm_import_memory"; + const char *image_ext = "cl_khr_image2d_from_buffer"; + int fail = 0; + + if (!opencl_check_extension(hwdev, drm_arm_ext)) { + av_log(hwdev, AV_LOG_VERBOSE, "The %s extension is " + "required for DRM to OpenCL mapping on ARM.\n", + drm_arm_ext); + fail = 1; + } + if (!opencl_check_extension(hwdev, image_ext)) { + av_log(hwdev, AV_LOG_VERBOSE, "The %s extension is " + "required for DRM to OpenCL mapping on ARM.\n", + image_ext); + fail = 1; + } + + // clImportMemoryARM() is linked statically. + + if (fail) { + av_log(hwdev, AV_LOG_WARNING, "DRM to OpenCL mapping on ARM " + "not usable.\n"); + priv->drm_arm_mapping_usable = 0; + } else { + priv->drm_arm_mapping_usable = 1; + } + } +#endif + #undef CL_FUNC return 0; @@ -1104,6 +1145,40 @@ static int opencl_filter_gpu_device(AVHWDeviceContext *hwdev, } #endif +#if HAVE_OPENCL_DRM_ARM +static int opencl_filter_drm_arm_platform(AVHWDeviceContext *hwdev, + cl_platform_id platform_id, + const char *platform_name, + void *context) +{ + const char *drm_arm_ext = "cl_arm_import_memory"; + + if (opencl_check_platform_extension(platform_id, drm_arm_ext)) { + return 0; + } else { + av_log(hwdev, AV_LOG_DEBUG, "Platform %s does not support the " + "%s extension.\n", platform_name, drm_arm_ext); + return 1; + } +} + +static int opencl_filter_drm_arm_device(AVHWDeviceContext *hwdev, + cl_device_id device_id, + const char *device_name, + void *context) +{ + const char *drm_arm_ext = "cl_arm_import_memory"; + + if (opencl_check_device_extension(device_id, drm_arm_ext)) { + return 0; + } else { + av_log(hwdev, AV_LOG_DEBUG, "Device %s does not support the " + "%s extension.\n", device_name, drm_arm_ext); + return 1; + } +} +#endif + static int opencl_device_derive(AVHWDeviceContext *hwdev, AVHWDeviceContext *src_ctx, int flags) @@ -1250,6 +1325,24 @@ static int opencl_device_derive(AVHWDeviceContext *hwdev, break; #endif +#if HAVE_OPENCL_DRM_ARM + case AV_HWDEVICE_TYPE_DRM: + { + OpenCLDeviceSelector selector = { + .platform_index = -1, + .device_index = -1, + .context = NULL, + .enumerate_platforms = &opencl_enumerate_platforms, + .filter_platform = &opencl_filter_drm_arm_platform, + .enumerate_devices = &opencl_enumerate_devices, + .filter_device = &opencl_filter_drm_arm_device, + }; + + err = opencl_device_create_internal(hwdev, &selector, NULL); + } + break; +#endif + default: err = AVERROR(ENOSYS); break; @@ -2556,6 +2649,165 @@ fail: #endif +#if HAVE_OPENCL_DRM_ARM + +typedef struct DRMARMtoOpenCLMapping { + int nb_objects; + cl_mem object_buffers[AV_DRM_MAX_PLANES]; + int nb_planes; + cl_mem plane_images[AV_DRM_MAX_PLANES]; +} DRMARMtoOpenCLMapping; + +static void opencl_unmap_from_drm_arm(AVHWFramesContext *dst_fc, + HWMapDescriptor *hwmap) +{ + DRMARMtoOpenCLMapping *mapping = hwmap->priv; + int i; + + for (i = 0; i < mapping->nb_planes; i++) + clReleaseMemObject(mapping->plane_images[i]); + + for (i = 0; i < mapping->nb_objects; i++) + clReleaseMemObject(mapping->object_buffers[i]); + + av_free(mapping); +} + +static int opencl_map_from_drm_arm(AVHWFramesContext *dst_fc, AVFrame *dst, + const AVFrame *src, int flags) +{ + AVHWFramesContext *src_fc = + (AVHWFramesContext*)src->hw_frames_ctx->data; + AVOpenCLDeviceContext *dst_dev = dst_fc->device_ctx->hwctx; + const AVDRMFrameDescriptor *desc; + DRMARMtoOpenCLMapping *mapping = NULL; + cl_mem_flags cl_flags; + const cl_import_properties_arm props[3] = { + CL_IMPORT_TYPE_ARM, CL_IMPORT_TYPE_DMA_BUF_ARM, 0, + }; + cl_int cle; + int err, i, j; + + desc = (const AVDRMFrameDescriptor*)src->data[0]; + + cl_flags = opencl_mem_flags_for_mapping(flags); + if (!cl_flags) + return AVERROR(EINVAL); + + mapping = av_mallocz(sizeof(*mapping)); + if (!mapping) + return AVERROR(ENOMEM); + + mapping->nb_objects = desc->nb_objects; + for (i = 0; i < desc->nb_objects; i++) { + int fd = desc->objects[i].fd; + + av_log(dst_fc, AV_LOG_DEBUG, "Map DRM PRIME fd %d to OpenCL.\n", fd); + + if (desc->objects[i].format_modifier) { + av_log(dst_fc, AV_LOG_DEBUG, "Warning: object %d fd %d has " + "nonzero format modifier %"PRId64", result may not " + "be as expected.\n", i, fd, + desc->objects[i].format_modifier); + } + + mapping->object_buffers[i] = + clImportMemoryARM(dst_dev->context, cl_flags, props, + &fd, desc->objects[i].size, &cle); + if (!mapping->object_buffers[i]) { + av_log(dst_fc, AV_LOG_ERROR, "Failed to create CL buffer " + "from object %d (fd %d, size %zu) of DRM frame: %d.\n", + i, fd, desc->objects[i].size, cle); + err = AVERROR(EIO); + goto fail; + } + } + + mapping->nb_planes = 0; + for (i = 0; i < desc->nb_layers; i++) { + const AVDRMLayerDescriptor *layer = &desc->layers[i]; + + for (j = 0; j < layer->nb_planes; j++) { + const AVDRMPlaneDescriptor *plane = &layer->planes[j]; + cl_mem plane_buffer; + cl_image_format image_format; + cl_image_desc image_desc; + cl_buffer_region region; + int p = mapping->nb_planes; + + err = opencl_get_plane_format(src_fc->sw_format, p, + src_fc->width, src_fc->height, + &image_format, &image_desc); + if (err < 0) { + av_log(dst_fc, AV_LOG_ERROR, "Invalid plane %d (DRM " + "layer %d plane %d): %d.\n", p, i, j, err); + goto fail; + } + + region.origin = plane->offset; + region.size = image_desc.image_row_pitch * + image_desc.image_height; + + plane_buffer = + clCreateSubBuffer(mapping->object_buffers[plane->object_index], + cl_flags, + CL_BUFFER_CREATE_TYPE_REGION, + ®ion, &cle); + if (!plane_buffer) { + av_log(dst_fc, AV_LOG_ERROR, "Failed to create sub-buffer " + "for plane %d: %d.\n", p, cle); + err = AVERROR(EIO); + goto fail; + } + + image_desc.buffer = plane_buffer; + + mapping->plane_images[p] = + clCreateImage(dst_dev->context, cl_flags, + &image_format, &image_desc, NULL, &cle); + + // Unreference the sub-buffer immediately - we don't need it + // directly and a reference is held by the image. + clReleaseMemObject(plane_buffer); + + if (!mapping->plane_images[p]) { + av_log(dst_fc, AV_LOG_ERROR, "Failed to create image " + "for plane %d: %d.\n", p, cle); + err = AVERROR(EIO); + goto fail; + } + + ++mapping->nb_planes; + } + } + + for (i = 0; i < mapping->nb_planes; i++) + dst->data[i] = (uint8_t*)mapping->plane_images[i]; + + err = ff_hwframe_map_create(dst->hw_frames_ctx, dst, src, + &opencl_unmap_from_drm_arm, mapping); + if (err < 0) + goto fail; + + dst->width = src->width; + dst->height = src->height; + + return 0; + +fail: + for (i = 0; i < mapping->nb_planes; i++) { + clReleaseMemObject(mapping->plane_images[i]); + } + for (i = 0; i < mapping->nb_objects; i++) { + if (mapping->object_buffers[i]) + clReleaseMemObject(mapping->object_buffers[i]); + } + av_free(mapping); + return err; +} + +#endif + static int opencl_map_from(AVHWFramesContext *hwfc, AVFrame *dst, const AVFrame *src, int flags) { @@ -2592,6 +2844,11 @@ static int opencl_map_to(AVHWFramesContext *hwfc, AVFrame *dst, if (priv->d3d11_mapping_usable) return opencl_map_from_d3d11(hwfc, dst, src, flags); #endif +#if HAVE_OPENCL_DRM_ARM + case AV_PIX_FMT_DRM_PRIME: + if (priv->drm_arm_mapping_usable) + return opencl_map_from_drm_arm(hwfc, dst, src, flags); +#endif } return AVERROR(ENOSYS); } @@ -2638,6 +2895,12 @@ static int opencl_frames_derive_to(AVHWFramesContext *dst_fc, } break; #endif +#if HAVE_OPENCL_DRM_ARM + case AV_HWDEVICE_TYPE_DRM: + if (!priv->drm_arm_mapping_usable) + return AVERROR(ENOSYS); + break; +#endif default: return AVERROR(ENOSYS); }