From patchwork Sun Sep 10 20:53:28 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 5083 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.36.26 with SMTP id f26csp2357255jaa; Sun, 10 Sep 2017 13:54:36 -0700 (PDT) X-Received: by 10.28.61.134 with SMTP id k128mr5831828wma.48.1505076876723; Sun, 10 Sep 2017 13:54:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1505076876; cv=none; d=google.com; s=arc-20160816; b=Br9+kM5o1+YrXblaMhQ8nBtn0vD5DZ4U0WUd6YkJAstzWmU/y7KKh9NO6JnWl9WBei acmJux3Q0zjT4Ff3zRjI0U0omIkFRnWDZi0uaUORs6amatJd+mDzDOkxGo6CyprsTkC0 wSCjXyQwaKwSST0VBx2cpYlfsARDDy6ejjgX3eQRoLusR7u2Rljw3P23HN+vgyR9IAQd GMAUg5TDwTZvEDNW1CyKqWS/Ag39DGut8mOuHJ/j3hLbKnQEKPWRVEwD/Tb6oc/rVW65 5tYz9WHS1pqlLxbkrI5jSJwd2qfQtLbSAoojdohsIFZBjMWxgRV8N9/T1rbH9GKckgzt ntNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=X1/SvrzgKddAzn2j1WKft6x1kit3jn81f+91Y06TMS8=; b=Cy8xTjX1FGQ+6+I83scpgVjgTTeKDFqRQFujIx6C7Et0cSEGSGfOMej2ovjDi6sTQm 94oSzKX7OOMntwWk7CCEeO9gyBsYxu5/0KaSacUAT0cZk3RccA7ZKHg3QIRJcMDtQBAY DbGmR8zYHoVKw4kpqj5xCSI/x6vvMGzb4VqzTjAYZ1AGqIegVqGuYNgX76x2Wb8j1WGQ kicgijFF2x2OQnobjsJRqAugH9g0mefpJYWDOol6N185kSkx+aNGBxv6yhqQMaj1Yo1Q VrKAtpwqJavp3bXCT2GZCR6+/QsSnGrOs6g8msf/oEDWeec6O+neDsfHyNWcnKn5Iuxi mosw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=g+BHSEy8; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id m19si1501193wrg.503.2017.09.10.13.54.36; Sun, 10 Sep 2017 13:54:36 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=g+BHSEy8; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D0777689CB2; Sun, 10 Sep 2017 23:53:49 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr0-f169.google.com (mail-wr0-f169.google.com [209.85.128.169]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C207F6899D2 for ; Sun, 10 Sep 2017 23:53:42 +0300 (EEST) Received: by mail-wr0-f169.google.com with SMTP id a43so11239237wrc.0 for ; Sun, 10 Sep 2017 13:53:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=1mpGIL6lUr463RKuk74Wd5Ihq+RO6hOok1AMSc27xOI=; b=g+BHSEy85G9Qheb1MZd0HXsRajprXQDehM7Cwwm6Dpq2IUKDvBqEKCYH4Tc6Ej/+P8 QI08SlUrPhRnCO1y7t5s5ohKL4ZSWbbWWhhCne8wztUCK/6lht9UQtqrXwZSLVPBphbq T+elQE1/apChKryBvC5xTcxTx0OjkavWKHZ/yAdHZOh8McMQ7rIAiEMGmYrItvCwUYsW psnCxClAAb+5tfOxyPzlWmXiRw5tkP78Lj3txcRahF7TWi8r3HzjemAPjEwyJiZRXpE/ /srUwfhJE5jaMySlw7/gFlhQqReFQU5ljcl/kCOUTqsDxl8JJleNv6BIAykcPfFsc+2X EsYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=1mpGIL6lUr463RKuk74Wd5Ihq+RO6hOok1AMSc27xOI=; b=PxeSzRNJcEb74ZNUniy5hE4zKPeMmxJcBWraoCZF9YFdtllC4Q9nM02XA5GTq+MRrO oOA92/B5F1ElcAyzAzuXnHTqCMEJQeWPZQdwUzd1tX10klXa7RQ4RyPDMaWqB9Eb052P fPI/jSM5qzJ067Uuxumf3N1AFcdHAN8F3ui9NcE+q5JaWBegwKyfjly7OF55NdR9qxzh qA06nax3cNwo0No5MA2KF4qtR7zQm/MhlV2STXCwMnltq3ySggyJGPOxLOerCt16A3vK gVtuCkJjCNVsYvzB7mASwkxDTRidMsv8BnD8Fie8fb/L7jbOWV1UiO6PzCKhfIDNKM1E yXpw== X-Gm-Message-State: AHPjjUh7C59syTob1j2PD4xwDYImoDgc3Oc50mp385T8pcsl2kAYdMLa 6Q53HSZCFzUE0lxma50= X-Google-Smtp-Source: ADKCNb5fT1gD5AUWSutDN4PNbpM73DKsQw2EWSmtelFQ3gn79W6bd3/d2i5AVZO0jCUr/GRkFYFumQ== X-Received: by 10.223.141.235 with SMTP id o98mr6653822wrb.242.1505076827129; Sun, 10 Sep 2017 13:53:47 -0700 (PDT) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id z51sm8742868wrz.80.2017.09.10.13.53.46 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 10 Sep 2017 13:53:46 -0700 (PDT) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 10 Sep 2017 21:53:28 +0100 Message-Id: <20170910205338.29687-5-sw@jkqxz.net> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170910205338.29687-1-sw@jkqxz.net> References: <20170910205338.29687-1-sw@jkqxz.net> Subject: [FFmpeg-devel] [PATCH 04/14] hwcontext_opencl: VAAPI to OpenCL mapping for Intel i965+beignet X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Supports all surface formats in common between the two. --- configure | 6 + libavutil/hwcontext_opencl.c | 298 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 304 insertions(+) diff --git a/configure b/configure index ee5ee58a51..fb21f290b2 100755 --- a/configure +++ b/configure @@ -2093,6 +2093,7 @@ HAVE_LIST=" makeinfo makeinfo_html MMAL_PARAMETER_VIDEO_MAX_NUM_CALLBACKS + opencl_vaapi_beignet perl pod2man section_data_rel_ro @@ -6187,6 +6188,11 @@ enabled vaapi && enabled vaapi && check_lib vaapi_x11 "va/va.h va/va_x11.h" vaGetDisplay -lva -lva-x11 -lX11 +if enabled_all opencl vaapi ; then + check_type "CL/cl_intel.h" "clCreateImageFromFdINTEL_fn" && + enable opencl_vaapi_beignet +fi + enabled vdpau && check_cpp_condition vdpau/vdpau.h "defined VDP_DECODER_PROFILE_MPEG4_PART2_ASP" || disable vdpau diff --git a/libavutil/hwcontext_opencl.c b/libavutil/hwcontext_opencl.c index 0fe25d9500..7f99f5af3f 100644 --- a/libavutil/hwcontext_opencl.c +++ b/libavutil/hwcontext_opencl.c @@ -29,6 +29,14 @@ #include "mem.h" #include "pixdesc.h" +#if HAVE_OPENCL_VAAPI_BEIGNET +#include +#include +#include +#include +#include "hwcontext_vaapi.h" +#endif + typedef struct OpenCLDeviceContext { // Default command queue to use for transfer/mapping operations on @@ -41,6 +49,10 @@ typedef struct OpenCLDeviceContext { cl_platform_id platform_id; // Platform/device-specific functions. +#if HAVE_OPENCL_VAAPI_BEIGNET + int vaapi_mapping_usable; + clCreateImageFromFdINTEL_fn clCreateImageFromFdINTEL; +#endif } OpenCLDeviceContext; typedef struct OpenCLFramesContext { @@ -589,6 +601,40 @@ static int opencl_device_init(AVHWDeviceContext *hwdev) return AVERROR(EIO); } +#define CL_FUNC(name, desc) do { \ + if (fail) \ + break; \ + priv->name = clGetExtensionFunctionAddressForPlatform( \ + priv->platform_id, #name); \ + if (!priv->name) { \ + av_log(hwdev, AV_LOG_VERBOSE, \ + desc " function not found (%s).\n", #name); \ + fail = 1; \ + } else { \ + av_log(hwdev, AV_LOG_VERBOSE, \ + desc " function found (%s).\n", #name); \ + } \ + } while (0) + +#if HAVE_OPENCL_VAAPI_BEIGNET + { + int fail = 0; + + CL_FUNC(clCreateImageFromFdINTEL, + "Intel DRM to OpenCL image mapping"); + + if (fail) { + av_log(hwdev, AV_LOG_WARNING, "VAAPI to OpenCL mapping " + "not usable.\n"); + priv->vaapi_mapping_usable = 0; + } else { + priv->vaapi_mapping_usable = 1; + } + } +#endif + +#undef CL_FUNC + return 0; } @@ -606,6 +652,52 @@ static void opencl_device_uninit(AVHWDeviceContext *hwdev) } } +static int opencl_device_derive(AVHWDeviceContext *hwdev, + AVHWDeviceContext *src_ctx, + int flags) +{ + int err; + switch (src_ctx->type) { + +#if HAVE_OPENCL_VAAPI_BEIGNET + case AV_HWDEVICE_TYPE_VAAPI: + { + // Surface mapping works via DRM PRIME fds with no special + // initialisation required in advance. This just finds the + // Beignet ICD by name. + AVDictionary *opts = NULL; + + err = av_dict_set(&opts, "platform_vendor", "Intel", 0); + if (err >= 0) + err = av_dict_set(&opts, "platform_version", "beignet", 0); + if (err >= 0) { + OpenCLDeviceSelector selector = { + .platform_index = -1, + .device_index = 0, + .context = opts, + .enumerate_platforms = &opencl_enumerate_platforms, + .filter_platform = &opencl_filter_platform, + .enumerate_devices = &opencl_enumerate_devices, + .filter_device = NULL, + }; + err = opencl_device_create_internal(hwdev, &selector, NULL); + } + av_dict_free(&opts); + } + break; +#endif + + default: + err = AVERROR(ENOSYS); + break; + } + + if (err < 0) + return err; + + return opencl_device_init(hwdev); +} + static int opencl_get_plane_format(enum AVPixelFormat pixfmt, int plane, int width, int height, cl_image_format *image_format, @@ -1263,6 +1355,177 @@ fail: return err; } +#if HAVE_OPENCL_VAAPI_BEIGNET + +typedef struct VAAPItoOpenCLMapping { + VAImage va_image; + VABufferInfo va_buffer_info; + + AVOpenCLFrameDescriptor frame; +} VAAPItoOpenCLMapping; + +static void opencl_unmap_from_vaapi(AVHWFramesContext *src_fc, + HWMapDescriptor *hwmap) +{ + VAAPItoOpenCLMapping *mapping = hwmap->priv; + AVVAAPIDeviceContext *src_dev = src_fc->device_ctx->hwctx; + VASurfaceID surface_id; + VAStatus vas; + cl_int cle; + int i; + + surface_id = (VASurfaceID)(uintptr_t)hwmap->source->data[3]; + av_log(src_fc, AV_LOG_DEBUG, "Unmap VAAPI surface %#x from OpenCL.\n", + surface_id); + + for (i = 0; i < mapping->frame.nb_planes; i++) { + cle = clReleaseMemObject(mapping->frame.planes[i]); + if (cle != CL_SUCCESS) { + av_log(src_fc, AV_LOG_ERROR, "Failed to release CL " + "buffer of plane %d of VA image %#x (derived " + "from surface %#x): %d.\n", i, + mapping->va_image.buf, surface_id, cle); + } + } + + vas = vaReleaseBufferHandle(src_dev->display, + mapping->va_image.buf); + if (vas != VA_STATUS_SUCCESS) { + av_log(src_fc, AV_LOG_ERROR, "Failed to release buffer " + "handle of image %#x (derived from surface %#x): " + "%d (%s).\n", mapping->va_image.buf, surface_id, + vas, vaErrorStr(vas)); + } + + vas = vaDestroyImage(src_dev->display, + mapping->va_image.image_id); + if (vas != VA_STATUS_SUCCESS) { + av_log(src_fc, AV_LOG_ERROR, "Failed to destroy image " + "derived from surface %#x: %d (%s).\n", + surface_id, vas, vaErrorStr(vas)); + } + + av_free(mapping); +} + +static int opencl_map_from_vaapi(AVHWFramesContext *dst_fc, AVFrame *dst, + const AVFrame *src, int flags) +{ + AVHWFramesContext *src_fc = + (AVHWFramesContext*)src->hw_frames_ctx->data; + AVVAAPIDeviceContext *src_dev = src_fc->device_ctx->hwctx; + AVOpenCLDeviceContext *dst_dev = dst_fc->device_ctx->hwctx; + OpenCLDeviceContext *priv = dst_fc->device_ctx->internal->priv; + VAAPItoOpenCLMapping *mapping = NULL; + VASurfaceID surface_id; + VAStatus vas; + cl_int cle; + int err, p; + + surface_id = (VASurfaceID)(uintptr_t)src->data[3]; + av_log(src_fc, AV_LOG_DEBUG, "Map VAAPI surface %#x to OpenCL.\n", + surface_id); + + mapping = av_mallocz(sizeof(*mapping)); + if (!mapping) + return AVERROR(ENOMEM); + + vas = vaDeriveImage(src_dev->display, surface_id, + &mapping->va_image); + if (vas != VA_STATUS_SUCCESS) { + av_log(src_fc, AV_LOG_ERROR, "Failed to derive image from " + "surface %#x: %d (%s).\n", + surface_id, vas, vaErrorStr(vas)); + err = AVERROR(EIO); + goto fail; + } + + mapping->va_buffer_info.mem_type = + VA_SURFACE_ATTRIB_MEM_TYPE_DRM_PRIME; + + vas = vaAcquireBufferHandle(src_dev->display, + mapping->va_image.buf, + &mapping->va_buffer_info); + if (vas != VA_STATUS_SUCCESS) { + av_log(src_fc, AV_LOG_ERROR, "Failed to get buffer " + "handle from image %#x (derived from surface %#x): " + "%d (%s).\n", mapping->va_image.buf, surface_id, + vas, vaErrorStr(vas)); + vaDestroyImage(src_dev->display, mapping->va_image.buf); + err = AVERROR(EIO); + goto fail_derived; + } + + av_log(dst_fc, AV_LOG_DEBUG, "DRM PRIME fd is %ld.\n", + mapping->va_buffer_info.handle); + + mapping->frame.nb_planes = mapping->va_image.num_planes; + for (p = 0; p < mapping->frame.nb_planes; p++) { + cl_import_image_info_intel image_info = { + .fd = mapping->va_buffer_info.handle, + .size = mapping->va_buffer_info.mem_size, + .type = CL_MEM_OBJECT_IMAGE2D, + .offset = mapping->va_image.offsets[p], + .row_pitch = mapping->va_image.pitches[p], + }; + cl_image_desc image_desc; + + err = opencl_get_plane_format(src_fc->sw_format, p, + mapping->va_image.width, + mapping->va_image.height, + &image_info.fmt, + &image_desc); + if (err < 0) { + av_log(dst_fc, AV_LOG_ERROR, "VA %#x (derived from " + "surface %#x) has invalid parameters: %d.\n", + mapping->va_image.buf, surface_id, err); + goto fail_mapped; + } + image_info.width = image_desc.image_width; + image_info.height = image_desc.image_height; + + mapping->frame.planes[p] = + priv->clCreateImageFromFdINTEL(dst_dev->context, + &image_info, &cle); + if (!mapping->frame.planes[p]) { + av_log(dst_fc, AV_LOG_ERROR, "Failed to create CL image " + "from plane %d of VA image %#x (derived from " + "surface %#x): %d.\n", p, + mapping->va_image.buf, surface_id, cle); + err = AVERROR(EIO); + goto fail_mapped; + } + + dst->data[p] = (uint8_t*)mapping->frame.planes[p]; + } + + err = ff_hwframe_map_create(src->hw_frames_ctx, + dst, src, &opencl_unmap_from_vaapi, + mapping); + if (err < 0) + goto fail_mapped; + + dst->width = src->width; + dst->height = src->height; + + return 0; + +fail_mapped: + for (p = 0; p < mapping->frame.nb_planes; p++) { + if (mapping->frame.planes[p]) + clReleaseMemObject(mapping->frame.planes[p]); + } + vaReleaseBufferHandle(src_dev->display, mapping->va_image.buf); +fail_derived: + vaDestroyImage(src_dev->display, mapping->va_image.image_id); +fail: + av_freep(&mapping); + return err; +} + +#endif + + static int opencl_map_from(AVHWFramesContext *hwfc, AVFrame *dst, const AVFrame *src, int flags) { @@ -1272,6 +1535,38 @@ static int opencl_map_from(AVHWFramesContext *hwfc, AVFrame *dst, return opencl_map_frame(hwfc, dst, src, flags); } +static int opencl_map_to(AVHWFramesContext *hwfc, AVFrame *dst, + const AVFrame *src, int flags) +{ + OpenCLDeviceContext *priv = hwfc->device_ctx->internal->priv; + av_assert0(dst->format == AV_PIX_FMT_OPENCL); + switch (src->format) { +#if HAVE_OPENCL_VAAPI_BEIGNET + case AV_PIX_FMT_VAAPI: + if (priv->vaapi_mapping_usable) + return opencl_map_from_vaapi(hwfc, dst, src, flags); +#endif + } + return AVERROR(ENOSYS); +} + +static int opencl_frames_derive_to(AVHWFramesContext *dst_fc, + AVHWFramesContext *src_fc, int flags) +{ + OpenCLDeviceContext *priv = dst_fc->device_ctx->internal->priv; + switch (src_fc->device_ctx->type) { +#if HAVE_OPENCL_VAAPI_BEIGNET + case AV_HWDEVICE_TYPE_VAAPI: + if (!priv->vaapi_mapping_usable) + return AVERROR(ENOSYS); + break; +#endif + default: + return AVERROR(ENOSYS); + } + return opencl_frames_init_command_queue(dst_fc); +} + const HWContextType ff_hwcontext_type_opencl = { .type = AV_HWDEVICE_TYPE_OPENCL, .name = "OpenCL", @@ -1282,6 +1577,7 @@ const HWContextType ff_hwcontext_type_opencl = { .frames_priv_size = sizeof(OpenCLFramesContext), .device_create = &opencl_device_create, + .device_derive = &opencl_device_derive, .device_init = &opencl_device_init, .device_uninit = &opencl_device_uninit, @@ -1295,6 +1591,8 @@ const HWContextType ff_hwcontext_type_opencl = { .transfer_data_from = &opencl_transfer_data_from, .map_from = &opencl_map_from, + .map_to = &opencl_map_to, + .frames_derive_to = &opencl_frames_derive_to, .pix_fmts = (const enum AVPixelFormat[]) { AV_PIX_FMT_OPENCL,