From patchwork Tue Nov 14 19:47:18 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 6062 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.94 with SMTP id m30csp3721127jah; Tue, 14 Nov 2017 11:48:13 -0800 (PST) X-Google-Smtp-Source: AGs4zMaMWDT0i/3s/1x2thNccz4fsSt4FYspjSInQDfdHpw2gNwzlJU3X09myRaEBekqNxF0n+Bk X-Received: by 10.28.29.130 with SMTP id d124mr4665427wmd.73.1510688893375; Tue, 14 Nov 2017 11:48:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510688893; cv=none; d=google.com; s=arc-20160816; b=MRnXdY0dQ3YUW6dUrCJk7D/OQV9ggooKUm+2wXhmY5Me1JQ8HnWVyY60DXFFmRqPRj L8lYtgZteMc6In3tnggzTG3Y3mTkHN0mLSqSZJqwvZKCKNeKyFJ54NCDLpsWHpY4NRZT Y+jhBCjBMXsnfr7PlBboXCO3iEGBNaQeuCYAO40BWnDdpdz3RBLXF5sttBb2WB7HOWwd 32qQKW8uZJ08aW+T/Jnv2f4UTNGa5qKfZ3HxjvK8zuxEl9Kop5y4DdPHAcgZKV2a6IWn Lkc8ZZva+qMKtDNck4upSyKqyhsi2SLMcsCctHCmbvNOgBQrtrk7bdzBdTiRLxl+1UiU kUxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=EvjTPSWwvC4xjaS2UVu4EQDqshnWpXH6wnO5bE8I+Pk=; b=rqwTXTi9XdOiYiBlYK85QPR8txY1MHaLyQQbSVz/kjzXVGISDW7ixGf11z2tcmCccJ FWBm6ZJWrl5VsPnWTXiz+wl1p+paPcbNSw/AT2cwcP08pXDtMXbJmGEMjjUJM8lm9nrP G2EpZ0XhgmA7I5XXq3OLMR4l0ZVZ31vt5kdk+HXWqTg6+eed3G5MPZi+f3+5Sv4phzys 7t/hLfbDBlfHbHJ7FdcKo3OFEyzGZ5ixmAHdRcNCKWqyIeHfGv6Vpt6gbNiG3gACT5IM QCPvxmQwJSbgWj02l23uaPD/eRWcp0+W+xrxFBW6ikKRIyRB43i9B0IStsBGpjmsLoOe Gh9A== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=yIY5TxGY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id u73si17366484wrb.529.2017.11.14.11.48.12; Tue, 14 Nov 2017 11:48:13 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=yIY5TxGY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9F2B168A13C; Tue, 14 Nov 2017 21:47:31 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f51.google.com (mail-wm0-f51.google.com [74.125.82.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3B47868043B for ; Tue, 14 Nov 2017 21:47:23 +0200 (EET) Received: by mail-wm0-f51.google.com with SMTP id l8so18229430wmg.4 for ; Tue, 14 Nov 2017 11:47:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=8BSgmB/1f6ZxUloPFqEfbTz5xNpvyF+B7szMhGjgodg=; b=yIY5TxGY4llWvrBjq6Qqg/hfrK66De2D4JqcE0/j8VBOgL12KtgpBywLWpgUjTVrJ8 zxWJIT1cq25KHudkVieHDaaOCP8Y51jD21Vhy2zAQBIXc0slRLnxzt8wsq9v5riGX+W2 BIIPBSVjXM0DpMB6u66SYOa6hhmocDeo1oLXNT+LLu+tWf7FSc2xahe9l9QDSmECuYd1 SXAph4Ss3rNzD/g8sE3jowHX7xtDjAyBF2QOHSY0frk2LAPiRiUjYNfKvJAlokzSZZnP nzr9zkZ8jKnOOXce7wrXo8bXEA9kkmmW14dDSF1FErL4109z92oqVvXXqJ0pkysBYRso LVtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=8BSgmB/1f6ZxUloPFqEfbTz5xNpvyF+B7szMhGjgodg=; b=HlSS1svqspPs+8lxSVN3WSZjv4bqbQbqbXzCDckzN61rGeIcSuDNfFiJEFdldUk2Cc /SEvMuVF9EV+o+xE4KlTHKoYusDMo7OZoHDfWdpj2+jPlPV+fKZGyr+wD7awumh8WPFL gTvbONHxNDDITf5SHBBPfD5WfCHDMivuKOhjIPXK/hLyAdzM975n5JIJOvyEN/8wNgl+ /2eqaTfFVlY6hepM2c4EWRNQP4OsDXUIWeByZB+ALRfLFycUkr/vMbIXze33GOlJ57KT KOS5EfnUj2DUxWM275Sc9q+A7E+8wclEPP6OB3E3reQkZSreX3FXaBPSP70jC9+9VIcd DmbQ== X-Gm-Message-State: AJaThX77yXpkDmNscgwwC8TQU4WPjOALXV2i0jS4B40h61sf5vv6LYvC dc1rYWkKySbJNhY5r2W/ghv45Nx9 X-Received: by 10.28.61.135 with SMTP id k129mr4659741wma.81.1510688857629; Tue, 14 Nov 2017 11:47:37 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id v35sm38938226wrc.13.2017.11.14.11.47.36 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 14 Nov 2017 11:47:37 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Tue, 14 Nov 2017 19:47:18 +0000 Message-Id: <20171114194730.11052-4-sw@jkqxz.net> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20171114194730.11052-1-sw@jkqxz.net> References: <20171114194730.11052-1-sw@jkqxz.net> Subject: [FFmpeg-devel] [PATCH 03/15] hwcontext_opencl: VAAPI to OpenCL mapping for Intel i965+beignet X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Supports all surface formats in common between the two. --- configure | 6 + libavutil/hwcontext_opencl.c | 298 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 304 insertions(+) diff --git a/configure b/configure index 167db274f1..dcdb5fee1f 100755 --- a/configure +++ b/configure @@ -2121,6 +2121,7 @@ HAVE_LIST=" $TYPES_LIST makeinfo makeinfo_html + opencl_vaapi_beignet perl pod2man texi2html @@ -6150,6 +6151,11 @@ enabled vaapi && check_cpp_condition "va/va.h" "VA_CHECK_VERSION(1, 0, 0)" && enable vaapi_1 +if enabled_all opencl vaapi ; then + check_type "CL/cl_intel.h" "clCreateImageFromFdINTEL_fn" && + enable opencl_vaapi_beignet +fi + enabled vdpau && check_cpp_condition vdpau/vdpau.h "defined VDP_DECODER_PROFILE_MPEG4_PART2_ASP" || disable vdpau diff --git a/libavutil/hwcontext_opencl.c b/libavutil/hwcontext_opencl.c index 0fe25d9500..7f99f5af3f 100644 --- a/libavutil/hwcontext_opencl.c +++ b/libavutil/hwcontext_opencl.c @@ -29,6 +29,14 @@ #include "mem.h" #include "pixdesc.h" +#if HAVE_OPENCL_VAAPI_BEIGNET +#include +#include +#include +#include +#include "hwcontext_vaapi.h" +#endif + typedef struct OpenCLDeviceContext { // Default command queue to use for transfer/mapping operations on @@ -41,6 +49,10 @@ typedef struct OpenCLDeviceContext { cl_platform_id platform_id; // Platform/device-specific functions. +#if HAVE_OPENCL_VAAPI_BEIGNET + int vaapi_mapping_usable; + clCreateImageFromFdINTEL_fn clCreateImageFromFdINTEL; +#endif } OpenCLDeviceContext; typedef struct OpenCLFramesContext { @@ -589,6 +601,40 @@ static int opencl_device_init(AVHWDeviceContext *hwdev) return AVERROR(EIO); } +#define CL_FUNC(name, desc) do { \ + if (fail) \ + break; \ + priv->name = clGetExtensionFunctionAddressForPlatform( \ + priv->platform_id, #name); \ + if (!priv->name) { \ + av_log(hwdev, AV_LOG_VERBOSE, \ + desc " function not found (%s).\n", #name); \ + fail = 1; \ + } else { \ + av_log(hwdev, AV_LOG_VERBOSE, \ + desc " function found (%s).\n", #name); \ + } \ + } while (0) + +#if HAVE_OPENCL_VAAPI_BEIGNET + { + int fail = 0; + + CL_FUNC(clCreateImageFromFdINTEL, + "Intel DRM to OpenCL image mapping"); + + if (fail) { + av_log(hwdev, AV_LOG_WARNING, "VAAPI to OpenCL mapping " + "not usable.\n"); + priv->vaapi_mapping_usable = 0; + } else { + priv->vaapi_mapping_usable = 1; + } + } +#endif + +#undef CL_FUNC + return 0; } @@ -606,6 +652,52 @@ static void opencl_device_uninit(AVHWDeviceContext *hwdev) } } +static int opencl_device_derive(AVHWDeviceContext *hwdev, + AVHWDeviceContext *src_ctx, + int flags) +{ + int err; + switch (src_ctx->type) { + +#if HAVE_OPENCL_VAAPI_BEIGNET + case AV_HWDEVICE_TYPE_VAAPI: + { + // Surface mapping works via DRM PRIME fds with no special + // initialisation required in advance. This just finds the + // Beignet ICD by name. + AVDictionary *opts = NULL; + + err = av_dict_set(&opts, "platform_vendor", "Intel", 0); + if (err >= 0) + err = av_dict_set(&opts, "platform_version", "beignet", 0); + if (err >= 0) { + OpenCLDeviceSelector selector = { + .platform_index = -1, + .device_index = 0, + .context = opts, + .enumerate_platforms = &opencl_enumerate_platforms, + .filter_platform = &opencl_filter_platform, + .enumerate_devices = &opencl_enumerate_devices, + .filter_device = NULL, + }; + err = opencl_device_create_internal(hwdev, &selector, NULL); + } + av_dict_free(&opts); + } + break; +#endif + + default: + err = AVERROR(ENOSYS); + break; + } + + if (err < 0) + return err; + + return opencl_device_init(hwdev); +} + static int opencl_get_plane_format(enum AVPixelFormat pixfmt, int plane, int width, int height, cl_image_format *image_format, @@ -1263,6 +1355,177 @@ fail: return err; } +#if HAVE_OPENCL_VAAPI_BEIGNET + +typedef struct VAAPItoOpenCLMapping { + VAImage va_image; + VABufferInfo va_buffer_info; + + AVOpenCLFrameDescriptor frame; +} VAAPItoOpenCLMapping; + +static void opencl_unmap_from_vaapi(AVHWFramesContext *src_fc, + HWMapDescriptor *hwmap) +{ + VAAPItoOpenCLMapping *mapping = hwmap->priv; + AVVAAPIDeviceContext *src_dev = src_fc->device_ctx->hwctx; + VASurfaceID surface_id; + VAStatus vas; + cl_int cle; + int i; + + surface_id = (VASurfaceID)(uintptr_t)hwmap->source->data[3]; + av_log(src_fc, AV_LOG_DEBUG, "Unmap VAAPI surface %#x from OpenCL.\n", + surface_id); + + for (i = 0; i < mapping->frame.nb_planes; i++) { + cle = clReleaseMemObject(mapping->frame.planes[i]); + if (cle != CL_SUCCESS) { + av_log(src_fc, AV_LOG_ERROR, "Failed to release CL " + "buffer of plane %d of VA image %#x (derived " + "from surface %#x): %d.\n", i, + mapping->va_image.buf, surface_id, cle); + } + } + + vas = vaReleaseBufferHandle(src_dev->display, + mapping->va_image.buf); + if (vas != VA_STATUS_SUCCESS) { + av_log(src_fc, AV_LOG_ERROR, "Failed to release buffer " + "handle of image %#x (derived from surface %#x): " + "%d (%s).\n", mapping->va_image.buf, surface_id, + vas, vaErrorStr(vas)); + } + + vas = vaDestroyImage(src_dev->display, + mapping->va_image.image_id); + if (vas != VA_STATUS_SUCCESS) { + av_log(src_fc, AV_LOG_ERROR, "Failed to destroy image " + "derived from surface %#x: %d (%s).\n", + surface_id, vas, vaErrorStr(vas)); + } + + av_free(mapping); +} + +static int opencl_map_from_vaapi(AVHWFramesContext *dst_fc, AVFrame *dst, + const AVFrame *src, int flags) +{ + AVHWFramesContext *src_fc = + (AVHWFramesContext*)src->hw_frames_ctx->data; + AVVAAPIDeviceContext *src_dev = src_fc->device_ctx->hwctx; + AVOpenCLDeviceContext *dst_dev = dst_fc->device_ctx->hwctx; + OpenCLDeviceContext *priv = dst_fc->device_ctx->internal->priv; + VAAPItoOpenCLMapping *mapping = NULL; + VASurfaceID surface_id; + VAStatus vas; + cl_int cle; + int err, p; + + surface_id = (VASurfaceID)(uintptr_t)src->data[3]; + av_log(src_fc, AV_LOG_DEBUG, "Map VAAPI surface %#x to OpenCL.\n", + surface_id); + + mapping = av_mallocz(sizeof(*mapping)); + if (!mapping) + return AVERROR(ENOMEM); + + vas = vaDeriveImage(src_dev->display, surface_id, + &mapping->va_image); + if (vas != VA_STATUS_SUCCESS) { + av_log(src_fc, AV_LOG_ERROR, "Failed to derive image from " + "surface %#x: %d (%s).\n", + surface_id, vas, vaErrorStr(vas)); + err = AVERROR(EIO); + goto fail; + } + + mapping->va_buffer_info.mem_type = + VA_SURFACE_ATTRIB_MEM_TYPE_DRM_PRIME; + + vas = vaAcquireBufferHandle(src_dev->display, + mapping->va_image.buf, + &mapping->va_buffer_info); + if (vas != VA_STATUS_SUCCESS) { + av_log(src_fc, AV_LOG_ERROR, "Failed to get buffer " + "handle from image %#x (derived from surface %#x): " + "%d (%s).\n", mapping->va_image.buf, surface_id, + vas, vaErrorStr(vas)); + vaDestroyImage(src_dev->display, mapping->va_image.buf); + err = AVERROR(EIO); + goto fail_derived; + } + + av_log(dst_fc, AV_LOG_DEBUG, "DRM PRIME fd is %ld.\n", + mapping->va_buffer_info.handle); + + mapping->frame.nb_planes = mapping->va_image.num_planes; + for (p = 0; p < mapping->frame.nb_planes; p++) { + cl_import_image_info_intel image_info = { + .fd = mapping->va_buffer_info.handle, + .size = mapping->va_buffer_info.mem_size, + .type = CL_MEM_OBJECT_IMAGE2D, + .offset = mapping->va_image.offsets[p], + .row_pitch = mapping->va_image.pitches[p], + }; + cl_image_desc image_desc; + + err = opencl_get_plane_format(src_fc->sw_format, p, + mapping->va_image.width, + mapping->va_image.height, + &image_info.fmt, + &image_desc); + if (err < 0) { + av_log(dst_fc, AV_LOG_ERROR, "VA %#x (derived from " + "surface %#x) has invalid parameters: %d.\n", + mapping->va_image.buf, surface_id, err); + goto fail_mapped; + } + image_info.width = image_desc.image_width; + image_info.height = image_desc.image_height; + + mapping->frame.planes[p] = + priv->clCreateImageFromFdINTEL(dst_dev->context, + &image_info, &cle); + if (!mapping->frame.planes[p]) { + av_log(dst_fc, AV_LOG_ERROR, "Failed to create CL image " + "from plane %d of VA image %#x (derived from " + "surface %#x): %d.\n", p, + mapping->va_image.buf, surface_id, cle); + err = AVERROR(EIO); + goto fail_mapped; + } + + dst->data[p] = (uint8_t*)mapping->frame.planes[p]; + } + + err = ff_hwframe_map_create(src->hw_frames_ctx, + dst, src, &opencl_unmap_from_vaapi, + mapping); + if (err < 0) + goto fail_mapped; + + dst->width = src->width; + dst->height = src->height; + + return 0; + +fail_mapped: + for (p = 0; p < mapping->frame.nb_planes; p++) { + if (mapping->frame.planes[p]) + clReleaseMemObject(mapping->frame.planes[p]); + } + vaReleaseBufferHandle(src_dev->display, mapping->va_image.buf); +fail_derived: + vaDestroyImage(src_dev->display, mapping->va_image.image_id); +fail: + av_freep(&mapping); + return err; +} + +#endif + + static int opencl_map_from(AVHWFramesContext *hwfc, AVFrame *dst, const AVFrame *src, int flags) { @@ -1272,6 +1535,38 @@ static int opencl_map_from(AVHWFramesContext *hwfc, AVFrame *dst, return opencl_map_frame(hwfc, dst, src, flags); } +static int opencl_map_to(AVHWFramesContext *hwfc, AVFrame *dst, + const AVFrame *src, int flags) +{ + OpenCLDeviceContext *priv = hwfc->device_ctx->internal->priv; + av_assert0(dst->format == AV_PIX_FMT_OPENCL); + switch (src->format) { +#if HAVE_OPENCL_VAAPI_BEIGNET + case AV_PIX_FMT_VAAPI: + if (priv->vaapi_mapping_usable) + return opencl_map_from_vaapi(hwfc, dst, src, flags); +#endif + } + return AVERROR(ENOSYS); +} + +static int opencl_frames_derive_to(AVHWFramesContext *dst_fc, + AVHWFramesContext *src_fc, int flags) +{ + OpenCLDeviceContext *priv = dst_fc->device_ctx->internal->priv; + switch (src_fc->device_ctx->type) { +#if HAVE_OPENCL_VAAPI_BEIGNET + case AV_HWDEVICE_TYPE_VAAPI: + if (!priv->vaapi_mapping_usable) + return AVERROR(ENOSYS); + break; +#endif + default: + return AVERROR(ENOSYS); + } + return opencl_frames_init_command_queue(dst_fc); +} + const HWContextType ff_hwcontext_type_opencl = { .type = AV_HWDEVICE_TYPE_OPENCL, .name = "OpenCL", @@ -1282,6 +1577,7 @@ const HWContextType ff_hwcontext_type_opencl = { .frames_priv_size = sizeof(OpenCLFramesContext), .device_create = &opencl_device_create, + .device_derive = &opencl_device_derive, .device_init = &opencl_device_init, .device_uninit = &opencl_device_uninit, @@ -1295,6 +1591,8 @@ const HWContextType ff_hwcontext_type_opencl = { .transfer_data_from = &opencl_transfer_data_from, .map_from = &opencl_map_from, + .map_to = &opencl_map_to, + .frames_derive_to = &opencl_frames_derive_to, .pix_fmts = (const enum AVPixelFormat[]) { AV_PIX_FMT_OPENCL,