From patchwork Wed Jul 5 23:05:22 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 4220 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.1.76 with SMTP id 73csp1442979vsb; Wed, 5 Jul 2017 16:05:35 -0700 (PDT) X-Received: by 10.28.16.17 with SMTP id 17mr23340777wmq.1.1499295935459; Wed, 05 Jul 2017 16:05:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1499295935; cv=none; d=google.com; s=arc-20160816; b=iU6xAUQRq3FhBPu/kl+6r18xP/esh/1uOiX1SR6pdKUiPneif6Wvh2xjHCj+b+1HO/ qjmC4bPxPVT/DhJ0DL/Zuas0b5NI5fdkvpTAosZEIvh7ruZyh5vCpXhaGfgAlseTVPQx FElMora1Q8o1OBLtf9nZq18uAbHG+u4f65AfZhO0xu2fO36zHLYmSmzn1YAZKSn59AXQ GQJ8FpmUFCgRwXshj0ekbiDtDR016n5KpvhlYJe/pOAN+Np3X2V726H0wCi5ZLMpMOxO dkwXb6B1If/vFpauCTdW01KNthyAinGQ5g9ca3lLW6n/LRHOiF5ACxQzsLt+ay4+fu2K aMRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:in-reply-to:mime-version:user-agent:date :message-id:from:references:to:dkim-signature:delivered-to :arc-authentication-results; bh=dSdLjQHj0EUZ2BfHodIo1KRwasYk8cu+vKetU6fYKEQ=; b=kW3mdeu/yEovgauctHpSELnhYfeAkbB4jU4vr4Wu9h1jp6aUYk5NV/5CivRcLYx1dC AS/cNsNHTfWfmML6qlxWnL0arZSe9JlXTiMXrCGv2aKgBu7GrLF/TXY3M6Faju7MMsAp XJMs6b4/eG8cC9T6Nub3M3ZoZvQj4vj4s5iesHI3cIGrp4Fu+fqxxNFmwsp0Taof6mP3 alZV9nMuKFCW41uAA3EtDWMN/bI4cOkEWb0bFSNJPstxtcV1pMu5q0AxWm/pXwYYxqnX e732jBvszPzwY3yotClv9DjTxYr4ymZlP9KPF2d5YQ16GQHxcOHuZBguKt9fUp1fVQiw 2Emg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.b=0Gi9RCA9; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id z128si17423034wmg.186.2017.07.05.16.05.34; Wed, 05 Jul 2017 16:05:35 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.b=0Gi9RCA9; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4E0266883A8; Thu, 6 Jul 2017 02:05:30 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr0-f173.google.com (mail-wr0-f173.google.com [209.85.128.173]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3CFBC680A65 for ; Thu, 6 Jul 2017 02:05:24 +0300 (EEST) Received: by mail-wr0-f173.google.com with SMTP id 77so4723928wrb.1 for ; Wed, 05 Jul 2017 16:05:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding; bh=AYvM0W3gE0ZlFBypLaOQH9oKePEYIA0KGI7smrl/fuc=; b=0Gi9RCA9ff7c6xxK5WJjgM5tHRuF8O9wHTluSuztcLM+2LM232oN8AUpm6CYpY/6+i hNG09TQNqhj3qN3gK4FlAMJOewhfhSEFX5S7mguR8Y3Nt2EXKbt/5je2I8dPktxUIo5X ST/ymiLX6GzL6uHBXD48FMowBL2uFC2TlvfEyUdRnuOzJWpd/XmUcA7SEDGlHpEjrg6y D1G9kk5nOWVNxv7TPiLrR5GFj5UInXweE7Sx4zXfHvM/TfnvtpL0YM4u2nYfiFuxY1IA D5AKONhYemDIfP27xE+3ppom31mOLeQEAJWMvF0rpGfY0FL6gkLWIHRPgIs0Lcxq9JW0 T/Mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=AYvM0W3gE0ZlFBypLaOQH9oKePEYIA0KGI7smrl/fuc=; b=NAuz7ngGb2j4dvey7OqtgqctVfgNF+0ZO0VtmCKwqaeuFiYYc9BKbqHkwTlHWaNmEV 0SiXrRBqqFVjGBbTsFrdGuxUdXYsaf+pqpjlDovcfHyuLEbqjoTEU6eG7eRpItrxgFug 42liW2opxymidWYOmabExSjFzvd3UrN940p9MEbvHG4rb8/oHItF9sllZo/AlpBNv7BD xCmgbb0nOVuKAfPf0vyWh+kLuWTDiX1WVwk3qQoxo+Q1bn22JCdtWTAL+gF1wQIuEMYq fLp+qoBb7rRXlNajyvD2J/gb5tkZPfTUEV7w98e2kPCfQzq/74takrx0NSD66yHXbRkF woAw== X-Gm-Message-State: AKS2vOy2Ve1DRWmNiko7Kur1pzKmstvUVqBTYIbBEjB9gL9ZtxRDYTS5 +VUVj5aM2n7ZJfbdBnI= X-Received: by 10.28.228.85 with SMTP id b82mr32354288wmh.65.1499295924090; Wed, 05 Jul 2017 16:05:24 -0700 (PDT) Received: from [192.168.0.8] (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id c203sm7944510wme.12.2017.07.05.16.05.23 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 05 Jul 2017 16:05:23 -0700 (PDT) To: FFmpeg development discussions and patches References: <5f1ebc54-9fe5-a129-7f16-272d13133887@jkqxz.net> From: Mark Thompson Message-ID: <738e0d06-a9f7-8560-035d-0a46de8c522a@jkqxz.net> Date: Thu, 6 Jul 2017 00:05:22 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <5f1ebc54-9fe5-a129-7f16-272d13133887@jkqxz.net> Subject: [FFmpeg-devel] [PATCH v2 2/2] test-only: DRM allocation for Intel, DRM to VAAPI mapping X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- For example: ./ffmpeg_g -y -init_hw_device drm:/dev/dri/card0 -i in.mp4 -an -filter_hw_device drm0 -vf 'format=nv12,hwupload,format=drm_prime,hwmap=derive_device=vaapi,format=vaapi' -c:v h264_vaapi out.mp4 (Note that it needs DRM master directly for the dumb buffer allocation, so this will fail if anything else (such as X) is running on the device.) libavutil/hwcontext_drm.c | 149 ++++++++++++++++++++++++++++++++++++++++++ libavutil/hwcontext_vaapi.c | 155 +++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 303 insertions(+), 1 deletion(-) diff --git a/libavutil/hwcontext_drm.c b/libavutil/hwcontext_drm.c index c6be3295f9..225dea676f 100644 --- a/libavutil/hwcontext_drm.c +++ b/libavutil/hwcontext_drm.c @@ -19,12 +19,15 @@ #include "config.h" #include +#include #include #if HAVE_UNISTD_H # include #endif #include +#include +#include #include #include "avassert.h" @@ -71,6 +74,150 @@ static int drm_device_create(AVHWDeviceContext *hwdev, const char *device, return 0; } +static int drm_frames_get_constraints(AVHWDeviceContext *hwdev, + const void *hwconfig, + AVHWFramesConstraints *constraints) +{ + // Hard-coded to NV12 with no size limits. + + constraints->valid_hw_formats = + av_malloc_array(2, sizeof(*constraints->valid_hw_formats)); + if (!constraints->valid_hw_formats) + return AVERROR(ENOMEM); + + constraints->valid_hw_formats[0] = AV_PIX_FMT_DRM_PRIME; + constraints->valid_hw_formats[1] = AV_PIX_FMT_NONE; + + constraints->valid_sw_formats = + av_malloc_array(2, sizeof(*constraints->valid_sw_formats)); + if (!constraints->valid_sw_formats) + return AVERROR(ENOMEM); + + constraints->valid_sw_formats[0] = AV_PIX_FMT_NV12; + constraints->valid_sw_formats[1] = AV_PIX_FMT_NONE; + + return 0; +} + +static void drm_pool_free(void *opaque, uint8_t *data) +{ + AVHWFramesContext *hwfc = opaque; + AVDRMDeviceContext *dev = hwfc->device_ctx->hwctx; + AVDRMFrameDescriptor *desc = (AVDRMFrameDescriptor*)data; + struct drm_mode_destroy_dumb destroy; + uint32_t handle; + int err, i; + + for (i = 0; i < desc->nb_objects; i++) { + err = drmPrimeFDToHandle(dev->fd, desc->objects[0].fd, &handle); + if (err < 0) { + av_log(hwfc, AV_LOG_ERROR, "Failed to retrieve DRM buffer handle " + "from fd %d: %d.\n", desc->objects[0].fd, errno); + continue; + } + + destroy.handle = handle; + ioctl(dev->fd, DRM_IOCTL_MODE_DESTROY_DUMB, &destroy); + } + + av_free(desc); +} + +static AVBufferRef *drm_pool_alloc(void *opaque, int size) +{ + AVHWFramesContext *hwfc = opaque; + AVDRMDeviceContext *dev = hwfc->device_ctx->hwctx; + AVDRMFrameDescriptor *desc; + AVBufferRef *ref; + struct drm_mode_create_dumb create; + int err, fd; + int aligned_height; + + desc = av_mallocz(sizeof(*desc)); + if (!desc) + return NULL; + + // This is to satisfy possible alignment constraints on the offset + // of the second plane (import to VAAPI on Intel requires 16-line + // alignment here). + aligned_height = FFALIGN(hwfc->height, 16); + + create.height = aligned_height * 3 / 2; + create.width = hwfc->width; + create.bpp = 8; + create.flags = 0; + + err = ioctl(dev->fd, DRM_IOCTL_MODE_CREATE_DUMB, &create); + if (err < 0) { + av_log(hwfc, AV_LOG_ERROR, "Failed to create DRM buffer: " + "%d.\n", errno); + return NULL; + } + + err = drmPrimeHandleToFD(dev->fd, create.handle, DRM_RDWR, &fd); + if (err < 0) { + struct drm_mode_destroy_dumb destroy; + + av_log(hwfc, AV_LOG_ERROR, "Failed to map DRM buffer to fd: " + "%d.\n", errno); + + destroy.handle = create.handle; + ioctl(dev->fd, DRM_IOCTL_MODE_DESTROY_DUMB, &destroy); + + return NULL; + } + + *desc = (AVDRMFrameDescriptor) { + .nb_objects = 1, + .objects[0] = { + .fd = fd, + .size = create.size, + .format_modifier = 0, + }, + .nb_layers = 2, + .layers[0] = { + .format = DRM_FORMAT_R8, + .nb_planes = 1, + .planes[0] = { + .object_index = 0, + .offset = 0, + .pitch = create.pitch, + }, + }, + .layers[1] = { + .format = DRM_FORMAT_RG88, + .nb_planes = 1, + .planes[0] = { + .object_index = 0, + .offset = aligned_height * create.pitch, + .pitch = create.pitch, + }, + }, + }; + + ref = av_buffer_create((uint8_t*)desc, sizeof(*desc), + &drm_pool_free, hwfc, 0); + if (!ref) { + drm_pool_free(hwfc, (uint8_t*)desc); + return NULL; + } + + return ref; +} + +static int drm_frames_init(AVHWFramesContext *hwfc) +{ + if (!hwfc->pool) { + hwfc->internal->pool_internal = + av_buffer_pool_init2(sizeof(AVDRMFrameDescriptor), hwfc, + &drm_pool_alloc, NULL); + if (!hwfc->internal->pool_internal) + return AVERROR(ENOMEM); + } + + return 0; +} + static int drm_get_buffer(AVHWFramesContext *hwfc, AVFrame *frame) { frame->buf[0] = av_buffer_pool_get(hwfc->pool); @@ -280,6 +427,8 @@ const HWContextType ff_hwcontext_type_drm = { .device_create = &drm_device_create, + .frames_get_constraints = &drm_frames_get_constraints, + .frames_init = &drm_frames_init, .frames_get_buffer = &drm_get_buffer, .transfer_get_formats = &drm_transfer_get_formats, diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index 3970726d30..328820e7ef 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -41,6 +41,11 @@ #include "pixdesc.h" #include "pixfmt.h" +#if CONFIG_LIBDRM +#include +# include "hwcontext_drm.h" +#endif + typedef struct VAAPIDevicePriv { #if HAVE_VAAPI_X11 Display *x11_display; @@ -897,6 +902,114 @@ static int vaapi_map_from(AVHWFramesContext *hwfc, AVFrame *dst, return 0; } +#if CONFIG_LIBDRM + +static void vaapi_unmap_from_drm(AVHWFramesContext *dst_fc, + HWMapDescriptor *hwmap) +{ + AVVAAPIDeviceContext *dst_dev = dst_fc->device_ctx->hwctx; + + VASurfaceID surface_id = (VASurfaceID)(uintptr_t)hwmap->priv; + + vaDestroySurfaces(dst_dev->display, &surface_id, 1); +} + +static int vaapi_map_from_drm(AVHWFramesContext *src_fc, AVFrame *dst, + const AVFrame *src, int flags) +{ + AVHWFramesContext *dst_fc = + (AVHWFramesContext*)dst->hw_frames_ctx->data; + AVVAAPIDeviceContext *dst_dev = dst_fc->device_ctx->hwctx; + const AVDRMFrameDescriptor *desc; + VASurfaceID surface_id; + VAStatus vas; + int err, plane; + + unsigned long buffer_handle; + VASurfaceAttribExternalBuffers buffer_desc; + VASurfaceAttrib attrs[2] = { + { + .type = VASurfaceAttribMemoryType, + .flags = VA_SURFACE_ATTRIB_SETTABLE, + .value.type = VAGenericValueTypeInteger, + .value.value.i = VA_SURFACE_ATTRIB_MEM_TYPE_DRM_PRIME, + }, + { + .type = VASurfaceAttribExternalBufferDescriptor, + .flags = VA_SURFACE_ATTRIB_SETTABLE, + .value.type = VAGenericValueTypePointer, + .value.value.p = &buffer_desc, + } + }; + + desc = (AVDRMFrameDescriptor*)src->data[0]; + + if (desc->nb_objects != 1) { + av_log(dst_fc, AV_LOG_ERROR, "VAAPI can only map frames " + "made from single DRM objects.\n"); + return AVERROR(EINVAL); + } + + av_log(dst_fc, AV_LOG_DEBUG, "Map DRM object %d to VAAPI.\n", + desc->objects[0].fd); + + buffer_handle = desc->objects[0].fd; + buffer_desc.pixel_format = VA_FOURCC_NV12; + buffer_desc.width = src_fc->width; + buffer_desc.height = src_fc->height; + buffer_desc.data_size = desc->objects[0].size; + + buffer_desc.num_planes = 2; + for (plane = 0; plane < 2; plane++) { + buffer_desc.pitches[plane] = desc->layers[plane].planes[0].pitch; + buffer_desc.offsets[plane] = desc->layers[plane].planes[0].offset; + } + + buffer_desc.buffers = &buffer_handle; + buffer_desc.num_buffers = 1; + buffer_desc.flags = 0; + + vas = vaCreateSurfaces(dst_dev->display, + VA_RT_FORMAT_YUV420, + src->width, src->height, + &surface_id, 1, + attrs, FF_ARRAY_ELEMS(attrs)); + if (vas != VA_STATUS_SUCCESS) { + av_log(dst_fc, AV_LOG_ERROR, "Failed to create surface from DRM " + "object: %d (%s).\n", vas, vaErrorStr(vas)); + return AVERROR(EIO); + } + + err = ff_hwframe_map_create(dst->hw_frames_ctx, dst, src, + &vaapi_unmap_from_drm, + (void*)(uintptr_t)surface_id); + if (err < 0) + return err; + + dst->width = src->width; + dst->height = src->height; + dst->data[3] = (uint8_t*)(uintptr_t)surface_id; + + av_log(dst_fc, AV_LOG_DEBUG, "Mapped DRM object %d to " + "surface %#x.\n", desc->objects[0].fd, surface_id); + + return 0; +} +#endif + +static int vaapi_map_to(AVHWFramesContext *hwfc, AVFrame *dst, + const AVFrame *src, int flags) +{ + switch (src->format) { +#if CONFIG_LIBDRM + case AV_PIX_FMT_DRM_PRIME: + return vaapi_map_from_drm(hwfc, dst, src, flags); +#endif + default: + return AVERROR(ENOSYS); + } +} + static void vaapi_device_free(AVHWDeviceContext *ctx) { AVVAAPIDeviceContext *hwctx = ctx->hwctx; @@ -999,6 +1112,45 @@ static int vaapi_device_create(AVHWDeviceContext *ctx, const char *device, return 0; } +static int vaapi_device_derive(AVHWDeviceContext *ctx, + AVHWDeviceContext *src_ctx, int flags) +{ +#if CONFIG_LIBDRM + if (src_ctx->type == AV_HWDEVICE_TYPE_DRM) { + AVDRMDeviceContext *src_hwctx = src_ctx->hwctx; + AVVAAPIDeviceContext *hwctx = ctx->hwctx; + VADisplay *display; + VAStatus vas; + int major, minor; + + if (src_hwctx->fd < 0) { + av_log(ctx, AV_LOG_ERROR, "DRM instance requires an associated " + "device to derive a VA display from.\n"); + return AVERROR(EINVAL); + } + + display = vaGetDisplayDRM(src_hwctx->fd); + if (!display) { + av_log(ctx, AV_LOG_ERROR, "Failed to open a VA display from " + "DRM device.\n"); + return AVERROR(EIO); + } + + vas = vaInitialize(display, &major, &minor); + if (vas != VA_STATUS_SUCCESS) { + av_log(ctx, AV_LOG_ERROR, "Failed to initialise VAAPI " + "connection: %d (%s).\n", vas, vaErrorStr(vas)); + vaTerminate(display); + return AVERROR(EIO); + } + + hwctx->display = display; + return 0; + } +#endif + return AVERROR(ENOSYS); +} + const HWContextType ff_hwcontext_type_vaapi = { .type = AV_HWDEVICE_TYPE_VAAPI, .name = "VAAPI", @@ -1010,6 +1162,7 @@ const HWContextType ff_hwcontext_type_vaapi = { .frames_priv_size = sizeof(VAAPIFramesContext), .device_create = &vaapi_device_create, + .device_derive = &vaapi_device_derive, .device_init = &vaapi_device_init, .device_uninit = &vaapi_device_uninit, .frames_get_constraints = &vaapi_frames_get_constraints, @@ -1019,7 +1172,7 @@ const HWContextType ff_hwcontext_type_vaapi = { .transfer_get_formats = &vaapi_transfer_get_formats, .transfer_data_to = &vaapi_transfer_data_to, .transfer_data_from = &vaapi_transfer_data_from, - .map_to = NULL, + .map_to = &vaapi_map_to, .map_from = &vaapi_map_from, .pix_fmts = (const enum AVPixelFormat[]) {