From patchwork Fri Feb 23 14:11:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 46456 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:26a3:b0:19e:cdac:8cce with SMTP id h35csp980486pze; Fri, 23 Feb 2024 06:12:11 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCUWHlOR0CiHSlcuX0wBS9MI5ZuW38MbLdZwcblz5DrhnkyuVE9c+9z5n6vm3lVOd0WVZI5yH8vVOEstfA8UOR69P829CL5WzRayqw== X-Google-Smtp-Source: AGHT+IG8O+P4qnJ6EwvYAvU0O7R2/nTGVGvYRfJln5u+f5UR5fW69x9a3yjyeHOvDKU0xkkdh1d+ X-Received: by 2002:aa7:c446:0:b0:565:4f82:b260 with SMTP id n6-20020aa7c446000000b005654f82b260mr1181583edr.14.1708697531708; Fri, 23 Feb 2024 06:12:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1708697531; cv=none; d=google.com; s=arc-20160816; b=bplRTe1Hb6taAtfCGkpgdyZV7qtuwnn2zFafmV7NIAZXo5td4RYihnkr4ziScHa7A9 MYCCSj6m7kjMVgzX3mwbCZawzX2TUpcgqS12Nk+IooZssSs6G/e5w9N9t7H/I6TE32n/ 7OWn41mBllXP586ucy7Wa0/CzaW8gBak78TR66Zi9pz8i6yXfAdTr3EYyABZ3oLtFj54 na70Sx6SH64CzjgHds3v4zXettFhFLnOy+PwLNFTpFuIy4sCYJa+ULuNCqCCORJtisPv 3gIu8VQFnR5oVKAK7ffH0lJ7e5v8Pui1bXm1m6UbqugMfdUpm8NL2zMzEJLfzkYWYClk 4lCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:date:to:from:message-id :dkim-signature:delivered-to; bh=DZJ6S0Dp9SHFjwBrdPa+LA6hJd54NKL9yCaquXwYyUU=; fh=HnHYuZ9XgUo86ZRXTLWWmQxhslYEI9B9taZ5X1DLFfc=; b=qRf0wAgAsnfOLKgAjzkJ0vo2BQw6kg1pgskrXr9om6/o5OM9Ui7kwblcvJQRJnF0Wi e4uqjZIxWD1Bgq0fVK0QoqRzxl2fjOcedHkS2G5141GtJjbeqBI+5L5pI5Q0ZR0sltd/ sNreE2DEvOkA2LlOoXl4h7Ir+vQZ4CMivCyBxmiQ7kxgTP+k0jin2eKj3EZIFZ7VuSCK 8TeXRMUhy6H/Zsl0oRShZ8XkwfCkj9ftd9foKuyQ7gwyHAMncPrdj/A9Ey12YstcCOck zqY2BsZuvz5C6zk/sUuZyg9ir18pInh5AipPPBS9cD3cvhK6bEaHdKqlQtfLL4XPeLH5 DJJA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=tWQEwin7; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b9-20020a056402278900b00565854101ffsi181872ede.449.2024.02.23.06.12.11; Fri, 23 Feb 2024 06:12:11 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=tWQEwin7; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 108B368C5E7; Fri, 23 Feb 2024 16:12:09 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-57-210.mail.qq.com (out162-62-57-210.mail.qq.com [162.62.57.210]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 470D1680122 for ; Fri, 23 Feb 2024 16:12:01 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1708697511; bh=kaTqSmBMQoeAqrSDfrfkJeOgz6kQmX49WyC95D5yABI=; h=From:To:Cc:Subject:Date; b=tWQEwin7vGP7MiV7B9LHshQjw8u3I87/DWiXDk1eYj4zGNlRaUnLLX0efA1A02t+E W5HQiMXmHKShk+qmDN4eJa6QYtw3rzaC05h5ZuCvxJVH+UpJircO1R9pC/huOxAec2 nB1oTC6MBRrEsN26WWuVhVvDIYxtdDc9uA4rmFuI= Received: from ZHILIZHAO-MB1.tencent.com ([119.147.10.198]) by newxmesmtplogicsvrszc5-0.qq.com (NewEsmtp) with SMTP id 2F18B87A; Fri, 23 Feb 2024 22:11:49 +0800 X-QQ-mid: xmsmtpt1708697509tf9kvlh2q Message-ID: X-QQ-XMAILINFO: MOpJcPT3Yy24DHNByNm6wbiR+QFs5RaPXRkSsIZVdRTww3W2MlQYgA5VDAc5Fz 2bP+NuatQY8YWCyMLqczXRX5+TuHwdtE8u1IkWLBXK+BQvFMD+7Hf4/QrQzXew8K39BOlVA4S8jR Uv0sQ25oZs+3p+Hr8dcOjQtC7sUoIhwvLX2LflFXacObQxzdlr93yu/HlwuQTrM5+ffsJa+Ss+MO UcYxGlzq+Uvey86sdvcuSRWvObgKFG8WKn6zqMXSa1Ulfkh5z0qDQeejMHCFitjBmXoIuJn6gDJR wIGJVnxI4PrIY42j5DNMI+BYHlZ9lhLYo86LSy2WyxCnlml67b2k3S4GyFh9bmfcPkwgNwEJmAub RPzxQYefrR2CoOvJtvewkMJdWs9P2zASneKxe5sMkUtMvNqJeUyRE8CYyrSfk1xf+fcM9lJa3bGd u4dnZ90/iyXvVtn6ErGeuxfxT66lgJFReb5aJ4dnvZ+hGSmuKi8FhKzeWShNRnfGdLNHG/9QoUDJ GUHqg/2xXovo5MtG45kQxxOiw5v9r0m1UKIY76CRuoFbF1vHgmnQ9a7rwqWlvTJgLR1g4jtCD6AT z0pRZ2+oM211Hq5LtOiMlgQ2qIR5MZhXHT3cQAVnHSz79K//AHK7KEUocv+S74y+xrUWw2IMnt3F 4bBwlXqIiw5R2bzSH43BuJctGCRod6J/iLuLbjBX43feoTOufwdzjY6RTsxIv+7+tIBDe/s+9DaC 0i0lEtnGoS9QzsA5D3qzgL08JFa+LPwp2nu4XjsKcKPLhykFo7fL4hWcvckgFBf1WxjFsSlfPW9F KKoKlbUG4bSTmG00BPKz1tY2unPeOuBCmMi4zz6pKxR3O7Oz0Shk54Gd29BwWERWrrtlzAJNBGxy oHk0ZuGuVrdoEL1PkAAH0TWLJ3i6kayiwxdcEa3HRFZoy5yun5jESfwIW9irJ6RCLhoJZhGXc9ib EycjSWBNVxiMTbVXtsvdI79DaCggoc7SKsuRef5qcZ/Glh1ZpewqfFMswP3+UD X-QQ-XMRINFO: NyFYKkN4Ny6FSmKK/uo/jdU= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Fri, 23 Feb 2024 22:11:42 +0800 X-OQ-MSGID: <20240223141143.16065-1-quinkblack@foxmail.com> X-Mailer: git-send-email 2.42.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] avutil/hwcontext_opencl: Fix missing linesize when map from opencl X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Zhao Zhili Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Y9rbmzXY6wKV From: Zhao Zhili --- libavutil/hwcontext_opencl.c | 1 + 1 file changed, 1 insertion(+) diff --git a/libavutil/hwcontext_opencl.c b/libavutil/hwcontext_opencl.c index c1764ed098..de093fffb1 100644 --- a/libavutil/hwcontext_opencl.c +++ b/libavutil/hwcontext_opencl.c @@ -2014,6 +2014,7 @@ static int opencl_map_frame(AVHWFramesContext *hwfc, AVFrame *dst, } dst->data[p] = map->address[p]; + dst->linesize[p] = row_pitch; av_log(hwfc, AV_LOG_DEBUG, "Map plane %d (%p -> %p).\n", p, src->data[p], dst->data[p]); From patchwork Fri Feb 23 14:11:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 46457 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:26a3:b0:19e:cdac:8cce with SMTP id h35csp980572pze; Fri, 23 Feb 2024 06:12:21 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCWte4xjZlwIpWhBr/aC9RZWffltwKHWECRwc0hPIEPlibyQ8l7BtrFmnJJ2iuKKvB4S9cH0fC8FERWjt5NflnThdCYejNLtotHaYA== X-Google-Smtp-Source: AGHT+IHMakjKl1E3kWe6pZx2XrEdqF0p2mVeMRhz0U/8o1Ip+LnhnPNXpd9Uc4C/UZVwbssWLWCh X-Received: by 2002:aa7:d899:0:b0:564:c034:8146 with SMTP id u25-20020aa7d899000000b00564c0348146mr1488339edq.37.1708697540977; Fri, 23 Feb 2024 06:12:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1708697540; cv=none; d=google.com; s=arc-20160816; b=jmJF/QEO69KoOTarihPfkO/sfqFVdfen5PolzVEZauJcXNGb9HRJvmEkQCtizFP8Bm tn3j4JbSrywn5TbNakaAG+5Vob30UqeAWbeuqlAqKCEbqyJZQEqjaE3BYfrW6hrWymCp E3meONZCPkvA1ef2xF1TERKtlNJnaso9fKYsM0YlAjW5AvFTbq4EmHePFqYSmTb7rP9z kbq1zf7FNmFdxOuM6f3ZezNQ3ywK/OBi1+awVSSWiv4zSo9LQxYhAYONOCi/w/ht9RLN 8lmbN1B2mqX6Ps4xFzNQngup2pfxiXMfYmCIz9+aJQgln/pxXrMKGIp05zgScMe621AB rsjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=Nqc9WxpNNRZD6TpvpUWBBny9nXmpE/M+iJk2X3lyd1s=; fh=HnHYuZ9XgUo86ZRXTLWWmQxhslYEI9B9taZ5X1DLFfc=; b=S2YwiZe/KnqE16dXc2MgGpHWD+ZlYw4M9tb2l0mf82dKKc4CtmvDcNwFpDiiuEjNID AYn+61Xx1jyhik+0ZMbFLysHqVl2LiVok3kYg0IR5LgjAA/IvUtbBw2Q4hT1kBbPJxF8 /ddozNxJHY4lQSV6FHa5lIsJaHMv83AxxvUI1F+eH154Yr0QbUBF6nzSvNCSQiGkrE2u 450zVN/C5TE72zKtxFGd2wd7ju+55KwUdecBNbZ2Q2hywQqHGnu5OeSpEOwpQujK4d+2 NET5uFWEYLSfjlg/Ol9txsm8SrjVlI/SUxsh2IDXZUTeDCUVHu0RMAg6siyW0hnRdiGL UdRA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b="XDX3K/qo"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x8-20020aa7cd88000000b005640c092a86si6359644edv.87.2024.02.23.06.12.20; Fri, 23 Feb 2024 06:12:20 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b="XDX3K/qo"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 300E368C5E6; Fri, 23 Feb 2024 16:12:12 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out203-205-251-60.mail.qq.com (out203-205-251-60.mail.qq.com [203.205.251.60]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 802E768C5E6 for ; Fri, 23 Feb 2024 16:12:04 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1708697516; bh=4Fj7AL3sMcOcJY4wlqbwl+KWiTs8OZSWUGL67dtZftk=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=XDX3K/qoZAA02Z5Bm8Zjp4iywRQ7c/SFDMWfkuI0Qqns3I9GdFITOfi5dgEevWz// vpp1X9XZdCFR6Gcu0BUbRx/Pu1Ds36vxwRVvQy1comSnfbyZLgaW5CezuZ1WxycGk5 dU6mlPMWrXDPCM7KP2JTiW/JZdxkWuPOT281O9Xk= Received: from ZHILIZHAO-MB1.tencent.com ([119.147.10.198]) by newxmesmtplogicsvrszc5-0.qq.com (NewEsmtp) with SMTP id 2F18B87A; Fri, 23 Feb 2024 22:11:49 +0800 X-QQ-mid: xmsmtpt1708697510t5bjpjf3b Message-ID: X-QQ-XMAILINFO: N4CzT1PKe+qFHsrQwYIuRQcMtHjSY0t/hMe1Kqsf5a+h1Qt8C3FGOmcauWhafP O8epPPUQsxCb/nYCN4qCggJHrGALf3Fa9oLHTh9iLzwJ2kI8FnSJXTk56YXPnzFCVyS481brep26 MhNmnisw0WnbhJCqVbcAVyOY+Ui6qTtxfDNnw02FAiD70u7hG9655RPgMQQL4PNXyzlCh0fqjX8W o5ZJf2exGSVcbeqJLZBmPeUcsUAbKDC5TqjOOhvEjn/Jz8tH7rrEgZwHpmnnT+t9YIwWgxxexB7b vGVfEtbvmYdTBfaTE3m64Ighq9+QvlzCYEeyg9Aea0tchf5BRVwYMvVwd9iiiJ9qAnD8ByiC5FAR JYjrOdI0wIFDRtsKGj7CubcDEtfZmlxa5J9IKOTLok3tWNlUU1/aUJTx9vJ/fOyJh2OB573pWSAx DwCDag02ugoggu0uJKkigPZmi3mMU3miMHhmnY0nUMb9TNvEFffMVkcGa9HQ4a2brpY4gpF4qYya CzN2xNqO5GV3QDWkvVLHIKttAq+Qjtw0DUfIWipLREpNlfj9BZqez5qJ4I3qIn8LoyqNYTVDZYM8 hFIP3K3VBr6QM++ebIYgJFiEm0SgSRJf+VPPAuwf/krFB3XFhb6RZzM5lYJgJjuHgRlb7VwzZTBu aS3lnMOFmgD4pm3VJVKTDI1x3cnstWz0yt0ovu3vOHDCHMibmD5bz3WLt55fN5EzCMFVKiDpjzNx rfdZ4fslQ+4EdxOTWAWdJxegpA3XDgnsUfE4VZd/6Yj29j4vWwjMWyGKSuu5xV3YblPwCwqF0She 6naNH+sJ5onAjpxWMH4aT7tFcdyMMr8yVGCfCX/3GT9j22yCNTAJFY7CV/pCxO3oNj9pPBvbGkzN O7GwaFyKaSVufc3JsOl+U6R4cnQZaQ3UDYEcjHaL0GpizDjgqfksPWLeTSqT8rbPcQ/LNrHWqeOq AoOBnfB7RzGjjJQ5OoNXt9ZG05zUqR2rFYf7+vWWgevQiAmd8JEtI3gWN3cDV0 X-QQ-XMRINFO: NS+P29fieYNw95Bth2bWPxk= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Fri, 23 Feb 2024 22:11:43 +0800 X-OQ-MSGID: <20240223141143.16065-2-quinkblack@foxmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240223141143.16065-1-quinkblack@foxmail.com> References: <20240223141143.16065-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] avutil/hwcontext_opencl: Add map from Videotoolbox to OpenCL X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Zhao Zhili Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: FtXQzckNrVl3 From: Zhao Zhili For example: ./ffmpeg -hwaccel videotoolbox \ -hwaccel_output_format videotoolbox_vld \ -i foo.mp4 \ -vf hwmap=derive_device=opencl,transpose_opencl=dir=clock,hwmap,format=nv12 \ -c:v hevc_videotoolbox \ -c:a copy \ -b:v 2M -tag:v hvc1 bar.mp4 --- configure | 6 +++ libavutil/hwcontext_opencl.c | 97 ++++++++++++++++++++++++++++++++++++ 2 files changed, 103 insertions(+) diff --git a/configure b/configure index 197f762b58..efcb4c4e49 100755 --- a/configure +++ b/configure @@ -2460,6 +2460,7 @@ HAVE_LIST=" opencl_dxva2 opencl_vaapi_beignet opencl_vaapi_intel_media + opencl_videotoolbox perl pod2man texi2html @@ -7217,6 +7218,11 @@ if enabled_all opencl d3d11va ; then enable opencl_d3d11 fi +if enabled_all opencl videotoolbox ; then + check_func_headers OpenCL/cl_gl_ext.h clCreateImageFromIOSurfaceWithPropertiesAPPLE -framework VideoToolbox -framework OpenCL && + enable opencl_videotoolbox +fi + enabled vdpau && check_cpp_condition vdpau vdpau/vdpau.h "defined VDP_DECODER_PROFILE_MPEG4_PART2_ASP" diff --git a/libavutil/hwcontext_opencl.c b/libavutil/hwcontext_opencl.c index de093fffb1..efd8d0e094 100644 --- a/libavutil/hwcontext_opencl.c +++ b/libavutil/hwcontext_opencl.c @@ -72,6 +72,11 @@ #include "hwcontext_drm.h" #endif +#if HAVE_OPENCL_VIDEOTOOLBOX +#include +#include +#endif + #if HAVE_OPENCL_VAAPI_INTEL_MEDIA && CONFIG_LIBMFX extern int ff_qsv_get_surface_base_handle(mfxFrameSurface1 *surf, enum AVHWDeviceType base_dev_typ, @@ -1364,6 +1369,12 @@ static int opencl_device_derive(AVHWDeviceContext *hwdev, break; #endif +#if HAVE_OPENCL_VIDEOTOOLBOX + case AV_HWDEVICE_TYPE_VIDEOTOOLBOX: + err = opencl_device_create(hwdev, NULL, NULL, 0); + break; +#endif + default: err = AVERROR(ENOSYS); break; @@ -2819,6 +2830,84 @@ fail: #endif +#if HAVE_OPENCL_VIDEOTOOLBOX + +static void opencl_unmap_from_vt(AVHWFramesContext *hwfc, + HWMapDescriptor *hwmap) +{ + uint8_t *desc = hwmap->priv; + opencl_pool_free(hwfc, desc); +} + +static int opencl_map_from_vt(AVHWFramesContext *dst_fc, AVFrame *dst, + const AVFrame *src, int flags) +{ + CVPixelBufferRef pixbuf = (CVPixelBufferRef) src->data[3]; + IOSurfaceRef io_surface_ref = CVPixelBufferGetIOSurface(pixbuf); + cl_int err = 0; + AVOpenCLFrameDescriptor *desc = NULL; + AVOpenCLDeviceContext *dst_dev = dst_fc->device_ctx->hwctx; + + if (!io_surface_ref) { + av_log(dst_fc, AV_LOG_ERROR, "Failed to get IOSurfaceRef\n"); + return AVERROR_EXTERNAL; + } + + desc = av_mallocz(sizeof(*desc)); + if (!desc) + return AVERROR(ENOMEM); + + for (int p = 0;; p++) { + cl_image_format image_format; + cl_image_desc image_desc; + cl_iosurface_properties_APPLE props[] = { + CL_IOSURFACE_REF_APPLE, (cl_iosurface_properties_APPLE) io_surface_ref, + CL_IOSURFACE_PLANE_APPLE, p, + 0 + }; + + err = opencl_get_plane_format(dst_fc->sw_format, p, + src->width, src->height, + &image_format, &image_desc); + if (err == AVERROR(ENOENT)) + break; + if (err < 0) + goto fail; + + desc->planes[p] = clCreateImageFromIOSurfaceWithPropertiesAPPLE(dst_dev->context, + opencl_mem_flags_for_mapping(flags), + &image_format, &image_desc, + props, &err); + if (!desc->planes[p]) { + av_log(dst_fc, AV_LOG_ERROR, "Failed to create image from IOSurfaceRef\n"); + err = AVERROR(EIO); + goto fail; + } + desc->nb_planes++; + } + + for (int i = 0; i < desc->nb_planes; i++) + dst->data[i] = (uint8_t *) desc->planes[i]; + + err = ff_hwframe_map_create(dst->hw_frames_ctx, dst, src, + opencl_unmap_from_vt, desc); + if (err < 0) + goto fail; + + dst->width = src->width; + dst->height = src->height; + + return 0; + +fail: + for (int i = 0; i < desc->nb_planes; i++) + clReleaseMemObject(desc->planes[i]); + av_freep(&desc); + return err; +} + +#endif + static int opencl_map_from(AVHWFramesContext *hwfc, AVFrame *dst, const AVFrame *src, int flags) { @@ -2864,6 +2953,10 @@ static int opencl_map_to(AVHWFramesContext *hwfc, AVFrame *dst, case AV_PIX_FMT_DRM_PRIME: if (priv->drm_arm_mapping_usable) return opencl_map_from_drm_arm(hwfc, dst, src, flags); +#endif +#if HAVE_OPENCL_VIDEOTOOLBOX + case AV_PIX_FMT_VIDEOTOOLBOX: + return opencl_map_from_vt(hwfc, dst, src, flags); #endif } return AVERROR(ENOSYS); @@ -2922,6 +3015,10 @@ static int opencl_frames_derive_to(AVHWFramesContext *dst_fc, if (!priv->drm_arm_mapping_usable) return AVERROR(ENOSYS); break; +#endif +#if HAVE_OPENCL_VIDEOTOOLBOX + case AV_HWDEVICE_TYPE_VIDEOTOOLBOX: + break; #endif default: return AVERROR(ENOSYS);