From patchwork Wed Feb 21 01:47:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: gnattu X-Patchwork-Id: 46404 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1b29:b0:19e:cdac:8cce with SMTP id ch41csp42358pzb; Tue, 20 Feb 2024 17:47:54 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCUaFDq3NijBsYHalp1+J1r3F4cTlDSWRorbjVCaDTwo9TVksqsZkzZXuVw62yTrFishCpPv7DQ59VPkdPzTyOapdhjSfjt4gQ8JHQ== X-Google-Smtp-Source: AGHT+IGZo25eE03UClgDgn/6XigmtvgAsfzDl9Vh1L4qlP05wAa4pvjqkWfeB6h4HCTRznb5XG8Y X-Received: by 2002:a05:6402:5179:b0:564:f9e2:df5b with SMTP id d25-20020a056402517900b00564f9e2df5bmr229534ede.38.1708480074063; Tue, 20 Feb 2024 17:47:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1708480074; cv=none; d=google.com; s=arc-20160816; b=w+xH56A8o8jdFVgBg93ObotpUuVulXlry6BOmjbRRaGlPv4Q+bIABMoaX7GSqx4jwg JbJ6AHaLgDOfHFSGVbupejtUwF3eYyBEeiXnj+63zb0ZzcQhXspheJC1p6+Gxc3Mu2vz pbUYSnE6m4zXg5fJbJL09Snofa/Nk8iHIj6E64mwxZJJfFWulVotkpP0+rykN5HE67Xc AkdZkApZG5L6uD/ZAPgpfNsOywnAgghhK0jtSvBk8LtiNhESKC8oIAIBqNqcWfDaL570 F2qdIz/JPjnekAC6eRh8ySoNGsINPSarr5/mWMAwC1AaqsZIRL13O//Tku8ruQ0+lMvT DyqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to :delivered-to; bh=cSdiUWa8whCDvjv0JLZrzND/RLs5xDNtwcIn35eQPsk=; fh=2gWgGvVN03792RqC4MXCXsB/4cbAJUKJ5Jr7thIsQ3A=; b=f49+icBqaXZnh+9D+B5MwvPfGwTHNrIJq2Gmyz2wZppzNErYfZJT49apHsmkKuETIz BgOYpTdDcvcPoPsQLBkVYf2fTk33gO0kYmR5UDKxufuAmBnT8uRs/+g0FGhfwVyohu1L ZbK3KiECiaFj+yyvinnTU76Wy/lzEpzYld74AH+yMBqVJTB+IKR6u4vD6ISq2ucjNcGN 4h0UeyzsQ5bQ1+QA9nP/dGrRuU0z/NBSElAN/BXdpafMctkdFgZbkVvSYqjTViAqok1/ YFw5R/C36BhBIg3fd4bw4DTRg5a4nJUg2JUfc6Ih8ibMXQPac5dhF2qFTgt7OZ1daRRj gwYQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id c1-20020a50d641000000b005644f49c838si3080240edj.553.2024.02.20.17.47.53; Tue, 20 Feb 2024 17:47:54 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BFA0D68D0F6; Wed, 21 Feb 2024 03:47:50 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mr85p00im-zteg06021901.me.com (mr85p00im-zteg06021901.me.com [17.58.23.194]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9F1B368D0B5 for ; Wed, 21 Feb 2024 03:47:43 +0200 (EET) Received: from Yakumo-Yukari.lan.lan (mr38p00im-dlb-asmtp-mailmevip.me.com [17.57.152.18]) by mr85p00im-zteg06021901.me.com (Postfix) with ESMTPSA id 29AC5740091; Wed, 21 Feb 2024 01:47:40 +0000 (UTC) To: ffmpeg-devel@ffmpeg.org Date: Wed, 21 Feb 2024 09:47:35 +0800 Message-Id: <20240221014735.4873-1-gnattuoc@me.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) MIME-Version: 1.0 X-Proofpoint-GUID: 4VonAa01NJZiSCRT394Uo7eA4KDB8Tct X-Proofpoint-ORIG-GUID: 4VonAa01NJZiSCRT394Uo7eA4KDB8Tct X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-02-20_06,2024-02-20_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 phishscore=0 mlxlogscore=999 malwarescore=0 clxscore=1015 adultscore=0 mlxscore=0 suspectscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2308100000 definitions=main-2402210012 Subject: [FFmpeg-devel] [PATCH v2] avfilter: add vf_overlay_videotoolbox X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: gnattu via ffmpeg-devel From: gnattu Reply-To: FFmpeg development discussions and patches Cc: gnattu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: eAqGcEWlfoFZ Overlay filter for VideoToolbox hwframes. Unlike most hardware overlay filters, this filter does not require the two inputs to have the same pixel format; instead, it will perform format conversion automatically with hardware accelerated methods. Signed-off-by: Gnattu OC --- Changelog | 1 + configure | 1 + doc/filters.texi | 52 +++++++++++++++++++++++++++++++++++++++ libavfilter/Makefile | 3 +++ libavfilter/allfilters.c | 1 + libavfilter/metal/utils.h | 1 - libavfilter/metal/utils.m | 7 ++++-- 7 files changed, 63 insertions(+), 3 deletions(-) diff --git a/Changelog b/Changelog index 610ee61dd6..3ecfdab81b 100644 --- a/Changelog +++ b/Changelog @@ -27,6 +27,7 @@ version : - a C11-compliant compiler is now required; note that this requirement will be bumped to C17 in the near future, so consider updating your build environment if it lacks C17 support +- VideoToolbox overlay filter version 6.1: - libaribcaption decoder diff --git a/configure b/configure index 23066efa32..a7c349d126 100755 --- a/configure +++ b/configure @@ -3807,6 +3807,7 @@ overlay_qsv_filter_deps="libmfx" overlay_qsv_filter_select="qsvvpp" overlay_vaapi_filter_deps="vaapi VAProcPipelineCaps_blend_flags" overlay_vulkan_filter_deps="vulkan spirv_compiler" +overlay_videotoolbox_filter_deps="metal corevideo coreimage videotoolbox" owdenoise_filter_deps="gpl" pad_opencl_filter_deps="opencl" pan_filter_deps="swresample" diff --git a/doc/filters.texi b/doc/filters.texi index e0436a5755..bfb77562cb 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -19033,6 +19033,58 @@ See @ref{framesync}. This filter also supports the @ref{framesync} options. +@section overlay_videotoolbox + +Overlay one video on top of another. + +This is the VideoToolbox variant of the @ref{overlay} filter. +It takes two inputs and has one output. The first input is the "main" video on which the second input is overlaid. +It only accepts VideoToolbox frames. The underlying input pixel formats do not have to match. +Different input pixel formats and color spaces will be automatically converted using hardware accelerated methods. +The final output will have the same pixel format and color space as the "main" input. + +The filter accepts the following options: + +@table @option + +@item x +Set the x coordinate of the overlaid video on the main video. +Default value is @code{0}. + +@item y +Set the y coordinate of the overlaid video on the main video. +Default value is @code{0}. + +@item eof_action +See @ref{framesync}. + +@item shortest +See @ref{framesync}. + +@item repeatlast +See @ref{framesync}. + +@end table + +@subsection Examples + +@itemize +@item +Overlay an image LOGO at the top-left corner of the INPUT video. +The INPUT video is in nv12 format and the LOGO image is in rgba format. +@example +-hwaccel videotoolbox -i INPUT -i LOGO -codec:v:0 h264_videotoolbox -filter_complex "[0:v]format=nv12,hwupload[a], [1:v]format=rgba,hwupload[b], [a][b]overlay_videotoolbox" OUTPUT +@end example +@item +Overlay an SDR video OVERLAY at the top-left corner of the HDR video MAIN. +The INPUT video is in p010 format and the LOGO image is in nv12 format. +The OUTPUT video will also be an HDR video with OVERLAY mapped to HDR. +@example +-hwaccel videotoolbox -i MAIN -i OVERLAY -codec:v:0 hevc_videotoolbox -tag:v hvc1 -filter_complex "[0:v]format=p010,hwupload[a], [1:v]format=nv12,hwupload[b], [a][b]overlay_videotoolbox" OUTPUT +@end example + +@end itemize + @section owdenoise Apply Overcomplete Wavelet denoiser. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index f6c1d641d6..ea1389ab57 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -413,6 +413,9 @@ OBJS-$(CONFIG_OVERLAY_OPENCL_FILTER) += vf_overlay_opencl.o opencl.o \ OBJS-$(CONFIG_OVERLAY_QSV_FILTER) += vf_overlay_qsv.o framesync.o OBJS-$(CONFIG_OVERLAY_VAAPI_FILTER) += vf_overlay_vaapi.o framesync.o vaapi_vpp.o OBJS-$(CONFIG_OVERLAY_VULKAN_FILTER) += vf_overlay_vulkan.o vulkan.o vulkan_filter.o +OBJS-$(CONFIG_OVERLAY_VIDEOTOOLBOX_FILTER) += vf_overlay_videotoolbox.o framesync.o \ + metal/vf_overlay_videotoolbox.metallib.o \ + metal/utils.o OBJS-$(CONFIG_OWDENOISE_FILTER) += vf_owdenoise.o OBJS-$(CONFIG_PAD_FILTER) += vf_pad.o OBJS-$(CONFIG_PAD_OPENCL_FILTER) += vf_pad_opencl.o opencl.o opencl/pad.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 149bf50997..ec9d975ecb 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -389,6 +389,7 @@ extern const AVFilter ff_vf_overlay_qsv; extern const AVFilter ff_vf_overlay_vaapi; extern const AVFilter ff_vf_overlay_vulkan; extern const AVFilter ff_vf_overlay_cuda; +extern const AVFilter ff_vf_overlay_videotoolbox; extern const AVFilter ff_vf_owdenoise; extern const AVFilter ff_vf_pad; extern const AVFilter ff_vf_pad_opencl; diff --git a/libavfilter/metal/utils.h b/libavfilter/metal/utils.h index 7350d42a35..d79c79751c 100644 --- a/libavfilter/metal/utils.h +++ b/libavfilter/metal/utils.h @@ -55,5 +55,4 @@ CVMetalTextureRef ff_metal_texture_from_pixbuf(void *avclass, int plane, MTLPixelFormat format) API_AVAILABLE(macos(10.11), ios(8.0)); - #endif /* AVFILTER_METAL_UTILS_H */ diff --git a/libavfilter/metal/utils.m b/libavfilter/metal/utils.m index f365d3ceea..db5c5f6f10 100644 --- a/libavfilter/metal/utils.m +++ b/libavfilter/metal/utils.m @@ -55,6 +55,9 @@ CVMetalTextureRef ff_metal_texture_from_pixbuf(void *ctx, { CVMetalTextureRef tex = NULL; CVReturn ret; + bool is_planer = CVPixelBufferIsPlanar(pixbuf); + size_t width = is_planer ? CVPixelBufferGetWidthOfPlane(pixbuf, plane) : CVPixelBufferGetWidth(pixbuf); + size_t height = is_planer ? CVPixelBufferGetHeightOfPlane(pixbuf, plane) : CVPixelBufferGetHeight(pixbuf); ret = CVMetalTextureCacheCreateTextureFromImage( NULL, @@ -62,8 +65,8 @@ CVMetalTextureRef ff_metal_texture_from_pixbuf(void *ctx, pixbuf, NULL, format, - CVPixelBufferGetWidthOfPlane(pixbuf, plane), - CVPixelBufferGetHeightOfPlane(pixbuf, plane), + width, + height, plane, &tex );