From patchwork Tue Dec 14 13:43:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 32491 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp6978047iog; Tue, 14 Dec 2021 05:43:47 -0800 (PST) X-Google-Smtp-Source: ABdhPJyuUix8+q5Xte6oBG5ts+VW63TLo9uXeWh62yrWAI7VRGPN6DZ3DJX3AkdfpHNOjNHNKETq X-Received: by 2002:a50:ff10:: with SMTP id a16mr8075360edu.275.1639489427344; Tue, 14 Dec 2021 05:43:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639489427; cv=none; d=google.com; s=arc-20160816; b=sFkx5uOVjsj+JnTqhOU8Vo0tcLUozcUifYr9E0i4acrXf17j2ozJdfI/yuS37lUDAp InW+3rQDos/St0wRnojvGl3iAaeQeXwWUlf1D2bQ7ZoxZ58qhCguHgYyApOllPydlKds bII4rfEO0jJqZk6uyR3I2rXLcsKMOH30u1ncAzW0I5xt2CTmz5UZ1hfH3BmMNOYVuwxX poMf/iVvyOxEjr6LI28Ml4WOZfyXF2Fq6iWF/GhKItmrQUcKFIwWgC872hoZ3b8AV8B8 G/s2FVbuuo0mdFNfuJMUPPIQ8IMyqx2kjWKqpnuthFb8Ce4xTcEgI2vVt+0gA1+NaaZ5 NyiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=ggNzD5ldNYGZB6ILYNxeToHutpuO+aUuQqGgsCJNpKU=; b=UZlJMiiDvL24g8pvWyB4c+GpfvwFGt41FH5jJJmcPughi3kvuyxwt6EB32CxfjNVzA +RRzvJftUejbY/1ksXtv28k5Tu22dOM+L1lj+HAVSutghzf0ByLGtyT6q+Ima9iRtUB2 pN2rAlWtYnMyVztVNvb/mFsZozQHovkh1BJkIzoPEFgnRhC7bMPY1Z1XVIJV6HnFzZs/ URX1WXCXeA9uhmN95HGUPNDBGuf1bm8M4RkJMdhPNkBnBEYZzru6NSHA/5J80+DtnuHY bSj8t1baqJcEgBTS8U98JuThZGDBT6CMuv/YvDK531iI/y1g37Wr/MZYULkqoTqvt9eg 5R3Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=ZExr65fc; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hb43si28327080ejc.159.2021.12.14.05.43.46; Tue, 14 Dec 2021 05:43:47 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=ZExr65fc; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A4E55687EBB; Tue, 14 Dec 2021 15:43:44 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 18993687EBB for ; Tue, 14 Dec 2021 15:43:38 +0200 (EET) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 4F1E3492D0; Tue, 14 Dec 2021 14:43:37 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1639489417; bh=pw73/mqCWFLN4Sxek+RiYWQNByjxeEML5RRQn0s5xUU=; h=From:To:Cc:Subject:Date:From; b=ZExr65fcVuhZ63rl5gDNKl+HxO+RkOSPdkfP0to88fcFj9RTMtS7of5A2VDwHTbeD Qe2w5wD1KgFIbgE6V6XnUDQsSwqBqLpyiMluhFM4sMCeFWhYCvcjm0jM+atNrbQr/o 7YMnUy6gLticRF/kma+kEiOq5Yy49cOfHQrvhqNg= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Tue, 14 Dec 2021 14:43:33 +0100 Message-Id: <20211214134333.130506-1-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] vf_libplacebo: switch to newer libplacebo helpers X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: P+yRiuh8uvEe From: Niklas Haas Support for mapping/unmapping hardware frames has been added into libplacebo itself, so we can scrap this code in favor of using the new functions. This has the additional benefit of being forwards-compatible as support for more complicated frame-related state management is added to libplacebo (e.g. mapping dolby vision metadata). It's worth pointing out that, technically, this would also allow `vf_libplacebo` to accept, practically unmodified, other frame types (e.g. vaapi or drm), or even software input formats. (Although we still need a vulkan *device* to be available) To keep things simple, though, retain the current restriction to vulkan frames. It's possible we could rethink this in a future commit, but for now I don't want to introduce any more potentially breaking changes. --- configure | 2 +- libavfilter/vf_libplacebo.c | 91 +++++-------------------------------- 2 files changed, 13 insertions(+), 80 deletions(-) diff --git a/configure b/configure index a7593ec2db..2abad854cf 100755 --- a/configure +++ b/configure @@ -6556,7 +6556,7 @@ enabled libopus && { require_pkg_config libopus opus opus_multistream.h opus_multistream_surround_encoder_create } } -enabled libplacebo && require_pkg_config libplacebo "libplacebo >= 4.173.0" libplacebo/vulkan.h pl_vulkan_create +enabled libplacebo && require_pkg_config libplacebo "libplacebo >= 4.184.0" libplacebo/vulkan.h pl_vulkan_create enabled libpulse && require_pkg_config libpulse libpulse pulse/pulseaudio.h pa_context_new enabled librabbitmq && require_pkg_config librabbitmq "librabbitmq >= 0.7.1" amqp.h amqp_new_connection enabled librav1e && require_pkg_config librav1e "rav1e >= 0.4.0" rav1e.h rav1e_context_new diff --git a/libavfilter/vf_libplacebo.c b/libavfilter/vf_libplacebo.c index e2749857d1..7ece9d829e 100644 --- a/libavfilter/vf_libplacebo.c +++ b/libavfilter/vf_libplacebo.c @@ -275,64 +275,18 @@ static void libplacebo_uninit(AVFilterContext *avctx) s->gpu = NULL; } -static int wrap_vkframe(pl_gpu gpu, const AVFrame *frame, int plane, pl_tex *tex) -{ - AVVkFrame *vkf = (AVVkFrame *) frame->data[0]; - const AVHWFramesContext *hwfc = (AVHWFramesContext *) frame->hw_frames_ctx->data; - const AVVulkanFramesContext *vkfc = hwfc->hwctx; - const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(hwfc->sw_format); - const VkFormat *vk_fmt = av_vkfmt_from_pixfmt(hwfc->sw_format); - const int chroma = plane == 1 || plane == 2; - - *tex = pl_vulkan_wrap(gpu, pl_vulkan_wrap_params( - .image = vkf->img[plane], - .format = vk_fmt[plane], - .width = AV_CEIL_RSHIFT(frame->width, chroma ? desc->log2_chroma_w : 0), - .height = AV_CEIL_RSHIFT(frame->height, chroma ? desc->log2_chroma_h : 0), - .usage = vkfc->usage, - )); - - if (!*tex) - return AVERROR(ENOMEM); - - pl_vulkan_release(gpu, *tex, vkf->layout[plane], (pl_vulkan_sem) { - .sem = vkf->sem[plane], - .value = vkf->sem_value[plane] - }); - return 0; -} - -static int unwrap_vkframe(pl_gpu gpu, AVFrame *frame, int plane, pl_tex *tex) -{ - AVVkFrame *vkf = (AVVkFrame *) frame->data[0]; - int ok = pl_vulkan_hold_raw(gpu, *tex, &vkf->layout[plane], - (pl_vulkan_sem) { vkf->sem[plane], vkf->sem_value[plane] + 1 }); - vkf->access[plane] = 0; - vkf->sem_value[plane] += !!ok; - return ok ? 0 : AVERROR_EXTERNAL; -} - -static void set_sample_depth(struct pl_frame *out_frame, const AVFrame *frame) -{ - const AVHWFramesContext *hwfc = (AVHWFramesContext *) frame->hw_frames_ctx->data; - pl_fmt fmt = out_frame->planes[0].texture->params.format; - struct pl_bit_encoding *bits = &out_frame->repr.bits; - bits->sample_depth = fmt->component_depth[0]; - - switch (hwfc->sw_format) { - case AV_PIX_FMT_P010: bits->bit_shift = 6; break; - default: break; - } -} - static int process_frames(AVFilterContext *avctx, AVFrame *out, AVFrame *in) { - int err = 0; + int err = 0, ok; LibplaceboContext *s = avctx->priv; struct pl_render_params params; struct pl_frame image, target; - pl_frame_from_avframe(&image, in); - pl_frame_from_avframe(&target, out); + ok = pl_map_avframe(s->gpu, &image, NULL, in); + ok &= pl_map_avframe(s->gpu, &target, NULL, out); + if (!ok) { + err = AVERROR_EXTERNAL; + goto fail; + } if (!s->apply_filmgrain) image.film_grain.type = PL_FILM_GRAIN_NONE; @@ -411,38 +365,17 @@ static int process_frames(AVFilterContext *avctx, AVFrame *out, AVFrame *in) RET(find_scaler(avctx, ¶ms.upscaler, s->upscaler)); RET(find_scaler(avctx, ¶ms.downscaler, s->downscaler)); - /* Ideally, we would persistently wrap all of these AVVkFrames into pl_tex - * objects, but for now we'll just create and destroy a wrapper per frame. - * Note that doing it this way is suboptimal, since it results in the - * creation and destruction of a VkSampler and VkFramebuffer per frame. - * - * FIXME: Can we do better? */ - for (int i = 0; i < image.num_planes; i++) - RET(wrap_vkframe(s->gpu, in, i, &image.planes[i].texture)); - for (int i = 0; i < target.num_planes; i++) - RET(wrap_vkframe(s->gpu, out, i, &target.planes[i].texture)); - - /* Since we-re mapping vkframes manually, the pl_frame helpers don't know - * about the mismatch between the sample format and the color depth. */ - set_sample_depth(&image, in); - set_sample_depth(&target, out); - pl_render_image(s->renderer, &image, &target, ¶ms); - - for (int i = 0; i < image.num_planes; i++) - RET(unwrap_vkframe(s->gpu, in, i, &image.planes[i].texture)); - for (int i = 0; i < target.num_planes; i++) - RET(unwrap_vkframe(s->gpu, out, i, &target.planes[i].texture)); + pl_unmap_avframe(s->gpu, &image); + pl_unmap_avframe(s->gpu, &target); /* Flush the command queues for performance */ pl_gpu_flush(s->gpu); + return 0; - /* fall through */ fail: - for (int i = 0; i < image.num_planes; i++) - pl_tex_destroy(s->gpu, &image.planes[i].texture); - for (int i = 0; i < target.num_planes; i++) - pl_tex_destroy(s->gpu, &target.planes[i].texture); + pl_unmap_avframe(s->gpu, &image); + pl_unmap_avframe(s->gpu, &target); return err; }