From patchwork Thu May 11 08:39:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 41577 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp4439314pzb; Thu, 11 May 2023 01:39:20 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7P0KyNydqnKDmzIulBSg0ml/INCRdiT2XZozYziEaNcx2QduGfKqtdSv7/cQ4Pzn52ErzK X-Received: by 2002:a17:907:940c:b0:960:ddba:e5bb with SMTP id dk12-20020a170907940c00b00960ddbae5bbmr18423831ejc.43.1683794359776; Thu, 11 May 2023 01:39:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683794359; cv=none; d=google.com; s=arc-20160816; b=OonFDEIZ6lVPWloOXEi8v79lRyk+aIlv1JU6mxDltG4HyZkiCF0hhRhax6PpYQsjTI Pquqp2Wd28Fj5w/WWgzdSN/NU/7risllgyQaxEj4KdwRgJlGTsZ02aScvgXrsFwtOnO7 Mw2uFm1KBUEALm0CWAy4aqmiN1DYUkI6r/NmFmoSu95AJGtwlfmJ3qClyR5rUbyZOo4f RrOnHOKGS3LfZZ4tApPZ8mCXJS3TIcj/AEXlZneYO8N3BneNjP7vWiy5bql4+ci7UXKq K3EawoGBBH22pfGEqh+pRkcAmSQWokAvmP3Dsw47wEeK4FNrQWFSv/z7Xn5gNPm4r7Es 8YAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=qHIDkoFxV3WAgNZlL9KlTjFn28MoEVFpDAhc0uqzCMo=; b=cUIfLnJpBHg9MN7J0t+58CQ08Ffa49EvLPY3limeI4yRGaxZxDklPxXkCV945OyhAV kZrHmF/lonxUS+9His89x+5nHsnpCmcbruIEb+9mhGJSJiCrT9TBCMvfX4kGe/38IHAV NFCTq+YExdTiJZL0o/zc6tiOUtg/Z50J7GzmpT1FYg/rBKeicLXVqq+rQApUhITzcxG9 Gd+DgKoIc5NNoKUXNOWC6DafPJy08qwjNvYVPGNPFlW6IdD3QW0zcSEKkHtFKvGSk8Xu x1KPvz27bIO6xvK6oKZyhfJ+ofIBJ07FU5j23fIRxLTkEkofSg68/M6093TCeLto0d0G UMbg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=SVkfnONy; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f10-20020a170906560a00b009538b4dacc8si4642114ejq.541.2023.05.11.01.39.19; Thu, 11 May 2023 01:39:19 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=SVkfnONy; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E9188680869; Thu, 11 May 2023 11:39:15 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 80EFC68BCFB for ; Thu, 11 May 2023 11:39:09 +0300 (EEST) Received: from localhost (217-74-0-168.hsi.r-kom.net [217.74.0.168]) by haasn.dev (Postfix) with ESMTPSA id 31CA140E98; Thu, 11 May 2023 10:39:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1683794349; bh=hNzh/C+/Q4IEQnR3aoR9Ox3n/Wg/Ix/+6lc/WtFnh2s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SVkfnONygwWWQU/9aP5LfTfsxeYjtV7MJ/Q1q27AN9g5GKFp4rz3wkEybod6in75A 4b4cy0L4lCw2QOtRtdubxgSKSOTGjB/4rgKto5Mgnx8SYNBpLbzQbRtDTGXbDh51mQ N8ML5uvdKYfcO0UlC4EjPZENPzRPTQ6GDpE2Nj8M= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Thu, 11 May 2023 10:39:02 +0200 Message-Id: <20230511083902.7262-1-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230511082850.5289-1-ffmpeg@haasn.xyz> References: <20230511082850.5289-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2] lavfi/vf_libplacebo: add vulkan device import fallback X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ThFw2jw7VgWN From: Niklas Haas Recent versions of libplacebo have required Vulkan versions incompatible with lavu Vulkan hwcontexts. While this is expected to change eventually, breaking vf_libplacebo every time there is such a transition period is obviously undesired behavior, as the following sea of bug reports shows. This commit adds a fallback path for pl_vulkan_import failures which simply creates an internal device instead. Also useful when no interop with lavu vulkan hwframes is needed or desired. Fixes: https://github.com/haasn/libplacebo/issues/170 Fixes: https://github.com/mpv-player/mpv/issues/9589#issuecomment-1535432185 Fixes: https://code.videolan.org/videolan/libplacebo/-/issues/270 --- libavfilter/vf_libplacebo.c | 92 +++++++++++++++++++------------------ 1 file changed, 48 insertions(+), 44 deletions(-) diff --git a/libavfilter/vf_libplacebo.c b/libavfilter/vf_libplacebo.c index 6fe3e0ea88..74ea3cbcc5 100644 --- a/libavfilter/vf_libplacebo.c +++ b/libavfilter/vf_libplacebo.c @@ -351,58 +351,50 @@ fail: return err; } -static int init_vulkan(AVFilterContext *avctx) +static int init_vulkan(AVFilterContext *avctx, const AVVulkanDeviceContext *hwctx) { int err = 0; LibplaceboContext *s = avctx->priv; - const AVHWDeviceContext *avhwctx; - const AVVulkanDeviceContext *hwctx; uint8_t *buf = NULL; size_t buf_len; - if (!avctx->hw_device_ctx) { - av_log(s, AV_LOG_ERROR, "Missing vulkan hwdevice for vf_libplacebo.\n"); - return AVERROR(EINVAL); - } - - avhwctx = (AVHWDeviceContext *) avctx->hw_device_ctx->data; - if (avhwctx->type != AV_HWDEVICE_TYPE_VULKAN) { - av_log(s, AV_LOG_ERROR, "Expected vulkan hwdevice for vf_libplacebo, got %s.\n", - av_hwdevice_get_type_name(avhwctx->type)); - return AVERROR(EINVAL); + if (hwctx) { + /* Import libavfilter vulkan context into libplacebo */ + s->vulkan = pl_vulkan_import(s->log, pl_vulkan_import_params( + .instance = hwctx->inst, + .get_proc_addr = hwctx->get_proc_addr, + .phys_device = hwctx->phys_dev, + .device = hwctx->act_dev, + .extensions = hwctx->enabled_dev_extensions, + .num_extensions = hwctx->nb_enabled_dev_extensions, + .features = &hwctx->device_features, + .queue_graphics = { + .index = hwctx->queue_family_index, + .count = hwctx->nb_graphics_queues, + }, + .queue_compute = { + .index = hwctx->queue_family_comp_index, + .count = hwctx->nb_comp_queues, + }, + .queue_transfer = { + .index = hwctx->queue_family_tx_index, + .count = hwctx->nb_tx_queues, + }, + /* This is the highest version created by hwcontext_vulkan.c */ + .max_api_version = VK_API_VERSION_1_2, + )); } - hwctx = avhwctx->hwctx; - - /* Import libavfilter vulkan context into libplacebo */ - s->vulkan = pl_vulkan_import(s->log, pl_vulkan_import_params( - .instance = hwctx->inst, - .get_proc_addr = hwctx->get_proc_addr, - .phys_device = hwctx->phys_dev, - .device = hwctx->act_dev, - .extensions = hwctx->enabled_dev_extensions, - .num_extensions = hwctx->nb_enabled_dev_extensions, - .features = &hwctx->device_features, - .queue_graphics = { - .index = hwctx->queue_family_index, - .count = hwctx->nb_graphics_queues, - }, - .queue_compute = { - .index = hwctx->queue_family_comp_index, - .count = hwctx->nb_comp_queues, - }, - .queue_transfer = { - .index = hwctx->queue_family_tx_index, - .count = hwctx->nb_tx_queues, - }, - /* This is the highest version created by hwcontext_vulkan.c */ - .max_api_version = VK_API_VERSION_1_2, - )); - if (!s->vulkan) { - av_log(s, AV_LOG_ERROR, "Failed importing vulkan device to libplacebo!\n"); - err = AVERROR_EXTERNAL; - goto fail; + s->vulkan = pl_vulkan_create(s->log, pl_vulkan_params( + .get_proc_addr = hwctx ? hwctx->get_proc_addr : NULL, + .queue_count = 0, /* enable all queues for parallelization */ + )); + if (!s->vulkan) { + av_log(s, AV_LOG_ERROR, "Failed creating fallback vulkan device!\n"); + err = AVERROR_EXTERNAL; + goto fail; + } } /* Create the renderer */ @@ -695,10 +687,17 @@ static int libplacebo_query_format(AVFilterContext *ctx) { int err; LibplaceboContext *s = ctx->priv; + const AVVulkanDeviceContext *vkhwctx = NULL; const AVPixFmtDescriptor *desc = NULL; AVFilterFormats *infmts = NULL, *outfmts = NULL; - RET(init_vulkan(ctx)); + if (ctx->hw_device_ctx) { + const AVHWDeviceContext *avhwctx = (void *) ctx->hw_device_ctx->data; + if (avhwctx->type == AV_HWDEVICE_TYPE_VULKAN) + vkhwctx = avhwctx->hwctx; + } + + RET(init_vulkan(ctx, vkhwctx)); while ((desc = av_pix_fmt_desc_next(desc))) { enum AVPixelFormat pixfmt = av_pix_fmt_desc_get_id(desc); @@ -710,6 +709,11 @@ static int libplacebo_query_format(AVFilterContext *ctx) continue; #endif + if (pixfmt == AV_PIX_FMT_VULKAN) { + if (!vkhwctx || vkhwctx->act_dev != s->vulkan->device) + continue; + } + if (!pl_test_pixfmt(s->gpu, pixfmt)) continue;