From patchwork Sun Jun 18 11:17:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 42201 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:be15:b0:121:b37c:e101 with SMTP id ge21csp57325pzb; Sun, 18 Jun 2023 04:23:03 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7SKRdppj7dwuxr3QBxbIuq6r3OQklYk1pCK97LNAL4AyLTt7jUDjZm2hELnMFIj6605EaC X-Received: by 2002:a17:907:2da1:b0:988:7428:2c1a with SMTP id gt33-20020a1709072da100b0098874282c1amr1433512ejc.7.1687087383576; Sun, 18 Jun 2023 04:23:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687087383; cv=none; d=google.com; s=arc-20160816; b=JPqu7CsaD06cFspVYOaFWudwu54klAYKrpNkNlv+KGiPp5ZnynpxwADIYzwSE8brMM XsTuyDjX9RRkmmOJiAchcaIzX+pEErBvC9ZMH7CBsHxdcoN10vf4cTFhMnbtgK4FxyAS ikQqLF05aweTHx0oS5zDX3ihMoo9iMvIaGr+1KBH9IfpPBG6lUp8tVkp3YdiNQDKqfvi +bmm11/pf9X4l/dhmDt24t7am2sKteEGmv4S8Rni/X9HrmfOMDSofI1QIxTeCoE7mA0d iOl5VdA5s8rZ2HlMqq7T5B891afdpNZUtNL4LEbMxZGkKYpL0b+Is9/84PX1etq9/kMs 7KHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=upSiwC3hS4DF3jcQZmNAezPNgy8X4eNw61lY7AzwvU4=; b=T8ny2t1bZRYH1JP5jQWROgs8Ykq8oSgFzAJQIoXwGoxOO71KkdnzIJiiYouZz41JQV PUHiVpt20cMoZr8SffKHiT5ERCS4riX9fr7ZqG2hu1sC6Gr5mK4ZsLS3hIPGwY5lef1j IlbS2ViOucf2iB31wlqCJly4ja4llraLWqFZA9Bed/n2rJx4fZtxN/lpT3Zz+BBslQln 8HhKOnXfrpC8nWqFy3BzVmetzRTnbnwPTpMVVTSsal2w8xBkrQQ6c5DjwWqGtg/Q9NQC hWNf1rlnLEtkWTA+m3pyF4C9gNfB1aDhIolwMzM9HXa+vzlQh19bWhf+xF5/waat9A8O AFUQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=RgU+wg4F; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a2-20020a170906368200b00965e4245c45si11801336ejc.256.2023.06.18.04.23.03; Sun, 18 Jun 2023 04:23:03 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=RgU+wg4F; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2C5CA68C220; Sun, 18 Jun 2023 14:20:32 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 66F4168C074 for ; Sun, 18 Jun 2023 14:20:14 +0300 (EEST) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 12C714AA92; Sun, 18 Jun 2023 13:20:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1687087211; bh=33DNQBemQg1KKt8P0pMqmJaFW8LFF5yAOgzJOlkkCGc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RgU+wg4FJ9kicJQB5PElE49FuX8DKuN50QyakL5cUwepdbzQDP9fclYEUJiQIBN0q JbdOJGsHbNHwx9x86d6ay2J3y6pL2r0wQd0SYpb9150BKNiwuQvLuof0yvSOtpHBFz ItwetUoF5iKkP/M1GC+j2c44GnNx9Dn1lIUGb63U= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Sun, 18 Jun 2023 13:17:03 +0200 Message-ID: <20230618111955.40994-13-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230618111955.40994-2-ffmpeg@haasn.xyz> References: <20230618111955.40994-2-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 11/22] lavfi/vf_libplacebo: support blending multiple inputs X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: F3qYy+g3Stky From: Niklas Haas Subsequent inputs require frame blending to be enabled, in order to not overwrite the existing frame contents. For output metadata, we implicitly copy the metadata of the *first* available stream (falling back to the second stream if the first has already reached EOF, and so on). This is done to resolve any conflicts between inputs with differing metadata. So when e.g. input 1 is HDR and output 2 is SDR, the output will be HDR, and vice versa. This logic could probablly be improved by dynamically determining some "superior" set of metadata, but I don't want to handle that complexity in this series. --- libavfilter/vf_libplacebo.c | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/libavfilter/vf_libplacebo.c b/libavfilter/vf_libplacebo.c index 5b45d4f8fb5..5ea6fcd7920 100644 --- a/libavfilter/vf_libplacebo.c +++ b/libavfilter/vf_libplacebo.c @@ -786,12 +786,18 @@ static int output_frame(AVFilterContext *ctx, int64_t pts) { int err = 0, ok, changed_csp; LibplaceboContext *s = ctx->priv; - LibplaceboInput *in = &s->inputs[0]; AVFilterLink *outlink = ctx->outputs[0]; const AVPixFmtDescriptor *outdesc = av_pix_fmt_desc_get(outlink->format); - const AVFrame *ref = ref_frame(&in->mix); struct pl_frame target; + const AVFrame *ref = NULL; AVFrame *out; + + /* Use the first active input as metadata reference */ + for (int i = 0; i < s->nb_inputs; i++) { + const LibplaceboInput *in = &s->inputs[i]; + if (in->qstatus == PL_QUEUE_OK && (ref = ref_frame(&in->mix))) + break; + } if (!ref) return 0; @@ -862,8 +868,18 @@ static int output_frame(AVFilterContext *ctx, int64_t pts) goto fail; } - update_crops(ctx, in, &target, out->pts * av_q2d(outlink->time_base)); - pl_render_image_mix(in->renderer, &in->mix, &target, &s->params); + /* Draw first frame opaque, others with blending */ + s->params.skip_target_clearing = false; + s->params.blend_params = NULL; + for (int i = 0; i < s->nb_inputs; i++) { + LibplaceboInput *in = &s->inputs[i]; + if (in->qstatus != PL_QUEUE_OK) + continue; + update_crops(ctx, in, &target, out->pts * av_q2d(outlink->time_base)); + pl_render_image_mix(in->renderer, &in->mix, &target, &s->params); + s->params.skip_target_clearing = true; + s->params.blend_params = &pl_alpha_overlay; + } if (outdesc->flags & AV_PIX_FMT_FLAG_HWACCEL) { pl_unmap_avframe(s->gpu, &target);