From patchwork Fri Jun 16 09:29:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 42146 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c526:b0:117:ac03:c9de with SMTP id gm38csp1391942pzb; Fri, 16 Jun 2023 02:32:51 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4c2tyV8WbHEV4LkJZlqghNOjXryfYneVW95zAz0CM/KUJg9ffM85fsIFkqMkR7WXs5rH2g X-Received: by 2002:a17:907:3687:b0:94f:449e:75db with SMTP id bi7-20020a170907368700b0094f449e75dbmr1451093ejc.52.1686907971307; Fri, 16 Jun 2023 02:32:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686907971; cv=none; d=google.com; s=arc-20160816; b=t7AawQsMF6MttRBmADm4g8Lh7NBJXTpxQgBzeEZSb4YCgU8bBDQzwxKVhneBjGdCIh Zwmh1o5585rGB6SRmMYrjl/xCGZR9mRB1fLu9Xu2axrbsa8CGb7T/wpo44R8j1zHLwIt /z3VI1SNGmV3Mjqae0SWExLeNR54mcBsCeeHpyPD5oTw42785HEZHsbOJGKRZLmGU6dE SKlv5Pq4i68szfS1V96QDvYahQoNyDDVwvj3GFk62s1UgoDkigjwQ1uwFaPNXDyFm/2h X3tS+VJFyIIvgQlbb8xSZUPGtY49uuXPhvy1SiwXOO45sLL8rQ2A0CQSdFJGXbFvsemZ dMag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=QsMRlup2KDehiTS7/S4ZT4WRoMSffTW0zhBpBHTAnlk=; b=wgxkyjgA2aqMvQpRkhJtygOSfbewCzbWsFSiDka+cS3T9YimKAlJrcMAo2JAFoYQ7D jYMLuuPtLg6Q7qOllcofdJQi5Z9M/dQ+J998mLaPge6WmDvt+vpOC95Xrt9m87aMcwQo 0aeXKBbpy7CyXSaDiOYL5eyi48nSmo+U378tIVyhcmFh7Fn78LbtGpVWT/hU/kShwpv8 y6YwzFcaicKr99sXmZEAWwkOA8xBZtLsIba9wlJIieHQ7uZYWz8B3Z/hZnWeVs2NzJQc 1Wk7aR0IVkqK1hk53QS9R1Za9wfQy+UjMORTmE86exZuTXL/pBc7GQZ5E8WlrNGTfS4n uU/g== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=rRmYna41; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id n18-20020aa7d052000000b00510dfff2057si1055988edo.62.2023.06.16.02.32.50; Fri, 16 Jun 2023 02:32:51 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=rRmYna41; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0B4DA68C60D; Fri, 16 Jun 2023 12:30:42 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id ACDB668C5C1 for ; Fri, 16 Jun 2023 12:30:26 +0300 (EEST) Received: from localhost (217-74-0-168.hsi.r-kom.net [217.74.0.168]) by haasn.dev (Postfix) with ESMTPSA id 93B01401C9; Fri, 16 Jun 2023 11:30:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1686907825; bh=2tTB0IMGtozgW9u82SARVZ//nWaM1y+3xxPJ1k1hf0w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rRmYna41lghluao6risBgqsGAvQiwPp5hIOyMXdd51w51LsU4HN0r6qzTeuAChiP3 rVCvJp7R1MFIIU3ZFRUXfYfSEMqSraJCflalVz0TXcwcJRn/KrY8wHkIcvLGHwR6AO bTI1PWs0M9l69ygcN0JQoPyJc/mfdCS/ep2xrh50= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Fri, 16 Jun 2023 11:29:48 +0200 Message-ID: <20230616092959.5247-11-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230616092959.5247-1-ffmpeg@haasn.xyz> References: <20230616092959.5247-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 11/22] lavfi/vf_libplacebo: support blending multiple inputs X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: BHyZpERA9tKc From: Niklas Haas Subsequent inputs required frame blending to be enable, in order to not overwrite the existing frame contents. For output metadata, we implicitly copy the metadata of the *first* available stream (falling back to the second stream if the first has already reached EOF, and so on). This is done to resolve any conflicts between inputs with differing metadata. So when e.g. input 1 is HDR and output 2 is SDR, the output will be HDR, and vice versa. This logic could probablly be improved by dynamically determining some "superior" set of metadata, but I don't want to handle that complexity in this series. --- libavfilter/vf_libplacebo.c | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/libavfilter/vf_libplacebo.c b/libavfilter/vf_libplacebo.c index d6f19f166d..3d812569f1 100644 --- a/libavfilter/vf_libplacebo.c +++ b/libavfilter/vf_libplacebo.c @@ -784,12 +784,18 @@ static int output_frame(AVFilterContext *ctx, int64_t pts) { int err = 0, ok, changed_csp; LibplaceboContext *s = ctx->priv; - LibplaceboInput *in = &s->inputs[0]; AVFilterLink *outlink = ctx->outputs[0]; const AVPixFmtDescriptor *outdesc = av_pix_fmt_desc_get(outlink->format); - const AVFrame *ref = ref_frame(&in->mix); struct pl_frame target; + const AVFrame *ref = NULL; AVFrame *out; + + /* Use the first active input as metadata reference */ + for (int i = 0; i < s->nb_inputs; i++) { + const LibplaceboInput *in = &s->inputs[i]; + if (in->qstatus == PL_QUEUE_OK && (ref = ref_frame(&in->mix))) + break; + } if (!ref) return 0; @@ -860,8 +866,18 @@ static int output_frame(AVFilterContext *ctx, int64_t pts) goto fail; } - update_crops(ctx, in, &target, out->pts * av_q2d(outlink->time_base)); - pl_render_image_mix(in->renderer, &in->mix, &target, &s->params); + /* Draw first frame opaque, others with blending */ + s->params.skip_target_clearing = false; + s->params.blend_params = NULL; + for (int i = 0; i < s->nb_inputs; i++) { + LibplaceboInput *in = &s->inputs[i]; + if (in->qstatus != PL_QUEUE_OK) + continue; + update_crops(ctx, in, &target, out->pts * av_q2d(outlink->time_base)); + pl_render_image_mix(in->renderer, &in->mix, &target, &s->params); + s->params.skip_target_clearing = true; + s->params.blend_params = &pl_alpha_overlay; + } if (outdesc->flags & AV_PIX_FMT_FLAG_HWACCEL) { pl_unmap_avframe(s->gpu, &target);