From patchwork Wed May 10 13:55:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 41568 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp3867692pzb; Wed, 10 May 2023 06:56:05 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4KSAoTNYK83Xq80dIOw+we2Sua/fueOwaCF2hwrhEJkCglWdFJIJ83aLl9QiFpOZj6eT6Z X-Received: by 2002:a05:6402:b1a:b0:506:b2c7:2552 with SMTP id bm26-20020a0564020b1a00b00506b2c72552mr14765277edb.13.1683726964828; Wed, 10 May 2023 06:56:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683726964; cv=none; d=google.com; s=arc-20160816; b=rUmRh1mHXZf/NNb5Vh/E+skly182OR1K1+mjdAzCLuDkdV4iglHnM7iaM8KaKrsKPK VePKu6MAN/Md0V7q7muilyBjHbgXdcJ06FfZJKKgPHz9mkEzfJPyEnKtmK4VW6oZu2Up 1wztuL6It0sM0sRE2aX1gefKZLcQxS1cc9WbSh2bEUnMCmZwHembQw0wtTsM12Yhhsk3 c7M8BQ+sV563a6qhpqxIOHvw/bVRyTfi0muP9MyAUDGLhFTys4ztfOhyRIzJ7bR9gP+C hcdIUEVA4bxSBe083RPFVbT0JqcttB5zAetHQvxumsrRHNZ/B27A2Mm2SY0RbyuYo2go OefQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=ONj4gi2KqIYeFRXLEWT6BmFSlvu7fsIdayCUcecY5MY=; b=nIWKTX1cXPqx8WPrzwWIIkd31GPQo9YzAkwebFaHu069zHgyQ4J0XlSQnhn4u0av+v qnX51ics+fpinhK6tigFsZG1fF4xB2qihrHMIvQmrqyW6rXA2FzZ4UPToL8Uw3XBA8rw /oFhF5DziU3DKeRkmvvxlM4anplgn6TN3Jg5121EiSijYFaCzTCLhlQOB078mM7rNcoA 8gqubsf5TrCsAcJuX7Wgr44d2Zu64almBLts0ETB9y+M5uDF+aa45U5H6GyEKKZRthHr hlp+iQybRrxIbjFHIEb4QLh1KhZ9subA0XFISs2dmvQvw+VQf+qWGyvtZ8mvQKNfEs7o WhZg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=uy2GfcOM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y10-20020aa7d50a000000b0050bc4dd2ccbsi3236589edq.168.2023.05.10.06.56.04; Wed, 10 May 2023 06:56:04 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=uy2GfcOM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 94E1768C173; Wed, 10 May 2023 16:55:27 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2227468B076 for ; Wed, 10 May 2023 16:55:17 +0300 (EEST) Received: from localhost (217-74-0-168.hsi.r-kom.net [217.74.0.168]) by haasn.dev (Postfix) with ESMTPSA id 0323F42B4A; Wed, 10 May 2023 15:55:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1683726916; bh=bUN2g0OsJBPIegBokt4wtZy4QYgvMEpGohBuUfIjU1M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uy2GfcOMFaVGJfDwypvkIFZ5vC4e4d8GbQnn0M0fTuCu5u/F1fbYqxB6lrj0GJVxh exqfsS3cYBvxTdvWF9NKxZHQ5AnB0v6j2+umWPiRzprVOXgDsUX4H6mypR1nCUhIFd 9b4ryhNOxxA2h84mKxJpCGNUZ/bKJ+U0gsvAGxoQ= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Wed, 10 May 2023 15:55:05 +0200 Message-Id: <20230510135509.4074-2-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230510135509.4074-1-ffmpeg@haasn.xyz> References: <20230510135509.4074-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/6] lavfi/vf_libplacebo: split and refactor logic X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: L1UrUfwZKXzv From: Niklas Haas This commit contains no functional change. The goal is merely to separate the highly intertwined `filter_frame` and `process_frames` functions into their separate concerns, specifically to separate frame uploading (which is now done directly in `filter_frame`) from emitting a frame (which is now done by a dedicated function `output_frame`). The overall idea here is to be able to ultimately call `output_frame` multiple times, to e.g. emit several output frames for a single input frame. --- libavfilter/vf_libplacebo.c | 172 +++++++++++++++++++----------------- 1 file changed, 93 insertions(+), 79 deletions(-) diff --git a/libavfilter/vf_libplacebo.c b/libavfilter/vf_libplacebo.c index 2ee9055c9d..e84f81c143 100644 --- a/libavfilter/vf_libplacebo.c +++ b/libavfilter/vf_libplacebo.c @@ -28,6 +28,14 @@ #include #include +/* Backwards compatibility with older libplacebo */ +#if PL_API_VER < 276 +static inline AVFrame *pl_get_mapped_avframe(const struct pl_frame *frame) +{ + return frame->user_data; +} +#endif + enum { TONE_MAP_AUTO, TONE_MAP_CLIP, @@ -564,35 +572,27 @@ fail: return err; } -static int process_frames(AVFilterContext *avctx, AVFrame *out, AVFrame *in) +static void update_crops(AVFilterContext *ctx, + struct pl_frame *image, + struct pl_frame *target, + const double target_pts) { - int err = 0, ok; - LibplaceboContext *s = avctx->priv; - const AVPixFmtDescriptor *outdesc = av_pix_fmt_desc_get(out->format); - struct pl_frame image, target; - ok = pl_map_avframe_ex(s->gpu, &image, pl_avframe_params( - .frame = in, - .tex = s->tex, - .map_dovi = s->apply_dovi, - )); - - if (outdesc->flags & AV_PIX_FMT_FLAG_HWACCEL) { - ok &= pl_map_avframe_ex(s->gpu, &target, pl_avframe_params( - .frame = out, - .map_dovi = false, - )); - } else { - ok &= pl_frame_recreate_from_avframe(s->gpu, &target, s->tex + 4, out); - } + LibplaceboContext *s = ctx->priv; + const AVFrame *in = pl_get_mapped_avframe(image); + const AVFilterLink *inlink = ctx->inputs[0]; + const double in_pts = in->pts * av_q2d(inlink->time_base); - if (!ok) { - err = AVERROR_EXTERNAL; - goto fail; - } + s->var_values[VAR_IN_T] = s->var_values[VAR_T] = in_pts; + s->var_values[VAR_OUT_T] = s->var_values[VAR_OT] = target_pts; + s->var_values[VAR_N] = inlink->frame_count_out; - if (!s->apply_filmgrain) - image.film_grain.type = PL_FILM_GRAIN_NONE; + /* Clear these explicitly to avoid leaking previous frames' state */ + s->var_values[VAR_CROP_W] = s->var_values[VAR_CW] = NAN; + s->var_values[VAR_CROP_H] = s->var_values[VAR_CH] = NAN; + s->var_values[VAR_POS_W] = s->var_values[VAR_PW] = NAN; + s->var_values[VAR_POS_H] = s->var_values[VAR_PH] = NAN; + /* Evaluate crop/pos dimensions first, and placement second */ s->var_values[VAR_CROP_W] = s->var_values[VAR_CW] = av_expr_eval(s->crop_w_pexpr, s->var_values, NULL); s->var_values[VAR_CROP_H] = s->var_values[VAR_CH] = @@ -602,72 +602,39 @@ static int process_frames(AVFilterContext *avctx, AVFrame *out, AVFrame *in) s->var_values[VAR_POS_H] = s->var_values[VAR_PH] = av_expr_eval(s->pos_h_pexpr, s->var_values, NULL); - image.crop.x0 = av_expr_eval(s->crop_x_pexpr, s->var_values, NULL); - image.crop.y0 = av_expr_eval(s->crop_y_pexpr, s->var_values, NULL); - image.crop.x1 = image.crop.x0 + s->var_values[VAR_CROP_W]; - image.crop.y1 = image.crop.y0 + s->var_values[VAR_CROP_H]; + image->crop.x0 = av_expr_eval(s->crop_x_pexpr, s->var_values, NULL); + image->crop.y0 = av_expr_eval(s->crop_y_pexpr, s->var_values, NULL); + image->crop.x1 = image->crop.x0 + s->var_values[VAR_CROP_W]; + image->crop.y1 = image->crop.y0 + s->var_values[VAR_CROP_H]; - target.crop.x0 = av_expr_eval(s->pos_x_pexpr, s->var_values, NULL); - target.crop.y0 = av_expr_eval(s->pos_y_pexpr, s->var_values, NULL); - target.crop.x1 = target.crop.x0 + s->var_values[VAR_POS_W]; - target.crop.y1 = target.crop.y0 + s->var_values[VAR_POS_H]; + target->crop.x0 = av_expr_eval(s->pos_x_pexpr, s->var_values, NULL); + target->crop.y0 = av_expr_eval(s->pos_y_pexpr, s->var_values, NULL); + target->crop.x1 = target->crop.x0 + s->var_values[VAR_POS_W]; + target->crop.y1 = target->crop.y0 + s->var_values[VAR_POS_H]; if (s->target_sar.num) { - float aspect = pl_rect2df_aspect(&target.crop) * av_q2d(s->target_sar); - pl_rect2df_aspect_set(&target.crop, aspect, s->pad_crop_ratio); - } - - pl_render_image(s->renderer, &image, &target, &s->params); - pl_unmap_avframe(s->gpu, &image); - - if (outdesc->flags & AV_PIX_FMT_FLAG_HWACCEL) { - pl_unmap_avframe(s->gpu, &target); - } else if (!pl_download_avframe(s->gpu, &target, out)) { - err = AVERROR_EXTERNAL; - goto fail; + float aspect = pl_rect2df_aspect(&target->crop) * av_q2d(s->target_sar); + pl_rect2df_aspect_set(&target->crop, aspect, s->pad_crop_ratio); } - - /* Flush the command queues for performance */ - pl_gpu_flush(s->gpu); - return 0; - -fail: - pl_unmap_avframe(s->gpu, &image); - pl_unmap_avframe(s->gpu, &target); - return err; } -static int filter_frame(AVFilterLink *link, AVFrame *in) +/* Construct and emit an output frame for `image` */ +static int output_frame(AVFilterContext *ctx, struct pl_frame *image) { - int err, changed_csp; - AVFilterContext *ctx = link->dst; + int err = 0, ok, changed_csp; LibplaceboContext *s = ctx->priv; AVFilterLink *outlink = ctx->outputs[0]; - + const AVPixFmtDescriptor *outdesc = av_pix_fmt_desc_get(outlink->format); AVFrame *out = ff_get_video_buffer(outlink, outlink->w, outlink->h); - if (!out) { - err = AVERROR(ENOMEM); - goto fail; - } - - pl_log_level_update(s->log, get_log_level()); + const AVFrame *in = pl_get_mapped_avframe(image); + struct pl_frame target; + if (!out) + return AVERROR(ENOMEM); RET(av_frame_copy_props(out, in)); out->width = outlink->w; out->height = outlink->h; - /* Dynamic variables */ - s->var_values[VAR_IN_T] = s->var_values[VAR_T] = - in->pts == AV_NOPTS_VALUE ? NAN : in->pts * av_q2d(link->time_base); - s->var_values[VAR_OUT_T] = s->var_values[VAR_OT] = - out->pts == AV_NOPTS_VALUE ? NAN : out->pts * av_q2d(outlink->time_base); - s->var_values[VAR_N] = link->frame_count_out; - /* Will be evaluated/set by `process_frames` */ - s->var_values[VAR_CROP_W] = s->var_values[VAR_CW] = NAN; - s->var_values[VAR_CROP_H] = s->var_values[VAR_CH] = NAN; - s->var_values[VAR_POS_W] = s->var_values[VAR_PW] = NAN; - s->var_values[VAR_POS_H] = s->var_values[VAR_PH] = NAN; - if (s->apply_dovi && av_frame_get_side_data(in, AV_FRAME_DATA_DOVI_METADATA)) { /* Output of dovi reshaping is always BT.2020+PQ, so infer the correct * output colorspace defaults */ @@ -703,18 +670,65 @@ static int filter_frame(AVFilterLink *link, AVFrame *in) if (s->apply_filmgrain) av_frame_remove_side_data(out, AV_FRAME_DATA_FILM_GRAIN_PARAMS); - RET(process_frames(ctx, out, in)); + /* Map, render and unmap output frame */ + if (outdesc->flags & AV_PIX_FMT_FLAG_HWACCEL) { + ok = pl_map_avframe_ex(s->gpu, &target, pl_avframe_params( + .frame = out, + .map_dovi = false, + )); + } else { + ok = pl_frame_recreate_from_avframe(s->gpu, &target, s->tex + 4, out); + } + if (!ok) { + err = AVERROR_EXTERNAL; + goto fail; + } - av_frame_free(&in); + update_crops(ctx, image, &target, out->pts * av_q2d(outlink->time_base)); + pl_render_image(s->renderer, image, &target, &s->params); + if (outdesc->flags & AV_PIX_FMT_FLAG_HWACCEL) { + pl_unmap_avframe(s->gpu, &target); + } else if (!pl_download_avframe(s->gpu, &target, out)) { + err = AVERROR_EXTERNAL; + goto fail; + } return ff_filter_frame(outlink, out); fail: - av_frame_free(&in); av_frame_free(&out); return err; } +static int filter_frame(AVFilterLink *link, AVFrame *in) +{ + int ret, ok; + AVFilterContext *ctx = link->dst; + LibplaceboContext *s = ctx->priv; + struct pl_frame image; + + pl_log_level_update(s->log, get_log_level()); + + /* Map input frame */ + ok = pl_map_avframe_ex(s->gpu, &image, pl_avframe_params( + .frame = in, + .tex = s->tex, + .map_dovi = s->apply_dovi, + )); + av_frame_free(&in); + + if (!s->apply_filmgrain) + image.film_grain.type = PL_FILM_GRAIN_NONE; + + if (!ok) + return AVERROR_EXTERNAL; + + ret = output_frame(ctx, &image); + + pl_unmap_avframe(s->gpu, &image); + return ret; +} + static int libplacebo_query_format(AVFilterContext *ctx) { int err;