From patchwork Wed May 10 13:55:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 41569 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dca6:b0:f3:34fa:f187 with SMTP id ky38csp3867779pzb; Wed, 10 May 2023 06:56:15 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4V87RtFWN3I05zOKuabEwwQlsBjgln7xKc8S9s+gR3++e37q/ssIPGt9XB+CsyPL3NrHvv X-Received: by 2002:a17:907:94c2:b0:966:3114:c790 with SMTP id dn2-20020a17090794c200b009663114c790mr13918533ejc.37.1683726975097; Wed, 10 May 2023 06:56:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683726975; cv=none; d=google.com; s=arc-20160816; b=GZNwL7HVmSuON8skCSPYd0PVU5Wk5JsLXDz/Fv02u0fdXsdFMD8ggeXtjYVMPdn68o MIsQZDR3iuX8CGmXROVE/Uah+FdVEFddbYvVEtac2vQo8alGa0xVeGlQxczXVt8rlkv2 ejWfcnNbljd1FM3cY2qubZ3OyKe+j8nunA3zpE0TuY0+PQG1tbRGizCy9MAPcFCDdXwM SesTiFTLAN7Kn4KeBHkXZcrPdH35sVJhkchDaqRFt0cMMzmHxdlglFsUV7aWNf3tVOft tVcq0pGm7rcB7LVgwIaVhsmKAH+UOE3NqgD1mRJz1/JpKBeGIXaSgSiuwU+wHg8aTcT/ mqsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=2ahvfivKRDm/MlbcTI2D0mTCITAtudcAq5kLqdvY5mI=; b=O46AXNQD2jkjiTYdDfRHL0e1ytYppGn0UKg4qboQ9QzjppkCOHbv0MwGbJrgNnXMhy iaBqiDd5g7G6oOq785psQ6gxINgtK8RuUFrwNdAy1hz7g/FLXYuBzVUy3GrQ65kgJ46U VseijdahZSGT3BRv0WvAiSj0HiHmxNDCqjPSMFUDHEHqZl0TVOCSP+QCsGo0Ldb/EaOT O9G8ZkXxxhQYp3zt4gYhqZ/1E78iQc2r9zaVk5LbXJpaJBqYC2d1llIv/04GXI6WK4Ie VrstWNbQh1HL3BAEVbeBUf2mamRKT1NoHzgAhvG2TdY2HiTOJc9lF5kN9lh8ivdFrdpk LC7w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=u66Qyz0a; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id qf30-20020a1709077f1e00b0094f76af1aa7si575265ejc.655.2023.05.10.06.56.14; Wed, 10 May 2023 06:56:15 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=u66Qyz0a; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A3B9568C189; Wed, 10 May 2023 16:55:28 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0983668B076 for ; Wed, 10 May 2023 16:55:18 +0300 (EEST) Received: from localhost (217-74-0-168.hsi.r-kom.net [217.74.0.168]) by haasn.dev (Postfix) with ESMTPSA id B8BE949F37; Wed, 10 May 2023 15:55:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1683726917; bh=9zLvwUWkkeB4P8bE0QMkD5XsMTYys7oetjCpNvwdkbA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=u66Qyz0aWTTHSowgEvVBTPTUSoj4b6EL8bmTPe488yM/fWhTpoKjXUj8F5YjZTtNY uowsY+L5xCQ1wTAYyQL7tJRy+s2WVJ/9Qf8KI5DQlV7DeA5ITFrPt0yvXZU5UZnxh/ mOSBn/0ixgPBDykKjysEHbtBHD136SiknECqLHFI= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Wed, 10 May 2023 15:55:08 +0200 Message-Id: <20230510135509.4074-5-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230510135509.4074-1-ffmpeg@haasn.xyz> References: <20230510135509.4074-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 5/6] lavfi/vf_libplacebo: allow fps conversion X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: T4tgxlSptvfA From: Niklas Haas This exposes libplacebo's frame mixing functionality to vf_libplacebo, by allowing users to specify a desired target fps to output at. Incoming frames will be smoothly resampled (in a manner determined by the `frame_mixer` option, to be added in the next commit). To generate a consistently timed output stream, we directly use the desired framerate as the timebase, and simply output frames in sequential order (tracked by the number of frames output so far). --- doc/filters.texi | 7 +++++++ libavfilter/vf_libplacebo.c | 38 ++++++++++++++++++++++++++++++------- 2 files changed, 38 insertions(+), 7 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 206fb23785..3e17ba1654 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -16034,6 +16034,13 @@ and @code{(oh-ph)/2}. Set the output placement width/height expressions, default values are @code{ow} and @code{oh}. +@item fps +Set the output frame rate. This can be rational, e.g. @code{60000/1001}. If +set to the special string @code{none} (the default), input timestamps will +instead be passed through to the output unmodified. Otherwise, the input video +frames will be interpolated as necessary to rescale the video to the specified +target framerate, in a manner as determined by the @option{frame_mixer} option. + @item format Set the output format override. If unset (the default), frames will be output in the same format as the respective input frames. Otherwise, format conversion diff --git a/libavfilter/vf_libplacebo.c b/libavfilter/vf_libplacebo.c index d8128351c8..2bec942358 100644 --- a/libavfilter/vf_libplacebo.c +++ b/libavfilter/vf_libplacebo.c @@ -139,6 +139,8 @@ typedef struct LibplaceboContext { double var_values[VAR_VARS_NB]; char *w_expr; char *h_expr; + char *fps_string; + AVRational fps; ///< parsed FPS, or 0/0 for "none" char *crop_x_expr, *crop_y_expr; char *crop_w_expr, *crop_h_expr; char *pos_x_expr, *pos_y_expr; @@ -166,6 +168,7 @@ typedef struct LibplaceboContext { float antiringing; int sigmoid; int skip_aa; + int skip_cache; float polar_cutoff; int disable_linear; int disable_builtin; @@ -400,7 +403,7 @@ static int update_settings(AVFilterContext *ctx) .num_hooks = s->num_hooks, .skip_anti_aliasing = s->skip_aa, - .skip_caching_single_frame = true, + .skip_caching_single_frame = s->skip_cache, .polar_cutoff = s->polar_cutoff, .disable_linear_scaling = s->disable_linear, .disable_builtin_scalers = s->disable_builtin, @@ -465,6 +468,8 @@ static int libplacebo_init(AVFilterContext *avctx) /* Initialize dynamic filter state */ s->out_pts = av_fifo_alloc2(1, sizeof(int64_t), AV_FIFO_FLAG_AUTO_GROW); + if (strcmp(s->fps_string, "none") != 0) + RET(av_parse_video_rate(&s->fps, s->fps_string)); /* Note: s->vulkan etc. are initialized later, when hwctx is available */ return 0; @@ -672,6 +677,8 @@ static int output_frame_mix(AVFilterContext *ctx, out->pts = pts; out->width = outlink->w; out->height = outlink->h; + if (s->fps.num) + out->duration = 1; if (s->apply_dovi && av_frame_get_side_data(ref, AV_FRAME_DATA_DOVI_METADATA)) { /* Output of dovi reshaping is always BT.2020+PQ, so infer the correct @@ -793,9 +800,11 @@ static int libplacebo_activate(AVFilterContext *ctx) .discard = discard_frame, }); - /* Internally queue an output frame for the same PTS */ - av_assert1(!av_cmp_q(link->time_base, outlink->time_base)); - av_fifo_write(s->out_pts, &in->pts, 1); + if (!s->fps.num) { + /* Internally queue an output frame for the same PTS */ + av_assert1(!av_cmp_q(link->time_base, outlink->time_base)); + av_fifo_write(s->out_pts, &in->pts, 1); + } } if (ret < 0) @@ -808,7 +817,8 @@ static int libplacebo_activate(AVFilterContext *ctx) /* Signal EOF to pl_queue, and enqueue this output frame to * make sure we see PL_QUEUE_EOF returned eventually */ pl_queue_push(s->queue, NULL); - av_fifo_write(s->out_pts, &pts, 1); + if (!s->fps.num) + av_fifo_write(s->out_pts, &pts, 1); } else { ff_outlink_set_status(outlink, status, pts); return 0; @@ -819,7 +829,9 @@ static int libplacebo_activate(AVFilterContext *ctx) struct pl_frame_mix mix; enum pl_queue_status ret; - if (av_fifo_peek(s->out_pts, &pts, 1, 0) < 0) { + if (s->fps.num) { + pts = outlink->frame_count_out; + } else if (av_fifo_peek(s->out_pts, &pts, 1, 0) < 0) { ff_inlink_request_frame(inlink); return 0; } @@ -835,7 +847,8 @@ static int libplacebo_activate(AVFilterContext *ctx) ff_inlink_request_frame(inlink); return 0; case PL_QUEUE_OK: - av_fifo_drain2(s->out_pts, 1); + if (!s->fps.num) + av_fifo_drain2(s->out_pts, 1); return output_frame_mix(ctx, &mix, pts); case PL_QUEUE_EOF: ff_outlink_set_status(outlink, AVERROR_EOF, pts); @@ -939,6 +952,7 @@ static int libplacebo_config_output(AVFilterLink *outlink) AVVulkanFramesContext *vkfc; AVRational scale_sar; + /* Frame dimensions */ RET(ff_scale_eval_dimensions(s, s->w_expr, s->h_expr, inlink, outlink, &outlink->w, &outlink->h)); @@ -962,6 +976,15 @@ static int libplacebo_config_output(AVFilterLink *outlink) outlink->sample_aspect_ratio = scale_sar; } + /* Frame rate */ + if (s->fps.num) { + outlink->frame_rate = s->fps; + outlink->time_base = av_inv_q(s->fps); + s->skip_cache = av_cmp_q(inlink->frame_rate, s->fps) > 0; + } else { + s->skip_cache = true; + } + /* Static variables */ s->var_values[VAR_IN_W] = s->var_values[VAR_IW] = inlink->w; s->var_values[VAR_IN_H] = s->var_values[VAR_IH] = inlink->h; @@ -1006,6 +1029,7 @@ fail: static const AVOption libplacebo_options[] = { { "w", "Output video frame width", OFFSET(w_expr), AV_OPT_TYPE_STRING, {.str = "iw"}, .flags = STATIC }, { "h", "Output video frame height", OFFSET(h_expr), AV_OPT_TYPE_STRING, {.str = "ih"}, .flags = STATIC }, + { "fps", "Output video frame rate", OFFSET(fps_string), AV_OPT_TYPE_STRING, {.str = "none"}, .flags = STATIC }, { "crop_x", "Input video crop x", OFFSET(crop_x_expr), AV_OPT_TYPE_STRING, {.str = "(iw-cw)/2"}, .flags = DYNAMIC }, { "crop_y", "Input video crop y", OFFSET(crop_y_expr), AV_OPT_TYPE_STRING, {.str = "(ih-ch)/2"}, .flags = DYNAMIC }, { "crop_w", "Input video crop w", OFFSET(crop_w_expr), AV_OPT_TYPE_STRING, {.str = "iw"}, .flags = DYNAMIC },