From patchwork Mon Jun 26 20:09:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas George X-Patchwork-Id: 42304 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3805:b0:126:ac68:d900 with SMTP id p5csp1417727pzf; Mon, 26 Jun 2023 13:09:56 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4+yjoll7c+SUFfxs1AX/fw1KhrAr1DHC2p/7MZsXor/qJq4l3ZKNJ86JJbjPLuryjZgzZ+ X-Received: by 2002:ac2:4db9:0:b0:4f8:57f1:cc7b with SMTP id h25-20020ac24db9000000b004f857f1cc7bmr17207745lfe.21.1687810196544; Mon, 26 Jun 2023 13:09:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687810196; cv=none; d=google.com; s=arc-20160816; b=mVJhDduI+Evn8in4oM/7vNDb7WWx/bION7tTI+wsV4JeJ5IOHXruvQEGJ0AZpdI22N FMfKXSrOtShLaZJfz2RxNjOesr/J+7F0Jg+x69GPbB4qoRKLLX5+pxS5wZIitsGAgPdX iLkT9V02GysNZ6QNycfcQqreRZ+/JmaU8C9HNJtE3JoAlUI/nMRO9Umm18mxH7VJDN82 LeytEwKr5PyavSgrNdR7xE+wZBl4718uvab4gbdMc+jT5a4ihdBzJQlO3OxMb3ArkZ9b 35rG6ZJBW5RN80vFXwq830teprs8gKi1y9paRn76V2+aKNWGC5VSi1ppT1W2AdIcgJ2p avOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=fbkQNmtMbWgMk6dicfbGn4CfEpxvwzg0RUbqCfYYdmw=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=VnsfirRJKp0SuP/A7CHwbv3GnzD51c8e/9ZuWcrpafGoSqP9yg+ZAk0DHGijT28b/I Yk5h1fYWkWY7gZlt3oMyKHwpLvOzBzYpbpiVdj9et9eBAWp8Ez9TtkgDSuiR10cl/nb/ nbJX0mFNT/GMaB9KWu0ItryodGjkSKAOKi97mwFnyjhIJ942PtsISfUI5vH97AcxnVDZ zruWbrGyyCZKxuUSvfT7H92Iu8cs3pvo5OYUTgQ0uzgz3chd5fG7vu96nm0RVl2lnE8Z T8jw98J9lCnhYgNTKA7s5DohKZ7SvOLR4ZwYdhpqwDnVVEr8Z8LrIgbCgiveBDvsBJM1 8KxA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id p13-20020a056402044d00b0051a5938cbf6si3004053edw.417.2023.06.26.13.09.56; Mon, 26 Jun 2023 13:09:56 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A8D6C68C2FB; Mon, 26 Jun 2023 23:09:42 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from nef.ens.fr (unknown [129.199.96.40]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B530768C2EE for ; Mon, 26 Jun 2023 23:09:35 +0300 (EEST) X-ENS-nef-client: 129.199.129.80 ( name = phare.normalesup.org ) Received: from phare.normalesup.org (phare.normalesup.org [129.199.129.80]) by nef.ens.fr (8.14.4/1.01.28121999) with ESMTP id 35QK9YNr007494 for ; Mon, 26 Jun 2023 22:09:34 +0200 Received: by phare.normalesup.org (Postfix, from userid 1001) id 47054E6C30; Mon, 26 Jun 2023 22:09:34 +0200 (CEST) From: Nicolas George To: ffmpeg-devel@ffmpeg.org Date: Mon, 26 Jun 2023 22:09:31 +0200 Message-Id: <20230626200932.1329118-1-george@nsup.org> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (nef.ens.fr [129.199.96.32]); Mon, 26 Jun 2023 22:09:34 +0200 (CEST) Subject: [FFmpeg-devel] [WIP] [PATCH 1/2] lavfi/framesync: support filters with multiple outputs X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: vFkngFrp8+Tk The filters will have to provide the logic to set the status and check for a wanted frame. Signed-off-by: Nicolas George --- libavfilter/framesync.c | 46 ++++++++++++++++++++++++++++++----------- libavfilter/framesync.h | 28 +++++++++++++++++++++++++ 2 files changed, 62 insertions(+), 12 deletions(-) Untested yet. diff --git a/libavfilter/framesync.c b/libavfilter/framesync.c index c748262ba6..0923e8c22b 100644 --- a/libavfilter/framesync.c +++ b/libavfilter/framesync.c @@ -75,6 +75,10 @@ enum { static int consume_from_fifos(FFFrameSync *fs); +static void default_on_eof(FFFrameSync *fs); + +static int default_on_miss(FFFrameSync *fs); + void ff_framesync_preinit(FFFrameSync *fs) { if (fs->class) @@ -83,28 +87,36 @@ void ff_framesync_preinit(FFFrameSync *fs) av_opt_set_defaults(fs); } -int ff_framesync_init(FFFrameSync *fs, AVFilterContext *parent, unsigned nb_in) +int ff_framesync_postinit(FFFrameSync *fs) { - /* For filters with several outputs, we will not be able to assume which - output is relevant for ff_outlink_frame_wanted() and - ff_outlink_set_status(). To be designed when needed. */ - av_assert0(parent->nb_outputs == 1); - - ff_framesync_preinit(fs); - fs->parent = parent; - fs->nb_in = nb_in; + if (fs->parent->nb_outputs == 1) { + if (!fs->on_eof) + fs->on_eof = default_on_eof; + if (!fs->on_miss) + fs->on_miss = default_on_miss; + } + av_assert0(fs->on_eof); + av_assert0(fs->on_miss); - fs->in = av_calloc(nb_in, sizeof(*fs->in)); + fs->in = av_calloc(fs->nb_in, sizeof(*fs->in)); if (!fs->in) return AVERROR(ENOMEM); return 0; } +int ff_framesync_init(FFFrameSync *fs, AVFilterContext *parent, unsigned nb_in) +{ + ff_framesync_preinit(fs); + fs->parent = parent; + fs->nb_in = nb_in; + return ff_framesync_postinit(fs); +} + static void framesync_eof(FFFrameSync *fs) { fs->eof = 1; fs->frame_ready = 0; - ff_outlink_set_status(fs->parent->outputs[0], AVERROR_EOF, AV_NOPTS_VALUE); + fs->on_eof(fs); } static void framesync_sync_level_update(FFFrameSync *fs) @@ -342,7 +354,7 @@ static int consume_from_fifos(FFFrameSync *fs) } } if (nb_miss) { - if (nb_miss == nb_active && !ff_outlink_frame_wanted(ctx->outputs[0])) + if (nb_miss == nb_active && !fs->on_miss(fs)) return FFERROR_NOT_READY; for (i = 0; i < fs->nb_in; i++) if (!fs->in[i].have_next && fs->in[i].state != STATE_EOF) @@ -422,3 +434,13 @@ int ff_framesync_dualinput_get_writable(FFFrameSync *fs, AVFrame **f0, AVFrame * } return 0; } + +static void default_on_eof(FFFrameSync *fs) +{ + ff_outlink_set_status(fs->parent->outputs[0], AVERROR_EOF, AV_NOPTS_VALUE); +} + +static int default_on_miss(FFFrameSync *fs) +{ + return ff_outlink_frame_wanted(fs->parent->outputs[0]); +} diff --git a/libavfilter/framesync.h b/libavfilter/framesync.h index 233f50a0eb..1957396c7f 100644 --- a/libavfilter/framesync.h +++ b/libavfilter/framesync.h @@ -193,6 +193,19 @@ typedef struct FFFrameSync { */ int (*on_event)(struct FFFrameSync *fs); + /** + * Callback called when EOF is reached; can be NULL + * The default is to ff_outlink_set_status() on the single output. + */ + void (*on_eof)(struct FFFrameSync *fs); + + /** + * Callback called when no input can be produced to decide if input must + * be requested + * The default is ff_outlink_frame_wanted() on the singe output. + */ + int (*on_miss)(struct FFFrameSync *fs); + /** * Opaque pointer, not used by the API */ @@ -240,6 +253,21 @@ typedef struct FFFrameSync { */ void ff_framesync_preinit(FFFrameSync *fs); +/** + * Finish the initialization of a frame sync structure + * + * Use it in combination with ff_framesync_preinit() and set fields and + * options in between. + * + * ff_framesync_preinit(fs); + * fs->parent = parent; + * fs->nb_in = nb_in; + * fs->field = value; + * ret = ff_framesync_postinit(fs); + * if (ret < 0) ... + */ +int ff_framesync_postinit(FFFrameSync *fs); + /** * Initialize a frame sync structure. * From patchwork Mon Jun 26 20:09:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas George X-Patchwork-Id: 42303 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3805:b0:126:ac68:d900 with SMTP id p5csp1417637pzf; Mon, 26 Jun 2023 13:09:46 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7z66IQeW9/p2eV6IshpclxoclASMvbI8hJ1ozqHPc8n2C1ZcYd5hUT4KA3w9df69dHhF1l X-Received: by 2002:a17:907:3687:b0:982:501a:62be with SMTP id bi7-20020a170907368700b00982501a62bemr23966164ejc.39.1687810186123; Mon, 26 Jun 2023 13:09:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687810186; cv=none; d=google.com; s=arc-20160816; b=V67/6BNjEOXN2H/15wsKaP/vJ6jWWBBBYnENxOsNGRfuHG1nyarN2GTWWntykJvf0c k2Bu9SXdGCl1SlM2Xud+ML1CLbZynAGxXU5oc5cZ14fST9877Fl3vxG182MNaQwXrgWc LHy0jPRYdk257u0dPkpEr5ZQaXPEOs+IjZgRHdzewnB+yJFh5iGeUfbQwvKHSmB4rqgg 4X8+5igethNYjxhlWpA3/CTerYwOfUB97Obj7Y16mMtuOWQye58s61zUOYkWdgtcSA+q aYGosuM7M1Vw6zIMwvd0rqjfUMXG753A0OXLBaz5UoLw0fthCpNSSSKZIDFSWuQ5Eczc eSiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=RqtlCNnVytJz3i9vgksoglAGkUyGlXMo7r5uwDyg4CE=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=fWTkm3u2YJoOb4baTfL0WMhBI3LDrQk3Noq9YfltduLBuXaNLxRNpDEkXX6CqBzwOB evHizcRx57Zj8HSCZFpYd0vp5NO0NJ7/JK1FKh57tY/yMp2auTdcCg//PDyldTLh5SlU mI7riT0rEEBglWafQJKHp3oWiLkCTS+udevlq/iJ5mXUkNRS25bNOk79OdEUKK8nbsC0 /71qmwtXB64LwMXq2Uy+8LTMVtyA6EcXkG6mpHfhNjoDiA1HP+dnCQCzK5/RBnkCGHmO TjNdwhjtVD8vyLINv2bm2NyNkprhQ4QUjMSjFrWhoqBJSUuaXJpufvnGk+gw8NN+/qOY /0Gg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hh14-20020a170906a94e00b00982a7fdd24csi3247671ejb.232.2023.06.26.13.09.45; Mon, 26 Jun 2023 13:09:46 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 74DE868C323; Mon, 26 Jun 2023 23:09:41 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from nef.ens.fr (unknown [129.199.96.40]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B279F68C2B9 for ; Mon, 26 Jun 2023 23:09:35 +0300 (EEST) X-ENS-nef-client: 129.199.129.80 ( name = phare.normalesup.org ) Received: from phare.normalesup.org (phare.normalesup.org [129.199.129.80]) by nef.ens.fr (8.14.4/1.01.28121999) with ESMTP id 35QK9Yi5007497 for ; Mon, 26 Jun 2023 22:09:34 +0200 Received: by phare.normalesup.org (Postfix, from userid 1001) id 8E636EB5BC; Mon, 26 Jun 2023 22:09:34 +0200 (CEST) From: Nicolas George To: ffmpeg-devel@ffmpeg.org Date: Mon, 26 Jun 2023 22:09:32 +0200 Message-Id: <20230626200932.1329118-2-george@nsup.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230626200932.1329118-1-george@nsup.org> References: <20230626200932.1329118-1-george@nsup.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (nef.ens.fr [129.199.96.32]); Mon, 26 Jun 2023 22:09:34 +0200 (CEST) Subject: [FFmpeg-devel] [WIP] [PATCH 2/2] lavfi/vf_framematch: add filter to output matching synchronized frames X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: u5SZIDH0A01E Signed-off-by: Nicolas George --- libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_framematch.c | 82 +++++++++++++++++++++++++++++++++++++ 3 files changed, 84 insertions(+) create mode 100644 libavfilter/vf_framematch.c Unfinished yet, but should be rather straightforward. I think I can squeeze some work on this the day after tomorrow. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 9b7813575a..290a473b63 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -313,6 +313,7 @@ OBJS-$(CONFIG_FIND_RECT_FILTER) += vf_find_rect.o lavfutils.o OBJS-$(CONFIG_FLOODFILL_FILTER) += vf_floodfill.o OBJS-$(CONFIG_FORMAT_FILTER) += vf_format.o OBJS-$(CONFIG_FPS_FILTER) += vf_fps.o +OBJS-$(CONFIG_FRAMEMATCH_FILTER) += vf_framematch.o OBJS-$(CONFIG_FRAMEPACK_FILTER) += vf_framepack.o OBJS-$(CONFIG_FRAMERATE_FILTER) += vf_framerate.o OBJS-$(CONFIG_FRAMESTEP_FILTER) += vf_framestep.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 9a7fadc58d..aa42b353c1 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -289,6 +289,7 @@ extern const AVFilter ff_vf_flip_vulkan; extern const AVFilter ff_vf_floodfill; extern const AVFilter ff_vf_format; extern const AVFilter ff_vf_fps; +extern const AVFilter ff_vf_framematch; extern const AVFilter ff_vf_framepack; extern const AVFilter ff_vf_framerate; extern const AVFilter ff_vf_framestep; diff --git a/libavfilter/vf_framematch.c b/libavfilter/vf_framematch.c new file mode 100644 index 0000000000..703b5ad93b --- /dev/null +++ b/libavfilter/vf_framematch.c @@ -0,0 +1,82 @@ + +#include "libavutil/avstring.h" +#include "libavutil/opt.h" + +#include "avfilter.h" +#include "framesync.h" +#include "internal.h" + +typedef struct MatchContext { + const AVClass *class; + int nb; + FFFrameSync fs; +} MatchContext; + +static av_cold int init(AVFilterContext *ctx) +{ + int i, ret; + MatchContext *s = ctx->priv; + + for (i = 0; i < s->nb; i++) { + AVFilterPad pad = { 0 }; + pad.type = AVMEDIA_TYPE_VIDEO; + pad.name = av_asprintf("input%d", i); + if (!pad.name) + return AVERROR(ENOMEM); + if ((ret = ff_append_inpad_free_name(ctx, &pad)) < 0) + return ret; + pad.name = av_asprintf("output%d", i); + if (!pad.name) + return AVERROR(ENOMEM); + if ((ret = ff_append_outpad_free_name(ctx, &pad)) < 0) + return ret; + } + return 0; +} + +static int query_formats(AVFilterContext *ctx) +{ + MatchContext *s = ctx->priv; + int i, ret; + + for (i = 0; i < s->nb; i++) { + AVFilterFormats *f = ff_all_formats(AVMEDIA_TYPE_AUDIO); + if ((ret = ff_formats_ref(f, &ctx->inputs [i]-> incfg.formats)) < 0 || + (ret = ff_formats_ref(f, &ctx->outputs[i]->outcfg.formats)) < 0) + return ret; + } + return 0; +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + MatchContext *s = ctx->priv; + //ff_framesync_uninit(&s->fs); +} + +static int activate(AVFilterContext *ctx) +{ + MatchContext *s = ctx->priv; + return ff_framesync_activate(&s->fs); +} + +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_FILTERING_PARAM +#define OFFSET(x) offsetof(MatchContext, x) +static const AVOption framematch_options[] = { + { "s", "set number of streams", OFFSET(nb), AV_OPT_TYPE_INT, {.i64=2}, 2, INT_MAX, .flags = FLAGS }, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(framematch); + +const AVFilter ff_vf_framematch = { + .name = "framematch", + .description = NULL_IF_CONFIG_SMALL("Match frames from multiple inputs"), + .priv_size = sizeof(MatchContext), + .priv_class = &framematch_class, + FILTER_QUERY_FUNC(query_formats), + .init = init, + .uninit = uninit, + .activate = activate, + .flags = AVFILTER_FLAG_DYNAMIC_INPUTS | AVFILTER_FLAG_DYNAMIC_OUTPUTS, +};