From patchwork Fri Jan 27 13:16:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 40133 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3ca3:b0:b9:1511:ac2c with SMTP id b35csp1337921pzj; Fri, 27 Jan 2023 05:17:35 -0800 (PST) X-Google-Smtp-Source: AK7set/AR6lfjNTHgEPs3MMabiy9+gwN9bIhHgTMhceVhWF2ERLIL0YaSdJ3aOHsZyaT7VWifaXb X-Received: by 2002:a17:907:a042:b0:87b:e105:5d9a with SMTP id gz2-20020a170907a04200b0087be1055d9amr1016796ejc.67.1674825455357; Fri, 27 Jan 2023 05:17:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674825455; cv=none; d=google.com; s=arc-20160816; b=wJpfCjb8M2WbgVpNX5QASSWMAUCI3vj4CgHgisn3FgOqY1wqwHQsleGld1oITZvycf Sy53UNLON2zDjpG6ybyYFREvlvCMZd1NqnvuL5jal5J6ThvS85+QidtfnceVdmOPF8Vm lKW7saYZAU19NWEo6kwHaof/OyrgwZYndkY9IpBFh7JvsRID420fwd2Mq1yX4XDQIneQ rXkyTq0GlXXj7+VvsFdqJt9KxuF5D3aLfDi9Tw0QV7uZblAWvYoJJeMPLwcbzT09TMuM wU8+8/1MJ3GjpQim3IOZCQwo2TxKTw1OIecaIeCPFkbYHZ5es66vfr8ZY3+SPEA8sgTi 5Y6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=rFH9UL2w03D76CUMEoIrppM1I18dmgDZX9eSI2VYe1Y=; b=PIC/LNlhOimHYFfr7ejkgJUMMMpeu34VTddHV5dcGXPaqjE8e+o+tzDZ691wYJ57Tk 88HK2RLgo6bu/rpMBNAc7N6aBXZ/gSqZIGXR0EdJWaehs0KvmvdF152gIGQuLu6uVlfO L1V+G4VahZwY6tPxV3W8HX+tzL5tOmr7qt+48769A/2reA5FP+Nnrt3PegSuEN2D6rRk x6i+S4Gtkg8I/WVcr9+Owdd/zh8GGn5FI8uIjF5uys1tzshV1aU3xy9ExNmxoOR22vpD nt3E8lDwSUAHji67Y6vqSFzaU++XY1mDnppU0dfWJvH1uz8ufS7cLrzvQKw7jTwEf6sK b1zQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 14-20020a17090600ce00b008731f7e5981si5120293eji.866.2023.01.27.05.17.34; Fri, 27 Jan 2023 05:17:35 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D776568BDD2; Fri, 27 Jan 2023 15:17:30 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 62C9268BD65 for ; Fri, 27 Jan 2023 15:17:24 +0200 (EET) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 01D8D240178 for ; Fri, 27 Jan 2023 14:17:24 +0100 (CET) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id MkYSxsLceeAI for ; Fri, 27 Jan 2023 14:17:23 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id CA2022404F5 for ; Fri, 27 Jan 2023 14:17:22 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id F08993A02AF for ; Fri, 27 Jan 2023 14:17:14 +0100 (CET) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Fri, 27 Jan 2023 14:16:36 +0100 Message-Id: <20230127131639.4928-1-anton@khirnov.net> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/4] lavfi/framesync: use a local variable to shorten code X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: UWkqqmLxCKjN --- libavfilter/framesync.c | 33 ++++++++++++++++----------------- 1 file changed, 16 insertions(+), 17 deletions(-) diff --git a/libavfilter/framesync.c b/libavfilter/framesync.c index ee91e4cf68..153db4fa21 100644 --- a/libavfilter/framesync.c +++ b/libavfilter/framesync.c @@ -201,24 +201,23 @@ static int framesync_advance(FFFrameSync *fs) break; } for (i = 0; i < fs->nb_in; i++) { - if (fs->in[i].pts_next == pts || - (fs->in[i].ts_mode == TS_NEAREST && - fs->in[i].have_next && - fs->in[i].pts_next != INT64_MAX && fs->in[i].pts != AV_NOPTS_VALUE && - fs->in[i].pts_next - pts < pts - fs->in[i].pts) || - (fs->in[i].before == EXT_INFINITY && - fs->in[i].state == STATE_BOF)) { - av_frame_free(&fs->in[i].frame); - fs->in[i].frame = fs->in[i].frame_next; - fs->in[i].pts = fs->in[i].pts_next; - fs->in[i].frame_next = NULL; - fs->in[i].pts_next = AV_NOPTS_VALUE; - fs->in[i].have_next = 0; - fs->in[i].state = fs->in[i].frame ? STATE_RUN : STATE_EOF; - if (fs->in[i].sync == fs->sync_level && fs->in[i].frame) + FFFrameSyncIn * const in = &fs->in[i]; + + if (in->pts_next == pts || + (in->ts_mode == TS_NEAREST && in->have_next && + in->pts_next != INT64_MAX && in->pts != AV_NOPTS_VALUE && + in->pts_next - pts < pts - in->pts) || + (in->before == EXT_INFINITY && in->state == STATE_BOF)) { + av_frame_free(&in->frame); + in->frame = in->frame_next; + in->pts = in->pts_next; + in->frame_next = NULL; + in->pts_next = AV_NOPTS_VALUE; + in->have_next = 0; + in->state = in->frame ? STATE_RUN : STATE_EOF; + if (in->sync == fs->sync_level && in->frame) fs->frame_ready = 1; - if (fs->in[i].state == STATE_EOF && - fs->in[i].after == EXT_STOP) + if (in->state == STATE_EOF && in->after == EXT_STOP) framesync_eof(fs); } } From patchwork Fri Jan 27 13:16:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 40135 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3ca3:b0:b9:1511:ac2c with SMTP id b35csp1338169pzj; Fri, 27 Jan 2023 05:17:54 -0800 (PST) X-Google-Smtp-Source: AK7set81vxH/IjRiD167W13lAodHrgEgTgX8dBwhZWOHFFN1BXTJney6SJx4wqqYgfEOYrbmc/66 X-Received: by 2002:a17:906:a007:b0:878:6631:7fe with SMTP id p7-20020a170906a00700b00878663107femr5854190ejy.20.1674825474666; Fri, 27 Jan 2023 05:17:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674825474; cv=none; d=google.com; s=arc-20160816; b=g4Bywue6NRhccsmCMZPNnM1X26QoNCed5W+bZ7CS2ED7XGr4X8PVwQ1AKVNUvfjI/Z SzzlOMwST3sVGfIfSXLcwV6sbxHqjTh/znZzMi0jg4vzZF3HQQ2+F/XPkhdhnkdg/I7H pkCWuNjEIKBS2ytNs93eBshiVumAUYoabLQlOtLlTgSC3cbuOIq7Zf48lucNHei89sLH /0NeW377jwh5bbv9xdY0Z6ee5B2wHOblOmhm3nIvgcShhMn2z6aQTEnquV2ZWq52R7NG XLXHLt2XbuerTHR6d6YM+kyUfalzxwua/WKBausM9GMdnPSgqiuRB3q46VTu/Lsm/jFQ cLGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=wXzl5b/zuYtSp/LitGiV4WnyhZib4DxbTE1X7fdTAo8=; b=AwLTvLAc6pJgY/0Xkr9CcuJ6uVkkeJGfgLjGEcUxWarVD/g+rRj6J5ZUUBsLh+/vvt g+46GLq4LYqDsjp/GXXe56fVzoyjKh+0lzihK2E70MrKLQzVIzKViV6XioofyAGTb9He ZYhntEprwyqaMM1D2DmOq/9EsiwbwJsTKlhvWtyjd1v87225owD+HmMtzRohvicvjthE Y2RKcI5Ifz9/illT3EPUPOXL6Lh/zFBlUNoA1LmJ4libMOAUquJsa7d6g6dXvPBPxXof DVcUJpsrak0/ziguwRBrSfxYQKWoWZTzTZjTce3rrLNm5xo/TpSThm6FadCQGImjiM8N pFcg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gh19-20020a170906e09300b008776a7302e8si4875226ejb.143.2023.01.27.05.17.54; Fri, 27 Jan 2023 05:17:54 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8D2C268BDEC; Fri, 27 Jan 2023 15:17:33 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6ED0C68BD7E for ; Fri, 27 Jan 2023 15:17:24 +0200 (EET) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 339E12404F5 for ; Fri, 27 Jan 2023 14:17:24 +0100 (CET) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id ZLpmJKAGf-20 for ; Fri, 27 Jan 2023 14:17:22 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id C81552404EC for ; Fri, 27 Jan 2023 14:17:22 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id 098DA3A0344 for ; Fri, 27 Jan 2023 14:17:15 +0100 (CET) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Fri, 27 Jan 2023 14:16:37 +0100 Message-Id: <20230127131639.4928-2-anton@khirnov.net> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20230127131639.4928-1-anton@khirnov.net> References: <20230127131639.4928-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/4] lavfi/framesync: reorder functions to avoid a forward declaration X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: uHeY444NKAYL --- libavfilter/framesync.c | 144 ++++++++++++++++++++-------------------- 1 file changed, 71 insertions(+), 73 deletions(-) diff --git a/libavfilter/framesync.c b/libavfilter/framesync.c index 153db4fa21..fdcc3b57c8 100644 --- a/libavfilter/framesync.c +++ b/libavfilter/framesync.c @@ -73,8 +73,6 @@ enum { STATE_EOF, }; -static int consume_from_fifos(FFFrameSync *fs); - void ff_framesync_preinit(FFFrameSync *fs) { if (fs->class) @@ -181,6 +179,77 @@ int ff_framesync_configure(FFFrameSync *fs) return 0; } +static void framesync_inject_frame(FFFrameSync *fs, unsigned in, AVFrame *frame) +{ + int64_t pts; + + av_assert0(!fs->in[in].have_next); + av_assert0(frame); + pts = av_rescale_q(frame->pts, fs->in[in].time_base, fs->time_base); + frame->pts = pts; + fs->in[in].frame_next = frame; + fs->in[in].pts_next = pts; + fs->in[in].have_next = 1; +} + +static int64_t framesync_pts_extrapolate(FFFrameSync *fs, unsigned in, + int64_t pts) +{ + /* Possible enhancement: use the link's frame rate */ + return pts + 1; +} + +static void framesync_inject_status(FFFrameSync *fs, unsigned in, int status, int64_t pts) +{ + av_assert0(!fs->in[in].have_next); + pts = fs->in[in].state != STATE_RUN || fs->in[in].after == EXT_INFINITY + ? INT64_MAX : framesync_pts_extrapolate(fs, in, fs->in[in].pts); + fs->in[in].sync = 0; + framesync_sync_level_update(fs); + fs->in[in].frame_next = NULL; + fs->in[in].pts_next = pts; + fs->in[in].have_next = 1; +} + +static int consume_from_fifos(FFFrameSync *fs) +{ + AVFilterContext *ctx = fs->parent; + AVFrame *frame = NULL; + int64_t pts; + unsigned i, nb_active, nb_miss; + int ret, status; + + nb_active = nb_miss = 0; + for (i = 0; i < fs->nb_in; i++) { + if (fs->in[i].have_next || fs->in[i].state == STATE_EOF) + continue; + nb_active++; + ret = ff_inlink_consume_frame(ctx->inputs[i], &frame); + if (ret < 0) + return ret; + if (ret) { + av_assert0(frame); + framesync_inject_frame(fs, i, frame); + } else { + ret = ff_inlink_acknowledge_status(ctx->inputs[i], &status, &pts); + if (ret > 0) { + framesync_inject_status(fs, i, status, pts); + } else if (!ret) { + nb_miss++; + } + } + } + if (nb_miss) { + if (nb_miss == nb_active && !ff_outlink_frame_wanted(ctx->outputs[0])) + return FFERROR_NOT_READY; + for (i = 0; i < fs->nb_in; i++) + if (!fs->in[i].have_next && fs->in[i].state != STATE_EOF) + ff_inlink_request_frame(ctx->inputs[i]); + return 0; + } + return 1; +} + static int framesync_advance(FFFrameSync *fs) { unsigned i; @@ -231,38 +300,6 @@ static int framesync_advance(FFFrameSync *fs) return 0; } -static int64_t framesync_pts_extrapolate(FFFrameSync *fs, unsigned in, - int64_t pts) -{ - /* Possible enhancement: use the link's frame rate */ - return pts + 1; -} - -static void framesync_inject_frame(FFFrameSync *fs, unsigned in, AVFrame *frame) -{ - int64_t pts; - - av_assert0(!fs->in[in].have_next); - av_assert0(frame); - pts = av_rescale_q(frame->pts, fs->in[in].time_base, fs->time_base); - frame->pts = pts; - fs->in[in].frame_next = frame; - fs->in[in].pts_next = pts; - fs->in[in].have_next = 1; -} - -static void framesync_inject_status(FFFrameSync *fs, unsigned in, int status, int64_t pts) -{ - av_assert0(!fs->in[in].have_next); - pts = fs->in[in].state != STATE_RUN || fs->in[in].after == EXT_INFINITY - ? INT64_MAX : framesync_pts_extrapolate(fs, in, fs->in[in].pts); - fs->in[in].sync = 0; - framesync_sync_level_update(fs); - fs->in[in].frame_next = NULL; - fs->in[in].pts_next = pts; - fs->in[in].have_next = 1; -} - int ff_framesync_get_frame(FFFrameSync *fs, unsigned in, AVFrame **rframe, unsigned get) { @@ -312,45 +349,6 @@ void ff_framesync_uninit(FFFrameSync *fs) av_freep(&fs->in); } -static int consume_from_fifos(FFFrameSync *fs) -{ - AVFilterContext *ctx = fs->parent; - AVFrame *frame = NULL; - int64_t pts; - unsigned i, nb_active, nb_miss; - int ret, status; - - nb_active = nb_miss = 0; - for (i = 0; i < fs->nb_in; i++) { - if (fs->in[i].have_next || fs->in[i].state == STATE_EOF) - continue; - nb_active++; - ret = ff_inlink_consume_frame(ctx->inputs[i], &frame); - if (ret < 0) - return ret; - if (ret) { - av_assert0(frame); - framesync_inject_frame(fs, i, frame); - } else { - ret = ff_inlink_acknowledge_status(ctx->inputs[i], &status, &pts); - if (ret > 0) { - framesync_inject_status(fs, i, status, pts); - } else if (!ret) { - nb_miss++; - } - } - } - if (nb_miss) { - if (nb_miss == nb_active && !ff_outlink_frame_wanted(ctx->outputs[0])) - return FFERROR_NOT_READY; - for (i = 0; i < fs->nb_in; i++) - if (!fs->in[i].have_next && fs->in[i].state != STATE_EOF) - ff_inlink_request_frame(ctx->inputs[i]); - return 0; - } - return 1; -} - int ff_framesync_activate(FFFrameSync *fs) { int ret; From patchwork Fri Jan 27 13:16:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 40136 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3ca3:b0:b9:1511:ac2c with SMTP id b35csp1338267pzj; Fri, 27 Jan 2023 05:18:04 -0800 (PST) X-Google-Smtp-Source: AK7set9C7Gn/WvNdvkqPb2ZJ4PoZpSgJU5rZBMqXZPTdec4R4/PcoYcVXVu6ht9koJa57lpNxb2l X-Received: by 2002:a17:906:2dd6:b0:878:714b:5e16 with SMTP id h22-20020a1709062dd600b00878714b5e16mr5769930eji.28.1674825484378; Fri, 27 Jan 2023 05:18:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674825484; cv=none; d=google.com; s=arc-20160816; b=sEXR58SzGzut9ArNDwi/GbJPRh3flKs1qFhj0CdlNmp4w8eu3LFVoHxtTbs7ufhvdR KzmUjNAjmHlbJ1RBGye8+gJGqfc9i3zE76YJoduoWwLAqRDMWbtjFC3sZgod6Bn053NP T8xG71+JAqTxIuDdpCPhon5FuJWkT/Szp1ExpB5ePHmdlVYo2QMOg+5D/u66LNO8eTFy SLGM4RAStX/NXxsWnwdNg7g1kIN8cibiNnGB0FHTa4gJEIERsTI6ppVetlZtLAF+FEk/ YPGOMGsK5K3j3hX2geHT1wYHdmo8s07GNqrREeYgCbgRgOnyr81LjawBI7EPFRrLWNcJ Tahg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=i8eTAxblw3VeNR3qthk2UjWjU2B9YMRrWlRRDo6hUlc=; b=FTnRPvOrgKa6oAcdOtzKxQfGVw2Vrmx1t5Dq8rBCYw4dXL7I+YltvsDOlgHKwS8kxo 6n8J5Fpf424xyLvQvX+AwcEMTreViXCm8x2MTX64oWx00KCIBLidc9TVmSrWA10zEjij IUuPXhIzhsrcLs6mo6mg2OgSmoaXCmqBAxogf0w4nDLdtohET3TjcucmhO5oY7JqkgOe iCUrK4E/XJ28aGK6n0jnHK1Q8C9QMAiDaRf7KPOZqHkRA4CDDfdInGv8594dLrqu+jS6 La8pWirIJofIjHg556ZEaV5oYumRiZ2Vh4keShhu5m97Pl3XGTid/PPCovoxeoIT7ZLw 803A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id wj4-20020a170907050400b008778ae57d38si4381375ejb.485.2023.01.27.05.18.03; Fri, 27 Jan 2023 05:18:04 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 98A8968BDF8; Fri, 27 Jan 2023 15:17:34 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0AF0D68BC43 for ; Fri, 27 Jan 2023 15:17:25 +0200 (EET) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id C3E8E2404EC for ; Fri, 27 Jan 2023 14:17:24 +0100 (CET) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id WQdVp81Eq8n2 for ; Fri, 27 Jan 2023 14:17:24 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id CBB6A2404F8 for ; Fri, 27 Jan 2023 14:17:22 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id 14F3F3A034A for ; Fri, 27 Jan 2023 14:17:15 +0100 (CET) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Fri, 27 Jan 2023 14:16:38 +0100 Message-Id: <20230127131639.4928-3-anton@khirnov.net> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20230127131639.4928-1-anton@khirnov.net> References: <20230127131639.4928-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/4] lavfi/framesync: add syncing via external timestamp map X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: aRe9J/3WNG0u Useful when there is some external process that determines canonical frame synchronization. E.g. the framerate conversion code in ffmpeg CLI. --- doc/filters.texi | 6 ++ libavfilter/framesync.c | 121 ++++++++++++++++++++++++++++++++++++++-- libavfilter/framesync.h | 11 ++++ 3 files changed, 132 insertions(+), 6 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index be70a2396b..2fc50f3a91 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -363,6 +363,12 @@ primary input frame. Frame from secondary input with the absolute nearest timestamp to the primary input frame. @end table + +@item ts_map +Specify an explicit timestamp map. The string should be composed of lines, one +per each output frame. The line should contain whitespace-separated times in +microseconds, one for every input. Frames with these timestamps will be matched +together to produces output events. @end table @c man end OPTIONS FOR FILTERS WITH SEVERAL INPUTS diff --git a/libavfilter/framesync.c b/libavfilter/framesync.c index fdcc3b57c8..b52cf318c0 100644 --- a/libavfilter/framesync.c +++ b/libavfilter/framesync.c @@ -49,6 +49,7 @@ static const AVOption framesync_options[] = { 0, AV_OPT_TYPE_CONST, { .i64 = TS_DEFAULT }, .flags = FLAGS, "ts_sync_mode" }, { "nearest", "Frame from secondary input with the absolute nearest timestamp to the primary input frame", 0, AV_OPT_TYPE_CONST, { .i64 = TS_NEAREST }, .flags = FLAGS, "ts_sync_mode" }, + { "ts_map", "Timestamp map", OFFSET(ts_map_str), AV_OPT_TYPE_STRING, .flags = FLAGS }, { NULL } }; static const AVClass framesync_class = { @@ -129,10 +130,78 @@ static void framesync_sync_level_update(FFFrameSync *fs) framesync_eof(fs); } +static int ts_map_parse(FFFrameSync *fs, const char *ts_map_str) +{ + while (*ts_map_str) { + int64_t *dst; + + ts_map_str += strspn(ts_map_str, " \t\r\n"); + + // skip comments + if (*ts_map_str == '#' || !*ts_map_str) + goto skip_line; + + dst = av_fast_realloc(fs->ts_map, &fs->ts_map_allocated, + sizeof(*fs->ts_map) * fs->nb_in * (fs->nb_ts_map + 1)); + if (!dst) + return AVERROR(ENOMEM); + + fs->ts_map = dst; + dst += fs->nb_in * fs->nb_ts_map; + fs->nb_ts_map++; + + // read a timestamp for each input + for (int i = 0; i < fs->nb_in; i++) { + char *p; + dst[i] = strtol(ts_map_str, &p, 0); + if (p == ts_map_str) { + av_log(fs, AV_LOG_ERROR, + "Invalid number in timestamp map on line %zu: %s\n", + fs->nb_ts_map - 1, ts_map_str); + return AVERROR_INVALIDDATA; + } + ts_map_str = p; + + if (fs->nb_ts_map > 1 && dst[i - (int)fs->nb_in] > dst[i]) { + av_log(fs, AV_LOG_ERROR, + "Timestamp map for input %d, frame %zu goes backwards\n", + i, fs->nb_ts_map - 1); + return AVERROR_INVALIDDATA; + } + + ts_map_str += strspn(p, " \t"); + } + + // skip everything after the needed timestamp +skip_line: + ts_map_str = strchr(ts_map_str, '\n'); + if (!ts_map_str) + break; + } + + return 0; +} + int ff_framesync_configure(FFFrameSync *fs) { unsigned i; + if (fs->ts_map_str) { + int ret; + + if (fs->opt_ts_sync_mode != TS_DEFAULT) { + av_log(fs, AV_LOG_ERROR, + "ts_sync_mode must be set to default when a map is used\n"); + return AVERROR(EINVAL); + } + + ret = ts_map_parse(fs, fs->ts_map_str); + if (ret < 0) { + av_log(fs, AV_LOG_ERROR, "Error reading the explicit timestamp map\n"); + return ret; + } + } + if (!fs->opt_repeatlast || fs->opt_eof_action == EOF_ACTION_PASS) { fs->opt_repeatlast = 0; fs->opt_eof_action = EOF_ACTION_PASS; @@ -250,17 +319,55 @@ static int consume_from_fifos(FFFrameSync *fs) return 1; } +static void frame_advance(FFFrameSyncIn *in) +{ + av_frame_free(&in->frame); + in->frame = in->frame_next; + in->pts = in->pts_next; + in->frame_next = NULL; + in->pts_next = AV_NOPTS_VALUE; + in->have_next = 0; +} + static int framesync_advance(FFFrameSync *fs) { unsigned i; int64_t pts; int ret; + if (fs->ts_map && fs->nb_events >= fs->nb_ts_map) { + framesync_eof(fs); + return 0; + } + while (!(fs->frame_ready || fs->eof)) { ret = consume_from_fifos(fs); if (ret <= 0) return ret; + if (fs->ts_map) { + fs->frame_ready = 1; + for (i = 0; i < fs->nb_in; i++) { + FFFrameSyncIn * const in = &fs->in[i]; + int64_t next_ts = av_rescale_q(fs->ts_map[fs->nb_events * fs->nb_in + i], + AV_TIME_BASE_Q, fs->time_base); + uint64_t delta_cur = in->frame ? FFABS(in->pts - next_ts) : UINT64_MAX; + uint64_t delta_next = in->frame_next ? FFABS(in->pts_next - next_ts) : UINT64_MAX; + + if (!in->frame || + (in->frame_next && delta_next < delta_cur)) { + frame_advance(in); + fs->frame_ready = 0; + in->state = in->frame ? STATE_RUN : STATE_EOF; + if (in->state == STATE_EOF) { + av_log(fs, AV_LOG_WARNING, + "Input stream %d ended before the timestamp map did\n", i); + framesync_eof(fs); + } + } + } + pts = fs->in[0].pts; + } else { pts = INT64_MAX; for (i = 0; i < fs->nb_in; i++) if (fs->in[i].have_next && fs->in[i].pts_next < pts) @@ -277,12 +384,7 @@ static int framesync_advance(FFFrameSync *fs) in->pts_next != INT64_MAX && in->pts != AV_NOPTS_VALUE && in->pts_next - pts < pts - in->pts) || (in->before == EXT_INFINITY && in->state == STATE_BOF)) { - av_frame_free(&in->frame); - in->frame = in->frame_next; - in->pts = in->pts_next; - in->frame_next = NULL; - in->pts_next = AV_NOPTS_VALUE; - in->have_next = 0; + frame_advance(in); in->state = in->frame ? STATE_RUN : STATE_EOF; if (in->sync == fs->sync_level && in->frame) fs->frame_ready = 1; @@ -295,6 +397,7 @@ static int framesync_advance(FFFrameSync *fs) if ((fs->in[i].state == STATE_BOF && fs->in[i].before == EXT_STOP)) fs->frame_ready = 0; + } fs->pts = pts; } return 0; @@ -347,6 +450,11 @@ void ff_framesync_uninit(FFFrameSync *fs) } av_freep(&fs->in); + + av_freep(&fs->ts_map_str); + av_freep(&fs->ts_map); + fs->nb_ts_map = 0; + fs->ts_map_allocated = 0; } int ff_framesync_activate(FFFrameSync *fs) @@ -359,6 +467,7 @@ int ff_framesync_activate(FFFrameSync *fs) if (fs->eof || !fs->frame_ready) return 0; ret = fs->on_event(fs); + fs->nb_events++; if (ret < 0) return ret; fs->frame_ready = 0; diff --git a/libavfilter/framesync.h b/libavfilter/framesync.h index 233f50a0eb..979f54e16e 100644 --- a/libavfilter/framesync.h +++ b/libavfilter/framesync.h @@ -188,6 +188,11 @@ typedef struct FFFrameSync { */ int64_t pts; + /** + * Number of times on_event() was called. + */ + uint64_t nb_events; + /** * Callback called when a frame event is ready */ @@ -229,6 +234,12 @@ typedef struct FFFrameSync { int opt_eof_action; int opt_ts_sync_mode; + char *ts_map_str; + + // explicit frame map + int64_t *ts_map; + size_t nb_ts_map; + unsigned int ts_map_allocated; } FFFrameSync; /** From patchwork Fri Jan 27 13:16:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 40134 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3ca3:b0:b9:1511:ac2c with SMTP id b35csp1338050pzj; Fri, 27 Jan 2023 05:17:46 -0800 (PST) X-Google-Smtp-Source: AMrXdXuJXxQxTIJDPETRV4kaS3kobfENcnkCu+Q1TBT62w4N9JpQjbmvNDCmGwNycj1PjJC1uDPR X-Received: by 2002:aa7:cad3:0:b0:49e:eb11:e40 with SMTP id l19-20020aa7cad3000000b0049eeb110e40mr27665240edt.34.1674825465735; Fri, 27 Jan 2023 05:17:45 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674825465; cv=none; d=google.com; s=arc-20160816; b=mRkA6ypmOLU+OJrHwP2Nj2nitauB22SmtoOsnR6wmaseJ+n5c+eVtXCUwqiYlpBNR+ /UchV+ra3O2snBy8UkfPBJlJO4jbbObUnjQ8PBIl7kkKFg3smbE4dN+iWeonnS37ZqlE u7aC5g6avcdPNIsPTUZnc0L7N3KjQTeB0R1ZdyRPXOxGwlRMQUGJny+wbUmtn1doWG/f k8S/Yon9Di0grTGtg2OY4LlfIDpTnzQALUldoGbv1ZpKfLMb2Ps1Uflw/EW8GPw+zjVb mVp34QG3PeeeZmj8kw0eXeHaINFceQAsj7ovDtCy5VfGxENf4HMxB7n7EnVlxqnvQ3Lr qjcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=u1bo+vvS2COysSqBSFfLRMUbx7oMGqaZeOjwkb3I3o8=; b=kSIY8vdhIi5JDMc1/I0RHC4b5ACUWdH16EUTU9rpY/s2PyNTocst77J7YtP1zwG1Oq lz5lc1CCZHggDZGH6DzsLTbMweBVgq+QYm7fuekNxYCak5Oy/WyRF0b+sIX3z5cJFVZO SVtGqxmk4jyWkAIf17qpe2F2K3djtZa9KJ6mkD7L/fU55CxtkSXUWRhcsL16jTQVKtW3 FxcTz/p4ZJDSP6aiMdJ1sDuOFGxzYJT3r6uqFLHdRLQvGGXdkvyRKsTb3JjG0JK1cZFh n8kDJnbuL4QXLdEi66jt+DvreWyGRhJIAOCP/fEQWveDQz0cqJLbJ4ERAMpBXPftv4Yo cUcA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d27-20020a056402401b00b0049e68398021si5962103eda.242.2023.01.27.05.17.45; Fri, 27 Jan 2023 05:17:45 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7515168BDE1; Fri, 27 Jan 2023 15:17:32 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5F69468B2C3 for ; Fri, 27 Jan 2023 15:17:24 +0200 (EET) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 64D1E240591 for ; Fri, 27 Jan 2023 14:17:23 +0100 (CET) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id r351XF8p2Gmm for ; Fri, 27 Jan 2023 14:17:22 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id C33D1240178 for ; Fri, 27 Jan 2023 14:17:22 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id 208E23A034C for ; Fri, 27 Jan 2023 14:17:15 +0100 (CET) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Fri, 27 Jan 2023 14:16:39 +0100 Message-Id: <20230127131639.4928-4-anton@khirnov.net> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20230127131639.4928-1-anton@khirnov.net> References: <20230127131639.4928-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/4] lavfi/framesync: reindent after previous commit X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: SNjX0AtXqWkO --- libavfilter/framesync.c | 56 ++++++++++++++++++++--------------------- 1 file changed, 28 insertions(+), 28 deletions(-) diff --git a/libavfilter/framesync.c b/libavfilter/framesync.c index b52cf318c0..9986b04e48 100644 --- a/libavfilter/framesync.c +++ b/libavfilter/framesync.c @@ -368,35 +368,35 @@ static int framesync_advance(FFFrameSync *fs) } pts = fs->in[0].pts; } else { - pts = INT64_MAX; - for (i = 0; i < fs->nb_in; i++) - if (fs->in[i].have_next && fs->in[i].pts_next < pts) - pts = fs->in[i].pts_next; - if (pts == INT64_MAX) { - framesync_eof(fs); - break; - } - for (i = 0; i < fs->nb_in; i++) { - FFFrameSyncIn * const in = &fs->in[i]; - - if (in->pts_next == pts || - (in->ts_mode == TS_NEAREST && in->have_next && - in->pts_next != INT64_MAX && in->pts != AV_NOPTS_VALUE && - in->pts_next - pts < pts - in->pts) || - (in->before == EXT_INFINITY && in->state == STATE_BOF)) { - frame_advance(in); - in->state = in->frame ? STATE_RUN : STATE_EOF; - if (in->sync == fs->sync_level && in->frame) - fs->frame_ready = 1; - if (in->state == STATE_EOF && in->after == EXT_STOP) - framesync_eof(fs); - } - } - if (fs->frame_ready) + pts = INT64_MAX; for (i = 0; i < fs->nb_in; i++) - if ((fs->in[i].state == STATE_BOF && - fs->in[i].before == EXT_STOP)) - fs->frame_ready = 0; + if (fs->in[i].have_next && fs->in[i].pts_next < pts) + pts = fs->in[i].pts_next; + if (pts == INT64_MAX) { + framesync_eof(fs); + break; + } + for (i = 0; i < fs->nb_in; i++) { + FFFrameSyncIn * const in = &fs->in[i]; + + if (in->pts_next == pts || + (in->ts_mode == TS_NEAREST && in->have_next && + in->pts_next != INT64_MAX && in->pts != AV_NOPTS_VALUE && + in->pts_next - pts < pts - in->pts) || + (in->before == EXT_INFINITY && in->state == STATE_BOF)) { + frame_advance(in); + in->state = in->frame ? STATE_RUN : STATE_EOF; + if (in->sync == fs->sync_level && in->frame) + fs->frame_ready = 1; + if (in->state == STATE_EOF && in->after == EXT_STOP) + framesync_eof(fs); + } + } + if (fs->frame_ready) + for (i = 0; i < fs->nb_in; i++) + if ((fs->in[i].state == STATE_BOF && + fs->in[i].before == EXT_STOP)) + fs->frame_ready = 0; } fs->pts = pts; }