From patchwork Mon Jul 17 14:19:25 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas George X-Patchwork-Id: 4331 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.1.76 with SMTP id 73csp209607vsb; Mon, 17 Jul 2017 07:20:29 -0700 (PDT) X-Received: by 10.28.8.144 with SMTP id 138mr4879621wmi.8.1500301229900; Mon, 17 Jul 2017 07:20:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1500301229; cv=none; d=google.com; s=arc-20160816; b=qh2b7jCzqGnEFiRRj9k902ZD5/sJhW0wNMnuaGCSwJbV276N3tgXesHOqmIX2+Wjbw hKoMjyDvq0vRgCwnkYs8eJuY4nKi2Gz6qAZFH9Kp2BWizQjtNaoNb4SYvf8EifSZ+k3z zCJNwWHHrNtThjO5/HNMvmNDI7RbL4Q3ZvShi8u/ehM3s2eiSGQyqZ0V9KoViByFFTs6 GufizfiWKTqEhZVEZxKwunDgQNgGNIpYVs5LXfU16xG0TYyUmTamIi/RDeLn639C8Ofl uK5Shn660zbkQe9nBWcQIpL/SXP5w/0BRoKNYZTkyBQVc1ixuvia+/rC4yddF3i/gvwL 6/uA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to:arc-authentication-results; bh=C2P+EWDYZyz9kFztuTeH2xPhrHoXx37EVtlXNLYAOP0=; b=SzVno/FmsDCLlysORKfnGRnaNu0mmKP39/TLnQ9zh9TvbOBfXBbilZHUh+MzpvSxfx RujJVdUe8RvP1jVd43/WOcNqJBG1XwegIcuGCG8XZOXkrSvp/VyOpJnb+KNTWEvnqf4A A+Tu3LE7wp4FJrdVK05CMMtFt3UqtbgvJ0BpT1B+evSO+mzbisGQJfXC9vO0PcnbluIT qeWv/RDxAa2Pf6wIOiTVFFpRb6qKSCmaytep5/I8n26PtnGYnbxFQwH87jCLKm96TBXr a1n6p5MEOEO5eZJJMj3Qe8haU24F5uL7X9r0pDcGOmjHFqcMoSLrJD0rg3MsBJHkp0Mh d5iw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h190si930992wme.146.2017.07.17.07.20.29; Mon, 17 Jul 2017 07:20:29 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1D86F689CFC; Mon, 17 Jul 2017 17:19:32 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from nef2.ens.fr (nef2.ens.fr [129.199.96.40]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0E4C4689C3C for ; Mon, 17 Jul 2017 17:19:25 +0300 (EEST) Received: from phare.normalesup.org (archicubes.ens.fr [129.199.129.80]) by nef2.ens.fr (8.13.6/1.01.28121999) with ESMTP id v6HEJVEC039909 for ; Mon, 17 Jul 2017 16:19:31 +0200 (CEST) Received: by phare.normalesup.org (Postfix, from userid 1001) id 759E5E00D0; Mon, 17 Jul 2017 16:19:31 +0200 (CEST) From: Nicolas George To: ffmpeg-devel@ffmpeg.org Date: Mon, 17 Jul 2017 16:19:25 +0200 Message-Id: <20170717141926.7442-6-george@nsup.org> X-Mailer: git-send-email 2.13.2 In-Reply-To: <20170717141926.7442-1-george@nsup.org> References: <20170717141926.7442-1-george@nsup.org> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (nef2.ens.fr [129.199.96.32]); Mon, 17 Jul 2017 16:19:31 +0200 (CEST) Subject: [FFmpeg-devel] [PATCH 6/7] lavfi/framesync2: implement activate design. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Nicolas George --- libavfilter/framesync2.c | 166 +++++++++++++++++++---------------------------- libavfilter/framesync2.h | 77 ++++------------------ 2 files changed, 82 insertions(+), 161 deletions(-) diff --git a/libavfilter/framesync2.c b/libavfilter/framesync2.c index 8d81bb5c48..0e9f6f210c 100644 --- a/libavfilter/framesync2.c +++ b/libavfilter/framesync2.c @@ -18,12 +18,9 @@ * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ -#define FF_INTERNAL_FIELDS 1 -#include "framequeue.h" - #include "libavutil/avassert.h" #include "avfilter.h" -#include "bufferqueue.h" +#include "filters.h" #include "framesync2.h" #include "internal.h" @@ -49,8 +46,13 @@ enum { STATE_EOF, }; -int ff_framesync2_init(FFFrameSync *fs, void *parent, unsigned nb_in) +int ff_framesync2_init(FFFrameSync *fs, AVFilterContext *parent, unsigned nb_in) { + /* For filters with several outputs, we will not be able to assume which + output is relevant for ff_outlink_frame_wanted() and + ff_outlink_set_status(). To be designed when needed. */ + av_assert0(parent->nb_outputs == 1); + fs->class = &framesync_class; fs->parent = parent; fs->nb_in = nb_in; @@ -61,6 +63,13 @@ int ff_framesync2_init(FFFrameSync *fs, void *parent, unsigned nb_in) return 0; } +static void framesync_eof(FFFrameSync *fs) +{ + fs->eof = 1; + fs->frame_ready = 0; + ff_outlink_set_status(fs->parent->outputs[0], AVERROR_EOF, AV_NOPTS_VALUE); +} + static void framesync_sync_level_update(FFFrameSync *fs) { unsigned i, level = 0; @@ -74,7 +83,7 @@ static void framesync_sync_level_update(FFFrameSync *fs) if (level) fs->sync_level = level; else - fs->eof = 1; + framesync_eof(fs); } int ff_framesync2_configure(FFFrameSync *fs) @@ -144,7 +153,7 @@ static void framesync_advance(FFFrameSync *fs) if (fs->in[i].pts_next < pts) pts = fs->in[i].pts_next; if (pts == INT64_MAX) { - fs->eof = 1; + framesync_eof(fs); break; } for (i = 0; i < fs->nb_in; i++) { @@ -162,11 +171,9 @@ static void framesync_advance(FFFrameSync *fs) fs->frame_ready = 1; if (fs->in[i].state == STATE_EOF && fs->in[i].after == EXT_STOP) - fs->eof = 1; + framesync_eof(fs); } } - if (fs->eof) - fs->frame_ready = 0; if (fs->frame_ready) for (i = 0; i < fs->nb_in; i++) if ((fs->in[i].state == STATE_BOF && @@ -188,45 +195,24 @@ static void framesync_inject_frame(FFFrameSync *fs, unsigned in, AVFrame *frame) int64_t pts; av_assert0(!fs->in[in].have_next); - if (frame) { - pts = av_rescale_q(frame->pts, fs->in[in].time_base, fs->time_base); - frame->pts = pts; - } else { - pts = fs->in[in].state != STATE_RUN || fs->in[in].after == EXT_INFINITY - ? INT64_MAX : framesync_pts_extrapolate(fs, in, fs->in[in].pts); - fs->in[in].sync = 0; - framesync_sync_level_update(fs); - } + av_assert0(frame); + pts = av_rescale_q(frame->pts, fs->in[in].time_base, fs->time_base); + frame->pts = pts; fs->in[in].frame_next = frame; fs->in[in].pts_next = pts; fs->in[in].have_next = 1; } -int ff_framesync2_add_frame(FFFrameSync *fs, unsigned in, AVFrame *frame) -{ - av_assert1(in < fs->nb_in); - if (!fs->in[in].have_next) - framesync_inject_frame(fs, in, frame); - else - ff_bufqueue_add(fs, &fs->in[in].queue, frame); - return 0; -} - -void ff_framesync2_next(FFFrameSync *fs) -{ - unsigned i; - - av_assert0(!fs->frame_ready); - for (i = 0; i < fs->nb_in; i++) - if (!fs->in[i].have_next && fs->in[i].queue.available) - framesync_inject_frame(fs, i, ff_bufqueue_get(&fs->in[i].queue)); - fs->frame_ready = 0; - framesync_advance(fs); -} - -void ff_framesync2_drop(FFFrameSync *fs) +static void framesync_inject_status(FFFrameSync *fs, unsigned in, int status, int64_t pts) { - fs->frame_ready = 0; + av_assert0(!fs->in[in].have_next); + pts = fs->in[in].state != STATE_RUN || fs->in[in].after == EXT_INFINITY + ? INT64_MAX : framesync_pts_extrapolate(fs, in, fs->in[in].pts); + fs->in[in].sync = 0; + framesync_sync_level_update(fs); + fs->in[in].frame_next = NULL; + fs->in[in].pts_next = pts; + fs->in[in].have_next = 1; } int ff_framesync2_get_frame(FFFrameSync *fs, unsigned in, AVFrame **rframe, @@ -273,71 +259,55 @@ void ff_framesync2_uninit(FFFrameSync *fs) for (i = 0; i < fs->nb_in; i++) { av_frame_free(&fs->in[i].frame); av_frame_free(&fs->in[i].frame_next); - ff_bufqueue_discard_all(&fs->in[i].queue); } av_freep(&fs->in); } -int ff_framesync2_process_frame(FFFrameSync *fs, unsigned all) +int ff_framesync2_activate(FFFrameSync *fs) { - int ret, count = 0; + AVFilterContext *ctx = fs->parent; + AVFrame *frame = NULL; + int64_t pts; + unsigned i, nb_active, nb_miss; + int ret, status; - av_assert0(fs->on_event); - while (1) { - ff_framesync2_next(fs); - if (fs->eof || !fs->frame_ready) - break; - if ((ret = fs->on_event(fs)) < 0) + nb_active = nb_miss = 0; + for (i = 0; i < fs->nb_in; i++) { + if (fs->in[i].have_next || fs->in[i].state == STATE_EOF) + continue; + nb_active++; + ret = ff_inlink_consume_frame(ctx->inputs[i], &frame); + if (ret < 0) return ret; - ff_framesync2_drop(fs); - count++; - if (!all) - break; + if (ret) { + av_assert0(frame); + framesync_inject_frame(fs, i, frame); + } else { + ret = ff_inlink_acknowledge_status(ctx->inputs[i], &status, &pts); + if (ret > 0) { + framesync_inject_status(fs, i, status, pts); + } else if (!ret) { + nb_miss++; + } + } + } + if (nb_miss) { + if (nb_miss == nb_active && !ff_outlink_frame_wanted(ctx->outputs[0])) + return FFERROR_NOT_READY; + for (i = 0; i < fs->nb_in; i++) + if (!fs->in[i].have_next && fs->in[i].state != STATE_EOF) + ff_inlink_request_frame(ctx->inputs[i]); + return 0; } - if (!count && fs->eof) - return AVERROR_EOF; - return count; -} - -int ff_framesync2_filter_frame(FFFrameSync *fs, AVFilterLink *inlink, - AVFrame *in) -{ - int ret; - if ((ret = ff_framesync2_process_frame(fs, 1)) < 0) - return ret; - if ((ret = ff_framesync2_add_frame(fs, FF_INLINK_IDX(inlink), in)) < 0) - return ret; - if ((ret = ff_framesync2_process_frame(fs, 0)) < 0) + framesync_advance(fs); + if (fs->eof || !fs->frame_ready) + return 0; + ret = fs->on_event(fs); + if (ret < 0) return ret; - return 0; -} - -int ff_framesync2_request_frame(FFFrameSync *fs, AVFilterLink *outlink) -{ - AVFilterContext *ctx = outlink->src; - int input, ret, i; + fs->frame_ready = 0; - if ((ret = ff_framesync2_process_frame(fs, 0)) < 0) - return ret; - if (ret > 0) - return 0; - if (fs->eof) - return AVERROR_EOF; - input = fs->in_request; - /* Detect status change early */ - for (i = 0; i < fs->nb_in; i++) - if (!ff_framequeue_queued_frames(&ctx->inputs[i]->fifo) && - ctx->inputs[i]->status_in && !ctx->inputs[i]->status_out) - input = i; - ret = ff_request_frame(ctx->inputs[input]); - if (ret == AVERROR_EOF) { - if ((ret = ff_framesync2_add_frame(fs, input, NULL)) < 0) - return ret; - if ((ret = ff_framesync2_process_frame(fs, 0)) < 0) - return ret; - ret = 0; - } - return ret; + return 0; } diff --git a/libavfilter/framesync2.h b/libavfilter/framesync2.h index e19d0f37e8..2b37636ebb 100644 --- a/libavfilter/framesync2.h +++ b/libavfilter/framesync2.h @@ -25,7 +25,6 @@ /* * TODO - * Callback-based API similar to dualinput. * Export convenient options. */ @@ -41,16 +40,9 @@ * situations where some stream extend beyond the beginning or the end of * others can be configured. * - * The basic working of this API is the following: - * - * - When a frame is available on any input, add it using - * ff_framesync2_add_frame(). - * - * - When a frame event is ready to be processed (i.e. after adding a frame - * or when requested on input): - * - call ff_framesync2_next(); - * - if fs->frame_ready is true, process the frames; - * - call ff_framesync2_drop(). + * The basic working of this API is the following: set the on_event + * callback, then call ff_framesync2_activate() from the filter's activate + * callback. */ /** @@ -83,11 +75,6 @@ enum FFFrameSyncExtMode { typedef struct FFFrameSyncIn { /** - * Queue of incoming AVFrame, and NULL to mark EOF - */ - struct FFBufQueue queue; - - /** * Extrapolation mode for timestamps before the first frame */ enum FFFrameSyncExtMode before; @@ -152,7 +139,11 @@ typedef struct FFFrameSyncIn { */ typedef struct FFFrameSync { const AVClass *class; - void *parent; + + /** + * Parent filter context. + */ + AVFilterContext *parent; /** * Number of input streams @@ -213,11 +204,11 @@ typedef struct FFFrameSync { * The entire structure is expected to be already set to 0. * * @param fs frame sync structure to initialize - * @param parent parent object, used for logging + * @param parent parent AVFilterContext object * @param nb_in number of inputs * @return >= 0 for success or a negative error code */ -int ff_framesync2_init(FFFrameSync *fs, void *parent, unsigned nb_in); +int ff_framesync2_init(FFFrameSync *fs, AVFilterContext *parent, unsigned nb_in); /** * Configure a frame sync structure. @@ -234,29 +225,6 @@ int ff_framesync2_configure(FFFrameSync *fs); void ff_framesync2_uninit(FFFrameSync *fs); /** - * Add a frame to an input - * - * Typically called from the filter_frame() method. - * - * @param fs frame sync structure - * @param in index of the input - * @param frame input frame, or NULL for EOF - */ -int ff_framesync2_add_frame(FFFrameSync *fs, unsigned in, AVFrame *frame); - -/** - * Prepare the next frame event. - * - * The status of the operation can be found in fs->frame_ready and fs->eof. - */ -void ff_framesync2_next(FFFrameSync *fs); - -/** - * Drop the current frame event. - */ -void ff_framesync2_drop(FFFrameSync *fs); - -/** * Get the current frame in an input. * * @param fs frame sync structure @@ -270,28 +238,11 @@ int ff_framesync2_get_frame(FFFrameSync *fs, unsigned in, AVFrame **rframe, unsigned get); /** - * Process one or several frame using the on_event callback. - * - * @return number of frames processed or negative error code - */ -int ff_framesync2_process_frame(FFFrameSync *fs, unsigned all); - - -/** - * Accept a frame on a filter input. - * - * This function can be the complete implementation of all filter_frame - * methods of a filter using framesync. - */ -int ff_framesync2_filter_frame(FFFrameSync *fs, AVFilterLink *inlink, - AVFrame *in); - -/** - * Request a frame on the filter output. + * Examine the frames in the filter's input and try to produce output. * - * This function can be the complete implementation of all filter_frame - * methods of a filter using framesync if it has only one output. + * This function can be the complete implementation of the activate + * method of a filter using framesync2. */ -int ff_framesync2_request_frame(FFFrameSync *fs, AVFilterLink *outlink); +int ff_framesync2_activate(FFFrameSync *fs); #endif /* AVFILTER_FRAMESYNC2_H */