From patchwork Thu Sep 7 07:42:24 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas George X-Patchwork-Id: 5020 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.15.201 with SMTP id 70csp1614242jao; Thu, 7 Sep 2017 00:43:30 -0700 (PDT) X-Google-Smtp-Source: ADKCNb7qpNMQOdp6i7uJKk+OFTTj+XEAAez4jBCFbByFWJDlgeOxG13+9JOC65ixeO9Iuhmn46ZV X-Received: by 10.28.236.91 with SMTP id k88mr1636852wmh.95.1504770210321; Thu, 07 Sep 2017 00:43:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1504770210; cv=none; d=google.com; s=arc-20160816; b=YzedbIlh4FPLejkSK9sH7wPsCBwM7e6ZSe8ilgQw5Xu8+v2S6eeK4jluOoBQ2P1yNb GilBbLxBxUUEMNAqzSaLdeqzzNWm/AuCUfanr5Biabyc5cyS2wKdQkqxiYeDBXDoQ4wP +DqS/v/Kl1gqWaZc0JhI+uG/AN8jtq4scrmSUbNeZquJTkLXTmlP4lbD+mAk4OKl81ND q/krJotxC4jUhY2sXFxPCfFnX4p6SR2nkknpX9Eq8wZpuPiZuMBaFGo07uknQV8hFQuC zvRPnuYUXV7EM3DivAMx/rHHfx3JcxEqDV56y8AOX61Ye0Ba6jinDzyKhZI4+BpPxPcS vcMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to:arc-authentication-results; bh=QGyPjnWmDXUyw95CNgNQyRZR3RlXSK1UMUgZJHmjTMQ=; b=eGjLkz1IRj9GrUBARPBhx5u9QvtQa3pGTvcw7jTV0nkhfWxTIxWpe7EdpxqF7isTLY QeIDj2ozhWJ8T00MSctMNrJrQACKI64CfKZ8OvPaDgouoUP0d60A8kiYJtt5WIfM3xD8 EeZYb1pjaswA3wcjhnRFCcaaVFMjIXNg+zgJB9l7P4YU8G0hrlE+skItCZ7cXN5vsTtI nbs14HTBm9dLAsyxgbAjvVDSsUsma8G+HvmwrYf8RALdBi78JQ4RJFbep1leN5GVrDDm lQM5nQXOz1jLuVLd1MMCxWLGdP8RNZChod8dle/uYjbP7uFYgMQuxv3xscG8E4AUJ23q VFfw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id q202si744966wme.148.2017.09.07.00.43.29; Thu, 07 Sep 2017 00:43:30 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1E18D68829E; Thu, 7 Sep 2017 10:42:35 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from nef2.ens.fr (nef2.ens.fr [129.199.96.40]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 167486883EA for ; Thu, 7 Sep 2017 10:42:27 +0300 (EEST) Received: from phare.normalesup.org (phare.normalesup.org [129.199.129.80]) by nef2.ens.fr (8.13.6/1.01.28121999) with ESMTP id v877gUVd051247 for ; Thu, 7 Sep 2017 09:42:30 +0200 (CEST) Received: by phare.normalesup.org (Postfix, from userid 1001) id 3C746E00AC; Thu, 7 Sep 2017 09:42:30 +0200 (CEST) From: Nicolas George To: ffmpeg-devel@ffmpeg.org Date: Thu, 7 Sep 2017 09:42:24 +0200 Message-Id: <20170907074226.7273-6-george@nsup.org> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20170907074226.7273-1-george@nsup.org> References: <20170907074226.7273-1-george@nsup.org> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (nef2.ens.fr [129.199.96.32]); Thu, 07 Sep 2017 09:42:30 +0200 (CEST) Subject: [FFmpeg-devel] [PATCH 6/8] lavfi: remove framesync. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Nicolas George --- libavfilter/framesync.c | 343 ------------------------------------------------ libavfilter/framesync.h | 297 ----------------------------------------- 2 files changed, 640 deletions(-) delete mode 100644 libavfilter/framesync.c delete mode 100644 libavfilter/framesync.h diff --git a/libavfilter/framesync.c b/libavfilter/framesync.c deleted file mode 100644 index eb05d66a86..0000000000 --- a/libavfilter/framesync.c +++ /dev/null @@ -1,343 +0,0 @@ -/* - * Copyright (c) 2013 Nicolas George - * - * This file is part of FFmpeg. - * - * FFmpeg is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public License - * as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * FFmpeg is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public License - * along with FFmpeg; if not, write to the Free Software Foundation, Inc., - * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA - */ - -#define FF_INTERNAL_FIELDS 1 -#include "framequeue.h" - -#include "libavutil/avassert.h" -#include "avfilter.h" -#include "bufferqueue.h" -#include "framesync.h" -#include "internal.h" - -#define OFFSET(member) offsetof(FFFrameSync, member) - -static const char *framesync_name(void *ptr) -{ - return "framesync"; -} - -static const AVClass framesync_class = { - .version = LIBAVUTIL_VERSION_INT, - .class_name = "framesync", - .item_name = framesync_name, - .category = AV_CLASS_CATEGORY_FILTER, - .option = NULL, - .parent_log_context_offset = OFFSET(parent), -}; - -enum { - STATE_BOF, - STATE_RUN, - STATE_EOF, -}; - -int ff_framesync_init(FFFrameSync *fs, void *parent, unsigned nb_in) -{ - fs->class = &framesync_class; - fs->parent = parent; - fs->nb_in = nb_in; - - fs->in = av_calloc(nb_in, sizeof(*fs->in)); - if (!fs->in) - return AVERROR(ENOMEM); - return 0; -} - -static void framesync_sync_level_update(FFFrameSync *fs) -{ - unsigned i, level = 0; - - for (i = 0; i < fs->nb_in; i++) - if (fs->in[i].state != STATE_EOF) - level = FFMAX(level, fs->in[i].sync); - av_assert0(level <= fs->sync_level); - if (level < fs->sync_level) - av_log(fs, AV_LOG_VERBOSE, "Sync level %u\n", level); - if (level) - fs->sync_level = level; - else - fs->eof = 1; -} - -int ff_framesync_configure(FFFrameSync *fs) -{ - unsigned i; - int64_t gcd, lcm; - - if (!fs->time_base.num) { - for (i = 0; i < fs->nb_in; i++) { - if (fs->in[i].sync) { - if (fs->time_base.num) { - gcd = av_gcd(fs->time_base.den, fs->in[i].time_base.den); - lcm = (fs->time_base.den / gcd) * fs->in[i].time_base.den; - if (lcm < AV_TIME_BASE / 2) { - fs->time_base.den = lcm; - fs->time_base.num = av_gcd(fs->time_base.num, - fs->in[i].time_base.num); - } else { - fs->time_base.num = 1; - fs->time_base.den = AV_TIME_BASE; - break; - } - } else { - fs->time_base = fs->in[i].time_base; - } - } - } - if (!fs->time_base.num) { - av_log(fs, AV_LOG_ERROR, "Impossible to set time base\n"); - return AVERROR(EINVAL); - } - av_log(fs, AV_LOG_VERBOSE, "Selected %d/%d time base\n", - fs->time_base.num, fs->time_base.den); - } - - for (i = 0; i < fs->nb_in; i++) - fs->in[i].pts = fs->in[i].pts_next = AV_NOPTS_VALUE; - fs->sync_level = UINT_MAX; - framesync_sync_level_update(fs); - - return 0; -} - -static void framesync_advance(FFFrameSync *fs) -{ - int latest; - unsigned i; - int64_t pts; - - if (fs->eof) - return; - while (!fs->frame_ready) { - latest = -1; - for (i = 0; i < fs->nb_in; i++) { - if (!fs->in[i].have_next) { - if (latest < 0 || fs->in[i].pts < fs->in[latest].pts) - latest = i; - } - } - if (latest >= 0) { - fs->in_request = latest; - break; - } - - pts = fs->in[0].pts_next; - for (i = 1; i < fs->nb_in; i++) - if (fs->in[i].pts_next < pts) - pts = fs->in[i].pts_next; - if (pts == INT64_MAX) { - fs->eof = 1; - break; - } - for (i = 0; i < fs->nb_in; i++) { - if (fs->in[i].pts_next == pts || - (fs->in[i].before == EXT_INFINITY && - fs->in[i].state == STATE_BOF)) { - av_frame_free(&fs->in[i].frame); - fs->in[i].frame = fs->in[i].frame_next; - fs->in[i].pts = fs->in[i].pts_next; - fs->in[i].frame_next = NULL; - fs->in[i].pts_next = AV_NOPTS_VALUE; - fs->in[i].have_next = 0; - fs->in[i].state = fs->in[i].frame ? STATE_RUN : STATE_EOF; - if (fs->in[i].sync == fs->sync_level && fs->in[i].frame) - fs->frame_ready = 1; - if (fs->in[i].state == STATE_EOF && - fs->in[i].after == EXT_STOP) - fs->eof = 1; - } - } - if (fs->eof) - fs->frame_ready = 0; - if (fs->frame_ready) - for (i = 0; i < fs->nb_in; i++) - if ((fs->in[i].state == STATE_BOF && - fs->in[i].before == EXT_STOP)) - fs->frame_ready = 0; - fs->pts = pts; - } -} - -static int64_t framesync_pts_extrapolate(FFFrameSync *fs, unsigned in, - int64_t pts) -{ - /* Possible enhancement: use the link's frame rate */ - return pts + 1; -} - -static void framesync_inject_frame(FFFrameSync *fs, unsigned in, AVFrame *frame) -{ - int64_t pts; - - av_assert0(!fs->in[in].have_next); - if (frame) { - pts = av_rescale_q(frame->pts, fs->in[in].time_base, fs->time_base); - frame->pts = pts; - } else { - pts = fs->in[in].state != STATE_RUN || fs->in[in].after == EXT_INFINITY - ? INT64_MAX : framesync_pts_extrapolate(fs, in, fs->in[in].pts); - fs->in[in].sync = 0; - framesync_sync_level_update(fs); - } - fs->in[in].frame_next = frame; - fs->in[in].pts_next = pts; - fs->in[in].have_next = 1; -} - -int ff_framesync_add_frame(FFFrameSync *fs, unsigned in, AVFrame *frame) -{ - av_assert1(in < fs->nb_in); - if (!fs->in[in].have_next) - framesync_inject_frame(fs, in, frame); - else - ff_bufqueue_add(fs, &fs->in[in].queue, frame); - return 0; -} - -void ff_framesync_next(FFFrameSync *fs) -{ - unsigned i; - - av_assert0(!fs->frame_ready); - for (i = 0; i < fs->nb_in; i++) - if (!fs->in[i].have_next && fs->in[i].queue.available) - framesync_inject_frame(fs, i, ff_bufqueue_get(&fs->in[i].queue)); - fs->frame_ready = 0; - framesync_advance(fs); -} - -void ff_framesync_drop(FFFrameSync *fs) -{ - fs->frame_ready = 0; -} - -int ff_framesync_get_frame(FFFrameSync *fs, unsigned in, AVFrame **rframe, - unsigned get) -{ - AVFrame *frame; - unsigned need_copy = 0, i; - int64_t pts_next; - int ret; - - if (!fs->in[in].frame) { - *rframe = NULL; - return 0; - } - frame = fs->in[in].frame; - if (get) { - /* Find out if we need to copy the frame: is there another sync - stream, and do we know if its current frame will outlast this one? */ - pts_next = fs->in[in].have_next ? fs->in[in].pts_next : INT64_MAX; - for (i = 0; i < fs->nb_in && !need_copy; i++) - if (i != in && fs->in[i].sync && - (!fs->in[i].have_next || fs->in[i].pts_next < pts_next)) - need_copy = 1; - if (need_copy) { - if (!(frame = av_frame_clone(frame))) - return AVERROR(ENOMEM); - if ((ret = av_frame_make_writable(frame)) < 0) { - av_frame_free(&frame); - return ret; - } - } else { - fs->in[in].frame = NULL; - } - fs->frame_ready = 0; - } - *rframe = frame; - return 0; -} - -void ff_framesync_uninit(FFFrameSync *fs) -{ - unsigned i; - - for (i = 0; i < fs->nb_in; i++) { - av_frame_free(&fs->in[i].frame); - av_frame_free(&fs->in[i].frame_next); - ff_bufqueue_discard_all(&fs->in[i].queue); - } - - av_freep(&fs->in); -} - -int ff_framesync_process_frame(FFFrameSync *fs, unsigned all) -{ - int ret, count = 0; - - av_assert0(fs->on_event); - while (1) { - ff_framesync_next(fs); - if (fs->eof || !fs->frame_ready) - break; - if ((ret = fs->on_event(fs)) < 0) - return ret; - ff_framesync_drop(fs); - count++; - if (!all) - break; - } - if (!count && fs->eof) - return AVERROR_EOF; - return count; -} - -int ff_framesync_filter_frame(FFFrameSync *fs, AVFilterLink *inlink, - AVFrame *in) -{ - int ret; - - if ((ret = ff_framesync_process_frame(fs, 1)) < 0) - return ret; - if ((ret = ff_framesync_add_frame(fs, FF_INLINK_IDX(inlink), in)) < 0) - return ret; - if ((ret = ff_framesync_process_frame(fs, 0)) < 0) - return ret; - return 0; -} - -int ff_framesync_request_frame(FFFrameSync *fs, AVFilterLink *outlink) -{ - AVFilterContext *ctx = outlink->src; - int input, ret, i; - - if ((ret = ff_framesync_process_frame(fs, 0)) < 0) - return ret; - if (ret > 0) - return 0; - if (fs->eof) - return AVERROR_EOF; - input = fs->in_request; - /* Detect status change early */ - for (i = 0; i < fs->nb_in; i++) - if (!ff_framequeue_queued_frames(&ctx->inputs[i]->fifo) && - ctx->inputs[i]->status_in && !ctx->inputs[i]->status_out) - input = i; - ret = ff_request_frame(ctx->inputs[input]); - if (ret == AVERROR_EOF) { - if ((ret = ff_framesync_add_frame(fs, input, NULL)) < 0) - return ret; - if ((ret = ff_framesync_process_frame(fs, 0)) < 0) - return ret; - ret = 0; - } - return ret; -} diff --git a/libavfilter/framesync.h b/libavfilter/framesync.h deleted file mode 100644 index 7ba99d5d86..0000000000 --- a/libavfilter/framesync.h +++ /dev/null @@ -1,297 +0,0 @@ -/* - * Copyright (c) 2013 Nicolas George - * - * This file is part of FFmpeg. - * - * FFmpeg is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public License - * as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * FFmpeg is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public License - * along with FFmpeg; if not, write to the Free Software Foundation, Inc., - * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA - */ - -#ifndef AVFILTER_FRAMESYNC_H -#define AVFILTER_FRAMESYNC_H - -#include "bufferqueue.h" - -/* - * TODO - * Callback-based API similar to dualinput. - * Export convenient options. - */ - -/** - * This API is intended as a helper for filters that have several video - * input and need to combine them somehow. If the inputs have different or - * variable frame rate, getting the input frames to match requires a rather - * complex logic and a few user-tunable options. - * - * In this API, when a set of synchronized input frames is ready to be - * procesed is called a frame event. Frame event can be generated in - * response to input frames on any or all inputs and the handling of - * situations where some stream extend beyond the beginning or the end of - * others can be configured. - * - * The basic working of this API is the following: - * - * - When a frame is available on any input, add it using - * ff_framesync_add_frame(). - * - * - When a frame event is ready to be processed (i.e. after adding a frame - * or when requested on input): - * - call ff_framesync_next(); - * - if fs->frame_ready is true, process the frames; - * - call ff_framesync_drop(). - */ - -/** - * Stream extrapolation mode - * - * Describe how the frames of a stream are extrapolated before the first one - * and after EOF to keep sync with possibly longer other streams. - */ -enum FFFrameSyncExtMode { - - /** - * Completely stop all streams with this one. - */ - EXT_STOP, - - /** - * Ignore this stream and continue processing the other ones. - */ - EXT_NULL, - - /** - * Extend the frame to infinity. - */ - EXT_INFINITY, -}; - -/** - * Input stream structure - */ -typedef struct FFFrameSyncIn { - - /** - * Queue of incoming AVFrame, and NULL to mark EOF - */ - struct FFBufQueue queue; - - /** - * Extrapolation mode for timestamps before the first frame - */ - enum FFFrameSyncExtMode before; - - /** - * Extrapolation mode for timestamps after the last frame - */ - enum FFFrameSyncExtMode after; - - /** - * Time base for the incoming frames - */ - AVRational time_base; - - /** - * Current frame, may be NULL before the first one or after EOF - */ - AVFrame *frame; - - /** - * Next frame, for internal use - */ - AVFrame *frame_next; - - /** - * PTS of the current frame - */ - int64_t pts; - - /** - * PTS of the next frame, for internal use - */ - int64_t pts_next; - - /** - * Boolean flagging the next frame, for internal use - */ - uint8_t have_next; - - /** - * State: before first, in stream or after EOF, for internal use - */ - uint8_t state; - - /** - * Synchronization level: frames on input at the highest sync level will - * generate output frame events. - * - * For example, if inputs #0 and #1 have sync level 2 and input #2 has - * sync level 1, then a frame on either input #0 or #1 will generate a - * frame event, but not a frame on input #2 until both inputs #0 and #1 - * have reached EOF. - * - * If sync is 0, no frame event will be generated. - */ - unsigned sync; - -} FFFrameSyncIn; - -/** - * Frame sync structure. - */ -typedef struct FFFrameSync { - const AVClass *class; - void *parent; - - /** - * Number of input streams - */ - unsigned nb_in; - - /** - * Time base for the output events - */ - AVRational time_base; - - /** - * Timestamp of the current event - */ - int64_t pts; - - /** - * Callback called when a frame event is ready - */ - int (*on_event)(struct FFFrameSync *fs); - - /** - * Opaque pointer, not used by the API - */ - void *opaque; - - /** - * Index of the input that requires a request - */ - unsigned in_request; - - /** - * Synchronization level: only inputs with the same sync level are sync - * sources. - */ - unsigned sync_level; - - /** - * Flag indicating that a frame event is ready - */ - uint8_t frame_ready; - - /** - * Flag indicating that output has reached EOF. - */ - uint8_t eof; - - /** - * Pointer to array of inputs. - */ - FFFrameSyncIn *in; - -} FFFrameSync; - -/** - * Initialize a frame sync structure. - * - * The entire structure is expected to be already set to 0. - * - * @param fs frame sync structure to initialize - * @param parent parent object, used for logging - * @param nb_in number of inputs - * @return >= 0 for success or a negative error code - */ -int ff_framesync_init(FFFrameSync *fs, void *parent, unsigned nb_in); - -/** - * Configure a frame sync structure. - * - * Must be called after all options are set but before all use. - * - * @return >= 0 for success or a negative error code - */ -int ff_framesync_configure(FFFrameSync *fs); - -/** - * Free all memory currently allocated. - */ -void ff_framesync_uninit(FFFrameSync *fs); - -/** - * Add a frame to an input - * - * Typically called from the filter_frame() method. - * - * @param fs frame sync structure - * @param in index of the input - * @param frame input frame, or NULL for EOF - */ -int ff_framesync_add_frame(FFFrameSync *fs, unsigned in, AVFrame *frame); - -/** - * Prepare the next frame event. - * - * The status of the operation can be found in fs->frame_ready and fs->eof. - */ -void ff_framesync_next(FFFrameSync *fs); - -/** - * Drop the current frame event. - */ -void ff_framesync_drop(FFFrameSync *fs); - -/** - * Get the current frame in an input. - * - * @param fs frame sync structure - * @param in index of the input - * @param rframe used to return the current frame (or NULL) - * @param get if not zero, the calling code needs to get ownership of - * the returned frame; the current frame will either be - * duplicated or removed from the framesync structure - */ -int ff_framesync_get_frame(FFFrameSync *fs, unsigned in, AVFrame **rframe, - unsigned get); - -/** - * Process one or several frame using the on_event callback. - * - * @return number of frames processed or negative error code - */ -int ff_framesync_process_frame(FFFrameSync *fs, unsigned all); - - -/** - * Accept a frame on a filter input. - * - * This function can be the complete implementation of all filter_frame - * methods of a filter using framesync. - */ -int ff_framesync_filter_frame(FFFrameSync *fs, AVFilterLink *inlink, - AVFrame *in); - -/** - * Request a frame on the filter output. - * - * This function can be the complete implementation of all filter_frame - * methods of a filter using framesync if it has only one output. - */ -int ff_framesync_request_frame(FFFrameSync *fs, AVFilterLink *outlink); - -#endif /* AVFILTER_FRAMESYNC_H */