From patchwork Sat Aug 25 18:35:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marton Balint X-Patchwork-Id: 10135 Delivered-To: ffmpegpatchwork@gmail.com Received: by 2002:a02:12c4:0:0:0:0:0 with SMTP id 65-v6csp4731969jap; Sat, 25 Aug 2018 11:35:13 -0700 (PDT) X-Google-Smtp-Source: ANB0VdaGDBPQ5ZyqzfTQswKJxpl+tvizTzxmN/WM43uGeKXn3oV8w6cGx9nZFTG1HdNpNUUSMkEX X-Received: by 2002:adf:9187:: with SMTP id 7-v6mr4439370wri.215.1535222113259; Sat, 25 Aug 2018 11:35:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535222113; cv=none; d=google.com; s=arc-20160816; b=JpUnhknIvMnCIsPAxEUi+jKQkyeRBliA7orgBJq2hovhcZmA2IbRKH9pWt8VHQruRo fEoK+6uHLOql3tiBSwJNZPo74o3DCat8hKSK5gl7VytROEccQQEdW4oi9hRXM5xGdkvx diTQlKivGMRlOa5gKtbHPnCxsmv3U/t2gD8MwjQT53EFn4dQDPpnkHdSm+9IlbFszemT Pof7gImB59xpVb2tNvXavA34YAxq32tVHN7+rhcqsG0H7DifXnyxTHhIceX5MEF+GHbT qoPNBAsiqFXJYaRuoBQtEMqTaXCet1CNwLjMqIvgV9W5IjktjX6SlzX0PAlQ25r9rOgC AptA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:delivered-to :arc-authentication-results; bh=1sNGjhr0mWlQKTKr5T07fw9n2j4+/a8fkTntYv1xmX4=; b=YaB2jLSkWqdkFYgjg/sdC0CIa1Yk7LqnZAs2gvBdf0mA4xrz2vIptWdz9RY8IIUajG NztLQSle23xMWViC3cmgri3Bn//+3bIVugoKq2ultab4CjieLImu21r2zna4rb6pY+66 SVfS8OkAFujW33d3CC0pwkOAmhc+I8n8/xGfpRo1NWBr5gRmY89KtmZQ2w1fHOUFIvQe vemukjA4zfCPwE9sbGYvtMZ/LMKmlGmut1cHMrVr1UoU9FPuCaqJV3kZui5b0yg82G2/ RNcOmWd7XsOfazqMZ59TQMp18FyR1dr4SiRWMrWO6JGh9zrZxa0XLibe+Im54u1Pw2nT CNug== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s5-v6si7919668wrm.364.2018.08.25.11.35.12; Sat, 25 Aug 2018 11:35:13 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7EA77689A0C; Sat, 25 Aug 2018 21:35:08 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from iq.passwd.hu (iq.passwd.hu [217.27.212.140]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7F6746805C1 for ; Sat, 25 Aug 2018 21:35:02 +0300 (EEST) Received: from localhost (localhost [127.0.0.1]) by iq.passwd.hu (Postfix) with ESMTP id 0FB90E12BB; Sat, 25 Aug 2018 20:35:05 +0200 (CEST) X-Virus-Scanned: amavisd-new at passwd.hu Received: from iq.passwd.hu ([127.0.0.1]) by localhost (iq.passwd.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pGcJJI9RLCwy; Sat, 25 Aug 2018 20:35:03 +0200 (CEST) Received: from bluegene.passwd.hu (localhost [127.0.0.1]) by iq.passwd.hu (Postfix) with ESMTP id A816CE06C8; Sat, 25 Aug 2018 20:35:03 +0200 (CEST) From: Marton Balint To: ffmpeg-devel@ffmpeg.org Date: Sat, 25 Aug 2018 20:35:01 +0200 Message-Id: <20180825183501.32385-1-cus@passwd.hu> X-Mailer: git-send-email 2.16.4 Subject: [FFmpeg-devel] [PATCH] avfilter/f_cue: add cue and acue filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Marton Balint MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" To delay filtering until a given wallclock timestamp. Signed-off-by: Marton Balint --- doc/filters.texi | 36 ++++++++++ libavfilter/Makefile | 2 + libavfilter/allfilters.c | 2 + libavfilter/f_cue.c | 182 +++++++++++++++++++++++++++++++++++++++++++++++ libavfilter/version.h | 2 +- 5 files changed, 223 insertions(+), 1 deletion(-) create mode 100644 libavfilter/f_cue.c diff --git a/doc/filters.texi b/doc/filters.texi index 32c95b591c..79eec0c808 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -551,6 +551,11 @@ Set LFO range. Set LFO rate. @end table +@section acue + +Delay audio filtering until a given wallclock timestamp. See the @ref{cue} +filter. + @section adeclick Remove impulsive noise from input audio. @@ -6987,6 +6992,37 @@ indicates 'never reset', and returns the largest area encountered during playback. @end table +@anchor{cue} +@section cue + +Delay video filtering until a given wallclock timestamp. The filter first +passes on @option{preroll} amount of frames, then it buffers at most +@option{buffer} amount of frames and waits for the cue. After reaching the cue +it forwards the buffered frames and also any subsequent frames coming in its +input. + +The filter can be used synchronize the output of multiple ffmpeg processes for +realtime output devices like decklink. By putting the delay in the filtering +chain and pre-buffering frames the process can pass on data to output almost +immediately after the target wallclock timestamp is reached. + +Perfect frame accuracy cannot be guaranteed, but the result is good enough for +some use cases. + +@table @option + +@item cue +The cue timestamp expressed in a UNIX timestamp in microseconds. Default is 0. + +@item preroll +The duration of content to pass on as preroll expressed in seconds. Default is 0. + +@item buffer +The maximum duration of content to buffer before waiting for the cue expressed +in seconds. Default is 0. + +@end table + @anchor{curves} @section curves diff --git a/libavfilter/Makefile b/libavfilter/Makefile index e5d3a57af7..37a06e0ec0 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -36,6 +36,7 @@ OBJS-$(CONFIG_ACONTRAST_FILTER) += af_acontrast.o OBJS-$(CONFIG_ACOPY_FILTER) += af_acopy.o OBJS-$(CONFIG_ACROSSFADE_FILTER) += af_afade.o OBJS-$(CONFIG_ACRUSHER_FILTER) += af_acrusher.o +OBJS-$(CONFIG_ACUE_FILTER) += f_cue.o OBJS-$(CONFIG_ADECLICK_FILTER) += af_adeclick.o OBJS-$(CONFIG_ADECLIP_FILTER) += af_adeclick.o OBJS-$(CONFIG_ADELAY_FILTER) += af_adelay.o @@ -178,6 +179,7 @@ OBJS-$(CONFIG_COREIMAGE_FILTER) += vf_coreimage.o OBJS-$(CONFIG_COVER_RECT_FILTER) += vf_cover_rect.o lavfutils.o OBJS-$(CONFIG_CROP_FILTER) += vf_crop.o OBJS-$(CONFIG_CROPDETECT_FILTER) += vf_cropdetect.o +OBJS-$(CONFIG_CUE_FILTER) += f_cue.o OBJS-$(CONFIG_CURVES_FILTER) += vf_curves.o OBJS-$(CONFIG_DATASCOPE_FILTER) += vf_datascope.o OBJS-$(CONFIG_DCTDNOIZ_FILTER) += vf_dctdnoiz.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 9732ae5345..6c6d0f43f0 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -27,6 +27,7 @@ extern AVFilter ff_af_abench; extern AVFilter ff_af_acompressor; extern AVFilter ff_af_acontrast; extern AVFilter ff_af_acopy; +extern AVFilter ff_af_acue; extern AVFilter ff_af_acrossfade; extern AVFilter ff_af_acrusher; extern AVFilter ff_af_adeclick; @@ -167,6 +168,7 @@ extern AVFilter ff_vf_coreimage; extern AVFilter ff_vf_cover_rect; extern AVFilter ff_vf_crop; extern AVFilter ff_vf_cropdetect; +extern AVFilter ff_vf_cue; extern AVFilter ff_vf_curves; extern AVFilter ff_vf_datascope; extern AVFilter ff_vf_dctdnoiz; diff --git a/libavfilter/f_cue.c b/libavfilter/f_cue.c new file mode 100644 index 0000000000..732b5e218a --- /dev/null +++ b/libavfilter/f_cue.c @@ -0,0 +1,182 @@ +/* + * Copyright (c) 2018 Marton Balint + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with FFmpeg; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/opt.h" +#include "libavutil/time.h" +#include "avfilter.h" +#include "filters.h" +#include "framequeue.h" +#include "internal.h" + +typedef struct CueContext { + const AVClass *class; + int64_t first_pts; + int64_t cue; + int64_t preroll; + int64_t buffer; + int status; + FFFrameQueue queue; +} CueContext; + +static av_cold int init(AVFilterContext *ctx) +{ + CueContext *s = ctx->priv; + ff_framequeue_init(&s->queue, &ctx->graph->internal->frame_queues); + return 0; +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + CueContext *s = ctx->priv; + ff_framequeue_free(&s->queue); +} + +static int activate(AVFilterContext *ctx) +{ + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + CueContext *s = ctx->priv; + int64_t pts; + AVFrame *frame = NULL; + + FF_FILTER_FORWARD_STATUS_BACK(outlink, inlink); + + if (s->status < 3 || s->status == 5) { + int ret = ff_inlink_consume_frame(inlink, &frame); + if (ret < 0) + return ret; + if (frame) + pts = av_rescale_q(frame->pts, inlink->time_base, AV_TIME_BASE_Q); + } + + if (!s->status && frame) { + s->first_pts = pts; + s->status++; + } + if (s->status == 1 && frame) { + if (pts - s->first_pts < s->preroll) + return ff_filter_frame(outlink, frame); + s->first_pts = pts; + s->status++; + } + if (s->status == 2 && frame) { + int ret = ff_framequeue_add(&s->queue, frame); + if (ret < 0) { + av_frame_free(&frame); + return ret; + } + frame = NULL; + if (!(pts - s->first_pts < s->buffer && (av_gettime() - s->cue) < 0)) + s->status++; + } + if (s->status == 3) { + int64_t diff; + while ((diff = (av_gettime() - s->cue)) < 0) + av_usleep(av_clip(-diff / 2, 100, 1000000)); + s->status++; + } + if (s->status == 4) { + if (ff_framequeue_queued_frames(&s->queue)) + return ff_filter_frame(outlink, ff_framequeue_take(&s->queue)); + s->status++; + } + if (s->status == 5 && frame) + return ff_filter_frame(outlink, frame); + + FF_FILTER_FORWARD_STATUS(inlink, outlink); + FF_FILTER_FORWARD_WANTED(outlink, inlink); + + return FFERROR_NOT_READY; +} + +#define OFFSET(x) offsetof(CueContext, x) +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_FILTERING_PARAM +static const AVOption options[] = { + { "cue", "cue unix timestamp in microseconds", OFFSET(cue), AV_OPT_TYPE_INT64, { .i64 = 0 }, 0, INT64_MAX, FLAGS }, + { "preroll", "preroll duration in seconds", OFFSET(preroll), AV_OPT_TYPE_DURATION, { .i64 = 0 }, 0, INT64_MAX, FLAGS }, + { "buffer", "buffer duration in seconds", OFFSET(buffer), AV_OPT_TYPE_DURATION, { .i64 = 0 }, 0, INT64_MAX, FLAGS }, + { NULL } +}; + +#if CONFIG_CUE_FILTER +#define cue_options options +AVFILTER_DEFINE_CLASS(cue); + +static const AVFilterPad cue_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, + { NULL } +}; + +static const AVFilterPad cue_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, + { NULL } +}; + +AVFilter ff_vf_cue = { + .name = "cue", + .description = NULL_IF_CONFIG_SMALL("Delay filtering to match a cue."), + .priv_size = sizeof(CueContext), + .priv_class = &cue_class, + .init = init, + .uninit = uninit, + .inputs = cue_inputs, + .outputs = cue_outputs, + .activate = activate, +}; +#endif /* CONFIG_CUE_FILTER */ + +#if CONFIG_ACUE_FILTER +#define acue_options options +AVFILTER_DEFINE_CLASS(acue); + +static const AVFilterPad acue_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + }, + { NULL } +}; + +static const AVFilterPad acue_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + }, + { NULL } +}; + +AVFilter ff_af_acue = { + .name = "acue", + .description = NULL_IF_CONFIG_SMALL("Delay filtering to match a cue."), + .priv_size = sizeof(CueContext), + .priv_class = &acue_class, + .init = init, + .uninit = uninit, + .inputs = acue_inputs, + .outputs = acue_outputs, + .activate = activate, +}; +#endif /* CONFIG_ACUE_FILTER */ diff --git a/libavfilter/version.h b/libavfilter/version.h index 0ac3a2f3a9..2ff2b6a318 100644 --- a/libavfilter/version.h +++ b/libavfilter/version.h @@ -30,7 +30,7 @@ #include "libavutil/version.h" #define LIBAVFILTER_VERSION_MAJOR 7 -#define LIBAVFILTER_VERSION_MINOR 26 +#define LIBAVFILTER_VERSION_MINOR 27 #define LIBAVFILTER_VERSION_MICRO 100 #define LIBAVFILTER_VERSION_INT AV_VERSION_INT(LIBAVFILTER_VERSION_MAJOR, \