From patchwork Sun Aug 1 14:21:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 29166 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:6c0f:0:0:0:0:0 with SMTP id a15csp872183ioh; Sun, 1 Aug 2021 08:21:08 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxOtkc4tcQz8e1m0LRKAXNQnT5Y/tEvfa07l313QcGfVP+spjh7+TS/rt+fMXTwchLMN01n X-Received: by 2002:a17:906:7b49:: with SMTP id n9mr9562401ejo.299.1627831268052; Sun, 01 Aug 2021 08:21:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1627831268; cv=none; d=google.com; s=arc-20160816; b=acYvVaEDPnoaOxh6E5D8trKCNBE1g9KBiDIEzhJnMF63I2K3JD8Em0QTme7C+vCQ6L S11EFWbbR5pCrlQfYWFTd9zTJqy0YuoDftgdUzcQ+B6FHlrzNqWjfl2seKPtsTPcLENI +s9FmM+TRIDyk+8Al1vNXupAOKeYdjHtuCJbBGpfm9+xGeLLFKpOdH2aOUb9pfNSa5Ty 4Q8SIPL4MCJuLt+uZrWBx+4BD1xPrp5XxJDNVLY/c31NVrRG8IqUruxEughJ2PkKu32i dE9fip39osc+ta7nWU/UiKB6dY/+CshONOKPA8xLfpxcV9NevzqLExCD6zZkPZhlwu7D oc2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to; bh=HeV4SxR7SZctxE3rdMxIa56wim6aLIQdFzqDl2lMywU=; b=vTNrtCoB/8RunZfumE76lugs5lWEYdIOr6+WkXP78R2VcLJ6YUP4MLI8km6xFhfnD8 NBP0glv27dpmcj4FRUnjgWbzLg9b5hxnOUPXyaSiuPKPe9e3HLWpXz+JMKT6dcG74b/T IuczCM8nv5/AI0BXgQROf+qwLRRdTZUqHyO7GsG7/bQcRtCcotkmtPjASxsFVi6ekTQv CtdPKcjmL2j7SiHiQJ519IkNbSKRrPlBNq78bjMH6HOPnsb2hT/7Bg5wqIRI28yObKKL UskeCi9UDtz9L4zpqm10bbcL1UMiNoGhWeVtCd5GLcEF+smm6amwMd7ifM5RVw4IZAec E8hw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b="bZIjT8a/"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id t14si5399581ejy.51.2021.08.01.08.21.07; Sun, 01 Aug 2021 08:21:08 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b="bZIjT8a/"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 695A268A5E6; Sun, 1 Aug 2021 18:21:05 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4270768A5AC for ; Sun, 1 Aug 2021 18:21:02 +0300 (EEST) Received: by mail-ed1-f43.google.com with SMTP id x14so20960569edr.12 for ; Sun, 01 Aug 2021 08:21:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=QSTteIPTJkaQkf39ITXKtCYm34vK3Uke8SLijH77huc=; b=bZIjT8a/27xDLyLxfaUPIzZbOGST2UUiGuqk6jgTHd5AqWYtTj9ec8K2vKnwoyUoUY DLwmlHfyJMnOiCpKbrACCdddvPuNqYexZ/7ZMAzb6bOKk4WwX63+zGBfWbletEjcnUlz oFG7t+W+J688OqNpSig+wNJ1JxYzFH3JoRwmeaZxCtaRBDBqxWAWupJ/Xl0XMRhU4XQQ eHGf6zevPI1WW7KTPJa+usBNxb3FtLfoblH1vsGJ39T6LMACOaeL0F1DXLBHXKCUH+sz 7Uoirxxhd8dbIhj1+rLRJe7C9MAmX7/M7TlIHBBLQUYBT0+bidoaDP0AbUFTr6St0hOB PAmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=QSTteIPTJkaQkf39ITXKtCYm34vK3Uke8SLijH77huc=; b=plFLFs9pFJbT4tdmRbNKcZFS2cj4K4ifmTReorPdBsNQrICiXt3auntWmdNKdGtZSo eG7/RtfICN9ryEJclXR1ly1xMmjvn/oU6fnvB7SYjxYmfJzNkDUfE6KuEs2F0GnR/bZ7 Vl5MtHhIxg95DF/pF80FWRVoP6uz4CDVmI1oJ0OPdG90FKwgOzFU6unlJn/+/o2akAn8 cNnUSYiYo/pz2wQ6ViTxy7FavTDiNZjYOBHAXbkZLlrckC5rawhKcNylMms3DU4r8zR0 pnbOAWatGddQbJlbvTRKjtrVf7PCp+P3SR4+E4pJhI4f7KYXvQIr6iEe+w98Ou7ywXcb +LBw== X-Gm-Message-State: AOAM533qPmnFnd2rlpk7w34El2KdPdcjg0kgUNYyfkMDP+Hlv+JmKFvT 8ZNUARpYGtjUpnpHFjonO9p6QQ3sMPA= X-Received: by 2002:adf:a40f:: with SMTP id d15mr13460239wra.70.1627827707547; Sun, 01 Aug 2021 07:21:47 -0700 (PDT) Received: from localhost.localdomain ([95.168.120.6]) by smtp.gmail.com with ESMTPSA id e5sm9746813wrr.36.2021.08.01.07.21.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Aug 2021 07:21:47 -0700 (PDT) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Sun, 1 Aug 2021 16:21:39 +0200 Message-Id: <20210801142139.25665-1-onemda@gmail.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH] avfilter: add (a)separate filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 7qX+fMzSxvCp Signed-off-by: Paul B Mahol --- doc/filters.texi | 31 ++++ libavfilter/Makefile | 2 + libavfilter/allfilters.c | 2 + libavfilter/f_separate.c | 346 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 381 insertions(+) create mode 100644 libavfilter/f_separate.c diff --git a/doc/filters.texi b/doc/filters.texi index 635179edb9..e40322417b 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -26067,6 +26067,37 @@ sendcmd=f=test.cmd,drawtext=fontfile=FreeSerif.ttf:text='',hue @end example @end itemize +@section separate, aseparate + +Separate single input stream into multiple streams. + +@code{separate} works on video frames, @code{aseparate} on audio samples. + +This filter accepts the following options: + +@table @option +@item durations +Durations of input at which to separate input. Each duration point is separated by '|'. + +@item frames, samples +Exact frame/sample at which to do separations of input video/audio stream. Each point +is separated by '|'. +@end table + +@subsection Examples + +@itemize + +@item +Separate input audio stream into three output audio streams, starting at start of input audio stream +and storing that in 1st output audio stream, then following at 60th second and storing than in 2nd +output audio stream, and last after 120th second of input audio stream store in 3rd output audio stream: +@example +aseparate=durations="60 | 120" +@end example + +@end itemize + @anchor{setpts} @section setpts, asetpts diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 772971521c..73c26f4870 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -80,6 +80,7 @@ OBJS-$(CONFIG_AREVERSE_FILTER) += f_reverse.o OBJS-$(CONFIG_ARNNDN_FILTER) += af_arnndn.o OBJS-$(CONFIG_ASELECT_FILTER) += f_select.o OBJS-$(CONFIG_ASENDCMD_FILTER) += f_sendcmd.o +OBJS-$(CONFIG_ASEPARATE_FILTER) += f_separate.o OBJS-$(CONFIG_ASETNSAMPLES_FILTER) += af_asetnsamples.o OBJS-$(CONFIG_ASETPTS_FILTER) += setpts.o OBJS-$(CONFIG_ASETRATE_FILTER) += af_asetrate.o @@ -408,6 +409,7 @@ OBJS-$(CONFIG_SCROLL_FILTER) += vf_scroll.o OBJS-$(CONFIG_SELECT_FILTER) += f_select.o OBJS-$(CONFIG_SELECTIVECOLOR_FILTER) += vf_selectivecolor.o OBJS-$(CONFIG_SENDCMD_FILTER) += f_sendcmd.o +OBJS-$(CONFIG_SEPARATE_FILTER) += f_separate.o OBJS-$(CONFIG_SEPARATEFIELDS_FILTER) += vf_separatefields.o OBJS-$(CONFIG_SETDAR_FILTER) += vf_aspect.o OBJS-$(CONFIG_SETFIELD_FILTER) += vf_setparams.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index e0276eb98c..c11c680564 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -73,6 +73,7 @@ extern const AVFilter ff_af_areverse; extern const AVFilter ff_af_arnndn; extern const AVFilter ff_af_aselect; extern const AVFilter ff_af_asendcmd; +extern const AVFilter ff_af_aseparate; extern const AVFilter ff_af_asetnsamples; extern const AVFilter ff_af_asetpts; extern const AVFilter ff_af_asetrate; @@ -389,6 +390,7 @@ extern const AVFilter ff_vf_scroll; extern const AVFilter ff_vf_select; extern const AVFilter ff_vf_selectivecolor; extern const AVFilter ff_vf_sendcmd; +extern const AVFilter ff_vf_separate; extern const AVFilter ff_vf_separatefields; extern const AVFilter ff_vf_setdar; extern const AVFilter ff_vf_setfield; diff --git a/libavfilter/f_separate.c b/libavfilter/f_separate.c new file mode 100644 index 0000000000..46f8871c8a --- /dev/null +++ b/libavfilter/f_separate.c @@ -0,0 +1,346 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include + +#include "libavutil/avstring.h" +#include "libavutil/channel_layout.h" +#include "libavutil/common.h" +#include "libavutil/log.h" +#include "libavutil/mathematics.h" +#include "libavutil/opt.h" +#include "libavutil/parseutils.h" +#include "libavutil/samplefmt.h" + +#include "audio.h" +#include "avfilter.h" +#include "filters.h" +#include "internal.h" + +typedef struct SeparateContext { + const AVClass *class; + + char *durations_str; + char *points_str; + + int current_point; + int nb_points; + + int64_t *points; + + int64_t current_frame; + int64_t current_sample; +} SeparateContext; + +static void count_points(char *item_str, int *nb_items) +{ + char *p; + + if (!item_str) + return; + + *nb_items = 1; + for (p = item_str; *p; p++) { + if (*p == '|') + (*nb_items)++; + } +} + +static int parse_durations(AVFilterContext *ctx, char *item_str, int nb_points, int64_t *points) +{ + char *arg, *p = item_str; + char *saveptr = NULL; + int ret; + + for (int i = 0; i < nb_points; i++) { + if (!(arg = av_strtok(p, "|", &saveptr))) + return AVERROR(EINVAL); + + p = NULL; + + ret = av_parse_time(&points[i], arg, 1); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Invalid durations supplied: %s\n", arg); + return AVERROR(EINVAL); + } + } + + return 0; +} + +static int parse_points(AVFilterContext *ctx, char *item_str, int nb_points, int64_t *points) +{ + char *arg, *p = item_str; + char *saveptr = NULL; + + for (int i = 0; i < nb_points; i++) { + if (!(arg = av_strtok(p, "|", &saveptr))) + return AVERROR(EINVAL); + + p = NULL; + + if (sscanf(arg, "%"PRId64, &points[i]) != 1) { + av_log(ctx, AV_LOG_ERROR, "Invalid points supplied: %s\n", arg); + return AVERROR(EINVAL); + } + } + + return 0; +} + +static av_cold int init(AVFilterContext *ctx, int type) +{ + SeparateContext *s = ctx->priv; + int ret; + + if (s->durations_str) { + count_points(s->durations_str, &s->nb_points); + s->nb_points++; + } else { + count_points(s->points_str, &s->nb_points); + s->nb_points++; + } + + s->points = av_calloc(s->nb_points, sizeof(*s->points)); + if (!s->points) + return AVERROR(ENOMEM); + + if (s->durations_str) { + ret = parse_durations(ctx, s->durations_str, s->nb_points - 1, s->points); + if (ret < 0) + return ret; + } else { + ret = parse_points(ctx, s->points_str, s->nb_points - 1, s->points); + if (ret < 0) + return ret; + } + + s->points[s->nb_points - 1] = INT64_MAX; + + for (int i = 0; i < s->nb_points; i++) { + AVFilterPad pad = { 0 }; + + pad.type = type; + pad.name = av_asprintf("output%d", i); + if (!pad.name) + return AVERROR(ENOMEM); + + if ((ret = ff_insert_outpad(ctx, i, &pad)) < 0) { + av_freep(&pad.name); + return ret; + } + } + + return 0; +} + +static int config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + SeparateContext *s = ctx->priv; + AVRational tb = (inlink->type == AVMEDIA_TYPE_VIDEO) ? + inlink->time_base : (AVRational){ 1, inlink->sample_rate }; + + for (int i = 0; i < s->nb_points - 1; i++) { + int64_t pts = av_rescale_q(s->points[i], AV_TIME_BASE_Q, tb); + + s->points[i] = pts; + } + + return 0; +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + SeparateContext *s = ctx->priv; + + av_freep(&s->points); + + for (int i = 0; i < ctx->nb_outputs; i++) + av_freep(&ctx->output_pads[i].name); +} + +#define OFFSET(x) offsetof(SeparateContext, x) +#define COMMON_OPTS \ + { "durations", "durations of input at which to separate input", OFFSET(durations_str), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, FLAGS }, \ + +#if CONFIG_SEPARATE_FILTER + +static av_cold int video_init(AVFilterContext *ctx) +{ + return init(ctx, AVMEDIA_TYPE_VIDEO); +} + +static int video_activate(AVFilterContext *ctx) +{ + SeparateContext *s = ctx->priv; + AVFrame *frame = NULL; + int ret, status; + int64_t pts; + + for (int i = s->current_point; i < s->nb_points; i++) { + FF_FILTER_FORWARD_STATUS_BACK_ALL(ctx->outputs[i], ctx); + } + + if ((ret = ff_inlink_consume_frame(ctx->inputs[0], &frame)) > 0) { + if (s->current_frame >= s->points[s->current_point]) { + ff_outlink_set_status(ctx->outputs[s->current_point], AVERROR_EOF, frame->pts); + s->current_point++; + } + + if (s->current_point >= s->nb_points) { + av_frame_free(&frame); + return AVERROR(EINVAL); + } + + s->current_frame++; + + ret = ff_filter_frame(ctx->outputs[s->current_point], frame); + } + + if (ret < 0) { + return ret; + } else if (ff_inlink_acknowledge_status(ctx->inputs[0], &status, &pts)) { + for (int i = s->current_point; i < s->nb_points; i++) + ff_outlink_set_status(ctx->outputs[i], status, pts); + return 0; + } else { + for (int i = s->current_point; i < s->nb_points; i++) { + if (ff_outlink_frame_wanted(ctx->outputs[i])) + ff_inlink_request_frame(ctx->inputs[0]); + } + return 0; + } +} + +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_FILTERING_PARAM +static const AVOption separate_options[] = { + { "frames", "frames at which to do separation", OFFSET(points_str), AV_OPT_TYPE_STRING, { .str = "25" }, 0, 0, FLAGS }, + COMMON_OPTS + { NULL } +}; +#undef FLAGS + +AVFILTER_DEFINE_CLASS(separate); + +static const AVFilterPad separate_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_input, + }, + { NULL } +}; + +const AVFilter ff_vf_separate = { + .name = "separate", + .description = NULL_IF_CONFIG_SMALL("Separate video stream."), + .init = video_init, + .uninit = uninit, + .priv_size = sizeof(SeparateContext), + .priv_class = &separate_class, + .activate = video_activate, + .inputs = separate_inputs, + .outputs = NULL, + .flags = AVFILTER_FLAG_DYNAMIC_OUTPUTS, +}; +#endif // CONFIG_SEPARATE_FILTER + +#if CONFIG_ASEPARATE_FILTER + +static av_cold int audio_init(AVFilterContext *ctx) +{ + return init(ctx, AVMEDIA_TYPE_AUDIO); +} + +static int audio_activate(AVFilterContext *ctx) +{ + SeparateContext *s = ctx->priv; + AVFrame *frame = NULL; + int ret, status; + int64_t pts; + + for (int i = s->current_point; i < s->nb_points; i++) { + FF_FILTER_FORWARD_STATUS_BACK_ALL(ctx->outputs[i], ctx); + } + + if ((ret = ff_inlink_consume_samples(ctx->inputs[0], 1, + FFMIN(s->points[s->current_point] - s->current_sample, INT_MAX), + &frame)) > 0) { + s->current_sample += frame->nb_samples; + + if (s->current_sample >= s->points[s->current_point]) { + ff_outlink_set_status(ctx->outputs[s->current_point], AVERROR_EOF, frame->pts); + s->current_point++; + } + + if (s->current_point >= s->nb_points) { + av_frame_free(&frame); + return AVERROR(EINVAL); + } + + ret = ff_filter_frame(ctx->outputs[s->current_point], frame); + } + + if (ret < 0) { + return ret; + } else if (ff_inlink_acknowledge_status(ctx->inputs[0], &status, &pts)) { + for (int i = s->current_point; i < s->nb_points; i++) + ff_outlink_set_status(ctx->outputs[i], status, pts); + return 0; + } else { + for (int i = s->current_point; i < s->nb_points; i++) { + if (ff_outlink_frame_wanted(ctx->outputs[i])) + ff_inlink_request_frame(ctx->inputs[0]); + } + return 0; + } +} + +#define FLAGS AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_FILTERING_PARAM +static const AVOption aseparate_options[] = { + { "samples", "samples at which to do separation", OFFSET(points_str), AV_OPT_TYPE_STRING, { .str = "44100" }, 0, 0, FLAGS }, + COMMON_OPTS + { NULL } +}; +#undef FLAGS + +AVFILTER_DEFINE_CLASS(aseparate); + +static const AVFilterPad aseparate_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + .config_props = config_input, + }, + { NULL } +}; + +const AVFilter ff_af_aseparate = { + .name = "aseparate", + .description = NULL_IF_CONFIG_SMALL("Separate audio stream."), + .init = audio_init, + .uninit = uninit, + .priv_size = sizeof(SeparateContext), + .priv_class = &aseparate_class, + .activate = audio_activate, + .inputs = aseparate_inputs, + .outputs = NULL, + .flags = AVFILTER_FLAG_DYNAMIC_OUTPUTS, +}; +#endif // CONFIG_ASEPARATE_FILTER