From patchwork Wed Oct 16 18:35:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 15792 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B436244A955 for ; Wed, 16 Oct 2019 21:36:04 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8BB77689F8E; Wed, 16 Oct 2019 21:36:04 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 291816883D0 for ; Wed, 16 Oct 2019 21:35:58 +0300 (EEST) Received: by mail-wr1-f46.google.com with SMTP id n14so29166942wrw.9 for ; Wed, 16 Oct 2019 11:35:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=+84Y2HM/qnwY6vvDwbrHO+0XwIKn1mtzFxzPPd0U10o=; b=l8mc3R4xvGUHqJhAfnufwNZCGLp3I+wm1ctbNNTDAqXchAsjYG4rwDC6PTpNdDEFxP qil5e5UL/w2rJkqNZFFUbv+fN6p458GXiEYCKLYuoUIiCANhEGyfD+9gLmYEEh3B6rdw ZZJggvueOrZKmbsW1ck61tkeG/K0HpGjiJPf1T5pVpFCivgaRu+doPrrV0wLv4Q59IZC Mq8AEcZLJxXAiBLTW1IBTpPh8KEJagR7sZ9+amEKvaLzxrKv1L4cNFWCHnH+lM34bqOG jtukrcWj2DM7Z6MYh20HcPkMiz9dVG0AOv7sDwwtI+9TsQQLq/Z4H7OUQmztfI+byqan 3YTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=+84Y2HM/qnwY6vvDwbrHO+0XwIKn1mtzFxzPPd0U10o=; b=BPtYaXdevhOXPT6W3wLObMb33i8DLpjk/OWVu9xNj8pXt11yl7UAUO0TZMj67/kXOk x7OEpbNkMkNWzqzEDtqi8DYmofROXjNUEi+yrjUzoSUgNlY4aKFAGCGMG1Q45wyWuCXu C+o9PSfaFHfjxR9yTt21n4PiJCRJTtaI9+T4q8p5L+R3YkbY/WViULgngsLpVvbG2bNh uWjC77/j3dn1fmLT97d8e4koKdlc76tfYU00A6sdx8SF0/vEHv68TCFbRZG3Kts26mzA YzzxraC5z72a9bTVY87h48UFMnxXVGMdy1E0NY66Zzem8B4S1BSiFwfFvuWtssw9+ViJ BlZg== X-Gm-Message-State: APjAAAW8qDAblKnuUzX2m2pr1vpfT0VTKU4Os3X2VUVa8mgy3nTOC+vC tiNemVX7uvuiVRE99Elfn0ZN5kIoZWQ= X-Google-Smtp-Source: APXvYqxYt8YFRbjQsixG8YA7//Gby36xLjduEQuz154x7lFnYlvYQrZKVynLsIeyAK8G920kiIMwKg== X-Received: by 2002:a5d:522f:: with SMTP id i15mr3732254wra.257.1571250957116; Wed, 16 Oct 2019 11:35:57 -0700 (PDT) Received: from localhost.localdomain ([212.15.160.218]) by smtp.gmail.com with ESMTPSA id q10sm44940892wrd.39.2019.10.16.11.35.55 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Oct 2019 11:35:56 -0700 (PDT) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Wed, 16 Oct 2019 20:35:46 +0200 Message-Id: <20191016183546.24832-1-onemda@gmail.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH] avfilter: add tdisplace filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Paul B Mahol --- doc/filters.texi | 21 ++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_tdisplace.c | 234 +++++++++++++++++++++++++++++++++++++ 4 files changed, 257 insertions(+) create mode 100644 libavfilter/vf_tdisplace.c diff --git a/doc/filters.texi b/doc/filters.texi index 5cebb26cbd..5d3a36e49f 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -17233,6 +17233,27 @@ the position in the file of the input frame, NAN if unknown @section swapuv Swap U & V plane. +@section tdisplace + +Displace pixels temporally as indicated by second stream. + +It takes two input streams and outputs one stream, the first input is the +source, and second input in temporal displacement map. + +The second input specifies how much to displace pixels along the +time-axis. +If temporal displacement map stream terminates, last frame from that stream +will be used. + +This filter accepts the following options: + +@table @option +@item edge +Set temporal displacement radius. Any value present in temporal displacement +map which is higher/lower than this one will be clipped. +Allowed range is from 1 to 127. Default is 127. +@end table + @section telecine Apply telecine process to the video. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index e9ac54daee..a76347af45 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -394,6 +394,7 @@ OBJS-$(CONFIG_SUPER2XSAI_FILTER) += vf_super2xsai.o OBJS-$(CONFIG_SWAPRECT_FILTER) += vf_swaprect.o OBJS-$(CONFIG_SWAPUV_FILTER) += vf_swapuv.o OBJS-$(CONFIG_TBLEND_FILTER) += vf_blend.o framesync.o +OBJS-$(CONFIG_TDISPLACE_FILTER) += vf_tdisplace.o framesync.o OBJS-$(CONFIG_TELECINE_FILTER) += vf_telecine.o OBJS-$(CONFIG_THRESHOLD_FILTER) += vf_threshold.o framesync.o OBJS-$(CONFIG_THUMBNAIL_FILTER) += vf_thumbnail.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index a7a165e0d8..a04dcfbf5b 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -375,6 +375,7 @@ extern AVFilter ff_vf_super2xsai; extern AVFilter ff_vf_swaprect; extern AVFilter ff_vf_swapuv; extern AVFilter ff_vf_tblend; +extern AVFilter ff_vf_tdisplace; extern AVFilter ff_vf_telecine; extern AVFilter ff_vf_threshold; extern AVFilter ff_vf_thumbnail; diff --git a/libavfilter/vf_tdisplace.c b/libavfilter/vf_tdisplace.c new file mode 100644 index 0000000000..abf4beba11 --- /dev/null +++ b/libavfilter/vf_tdisplace.c @@ -0,0 +1,234 @@ +/* + * Copyright (c) 2019 Paul B Mahol + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "avfilter.h" +#include "filters.h" +#include "formats.h" +#include "framesync.h" +#include "internal.h" +#include "video.h" + +#define MAX_RADIUS 127 +#define MAX_FRAMES (MAX_RADIUS * 2 + 1) + +typedef struct TDisplaceContext { + const AVClass *class; + int radius; + + int max_frames; + + int planewidth[4], planeheight[4]; + int nb_planes; + + AVFrame *frames[MAX_FRAMES]; + int nb_frames; + + FFFrameSync fs; +} TDisplaceContext; + +#define OFFSET(x) offsetof(TDisplaceContext, x) +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM + +static const AVOption tdisplace_options[] = { + { "radius", "set temporal radius", OFFSET(radius), AV_OPT_TYPE_INT, {.i64=MAX_RADIUS}, 1, MAX_RADIUS, FLAGS }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(tdisplace); + +static int tdisplace_frame(FFFrameSync *fs) +{ + AVFilterContext *ctx = fs->parent; + AVFilterLink *outlink = ctx->outputs[0]; + TDisplaceContext *s = ctx->priv; + AVFrame *in, *tmap, *out; + int ret; + + ret = ff_framesync_dualinput_get(fs, &in, &tmap); + if (ret < 0) + return ret; + + if (s->nb_frames < s->max_frames) { + s->frames[s->nb_frames] = in; + if (s->frames[s->nb_frames]) + s->nb_frames++; + } else { + av_frame_free(&s->frames[0]); + memmove(&s->frames[0], &s->frames[1], (s->nb_frames - 1) * sizeof(s->frames[0])); + s->frames[s->max_frames - 1] = in; + if (!s->frames[s->max_frames - 1]) + s->nb_frames--; + } + + out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!out) + return AVERROR(ENOMEM); + av_frame_copy_props(out, in); + + for (int p = 0; p < s->nb_planes; p++) { + uint8_t *dst = out->data[p]; + const uint8_t *tsrc = tmap->data[p]; + const int tlinesize = tmap->linesize[p]; + const int dlinesize = out->linesize[p]; + const int half = (s->nb_frames + 1) >> 1; + + for (int y = 0; y < s->planeheight[p]; y++) { + for (int x = 0; x < s->planewidth[p]; x++) { + const int frame = tsrc[x] - s->max_frames / 2; + const int index = av_clip(half + frame, 0, s->nb_frames - 1); + const int linesize = s->frames[index]->linesize[p]; + + dst[x] = s->frames[index]->data[p][linesize * y + x]; + } + + tsrc += tlinesize; + dst += dlinesize; + } + } + + return ff_filter_frame(outlink, out); +} + +static av_cold int init(AVFilterContext *ctx) +{ + TDisplaceContext *s = ctx->priv; + + s->max_frames = s->radius * 2 + 1; + s->fs.on_event = tdisplace_frame; + + return 0; +} + +static int query_formats(AVFilterContext *ctx) +{ + static const enum AVPixelFormat pix_fmts[] = { + AV_PIX_FMT_YUVA444P, AV_PIX_FMT_YUV444P, AV_PIX_FMT_YUV440P, + AV_PIX_FMT_YUVJ444P, AV_PIX_FMT_YUVJ440P, + AV_PIX_FMT_YUVA422P, AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUVA420P, AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUVJ422P, AV_PIX_FMT_YUVJ420P, + AV_PIX_FMT_YUVJ411P, AV_PIX_FMT_YUV411P, AV_PIX_FMT_YUV410P, + AV_PIX_FMT_GBRP, AV_PIX_FMT_GBRAP, + AV_PIX_FMT_GRAY8, AV_PIX_FMT_NONE + }; + + return ff_set_common_formats(ctx, ff_make_format_list(pix_fmts)); +} + +static int config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + TDisplaceContext *s = ctx->priv; + AVFilterLink *srclink = ctx->inputs[0]; + AVFilterLink *tlink = ctx->inputs[1]; + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(outlink->format); + int ret, vsub, hsub; + + if (srclink->format != tlink->format) { + av_log(ctx, AV_LOG_ERROR, "inputs must be of same pixel format\n"); + return AVERROR(EINVAL); + } + if (srclink->w != tlink->w || + srclink->h != tlink->h) { + av_log(ctx, AV_LOG_ERROR, "First input link %s parameters " + "(size %dx%d) do not match the corresponding " + "second input link %s parameters (%dx%d)\n", + ctx->input_pads[0].name, srclink->w, srclink->h, + ctx->input_pads[1].name, tlink->w, tlink->h); + return AVERROR(EINVAL); + } + + outlink->w = srclink->w; + outlink->h = srclink->h; + outlink->time_base = srclink->time_base; + outlink->sample_aspect_ratio = srclink->sample_aspect_ratio; + outlink->frame_rate = srclink->frame_rate; + + s->nb_planes = av_pix_fmt_count_planes(outlink->format); + + hsub = desc->log2_chroma_w; + vsub = desc->log2_chroma_h; + + s->planeheight[1] = s->planeheight[2] = AV_CEIL_RSHIFT(outlink->h, vsub); + s->planeheight[0] = s->planeheight[3] = outlink->h; + s->planewidth[1] = s->planewidth[2] = AV_CEIL_RSHIFT(outlink->w, hsub); + s->planewidth[0] = s->planewidth[3] = outlink->w; + + if ((ret = ff_framesync_init_dualinput(&s->fs, ctx)) < 0) + return ret; + + ret = ff_framesync_configure(&s->fs); + outlink->time_base = s->fs.time_base; + + return ret; +} + +static int activate(AVFilterContext *ctx) +{ + TDisplaceContext *s = ctx->priv; + return ff_framesync_activate(&s->fs); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + TDisplaceContext *s = ctx->priv; + + ff_framesync_uninit(&s->fs); + + for (int i = 0; i < s->nb_frames; i++) { + av_frame_free(&s->frames[i]); + } + s->nb_frames = 0; +} + +static const AVFilterPad tdisplace_inputs[] = { + { + .name = "in", + .type = AVMEDIA_TYPE_VIDEO, + }, + { + .name = "tmap", + .type = AVMEDIA_TYPE_VIDEO, + }, + { NULL } +}; + +static const AVFilterPad tdisplace_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_output, + }, + { NULL } +}; + +AVFilter ff_vf_tdisplace = { + .name = "tdisplace", + .description = NULL_IF_CONFIG_SMALL("Temporal pixel displacement."), + .priv_size = sizeof(TDisplaceContext), + .priv_class = &tdisplace_class, + .query_formats = query_formats, + .init = init, + .uninit = uninit, + .activate = activate, + .inputs = tdisplace_inputs, + .outputs = tdisplace_outputs, +};