From patchwork Mon Dec 25 20:38:41 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 6963 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.79.195 with SMTP id r64csp6108121jad; Mon, 25 Dec 2017 12:39:33 -0800 (PST) X-Google-Smtp-Source: ACJfBotTaEF+Ek97bakwb0s2wioPDQPp3j5hzyEg0+9Li9IqtEOqQomPBUhRUvou8fL82OVDcDHp X-Received: by 10.28.57.11 with SMTP id g11mr20409601wma.92.1514234373387; Mon, 25 Dec 2017 12:39:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1514234373; cv=none; d=google.com; s=arc-20160816; b=YjtzXFjdRwoYQKP4Je4HHkW+uqT+meFSGscY/SXl/mqCoLqCzaZ+WIDQ72ji6WeUqr wEmmls4aAmx1mGED9rHisyGstVwEM38pR5raxiV9utC08a6iEhHobCPPGnyEOqHY/Qw5 oz0C6/L0X4gEXVJ3hxp45tgwD7D4ZYH+P59zRefX85AFYC2U/xk/Vg6WpmJfNShLfFEK 0qjmcSutHECStb70xZdGYYDiXkszRRY54G7ZW1cWb2eWRlp5PK025wLVLspKHrpvZ9j6 Q2ZdkhT59oBWsGo1ZQ5gd7Ay0R7WwRm9lyn4hc7FmyzqSMXaj+tCJ9Q/GPzKxNi6n16v EinQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=aR806eSlq4a/Pd6Hh7Yxj92enidyaPvJjjyL040rI1w=; b=v4syyjac/xi5g+Fqatq/Xstauineo0GjhDeD4SUzsGr49P7M9INf2ZjFXSsf4yn7LM sR4h6gPlllVsYe2t730hqG8L1z+50kox654tyC0ysHMrAKi+vFyh8Vp9IB6WFvQqJvkk mUrAOMCNWGY5RkQ7nLRkxL9K+J7j9/4Ma3LCxjsIj2qDTmUumy9USpqdlGl7lvvktxM2 3GtsEX1HLEqFTqt/CmWNo3sMSL+i3xc4C7xa1DcTaHACZV+i7SFP1t2BwO/S941lfMt6 0kdC21U7BlipyDs+QcoEY/VpolE/5kj9z5uh4tMboL8wiR29hsFkWQ5pbsN7W+YeIM9d XVQA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=dnuOi0g9; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 134si11297915wmn.234.2017.12.25.12.39.32; Mon, 25 Dec 2017 12:39:33 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=dnuOi0g9; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id AA2896805FF; Mon, 25 Dec 2017 22:39:17 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr0-f195.google.com (mail-wr0-f195.google.com [209.85.128.195]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0F9DE6805A1 for ; Mon, 25 Dec 2017 22:39:11 +0200 (EET) Received: by mail-wr0-f195.google.com with SMTP id p69so24553819wrb.8 for ; Mon, 25 Dec 2017 12:39:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=Okzs4BFVsXJ7gXJZgRoGPAueOt6/iN9fNJFIkhzHKcs=; b=dnuOi0g9CiydaAxF22XZnaib8hcm3bcU548Wz1KKlJl/auRDM/PCYye4B5J9GHU5OQ flLwB02ySFvq8PPTAKgbdRyoviyFVa3Sa6PodJlEDBUcQl81p/ap4qYgf7fSzseby6i9 nXrdADYnlnudY/Xsrcvnn5m3nRS5zgrOILszTPdnixDWpsRSm10R4eXsJzp7if/vbFgy eMeLDLF79t9jp5vxhA8PHyivu4cpOoscdIV/DJg4kNz/Xyv1qr7hjbJ2lRJgvpiXwQpe XiFR9THpsIlXjAWY1O60H4FbksyXGgkWEunT0x8tCcCVDY2E9nKJ7WNC8YOMetSE8uAe MXBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=Okzs4BFVsXJ7gXJZgRoGPAueOt6/iN9fNJFIkhzHKcs=; b=uAAwpZ25ZDVZgui6ZJkpvGUPshH+jShUEI55C8IEmoVLRyNVBMax3sTJOkQpOwOE1x JXqE23tFY4+5h6IEtGb2ZZEpg1lnzgb5FIaDg39iA/yjbcF6XsiE2kl3OQCjGUP6OfFn PF5TMpjxvVnVaHNkz6r3jOcRTeAnYoI2E+BC2jTVHuf9xUilWYrlRjdEBNivVSUXxWmU XUT4brv5/SzvzhEzUP/6AY5AFcaH+7hEyB7lAkSAmrZchvf+ZUdCRmVDIoN/muObKsKm f5kyf+J1mWBgBpHO3MOuZ8OGoto8EXsXieIAGv80bcguFOBmXSH3oqPzMmNZ6iSzIUjb S83g== X-Gm-Message-State: AKGB3mIAav7xSC8/bKWSQP6IkhIqKXmQ48RBTBBwKFxEZpSX5qySNiKK nSwOmzgNzJSuN6aZ/dxla9EXLw== X-Received: by 10.223.181.150 with SMTP id c22mr26248868wre.0.1514234363123; Mon, 25 Dec 2017 12:39:23 -0800 (PST) Received: from localhost.localdomain ([94.250.174.60]) by smtp.gmail.com with ESMTPSA id p90sm30334494wrb.28.2017.12.25.12.39.21 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 25 Dec 2017 12:39:22 -0800 (PST) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Mon, 25 Dec 2017 21:38:41 +0100 Message-Id: <20171225203841.30573-1-onemda@gmail.com> X-Mailer: git-send-email 2.11.0 Subject: [FFmpeg-devel] [PATCH] avfilter: add deconvolve filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Paul B Mahol --- doc/filters.texi | 22 +++++++++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_convolve.c | 99 ++++++++++++++++++++++++++++++++++++++++++----- 4 files changed, 113 insertions(+), 10 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 45515966e8..0220b7786a 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -6937,6 +6937,28 @@ Set whether or not chroma is considered in the metric calculations. Default is @code{1}. @end table +@section deconvolve + +Apply 2D deconvolution of video stream in frequency domain using second stream +as impulse. + +The filter accepts the following options: + +@table @option +@item planes +Set which planes to process. + +@item impulse +Set which impulse video frames will be processed, can be @var{first} +or @var{all}. Default is @var{all}. + +@item noise +Set noise when doing divisions. Default is @var{0}. Useful when width and height +are not same and not power of 2. +@end table + +The @code{deconvolve} filter also supports the @ref{framesync} options. + @section deflate Apply deflate effect to the video. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 6b06d57234..8bde542163 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -165,6 +165,7 @@ OBJS-$(CONFIG_DATASCOPE_FILTER) += vf_datascope.o OBJS-$(CONFIG_DCTDNOIZ_FILTER) += vf_dctdnoiz.o OBJS-$(CONFIG_DEBAND_FILTER) += vf_deband.o OBJS-$(CONFIG_DECIMATE_FILTER) += vf_decimate.o +OBJS-$(CONFIG_DECONVOLVE_FILTER) += vf_convolve.o framesync.o OBJS-$(CONFIG_DEFLATE_FILTER) += vf_neighbor.o OBJS-$(CONFIG_DEFLICKER_FILTER) += vf_deflicker.o OBJS-$(CONFIG_DEINTERLACE_QSV_FILTER) += vf_deinterlace_qsv.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 707faad777..67c073091f 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -175,6 +175,7 @@ static void register_all(void) REGISTER_FILTER(DCTDNOIZ, dctdnoiz, vf); REGISTER_FILTER(DEBAND, deband, vf); REGISTER_FILTER(DECIMATE, decimate, vf); + REGISTER_FILTER(DECONVOLVE, deconvolve, vf); REGISTER_FILTER(DEFLATE, deflate, vf); REGISTER_FILTER(DEFLICKER, deflicker, vf); REGISTER_FILTER(DEINTERLACE_QSV,deinterlace_qsv,vf); diff --git a/libavfilter/vf_convolve.c b/libavfilter/vf_convolve.c index 88ae884a19..1b96848b87 100644 --- a/libavfilter/vf_convolve.c +++ b/libavfilter/vf_convolve.c @@ -18,6 +18,8 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include + #include "libavutil/imgutils.h" #include "libavutil/opt.h" #include "libavutil/pixdesc.h" @@ -51,8 +53,11 @@ typedef struct ConvolveContext { int depth; int planes; int impulse; + float noise; int nb_planes; int got_impulse[4]; + + int (*filter)(AVFilterContext *ctx, void *arg, int jobnr, int nb_jobs); } ConvolveContext; #define OFFSET(x) offsetof(ConvolveContext, x) @@ -121,16 +126,16 @@ static int config_input_main(AVFilterLink *inlink) s->fft_bits[i] = fft_bits; s->fft_len[i] = 1 << s->fft_bits[i]; - if (!(s->fft_hdata[i] = av_calloc(s->fft_len[i] + 1, s->fft_len[i] * sizeof(FFTComplex)))) + if (!(s->fft_hdata[i] = av_calloc(s->fft_len[i], s->fft_len[i] * sizeof(FFTComplex)))) return AVERROR(ENOMEM); - if (!(s->fft_vdata[i] = av_calloc(s->fft_len[i] + 1, s->fft_len[i] * sizeof(FFTComplex)))) + if (!(s->fft_vdata[i] = av_calloc(s->fft_len[i], s->fft_len[i] * sizeof(FFTComplex)))) return AVERROR(ENOMEM); - if (!(s->fft_hdata_impulse[i] = av_calloc(s->fft_len[i] + 1, s->fft_len[i] * sizeof(FFTComplex)))) + if (!(s->fft_hdata_impulse[i] = av_calloc(s->fft_len[i], s->fft_len[i] * sizeof(FFTComplex)))) return AVERROR(ENOMEM); - if (!(s->fft_vdata_impulse[i] = av_calloc(s->fft_len[i] + 1, s->fft_len[i] * sizeof(FFTComplex)))) + if (!(s->fft_vdata_impulse[i] = av_calloc(s->fft_len[i], s->fft_len[i] * sizeof(FFTComplex)))) return AVERROR(ENOMEM); } @@ -166,7 +171,7 @@ static int fft_horizontal(AVFilterContext *ctx, void *arg, int jobnr, int nb_job FFTComplex *hdata = td->hdata; const int plane = td->plane; const int n = td->n; - int start = (n * jobnr ) / nb_jobs; + int start = (n * jobnr) / nb_jobs; int end = (n * (jobnr+1)) / nb_jobs; int y; @@ -261,7 +266,7 @@ static int fft_vertical(AVFilterContext *ctx, void *arg, int jobnr, int nb_jobs) FFTComplex *vdata = td->vdata; const int plane = td->plane; const int n = td->n; - int start = (n * jobnr ) / nb_jobs; + int start = (n * jobnr) / nb_jobs; int end = (n * (jobnr+1)) / nb_jobs; int y, x; @@ -286,7 +291,7 @@ static int ifft_vertical(AVFilterContext *ctx, void *arg, int jobnr, int nb_jobs FFTComplex *vdata = td->vdata; const int plane = td->plane; const int n = td->n; - int start = (n * jobnr ) / nb_jobs; + int start = (n * jobnr) / nb_jobs; int end = (n * (jobnr+1)) / nb_jobs; int y, x; @@ -310,7 +315,7 @@ static int ifft_horizontal(AVFilterContext *ctx, void *arg, int jobnr, int nb_jo FFTComplex *hdata = td->hdata; const int plane = td->plane; const int n = td->n; - int start = (n * jobnr ) / nb_jobs; + int start = (n * jobnr) / nb_jobs; int end = (n * (jobnr+1)) / nb_jobs; int y; @@ -383,7 +388,7 @@ static int complex_multiply(AVFilterContext *ctx, void *arg, int jobnr, int nb_j FFTComplex *input = td->hdata; FFTComplex *filter = td->vdata; const int n = td->n; - int start = (n * jobnr ) / nb_jobs; + int start = (n * jobnr) / nb_jobs; int end = (n * (jobnr+1)) / nb_jobs; int y, x; @@ -406,6 +411,38 @@ static int complex_multiply(AVFilterContext *ctx, void *arg, int jobnr, int nb_j return 0; } +static int complex_divide(AVFilterContext *ctx, void *arg, int jobnr, int nb_jobs) +{ + ConvolveContext *s = ctx->priv; + ThreadData *td = arg; + FFTComplex *input = td->hdata; + FFTComplex *filter = td->vdata; + const float noise = s->noise; + const int n = td->n; + int start = (n * jobnr) / nb_jobs; + int end = (n * (jobnr+1)) / nb_jobs; + int y, x; + + for (y = start; y < end; y++) { + int yn = y * n; + + for (x = 0; x < n; x++) { + FFTSample re, im, ire, iim, div; + + re = input[yn + x].re; + im = input[yn + x].im; + ire = filter[yn + x].re; + iim = filter[yn + x].im; + div = ire * ire + iim * iim + noise; + + input[yn + x].re = (ire * re + iim * im) / div; + input[yn + x].im = (ire * im - iim * re) / div; + } + } + + return 0; +} + static int do_convolve(FFFrameSync *fs) { AVFilterContext *ctx = fs->parent; @@ -474,7 +511,7 @@ static int do_convolve(FFFrameSync *fs) td.hdata = input; td.vdata = filter; - ctx->internal->execute(ctx, complex_multiply, &td, NULL, FFMIN3(MAX_THREADS, n, ff_filter_get_nb_threads(ctx))); + ctx->internal->execute(ctx, s->filter, &td, NULL, FFMIN3(MAX_THREADS, n, ff_filter_get_nb_threads(ctx))); td.hdata = s->fft_hdata[plane]; td.vdata = s->fft_vdata[plane]; @@ -525,6 +562,21 @@ static int activate(AVFilterContext *ctx) return ff_framesync_activate(&s->fs); } +static av_cold int init(AVFilterContext *ctx) +{ + ConvolveContext *s = ctx->priv; + + if (!strcmp(ctx->filter->name, "convolve")) { + s->filter = complex_multiply; + } else if (!strcmp(ctx->filter->name, "deconvolve")) { + s->filter = complex_divide; + } else { + return AVERROR_BUG; + } + + return 0; +} + static av_cold void uninit(AVFilterContext *ctx) { ConvolveContext *s = ctx->priv; @@ -571,6 +623,7 @@ AVFilter ff_vf_convolve = { .name = "convolve", .description = NULL_IF_CONFIG_SMALL("Convolve first video stream with second video stream."), .preinit = convolve_framesync_preinit, + .init = init, .uninit = uninit, .query_formats = query_formats, .activate = activate, @@ -580,3 +633,29 @@ AVFilter ff_vf_convolve = { .outputs = convolve_outputs, .flags = AVFILTER_FLAG_SUPPORT_TIMELINE_INTERNAL | AVFILTER_FLAG_SLICE_THREADS, }; + +static const AVOption deconvolve_options[] = { + { "planes", "set planes to deconvolve", OFFSET(planes), AV_OPT_TYPE_INT, {.i64=7}, 0, 15, FLAGS }, + { "impulse", "when to process impulses", OFFSET(impulse), AV_OPT_TYPE_INT, {.i64=1}, 0, 1, FLAGS, "impulse" }, + { "first", "process only first impulse, ignore rest", 0, AV_OPT_TYPE_CONST, {.i64=0}, 0, 0, FLAGS, "impulse" }, + { "all", "process all impulses", 0, AV_OPT_TYPE_CONST, {.i64=1}, 0, 0, FLAGS, "impulse" }, + { "noise", "set noise", OFFSET(noise), AV_OPT_TYPE_FLOAT, {.dbl=0}, 0, 1, FLAGS }, + { NULL }, +}; + +FRAMESYNC_DEFINE_CLASS(deconvolve, ConvolveContext, fs); + +AVFilter ff_vf_deconvolve = { + .name = "deconvolve", + .description = NULL_IF_CONFIG_SMALL("Deconvolve first video stream with second video stream."), + .preinit = deconvolve_framesync_preinit, + .init = init, + .uninit = uninit, + .query_formats = query_formats, + .activate = activate, + .priv_size = sizeof(ConvolveContext), + .priv_class = &deconvolve_class, + .inputs = convolve_inputs, + .outputs = convolve_outputs, + .flags = AVFILTER_FLAG_SUPPORT_TIMELINE_INTERNAL | AVFILTER_FLAG_SLICE_THREADS, +};