From patchwork Mon May 25 14:58:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 19861 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id AE53144AD20 for ; Mon, 25 May 2020 17:58:51 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8AD4D689AE3; Mon, 25 May 2020 17:58:51 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f47.google.com (mail-ej1-f47.google.com [209.85.218.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1F138687FA0 for ; Mon, 25 May 2020 17:58:45 +0300 (EEST) Received: by mail-ej1-f47.google.com with SMTP id h21so20753123ejq.5 for ; Mon, 25 May 2020 07:58:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=qF60EK2ZG6r+t5bb9mnzmWUl+uGGenk2DeyolTwErKU=; b=bpO7PW5Rv7rOryUEkZ8HD2s3/PsEfFHYW+PgvQs6owwtjoj61i1XB6BmrrGr/FfmVp QXeiWSSBUlbdVN8CDUvw6PeIhq6SX8M8ikBLuUWMuA/L18iXu1F8BQTHozn7xtA+eA8B EpF0XkCVmAUryvP2T6+ZC3+PL9hNHFTT8wmWdh5YVJzYHu7Z1isqRpVcmn5GvNMvS51u hNOJKtSOeXI9ZpFjE2zv8IHcQwVt4x+VXqkHRa9bDApTs+Tupnfu4vza7Mv9eLBFItfe UaJK9E4LqTIslQR180V8odXLTOMFRfCijp7AjCMvt8Xdl9OPwMWkpmr4kPTwzlner7tu Oxvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=qF60EK2ZG6r+t5bb9mnzmWUl+uGGenk2DeyolTwErKU=; b=p5xAV/Z/mKKURUjA55BHAA7J9WusIbH+OEJp51rjNkYVXXXPMzRvrowuE0Ocw41pNn Ud/Oq4Gjc4TZKLsJfhxh/1VYsRgNBJyGyLNw3Qgc8LfViGOurbwVEQaY4DhAW2LUmBPm e+1ht3w+78xzFEEHkSnjBq+boJDeHJSVKAPCo7TUcNoHo8HVukjqqPA/oK8Fs7Wzif5X /QqtboWYZRibivoiVKPSSI7i4TuuwOwTDUL7Ifeps1/ML+oE4a6/mowPYWRVDSLC5ONs RauduSggbVWNXTdOlc6YmIlK/Y2bijfMzfBxgudXP37mJOmM3zXwDCab+0GTp4wazMBt 9AhA== X-Gm-Message-State: AOAM531VtcA2VRobrLtFblQlaL9kibKG4ZABMILc+PxFZa43PLRcM1gC evuKzZd1K68Ct6zkHKJOo7naIz/24qU= X-Google-Smtp-Source: ABdhPJwrbqVfDkqQLM1f2n7t4i/9i/IfgseTLvTH94WKfQIl0txB5t2hZnJC2UCxNq3i0rI99UJBBg== X-Received: by 2002:a17:906:b348:: with SMTP id cd8mr18825705ejb.139.1590418724034; Mon, 25 May 2020 07:58:44 -0700 (PDT) Received: from localhost.localdomain ([37.244.239.188]) by smtp.gmail.com with ESMTPSA id qp13sm16017515ejb.8.2020.05.25.07.58.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 May 2020 07:58:43 -0700 (PDT) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Mon, 25 May 2020 16:58:31 +0200 Message-Id: <20200525145831.26292-1-onemda@gmail.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH] avfilter: add iir video filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Paul B Mahol --- doc/filters.texi | 18 ++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_iir.c | 410 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 430 insertions(+) create mode 100644 libavfilter/vf_iir.c diff --git a/doc/filters.texi b/doc/filters.texi index 773473f721..7675a0e9b1 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -12500,6 +12500,24 @@ further computations. This allows inserting the idet filter as a low computation method to clean up the interlaced flag @end table +@section iir +Apply 2D Infinite Impulse Response filter to input video stream. + +The filter accepts the following options: + +@table @option +@item hn +@item vn +Set horizontal and vertical numerator coefficients of transfer function. + +@item hd +@item vd +Set horizontal and vertical denominator coefficients of transfer function. + +@item planes +Set which planes to filter. Default is all. Allowed range is from 0 to 15. +@end table + @section il Deinterleave or interleave fields. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 994a4172a3..5344eb8ca9 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -281,6 +281,7 @@ OBJS-$(CONFIG_HWUPLOAD_CUDA_FILTER) += vf_hwupload_cuda.o OBJS-$(CONFIG_HWUPLOAD_FILTER) += vf_hwupload.o OBJS-$(CONFIG_HYSTERESIS_FILTER) += vf_hysteresis.o framesync.o OBJS-$(CONFIG_IDET_FILTER) += vf_idet.o +OBJS-$(CONFIG_IIR_FILTER) += vf_iir.o OBJS-$(CONFIG_IL_FILTER) += vf_il.o OBJS-$(CONFIG_INFLATE_FILTER) += vf_neighbor.o OBJS-$(CONFIG_INTERLACE_FILTER) += vf_tinterlace.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index f2a44b0090..f6ecf96b9b 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -266,6 +266,7 @@ extern AVFilter ff_vf_hwupload; extern AVFilter ff_vf_hwupload_cuda; extern AVFilter ff_vf_hysteresis; extern AVFilter ff_vf_idet; +extern AVFilter ff_vf_iir; extern AVFilter ff_vf_il; extern AVFilter ff_vf_inflate; extern AVFilter ff_vf_interlace; diff --git a/libavfilter/vf_iir.c b/libavfilter/vf_iir.c new file mode 100644 index 0000000000..193856adf3 --- /dev/null +++ b/libavfilter/vf_iir.c @@ -0,0 +1,410 @@ +/* + * Copyright (c) 2020 Paul B Mahol + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/avstring.h" +#include "avfilter.h" +#include "formats.h" +#include "internal.h" +#include "video.h" + +typedef struct IIRContext { + const AVClass *class; + + char *h_num_str; + char *h_den_str; + char *v_num_str; + char *v_den_str; + + int h_nb_num; + int h_nb_den; + int v_nb_num; + int v_nb_den; + + float *hnum; + float *hden; + float *vnum; + float *vden; + + float **honum; + float **hoden; + float **vonum; + float **voden; + + int planes; + int nb_planes; + int nb_threads; + + int planewidth[4]; + int planeheight[4]; +} IIRContext; + +#define OFFSET(x) offsetof(IIRContext, x) +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM + +static const AVOption iir_options[] = { + { "hn", "set horizontal numerator", OFFSET(h_num_str), AV_OPT_TYPE_STRING, {.str=0}, 0, 0, FLAGS }, + { "hd", "set horizotnal denominator", OFFSET(h_den_str), AV_OPT_TYPE_STRING, {.str=0}, 0, 0, FLAGS }, + { "vn", "set vertical numerator", OFFSET(v_num_str), AV_OPT_TYPE_STRING, {.str=0}, 0, 0, FLAGS }, + { "vd", "set vertical denominator", OFFSET(v_den_str), AV_OPT_TYPE_STRING, {.str=0}, 0, 0, FLAGS }, + { "planes", "set planes to filter", OFFSET(planes), AV_OPT_TYPE_INT, {.i64=0xF}, 0, 0xF, FLAGS }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(iir); + +typedef struct ThreadData { + int height; + int width; + + float *dst; + float *src; + int dst_linesize; + int src_linesize; +} ThreadData; + +static void horiz_slice(float *dst, float *src, + int dst_linesize, int src_linesize, + int nb_num, int nb_den, + const float *num, const float *den, + float *onum, float *oden, + int width, int height) +{ + for (int y = 0; y < height; y++) { + for (int i = 0; i < nb_num; i++) + onum[i] = src[0]; + for (int i = 0; i < nb_den; i++) + oden[i] = src[0]; + + for (int x = 0; x < width; x++) { + float sample = 0.f; + + memmove(&onum[1], &onum[0], (nb_num - 1) * sizeof(*onum)); + memmove(&oden[1], &oden[0], (nb_den - 1) * sizeof(*oden)); + onum[0] = src[x]; + for (int i = 0; i < nb_num; i++) + sample += num[i] * onum[i]; + + for (int i = 1; i < nb_den; i++) + sample -= den[i] * oden[i]; + + dst[x] = oden[0] = sample; + } + + dst += dst_linesize; + src += src_linesize; + } +} + +static int filter_horizontally(AVFilterContext *ctx, void *arg, int jobnr, int nb_jobs) +{ + IIRContext *s = ctx->priv; + ThreadData *td = arg; + const int height = td->height; + const int width = td->width; + const int slice_start = (height * jobnr ) / nb_jobs; + const int slice_end = (height * (jobnr+1)) / nb_jobs; + + horiz_slice(td->dst + td->dst_linesize * slice_start, + td->src + td->src_linesize * slice_start, + td->dst_linesize, td->src_linesize, + s->h_nb_num, s->h_nb_den, + s->hnum, s->hden, s->honum[jobnr], s->hoden[jobnr], + width, slice_end - slice_start); + + return 0; +} + +static void do_vertical_columns(float *pdst, float *psrc, + int dst_linesize, int src_linesize, + int nb_num, int nb_den, + const float *num, const float *den, + float *onum, float *oden, + int width, int height) +{ + for (int x = 0; x < width; x++) { + float *src = psrc; + float *dst = pdst; + + for (int i = 0; i < nb_num; i++) + onum[i] = src[x]; + for (int i = 0; i < nb_den; i++) + oden[i] = src[x]; + + for (int y = 0; y < height; y++) { + float sample = 0.f; + + memmove(&onum[1], &onum[0], (nb_num - 1) * sizeof(*onum)); + memmove(&oden[1], &oden[0], (nb_den - 1) * sizeof(*oden)); + onum[0] = src[x]; + for (int i = 0; i < nb_num; i++) + sample += num[i] * onum[i]; + + for (int i = 1; i < nb_den; i++) + sample -= den[i] * oden[i]; + + dst[x] = oden[0] = sample; + + dst += dst_linesize; + src += src_linesize; + } + } +} + +static int filter_vertically(AVFilterContext *ctx, void *arg, int jobnr, int nb_jobs) +{ + IIRContext *s = ctx->priv; + ThreadData *td = arg; + const int height = td->height; + const int width = td->width; + const int slice_start = (width * jobnr ) / nb_jobs; + const int slice_end = (width * (jobnr+1)) / nb_jobs; + + do_vertical_columns(td->dst + slice_start, + td->src + slice_start, + td->dst_linesize, td->src_linesize, + s->v_nb_num, s->v_nb_den, + s->vnum, s->vden, s->vonum[jobnr], s->voden[jobnr], + slice_end - slice_start, height); + + return 0; +} + +static void iir2d(AVFilterContext *ctx, AVFrame *out, AVFrame *in, int plane) +{ + IIRContext *s = ctx->priv; + const int width = s->planewidth[plane]; + const int height = s->planeheight[plane]; + ThreadData td; + + td.width = width; + td.height = height; + td.dst = (float *)out->data[plane]; + td.src = (float *)in->data[plane]; + td.dst_linesize = out->linesize[plane] / 4; + td.src_linesize = in->linesize[plane] / 4; + ctx->internal->execute(ctx, filter_horizontally, &td, NULL, FFMIN(height, s->nb_threads)); + td.src = (float *)out->data[plane]; + td.src_linesize = out->linesize[plane] / 4; + ctx->internal->execute(ctx, filter_vertically, &td, NULL, FFMIN(width, s->nb_threads)); +} + +static int query_formats(AVFilterContext *ctx) +{ + static const enum AVPixelFormat pix_fmts[] = { + AV_PIX_FMT_GBRPF32, AV_PIX_FMT_GBRAPF32, + AV_PIX_FMT_NONE + }; + + return ff_set_common_formats(ctx, ff_make_format_list(pix_fmts)); +} + +static void count_coefficients(char *item_str, int *nb_items) +{ + char *p; + + if (!item_str) + return; + + *nb_items = 1; + for (p = item_str; *p; p++) { + if (*p == ' ') + (*nb_items)++; + } +} + +static int read_coefficients(AVFilterContext *ctx, char *item_str, int nb_items, float *dst) +{ + char *p, *arg, *old_str, *saveptr = NULL; + int i; + + p = old_str = av_strdup(item_str); + if (!p) + return AVERROR(ENOMEM); + + for (i = 0; i < nb_items; i++) { + if (!(arg = av_strtok(p, " ", &saveptr))) + break; + + p = NULL; + if (av_sscanf(arg, "%f", &dst[i]) != 1) { + av_log(ctx, AV_LOG_ERROR, "Invalid coefficients supplied: %s\n", arg); + av_freep(&old_str); + return AVERROR(EINVAL); + } + } + + av_freep(&old_str); + + return 0; +} + +static int config_input(AVFilterLink *inlink) +{ + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(inlink->format); + AVFilterContext *ctx = inlink->dst; + IIRContext *s = ctx->priv; + int ret; + + s->nb_threads = ff_filter_get_nb_threads(ctx); + s->planewidth[1] = s->planewidth[2] = AV_CEIL_RSHIFT(inlink->w, desc->log2_chroma_w); + s->planewidth[0] = s->planewidth[3] = inlink->w; + s->planeheight[1] = s->planeheight[2] = AV_CEIL_RSHIFT(inlink->h, desc->log2_chroma_h); + s->planeheight[0] = s->planeheight[3] = inlink->h; + + s->nb_planes = av_pix_fmt_count_planes(inlink->format); + + count_coefficients(s->h_num_str, &s->h_nb_num); + count_coefficients(s->h_den_str, &s->h_nb_den); + count_coefficients(s->v_num_str, &s->v_nb_num); + count_coefficients(s->v_den_str, &s->v_nb_den); + + if (!s->h_nb_num || !s->h_nb_den || + !s->v_nb_num || !s->v_nb_den) + return AVERROR(EINVAL); + + s->hnum = av_calloc(s->h_nb_num, sizeof(*s->hnum)); + s->hden = av_calloc(s->h_nb_den, sizeof(*s->hden)); + s->honum = av_calloc(s->nb_threads, sizeof(*s->honum)); + s->hoden = av_calloc(s->nb_threads, sizeof(*s->hoden)); + s->vnum = av_calloc(s->h_nb_num, sizeof(*s->vnum)); + s->vden = av_calloc(s->h_nb_den, sizeof(*s->vden)); + s->vonum = av_calloc(s->nb_threads, sizeof(*s->vonum)); + s->voden = av_calloc(s->nb_threads, sizeof(*s->voden)); + if (!s->hnum || !s->hden || !s->honum || !s->hoden || + !s->vnum || !s->vden || !s->vonum || !s->voden) + return AVERROR(ENOMEM); + + for (int i = 0; i < s->nb_threads; i++) { + s->honum[i] = av_calloc(s->h_nb_num, sizeof(**s->honum)); + s->hoden[i] = av_calloc(s->h_nb_den, sizeof(**s->hoden)); + s->vonum[i] = av_calloc(s->v_nb_num, sizeof(**s->vonum)); + s->voden[i] = av_calloc(s->v_nb_den, sizeof(**s->voden)); + if (!s->honum[i] || !s->hoden[i] || + !s->vonum[i] || !s->voden[i]) + return AVERROR(ENOMEM); + } + + ret = read_coefficients(ctx, s->h_num_str, s->h_nb_num, s->hnum); + if (ret < 0) + return ret; + ret = read_coefficients(ctx, s->h_den_str, s->h_nb_den, s->hden); + if (ret < 0) + return ret; + ret = read_coefficients(ctx, s->v_num_str, s->v_nb_num, s->vnum); + if (ret < 0) + return ret; + ret = read_coefficients(ctx, s->v_den_str, s->v_nb_den, s->vden); + if (ret < 0) + return ret; + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx = inlink->dst; + IIRContext *s = ctx->priv; + AVFilterLink *outlink = ctx->outputs[0]; + AVFrame *out; + int plane; + + out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!out) { + av_frame_free(&in); + return AVERROR(ENOMEM); + } + av_frame_copy_props(out, in); + + for (plane = 0; plane < s->nb_planes; plane++) { + const int height = s->planeheight[plane]; + const int width = s->planewidth[plane]; + + if (!(s->planes & (1 << plane))) { + av_image_copy_plane(out->data[plane], out->linesize[plane], + in->data[plane], in->linesize[plane], + width * sizeof(float), height); + continue; + } + + iir2d(ctx, out, in, plane); + } + + av_frame_free(&in); + return ff_filter_frame(outlink, out); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + IIRContext *s = ctx->priv; + + av_freep(&s->hnum); + av_freep(&s->hden); + av_freep(&s->vnum); + av_freep(&s->vden); + + for (int i = 0; i < s->nb_threads; i++) { + if (s->honum) + av_freep(&s->honum[i]); + if (s->hoden) + av_freep(&s->hoden[i]); + if (s->vonum) + av_freep(&s->vonum[i]); + if (s->voden) + av_freep(&s->voden[i]); + } + + av_freep(&s->honum); + av_freep(&s->hoden); + av_freep(&s->vonum); + av_freep(&s->voden); +} + +static const AVFilterPad iir_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_input, + .filter_frame = filter_frame, + }, + { NULL } +}; + +static const AVFilterPad iir_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, + { NULL } +}; + +AVFilter ff_vf_iir = { + .name = "iir", + .description = NULL_IF_CONFIG_SMALL("Apply 2D Infinite Impulse Response filter with supplied coefficients."), + .priv_size = sizeof(IIRContext), + .priv_class = &iir_class, + .uninit = uninit, + .query_formats = query_formats, + .inputs = iir_inputs, + .outputs = iir_outputs, + .flags = AVFILTER_FLAG_SUPPORT_TIMELINE_GENERIC | AVFILTER_FLAG_SLICE_THREADS, +};