From patchwork Fri Feb 1 18:04:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul B Mahol X-Patchwork-Id: 11945 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 45F2A44576A for ; Fri, 1 Feb 2019 20:13:00 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2522B68A623; Fri, 1 Feb 2019 20:13:00 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5A901680806 for ; Fri, 1 Feb 2019 20:12:53 +0200 (EET) Received: by mail-wr1-f67.google.com with SMTP id q18so8056672wrx.9 for ; Fri, 01 Feb 2019 10:12:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id; bh=m4qKODBDXYlkFpTIcLVOgcLn2ZTtPxq90vv7sy3IrBA=; b=VpKJ34jxtm/PqlJtmNTHmYTBCrhqADGe2EAnMTGAJWR+rW2Gmcc+wBbfjZGVhPaBds P+/XcYC6YbgA7pIB71rcCDv21jgnk21za6TG/vuZheWWVYO+CIniY0uHWOtYdI/vh5VA 3AzLWz5ZIxnyb8KUDI6dsVnLePuqOAnOQGdvbMVWZMVGBGscO0xIgpnQMudqmBLw+7ta Q8Z+Ou4EazDpQh0WSyOuZJF/Uicts5XcpaUSegwgcTBYChbDzFZ0WAEyuwnJkeBu0dlN gAsWAedtwY1qLCfbeTxoB8SsXysAjg8WTPmQ4pOhKFP9SL9Dd98UkWOiK+/WeXS6KYFD IPWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=m4qKODBDXYlkFpTIcLVOgcLn2ZTtPxq90vv7sy3IrBA=; b=QXcwyu/ZjtWafrsc+/a/3icplFimRaFrOH5jZX78dJkJ7K6J/nMlwqbAkgspJfenN5 x79hfXb0iju5Xt6PTjlCSDkC/x6KM+VDe3IBOh3Qh6k/D/ocLPFCTFI0+XYbo/jmWbyv /4oGvTlN6zIlBDPO+Q69UsndZX7kn8bIDpdWMUqH8zWbsJcQOhp1gLCCplRjeIZFe1Kr ELfO61DaW23CcC82hSv6LAiz3kekm24sbHLBosLyJ1ikux4a1Qw+TsQ25B8pCv68tbiJ Krgaknrz/LwbSNY3aBmv9bQkXn+PvRvggNjhNb2bEs9jasYSMtKD7yuTQXx1CI2SiUJn ACXA== X-Gm-Message-State: AJcUukdbcqkjEh89zFnFKZg/xOqbvy2bpR8Nc1ILfmJUUQ/4LxzDgov1 8zSAbHQ4fartZt56LotfIQQNNwZj X-Google-Smtp-Source: ALg8bN6TmibTNFL1SsbbeTlF91w65uyzAIN1ivHcy3ZUpnn+PT4ERTRrrUalZSCjJ7uqO/SiSnOJuQ== X-Received: by 2002:a5d:4fcb:: with SMTP id h11mr40400021wrw.139.1549044269311; Fri, 01 Feb 2019 10:04:29 -0800 (PST) Received: from localhost.localdomain ([94.250.174.60]) by smtp.gmail.com with ESMTPSA id y138sm4343008wmc.16.2019.02.01.10.04.27 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Feb 2019 10:04:28 -0800 (PST) From: Paul B Mahol To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Feb 2019 19:04:18 +0100 Message-Id: <20190201180418.19551-1-onemda@gmail.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH] avfilter: add median filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Paul B Mahol --- doc/filters.texi | 15 +++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/median.h | 47 ++++++++ libavfilter/median_template.c | 152 +++++++++++++++++++++++ libavfilter/vf_median.c | 220 ++++++++++++++++++++++++++++++++++ 6 files changed, 436 insertions(+) create mode 100644 libavfilter/median.h create mode 100644 libavfilter/median_template.c create mode 100644 libavfilter/vf_median.c diff --git a/doc/filters.texi b/doc/filters.texi index fc98323af0..4dc5063613 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -11987,6 +11987,21 @@ Higher values should result in a smoother motion vector field but less optimal individual vectors. Default value is 1. @end table +@section median + +Pick median pixel from certain rectangle defined by radius. + +This filter accepts the following options: + +@table @option +@item radius +Set horizontal and vertical radius size. Default value is @code{1}. +Allowed range is integer from 1 to 127. + +@item planes +Set which planes to process. Default is @code{15}, which is all available planes. +@end table + @section mergeplanes Merge color channel components from several video streams. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index bc642acb01..c1587d962b 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -282,6 +282,7 @@ OBJS-$(CONFIG_MASKEDCLAMP_FILTER) += vf_maskedclamp.o framesync.o OBJS-$(CONFIG_MASKEDMERGE_FILTER) += vf_maskedmerge.o framesync.o OBJS-$(CONFIG_MASKFUN_FILTER) += vf_maskfun.o OBJS-$(CONFIG_MCDEINT_FILTER) += vf_mcdeint.o +OBJS-$(CONFIG_MEDIAN_FILTER) += vf_median.o OBJS-$(CONFIG_MERGEPLANES_FILTER) += vf_mergeplanes.o framesync.o OBJS-$(CONFIG_MESTIMATE_FILTER) += vf_mestimate.o motion_estimation.o OBJS-$(CONFIG_METADATA_FILTER) += f_metadata.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index c51ae0f3c7..1d69e12bf4 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -268,6 +268,7 @@ extern AVFilter ff_vf_maskedclamp; extern AVFilter ff_vf_maskedmerge; extern AVFilter ff_vf_maskfun; extern AVFilter ff_vf_mcdeint; +extern AVFilter ff_vf_median; extern AVFilter ff_vf_mergeplanes; extern AVFilter ff_vf_mestimate; extern AVFilter ff_vf_metadata; diff --git a/libavfilter/median.h b/libavfilter/median.h new file mode 100644 index 0000000000..cb682dabe6 --- /dev/null +++ b/libavfilter/median.h @@ -0,0 +1,47 @@ +/* + * Copyright (c) 2019 Paul B Mahol + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + * Redistribution and use in source and binary forms, with or without modification, + * are permitted provided that the following conditions are met: + */ + +#ifndef AVFILTER_MEDIAN_H +#define AVFILTER_MEDIAN_H + +#include "avfilter.h" + +typedef struct MedianContext { + const AVClass *class; + + int planes; + int radius; + + int planewidth[4]; + int planeheight[4]; + int depth; + int nb_planes; + + uint16_t *coarse, *fine; + int bins; + int t; + + void (*filter_plane)(AVFilterContext *ctx, uint8_t *ssrc, int src_linesize, + uint8_t *ddst, int dst_linesize, int width, int height); +} MedianContext; + +#endif diff --git a/libavfilter/median_template.c b/libavfilter/median_template.c new file mode 100644 index 0000000000..6a1091c111 --- /dev/null +++ b/libavfilter/median_template.c @@ -0,0 +1,152 @@ +/* + * Copyright (c) 2019 Paul B Mahol + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + * Redistribution and use in source and binary forms, with or without modification, + * are permitted provided that the following conditions are met: + */ + +#include "libavutil/avassert.h" +#include "avfilter.h" +#include "formats.h" +#include "internal.h" +#include "video.h" + +#undef pixel +#if DEPTH == 8 +#define pixel uint8_t +#else +#define pixel uint16_t +#endif + +#undef fn +#undef fn2 +#undef fn3 +#define SHIFT (DEPTH / 2) +#define BINS (1 << SHIFT) +#define MASK (BINS - 1) +#define fn3(a,b) a##_##b +#define fn2(a,b) fn3(a,b) +#define fn(a) fn2(a, DEPTH) + +#define PICK_COARSE_BIN(x, y) (BINS * (x) + ((y) >> SHIFT)) +#define PICK_FINE_BIN(x, y, z) (BINS * ((x) * ((y) >> SHIFT) + (z)) + ((y) & MASK)) + +static void fn(filter_plane)(AVFilterContext *ctx, uint8_t *ssrc, int src_linesize, + uint8_t *ddst, int dst_linesize, int width, int height) +{ + MedianContext *s = ctx->priv; + uint16_t coarse[BINS]; + uint16_t fine[BINS][BINS]; + uint16_t luc[BINS]; + const int radius = s->radius; + const int t = s->t; + const pixel *src = (const pixel *)ssrc; + pixel *dst = (pixel *)ddst; + const pixel *srcp = src; + const pixel *p, *q; + + src_linesize /= sizeof(pixel); + dst_linesize /= sizeof(pixel); + + memset(s->coarse, 0, BINS * width * sizeof(*s->coarse)); + memset(s->fine, 0, BINS * BINS * width * sizeof(*s->fine)); + + for (int i = 0; i < width; i++) { + s->coarse[PICK_COARSE_BIN(i, src[i])] += radius + 1; + s->fine[PICK_FINE_BIN(width, src[i], i)] += radius + 1; + } + + for (int i = 0; i < radius; i++) { + for (int j = 0; j < width; j++) { + s->coarse[PICK_COARSE_BIN(j, srcp[j])]++; + s->fine[PICK_FINE_BIN(width, srcp[j], j)]++; + } + srcp += src_linesize; + } + + srcp = src; + + for (int i = 0; i < height; i++) { + p = srcp + src_linesize * FFMAX(0, i - radius - 1); + q = p + width; + for (int j = 0; p != q; j++, p++) { + s->coarse[PICK_COARSE_BIN(j, *p)]--; + s->fine[PICK_FINE_BIN(width, *p, j)]--; + } + + p = srcp + src_linesize * FFMIN(height - 1, i + radius); + q = p + width; + for (int j = 0; p != q; j++, p++) { + s->coarse[PICK_COARSE_BIN(j, *p)]++; + s->fine[PICK_FINE_BIN(width, *p, j)]++; + } + + memset(coarse, 0, sizeof(coarse)); + memset(fine, 0, sizeof(fine)); + memset(luc, 0, sizeof(luc)); + hmuladd(coarse, &s->coarse[0], radius, BINS); + for (int j = 0; j < radius; j++) + hadd(coarse, &s->coarse[BINS * j], BINS); + for (int k = 0; k < BINS; k++) + hmuladd(&fine[k][0], &s->fine[BINS * width * k], 2 * radius + 1, BINS); + + for (int j = 0; j < width; j++) { + uint16_t sum = 0; + uint16_t *segment; + int k, b; + + hadd(coarse, &s->coarse[BINS * FFMIN(j + radius, width - 1)], BINS); + + for (k = 0; k < BINS; k++) { + sum += coarse[k]; + if (sum > t) { + sum -= coarse[k]; + break; + } + } + av_assert0(k < BINS); + + if (luc[k] <= j - radius) { + memset(&fine[k], 0, BINS * sizeof(uint16_t)); + for (luc[k] = j - radius; luc[k] < FFMIN(j + radius + 1, width); luc[k]++) + hadd(fine[k], &s->fine[BINS * (width * k + luc[k])], BINS); + if (luc[k] < j + radius + 1) { + hmuladd(&fine[k][0], &s->fine[BINS * (width * k + width - 1)], j + radius + 1 - width, BINS); + luc[k] = j + radius + 1; + } + } else { + for (; luc[k] < j + radius + 1; luc[k]++) { + hsub(fine[k], &s->fine[BINS * (width * k + FFMAX(luc[k] - 2 * radius - 1, 0))], BINS); + hadd(fine[k], &s->fine[BINS * (width * k + FFMIN(luc[k], width - 1))], BINS); + } + } + + hsub(coarse, &s->coarse[BINS * FFMAX(j - radius, 0)], BINS); + + segment = fine[k]; + for (b = 0; b < BINS; b++) { + sum += segment[b]; + if (sum > t) { + dst[dst_linesize * i + j] = BINS * k + b; + break; + } + } + av_assert0(b < BINS); + } + } +} diff --git a/libavfilter/vf_median.c b/libavfilter/vf_median.c new file mode 100644 index 0000000000..782cf0cfe6 --- /dev/null +++ b/libavfilter/vf_median.c @@ -0,0 +1,220 @@ +/* + * Copyright (c) 2019 Paul B Mahol + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + * Redistribution and use in source and binary forms, with or without modification, + * are permitted provided that the following conditions are met: + */ + +#include "libavutil/avassert.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "avfilter.h" +#include "formats.h" +#include "internal.h" +#include "median.h" +#include "video.h" + +static void hadd(uint16_t *dst, const uint16_t *src, int bins) +{ + for (int i = 0; i < bins; i++) + dst[i] += src[i]; +} + +static void hsub(uint16_t *dst, const uint16_t *src, int bins) +{ + for (int i = 0; i < bins; i++) + dst[i] -= src[i]; +} + +static void hmuladd(uint16_t *dst, const uint16_t *src, uint16_t f, int bins) +{ + for (int i = 0; i < bins; i++) + dst[i] += f * src[i]; +} + +#define DEPTH 8 +#include "median_template.c" + +#undef DEPTH +#define DEPTH 10 +#include "median_template.c" + +#undef DEPTH +#define DEPTH 12 +#include "median_template.c" + +#undef DEPTH +#define DEPTH 14 +#include "median_template.c" + +#undef DEPTH +#define DEPTH 16 +#include "median_template.c" + +#define OFFSET(x) offsetof(MedianContext, x) +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM + +static const AVOption median_options[] = { + { "radius", "set median radius", OFFSET(radius), AV_OPT_TYPE_INT, {.i64=1}, 1, 127, FLAGS }, + { "planes", "set planes to filter", OFFSET(planes), AV_OPT_TYPE_INT, {.i64=0xF}, 0, 0xF, FLAGS }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(median); + +static int query_formats(AVFilterContext *ctx) +{ + static const enum AVPixelFormat pix_fmts[] = { + AV_PIX_FMT_YUVA444P, AV_PIX_FMT_YUV444P, AV_PIX_FMT_YUV440P, + AV_PIX_FMT_YUVJ444P, AV_PIX_FMT_YUVJ440P, + AV_PIX_FMT_YUVA422P, AV_PIX_FMT_YUV422P, AV_PIX_FMT_YUVA420P, AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUVJ422P, AV_PIX_FMT_YUVJ420P, + AV_PIX_FMT_YUVJ411P, AV_PIX_FMT_YUV411P, AV_PIX_FMT_YUV410P, + AV_PIX_FMT_GBRP, AV_PIX_FMT_GBRAP, AV_PIX_FMT_GRAY8, + AV_PIX_FMT_YUV420P10, AV_PIX_FMT_YUV422P10, AV_PIX_FMT_YUV444P10, + AV_PIX_FMT_YUV420P12, AV_PIX_FMT_YUV422P12, AV_PIX_FMT_YUV444P12, AV_PIX_FMT_YUV440P12, + AV_PIX_FMT_YUV420P14, AV_PIX_FMT_YUV422P14, AV_PIX_FMT_YUV444P14, + AV_PIX_FMT_YUV420P16, AV_PIX_FMT_YUV422P16, AV_PIX_FMT_YUV444P16, + AV_PIX_FMT_YUVA420P10, AV_PIX_FMT_YUVA422P10, AV_PIX_FMT_YUVA444P10, + AV_PIX_FMT_YUVA420P16, AV_PIX_FMT_YUVA422P16, AV_PIX_FMT_YUVA444P16, + AV_PIX_FMT_GBRP10, AV_PIX_FMT_GBRP12, AV_PIX_FMT_GBRP14, AV_PIX_FMT_GBRP16, + AV_PIX_FMT_GBRAP10, AV_PIX_FMT_GBRAP12, AV_PIX_FMT_GBRAP16, + AV_PIX_FMT_GRAY10, AV_PIX_FMT_GRAY12, AV_PIX_FMT_GRAY14, AV_PIX_FMT_GRAY16, + AV_PIX_FMT_NONE + }; + + return ff_set_common_formats(ctx, ff_make_format_list(pix_fmts)); +} + +static int config_input(AVFilterLink *inlink) +{ + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(inlink->format); + MedianContext *s = inlink->dst->priv; + + s->depth = desc->comp[0].depth; + s->planewidth[1] = s->planewidth[2] = AV_CEIL_RSHIFT(inlink->w, desc->log2_chroma_w); + s->planewidth[0] = s->planewidth[3] = inlink->w; + s->planeheight[1] = s->planeheight[2] = AV_CEIL_RSHIFT(inlink->h, desc->log2_chroma_h); + s->planeheight[0] = s->planeheight[3] = inlink->h; + + s->nb_planes = av_pix_fmt_count_planes(inlink->format); + s->t = 2 * s->radius * s->radius + 2 * s->radius; + + for (int i = 0; i < s->nb_planes; i++) { + if (!(s->planes & (1 << i))) + continue; + + if (s->planewidth[i] < s->radius * 2 + 1) { + av_log(inlink->dst, AV_LOG_ERROR, "The %d plane width must be not less than %d\n", i, s->radius * 2 + 1); + return AVERROR(EINVAL); + } + + if (s->planeheight[i] < s->radius * 2 + 1) { + av_log(inlink->dst, AV_LOG_ERROR, "The %d plane height must be not less than %d\n", i, s->radius * 2 + 1); + return AVERROR(EINVAL); + } + } + + s->bins = 1 << (s->depth / 2); + s->coarse = av_malloc_array(s->bins * inlink->w, sizeof(*s->coarse)); + s->fine = av_malloc_array(s->bins * s->bins * inlink->w, sizeof(*s->fine)); + if (!s->coarse || !s->fine) + return AVERROR(ENOMEM); + + switch (s->depth) { + case 8: s->filter_plane = filter_plane_8; break; + case 10: s->filter_plane = filter_plane_10; break; + case 12: s->filter_plane = filter_plane_12; break; + case 14: s->filter_plane = filter_plane_14; break; + case 16: s->filter_plane = filter_plane_16; break; + } + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx = inlink->dst; + MedianContext *s = ctx->priv; + AVFilterLink *outlink = ctx->outputs[0]; + AVFrame *out; + int plane; + + out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!out) { + av_frame_free(&in); + return AVERROR(ENOMEM); + } + av_frame_copy_props(out, in); + + for (plane = 0; plane < s->nb_planes; plane++) { + if (!(s->planes & (1 << plane))) { + av_image_copy_plane(out->data[plane], out->linesize[plane], + in->data[plane], in->linesize[plane], + s->planewidth[plane] * ((s->depth + 7) / 8), + s->planeheight[plane]); + continue; + } + + s->filter_plane(ctx, in->data[plane], in->linesize[plane], + out->data[plane], out->linesize[plane], + s->planewidth[plane], s->planeheight[plane]); + } + + av_frame_free(&in); + return ff_filter_frame(outlink, out); +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + MedianContext *s = ctx->priv; + + av_freep(&s->coarse); + av_freep(&s->fine); +} + +static const AVFilterPad median_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_input, + .filter_frame = filter_frame, + }, + { NULL } +}; + +static const AVFilterPad median_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, + { NULL } +}; + +AVFilter ff_vf_median = { + .name = "median", + .description = NULL_IF_CONFIG_SMALL("Apply Median filter."), + .priv_size = sizeof(MedianContext), + .priv_class = &median_class, + .uninit = uninit, + .query_formats = query_formats, + .inputs = median_inputs, + .outputs = median_outputs, + .flags = AVFILTER_FLAG_SUPPORT_TIMELINE_GENERIC | AVFILTER_FLAG_SLICE_THREADS, +};