From patchwork Mon May 10 13:42:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuewei Meng <928826483@qq.com> X-Patchwork-Id: 27709 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:b214:0:0:0:0:0 with SMTP id b20csp3016974iof; Mon, 10 May 2021 06:43:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxsZ4EsEwkJPHKOnkaDVv/K7aLj0sRd/X/RKmBQU6tH446jEQoqo3oLIYIqdXrDw3hbfa42 X-Received: by 2002:a17:906:5654:: with SMTP id v20mr25505221ejr.462.1620654180120; Mon, 10 May 2021 06:43:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1620654180; cv=none; d=google.com; s=arc-20160816; b=Urpu0bgK8Wcu4/NG530nf5jm5xReohIPKyISRCuEG4pR8U82owqGfJZgzu8IED14sr P4YE/AMvjlC57y0GiQRywd2pxhsVlCWVoAenFqcG5s7AHHMz6Ide+YTlZb5Ayp2rKDwR XIHPAqFtnysptkOfLhv1H1sfQK4FhSk+LwaO6mM7zqoZQhXXVfj228zuynBc/+dMzrZ4 FU3OumwLFxNRi5WgLIpnIqWOBNIhlfEIo8yZgZQ9LgixWAji4EFcEtn9wcbAIIYuwIrh fF28OzgvGcSa4bMfxCgd0WDvQMwjAst1dg6cTGkUiDmkzKSvMIjqV/CYlkqsECgsGZxj WQ7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:date:to:from:message-id:dkim-signature :delivered-to; bh=2GqTcejAIYI9+KfRWVyhveeKNsVRAPPs1VMxD3MFAKU=; b=FZ0wlaLPJFSmKyJoXPuiYsrLVe2ln1L/Tof2xIH9BciZAXTNqV5IqufSESgUntgWKj eJ/XG/sbd7yGkB5E8YbyscdZjp1tBRGLqc9Uuu6V65QXQe4JcK4XXEOqQkH2MHZ4SCIt aoLA3RHfaV4RrCMjSk0OEXy3nm4dZ15QSfcA7+nE/J4PiHVfrjRjgesYlMJhyuVgJDUQ A2XjOkIanaaerMoaiZjJ24yWULyff5LYs13qSYq7A1cGkmfuGS9sDlby6PyZ26RlXwob QScedLwL53FLZKjYN8HjMHFJ8tZDUHxOuuLGqDYY/h5MUcCt97P9P5tcdvcWfg458eu8 JRQQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@qq.com header.s=s201512 header.b=zhTpvFaJ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=qq.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id j14si13941854eja.369.2021.05.10.06.42.59; Mon, 10 May 2021 06:43:00 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@qq.com header.s=s201512 header.b=zhTpvFaJ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=qq.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6DCC9680B73; Mon, 10 May 2021 16:42:55 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from qq.com (out203-205-221-153.mail.qq.com [203.205.221.153]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 545FD68056B for ; Mon, 10 May 2021 16:42:46 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qq.com; s=s201512; t=1620654161; bh=82KiNY6KmGd2ofd30Ddxnj08piJkSg2bMFplCosp2LU=; h=From:To:Cc:Subject:Date; b=zhTpvFaJMAiOFIBiyuhGv65XVct3lyKXKRZw00cDuLSkSXhXNKuLCQudGwhT5ECL5 Yw/QBHL/SdYZ6juh60baWNrxic+BMNGqc4KK/hPQ8Md8dXAc8Wscd5ePywkV6Re+5t LInfc3IJQFkvSdd1ZPD2EXGjEQ04ulo468AVuco4= Received: from meng.localdomain ([162.105.23.172]) by newxmesmtplogicsvrszb6.qq.com (NewEsmtp) with SMTP id AA839480; Mon, 10 May 2021 21:42:40 +0800 X-QQ-mid: xmsmtpt1620654160taawb3h4k Message-ID: X-QQ-XMAILINFO: MvbeDoT2wgnLwDDD5mSCMTfvERKXz34J1ivfz5tI9CV3XfYVlnKU+YVAGNez0g 9+ZRjbzsTodba81kgIYErOPmNqSpDuxm1Q8ZTucZAKecMSjMo2K19aIY2ZNpanYgzPZNi34Cv3EU v+NoxrPo8iOC2bIJ+cKc3M94X0bllFoVE2GKnW38klE+ZAtSXuN8oHs8uhLO6FybWQZNMU00OxrS b16UAIsfxdpW25RfwrzLfT+u51BYQatps7Hor4u2A6id6E1OiQ9ljlV0MRihy3LzL3wcLONPTmPI e3c5rEjayjvWk7QAze/GP32SekLIaRWN1UoeooFaox4JIbc09jOJ50zwrhayoqAqrsft1DCGWslw c+XCoVerEblxmyEQshVVxhbptKKaranY6dLo1WbI+X5SbYwGvEYhfYBZpyKOkKPN3H9OqPj6njpe mr3+k1BoEmbojj1fp6eVRzrL7GQG/du3Osb2MqP85ubNvEPOYfQsSsCPti6EFwUKneweMOClrJqi Owb2J0kPeqH1HxnmWPPirlBAPeVZ7efjm/MGXlOhie1q0IJK6dIAoKtwYgvFo/KtYXYgBtKPeYmQ GkSe15f4OCstU5NqnB+n1Uo++8BmUc4GxtpoHE82doljyUDnMwAJVkgghV5Sr+OxDmsqITkZqcHB nD8GEZsI/ehZXWkvuVObj88flPNMQIklHHc/LeUk8nHlTy3HQ+pCIy1Srlre/YteeBJ9TpeIq6nr fozs7FYsvlWr8mCJAjFASfedSesZa3mgfl3/sqc3cGJdUfVXqwOeHs6TAo3P3oWSJOF7hwPG8txk AZ9oy1DeCPJSHUQR58UJDp7mB1Zoa3W6w= From: Xuewei Meng <928826483@qq.com> To: ffmpeg-devel@ffmpeg.org Date: Mon, 10 May 2021 21:42:31 +0800 X-OQ-MSGID: <1620654151-178-1-git-send-email-928826483@qq.com> X-Mailer: git-send-email 1.9.1 Subject: [FFmpeg-devel] [PATCH] GSoC: Support fast guided filter. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Xuewei Meng MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: TZl8Oe5r5zuo From: Xuewei Meng Two modes are supported in guided filter, basic mode and fast mode. Basic mode is the initial pushed guided filter without optimization. Fast mode is implemented based on the basic one by sub-sampling method. The sub-sampling ratio which can be defined by users controls the algorithm complexity. The larger the sub-sampling ratio, the lower the algorithm complexity. Signed-off-by: Xuewei Meng --- doc/filters.texi | 20 +++++++--- libavfilter/vf_guided.c | 104 ++++++++++++++++++++++++++++++++---------------- 2 files changed, 85 insertions(+), 39 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 03ca9ae..eb747cb 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -12963,12 +12963,22 @@ Apply guided filter for edge-preserving smoothing, dehazing and so on. The filter accepts the following options: @table @option @item radius -Set the radius in pixels. +Set the box radius in pixels. Allowed range is 1 to 20. Default is 3. @item eps -Set regularization parameter. -Allowed range is 0 to 1. Default is 0.1. +Set regularization parameter (with square). +Allowed range is 0 to 1. Default is 0.01. + +@item mode +Set filter mode. Can be @code{basic} or @code{fast}. +Default is @code{basic}. + +@item sub +Set subsampling ratio. +Allowed range is 1 to 64. +Default is always 1 for @code{basic} value of @var{mode} option, +and 4 for @code{fast} value of @var{mode} option. @item planes Set planes to filter. Default is first only. @@ -12987,8 +12997,8 @@ ffmpeg -i in.png -i in.png -filter_complex guided out.png @item Dehazing, structure-transferring filtering, detail enhancement with guided filter. -For the generation of guidance image, -see @url{http://kaiminghe.com/publications/pami12guidedfilter.pdf}. +For the generation of guidance image, refer to paper "Guided Image Filtering". +See: @url{http://kaiminghe.com/publications/pami12guidedfilter.pdf}. @example ffmpeg -i in.png -i guidance.png -filter_complex guided out.png @end example diff --git a/libavfilter/vf_guided.c b/libavfilter/vf_guided.c index 86c0db5..230fb7b 100644 --- a/libavfilter/vf_guided.c +++ b/libavfilter/vf_guided.c @@ -27,12 +27,20 @@ #include "internal.h" #include "video.h" +enum FilterModes { + BASIC, + FAST, + NB_MODES, +}; + typedef struct GuidedContext { const AVClass *class; FFFrameSync fs; int radius; float eps; + int mode; + int sub; int planes; @@ -51,9 +59,13 @@ typedef struct GuidedContext { #define FLAGS AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_RUNTIME_PARAM static const AVOption guided_options[] = { - { "radius", "set the box radius", OFFSET(radius), AV_OPT_TYPE_INT, {.i64=3 }, 1, 20, FLAGS }, - { "eps", "set the regularization parameter (with square)", OFFSET(eps), AV_OPT_TYPE_FLOAT, {.dbl=0.01 }, 0.0, 1, FLAGS }, - { "planes", "set planes to filter", OFFSET(planes), AV_OPT_TYPE_INT, {.i64=1 }, 0, 0xF, FLAGS }, + { "radius", "set the box radius", OFFSET(radius), AV_OPT_TYPE_INT, {.i64 = 3 }, 1, 20, FLAGS }, + { "eps", "set the regularization parameter (with square)", OFFSET(eps), AV_OPT_TYPE_FLOAT, {.dbl = 0.01 }, 0.0, 1, FLAGS }, + { "mode", "set filtering mode (0: basic mode; 1: fast mode)", OFFSET(mode), AV_OPT_TYPE_INT, {.i64 = BASIC}, 0, NB_MODES - 1, FLAGS, "mode" }, + { "basic", "basic guided filter", 0, AV_OPT_TYPE_CONST, {.i64 = BASIC}, 0, 0, FLAGS, "mode" }, + { "fast", "fast guided filter", 0, AV_OPT_TYPE_CONST, {.i64 = FAST }, 0, 0, FLAGS, "mode" }, + { "sub", "subsampling ratio", OFFSET(sub), AV_OPT_TYPE_INT, {.i64 = 1 }, 1, 64, FLAGS }, + { "planes", "set planes to filter", OFFSET(planes), AV_OPT_TYPE_INT, {.i64=1 }, 0, 0xF, FLAGS }, { NULL } }; @@ -147,6 +159,26 @@ static int config_input(AVFilterLink *inlink) return AVERROR(EINVAL); } + if (s->mode == BASIC) { + if (s->sub != 1) { + av_log(ctx, AV_LOG_WARNING, "Subsampling ratio is 1 in basic mode.\n"); + s->sub = 1; + } + } + else if (s->mode == FAST) { + if (s->sub == 1) { + av_log(ctx, AV_LOG_WARNING, "Subsampling ratio is larger than 1 in fast mode.\n"); + s->sub = 4; + } + if (s->radius >= s->sub) + s->radius = s->radius / s->sub; + else { + s->radius = 1; + } + } + else { + return AVERROR_BUG; + } s->depth = desc->comp[0].depth; s->width = ctx->inputs[0]->w; @@ -174,6 +206,10 @@ static int guided_##name(AVFilterContext *ctx, GuidedContext *s, const type *src = (const type *)ssrc; \ const type *srcRef = (const type *)ssrcRef; \ \ + int sub = s->sub; \ + int h = (height % sub) == 0 ? height / sub : height / sub + 1; \ + int w = (width % sub) == 0 ? width / sub : width / sub + 1; \ + \ ThreadData t; \ const int nb_threads = ff_filter_get_nb_threads(ctx); \ float *I; \ @@ -189,55 +225,55 @@ static int guided_##name(AVFilterContext *ctx, GuidedContext *s, float *meanA; \ float *meanB; \ \ - I = av_calloc(width * height, sizeof(float)); \ - II = av_calloc(width * height, sizeof(float)); \ - P = av_calloc(width * height, sizeof(float)); \ - IP = av_calloc(width * height, sizeof(float)); \ - meanI = av_calloc(width * height, sizeof(float)); \ - meanII = av_calloc(width * height, sizeof(float)); \ - meanP = av_calloc(width * height, sizeof(float)); \ - meanIP = av_calloc(width * height, sizeof(float)); \ + I = av_calloc(w * h, sizeof(float)); \ + II = av_calloc(w * h, sizeof(float)); \ + P = av_calloc(w * h, sizeof(float)); \ + IP = av_calloc(w * h, sizeof(float)); \ + meanI = av_calloc(w * h, sizeof(float)); \ + meanII = av_calloc(w * h, sizeof(float)); \ + meanP = av_calloc(w * h, sizeof(float)); \ + meanIP = av_calloc(w * h, sizeof(float)); \ \ - A = av_calloc(width * height, sizeof(float)); \ - B = av_calloc(width * height, sizeof(float)); \ - meanA = av_calloc(width * height, sizeof(float)); \ - meanB = av_calloc(width * height, sizeof(float)); \ + A = av_calloc(w * h, sizeof(float)); \ + B = av_calloc(w * h, sizeof(float)); \ + meanA = av_calloc(w * h, sizeof(float)); \ + meanB = av_calloc(w * h, sizeof(float)); \ \ if (!I || !II || !P || !IP || !meanI || !meanII || !meanP || \ !meanIP || !A || !B || !meanA || !meanB){ \ ret = AVERROR(ENOMEM); \ goto end; \ } \ - for (int i = 0;i < height;i++) { \ - for (int j = 0;j < width;j++) { \ - int x = i * width + j; \ - I[x] = src[i * src_stride + j] / maxval; \ + for (int i = 0;i < h;i++) { \ + for (int j = 0;j < w;j++) { \ + int x = i * w + j; \ + I[x] = src[(i * src_stride + j) * sub] / maxval; \ II[x] = I[x] * I[x]; \ - P[x] = srcRef[i * src_ref_stride + j] / maxval; \ + P[x] = srcRef[(i * src_ref_stride + j) * sub] / maxval; \ IP[x] = I[x] * P[x]; \ } \ } \ \ - t.width = width; \ - t.height = height; \ - t.srcStride = width; \ - t.dstStride = width; \ + t.width = w; \ + t.height = h; \ + t.srcStride = w; \ + t.dstStride = w; \ t.src = I; \ t.dst = meanI; \ - ctx->internal->execute(ctx, s->box_slice, &t, NULL, FFMIN(height, nb_threads)); \ + ctx->internal->execute(ctx, s->box_slice, &t, NULL, FFMIN(h, nb_threads)); \ t.src = II; \ t.dst = meanII; \ - ctx->internal->execute(ctx, s->box_slice, &t, NULL, FFMIN(height, nb_threads)); \ + ctx->internal->execute(ctx, s->box_slice, &t, NULL, FFMIN(h, nb_threads)); \ t.src = P; \ t.dst = meanP; \ - ctx->internal->execute(ctx, s->box_slice, &t, NULL, FFMIN(height, nb_threads)); \ + ctx->internal->execute(ctx, s->box_slice, &t, NULL, FFMIN(h, nb_threads)); \ t.src = IP; \ t.dst = meanIP; \ - ctx->internal->execute(ctx, s->box_slice, &t, NULL, FFMIN(height, nb_threads)); \ + ctx->internal->execute(ctx, s->box_slice, &t, NULL, FFMIN(h, nb_threads)); \ \ - for (int i = 0;i < height;i++) { \ - for (int j = 0;j < width;j++) { \ - int x = i * width + j; \ + for (int i = 0;i < h;i++) { \ + for (int j = 0;j < w;j++) { \ + int x = i * w + j; \ float varI = meanII[x] - (meanI[x] * meanI[x]); \ float covIP = meanIP[x] - (meanI[x] * meanP[x]); \ A[x] = covIP / (varI + eps); \ @@ -247,14 +283,14 @@ static int guided_##name(AVFilterContext *ctx, GuidedContext *s, \ t.src = A; \ t.dst = meanA; \ - ctx->internal->execute(ctx, s->box_slice, &t, NULL, FFMIN(height, nb_threads)); \ + ctx->internal->execute(ctx, s->box_slice, &t, NULL, FFMIN(h, nb_threads)); \ t.src = B; \ t.dst = meanB; \ - ctx->internal->execute(ctx, s->box_slice, &t, NULL, FFMIN(height, nb_threads)); \ + ctx->internal->execute(ctx, s->box_slice, &t, NULL, FFMIN(h, nb_threads)); \ \ for (int i = 0;i < height;i++) { \ for (int j = 0;j < width;j++) { \ - int x = i * width + j; \ + int x = i / sub * w + j / sub; \ dst[i * dst_stride + j] = meanA[x] * src[i * src_stride + j] + \ meanB[x] * maxval; \ } \