From patchwork Wed Dec 26 17:21:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Uwe Freese X-Patchwork-Id: 11555 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id D029A44D24F for ; Wed, 26 Dec 2018 19:21:13 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 81D2468B060; Wed, 26 Dec 2018 19:21:10 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.126.135]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C879568AEB7 for ; Wed, 26 Dec 2018 19:21:03 +0200 (EET) Received: from butler2.ufnet ([84.57.61.127]) by mrelayeu.kundenserver.de (mreue012 [212.227.15.167]) with ESMTPSA (Nemesis) id 1Mw9oq-1hU2RE2dLY-00s4sp for ; Wed, 26 Dec 2018 18:21:08 +0100 Received: from localhost ([127.0.0.1] helo=[IPv6:::1]) by butler2.ufnet with esmtp (Exim 4.89) (envelope-from ) id 1gcCrk-0005QC-0C for ffmpeg-devel@ffmpeg.org; Wed, 26 Dec 2018 18:21:08 +0100 To: ffmpeg-devel@ffmpeg.org References: <04cf53a1-0749-a4b5-1d1a-cef28c086a57@uwe-freese.de> <984b88c3-605e-a281-ec48-4b73956a4288@uwe-freese.de> <5f86c630-efe4-0b8b-9a68-8c624e522573@gmx.de> <5A696EAA-6AF5-4ED2-A977-97AFF0819A6A@gmail.com> <5258e8cb-7771-cf89-4e54-f60c37e3c2fd@gmx.de> From: Uwe Freese Message-ID: Date: Wed, 26 Dec 2018 18:21:07 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.3.0 MIME-Version: 1.0 In-Reply-To: <5258e8cb-7771-cf89-4e54-f60c37e3c2fd@gmx.de> Content-Language: en-US X-Provags-ID: V03:K1:jonxgBWPDmv5RDv2yC315E097njzVqMwxJfgVEVAyob3IdFI1rm QmF4SI5SX8EF9MxPJfzzCnzv5OQwsFyQZQhCznP4DqtY3bkI9gSClTNcvgErsjwt194WDjx duGMKIWmC0wbJkctbakuzLtc86WRywXukQecKNc6CFHWXTpoEBCVidTUWgEt6tYplkmhVAa 7Fa87GBVgPtkVIOhTJ5qQ== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1; V03:K0:6zsNRRSIj6M=:9dnC1HvtnlJzzUPYwR8DWR 8QA+klCW41939rh0acU8Mi2PC7YlSOWnkwqjwl0y+vAukwJKeiMh9BC+p05J/p1hr8FHK9Tie cZtKBP3XQxrwzaESjIBCukkLXQHVfnmbpKTyPRkpMgUZrNgrc5lBxxx63qWcY8UuNhXLUSz6D ty+5HXodtT7tiN5eZ69gA9PRm9qdgn3Dp7r300xUey90qFp2JuDcXay6FzFCpIrEublSYSiLq rYHruEdBrSq3G98TDpO0pwmcvggDj1NpyaZXKk1m1+2d4Y+npc29AKvI9YUhNC1/4qq22D8LN DpTv83uj9GU9Tf9Iqm5t/X3ndpmWPAhzOY7fuiyMUvtRvnPvA7+/sdof54pkOCT6c5Xl+yeVj lSHBI7HyPoz3IxZ8xRZZP4CgC8GoqrSVwlIwXObmcYnESQlL8IZ8JNwTI2kQrZGQR0Xwks3Us WFRBN903SLY9ca5dzgF397MWnF6Llsmz1vDsmRO1KV360EMVRb4avoMeRBdHCQ4eDdsVb4LAc xJ4+M9eTlXXLGVYTEB142BDwT08mtPh+acJ6TD79f0lSCaoumhlp+TnmO7rN4f4xWzCAGZLFC n/oA+uU2LgSJZmRJ0Mp3jOJBK82Lj0zC9PZFcC+kbI0/KEi5JNoIFuvklrJTYhO29agY+RGDc MBQoscW64vTn9W2fzod277LrI0f+KOvlg7FX4Rt97PlOBOKBDDfjhdgqRMSyE4tE5Sl+qS4au ZyFPNXU2lMrx5CIrBfVTNNMhKTtjDfgEijj08g== Subject: Re: [FFmpeg-devel] [PATCH] delogo filter: new "uglarm" interpolation mode added X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Hello, attached you can find a new patch, with no trailing spaces and avoiding additional indentation in existing code. Regards, Uwe Am 26.12.18 um 16:48 schrieb Uwe Freese: > Hello, >>> so now I've taken the time to integrate the alternative >>> interpolation algorithm 'uglarm' into the delogo filter. >> Please do not re-indent existing code in this patch as this makes >> reviewing the changes more difficult > > Parts of the previous code are now in an "if" statement block > (function apply_delogo), so indentation is correct. Even it would > maybe be better to read the diff, it wouldn't be correctly indented. > > In the parameter lists, spaces are added to have the same layout of > the descriptions in the right part for all lines. > > So what should I do in this case? > > I could create a diff of course with "wrong" indentation (not > correctly indented and not good to read in the final file, but the > diff is better to read...). Would this help for the first review? > >>   and please do not add trailing white space as it cannot be >> committed to our git repository. > > OK, I'll remove them, no problem. (I'll send a new patch to this list > after I know how to handle also the first point.) > >> (I did not check if you are possibly changing the license of code you >> did not write yourself, please be extra careful to make sure this >> does not happen.) > > The TimgFilterLogoaway.cpp from ffdshow also has GPL 2 and up, the > same as vf_delogo.c from ffmpeg. -> OK. > > https://sourceforge.net/p/ffdshow-tryout/code/HEAD/tree/trunk/src/imgFilters/TimgFilterLogoaway.cpp > > > Regards, > Uwe > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel From 221ecf222eb855303055a32bdc27ca8885217a2c Mon Sep 17 00:00:00 2001 From: breaker27 Date: Wed, 26 Dec 2018 18:16:48 +0100 Subject: [PATCH] Add new delogo interpolation mode uglarm. --- libavfilter/vf_delogo.c | 182 +++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 173 insertions(+), 9 deletions(-) diff --git a/libavfilter/vf_delogo.c b/libavfilter/vf_delogo.c index 065d093641..608cff2e4d 100644 --- a/libavfilter/vf_delogo.c +++ b/libavfilter/vf_delogo.c @@ -2,6 +2,7 @@ * Copyright (c) 2002 Jindrich Makovicka * Copyright (c) 2011 Stefano Sabatini * Copyright (c) 2013, 2015 Jean Delvare + * Copyright (c) 2019 Uwe Freese * * This file is part of FFmpeg. * @@ -25,6 +26,9 @@ * A very simple tv station logo remover * Originally imported from MPlayer libmpcodecs/vf_delogo.c, * the algorithm was later improved. + * The "UGLARM" mode was first implemented 2001 by Uwe Freese for Virtual + * Dub's LogoAway filter (by Krzysztof Wojdon), taken over into ffdshow's + * logoaway filter (by Milan Cutka), from where it was ported to ffmpeg. */ #include "libavutil/common.h" @@ -50,6 +54,10 @@ * @param logo_w width of the logo * @param logo_h height of the logo * @param band the size of the band around the processed area + * @param *uglarmtable pointer to weight table in UGLARM interpolation mode, + * zero when x-y mode is used + * @param *uglarmweightsum pointer to weight sum table in UGLARM interpolation mode, + * zero when x-y mode is used * @param show show a rectangle around the processed area, useful for * parameters tweaking * @param direct if non-zero perform in-place processing @@ -58,7 +66,8 @@ static void apply_delogo(uint8_t *dst, int dst_linesize, uint8_t *src, int src_linesize, int w, int h, AVRational sar, int logo_x, int logo_y, int logo_w, int logo_h, - unsigned int band, int show, int direct) + unsigned int band, double *uglarmtable, + double *uglarmweightsum, int show, int direct) { int x, y; uint64_t interp, weightl, weightr, weightt, weightb, weight; @@ -89,6 +98,7 @@ static void apply_delogo(uint8_t *dst, int dst_linesize, dst += (logo_y1 + 1) * dst_linesize; src += (logo_y1 + 1) * src_linesize; + if (!uglarmtable) { for (y = logo_y1+1; y < logo_y2; y++) { left_sample = topleft[src_linesize*(y-logo_y1)] + topleft[src_linesize*(y-logo_y1-1)] + @@ -151,12 +161,121 @@ static void apply_delogo(uint8_t *dst, int dst_linesize, dst += dst_linesize; src += src_linesize; } + } else { + int bx, by; + double interpd; + + for (y = logo_y1 + 1; y < logo_y2; y++) { + for (x = logo_x1 + 1, + xdst = dst + logo_x1 + 1, + xsrc = src + logo_x1 + 1; x < logo_x2; x++, xdst++, xsrc++) { + + if (show && (y == logo_y1 + 1 || y == logo_y2 - 1 || + x == logo_x1 + 1 || x == logo_x2 - 1)) { + *xdst = 0; + continue; + } + + interpd = 0; + + for (bx = 0; bx < logo_w; bx++) { + interpd += topleft[bx] * + uglarmtable[abs(bx - (x - logo_x1)) + (y - logo_y1) * (logo_w - 1)]; + interpd += botleft[bx] * + uglarmtable[abs(bx - (x - logo_x1)) + (logo_h - (y - logo_y1) - 1) * (logo_w - 1)]; + } + + for (by = 1; by < logo_h - 1; by++) { + interpd += topleft[by * src_linesize] * + uglarmtable[(x - logo_x1) + abs(by - (y - logo_y1)) * (logo_w - 1)]; + interpd += topleft[by * src_linesize + (logo_w - 1)] * + uglarmtable[logo_w - (x - logo_x1) - 1 + abs(by - (y - logo_y1)) * (logo_w - 1)]; + } + + interp = (uint64_t)(interpd / + uglarmweightsum[(x - logo_x1) - 1 + (y - logo_y1 - 1) * (logo_w - 2)]); + *xdst = interp; + } + + dst += dst_linesize; + src += src_linesize; + } + } +} + +/** + * Calculate the lookup tables to be used in UGLARM interpolation mode. + * + * @param *uglarmtable Pointer to table containing weigths for each possible + * diagonal distance between a border pixel and an inner + * logo pixel. + * @param *uglarmweightsum Pointer to a table containing the weight sum to divide + * by for each pixel within the logo area. + * @param sar The sar to take into account when calculating lookup + * tables. + * @param logo_w width of the logo + * @param logo_h height of the logo + * @param power power of uglarm interpolation + */ +static void calcUGLARMTables(double *uglarmtable, double *uglarmweightsum, + AVRational sar, int logo_w, int logo_h, int power) +{ + double e = 0.2 * power; + double aspect = (double)sar.num / sar.den; + int x, y; + + /* uglarmtable will contain a weigth for each possible diagonal distance + * between a border pixel and an inner logo pixel. The maximum distance in + * each direction between border and an inner pixel can be logo_w - 1. The + * weight of a border pixel which is x,y pixels away is stored at position + * x + y * (logo_w - 1). */ + for (y = 0; y < logo_h - 1; y++) + for (x = 0; x < logo_w - 1; x++) { + if (x + y != 0) { + double d = pow(sqrt((double)(x * x * aspect * aspect + y * y)), e); + uglarmtable[x + y * (logo_w - 1)] = 1.0 / d; + } else { + uglarmtable[x + y * (logo_w - 1)] = 1.0; + } + } + + /* uglarmweithsum will conatain the sum of all weigths which is used when + * an inner pixel of the logo at position x,y is calculated out of the + * border pixels. The aggregated value has to be divided by that. The value + * to use for the inner 1-based logo position x,y is stored at + * (x - 1) + (y - 1) * (logo_w - 2). */ + for (y = 1; y < logo_h - 1; y++) + for (x = 1; x < logo_w - 1; x++) { + double weightsum = 0; + + for (int bx = 0; bx < logo_w; bx++) { + /* top border */ + weightsum += uglarmtable[abs(bx - x) + y * (logo_w - 1)]; + /* bottom border */ + weightsum += uglarmtable[abs(bx - x) + (logo_h - y - 1) * (logo_w - 1)]; + } + + for (int by = 1; by < logo_h - 1; by++) { + /* left border */ + weightsum += uglarmtable[x + abs(by - y) * (logo_w - 1)]; + /* right border */ + weightsum += uglarmtable[(logo_w - x - 1) + abs(by - y) * (logo_w - 1)]; + } + + uglarmweightsum[(x - 1) + (y - 1) * (logo_w - 2)] = weightsum; + } } +enum mode { + MODE_XY, + MODE_UGLARM +}; + typedef struct DelogoContext { const AVClass *class; - int x, y, w, h, band, show; -} DelogoContext; + int x, y, w, h, band, mode, power, show; + double *uglarmtable[10], *uglarmweightsum[10]; +} DelogoContext; #define OFFSET(x) offsetof(DelogoContext, x) #define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM @@ -171,6 +290,10 @@ static const AVOption delogo_options[]= { { "band", "set delogo area band size", OFFSET(band), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, FLAGS }, { "t", "set delogo area band size", OFFSET(band), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, FLAGS }, #endif + {"mode", "set the interpolation mode", OFFSET(mode), AV_OPT_TYPE_INT, { .i64 = MODE_XY}, 0, 1, FLAGS, "mode"}, + {"xy", "use pixels in straight x any y direction", OFFSET(mode), AV_OPT_TYPE_CONST, { .i64 = MODE_XY}, 0, 0, FLAGS, "mode"}, + {"uglarm", "UGLARM mode, use full border", OFFSET(mode), AV_OPT_TYPE_CONST, { .i64 = MODE_UGLARM}, 0, 0, FLAGS, "mode"}, + { "power", "power of UGLARM interpolation", OFFSET(power), AV_OPT_TYPE_INT, { .i64 = 15 }, 0, 30, FLAGS }, { "show", "show delogo area", OFFSET(show), AV_OPT_TYPE_BOOL,{ .i64 = 0 }, 0, 1, FLAGS }, { NULL } }; @@ -215,8 +338,8 @@ static av_cold int init(AVFilterContext *ctx) #else s->band = 1; #endif - av_log(ctx, AV_LOG_VERBOSE, "x:%d y:%d, w:%d h:%d band:%d show:%d\n", - s->x, s->y, s->w, s->h, s->band, s->show); + av_log(ctx, AV_LOG_VERBOSE, "x:%d y:%d, w:%d h:%d band:%d mode:%d power:%d show:%d\n", + s->x, s->y, s->w, s->h, s->band, s->mode, s->power, s->show); s->w += s->band*2; s->h += s->band*2; @@ -226,6 +349,21 @@ static av_cold int init(AVFilterContext *ctx) return 0; } +static av_cold void uninit(AVFilterContext *ctx) +{ + DelogoContext *s = ctx->priv; + + if (s->mode == MODE_UGLARM) + { + for (int plane = 0; plane < 10; plane++) { + if (s->uglarmtable[plane]) + av_free(s->uglarmtable[plane]); + if (s->uglarmweightsum[plane]) + av_free(s->uglarmweightsum[plane]); + } + } +} + static int config_input(AVFilterLink *inlink) { DelogoContext *s = inlink->dst->priv; @@ -274,16 +412,41 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) int hsub = plane == 1 || plane == 2 ? hsub0 : 0; int vsub = plane == 1 || plane == 2 ? vsub0 : 0; + /* Up and left borders were rounded down, inject lost bits + * into width and height to avoid error accumulation */ + int logo_w = AV_CEIL_RSHIFT(s->w + (s->x & ((1<h + (s->y & ((1<mode == MODE_UGLARM) && (!s->uglarmtable[plane])) { + s->uglarmtable[plane] = + (double*)av_malloc((logo_w - 1) * (logo_h - 1) * sizeof(double)); + + if (!s->uglarmtable[plane]) { + return AVERROR(ENOMEM); + } + + s->uglarmweightsum[plane] = + (double*)av_malloc((logo_w - 2) * (logo_h - 2) * sizeof(double)); + + if (!s->uglarmweightsum[plane]) { + return AVERROR(ENOMEM); + } + + calcUGLARMTables(s->uglarmtable[plane], + s->uglarmweightsum[plane], + sar, logo_w, logo_h, s->power); + } + apply_delogo(out->data[plane], out->linesize[plane], in ->data[plane], in ->linesize[plane], AV_CEIL_RSHIFT(inlink->w, hsub), AV_CEIL_RSHIFT(inlink->h, vsub), sar, s->x>>hsub, s->y>>vsub, - /* Up and left borders were rounded down, inject lost bits - * into width and height to avoid error accumulation */ - AV_CEIL_RSHIFT(s->w + (s->x & ((1<h + (s->y & ((1<band>>FFMIN(hsub, vsub), + s->uglarmtable[plane], + s->uglarmweightsum[plane], s->show, direct); } @@ -317,6 +480,7 @@ AVFilter ff_vf_delogo = { .priv_size = sizeof(DelogoContext), .priv_class = &delogo_class, .init = init, + .uninit = uninit, .query_formats = query_formats, .inputs = avfilter_vf_delogo_inputs, .outputs = avfilter_vf_delogo_outputs, -- 2.11.0