From patchwork Tue Nov 16 12:55:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thilo Borgmann X-Patchwork-Id: 31444 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:d206:0:0:0:0:0 with SMTP id q6csp7344285iob; Tue, 16 Nov 2021 04:56:07 -0800 (PST) X-Google-Smtp-Source: ABdhPJzEkTpYj+D7S2MBNzp9XHosYZL0ugxQnY2WBbSVb12QBVb4bZ438Xcm2RpZam8gaCYVt5/n X-Received: by 2002:a05:6402:1014:: with SMTP id c20mr9955854edu.186.1637067367370; Tue, 16 Nov 2021 04:56:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1637067367; cv=none; d=google.com; s=arc-20160816; b=UJSvvzEkC4XPSvUF4OtKmRQaQ6JPgvmSzonYVl6U3GeTDhwYE9cC+rVJYnp5h6hG18 HCaeDJ0z22M2OwzfZ7um/NeYX2IxPUI7JBhrr/TsZ49jhyxQtffwsbBHT+XCKhTtFpeD /5G+Edib1gsp3q7I2or1iOhKPSvsOq3hUB/FcYnI2m9wiucPq1vUfrQRf9EoOdBOvxAa yPl86zW+DCvX9Jmi77Qxa5/tUztoQ5OVbmvgkBaivZtFMFEm6ScAvMJaQ0EL/oWGoo+d zWsAW9qMubT3NZpeervvpJAmZdP7kQ6EyBTMSK1135coJwZM1AJ19onXd1XCsvsf+jPb dDNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject :content-language:mime-version:date:message-id:from:to :dkim-signature:delivered-to; bh=tgj1+57ilySPP+bI7nAVdGyOXUydGVMvsPeJRwW5CWg=; b=jk2OJxJgcfw4kZSpiH3hU/lPbM/OexO30kLxm62xpebvNxY+ZKrhb9fDjtGwpByNtj ovDoF4SlP6ryAH3GV7VPpt2CpMYW+RHav6UAs/FwD8xkNaETukXPQbkFpwOBFiIyKk6Y YYTSsYHSkegkszC7GFRFFOct4rPi0chcPZOS/ElpXjJZwd1xBmmE546aB3XWrYdig93a 2P5XTVJoHHQyK1gsBF6/69oy4ZOyM4heCWSh4oSTRKSu9/BRQI2KD3pf3aHNrbLBNN6n 6SH3mNc7KxlywTIaI1jgn5QjBcDPu3aynYaLWAZwyw4Hb9x8i258U0BbSgrwpvtg600S 4I8A== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@mail.de header.s=mailde202009 header.b=Tn8S2npF; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=mail.de Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id l13si28253019ejs.76.2021.11.16.04.56.06; Tue, 16 Nov 2021 04:56:07 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@mail.de header.s=mailde202009 header.b=Tn8S2npF; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=mail.de Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2BEE768A93D; Tue, 16 Nov 2021 14:56:04 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from shout01.mail.de (shout01.mail.de [62.201.172.24]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5245C687F96 for ; Tue, 16 Nov 2021 14:55:57 +0200 (EET) Received: from postfix01.mail.de (postfix01.bt.mail.de [10.0.121.125]) by shout01.mail.de (Postfix) with ESMTP id A748AA36CE for ; Tue, 16 Nov 2021 13:55:56 +0100 (CET) Received: from smtp02.mail.de (smtp02.bt.mail.de [10.0.121.212]) by postfix01.mail.de (Postfix) with ESMTP id 8CCB6801B8 for ; Tue, 16 Nov 2021 13:55:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mail.de; s=mailde202009; t=1637067356; bh=0v7t9NH1v0uTTCReR8UnmQ07epxbilOKiT7vcI2+YPU=; h=To:From:Subject:Date:From; b=Tn8S2npFKKDLktLLxJuljyawhHj9XiI3CP0JE8EsLbtD2K3KIvod9fQpNgNOa+vkq Hyc701XRDULa+XT2eWDQkTL9cedv8DvHE+6mrwtO4qvqN/gCXyMc5SwWgX6Ke9yP8h TCwdkQ6EIf6XHBbtCdBF0KIO1F8+mGYQaNICdXx82Qj9YLNEGnSbCWXj0vA/N7+9NU dF8sqRVzlJt65qwPL+VZc6Y2V7Wiai/muYECf4nmp23DWuiLftHoHcEnonO/8MmqBc 2tdDuHwYw8BXzwYTXCqE2jmo29+344nzHdpXuYiZZHRf6RPQts/mEJTajrnKL0D/UY k+32MDbfxcb7A== Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by smtp02.mail.de (Postfix) with ESMTPSA id 57756A073F for ; Tue, 16 Nov 2021 13:55:56 +0100 (CET) To: FFmpeg development discussions and patches From: Thilo Borgmann Message-ID: <0c6142b3-ab81-120a-f38a-a88c3dc66ac0@mail.de> Date: Tue, 16 Nov 2021 13:55:54 +0100 MIME-Version: 1.0 Content-Language: en-US X-purgate: clean X-purgate: This mail is considered clean (visit http://www.eleven.de for further information) X-purgate-type: clean X-purgate-Ad: Categorized by eleven eXpurgate (R) http://www.eleven.de X-purgate: This mail is considered clean (visit http://www.eleven.de for further information) X-purgate: clean X-purgate-size: 20176 X-purgate-ID: 154282::1637067356-00005767-3E645D9E/0/0 Subject: [FFmpeg-devel] [PATCH 1/2] lafi/vf_edgedetect: Move some functions to lavu/imgutils X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: tFHRuSyJ18BT Hi, shared code with vf_blurriness.c in patch 2/2. RFC: is there a better place to put these? -Thilo From b4e3701b2ed034b8d60aeff5d5ed75e4d4332a9f Mon Sep 17 00:00:00 2001 From: Thilo Borgmann Date: Tue, 16 Nov 2021 13:35:02 +0100 Subject: [PATCH 1/2] lafi/vf_edgedetect: Move some functions to lavu/imgutils --- libavfilter/vf_edgedetect.c | 127 +----------------------------------- libavutil/imgutils.c | 120 ++++++++++++++++++++++++++++++++++ libavutil/imgutils.h | 63 ++++++++++++++++++ 3 files changed, 186 insertions(+), 124 deletions(-) diff --git a/libavfilter/vf_edgedetect.c b/libavfilter/vf_edgedetect.c index 3eea34e325..9750ba5f98 100644 --- a/libavfilter/vf_edgedetect.c +++ b/libavfilter/vf_edgedetect.c @@ -188,127 +188,6 @@ static void gaussian_blur(AVFilterContext *ctx, int w, int h, memcpy(dst, src, w); } -enum { - DIRECTION_45UP, - DIRECTION_45DOWN, - DIRECTION_HORIZONTAL, - DIRECTION_VERTICAL, -}; - -static int get_rounded_direction(int gx, int gy) -{ - /* reference angles: - * tan( pi/8) = sqrt(2)-1 - * tan(3pi/8) = sqrt(2)+1 - * Gy/Gx is the tangent of the angle (theta), so Gy/Gx is compared against - * , or more simply Gy against *Gx - * - * Gx and Gy bounds = [-1020;1020], using 16-bit arithmetic: - * round((sqrt(2)-1) * (1<<16)) = 27146 - * round((sqrt(2)+1) * (1<<16)) = 158218 - */ - if (gx) { - int tanpi8gx, tan3pi8gx; - - if (gx < 0) - gx = -gx, gy = -gy; - gy *= (1 << 16); - tanpi8gx = 27146 * gx; - tan3pi8gx = 158218 * gx; - if (gy > -tan3pi8gx && gy < -tanpi8gx) return DIRECTION_45UP; - if (gy > -tanpi8gx && gy < tanpi8gx) return DIRECTION_HORIZONTAL; - if (gy > tanpi8gx && gy < tan3pi8gx) return DIRECTION_45DOWN; - } - return DIRECTION_VERTICAL; -} - -static void sobel(int w, int h, - uint16_t *dst, int dst_linesize, - int8_t *dir, int dir_linesize, - const uint8_t *src, int src_linesize) -{ - int i, j; - - for (j = 1; j < h - 1; j++) { - dst += dst_linesize; - dir += dir_linesize; - src += src_linesize; - for (i = 1; i < w - 1; i++) { - const int gx = - -1*src[-src_linesize + i-1] + 1*src[-src_linesize + i+1] - -2*src[ i-1] + 2*src[ i+1] - -1*src[ src_linesize + i-1] + 1*src[ src_linesize + i+1]; - const int gy = - -1*src[-src_linesize + i-1] + 1*src[ src_linesize + i-1] - -2*src[-src_linesize + i ] + 2*src[ src_linesize + i ] - -1*src[-src_linesize + i+1] + 1*src[ src_linesize + i+1]; - - dst[i] = FFABS(gx) + FFABS(gy); - dir[i] = get_rounded_direction(gx, gy); - } - } -} - -static void non_maximum_suppression(int w, int h, - uint8_t *dst, int dst_linesize, - const int8_t *dir, int dir_linesize, - const uint16_t *src, int src_linesize) -{ - int i, j; - -#define COPY_MAXIMA(ay, ax, by, bx) do { \ - if (src[i] > src[(ay)*src_linesize + i+(ax)] && \ - src[i] > src[(by)*src_linesize + i+(bx)]) \ - dst[i] = av_clip_uint8(src[i]); \ -} while (0) - - for (j = 1; j < h - 1; j++) { - dst += dst_linesize; - dir += dir_linesize; - src += src_linesize; - for (i = 1; i < w - 1; i++) { - switch (dir[i]) { - case DIRECTION_45UP: COPY_MAXIMA( 1, -1, -1, 1); break; - case DIRECTION_45DOWN: COPY_MAXIMA(-1, -1, 1, 1); break; - case DIRECTION_HORIZONTAL: COPY_MAXIMA( 0, -1, 0, 1); break; - case DIRECTION_VERTICAL: COPY_MAXIMA(-1, 0, 1, 0); break; - } - } - } -} - -static void double_threshold(int low, int high, int w, int h, - uint8_t *dst, int dst_linesize, - const uint8_t *src, int src_linesize) -{ - int i, j; - - for (j = 0; j < h; j++) { - for (i = 0; i < w; i++) { - if (src[i] > high) { - dst[i] = src[i]; - continue; - } - - if (!(!i || i == w - 1 || !j || j == h - 1) && - src[i] > low && - (src[-src_linesize + i-1] > high || - src[-src_linesize + i ] > high || - src[-src_linesize + i+1] > high || - src[ i-1] > high || - src[ i+1] > high || - src[ src_linesize + i-1] > high || - src[ src_linesize + i ] > high || - src[ src_linesize + i+1] > high)) - dst[i] = src[i]; - else - dst[i] = 0; - } - dst += dst_linesize; - src += src_linesize; - } -} - static void color_mix(int w, int h, uint8_t *dst, int dst_linesize, const uint8_t *src, int src_linesize) @@ -365,7 +244,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) in->data[p], in->linesize[p]); /* compute the 16-bits gradients and directions for the next step */ - sobel(width, height, + av_image_sobel(width, height, gradients, width, directions,width, tmpbuf, width); @@ -373,13 +252,13 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) /* non_maximum_suppression() will actually keep & clip what's necessary and * ignore the rest, so we need a clean output buffer */ memset(tmpbuf, 0, width * height); - non_maximum_suppression(width, height, + av_image_non_maximum_suppression(width, height, tmpbuf, width, directions,width, gradients, width); /* keep high values, or low values surrounded by high values */ - double_threshold(edgedetect->low_u8, edgedetect->high_u8, + av_image_double_threshold(edgedetect->low_u8, edgedetect->high_u8, width, height, out->data[p], out->linesize[p], tmpbuf, width); diff --git a/libavutil/imgutils.c b/libavutil/imgutils.c index 9ab5757cf6..fdb1331077 100644 --- a/libavutil/imgutils.c +++ b/libavutil/imgutils.c @@ -690,3 +690,123 @@ int av_image_fill_black(uint8_t *dst_data[4], const ptrdiff_t dst_linesize[4], return 0; } + +// Internal helper for av_image_sobel() +static int get_rounded_direction(int gx, int gy) +{ + /* reference angles: + * tan( pi/8) = sqrt(2)-1 + * tan(3pi/8) = sqrt(2)+1 + * Gy/Gx is the tangent of the angle (theta), so Gy/Gx is compared against + * , or more simply Gy against *Gx + * + * Gx and Gy bounds = [-1020;1020], using 16-bit arithmetic: + * round((sqrt(2)-1) * (1<<16)) = 27146 + * round((sqrt(2)+1) * (1<<16)) = 158218 + */ + if (gx) { + int tanpi8gx, tan3pi8gx; + + if (gx < 0) + gx = -gx, gy = -gy; + gy *= (1 << 16); + tanpi8gx = 27146 * gx; + tan3pi8gx = 158218 * gx; + if (gy > -tan3pi8gx && gy < -tanpi8gx) return DIRECTION_45UP; + if (gy > -tanpi8gx && gy < tanpi8gx) return DIRECTION_HORIZONTAL; + if (gy > tanpi8gx && gy < tan3pi8gx) return DIRECTION_45DOWN; + } + return DIRECTION_VERTICAL; +} + +// Simple sobel operator to get rounded gradients +void av_image_sobel(int w, int h, + uint16_t *dst, int dst_linesize, + int8_t *dir, int dir_linesize, + const uint8_t *src, int src_linesize) +{ + int i, j; + + for (j = 1; j < h - 1; j++) { + dst += dst_linesize; + dir += dir_linesize; + src += src_linesize; + for (i = 1; i < w - 1; i++) { + const int gx = + -1*src[-src_linesize + i-1] + 1*src[-src_linesize + i+1] + -2*src[ i-1] + 2*src[ i+1] + -1*src[ src_linesize + i-1] + 1*src[ src_linesize + i+1]; + const int gy = + -1*src[-src_linesize + i-1] + 1*src[ src_linesize + i-1] + -2*src[-src_linesize + i ] + 2*src[ src_linesize + i ] + -1*src[-src_linesize + i+1] + 1*src[ src_linesize + i+1]; + + dst[i] = FFABS(gx) + FFABS(gy); + dir[i] = get_rounded_direction(gx, gy); + } + } +} + +// Filters rounded gradients to drop all non-maxima +// Expects gradients generated by av_image_sobel() +// Expects zero's destination buffer +void av_image_non_maximum_suppression(int w, int h, + uint8_t *dst, int dst_linesize, + const int8_t *dir, int dir_linesize, + const uint16_t *src, int src_linesize) +{ + int i, j; + +#define COPY_MAXIMA(ay, ax, by, bx) do { \ + if (src[i] > src[(ay)*src_linesize + i+(ax)] && \ + src[i] > src[(by)*src_linesize + i+(bx)]) \ + dst[i] = av_clip_uint8(src[i]); \ +} while (0) + + for (j = 1; j < h - 1; j++) { + dst += dst_linesize; + dir += dir_linesize; + src += src_linesize; + for (i = 1; i < w - 1; i++) { + switch (dir[i]) { + case DIRECTION_45UP: COPY_MAXIMA( 1, -1, -1, 1); break; + case DIRECTION_45DOWN: COPY_MAXIMA(-1, -1, 1, 1); break; + case DIRECTION_HORIZONTAL: COPY_MAXIMA( 0, -1, 0, 1); break; + case DIRECTION_VERTICAL: COPY_MAXIMA(-1, 0, 1, 0); break; + } + } + } +} + +// Filter to keep all pixels > high, and keep all pixels > low where all surrounding pixels > high +void av_image_double_threshold(int low, int high, int w, int h, + uint8_t *dst, int dst_linesize, + const uint8_t *src, int src_linesize) +{ + int i, j; + + for (j = 0; j < h; j++) { + for (i = 0; i < w; i++) { + if (src[i] > high) { + dst[i] = src[i]; + continue; + } + + if (!(!i || i == w - 1 || !j || j == h - 1) && + src[i] > low && + (src[-src_linesize + i-1] > high || + src[-src_linesize + i ] > high || + src[-src_linesize + i+1] > high || + src[ i-1] > high || + src[ i+1] > high || + src[ src_linesize + i-1] > high || + src[ src_linesize + i ] > high || + src[ src_linesize + i+1] > high)) + dst[i] = src[i]; + else + dst[i] = 0; + } + dst += dst_linesize; + src += src_linesize; + } +} diff --git a/libavutil/imgutils.h b/libavutil/imgutils.h index cb2d74728e..a0514a67f0 100644 --- a/libavutil/imgutils.h +++ b/libavutil/imgutils.h @@ -301,6 +301,69 @@ int av_image_fill_black(uint8_t *dst_data[4], const ptrdiff_t dst_linesize[4], enum AVPixelFormat pix_fmt, enum AVColorRange range, int width, int height); +/** + * @brief Rounded directions used in av_image_sobel() + */ +enum AVRoundedDirection { + DIRECTION_45UP, + DIRECTION_45DOWN, + DIRECTION_HORIZONTAL, + DIRECTION_VERTICAL, +}; + +/** + * Simple sobel operator to get rounded gradients + * + * @param w the width of the image in pixels + * @param h the height of the image in pixels + * @param dst data pointers to magnitude image + * @param dst_linesize linesizes for the magnitude image + * @param dir data pointers to direction image + * @param dir_linesize linesizes for the direction image + * @param src data pointers to source image + * @param src_linesize linesizes for the source image + */ +void av_image_sobel(int w, int h, + uint16_t *dst, int dst_linesize, + int8_t *dir, int dir_linesize, + const uint8_t *src, int src_linesize); + +/** + * Filters rounded gradients to drop all non-maxima pixels in the magnitude image + * Expects gradients generated by av_image_sobel() + * Expects zero's in the destination buffer dst + * + * @param w the width of the image in pixels + * @param h the height of the image in pixels + * @param dst data pointers to magnitude image + * @param dst_linesize linesizes for the magnitude image + * @param dir data pointers to direction image + * @param dir_linesize linesizes for the direction image + * @param src data pointers to source image + * @param src_linesize linesizes for the source image + */ +void av_image_non_maximum_suppression(int w, int h, + uint8_t *dst, int dst_linesize, + const int8_t *dir, int dir_linesize, + const uint16_t *src, int src_linesize); + +/** + * Filters all pixels in src to keep all pixels > high, + * and keep all pixels > low where all surrounding pixels > high + * + * @param low the low threshold value + * @param high the hegh threshold value + * @param w the width of the image in pixels + * @param h the height of the image in pixels + * @param dst data pointers to destination image + * @param dst_linesize linesizes for the destination image + * @param src data pointers to source image + * @param src_linesize linesizes for the source image + */ +void av_image_double_threshold(int low, int high, int w, int h, + uint8_t *dst, int dst_linesize, + const uint8_t *src, int src_linesize); + /** * @} */