From patchwork Mon Jul 3 19:04:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Cox X-Patchwork-Id: 42405 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b1e:b0:12b:9ae3:586d with SMTP id c30csp4574497pzh; Mon, 3 Jul 2023 12:05:37 -0700 (PDT) X-Google-Smtp-Source: APBJJlH6FJG3OqJlRB+iQlCvc3KPlMfJMmjlZKvbxxH3RH3r7BJ3LkEMpzRdUhFkx7Uj2SZ1b4Em X-Received: by 2002:a17:906:4e06:b0:98d:dd76:7201 with SMTP id z6-20020a1709064e0600b0098ddd767201mr7821045eju.16.1688411137448; Mon, 03 Jul 2023 12:05:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688411137; cv=none; d=google.com; s=arc-20160816; b=rQ7B6xAV0V/fEck+8zkDF7pz9BCcGb3FKSHH/J4F2NhBkTUWJBKOkuaxkyianlZ3vt sy5XKUNB0O/nqimt09FgTmHoyEpAE02Bs1IrtkEyHKYVihZ6bcx90i3e5TFqEQRA4LT2 +roOzYEyWrLGv5/EyHj+z5YNrIE+7aZX57KPh4Jhn2JBkBi1R1/udITnR0ngcMEs7PyD yjdfEFn2zIZgWxQZqAInuq5nBhpDzELCQW2vGHpsSl7eWYm7rW3qy3417hyiX9nS4qFf 1EFnPgvkr0cBvMJnN7I/PpdgeMZ9PYalnwHHMQxtbrSuk4vsRs23oyxawpJptB8aFC/Q cs3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=KuC+g7Q4gtIKbHE9s36pEYQFhuZUF79rHO2rGBTCv70=; fh=2QQVLAqz5Dgp0O7PTQ7hb1i3rOEvtuxkp5BnHStC38U=; b=IIKRlDKfe/OUP7nlw41yFU4AuDDZxb4Us4Df/1omS6RCrEPXdv/AAoCEtqT8qu/CgV 8gLKlMx84PhK5EpF55AeJjYYSgEub58pVU8PnBFYxgnmPSRxcQD0ltbj7k10Rfwg/eOI lbCOpr813WXlj+/tccNPyupQy9aF5wrRTXzbmi8BrO6RADPO5cIMXxVsT2fFjvEPYpyE Igo0EBkSVbAhD/tdy8rSOUm+66GGvujH39EXtHddqUshQWREcPHKHeIIotKxubc7Vn6W xdg8VkJEDPtI+/1+d/jrDFogfQub/hqmqwyOYt92cRU0hIM+iUpR3Wp7YJhOREHbK+Cz 2/pg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@kynesim-co-uk.20221208.gappssmtp.com header.s=20221208 header.b=KZA0cRr7; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id pk21-20020a170906d7b500b00992ac8614b7si2741123ejb.69.2023.07.03.12.05.37; Mon, 03 Jul 2023 12:05:37 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@kynesim-co-uk.20221208.gappssmtp.com header.s=20221208 header.b=KZA0cRr7; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 59AFF68C59A; Mon, 3 Jul 2023 22:04:47 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 303F868C57D for ; Mon, 3 Jul 2023 22:04:38 +0300 (EEST) Received: by mail-wr1-f45.google.com with SMTP id ffacd0b85a97d-31438512cafso1283698f8f.2 for ; Mon, 03 Jul 2023 12:04:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kynesim-co-uk.20221208.gappssmtp.com; s=20221208; t=1688411077; x=1691003077; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=H687VJhKJFwkKLWCcK28jqX8tG6UW/1CdnXuKPDp1d0=; b=KZA0cRr7EeuQMMEa1xm59/R0AvTaxHmp2STkAAus6iKj3x5rhzYD+hbnkeUzf9mC2q UUPYapeApP5Q/pHVmzNVcLs5ULWN5ETZ2yJNMbzw8dqleOC90QSHa+xsMTh84fJhBA+C W/vdYsfEkYw6+Dt6FHbgJTSmo4JQQ1+tLnZf+9/o2AelfHF7EU7FbZs47W3Bi11YxNbd JIEcA/yxm/V8NYZooFqRP5dsMuIYnu3czLdePHyVxiXXjNkHP65dzTkacM6FU8q+4zoP 61XS1pguJmB1FdwSKRqxPlZLxawcpP5Nh5XwK8U2F0zgpAOuMqf05s5zN9jujp+B8V/N OU1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688411077; x=1691003077; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=H687VJhKJFwkKLWCcK28jqX8tG6UW/1CdnXuKPDp1d0=; b=FhFgkaAmOgp2+yhjFk+SLe53qRunVN1a2l9tX86WsaRTz5cAyn6Scnk1vIReKuFvYi Fe8C4RaV3LoFQVXTX2hfagqMRU57cXTrkK8zZ2jQjnreIcNODWaTZTk7pkID2eaF2q31 UaQKoOVwF4KNIGQkRKa5zfp2jPM+dQOBfTT91Yw5UsCkDZBImAuZcWoh42kEQ5twoJZy Ak8+8tIDq9FI0AB3ivztGPfxvG+eYeU6FnQFNQ587LW9vYmdqdrVZBFJaP0CwmHb9J+e MrAom77dTkXOs6FIJL9xOfQEXA5t//e/KfvEtRigzFDrjw6TCGWi/aXZ0wuxHncBvuH4 DdYA== X-Gm-Message-State: ABy/qLY5aguGqQ5lrlrSB0jaZkl4KXZZaK3GogXjrgyiFHezTmh83mBI NAmWxJLYcZcMRsm+/1AyylOG+ILvDTRGVwPXx5A= X-Received: by 2002:a5d:68c2:0:b0:314:336e:d4d8 with SMTP id p2-20020a5d68c2000000b00314336ed4d8mr4459790wrw.8.1688411077638; Mon, 03 Jul 2023 12:04:37 -0700 (PDT) Received: from sucnaath.outer.uphall.net (cpc1-cmbg20-2-0-cust759.5-4.cable.virginm.net. [86.21.218.248]) by smtp.gmail.com with ESMTPSA id o18-20020a056000011200b00313f9085119sm18972035wrx.113.2023.07.03.12.04.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Jul 2023 12:04:37 -0700 (PDT) From: John Cox To: ffmpeg-devel@ffmpeg.org Date: Mon, 3 Jul 2023 19:04:09 +0000 Message-Id: <20230703190410.237473-7-jc@kynesim.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230703190410.237473-1-jc@kynesim.co.uk> References: <20230703190410.237473-1-jc@kynesim.co.uk> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 6/7] avfilter/vf_bwdif: Add a filter_line3 method for optimisation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: thomas.mundt@hr.de, John Cox , martin@martin.st Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: zLQEHic4cH2O Add an optional filter_line3 to the available optimisations. filter_line3 is equivalent to filter_line, memcpy, filter_line filter_line shares quite a number of loads and some calculations in common with its next iteration and testing shows that using aarch64 neon filter_line3s performance is 30% better than two filter_lines and a memcpy. Adds a test for vf_bwdif filter_line3 to checkasm Rounds job start lines down to a multiple of 4. This means that if filter_line3 exists then filter_line will not sometimes be called once at the end of a slice depending on thread count. The final slice may do up to 3 extra lines but filter_edge is faster than filter_line so it is unlikely to create any noticable thread load variation. Signed-off-by: John Cox --- libavfilter/bwdif.h | 7 ++++ libavfilter/vf_bwdif.c | 44 +++++++++++++++++++-- tests/checkasm/vf_bwdif.c | 81 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 129 insertions(+), 3 deletions(-) diff --git a/libavfilter/bwdif.h b/libavfilter/bwdif.h index cce99953f3..496cec72ef 100644 --- a/libavfilter/bwdif.h +++ b/libavfilter/bwdif.h @@ -35,6 +35,9 @@ typedef struct BWDIFContext { void (*filter_edge)(void *dst, void *prev, void *cur, void *next, int w, int prefs, int mrefs, int prefs2, int mrefs2, int parity, int clip_max, int spat); + void (*filter_line3)(void *dst, int dstride, + const void *prev, const void *cur, const void *next, int prefs, + int w, int parity, int clip_max); } BWDIFContext; void ff_bwdif_init_filter_line(BWDIFContext *bwdif, int bit_depth); @@ -53,4 +56,8 @@ void ff_bwdif_filter_line_c(void *dst1, void *prev1, void *cur1, void *next1, int prefs3, int mrefs3, int prefs4, int mrefs4, int parity, int clip_max); +void ff_bwdif_filter_line3_c(void * dst1, int d_stride, + const void * prev1, const void * cur1, const void * next1, int s_stride, + int w, int parity, int clip_max); + #endif /* AVFILTER_BWDIF_H */ diff --git a/libavfilter/vf_bwdif.c b/libavfilter/vf_bwdif.c index 26349da1fd..6701208efe 100644 --- a/libavfilter/vf_bwdif.c +++ b/libavfilter/vf_bwdif.c @@ -150,6 +150,31 @@ void ff_bwdif_filter_line_c(void *dst1, void *prev1, void *cur1, void *next1, FILTER2() } +#define NEXT_LINE()\ + dst += d_stride; \ + prev += prefs; \ + cur += prefs; \ + next += prefs; + +void ff_bwdif_filter_line3_c(void * dst1, int d_stride, + const void * prev1, const void * cur1, const void * next1, int s_stride, + int w, int parity, int clip_max) +{ + const int prefs = s_stride; + uint8_t * dst = dst1; + const uint8_t * prev = prev1; + const uint8_t * cur = cur1; + const uint8_t * next = next1; + + ff_bwdif_filter_line_c(dst, (void*)prev, (void*)cur, (void*)next, w, + prefs, -prefs, prefs * 2, - prefs * 2, prefs * 3, -prefs * 3, prefs * 4, -prefs * 4, parity, clip_max); + NEXT_LINE(); + memcpy(dst, cur, w); + NEXT_LINE(); + ff_bwdif_filter_line_c(dst, (void*)prev, (void*)cur, (void*)next, w, + prefs, -prefs, prefs * 2, - prefs * 2, prefs * 3, -prefs * 3, prefs * 4, -prefs * 4, parity, clip_max); +} + void ff_bwdif_filter_edge_c(void *dst1, void *prev1, void *cur1, void *next1, int w, int prefs, int mrefs, int prefs2, int mrefs2, int parity, int clip_max, int spat) @@ -212,6 +237,13 @@ static void filter_edge_16bit(void *dst1, void *prev1, void *cur1, void *next1, FILTER2() } +// Round job start line down to multiple of 4 so that if filter_line3 exists +// and the frame is a multiple of 4 high then filter_line will never be called +static inline int job_start(const int jobnr, const int nb_jobs, const int h) +{ + return jobnr >= nb_jobs ? h : ((h * jobnr) / nb_jobs) & ~3; +} + static int filter_slice(AVFilterContext *ctx, void *arg, int jobnr, int nb_jobs) { BWDIFContext *s = ctx->priv; @@ -221,8 +253,8 @@ static int filter_slice(AVFilterContext *ctx, void *arg, int jobnr, int nb_jobs) int clip_max = (1 << (yadif->csp->comp[td->plane].depth)) - 1; int df = (yadif->csp->comp[td->plane].depth + 7) / 8; int refs = linesize / df; - int slice_start = (td->h * jobnr ) / nb_jobs; - int slice_end = (td->h * (jobnr+1)) / nb_jobs; + int slice_start = job_start(jobnr, nb_jobs, td->h); + int slice_end = job_start(jobnr + 1, nb_jobs, td->h); int y; for (y = slice_start; y < slice_end; y++) { @@ -244,6 +276,11 @@ static int filter_slice(AVFilterContext *ctx, void *arg, int jobnr, int nb_jobs) refs << 1, -(refs << 1), td->parity ^ td->tff, clip_max, (y < 2) || ((y + 3) > td->h) ? 0 : 1); + } else if (s->filter_line3 && y + 2 < slice_end && y + 6 < td->h) { + s->filter_line3(dst, td->frame->linesize[td->plane], + prev, cur, next, linesize, td->w, + td->parity ^ td->tff, clip_max); + y += 2; } else { s->filter_line(dst, prev, cur, next, td->w, refs, -refs, refs << 1, -(refs << 1), @@ -280,7 +317,7 @@ static void filter(AVFilterContext *ctx, AVFrame *dstpic, td.plane = i; ff_filter_execute(ctx, filter_slice, &td, NULL, - FFMIN(h, ff_filter_get_nb_threads(ctx))); + FFMIN((h+3)/4, ff_filter_get_nb_threads(ctx))); } if (yadif->current_field == YADIF_FIELD_END) { yadif->current_field = YADIF_FIELD_NORMAL; @@ -357,6 +394,7 @@ static int config_props(AVFilterLink *link) av_cold void ff_bwdif_init_filter_line(BWDIFContext *s, int bit_depth) { + s->filter_line3 = 0; if (bit_depth > 8) { s->filter_intra = filter_intra_16bit; s->filter_line = filter_line_c_16bit; diff --git a/tests/checkasm/vf_bwdif.c b/tests/checkasm/vf_bwdif.c index 5fdba09fdc..3399cacdf7 100644 --- a/tests/checkasm/vf_bwdif.c +++ b/tests/checkasm/vf_bwdif.c @@ -28,6 +28,10 @@ for (size_t i = 0; i < count; i++) \ buf0[i] = buf1[i] = rnd() & mask +#define randomize_overflow_check(buf0, buf1, mask, count) \ + for (size_t i = 0; i < count; i++) \ + buf0[i] = buf1[i] = (rnd() & 1) != 0 ? mask : 0; + #define BODY(type, depth) \ do { \ type prev0[9*WIDTH], prev1[9*WIDTH]; \ @@ -83,6 +87,83 @@ void checkasm_check_vf_bwdif(void) report("bwdif10"); } + if (!ctx_8.filter_line3) + ctx_8.filter_line3 = ff_bwdif_filter_line3_c; + + { + LOCAL_ALIGNED_16(uint8_t, prev0, [11*WIDTH]); + LOCAL_ALIGNED_16(uint8_t, prev1, [11*WIDTH]); + LOCAL_ALIGNED_16(uint8_t, next0, [11*WIDTH]); + LOCAL_ALIGNED_16(uint8_t, next1, [11*WIDTH]); + LOCAL_ALIGNED_16(uint8_t, cur0, [11*WIDTH]); + LOCAL_ALIGNED_16(uint8_t, cur1, [11*WIDTH]); + LOCAL_ALIGNED_16(uint8_t, dst0, [WIDTH*3]); + LOCAL_ALIGNED_16(uint8_t, dst1, [WIDTH*3]); + const int stride = WIDTH; + const int mask = (1<<8)-1; + int parity; + + for (parity = 0; parity != 2; ++parity) { + if (check_func(ctx_8.filter_line3, "bwdif8.line3.rnd.p%d", parity)) { + + declare_func(void, void * dst1, int d_stride, + const void * prev1, const void * cur1, const void * next1, int prefs, + int w, int parity, int clip_max); + + randomize_buffers(prev0, prev1, mask, 11*WIDTH); + randomize_buffers(next0, next1, mask, 11*WIDTH); + randomize_buffers( cur0, cur1, mask, 11*WIDTH); + + call_ref(dst0, stride, + prev0 + stride * 4, cur0 + stride * 4, next0 + stride * 4, stride, + WIDTH, parity, mask); + call_new(dst1, stride, + prev1 + stride * 4, cur1 + stride * 4, next1 + stride * 4, stride, + WIDTH, parity, mask); + + if (memcmp(dst0, dst1, WIDTH*3) + || memcmp(prev0, prev1, WIDTH*11) + || memcmp(next0, next1, WIDTH*11) + || memcmp( cur0, cur1, WIDTH*11)) + fail(); + + bench_new(dst1, stride, + prev1 + stride * 4, cur1 + stride * 4, next1 + stride * 4, stride, + WIDTH, parity, mask); + } + } + + // Use just 0s and ~0s to try to provoke bad cropping or overflow + // Parity makes no difference to this test so just test 0 + if (check_func(ctx_8.filter_line3, "bwdif8.line3.overflow")) { + + declare_func(void, void * dst1, int d_stride, + const void * prev1, const void * cur1, const void * next1, int prefs, + int w, int parity, int clip_max); + + randomize_overflow_check(prev0, prev1, mask, 11*WIDTH); + randomize_overflow_check(next0, next1, mask, 11*WIDTH); + randomize_overflow_check( cur0, cur1, mask, 11*WIDTH); + + call_ref(dst0, stride, + prev0 + stride * 4, cur0 + stride * 4, next0 + stride * 4, stride, + WIDTH, 0, mask); + call_new(dst1, stride, + prev1 + stride * 4, cur1 + stride * 4, next1 + stride * 4, stride, + WIDTH, 0, mask); + + if (memcmp(dst0, dst1, WIDTH*3) + || memcmp(prev0, prev1, WIDTH*11) + || memcmp(next0, next1, WIDTH*11) + || memcmp( cur0, cur1, WIDTH*11)) + fail(); + + // No point to benching + } + + report("bwdif8.line3"); + } + { LOCAL_ALIGNED_16(uint8_t, prev0, [11*WIDTH]); LOCAL_ALIGNED_16(uint8_t, prev1, [11*WIDTH]);