From patchwork Thu Feb 4 11:32:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Josh Dekker X-Patchwork-Id: 25384 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 7223F44A6C2 for ; Thu, 4 Feb 2021 13:33:18 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5C7B468A2A9; Thu, 4 Feb 2021 13:33:18 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BDD3A689ADA for ; Thu, 4 Feb 2021 13:33:10 +0200 (EET) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id 7865E5C0182; Thu, 4 Feb 2021 06:33:09 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Thu, 04 Feb 2021 06:33:09 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=itanimul.li; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=fm1; bh= a1v719MbPHUFSidCBIY69p9dmOj/1Y1GO79rU2TO/DI=; b=i4TQZpxZ778FE6Rl nW526CuwpHiQph65IHxgkQ/QOAs5TVN0eKcx0P6FZZQ68uIn7M4I46LXD5mVOBZq ANyG2ZKrdZHwCCZIZW8mPYoqmf3NFjTFF5HbepHqs5verI5ewBfXx7MoVBi6XKKI uugW8p0sibFrSc1x8l2pHd0N/jm3Ngyyo2OF/Dqw0H25pwlyV6pBoHTDqwxZxsxB LCxpOzrEeFdAfAcBsDOOI15Eu0hm+EJk8ICvYRhQe1rNsASJC3FNua80w0nCa4pE YatxeCBMa1jcCM+AOsETn0B0SdZug+Fz5C8sjnxCR4TgqHehIi1S7fKKjuWhoEb5 z8BQCQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=a1v719MbPHUFSidCBIY69p9dmOj/1Y1GO79rU2TO/ DI=; b=UECTXJZ3u9LvAufU5EpFG2eO1qce1o74iriPHnhLJ/vZAJI+m3KZYWbJS tkdxg3Ru0yXmW6I8LtsZqkO9RdGU7B7mxxxozFPfT+b6cGd2A7wR9PWInYmeDsYZ KRZTdxGwlwY6d1FvDtyV3xLwy3/vGChyh7wxv0YgwZWpEdQaVj2+Bs6OgEUAFmbW miayHxldee+VJs+DEGjPohwUhtqXQLxmS4S6jz/ipUHub69itoiJtf6DKzVSfrhW jFoNdYxeHys3QGlAufJGXfJZN1v5m0/w7Txz93bvPWwrJ2CAyPnsIR0ZipETcsjn qKmt3PnPqIESMNHq/55mAiarcr0Ew== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrgeeggddvtdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkffojghfgggtgfesthekre dtredtjeenucfhrhhomheplfhoshhhucffvghkkhgvrhcuoehjohhshhesihhtrghnihhm uhhlrdhliheqnecuggftrfgrthhtvghrnhepjeeikeduvdevleegleejtdetledvtdduje fgteegkeduleeludekudefveehffeinecuffhomhgrihhnpehnvghonhdrshgsnecukfhp peekjedruddvfedrudelvddrudehheenucevlhhushhtvghrufhiiigvpedtnecurfgrrh grmhepmhgrihhlfhhrohhmpehjohhshhesihhtrghnihhmuhhlrdhlih X-ME-Proxy: Received: from localhost.localdomain (i577bc09b.versanet.de [87.123.192.155]) by mail.messagingengine.com (Postfix) with ESMTPA id DECEF1080059; Thu, 4 Feb 2021 06:33:08 -0500 (EST) From: Josh Dekker To: ffmpeg-devel@ffmpeg.org Date: Thu, 4 Feb 2021 12:32:57 +0100 Message-Id: <20210204113259.20112-3-josh@itanimul.li> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20210204113259.20112-1-josh@itanimul.li> References: <20210204113259.20112-1-josh@itanimul.li> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 2/4] avcodec/aarch64/hevcdsp: port add_residual functions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: =?utf-8?q?Reimar_D=C3=B6ffinger?= Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Reimar Döffinger Speedup is fairly small, around 1.5%, but these are fairly simple. Signed-off-by: Josh Dekker --- libavcodec/aarch64/hevcdsp_idct_neon.S | 190 ++++++++++++++++++++++ libavcodec/aarch64/hevcdsp_init_aarch64.c | 24 +++ 2 files changed, 214 insertions(+) diff --git a/libavcodec/aarch64/hevcdsp_idct_neon.S b/libavcodec/aarch64/hevcdsp_idct_neon.S index c70d6a906d..329038a958 100644 --- a/libavcodec/aarch64/hevcdsp_idct_neon.S +++ b/libavcodec/aarch64/hevcdsp_idct_neon.S @@ -36,6 +36,196 @@ const trans, align=4 .short 31, 22, 13, 4 endconst +.macro clip10 in1, in2, c1, c2 + smax \in1, \in1, \c1 + smax \in2, \in2, \c1 + smin \in1, \in1, \c2 + smin \in2, \in2, \c2 +.endm + +function ff_hevc_add_residual_4x4_8_neon, export=1 + ld1 {v0.8h-v1.8h}, [x1] + ld1 {v2.s}[0], [x0], x2 + ld1 {v2.s}[1], [x0], x2 + ld1 {v2.s}[2], [x0], x2 + ld1 {v2.s}[3], [x0], x2 + sub x0, x0, x2, lsl #2 + uxtl v6.8h, v2.8B + uxtl2 v7.8h, v2.16B + sqadd v0.8h, v0.8h, v6.8h + sqadd v1.8h, v1.8h, v7.8h + sqxtun v0.8B, v0.8h + sqxtun2 v0.16B, v1.8h + st1 {v0.s}[0], [x0], x2 + st1 {v0.s}[1], [x0], x2 + st1 {v0.s}[2], [x0], x2 + st1 {v0.s}[3], [x0], x2 + ret +endfunc + +function ff_hevc_add_residual_4x4_10_neon, export=1 + mov x12, x0 + ld1 {v0.8h-v1.8h}, [x1] + ld1 {v2.d}[0], [x12], x2 + ld1 {v2.d}[1], [x12], x2 + ld1 {v3.d}[0], [x12], x2 + sqadd v0.8h, v0.8h, v2.8h + ld1 {V3.d}[1], [x12], x2 + movi v4.8h, #0 + sqadd v1.8h, v1.8h, v3.8h + mvni v5.8h, #0xFC, LSL #8 // movi #0x3FF + clip10 v0.8h, v1.8h, v4.8h, v5.8h + st1 {v0.d}[0], [x0], x2 + st1 {v0.d}[1], [x0], x2 + st1 {v1.d}[0], [x0], x2 + st1 {v1.d}[1], [x0], x2 + ret +endfunc + +function ff_hevc_add_residual_8x8_8_neon, export=1 + add x12, x0, x2 + add x2, x2, x2 + mov x3, #8 +1: subs x3, x3, #2 + ld1 {v2.d}[0], [x0] + ld1 {v2.d}[1], [x12] + uxtl v3.8h, v2.8B + ld1 {v0.8h-v1.8h}, [x1], #32 + uxtl2 v2.8h, v2.16B + sqadd v0.8h, v0.8h, v3.8h + sqadd v1.8h, v1.8h, v2.8h + sqxtun v0.8B, v0.8h + sqxtun2 v0.16B, v1.8h + st1 {v0.d}[0], [x0], x2 + st1 {v0.d}[1], [x12], x2 + bne 1b + ret +endfunc + +function ff_hevc_add_residual_8x8_10_neon, export=1 + add x12, x0, x2 + add x2, x2, x2 + mov x3, #8 + movi v4.8h, #0 + mvni v5.8h, #0xFC, LSL #8 // movi #0x3FF +1: subs x3, x3, #2 + ld1 {v0.8h-v1.8h}, [x1], #32 + ld1 {v2.8h}, [x0] + sqadd v0.8h, v0.8h, v2.8h + ld1 {v3.8h}, [x12] + sqadd v1.8h, v1.8h, v3.8h + clip10 v0.8h, v1.8h, v4.8h, v5.8h + st1 {v0.8h}, [x0], x2 + st1 {v1.8h}, [x12], x2 + bne 1b + ret +endfunc + +function ff_hevc_add_residual_16x16_8_neon, export=1 + mov x3, #16 + add x12, x0, x2 + add x2, x2, x2 +1: subs x3, x3, #2 + ld1 {v16.16B}, [x0] + ld1 {v0.8h-v3.8h}, [x1], #64 + ld1 {v19.16B}, [x12] + uxtl v17.8h, v16.8B + uxtl2 v18.8h, v16.16B + uxtl v20.8h, v19.8B + uxtl2 v21.8h, v19.16B + sqadd v0.8h, v0.8h, v17.8h + sqadd v1.8h, v1.8h, v18.8h + sqadd v2.8h, v2.8h, v20.8h + sqadd v3.8h, v3.8h, v21.8h + sqxtun v0.8B, v0.8h + sqxtun2 v0.16B, v1.8h + sqxtun v1.8B, v2.8h + sqxtun2 v1.16B, v3.8h + st1 {v0.16B}, [x0], x2 + st1 {v1.16B}, [x12], x2 + bne 1b + ret +endfunc + +function ff_hevc_add_residual_16x16_10_neon, export=1 + mov x3, #16 + movi v20.8h, #0 + mvni v21.8h, #0xFC, LSL #8 // movi #0x3FF + add x12, x0, x2 + add x2, x2, x2 +1: subs x3, x3, #2 + ld1 {v16.8h-v17.8h}, [x0] + ld1 {v0.8h-v3.8h}, [x1], #64 + sqadd v0.8h, v0.8h, v16.8h + ld1 {v18.8h-v19.8h}, [x12] + sqadd v1.8h, v1.8h, v17.8h + sqadd v2.8h, v2.8h, v18.8h + sqadd v3.8h, v3.8h, v19.8h + clip10 v0.8h, v1.8h, v20.8h, v21.8h + clip10 v2.8h, v3.8h, v20.8h, v21.8h + st1 {v0.8h-v1.8h}, [x0], x2 + st1 {v2.8h-v3.8h}, [x12], x2 + bne 1b + ret +endfunc + +function ff_hevc_add_residual_32x32_8_neon, export=1 + add x12, x0, x2 + add x2, x2, x2 + mov x3, #32 +1: subs x3, x3, #2 + ld1 {v20.16B, v21.16B}, [x0] + uxtl v16.8h, v20.8B + uxtl2 v17.8h, v20.16B + ld1 {v22.16B, v23.16B}, [x12] + uxtl v18.8h, v21.8B + uxtl2 v19.8h, v21.16B + uxtl v20.8h, v22.8B + ld1 {v0.8h-v3.8h}, [x1], #64 + ld1 {v4.8h-v7.8h}, [x1], #64 + uxtl2 v21.8h, v22.16B + uxtl v22.8h, v23.8B + uxtl2 v23.8h, v23.16B + sqadd v0.8h, v0.8h, v16.8h + sqadd v1.8h, v1.8h, v17.8h + sqadd v2.8h, v2.8h, v18.8h + sqadd v3.8h, v3.8h, v19.8h + sqadd v4.8h, v4.8h, v20.8h + sqadd v5.8h, v5.8h, v21.8h + sqadd v6.8h, v6.8h, v22.8h + sqadd v7.8h, v7.8h, v23.8h + sqxtun v0.8B, v0.8h + sqxtun2 v0.16B, v1.8h + sqxtun v1.8B, v2.8h + sqxtun2 v1.16B, v3.8h + sqxtun v2.8B, v4.8h + sqxtun2 v2.16B, v5.8h + st1 {v0.16B, v1.16B}, [x0], x2 + sqxtun v3.8B, v6.8h + sqxtun2 v3.16B, v7.8h + st1 {v2.16B, v3.16B}, [x12], x2 + bne 1b + ret +endfunc + +function ff_hevc_add_residual_32x32_10_neon, export=1 + mov x3, #32 + movi v20.8h, #0 + mvni v21.8h, #0xFC, LSL #8 // movi #0x3FF +1: subs x3, x3, #1 + ld1 {v0.8h-v3.8h}, [x1], #64 + ld1 {v16.8h-v19.8h}, [x0] + sqadd v0.8h, v0.8h, v16.8h + sqadd v1.8h, v1.8h, v17.8h + sqadd v2.8h, v2.8h, v18.8h + sqadd v3.8h, v3.8h, v19.8h + clip10 v0.8h, v1.8h, v20.8h, v21.8h + clip10 v2.8h, v3.8h, v20.8h, v21.8h + st1 {v0.8h-v3.8h}, [x0], x2 + bne 1b + ret +endfunc + .macro sum_sub out, in, c, op, p .ifc \op, + smlal\p \out, \in, \c diff --git a/libavcodec/aarch64/hevcdsp_init_aarch64.c b/libavcodec/aarch64/hevcdsp_init_aarch64.c index 19d9a7f9ed..4c29daa6d5 100644 --- a/libavcodec/aarch64/hevcdsp_init_aarch64.c +++ b/libavcodec/aarch64/hevcdsp_init_aarch64.c @@ -25,6 +25,22 @@ #include "libavutil/aarch64/cpu.h" #include "libavcodec/hevcdsp.h" +void ff_hevc_add_residual_4x4_8_neon(uint8_t *_dst, int16_t *coeffs, + ptrdiff_t stride); +void ff_hevc_add_residual_4x4_10_neon(uint8_t *_dst, int16_t *coeffs, + ptrdiff_t stride); +void ff_hevc_add_residual_8x8_8_neon(uint8_t *_dst, int16_t *coeffs, + ptrdiff_t stride); +void ff_hevc_add_residual_8x8_10_neon(uint8_t *_dst, int16_t *coeffs, + ptrdiff_t stride); +void ff_hevc_add_residual_16x16_8_neon(uint8_t *_dst, int16_t *coeffs, + ptrdiff_t stride); +void ff_hevc_add_residual_16x16_10_neon(uint8_t *_dst, int16_t *coeffs, + ptrdiff_t stride); +void ff_hevc_add_residual_32x32_8_neon(uint8_t *_dst, int16_t *coeffs, + ptrdiff_t stride); +void ff_hevc_add_residual_32x32_10_neon(uint8_t *_dst, int16_t *coeffs, + ptrdiff_t stride); void ff_hevc_idct_8x8_8_neon(int16_t *coeffs, int col_limit); void ff_hevc_idct_8x8_10_neon(int16_t *coeffs, int col_limit); void ff_hevc_idct_16x16_8_neon(int16_t *coeffs, int col_limit); @@ -35,10 +51,18 @@ av_cold void ff_hevc_dsp_init_aarch64(HEVCDSPContext *c, const int bit_depth) if (!have_neon(av_get_cpu_flags())) return; if (bit_depth == 8) { + c->add_residual[0] = ff_hevc_add_residual_4x4_8_neon; + c->add_residual[1] = ff_hevc_add_residual_8x8_8_neon; + c->add_residual[2] = ff_hevc_add_residual_16x16_8_neon; + c->add_residual[3] = ff_hevc_add_residual_32x32_8_neon; c->idct[1] = ff_hevc_idct_8x8_8_neon; c->idct[2] = ff_hevc_idct_16x16_8_neon; } if (bit_depth == 10) { + c->add_residual[0] = ff_hevc_add_residual_4x4_10_neon; + c->add_residual[1] = ff_hevc_add_residual_8x8_10_neon; + c->add_residual[2] = ff_hevc_add_residual_16x16_10_neon; + c->add_residual[3] = ff_hevc_add_residual_32x32_10_neon; c->idct[1] = ff_hevc_idct_8x8_10_neon; c->idct[2] = ff_hevc_idct_16x16_10_neon; }