From patchwork Thu Jan 7 12:10:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Dekker X-Patchwork-Id: 24828 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 14A0E44AF79 for ; Thu, 7 Jan 2021 14:10:35 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E96E6689818; Thu, 7 Jan 2021 14:10:34 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6123D6880D7 for ; Thu, 7 Jan 2021 14:10:28 +0200 (EET) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id D60681773 for ; Thu, 7 Jan 2021 07:10:26 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Thu, 07 Jan 2021 07:10:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=itanimul.li; h= from:to:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm3; bh=cjiSWAuSNKs6F bw5V95cw6baoGmSbeVigxxIZsPnwMs=; b=xpqqiIdFsFbAhNHCipp4/dZihdblx UIbubFoKsLrABFzJgMpECb21JzwzJCDRbfJZRHtA9Aw0jx88o5VgMRUSLWs2pA9j ZEwE8C6N/yEld10T3KRCbjCk3tNm65J6C2gvhReWdfEAG/e4dhk5v5e8qwzf6E6Q G76VjTuzNyeEblIRhl7EOl9/YjQspnoTGrjLm+t85Ceq+d0nlWhcukZCDY0G/GjB FDPDVRhQfZZ7XVVcbGjZgDkiL5XT92m4pNhDjIOCK3dT54iPVl25iFosQab+fua1 jbaTHzMmLVPiR3InoFjYvWl+9K695iT9tZV1TroArmdhJG+KK51s7mTTw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=cjiSWAuSNKs6Fbw5V95cw6baoGmSbeVigxxIZsPnwMs=; b=n4UUHHvm X2acXVJoh1jWegux68BaDgzKrQcCXyoZfG1dU/SAayKLlnZidei5yjPOXU667QQX sF53XHNcj2tFaAQXaFjyBh5/ooTe3Z5XLr3GNGwPBLGPfcVrgBq9Q2x5W4vllpku gz4kyBQvWbmlBJ1CXNV9yXF5h52jwbyCHHHlFolG8CwW9gSQVgOBVT3fRrHRjMoU hZ9CIAGlZZ63zkWgRdJxpRhU/IBIYRwwjLDP/N0H3Up8rGSsRMH2HPHBLEB4vPVK 00Zx24ZYyfJ+Ot4YZj6fviI86gcjkZQLsy/E9mgcZpXvEO6bvou9AvseteIVgESr MqgbBOLdDavAiQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrvdegvddgfeeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffkofgjfhgggfestdekre dtredttdenucfhrhhomheplfhoshhhucffvghkkhgvrhcuoehjohhshhesihhtrghnihhm uhhlrdhliheqnecuggftrfgrthhtvghrnhepgeduieeltdeltedugfegffehfefgudduhe ehgfejgeeuiedukedvtdejiefgheeunecuffhomhgrihhnpehnvghonhdrshgsnecukfhp peekkedrudeftddrgeekrddufeejnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrg hmpehmrghilhhfrhhomhepjhhoshhhsehithgrnhhimhhulhdrlhhi X-ME-Proxy: Received: from localhost.localdomain (mue-88-130-48-137.dsl.tropolys.de [88.130.48.137]) by mail.messagingengine.com (Postfix) with ESMTPA id 91F4524005D for ; Thu, 7 Jan 2021 07:10:25 -0500 (EST) From: Josh Dekker To: ffmpeg-devel@ffmpeg.org Date: Thu, 7 Jan 2021 13:10:17 +0100 Message-Id: <20210107121020.86179-2-josh@itanimul.li> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20210107121020.86179-1-josh@itanimul.li> References: <20210107121020.86179-1-josh@itanimul.li> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/4] lavc/aarch64: add HEVC add_residual NEON X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Josh Dekker --- libavcodec/aarch64/Makefile | 2 + libavcodec/aarch64/hevcdsp_add_res_neon.S | 298 ++++++++++++++++++++++ libavcodec/aarch64/hevcdsp_init.c | 59 +++++ libavcodec/hevcdsp.c | 2 + libavcodec/hevcdsp.h | 1 + 5 files changed, 362 insertions(+) create mode 100644 libavcodec/aarch64/hevcdsp_add_res_neon.S create mode 100644 libavcodec/aarch64/hevcdsp_init.c diff --git a/libavcodec/aarch64/Makefile b/libavcodec/aarch64/Makefile index f6434e40da..4bdd554e7e 100644 --- a/libavcodec/aarch64/Makefile +++ b/libavcodec/aarch64/Makefile @@ -17,6 +17,7 @@ OBJS-$(CONFIG_VP8DSP) += aarch64/vp8dsp_init_aarch64.o OBJS-$(CONFIG_AAC_DECODER) += aarch64/aacpsdsp_init_aarch64.o \ aarch64/sbrdsp_init_aarch64.o OBJS-$(CONFIG_DCA_DECODER) += aarch64/synth_filter_init.o +OBJS-$(CONFIG_HEVC_DECODER) += aarch64/hevcdsp_init.o OBJS-$(CONFIG_OPUS_DECODER) += aarch64/opusdsp_init.o OBJS-$(CONFIG_RV40_DECODER) += aarch64/rv40dsp_init_aarch64.o OBJS-$(CONFIG_VC1DSP) += aarch64/vc1dsp_init_aarch64.o @@ -53,6 +54,7 @@ NEON-OBJS-$(CONFIG_VP8DSP) += aarch64/vp8dsp_neon.o # decoders/encoders NEON-OBJS-$(CONFIG_AAC_DECODER) += aarch64/aacpsdsp_neon.o NEON-OBJS-$(CONFIG_DCA_DECODER) += aarch64/synth_filter_neon.o +NEON-OBJS-$(CONFIG_HEVC_DECODER) += aarch64/hevcdsp_add_res_neon.o NEON-OBJS-$(CONFIG_OPUS_DECODER) += aarch64/opusdsp_neon.o NEON-OBJS-$(CONFIG_VORBIS_DECODER) += aarch64/vorbisdsp_neon.o NEON-OBJS-$(CONFIG_VP9_DECODER) += aarch64/vp9itxfm_16bpp_neon.o \ diff --git a/libavcodec/aarch64/hevcdsp_add_res_neon.S b/libavcodec/aarch64/hevcdsp_add_res_neon.S new file mode 100644 index 0000000000..4005366192 --- /dev/null +++ b/libavcodec/aarch64/hevcdsp_add_res_neon.S @@ -0,0 +1,298 @@ +/* -*-arm64-*- + * + * AArch64 NEON optimised add residual functions for HEVC decoding + * + * Copyright (c) 2020 Josh Dekker + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/aarch64/asm.S" + +.macro clip10 in1, in2, c1, c2 + smax \in1, \in1, \c1 + smax \in2, \in2, \c1 + smin \in1, \in1, \c2 + smin \in2, \in2, \c2 +.endm + +function ff_hevc_add_residual_4x4_8_neon, export=1 + mov x3, x0 + ld1 {v0.S}[0], [x3], x2 + ld1 {v0.S}[1], [x3], x2 + ld1 {v1.S}[0], [x3], x2 + ld1 {v1.S}[1], [x3], x2 + ld1 { v2.8H-v3.8H}, [x1] + ushll v4.8H, v0.8B, #0 + ushll v5.8H, v1.8B, #0 + add v6.8H, v4.8H, v2.8H + add v7.8H, v5.8H, v3.8H + sqxtun v0.8B, v6.8H + sqxtun v1.8B, v7.8H + st1 {v0.S}[0], [x0], x2 + st1 {v0.S}[1], [x0], x2 + st1 {v1.S}[0], [x0], x2 + st1 {v1.S}[1], [x0], x2 + ret +endfunc + +function ff_hevc_add_residual_4x4_10_neon, export=1 + mov x3, x0 + movi v4.8H, #0 + mvni v5.8H, #0xFC, lsl #8 + ld1 {v0.D}[0], [x3], x2 + ld1 {v0.D}[1], [x3], x2 + ld1 {v1.D}[0], [x3], x2 + ld1 {v1.D}[1], [x3], x2 + ld1 { v2.8H-v3.8H}, [x1] + add v2.8H, v0.8H, v2.8H + add v3.8H, v1.8H, v3.8H + clip10 v2.8H, v3.8H, v4.8H, v5.8H + st1 {v2.D}[0], [x0], x2 + st1 {v2.D}[1], [x0], x2 + st1 {v3.D}[0], [x0], x2 + st1 {v3.D}[1], [x0], x2 + ret +endfunc + +function ff_hevc_add_residual_8x8_8_neon, export=1 + mov x3, x0 + ld1 {v0.8B}, [x3], x2 + ld1 {v1.8B}, [x3], x2 + ld1 {v2.8B}, [x3], x2 + ld1 {v3.8B}, [x3], x2 + ld1 {v4.8B}, [x3], x2 + ld1 {v5.8B}, [x3], x2 + ld1 {v6.8B}, [x3], x2 + ld1 {v7.8B}, [x3], x2 + ld1 { v16.8H-v19.8H}, [x1], #64 + ld1 { v20.8H-v23.8H}, [x1] + ushll v24.8H, v0.8B, #0 + ushll v25.8H, v1.8B, #0 + ushll v26.8H, v2.8B, #0 + ushll v27.8H, v3.8B, #0 + ushll v28.8H, v4.8B, #0 + ushll v29.8H, v5.8B, #0 + ushll v30.8H, v6.8B, #0 + ushll v31.8H, v7.8B, #0 + add v0.8H, v24.8H, v16.8H + add v1.8H, v25.8H, v17.8H + add v2.8H, v26.8H, v18.8H + add v3.8H, v27.8H, v19.8H + add v4.8H, v28.8H, v20.8H + add v5.8H, v29.8H, v21.8H + add v6.8H, v30.8H, v22.8H + add v7.8H, v31.8H, v23.8H + sqxtun v24.8B, v0.8H + sqxtun v25.8B, v1.8H + sqxtun v26.8B, v2.8H + sqxtun v27.8B, v3.8H + sqxtun v28.8B, v4.8H + sqxtun v29.8B, v5.8H + sqxtun v30.8B, v6.8H + sqxtun v31.8B, v7.8H + st1 {v24.8B}, [x0], x2 + st1 {v25.8B}, [x0], x2 + st1 {v26.8B}, [x0], x2 + st1 {v27.8B}, [x0], x2 + st1 {v28.8B}, [x0], x2 + st1 {v29.8B}, [x0], x2 + st1 {v30.8B}, [x0], x2 + st1 {v31.8B}, [x0], x2 + ret +endfunc + +function ff_hevc_add_residual_8x8_10_neon, export=1 + mov x3, x0 + movi v0.8H, #0 + mvni v1.8H, #0xFC, lsl #8 + ld1 {v24.8H}, [x3], x2 + ld1 {v25.8H}, [x3], x2 + ld1 {v26.8H}, [x3], x2 + ld1 {v27.8H}, [x3], x2 + ld1 {v28.8H}, [x3], x2 + ld1 {v29.8H}, [x3], x2 + ld1 {v30.8H}, [x3], x2 + ld1 {v31.8H}, [x3], x2 + ld1 {v16.8H-v19.8H}, [x1], #64 + ld1 {v20.8H-v23.8H}, [x1] + add v16.8H, v16.8H, v24.8H + add v17.8H, v17.8H, v25.8H + clip10 v16.8H, v17.8H, v0.8H, v1.8H + add v18.8H, v18.8H, v26.8H + add v19.8H, v19.8H, v27.8H + clip10 v18.8H, v19.8H, v0.8H, v1.8H + add v20.8H, v20.8H, v28.8H + add v21.8H, v21.8H, v29.8H + clip10 v20.8H, v21.8H, v0.8H, v1.8H + add v22.8H, v22.8H, v30.8H + add v23.8H, v23.8H, v31.8H + clip10 v22.8H, v23.8H, v0.8H, v1.8H + st1 {v16.8H}, [x0], x2 + st1 {v17.8H}, [x0], x2 + st1 {v18.8H}, [x0], x2 + st1 {v19.8H}, [x0], x2 + st1 {v20.8H}, [x0], x2 + st1 {v21.8H}, [x0], x2 + st1 {v22.8H}, [x0], x2 + st1 {v23.8H}, [x0], x2 + ret +endfunc + +function ff_hevc_add_residual_16x16_8_neon, export=1 + mov x3, x0 + mov x4, #4 +1: + subs x4, x4, #1 + ld1 { v0.8B- v1.8B}, [x3], x2 + ld1 { v2.8B- v3.8B}, [x3], x2 + ld1 { v4.8B- v5.8B}, [x3], x2 + ld1 { v6.8B- v7.8B}, [x3], x2 + ld1 {v16.8H-v19.8H}, [x1], #64 + ld1 {v20.8H-v23.8H}, [x1], #64 + ushll v24.8H, v0.8B, #0 + ushll v25.8H, v1.8B, #0 + ushll v26.8H, v2.8B, #0 + ushll v27.8H, v3.8B, #0 + ushll v28.8H, v4.8B, #0 + ushll v29.8H, v5.8B, #0 + ushll v30.8H, v6.8B, #0 + ushll v31.8H, v7.8B, #0 + add v0.8H, v24.8H, v16.8H + add v1.8H, v25.8H, v17.8H + add v2.8H, v26.8H, v18.8H + add v3.8H, v27.8H, v19.8H + add v4.8H, v28.8H, v20.8H + add v5.8H, v29.8H, v21.8H + add v6.8H, v30.8H, v22.8H + add v7.8H, v31.8H, v23.8H + sqxtun v24.8B, v0.8H + sqxtun v25.8B, v1.8H + sqxtun v26.8B, v2.8H + sqxtun v27.8B, v3.8H + sqxtun v28.8B, v4.8H + sqxtun v29.8B, v5.8H + sqxtun v30.8B, v6.8H + sqxtun v31.8B, v7.8H + st1 {v24.8B-v25.8B}, [x0], x2 + st1 {v26.8B-v27.8B}, [x0], x2 + st1 {v28.8B-v29.8B}, [x0], x2 + st1 {v30.8B-v31.8B}, [x0], x2 + b.ne 1b + ret +endfunc + +function ff_hevc_add_residual_16x16_10_neon, export=1 + mov x3, x0 + mov x4, #4 + movi v0.8H, #0 + mvni v1.8H, #0xFC, lsl #8 +1: + subs x4, x4, #1 + ld1 {v16.8H-v17.8H}, [x3], x2 + ld1 {v18.8H-v19.8H}, [x3], x2 + ld1 {v20.8H-v21.8H}, [x3], x2 + ld1 {v22.8H-v23.8H}, [x3], x2 + ld1 {v24.8H-v27.8H}, [x1], #64 + ld1 {v28.8H-v31.8H}, [x1], #64 + add v16.8H, v16.8H, v24.8H + add v17.8H, v17.8H, v25.8H + clip10 v16.8H, v17.8H, v0.8H, v1.8H + add v18.8H, v18.8H, v26.8H + add v19.8H, v19.8H, v27.8H + clip10 v18.8H, v19.8H, v0.8H, v1.8H + add v20.8H, v20.8H, v28.8H + add v21.8H, v21.8H, v29.8H + clip10 v20.8H, v21.8H, v0.8H, v1.8H + add v22.8H, v22.8H, v30.8H + add v23.8H, v23.8H, v31.8H + clip10 v22.8H, v23.8H, v0.8H, v1.8H + st1 {v16.8H-v17.8H}, [x0], x2 + st1 {v18.8H-v19.8H}, [x0], x2 + st1 {v20.8H-v21.8H}, [x0], x2 + st1 {v22.8H-v23.8H}, [x0], x2 + b.ne 1b + ret +endfunc + +function ff_hevc_add_residual_32x32_8_neon, export=1 + mov x3, x0 + mov x4, #16 +1: + subs x4, x4, #1 + ld1 { v0.8B- v3.8B}, [x3], x2 + ld1 { v4.8B- v7.8B}, [x3], x2 + ld1 {v16.8H-v19.8H}, [x1], #64 + ld1 {v20.8H-v23.8H}, [x1], #64 + ushll v24.8H, v0.8B, #0 + ushll v25.8H, v1.8B, #0 + ushll v26.8H, v2.8B, #0 + ushll v27.8H, v3.8B, #0 + ushll v28.8H, v4.8B, #0 + ushll v29.8H, v5.8B, #0 + ushll v30.8H, v6.8B, #0 + ushll v31.8H, v7.8B, #0 + add v0.8H, v24.8H, v16.8H + add v1.8H, v25.8H, v17.8H + add v2.8H, v26.8H, v18.8H + add v3.8H, v27.8H, v19.8H + add v4.8H, v28.8H, v20.8H + add v5.8H, v29.8H, v21.8H + add v6.8H, v30.8H, v22.8H + add v7.8H, v31.8H, v23.8H + sqxtun v24.8B, v0.8H + sqxtun v25.8B, v1.8H + sqxtun v26.8B, v2.8H + sqxtun v27.8B, v3.8H + sqxtun v28.8B, v4.8H + sqxtun v29.8B, v5.8H + sqxtun v30.8B, v6.8H + sqxtun v31.8B, v7.8H + st1 {v24.8B-v27.8B}, [x0], x2 + st1 {v28.8B-v31.8B}, [x0], x2 + b.ne 1b + ret +endfunc + +function ff_hevc_add_residual_32x32_10_neon, export=1 + mov x3, x0 + mov x4, #16 + movi v0.8H, #0 + mvni v1.8H, #0xFC, lsl #8 +1: + subs x4, x4, #1 + ld1 {v16.8H-v19.8H}, [x3], x2 + ld1 {v20.8H-v23.8H}, [x3], x2 + ld1 {v24.8H-v27.8H}, [x1], #64 + ld1 {v28.8H-v31.8H}, [x1], #64 + add v16.8H, v16.8H, v24.8H + add v17.8H, v17.8H, v25.8H + clip10 v16.8H, v17.8H, v0.8H, v1.8H + add v18.8H, v18.8H, v26.8H + add v19.8H, v19.8H, v27.8H + clip10 v18.8H, v19.8H, v0.8H, v1.8H + add v20.8H, v20.8H, v28.8H + add v21.8H, v21.8H, v29.8H + clip10 v20.8H, v21.8H, v0.8H, v1.8H + add v22.8H, v22.8H, v30.8H + add v23.8H, v23.8H, v31.8H + clip10 v22.8H, v23.8H, v0.8H, v1.8H + st1 {v16.8H-v19.8H}, [x0], x2 + st1 {v20.8H-v23.8H}, [x0], x2 + b.ne 1b + ret +endfunc diff --git a/libavcodec/aarch64/hevcdsp_init.c b/libavcodec/aarch64/hevcdsp_init.c new file mode 100644 index 0000000000..f0a617ab39 --- /dev/null +++ b/libavcodec/aarch64/hevcdsp_init.c @@ -0,0 +1,59 @@ +/* + * Copyright (c) 2014 Seppo Tomperi + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/attributes.h" +#include "libavutil/aarch64/cpu.h" +#include "libavcodec/hevcdsp.h" +#include "libavcodec/avcodec.h" + +void ff_hevc_add_residual_4x4_8_neon(uint8_t *_dst, int16_t *coeffs, + ptrdiff_t stride); +void ff_hevc_add_residual_4x4_10_neon(uint8_t *_dst, int16_t *coeffs, + ptrdiff_t stride); +void ff_hevc_add_residual_8x8_8_neon(uint8_t *_dst, int16_t *coeffs, + ptrdiff_t stride); +void ff_hevc_add_residual_8x8_10_neon(uint8_t *_dst, int16_t *coeffs, + ptrdiff_t stride); +void ff_hevc_add_residual_16x16_8_neon(uint8_t *_dst, int16_t *coeffs, + ptrdiff_t stride); +void ff_hevc_add_residual_16x16_10_neon(uint8_t *_dst, int16_t *coeffs, + ptrdiff_t stride); +void ff_hevc_add_residual_32x32_8_neon(uint8_t *_dst, int16_t *coeffs, + ptrdiff_t stride); +void ff_hevc_add_residual_32x32_10_neon(uint8_t *_dst, int16_t *coeffs, + ptrdiff_t stride); + +av_cold void ff_hevc_dsp_init_aarch64(HEVCDSPContext *c, const int bit_depth) +{ + int cpu_flags = av_get_cpu_flags(); + if (have_neon(cpu_flags) && bit_depth == 8) { + c->add_residual[0] = ff_hevc_add_residual_4x4_8_neon; + c->add_residual[1] = ff_hevc_add_residual_8x8_8_neon; + c->add_residual[2] = ff_hevc_add_residual_16x16_8_neon; + c->add_residual[3] = ff_hevc_add_residual_32x32_8_neon; + } + + if (have_neon(cpu_flags) && bit_depth == 10) { + c->add_residual[0] = ff_hevc_add_residual_4x4_10_neon; + c->add_residual[1] = ff_hevc_add_residual_8x8_10_neon; + c->add_residual[2] = ff_hevc_add_residual_16x16_10_neon; + c->add_residual[3] = ff_hevc_add_residual_32x32_10_neon; + } +} diff --git a/libavcodec/hevcdsp.c b/libavcodec/hevcdsp.c index 957e40d5ff..fe272ac1ce 100644 --- a/libavcodec/hevcdsp.c +++ b/libavcodec/hevcdsp.c @@ -257,6 +257,8 @@ int i = 0; break; } + if (ARCH_AARCH64) + ff_hevc_dsp_init_aarch64(hevcdsp, bit_depth); if (ARCH_ARM) ff_hevc_dsp_init_arm(hevcdsp, bit_depth); if (ARCH_PPC) diff --git a/libavcodec/hevcdsp.h b/libavcodec/hevcdsp.h index c605a343d6..0e013a8328 100644 --- a/libavcodec/hevcdsp.h +++ b/libavcodec/hevcdsp.h @@ -129,6 +129,7 @@ void ff_hevc_dsp_init(HEVCDSPContext *hpc, int bit_depth); extern const int8_t ff_hevc_epel_filters[7][4]; extern const int8_t ff_hevc_qpel_filters[3][16]; +void ff_hevc_dsp_init_aarch64(HEVCDSPContext *c, const int bit_depth); void ff_hevc_dsp_init_arm(HEVCDSPContext *c, const int bit_depth); void ff_hevc_dsp_init_ppc(HEVCDSPContext *c, const int bit_depth); void ff_hevc_dsp_init_x86(HEVCDSPContext *c, const int bit_depth); From patchwork Thu Jan 7 12:10:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Dekker X-Patchwork-Id: 24829 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 2F97944AF79 for ; Thu, 7 Jan 2021 14:10:36 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 14BBB689DF4; Thu, 7 Jan 2021 14:10:36 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6DC93688288 for ; Thu, 7 Jan 2021 14:10:28 +0200 (EET) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id DA9F21796 for ; Thu, 7 Jan 2021 07:10:26 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Thu, 07 Jan 2021 07:10:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=itanimul.li; h= from:to:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm3; bh=MtdnGxb/avBV2 CPGiB4DtV/53ODqjPj4yA/DcIab5h8=; b=FPd6LsEhhR4pOvTZCVV3EX3yNd1gz 9ghcEJ90JWGZKwJcvfk1C1tvl8JjPwfA0pvSUVnkPJKUE/+ESZSMCKJm8Rr124yM UhzdIrOhNB14cadoyn17ZhbAP2Sbcsb5goV/1YEpI8ww5H9hdTPMKtaEaHM2asu1 mit0VHWR15ipJ1qRiUVpbNNTrRVdVGrb4eMlXSbgGZ3m+RdZFlg2Gi4lCDHf+Ero YjLJg3VScz0/kTHXZs1IoWpX+qQgSgnXRJz5VzpPRetxJJ3OZndJnxNBGoG4E/TR Nj5pyxTuyvbfkxpDtqt3BQFun4diHPBDfWJ45vO3Vkqu+p2wMTPT8OLNw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=MtdnGxb/avBV2CPGiB4DtV/53ODqjPj4yA/DcIab5h8=; b=MvOBdbra ZwkHBPUZA9aNeyuyjQFPDAc8vW+K4w+zjBzd9Q9CFDHGRHL9Eg/f3stf+sUOz8CF szvL0EPg6FVUwYCh1zm4NaOAjS0GXfnu3+JOmVyB0HO032XlPQsHTcxLleCYSX8z wrCWbCPf9nOhlOnQjyouXtEcwZCYx8HPQASqgXcqAN107nY6enjSavXYXg44YhyL Y47mWyrKs+lvQEuPgZwnJDL9EZYwBZsZZTX2jXsfEKe55RliO1HrbkROojHfXEKs huhnNoJflQa0ej+eB/Q34L/n2Z/bBOrDcyf19bYkZKzawc7RPC7HlY45tjTNZ9In 9CNreWCBjfcJEQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrvdegvddgfeejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffkofgjfhgggfestdekre dtredttdenucfhrhhomheplfhoshhhucffvghkkhgvrhcuoehjohhshhesihhtrghnihhm uhhlrdhliheqnecuggftrfgrthhtvghrnhepgeduieeltdeltedugfegffehfefgudduhe ehgfejgeeuiedukedvtdejiefgheeunecuffhomhgrihhnpehnvghonhdrshgsnecukfhp peekkedrudeftddrgeekrddufeejnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrg hmpehmrghilhhfrhhomhepjhhoshhhsehithgrnhhimhhulhdrlhhi X-ME-Proxy: Received: from localhost.localdomain (mue-88-130-48-137.dsl.tropolys.de [88.130.48.137]) by mail.messagingengine.com (Postfix) with ESMTPA id 1A985240066 for ; Thu, 7 Jan 2021 07:10:26 -0500 (EST) From: Josh Dekker To: ffmpeg-devel@ffmpeg.org Date: Thu, 7 Jan 2021 13:10:18 +0100 Message-Id: <20210107121020.86179-3-josh@itanimul.li> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20210107121020.86179-1-josh@itanimul.li> References: <20210107121020.86179-1-josh@itanimul.li> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/4] lavc/aarch64: add HEVC idct_dc NEON X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Josh Dekker --- libavcodec/aarch64/Makefile | 3 +- libavcodec/aarch64/hevcdsp_idct_neon.S | 74 ++++++++++++++++++++++++++ libavcodec/aarch64/hevcdsp_init.c | 19 +++++++ 3 files changed, 95 insertions(+), 1 deletion(-) create mode 100644 libavcodec/aarch64/hevcdsp_idct_neon.S diff --git a/libavcodec/aarch64/Makefile b/libavcodec/aarch64/Makefile index 4bdd554e7e..42d80bf74c 100644 --- a/libavcodec/aarch64/Makefile +++ b/libavcodec/aarch64/Makefile @@ -54,7 +54,8 @@ NEON-OBJS-$(CONFIG_VP8DSP) += aarch64/vp8dsp_neon.o # decoders/encoders NEON-OBJS-$(CONFIG_AAC_DECODER) += aarch64/aacpsdsp_neon.o NEON-OBJS-$(CONFIG_DCA_DECODER) += aarch64/synth_filter_neon.o -NEON-OBJS-$(CONFIG_HEVC_DECODER) += aarch64/hevcdsp_add_res_neon.o +NEON-OBJS-$(CONFIG_HEVC_DECODER) += aarch64/hevcdsp_add_res_neon.o \ + aarch64/hevcdsp_idct_neon.o NEON-OBJS-$(CONFIG_OPUS_DECODER) += aarch64/opusdsp_neon.o NEON-OBJS-$(CONFIG_VORBIS_DECODER) += aarch64/vorbisdsp_neon.o NEON-OBJS-$(CONFIG_VP9_DECODER) += aarch64/vp9itxfm_16bpp_neon.o \ diff --git a/libavcodec/aarch64/hevcdsp_idct_neon.S b/libavcodec/aarch64/hevcdsp_idct_neon.S new file mode 100644 index 0000000000..cd886bb6dc --- /dev/null +++ b/libavcodec/aarch64/hevcdsp_idct_neon.S @@ -0,0 +1,74 @@ +/* -*-arm64-*- + * + * AArch64 NEON optimised IDCT functions for HEVC decoding + * + * Copyright (c) 2020 Josh Dekker + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/aarch64/asm.S" + +.macro idct_dc size bitdepth +function ff_hevc_idct_\size\()x\size\()_dc_\bitdepth\()_neon, export=1 + ldrsh w1, [x0] + mov w2, #(1 << (13 - \bitdepth)) + add w1, w1, #1 + asr w1, w1, #1 + add w1, w1, w2 + asr w1, w1, #(14 - \bitdepth) + dup v0.8h, w1 + dup v1.8h, w1 +.if \size > 4 + dup v2.8h, w1 + dup v3.8h, w1 +.if \size > 16 /* dc 32x32 */ + mov x2, #4 +1: + subs x2, x2, #1 +.endif +.if \size > 8 /* dc 16x16 */ + st1 {v0.8h-v3.8h}, [x0], #64 + st1 {v0.8h-v3.8h}, [x0], #64 + st1 {v0.8h-v3.8h}, [x0], #64 + st1 {v0.8h-v3.8h}, [x0], #64 + st1 {v0.8h-v3.8h}, [x0], #64 + st1 {v0.8h-v3.8h}, [x0], #64 +.endif /* dc 8x8 */ + st1 {v0.8h-v3.8h}, [x0], #64 + st1 {v0.8h-v3.8h}, [x0], #64 +.if \size > 16 /* dc 32x32 */ + bne 1b +.endif +.else /* dc 4x4 */ + st1 {v0.8h-v1.8h}, [x0] +.endif + ret +endfunc +.endm + +idct_dc 4 8 +idct_dc 4 10 + +idct_dc 8 8 +idct_dc 8 10 + +idct_dc 16 8 +idct_dc 16 10 + +idct_dc 32 8 +idct_dc 32 10 diff --git a/libavcodec/aarch64/hevcdsp_init.c b/libavcodec/aarch64/hevcdsp_init.c index f0a617ab39..2cd7ef3a6c 100644 --- a/libavcodec/aarch64/hevcdsp_init.c +++ b/libavcodec/aarch64/hevcdsp_init.c @@ -23,6 +23,15 @@ #include "libavcodec/hevcdsp.h" #include "libavcodec/avcodec.h" +void ff_hevc_idct_4x4_dc_8_neon(int16_t *coeffs); +void ff_hevc_idct_8x8_dc_8_neon(int16_t *coeffs); +void ff_hevc_idct_16x16_dc_8_neon(int16_t *coeffs); +void ff_hevc_idct_32x32_dc_8_neon(int16_t *coeffs); +void ff_hevc_idct_4x4_dc_10_neon(int16_t *coeffs); +void ff_hevc_idct_8x8_dc_10_neon(int16_t *coeffs); +void ff_hevc_idct_16x16_dc_10_neon(int16_t *coeffs); +void ff_hevc_idct_32x32_dc_10_neon(int16_t *coeffs); + void ff_hevc_add_residual_4x4_8_neon(uint8_t *_dst, int16_t *coeffs, ptrdiff_t stride); void ff_hevc_add_residual_4x4_10_neon(uint8_t *_dst, int16_t *coeffs, @@ -48,6 +57,11 @@ av_cold void ff_hevc_dsp_init_aarch64(HEVCDSPContext *c, const int bit_depth) c->add_residual[1] = ff_hevc_add_residual_8x8_8_neon; c->add_residual[2] = ff_hevc_add_residual_16x16_8_neon; c->add_residual[3] = ff_hevc_add_residual_32x32_8_neon; + + c->idct_dc[0] = ff_hevc_idct_4x4_dc_8_neon; + c->idct_dc[1] = ff_hevc_idct_8x8_dc_8_neon; + c->idct_dc[2] = ff_hevc_idct_16x16_dc_8_neon; + c->idct_dc[3] = ff_hevc_idct_32x32_dc_8_neon; } if (have_neon(cpu_flags) && bit_depth == 10) { @@ -55,5 +69,10 @@ av_cold void ff_hevc_dsp_init_aarch64(HEVCDSPContext *c, const int bit_depth) c->add_residual[1] = ff_hevc_add_residual_8x8_10_neon; c->add_residual[2] = ff_hevc_add_residual_16x16_10_neon; c->add_residual[3] = ff_hevc_add_residual_32x32_10_neon; + + c->idct_dc[0] = ff_hevc_idct_4x4_dc_10_neon; + c->idct_dc[1] = ff_hevc_idct_8x8_dc_10_neon; + c->idct_dc[2] = ff_hevc_idct_16x16_dc_10_neon; + c->idct_dc[3] = ff_hevc_idct_32x32_dc_10_neon; } } From patchwork Thu Jan 7 12:10:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Dekker X-Patchwork-Id: 24830 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 293F544AF79 for ; Thu, 7 Jan 2021 14:10:37 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1655468A370; Thu, 7 Jan 2021 14:10:37 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id EFD43688288 for ; Thu, 7 Jan 2021 14:10:28 +0200 (EET) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id 6D546180E for ; Thu, 7 Jan 2021 07:10:27 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Thu, 07 Jan 2021 07:10:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=itanimul.li; h= from:to:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm3; bh=hMfy+BKFNus/I fgQgEkmXZTwz8LdL/U9gmrGSu5zy3o=; b=SUNvHOGuKSBaD0TIk86cGBSMftWOl s3bInVVjfO5Ph4QtvnzSDjMmqhDpSr1ZYBugxkiSrhoW8i+tW38XYvB4hx7EvXuH wKJbEZFKCOWvO9ZLVS8bW5sqoghGiuRBCCPMnJXnG24bmot8tUDgBelZJ9ZcnPE4 b26FnPayKeWvjtxGbAdJaRhA7FxTY7XcH8Kve3qew/ieav2n/LSPBdGqsOzJhmAL QGyS0rBBkXvtDJYAuT38UqESZNAFbEpGybTs5+aTH6z87xVyt6lkO5V35cIa6tCi iFZnONEAcdYIbZIItHVfTAvGgY6cr4rqiFy+VQ1++6DUtjQZeQ7iRm+Tg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=hMfy+BKFNus/IfgQgEkmXZTwz8LdL/U9gmrGSu5zy3o=; b=KKmSgPEr Cl+Lt4K1xGDAzIvICZOogaKLC7ttUYGCMem2lX0ruoCa2Ef9NKFMEvrFtFOGBlKU qii0aGHpsWXaHNj/gNPmb+tF6p2P2gXRnQgm2CCKmELvSi+IlUW3gFlvoR3YPu0c 9bdKpVa3nZQT09fSGSoWbagaonOie9dLhYf0niYA3OgRznktcCaPIbSs5QFxRyOU iOPAfeAX2eHbhRHhP0jM+mzINaWuygrkQoE5wH6rXFNNNd8ljOThmYewAmKeoyyt LWeC1e2Mncsg2c5VP5nbtxE6w+5NTJdUHGA8CEJfA0IGjV4jgDYMp71zwjASQxDf YjZwY/HNqDHtYA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrvdegvddgfeejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffkofgjfhgggfestdekre dtredttdenucfhrhhomheplfhoshhhucffvghkkhgvrhcuoehjohhshhesihhtrghnihhm uhhlrdhliheqnecuggftrfgrthhtvghrnhepgeduieeltdeltedugfegffehfefgudduhe ehgfejgeeuiedukedvtdejiefgheeunecuffhomhgrihhnpehnvghonhdrshgsnecukfhp peekkedrudeftddrgeekrddufeejnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrg hmpehmrghilhhfrhhomhepjhhoshhhsehithgrnhhimhhulhdrlhhi X-ME-Proxy: Received: from localhost.localdomain (mue-88-130-48-137.dsl.tropolys.de [88.130.48.137]) by mail.messagingengine.com (Postfix) with ESMTPA id 9600024005A for ; Thu, 7 Jan 2021 07:10:26 -0500 (EST) From: Josh Dekker To: ffmpeg-devel@ffmpeg.org Date: Thu, 7 Jan 2021 13:10:19 +0100 Message-Id: <20210107121020.86179-4-josh@itanimul.li> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20210107121020.86179-1-josh@itanimul.li> References: <20210107121020.86179-1-josh@itanimul.li> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/4] lavc/aarch64: add HEVC sao_band NEON X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Only works for 8x8. Signed-off-by: Josh Dekker --- libavcodec/aarch64/Makefile | 3 +- libavcodec/aarch64/hevcdsp_init.c | 7 +++ libavcodec/aarch64/hevcdsp_sao_neon.S | 87 +++++++++++++++++++++++++++ 3 files changed, 96 insertions(+), 1 deletion(-) create mode 100644 libavcodec/aarch64/hevcdsp_sao_neon.S diff --git a/libavcodec/aarch64/Makefile b/libavcodec/aarch64/Makefile index 42d80bf74c..1f54fc31f4 100644 --- a/libavcodec/aarch64/Makefile +++ b/libavcodec/aarch64/Makefile @@ -55,7 +55,8 @@ NEON-OBJS-$(CONFIG_VP8DSP) += aarch64/vp8dsp_neon.o NEON-OBJS-$(CONFIG_AAC_DECODER) += aarch64/aacpsdsp_neon.o NEON-OBJS-$(CONFIG_DCA_DECODER) += aarch64/synth_filter_neon.o NEON-OBJS-$(CONFIG_HEVC_DECODER) += aarch64/hevcdsp_add_res_neon.o \ - aarch64/hevcdsp_idct_neon.o + aarch64/hevcdsp_idct_neon.o \ + aarch64/hevcdsp_sao_neon.o NEON-OBJS-$(CONFIG_OPUS_DECODER) += aarch64/opusdsp_neon.o NEON-OBJS-$(CONFIG_VORBIS_DECODER) += aarch64/vorbisdsp_neon.o NEON-OBJS-$(CONFIG_VP9_DECODER) += aarch64/vp9itxfm_16bpp_neon.o \ diff --git a/libavcodec/aarch64/hevcdsp_init.c b/libavcodec/aarch64/hevcdsp_init.c index 2cd7ef3a6c..8f0a923ab1 100644 --- a/libavcodec/aarch64/hevcdsp_init.c +++ b/libavcodec/aarch64/hevcdsp_init.c @@ -23,6 +23,11 @@ #include "libavcodec/hevcdsp.h" #include "libavcodec/avcodec.h" +void ff_hevc_sao_band_filter_8x8_8_neon(uint8_t *_dst, uint8_t *_src, + ptrdiff_t stride_dst, ptrdiff_t stride_src, + int16_t *sao_offset_val, int sao_left_class, + int width, int height); + void ff_hevc_idct_4x4_dc_8_neon(int16_t *coeffs); void ff_hevc_idct_8x8_dc_8_neon(int16_t *coeffs); void ff_hevc_idct_16x16_dc_8_neon(int16_t *coeffs); @@ -53,6 +58,8 @@ av_cold void ff_hevc_dsp_init_aarch64(HEVCDSPContext *c, const int bit_depth) { int cpu_flags = av_get_cpu_flags(); if (have_neon(cpu_flags) && bit_depth == 8) { + c->sao_band_filter[0] = ff_hevc_sao_band_filter_8x8_8_neon; + c->add_residual[0] = ff_hevc_add_residual_4x4_8_neon; c->add_residual[1] = ff_hevc_add_residual_8x8_8_neon; c->add_residual[2] = ff_hevc_add_residual_16x16_8_neon; diff --git a/libavcodec/aarch64/hevcdsp_sao_neon.S b/libavcodec/aarch64/hevcdsp_sao_neon.S new file mode 100644 index 0000000000..25b6c25117 --- /dev/null +++ b/libavcodec/aarch64/hevcdsp_sao_neon.S @@ -0,0 +1,87 @@ +/* -*-arm64-*- + * vim: syntax=arm64asm + * + * AArch64 NEON optimised SAO functions for HEVC decoding + * + * Copyright (c) 2020 Josh Dekker + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/aarch64/asm.S" + +// void sao_band_filter(uint8_t *_dst, uint8_t *_src, +// ptrdiff_t stride_dst, ptrdiff_t stride_src, +// int16_t *sao_offset_val, int sao_left_class, +// int width, int height) +function ff_hevc_sao_band_filter_8x8_8_neon, export=1 + sub sp, sp, #64 + stp xzr, xzr, [sp] + stp xzr, xzr, [sp, #16] + stp xzr, xzr, [sp, #32] + stp xzr, xzr, [sp, #48] + mov w8, #4 +.setup: + ldrsh x9, [x4, x8, lsl #1] // x9 = sao_offset_val[k+1] + subs w8, w8, #1 + add w10, w8, w5 // x10 = k + sao_left_class + and w10, w10, #0x1F + strh w9, [sp, x10, lsl #1] + bne .setup + ld1 {v16.16B-v19.16B}, [sp], #64 + movi v20.8H, #1 +0: // beginning of line + mov w8, w6 +8: + // Simple layout for accessing 16bit values + // with 8bit LUT. + // + // 00 01 02 03 04 05 06 07 + // +-----------------------------------> + // |xDE#xAD|xCA#xFE|xBE#xEF|xFE#xED|.... + // +-----------------------------------> + // i-0 i-1 i-2 i-3 + // dst[x] = av_clip_pixel(src[x] + offset_table[src[x] >> shift]); + ld1 {v2.8B}, [x1] + // load src[x] + ushll v0.8H, v2.8B, #0 + // >> shift + ushr v2.8H, v0.8H, #3 // BIT_DEPTH - 3 + // x2 (access lower short) + shl v1.8H, v2.8H, #1 // low (x2, accessing short) + // +1 access upper short + add v3.8H, v1.8H, v20.8H + // shift insert index to upper byte + sli v1.8H, v3.8H, #8 + // table + tbx v2.16B, {v16.16B-v19.16B}, v1.16B + // src[x] + table + add v1.8H, v0.8H, v2.8H + // clip + narrow + sqxtun v4.8B, v1.8H + // store + st1 {v4.8B}, [x0] + // done 8 pixels + subs w8, w8, #8 + bne 8b + // finished line + subs w7, w7, #1 + add x0, x0, x2 // dst += stride_dst + add x1, x1, x3 // src += stride_src + bne 0b + ret +endfunc From patchwork Thu Jan 7 12:10:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Dekker X-Patchwork-Id: 24831 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 47E3344AF79 for ; Thu, 7 Jan 2021 14:10:38 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2A16A68A49E; Thu, 7 Jan 2021 14:10:38 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C02E2689F86 for ; Thu, 7 Jan 2021 14:10:29 +0200 (EET) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id 27DE71833; Thu, 7 Jan 2021 07:10:28 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Thu, 07 Jan 2021 07:10:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=itanimul.li; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm3; bh=zp+pY5z5e85M0 LX42xoitoucDBCJgQw583AaOQuh684=; b=f3RMWYQIN7QBzBjH+bopWUearvOXE gmEAxy5yf4P6zVen471eDIeJAYlssIH72qLiwqxv+cjjcQAZsohemoSxW+MjA+LE +bRa9T1sXo6v06UoGNzG6VeSFJgEa5LrhMPxdloBpQDwM76uWP+H23T/AjjiXw7J CBTA2LOfHsMpk5q5XWVoxOwIQgIuSOcUXDrIGjhvGLW3B7/6ZpxyFoBxZMhsae1d +0zZiX5xovNBfFrWsdYfuZF/eTUqx1i+j/VluR1rbzawqy6JiBhQD+wwWBGD1myI v0oEzQTR2Y7SZGHQDrCRi47ZAkhA56Vu4MWXGS5TAVmFOERPz6JVylR7Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=zp+pY5z5e85M0LX42xoitoucDBCJgQw583AaOQuh684=; b=mJMFbn1Y 0tEK77c/a8CC3iuF7j18FKtw2dLX4F3Y2BovaJJJ5Y8+EHuNzfVWQw4IAXrHZ5h5 32fpAA12PRDbfvhI0oQnBDsM0t4hRFFZVlm9E+dzwoHu4YCRXKV7x96WApoTud3h pi2cVsq8RQP+mX4bpWUeNlumsPzK8l1ILt1waGMGOzjMhyp5YGcvRQNbyqOHii9f khxK/luwN/tTcFxfJ9AUaeRZGsVbwHMxImsZp65eM0Xk/tO+oAETxpNBBmC5x6Mp jKcL4wk+2SIvv/D5nMmWSDcu8DH6y97MXZm8BPKyrWwOJDSi/3txlio0iefh1QrB v2YT3Cqa5mcyzA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrvdegvddgfeejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucgfrhhlucfvnfffucdlfeehmdenucfjughrpefhvf fufffkofgjfhgggfestdekredtredttdenucfhrhhomheplfhoshhhucffvghkkhgvrhcu oehjohhshhesihhtrghnihhmuhhlrdhliheqnecuggftrfgrthhtvghrnhepfedvvdeutd efuedthfffudeludekjeehuddvheeghedtledvgeelhffgteelvdefnecukfhppeekkedr udeftddrgeekrddufeejnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg hilhhfrhhomhepjhhoshhhsehithgrnhhimhhulhdrlhhi X-ME-Proxy: Received: from localhost.localdomain (mue-88-130-48-137.dsl.tropolys.de [88.130.48.137]) by mail.messagingengine.com (Postfix) with ESMTPA id 219AD240069; Thu, 7 Jan 2021 07:10:27 -0500 (EST) From: Josh Dekker To: ffmpeg-devel@ffmpeg.org Date: Thu, 7 Jan 2021 13:10:20 +0100 Message-Id: <20210107121020.86179-5-josh@itanimul.li> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20210107121020.86179-1-josh@itanimul.li> References: <20210107121020.86179-1-josh@itanimul.li> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/4] checkasm: add hevc_pel tests X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Co-authored-by: Niklas Haas Signed-off-by: Josh Dekker --- tests/checkasm/Makefile | 2 +- tests/checkasm/checkasm.c | 10 + tests/checkasm/checkasm.h | 10 + tests/checkasm/hevc_pel.c | 523 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 544 insertions(+), 1 deletion(-) create mode 100644 tests/checkasm/hevc_pel.c diff --git a/tests/checkasm/Makefile b/tests/checkasm/Makefile index 9e9569777b..1827a4e134 100644 --- a/tests/checkasm/Makefile +++ b/tests/checkasm/Makefile @@ -24,7 +24,7 @@ AVCODECOBJS-$(CONFIG_HUFFYUV_DECODER) += huffyuvdsp.o AVCODECOBJS-$(CONFIG_JPEG2000_DECODER) += jpeg2000dsp.o AVCODECOBJS-$(CONFIG_OPUS_DECODER) += opusdsp.o AVCODECOBJS-$(CONFIG_PIXBLOCKDSP) += pixblockdsp.o -AVCODECOBJS-$(CONFIG_HEVC_DECODER) += hevc_add_res.o hevc_idct.o hevc_sao.o +AVCODECOBJS-$(CONFIG_HEVC_DECODER) += hevc_add_res.o hevc_idct.o hevc_sao.o hevc_pel.o AVCODECOBJS-$(CONFIG_UTVIDEO_DECODER) += utvideodsp.o AVCODECOBJS-$(CONFIG_V210_DECODER) += v210dec.o AVCODECOBJS-$(CONFIG_V210_ENCODER) += v210enc.o diff --git a/tests/checkasm/checkasm.c b/tests/checkasm/checkasm.c index b3ac76c325..8338e8ff58 100644 --- a/tests/checkasm/checkasm.c +++ b/tests/checkasm/checkasm.c @@ -116,6 +116,16 @@ static const struct { #if CONFIG_HEVC_DECODER { "hevc_add_res", checkasm_check_hevc_add_res }, { "hevc_idct", checkasm_check_hevc_idct }, + { "hevc_qpel", checkasm_check_hevc_qpel }, + { "hevc_qpel_uni", checkasm_check_hevc_qpel_uni }, + { "hevc_qpel_uni_w", checkasm_check_hevc_qpel_uni_w }, + { "hevc_qpel_bi", checkasm_check_hevc_qpel_bi }, + { "hevc_qpel_bi_w", checkasm_check_hevc_qpel_bi_w }, + { "hevc_epel", checkasm_check_hevc_epel }, + { "hevc_epel_uni", checkasm_check_hevc_epel_uni }, + { "hevc_epel_uni_w", checkasm_check_hevc_epel_uni_w }, + { "hevc_epel_bi", checkasm_check_hevc_epel_bi }, + { "hevc_epel_bi_w", checkasm_check_hevc_epel_bi_w }, { "hevc_sao", checkasm_check_hevc_sao }, #endif #if CONFIG_HUFFYUV_DECODER diff --git a/tests/checkasm/checkasm.h b/tests/checkasm/checkasm.h index 0190bc912c..ef6645e3a2 100644 --- a/tests/checkasm/checkasm.h +++ b/tests/checkasm/checkasm.h @@ -58,6 +58,16 @@ void checkasm_check_h264pred(void); void checkasm_check_h264qpel(void); void checkasm_check_hevc_add_res(void); void checkasm_check_hevc_idct(void); +void checkasm_check_hevc_qpel(void); +void checkasm_check_hevc_qpel_uni(void); +void checkasm_check_hevc_qpel_uni_w(void); +void checkasm_check_hevc_qpel_bi(void); +void checkasm_check_hevc_qpel_bi_w(void); +void checkasm_check_hevc_epel(void); +void checkasm_check_hevc_epel_uni(void); +void checkasm_check_hevc_epel_uni_w(void); +void checkasm_check_hevc_epel_bi(void); +void checkasm_check_hevc_epel_bi_w(void); void checkasm_check_hevc_sao(void); void checkasm_check_huffyuvdsp(void); void checkasm_check_jpeg2000dsp(void); diff --git a/tests/checkasm/hevc_pel.c b/tests/checkasm/hevc_pel.c new file mode 100644 index 0000000000..236404f8ff --- /dev/null +++ b/tests/checkasm/hevc_pel.c @@ -0,0 +1,523 @@ +/* + * Copyright (c) 2015 Henrik Gramner + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with FFmpeg; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + */ + +#include +#include "checkasm.h" +#include "libavcodec/hevcdsp.h" +#include "libavutil/common.h" +#include "libavutil/internal.h" +#include "libavutil/intreadwrite.h" + +static const uint32_t pixel_mask[] = { 0xffffffff, 0x01ff01ff, 0x03ff03ff, 0x07ff07ff, 0x0fff0fff }; +static const uint32_t pixel_mask16[] = { 0x00ff00ff, 0x01ff01ff, 0x03ff03ff, 0x07ff07ff, 0x0fff0fff }; +static const int sizes[] = { -1, 4, 6, 8, 12, 16, 24, 32, 48, 64 }; +static const int weights[] = { 0, 128, 255, -1 }; +static const int denoms[] = {0, 7, 12, -1 }; +static const int offsets[] = {0, 255, -1 }; + +#define SIZEOF_PIXEL ((bit_depth + 7) / 8) +#define BUF_SIZE (2 * MAX_PB_SIZE * (2 * 4 + MAX_PB_SIZE)) + +#define randomize_buffers() \ + do { \ + uint32_t mask = pixel_mask[bit_depth - 8]; \ + int k; \ + for (k = 0; k < BUF_SIZE; k += 4) { \ + uint32_t r = rnd() & mask; \ + AV_WN32A(buf0 + k, r); \ + AV_WN32A(buf1 + k, r); \ + r = rnd(); \ + AV_WN32A(dst0 + k, r); \ + AV_WN32A(dst1 + k, r); \ + } \ + } while (0) + +#define randomize_buffers_ref() \ + randomize_buffers(); \ + do { \ + uint32_t mask = pixel_mask16[bit_depth - 8]; \ + int k; \ + for (k = 0; k < BUF_SIZE; k += 2) { \ + uint32_t r = rnd() & mask; \ + AV_WN32A(ref0 + k, r); \ + AV_WN32A(ref1 + k, r); \ + } \ + } while (0) + +#define src0 (buf0 + 2 * 4 * MAX_PB_SIZE) /* hevc qpel functions read data from negative src pointer offsets */ +#define src1 (buf1 + 2 * 4 * MAX_PB_SIZE) + +void checkasm_check_hevc_qpel(void) +{ + LOCAL_ALIGNED_32(uint8_t, buf0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, buf1, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst1, [BUF_SIZE]); + + HEVCDSPContext h; + int size, bit_depth, i, j, row; + declare_func_emms(AV_CPU_FLAG_MMX | AV_CPU_FLAG_MMXEXT, void, int16_t *dst, uint8_t *src, ptrdiff_t srcstride, + int height, intptr_t mx, intptr_t my, int width); + + for (bit_depth = 8; bit_depth <= 12; bit_depth++) { + ff_hevc_dsp_init(&h, bit_depth); + + for (i = 0; i < 2; i++) { + for (j = 0; j < 2; j++) { + for (size = 1; size < 10; size++) { + const char *type; + switch ((j << 1) | i) { + case 0: type = "pel_pixels"; break; // 0 0 + case 1: type = "qpel_h"; break; // 0 1 + case 2: type = "qpel_v"; break; // 1 0 + case 3: type = "qpel_hv"; break; // 1 1 + } + + if (check_func(h.put_hevc_qpel[size][j][i], "put_hevc_%s%d_%d", type, sizes[size], bit_depth)) { + int16_t *dstw0 = (int16_t *) dst0, *dstw1 = (int16_t *) dst1; + randomize_buffers(); + call_ref(dstw0, src0, sizes[size] * SIZEOF_PIXEL, sizes[size], i, j, sizes[size]); + call_new(dstw1, src1, sizes[size] * SIZEOF_PIXEL, sizes[size], i, j, sizes[size]); + for (row = 0; row < size[sizes]; row++) { + if (memcmp(dstw0 + row * MAX_PB_SIZE, dstw1 + row * MAX_PB_SIZE, sizes[size] * SIZEOF_PIXEL)) + fail(); + } + bench_new(dstw1, src1, sizes[size] * SIZEOF_PIXEL, sizes[size], i, j, sizes[size]); + } + } + } + } + } + report("qpel"); +} + +void checkasm_check_hevc_qpel_uni(void) +{ + LOCAL_ALIGNED_32(uint8_t, buf0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, buf1, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst1, [BUF_SIZE]); + + HEVCDSPContext h; + int size, bit_depth, i, j; + declare_func_emms(AV_CPU_FLAG_MMX | AV_CPU_FLAG_MMXEXT, void, uint8_t *dst, ptrdiff_t dststride, uint8_t *src, ptrdiff_t srcstride, + int height, intptr_t mx, intptr_t my, int width); + + for (bit_depth = 8; bit_depth <= 12; bit_depth++) { + ff_hevc_dsp_init(&h, bit_depth); + + for (i = 0; i < 2; i++) { + for (j = 0; j < 2; j++) { + for (size = 1; size < 10; size++) { + const char *type; + switch ((j << 1) | i) { + case 0: type = "pel_uni_pixels"; break; // 0 0 + case 1: type = "qpel_uni_h"; break; // 0 1 + case 2: type = "qpel_uni_v"; break; // 1 0 + case 3: type = "qpel_uni_hv"; break; // 1 1 + } + + if (check_func(h.put_hevc_qpel_uni[size][j][i], "put_hevc_%s%d_%d", type, sizes[size], bit_depth)) { + randomize_buffers(); + call_ref(dst0, sizes[size] * SIZEOF_PIXEL, src0, sizes[size] * SIZEOF_PIXEL, sizes[size], i, j, sizes[size]); + call_new(dst1, sizes[size] * SIZEOF_PIXEL, src1, sizes[size] * SIZEOF_PIXEL, sizes[size], i, j, sizes[size]); + if (memcmp(dst0, dst1, sizes[size] * sizes[size] * SIZEOF_PIXEL)) + fail(); + bench_new(dst1, sizes[size] * SIZEOF_PIXEL, src1, sizes[size] * SIZEOF_PIXEL, sizes[size], i, j, sizes[size]); + } + } + } + } + } + report("qpel_uni"); +} + +void checkasm_check_hevc_qpel_uni_w(void) +{ + LOCAL_ALIGNED_32(uint8_t, buf0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, buf1, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst1, [BUF_SIZE]); + + HEVCDSPContext h; + int size, bit_depth, i, j; + const int *denom, *wx, *ox; + declare_func_emms(AV_CPU_FLAG_MMX | AV_CPU_FLAG_MMXEXT, void, uint8_t *dst, ptrdiff_t dststride, uint8_t *src, ptrdiff_t srcstride, + int height, int denom, int wx, int ox, intptr_t mx, intptr_t my, int width); + + for (bit_depth = 8; bit_depth <= 12; bit_depth++) { + ff_hevc_dsp_init(&h, bit_depth); + + for (i = 0; i < 2; i++) { + for (j = 0; j < 2; j++) { + for (size = 1; size < 10; size++) { + const char *type; + switch ((j << 1) | i) { + case 0: type = "pel_uni_w_pixels"; break; // 0 0 + case 1: type = "qpel_uni_w_h"; break; // 0 1 + case 2: type = "qpel_uni_w_v"; break; // 1 0 + case 3: type = "qpel_uni_w_hv"; break; // 1 1 + } + + if (check_func(h.put_hevc_qpel_uni_w[size][j][i], "put_hevc_%s%d_%d", type, sizes[size], bit_depth)) { + for (denom = denoms; *denom >= 0; denom++) { + for (wx = weights; *wx >= 0; wx++) { + for (ox = offsets; *ox >= 0; ox++) { + randomize_buffers(); + call_ref(dst0, sizes[size] * SIZEOF_PIXEL, src0, sizes[size] * SIZEOF_PIXEL, sizes[size], *denom, *wx, *ox, i, j, sizes[size]); + call_new(dst1, sizes[size] * SIZEOF_PIXEL, src1, sizes[size] * SIZEOF_PIXEL, sizes[size], *denom, *wx, *ox, i, j, sizes[size]); + if (memcmp(dst0, dst1, sizes[size] * sizes[size] * SIZEOF_PIXEL)) + fail(); + bench_new(dst1, sizes[size] * SIZEOF_PIXEL, src1, sizes[size] * SIZEOF_PIXEL, sizes[size], *denom, *wx, *ox, i, j, sizes[size]); + } + } + } + } + } + } + } + } + report("qpel_uni_w"); +} + +void checkasm_check_hevc_qpel_bi(void) +{ + LOCAL_ALIGNED_32(uint8_t, buf0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, buf1, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst1, [BUF_SIZE]); + LOCAL_ALIGNED_32(int16_t, ref0, [BUF_SIZE]); + LOCAL_ALIGNED_32(int16_t, ref1, [BUF_SIZE]); + + HEVCDSPContext h; + int size, bit_depth, i, j; + declare_func_emms(AV_CPU_FLAG_MMX | AV_CPU_FLAG_MMXEXT, void, uint8_t *dst, ptrdiff_t dststride, uint8_t *src, ptrdiff_t srcstride, + int16_t *src2, + int height, intptr_t mx, intptr_t my, int width); + + for (bit_depth = 8; bit_depth <= 12; bit_depth++) { + ff_hevc_dsp_init(&h, bit_depth); + + for (i = 0; i < 2; i++) { + for (j = 0; j < 2; j++) { + for (size = 1; size < 10; size++) { + const char *type; + switch ((j << 1) | i) { + case 0: type = "pel_bi_pixels"; break; // 0 0 + case 1: type = "qpel_bi_h"; break; // 0 1 + case 2: type = "qpel_bi_v"; break; // 1 0 + case 3: type = "qpel_bi_hv"; break; // 1 1 + } + + if (check_func(h.put_hevc_qpel_bi[size][j][i], "put_hevc_%s%d_%d", type, sizes[size], bit_depth)) { + randomize_buffers_ref(); + call_ref(dst0, sizes[size] * SIZEOF_PIXEL, src0, sizes[size] * SIZEOF_PIXEL, ref0, sizes[size], i, j, sizes[size]); + call_new(dst1, sizes[size] * SIZEOF_PIXEL, src1, sizes[size] * SIZEOF_PIXEL, ref1, sizes[size], i, j, sizes[size]); + if (memcmp(dst0, dst1, sizes[size] * sizes[size] * SIZEOF_PIXEL)) + fail(); + bench_new(dst1, sizes[size] * SIZEOF_PIXEL, src1, sizes[size] * SIZEOF_PIXEL, ref1, sizes[size], i, j, sizes[size]); + } + } + } + } + } + report("qpel_bi"); +} + +void checkasm_check_hevc_qpel_bi_w(void) +{ + LOCAL_ALIGNED_32(uint8_t, buf0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, buf1, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst1, [BUF_SIZE]); + LOCAL_ALIGNED_32(int16_t, ref0, [BUF_SIZE]); + LOCAL_ALIGNED_32(int16_t, ref1, [BUF_SIZE]); + + HEVCDSPContext h; + int size, bit_depth, i, j; + const int *denom, *wx, *ox; + declare_func_emms(AV_CPU_FLAG_MMX | AV_CPU_FLAG_MMXEXT, void, uint8_t *dst, ptrdiff_t dststride, uint8_t *src, ptrdiff_t srcstride, + int16_t *src2, + int height, int denom, int wx0, int wx1, + int ox0, int ox1, intptr_t mx, intptr_t my, int width); + + for (bit_depth = 8; bit_depth <= 12; bit_depth++) { + ff_hevc_dsp_init(&h, bit_depth); + + for (i = 0; i < 2; i++) { + for (j = 0; j < 2; j++) { + for (size = 1; size < 10; size++) { + const char *type; + switch ((j << 1) | i) { + case 0: type = "pel_bi_w_pixels"; break; // 0 0 + case 1: type = "qpel_bi_w_h"; break; // 0 1 + case 2: type = "qpel_bi_w_v"; break; // 1 0 + case 3: type = "qpel_bi_w_hv"; break; // 1 1 + } + + if (check_func(h.put_hevc_qpel_bi_w[size][j][i], "put_hevc_%s%d_%d", type, sizes[size], bit_depth)) { + for (denom = denoms; *denom >= 0; denom++) { + for (wx = weights; *wx >= 0; wx++) { + for (ox = offsets; *ox >= 0; ox++) { + randomize_buffers_ref(); + call_ref(dst0, sizes[size] * SIZEOF_PIXEL, src0, sizes[size] * SIZEOF_PIXEL, ref0, sizes[size], *denom, *wx, *wx, *ox, *ox, i, j, sizes[size]); + call_new(dst1, sizes[size] * SIZEOF_PIXEL, src1, sizes[size] * SIZEOF_PIXEL, ref1, sizes[size], *denom, *wx, *wx, *ox, *ox, i, j, sizes[size]); + if (memcmp(dst0, dst1, sizes[size] * sizes[size] * SIZEOF_PIXEL)) + fail(); + bench_new(dst1, sizes[size] * SIZEOF_PIXEL, src1, sizes[size] * SIZEOF_PIXEL, ref1, sizes[size], *denom, *wx, *wx, *ox, *ox, i, j, sizes[size]); + } + } + } + } + } + } + } + } + report("qpel_bi_w"); +} + +void checkasm_check_hevc_epel(void) +{ + LOCAL_ALIGNED_32(uint8_t, buf0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, buf1, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst1, [BUF_SIZE]); + + HEVCDSPContext h; + int size, bit_depth, i, j, row; + declare_func_emms(AV_CPU_FLAG_MMX | AV_CPU_FLAG_MMXEXT, void, int16_t *dst, uint8_t *src, ptrdiff_t srcstride, + int height, intptr_t mx, intptr_t my, int width); + + for (bit_depth = 8; bit_depth <= 12; bit_depth++) { + ff_hevc_dsp_init(&h, bit_depth); + + for (i = 0; i < 2; i++) { + for (j = 0; j < 2; j++) { + for (size = 1; size < 10; size++) { + const char *type; + switch ((j << 1) | i) { + case 0: type = "pel_pixels"; break; // 0 0 + case 1: type = "epel_h"; break; // 0 1 + case 2: type = "epel_v"; break; // 1 0 + case 3: type = "epel_hv"; break; // 1 1 + } + + if (check_func(h.put_hevc_epel[size][j][i], "put_hevc_%s%d_%d", type, sizes[size], bit_depth)) { + int16_t *dstw0 = (int16_t *) dst0, *dstw1 = (int16_t *) dst1; + randomize_buffers(); + call_ref(dstw0, src0, sizes[size] * SIZEOF_PIXEL, sizes[size], i, j, sizes[size]); + call_new(dstw1, src1, sizes[size] * SIZEOF_PIXEL, sizes[size], i, j, sizes[size]); + for (row = 0; row < size[sizes]; row++) { + if (memcmp(dstw0 + row * MAX_PB_SIZE, dstw1 + row * MAX_PB_SIZE, sizes[size] * SIZEOF_PIXEL)) + fail(); + } + bench_new(dstw1, src1, sizes[size] * SIZEOF_PIXEL, sizes[size], i, j, sizes[size]); + } + } + } + } + } + report("epel"); +} + +void checkasm_check_hevc_epel_uni(void) +{ + LOCAL_ALIGNED_32(uint8_t, buf0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, buf1, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst1, [BUF_SIZE]); + + HEVCDSPContext h; + int size, bit_depth, i, j; + declare_func_emms(AV_CPU_FLAG_MMX | AV_CPU_FLAG_MMXEXT, void, uint8_t *dst, ptrdiff_t dststride, uint8_t *src, ptrdiff_t srcstride, + int height, intptr_t mx, intptr_t my, int width); + + for (bit_depth = 8; bit_depth <= 12; bit_depth++) { + ff_hevc_dsp_init(&h, bit_depth); + + for (i = 0; i < 2; i++) { + for (j = 0; j < 2; j++) { + for (size = 1; size < 10; size++) { + const char *type; + switch ((j << 1) | i) { + case 0: type = "pel_uni_pixels"; break; // 0 0 + case 1: type = "epel_uni_h"; break; // 0 1 + case 2: type = "epel_uni_v"; break; // 1 0 + case 3: type = "epel_uni_hv"; break; // 1 1 + } + + if (check_func(h.put_hevc_epel_uni[size][j][i], "put_hevc_%s%d_%d", type, sizes[size], bit_depth)) { + randomize_buffers(); + call_ref(dst0, sizes[size] * SIZEOF_PIXEL, src0, sizes[size] * SIZEOF_PIXEL, sizes[size], i, j, sizes[size]); + call_new(dst1, sizes[size] * SIZEOF_PIXEL, src1, sizes[size] * SIZEOF_PIXEL, sizes[size], i, j, sizes[size]); + if (memcmp(dst0, dst1, sizes[size] * sizes[size] * SIZEOF_PIXEL)) + fail(); + bench_new(dst1, sizes[size] * SIZEOF_PIXEL, src1, sizes[size] * SIZEOF_PIXEL, sizes[size], i, j, sizes[size]); + } + } + } + } + } + report("epel_uni"); +} + +void checkasm_check_hevc_epel_uni_w(void) +{ + LOCAL_ALIGNED_32(uint8_t, buf0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, buf1, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst1, [BUF_SIZE]); + + HEVCDSPContext h; + int size, bit_depth, i, j; + const int *denom, *wx, *ox; + declare_func_emms(AV_CPU_FLAG_MMX | AV_CPU_FLAG_MMXEXT, void, uint8_t *dst, ptrdiff_t dststride, uint8_t *src, ptrdiff_t srcstride, + int height, int denom, int wx, int ox, intptr_t mx, intptr_t my, int width); + + for (bit_depth = 8; bit_depth <= 12; bit_depth++) { + ff_hevc_dsp_init(&h, bit_depth); + + for (i = 0; i < 2; i++) { + for (j = 0; j < 2; j++) { + for (size = 1; size < 10; size++) { + const char *type; + switch ((j << 1) | i) { + case 0: type = "pel_uni_w_pixels"; break; // 0 0 + case 1: type = "epel_uni_w_h"; break; // 0 1 + case 2: type = "epel_uni_w_v"; break; // 1 0 + case 3: type = "epel_uni_w_hv"; break; // 1 1 + } + + if (check_func(h.put_hevc_epel_uni_w[size][j][i], "put_hevc_%s%d_%d", type, sizes[size], bit_depth)) { + for (denom = denoms; *denom >= 0; denom++) { + for (wx = weights; *wx >= 0; wx++) { + for (ox = offsets; *ox >= 0; ox++) { + randomize_buffers(); + call_ref(dst0, sizes[size] * SIZEOF_PIXEL, src0, sizes[size] * SIZEOF_PIXEL, sizes[size], *denom, *wx, *ox, i, j, sizes[size]); + call_new(dst1, sizes[size] * SIZEOF_PIXEL, src1, sizes[size] * SIZEOF_PIXEL, sizes[size], *denom, *wx, *ox, i, j, sizes[size]); + if (memcmp(dst0, dst1, sizes[size] * sizes[size] * SIZEOF_PIXEL)) + fail(); + bench_new(dst1, sizes[size] * SIZEOF_PIXEL, src1, sizes[size] * SIZEOF_PIXEL, sizes[size], *denom, *wx, *ox, i, j, sizes[size]); + } + } + } + } + } + } + } + } + report("epel_uni_w"); +} + +void checkasm_check_hevc_epel_bi(void) +{ + LOCAL_ALIGNED_32(uint8_t, buf0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, buf1, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst1, [BUF_SIZE]); + LOCAL_ALIGNED_32(int16_t, ref0, [BUF_SIZE]); + LOCAL_ALIGNED_32(int16_t, ref1, [BUF_SIZE]); + + HEVCDSPContext h; + int size, bit_depth, i, j; + declare_func_emms(AV_CPU_FLAG_MMX | AV_CPU_FLAG_MMXEXT, void, uint8_t *dst, ptrdiff_t dststride, uint8_t *src, ptrdiff_t srcstride, + int16_t *src2, + int height, intptr_t mx, intptr_t my, int width); + + for (bit_depth = 8; bit_depth <= 12; bit_depth++) { + ff_hevc_dsp_init(&h, bit_depth); + + for (i = 0; i < 2; i++) { + for (j = 0; j < 2; j++) { + for (size = 1; size < 10; size++) { + const char *type; + switch ((j << 1) | i) { + case 0: type = "pel_bi_pixels"; break; // 0 0 + case 1: type = "epel_bi_h"; break; // 0 1 + case 2: type = "epel_bi_v"; break; // 1 0 + case 3: type = "epel_bi_hv"; break; // 1 1 + } + + if (check_func(h.put_hevc_epel_bi[size][j][i], "put_hevc_%s%d_%d", type, sizes[size], bit_depth)) { + randomize_buffers_ref(); + call_ref(dst0, sizes[size] * SIZEOF_PIXEL, src0, sizes[size] * SIZEOF_PIXEL, ref0, sizes[size], i, j, sizes[size]); + call_new(dst1, sizes[size] * SIZEOF_PIXEL, src1, sizes[size] * SIZEOF_PIXEL, ref1, sizes[size], i, j, sizes[size]); + if (memcmp(dst0, dst1, sizes[size] * sizes[size] * SIZEOF_PIXEL)) + fail(); + bench_new(dst1, sizes[size] * SIZEOF_PIXEL, src1, sizes[size] * SIZEOF_PIXEL, ref1, sizes[size], i, j, sizes[size]); + } + } + } + } + } + report("epel_bi"); +} + +void checkasm_check_hevc_epel_bi_w(void) +{ + LOCAL_ALIGNED_32(uint8_t, buf0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, buf1, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst0, [BUF_SIZE]); + LOCAL_ALIGNED_32(uint8_t, dst1, [BUF_SIZE]); + LOCAL_ALIGNED_32(int16_t, ref0, [BUF_SIZE]); + LOCAL_ALIGNED_32(int16_t, ref1, [BUF_SIZE]); + + HEVCDSPContext h; + int size, bit_depth, i, j; + const int *denom, *wx, *ox; + declare_func_emms(AV_CPU_FLAG_MMX | AV_CPU_FLAG_MMXEXT, void, uint8_t *dst, ptrdiff_t dststride, uint8_t *src, ptrdiff_t srcstride, + int16_t *src2, + int height, int denom, int wx0, int wx1, + int ox0, int ox1, intptr_t mx, intptr_t my, int width); + + for (bit_depth = 8; bit_depth <= 12; bit_depth++) { + ff_hevc_dsp_init(&h, bit_depth); + + for (i = 0; i < 2; i++) { + for (j = 0; j < 2; j++) { + for (size = 1; size < 10; size++) { + const char *type; + switch ((j << 1) | i) { + case 0: type = "pel_bi_w_pixels"; break; // 0 0 + case 1: type = "epel_bi_w_h"; break; // 0 1 + case 2: type = "epel_bi_w_v"; break; // 1 0 + case 3: type = "epel_bi_w_hv"; break; // 1 1 + } + + if (check_func(h.put_hevc_epel_bi_w[size][j][i], "put_hevc_%s%d_%d", type, sizes[size], bit_depth)) { + for (denom = denoms; *denom >= 0; denom++) { + for (wx = weights; *wx >= 0; wx++) { + for (ox = offsets; *ox >= 0; ox++) { + randomize_buffers_ref(); + call_ref(dst0, sizes[size] * SIZEOF_PIXEL, src0, sizes[size] * SIZEOF_PIXEL, ref0, sizes[size], *denom, *wx, *wx, *ox, *ox, i, j, sizes[size]); + call_new(dst1, sizes[size] * SIZEOF_PIXEL, src1, sizes[size] * SIZEOF_PIXEL, ref1, sizes[size], *denom, *wx, *wx, *ox, *ox, i, j, sizes[size]); + if (memcmp(dst0, dst1, sizes[size] * sizes[size] * SIZEOF_PIXEL)) + fail(); + bench_new(dst1, sizes[size] * SIZEOF_PIXEL, src1, sizes[size] * SIZEOF_PIXEL, ref1, sizes[size], *denom, *wx, *wx, *ox, *ox, i, j, sizes[size]); + } + } + } + } + } + } + } + } + report("epel_bi_w"); +}