From patchwork Wed Sep 6 14:24:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Kelly X-Patchwork-Id: 43592 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4f16:b0:149:dfde:5c0a with SMTP id gi22csp2562612pzb; Wed, 6 Sep 2023 07:24:58 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFsdeXSs7v5GT0WhbevCuXupRc2iN94CjCSLr5+W3a0mVXr2xO9gRCbmVytbEDzaRIYaJbu X-Received: by 2002:a17:906:20d6:b0:9a6:5696:3887 with SMTP id c22-20020a17090620d600b009a656963887mr2170399ejc.59.1694010298182; Wed, 06 Sep 2023 07:24:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694010298; cv=none; d=google.com; s=arc-20160816; b=Pzqm94qlIc4kVEuaj/ab78sAuJrIf7IPRJBHN0HmnZSkWxWjkfq1Jx08rAkIlCv7PQ YwJM+bCQDvUGmUqVyKHl7ypJLXp39e4K/uePAluEO+YEvI2VtZE2fElC7HL+P6Mk5tja uJhZmsdz/RbgwxdfkVJ+6qYACehDKA62HFISvef4c7J1qW/BAdIINHdtB9DAMpVgAA/r zjcG20YpTvmWChQPO0zTSldyOAf+TReogmMdg4nIM1mFk/IXoH9dJa3CSvNwgGFSJRfv 1gCCPepJA7Od3rfpSnpOX7Xv1zI+kJgBxi69bt5btN2OZnRv8u/+8PYWYSQReLnOnmt2 8VEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:message-id:mime-version:date :delivered-to; bh=fmPApyhrcQUR/BEUcyPMgsdczbf0OLvuTaVGp0Hbcpc=; fh=pIYkOB0DS4x4m8fX0HHRFiKDzn/WWz+bfA1ms/Md3Hc=; b=cr1zrh6sUgY+zpqwl9WmORhwePK6vBUQY7E5EuoBuTqB/LGijVlhIt+niWFD0tuVGp nA/PsKAZEpTntX1ITeDPGots1o9jhB54H7THBtxUrh0mGFgnqC+kn4PqpsoYOjC3hhsL 7b+8et8oR95z9GgXGKe/iG0O0mvmy771cJzEm58Xvy8OH5kFrsKScELQKBVqj7P2hPCu ZP67diIhpLj1uhsb4Sf+bz7jwU30EUJ/jmjkeQbZ/56yGC6PBE4x3NYqLde7BViJLsDB +XKl3ARRZEyAgydYyVp5TA3kUDonJrI7pT7lA5CYVlIY00SPTgGVufbyNI+Dn5/dePbA XPZw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h22-20020a17090634d600b009922cdfaf62si9776637ejb.42.2023.09.06.07.24.57; Wed, 06 Sep 2023 07:24:58 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 89FCE68BE7B; Wed, 6 Sep 2023 17:24:54 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B85FA68BE7B for ; Wed, 6 Sep 2023 17:24:48 +0300 (EEST) Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-594e5e2e608so37880267b3.2 for ; Wed, 06 Sep 2023 07:24:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1694010287; x=1694615087; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ODT5A64Q5kw3MrvgUNbAZ39zXEH01bKlxMDt+7YQX0o=; b=bmePD5j+s5Wf3ZcdGUydV6f4X7B7bVPfXcU4YyucW0SkIQYbmzYGouwvzzIC9AGekw w6S0CZAC2BhpHXBzPPenWOacjonGpoCRhqvCHZgLKCs6BajQSkpmNtmxnhNA3uxQDgTS BaDnEI9Lbn7gB6gpvseH2tYg8moS5j/bSS5STbaw8N6zei7XuU0pSrR+7HEcCd/+Okc/ zKHHjkBN+offdSP0dRwX8y3w4AZzWnLkCHKoCvSzjh5VoKasBZg2f8pFpyLkzHU57+S2 v1FHyiHoxrlXD4DOcie3E1fPf02jUY5gjpD+3kvoif9SK9hsWNmXsGRcDUk30wRZfYsy E2Tg== X-Gm-Message-State: AOJu0YygPhT+lncHOVcJEFwLVzZ+Bb+Mu3lXag/0vP/KnOV/ZzF9u3Ye myNyAiOleuuVXBOXglXXrGuekcPknUgP+zTveXOngnw6yQsPx1ddEQUvTCHbQLBOSZilzdDTLtY llLeamI/Ckj877XJo1YrcTj7xyVvX1zrY1vaSqwqeSChK6E19z6meyH8n+DqVjPVRtvWNRqU= X-Received: from alankelly0.zrh.corp.google.com ([2a00:79e0:9e:2:7143:767c:f2cf:9d6d]) (user=alankelly job=sendgmr) by 2002:a81:4306:0:b0:58c:8552:458d with SMTP id q6-20020a814306000000b0058c8552458dmr398999ywa.3.1694010286883; Wed, 06 Sep 2023 07:24:46 -0700 (PDT) Date: Wed, 6 Sep 2023 16:24:29 +0200 Mime-Version: 1.0 X-Mailer: git-send-email 2.42.0.283.g2d96d420d3-goog Message-ID: <20230906142442.1634320-1-alankelly@google.com> To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 1/2] swscale/x86/yuv2yuvX: Add yuv2yuvX avx512 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Alan Kelly via ffmpeg-devel From: Alan Kelly Reply-To: FFmpeg development discussions and patches Cc: Alan Kelly Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: WZ84mvGw1LtE --- libswscale/x86/swscale.c | 7 +++++++ libswscale/x86/yuv2yuvX.asm | 19 ++++++++++++++++++- 2 files changed, 25 insertions(+), 1 deletion(-) diff --git a/libswscale/x86/swscale.c b/libswscale/x86/swscale.c index ff16398988..00e42b4bec 100644 --- a/libswscale/x86/swscale.c +++ b/libswscale/x86/swscale.c @@ -225,6 +225,9 @@ YUV2YUVX_FUNC(sse3, 32) #if HAVE_AVX2_EXTERNAL YUV2YUVX_FUNC(avx2, 64) #endif +#if ARCH_X86_64 && HAVE_AVX512_EXTERNAL +YUV2YUVX_FUNC(avx512, 128) +#endif #define SCALE_FUNC(filter_n, from_bpc, to_bpc, opt) \ void ff_hscale ## from_bpc ## to ## to_bpc ## _ ## filter_n ## _ ## opt( \ @@ -467,6 +470,10 @@ av_cold void ff_sws_init_swscale_x86(SwsContext *c) #if HAVE_AVX2_EXTERNAL if (EXTERNAL_AVX2_FAST(cpu_flags)) c->yuv2planeX = yuv2yuvX_avx2; +#endif +#if ARCH_X86_64 && HAVE_AVX512_EXTERNAL + if (EXTERNAL_AVX512ICL(cpu_flags)) + c->yuv2planeX = yuv2yuvX_avx512; #endif } #if ARCH_X86_32 && !HAVE_ALIGNED_STACK diff --git a/libswscale/x86/yuv2yuvX.asm b/libswscale/x86/yuv2yuvX.asm index 369c850674..57bfa09d66 100644 --- a/libswscale/x86/yuv2yuvX.asm +++ b/libswscale/x86/yuv2yuvX.asm @@ -22,6 +22,10 @@ %include "libavutil/x86/x86util.asm" +SECTION_RODATA 64 + +permutation: dq 0, 2, 4, 6, 1, 3, 5, 7 + SECTION .text ;----------------------------------------------------------------------------- @@ -50,6 +54,10 @@ cglobal yuv2yuvX, 7, 7, 8, filter, filterSize, src, dest, dstW, dither, offset %else movq xm3, [ditherq] %endif ; avx2 + +%if cpuflag(avx512) + mova m15, [permutation] +%endif cmp offsetd, 0 jz .offset @@ -109,7 +117,10 @@ cglobal yuv2yuvX, 7, 7, 8, filter, filterSize, src, dest, dstW, dither, offset packuswb m6, m6, m1 %endif mov srcq, [filterq] -%if cpuflag(avx2) +%if cpuflag(avx512) + vpermt2q m3, m15, m3 + vpermt2q m6, m15, m6 +%elif cpuflag(avx2) vpermq m3, m3, 216 vpermq m6, m6, 216 %endif @@ -131,4 +142,10 @@ YUV2YUVX_FUNC %if HAVE_AVX2_EXTERNAL INIT_YMM avx2 YUV2YUVX_FUNC +%if HAVE_AVX512_EXTERNAL +%if ARCH_X86_64 +INIT_ZMM avx512 +YUV2YUVX_FUNC +%endif +%endif %endif From patchwork Wed Sep 6 14:25:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Kelly X-Patchwork-Id: 43593 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4f16:b0:149:dfde:5c0a with SMTP id gi22csp2563000pzb; Wed, 6 Sep 2023 07:25:35 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFL7+MmdR4eNi2j2hk6NzOTZKFeEf+TlkR4Ho/hhwf6TnxnptT6/JVq5HTSGhN0Imuz0+a7 X-Received: by 2002:a05:6402:50d:b0:523:493e:929c with SMTP id m13-20020a056402050d00b00523493e929cmr2487795edv.10.1694010334791; Wed, 06 Sep 2023 07:25:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694010334; cv=none; d=google.com; s=arc-20160816; b=c8MXkuWj/a74j9heKVMAVlueffs4Q3VmsbucoujGbzMIdMw81X420sb2aQWIyZ0PpR +x7Fcl/TBIkuUc+NcRPrlSrhrsbW243vMkz+Dv8PqUnGC7yYZ7QYPi7EUoFFhe6CTC2Q 0tqJLj1vKbVRZk9UP5X0E4tzyruo5hKe6o43Ee5tn1lwB5waKREKMZuByFXxHxwFDbni /bWq4fDlxKBOWgrGuMtIN4Ev251neo3itoDxwVY+XsMPZDoWWiVMSPTtr7QumQIARvqx sW2H4ogvRz8fVxZjVSHu4WH5tiFeuePlNzsJtunOZh5E8k1KE1P6i96/pBMQCdpH8A2T oCfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:message-id:mime-version:date :delivered-to; bh=M6OLiyvLIaVU7q8BmIxPAgR1nT/fy5YPFg3jtH/4YnQ=; fh=pIYkOB0DS4x4m8fX0HHRFiKDzn/WWz+bfA1ms/Md3Hc=; b=NTQmxb7oo8nDlFQxnN5xD1l0+zakMLm7EL8n3qD6DMlEF3/aAIQp9BSOq67pyR3MH4 YuSBe2nKCODybAC0ir9iwwVZZiVgAWi2HH1WDs/iGqrds0BxCRfD/PJWe4Hta9laCAD6 zIFGdRNVe1gPYXm0HFWTuBFJ9R537BwK/XxvtGBP2NDuQg2gaEsbmiyBWf7a+ivdfeOE pY/3ohnyP99vIeseQp40tE+KnfCm7mF2gY1tosKHtuu600Z0tPJhhVawcVsT8697mgb+ BN8mPvEDcofYycYFEFGLxrGEkO3xvxoIch9rERAguSojxXsMh8tsqWzwkaCpbjvdNS0F Qs+g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f26-20020a056402151a00b005256e2d6d7dsi9274963edw.38.2023.09.06.07.25.31; Wed, 06 Sep 2023 07:25:34 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B6A0368C7E7; Wed, 6 Sep 2023 17:25:29 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4DF0368C66A for ; Wed, 6 Sep 2023 17:25:23 +0300 (EEST) Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-59504967dc4so38597577b3.0 for ; Wed, 06 Sep 2023 07:25:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1694010322; x=1694615122; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=4ZO1k/PhjdQ4f0MP6I+laK//0bSuZr4URDjR//YMU3E=; b=cqoDf97qT/x6cZefUV2VoiGEd2FVlA9PdL8WcxAz+Qr9EmSBsolrqya3iuglAgny1o PVZHZvxOLi57H5gu8axlAha9QkN1qYMLVWM8ijnY7FMlwJqmsICZ3H7AWd9Ltv+7ggM/ urFXQ7KP01WtpeVMCoyUGc8mP3Bv2NEVQqFvh/uDaIA4kLhSFk8tsHenh5RgBHcAIA2a /iig/9d/enMsC8DRAOx41aEsfpZvK3thXwKZbqZQRjqYELE+StYAoojQ2areeftDgsTN k1JHdOq5PREVPrffzFzUTSHMaeLKw6+1QkvZqqsfcxvlNkKkjAofzGLP4HzbvnZCUAKu Nhiw== X-Gm-Message-State: AOJu0YwX8WJDHqcr5Hr623AoLYMnu6DWsavfkn/v5alvTw6CzNZMdLUn rWB06ictYMcvJpA7J5WQIow/fTHAEubbFSx2PyVSUrinzCOG1pkqcgqoN9G/a1YFwWj986b2R5N rfRJogMC3pWXx8KsvebWfn/dgUXvR8Mii6YBzfoqeV3t1+skvBXtBHOg6uj2E6PScIMG1yro= X-Received: from alankelly0.zrh.corp.google.com ([2a00:79e0:9e:2:7143:767c:f2cf:9d6d]) (user=alankelly job=sendgmr) by 2002:a81:b709:0:b0:579:e07c:2798 with SMTP id v9-20020a81b709000000b00579e07c2798mr398653ywh.2.1694010322104; Wed, 06 Sep 2023 07:25:22 -0700 (PDT) Date: Wed, 6 Sep 2023 16:25:00 +0200 Mime-Version: 1.0 X-Mailer: git-send-email 2.42.0.283.g2d96d420d3-goog Message-ID: <20230906142518.1634649-1-alankelly@google.com> To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 2/2] swscale/x86/yuv2yuvX: Process tails by jumping back into the main loop. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Alan Kelly via ffmpeg-devel From: Alan Kelly Reply-To: FFmpeg development discussions and patches Cc: Alan Kelly Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 375FhnfLFwBi --- libswscale/x86/swscale.c | 19 ++++++++----------- libswscale/x86/yuv2yuvX.asm | 24 ++++++++++++++++++------ 2 files changed, 26 insertions(+), 17 deletions(-) diff --git a/libswscale/x86/swscale.c b/libswscale/x86/swscale.c index 00e42b4bec..6980002e9e 100644 --- a/libswscale/x86/swscale.c +++ b/libswscale/x86/swscale.c @@ -194,7 +194,7 @@ static void yuv2yuvX_ ##opt(const int16_t *filter, int filterSize, \ return; \ } -#define YUV2YUVX_FUNC(opt, step) \ +#define YUV2YUVX_FUNC(opt, step, tail) \ void ff_yuv2yuvX_ ##opt(const int16_t *filter, int filterSize, int srcOffset, \ uint8_t *dest, int dstW, \ const uint8_t *dither, int offset); \ @@ -202,17 +202,14 @@ static void yuv2yuvX_ ##opt(const int16_t *filter, int filterSize, \ const int16_t **src, uint8_t *dest, int dstW, \ const uint8_t *dither, int offset) \ { \ - int remainder = (dstW % step); \ - int pixelsProcessed = dstW - remainder; \ if(((uintptr_t)dest) & 15){ \ yuv2yuvX_mmxext(filter, filterSize, src, dest, dstW, dither, offset); \ return; \ } \ - if(pixelsProcessed > 0) \ - ff_yuv2yuvX_ ##opt(filter, filterSize - 1, 0, dest - offset, pixelsProcessed + offset, dither, offset); \ - if(remainder > 0){ \ - ff_yuv2yuvX_mmxext(filter, filterSize - 1, pixelsProcessed, dest - offset, pixelsProcessed + remainder + offset, dither, offset); \ - } \ + if (dstW >= step) \ + ff_yuv2yuvX_ ##opt(filter, filterSize - 1, 0, dest - offset, dstW + offset, dither, offset); \ + else \ + yuv2yuvX_ ##tail(filter, filterSize, src, dest, dstW, dither, offset); \ return; \ } @@ -220,13 +217,13 @@ static void yuv2yuvX_ ##opt(const int16_t *filter, int filterSize, \ YUV2YUVX_FUNC_MMX(mmxext, 16) #endif #if HAVE_SSE3_EXTERNAL -YUV2YUVX_FUNC(sse3, 32) +YUV2YUVX_FUNC(sse3, 32, mmxext) #endif #if HAVE_AVX2_EXTERNAL -YUV2YUVX_FUNC(avx2, 64) +YUV2YUVX_FUNC(avx2, 64, sse3) #endif #if ARCH_X86_64 && HAVE_AVX512_EXTERNAL -YUV2YUVX_FUNC(avx512, 128) +YUV2YUVX_FUNC(avx512, 128, avx2) #endif #define SCALE_FUNC(filter_n, from_bpc, to_bpc, opt) \ diff --git a/libswscale/x86/yuv2yuvX.asm b/libswscale/x86/yuv2yuvX.asm index 57bfa09d66..03bfd6ad1d 100644 --- a/libswscale/x86/yuv2yuvX.asm +++ b/libswscale/x86/yuv2yuvX.asm @@ -54,6 +54,8 @@ cglobal yuv2yuvX, 7, 7, 8, filter, filterSize, src, dest, dstW, dither, offset %else movq xm3, [ditherq] %endif ; avx2 + mov ditherq, dstWq + sub dstWq, mmsize * unroll %if cpuflag(avx512) mova m15, [permutation] @@ -92,13 +94,17 @@ cglobal yuv2yuvX, 7, 7, 8, filter, filterSize, src, dest, dstW, dither, offset %else mova m0, [filterSizeq + 8] %endif - pmulhw m2, m0, [srcq + offsetq * 2] - pmulhw m5, m0, [srcq + offsetq * 2 + mmsize] + movu m2, [srcq + offsetq * 2] + movu m5, [srcq + offsetq * 2 + mmsize] + pmulhw m2, m0, m2 + pmulhw m5, m0, m5 paddw m3, m3, m2 paddw m4, m4, m5 %if cpuflag(sse3) - pmulhw m2, m0, [srcq + offsetq * 2 + 2 * mmsize] - pmulhw m5, m0, [srcq + offsetq * 2 + 3 * mmsize] + movu m2, [srcq + offsetq * 2 + 2 * mmsize] + movu m5, [srcq + offsetq * 2 + 3 * mmsize] + pmulhw m2, m0, m2 + pmulhw m5, m0, m5 paddw m6, m6, m2 paddw m1, m1, m5 %endif @@ -131,8 +137,14 @@ cglobal yuv2yuvX, 7, 7, 8, filter, filterSize, src, dest, dstW, dither, offset add offsetq, mmsize * unroll mov filterSizeq, filterq cmp offsetq, dstWq - jb .outerloop - RET + jb .outerloop + + mov dstWq, offsetq + mov offsetq, ditherq + sub offsetq, mmsize * unroll + cmp dstWq, ditherq + jb .outerloop + REP_RET %endmacro INIT_MMX mmxext