From patchwork Wed Jul 27 17:19:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Swinney, Jonathan" X-Patchwork-Id: 37011 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:1649:b0:8b:613a:194d with SMTP id no9csp490525pzb; Wed, 27 Jul 2022 10:19:44 -0700 (PDT) X-Google-Smtp-Source: AGRyM1sVhPD4XIe3jiikfmuX52ObFG1JGRov4Bv7Et1zazPhoOvunJWtoRwiS0k5f8NIIu0DZSfk X-Received: by 2002:a17:907:97cd:b0:72f:2df:274f with SMTP id js13-20020a17090797cd00b0072f02df274fmr17917423ejc.766.1658942384428; Wed, 27 Jul 2022 10:19:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1658942384; cv=none; d=google.com; s=arc-20160816; b=Ewgx6c5dTvbsPIEHc9dXbng7iqX+whOS9KSeVts2CLh4kFbto1KrLNvNIC1Vfw7Z0Q 3NUb1sdIKxWgGdPe7pLrp4s++0NTSoHpHwFVX0h7laXXV1bsZfhraVIhZyyPDhCT6oWQ yhHQ7pgFYwor4W3kVtYiwYJW6t4dzAjrexM3dAnlgC3DJjClwceGROYwpetvkihISLJe eeg0ObjZehot+GFLBTWFrQsKx1lWahznrVq8/aymGf1R/nbrs28Sv2jnrzawTPi+ndl6 yWAv+fKwkEHaRi+j9YByYPr+CKAD2Fga0XwjDoS2Zxj0kymz3B3itSXoWgC7wkohhvlR +Lnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:content-language :accept-language:message-id:date:thread-index:thread-topic:to:from :dkim-signature:delivered-to; bh=RjdHWmqp4Y7KD2kTW9p+Zs0wmQmB4esYj5RqXKHSlrI=; b=QGPhRoAFSQ9o+HjdOomLsYMEic10JbRDSA4gdefTIeLQgYCXKZo+1+wuVdzbT/BVUG tIohkPZkkn9hETDAHo1EXpopGHmR8ibzwViDxyr7+YCJvW1zme3bJdedWVQpSZqHbh8w aJ1BVcs1EFtBAkqipyNrxA7cEcWAgEdS/jTgR7aQTNtK4FHRnUbL2HzQcm4NSOQw2aBC NVM7SKXG5az6khusO5L8/TIpBgILV1jTHwLrfPKIG8xT7I2LRtCaOS0XO6PZy/PkIXGo Dl+4q0Kt3aBE9iL/4DV+uiYGAIHzoRY62EVtSlxsg+d+PhjsRJL5LZ2V9Yobzl4KTKWI C5iw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@amazon.com header.s=amazon201209 header.b=XArhdPkd; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amazon.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i4-20020aa7c704000000b0043cc90dc170si1049064edq.96.2022.07.27.10.19.42; Wed, 27 Jul 2022 10:19:44 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@amazon.com header.s=amazon201209 header.b=XArhdPkd; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amazon.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id AE0CD68B8F0; Wed, 27 Jul 2022 20:19:39 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from smtp-fw-9103.amazon.com (smtp-fw-9103.amazon.com [207.171.188.200]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B93F768B8E0 for ; Wed, 27 Jul 2022 20:19:32 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1658942378; x=1690478378; h=from:to:cc:subject:date:message-id: content-transfer-encoding:mime-version; bh=X76asSlkPbHi9zt3LbLOFOjBap2XY13FH5enOnIzzcQ=; b=XArhdPkd2hGO6c9AGQBpvgnCnZnVM+DMb5SDfKGzaJMhLBU8hSL8dylt oihXWTP8K09vp80Y7vnliMEbxnj8123IGvqsC4nfViAko0onCAxt1iXnQ j6bBjTxyMLFZ6DYjiUJGrnz/XVfpKt0IlLIG7MX8oBhj+r/gO72p0W21/ I=; X-IronPort-AV: E=Sophos;i="5.93,196,1654560000"; d="scan'208";a="1038424914" Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO email-inbound-relay-iad-1d-9a235a16.us-east-1.amazon.com) ([10.25.36.214]) by smtp-border-fw-9103.sea19.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jul 2022 17:19:27 +0000 Received: from EX13MTAUWB001.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan3.iad.amazon.com [10.40.163.38]) by email-inbound-relay-iad-1d-9a235a16.us-east-1.amazon.com (Postfix) with ESMTPS id AE3898017D; Wed, 27 Jul 2022 17:19:24 +0000 (UTC) Received: from EX19D007UWB003.ant.amazon.com (10.13.138.28) by EX13MTAUWB001.ant.amazon.com (10.43.161.207) with Microsoft SMTP Server (TLS) id 15.0.1497.36; Wed, 27 Jul 2022 17:19:24 +0000 Received: from EX19D007UWB001.ant.amazon.com (10.13.138.75) by EX19D007UWB003.ant.amazon.com (10.13.138.28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.9; Wed, 27 Jul 2022 17:19:23 +0000 Received: from EX19D007UWB001.ant.amazon.com ([fe80::bcaa:e18f:a569:3851]) by EX19D007UWB001.ant.amazon.com ([fe80::bcaa:e18f:a569:3851%6]) with mapi id 15.02.1118.009; Wed, 27 Jul 2022 17:19:23 +0000 From: "Swinney, Jonathan" To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v2 1/2] checkasm: updated tests for sw_scale Thread-Index: Adih3QN2jIwXjY+rSByLKY5wWJV00g== Date: Wed, 27 Jul 2022 17:19:23 +0000 Message-ID: <8ce08abb20944ba4987c8bc1ba41c55f@amazon.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.43.161.113] MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 1/2] checkasm: updated tests for sw_scale X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: =?utf-8?q?Martin_Storsj=C3=B6?= , "J. Dekker" , "Pop, Sebastian" , Hubert Mazur Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: hyUZ4frKMi8t - added a test for yuv2plane1 - fixed test for yuv2planeX for aarch64 which was previously not working at all - updated the test for yuv2planeX to check exact results or approximated results Signed-off-by: Jonathan Swinney --- libswscale/x86/swscale.c | 8 +- tests/checkasm/sw_scale.c | 189 ++++++++++++++++++++++++++++++-------- 2 files changed, 155 insertions(+), 42 deletions(-) diff --git a/libswscale/x86/swscale.c b/libswscale/x86/swscale.c index 628f12137c..32d441245d 100644 --- a/libswscale/x86/swscale.c +++ b/libswscale/x86/swscale.c @@ -534,7 +534,8 @@ switch(c->dstBpc){ \ ASSIGN_SSE_SCALE_FUNC(c->hcScale, c->hChrFilterSize, sse2, sse2); ASSIGN_VSCALEX_FUNC(c->yuv2planeX, sse2, , HAVE_ALIGNED_STACK || ARCH_X86_64); - ASSIGN_VSCALE_FUNC(c->yuv2plane1, sse2); + if (!(c->flags & SWS_ACCURATE_RND)) + ASSIGN_VSCALE_FUNC(c->yuv2plane1, sse2); switch (c->srcFormat) { case AV_PIX_FMT_YA8: @@ -583,14 +584,15 @@ switch(c->dstBpc){ \ ASSIGN_VSCALEX_FUNC(c->yuv2planeX, sse4, if (!isBE(c->dstFormat)) c->yuv2planeX = ff_yuv2planeX_16_sse4, HAVE_ALIGNED_STACK || ARCH_X86_64); - if (c->dstBpc == 16 && !isBE(c->dstFormat)) + if (c->dstBpc == 16 && !isBE(c->dstFormat) && !(c->flags & SWS_ACCURATE_RND)) c->yuv2plane1 = ff_yuv2plane1_16_sse4; } if (EXTERNAL_AVX(cpu_flags)) { ASSIGN_VSCALEX_FUNC(c->yuv2planeX, avx, , HAVE_ALIGNED_STACK || ARCH_X86_64); - ASSIGN_VSCALE_FUNC(c->yuv2plane1, avx); + if (!(c->flags & SWS_ACCURATE_RND)) + ASSIGN_VSCALE_FUNC(c->yuv2plane1, avx); switch (c->srcFormat) { case AV_PIX_FMT_YUYV422: diff --git a/tests/checkasm/sw_scale.c b/tests/checkasm/sw_scale.c index b643a47c30..b01fc31fd9 100644 --- a/tests/checkasm/sw_scale.c +++ b/tests/checkasm/sw_scale.c @@ -18,6 +18,7 @@ */ #include +#include #include "libavutil/common.h" #include "libavutil/intreadwrite.h" @@ -35,40 +36,140 @@ AV_WN32(buf + j, rnd()); \ } while (0) -// This reference function is the same approximate algorithm employed by the -// SIMD functions -static void ref_function(const int16_t *filter, int filterSize, - const int16_t **src, uint8_t *dest, int dstW, - const uint8_t *dither, int offset) +static void yuv2planeX_8_ref(const int16_t *filter, int filterSize, + const int16_t **src, uint8_t *dest, int dstW, + const uint8_t *dither, int offset) { - int i, d; - d = ((filterSize - 1) * 8 + dither[0]) >> 4; - for ( i = 0; i < dstW; i++) { - int16_t val = d; + // This corresponds to the yuv2planeX_8_c function + int i; + for (i = 0; i < dstW; i++) { + int val = dither[(i + offset) & 7] << 12; int j; - union { - int val; - int16_t v[2]; - } t; - for (j = 0; j < filterSize; j++){ - t.val = (int)src[j][i + offset] * (int)filter[j]; - val += t.v[1]; + for (j = 0; j < filterSize; j++) + val += src[j][i] * filter[j]; + + dest[i]= av_clip_uint8(val >> 19); + } +} + +static int cmp_off_by_n(const uint8_t *ref, const uint8_t *test, size_t n, int accuracy) +{ + for (size_t i = 0; i < n; i++) { + if (abs(ref[i] - test[i]) > accuracy) + return 1; + } + return 0; +} + +static void print_data(uint8_t *p, size_t len, size_t offset) +{ + size_t i = 0; + for (; i < len; i++) { + if (i % 8 == 0) { + printf("0x%04zx: ", i+offset); + } + printf("0x%02x ", (uint32_t) p[i]); + if (i % 8 == 7) { + printf("\n"); } - dest[i]= av_clip_uint8(val>>3); } + if (i % 8 != 0) { + printf("\n"); + } +} + +static size_t show_differences(uint8_t *a, uint8_t *b, size_t len) +{ + for (size_t i = 0; i < len; i++) { + if (a[i] != b[i]) { + size_t offset_of_mismatch = i; + size_t offset; + if (i >= 8) i-=8; + offset = i & (~7); + printf("test a:\n"); + print_data(&a[offset], 32, offset); + printf("\ntest b:\n"); + print_data(&b[offset], 32, offset); + printf("\n"); + return offset_of_mismatch; + } + } + return len; } -static void check_yuv2yuvX(void) +static void check_yuv2yuv1(bool accurate) +{ + struct SwsContext *ctx; + int osi, isi; + int dstW, offset; + size_t fail_offset; + const int input_sizes[] = {8, 24, 128, 144, 256, 512}; + const int INPUT_SIZES = sizeof(input_sizes)/sizeof(input_sizes[0]); + #define LARGEST_INPUT_SIZE 512 + + const int offsets[] = {0, 3, 8, 11, 16, 19}; + const int OFFSET_SIZES = sizeof(offsets)/sizeof(offsets[0]); + const char *accurate_str = (accurate) ? "accurate" : "approximate"; + + declare_func_emms(AV_CPU_FLAG_MMX, void, + const int16_t *src, uint8_t *dest, + int dstW, const uint8_t *dither, int offset); + + LOCAL_ALIGNED_8(int16_t, src_pixels, [LARGEST_INPUT_SIZE]); + LOCAL_ALIGNED_8(uint8_t, dst0, [LARGEST_INPUT_SIZE]); + LOCAL_ALIGNED_8(uint8_t, dst1, [LARGEST_INPUT_SIZE]); + LOCAL_ALIGNED_8(uint8_t, dither, [8]); + + randomize_buffers((uint8_t*)dither, 8); + randomize_buffers((uint8_t*)src_pixels, LARGEST_INPUT_SIZE * sizeof(int16_t)); + ctx = sws_alloc_context(); + if (accurate) + ctx->flags |= SWS_ACCURATE_RND; + if (sws_init_context(ctx, NULL, NULL) < 0) + fail(); + + ff_sws_init_scale(ctx); + for (isi = 0; isi < INPUT_SIZES; ++isi) { + dstW = input_sizes[isi]; + for (osi = 0; osi < OFFSET_SIZES; osi++) { + offset = offsets[osi]; + if (check_func(ctx->yuv2plane1, "yuv2yuv1_%d_%d_%s", offset, dstW, accurate_str)){ + memset(dst0, 0, LARGEST_INPUT_SIZE * sizeof(dst0[0])); + memset(dst1, 0, LARGEST_INPUT_SIZE * sizeof(dst1[0])); + + call_ref(src_pixels, dst0, dstW, dither, offset); + call_new(src_pixels, dst1, dstW, dither, offset); + if (cmp_off_by_n(dst0, dst1, dstW * sizeof(dst0[0]), accurate ? 0 : 2)) { + fail(); + printf("failed: yuv2yuv1_%d_%di_%s\n", offset, dstW, accurate_str); + fail_offset = show_differences(dst0, dst1, LARGEST_INPUT_SIZE * sizeof(dst0[0])); + printf("failing values: src: 0x%04x dither: 0x%02x dst-c: %02x dst-asm: %02x\n", + (int) src_pixels[fail_offset], + (int) dither[(fail_offset + fail_offset) & 7], + (int) dst0[fail_offset], + (int) dst1[fail_offset]); + } + if(dstW == LARGEST_INPUT_SIZE) + bench_new(src_pixels, dst1, dstW, dither, offset); + } + } + } + sws_freeContext(ctx); +} + +static void check_yuv2yuvX(bool accurate) { struct SwsContext *ctx; int fsi, osi, isi, i, j; int dstW; #define LARGEST_FILTER 16 -#define FILTER_SIZES 4 - static const int filter_sizes[FILTER_SIZES] = {1, 4, 8, 16}; + // ff_yuv2planeX_8_sse2 can't handle odd filter sizes + const int filter_sizes[] = {2, 4, 8, 16}; + const int FILTER_SIZES = sizeof(filter_sizes)/sizeof(filter_sizes[0]); #define LARGEST_INPUT_SIZE 512 -#define INPUT_SIZES 6 - static const int input_sizes[INPUT_SIZES] = {8, 24, 128, 144, 256, 512}; + static const int input_sizes[] = {8, 24, 128, 144, 256, 512}; + const int INPUT_SIZES = sizeof(input_sizes)/sizeof(input_sizes[0]); + const char *accurate_str = (accurate) ? "accurate" : "approximate"; declare_func_emms(AV_CPU_FLAG_MMX, void, const int16_t *filter, int filterSize, const int16_t **src, uint8_t *dest, @@ -89,6 +190,8 @@ static void check_yuv2yuvX(void) randomize_buffers((uint8_t*)src_pixels, LARGEST_FILTER * LARGEST_INPUT_SIZE * sizeof(int16_t)); randomize_buffers((uint8_t*)filter_coeff, LARGEST_FILTER * sizeof(int16_t)); ctx = sws_alloc_context(); + if (accurate) + ctx->flags |= SWS_ACCURATE_RND; if (sws_init_context(ctx, NULL, NULL) < 0) fail(); @@ -96,33 +199,37 @@ static void check_yuv2yuvX(void) for(isi = 0; isi < INPUT_SIZES; ++isi){ dstW = input_sizes[isi]; for(osi = 0; osi < 64; osi += 16){ - for(fsi = 0; fsi < FILTER_SIZES; ++fsi){ + if (dstW <= osi) + continue; + for (fsi = 0; fsi < FILTER_SIZES; ++fsi) { src = av_malloc(sizeof(int16_t*) * filter_sizes[fsi]); vFilterData = av_malloc((filter_sizes[fsi] + 2) * sizeof(union VFilterData)); memset(vFilterData, 0, (filter_sizes[fsi] + 2) * sizeof(union VFilterData)); - for(i = 0; i < filter_sizes[fsi]; ++i){ + for (i = 0; i < filter_sizes[fsi]; ++i) { src[i] = &src_pixels[i * LARGEST_INPUT_SIZE]; - vFilterData[i].src = src[i]; + vFilterData[i].src = src[i] - osi; for(j = 0; j < 4; ++j) vFilterData[i].coeff[j + 4] = filter_coeff[i]; } - if (check_func(ctx->yuv2planeX, "yuv2yuvX_%d_%d_%d", filter_sizes[fsi], osi, dstW)){ + if (check_func(ctx->yuv2planeX, "yuv2yuvX_%d_%d_%d_%s", filter_sizes[fsi], osi, dstW, accurate_str)){ + // use vFilterData for the mmx function + const int16_t *filter = ctx->use_mmx_vfilter ? (const int16_t*)vFilterData : &filter_coeff[0]; memset(dst0, 0, LARGEST_INPUT_SIZE * sizeof(dst0[0])); memset(dst1, 0, LARGEST_INPUT_SIZE * sizeof(dst1[0])); - // The reference function is not the scalar function selected when mmx - // is deactivated as the SIMD functions do not give the same result as - // the scalar ones due to rounding. The SIMD functions are activated by - // the flag SWS_ACCURATE_RND - ref_function(&filter_coeff[0], filter_sizes[fsi], src, dst0, dstW - osi, dither, osi); - // There's no point in calling new for the reference function - if(ctx->use_mmx_vfilter){ - call_new((const int16_t*)vFilterData, filter_sizes[fsi], src, dst1, dstW - osi, dither, osi); - if (memcmp(dst0, dst1, LARGEST_INPUT_SIZE * sizeof(dst0[0]))) - fail(); - if(dstW == LARGEST_INPUT_SIZE) - bench_new((const int16_t*)vFilterData, filter_sizes[fsi], src, dst1, dstW - osi, dither, osi); + // We can't use call_ref here, because we don't know if use_mmx_vfilter was set for that + // function or not, so we can't pass it the parameters correctly. + yuv2planeX_8_ref(&filter_coeff[0], filter_sizes[fsi], src, dst0, dstW - osi, dither, osi); + + call_new(filter, filter_sizes[fsi], src, dst1, dstW - osi, dither, osi); + if (cmp_off_by_n(dst0, dst1, LARGEST_INPUT_SIZE * sizeof(dst0[0]), accurate ? 0 : 2)) { + fail(); + printf("failed: yuv2yuvX_%d_%d_%d_%s\n", filter_sizes[fsi], osi, dstW, accurate_str); + show_differences(dst0, dst1, LARGEST_INPUT_SIZE * sizeof(dst0[0])); } + if(dstW == LARGEST_INPUT_SIZE) + bench_new((const int16_t*)vFilterData, filter_sizes[fsi], src, dst1, dstW - osi, dither, osi); + } av_freep(&src); av_freep(&vFilterData); @@ -245,6 +352,10 @@ void checkasm_check_sw_scale(void) { check_hscale(); report("hscale"); - check_yuv2yuvX(); + check_yuv2yuv1(false); + check_yuv2yuv1(true); + report("yuv2yuv1"); + check_yuv2yuvX(false); + check_yuv2yuvX(true); report("yuv2yuvX"); } From patchwork Wed Jul 27 17:19:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Swinney, Jonathan" X-Patchwork-Id: 37012 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:1649:b0:8b:613a:194d with SMTP id no9csp490601pzb; Wed, 27 Jul 2022 10:19:52 -0700 (PDT) X-Google-Smtp-Source: AGRyM1teHfTeqwSTvrG+Xl/ZZnAXtDKxvspJkqzLXT9hm6GbmkkRlSv0ACS1KlHDCS/ZOfdXupUd X-Received: by 2002:a50:fe89:0:b0:43b:22af:4c8b with SMTP id d9-20020a50fe89000000b0043b22af4c8bmr24614415edt.296.1658942391925; Wed, 27 Jul 2022 10:19:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1658942391; cv=none; d=google.com; s=arc-20160816; b=paWVy+ToS47Symks3kF9UNd77pkfjdi30yrlEdTSuKsCFKNb7xj/660wi69pIwZFbg m0/lPK2gMRCd7G5yRpdN/ncT5le8STWk8/7ztPOzuh6U5M3pVgN7eXFLW1wDV+4QKs72 M15AxdA9t02I/pH/0CoPSuT2D3Zm5Lr+JURYFJlpto3TsuFIFFUTRi0BJByx038rxyfc e+FXam405Y9PFS9Iaa7qsvMdleoEzYFDVH4vkT6A+nSkG0n1AwvzCPCoaoVMJmkB6KhN o702Ern+hTDfY8ilY5CNWNJxlMXQMNNewClTG8TX8H4oq6dNZk9Y7OH1ZVpdiMCuIBQJ WqXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:content-language :accept-language:message-id:date:thread-index:thread-topic:to:from :dkim-signature:delivered-to; bh=knUNw6CYQU8LowG4Bo/rIQfBfMBSYfyFqmrWup9UrCY=; b=rdRIiLeD0dWte2Q1Qq9TJaJHh2B2QNPh1fCsg0kIDboihCWX0mz0REN/oPQFgHtJoC 6CnNpc91R3zs+GHCL1LzhpTva1KM6X6uTbRZznIbohSoqgzyeq8NPjOuxwC3My3meBtt X7QJ4RJStX9KMCJHNQVMHQ0+L/vDQRicHeZtGRkZzfQLLn3mq5/F7gw9Fc7sP+eKhQtB M5rFSf889Pf6zccjTWW3ULdHldgV6gMxOw0UV3cYnpIad7gElN6an2GEuyk99KouOAWg CkZFZ7vKU8PuGWZWuy1Bf3NoE0gHQ7ERZWEyJ95/TYxEjtXH+9dArwFVCeHhvzumZdsp tQCw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@amazon.com header.s=amazon201209 header.b=qsHa2q4N; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amazon.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v5-20020a1709061dc500b0072ab62adfe9si15590248ejh.237.2022.07.27.10.19.51; Wed, 27 Jul 2022 10:19:51 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@amazon.com header.s=amazon201209 header.b=qsHa2q4N; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amazon.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B1B2568B913; Wed, 27 Jul 2022 20:19:45 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from smtp-fw-6002.amazon.com (smtp-fw-6002.amazon.com [52.95.49.90]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A2B5968B8DA for ; Wed, 27 Jul 2022 20:19:38 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1658942384; x=1690478384; h=from:to:cc:subject:date:message-id: content-transfer-encoding:mime-version; bh=Bq6KYni1Q5QC4vpPbAPMa1B7rG92HBM7sHMwc4Gpk9Y=; b=qsHa2q4NQRO4Ukkam1Y9KGQnsbeZMwbMmCtxaktfu+xJXW3JkongBxHd K7CYJBsYB20BHotoCNeB4ZHzFoRuGKkK73WwPN0I07BzILDUIMpGr9pvp 0IFiQYAJWqb0ss3QWEpVwnkPlmCeNN+2ewVLBE7oDtSLO+e9gluWG8Asn I=; X-IronPort-AV: E=Sophos;i="5.93,196,1654560000"; d="scan'208";a="226404735" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-iad-1e-f771ae83.us-east-1.amazon.com) ([10.43.8.6]) by smtp-border-fw-6002.iad6.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jul 2022 17:19:38 +0000 Received: from EX13MTAUWB001.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan2.iad.amazon.com [10.40.163.34]) by email-inbound-relay-iad-1e-f771ae83.us-east-1.amazon.com (Postfix) with ESMTPS id 77169123AC6; Wed, 27 Jul 2022 17:19:36 +0000 (UTC) Received: from EX19D007UWB004.ant.amazon.com (10.13.138.60) by EX13MTAUWB001.ant.amazon.com (10.43.161.207) with Microsoft SMTP Server (TLS) id 15.0.1497.36; Wed, 27 Jul 2022 17:19:35 +0000 Received: from EX19D007UWB001.ant.amazon.com (10.13.138.75) by EX19D007UWB004.ant.amazon.com (10.13.138.60) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.9; Wed, 27 Jul 2022 17:19:35 +0000 Received: from EX19D007UWB001.ant.amazon.com ([fe80::bcaa:e18f:a569:3851]) by EX19D007UWB001.ant.amazon.com ([fe80::bcaa:e18f:a569:3851%6]) with mapi id 15.02.1118.009; Wed, 27 Jul 2022 17:19:35 +0000 From: "Swinney, Jonathan" To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH v2 2/2] swscale/aarch64: add vscale specializations Thread-Index: Adih3QqeJa73cTxzTb6Qq4QNGJON4A== Date: Wed, 27 Jul 2022 17:19:35 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.43.161.113] MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 2/2] swscale/aarch64: add vscale specializations X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: =?utf-8?q?Martin_Storsj=C3=B6?= , "J. Dekker" , "Pop, Sebastian" , Hubert Mazur Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 9D7Mjwe3AVls This commit adds new code paths for vscale when filterSize is 2, 4, or 8. By using specialized code with unrolling to match the filterSize we can improve performance. This patch also corrects the behavor for filterSize 1 which was previously failing the checkasm test. On AWS c7g (Graviton 3, Neoverse V1) instances: before after yuv2yuvX_2_0_512_neon: 355.1 270.4 yuv2yuvX_4_0_512_neon: 621.4 436.6 yuv2yuvX_8_0_512_neon: 1134.4 817.6 yuv2yuvX_16_0_512_neon: 2071.9 1845.1 Signed-off-by: Jonathan Swinney --- libswscale/aarch64/output.S | 188 +++++++++++++++++++++++++++++++++-- libswscale/aarch64/swscale.c | 12 +++ 2 files changed, 194 insertions(+), 6 deletions(-) diff --git a/libswscale/aarch64/output.S b/libswscale/aarch64/output.S index af71de6050..a18e2c7541 100644 --- a/libswscale/aarch64/output.S +++ b/libswscale/aarch64/output.S @@ -21,29 +21,48 @@ #include "libavutil/aarch64/asm.S" function ff_yuv2planeX_8_neon, export=1 +// x0 - const int16_t *filter, +// x1 - int filterSize, +// x2 - const int16_t **src, +// x3 - uint8_t *dest, +// x4 - int dstW, +// x5 - const uint8_t *dither, +// x6 - int offset + ld1 {v0.8B}, [x5] // load 8x8-bit dither + and w6, w6, #7 cbz w6, 1f // check if offsetting present ext v0.8B, v0.8B, v0.8B, #3 // honor offsetting which can be 0 or 3 only 1: uxtl v0.8H, v0.8B // extend dither to 16-bit ushll v1.4S, v0.4H, #12 // extend dither to 32-bit with left shift by 12 (part 1) ushll2 v2.4S, v0.8H, #12 // extend dither to 32-bit with left shift by 12 (part 2) + cmp w1, #8 // if filterSize == 8, branch to specialized version + b.eq 6f + cmp w1, #4 // if filterSize == 4, branch to specialized version + b.eq 8f + cmp w1, #2 // if filterSize == 2, branch to specialized version + b.eq 10f + +// The filter size does not match of the of specialized implementations. It is either even or odd. If it is even +// then use the first section below. mov x7, #0 // i = 0 + tbnz w1, #0, 4f // if filterSize % 2 != 0 branch to specialized version +// fs % 2 == 0 2: mov v3.16B, v1.16B // initialize accumulator part 1 with dithering value mov v4.16B, v2.16B // initialize accumulator part 2 with dithering value mov w8, w1 // tmpfilterSize = filterSize mov x9, x2 // srcp = src mov x10, x0 // filterp = filter 3: ldp x11, x12, [x9], #16 // get 2 pointers: src[j] and src[j+1] + ldr s7, [x10], #4 // read 2x16-bit coeff X and Y at filter[j] and filter[j+1] add x11, x11, x7, lsl #1 // &src[j ][i] add x12, x12, x7, lsl #1 // &src[j+1][i] ld1 {v5.8H}, [x11] // read 8x16-bit @ src[j ][i + {0..7}]: A,B,C,D,E,F,G,H ld1 {v6.8H}, [x12] // read 8x16-bit @ src[j+1][i + {0..7}]: I,J,K,L,M,N,O,P - ld1r {v7.8H}, [x10], #2 // read 1x16-bit coeff X at filter[j ] and duplicate across lanes - ld1r {v16.8H}, [x10], #2 // read 1x16-bit coeff Y at filter[j+1] and duplicate across lanes - smlal v3.4S, v5.4H, v7.4H // val0 += {A,B,C,D} * X - smlal2 v4.4S, v5.8H, v7.8H // val1 += {E,F,G,H} * X - smlal v3.4S, v6.4H, v16.4H // val0 += {I,J,K,L} * Y - smlal2 v4.4S, v6.8H, v16.8H // val1 += {M,N,O,P} * Y + smlal v3.4S, v5.4H, v7.H[0] // val0 += {A,B,C,D} * X + smlal2 v4.4S, v5.8H, v7.H[0] // val1 += {E,F,G,H} * X + smlal v3.4S, v6.4H, v7.H[1] // val0 += {I,J,K,L} * Y + smlal2 v4.4S, v6.8H, v7.H[1] // val1 += {M,N,O,P} * Y subs w8, w8, #2 // tmpfilterSize -= 2 b.gt 3b // loop until filterSize consumed @@ -55,4 +74,161 @@ function ff_yuv2planeX_8_neon, export=1 add x7, x7, #8 // i += 8 b.gt 2b // loop until width consumed ret + +// If filter size is odd (most likely == 1), then use this section. +// fs % 2 != 0 +4: mov v3.16B, v1.16B // initialize accumulator part 1 with dithering value + mov v4.16B, v2.16B // initialize accumulator part 2 with dithering value + mov w8, w1 // tmpfilterSize = filterSize + mov x9, x2 // srcp = src + mov x10, x0 // filterp = filter +5: ldr x11, [x9], #8 // get 1 pointer: src[j] + ldr h6, [x10], #2 // read 1 16 bit coeff X at filter[j] + add x11, x11, x7, lsl #1 // &src[j ][i] + ld1 {v5.8H}, [x11] // read 8x16-bit @ src[j ][i + {0..7}]: A,B,C,D,E,F,G,H + smlal v3.4S, v5.4H, v6.H[0] // val0 += {A,B,C,D} * X + smlal2 v4.4S, v5.8H, v6.H[0] // val1 += {E,F,G,H} * X + subs w8, w8, #1 // tmpfilterSize -= 2 + b.gt 5b // loop until filterSize consumed + + sqshrun v3.4h, v3.4s, #16 // clip16(val0>>16) + sqshrun2 v3.8h, v4.4s, #16 // clip16(val1>>16) + uqshrn v3.8b, v3.8h, #3 // clip8(val>>19) + st1 {v3.8b}, [x3], #8 // write to destination + subs w4, w4, #8 // dstW -= 8 + add x7, x7, #8 // i += 8 + b.gt 4b // loop until width consumed + ret + +6: // fs=8 + ldp x5, x6, [x2] // load 2 pointers: src[j ] and src[j+1] + ldp x7, x9, [x2, #16] // load 2 pointers: src[j+2] and src[j+3] + ldp x10, x11, [x2, #32] // load 2 pointers: src[j+4] and src[j+5] + ldp x12, x13, [x2, #48] // load 2 pointers: src[j+6] and src[j+7] + + // load 8x16-bit values for filter[j], where j=0..7 + ld1 {v6.8H}, [x0] +7: + mov v3.16B, v1.16B // initialize accumulator part 1 with dithering value + mov v4.16B, v2.16B // initialize accumulator part 2 with dithering value + + ld1 {v24.8H}, [x5], #16 // load 8x16-bit values for src[j + 0][i + {0..7}] + ld1 {v25.8H}, [x6], #16 // load 8x16-bit values for src[j + 1][i + {0..7}] + ld1 {v26.8H}, [x7], #16 // load 8x16-bit values for src[j + 2][i + {0..7}] + ld1 {v27.8H}, [x9], #16 // load 8x16-bit values for src[j + 3][i + {0..7}] + ld1 {v28.8H}, [x10], #16 // load 8x16-bit values for src[j + 4][i + {0..7}] + ld1 {v29.8H}, [x11], #16 // load 8x16-bit values for src[j + 5][i + {0..7}] + ld1 {v30.8H}, [x12], #16 // load 8x16-bit values for src[j + 6][i + {0..7}] + ld1 {v31.8H}, [x13], #16 // load 8x16-bit values for src[j + 7][i + {0..7}] + + smlal v3.4S, v24.4H, v6.H[0] // val0 += src[0][i + {0..3}] * filter[0] + smlal2 v4.4S, v24.8H, v6.H[0] // val1 += src[0][i + {4..7}] * filter[0] + smlal v3.4S, v25.4H, v6.H[1] // val0 += src[1][i + {0..3}] * filter[1] + smlal2 v4.4S, v25.8H, v6.H[1] // val1 += src[1][i + {4..7}] * filter[1] + smlal v3.4S, v26.4H, v6.H[2] // val0 += src[2][i + {0..3}] * filter[2] + smlal2 v4.4S, v26.8H, v6.H[2] // val1 += src[2][i + {4..7}] * filter[2] + smlal v3.4S, v27.4H, v6.H[3] // val0 += src[3][i + {0..3}] * filter[3] + smlal2 v4.4S, v27.8H, v6.H[3] // val1 += src[3][i + {4..7}] * filter[3] + smlal v3.4S, v28.4H, v6.H[4] // val0 += src[4][i + {0..3}] * filter[4] + smlal2 v4.4S, v28.8H, v6.H[4] // val1 += src[4][i + {4..7}] * filter[4] + smlal v3.4S, v29.4H, v6.H[5] // val0 += src[5][i + {0..3}] * filter[5] + smlal2 v4.4S, v29.8H, v6.H[5] // val1 += src[5][i + {4..7}] * filter[5] + smlal v3.4S, v30.4H, v6.H[6] // val0 += src[6][i + {0..3}] * filter[6] + smlal2 v4.4S, v30.8H, v6.H[6] // val1 += src[6][i + {4..7}] * filter[6] + smlal v3.4S, v31.4H, v6.H[7] // val0 += src[7][i + {0..3}] * filter[7] + smlal2 v4.4S, v31.8H, v6.H[7] // val1 += src[7][i + {4..7}] * filter[7] + + sqshrun v3.4h, v3.4s, #16 // clip16(val0>>16) + sqshrun2 v3.8h, v4.4s, #16 // clip16(val1>>16) + uqshrn v3.8b, v3.8h, #3 // clip8(val>>19) + subs w4, w4, #8 // dstW -= 8 + st1 {v3.8b}, [x3], #8 // write to destination + b.gt 7b // loop until width consumed + ret + +8: // fs=4 + ldp x5, x6, [x2] // load 2 pointers: src[j ] and src[j+1] + ldp x7, x9, [x2, #16] // load 2 pointers: src[j+2] and src[j+3] + + // load 4x16-bit values for filter[j], where j=0..3 and replicated across lanes + ld1 {v6.4H}, [x0] +9: + mov v3.16B, v1.16B // initialize accumulator part 1 with dithering value + mov v4.16B, v2.16B // initialize accumulator part 2 with dithering value + + ld1 {v24.8H}, [x5], #16 // load 8x16-bit values for src[j + 0][i + {0..7}] + ld1 {v25.8H}, [x6], #16 // load 8x16-bit values for src[j + 1][i + {0..7}] + ld1 {v26.8H}, [x7], #16 // load 8x16-bit values for src[j + 2][i + {0..7}] + ld1 {v27.8H}, [x9], #16 // load 8x16-bit values for src[j + 3][i + {0..7}] + + smlal v3.4S, v24.4H, v6.H[0] // val0 += src[0][i + {0..3}] * filter[0] + smlal2 v4.4S, v24.8H, v6.H[0] // val1 += src[0][i + {4..7}] * filter[0] + smlal v3.4S, v25.4H, v6.H[1] // val0 += src[1][i + {0..3}] * filter[1] + smlal2 v4.4S, v25.8H, v6.H[1] // val1 += src[1][i + {4..7}] * filter[1] + smlal v3.4S, v26.4H, v6.H[2] // val0 += src[2][i + {0..3}] * filter[2] + smlal2 v4.4S, v26.8H, v6.H[2] // val1 += src[2][i + {4..7}] * filter[2] + smlal v3.4S, v27.4H, v6.H[3] // val0 += src[3][i + {0..3}] * filter[3] + smlal2 v4.4S, v27.8H, v6.H[3] // val1 += src[3][i + {4..7}] * filter[3] + + sqshrun v3.4h, v3.4s, #16 // clip16(val0>>16) + sqshrun2 v3.8h, v4.4s, #16 // clip16(val1>>16) + uqshrn v3.8b, v3.8h, #3 // clip8(val>>19) + st1 {v3.8b}, [x3], #8 // write to destination + subs w4, w4, #8 // dstW -= 8 + b.gt 9b // loop until width consumed + ret + +10: // fs=2 + ldp x5, x6, [x2] // load 2 pointers: src[j ] and src[j+1] + + // load 2x16-bit values for filter[j], where j=0..1 and replicated across lanes + ldr s6, [x0] +11: + mov v3.16B, v1.16B // initialize accumulator part 1 with dithering value + mov v4.16B, v2.16B // initialize accumulator part 2 with dithering value + + ld1 {v24.8H}, [x5], #16 // load 8x16-bit values for src[j + 0][i + {0..7}] + ld1 {v25.8H}, [x6], #16 // load 8x16-bit values for src[j + 1][i + {0..7}] + + smlal v3.4S, v24.4H, v6.H[0] // val0 += src[0][i + {0..3}] * filter[0] + smlal2 v4.4S, v24.8H, v6.H[0] // val1 += src[0][i + {4..7}] * filter[0] + smlal v3.4S, v25.4H, v6.H[1] // val0 += src[1][i + {0..3}] * filter[1] + smlal2 v4.4S, v25.8H, v6.H[1] // val1 += src[1][i + {4..7}] * filter[1] + + sqshrun v3.4h, v3.4s, #16 // clip16(val0>>16) + sqshrun2 v3.8h, v4.4s, #16 // clip16(val1>>16) + uqshrn v3.8b, v3.8h, #3 // clip8(val>>19) + st1 {v3.8b}, [x3], #8 // write to destination + subs w4, w4, #8 // dstW -= 8 + b.gt 11b // loop until width consumed + ret +endfunc + +function ff_yuv2plane1_8_neon, export=1 +// x0 - const int16_t *src, +// x1 - uint8_t *dest, +// x2 - int dstW, +// x3 - const uint8_t *dither, +// x4 - int offset + ld1 {v0.8B}, [x3] // load 8x8-bit dither + and w4, w4, #7 + cbz w4, 1f // check if offsetting present + ext v0.8B, v0.8B, v0.8B, #3 // honor offsetting which can be 0 or 3 only +1: uxtl v0.8H, v0.8B // extend dither to 32-bit + uxtl v1.4s, v0.4h + uxtl2 v2.4s, v0.8h +2: + ld1 {v3.8h}, [x0], #16 // read 8x16-bit @ src[j ][i + {0..7}]: A,B,C,D,E,F,G,H + sxtl v4.4s, v3.4h + sxtl2 v5.4s, v3.8h + add v4.4s, v4.4s, v1.4s + add v5.4s, v5.4s, v2.4s + sqshrun v4.4h, v4.4s, #6 + sqshrun2 v4.8h, v5.4s, #6 + + uqshrn v3.8b, v4.8h, #1 // clip8(val>>7) + subs w2, w2, #8 // dstW -= 8 + st1 {v3.8b}, [x1], #8 // write to destination + b.gt 2b // loop until width consumed + ret endfunc diff --git a/libswscale/aarch64/swscale.c b/libswscale/aarch64/swscale.c index ab28be4da6..321d1f844e 100644 --- a/libswscale/aarch64/swscale.c +++ b/libswscale/aarch64/swscale.c @@ -39,6 +39,12 @@ ALL_SCALE_FUNCS(neon); void ff_yuv2planeX_8_neon(const int16_t *filter, int filterSize, const int16_t **src, uint8_t *dest, int dstW, const uint8_t *dither, int offset); +void ff_yuv2plane1_8_neon( + const int16_t *src, + uint8_t *dest, + int dstW, + const uint8_t *dither, + int offset); #define ASSIGN_SCALE_FUNC2(hscalefn, filtersize, opt) do { \ if (c->srcBpc == 8 && c->dstBpc <= 14) { \ @@ -54,6 +60,11 @@ void ff_yuv2planeX_8_neon(const int16_t *filter, int filterSize, ASSIGN_SCALE_FUNC2(hscalefn, X8, opt); \ break; \ } +#define ASSIGN_VSCALE_FUNC(vscalefn, opt) \ + switch (c->dstBpc) { \ + case 8: vscalefn = ff_yuv2plane1_8_ ## opt; break; \ + default: break; \ + } av_cold void ff_sws_init_swscale_aarch64(SwsContext *c) { @@ -62,6 +73,7 @@ av_cold void ff_sws_init_swscale_aarch64(SwsContext *c) if (have_neon(cpu_flags)) { ASSIGN_SCALE_FUNC(c->hyScale, c->hLumFilterSize, neon); ASSIGN_SCALE_FUNC(c->hcScale, c->hChrFilterSize, neon); + ASSIGN_VSCALE_FUNC(c->yuv2plane1, neon); if (c->dstBpc == 8) { c->yuv2planeX = ff_yuv2planeX_8_neon; }