From patchwork Mon Jul 17 09:23:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Kelly X-Patchwork-Id: 42782 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:6da1:b0:131:a7d0:bc6d with SMTP id gl33csp5254328pzb; Mon, 17 Jul 2023 02:24:01 -0700 (PDT) X-Google-Smtp-Source: APBJJlFIn5RwsBZb+0/0s0/BwiJTNyOBfVeEQKlz66O7XzkXhqXD1chxNAW69OgrTLurlmM/fIJ1 X-Received: by 2002:aa7:cb52:0:b0:516:af22:bcc6 with SMTP id w18-20020aa7cb52000000b00516af22bcc6mr9934102edt.21.1689585840945; Mon, 17 Jul 2023 02:24:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689585840; cv=none; d=google.com; s=arc-20160816; b=QlhJm9T4/RRGBC1WipmCI9ecPoMb/x6CibgZrrWI9SBskIPlh5nlgwOlOPPwXzvTpG UrkPcUo58Qp4MPsSqiXzk8ILtKHFGZdYPEHzm7EXFQJvJK5AnATC3iBp3Bq5jMHMML4j VxMx8uqqjpZzF1wwXTjDB9X9i7RYdgxtzMtoJekU4skiJwFFBQk9DhWkTHTGg4Cd36HF u0EeWLlIqRcAYKz/zIJuPMx/05MSXIl0Ps9wlK/a0HiroSgjEQM/0FNiWUUu6Y6Zoe8m gwGizOMhbtkIi1K7yeHrxykxwsf/f/WTeBjM+sZiRGtc/Stb8eB++baL0AhKf/b1TxFZ xygA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:from:message-id:references :mime-version:in-reply-to:date:dkim-signature:delivered-to; bh=BWLtyvpyQnX3yhJEpZdml862th2OLVz+zuSPF75NS3E=; fh=/dichZGS11g2zj1pQZN5I1lofI3DSP0AKY/Bo6XKkZk=; b=PrBGCLZqJIeCclfpunZJI7rw2RJzVTefiLGcauBeD4LCZiOsQab257de40o1KDXY2D BPOllA9aJXjycin18czbqJf6ZbUksHJa9hSitR5qUH7XyiCbB2RUTj52Yl46czvxP1ez U06KL69wYuluzhE5PCQP8nWtqpj/MIw+AOywmLb82FRrU8LpiRK4Q6igzLlkjNJE2S/n B32w8JhCZakvM3EqvECXpPVMjKHPkeB5EBk7cLvfporsUyYVbbt2LuZ8kkdGOlVXPt9I 3Q/0IGN1/DEQ05i1HMpoqoKLdjq7Nh4mYfYdPnKJdDAEHqF4jPVHUP2Z2ODWNRn07JoN Q7mQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20221208 header.b=LthQP2K3; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b7-20020aa7df87000000b0051de3b9e331si14047788edy.561.2023.07.17.02.24.00; Mon, 17 Jul 2023 02:24:00 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20221208 header.b=LthQP2K3; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2058B68C622; Mon, 17 Jul 2023 12:23:58 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0DF6D68C518 for ; Mon, 17 Jul 2023 12:23:50 +0300 (EEST) Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-57704aa6c69so38662577b3.0 for ; Mon, 17 Jul 2023 02:23:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689585829; x=1692177829; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=MNed8kY8JgEV6ITOmSb7XRQlOH2qYweaiTkkCcYuwCI=; b=LthQP2K36Xxlr5WH7hY/jtWAawSz7VJObguIkW3E2WSKnlORxOdkRhOTngbbH6727f 4ry8JeN/NH9CRVvQwCCKgqcTcCnWJLJQlXCAROcV4rTrqRTtXy9bNxHLTaLQFzlcnpb0 zUTJkQtxtC+hDFN7VfVdeQPIRb+HX+yf0YU1WtyvjUMBCnDhcE72sqAvPxnEtI2Rzfq5 dilsHtWtyXlLoKiWyKuU4pgI5FirpbVViIIGKhPR4bxPKL5YulMP4Xc+dIemXgHvz5Xu aVGtj+aNWqPgBl/4WV0mBk/RNRJpDWZ6IJDOWnRjYTxF+xYlWT1BMeZLRR7mZhuujW3z AD2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689585829; x=1692177829; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=MNed8kY8JgEV6ITOmSb7XRQlOH2qYweaiTkkCcYuwCI=; b=gsCKFlmf/ZDqW6NRFDbyHl4dw9WgpT8TSGgXB3tFzOtmBIxGcWaWu+J+vSjfMQ0jEb pkS1IlsNGeAD5iJXvtQ8KfalrIjKBlT1r/cqAXh/AeRo4jquvfaRLupw0qIiOjBEtIiO BPOYIf4e9nhFqgYxBpQl960mXdPBEa8eKWubeGQ8CPLZDEObuXgSW5cACeE7Eh8ZCJay sHJlcqzAYgVZ8ErQ14cnR2tYQ8N+bOqYqlDWVtIHrTn6ifjJ3YS83eZRqNZquXadY0dA UuRB1BIe3PLaF4pcZHoBr4F83jxno5lwP/anzjvgUokxo+ukChjnP1o+xFwHs8kW3eQx 4HgQ== X-Gm-Message-State: ABy/qLZpO6VrDJxRhy3rNyUF+sB5T9YCBLo62Og6h5UJMqHTROV61rWo mgU+k8LPmeAE8/7D1kp6XXw/+WXfTEGA9d5mkm2Z2YoSCsV1rjxFZWE/eJ8ZEIzyQqBi+o0bRl7 H34lOVnPHFbzEUXgqd8V9bp8DyKzVOO8jApNXkvGMw+le1HwbLnhAbTUCSRgp5/6u903ou+8= X-Received: from alankelly0.zrh.corp.google.com ([2a00:79e0:9e:2:7bb6:b144:b7b0:2569]) (user=alankelly job=sendgmr) by 2002:a81:a709:0:b0:576:6e4e:b87f with SMTP id e9-20020a81a709000000b005766e4eb87fmr152187ywh.10.1689585828877; Mon, 17 Jul 2023 02:23:48 -0700 (PDT) Date: Mon, 17 Jul 2023 11:23:33 +0200 In-Reply-To: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230717092338.2075646-1-alankelly@google.com> From: Alan Kelly To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 2/3] swscale/x86/yuv2yuvX: Add yuv2yuvX avx512 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Alan Kelly Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: eiFevf+onI5O --- Checks for EXTERNAL_AVX512ICL to prevent downclocking on Skylake libswscale/x86/swscale.c | 7 +++++++ libswscale/x86/yuv2yuvX.asm | 19 ++++++++++++++++++- 2 files changed, 25 insertions(+), 1 deletion(-) diff --git a/libswscale/x86/swscale.c b/libswscale/x86/swscale.c index 8c67bf4fab..600c7d6c91 100644 --- a/libswscale/x86/swscale.c +++ b/libswscale/x86/swscale.c @@ -225,6 +225,9 @@ YUV2YUVX_FUNC(sse3, 32, mmxext) #if HAVE_AVX2_EXTERNAL YUV2YUVX_FUNC(avx2, 64, sse3) #endif +#if ARCH_X86_64 && HAVE_AVX512_EXTERNAL +YUV2YUVX_FUNC(avx512, 128, avx2) +#endif #define SCALE_FUNC(filter_n, from_bpc, to_bpc, opt) \ void ff_hscale ## from_bpc ## to ## to_bpc ## _ ## filter_n ## _ ## opt( \ @@ -467,6 +470,10 @@ av_cold void ff_sws_init_swscale_x86(SwsContext *c) #if HAVE_AVX2_EXTERNAL if (EXTERNAL_AVX2_FAST(cpu_flags)) c->yuv2planeX = yuv2yuvX_avx2; +#endif +#if ARCH_X86_64 && HAVE_AVX512_EXTERNAL + if (EXTERNAL_AVX512ICL(cpu_flags)) + c->yuv2planeX = yuv2yuvX_avx512; #endif } #if ARCH_X86_32 && !HAVE_ALIGNED_STACK diff --git a/libswscale/x86/yuv2yuvX.asm b/libswscale/x86/yuv2yuvX.asm index 369c850674..57bfa09d66 100644 --- a/libswscale/x86/yuv2yuvX.asm +++ b/libswscale/x86/yuv2yuvX.asm @@ -22,6 +22,10 @@ %include "libavutil/x86/x86util.asm" +SECTION_RODATA 64 + +permutation: dq 0, 2, 4, 6, 1, 3, 5, 7 + SECTION .text ;----------------------------------------------------------------------------- @@ -50,6 +54,10 @@ cglobal yuv2yuvX, 7, 7, 8, filter, filterSize, src, dest, dstW, dither, offset %else movq xm3, [ditherq] %endif ; avx2 + +%if cpuflag(avx512) + mova m15, [permutation] +%endif cmp offsetd, 0 jz .offset @@ -109,7 +117,10 @@ cglobal yuv2yuvX, 7, 7, 8, filter, filterSize, src, dest, dstW, dither, offset packuswb m6, m6, m1 %endif mov srcq, [filterq] -%if cpuflag(avx2) +%if cpuflag(avx512) + vpermt2q m3, m15, m3 + vpermt2q m6, m15, m6 +%elif cpuflag(avx2) vpermq m3, m3, 216 vpermq m6, m6, 216 %endif @@ -131,4 +142,10 @@ YUV2YUVX_FUNC %if HAVE_AVX2_EXTERNAL INIT_YMM avx2 YUV2YUVX_FUNC +%if HAVE_AVX512_EXTERNAL +%if ARCH_X86_64 +INIT_ZMM avx512 +YUV2YUVX_FUNC +%endif +%endif %endif From patchwork Mon Jul 17 09:30:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Kelly X-Patchwork-Id: 42783 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:6da1:b0:131:a7d0:bc6d with SMTP id gl33csp5256863pzb; Mon, 17 Jul 2023 02:30:32 -0700 (PDT) X-Google-Smtp-Source: APBJJlEH2wHD6Jmqr7hGQ318iZw5pN2kzze6ZrMpnFZk3Y1AnMB1pcnwom0uvRK4cdFyl1GUtinP X-Received: by 2002:a05:6402:1b19:b0:51e:156c:bb32 with SMTP id by25-20020a0564021b1900b0051e156cbb32mr11036440edb.12.1689586232171; Mon, 17 Jul 2023 02:30:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689586232; cv=none; d=google.com; s=arc-20160816; b=FqoKy5REorl7VUOFYOseD7menK0Qk6tZk5qfAqHV5VL1PcQ8HmKbX0ZYouC0zOART5 YXrB1nR2ggUAFmp1FDTldP3XD2Zbu5PVf9V3AWnb2BNy9cByLA94W9AKDaXIZSaG+Fdd qy4bVfm2iYyGEvMu5XmGAfD3yS6IULNdznkoq+xL/+wiS/ds0Qv4rpdGxB8qrZP6k7VT ZM2+CNRIJbNDanaHnETAYEDNucyq6LW7cBUtXBqJOlUPKEzKk8fjCMsUHkHrvKt66jNp zbFPN+Unh4U3tVLCQrY4Wa+q9kpnh4rEKZOcrxtxAXP5LhLIs9cRQeYIu7ITePzfgFJc eRlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:from:message-id:references :mime-version:in-reply-to:date:dkim-signature:delivered-to; bh=4374cgUsYDWVGiaTNftqVrcqHca+vEMJ2hwN5NTp7WU=; fh=/dichZGS11g2zj1pQZN5I1lofI3DSP0AKY/Bo6XKkZk=; b=D937sb8hJIvTVvwoCpY12nl0KLyy1GHEh10sj5h0GuSRSRqBKr44PRT/Eho0Sf8XkK D6cO9j/1ObPKIL1ILsuC9HQaFXzNS7pLb4vzqxiKQS4QljBCPSFzRPIkNzNLf/qioaLQ i+BvUXhYh9CScnVzxOcnZnsp4fJEs3Ws6BkrwbW4VoGQmBoFtub7PaIE6Q5otwLijUI5 SF752w3b2EwxlgjpxD3QnGTHxJI2BR0N4tXSJpq/Bl5Y49ze+j31pBXWI6BcYhfVtmwU bB/I4gxbg935K2HW9bh9yoG7xuQZL/PXv+WS1rGBcbTkXajwAP5sbQ5wLOlstQ3nL7h1 dq8w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20221208 header.b=amF0jGhM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d8-20020a05640208c800b0052177c07808si3622354edz.487.2023.07.17.02.30.31; Mon, 17 Jul 2023 02:30:32 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20221208 header.b=amF0jGhM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BEFCC68C612; Mon, 17 Jul 2023 12:30:29 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id F09FC68C4F3 for ; Mon, 17 Jul 2023 12:30:22 +0300 (EEST) Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-31443e13f9dso2464153f8f.0 for ; Mon, 17 Jul 2023 02:30:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689586222; x=1692178222; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=dmjPv0Ly9VG9HtGeIA9JK3EK7trqxbl+M61hQ7+Wdtk=; b=amF0jGhMpuUU70nfcDFka5W0a+3EJiwYCZ/oo225JZNGanFRfnasi3icd5xQe3m16y 180veAJb32ZF9gFaxxGM7rE57wKq65lo2HL+8TtoEu+pOQWBK8loELTi+Clke5d4HyvI EJqaZ9SoifeC8/riBjITMj1qdzbe93EI6Ak4OZUQ16dabFX/jA40S1y3JM+ehG8Cqrk9 kOezNwtOfz8l7eaWkwWqsTW1IvMqvGPlQ25UAL5CBOHTFSkT12Q8HkxhRoCitUnTRuTT 1Wszl1mMeJYYvnWaT0LxD3oDT+3R5BHRMHmg+HfpFXgXZjn4lJtbcEMpxzDCJnWX318Z wxkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689586222; x=1692178222; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dmjPv0Ly9VG9HtGeIA9JK3EK7trqxbl+M61hQ7+Wdtk=; b=e4gpeM6xlnTOH3TZd/HJkAqStc17wKu9Gas6uhpnIcHJzKetsBghdyfxzfiq0fWNxo ONVJ8CwXsBxwGe0yF0ne1baeEeBRdMC9OrD0/xBNJHM9Sl1qmfTbEQLQZ9lJK9W/RzcU vMWEY3jKfc0WYmYKJxjZ/m96trOSUDXFH/fSXg2Yzzm3M2ZQDcanE9LRjfT5p1PEPy83 NcsiR6wTD1ohfunZEfdNfKyjt1ZR11D4DXUmsuWH+789CDMNN8x4NN8/E5o+lAa+0v/h 87qKXywwQlnXtbiQXCe9NY2SWFCS+igqRSBPi4l6Ea6zJXXFufT6hNfGmiS6lUior8kF fFdA== X-Gm-Message-State: ABy/qLbe6jTkAcLkRFkqCHTBTvLP/o/RLWOasxXw6kF6mzwKkQ59dFKN i2d8LSlOsh+bAZZNiq6+7g4K8vn2+HOMc/yl/vyET/6eGgLwJ5wh1/pI8k/N8IQMcUP26rXjC8A 6WJBatn4GwZ5RSopP+/gZv+JiN6/ztXoGIiCb7R14zhOzb9ZH0ueHte+sptVCRxreeA3ftWs= X-Received: from alankelly0.zrh.corp.google.com ([2a00:79e0:9e:2:7bb6:b144:b7b0:2569]) (user=alankelly job=sendgmr) by 2002:adf:ef12:0:b0:314:1043:1d98 with SMTP id e18-20020adfef12000000b0031410431d98mr82227wro.13.1689586222234; Mon, 17 Jul 2023 02:30:22 -0700 (PDT) Date: Mon, 17 Jul 2023 11:30:09 +0200 In-Reply-To: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230717093018.2076410-1-alankelly@google.com> From: Alan Kelly To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 3/3] swscale/x86/yuv2yuvX: Process tails by jumping back into the main loop. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Alan Kelly Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: smsTqu4mkghW --- libswscale/x86/swscale.c | 11 ++++------- libswscale/x86/yuv2yuvX.asm | 24 ++++++++++++++++++------ 2 files changed, 22 insertions(+), 13 deletions(-) diff --git a/libswscale/x86/swscale.c b/libswscale/x86/swscale.c index 600c7d6c91..6980002e9e 100644 --- a/libswscale/x86/swscale.c +++ b/libswscale/x86/swscale.c @@ -202,17 +202,14 @@ static void yuv2yuvX_ ##opt(const int16_t *filter, int filterSize, \ const int16_t **src, uint8_t *dest, int dstW, \ const uint8_t *dither, int offset) \ { \ - int remainder = (dstW % step); \ - int pixelsProcessed = dstW - remainder; \ if(((uintptr_t)dest) & 15){ \ yuv2yuvX_mmxext(filter, filterSize, src, dest, dstW, dither, offset); \ return; \ } \ - if(pixelsProcessed > 0) \ - ff_yuv2yuvX_ ##opt(filter, filterSize - 1, 0, dest - offset, pixelsProcessed + offset, dither, offset); \ - if(remainder > 0){ \ - yuv2yuvX_ ##tail(filter, filterSize, src, dest, dstW, dither, offset); \ - } \ + if (dstW >= step) \ + ff_yuv2yuvX_ ##opt(filter, filterSize - 1, 0, dest - offset, dstW + offset, dither, offset); \ + else \ + yuv2yuvX_ ##tail(filter, filterSize, src, dest, dstW, dither, offset); \ return; \ } diff --git a/libswscale/x86/yuv2yuvX.asm b/libswscale/x86/yuv2yuvX.asm index 57bfa09d66..03bfd6ad1d 100644 --- a/libswscale/x86/yuv2yuvX.asm +++ b/libswscale/x86/yuv2yuvX.asm @@ -54,6 +54,8 @@ cglobal yuv2yuvX, 7, 7, 8, filter, filterSize, src, dest, dstW, dither, offset %else movq xm3, [ditherq] %endif ; avx2 + mov ditherq, dstWq + sub dstWq, mmsize * unroll %if cpuflag(avx512) mova m15, [permutation] @@ -92,13 +94,17 @@ cglobal yuv2yuvX, 7, 7, 8, filter, filterSize, src, dest, dstW, dither, offset %else mova m0, [filterSizeq + 8] %endif - pmulhw m2, m0, [srcq + offsetq * 2] - pmulhw m5, m0, [srcq + offsetq * 2 + mmsize] + movu m2, [srcq + offsetq * 2] + movu m5, [srcq + offsetq * 2 + mmsize] + pmulhw m2, m0, m2 + pmulhw m5, m0, m5 paddw m3, m3, m2 paddw m4, m4, m5 %if cpuflag(sse3) - pmulhw m2, m0, [srcq + offsetq * 2 + 2 * mmsize] - pmulhw m5, m0, [srcq + offsetq * 2 + 3 * mmsize] + movu m2, [srcq + offsetq * 2 + 2 * mmsize] + movu m5, [srcq + offsetq * 2 + 3 * mmsize] + pmulhw m2, m0, m2 + pmulhw m5, m0, m5 paddw m6, m6, m2 paddw m1, m1, m5 %endif @@ -131,8 +137,14 @@ cglobal yuv2yuvX, 7, 7, 8, filter, filterSize, src, dest, dstW, dither, offset add offsetq, mmsize * unroll mov filterSizeq, filterq cmp offsetq, dstWq - jb .outerloop - RET + jb .outerloop + + mov dstWq, offsetq + mov offsetq, ditherq + sub offsetq, mmsize * unroll + cmp dstWq, ditherq + jb .outerloop + REP_RET %endmacro INIT_MMX mmxext