From patchwork Mon Sep 23 12:40:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ramiro Polla X-Patchwork-Id: 51752 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:d154:0:b0:48e:c0f8:d0de with SMTP id bt20csp2472994vqb; Mon, 23 Sep 2024 06:59:23 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWLy551bPfXO19n17oANUZOGXXUFeAtZMPlSrWOAeD5UkmXIodbe5+OXqgppv1QFXf3NiqOtKtwH2K5XtpeuQf6@gmail.com X-Google-Smtp-Source: AGHT+IHKx+Plgx4CK7cQwnKNOaNh6pR0ZSyC2N2jQBVZEbaAC/9emnc5ZcGVCl6QR+b2G0StkkAO X-Received: by 2002:a05:6512:1589:b0:536:7d0b:ece3 with SMTP id 2adb3069b0e04-536ac346abemr1582288e87.10.1727099962898; Mon, 23 Sep 2024 06:59:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1727099962; cv=none; d=google.com; s=arc-20240605; b=FAT498+9XZfcr2h5P+LtC5w4yBJWF4akRFOy2cixlDRsWaY/GeNxQ6SgWKRkUSDXcO avtjvmigLOATSwv30dNh4BajA5bQYKMvH5Jo5CEPw5CfgnHbD9l5Tgp7ZiuqXFuGQhXh AghO7t/h3/14UT9ChV+jbpw0raf8RloGWw0w6f5RgsPaCAFZQ01vsOStMUHsKz7Lw4Um SIFJkNAUhsSDUKeJV+L37nFyWmouWRgW1z5yY/Z4jLhJZ9RBNAJ5LY7of3I5XlZUih8l lRRv+MMX8obfpeaNRNJlvEnLgWMlegxkAMWvD2Py9aWxEQZXUArpt1fBalntdAJ4JFml LjcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=v/ctQwLmk78ufR+IBXXtL43tYmspuDTwvxHok+QTZyE=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=jC60dzgtBYMIsg4v9Ag9uN8Dtg4h8eRa7KvECV8d/7y2eNSjNxvUpV3WkO/Wse+Sak NiIOVPhR8tj1ftMpaigjWNKMjZqYt1dqjjH2WNl5Dz/mSHPDCdBcOeMO70NYLtwNCmlA 4yBy4F8WOkdDxdd57mFvkUWqFHnz9flFbOjMdwHEgB48Z52Q51cKt5uTDs4xWhMqCcDN P+RXj0NFxGVd9D4MtwFTOIdJXI+WoZBLKO7V4EVWH4GnvQX/cQ+uVMXzInR6+jks4eFv uesvvrM1jzoN6G6S0NMZctOmrAMT23vnUVvs7+bw6DzWTh247/WTRosGnKaHdh7n6zsE dz7w==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=eAw4Wu8t; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 2adb3069b0e04-536870afe53si7178944e87.485.2024.09.23.06.59.20; Mon, 23 Sep 2024 06:59:22 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=eAw4Wu8t; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7DB2F68DAE2; Mon, 23 Sep 2024 15:40:44 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0654068DB5E for ; Mon, 23 Sep 2024 15:40:35 +0300 (EEST) Received: by mail-wr1-f49.google.com with SMTP id ffacd0b85a97d-37a413085cbso2135463f8f.1 for ; Mon, 23 Sep 2024 05:40:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727095235; x=1727700035; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=C0QZDVZJkVyF4d5LexRqtvx3J9Q6qCTo0vqcFfMh3og=; b=eAw4Wu8tP+XltdiPV2t5VnEUT3ektMErhOPyJQlXhocmd0MPEvrk7dHpYG4PVLI7+U iSHU3+GRvfE814c6x0sfbpS/ShTw3HZoYwjPIHVBy0MwkteVAouW9zS/Ph+Wzdjj3PEX WdM6SE76m6Vta9rqUwoRE5Vr0hoidXu6DL+Ui4fkPP1jgIeb1oF7305W2Hr5nSxmEykd iiMS6RomvqDjn+k3Wj+7tmneBRFea9YswHyllmeoB6WhHFM7aLAkx7TveIWAosti74Rw yRiO2anH4LJFkukPcuoUIX8FGu13GjVOD8TsuZASw/Bgp1MTGQZytCWWhdEKW8kAOFAn IJ8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727095235; x=1727700035; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C0QZDVZJkVyF4d5LexRqtvx3J9Q6qCTo0vqcFfMh3og=; b=gELR/FVFoTv+iOzaqnRVp44yQLAOxeFPasZgg9z1UUuBs2l5p5alxCcDAFoV41oF7L ztdXjdP0Am8FWT5yGesTtl3CVbDdjujMticgxbomXEAvQH5KWLm+jauvvt72camOxMBN w6h6I1Cxi0QwbW3bU0AvJdiavzgL5fv7jvaBfzG+4f8lT/eEdQQzeWZA99gm0nthKwNN Szysqs7bM+zcvnFLycGbkSTeyNTMK2Dqxg2VZOAIIAlkKMqkSkMpqQnb0dvDnfkphOv5 FJV4RrfCaUP5ieA1aj6Tcer0vwpczSXcMEwnf9A/pSFFCiP5lIxLthNpfbfXjQ4ed2Lk y1Ew== X-Gm-Message-State: AOJu0YzEjmnf6zwcBIWwv8t6fWvSV7rqRnj7igeb7dkYb5KYf7Fi8vdP /AnIe5enDFu2O34bR9PjywmIVduup0QsLrfjXQUcWjXfaDVzW7/LgUS+bg== X-Received: by 2002:adf:e348:0:b0:374:c79d:5f7e with SMTP id ffacd0b85a97d-379a860b6f9mr8438321f8f.26.1727095234577; Mon, 23 Sep 2024 05:40:34 -0700 (PDT) Received: from localhost.localdomain (213.95-240-81.adsl-dyn.isp.belgacom.be. [81.240.95.213]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-378e71f0683sm24424345f8f.13.2024.09.23.05.40.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 05:40:33 -0700 (PDT) From: Ramiro Polla To: ffmpeg-devel@ffmpeg.org Date: Mon, 23 Sep 2024 14:40:15 +0200 Message-Id: <20240923124017.33659-13-ramiro.polla@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240923124017.33659-1-ramiro.polla@gmail.com> References: <20240923124017.33659-1-ramiro.polla@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 12/14] swscale/x86: add sse2, sse4, and avx2 {lum, chr}ConvertRange16 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: B8L9YWL66JmH chrRangeFromJpeg16_1920_c: 3893.0 ( 1.00x) chrRangeFromJpeg16_1920_sse2: 3249.0 ( 1.20x) chrRangeFromJpeg16_1920_avx2: 1636.0 ( 2.38x) chrRangeToJpeg16_1920_c: 7731.0 ( 1.00x) chrRangeToJpeg16_1920_sse2: 4722.8 ( 1.64x) chrRangeToJpeg16_1920_sse4: 3715.0 ( 2.08x) chrRangeToJpeg16_1920_avx2: 1859.0 ( 4.16x) lumRangeFromJpeg16_1920_c: 2385.5 ( 1.00x) lumRangeFromJpeg16_1920_sse2: 1636.0 ( 1.46x) lumRangeFromJpeg16_1920_avx2: 839.8 ( 2.84x) lumRangeToJpeg16_1920_c: 3533.0 ( 1.00x) lumRangeToJpeg16_1920_sse2: 2362.2 ( 1.50x) lumRangeToJpeg16_1920_sse4: 1861.0 ( 1.90x) lumRangeToJpeg16_1920_avx2: 946.0 ( 3.73x) --- libswscale/x86/range_convert.asm | 133 ++++++++++++++++++++++++++----- libswscale/x86/swscale.c | 50 +++++++++--- 2 files changed, 149 insertions(+), 34 deletions(-) diff --git a/libswscale/x86/range_convert.asm b/libswscale/x86/range_convert.asm index d1aff63d7c..83dbe6c4b8 100644 --- a/libswscale/x86/range_convert.asm +++ b/libswscale/x86/range_convert.asm @@ -25,20 +25,28 @@ SECTION .text ;----------------------------------------------------------------------------- ; lumConvertRange ; -; void ff_lumRangeToJpeg_(int16_t *dst, int width, -; int amax, int coeff, int64_t offset); -; void ff_lumRangeFromJpeg_(int16_t *dst, int width, -; int amax, int coeff, int64_t offset); +; void ff_lumRangeToJpeg{8,16}_(int16_t *dst, int width, +; int amax, int coeff, int64_t offset); +; void ff_lumRangeFromJpeg{8,16}_(int16_t *dst, int width, +; int amax, int coeff, int64_t offset); ; ;----------------------------------------------------------------------------- -%macro LUMCONVERTRANGE 1 +%macro LUMCONVERTRANGE 2 %ifidni %1,To -cglobal lumRange%1Jpeg, 5, 5, 6, dst, width, amax, coeff, offset +%if %2 == 16 +cglobal lumRange%1Jpeg%2, 5, 5, 5, dst, width, amax, coeff, offset +%elif %2 == 8 +cglobal lumRange%1Jpeg%2, 5, 5, 6, dst, width, amax, coeff, offset +%endif ; %2 == 8/16 %else -cglobal lumRange%1Jpeg, 5, 5, 5, dst, width, amax, coeff, offset +cglobal lumRange%1Jpeg%2, 5, 5, 5, dst, width, amax, coeff, offset %endif +%if %2 == 16 + shl widthd, 2 +%elif %2 == 8 shl widthd, 1 +%endif ; %2 == 8/16 movd xm2, coeffd VBROADCASTSS m2, xm2 %if ARCH_X86_64 @@ -46,16 +54,40 @@ cglobal lumRange%1Jpeg, 5, 5, 5, dst, width, amax, coeff, offset %else movq xm3, offsetm %endif +%if %2 == 16 + VBROADCASTSD m3, xm3 +%elif %2 == 8 VBROADCASTSS m3, xm3 pxor m4, m4 +%endif ; %2 == 8/16 %ifidni %1,To +%if %2 == 16 + movd xm4, amaxd + VBROADCASTSS m4, xm4 +%elif %2 == 8 movd xm5, amaxd SPLATW m5, xm5 +%endif ; %2 == 8/16 %endif add dstq, widthq neg widthq .loop: movu m0, [dstq+widthq] +%if %2 == 16 +%ifidni %1,To + PMINSD m0, m4, m1 +%endif + pshufd m1, m0, 0xb1 + pmuludq m0, m2 + pmuludq m1, m2 + paddq m0, m3 + paddq m1, m3 + psrlq m0, 18 + psrlq m1, 18 + pshufd m0, m0, 0xd8 + pshufd m1, m1, 0xd8 + punpckldq m0, m1 +%elif %2 == 8 %ifidni %1,To pminsw m0, m5 %endif @@ -68,6 +100,7 @@ cglobal lumRange%1Jpeg, 5, 5, 5, dst, width, amax, coeff, offset psrad m0, 14 psrad m1, 14 packssdw m0, m1 +%endif ; %2 == 8/16 movu [dstq+widthq], m0 add widthq, mmsize jl .loop @@ -77,20 +110,28 @@ cglobal lumRange%1Jpeg, 5, 5, 5, dst, width, amax, coeff, offset ;----------------------------------------------------------------------------- ; chrConvertRange ; -; void ff_chrRangeToJpeg_(int16_t *dstU, int16_t *dstV, int width, -; int amax, int coeff, int64_t offset); -; void ff_chrRangeFromJpeg_(int16_t *dstU, int16_t *dstV, int width, -; int amax, int coeff, int64_t offset); +; void ff_chrRangeToJpeg{8,16}_(int16_t *dstU, int16_t *dstV, int width, +; int amax, int coeff, int64_t offset); +; void ff_chrRangeFromJpeg{8,16}_(int16_t *dstU, int16_t *dstV, int width, +; int amax, int coeff, int64_t offset); ; ;----------------------------------------------------------------------------- -%macro CHRCONVERTRANGE 1 +%macro CHRCONVERTRANGE 2 %ifidni %1,To -cglobal chrRange%1Jpeg, 6, 6, 8, dstU, dstV, width, amax, coeff, offset +%if %2 == 16 +cglobal chrRange%1Jpeg%2, 6, 6, 7, dstU, dstV, width, amax, coeff, offset +%elif %2 == 8 +cglobal chrRange%1Jpeg%2, 6, 6, 8, dstU, dstV, width, amax, coeff, offset +%endif ; %2 == 8/16 %else -cglobal chrRange%1Jpeg, 6, 6, 7, dstU, dstV, width, amax, coeff, offset +cglobal chrRange%1Jpeg%2, 6, 6, 7, dstU, dstV, width, amax, coeff, offset %endif +%if %2 == 16 + shl widthd, 2 +%elif %2 == 8 shl widthd, 1 +%endif ; %2 == 8/16 movd xm4, coeffd VBROADCASTSS m4, xm4 %if ARCH_X86_64 @@ -98,11 +139,20 @@ cglobal chrRange%1Jpeg, 6, 6, 7, dstU, dstV, width, amax, coeff, offset %else movq xm5, offsetm %endif +%if %2 == 16 + VBROADCASTSD m5, xm5 +%elif %2 == 8 VBROADCASTSS m5, xm5 pxor m6, m6 +%endif ; %2 == 8/16 %ifidni %1,To +%if %2 == 16 + movd xm6, amaxd + VBROADCASTSS m6, xm6 +%elif %2 == 8 movd xm7, amaxd SPLATW m7, xm7 +%endif ; %2 == 8/16 %endif add dstUq, widthq add dstVq, widthq @@ -110,6 +160,32 @@ cglobal chrRange%1Jpeg, 6, 6, 7, dstU, dstV, width, amax, coeff, offset .loop: movu m0, [dstUq+widthq] movu m2, [dstVq+widthq] +%if %2 == 16 +%ifidni %1,To + PMINSD m0, m6, m1 + PMINSD m2, m6, m3 +%endif + pshufd m1, m0, 0xb1 + pshufd m3, m2, 0xb1 + pmuludq m0, m4 + pmuludq m1, m4 + pmuludq m2, m4 + pmuludq m3, m4 + paddq m0, m5 + paddq m1, m5 + paddq m2, m5 + paddq m3, m5 + psrlq m0, 18 + psrlq m1, 18 + psrlq m2, 18 + psrlq m3, 18 + pshufd m0, m0, 0xd8 + pshufd m1, m1, 0xd8 + pshufd m2, m2, 0xd8 + pshufd m3, m3, 0xd8 + punpckldq m0, m1 + punpckldq m2, m3 +%elif %2 == 8 %ifidni %1,To pminsw m0, m7 pminsw m2, m7 @@ -132,6 +208,7 @@ cglobal chrRange%1Jpeg, 6, 6, 7, dstU, dstV, width, amax, coeff, offset psrad m3, 14 packssdw m0, m1 packssdw m2, m3 +%endif ; %2 == 8/16 movu [dstUq+widthq], m0 movu [dstVq+widthq], m2 add widthq, mmsize @@ -140,15 +217,27 @@ cglobal chrRange%1Jpeg, 6, 6, 7, dstU, dstV, width, amax, coeff, offset %endmacro INIT_XMM sse2 -LUMCONVERTRANGE To -CHRCONVERTRANGE To -LUMCONVERTRANGE From -CHRCONVERTRANGE From +LUMCONVERTRANGE To, 8 +LUMCONVERTRANGE To, 16 +CHRCONVERTRANGE To, 8 +CHRCONVERTRANGE To, 16 +LUMCONVERTRANGE From, 8 +LUMCONVERTRANGE From, 16 +CHRCONVERTRANGE From, 8 +CHRCONVERTRANGE From, 16 + +INIT_XMM sse4 +LUMCONVERTRANGE To, 16 +CHRCONVERTRANGE To, 16 %if HAVE_AVX2_EXTERNAL INIT_YMM avx2 -LUMCONVERTRANGE To -CHRCONVERTRANGE To -LUMCONVERTRANGE From -CHRCONVERTRANGE From +LUMCONVERTRANGE To, 8 +LUMCONVERTRANGE To, 16 +CHRCONVERTRANGE To, 8 +CHRCONVERTRANGE To, 16 +LUMCONVERTRANGE From, 8 +LUMCONVERTRANGE From, 16 +CHRCONVERTRANGE From, 8 +CHRCONVERTRANGE From, 16 %endif diff --git a/libswscale/x86/swscale.c b/libswscale/x86/swscale.c index 2377365e91..c49f32c5ef 100644 --- a/libswscale/x86/swscale.c +++ b/libswscale/x86/swscale.c @@ -454,26 +454,46 @@ INPUT_PLANAR_RGB_A_ALL_DECL(avx2); #define RANGE_CONVERT_FUNCS(opt) do { \ if (c->dstBpc <= 14) { \ if (c->srcRange) { \ - c->lumConvertRange = ff_lumRangeFromJpeg_ ##opt; \ - c->chrConvertRange = ff_chrRangeFromJpeg_ ##opt; \ + c->lumConvertRange = ff_lumRangeFromJpeg8_ ##opt; \ + c->chrConvertRange = ff_chrRangeFromJpeg8_ ##opt; \ } else { \ - c->lumConvertRange = ff_lumRangeToJpeg_ ##opt; \ - c->chrConvertRange = ff_chrRangeToJpeg_ ##opt; \ + c->lumConvertRange = ff_lumRangeToJpeg8_ ##opt; \ + c->chrConvertRange = ff_chrRangeToJpeg8_ ##opt; \ + } \ + } else { \ + if (c->srcRange) { \ + c->lumConvertRange = ff_lumRangeFromJpeg16_ ##opt; \ + c->chrConvertRange = ff_chrRangeFromJpeg16_ ##opt; \ + } else { \ + c->lumConvertRange = ff_lumRangeToJpeg16_ ##opt; \ + c->chrConvertRange = ff_chrRangeToJpeg16_ ##opt; \ } \ } \ } while (0) #define RANGE_CONVERT_FUNCS_DECL(opt) \ -void ff_lumRangeFromJpeg_ ##opt(int16_t *dst, int width, \ +void ff_lumRangeFromJpeg8_ ##opt(int16_t *dst, int width, \ + int amax, int coeff, int64_t offset); \ +void ff_chrRangeFromJpeg8_ ##opt(int16_t *dstU, int16_t *dstV, int width, \ + int amax, int coeff, int64_t offset); \ +void ff_lumRangeToJpeg8_ ##opt(int16_t *dst, int width, \ + int amax, int coeff, int64_t offset); \ +void ff_chrRangeToJpeg8_ ##opt(int16_t *dstU, int16_t *dstV, int width, \ + int amax, int coeff, int64_t offset); \ +void ff_lumRangeFromJpeg16_ ##opt(int16_t *dst, int width, \ + int amax, int coeff, int64_t offset); \ +void ff_chrRangeFromJpeg16_ ##opt(int16_t *dstU, int16_t *dstV, int width, \ + int amax, int coeff, int64_t offset); \ +void ff_lumRangeToJpeg16_ ##opt(int16_t *dst, int width, \ int amax, int coeff, int64_t offset); \ -void ff_chrRangeFromJpeg_ ##opt(int16_t *dstU, int16_t *dstV, int width, \ +void ff_chrRangeToJpeg16_ ##opt(int16_t *dstU, int16_t *dstV, int width, \ int amax, int coeff, int64_t offset); \ -void ff_lumRangeToJpeg_ ##opt(int16_t *dst, int width, \ - int amax, int coeff, int64_t offset); \ -void ff_chrRangeToJpeg_ ##opt(int16_t *dstU, int16_t *dstV, int width, \ - int amax, int coeff, int64_t offset); \ RANGE_CONVERT_FUNCS_DECL(sse2); +void ff_lumRangeToJpeg16_sse4(int16_t *dst, int width, + int amax, int coeff, int64_t offset); +void ff_chrRangeToJpeg16_sse4(int16_t *dstU, int16_t *dstV, int width, + int amax, int coeff, int64_t offset); RANGE_CONVERT_FUNCS_DECL(avx2); av_cold void ff_sws_init_range_convert_x86(SwsContext *c) @@ -481,8 +501,14 @@ av_cold void ff_sws_init_range_convert_x86(SwsContext *c) int cpu_flags = av_get_cpu_flags(); if (EXTERNAL_AVX2_FAST(cpu_flags)) { RANGE_CONVERT_FUNCS(avx2); - } else if (EXTERNAL_SSE2(cpu_flags)) { - RANGE_CONVERT_FUNCS(sse2); + } else { + if (EXTERNAL_SSE2(cpu_flags)) { + RANGE_CONVERT_FUNCS(sse2); + } + if (EXTERNAL_SSE4(cpu_flags) && c->dstBpc > 14 && !c->srcRange) { + c->lumConvertRange = ff_lumRangeToJpeg16_sse4; + c->chrConvertRange = ff_chrRangeToJpeg16_sse4; + } } }