From patchwork Wed Jan 17 02:15:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nuo Mi X-Patchwork-Id: 45621 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c58a:b0:199:de12:6fa6 with SMTP id gn10csp87218pzb; Tue, 16 Jan 2024 18:16:59 -0800 (PST) X-Google-Smtp-Source: AGHT+IGVLTJa6cdaxfpHOkV1oGfLO+1iXEYuHCR2Io4sJ03fUvKZnFqolyk7qqAQUyrIr/SBtCor X-Received: by 2002:aa7:c68c:0:b0:559:dc26:9e0a with SMTP id n12-20020aa7c68c000000b00559dc269e0amr35446edq.52.1705457819615; Tue, 16 Jan 2024 18:16:59 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id c10-20020a0564021f8a00b0055967c65c49si2034542edc.196.2024.01.16.18.16.59; Tue, 16 Jan 2024 18:16:59 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=mdYznF5a; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7FE2C68D08B; Wed, 17 Jan 2024 04:16:50 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from APC01-SG2-obe.outbound.protection.outlook.com (mail-sgaapc01olkn2100.outbound.protection.outlook.com [40.92.53.100]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 923D868D081 for ; Wed, 17 Jan 2024 04:16:43 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XoYhTjqS0GFV+c1knHtrIGvN3zrWomrd0fRIKwD9J1RRhngn6GgwznbwnR2KdV7akFnBxQfJZFeqmubnvLyQiywqodlegkwQafBs4+nZgAlwW0VqP9RAdXetgEpS8TGeIOCn3S/1iZeL3hZl8Bf3T1phyXSI0y9ye9F7xlakDExwoHYpPYT4NFWA/PZQwglrlkeyJRP9TkaTS33S0qTfAeznZIL42mSPSh6oISVJhSWrP1gq3Gq2vZd1nJEcN2gauIfVBg1Y81OULQfmNJyLWsPUbB+UzyM8kcfmTtZECTa9jWQdzaQdWxdclJw4Ix0bIrRxd9ElOoz5RHxB4zeZxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NbVdQ7Mhx5buZ9IsnGN4CCXkXoBUQk+2XUsakLWspaE=; b=IKKkLzouHzOHz/GYz+66oQJtClBFBpI4i60lDdb6yS1HINq6U4wtLVHDPTeF448TIgYKCj0OEJyyAK6/3kMYWXsSbCfZu82/uNnpssQk/j1cFnhM3WCzNAaptJt3L+mbg8Kb7R27haweG3soH4c51Nbc5hFfl0nAgWuy5KOG6SaRBppwBGZbcbRmKo4PwQbSDHWpNTTkEzunA9VYfUyA6oZ3Zfc+RCGn6WdtuHKjggkpRgX03Mia4LbdSQAmULjdqt8taR4yrweSvDg7103U4n9F3/Scu7iTMjRXRoUQtCrhlqnPImSMf8a3QbTbG6R+Fz868t/YVuj85o7EnTGUFQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NbVdQ7Mhx5buZ9IsnGN4CCXkXoBUQk+2XUsakLWspaE=; b=mdYznF5aBIozGGd+yEI0Q3+ip++mcHKjs0c1wEJdUC9U7rP8Ueoot42lElv1tDbgcLVv2YMEOzD7oPi5Y5+7Pq10OpRy7hHKIofepA0a03EfA6SFl1u4VvHR3AQ6N1PKU0IFfsOjiLfP8N9vlnYgxokR7PiqcwfzDckgY7CSy+E7yhKY8ILE5yzFPH2egkg+ZOssbIxddZHUmbn1xdGiO608drBgepaXx7hg9CgTWhg5Us6P526hWkKb8AfQo0Cs6bsByZiLAYtx09w2TooV2YUtLByzvq416D1eJPv5qMD7sjrYpZiBzOPgSsJTKbSDSoUa2v0e/4+PFd+2z8zqYg== Received: from TYSPR06MB6433.apcprd06.prod.outlook.com (2603:1096:400:47a::6) by SEYPR06MB5255.apcprd06.prod.outlook.com (2603:1096:101:84::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7181.26; Wed, 17 Jan 2024 02:16:22 +0000 Received: from TYSPR06MB6433.apcprd06.prod.outlook.com ([fe80::e139:fd7a:1b66:a3f7]) by TYSPR06MB6433.apcprd06.prod.outlook.com ([fe80::e139:fd7a:1b66:a3f7%7]) with mapi id 15.20.7202.020; Wed, 17 Jan 2024 02:16:22 +0000 From: Nuo Mi To: ffmpeg-devel@ffmpeg.org Date: Wed, 17 Jan 2024 10:15:32 +0800 Message-ID: X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240117021533.2965-1-nuomi2021@gmail.com> References: <20240117021533.2965-1-nuomi2021@gmail.com> X-TMN: [CqEzawnRzUi/i8+XvMJG+uibgZex708b] X-ClientProxiedBy: TYCP286CA0261.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:455::12) To TYSPR06MB6433.apcprd06.prod.outlook.com (2603:1096:400:47a::6) X-Microsoft-Original-Message-ID: <20240117021533.2965-2-nuomi2021@gmail.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 2 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TYSPR06MB6433:EE_|SEYPR06MB5255:EE_ X-MS-Office365-Filtering-Correlation-Id: 56556de0-682e-4deb-89b8-08dc17024c9f X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lhAb6f3kJa/RpF1Y9y90M+SVwKdSnYdCdhGUdv4Yj9/n0CRm2Y/vWAjkm0nxEJcDDI4mGGMBxg7PEvXBnWDXnPW9jsagr8RM0hh+C48cL6GLVtFT4BsE86n+nHALT4RGhY5Du4esmhRaQwPlhXprOJoryTFj+P+/7fyPoyiSP1S0Td3YNax2sLY+MFLp7AeVL1ttqTmVBjuNHLTM9oPne0zrRntxkLruKKh0mjBQsa0dwi5OsTjaZcpJyvxTlETQkZJ06K7/pnVQohTbrA8hoXb4px9NW0ANdFJYI+ITwVkAkfrlqAfP5jiaDRgOLSmpq5rcXaavOgGUeCgwJica5iSx4K/Qmg9KUjdV4Gv+fily59d5BHQRjvR7YJy1Z8nSX+VCHZPTLX8WY2ElLTbMMKOWXzrI/WBnH9W4RoR8otAe4kXQx72ZCgynlvgsLfJ+lEtw+47jhX4dNWYMQMEF/GLBsBDn4RQDIPrdAAF6IHIsXR+yxSk4QkI4qEV28f6v2niYhxeysskunV2P6suaS0sbbAVyxZcJM+9NEVfrg2FkyrcsdLliybkB+EVCXzW/Mej1c0useGS1ScJeEe+Ge66j41OwdgYZDzJGkpHafXqbMKNU/XAh2WtiyUP3Ehf6 X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: JZ4d5zT4PJjvv+kwXDl3Fl++Xp0ETHhXl9KFxTbkPrd+fQO3GZAGfG3aHGEH6LyMKJRUicaVSfyU0RNWkhZD5OS20SnwhihL8qkC+hvDVtLjGmb4gbRnoIiJLDo27oIhvbMnVNSDrGBnt0okWantgtuF2XrKkhliKLSQnHQAqwAfFqHucYNIvPUUNrSIV9qtGBJe+wuo/3iLt6v5LhM86sj23tPbPIiSsnb6vujwAQtBVRAzfy8ax2yd8Ar29otbyGv6ymaxRcKuFxxgjRDYv26kB6FGlUdMPRwePFpM2Qgenwm/TjIqwnHcDmOPhPtfVBSmPjO0ELcOAqCRPP01NUAWGo0Smbfqdxm1vit9oEs1zQBs2OEW9eMEsKFwZXeXMUWBCGdyQIxQu2lvzf/256ZYJUnhJz9FX145RZKIyn7V/xIs4d/mhcxHNWVWjE9dqGPUc2ToNXl73gibFdNCFKNfn1JDP4o/mIYFQc/JlFI/7J5lT3rgsRplr2ks2t07T2GmIar2qzdt1Kbjq8AqNS4/yWUdE3ok6pzM6syAKhsx/o+VLjnCZgv9VOCgw/zztyVG4XwF6oiSi3Z25ualw9hyCgeD7g9EfzsJ5knIYnqOhcJ/Ts4d88fw/kucVowtWE1R6PD1oZnEnIyFnCCgsS0Em1sKH3WcKAnGZLOKsBS715tfkpBzswc9nUvR63JfAo84PC8GXfBKIL7VAl2EUym7vujcrXVGMCX31IIiM/6DDvNigzLXs/pafXiQzDUgFy9pPjnTa4rQoKF5SOeLVRYoj4x+k+DrnXwwX39uQ/4eSu6YRzaGdbbwhrjGSjAmHT9o7+5rhUmqKjrdToszRUd8qwWKHkG7axNvPBygZyynwiWxJhRhJnpSCZPy6WyT2ThdIW47lk4uEufuMv8QdeS7zX8eRSHq3iVBjHiSpoVMeshtkXCUp8O3/IDNGL8j5MCQNwF+zMjk+Em+l3Ew3hzKAW9M5xHqUKj3Vz8CHOivhNPcpSMELeRaT+vf8YmfOgUKgO1VrUufFQuY0IDBxkrL+dm7R3Ue2cC04CcA2H38r6CMYyA/+w86QnOWrVI3jVB6ubirmDjkDSBSU/7su5r5fJefxVI7NVzhoTGmXnk/RgbY+OYWxRkHeBXa31OWY1Gqi/XTZmZlcsxj80n7ObbAhu6/m3JFqd2LIzYlNEBfepwH9TAOnS2a6Ar6noobjFE7mseWLIkJ4LFXXxYjYYrwbqQXjZVKKr+RV/Rga150fDLVH10sHITcr5jeVe++ X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 56556de0-682e-4deb-89b8-08dc17024c9f X-MS-Exchange-CrossTenant-AuthSource: TYSPR06MB6433.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jan 2024 02:16:22.6320 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEYPR06MB5255 Subject: [FFmpeg-devel] [PATCH v2 1/2] avcodec/vvcdec: deblock, fix uninitialized values X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nuo Mi , James Almer Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 4Tk7qbaTvxuC see https://fate.ffmpeg.org/report.cgi?slot=x86_64-archlinux-gcc-valgrind&time=20240105201935 If tc is zero, the max_len_q, max_len_p are uninitialized. Reported-by: James Almer --- libavcodec/vvc/vvc_filter_template.c | 236 ++++++++++++++------------- 1 file changed, 119 insertions(+), 117 deletions(-) diff --git a/libavcodec/vvc/vvc_filter_template.c b/libavcodec/vvc/vvc_filter_template.c index 671ed7de4e..845c8a4181 100644 --- a/libavcodec/vvc/vvc_filter_template.c +++ b/libavcodec/vvc/vvc_filter_template.c @@ -551,81 +551,80 @@ static void FUNC(vvc_loop_filter_luma)(uint8_t* _pix, ptrdiff_t _xstride, ptrdif const ptrdiff_t ystride = _ystride / sizeof(pixel); for (int i = 0; i < 2; i++) { - pixel* pix = (pixel*)_pix + i * 4 * ystride; - const int dp0 = abs(P2 - 2 * P1 + P0); - const int dq0 = abs(Q2 - 2 * Q1 + Q0); - const int dp3 = abs(TP2 - 2 * TP1 + TP0); - const int dq3 = abs(TQ2 - 2 * TQ1 + TQ0); - const int d0 = dp0 + dq0; - const int d3 = dp3 + dq3; #if BIT_DEPTH < 10 const int tc = (_tc[i] + (1 << (9 - BIT_DEPTH))) >> (10 - BIT_DEPTH); #else const int tc = _tc[i] << (BIT_DEPTH - 10); #endif - const int tc25 = ((tc * 5 + 1) >> 1); - - const int no_p = _no_p[i]; - const int no_q = _no_q[i]; - - int max_len_p = _max_len_p[i]; - int max_len_q = _max_len_q[i]; - - const int large_p = (max_len_p > 3 && !hor_ctu_edge); - const int large_q = max_len_q > 3; - const int beta = _beta[i] << BIT_DEPTH - 8; - - const int beta_3 = beta >> 3; - const int beta_2 = beta >> 2; - - if (!tc) - continue; - - if (large_p || large_q) { - const int dp0l = large_p ? ((dp0 + abs(P5 - 2 * P4 + P3) + 1) >> 1) : dp0; - const int dq0l = large_q ? ((dq0 + abs(Q5 - 2 * Q4 + Q3) + 1) >> 1) : dq0; - const int dp3l = large_p ? ((dp3 + abs(TP5 - 2 * TP4 + TP3) + 1) >> 1) : dp3; - const int dq3l = large_q ? ((dq3 + abs(TQ5 - 2 * TQ4 + TQ3) + 1) >> 1) : dq3; - const int d0l = dp0l + dq0l; - const int d3l = dp3l + dq3l; - const int beta53 = beta * 3 >> 5; - const int beta_4 = beta >> 4; - max_len_p = large_p ? max_len_p : 3; - max_len_q = large_q ? max_len_q : 3; - - if (d0l + d3l < beta) { - const int sp0l = abs(P3 - P0) + (max_len_p == 7 ? abs(P7 - P6 - P5 + P4) : 0); - const int sq0l = abs(Q0 - Q3) + (max_len_q == 7 ? abs(Q4 - Q5 - Q6 + Q7) : 0); - const int sp3l = abs(TP3 - TP0) + (max_len_p == 7 ? abs(TP7 - TP6 - TP5 + TP4) : 0); - const int sq3l = abs(TQ0 - TQ3) + (max_len_q == 7 ? abs(TQ4 - TQ5 - TQ6 + TQ7) : 0); - const int sp0 = large_p ? ((sp0l + abs(P3 - P(max_len_p)) + 1) >> 1) : sp0l; - const int sp3 = large_p ? ((sp3l + abs(TP3 - TP(max_len_p)) + 1) >> 1) : sp3l; - const int sq0 = large_q ? ((sq0l + abs(Q3 - Q(max_len_q)) + 1) >> 1) : sq0l; - const int sq3 = large_q ? ((sq3l + abs(TQ3 - TQ(max_len_q)) + 1) >> 1) : sq3l; - if (sp0 + sq0 < beta53 && abs(P0 - Q0) < tc25 && - sp3 + sq3 < beta53 && abs(TP0 - TQ0) < tc25 && - (d0l << 1) < beta_4 && (d3l << 1) < beta_4) { - FUNC(loop_filter_luma_large)(pix, xstride, ystride, tc, no_p, no_q, max_len_p, max_len_q); - continue; + if (tc) { + pixel* pix = (pixel*)_pix + i * 4 * ystride; + const int dp0 = abs(P2 - 2 * P1 + P0); + const int dq0 = abs(Q2 - 2 * Q1 + Q0); + const int dp3 = abs(TP2 - 2 * TP1 + TP0); + const int dq3 = abs(TQ2 - 2 * TQ1 + TQ0); + const int d0 = dp0 + dq0; + const int d3 = dp3 + dq3; + const int tc25 = ((tc * 5 + 1) >> 1); + + const int no_p = _no_p[i]; + const int no_q = _no_q[i]; + + int max_len_p = _max_len_p[i]; + int max_len_q = _max_len_q[i]; + + const int large_p = (max_len_p > 3 && !hor_ctu_edge); + const int large_q = max_len_q > 3; + + const int beta = _beta[i] << BIT_DEPTH - 8; + const int beta_3 = beta >> 3; + const int beta_2 = beta >> 2; + + if (large_p || large_q) { + const int dp0l = large_p ? ((dp0 + abs(P5 - 2 * P4 + P3) + 1) >> 1) : dp0; + const int dq0l = large_q ? ((dq0 + abs(Q5 - 2 * Q4 + Q3) + 1) >> 1) : dq0; + const int dp3l = large_p ? ((dp3 + abs(TP5 - 2 * TP4 + TP3) + 1) >> 1) : dp3; + const int dq3l = large_q ? ((dq3 + abs(TQ5 - 2 * TQ4 + TQ3) + 1) >> 1) : dq3; + const int d0l = dp0l + dq0l; + const int d3l = dp3l + dq3l; + const int beta53 = beta * 3 >> 5; + const int beta_4 = beta >> 4; + max_len_p = large_p ? max_len_p : 3; + max_len_q = large_q ? max_len_q : 3; + + if (d0l + d3l < beta) { + const int sp0l = abs(P3 - P0) + (max_len_p == 7 ? abs(P7 - P6 - P5 + P4) : 0); + const int sq0l = abs(Q0 - Q3) + (max_len_q == 7 ? abs(Q4 - Q5 - Q6 + Q7) : 0); + const int sp3l = abs(TP3 - TP0) + (max_len_p == 7 ? abs(TP7 - TP6 - TP5 + TP4) : 0); + const int sq3l = abs(TQ0 - TQ3) + (max_len_q == 7 ? abs(TQ4 - TQ5 - TQ6 + TQ7) : 0); + const int sp0 = large_p ? ((sp0l + abs(P3 - P(max_len_p)) + 1) >> 1) : sp0l; + const int sp3 = large_p ? ((sp3l + abs(TP3 - TP(max_len_p)) + 1) >> 1) : sp3l; + const int sq0 = large_q ? ((sq0l + abs(Q3 - Q(max_len_q)) + 1) >> 1) : sq0l; + const int sq3 = large_q ? ((sq3l + abs(TQ3 - TQ(max_len_q)) + 1) >> 1) : sq3l; + if (sp0 + sq0 < beta53 && abs(P0 - Q0) < tc25 && + sp3 + sq3 < beta53 && abs(TP0 - TQ0) < tc25 && + (d0l << 1) < beta_4 && (d3l << 1) < beta_4) { + FUNC(loop_filter_luma_large)(pix, xstride, ystride, tc, no_p, no_q, max_len_p, max_len_q); + continue; + } } } - } - if (d0 + d3 < beta) { - if (max_len_p > 2 && max_len_q > 2 && - abs(P3 - P0) + abs(Q3 - Q0) < beta_3 && abs(P0 - Q0) < tc25 && - abs(TP3 - TP0) + abs(TQ3 - TQ0) < beta_3 && abs(TP0 - TQ0) < tc25 && - (d0 << 1) < beta_2 && (d3 << 1) < beta_2) { - FUNC(loop_filter_luma_strong)(pix, xstride, ystride, tc, tc << 1, tc * 3, no_p, no_q); - } else { // weak filtering - int nd_p = 1; - int nd_q = 1; - if (max_len_p > 1 && max_len_q > 1) { - if (dp0 + dp3 < ((beta + (beta >> 1)) >> 3)) - nd_p = 2; - if (dq0 + dq3 < ((beta + (beta >> 1)) >> 3)) - nd_q = 2; + if (d0 + d3 < beta) { + if (max_len_p > 2 && max_len_q > 2 && + abs(P3 - P0) + abs(Q3 - Q0) < beta_3 && abs(P0 - Q0) < tc25 && + abs(TP3 - TP0) + abs(TQ3 - TQ0) < beta_3 && abs(TP0 - TQ0) < tc25 && + (d0 << 1) < beta_2 && (d3 << 1) < beta_2) { + FUNC(loop_filter_luma_strong)(pix, xstride, ystride, tc, tc << 1, tc * 3, no_p, no_q); + } else { + int nd_p = 1; + int nd_q = 1; + if (max_len_p > 1 && max_len_q > 1) { + if (dp0 + dp3 < ((beta + (beta >> 1)) >> 3)) + nd_p = 2; + if (dq0 + dq3 < ((beta + (beta >> 1)) >> 3)) + nd_q = 2; + } + FUNC(loop_filter_luma_weak)(pix, xstride, ystride, tc, beta, no_p, no_q, nd_p, nd_q); } - FUNC(loop_filter_luma_weak)(pix, xstride, ystride, tc, beta, no_p, no_q, nd_p, nd_q); } } } @@ -689,65 +688,68 @@ static void FUNC(vvc_loop_filter_chroma)(uint8_t *_pix, const ptrdiff_t _xstrid const int end = 8 / size; // 8 samples a loop for (int i = 0; i < end; i++) { - pixel *pix = (pixel *)_pix + i * size * ystride; - const uint8_t no_p = _no_p[i]; - const uint8_t no_q = _no_q[i]; - const int beta = _beta[i] << (BIT_DEPTH - 8); - const int beta_3 = beta >> 3; - const int beta_2 = beta >> 2; - #if BIT_DEPTH < 10 const int tc = (_tc[i] + (1 << (9 - BIT_DEPTH))) >> (10 - BIT_DEPTH); #else const int tc = _tc[i] << (BIT_DEPTH - 10); #endif - const int tc25 = ((tc * 5 + 1) >> 1); - - uint8_t max_len_p = _max_len_p[i]; - uint8_t max_len_q = _max_len_q[i]; - - if (!max_len_p || !max_len_q || !tc) - continue; - - if (max_len_q == 3){ - const int p1n = shift ? FP1 : TP1; - const int p2n = max_len_p == 1 ? p1n : (shift ? FP2 : TP2); - const int p0n = shift ? FP0 : TP0; - const int q0n = shift ? FQ0 : TQ0; - const int q1n = shift ? FQ1 : TQ1; - const int q2n = shift ? FQ2 : TQ2; - const int p3 = max_len_p == 1 ? P1 : P3; - const int p2 = max_len_p == 1 ? P1 : P2; - const int p1 = P1; - const int p0 = P0; - const int dp0 = abs(p2 - 2 * p1 + p0); - const int dq0 = abs(Q2 - 2 * Q1 + Q0); + if (tc) { + pixel *pix = (pixel *)_pix + i * size * ystride; + const uint8_t no_p = _no_p[i]; + const uint8_t no_q = _no_q[i]; + + const int beta = _beta[i] << (BIT_DEPTH - 8); + const int beta_3 = beta >> 3; + const int beta_2 = beta >> 2; + + const int tc25 = ((tc * 5 + 1) >> 1); + + uint8_t max_len_p = _max_len_p[i]; + uint8_t max_len_q = _max_len_q[i]; - const int dp1 = abs(p2n - 2 * p1n + p0n); - const int dq1 = abs(q2n - 2 * q1n + q0n); - const int d0 = dp0 + dq0; - const int d1 = dp1 + dq1; - - if (d0 + d1 < beta) { - const int p3n = max_len_p == 1 ? p1n : (shift ? FP3 : TP3); - const int q3n = shift ? FQ3 : TQ3; - const int dsam0 = (d0 << 1) < beta_2 && (abs(p3 - p0) + abs(Q0 - Q3) < beta_3) && - abs(p0 - Q0) < tc25; - const int dsam1 = (d1 << 1) < beta_2 && (abs(p3n - p0n) + abs(q0n - q3n) < beta_3) && - abs(p0n - q0n) < tc25; - if (!dsam0 || !dsam1) + if (!max_len_p || !max_len_q) + continue; + + if (max_len_q == 3){ + const int p1n = shift ? FP1 : TP1; + const int p2n = max_len_p == 1 ? p1n : (shift ? FP2 : TP2); + const int p0n = shift ? FP0 : TP0; + const int q0n = shift ? FQ0 : TQ0; + const int q1n = shift ? FQ1 : TQ1; + const int q2n = shift ? FQ2 : TQ2; + const int p3 = max_len_p == 1 ? P1 : P3; + const int p2 = max_len_p == 1 ? P1 : P2; + const int p1 = P1; + const int p0 = P0; + const int dp0 = abs(p2 - 2 * p1 + p0); + const int dq0 = abs(Q2 - 2 * Q1 + Q0); + + const int dp1 = abs(p2n - 2 * p1n + p0n); + const int dq1 = abs(q2n - 2 * q1n + q0n); + const int d0 = dp0 + dq0; + const int d1 = dp1 + dq1; + + if (d0 + d1 < beta) { + const int p3n = max_len_p == 1 ? p1n : (shift ? FP3 : TP3); + const int q3n = shift ? FQ3 : TQ3; + const int dsam0 = (d0 << 1) < beta_2 && (abs(p3 - p0) + abs(Q0 - Q3) < beta_3) && + abs(p0 - Q0) < tc25; + const int dsam1 = (d1 << 1) < beta_2 && (abs(p3n - p0n) + abs(q0n - q3n) < beta_3) && + abs(p0n - q0n) < tc25; + if (!dsam0 || !dsam1) + max_len_p = max_len_q = 1; + } else { max_len_p = max_len_q = 1; - } else { - max_len_p = max_len_q = 1; + } } - } - if (max_len_p == 3 && max_len_q == 3) - FUNC(loop_filter_chroma_strong)(pix, xstride, ystride, size, tc, no_p, no_q); - else if (max_len_q == 3) - FUNC(loop_filter_chroma_strong_one_side)(pix, xstride, ystride, size, tc, no_p, no_q); - else - FUNC(loop_filter_chroma_weak)(pix, xstride, ystride, size, tc, no_p, no_q); + if (max_len_p == 3 && max_len_q == 3) + FUNC(loop_filter_chroma_strong)(pix, xstride, ystride, size, tc, no_p, no_q); + else if (max_len_q == 3) + FUNC(loop_filter_chroma_strong_one_side)(pix, xstride, ystride, size, tc, no_p, no_q); + else + FUNC(loop_filter_chroma_weak)(pix, xstride, ystride, size, tc, no_p, no_q); + } } } From patchwork Wed Jan 17 02:15:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nuo Mi X-Patchwork-Id: 45620 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c58a:b0:199:de12:6fa6 with SMTP id gn10csp87151pzb; Tue, 16 Jan 2024 18:16:49 -0800 (PST) X-Google-Smtp-Source: AGHT+IHyU/blevl1iyfdyiu/WWHX2FRvdeFbCogCksZaW7daYiUr91k8BVGYz3BAevI9DlJoHf4D X-Received: by 2002:a17:906:4355:b0:a2b:96ab:3c2f with SMTP id z21-20020a170906435500b00a2b96ab3c2fmr3826013ejm.93.1705457809147; Tue, 16 Jan 2024 18:16:49 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id t21-20020a170906065500b00a28b34d8d6asi5304864ejb.218.2024.01.16.18.16.48; Tue, 16 Jan 2024 18:16:49 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=oQWV3QQt; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EEB2968CBC3; Wed, 17 Jan 2024 04:16:44 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from APC01-SG2-obe.outbound.protection.outlook.com (mail-sgaapc01olkn2100.outbound.protection.outlook.com [40.92.53.100]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0EEE768CBC3 for ; Wed, 17 Jan 2024 04:16:38 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LuQ90/X5fgws1IMMksliksu7LXClhnUMEsd61V42743B5Ti1eT3MtbzHAa7F03pETaveCTEBHgTQvr0GZamMEj7Wr0eVtmuvqpPoKQMtd6PJY+6b7GWVzttAXME6DgJka3f9jY11INzDvtLS82uoEXPo2mVTI/AaPZYrBgUcOh4nfc7ePcFFjMiQlu1mZkvkk+iFS4oT+LCnFEz+OA9DvmxuOljV/LEE1xdzHqtxvkf1MJFl8UwGLTHBbVsFQ6vEOsTlS4MzraBwVNaIg51Mm8iQNjyq8J+aG2jr1aCXjBu3C6HL9h/IAoPDTf4PIdf3TJvzcSLCJi6mpJTlgtVUJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4InPavYaLl0n1Q3EgNXkVxAfmDwB+dRW97QnvzpGFwg=; b=m6PgcHO5n743wc8tzrIKVrRsWs6Yf1RfZ/zxw4jl3o1iaSGfEnSfLhVJ6F/MfR8haVhEq/H6fsIjDzSxvL87WALa9yLGMpK1TPmV1r5YJQ+f5ryShTFORFQxLLk/udS3tIyoPzqvurnLUuA5WJV2mAubhYpkx1f5CHKkrRHptCCiQUnvqBkdr4UlcdtumkE3Ph/sN4gYXEg2M1JT2wWphJ1aI636KKzD2i/3wq3iPhVF1D5RZYgDwX/wH7etBSrpTkp5kMZmzWkgO8XW0jPtO+tivecJh0RcVJKmL/FYlMksuepa/yf97T7i9vaKQSiviNxvQYh9yV6tth2uIAsPxg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4InPavYaLl0n1Q3EgNXkVxAfmDwB+dRW97QnvzpGFwg=; b=oQWV3QQtJTffJ5YMUA9T0zYQeG9GIGRrSaORk5rWwYtkCTWRllE4n/tbjN3e7oB64RLVxMaE4gklhXbfWY7neo6WF2Yp60m9UzBsfr6FyEtv+Ot6oi2ECD4hdSY7DDjJeiaxhqwM1K6bnQOucam5MToRqcLYHd2ZSaxRggYG6Y87CPPalsPAFXo5u7z7TQ5fcYqDoldn3W5fJa1SuXc7NNdq0pfrHEiUf8acdGtF/h+/YEulrqP0SZ7AeHu9Ps5rtSlQInED9g1JXMaN0TpbRiOcnPFlFRSeN7CW8J0P9AeiuAMTEim3qoKiIMWI39wg1D24a4wWrbFECbtS6yrPVQ== Received: from TYSPR06MB6433.apcprd06.prod.outlook.com (2603:1096:400:47a::6) by SEYPR06MB5255.apcprd06.prod.outlook.com (2603:1096:101:84::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7181.26; Wed, 17 Jan 2024 02:16:23 +0000 Received: from TYSPR06MB6433.apcprd06.prod.outlook.com ([fe80::e139:fd7a:1b66:a3f7]) by TYSPR06MB6433.apcprd06.prod.outlook.com ([fe80::e139:fd7a:1b66:a3f7%7]) with mapi id 15.20.7202.020; Wed, 17 Jan 2024 02:16:23 +0000 From: Nuo Mi To: ffmpeg-devel@ffmpeg.org Date: Wed, 17 Jan 2024 10:15:33 +0800 Message-ID: X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240117021533.2965-1-nuomi2021@gmail.com> References: <20240117021533.2965-1-nuomi2021@gmail.com> X-TMN: [yCrHn5QGmj5+NbNNQLd+9/RkD6C5H+U+] X-ClientProxiedBy: TYCP286CA0261.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:455::12) To TYSPR06MB6433.apcprd06.prod.outlook.com (2603:1096:400:47a::6) X-Microsoft-Original-Message-ID: <20240117021533.2965-3-nuomi2021@gmail.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 2 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TYSPR06MB6433:EE_|SEYPR06MB5255:EE_ X-MS-Office365-Filtering-Correlation-Id: 2ec7c738-edd3-4d94-4a50-08dc17024d20 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JondVTtvrUwQrRYc+c8eWQHaF86+LPlapcmDirgl5e8Y7SDz4t6B7hWcnRejPlatu414/T9aFOhlzYvJK+dVZ7gPi3Me8IdTIwFWIpwOQ389aENno4mPdin6iZsVU26N6QaP+jRiqfphtsskzZzA12W9D9jUYRHlQYGy8zGeiD2QQXzu67E8St4jXiHyf922WNBEskZx8uL/irtEoQ4kmHkmT5YPD7AynYSndZRyVWdHJx/nVxzF3b3I/xWb3GmlUctrvX/tLODyEn1YRmdwKREOOhEh/wZoTlaheU9FJHmbutmcnwT71ZvlNHzGPSAx84ugIq7OzG1skEiso8mEM3WxAF67I74WYBV/oKmIu9mwFoiktjnesLvjGm7niK66j9IVVDAtZc9wOKBI8FSX3bQktqzPTV1vpZbZF9TWmxt399e8zlLjfR9fudtJABH6BWD9y10+gO8PJ48XZ7fdoIIdTNe+xXDxTar+QCWtc0OVmeza22/9tH1VTv+vefcwu650XMunXM9iiEixrJUGWRTyi86pp7OOJbhfwSOB1SX3gU1jssWs3MpjNU8Yvb25gU40+fELI1pRXiEQKviMImk+BvEKxlWY53uLtGhrL6NG36gHnnm5UVfdY8oEzMhm X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: cCuAU+P1kzjMd5KzCflWZ97/3LKbG25YJONG4dWtEcf9vz2X1yMUGiPZoaE7n9YgqFP6jvvsNYMDJmtDzcZjx1lzmYSaGiXFDC8PUTiggayxMECnjJGa9zk2gIpWibYpjK0L3uS7GHzihbXCgVr8QUVP6DBDuFADixmeMCc/XuQt3InxW3l8en9mnw5HXP/+i477y0Q+ddXklfPLnHv1LERwNiMaTU0zIVz3NHEtXXUFPJ0t7YxvYK8ETbML+Cxoe5L+CZiyUrrTSECPhJd0S9RaU3Pnqplo2Wu17RwRkefzbHF6hISoT0Lsy70hzEqWsnwADBp4e+K08OCbS18LZs8nWaUm1x1OQ3Yn8Jhlt7pz19wYn6DkRMFj3Zq5Iw6iErCD/PfCww5pvccjr2/kL9jA9fQVa1hIEKC8TUpngB1SPlrpUVwTjIfIAa7jRW2zrJ3ZncUEHzSNR5VHFc768McYCcA5K7sjH0htQrzNpQ9WHVUjhqclxMfAxbhKAJXB4uLjJGC9NGQm9EKOCR/zOHsPPUFswhPAbVAvWiRnixYC5xB92AW435UT/Lo3A0AYuZo9XorJlV9pAUcK6bhDvAwHBJrVstcyVF6DAOIZ66zW0JOQJWs609V2w5PPlTEWNBo7nt2GrAojJx0tGktF2xazwtfwvhQIsW12vwuaEp6iZE04d+aEhXOiF293HceO826C3OFYQFZcsG6tx5uIU2ZmTzdMyNipOik9mffM0n2i8KPBM1C3B3IPOSLMtEoIoN16NyGZBPJf+PUfDISwpqB67JSItPVVFNR3Pj6XCdWqzDx0KWQh49UU2JbRP/Vh7FL0yzBQfQfpfPm/zqWBBD+6vpUPHG8BhKk61W37y15jUIehe7WLzskGCTg9hlyNSwLmegi0Aswbef4FUjg9l74c9b/cKCoaqRS7Sv3WHykHEWnV6ivEOO3HdMFLNiuNEkL4AC9aMrBjjdzfQ+1H0tFbs8g2WUnvMgY4kbHyND5Ccgz355S3PrJBjssnPf/0EqlEQhg/juYTEqK4ndEnqqg2pKHuVmCxPkdwizLC7ElK5+8dXleI8KIr5533nNQ4dQXYYVwg9axjXxbXNts82fVwhoOotxUYm0BtIZkAAgjeJsPUaKg5JDMd5tLJs2jS3l3ySHkcsKdHZ15dEqSkfnanMlxD6aPXY8fjk44GCNSiMvvoudttmUDXsZTTidSfuYbydvevkyGxL6VSsMnN7VcEzkhPXc5N93KfNJ8dPpm5jScBuDqSrf0/xjDMcP+P X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2ec7c738-edd3-4d94-4a50-08dc17024d20 X-MS-Exchange-CrossTenant-AuthSource: TYSPR06MB6433.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jan 2024 02:16:23.4674 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEYPR06MB5255 Subject: [FFmpeg-devel] [PATCH v2 2/2] avcodec/vvcdec: misc, constify hor_ctu_edge X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nuo Mi Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: VQVUNGRoXgef --- libavcodec/vvc/vvc_filter_template.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/libavcodec/vvc/vvc_filter_template.c b/libavcodec/vvc/vvc_filter_template.c index 845c8a4181..b7eaef5125 100644 --- a/libavcodec/vvc/vvc_filter_template.c +++ b/libavcodec/vvc/vvc_filter_template.c @@ -545,7 +545,7 @@ static void FUNC(loop_filter_luma_large)(pixel *pix, const ptrdiff_t xstride, co static void FUNC(vvc_loop_filter_luma)(uint8_t* _pix, ptrdiff_t _xstride, ptrdiff_t _ystride, const int32_t *_beta, const int32_t *_tc, const uint8_t *_no_p, const uint8_t *_no_q, - const uint8_t *_max_len_p, const uint8_t *_max_len_q, int hor_ctu_edge) + const uint8_t *_max_len_p, const uint8_t *_max_len_q, const int hor_ctu_edge) { const ptrdiff_t xstride = _xstride / sizeof(pixel); const ptrdiff_t ystride = _ystride / sizeof(pixel); @@ -771,7 +771,7 @@ static void FUNC(vvc_v_loop_filter_chroma)(uint8_t *pix, ptrdiff_t stride, static void FUNC(vvc_h_loop_filter_luma)(uint8_t *pix, ptrdiff_t stride, const int32_t *beta, const int32_t *tc, const uint8_t *no_p, const uint8_t *no_q, - const uint8_t *max_len_p, const uint8_t *max_len_q, int hor_ctu_edge) + const uint8_t *max_len_p, const uint8_t *max_len_q, const int hor_ctu_edge) { FUNC(vvc_loop_filter_luma)(pix, stride, sizeof(pixel), beta, tc, no_p, no_q, max_len_p, max_len_q, hor_ctu_edge); @@ -779,7 +779,7 @@ static void FUNC(vvc_h_loop_filter_luma)(uint8_t *pix, ptrdiff_t stride, static void FUNC(vvc_v_loop_filter_luma)(uint8_t *pix, ptrdiff_t stride, const int32_t *beta, const int32_t *tc, const uint8_t *no_p, const uint8_t *no_q, - const uint8_t *max_len_p, const uint8_t *max_len_q, int hor_ctu_edge) + const uint8_t *max_len_p, const uint8_t *max_len_q, const int hor_ctu_edge) { FUNC(vvc_loop_filter_luma)(pix, sizeof(pixel), stride, beta, tc, no_p, no_q, max_len_p, max_len_q, hor_ctu_edge);