From patchwork Sun Jan 7 04:42:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nuo Mi X-Patchwork-Id: 45515 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:2449:b0:199:85ec:e0df with SMTP id t9csp350427pzc; Sat, 6 Jan 2024 20:44:12 -0800 (PST) X-Google-Smtp-Source: AGHT+IFl4WLzqPyzFv13Y7AvgpcNctEjDPmnCN6JVAvOJ+FlnonLH1U9Ze3CH+3fcan5Qocki2SK X-Received: by 2002:a19:430c:0:b0:50e:6784:8614 with SMTP id q12-20020a19430c000000b0050e67848614mr260775lfa.101.1704602651735; Sat, 06 Jan 2024 20:44:11 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id t10-20020a50ab4a000000b0055411420e18si2091997edc.232.2024.01.06.20.44.11; Sat, 06 Jan 2024 20:44:11 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=N2f3WGFx; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E232268C4DD; Sun, 7 Jan 2024 06:44:07 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from APC01-SG2-obe.outbound.protection.outlook.com (mail-sgaapc01olkn2038.outbound.protection.outlook.com [40.92.53.38]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E99B768C8F6 for ; Sun, 7 Jan 2024 06:43:57 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Xbt+bDrYRsBZ3yYMFVs1sKlIm18oPaayhWhqWQJK4W7c8LtnWMVOFXfaezkFMz78XjmXNfXf9oYoAxA66Mnc1BC0ddYEKMOUv0gZthb2dlnRK6Bu2TPiwrHpIe8UR4uNX6M3D+zeJv0++ayeGK4HCZfu8CpNQdCahvU8p7mzTJ51hIgNKxnAZMOXCDnMajKsFXHXQtQbOZm2gjjSSxfrgzWQ6v8CZ9Dh/wUWek6GZhcY0qG0cNF406+NaOo3Sx+NwgCCZ1x9J12FyOIm0C1TdL9zF3s9u3k2PsTn32AMzN4mlstgY2d0Ash1ZLGrbwnXzklPivQy61E2gczLCjZRvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=65f0/mHUm43RXVbbczXxrZQahg8UdBGoZLBJ/K40874=; b=WJgg0xm6KLCT6sk8UElPUB4LTS2iKcJBvMhAMdFXbmAzMUFz2yINp6e//BNO4SB9b1Su9XZmDvWMuhGslF1NEAQTIFziYJwnSI+2M7WDu7uQTYLNkTEejkqO1KfGbUNHFF+qMdvai2+aIerF5XWvmSongjY2G8eEm5Ne6qmv/JR6fHwvAE5hK7q0EVTL5MW6t5g6gCM7tMbdo6r9n2lr56hDa+UEiUEHpIa3UXP0HibYrfVFPjUBVztSFLSVyqHEeVlsGO1ivmj1I1E5JAb+XaAMs92f+YnQcmFzFoBaO1GxjRWfIivQ0DbEH/hOdWJ20z/QyD1L4+JKuAJOOM7lEg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=65f0/mHUm43RXVbbczXxrZQahg8UdBGoZLBJ/K40874=; b=N2f3WGFxdp16HVFEX6TSVLO6VQZMOxUYyKvwDffN0LyBqaeJs8FZ40hghaPOxxxTEbdojx19Z8sGuthmytUUpicwgd5jj98zkBZWjYeTJk3ZfJqltF1BVQ+f7mI6CUQQVBG52T/RLECOf/q3q0MCL73kTV99yGpCbT0ea2EoqXv0EJhbgQVvwv53bByUIxZYKIkTC18Ut/0K+ChdwfAKnwtRkTqJE4kUCO+z3vgvMdmFY9d4xe44dDAqDO0Xx05DZdp29U4Z09PW+VXMhu/POb4u3DvnDpqLJv0vJ5nE2NvwcxZSMmnm98YHj/umBXOhiri3nGWb3XkHr89XfdcWuA== Received: from TYSPR06MB6433.apcprd06.prod.outlook.com (2603:1096:400:47a::6) by SI2PR06MB5268.apcprd06.prod.outlook.com (2603:1096:4:1e7::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7159.13; Sun, 7 Jan 2024 04:43:52 +0000 Received: from TYSPR06MB6433.apcprd06.prod.outlook.com ([fe80::e139:fd7a:1b66:a3f7]) by TYSPR06MB6433.apcprd06.prod.outlook.com ([fe80::e139:fd7a:1b66:a3f7%7]) with mapi id 15.20.7159.018; Sun, 7 Jan 2024 04:43:52 +0000 From: Nuo Mi To: ffmpeg-devel@ffmpeg.org Date: Sun, 7 Jan 2024 12:42:57 +0800 Message-ID: X-Mailer: git-send-email 2.25.1 X-TMN: [tXW4WQ+QopHu7GN9T1xkw9hWFJaKiGhO] X-ClientProxiedBy: TY2PR0101CA0036.apcprd01.prod.exchangelabs.com (2603:1096:404:8000::22) To TYSPR06MB6433.apcprd06.prod.outlook.com (2603:1096:400:47a::6) X-Microsoft-Original-Message-ID: <20240107044258.3926-1-nuomi2021@gmail.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 2 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TYSPR06MB6433:EE_|SI2PR06MB5268:EE_ X-MS-Office365-Filtering-Correlation-Id: 1f616f9a-ff48-4c1d-5d46-08dc0f3b3f4e X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: S8Hio8cWEkQbvBSi6BaPDCP79dM0a4yQeY0wdy8akoQZox0Lfupkk5IeMU9Owg+VEujJ6IwifYxcmIQL780xOGb+Kps5xVbfa9Oq50MPGdWqRDJ4jrfAxY9yxdV5CeDBkBReCOvGoM6pmZemcRbHFpmx6TLV21DhFnlEgNJeh5vJYiE78QU1mpF+CeYxmrVqab7FfzBkpK7kUHcFbV87dbCx0XA8b1gz3ihOHi0zRwEA4LHRR61RFBDmySQhMswfQ/LivBHM83tncbGNZgaLvWGQYDJuASnekrqJ0+/7zWQ0iij1HTxdo0TEENZDCVqPMKcdr7A/33VpUVKgZt9NMRvB7+oSkBEvOtHIDsItKUq5DSc00NpZJtW0WrEZGWmFV9AaPenav5Q3NniFhuA7obbPuBmrki9KnF8aboFbtlhAl1jgrwYCelfm9ikAxVVz5nvPs+SpiD1MtD9fKbWchJq3ldb9pWUMC43R1jsQVdfewtK/SeQVmUXW6uiTswDjW9rl7vVRMCvcEzvbx2pRfXuekLkSLYIGgRtwSG+5f1pH77ZsaQXUVPUA4WkEvb3JYj3MYIH0b0uI+q942M5iTHxluKzRyl15pHiOoOT43RSqZesopg8UL3MEgQwYc+mX X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Co0d9/F+RFtWVOmXfPYWE8RsFpnAM1/CfezDfm3vlbkj5D413H01ISCwDg6gtQfQS/UDNKHgo3VmpuAcUGwQkH08V5EkJr3zjgZFC9Xgws89Mv1pham/o2bEKgE9egETgyLne91GzBWIWwQmgBq9PrbsCTRR4ACFtPOmkdWf1zLbfKL4ejxGGgRLtbJFHhOu0MC09T2yDh98zkUBwikQoPGjT4LD0ZUMcCCtkq/LgigoMCZnjUJqb7BpDjR7f2Db9QWkTiUGlzQjvpCw19WzBu+UBGkgJtlBkLkkBs6xR4wE3NWKrGiYzReMluF7xhCGBgGDoaiGNpvTyYsWOMyGAc4qz97i8VkDB2YL7G8KHahB/NTCi1XtYQ/hb+l43VwGIh8oIlcj/FV/RlNBYuuoWlUdqddj3GXM9gGK58A675MzbqDj9NNlOmtBYlsmYQr8F64G7iDdkyLYbF1BTd8dNdsAV9lDNlQUITcG/6YSYUMHdBu2kw3EVelg1oO4kJBpEtysy4KyKo0ubxx3X3nvVCrjvkH3Og+rMNL2Aa9KLjmNg6obX+s6XghTEO7T5gnnWKmR5nbJlpWvOHWuzd5kdUFHL1QLv3imQEpcBqKeVk4uuaY0s1zzCKIro7ahG3SttVkbsy6vAVa10iggErP308klii/+sWHsvFU9rMLee4cYKPq90+ZoHuPB1rjZNfyLREqLjmSQ2v8Mn9KiCh53+y2sYbSqay3n9EBvWquq9hmRySy+Th0KVIpx0Z7/PgiXCXH1S7aYRVSH2z4G2Kl+qUjwNIzWxloVt8nnNbb5CWeG6ZDty2c8CyTOehaO+7DjQ1jgM9x7qgMfmTyTSIVrNshY06qZnUox64O43FhCIB547ml6gj0NOrQbh4l23tvgSx1PzGm2GnN+vco2eflwJLe7sqmxNbe07rpRCSVVE0NXdeSBOmNzZd6cwIkfTZPXMqwdEg5ahybgFNwQ51WFQ9UneUOMrMuKa1jHtrSdYWXclqWesOaxaGpNU/822aA01ZLn/DqPvxxXVkwuhES/AL0XkmPHPZrK+xq8aazJNi61KI/xADTn0D93Pu6SBu3wrJ9WDbojFs1FkVIgqDRD8GzX8PYxGWtmgcQy5hzb37m3PYEI+sye9pSPSx0pnSaIQcgoZFz9e/x8JSAC62AGHU+uBQKCWszvZyqqHQUDoXa1uSJ1lMMLZl8Q8wWM2/+zZ/25o12I4UBaTaHw0CEhXcVnHGVM6Jipstled5l3oOjO4t6Bb6yrzLB28Gu/H4OB X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1f616f9a-ff48-4c1d-5d46-08dc0f3b3f4e X-MS-Exchange-CrossTenant-AuthSource: TYSPR06MB6433.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jan 2024 04:43:52.2792 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SI2PR06MB5268 Subject: [FFmpeg-devel] [PATCH 1/2] avcodec/vvcdec: deblock, fix uninitialized values X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nuo Mi , James Almer Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 8WEin+r5+/cl see https://fate.ffmpeg.org/report.cgi?slot=x86_64-archlinux-gcc-valgrind&time=20240105201935 If tc is zero, the max_len_q, max_len_p are uninitialized. Reported-by: James Almer --- libavcodec/vvc/vvc_filter_template.c | 231 +++++++++++++-------------- 1 file changed, 114 insertions(+), 117 deletions(-) diff --git a/libavcodec/vvc/vvc_filter_template.c b/libavcodec/vvc/vvc_filter_template.c index a4f1792ec4..e30deaac96 100644 --- a/libavcodec/vvc/vvc_filter_template.c +++ b/libavcodec/vvc/vvc_filter_template.c @@ -798,81 +798,78 @@ static void FUNC(vvc_loop_filter_luma)(uint8_t* _pix, ptrdiff_t _xstride, ptrdif const ptrdiff_t ystride = _ystride / sizeof(pixel); for (int i = 0; i < 2; i++) { - pixel* pix = (pixel*)_pix + i * 4 * ystride; - const int dp0 = abs(P2 - 2 * P1 + P0); - const int dq0 = abs(Q2 - 2 * Q1 + Q0); - const int dp3 = abs(TP2 - 2 * TP1 + TP0); - const int dq3 = abs(TQ2 - 2 * TQ1 + TQ0); - const int d0 = dp0 + dq0; - const int d3 = dp3 + dq3; #if BIT_DEPTH < 10 const int tc = (_tc[i] + (1 << (9 - BIT_DEPTH))) >> (10 - BIT_DEPTH); #else const int tc = _tc[i] << (BIT_DEPTH - 10); #endif - const int tc25 = ((tc * 5 + 1) >> 1); - - const int no_p = _no_p[i]; - const int no_q = _no_q[i]; - - int max_len_p = _max_len_p[i]; - int max_len_q = _max_len_q[i]; - - const int large_p = (max_len_p > 3 && !hor_ctu_edge); - const int large_q = max_len_q > 3; - const int beta = _beta[i] << BIT_DEPTH - 8; - - const int beta_3 = beta >> 3; - const int beta_2 = beta >> 2; - - if (!tc) - continue; - - if (large_p || large_q) { - const int dp0l = large_p ? ((dp0 + abs(P5 - 2 * P4 + P3) + 1) >> 1) : dp0; - const int dq0l = large_q ? ((dq0 + abs(Q5 - 2 * Q4 + Q3) + 1) >> 1) : dq0; - const int dp3l = large_p ? ((dp3 + abs(TP5 - 2 * TP4 + TP3) + 1) >> 1) : dp3; - const int dq3l = large_q ? ((dq3 + abs(TQ5 - 2 * TQ4 + TQ3) + 1) >> 1) : dq3; - const int d0l = dp0l + dq0l; - const int d3l = dp3l + dq3l; - const int beta53 = beta * 3 >> 5; - const int beta_4 = beta >> 4; - max_len_p = large_p ? max_len_p : 3; - max_len_q = large_q ? max_len_q : 3; - - if (d0l + d3l < beta) { - const int sp0l = abs(P3 - P0) + (max_len_p == 7 ? abs(P7 - P6 - P5 + P4) : 0); - const int sq0l = abs(Q0 - Q3) + (max_len_q == 7 ? abs(Q4 - Q5 - Q6 + Q7) : 0); - const int sp3l = abs(TP3 - TP0) + (max_len_p == 7 ? abs(TP7 - TP6 - TP5 + TP4) : 0); - const int sq3l = abs(TQ0 - TQ3) + (max_len_q == 7 ? abs(TQ4 - TQ5 - TQ6 + TQ7) : 0); - const int sp0 = large_p ? ((sp0l + abs(P3 - P(max_len_p)) + 1) >> 1) : sp0l; - const int sp3 = large_p ? ((sp3l + abs(TP3 - TP(max_len_p)) + 1) >> 1) : sp3l; - const int sq0 = large_q ? ((sq0l + abs(Q3 - Q(max_len_q)) + 1) >> 1) : sq0l; - const int sq3 = large_q ? ((sq3l + abs(TQ3 - TQ(max_len_q)) + 1) >> 1) : sq3l; - if (sp0 + sq0 < beta53 && abs(P0 - Q0) < tc25 && - sp3 + sq3 < beta53 && abs(TP0 - TQ0) < tc25 && - (d0l << 1) < beta_4 && (d3l << 1) < beta_4) { - FUNC(loop_filter_luma_large)(pix, xstride, ystride, tc, no_p, no_q, max_len_p, max_len_q); - continue; + if (tc) { + pixel* pix = (pixel*)_pix + i * 4 * ystride; + const int dp0 = abs(P2 - 2 * P1 + P0); + const int dq0 = abs(Q2 - 2 * Q1 + Q0); + const int dp3 = abs(TP2 - 2 * TP1 + TP0); + const int dq3 = abs(TQ2 - 2 * TQ1 + TQ0); + const int d0 = dp0 + dq0; + const int d3 = dp3 + dq3; + const int tc25 = ((tc * 5 + 1) >> 1); + const int no_p = _no_p[i]; + const int no_q = _no_q[i]; + + int max_len_p = _max_len_p[i]; + int max_len_q = _max_len_q[i]; + const int large_p = (max_len_p > 3 && !hor_ctu_edge); + const int large_q = max_len_q > 3; + + const int beta = _beta[i] << BIT_DEPTH - 8; + const int beta_3 = beta >> 3; + const int beta_2 = beta >> 2; + + if (large_p || large_q) { + const int dp0l = large_p ? ((dp0 + abs(P5 - 2 * P4 + P3) + 1) >> 1) : dp0; + const int dq0l = large_q ? ((dq0 + abs(Q5 - 2 * Q4 + Q3) + 1) >> 1) : dq0; + const int dp3l = large_p ? ((dp3 + abs(TP5 - 2 * TP4 + TP3) + 1) >> 1) : dp3; + const int dq3l = large_q ? ((dq3 + abs(TQ5 - 2 * TQ4 + TQ3) + 1) >> 1) : dq3; + const int d0l = dp0l + dq0l; + const int d3l = dp3l + dq3l; + const int beta53 = beta * 3 >> 5; + const int beta_4 = beta >> 4; + max_len_p = large_p ? max_len_p : 3; + max_len_q = large_q ? max_len_q : 3; + + if (d0l + d3l < beta) { + const int sp0l = abs(P3 - P0) + (max_len_p == 7 ? abs(P7 - P6 - P5 + P4) : 0); + const int sq0l = abs(Q0 - Q3) + (max_len_q == 7 ? abs(Q4 - Q5 - Q6 + Q7) : 0); + const int sp3l = abs(TP3 - TP0) + (max_len_p == 7 ? abs(TP7 - TP6 - TP5 + TP4) : 0); + const int sq3l = abs(TQ0 - TQ3) + (max_len_q == 7 ? abs(TQ4 - TQ5 - TQ6 + TQ7) : 0); + const int sp0 = large_p ? ((sp0l + abs(P3 - P(max_len_p)) + 1) >> 1) : sp0l; + const int sp3 = large_p ? ((sp3l + abs(TP3 - TP(max_len_p)) + 1) >> 1) : sp3l; + const int sq0 = large_q ? ((sq0l + abs(Q3 - Q(max_len_q)) + 1) >> 1) : sq0l; + const int sq3 = large_q ? ((sq3l + abs(TQ3 - TQ(max_len_q)) + 1) >> 1) : sq3l; + if (sp0 + sq0 < beta53 && abs(P0 - Q0) < tc25 && + sp3 + sq3 < beta53 && abs(TP0 - TQ0) < tc25 && + (d0l << 1) < beta_4 && (d3l << 1) < beta_4) { + FUNC(loop_filter_luma_large)(pix, xstride, ystride, tc, no_p, no_q, max_len_p, max_len_q); + continue; + } } } - } - if (d0 + d3 < beta) { - if (max_len_p > 2 && max_len_q > 2 && - abs(P3 - P0) + abs(Q3 - Q0) < beta_3 && abs(P0 - Q0) < tc25 && - abs(TP3 - TP0) + abs(TQ3 - TQ0) < beta_3 && abs(TP0 - TQ0) < tc25 && - (d0 << 1) < beta_2 && (d3 << 1) < beta_2) { - FUNC(loop_filter_luma_strong)(pix, xstride, ystride, tc, no_p, no_q); - } else { // weak filtering - int nd_p = 1; - int nd_q = 1; - if (max_len_p > 1 && max_len_q > 1) { - if (dp0 + dp3 < ((beta + (beta >> 1)) >> 3)) - nd_p = 2; - if (dq0 + dq3 < ((beta + (beta >> 1)) >> 3)) - nd_q = 2; + if (d0 + d3 < beta) { + if (max_len_p > 2 && max_len_q > 2 && + abs(P3 - P0) + abs(Q3 - Q0) < beta_3 && abs(P0 - Q0) < tc25 && + abs(TP3 - TP0) + abs(TQ3 - TQ0) < beta_3 && abs(TP0 - TQ0) < tc25 && + (d0 << 1) < beta_2 && (d3 << 1) < beta_2) { + FUNC(loop_filter_luma_strong)(pix, xstride, ystride, tc, no_p, no_q); + } else { // weak filtering + int nd_p = 1; + int nd_q = 1; + if (max_len_p > 1 && max_len_q > 1) { + if (dp0 + dp3 < ((beta + (beta >> 1)) >> 3)) + nd_p = 2; + if (dq0 + dq3 < ((beta + (beta >> 1)) >> 3)) + nd_q = 2; + } + FUNC(loop_filter_luma_weak)(pix, xstride, ystride, tc, beta, no_p, no_q, nd_p, nd_q); } - FUNC(loop_filter_luma_weak)(pix, xstride, ystride, tc, beta, no_p, no_q, nd_p, nd_q); } } } @@ -954,65 +951,65 @@ static void FUNC(vvc_loop_filter_chroma)(uint8_t *_pix, const ptrdiff_t _xstrid const int end = 8 / size; // 8 samples a loop for (int i = 0; i < end; i++) { - pixel *pix = (pixel *)_pix + i * size * ystride; - const uint8_t no_p = _no_p[i]; - const uint8_t no_q = _no_q[i]; - const int beta = _beta[i] << (BIT_DEPTH - 8); - const int beta_3 = beta >> 3; - const int beta_2 = beta >> 2; - #if BIT_DEPTH < 10 const int tc = (_tc[i] + (1 << (9 - BIT_DEPTH))) >> (10 - BIT_DEPTH); #else const int tc = _tc[i] << (BIT_DEPTH - 10); #endif - const int tc25 = ((tc * 5 + 1) >> 1); - - uint8_t max_len_p = _max_len_p[i]; - uint8_t max_len_q = _max_len_q[i]; - - if (!max_len_p || !max_len_q || !tc) - continue; - - if (max_len_q == 3){ - const int p1n = shift ? FP1 : TP1; - const int p2n = max_len_p == 1 ? p1n : (shift ? FP2 : TP2); - const int p0n = shift ? FP0 : TP0; - const int q0n = shift ? FQ0 : TQ0; - const int q1n = shift ? FQ1 : TQ1; - const int q2n = shift ? FQ2 : TQ2; - const int p3 = max_len_p == 1 ? P1 : P3; - const int p2 = max_len_p == 1 ? P1 : P2; - const int p1 = P1; - const int p0 = P0; - const int dp0 = abs(p2 - 2 * p1 + p0); - const int dq0 = abs(Q2 - 2 * Q1 + Q0); + if (tc) { + pixel *pix = (pixel *)_pix + i * size * ystride; + const uint8_t no_p = _no_p[i]; + const uint8_t no_q = _no_q[i]; + const int beta = _beta[i] << (BIT_DEPTH - 8); + const int beta_3 = beta >> 3; + const int beta_2 = beta >> 2; + const int tc25 = ((tc * 5 + 1) >> 1); + uint8_t max_len_p = _max_len_p[i]; + uint8_t max_len_q = _max_len_q[i]; + + if (!max_len_p || !max_len_q) + continue; - const int dp1 = abs(p2n - 2 * p1n + p0n); - const int dq1 = abs(q2n - 2 * q1n + q0n); - const int d0 = dp0 + dq0; - const int d1 = dp1 + dq1; - - if (d0 + d1 < beta) { - const int p3n = max_len_p == 1 ? p1n : (shift ? FP3 : TP3); - const int q3n = shift ? FQ3 : TQ3; - const int dsam0 = (d0 << 1) < beta_2 && (abs(p3 - p0) + abs(Q0 - Q3) < beta_3) && - abs(p0 - Q0) < tc25; - const int dsam1 = (d1 << 1) < beta_2 && (abs(p3n - p0n) + abs(q0n - q3n) < beta_3) && - abs(p0n - q0n) < tc25; - if (!dsam0 || !dsam1) + if (max_len_q == 3) { + const int p1n = shift ? FP1 : TP1; + const int p2n = max_len_p == 1 ? p1n : (shift ? FP2 : TP2); + const int p0n = shift ? FP0 : TP0; + const int q0n = shift ? FQ0 : TQ0; + const int q1n = shift ? FQ1 : TQ1; + const int q2n = shift ? FQ2 : TQ2; + const int p3 = max_len_p == 1 ? P1 : P3; + const int p2 = max_len_p == 1 ? P1 : P2; + const int p1 = P1; + const int p0 = P0; + const int dp0 = abs(p2 - 2 * p1 + p0); + const int dq0 = abs(Q2 - 2 * Q1 + Q0); + + const int dp1 = abs(p2n - 2 * p1n + p0n); + const int dq1 = abs(q2n - 2 * q1n + q0n); + const int d0 = dp0 + dq0; + const int d1 = dp1 + dq1; + + if (d0 + d1 < beta) { + const int p3n = max_len_p == 1 ? p1n : (shift ? FP3 : TP3); + const int q3n = shift ? FQ3 : TQ3; + const int dsam0 = (d0 << 1) < beta_2 && (abs(p3 - p0) + abs(Q0 - Q3) < beta_3) && + abs(p0 - Q0) < tc25; + const int dsam1 = (d1 << 1) < beta_2 && (abs(p3n - p0n) + abs(q0n - q3n) < beta_3) && + abs(p0n - q0n) < tc25; + if (!dsam0 || !dsam1) + max_len_p = max_len_q = 1; + } else { max_len_p = max_len_q = 1; - } else { - max_len_p = max_len_q = 1; + } } - } - if (max_len_p == 3 && max_len_q == 3) - FUNC(loop_filter_chroma_strong)(pix, xstride, ystride, size, tc, no_p, no_q); - else if (max_len_q == 3) - FUNC(loop_filter_chroma_strong_one_side)(pix, xstride, ystride, size, tc, no_p, no_q); - else - FUNC(loop_filter_chroma_weak)(pix, xstride, ystride, size, tc, no_p, no_q); + if (max_len_p == 3 && max_len_q == 3) + FUNC(loop_filter_chroma_strong)(pix, xstride, ystride, size, tc, no_p, no_q); + else if (max_len_q == 3) + FUNC(loop_filter_chroma_strong_one_side)(pix, xstride, ystride, size, tc, no_p, no_q); + else + FUNC(loop_filter_chroma_weak)(pix, xstride, ystride, size, tc, no_p, no_q); + } } } From patchwork Sun Jan 7 04:42:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nuo Mi X-Patchwork-Id: 45516 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:2449:b0:199:85ec:e0df with SMTP id t9csp350518pzc; Sat, 6 Jan 2024 20:44:30 -0800 (PST) X-Google-Smtp-Source: AGHT+IH0fqB/a1PfEsoqBB6qn+XqIHeZFIGN7m9i2p1x31ZfoYdSlddtRg27G3pJf3hWHidjZ7rS X-Received: by 2002:a17:906:39ca:b0:a26:e9fe:f59f with SMTP id i10-20020a17090639ca00b00a26e9fef59fmr704371eje.133.1704602669946; Sat, 06 Jan 2024 20:44:29 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id su9-20020a17090703c900b00a2a2c6da5acsi569208ejb.93.2024.01.06.20.44.19; Sat, 06 Jan 2024 20:44:29 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=uNknn3P1; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0FA6668CA85; Sun, 7 Jan 2024 06:44:11 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from APC01-SG2-obe.outbound.protection.outlook.com (mail-sgaapc01olkn2038.outbound.protection.outlook.com [40.92.53.38]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B2DAF68CA85 for ; Sun, 7 Jan 2024 06:44:03 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kl0zywMRDN1olHgqz1AVZK4TBYHIaYWj9qQ6utS+QPRsCL4DEhc6Pd87aQ0UCW3OGD8UhC9gMBkLQeuq12RQM6AU9nmEl6yfSsBA75jo/1pIeQou06azmekZL1HOB1a9SkR/Pz5M8lLoA/Gkb4kAHUVaNT9aYtQQC0COsb3EEzpXmNhGdl597j81VcmUTOb1LiI91g1WrP24Cx8KcwBtbT+qbDdG64WaRNDU0wbQyIZ/cJl0SMgBrdG5iL7xWxftgjzy++eC6R8Z3xPowc/IG8N6f6rI+nKQpxPveX69cIzLSYoHsEcHAe5vHIs1c86cWkjzZ8wDVnS0FAepBF3CjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tgjjpojfg1m4q8bO3TC6q78Mg5NYgKR4bHeCp7LjZe0=; b=jOSypOpfxfukFYFPqHVAhfuGVOevDxqGc4rXpxNyBQ9iNagHAheJ3xOtyDwTdz5LbbWvLtkmlNssusd5bzN/dpvxE34jQbzElBXPT54laXpvx3TuPzmURgycOgRaYyPECWefyNj3MoKR6WUoJcz2yHhD0W6jqx0sBhhzYYnpwIEY6T8ZDiy4G40crct317ftbQuP9xCW3pIDWlBAGm+48sMPgbe8BI813MLRvRDlJM9Tc0ygAelLhe2R8i6h2IN56pMIrEMO+fgVljc8fotgUhOGQ+yFF+pcYQDNEHwEIRBb7zrJCPznrwxfsA015PabRqAtXamKSx7C/0PmT/1xng== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tgjjpojfg1m4q8bO3TC6q78Mg5NYgKR4bHeCp7LjZe0=; b=uNknn3P1GAl3vVEg3pLSCaKNp8yheXX9BsGUk510C8ibpa0FMbOFE6PDohDgcA0Zar5Vd6Ob7z2PWIizG+w22woq6Th9a2GbX95cleS9LQFvk1BmOfIX4TqYBuPmoq8Fpm9BCP57SXvoyUFhnfMqgwDgrsiAerd1TNp/uE+EMHl514XZEvwqxdhlN2GtuLiHsOkdaRrhTX5i6ETP2bjNxfNgjmF8/qX5wGBjYxlQ2Ul6IQnGQReQLRvniSI2fzakDmVPBAnR8LuesCFNt79dd3ITs8OD4PKJttGwLUhf3vfbRzKC2o3u6JjtonHF+/ovDhuwYGr0rqjldGvaAOSR3Q== Received: from TYSPR06MB6433.apcprd06.prod.outlook.com (2603:1096:400:47a::6) by SI2PR06MB5268.apcprd06.prod.outlook.com (2603:1096:4:1e7::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7159.13; Sun, 7 Jan 2024 04:43:53 +0000 Received: from TYSPR06MB6433.apcprd06.prod.outlook.com ([fe80::e139:fd7a:1b66:a3f7]) by TYSPR06MB6433.apcprd06.prod.outlook.com ([fe80::e139:fd7a:1b66:a3f7%7]) with mapi id 15.20.7159.018; Sun, 7 Jan 2024 04:43:53 +0000 From: Nuo Mi To: ffmpeg-devel@ffmpeg.org Date: Sun, 7 Jan 2024 12:42:58 +0800 Message-ID: X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240107044258.3926-1-nuomi2021@gmail.com> References: <20240107044258.3926-1-nuomi2021@gmail.com> X-TMN: [HbEB2Brc1cztJ7sYJG/qrkuaHC+5BCCi] X-ClientProxiedBy: TY2PR0101CA0036.apcprd01.prod.exchangelabs.com (2603:1096:404:8000::22) To TYSPR06MB6433.apcprd06.prod.outlook.com (2603:1096:400:47a::6) X-Microsoft-Original-Message-ID: <20240107044258.3926-2-nuomi2021@gmail.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 2 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TYSPR06MB6433:EE_|SI2PR06MB5268:EE_ X-MS-Office365-Filtering-Correlation-Id: 9df0bfe4-f85a-4d05-80b4-08dc0f3b3fd1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: BO7eA2O8h/AXwPDysl5s3Gdu5lMUq8RRy7VGggApoZDplViSCiGdGv2CXaLT5bSMk4zHJg+28ZRulbddkxAEL6xeQQLyqGpv99MI832RAlTOVLBN4SYdkkefzX6MUPflt+l3fcp4eO2ao+Tu7wEwGrVbmJvByjOlcQ62VZMUTkQsYAAOuv3ra5FvlvgfGRf7yD6YgD4hyGsQq1jgpPZirLicYEyWQ+Odb5RsqMByefHfIswvybKIKZOtcPGe6XmQG2WmFK3jS5shLPuGKIZCF2QS5hvN0/mpbXVpjpgw6vrSV8YdO4yiXTpXmvzSoQO2oPY/UCva4bTCUQacuC+7e7DRRrphzal4CfE+nYzFdUpzl8U7GIR2D+6R11ZaIJDcCRhIEp80CCwoKdjeKmRG2FQOz3DM74tL4mtWbic2bKDPaFe8X6wyFUVXR9B21fPTE9C90BpeD3hsj8HnhFbvRcwTy/6cG32hDq0Gg4QukTsDnmkHP2nBzh0YxdiTzS/S646prb/rSnEJNK1brD026n+i2OUwyfLL0EkoeHE+HrwKiX0VLM67XkRoG5y+VsooWKNcqkWv3/pJnlK84S8ui4o3KlWDGw8ZXkJ6Iv0x2TINl5lo6HIBE2evmcCL/ORU X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Nqp91H4Eji+MkgMXRxReD7gWo0ECSdi9+SWhRC4nzKOjBmcSRUe7MPxD7lYHMtGwwtxDaENFmfsjT8qQypfPnUsv9wYwFFk4CdVglJ55ls0F9IvaDeKXo1Yc1Gr5hVWAhvzKQTCb5eL3qQzKkJBvti8YKW6AVPVV9neR3P3Rdft/cqZWACO6eeVvHG0BQ8rnrgDC1lpDdIIdHlMeNznbozdHda0IHxoLzLkYUIlelZEF5545fNF8OCzpwB3jg+s6RBNhntlhmZXo/QgWnP3IVa0nlL3Q3YfuqNrs+xggfoT3iFLK87GaAE8yd1AXyElVZJXaZPnmpHs5Z1rUKmwGeC6VvFnZXp8lXiEmV8oANcBB2PQORYW62WG7N5E2R5kbl385jCdUKG6pPzf1yWkRxM6J54DZ6SdvnJGLwLNdXHs59hw36mIlwNtmpUaNi6CNrfGQj1x9bVDK/plHcqVJtVvgi4fxULegOVAdU3dC881iHYNLExqSL1YMDZigh6wSI0s3TB4PTcG+7W1DTsvf0MCzKN3Eoj8oM8IPEyvBB/zcqxfVcvjTTQXI9G7vsiVHlf8OMyPpEgTyU+vf5goMUOc+pQyOf6SwpObE3EA57NYu/ik4Av10sxiyEd2fkNiQaOyay657doA0iT38XYF6oafOfb36f8hhVlpbqPqqmi2ob5rl3sFZxUCIFR0EJ1h6ZzH0nEzOKr4ZONJCVDVdhfT4EXk545RyX+pBP1uGjvKBa+Hf4DJyTr7yzLxUqFgzzb3Gv/DkyTtU7FSr77QE5PflFTw9s6URFlr877dYfM4r1KPXtYHsmFqt8LYWlrBAB6sC0HrfOe20OOfJ3/yrQhhj4b/lx/089OLXFIWl9Fw4AZ1zJApX7GChdl9GCW9w+onFb/cPmEN0e8HtJ51Or+kSwbBq9Kxl3ZpbqvUO0DIxNDBUlKFGkUsNI2Y2LdmuLVUtwRGQ4hYU6ZVIdzx48Z0IzMgocPY9XuJPOI/MWcP1smylEtaTfx4SRpM4gSQRjq9Nh8/RScGNfJO4nVs2xdrp7gxMENGeqQ3xEZ3mRoKMnzT+O5CQTVKrWYPtmLcIs6SREvnzvURiJGNELoSfzyQda5OBnDaP45uM0T69+2ddMdIeJH/OqRTuMw8mlc11m37dwHbDn9iqLgw13wmLsbEKPxIRQ8oQTCwPzsqOwz1u7Ku2rKEPpAqi8UV6ud4iPtwuVHnwFak0v2sZdN+0ceF4vuPgDP3izBjcwRHhZI+RGURQvmy5FgIM764xDm9U X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9df0bfe4-f85a-4d05-80b4-08dc0f3b3fd1 X-MS-Exchange-CrossTenant-AuthSource: TYSPR06MB6433.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jan 2024 04:43:53.0404 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SI2PR06MB5268 Subject: [FFmpeg-devel] [PATCH 2/2] avcodec/vvcdec: misc, constify hor_ctu_edge X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nuo Mi Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: pIu4EsNchSiH --- libavcodec/vvc/vvc_filter_template.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/libavcodec/vvc/vvc_filter_template.c b/libavcodec/vvc/vvc_filter_template.c index e30deaac96..8687d5b24d 100644 --- a/libavcodec/vvc/vvc_filter_template.c +++ b/libavcodec/vvc/vvc_filter_template.c @@ -792,7 +792,7 @@ static void FUNC(loop_filter_luma_weak)(pixel *pix, const ptrdiff_t xstride, con static void FUNC(vvc_loop_filter_luma)(uint8_t* _pix, ptrdiff_t _xstride, ptrdiff_t _ystride, const int32_t *_beta, const int32_t *_tc, const uint8_t *_no_p, const uint8_t *_no_q, - const uint8_t *_max_len_p, const uint8_t *_max_len_q, int hor_ctu_edge) + const uint8_t *_max_len_p, const uint8_t *_max_len_q, const int hor_ctu_edge) { const ptrdiff_t xstride = _xstride / sizeof(pixel); const ptrdiff_t ystride = _ystride / sizeof(pixel); @@ -1031,7 +1031,7 @@ static void FUNC(vvc_v_loop_filter_chroma)(uint8_t *pix, ptrdiff_t stride, static void FUNC(vvc_h_loop_filter_luma)(uint8_t *pix, ptrdiff_t stride, const int32_t *beta, const int32_t *tc, const uint8_t *no_p, const uint8_t *no_q, - const uint8_t *max_len_p, const uint8_t *max_len_q, int hor_ctu_edge) + const uint8_t *max_len_p, const uint8_t *max_len_q, const int hor_ctu_edge) { FUNC(vvc_loop_filter_luma)(pix, stride, sizeof(pixel), beta, tc, no_p, no_q, max_len_p, max_len_q, hor_ctu_edge); @@ -1039,7 +1039,7 @@ static void FUNC(vvc_h_loop_filter_luma)(uint8_t *pix, ptrdiff_t stride, static void FUNC(vvc_v_loop_filter_luma)(uint8_t *pix, ptrdiff_t stride, const int32_t *beta, const int32_t *tc, const uint8_t *no_p, const uint8_t *no_q, - const uint8_t *max_len_p, const uint8_t *max_len_q, int hor_ctu_edge) + const uint8_t *max_len_p, const uint8_t *max_len_q, const int hor_ctu_edge) { FUNC(vvc_loop_filter_luma)(pix, sizeof(pixel), stride, beta, tc, no_p, no_q, max_len_p, max_len_q, hor_ctu_edge);