From patchwork Thu Feb 22 07:14:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nuo Mi X-Patchwork-Id: 46434 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:26a3:b0:19e:cdac:8cce with SMTP id h35csp173891pze; Wed, 21 Feb 2024 23:15:53 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCVJ8CUHIhhlbHuUJ2i8EZw7iRv3aQewZg+Sq+3YLbOarZPkJES78ab3IZPAMKxH0tXnJUBwb+9Traeb+KqnIdYeoCMmpa7zbGQVwg== X-Google-Smtp-Source: AGHT+IEqUZdId/wj2vHjS4AnkA6IUsDkGfjvQ7IGAiegWPFjH1k3eH2Y0n+cyeeASghAAR0fTNLL X-Received: by 2002:aa7:da53:0:b0:565:21f1:4f27 with SMTP id w19-20020aa7da53000000b0056521f14f27mr1417294eds.23.1708586153492; Wed, 21 Feb 2024 23:15:53 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id p12-20020aa7cc8c000000b0056069a0bd04si5171145edt.190.2024.02.21.23.15.53; Wed, 21 Feb 2024 23:15:53 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=bbU0e1Fq; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5154E68D22C; Thu, 22 Feb 2024 09:15:12 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from APC01-SG2-obe.outbound.protection.outlook.com (mail-sgaapc01olkn2011.outbound.protection.outlook.com [40.92.53.11]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8DF7368D18B for ; Thu, 22 Feb 2024 09:15:09 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hN7Ru2bGMAdDAHVQz/FELKgbXM8+HIwcb6XME9yCzHMnW7gtdtcRJ/GNtigu1NWlr9Fh3h7rIbQR1kJUr9ooWxzIFwL5MLH736gwPaL44UThuKeKYxU0G4jh3eZ7oPaK3CNf5kHbkJGjwb2LyXUjme6IH2FT8teaBlZX2kKtU0q2QhQ/uFBr8hEMOtyIdQTrqkUck+quhX+2bYAqy9EtwtWgUCaZXcahcWpvzEA2gROE4O7BmQcrt0CQcnJXoGNPyR1/ttbec47MhrtN1DA+8Ik41WrbqIx5UJKCqu5kSN8Npcue+9oKuCjMtAhJSsumDZimxTQcC65xVVlWL22g1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=N7fWMebf3fo9NcjrgaJeRXnV1TM3EKbFaW8fr9tsb1Q=; b=brgcH65Ji2r7rV0GpObxNQNvETtUwsA9ujbeDVA8jL1rjFoMhA0OvX39AaJkgNCDS4804J4ZXQvqf4dLzMAMlg+Jztj9SCGMuj2DDy3CTlfnWSK0QDGaPs2nrHyRp2opic7WUst2zebI+nslJRJXxrxQw7tnME3uNNx8OyCtw/OG0QvusVe+4il8HmlLonkeOK1EgaffS6d1T8INwZvD5J8juS54UHAVyXBetfaCYsw4rNOrkswCvsmZFLIXIioVLsnowJ7o2lNk1tQPJzsYWUSxXn68PPEQlVcnnrI4tbjOr1FLBy7pLNlwzfLDX6kz4ZAiD7x0MwWiS6yhTyWMCA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N7fWMebf3fo9NcjrgaJeRXnV1TM3EKbFaW8fr9tsb1Q=; b=bbU0e1Fqhj/dTRJXySoUDOFdzXDdGhq+HeiAkwC858IIOF38a/h503gdAnyto8kBxoMnZSmQc2x1XBXfzEIApqStGHwDE3VBo9HWzcUNojR2h32GKQrX7Zzrn6ftVpXXDE+/N4rgnGk3MeWxjiNJATl4SxA7LH7CCsQeThdJF/tFdvSNZC1uj4jMWQTJnWw6icKrcIq13dS5jGWABtBuUEHITRYRU4vk3GL5ISPS3wYcjRHxXeYFH083z1NbJfKpUuo4W+0/C4g7EdvOKi7Pq7P9quHPlRMoWpV4qgz8MaMn7Xe16DQ9iGS80BytompvSBrEezxNuD7Las3lfSYqQQ== Received: from TYSPR06MB6433.apcprd06.prod.outlook.com (2603:1096:400:47a::6) by TYUPR06MB6241.apcprd06.prod.outlook.com (2603:1096:400:358::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.38; Thu, 22 Feb 2024 07:14:54 +0000 Received: from TYSPR06MB6433.apcprd06.prod.outlook.com ([fe80::e139:fd7a:1b66:a3f7]) by TYSPR06MB6433.apcprd06.prod.outlook.com ([fe80::e139:fd7a:1b66:a3f7%7]) with mapi id 15.20.7292.036; Thu, 22 Feb 2024 07:14:54 +0000 From: Nuo Mi To: ffmpeg-devel@ffmpeg.org Date: Thu, 22 Feb 2024 15:14:06 +0800 Message-ID: X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240222071406.5714-1-nuomi2021@gmail.com> References: <20240222071406.5714-1-nuomi2021@gmail.com> X-TMN: [wC/U/se3exR1lcRfBb+B24WBubgqf/k8] X-ClientProxiedBy: TYCP286CA0049.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:2b5::7) To TYSPR06MB6433.apcprd06.prod.outlook.com (2603:1096:400:47a::6) X-Microsoft-Original-Message-ID: <20240222071406.5714-12-nuomi2021@gmail.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 2 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: TYSPR06MB6433:EE_|TYUPR06MB6241:EE_ X-MS-Office365-Filtering-Correlation-Id: 0c3303a4-6def-46ad-5460-08dc3375f801 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: c+LoFkADCS/Ul2HP9T5/KZ47JtbrF+3xCqx7Qc7m9fa32c9n6Cij8ogXaiMc0ylkTsLQS/3FMQlZxiN/aMy7bkJw3hIZM7p/OACr6lZOxdmSOE3K0RVABjLkbYM0YkgEIvIr0AoRhRDp828mpVlFRPmeCXqEK9YqMIk1BURJ17CoFzuIOObCD/7nvy0WLXv4BfmUGTyenRjBu94BoOnP+xxtILpEHZQ4noP+rqDNBE65bDWJeYhwTRBijcbbNqguUHPyYtWJBXlY74kNK3WjDuOiUWeU+pPk1F1gTKqwJWFp2hY69VPAeeJeAX491vkk570phTknIrCUk+/ohHfj4Wu2L56SxuMlP/YQsM8kZlQg/BSbLDZlRLniQYqnYWvdl6mkXbFykXF/NCAl9ZmN/30GHufnk2Se85ImyWdS74JgGm60hwqjwwX1o5Tnncjh+T5oQEqfqTF9hU4OAYXyHp7Yt1kyEcQLobU2vBFp/5aj8Ol0SNaFEwdxYjWqDo0H00hBVuwZDhwbBinUdWGDDIN0iJBHZkWm1tRS2FZGRrGtXZuWgsM2s8VMTNBVh8oBa4hq4UpAG8GoKeRZlxbe/lbUim0XNknk92GshFfx8JBR78vHZanCIxsvszlPSIxk X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: V58Spss4iXDL0GmbRQlR9nlfToBGOybQCPZB/o11MTa2qn6BOQECkStN4CoISZJIj3G+kQz8JndTcs40LtJOoQq/zGgeyYWfzzbzo7dsxg3u1pL/47vhoxnPnMvhwOWxTee1jeom1TiSHOTZ7HPJvO0I20LbG+txbKpKj5R82pKdPYiwEADUXQGVsfJcxi9OKrpPrWHNrnOq9GGwEViUpBMglbq7oHtAg6KrGfR/ah2duWw3/g746ld88GFZbTrFHKPNXgbcnCYTatQJzMi30sGtMnW+t3HfN1q3AUAzgtwm85xrvGYUFbZxJUqkRPY7s4ICTL7/5LDJaq/4MuuC9jrJ8wHzpgc5EcaeV+mUrh2E2i8HF/4aCuGjlZafWzEud5El07P+RPie/fPxkx+WHpPqSRhZV7HXsSWl2lx7DBft4NJaZeuBAZw08DtbHQnVfAYB25VQo2AcH4oQvxO+iMymGUo7Y8EKo3G2PeyBkEja+3Y8EbffMd4LSJ5eoPht+aagUGfRtFz6FGUGwJo5ItSNvZ8MIQBxJebPxXeZ2fENiDe+KTKHlcGwbOY27PM/a89w31JrN0jXeHKfV8UpojofbAU0/eXPcFpbRgGV361zgmpQcRHD/W5d7gkUeMVs/3Ci6fWuyDYWuAdtVvpLM+n3y5LBobOORZW0nf6OGDMQgCBuNxUgLr4BqnU8peZlo1Ei9PUWvDNPKzbCuObza1Wj4KGemM48xxACEfz9hAMkiO0hqbUb0In7Bv/Ht2qCwkE1ck1S/Jaln65Xl5xuyCG77VCRL5260owL1+LWyyMq5zzzHL/9dXHijmQdEAA9wtAFny/7+jzVKfzbmi3ofh6Pd5IWZ7xqQmChwCcqAOCd/UvBverlmQVoOxRwAZXILbWXTUQWCP4wGaWUL0Ab1fqr45VsmVRyhOj/cymbH/uaorMz+qt4VhTNUqmratihGRAdmets6EXUli/BMWGQIT7AnWi+Jhn36RcqHshvB9dwuCE2X8WW62aO6MsYqnXCRcQBfYvJnWhusBoqC85fqLjcfcW+9jKV1+bwM2AiydGpaUrglOVARwsecZE/Pk6Hq3CejP/94pnQLCk8hw+0eO4OfvKVbkZVLngZ7BO/U8nuqIj1gXGd3pnLYeTG4DqYpPTop4a06f25c+2RC1R4qOLq9heCkW4QNJnhS8Vj15ZLeg/NGE3ZY9p0ntWKKuOSpOTh2RGQ4goWjC+n4h+LSFe6CmNVWwjzCFdaTgQ9Bj6Idw7/V6eJvSrUPPVJ40oa/MgmD5NdAg+AT9BMZmPO9Q== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0c3303a4-6def-46ad-5460-08dc3375f801 X-MS-Exchange-CrossTenant-AuthSource: TYSPR06MB6433.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Feb 2024 07:14:54.7191 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: TYUPR06MB6241 Subject: [FFmpeg-devel] [PATCH 11/11] avcodec/vvcdec: add Intra Block Copy decoder X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nuo Mi , Wu Jianhua Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: pXhwqXavS+uw From: Wu Jianhua Introduction at https://ieeexplore.ieee.org/document/9408666 passed files: 10b444_A_Kwai_3.bit 10b444_B_Kwai_3.bit CodingToolsSets_D_Tencent_2.bit IBC_A_Tencent_2.bit IBC_B_Tencent_2.bit IBC_C_Tencent_2.bit IBC_D_Tencent_2.bit IBC_E_Tencent_1.bit LOSSLESS_B_HHI_3.bit Signed-off-by: Wu Jianhua Signed-off-by: Nuo Mi --- libavcodec/vvc/vvc_intra.c | 81 ++++++++++++++++++++++++++++++++++++++ libavcodec/vvc/vvcdec.c | 25 ++++++++++++ libavcodec/vvc/vvcdec.h | 3 ++ 3 files changed, 109 insertions(+) diff --git a/libavcodec/vvc/vvc_intra.c b/libavcodec/vvc/vvc_intra.c index fb001d6713..58dd492478 100644 --- a/libavcodec/vvc/vvc_intra.c +++ b/libavcodec/vvc/vvc_intra.c @@ -20,11 +20,13 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ #include "libavutil/frame.h" +#include "libavutil/imgutils.h" #include "vvc_data.h" #include "vvc_inter.h" #include "vvc_intra.h" #include "vvc_itx_1d.h" +#include "vvc_mvs.h" static int is_cclm(enum IntraPredMode mode) { @@ -580,6 +582,81 @@ static int reconstruct(VVCLocalContext *lc) return 0; } +#define POS(c_idx, x, y) \ + &fc->frame->data[c_idx][((y) >> fc->ps.sps->vshift[c_idx]) * fc->frame->linesize[c_idx] + \ + (((x) >> fc->ps.sps->hshift[c_idx]) << fc->ps.sps->pixel_shift)] + +#define IBC_POS(c_idx, x, y) \ + (fc->tab.ibc_vir_buf[c_idx] + \ + (x << ps) + (y + ((cu->y0 & ~(sps->ctb_size_y - 1)) >> vs)) * ibc_stride) +#define IBC_X(x) ((x) & ((fc->tab.sz.ibc_buffer_width >> hs) - 1)) +#define IBC_Y(y) ((y) & ((1 << sps->ctb_log2_size_y >> vs) - 1)) + +static void intra_block_copy(const VVCLocalContext *lc, const int c_idx) +{ + const CodingUnit *cu = lc->cu; + const PredictionUnit *pu = &cu->pu; + const VVCFrameContext *fc = lc->fc; + const VVCSPS *sps = fc->ps.sps; + const Mv *bv = &pu->mi.mv[L0][0]; + const int hs = sps->hshift[c_idx]; + const int vs = sps->vshift[c_idx]; + const int ps = sps->pixel_shift; + const int ref_x = IBC_X((cu->x0 >> hs) + (bv->x >> (4 + hs))); + const int ref_y = IBC_Y((cu->y0 >> vs) + (bv->y >> (4 + vs))); + const int w = cu->cb_width >> hs; + const int h = cu->cb_height >> vs; + const int ibc_buf_width = fc->tab.sz.ibc_buffer_width >> hs; ///< IbcBufWidthY and IbcBufWidthC + const int rw = FFMIN(w, ibc_buf_width - ref_x); + const int ibc_stride = ibc_buf_width << ps; + const int dst_stride = fc->frame->linesize[c_idx]; + const uint8_t *ibc_buf = IBC_POS(c_idx, ref_x, ref_y); + uint8_t *dst = POS(c_idx, cu->x0, cu->y0); + + av_image_copy_plane(dst, dst_stride, ibc_buf, ibc_stride, rw << ps, h); + + if (w > rw) { + //wrap around, left part + ibc_buf = IBC_POS(c_idx, 0, ref_y); + dst += rw << ps; + av_image_copy_plane(dst, dst_stride, ibc_buf, ibc_stride, (w - rw) << ps, h); + } +} + +static void vvc_predict_ibc(const VVCLocalContext *lc) +{ + const H266RawSPS *rsps = lc->fc->ps.sps->r; + + intra_block_copy(lc, LUMA); + if (lc->cu->tree_type == SINGLE_TREE && rsps->sps_chroma_format_idc) { + intra_block_copy(lc, CB); + intra_block_copy(lc, CR); + } +} + +static void ibc_fill_vir_buf(const VVCLocalContext *lc, const CodingUnit *cu) +{ + const VVCFrameContext *fc = lc->fc; + const VVCSPS *sps = fc->ps.sps; + const int has_chroma = sps->r->sps_chroma_format_idc && cu->tree_type != DUAL_TREE_LUMA; + const int start = cu->tree_type == DUAL_TREE_CHROMA; + const int end = has_chroma ? CR : LUMA; + + for (int c_idx = start; c_idx <= end; c_idx++) { + const int hs = sps->hshift[c_idx]; + const int vs = sps->vshift[c_idx]; + const int ps = sps->pixel_shift; + const int x = IBC_X(cu->x0 >> hs); + const int y = IBC_Y(cu->y0 >> vs); + const int src_stride = fc->frame->linesize[c_idx]; + const int ibc_stride = fc->tab.sz.ibc_buffer_width >> hs << ps; + const uint8_t *src = POS(c_idx, cu->x0, cu->y0); + uint8_t *ibc_buf = IBC_POS(c_idx, x, y); + + av_image_copy_plane(ibc_buf, ibc_stride, src, src_stride, cu->cb_width >> hs << ps , cu->cb_height >> vs); + } +} + int ff_vvc_reconstruct(VVCLocalContext *lc, const int rs, const int rx, const int ry) { const VVCFrameContext *fc = lc->fc; @@ -599,6 +676,8 @@ int ff_vvc_reconstruct(VVCLocalContext *lc, const int rs, const int rx, const in if (cu->ciip_flag) ff_vvc_predict_ciip(lc); + else if (cu->pred_mode == MODE_IBC) + vvc_predict_ibc(lc); if (cu->coded_flag) { ret = reconstruct(lc); } else { @@ -607,6 +686,8 @@ int ff_vvc_reconstruct(VVCLocalContext *lc, const int rs, const int rx, const in if (sps->r->sps_chroma_format_idc && cu->tree_type != DUAL_TREE_LUMA) add_reconstructed_area(lc, CHROMA, cu->x0, cu->y0, cu->cb_width, cu->cb_height); } + if (sps->r->sps_ibc_enabled_flag) + ibc_fill_vir_buf(lc, cu); cu = cu->next; } ff_vvc_ctu_free_cus(ctu); diff --git a/libavcodec/vvc/vvcdec.c b/libavcodec/vvc/vvcdec.c index e88e746de4..caba1b28c6 100644 --- a/libavcodec/vvc/vvcdec.c +++ b/libavcodec/vvc/vvcdec.c @@ -262,6 +262,30 @@ static void ispmf_tl_init(TabList *l, VVCFrameContext *fc) TL_ADD(ispmf, w64 * h64); } +static void ibc_tl_init(TabList *l, VVCFrameContext *fc) +{ + const VVCSPS *sps = fc->ps.sps; + const VVCPPS *pps = fc->ps.pps; + const int ctu_height = pps ? pps->ctb_height : 0; + const int ctu_size = sps ? sps->ctb_size_y : 0; + const int ps = sps ? sps->pixel_shift : 0; + const int chroma_idc = sps ? sps->r->sps_chroma_format_idc : 0; + const int changed = fc->tab.sz.chroma_format_idc != chroma_idc || + fc->tab.sz.ctu_height != ctu_height || + fc->tab.sz.ctu_size != ctu_size || + fc->tab.sz.pixel_shift != ps; + + fc->tab.sz.ibc_buffer_width = ctu_size ? 2 * MAX_CTU_SIZE * MAX_CTU_SIZE / ctu_size : 0; + + tl_init(l, 0, changed); + + for (int i = LUMA; i < VVC_MAX_SAMPLE_ARRAYS; i++) { + const int hs = sps ? sps->hshift[i] : 0; + const int vs = sps ? sps->vshift[i] : 0; + TL_ADD(ibc_vir_buf[i], fc->tab.sz.ibc_buffer_width * ctu_size * ctu_height << ps >> hs >> vs); + } +} + typedef void (*tl_init_fn)(TabList *l, VVCFrameContext *fc); static int frame_context_for_each_tl(VVCFrameContext *fc, int (*unary_fn)(TabList *l)) @@ -276,6 +300,7 @@ static int frame_context_for_each_tl(VVCFrameContext *fc, int (*unary_fn)(TabLis pixel_buffer_nz_tl_init, msm_tl_init, ispmf_tl_init, + ibc_tl_init, }; for (int i = 0; i < FF_ARRAY_ELEMS(init); i++) { diff --git a/libavcodec/vvc/vvcdec.h b/libavcodec/vvc/vvcdec.h index 4f7ef3a32f..aa3d715524 100644 --- a/libavcodec/vvc/vvcdec.h +++ b/libavcodec/vvc/vvcdec.h @@ -170,6 +170,8 @@ typedef struct VVCFrameContext { int *coeffs; struct CTU *ctus; + uint8_t *ibc_vir_buf[VVC_MAX_SAMPLE_ARRAYS]; ///< IbcVirBuf[] + //used in arrays_init only struct { int ctu_count; @@ -185,6 +187,7 @@ typedef struct VVCFrameContext { int pixel_shift; int bs_width; int bs_height; + int ibc_buffer_width; ///< IbcBufWidth } sz; } tab; } VVCFrameContext;