From patchwork Fri Oct 4 14:31:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nuo Mi X-Patchwork-Id: 52051 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:938f:0:b0:48e:c0f8:d0de with SMTP id z15csp633003vqg; Fri, 4 Oct 2024 12:39:06 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVm2mtIHL/Ta62XtJoiBIKsfs4KrHehkOvGji2MnTAAHR8RJ2TxJL5Vr5aYb+HJgC13cFy622qwm5XBLZd0zXJ5@gmail.com X-Google-Smtp-Source: AGHT+IE1ET/m/vCZ6cz3Xf+faBCToZr8v9cvHNKqXOPh13S1DpQP/2L9tM/HNA0hcdj3H8Hd7qaE X-Received: by 2002:a05:6512:31cb:b0:539:8d67:1b1b with SMTP id 2adb3069b0e04-539ab86b7demr2448307e87.26.1728070745955; Fri, 04 Oct 2024 12:39:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1728070745; cv=none; d=google.com; s=arc-20240605; b=Ys6KRo9PFn5A/xG07NjBS4+nmDCgj2rfqeuSP+T4fb8tSPpKVVYUmdxn9/nc1jRmyt F61nW8fS8R48bAQKzRu9q8zQsxhLpPXEi1h/zaXU3csmD5+yFkmAZyWdcA6iSeMPclvh ovtvfVf+A3CrxCH2kabFMmXEYaLUPXUdAgCKV1XceBQSkuTlVP10VZPTKRT9P7SL5+LT mftVVhLInLUAPzzB8kO+5BNESY7IZTiRUFVHdqrqMitf9gAxdypNaomDNJAwYnsIkJeI aPwwnQrr8FWQkNbb2iq6jEoM7Oct8O6yvKok6eWbRh95f+wGRAPKCKchZd93nTGkZFPk jXbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=ccM4+5VigvUoulLLLZZzxeEv2FQKufDaaKTt3ACr/4Q=; fh=mZk9AfRmPBMGW9h158yccPeJgZmEjzU2tMQtLZcF184=; b=l1y0xmkiGh7CAj1CFpDbFWa+h6WwsCstI4iATsVDAErbU0q1smDmsZqYKHE02keHMO FREmuf2/0R/UrddJ1+J0chK7vcmTtloUULFTDqKIGSA9kmidGQqAqRyC+GsVkuuSQl0D h3A5fJLGoMY6OVUwiUuTUwrMwnZNXqaMVIG4Ghqb/Aa5D8nmtfvhLYDGsJ31TTfHna3X Jmi+Yg+V0rhGny6N2+Box8qs2dkVUDkyTd3qrWpnemcvlpVoSL7dr3x62/jGGmfXbdgY svr2ZKJdce5lbVxvlhg8G3s2Qjz0ff22FuTqdn1J87q8o99Rt7IbXO9FktGI8L5f3E8W F+mg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=G2JsYREb; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 2adb3069b0e04-539b002867bsi98658e87.603.2024.10.04.12.39.05; Fri, 04 Oct 2024 12:39:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=G2JsYREb; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1F21368DA61; Fri, 4 Oct 2024 17:31:41 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1D1AA68DAA8 for ; Fri, 4 Oct 2024 17:31:35 +0300 (EEST) Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-20b7463dd89so24272605ad.2 for ; Fri, 04 Oct 2024 07:31:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1728052293; x=1728657093; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mN1FJYYyk2vB1BcuN58GnuxbCRRkKGiARstYdM22le4=; b=G2JsYREbRbIy2RlAIN27EIh0uWDCJ0UmElGWkzqJKBge4qsxvTSmgUUmQ1qzOltNiY Qmm4pNTNik3gTroh7nmys6+Shyuo3hm+Vh/Pdgqa9VBElTcn4Zvwl//UPaTYFtpczrrt 2/XA8NHKM5RIsUrObXfy9Q/UbD9KV6UVhedD1nBwXc4gFK19joPDMNLlIxICiyWB7P58 Ok8fasavzi4A9XLjL+RMNdwsy/q1uTliEUdOiLMwoY44LNkuklrJHqiy1dKSo/x+GZRr 0tDzGx9AbGGzSqtLXCkKX/bGRCFi8RVVOG+ufbWP/ykFK0y1WsUyJQ12/JzFRDdoyOzn EDMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728052293; x=1728657093; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mN1FJYYyk2vB1BcuN58GnuxbCRRkKGiARstYdM22le4=; b=rjT8X8tgwvYNI4e27Gl+gStS31N3Fpr1lRkTw699NLsw/gDfShBaYAgvEI0Xl7+u7b vqmlnHGXAZux6faaz40LY9H73dsl642qLQjQjbc3tTjQurZfWyykrMtUTTNHXaFWXMsa q58EBPG7Cky1+0XGArwHS6xXXTFx7d+lTuUp8f+p4WSt/9ixX1whMiG1oAbh4Jb8dgSP ++iP94AyV8AKK2bB9HihM2wQQTkL/hPEyVciPqnJzYVZ8PNuiXSc0+R7Z5QQDNJiPRWe jz04Go6FJsckHK0LY8ccUYuoZ3dNKoLfeCwkYeYhFeHPJEyAynzcjXkZZ0xsbwnYXbkM +qDA== X-Gm-Message-State: AOJu0Yy+BAE2Z3u70V93V2WEbFrxRtNHvWCPb72BPG2u7hQEnPFs0nXe k8pqAU6jmiPmCHwTMa1zTOVo75AWtoRHkoNcpIvfiVOuqifRXiPJya50EfHv X-Received: by 2002:a17:902:ea11:b0:20b:7a31:4522 with SMTP id d9443c01a7336-20bff04a762mr41020285ad.42.1728052292972; Fri, 04 Oct 2024 07:31:32 -0700 (PDT) Received: from localhost ([112.64.8.17]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20bef70704bsm24648025ad.268.2024.10.04.07.31.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Oct 2024 07:31:32 -0700 (PDT) From: Nuo Mi To: ffmpeg-devel@ffmpeg.org Date: Fri, 4 Oct 2024 22:31:14 +0800 Message-Id: <20241004143115.382070-3-nuomi2021@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241004143115.382070-1-nuomi2021@gmail.com> References: <20241004143115.382070-1-nuomi2021@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/4] avcodec/vvdec: refact, ff_vvc_deblock_bs use CodingUnit/TransformUnit instead of fc->tabs X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Nuo Mi Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: WZ1HywXXZbzI perf result for: "perf record -F 99 ./ffmpeg_g -i Tango2_3840x2160_60_10_420_27_LD.266 -f null -" before: 5.24% 1.87% ffmpeg_g [.] vvc_deblock_bs_chroma 1.72% ffmpeg_g [.] ff_vvc_deblock_bs 1.65% ffmpeg_g [.] vvc_deblock_bs_luma after: 3.48% 1.84% ffmpeg_g [.] vvc_deblock_bs_chroma 1.64% ffmpeg_g [.] ff_vvc_deblock_bs + vvc_deblock_bs_luma(inlined) --- libavcodec/vvc/ctu.c | 2 + libavcodec/vvc/ctu.h | 3 ++ libavcodec/vvc/filter.c | 90 +++++++++++++++++------------------------ 3 files changed, 42 insertions(+), 53 deletions(-) diff --git a/libavcodec/vvc/ctu.c b/libavcodec/vvc/ctu.c index 8210ab520f..e49976c66b 100644 --- a/libavcodec/vvc/ctu.c +++ b/libavcodec/vvc/ctu.c @@ -241,6 +241,7 @@ static TransformUnit* add_tu(VVCFrameContext *fc, CodingUnit *cu, const int x0, tu->height = tu_height; tu->joint_cbcr_residual_flag = 0; memset(tu->coded_flag, 0, sizeof(tu->coded_flag)); + tu->avail[LUMA] = tu->avail[CHROMA] = 0; tu->nb_tbs = 0; return tu; @@ -267,6 +268,7 @@ static TransformBlock* add_tb(TransformUnit *tu, VVCLocalContext *lc, tb->ts = 0; tb->coeffs = lc->coeffs; lc->coeffs += tb_width * tb_height; + tu->avail[!!c_idx] = true; return tb; } diff --git a/libavcodec/vvc/ctu.h b/libavcodec/vvc/ctu.h index eab4612561..c5533c1ad0 100644 --- a/libavcodec/vvc/ctu.h +++ b/libavcodec/vvc/ctu.h @@ -23,6 +23,8 @@ #ifndef AVCODEC_VVC_CTU_H #define AVCODEC_VVC_CTU_H +#include + #include "libavcodec/cabac.h" #include "libavutil/mem_internal.h" @@ -172,6 +174,7 @@ typedef struct TransformUnit { int y0; int width; int height; + bool avail[CHROMA + 1]; // contains luma/chroma block uint8_t joint_cbcr_residual_flag; ///< tu_joint_cbcr_residual_flag diff --git a/libavcodec/vvc/filter.c b/libavcodec/vvc/filter.c index 9a45a735e0..a7f102bc64 100644 --- a/libavcodec/vvc/filter.c +++ b/libavcodec/vvc/filter.c @@ -451,15 +451,15 @@ static int boundary_strength(const VVCLocalContext *lc, const MvField *curr, con //part of 8.8.3.3 Derivation process of transform block boundary static void derive_max_filter_length_luma(const VVCFrameContext *fc, const int qx, const int qy, - const int is_intra, const int has_subblock, const int vertical, uint8_t *max_len_p, uint8_t *max_len_q) + const int size_q, const int has_subblock, const int vertical, uint8_t *max_len_p, uint8_t *max_len_q) { const int px = vertical ? qx - 1 : qx; const int py = !vertical ? qy - 1 : qy; const uint8_t *tb_size = vertical ? fc->tab.tb_width[LUMA] : fc->tab.tb_height[LUMA]; const int size_p = tb_size[(py >> MIN_TU_LOG2) * fc->ps.pps->min_tu_width + (px >> MIN_TU_LOG2)]; - const int size_q = tb_size[(qy >> MIN_TU_LOG2) * fc->ps.pps->min_tu_width + (qx >> MIN_TU_LOG2)]; const int min_cb_log2 = fc->ps.sps->min_cb_log2_size_y; const int off_p = (py >> min_cb_log2) * fc->ps.pps->min_cb_width + (px >> min_cb_log2); + if (size_p <= 4 || size_q <= 4) { *max_len_p = *max_len_q = 1; } else { @@ -525,7 +525,7 @@ static void vvc_deblock_subblock_bs(const VVCLocalContext *lc, } static av_always_inline int deblock_bs(const VVCLocalContext *lc, - const int x_p, const int y_p, const int x_q, const int y_q, + const int x_p, const int y_p, const int x_q, const int y_q, const CodingUnit *cu, const TransformUnit *tu, const RefPicList *rpl_p, const int c_idx, const int off_to_cb, const uint8_t has_sub_block) { const VVCFrameContext *fc = lc->fc; @@ -542,12 +542,10 @@ static av_always_inline int deblock_bs(const VVCLocalContext *lc, const MvField *mvf_q = &tab_mvf[pu_q]; const uint8_t chroma = !!c_idx; const int tu_p = (y_p >> log2_min_tu_size) * min_tu_width + (x_p >> log2_min_tu_size); - const int tu_q = (y_q >> log2_min_tu_size) * min_tu_width + (x_q >> log2_min_tu_size); const int cb_p = (y_p >> log2_min_cb_size) * min_cb_width + (x_p >> log2_min_cb_size); - const int cb_q = (y_q >> log2_min_cb_size) * min_cb_width + (x_q >> log2_min_cb_size); - const uint8_t pcmf = fc->tab.pcmf[chroma][cb_p] && fc->tab.pcmf[chroma][cb_q]; - const uint8_t intra = fc->tab.cpm[chroma][cb_p] == MODE_INTRA || fc->tab.cpm[chroma][cb_q] == MODE_INTRA; - const uint8_t same_mode = fc->tab.cpm[chroma][cb_p] == fc->tab.cpm[chroma][cb_q]; + const uint8_t pcmf = fc->tab.pcmf[chroma][cb_p] && cu->bdpcm_flag[chroma]; + const uint8_t intra = fc->tab.cpm[chroma][cb_p] == MODE_INTRA || cu->pred_mode == MODE_INTRA; + const uint8_t same_mode = fc->tab.cpm[chroma][cb_p] == cu->pred_mode; if (pcmf) return 0; @@ -557,12 +555,12 @@ static av_always_inline int deblock_bs(const VVCLocalContext *lc, if (chroma) { return fc->tab.tu_coded_flag[c_idx][tu_p] || - fc->tab.tu_coded_flag[c_idx][tu_q] || fc->tab.tu_joint_cbcr_residual_flag[tu_p] || - fc->tab.tu_joint_cbcr_residual_flag[tu_q]; + tu->coded_flag[c_idx] || + tu->joint_cbcr_residual_flag; } - if (fc->tab.tu_coded_flag[LUMA][tu_p] || fc->tab.tu_coded_flag[LUMA][tu_q]) + if (fc->tab.tu_coded_flag[LUMA][tu_p] || tu->coded_flag[LUMA]) return 1; if ((off_to_cb && ((off_to_cb % 8) || !has_sub_block))) @@ -606,27 +604,23 @@ static int deblock_is_boundary(const VVCLocalContext *lc, const int boundary, } static void vvc_deblock_bs_luma(const VVCLocalContext *lc, - const int x0, const int y0, const int width, const int height, const int rs, const int vertical) + const int x0, const int y0, const int width, const int height, + const CodingUnit *cu, const TransformUnit *tu, int rs, const int vertical) { - const VVCFrameContext *fc = lc->fc; - const MvField *tab_mvf = fc->tab.mvf; - const int mask = LUMA_GRID - 1; - const int log2_min_pu_size = MIN_PU_LOG2; - const int min_pu_width = fc->ps.pps->min_pu_width; - const int min_cb_log2 = fc->ps.sps->min_cb_log2_size_y; - const int min_cb_width = fc->ps.pps->min_cb_width; - const int pos = vertical ? x0 : y0; - const int off_q = (y0 >> min_cb_log2) * min_cb_width + (x0 >> min_cb_log2); - const int cb = (vertical ? fc->tab.cb_pos_x : fc->tab.cb_pos_y )[LUMA][off_q]; - const int is_intra = tab_mvf[(y0 >> log2_min_pu_size) * min_pu_width + - (x0 >> log2_min_pu_size)].pred_flag == PF_INTRA; + const VVCFrameContext *fc = lc->fc; + const PredictionUnit *pu = &cu->pu; + const int mask = LUMA_GRID - 1; + const int pos = vertical ? x0 : y0; + const int cb = vertical ? cu->x0 : cu->y0; + const int is_intra = cu->pred_mode == MODE_INTRA; + const int cb_size = vertical ? cu->cb_width : cu->cb_height; + const int has_sb = !is_intra && (pu->merge_subblock_flag || pu->inter_affine_flag) && cb_size > 8; if (deblock_is_boundary(lc, pos > 0 && !(pos & mask), pos, rs, vertical)) { const int is_vb = is_virtual_boundary(fc, pos, vertical); const int size = vertical ? height : width; + const int size_q = vertical ? width : height; const int off = cb - pos; - const int cb_size = (vertical ? fc->tab.cb_width : fc->tab.cb_height)[LUMA][off_q]; - const int has_sb = !is_intra && (fc->tab.msf[off_q] || fc->tab.iaf[off_q]) && cb_size > 8; const int flag = vertical ? BOUNDARY_LEFT_SLICE : BOUNDARY_UPPER_SLICE; const RefPicList *rpl_p = (lc->boundary_flags & flag) ? ff_vvc_get_ref_list(fc, fc->ref, x0 - vertical, y0 - !vertical) : lc->sc->rpl; @@ -635,24 +629,23 @@ static void vvc_deblock_bs_luma(const VVCLocalContext *lc, const int x = x0 + i * !vertical; const int y = y0 + i * vertical; uint8_t max_len_p, max_len_q; - const int bs = is_vb ? 0 : deblock_bs(lc, x - vertical, y - !vertical, x, y, rpl_p, LUMA, off, has_sb); + const int bs = is_vb ? 0 : deblock_bs(lc, x - vertical, y - !vertical, x, y, cu, tu, rpl_p, LUMA, off, has_sb); TAB_BS(fc->tab.bs[vertical][LUMA], x, y) = bs; - derive_max_filter_length_luma(fc, x, y, is_intra, has_sb, vertical, &max_len_p, &max_len_q); + derive_max_filter_length_luma(fc, x, y, size_q, has_sb, vertical, &max_len_p, &max_len_q); TAB_MAX_LEN(fc->tab.max_len_p[vertical], x, y) = max_len_p; TAB_MAX_LEN(fc->tab.max_len_q[vertical], x, y) = max_len_q; } } - if (!is_intra) { - if (fc->tab.msf[off_q] || fc->tab.iaf[off_q]) - vvc_deblock_subblock_bs(lc, cb, x0, y0, width, height, vertical); - } + if (has_sb) + vvc_deblock_subblock_bs(lc, cb, x0, y0, width, height, vertical); } static void vvc_deblock_bs_chroma(const VVCLocalContext *lc, - const int x0, const int y0, const int width, const int height, const int rs, const int vertical) + const int x0, const int y0, const int width, const int height, + const CodingUnit *cu, const TransformUnit *tu, const int rs, const int vertical) { const VVCFrameContext *fc = lc->fc; const int shift = (vertical ? fc->ps.sps->hshift : fc->ps.sps->vshift)[CHROMA]; @@ -667,7 +660,7 @@ static void vvc_deblock_bs_chroma(const VVCLocalContext *lc, for (int i = 0; i < size; i += 2) { const int x = x0 + i * !vertical; const int y = y0 + i * vertical; - const int bs = is_vb ? 0 : deblock_bs(lc, x - vertical, y - !vertical, x, y, NULL, c_idx, 0, 0); + const int bs = is_vb ? 0 : deblock_bs(lc, x - vertical, y - !vertical, x, y, cu, tu, NULL, c_idx, 0, 0); TAB_BS(fc->tab.bs[vertical][c_idx], x, y) = bs; } @@ -682,29 +675,20 @@ void ff_vvc_deblock_bs(VVCLocalContext *lc, const int rx, const int ry, const in { const VVCFrameContext *fc = lc->fc; const VVCSPS *sps = fc->ps.sps; - const VVCPPS *pps = fc->ps.pps; - const int ctb_size = sps->ctb_size_y; const int x0 = rx << sps->ctb_log2_size_y; const int y0 = ry << sps->ctb_log2_size_y; - const int x_end = FFMIN(x0 + ctb_size, pps->width) >> MIN_TU_LOG2; - const int y_end = FFMIN(y0 + ctb_size, pps->height) >> MIN_TU_LOG2; - const int has_chroma = !!sps->r->sps_chroma_format_idc; - deblock_bs_fn deblock_bs[] = { - vvc_deblock_bs_luma, vvc_deblock_bs_chroma - }; ff_vvc_decode_neighbour(lc, x0, y0, rx, ry, rs); - for (int vertical = 0; vertical <= 1; vertical++) { - for (int is_chroma = 0; is_chroma <= has_chroma; is_chroma++) { - const int hs = sps->hshift[is_chroma]; - const int vs = sps->vshift[is_chroma]; - for (int y = y0 >> MIN_TU_LOG2; y < y_end; y++) { - for (int x = x0 >> MIN_TU_LOG2; x < x_end; x++) { - const int off = y * fc->ps.pps->min_tu_width + x; - if ((fc->tab.tb_pos_x0[is_chroma][off] >> MIN_TU_LOG2) == x && (fc->tab.tb_pos_y0[is_chroma][off] >> MIN_TU_LOG2) == y) { - deblock_bs[is_chroma](lc, x << MIN_TU_LOG2, y << MIN_TU_LOG2, - fc->tab.tb_width[is_chroma][off] << hs, fc->tab.tb_height[is_chroma][off] << vs, rs, vertical); - } + for (const CodingUnit *cu = fc->tab.cus[rs]; cu; cu = cu->next) { + for (const TransformUnit *tu = cu->tus.head; tu; tu = tu->next) { + for (int vertical = 0; vertical <= 1; vertical++) { + if (tu->avail[LUMA]) + vvc_deblock_bs_luma(lc, tu->x0, tu->y0, tu->width, tu->height, cu, tu, rs, vertical); + if (tu->avail[CHROMA]) { + if (cu->isp_split_type != ISP_NO_SPLIT && cu->tree_type == SINGLE_TREE) + vvc_deblock_bs_chroma(lc, cu->x0, cu->y0, cu->cb_width, cu->cb_height, cu, tu, rs, vertical); + else + vvc_deblock_bs_chroma(lc, tu->x0, tu->y0, tu->width, tu->height, cu, tu, rs, vertical); } } }