From patchwork Fri Dec 17 08:46:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?6buE5ZOy6ZuE?= X-Patchwork-Id: 32680 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp1327337iog; Fri, 17 Dec 2021 00:47:00 -0800 (PST) X-Google-Smtp-Source: ABdhPJyyJntUSI+m7YX+TMrMd/zz+o4STuOTR4DBmVJOI9o0psGedEUTfoROlIfLh21JKlUKvMNB X-Received: by 2002:a17:906:6a22:: with SMTP id qw34mr1721470ejc.329.1639730819857; Fri, 17 Dec 2021 00:46:59 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639730819; cv=none; d=google.com; s=arc-20160816; b=r7PTtA6+DhdtFYoOkUndbdEHzCX21AQq/NdRIdkJpViaYfh6BVuM5ofwogzqFZ0DPX ZshbUrs3aPoolfEn4rA7bPYuGRDiC/CdaF15sL/IKOUrZFggbjFk5mC2wcyqm7u7EHOz KB1HlY1EG3k07/qjIH1HVGvgiC0JLYlO6sKq7pCy2cRxk1qEuk50xtGlF0exYGKigIg4 R3JisDMZzBpeCUW2tz7jyol/7bIZigVKLawvJJAQV/tI/morVMbewLJV88eALH+qrSow +0HwpQZoQC/piONJen6NnIMC+JElqEnO8qPs30qT0Dfdo8E6Yb42DvONO463LYzGzgps gEWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:from:to:mime-version:date:message-id :dkim-signature:delivered-to; bh=gQcwS8mUYDPBgspvhvEV3PSQmC+GNTyC+8p9g3OBHYQ=; b=xJvPhLp8iBSLXhaK+xlUayliCpSiPSLLgq1w6mO8oZxAcWWJxKpU7tYWqvFPdfzxu2 2/pU4zCyoQjQvEgG1/SfY98xIUOomx05jfsrwXIBp1dhZTw95Xhg/+EaAGuhqiXMejyd W7A6n1HmzcsfVgio4ySngD2ebmZmbuRJ4C3DfHjZ9HufhmYmLdkpufE0Plt9JJm2OKiH bu13cYpIynPXsKX+4rB6IKVqzalO7s4o+eOpCZyH+9S7ljj7pdTeM/BCHux+IYCJ2MR/ LWJYZ1LACF1gZuB5X+el8unqg6fyIXLcmyxIK6JQKhPL6RjPF7NZ7Ill0G1oqKBdGTtA lVPA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@bytedance-com.20200927.dkim.feishu.cn header.s=s1 header.b=cBFTJoYW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ji10si6545413ejc.539.2021.12.17.00.46.59; Fri, 17 Dec 2021 00:46:59 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@bytedance-com.20200927.dkim.feishu.cn header.s=s1 header.b=cBFTJoYW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3E56768AE7E; Fri, 17 Dec 2021 10:46:54 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from v02.bc.feishu.cn (v02.bc.feishu.cn [101.36.218.27]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1DC0B68A255 for ; Fri, 17 Dec 2021 10:46:46 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=s1; d=bytedance-com.20200927.dkim.feishu.cn; t=1639730797; h=mime-version:from:date:message-id:subject:to; bh=BGwArzVcZtPQDFcQiPIApMRj13MbMUOzwVp7mtPQpsY=; b=cBFTJoYWlmlxZlDm7KZ0F0xcR+sC8rjkdhw2Vy6+YDv+RqB276euDa6try03WrT04rUvM0 NSgSqVf7fuktWn+49WAZgfOVK25S6yFJprTvbuNBoDQU3b3FdNiLSt9Jz7SCNrkrlucROG x1bEtp1Kfhgo3vcDah8xLGi4pp7ddcLpeXuirKOXwmGl6amVaiO5P98pVdEj3uXVwCKLag 0FFzXiBRLU/Pa/iPIIbR8Z7HJKFX2d+UgUZulao0LMcIV2gTX/k3znVApFaBx1AjgPPtjS HqB0qrUOTFOuAqJugNpqGsJcKQ3IxzWEEQ6dygb1m86Idu7oC00I+fYgs927jA== Message-Id: <20211217084611.3234-1-huangzhexiong@bytedance.com> X-Mailer: git-send-email 2.25.1 Date: Fri, 17 Dec 2021 16:46:37 +0800 Mime-Version: 1.0 To: From: "Zhexiong Huang" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: [FFmpeg-devel] [PATCH] avcodec/hevcdec: fix the order of in-loop filters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Zhexiong Huang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 4VKCqill02z4 As we know, in-loop filters in HEVC are applied by the following ordered steps: firstly deblocking filter, then sample adaptive offset filter if enabled. However, in the current version of FFmpeg, pixels without being deblocking-filtered could be used by SAO filter when CTU size is 16 and chroma format is 4:2:0 or 4:2:2, which could lead to the wrong result in chroma components. This patch changes the algorithm of deblocking filter, which ensures that SAO filter is applied after deblocking filter for all the related pixels. The new algorithm fixes this decoding problem when CTU size is 16, and shall not affect performance and correctness when CTU size is 32 or 64. Signed-off-by: Zhexiong Huang --- libavcodec/hevc_filter.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/libavcodec/hevc_filter.c b/libavcodec/hevc_filter.c index 3c45b5a39e..c53875d85f 100644 --- a/libavcodec/hevc_filter.c +++ b/libavcodec/hevc_filter.c @@ -512,10 +512,10 @@ static void deblocking_filter_CTB(HEVCContext *s, int x0, int y0) x_end2 = x_end; if (x_end2 != s->ps.sps->width) - x_end2 -= 8; + x_end2 += 8; for (y = y0; y < y_end; y += 8) { // vertical filtering luma - for (x = x0 ? x0 : 8; x < x_end; x += 8) { + for (x = x0 + 8; x < x_end2; x += 8) { const int bs0 = s->vertical_bs[(x + y * s->bs_width) >> 2]; const int bs1 = s->vertical_bs[(x + (y + 4) * s->bs_width) >> 2]; if (bs0 || bs1) { @@ -545,7 +545,7 @@ static void deblocking_filter_CTB(HEVCContext *s, int x0, int y0) continue; // horizontal filtering luma - for (x = x0 ? x0 - 8 : 0; x < x_end2; x += 8) { + for (x = x0; x < x_end; x += 8) { const int bs0 = s->horizontal_bs[( x + y * s->bs_width) >> 2]; const int bs1 = s->horizontal_bs[((x + 4) + y * s->bs_width) >> 2]; if (bs0 || bs1) { @@ -579,9 +579,13 @@ static void deblocking_filter_CTB(HEVCContext *s, int x0, int y0) int h = 1 << s->ps.sps->hshift[chroma]; int v = 1 << s->ps.sps->vshift[chroma]; + x_end2 = x_end; + if (x_end2 != s->ps.sps->width) + x_end2 += 8 * h; + // vertical filtering chroma for (y = y0; y < y_end; y += (8 * v)) { - for (x = x0 ? x0 : 8 * h; x < x_end; x += (8 * h)) { + for (x = x0 + 8 * h; x < x_end2; x += (8 * h)) { const int bs0 = s->vertical_bs[(x + y * s->bs_width) >> 2]; const int bs1 = s->vertical_bs[(x + (y + (4 * v)) * s->bs_width) >> 2]; @@ -612,10 +616,7 @@ static void deblocking_filter_CTB(HEVCContext *s, int x0, int y0) // horizontal filtering chroma tc_offset = x0 ? left_tc_offset : cur_tc_offset; - x_end2 = x_end; - if (x_end != s->ps.sps->width) - x_end2 = x_end - 8 * h; - for (x = x0 ? x0 - 8 * h : 0; x < x_end2; x += (8 * h)) { + for (x = x0; x < x_end; x += (8 * h)) { const int bs0 = s->horizontal_bs[( x + y * s->bs_width) >> 2]; const int bs1 = s->horizontal_bs[((x + 4 * h) + y * s->bs_width) >> 2]; if ((bs0 == 2) || (bs1 == 2)) {