From patchwork Tue Oct 12 08:23:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fei Wang X-Patchwork-Id: 31071 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2084:0:0:0:0 with SMTP id a4csp4527980ioa; Tue, 12 Oct 2021 01:26:26 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxHr6nlcK+5bdH8bkJSz5RJQdKgTOJUi/8ncsj3x59sHgaYpect3mEWJ2niBt5jWyxNjdTP X-Received: by 2002:a05:6402:21eb:: with SMTP id ce11mr47684085edb.153.1634027186456; Tue, 12 Oct 2021 01:26:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1634027186; cv=none; d=google.com; s=arc-20160816; b=zNQuLOeKavl2LK2MV7ThyD6pkeJDBw3yCFaTVtlGt5Nm6SFwIYpmIB3pKbtwYKxq7H o/nlBAMLmhO6/Y9759kxPGzm62Bqp+ptVxK30nX8OZltkw1D8BBiu3Q5O05gPuMLAHqS wcrDVKNEoyR7ROlxCb7eKXDitbpk30yjs/QzdJ2NFOsjR24czxrHXO7TcGAgJTmPXaOv ZHk+SVY+vIBNKPVp/Y4Am9HSlL/xkHfg2rDFYp+58cMbkYmHre7MngF3cdwzSyaZ9cAl PLXTw2q1E88CHkxvo5RMOwaPF3VsDyMVu5AqvWNgVl3poyLM12+Gml5pPYZmu5Xt1YDj NZFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=GANFaPuoWkrEZ89hLO5rMbx2TidXe0x91r4aDVp8PGs=; b=IEmNRnxGRQJobWCMoKhvRRJe0daPPbaYfEdPklVQnUhwg3QiQoxWpU0lGOGPFXxQ03 niB08kmMu5Tmdr5gEprEsOBLGmIc3kv4KU+5TC9mTm0oRhUXTAvt3I3H6aSmHH0YBezz GOBaAFoKeIRZHPBC3+jI2VHewYKMv33K4oM35WHKgauOfUHMo7pV51/OZW4tvKUDukwb DGPdfPC9BpVLmhxHU3s/fHBXF2e/JrGd9w+0thKxv9wCo7Ox/ByAhX3r1xMvq7kn63on KTb6mMujdnx08p/NyErryjQP7X84IxcUdpa2cYxPabO44Zru81hxENcVu32jFQE+wlL1 PCxA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v23si14341213ejo.762.2021.10.12.01.26.25; Tue, 12 Oct 2021 01:26:26 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4A40868A8EE; Tue, 12 Oct 2021 11:26:21 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 02FAB689C54 for ; Tue, 12 Oct 2021 11:26:13 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10134"; a="225852077" X-IronPort-AV: E=Sophos;i="5.85,367,1624345200"; d="scan'208";a="225852077" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2021 01:26:11 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,367,1624345200"; d="scan'208";a="524139840" Received: from t.sh.intel.com ([10.239.159.147]) by orsmga001.jf.intel.com with ESMTP; 12 Oct 2021 01:26:09 -0700 From: Fei Wang To: ffmpeg-devel@ffmpeg.org Date: Tue, 12 Oct 2021 16:23:56 +0800 Message-Id: <20211012082404.31639-1-fei.w.wang@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v5 1/9] cbs_av1: fix incorrect data type X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Fei Wang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: EXhlsFf5kUF/ Since order_hint_bits_minus_1 range is 0~7, cur_frame_hint can be most 128. And similar return value for cbs_av1_get_relative_dist. So if plus them and use int8_t for the result may lose its precision. Signed-off-by: Fei Wang --- update: 1. move additional film grain frame from av1dec.c to vaapi_av1.c 2. patch 3~5 can fix clip: https://drive.google.com/file/d/1Qdx_18_BFcFf_5_XXSZOVohLpaAb-yIw/view?usp=sharing libavcodec/cbs_av1_syntax_template.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/cbs_av1_syntax_template.c b/libavcodec/cbs_av1_syntax_template.c index 6fe6e9a4f3..d98d3d42de 100644 --- a/libavcodec/cbs_av1_syntax_template.c +++ b/libavcodec/cbs_av1_syntax_template.c @@ -355,7 +355,7 @@ static int FUNC(set_frame_refs)(CodedBitstreamContext *ctx, RWContext *rw, AV1_REF_FRAME_ALTREF2, AV1_REF_FRAME_ALTREF }; int8_t ref_frame_idx[AV1_REFS_PER_FRAME], used_frame[AV1_NUM_REF_FRAMES]; - int8_t shifted_order_hints[AV1_NUM_REF_FRAMES]; + int16_t shifted_order_hints[AV1_NUM_REF_FRAMES]; int cur_frame_hint, latest_order_hint, earliest_order_hint, ref; int i, j;