From patchwork Sat Jun 24 17:12:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 42292 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3805:b0:126:ac68:d900 with SMTP id p5csp137850pzf; Sat, 24 Jun 2023 10:13:14 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7tYEiRqzXhcRmHDkNeS6Zj86HsKp9vyngCqLD3qgZ5o/j/KEh9hwEe49lGPOoGLAaLvImn X-Received: by 2002:a17:907:268a:b0:987:fac5:9fb9 with SMTP id bn10-20020a170907268a00b00987fac59fb9mr16755261ejc.35.1687626794212; Sat, 24 Jun 2023 10:13:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687626794; cv=none; d=google.com; s=arc-20160816; b=B/US3K5fG40Ea9AOdOhbZQf5o3oOTWCoRUBJwyPymVm6Aes3BBkIeIHr6QtDqR+cAF 8cVjN6ljlK5/dUj0+1fYLhFamEoael+P1TbfbAqS2I2C1NlU7bYCMguvIsSV68yfDc96 038haaK2VVypAIl8QE0jzbeeTdu9m/VWMecfSCn7Oqc5u3zyWiJxKSRFfDU716FYD64f QcIoBISuaAtximjFMkUHmEywlybszLxWWfQYeDGtYizgpFUyjHd9TwCZ/RU50O9J/kIr YTul1qUAmwcBoZF1Fij1t+Gy5QNZgF7jFMdCuqDGek059qiP57ocOJbyzNUudHtU3GFD gy5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=AVUopIMBbYs8F7GJKlqDl0OBb7UWiq+t72b+KvqY0A0=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=0GFe7Ho77vbbK9Bdn6VBNfA32hFBCHeJW+H2rCqObKwwEeesFIer4bSvxCucLheAOz BQ1RgzqxbxyRjThZgZaE8qgzGprIhLXuv5dFzjU2H8P0lWc3Sa1O86Uih+wLaLDI8+lV FMw1I4VykN/qvb57/Xclpn1plcv2Wd2yp3QtNHeCt3tIgiPIFpxwkqQLndzwalUN8FnC OQFmMku9Z12EmCr/soF/51rbhS0yIb/rfM7n692CB74IU2psrhpSBqbT02heyQPhw8p0 RmaDfi0IPs9cEpIQ/PrZzOGf8/0gA57LeQgeUmzlwKYPThZqqXxZIabYssj/Kd2xJuoL wgSw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20221208 header.b=hNQDwE1Y; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b11-20020a170906d10b00b009827e07cca6si941621ejz.17.2023.06.24.10.13.13; Sat, 24 Jun 2023 10:13:14 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20221208 header.b=hNQDwE1Y; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BE20468C100; Sat, 24 Jun 2023 20:13:09 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oo1-f54.google.com (mail-oo1-f54.google.com [209.85.161.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E8D4D68BF54 for ; Sat, 24 Jun 2023 20:13:03 +0300 (EEST) Received: by mail-oo1-f54.google.com with SMTP id 006d021491bc7-55b3d77c9deso1300532eaf.0 for ; Sat, 24 Jun 2023 10:13:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687626782; x=1690218782; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=29eY9xjZgPqrzNwUFQUbTGMVmfHpLU8XeDxOTMAZKQQ=; b=hNQDwE1YvYQggGn/XPTczrS0myQInmnafx6n8IKCkGntOeTOJGJ6Zgnwmkm0FAyWwa Iy4P/afzsiWGj6X7Cn/jpxD9HCxQ2BvV9QMeqgKherEjfLyTcL/SqYFFEJxMc+VXCENT f3GFujroE2ss5g7OtAPLeiUT+8Aj+HDbc113xEchIJuw58UjzCvGvcpXWZ65m7d5dZtR zZly5CWSLlTOt7+OWoJ2abW291kSSv69u+rsYc9hs8+VQxUFKCz0PeGjInqLjFsWmmsl Wwnp/lluolbyvVEkZDkQkfXpBugd3uXvfQgaU3fD8W7oggt3OUxo8KfGO5g9BYgOuEeE dvYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687626782; x=1690218782; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=29eY9xjZgPqrzNwUFQUbTGMVmfHpLU8XeDxOTMAZKQQ=; b=IqNURDpFRCrAm1+WTrwtc51xMER2lYJ8er/WStQsKJVhwKULnPjKUlP4+RxU8+RC1C 0OskZvFzMC7Ifi6nzeIzt3lV1PCteyJ7ImLh6W2ohprECn7kuLyPgD0DUeAvBtv3Y574 M3CvPqa+FQKxvuoZ+UJObGMRdFaJpfZBt2/Jm/OAqeY6TjsGGHkH3L3rpxO9nAnpWVD+ LKOPeWQQZMU9N7++aWPQZJffDKGcNgd06FKl8jZY3LYzs/yiLpJfgZF/3zcUKrp3+IAX AIbM6KmSpa7dS1KhqjPLqgQrNjjSfAaVAyc9rPnomIVTNV7Gi92T5nz6T24mzaMz2hEv yVYw== X-Gm-Message-State: AC+VfDxR/6yjlN3Oi1t9hB3QuMxQsu5UNSYykf3kH2YDzKxDvOrBV4lD AvDBFU+HOMYhCwwGr3KA0EIlrH+qpG0= X-Received: by 2002:a4a:c807:0:b0:555:91b1:546a with SMTP id s7-20020a4ac807000000b0055591b1546amr20156328ooq.2.1687626781593; Sat, 24 Jun 2023 10:13:01 -0700 (PDT) Received: from localhost.localdomain (host197.190-225-105.telecom.net.ar. [190.225.105.197]) by smtp.gmail.com with ESMTPSA id bs18-20020a056820179200b0055a0166070asm831273oob.30.2023.06.24.10.12.59 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 24 Jun 2023 10:13:00 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sat, 24 Jun 2023 14:12:45 -0300 Message-ID: <20230624171245.740-1-jamrial@gmail.com> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avcodec/j2kenc: use uint64_t type for the lambda value X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: GvvSNkByw+Z8 Should fix integer overflows, and improve encoding results. Signed-off-by: James Almer --- libavcodec/j2kenc.c | 4 ++-- tests/ref/vsynth/vsynth1-jpeg2000-yuva444p16 | 8 ++++---- tests/ref/vsynth/vsynth2-jpeg2000-yuva444p16 | 8 ++++---- tests/ref/vsynth/vsynth3-jpeg2000-yuva444p16 | 8 ++++---- tests/ref/vsynth/vsynth_lena-jpeg2000-yuva444p16 | 8 ++++---- 5 files changed, 18 insertions(+), 18 deletions(-) diff --git a/libavcodec/j2kenc.c b/libavcodec/j2kenc.c index 2b0c58cff4..5f95b772d1 100644 --- a/libavcodec/j2kenc.c +++ b/libavcodec/j2kenc.c @@ -127,7 +127,7 @@ typedef struct { uint8_t *buf_end; int bit_index; - int64_t lambda; + uint64_t lambda; Jpeg2000CodingStyle codsty; Jpeg2000QuantStyle qntsty; @@ -1348,7 +1348,7 @@ static void makelayers(Jpeg2000EncoderContext *s, Jpeg2000Tile *tile) } } -static int getcut(Jpeg2000Cblk *cblk, int64_t lambda, int dwt_norm) +static int getcut(Jpeg2000Cblk *cblk, uint64_t lambda, int dwt_norm) { int passno, res = 0; for (passno = 0; passno < cblk->npasses; passno++){ diff --git a/tests/ref/vsynth/vsynth1-jpeg2000-yuva444p16 b/tests/ref/vsynth/vsynth1-jpeg2000-yuva444p16 index 363ddc4ffa..f3d3ba8a6c 100644 --- a/tests/ref/vsynth/vsynth1-jpeg2000-yuva444p16 +++ b/tests/ref/vsynth/vsynth1-jpeg2000-yuva444p16 @@ -1,4 +1,4 @@ -9261706cfd4dfd8db38fc9bdafe6bef7 *tests/data/fate/vsynth1-jpeg2000-yuva444p16.avi -12433462 tests/data/fate/vsynth1-jpeg2000-yuva444p16.avi -6bc7131cd19dbb59339bb13a6d53b94d *tests/data/fate/vsynth1-jpeg2000-yuva444p16.out.rawvideo -stddev: 12.97 PSNR: 25.87 MAXDIFF: 169 bytes: 7603200/ 7603200 +4c051562f5ac7c9871e034917326787c *tests/data/fate/vsynth1-jpeg2000-yuva444p16.avi +12502390 tests/data/fate/vsynth1-jpeg2000-yuva444p16.avi +59d53e883f76d41d17c926236ca9d7d2 *tests/data/fate/vsynth1-jpeg2000-yuva444p16.out.rawvideo +stddev: 2.66 PSNR: 39.62 MAXDIFF: 44 bytes: 7603200/ 7603200 diff --git a/tests/ref/vsynth/vsynth2-jpeg2000-yuva444p16 b/tests/ref/vsynth/vsynth2-jpeg2000-yuva444p16 index bec08da307..3cbe5f4ad2 100644 --- a/tests/ref/vsynth/vsynth2-jpeg2000-yuva444p16 +++ b/tests/ref/vsynth/vsynth2-jpeg2000-yuva444p16 @@ -1,4 +1,4 @@ -54063ab98e48b2a079b63643fe26e082 *tests/data/fate/vsynth2-jpeg2000-yuva444p16.avi -11489842 tests/data/fate/vsynth2-jpeg2000-yuva444p16.avi -a4232b2959472f88b5310572456fa900 *tests/data/fate/vsynth2-jpeg2000-yuva444p16.out.rawvideo -stddev: 11.69 PSNR: 26.77 MAXDIFF: 188 bytes: 7603200/ 7603200 +37555b5b00bba2773d5a4212f6c7debd *tests/data/fate/vsynth2-jpeg2000-yuva444p16.avi +11501630 tests/data/fate/vsynth2-jpeg2000-yuva444p16.avi +61be7920a21190d946c22cb2c945ab6f *tests/data/fate/vsynth2-jpeg2000-yuva444p16.out.rawvideo +stddev: 0.53 PSNR: 53.49 MAXDIFF: 13 bytes: 7603200/ 7603200 diff --git a/tests/ref/vsynth/vsynth3-jpeg2000-yuva444p16 b/tests/ref/vsynth/vsynth3-jpeg2000-yuva444p16 index dae44f40bd..ce92bf7e43 100644 --- a/tests/ref/vsynth/vsynth3-jpeg2000-yuva444p16 +++ b/tests/ref/vsynth/vsynth3-jpeg2000-yuva444p16 @@ -1,4 +1,4 @@ -afdff47e9c71d912153e59a56afdf2a9 *tests/data/fate/vsynth3-jpeg2000-yuva444p16.avi -194250 tests/data/fate/vsynth3-jpeg2000-yuva444p16.avi -9e0ef11f8c85206a950a919cb679f129 *tests/data/fate/vsynth3-jpeg2000-yuva444p16.out.rawvideo -stddev: 10.72 PSNR: 27.53 MAXDIFF: 82 bytes: 86700/ 86700 +ae5951cee3db63c7c6f81d23b601c0fb *tests/data/fate/vsynth3-jpeg2000-yuva444p16.avi +194668 tests/data/fate/vsynth3-jpeg2000-yuva444p16.avi +d959793c1f0bb86b0fc509d5747e03c7 *tests/data/fate/vsynth3-jpeg2000-yuva444p16.out.rawvideo +stddev: 3.06 PSNR: 38.39 MAXDIFF: 40 bytes: 86700/ 86700 diff --git a/tests/ref/vsynth/vsynth_lena-jpeg2000-yuva444p16 b/tests/ref/vsynth/vsynth_lena-jpeg2000-yuva444p16 index d9e856735b..6dc522bb01 100644 --- a/tests/ref/vsynth/vsynth_lena-jpeg2000-yuva444p16 +++ b/tests/ref/vsynth/vsynth_lena-jpeg2000-yuva444p16 @@ -1,4 +1,4 @@ -e9c56e128cf980db7c8e965d2afc0ea6 *tests/data/fate/vsynth_lena-jpeg2000-yuva444p16.avi -11109882 tests/data/fate/vsynth_lena-jpeg2000-yuva444p16.avi -3e1a4f9ca46c2dd3fd997c01a0dfb130 *tests/data/fate/vsynth_lena-jpeg2000-yuva444p16.out.rawvideo -stddev: 11.67 PSNR: 26.79 MAXDIFF: 150 bytes: 7603200/ 7603200 +fcfb2e4738ad6afd7f9d4e9f92084415 *tests/data/fate/vsynth_lena-jpeg2000-yuva444p16.avi +11123818 tests/data/fate/vsynth_lena-jpeg2000-yuva444p16.avi +62ac96a84eb370e5543582512dbe5d7c *tests/data/fate/vsynth_lena-jpeg2000-yuva444p16.out.rawvideo +stddev: 0.45 PSNR: 54.88 MAXDIFF: 7 bytes: 7603200/ 7603200