From patchwork Thu Jul 9 06:00:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gautam Ramakrishnan X-Patchwork-Id: 20905 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 632B64496C7 for ; Thu, 9 Jul 2020 09:29:22 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 39CA468B00A; Thu, 9 Jul 2020 09:29:22 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oi1-f193.google.com (mail-oi1-f193.google.com [209.85.167.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id EA4CA68A0C7 for ; Thu, 9 Jul 2020 09:29:15 +0300 (EEST) Received: by mail-oi1-f193.google.com with SMTP id t4so1025362oij.9 for ; Wed, 08 Jul 2020 23:29:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=YAmshoRXXpr9BBAZjtgOpG6ohUFqtkbxnho5pVLwnzE=; b=r6JkpexvkdrdY0aveVDoUzmNtJhHiP5Zm4Qps2zNfe8EUm89Z13qHYNjs0Na2lEb/N 0rgRwR6ovP6kSL6x/M5SxRBgUsJdQJseld1N25YLNUGr2CNgV1qgMJ9ayrHoVoYTMPCi l8KIz/n1uOnisrqJfgHJrW4qGi5dPOoxRzmAOEVsfvEkR1570goUNveTkyoRMyYRVCR7 /hHv5sa41IBb4KdbYv6FwXzOA2AcNUL6LZ7fuZw/KmDQ0VW+J10EIvCpSv1U3oS0/A36 jhWCGdIhxoTwvuDJn3w/oZmKmEnXZaposNt4EGvr+NbRUjO3BWBS/A10GVqNhyH9Gl1d +dhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=YAmshoRXXpr9BBAZjtgOpG6ohUFqtkbxnho5pVLwnzE=; b=t7ZVPLSpP8absgnf3lu+S5NMibwSW8RQbTzlPbLBXAgy9N/XfNNbYoO/1CxC53cifQ d5ax5W18aU70FebJLRBIlV4qkSem3Hq5nrG68dGEQgXnZ/X8LkFBmc1uTs+bk/aAxbgn 5igVGAzN8dwOMTNU6++DONP2eHBya0ha+mOstNqqE8xRUbwbqYxZpfKQe/TKd910KkQ8 saXXyDqq1sQRT+a1TR8TPk/VGHtlQhgl4m/8Fi2Y+tVpPWvVgiQKVvjc2mz+p0meSO5w 3qmRvnX+bqwfob4XrERrcgSiDHx+7yZg7NGarcuV8IeSG7BVaEFqNt7oxkmQeX8yIuOy 5Plg== X-Gm-Message-State: AOAM533XxLXb1E5Z5CJXZM+GgRwrxl4rBcf51b5ejhZ16RUpu3OTmJq6 1oXx5EkgnlNur+KCS1gKnOTfTXNul7HAYA== X-Google-Smtp-Source: ABdhPJy9U4u19nqzTrlLXntNDczP8VhPSeilkSgFXmaxz8ghBZ+MXgJ9Py9iDB8kUMfv7X8sfFbiIQ== X-Received: by 2002:a17:90a:8a07:: with SMTP id w7mr13310482pjn.219.1594274424686; Wed, 08 Jul 2020 23:00:24 -0700 (PDT) Received: from localhost.localdomain ([223.226.75.84]) by smtp.gmail.com with ESMTPSA id fh12sm1181188pjb.5.2020.07.08.23.00.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 23:00:23 -0700 (PDT) From: gautamramk@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Thu, 9 Jul 2020 11:30:18 +0530 Message-Id: <20200709060018.11888-1-gautamramk@gmail.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH] libavcodec/jpeg2000dec: Fix RPCL progression order X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Gautam Ramakrishnan MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Gautam Ramakrishnan This patch fixes a check in the RPCL progression order, tested on p1_07.j2k. --- libavcodec/jpeg2000dec.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/libavcodec/jpeg2000dec.c b/libavcodec/jpeg2000dec.c index 18a933077e..3ea939f7d9 100644 --- a/libavcodec/jpeg2000dec.c +++ b/libavcodec/jpeg2000dec.c @@ -1393,22 +1393,28 @@ static int jpeg2000_decode_packets_po_iteration(Jpeg2000DecoderContext *s, Jpeg2 uint8_t reducedresno = codsty->nreslevels - 1 -reslevelno; // ==> N_L - r Jpeg2000ResLevel *rlevel = comp->reslevel + reslevelno; unsigned prcx, prcy; + int trx0, try0; - int xc = x / s->cdx[compno]; - int yc = y / s->cdy[compno]; + if (!s->cdx[compno] || !s->cdy[compno]) + return AVERROR_INVALIDDATA; + + trx0 = ff_jpeg2000_ceildiv(tile->coord[0][0], s->cdx[compno] << reslevelno); + try0 = ff_jpeg2000_ceildiv(tile->coord[1][0], s->cdy[compno] << reslevelno); if (reslevelno >= codsty->nreslevels) continue; - if (yc % (1LL << (rlevel->log2_prec_height + reducedresno)) && y != tile->coord[1][0]) //FIXME this is a subset of the check + if (!(y % ((uint64_t)s->cdy[compno] << (rlevel->log2_prec_height + reducedresno)) == 0 || + (y == tile->coord[1][0] && (try0 << reducedresno) % (1U << (reducedresno + rlevel->log2_prec_height))))) continue; - if (xc % (1LL << (rlevel->log2_prec_width + reducedresno)) && x != tile->coord[0][0]) //FIXME this is a subset of the check + if (!(x % ((uint64_t)s->cdx[compno] << (rlevel->log2_prec_width + reducedresno)) == 0 || + (x == tile->coord[0][0] && (trx0 << reducedresno) % (1U << (reducedresno + rlevel->log2_prec_width))))) continue; // check if a precinct exists - prcx = ff_jpeg2000_ceildivpow2(xc, reducedresno) >> rlevel->log2_prec_width; - prcy = ff_jpeg2000_ceildivpow2(yc, reducedresno) >> rlevel->log2_prec_height; + prcx = ff_jpeg2000_ceildiv(x, s->cdx[compno] << reducedresno) >> rlevel->log2_prec_width; + prcy = ff_jpeg2000_ceildiv(y, s->cdy[compno] << reducedresno) >> rlevel->log2_prec_height; prcx -= ff_jpeg2000_ceildivpow2(comp->coord_o[0][0], reducedresno) >> rlevel->log2_prec_width; prcy -= ff_jpeg2000_ceildivpow2(comp->coord_o[1][0], reducedresno) >> rlevel->log2_prec_height; @@ -1906,7 +1912,6 @@ static inline void tile_codeblocks(Jpeg2000DecoderContext *s, Jpeg2000Tile *tile continue; x = cblk->coord[0][0] - band->coord[0][0]; y = cblk->coord[1][0] - band->coord[1][0]; - if (comp->roi_shift) roi_scale_cblk(cblk, comp, &t1); if (codsty->transform == FF_DWT97)