From patchwork Mon Jul 13 17:20:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gautam Ramakrishnan X-Patchwork-Id: 20995 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 3EED34493BF for ; Mon, 13 Jul 2020 20:27:56 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1E81B68A3F8; Mon, 13 Jul 2020 20:27:56 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f195.google.com (mail-pl1-f195.google.com [209.85.214.195]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7136668828E for ; Mon, 13 Jul 2020 20:27:49 +0300 (EEST) Received: by mail-pl1-f195.google.com with SMTP id k5so5789415plk.13 for ; Mon, 13 Jul 2020 10:27:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=W0vkLVpbGC8iplakE5HggLPIkZErAF5DCoPm30qoMME=; b=va+dPHl/FyO4vBsMdicIKdWmzwa/UZf+BTQTRaHs7EHzNlCqO8yL1aLJcdUzt0Pp3k e1hqqtD3DjuUWu7y32ULwoMPAMJCK6cUeYvBkCGYJRQ+txhWqdpJMTNCzmeAW/NI9GUc a1N3QKq6W+1bDDyH3EtkpoYdD7v1sxQQ0rseCoy1MIz17ouXC1NFQDpu0NqG7uOWd2t5 BJi+pviyaOEX/iR1qq945x+sVsAGL5vC8AHhogJ87l6LKM7ZykqkS18P+Ph7p4FTkpfe pB7NQ9FcUXrJX5XPtI6ESW2zVypw0QpbfX5oUgxGcKy1VEMOVrhe1hgrnrySAZhVPsLp aeIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=W0vkLVpbGC8iplakE5HggLPIkZErAF5DCoPm30qoMME=; b=UnktevNx4JWdWBdMw3LcCmhQgplNTp4TYPgMrD8bB4ERaosDAfRbOU/RMLgyLfi01X +uKQmIJRNg9FdyStGC1x0VdezNQoIDqhxEgCwtfxX+wkFgMsA9c0XMFwZKBQ2v0u5gBN 49X2HYwMVa4W4X7I0aS5ACmxiyDXRJyznHbiJcA/4JmOmnICB9Fulyy2K6kiPM7qz8ze ZK2HYgVXVpWPVaazABVP1F1ptWoxz/3ZIFKb4H0dgYJlhg8VHoCrpTg00wfbT3WAnchk pLuOGdz28hJXGBD9WCXX9WosBTZJRRRA7vDG/+CF2QMfHWE8q/S3PQQDn8QDVfwxP0xz SNeg== X-Gm-Message-State: AOAM531SSsXCQGS9YxOQQWPD5yBquw+NrRdjz9qJpvBbErRtBVwgjsbb EBJnb+s88PY8KGYRi0Ig4ftzjzKE91tcXw== X-Google-Smtp-Source: ABdhPJwcC+7r/BRlWqHVxNeh8+lU9qWAhlD7gPJgbix1R1hHW98Bg+O+QnDgHFN5N1/Yhz/IL8r4iQ== X-Received: by 2002:a17:902:8b81:: with SMTP id ay1mr577076plb.304.1594660811390; Mon, 13 Jul 2020 10:20:11 -0700 (PDT) Received: from localhost.localdomain ([122.172.60.196]) by smtp.gmail.com with ESMTPSA id 17sm168440pjl.30.2020.07.13.10.20.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jul 2020 10:20:10 -0700 (PDT) From: gautamramk@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Mon, 13 Jul 2020 22:50:00 +0530 Message-Id: <20200713172002.14158-2-gautamramk@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200713172002.14158-1-gautamramk@gmail.com> References: <20200713172002.14158-1-gautamramk@gmail.com> Subject: [FFmpeg-devel] [PATCH 2/4] libavcodec/jpeg2000 Fix PCRL Progression Order check X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Gautam Ramakrishnan MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Gautam Ramakrishnan The PCRL progression checks were incomplete. This patch modifes completes the check. Tested on p1_05.j2k. --- libavcodec/jpeg2000dec.c | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/libavcodec/jpeg2000dec.c b/libavcodec/jpeg2000dec.c index 48ca1c37a5..adaac192e9 100644 --- a/libavcodec/jpeg2000dec.c +++ b/libavcodec/jpeg2000dec.c @@ -1465,23 +1465,30 @@ static int jpeg2000_decode_packets_po_iteration(Jpeg2000DecoderContext *s, Jpeg2 Jpeg2000Component *comp = tile->comp + compno; Jpeg2000CodingStyle *codsty = tile->codsty + compno; Jpeg2000QuantStyle *qntsty = tile->qntsty + compno; - int xc = x / s->cdx[compno]; - int yc = y / s->cdy[compno]; + + if (!s->cdx[compno] || !s->cdy[compno]) + return AVERROR_INVALIDDATA; for (reslevelno = RSpoc; reslevelno < FFMIN(codsty->nreslevels, REpoc); reslevelno++) { unsigned prcx, prcy; uint8_t reducedresno = codsty->nreslevels - 1 -reslevelno; // ==> N_L - r Jpeg2000ResLevel *rlevel = comp->reslevel + reslevelno; + int trx0, try0; - if (yc % (1LL << (rlevel->log2_prec_height + reducedresno)) && y != tile->coord[1][0]) //FIXME this is a subset of the check - continue; + trx0 = ff_jpeg2000_ceildiv(tile->coord[0][0], s->cdx[compno] << reducedresno); + try0 = ff_jpeg2000_ceildiv(tile->coord[1][0], s->cdy[compno] << reducedresno); - if (xc % (1LL << (rlevel->log2_prec_width + reducedresno)) && x != tile->coord[0][0]) //FIXME this is a subset of the check - continue; + if (!(y % ((uint64_t)s->cdy[compno] << (rlevel->log2_prec_height + reducedresno)) == 0 || + (y == tile->coord[1][0] && (try0 << reducedresno) % (1U << (reducedresno + rlevel->log2_prec_height))))) + continue; + + if (!(x % ((uint64_t)s->cdx[compno] << (rlevel->log2_prec_width + reducedresno)) == 0 || + (x == tile->coord[0][0] && (trx0 << reducedresno) % (1U << (reducedresno + rlevel->log2_prec_width))))) + continue; // check if a precinct exists - prcx = ff_jpeg2000_ceildivpow2(xc, reducedresno) >> rlevel->log2_prec_width; - prcy = ff_jpeg2000_ceildivpow2(yc, reducedresno) >> rlevel->log2_prec_height; + prcx = ff_jpeg2000_ceildiv(x, s->cdx[compno] << reducedresno) >> rlevel->log2_prec_width; + prcy = ff_jpeg2000_ceildiv(y, s->cdy[compno] << reducedresno) >> rlevel->log2_prec_height; prcx -= ff_jpeg2000_ceildivpow2(comp->coord_o[0][0], reducedresno) >> rlevel->log2_prec_width; prcy -= ff_jpeg2000_ceildivpow2(comp->coord_o[1][0], reducedresno) >> rlevel->log2_prec_height;