From patchwork Tue Jul 14 16:43:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gautam Ramakrishnan X-Patchwork-Id: 21012 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 44FB844AD8E for ; Tue, 14 Jul 2020 19:43:46 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2AA1768B2DB; Tue, 14 Jul 2020 19:43:46 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f67.google.com (mail-pj1-f67.google.com [209.85.216.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 98C9368B2B1 for ; Tue, 14 Jul 2020 19:43:39 +0300 (EEST) Received: by mail-pj1-f67.google.com with SMTP id mn17so1863636pjb.4 for ; Tue, 14 Jul 2020 09:43:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=GxxeNT+L6uSZsTtk3vnt4FajWyZX3oi41AD6Ju9SDiw=; b=RQtewuwiZezfUZKwotMGFstJR0xP+KlyOkaqVoC4Enbg32Wt+Xuc2/lX0L0kGvSie0 RNbS+qrn7ByCX2WO+9j7/hdaPEBH2NNRnlSW6/jyxW16e0n/in2QSbZ0KKcfFA6HuIMV JNp8ZO1U8+DkidzyFKCDvyeK8j8iKvGn7uswxm+cQgrCQsln6Jaoj3c3OIj7GEdR4OKT VK1ExJDk6Ic9GotQaAzFX6CU0unQpWgggMpyZvR7uJe5UNeZQR01jKSzNR/Q0lWM2xpg VYz95Bf8XVKnYXguEpPmfcZ8cQuxqakZ+V5jbzzrocaVqN+aOni4IhPr+dHOsEc43Lea fSkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=GxxeNT+L6uSZsTtk3vnt4FajWyZX3oi41AD6Ju9SDiw=; b=b1gGMWuyRn8b1meKCDm+Cj8WZmt/eaMFBNNW+CY6qSnw+2OFx+CmvorTN/uLiWR1iz sPX9H5bIdf1uYSQT8LASP5wZs+Rc3PQz/L1f1CMWhzouxsqF+iYdHSdKKG7+K1NJSlcQ nNDMarP4p3OLlH4RMKr9M7NO+thwfeH/+Ih8CCK7v18pBHJE98rFhUumDZUl0Zo07sH2 KQIUAThS7kRVKih1qU7TnVIDJCbZvMEo7/iQh8jejIrreY1pxu1z3R5pSO5o0zSH8Byd d6Zm8DVT4ntu6ayP6lDPzxoufn+98xdwurhbhg4e16PvM+j3N4tbcgy60ziC0b32Cd0a hKuA== X-Gm-Message-State: AOAM531B0SALcB0WqZOG3f55rwlxG6zGgsnENCJo9OVJzwZyfCe2Ry+T dN3J4ITGJXsWzKZYMF7LAJn9QKP9vt8v7g== X-Google-Smtp-Source: ABdhPJzMms59hgkZFFQRuOd93STFwiz5oj6USng10ATCEM2p9GBdB3tILUcAxd+ck8e8pPbkEKX6wA== X-Received: by 2002:a17:90b:3010:: with SMTP id hg16mr5620247pjb.69.1594745017621; Tue, 14 Jul 2020 09:43:37 -0700 (PDT) Received: from localhost.localdomain ([122.172.60.196]) by smtp.gmail.com with ESMTPSA id c1sm3295983pje.9.2020.07.14.09.43.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jul 2020 09:43:37 -0700 (PDT) From: gautamramk@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 14 Jul 2020 22:13:15 +0530 Message-Id: <20200714164316.32120-3-gautamramk@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200714164316.32120-1-gautamramk@gmail.com> References: <20200714164316.32120-1-gautamramk@gmail.com> Subject: [FFmpeg-devel] [PATCH v2 3/4] libavcodec/jpeg2000 Fix RPCL Progression order check X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Gautam Ramakrishnan MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Gautam Ramakrishnan The RPCL progression order check was incomplete. This patch completes the check. Tested on p1_07.j2k. --- libavcodec/jpeg2000dec.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/libavcodec/jpeg2000dec.c b/libavcodec/jpeg2000dec.c index adaac192e9..5ea6fd0b9a 100644 --- a/libavcodec/jpeg2000dec.c +++ b/libavcodec/jpeg2000dec.c @@ -1395,22 +1395,28 @@ static int jpeg2000_decode_packets_po_iteration(Jpeg2000DecoderContext *s, Jpeg2 uint8_t reducedresno = codsty->nreslevels - 1 -reslevelno; // ==> N_L - r Jpeg2000ResLevel *rlevel = comp->reslevel + reslevelno; unsigned prcx, prcy; + int trx0, try0; - int xc = x / s->cdx[compno]; - int yc = y / s->cdy[compno]; + if (!s->cdx[compno] || !s->cdy[compno]) + return AVERROR_INVALIDDATA; + + trx0 = ff_jpeg2000_ceildiv(tile->coord[0][0], s->cdx[compno] << reducedresno); + try0 = ff_jpeg2000_ceildiv(tile->coord[1][0], s->cdy[compno] << reducedresno); if (reslevelno >= codsty->nreslevels) continue; - if (yc % (1LL << (rlevel->log2_prec_height + reducedresno)) && y != tile->coord[1][0]) //FIXME this is a subset of the check + if (!(y % ((uint64_t)s->cdy[compno] << (rlevel->log2_prec_height + reducedresno)) == 0 || + (y == tile->coord[1][0] && (try0 << reducedresno) % (1U << (reducedresno + rlevel->log2_prec_height))))) continue; - if (xc % (1LL << (rlevel->log2_prec_width + reducedresno)) && x != tile->coord[0][0]) //FIXME this is a subset of the check + if (!(x % ((uint64_t)s->cdx[compno] << (rlevel->log2_prec_width + reducedresno)) == 0 || + (x == tile->coord[0][0] && (trx0 << reducedresno) % (1U << (reducedresno + rlevel->log2_prec_width))))) continue; // check if a precinct exists - prcx = ff_jpeg2000_ceildivpow2(xc, reducedresno) >> rlevel->log2_prec_width; - prcy = ff_jpeg2000_ceildivpow2(yc, reducedresno) >> rlevel->log2_prec_height; + prcx = ff_jpeg2000_ceildiv(x, s->cdx[compno] << reducedresno) >> rlevel->log2_prec_width; + prcy = ff_jpeg2000_ceildiv(y, s->cdy[compno] << reducedresno) >> rlevel->log2_prec_height; prcx -= ff_jpeg2000_ceildivpow2(comp->coord_o[0][0], reducedresno) >> rlevel->log2_prec_width; prcy -= ff_jpeg2000_ceildivpow2(comp->coord_o[1][0], reducedresno) >> rlevel->log2_prec_height;