From patchwork Mon Jul 13 17:20:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gautam Ramakrishnan X-Patchwork-Id: 20997 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 1D32544A041 for ; Mon, 13 Jul 2020 20:48:19 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id ECF3568A0BE; Mon, 13 Jul 2020 20:48:18 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oi1-f195.google.com (mail-oi1-f195.google.com [209.85.167.195]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0B1D9688105 for ; Mon, 13 Jul 2020 20:48:13 +0300 (EEST) Received: by mail-oi1-f195.google.com with SMTP id x83so11654401oif.10 for ; Mon, 13 Jul 2020 10:48:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=GxxeNT+L6uSZsTtk3vnt4FajWyZX3oi41AD6Ju9SDiw=; b=A1m1W4eeXULMqyOtEU9b/HMuDTHZH5GZskg24iLGeWACAzO89vgBQgKa/lldujOlet VhjzlUNrCEH3W7Sm8GzOjF0ouFz8HEBaHJEJu/XDTqCCjZtKFuWLzE4vS4uqVKjfIn0D Q0C+7J7KPJS7ZLYI8bDLvmRfqUsMIitpMIX3r28RadGNa5y6kmpXUFWfBts8NqYLu52M sx+n/xWF6ixkYaP5rAnt7GAEgyQTz8JmV4beIUMa401n+RMYKGZ8yaokKT1ffMZG4+SF 6Ktn39qLOdB12wGBdM6psAn/9C0nSZrz0Tb3buXqfUGarcSrL4SC8n8EX4FsK7hwGBu1 WIlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=GxxeNT+L6uSZsTtk3vnt4FajWyZX3oi41AD6Ju9SDiw=; b=aOt6L3G1T49l231+76om4la+sxtFloh38DhiNI9W+kYNMcly83tv0Omg521ggJ+0zm BJmLGpmHXkkDAxDh/zO27bwQffaqzeIwLFjrbqk4R97xS+ihz/eDp2hCz6F+HXdjChlk zICBt8auOPrdwkJl0LgovDsp6wtAKEewfk5sJRbtwwclQavzt+T7O75TO+YyYl2gIvzF llWcooVpD0kkuoxCNMUwaFhJnHHR+LDOlX+5KIixqXzmUPQCFyrBjeDv/ANTL6TDu199 BbagROGnaM3Cj1v2y5yB9yig58NFswszzHEbdXIvtj5ET3GvzRNMYdr+orC4imKG3u7a enEg== X-Gm-Message-State: AOAM530m/uxqrJR5JLkRbSzJa1xTP4LM0jpUxn1Ebsi+NaaWmJq263sE FgZwbkvJbPYLChKWIwmNxAjLIyphxLGkOQ== X-Google-Smtp-Source: ABdhPJyBUxcs+473YVrYW3Y3f2zit25x1GGGb79mbPZ/bwMxlGEV0FMtPv667fQ06h+AC8QjpmDzoA== X-Received: by 2002:a17:90b:918:: with SMTP id bo24mr400692pjb.191.1594660813349; Mon, 13 Jul 2020 10:20:13 -0700 (PDT) Received: from localhost.localdomain ([122.172.60.196]) by smtp.gmail.com with ESMTPSA id 17sm168440pjl.30.2020.07.13.10.20.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jul 2020 10:20:12 -0700 (PDT) From: gautamramk@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Mon, 13 Jul 2020 22:50:01 +0530 Message-Id: <20200713172002.14158-3-gautamramk@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200713172002.14158-1-gautamramk@gmail.com> References: <20200713172002.14158-1-gautamramk@gmail.com> Subject: [FFmpeg-devel] [PATCH 3/4] libavcodec/jpeg2000 Fix RPCL Progression order check X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Gautam Ramakrishnan MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Gautam Ramakrishnan The RPCL progression order check was incomplete. This patch completes the check. Tested on p1_07.j2k. --- libavcodec/jpeg2000dec.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/libavcodec/jpeg2000dec.c b/libavcodec/jpeg2000dec.c index adaac192e9..5ea6fd0b9a 100644 --- a/libavcodec/jpeg2000dec.c +++ b/libavcodec/jpeg2000dec.c @@ -1395,22 +1395,28 @@ static int jpeg2000_decode_packets_po_iteration(Jpeg2000DecoderContext *s, Jpeg2 uint8_t reducedresno = codsty->nreslevels - 1 -reslevelno; // ==> N_L - r Jpeg2000ResLevel *rlevel = comp->reslevel + reslevelno; unsigned prcx, prcy; + int trx0, try0; - int xc = x / s->cdx[compno]; - int yc = y / s->cdy[compno]; + if (!s->cdx[compno] || !s->cdy[compno]) + return AVERROR_INVALIDDATA; + + trx0 = ff_jpeg2000_ceildiv(tile->coord[0][0], s->cdx[compno] << reducedresno); + try0 = ff_jpeg2000_ceildiv(tile->coord[1][0], s->cdy[compno] << reducedresno); if (reslevelno >= codsty->nreslevels) continue; - if (yc % (1LL << (rlevel->log2_prec_height + reducedresno)) && y != tile->coord[1][0]) //FIXME this is a subset of the check + if (!(y % ((uint64_t)s->cdy[compno] << (rlevel->log2_prec_height + reducedresno)) == 0 || + (y == tile->coord[1][0] && (try0 << reducedresno) % (1U << (reducedresno + rlevel->log2_prec_height))))) continue; - if (xc % (1LL << (rlevel->log2_prec_width + reducedresno)) && x != tile->coord[0][0]) //FIXME this is a subset of the check + if (!(x % ((uint64_t)s->cdx[compno] << (rlevel->log2_prec_width + reducedresno)) == 0 || + (x == tile->coord[0][0] && (trx0 << reducedresno) % (1U << (reducedresno + rlevel->log2_prec_width))))) continue; // check if a precinct exists - prcx = ff_jpeg2000_ceildivpow2(xc, reducedresno) >> rlevel->log2_prec_width; - prcy = ff_jpeg2000_ceildivpow2(yc, reducedresno) >> rlevel->log2_prec_height; + prcx = ff_jpeg2000_ceildiv(x, s->cdx[compno] << reducedresno) >> rlevel->log2_prec_width; + prcy = ff_jpeg2000_ceildiv(y, s->cdy[compno] << reducedresno) >> rlevel->log2_prec_height; prcx -= ff_jpeg2000_ceildivpow2(comp->coord_o[0][0], reducedresno) >> rlevel->log2_prec_width; prcy -= ff_jpeg2000_ceildivpow2(comp->coord_o[1][0], reducedresno) >> rlevel->log2_prec_height;