From patchwork Tue Jun 14 14:39:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Tomas_H=C3=A4rdin?= X-Patchwork-Id: 36222 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1a22:b0:84:42e0:ad30 with SMTP id cj34csp1113066pzb; Tue, 14 Jun 2022 07:39:14 -0700 (PDT) X-Google-Smtp-Source: AGRyM1sKWoQOlaL/4rzbStv76pc3HSl+AdGLaS/V1JizvAp532oPslO/Blnfd+s9vJeactyAFcBu X-Received: by 2002:a17:906:5e59:b0:711:2d24:5152 with SMTP id b25-20020a1709065e5900b007112d245152mr4622604eju.71.1655217553660; Tue, 14 Jun 2022 07:39:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1655217553; cv=none; d=google.com; s=arc-20160816; b=cUJGMUgZqIxwxY6lJzavilw5b2jSkO5BRufOFCK2dmlTcqWc+TyP9gPTigpeOjz26D PaOCVV50wrBYyMilELupMn998z78ehNzNSEB6/XyhKuRnrhMCuCO9rPkVdarg03a9qaB 68FBgQ82nKfWc6xl9lub1DYk0BzC1U071oU6GWUE+yog+i+Jh9H1UiEghLv8WaKDTe6Y Plw5mmxvVryqQ4ZjdJTkg2MmgyTpObVP/Ovxf3PxEQ+D5IIrU0hGVEyhNx/xeBhGwIGy HebmA9+bm28TmSrqXNrcjJ4poNBqV0I6IMtE1Ij403pIj7GTegNs8Wpd7junR+W9OCWP 2fnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:user-agent:date:to:from:message-id:dkim-signature :dkim-signature:delivered-to; bh=ZcDheWvRk4Mdac3/Q42YbA0PjZFPfSl+5+xlPXc9gkM=; b=LX7tJkN1oB01yIfnBafGt5abJVw2D5D9lxueLPrqwSwxOKr3UZ0Zl7KLe3io8tS3Wh OdMmULWzm87NAMpF6AM8RbSOM4uH/WjQmwqUrWrZsVA76/EcKTzHkWKXcL7ua9IlP1P8 +gr6Jn+17v71IGYIWbzgs9IUdwT2Qv2CSJVgAKTrs4ceyciBt9+yIcHeh8j8YCoj8wyv r0mApcN/bg0dwt7fPeQ7D/KdCdXiRIQp69PpEEnh3f3Tyuqp1vrMbPTUBQlxtrgByoce kaj6OJJryNpI6EwLr3MG1KLKYhnBa3DmBxS5HOpqObGVextvD5imiGELZUqP2Y58Kypm eGqQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@acc.umu.se header.s=mail1 header.b=Ma236UBT; dkim=neutral (body hash did not verify) header.i=@acc.umu.se header.s=mail1 header.b=Ur96UjzA; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=acc.umu.se Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id q20-20020a056402519400b0042600959715si13881550edd.204.2022.06.14.07.39.13; Tue, 14 Jun 2022 07:39:13 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@acc.umu.se header.s=mail1 header.b=Ma236UBT; dkim=neutral (body hash did not verify) header.i=@acc.umu.se header.s=mail1 header.b=Ur96UjzA; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=acc.umu.se Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2164568B647; Tue, 14 Jun 2022 17:39:10 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.acc.umu.se (mail.acc.umu.se [130.239.18.156]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2CBFF68B194 for ; Tue, 14 Jun 2022 17:39:03 +0300 (EEST) Received: from localhost (localhost.localdomain [127.0.0.1]) by amavisd-new (Postfix) with ESMTP id 3A0A744DCB for ; Tue, 14 Jun 2022 16:39:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=acc.umu.se; s=mail1; t=1655217542; bh=Pus9GxSPzoGwuHuJRUx9JMCn+j9BxbZfSxUlhOwV9C8=; h=Subject:From:To:Date:From; b=Ma236UBTdfPLvzIa0hQiZHfT04uNDonLipv26ZoAaBy5DFS4ydOqGd10ac8U/1VXU UjVHGllP+YoG5BCSE7FJLaQPvFXV+otD43ftVqAFGrg6OXIxscFB3LGkfzEfrh47Pd qHm65LkwtyEbQrEUknDY97eDlduE4SR4SJXJMkkYYLsC2FFJbPvZ89kNvDDW4I8blu Sb/IzcQKt04sqGYA0lU2xbtMbvtX8zh8tMx0Kee/6shKzCKXHthkc2CzWr7X43iUDH kp9X050HAMOT0P5MkuDUn9Oqx5Xek8BQo/9hmAvej9cDcmo8L4dL6vToY4O75gjApx g1dxpNEmhtxcg== Received: from debian.lan (unknown [IPv6:2a00:66c0:a::72c]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: tjoppen) by mail.acc.umu.se (Postfix) with ESMTPSA id 6463644DC6 for ; Tue, 14 Jun 2022 16:39:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=acc.umu.se; s=mail1; t=1655217541; bh=Pus9GxSPzoGwuHuJRUx9JMCn+j9BxbZfSxUlhOwV9C8=; h=Subject:From:To:Date:From; b=Ur96UjzAH9MaHzCJ0qV1AyMcuVYjPtUg5WRNKdWq7mGctLGyGnveQBv+Hr1uM0tGy vekNQlb94EyHYPA3Bz+EreA47mJNfVKA5ghqyIgEj5IWCVGgbtn+dEuDDzpfU4wuOj +qAaVOf8P9afHepxgcsAl98k5/qVX/2Mz6bFNhY19Wo99EQF4eWTJyGm0gkt5e1OpZ qKUcx8jBQwTIo7TlYGY7w+Hlx7VTCyAlOrA1OeW5Pa5NOXSEBP40kmg4zu0mzfjrhv GrP2zdI0jGx8ZPrzXcbhYYlUENt6INOUisoe1rAYjycgN189ggLZsz5rL7DyqYF4bS 9pZeWY8bCjEVQ== Message-ID: <10ec51ef44325c2de6d5de7b994a9b6c8eb5e3a2.camel@acc.umu.se> From: Tomas =?iso-8859-1?q?H=E4rdin?= To: FFmpeg development discussions and patches Date: Tue, 14 Jun 2022 16:39:00 +0200 User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 01/13] lavc/jpeg2000dec: Finer granularity threading X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: GQazqJC7Gq7A Patch 12 in this series is optional since it's just me getting the speed up on a specific machine /Tomas From 115aa26c343419e81c1b5ba0bfdb1615cbec27e9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tomas=20H=C3=A4rdin?= Date: Fri, 10 Jun 2022 14:10:02 +0200 Subject: [PATCH 01/13] lavc/jpeg2000dec: Finer granularity threading Decoding and dequant is now threaded on codeblock level. IDWT is threaded on component level. MCT and write_frame() remain threaded on tile level. This brings lossless 4K J2K with -lowres 2 -thread_type slice -threads 96 on an AMD EPYC 7R32 from 4.8 fps (177% CPU) to 31 fps (1284% CPU). --- libavcodec/jpeg2000dec.c | 196 ++++++++++++++++++++++++++++----------- 1 file changed, 142 insertions(+), 54 deletions(-) diff --git a/libavcodec/jpeg2000dec.c b/libavcodec/jpeg2000dec.c index 92966b11f5..d9754fc50e 100644 --- a/libavcodec/jpeg2000dec.c +++ b/libavcodec/jpeg2000dec.c @@ -92,6 +92,15 @@ typedef struct Jpeg2000Tile { int coord[2][2]; // border coordinates {{x0, x1}, {y0, y1}} } Jpeg2000Tile; +typedef struct Jpeg2000IdwtThread { + int cb_start, cb_end; +} Jpeg2000IdwtThread; + +typedef struct Jpeg2000CodeblockThread { + int tileno, compno, reslevelno, bandno, precno, cblkno; + int coded; +} Jpeg2000CodeblockThread; + typedef struct Jpeg2000DecoderContext { AVClass *class; AVCodecContext *avctx; @@ -136,6 +145,11 @@ typedef struct Jpeg2000DecoderContext { /*options parameters*/ int reduction_factor; + + Jpeg2000IdwtThread *idwt; + unsigned int idwt_size; + Jpeg2000CodeblockThread *cb; + unsigned int cb_size; } Jpeg2000DecoderContext; /* get_bits functions for JPEG2000 packet bitstream @@ -1937,54 +1951,33 @@ static inline void roi_scale_cblk(Jpeg2000Cblk *cblk, } } -static inline void tile_codeblocks(Jpeg2000DecoderContext *s, Jpeg2000Tile *tile) +static int jpeg2000_decode_cb(AVCodecContext *avctx, void *td, + int jobnr, int threadnr) { Jpeg2000T1Context t1; - - int compno, reslevelno, bandno; - - /* Loop on tile components */ - for (compno = 0; compno < s->ncomponents; compno++) { - Jpeg2000Component *comp = tile->comp + compno; - Jpeg2000CodingStyle *codsty = tile->codsty + compno; - int coded = 0; - - t1.stride = (1<log2_cblk_width) + 2; - - /* Loop on resolution levels */ - for (reslevelno = 0; reslevelno < codsty->nreslevels2decode; reslevelno++) { - Jpeg2000ResLevel *rlevel = comp->reslevel + reslevelno; - /* Loop on bands */ - for (bandno = 0; bandno < rlevel->nbands; bandno++) { - int nb_precincts, precno; - Jpeg2000Band *band = rlevel->band + bandno; - int cblkno = 0, bandpos; - - bandpos = bandno + (reslevelno > 0); - - if (band->coord[0][0] == band->coord[0][1] || - band->coord[1][0] == band->coord[1][1]) - continue; - - nb_precincts = rlevel->num_precincts_x * rlevel->num_precincts_y; - /* Loop on precincts */ - for (precno = 0; precno < nb_precincts; precno++) { - Jpeg2000Prec *prec = band->prec + precno; - - /* Loop on codeblocks */ - for (cblkno = 0; - cblkno < prec->nb_codeblocks_width * prec->nb_codeblocks_height; - cblkno++) { - int x, y; - Jpeg2000Cblk *cblk = prec->cblk + cblkno; - int ret = decode_cblk(s, codsty, &t1, cblk, + Jpeg2000DecoderContext *s = avctx->priv_data; + Jpeg2000CodeblockThread *cb = s->cb + jobnr; + Jpeg2000Tile *tile = s->tile + cb->tileno; + Jpeg2000Component *comp = tile->comp + cb->compno; + Jpeg2000CodingStyle *codsty = tile->codsty + cb->compno; + Jpeg2000ResLevel *rlevel = comp->reslevel + cb->reslevelno; + Jpeg2000Band *band = rlevel->band + cb->bandno; + Jpeg2000Prec *prec = band->prec + cb->precno; + Jpeg2000Cblk *cblk = prec->cblk + cb->cblkno; + int ret, x, y, bandpos = cb->bandno + (cb->reslevelno > 0); + + t1.stride = (1<log2_cblk_width) + 2; + cb->coded = 0; + + ret = decode_cblk(s, codsty, &t1, cblk, cblk->coord[0][1] - cblk->coord[0][0], cblk->coord[1][1] - cblk->coord[1][0], bandpos, comp->roi_shift); if (ret) - coded = 1; + cb->coded = 1; else - continue; + return 0; + x = cblk->coord[0][0] - band->coord[0][0]; y = cblk->coord[1][0] - band->coord[1][0]; @@ -1996,16 +1989,28 @@ static inline void tile_codeblocks(Jpeg2000DecoderContext *s, Jpeg2000Tile *tile dequantization_int_97(x, y, cblk, comp, &t1, band); else dequantization_int(x, y, cblk, comp, &t1, band); - } /* end cblk */ - } /*end prec */ - } /* end band */ - } /* end reslevel */ - /* inverse DWT */ - if (coded) + return 0; +} + +static int jpeg2000_idwt(AVCodecContext *avctx, void *td, + int jobnr, int threadnr) +{ + Jpeg2000DecoderContext *s = avctx->priv_data; + Jpeg2000IdwtThread *idwt = s->idwt + jobnr; + Jpeg2000Tile *tile = s->tile + jobnr / s->ncomponents; + int compno = jobnr % s->ncomponents; + Jpeg2000Component *comp = tile->comp + compno; + Jpeg2000CodingStyle *codsty = tile->codsty + compno; + + for (int i = idwt->cb_start; i < idwt->cb_end; i++) { + if (s->cb[i].coded) { ff_dwt_decode(&comp->dwt, codsty->transform == FF_DWT97 ? (void*)comp->f_data : (void*)comp->i_data); + break; + } + } - } /*end comp */ + return 0; } #define WRITE_FRAME(D, PIXEL) \ @@ -2075,15 +2080,13 @@ WRITE_FRAME(16, uint16_t) #undef WRITE_FRAME -static int jpeg2000_decode_tile(AVCodecContext *avctx, void *td, - int jobnr, int threadnr) +static int jpeg2000_mct_write_frame(AVCodecContext *avctx, void *td, + int jobnr, int threadnr) { Jpeg2000DecoderContext *s = avctx->priv_data; AVFrame *picture = td; Jpeg2000Tile *tile = s->tile + jobnr; - tile_codeblocks(s, tile); - /* inverse MCT transformation */ if (tile->codsty[0].mct) mct_decode(s, tile); @@ -2473,11 +2476,80 @@ static av_cold int jpeg2000_decode_init(AVCodecContext *avctx) return 0; } +static int jpeg2000_setup_cbs(Jpeg2000DecoderContext *s, int *cbs_out) +{ + if (s->numXtiles * s->numYtiles > INT_MAX/sizeof(*s->idwt)/s->ncomponents) + return AVERROR(ENOMEM); + + av_fast_malloc(&s->idwt, &s->idwt_size, s->numXtiles * s->numYtiles * s->ncomponents * sizeof(*s->idwt)); + if (!s->idwt) + return AVERROR(ENOMEM); + + for (int pass = 0; pass < 2; pass++) { + int cbs = 0; + for (int tileno = 0; tileno < s->numXtiles * s->numYtiles; tileno++) { + for (int compno = 0; compno < s->ncomponents; compno++) { + Jpeg2000Tile *tile = s->tile + tileno; + Jpeg2000Component *comp = tile->comp + compno; + Jpeg2000CodingStyle *codsty = tile->codsty + compno; + Jpeg2000IdwtThread *idwt = s->idwt + compno + tileno * s->ncomponents; + + idwt->cb_start = cbs; + + for (int reslevelno = 0; reslevelno < codsty->nreslevels2decode; reslevelno++) { + Jpeg2000ResLevel *rlevel = comp->reslevel + reslevelno; + for (int bandno = 0; bandno < rlevel->nbands; bandno++) { + int nb_precincts = rlevel->num_precincts_x * rlevel->num_precincts_y; + Jpeg2000Band *band = rlevel->band + bandno; + + if (band->coord[0][0] == band->coord[0][1] || + band->coord[1][0] == band->coord[1][1]) + continue; + + for (int precno = 0; precno < nb_precincts; precno++) { + Jpeg2000Prec *prec = band->prec + precno; + int prec_cbs = prec->nb_codeblocks_width * prec->nb_codeblocks_height; + + if (cbs > INT_MAX - prec_cbs) + return AVERROR(ENOMEM); + + for (int cblkno = 0; cblkno < prec_cbs; cblkno++, cbs++) { + if (pass == 1) { + Jpeg2000CodeblockThread *cb = s->cb + cbs; + cb->tileno = tileno; + cb->compno = compno; + cb->reslevelno = reslevelno; + cb->bandno = bandno; + cb->precno = precno; + cb->cblkno = cblkno; + } + } + } + } + } + + idwt->cb_end = cbs; + } + } + + if (pass == 0) { + if (cbs > INT_MAX/sizeof(*s->cb)) + return AVERROR(ENOMEM); + av_fast_malloc(&s->cb, &s->cb_size, cbs*sizeof(*s->cb)); + if (!s->cb) + return AVERROR(ENOMEM); + } + + *cbs_out = cbs; + } + return 0; +} + static int jpeg2000_decode_frame(AVCodecContext *avctx, AVFrame *picture, int *got_frame, AVPacket *avpkt) { Jpeg2000DecoderContext *s = avctx->priv_data; - int ret; + int ret, cbs; s->avctx = avctx; bytestream2_init(&s->g, avpkt->data, avpkt->size); @@ -2535,7 +2607,12 @@ static int jpeg2000_decode_frame(AVCodecContext *avctx, AVFrame *picture, } } - avctx->execute2(avctx, jpeg2000_decode_tile, picture, NULL, s->numXtiles * s->numYtiles); + if ((ret = jpeg2000_setup_cbs(s, &cbs))) + goto end; + + avctx->execute2(avctx, jpeg2000_decode_cb, NULL, NULL, cbs); + avctx->execute2(avctx, jpeg2000_idwt, NULL, NULL, s->numXtiles * s->numYtiles * s->ncomponents); + avctx->execute2(avctx, jpeg2000_mct_write_frame, picture, NULL, s->numXtiles * s->numYtiles); jpeg2000_dec_cleanup(s); @@ -2554,6 +2631,16 @@ end: return ret; } +static av_cold int jpeg2000_decode_close(AVCodecContext *avctx) +{ + Jpeg2000DecoderContext *s = avctx->priv_data; + + av_freep(&s->idwt); + av_freep(&s->cb); + + return 0; +} + #define OFFSET(x) offsetof(Jpeg2000DecoderContext, x) #define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM @@ -2579,6 +2666,7 @@ const FFCodec ff_jpeg2000_decoder = { .priv_data_size = sizeof(Jpeg2000DecoderContext), .init = jpeg2000_decode_init, FF_CODEC_DECODE_CB(jpeg2000_decode_frame), + .close = jpeg2000_decode_close, .p.priv_class = &jpeg2000_class, .p.max_lowres = 5, .p.profiles = NULL_IF_CONFIG_SMALL(ff_jpeg2000_profiles), -- 2.30.2