From patchwork Wed Sep 28 10:04:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Tomas_H=C3=A4rdin?= X-Patchwork-Id: 38414 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b1c:b0:96:9ee8:5cfd with SMTP id c28csp97529pzh; Wed, 28 Sep 2022 03:04:46 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5Dbz2kmh+/26RsPIUXIqpddbVhE4Mh/V87/DUanmCVb/6cD6FmbCVnVI7AjO6lqMEjdin/ X-Received: by 2002:a17:906:cc4e:b0:77c:b7a:9de6 with SMTP id mm14-20020a170906cc4e00b0077c0b7a9de6mr26205007ejb.531.1664359486024; Wed, 28 Sep 2022 03:04:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1664359486; cv=none; d=google.com; s=arc-20160816; b=C4Fgv7nWtugvLJuHmcW/Yf+JCSf1tcNRHBvzsNYCbz3ZdO4lkd7MIDBojmhfJl20z3 uV8Wy91EGy2b+RqwAbseQ3UHpt/pMPCBEiQel5qXXYN1l5a2AYl4F5SPpVun3GRzifU6 yBP93Snh+WJd9+Qekq92vTwqkY/IU1z78Q1S9yoQcKbCTRV/ZsB4Wl0jZ0l4wiAftGvN 0UxFUtnS7256WdjdZ8KHxnBZz4JSS6JzOIqcZ638OjzAPIcQ2/MZNDJek8evuNrUzzFy SGUmYD+fzxy2pZpY99W3KJTv+TQdGKR4YOkTbglKJgNPfOx0gQ6YNSqG+Fsm5nIhAMry WDoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:user-agent:date:to:from:message-id:delivered-to; bh=vF/iK6I1YWQHc0I777R9M+3Q3lqE/ccciQCcWJf7IBw=; b=CIPBY3EtMKm1qj5+xmaEtV8+/iTHUuFlg6waFnAEtLp9/SFZ+KfX9JPaCGkWepDTJc vPSY7dXLF2QMpmWMzXVXghEQCSkuGm5L80jhYvkPTPW0+ZMEnXCGlCE3MfmbTA023Vnx 6opilz5oOL12FaoVMDeTgXO5Az4UD7MaG3yBmDtTmly0Z7qIfrXrIZDTVPn/0bGE9dNo 1hEHXcJTyF+iBO7TDllxzDmaITQ+Imk03H8aSAXCT+SK7bT4OLwMwHjrefppp50WPBJA P7wArwrE37lfF2KKHyBtXQAUPWut7y3yiROs6fNBZ1hytC/S50ox+hxdcmhXhjP7UMlH DDTA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v5-20020a170906858500b007394b02127bsi3302322ejx.1002.2022.09.28.03.04.45; Wed, 28 Sep 2022 03:04:46 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BD9D468BBFF; Wed, 28 Sep 2022 13:04:42 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.frobbit.se (mail.frobbit.se [85.30.129.176]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 73A9168BB5D for ; Wed, 28 Sep 2022 13:04:35 +0300 (EEST) Received: from debian.lan (unknown [IPv6:2a00:66c0:a::72c]) by mail.frobbit.se (Postfix) with ESMTPSA id D4A2420403 for ; Wed, 28 Sep 2022 12:04:34 +0200 (CEST) Message-ID: <65e79fe701374868bb2f4b70ce8fd220938e2e86.camel@haerdin.se> From: Tomas =?iso-8859-1?q?H=E4rdin?= To: ffmpeg-devel Date: Wed, 28 Sep 2022 12:04:33 +0200 User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 01/11] lavc/jpeg2000dec: Finer granularity threading X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 3y/5dOL6H66k Hi This patchset depends both on my execute2() patchset and on av_realloc*_array_reuse(). The performance numbers aren't fully up to date, but I expect they won't have changed much. /Tomas From 9f70f673b2977e969c2c6df51d7a2b7d73302f08 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tomas=20H=C3=A4rdin?= Date: Fri, 10 Jun 2022 14:10:02 +0200 Subject: [PATCH 01/11] lavc/jpeg2000dec: Finer granularity threading Decoding and dequant is now threaded on codeblock level. IDWT is threaded on component level. MCT and write_frame() remain threaded on tile level. This brings lossless 4K J2K with -lowres 2 -thread_type slice -threads 96 on an AMD EPYC 7R32 from 4.8 fps (177% CPU) to 31 fps (1284% CPU). --- libavcodec/jpeg2000dec.c | 195 ++++++++++++++++++++++++++++----------- 1 file changed, 141 insertions(+), 54 deletions(-) diff --git a/libavcodec/jpeg2000dec.c b/libavcodec/jpeg2000dec.c index c2b81ec103..e823ae58ec 100644 --- a/libavcodec/jpeg2000dec.c +++ b/libavcodec/jpeg2000dec.c @@ -92,6 +92,15 @@ typedef struct Jpeg2000Tile { int coord[2][2]; // border coordinates {{x0, x1}, {y0, y1}} } Jpeg2000Tile; +typedef struct Jpeg2000IdwtThread { + int cb_start, cb_end; +} Jpeg2000IdwtThread; + +typedef struct Jpeg2000CodeblockThread { + int tileno, compno, reslevelno, bandno, precno, cblkno; + int coded; +} Jpeg2000CodeblockThread; + typedef struct Jpeg2000DecoderContext { AVClass *class; AVCodecContext *avctx; @@ -136,6 +145,11 @@ typedef struct Jpeg2000DecoderContext { /*options parameters*/ int reduction_factor; + + Jpeg2000IdwtThread *idwt; + size_t idwt_allocated; + Jpeg2000CodeblockThread *cb; + size_t cb_allocated; } Jpeg2000DecoderContext; /* get_bits functions for JPEG2000 packet bitstream @@ -1941,54 +1955,33 @@ static inline void roi_scale_cblk(Jpeg2000Cblk *cblk, } } -static inline void tile_codeblocks(const Jpeg2000DecoderContext *s, Jpeg2000Tile *tile) +static int jpeg2000_decode_cb(AVCodecContext *avctx, void *td, + int jobnr, int threadnr) { Jpeg2000T1Context t1; - - int compno, reslevelno, bandno; - - /* Loop on tile components */ - for (compno = 0; compno < s->ncomponents; compno++) { - Jpeg2000Component *comp = tile->comp + compno; - Jpeg2000CodingStyle *codsty = tile->codsty + compno; - int coded = 0; - - t1.stride = (1<log2_cblk_width) + 2; - - /* Loop on resolution levels */ - for (reslevelno = 0; reslevelno < codsty->nreslevels2decode; reslevelno++) { - Jpeg2000ResLevel *rlevel = comp->reslevel + reslevelno; - /* Loop on bands */ - for (bandno = 0; bandno < rlevel->nbands; bandno++) { - int nb_precincts, precno; - Jpeg2000Band *band = rlevel->band + bandno; - int cblkno = 0, bandpos; - - bandpos = bandno + (reslevelno > 0); - - if (band->coord[0][0] == band->coord[0][1] || - band->coord[1][0] == band->coord[1][1]) - continue; - - nb_precincts = rlevel->num_precincts_x * rlevel->num_precincts_y; - /* Loop on precincts */ - for (precno = 0; precno < nb_precincts; precno++) { - Jpeg2000Prec *prec = band->prec + precno; - - /* Loop on codeblocks */ - for (cblkno = 0; - cblkno < prec->nb_codeblocks_width * prec->nb_codeblocks_height; - cblkno++) { - int x, y; - Jpeg2000Cblk *cblk = prec->cblk + cblkno; - int ret = decode_cblk(s, codsty, &t1, cblk, + const Jpeg2000DecoderContext *s = avctx->priv_data; + Jpeg2000CodeblockThread *cb = s->cb + jobnr; + Jpeg2000Tile *tile = s->tile + cb->tileno; + Jpeg2000Component *comp = tile->comp + cb->compno; + Jpeg2000CodingStyle *codsty = tile->codsty + cb->compno; + Jpeg2000ResLevel *rlevel = comp->reslevel + cb->reslevelno; + Jpeg2000Band *band = rlevel->band + cb->bandno; + Jpeg2000Prec *prec = band->prec + cb->precno; + Jpeg2000Cblk *cblk = prec->cblk + cb->cblkno; + int ret, x, y, bandpos = cb->bandno + (cb->reslevelno > 0); + + t1.stride = (1<log2_cblk_width) + 2; + cb->coded = 0; + + ret = decode_cblk(s, codsty, &t1, cblk, cblk->coord[0][1] - cblk->coord[0][0], cblk->coord[1][1] - cblk->coord[1][0], bandpos, comp->roi_shift); if (ret) - coded = 1; + cb->coded = 1; else - continue; + return 0; + x = cblk->coord[0][0] - band->coord[0][0]; y = cblk->coord[1][0] - band->coord[1][0]; @@ -2000,16 +1993,28 @@ static inline void tile_codeblocks(const Jpeg2000DecoderContext *s, Jpeg2000Tile dequantization_int_97(x, y, cblk, comp, &t1, band); else dequantization_int(x, y, cblk, comp, &t1, band); - } /* end cblk */ - } /*end prec */ - } /* end band */ - } /* end reslevel */ - /* inverse DWT */ - if (coded) + return 0; +} + +static int jpeg2000_idwt(AVCodecContext *avctx, void *td, + int jobnr, int threadnr) +{ + const Jpeg2000DecoderContext *s = avctx->priv_data; + Jpeg2000IdwtThread *idwt = s->idwt + jobnr; + Jpeg2000Tile *tile = s->tile + jobnr / s->ncomponents; + int compno = jobnr % s->ncomponents; + Jpeg2000Component *comp = tile->comp + compno; + Jpeg2000CodingStyle *codsty = tile->codsty + compno; + + for (int i = idwt->cb_start; i < idwt->cb_end; i++) { + if (s->cb[i].coded) { ff_dwt_decode(&comp->dwt, codsty->transform == FF_DWT97 ? (void*)comp->f_data : (void*)comp->i_data); + break; + } + } - } /*end comp */ + return 0; } #define WRITE_FRAME(D, PIXEL) \ @@ -2079,15 +2084,13 @@ WRITE_FRAME(16, uint16_t) #undef WRITE_FRAME -static int jpeg2000_decode_tile(AVCodecContext *avctx, void *td, - int jobnr, int threadnr) +static int jpeg2000_mct_write_frame(AVCodecContext *avctx, void *td, + int jobnr, int threadnr) { const Jpeg2000DecoderContext *s = avctx->priv_data; AVFrame *picture = td; Jpeg2000Tile *tile = s->tile + jobnr; - tile_codeblocks(s, tile); - /* inverse MCT transformation */ if (tile->codsty[0].mct) mct_decode(s, tile); @@ -2477,11 +2480,79 @@ static av_cold int jpeg2000_decode_init(AVCodecContext *avctx) return 0; } +static int jpeg2000_setup_cbs(Jpeg2000DecoderContext *s, int *cbs_out) +{ + if (s->numXtiles * s->numYtiles > INT_MAX/s->ncomponents) + return AVERROR(ENOMEM); + + if (av_realloc_array_reuse(&s->idwt, &s->idwt_allocated, + s->numXtiles * s->numYtiles * s->ncomponents, + INT_MAX, sizeof(*s->idwt)) < 0) + return AVERROR(ENOMEM); + + for (int pass = 0; pass < 2; pass++) { + int cbs = 0; + for (int tileno = 0; tileno < s->numXtiles * s->numYtiles; tileno++) { + for (int compno = 0; compno < s->ncomponents; compno++) { + Jpeg2000Tile *tile = s->tile + tileno; + Jpeg2000Component *comp = tile->comp + compno; + Jpeg2000CodingStyle *codsty = tile->codsty + compno; + Jpeg2000IdwtThread *idwt = s->idwt + compno + tileno * s->ncomponents; + + idwt->cb_start = cbs; + + for (int reslevelno = 0; reslevelno < codsty->nreslevels2decode; reslevelno++) { + Jpeg2000ResLevel *rlevel = comp->reslevel + reslevelno; + for (int bandno = 0; bandno < rlevel->nbands; bandno++) { + int nb_precincts = rlevel->num_precincts_x * rlevel->num_precincts_y; + Jpeg2000Band *band = rlevel->band + bandno; + + if (band->coord[0][0] == band->coord[0][1] || + band->coord[1][0] == band->coord[1][1]) + continue; + + for (int precno = 0; precno < nb_precincts; precno++) { + Jpeg2000Prec *prec = band->prec + precno; + int prec_cbs = prec->nb_codeblocks_width * prec->nb_codeblocks_height; + + if (cbs > INT_MAX - prec_cbs) + return AVERROR(ENOMEM); + + for (int cblkno = 0; cblkno < prec_cbs; cblkno++, cbs++) { + if (pass == 1) { + Jpeg2000CodeblockThread *cb = s->cb + cbs; + cb->tileno = tileno; + cb->compno = compno; + cb->reslevelno = reslevelno; + cb->bandno = bandno; + cb->precno = precno; + cb->cblkno = cblkno; + } + } + } + } + } + + idwt->cb_end = cbs; + } + } + + if (pass == 0) { + if (av_realloc_array_reuse(&s->cb, &s->cb_allocated, + cbs, INT_MAX, sizeof(*s->cb)) < 0) + return AVERROR(ENOMEM); + } + + *cbs_out = cbs; + } + return 0; +} + static int jpeg2000_decode_frame(AVCodecContext *avctx, AVFrame *picture, int *got_frame, AVPacket *avpkt) { Jpeg2000DecoderContext *s = avctx->priv_data; - int ret; + int ret, cbs; s->avctx = avctx; bytestream2_init(&s->g, avpkt->data, avpkt->size); @@ -2548,7 +2619,12 @@ static int jpeg2000_decode_frame(AVCodecContext *avctx, AVFrame *picture, } } - avctx->execute2(avctx, jpeg2000_decode_tile, picture, NULL, s->numXtiles * s->numYtiles); + if ((ret = jpeg2000_setup_cbs(s, &cbs)) < 0 || + (ret = avctx->execute2(avctx, jpeg2000_decode_cb, NULL, NULL, cbs)) < 0 || + (ret = avctx->execute2(avctx, jpeg2000_idwt, NULL, NULL, s->numXtiles * s->numYtiles * s->ncomponents)) < 0 || + (ret = avctx->execute2(avctx, jpeg2000_mct_write_frame, picture, NULL, s->numXtiles * s->numYtiles)) < 0) + goto end; + jpeg2000_dec_cleanup(s); @@ -2564,6 +2640,16 @@ end: return ret; } +static av_cold int jpeg2000_decode_close(AVCodecContext *avctx) +{ + Jpeg2000DecoderContext *s = avctx->priv_data; + + av_freep(&s->idwt); + av_freep(&s->cb); + + return 0; +} + #define OFFSET(x) offsetof(Jpeg2000DecoderContext, x) #define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM @@ -2589,6 +2675,7 @@ const FFCodec ff_jpeg2000_decoder = { .priv_data_size = sizeof(Jpeg2000DecoderContext), .init = jpeg2000_decode_init, FF_CODEC_DECODE_CB(jpeg2000_decode_frame), + .close = jpeg2000_decode_close, .p.priv_class = &jpeg2000_class, .p.max_lowres = 5, .p.profiles = NULL_IF_CONFIG_SMALL(ff_jpeg2000_profiles), -- 2.30.2