From patchwork Fri Mar 11 15:59:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 34704 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6838:3486:0:0:0:0 with SMTP id ek6csp1819240nkb; Fri, 11 Mar 2022 07:59:55 -0800 (PST) X-Google-Smtp-Source: ABdhPJyNdYF9N3ODLjBwlCkanDA6sFoL0ASaro//gZLfcj3/UKwLkYyJl9hS+OhS+tcIZhKgReWX X-Received: by 2002:a05:6402:42c9:b0:415:e324:c211 with SMTP id i9-20020a05640242c900b00415e324c211mr9771688edc.234.1647014395541; Fri, 11 Mar 2022 07:59:55 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 16-20020a170906301000b006b2a4769e72si4800002ejz.325.2022.03.11.07.59.55; Fri, 11 Mar 2022 07:59:55 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@outlook.com header.s=selector1 header.b=DlZt8KeN; arc=fail (body hash mismatch); spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=outlook.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F214B68B143; Fri, 11 Mar 2022 17:59:52 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-oln040092074074.outbound.protection.outlook.com [40.92.74.74]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9D4C9689F00 for ; Fri, 11 Mar 2022 17:59:46 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=W80hGx3pvLmB22X9ZCC0x5LuclXUoGdHSO8+bo1t48+fcxAJHqqBEJ4qp/Ay99OsOiOu4E3Uh5ZYUkIn+pOTnNIJr/AJaQEOz2fMLni+9hJ1zfV4FY6b0s2CU8bSE94YiuEq1qlzvXAaQiwfNfiNh8p9zFUhpMizjzxirOvLhoreIKAww/dvmVs68J9x8XOiNLU70sP98vIeMyLEuCZmLr0WMDcwWTi2BXDVR36WJ7QFLKDllNcf/vhuEXYs7SeDJedhh7F1LoD5Es2rPifqWWXKxcflrQDPwa5CBFL3hbbpbbiBB9a36hVCfX3oWYujCjEwg0ED8DRFlT00zfUdfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ttVH7jXIUpD7kFUVeOrbolBcw+If+lKVkbNvN4jxml4=; b=DX3yAIhT9bb6/rsW4m8AHtME1K1L7Ztj/BTiq3FnTrw2RiDVof0PkFeMCSMC5dBFqtUjIEOIEICWRjY5xqwlmCV39ZRc8esrTn+UZ9OVTO0Eow3wG6IAuMtFKHn2PUheD7ikXkFZ7mnZLTO/ieD+xY0uYpdYxjlTpp/6rMVksAuTjYUQbXF1SxOcRFvaiBWnp81GXDF6wFlrquJYqYXZzACcXSwCHd9qFCZZErqmW2wnqRiIA9VR0KFIYrmRyihlPxw/CZXLzkA/u4dbk/2Ia5+33sFpa8SAO7aCKBaXyPb3oEWPYElRyip2eTG8p9yK+xsb+8qAFoiPcyLQBjvrfg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ttVH7jXIUpD7kFUVeOrbolBcw+If+lKVkbNvN4jxml4=; b=DlZt8KeNflsciII2mMVMizbsvtNOJr0hBjIvXeFeXFPILM/V/mj2S7J8OlWJEaXvlq8AWu9DBOOvvP5NM1pnWuolssLw1in27QRmA17BO7Nd22CMuiXDD++d39M5GBy2QEkm2h5q4diLLW6YjkESJoqYzZtIkcOdi8EgfdND7K+1AURLzFwrcoCICwGKNBaliaNs/2KYDGiM/J85R/UbJSTOyKK/wMWKPUeOTzo0l2O7anvF0crIiRFq4xNF8dAl0NKBf1fFxSrI5EtDWjgBYZjEtHfYY43W6vLX/1cky48ASk4Hl3tpGM4ka0bMy4x57gOrUcxVT9qkV9ql66a+Dw== Received: from AM7PR03MB6660.eurprd03.prod.outlook.com (2603:10a6:20b:1c1::22) by AM6PR03MB3991.eurprd03.prod.outlook.com (2603:10a6:20b:25::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5061.22; Fri, 11 Mar 2022 15:59:44 +0000 Received: from AM7PR03MB6660.eurprd03.prod.outlook.com ([fe80::bc6c:af66:7adb:7a3d]) by AM7PR03MB6660.eurprd03.prod.outlook.com ([fe80::bc6c:af66:7adb:7a3d%7]) with mapi id 15.20.5061.022; Fri, 11 Mar 2022 15:59:44 +0000 From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 11 Mar 2022 16:59:37 +0100 Message-ID: X-Mailer: git-send-email 2.32.0 In-Reply-To: References: X-TMN: [T+1LP58oNcKdg/vA0d+nWM7xbaD+6MV6] X-ClientProxiedBy: AM6PR04CA0012.eurprd04.prod.outlook.com (2603:10a6:20b:92::25) To AM7PR03MB6660.eurprd03.prod.outlook.com (2603:10a6:20b:1c1::22) X-Microsoft-Original-Message-ID: <20220311155937.220285-1-andreas.rheinhardt@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0279ff71-97a8-426c-586f-08da037828b3 X-MS-Exchange-SLBlob-MailProps: gjx25WM8ZNW55cP4YTouUlFWMuNNcdTJ2eDlnZJQJb7CdkeRnLdrKQcJqQHsvTREBw4sKLYa73i6pKWViyRLesDB4SyQCXp7VwYy/8H9o2zxg9I5COkFmwhzpLS5q9Uybzwm4B3Mu8UupCKD8AFrNS45DWgXocEYDJOfmyQK9bBx3scv7DKvAN5+f2i7fMtnW9eOTpc5yxi/OaS3mU1vJ9sm/wgqqAmIRNXE/KtCckbAIxRXPPvquRsE74avhqKej11ID8ahJdCRLakWYPDF3M9NhxKtnAGi5coh8ucanJAW5S5vLe0HRvTjXt+PjxLRmwf3YQjWtLq16A5Q3chlPjXZ3rC1IjHmUFTUpwOy0IjLl9Gg1H0FX8UL/aKoYWQW34Dj1ygWsxaROcCFnJkcTVxmo0r4tUee4kmWLFJugGHvH10y2S07RUx1INUDBl8aWUNhIMboImYreIo5iBsbk7GOwSRd+euKMcd/QAADLCTEx2e74RXvQihLph8p9W7POMTvYgwARcW6W+zm9/M39P95gHko6cwJtuKGENpjAx5DIiLNb23X0LAtxtfQVlmBkLRRIJ7qj4cXywxphxElJsCdkvFpjlciI8kJUcPHdTtmO0VyGR2QALRoKZ4u1fWNihkdmbeEA+kmyLuXcpZJpHtWUik6Cy/LDMgkoF2ehH8frTj9DpXcD3eDQ/A5+RSIEf6XTahxTxfwdrY/3KSthfFw7BGBc02S6JbtXaVGpuw= X-MS-TrafficTypeDiagnostic: AM6PR03MB3991:EE_ X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Q93SCwgxeHhs8O+tEHPARbNViahsGFMKhlGpwqeJryLuGYSBAoaHTLUtS2sRKsnxINsUk9D8YIjjKJ70ozkUJzSyb1ZovtqG6TIwlgvBKIX5Q34R44TbBG3LjubxTvG9f3hm7Qe9VJf90GMEQDC8UUgpIeo9Od0OTEpyOumqxIgkholSelJlzsiVrIfS3cC4W7+ot8lV0Nicfx8RkcG9zvTclV042ho5k1egGH+zUlV3PrUzVv6OR9RiGZgURMiaPGvcog2j2Yn9tb77ObTSGPb7geeo+quTdJZ189ACgPX8r69gqx9QdDABJY/m4NEWjTLSGYf7bLfWGFJkOp0604LhSSaTXT3dzAOD6NSEcfUv/gajhg7OojIgMhT51lpnm31fZqc7kxHFf5QaA7EeQhjk2eiGsqUD3JFPNVikB4LeizlqmLS/fOLSIYf6Uhk3khJphMZWMNFa2WisV7lVzpT2gm40ZvaCdXE6juCB/SQASDaT/EBQNy+l6f9T0rUMLmaQxvMrroAkoq6zaV8RHiCeZQ0VKBix6tiFoduQyCTzC1kUo1/Xwta95BrWigTc8jcdqUVCSUxO4f6eyq769A== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Fsr0XqLEF25ZA3wk6Auq5ZhWl3uo7Qk9cw7CbowzxG2ItRR+dbblJ3TlTRNG6a8FRCnhW5s+sFdEl4p1sZ6jljYiy7XmjIuz0q0+BybsTdZva9uS/2ndOP8oSmPjtgiJwasA+SfrWjjkAKS7EeaTUdFnwCJgFMi6K4WWaKXRPkr14bKcWpzgv9cPtP8gBxRTJKZTZWRvRBJVNiC0jJdcmkhQopDNHJPAVDw236cBnIZleqXdvqOP5pVKQ2XKLLf/OmJcVw4dnvzSYzATbK4X2DURjCyFGQ8SWJ//eUXfFzePgjKfWD54W9jhoOYFPQVk4+CdAx/qG42EY+YmqQxg2CC3og6BpiCaeV8dKj5vTp4VzSdTVvo5C/S7bMCab0RODrpd9Nuhbr1og6rMHdG4rI5Nnp80mv32ny1b+ex6jPRO3ua235S2X1VTsSbXnhuCfzm6kIIi8zJjbqX1s4ky1MPtYBQHkg7c8gPSKHjffsaKYb5todm+lCS491NqqmpKrPzub/N1Z9atEDObeDaHSrkgVuVUaYVhcyBPd73XIbdrf/WQC+lK7jkjdgD5mYnvvseASx0D0a/JDMiV1CI/h0ribhqtjsGhZab+ZPV3yknU73oB4bn3XI2s3+KTRiGOqrX/vvXcK3uRHWa7IJ57NO9NQj1yfdo3ZZ6NU2rJZ0dwL+89A5mEkxEVWwBA46jaN2xLDMP99ivgO69iQPeTOczF8esR8+SofgD6z69/HjG4hC6I02fYBzFQ42o2d8y0eV9jVmGpdwdgDeSJ5bEYDjyL9qz/T/slWgDxYdrMl0Nm1W8yNsnlnZ9cGOdxFO3MAnAOJTgQSoseYLubV9htMXCz0i32KLXESaEBcwmV6H/fytlToJ5xo2+NHfNU/merxY4uyvMlqahrN21MQBLFsdJ7P7rOXEcnWUvRGGJqFz0D4qpYmtP2bQVXB1P6JOgv4drVc1anV/vDknQdwHmVCVbOjtek6B/l4oB2IyQK9lXe+sxov98/CqgcxAZqC+JG6yGfiuGISzThfuXsOeqchnlK+U+EVRdagdQO+CcRoicjKX4HdQI8Anl/JWfjRMErHDTbJ5FfAMyrkLNWBo2tSmmzsTB5X4FvVv3cLdyzXvJxU4aAGCkQ3HYLyGudSHqXOKjDDAjCgqc+mrXfSikwRn2yC/znHNsFCOKAufQcgJ0vpIFJAWRssCmmluK3Xmjx X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0279ff71-97a8-426c-586f-08da037828b3 X-MS-Exchange-CrossTenant-AuthSource: AM7PR03MB6660.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2022 15:59:44.3272 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR03MB3991 Subject: [FFmpeg-devel] [PATCH 2/2] avcodec/flashsv: Avoid deflating data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: W88X3Rbd4AWm Currently priming the zlib decompressor involves compressing data directly after having decompressed it and decompressing it again in order to set the "dictionary" and to initialize the adler32-checksum. Yet this is wasteful and can be simplified by synthetizing the compressed data via non-compressed blocks. This reduced the amount of allocations for the decoding part of fate-vsynth1-flashsv2, namely from total heap usage: 9,135 allocs, 9,135 frees, 376,503,427 bytes allocated to total heap usage: 2,373 allocs, 2,373 frees, 14,144,083 bytes allocated Signed-off-by: Andreas Rheinhardt --- Unfortunately one can't simply use inflateSetDictionary() to set the dictionary, as this does not initialize the adler32-checksum. More precisely, the main adler32 does not cover the dictionary data; so it would decode correctly, but one would get checksum errors. Notice that I do not have a single flashsv2 file that has not been created by our encoder and it might be that our encoder is wrong. Looking at the mailing list archive showed lots of mails that complain about the paucacity of samples. The spec simply speaks of "the ZLIB priming technique of compression". How very helpful. libavcodec/flashsv.c | 89 +++++++++++++++++++------------------------- 1 file changed, 38 insertions(+), 51 deletions(-) diff --git a/libavcodec/flashsv.c b/libavcodec/flashsv.c index 5f0bc0c6df..e284439972 100644 --- a/libavcodec/flashsv.c +++ b/libavcodec/flashsv.c @@ -63,11 +63,10 @@ typedef struct FlashSVContext { AVBufferRef *keyframedata_buf; uint8_t *keyframe; BlockInfo *blocks; - uint8_t *deflate_block; - int deflate_block_size; int color_depth; int zlibprime_curr, zlibprime_prev; int diff_start, diff_height; + uint8_t tmp[UINT16_MAX]; } FlashSVContext; static int decode_hybrid(const uint8_t *sptr, const uint8_t *sptr_end, uint8_t *dptr, int dx, int dy, @@ -141,41 +140,59 @@ static av_cold int flashsv_decode_init(AVCodecContext *avctx) static int flashsv2_prime(FlashSVContext *s, const uint8_t *src, int size) { - z_stream zs; int zret; // Zlib return code + static const uint8_t zlib_header[] = { 0x78, 0x01 }; + uint8_t *data = s->tmpblock; + unsigned remaining; if (!src) return AVERROR_INVALIDDATA; - zs.zalloc = NULL; - zs.zfree = NULL; - zs.opaque = NULL; - s->zstream.next_in = src; s->zstream.avail_in = size; - s->zstream.next_out = s->tmpblock; + s->zstream.next_out = data; s->zstream.avail_out = s->block_size * 3; inflate(&s->zstream, Z_SYNC_FLUSH); - - if (deflateInit(&zs, 0) != Z_OK) - return -1; - zs.next_in = s->tmpblock; - zs.avail_in = s->block_size * 3 - s->zstream.avail_out; - zs.next_out = s->deflate_block; - zs.avail_out = s->deflate_block_size; - deflate(&zs, Z_SYNC_FLUSH); - deflateEnd(&zs); + remaining = s->block_size * 3 - s->zstream.avail_out; if ((zret = inflateReset(&s->zstream)) != Z_OK) { av_log(s->avctx, AV_LOG_ERROR, "Inflate reset error: %d\n", zret); return AVERROR_UNKNOWN; } - s->zstream.next_in = s->deflate_block; - s->zstream.avail_in = s->deflate_block_size - zs.avail_out; - s->zstream.next_out = s->tmpblock; - s->zstream.avail_out = s->block_size * 3; + /* Create input for zlib that is equivalent to encoding the output + * from above and decoding it again (the net result of this is that + * the dictionary of past decoded data is correctly primed and + * the adler32 checksum is correctly initialized). + * This is accomplished by synthetizing blocks of uncompressed data + * out of the output from above. See section 3.2.4 of RFC 1951. */ + s->zstream.next_in = zlib_header; + s->zstream.avail_in = sizeof(zlib_header); inflate(&s->zstream, Z_SYNC_FLUSH); + while (remaining > 0) { + unsigned block_size = FFMIN(UINT16_MAX, remaining); + uint8_t header[5]; + /* Bit 0: Non-last-block, bits 1-2: BTYPE for uncompressed block */ + header[0] = 0; + /* Block size */ + AV_WL16(header + 1, block_size); + /* Block size (one's complement) */ + AV_WL16(header + 3, block_size ^ 0xFFFF); + s->zstream.next_in = header; + s->zstream.avail_in = sizeof(header); + s->zstream.next_out = s->tmp; + s->zstream.avail_out = sizeof(s->tmp); + zret = inflate(&s->zstream, Z_SYNC_FLUSH); + if (zret != Z_OK) + return AVERROR_UNKNOWN; + s->zstream.next_in = data; + s->zstream.avail_in = block_size; + zret = inflate(&s->zstream, Z_SYNC_FLUSH); + if (zret != Z_OK) + return AVERROR_UNKNOWN; + data += block_size; + remaining -= block_size; + } return 0; } @@ -248,22 +265,6 @@ static int flashsv_decode_block(AVCodecContext *avctx, const AVPacket *avpkt, return 0; } -static int calc_deflate_block_size(int tmpblock_size) -{ - z_stream zstream; - int size; - - zstream.zalloc = Z_NULL; - zstream.zfree = Z_NULL; - zstream.opaque = Z_NULL; - if (deflateInit(&zstream, 0) != Z_OK) - return -1; - size = deflateBound(&zstream, tmpblock_size); - deflateEnd(&zstream); - - return size; -} - static int flashsv_decode_frame(AVCodecContext *avctx, void *data, int *got_frame, AVPacket *avpkt) { @@ -322,19 +323,6 @@ static int flashsv_decode_frame(AVCodecContext *avctx, void *data, "Cannot allocate decompression buffer.\n"); return err; } - if (s->ver == 2) { - s->deflate_block_size = calc_deflate_block_size(tmpblock_size); - if (s->deflate_block_size <= 0) { - av_log(avctx, AV_LOG_ERROR, - "Cannot determine deflate buffer size.\n"); - return -1; - } - if ((err = av_reallocp(&s->deflate_block, s->deflate_block_size)) < 0) { - s->block_size = 0; - av_log(avctx, AV_LOG_ERROR, "Cannot allocate deflate buffer.\n"); - return err; - } - } } s->block_size = s->block_width * s->block_height; @@ -570,7 +558,6 @@ static av_cold int flashsv2_decode_end(AVCodecContext *avctx) av_buffer_unref(&s->keyframedata_buf); av_freep(&s->blocks); av_freep(&s->keyframe); - av_freep(&s->deflate_block); flashsv_decode_end(avctx); return 0;