From patchwork Sat Feb 24 20:06:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Connor Worley X-Patchwork-Id: 46505 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c51b:b0:19e:cdac:8cce with SMTP id gm27csp598370pzb; Sat, 24 Feb 2024 12:06:30 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCWWHVEC1qu7/S71F/XHZ0fHC+akniSzvVOHPJ4fMYTnot2WMnVMUDYq5ETvUCiQNy9I/N49ArVtCCUpNkC4x2kR5v606lvOnZDGjQ== X-Google-Smtp-Source: AGHT+IEzJCUZh2J+JAdWEOhlf9QccHynJkoD9gwj/jJefNqjVfwkQsR0CIR34puaGf0jpmMTOaxB X-Received: by 2002:a17:906:4748:b0:a3f:357b:d995 with SMTP id j8-20020a170906474800b00a3f357bd995mr2013093ejs.1.1708805189979; Sat, 24 Feb 2024 12:06:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1708805189; cv=none; d=google.com; s=arc-20160816; b=SRgJzfxoSV7eQ58pL2E44QDD3Zun7KlLJQffeal+unAfesN/ZFxfA+Tkm3rQYLkbRu A/9ZubSajAPX1H3ONJcGdNPXuUuHxWBbr8PxBKiv1z+AjMyXKLKQuDggKB5jc3QcgJPg s7StHnHjchcmZvX6b+DFLYsTEkrue592MuZ5RcK2Lo+rVFukvUC7q00lzw3bpQjrfuqS yli8wJnid6TSQDKSAUd+0pqv7NoCQOaYvC5V6TZjUQuHNS9FL69SQfz8McMSkV4nXF+b VwDPfr2TTSZka1ADY43rpMX7+Qqv2g8c8olNfgn16f1fY4IIvHSkAM60TdB6UFG08Ex7 uiyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=7chkcYnnKGVaYs3o2Cp1ZmvkCvjUBhFvepluK+HOybc=; fh=4o9WedBEbOrq7GvNs0s+TOKeyzmDZRM3O7oe6ZP6vYM=; b=I4Zp8BaEHKscl9/UqkQPkYxYC/oCQDHo732U6d9VlHjYBCCxQfezJzV33QZX6jMzBK q+r5Pr5iRVrkehOGnHfR9qd4YOFNEgJjrRWlnOx/KOoTV91Be7Km6WvtoEtyZr2nxQod QEWxy/pMgkjNp89H6UJAu2wjhkb4G1TMSiHKu1SgX8k+zCnCw7RNLaSz6t+/2NEY8dIO sEcuy3zNFakTSSf6ZrwTLLr69/Gn3pqSYD6iPtSvjIDvRdLYxPrySdGLoe7eyiPvgNX4 R1xZ9sECYUpg6QJdzcc6YRAEqQMf0pWs2A07bp33ux+5QzAHd6A94xlQMEKFrjV4T278 lxLA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=bwuPgDdg; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id qk39-20020a1709077fa700b00a43124be50csi161373ejc.377.2024.02.24.12.06.29; Sat, 24 Feb 2024 12:06:29 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=bwuPgDdg; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3272068C6CE; Sat, 24 Feb 2024 22:06:18 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2998468C695 for ; Sat, 24 Feb 2024 22:06:11 +0200 (EET) Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-6da202aa138so1280266b3a.2 for ; Sat, 24 Feb 2024 12:06:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708805168; x=1709409968; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Cut6hcq27mn8Wj7NZr4ZByruIoDRge47+ebkhb9J4NQ=; b=bwuPgDdg/ArpPX07f5scTVCLPY1cSdjEknWSv9WyRNUbjCtX+sMQXjFgl8udXnX3z+ P8m2UkPuH9hxTEoo3Xugvutcl7cN57+YMfeBrkcwNRbBGG8bswLoGCkR+E34CXRB89bV fRzjCUYLc4FajKroSWL9L7GHqN/D5gYkwcz94QaWNGpOfTKmjIRXCbt+usPzJR/QTF9n z05aTRyYBgwy2OfumTk/4VS8Y9OpsAkLSjQYUieG2OZfTExeQvf4QtEhvyjheoI3PODO jMUyStcx6nHlHD2MrjvKj1diOEFje22Hp67f7sf80nCzF/sMqrWtOnBrVIxdHpzC/yyX qz0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708805168; x=1709409968; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Cut6hcq27mn8Wj7NZr4ZByruIoDRge47+ebkhb9J4NQ=; b=b8Yow+ek7iQInFPv7MuHx/9Tktfyv3CZAhcsv1aTcmq5fMW+DKCXVYv5koRweL5STB ATlOrO1sJJKdsUsZSGS6LB0ziQe6ld2EktY9fN4ukzluPWX0Ndtg0pyw52C8hRi3MxR2 AQI2/2deM2RsCgW+S23j+eNlCekkrz4wsd6xuOIFcVLODN/AyLnNhyK2IZzae0BrzQnu X36v79X1pHzGeh400Ui7TFrZBIkW6hAJU3hq6Go5EmWxxVxDpoRvgq7msd9WOdMVvnvZ j3AH9w5LKx5J1l4jueHYdCYwpR69z5uf58iUtL923/h18rKn7vf2iO/ia6A5673GGnVj +rzQ== X-Gm-Message-State: AOJu0YycxDpnU39cOEortDkDdswyP9yziqjSvSz4io94QtibmWciG2QW UEq5RBfmKO9xe8Iz8rVgLIQVLjMyiFNloCUhdw5D984hbo62XWwW80rfeNLz3zg= X-Received: by 2002:a05:6a00:b8f:b0:6e4:f678:f694 with SMTP id g15-20020a056a000b8f00b006e4f678f694mr2384047pfj.19.1708805167910; Sat, 24 Feb 2024 12:06:07 -0800 (PST) Received: from localhost ([2601:647:4600:84e0:21ab:155b:fbb:f2b0]) by smtp.gmail.com with UTF8SMTPSA id ei6-20020a056a0080c600b006e50a28ecf5sm25970pfb.171.2024.02.24.12.06.07 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 24 Feb 2024 12:06:07 -0800 (PST) From: Connor Worley To: ffmpeg-devel@ffmpeg.org Date: Sat, 24 Feb 2024 12:06:03 -0800 Message-ID: <20240224200603.6613-2-connorbworley@gmail.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240224200603.6613-1-connorbworley@gmail.com> References: <20240224200603.6613-1-connorbworley@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 2/2] lavc/dxvenc: migrate DXT1 encoder to lavu hashtable X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Connor Worley Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: JGMkDWkS0zU0 Offers a modest performance gain due to the switch from naive linear probling to robin hood. Signed-off-by: Connor Worley --- libavcodec/dxvenc.c | 121 ++++++++++++-------------------------------- 1 file changed, 33 insertions(+), 88 deletions(-) diff --git a/libavcodec/dxvenc.c b/libavcodec/dxvenc.c index 1ce2b1d014..980269657d 100644 --- a/libavcodec/dxvenc.c +++ b/libavcodec/dxvenc.c @@ -21,7 +21,7 @@ #include -#include "libavutil/crc.h" +#include "libavutil/hashtable.h" #include "libavutil/imgutils.h" #include "libavutil/opt.h" @@ -38,72 +38,9 @@ * appeared in the decompressed stream. Using a simple hash table (HT) * significantly speeds up the lookback process while encoding. */ -#define LOOKBACK_HT_ELEMS 0x40000 +#define LOOKBACK_HT_ELEMS 0x20202 #define LOOKBACK_WORDS 0x20202 -typedef struct HTEntry { - uint32_t key; - uint32_t pos; -} HTEntry; - -static void ht_init(HTEntry *ht) -{ - for (size_t i = 0; i < LOOKBACK_HT_ELEMS; i++) { - ht[i].pos = -1; - } -} - -static uint32_t ht_lookup_and_upsert(HTEntry *ht, const AVCRC *hash_ctx, - uint32_t key, uint32_t pos) -{ - uint32_t ret = -1; - size_t hash = av_crc(hash_ctx, 0, (uint8_t*)&key, 4) % LOOKBACK_HT_ELEMS; - for (size_t i = hash; i < hash + LOOKBACK_HT_ELEMS; i++) { - size_t wrapped_index = i % LOOKBACK_HT_ELEMS; - HTEntry *entry = &ht[wrapped_index]; - if (entry->key == key || entry->pos == -1) { - ret = entry->pos; - entry->key = key; - entry->pos = pos; - break; - } - } - return ret; -} - -static void ht_delete(HTEntry *ht, const AVCRC *hash_ctx, - uint32_t key, uint32_t pos) -{ - HTEntry *removed_entry = NULL; - size_t removed_hash; - size_t hash = av_crc(hash_ctx, 0, (uint8_t*)&key, 4) % LOOKBACK_HT_ELEMS; - - for (size_t i = hash; i < hash + LOOKBACK_HT_ELEMS; i++) { - size_t wrapped_index = i % LOOKBACK_HT_ELEMS; - HTEntry *entry = &ht[wrapped_index]; - if (entry->pos == -1) - return; - if (removed_entry) { - size_t candidate_hash = av_crc(hash_ctx, 0, (uint8_t*)&entry->key, 4) % LOOKBACK_HT_ELEMS; - if ((wrapped_index > removed_hash && (candidate_hash <= removed_hash || candidate_hash > wrapped_index)) || - (wrapped_index < removed_hash && (candidate_hash <= removed_hash && candidate_hash > wrapped_index))) { - *removed_entry = *entry; - entry->pos = -1; - removed_entry = entry; - removed_hash = wrapped_index; - } - } else if (entry->key == key) { - if (entry->pos <= pos) { - entry->pos = -1; - removed_entry = entry; - removed_hash = wrapped_index; - } else { - return; - } - } - } -} - typedef struct DXVEncContext { AVClass *class; @@ -120,10 +57,8 @@ typedef struct DXVEncContext { DXVTextureFormat tex_fmt; int (*compress_tex)(AVCodecContext *avctx); - const AVCRC *crc_ctx; - - HTEntry color_lookback_ht[LOOKBACK_HT_ELEMS]; - HTEntry lut_lookback_ht[LOOKBACK_HT_ELEMS]; + struct AVHashtableContext *color_ht; + struct AVHashtableContext *lut_ht; } DXVEncContext; /* Converts an index offset value to a 2-bit opcode and pushes it to a stream. @@ -158,27 +93,32 @@ static int dxv_compress_dxt1(AVCodecContext *avctx) DXVEncContext *ctx = avctx->priv_data; PutByteContext *pbc = &ctx->pbc; uint32_t *value; - uint32_t color, lut, idx, color_idx, lut_idx, prev_pos, state = 16, pos = 2, op = 0; + uint32_t color, lut, idx, color_idx, lut_idx, prev_pos, state = 16, pos = 0, op = 0; - ht_init(ctx->color_lookback_ht); - ht_init(ctx->lut_lookback_ht); + av_hashtable_clear(ctx->color_ht); + av_hashtable_clear(ctx->lut_ht); bytestream2_put_le32(pbc, AV_RL32(ctx->tex_data)); + av_hashtable_set(ctx->color_ht, ctx->tex_data, &pos); + pos++; bytestream2_put_le32(pbc, AV_RL32(ctx->tex_data + 4)); - - ht_lookup_and_upsert(ctx->color_lookback_ht, ctx->crc_ctx, AV_RL32(ctx->tex_data), 0); - ht_lookup_and_upsert(ctx->lut_lookback_ht, ctx->crc_ctx, AV_RL32(ctx->tex_data + 4), 1); + av_hashtable_set(ctx->lut_ht, ctx->tex_data + 4, &pos); + pos++; while (pos + 2 <= ctx->tex_size / 4) { idx = 0; + color_idx = 0; + lut_idx = 0; color = AV_RL32(ctx->tex_data + pos * 4); - prev_pos = ht_lookup_and_upsert(ctx->color_lookback_ht, ctx->crc_ctx, color, pos); - color_idx = prev_pos != -1 ? pos - prev_pos : 0; + if (av_hashtable_get(ctx->color_ht, &color, &prev_pos)) + color_idx = pos - prev_pos; + av_hashtable_set(ctx->color_ht, &color, &pos); + if (pos >= LOOKBACK_WORDS) { uint32_t old_pos = pos - LOOKBACK_WORDS; - uint32_t old_color = AV_RL32(ctx->tex_data + old_pos * 4); - ht_delete(ctx->color_lookback_ht, ctx->crc_ctx, old_color, old_pos); + if (av_hashtable_get(ctx->color_ht, ctx->tex_data + old_pos * 4, &prev_pos) && prev_pos <= old_pos) + av_hashtable_delete(ctx->color_ht, ctx->tex_data + old_pos * 4); } pos++; @@ -187,13 +127,14 @@ static int dxv_compress_dxt1(AVCodecContext *avctx) idx = color_idx; } else { idx = 0; - prev_pos = ht_lookup_and_upsert(ctx->lut_lookback_ht, ctx->crc_ctx, lut, pos); - lut_idx = prev_pos != -1 ? pos - prev_pos : 0; + if (av_hashtable_get(ctx->lut_ht, &lut, &prev_pos)) + lut_idx = pos - prev_pos; + av_hashtable_set(ctx->lut_ht, &lut, &pos); } if (pos >= LOOKBACK_WORDS) { uint32_t old_pos = pos - LOOKBACK_WORDS; - uint32_t old_lut = AV_RL32(ctx->tex_data + old_pos * 4); - ht_delete(ctx->lut_lookback_ht, ctx->crc_ctx, old_lut, old_pos); + if (av_hashtable_get(ctx->lut_ht, ctx->tex_data + old_pos * 4, &prev_pos) && prev_pos <= old_pos) + av_hashtable_delete(ctx->lut_ht, ctx->tex_data + old_pos * 4); } pos++; @@ -305,11 +246,12 @@ static av_cold int dxv_init(AVCodecContext *avctx) return AVERROR(ENOMEM); } - ctx->crc_ctx = av_crc_get_table(AV_CRC_32_IEEE); - if (!ctx->crc_ctx) { - av_log(avctx, AV_LOG_ERROR, "Could not initialize CRC table.\n"); - return AVERROR_BUG; - } + ret = av_hashtable_alloc(&ctx->color_ht, sizeof(uint32_t), sizeof(uint32_t), LOOKBACK_HT_ELEMS); + if (ret < 0) + return ret; + ret = av_hashtable_alloc(&ctx->lut_ht, sizeof(uint32_t), sizeof(uint32_t), LOOKBACK_HT_ELEMS); + if (ret < 0) + return ret; return 0; } @@ -320,6 +262,9 @@ static av_cold int dxv_close(AVCodecContext *avctx) av_freep(&ctx->tex_data); + av_hashtable_freep(&ctx->color_ht); + av_hashtable_freep(&ctx->lut_ht); + return 0; }