From patchwork Mon Feb 5 05:27:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Connor Worley X-Patchwork-Id: 46032 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:7b08:b0:19e:8a94:b663 with SMTP id s8csp554862pzh; Sun, 4 Feb 2024 21:27:53 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCUYMJap+hcx+FUIxwVlRU3WrLAYAOt5g4vewWm0whU12hwOyLkZeloAFJjh/u1vmPi6HxFDYZKxUPi1rK1NbdCwK5WwJeanZJtKlA== X-Google-Smtp-Source: AGHT+IE+RE+hXAF7hnE6vB7m0JUHsafQgGhM/yC9LpLHa3o/yKHLQ3l39OOq79WczlblQertDNh/ X-Received: by 2002:a17:906:6b92:b0:a37:7f72:574c with SMTP id l18-20020a1709066b9200b00a377f72574cmr2444915ejr.68.1707110873491; Sun, 04 Feb 2024 21:27:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1707110873; cv=none; d=google.com; s=arc-20160816; b=OGWMGQhf9oGKAGNnY51ZVfDNVxEUvneINFCep/hXKP2B6ZomXz6X3F+H+d87+MDNjK Kv8MMpVvBXCeeZGH/D4jL2LV6+VLFz5x4Xc0LZJ57XrjDW4YbiKC+IEd1Kg9AUPeV1yc 1++WCTNI3OwAG4U1ZYeJHYwnROUa3e/fpGkC3qq7OnyhZqB5Grk7zQG5j9zuYNLZL3ZG PS1sRBUGyblKFaozw0OpU/36eSwV4s/wKekhaB/801QOqPutEMvfYeP+CSUnWE3LqlJE aDBHsU6mOWeE/BsSNjNO8PsXxfb0RxnkXTH81BJkjKiXla2vWo6vsPRfCdcSJHUyM2Rx upFg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=lTgklaYbZAiH1dYP2rj4NEo0Yrt98zo2oxTK4A4ypzc=; fh=ZyuWHoWvHXJVA3/fYpt+jJeXzo/kditBj74Pi8IhXiA=; b=ONyIWLqRQOYnrY1tNq9Ux3NvcH5SJ+QOENFI6ve46ZW9D7vzGyD9oPb097wOxmetJb KSVGKzEoVmEIE4CsDwahDcOASo9hovHgKe2U0q0YPSPdctQrzlr9kmoYHbKG0tyu0eII kq979RH/L/v5+F8X5beZNr7DB+zp1MaxK41+/jhJRsOslfT4TQ2tVbpY3sHbnX4tZBRb NGAm+VIeP6SfIP0TDqrFyU3unRYondcXXNwjzGPdzUsAD+JcKFfw6M/AwLuSdOM6vOtJ zscH8cSUp06o4KovdlW6anVUXoELxY+cH11GtsxIpqWmZAmm1aQdiQ6az2D56K/CevK/ KIvw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=dXIJFEJT; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com X-Forwarded-Encrypted: i=0; AJvYcCXyN8n3Q89OHBPtepBKrrgQQARjNG1uhNBLU/grm4qEGbpP742PeZqHyRPnPPnx1FZgQ9XgKa3NRYfsUqsqI2SIgNr5sq4LubTIaNtPPyjCAaW/VfARaXb3KmQpg5Wory806tzzWybUJoQRTDNl/KbRhBBltJ7Bk5ee/Di/a3XOh9xK4U09WNoRUc6odH1qzKeEMjkVWgpYIc5anjsnWSE+qPouyye5lmxplcBtwd+4yH4+aiUHnyNjFmoMrIVqJOIMupDfcuBXgLZrONgXJjA2Kyh1Ao7z8KSwyaJgAg1EGQTYgybRZCXhNbxkDgXheizoSTCfglf4YOUIGAZBszcSTp16LbnXHcZyewjqt12rBwfmn3ywLZ11j80fjzVKn/FUCH9EPoMWaRl70tU2X9S18a/fWuaZkFlVhGLKF3GkXJxJn+Q8lTcsv+U3tgZFDbMEECeIj/RAT+P+2TX/qx4faIKbybW9KJmYbdUR5SJwWNFp0RhjsoZdjmPXO4eBMxj2RiXvX4Q+jrxZmKBg5xaF3w/8YWNGxijetsLh8xMoaPRDETG2tuPcgPGvelUFz/FPyYhe/nCTQQ/6nBnJ932OLMz0/nwmIHGD/HK566Mpsw53lz0CRqCSLJIPybzSnreQoS4C/BRrDE3roUUP5V3GSOok9FAkgLje5Ppio8urSVH5OAqQ9bP215QqqlcjdllIdDsFEzaam9IE+InJUt4QnwrYvJm/Yqwfsi5NfW1CvOWby5JJyiUI2DMEjvTTVOkX3kciEqEA3XGTqklFYVQsPtyvUhVUFDJIhkI22bc6x+l9eo0QA+XPiZYRfEzFLugD7n4MvMZhUBIHukpZaPZnUiPYqwOu1T7pKVxhkWmjz1Rl+n2tC7MOLv9G5jKDRlkaJDjJ5Q8eahsUMhnKGysHD+eM7z/ioqXuQrwfFjQ7uUGYi3e+YC5b998bH8NjeXZA4E Y6Qy4kVYwlDPCMrhBRNY3Vmvqj5yOXpYb1B6kNM7GDdB3FzaKfzPbNfCNi0oMPDI3/TV9EgacpiCayP8wgtjCoBdBM1sY0yZ2M2MikqDjBX7u2eL+a6OgPCTiv68L08Yhj1Tv3J53fAV5hywvMqFHVWZGvMCrqb+6Re9V8XjF5wpiTCKHbxL8WYRs0AHVoMf4r7oBI84j7j2hYC1Ar/vgFC0Gs1l18CmnHgieZnFyS/p70/hbA2vV55eaDBY0LVF8gs21p8rtMGkW9pskxNc3wjfHXSxnwOtFXArd8QnbjZi1nW98JpPuAOF62ihaYjopmNEj9kbBcf0QA10iPKqkDeot8W+2HrIAL3edFdfhDAIKQOFOe8V7XwfFFO+3KNFVlyYIachZYPMBoO6G6KRlxa44p9Nd6HlcywOwILIYsWqHcaeV5e3ZILwiuB1uHD3y7RuhyyxXe4zJq+vTxp5Bs95b3QlTRd/s9heqoKQuRUw5+cYj14n6gFbwFNPomG84iPSskAafxj/WDnpsWVrQ/Vs4kWRghBAsLog2zbWNx9juELMZBUWf63RpXYMjiafEKUYtUD3xr+dpvYI/5Qt0CXFAZSUD9ov7UZ9qs/QmH/f/Q7KkgdisaDMIIOEmYtCr85s7Z7ABcmOpsuQ== Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h20-20020a170906111400b00a3707cc46a2si3241343eja.264.2024.02.04.21.27.53; Sun, 04 Feb 2024 21:27:53 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=dXIJFEJT; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id ED62B68D155; Mon, 5 Feb 2024 07:27:40 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oi1-f179.google.com (mail-oi1-f179.google.com [209.85.167.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E6E3E68C8CA for ; Mon, 5 Feb 2024 07:27:34 +0200 (EET) Received: by mail-oi1-f179.google.com with SMTP id 5614622812f47-3bd884146e9so3258586b6e.0 for ; Sun, 04 Feb 2024 21:27:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707110852; x=1707715652; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=83heej+m+Flkls8R3Ql0th3gAUYejDMeWpDy6jG3Rpo=; b=dXIJFEJTvNsu1tNWHwohZ8Djk9SKPIU2t/ifxQMTNZAkcJmzWXcO10m/UbEn9D+mtg SmgHNTOWsEtBI4VbcPY39v/RlcC6Q8SB44YmY2Il4UqqdEShFkhCj2E0Pq2/oPCL9zEO JxmaeeEcylciYmrU/fy/vVHqN/AWjVhrvdG1275jJ6kEarvA0BsHfMBaOTvmzdcudPEs 1v4UK8JFlQLefHMrN7BGG23WIhL1MaO8RguY6bXBuSlacwzvjLRez/0C+W8ZpdOoFKi/ ACrUvTFliSPLVaGOy0dwyF+ZxZ2bDTfjPiNHzrFP+jRneujNw60Cw9NQWa5Q0XPMbfRJ gnTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707110852; x=1707715652; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=83heej+m+Flkls8R3Ql0th3gAUYejDMeWpDy6jG3Rpo=; b=Gabje/yvynMd1kbtqIo7eUNKKug+0ncskmFQsYJaj5Jcjp0+jC9eH/ryWhATqAdET3 9LUHVRcf1ji1Q8JohHj6bsZN9akAwKoFvT0+dqV+UHzmONw22qIfA5SLj38W0tOc2JO9 R47QlNYTQ/Yl7u1SNuMZj78R0R6V825mR0tZTCHL2/N5UvbjnI89lxz6oLi1ek3ZWRxN mVXYwT47XxEU4P8L3PWgpYyLlmFrAkBxIzMvzleJODFHcw79Xqj/e3bP/5vQWSlnICuj dls5mmD3hWTjDHJ7zLWDjo3RbWfmMjry0TNilQli9Ezcruh39Hw82Eo6sRgbLTVyAu7W MWAg== X-Gm-Message-State: AOJu0YwYoDupm9IsaJJQbI92F1bZd90XleztQA1EvgZpYlZmQ85SxlqI m66qG5/pYh72dcMfp2eXQap8tKzbHnXWN9FJnupBv5kCRR/gH7/kHpNKDABk X-Received: by 2002:a05:6808:14d5:b0:3bf:dfa8:5190 with SMTP id f21-20020a05680814d500b003bfdfa85190mr561662oiw.10.1707110852026; Sun, 04 Feb 2024 21:27:32 -0800 (PST) Received: from localhost (c-24-4-236-173.hsd1.ca.comcast.net. [24.4.236.173]) by smtp.gmail.com with UTF8SMTPSA id p21-20020a056a000a1500b006e04a3a3df6sm550614pfh.185.2024.02.04.21.27.31 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 04 Feb 2024 21:27:31 -0800 (PST) From: Connor Worley To: ffmpeg-devel@ffmpeg.org Date: Sun, 4 Feb 2024 21:27:24 -0800 Message-Id: <20240205052724.26862-2-connorbworley@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240205052724.26862-1-connorbworley@gmail.com> References: <20240205052724.26862-1-connorbworley@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] lavc/dxvenc: migrate DXT1 encoder to lavu hashtable X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Connor Worley Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: nHLE0x3pOzxe Offers a modest performance gain due to the switch from naive linear probling to robin hood. Signed-off-by: Connor Worley --- libavcodec/dxvenc.c | 121 +++++++++++++------------------------------- 1 file changed, 35 insertions(+), 86 deletions(-) diff --git a/libavcodec/dxvenc.c b/libavcodec/dxvenc.c index b274175689..e17b3b2c36 100644 --- a/libavcodec/dxvenc.c +++ b/libavcodec/dxvenc.c @@ -21,7 +21,7 @@ #include -#include "libavutil/crc.h" +#include "libavutil/hashtable.h" #include "libavutil/imgutils.h" #include "libavutil/opt.h" @@ -44,69 +44,6 @@ enum DXVTextureFormat { DXV_FMT_DXT1 = MKBETAG('D', 'X', 'T', '1'), }; -typedef struct HTEntry { - uint32_t key; - uint32_t pos; -} HTEntry; - -static void ht_init(HTEntry *ht) -{ - for (size_t i = 0; i < LOOKBACK_HT_ELEMS; i++) { - ht[i].pos = -1; - } -} - -static uint32_t ht_lookup_and_upsert(HTEntry *ht, const AVCRC *hash_ctx, - uint32_t key, uint32_t pos) -{ - uint32_t ret = -1; - size_t hash = av_crc(hash_ctx, 0, (uint8_t*)&key, 4) % LOOKBACK_HT_ELEMS; - for (size_t i = hash; i < hash + LOOKBACK_HT_ELEMS; i++) { - size_t wrapped_index = i % LOOKBACK_HT_ELEMS; - HTEntry *entry = &ht[wrapped_index]; - if (entry->key == key || entry->pos == -1) { - ret = entry->pos; - entry->key = key; - entry->pos = pos; - break; - } - } - return ret; -} - -static void ht_delete(HTEntry *ht, const AVCRC *hash_ctx, - uint32_t key, uint32_t pos) -{ - HTEntry *removed_entry = NULL; - size_t removed_hash; - size_t hash = av_crc(hash_ctx, 0, (uint8_t*)&key, 4) % LOOKBACK_HT_ELEMS; - - for (size_t i = hash; i < hash + LOOKBACK_HT_ELEMS; i++) { - size_t wrapped_index = i % LOOKBACK_HT_ELEMS; - HTEntry *entry = &ht[wrapped_index]; - if (entry->pos == -1) - return; - if (removed_entry) { - size_t candidate_hash = av_crc(hash_ctx, 0, (uint8_t*)&entry->key, 4) % LOOKBACK_HT_ELEMS; - if ((wrapped_index > removed_hash && (candidate_hash <= removed_hash || candidate_hash > wrapped_index)) || - (wrapped_index < removed_hash && (candidate_hash <= removed_hash && candidate_hash > wrapped_index))) { - *removed_entry = *entry; - entry->pos = -1; - removed_entry = entry; - removed_hash = wrapped_index; - } - } else if (entry->key == key) { - if (entry->pos <= pos) { - entry->pos = -1; - removed_entry = entry; - removed_hash = wrapped_index; - } else { - return; - } - } - } -} - typedef struct DXVEncContext { AVClass *class; @@ -123,10 +60,8 @@ typedef struct DXVEncContext { enum DXVTextureFormat tex_fmt; int (*compress_tex)(AVCodecContext *avctx); - const AVCRC *crc_ctx; - - HTEntry color_lookback_ht[LOOKBACK_HT_ELEMS]; - HTEntry lut_lookback_ht[LOOKBACK_HT_ELEMS]; + AVHashtableContext color_ht; + AVHashtableContext lut_ht; } DXVEncContext; /* Converts an index offset value to a 2-bit opcode and pushes it to a stream. @@ -161,27 +96,32 @@ static int dxv_compress_dxt1(AVCodecContext *avctx) DXVEncContext *ctx = avctx->priv_data; PutByteContext *pbc = &ctx->pbc; uint32_t *value; - uint32_t color, lut, idx, color_idx, lut_idx, prev_pos, state = 16, pos = 2, op = 0; + uint32_t color, lut, idx, color_idx, lut_idx, prev_pos, state = 16, pos = 0, op = 0; - ht_init(ctx->color_lookback_ht); - ht_init(ctx->lut_lookback_ht); + av_hashtable_clear(&ctx->color_ht); + av_hashtable_clear(&ctx->lut_ht); bytestream2_put_le32(pbc, AV_RL32(ctx->tex_data)); + av_hashtable_set(&ctx->color_ht, ctx->tex_data, &pos); + pos++; bytestream2_put_le32(pbc, AV_RL32(ctx->tex_data + 4)); - - ht_lookup_and_upsert(ctx->color_lookback_ht, ctx->crc_ctx, AV_RL32(ctx->tex_data), 0); - ht_lookup_and_upsert(ctx->lut_lookback_ht, ctx->crc_ctx, AV_RL32(ctx->tex_data + 4), 1); + av_hashtable_set(&ctx->lut_ht, ctx->tex_data + 4, &pos); + pos++; while (pos + 2 <= ctx->tex_size / 4) { idx = 0; + color_idx = 0; + lut_idx = 0; color = AV_RL32(ctx->tex_data + pos * 4); - prev_pos = ht_lookup_and_upsert(ctx->color_lookback_ht, ctx->crc_ctx, color, pos); - color_idx = prev_pos != -1 ? pos - prev_pos : 0; + if (av_hashtable_get(&ctx->color_ht, &color, &prev_pos)) + color_idx = pos - prev_pos; + av_hashtable_set(&ctx->color_ht, &color, &pos); + if (pos >= LOOKBACK_WORDS) { uint32_t old_pos = pos - LOOKBACK_WORDS; - uint32_t old_color = AV_RL32(ctx->tex_data + old_pos * 4); - ht_delete(ctx->color_lookback_ht, ctx->crc_ctx, old_color, old_pos); + if (av_hashtable_get(&ctx->color_ht, ctx->tex_data + old_pos * 4, &prev_pos) && prev_pos <= old_pos) + av_hashtable_delete(&ctx->color_ht, ctx->tex_data + old_pos * 4); } pos++; @@ -190,13 +130,14 @@ static int dxv_compress_dxt1(AVCodecContext *avctx) idx = color_idx; } else { idx = 0; - prev_pos = ht_lookup_and_upsert(ctx->lut_lookback_ht, ctx->crc_ctx, lut, pos); - lut_idx = prev_pos != -1 ? pos - prev_pos : 0; + if (av_hashtable_get(&ctx->lut_ht, &lut, &prev_pos)) + lut_idx = pos - prev_pos; + av_hashtable_set(&ctx->lut_ht, &lut, &pos); } if (pos >= LOOKBACK_WORDS) { uint32_t old_pos = pos - LOOKBACK_WORDS; - uint32_t old_lut = AV_RL32(ctx->tex_data + old_pos * 4); - ht_delete(ctx->lut_lookback_ht, ctx->crc_ctx, old_lut, old_pos); + if (av_hashtable_get(&ctx->lut_ht, ctx->tex_data + old_pos * 4, &prev_pos) && prev_pos <= old_pos) + av_hashtable_delete(&ctx->lut_ht, ctx->tex_data + old_pos * 4); } pos++; @@ -298,10 +239,15 @@ static av_cold int dxv_init(AVCodecContext *avctx) return AVERROR(ENOMEM); } - ctx->crc_ctx = av_crc_get_table(AV_CRC_32_IEEE); - if (!ctx->crc_ctx) { - av_log(avctx, AV_LOG_ERROR, "Could not initialize CRC table.\n"); - return AVERROR_BUG; + ret = av_hashtable_init(&ctx->color_ht, sizeof(uint32_t), sizeof(uint32_t), LOOKBACK_HT_ELEMS); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Could not initialize color lookback table.\n"); + return ret; + } + ret = av_hashtable_init(&ctx->lut_ht, sizeof(uint32_t), sizeof(uint32_t), LOOKBACK_HT_ELEMS); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Could not initialize LUT lookback table.\n"); + return ret; } return 0; @@ -313,6 +259,9 @@ static av_cold int dxv_close(AVCodecContext *avctx) av_freep(&ctx->tex_data); + av_hashtable_destroy(&ctx->color_ht); + av_hashtable_destroy(&ctx->lut_ht); + return 0; }