From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <ffmpeg-devel-bounces@ffmpeg.org>
Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100])
	by master.gitmailbox.com (Postfix) with ESMTPS id 6A35444B2A
	for <ffmpegdev@gitmailbox.com>; Tue, 22 Apr 2025 07:43:34 +0000 (UTC)
Received: from [127.0.1.1] (localhost [127.0.0.1])
	by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9AE1E68A5B9;
	Tue, 22 Apr 2025 10:43:27 +0300 (EEST)
Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com
 [209.85.216.44])
 by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BC32468A48C
 for <ffmpeg-devel@ffmpeg.org>; Tue, 22 Apr 2025 10:43:20 +0300 (EEST)
Received: by mail-pj1-f44.google.com with SMTP id
 98e67ed59e1d1-30384072398so3923578a91.0
 for <ffmpeg-devel@ffmpeg.org>; Tue, 22 Apr 2025 00:43:20 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=emma-gg.20230601.gappssmtp.com; s=20230601; t=1745307798; x=1745912598;
 darn=ffmpeg.org; 
 h=content-transfer-encoding:mime-version:references:in-reply-to
 :message-id:date:subject:cc:to:from:from:to:cc:subject:date
 :message-id:reply-to;
 bh=Gw1AcOPQ6DrIRn2CCn9CQ9Bh+S3e/ImhP+8DD4PyNGs=;
 b=mm6y3WxKrrjQiHHfegWy3j0tKbjUd+o4m6CMa1msoE2MntE6IkPvyYZXlJu3QTxaV6
 9Ro9oXYH8cod9weo2IH3pwXEbAhoZ7/6tqmr5HmeePUjjkVdsLVAZFGZ0K++30xDOu3o
 XZs3nsOrJ/NXg8kDLUXktU43U1LJnfNnl18FTXZvuxu2yDQJzq3JB0rvCbZ1RnAIEYt1
 RT63Xs31Z6E8jiLU1fC2mEbn6DeIFwRlDyZFkp2q/9zRt7GtM4Cdu6eCPPzMp+DyEXEz
 lqwJTLloMSlbLu8yf0DGcOauCa7SN/mNRNRCFBvbwAwcMiKd0iA87Jkb/+MjVrzgjmc8
 miSA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1745307798; x=1745912598;
 h=content-transfer-encoding:mime-version:references:in-reply-to
 :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=Gw1AcOPQ6DrIRn2CCn9CQ9Bh+S3e/ImhP+8DD4PyNGs=;
 b=Sef4G8C40qImhgzsTv53miPPuL+q51a+7DKILQh9YvKLOGhDXlQnZ5VisXgWJecn5Y
 KCNg3C7NwApivAfAxyuE6KaT0Wuqq/rwLisissZVSg4W5ad8DYH6qWx5Su833LpHF87/
 kMiQ3G1yF3qL4phAJhqURqLiUWm7d09dX5zqEN44Ng95njaZtQmt+J2m7hxvg8TKJdXi
 Hhu3fWb8rLYrohymZlzoaMYYUyEsgaUioGJbgBs+uyIxMRde0XYH55ocgfhY8xu6CLNC
 aRfXlwuIUzqNmkhxdVtrECQ26NWPl6KkGhWM3dgyQzro8tfacg2nWfiP3XjImtFfEheY
 XjAw==
X-Gm-Message-State: AOJu0Yw503CahALLHtrBC8QZX06El34fGzv7K+/6ONSGJi4DjQufaG2S
 Uy2uDXQLFNvU3HqB1WsggHIXfxJdxr/tXwt0O1KojxpTZxXLx/+owfPNoY7wgxJf08NhF8zBCIT
 e
X-Gm-Gg: ASbGncvw9T+E0KDkQLplmyXUp57ms1Vjqi65zAIUtM8rsIMMUrKRSMEZaaVZZlBUlZ/
 012BY2QE2O2ndwdFLG0DSCeQdU10gn8EGjX5BgVp+5Vd1QNPF5ciHidFvhg+yPN4bpIjys8Ul1t
 ocAlcG45+o3HJSU7M2h4s7zBz2kWWas8hp/KtRDpVg6gTqiXACGpypFaxsRHsBqWWmMTBybjrCn
 kvnT8U/YkeRMtguoujqv9gbe0u4Pj0yo7uLVZzyboU0Bt2VRzodbfvP+H4qRei8HLxgS+E9ibpS
 Uphua/yY+reGmZWXjDTi6yk8nG50SkDnMeoOnxSlHFKevhpZjw==
X-Google-Smtp-Source: AGHT+IFynZ3/RnkiKER/v0I4Exz03l+BlA/+ue5ev1pkM3LPFPPNt0AqSIJ69SBb/zZVkNKCpXn4qg==
X-Received: by 2002:a17:90b:1c0a:b0:2fe:994d:613b with SMTP id
 98e67ed59e1d1-3087bbcb45dmr20203843a91.35.1745307798220; 
 Tue, 22 Apr 2025 00:43:18 -0700 (PDT)
Received: from localhost ([2600:1700:65a7:3600:e43f:b3c9:c97a:8ef3])
 by smtp.gmail.com with UTF8SMTPSA id
 98e67ed59e1d1-309d3bd1a2csm448971a91.1.2025.04.22.00.43.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 22 Apr 2025 00:43:17 -0700 (PDT)
From: Emma Worley <emma@emma.gg>
To: ffmpeg-devel@ffmpeg.org
Date: Tue, 22 Apr 2025 00:42:53 -0700
Message-ID: <20250422074254.77328-2-emma@emma.gg>
X-Mailer: git-send-email 2.48.0
In-Reply-To: <20250422074254.77328-1-emma@emma.gg>
References: <20250422074254.77328-1-emma@emma.gg>
MIME-Version: 1.0
Subject: [FFmpeg-devel] [PATCH v4 2/3] lavc/dxvenc: migrate DXT1 encoder to
 lavc hashtable
X-BeenThere: ffmpeg-devel@ffmpeg.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: FFmpeg development discussions and patches <ffmpeg-devel.ffmpeg.org>
List-Unsubscribe: <https://ffmpeg.org/mailman/options/ffmpeg-devel>,
 <mailto:ffmpeg-devel-request@ffmpeg.org?subject=unsubscribe>
List-Archive: <https://ffmpeg.org/pipermail/ffmpeg-devel>
List-Post: <mailto:ffmpeg-devel@ffmpeg.org>
List-Help: <mailto:ffmpeg-devel-request@ffmpeg.org?subject=help>
List-Subscribe: <https://ffmpeg.org/mailman/listinfo/ffmpeg-devel>,
 <mailto:ffmpeg-devel-request@ffmpeg.org?subject=subscribe>
Reply-To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Cc: Emma Worley <emma@emma.gg>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: ffmpeg-devel-bounces@ffmpeg.org
Sender: "ffmpeg-devel" <ffmpeg-devel-bounces@ffmpeg.org>
Archived-At: <https://master.gitmailbox.com/ffmpegdev/20250422074254.77328-2-emma@emma.gg/>
List-Archive: <https://master.gitmailbox.com/ffmpegdev/>
List-Post: <mailto:ffmpegdev@gitmailbox.com>

Offers a modest performance gain due to the switch from naive linear
probling to robin hood.

Signed-off-by: Emma Worley <emma@emma.gg>
---
 libavcodec/dxvenc.c | 121 ++++++++++++--------------------------------
 1 file changed, 33 insertions(+), 88 deletions(-)

diff --git a/libavcodec/dxvenc.c b/libavcodec/dxvenc.c
index 808d8daedb..207ac1f24f 100644
--- a/libavcodec/dxvenc.c
+++ b/libavcodec/dxvenc.c
@@ -21,7 +21,7 @@
 
 #include <stdint.h>
 
-#include "libavutil/crc.h"
+#include "libavcodec/hashtable.h"
 #include "libavutil/imgutils.h"
 #include "libavutil/mem.h"
 #include "libavutil/opt.h"
@@ -39,72 +39,9 @@
  * appeared in the decompressed stream. Using a simple hash table (HT)
  * significantly speeds up the lookback process while encoding.
  */
-#define LOOKBACK_HT_ELEMS 0x40000
+#define LOOKBACK_HT_ELEMS 0x20202
 #define LOOKBACK_WORDS    0x20202
 
-typedef struct HTEntry {
-    uint32_t key;
-    uint32_t pos;
-} HTEntry;
-
-static void ht_init(HTEntry *ht)
-{
-    for (size_t i = 0; i < LOOKBACK_HT_ELEMS; i++) {
-        ht[i].pos = -1;
-    }
-}
-
-static uint32_t ht_lookup_and_upsert(HTEntry *ht, const AVCRC *hash_ctx,
-                                    uint32_t key, uint32_t pos)
-{
-    uint32_t ret = -1;
-    size_t hash = av_crc(hash_ctx, 0, (uint8_t*)&key, 4) % LOOKBACK_HT_ELEMS;
-    for (size_t i = hash; i < hash + LOOKBACK_HT_ELEMS; i++) {
-        size_t wrapped_index = i % LOOKBACK_HT_ELEMS;
-        HTEntry *entry = &ht[wrapped_index];
-        if (entry->key == key || entry->pos == -1) {
-            ret = entry->pos;
-            entry->key = key;
-            entry->pos = pos;
-            break;
-        }
-    }
-    return ret;
-}
-
-static void ht_delete(HTEntry *ht, const AVCRC *hash_ctx,
-                      uint32_t key, uint32_t pos)
-{
-    HTEntry *removed_entry = NULL;
-    size_t removed_hash;
-    size_t hash = av_crc(hash_ctx, 0, (uint8_t*)&key, 4) % LOOKBACK_HT_ELEMS;
-
-    for (size_t i = hash; i < hash + LOOKBACK_HT_ELEMS; i++) {
-        size_t wrapped_index = i % LOOKBACK_HT_ELEMS;
-        HTEntry *entry = &ht[wrapped_index];
-        if (entry->pos == -1)
-            return;
-        if (removed_entry) {
-            size_t candidate_hash = av_crc(hash_ctx, 0, (uint8_t*)&entry->key, 4) % LOOKBACK_HT_ELEMS;
-            if ((wrapped_index > removed_hash && (candidate_hash <= removed_hash || candidate_hash > wrapped_index)) ||
-                (wrapped_index < removed_hash && (candidate_hash <= removed_hash && candidate_hash > wrapped_index))) {
-                *removed_entry = *entry;
-                entry->pos = -1;
-                removed_entry = entry;
-                removed_hash = wrapped_index;
-            }
-        } else if (entry->key == key) {
-            if (entry->pos <= pos) {
-                entry->pos = -1;
-                removed_entry = entry;
-                removed_hash = wrapped_index;
-            } else {
-                return;
-            }
-        }
-    }
-}
-
 typedef struct DXVEncContext {
     AVClass *class;
 
@@ -121,10 +58,8 @@ typedef struct DXVEncContext {
     DXVTextureFormat tex_fmt;
     int (*compress_tex)(AVCodecContext *avctx);
 
-    const AVCRC *crc_ctx;
-
-    HTEntry color_lookback_ht[LOOKBACK_HT_ELEMS];
-    HTEntry lut_lookback_ht[LOOKBACK_HT_ELEMS];
+    struct FFHashtableContext *color_ht;
+    struct FFHashtableContext *lut_ht;
 } DXVEncContext;
 
 /* Converts an index offset value to a 2-bit opcode and pushes it to a stream.
@@ -159,27 +94,32 @@ static int dxv_compress_dxt1(AVCodecContext *avctx)
     DXVEncContext *ctx = avctx->priv_data;
     PutByteContext *pbc = &ctx->pbc;
     void *value;
-    uint32_t color, lut, idx, color_idx, lut_idx, prev_pos, state = 16, pos = 2, op = 0;
+    uint32_t color, lut, idx, color_idx, lut_idx, prev_pos, state = 16, pos = 0, op = 0;
 
-    ht_init(ctx->color_lookback_ht);
-    ht_init(ctx->lut_lookback_ht);
+    ff_hashtable_clear(ctx->color_ht);
+    ff_hashtable_clear(ctx->lut_ht);
 
     bytestream2_put_le32(pbc, AV_RL32(ctx->tex_data));
+    ff_hashtable_set(ctx->color_ht, ctx->tex_data, &pos);
+    pos++;
     bytestream2_put_le32(pbc, AV_RL32(ctx->tex_data + 4));
-
-    ht_lookup_and_upsert(ctx->color_lookback_ht, ctx->crc_ctx, AV_RL32(ctx->tex_data), 0);
-    ht_lookup_and_upsert(ctx->lut_lookback_ht, ctx->crc_ctx, AV_RL32(ctx->tex_data + 4), 1);
+    ff_hashtable_set(ctx->lut_ht, ctx->tex_data + 4, &pos);
+    pos++;
 
     while (pos + 2 <= ctx->tex_size / 4) {
         idx = 0;
+        color_idx = 0;
+        lut_idx = 0;
 
         color = AV_RL32(ctx->tex_data + pos * 4);
-        prev_pos = ht_lookup_and_upsert(ctx->color_lookback_ht, ctx->crc_ctx, color, pos);
-        color_idx = prev_pos != -1 ? pos - prev_pos : 0;
+        if (ff_hashtable_get(ctx->color_ht, &color, &prev_pos))
+            color_idx = pos - prev_pos;
+        ff_hashtable_set(ctx->color_ht, &color, &pos);
+
         if (pos >= LOOKBACK_WORDS) {
             uint32_t old_pos = pos - LOOKBACK_WORDS;
-            uint32_t old_color = AV_RL32(ctx->tex_data + old_pos * 4);
-            ht_delete(ctx->color_lookback_ht, ctx->crc_ctx, old_color, old_pos);
+            if (ff_hashtable_get(ctx->color_ht, ctx->tex_data + old_pos * 4, &prev_pos) && prev_pos <= old_pos)
+                ff_hashtable_delete(ctx->color_ht, ctx->tex_data + old_pos * 4);
         }
         pos++;
 
@@ -188,13 +128,14 @@ static int dxv_compress_dxt1(AVCodecContext *avctx)
             idx = color_idx;
         } else {
             idx = 0;
-            prev_pos = ht_lookup_and_upsert(ctx->lut_lookback_ht, ctx->crc_ctx, lut, pos);
-            lut_idx = prev_pos != -1 ? pos - prev_pos : 0;
+            if (ff_hashtable_get(ctx->lut_ht, &lut, &prev_pos))
+                lut_idx = pos - prev_pos;
+            ff_hashtable_set(ctx->lut_ht, &lut, &pos);
         }
         if (pos >= LOOKBACK_WORDS) {
             uint32_t old_pos = pos - LOOKBACK_WORDS;
-            uint32_t old_lut = AV_RL32(ctx->tex_data + old_pos * 4);
-            ht_delete(ctx->lut_lookback_ht, ctx->crc_ctx, old_lut, old_pos);
+            if (ff_hashtable_get(ctx->lut_ht, ctx->tex_data + old_pos * 4, &prev_pos) && prev_pos <= old_pos)
+                ff_hashtable_delete(ctx->lut_ht, ctx->tex_data + old_pos * 4);
         }
         pos++;
 
@@ -306,11 +247,12 @@ static av_cold int dxv_init(AVCodecContext *avctx)
         return AVERROR(ENOMEM);
     }
 
-    ctx->crc_ctx = av_crc_get_table(AV_CRC_32_IEEE);
-    if (!ctx->crc_ctx) {
-        av_log(avctx, AV_LOG_ERROR, "Could not initialize CRC table.\n");
-        return AVERROR_BUG;
-    }
+    ret = ff_hashtable_alloc(&ctx->color_ht, sizeof(uint32_t), sizeof(uint32_t), LOOKBACK_HT_ELEMS);
+    if (ret < 0)
+        return ret;
+    ret = ff_hashtable_alloc(&ctx->lut_ht, sizeof(uint32_t), sizeof(uint32_t), LOOKBACK_HT_ELEMS);
+    if (ret < 0)
+        return ret;
 
     return 0;
 }
@@ -321,6 +263,9 @@ static av_cold int dxv_close(AVCodecContext *avctx)
 
     av_freep(&ctx->tex_data);
 
+    ff_hashtable_freep(&ctx->color_ht);
+    ff_hashtable_freep(&ctx->lut_ht);
+
     return 0;
 }
 
-- 
2.48.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".