From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <ffmpeg-devel-bounces@ffmpeg.org> Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id 6A35444B2A for <ffmpegdev@gitmailbox.com>; Tue, 22 Apr 2025 07:43:34 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9AE1E68A5B9; Tue, 22 Apr 2025 10:43:27 +0300 (EEST) Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BC32468A48C for <ffmpeg-devel@ffmpeg.org>; Tue, 22 Apr 2025 10:43:20 +0300 (EEST) Received: by mail-pj1-f44.google.com with SMTP id 98e67ed59e1d1-30384072398so3923578a91.0 for <ffmpeg-devel@ffmpeg.org>; Tue, 22 Apr 2025 00:43:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=emma-gg.20230601.gappssmtp.com; s=20230601; t=1745307798; x=1745912598; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Gw1AcOPQ6DrIRn2CCn9CQ9Bh+S3e/ImhP+8DD4PyNGs=; b=mm6y3WxKrrjQiHHfegWy3j0tKbjUd+o4m6CMa1msoE2MntE6IkPvyYZXlJu3QTxaV6 9Ro9oXYH8cod9weo2IH3pwXEbAhoZ7/6tqmr5HmeePUjjkVdsLVAZFGZ0K++30xDOu3o XZs3nsOrJ/NXg8kDLUXktU43U1LJnfNnl18FTXZvuxu2yDQJzq3JB0rvCbZ1RnAIEYt1 RT63Xs31Z6E8jiLU1fC2mEbn6DeIFwRlDyZFkp2q/9zRt7GtM4Cdu6eCPPzMp+DyEXEz lqwJTLloMSlbLu8yf0DGcOauCa7SN/mNRNRCFBvbwAwcMiKd0iA87Jkb/+MjVrzgjmc8 miSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745307798; x=1745912598; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Gw1AcOPQ6DrIRn2CCn9CQ9Bh+S3e/ImhP+8DD4PyNGs=; b=Sef4G8C40qImhgzsTv53miPPuL+q51a+7DKILQh9YvKLOGhDXlQnZ5VisXgWJecn5Y KCNg3C7NwApivAfAxyuE6KaT0Wuqq/rwLisissZVSg4W5ad8DYH6qWx5Su833LpHF87/ kMiQ3G1yF3qL4phAJhqURqLiUWm7d09dX5zqEN44Ng95njaZtQmt+J2m7hxvg8TKJdXi Hhu3fWb8rLYrohymZlzoaMYYUyEsgaUioGJbgBs+uyIxMRde0XYH55ocgfhY8xu6CLNC aRfXlwuIUzqNmkhxdVtrECQ26NWPl6KkGhWM3dgyQzro8tfacg2nWfiP3XjImtFfEheY XjAw== X-Gm-Message-State: AOJu0Yw503CahALLHtrBC8QZX06El34fGzv7K+/6ONSGJi4DjQufaG2S Uy2uDXQLFNvU3HqB1WsggHIXfxJdxr/tXwt0O1KojxpTZxXLx/+owfPNoY7wgxJf08NhF8zBCIT e X-Gm-Gg: ASbGncvw9T+E0KDkQLplmyXUp57ms1Vjqi65zAIUtM8rsIMMUrKRSMEZaaVZZlBUlZ/ 012BY2QE2O2ndwdFLG0DSCeQdU10gn8EGjX5BgVp+5Vd1QNPF5ciHidFvhg+yPN4bpIjys8Ul1t ocAlcG45+o3HJSU7M2h4s7zBz2kWWas8hp/KtRDpVg6gTqiXACGpypFaxsRHsBqWWmMTBybjrCn kvnT8U/YkeRMtguoujqv9gbe0u4Pj0yo7uLVZzyboU0Bt2VRzodbfvP+H4qRei8HLxgS+E9ibpS Uphua/yY+reGmZWXjDTi6yk8nG50SkDnMeoOnxSlHFKevhpZjw== X-Google-Smtp-Source: AGHT+IFynZ3/RnkiKER/v0I4Exz03l+BlA/+ue5ev1pkM3LPFPPNt0AqSIJ69SBb/zZVkNKCpXn4qg== X-Received: by 2002:a17:90b:1c0a:b0:2fe:994d:613b with SMTP id 98e67ed59e1d1-3087bbcb45dmr20203843a91.35.1745307798220; Tue, 22 Apr 2025 00:43:18 -0700 (PDT) Received: from localhost ([2600:1700:65a7:3600:e43f:b3c9:c97a:8ef3]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-309d3bd1a2csm448971a91.1.2025.04.22.00.43.17 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 22 Apr 2025 00:43:17 -0700 (PDT) From: Emma Worley <emma@emma.gg> To: ffmpeg-devel@ffmpeg.org Date: Tue, 22 Apr 2025 00:42:53 -0700 Message-ID: <20250422074254.77328-2-emma@emma.gg> X-Mailer: git-send-email 2.48.0 In-Reply-To: <20250422074254.77328-1-emma@emma.gg> References: <20250422074254.77328-1-emma@emma.gg> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 2/3] lavc/dxvenc: migrate DXT1 encoder to lavc hashtable X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches <ffmpeg-devel.ffmpeg.org> List-Unsubscribe: <https://ffmpeg.org/mailman/options/ffmpeg-devel>, <mailto:ffmpeg-devel-request@ffmpeg.org?subject=unsubscribe> List-Archive: <https://ffmpeg.org/pipermail/ffmpeg-devel> List-Post: <mailto:ffmpeg-devel@ffmpeg.org> List-Help: <mailto:ffmpeg-devel-request@ffmpeg.org?subject=help> List-Subscribe: <https://ffmpeg.org/mailman/listinfo/ffmpeg-devel>, <mailto:ffmpeg-devel-request@ffmpeg.org?subject=subscribe> Reply-To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org> Cc: Emma Worley <emma@emma.gg> Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" <ffmpeg-devel-bounces@ffmpeg.org> Archived-At: <https://master.gitmailbox.com/ffmpegdev/20250422074254.77328-2-emma@emma.gg/> List-Archive: <https://master.gitmailbox.com/ffmpegdev/> List-Post: <mailto:ffmpegdev@gitmailbox.com> Offers a modest performance gain due to the switch from naive linear probling to robin hood. Signed-off-by: Emma Worley <emma@emma.gg> --- libavcodec/dxvenc.c | 121 ++++++++++++-------------------------------- 1 file changed, 33 insertions(+), 88 deletions(-) diff --git a/libavcodec/dxvenc.c b/libavcodec/dxvenc.c index 808d8daedb..207ac1f24f 100644 --- a/libavcodec/dxvenc.c +++ b/libavcodec/dxvenc.c @@ -21,7 +21,7 @@ #include <stdint.h> -#include "libavutil/crc.h" +#include "libavcodec/hashtable.h" #include "libavutil/imgutils.h" #include "libavutil/mem.h" #include "libavutil/opt.h" @@ -39,72 +39,9 @@ * appeared in the decompressed stream. Using a simple hash table (HT) * significantly speeds up the lookback process while encoding. */ -#define LOOKBACK_HT_ELEMS 0x40000 +#define LOOKBACK_HT_ELEMS 0x20202 #define LOOKBACK_WORDS 0x20202 -typedef struct HTEntry { - uint32_t key; - uint32_t pos; -} HTEntry; - -static void ht_init(HTEntry *ht) -{ - for (size_t i = 0; i < LOOKBACK_HT_ELEMS; i++) { - ht[i].pos = -1; - } -} - -static uint32_t ht_lookup_and_upsert(HTEntry *ht, const AVCRC *hash_ctx, - uint32_t key, uint32_t pos) -{ - uint32_t ret = -1; - size_t hash = av_crc(hash_ctx, 0, (uint8_t*)&key, 4) % LOOKBACK_HT_ELEMS; - for (size_t i = hash; i < hash + LOOKBACK_HT_ELEMS; i++) { - size_t wrapped_index = i % LOOKBACK_HT_ELEMS; - HTEntry *entry = &ht[wrapped_index]; - if (entry->key == key || entry->pos == -1) { - ret = entry->pos; - entry->key = key; - entry->pos = pos; - break; - } - } - return ret; -} - -static void ht_delete(HTEntry *ht, const AVCRC *hash_ctx, - uint32_t key, uint32_t pos) -{ - HTEntry *removed_entry = NULL; - size_t removed_hash; - size_t hash = av_crc(hash_ctx, 0, (uint8_t*)&key, 4) % LOOKBACK_HT_ELEMS; - - for (size_t i = hash; i < hash + LOOKBACK_HT_ELEMS; i++) { - size_t wrapped_index = i % LOOKBACK_HT_ELEMS; - HTEntry *entry = &ht[wrapped_index]; - if (entry->pos == -1) - return; - if (removed_entry) { - size_t candidate_hash = av_crc(hash_ctx, 0, (uint8_t*)&entry->key, 4) % LOOKBACK_HT_ELEMS; - if ((wrapped_index > removed_hash && (candidate_hash <= removed_hash || candidate_hash > wrapped_index)) || - (wrapped_index < removed_hash && (candidate_hash <= removed_hash && candidate_hash > wrapped_index))) { - *removed_entry = *entry; - entry->pos = -1; - removed_entry = entry; - removed_hash = wrapped_index; - } - } else if (entry->key == key) { - if (entry->pos <= pos) { - entry->pos = -1; - removed_entry = entry; - removed_hash = wrapped_index; - } else { - return; - } - } - } -} - typedef struct DXVEncContext { AVClass *class; @@ -121,10 +58,8 @@ typedef struct DXVEncContext { DXVTextureFormat tex_fmt; int (*compress_tex)(AVCodecContext *avctx); - const AVCRC *crc_ctx; - - HTEntry color_lookback_ht[LOOKBACK_HT_ELEMS]; - HTEntry lut_lookback_ht[LOOKBACK_HT_ELEMS]; + struct FFHashtableContext *color_ht; + struct FFHashtableContext *lut_ht; } DXVEncContext; /* Converts an index offset value to a 2-bit opcode and pushes it to a stream. @@ -159,27 +94,32 @@ static int dxv_compress_dxt1(AVCodecContext *avctx) DXVEncContext *ctx = avctx->priv_data; PutByteContext *pbc = &ctx->pbc; void *value; - uint32_t color, lut, idx, color_idx, lut_idx, prev_pos, state = 16, pos = 2, op = 0; + uint32_t color, lut, idx, color_idx, lut_idx, prev_pos, state = 16, pos = 0, op = 0; - ht_init(ctx->color_lookback_ht); - ht_init(ctx->lut_lookback_ht); + ff_hashtable_clear(ctx->color_ht); + ff_hashtable_clear(ctx->lut_ht); bytestream2_put_le32(pbc, AV_RL32(ctx->tex_data)); + ff_hashtable_set(ctx->color_ht, ctx->tex_data, &pos); + pos++; bytestream2_put_le32(pbc, AV_RL32(ctx->tex_data + 4)); - - ht_lookup_and_upsert(ctx->color_lookback_ht, ctx->crc_ctx, AV_RL32(ctx->tex_data), 0); - ht_lookup_and_upsert(ctx->lut_lookback_ht, ctx->crc_ctx, AV_RL32(ctx->tex_data + 4), 1); + ff_hashtable_set(ctx->lut_ht, ctx->tex_data + 4, &pos); + pos++; while (pos + 2 <= ctx->tex_size / 4) { idx = 0; + color_idx = 0; + lut_idx = 0; color = AV_RL32(ctx->tex_data + pos * 4); - prev_pos = ht_lookup_and_upsert(ctx->color_lookback_ht, ctx->crc_ctx, color, pos); - color_idx = prev_pos != -1 ? pos - prev_pos : 0; + if (ff_hashtable_get(ctx->color_ht, &color, &prev_pos)) + color_idx = pos - prev_pos; + ff_hashtable_set(ctx->color_ht, &color, &pos); + if (pos >= LOOKBACK_WORDS) { uint32_t old_pos = pos - LOOKBACK_WORDS; - uint32_t old_color = AV_RL32(ctx->tex_data + old_pos * 4); - ht_delete(ctx->color_lookback_ht, ctx->crc_ctx, old_color, old_pos); + if (ff_hashtable_get(ctx->color_ht, ctx->tex_data + old_pos * 4, &prev_pos) && prev_pos <= old_pos) + ff_hashtable_delete(ctx->color_ht, ctx->tex_data + old_pos * 4); } pos++; @@ -188,13 +128,14 @@ static int dxv_compress_dxt1(AVCodecContext *avctx) idx = color_idx; } else { idx = 0; - prev_pos = ht_lookup_and_upsert(ctx->lut_lookback_ht, ctx->crc_ctx, lut, pos); - lut_idx = prev_pos != -1 ? pos - prev_pos : 0; + if (ff_hashtable_get(ctx->lut_ht, &lut, &prev_pos)) + lut_idx = pos - prev_pos; + ff_hashtable_set(ctx->lut_ht, &lut, &pos); } if (pos >= LOOKBACK_WORDS) { uint32_t old_pos = pos - LOOKBACK_WORDS; - uint32_t old_lut = AV_RL32(ctx->tex_data + old_pos * 4); - ht_delete(ctx->lut_lookback_ht, ctx->crc_ctx, old_lut, old_pos); + if (ff_hashtable_get(ctx->lut_ht, ctx->tex_data + old_pos * 4, &prev_pos) && prev_pos <= old_pos) + ff_hashtable_delete(ctx->lut_ht, ctx->tex_data + old_pos * 4); } pos++; @@ -306,11 +247,12 @@ static av_cold int dxv_init(AVCodecContext *avctx) return AVERROR(ENOMEM); } - ctx->crc_ctx = av_crc_get_table(AV_CRC_32_IEEE); - if (!ctx->crc_ctx) { - av_log(avctx, AV_LOG_ERROR, "Could not initialize CRC table.\n"); - return AVERROR_BUG; - } + ret = ff_hashtable_alloc(&ctx->color_ht, sizeof(uint32_t), sizeof(uint32_t), LOOKBACK_HT_ELEMS); + if (ret < 0) + return ret; + ret = ff_hashtable_alloc(&ctx->lut_ht, sizeof(uint32_t), sizeof(uint32_t), LOOKBACK_HT_ELEMS); + if (ret < 0) + return ret; return 0; } @@ -321,6 +263,9 @@ static av_cold int dxv_close(AVCodecContext *avctx) av_freep(&ctx->tex_data); + ff_hashtable_freep(&ctx->color_ht); + ff_hashtable_freep(&ctx->lut_ht); + return 0; } -- 2.48.0 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".