From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <ffmpeg-devel-bounces@ffmpeg.org>
Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100])
by master.gitmailbox.com (Postfix) with ESMTPS id 55B044D7D6
for <ffmpegdev@gitmailbox.com>; Sun, 20 Apr 2025 13:09:19 +0000 (UTC)
Received: from [127.0.1.1] (localhost [127.0.0.1])
by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5C736687DBE;
Sun, 20 Apr 2025 16:09:11 +0300 (EEST)
Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com
[209.85.210.170])
by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8CD20687CF0
for <ffmpeg-devel@ffmpeg.org>; Sun, 20 Apr 2025 16:08:54 +0300 (EEST)
Received: by mail-pf1-f170.google.com with SMTP id
d2e1a72fcca58-7390d21bb1cso2820647b3a.2
for <ffmpeg-devel@ffmpeg.org>; Sun, 20 Apr 2025 06:08:54 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=emma-gg.20230601.gappssmtp.com; s=20230601; t=1745154532; x=1745759332;
darn=ffmpeg.org;
h=content-transfer-encoding:mime-version:references:in-reply-to
:message-id:date:subject:cc:to:from:from:to:cc:subject:date
:message-id:reply-to;
bh=HtXyJklU890KOf6LckPSf3ZT2GJpZTsJZRcH35xkmAs=;
b=YnS0/G75z123Yh/pAWLBSPcuflfEZ46xpzVM3dhQPpGFF93Fu/uPsdy1j8SJ87/3ML
p+ge5QDajPg6/5jpl9CCDBuIMjd2iD4NsBJcAHDhVEO4Fo4NqW2VyJxFHAdPy3Fue0WW
64o+xzJfuvc69SMREGT7zFT4r4/udOOGJmdM4Bk9V+STNteSlN0Tol+R0Qwm2hRee6ki
l0YpPsD3tWtUqU5w77mj0UpgAzfJ6GqxG5jB8p60PC5QxnH+H885fpNzPfwiF+kqiXLz
g9dO9e+K99/BPdA10Ye99LPZ/86k/jwX8fho6/Fk8m5NWRlpo04/21IZmKogZSPEXjkv
cNEQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1745154532; x=1745759332;
h=content-transfer-encoding:mime-version:references:in-reply-to
:message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=HtXyJklU890KOf6LckPSf3ZT2GJpZTsJZRcH35xkmAs=;
b=ZKekDmAvQxXkBbWrd+5pKH5v8iYdyFIQDN6KWe6NMVe/ZXC0ceulw7jPnm+Gt0yLwU
Tpf7W+rDx9dKFhcyRCCC1IXxm9u/5k7QpuApRtubXFMOTEl29fc76hOMg/HswBx9Dqtd
GY4kByDS4EHrwriGs4/k22B77Z3Q+XC57tQUHXZYlPhfAQLYeXZjUMOZ0EdGKfvlBtm4
mKFffzW3M/4Dh5mJWhe6+ikXTbVCQAHz3zgYjldBZ7nslYhb7d8ohetBaNMGjYaUQsLY
cq1yzDh6CGGKGoJ40fL/WjhrA6cm4hsYUkBYwiaYQROxHm3moiTk8v5BLhp6p+wrBfzm
meOw==
X-Gm-Message-State: AOJu0YyHazr4H+VP2VHIQQ330lBve0nnE6wwZGrde31cx4x8JyajLOwG
v6RuurMUhlkrTeMIoM64vGWqlXG8/xE4N2KMFQnmNGHTD0XM5c+Ms09kuXroBsfjX1Q6QqWMPle
z
X-Gm-Gg: ASbGncsDNbQzEcANF3vyC9w2f5f60P+r1hG05uhVEtOONS6X+mCT5/+J6Rg5boarX8W
8FDlLewZrjg1xdGi93NmrLCPFXuiMyRHkUSlRmuGwFVu1wSRmMIF2/yhByIBNqANHtQBOl+MAO0
HMJpPUhVNQ4IAjP0Y3P5963l4pt709J6yIN1oVX2l02B+tMl6hPeLssUCxs/hHrCGgzt0UU3ema
cm7HAiRGx2GeMK2hPB651yruADXy19WqJEhvZIycxfAy3WbVTGR3nPzdOHd2Wjfm4LxAbMh9n8t
z/oXIrbCQfzBdcc2kKyphvPV8JE5uNV3x4pjxSo/UyuzasJzc3PfFBEliAuY
X-Google-Smtp-Source: AGHT+IEm4uqzYg7+cMl/DbbvTE841D/7hlFoSX07RjX/VQWY6hYF5M3weE6922YXDRWSv4PJEqXK+A==
X-Received: by 2002:a05:6a00:4acc:b0:736:33fd:f57d with SMTP id
d2e1a72fcca58-73dc156a8b4mr10519391b3a.17.1745154532220;
Sun, 20 Apr 2025 06:08:52 -0700 (PDT)
Received: from localhost ([2600:1700:65a7:3600:b90a:a0ba:c740:60a5])
by smtp.gmail.com with UTF8SMTPSA id
d2e1a72fcca58-73dbf8c0200sm4681483b3a.1.2025.04.20.06.08.51
(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
Sun, 20 Apr 2025 06:08:51 -0700 (PDT)
From: Emma Worley <emma@emma.gg>
To: ffmpeg-devel@ffmpeg.org
Date: Sun, 20 Apr 2025 06:08:41 -0700
Message-ID: <20250420130841.46245-2-emma@emma.gg>
X-Mailer: git-send-email 2.48.0
In-Reply-To: <20250420130841.46245-1-emma@emma.gg>
References: <20250420130841.46245-1-emma@emma.gg>
MIME-Version: 1.0
Subject: [FFmpeg-devel] [PATCH 2/2] lavc/dxvenc: migrate DXT1 encoder to
lavu hashtable and improve Resolume compatibility
X-BeenThere: ffmpeg-devel@ffmpeg.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: FFmpeg development discussions and patches <ffmpeg-devel.ffmpeg.org>
List-Unsubscribe: <https://ffmpeg.org/mailman/options/ffmpeg-devel>,
<mailto:ffmpeg-devel-request@ffmpeg.org?subject=unsubscribe>
List-Archive: <https://ffmpeg.org/pipermail/ffmpeg-devel>
List-Post: <mailto:ffmpeg-devel@ffmpeg.org>
List-Help: <mailto:ffmpeg-devel-request@ffmpeg.org?subject=help>
List-Subscribe: <https://ffmpeg.org/mailman/listinfo/ffmpeg-devel>,
<mailto:ffmpeg-devel-request@ffmpeg.org?subject=subscribe>
Reply-To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Cc: Emma Worley <emma@emma.gg>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: ffmpeg-devel-bounces@ffmpeg.org
Sender: "ffmpeg-devel" <ffmpeg-devel-bounces@ffmpeg.org>
Archived-At: <https://master.gitmailbox.com/ffmpegdev/20250420130841.46245-2-emma@emma.gg/>
List-Archive: <https://master.gitmailbox.com/ffmpegdev/>
List-Post: <mailto:ffmpegdev@gitmailbox.com>
Offers a modest performance gain due to the switch from naive linear
probling to robin hood, as well as improved compatibility with Resolume.
Signed-off-by: Emma Worley <emma@emma.gg>
---
libavcodec/dxvenc.c | 190 +++++++++++++-----------------------
tests/ref/fate/dxv3enc-dxt1 | 2 +-
2 files changed, 69 insertions(+), 123 deletions(-)
diff --git a/libavcodec/dxvenc.c b/libavcodec/dxvenc.c
index 808d8daedb..b2972435a2 100644
--- a/libavcodec/dxvenc.c
+++ b/libavcodec/dxvenc.c
@@ -21,7 +21,7 @@
#include <stdint.h>
-#include "libavutil/crc.h"
+#include "libavutil/hashtable.h"
#include "libavutil/imgutils.h"
#include "libavutil/mem.h"
#include "libavutil/opt.h"
@@ -34,77 +34,19 @@
#define DXV_HEADER_LENGTH 12
+/*
+ * Resolume will refuse to display frames that are not padded to 16x16 pixels.
+ */
+#define DXV_PAD(x) (x + 15 & ~15)
+
/*
* DXV uses LZ-like back-references to avoid copying words that have already
* appeared in the decompressed stream. Using a simple hash table (HT)
* significantly speeds up the lookback process while encoding.
*/
-#define LOOKBACK_HT_ELEMS 0x40000
+#define LOOKBACK_HT_ELEMS 0x20202
#define LOOKBACK_WORDS 0x20202
-typedef struct HTEntry {
- uint32_t key;
- uint32_t pos;
-} HTEntry;
-
-static void ht_init(HTEntry *ht)
-{
- for (size_t i = 0; i < LOOKBACK_HT_ELEMS; i++) {
- ht[i].pos = -1;
- }
-}
-
-static uint32_t ht_lookup_and_upsert(HTEntry *ht, const AVCRC *hash_ctx,
- uint32_t key, uint32_t pos)
-{
- uint32_t ret = -1;
- size_t hash = av_crc(hash_ctx, 0, (uint8_t*)&key, 4) % LOOKBACK_HT_ELEMS;
- for (size_t i = hash; i < hash + LOOKBACK_HT_ELEMS; i++) {
- size_t wrapped_index = i % LOOKBACK_HT_ELEMS;
- HTEntry *entry = &ht[wrapped_index];
- if (entry->key == key || entry->pos == -1) {
- ret = entry->pos;
- entry->key = key;
- entry->pos = pos;
- break;
- }
- }
- return ret;
-}
-
-static void ht_delete(HTEntry *ht, const AVCRC *hash_ctx,
- uint32_t key, uint32_t pos)
-{
- HTEntry *removed_entry = NULL;
- size_t removed_hash;
- size_t hash = av_crc(hash_ctx, 0, (uint8_t*)&key, 4) % LOOKBACK_HT_ELEMS;
-
- for (size_t i = hash; i < hash + LOOKBACK_HT_ELEMS; i++) {
- size_t wrapped_index = i % LOOKBACK_HT_ELEMS;
- HTEntry *entry = &ht[wrapped_index];
- if (entry->pos == -1)
- return;
- if (removed_entry) {
- size_t candidate_hash = av_crc(hash_ctx, 0, (uint8_t*)&entry->key, 4) % LOOKBACK_HT_ELEMS;
- if ((wrapped_index > removed_hash && (candidate_hash <= removed_hash || candidate_hash > wrapped_index)) ||
- (wrapped_index < removed_hash && (candidate_hash <= removed_hash && candidate_hash > wrapped_index))) {
- *removed_entry = *entry;
- entry->pos = -1;
- removed_entry = entry;
- removed_hash = wrapped_index;
- }
- } else if (entry->key == key) {
- if (entry->pos <= pos) {
- entry->pos = -1;
- removed_entry = entry;
- removed_hash = wrapped_index;
- } else {
- return;
- }
- }
- }
-}
-
typedef struct DXVEncContext {
AVClass *class;
@@ -121,10 +63,9 @@ typedef struct DXVEncContext {
DXVTextureFormat tex_fmt;
int (*compress_tex)(AVCodecContext *avctx);
- const AVCRC *crc_ctx;
-
- HTEntry color_lookback_ht[LOOKBACK_HT_ELEMS];
- HTEntry lut_lookback_ht[LOOKBACK_HT_ELEMS];
+ struct AVHashtableContext *color_ht;
+ struct AVHashtableContext *lut_ht;
+ struct AVHashtableContext *combo_ht;
} DXVEncContext;
/* Converts an index offset value to a 2-bit opcode and pushes it to a stream.
@@ -159,58 +100,63 @@ static int dxv_compress_dxt1(AVCodecContext *avctx)
DXVEncContext *ctx = avctx->priv_data;
PutByteContext *pbc = &ctx->pbc;
void *value;
- uint32_t color, lut, idx, color_idx, lut_idx, prev_pos, state = 16, pos = 2, op = 0;
+ uint64_t combo;
+ uint32_t color, lut, idx, combo_idx, prev_pos, old_pos, state = 16, pos = 0, op = 0;
- ht_init(ctx->color_lookback_ht);
- ht_init(ctx->lut_lookback_ht);
+ av_hashtable_clear(ctx->color_ht);
+ av_hashtable_clear(ctx->lut_ht);
+ av_hashtable_clear(ctx->combo_ht);
+
+ av_hashtable_set(ctx->combo_ht, ctx->tex_data, &pos);
bytestream2_put_le32(pbc, AV_RL32(ctx->tex_data));
+ av_hashtable_set(ctx->color_ht, ctx->tex_data, &pos);
+ pos++;
bytestream2_put_le32(pbc, AV_RL32(ctx->tex_data + 4));
-
- ht_lookup_and_upsert(ctx->color_lookback_ht, ctx->crc_ctx, AV_RL32(ctx->tex_data), 0);
- ht_lookup_and_upsert(ctx->lut_lookback_ht, ctx->crc_ctx, AV_RL32(ctx->tex_data + 4), 1);
+ av_hashtable_set(ctx->lut_ht, ctx->tex_data + 4, &pos);
+ pos++;
while (pos + 2 <= ctx->tex_size / 4) {
- idx = 0;
-
- color = AV_RL32(ctx->tex_data + pos * 4);
- prev_pos = ht_lookup_and_upsert(ctx->color_lookback_ht, ctx->crc_ctx, color, pos);
- color_idx = prev_pos != -1 ? pos - prev_pos : 0;
+ combo = AV_RL64(ctx->tex_data + pos * 4);
+ combo_idx = av_hashtable_get(ctx->combo_ht, &combo, &prev_pos) ? pos - prev_pos : 0;
+ idx = combo_idx;
+ PUSH_OP(2);
if (pos >= LOOKBACK_WORDS) {
- uint32_t old_pos = pos - LOOKBACK_WORDS;
- uint32_t old_color = AV_RL32(ctx->tex_data + old_pos * 4);
- ht_delete(ctx->color_lookback_ht, ctx->crc_ctx, old_color, old_pos);
+ old_pos = pos - LOOKBACK_WORDS;
+ if (av_hashtable_get(ctx->combo_ht, ctx->tex_data + old_pos * 4, &prev_pos) && prev_pos <= old_pos)
+ av_hashtable_delete(ctx->combo_ht, ctx->tex_data + old_pos * 4);
}
- pos++;
+ av_hashtable_set(ctx->combo_ht, &combo, &pos);
- lut = AV_RL32(ctx->tex_data + pos * 4);
- if (color_idx && lut == AV_RL32(ctx->tex_data + (pos - color_idx) * 4)) {
- idx = color_idx;
- } else {
- idx = 0;
- prev_pos = ht_lookup_and_upsert(ctx->lut_lookback_ht, ctx->crc_ctx, lut, pos);
- lut_idx = prev_pos != -1 ? pos - prev_pos : 0;
+ color = AV_RL32(ctx->tex_data + pos * 4);
+ if (!combo_idx) {
+ idx = av_hashtable_get(ctx->color_ht, &color, &prev_pos) ? pos - prev_pos : 0;
+ PUSH_OP(2);
+ if (!idx)
+ bytestream2_put_le32(pbc, color);
}
if (pos >= LOOKBACK_WORDS) {
- uint32_t old_pos = pos - LOOKBACK_WORDS;
- uint32_t old_lut = AV_RL32(ctx->tex_data + old_pos * 4);
- ht_delete(ctx->lut_lookback_ht, ctx->crc_ctx, old_lut, old_pos);
+ old_pos = pos - LOOKBACK_WORDS;
+ if (av_hashtable_get(ctx->color_ht, ctx->tex_data + old_pos * 4, &prev_pos) && prev_pos <= old_pos)
+ av_hashtable_delete(ctx->color_ht, ctx->tex_data + old_pos * 4);
}
+ av_hashtable_set(ctx->color_ht, &color, &pos);
pos++;
- PUSH_OP(2);
-
- if (!idx) {
- idx = color_idx;
- PUSH_OP(2);
- if (!idx)
- bytestream2_put_le32(pbc, color);
-
- idx = lut_idx;
+ lut = AV_RL32(ctx->tex_data + pos * 4);
+ if (!combo_idx) {
+ idx = av_hashtable_get(ctx->lut_ht, &lut, &prev_pos) ? pos - prev_pos : 0;
PUSH_OP(2);
if (!idx)
- bytestream2_put_le32(pbc, lut);
+ bytestream2_put_le32(pbc, lut);
}
+ if (pos >= LOOKBACK_WORDS) {
+ old_pos = pos - LOOKBACK_WORDS;
+ if (av_hashtable_get(ctx->lut_ht, ctx->tex_data + old_pos * 4, &prev_pos) && prev_pos <= old_pos)
+ av_hashtable_delete(ctx->lut_ht, ctx->tex_data + old_pos * 4);
+ }
+ av_hashtable_set(ctx->lut_ht, &lut, &pos);
+ pos++;
}
return 0;
@@ -234,8 +180,8 @@ static int dxv_encode(AVCodecContext *avctx, AVPacket *pkt,
ctx->enc.tex_data.out = ctx->tex_data;
ctx->enc.frame_data.in = frame->data[0];
ctx->enc.stride = frame->linesize[0];
- ctx->enc.width = avctx->width;
- ctx->enc.height = avctx->height;
+ ctx->enc.width = DXV_PAD(avctx->width);
+ ctx->enc.height = DXV_PAD(avctx->height);
ff_texturedsp_exec_compress_threads(avctx, &ctx->enc);
} else {
/* unimplemented: YCoCg formats */
@@ -275,14 +221,6 @@ static av_cold int dxv_init(AVCodecContext *avctx)
return ret;
}
- if (avctx->width % TEXTURE_BLOCK_W || avctx->height % TEXTURE_BLOCK_H) {
- av_log(avctx,
- AV_LOG_ERROR,
- "Video size %dx%d is not multiple of "AV_STRINGIFY(TEXTURE_BLOCK_W)"x"AV_STRINGIFY(TEXTURE_BLOCK_H)".\n",
- avctx->width, avctx->height);
- return AVERROR_INVALIDDATA;
- }
-
ff_texturedspenc_init(&texdsp);
switch (ctx->tex_fmt) {
@@ -296,21 +234,25 @@ static av_cold int dxv_init(AVCodecContext *avctx)
return AVERROR_INVALIDDATA;
}
ctx->enc.raw_ratio = 16;
- ctx->tex_size = avctx->width / TEXTURE_BLOCK_W *
- avctx->height / TEXTURE_BLOCK_H *
+ ctx->tex_size = DXV_PAD(avctx->width) / TEXTURE_BLOCK_W *
+ DXV_PAD(avctx->height) / TEXTURE_BLOCK_H *
ctx->enc.tex_ratio;
- ctx->enc.slice_count = av_clip(avctx->thread_count, 1, avctx->height / TEXTURE_BLOCK_H);
+ ctx->enc.slice_count = av_clip(avctx->thread_count, 1, DXV_PAD(avctx->height) / TEXTURE_BLOCK_H);
ctx->tex_data = av_malloc(ctx->tex_size);
if (!ctx->tex_data) {
return AVERROR(ENOMEM);
}
- ctx->crc_ctx = av_crc_get_table(AV_CRC_32_IEEE);
- if (!ctx->crc_ctx) {
- av_log(avctx, AV_LOG_ERROR, "Could not initialize CRC table.\n");
- return AVERROR_BUG;
- }
+ ret = av_hashtable_alloc(&ctx->color_ht, sizeof(uint32_t), sizeof(uint32_t), LOOKBACK_HT_ELEMS);
+ if (ret < 0)
+ return ret;
+ ret = av_hashtable_alloc(&ctx->lut_ht, sizeof(uint32_t), sizeof(uint32_t), LOOKBACK_HT_ELEMS);
+ if (ret < 0)
+ return ret;
+ ret = av_hashtable_alloc(&ctx->combo_ht, sizeof(uint64_t), sizeof(uint32_t), LOOKBACK_HT_ELEMS);
+ if (ret < 0)
+ return ret;
return 0;
}
@@ -321,6 +263,10 @@ static av_cold int dxv_close(AVCodecContext *avctx)
av_freep(&ctx->tex_data);
+ av_hashtable_freep(&ctx->color_ht);
+ av_hashtable_freep(&ctx->lut_ht);
+ av_hashtable_freep(&ctx->combo_ht);
+
return 0;
}
diff --git a/tests/ref/fate/dxv3enc-dxt1 b/tests/ref/fate/dxv3enc-dxt1
index 74849a8031..1e1b86aa6c 100644
--- a/tests/ref/fate/dxv3enc-dxt1
+++ b/tests/ref/fate/dxv3enc-dxt1
@@ -3,4 +3,4 @@
#codec_id 0: dxv
#dimensions 0: 1920x1080
#sar 0: 1/1
-0, 0, 0, 1, 76521, 0xed387a5e
+0, 0, 0, 1, 76563, 0x6b028fff
--
2.48.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".