Lock-free Concurrent Level Hashing for Persistent Memory Zhangyu - - PowerPoint PPT Presentation
Lock-free Concurrent Level Hashing for Persistent Memory Zhangyu Chen , Yu Hua, Bo Ding, Pengfei Zuo Huazhong University of Science and Technology USENIX ATC 2020 Persistent Memory (PM) PM features Non-volatility Byte-addressability
USENIX ATC 2020
2
Intel Optane DC Persistent Memory 512 GB per module at most DIMM compatible
3
4
CPU Cache
Volatile Persistent
Bus
Partial update
4
CPU Cache
Volatile Persistent
Bus
Partial update data (32 B)
4
CPU Cache
Volatile Persistent
Bus 8-byte atomic write
Partial update
4
CPU Cache
Volatile Persistent
Bus
Partial update
4
CPU Cache
Volatile Persistent
Bus
Inconsistency Partial update
4
CPU Cache
Volatile Persistent
Bus
Inconsistency
CPU Cache
clwb sfence
Bus
Partial update
4
CPU Cache
Volatile Persistent
Bus
Inconsistency
CPU Cache
clwb sfence
Bus
Partial update
4
CPU Cache
Volatile Persistent
Bus
Inconsistency
CPU Cache
clwb sfence
Bus
Partial update
4
CPU Cache
Volatile Persistent
Bus
Inconsistency
CPU Cache
clwb sfence
Bus
Partial update
5
CPU Cache Bus
slots Program order slots
5
CPU Cache Bus
kv_t item = new kv_t(k, v); slots[0] = &item;
slots item Program order slots slots
5
CPU Cache Bus
kv_t item = new kv_t(k, v); slots[0] = &item;
slots item Cache Reordering Program order slots slots slots
5
CPU Cache Bus
kv_t item = new kv_t(k, v); slots[0] = &item;
Cache Reordering Program order slots slots Inconsistency
5
CPU Cache Bus
kv_t item = new kv_t(k, v); slots[0] = &item;
Program order slots slots Inconsistency
clwb(item); sfence;
6
6
key
6
Hash(key)
key key
6
Hash(key)
key key
7
x
Hash(y)
y Collision
7
x y
Linear probing
probing distance
7
x y
Linear probing
x y
Linked list
probing distance
7
Old hash table
x y
Linear probing
x y
Linked list
probing distance
7
Old hash table New hash table Rehashing
x y
Linear probing
x y
Linked list
probing distance
8
Writers Readers Concurrent resizing
9
112 102 012 002
... ... ... ...
Bucket 0 Bucket 1 Bucket 254 Bucket 255 Bucket 0 Bucket 1 Bucket 254 Bucket 255 Segment 0 Segment 1 Directory
9
− Segment reader/writer locks for queries
112 102 012 002
... ... ... ...
Bucket 0 Bucket 1 Bucket 254 Bucket 255 Bucket 0 Bucket 1 Bucket 254 Bucket 255 Segment 0 Segment 1 Directory
9
− Segment reader/writer locks for queries − Dynamic resizing with segment splitting and directory doubling
112 102 012 002
... ... ... ...
Bucket 0 Bucket 1 Bucket 254 Bucket 255 Bucket 0 Bucket 1 Bucket 254 Bucket 255 Segment 0 Segment 1 Directory
9
− Segment reader/writer locks for queries − Dynamic resizing with segment splitting and directory doubling
112 102 012 002
... ... ... ...
Bucket 0 Bucket 1 Bucket 254 Bucket 255 Bucket 0 Bucket 1 Bucket 254 Bucket 255 Segment 0 Segment 1 Directory
9
− Segment reader/writer locks for queries − Dynamic resizing with segment splitting and directory doubling
− Lock-free search and bucket lock for writes
112 102 012 002
... ... ... ...
Bucket 0 Bucket 1 Bucket 254 Bucket 255 Bucket 0 Bucket 1 Bucket 254 Bucket 255 Segment 0 Segment 1 Directory
9
− Segment reader/writer locks for queries − Dynamic resizing with segment splitting and directory doubling
− Lock-free search and bucket lock for writes − Full-table resizing with one helper thread
112 102 012 002
... ... ... ...
Bucket 0 Bucket 1 Bucket 254 Bucket 255 Bucket 0 Bucket 1 Bucket 254 Bucket 255 Segment 0 Segment 1 Directory
Thread-1: resize Thread-2: help resizing Thread-3~n: wait for finishing resizing…
9
− Segment reader/writer locks for queries − Dynamic resizing with segment splitting and directory doubling
− Lock-free search and bucket lock for writes − Full-table resizing with one helper thread
112 102 012 002
... ... ... ...
Bucket 0 Bucket 1 Bucket 254 Bucket 255 Bucket 0 Bucket 1 Bucket 254 Bucket 255 Segment 0 Segment 1 Directory
Thread-1: resize Thread-2: help resizing Thread-3~n: wait for finishing resizing…
1 2N-2 2N-1
10
N-1
Top level Bottom level
1 2N-2 2N-1
10
Two-level structure
N-1
Top level Bottom level
1 2N-2 2N-1
10
Two-level structure
KV1 KV2 KV3 KV4 Slots Tokens
A 4-slot bucket
N-1
Top level Bottom level
1 2N-2 2N-1
10
key h1(key) h2(key) Two-level structure
KV1 KV2 KV3 KV4 Slots Tokens
A 4-slot bucket One-step movement One extra write at most
N-1
Top level Bottom level
1 2N-2 2N-1
10
key h1(key) h2(key) Two-level structure
KV1 KV2 KV3 KV4 Slots Tokens
A 4-slot bucket One-step movement One extra write at most
N-1
Top level Bottom level
1 2N-2 2N-1
10
key h1(key) h2(key) Two-level structure
KV1 KV2 KV3 KV4 Slots Tokens
A 4-slot bucket One-step movement
(Atomic update)
One extra write at most
N-1
Top level Bottom level
1 2N-2 2N-1
10
N-1
Top level Bottom level
1 2 4N-3 4N-1 4N-2
1 2N-2 2N-1
10
Top level Bottom level
1 2N-2 2N-1
11
Top level Bottom level
1 2N-2 2N-1
N-1
Top level Bottom level
N-1
x
Thread-1: search(x) Thread-2: insert(key)
1 2N-2 2N-1
11
Thread-2: relocate x Top level Bottom level
1 2N-2 2N-1
N-1
Top level Bottom level
N-1
x
Thread-1: search x Thread-1: search(x) Thread-2: insert(key)
1 2N-2 2N-1
11
Thread-2: relocate x Top level Bottom level
1 2N-2 2N-1
N-1
Top level Bottom level
N-1
x
Thread-1: search x
No “x” is found Thread-1: search(x) Thread-2: insert(key)
1 2N-2 2N-1
11
Thread-2: relocate x Top level Bottom level
1 2 4N-3 4N-1 4N-2
1 2N-2 2N-1
N-1
Top level Bottom level
N-1
x
Thread-1: search x
No “x” is found Thread-1: search(x) Thread-2: insert(key) Thread-1: insert(key) and trigger resizing…
1 2N-2 2N-1
11
Thread-2: relocate x Top level Bottom level
1 2 4N-3 4N-1 4N-2
1 2N-2 2N-1
N-1
Top level Bottom level
N-1
x
Thread-1: search x
No “x” is found Thread-1: search(x) Thread-2: insert(key) Thread-1: insert(key) and trigger resizing… Thread-2~n: wait for finishing resizing…
1 2N-2 2N-1
11
Thread-2: relocate x Top level Bottom level
1 2 4N-3 4N-1 4N-2
1 2N-2 2N-1
N-1
Top level Bottom level
N-1
x
Thread-1: search x
No “x” is found Thread-1: search(x) Thread-2: insert(key) Thread-1: insert(key) and trigger resizing… Thread-2~n: wait for finishing resizing…
12
13
Rehashing threads
...
Worker threads
...
Thread-local context ptr. A thread
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Rehashing
key
H1(key) H2(key)
Level list
last_level first_level is_resizing
Context
Global context ptr.
13
Rehashing threads
...
Worker threads
...
Thread-local context ptr. A thread
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Rehashing
key
H1(key) H2(key)
Level list
last_level first_level is_resizing
Context
Global context ptr.
13
Rehashing threads
...
Worker threads
...
Thread-local context ptr. A thread
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Rehashing
key
H1(key) H2(key)
Level list
last_level first_level is_resizing
Context
Global context ptr.
13
Rehashing threads
...
Worker threads
...
Thread-local context ptr. A thread
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Rehashing
key
H1(key) H2(key)
Level list
last_level first_level is_resizing
Context
Global context ptr.
14
15
16
17
KV_PTR1 Slots (each 8 bytes)
A bucket
KV_PTR8
18
H1(key) H2(key)
19
2N-1 2N-2 2N-3 1 2
N-1 N-2 1
H1(key) H2(key)
19
2N-1 2N-2 2N-3 1 2
N-1 N-2 1
H1(key) H2(key)
20
21
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
...
key
H1(key) H2(key)
21
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
...
key
H1(key) H2(key)
...
Level list
22
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
key
H1(key) H2(key)
Level list
last_level first_level is_resizing
Context
Global context ptr.
23
1. Make a local copy of the global context pointer
Worker threads
...
Thread-local context ptr. A thread 1
... ...
N-1 N-2 1
... ...
key
H1(key) H2(key) last_level first_level is_resizing
Level list Context
Global context ptr.
24
1. Make a local copy of the global context pointer 2. CAS to append a new level
Worker threads
...
Thread-local context ptr. A thread 1
... ...
N-1 N-2 1
... ...
last_level first_level is_resizing
Level list Context
Global context ptr.
...
2N-1 2N-2 2N-3 1 2
key
H1(key) H2(key) 2
25
1. Make a local copy of the global context pointer 2. CAS to append a new level 3. CoW + CAS to update the first_level
Worker threads
...
Thread-local context ptr. A thread 1
... ...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
key
H1(key) H2(key) 2 last_level first_level is_resizing Global context ptr. 3
Context
Context size: 17 bytes
last_level first_level is_resizing
8 B 8 B 1 B Lightweight CoW
26
1. Make a local copy of the global context pointer 2. CAS to append a new level 3. CoW + CAS to update the first_level 4. Rehash items in the last level
Worker threads
...
Thread-local context ptr. A thread 1
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Rehashing
key
H1(key) H2(key)
Level list
2 4 last_level first_level is_resizing Global context ptr. 3
Context
27
1. Make a local copy of the global context pointer 2. CAS to append a new level 3. CoW + CAS to update the first_level 4. Rehash items in the last level 5. CoW + CAS to update the last_level
Worker threads
...
Thread-local context ptr. A thread 1
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Rehashing
key
H1(key) H2(key)
Level list
last_level first_level is_resizing
Context
Global context ptr. 2 3 4 5
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Rehashing
key
H1(key) H2(key)
Level list
last_level first_level is_resizing
Context
Global context ptr.
28
1. Make a local copy of the global context pointer 2. CAS to append a new level 3. CoW + CAS to update the first_level 4. Rehash items in the last level 5. CoW + CAS to update the last_level
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Rehashing
key
H1(key) H2(key)
Level list
last_level first_level is_resizing
Context
Global context ptr.
28
1. Make a local copy of the global context pointer 2. CAS to append a new level 3. CoW + CAS to update the first_level 4. Rehash items in the last level 5. CoW + CAS to update the last_level
Expansion stage Rehashing stage Expansion stage Rehashing stage Resizing steps Queries
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Rehashing
key
H1(key) H2(key)
Level list
last_level first_level is_resizing
Context
Global context ptr.
28
1. Make a local copy of the global context pointer 2. CAS to append a new level 3. CoW + CAS to update the first_level 4. Rehash items in the last level 5. CoW + CAS to update the last_level
− Rehashing threads: rehash until there are 2 levels left
Expansion stage Rehashing stage Expansion stage Rehashing stage Rehashing threads (background) Resizing steps Queries
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Rehashing
key
H1(key) H2(key)
Level list
last_level first_level is_resizing
Context
Global context ptr.
28
1. Make a local copy of the global context pointer 2. CAS to append a new level 3. CoW + CAS to update the first_level 4. Rehash items in the last level 5. CoW + CAS to update the last_level
− Rehashing threads: rehash until there are 2 levels left
Expansion stage Rehashing stage Expansion stage Rehashing stage Rehashing threads (background) Resizing steps Worker threads Queries
29
30
30
Update tag and pointer in an atomic manner
Tag (2 B)
KV_PTR1
A bucket
KV_PTR8
A slot
31
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
key
H1(key) H2(key)
Level list
...
: a pointer in one slot
31
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
key
H1(key) H2(key)
Level list
Thread-2: rehashing Thread-1: search
...
2N-1 2N-2 2N-3 1 2
Missing
: a pointer in one slot
31
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
key
H1(key) H2(key)
Level list
Thread-2: rehashing Thread-1: b2t search
...
2N-1 2N-2 2N-3 1 2
: a pointer in one slot
...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
32
...
...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
32
x
H1(x) H2(x)
...
...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
32
Thread-1: insert(x) Thread-2: insert(x)
... ...
...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
32
Thread-1: insert(x) Thread-2: rehashing
...
...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
32
Thread-1: insert(x) Thread-2: rehashing
... ...
...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
32
Thread-1: insert(x) Thread-2: rehashing
... ...
...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
32
Thread-1: insert(x)
...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
32
Thread-1: insert(x)
Loss
...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
32
Thread-1: insert(x)
Loss
33
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
33
− Concurrent insertions with the same key
...
2N-1 2N-2 2N-3 1 2
Thread-1: insert(x) Thread-2: insert(x)
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
33
− Concurrent insertions with the same key − Retry of context-aware insertion
...
2N-1 2N-2 2N-3 1 2
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
...
Thread-1: insert(x) Thread-2: insert(x) Thread-1: insert(x) Thread-1: redo insert(x)
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
33
− Concurrent insertions with the same key − Retry of context-aware insertion − Data movement for rehashing
...
2N-1 2N-2 2N-3 1 2
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
... ... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
...
Rehashing thread
Thread-1: insert(x) Thread-2: insert(x) Thread-1: insert(x) Thread-1: redo insert(x)
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
33
− Concurrent insertions with the same key − Retry of context-aware insertion − Data movement for rehashing
...
2N-1 2N-2 2N-3 1 2
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
... ... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
...
Rehashing thread
Thread-1: insert(x) Thread-2: insert(x) Thread-1: insert(x) Thread-1: redo insert(x)
Two pointers to different items
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
33
− Concurrent insertions with the same key − Retry of context-aware insertion − Data movement for rehashing
...
2N-1 2N-2 2N-3 1 2
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
... ... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
...
Rehashing thread
Thread-1: insert(x) Thread-2: insert(x) Thread-1: insert(x) Thread-1: redo insert(x)
Two pointers to different items Two pointers to the same item
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
33
− Concurrent insertions with the same key − Retry of context-aware insertion − Data movement for rehashing
...
2N-1 2N-2 2N-3 1 2
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
... ... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
...
Rehashing thread
Thread-1: insert(x) Thread-2: insert(x) Thread-1: insert(x) Thread-1: redo insert(x)
Two pointers to different items Two pointers to the same item
− B2t search to find two pointers to duplicate items
B2t search
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
33
− Concurrent insertions with the same key − Retry of context-aware insertion − Data movement for rehashing
...
2N-1 2N-2 2N-3 1 2
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
... ... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
...
Rehashing thread
Thread-1: insert(x) Thread-2: insert(x) Thread-1: insert(x) Thread-1: redo insert(x)
Two pointers to different items Two pointers to the same item
− B2t search to find two pointers to duplicate items − Check if two pointers refer to the same item
B2t search
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
33
− Concurrent insertions with the same key − Retry of context-aware insertion − Data movement for rehashing
...
2N-1 2N-2 2N-3 1 2
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
Rehashing thread
Thread-1: insert(x) Thread-2: insert(x) Thread-1: redo insert(x)
Two pointers to different items Two pointers to the same item
− B2t search to find two pointers to duplicate items − Check if two pointers refer to the same item
B2t search
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
33
− Concurrent insertions with the same key − Retry of context-aware insertion − Data movement for rehashing
...
2N-1 2N-2 2N-3 1 2
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
Rehashing thread
Thread-2: insert(x) Thread-1: redo insert(x)
Two pointers to different items Two pointers to the same item
− B2t search to find two pointers to duplicate items − Check if two pointers refer to the same item
matching the key
B2t search
...
2N-1 2N-2 2N-3 1 2
34
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
Timeline
Thread 1: update Thread-2: rehashing
34
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
Timeline
Thread 1: update Thread-2: rehashing
Find
34
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
...
Timeline
Thread 1: update Thread-2: rehashing
Find copy
Rehashing
34
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
Timeline
Thread 1: update Thread-2: rehashing
Find update copy
...
34
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
Timeline
Thread 1: update Thread-2: rehashing
Find update copy delete
34
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
Timeline
Thread 1: update Thread-2: rehashing
Find update copy delete
34
... ...
2N-1 2N-2 2N-3 1 2
...
N-1 N-2 1
... ...
Level list
...
2N-1 2N-2 2N-3 1 2
Timeline
Thread 1: update Thread-2: rehashing
Find update copy delete
35
36
37
− Intel Optane DC PMM configured in App Direct mode − 36 threads in one NUMA node − PMDK
− LEVEL: original level hashing [OSDI ’18] − CCEH: lazy deletion version, default probing distance (16 slots) [FAST ’19] − CMAP: concurrent_hash_map engine from Intel pmemkv − P-CLHT: PM variant of CLHT converted by RECIPE [SOSP ’19] − CLEVEL: our Clevel hashing
38
200 400 600 800 1000 20 40 60 80 100
Load factor (%) Inserted items (k)
P-CLHT CCEH LEVEL CLEVEL
38
200 400 600 800 1000 20 40 60 80 100
Load factor (%) Inserted items (k)
P-CLHT CCEH LEVEL CLEVEL
39
Positive Negative 5 10 15 20 25
Average latency (us)
P-CLHT LEVEL CCEH CMAP LEVEL-TBB CCEH-TBB CMAP-TBB CLEVEL
Insertion Update Deletion 10 20 30 40
46 101 86 57
Average latency (us)
P-CLHT LEVEL CCEH CMAP LEVEL-TBB CCEH-TBB CMAP-TBB CLEVEL
106
* Lack of implementation of update and deletion in open-source code
39
summary tags, Clevel hashing obtains
− 1.2×−5.0× speedup for positive search − 1.4×−9.0× speedup for negative search
Positive Negative 5 10 15 20 25
Average latency (us)
P-CLHT LEVEL CCEH CMAP LEVEL-TBB CCEH-TBB CMAP-TBB CLEVEL
Insertion Update Deletion 10 20 30 40
46 101 86 57
Average latency (us)
P-CLHT LEVEL CCEH CMAP LEVEL-TBB CCEH-TBB CMAP-TBB CLEVEL
106
* Lack of implementation of update and deletion in open-source code
39
summary tags, Clevel hashing obtains
− 1.2×−5.0× speedup for positive search − 1.4×−9.0× speedup for negative search
Positive Negative 5 10 15 20 25
Average latency (us)
P-CLHT LEVEL CCEH CMAP LEVEL-TBB CCEH-TBB CMAP-TBB CLEVEL
Insertion Update Deletion 10 20 30 40
46 101 86 57
Average latency (us)
P-CLHT LEVEL CCEH CMAP LEVEL-TBB CCEH-TBB CMAP-TBB CLEVEL
106
* Lack of implementation of update and deletion in open-source code
correctness guarantee
40
Read ratio (%): 50 95 100 Write ratio (%): 100 50 5
Load A A B C 1 2 3 4 5
1.32 M op/s 1.81 M op/s 0.45 M op/s
Throughput ratio wrt P-CLHT
P-CLHT LEVEL CCEH CMAP LEVEL-TBB CCEH-TBB CMAP-TBB CLEVEL 0.91 M op/s
40
Read ratio (%): 50 95 100 Write ratio (%): 100 50 5
Load A A B C 1 2 3 4 5
1.32 M op/s 1.81 M op/s 0.45 M op/s
Throughput ratio wrt P-CLHT
P-CLHT LEVEL CCEH CMAP LEVEL-TBB CCEH-TBB CMAP-TBB CLEVEL 0.91 M op/s
control and non-blocking resizing
4.2×
41