Nitro: A Fast, Scalable In-Memory Storage Engine for NoSQL Global - - PowerPoint PPT Presentation

▶
nitro a fast scalable in memory storage engine for nosql
SMART_READER_LITE
LIVE PREVIEW

Nitro: A Fast, Scalable In-Memory Storage Engine for NoSQL Global - - PowerPoint PPT Presentation

Jan 15, 2023 623 likes •994 views

Nitro: A Fast, Scalable In-Memory Storage Engine for NoSQL Global Secondary Index Sarath Lakshman, Sriram Melkote, John Liang, Ravi Mayuram Couchbase, Inc Presenter: Xiaoyao Qian 04.04.2017 4 million entries/sec 10 million lookups/sec 2

slide-1
SLIDE 1

Nitro: A Fast, Scalable In-Memory Storage Engine for NoSQL Global Secondary Index

Sarath Lakshman, Sriram Melkote, John Liang, Ravi Mayuram Couchbase, Inc

Presenter: Xiaoyao Qian • 04.04.2017

slide-2
SLIDE 2

10 million lookups/sec 4 million entries/sec

2

slide-3
SLIDE 3

3 https://www.mysql.com/why-mysql/benchmarks/

slide-4
SLIDE 4

Motivation

4

slide-5
SLIDE 5

5

slide-6
SLIDE 6

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

6

Ordered Linked List

slide-7
SLIDE 7

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

7

n: #nodes in next level f: fanout factor Avg O(logN): insert, lookup, delete

slide-8
SLIDE 8

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

8

Lock-free List Operations

slide-9
SLIDE 9

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

9

DoubleCAS 1 4 6 8

isdeleted=0 isdeleted=1

slide-10
SLIDE 10

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

10

MVCC: Multi-Version Concurrency Control

  • Immutable snapshots
  • Fast and low overhead snapshots
  • Avoid phantom reads
  • Memory efficiency
  • Fast and scalable garbage collection
slide-11
SLIDE 11

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

11

Descriptor: refcount = x Descriptor: refcount = y MVCC primitives: lifetime and descriptor

slide-12
SLIDE 12

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

12

Snapshot Iteration filter with bornSn>termSn && deadSn>=termSn

slide-13
SLIDE 13

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

13

Comparison with Copy-On-Write B+ Tree (COW B+)

slide-14
SLIDE 14

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

14

1. The snapshot Sn(x) descriptor shows refcount = 0 2. The previous snapshot Sn(x-1) has been garbage collected, i.e garbage collection of snapshots can only be performed in the sequential order of the snapshot termSn 3. #gc_workers = #concurrent_writers 4. Writers keep track of deadList which is attached to the snapshot

  • descriptor. Whenever a node is marked as deleted, add to deadList.

5. GC workers use deadList of a snapshot to perform physical node removal from the skiplist

slide-15
SLIDE 15

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

15

1. Traverse level 0 linked list of the skiplist, and write out the entries into data files 2. All entries that don’t belong to the snapshot are ignored 3. Node metadata (i.e lifetime) are not

  • serialized. They can be recreated during

recovery ✓ Minimum backup file size ✓ Compression friendly ✓ Since skiplist is ordered, the data written to disk is also ordered ❌ Could block garbage collection

slide-16
SLIDE 16

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

16

Backup shard1 Backup shard2 Backup shard3

slide-17
SLIDE 17

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

17

Recovery Buf: [nil, nil, nil, nil]

slide-18
SLIDE 18

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

18

Recovery Buf: [nil, nil, nil, nil] -> [n1, n1, n1, n1]

slide-19
SLIDE 19

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

19

Recovery Buf: [n1, n1, n1, n1] -> [n2, n2, n1, n1]

slide-20
SLIDE 20

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

20

Recovery Buf: [n2, n2, n1, n1] -> [n3, n3, n3, n3]

slide-21
SLIDE 21

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

21

Recovery Buf: [n3, n3, n3, n3] -> [n4, n3, n3, n3]

slide-22
SLIDE 22

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

22

Recovery Buf: [n4, n3, n3, n3] -> [n5, n5, n5, n5]

slide-23
SLIDE 23

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

23

Recovery Buf: [n5, n5, n5, n5] -> [n6, n6, n6, n5]

slide-24
SLIDE 24

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

24

Recovery Buf: [n6, n6, n6, n5] -> [n7, n6, n6, n5]

slide-25
SLIDE 25

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

25

Recovery Buf: [n7, n6, n6, n5] -> [nil, nil, nil, nil]

slide-26
SLIDE 26

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

26

Backup worker Garbage collector

INIT

Backing up termSn ack

ACTIVE Unlink, and write eligible data to delta backup files TERMINATE

Are you done? ack

Close delta backup files

Non-intrusive Backup

slide-27
SLIDE 27

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

27

slide-28
SLIDE 28

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

28

AccessBarrier t1 t2 t3

BarrierSession: liveCount = 2

slide-29
SLIDE 29

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

29

AccessBarrier t1 t2 t3

BarrierSession: liveCount = 2

BarrierSessionClos e

slide-30
SLIDE 30

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

30

AccessBarrier t1 t2 t3

BarrierSession: liveCount = 2

Terminated

slide-31
SLIDE 31

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

31

slide-32
SLIDE 32

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

32

slide-33
SLIDE 33

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

33

Global Secondary Index architecture

slide-34
SLIDE 34

Lock-Free Skiplist MVCC GC Backup & Recovery

Memory Reclamation

Evaluation GSI

34

slide-35
SLIDE 35

35

“TALK IS CHEAP, SHOW ME THE CODE”

https://github.com/couchbase/nitro ~15,000 lines of code mainly in Golang, with a little C/C++ Apache 2.0 Licence

slide-36
SLIDE 36

Questions & Discussions

1.

#GC_workers = #writers? Wouldn’t that be too intense? 2. Skiplist may not be good in cache utilization because of not consecutive

  • memory. Can this be optimized?

3. How can a single large index be distributed?

36