CockroachDBs Survivability Model Scalable, Survivable, Consistent, - - PowerPoint PPT Presentation

▶

Apr 12, 2023 484 likes •833 views

CockroachDBs Survivability Model Scalable, Survivable, Consistent, SQL presented by Marc Berhault / Engineer @cockroachdb CockroachDB: Make Data Easy Scalable Survivable Strongly Consistent SQL And... Open Source

SLIDE 1

@cockroachdb

Scalable, Survivable, Consistent, SQL

CockroachDB’s Survivability Model

presented by Marc Berhault / Engineer

SLIDE 2

@cockroachdb

■ Scalable ■ Survivable ■ Strongly Consistent ■ SQL And... ■ Open Source

CockroachDB: Make Data Easy

SLIDE 3

@cockroachdb

Architecture: ■ SQL layer ■ Transactions ■ Sharding ■ Replication Survivability: ■ Rebalancing ■ Repairs

Agenda

SLIDE 4

@cockroachdb

Architecture

SLIDE 5

@cockroachdb

Architecture (high-level)

SQL Store Range Transactional KV Distribution Replication Range Range Store Range Range Range Storage Store Range Range Range Store Range Range Range SQL SQL Storage Storage * * Abstraction stack: Node 1 Node 2 In the network:

SLIDE 6

@cockroachdb

CREATE TABLE inventory ( id INTEGER PRIMARY KEY, name VARCHAR, quantity INTEGER, INDEX name_index (name)); INSERT INTO inventory VALUES (1, “Apple”, 3);

SQL

SQL Transactional KV Distribution Replication

SLIDE 7

@cockroachdb

■ Tables ■ Rows ■ Columns ■ Indexes

SQL: Data model

id name quantity 1 Apple 3 2 Orange 12 3 Cherry 5 4 Banana 7 inventory name id Apple 1 Banana 4 Cherry 3 Orange 2 name_index

SLIDE 8

@cockroachdb

INSERT INTO inventory VALUES (1, “Apple”, 3);

SQL: Key anatomy

Key: /<table>/<index>/<key>/<column> Value /inventory/primary/ 1/name Apple /inventory/primary/ 1/quantity 3 Key: /<table>/<index>/<key> Value /inventory/name_index/Apple 1 inventory name_index

SLIDE 9

@cockroachdb

■ Update all keys atomically ■ Track across multiple commands ■ Retry when necessary

Transactional KV: consistency

SQL Transactional KV Distribution Replication

SLIDE 10

@cockroachdb

■ CockroachDB uses optimistic concurrency control for lock-free transactions ■ In case of conflict: the losing transaction restarts

Optimistic Concurrency

SLIDE 11

@cockroachdb

Distribution: scalability

SQL Transactional KV Distribution Replication

■ Route KV commands to the appropriate shards ■ Split batches if necessary

SLIDE 12

@cockroachdb

Sharding: Index Each shard holds a contiguous span of the keyspace

Ø-lem

apricot banana blueberry cherry grape

lem-pea

lemon lime mango melon

range

pea-∞

peach pear pineapple raspberry strawberry

SLIDE 13

@cockroachdb

Sharding: Index An index maps from key to range ID

Ø-lem

apricot banana blueberry cherry grape

lem-pea

lemon lime mango melon

range

pea-∞

peach pear pineapple raspberry strawberry Ø-lem lem-pea pea-∞

shard index

SLIDE 14

@cockroachdb

Sharding: Split

Ø-lem

apricot banana blueberry cherry grape

lem-pea

lemon lime mango melon

range

pea-str

peach pear pineapple raspberry Ø-lem lem-pea

shard index str-∞

strawberry tamarillo tamarind pea-∞ pea-str

Split when a shard is too large

SLIDE 15

@cockroachdb

Replication: survivability

SQL Transactional KV Distribution Replication

■ Each range is replicated to three

r more nodes

■ One replica of each range is the leader

SLIDE 16

@cockroachdb

Replication

Node 1

Range 1 Range 2

Node 2

Range 1 Range 2

Node 3

Range 1

Node 4

Range 2

Range 3 Range 3 Range 2 Range 3

■ Each set of replicas is a Raft group ■ Consistency provided by quorum

SLIDE 17

@cockroachdb

Replication: Node storage

■ Data is stored locally in RocksDB ■ Embedded KV database ■ Provides atomic writes to multiple keys ■ Supports ordered scans

SLIDE 18

@cockroachdb

Reliability

SLIDE 19

@cockroachdb

■ Symmetric nodes ■ Auto-balancing ■ Self-healing

Reliability

SLIDE 20

@cockroachdb

Reliability: Rebalancing

Node 1

Range 1 Range 2

Node 2

Range 1 Range 2

Node 3

Range 1

Range 2

Range 3 Range 3 Range 2 Range 3

SLIDE 21

@cockroachdb

Adding a new (empty) node

Reliability: Rebalancing

Node 1

Range 1 Range 2

Node 2

Range 1 Range 2

Node 3

Range 1

Node 4

Range 2

Range 3 Range 3 Range 2 Range 3

SLIDE 22

@cockroachdb

A new replica is allocated, data is copied.

Reliability: Rebalancing

Node 1

Range 1 Range 2

Node 2

Range 1 Range 2

Node 3

Range 1

Node 4

Range 2

Range 3 Range 3 Range 2 Range 3 Range 3

SLIDE 23

@cockroachdb

The new replica is made live, replacing another.

Reliability: Rebalancing

Node 1

Range 1 Range 2

Node 2

Range 1 Range 2

Node 3

Range 1

Node 4

Range 2

Range 3 Range 3 Range 2 Range 3 Range 3

SLIDE 24

@cockroachdb

The old (inactive) replica is deleted.

Reliability: Rebalancing

Node 1

Range 1 Range 2

Node 2

Range 1 Range 2

Node 3

Range 1

Node 4

Range 2

Range 3 Range 3 Range 2 Range 3

SLIDE 25

@cockroachdb

Process continues until nodes are balanced.

Reliability: Rebalancing

Node 1

Range 1 Range 2 Range 2

Node 2 Node 3

Range 1

Node 4

Range 2

Range 3 Range 1 Range 3 Range 2 Range 3

SLIDE 26

@cockroachdb

Reliability: Recovery

Node 1

Range 1 Range 2 Range 2

Node 2 Node 3

Range 1

Node 4

Range 2

Range 3 Range 1 Range 3 Range 2 Range 3

SLIDE 27

@cockroachdb

Losing a node causes recovery of its replicas.

Reliability: Recovery

Node 1

Range 1 Range 2 Range 2

Node 2 Node 3

Range 1

Node 4

Range 2

Range 3 Range 1 Range 3 Range 2 Range 3

X

SLIDE 28

@cockroachdb

A new replica gets created on an existing node.

Reliability: Recovery

Node 1

Range 1 Range 2 Range 2

Node 2 Node 3

Range 1

Node 4

Range 2

Range 3 Range 1 Range 3 Range 2 Range 3

X

Range 1 Range 3

SLIDE 29

@cockroachdb

Once at full replication, the old replicas are forgotten.

Reliability: Recovery

Node 1

Range 1 Range 2 Range 2

Node 3

Range 1

Node 4

Range 2

Range 3 Range 2 Range 3 Range 1 Range 3

SLIDE 30

@cockroachdb

Zone configuration ■ Replication factor (default 3) ■ Geographical location (eg: 2 in Europe, 1 in US) ■ Machine attributes (ssd vs disk)

SLIDE 31

@cockroachdb

Status: BETA

SLIDE 32

@cockroachdb

Ready for development testing Roadmap: ■ Stability ■ Performance ■ Distributed SQL ■ Optimized JOINs

Status: Beta

SLIDE 33

@cockroachdb

CockroachDB’s Survivability Model

■ Scalable ■ Survivable ■ Strongly Consistent ■ SQL And... ■ Open Source

CockroachDB: Make Data Easy

Architecture: ■ SQL layer ■ Transactions ■ Sharding ■ Replication Survivability: ■ Rebalancing ■ Repairs

Agenda

Architecture

Architecture (high-level)

CREATE TABLE inventory ( id INTEGER PRIMARY KEY, name VARCHAR, quantity INTEGER, INDEX name_index (name)); INSERT INTO inventory VALUES (1, “Apple”, 3);

SQL

■ Tables ■ Rows ■ Columns ■ Indexes

SQL: Data model

INSERT INTO inventory VALUES (1, “Apple”, 3);

SQL: Key anatomy

■ Update all keys atomically ■ Track across multiple commands ■ Retry when necessary

Transactional KV: consistency

■ CockroachDB uses optimistic concurrency control for lock-free transactions ■ In case of conflict: the losing transaction restarts

Optimistic Concurrency

Distribution: scalability

■ Route KV commands to the appropriate shards ■ Split batches if necessary

Sharding: Index Each shard holds a contiguous span of the keyspace

Sharding: Index An index maps from key to range ID

Sharding: Split

Split when a shard is too large

Replication: survivability

■ Each range is replicated to three

■ One replica of each range is the leader

Replication

■ Each set of replicas is a Raft group ■ Consistency provided by quorum

Replication: Node storage

■ Data is stored locally in RocksDB ■ Embedded KV database ■ Provides atomic writes to multiple keys ■ Supports ordered scans

Reliability

■ Symmetric nodes ■ Auto-balancing ■ Self-healing

Reliability

Reliability: Rebalancing

Adding a new (empty) node

Reliability: Rebalancing

A new replica is allocated, data is copied.

Reliability: Rebalancing

The new replica is made live, replacing another.

Reliability: Rebalancing

The old (inactive) replica is deleted.

Reliability: Rebalancing

Process continues until nodes are balanced.

Reliability: Rebalancing

Reliability: Recovery

Losing a node causes recovery of its replicas.

Reliability: Recovery

X

A new replica gets created on an existing node.

Reliability: Recovery

X

Once at full replication, the old replicas are forgotten.

Reliability: Recovery

Zone configuration ■ Replication factor (default 3) ■ Geographical location (eg: 2 in Europe, 1 in US) ■ Machine attributes (ssd vs disk)

Status: BETA

Ready for development testing Roadmap: ■ Stability ■ Performance ■ Distributed SQL ■ Optimized JOINs

Status: Beta

github.com/cockroachdb/cockroach CockroachLabs.com Gitter: cockroachdb @cockroachdb Thank You