Evan Tschannen 1 Evan Tschannen Worked on FoundationDB for 8 - - PowerPoint PPT Presentation

▶
evan tschannen
SMART_READER_LITE
LIVE PREVIEW

Evan Tschannen 1 Evan Tschannen Worked on FoundationDB for 8 - - PowerPoint PPT Presentation

Mar 20, 2023 482 likes •853 views

Evan Tschannen 1 Evan Tschannen Worked on FoundationDB for 8 years Touched every core component of the database Recently added multi-region asynchronous replication https://www.foundationdb.org

slide-1
SLIDE 1

Evan Tschannen

  • 1
slide-2
SLIDE 2
  • Worked on FoundationDB for 8 years
  • Touched every core component of the database
  • Recently added multi-region asynchronous replication

Evan Tschannen

slide-3
SLIDE 3
  • https://www.foundationdb.org
  • https://github.com/apple/foundationdb
  • Open Source, Apache License (v2)
slide-4
SLIDE 4

Why are there so many databases?

4

slide-5
SLIDE 5

5

  • Tension between performance, scalability, and consistency
  • A wide variety of data models

Why are there so many databases?

slide-6
SLIDE 6

FoundationDB Performance

  • Organizes many instances of a single process database

into a single database

  • Read and write throughput 90% of the aggregate individual

read and write throughput

  • Single hop read latencies
  • Four hop write latencies
slide-7
SLIDE 7

FoundationDB is a…

  • Highly performant,
  • Scalable,
  • Ordered key value store,
  • That supports ACID transactions.
slide-8
SLIDE 8

8

A B C

Writer Servers Reader Wait for 2 of 3 responses Wait for 2 of 3 responses

slide-9
SLIDE 9

9

A B C

Writer Servers Reader Wait for 2 of 3 responses Wait for 2 of 3 responses

slide-10
SLIDE 10

10

A B C

Writer Servers Reader Wait for 3 of 3 responses Wait for 1 of 3 responses

slide-11
SLIDE 11

11

A B C

Writer Servers Reader Wait for 3 of 3 responses Wait for 1 of 3 responses

slide-12
SLIDE 12

12

A B C

Writer Servers Reader Wait for 3 of 3 responses Wait for 1 of 3 responses Versions (Stored in a different database)

D

0: A,B,C 101: B,C,D

slide-13
SLIDE 13

13

Coordinators Storage Servers ZooKeeper (Failure Handling) SQLite (Reads) Transaction Logs FoundationDB (Writes)

slide-14
SLIDE 14

14

Coordinators Storage Servers Transaction Logs Cluster Controller

slide-15
SLIDE 15

15

Coordinators Storage Servers Transaction Logs Cluster Controller

slide-16
SLIDE 16

16

Get Read Version

Read Write Commit

FoundationDB API

Versions 200 400 100 A : apple (200) (A, 200) (B, banana)

slide-17
SLIDE 17

17

Get Read Version

Read Write Commit

FoundationDB API

Versions 200 400 100 A : apple (200) (A, 200) (B, banana) 300 A : orange

slide-18
SLIDE 18

18

Coordinators Storage Servers Cluster Controller Master Proxies Resolvers Transaction Logs

slide-19
SLIDE 19

19

Coordinators Cluster Controller Master Proxies Resolvers Transaction Logs Storage Servers Get Read Version Reads Writes Commit

slide-20
SLIDE 20

20

Coordinators Cluster Controller Master Proxies Resolvers Transaction Logs Storage Servers Get Read Version Reads Writes Commit (200) (A, 200) (B : banana) Data Distribution

slide-21
SLIDE 21

21

Coordinators Cluster Controller Master Proxies Resolvers Transaction Logs Storage Servers Get Read Version Reads Writes Commit (200) (A, 200) (400) (B : banana)

slide-22
SLIDE 22

22

Coordinators Cluster Controller Master Proxies Resolvers Transaction Logs Storage Servers Get Read Version Reads Writes Commit (200) (A, 200) (400) (B : banana)

slide-23
SLIDE 23

23

Coordinators Cluster Controller Master Proxies Resolvers Transaction Logs Storage Servers Get Read Version Reads Writes Commit (200) (A, 200) (B : banana)

slide-24
SLIDE 24

24

Coordinators Cluster Controller Master Proxies Resolvers Transaction Logs Storage Servers Get Read Version Reads Writes Commit (200) (A, 200) (400) (B : banana)

slide-25
SLIDE 25

25

Coordinators Cluster Controller Master Proxies Resolvers Transaction Logs Storage Servers Get Read Version Reads Writes Commit

slide-26
SLIDE 26

26

Coordinators Cluster Controller Master Proxies Resolvers Transaction Logs Storage Servers Get Read Version Reads Writes Commit (400) (300) (200)

slide-27
SLIDE 27

27

Coordinators Cluster Controller Master Proxies Resolvers Transaction Logs Storage Servers Get Read Version Reads Writes Commit (400) (400) (300) (200)

slide-28
SLIDE 28

28

Coordinators Cluster Controller Master Proxies Resolvers Transaction Logs Storage Servers Get Read Version Reads Writes Commit

slide-29
SLIDE 29

29

Coordinators Cluster Controller Master Proxies Resolvers Transaction Logs Storage Servers Get Read Version Reads Writes Commit

slide-30
SLIDE 30

30

Coordinators Cluster Controller Master Proxies Resolvers Transaction Logs Storage Servers Get Read Version Reads Writes Commit

slide-31
SLIDE 31

31

Coordinators Cluster Controller Master Proxies Resolvers Transaction Logs Storage Servers Get Read Version Reads Writes Commit Data Distribution

slide-32
SLIDE 32

32

Coordinators Cluster Controller Master Proxies Resolvers Transaction Logs Storage Servers Get Read Version Reads Writes Commit

slide-33
SLIDE 33

FoundationDB Performance

  • Organizes many instances of a single process database

into a single database

  • Read and write throughput 90% of the aggregate individual

read and write throughput

  • Single hop read latencies
  • Four hop write latencies
slide-34
SLIDE 34

Does it work?

  • Run the entire distributed database in a single process
  • Inject random failures
  • Check correctness
  • Reproduce errors
slide-35
SLIDE 35

https://www.foundationdb.org

35