Database Scalability {Patterns} / Robert Treat robert treat - - PowerPoint PPT Presentation

▶
database scalability patterns
SMART_READER_LITE
LIVE PREVIEW

Database Scalability {Patterns} / Robert Treat robert treat - - PowerPoint PPT Presentation

Mar 01, 2023 141 likes •563 views

Database Scalability {Patterns} / Robert Treat robert treat omniti postgres oracle - mysql mssql - sqlite - nosql What are Database Scalability Patterns? Part Design Patterns Part Application Life-Cycle Part Design Patterns Part

slide-1
SLIDE 1

/ Robert Treat

Database Scalability {Patterns}

slide-2
SLIDE 2

robert treat

  • mniti

postgres

  • racle - mysql

mssql - sqlite - nosql

slide-3
SLIDE 3

What are “Database Scalability Patterns?”

slide-4
SLIDE 4

Part Design Patterns Part Application Life-Cycle

slide-5
SLIDE 5

Part Design Patterns Part Application Life-Cycle

slide-6
SLIDE 6

Part Design Patterns Part Application Life-Cycle

slide-7
SLIDE 7

Part Design Patterns Part Application Life-Cycle

slide-8
SLIDE 8

MyFirstDatabase

slide-9
SLIDE 9

Vertical Partitioning

slide-10
SLIDE 10

Vertical Scaling

slide-11
SLIDE 11

Vertical Scaling

slide-12
SLIDE 12

Vertical Scaling

slide-13
SLIDE 13

Horizontal Scaling Federated Data Storage “sharding” Horizontal Partitioning Multi-Master Read Slaves

slide-14
SLIDE 14

Read Slaves / Master - Slave Scale Read Load

slide-15
SLIDE 15

Read Slaves / Master - Slave Scale Read Load writes go here!

slide-16
SLIDE 16

Read Slaves / Master - Slave Scale Read Load writes go here! reads go here (or here) (or here)

slide-17
SLIDE 17

database writes data to slaves Read Slaves / Master - Slave Scale Read Load writes go here! reads go here (or here) (or here) slave db slave db slave db

slide-18
SLIDE 18

Read Slaves / Master - Slave Scale Read Load writes go here! reads go here (or here) (or here) app writes data everywhere memcached memcached memcached

slide-19
SLIDE 19

Read Slaves / Master - Slave Scale Read Load

  • Typically
  • Full Copy of Data

On Each Node

  • Asynchronous
slide-20
SLIDE 20

Read Slaves / Master - Slave Scale Read Load

  • Typically
  • Full Copy of Data

On Each Node

  • Asynchronous
  • Consider
  • Partial Copy
  • Synchronous
  • Don’t use a RDBMS?
slide-21
SLIDE 21

Read Slaves / Master - Slave Scale Read Load

  • Typically
  • Full Copy of Data

On Each Node

  • Asynchronous
  • Consider
  • Partial Copy
  • Synchronous
  • Don’t use a RDBMS?

“easy” Requires Application Changes

slide-22
SLIDE 22

Scaling Writes “not easy”

slide-23
SLIDE 23

Multi-Master many different ways to implement this, few that actually work in production

slide-24
SLIDE 24

many different ways to implement this, few that actually work in production write to any node, database syncs data Multi-Master

slide-25
SLIDE 25

many different ways to implement this, few that actually work in production write to any node, database syncs data can reduce cpu, doesn’t reduce i/o Multi-Master

slide-26
SLIDE 26

many different ways to implement this, few that actually work in production write to any node, database syncs data can reduce cpu, doesn’t reduce i/o failover solution not a scalability solution Multi-Master

slide-27
SLIDE 27

Horizontal Partitioning “sharding” “SOA”

slide-28
SLIDE 28

Horizontal Partitioning “sharding” “SOA”

slide-29
SLIDE 29
  • Divide schema by job operations
  • Move each piece to own server
  • Duplicate some data as needed

Horizontal Partitioning

slide-30
SLIDE 30

users items forums

  • Divide schema by job operations
  • Move each piece to own server
  • Duplicate some data as needed

Horizontal Partitioning

slide-31
SLIDE 31

users items forums

  • Divide schema by job operations
  • Move each piece to own server
  • Duplicate some data as needed
  • You must separate dependencies

in the app code first! Horizontal Partitioning

slide-32
SLIDE 32

users items forums

  • Divide schema by job operations
  • Move each piece to own server
  • Duplicate some data as needed
  • You must separate dependencies

in the app code first! Each node is a new instance of vertical scaling Horizontal Partitioning

slide-33
SLIDE 33
  • data split across servers based on algorithm
  • data dropped into buckets (multiple?)

Horizontal Scaling

magic hash algorithm

app

slide-34
SLIDE 34
  • data split across servers based on algorithm
  • data dropped into buckets (multiple?)

Horizontal Scaling

magic hash algorithm

app

slide-35
SLIDE 35
  • data split across servers based on algorithm
  • data dropped into buckets (multiple?)
  • someone must keep track of data, and provide

lookup services Horizontal Scaling

magic hash algorithm

app

slide-36
SLIDE 36

Universal Truths of Scaling Databases Vertical Scalability is Helpful for Every Pattern Even in a horizontally scaled, fully distributed database, the number of nodes needed is affected by vertical scalability

slide-37
SLIDE 37

Universal Truths of Scaling Databases New Nodes Are Never Free

  • Add points of failure
  • Add management costs
  • Add complexity to architecture
  • Add complexity to your app code
slide-38
SLIDE 38

MyFirstDB V Vertical Partitioning V Vertical Scaling V Read Slaves V Horizontal Partitioning

slide-39
SLIDE 39

plan for layered data sources read / write connections in code use schemas to separate services tips

slide-40
SLIDE 40

THANKS! more: xzilla.net @robtreat2

  • mniti.com/surge

(scalability conference)