Learnings from Operating 200 PB of Disk-Based Storage What MSST - - PowerPoint PPT Presentation

▶
learnings from operating 200 pb of
SMART_READER_LITE
LIVE PREVIEW

Learnings from Operating 200 PB of Disk-Based Storage What MSST - - PowerPoint PPT Presentation

Jan 21, 2024 7 likes •397 views

` Learnings from Operating 200 PB of Disk-Based Storage What MSST May 2016 When Who Gleb Budman, CEO @GlebBudman @GlebBudman @GlebBudman Backblaze B2 is still 40% - 250% lower cost Doesnt make you wait for your data

slide-1
SLIDE 1

@GlebBudman What When Who MSST May 2016 Gleb Budman, CEO

Learnings from Operating 200 PB of Disk-Based Storage

`

slide-2
SLIDE 2

@GlebBudman

slide-3
SLIDE 3

@GlebBudman

slide-4
SLIDE 4

@GlebBudman

  • Backblaze B2 is still 40% - 250% lower cost
  • Doesn’t make you wait for your data
  • Doesn’t penalize for deletes
  • Doesn’t penalize for access
  • Upload transactions are free
slide-5
SLIDE 5

@GlebBudman

200+

PETABYTES

slide-6
SLIDE 6

@GlebBudman

The Cloud Storage Stack

Culture Business Process Software Hardware Datacenter

slide-7
SLIDE 7

@GlebBudman

slide-8
SLIDE 8

@GlebBudman

Goal:

  • Convert kilowatts-to-bits

Considerations:

  • Local cost of power and real estate
  • Don’t ignore taxes, economic zones
  • Climate
  • Building and system efficiency
  • Proximity to ops team & good people
  • Connectivity

Resource: Backblaze Datacenter RFP

https://www.backblaze.com/blog/backblaze-datacenter-grows-1000x-datacenter-2-0-needed-apply-within/
slide-9
SLIDE 9

@GlebBudman

Hardware Connect Hard Drives to the Internet

slide-10
SLIDE 10

@GlebBudman

Backblaze Storage Pod

slide-11
SLIDE 11

@GlebBudman

Don’t Make Hardware Redundant

slide-12
SLIDE 12

@GlebBudman

Server Power Supply Desktop Power Supply

VS

Use Commodity Parts

slide-13
SLIDE 13

@GlebBudman

Use Consumer Hard Drives

slide-14
SLIDE 14

@GlebBudman

Goal:

  • Connect hard drives to

the Internet Considerations:

  • Don’t make hardware redundant
  • Use commodity parts
  • Use consumer hard drives

Resource: Backblaze Storage Pod

https://www.backblaze.com/storage-pod.html/

slide-15
SLIDE 15

@GlebBudman

Software Put All Intelligence Here

slide-16
SLIDE 16

@GlebBudman

Backblaze Vault

slide-17
SLIDE 17

@GlebBudman

Avoid Choke Points

slide-18
SLIDE 18

@GlebBudman

slide-19
SLIDE 19

@GlebBudman

VS

Plan for Silent Corruption

slide-20
SLIDE 20

@GlebBudman

Put Replication Above the File System

ide
slide-21
SLIDE 21

@GlebBudman

Run Out of Resources Simultaneously

slide-22
SLIDE 22

@GlebBudman

Model & Monitor Storage Burn

slide-23
SLIDE 23

@GlebBudman

Goal:

  • Put all intelligence here

Considerations:

  • Avoid choke points
  • Algorithmically monitor SMART stats
  • Plan for Silent Corruption
  • Put replication above the file system
  • Run out of resources simultaneously
  • Model & monitor storage burn

Resource: Backblaze Vault

https://www.backblaze.com/blog/vault-cloud-storage-architecture/

slide-24
SLIDE 24

@GlebBudman

Business Processes Optimize for TCO

slide-25
SLIDE 25

@GlebBudman

Design for Failure…

slide-26
SLIDE 26

@GlebBudman

But Fix

slide-27
SLIDE 27

@GlebBudman

slide-28
SLIDE 28

@GlebBudman

slide-29
SLIDE 29

@GlebBudman

slide-30
SLIDE 30

@GlebBudman

slide-31
SLIDE 31

@GlebBudman

Purchases

slide-32
SLIDE 32

@GlebBudman

Goal:

  • Optimize for TCO

Resource: Backblaze Drive Process

https://www.backblaze.com/blog/alas-poor-stephen-is-dead/

Considerations:

  • Design for failure, but fix failures quickly
  • Create repeatable repairs
  • Standardize pod chassis
  • ROI drives automation
  • Workflow for storage buffer
  • Insource & use math for drive purchases
slide-33
SLIDE 33

@GlebBudman

Image by Michael Cara https://www.flickr.com/photos/michael_cala/
slide-34
SLIDE 34

@GlebBudman

No Hardware Worshippers

slide-35
SLIDE 35

@GlebBudman

http //365datacenters com/b og/ ocal-cloud-storage-beats-publ c-cloud/

Agile Extends to Hardware

slide-36
SLIDE 36

@GlebBudman

Relentless Focus on Cost

  • Is this required?
  • Is there a comparable lower cost option?
  • Can business processes work around this?
  • Can software work around this?
slide-37
SLIDE 37

@GlebBudman

Culture

Goal:

  • Question “conventional wisdom”

Considerations:

  • No hardware worshippers
  • Agile extends to hardware
  • Relentless focus on cost

Resource: Backblaze Agile Hardware Process

https://www.backblaze.com/blog/designing-the-next-storage-pod/

slide-38
SLIDE 38

@GlebBudman

Backblaze B2 Use Cases

When cost matters…

  • Archive
  • Backup
  • Tape Replacement
  • Compliance
  • Disaster Recovery

But also…

  • Any storage that is ok with Internet-latency
slide-39
SLIDE 39

@GlebBudman

Learnings from Operating 200 PB of Disk-Based Storage

Backblaze.com/B2 => Lowest cost cloud storage Backblaze.com/blog => Cloud storage resources

Gleb Budman @glebbudman Gleb.Budman@Backblaze.com