OpenStack Swift OpenStack Summit Atlanta 2014 Martin Lanner & - - PowerPoint PPT Presentation

▶
openstack swift
SMART_READER_LITE
LIVE PREVIEW

OpenStack Swift OpenStack Summit Atlanta 2014 Martin Lanner & - - PowerPoint PPT Presentation

Dec 13, 2022 540 likes •1.15k views

OpenStack Swift OpenStack Summit Atlanta 2014 Martin Lanner & Hugo Kuo May 15, 2014 Agenda Object Storage & Swift How Swift Works Installing Swift: Hands-on Lab Manual Swift installation (20 minutes)

slide-1
SLIDE 1

OpenStack Swift

OpenStack Summit Atlanta 2014

Martin Lanner & Hugo Kuo May 15, 2014

slide-2
SLIDE 2

Property of SwiftStack Inc. 2

Agenda

  • Object Storage & Swift

  • How Swift Works

  • Installing Swift: Hands-on Lab
  • Manual Swift installation (20 minutes)
  • SwiftStack installation (15 minutes)

  • Operating, Managing and Monitoring Swift

  • Failure Handling
slide-3
SLIDE 3

Property of SwiftStack Inc. 3

Why Object Storage?

  • Data grows at ~50% per year

  • 50%-75% of all data is unstructured or of archival nature

  • Modern application design, using RESTful API (HTTP)

  • High availability
  • Agile data centers
slide-4
SLIDE 4

Property of SwiftStack Inc. 4

Why Swift?

  • Open source

  • Proven at scale: > 100PB

  • Actively developed by ~15 core devs and > 150 committers

  • Deploy in your own data center
  • Unique features:
  • Multi-region cluster, geographic distribution of data
  • Storage policies
  • Erasure Coding
slide-5
SLIDE 5

How Swift Works

slide-6
SLIDE 6

Property of SwiftStack Inc. 6

Swift Design Goals

Reliable 
 Highly scalable 
 Hardware proof

No Single Point of Failure (High Availability) Assumes unreliable hardware Mix & match hardware vendors Configurable replica model with zones & regions Easy to use HTTP API – Developers don’t shard High Concurrency (Supports lots of users) Multi-tenant – each account has its own namespace Tier & scale any component in the system

slide-7
SLIDE 7

Property of SwiftStack Inc. 7

The Swift API

https:/ /swift.example.com/v1/AUTH_acct/cont/obj

Prefix API version

Account Container Object

slide-8
SLIDE 8

Property of SwiftStack Inc. 8

The Swift API

Write a new object: PUT /v1/account/container/object_name

  • Read an object:

GET /v1/account/container/object_name

slide-9
SLIDE 9

Property of SwiftStack Inc. 9

Swift Overview

  • RESTful API

  • Swift Components
  • Proxy Server
  • Account Server
  • Container Server
  • Object Server

  • The Rings
  • Regions
  • Zones
  • Devices
  • Partitions
slide-10
SLIDE 10

Property of SwiftStack Inc. 10

Parts of Swift

Proxy Account ¡| ¡Container ¡| ¡Object

slide-11
SLIDE 11

Property of SwiftStack Inc. 11

System Components: Proxy

WRITE REQUEST

Write Request

Client sends a write request to the cluster One proxy is randomly selected to serve request

  • CLIENT

REQUEST PROXY OBJECT
 NODES

slide-12
SLIDE 12

Property of SwiftStack Inc. 12

System Components: Proxy

WRITE REQUEST

Write Request

Proxy streams 3 writes to each storage location simultaneously There is no ‘master’ object

CLIENT REQUEST PROXY OBJECT
 NODES

slide-13
SLIDE 13

Property of SwiftStack Inc. 13

System Components: Proxy

WRITE REQUEST

Write Request

Client receives ‘OK’ if quorum write is successful

CLIENT REQUEST PROXY OBJECT
 NODES

slide-14
SLIDE 14

Property of SwiftStack Inc. 14

System Components: Proxy

READ REQUEST

Read Request

Proxy contacts a single available storage node and streams back data Prioritizes nearby storage locations Will try alternate replicas when there is failures

CLIENT REQUEST PROXY OBJECT
 NODES

slide-15
SLIDE 15

Property of SwiftStack Inc. 15

Durability with Replicas

  • Swift stores multiple replicas to protect data
  • 3 replicas provide a good balance between cost and durability guarantees

  • The number of replicas is determined by the Ring
slide-16
SLIDE 16

Property of SwiftStack Inc.

  • Swift stores MD5 checksum with each object

  • Returned in header so client can test

  • Uploads with a checksum

  • Swift only saves if checksums match

16

Durability with Checksums

MD5 Checksum MD5 Checksum MD5 Checksum + Checksum

slide-17
SLIDE 17

Property of SwiftStack Inc. 17

Swift Background Processes: Auditor & Replicator

Replicator Replicator Replicator

Missing?

  • Active auditing and replication

  • Asks other servers if they have a copy of this object

  • Replicator will push object where missing
slide-18
SLIDE 18

Property of SwiftStack Inc. 18

Disk Weights

Weight: 2000 Weight: 3000

3TB 2TB

3TB Disks 2TB Disks

3TB 2TB 3TB

Any size disk can be used in a cluster. A weight is assigned to each disk. Every individual disk’s weight is relative to all other disks in the cluster.

slide-19
SLIDE 19

Property of SwiftStack Inc. 19

Swift Partitions

Swift partitions are NOT your regular Linux disk partitions.

slide-20
SLIDE 20

Property of SwiftStack Inc. 20

Swift Partitions: Directories on Disk

swiftstack@node1:/srv/node/d14/objects$ ls

  • 100005 107626 115455 130228 140520 148705 157620 166997 176011 193357 202156

100009 10763 115459 130238 140533 148724 157636 167 176027 193370 202161 100014 107634 115471 130240 140535 148734 157642 167003 176044 193377 202165 100020 107636 115476 130268 140536 148743 15765 167005 176045 193382 202167 100028 107648 115479 13029 140541 148747 157664 167006 176051 193388 202169 100056 107656 11550 130296 140544 148749 157666 167008 176054 193390 202170 100071 107659 115504 130297 14055 148751 157671 167026 176055 193391 202177 100072 107663 115512 130302 140560 148752 157675 167033 176061 193395 202197 100084 107666 115513 13033 140593 148761 157678 167041 176065 193397 202199 100095 107667 115522 130330 140617 148762 15768 167042 176070 19340 202219 100103 10767 115528 130335 140622 148766 157691 167054 17608 193402 202223 100105 107675 115535 130336 140629 148785 157702 167074 176084 193428 202224 100114 107685 115537 130340 140631 148788 157707 167076 17609 193435 202228 100115 107692 11554 130348 140637 14879 157709 167086 176092 19344 202236

slide-21
SLIDE 21

Property of SwiftStack Inc. 21

Swift Partitions - 1 Node

Node 1 8 Disks - 16 Partitions/Disk

Disk 1 Disk 2 Disk 3 Disk 4 Disk 5 Disk 6 Disk 7 Disk 8

8 * 16 = 128 partitions

Example: Assuming equally weighted disks.

slide-22
SLIDE 22

Property of SwiftStack Inc. 22

Swift Partitions - Adding A Node: Partitions Are Reassigned

Node 2 8 Disks - 8 Partitions/Disk

Disk 9 Disk 10 Disk 11 Disk 12 Disk 13 Disk 14 Disk 15 Disk 16

Node 1 8 Disks - 8 Partitions/Disk

Disk 1 Disk 2 Disk 3 Disk 4 Disk 5 Disk 6 Disk 7 Disk 8

16 * 8 = 128 partitions

+

slide-23
SLIDE 23

Property of SwiftStack Inc. 23

The Ring - Like An Encyclopedia

A C E G I K M O Q S U W Y AA AC … B D F H J L N P R T V X Z AB … …

Swift

slide-24
SLIDE 24

Property of SwiftStack Inc. 24

But instead of letters,

Swift uses hashes

for each:

The Ring - Like An Encyclopedia

slide-25
SLIDE 25

Property of SwiftStack Inc.

Hashed location on disk:

/srv/node/d16/objects/53180/60a/33ef27644ca718d383c98199c31cb60a/1388912539.97756.data

  • 25

The Ring - Object Location Mapping

Account: AUTH_user1 Container: photos Object: cloudcat.jpg

  • Partition: 53180

Hash: 33ef27644ca718d383c98199c31cb60a Three last characters Full hash Object timestamp

slide-26
SLIDE 26

Property of SwiftStack Inc. 26

The Ring - Partition

/53180/60a/33ef27644ca718d383c98199c31cb60a/1388912539.97756.data

Partition

53180

slide-27
SLIDE 27

Property of SwiftStack Inc. 27

Ring Building Process

Ensures that only one replica is in flight

min_part_hours min_part_hours

Min-Part-Hours: 24 hours default setting

How quickly partitions can move

3

Ring Builder Database

1

Replicas

2

Partitions

3

Min-Part- Hours

Number of replicas

1

Single Region Cluster Recommendation: 3

Number of partitions

2

How big will the cluster be? Rule of thumb: 100 partitions * the number of drives that you think 
 you will ever have, rounded up to the nearest power of 2

slide-28
SLIDE 28

Property of SwiftStack Inc. 28

As Unique As Possible

DC 1 DC 2

Single Node Three Nodes Multiple Zones Multi-Region Cluster Three Disks

slide-29
SLIDE 29

Installing Swift - Lab

Manual Swift Installation (20 min)

slide-30
SLIDE 30

Property of SwiftStack Inc. 30

Log in to Your Server

$ ssh demo@<your-vm-ip>

  • See *** Swift from CLI *** card for IP address
  • Password: password
slide-31
SLIDE 31

Property of SwiftStack Inc. 31

Step 2: Format Devices

$ df -h $ blkid -o list $ sudo su -
 # mkfs.xfs -f -i size=512 -L d1 /dev/mapper/v-xvdd
 # mkfs.xfs -f -i size=512 -L d2 /dev/mapper/v-xvde
 # mkfs.xfs -f -i size=512 -L d3 /dev/mapper/v-xvdf # mkfs.xfs -f -i size=512 -L d4 /dev/mapper/v-xvdg # mkfs.xfs -f -i size=512 -L d5 /dev/mapper/v-xvdh $ blkid -o list

slide-32
SLIDE 32

Property of SwiftStack Inc. 32

Step 3: Mount Drives

# mkdir -p /srv/node/d1
 # mkdir -p /srv/node/d2
 # mkdir -p /srv/node/d3 # mkdir -p /srv/node/d4 # mkdir -p /srv/node/d5 # mount -t xfs -L d1 /srv/node/d1
 # mount -t xfs -L d2 /srv/node/d2
 # mount -t xfs -L d3 /srv/node/d3 # mount -t xfs -L d4 /srv/node/d4 # mount -t xfs -L d5 /srv/node/d5 # chown -R swift:swift /srv/node

slide-33
SLIDE 33

Property of SwiftStack Inc. 33

Step 4: Create The Builder Files

# cd /etc/swift
 # swift-ring-builder account.builder create 14 3 1
 # swift-ring-builder container.builder create 14 3 1
 # swift-ring-builder object.builder create 14 3 1 # cd /etc/swift
 # p=0
 # for t in object container account; do
 > for i in 1 2 3 4 5; do
 > swift-ring-builder $t.builder add z$i-127.0.0.1:600$p/d$i 100
 > done
 > let p++
 > done

slide-34
SLIDE 34

Property of SwiftStack Inc. 34

Step 4: Continued …

# swift-ring-builder account.builder
 # swift-ring-builder container.builder
 # swift-ring-builder object.builder

slide-35
SLIDE 35

Property of SwiftStack Inc. 35

Step: Create The Rings

# cd /etc/swift
 # swift-ring-builder account.builder rebalance
 # swift-ring-builder container.builder rebalance
 # swift-ring-builder object.builder rebalance # ls *.ring.gz account.ring.gz container.ring.gz object.ring.gz

Output should be:

slide-36
SLIDE 36

Property of SwiftStack Inc. 36

Step 6: Start Swift

# swift-init main restart # tail -f /var/log/swift/all.log

slide-37
SLIDE 37

Property of SwiftStack Inc. 37

Step 7: Use The Swift CLI Client

# cd /home/demo # swift -U admin:admin -K admin \


  • A http://127.0.0.1/auth/v1.0 upload cats cloudcat.jpg

Upload the object cloudcats.jpg into the cats container:

# swift -U admin:admin -K admin \

  • A http://127.0.0.1/auth/v1.0 list cats

List files in the cats container:

# swift -U admin:admin -K admin \

  • A http://127.0.0.1/auth/v1.0 download cats cloudcat.jpg

Download cloudcats.jpg from the cats container:

slide-38
SLIDE 38

Property of SwiftStack Inc. 38

Title text

Step 8: Serving Data out of Swift

# swift -U admin:admin -K admin \

  • A http://127.0.0.1/auth/v1.0/ post -r '.r:*' cats

Make the cats container world readable: T

  • view the image, open your web browser and go to:

http://<your-vm-ip>/v1/AUTH_admin/cats/cloudcat.jpg

slide-39
SLIDE 39

Installing Swift - Lab

SwiftStack Swift Installation (15 min)

slide-40
SLIDE 40

Property of SwiftStack Inc. 40

Log in to Your SwiftStack Node

$ ssh demo@<your-vm-ip>

  • See SwiftStack handout for your specific IP address
  • Password: password
slide-41
SLIDE 41

Property of SwiftStack Inc. 41

Log in to SwiftStack Controller

https:/ /try.swiftstack.com

slide-42
SLIDE 42

Property of SwiftStack Inc. 42

Install Swift Using SwiftStack

$ curl https://try.swiftstack.com/install_ubuntu | bash

+----------------------------------------------------------------------------+ | | | Your claim URL is: | | https://try.swiftstack.com/claim/09f7d921-4756-11e3-8016-bc764e04efd3 | | | +----------------------------------------------------------------------------+

Output should look similar to: SwiftStack installation command:

slide-43
SLIDE 43

Property of SwiftStack Inc. 43

Create New Cluster & Configure

slide-44
SLIDE 44

Property of SwiftStack Inc. 44

Deploying your single node cluster

slide-45
SLIDE 45

Property of SwiftStack Inc. 45

Authentication

  • SwiftStack Auth
  • LDAP
  • Keystone
  • Active Directory
slide-46
SLIDE 46

Property of SwiftStack Inc. 46

Integrations

  • Load Balancing
  • SSL
  • CDN Integration
  • Billing / Utilization
  • Quotas
  • CIFS / NFS Gateways
  • File Managers & File System Adapters
  • Backup
slide-47
SLIDE 47

Property of SwiftStack Inc. 47

Upload An Object

$ cd /home/demo $ swift -A localhost/auth/v1.0 -U user1 \

  • K password photos cloudcat.jpg

Using the Swift Command Line Client, upload an object to the cluster:

slide-48
SLIDE 48

Property of SwiftStack Inc. 48

Find Where Objects Are

$ sudo /opt/ss/bin/swift-get-nodes \ /etc/swift/object.ring.gz \ AUTH_demo/photos/cloudcat.jpg

Using the swift-get-nodes command, find on which disks

  • bjects are located:
slide-49
SLIDE 49

Operating, Managing and Monitoring Swift

slide-50
SLIDE 50

Property of SwiftStack Inc. 50

Capacity Adjustments & Monitoring

slide-51
SLIDE 51

Property of SwiftStack Inc. 51

Swift Stats

slide-52
SLIDE 52

Property of SwiftStack Inc. 52

Node Stats

slide-53
SLIDE 53

Failure Handing

slide-54
SLIDE 54

Property of SwiftStack Inc. 54

CLIENT REQUEST

OBJECT
 NODES

3 Replicas: Successfully written

WRITE REQUEST PROXY

slide-55
SLIDE 55

Property of SwiftStack Inc. 55

PROXY OBJECT
 NODES

Failure Handling: Bad Disk

AUDITOR & REPLICATOR

Handoff Locations

The replicators will proactively push
 replicas to handoff locations CLIENT REQUEST

slide-56
SLIDE 56

Property of SwiftStack Inc. 56

Failure Handling: Node Down

Replication works on Swift partitions on disks

Partitions, not drives or files are replicated during a recovery

Replication will move partitions to other nodes

slide-57
SLIDE 57

Cluster Topology Example

slide-58
SLIDE 58

Property of SwiftStack Inc. 58

Cluster Layout - Sample

Swi$Stack ¡ Controller Firewall Load ¡Balancer ¡ HA ¡(10G) Core ¡Switch Core ¡Switch Aggrega7on ¡Switch

LEGEND ¡

  • O utward-­‑fac ing ¡ (10G ) ¡
  • C luste r-­‑fac ing ¡ (10G ) ¡
  • C ontrolle r ¡ networ k ¡ (1G ) ¡
  • O ut-­‑ of-­‑ band ¡ (1G )

ToR ¡Switch ¡(10G) Proxy Account ¡| ¡Container ¡| ¡Object Account ¡| ¡Container ¡| ¡Object Account ¡| ¡Container ¡| ¡Object Account ¡| ¡Container ¡| ¡Object Mgmt ¡Switch ¡(1G) Rack 1 Zone 1 Rack 2 Zone 2 Rack 3 Zone 3 ToR ¡Switch ¡(10G) Proxy Account ¡| ¡Container ¡| ¡Object Account ¡| ¡Container ¡| ¡Object Account ¡| ¡Container ¡| ¡Object Account ¡| ¡Container ¡| ¡Object Mgmt ¡Switch ¡(1G) Aggrega7on ¡Switch ToR ¡Switch ¡(10G) Proxy Account ¡| ¡Container ¡| ¡Object Account ¡| ¡Container ¡| ¡Object Account ¡| ¡Container ¡| ¡Object Account ¡| ¡Container ¡| ¡Object Mgmt ¡Switch ¡(1G)

slide-59
SLIDE 59

Thank You!