Outline PAST goals Storage management and caching PAST api in - - PowerPoint PPT Presentation

Storage management and caching in PAST

Antony Rowstron and Peter Druschel

Presented to cs294-4 by Owen Cooper

Outline

• PAST goals
• PAST api
• File storage overview
• File and replica diversion
• Replica management
• Caching
• Performance
• Discussion

PAST (non)goals

• P2P global storage network

– Use properties of existing p2p systems (Pastry) – Support for strong persistence

• Via a core set of replicas

– High availability

• Via local caching

– Scalable

• Obtain high storage utilization via local cooperation

– Secure

• Design goals do not include

– Replacing the file system – Updatable files – Directory or lookup service

Security Model

• Pastry node ids are a hash of a public key
• Smartcard based security

– Provides keys – Quota management

• Nodeid and fileid generation controlled

– Try to stop nodes from getting consecutive ids – Or clients from overloading parts of the network

• But node id and real world identity may not be

linked

• Data not encrypted

PAST API’s

• In PAST, files are immutable
• Fileid=Insert(filename,credentials, k, file)

– Insert k copies of the file into the network, or fail. – Fileid a signed (filename, credentials, salt) – Successful if ack with receipts from k nodes

• File=lookup(fileid)

– Return a copy of the file if it exists

• Reclaim(fileid, cradentials)

– Reclaim accepted if requested by the owner – Allows, but does not require, storage reclamation

File insertion

• Insert(name, c, k, file)

– Computes a storage certificate

• Contains fileid, hash of content, k, salt

– Deducts k*filesize from quota – Routes file and storage certificate using pastry using fileid. – Node verifies the integrity of the file, stores it, and asks k-1 closest nodes to store the file.

• K-1 nodes in leaf set (k-1 <= l)

– Node returns ack with k signed storage receipts, or a nak.

Lookup and Reclamation

• Pastry ensures replica is found

– Since a lookup is routed to the closest nodeid

• Reclamation

– Client generates a reclaim certificate – Sends it to the fileid via pastry – Recipients verify the certificate & issue receipt – Client reclaims quota

Diversion

• A file or replica can be relocated
• For a replica, to another close node

– If one of the K closest is overloaded

• For a file, to another set of nodes in the idspace

– If the nodes around a fileid are (possibly locally) congested

• Why is this necessary?

– Differing storage capacity at nodes – Differing file size for inserted files

Replica Diversion

• Node responsible for fileid asks k-1 neighbors to

store the file

• Neighbor (N) may divert a copy to a node in its

leaf set

– Pointer to copy inserted at N – N issues storage certificate – N also inserts a pointer on the k+1th closest node

• No orphan if N fails
• N remains responsible for pointer maintenance

File Diversion

• Replica diversion is local

– Allows storage choice between nodes around fileid

• File Diversion

– Triggered when an insert with a fileid fails – Insert is tried a total of three times – New fileid generated by changing the salt

Storage Policy

• How does a node choose to accept or reject a

replica?

– Computes sizeof(file)/sizeof(free_space) – Compares to Tpri or Tdiv depending node’s role – Tpri > Tdiv

• How is node chosen for replica diversion

– Search leaf set for the node that

• Has maximal free space
• Doesn’t already hold a diverted or primary replica
• File diversion

– K copies cannot be located (via primary or diversion)

Replica maintenance

• Node join/leave causes responsibility shift

– Pastry node failure detection will cause leaf set updates

• Past detects responsibility shifts this way
• Newly responsible node must copy files

– Make a copy immediately, OR – pointer to old owner & copy lazily

• Diverted replicas

– Target of diversion may move out of leaf set

• Node to store repica can be any one in leaf set

– Must exchange keepalive messages themselves – Should be relocated

Replica maintenance (2)

• Node failure may cause storage shortage

– No node in leaf set can take over ownership

• Search space is widened

– Ask most extreme nodes to locate storage

• Increases search space to 2l nodes

– If no storage space found, fail.

Caching

• Pastry’s locality based routing will tend to direct

requests to nearby copies

• PAST also stores cached copies

– Along routing path between client and fileid – For insert and lookup operations – Cache maintained using GD-size algorithm

• Weight per file: 1/size(file)
• Eviction:

– Pick file with minimum weight – Subtract weight of evicted file from all others

Experiments: without diversion

• Experiments use

– Large trace from web server – Files from local web server

• The case for diversion with web trace

– Without diversion:

• 51.1% of insertions failed
• 60.8% storage utilization

Experiments (2): with diversion

• With diversion

– Bigger leaf set size a plus

Experiments (3):varying Tpri

• Effects of varying Tpri
• # files stored v.s. size of file

Experiments (4): Varying Tdiv

• Varying Tdiv
• Tpri is constant

File and Replica Diversion caching

• 8 traces combined
• Requests from clients in each trace are mapped to close

PAST nodes