[P2P S YSTEMS ] Shrideep Pallickara Computer Science Colorado - PDF document

CS555: Distributed Systems [Fall 2019] Dept. Of Computer Science , Colorado State University CS 555: D ISTRIBUTED S YSTEMS [P2P S YSTEMS ] Shrideep Pallickara Computer Science Colorado State University CS555: Distributed Systems [Fall 2019] September 10, 2019 L5.1 Dept. Of Computer Science , Colorado State University Frequently asked questions from the previous class survey ¨ Can vector clocks be reset if the count reaches Long.MAX_VALUE? ¨ Can vector clocks handle processes failing at runtime? ¨ In the case of multicasting with Vector clocks, what happens if the message is lost? L5. 2 CS555: Distributed Systems [Fall 2019] September 10, 2019 Dept. Of Computer Science , Colorado State University Professor: S HRIDEEP P ALLICKARA L5.1 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA

CS555: Distributed Systems [Fall 2019] Dept. Of Computer Science , Colorado State University Topics covered in this lecture ¨ Peer to Peer (P2P) Systems ¤ P2P middleware and requirements ¤ Overlays ¤ The P2P lookup problem ¤ Implementing DHTs L5. 3 CS555: Distributed Systems [Fall 2019] September 10, 2019 Dept. Of Computer Science , Colorado State University Professor: S HRIDEEP P ALLICKARA P2P M IDDLEWARE CS555: Distributed Systems [Fall 2019] September 10, 2019 L5.4 Dept. Of Computer Science , Colorado State University L5.2 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA

CS555: Distributed Systems [Fall 2019] Dept. Of Computer Science , Colorado State University P2P middleware is designed to orchestrate ¨ Automatic placement of resources (data items, objects, files, etc. ) ¨ Subsequent location ( discovery ) of distributed resources L5. 5 CS555: Distributed Systems [Fall 2019] September 10, 2019 Dept. Of Computer Science , Colorado State University Professor: S HRIDEEP P ALLICKARA How different P2P generations cope with this issue ¨ 1 st Generation ¤ Maintain a centralized index of available files ¤ Files are stored at the peers ¨ 2 nd Generation ¤ Systems such a Gnutella & Freenet employ partitioned distributed indexes ¨ 3 rd Generation ¤ Rely on Overlays L5. 6 CS555: Distributed Systems [Fall 2019] September 10, 2019 Dept. Of Computer Science , Colorado State University Professor: S HRIDEEP P ALLICKARA L5.3 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA

CS555: Distributed Systems [Fall 2019] Dept. Of Computer Science , Colorado State University Requirements for P2P systems ¨ Functional ¤ Specific behaviors or functions that must be supported ¨ Non-functional (or evaluation metrics) ¤ Criteria that can be used to judge the operation of a system L5. 7 CS555: Distributed Systems [Fall 2019] September 10, 2019 Dept. Of Computer Science , Colorado State University Professor: S HRIDEEP P ALLICKARA Functional requirements for P2P middleware ¨ Locate and communicate with any resource made available to the system ¤ Even though resources are dispersed over a large number of nodes ¨ The ability to add and remove both resources and nodes at will L5. 8 CS555: Distributed Systems [Fall 2019] September 10, 2019 Dept. Of Computer Science , Colorado State University Professor: S HRIDEEP P ALLICKARA L5.4 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA

CS555: Distributed Systems [Fall 2019] Dept. Of Computer Science , Colorado State University Non-functional requirements for P2P systems ¨ Scalability ¨ Load balancing ¨ Dynamic host availability L5. 9 CS555: Distributed Systems [Fall 2019] September 10, 2019 Dept. Of Computer Science , Colorado State University Professor: S HRIDEEP P ALLICKARA Non-functional requirements: Load balancing ¨ Achieved via random placement of resources ¨ Replicas of heavily used resources are created L5. 10 CS555: Distributed Systems [Fall 2019] September 10, 2019 Dept. Of Computer Science , Colorado State University Professor: S HRIDEEP P ALLICKARA L5.5 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA

CS555: Distributed Systems [Fall 2019] Dept. Of Computer Science , Colorado State University Accommodate highly dynamic host availability ¨ Host computers are free to join or leave at any time ¨ Provide a dependable service, from unreliable nodes ¨ As nodes join the system ¤ Must be integrated into the system ¤ Load must be redistributed to exploit their capabilities ¨ As nodes leave the system (voluntarily or involuntarily)? ¤ Redistribute their load and resources n Replication levels for some resources must be preserved L5. 11 CS555: Distributed Systems [Fall 2019] September 10, 2019 Dept. Of Computer Science , Colorado State University Professor: S HRIDEEP P ALLICKARA S YSTEMS THAT WE WILL LOOK AT CS555: Distributed Systems [Fall 2019] September 10, 2019 L5.12 Dept. Of Computer Science , Colorado State University L5.6 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA

CS555: Distributed Systems [Fall 2019] Dept. Of Computer Science , Colorado State University Systems that we will observe closely ¨ 1 st Generation ¤ Napster ¨ 3 rd Generation ¤ Chord ¤ Pastry ¤ Tapestry ¨ Unstructured P2P or 2 nd Generation ¤ Gnutella and BitTorrent L5. 13 CS555: Distributed Systems [Fall 2019] September 10, 2019 Dept. Of Computer Science , Colorado State University Professor: S HRIDEEP P ALLICKARA N APSTER CS555: Distributed Systems [Fall 2019] September 10, 2019 L5.14 Dept. Of Computer Science , Colorado State University L5.7 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA

CS555: Distributed Systems [Fall 2019] Dept. Of Computer Science , Colorado State University Napster ¨ First application in which demand for massively scalable storage and retrieval arose ¤ Downloading of digital music files ¨ Became very popular soon after its launch ¨ At its peak ¤ Several million users ¤ Thousands swapped music files simultaneously L5. 15 CS555: Distributed Systems [Fall 2019] September 10, 2019 Dept. Of Computer Science , Colorado State University Professor: S HRIDEEP P ALLICKARA Key features of the architecture ¨ Centralized indexes ¨ Users supplied the files ¤ Stored and accessed on their personal computers ¨ Clients add their own music files to the pool of shared resources ¤ Transmit a link to Napster’s indexing service for each available file ¤ Shared resources at the “ edge of the internet ” L5. 16 CS555: Distributed Systems [Fall 2019] September 10, 2019 Dept. Of Computer Science , Colorado State University Professor: S HRIDEEP P ALLICKARA L5.8 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA

CS555: Distributed Systems [Fall 2019] Dept. Of Computer Science , Colorado State University Napster Architecture Peer Peer Peer User’s User’s User’s Napster Server Computer Computer Computer Index (1) File location request (3) File Request Peer (2) List of peers User’s offering the file Computer Peer (4) File Delivered User’s (5) Index Update Computer L5. 17 CS555: Distributed Systems [Fall 2019] September 10, 2019 Dept. Of Computer Science , Colorado State University Professor: S HRIDEEP P ALLICKARA O VERLAYS [ U SED IN 3 RD G ENERATION P2P S YSTEMS ] CS555: Distributed Systems [Fall 2019] September 10, 2019 L5.18 Dept. Of Computer Science , Colorado State University L5.9 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA

CS555: Distributed Systems [Fall 2019] Dept. Of Computer Science , Colorado State University Overlays ¨ A distributed algorithm takes the responsibility of locating nodes and objects ¤ This is the routing overlay ¨ Denotes that the middleware is a layer that is responsible for routing requests ¤ From a client to the host that holds the requested object L5. 19 CS555: Distributed Systems [Fall 2019] September 10, 2019 Dept. Of Computer Science , Colorado State University Professor: S HRIDEEP P ALLICKARA But why call it an overlay? ¨ Denotes that it implements a routing mechanism in the application layer ¤ This is different from routing mechanisms deployed at the network level, e.g., IP ¨ A logical hop in the routing overlay, encompasses multiple underlying router hops L5. 20 CS555: Distributed Systems [Fall 2019] September 10, 2019 Dept. Of Computer Science , Colorado State University Professor: S HRIDEEP P ALLICKARA L5.10 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA

CS555: Distributed Systems [Fall 2019] Dept. Of Computer Science , Colorado State University What does the routing overlay do? ¨ Ensures any node can access any object ¨ Routes requests through a sequence of nodes ¤ Exploits (local) knowledge at each of the intermediate nodes to locate the destination object ¨ If there are multiple replicas of objects? ¤ Overlay maintains knowledge of all available replicas, and then delivers request to the nearest “live” node L5. 21 CS555: Distributed Systems [Fall 2019] September 10, 2019 Dept. Of Computer Science , Colorado State University Professor: S HRIDEEP P ALLICKARA O VERLAY VS IP R OUTING CS555: Distributed Systems [Fall 2019] September 10, 2019 L5.22 Dept. Of Computer Science , Colorado State University L5.11 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA