PROST: A Programmable Structured Peer-to-Peer overlay Network - - PowerPoint PPT Presentation

PROST: A Programmable Structured Peer-to-Peer

• verlay Network

Marius Portmann, S Marius Portmann, Sé ébastien Ardon bastien Ardon Patrick S Patrick Sé énac nac

University of Queensland, Brisbane, Australia ENSICA Toulouse, France

Prost: a Programmable structured P2P system

Context: overlay networks

NETWORK END-HOSTS OVERLAY

Prost: a Programmable structured P2P system

Structured peer-to-peer systems

Each node assigned and responsible for a chunk of key Each node assigned and responsible for a chunk of key space space Mapping of resources to key space done using a hash Mapping of resources to key space done using a hash function (e.g. SHA-1) function (e.g. SHA-1)

– – Most famous: Distributed Hash Tables Most famous: Distributed Hash Tables

2

128

chunk chunk key space

Prost: a Programmable structured P2P system

Example: DHT with CHORD

Circular Circular ID ID space space ( (wraps wraps) ) Each node stores (key,values) for all keys Each node stores (key,values) for all keys smaller than its node id but greater than smaller than its node id but greater than previous node id. previous node id. When node join/leave key,values are When node join/leave key,values are migrated to neighbor nodes migrated to neighbor nodes hash function is SHA1: nice uniform random hash function is SHA1: nice uniform random

• utput (load balancing)
• utput (load balancing)

Routing is used Routing is used to to find the node responsible find the node responsible for a for a particular key particular key A A node node uses « uses « shortcuts shortcuts » » along the along the ring: ring: fingers fingers Fingers Fingers are are arranged exponentially starting arranged exponentially starting from the node from the node ID. ID. Each node need Each node need to to find the closest find the closest predecessor predecessor to to the the destination destination key key O(log N) messages O(log N) messages exchanged exchanged for for routing routing from any from any source to source to any any destination destination Each node export simple hash tables Each node export simple hash tables manipulations functions (get/put) manipulations functions (get/put)

4

50 80 60

dest = 78

7

27 14

keyspace

14 27 7

…

50

Node id

IP@

Prost: a Programmable structured P2P system

Structured P2P applications

Cooperative File System (CFS) Cooperative File System (CFS)

– – File storage, uses CHORD at the block level File storage, uses CHORD at the block level

PAST PAST

– – large-scale, Internet-based, global storage utility that provides scalability, high large-scale, Internet-based, global storage utility that provides scalability, high availability, persistence and security availability, persistence and security

I3 I3

– – Communication primitive (indirection) based on CHORD Communication primitive (indirection) based on CHORD

SCRIBE SCRIBE

– – scalable application-level multicast infrastructure, event notification infrastructure scalable application-level multicast infrastructure, event notification infrastructure

SplitStream SplitStream

– – High-bandwidth content distribution system based on application-level multicast High-bandwidth content distribution system based on application-level multicast

BAYEUX BAYEUX

– – application-level multicast built on top of Tapestry application-level multicast built on top of Tapestry

OCEANSTORE OCEANSTORE

– – persistent and resilient data storage persistent and resilient data storage

Prost: a Programmable structured P2P system

PROST idea

Proliferation of new Proliferation of new “ “DHT DHT” ”-based applications, each with

• based applications, each with

their own topology their own topology

– – Re-inventing the wheel Re-inventing the wheel

One network per application! One network per application!

– – Overhead: topology maintenance Overhead: topology maintenance

Idea: share the topology, routing and topology Idea: share the topology, routing and topology maintenance between applications maintenance between applications

– – Common DHT API Common DHT API

Not enough: most applications need more than DHT Not enough: most applications need more than DHT

Dynamically deploy application code on end-nodes Dynamically deploy application code on end-nodes (peers) (peers)

– – Take the programmable network paradigm to the peer-to-peer Take the programmable network paradigm to the peer-to-peer layer layer – – Allow to deploy new applications on a large scale, by leveraging Allow to deploy new applications on a large scale, by leveraging an existing peer base an existing peer base

Prost: a Programmable structured P2P system

A Layered model of structured p2p

Dabek, F. et al, “Towards a common API for structured P2P overlays”, IPTPS’03

Prost: a Programmable structured P2P system

PROST

Started as an enhanced DHT Started as an enhanced DHT

– – Adding functionalities to DHT Adding functionalities to DHT – – Problem to maintain consistency and versioning Problem to maintain consistency and versioning

Why not use DOLR? Why not use DOLR?

– – Objects residing on peers to provide application Objects residing on peers to provide application functionalities functionalities

must be previously deployed must be previously deployed

Solution: dynamically deploy objects into peers, Solution: dynamically deploy objects into peers,

• n-demand
• n-demand

Prost: a Programmable structured P2P system

Node architecture

Key-based Routing Layer (KBR ) Peerlet Manager Peerlet Peerlet Peerlet Programmable Peer Layer User Interface Component User Interface Component

Prost: a Programmable structured P2P system

Two types of applications

KBR

Application (Peerlet)

. . .

KBR KBR

Application (Peerlet)

KBR

Application (Peerlet)

. . .

KBR KBR

Application (Peerlet) Application (Peerlet)

(a) (b)

End-node applications: file sharing, storage, etc Per-hop applications: multicast communication, etc

Prost: a Programmable structured P2P system

PROST messages

DST KEY: destination key (ID) for this message DST KEY: destination key (ID) for this message APP ID: unique application-specific ID APP ID: unique application-specific ID APP TYPE: 1: end-node application, 0: per-hop APP TYPE: 1: end-node application, 0: per-hop application application

– – Analogy to Router Alert IP header field Analogy to Router Alert IP header field

Peerlet Code Locator: where to download the code if Peerlet Code Locator: where to download the code if needed needed Application Data: application-specific data, e.g. get/put Application Data: application-specific data, e.g. get/put

• peration and parameters
• peration and parameters

dst key app id app type peerlet code locator application data

Prost: a Programmable structured P2P system

PROST API

Based on Based on Dabek Dabek et al. et al. “ “Towards a common API Towards a common API for structured P2P overlays for structured P2P overlays” ”

– – route(k, route(k,msg msg, ,nextHopHint nextHopHint): ): send a message send a message to destination key k to destination key k – – forward(k, forward(k,msg msg, ,nextHop nextHop): ): upcall to peerlet at upcall to peerlet at intermediate nodes intermediate nodes – – deliver(k, deliver(k,msg msg) ) – – Plus access to local routing information: Plus access to local routing information:

local_lookup local_lookup( (… …), ), neighbour_set neighbour_set( (… …), ), replicaSet replicaSet( (… …), update( ), update(… …), range( ), range(… …) )

Prost: a Programmable structured P2P system

Typical operation

dst key = 78 App ID = ‘P2PTV v1.9’ App TYPE = ‘1’ Peerlet code locator = ‘https://secure.p2ptv.com/cgi?code-current App DATA: addReceiver(4,…)

4

50 80 60

7

27 14

PROST NETWORK

appType=1? appID installed? yes

download and install peerlets

no forward(78,appData,60) yes

P2PTV peerlet

route(80,60,appData) deliver(78,appData)

Prost: a Programmable structured P2P system

Node security policies

Security policies are configured in each Security policies are configured in each node node

– – allow/deny peerlets to be installed, using allow/deny peerlets to be installed, using

peerlet provider identity peerlet provider identity application ID application ID local policy (deny/allow all) local policy (deny/allow all)

Allow graceful degradation of the service Allow graceful degradation of the service

– – if peerlet denied installation, try first node in if peerlet denied installation, try first node in replicaSet replicaSet (recursively) (recursively)

Prost: a Programmable structured P2P system

Challenges

Similar to programmable networks: Similar to programmable networks:

– – resource management resource management

admission control admission control

– – security security

trust model, code authentication trust model, code authentication sandboxing sandboxing use to Crypto-based IDs use to Crypto-based IDs

New to p2p-based New to p2p-based “ “active active” ” networking networking

– – dynamicity of execution environment: nodes join and dynamicity of execution environment: nodes join and leave leave

process migration? process migration?

Prost: a Programmable structured P2P system

Implementation

Implementation with CHORD, in java Implementation with CHORD, in java dynamic code execution through network class dynamic code execution through network class loader loader No resource management, plan to use existing No resource management, plan to use existing system such as system such as KaffeOS KaffeOS

– – support the OS abstraction of support the OS abstraction of process process – – process separation process separation – – precise memory and CPU accounting precise memory and CPU accounting

No process migration, for the moment consider No process migration, for the moment consider process are stateless process are stateless

Prost: a Programmable structured P2P system

Future work

further investigate the use of strong verifiable further investigate the use of strong verifiable identities identities admission control, resource management admission control, resource management

– – use results from the AN community use results from the AN community

develop and promote PROST as a P2P develop and promote PROST as a P2P research experimentation platform research experimentation platform

– – implementation using the Bamboo DHT implementation using the Bamboo DHT – – large scale deployment over large scale deployment over planetlab planetlab

Prost: a Programmable structured P2P system

questions questions