Lab 1 Introduction Bulletin Board System Architecture and - PDF document

Schedule • The Ensemble system Lab 1 – Introduction Bulletin Board System – Architecture and Protocols – How does Ensemble achieve the group communication properties ? • The Bulletin Board System Andreas Larsson 2009-01-26 Lab 1 - Bulletin Board System 2 The Ensemble System Terminology • A library of protocols that support group • Deliver a message communication. – Send it upwards in the stack • Ensemble Provides – Here from ensemble to the program that uses ensemble – Group membership service – Reliable communication – Failure detector – Secure communication Lab 1 - Bulletin Board System 3 Lab 1 - Bulletin Board System 4 Group membership service Reliable communication • Endpoints • Multicast communication – Abstraction for a communicating entity – Messages are delivered by all group members – Normally one per process • in the current view of the sender. – Based on IP-multicast • Groups – Corresponds to a set of endpoints that communicates • Point-to-Point communication – Just a name for endpoints to use • Properties: • Views – Virtual synchrony – A snapshot of the group membership at a specified – Stability point – Ordering • May change from time to time – Maintaining membership Lab 1 - Bulletin Board System 5 Lab 1 - Bulletin Board System 6

Virtual synchrony Virtual Synchrony • AKA: View-synchronous group communication V 0 ={a,b,c} V 1 ={a,b,c,d} V 2 ={b,c,d} V 3 ={b,c,d,e} • Integrity a crash – A correct process delivers a message at most once. b • Validity c – A message from a correct process will be delivered d eventually by that process d want to join e • Agreement – A message delivered by one correct process will be e want to join delivered by all correct processes Lab 1 - Bulletin Board System 7 Lab 1 - Bulletin Board System 8 Schedule Infrastructure • The Ensemble system • Layered protocol architecture – All features are implemented as micro- – Introduction protocols/layers – Architecture & Protocols – A stack/combination ~ a high-level protocol – How does Ensemble achieve the group communication properties ? • A new stack is created for a new • The Bulletin Board System configuration at each endpoint • Ability to change the group protocol on the fly Lab 1 - Bulletin Board System 9 Lab 1 - Bulletin Board System 10 Layers Stacks • Layers are implemented as a set of • Combinations of layers that work together to callbacks that handle events passed to provide high-level protocols them. • Stack creation: – Each layer gives the system 2 callbacks to – A new protocol stack is created at each endpoint of a handle events from its adjacent layers group whenever the configuration (e.g. the view) of the group changes. – Layers use 2 callbacks of its adjacent layers for passing events. – All endpoint in the same partition receive the same ViewState record to create their stack: • select appropriate layers according to the ViewState • Each instance of a layer maintain a private • create a new local state for each layer local state. • compose the protocol layers • connect to the network Lab 1 - Bulletin Board System 11 Lab 1 - Bulletin Board System 12

Schedule The basic stack • Each group has a Layers Functionality • The Ensemble system leader for the – Introduction membership protocol. Gmp Membership algorithm (7 layers) Slander Failure suspicion sharing – Architecture & Protocols Synch Block during membership change – How does Ensemble achieve the group Top_appl Interface to the application communication properties ? Sequencer Total ordering • The Bulletin Board System Suspect Failure detector Stable Stability detection Mnak Reliable fifo Bottom Interface to the network Lab 1 - Bulletin Board System 13 Lab 1 - Bulletin Board System 14 Failure detector Stability • Suspect layer: • Stable layer: – Regularly ping other members to check for suspected – Track the stability of multicast messages failures – Protocol: – Protocol: • If (#unacknowledged Ping messages for a member > threshold) • Maintain Acks[N][N] by unreliable multicast: send a Suspect event down – Acks[s][t]: #( s ’ messages) that t has acknowledged • Slander layer: – Stability vector – Share suspicions between members of a partition StblVct = {(minimum of row s): ∀ s} • The leader is informed so that faulty members are removed, even – NumCast vector if the leader does not detect the failures. NumCast = {(maximum of row s): ∀ s} – Protocol: • Occasionally, recompute StblVct and NumCast , then • The protocol multicasts slander messages to other members send them down in a Stable event. whenever receiving a new Suspect event Lab 1 - Bulletin Board System 15 Lab 1 - Bulletin Board System 16 Reliable multicast Ordering property • Sequencer layer: • Mnak layer: – Provide total ordering – Implements a reliable FIFO-ordered multicast protocol – Protocol: • Messages from live members are delivered reliably • Messages from faulty members are retransmitted by live • Members buffer all messages received from below in members a local buffer – Protocol: • The leader periodically multicasts an ordering • Keep a record of all multicast messages to retransmit on message demand • Members deliver the buffered messages according to • Use Stable event from Stable layer: the leader’s instructions – StblVct vector is used for garbage collection – NumCast vector gives an indication to lost messages � • See Causal layer for causal ordering recover them Lab 1 - Bulletin Board System 17 Lab 1 - Bulletin Board System 18

Maintaining membership (2) Maintaining membership (1) • Handle Failure by splitting a group into several subgroups: 1 • Recover failure by merging non-primary primary and many non-primary (partitionable) subgroups to the primary subgroup • Protocol: – Each member keeps a list of suspected members via Suspect layer • Protocol: – A member shares its suspicions via Slander layer – View leader l: l : local leader, r : remote leader • collect all suspicions 1. l synchronizes its view • reliably multicast a fail(p i0 ,…,p ik ) message • synchronize the view via Synch layer 2. l sends a merge request to r • Install a new view without p i0 ,…,p ik – A new leader is elected for the view without leader 3. r synchronizes its view • If p k in view V 1 suspects that all lower ranked members are faulty, it 4. r installs a new view with its mergers and sends the elects itself as leader and does like l. • A member that agrees with p k , continues with p k to the new view V 2 with view to l p k as the leader. 5. l installs the new view in its subgroup • A member that disagrees with p k , suspects p k . Lab 1 - Bulletin Board System 19 Lab 1 - Bulletin Board System 20 Virtual synchrony Join Group • Achieved by a simple leader-based protocol: – Idea: V 0 ={a,b,c} V 1 ={a,b,c,d} V 2 ={b,c,d} V 3 ={b,c,d,e} • Before a membership change from V 1 to V 2 all a crash messages in V 1 must become stable b – Protocol: before any membership change c • The leader activates the Synch protocol � the set, d M V1 , of messages needed to deliver in V 1 is bounded. d want to join • The leader waits until live members agree on M V1 via e sending negative acknowledgements and recovering e want to join lost messages (i.e. StblVct = NumCast ) Lab 1 - Bulletin Board System 21 Lab 1 - Bulletin Board System 22 Schedule Virtual Synchrony • The Ensemble system – Introduction V 0 ={a,b,c} V 1 ={a,b,c,d} V 2 ={b,c,d} V 3 ={b,c,d,e} – Architecture and Protocols a crash – How does Ensemble achieve the group b communication properties ? c • The Bulletin Board System d d want to join e e want to join Lab 1 - Bulletin Board System 23 Lab 1 - Bulletin Board System 24

Ensembl ed The Bulletin Board System • Bulletin board with messages • An ensembled process needs to run at each – Subject, ID, sender, time computer – Arriving messages shall be displayed immediately – If none is running at your computer run – No agreed order of messages is needed /chalmers/users/larandr/ensemble/ensembled – But: Replies should always be after their parent • Take advantage of ensemble to do this – Already runs on: • Peer to peer application remote{1,2,3,4,5}.student.chalmers.se – The bulletin board is shared • Ensembled is providing the ensemble service – No server that keeps the messages – Ensemble servers are not centralized servers – Stability while endpoints join and leaves • The application should be able to stand the loss of any – The server serves one host client – The servers connect to each other and form the – Except last one network – Warn client when it is the only one left Lab 1 - Bulletin Board System 25 Lab 1 - Bulletin Board System 26 Get to know the system • Look at the documentation: – Ensemble tutorial • Chapter 8 for the java interface • 5.8 for quick view of properties/layers – Ensemble reference manual • Chapter 11 for more details on the layers • For additional information – client/java/ensemble/JoinOps.java • Under /chalmers/users/larandr/ensemble/ • To see how to select layers • Understand the example program – Get the Talk app from the course page – Get it to run and figure out how it works Lab 1 - Bulletin Board System 27