Distributed Systems anycast 1 nearest 1 of several identical nodes - PDF document

Modes of communication • unicast – 1 1 – Point-to-point Distributed Systems • anycast – 1 nearest 1 of several identical nodes – Introduced with IPv6; used with BGP • netcast Group Communication – 1 many, 1 at a time • multicast – 1 many Paul Krzyzanowski – group communication pxk@cs.rutgers.edu • broadcast – 1 all Except as otherwise noted, the content of this presentation is licensed under the Creative Commons Attribution 2.5 License. Page 1 Page 1 Page 2 Groups Design Issues Groups are dynamic • Closed vs. Open – Created and destroyed – Closed: only group members can sent messages – Processes can join or leave • Peer vs. Hierarchical • May belong to 0 or more groups – Peer: each member communicates with group Send message to one entity – Hierarchical: go through coordinator – Deliver to entire group • Managing membership – Distributed vs. centralized Deal with collection of processes as one • Leaving & joining must be synchronous abstraction • Fault tolerance? Page 3 Page 4 Hardware multicast Hardware support for multicast – Group members listen on network address Implementing Group Communication send addr = a 1 Mechanisms listen addr=a 1 listen addr=a 1 listen addr=a 1 Page 5 Page 5 Page 6 1

Hardware broadcast Software: netcast Hardware support for broadcast Multiple unicasts ( netcast ) – Software filters multicast address – Sender knows group members • May be auxiliary address discard id= m broadcast(id= m ) accept id=m listen local addr=a2 send(a3) accept id=m listen local addr=a3 discard id= m listen local addr=a5 accept id=m Page 7 Page 8 Software Multiple unicasts via group coordinator – coordinator knows group members Reliability of multicasts coordinator listen local addr send(a3) listen local addr send(c) listen local addr Page 9 Page 10 Page 10 Atomic multicast Achieving atomicity (2-phase commit variation) Atomicity Retry through network failures & system downtime Sender and receivers maintain persistent log Message sent to a group arrives at all group members 1. Send message to all group members • If it fails to arrive at any member, no member will process it. • Each receiver acknowledges message • Saves message and acknowledgement in log Problems • Does not pass message to application 2. Sender waits for all acknowledgements Unreliable network • Retransmits message to non-responding members • Each message should be acknowledged – Again and again… until response received • Acknowledgements can be lost 3. Sender sends “go” message to all members Message sender might die • Each recipient passes message to application • Sends reply to server Page 11 Page 12 2

Achieving atomicity Reliable multicast Best effort All members will eventually get the message – Assume sender will remain alive – Retransmit undelivered messages Phase 1: • Send message – Make sure that everyone gets the message • Wait for acknowledgement from each group member Phase 2: • Retransmit to non-responding members – Once everyone has confirmed receipt, let the application see it Page 13 Page 14 Unreliable multicast • Basic multicast • Hope it gets there Message ordering Page 15 Page 16 Page 16 Good Ordering Bad Ordering order received order received message a message a a, b a, b Process 0 Process 0 message b message b a, b b, a Page 17 Page 18 3

Good Ordering Bad Ordering order received order received message a message a a, b a, b Process 0 Process 0 message b message b a, b b, a Process 1 Process 1 Page 19 Page 20 Sending versus Delivering Sending, delivering, holding back • Multicast receiver algorithm decides when to deliver a message to the process. sender receiver • A received message may be: – Delivered immediately delivering (put on a delivery queue that the process reads) Multicast sending – Placed on a hold-back queue algorithm (because we need to wait for an earlier message) – Rejected/discarded delivery ? queue (duplicate or earlier message that we no longer want) sending hold-back queue discard Multicast receiving algorithm Page 21 Page 22 Total ordering Global time ordering • Consistent ordering everywhere • All messages arrive in exact order sent • All messages arrive at all group members in the same • Assumes two events never happen at the order exact same time! 1. If a process sends m before m ’ then any other process that delivers m ’ will have delivered m . • Difficult (impossible) to achieve 2. If a process delivers m ’ before m ” then every other process will have delivered m ’ before m ” . • Implementation: – Attach unique totally sequenced message ID – Receiver delivers a message to the application only if it has received all messages with a smaller ID Page 23 Page 24 4

Causal ordering Sync ordering • Partial ordering • Messages can arrive in any order – Messages sequenced by Lamport or Vector • Special message type timestamps – Synchronization primitive If multicast(G, m) -> multicast(G, m ’) – Ensure all pending messages are delivered before then every process that delivers m ’ will any additional (post-sync) messages are accepted have delivered m • Implementation – Deliver messages in timestamp order per-source. Page 25 Page 26 FIFO ordering Unordered multicast • Messages can be delivered in different order • Messages can be delivered in different order to different members to different members • Message m must be delivered before message • Order per-source does not matter. m’ iff m was sent before m’ from the same host If a process issues a multicast of m followed by m’, then every process that delivers m’ will have already delivered m. Page 27 Page 28 Multicasting considerations atomic Reliability reliable IP Multicasting unreliable Message Ordering Page 29 Page 30 Page 30 5

IP Broadcasting IP Multicasting • 255.255.255.255 Class D network created for IP multicasting – Limited broadcast: send to all connected networks 1110 28-bit multicast address • Host bits all 1 (128.6.255.255, 192.168.0.255) – Directed broadcast on subnet 224.0.0.0/4 224.0.0.0 – 239.255.255.255 Host group – Set of machines listening to a particular multicast address Page 31 Page 32 IP multicasting IP multicast addresses • Can span multiple physical networks • Addresses chosen arbitrarily • Well-known addresses assigned by IANA • Dynamic membership – Internet Assigned Numbers Authority – Machine can join or leave at any time – RFC 1340 – Similar to ports – service-based allocation • No restriction on number of hosts in a group • FTP: port 21, SMTP: port 25, HTTP: port 80 • Machine does not need to be a member to send messages 224.0.0.1: all systems on this subnet 224.0.0.2: all multicast routers on subnet 224.0.1.16: music service 224.0.1.2: SGI’s dogfight 224.0.1.7: Audionews service Page 33 Page 34 LAN (Ethernet) multicasting LAN (Ethernet) multicasting example LAN cards support multicast in one (or both) of Intel 82546EB Dual Port Gigabit Ethernet two ways: Controller 10/100/1000 BaseT Ethernet – Packets filtered based on hash(mcast addr) • Some unwanted packets may pass through • Simplified circuitry Supports: - 16 exact MAC address matches – Exact match on small number of addresses • If host needs more, put LAN card in multicast - 4096-bit hash filter for multicast frames promiscuous mode - promiscuous unicast & promiscuous multicast – Receive all hardware multicast packets transfer modes Device driver must check to see if the packet was really needed Page 35 Page 36 6

IP multicast on a LAN IP multicast on a LAN Joining a multicast group • Sender specifies class D address in packet Receiving process: • Driver must translate 28-bit IP multicast group to – Notifies IP layer that it wants to receive multicast Ethernet address datagrams addressed to a certain host group – IANA allocated range of Ethernet MAC addresses for multicast – Device driver must enable reception of Ethernet – Copy least significant 23 bits of IP address to packets for that IP address MAC address • Then filter exact packets Bottom 23 bits • 01:00:5e: xx : xx : xx of IP address • Send out multicast Ethernet packet – Contains multicast IP packet Page 37 Page 38 Beyond the physical network IGMP v1 Packets pass through routers which bridge • Datagram-based protocol networks together • Fixed-size messages: – 20 bytes header, 8 bytes data Multicast-aware router needs to know: • 4-bit version – are any hosts on a LAN that belong to a multicast • 4-bit operation (1=query by router, 2=response) group? • 16-bit checksum • 32-bit IP class D address IGMP: – Internet Group Management Protocol – Designed to answer this question – RFC 1112 (v1) , 2236 (v2) , 3376 (v3) Page 39 Page 40 Leaving a multicast group with IGMP Joining multicast group with IGMP • Machine sends IGMP report: – “I’m interested in this multicast address” • No response to an IGMP query – Machine has no more processes which are interested • Each multicast router broadcasts IGMP • Eventually router will stop forwarding packets queries at regular intervals to network when it gets no IGMP responses – See if any machines are still interested – One query per network interface • When machine receives query – Send IGMP response packet for each group for which it is still interested in receiving packets Page 41 Page 42 7