C AUSALITY Prasun Dewan Department of Computer Science University - PowerPoint PPT Presentation

C AUSALITY Prasun Dewan Department of Computer Science University of North Carolina at Chapel Hill dewan@cs.unc.edu

M ESSAGE O RDERING  Assume messages received reliably but not necessarily in order  Communication is direct (P2P) 2

M ULTICAST Same Message directed multiple processes User Process Receiver of multicast message can also multicast Consistency ? Consistency in 2- computer case? User Process User Process communicator.toOthers( new ARemoteInput(theNextInput)); 3

U NICAST Message directed to a User Process single process Message User Process communicator.toUser (“ alice ”, new AFloorControlRequest()); 4

D ECOUPLING R ELIABILITY AND O RDER Sliding window ensures in- order processing and reliable delivery User Process Assume reliable delivery Message How in-order processing? Not practical to decouple, but will help us draw out principles for the N-Computer, multicast case, where reliability assumed User Process communicator.toUser (“ alice ”, new AFloorControlRequest()); 5

O UT OF O RDER U NICAST PC PC 1 2 Divergent state! Sequence numbers! Done? Lunch? Lunch? Done? Yes, No Yes, No 6

O UT OF O RDER U NICAST (R EVIEW ) PC PC 1 2 Divergent state! Sequence numbers! Done? Lunch? Lunch? Done? Yes, No Yes, No 7

U NICAST S EQUENCE N UMBERS Each computer pair keeps count Local 0 Local 0 1 2 of #messages sent to other party Remote 0 Remote 0 2 1 Send message: increment and PC PC attach local count as time stamp 1 2 Each computer keeps last processed remote # and ordered buffer for other party When message received, put message in ordered buffer 1. If buffer empty or message# !=successor ( remote#) return Done? Done? 2. Remove message from Lunch? Lunch? buffer, process it PC2 Buffer 3 remote# = message# Done? 1 4. Go to 1 Lunch? 2 8

N-U SER U NICAST Local 2 0 Local 3 5 Local 1 1 Local 3 3 Local 1 0 Local 2 2 Remote 2 1 Remote 3 0 Remote 1 0 Remote 3 2 Remote 1 5 Remote 2 3 PC PC PC 1 2 3 Buffer 2 Buffer 3 Buffer 1 Buffer 3 Buffer 1 Buffer 2 Supports pairwise connections (IMs) Group IM? 9

O UT O F O RDER M ULTICAST M 1 causes (<) M 2 Causal relations if M 2 sent from site S after M 1 PC PC PC 1 2 3 received or generated at Site S Causal multicast: At all sites, if M 1 < M 2 , M 2 should be processed after Yes, No M 1 Done? Done? Lunch? How to detect and Lunch? Lunch? Done? ensure causal broadcast? Yes, No Yes, No Yes, No Correct Order? 10

R EAL -T IME S CALAR S TAMP M 2 caused by M 1  RTS (M 2 ) > RTS (M 1 ) PC PC PC 1 2 3 Process received messages in order of RTS Not RTS (M 2 ) > RTS (M 1 )  M 2 caused by M 1 Done? Done? Lunch? Do not know if there Lunch? Lunch? Done? is an in-transit Yes, No Yes, No Yes, No previous message, how long to wait? Clocks at different sites not 11 synchronized!

M ESSAGE H ISTORY S TAMP Stamp: Global Local 0 1 Local 0 1 Local 0 1 message id + history of ids of PC PC PC 1 2 3 sent/received msgs Global id: unique site it + sequence number Yes, No, (3,1), {(1,1), (2,1)} History can get large and compression needed Done? Done? Lunch? Lunch? Lunch? Done? Simpler scheme Yes, No Yes, No Yes, No possible if message not multicast to arbitrary user set 12

M ESSAGE H ISTORY S TAMP (R EVIEW ) Stamp: Global Local 0 1 Local 0 1 Local 0 1 message id + history of ids of PC PC PC 1 2 3 sent/received msgs Global id: unique site it + sequence number Yes, No, (3,1), {(1,1), (2,1)} History can get large and compression needed Done? Done? Lunch? Lunch? Lunch? Done? Simpler scheme Yes, No Yes, No Yes, No possible if message not multicast to arbitrary user set 13

G LOBAL S CALAR I D : L OGICAL C LOCK , A SSUMING A LL M ESSAGES B ROADCAST Every site keeps a global id initialized to 0 1 0 When a site generates a message it PC increments id and time stamps message 2 2 0 1 with it PC 1 0 A site delivers a message if its global id is 3 the successor of current global id; 2 0 1 PC otherwise it buffers the message to be 1 PC delivered later 4 On delivering/processing a received message, a site sets its global id to the message id If no concurrent messages ever occur, It should allow detection of concurrent this scheme should work messages as soon as they arrive Causal broadcast does not indicate what should happen with concurrent It should not deliver a message before its messages – immediate delivery, (fatal) cause 14 error

L AST S ENDER T IME S TAMP , A SSUMING A LL M ESSAGES B ROADCAST Every site keeps a receive count for 0 0 0 0 each site PC 2 When a site generates a message it 0 1 0 0 0 sends the sender and count of the last PC message it received 0 0 0 0 3 PC A site delivers a message if the 1 PC received count for the site is the same 4 as its count for that site; otherwise it buffers the message for later delivery 0 1 0 0 0 On delivering/processing a received message, a site increments local count for that site 15

L AST S ENDER T IME S TAMP : M ULTIPLE C AUSES 0 0 0 0 Every site keeps a receive count for PC 2 each site When a site generates a message it PC sends the sender and count of the last 1 1 0 0 1 0 0 message it received 0 0 0 0 PC 3 A site delivers a message if the received count for the site is the same as its count for that site; otherwise it buffers the message for later delivery PC On delivering/processing a received 4 message, a site increments local count 1 0 0 0 1 0 for that site A message may have multiple causes, and this scheme sends only the most recent cause 16

F ROM H ISTORY TO V ECTOR T IMESTAMPS Local 0 1 Local 0 1 Local 0 1 Assume: each message broadcast to PC PC PC all other users in an 1 2 3 app session IM and many other apps follow this assumption Yes, No, (3,1), {(1,1), (2,1)} Counts of sent/received messages replace Done? Done? Lunch? history Lunch? Lunch? Done? Yes, No Yes, No Yes, No 17

E XTENSION OF T WO -U SER U NICAST Number for each Local 0 Local 0 2 1 user: vector timestamp Remote 0 1 2 0 Remote 0 0 PC PC Message has vector 1 2 time stamp Ordered buffer for messages arriving early Need: < and == for sorting buffer increment Done? Done? operation before Lunch? Lunch? sending message and after receiving PC2 Buffer message Done? 1 succ function for picking next Lunch? 2 18 received msg

V ECTOR T IME S TAMPS Site S j has broadcast x i messages to v = (x 1 , .. x n ) at Site S j   other sites and for all 1≤ i ≤ n, i != j Site S j has received x i messages from Site S i v 1 = (a 1 , .. a n ) ==  for all 1≤ i ≤ n, a i == b i v 2 = (b 1 , .. b n ) v 1 = (a 1 , .. a n ) for all 1≤ i ≤ n, a i ≤ b i <  for some 1 ≤ i ≤ n, a i < b i v 2 = (b 1 , .. b n ) Vector time stamps do not create total order Possible that a i < b i and a j > b j for some 1 ≤ i , j ≤ n  concurrent message, v 1 ||v 2 Causal broadcast does not impose order on concurrent messages For causal broadcast, will assume no concurrent messages are generated. 19

E XTENSION OF T WO -U SER U NICAST Number for each Local 0 Local 0 2 1 user: vector timestamp Remote 0 1 2 0 Remote 0 0 PC PC Message has vector 1 2 time stamp Ordered buffer for messages arriving early Need: < and == for sorting buffer increment Done? Done? operation before Lunch? Lunch? sending message and after receiving PC2 Buffer message Done? 1 succ function for picking next Lunch? 2 20 received msg

I NCREMENT AND S UCCESSOR v 1 = (a 1 , .. a n ) There exists 1 ≤ i ≤ n, a i == 1 + b i is a successor of  for all j != i, a j == b j v 2 =(b 1 , .. b n ) v = (a 1 , .., a i + 1, … a n ) inc( i, v = (a 1 , .., a i , .., a n ))  A message has multiple successors Inc with respect to a site 21

U NICAST VS . M ULITCAST Each pair of communicating computers Each site i keeps a local vector time keeps a count of how many messages it stamp, v i = (i 1 , .. i n ) has sent to other party and next expected remote# for other party Send message: increment i i and attach Send message: attach and increment local count vector time stamp Each site keeps ordered buffer for other Each site i keeps ordered buffer i for all party parties When message received, put message in When message received from site i, put ordered buffer message in ordered buffer 1. If buffer empty or message TS != 1. If buffer empty or message# successor (local TS) return !=successor (remote#) return 2. Remove message from buffer, 2. Remove message from buffer, process it process it 3. Local TS i  message TS i 3 remote#  message# 22 4. Go to 1 4. Go to 1

C AUSAL M ULTICAST Send message: v 3 v 1 v 2 increment i i and attach vector time 0 1 0 0 0 0 1 1 0 1 0 0 0 0 0 0 1 1 0 0 1 0 0 0 stamp PC PC PC 1 2 3 Received message is put in ordered buffer 1. If buffer empty or message TS !=successor ( local TS) return 2. Remove message Done? Done? Done? from buffer and Lunch? Lunch? process it 3 Local TS i  msg Buffer 1 Buffer 2 Buffer 3 TS i Done? 1 0 0 Done? 1 0 0 4. Go to 1 Lunch? 1 1 0 23