Approche Algorithmique des Syst` emes R epartis (AASR) Guillaume - PowerPoint PPT Presentation

Approche Algorithmique des Syst` emes R´ epartis (AASR) Guillaume Pierre guillaume.pierre@irisa.fr D’apr` es un jeu de transparents de Maarten van Steen VU Amsterdam, Dept. Computer Science 06a: Synchronization (1/2)

Contents Chapter 01: Introduction 02: Architectures 03: Processes 04: Communication (1/2) 04: Communication (2/2) 05: Naming (1/2) 05: Naming (2/2) 06: Synchronization (1/2) 06: Synchronization (2/2) 07: Consistency & Replication 08: Fault Tolerance 09: Security 2 / 41

Clock Synchronization Physical clocks Logical clocks Vector clocks 3 / 41

Physical clocks 4 / 41

Physical clocks Problem Sometimes we simply need the exact time, not just an ordering. Solution Universal Coordinated Time (UTC): Based on the number of transitions per second of the cesium 133 atom (pretty accurate). At present, the real time is taken as the average of some 50 cesium-clocks around the world. Introduces a leap second from time to time to compensate that days are getting longer. Note UTC is broadcast through short wave radio and satellite. Satellites can give an accuracy of about ± 0 . 5 ms. 5 / 41

Physical clocks Problem Suppose we have a distributed system with a UTC-receiver somewhere in it ⇒ we still have to distribute its time to each machine. Basic principle Every machine has a timer that generates an interrupt H times per second. There is a clock in machine p that ticks on each timer interrupt. Denote the value of that clock by C p ( t ) , where t is UTC time. Ideally, we have that for each machine p , C p ( t ) = t , or, in other words, dC / dt = 1. 6 / 41

Physical clocks dC dt > 1 dC Clock time, C dt = 1 k Fast clock c o l c t c dC e dt < 1 f r k e c P o c l w o S l UTC, t In practice: 1 − ρ ≤ dC dt ≤ 1 + ρ . Goal Never let two clocks in any system differ by more than δ time units ⇒ synchronize at least every δ / ( 2 ρ ) seconds. 7 / 41

Global positioning system Basic idea You can get an accurate account of time as a side-effect of GPS. Height r = 25.9 (4.5,28.5) (17.8,17.8) r = 19 (-7.6,7.6) x r = 11.4 8 / 41

Global positioning system Problem Assuming that the clocks of the satellites are accurate and synchronized: It takes a while before a signal reaches the receiver The receiver’s clock is definitely out of synch with the satellite 9 / 41

Global positioning system Principal operation ∆ r : unknown deviation of the receiver’s clock. x r , y r , z r : unknown coordinates of the receiver. T i : timestamp on a message from satellite i ∆ i = ( T now − T i )+∆ r : measured delay of the message sent by satellite i . Measured distance to satellite i : c × ∆ i ( c is speed of light) Real distance is: � ( x i − x r ) 2 +( y i − y r ) 2 +( z i − z r ) 2 d i = c ∆ i − c ∆ r = Observation 4 satellites ⇒ 4 equations in 4 unknowns (with ∆ r as one of them). Accuracy ∼ 20 – 40 ns 10 / 41

Clock synchronization principles Principle I Every machine asks a time server for the accurate time at least once every δ / ( 2 ρ ) seconds ( Network Time Protocol ). Note Okay, but you need an accurate measure of round trip delay, including interrupt handling and processing incoming messages. 11 / 41

Clock synchronization principles Principle II Let the time server scan all machines periodically, calculate an average, and inform each machine how it should adjust its time relative to its present time. Note Okay, you’ll probably get every machine in sync. You don’t even need to propagate UTC time. Fundamental You’ll have to take into account that setting the time back is never allowed ⇒ smooth adjustments. 12 / 41

Logical clocks 13 / 41

Leslie Lamport Logical clocks Vector clocks Paxos LaTeX . . . 14 / 41

The Happened-before relationship Problem We first need to introduce a notion of ordering before we can order anything. The happened-before relation If a and b are two events in the same process, and a comes before b , then a → b . If a is the sending of a message, and b is the receipt of that message, then a → b If a → b and b → c , then a → c Note This introduces a partial ordering of events in a system with concurrently operating processes. 15 / 41

Logical clocks Problem How do we maintain a global view on the system’s behavior that is consistent with the happened-before relation? P P P 1 2 3 0 0 0 6 m 1 8 10 12 16 20 18 24 m 2 30 24 32 40 30 40 50 36 48 60 m 3 42 56 70 48 64 80 m 4 54 72 90 60 80 100 16 / 41

Logical clocks Solution Attach a timestamp C ( e ) to each event e , satisfying the following properties: P1 If a and b are two events in the same process, and a → b , then we demand that C ( a ) < C ( b ) . P2 If a corresponds to sending a message m , and b to the receipt of that message, then also C ( a ) < C ( b ) . Problem How to attach a timestamp to an event when there’s no global clock ⇒ maintain a consistent set of logical clocks, one per process. 17 / 41

Logical clocks Solution Each process P i maintains a local counter C i and adjusts this counter according to the following rules: 1: For any two successive events that take place within P i , C i is incremented by 1. 2: Each time a message m is sent by process P i , the message receives a timestamp ts ( m ) = C i . 3: Whenever a message m is received by a process P j , P j adjusts its local counter C j to max { C j , ts ( m ) } ; then executes step 1 before passing m to the application. Notes Property P1 is satisfied by (1); Property P2 by (2) and (3). It can still occur that two events happen at the same time. Avoid this by breaking ties through process IDs. 18 / 41

Logical clocks – example P P P 1 2 3 0 0 0 6 m 1 8 10 12 16 20 18 24 m 2 30 24 32 40 P adjusts 30 40 50 2 its clock 36 48 60 m 3 42 61 70 48 69 80 m 4 70 77 90 76 85 100 P adjusts 1 its clock 19 / 41

Logical clocks – example Note Adjustments take place in the middleware layer Application layer Application sends message Message is delivered to application Adjust local clock� Adjust local clock Middleware layer and timestamp message Middleware sends message Message is received Network layer 20 / 41

Example: Totally ordered multicast Problem We sometimes need to guarantee that concurrent updates on a replicated database are seen in the same order everywhere: P 1 adds $100 to an account (initial value: $1000) P 2 increments account by 1% There are two replicas Update 1 Update 2 Replicated database Update 1 is Update 2 is performed before performed before update 2 update 1 Result In absence of proper synchronization: replica #1 ← $1111, while replica #2 ← $1110. 21 / 41

Example: Totally ordered multicast Solution Process P i sends timestamped message msg i to all others. The message itself is put in a local queue queue i . Any incoming message at P j is queued in queue j , according to its timestamp, and acknowledged to every other process. P j passes a message msg i to its application if: (1) msg i is at the head of queue j (2) for each process P k , there is a message msg k in queue j with a larger timestamp. Note We are assuming that communication is reliable and FIFO ordered. 22 / 41

Vector clocks 23 / 41

Vector clocks Lamport’s clocks are not perfect They guarantee that if a → b then C ( a ) < C ( b ) . They do not guarantee that if C ( a ) < C ( b ) then a → b . P 1 P 2 P 3 0� 0� 0� 6� m 1 8� 10� m 2 12� 16� 20� Observation 18� 24� 30� Event a : m 1 is received at T = 16; m 3 24� 32� 40� 30� 40� 50� Event b : m 2 is sent at T = 20. 36� 48� 60� 42� 61� 70� m 4 48� 69� 80� m 5 70� 77� 90� 76 85 100 Note We cannot conclude that a causally precedes b . 24 / 41

Vector clocks Solution Each process P i has an array VC i [ 1 .. n ] , where VC i [ j ] denotes the number of events that process P i knows have taken place at process P j . When P i sends a message m , it adds 1 to VC i [ i ] , and sends VC i along with m as vector timestamp vt ( m ) . Result: upon arrival, recipient knows P i ’s timestamp. When a process P j delivers a message m that it received from P i with vector timestamp ts ( m ) , it (1) updates each VC j [ k ] to max { VC j [ k ] , ts ( m )[ k ] } (2) increments VC j [ j ] by 1. Question What does VC i [ j ] = k mean in terms of messages sent and received? 25 / 41

Causally ordered multicasting Observation We can now ensure that a message is delivered only if all causally preceding messages have already been delivered. Adjustment P i increments VC i [ i ] only when sending a message, and P j “adjusts” VC j when receiving a message (i.e., effectively does not change VC j [ j ] ). P j postpones delivery of m until: ts ( m )[ i ] = VC j [ i ]+ 1. ts ( m )[ k ] ≤ VC j [ k ] for k � = i . 26 / 41

Causally ordered multicasting Example VC = (1,0,0) VC = (1,1,0) 0 0 P 0 m P 1 m* VC = (1,1,0) VC = (1,1,0) 1 2 P 2 � VC = (0,0,0) VC = (1,0,0) 2 2 Example Take VC 2 = [ 0 , 2 , 2 ] , ts ( m ) = [ 1 , 3 , 0 ] from P 0 . What information does P 2 have, and what will it do when receiving m (from P 0 )? 27 / 41