Distributed Systems Synchronization between senders and receivers - PDF document

What’s it for? • Temporal ordering of events produced by concurrent processes Distributed Systems • Synchronization between senders and receivers of messages • Coordination of joint activity Clock Synchronization: Physical Clocks • Serialization of concurrent access for shared objects Paul Krzyzanowski pxk@cs.rutgers.edu Except as otherwise noted, the content of this presentation is licensed under the Creative Commons Attribution 2.5 License. Page 1 Page 1 Page 2 Logical vs. physical clocks Logical clock keeps track of event ordering – among related (causal) events Physical clocks Physical clocks keep time of day – Consistent across systems Page 3 Page 3 Page 4 Quartz clocks Atomic clocks • 1880: Piezoelectric effect • Second is defined as 9,192,631,770 periods of – Curie brothers radiation corresponding to the transition – Squeeze a quartz crystal & it generates an electric field between two hyperfine levels of cesium-133 – Apply an electric field and it bends • 1929: Quartz crystal clock • Accuracy: – Resonator shaped like tuning fork better than 1 second in six million years – Laser-trimmed to vibrate at 32,768 Hz – Standard resonators accurate to 6 parts per million at 31 ° C – Watch will gain/lose < ½ sec/day – Stability > accuracy: stable to 2 sec/month • NIST standard since 1960 – Good resonator can have accuracy of 1 second in 10 years • Frequency changes with age, temperature, and acceleration Page 5 Page 6 1

UTC UTC • UT0 • Coordinated Universal Time – Mean solar time on Greenwich meridian • Temps Universel Coordonné – Obtained from astronomical observation • UT1 – Kept within 0.9 seconds of UT1 – UT0 corrected for polar motion – Atomic clocks cannot keep mean time • UT2 • Mean time is a measure of Earth’s rotation – UT1 corrected for seasonal variations in Earth’s rotation • UTC – Civil time measured on an atomic time scale Page 7 Page 8 Physical clocks in computers Problem Getting two systems to agree on time Real-time Clock: CMOS clock (counter) circuit – Two clocks hardly ever agree driven by a quartz oscillator – Quartz oscillators oscillate at slightly different – battery backup to continue measuring time when frequencies power is off Clocks tick at different rates OS generally programs a timer circuit to – Create ever-widening gap in perceived time generate an interrupt periodically – Clock Drift – e . g., 60, 100, 250, 1000 interrupts per second Difference between two clocks at one point in time (Linux 2.6+ adjustable up to 1000 Hz) – Programmable Interval Timer (PIT) – Intel 8253, 8254 – Clock Skew – Interrupt service procedure adds 1 to a counter in memory Page 9 Page 10 8:00:00 8:00:00 8:01:24 8:01:48 Skew = +84 seconds Skew = +108 seconds Sept 18, 2006 Oct 23, 2006 +84 seconds/35 days +108 seconds/35 days 8:00:00 8:00:00 Drift = +2.4 sec/day Drift = +3.1 sec/day Page 11 Page 12 2

Perfect clock Drift with slow clock Computer’s time, C Computer’s time, C skew UTC time, t UTC time, t Page 13 Page 14 Drift with fast clock Dealing with drift Assume we set computer to true time Computer’s time, C Not good idea to set clock back skew – Illusion of time moving backwards can confuse message ordering and software development environments UTC time, t Page 15 Page 16 Dealing with drift Dealing with drift Go for gradual clock correction OS can do this: Change rate at which it requests interrupts If fast: e.g.: if system requests interrupts every Make clock run slower until it synchronizes 17 msec but clock is too slow: request interrupts at (e.g.) 15 msec If slow: Or software correction: redefine the interval Make clock run faster until it synchronizes Adjustment changes slope of system time: Linear compensating function Page 17 Page 18 3

Compensating for a fast clock Compensating for a fast clock Computer’s time, C Computer’s time, C Clock synchronized skew Linear compensating function applied UTC time, t UTC time, t Page 19 Page 20 Resynchronizing Getting accurate time After synchronization period is reached • Attach GPS receiver to each computer – Resynchronize periodically ± 1 msec of UTC – Successive application of a second linear • Attach WWV radio receiver compensating function can bring us closer to true Obtain time broadcasts from Boulder or DC slope ± 3 msec of UTC (depending on distance) • Attach GOES receiver Keep track of adjustments and apply ± 0.1 msec of UTC continuously – e.g., U NIX adjtime system call Not practical solution for every machine – Cost, size, convenience, environment Page 21 Page 22 Getting accurate time RPC Synchronize from another machine Simplest synchronization technique – One with a more accurate clock – Issue RPC to obtain time – Set time Machine/service that provides time information: what’s the time? client server Time server 3:42:19 Does not account for network or processing latency Page 23 Page 24 4

Cristian’s algorithm Cristian’s algorithm Compensate for delays Client sets time to: T server – Note times: server • request sent: T 0 request reply • reply received: T 1 client – Assume network delays are symmetric T 0 T 1 time T server = estimated overhead server in each direction request reply client T 0 T 1 time Page 25 Page 26 Error bounds Error bounds T server If minimum message transit time (T min ) is known: server request reply Place bounds on accuracy of result client T 0 T 1 time T min T min Earliest time Latest time message arrives message leaves range = T 1 -T 0 -2T min accuracy of result = Page 27 Page 28 Cristian’s algorithm: example Cristian’s algorithm: example • Send request at 5:08:15.100 ( T 0 ) If best-case message time=200 msec T 0 = 5:08:15.100 T 1 = 5:08:15.900 • Receive response at 5:08:15.900 ( T 1 ) T s = 5:09:25:300 T server T min = 200msec – Response contains 5:09:25.300 ( T server ) server request reply • Elapsed time is T 1 - T 0 client T 0 T 1 time 5:08:15.900 - 5:08:15.100 = 800 msec 200 200 • Best guess: timestamp was generated 400 msec ago 800 • Set time to T server + elapsed time 5:09:25.300 + 400 = 5:09.25.700 Error = Page 29 Page 30 5

Berkeley Algorithm Berkeley Algorithm • Gusella & Zatti, 1989 • Machines run time dæmon – Process that implements protocol • One machine is elected (or designated) as the • Assumes no machine has an accurate time server ( master ) source – Others are slaves • Obtains average from participating computers • Synchronizes all clocks to average Page 31 Page 32 Berkeley Algorithm Berkeley Algorithm • Master polls each machine periodically Algorithm has provisions for ignoring readings – Ask each machine for time from clocks whose skew is too great • Can use Cristian’s algorithm to compensate for network – Compute a fault-tolerant average latency • When results are in, compute average If master fails – Including master’s time • Hope: average cancels out individual clock’s – Any slave can take over tendencies to run fast or slow • Send offset by which each clock needs adjustment to each slave – Avoids problems with network delays if we send a time stamp Page 33 Page 34 Berkeley Algorithm: example Berkeley Algorithm: example 3:00 3:00 2:50 2:50 3:25 2:50 9:10 3:25 2:50 9:10 1. Request timestamps from all slaves 2. Compute fault-tolerant average: Page 35 Page 36 6

Berkeley Algorithm: example Network Time Protocol, NTP 1991, 1992 +0.15 3:00 Internet Standard, version 3: RFC 1305 +0:15 3:25 2:50 9:10 3. Send offset to each client Page 37 Page 38 NTP Goals NTP servers • Enable clients across Internet to be accurately Arranged in strata synchronized to UTC despite message delays – 1 st stratum: machines 1 – Use statistical techniques to filter data and gauge quality of connected directly to results 2 accurate time source • Provide reliable service – 2 nd stratum: machines 3 – Survive lengthy losses of connectivity synchronized from 1 st – Redundant paths stratum machines 4 – Redundant servers – … • Enable clients to synchronize frequently – offset effects of clock drift • Provide protection against interference S YNCHRONIZATION S UBNET – Authenticate source of data Page 39 Page 40 NTP Synchronization Modes NTP messages • Procedure call and symmetric mode Multicast mode – Messages exchanged in pairs – for high speed LANS • NTP calculates: – Lower accuracy but efficient – Offset for each pair of messages Procedure call mode • Estimate of offset between two clocks – Similar to Cristian’s algorithm – Delay Symmetric mode • Transmit time between two messages – Filter Dispersion – Intended for master servers • Estimate of error – quality of results – Pair of servers exchange messages and retain data • Based on accuracy of server’s clock and consistency of to improve synchronization over time network transit time • Use this data to find preferred server: – lower stratum & lowest total dispersion All messages delivered unreliably with UDP Page 41 Page 42 7