Issues with Clocks Context The tree correction protocol was based - - PowerPoint PPT Presentation

Issues with Clocks

Context

• The tree correction protocol was based
• n the idea of local detection and

correction.

• Protocols of this type are complex to

build

• We will study some of the classic

protocols that help solve different problems

Models

• We reviewed the idea of different

models

– Higher level models make it easier to write the programs – Lower level models are easy to implement

Strongest vs Weakest Model

• What is strongest model?

– No private memory – Can update all variables at once – Essentially a centralized program

• What is the weakest model

– Process speed arbitrary – No bounds on delays – No global time

• Which should we study first?

Asynchronous Model

• Fairly weak model

– No global clock – No shared memory – Arbitrary process speed – Arbitrary message delays – We will assume messages are eventually delivered but after arbitrary delay

Problems in Asynchronous Systems

• We will study problems that we solve

easily in centralized systems

– Examples:

• Clocks
• Ordering among events
• Checkpoints
• ..
• We will study solving these problems in

asynchronous systems

Use of Clocks

• Clocks allow us to order events
• In asynchronous systems, there is no

notion of time.

• What is the purpose of time

– To say: something happened at 5pm

• We cannot do this in asynchronous systems

– To say: x1 happened first then y1

• We can do this sometimes in asynchronous

systems

– E.g., the class began before it ended

Problem

• Lack of global time

– Need to compare different events in a distributed system

• In asynchronous systems, time cannot

be used to order events

– We need to develop an alternative to order events in such a system

Problem (Continued)

• Consider the problem of detecting insider

trading in a stock exchange

– Assume that no communication occurs outside the computer systems – Each entity is represented by a computer and they communicate among themselves – Whenever a process sends a message, it includes ALL the information it has learnt so far

• Lets not worry about the cost of implementing this

– Each entity is required to follow the protocol you choose

Problem (Continued)

• Consider two events

– An inside event, say e, that affects the company A – An event f, where an officer, X, of company A sells stock

• Question

– Is it possible that X is guilty of insider trading?

Situation 1

• Assume global time

– Event e occurred at 8am – Event f occurred at 10am (same day) – Answer: – Event e occurred at 10am – Event f occurred at 8am (same day) – Answer:

Approach

• What we need is a way to define

causality

– Can event e affect event f? – This is defined as the happened before relation – Please do not confuse it with the English meaning of this

A Realistic Variation

• Consider a experiment that you are

performing that includes

– Events e and event f – Event f is a safety violation – Can changing what happens in event e affect what happens in f?

• Important in debugging, checkpointing,

etc.

An Example

Consider the following code segment x = 2; GOTO L1 x = 3 L2 z = y /x L1 x = x - 2 GOTO L2

Another Problem from Debugging

• Consider the requirement:

– Valve must be open if pressure exceeds 10 – Event e : at time 8:00:00:valve = open – Event f : at time 8:00:01, pressure = 11 – Looks ok. – But is it possible that due to some race conditions (e.g., processor @e being slow) delays e and violates this constraint.

happened before

• Let a, b, c be events

– An event can be a local event, send event or a receive event

• Definition :
• a  b (read as a happened before b) iff either
• ne of the following condition is true

– a and b are events on the same process and a

• ccurred before b

– a is a send event and b is the corresponding receive event – there exists event c such that a  c and c  b

Revisiting the Previous Problems

• Consider two events

– An inside event, say e, that affects the company A – An event f, where an officer, X, of company A sells stock

• If e  f then X is guilty of insider trading

Revisiting the Previous Problems

• Parallel experiment
• If e  f then changing what happens in

e can affect what happens in f

Concurrent Events

• a || b iff (NOT (a  b)) & (NOT (b  a))

– Question?

• (a || b) & (b || c) => (a || c) ?

Using Event Diagrams to Understand Causality

Logical Clocks

• Goal of logical clock is

– Assign each event a timestamp – If e  f then the timestamp of e should be less than that of f. – To solve this, each process maintains a logical clock cl

• cl.j : clock value of process j
• cl.m : clock value of message m
• cl.a : clock value of event a

Program for Logical Timestamps

• Let a be a new event

– If a is a local event at j

• cl.j := cl.j + 1
• cl.a := cl.j

– If a is a send event at j

• cl.j := cl.j + 1
• cl.m := cl.j
• cl.a := cl.j

– If a is a receive of message m at j

• cl.j := max(cl.j, cl.m) + 1
• cl.a := cl.j

Properties of Logical Clocks

• If a  b then cl.a < cl.b

Proving Properties of Logical Clocks

• Theorem: At any time, the following statement

is true for all events (that have occurred in the system so far)

– ∀ x, y :: x  y ⇒ cl.x < cl.y, – where x and y range over all the events, processes (latest event on the process) and messages (corresponding send event)

• Proof by induction
• Base case

– No events created. Hence, trivially true – Show that whenever a new event is created

Invariant

• The predicate

– a  b ⇒ cl.a < cl.b – Is an invariant of the logical timestamp program.

Logical Timestamps

• The time associated with an event is a pair,

the clock and the process where the event

• ccurred.
• For event a at process j, the timestamp ts.a is

– ts.a = <cl.a, j>

Lexicographical comparison < x1, x2 > < <y1, y2> iff x1 < y1 ∨ ( (x1 = y1) ∧ (x2 < y2) )

Observation about Logical Clocks

• For any two distinct events a and b,

either

• ts.a < ts.b ∨ ts.b < ts.a
• The event timestamps form a total
• rder.

So What?

• Consider two events

– An inside event, say e, that affects the company A – An event f, where an officer, X, of company A sells stock

• Question

– Is it possible that X is guilty of insider trading?

• Case 1

– cl.e = 8, cl.f = 10 – Answer:

• Case 2

– cl.e = 10, cl.f = 8 – Answer:

Problem

• Logical timestamps provide a partial

information about causality.

• Extending logical timestamps for

complete knowledge of causality

Accurate Causality Information

• Desire

– a  b  clock.a < clock.b

• Can clock.a be an integer?

Accurate Causality Information

• To obtain accurate causality

information, clock must consist of several integers

– It turns out that accurate causality information requires clocks to consist of n values

Vector Clocks

• Each process maintains a vector vc

– vc.j is the clock maintained by j – vc.j is an array

• vc.j.k denotes the knowledge that j has about

the clock of k => vc.j.j denotes the clock of j

Updating Vector Clocks

• Let a be a new event

– If a is a send event at j

• vc.j.j++
• vc.m := vc.j
• vc.a := vc.j

– If a is a receive of message m at j

• vc.j := max(vc.j, vc.m)
• vc.j.j++
• vc.a := vc.j

Causality Checking with Vector Clocks

• vc.a < vc.b iff

– (Subject to requirement a  b iff vc.a < vc.b)

Causality Checking with Vector Clocks

• Suppose a is an event at process j and

b is an event at process k

– vc.a < vc.b iff

Size of Vector Clocks

• Our implementation requires clocks of

size n

– Can we do better?

Lower bound on size of VC

• Consider the network with n processes

– Consider the following communication

• Initial event on process j is aj
• Each process j sends a message to everyone

except (j-1) mod n

• Each process receives all messages intended

for it.

– All receives are after all sends

• Final event on process j is bj

Lower bound on VC (continued)

• Observations

– aj+1 || bj – For j≠k

• ak+1  bj

• Suppose in contradiction, there is a way

to implement vector clocks with k- vectors of reals, where k < n.

• aj+1 || bj

=> V(aj+1) and V(bj) are incomparable => V(aj+1) is larger than V(bj) in some coordinate Denote this value by h(i)

• h : {0,…,n-1} → {0,…,k-1}

• h cannot be a one-to-one function

– There exists j and k such that h(j) = j(k)

• Let this value be r
• Compare V(bj), V(aj+1), V(bj) and V(ak+1)

– Evaluate with respect to element r

Causal Delivery

• Problem requirement

– Given two messages m1 and m2 sent to the same process

• If send(m1)  send(m2)
• Then Delivery(m1)  Delivery(m2)

Solving Causal Broadcast

• Special case of causal delivery where

all messages are broadcast in nature

– Applications in checkpointing etc.

• Variables

– num.j.k = knowledge that j has about the number of messages sent by k

• num.j.j = number of messages sent by j

Solving Causal Broadcast

• Algorithm

– When j sends a message

• Num.j.j++

– When j receives a message.

• Buffer it until j is certain that causal delivery

constraints wont be violated

• Constraint j should check before delivering a

message from k

• Suppose j has a message m from k in its
• buffer. J can deliver it iff the following

condition is satisfied:

• Comparison between num.j.k & num.m.k

– Check if num.j.k = num.m.k – 1

• Comparison between num.j.l and num.m.l,

where l is any process other than k

– Check if num.j.l >= num.m.l

• If all these conditions are satisfied then

deliver m

• How do I update num upon delivery?

– Num.j.k++

Observations Regarding Vector Clocks

• Could we have used a `reasonably fine-

grained’ physical clock to obtain vc.j.j?

Augmented Time (Brief Overview)

• What if clocks are synchronized within some

epsilon and we are interested in ordering events in the system.

• For example, let epsilon = 10

– Can we say something about events e and f such that

• ph.e = 10, ph.f = 15
• ph.e = 10, ph.f = 1000

– Note that this was not possible without clock synchronization

Augmented Time

• Create timestamp with clocks

synchronized upto epsilon

– Need to order these events

• ph.e = 10, ph.f = 15

– No need to order these events

• ph.e = 10, ph.f = 1000

Idea

• Each process maintains at.j.k

– at.j.k = knowledge j has about clock of k

• How would this variable be updated?

– Upon send? – Upon receive?

Space requirement for augmented time

• Worst case
• Average case?
• Can we optimize?

Properties of Augmented Time

• If two events are close
• If two events are far off

Augmented Logical Time

Observations (Continued)

• What if this physical clock was accurate

within some time bound

– Notion of communication via time – This could potentially bound the size of vector clocks

Observations (Continued)

• Does the clock have to be a vector of

size n?

– In general yes – But under certain topologies, it can be reduced

Observations (Continued)

• Causality captures whether event e

can potentially affect event f

– Illustration with simple code below:

• 1. ptr = NULL;
• 2. X = 0
• 3. ptr->next = …
• event2 happened before event3

Applications of Causality

• Checkpointing and Recovery
• Concurrent database updates
• …