Consensus in Distributed Systems Course: Distributed Computing - - PowerPoint PPT Presentation

Consensus in Distributed Systems

Course: Distributed Computing Faculty: Dr. Rajendra Prasath

Spring 2019

About this topic

This course covers various concepts in Consensus in Distributed Systems. We will also focus on the essential aspects of consensus protocols in distributed systems.

2

Rajendra, IIIT Sri City

What did you learn so far? What did you learn so far?

è Challenges in Message Passing systems è Distributed Sorting è Space-Time Diagram è Partial Ordering / Causal Ordering è Concurrent Events è Local Clocks and Vector Clocks è Distributed Snapshots è Termination Detection è Topology Abstraction and Overlays è Leader Election Problem in Rings è Message Ordering / Group Communications è Distributed Mutual Exclusion Algorithms

RECAP

Rajendra, IIIT Sri City

3

Topics to focus on

• pics to focus on …

è Distributed Mutual Exclusion è Deadlock Detection è Check Pointing and Rollback Recovery è Self-Stabilization Self-Study:

è Distributed Consensus

è Peer – to – peer computing and Overlays è Authentication in Distributed Systems

Rajendra, IIIT Sri City

4

Definition Definition

A system is self-stabilizing if and only if: è Convergence: Starting from any state, it is guaranteed that the system will eventually reach a correct state è Closure: Given that the system is in a correct state, it is guaranteed to stay in a correct state, provided that no fault happens è A system is said to be randomized self-stabilizing if and only if it is self-stabilizing and the expected number of rounds needed to reach a correct state is bounded by some constant k

5

Rajendra, IIIT Sri City

Dijkstr Dijkstra's a's Algorithm Algorithm

è For any machine:

è S – State of its own è L – State of the left neighbor and è R - State of the right neighbor on the ring

è The exceptional machine:

è If L = S then S = (S+1) mod K;

è All other machines:

è If L = S then S = L;

6

Rajendra, IIIT Sri City

Dijkstr Dijkstra's a's Algorithm Algorithm

è A Privilege of a machine is able to change its current state on a Boolean predicate that consists of its current state and the states of its neighbors è When a machine has a privilege, it is able to change its current state, which is referred to as a move.

Second solution (K = 3)

è The bottom machine, machine 0:

è If (S+1) mod 3 = R then S = (S−1) mod 3;

è The top machine, machine n−1:

è If L = R and (L+1) mod 3 = S then S = (L+1) mod 3;

è The other machines:

è If (S+1) mod 3 = L then S = L;

7

Rajendra, IIIT Sri City

Consensus in Distributed Systems

Let us explore distributed consensus algorithms in Distributed Systems

Rajendra, IIIT Sri City

8

Distributed Consensus Distributed Consensus

Reaching agr Reaching agreement eement is a fundamental pr is a fundamental problem in

• blem in

distributed computing distributed computing Examples: Examples:

è Leader election / Mutual Exclusion è Commit or Abort in distributed transactions è Reaching agreement about which process has failed è Clock phase synchronization è Air traffic control system: all aircrafts must have the same view

Rajendra, IIIT Sri City

9

Problem

• blem Specification

Specification

u3 u2 u1 u0 v v v v

è Each process pi has an input value ui è These processes exchanges their inputs, so that the

• utputs of all non-faulty processes become identical,

even if one or more processes fail at any time è Output v must be equal to the value of at least one process

input

• utput

p0 p1 p2 p3

Rajendra, IIIT Sri City

10

Problem Specification

• blem Specification

Termination ermination Every non-faulty process must eventually decide. Agr Agreement eement The final decision of every non-faulty process must be identical. Validity alidity If every non-faulty process begins with the same initial value v, then their final decision must be v

Rajendra, IIIT Sri City

11

Observation Observation

• If there is no failure, then reaching

consensus is trivial. All-to-all broadcast followed by a applying a choice function …

• Consensus in presence of failures can

however be complex. The complexity depends on the system model and the type of failures

Rajendra, IIIT Sri City

12

Asynchr Asynchronous Consensus

• nous Consensus

Seven members of a busy household decided to to hir hire e a a cook cook, since they do not have time to prepare their own food Each member separ separately ately interviewed interviewed every applicant for the cook’s position. Depending on how it went, each member voted "yes" (means "yes" (means “hir hire”) or "no" (means "no" (means “don't hir don't hire”). These members will now have to communicate with one another to reach each a a uniform uniform final final decision decision about whether the applicant will be hired The process will be repeated with the next applicant, until someone is hired. Consider various modes of communication like shared memory or message passing Also assume that one process may crash at any time

Rajendra, IIIT Sri City

13

Asynchr Asynchronous Consensus

• nous Consensus

Theor heorem em:

In a pur purely asynchr ely asynchronous

• nous distributed system,

the consensus problem is impossible impossible to solve if even a single pr single process cr

• cess crashes

ashes Fischer ischer, L Lynch, ynch, P Pat atterson terson (commonly known as FLP 85). Received the most influential paper award of ACM PODC in 2001

Rajendra, IIIT Sri City

14

Proof

• of

Bivalent and Univalent states Bivalent and Univalent states A decision state is bivalent bivalent, if starting from that state, there exist two distinct executions leading to two distinct decision values 0 or 1 1 Otherwise it is univalent univalent An univalent univalent state may be either 0-valent 0-valent or 1- 1- valent. valent.

Rajendra, IIIT Sri City

15

Proof

• of

Lemma: emma: Every consensus protocol must have a bivalent initial state bivalent initial state. Proof

• of by

by contr contradiction adiction: Suppose not. Then consider the following input patterns: n-1 2 1 0 s[0] 0 0 0 0 0 0 …0 0 0 {0-valent) There must be a j: 0 0 0 0 0 0 …0 0 1 s[j] is 0-valent 0 0 0 0 0 0 …0 1 1 s[j+1] is 1-valent … … … … s[n] 1 1 1 1 1 1 …1 1 1 {1-valent}

What if process j crashes at the first step?

Rajendra, IIIT Sri City

16

Consensus in Synchr Consensus in Synchronous Systems:

• nous Systems:

Byzantine Gener Byzantine Generals P als Problem

• blem
• Describes and solves the consensus problem
• n the synchronous Communication model
• Processor speeds have lower bounds and

communication delays have upper bounds.

• The network is completely connected
• Processes undergo byzantine failures, the

worst possible kind of failure

Rajendra, IIIT Sri City

17

Byzantine Gener Byzantine Generals P als Problem

• blem
• n generals {0, 1, 2, . . . , n-1} decide about whether

to "attack" or to "retreat" during a particular phase of a war. The goal is to agr agree ee upon upon the the same plan of same plan of action action

• Some generals may be "tr

"traitors" aitors" and therefore send either no input, or send conflicting inputs to prevent the "loyal" "loyal" gener generals als fr from

• m reaching

eaching an an agr agreement eement

• Devise a strategy, by which every loyal general

eventually agrees upon the same plan, regardless

• f the action of the traitors

Rajendra, IIIT Sri City

18

Byzantine Gener Byzantine Generals als

3 2 1

Attack = 1 Attack=1 Retreat = 0 Retreat = 0 {1, 1, 0, 0} {1, 1, 0, 0}

Every general will broadcast his judgment to everyone else. These are inputs to the consensus protocol.

{1, 1, 0, 1} {1, 1, 0, 0}

traitor

The tr he traitor aitor may send may send

• ut
• ut

conflicting conflicting inputs inputs

Rajendra, IIIT Sri City

19

Byzantine Gener Byzantine Generals als

We need to devise a protocol so that every peer (call it a lieutenant lieutenant) receives eceives the same value the same value from any given general (call it a commander commander) Clearly, the lieutenants will have to use secondary secondary information information Note that the roles of the commander and the lieutenants will rotate among the generals

Rajendra, IIIT Sri City

20

Inter Interactive consistency specifications active consistency specifications

IC1: IC1: Every loyal lieutenant receives the same same or

• rder

der from the commander IC2: IC2: If the commander is loyal, then every loyal lieutenant receives the

• rder that the commander

sends

commander commander

lieutenants

Rajendra, IIIT Sri City

21

The Communication Model he Communication Model

Or Oral Messages (OM) al Messages (OM)

• 1. Messages are not corrupted during the transit
• 2. Messages can be lost, but the absence of message

can be detected

• 3. When a message is received (or its absence is

detected), the receiver knows the identity of the sender (or the defaulter) OM(m) OM(m) represents an interactive consistency protocol in presence of at most m traitors.

Rajendra, IIIT Sri City

22

An Impossibility Result An Impossibility Result

commander 0 commander 0 lieutenent 1 lieutenant 2 lieutenent 1 lieutenant 2 1 1 1 1 (a) (b)

Using oral messages, no solution no solution to the Byzantine Generals problem exists with thr three or fewer ee or fewer gener generals als and one tr

• ne traitor
• aitor. Consider the two cases:

In (a), to satisfy IC2, lieutenant 1 must trust the commander, but in IC2, the same idea leads to the violation of IC1.

Rajendra, IIIT Sri City

23

Impossibility r Impossibility result esult

Using oral messages, no no solution solution to to the the Byzantine Byzantine Gener Generals als pr problem

• blem exists

exists with 3m 3m or fewer generals and m traitors (m > 0 m > 0) The pr he proof

• of is by contr

is by contradiction adiction: Assume that such a solution

• exists. Now, divide the 3m generals into three groups of m

generals each, such that all the traitors belong to one

• group. Let one general simulate each of these three
• groups. This scenario is equivalent to the case of three

generals and one traitor. We already know that such a solution does not exist. Note Note: Be always suspicious about such an informal reasoning (refer to Lamport’s original paper)

Rajendra, IIIT Sri City

24

The OM(m) algorithm he OM(m) algorithm

Recursive algorithm

OM(m) OM(m-1) OM(m-2) OM(0)

OM(0) = Dir OM(0) = Direct br ect broadcast

• adcast

OM(0)

Rajendra, IIIT Sri City

25

The OM(m) algorithm he OM(m) algorithm

1. Commander i sends out a value v (0 or 1) 2. If m > 0 0, then every lieutenant j ≠ i, after receiving v , acts as a commander and initiates OM( OM(m-1) with everyone except i 3. Every lieutenant, collects (n-1)

• 1) values:

à (n-2) values received from the lieutenants using OM( OM(m-1)

• 1) and

à one direct value from the commander Then he picks the the majority majority of

• f these values

as the or

• rder

der from i

Rajendra, IIIT Sri City

26

Example of Example of OM(1) OM(1)

1 1 (a) 1 2 2 3 2 3 3 1 1 2 1 1 1 1 1 1 1 1 2 2 3 2 3 3 1 1 2 1 1 1 1 (b) commander commander

Rajendra, IIIT Sri City

27

Example of Example of OM(2) OM(2)

1 2 2 3 4 5 6 4 5 6 5 6 2 4 6 2 4 5 Commander

OM(2) OM(1) v v v v v v v v v v v v v v v v v v 5 6 2 6 2 5 OM(0) v v v v v v

OM(2) OM(2) OM(1) OM(1) OM(0) OM(0)

Rajendra, IIIT Sri City

28

Proof

• of of
• f OM(m)

OM(m)

Lemma: Let the commander be loyal, and n > 2m + k n > 2m + k, where m = maximum number of traitors. Then OM(k) OM(k) satisfies IC2 IC2

m traitors n-m-1 loyal lieutenants values received via OM(r) loyal commander

Rajendra, IIIT Sri City

29

Proof

• of of
• f OM(m)

OM(m)

Proof: If k=0, then the result trivially holds. Let it hold for k = r k = r (r > 0) i.e. i.e. OM(r) OM(r) satisfies IC2 satisfies IC2. We have to show that it holds for k = r + 1 k = r + 1 too. By definition, n > 2m+ r+1, so n-1 > 2m+ r So So OM(r) holds OM(r) holds for the lieutenants in the for the lieutenants in the bot bottom r tom row

• w.

Each loyal lieutenant collects n-m-1 n-m-1 identical good values and m bad values. So bad values are voted

• ut (n-m-1 >

n-m-1 > m+r m+r implies n-m-1 > m n-m-1 > m)

m traitors n-m-1 loyal lieutenants values received via OM(r) loyal commander

“OM(r) holds” means each loyal lieutenant receives identical values from every loyal commander

Rajendra, IIIT Sri City

30

The F he Final theor inal theorem em

Theor heorem em: If n > 3m n > 3m where m is the maximum number of traitors, then OM(m) OM(m) satisfies both IC1 IC1 and IC2 IC2 Proof:

• of: Consider two cases:

Case 1 ase 1: Commander is loyal The theorem follows from the previous lemma (substitute k = m). Case 2 ase 2: Commander is a traitor We prove it by induction: Base case: m=0 trivial. (Induction hypothesis) Let the theorem hold for m = r We have to show that it holds for m = r+1 too.

Rajendra, IIIT Sri City

31

Proof

• of (continued)

(continued)

There are n > 3(r + 1) > 3(r + 1) generals and r + 1 r + 1 traitors Excluding the commander, there are > 3r+2 gener > 3r+2 generals als of which there are r tr r traitors

• aitors. So > 2r+2 lieutenants are loyal.

Since 3r+ 2 > 3r è OM(r) satisfies IC1 and IC2

> 2r+2 > 2r+2 r tr r traitors aitors

Rajendra, IIIT Sri City

32

Proof

• of (continued)

(continued)

In OM(r+1), a loyal lieutenant chooses the majority from (1) > 2r+1 values obtained from the loyal lieutenants via OM(r), (2) the r values from the traitors, and (3) the value directly from the commander.

> 2r+2 > 2r+2 r tr r traitors aitors

The set of values collected in part (1) & (3) are the same for all loyal lieutenants - it is the same set of values that these lieutenants received from the commander Also, by the induction hypothesis, in part (2) each loyal lieutenant receives identical values from each traitor. So every loyal lieutenant eventually collects the same set of values

Rajendra, IIIT Sri City

33

Applications Applications

Rajendra, IIIT Sri City

34 The real world applications include

è Block Chain Transactions – Bit Coins è Clock Synchronization è PageRank è Opinion Formation è Smart Power Grids è State Estimation è Control of UAVs (and multiple robots / agents in general) è Load Balancing and many other domains è Block Chain

è Distributed Consensus is really important

Summary Summary

è State of Machines è Legitimate / Illegitimate States è Self-Stabilizing Algorithms

è Dijkstra’s Algorithm (token rings) è Constructing a Breadth First Tree

è Fault Tolerance è Costs of self-stabilization

è Stay tuned ... More to come up … !!

Rajendra, IIIT Sri City

35

How to r How to reach me? each me?

è Please leave me an email:

rajendra [DOT] prasath [AT] iiits [DOT] in

è Visit my homepage @

è http://www.iiits.ac.in/FacPages/index- rajendra.html OR è http://rajendra.2power3.com 36

Rajendra, IIIT Sri City

• Perspective Students (having CGPA above 8.5

and above)

• Promising Students (having CGPA above 6.5

and less than 8.5)

• Needy Students (having CGPA less than 6.5)
• Can the above group help these students? (Your

work will also be rewarded)

• You may grow a culture of collaborative

learning by helping the needy students

Help among Yourselves?

37

Rajendra, IIIT Sri City

… Questions ???

Thanks …

Rajendra, IIIT Sri City

38