[PPT] - Local Algorithms: Past, Present, Future Jukka Suomela Helsinki PowerPoint Presentation

SLIDE 1

Jukka Suomela

Helsinki Institute for Information Technology HIIT University of Helsinki Wireless Networks and Mobile Computing Carleton University 26–27 April 2011

www.hiit.fi/jukka.suomela/tut-2011

Local Algorithms: Past, Present, Future

SLIDE 2

About This Tutorial

Two parts:
Part A, Tuesday 11:00–12:30
Part B, Wednesday 11:00–12:30
www.iki.fi/suo/tut
Slides, additional material, further reading

2

SLIDE 3

Background

3

SLIDE 4

Setting

Graphs

4

SLIDE 5

Setting

Graphs
Algorithms for

graph problems

Independent sets

5

SLIDE 6

Setting

Graphs
Algorithms for

graph problems

Independent sets,

matchings

6

SLIDE 7

Setting

Graphs
Algorithms for

graph problems

Independent sets,

matchings, vertex covers

7

SLIDE 8

Setting

Graphs
Algorithms for

graph problems

Independent sets,

matchings, vertex covers, dominating sets

8

SLIDE 9

Setting

Graphs
Algorithms for

graph problems

Independent sets,

matchings, vertex covers, dominating sets, edge dominating sets

9

SLIDE 10

Setting

Graphs
Algorithms for

graph problems

Independent sets,

matchings, vertex covers, dominating sets, edge dominating sets, graph colourings

10

SLIDE 11

Setting

Graphs
Algorithms for

graph problems

Independent sets,

matchings, vertex covers, dominating sets, edge dominating sets, graph colourings, …

11

SLIDE 12

Local Algorithms

Local neighbourhood:

nodes at distance r

Here r = O(1),

independent of number of nodes

Shortest-path distance,

number of edges

12

SLIDE 13

Local Algorithms

Local algorithm:

each node operates based on its local neighbourhood only

Output is a function
f local neighbourhood

13

SLIDE 14

Local Algorithms

Same neighbourhood,

same output

14

SLIDE 15

Local Algorithms

Equivalently:
Constant-time

distributed algorithm

Time = number
f synchronous

communication rounds

15

SLIDE 16

Advantages

Fast and scalable distributed algorithm
By definition…
Fault-tolerant and robust
Changes in input (or network structure):
nly local changes in output
We can quickly recover from any failures
But do these exist?

16

SLIDE 17

Past

17

SLIDE 18

Bad News

Long history of very strong negative results
Linial (1992)
Naor & Stockmeyer (1995)
Czygrinow, Hańćkowiak & Wawrzyniak (2008)
Lenzen & Wattenhofer (2008)
using, e.g., results that date back to Ramsey (1930)

18

SLIDE 19

Bad News

Even if your graph is a cycle…

19

SLIDE 20

Bad News

Even if your graph is a cycle…
And even if you have

unique node identifiers…

20

19 30 12 5 72 4 18 34 2 68 77 15 19

SLIDE 21

Bad News

Even if your graph is a cycle…
And even if you have

unique node identifiers…

And orientation…

21

19 30 12 5 72 4 18 34 2 68 77 15 19

SLIDE 22

Bad News

Even if your graph is a cycle…
And even if you have

unique node identifiers…

And orientation…
Then no matter which

local algorithm you use, there is a “bad input”

22

19 30 12 5 72 4 18 34 2 68 77 15 19

SLIDE 23

Bad News

“Bad input”:
Almost all nodes will

produce the same output

Graph colouring

not possible

You can find only trivial

independent sets, matchings, vertex covers, dominating sets, …

23

19 30 12 5 72 4 18 34 2 68 77 15 19

SLIDE 24

Bad News

Example: A is a local algorithm with r = 2,
utputs from {1, 2, …, k}
Focus on oriented cycles
A maps 5-tuples of

identifiers to local outputs

A(15, 72, 5, 12, 30) = …

24

19 30 12 5 72 4 18 34 2 68 77 15 19

SLIDE 25

Bad News

Example: A is a local algorithm with r = 2,
utputs from {1, 2, …, k}
Set of identifiers: I = {1, 2, …, N}
Let X = {a, b, c, d, e} ⊆ I,

a < b < c < d < e

Define the colour C(X) of X:

C(X) = A(a, b, c, d, e)

25

? e d c b ? ? ? ? ? ? a ?

SLIDE 26

Bad News

Example: A is a local algorithm with r = 2,
utputs from {1, 2, …, k}
Set of identifiers: I = {1, 2, …, N}
Let X = {a, b, c, d, e} ⊆ I,

a < b < c < d < e

Define the colour C(X) of X:

C(X) = A(a, b, c, d, e)

We will colour all 5-subsets of I

26

? e d c b ? ? ? ? ? ? a ?

SLIDE 27

Bad News

Example: A is a local algorithm with r = 2,
utputs from {1, 2, …, k}
Set of identifiers: I = {1, 2, …, N},

colouring C(X) of 5-subsets

Ramsey: if N is large enough,

there exists a large monochromatic subset M ⊆ I

All 5-subsets X ⊆ M have

the same colour C(X)

27

? e d c b ? ? ? ? ? ? a ?

SLIDE 28

Bad News

Example: A is a local algorithm with r = 2,
utputs from {1, 2, …, k}
Assume that M = {a, b, c, d, e, f}

is a monochromatic subset, a < b < c < d < e < f

C({a, b, c, d, e}) =

C({b, c, d, e, f})

A(a, b, c, d, e) = A(b, c, d, e, f)

28

f e d c b ? ? ? ? ? ? a ?

SLIDE 29

Bad News

Example: A is a local algorithm with r = 2,
utputs from {1, 2, …, k}
We have found a “bad input”:

nodes with identifiers c and d are adjacent and they produce the same output

We already proved that

A cannot produce a valid graph colouring!

29

f e d c b ? ? ? ? ? ? a ?

SLIDE 30

Bad News

Example: A is a local algorithm with r = 2,
utputs from {1, 2, …, k}
We can apply the same idea

for any value of r

And we can “boost”

the argument and show that almost all nodes will produce the same output

30

f e d c b ? ? ? ? ? ? a ?

SLIDE 31

Bad News

For
any local algorithm A that finds an independent set,
any constant ε > 0, and
sufficiently large n,

we can choose unique identifiers in an n-cycle so that A outputs an independent set with only εn nodes

31

SLIDE 32

Bad News

For
any local algorithm A that finds a vertex cover,
any constant ε > 0, and
sufficiently large n,

we can choose unique identifiers in an n-cycle so that A outputs a vertex cover with at least (1 − ε)n nodes

32

SLIDE 33

Dealing with Bad News

Three traditional escapes:
Randomised algorithms
Geometric information
“Almost local” algorithms

33

SLIDE 34

Dealing with Bad News

Three traditional escapes:
Randomised algorithms
Geometric information
“Almost local” algorithms

34

SLIDE 35

Randomised Algorithms

Nodes can roll dice

35

SLIDE 36

Randomised Algorithms

Nodes can roll dice or toss coins

36

SLIDE 37

Randomised Algorithms

Nodes can roll dice or toss coins
We cannot guarantee that we find

a good solution

Worst case: all coin tosses equal, no new information
But we can find a good solution

with high probability or in expectation

37

SLIDE 38

Randomised Algorithms

Example: finding an independent set I
Each node v picks

uniformly at random X(v) = ⚀, ⚁, ⚂, ⚃, ⚄, ⚅

38

⚃ ⚀ ⚁ ⚅ ⚂ ⚀ ⚁ ⚂ ⚄ ⚄ ⚂ ⚃ ⚂

SLIDE 39

Randomised Algorithms

Example: finding an independent set I
Each node v picks

uniformly at random X(v) = ⚀, ⚁, ⚂, ⚃, ⚄, ⚅

Node v joins I if X(v) is

(strict) local maximum

39

⚃ ⚀ ⚁ ⚅ ⚂ ⚀ ⚁ ⚂ ⚄ ⚄ ⚂ ⚃ ⚂

SLIDE 40

Randomised Algorithms

Example: finding an independent set I
Each node v picks

uniformly at random X(v) = ⚀, ⚁, ⚂, ⚃, ⚄, ⚅

Node v joins I if X(v) is

(strict) local maximum

By construction,

I is an independent set

40

⚃ ⚀ ⚁ ⚅ ⚂ ⚀ ⚁ ⚂ ⚄ ⚄ ⚂ ⚃ ⚂

SLIDE 41

Randomised Algorithms

Example: finding an independent set I
Each node v picks

uniformly at random X(v) = ⚀, ⚁, ⚂, ⚃, ⚄, ⚅

Node v joins I if X(v) is

(strict) local maximum

Expected size of I

is reasonably large

41

⚃ ⚀ ⚁ ⚅ ⚂ ⚀ ⚁ ⚂ ⚄ ⚄ ⚂ ⚃ ⚂

SLIDE 42

Randomised Algorithms

Example: finding an independent set I
A local randomised

algorithm can find a large independent set

Approximation algorithm

(in expectation)

However, we cannot find

maximum independent set or maximal independent set

42

⚃ ⚀ ⚁ ⚅ ⚂ ⚀ ⚁ ⚂ ⚄ ⚄ ⚂ ⚃ ⚂

SLIDE 43

Dealing with Bad News

Three traditional escapes:
Randomised algorithms
Geometric information
“Almost local” algorithms

43

SLIDE 44

Geometric Graphs

Assume that nodes are

points in the plane

Assume “reasonable”

embedding

44

SLIDE 45

Geometric Graphs

Assume that nodes are

points in the plane

Assume “reasonable”

embedding

Civilised graph

45

SLIDE 46

Geometric Graphs

Assume that nodes are

points in the plane

Assume “reasonable”

embedding

Civilised graph:

edges not too long…

46

no!

SLIDE 47

Geometric Graphs

Assume that nodes are

points in the plane

Assume “reasonable”

embedding

Civilised graph:

edges not too long, nodes not in too dense

47

no!

SLIDE 48

Geometric Graphs

Assume that nodes are

points in the plane

Assume “reasonable”

embedding

Civilised graph
Unit disk graph
Quasi unit disk graph…

48

SLIDE 49

Geometric Graphs

Exploit coordinates
a simple approach:

divide-and-conquer

e.g., partition the plane

in rectangular tiles

49

SLIDE 50

Geometric Graphs

Exploit coordinates
each tile defines

a constant-size subproblem

solve the subproblem

locally within each tile (in parallel for all tiles)

50

SLIDE 51

Geometric Graphs

Exploit coordinates
each tile defines

a constant-size subproblem

solve the subproblem

locally within each tile

merge the solutions
f the subproblems

51

SLIDE 52

Geometric Graphs

Graph colouring:
f = 3-colouring of tiles
all edges are short
there is no edge

that joins e.g. a blue tile and another blue tile

52

SLIDE 53

Geometric Graphs

Graph colouring:
f = 3-colouring of tiles
g = k-colouring that

is valid inside each tile

can be solved

by brute force

53

SLIDE 54

Geometric Graphs

Graph colouring:
f = 3-colouring of tiles
g = k-colouring that

is valid inside each tile

Output: (f, g)
Valid 3k-colouring!

54

SLIDE 55

Geometric Graphs

Simple local

algorithms:

maximal matchings,

independent sets, …

approximation

algorithms for vertex covers, dominating sets, colourings, …

55

SLIDE 56

Dealing with Bad News

Three traditional escapes:
Randomised algorithms
Geometric information
“Almost local” algorithms

56

SLIDE 57

Almost Local Algorithms

We cannot find non-trivial solutions

in a cycle in O(1) rounds

But we can do it in O(log* n) rounds!
log* n = iterated logarithm
0 ≤ log* n ≤ 7 for all real-world values of n
Good enough?

57

SLIDE 58

Almost Local Algorithms

Main tool: colour reduction
Cole & Vishkin (1986)
Goldberg, Plotkin & Shannon (1988)
Bit manipulation trick:
From k colours to O(log k) colours in one step
Initially poly(n) colours: unique identifiers
Iterate O(log* n) times until O(1) colours

58

SLIDE 59

Almost Local Algorithms

59

13573 1029 13348 9252 3108 7491 3139 5443 3427 15683 9315 15715 7523

Initial colouring

SLIDE 60

Almost Local Algorithms

60

13573 1029 13348 9252 3108 7491 3139 5443 3427 15683 9315 15715 7523

110001000011 11110101000011 Key idea: inspect the binary encodings

f old colours

SLIDE 61

Almost Local Algorithms

61

13573 1029 13348 9252 3108 7491 3139 5443 3427 15683 9315 15715 7523

110001000011 11110101000011 Bit number 8 differs (8, 1) 10001

SLIDE 62

Almost Local Algorithms

62

13573 1029 13348 9252 3108 7491 3139 5443 3427 15683 9315 15715 7523

1110101000011 110001000011 11110101000011 Bit number 8 differs (8, 0) (8, 1) 10000 10001

SLIDE 63

Almost Local Algorithms

63

13573 1029 13348 9252 3108 7491 3139 5443 3427 15683 9315 15715 7523

1110101000011 110001000011 1010101000011 11110101000011 (11, 1) Bit number 11 differs (8, 0) (8, 1) 10111 10000 10001

SLIDE 64

Almost Local Algorithms

64

13573 1029 13348 9252 3108 7491 3139 5443 3427 15683 9315 15715 7523

17 1 25 22 23 16 3 17 16 17 25

A proper colouring!

SLIDE 65

Almost Local Algorithms

65

17 1 25 22 23 16 3 … 17 16 17 25 17 1 25 22 23 16 3 17 16 17 25

Update colours…

SLIDE 66

Almost Local Algorithms

66

17 1 25 22 23 16 3 … 17 16 17 25

After one round

SLIDE 67

Almost Local Algorithms

67

9 6 1 3 5 3 … 1 1 …

After two rounds

SLIDE 68

Almost Local Algorithms

68

1 2 1 2 3 … 1 … 1 …

After three rounds

SLIDE 69

Almost Local Algorithms

69

Graph colouring in O(log* n) rounds
Paths or cycles, 3-colouring
Generalisations:
Trees, bounded-degree graphs, …
Graphs of maximum degree ∆:

(∆+1)-colouring in O(∆ + log* n) rounds

SLIDE 70

Almost Local Algorithms

70

Graph colouring in O(log* n) rounds
Many applications:
Maximal independent set:

first try to add nodes of colour 0 (in parallel), then try to add nodes of colour 1 (in parallel), …

Maximal matching
Greedy algorithm for dominating sets

SLIDE 71

Almost Local Algorithms

71

Graph colouring in O(log* n) rounds
Many applications
Fast, but not strictly local
And inherently depends on the existence of

small, unique, numerical identifiers

SLIDE 72

Past: Summary

Bad news:
Cannot break symmetry in cycles
Three traditional escapes:
Randomised algorithms
Geometric information
“Almost local” algorithms

72

SLIDE 73

Present

73

! ! ! ! ! ! ! ! ! ! ! ! !

SLIDE 74

Dealing with Bad News

You cannot break symmetry in cycles…
Which problems do not require

symmetry breaking in cycles?

74

SLIDE 75

Tractable Problems

Linear programs (LPs)
Many resource-allocation

problems can be modelled as LPs

If the input is symmetric,

a trivial solution is an optimal solution!

Only non-symmetric

inputs are challenging…

75

! ! ! ! ! ! ! ! ! ! ! ! !

SLIDE 76

Tractable Problems

Linear programs (LPs)
Approximation scheme for

packing and covering LPs

Local algorithm
Kuhn, Moscibroda &

Wattenhofer (2006)

76

! ! ! ! ! ! ! ! ! ! ! ! !

SLIDE 77

Tractable Problems

Vertex covers
2-approximation is the best that we can find

with centralised polynomial-time algorithms

Nobody knows how to find

1.9999-approximation efficiently

Hence if we could find a 2-approximation

with local algorithms, it would be amazing!

77

SLIDE 78

Tractable Problems

Vertex covers
2-approximation does not

require symmetry breaking

In a regular graph, trivial

solution (all nodes) is 2-approximation

Again, only non-symmetric

inputs are challenging…

78

SLIDE 79

Tractable Problems

Vertex covers
2-approximation of vertex cover

in bounded-degree graphs

Local algorithm
Åstrand & Suomela (2010)

79

SLIDE 80

Tractable Problems

Vertex covers
2-approximation of vertex cover

in bounded-degree graphs

Local algorithm
A bit complicated…
Let’s have a look at

a simpler local algorithm: 3-approximation of vertex cover

80

SLIDE 81

Vertex Cover

A simple local algorithm: 3-approximation of minimum vertex cover

81

SLIDE 82

Vertex Cover

Construct a virtual graph: two copies of each node; edges across

82

SLIDE 83

Vertex Cover

The virtual graph is 2-coloured: all edges are from white to black

83

SLIDE 84

Vertex Cover

The virtual graph is 2-coloured – therefore we can find a maximal matching!

84

SLIDE 85

Vertex Cover

White nodes send proposals to their black neighbours

85

SLIDE 86

Vertex Cover

Black nodes accept one of the proposals

86

SLIDE 87

Vertex Cover

White nodes send proposals to another black neighbour if they were rejected

87

SLIDE 88

Vertex Cover

Again, black nodes accept one proposal – unless they were already matched

88

SLIDE 89

Vertex Cover

Continue until all white nodes are matched –

r they are rejected by all black neighbours

89

SLIDE 90

Vertex Cover

End result: a maximal matching in the virtual graph

90

SLIDE 91

Vertex Cover

Take all original nodes that were matched – 3-approximation of minimum vertex cover!

91

SLIDE 92

Present: Summary

You cannot break symmetry in cycles…
But we can study problems

that do not require symmetry breaking!

Linear programs: local approximation schemes
Vertex covers: local 2-approximation algorithm
Edge dominating sets: local approximation algorithm
…

92

SLIDE 93

Future

93

SLIDE 94

Dealing with Bad News

Let’s have a fresh look at the lower bounds!
Exactly what was proved?

94

SLIDE 95

Lower Bounds

Only trivial solutions in cycles
Assumption:

constant-size output

Each node outputs

constant number of bits

Innocuous?

95

19 30 12 5 72 4 18 34 2 68 77 15 19

SLIDE 96

Output Size

Vertex cover, independent set,

dominating set, cut: 1 bit per node

Matching, edge dominating set,

edge cover: 1 bit per edge

In a cycle, this is O(1) bits per node

96

SLIDE 97

Output Size

Graph colouring:
O(1) colours should be enough in a cycle
Hence O(1) bits per node is enough to

encode the solution

Linear programs:
For a near-optimal solution, we can use

finite-precision rational numbers

97

SLIDE 98

Output Size

Natural problems seem to have

constant-size output

Hence the negative results apply
Unique identifiers do not help in cycles
We can only produce trivial solutions in cycles
We can only solve problems that

do not require symmetry-breaking

98

SLIDE 99

Output Size

Natural problems seem to have

constant-size output

Hence the negative results apply
Did we miss anything?

99

SLIDE 100

Scheduling Problems

Neighbours not working simultaneously,

everyone must get 3 units of work done

100

time: 1 2 3 4 5 6 7 node 1: node 2: node 3: node 4: node 5: node 6:

SLIDE 101

Scheduling Problems

Time is continuous, we can use

a more fine-grained schedule if it helps

101

time: 1 2 3 4 5 6 7 node 1: node 2: node 3: node 4: node 5: node 6:

SLIDE 102

Scheduling Problems

Time is continuous, we can use

a more fine-grained schedule if it helps

A good solution does not necessarily

have a constant-size description

Existing lower bounds do not apply

102

SLIDE 103

Scheduling Problems

Time is continuous, we can use

a more fine-grained schedule if it helps

A good solution does not necessarily

have a constant-size description

Existing lower bounds do not apply
Proof artefact? Uninteresting technicality?

Just derive a bit stronger negative result?

103

SLIDE 104

Scheduling Problems

Time is continuous, we can use

a more fine-grained schedule if it helps

A good solution does not necessarily

have a constant-size description

Existing lower bounds do not apply
Proof artefact? Uninteresting technicality?

Just derive a bit stronger negative result?

Wrong! There is a local approximation algorithm!

104

SLIDE 105

Scheduling Problems

Local approximation algorithms
Scheduling problems:

fractional graph colouring, fractional domatic partition, …

First example of a local algorithm that

actually requires unique numerical identifiers

Hasemann, Hirvonen, Rybicki & Suomela

(work in progress)

105

SLIDE 106

More New Directions

Deterministic local algorithm
cf. deterministic Turing machine – class P
Randomised local algorithm
cf. probabilistic Turing machine – class BPP, etc.
Nondeterministic local algorithm
cf. nondeterministic Turing machine – class NP

106

SLIDE 107

Decision Problems

Back to very basics: decision problems
Is this graph bipartite? Acyclic? Hamiltonian?

Eulerian? Connected? 3-colourable? Symmetric?

Decision problems form the foundation
f classical complexity theory…

107

SLIDE 108

Decision Problems

Decision problems in distributed setting:
yes-instance: all nodes happy
no-instance: at least one node raises alarm
Few decision problems can be solved

with deterministic local algorithms

But now we have a very natural extension…

108

SLIDE 109

Decision Problems

Nondeterministic local algorithms
Yes-instances have a compact certificate

that can be verified with a local algorithm

“locally checkable proof”
Cf. class NP:
Yes-instances have a compact certificate

that can be verified in P

109

SLIDE 110

Locally Checkable Proofs

Key question: what is the size of the proof?
How many bits per node are needed?
For example, it is easy to show that a graph is

bipartite: just give a 2-colouring, 1 bit per node

How do you prove that a graph is not bipartite?

110

SLIDE 111

Locally Checkable Proofs

Key question: what is the size of the proof?
How many bits per node are needed?
For example, it is easy to show that a graph is

bipartite: just give a 2-colouring, 1 bit per node

How do you prove that a graph is not bipartite?
Find an odd cycle, and prove that it exists
O(log n) bits is enough, Ω(log n) bits necessary

111

SLIDE 112

Locally Checkable Proofs

Natural hierarchy of proof complexities:
2-colourable graphs: Θ(1) bits per node
Non-2-colourable graphs: Θ(log n) bits per node
Non-3-colourable graphs: poly(n) bits per node
Göös & Suomela (2011)

112

SLIDE 113

Summary

Local algorithms
Strong lower bounds
Nevertheless,

a lot of progress!

Latest hot topics
Scheduling problems
Nondeterministic models

113

SLIDE 114