Three Challenges in Distributed Optimization Keren Censor-Hillel - - PowerPoint PPT Presentation

β–Ά
three challenges in distributed optimization
SMART_READER_LITE
LIVE PREVIEW

Three Challenges in Distributed Optimization Keren Censor-Hillel - - PowerPoint PPT Presentation

Jun 07, 2023 266 likes β€’969 views

Three Challenges in Distributed Optimization Keren Censor-Hillel Technion Workshop on Local Algorithms WOLA 2019 This project has received funding from the European Unions Horizon 2020 Research and Innovation Programme under grant agreement

slide-1
SLIDE 1

Three Challenges in Distributed Optimization

Keren Censor-Hillel Technion Workshop on Local Algorithms WOLA 2019

This project has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under grant agreement no. 755839

1

slide-2
SLIDE 2

Optimization problems

2

slide-3
SLIDE 3

Optimization problems

3

Minimum vertex cover

slide-4
SLIDE 4

Optimization problems

4

Minimum vertex cover NP-hard (2-Ξ΅)-approximation is UG-hard (some algs. with a smaller-than-2 apx)

slide-5
SLIDE 5

Distributed optimization

5

Minimum vertex cover Complexity ?

slide-6
SLIDE 6

Distributed graph algorithms

6

#Nodes = n #Bandwidth = B (typically O(logn)) Knowledge of neighbors #Rounds = ? CONGEST

slide-7
SLIDE 7

Distributed graph algorithms

7

#Nodes = n #Bandwidth = B (typically O(logn)) Knowledge of neighbors #Rounds = ? Models: CONGEST LOCAL

slide-8
SLIDE 8

Distributed graph algorithms

8

#Nodes = n #Bandwidth = B (typically O(logn)) Knowledge of neighbors #Rounds = ? Models: CONGEST LOCAL

slide-9
SLIDE 9

Distributed graph algorithms

9

#Nodes = n #Bandwidth = B (typically O(logn)) Knowledge of neighbors #Rounds = ? Models: CONGEST 𝑃 𝑛 = 𝑃(π‘œ&) LOCAL 𝑃(𝐸)

slide-10
SLIDE 10

10

Challenge 1: Distances

slide-11
SLIDE 11

Challenge 1: Distances

11

slide-12
SLIDE 12

Distances

12

Exact minimum vertex cover: Ω(𝐸) rounds

slide-13
SLIDE 13

Distances

13

Exact minimum vertex cover: Ξ©(𝐸) rounds Approximations: LOCAL: (1 + πœ—)-approximation in O(π‘žπ‘π‘šπ‘§ (log π‘œ /πœ—)) rounds [Ghaffari, Kuhn, Maus β€˜17]

slide-14
SLIDE 14

Distances

14

Exact minimum vertex cover: Ξ©(𝐸) rounds Approximations: LOCAL: (1 + πœ—)-approximation in O(π‘žπ‘π‘šπ‘§ (log π‘œ /πœ—)) rounds [Ghaffari, Kuhn, Maus β€˜17] Ξ©(log Ξ”/ log log Ξ”) , Ξ©(√(log π‘œ / log log π‘œ)) rounds [Kuhn, Moscibroda, Wattenhofer β€˜04] Ξ©(1/πœ—) [Ben-Basat, Kawarabayashi, Schwartzman β€˜18]

slide-15
SLIDE 15

Optimal solutions for pieces of the graph

LOCAL: (1 + πœ—)-approximation in O(π‘žπ‘π‘šπ‘§ (log π‘œ /πœ—)) rounds [Ghaffari, Kuhn, Maus β€˜17] Come to Yannic’s talk tomorrow!

15

slide-16
SLIDE 16

Optimal solutions for pieces of the graph

LOCAL: (1 + πœ—)-approximation in O(π‘žπ‘π‘šπ‘§ (log π‘œ /πœ—)) rounds [Ghaffari, Kuhn, Maus β€˜17] Come to Yannic’s talk tomorrow!

16

slide-17
SLIDE 17

Optimal solutions for pieces of the graph

LOCAL: (1 + πœ—)-approximation in O(π‘žπ‘π‘šπ‘§ (log π‘œ /πœ—)) rounds [Ghaffari, Kuhn, Maus β€˜17] Come to Yannic’s talk tomorrow!

17

slide-18
SLIDE 18

Optimal solutions for pieces of the graph

LOCAL: (1 + πœ—)-approximation in O(π‘žπ‘π‘šπ‘§ (log π‘œ /πœ—)) rounds [Ghaffari, Kuhn, Maus β€˜17] Come to Yannic’s talk tomorrow!

18

slide-19
SLIDE 19

Optimal solutions for pieces of the graph

LOCAL: (1 + πœ—)-approximation in O(π‘žπ‘π‘šπ‘§ (log π‘œ /πœ—)) rounds [Ghaffari, Kuhn, Maus β€˜17] Come to Yannic’s talk tomorrow!

19

slide-20
SLIDE 20

Challenge 2: Congestion

20

slide-21
SLIDE 21

21

Challenge 2: Congestion

slide-22
SLIDE 22

Congestion

LOCAL: (1 + πœ—)-approximation in O(π‘žπ‘π‘šπ‘§ (log π‘œ /πœ—)) rounds [Ghaffari, Kuhn, Maus β€˜17] Cannot collect dense neighborhoods quickly

22

slide-23
SLIDE 23

23

(2+Ξ΅)-approximation in 𝑃(log Ξ”/ log log Ξ”) rounds

[Bar-Yehuda, C., Schwartzman β€˜16]

2-approximation [Ben-Basat , Even, Kawarabayashi, Schwartzman β€˜18]

  • (n2), (2-Ξ΅)-approximation [Ben-Basat , Kawarabayashi, Schwartzman β€˜18]

Ξ©(log Ξ”/ log log Ξ”) , Ξ©(√(log π‘œ / log log π‘œ)) rounds for approximation

[Kuhn, Moscibroda, Wattenhofer β€˜04]

CONGEST

slide-24
SLIDE 24

24

(2+Ξ΅)-approximation in 𝑃(log Ξ”/ log log Ξ”) rounds

[Bar-Yehuda, C., Schwartzman β€˜16]

2-approximation [Ben-Basat , Even, Kawarabayashi, Schwartzman β€˜18]

  • (n2), (2-Ξ΅)-approximation [Ben-Basat , Kawarabayashi, Schwartzman β€˜18]

Ξ©(log Ξ”/ log log Ξ”) , Ξ©(√(log π‘œ / log log π‘œ)) rounds for approximation

[Kuhn, Moscibroda, Wattenhofer β€˜04]

CONGEST

slide-25
SLIDE 25

25

(2+Ξ΅)-approximation in 𝑃(log Ξ”/ log log Ξ”) rounds

[Bar-Yehuda, C., Schwartzman β€˜16]

2-approximation [Ben-Basat , Even, Kawarabayashi, Schwartzman β€˜18]

  • (n2), (2-Ξ΅)-approximation [Ben-Basat , Kawarabayashi, Schwartzman β€˜18]

Ξ©(log Ξ”/ log log Ξ”) , Ξ©(√(log π‘œ / log log π‘œ)) rounds for approximation

[Kuhn, Moscibroda, Wattenhofer β€˜04]

Ξ©(π‘œ&/π‘žπ‘π‘šπ‘§ log π‘œ) rounds for exact [C., Khoury, Paz β€˜17]

CONGEST

slide-26
SLIDE 26

26

Minimum vertex cover in CONGEST

approximation #rounds 9 Ξ©(π‘œ&) 1 (exact) 2+Ξ΅ 2 [C., Khoury, Paz β€˜17] [Bar-Yehuda, C., Schwartzman β€˜16] [Kuhn, Moscibroda, Wattenhofer β€˜04] [Ben-Basat, Even, Kawarabayashi, Schwartzman β€˜18]

slide-27
SLIDE 27

27

Minimum vertex cover in CONGEST

approximation #rounds 9 Ξ©(π‘œ&) [C., Khoury, Paz β€˜17] 1 (exact) 2+Ξ΅ 2 [Bar-Yehuda, C., Schwartzman β€˜16] [Kuhn, Moscibroda, Wattenhofer β€˜04] [Ben-Basat, Even, Kawarabayashi, Schwartzman β€˜18]

slide-28
SLIDE 28

28

Minimum vertex cover in CONGEST

approximation #rounds 9 Ξ©(π‘œ&) 1 (exact) 2+Ξ΅ 2 [C., Khoury, Paz β€˜17] 1.5 [Bachrach, C., Dory, Efron, Leitersdorf, Paz β€˜19] [Bar-Yehuda, C., Schwartzman β€˜16] [Kuhn, Moscibroda, Wattenhofer β€˜04] [Ben-Basat, Even, Kawarabayashi, Schwartzman β€˜18]

slide-29
SLIDE 29

29

Minimum vertex cover in CONGEST

approximation #rounds 9 Ξ©(π‘œ&) 1 (exact) 2+Ξ΅ 2 [C., Khoury, Paz β€˜17] 1.5 [Bachrach, C., Dory, Efron, Leitersdorf, Paz β€˜19] [Bar-Yehuda, C., Schwartzman β€˜16] [Kuhn, Moscibroda, Wattenhofer β€˜04] [Ben-Basat, Even, Kawarabayashi, Schwartzman β€˜18]

slide-30
SLIDE 30

30

Minimum vertex cover in CONGEST

approximation #rounds 9 Ξ©(π‘œ&) 1 (exact) 2+Ξ΅ 2 [C., Khoury, Paz β€˜17] 1.5 [Bachrach, C., Dory, Efron, Leitersdorf, Paz β€˜19] [Bar-Yehuda, C., Schwartzman β€˜16] [Kuhn, Moscibroda, Wattenhofer β€˜04] [Ben-Basat, Even, Kawarabayashi, Schwartzman β€˜18]

slide-31
SLIDE 31

2-Party communication

[Yao β€˜79]

31

slide-32
SLIDE 32

History

32

slide-33
SLIDE 33

History

33

[Peleg, Rubinovich β€˜97] [Lotker, Patt-Shamir, Peleg ’01] [Elkin β€˜04] [Das-Sarma, Holzer, Kor, Korman, Nanongkai, Pandurangan, Peleg, Wattenhofer β€˜11] [Frischknecht, Holzer, Wattenhofer β€˜12] [Ghaffari, Kuhn ’13] [Drucker, Kuhn, Oshman β€˜14] [Nanongkai, Das-Sarma, Pandurangan β€˜14] [Das-Sarma, Molla, Pandurangan β€˜15] [Holzer, Pinsker, β€˜15] [Pandurangan, Peleg, Scquizzato β€˜16] [Pandurangan, Robinson, Scquizzato β€˜16] [C., Kavitha, Paz, Yehudayoff ’16] [Fischer, Gonen, Kuhn, Oshman β€˜18] [C., Dory β€˜17] [C., Dory ’18]

slide-34
SLIDE 34

2-Party communication

34

Alice: x=x1,…,xk Bob: y=y1,…,yk Goal: f(x,y)

slide-35
SLIDE 35

2-Party communication

Alice: x=x1,…,xk

35

Bob: y=y1,…,yk Goal: f(x,y)

[Kalyanasundaram and Schnitger β€˜87] [Razborov β€˜90] [Bar-Yossef et al. β€˜04]

Set-Disjointess: i: xi=yi=1 ? cost=Ξ©(k)

slide-36
SLIDE 36

2-Party communication Γ¨ CONGEST

36

Graph property P of Gx,y determines f(x,y) Alice: x=x1,…,xk Bob: y=y1,…,yk

slide-37
SLIDE 37

2-Party communication Γ¨ CONGEST

37

RoundsŸcutŸB = Ξ©(cost(f(k))) Graph property P of Gx,y determines f(x,y) Alice: x=x1,…,xk Bob: y=y1,…,yk

slide-38
SLIDE 38

2-Party communication Γ¨ CONGEST

38

RoundsŸcutŸB = Ξ©(cost(f(k))) Rounds = Ξ©(cost(f(k))/cutŸB) Graph property P of Gx,y determines f(x,y) Alice: x=x1,…,xk Bob: y=y1,…,yk

slide-39
SLIDE 39

Warm-up: MVC, CONGEST

39

: π‘œ Add edge iff input is 0 cut = : π‘œ = π‘œ/6

slide-40
SLIDE 40

Warm-up: MVC, CONGEST

40

Add edge iff input is 0 cut = : π‘œ = π‘œ/6 : π‘œ

slide-41
SLIDE 41

Warm-up: MVC, CONGEST

41

Add edge iff input is 0 cut = : π‘œ = π‘œ/6 𝑙 = : π‘œ& : π‘œ

slide-42
SLIDE 42

Warm-up: MVC, CONGEST

42

: π‘œ

MVC = 4: π‘œ-2 inputs not disjoint

slide-43
SLIDE 43

Warm-up: MVC, CONGEST

43

: π‘œ

MVC = 4: π‘œ-2 inputs not disjoint MVC β‰₯ 4: π‘œ-1 inputs disjoint

slide-44
SLIDE 44

Warm-up: MVC, CONGEST

44

MVC = 4: π‘œ-2 inputs not disjoint

: π‘œ

MVC β‰₯ 4: π‘œ-1 inputs disjoint Lower bound: Ξ©(𝑙/π‘œ log π‘œ) = 9 Ξ©(π‘œ) rounds

slide-45
SLIDE 45

45

Minimum vertex cover in CONGEST

approximation #rounds 9 Ξ©(π‘œ&) 1 (exact) 2+Ξ΅ 2 [C., Khoury, Paz β€˜17] 1.5 [Bachrach, C., Dory, Efron, Leitersdorf, Paz β€˜19] [Bar-Yehuda, C., Schwartzman β€˜16] [Kuhn, Moscibroda, Wattenhofer β€˜04] [Ben-Basat, Even, Kawarabayashi, Schwartzman β€˜18]

slide-46
SLIDE 46

Small cuts for MVC

[C., Khoury, Paz β€˜17]

Bit-gadget used for, e.g., diameter in [Abboud, C., Khoury β€˜16]

46

0 1 1

slide-47
SLIDE 47

Small cuts for MVC

[C., Khoury, Paz β€˜17]

47

MVC = 4(: π‘œ-1+log : π‘œ) inputs not disjoint

0 1 1

slide-48
SLIDE 48

Small cuts for MVC

[C., Khoury, Paz β€˜17]

48

0 1 1

k=Θ(n2), cut=Θ(logn) Rounds = Ω(cost(f(k))/cutŸlogn) = Ω(n2/log2n)

slide-49
SLIDE 49

49

Minimum vertex cover in CONGEST

approximation #rounds 9 Ξ©(π‘œ&) 1 (exact) 2+Ξ΅ 2 [C., Khoury, Paz β€˜17] 1.5 [Bachrach, C., Dory, Efron, Leitersdorf, Paz β€˜19] [Bar-Yehuda, C., Schwartzman β€˜16] [Kuhn, Moscibroda, Wattenhofer β€˜04] [Ben-Basat, Even, Kawarabayashi, Schwartzman β€˜18]

slide-50
SLIDE 50

Why not for 1.5-approximation?

  • Alice/Bob compute OPTA and OPTB for their parts
  • The smaller is at most OPT/2

50

slide-51
SLIDE 51

Why not for 1.5-approximation?

  • Alice/Bob compute OPTA and OPTB for their parts
  • The smaller is at most OPT/2

51

slide-52
SLIDE 52

Why not for 1.5-approximation?

  • Alice/Bob compute OPTA and OPTB for their parts
  • The smaller is at most OPT/2
  • The other side fills in the rest, at most OPT

52

slide-53
SLIDE 53

Optimization problems

53

slide-54
SLIDE 54

54

Maximum independent set in CONGEST

approximation #rounds 9 Ξ©(π‘œ&) 1 (exact) 𝑃(1/Ξ”) [C., Khoury, Paz β€˜17] [Kawarabayashi, Khoury, Schild, Schwartzman β€˜19] [Boppana, Halldorsson, Rawitz β€˜18]

slide-55
SLIDE 55

55

Maximum independent set in CONGEST

approximation #rounds 9 Ξ©(π‘œ&) 1 (exact) 𝑃(1/Ξ”) [C., Khoury, Paz β€˜17] 7/8 [Bachrach, C., Dory, Efron, Leitersdorf, Paz β€˜19] [Kawarabayashi, Khoury, Schild, Schwartzman β€˜19] [Boppana, Halldorsson, Rawitz β€˜18]

slide-56
SLIDE 56

56

Maximum independent set in CONGEST

approximation #rounds 9 Ξ©(π‘œ&) 1 (exact) 𝑃(1/Ξ”) 1/2 [C., Khoury, Paz β€˜17] 7/8 [Bachrach, C., Dory, Efron, Leitersdorf, Paz β€˜19] [Kawarabayashi, Khoury, Schild, Schwartzman β€˜19] [Boppana, Halldorsson, Rawitz β€˜18]

slide-57
SLIDE 57

7/8-approximation for MaxIS

[Bachrach, C., Dory, Efron, Leitersdorf, Paz β€˜19] Bit-gadget

57

0 1 1

slide-58
SLIDE 58

7/8-approximation for MaxIS

[Bachrach, C., Dory, Efron, Leitersdorf, Paz β€˜19] Code-gadget: non-binary codewords with a polylog distance 9 Ξ©(π‘œ&) rounds

58

slide-59
SLIDE 59

Optimization problems

  • Minimum vertex cover
  • Maximum independent set

59

slide-60
SLIDE 60

Optimization problems

  • Minimum vertex cover
  • Maximum independent set
  • Max cut

[Bachrach, C., Dory, Efron, Leitersdorf, Paz β€˜19] – Requires 9 Ξ©(π‘œ&) rounds – (1 βˆ’ πœ—)-approximation in ? 𝑃(π‘œ) rounds [Zelke β€˜09]

60

slide-61
SLIDE 61

Optimization problems

  • Minimum vertex cover
  • Maximum independent set
  • Max cut

[Bachrach, C., Dory, Efron, Leitersdorf, Paz β€˜19] – Requires 9 Ξ©(π‘œ&) rounds – (1 βˆ’ πœ—)-approximation in ? 𝑃(π‘œ) rounds [Zelke β€˜09]

  • FT-BFS

[Ghaffari, Parter β€˜16] – 2-approximation in ?

𝑃(𝐸) rounds

– sequential approximation implies set cover

61

slide-62
SLIDE 62

Challenge 3: ???

62

slide-63
SLIDE 63

63

Challenge 3: ???

slide-64
SLIDE 64

Alice-Bob limitations

  • For triangle detection [Drucker, Kuhn, Oshman β€˜14]

– [Izumi, Le Gall β€˜17]: ? 𝑃(π‘œ&/@) – [Chang, Pettie, Zhang, ’18]: ? 𝑃(π‘œA/&) – [Chang, Saranurak β€˜19]: ? 𝑃(π‘œA/@)

  • For weighted APSP above Ξ© π‘œ [C., Khoury, Paz ’17]

– [Elkin ’17]: ? 𝑃(π‘œB/@) – [Huang, Nanongkai, Saranurak ’17]: ? 𝑃(π‘œB/C) – [Agarwal, Ramachandran, King, Pontecorvi ’18]: ? 𝑃(π‘œ@/&) det. – [Bernstein, Nanongkai β€˜19]: ? 𝑃(π‘œ)

  • For 4-clique detection above Ξ©(π‘œA/&) [Czumaj, Konrad β€˜18]

– [Eden, Fiat, Fischer, Kuhn, Oshman β€˜19]: first sublinear algorithm

64

slide-65
SLIDE 65
  • 1. Distances
  • 2. Congestion
  • 3. ???

Can we have better approximation factors in the CONGEST model?

65

Distributed optimization

slide-66
SLIDE 66
  • 1. Distances
  • 2. Congestion
  • 3. ???

Can we have better approximation factors in the CONGEST model? CONGESTED CLIQUE? MPC? Streaming?

66

Distributed optimization

A challenge for WOLA 2020

slide-67
SLIDE 67
  • 1. Distances
  • 2. Congestion
  • 3. ???

Can we have better approximation factors in the CONGEST model? CONGESTED CLIQUE? MPC? Streaming?

67

Distributed optimization

A challenge for WOLA 2020

slide-68
SLIDE 68
  • 1. Distances
  • 2. Congestion
  • 3. ???

Can we have better approximation factors in the CONGEST model? CONGESTED CLIQUE? MPC? Streaming? Thank you!

68

Distributed optimization

A challenge for WOLA 2020