An Application-Level Network Mapper Arnaud Legrand 1 eric Mazoit 2 - - PowerPoint PPT Presentation

SLIDE 1

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

An Application-Level Network Mapper

Arnaud Legrand1 Fr´ ed´ eric Mazoit2 Martin Quinson1.

1: ID – UMR 5132 (CNRS – INPG – INRIA – UJF), Grenoble, France. 2: LIP – UMR 5668 (CNRS – ENS-Lyon – UCBL – INRIA), Lyon, France.

SLIDE 2

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Outline

Introduction Context Motivation and goals State of the art ALNeM Model used Measurement methodology Problem statement Mathematical tools, algorithms Total interference and separators Reconstruction trees and cliques of trees Extension for cycles Implementation Data collection Conclusion Contributions and future work

SLIDE 3

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Introduction (1/2)

Motivation Modern platforms (Grid, P2P systems) heterogeneous and dynamic. Distributed applications have to be reactive and network-aware. Quantitative information (bandwidth) well studied [NWS, RPS, ganglia]. Qualitative information (topology) seldom known, but needed for:

Host siting and automatic configuration Group communication

Definitions of topology Almost as many as layers in the OSI model. Physical interconnexion map (wires in the walls) Routing infrastructure (path of network packets, from router to switch) Application level (focus on effects – bandwidth, latency – not causes)

SLIDE 4

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Introduction (2/2)

Our context is at application level Grid or P2P systems = multi-organization platforms.

System heterogeneity ⇒ cannot rely on specific system feature Trust issue ⇒ no privileges for grid administrators (“root” or other)

Our Goal is... Discover What Applications can Expect from the Platform Given 4 hosts (a, b, c, d), determine whether a → b impact c → d (perfs). Intuition: if they share a link, they share the bandwidth. Our goal is not... Discover network bottleneck and configuration issues Discover packet paths

SLIDE 5

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Topology discovery methodologies

State of the art

Method Restriction Focus Routers Notes SNMP authorized path all passive, LAN traceroute ICMP path all level 3 of OSI pathchar root path all link bandwidth, slow Other no path din = dout tree tomography bipartite [Rabbat03] ENV no interference some tree only ALNeM no interference ? complete graph

SLIDE 6

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Outline

Introduction Context Motivation and goals State of the art ALNeM Model used Measurement methodology Problem statement Mathematical tools, algorithms Total interference and separators Reconstruction trees and cliques of trees Extension for cycles Implementation Data collection Conclusion Contributions and future work

SLIDE 7

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Model used

Definition: routed graph G = (V , E, r) Non-oriented graph with routing function (r : V × V → V ).

• u −

→

G v

• is the path (set of vertices encountered in the graph G).

Definition: (ab) interfere with (cd) in G (ab) G (cd) ⇐ ⇒

• a −

→

G b

• ∩
• c −

→

G d

• = ∅

Symmetric relation: (ab) G (cd) ⇔ (cd) G (ab) Routing not symmetric: (ab) G (cd) ⇔ (ab) G (dc)

Definition: (ab) does not interfere with (cd) in G (ab) G (cd) ⇐ ⇒ ¬

• (ab)

G (cd)

SLIDE 8

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Measurement methodology

Notation bw(ab): bandwidth on a → b. bwcd(ab): bandwidth on a → b when c → d is saturated. Definition of the measured interference (ab) mes (cd) ⇐ ⇒

bwcd(ab) bw(ab)

< 0.7 ; (ab) mes (cd) if ratio > 0.9

Not symmetric:

10 Mo/s 100 Mo/s 100 Mo/s a c b d

(ab) mes (cd) and (cd) mes (ab).

Definition of the “real” interference (to reintroduce symmetry) (ab) rl (cd) ⇐ ⇒

• (ab)

mes (cd) (cd) mes (ab) (or) ⇐ ⇒ ¬(ab) rl (cd)

SLIDE 9

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Problem statement

Notations H : set of nodes Interference matrix I(H, rl): I(H, rl)(a,b,c,d) =

• 1

if (ab) rl (cd) else Definition InterferenceGraph: Given H and I(H, e

G), find a routed graph

G = (V , E, r) such that:      H ⊂ V ; I(H, e

G) = I(H,

G) ; |V | is minimal.

SLIDE 10

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Outline

Introduction Context Motivation and goals State of the art ALNeM Model used Measurement methodology Problem statement Mathematical tools, algorithms Total interference and separators Reconstruction trees and cliques of trees Extension for cycles Implementation Data collection Conclusion Contributions and future work

SLIDE 11

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Mathematical tools: Total interference and separators

Definition of the total interference a ⊥ b ⇐ ⇒ ∀(u, v) ∈ H, (au) rl (bv) Lemma (Separation) a ⊥ b ⇐ ⇒ ∃ρ ∈ V

• ∀z ∈ H : ρ ∈ (a −

→ z) ∩ (b − → z) . Such a ρ is said to be a separator of a and b. Theorem: ⊥ is an equivalence relation

(under some assumptions)

Moreover, ∀ equivalence class, ∃ common separator for all pair of elements. Theorem (Representativity) Let C be an equivalence class for ⊥ and ρ a separator of its elements. ∀a ∈ C, ∀b, u, v ∈ H, (a, u) rl (b, v) ⇔ (ρ, u) rl (b, v)

SLIDE 12

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Reconstructing algorithm

Handling trees and cliques of trees Equivalence class ⇒ greedy algorithm eating the leaves.

A C D E F G H I B A B C D E F G H I

SLIDE 13

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Reconstructing algorithm

Handling trees and cliques of trees Equivalence class ⇒ greedy algorithm eating the leaves.

A C D E F G H I B B D G H I F E C A

SLIDE 14

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Reconstructing algorithm

Handling trees and cliques of trees Equivalence class ⇒ greedy algorithm eating the leaves.

A C D E F G H I B B D G H I F E C A

SLIDE 15

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Reconstructing algorithm

Handling trees and cliques of trees Equivalence class ⇒ greedy algorithm eating the leaves.

A C D E F G H I B D G B H I F E C A

SLIDE 16

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Reconstructing algorithm

Handling trees and cliques of trees Equivalence class ⇒ greedy algorithm eating the leaves.

A C D E F G H I B D G B H I F E C A

Theorem: When |Cinf| = 1, the graph built is a solution. Theorem: If a tree being a solution exists, |Cinf| = 1. Remark: The graph built is optimal (wrt |V | since V = H)

SLIDE 17

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Reconstructing algorithm

Handling trees and cliques of trees Equivalence class ⇒ greedy algorithm eating the leaves.

A C D E F G H I B D G B H I F E C A

Theorem: When |Cinf| = 1, the graph built is a solution. Theorem: If a tree being a solution exists, |Cinf| = 1. Remark: The graph built is optimal (wrt |V | since V = H) Theorem: When no interferences in I, clique of Ci is valid solution Remark: It is also optimal

SLIDE 18

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Reconstructing algorithm: Extension for cycles

Idea Find a and b close to each other on a cycle; cut the cycle in between; iterate previous algorithm; reintroduce the cycle: reconnect (a, b).

a b

Finding out how to cut a, b: nodes with the most interferences (i.e., maximizing {u, v : au

bv})

SLIDE 19

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Reconstructing algorithm: Extension for cycles

Idea Find a and b close to each other on a cycle; cut the cycle in between; iterate previous algorithm; reintroduce the cycle: reconnect (a, b).

a b

β α

Finding out how to cut a, b: nodes with the most interferences (i.e., maximizing {u, v : au

bv})

SLIDE 20

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Reconstructing algorithm: Extension for cycles

Idea Find a and b close to each other on a cycle; cut the cycle in between; iterate previous algorithm; reintroduce the cycle: reconnect (a, b).

a b

I1 I2 I3

Finding out how to cut a, b: nodes with the most interferences (i.e., maximizing {u, v : au

bv}) 8 > > > < > > > : I1 = ˘ u ∈ Ci : a ∈ (b − → u) and b ∈ (a − → u) ¯ I2 = ˘ u ∈ Ci : a ∈ (b − → u) and b ∈ (a − → u) ¯ I3 = ˘ u ∈ Ci : a ∈ (b − → u) and b ∈ (a − → u) ¯ I4 = ˘ u ∈ Ci : a ∈ (b − → u) and b ∈ (a − → u) ¯

I4 = {a; b} or

a b u

SLIDE 21

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Reconstructing algorithm: Extension for cycles

Idea Find a and b close to each other on a cycle; cut the cycle in between; iterate previous algorithm; reintroduce the cycle: reconnect (a, b).

u v v u a b

I1 I2 I3

Finding out how to cut a, b: nodes with the most interferences (i.e., maximizing {u, v : au

bv}) 8 > > > < > > > : I1 = ˘ u ∈ Ci : a ∈ (b − → u) and b ∈ (a − → u) ¯ I2 = ˘ u ∈ Ci : a ∈ (b − → u) and b ∈ (a − → u) ¯ I3 = ˘ u ∈ Ci : a ∈ (b − → u) and b ∈ (a − → u) ¯ I4 = ˘ u ∈ Ci : a ∈ (b − → u) and b ∈ (a − → u) ¯

b α β a b α β a

1 1

} } }

u

1 1 1

v

0\1

I2 I3 I1

SLIDE 22

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Reconstructing algorithm: Extension for cycles

Idea Find a and b close to each other on a cycle; cut the cycle in between; iterate previous algorithm; reintroduce the cycle: reconnect (a, b).

u v v u a b

I1 I2 I3

Finding out how to cut a, b: nodes with the most interferences (i.e., maximizing {u, v : au

bv}) 8 > > > < > > > : I1 = ˘ u ∈ Ci : a ∈ (b − → u) and b ∈ (a − → u) ¯ I2 = ˘ u ∈ Ci : a ∈ (b − → u) and b ∈ (a − → u) ¯ I3 = ˘ u ∈ Ci : a ∈ (b − → u) and b ∈ (a − → u) ¯ I4 = ˘ u ∈ Ci : a ∈ (b − → u) and b ∈ (a − → u) ¯

b α β a b α β a

1 1

} } }

u

1 1 1

v

0\1

I2 I3 I1

Topological sort on the graph associated to the matrix slice gives I1, I2, I3

SLIDE 23

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Reconstructing algorithm: Extension for cycles

Idea Find a and b close to each other on a cycle; cut the cycle in between; iterate previous algorithm; reintroduce the cycle: reconnect (a, b).

a b

I1 I2 I3

Finding out how to cut . . . How to connect parts afterward

SLIDE 24

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Reconstructing algorithm: Extension for cycles

Idea Find a and b close to each other on a cycle; cut the cycle in between; iterate previous algorithm; reintroduce the cycle: reconnect (a, b).

• ✁

✁ ✁ ✁ ✁ ✁ ✂ ✂ ✂ ✂ ✂ ✂ ✄ ✄ ✄ ✄ ✄ ✄ ☎ ☎ ☎ ☎ ☎ ☎ ☎ ☎ ☎ ✆ ✆ ✆ ✆ ✆ ✆ ✝ ✝ ✝ ✝ ✝ ✝ ✞ ✞ ✞ ✞ ✞ ✞

a b

I1 I2 I3

Finding out how to cut . . . How to connect parts afterward First step on I1 → Finds 2 classes I1a and I1α; a ∈ I1a. First step on I3 → Finds 2 classes I1b and I1β; b ∈ I1b.

SLIDE 25

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Reconstructing algorithm: Extension for cycles

Idea Find a and b close to each other on a cycle; cut the cycle in between; iterate previous algorithm; reintroduce the cycle: reconnect (a, b).

• ✁

✁ ✁ ✁ ✁ ✁ ✂ ✂ ✂ ✂ ✂ ✂ ✄ ✄ ✄ ✄ ✄ ✄ ☎ ☎ ☎ ☎ ☎ ☎ ☎ ☎ ☎ ✆ ✆ ✆ ✆ ✆ ✆ ✝ ✝ ✝ ✝ ✝ ✝ ✞ ✞ ✞ ✞ ✞ ✞

a b

I1 I2 I3

Finding out how to cut . . . How to connect parts afterward First step on I1 → Finds 2 classes I1a and I1α; a ∈ I1a. First step on I3 → Finds 2 classes I1b and I1β; b ∈ I1b. Reconnect I1a and I1b ; Reconnect I1α and I1β.

SLIDE 26

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Reconstructing algorithm: Extension for cycles

Idea Find a and b close to each other on a cycle; cut the cycle in between; iterate previous algorithm; reintroduce the cycle: reconnect (a, b).

• ✁

✁ ✁ ✁ ✁ ✁ ✂ ✂ ✂ ✂ ✂ ✂ ✄ ✄ ✄ ✄ ✄ ✄ ☎ ☎ ☎ ☎ ☎ ☎ ☎ ☎ ☎ ✆ ✆ ✆ ✆ ✆ ✆ ✝ ✝ ✝ ✝ ✝ ✝ ✞ ✞ ✞ ✞ ✞ ✞

a b

I1 I2 I3

Finding out how to cut . . . How to connect parts afterward First step on I1 → Finds 2 classes I1a and I1α; a ∈ I1a. First step on I3 → Finds 2 classes I1b and I1β; b ∈ I1b. Reconnect I1a and I1b ; Reconnect I1α and I1β. No demonstration of this...

SLIDE 27

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Example of reconstruction

32 42 5 6 1 8 10 16

100

101

102

103

104

20 105 106

107 108

109 22 11 12 14 19 110

111

112

113

114 39

115 116 117

118 119

120

121

122 123

124 34

125

126

127 128 129

36 13 15 130 131

132

133 134 31

135

136

137

138

139

46 60

140

141

142

143 144 40

145

146 147 148

149

4 7

150 151

152 153 154 47 155

156 157

158 159 44 18 75

160

161

162 163

164 58 17 80 170 171 172 173 174 52 175 176 177

178

179 59

65

70 180

181

182 183

184

53 2 27 3 51 21 25 28 85

95

23 24 29 26

90

30 35 33 37 38 9 45 41 50 55 56 57 61 62 63 64 66 67 68 69 71 72 73 74 76

77

78 79

81 82 83 84

86

87 88 89 91

92

93

94 96

97

98 99

SLIDE 28

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Example of reconstruction

SLIDE 29

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Example of reconstruction

SLIDE 30

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Example of reconstruction

SLIDE 31

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Example of reconstruction

SLIDE 32

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Example of reconstruction

SLIDE 33

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Example of reconstruction

SLIDE 34

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Example of reconstruction

SLIDE 35

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Example of reconstruction

SLIDE 36

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Example of reconstruction

SLIDE 37

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Example of reconstruction

SLIDE 38

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Example of reconstruction

SLIDE 39

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Example of reconstruction

SLIDE 40

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Example of reconstruction

SLIDE 41

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Outline

Introduction Context Motivation and goals State of the art ALNeM Model used Measurement methodology Problem statement Mathematical tools, algorithms Total interference and separators Reconstruction trees and cliques of trees Extension for cycles Implementation Data collection Conclusion Contributions and future work

SLIDE 42

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Data collection

Intuitive algorithm

1 Measure the bandwidth on (ab) ; 2 Measure the bandwidth on (ab) when the link (cd) is saturated ; 3 Compute the ratio.

N4, 30s. per step ⇒ 50 days for 20 hosts. Speeding things up Using traceroute or other tomography

• Independent tests in parallel
• Validation of information sets

Refinement of existing graph?

SLIDE 43

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Outline

Introduction Context Motivation and goals State of the art ALNeM Model used Measurement methodology Problem statement Mathematical tools, algorithms Total interference and separators Reconstruction trees and cliques of trees Extension for cycles Implementation Data collection Conclusion Contributions and future work

SLIDE 44

Introduction ALNeM Mathematical tools, algorithms Implementation Conclusion

Conclusion

Contributions Retrieve the interference-based topology from direct measurements Strong mathemathical basements (optimal for cliques of trees) More generic than ENV (partial cycle handling) Based on GRAS (development of distributed applications on simulator) Future work NP-hardness Experimentation on real platform (measurements optimization) Iterative algorithm (modification detection) Couple measurement and reconstruction phases Integration within the NWS (auto-configuration; provide information)