Load-Optimization in Reconfigurable Networks: Algorithms and - PowerPoint PPT Presentation

Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing Wenkai Dai, Klaus-T. Foerster, David Fuchssteiner, Stefan Schmid (CT Group, University of Vienna)

Motivation: Interconnecting Top of Rack in Datacenter 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 2

Fat-Tree (Clos) Topology for Data Centers • Fat-Tree is good for all-to-all traffic 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 3

“ Data reveal that 46-99% of the rack Data Center Traffic ≠ Uniform pairs exchange no traffic at all ” • However, DCN traffic is often not all-to-all Heatmap of rack to rack traffic. Color intensity is log- scale and normalized. Ghobadi et al., SIGCOMM’16 Traffic demands (normalized) between ToR switches. Halperin et al., SIGCOMM’11 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 4

Circuit Switches vs Packet Switches 1. Circuit Switches: usually optical ◦ Fast (high bandwidth) ◦ Connection between ports can be adjusted dynamically https://www.laserfocusworld.com/optics/article/16556781/ma ny-approaches-taken-for-alloptical-switching (Hecht, 2001) 2. Packet Switches: usually electronic ◦ Low bandwidth ◦ The connections of links are fixed after deployment 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 5

Understand Circuit Switches Physical layer: It‘s a Match( ing)! B Reconfigurable Switch • Idea: implement “physical” connections Difference: Not all-to-all switch – E.g. just 1 connection per node • C A • A matching is selected to connect nodes D (b) (a) (c) B B B C C A A C A D D D 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 6

Hybrid Architecture for Datacenter (Helios, Farrington et al., SIGCOMM ‘10) Helios • Adjust the topology dynamically for variant demands: ◦ Elephant (big) flows → Circuit Switches ◦ Mice (small) flows → Packet Switches 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 7

Reconfigurable Data Center Networks (DCNs) Helios (core) c-Through ( HyPaC architecture) ProjecToR interconnect Farrington et al., SIGCOMM ‘10 Wang et al., SIGCOMM ‘10 Ghobadi et al., SIGCOMM ‘16 Rotornet (rotor switches) Solstice (architecture & scheduling) REACToR FireFly Mellette et al., SIGCOMM ‘17 Liu et al., CoNEXT ‘15 Liu et al., NSDI ‘15 Hamedazimi et al., SIGCOMM ‘14 … and many more … 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 8

Routing Models: Unsplittable vs Splittable • For each demand, e.g., A→E: 10 5.5 10 A C E A C E A C E 4.5 10 B D B D B D Splittable Unsplittable Unsplittable 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 9

Routing Models: Segregated vs Nonsegregated • In a reconfigurable datacenter, for each demand: E.g., demand: A→E Circuit Switch Circuit Switch Circuit Switch A C E G A C E G A C E G B D F B D F B D F Segregated Nonsegregated Segregated 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 10

Four Routing Models in Reconfigurable Networks Routing Models Segregation Model Nonsegregation Model SS SN Splittable Model US UN Unsplittable Model 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 11

Load-Optimization Reconfiguration Problem (Our Problem) • Given: A routing model 𝜐 ∈ {𝑇𝑇, 𝑇𝑂, 𝑉𝑇, 𝑉𝑂} Circuit Switches Static Network N = 𝑊, 𝐹, 𝐷 From: calient.net From: Al-Fares et al. 2008 Demands Matrix 𝐸 Set of reconfigurable links ℇ 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 12

Load-Optimization Reconfiguration Problem (Our Problem) • Compute: a matching from reconfigurable links; and optimal routing schemes for demands Circuit Switches Static Network N = 𝑊, 𝐹, 𝐷 From: cisco.com From: calient.net From: Al-Fares et al. 2008 + A routing model 𝜐 ∈ {𝑇𝑇, 𝑇𝑂, 𝑉𝑇, 𝑉𝑂} Matching 𝑁 ⊂ ℇ Optimal routing schemes for demands Set of reconfigurable links ℇ in the hybrid network 𝑊, 𝐹ڂ 𝑁 , 𝐷 • Objective: minimize the maximum link load in the hybrid network 𝑊, 𝐹ڂ 𝑁 , 𝐷 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 13

An Example For Load-Optimization Reconfiguration Problem From: Al-Fares et al. 2008 C A B E A D B D C E E A D B Reconfigurable links Reconfigurable network 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 14

Example: Loads Depend on Reconfigurations • Consider demands D: A→B: 8, A→C: 6, C→B: 6, D→B: 6, A→E: 6 • Goal: determine a matching in reconfigurable links to minimize the maximum load A A Compute flows for demands without reconfigurable links. 20 6 6 D C E D C E 20 B B Maximum load 20 Reconfigurable network 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 15

Example: Determine Matching by Greedy • Demands D : A→B: 8, A→C: 6, C→B: 6, D→B: 6, A→E: 6 ◦ Greedy chooses 𝐵, 𝐶 to serve A→B , then the matching is 𝐵, 𝐶 and 𝐸, 𝐹 A A Links: 𝐵, 𝐶 , 𝐸, 𝐹 configured 12 20 6 6 6 6 8 D C E C E D 12 20 B A→B: 8 B 0 Maximum load 20 Greedy -> maximum load 12 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 16

Example: Optimal Matching • Demands D : A→B: 8, A→C: 6, C→B: 6, D→B: 6, A→E: 6 ◦ The optimal matching is 𝐸, 𝐶 and 𝐵, 𝐹 A A Links: 𝐵, 𝐹 , 𝐸, 𝐶 configured 10 20 4 4 6 6 D C E C E D 10 20 B B Maximum load 20 Optimal -> maximum load 10 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 17

Height =2 Complexity for Simple Trees • If the given static network is a tree with a height >=2, then Time Complexity Segregation Model Nonsegregation Model Splittable Model SS is strongly NP-hard SN is strongly NP-hard Unsplittable Model US is strongly NP-hard UN is strongly NP-hard ◦ Reduction from 3-Partition problem • Especially, UN model is weakly NP-hard for star networks ◦ Reduction from 2-Partition problem ◦ Not hard anymore for small demands 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 18

Non-Blocking Interconnects, e.g., Clos, Fat-Tree etc. Non-Blocking Interconnections in above layers 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 19

Simplified Problem defined by Non-Blocking Interconnections Above layers abstracted as a packet switch. (Mohammad Alizadeh et al. 2016). 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 20

Optimal Algorithms for Simplified Problem (Notations) • Consider a decision problem A • Assume the optimized maximum load: 𝜄 20 6 6 • Let 𝑇 be the set of possible values for 𝜄 D C E • 𝑇 contains the load for each static link before reconfiguration 20 • Next, we show how to compute the set 𝑇 B The set 𝑇 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 21

Useful Observations • For each reconfigurable link 𝑌, 𝑍 , focus on its triangle. E.g., demand : B→E • E.g., the triangle 𝐵, 𝐹, 𝐷 A A A D C E D C E E D C B B B Any demand : X→Y, one node in the triangle, the other not in, then flows always go through the center 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 22

Local Optimization For Each Triangle • For each reconfigurable link 𝑌, 𝑍 , in the triangle 𝑌, 𝑍, 𝐷 : ◦ Compute local demands, and find optimal load for the local demands A A A Local demands D’ D C E D C E E C • Find optimal routing in 𝑃(1) B B • Let the maximum load be Δ 𝑗 • Put Δ 𝑗 into the set 𝑇 Local demands :D’( C→E )=D( B→E)+ D( D→E) D’( E→C )=D(E →B)+ D( E→D) 06.10.2020 Load-Optimization in Reconfigurable Networks: Algorithms and Complexity of Flow Routing (Performance 2020) Page 23