The Demand of Bulk Transfers over WAN More demanding 1.Transfer - - PowerPoint PPT Presentation

▶

May 18, 2023 339 likes •675 views

The Demand of Bulk Transfers over WAN More demanding 1.Transfer large size 2.Minimize completion time More willing to 1.Provide demand information 2.Control its transfers 1 Software-Defined Networking (SDN) in WAN Global traffic engineering

SLIDE 1

The Demand of Bulk Transfers over WAN

1

More willing to

1.Provide demand information 2.Control its transfers

More demanding

1.Transfer large size 2.Minimize completion time

SLIDE 2

Software-Defined Networking (SDN) in WAN

Controller

Global traffic engineering with centralized control, e.g., Google B4, Microsoft SWAN

Given Traffic Demand

Network Topology

Compute Routing Rate Allocation Optimize Completion Time Deadlines Met

Traffic Engineering Daily Timescales Central Control Minutes

2

SLIDE 3

Network Layer over Optical Layer

Network Layer Optical Layer

Network Link Optical Circuit Seattle Los Angeles Seattle Los Angeles Network switch (Router) Optical switch

Manual Config. Monthly Timescales Automated Config. Minutes

3

SLIDE 4

Technology Trends

Bulk-transfer applications with demand information
Fast centralized control with SDN
Fast reconfigurable optics

4

SLIDE 5

Reconfigure Optical Layer to Change Network-Layer Topology

R0 O0 R2 R3 R1 O1 O3 O2

Optical Layer

Router Optical Switch R0 R2 R3 R1

10 10 10 10

Network Layer Configuration A

5

SLIDE 6

Reconfigure Optical Layer to Change Network-Layer Topology

R0 O0 R2 R3 R1 O1 O3 O2 R0 R2 R3 R1

10 10 10 10

Network Layer Optical Layer

Router Optical Switch

Configuration A

R0 O0 R2 R3 R1 O1 O3 O2 R0 R2 R3 R1

10 10 10 10

Configuration B

6

SLIDE 7

Reduce Average Transfer Completion Time

R0 R2 R3 R1

10 10 10 10

F0(Demand=10) F1(Demand=10) Routing R0 R2 R3 R1

10 10 10 10

F0(Demand=10) R0 R2 R3 R1

10 10 10 10

F1(Demand=10) Routing + Rate allocation Step 1 Step 2 F0(Demand=10) F1(Demand=10) R0 R2 R3 R1

10 10 10 10

Routing + Rate + Topology

Unused Capacity Inefficiently Used Capacity

7

SLIDE 8

Joint Optimization and Challenges

8

SLIDE 9

Joint Optimization

Constraints # of Router Ports Optical Reach

# of Regenerators

# of Wavelengths Link Capacity # of Router Ports Optical Reach

O0 O1 O2 Wide Area Network Client DC Router Optical Switch C0 C1 C2 9

R1 R0 R2

CurrentTopology Optical Infrastructure

Routing Rate Allocation Optimize Completion Time Deadlines Met Given Traffic Demand Compute

NewTopology Optical Infrastructure NewTopology

SLIDE 10

Challenges

Efficient joint optimization
Routing
Rate allocation
Topology
Transition gracefully
Minimize disruption during update

10

SLIDE 11

Throughput

Finding Good Configuration with Small Change

Current 11 Good Close

Configuration Space

SLIDE 12

Simulated Annealing Algorithm

12 Current

Evaluate Neighbor

Choose Random Neighbor

SLIDE 13

Owan's Solution Overview

13 Evaluate Neighbor

Random Neighbor Topo. Optimize Network Layer Choose Random Neighbor

Consistent Update

Joint optimization
Avoids disruption

efficiently

SLIDE 14

14

Owan Algorithm

SLIDE 15

Random Neighbor Topology

15 Evaluate Neighbor

Random Neighbor Topo. Optimize Network Layer

Consistent Update

1. Make random local change
2. Select optical circuits

SLIDE 16

Random Neighbor Topology

Make random local change
Minimize changes to the network
Satisfy the port number constraints
Select optical circuits for each link
Use graph algorithm
For each path
Minimize regenerator usage
Balance regenerator usage across sites

16

Current Topology

R0 R2 R3 R1

10 10 10 10 New Topology

R0 R2 R3 R1

20 20 Random Local Change

R0 R2 R3 R1

10+10 10+10 10-10 10-10

SLIDE 17

Optimize Network Layer

17 Evaluate Neighbor

Random Neighbor Topo. Optimize Network Layer

Consistent Update

1. Routing
2. Rate allocation

SLIDE 18

Order transfers with classic scheduling disciplines
Prioritize short paths in rate allocation

Schedule Transfers on the New Topology

Avg.Transfer CompletionTime

SJF EDF ……

# of Deadlines Met Other Objective

18

SLIDE 19

Evaluate Neighbor Topology

19 Evaluate Neighbor

Random Neighbor Topo. Optimize Network Layer

Consistent Update

Throughput: sum of rates

SLIDE 20

Consistent Update

20 Evaluate Neighbor

Random Neighbor Topo. Optimize Network Layer

Consistent Update

Dependencies of operations

SLIDE 21

21

Implementation and Evaluation

SLIDE 22

Testbed Implementation

9 Sites
Emulating Internet2 network
135 servers
Two 6-core Intel E5-2620v2
10GE

ROADM Arista Switch Servers

One Site

22

SLIDE 23

Workload
Generate transfers for 2 hours
Draw transfer size from exponential distribution
Mean 500GB/5TB for testbed/simulation
Evaluation
Testbed experiments, with 9 sites
Large-scale simulations, with about 40 sites
Results
Average transfer completion time: 3.5-4.4x
Number of transfers that meet deadlines: 1.1-1.3x

Evaluation

23

SLIDE 24

Deadline-Unconstrained Traffic

Performance metric
Transfer completion time
Other approaches
MaxFlow
MaxMinFract
SWAN[1]

24

[1] Hong, Chi-Yao, et al., Achieving High Utilization with Software-Driven WAN, SIGCOMM 2013

SLIDE 25

Better Average Completion Time

4.45x

Better

25

SLIDE 26

Deadline-Constrained Traffic

Performance metric
Percentage of transfers that meet deadlines
Amount of bytes that finish before deadlines
Other approaches
Deadline-unconstrained approaches
Amoeba[1]

26

[1] Zhang, Hong, et al., Guaranteeing deadlines for inter-datacenter transfers, EuroSys 2015

SLIDE 27

More Transfers Meet Deadlines

27

1.36x

Better

SLIDE 28

Consistent Update Avoids Disruptions

28 Topology Update

SLIDE 29

Conclusions

Optical control improves WAN performance
Efficient algorithms for joint optimization
Transition gracefully

29

SLIDE 30

Thanks! Q&A

SLIDE 31

Build Optical Circuits for Each Link

Build regenerator graph
Balance regenerator consumption

O0 O1 O2 O3 O4 0.2 0.25 1

Distance <= Optical Reach Inverse of # Regenerators

Goal: Find path with min total node weight Shortest path problem

n directed graph

31