On On l low-la latenc ency-ca capable to topologies, and their - - PowerPoint PPT Presentation

On On l low-la latenc ency-ca capable to topologies, and their impact on th the desi sign of

• f intr

tra-do domain ain ro routing

Nikola Gvozdiev, Stefano Vissicchio, Brad Karp, Mark Handley University College London (UCL)

We want low latency!

We want low latency! In the datacenter

We want low latency! In the datacenter In the enterprise WAN [B4, BwE, SWAN …]

• perator controls both WAN and sources
• … so demands are predictable

We want low latency! In the datacenter In the enterprise WAN [B4, BwE, SWAN …]

• perator controls both WAN and sources
• … so demands are predictable

In the ISP this talk

• ISP operator does not control sources

How do we get low latency in a loaded ISP network?

How do we get low latency in a loaded ISP network?

The topology?

Topology must offer diverse low- latency paths…

How do we get low latency in a loaded ISP network?

The topology? The routing?

…and routing system must make good use of those low-latency paths Topology must offer diverse low- latency paths…

How do we get low latency in a loaded ISP network?

The topology? The routing?

…and routing system must make good use of those low-latency paths Topology must offer diverse low- latency paths…

All possible topologies connecting a set of PoPs

Venn diagram

Other topologies SP routing gives lowest possible latency

Shortest-path routing doesn’t yield lowest latency on all topologies

Other topologies

SP routing gives lowest possible latency

Shortest-path routing doesn’t yield lowest latency on all topologies

Other topologies

SP routing gives lowest possible latency

Shortest-path routing doesn’t yield lowest latency on all topologies

capacity: 1.5

Other topologies

SP routing gives lowest possible latency

demands: 1 each

Shortest-path routing doesn’t yield lowest latency on all topologies

capacity: 1.5

Other topologies

SP routing gives lowest possible latency

demands: 1 each

Shortest-path routing doesn’t yield lowest latency on all topologies

capacity: 1.5

Other topologies

SP routing gives lowest possible latency

Can’t achieve lower latency on a tree!

demands: 1 each

Shortest-path routing doesn’t yield lowest latency on all topologies

capacity: 1.5

Let’s improve the topology, let’s add redundancy!

demands: 1 each

Better Topologies May Need Better Routing

capacity: 1.5

Congestion inflates latency if both aggregates don’t fit.

demands: 1 each

Better Topologies May Need Better Routing

capacity: 1.5

Congestion control makes aggregates fit; hurts throughput

demands: 1 each

Better Topologies May Need Better Routing

capacity: 1.5

Other topologies

SP routing gives lowest possible latency

demands: 1 each

Better Topologies May Need Better Routing

capacity: 1.5

Other topologies

SP routing gives lowest possible latency

Better Topologies May Need Better Routing

Other topologies

Better Topologies May Need Better Routing

Modern TE (B4, MPLS-TE) give lowest possible latency

Other topologies

Modern TE (B4, MPLS-TE) give lowest possible latency

demands: 1 each

Better Topologies May Need Better Routing

capacity: 1.5

Other topologies Route as much as possible on shortest paths

demands: 1 each

Better Topologies May Need Better Routing

Modern TE (B4, MPLS-TE) give lowest possible latency

capacity: 1.5

Other topologies Traffic is split, low-latency top path fully loaded

demands: 1 each

Better Topologies May Need Better Routing

Modern TE (B4, MPLS-TE) give lowest possible latency

capacity: 1.5

Other topologies

• Do any topologies fall in

this region?

Do Even Better Topologies Need Even Better Routing?

Modern TE (B4, MPLS-TE) give lowest possible latency

Other topologies

• Do any topologies fall in

this region?

• If so, do any of them have

a greater potential to provide low latency?

Modern TE (B4, MPLS-TE) give lowest possible latency

Do Even Better Topologies Need Even Better Routing?

Other topologies

• Do any topologies fall in

this region?

• If so, do any of them have

a greater potential to provide low latency?

• Why does current routing

do poorly on those topologies?

Modern TE (B4, MPLS-TE) give lowest possible latency

Do Even Better Topologies Need Even Better Routing?

Limitations of Today’s Routing

Central European network of GTS, 2010

Proof by example. Consider this real-world ISP topology…

0.0 0.2 0.4 0.6 Fraction of flows that encounter congestion 0.0 0.2 0.4 0.6 0.8 1.0 CDF Shortest path

CDF over 100 runs of SP routing on synthetic traffic matrices on GTS’s topology

0.0 0.2 0.4 0.6 Fraction of flows that encounter congestion 0.0 0.2 0.4 0.6 0.8 1.0 CDF Shortest path

Fraction of all flows in each traffic matrix that cross at least one congested link

SP does poorly, as expected

0.0 0.2 0.4 0.6 Fraction of flows that encounter congestion 0.0 0.2 0.4 0.6 0.8 1.0 CDF Shortest path

Each point is a run of the routing system on a different traffic matrix

0.0 0.2 0.4 0.6 Fraction of flows that encounter congestion 0.0 0.2 0.4 0.6 0.8 1.0 CDF Shortest path B4

B4 for the win … sort of

Jain, Sushant, et al. "B4: Experience with a globally-deployed software defined WAN." ACM SIGCOMM 2013

Where does greedy routing such as B4 go wrong?

You are here!

Let’s focus on a small part of the network

Where does greedy routing such as B4 go wrong?

Limitations of greedy routing

Limitations of greedy routing

1.Allocate as much as possible on shortest path

Limitations of greedy routing

V G

1.Allocate as much as possible on shortest path

Local flows between Veszprem (V) and Gyor (G)

Limitations of greedy routing

Local flows between Veszprem (V) and Gyor (G)

V G

Through flows

1.Allocate as much as possible on shortest path

Limitations of greedy routing

First link on V->G aggregate’s shortest path fills eastbound

V G

1.Allocate as much as possible on shortest path

Limitations of greedy routing

First link on V->G aggregate’s shortest path fills eastbound

V G

1.Allocate as much as possible on shortest path 2.Allocate to longer paths

Limitations of greedy routing

First link on V->G aggregate’s shortest path fills eastbound

V G

?

1.Allocate as much as possible on shortest path 2.Allocate to longer paths

Limitations of greedy routing

First link on V->G aggregate’s shortest path fills eastbound

V G

V->G’s second best path is already full, using it results in congestion!

?

1.Allocate as much as possible on shortest path 2.Allocate to longer paths

Limitations of greedy routing

Limitations of greedy routing

Rings embedded in a topology can trigger this problem with greedy routing

0.0 0.2 0.4 0.6 Fraction of flows that encounter congestion 0.0 0.2 0.4 0.6 0.8 1.0 CDF Shortest path B4

B4 for the win … sort of

Can we do better?

0.0 0.2 0.4 0.6 Fraction of flows that encounter congestion 0.0 0.2 0.4 0.6 0.8 1.0 CDF Shortest path B4 Latency-optimal

Yes, a placement which both avoids congestion and minimizes propagation delay does exist!

Can we do better?

0.0 0.2 0.4 0.6 Fraction of flows that encounter congestion 0.0 0.2 0.4 0.6 0.8 1.0 CDF Shortest path B4 Latency-optimal

Yes, a placement which both avoids congestion and minimizes propagation delay does exist! So GTS is amenable to low latency. Are other topologies?

How might we quantify a topology’s potential for low latency under load?

• Want a metric to capture a topology’s inherent potential

for low latency

• Should be:
• traffic matrix-agnostic
• routing algorithm-agnostic

How might we quantify a topology’s potential for low latency under load?

• Want a metric to capture a topology’s inherent potential

for low latency

• Should be:
• traffic matrix-agnostic
• routing algorithm-agnostic
• Want to capture two things:
• topology’s potential for routing around congestion hot spots
• …without incurring long propagation delay

How might we quantify a topology’s potential for low latency under load?

• Want a metric to capture a topology’s inherent potential

for low latency

• Should be:
• traffic matrix-agnostic
• routing algorithm-agnostic
• Want to capture two things:
• topology’s potential for routing around congestion hot spots
• …without incurring long propagation delay

We want a metric that rewards alternate paths with short propagation delay

Alternate Path Availability (APA)

Y Gbps

Alternate Path Availability (APA)

Y Gbps

Shortest path: T ms total propagation delay Y Gbps SP capacity

Alternate Path Availability (APA)

Y Gbps

Shortest path: T ms total propagation delay Y Gbps SP capacity Exclude each link on the shortest path; can we route Y Gbps over

• ne or more alternative paths with delay < 1.4 T?

Alternate Path Availability (APA)

Y Gbps

Shortest path: T ms total propagation delay Y Gbps SP capacity Exclude each link on the shortest path; can we route Y Gbps over

• ne or more alternative paths with delay < 1.4 T?

Alternate Path Availability (APA)

Y Gbps

Shortest path: T ms total propagation delay Y Gbps SP capacity Exclude each link on the shortest path; can we route Y Gbps over

• ne or more alternative paths with delay < 1.4 T?

Links with viable alternative paths Links on shortest path

1 5

Alternate Path Availability (APA)

Y Gbps

Shortest path: T ms total propagation delay Y Gbps SP capacity Exclude each link on the shortest path; can we route Y Gbps over

• ne or more alternative paths with delay < 1.4 T?

1 5

Alternate Path Availability (APA)

Y Gbps

Shortest path: T ms total propagation delay Y Gbps SP capacity Exclude each link on the shortest path; can we route Y Gbps over

• ne or more alternative paths with delay < 1.4 T?

2 5

Alternate Path Availability (APA)

Y Gbps

Shortest path: T ms total propagation delay Y Gbps SP capacity Exclude each link on the shortest path; can we route Y Gbps over

• ne or more alternative paths with delay < 1.4 T?

3 5

Alternate Path Availability (APA)

Y Gbps

Shortest path: T ms total propagation delay Y Gbps SP capacity Exclude each link on the shortest path; can we route Y Gbps over

• ne or more alternative paths with delay < 1.4 T?

3 5

Path too long or not enough capacity

Alternate Path Availability (APA)

Y Gbps

Shortest path: T ms total propagation delay Y Gbps SP capacity Exclude each link on the shortest path; can we route Y Gbps over

• ne or more alternative paths with delay < 1.4 T?

4 5

Alternate Path Availability (APA)

Y Gbps

Shortest path: T ms total propagation delay Y Gbps SP capacity Exclude each link on the shortest path; can we route Y Gbps over

• ne or more alternative paths with delay < 1.4 T?

4 5 = 0.8

For this PoP pair 80% of the links on the SP have an alternate path with acceptable low latency

Low-latency path diversity (LLPD)

• 1. Compute APA for all PoP pairs

Low-latency path diversity (LLPD)

• 2. Compute LLPD =
• 1. Compute APA for all PoP pairs

Fraction of PoP pairs with “good” path availability

Low-latency path diversity (LLPD)

number of PoP pairs with APA ≥ 0.7 total number of PoP pairs

• 2. Compute LLPD =
• 1. Compute APA for all PoP pairs

Fraction of PoP pairs with “good” path availability =

Low-latency path diversity (LLPD)

number of PoP pairs with APA ≥ 0.7 total number of PoP pairs

• 2. Compute LLPD =
• 1. Compute APA for all PoP pairs

Fraction of PoP pairs with “good” path availability = Empirically derived; metric not sensitive to picking different values

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 0.0 0.5 fraction of pairs congested GTS 90th percentile Median

100+ real-world ISP topologies, ranked by low-latency path diversity (LLPD)

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 0.0 0.5 fraction of pairs congested GTS 90th percentile Median

Generate TMs for each topology; plot fraction of (Src,Dst) PoP pairs in each TM that crosses at least one congested link

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 0.0 0.5 fraction of pairs congested GTS 90th percentile Median

Shortest path routing congests links

Two points per topology: median TM and 90th percentile TM; line shows spread of distribution

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 0.0 0.5 fraction of pairs congested GTS 90th percentile Median

Networks with high LLPD offer lots of alternative paths à shortest path routing experiences congestion

Shortest path routing congests links

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 0.0 0.5 fraction of pairs congested GTS 90th percentile Median

Networks with high LLPD offer lots of alternative paths à shortest path routing experiences congestion

Shortest path routing congests links

No surprises here. What about B4?

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 0.0 0.5 fraction of pairs congested GTS 90th percentile Median

• Better at using

alternative paths

B4 congests networks with high potential for low latency

0.0 0.5 fraction of pairs congested GTS 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

• Better at using

alternative paths

total prop delay of all flows total prop delay if all flows routed on SP

B4 congests networks with high potential for low latency

0.0 0.5 fraction of pairs congested GTS 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

Congestion

B4 congests networks with high potential for low latency

Better Better Propagation delay

0.0 0.5 fraction of pairs congested GTS 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

• Better at using

alternative paths

• Some flows routed
• n non-shortest paths

B4 congests networks with high potential for low latency

0.0 0.5 fraction of pairs congested GTS 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

• Better at using

alternative paths

• Some flows routed
• n non-shortest paths
• Still incurs

congestion, and precisely on high-LLPD networks!

B4 congests networks with high potential for low latency

0.0 0.5 fraction of pairs congested GTS 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

• Better at using

alternative paths

• Some flows routed
• n non-shortest paths
• Still incurs

congestion, and precisely on high-LLPD networks!

B4 congests networks with high potential for low latency

Need a different routing scheme. How about one that prioritizes avoiding congestion above all else?

Minimizing utilization avoids congestion

Minimizing utilization avoids congestion

33%

• Spread traffic out to leave

spare capacity in case traffic levels increase

• A well-known technique

called MinMax

• Does not care about

propagation delay

Minimizing utilization avoids congestion

33%

• Spread traffic out to leave

spare capacity in case traffic levels increase

• A well-known technique

called MinMax

• Does not care about

propagation delay How does MinMax do?

MinMax inflates propagation delay

0.0 0.5 fraction of pairs congested 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

• Minimizes utilization,

designed to avoid congestion

0.0 0.5 fraction of pairs congested 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

• Minimizes utilization,

designed to avoid congestion

• Routes some flows on

paths with high propagation delay

MinMax inflates propagation delay

0.0 0.5 fraction of pairs congested 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

• Minimizes utilization,

designed to avoid congestion

• Routes some flows on

paths with high propagation delay One extreme of the design

• space. The other one?

MinMax inflates propagation delay

Latency-optimal placement

0.0 0.5 fraction of pairs congested 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

• Minimizes prop delay

and avoids congestion

• Maximizes utilization
• f links on low-delay

paths

Latency-optimal placement

0.0 0.5 fraction of pairs congested 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

• Minimizes prop delay

and avoids congestion

• Maximizes utilization
• f links on low-delay

paths Assume it is possible to compute this at scale, more about that later…

0.0 0.5 fraction of pairs congested 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

Two extremes of congestion-free routing

0.0 0.5 fraction of pairs congested 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

Minimize utilization (MinMax) Minimize propagation delay and avoid congestion

0.0 0.5 fraction of pairs congested 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

Two extremes of congestion-free routing

0.0 0.5 fraction of pairs congested 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

Minimize utilization (MinMax) Minimize propagation delay and avoid congestion

0.0 0.5 fraction of pairs congested 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

Two extremes of congestion-free routing

0.0 0.5 fraction of pairs congested 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

Minimize utilization (MinMax) Minimize propagation delay and avoid congestion

GTS

GTS

0.0 0.5 fraction of pairs congested 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

Two extremes of congestion-free routing

0.0 0.5 fraction of pairs congested 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

Minimize utilization (MinMax) Minimize propagation delay and avoid congestion

GTS

GTS

GTS sees 5x difference in propagation delay!

0.0 0.5 fraction of pairs congested 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

Two extremes of congestion-free routing

0.0 0.5 fraction of pairs congested 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 LLPD 1.0 1.2 1.4 latency stretch GTS 90th percentile Median

Minimize utilization (MinMax) Minimize propagation delay and avoid congestion

GTS

GTS

GTS sees 5x difference in propagation delay! Let’s focus on a single traffic matrix

0.0 0.2 0.4 0.6 0.8 1.0 link utilization 0.0 0.5 1.0 CDF Latency-optimal (mean 0.32) MinMax (mean 0.30)

Two extremes of congestion-free routing

Significant delta in prop delay, but mean utilization the same

0.0 0.2 0.4 0.6 0.8 1.0 link utilization 0.0 0.5 1.0 CDF Latency-optimal (mean 0.32) MinMax (mean 0.30)

Links on short prop-delay paths “in demand” in latency-optimal placement

Two extremes of congestion-free routing

0.6 0.7 0.8 0.9 1.0 link utilization 0.80 0.85 0.90 0.95 1.00 CDF Latency-optimal (mean 0.32) MinMax (mean 0.30)

Two extremes of congestion-free routing

0.6 0.7 0.8 0.9 1.0 link utilization 0.80 0.85 0.90 0.95 1.00 CDF Latency-optimal (mean 0.32) MinMax (mean 0.30)

Two extremes of congestion-free routing

All possible congestion-free routing solutions lie in this range

0.6 0.7 0.8 0.9 1.0 link utilization 0.80 0.85 0.90 0.95 1.00 CDF Latency-optimal (mean 0.32) MinMax (mean 0.30)

Two extremes of congestion-free routing

100% utilization? On an ISP?

10 20 30 40 50 60 time (s) +3.2e3 1 2 3 4 Gbps

A minute from a core link

Source: CAIDA

10 20 30 40 50 60 time (s) +3.2e3 1 2 3 4 Gbps

Mean rate. Could run traffic through a path with this capacity, but long queuing delay.

A minute from a core link

10 20 30 40 50 60 time (s) +3.2e3 1 2 3 4 Gbps

Need to allocate headroom to allow for variability

A minute from a core link

0.6 0.7 0.8 0.9 1.0 link utilization 0.80 0.85 0.90 0.95 1.00 CDF Latency-optimal (mean 0.32) MinMax (mean 0.30)

The headroom dial

Not feasible because of variability

0.6 0.7 0.8 0.9 1.0 link utilization 0.80 0.85 0.90 0.95 1.00 CDF Latency-optimal (mean 0.32) MinMax (mean 0.30)

The headroom dial

MinMax is one extreme of the headroom dial

0.6 0.7 0.8 0.9 1.0 link utilization 0.80 0.85 0.90 0.95 1.00 CDF Latency-optimal (mean 0.32) MinMax (mean 0.30) 7% headroom

The headroom dial

0.6 0.7 0.8 0.9 1.0 link utilization 0.80 0.85 0.90 0.95 1.00 CDF Latency-optimal (mean 0.32) MinMax (mean 0.30) 27% headroom

0.6 0.7 0.8 0.9 1.0 link utilization 0.80 0.85 0.90 0.95 1.00 CDF Latency-optimal (mean 0.32) MinMax (mean 0.30) 7% headroom

The headroom dial

0.6 0.7 0.8 0.9 1.0 link utilization 0.80 0.85 0.90 0.95 1.00 CDF Latency-optimal (mean 0.32) MinMax (mean 0.30) 22% headroom

0.6 0.7 0.8 0.9 1.0 link utilization 0.80 0.85 0.90 0.95 1.00 CDF Latency-optimal (mean 0.32) MinMax (mean 0.30) 7% headroom

The headroom dial

0.6 0.7 0.8 0.9 1.0 link utilization 0.80 0.85 0.90 0.95 1.00 CDF Latency-optimal (mean 0.32) MinMax (mean 0.30) 15% headroom

0.6 0.7 0.8 0.9 1.0 link utilization 0.80 0.85 0.90 0.95 1.00 CDF Latency-optimal (mean 0.32) MinMax (mean 0.30) 7% headroom

The headroom dial

0.6 0.7 0.8 0.9 1.0 link utilization 0.80 0.85 0.90 0.95 1.00 CDF Latency-optimal (mean 0.32) MinMax (mean 0.30) 7% headroom

0.6 0.7 0.8 0.9 1.0 link utilization 0.80 0.85 0.90 0.95 1.00 CDF Latency-optimal (mean 0.32) MinMax (mean 0.30) 2% headroom

The headroom dial

Need to allow the minimal amount

• f headroom to cope with variability

Towards a low-latency routing system

Towards a low-latency routing system

Compute latency-optimal routing solution, sans headroom

• expressed the problem as one big linear program

(largely straightforward)

• efficient iterative solution: add paths, solve, repeat …
• 400+ nodes, less than one second (vs. tens of minutes…)

Towards a low-latency routing system

Compute latency-optimal routing solution, sans headroom

• expressed the problem as one big linear program

(largely straightforward)

• efficient iterative solution: add paths, solve, repeat …
• 400+ nodes, less than one second (vs. tens of minutes…)

Tune headroom dial to drive routing as close as possible to optimal solution while avoiding congestion

• predict how aggregates will statistically multiplex on a path

by convolving their past demands

Towards a low-latency routing system

Compute latency-optimal routing solution, sans headroom

• expressed the problem as one big linear program

(largely straightforward)

• efficient iterative solution: add paths, solve, repeat …
• 400+ nodes, less than one second (vs. tens of minutes…)

Tune headroom dial to drive routing as close as possible to optimal solution while avoiding congestion

• predict how aggregates will statistically multiplex on a path

by convolving their past demands

More details in the paper!

Are high-LLPD networks viable?

• A routing system may not be able to unlock the low-

latency potential of a topology

• LLPD indicates that a topology has good potential for

low latency

Are high-LLPD networks viable?

• A routing system may not be able to unlock the low-

latency potential of a topology

• LLPD indicates that a topology has good potential for

low latency

• But will anyone ever really build a modern WAN with

high LLPD?

Are high-LLPD networks viable?

0.0 0.2 0.4 0.6 0.8 LLPD 0.0 0.5 fraction of pairs congested Google 90th percentile Median

• Repeated SP experiment, but added Google’s network

Are high-LLPD networks viable?

0.0 0.2 0.4 0.6 0.8 LLPD 0.0 0.5 fraction of pairs congested Google 90th percentile Median

• Repeated SP experiment, but added Google’s network

Modern high-LLPD network!

Are high-LLPD networks viable?

0.0 0.2 0.4 0.6 0.8 LLPD 0.0 0.5 fraction of pairs congested Google 90th percentile Median

• Repeated SP experiment, but added Google’s network

Modern high-LLPD network! B4 however, does great on that network! Could it be because the routing and the topology co-evolved?

What topologies would people build if they knew the routing system would always extract the best from it?

Conclusions

• To achieve low latency:
• topology must provide low-latency paths
• the routing system must use them effectively
• State-of-the-art routing falters on high-LLPD topologies—

precisely those with best potential for low latency

• Practical routing approach for high-LLPD topologies:
• Efficient LP solution for optimal traffic placement
• Tune headroom dial to avoid congestion (but as little

toward MinMax as possible)

System Design

• Simple, centralized design

System Design

• Simple, centralized design

But measure what?

Measurements

• Only need measurements per aggregate, not per flow!
• Need to know enough to figure out both long and

short-term variability for each aggregate

• Sampling traffic level 10 times per second is enough

to capture short-term variability due to TCP’s congestion control…

• …since RTTs in the ISP are long (order of 100ms)
• Sampling 10 times per second well within reach of

recent hardware [DevoFlow SIGCOMM 2011]

What about prioritization?

• If you can you should definitely prioritize delay-

sensitive traffic

• but identifying this traffic in the ISP setting may not

be trivial, since no single operator controls all sources

• also, what about bandwidth-hungry low-latency

traffic (e.g., VR)