Peering Planning Cooperation w ithout Revealing Confidential I nform ation Arman Maghbouleh am at cariden dot com Apricot 2006 Perth, Australia w w w .cariden.com (c) cariden technologies Failover Matrices– Cariden 1 APRI COT 2006
The I ssue • Multi-Homed Neighbor, 2 or more links > 50% • Example – 1000Mbps connections to Peer X in 3 locations – SJC-to-Peer = 600Mbps, NYC = 100, WDC = 600 – SJC-to-Peer link fails • Are we in trouble? ... or ... ? ? SEA BOS SEA BOS CHI CHI 1 0 0 Mbps NYC NYC ( congested) WDC WDC KCY SJC KCY SJC 7 0 0 X 1 3 0 0 X 6 0 0 HST ATL HST ATL LAX LAX Peer X MIA Peer X MIA Failover Matrices– Cariden 2 APRI COT 2006
Capacity Planning Utopia • Uniform capacity links • Diverse connections (unlikely double failures at Layer 3) • Upgrade at 50% (planning objective is to be resilient to single failures) Failover Matrices– Cariden 3 APRI COT 2006
Capacity Planning Reality • Range of capacities • Multiple Layer 3 failures • Upgrade impediments (money, cable plant, ...) Failover Matrices– Cariden 4 APRI COT 2006
I GP Different from BGP • Failure behavior is predictable • Established process for within AS planning – Gather Data • Topology (OSPF, IS-IS, ...) • Traffic matrix [ 1] • Estimate growth – Simulate for failures – Perform traffic engineering (optional) [ 2] – Upgrade as necessary • Commercial and free tools [ 1] APRICOT 2005 tutorial: Best Practices for Determining the Traffic Matrix in IP Networks [ 2] APRICOT 2004 tutorial: Traffic Engineering Beyond MPLS Failover Matrices– Cariden 5 APRI COT 2006
The Trouble w ith BGP • Planning practices not well established • BGP decision process complicated • Amount of data can be large • Failure behavior often depends on someone else’s network! subject of this talk – e.g., incoming traffic from a peer Failover Matrices– Cariden 6 APRI COT 2006
BGP Path Decision Algorithm [ 1 ] 1. Reachable next hop 2. Highest Weight 3. Highest Local Preference 4. Locally originated routes 5. Shortest AS-path length 6. IGP > EGP > Incomplete Respect MEDs 7. Lowest MED 8. EBGP > IBGP Shortest Exit Routing 9. Lowest IGP cost to next hop 10. Shortest route reflection cluster list 11. Lowest BGP router ID 12. Lowest peer remote address [ 1] Junos algorithm shown here. Cisco IOS uses a slightly different algorithm. Failover Matrices– Cariden 7 APRI COT 2006
Com m on Routing Policies • Shortest Exit – Often used for sending to peers – Get packet out of network as soon as possible – Local Prefs used to determine which neighbor, IGP costs used to determine which exit • Respect MEDs – Often used for customers who buy transit – Deliver packets closest to destination – Neighbor forwards IGP costs as MEDs (multi-exit discriminators) Failover Matrices– Cariden 8 APRI COT 2006
Blind Spots • Cannot predict behavior when routing depends on other network (see 3 cases below). Relationship Routing To Routing From to Remote AS Remote AS Remote AS Shortest Exit in Shortest Exit in Peer known network unknown network Respect MEDs Shortest Exit in Customer from unknown unknown network Transit Shortest Exit in Respect our MEDs known network Provider Failover Matrices– Cariden 9 APRI COT 2006
Failover Matrices • Solution to peering planning blind spots • Procedure – Gather data • Topology, Traffic, Routing Configurations – Simulate knowable effects • Generate Failover Matrices – Share Failover Matrices for unknowables • e.g., peer gives failover matrix for traffic it delivers, we provide peer failover matrix for traffic we deliver • Both sides benefit from cooperating • AS-Internal information is kept confidential Failover Matrices– Cariden 1 0 APRI COT 2006
Failover Matrix Exam ple Traffic: % Traffic: % Traffic: % Traffic: Node: Interface no failure fail_SJC fail_nyc fail_wdc ar1.sjc:Gig3/2 600 - 10% (610) 1% (606) ar1.nyc:ge-2/1 100 48% (388) - 95% (670) ar2.wdc:ge-2/2 600 52% (912) 70% (670) - Note: 388Mbps= 100Mbps+ (0.48* 600Mbps), 912= 600+ (0.52* 600), ... Failover Matrices– Cariden 1 1 APRI COT 2006
Failover Exam ple ( from real netw ork) Peer Circuit 1: Traffic levels at five minute intervals Peer Circuit 2: Traffic levels at five minute intervals Peer Circuit 3: Traffic levels at five minute intervals Peer Circuit 4: Traffic levels at five minute intervals • Circuit 2 fails. Traffic shifts to circuit 4. Failover Matrices– Cariden 1 2 APRI COT 2006
Failover Exam ple ( from real netw ork) Peer Circuit 1: Traffic levels at five minute intervals Peer Circuit 2: Traffic levels at five minute intervals Peer Circuit 3: Traffic levels at five minute intervals Peer Circuit 4: Traffic levels at five minute intervals • Circuit 1 fails. Some traffic shifts to 2 & 4 • Some “leaks” to other AS’s Failover Matrices– Cariden 1 3 APRI COT 2006
Questions • How do I calculate a failover matrix? • How do I use a failover matrix from a peer? • What if my peer does not cooperate? • What if a substantial amount of traffic “leaks” to another AS? Failover Matrices– Cariden 1 4 APRI COT 2006
Calculating Failover Matrices • Accurate and Detailed [ 1,2] – Per prefix routing and traffic statistics – Full BGP simulation • Simple and Scalable [ 3] – Traffic matrix based on ingress-egress pairs • e.g., Peer1.LAX-AR1.CHI (measure and/ or estimate) instead of 192.12.3.0/ 24-208.43.0.0/ 16 – Limited simulation model • Shortest Path, Respect MEDs • “Our” AS plus immediate neighbors [ 1] “Modeling the routing of an Autonomous System with C-BGP,” B. Quoitin and S. Uhlig, IEEE Network, Vol 19(6), November 2005. [ 2] “Network-wide BGP route prediction for traffic engineering,” N. Feamster and J. Rexford, in Proc. Workshop on Scalability and Traffic Control in IP Networks, SPIE ITCOM Conference, August 2002. [ 3] Cariden MATE, available at http: / / www.cariden.com. Failover Matrices– Cariden 1 5 APRI COT 2006
Using Failover Matrix from Peers • Peer calculates failover matrix • Peer exports failover matrix using IP addresses of peering links • We import failover matrix • We include in a representative model of peer network • Use Failover Matrix in simulation Failover Matrices– Cariden 1 6 APRI COT 2006
Estim ate if Peer not Cooperate bbs1 bhx1 sel2 � Group own sea1 lba1 man1 det1 cph1 min1 nqt1 sources based on sfo1 roc1 bos1 nrt4 ewr1 nrt1 ham1 chi1 jfk1 lon3 exit location osa1 sjc2 sac1 pao2 kcy1 cle1 nyc2 wsh1 ams2 fra2 (4 groups here) tpe3 snv2 kul1 lax1 cdg2 den2 dal1 phi1 wdc2 ams1 lin1 sin1 bru1 hkg1 atl1 phx1 sna1 hou1 tpa1 mia1 bbs1 bhx1 sel2 pty1 mex1 gru1 syd1 sea1 lba1 man1 eze1 det1 cph1 min1 nqt1 sfo1 roc1 bos1 nrt4 ewr1 nrt1 ham1 chi1 jfk1 � Quantify shift (to 3 lon3 osa1 sjc2 sac1 pao2 kcy1 cle1 nyc2 wsh1 groups) after failure ams2 fra2 tpe3 snv2 kul1 lax1 cdg2 Assume similar for den2 dal1 phi1 wdc2 ams1 lin1 sin1 bru1 hkg1 atl1 phx1 other side sna1 hou1 tpa1 mia1 pty1 mex1 gru1 syd1 eze1 • Valid if topology and traffic distributions are similar Failover Matrices– Cariden 1 7 APRI COT 2006
Leaks to Other AS’s • Simple option – Leaks between peers relatively small • Ignore – Shifts between transit providers can be large • Equal AS-path length to most destinations: x A INTERNET B • Assume complete shift (easy to model) • Accurate option – Extend model to more than one AS away – Add columns in traffic matrix to designate extra traffic in case of other network failures Failover Matrices– Cariden 1 8 APRI COT 2006
W ork in Progress • Evaluating goodness of models – Compare actual failures to models • Evaluating goodness of failover estimates – Work with both sides of a peering arrangement, compare failover estimates to simulations – Compare estimated failover matrices to actual failures • Streamlining sharing of information • Contact me to participate in the above Failover Matrices– Cariden 1 9 APRI COT 2006
Sum m ary • Peering/ transit links are some of the most expensive and difficult to provision links • We can improve capacity planning on such links by modeling the network • BGP modeling can be much more complex than IGP modeling – Some required information is not even available • Failover Matrices provide a simple way to share information without giving away details • Failover Matrices can be estimated using one’s own network details Failover Matrices– Cariden 2 0 APRI COT 2006
Acknow ledgm ents • Jon Aufderheide (Global Crossing) • Clarence Filsfils (Cisco) Failover Matrices– Cariden 2 1 APRI COT 2006
Recommend
More recommend
