CompSci 514: Computer Networks Lecture 10: BGP problems Xiaowei - - PowerPoint PPT Presentation

1

CompSci 514: Computer Networks Lecture 10: BGP problems

Xiaowei Yang

2

Today

• Known problems of BGP

– Multi-homing – Instability – Delayed convergence

• Slow failover
• Discussing fixes

– Root cause, ghost flushing etc.

Background on the paper

• Another ACM SIGCOMM test of time

award paper

• A first large scale study of BGP traffic
• Motivated much improvement to BGP

3

Internet Routing Instability

Craig Labovitz, Student Member, IEEE, G. Robert Malan, Student Member, IEEE, and Farnam Jahanian, Member, IEEE

Abstract—This paper examines the network interdomain rout- ing information exchanged between backbone service providers at the major U.S. public Internet exchange points. Internet rout- ing instability, or the rapid fluctuation of network reachability information, is an important problem currently facing the In- ternet engineering community. High levels of network instability can lead to packet loss, increased network latency and time to

• convergence. At the extreme, high levels of routing instability

have led to the loss of internal connectivity in wide-area, national

• networks. In this paper, we describe several unexpected trends

in routing instability, and examine a number of anomalies and pathologies observed in the exchange of inter-domain routing

• information. The analysis in this paper is based on data collected

from BGP routing messages generated by border routers at five

• f the Internet core’s public exchange points during a nine month
• period. We show that the volume of these routing updates is

several orders of magnitude more than expected and that the majority of this routing information is redundant, or pathological. Furthermore, our analysis reveals several unexpected trends and ill-behaved systematic properties in Internet routing. We finally posit a number of explanations for these anomalies and evaluate their potential impact on the Internet infrastructure. flaps have led to the transient loss of connectivity for large portions of the Internet. Overall, instability has three primary effects: increased packet loss, delays in the time for network convergence, and additional resource overheard (memory, CPU, etc.) within the Internet infrastructure. The Internet is comprised of a large number of intercon- nected regional and national backbones. The large public exchange points are often considered the “core” of the Internet, where backbone service providers peer, or exchange traffic and routing information with one another. Backbone service providers participating in the Internet core must maintain a complete map, or default-free routing table, of all globally vis- ible network-layer addresses reachable throughout the Internet. The Internet is divided into a large number of different re- gions of administrative control commonly called autonomous

• systems. These autonomous systems (AS’s) usually have dis-

tinct routing policies and connect to one or more remote AS’s at private or public exchange points. AS’s are traditionally composed of network service providers or large organizational

4

Failover

• BGP is designed for scaling more than fast

failover

– Many mechanisms favor this balance – Route flap damping, for example.

• If excess routing changes (flapping), ignore for some

time.

• Has unexpected effects on convergence times.

– Route advertisement/withdrawal timers in the 30 second range – Effect: tens of seconds to many minutes to recover from simple failures. – 15-30 minute outages not uncommon.

5

Multi-homing

• Connect to multiple providers

– Goal: Higher availability, more capacity

• Problems:

– Provider-based addressing breaks – Everyone needs their own address space

6

Multi-homing increases routing table size

Mutil-home.com 128.0.0.0/8 204.0.0.0/8 204.1.0.0/16 ISP2 ISP1 You can reach 128.0.0.0/8 And 204.1.0.0/16 via ISP1 ISP3 204.1.0.0/16 ISP1 204.1.0.0/16 128.0.0.0/8 ISP1 204.1.0.0/16 ISP2 204.0.0.0/8 ISP2

7

Global routing tables continue to grow

Source: http://bgp.potaroo.net/as6447/

8

Other BGP problems

• Convergence: BGP may explore many

routes before finding the right new one.

– Labovitz et al., SIGCOMM 2000

• Correctness: routes may not be valid, visible,
• r loop-free.
• Security: There is none!

– Some providers filter what announcements their customers can make. Not all do. – See paper discussion site for pointers

9

Measurement studies

• Two papers (measurement)

– End-to-end traffic – Routing messages

• Experimental techniques
• Results

10

Internet Routing Instability

• Goals: how often BGP sends updates to change

routes

• Methodology:

– Analyzing BGP logs for a long time

Terms

• WADiff: withdrawal à announcement
• AADiff: announcement à announcement
• WADup: same route withdrawal à

announcement

• AADup: same route announcement à

announcement

• WWDup: same route withdrawal à

withdrawal

10/2/18 CPS 214 11

Observed pathologies

• Repeated WWDup, WADup, AADup
• Why are they pathologies?

10/2/18 CPS 214 12

10/2/18 13

• Majority of BGP updates are WWDup
• WWDup belong to ASes that never announce them
• Why?

– Stateless BGP, does not remember what have sent to peers – Send withdrawals to all peers

Possible origins of instability

• Stateless BGP
• Physical link errors
• Unjittered timers
• IGP, BGP interactions
• Conflicting routing policies

10/2/18 14

10/2/18 15

Data analysis techniques

• Time series analysis
• Frequency analysis

– Fast Fourier transform – Maximum entropy spectral estimation

• Different estimation methods, but both

find significant frequencies at seven days, and 24 hours

Main results

• Much more updates than expected

– 99% is pathological. Impressive! – A taxonomy to analyze pathologies

• Speculation of causes

– Configuration errors, router bugs – Correlate with traffic load, perhaps due to router architectures – Open research questions: root cause of updates

• Motivated much follow-up work

10/2/18 16

17

IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 9, NO. 3, JUNE 2001 293

Delayed Internet Routing Convergence

Craig Labovitz, Member, IEEE, Abha Ahuja, Member, IEEE, Abhijit Bose, and Farnam Jahanian

Abstract—This paper examines the latency in Internet path failure, failover, and repair due to the convergence properties of interdomain routing. Unlike circuit-switched paths which exhibit failover on the order of milliseconds, our experimental mea- surements show that interdomain routers in the packet-switched Internet may take tens of minutes to reach a consistent view of the network topology after a fault. These delays stem from temporary routing table fluctuations formed during the operation of the Border Gateway Protocol (BGP) path selection process on Internet backbone routers. During these periods of delayed convergence, we show that end-to-end Internet paths will experience intermittent loss of connectivity, as well as increased packet loss and latency. We present a two-year study of Internet routing convergence through the experimental instrumentation of key portions of the Internet infrastructure, including both passive data collection and fault-injection machines at major Internet exchange points. Based on data from the injection and measurement of several

• tions. For example, transient disruptions in backbone networks

that previously impacted a handful of scientists may now cause enormous financial loss and disrupt hundreds of thousands of end users. Since its commercial inception in 1995, the Internet has lagged behind the public switched telephone network (PSTN) in availability, reliability, and quality of service (QoS). Factors contributing to these differences between the commercial Internet infrastructure and the PSTN have been discussed in various literature [26], [18]. Although recent advances in the IETF’s Differentiated Services working group promise to improve the performance of application-level services within some networks, across the wide-area Internet these QoS algorithms are usually predicated on the existence of a stable

10/2/18 18

Delayed Internet Convergence

Measurement Problem discovery Modeling & analysis Improvement

• Methodologies

10/2/18 19

Experiments setup

• Actively inject BGP faults

– How is fault injected?

• Passively listen at peering sessions, and use NTP

synchronized machines to calculate the convergence time

• Actively send probe packets to observe end-to-end

packet loss and latency

• Much BGP work later uses similar measurement

techniques.

10/2/18 20

Results show delayed convergence

• Bad news travels slow.

10/2/18 21

Slow routing convergence results in poor end-to-end performance

10/2/18 22

What causes the delayed routing convergence?

• A simple BGP convergence model reveals that in

the worse case, all possible paths are explored before a prefix is withdrawn.

• No minimum advertisement timer: synchronized

network, global message queue

1 2 R (*0R, ∞,, 2R) (∞, ∞, *2R) (∞, ∞, *20R) (*0R, 1R, ∞,) (01R, *1R, ∞) (*01R, 10R, ∞,) (∞, *1R, 2R) (∞, ∞, *2R) (∞, ∞, ∞) 01R 01R 10R 10R 20R 20R

10/2/18 23

Min router Advertisement interval timer(MRAI) reduces message count

• Why?

– MATI introduces synchronization. Multiple announcements are combined into one announcement, reducing the total message count.

• However, the convergence time

becomes proportional to timer_interval * (n-3)

10/2/18 24

Lets brain storm…

• How can we fix the slow convergence

problem?

– What is the solution proposed by the authors?

• Sender-side loop detection. When a sender detects

a loop, it sends a withdrawal to a neighbor

• immediately. Since withdrawal is not subject to

MATI delay, this improvement reduces both message count and convergence time.

– What exactly is the root cause of BGPs slow convergence problem? – Can you come up with any solution?

10/2/18 25

Sender-side loop detection

• Without sender-side loop detection:

– AS3 à AS1: 301R – This announcement is sent out when MRAI timer expires

• With sender-side loop detection:

– AS3->AS1: withdrawal – Withdrawal is sent out immediately. AS1 knows it has no path.

10/2/18 26

BGP assertion

• Detect path inconsistency between different neighbors
• If inconsistency is found, give path learned from direct

neighbors high priority

• Sensitive to topology
• Does not eliminate all invalid paths

N1 R D X Y N2

X

N1à R: N1 X Y D R: N1 D N2 N1 D

10/2/18 27

Ghost flushing

• If new path is worse than last announced path, and

router advertisement timer has not expired yet, send a withdrawal immediately.

• The withdrawal flushes ghost information.
• Reduces the convergence time because withdrawals

are not delayed by MRAI, but does not help much with Tlong.

N1 R D X Y N2

X

N1à R: withdrawal R: N1 D N2 N1 D

10/2/18 28

BGP root cause notification

• Neither BGP-assertion nor ghost flushing works well in this

topology.

– Why? – BGP-assertion: 3 and 6 are both direct neighbors to 5, but their announcements may be inconsistent – BGP ghost flushing: the newer path is subject to MATI delay

• Explicitly send out link up/down information
• Essentially adds link-state information into BGP
• Sequence number is used to order the notifications.
• Open research problem: can you get rid of sequence number?

5 3 4 2 1

X

6

29

Background

• Route import/export follow business

relationship between ASes

• Routing advertisements include AS path

information

• à Use this information to infer AS-level

topology

30

An annotated AS Graph

• Sibling: provide transit services for each
• ther.

– May belong to the same company

31

AS5 AS1 AS2 AS3 AS4 peer-to-peer sibling-to- sibling edge edge AS6 AS7 provider-to

• customer edge

The valley-free property

• Valley-free: After traversing a provider-

to-customer or peer-to-peer edge, the AS path can not traverse a customer-to- provider or peer-to-peer edge

32

What type of paths are legitimate?

33

The basic algorithm

• The node with the largest number of

neighbors is the top-level provider on an AS path

34

35

Taken into configuration errors

36

Inferring peering relationships

37

Evalution

• Verified using AT&T internal information

38

10/2/18 39

Summary

• BGPs slow convergence problem and
• ther problems
• It represents a message overhead,

processing overhead, and latency tradeoff

• How to infer AS-level topology

Comments

• Measurement paper

– Data – Data collection techniques – Data analysis

• When to write a measurement paper

10/2/18 40