CHALLENGES IN INFERRING INTERNET CONGESTION USING THROUGHPUT TESTS - - PowerPoint PPT Presentation

▶
challenges in inferring internet congestion using
SMART_READER_LITE
LIVE PREVIEW

CHALLENGES IN INFERRING INTERNET CONGESTION USING THROUGHPUT TESTS - - PowerPoint PPT Presentation

Nov 01, 2023 30 likes •576 views

CHALLENGES IN INFERRING INTERNET CONGESTION USING THROUGHPUT TESTS Amogh Dhamdhere amogh@caida.org with Srikanth Sundaresan (Princeton) Danny Lee (Georgia Tech) Xiaohong Deng, Yun Feng (UNSW) 1 w w w . cai da. or In the Press

slide-1
SLIDE 1

w w w . cai da.

  • r

CHALLENGES IN INFERRING INTERNET CONGESTION USING THROUGHPUT TESTS

1

Amogh Dhamdhere amogh@caida.org with Srikanth Sundaresan (Princeton) Danny Lee (Georgia Tech) Xiaohong Deng, Yun Feng (UNSW)

slide-2
SLIDE 2

w w w . cai da.

  • r

(sub-title)

In the Press

2

slide-3
SLIDE 3

w w w . cai da.

  • r

Measurement Lab (M-lab)

  • M-lab operates a

large server-side infrastructure with hundreds of testing servers

  • Tools deployed on

server-side: NDT, Glasnost, Shaperprobe..

3

slide-4
SLIDE 4

w w w . cai da.

  • r

NDT test on M-lab

  • Runs upload and download throughput measurements
  • Web interface directs client to closest NDT server

4

slide-5
SLIDE 5

w w w . cai da.

  • r

NDT test on M-lab

  • Runs upload and download throughput measurements
  • Web interface directs client to closest NDT server

4

slide-6
SLIDE 6

w w w . cai da.

  • r

NDT test on M-lab

  • Runs upload and download throughput measurements
  • Web interface directs client to closest NDT server

4

slide-7
SLIDE 7

w w w . cai da.

  • r

Using NDT Data to Infer Congestion

5

October 2014 May 2015

slide-8
SLIDE 8

w w w . cai da.

  • r

Using NDT Data to Infer Congestion

6

M-lab server ISP X Access ISP A Access ISP B M-lab server ISP Y

slide-9
SLIDE 9

w w w . cai da.

  • r

Using NDT Data to Infer Congestion

7

? ? ?

M-lab server ISP X M-lab server ISP Y Access ISP A Access ISP B

slide-10
SLIDE 10

w w w . cai da.

  • r

Using NDT Data to Infer Congestion

8

M-lab server ISP X M-lab server ISP Y Access ISP A Access ISP B

slide-11
SLIDE 11

w w w . cai da.

  • r

Using NDT Data to Infer Congestion

9

M-lab server ISP X M-lab server ISP Y Access ISP A Access ISP B

slide-12
SLIDE 12

w w w . cai da.

  • r

Using NDT Data to Infer Congestion

10

!

M-lab server ISP X M-lab server ISP Y Access ISP A Access ISP B

slide-13
SLIDE 13

w w w . cai da.

  • r

Policy Implications

11

slide-14
SLIDE 14

w w w . cai da.

  • r

Policy Implications

11

slide-15
SLIDE 15

w w w . cai da.

  • r

Policy Implications

11

slide-16
SLIDE 16

w w w . cai da.

  • r

Policy Implications

11

slide-17
SLIDE 17

w w w . cai da.

  • r

In This Paper

  • Analysis of methodology, assumptions, and challenges in

inferring interconnect congestion using throughput tests

  • Topology
  • Placement and coverage
  • Statistical challenges
  • Recommendations for measurement and analysis

12

slide-18
SLIDE 18

w w w . cai da.

  • r

Not In This Paper

  • Challenging specific conclusions of the M-lab reports
  • Undermining M-lab’s contributions to building an open

server-side measurement platform

13

slide-19
SLIDE 19

w w w . cai da.

  • r

Challenge: Topology

  • Implicit assumptions involved in using throughput tests to

infer interconnection congestion

  • A1: No congestion internal to ASes, any congestion is at

borders

  • A2: Server and client ASes are directly connected
  • A3: All tests between server and client AS pass through

same IP link (or “similar” IP links)

14

slide-20
SLIDE 20

w w w . cai da.

  • r

Challenge: Topology

  • Implicit assumptions involved in using throughput tests to

infer interconnection congestion

  • A1: No congestion internal to ASes, any congestion is at

borders

  • A2: Server and client ASes are directly connected
  • A3: All tests between server and client AS pass through

same IP link (or “similar” IP links)

14

slide-21
SLIDE 21

w w w . cai da.

  • r

Are Server and Client AS Directly Connected ?

  • Method: Use Paris traceroutes associated with NDT tests

to infer connectivity between server and client AS

  • Use MAP-IT to infer AS boundaries in traceroute, infer

which tests traverse a single AS hop

  • Case study: NDT servers to top US ISPs from the FCC

Measurement Broadband America (MBA) report in 2015

15 Marder, Smith, “MAP-IT: Multi-pass Accurate Passive Inferences from Traceroute”, IMC 2016

slide-22
SLIDE 22

w w w . cai da.

  • r

Direct Connectivity

16

0.2 0.4 0.6 0.8 1 1.2 Comcast ATT TWC Verizon Centurylink Charter Cox Frontier Windstream Fraction of tests 1 hop 2 hops 2+ hops 117k 89k 56k 59k 13k 1k 39k 6k 4k

Server AS Client AS 1 hop Server AS Client AS 2 hops T

slide-23
SLIDE 23

w w w . cai da.

  • r

Direct Connectivity

17

0.2 0.4 0.6 0.8 1 1.2 Comcast ATT TWC Verizon Centurylink Charter Cox Frontier Windstream Fraction of tests 1 hop 2 hops 2+ hops 117k 89k 56k 59k 13k 1k 39k 6k 4k

Top-5 U.S. ISPs in Q3 2015

slide-24
SLIDE 24

w w w . cai da.

  • r

Direct Connectivity

17

0.2 0.4 0.6 0.8 1 1.2 Comcast ATT TWC Verizon Centurylink Charter Cox Frontier Windstream Fraction of tests 1 hop 2 hops 2+ hops 117k 89k 56k 59k 13k 1k 39k 6k 4k

Top-5 U.S. ISPs in Q3 2015 ISPs in top 10 but not top-5

slide-25
SLIDE 25

w w w . cai da.

  • r

Direct Connectivity

  • Overall 82% of traces were a single hop from server AS to

client AS (or siblings)

  • Depends on (server AS, client AS) pair
  • Direct connection assumption invalid for some ISPs in the top-10
  • Driven by the economic incentives of ASes hosting M-lab

servers and client ASes

  • AS-level ecosystem is dynamic, warrants periodic re-examination

18

slide-26
SLIDE 26

w w w . cai da.

  • r

Diversity of Interconnection

  • How many IP-level links are traversed in NDT tests from a

server to client AS?

  • Are the traversed links in the same geographical area?
  • Congestion can have geographical effects; aggregation

across different geographical areas could be misleading

19 Claffy et al., “Policy Challenges in Mapping Internet Interdomain Congestion”, TPRC 2016

slide-27
SLIDE 27

w w w . cai da.

  • r

Diversity of Interconnection

  • Case study: M-lab’s NDT server in Atlanta (atl02) hosted

by Level 3

  • Used MAP-IT to infer all IP-level links traversed from atl02

to large U.S. access ISPs in May 2015

  • Associated each IP link with the number of tests crossing

the link

20

slide-28
SLIDE 28

w w w . cai da.

  • r

Diversity of Interconnection

21

Client ISP # IP Links

#NDT tests per link

Comcast (AS7922) 2 1759, 8 Comcast (AS7725) 1 1650 Comcast (AS 22909) 1 1130 AT&T (AS7018) 14 2395, 820, 770, 216, 137, 25,21, 19,19, Verizon (AS701) 8 548, 62,54,42,20,2,1,1 Verizon (AS6167) 2 3,3 Cox (AS22773) 39 total 817, max 378 Frontier (AS5650) 1 107 CenturyLink 4 383, 39, 22, 1

slide-29
SLIDE 29

w w w . cai da.

  • r

Diversity of Interconnection

21

Client ISP # IP Links

#NDT tests per link

Comcast (AS7922) 2 1759, 8 Comcast (AS7725) 1 1650 Comcast (AS 22909) 1 1130 AT&T (AS7018) 14 2395, 820, 770, 216, 137, 25,21, 19,19, Verizon (AS701) 8 548, 62,54,42,20,2,1,1 Verizon (AS6167) 2 3,3 Cox (AS22773) 39 total 817, max 378 Frontier (AS5650) 1 107 CenturyLink 4 383, 39, 22, 1

Total 18 ASNs and 30 IP links with Comcast

slide-30
SLIDE 30

w w w . cai da.

  • r

Diversity of Interconnection

21

Client ISP # IP Links

#NDT tests per link

Comcast (AS7922) 2 1759, 8 Comcast (AS7725) 1 1650 Comcast (AS 22909) 1 1130 AT&T (AS7018) 14 2395, 820, 770, 216, 137, 25,21, 19,19, Verizon (AS701) 8 548, 62,54,42,20,2,1,1 Verizon (AS6167) 2 3,3 Cox (AS22773) 39 total 817, max 378 Frontier (AS5650) 1 107 CenturyLink 4 383, 39, 22, 1

Total 18 ASNs and 30 IP links with Comcast That’s a lot!

slide-31
SLIDE 31

w w w . cai da.

  • r

Location of Interconnection (DNS)

22

Level3 Interconnections with Cox from atl02

slide-32
SLIDE 32

w w w . cai da.

  • r

Location of Interconnection (DNS)

22

Level3 Interconnections with Cox from atl02

Atlanta NDT SERVER

slide-33
SLIDE 33

w w w . cai da.

  • r

Location of Interconnection (DNS)

22

Level3 Interconnections with Cox from atl02

Washington, D.C. Atlanta Dallas Los Angeles San Jose NDT SERVER 7 links 5 links 12 links 9 links

slide-34
SLIDE 34

w w w . cai da.

  • r

Location of Interconnection (DNS)

22

Level3 Interconnections with Cox from atl02

Washington, D.C. Atlanta Dallas Los Angeles San Jose

Large geographical diversity in traversed interconnections

NDT SERVER 7 links 5 links 12 links 9 links

slide-35
SLIDE 35

w w w . cai da.

  • r

Summary and Recommendations

  • Assumption of direct connectivity may not always be valid,

analysis must use path information to verify assumption

  • Traceroutes in both directions would be most useful
  • But clients typically run NDT from a web interface
  • Tools like bdrmap or MAP-IT on server-side infrastructure

would help match NDT tests with interconnection links traversed

23

slide-36
SLIDE 36

w w w . cai da.

  • r

What is the Coverage of Testing Servers?

  • Which interconnections of an access network are

“testable” using M-lab and speedtest.net servers?

  • Which interconnections traversed on the path to popular

web content are testable using M-lab and speedtest.net servers?

24

slide-37
SLIDE 37

w w w . cai da.

  • r

Methodology to Assess Coverage

25

  • Use bdrmap from

Ark VPs in a network to discover all interdomain links

  • f that network

visible from that VP

Luckie et al., “bdrmap: Inference of borders between IP networks”, IMC 2016 AS 1 AS 2 AS 3 AS 4 ARK VP

slide-38
SLIDE 38

w w w . cai da.

  • r

Methodology to Assess Coverage

26

  • Trace toward all

testing servers

  • Assess which

interconnections are “testable”

NDT Server NDT Server ARK VP AS 1 AS 2 AS 3 AS 4

slide-39
SLIDE 39

w w w . cai da.

  • r

Methodology to Assess Coverage

26

  • Trace toward all

testing servers

  • Assess which

interconnections are “testable”

NDT Server NDT Server ARK VP AS 1 AS 2 AS 3 AS 4

slide-40
SLIDE 40

w w w . cai da.

  • r

Methodology to Assess Coverage

27

  • Trace toward

domains in Alexa top 500 for U.S.

  • Identify

interconnections traversed

ARK VP AS 1 AS 2 AS 3 AS 4

slide-41
SLIDE 41

w w w . cai da.

  • r

Methodology to Assess Coverage

27

  • Trace toward

domains in Alexa top 500 for U.S.

  • Identify

interconnections traversed

ARK VP AS 1 AS 2 AS 3 AS 4

slide-42
SLIDE 42

w w w . cai da.

  • r

Methodology to Assess Coverage

27

  • Trace toward

domains in Alexa top 500 for U.S.

  • Identify

interconnections traversed

Alexa Alexa ARK VP AS 1 AS 2 AS 3 AS 4

slide-43
SLIDE 43

w w w . cai da.

  • r

Measurement Study

  • 16 Ark VPs in the US: Comcast (5), Time Warner (3), Cox

(2), AT&T, Verizon, CenturyLink, Charter, Frontier, Sonic, RCN

  • Crawled Alexa U.S. top 500, resolved domains and

subdomains locally on monitor

  • Traceroutes toward Alexa targets, M-lab servers,

speedtest.net servers

28

slide-44
SLIDE 44

w w w . cai da.

  • r

Low Coverage of Interconnections

  • Between 0.4% (AT&T) and 9% (Frontier) of

interconnections were testable with M-lab servers

29

1 10 100 1000 10000 COM-1 COM-2 COM-3 COM-4 COM-5 VZ TWC-1 TWC-2 TWC-3 COX-1 COX-2 CENT SONC RCN FRON ATT AS interconnections Bordermap Mlab Speedtest

slide-45
SLIDE 45

w w w . cai da.

  • r

Low Coverage of Interconnections

29

1 10 100 1000 10000 COM-1 COM-2 COM-3 COM-4 COM-5 VZ TWC-1 TWC-2 TWC-3 COX-1 COX-2 CENT SONC RCN FRON ATT AS interconnections Bordermap Mlab Speedtest

slide-46
SLIDE 46

w w w . cai da.

  • r

Low Coverage of Interconnections

29

1 10 100 1000 10000 COM-1 COM-2 COM-3 COM-4 COM-5 VZ TWC-1 TWC-2 TWC-3 COX-1 COX-2 CENT SONC RCN FRON ATT AS interconnections Bordermap Mlab Speedtest

  • Better coverage with speedtest.net servers: between 2.3%

(AT&T) and 28% (Sonic)

slide-47
SLIDE 47

w w w . cai da.

  • r

Peer Interconnections

  • Between 2.8% (RCN) and 30% (Sonic) of peer

interconnections were testable with M-lab servers

30

1 10 100 COM-1 COM-2 COM-3 COM-4 COM-5 VZ TWC-1 TWC-2 TWC-3 COX-1 COX-2 CENT SONC RCN FRON ATT AS interconnections Bordermap Mlab Speedtest

slide-48
SLIDE 48

w w w . cai da.

  • r

Overlap with Popular Web Content

  • Significant number of AS interconnections on paths to

Alexa web content not testable using M-lab or Speedtest

31

20 40 60 80 100 COM-1 COM-2 COM-3 COM-4 COM-5 VZ TWC-1 TWC-2 TWC-3 COX-1 COX-2 CENT SONC RCN FRON ATT AS interconnections Mlab-Alexa Alexa-Mlab Speedtest-Alexa Alexa-Speedtest

slide-49
SLIDE 49

w w w . cai da.

  • r

Summary and Recommendations

  • Currently a small fraction of interdomain interconnections
  • f an access network are “testable” using M-lab or

Speedtest

  • Better coverage for Speedtest, but not an open testing platform
  • Prioritize placement of testing servers on paths to popular

content

  • Our analysis does not capture video providers, need

different methodology

32

slide-50
SLIDE 50

w w w . cai da.

  • r

Statistical Challenges

  • Limitations of crowdsourcing
  • Samples cannot be controlled: clients test when they will
  • Time of day variations: more samples during peak than off-peak
  • Service plan variations
  • Home network performance problems
  • Thresholds to detect congestion

33

slide-51
SLIDE 51

w w w . cai da.

  • r

Statistical Challenges

  • What is the right threshold to conclude that a diurnal

throughput trend is due to congestion on the path?

34

6 12 18 24

Time of day (local)

10 20 30 40 50 60

Throughput (Mbps)

6 12 18 24

Time of day (local)

20 40 60 80 100 120 140

  • No. of samples
slide-52
SLIDE 52

w w w . cai da.

  • r

Statistical Challenges

  • 20-30% difference in throughput between peak and off-

peak

35

6 12 18 24

Time of day (local)

10 20 30 40 50 60

Throughput (Mbps)

6 12 18 24

Time of day (local)

50 100 150 200 250

  • No. of samples
slide-53
SLIDE 53

w w w . cai da.

  • r

Summary and Recommendations

  • Analysis should ensure enough samples are available for

statistical significance

  • Particularly if analysis is done per-link, samples will be sparse
  • Need periodic, deterministic measurements to overcome

the limitations of crowdsourcing

36

slide-54
SLIDE 54

w w w . cai da.

  • r

Thanks! amogh@caida.org

37