using Large Scale Measurement Platforms Understanding the Impact of - - PowerPoint PPT Presentation

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

Understanding the Impact of Network Infrastructure Changes using Large Scale Measurement Platforms

Vaibhav Bajpai TU Munich IM 2017 Conference Lisbon, Portugal Dissertation Committee Jürgen Schönwälder Jacobs University Bremen, Germany Kinga Lipskoch Jacobs University Bremen, Germany Filip De Turck University of Ghent, Belgium May 2017

1 / 19

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

Survey on Internet Performance Measurement Platforms [1]

[COMST ′15]

Measuring IPv6 Performance

▶ Measuring Web Similarity [2] [CNSM ′16] ▶ Measuring TCP Connect Times [3] [NETWORKING ′15] ▶ Measuring YouTube Performance [4] [PAM ′15] ▶ Measuring Efgects of Happy Eyeballs [5] [ANRW ′16]

Measuring Access Network Performance

▶ RIPE Atlas Vantage Point Selection [6] [IM ′17] ▶ Dissecting Last-mile Latency Characteristics [∗] ▶ Lessons Learned from using RIPE Atlas [7] [SIGCOMM CCR ′15]

* entries are papers currently under review.

2 / 19

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

IPv6 Performance

▶ Literature focus largely on IPv6 adoption. ▶ Very little work on measuring IPv6 performance. ▶ Tiis study closes the gap.

2009 2010 2011 2012 2013 2014 2015 2016 2017 0% 5% 10% 15% Google IPv6 Adoption shaded region represents the duration of the longitudinal study.

We measure from ∼100 dual-stacked SamKnows probes.

NETWORK TYPE # RESIDENTIAL 78 NREN / RESEARCH 10 BUSINESS / DATACENTER 08 OPERATOR LAB 04 IXP 01 RIR # RIPE 60 ARIN 29 APNIC 10 AFRINIC 01 LACNIC 01 3 / 19

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

Complete Failures

2010 2011 2012 2013 2014 2015 2016 2017 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% W6D W6LD ALEXA 1M with AAAA entries HTTP Failure

▶ Failures reduced from 40% (2009) to 3% today.

0.1K 1K 10K 100K 1000K ALEXA Rank 0.0 0.2 0.4 0.6 0.8 1.0 CDF Failing AAAA Websites

4.3K

[Mar '17]

▶ 88% failing websites rank > 100K. ▶ 1% rank < 10K, six websites rank < 300.

100 101 102 103 www.bing.com 102 103 www.detik.com 100 101 102 103 www.engadget.com 102 103 www.nifty.com 100 101 102 103 104 www.qq.com Jan 2013 Jan 2014 Jan 2015 Jan 2016 Jul Jul Jul 102 103 www.sakura.ne.jp IPv6 IPv4 TCP Connect Times (ms)

Metrics should account for changes in IPv6-readiness.

4 / 19

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

Partial Failures

ALEXA top 100 websites with AAAA entries. ▶ 27% show some rate of failure over IPv6. ▶ 9% exhibit more than 50% failures over IPv6.

20 40 60 80 100 Success Rate (%) 0.0 0.2 0.4 0.6 0.8 1.0 CDF IPv6 (100) IPv4 (100)

▶ Limiting to root webpage can lead to

• verestimation of IPv6 adoption numbers

▶ Unclear whether websites with partial failures can be deemed IPv6-ready ▶ ISOC now supporting [8] development of tools that identify such partial failures

# Webpage Success Rate (%) W6LD IPv6(↓) IPv4 01 www.bing.com 100 ✓ 02 www.detik.com 100 ✓ 03 www.engadget.com 100 ✓ 04 www.nifty.com 100 05 www.qq.com 100 06 www.sakura.ne.jp 100 07 www.flipkart.com 09 99 ✓ 08 www.folha.uol.com.br 13 100 09 www.aol.com 48 100 ✓ 10 www.comcast.net 52 100 ✓ 11 www.yahoo.com 72 100 ✓ 12 www.mozilla.org 84 100 ✓ 13 www.orange.fr 86 100 ✓ 14 www.seznam.cz 89 100 ✓ 15 www.mobile.de 90 100 ✓ 16 www.wikimedia.org 90 100 17 www.t-online.de 93 100 ✓ 18 www.free.fr 95 100 19 www.usps.com 95 100 20 www.vk.com 95 100 ✓ 21 www.wikipedia.org 95 100 ✓ 22 www.wiktionary.org 95 100 23 www.elmundo.es 96 100 ✓ 24 www.uol.com.br 96 100 ✓ 25 www.marca.com 97 100 ✓ 26 www.terra.com.br 98 100 ✓ 27 www.youm7.com 99 100 5 / 19

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

Partial Failures | Root Cause Analysis

30 60 90 www.youm7.com (1%) www.terra.com.br (2%) www.marca.com (3%) www.uol.com.br (4%) www.elmundo.es (4%) www.wiktionary.org (5%) www.wikipedia.org (5%) www.vk.com (5%) www.usps.com (5%) www.free.fr (5%) www.t-online.de (7%) www.wikimedia.org (10%) www.mobile.de (10%) www.seznam.cz (11%) www.orange.fr (14%) www.mozilla.org (16%) www.yahoo.com (28%) www.comcast.net (48%) www.aol.com (52%) www.folha.uol.com.br (87%) www.flipkart.com (91%) www.sakura.ne.jp (100%) www.qq.com (100%) www.nifty.com (100%) www.engadget.com (100%) www.detik.com (100%) www.bing.com (100%) Network Level

CURLE_OK CURLE_COULDNT_RESOLVE_HOST CURLE_COULDNT_CONNECT CURLE_OPERATION_TIMEDOUT CURLE_GOT_NOTHING CURLE_RECV_ERROR

30 60 90 Contribution (%) Content Level

*/css */html */javascript, */json */octet-stream */plain */rdf */xml image/*

30 60 90 Service Level

SAME ORIGIN CROSS ORIGIN

Website failing over IPv6

▶ Failures silently exist; clients do not notice them due to IPv4 fallback. ▶ Identifjcation of operational issues relevant for upcoming IPv6-only networks ▶ Failures due to DNS resolution error on image/*, */javascript, */json and */css content. ▶ 12% websites have more than 50% content that belongs to same-origin source and fails over IPv6, ▶ Content failing from cross-origin sources consists of analytics and third-party advertisements.

6 / 19

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

Latency | Websites

∆sa(u) = t4(u) − t6(u) where t(u) is the time taken to establish TCP connection to website u.

▶ ISPs in early stages of IPv6 deployment should ensure their CDN caches are dual-stacked.

−150 −100 −50 50 TCP Connect Times [∆sa (ms)] www.bing.com www.facebook.com www.wikipedia.org www.youtube.com 2013 2014 2015 2016 2017 −60 −40 −20 20 www.blogspot.* www.google.* www.netflix.com www.yahoo.com

▶ TCP connect times to popular websites over IPv6 have considerably improved over time. ▶ Infmated latency over IPv6 was due to missing content caches over IPv6

7 / 19

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

Latency | Websites - Who connects faster?

ALEXA top 10K websites (as of Jan 2017):

▶ 40% are faster over IPv6. ▶ 94% of the rest are at most 1 ms slower. ▶ 3% are at least 10 ms slower. ▶ 1% are at least 100 ms slower.

−1.0 −0.5 0.0 0.5 1.0 ∆sa (ms) 0.0 0.2 0.4 0.6 0.8 1.0 CDF netflix yahoo google linkedin microsoft facebook wikipedia cloudflare heise

• penstreetmap

ALEXA (10K) [01/2017] ∆sa(u) = t4(u) − t6(u)

▶ Relevant for content providers to get insights on how their service delivery compares over IPv6.

8 / 19

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

YouTube

Latency is consistently higher over IPv6.

▶ TCP connect times

▶ < 1 ms slower over IPv6 ▶ Higher towards webpages

▶ Prebufgering durations

▶ > 25 ms slower over IPv6

▶ Startup delay

▶ > 100 ms slower over IPv6

▶ ISPs should make their GGC nodes dual-stacked.

−5 −4 −3 −2 −1 ∆t (ms) TCP Connect Times Web −0.4 −0.3 −0.2 −0.1 0.0 ∆t (ms) TCP Connect Times Audio Video −120 −80 −40 ∆p (ms) Prebuffering Duration Oct Jan 2015 Apr Jul Oct Jan 2016 Apr −400 −300 −200 −100 ∆s (ms) Startup Delay

∆t(y) = tc4(y) − tc6(y) ∆p(y) = pd4(y) − pd6(y) ∆s(y) = sd4(y) − sd6(y) 9 / 19

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

Happy Eyeballs

▶ Only ∼1% of samples above

HE timer value > 300 ms

Samples where HE prefers IPv6 −

▶ HE prefers slower IPv6

connections 90% of the time.

▶ HE timer of 150 ms maintains

same IPv6 preference levels.

10-2 10-1 100 101 102 103 104 TCP Connect Times (ms) 0.0 0.2 0.4 0.6 0.8 1.0

CDF

300 ms IPv6 (462K) IPv4 (462K) ['13 - '17] −40 −30 −20 −10 10 ∆sa (ms) 0.0 0.2 0.4 0.6 0.8 1.0 CDF 1% 2% 7% 30% 93% 99% 462K ['13 - '17]

▶ RFC 6555 should have used 150 ms timer. Measurements should inform protocol engineering. ▶ Drive an RFC 6555 update with operational experience within the IETF.

10 / 19

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

Survey on Internet Performance Measurement Platforms

[COMST ′15]

Measuring IPv6 Performance

▶ Measuring Web Similarity [CNSM ′16] ▶ Measuring TCP Connect Times [NETWORKING ′15] ▶ Measuring YouTube Performance [PAM ′15] ▶ Measuring Efgects of Happy Eyeballs [ANRW ′16]

Measuring Access Network Performance

▶ RIPE Atlas Vantage Point Selection [IM ′17] ▶ Dissecting Last-mile Latency Characteristics [∗] ▶ Lessons Learned from using RIPE Atlas [SIGCOMM CCR ′15]

* entries are papers currently under review.

11 / 19

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

Last-mile Latency

▶ Latency becomes a critical factor [9] when downstream throughput > 16 Mb/s. ▶ Last-mile latency is a major contributor[9] to end-to-end latency. ▶ However, little is known [10, 11] about characteristics of last-mile latency. ▶ 696 RIPE Atlas v3 residential probes (blue) ▶ 1245 SamKnows residential probes (red) Methodology described to isolate residential probes useful for future broadband measurement studies using these platforms.

12 / 19

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

Last-mile Latency | Home Network Latency

Tie home network should not be accounted when measuring last-mile latency.

10-1 100 101 102 103 104 hop1/hop2 (%) 0.0 0.2 0.4 0.6 0.8 1.0 CDF Residential Probes SamKnows (1.1K) RIPE Atlas (0.6K)

▶ hop1 > 10% of hop2 latency (∼19% probes). Last-mile latency should be the difgerence between the hop2 and hop1 latency.

13 / 19

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

Last-mile Latency | Interleaving Depths

▶ DSL networks not only enable interleaving [11] but … ▶ …also employ multiple interleaving depth levels that change with time.

1 4 16 64 0.2 0.6 1.0 CDF FREE 1 4 16 64 FREE 1 4 16 64 PLUSNET 1 4 16 64 TISCALI 29/ 12/ 26/ 1 4 16 64 RTT (ms) 29/ 12/ 26/ 05/ 19/ 29/ 12/ 26/ hop1 hop2

▶ Interleaving depths show a step-wise functional change. ▶ hop2 latency transitions correlate with corresponding timeseries.

14 / 19

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

Last-mile Latency | Time of Day Efgects

▶ Last-mile latencies are stable over time. ▶ Last-mile latencies do not exhibit diurnal load patterns. ▶ Simulation studies can now accurately model access links. ▶ CDN providers benefjt from characteristics of the last-mile. ▶ Promotes ISPs to cache popular content close to the CPE.

01h 05h 09h 13h 17h 21h [35 days] 01h 05h 09h 13h 17h 21h [35 days] 1 2 4 8 163264 Last-mile latency (ms) 01h 05h 09h 13h 17h 21h [35 days] 424 424 425 426 425 425 [# Probes] DSL (RIPE Atlas) 223 223 223 223 223 223 [# Probes] CABLE (RIPE Atlas) 36 36 36 36 36 36 [# Probes] FIBRE (RIPE Atlas)

15 / 19

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

Last-mile Latency | Subscriber Location

▶ Not all cable deployments [10, 11] show last-mile latencies < DSL.

1 2 4 8 16 32 64 Last-mile latency (ms) 0.0 0.2 0.4 0.6 0.8 1.0 CDF (84 probes) COMCAST (RIPE Atlas) EST (44) PST (40)

▶ Last-mile latencies:

▶ can depend on geographic location of the subscriber. ▶ are considerably difgerent along US east (∼32 ms) and west (∼8 ms) coast. 16 / 19

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

Last-mile Latency | Broadband Speeds

Last-mile latencies vary by broadband speeds.

20 21 22 23 24 25 26 Last-mile latency (ms) 0.0 0.2 0.4 0.6 0.8 1.0 CDF (306) BT (SamKnows)

80 Mbps (100) 40 Mbps (37) 20 Mbps (88) 8 Mbps (81)

▶ Input for future standards (QUIC, TLS 1.3) work that targets operation in 0-RTT mode. ▶ ADSL2+ and VDSL with higher transmission rates help reduce interleaving delays. ▶ Last-mile latencies for VDSL < ADSL/ADSL2+

17 / 19

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

Tiis thesis would not have been possible without these amazing people!

• –

– – – –

• What’s ¡missing: ¡Many ¡things, ¡but ¡in

18 / 19

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

• 1. Survey on Internet Performance Measurement Platforms

[COMST ′15]

• 2. Measuring IPv6 Performance

▶ Measuring Web Similarity [CNSM ′16] ▶ Measuring TCP Connect Times [NETWORKING ′15] ▶ Measuring YouTube Performance [PAM ′15] ▶ Measuring Efgects of Happy Eyeballs [ANRW ′16]

• 3. Measuring Access Network Performance

▶ RIPE Atlas Vantage Point Selection [IM ′17] ▶ Dissecting Last-mile Latency Characteristics [∗] ▶ Lessons Learned from using RIPE Atlas [SIGCOMM CCR ′15]

www.vaibhavbajpai.com bajpaiv@in.tum.de | @bajpaivaibhav

19 / 19

Contributions IPv6 Performance

Failures Latency YouTube Happy Eyeballs

Last-mile Latency Tianks! Q/A

References

[1]

• V. Bajpai and J. Schönwälder, “A Survey on Internet Performance

Measurement Platforms and Related Standardization Efgorts,” ser. IEEE Communications Surveys and Tutorials, 2015. [Online]. Available: http://dx.doi.org/10.1109/COMST.2015.2418435 [2]

• S. J. Eravuchira, V. Bajpai, J. Schönwälder, and S. Crawford,

“Measuring Web Similarity from Dual-stacked Hosts,” ser. Conference on Network and Service Management, 2016, pp. 181–187. [Online]. Available: http://dx.doi.org/10.1109/CNSM.2016.7818415 [3]

• V. Bajpai and J. Schönwälder, “IPv4 versus IPv6 - who connects

faster?” ser. IFIP Networking Conference, 2015, pp. 1–9. [Online]. Available: http://dx.doi.org/10.1109/IFIPNetworking.2015.7145323 [4]

• S. Ahsan, V. Bajpai, J. Ott, and J. Schönwälder, “Measuring YouTube

from Dual-Stacked Hosts,” ser. Passive and Active Measurement Conference, 2015, pp. 249–261. [Online]. Available: http://dx.doi.org/10.1007/978-3-319-15509-8_19 [5]

• V. Bajpai and J. Schönwälder, “Measuring the Efgects of Happy

Eyeballs,” ser. Applied Networking Research Workshop, 2016. [Online]. Available: http://dl.acm.org/citation.cfm?id=2959429 [6]

• V. Bajpai, S. J. Eravuchira, J. Schönwälder, R. Kisteleki, and E. Aben,

“Vantage Point Selection for IPv6 Measurements: Benefjts and Limitations of RIPE Atlas Tags,” ser. International Symposium on Integrated Network Management (IM), 2017 (to appear). [7]

• V. Bajpai, S. J. Eravuchira, and J. Schönwälder, “Lessons Learned

From Using the RIPE Atlas Platform for Measurement Research,” ser. Computer Communication Review, vol. 45, no. 3, 2015, pp. 35–42. [Online]. Available: http://doi.acm.org/10.1145/2805789.2805796 [8] “NAT64 Check,” nat64check.ipv6-lab.net, [Accessed 15-Apr-2017]. [9]

• S. Sundaresan, N. Feamster, R. Teixeira, and N. Magharei, “Measuring

and Mitigating Web Performance Bottlenecks in Broadband Access Networks,” ser. IMC, 2013. [Online]. Available: http://doi.acm.org/10.1145/2504730.2504741 [10]

• M. Dischinger, A. Haeberlen, P. K. Gummadi, and S. Saroiu,

“Characterizing Residential Broadband Networks,” ser. Internet Measurement Conference, 2007, pp. 43–56. [Online]. Available: http://doi.acm.org/10.1145/1298306.1298313 [11]

• S. Sundaresan, W. de Donato, N. Feamster, R. Teixeira, S. Crawford,

and A. Pescapè, “Broadband Internet Performance: A View From the Gateway,” ser. SIGCOMM, 2011, pp. 134–145. [Online]. Available: http://doi.acm.org/10.1145/2018436.2018452 19 / 19