Mobile Network Performance from User Devices: A Longitudinal, - PowerPoint PPT Presentation

Mobile Network Performance from User Devices: A Longitudinal, Multidimensional Analysis Ashkan Nikravesh , David R. Choffnes, Ethan Katz-Bassett Z. Morley Mao, Matt Welsh

Problem Mobile Network Performance: ✘ Poor visibility into user perceived performance. ✘ It is difficult to capture a view of network performance. A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 2 / 16

Problem Mobile Network Performance: ✘ Poor visibility into user perceived performance. ✘ It is difficult to capture a view of network performance. Why is that difficult? • Performance depends on many factors A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 2 / 16

Problem Mobile Network Performance: ✘ Poor visibility into user perceived performance. ✘ It is difficult to capture a view of network performance. Why is that difficult? • Performance depends on many factors How to improve the visibility? A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 2 / 16

Problem Mobile Network Performance: ✘ Poor visibility into user perceived performance. ✘ It is difficult to capture a view of network performance. Why is that difficult? • Performance depends on many factors How to improve the visibility? • Pervasive network monitoring is needed: ◦ Continuous ◦ Large-scale A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 2 / 16

Problem Mobile Network Performance: ✘ Poor visibility into user perceived performance. ✘ It is difficult to capture a view of network performance. Why is that difficult? • Performance depends on many factors How to improve the visibility? • Pervasive network monitoring is needed: ◦ Continuous ◦ Large-scale • Sampling performance of devices across: ◦ Carriers ◦ Access Technologies ◦ Location ◦ Time A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 2 / 16

Previous Works Their Limitations: • Passively collected from cellular network infrastructure ( e.g. GGSN) • One month of data • Limited to a single carrier A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 3 / 16

Previous Works Their Limitations: • Passively collected from cellular network infrastructure ( e.g. GGSN) • One month of data • Limited to a single carrier • Collected from mobile devices , but not continuously A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 3 / 16

Previous Works Their Limitations: • Passively collected from cellular network infrastructure ( e.g. GGSN) • One month of data • Limited to a single carrier • Collected from mobile devices , but not continuously Our work differs from previous related work: • Longitudinal • Continuous • Gathered from mobile devices using controlled experiments. A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 3 / 16

Data Analysis ✔ Analyzing the data collected from: ✔ 144 carriers ✔ 17 months ✔ 11 cellular networks A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 4 / 16

Data Analysis ✔ Analyzing the data collected from: ✔ 144 carriers ✔ 17 months ✔ 11 cellular networks ✔ Identify patterns, trends, anomalies, and evolution of cellular networks’ performance. A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 4 / 16

Data Analysis ✔ Analyzing the data collected from: ✔ 144 carriers ✔ 17 months ✔ 11 cellular networks ✔ Identify patterns, trends, anomalies, and evolution of cellular networks’ performance. We find: • Significant variance in end-to-end performance for all carriers. A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 4 / 16

Data Analysis ✔ Analyzing the data collected from: ✔ 144 carriers ✔ 17 months ✔ 11 cellular networks ✔ Identify patterns, trends, anomalies, and evolution of cellular networks’ performance. We find: • Significant variance in end-to-end performance for all carriers. • Part of the high variability is due to the geographic and temporal properties of network. A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 4 / 16

Data Analysis ✔ Analyzing the data collected from: ✔ 144 carriers ✔ 17 months ✔ 11 cellular networks ✔ Identify patterns, trends, anomalies, and evolution of cellular networks’ performance. We find: • Significant variance in end-to-end performance for all carriers. • Part of the high variability is due to the geographic and temporal properties of network. • Routing and signal strength are potential sources of performance variability. A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 4 / 16

Data Analysis ✔ Analyzing the data collected from: ✔ 144 carriers ✔ 17 months ✔ 11 cellular networks ✔ Identify patterns, trends, anomalies, and evolution of cellular networks’ performance. We find: • Significant variance in end-to-end performance for all carriers. • Part of the high variability is due to the geographic and temporal properties of network. • Routing and signal strength are potential sources of performance variability. • Performance is inherently unstable . A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 4 / 16

Methodology • User perceived performance: A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 5 / 16

Methodology • User perceived performance: • HTTP GET Throughput • Ping RTT • DNS Lookup time A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 5 / 16

Methodology • User perceived performance: • HTTP GET Throughput • Ping RTT • DNS Lookup time ✔ Traceroute ✔ Carrier + Cellular network technology ✔ Signal strength ✔ Location ✔ Timestamp A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 5 / 16

Methodology • User perceived performance: • HTTP GET Throughput • Ping RTT • DNS Lookup time ✔ Traceroute ✔ Carrier + Cellular network technology ✔ Signal strength ✔ Location ✔ Timestamp To identify and isolate the performance impact of each factor A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 5 / 16

Methodology • User perceived performance: • HTTP GET Throughput • Ping RTT • DNS Lookup time ✔ Traceroute ✔ Carrier + Cellular network technology ✔ Signal strength ✔ Location ✔ Timestamp To identify and isolate the performance impact of each factor A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 5 / 16

Dataset • Mainly collected by Speedometer : • 2011-10 to 2013-2 (17 months) • Internal android app developed by Google • Anonymized data A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 6 / 16

Dataset • Mainly collected by Speedometer : • 2011-10 to 2013-2 (17 months) • Internal android app developed by Google • Anonymized data • Mobiperf • 11 months • Only used for our signal strength analysis A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 6 / 16

Dataset • Mainly collected by Speedometer : • 2011-10 to 2013-2 (17 months) • Internal android app developed by Google • Anonymized data • Mobiperf • 11 months • Only used for our signal strength analysis • Controlled experiments Speedometer dataset: 4-5 measurements per minute A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 6 / 16

Dataset • Mainly collected by Speedometer : • 2011-10 to 2013-2 (17 months) • Internal android app developed by Google • Anonymized data • Mobiperf • 11 months • Only used for our signal strength analysis • Controlled experiments Speedometer dataset: 4-5 measurements per minute • Code is open source and data is publicly available A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 6 / 16

Performance across Carriers How observed performance matches with the expectations across access technologies? A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 7 / 16

Performance across Carriers How observed performance matches with the expectations across access technologies? • Ping RTT Latency 1000 GPRS EDGE UMTS HSDPA HSPA Ping RTT (ms) 100 T A Y S V V V V O A T R S N T S S E - e e M T e w o o o o 2 i r o i n T F K m s d d d d t l l & i ( e k g g T s R o o O s a a a a U e T T s l o T t b b f f f f K r D r e p c o o o o m s e a i i o l e l l t o n n n n ) s l e e u C c s m e e e e e o ( ( ( ( l o D N I U M m E E L K ) o ) ) ) A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, M. Welsh PAM 2014 7 / 16