Analyzing Throughput and Stability in Cellular Networks Ermias - - PowerPoint PPT Presentation

Analyzing Throughput and Stability in Cellular Networks

Ermias Walelgne 1 Jukka Manner 1 Vaibhav Bajpai 2 J¨

• rg Ott 2

April 25, 2018 — NOMS’18, Taipei

1(ermias.walelgne | jukka.manner)@aalto.fi,Aalto University, Finland 2(bajpaiv | ott)@in.tum.de, Technical University of Munich, Germany

April 25, 2018 — NOMS’18, Taipei

Introduction

Introduction

Introduction — Motivation

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 1 / 20

Introduction

Introduction — Motivation

Mobile-broadband subscriptions have grown more than 20% annually in the last five years (ITU 2017) [1]

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 1 / 20

Introduction

Introduction — Motivation

Mobile-broadband subscriptions have grown more than 20% annually in the last five years (ITU 2017) [1] Smartphones (in 2017) have 70% share of the total market of all device types (IDC) [2].

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 1 / 20

Introduction

Introduction — Motivation

Mobile-broadband subscriptions have grown more than 20% annually in the last five years (ITU 2017) [1] Smartphones (in 2017) have 70% share of the total market of all device types (IDC) [2]. Quality & performance of the cellular network depends:

radio technology, limitations of device hardware wireless link characteristics (e.g interference, fading, etc.) mobility, location and time of the day Infrastructure of Mobile Network Operators (MNO)

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 1 / 20

Introduction

Introduction — Motivation

Mobile-broadband subscriptions have grown more than 20% annually in the last five years (ITU 2017) [1] Smartphones (in 2017) have 70% share of the total market of all device types (IDC) [2]. Quality & performance of the cellular network depends:

radio technology, limitations of device hardware wireless link characteristics (e.g interference, fading, etc.) mobility, location and time of the day Infrastructure of Mobile Network Operators (MNO)

About 30% of cellular measurements from netradar [3] experience sudden drops to zero bitrate for(≥ 200 ms).

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 1 / 20

Introduction

Introduction — Research Question

What are the factors affecting the throughput and stability of cellular networks?

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 2 / 20

Introduction

Introduction — Contribution

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 3 / 20

Introduction

Introduction — Contribution

1

Time of a day and the location affect the throughput:

Throughput drops during peak hours in cities as congestion increases. Radio technology switches from legacy (UMTS) to advanced (HSDPA) during the course of a day.

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 3 / 20

Introduction

Introduction — Contribution

1

Time of a day and the location affect the throughput:

Throughput drops during peak hours in cities as congestion increases. Radio technology switches from legacy (UMTS) to advanced (HSDPA) during the course of a day.

2

Predict the probability sudden drops in bit rate in a cellular network with 90% accuracy

relying on easily accessible information (e.g device model, location, network technology).

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 3 / 20

Introduction

Introduction — Contribution

1

Time of a day and the location affect the throughput:

Throughput drops during peak hours in cities as congestion increases. Radio technology switches from legacy (UMTS) to advanced (HSDPA) during the course of a day.

2

Predict the probability sudden drops in bit rate in a cellular network with 90% accuracy

relying on easily accessible information (e.g device model, location, network technology).

3

Predict the average TCP download speed based on the first 5 seconds of median bit rate value of throughput.

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 3 / 20

Methodology

Methodology

Methodology — Measurement platform

1 2 3 4 5 2.5 7.5 10 5 Time (Sec.) TCP downlik speed (Mbps)

It measures & collects information including throughput (TCP), signal strength, radio technology type, RTT( UDP ) etc. towards Amazon Cloud instances & Aalto University servers. The measurement server tests the download speed by sending a random data

• ver TCP for 10 seconds.

During the measurement session, both the client and the server record the number of bytes transferred every 50 ms.

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 4 / 20

Methodology

Methodology — Data Set and Measurement Trials

Based on a longitudinal dataset collected using the netradar measurement platform [3]. Focused on stationary nodes during a ten second measurement session — to minimize the variability that might arise from mobility. A year-long measurement data (∼750K ) from 3 Finnish MNO.

Network Total # Of LTE HSPA HSPA+ HSDPA UMTS Others Operator Measurements Elisa 373K 45.75% 8.49 % 14.70% 1.12% 25.43% 5% DNA 235K 63.30% 7.52% 8.17% 10.69% 0.9% 9.42% TeliaSonera 140K 34.74% 1.04% 14.27% 1.29% 33.60% 15.06% Table: Measurement Distribution per radio Technology for each MNO

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 5 / 20

Data Analysis

Data Analysis

Data Analysis — Device Model

EDGE GPRS HSDPA HSPA HSPA+ LTE UMTS

2011 2012 2013 2014 2015 2011 2012 2013 2014 2015 2011 2012 2013 2014 2015 2011 2012 2013 2014 2015 2011 2012 2013 2014 2015 2011 2012 2013 2014 2015 2011 2012 2013 2014 2015 10−3 10−2 10−1 100 101 102 Release year of the device Download speed (Mbps)

Figure 1: TCP download speed of different device models per network technology.

The release year of the device models does not correlate to TCP download speed. It is not always the newest device model whose TCP download performance is best.

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 6 / 20

Data Analysis

Data Analysis — Mobile Network Operator

0.00 0.25 0.50 0.75 1.00 10 20 30 40 50 60 70 80 TCP Download Average (MBps) CDF DNA Elisa TeliaSonera 0.00 0.25 0.50 0.75 1.00 10 20 30 40 50 TCP Upload Average (MBps) CDF DNA Elisa TeliaSonera

Figure 2: Mean TCP throughput for LTE networks of 3 MNO downlink (left) and uplink (right) speed.

Clear variation between MNOs on mean uploading speed for LTE. MNO’s are significant for network performance variation.

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 7 / 20

Data Analysis

Data Analysis — Subscribers Location

Elisa (Network Operator With LTE)

10 Mbps 15 Mbps 20 Mbps 25 Mbps 30 Mbps 35 Mbps 40 Mbps

DNA (Network Operator With LTE)

10 Mbps 20 Mbps 30 Mbps 40 Mbps 50 Mbps 60 Mbps 70 Mbps 80 Mbps 90 Mbps 100 Mbps 110 Mbps

Sonera (Network Operator With LTE)

10 Mbps 20 Mbps 30 Mbps 40 Mbps 50 Mbps 60 Mbps 70 Mbps 80 Mbps 90 Mbps

Figure 3: Mean TCP throughput distribution by area in Finland for LTE networks of three MNOs: Elisa (left), DNA (middle), TeliaSonera (right).

The comparison across MNOs shows a large variation in throughput per locations. Users in a metropolitan area & (subscribed to DNA or Elisa) get better throughput than urban areas.

Better infrastructure provisioning — base station density — sufficient core network capacity.

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 8 / 20

Data Analysis

Data Analysis — Radio Technology Changes and Time of Day

HSPA to HSPA+ UMTS to HSDPA

5 10 15 20 0% 2% 4% 6% 0% 2% 4% 6% 8%

Time of a day Handover frequency

Figure 4: Frequency of radio technology switches over time of day, for the TeliaSonera network (similar to other MNOs).

The occurrence of switches (from legacy to more advanced technology e.g UMTS to HSDPA) increases during peak hours.

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 9 / 20

Data Analysis

Data Analysis — Network Stability

1

2 3

4

5

2.5 7.5 10 5 Time (Sec.) TCP downlik speed (Mbps)

Classified the data into two groups: dropped: measurement sessions that experience a sudden dropout >200 ms. non-dropped: sessions without this phenomenon.

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 10 / 20

Data Analysis

Data Analysis — Received Data per Recording Interval

After sudden dropout Before sudden dropout 'P

200

+

Q*

175

Q*

150

• /-

tlm tlm

1 25

• 'P .1

100

<12:S;

75

• D

rm

50

• 25

0.0 0.51.0l.52.02.53.03.54.04.5 5.0 0.0 0.51.0l.52.02.53.03.54.04.5 5.0 Sudden dropout duration (sec)

• a

s HSPA LTE UMTS Download speed (Mbps)

Figure 5: TCP maximum download rate observed before and after a sudden dropout of a certain duration (≥ 200 ms) per radio technology.

A sudden dropout duration that stayed for at least 200 ms does have an impact on download bit rate. The impact is visible especially after the dropout period is over (left side of the figure).

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 11 / 20

Data Analysis

Data Analysis — Received Data per Recording Interval

mode mode

dropout dropout

Bytes downloaded at every 50 ms sample time span Bytes downloaded at every 50 ms sample time span

Measurements with NO sudden dropout Measurements with sudden dropout

Mean & median of dropped diverge with a relatively higher standard deviation than the non-dropped measurements. Network inconsistency and jitter is present in the dropped measurements than in non-dropped ones.

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 12 / 20

Data Analysis

Data Analysis — Received Data per Recording Interval

10 1 101 102 100 0.0 0.2 0.4 0.6 0.8 1.0 CDF

Average of the first 3 bit-rates samples before and after dropout Average of the first 3 bit-rates before dropout Average of the first 3 bit-rates after dropout

Download Speed (Mbps) Figure 6: Average of the first 3 bit rates samples before and after a sudden dropout.

The effect of sudden dropout is reflected even after the sudden dropout (zero bit rate) is over.

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 13 / 20

Data Analysis

Data Analysis — Sudden Dropout by Network Technology

EDGE GPRS HSDPA HSPA HSPA+ LTE UMTS

5 10 15 20

0% 2% 4% 6% 0% 2% 4% 6% 0% 2% 4% 6% 0% 2% 4% 6% 8% 0% 2% 4% 6% 0% 2% 4% 6% 0% 2% 4% 6%

Time of a day Sudden dropout frequency

The sudden dropouts is distributed in all radio technology that has been used. Some technologies such as UMTS and HSPA show frequent sudden dropouts during daytime.

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 14 / 20

Data Analysis

Data Analysis — Switching Radio Technology

0.00 0.25 0.50 0.75 1.00 10 20 30 40 50 Download speed (Mbps) CDF

Unknown to LTE Unknown to UMTS HSPA HSPA to HSPA+ LTE HSPA+ UMTS to HSPA+ UMTS HSDPA

Figure 7: Impact of radio technology switches for download speed.

When sudden dropout happens a switch in radio technology causes a significant variation in TCP download speed.

E.g. a change from UMTS to HSPA+ has better download speed than from HSPA to HSPA+.

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 15 / 20

Data Analysis

Data Analysis — Feature Importance and Selections

Downlink inter-arrival(variance) Average throughput(TCP) Device Model Area province Radio type at the beginning Latency(UDP) Radio type at the end Uplink speed Signal strengths Mobile Network Operator Time of a day(hours) Battery level Handover Cell ID Changes Day of week Frequency of handover Platform Byte inter-arrival(mean) 250 500 750

Mean Decrease Accuracy

Downlink (TCP) Downlink inter-arrival(variance) Area province Uplink speed Device Model Latency(UDP) Radio type at the beginning Day of week Battery level Time of a day(hours) Signal strengths Radio type at the end Handover Mobile Network Operator Cell ID Changes Frequency of handover Byte inter-arrival(mean) Platfprm 500 1000 1500 2000

Mean Decrease Gini

(a) with TCP throughput

RadioHandoverStartEnd StartingNetTechNames EndingNetTechName radioEvolution RadioHandover deviceVendor CellIDChanged battery_level weekday DModel hours signalStrengths carrier latency 25 50 75 Mean Decrease Accuracy

(b) without TCP throughput

Latency, carrier network, signal strength radio network technology, time of the day and device model found to be important predictive variables for classification.

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 16 / 20

Data Analysis

Data Analysis — Classification model

0.0 0.2

ROC

0.4

• 0.6

0.8 1.0

False positive rate True positive rate

Models:

Conditional Inference Tree Bagging RandomForest

• .o

0.2 0.4 0.6 0.8 1.0

Figure 9: True positive and false positive rates of the three classification models; random forest shows the best Receiver Operating Characteristic (ROC) curve.

Random forest based classification produces better prediction with accuracy

• f 90% & error rate of 10.2% on the testing dataset.

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 17 / 20

Data Analysis

Data Analysis — Predicting the Average Throughput

Prediction of througput is useful (eg. to improve video performance in cellular networks [4]). How to predict the overall mean throughput only using the first five seconds

• f TCP bit rate measurement?

Train a model using Randome Forest algorithm from caret [5] package. The model predict average throughput: using only the first 5 sec. download rates (which would be easily available right after startup) with Root-Mean-Square Error (RMSE) ( 0.003 Mbps) & 98% R-squared.

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 18 / 20

Conclusion

Conclusion

Conclusion

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 19 / 20

Conclusion

Conclusion

1

Throughput observed by a cellular network user depends on various factors.

E.g location and time of the day where metropolitan areas during peak hours showed more drops in throughput.

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 19 / 20

Conclusion

Conclusion

1

Throughput observed by a cellular network user depends on various factors.

E.g location and time of the day where metropolitan areas during peak hours showed more drops in throughput.

2

Network stability:

TCP being sensitive to losses and jitter ∼30% sudden drops to zero bitrate - could create performance degradations. classify stability based on sudden dropouts only relying on easily accessible information.

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 19 / 20

Conclusion

Conclusion

1

Throughput observed by a cellular network user depends on various factors.

E.g location and time of the day where metropolitan areas during peak hours showed more drops in throughput.

2

Network stability:

TCP being sensitive to losses and jitter ∼30% sudden drops to zero bitrate - could create performance degradations. classify stability based on sudden dropouts only relying on easily accessible information.

3

Predicting the average throughput:

useful to anticipate future performance & to adjust application demands. trained a model that predicts average throughput based on throughput of TCP slow start phase.

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 19 / 20

Conclusion

Conclusion

1

Throughput observed by a cellular network user depends on various factors.

E.g location and time of the day where metropolitan areas during peak hours showed more drops in throughput.

2

Network stability:

TCP being sensitive to losses and jitter ∼30% sudden drops to zero bitrate - could create performance degradations. classify stability based on sudden dropouts only relying on easily accessible information.

3

Predicting the average throughput:

useful to anticipate future performance & to adjust application demands. trained a model that predicts average throughput based on throughput of TCP slow start phase.

4

Future work extending — our predictive approach :

Using Crowdsourcing Data for Adaptive Video Streaming in Cellular Network.

ermias.walelgne@aalto.fi

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 19 / 20

Conclusion

References

ITU, “ITU releases 2017 global information and communication technology facts and figures.” IDC, “IDC idc 50th anniversary transformation everywhere,” 2017. “Netradar measurement platform.” http://www.netradar.org/.

• X. K. Zou, J. Erman, V. Gopalakrishnan, E. Halepovic, R. Jana, X. Jin,
• J. Rexford, and R. K. Sinha, “Can Accurate Predictions Improve Video

Streaming in Cellular Networks?,” HotMobile, pp. 57–62, 2015.

• M. Kuhn, “Building Predictive Models in R Using the caret Package,”

Journal of Statistical Software, vol. 28, no. 5, pp. 1–26, 2008.

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 20 / 20