Measuring the Mobile Internet David Choffnes Northeastern - - PowerPoint PPT Presentation

▶

Oct 28, 2023 296 likes •450 views

Measuring the Mobile Internet David Choffnes Northeastern University with U. Michigan, USC, Google, INRIA, and many others Motivation Most of the time people spend online is via a mobile device* When YouTube hangs, is it } A slow device?

SLIDE 1

Measuring the Mobile Internet

David Choffnes Northeastern University

with U. Michigan, USC, Google, INRIA, and many others

SLIDE 2

Measuring the Mobile Internet

Motivation

Most of the time people spend online is via a mobile device* When YouTube hangs, is it

} A slow device? } Weak signal? } Contention for bandwidth? } Bad path to Google? } ISP shaping? } Bad server?

Key challenge: We need extensive network measurements to perform root cause analysis

SLIDE 3

Measuring the Mobile Internet

There’s an app for that

Solution: App-based measurement

} Speedometer } Mobiperf } MySpeedTest } Netalyzr Mobile } Namehelp Mobile } …

How is this working out for us?

SLIDE 4

Measuring the Mobile Internet

Lessons learned from mobile measurement

} What you measure may not be what you think

Cellular Link Wired Link Non-Split: Split:

① HTTP GET www… ① HTTP 200 OK ① HTTP GET www… ① HTTP 200 OK ② HTTP GET www… ② HTTP 200 OK

Example: Estimate page load time for google.com (low cost)

1. DNS lookup for google.com
2. Ping google.com
3. Estimate latency as DNS lookup time + N RTTs

} But when you do an HTTP GET  

for google.com…

} Connection is split } New DNS lookup based on  

Host: field

} …except for

YouTube + T

Mobile

Extra DNS lookup

SLIDE 5

Measuring the Mobile Internet

Lessons learned from mobile measurement

} Spray and pray measurement can get you breadth…

HSPA HSDPA UMTS EDGE GPRS LTE EVDO eHRPD 1xRTT # of Measurements 439K 2326K 563K 506K 58K 1460K 2183K 301K 68K # of Carriers 50 111 96 85 48 7 8 2 3

100 1000 T

i l e A T & T Y e s O p t u s S w i s s c

a f

e ( D E ) V

a f

e ( N L ) V

a f

e ( I E ) V

a f

e ( U K ) O 2 ( U K ) A i r t e l T e l k

s e l R

e r s S i n g T e l N T T D

e l s t r a S F R S K T e l e c

E m

i l e Ping RTT (ms) GPRS EDGE UMTS HSDPA HSPA

} …but noise in the data frustrates characterization

SLIDE 6

Measuring the Mobile Internet

Lessons learned from mobile measurement

How do we isolate cases of bad performance? [PAM ’14] Example: Path inflation evolution in mobile networks

} Combination of traceroute and DNS lookups } Knowledge of Google’s infrastructure } Continuous measurements

Example: Shotgun correlations

} Vary signal strength only,  

bserve impact on performance

} Doesn’t really generalize to factors  

such as location

0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.24 0.26

5 10 15 20 25 30

400 600 800 1000 1200 1400 1600 1800 Mean Packet Loss (%) ASU Ping RTT Packet Loss

SLIDE 7

Measuring the Mobile Internet

Next generation of mobile measurement

Our 1st-generation apps have been useful!   Making further strides in this domain

} Needs a large(r) set of participating devices } Real-time monitoring of performance } On-demand (adaptive) measurements for RCA

Great, let’s all go build new apps!

SLIDE 8

Measuring the Mobile Internet

Pitfalls of App Proliferation

Coverage

} Who has a billion-install idea? Million-install?

}

If so, what the heck are you doing here?

Consistency

} How do we join datasets? } What measurement implementations?

Cost

} How many times do we need to run the same ping?

Interference

} What if MST runs during Mobiperf’s ping test?

SLIDE 9

Measuring the Mobile Internet

Mobilyzer: Mobile Measurement Library

Our proposal: One measurement library, many apps

} API for measurements  

simplifies app dev

click on “Layout” to see the “Design” menu and click on “Colors”. You can This PowerPoint 2007 template produces a 36”x48”

Ping,“google.com”, Modify scheduler’s configuration

SLIDE 10

Measuring the Mobile Internet

Mobilyzer: Mobile Measurement Library

Our proposal: One measurement library, many apps

} API for measurements  

simplifies app dev

} Single scheduler per  

device allows apps to   coexist

click on “Layout” to see the “Design” menu and click on “Colors”. You can This PowerPoint 2007 template produces a 36”x48”

Ping,“google.com”, Modify scheduler’s configuration

SLIDE 11

Measuring the Mobile Internet

Mobilyzer: Mobile Measurement Library

Our proposal: One measurement library, many apps

} API for measurements  

simplifies app dev

} Single scheduler per  

device allows apps to   coexist

} Data collection

click on “Layout” to see the “Design” menu and click on “Colors”. You can This PowerPoint 2007 template produces a 36”x48”

Ping,“google.com”, Modify scheduler’s configuration

SLIDE 12

Measuring the Mobile Internet

Why Mobilyzer?

Designed with researcher incentives in mind

} Simplifies app development

} Just ask MySpeedTest dev (GaTech student)

} PlanetLab-like participation model

} Bring one Mobilyzed app, run measurements on all devices

} Dynamic server-side control of measurements

SLIDE 13

Measuring the Mobile Internet

One last thing

What about detecting traffic differentiation in mobile?

SLIDE 14

Measuring the Mobile Internet

One last thing

What about detecting traffic differentiation in mobile?

} Challenges

} What traffic triggers differentiation? } How do we tell that the ISP is doing anything (vs. noise)?

} Our approach

} Record and replay traces of real traffic from mobile apps } Use

VPN tunnels (Meddle) as controlled experiments

App to be released soon (works without root)!

SLIDE 15

Measuring the Mobile Internet

Measuring the Mobile Internet

David Choffnes Northeastern University

with U. Michigan, USC, Google, INRIA, and many others

Motivation

Most of the time people spend online is via a mobile device* When YouTube hangs, is it

} A slow device? } Weak signal? } Contention for bandwidth? } Bad path to Google? } ISP shaping? } Bad server?

Key challenge: We need extensive network measurements to perform root cause analysis

There’s an app for that

Solution: App-based measurement

} Speedometer } Mobiperf } MySpeedTest } Netalyzr Mobile } Namehelp Mobile } …

How is this working out for us?

Lessons learned from mobile measurement

} What you measure may not be what you think

Example: Estimate page load time for google.com (low cost)

} But when you do an HTTP GET

for google.com…

} Connection is split } New DNS lookup based on

Host: field

} …except for

YouTube + T

Extra DNS lookup

Lessons learned from mobile measurement

} Spray and pray measurement can get you breadth…

} …but noise in the data frustrates characterization

Lessons learned from mobile measurement

How do we isolate cases of bad performance? [PAM ’14] Example: Path inflation evolution in mobile networks

} Combination of traceroute and DNS lookups } Knowledge of Google’s infrastructure } Continuous measurements

Example: Shotgun correlations

} Vary signal strength only,

} Doesn’t really generalize to factors

such as location

Next generation of mobile measurement

Our 1st-generation apps have been useful! Making further strides in this domain

} Needs a large(r) set of participating devices } Real-time monitoring of performance } On-demand (adaptive) measurements for RCA

Great, let’s all go build new apps!

Pitfalls of App Proliferation

Coverage

} Who has a billion-install idea? Million-install?

Consistency

} How do we join datasets? } What measurement implementations?

Cost

} How many times do we need to run the same ping?

Interference

} What if MST runs during Mobiperf’s ping test?

Mobilyzer: Mobile Measurement Library

Our proposal: One measurement library, many apps

} API for measurements

simplifies app dev

Ping,“google.com”, Modify scheduler’s configuration

Mobilyzer: Mobile Measurement Library

Our proposal: One measurement library, many apps

} API for measurements

simplifies app dev

} Single scheduler per

device allows apps to coexist

Ping,“google.com”, Modify scheduler’s configuration

Mobilyzer: Mobile Measurement Library

Our proposal: One measurement library, many apps

} API for measurements

simplifies app dev

} Single scheduler per

device allows apps to coexist

} Data collection

Ping,“google.com”, Modify scheduler’s configuration

Why Mobilyzer?

Designed with researcher incentives in mind

} Simplifies app development

} Just ask MySpeedTest dev (GaTech student)

} PlanetLab-like participation model

} Bring one Mobilyzed app, run measurements on all devices

} Dynamic server-side control of measurements

One last thing

What about detecting traffic differentiation in mobile?

One last thing

What about detecting traffic differentiation in mobile?

} Challenges

} What traffic triggers differentiation? } How do we tell that the ISP is doing anything (vs. noise)?

} Our approach

} Record and replay traces of real traffic from mobile apps } Use

VPN tunnels (Meddle) as controlled experiments

} But when you do an HTTP GET  

} Connection is split } New DNS lookup based on  

} Vary signal strength only,  

} Doesn’t really generalize to factors  

Our 1st-generation apps have been useful!   Making further strides in this domain

} API for measurements  

} API for measurements  

} Single scheduler per  

device allows apps to   coexist

} API for measurements  

} Single scheduler per  

device allows apps to   coexist