SLIDE 1
Supported by a Google Grant
Traffic differentiation
2
Traffic differentiation selectively changing the performance of network traffic
Reasons for differentiation:
¤ traffic engineering ¤ bandwidth management ¤ business reasons
Towards Detecting Differential QoE David Choffnes, Northeastern - - PowerPoint PPT Presentation
Towards Detecting Differential QoE David Choffnes, Northeastern - - PowerPoint PPT Presentation
Towards Detecting Differential QoE David Choffnes, Northeastern (joint work with SBU) Supported by a Google Grant Traffic differentiation 2 Traffic differentiation selectively changing the performance of network traffic Reasons for
SLIDE 2
SLIDE 3
Traffic differentiation
2
Traffic differentiation selectively changing the performance of network traffic
Reasons for differentiation:
¤ traffic engineering ¤ bandwidth management ¤ business reasons
Do certain types of network traffic receive better (or worse) QoE?
SLIDE 4
Motivation: Differentiation in the wild
3
Tested in early 2015
ISP
YouTube Netflix Spotify Verizon
m m m
Tmobile
- ATT
m m m
Sprint
m m m
Boost
m m m
BlackWireless
60%
- H2O
37% 45% 65%
SimpleMobile
36%
- NET10
p p p
- m: content modified on the fly
- p: translucent proxies change
connection behavior
SLIDE 5
Motivation: Differentiation in the wild
3
Tested in early 2015
ISP
YouTube Netflix Spotify Verizon
m m m
Tmobile
- ATT
m m m
Sprint
m m m
Boost
m m m
BlackWireless
60%
- H2O
37% 45% 65%
SimpleMobile
36%
- NET10
p p p
- m: content modified on the fly
- p: translucent proxies change
connection behavior
SLIDE 6
Motivation: Differentiation in the wild
3
Tested in early 2015
ISP
YouTube Netflix Spotify Verizon
m m m
Tmobile
- ATT
m m m
Sprint
m m m
Boost
m m m
BlackWireless
60%
- H2O
37% 45% 65%
SimpleMobile
36%
- NET10
p p p
- m: content modified on the fly
- p: translucent proxies change
connection behavior
SLIDE 7
Motivation: Differentiation in the wild
3
Tested in early 2015 Again in late 2015: No observed differences
ISP
YouTube Netflix Spotify Verizon
m m m
Tmobile
- ATT
m m m
Sprint
m m m
Boost
m m m
BlackWireless
60%
- H2O
37% 45% 65%
SimpleMobile
36%
- NET10
p p p
- m: content modified on the fly
- p: translucent proxies change
connection behavior
SLIDE 8
Key Questions
4
What do you test for differentiation?
¤ How do you generate traffic? ¤ What triggers differentiation?
How can you tell if there is differentiation?
¤ How do you do a controlled experiment? ¤ How do you rule out other reasons for differential service?
What is the impact on QoE?
¤ How do you map observed degradation to QoE? ¤ How do you scale this to arbitrary applications?
SLIDE 9
How do you test for differentiation?
5
What triggers differentiation?
¤ We don’t know
They might trigger on
¤ IP addresses ¤ ports ¤ payload signatures ¤ total number of connections ¤ total bandwidth ¤ time of day
(This is consistent with online manuals for DPI boxes)
SLIDE 10
How do you test for differentiation?
6
What triggers differentiation?
¤ We tested using carrier-grade DPI boxes
They might trigger on
¤ IP addresses ¤ ports ¤ payload signatures ¤ total number of connections ¤ total bandwidth ¤ time of day
SLIDE 11
What triggers differentiation?
7
HTTP
¤ Host: and GET fields, typically regex ¤ Examples: youtube, facebook, netflix
HTTPS
¤ Server cert is sent in plaintext ¤ Searches on SNI, CN fields
Other protocols
¤ Can identify Skype using some knowledge of handshake format ¤ Open question: how to reverse engineer classifiers?
What is not important
¤ IP addresses don’t seem to matter!
SLIDE 12
Triggering differentiation
8
SLIDE 13
Triggering differentiation
8
SLIDE 14
Triggering differentiation
8
end user
SLIDE 15
Triggering differentiation
8
end user
SLIDE 16
Triggering differentiation
8
end user
SLIDE 17
Triggering differentiation
8
end user
VPN TUNNEL
VPN proxy
SLIDE 18
Triggering differentiation
8
end user
VPN TUNNEL
VPN proxy
Record:
SLIDE 19
Triggering differentiation
8
end user
VPN TUNNEL
VPN proxy
Record: Replay:
SLIDE 20
Triggering differentiation
8
end user
VPN TUNNEL
VPN proxy
Record: Replay:
end user
SLIDE 21
Triggering differentiation
8
end user
VPN TUNNEL
VPN proxy
Record: Replay:
end user Replay server
SLIDE 22
Triggering differentiation
8
end user
VPN TUNNEL
VPN proxy
Record:
VPN TUNNEL
Replay:
end user Replay server
SLIDE 23
Triggering differentiation
8
end user
VPN TUNNEL
VPN proxy
Record:
VPN TUNNEL
Replay:
end user Replay server
SLIDE 24
Triggering differentiation
8
end user
VPN TUNNEL
VPN proxy
Record:
VPN TUNNEL
Replay:
end user Replay server
Shaper
SLIDE 25
Triggering differentiation
8
end user
VPN TUNNEL
VPN proxy
Record:
VPN TUNNEL
Replay:
end user Replay server
Shaper
SLIDE 26
Triggering differentiation
8
end user
VPN TUNNEL
VPN proxy
Record:
VPN TUNNEL
Replay:
end user Replay server
VPN TUNNEL Shaper
SLIDE 27
How to tell if there is differentiation?
9
Developed and validated new detection technique
¤ Back-to-back tests of same trace ¤ Includes VPN and random payload (but not ports) ¤ Send only at recorded rate (this is not a speed test) ¤ Statistical tests to reliably find differentiation
SLIDE 28
How to tell if there is differentiation?
9
Developed and validated new detection technique
¤ Back-to-back tests of same trace ¤ Includes VPN and random payload (but not ports) ¤ Send only at recorded rate (this is not a speed test) ¤ Statistical tests to reliably find differentiation
KS Test statistic Throughput (KB/s) CDF
SLIDE 29
How to tell if there is differentiation?
9
Developed and validated new detection technique
¤ Back-to-back tests of same trace ¤ Includes VPN and random payload (but not ports) ¤ Send only at recorded rate (this is not a speed test) ¤ Statistical tests to reliably find differentiation
Area Test = a/w Throughput (KB/s) CDF w a KS Test statistic Throughput (KB/s) CDF
SLIDE 30
Mapping to QoE
10
Currently measure throughput, loss, delay, jitter
¤Some clear mappings to video streaming bitrate ¤Fairly clear mapping to VoIP ¤… but unclear how it impacts user-perceived performance
Key challenge: Applications are adaptive, servers dynamic
¤Users may not perceive impairment
Current focus: Expose QoE metrics
¤YouTube does this, Netflix does not ¤What is the right format? ¤How do we know if users notice?
SLIDE 31
Summary
11
Differentiation and its impact on QoE important to solve Detecting differentiation requires care
¤ What you send (trigger it), how you send it (avoid it) ¤ Detection approach should be resilient to noise
Need a deeper understanding of impact on QoE
¤ Adaptive applications pose a challenge ¤ Potential approach is combination of ¤Exposing more QoE data from applications ¤Building better models to map QoS to QoE
SLIDE 32