SLIDE 1
2
Impact of Duration on Active Video testing Saba Ahsan - - PowerPoint PPT Presentation
Impact of Duration on Active Video testing Saba Ahsan - - PowerPoint PPT Presentation
Impact of Duration on Active Video testing Saba Ahsan Department of Communica5ons and Networking Aalto University Varun Singh (callstat.io), Jrg OG ( Technische
SLIDE 2
SLIDE 3
Motivation
- Actively measure performance of
Internet video (YouTube) from a user’s perspective
- Finish the test as quickly as
possible but still get reliable results
SLIDE 4
Talkshow Trailer 2 4 6 50 100 150 0 50 100 150
Time(s) Mbps
Instantaneous video bit rate of Full HD (1080p) YouTube videos
SLIDE 5
Problem Statement
- When conducting active video tests in a time
constrained environment, what is the minimum that the test should run for…
– Can we represent the bit rate magnitude and variations in Internet video with a short clip?
5
SLIDE 6
Dataset
- Popular YouTube videos by geographical area
- Mp4 videos (frame sizes and timing)
– Non-adaptive 720p > 32k – Non-adaptive 360p > 62k – DASH 1080p > 8k – DASH 720p > 19k
6
SLIDE 7
Selecting a cut-off value
7
Clip − S2 Full − S1 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0 20 40 60 100 200 300
Time(s) Mbps
SLIDE 8
Selecting a cut-off value
8
Clip − S2 Full − S1 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0 20 40 60 100 200 300
Time(s) Mbps
SLIDE 9
Selecting a cut-off value
9
Clip − S2 Full − S1 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0 20 40 60 100 200 300
Time(s) Mbps
SLIDE 10
Selecting a cut-off value
10
Clip − S2 Full − S1 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0 20 40 60 100 200 300
Time(s) Mbps
Take from the beginning, and startup delay is unchanged
SLIDE 11
Kolmogorov-Smirnov Test
11
DASH 1080p DASH 720p Non−adapt 360p Non−adapt 720p 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.4 0.8 0.0 0.4 0.8 0.0 0.4 0.8 0.0 0.4 0.8
KS Test p−value CDF Cutoff (s)
10 60 120 180 240 300
SLIDE 12
Autocorrelation Function-based dissimilarity
12
- 1
2 3 200 400 600
Cutoff length (s) ACF dissimilarity
- DASH 1080p
DASH 720p Non−adapt 360p Non−adapt 720p
90th percentile
SLIDE 13
Does it really work?
- Do we get similar user experience metrics even if we
cut-off the test before the video ends?
– YES
13
SLIDE 14
Does it really work?
- Do we get similar user experience metrics even if we
cut-off the test before the video ends?
– YES (We tested with our own client in emulated networks) we measured stall ratio= total stall duration/total playable video
14
SLIDE 15
Testing in Experiment network with 10 YouTube videos of different lengths
- 0.01
0.10 200 400 600
Cutoff length(s) Stall ratio Network rate
- 2Mbps
3Mbps 4Mbps
15
SLIDE 16
Testing in Experiment network with 10 YouTube videos longer than 10 min (600 s)
16
- 0.000
0.025 0.050 0.075 200 400 600
Cutoff length(s) Stall ratio Network rate
- 14Mbps
15Mbps 16Mbps 17Mbps
SLIDE 17
Summary and Future work…
- Test durations should run at least for 1 min, but 3 min
gives more reliable results.
17
SLIDE 18
Summary and Future work…
- Test durations should run at least for 1 min, but 3 min
gives more reliable results.
- What about DASH related metrics?
18
SLIDE 19
Summary and Future work…
- Test durations should run at least for 1 min, but 3 min
gives more reliable results.
- What about DASH related metrics?
- Results “probably” not suitable for feature-length movies
19
SLIDE 20