VIDEO AND WEB SERVICE QUALITY MONITORING USING ITU-T STANDARDS AT DEUTSCHE TELEKOM Werner Robitza 1 , Bernhard Feiten 2 , Alexander Raake 1 , Ulf Wüstenhagen 2 , Peter List 2 , Steve Göring 1 , Rakesh Rao 1 1 Technische Universität Ilmenau, 2 Deutsche Telekom AG
DEUTSCHE TELEKOM — WE HAVE THE BEST NETWORK … make this provable, tangible and understandable for millions of customers. GLOBALLY, IP VIDEO TRAFFIC WILL BE 82 PERCENT OF ALL CONSUMER INTERNET TRAFFIC BY 2021 WHITE PAPER: CISCO, “VNI FORECAST AND METHODOLOGY, 2016-2021” 2
FROM KPI TO KQI, Q O S TO Q O E Provider perspective: Quality Elements Service Quality of Service Key Performance Design Indicators (KPIs) User perspective: Quality Features Service Quality of Experience Key Quality Perception Indicators (KQIs) S. Moeller and A. Raake, Quality of Experience: Advanced Concepts, Applications and 3 Methods. Springer, 2014
TECHNOLOGY HTTP Adaptive Streaming (HAS) as the leading technology for video streaming today. QoE may be degraded due to: Inaccessible services § Initial loading delays, stalling events § Reduced video resolution / quality switching § Quality switches quality level Bitrate / t Stalling Initial loading delay https://youtube.googleblog.com 4
ITU-T RECOMMENDATION P.1203 (2017) § First standardized QoE model for HAS § Predicts Mean Opinion Scores for sequences up to 5 min length ARCHITECTURE A/V Quality Integration Module Audio Quality Module Quality Audiovisual Integral Quality Integration Parameter Video Quality Exctraction Module Stalling Impact Stalling Quality 5
EXTERNAL VALIDATION OF P.1203 PERFORMANCE Network Network P.1203 (Modes 0, 1) YT, Vimeo, Emulator Traces Amazon Performance Prime, Subjective DASH ratings S. Satti, et al., P. 1203 evaluation of real OTT video services, QoMEX 2017 6
CLIENT-CENTRIC QOE MONITORING COMPLEMENTING NETWORK-CENTRIC MONITORING 7
MULTI-SERVICE QUALITY MONITORING SYSTEM (MSQMS) Distributed quality monitoring system § Fully automated probes across Germany § Measure video streams and website calls, directly § from the browser, like a customer would do Different KPIs and KQIs measured: § DNS times, ping § Upload and download speed tests § Video loading times § Video quality according to P.1203 … and many more § 8
EXAMPLE CASE: QOE OF VIDEO STREAMS MOS Different baseline for MOS due to player/window size or maximum delivered video resolution Monitoring of different video providers over the period of several days 9
PLANNING NETWORK CAPACITY … WITH QOE IN MIND 10
PREDICTING YOUTUBE QOE FROM DOWNLOAD SPEEDS What is the customer’s video QoE for a given bandwidth ? § Not universally answerable! § Depends on the service / technology, client device, … § Example case: YouTube streaming to desktop clients § TEST ARCHITECTURE Test PC Measurement Logs Add-on for Chrome Traffic Shaper Video bitstreams P.1203 QoE W. Robitza et al., Measuring YouTube QoE with ITU-T P.1203 under Constrained Bandwidth Conditions, 11 QoE Management Workshop at QoMEX 2018
EXAMPLE: VIDEO CHUNKS AS A FUNCTION OF DOWNLOAD SPEED 100% Resolution hd1080 75% hd720 large 50% medium small 25% tiny 0% 16384 37499 1024 1536 2048 3072 4096 8192 256 384 512 768 Download 12
YOUTUBE QOE AS A FUNCTION OF DOWNLOAD SPEED 13
CONCLUSION AND OUTLOOK 14
CONCLUSION AND FUTURE WORK Deutsche Telekom as a leading provider in Germany § QoE as an important measure for network quality § Development of QoE metrics like P.1203 § Focus on QoE in monitoring systems: § Client-centric monitoring as complement to § network-centric solutions Predicting QoE from network parameters § OUTLOOK ITU competition P.NATS Phase 2 / AVHD-AS: § New models for 4K/UHD and new codecs Monitoring of encrypted video streams – predicting QoE from § network-centric monitoring systems 15
THANK YOU! Acknowledgements for their supporting work: Dhananjay Kittur (BTH Sweden), Alexander Dethof (TU Berlin), Marie-Neige Garcia (TU Berlin), and the colleagues in ITU-T Study Group 12, Q14 for jointly developing the P.1203 recommendation. 16
Recommend
More recommend
