POI360 Panoramic Mobile Video Telephony over LTE Cellular Networks - PowerPoint PPT Presentation

POI360 Panoramic Mobile Video Telephony over LTE Cellular Networks Xiufeng Xie Xinyu Zhang University of Michigan-Ann Arbor University of California San Diego CoNEXT 2017

Background: 360 ° Video for VR 360 ° camera Sphere view Panoramic frame 360 ° video for VR 30FPS time

360 ° Video + Video Telephony = Interactive VR! Coverage Mobility

Challenges & Solution Spaces

Huge VR Traffic Load Calls for Compression • 360 ° frame  High VR stream bitrate: ▪ 10~20Mbps for 4K MP4 format ▪ Exceed LTE UL (5Mbps)/DL (12Mbps) bandwidth • Compression based on region of interest (ROI) Human eye can only see part of 360 ° Compress unseen parts Quality Region-of-Interest (ROI) Spatial position

Challenge 1: Compression Fails over LTE Update ROI knowledge t Compressed frame Low Low High Low High Low VR stream compressed ROI quality quality quality quality quality quality with new ROI User-perceived VR quality always fluctuates over LTE ROI change ROI quality recover Lower ROI quality for one RTT • Does not matter if RTT < VR frame interval (e.g., 33ms for 30fps) ▪ Typical wireline network ✓ • LTE has unstable RTT ( 5~500ms ) depending on traffic & channel

ROI Prediction? • Predict the ROI by reviewer’s motion? ▪ Oculus measurements [1]: • Avg. head angular speed: 60 Τ ° 𝑡 ° 𝑡 2 • Avg. head angular acceleration: 500 Τ • Head can stop rotation within 120ms ▪ Typical end-to-end LTE video latency can be more than 500ms Prediction: 120ms Need: 500ms ROI prediction does not work on LTE networks! [1] S.M.LaValle, A.Yershova, M.Katsev, and M.Antonov , “Head Tracking for The Oculus Rift,” in Robotics and Automation (ICRA), 2014 IEEE International Conference on , 2014.

Solution: Adaptive Compression Many ways to redistribute ROI center Aggressive the quality Video quality Sharp quality drop Spatial position … Adaptive compression Conservative • Responsive ROI update  Aggressive Smooth quality drop ▪ Maximize the user-perceived quality • Irresponsive ROI update  Conservative ▪ Guarantee the stability of VR quality

Challenge 2: Irresponsive Rate Control • Insufficient VR rate control responsiveness VR users: sensitive to video freezes in immersive environment LTE network: highly dynamic bandwidth Conventional video rate control Measure network-layer statistics Network Sluggish loop: Request suitable rate large RTT over LTE

Solution: Cellular Link-Informed Adaptation • Cellular link info as congestion indicator ▪ LTE uplink: typical bottleneck for mobile VR telephony ▪ Diagnostic interface: status of UL firmware buffer Uplink congestion control based on UL buffer status VR stream LTE uplink Network End-to-end congestion control Shortcut: shorter adaptation path  better responsiveness

Challenge 3: UL Bandwidth Underutilization LTE UL firmware buffer Video data UL throughput LTE uplink resource scheduling: UL throughput depends on its own buffer level Before UL congestion, higher buffer level  higher uplink rate • Existing rate control: unaware of this unique feature ▪ Buffer left empty (0 throughput) for 40% of time! ▪ UL throughput << available bandwidth

Solution: Adapt to UL Buffer Level • Learn relation between UL throughput & buffer level • Push firmware buffer level to the “sweet” region ▪ Sweet region: maximize throughput without congestion • Buffer level too high: slow down traffic to avoid congestion • Buffer level too low: speed up traffic to exploit bandwidth

POI360 System Design

Design Overview Viewer ROI Adaptive Spatial 360 ° Cam Compression Compressed VR stream Buffer Aware Rate Control Buffer level RTP traffic Firmware Buffer Sender Cellular uplink

Adaptive Spatial Compression • Adapt compression mode Video quality Aggressive ▪ Balance ROI quality and stability of ROI quality Conservative Spatial position • Design: • Switch mode following ROI update responsiveness • Responsiveness metric: T3-T1 (duration of lower ROI quality) T1: ROI change T3: ROI quality recovered T2: sender updates ROI knowledge

Buffer Aware Rate Control Compressed frame H.264 Encoding Rate Control Video bitrate Application layer PHY buffer level ▪ Cross-layer design Packet Pacer • Learn buffer’s “sweet” region • PHY buffer level too high  reduce RTP & video RTP bitrate Transport layer bitrate • PHY buffer level too low  increase RTP & video bitrate UL Firmware Buffer PHY bitrate Physical layer

Implementation VR player QXDM Live stream 360° video Diag. interface Client’s ROI LTE phone

Evaluation

Micro-benchmark Evaluation Video-frame-level delay • Validate VR compression design • Benchmark algorithm: ▪ CMU--Conduit (crop & send ROI) ▪ Facebook--Pyramid encoding ROI quality (PSNR) Reduce delay by 15% ROI quality stability 11~13dB of improvement Reduce variation by 5X

Micro-benchmark Evaluation • Validate our UL buffer-based rate control design ▪ Compare with Google Congestion Control (GCC, default rate control of Google Hangouts & Facebook Messenger) ▪ Our rate control FBCC keeps UL buffer level in the “sweet” region (green) for most of the time High usage Overuse (saturation) Low usage

System-Level Test • Test POI360 system under various network conditions ▪ Different LTE background traffic load ▪ Different physical channel quality ▪ Different mobility level • Performance metrics ▪ Smoothness • Video freezing ratio ▪ Quality • Frame-level PSNR • Mean Opinion Score(MOS)

Different Background Traffic Load • Light LTE background traffic load (early morning) ▪ 1% video freeze • Heavy LTE background traffic load (noon) ▪ 4% video freeze & 2dB PSNR drop ▪ Majority of the frames have either excellent or good quality PSNR & Video freezing ratio MOS

Different Physical Channel Quality • Test at places with different channel quality ▪ Weak (-115dB RSS); Moderate (-82dB RSS); Strong (-73dB RSS) ▪ Better channel: higher PSNR & MOS, less video freezes ▪ Even the weak channel achieves <3% video freezes PSNR & Video freezing ratio MOS

Different Mobility Level • Test under 3 different mobility levels ▪ Slow (15mph); urban driving (30mph); highway (50mph) ▪ PSNR & MOS drop with higher mobility. But still have good or excellent quality even under 50mph mobility ▪ More freezes with high mobility: 1% for slow driving, 7% for urban driving. Comparable to legacy non-360 LTE video chat MOS PSNR & Video freezing ratio

POI360 Summary • Unique challenges when 360 ° VR video meets LTE ▪ Long RTT of LTE breaks spatial VR compression ▪ Heavy VR traffic load ▪ Low cellular bandwidth utilization • POI360: the first adaptive 360 ° VR compression ▪ Adapt compression strategy based on network condition ▪ Achieve balance between traffic load & smoothness ▪ Leverage cellular statistics to enable responsive rate control • Other works in cellular network-informed mobile applications * “ piStream: Physical Layer Informed Adaptive Video Streaming Over LTE ”, Xiufeng Xie, Xinyu Zhang, Swarun Kumar, Li Erran Li, ACM MobiCom’15 * “ Accelerating Mobile Web Loading Using Cellular Link Information ”, Xiufeng Xie, Xinyu Zhang, Shilin Zhu, ACM MobiSys’17

Thank you! Q & A