POI360 Panoramic Mobile Video Telephony over LTE Cellular Networks - - PowerPoint PPT Presentation

POI360

Panoramic Mobile Video Telephony over LTE Cellular Networks

Xiufeng Xie University of Michigan-Ann Arbor Xinyu Zhang University of California San Diego CoNEXT 2017

Background: 360° Video for VR

Sphere view Panoramic frame 360° camera

time

360° video for VR 30FPS

360° Video + Video Telephony = Interactive VR!

Mobility Coverage

Challenges & Solution Spaces

Huge VR Traffic Load Calls for Compression

• 360° frame  High VR stream bitrate:

▪ 10~20Mbps for 4K MP4 format ▪ Exceed LTE UL (5Mbps)/DL (12Mbps) bandwidth

• Compression based on region of interest (ROI)

Human eye can only see part of 360°

Quality Spatial position

Region-of-Interest (ROI)

Compress unseen parts

Challenge 1: Compression Fails over LTE

• Does not matter if RTT < VR frame interval (e.g., 33ms for 30fps)

▪ Typical wireline network✓

• LTE has unstable RTT (5~500ms) depending on traffic & channel

High quality Low quality Low quality

Compressed frame

High quality Low quality Low quality

t ROI Update ROI knowledge ROI change ROI quality recover

Lower ROI quality for one RTT

VR stream compressed with new ROI

User-perceived VR quality always fluctuates over LTE

ROI Prediction?

• Predict the ROI by reviewer’s motion?

▪ Oculus measurements [1]:

• Avg. head angular speed: 60 Τ

° 𝑡

• Avg. head angular acceleration: 500 Τ

° 𝑡2

• Head can stop rotation within 120ms

▪ Typical end-to-end LTE video latency can be more than 500ms Prediction: 120ms Need: 500ms

[1] S.M.LaValle, A.Yershova, M.Katsev, and M.Antonov, “Head Tracking for The Oculus Rift,” in Robotics and Automation (ICRA), 2014 IEEE International Conference on, 2014.

ROI prediction does not work on LTE networks!

Solution: Adaptive Compression

• Responsive ROI update  Aggressive

▪ Maximize the user-perceived quality

• Irresponsive ROI update  Conservative

▪ Guarantee the stability of VR quality Conservative Aggressive Smooth quality drop Sharp quality drop

Video quality Spatial position ROI center

…

Adaptive compression

Many ways to redistribute the quality

Challenge 2: Irresponsive Rate Control

• Insufficient VR rate control responsiveness

Sluggish loop: large RTT over LTE Request suitable rate Measure network-layer statistics

Network

Conventional video rate control VR users: sensitive to video freezes in immersive environment LTE network: highly dynamic bandwidth

VR stream LTE uplink

Solution: Cellular Link-Informed Adaptation

• Cellular link info as congestion indicator

▪ LTE uplink: typical bottleneck for mobile VR telephony ▪ Diagnostic interface: status of UL firmware buffer Uplink congestion control based on UL buffer status

Network

End-to-end congestion control Shortcut: shorter adaptation path  better responsiveness

Challenge 3: UL Bandwidth Underutilization

• Existing rate control: unaware of this

unique feature

▪ Buffer left empty (0 throughput) for 40% of time! ▪ UL throughput << available bandwidth Video data UL throughput LTE uplink resource scheduling: UL throughput depends on its own buffer level LTE UL firmware buffer Before UL congestion, higher buffer level  higher uplink rate

Solution: Adapt to UL Buffer Level

• Learn relation between UL throughput & buffer level
• Push firmware buffer level to the “sweet” region

▪ Sweet region: maximize throughput without congestion

• Buffer level too high: slow down traffic to avoid congestion
• Buffer level too low: speed up traffic to exploit bandwidth

POI360 System Design

Design Overview

360° Cam Firmware Buffer Buffer level Buffer Aware Rate Control RTP traffic ROI Adaptive Spatial Compression Compressed VR stream Viewer Cellular uplink Sender

Adaptive Spatial Compression

• Adapt compression mode

▪ Balance ROI quality and stability of ROI quality

• Design:
• Switch mode following ROI update responsiveness
• Responsiveness metric: T3-T1 (duration of lower ROI quality)

T2: sender updates ROI knowledge T3: ROI quality recovered

Conservative Aggressive

Video quality Spatial position

T1: ROI change

Buffer Aware Rate Control

▪ Cross-layer design

• Learn buffer’s “sweet” region
• PHY buffer level too high  reduce RTP & video

bitrate

• PHY buffer level too low  increase RTP & video

bitrate

Video bitrate

Application layer

H.264 Encoding Packet Pacer RTP bitrate

Transport layer

Compressed frame PHY bitrate UL Firmware Buffer

Physical layer

Rate Control

PHY buffer level

Implementation

Live stream 360° video VR player LTE phone

Client’s ROI

QXDM

• Diag. interface

Evaluation

Micro-benchmark Evaluation

• Validate VR compression design
• Benchmark algorithm:

▪ CMU--Conduit (crop & send ROI) ▪ Facebook--Pyramid encoding

ROI quality (PSNR) Video-frame-level delay ROI quality stability 11~13dB of improvement Reduce delay by 15% Reduce variation by 5X

Micro-benchmark Evaluation

• Validate our UL buffer-based rate control design

▪ Compare with Google Congestion Control (GCC, default rate control of

Google Hangouts & Facebook Messenger)

▪ Our rate control FBCC keeps UL buffer level in the “sweet” region (green) for most of the time

Low usage High usage Overuse (saturation)

System-Level Test

• Test POI360 system under various network conditions

▪ Different LTE background traffic load ▪ Different physical channel quality ▪ Different mobility level

• Performance metrics

▪ Smoothness

• Video freezing ratio

▪ Quality

• Frame-level PSNR
• Mean Opinion Score(MOS)

Different Background Traffic Load

• Light LTE background traffic load (early morning)

▪ 1% video freeze

• Heavy LTE background traffic load (noon)

▪ 4% video freeze & 2dB PSNR drop ▪ Majority of the frames have either excellent or good quality

PSNR & Video freezing ratio MOS

Different Physical Channel Quality

• Test at places with different channel quality

▪ Weak (-115dB RSS); Moderate (-82dB RSS); Strong (-73dB RSS) ▪ Better channel: higher PSNR & MOS, less video freezes ▪ Even the weak channel achieves <3% video freezes

PSNR & Video freezing ratio MOS

Different Mobility Level

• Test under 3 different mobility levels

▪ Slow (15mph); urban driving (30mph); highway (50mph) ▪ PSNR & MOS drop with higher mobility. But still have good or excellent quality even under 50mph mobility ▪ More freezes with high mobility: 1% for slow driving, 7% for urban driving. Comparable to legacy non-360 LTE video chat

PSNR & Video freezing ratio MOS

POI360 Summary

• Unique challenges when 360° VR video meets LTE

▪ Long RTT of LTE breaks spatial VR compression ▪ Heavy VR traffic load ▪ Low cellular bandwidth utilization

• POI360: the first adaptive 360° VR compression

▪ Adapt compression strategy based on network condition ▪ Achieve balance between traffic load & smoothness ▪ Leverage cellular statistics to enable responsive rate control

• Other works in cellular network-informed mobile applications

* “Accelerating Mobile Web Loading Using Cellular Link Information”, Xiufeng Xie, Xinyu Zhang, Shilin Zhu, ACM MobiSys’17 * “piStream: Physical Layer Informed Adaptive Video Streaming Over LTE”, Xiufeng Xie, Xinyu Zhang, Swarun Kumar, Li Erran Li, ACM MobiCom’15

Thank you!

Q & A