Page 1 t : l Transport Problem for AR&VR t draft-han-iccrg-arvr-transport-problem 黑体 Huawei USA, Future Network Lab: Lin Han t Vodafone: Kevin Smith t 黑色 : lar l t t 黑色 细黑体 IETF 98, Chicago, 2017 draft-han-iccrg-arvr-transport-problem HISILICON SEMICONDUCTOR
Page 2 t Goals of this draft : • l Quantify the network problems for network based AR&VR › Bandwidth, Latency, Burst t › Transport technology • 黑体 Attract telnets from different organizations to study and give more solutions › Ecosystem › SDO liaison • Technology ready for AR&VR integrated with Internet and cloud t • Make Internet able to support similar extreme applications (V2X, tactile network, etc) t 黑色 : lar l t t 黑色 细黑体 IETF 98, Chicago, 2017 draft-han-iccrg-arvr-transport-problem HISILICON SEMICONDUCTOR
Page 3 t New Era is coming : 4 th time of tech waves l PC, Internet, Mobile, AR&VR • Has attracted tremendous VC investment t to a lot startup • FB, Google, and many other companies also put a lot resource on it. 黑体 • Some old industry map may be dramatically changed, mobile, computer, TV, game, etc • The market of software and hardware for AR&VR could reach $180bn by 2025 ( Goldman Sachs Global Investment Research 2016 ) t 4 th generation of computing platform t 黑色 Main frame, PC, Smart phone, AR&VR : • What you see from AR&VR is your new desktop, lar and all applications and running results will be l displayed on top of it as objects in virtual world. • What you act to the virtual objects will be the t input to the computer. t 黑色 细黑体 IETF 98, Chicago, 2017 draft-han-iccrg-arvr-transport-problem HISILICON SEMICONDUCTOR
Page 4 t AR&VR integrated with Internet and Cloud : Network based VR l Localized VR t HMD Rendering 黑体 HDMI GW Internet VR Service Cloud VR Service + Local Rendering t t 黑色 HMD : lar GW Internet l VR Service+Rendering t Cloud VR Service + Cloud Rendering t 黑色 http://www.trendhunter.com/trends/backpack-computer 细黑体 IETF 98, Chicago, 2017 draft-han-iccrg-arvr-transport-problem HISILICON SEMICONDUCTOR
Page 5 t New Applications of Network Based AR&VR : VR online Games VR online shopping VR live broadcast l t 黑体 t t 黑色 Oculus focus on development of VR Alibaba is popularizing the concept of NextVR focuses on the VR live : online game . Buy+ , which adopts the VR broadcast operation (e.g. NBA, Boxing, lar technology to provide a interactive 3D presidential debate). It has been acquired by Facebook for online shopping environment for the l $2 billion. It gained $3.5 million investment from customers. Comcast & Time Warner. t t 黑色 细黑体 IETF 98, Chicago, 2017 draft-han-iccrg-arvr-transport-problem HISILICON SEMICONDUCTOR
Page 6 t Technologies ready? : • l Sensor › Action detection/capturing t › Action simulation • Computing 黑体 › Rendering › Color sampling › Coding/decoding › … t • Display t › Super high resolution 黑色 › Super fast response time : • Power Supplier lar › Power consumption l › Heat Dissipation • Network t › Transport technologies t › Throughput, Latency, Packet Loss, etc 黑色 细黑体 IETF 98, Chicago, 2017 draft-han-iccrg-arvr-transport-problem HISILICON SEMICONDUCTOR
Page 7 t Video quality, Encoding and Bitrate : • l Normal Encoder setting › Size t › FPS https://www.sri.com/sites/default/files/publications/3_07_h264_format_bitrate_quality_tradeoff_study.pdf › Color, sampling 黑体 › Profile › Bitrate mode (CBR, VBR) › Pass number › Bitrate, maximum bitrate (VBR) t › GOP distance t › B-frame 黑色 › …. : lar No exact formula for bitrate l t t J.R.Ohm et al.,”Comparison of the Coding Efficiency of Video Coding Standards — Including High Efficiency Video Coding (HEVC )”, 黑色 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 22, NO. 12, DECEMBER 2012 细黑体 IETF 98, Chicago, 2017 draft-han-iccrg-arvr-transport-problem HISILICON SEMICONDUCTOR
Page 8 t Math for Bit rate, Peak bit rate, Burst : Bit rate = W * H * FPS * Rank * 0.07 Adobe, "H.264 Primer", 2016 l W: Pixel number in horizontal direction H: Pixel number in Vertical direction t FPS: Frame per second Rank: Motion rank, it can be Low motion, Medium motion or High motion. 黑体 Low motion = 1, Medium motion = 2, High motion = 4. Low motion: video that has minimal movement Medium motion: video that has some degree of movement High motion: video that has a lot movements and movement is unpredictable. Average Bit rate = T * W * H * S * d * FPS / Cv t Bit rate for I-frame = T * W * H * S * d * FPS / Cj t Burst size = T * W * H * S * d / Cj Burst time = 1/FPS 黑色 T: Type of video, 1 for 2D, 2 for 3D : W: Pixel number in horizontal direction lar H: Pixel number in Vertical direction S: scale factor, l 1 for YUV400, 1.5 for YUV420, 2 for YUV422, 3 for YUV444 d: Color depth bits t FPS: Frame per second Cv: Average Compression ratio for video. t Cj: Compression ratio for I-frame 黑色 细黑体 IETF 98, Chicago, 2017 draft-han-iccrg-arvr-transport-problem Page 8 HISILICON SEMICONDUCTOR
Page 9 t MTP Latency – Motion-to-Photon Latency : 【 1 】 Motion-to-Photon latency is the time needed for a user movement to be fully reflected on a display screen. l (http://www.chioka.in/what-is-motion-to-photon-latency/) t 【 2 】 Motion-to-photon latency also known as the End-to-end latency is the delay between the movement of the user's head and the change of the VR device's display reflecting the user's movement. As soon as the user's head 黑体 moves, the VR scenery should match the movement. The more delay (latency) between these 2 actions, the more unrealistic the VR world seems. To make the VR world realistic, VR systems want low latency of <20ms and even really low latency of <7ms (http://xinreality.com/wiki/Motion-to-photon_latency) t However, still some difference t MTP (from the movement of user header to … Reference 黑色 : Oculus, Atman Binstock the screen starts the display of new image https://developer.oculus.com/blog/optimizing-vr-graphics- lar with-late-latching/ l Oculus, John Carmack the screen finishes the display of new image https://www.twentymilliseconds.com/post/latency-mitigation- strategies t Advanced VR Rendering ， GDC 2015 Valve, Alex Vlachos the screen finishes the display of new image t Qualcomm the screen center starts the display of new image http://www.qualcomm.cn/news/blog-2016-07-20 黑色 细黑体 IETF 98, Chicago, 2017 draft-han-iccrg-arvr-transport-problem HISILICON SEMICONDUCTOR
Page 10 Network Latency is Bigger Challenge t MTP < 20ms : The maximum latency for network device = 5~7 ms – Propagation Delay(200km/ms) Network latency must be User to server distance 500km (round propagation delay= 5ms), < 5-7ms The maximum queuing latency accumulated on all device (one direction) < 1ms l Dynamic refreshing 、 HW parallel TimeWap Total latency of network based VR APP coding t Latency of Localized VR APP ：～ 18ms by current technologies ： >>20ms HDMI 黑体 Network Major optimization for processing time in future VR ： ～ 1ms ~11ms ～ 1ms ～ 5ms t Action capture ~= 1ms T2 ： T1 ： T1 ： T2 ： ROI encoding Action capture Display refreshing ~= 0.01ms （ AMOLED screen ， t Action capture ROI rendering dynamic refreshing 、 TimeWap ） 黑色 ～ 10ms Server coding ~= 2ms （ HW parallel coding ） : 200us-200ms@hop ～ 10ms Streaming re-order ~= 5ms 5us@1KM T6 ： Display lar Nanjing to Beijing: 1000KM Terminal decoding ~= 5ms T4 ： l Display refresh Measured average latency 112ms Network transport ~= 5ms － 7ms T5 ： ～ 2ms t T4 ： T3 ： GOP Terminal network transport framing/streaming decoding t T3 ： Local transmit ～ 5ms ～ 110ms － 1s Latency in future networks must be 黑色 about 5~7 ms, considering the 细黑体 Network based AR/VR APP has 4 more processing than localized APP, technology advances in future Total Latency >> 20ms IETF 98, Chicago, 2017 Latency of Localized VR APP <20ms draft-han-iccrg-arvr-transport-problem HISILICON SEMICONDUCTOR
Page 11 t Summary : • Network needs to support “High throughput + Ultra - short latency + High burst” l • Network quality greatly impacts AR&VR User Experience t • Much lower tolerance of packet loss or delay than watching HD TV. 黑体 • Compromise of network quality may cause CyberSickness t t 黑色 : lar l t t 黑色 细黑体 IETF 98, Chicago, 2017 draft-han-iccrg-arvr-transport-problem HISILICON SEMICONDUCTOR
Recommend
More recommend
