Power and Bandwidth Optimization in 360-Degree Immersive Mobile - PowerPoint PPT Presentation

Power and Bandwidth Optimization in 360-Degree Immersive Mobile Video Streaming Sheng Wei, Assistant Professor Rutgers ECE May 2019

Context: Research Paths Hardware Security . . Mobile Video Systems Assistant Professor Assistant Professor PhD Student Intern/Research Scientist 2008 2010 2015 2018 2 6/25/2019

All about Videos (AR) (VR) Volumetric Immersive 2D Video 3D Video Video Video 3 6/25/2019

The 360-degree Challenges Power Bandwidth 4 6/25/2019

Power Profiling 5 6/25/2019

Power Optimization Solution  Key Idea: Reducing the framerate  The Problem: Video quality degradation  Solution: Reducing framerate during switching only (Courtesy of F. Qian et al.) 6 6/25/2019

Power Optimization Framework: QuRate Video Quality Framerate 7 6/25/2019

Power Optimization Results  6-video, 59-user head movement dataset Frequency of View Switching 8 6/25/2019

Power Optimization Results Power Quality Battery Life 9 6/25/2019

Future Work (x, y, z) Content Distribution Network Edge Node Remote Media Server Hardware Viewport Prediction Edge Computing Acceleration 10 6/25/2019

Viewport Prediction  Bandwidth/Power Optimization Opportunity 11 6/25/2019

Problem Definition  Scenario: Live 360-degree Video Streaming  Problem: Predict the user viewport for a few seconds 12 6/25/2019

Solution Space 13 6/25/2019

Challenges 14 6/25/2019

[Ubicomp 19] LiveMotion: Motion-tracking-based Viewport Prediction  Key Idea: User watches moving objects, so let’s track the motion 15 6/25/2019

LiveMotion Results 16 6/25/2019

LiveMotion Limitations  Does not work for videos shot by moving cameras  Does not work for complicated moving background  User may want to watch non-moving objects Solution: Let’s understand the users better 17 6/25/2019

LiveObj: Object-semantics-based viewport prediction  Key Idea: user watches meaningful objects 18 6/25/2019

LiveObj Design 19 6/25/2019

LiveObj Results Public head movement dataset involving 48 users watching 10 videos 20 6/25/2019

Future work in viewport prediction  Accomplished  User watches moving objects → LiveMotion [ubicomp19]  In Progress  User watches meaningful objects → LiveObj  Future Work  User watches meaningful actions → LiveROI  User watches video content based on audio guide → LiveAudio  User/Content-based adaptive viewport algorithm selection LiveMotion LiveRoI ? LiveObj LiveAudio 21 6/25/2019

Future work: Volumetric Video Security  Security/Privacy concerns  Threat model  Bypassing face ID authentication  Solution  Encryption (too much overhead)  Proposed: Opposite use of adversarial attack Volumetric Video 22 6/25/2019

Acknowledgement Project Repositories: gitlab.com/hwsel 23 6/25/2019