Power and Bandwidth Optimization in 360-Degree Immersive Mobile - - PowerPoint PPT Presentation

▶
power and bandwidth optimization in 360 degree immersive
SMART_READER_LITE
LIVE PREVIEW

Power and Bandwidth Optimization in 360-Degree Immersive Mobile - - PowerPoint PPT Presentation

Aug 29, 2023 471 likes •717 views

Power and Bandwidth Optimization in 360-Degree Immersive Mobile Video Streaming Sheng Wei, Assistant Professor Rutgers ECE May 2019 Context: Research Paths Hardware Security . . Mobile Video Systems Assistant Professor Assistant Professor

slide-1
SLIDE 1

Power and Bandwidth Optimization in 360-Degree Immersive Mobile Video Streaming

Sheng Wei, Assistant Professor Rutgers ECE May 2019

slide-2
SLIDE 2

Context: Research Paths

6/25/2019 2

Hardware Security Mobile Video Systems 2008 2010 2015 2018

. .

PhD Student Intern/Research Scientist Assistant Professor Assistant Professor

slide-3
SLIDE 3

All about Videos

6/25/2019 3

2D Video 3D Video Immersive Video Volumetric Video (VR) (AR)

slide-4
SLIDE 4

The 360-degree Challenges

6/25/2019 4

Power Bandwidth

slide-5
SLIDE 5

Power Profiling

6/25/2019 5

slide-6
SLIDE 6

Power Optimization Solution

6/25/2019 6

 Key Idea: Reducing the framerate  The Problem: Video quality degradation  Solution: Reducing framerate during switching only

(Courtesy of F. Qian et al.)

slide-7
SLIDE 7

Power Optimization Framework: QuRate

6/25/2019 7

Video Quality Framerate

slide-8
SLIDE 8

Power Optimization Results

6/25/2019 8

 6-video, 59-user head movement dataset

Frequency of View Switching

slide-9
SLIDE 9

Power Optimization Results

6/25/2019 9

Power Quality Battery Life

slide-10
SLIDE 10

Future Work

6/25/2019 10

Content Distribution Network Edge Node Remote Media Server (x, y, z)

Edge Computing Hardware Acceleration Viewport Prediction

slide-11
SLIDE 11

Viewport Prediction

6/25/2019 11

 Bandwidth/Power Optimization Opportunity

slide-12
SLIDE 12

Problem Definition

6/25/2019 12

 Scenario: Live 360-degree Video Streaming  Problem: Predict the user viewport for a few seconds

slide-13
SLIDE 13

Solution Space

6/25/2019 13

slide-14
SLIDE 14

Challenges

6/25/2019 14

slide-15
SLIDE 15

LiveMotion: Motion-tracking-based Viewport Prediction

6/25/2019 15

 Key Idea: User watches moving objects, so let’s track the motion

[Ubicomp 19]

slide-16
SLIDE 16

LiveMotion Results

6/25/2019 16

slide-17
SLIDE 17

LiveMotion Limitations

6/25/2019 17

 Does not work for videos shot by moving cameras  Does not work for complicated moving background  User may want to watch non-moving objects

Solution: Let’s understand the users better

slide-18
SLIDE 18

LiveObj: Object-semantics-based viewport prediction

6/25/2019 18

 Key Idea: user watches meaningful objects

slide-19
SLIDE 19

LiveObj Design

6/25/2019 19

slide-20
SLIDE 20

LiveObj Results

6/25/2019 20

Public head movement dataset involving 48 users watching 10 videos

slide-21
SLIDE 21

Future work in viewport prediction

6/25/2019 21

 Accomplished

 User watches moving objects → LiveMotion [ubicomp19]

 In Progress

 User watches meaningful objects → LiveObj

 Future Work

 User watches meaningful actions → LiveROI  User watches video content based on audio guide → LiveAudio

 User/Content-based adaptive viewport algorithm selection

LiveMotion LiveObj LiveRoI LiveAudio ?

slide-22
SLIDE 22

Future work: Volumetric Video Security

6/25/2019 22

Volumetric Video  Security/Privacy concerns  Threat model

 Bypassing face ID authentication

 Solution

 Encryption (too much overhead)  Proposed: Opposite use of adversarial attack

slide-23
SLIDE 23

Acknowledgement

6/25/2019 23

Project Repositories: gitlab.com/hwsel