SLIDE 1
Zoomable Video Playback
- n Mobile Devices by
Zoomable Video Playback
- n Mobile Devices by
Zoomable Video Playback on Mobile Devices by Selective Decoding - - PowerPoint PPT Presentation
Zoomable Video Playback on Mobile Devices by Selective Decoding - - PowerPoint PPT Presentation
Zoomable Video Playback on Mobile Devices by Selective Decoding Feipeng Liu and Wei Tsang Ooi National University of Singapore Zoomable Video Playback on Mobile Devices by Selective Decoding Feipeng Liu and Wei Tsang Ooi National University
SLIDE 2
SLIDE 3
UHDTV Resolution Chart, from Wikipedia
SLIDE 4
bits captured > bits displayed
SLIDE 5
NUS.SoC.CS2105 Ooi Wei Tsang
We want to zoom and pan in videos, just like in photos and Web pages
SLIDE 6
SLIDE 7
Research Issues: compression streaming interaction :
SLIDE 8
Zoomable Video Playback
- n Mobile Devices by
Selective Decoding
Feipeng Liu and Wei Tsang Ooi
National University of Singapore
SLIDE 9
decode scale
SLIDE 10
SLIDE 11
ROI
SLIDE 12
decode crop scale
SLIDE 13
SLIDE 14
decode crop scale
SLIDE 15
Zoomable Video Playback
- n Mobile Devices by
Selective Decoding
Feipeng Liu and Wei Tsang Ooi
National University of Singapore
SLIDE 16
need to save computation and power as much as possible
SLIDE 17
decode crop scale
not necessary to decode this area
SLIDE 18
Zoomable Video Playback
- n Mobile Devices by
Selective Decoding
Feipeng Liu and Wei Tsang Ooi
National University of Singapore
SLIDE 19
decode crop scale
how to selectively decode the macroblocks in the ROI?
SLIDE 20
decode crop scale
need to decode the macroblocks of the region plus other macroblocks that it depends on
SLIDE 21
for each macroblock m if m is in ROI or m is needed by m’ in ROI (curr or future frames) decode m
SLIDE 22
for each macroblock m if m is in ROI or m is needed by m’ in ROI (curr or future frames) decode m
SLIDE 23
Questions:
- 1. how to check if m is
needed by m’ in ROI?
- 2. how to reduce the
number of such m?
SLIDE 24
Requirements:
- 1. work with standard codec
- 2. no re-encoding of video
SLIDE 25
Our approach:
video meta-data
preprocess selective decoding
User Input ROI
SLIDE 26
video meta-data
preprocess selective decoding
User Input ROI
SLIDE 27
how it works with MPEG-4 SP
(can be generalized to other codec)
SLIDE 28
MB
starting bit position ending bit position AC/DC prediction direction MV values
meta-data
=
SLIDE 29
Our approach:
video meta-data
preprocess selective decoding
User Input ROI
SLIDE 30
construct inter-frame dependency graph by tracing the motion vectors
SLIDE 31
MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB
frame N frame N-1 frame N-2
SLIDE 32
construct intra-frame dependency graph by tracing the AC/DC prediction directions
SLIDE 33
MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB MB
frame N frame N-1 frame N-2
SLIDE 34
Questions:
- 1. how to check if m is
needed by m’ in ROI? Answer: lookup the data structure
SLIDE 35
Questions:
- 2. how to reduce the
amount of dependencies? Answer: storing AC/DC prediction directions and MV vectors
SLIDE 36
for each macroblock m if m is in ROI or m is needed by m’ in ROI (curr or future frames) mark m for decoding for each marked macroblock m seek to m decode and display m
SLIDE 37
Recall: aim to save computation and power as much as possible
SLIDE 38
7.5 15 22.5 30 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Frame Rate (fps) ROI Size (ratio to video size)
Standard Decoding Selective Decoding
SLIDE 39
0.02 0.04 0.06 0.08 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 J/frame ROI Size (ratio to video size)
Standard Decoding Selective Decoding
CPU power consumption (by PowerTutor)
SLIDE 40
at the cost of huge meta-data file (up to 5 times the video size)
SLIDE 41