Multidimensional Transcoding for Adaptive Video Streaming Jens - - PowerPoint PPT Presentation

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Multidimensional Transcoding for Adaptive Video Streaming

Jens Brandt, Lars Wolf

Institut f¨ ur Betriebssystem und Rechnerverbund Technische Universit¨ at Braunschweig Germany

NOSSDAV 2007, June 4-5

Jens Brandt, Lars Wolf IBR, TU Braunschweig 1

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Outline

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Jens Brandt, Lars Wolf IBR, TU Braunschweig 2

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Adaptive Media Streaming for Mobile Devices

Mobile devices:

Smartphones, PDAs, pocket multimedia players 300 – 600 MHz CPU 64 – 128 MB RAM Small screen resolutions ⇒ Great heterogeneity of mobile devices

Video streams:

High data rates Complex to decode Fixed encoding parameters

Jens Brandt, Lars Wolf IBR, TU Braunschweig 3

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Video Streaming

Video adaptation to support different devices ⇒ Different versions of video streams Traditional approach: Differently encoded versions of each stream on each server Dynamic approach: Individual adaptation on each server or in the network

Jens Brandt, Lars Wolf IBR, TU Braunschweig 4

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Adaptation Dimensions

Spatial resolution: reducing the resolution of each frame Temporal resolution: reducing the number of frames Detail resolution: increasing the quantization level Bit stream syntax: changing the syntax of the stream Semantic: changing the content of a video . . .

Jens Brandt, Lars Wolf IBR, TU Braunschweig 5

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Decoder - Encoder

Encoding parameters are defined at encoding time Cascaded Pixel Domain Transcoder (CPDT):

Data Manipulation Transcoder Decoder Encoder

very flexible, but computational intensive

Improvements:

Minimizing the needed decoding steps Reusing information from the original data

⇒ Transcoding = manipulation in the compressed domain

Jens Brandt, Lars Wolf IBR, TU Braunschweig 6

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Multidimensional Transcoding

Different specialized transcoding architectures exist A combined architecture is missing Multidimensional transcoding by combining existing transcoder architectures Focus of this work: resolution reduction in three dimensions

Temporal Resolution Spatial Resolution Detail Resolution

Jens Brandt, Lars Wolf IBR, TU Braunschweig 7

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Transcoding in Three Dimensions

Temporal Resolution by

B-Frame Skipping P-Frame Skipping

⇒ Motion Vector Refinement

Spatial Resolution by

Macro Block Aggregation

⇒ Motion Vector Refinement

Detail Resolution by

Requantization

⇒ Drift-Error Correction

Jens Brandt, Lars Wolf IBR, TU Braunschweig 8

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Transcoding in Three Dimensions

Temporal Resolution by

B-Frame Skipping P-Frame Skipping

⇒ Motion Vector Refinement

Spatial Resolution by

Macro Block Aggregation

⇒ Motion Vector Refinement

Detail Resolution by

Requantization

⇒ Drift-Error Correction

⇐ ⇐ ⇐

Inverse Motion Compensation needed

Jens Brandt, Lars Wolf IBR, TU Braunschweig 8

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Transcoder Chain

Jens Brandt, Lars Wolf IBR, TU Braunschweig 9

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Processing Architecture

Pull architecture Passive transcoder modules: Ti One active component: Controller Events: ei

Jens Brandt, Lars Wolf IBR, TU Braunschweig 10

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Implementation

Prototype implementation in C++ Transcoding in three dimensions: temporal, spatial, detail Features:

B-Frame Skipping P-Frame Skipping Downscaling by a factor of 2 Requantization Static Controller MPEG-4 Simple Profile

Jens Brandt, Lars Wolf IBR, TU Braunschweig 11

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Implemented Transcoder Chain

Data exchanged between transcoders = partially decoded Frames

Jens Brandt, Lars Wolf IBR, TU Braunschweig 12

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Frame States

Jens Brandt, Lars Wolf IBR, TU Braunschweig 13

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Testvideos

Source:

12 seconds (300 frames, 25 fps, 30 frames GOV length) CIF (352x288) reolution Encoded by XviD Akiyo Foreman

Transcoded to 150 frames, 12.5 fps, QCIF

Jens Brandt, Lars Wolf IBR, TU Braunschweig 14

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Akiyo - Average PSNR Values

26 27 28 29 30 31 20 40 60 80 100 120 140 160 180 PSNR Bit Rate [KBit/s] Akiyo recoded Akiyo transcoded

Jens Brandt, Lars Wolf IBR, TU Braunschweig 15

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Foreman - Average PSNR Values

20 21 22 23 24 25 26 27 28 29 100 200 300 400 500 600 700 PSNR Bit Rate [KBit/s] Foreman recoded Foreman transcoded

Jens Brandt, Lars Wolf IBR, TU Braunschweig 16

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Runtime Performance

Intel Pentium 4 CPU 3.20GHz, 512 MB RAM: Sequence Processing Time Processed Frames Akiyo 4.31 s 34.80 fps Foreman 10.71 s 14.01 fps Transcoded video: 150 frames, 12.5 fps, 12 s

Jens Brandt, Lars Wolf IBR, TU Braunschweig 17

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Conclusions & Future Work

Flexible processing architecture for multidimensional transcoding Three transcoding dimensions for fine grain adaptation Transcoder implementation for this dimensions Promising evaluation results

Future Work

Optimization to improve runtime Further features of MPEG-4 to reduce bit rate Dynamic rate control mechanisms Quality evaluation by subjective tests

Jens Brandt, Lars Wolf IBR, TU Braunschweig 18

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work

Questions?

Jens Brandt, Lars Wolf {brandt|wolf}@ibr.cs.tu-bs.de Institut f¨ ur Betriebssystem und Rechnerverbund Technische Universit¨ at Braunschweig Germany http://www.ibr.cs.tu-bs.de

Jens Brandt, Lars Wolf IBR, TU Braunschweig 19