Multidimensional Transcoding for Adaptive Video Streaming Jens - PowerPoint PPT Presentation

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Multidimensional Transcoding for Adaptive Video Streaming Jens Brandt, Lars Wolf Institut f¨ ur Betriebssystem und Rechnerverbund Technische Universit¨ at Braunschweig Germany NOSSDAV 2007, June 4-5 Jens Brandt, Lars Wolf IBR, TU Braunschweig 1

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Outline Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Jens Brandt, Lars Wolf IBR, TU Braunschweig 2

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Adaptive Media Streaming for Mobile Devices Mobile devices: Smartphones, PDAs, pocket multimedia players 300 – 600 MHz CPU 64 – 128 MB RAM Small screen resolutions ⇒ Great heterogeneity of mobile devices Video streams: High data rates Complex to decode Fixed encoding parameters Jens Brandt, Lars Wolf IBR, TU Braunschweig 3

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Video Streaming Video adaptation to support different devices ⇒ Different versions of video streams Traditional approach: Differently encoded versions of each stream on each server Dynamic approach: Individual adaptation on each server or in the network Jens Brandt, Lars Wolf IBR, TU Braunschweig 4

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Adaptation Dimensions Spatial resolution: reducing the resolution of each frame Temporal resolution: reducing the number of frames Detail resolution: increasing the quantization level Bit stream syntax: changing the syntax of the stream Semantic: changing the content of a video . . . Jens Brandt, Lars Wolf IBR, TU Braunschweig 5

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Decoder - Encoder Encoding parameters are defined at encoding time Cascaded Pixel Domain Transcoder (CPDT): Transcoder Data Decoder Encoder Manipulation very flexible, but computational intensive Improvements: Minimizing the needed decoding steps Reusing information from the original data ⇒ Transcoding = manipulation in the compressed domain Jens Brandt, Lars Wolf IBR, TU Braunschweig 6

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Multidimensional Transcoding Different specialized transcoding architectures exist A combined architecture is missing Multidimensional transcoding by combining existing transcoder architectures Focus of this work: resolution reduction in three dimensions Temporal Resolution Spatial Resolution Detail Resolution Jens Brandt, Lars Wolf IBR, TU Braunschweig 7

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Transcoding in Three Dimensions Temporal Resolution by B-Frame Skipping P-Frame Skipping ⇒ Motion Vector Refinement Spatial Resolution by Macro Block Aggregation ⇒ Motion Vector Refinement Detail Resolution by Requantization ⇒ Drift-Error Correction Jens Brandt, Lars Wolf IBR, TU Braunschweig 8

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Transcoding in Three Dimensions Temporal Resolution by B-Frame Skipping P-Frame Skipping ⇐ ⇒ Motion Vector Refinement Spatial Resolution by Inverse Motion Macro Block Aggregation ⇐ Compensation ⇒ Motion Vector Refinement Detail Resolution by needed Requantization ⇐ ⇒ Drift-Error Correction Jens Brandt, Lars Wolf IBR, TU Braunschweig 8

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Transcoder Chain Jens Brandt, Lars Wolf IBR, TU Braunschweig 9

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Processing Architecture Pull architecture Passive transcoder modules: T i One active component: Controller Events: e i Jens Brandt, Lars Wolf IBR, TU Braunschweig 10

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Implementation Prototype implementation in C++ Transcoding in three dimensions: temporal, spatial, detail Features: B-Frame Skipping P-Frame Skipping Downscaling by a factor of 2 Requantization Static Controller MPEG-4 Simple Profile Jens Brandt, Lars Wolf IBR, TU Braunschweig 11

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Implemented Transcoder Chain Data exchanged between transcoders = partially decoded Frames Jens Brandt, Lars Wolf IBR, TU Braunschweig 12

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Frame States Jens Brandt, Lars Wolf IBR, TU Braunschweig 13

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Testvideos Source: 12 seconds (300 frames, 25 fps, 30 frames GOV length) CIF (352x288) reolution Encoded by XviD Akiyo Foreman Transcoded to 150 frames, 12.5 fps, QCIF Jens Brandt, Lars Wolf IBR, TU Braunschweig 14

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Akiyo - Average PSNR Values Akiyo recoded Akiyo transcoded 31 30 29 PSNR 28 27 26 0 20 40 60 80 100 120 140 160 180 Bit Rate [KBit/s] Jens Brandt, Lars Wolf IBR, TU Braunschweig 15

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Foreman - Average PSNR Values 29 Foreman recoded Foreman transcoded 28 27 26 25 PSNR 24 23 22 21 20 0 100 200 300 400 500 600 700 Bit Rate [KBit/s] Jens Brandt, Lars Wolf IBR, TU Braunschweig 16

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Runtime Performance Intel Pentium 4 CPU 3.20GHz, 512 MB RAM: Sequence Processing Time Processed Frames Akiyo 4.31 s 34.80 fps Foreman 10.71 s 14.01 fps Transcoded video: 150 frames, 12.5 fps, 12 s Jens Brandt, Lars Wolf IBR, TU Braunschweig 17

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Conclusions & Future Work Flexible processing architecture for multidimensional transcoding Three transcoding dimensions for fine grain adaptation Transcoder implementation for this dimensions Promising evaluation results Future Work Optimization to improve runtime Further features of MPEG-4 to reduce bit rate Dynamic rate control mechanisms Quality evaluation by subjective tests Jens Brandt, Lars Wolf IBR, TU Braunschweig 18

Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Questions? Jens Brandt, Lars Wolf { brandt|wolf } @ibr.cs.tu-bs.de Institut f¨ ur Betriebssystem und Rechnerverbund Technische Universit¨ at Braunschweig Germany http://www.ibr.cs.tu-bs.de Jens Brandt, Lars Wolf IBR, TU Braunschweig 19