In the name of Allah f the compassionate, the merciful p , - PowerPoint PPT Presentation

In the name of Allah f the compassionate, the merciful p ,

Digital Video Systems g y S. Kasaei S. Kasaei Room: CE 307 Room: CE 307 Department of Computer Engineering Sharif University of Technology E-Mail: skasaei@sharif.edu a s asae @s a edu Webpage: http://sharif.edu/~skasaei Lab. Website: http://ipl.ce.sharif.edu

Acknowledgment g Most of the slides used in this course have been provided by: Prof. Yao Wang (Polytechnic University, Brooklyn) based on the book: Video Processing & Communications written by: Yao Wang, Jom Ostermann, & Ya-Oin Zhang Prentice Hall, 1 st edition, 2001, ISBN: 0130175471. [SUT Code: TK 5105 .2 .W36 2001].

Chapter 9 p Waveform-Based Coding: g Transform & Predictive Coding

Outline � Overview of video coding systems � Overview of video coding systems � Transform coding � Predictive coding Predictive coding 6 Kasaei

Components in a Coding System Focus of this lecture. 7 Kasaei

Video Coding � Reduces the data rate of a video sequence by Reduces the data rate of a video sequence by exploiting the spatial & spectral correlation between neighboring pixels as well as temporal correlation b t between consecutive frames of the sequence. ti f f th � Spatial & spectral correlation is due to the fact that color values of adjacent pixels in the same video frame usually j p y change smoothly. � Temporal correlation refers to the fact that consecutive frames usually show the same physical scene occupied by frames usually show the same physical scene, occupied by the same object that may have moved. 8 Kasaei

Transform Coding � Motivation: � Motivation: � Represent a vector ( e.g., a block of image pels) as the superposition of some typical vectors p p yp (basic patterns, transform basic functions). � Quantize & code the coefficients. � Can be thought of as a constrained vector quantizer. transform basis transform coefficients + t 1 t 2 t 3 t 4 9 Kasaei

Transform Coding Block Diagram 10 Kasaei

Which Transform Basis to Use? � The transform should: � The transform should: � Minimize the correlation among resulting coefficients, so that scalar quantization can be , q employed without losing too much in coding efficiency (compared to vector quantization). � Compact the energy into as few coefficients as possible. 11 Kasaei

Which Transform Basis to Use? � Optimal transform: Optimal transform: � Karhunen-Loeve transform (KLT): signal statistics dependent (ensemble dependent, 2 nd order statistics). t ti ti ) � Suboptimal transform: � Discrete cosine transform (DCT): nearly as good as Discrete cosine transform (DCT): nearly as good as the KLT for common image signals (highly correlated signals). � Wavelet transform (WT): a powerful multiresolution tool that utilizes space-frequency properties of non- stationary signals stationary signals. 12 Kasaei

General Linear Transform � Linearly independent basis vectors (or blocks): � Linearly independent basis vectors (or blocks): � Inverse transform represents a vector (or � Inverse transform represents a vector (or block) as the superposition of basis vectors (or blocks): blocks): � Forward transform determines the contribution � Forward transform determines the contribution (weight) of each basis vector: s t s V U 13 Kasaei

Unitary Transform � Unitary (orthonormal) basis: � Unitary (orthonormal) basis: � Basis vectors are orthogonal to each other & each has length 1: g � Transform coefficient associated with a basis vector is simply the projection of the input vector is simply the projection of the input vector onto the basis vector: 14 Kasaei

Discrete Cosine Transform Basis Images 15 Kasaei

Energy Distribution of DCT Coefficients in Typical Images 16 Kasaei

Images Approximated by Different Number of DCT Coefficients With 16/64 Original Original Coefficients With 8/64 With 4/64 Coefficients Coefficients 17 Kasaei

Demos � Use Matlab demo to demonstrate � Use Matlab demo to demonstrate approximation by using different number of DCT coefficients. (dctdemo.m) 18 Kasaei

Distortion in Transform Coding � Distortion in sample domain: � Distortion in sample domain: � Distortion in coefficient domain: � These two distortions equal when using � These two distortions equal when using unitary transform: 19 Kasaei

Modeling of Distortion due to Coefficient Quantization � High Resolution Approximation of Scalar � High Resolution Approximation of Scalar Quantization: � With the MMSE quantizer, when each coefficient � With the MMSE quantizer, when each coefficient is scalar quantized with sufficiently high rates, so that the pdf in each quantization bin is approximately flat we have: i l fl h Depends on the pdf of the k th coefficient Depends on the pdf of the k -th coefficient. 20 Kasaei

Optimal Bit Allocation among Coefficients � How many bits to use for each coefficient? � How many bits to use for each coefficient? � Can be formulated as an constrained optimization problem: p Minimize: Subject to: � The constrained problem can be converted to unconstrained one using the Lagrange multiplier method: Minimize: 21 Kasaei

Derivation & Result 22 Kasaei

Implication of Optimal Bit Allocation � Bitrate for a coefficient proportional to its variance Bitrate for a coefficient proportional to its variance (energy): geometric mean � Larger variance, more bits. g � Distortion is equalized among all coefficients & depends on the geometric mean of the coefficient variances: i 23 Kasaei

Transform Coding Gain over PCM � Distortion for PCM (SQ samples) if each sample is Distortion for PCM (SQ samples) if each sample is quantized to R bit: � Gain over PCM: arithmetic mean geometric mean 24 Kasaei

Transform Coding Gain over PCM � For any arbitrary set of values, the arithmetic mean For any arbitrary set of values, the arithmetic mean is equal to (or larger than) the geometric mean ( � G >=1). � For Gaussian sources: � If each sample is Gaussian, then coefficients are also p , Gaussian (& are all the same). Then: 25 Kasaei

Example � Determine the optimal bit allocation & corresponding TC p p g gain for coding 2x2 image block, using 2x2 DCT. Assume that the image is a stationary Gaussian process with inter-sample correlation as shown below. ith i t l l ti h b l 26 Kasaei

Example (cnt’d) � Correlation matrix: zero-mean data. Correlation matrix: zero mean data. � DCT basis images: � Equivalent transform matrix: 27 Kasaei

Example (cnt’d) covar. matrix geo. mean g 0 For R=2 : For R=2 : 28 Kasaei

Optimal Transform Design � Optimal transform: p � Should minimize the distortion for a given average bitrate. � Equivalent to minimize the geometric mean of the coefficient variances variances. � When the source is Gaussian, the optimal transform is the KLT (which depends on the covariance matrix of the samples), ensemble dependent. � Basis vectors are the eigenvectors of the covariance matrix, the coefficient variances are the eigenvalues: k- th coeff. var. KLT unit. minimal geo. mean maximize TC 29 Kasaei

Example � Determine the KLT for the 2x2 image block in the g previous example. Determine the eigenvalues by solving: (Same as the coefficient variances with DCT!) Determine the eigenvectors by solving: Resulting transform is the DCT! Resulting transform is the DCT! 30 Kasaei

Data Compression Symbol Data Quantization Encoding Transform O t Output t Original O i i l Bit Stream Image Transform coding s stem Transform coding system. 31 4/17/2007 S. Kasaei

JPEG-98 Compression Run- 8 × 8 Scalar Length DCT Pixel Pixel Quantizer Quantizer & & Block Huffman Original Output I Image Bitstream JPEG 98 encoder JPEG-98 encoder. 32 4/17/2007 S. Kasaei

DCT Transform DCT transform: � + + C C 7 7 2 x 1 2 y 1 ∑∑ = π π u v T S (cos u )(cos v ) u , v x , y 4 4 16 16 16 16 = = y 0 0 x 0 0  2 u, v = 0  = = 2 2 C C u C , C   v  otherwise 1  S: source block, T: transformed block. T has the same size as S with real coefficients. 33 4/17/2007 S. Kasaei

DCT Transform � In matrix T, elements , Lower Lower containing high energy Frequency levels (with low frequencies) are placed in frequencies) are placed in upper left corner. � When going towards lower Higher right corner, energy level Frequency q y goes down (& frequency goes down (& frequency goes up). DCT coefficients. 34 4/17/2007 S. Kasaei

Scalar Quantization Quantization tables used by JPEG (entries are the stepsizes) Quantization tables used by JPEG (entries are the stepsizes). 35 4/17/2007 S. Kasaei

Scalar Quantization � Quantized matrix has the same size as S & T � Quantized matrix has the same size as S & T, with elements of: T T QT = Q Q: quantization matrix, QT: quantized T. 36 4/17/2007 S. Kasaei