Time Domain Lapped Transform and its Applications Jie Liang - PowerPoint PPT Presentation

Jie Liang Simon Fraser University Time Domain Lapped Transform and its Applications Jie Liang (jiel@sfu.ca) School of Engineering Science Simon Fraser University Vancouver, BC, Canada http://www.ensc.sfu.ca/people/faculty/jiel/ BIRS Workshop on Multimedia and Mathematics Acknowledgements Prof. Trac D. Tran, The Johns Hopkins University Dr. Chengjie Tu, Microsoft Corporation Dr. Lu Gan, The University of Newcastle, Australia Banff, AB, Canada, July 23-28, 2005 �

Jie Liang Simon Fraser University Outline � Introduction � From filter bank to time-domain lapped transform (TDLT) � Fast TDLT � Pre/post-filtering for 2D and 3D Wavelet transform � Error Resilient TDLT � Current Works � Summary Banff July 27, 2005 �

Jie Liang Simon Fraser University Filter Bank Fundamental x [ n ] H 0 z F 0 z ( ) M ( ) M M M Processing H 1 z F 1 z ( ) M M M M ( ) x ˆ n [ ] H M − 1 z F M − 1 z ( ) M M ( ) M M       � � � �       y 0 P � P P 0 x       − m K m 1 1 0       = P � y 0 P P 0 x + − + m K m       1 1 1 0 1       y 0 P � P P x + − + m K m 2 1 1 0 2             � � � � � = + + + y P x P x � P x − − − + m m m K m K 0 1 1 1 1 ( ) + = + + + = z z z z -1 -K 1 � Y ( ) P P z P z X ( ) P ( ) X ( ) K − 0 1 1 Banff July 27, 2005 �

Jie Liang Simon Fraser University Filter Bank Fundamental x [ n ] H 0 z F 0 z ( ) M M M ( ) M H 1 z F 1 z ( ) M M ( ) M M ˆ n x [ ] H M − 1 z F M − 1 z ( ) M M ( ) M M 0 ˆ z ( z ( z x ) x ( ) y ) 1 P (z) R (z) M-1 � Perfect Reconstruction: R (z) P (z) = I , or R (z) = P -1 (z). � Fast Implementation: Factorization of P (z) … … x y 1 z 1 z 0 z P (z) G G G ( ) ( ) ( ) m m K − … Banff July 27, 2005 �

Jie Liang Simon Fraser University Linear Phase Filter Banks � Linear phase: � Desired property for image/video coding � General structure [Vaidyanathan93, Gao01, Gan01] U 0 − 1 z 2 z 1 z G ( ) G ( ) K − − V 1 z V 0 1 1 z G G ( ) 0 U i V , : Invertible matrices. i Can be optimized for different applications. Banff July 27, 2005 �

Jie Liang Simon Fraser University Rate-Distortion Optimization � Objective: Design the filter bank to minimize the MSE for a given bit rate. = + ˆ x x e x -1 { } P (z) P (z) δ = 2 trace R P ee Q � Design Criterion: Coding Gain MSE reduction of transform coding w.r.t. PCM   σ 2 γ = P  CM  10log σ 10 2   P Banff July 27, 2005 �

Jie Liang Simon Fraser University A Special Case: Block Transform x x − P P ˆ 1 Q m m ( z P P ) x x − P P ˆ 1 Q + m + m 1 1 � Karehunen-Loeve Transform (KLT): � Optimal block transform � Signal dependent, No fast algorithm � Discrete Cosine Transform (DCT): � Fast approx. of the KLT for AR(1) signals. � Drawbacks of block transform: � Blocking artifact � Limited compression capability Banff July 27, 2005 �

Jie Liang Simon Fraser University Lapped Transform [Malvar et al . 1985] � Apply post-processing of the DCT to � Improve compression efficiency and reduce blocking artifact A special case of linear phase filter banks � … … IDCT DCT Q V − V 1 IDCT DCT Q V − V 1 IDCT DCT … … … … post-processing pre-processing Banff July 27, 2005 �

Jie Liang Simon Fraser University Time-Domain Lapped Transform � [Tran-2001] Q DCT IDCT V − V 1 Q IDCT DCT V − V 1 IDCT DCT Q � More compatible to DCT-based schemes � Also a special case of linear phase filter banks � Adopted by MS WMV-9, SMPTE VC-1, HD-DVD. Banff July 27, 2005 �

Jie Liang Simon Fraser University Effect of Prefiltering � A flattened image Banff July 27, 2005 ��

Jie Liang Simon Fraser University DCT vs LT: Basis Functions 8-point DCT 8 x 16 TDLT Banff July 27, 2005 ��

Jie Liang Simon Fraser University Frequency Responses 8-point DCT 8 x 16 TDLT 8.83 dB 9.61 dB DC Att. ≥ 409.0309, Mirr Att. ≥ 320.1639, Stopband ≥ 9.9559, Coding Gain = 8.8259 dB DC Att. ≥ 322.1021, Mirr Att. ≥ 305.2548, Stopband ≥ 11.9696, Coding Gain = 9.6115 dB 5 5 0 0 -5 -5 Magnitude Response (dB) Magnitude Response (dB) -10 -10 -15 -15 -20 -20 -25 -25 -30 -30 -35 -35 -40 -40 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0 0.1 0.2 0.3 0.4 0.5 Normalized Frequency Normalized Frequency Banff July 27, 2005 ��

Jie Liang Simon Fraser University Applications & Generalizations � Fast TDLT [Liang et al. 2001] � Pre/Post-filtering for Wavelet [Liang et al. 2003] Error Resilient TDLT [Tu et al. 2002, Liang et al. 2005] � � Generalized Lapped Transform [Liang et al. 2002] � Adaptive Entropy Coding for TDLT [Tu et al. 2001] � Oversampled TDLT [Gan-Ma-2002] � Undersampled TDLT [Tu et al. 2004] � Regularity Constrained TDLT [Dai et al. 2001] � Adaptive TDLT [Dai et al. 2005] Banff July 27, 2005 ��

Jie Liang Simon Fraser University Outline � Introduction � Fast TDLT � Pre/post-filtering for 2D and 3D Wavelet transform � Error Resilient TDLT � Current Works � Summary Banff July 27, 2005 ��

Jie Liang Simon Fraser University Fast Orthogonal TDLT M x M Prefilter: 0 1 M/2 V M-1 − 1 = T V V � � General structure of V : θ cos θ 0 − θ sin θ θ θ θ θ 1 sin 2 3 4 θ θ cos 5 Banff July 27, 2005 ��

Jie Liang Simon Fraser University Fast Orthogonal TDLT � Quasi-optimal coding gain [Liang-Tran-Tu-01]: � 9.26 dB for M = 8 � Close to optimal filter bank � Can be generalized to large block size (e.g., 128) -0.17 π -0.12 π -0.05 π Banff July 27, 2005 ��

Jie Liang Simon Fraser University Fast Biorthogonal TDLT − ≠ T 1 V V : More freedoms, better performance � � Fast approximation (lifting steps, LU factorization): s0 P2 U2 s1 s2 P1 U1 s3 P0 U0 � Integer Solutions: > 0.3 dB higher than orthogonal TDLT Gain S0 S1 S2 S3 P0 U0 P1 U1 P2 U2 4/3 8/7 8/7 8/7 -1/16 1 / 4 -1/4 1 / 2 -3/8 3 / 4 9.59 3/2 9/8 9/8 9/8 -1/16 1 / 4 -1/4 1 / 2 -3/8 3 / 4 9.58 1 1 1 1 0 1 / 4 -1/4 1 / 2 -1 / 2 3 / 4 9.37 Banff July 27, 2005 ��

Jie Liang Simon Fraser University Image Coding Performance � TDLT vs Wavelet � Both coded by SPIHT [Said, Pearlman, 1996] � (Improved entropy coding in [Tu, Tran, 2001]) 42 WT TDLT 40 38 Lena 36 34 PSNR 32 30 > 1.5dB 28 26 Barb 24 22 0 10 20 30 40 50 60 70 80 90 100 Comp. Ratio Banff July 27, 2005 ��

Jie Liang Simon Fraser University Image Coding Performance � TDLT 32:1: 28.95 dB � WT 32:1: 27.58 dB Banff July 27, 2005 ��

Jie Liang Simon Fraser University Outline � Introduction � Fast TDLT � Pre/post-filtering for 2D and 3D Wavelet transform � Error Resilient TDLT � Current Works � Summary Banff July 27, 2005 ��

Jie Liang Simon Fraser University JPEG 2000 & Tiling Artifact � No blocking artifact if WT is applied to the entire image � Used by JPEG 2000 � Problem: Memory requirement � Tradeoff: Tiling approach Tiling Artifact Tile size: 64 x 64, 0.2bpp Banff July 27, 2005 ��

Jie Liang Simon Fraser University Average MSE of All Rows & Columns � MSE is more than doubled at tile boundaries 400 300 MSE 200 100 0 0 50 100 150 200 250 300 350 400 450 500 X Pixel 300 200 MSE 100 0 0 50 100 150 200 250 300 350 400 450 500 Y Pixel Banff July 27, 2005 ��

Jie Liang Simon Fraser University Pre/post-filtering for WT � Apply small pre/post filters at tile boundaries … I W Q W T T -1 P P I … … W Q W T T -1 P P I W W Q T T Banff July 27, 2005 ��

Jie Liang Simon Fraser University Problem Formulation k LP � Effect of pre/post-filters: 1 γ + � In LT: affect all subbands z − γ + z ( 1 ) 1 ( 1 ) 1 2 � In WT: affect some subbands k HP 2 5/3 WT x 7 y 6 x 6 y 5 x 5 x y 4 ˆ x 4 4 x y 3 x 3 ˆ 3 x y 2 x 2 ˆ 2 x y 1 x 1 ˆ 1 y 0 x 0 x ˆ -1 0 P P Banff July 27, 2005 ��

Jie Liang Simon Fraser University Problem Formulation � Boundary Filter Bank � Optimization can be performed similar to LT u u { . 0 0 . . Processing . x { . { . . F 0 0 . . . . y y . . G 0 0 0 . . -1 P P Processing { { { y y 1 G 1 1 . . . . x F 1 . . . . . . . . 1 . . . u u 1 1 Banff July 27, 2005 ��

Jie Liang Simon Fraser University Fast Structure � Optimal pre/post-filters for WT and DCT are similar 1/2 1/2 � � � 1/2 1/2 S 0 1/2 S � U 1/2 ��� � � � 1/2 S ��� S ��� U 0 1/2 � Examples: S = [5/3, 4/3, 6/5, 9/8], U = [1/8, 1/4, 5/8], S = [2, 1, 1, 1], U = [1/8, 1/4, 1/2], (lossless) Banff July 27, 2005 ��