Sparsity Based Spectral Embedding: Application to Multi-Atlas - PowerPoint PPT Presentation

Sparsity Based Spectral Embedding: Application to Multi-Atlas Echocardiography Segmentation � Ozan Oktay, Wenzhe Shi, Jose Caballero, � Kevin Keraudren, and Daniel Rueckert � � Department ¡of ¡Compu.ng ¡ � Imperial ¡College ¡London ¡ � � � Challenge on Endocardial Three-dimensional � Ultrasound Segmentation, MICCAI 2014 � 14 th September 2014 �

STMI’14 – September 2014 � 2 � LA � RA � Problem Definition and Literature Review � RV � Problem: � LV � Left ventricle (LV) endocardium segmentation in 3D Echo Images � � Motivation: � Estimation of clinical indices: (1) ejection fraction, � (2) stroke volume, and (3) cardiac motion � � The existing work in the literature: � 1. B-spline based active surfaces [Barbosa et al., 2013] � 2. Statistical-shape models [Butakoff et al., 2011] � 3. Edge based level-set segmentation [Rajpoot et al., 2011] � �

STMI’14 – September 2014 � 3 � Multi-Atlas Image Segmentation � Atlas-2 � § Atlas-1 � Atlas-3,4,5 � It uses the manually labeled atlases to segment the target organ. � § It does not require any training or prior estimation. � Linear and � § Successfully applied in � deformable registration � Target � i. Brain MRI Segmentation � image � � [Aljabar et al. 2009 NeuroImage] � � Propagate the labels ii. Cardiac MRI Segmentation � & majority voting � � � [Isgum et al. 2009 TMI] �

STMI’14 – September 2014 � 4 � Proposed Segmentation Framework � Atlas � Speckle Structural Image � Reduction � Representation � Target � Speckle Structural Image Segmentation Image � Reduction � Representation � Registration � Output �

STMI’14 – September 2014 � 5 � Patch Based Spectral Representation � Input image � Patch matrix � § Structural image representation � 20 40 60 § Unsupervised learning of shape 80 100 and contextual information. � 120 140 160 § Laplacian Eigenmaps. � 180 Lower dimensional � Manifold � § Useful for echo images since embedding � intensity data does not explicitly reveal the structural information. � Structural representation � 1. Wachinger C. and Navab N.: “Entropy 20 20 20 40 40 40 and Laplacian images: Structural 60 60 60 representations for multi-modal 80 80 80 100 100 100 registration” � 120 120 120 Medical Image Analysis 2012. � 140 140 140 160 160 160 180 180 180

STMI’14 – September 2014 � 6 � Dictionary Based Spectral Representation � Computationally prohibitive � Spectral embedding Target image � of image patches �

STMI’14 – September 2014 � 7 � Dictionary Based Spectral Representation � Training � Dictionary � Spectral Embedding Images � Learning � of Dictionary Atoms � What is the main motivation for the sparsity and dictionary learning ? � � § Computationally efficient due to elimination of redundancy � § Large number of images can be mapped to the same embedding space �

STMI’14 – September 2014 � 8 � Dictionary Based Spectral Representation � Training � Dictionary � Spectral Embedding Images � Learning � of Dictionary Atoms � Mapping to Spectral Query Image � Sparse and the manifold Representation � Patches � Local Coding � space �

STMI’14 – September 2014 � 9 � Dictionary Based Spectral Representation � 20 40 C , X k Y � CX k 2 60 min 80 100 8 i , k x i k 0  T 0 . s.t. 120 140 160 20 40 60 80 100 120 140 160 Input Image � Dictionary Learning �

STMI’14 – September 2014 � 10 � Dictionary Based Spectral Representation � 20 40 C , X k Y � CX k 2 60 min 80 100 8 i , k x i k 0  T 0 . s.t. 120 140 160 20 40 60 80 100 120 140 160 Input Image � Dictionary Learning � 2 2 4 4 Laplacian Graph � 6 6 8 8 10 10 12 12 2 4 6 8 10 12 L = D − W Eigen decomposition � L = V > Λ V 2 2 4 4 6 6 8 8 Dictionary Atoms � 10 10 12 12 2 4 6 8 10 12 2 4 6 8 10 12 Spectral Embedding �

STMI’14 – September 2014 � 11 � Sparse and Local Coding for Spectral Representation � Input Image � 20 40 60 80 100 120 140 160 2 2 2 2 20 40 60 80 100 120 140 160 4 4 4 4 b 1 × ¡ + ¡b 2 × ¡ + ¡b 3 × ¡ = ¡ 6 6 6 6 8 8 8 8 10 10 10 12 12 12 2 4 6 8 10 12 2 4 6 8 10 12 2 4 6 8 10 12 2 4 6 8 10 12 Query Patch � Atom 2 � Atom 3 � Atom 1 � Locality constrained linear coding � N x n k 2 + λ k b n � ˜ x n k 2 s.t. 8 n , 1 > ˜ P k y n � C˜ min x n = 1 X n =1 k y n � c m k 2 / σ � � b ( n,m ) = exp b n = [ b ( n, 1) , . . . , b ( n,M ) ] 1. K. Yu et al. : “Nonlinear learning using local coordinate coding” NIPS 2009 � 2. J. Wang et al. : “Locality-constrained linear coding for image classification” CVPR 2010 �

STMI’14 – September 2014 � 12 � Dictionary Based Spectral Representation � 20 40 60 80 100 20 120 40 140 60 160 80 20 40 60 80 100 120 140 160 First Spectral Mode � 100 120 20 140 40 160 20 40 60 80 100 120 140 160 Input Image � 60 80 100 120 140 160 20 40 60 80 100 120 140 160 Second Spectral Mode �

STMI’14 – September 2014 � 13 � Registration Strategy � Atlas Image � Target Image � K k S A k ( T ( p ) ) � S T k ( p ) k 2 + β R ( p ) X T k =1 Mode 1 � Mode 1 � p 2 R 3 , T : R 3 7! R 3 . � . � . � . � T . � . � § B-spline free-form deformations. [Rueckert et al. TMI 99] � T § Sum of squared differences similarity Mode K � Mode K � measure. � ( S A K ) ( S T K ) § Number of modes K = 4. � Single deformation field ( T ) for all 3D-3D spectral image pairs

STMI’14 – September 2014 � 14 � The Proposed Segmentation Framework � Locality Target � Multi-atlas Target � Speckle constrained image shape segmentation � image � reduction � sparse coding � representation � Training � Atlas shape Dictionary � Spectral dataset representations � learning � embedding � (atlases) �

STMI’14 – September 2014 � 15 � Validation Dataset � 1. Training Dataset (Atlases) (15 Patients) � • 3D+T echo scans. � • Cross-validation is performed on the training dataset. � • Ground-truth segmentations are available for ED and ES frames. � 2. Testing Dataset (15 Patients) � • 3D+T echo scans, obtained from different view angles. � • Only the ED and ES frames are segmented �

STMI’14 – September 2014 � 16 � Other Image Representations � Intensity and phase features � � • Encodes only the tissue boundary information. � • It is not sufficient for image analysis applications � Original echo image � Local phase image [2] � Spectral representation � � • Encodes the contextual information � � 1. Rajpoot, K. et al.: ISBI 2009 � 2. Zhuang, X. et al.: ISBI 2010 � Boundary Image [1] � Spectral Representation �

17 � Surface to Surface Distance Errors � Unprocessed Images Phase Symmetry Images Spectral Representation Speckle Reduced Images Local Phase Images 4 . 5 1 . 0 Mean Surface Distance 4 . 0 3 . 5 Mesh Surface Distance Errors (mm) 0 . 8 3 . 0 2 . 5 2 . 0 0 . 6 1 . 5 Testing Dataset Training Dataset (Cross-validation) 13 Maximum Surface Distance 0 . 4 12 11 10 0 . 2 9 8 7 0 . 0 6 0 . 0 0 . 2 0 . 4 0 . 6 0 . 8 1 . 0 Testing Dataset Training Dataset (Cross-validation)

18 � Estimation of Clinical Indices � Unprocessed Images Phase Symmetry Images Spectral Representation Speckle Reduced Images Local Phase Images 1 . 0 1 . 0 Percentage Agreement with the Ground Truth Values Ejection Fraction and Stroke Volume Correlation with Reference Values 0 . 9 0 . 8 0 . 8 0 . 7 0 . 6 0 . 6 0 . 5 Testing EF Testing SV Training EF Training SV 1 . 00 Dice Coefficient 0 . 4 0 . 95 0 . 90 0 . 2 0 . 85 0 . 80 0 . 75 0 . 0 0 . 0 0 . 2 0 . 4 0 . 6 0 . 8 1 . 0 Testing Dataset Training Dataset (Cross-validation)

STMI’14 – September 2014 � 19 � Qualitative Results � Segmentation using the proposed spectral representation Segmentation using local phase image Testing Dataset Training Dataset Ground-truth segmentation

20 � Conclusion & Acknowledgements � § Conclusion � • We propose a novel structural representation for echocardiography � • This representation enables accurate multi-atlas segmentation � § Future work � • The linear approximation of non-linear manifold can be improved � • Application of the proposed feature in echocardiography strain imaging � • Explore application to multi modal image registration (echo / CT) �

21 � Sparsity Based Spectral Embedding: Application to Multi-Atlas Echocardiography Segmentation � Ozan Oktay*, Wenzhe Shi, Jose Caballero, � Kevin Keraudren, and Daniel Rueckert � *E: o.oktay13@imperial.ac.uk � � � � � � � § Acknowledgements: �