Structured Decision Forests For Multi-modal Ultrasound Image - - PowerPoint PPT Presentation

Structured Decision Forests For Multi-modal Ultrasound Image Registration

Ozan Oktay1, Andreas Schuh1, Martin Rajchl1, Kevin Keraudren1, Alberto Gomez3, Mattias P. Heinrich2, Graeme Penney3, Daniel Rueckert1

1 Department of Computing, Imperial College London, UK 2 Institute of Medical Informatics, University of Lübeck, Germany 3 Division of Imaging Sciences and Biomedical Engineering, King’s College London, UK

18th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2015)

2

Intra-Operative Image Guidance with TOE Ultrasound Images

Image Guided Cardiac Interventions

Pre-Operative Stage CT and MR Image Acquisitions

18th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2015)

Spatial Alignment of Pre- and Intra-Operative Images (for Better Image Guidance) Spatial Alignment of Pre- and Intra- Operative Images

3

Probabilistic Edge Maps (PEMs)

SSC-MRI GM-MRI SSC-US

SSC [1] and GM [2]

Input Images Input Images PEMs PEMs

18th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2015)

1. Heinrich et al.: “Towards real-time multimodal fusion for image guided interventions using self-similarities.” MICCAI’13 2. Wein et al.: “Global registration of US to MRI using the LC2 metric for enabling neurosurgical guidance.” MICCAI’13 GM: Intensity Gradient Magnitude – SSC: Self-Similarity Context Descriptor

4

Advantages of Probabilistic Edge Maps

• A. Modality independent (e.g. CT, MRI, US)
• A. Computationally efficient ( 20s per image )
• A. Target organ specific image registration
• B. Accurate and smooth anatomical representation
• A. Same training and testing configuration is

applied to all three modalities.

• B. It does not require image segmentation.

18th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2015)

5

Structured Decision Forest (SDF)

ψ1, θ1 ψ2, θ2 L3

Input Image

• Each voxel is voted for Nt x (Me)3
• Nt is the number of trees.
• All the votes are aggregated by averaging.

18th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2015)

L1

Ma Input Space: xi ∈ X

Image features Structured Decision Tree

3. Dollar et al.: “Structured forests for fast edge detection.” ICCV 2013 4. Kontschieder et al.: “Structured class-labels in random forests for semantic image labeling.” ICCV 2011

Me

Output Space: yi ∈ Y

Output edge patch labels

L2

PEM Representation

6

PEM-CT PEM-US

The Proposed Multi-Modal Registration Framework

Input cardiac images PEM representation Initial Alignment

• f the images

Global alignment with robust block matching [2] B-spline FFD based non-rigid registration [1]

5. Rueckert et al.: “Non-rigid registration using free-form deformations: Application to breast MR images.” TMI’99 6. Ourselin et al.: “Reconstructing a 3D structure from serial histological sections.” Image and Vision Computing ’01 Computation Time (Quad-core 3.0GHz) ~20s per image ~21s per image ~73s per image

18th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2015)

7

US/CT and US/MR Image Alignment Results

18th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2015)

Subject 1 Subject 1 Subject 2 Subject 2 Subject 3 Subject 3 Subject 4 Subject 4

8

Structured Regression Forest (SRF)

ψ1, θ1 ψ2, θ2 L1 L2 L3

L4

7. Gall J., et al. ”Class-Specific Hough Forests for Object Detection.” CVPR 2009. 8. Criminisi A., et al. “Regression Forests for Efficient Anatomy Detection and Localization in CT Studies.” MCV 2010.

(y1, dn

1, Λn 1)

(y2, dn

2, Λn 2) (y3, dn 3, Λn 3)

(y4, dn

4, Λn 4)

Γ3, θ3

18th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2015)

CT image & PEM contours Regression for Septal-wall MR image & PEM contours Regression for Mid-ventricle : Leaf Node : Classification Node : Regression Node

The detected landmark points are used to initialize the multi- modal image registration.

9

Structured Decision Forests For Multi-modal Ultrasound Image Registration

Original Fast x4

§ Acknowledgements:

• Source code will be available at http://www.doc.ic.ac.uk/~oo2113/

18th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2015)