[PPT] - Multi-Channel MR Images PhD Defense Matthias Becker Geneva, PowerPoint Presentation

SLIDE 1

PhD Defense Matthias Becker

Geneva, September 20, 2016

Efficient Extraction of Musculoskeletal Structures from Multi-Channel MR Images

PhD Director

Prof. Nadia Magnenat Thalmann

MIRALab-CUI, University of Geneva

PhD Director

Prof. Jose Rolim

TCS-CUI, University of Geneva

Jury members

Prof. Elsa Angelini
Prof. Eric Stindel

SLIDE 2

2

Outline

Introduction
Related work
Contributions in four areas
1. Image acquisition
2. Image processing
3. Model initialisation
4. Image segmentation
Conclusions

 Conclusions, limitations and future work

PhD Defense Matthias Becker

SLIDE 3

3

Research problem

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions
Musculoskeletal diseases

 morphology of anatomical structures explain some of the

rigins
MRI

 Highlights multiple structures  High contrast in soft tissue  Number of MR scans increases  Generates big amounts of data  Multi-channel acquisition

9000000 14000000 19000000 24000000 29000000 34000000 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

MRI scans (USA) based on OECD data

year # of scans

SLIDE 4

4

Research problem

Development of an imaging protocol
Tissue labelling in image data
Exploitation of multi-channel image data during the

segmentation

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

SLIDE 5

5

Outline

Introduction
Related work
Contributions in four areas
1. Image acquisition
2. Image processing
3. Model initialisation
4. Image segmentation
Conclusions

 Conclusions, limitations and future work

PhD Defense Matthias Becker

SLIDE 6

6

Muscle segmentation

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

 Method for a semi-automatic muscle segmentation  User manually outlines the contour

in pivot slices

 Interpolation and optimization

[JDR+14] E. Jolivet, E. Dion, P. Rouch, G. Dubois, R. Charrier, C. Payan, and W. Skalli, “Skeletal muscle segmentation from MRI dataset using a model-based approach,” Comput. Methods Biomech. Biomed. Eng. Imaging Vis., no. April 2014, pp. 1–8, Apr. 2014.

Example based muscle segmentation

SLIDE 7

7

Muscle segmentation

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

[BACP12] Baudin, P.Y., Azzabou, N., Carlier, P.G., Paragios, N.: Prior knowledge, random walks and human skeletal muscle segmentation. MICCAI. 15, 569–76 (2012). [G07] Gilles, B.: Anatomical and Kinematical Modelling of the Musculoskeletal System from MRI, Thesis, (2007). [TNL+14] M. S. Thomas, D. Newman, O. D. Leinhard, B. Kasmai, R. Greenwood, P. N. Malcolm,

A. Karlsson, J. Rosander, M. Borga, and A. P. Toms, “Test-retest reliability of automated whole

body and compartmental muscle volume measurements on a wide bore 3T MR system.,” Eur. Radiol., May 2014. [BACP12] [G07] [TNL+14]

Complex task,

muscles hard to distinguish

 Limit to compartments  Previous work shows problems in accuracy [BACP12] and performance [G07]

SLIDE 8

8

Muscle segmentation

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions
Statistical Shape Models

 Andrews et al.: SSM approach for muscle segmentation in the thigh  Alignment and PCA from 39 data sets  Segmentation uses energy minimization using image features (intensities) and derived features (gradients, curvature).  Avg. DSC of 0.92

[AHY+11] S. Andrews, G. Hamarneh, A. Yazdanpanah, B. Haj Ghanbari, and W. D. Reid, “Probabilistic multi-shape segmentation

f knee extensor and flexor muscles.,” Med. Image Comput. Comput. Assist. Interv., vol. 14, no. Pt 3, pp. 651–8, Jan. 2011.

SLIDE 9

9

Multi-channel segmentation

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions
Multi-channel applications

[CSV00] Edge-less level sets

n RGB

images [F11] - Left ventricle in CT and PET [F11] Fechter, T. Deformation Based Manual Segmentation in Three and Four Dimensions., 2011 [CSV00] Chan, T. F., Sandberg, B. Y., & Vese, L. a. (2000). Active Contours without Edges for Vector-Valued Images. Journal of Visual Communication and Image Representation, 11(2), 130–141. doi:10.1006/jvci.1999.0442 [KJC13] I. Kopriva, A. Jukić, and X. Chen, “Sparseness constrained nonnegative matrix factorization for unsupervised 3D segmentation of multichannel images: demonstration on multispectral magnetic resonance image of the brain,” vol. 8669,Mar. 2013. [GCM+11] E. Geremia, O. Clatz, B. H. Menze, E. Konukoglu, A. Criminisi, and N. Ayache, “Spatial decision forests for MS lesion segmentation in multi-channel magnetic resonance images.,” Neuroimage, vol. 57, no. 2, pp. 378–90, Jul. 2011. [KJC13] [GCM+11]

SLIDE 10

10

Related work

Model based approaches have high potential in

musculoskeletal segmentation

 Comprehensive approach, including anatomical knowledge [AHY+11]  Multi-channel provides additional image information  Results in higher segmentation quality [BFS+07]

Our proposal: Multi-channel + Deformable

Models

 Need for higher efficiency: Larger amount of data requires more processing

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

[AHY+11] S. Andrews, G. Hamarneh, A. Yazdanpanah, B. Haj Ghanbari, and W. D. Reid, “Probabilistic multi-shape segmentation of knee extensor and flexor muscles.,” Med. Image Comput. Comput. Assist. Interv., vol. 14, no. Pt 3, pp. 651–8, Jan. 2011. [BFS+07] P. Bourgeat, J. Fripp, P. Stanwell, S. Ramadan, and S. Ourselin, “MR image segmentation of the knee bone using phase information,” Med. Image Anal., vol. 11, no. 4, pp. 325–335, 2007.

SLIDE 11

11

Outline

Introduction
Related work
Contributions in four areas
1. Image acquisition
2. Image processing
3. Model initialisation
4. Image segmentation
Conclusions

 Conclusions, limitations and future work

PhD Defense Matthias Becker

SLIDE 12

12

Contributions

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions
1. Image acquisition
2. Image processing
3. Model initialisation
4. Segmentation

SLIDE 13

13

1. Image acquisition

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions
MRI Acquisition [GDT+14]

 Surface coil

Higher resolution, shorter duration, better SNR

 Only covers 30cm  Subject must not move scanner

coil scanner table wooden board

[GDT+14] D. García, B. M. A. Delattre, S. Trombella, S. Lynch, M. Becker, H. F. Choi, and O. Ratib, “Open framework for management and processing of multi-modality and multidimensional imaging data for analysis and modeling muscular function,” Proc. SPIE 9036, Medical Imaging 2014: Image-Guided Procedures, Robotic Interventions, and Modeling, 90361W (March 12, 2014).

SLIDE 14

14

1. Image acquisition

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

[GDT+14] D. García, B. M. A. Delattre, S. Trombella, S. Lynch, M. Becker, H. F. Choi, and O. Ratib, “Open framework for management and processing of multi-modality and multidimensional imaging data for analysis and modeling muscular function,” Proc. SPIE 9036, Medical Imaging 2014: Image-Guided Procedures, Robotic Interventions, and Modeling, 90361W (March 12, 2014).

SLIDE 15

15

1. Image acquisition

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

Multi-channel acquisition: mDixon and T1/TSE

29 to 52 minutes (3-5 stacks)

magnitude T1/TSE in- phase

pposed-

phase water fat mDixon

Survey: 0:10

Reference: 0:31 Dixon: 1:42 T1 TSE: 4:51 Stack Survey: 0:50 Reference: 0:31 T1 TSE cor: 2:00 3D DP Vista: 3:16 t Knee scan Static scan

…

Survey: 0:10

Reference: 0:31 Dixon: 1:42 T1 TSE: 4:51

Stack

Survey: 0:10

Reference: 0:31 Dixon: 1:42 T1 TSE: 4:51 Stack Move setup: 4:00 Move setup: 4:00 Move setup: 5:00

SLIDE 16

16

1. Image acquisition

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

Subject 1 Subject 2 Subject 3 Subject 4 Subject 5 Age 26 24 29 22 32 Height 175 172 197 176 173 Weight 85 65 84 68 58 Activity None Running Running Soccer Soccer

Study subjects

SLIDE 17

17

1. Image acquisition

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

4x Erosion 4x Dilation Threshold Connected component Inversion Connected component Addition

Uncropped Cropped Reduction S1 1877 MiB 701 MiB 62 % S2 1973 MiB 571 MiB 71 % S3 2133 MiB 728 MiB 66 % S4 1998 MiB 684 MiB 66 % S5 2418 MiB 658 MiB 72 %

Air labelling
Can be used to crop files

SLIDE 18

18

2. Image processing

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions
1. Image acquisition
2. Image processing
3. Model initialisation
4. Segmentation

SLIDE 19

19

2. Image processing

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

5000 10000 15000 20000 25000 30 60 90 120 150 180 210 240 270 300 330

Intensity Slice (transversal plane)

Intensity Window Distribution

Min Max

Intensity correction

Intra-stack

 Intensity range differences between stacks

Inter-stack

 Bias field of scanner  Non-parametric non-uniform intensity normalisation (N3)

100000 200000 300000 400000 500000 600000 700000 800000 62 125 187 250 312 375 437 500 562 625 687 750 812 875 937 1000 1062 1125 1187 1250 1312 1375 1437 1500 1562 1625 1687 1750 1812 1875

Frequency (# of voxels) Intensity value

with bias without bias

J. G. Sled, A. P. Zijdenbos, and A. C. Evans, “A nonparametric method

for automatic correction of intensity nonuniformity in MRI data.” IEEE Transactions on Medical Imaging, vol. 17, no. 1, pp. 87–97, 1998.

SLIDE 20

20

2. Image processing

Non-local means filtering for noise reduction SNR improvement from 5.1 to 5.4

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

200000 400000 600000 800000

Frequency (# of voxels) Intensity value

before denoising after denoising

A. Tristán-Vega, V. García-Pérez, S. Aja-

Fernández, and C. F. Westin, “Efficient and robust nonlocal means denoising of MR data based on salient features matching,” Computer Methods and Programs in Biomedicine, vol. 105, no. 2, pp. 131–144, 2012

SLIDE 21

21

2. Image processing

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

Fat image Air mask 4x Erosion 4x Dilation Thresholding Inversion Connected components Dilation Addition

Muscles

Connected components

[BMT15] M. Becker, and N. Magnenat-Thalmann, “Muscle Tissue Labeling of Human Lower Limb in Multi-Channel mDixon MR Imaging: Concepts and Applications,” in Computational Biology and Bioinformatics, IEEE/ACM Transactions on, 2015, online.

SLIDE 22

22

2. Image processing

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

4x Erosion 4x Dilation Thresholding Inversion Connected components Dilation Addition

Muscles

Connected components

SLIDE 23

23

2. Image processing

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

4x Erosion 4x Dilation Thresholding Inversion Connected components Dilation Addition

Muscles

Connected components

SLIDE 24

24

2. Image processing

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

4x Erosion 4x Dilation Thresholding Inversion Connected components Dilation Addition

Muscles

Connected components

SLIDE 25

25

2. Image processing

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

4x Erosion 4x Dilation Thresholding Inversion Connected components Dilation Addition

Muscles

Connected components

SLIDE 26

26

2. Image processing

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

4x Erosion 4x Dilation Thresholding Inversion Connected components Dilation Addition

Muscles

Connected components

SLIDE 27

27

2. Image processing

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

4x Erosion 4x Dilation Thresholding Inversion Connected components Dilation Addition

Muscles

Connected components

SLIDE 28

28

2. Image processing

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

4x Erosion 4x Dilation Thresholding Inversion Connected components Dilation Addition

Muscles

Connected components

SLIDE 29

29

2. Image processing

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

4x Erosion 4x Dilation Thresholding Inversion Connected components Dilation Addition

Muscles

Connected components

SLIDE 30

30

2. Image processing

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

11 13 15 17 19 21 23 25

11 13 15 17 19 21 23 25 automatic (ml) manual (ml) Method all (SDC) Main (SDC) Joints (SDC) w/o fat reduction 0.95 0.97 0.91 with fat reduction 0.96 0.97 0.93 difference 0.01 0.003 0.027

Muscle labelling results
Comparison to 7 manually

segmented slices

Sørensen Dice Coefficient 𝑇𝐸𝐷 𝑇, 𝑈 = 2

𝑇∩𝑈 𝑇 + 𝑈

Range: [0,1]

SLIDE 31

31

2. Image processing

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions
Thigh result comparison with SoA

Method Measure Result [KPV 07] fuzzy C-means Accuracy 0.91 [PHKTA06] surface expansion-based Accuracy 0.94 [AHY+11] model-based with prior knowledge SDC 0.95 [BMT15] w/o fat reduction SDC 0.95 [BMT15] with fat reduction SDC 0.97

[KPV07] H. Kang, A. Pinti, L. Vermeiren, A. Taleb-Ahmed, and X. Zeng, “An automatic FCM-based method for tissue classification: Application to MRI of thigh,” in 2007 1st International Conference on Bioinformatics and Biomedical Engineering, ICBBE, 2007, pp. 510–514. [PHKTA06] A. Pinti, P. Hêdoux, H. Kang, and A. Taleb-Ahmed, “An automated pixel classification method using surface expansion Application to MRI image sequence,” IMACS Multiconference on "Computational Engineering in Systems Applications", CESA, pp. 1514–1519, 2006. [AHY+11] ]S. Andrews, G. Hamarneh, A. Yazdanpanah, B. HajGhanbari, and W. D. Reid, “Probabilistic multi-shape segmentation of knee extensor and flexor muscles.” Medical image computing and computer- assisted intervention : MICCAI International Conference on Medical Image Computing and Computer-Assisted Intervention, vol. 14, no. Pt 3, pp. 651–8, jan 2011. [BMT15] M. Becker, and N. Magnenat-Thalmann, “Muscle Tissue Labeling of Human Lower Limb in Multi-Channel mDixon MR Imaging: Concepts and Applications,” in Computational Biology and Bioinformatics, IEEE/ACM Transactions on, 2015.

SLIDE 32

32

2. Image processing
Image processing: GPU

 Processing time critical for applications  GPUs available from image reconstruction  Parallel morphological operations, CCL, addition, thresholding: in-place, to new image  CUDA or OpenCL

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

Block (0,0) Block (0,1) Block (1,0) Block (1,1) Block (2,0) Block (2,1) Thread (0,0) Thread (0,1) Thread (0,2) Thread (1,0) Thread (1,1) Thread (1,2) Thread (2,0) Thread (2,1) Thread (2,2) Grid (3x2) Block (3x3) SM 0 SM 1

SLIDE 33

33

2. Image processing
CCL

 Divide-and-conquer: CCL and merge [HLP10]  Scaling based on number of blocks  Large number

f possible

labels  Contributions to handling memory constraints

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

[HLP10] K. Hawick, A. Leist, and D. Playne, “Parallel graph component labelling with GPUs and CUDA,” Parallel Computing, pp. 655–678, 2010.

SLIDE 34

34

10 20 30 40 50 60 70 S1 S2 S3 S4 S5 S1 (crop) S2 (crop) S3 (crop) S4 (crop) S5 (crop) S1 (half)

Processing time (seconds)

10 20 30 40 50 60 70 80 90 100 110 S1 S2 S3 S4 S5 S1 (crop) S2 (crop) S3 (crop) S4 (crop) S5 (crop) S1 (half)

Processing time(seconds)

i7 3770 GTX 470 GTX 660 GTX 750 Ti M2050 GRID K520

2. Image processing

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions
Results: Comparison CPU and five nVidia

GPUs

Muscle Air

M. Becker, and N. Magnenat-Thalmann, “Parallel graph component labelling on GPUs in low memory scenarios” (in preparation)

S1-5: Subject 1-5, cropped: removed air, half: half resolution

SLIDE 35

35

2. Image processing

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions
Comparison with SoA

Method Computation time Unsupervised recursive algorithm [UMV09] ≈ 300s / slice Saliency-based [IGTG+14] ≈ 1s / slice Fuzzy approach [IGTH+15] ≈ 10s / slice CPU version [BMT15] ≈ 0.1s / slice GPU version [BMT15] ≈ 0.033 - 0.05s / slice

[UMV09] L. Urricelqui, A. Malanda, and A. Villanueva, “Automatic segmentation of thigh magnetic resonance images,” World Acad Sci Eng Technol, vol. 3, no. 10, pp. 953–959, 2009. [IGTG+14] ]N. Imamoglu, J. Gomez-Tames, J. Gonzalez, D. Gu, and W. Yu, “Pulse-Coupled Neural Network Segmentation and Bottom-Up Saliency-On Feature Extraction for Thigh Magnetic Resonance Imaging Based 3D Model Construction,” Journal of Medical Imaging and Health Informatics, vol. 4,

no. 2, pp. 220– 229, 2014.

[IGTH+15] N. Imamoglu, J. Gomez-Tames, S. He, D. Y. Gu, K. Kita, and W. Yu, “Unsupervised muscle region extraction by fuzzy decision based saliency feature integration on thigh MRI for 3D modeling,” Proceedings of the 14th IAPR International Conference on Machine Vision Applications, MVA 2015, pp. 150–153, 2015. [BMT15] M. Becker, and N. Magnenat-Thalmann, “Muscle Tissue Labeling of Human Lower Limb in Multi-Channel mDixon MR Imaging: Concepts and Applications,” in Computational Biology and Bioinformatics, IEEE/ACM Transactions on, 2015.

SLIDE 36

36

3. Model initialisation

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions
1. Image acquisition
2. Image processing
3. Model initialisation
4. Segmentation

SLIDE 37

37

3. Model initialisation

Joint localisation (besides GHT [Kho07])

Histogram hints the location of the joints
Template histogram matching using scaling and

translation

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

20000 40000 60000 80000 100000 120000 140000 35 70 105 140 175 210 245 280 315 350 385 420 455 490 525 560

Volume (voxels) Transversal slice index, superior direction

Tissue volume

10000 20000 30000 40000 50000 60000 70000 80000 90000 34 68 102 136 170 204 238 272 306 340 374 408 442 476 510 544 578 612 646 680 714

Volume (voxels) Transversal slice index, superior direction

Muscle volume

Subject1 Subject2 Subject3 Subject4 Subject5

Ankle Knee Hip Error (offset) 16.2 mm 2 mm 2.1 mm

[Kho07] K. Khoshelham, “Extending Generalized Hough Transform to Detect 3D Objects in Laser Range Data,” ISPRS Workshop on Laser Scanning and SilviLaser 2007, pp. 206–210, 2007.

SLIDE 38

38

3. Model initialisation
Bone placement

 Centroid sampled along shaft  On three slices  Translation to

verlap fixed

 Best neighbour registration

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

fixed rotating rotating Min distance: 0.01 mm Max distance: 16.79 mm Mean distance: 4.27 mm RMS: 5.48 mm

0 mm 16 mm

SLIDE 39

39

3. Model initialisation

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions
Fast muscle mesh placement

SLIDE 40

40

3. Model initialisation

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

SLIDE 41

41

3. Model initialisation
Muscle correction
Template offset
Use labels to adapt meshes to avoid

image-to-image registration

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

X S T B A ℎ 𝑐 𝑏 𝑕 ℎ1 ℎ2 𝑢 𝑡 D C X’ S’ T’ B A ℎ′ 𝑐′ 𝑏′ 𝑕 ℎ1 ℎ2 𝑢 𝑡 C’ D’

[BMT15] M. Becker, and N. Magnenat-Thalmann, “Muscle Tissue Labeling of Human Lower Limb in Multi-Channel mDixon MR Imaging: Concepts and Applications,” in Computational Biology and Bioinformatics, IEEE/ACM Transactions on, 2015, online.

Muscle tissue Muscle tissue Muscle mesh Muscle mesh

SLIDE 42

42

3. Model initialisation

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

0% 20% 40% 60% 80% 100% Before After Before After Before After Before After Before After Subject 1 Subject 2 Subject 3 Subject 4 Subject 5 Sartorius Biceps femoris Rectus femoris

0.5 0.6 0.7 0.8 0.9 1

Original 18 36 72 144

[BMT15] M. Becker, and N. Magnenat-Thalmann, “Muscle Tissue Labeling of Human Lower Limb in Multi-Channel mDixon MR Imaging: Concepts and Applications,” in Computational Biology and Bioinformatics, IEEE/ACM Transactions on, 2015, online.

Overlap Overlap

SLIDE 43

43

4. Image Segmentation

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions
1. Image acquisition
2. Image processing
3. Model initialisation
4. Segmentation

SLIDE 44

44

4. Image Segmentation

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

[BLS03] J. Bredno, T. M. T. M. Lehmann, and K. Spitzer, “A General Discrete Contour Model in Two, Three, and Four Dimensions for Topology-Adaptive Multichannel Segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 5, pp. 550–563, 2003. [JKM11] D. Jayadevappa, S. Kumar, and D. Murty, “Medical Image Segmentation Algorithms using Deformable Models: A Review,” Iete Tech. Rev., vol. 28, no. 3, pp. 248–255, 2011.

Intensity profiles Shape preservation Intersection handling Smoothing Force calculation Deformation simulation

SLIDE 45

45

4. Image Segmentation
Forces

 Muscle labels

used as indication
distance vector field

 Air mask and bones

rigid boundary

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

[MR03] C. Maurer and V. Raghavan, “A linear time algorithm for computing exact Euclidean distance transforms of binary images in arbitrary dimensions,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 265–270, 2003.

SLIDE 46

46

4. Image Segmentation

RF VM VI VL IP 0.78 0.78 0.76 0.68 W(ater) 0.85 0.81 0.78 0.74 F(at) 0.72 0.76 0.72 0.71 OP 0.79 0.79 0.77 0.71 F+W 0.74 0.80 0.77 0.74 F+W+OP 0.83 0.80 0.76 0.74 F+IP+OP 0.81 0.78 0.77 0.71 F+W+IP+OP 0.82 0.79 0.78 0.72

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

RF VM VI VL IP 0.76 0.79 0.75 0.70 W(ater) 0.81 0.83 0.79 0.71 F(at) 0.71 0.78 0.72 0.71 OP 0.76 0.81 0.75 0.72 F+W 0.71 0.77 0.76 0.72 F+W+OP 0.84 0.82 0.76 0.72 F+IP+OP 0.81 0.81 0.75 0.72 F+W+IP+OP 0.84 0.81 0.75 0.75

Subject 1 Subject 2

SDC (Sørensen Dice Coefficient)

Quadriceps (RF,VM,VI,VL)
Gradient based image forces
M. Becker, and N. Magnenat-Thalmann, “Exploiting MRI Multi-Channel Information in Quadriceps Muscle Segmentation” (in preparation)

SLIDE 47

47

4. Image Segmentation

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions
Efficient segmentation: coupled parallel calculation of

different resolutions

[BPB13] Bogovic, Prince, Bazin, “A Multiple Object Geometric Deformable Model for Image Segmentation.” Comp vis and image understanding : 2013. [EWS+10] Edwards, Williams,Sjaardema, D. G. Baur, and W. K. Cochran, “SIERRA Toolkit Computational Mesh Conceptual Model,”, 2010. [TRGP10] Z. Tang, G. Rong, X. Guo, and B. Prabhakaran, “Streaming 3D shape deformations in collaborative virtual environment,” 2010.[SEYB08] S. Sumengen, M. T. Eren, S. Yesilyurt, and S. Balcisoy, “A multi-resolution mesh representation for deformable objects in collaborative virtual environments,” in Comm. in Comp. and Inf. Science, vol. 21 CCIS, 2008, pp. 75–87. [BNMT15] M. Becker, N. Nijdam, and N. Magnenat-Thalmann, “Coupling strategies for multi-resolution deformable meshes: expanding the pyramid approach beyond its one-way nature,” Int. J. Comput. Assist. Radiol. Surg., vol. 11, no. 5, pp. 695-705, 2016.

Related work

Coupled components

[BPB13]

Mesh consistency

[EWS+10]

Mesh distributions in

collaborative environments [TRGP10]

Multi-resolution FEMs

for distributed environments [SEYB08]

SLIDE 48

48

4. Image Segmentation

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions
Architecture update for parallel models

[BNMT15] M. Becker, N. Nijdam, and N. Magnenat-Thalmann, “Coupling strategies for multi-resolution deformable meshes: expanding the pyramid approach beyond its one-way nature,” Int. J. Comput. Assist. Radiol. Surg., vol. 11, no. 5, pp. 695-705, 2016.

SLIDE 49

49

Approaches and results
4. Image Segmentation

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

Per Step Updates (PSU) Scaled Updates (SU) Partial Updates (PU) Sync Sync

update ulock calc clock update ulock calc clock update ulock calc clock

[BNMT15] M. Becker, N. Nijdam, and N. Magnenat-Thalmann, “Coupling strategies for multi-resolution deformable meshes: expanding the pyramid approach beyond its one-way nature,” Int. J. Comput. Assist. Radiol. Surg., vol. 11, no. 5, pp. 695-705, 2016. Time (μs) Time (μs) Time (μs) Thread index Thread index Thread index

SLIDE 50

50

Outline

Introduction
Related work
Contributions in four areas
1. Image acquisition
2. Image processing
3. Model initialisation
4. Image segmentation
Conclusions

 Conclusions, limitations and future work

PhD Defense Matthias Becker

SLIDE 51

51

Conclusions

1. Image acquisition

 Coil support for seamless acquisition [GDT+14]

2. Image processing

 Fast and correct muscle tissue labelling [BMT15]

3. Model initialisation

 Label based mesh correction [BMT15]

4. Image segmentation

 Multi-channel image segmentation [in preperation]  Efficient coupling of multi-resolution models [BNMT15]

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

SLIDE 52

52

Limitations & Future Work

1. Image acquisition

 Long acquisition, T1/TSE optional?  Optimise coil support

2. Image data processing

 Stitching error depends on slice thickness  Muscle labelling near joints, full-body

3. Model initialisation

 Ankle accuracy  Other limbs

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

SLIDE 53

53

Limitations & Future work

4. Multi-channel image segmentation

 Improvements through SSMs  Intensity profiles  More data sets

4. Multi-resolution mesh coupling

 Network-coupling  User study  Collaboration

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions

SLIDE 54

54

Reserve

PhD Defense Matthias Becker

SLIDE 55

55

Image Segmentation

PhD Defense Matthias Becker

Introduction
Related Work
Contributions
Conclusions
Implementation

 Scripting: Lua  Visualization: vtk  Simulation: Bullet Physics

Accurate simulation
Easy integration
GPU pipeline

 Image handling: ITK  Models, forces, management

Custom code, C++, 162 files, 15k+ LoC

 Threading, System: Boost  Multi-platform: Windows, Linux, macOS

SLIDE 56

56

Publications

M. Becker, and N. Magnenat-Thalmann, “Muscle Tissue Labeling of Human Lower Limb in Multi-Channel mDixon MR Imaging:

Concepts and Applications,” in Computational Biology and Bioinformatics, IEEE/ACM Transactions on, 2015, online. (IF = 1.609)

M. Becker, N. Nijdam, and N. Magnenat-Thalmann, “Coupling strategies for multi-resolution deformable meshes: expanding the

pyramid approach beyond its one-way nature,” in Int. J. Comput. Assist. Radiol. Surg., vol. 11, no. 5, pp. 695-705, May 2016. (IF = 1.827)

A. Chincisan, K. Tecante, M. Becker, N. Magnenat-Thalmann, C. Hurschler, and H.F. Choi, “A Computational Approach to

Calculate Personalized Pennation Angle based on MRI: Effect on Gait Analysis,” in Int. J. Comput. Assist. Radiol. Surg., vol. 11,

no. 5, pp. 683-693, May 2016. (IF = 1.827)
M. E. H. Wagner, N.-C. Gellrich, K.-I. Friese, M. Becker, F.-E. Wolter, J. T. Lichtenstein, M. Stoetzer, M. Rana, and H. Essig,

“Model-based segmentation in orbital volume measurement with cone beam computed tomography and evaluation against current concepts,” in Int. J. Comput. Assist. Radiol. Surg., vol. 11, no. 1, pp 1-9, January 2016. (IF = 1.827)

H. F. Choi, A. Chincisan, M. Becker, and N. Magnenat-Thalmann, “Multimodal composition of the digital patient: a strategy for

the knee articulation,” in Vis. Comput., vol. 30, no. 6, pp. 1–11, May 2014. (IF = 0.957)

M. Becker, and N. Magnenat-Thalmann, "Muscle tissue labeling of human lower extremities in multi-channel mDixon MR

imaging: Concepts and applications," in Bioinformatics and Biomedicine (BIBM), 2014 IEEE International Conference on, pp.279,284, 2-5 Nov. 2014, Best Overall Paper Award.

D. García, B. M. A. Delattre, S. Trombella, S. Lynch, M. Becker, H. F. Choi, and O. Ratib, “Open framework for management

and processing of multi-modality and multidimensional imaging data for analysis and modeling muscular function,” Proc. SPIE 9036, Medical Imaging 2014: Image-Guided Procedures, Robotic Interventions, and Modeling, 90361W (March 12, 2014).

L. Assassi, M. Becker, and N. Magnenat-Thalmann, “Dynamic Skin Deformation based on Biomechanical Modeling,” in 25th

Annual Conference on Computer Animation and Social Agents (CASA 2012) May 9-11, 2012 Nanyang Technological University, Singapore.

M. Becker, K.-I. Friese, F.-E. Wolter, N.-C. Gellrich, and H. Essig, “Development of a Reliable Method for Orbit Segmentation &

Measuring,” in 2015 IEEE International Symposium on Medical Measurements and Applications (MeMeA 2015), 2015, pp. 285– 290.

M. Becker, and N. Magnenat-Thalmann, “Deformable Models in Medical Image Segmentation,” in 3D Multiscale Physiological

Human, 1st ed. N. Magnenat-Thalmann, O. Ratib, and H. F. Choi, Eds. Springer-Verlag London, 2014 PhD Defense Matthias Becker

SLIDE 57

57 PhD Defense Matthias Becker

SLIDE 58

58

Acknowledgments

Supervisors and jury members

 Prof. Nadia Magnenat-Thalmann, Prof. José Rolim, Prof. Elsa Angelini,

Prof. Eric Stindel
Collaborations

 MIRALab: Dr. H.F. Choi, Dr. A. Chincisan, Dr. N. Nijdam, Dr. L. Assassi  MultiScaleHuman partners: HUG, LBB, volunteers

MIRALab members
Family and friends

PhD Defense Matthias Becker

This work has been funded by the EU FP7 Marie Curie Initial Training Network project Multi-scale Biological Modalities for Physiological Human Articulation (MultiScaleHuman) under Grant number 289897.