SEGMENTATION OF MEDICAL VOLUMES USING CONVOLUTIONAL NEURAL NETWORKS - - PowerPoint PPT Presentation

▶
segmentation of medical volumes using convolutional
SMART_READER_LITE
LIVE PREVIEW

SEGMENTATION OF MEDICAL VOLUMES USING CONVOLUTIONAL NEURAL NETWORKS - - PowerPoint PPT Presentation

Aug 15, 2023 179 likes •419 views

FAUSTO MILLETARI SEGMENTATION OF MEDICAL VOLUMES USING CONVOLUTIONAL NEURAL NETWORKS MEDICAL IMAGE SEGMENTATION Computer assisted diagnosis Computer assisted intervention Treatment Planning Enhanced visualisation MEDICAL IMAGING

slide-1
SLIDE 1

SEGMENTATION OF MEDICAL VOLUMES USING CONVOLUTIONAL NEURAL NETWORKS

FAUSTO MILLETARI

slide-2
SLIDE 2

MEDICAL IMAGE SEGMENTATION

▸ Treatment Planning ▸ Computer assisted diagnosis ▸ Computer assisted intervention ▸ Enhanced visualisation

slide-3
SLIDE 3

MEDICAL IMAGING

▸ Magnetic Resonance Imaging (MRI)

  • De-facto standard in neuroimaging
  • Complex and expensive
  • Good soft tissue contrast, limited artefacts, high SNR
slide-4
SLIDE 4

▸ Ultrasound

  • Non-invasive, mobile, inexpensive, real-time, safe
  • Emits high frequency sound, forms images from tissue echoes
  • Noisy signal, artefacts, shadows and poor contrast
  • Early Parkinson’s diagnosis

MEDICAL IMAGING

slide-5
SLIDE 5

A LEARNING BASED APPROACH TO SEGMENTATION

▸ Automatic segmentation

▸ Anatomy localisation ▸ Prior shape knowledge

▸ Time efficiency ▸ Limited training data

slide-6
SLIDE 6

SEGMENTATION WITH CONVOLUTIONAL NEURAL NETWORKS

▸ CNNs for image segmentation

  • How to use CNNs for segmentation?
  • How to include shape priors?
  • What is a good network architecture choice?
  • Can we process 3D data?
  • How to be robust to limited amount of

training data?

slide-7
SLIDE 7

SEGMENTATION WITH CONVOLUTIONAL NEURAL NETWORKS

▸ Voxel-wise classification

…

CNN

▸ Segmentation mask prediction for whole volume

CNN

▸ Voting strategy for localisation and segmentation

…

CNN

x Localisation Accurate segmentation

slide-8
SLIDE 8

TRAINING HOUGH-CNN

Training set of (overlapping) patches and votes CNN

Softmax FG BG

▸ Train CNN classification ▸ Extract descriptors foreground patches

CNN

Features (128-d) f1 f2 f3 f4 f5 f6

▸ Build database features/votes/segmentation patches

f4 f5 f6 f3 f1 f2

slide-9
SLIDE 9

TESTING HOUGH-CNN

Collect overlapping patches CNN

Softmax (class) Features (128-d)

▸ CNN classification & Feature Extraction

slide-10
SLIDE 10

TESTING HOUGH-CNN

Binary classification result CNN

Softmax (class) Features (128-d)

▸ CNN classification & Feature Extraction

slide-11
SLIDE 11

TESTING HOUGH-CNN

Binary classification result CNN

Softmax (class) Features (128-d)

▸ CNN classification & Feature Extraction ▸ Use features to retrieve votes from database

Find K-NN 
 within range r Features (128-d) 


  • nly foreground patches (circles)

Example neighbours

slide-12
SLIDE 12

TESTING HOUGH-CNN

Votes towards object centroid CNN

Softmax (class) Features (128-d)

▸ CNN classification & Feature Extraction ▸ Cast votes and find peak in vote map ▸ Use features to retrieve votes from database

Find K-NN 
 within range r Features (128-d) 


  • nly foreground patches (circles)

Example neighbours

slide-13
SLIDE 13

TESTING HOUGH-CNN

Collect overlapping patches CNN

Softmax (class) Features (128-d)

▸ CNN classification & Feature Extraction ▸ Cast votes and find peak in vote map

▸ ▸

▸ Trace back correct votes. 


Project associated segmentation patches (from database)

▸ Use features to retrieve votes from database

Find K-NN 
 within range r Features (128-d) 


  • nly foreground patches (circles)

Example neighbours

slide-14
SLIDE 14

EXPERIMENTAL EVALUATION

▸ Ultrasound and MRI volumes ▸ Training dataset size ▸ Data dimensionality

Experiments

  • Variation of

– network architecture

  • different depth (3, 5 and 8 convolutional layers)
  • different convolution filter sizes (side length of 3,5,7,9,11 voxels)

– Patch size

  • quadratic/cubic patches with side length of 31 or 51 voxels

– Patch dimensionality 2D 2.5D 3D – training set size

  • MRI: 100,1.000 or 10.000 patches/region/volume ~ 116.000, 765.000 or 3.100.000 patches
  • US: 1.000 or 5.000 patches/region/volume

~ 80.000 or 400.000 patches

Segmentation of MRI and Ultrasound Scans Using Deep Convolutional Neural Networks 10

Experiments

  • Variation of

– network architecture

  • different depth (3, 5 and 8 convolutional layers)
  • different convolution filter sizes (side length of 3,5,7,9,11 voxels)

– Patch size

  • quadratic/cubic patches with side length of 31 or 51 voxels

– Patch dimensionality 2D 2.5D 3D – training set size

  • MRI: 100,1.000 or 10.000 patches/region/volume ~ 116.000, 765.000 or 3.100.000 patches
  • US: 1.000 or 5.000 patches/region/volume

~ 80.000 or 400.000 patches

Segmentation of MRI and Ultrasound Scans Using Deep Convolutional Neural Networks 10

Experiments

  • Variation of

– network architecture

  • different depth (3, 5 and 8 convolutional layers)
  • different convolution filter sizes (side length of 3,5,7,9,11 voxels)

– Patch size

  • quadratic/cubic patches with side length of 31 or 51 voxels

– Patch dimensionality 2D 2.5D 3D – training set size

  • MRI: 100,1.000 or 10.000 patches/region/volume ~ 116.000, 765.000 or 3.100.000 patches
  • US: 1.000 or 5.000 patches/region/volume

~ 80.000 or 400.000 patches

Segmentation of MRI and Ultrasound Scans Using Deep Convolutional Neural Networks 10

▸ Network Architecture

slide-15
SLIDE 15

EXPERIMENTAL EVALUATION

▸ Hough-CNN vs. voxel-wise classification

VOXEL-WISE CLASSIFICATION

0,25 0,5 0,75 1

0,55 0,33

VOXEL-WISE CLASSIFICATION

0,25 0,5 0,75 1

0,47 0,29

MRI US

slide-16
SLIDE 16

EXPERIMENTAL EVALUATION

▸ Hough-CNN vs. voxel-wise classification

Biggest Training set Smallest Training Set

slide-17
SLIDE 17

EXPERIMENTAL EVALUATION

▸ Hough-CNN vs. voxel-wise classification

slide-18
SLIDE 18

EXPERIMENTAL EVALUATION

▸ Patch dimensionality (2D, 2.5D, 3D)

2D

0,25 0,5 0,75 1 MRI US

0,82 0,77 0,83 0,68 0,85 0,68

Experiments

  • Variation of

– network architecture

  • different depth (3, 5 and 8 convolutional layers)
  • different convolution filter sizes (side length of 3,5,7,9,11 voxels)

– Patch size

  • quadratic/cubic patches with side length of 31 or 51 voxels

– Patch dimensionality 2D 2.5D 3D – training set size

  • MRI: 100,1.000 or 10.000 patches/region/volume ~ 116.000, 765.000 or 3.100.000 patches
  • US: 1.000 or 5.000 patches/region/volume

~ 80.000 or 400.000 patches

Segmentation of MRI and Ultrasound Scans Using Deep Convolutional Neural Networks 10

Experiments

  • Variation of

– network architecture

  • different depth (3, 5 and 8 convolutional layers)
  • different convolution filter sizes (side length of 3,5,7,9,11 voxels)

– Patch size

  • quadratic/cubic patches with side length of 31 or 51 voxels

– Patch dimensionality 2D 2.5D 3D – training set size

  • MRI: 100,1.000 or 10.000 patches/region/volume ~ 116.000, 765.000 or 3.100.000 patches
  • US: 1.000 or 5.000 patches/region/volume

~ 80.000 or 400.000 patches

Segmentation of MRI and Ultrasound Scans Using Deep Convolutional Neural Networks 10

Experiments

  • Variation of

– network architecture

  • different depth (3, 5 and 8 convolutional layers)
  • different convolution filter sizes (side length of 3,5,7,9,11 voxels)

– Patch size

  • quadratic/cubic patches with side length of 31 or 51 voxels

– Patch dimensionality 2D 2.5D 3D – training set size

  • MRI: 100,1.000 or 10.000 patches/region/volume ~ 116.000, 765.000 or 3.100.000 patches
  • US: 1.000 or 5.000 patches/region/volume

~ 80.000 or 400.000 patches

Segmentation of MRI and Ultrasound Scans Using Deep Convolutional Neural Networks 10

2D 2.5D 3D

Experiments

  • Variation of

– network architecture

  • different depth (3, 5 and 8 convolutional layers)
  • different convolution filter sizes (side length of 3,5,7,9,11 voxels)

– Patch size

  • quadratic/cubic patches with side length of 31 or 51 voxels

– Patch dimensionality 2D 2.5D 3D – training set size

  • MRI: 100,1.000 or 10.000 patches/region/volume ~ 116.000, 765.000 or 3.100.000 patches
  • US: 1.000 or 5.000 patches/region/volume

~ 80.000 or 400.000 patches

Segmentation of MRI and Ultrasound Scans Using Deep Convolutional Neural Networks 10

Experiments

  • Variation of

– network architecture

  • different depth (3, 5 and 8 convolutional layers)
  • different convolution filter sizes (side length of 3,5,7,9,11 voxels)

– Patch size

  • quadratic/cubic patches with side length of 31 or 51 voxels

– Patch dimensionality 2D 2.5D 3D – training set size

  • MRI: 100,1.000 or 10.000 patches/region/volume ~ 116.000, 765.000 or 3.100.000 patches
  • US: 1.000 or 5.000 patches/region/volume

~ 80.000 or 400.000 patches

Segmentation of MRI and Ultrasound Scans Using Deep Convolutional Neural Networks 10

Experiments

  • Variation of

– network architecture

  • different depth (3, 5 and 8 convolutional layers)
  • different convolution filter sizes (side length of 3,5,7,9,11 voxels)

– Patch size

  • quadratic/cubic patches with side length of 31 or 51 voxels

– Patch dimensionality 2D 2.5D 3D – training set size

  • MRI: 100,1.000 or 10.000 patches/region/volume ~ 116.000, 765.000 or 3.100.000 patches
  • US: 1.000 or 5.000 patches/region/volume

~ 80.000 or 400.000 patches

Segmentation of MRI and Ultrasound Scans Using Deep Convolutional Neural Networks 10

2D 2.5D 3D

slide-19
SLIDE 19

EXPERIMENTAL EVALUATION

▸ Network architectures

2D

0,25 0,5 0,75 1

3-3-3-3-3 3-3-3-3-3-3-3-3 5-5-5-5-5 7-5-3 9-7-5-3-3 SMALL ALEX

0,72 0,68 0,77 0,71 0,7 0,72

MRI

0,25 0,5 0,75 1

3-3-3-3-3 3-3-3-3-3-3-3-3 5-5-5-5-5 7-5-3 9-7-5-3-3 SMALL ALEX

0,83 0,82 0,83 0,83 0,85 0,84

US

0,77 0,85

Architecture Architecture

slide-20
SLIDE 20

CONCLUSION

▸ Need precision and robustness in medical domain
 ▸ Volumetric CNNs via CuDNN and powerful GPUs
 ▸ Localisation strategy minimises impact of outliers
 ▸ Segmentation makes use of previous shape knowledge
 ▸ Results show excellent results in challenging situations

slide-21
SLIDE 21

ADDITIONAL RESOURCES

▸ Our Hough-CNN paper

http://arxiv.org/abs/1601.07014

  • Dr. Seyed-Ahmad Ahmadi

Fausto Milletari

  • Prof. Nassir Navab

▸ Our team at

Christine Kroll

  • Prof. Kai Bötzel
slide-22
SLIDE 22

THANK YOU!

contact: fausto.milletari@gmail.com