Segmentation Fabian Isensee Division of Medical Image Computing - - PowerPoint PPT Presentation

β–Ά
segmentation
SMART_READER_LITE
LIVE PREVIEW

Segmentation Fabian Isensee Division of Medical Image Computing - - PowerPoint PPT Presentation

Jun 25, 2023 201 likes β€’513 views

Segmentation Fabian Isensee Division of Medical Image Computing DKFZ, Heidelberg Author Division 3/14/2018 | Page2 Classification Segmentation Detection Cat 3/14/2018 | Page2 Fabian Isensee, Division of Medical Image Computing, DKFZ

slide-1
SLIDE 1

Segmentation

Fabian Isensee Division of Medical Image Computing DKFZ, Heidelberg

slide-2
SLIDE 2

Page2 3/14/2018 | Author Division 3/14/2018 | Page2 Fabian Isensee, Division of Medical Image Computing, DKFZ

Classification Segmentation Detection

Cat

slide-3
SLIDE 3

Page3 3/14/2018 | Author Division 3/14/2018 | Page3 3/14/2018 | Fabian Isensee, Division of Medical Image Computing, DKFZ

WMH ACDC BraTS Endovis Camelyon LiTS

slide-4
SLIDE 4

Page4 3/14/2018 | Author Division 3/14/2018 | Page4

Classification

Cat

Segmentation

skyskyskyskyskyskyskyskysky skyskyskyskysky tree tree tree tree

catcatcatcatcatcatcattree tree

tree tree grasscatcatcatcat grassgrass

grassgrassgrassgrasscatcatcat grassgrass grassgrassgrassgrasscatcatcat grassgrass grassgrassgrassgrassgrasscat grassgrassgrass grassgrassgrassgrassgrassgrassgrassgrassgrass

  • ne label per image
  • ne label per pixel
slide-5
SLIDE 5

Page5 3/14/2018 | Author Division 3/14/2018 | Page5

Classification

Simonyan, Zisserman, CVPR, 2014 Semantic Information Spatial Information

Segmentation

Semantic Information Spatial Information

Fabian Isensee, Division of Medical Image Computing, DKFZ

slide-6
SLIDE 6

Page6 3/14/2018 | Author Division 3/14/2018 | Page6

Fully Convolutional Networks for Semantic Segmentation

Fabian Isensee, Division of Medical Image Computing, DKFZ

Long et al., CVPR, 2015

slide-7
SLIDE 7

Page7 3/14/2018 | Author Division 3/14/2018 | Page7

Fully Convolutional Networks for Semantic Segmentation

Fabian Isensee, Division of Medical Image Computing, DKFZ

Long et al., CVPR, 2015

slide-8
SLIDE 8

Page8 3/14/2018 | Author Division 3/14/2018 | Page8 Fabian Isensee, Division of Medical Image Computing, DKFZ

Fully Convolutional Networks for Semantic Segmentation

Long et al., CVPR, 2015

slide-9
SLIDE 9

Page9 3/14/2018 | Author Division 3/14/2018 | Page9 Fabian Isensee, Division of Medical Image Computing, DKFZ

Fully Convolutional Networks for Semantic Segmentation

Long et al., CVPR, 2015

slide-10
SLIDE 10

Page10 3/14/2018 | Author Division 3/14/2018 | Page10

DeepLabv3

Chen et al., arXiv, 2017

Fabian Isensee, Division of Medical Image Computing, DKFZ

slide-11
SLIDE 11

Page11 3/14/2018 | Author Division 3/14/2018 | Page11

DeepLabv3

Chen et al., arXiv, 2017

Fabian Isensee, Division of Medical Image Computing, DKFZ

slide-12
SLIDE 12

Page12 3/14/2018 | Author Division 3/14/2018 | Page12

Pretrained ResNet

Fabian Isensee, Division of Medical Image Computing, DKFZ

DeepLabv3

+ Effective utilization of pretrained network + Performs very well on semantically demanding tasks

  • Output stride 16/8 -> no precise localization
  • No pretraining possible for MIC

Chen et al., arXiv, 2017

slide-13
SLIDE 13

Page13 3/14/2018 | Author Division 3/14/2018 | Page13

  • UNET

Fabian Isensee, Division of Medical Image Computing, DKFZ

Ronneberger et al., MICCAI, 2015

Encoder-Decoder: UNet

Encoder Decoder Skip Connections

  • utput stride 1!
slide-14
SLIDE 14

Page14 3/14/2018 | Author Division 3/14/2018 | Page14

UNet variants: UNet3D

Fabian Isensee, Division of Medical Image Computing, DKFZ

Cicek et al., MICCAI, 2016

slide-15
SLIDE 15

Page15 3/14/2018 | Author Division 3/14/2018 | Page15

UNet variants: V-Net

Fabian Isensee, Division of Medical Image Computing, DKFZ

Milletari et al., 3DV, 2016

slide-16
SLIDE 16

Page16 3/14/2018 | Author Division 3/14/2018 | Page16

UNet variants: Dense UNet

Jegou et al., CVPRW, 2017 Huang et al., CVPR, 2017

slide-17
SLIDE 17

Page17 3/14/2018 | Author Division 3/14/2018 | Page17

DeepLabv3+

Chen et al., arXiv, 2018

DeepLabv3 Encoder-Decoder DeepLabv3+

Fabian Isensee, Division of Medical Image Computing, DKFZ

slide-18
SLIDE 18

Page18 3/14/2018 | Author Division 3/14/2018 | Page18

DeepLabv3+

Chen et al., arXiv, 2018

DeepLabv3 Encoder-Decoder DeepLabv3+

Fabian Isensee, Division of Medical Image Computing, DKFZ

slide-19
SLIDE 19

Page19 3/14/2018 | Author Division 3/14/2018 | Page19

Chen et al., arXiv, 2018

Fabian Isensee, Division of Medical Image Computing, DKFZ

DeepLabv3+

slide-20
SLIDE 20

Page20 3/14/2018 | Author Division 3/14/2018 | Page20

𝑀𝐷𝐹 π‘ž, 𝑕 = βˆ’ 1 𝑂

π‘—βˆˆπ‘‚ π‘™βˆˆπΏ

𝑕𝑗,𝑙 log π‘žπ‘—,𝑙 Categorical Crossentropy

Fabian Isensee, Division of Medical Image Computing, DKFZ

Medical Image Segmentation – Loss functions

skyskyskyskyskyskyskyskysky skyskyskyskyskytree tree tree tree

catcatcatcatcatcatcat tree tree

tree tree grasscatcatcatcat grass grass

grass grass grass grasscatcatcat grass grass grass grass grass grasscatcatcat grass grass grass grass grass grass grasscat grass grass grass grass grass grass grass grass grass grass grass grass

slide-21
SLIDE 21

Page21 3/14/2018 | Author Division 3/14/2018 | Page21

Medical Image Segmentation – Loss functions

Fabian Isensee, Division of Medical Image Computing, DKFZ

slide-22
SLIDE 22

Page22 3/14/2018 | Author Division 3/14/2018 | Page22

𝑀𝑋𝐷𝐹 π‘ž, 𝑕 = βˆ’ 1 𝑂

π‘—βˆˆπ‘‚ π‘™βˆˆπΏ

π‘₯𝑗𝑕𝑗,𝑙 log π‘žπ‘—,𝑙 Categorical Crossentropy (weighted)

Medical Image Segmentation – Loss functions

Ronneberger et al., MICCAI, 2015

slide-23
SLIDE 23

Page23 3/14/2018 | Author Division 3/14/2018 | Page23

𝑀𝐸𝐽𝐷𝐹 π‘ž, 𝑕 = βˆ’ 2 |𝐿|

π‘™βˆˆπΏ

π‘—βˆˆπ‘‚ π‘žπ‘—,𝑙𝑕𝑗,𝑙 π‘—βˆˆπ‘‚ π‘žπ‘—,𝑙 + π‘—βˆˆπ‘‚ 𝑕𝑗,𝑙 Dice Loss 𝐸𝐽𝐷𝐹 𝐡, 𝐢 = 2 𝐡 ∩ 𝐢 𝐡 + |𝐢|

Fabian Isensee, Division of Medical Image Computing, DKFZ

Milletari et al., 3DV, 2016 Sudre et al., DLMIA/ML-CDS (MICCAI), 2017 Drozdzal et al., DLMIA/LABELS (MICCAI), 2016

Medical Image Segmentation – Loss functions

slide-24
SLIDE 24

Page24 3/14/2018 | Author Division 3/14/2018 | Page24

Medical Image Segmentation – Example: BraTS

www.braintumorsegmentation.org

Fabian Isensee, Division of Medical Image Computing, DKFZ

Whole Tumor Tumor Core Necrosis and Enhancing Tumor

slide-25
SLIDE 25

Page25 3/14/2018 | Author Division 3/14/2018 | Page25

BraTS 2017 3rd Place (you can get a long way with a well trained UNet)

  • Train on large patches (128x128x128)
  • DICE loss
  • A lot of data augmentation

Fabian Isensee, Division of Medical Image Computing, DKFZ

slide-26
SLIDE 26

Page26 3/14/2018 | Author Division 3/14/2018 | Page26

BraTS 2017 2nd Place

Wang et al., MICCAI-BRATS, 2017

Fabian Isensee, Division of Medical Image Computing, DKFZ

Cascade: simulate annotation procedure

slide-27
SLIDE 27

Page27 3/14/2018 | Author Division 3/14/2018 | Page27

BraTS 2017 2nd Place 144x144x19 96x96x19 64x64x19

Fabian Isensee, Division of Medical Image Computing, DKFZ

Wang et al., MICCAI-BRATS, 2017

slide-28
SLIDE 28

Page28 3/14/2018 | Author Division 3/14/2018 | Page28

BraTS 2017 1st Place

Kamnitsas et al., MICCAI-BRATS, 2017

UNet (3D) FCN (3D) DeepMedic

slide-29
SLIDE 29

Page29 3/14/2018 | Author Division 3/14/2018 | Page29

  • Use Encoder-Decoder with OS1
  • Think about your loss function
  • Properly aggregate semantic information
  • Data Augmentation
  • Ensembling

Segmentation in Medical Image Computing: Short Summary

slide-30
SLIDE 30

Page30 3/14/2018 | Author Division 3/14/2018 | Page30

Thank you!