Segmentation Fabian Isensee Division of Medical Image Computing - PowerPoint PPT Presentation

Segmentation Fabian Isensee Division of Medical Image Computing DKFZ, Heidelberg

Author Division 3/14/2018 | Page2 Classification Segmentation Detection Cat 3/14/2018 | Page2 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division 3/14/2018 | Page3 WMH ACDC LiTS BraTS Endovis Camelyon 3/14/2018 | 3/14/2018 | Page3 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division Classification 3/14/2018 | Page4 Cat one label per image Segmentation skyskyskyskyskyskyskyskysky skyskyskyskysky tree tree tree tree catcatcatcatcatcatcat tree tree tree tree grass catcatcatcat grassgrass grassgrassgrassgrass catcatcat grassgrass grassgrassgrassgrass catcatcat grassgrass grassgrassgrassgrassgrass cat grassgrassgrass grassgrassgrassgrassgrassgrassgrassgrassgrass one label per pixel 3/14/2018 | Page4

Author Division Classification 3/14/2018 | Page5 Simonyan, Zisserman, CVPR, 2014 Semantic Information Spatial Information Segmentation Semantic Information Spatial Information 3/14/2018 | Page5 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division Fully Convolutional Networks for Semantic Segmentation 3/14/2018 | Page6 Long et al., CVPR, 2015 3/14/2018 | Page6 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division Fully Convolutional Networks for Semantic Segmentation 3/14/2018 | Page7 Long et al., CVPR, 2015 3/14/2018 | Page7 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division Fully Convolutional Networks for Semantic Segmentation 3/14/2018 | Page8 Long et al., CVPR, 2015 3/14/2018 | Page8 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division Fully Convolutional Networks for Semantic Segmentation 3/14/2018 | Page9 Long et al., CVPR, 2015 3/14/2018 | Page9 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division DeepLabv3 3/14/2018 | Page10 Chen et al., arXiv, 2017 3/14/2018 | Page10 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division DeepLabv3 3/14/2018 | Page11 Chen et al., arXiv, 2017 3/14/2018 | Page11 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division DeepLabv3 3/14/2018 | Page12 Pretrained ResNet + Effective utilization of pretrained network + Performs very well on semantically demanding tasks - Output stride 16/8 -> no precise localization - Chen et al., arXiv, 2017 No pretraining possible for MIC 3/14/2018 | Page12 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division Encoder-Decoder: UNet output stride 1! 3/14/2018 | Page13 • UNET Skip Connections Encoder Decoder Ronneberger et al., MICCAI, 2015 3/14/2018 | Page13 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division UNet variants: UNet3D 3/14/2018 | Page14 Cicek et al., MICCAI, 2016 3/14/2018 | Page14 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division UNet variants: V-Net 3/14/2018 | Page15 Milletari et al., 3DV, 2016 3/14/2018 | Page15 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division UNet variants: Dense UNet 3/14/2018 | Page16 Huang et al., CVPR, 2017 Jegou et al., CVPRW, 2017 3/14/2018 | Page16

Author Division DeepLabv3+ 3/14/2018 | Page17 DeepLabv3 DeepLabv3+ Encoder-Decoder Chen et al., arXiv, 2018 3/14/2018 | Page17 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division DeepLabv3+ 3/14/2018 | Page18 DeepLabv3 DeepLabv3+ Encoder-Decoder Chen et al., arXiv, 2018 3/14/2018 | Page18 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division DeepLabv3+ 3/14/2018 | Page19 Chen et al., arXiv, 2018 3/14/2018 | Page19 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division Medical Image Segmentation – Loss functions 3/14/2018 | Page20 skyskyskyskyskyskyskyskysky skyskyskyskysky tree tree tree tree catcatcatcatcatcatcat tree tree tree tree grass catcatcatcat grass grass grass grass grass grass catcatcat grass grass grass grass grass grass catcatcat grass grass grass grass grass grass grass cat grass grass grass grass grass grass grass grass grass grass grass grass Categorical Crossentropy 𝑀 𝐷𝐹 𝑞, 𝑕 = − 1 𝑂 𝑕 𝑗,𝑙 log 𝑞 𝑗,𝑙 𝑗∈𝑂 𝑙∈𝐿 3/14/2018 | Page20 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division Medical Image Segmentation – Loss functions 3/14/2018 | Page21 3/14/2018 | Page21 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division Medical Image Segmentation – Loss functions 3/14/2018 | Page22 Categorical Crossentropy (weighted) 𝑀 𝑋𝐷𝐹 𝑞, 𝑕 = − 1 𝑂 𝑥 𝑗 𝑕 𝑗,𝑙 log 𝑞 𝑗,𝑙 𝑗∈𝑂 𝑙∈𝐿 Ronneberger et al., MICCAI, 2015 3/14/2018 | Page22

Author Division Medical Image Segmentation – Loss functions 3/14/2018 | Page23 𝐸𝐽𝐷𝐹 𝐵, 𝐶 = 2 𝐵 ∩ 𝐶 𝐵 + |𝐶| Dice Loss 𝑗∈𝑂 𝑞 𝑗,𝑙 𝑕 𝑗,𝑙 𝑀 𝐸𝐽𝐷𝐹 𝑞, 𝑕 = − 2 |𝐿| 𝑗∈𝑂 𝑞 𝑗,𝑙 + 𝑗∈𝑂 𝑕 𝑗,𝑙 𝑙∈𝐿 Milletari et al., 3DV, 2016 Sudre et al., DLMIA/ML-CDS (MICCAI), 2017 Drozdzal et al., DLMIA/LABELS (MICCAI), 2016 3/14/2018 | Page23 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division Medical Image Segmentation – Example: BraTS 3/14/2018 | Page24 Necrosis and Enhancing Tumor Whole Tumor Tumor Core www.braintumorsegmentation.org 3/14/2018 | Page24 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division BraTS 2017 3rd Place (you can get a long way with a well trained UNet) 3/14/2018 | Page25 - Train on large patches (128x128x128) - DICE loss - A lot of data augmentation 3/14/2018 | Page25 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division BraTS 2017 2nd Place 3/14/2018 | Page26 Cascade: simulate annotation procedure Wang et al., MICCAI-BRATS, 2017 3/14/2018 | Page26 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division BraTS 2017 2nd Place 3/14/2018 | Page27 144x144x19 64x64x19 96x96x19 Wang et al., MICCAI-BRATS, 2017 3/14/2018 | Page27 Fabian Isensee, Division of Medical Image Computing, DKFZ

Author Division BraTS 2017 1st Place 3/14/2018 | Page28 UNet (3D) FCN (3D) DeepMedic Kamnitsas et al., MICCAI-BRATS, 2017 3/14/2018 | Page28

Author Division Segmentation in Medical Image 3/14/2018 | Page29 Computing: Short Summary • Use Encoder-Decoder with OS1 • Think about your loss function • Properly aggregate semantic information • Data Augmentation • Ensembling 3/14/2018 | Page29

Author Division 3/14/2018 | Page30 Thank you! 3/14/2018 | Page30