Mutual information deep regularization for semi- supervised - - PowerPoint PPT Presentation

Mutual information deep regularization for semi- supervised segmentation

• J. Peng, M. Pedersoli, C. Desrosiers

MIDL 2020 presentation

Outline

We proposed a semi-supervised segmentation method for medical image regularized by Mutual Information

• Introduction on Semi-Supervised Segmentation
• Explanation on Mutual information concept for segmentation
• Our proposed scheme
• Experimental setup and results
• Conclusion

Medical Image segmentation

• Important stage for quantification (volume), visualization, intra-operative navigation,

radiotherapy and clinical-oriented analysis

• Widely employed with CT MRI, and X-ray and Ultrasound for organs such as brain,

lung, spleen, prostate organs.

Modalities: MRI CT Ultrasound Cardiac Segmentation

Limited access to annotations

Well know constraint for MIDL:

• require experts with long experience
• require annotation slice by slice
• cost hours for a single patient
• privacy requirement

Labeling is hard for 3D volumns Three different views of a patient from [1]

ref: [1]: Zhuang, Xiahai, et al. "Evaluation of algorithms for Multi-Modality Whole Heart Segmentation: An open-access grand challenge." Medical image analysis 58 (2019): 101537.

Semi-supervised learning framework

Labeled images Supervised loss: ➢ Dice loss [1] ➢ Cross entropy loss [2] ➢ Uncertainty based loss [3] Regularization loss: ➢ Consistency based reg. [4-7] ➢ Prior-enabled based reg.[8] ➢ Entropy based reg.[9] ➢ Mutual information based reg. Unlabeled images

Ref: [1]: Sudre, Carole H., et al. [2]:Zhang, Zhilu, and Mert Sabuncu, 2018. [3]: Kendall, Alex, et al.,CVPR 2018. [4]: Perone, et al., 2018. [5]: Li, Xiaomeng, et al.,

• 2018. [6]: Wang, Fan, et al.,.ICCV 2013, [7]: Cui, Wenhui, et al. 2019. [8]: Zheng, Han, et al. MICCAI 2019. [9]: Vu, Tuan-Hung, et al. CVPR 2019.

Mutual information

measures the amount of information that two variables X, Y share:. if X, Y are independent, then p(X,Y) = p(X)p(Y), I (X;Y) = 0

MI = 0 MI Maximized MI Maximized

Mutual information for different transformation

measures the amount of information that two variables X, Y share:. p(X)

How about having X and Y as a segmentation distribution? Case1: We explore the consistency with MI

p(Y) Maximazing I(X;Y)

Afffine transfrom Same transfrom

Mutual information on nearby patches

measures the amount of information that two variables X, Y share:. p(X)

How about having X and Y as a segmentation distribution? Case2: We explore the structural information of nearby patches by maximazing the MI. MI does not require a strict assignment mapping.

p(Y)

shift n pixels

unlabeled image Maximazing I(X;Y)

How to compute I(X; Y) for 2D?

We compute the joint distribution by using product of the two marginal distribution (conditionally independent given the same input image) We compute the MI from the joint probability matrix.

2D convolution

The proposed scheme

ref: [1]: Perone, Christian S., and Julien Cohen-Adad. "Deep semi-supervised segmentation with weight-averaged consistency targets." Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Springer, Cham, 2018. 12-19. [2]: Vu, Tuan-Hung, et al. "Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019.

We examine the proposed idea in a semi supervised learning setting on three benchmark datasets: ACDC, prostate and spleen. ACDC: 4% data as labeled, 83.5% as unlabeled, 12.5% as validation Prostate: 14% data as labeled, 66% as unlabeled, 20% as validation Spleen: 10% as labeled data, 78% as unlabeled, 12% as validation. We compared our method against mean teacher [1] and entropy minimization [2]

Experimental results

Experimental results

ref: [1]: Zhuang, Xiahai, et al. "Evaluation of algorithms for Multi-Modality Whole Heart Segmentation: An open-access grand challenge." Medical image analysis 58 (2019): 101537.

Conclusion

• We proposed a semi-supervised learning algorithm employing mutual information

as a deep regularization

• Mutual information can capture structural/schematic information of organs which

are usually under regular shape.

• We examined the proposed idea on three benchmark dataset and have proven

that the proposed method achieved significant improvement compared with baseline method and comparable performance compared with SOTA methods.