Lung Nodule Detection based on Convolutional Neural Networks Julio - PowerPoint PPT Presentation

Lung Nodule Detection based on Convolutional Neural Networks Julio Mendoza Bobadilla Advisor: Prof. Helio Pedrini Master’s Defense Institute of Computing University of Campinas December 18, 2017 Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 1 / 43

Summary 1. Introduction Problem Motivation Hypotheses Contributions 2. Background CAD Systems Lung Area Segmentation Candidate Nodule Localization Candidate Nodule Classification 3. Methodology 4. Results 5. Conclusions Future Work Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 2 / 43

Introduction Problem Input: Chest X-Ray Output: Potential nodules highlighted Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 3 / 43

Introduction Motivation Estimated number of cancer deaths on both sexes (Source: World Cancer Report 2014 1 ). Improve the performance of radiologists on lung cancer screening. Address the drawbacks of lung cancer screening with CT images. Explore the potential role of CAD systems with CXR for lung cancer screening. Leverage the advantages of deep learning approaches on lung nodule detection. 1McGuire, “World Cancer Report 2014. Geneva, Switzerland: World Health Organization, International Agency for Research on Cancer, WHO Press”, 2016. Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 4 / 43

Introduction Hypotheses Main hypothesis: Convolutional Neural Networks (CNNs) trained from the scratch can perform well on lung nodule classification by setting proper regularization and optimization methods. Secondary hypotheses: The design of a specialized architecture for the classification stage of lung nodule detection improve the performance of our method. The regularization effects of data augmentation, dropout and weight penalties critical for lung nodule classification due to the amount of samples used in training. The regularization and optimization effects of unsupervised objectives in loss functions improve CNN performance in lung nodule classification. Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 5 / 43

Introduction Contributions Main contributions: An analysis and evaluation of methods for lung area segmentation and candidate nodule localization. The proposition of a method for lung nodule classification based a CNN trained from the scratch. The comparison of the our lung nodule detection pipeline with other approaches of the literature. Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 6 / 43

Background CAD Systems Usage modalities: As first reader: radiologists analyze the regions detected by the CAD system. As second reader: CAD system may detect nodules missed by radiologists. CAD systems usually detect nodules solving three subproblems: Lung Area Candidate Nodule Candidate Nodule Segmentation Localization Classi fi cation Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 7 / 43

Background Lung Area Segmentation Lung Area Lung Area Candidate Nodule Candidate Nodule Candidate Nodule Candidate Nodule Segmentation Segmentation Localization Localization Classi fi cation Classi fi cation Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 8 / 43

Background Active Appearance Model Shape model ◮ Normalization with Procrustes Analysis. ◮ Finding bases with Principal Component Analysis. n � s = ¯ + p i s s i ���� ���� i =1 mean shape shape bases Appearance model ◮ Appearance representation using visual descriptors. ◮ Finding bases with Principal Component Analysis. m � ¯ A = A + c i A i ���� ���� i =1 mean appearance appearance bases Model instantiation m � M ( W ( x , p ) ) ≃ ¯ A + c i A i i =1 Model fitting � � �� 2 m � � � p ∗ , c ∗ = arg min ¯ � � � M ( W ( x , p ) ) − A + c i A i � � p , c � i =1 Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 9 / 43

Background Active Appearance Model Source: Matthews and Baker 2 2Matthews and Baker, “Active Appearance Models Revisited”, 2004. Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 10 / 43

Background Candidate Nodule Localization Lung Area Candidate Nodule Candidate Nodule Segmentation Localization Classi fi cation Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 11 / 43

Background Scale-Space based Detectors t t local extrema scale scale L ( · , · ; t ) = g ( · , · ; t ) ∗ f ( · , · ) Laplacian of Gaussian blobs Scale-space blob detection. normalized Laplacian of Gaussian (LoG): t ∇ L = t ( L xx + L yy ) normalized Determinant of Hessian (DoH): t ∇ L = t 2 ( L xx L yy − L 2 xy ) Difference of Gaussians (DoG): DoG ( t ) = L ( · , · ; t + ∆ t ) − L ( · , · ; t ) Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 12 / 43

Background Convergence Index based detectors R gradient vector Q θ (k, l) P (i, j) Convergence degree at point Q : cos θ ( k , l ) Convergence index at point P with support region R : CI ( i , j ) = 1 � cos θ ( k , l ) M ( k , l ) ∈ R Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 13 / 43

Background Sliding Band Filter R ... B 3 B 2 B 1 d P B 0 B 15 B 14 ... r max +d r min Convergence index at point P with support region R :         N − 1 n + d SB ( x , y ) = 1 1 � �     max cos θ i , m     N d     r min ≤ n ≤ r max   m = n   i =0 � �� � CI of a band in the line B i Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 14 / 43

Background Candidate Nodule Classification Lung Area Candidate Nodule Candidate Nodule Segmentation Localization Classi fi cation Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 15 / 43

Background Feature Engineering Classi fi er Input nodules Feature Extraction . . . Feature extraction: geometric, shape, intensity and gradient features. Binary classification. Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 16 / 43

Background Feature Engineering Classi fi er Input nodules Nodule Feature Feature Segmentation Extraction Selection / Reduction . . . Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 17 / 43

Background Convolutional Neural Networks Convolutional Neural Networks ◮ Convolution ◮ Pooling Input nodules ConvNet . . . Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 18 / 43

Methodology Overview Overview of the main steps of the proposed methodology. We segment the lung area using patch-based AAM. We find candidate locations with an SB filter. We estimate the probability of nodules being candidates using a CNN. Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 19 / 43

Methodology Segmentation Overview of the segmentation stage. Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 20 / 43

Methodology Segmentation Low resolution model High resolution model Visualization of Patch-based AAM (for low and high resolution models) fitted on the sample JPCLN001.IMG from the JSRT dataset. Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 21 / 43

Methodology Detection Non-maxima suppression with r sbf = 7px Input X-ray ( JPCLN001.IMG ) SB filter output Candidate nodules detected with an SB filter. Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 22 / 43

Methodology Classification Overview of the classification stage. Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 23 / 43

Methodology Data Preparation and Augmentation Data preparation: Labeling criterion: 25mm. Z-score normalization. Data augmentation: Affine transformations (translation, rotation, scaling, flip). Intensity shifts. Julio Mendoza Bobadilla IC/UNICAMP Lung Nodule Detection based on Convolutional Neural Networks December 18, 2017 24 / 43