Combined Radiology and Pathology Classification of Brain Tumors - - PowerPoint PPT Presentation

The problem State of the art Proposed approach Achieved results

Combined Radiology and Pathology Classification of Brain Tumors

Rozpoznanie guza mózgu na podstawie obrazu radiologicznego i patologicznego Piotr Giedziun

Supervisor: dr hab. in˙

• z. Henryk Maciejewski

4 March 2016

Piotr Giedziun Wrocław University of Technology 4 March 2016 1 / 25

The problem State of the art Proposed approach Achieved results

Outline

1

The problem

2

State of the art

3

Proposed approach

4

Achieved results

Piotr Giedziun Wrocław University of Technology 4 March 2016 2 / 25

The problem State of the art Proposed approach Achieved results

Definitions

Cancer Cancer occurs when abnormal cells grow out of control Brain tumor Benign or Malignant Over time, a low-grade tumor can become a high-grade tumor Brain tumors are classified as grade I, grade II, or grade III, or grade IV

Piotr Giedziun Wrocław University of Technology 4 March 2016 3 / 25

The problem State of the art Proposed approach Achieved results

Brain tumor - Survival rate (5 years or more)

FIGURE – Based on data from SEER 18 2005-2011, cancer.gov

Piotr Giedziun Wrocław University of Technology 4 March 2016 4 / 25

The problem State of the art Proposed approach Achieved results

Brain tumor - Survival by stage

FIGURE – Ovarian cancer, Five-year stage-specific relative survival rates, adults (ages 15-99), Anglia Cancer Net-

work, 1987-2008

Piotr Giedziun Wrocław University of Technology 4 March 2016 5 / 25

The problem State of the art Proposed approach Achieved results

Brain tumor - Diagnosis process

General Practitioner Neurologist MR or CT - Radiologists (tumor confirmed) Neurosurgeon Benign Pathologist (final diagnosis) Malignant 2 Pathologists (final diagnosis) (clear image)

Piotr Giedziun Wrocław University of Technology 4 March 2016 6 / 25

The problem State of the art Proposed approach Achieved results

Diagnosis problems

Problems Diverse shapes, sizes and appearances of tumors Relies on histopathologic examination (biopsy examination) Waiting for tests and to start treatment Radiology imaging is used only to establish location, size and whether it is benign and malignant tumor

FIGURE – Glioblastoma cells FIGURE – Oligodendroglioma cells

Piotr Giedziun Wrocław University of Technology 4 March 2016 7 / 25

The problem State of the art Proposed approach Achieved results

Diagnosis problems

Problems Diverse shapes, sizes and appearances of tumors Relies on histopathologic examination (biopsy examination) Waiting for tests and to start treatment Radiology imaging is used only to establish location, size and whether it is benign and malignant tumor Targets in the UK No more than 2 months wait between the date the hospital receives an urgent GP referral for suspected cancer and starting treatment

FIGURE – Glioblastoma cells FIGURE – Oligodendroglioma cells

Piotr Giedziun Wrocław University of Technology 4 March 2016 7 / 25

The problem State of the art Proposed approach Achieved results

Aims & Limitations

Aims Research & build a segmentation mechanism for the MRI scans (ROI selection) Research & build a classifier based on the segmented radiological images (if possible) Combine the Pathology-based classification with radiology-based classifier Limitations Limited access to the MRI samples with the diadnosis provided by the doctor Conservative environment - only non-black box models

Piotr Giedziun Wrocław University of Technology 4 March 2016 8 / 25

The problem State of the art Proposed approach Achieved results

Related work

Brain tumor segmentation The topic of brain segmentation is relatively popular thanks to BraTS challenge Several supervised and unsupervised algorithms were proposed

Random Decision Forest that classifies voxels Fuzzy C-means clustering Mean Shift and K-means clustering

Brain tumor classification Slightly less popular subject (current diagnosis fully rely on histopathology imaging) Feature extraction

Extraction of structure information Feature selection

GLCM (Gray-Level Co-occurrence Matrix)

Piotr Giedziun Wrocław University of Technology 4 March 2016 9 / 25

The problem State of the art Proposed approach Achieved results

Influential articles

Joana Festa and Sérgio Pereira and José António Mariz and Nuno Sousa and Carlos A. Silva Automatic Brain Tumor Segmentation of Multi-sequence MR images using Random Decision Forests Proceedings of NCI-MICCAI BRATS 2013, Nagoya, Japan, 2013 Nitish Zulpe and Vrushsen Pawar GLCM Textural Features for Brain Tumor International Journal of Computer Science, 2012 Hassan Khotanlou, Olivier Colliot, and Isabelle Bloch Automatic brain tumor segmentation using symmetry analysis and deformable models Nationale Superieure des Telecommunications, 2007

Piotr Giedziun Wrocław University of Technology 4 March 2016 10 / 25

The problem State of the art Proposed approach Achieved results

Brain tumor - Modified diagnosis process

General Practitioner Neurologist MR or CT - Radiologists & piece of software (tumor confirmed & partial diagnosis) Neurosurgeon Benign Pathologist (final diagnosis) Malignant 2 Pathologists (final diagnosis) (clear image)

Piotr Giedziun Wrocław University of Technology 4 March 2016 11 / 25

The problem State of the art Proposed approach Achieved results

Data set

5 10 15 20 25 30 35 Sample No. 100 200 300 400 500 600 700 800 900 Average intensity 5 10 15 20 25 30 35 Sample No. 1000 2000 3000 4000 5000 6000 7000 Max intensity 5 10 15 20 25 30 35 Sample No. 200 250 300 350 400 450 500 550 Width

FIGURE – Plots of different attributes of the data set

Angle 1 Angle 2 Angle 3

FIGURE – Viewing angles of MRI scan

Piotr Giedziun Wrocław University of Technology 4 March 2016 12 / 25

The problem State of the art Proposed approach Achieved results

Data set

Summary 27 cases with lower grade glioma tumors 13 of them with Oligodendroglioma and 14 with Astrocytoma Each case has 3 or 4 MRI scans (T1, T1C, FLAIR, and T2) Provided samples were taken using different hardware

5 10 15 20 25 30 35 Sample No. 100 200 300 400 500 600 700 800 900 Average intensity 5 10 15 20 25 30 35 Sample No. 1000 2000 3000 4000 5000 6000 7000 Max intensity 5 10 15 20 25 30 35 Sample No. 200 250 300 350 400 450 500 550 Width

FIGURE – Plots of different attributes of the data set

Angle 1 Angle 2 Angle 3

FIGURE – Viewing angles of MRI scan

Piotr Giedziun Wrocław University of Technology 4 March 2016 13 / 25

The problem State of the art Proposed approach Achieved results

Pre-processing

50 100 150 200 250 Intensity level 1000 2000 3000 4000 5000 Numer of pixels

Histogram Tresholded binary map Extracted skull binary map

FIGURE – Process of skull extraction

FLAIR skull figure T2 skull figure

FIGURE – Skulls properties in FLAIR and T2

Piotr Giedziun Wrocław University of Technology 4 March 2016 14 / 25

The problem State of the art Proposed approach Achieved results

Pre-processing

20 40 60 80 100 120 140 160 180 Intensity level 1000 2000 3000 4000 5000 6000 Numer of pixels

Histogram before FLAIR before

20 40 60 80 100 120 140 160 180 Intensity level 1000 2000 3000 4000 5000 6000 7000 8000 Numer of pixels

Histogram after FLAIR after

FIGURE – Median filter effect on image histogram

Piotr Giedziun Wrocław University of Technology 4 March 2016 15 / 25

The problem State of the art Proposed approach Achieved results

Segmentation - K-Means

0.1 0.0 0.2 0.4 0.6 0.8 1.0 The silhouette coefficient values Cluster label 1 2 3

The silhouette plot for the various clusters. Feature space for the X 0.0 0.2 0.4 0.6 0.8 1.0 1.2 Feature space for the Y 0.0 0.2 0.4 0.6 0.8 1.0 1.2 Feature space for the Intensity 2 2 4 6 8 10 12 14

The visualization of the clustered data.

FIGURE – Silhouette analysis for K-Means(k=5)

Piotr Giedziun Wrocław University of Technology 4 March 2016 16 / 25

The problem State of the art Proposed approach Achieved results

Segmentation - Combined

K-Means segmentation Symetry analysis segmentation Result K-Means segmentation Symetry analysis segmentation Result K-Means segmentation Symetry analysis segmentation Result

FIGURE – Segmentation with results

Piotr Giedziun Wrocław University of Technology 4 March 2016 17 / 25

The problem State of the art Proposed approach Achieved results

Segmentation - Alternatives

3 4 5 6 7 8 Number of clusters 20 40 60 80 100 120 Scores Modified K-Means vs K-Means Mini Batch K-Means K-Means

FIGURE – Mini K-Means

3 4 5 6 7 8 Number of clusters 20 40 60 80 100 120 Scores Agglomerative Clustering vs K-Means Agglomerative K-Means

FIGURE – Agglomerative cluste-

ring

3 4 5 6 7 8 Number of clusters 20 40 60 80 100 120 Scores Modified K-Means vs K-Means Modified K-Means K-Means

FIGURE – K-Means with position

Piotr Giedziun Wrocław University of Technology 4 March 2016 18 / 25

The problem State of the art Proposed approach Achieved results

Classification

Tested methods Feature extraction & evaluation Texture features extraction with Gray-Level Co-Occurrence Matrix Texture features extraction with Local Binary Pattern Classification algorithms SVM (Support vector machine) Gaussian Naive Bayes Logistic Regression Random Forest

Piotr Giedziun Wrocław University of Technology 4 March 2016 19 / 25

The problem State of the art Proposed approach Achieved results

Classification - Feature extraction & evaluation

Selected features (out of 59) Tumor volume (in mm3) Tumor position (x,y,z) calculated from the middle of the brain Metrics intensity of tumor area 8 bins of intensity histogram

2000 4000 6000 8000 10000 12000 14000 16000 Values of FLAIR 4th histogram bin 5000 10000 15000 20000 Values of FLAIR 5th histogram bin Oligodendroglioma Astrocytoma

FIGURE – Selected features extracted from data set

pos_x 0.6 0.4 0.2 0.0 0.2 0.4 0.6 pos_y 0.6 0.4 0.2 0.0 0.2 0.4 0.6 p

• s

_ z 0.4 0.3 0.2 0.1 0.0 0.1 0.2 0.3 0.4 The visualization of tumor position. Temporal lobe Frontal lobe Brain Oligodendroglioma Astrocytoma

FIGURE – Tumor positional features

Piotr Giedziun Wrocław University of Technology 4 March 2016 20 / 25

The problem State of the art Proposed approach Achieved results

Classification - Texture features extraction with GLCM & LBP

Oligodendroglioma Astrocytoma O #1 O #2 O #3 O #4 O #5 A #1 A #2 A #3 A #4 A #5 10 20 30 40 50 60 GLCM Dissimilarity 0.2 0.0 0.2 0.4 0.6 0.8 1.0 GLVM Correlation 1 2 3 4 5 1 2 3 4 5 O A

FIGURE – Co-occurence matrix features for Oligodendroglioma and Astrocytoma

Piotr Giedziun Wrocław University of Technology 4 March 2016 21 / 25

The problem State of the art Proposed approach Achieved results

Radiology imaging

Tumor segmentation METHOD BEST SCORE Mini Batch K-Means (5 clusters) 89.027% (std : 5.408) K-Means (5 clusters) 88.168% (std : 5.264) K-Means with position (5 clusters) 86.026% (std : 5.282) Agglomerative Clustering 88.956% (std : 10.632) Cancer classification METHOD BEST SCORE Random Forest Classifier 87.000% (std : 12.991) Logistic Regression 81.297% (std : 5.744) Logistic Regression (texture) 68.285% (std : 0.082)

Piotr Giedziun Wrocław University of Technology 4 March 2016 22 / 25

The problem State of the art Proposed approach Achieved results

Combined Radiology and Pathology

0.0 0.2 0.4 0.6 0.8 1.0 Radiology Classifier estimate 0.0 0.2 0.4 0.6 0.8 1.0 Pathology Classifier estimate

Oligodendroglioma Astrocytoma

FIGURE – Comparison of Pathology and Radiology results (average estimations of Oligodendroglioma cancer for

each sample)

Piotr Giedziun Wrocław University of Technology 4 March 2016 23 / 25

The problem State of the art Proposed approach Achieved results

Results

Conclusion Random Forest classifier validated with k-fold cross validation had average accuracy of 87.0% Pre-processing of the input data is a hand-crafted process, that had to be performed K-Means had the best score out of Mini Batch K-Means, K-Means with modified input vector (with position), and Agglomerative clustering

Piotr Giedziun Wrocław University of Technology 4 March 2016 24 / 25

The problem State of the art Proposed approach Achieved results

Combined Radiology and Pathology Classification of Brain Tumors

Rozpoznanie guza mózgu na podstawie obrazu radiologicznego i patologicznego Piotr Giedziun

Supervisor: dr hab. in˙

• z. Henryk Maciejewski

4 March 2016

Piotr Giedziun Wrocław University of Technology 4 March 2016 25 / 25