[PDF] - Computer- -Aided Diagnosis in Aided Diagnosis in Computer Medical PDF Document

SLIDE 1

Axel Wismüller, Dept. of Radiology, University of Munich

Computer Computer-

Aided Diagnosis in

Aided Diagnosis in Medical Imaging: From Pattern Medical Imaging: From Pattern Recognition to Clinical Validation Recognition to Clinical Validation

Axel Wismüller

Dept. of Radiology, Klinikum Innenstadt,

University of Munich, Germany

http://www.wismueller.de

Axel Wismüller, Dept. of Radiology, University of Munich

Computer Computer-

Aided

Aided Diagnosis Diagnosis

SLIDE 2

Axel Wismüller, Dept. of Radiology, University of Munich

Diagnosis

„ „Vision“: Computer Vision“: Computer-

Aided Diagnosis

Aided Diagnosis

Computer Output Image Medical Expert Image

Axel Wismüller, Dept. of Radiology, University of Munich

„ „Vision“: Computer Vision“: Computer-

Aided Diagnosis

Aided Diagnosis

Glioblastoma multiforme p = 0.95 Perifocal edema Midline shift

Science Science Fiction Fiction ?! ?!

SLIDE 3

Axel Wismüller, Dept. of Radiology, University of Munich

Medical Medical Image Image Processing Processing

Segmentation
Registration
Classification
Image Sequence Analysis

Axel Wismüller, Dept. of Radiology, University of Munich

Medical Medical Image Image Processing Processing

Segmentation: Identification of `meaningful´ image components

SLIDE 4

Axel Wismüller, Dept. of Radiology, University of Munich

Medical Medical Image Image Processing Processing

the unsolved problem of medical image processing and computer visualization Segmentation is

C. Pelizzari

Axel Wismüller, Dept. of Radiology, University of Munich

Image Image Segmentation Segmentation

Manual segmentation by human experts is

time-consuming, expensive

Not feasible in clinical practice

→ Automatic segmentation is desirable !

SLIDE 5

Axel Wismüller, Dept. of Radiology, University of Munich

Image Image Segmentation Segmentation

Relevant application problem in neuroradiology: Segmentation of 3D MRI data sets

f the human brain into the structure classes

`Gray Matter´, `White Matter´, and `Cerebrospinal Fluid´ (CSF)

Axel Wismüller, Dept. of Radiology, University of Munich

Why Why Brain Brain Segmentation Segmentation? ?

Potential applications in neurology / psychiatry:

Alzheimer´s Dementia: Identification and monitoring
f structural changes by precise volume measurements
Multiple Sclerosis: White Matter Lesions (WML) and

quantitative measures for brain atrophy

In general: Clinical studies based on quantitative

evaluation of disease progression and therapeutic effects

SLIDE 6

Axel Wismüller, Dept. of Radiology, University of Munich

Motivation: Motivation: Satellite Satellite Remote Remote Sensing Sensing

Axel Wismüller, Dept. of Radiology, University of Munich

Motivation: Motivation: Satellite Satellite Remote Remote Sensing Sensing

Segmentation

f multispectral

satellite data LANDSAT 6 channels

SLIDE 7

Axel Wismüller, Dept. of Radiology, University of Munich

Multispectral Multispectral MRI MRI Data Data Sets Sets

T1 T1-

weighted

weighted T2 T2-

weighted

weighted Proton Proton Density Density Inversion Inversion Recovery Recovery

Axel Wismüller, Dept. of Radiology, University of Munich

Multispectral Multispectral Image Analysis Image Analysis

Registration

⇒ Finally: Construction of feature vectors x = (gT1, gT2, gPD, gIR) gj , j ∈{T1, T2, PD, IR}: signal intensities

Pre-segmentation

SLIDE 8

Axel Wismüller, Dept. of Radiology, University of Munich

Segmentation Segmentation Approaches Approaches

Unsupervised cluster analysis

(Vector quantization) Minimal free energy VQ, self-organizing maps, fuzzy c-means, etc.

Supervised classification

(GRBF neural network)

Deformable feature map: Mixture between

unsupervised and supervised learning component

Axel Wismüller, Dept. of Radiology, University of Munich

Unsupervised Unsupervised Cluster Analysis Cluster Analysis

`White matter´ `CSF´ `Gray matter´ Assign pixels to codebook vectors according to minimal distance criterion in the feature space...

SLIDE 9

Axel Wismüller, Dept. of Radiology, University of Munich

Unsupervised Unsupervised Cluster Analysis Cluster Analysis

T1-weighted image Segmentation result Combination of all the codebook vectors belonging to a specific tissue class ⇒ Segmentation

Axel Wismüller, Dept. of Radiology, University of Munich

GRBF GRBF Neural Neural Network Network

Hidden layer: ∑ =

⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ ρ − − ⎟ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎜ ⎝ ⎛ ρ − − =

N i i i j j j

a

1 2 2 2 2

2 exp 2 exp ) ( w x w x x

Output layer:

) ( ) (

1

x s x y

j N j ja

∑

=

Structure

sj wj

Input layer:

x ∈ Rn

Information Flow

SLIDE 10

Axel Wismüller, Dept. of Radiology, University of Munich

Supervised Supervised Classification Classification

Acquisition of a training data set: Manual classification of a pixel subset (ca. 1 %)

Axel Wismüller, Dept. of Radiology, University of Munich

Supervised Supervised Classification Classification

T1-weighted image Segmentation result of GRBF classification

SLIDE 11

Axel Wismüller, Dept. of Radiology, University of Munich

Can we re-utilize prevoiously acquired knowledge in order to economize the segmentation procedure? Question:

Automatic Automatic Segmentation Segmentation

Axel Wismüller, Dept. of Radiology, University of Munich

Deformable Deformable Feature Feature Map Map

Reference data Test data

+ + + + + +

Source Space X Target Space Y

{ }

q ,..., 1 , ∈ μ

μ

x

{ }

N j

j

,..., 1 , ∈ w

n j

R ∈ w x ,

μ

{ }

p ,..., 1 , ∈ ν

ν

y

{ }

N j

j

,..., 1 , ∈ r

n j

R ∈ r y ,

ν

Feature Vectors Codebook Vectors

S

A. Wismüller and H. Ritter: The Deformable Feature Map – A Novel

Neurocomputing Algorithm for Adaptive Plasticity in Pattern Analysis. Neurocomputing 48:107-139 (2002)

A. Wismüller et al.: Fully Automated Biomedical

Image Segmentation by Self-Organized Model Adaptiation. Neural Networks 17:1327-1344 (2004)

SLIDE 12

Axel Wismüller, Dept. of Radiology, University of Munich

Deformable Deformable Feature Feature Map Map

Reference data Test data

+ + + + + +

Source Space X Target Space Y

{ }

q ,..., 1 , ∈ μ

μ

x

{ }

N j

j

,..., 1 , ∈ w

n j

R ∈ w x ,

μ

{ }

p ,..., 1 , ∈ ν

ν

y

{ }

N j

j

,..., 1 , ∈ r

n j

R ∈ r y ,

ν

Feature Vectors Codebook Vectors

S Reference: Individual Y Test data: Individual X

Axel Wismüller, Dept. of Radiology, University of Munich

Cluster assignment and classification Volumes WM, GM, CSF, WML RC

Interpreta- tion of the CVs Vector quantization Spatial contingency threshold Composite cluster assignment maps Gray level spectrum of the CVs

Volumes WM, GM, CSF, WML RC

PBV evaluation Spatial contingency threshold GRBF classification on ICC Raw data Training data, ICC Preliminary GRBF classification Gray level inhomogeneity correction Co-registration gray level rescaling Iteration loop

System System Development Development

SLIDE 13

Axel Wismüller, Dept. of Radiology, University of Munich

Cluster assignment and classification Volumes WM, GM, CSF, WML RC

Interpreta- tion of the CVs Vector quantization Spatial contingency threshold Composite cluster assignment maps Gray level spectrum of the CVs

Volumes WM, GM, CSF, WML RC

PBV evaluation Spatial contingency threshold GRBF classification on ICC Raw data Training data, ICC Preliminary GRBF classification Gray level inhomogeneity correction Co-registration gray level rescaling Iteration loop

Preprocessing Preprocessing

Raw data Training data, ICC Preliminary GRBF classification Gray level inhomogeneity correction Co-registration gray level rescaling Iteration loop Axel Wismüller, Dept. of Radiology, University of Munich

Cluster assignment and classification Volumes WM, GM, CSF, WML RC

Interpreta- tion of the CVs Vector quantization Spatial contingency threshold Composite cluster assignment maps Gray level spectrum of the CVs

Volumes WM, GM, CSF, WML RC

PBV evaluation Spatial contingency threshold GRBF classification on ICC Raw data Training data, ICC Preliminary GRBF classification Gray level inhomogeneity correction Co-registration gray level rescaling Iteration loop

Unsupervised Learning Unsupervised Unsupervised Learning Learning

Raw data Training data, ICC Preliminary GRBF classification Gray level inhomogeneity correction Co-registration gray level rescaling Iteration loop

Cluster assignment and classification Volumes WM, GM, CSF, WML RC

Interpreta- tion of the CVs Vector quantization Spatial contingency threshold Composite cluster assignment maps Gray level spectrum of the CVs

SLIDE 14

Axel Wismüller, Dept. of Radiology, University of Munich

Cluster assignment and classification Volumes WM, GM, CSF, WML RC

Interpreta- tion of the CVs Vector quantization Spatial contingency threshold Composite cluster assignment maps Gray level spectrum of the CVs

Volumes WM, GM, CSF, WML RC

PBV evaluation Spatial contingency threshold GRBF classification on ICC Raw data Training data, ICC Preliminary GRBF classification Gray level inhomogeneity correction Co-registration gray level rescaling Iteration loop

Supervised Learning (GRBF Network) Supervised Supervised Learning Learning (GRBF (GRBF Network Network) )

Raw data Training data, ICC Preliminary GRBF classification Gray level inhomogeneity correction Co-registration gray level rescaling Iteration loop

Cluster assignment and classification Volumes WM, GM, CSF, WML RC

Interpreta- tion of the CVs Vector quantization Spatial contingency threshold Composite cluster assignment maps Gray level spectrum of the CVs

Volumes WM, GM, CSF, WML RC

PBV evaluation Spatial contingency threshold GRBF classification on ICC Axel Wismüller, Dept. of Radiology, University of Munich

Multiple Multiple Sclerosis Sclerosis: Image : Image Data Data

PD T2 FLAIR T1 T1+C MT MT+C

SLIDE 15

Axel Wismüller, Dept. of Radiology, University of Munich

Brain Brain Segmentation Segmentation in MS in MS

Cluster assignment maps Codebook vectors

Axel Wismüller, Dept. of Radiology, University of Munich

Assignment maps of demyelination plaques: Clinical application: Quantitative therapy control

` `Lesion Lesion Load Load´ in Multiple ´ in Multiple Sclerosis Sclerosis

SLIDE 16

Axel Wismüller, Dept. of Radiology, University of Munich

Conclusion Conclusion: : Segmentation Segmentation

Development and evaluation of an automatic segmentation

system for multispectral MRI data of the human brain

Clinical relevance: practical application for high-precision

quantitative therapy control in Multiple Sclerosis

New algorithm for self-organized model adaptation in high-

dimensional feature spaces: `Deformable Feature Map´

Special features: Inhomogeneity correction, automatic

extraction of interesting image regions, multispectral image synthesis

Axel Wismüller, Dept. of Radiology, University of Munich

Problems Problems Related Related to to This This Workshop Workshop

Number of well-documented data sets for

validation of such CAD approaches is small

Data are, in general, not publicly available on

the web

Data available lack clinically relevant MR

sequences (e.g. FLAIR)

However, creation and documentation of such

data is expensive, requires funding.

This is why we are here!

SLIDE 17

Axel Wismüller, Dept. of Radiology, University of Munich

Image Image Sequence Sequence Analysis Analysis

Axel Wismüller, Dept. of Radiology, University of Munich

Example Example: : Functional Functional MRI MRI

Interpretation of spatio-temporal fMRI activation patterns induced by a so-called `stimulus´ (optic, acoustic, motor, cognitive...)

Information processing in the brain
Localized activity of neural tissue
Changing physiological parameters
Changing MR imaging properties

SLIDE 18

Axel Wismüller, Dept. of Radiology, University of Munich

Conventional Conventional fMRI fMRI Analysis Analysis

... requires knowledge about the stimulus
However, this knowledge is often

not available not reproducible

Example: spontaneuous neurological events

(e.g. epileptic fits, hallucinations, sleep)

Axel Wismüller, Dept. of Radiology, University of Munich

Problem Problem

Is there a fMRI data analysis method that does not require knowledge about the stimulus function?

SLIDE 19

Axel Wismüller, Dept. of Radiology, University of Munich

Answers Answers

Vector Quantization

(e.g. Minimal Free Energy VQ)

Mutual Connectivity Analysis

(MCA)

Axel Wismüller, Dept. of Radiology, University of Munich

Functional Functional MRI MRI Time Time-

Series

Series VQ VQ

Cluster Assignment Maps Codebook Vectors

SLIDE 20

Axel Wismüller, Dept. of Radiology, University of Munich

Cluster Assignment Maps Codebook Vectors

Functional Functional MRI MRI Time Time-

Series

Series VQ VQ

Axel Wismüller, Dept. of Radiology, University of Munich

Outlook: Outlook: Brain Brain Connectivity Connectivity Analysis Analysis

Assignment Map Motor Cortex Averaged cluster- specific time-series Spontaneous activity of the resting brain without stimulus:

SLIDE 21

Axel Wismüller, Dept. of Radiology, University of Munich

Medical Medical Image Image Sequence Sequence Analysis Analysis

Functional MRI MRI Mammography Perfusion MRI

A. Wismüller et al.: Cluster analysis of biomedical image time-series.

International Journal of Computer Vision 46(2), 2002.

A. Wismüller, A. Meyer-Baese, et al.: Model-free fMRI analysis based on

unsupervised clustering. Journal of Biomedical Informatics 37(1), 2004.

Axel Wismüller, Dept. of Radiology, University of Munich

Problems Problems Related Related to to This This Workshop Workshop

Number of well-documented data sets for

validation of such CAD approaches is small

Data are, in general, not publicly available on

the web

Each clinical center tries validation of CAD

approaches in MRI mammography on its own small database

However, multi-center fusion and

documentation of such data is expensive, requires funding. Thi i h h !

SLIDE 22

Axel Wismüller, Dept. of Radiology, University of Munich

Gene expression time courses from microarray hybridization experiments (budding yeast)

Bioinformatics Bioinformatics: : Functional Functional Genomics Genomics

Axel Wismüller, Dept. of Radiology, University of Munich

Bioinformatics Bioinformatics Functional Functional Genomics Genomics

Gene expression time- series from microarray hybridization experiments: partitioning by minimal free energy VQ

C2 YPR074C TKL1 PENTOSE PHOSPHATE CYCLE TRANSKETOLASE C2 YLR134W PDC5 GLYCOLYSIS PYRUVATE DECARBOXYLASE C2 YLR044C PDC1 GLYCOLYSIS PYRUVATE DECARBOXYLASE C2 YAL038W CDC19 GLYCOLYSIS PYRUVATE KINASE C2 YOR344C TYE7 GLYCOLYSIS BASIC H-L-H TRANSCRIPTION FACTOR C2 YPL061W ALD6 ETHANOL UTILIZATION ACETALDEHYDE DEHYDROGENASE C3 YPR104C FHL1 TRANSCRIPTION TRANSCRIPTIONAL ACTIVATOR C3 YBL021C HAP3 TRANSCRIPTION COMPONENT OF HETEROTRIMERIC CCAAT-BINDING FACTOR C3 YNL216W RAP1 TRANSCRIPTION TRANSCRIPTIONAL REPRESSOR AND ACTIVATOR ... ... ... ... ... C1 YHR193C EGD2 PROTEIN SYNTHESIS (PUTAT HOMOLOG OF HUMAN NASCENT-POLYPEPTIDE-ASSOCIATED COMPLEX SUBUNIT C1 YHR021C RPS27B PROTEIN SYNTHESIS RIBOSOMAL PROTEIN S27B C1 YDL191W RPL35A PROTEIN SYNTHESIS RIBOSOMAL PROTEIN L35A C1 YDL136W RPL35B PROTEIN SYNTHESIS RIBOSOMAL PROTEIN L35B C1 YOR184W SER1 SERINE BIOSYNTHESIS PHOSPHOSERINE C2 YMR058W FET3 TRANSPORT CELL SURFACE FERROXIDASE C2 YJL167W ERG20 STEROL METABOLISM FARNESYL-PYROPHOSPHATE SYNTHETASE C2 YDR050C TPI1 GLYCOLYSIS TRIOSEPHOSPHATE ISOMERASE C2 YKL152C GPM1 GLYCOLYSIS PHOSPHOGLYCERATE MUTASE C2 YCR012W PGK1 GLYCOLYSIS PHOSPHOGLYCERATE KINASE C2 YGR192C TDH3 GLYCOLYSIS GLYCERALDEHYDE-3-PHOSPHATE DEHYDROGENASE 3 C2 YJR009C TDH2 GLYCOLYSIS GLYCERALDEHYDE-3-PHOSPHATE DEHYDROGENASE 2 C2 YHR174W ENO2 GLYCOLYSIS ENOLASE II C2 YJL052W TDH1 GLYCOLYSIS GLYCERALDEHYDE-3-PHOSPHATE DEHYDROGENASE 1 C2 YKL060C FBA1 GLYCOLYSIS ALDOLASE

SLIDE 23

Axel Wismüller, Dept. of Radiology, University of Munich

Computer Computer-

Aided

Aided Diagnosis Diagnosis

Axel Wismüller, Dept. of Radiology, University of Munich

Medizinische Bildverarbeitung Medizinische Bildverarbeitung

Path from computer science to medicine is not a one-way trip.
Computer science not only a ‘warehouse’ where ready

solutions for applied problems in medical data processing can simply be picked up.

In particular: Unsolved problems in medical applications

motivate the invention of new algorithms for pattern recognition and machine learning (such as the `Deformable Feature Map´).

SLIDE 24

Axel Wismüller, Dept. of Radiology, University of Munich

Computer Computer-

Aided

Aided Diagnosis Diagnosis

Segmentation, registration, and time-series-analysis in

multidimensional data from biomedical imaging

Aim (`Vision´): Computer-assisted Diagnosis (CAD)
Challenge: Search for applications that fulfill two criteria at the

same time:

Clinical relevance
Methodological feasibility with respect to image processing,

data analysis, and pattern recognition

Independent development of several image analysis systems

applied to relevant real-world problems in practice

Axel Wismüller, Dept. of Radiology, University of Munich

`Hard´ real-world application problems, not just

`toy examples´

Interesting perspectives
Various imaging modalities (MRI, CT, conventional

X-ray): `Structure´ and `function´

Clinical applications (fMRI, MRI mammography,

MRI perfusion imaging, nuclear medicine, neurology (Alzheimer, Multiple Sclerosis), vessel diagnosis ...

Intellectual exchange with other disciplines such as

genome research, astronomy, speech processing, psychology, etc.

Computer Computer-

Aided

Aided Diagnosis Diagnosis

SLIDE 25

Axel Wismüller, Dept. of Radiology, University of Munich

Thanks Thanks to ... to ...

Prof. Dr. Anke Meyer-Baese, Florida State University, USA
Prof. Dr. DeWitt Sumners, Florida State University, USA
Prof. Dr. Dorothee Auer, University of Nottingham, UK
Prof. Dr. Helge Ritter, University of Bielefeld
Prof. Dr. Bert Sakmann, MPI Heidelberg
Prof. Dr. Gerhard Rigoll, TU Munich
Prof. Dr. Gert Hauske, TU Munich
Prof. Dr. Klaus-Robert Müller, GMD FIRST Berlin
Prof. Dr. Klaus Hahn, University of Munich
Prof. Dr. Herbert Witte, University of Jena
Prof. Dr. Christoph v. d. Malsburg, UCLA, USA
Prof. Dr. Maximilian Reiser, University of Munich
Dr. Frank Vietze, Basler Vision Technologies, Hamburg
Dr. Dominik Dersch, Crux Cybernetics, Sydney, Australia

Dipl.-Ing. Oliver Lange, University of Munich Dipl.-Ing. Johannes Behrends, University of Munich Dipl.-Math. Johannes Kurz, TU Munich

Dr. Gerda Leinsinger, University of Munich
Dr. Dirk Heiss, Gartner Group, Munich
Cand. med. Mirjana Jukic, University of Munich
Cand. med. Claudia Krammer, University of Munich
Dr. Thomas Schlossbauer, University of Munich

Axel Wismüller, Dept. of Radiology, University of Munich