from snakes from snakes to organisms to organisms Ghassan Hamarneh - - PowerPoint PPT Presentation

▶
from snakes from snakes to organisms to organisms
SMART_READER_LITE
LIVE PREVIEW

from snakes from snakes to organisms to organisms Ghassan Hamarneh - - PowerPoint PPT Presentation

Aug 29, 2023 216 likes •637 views

Multimedia and Mathematics 2005 Deformable Models for Biomedical Image Analysis from snakes from snakes to organisms to organisms Ghassan Hamarneh Simon Fraser University Talk Overview Multimedia patient records

slide-1
SLIDE 1

Deformable Models for Biomedical Image Analysis

from ‘snakes’ from ‘snakes’ to ‘organisms to ‘organisms’

Ghassan Hamarneh Multimedia and Mathematics 2005

Simon Fraser University

slide-2
SLIDE 2

27Jul’05 B A N F F

2

Talk Overview

  • Multimedia patient records
  • Medical images
  • Medical images analysis
  • Image segmentation and registration
  • Deformable models: Snakes
  • Controlling shape deformation
  • Deformable organisms

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

slide-3
SLIDE 3

27Jul’05 B A N F F

3

Multimedia Patient Record

audio alphanumeric natural language speech images bio-signals video graphical objects

slide-4
SLIDE 4

27Jul’05 B A N F F

4

Medical Images

Need to: Store, Communicate, Visualize, Process, Analyze

Chudler, U of Washington University of Bergen - Norway Scientific Computing & Imaging, Utah BrighamRAD www.Brain-Spect.com Mayo Clinic Center for Neural Science at NYU Visible Human University College London Koizumi, Hitachi Philips Medical MR, CT, SPECT, MRA, EIT, MRE, hist., optical, PET, DTMRI, fMRI…

slide-5
SLIDE 5

27Jul’05 B A N F F

5

Medical Image Analysis

  • Medical Images:

2D/3D+Time, scalar/vector/tensor fields, non rigid tissue, patient info

  • Manual Analysis:
  • Tedious. Time consuming. Inter-, intra-operator variability
  • General goals:
  • Automation. Quantification. Classification. Data reduction. Visualization
  • General Methodologies:

Image restoration. Image enhancement. Visualization techniques, image segmentation. Image registration. Shape analysis

  • Mathematics:

Inverse problems, PDEs, transforms, optimization, statistics,…

  • Numerous Applications… Computer-aided diagnosis. Computer assisted intervention. Image guided therapy,

therapy evaluation. Surgical simulation, planning, and navigation. Image data fusion. Quantitative & time series analysis. Statistical Structural Shape Analysis Anatomical atlases. Virtual, augmented reality. Instrument, patient localization, tracking. Medical tele-presence and tele-surgery. Functional brain mapping. Screening and functional genomics.

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

http://www.uib.no/med/avd/miapr/arvid/matematisk98/index.htm

slide-6
SLIDE 6

27Jul’05 B A N F F

6

Image Segmentation

  • Partition an image into regions
  • Assign labels to pixels (binary/fuzzy)
  • Obtain higher-level representation

Voxel-Man 3D Navigator University Medical Center Utrecht

http://www-dsv.cea.fr

slide-7
SLIDE 7

27Jul’05 B A N F F

7

Pixel intensity Count

  • b

j e c t background

Feature 1 Feature 2 Feature 3

pixel

Thresholding and Clustering

classifier

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

slide-8
SLIDE 8

27Jul’05 B A N F F

8

Region-based Methods

…Merging

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

Growing

Courtesy: Tina Kapur

Splitting

slide-9
SLIDE 9

27Jul’05 B A N F F

9

2D3D

Edge Detection and Linking…

Laplacian zero-crossing, canny-edge, gradient magnitude and direction

“Livewire”

slide-10
SLIDE 10

27Jul’05 B A N F F

10

Image Registration and Atlas-based Segmentation

  • Registration: Find optimal

spatial transformation (warping)

  • f one image to maximize

“similarity” to another image

  • Segmentation via registration

new image reference image labelled reference (atlas)

register warp labels using same transform

labelled image

slide-11
SLIDE 11

27Jul’05 B A N F F

11

Registration and deformation analysis

Extension and Flexion

slide-12
SLIDE 12

27Jul’05 B A N F F

12

  • Contours, surfaces, volumes
  • Initialized in the image space
  • Deform according to image data
  • … & “shape” constraints
  • Originally: 2D semi-automatic tools
  • Integrate boundary elements, robust

to image noise, boundary gaps

  • Implemented on the continuum

achieving sub-pixel accuracy

Shape representation Initialization Model deformation

Deformable Models

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

slide-13
SLIDE 13

27Jul’05 B A N F F

13

Snake or Active Contour Models: Deformable contours, initialized in the image, deform according to internal and external constraints

v(x(s),y(s))

s=0

y

s=1

x

s=0.2

Classical “snakes”

tensile flexural external inflation i i i i i i

µ γ α β + + + = + v v F F F F

slide-14
SLIDE 14

27Jul’05 B A N F F

14

Deformable Surfaces

2 2 i i int ext

d V dV m F F dt dt δ α β + = +

slide-15
SLIDE 15

27Jul’05 B A N F F

15

Dimensionality: 2D, 2D+T,3D,3D+T, color, stereo… Shape representations: Wavelets, Splines, Fourier descriptors,... Energy/forces: inflation, distance transform, texture/appearance… Optimization: GA, SA, ANN, DP,… Topological changes: T-snakes, Level-sets

Some DM Extensions

Framework: Bayesian, wavefront propagation, geodesic computation

Sethian, Berkley McInerney, Ryerson

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

slide-16
SLIDE 16

27Jul’05 B A N F F

16

DM Problems

  • Leaking from weak edges
  • Low level parameter selection problematic
  • Lack of high level control (rely on human guidance, user interaction)
  • Modest prior shape knowledge (amorphous shapes, smoothness constraints)
  • Sensitivity to initialization

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

slide-17
SLIDE 17

27Jul’05 B A N F F

17

CC Segmentation for MS

407 images

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

slide-18
SLIDE 18

27Jul’05 B A N F F

18

Training set Labeling corresponding landmarks, aligning shapes Prior shape knowledge: Point Distribution Model Allowable Shape Domain PDM utilized in segmentation: Active Shape Models

ORL database

alignment

S S = + Pb

PCA

1 k >> 2 2 k − < <

i

k λ −

i

k λ +

Global Shape Statistics

1 1 2 2 n n

x y x y x y                      

  • hamarneh@cs.sfu.ca

http://www.cs.sfu.ca/~hamarneh

slide-19
SLIDE 19

27Jul’05 B A N F F

19

1st mode 4th mode (too much) Allowable shape domain Un-allowable shape

PDM/ASM Segmentation

http://www2.imm.dtu.dk/~aam/datasets/

slide-20
SLIDE 20

27Jul’05 B A N F F

20

“Fourier Snakes”

1 1 2 2 n n

x y x y x y                      

  • x1

y1 yn

spatial representaion DCT frequency domain

DCT coeff2 DCT coeff1 DCT coeffN

projection

  • n subspace

PC1 PC2

allowable shape domain

mean 1st PC 1st PC 2nd PC 2nd PC

mean PC2 ±1std PC1 ±1std

mean 1st PC 1st PC 2nd PC 2nd PC

mean PC2 ±1std PC1 ±1std

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

slide-21
SLIDE 21

27Jul’05 B A N F F

21

  • Controlling deformation

– User interaction – Global to local deformations – Global shape statistics – Setting low-level parameters – New cost/force terms

  • Utilize high-level knowledge to guide model-fitting
  • Difficult to encode knowledge in low-level terms
  • Require intuitive, controlled shape deformation handles
  • Artificial-Life framework that complements

– bottom-up data-driven functionality of deformable models with… – top-down knowledge-driven model-fitting strategies

Deformable Organisms Deformable Organisms

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

slide-22
SLIDE 22

27Jul’05 B A N F F

22

Deformable Organisms

Perception perceptual attention mechanism sensors Skeleton underlying medial-based shape representation muscles and limbs muscle actuation

causing shape deformation

cognitive center

Geometry Physics Behaviours Cognition

Plan or schedule Interaction with other organisms Memory and prior knowledge

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

slide-23
SLIDE 23

27Jul’05 B A N F F

23

Length, Orientation, Left and right thickness

Medial-Based Shape Profiles

Geometry

Controlling Shape Deformation

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

slide-24
SLIDE 24

27Jul’05 B A N F F

24

d d dls dls dlst dlst l s t

p p M w k α     = + +        

∑∑ ∑

Type: bending stretching thickening location scale variation mode

  • perator type

amplitude

Medial-Based Shape Profiles

Physics

HR-PCA

Controlling Shape Deformation

slide-25
SLIDE 25

27Jul’05 B A N F F

25

Physics-Based Shape Deformations

  • Mass-spring model
  • User interaction
  • Intuitive deformations
  • Feasible shapes

Physics

Controlling Shape Deformation

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

slide-26
SLIDE 26

27Jul’05 B A N F F

26

Spring actuation External forces

( )( )(

)

( )

2 1 1 1 1

  • ld

ij ij

d r K r R θ π = − − − +

R d C θ

ij

s

i

n

j

n D

  • R

d C θ

ij

s

i

n

j

n D

  • ,

, , , , , , , def loc scl def loc scl def loc scl def loc scl

M = + r r w

Statistics-based deformation

Physics

HR-PCA Deformations:

Translation, rotation, scaling, Bending, bulging, tapering,…

Controlling Shape Deformation

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

slide-27
SLIDE 27

27Jul’05 B A N F F

27

Controlling Shape Deformation

slide-28
SLIDE 28

27Jul’05 B A N F F

28

Controlling Shape Deformation, 3D

  • In-sheet and out-of-sheet bending values
  • Upper and lower thickness values
  • Elongation values

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

slide-29
SLIDE 29

27Jul’05 B A N F F

29

3D Deformations

Apply operators Intuitive, controlled shape deformation

slide-30
SLIDE 30

27Jul’05 B A N F F

30

Bicipital Groove

slide-31
SLIDE 31

27Jul’05 B A N F F

31

Behaviours

Procedural Plan and Behavioral Routines

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

slide-32
SLIDE 32

27Jul’05 B A N F F

32

To carry out:

active, explicit searches for object features by activating behavior subroutines.

Combines:

sensory information, prior knowledge, instructions from a pre-stored segmentation plan, interaction with other

  • rganisms

Geometry Physics behaviour

Cognition

“brain” Perception perceptual attention mechanism Memory and prior knowledge Plan or schedule Interactions with

  • ther organisms

muscle actuation

causing shape deformation

Cognitive Centre

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

slide-33
SLIDE 33

27Jul’05 B A N F F

33

Behavioral routines find-top-of-head find-upper-boundary latch-to-boundary find-genu, find-rostrum, find-splenium

Deformable Organism

genu rostrum splenium fornix body

Corpus callosum

slide-34
SLIDE 34

27Jul’05 B A N F F

34

  • Global model alignment:

– Deform: Translational forces – Sense: norm mean & variance – Decide: max mean, min variance

  • Model parts’ alignment:

– For each: Splenium, Genu, Rostrum – Deform: rotational & translational forces – Sense: norm mean & variance – Decide: max mean, min variance

  • Model parts’

contraction/expansion:

– For each: Splenium, Genu, Rostrum

– Deform: contraction/expansion – Sense: norm regional mean, variance, area – Decide: max mean, min variance, max area

  • Medial-axis alignment

– Deform: Stretch along thickness springs :

  • Fitting to Boundary

– Deform: Forces at boundary nodes along thickness spring – Sense: edge strength (+ mean, var) :

  • Detect/repair fornix dip:

– Detect based on parallelism to medial/upper-boundary, gradient, thickness – Repair by interpolate, …

Physics-Based Deformable Organisms

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

slide-35
SLIDE 35

27Jul’05 B A N F F

35

Statistical Shape Analysis

Thickness and orientation profiles Shape histogram

Student Version of MATLAB Student Version of MATLAB Student Version of MATLAB Student Version of MATLAB Student Version of MATLAB

1st, 2nd, 3rd, 4th main modes of global bending explaining 41.07, 14.15, 11.96, 6.68% of the total variation Main modes of localized and deformation-specific shape variation

slide-36
SLIDE 36

27Jul’05 B A N F F

36

Deformable Organisms:

Lateral Ventricles, Caudate Nuclei, and Putamina

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

slide-37
SLIDE 37

27Jul’05 B A N F F

37

Overlap Bifurcation

5 10 15 20 25 30 35 40 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 5 10 15 20 25 30 35 40 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

2D Vessel Crawler

slide-38
SLIDE 38

27Jul’05 B A N F F

38

3D Vessel Crawler

Segmentation & Analysis

Radius Medial axis

slide-39
SLIDE 39

27Jul’05 B A N F F

39

Summary

  • Multimedia patient record
  • Medical images & analysis
  • Deformable models for segmentation
  • Synthesizing and analyzing deformable shapes
  • Deformable organisms, A-Life approach to MIA

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh

  • Challenges…

– Robust, highly-automated MIA – Large data… store, communicate, visualize, process, analyze, access, link to patient records… – Small displays and mobile users

slide-40
SLIDE 40

27Jul’05 B A N F F

40

Acknowledgements

  • Students:

– Yvonne Li, Michael Bodnyk, Kevin Stevenson, Aaron Ward, Chris McIntosh, Vincent Chu, Johnson Yang, Morgan Langille, Tong Liu

  • Collaborators:

– NYU (Dr. Schweitzer), Chalmers (Dr. Chodorowski), Kinesiology (Dr. Finegood), UBC MS/MRI (Dr. Traboulsee).

  • Funding:

– NSERC, CFI, SFU

hamarneh@cs.sfu.ca http://www.cs.sfu.ca/~hamarneh