Localized Statistical Models in Computer Vision Shawn Lankton - - PowerPoint PPT Presentation

▶
localized statistical models in computer vision
SMART_READER_LITE
LIVE PREVIEW

Localized Statistical Models in Computer Vision Shawn Lankton - - PowerPoint PPT Presentation

Aug 30, 2022 38 likes •991 views

Localized Statistical Models in Computer Vision Shawn Lankton Ph.D. Thesis Defense September 8, 2009 Professor Allen Tannenbaum - Academic Advisor Professor Anthony Yezzi - Committee Chair Professor Jeff Shamma - Committee Member

slide-1
SLIDE 1

Localized Statistical Models in Computer Vision

Shawn Lankton Ph.D. Thesis Defense September 8, 2009

Professor Allen Tannenbaum Professor Anthony Yezzi Professor Jeff Shamma Professor Ghassan Al Regib Professor Arthur Stillman Professor Marc Niethammer

  • Academic Advisor
  • Committee Chair
  • Committee Member
  • Committee Member
  • Committee Member
  • Committee Member

Tuesday, September 8, 2009

slide-2
SLIDE 2

Outline

  • Introduction and Background
  • Localized Segmentation Framework
  • Vessel Analysis and Plaque Detection
  • Conclusions

2

Tuesday, September 8, 2009

slide-3
SLIDE 3

Computer Vision

3

Chapter 1

Tuesday, September 8, 2009

slide-4
SLIDE 4

Computer Vision

4

Image Understanding Image Acquisition Image Processing

Tuesday, September 8, 2009

slide-5
SLIDE 5

Image Processing

5

Image Understanding Image Acquisition Image Processing

Detection Tracking Registration Shape Analysis Segmentation

Image Understanding Image Acquisition Image Processing

Tuesday, September 8, 2009

slide-6
SLIDE 6

Hypothesis

6

By localizing the analysis of visual information, assumptions about image-makeup and

  • bject-appearances can be relaxed...

thereby improving the accuracy of segmentation, detection, and tracking results.

Tuesday, September 8, 2009

slide-7
SLIDE 7

Hypothesis

7

localization improves results.

Tuesday, September 8, 2009

slide-8
SLIDE 8

Active Contours and Level Set Methods

8

Chapter 2

Tuesday, September 8, 2009

slide-9
SLIDE 9

Segmentation

9

Tuesday, September 8, 2009

slide-10
SLIDE 10

Segmentation

  • Thresholding
  • Region Growing
  • Graph Cuts
  • Snakes/Active Contours

10

Tuesday, September 8, 2009

slide-11
SLIDE 11

Active Contours

11

Tuesday, September 8, 2009

slide-12
SLIDE 12

Implementation

  • Represent the contour
  • Parametrized
  • Implicit

12

Tuesday, September 8, 2009

slide-13
SLIDE 13

Level Sets

13

Tuesday, September 8, 2009

slide-14
SLIDE 14

Definitions

  • 14
  • Sethian. Level Set Methods and Fast Marching Methods. 1999

A Contour Γ embedded in φ : RN → R Such that Γ = {x ∈ Ω|φ(x) = 0} An Image I : RN → R on the domain Ω

Tuesday, September 8, 2009

slide-15
SLIDE 15

Definitions

15

Hφ =    1 φ < −ǫ φ > ǫ smooth

  • therwise

inside

  • utside

δφ =    1 φ = 0 |φ| < ǫ smooth

  • therwise

the contour the rest

Tuesday, September 8, 2009

slide-16
SLIDE 16

Define an Energy

  • Observe the image
  • Make an assumption
  • Craft an energy accordingly

16

Mumford and Shah “Boundary Detection by Minimizing Functionals,” JIU, 1988

Tuesday, September 8, 2009

slide-17
SLIDE 17

Uniform Mean Modeling Energy

Assumption: The foreground and background will be approximately constant.

Chan and Vese. “Active contours without edges,” TIP, 2001

17

E(φ) =

  • Ω

Hφ(I − µin)2dx +

  • Ω

(1 − Hφ)(I − µout)2dx + + λ

  • Ω

δφ(x)∇φ(x)dx

0 if I = µin inside Γ 0 if I = µout outside Γ small if Γ is smooth

Tuesday, September 8, 2009

slide-18
SLIDE 18

Energy Minimization

φt+1 = φt + dt · dφ dt

18

dφ dt = δφ

  • (I − µin)2 − (I − µout)2 + λ div

∇φ |∇φ|

  • ∇φ
  • ∇φE(φ) = −dφ

dt

Tuesday, September 8, 2009

slide-19
SLIDE 19

Complex Images

19

Tuesday, September 8, 2009

slide-20
SLIDE 20

A Localized Active Contour Model

20

Chapter 3

Tuesday, September 8, 2009

slide-21
SLIDE 21

Localizing

x

B(x, y) B(x, y) · Hφ(y)

B(x, y) · (1 − Hφ(y))

≤ r.

21

Lankton and Tannenbaum “Localized Region-Based Active Contours,” TIP, 2008

Tuesday, September 8, 2009

slide-22
SLIDE 22

Localized Contours

22

+ λ

  • Ωx

δφ(x)∇φ(x)dx

∂φ ∂t

(x) = δφ(x)

  • Ωy

B(x, y)·∇φ(y)F(I, φ, x, y)dy+λδφ(x) div ∇φ(x) |∇φ(x)|

  • ∇φ(x)

E(φ) =

  • Ωx

δφ(x)

  • Ωy

B(x, y) · F(I, φ, x, y) dy dy dx

Tuesday, September 8, 2009

slide-23
SLIDE 23

Computing the Energy

23

E(φ) =

+

dF dF dF dF dF dF

+ + + + +

... etc.

Tuesday, September 8, 2009

slide-24
SLIDE 24

Internal Energies

  • Local Uniform Modeling
  • Local Mean Separation
  • Local Histogram Separation

24

Tuesday, September 8, 2009

slide-25
SLIDE 25

Localized Means

25

µin(x) =

  • Ωy B(x, y) · Hφ(y) · I(y)dy
  • Ωy B(x, y) · Hφ(y)dy

µout(x) =

  • Ωy B(x, y) · (1 − Hφ(y)) · I(y)dy
  • Ωy B(x, y) · (1 − Hφ(y))dy

Tuesday, September 8, 2009

slide-26
SLIDE 26

Local Uniform Mean Modeling Energy

26

Assumption: The foreground and background are approximately constant locally.

Fum = Hφ(y)(I(y) − µin(x))2 + (1 − Hφ(y))(I(y) − µout(x))2

Tuesday, September 8, 2009

slide-27
SLIDE 27

Local Uniform Mean Modeling Energy

27

(a) Initialization (b) Global UM (c) Local UM

Assumption: The foreground and background are approximately constant locally.

Tuesday, September 8, 2009

slide-28
SLIDE 28

Local Mean Separation Energy

28

Yezzi et al. “A Fully Global Approach to Image Segmentation... ,” JVCIR 2002

Assumption: The foreground and background are different locally.

Fms = −(µin(x) − µout(x))2

Tuesday, September 8, 2009

slide-29
SLIDE 29

Local Mean Separation Energy

29

(a) Initialization (b) Global MS (c) Local MS

Assumption: The foreground and background are different locally.

Tuesday, September 8, 2009

slide-30
SLIDE 30

Local Histogram Separation Energy

30

Assumption: The foreground and background have different histograms locally.

Bhattacharyya Measure

Fhs =

  • z
  • Pu,x(z)Pv,x(z)dz

Michailovich et. al. “Image segmentation using … Bhattacharyya ...” TIP 2007

Tuesday, September 8, 2009

slide-31
SLIDE 31

Local Histogram Separation Energy

31

(a) Initialization (b) Global HS (c) Local HS

Assumption: The foreground and background have different histograms locally.

Tuesday, September 8, 2009

slide-32
SLIDE 32

Back to These Guys

32

Local Mean Separation

Tuesday, September 8, 2009

slide-33
SLIDE 33

Studying Local Energies

  • Choosing the radius
  • Initializing the contour

33

Tuesday, September 8, 2009

slide-34
SLIDE 34

Choosing the Radius

34

(a) Initialization (b) r = 3 (c) r = 5 (d) r = 7 (e) r = 9 (f) r = 15 r = 7 r = 9 r = 15

Tuesday, September 8, 2009

slide-35
SLIDE 35

Choosing the Radius

35

Tuesday, September 8, 2009

slide-36
SLIDE 36

How to Initialize

36

(a) (b) (c) (d) (e) (f) (g) (h)

Tuesday, September 8, 2009

slide-37
SLIDE 37

How to Initialize

37

(a) Brain 1 (b) Brain 2

Tuesday, September 8, 2009

slide-38
SLIDE 38

Benefits of Localized Segmentation

  • Complex problems become simple
  • Natural solution to many problems
  • Scale is controllable

38

Tuesday, September 8, 2009

slide-39
SLIDE 39

Volumetric Quantification of Neural Pathways

39

Chapter 4

Tuesday, September 8, 2009

slide-40
SLIDE 40

Tractography

40

  • Diffusion Imaging
  • Brain Connectivity
  • Localize Orientation Analysis

Lankton et. al. “Localized … Fiber Bundle Segmentation.” MMBIA, 2008

Tuesday, September 8, 2009

slide-41
SLIDE 41

Cingulum Bundle Segmentation

41

Tuesday, September 8, 2009

slide-42
SLIDE 42

Soft Plaque Detection in Coronary Arteries

42

Chapter 5

Tuesday, September 8, 2009

slide-43
SLIDE 43

Vessel Analysis

  • Important
  • Challenging
  • Segmentation
  • Plaque Detection

43

Lankton et al. “Soft Plaque Detection...,” MICCAI Workshop 2009

Tuesday, September 8, 2009

slide-44
SLIDE 44

Coronary Anatomy

  • Coronaries
  • RCA
  • LAD
  • LCX

44

Tuesday, September 8, 2009

slide-45
SLIDE 45

CTA Imagery

  • In-vivo, 3-D scan
  • X-ray Attenuation
  • Contrast Agent

45

Tuesday, September 8, 2009

slide-46
SLIDE 46

Vessel Segmentation

  • Simple initialization
  • No leaks
  • Branch handling
  • No shape information

46

Tuesday, September 8, 2009

slide-47
SLIDE 47

Local Means

47

Tuesday, September 8, 2009

slide-48
SLIDE 48

Vessel Segmentation

48

Localized Uniform Mean Modeling Energy

Fum = Hφ(y)(I(y) − µin(x))2 + (1 − Hφ(y))(I(y) − µout(x))2

Domain restriction:

˜ Ω =Ω ∩ (I < −600 HU)

Tuesday, September 8, 2009

slide-49
SLIDE 49

Vessel Segmentation

49

LAD RCA

Tuesday, September 8, 2009

slide-50
SLIDE 50

Soft Plaque Detection

  • Dangerous
  • Hard to see
  • No good tools

50

Tuesday, September 8, 2009

slide-51
SLIDE 51

Soft Plaque Detection

  • Two-front approach
  • Inside moves out
  • Outside moves in

51

Tuesday, September 8, 2009

slide-52
SLIDE 52

= (µin(x) − µout(x))2 Fms = ( x) −

Detection Energy

52

Clever initializations are required Localized Means Separation Energy

Tuesday, September 8, 2009

slide-53
SLIDE 53

Creating Initializations

53

Eshrink(φ) =

  • Ωx

δφ(x)

  • Ωy

(B(x, y) · Hφ(y)) y)) dy dx + λ

  • Ωx

δφ(x)∇φ(x)dx Egrow(φ) = −Hφ(x) dx + λ

  • Ωx

δφ(x)∇φ(x)dx

Tuesday, September 8, 2009

slide-54
SLIDE 54

Steps for Detection

  • Segment the Vessel
  • Create Initialization
  • Run Local Mean Separation
  • Check for Differences

54

Tuesday, September 8, 2009

slide-55
SLIDE 55

3-D Example

55

Tuesday, September 8, 2009

slide-56
SLIDE 56

2-D Results

56

(a) Initial Surfaces (b) Result of Evolution (c) Expert Marking (d) Detected Plaque

Tuesday, September 8, 2009

slide-57
SLIDE 57

2-D Results

57

(a) Initial Surfaces (b) Result of Evolution (c) Expert Marking (d) Detected Plaque

Tuesday, September 8, 2009

slide-58
SLIDE 58

3-D Results (LCX)

58

Tuesday, September 8, 2009

slide-59
SLIDE 59

3-D Results (LAD)

59

Tuesday, September 8, 2009

slide-60
SLIDE 60

3-D Results (RCA)

60

Tuesday, September 8, 2009

slide-61
SLIDE 61

3-D Results (RCA)

61

Tuesday, September 8, 2009

slide-62
SLIDE 62

Temporal Localization for Visual Tracking

62

Chapter 6

Tuesday, September 8, 2009

slide-63
SLIDE 63

Visual Tracking

  • Temporal coherency of video
  • Isolate local temporal regions

63

Tuesday, September 8, 2009

slide-64
SLIDE 64

Temporal Localization

64

Lankton et al. “Tracking Through Changes in Scale,” ICIP 2008

Tuesday, September 8, 2009

slide-65
SLIDE 65

Conclusions

65

Chapter 7

Tuesday, September 8, 2009

slide-66
SLIDE 66

Conclusions

“By localizing the analysis of visual information, assumptions about image-makeup and

  • bject-appearances can be relaxed

thereby improving the accuracy of segmentation, detection, and tracking results.”

66

Tuesday, September 8, 2009

slide-67
SLIDE 67

Conclusions

67

localization can indeed improve results.

Tuesday, September 8, 2009

slide-68
SLIDE 68

Conclusions

  • Medical imaging applications
  • Anatomy informs radius choice
  • Localization makes sense

68

Tuesday, September 8, 2009

slide-69
SLIDE 69

Since the Proposal

69

  • Accepted to MICCAI Workshop
  • Localized contour coupling
  • Re-implementation (much faster)
  • More plaque detection examples

Tuesday, September 8, 2009

slide-70
SLIDE 70

Future Research

  • Setting localization radius
  • Testing new energies
  • Performing population studies

70

Tuesday, September 8, 2009

slide-71
SLIDE 71

Key Publications

71

  • 3. “Localizing Region-Based Active Contours.” S. Lankton and
  • A. Tannenbaum. IEEE Transactions on Image Processing. Vol.

17, No. 11, pp 2029-2039, Nov. 2008.

  • 4. “Localized Statistics for DW-MRI Fiber Bundle

Segmentation.” S. Lankton, J. Melonakos, J. Malcolm, S. Dambreville, and A. Tannenbaum. Wkshp. Mathematical Methods in Biomedical Image Analysis (CVPR). Jun. 2008.

  • 5. “Soft Plaque Detection and Automatic Vessel Segmentation.”
  • S. Lankton, A. Stillman, P. Raggi, A. Tannenbaum. Wkshp.

Probabilistic Methods in Medical Image Analysis (MICCAI).

  • Sept. 2009. (in press)
  • 6. “Tracking Through Changes in Scale.” S. Lankton, J. Malcolm,
  • A. Nakhmani, and A. Tannenbaum. IEEE International

Conference on Image Processing. pp. 241 - 244, Oct. 2008

Tuesday, September 8, 2009

slide-72
SLIDE 72

Additional Publications

72

1. “Tracking with 3DLADAR for Tactical Applications.” R. Sandhu*, S. Lankton*, S. Dambreville, S. Shaw, D. Murphy, S. Michael, A. Tannenbaum. Book Chapter. (in press) 2. “Statistical Shape Learning for 3D Tracking.” R. Sandhu, S. Lankton, S. Dambreville, A. Tannenbaum. CDC, 2009. (accepted, in press) 3. “TAC: Thresholding Active Contours.” S. Dambreville, A. Yezzi, S. Lankton, and A. Tannenbaum. ICIP, 2008. 4. “Improved Tracking by Decoupling Target and Camera Motion.”

  • S. Lankton and A. Tannenbaum. SPIE Electronic Imaging, 2008.

5. “Fast Optimal Mass Transport for Dynamic Active Contour Tracking on the GPU.” G. Pryor, T. Rehman, S. Lankton, P. Vela, and A. Tannenbaum. CDC, 2007. 6. “Hybrid Geodesic Region-Based Curve Evolutions for Image Segmentation.”

  • S. Lankton, D. Nain, A. Yezzi, A. Tannenbaum. SPIE Medical Imaging, 2007.

7. “Fusion of IVUS and OCT Through Semi-Automatic Registration.”

  • G. Unal, S. Lankton, S. Carlier, G. Slabuagh, and Y. Chen. MICCAI, 2006.

Tuesday, September 8, 2009

slide-73
SLIDE 73

Thank You.

73

Tuesday, September 8, 2009

slide-74
SLIDE 74

Supplemental Material

74

Tuesday, September 8, 2009

slide-75
SLIDE 75

Volumetric Quantification of Neural Pathways

75

Chapter 4

Tuesday, September 8, 2009

slide-76
SLIDE 76

Fiber Bundles

  • DTI
  • 3x3 Tensor Data
  • Magnitude
  • Orientation
  • Anisotropy

76

Lankton et al. “Localized Statistics for DWI Fiber Bundle Segmentation,” MMBIA, 2008

Tuesday, September 8, 2009

slide-77
SLIDE 77

Fiber Bundles

77

Tuesday, September 8, 2009

slide-78
SLIDE 78

Bundle Segmentation

  • Initialize with single fiber
  • Evolve to capture full bundle
  • Uses “tensor uniform mean model”

78

  • Pichon. Novel Methods for Multidimensional Image Segmentation, Ph.D. Thesis, 2005

Tuesday, September 8, 2009

slide-79
SLIDE 79

Log-Euclidean

79

  • Log-Euclidean Representation
  • Tensors lie on a manifold
  • Linear comparisons become possible

Arsigny et. al. “Log-Euclidean Metrics for Fast and Simple…” MRM 2006

T1 + T2 exp

  • log(

)

  • Tuesday, September 8, 2009
slide-80
SLIDE 80

Local Tensor Means

80

µin

(x) = exp

  • Ωy B(x, y) Hφ(y) log (T(y)) dy
  • Ωy B(x, y) Hφ(y)dy
  • µout

(x) = exp

  • Ωy B(x, y) (1 − Hφ(y)) log (T(y)) dy
  • Ωy B(x, y) (1 − Hφ(y))dy
  • One can also define a log-Euclidean distance,

Tuesday, September 8, 2009

slide-81
SLIDE 81

Localized Uniform Tensor Modeling

81

dLE[T1, T2] = log(T1) − log(T2)2, F = Hφ(y) dLE[T(y), ux] + (1 − Hφ(y)) dLE[T(y), vx].

Lankton et. al. “Localized … Fiber Bundle Segmentation.” MMBIA, 2008

Tuesday, September 8, 2009

slide-82
SLIDE 82

Fiber Bundles

82

Tuesday, September 8, 2009

slide-83
SLIDE 83

Temporal Localization for Visual Tracking

83

Chapter 6

Tuesday, September 8, 2009

slide-84
SLIDE 84

Tracking Schematic

84

It ˆ Mt−1 ˜ M Register Register + Warp ˆ Mt p1 p2 p { ˆ Mi}N

i=0

˜ Mk+1 = argmin

ˆ Mi

ˆ Mi − ˜ Mk2 ˜ Mk+1 Updated every frame Updated every N frames TEMPLATE MATCHING TEMPLATE UPDATE

Tuesday, September 8, 2009

slide-85
SLIDE 85

Template Matching

  • 85

An image I(x) and a template T(x) A set of warping parameters p A warping function W(x; p)

Tuesday, September 8, 2009

slide-86
SLIDE 86

Registration Procedure

86

p∗ = argmin

p

  • x [I(W(x; p)) − T(x)]2

∆p =

  • xH−1
  • ∇I· ∂W

∂p T [T(x) − I(W(x; p))] H =

  • x
  • ∇I· ∂W

∂p T ∇I· ∂W ∂p

  • Tuesday, September 8, 2009
slide-87
SLIDE 87

Tracking Schematic

87

It ˆ Mt−1 ˜ M Register Register + Warp ˆ Mt p1 p2 p { ˆ Mi}N

i=0

˜ Mk+1 = argmin

ˆ Mi

ˆ Mi − ˜ Mk2 ˜ Mk+1 Updated every frame Updated every N frames TEMPLATE MATCHING TEMPLATE UPDATE

Tuesday, September 8, 2009

slide-88
SLIDE 88

Template Update

88

e( ˆ M, ˜ M) = ˆ M − ˜ M2 ˜ Mk+1 = argmin

ˆ Mi

e({ ˆ Mi}N

i=1, ˜

Mk)

Tuesday, September 8, 2009

slide-89
SLIDE 89

Template Update

89

˜ Mk ˜ Mk+1

               e = 469 e = 517 e = 449

· · ·

e = 548               

Tuesday, September 8, 2009

slide-90
SLIDE 90

Tracking Schematic

90

It ˆ Mt−1 ˜ M Register Register + Warp ˆ Mt p1 p2 p { ˆ Mi}N

i=0

˜ Mk+1 = argmin

ˆ Mi

ˆ Mi − ˜ Mk2 ˜ Mk+1 Updated every frame Updated every N frames TEMPLATE MATCHING TEMPLATE UPDATE

Tuesday, September 8, 2009

slide-91
SLIDE 91

Temporal Localization

91

Tuesday, September 8, 2009

slide-92
SLIDE 92

Temporal Localization

92

Lankton et al. “Tracking Through Changes in Scale,” ICIP 2008

Tuesday, September 8, 2009

slide-93
SLIDE 93

Temporal Localization

93

Tuesday, September 8, 2009

slide-94
SLIDE 94

Temporal Localization

94

Tuesday, September 8, 2009

slide-95
SLIDE 95

End.

95

Tuesday, September 8, 2009