Lecture 10: Point Clouds, Eigenvectors, PCA COMPSCI/MATH 290-04 - - PowerPoint PPT Presentation

▶
lecture 10 point clouds eigenvectors pca
SMART_READER_LITE
LIVE PREVIEW

Lecture 10: Point Clouds, Eigenvectors, PCA COMPSCI/MATH 290-04 - - PowerPoint PPT Presentation

Jan 13, 2024 20 likes •661 views

Lecture 10: Point Clouds, Eigenvectors, PCA COMPSCI/MATH 290-04 Chris Tralie, Duke University 2/16/2016 COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA Announcements Group Assignment 1 Part 1, Due This Sunday 11:55 PM

slide-1
SLIDE 1

Lecture 10: Point Clouds, Eigenvectors, PCA

COMPSCI/MATH 290-04

Chris Tralie, Duke University

2/16/2016

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-2
SLIDE 2

Announcements

◮ Group Assignment 1 Part 1, Due This Sunday 11:55 PM ◮ Note on Assignment Difficulty ◮ Hackathon 2/26, 2/27, or 2/28 (upcoming poll) ◮ Choosing final projects before spring break ◮ Visitors Coming...

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-3
SLIDE 3

Table of Contents

◮ Point Clouds Unit Overview ⊲ Eigenvalues / Eigenvectors ⊲ Principal Component Analysis

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-4
SLIDE 4

What Is A Point Cloud?

Interactive Demo

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-5
SLIDE 5

What’s In This Unit?

Point Cloud Registration (Alignment) Courtesy of http://www.pointclouds.org

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-6
SLIDE 6

What’s In This Unit?

“Shape Google” (Point Cloud Statistics) (my undergrad senior thesis)

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-7
SLIDE 7

What’s In This Unit?

Symmetries

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-8
SLIDE 8

What’s In This Unit?

Symmetries

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-9
SLIDE 9

What’s In This Unit?

Symmetries

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-10
SLIDE 10

What’s In This Unit?

Surface Reconstruction

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-11
SLIDE 11

What’s In This Unit?

Surface Reconstruction

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-12
SLIDE 12

Table of Contents

⊲ Point Clouds Unit Overview ◮ Eigenvalues / Eigenvectors ⊲ Principal Component Analysis

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-13
SLIDE 13

Eigenvalues/Eigenvectors

Ax = λx A is N × N matrix x is N × 1 column matrix (vector) λ is a scalar called the eigenvalue of x

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-14
SLIDE 14

Eigenvalues/Eigenvectors

Ax = λx A is N × N matrix x is N × 1 column matrix (vector) λ is a scalar called the eigenvalue of x ⊲ A does not change the direction of x

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-15
SLIDE 15

Eigenvalues/Eigenvectors Example: X Scale

2 1 1

  • =

Before

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-16
SLIDE 16

Eigenvalues/Eigenvectors Example: X Scale

2 1 1

  • =

1

  • Yes! λ = 1

Before After

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-17
SLIDE 17

Eigenvalues/Eigenvectors Example: X Scale

2 1 1

  • =

Before

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-18
SLIDE 18

Eigenvalues/Eigenvectors Example: X Scale

2 1 1

  • =

2

  • Yes! λ = 2

Before After

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-19
SLIDE 19

Eigenvalues/Eigenvectors Example: X Scale

2 1 1 1

  • =

Before

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-20
SLIDE 20

Eigenvalues/Eigenvectors Example: X Scale

2 1 1 1

  • =

2 1

  • No, changes direction

Before After

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-21
SLIDE 21

Eigenvalues/Eigenvectors Example: Shear

1 1 1 1

  • =

Before

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-22
SLIDE 22

Eigenvalues/Eigenvectors Example: Shear

1 1 1 1

  • =

1

  • Yes! λ = 1

Before After

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-23
SLIDE 23

Eigenvalues/Eigenvectors Example: Shear

1 1 1 1

  • =

Before

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-24
SLIDE 24

Eigenvalues/Eigenvectors Example: Shear

1 1 1 1

  • =

1 1

  • No, changes direction

Before After

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-25
SLIDE 25

Eigenvalues/Eigenvectors Example: Flip X, Scale Y

−1 2 1

  • =

Before

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-26
SLIDE 26

Eigenvalues/Eigenvectors Example: Flip X, Scale Y

−1 2 1

  • =

−1

  • Yes! λ = −1 (tricky)

Before After

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-27
SLIDE 27

Eigenvalues/Eigenvectors Example: Flip X, Scale Y

−1 2 1

  • =

Before

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-28
SLIDE 28

Eigenvalues/Eigenvectors Example: Flip X, Scale Y

−1 2 1

  • =

2

  • Yes! λ = 2

Before After

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-29
SLIDE 29

Eigenvalues/Eigenvectors Example: Flip X, Scale Y

−1 2 1 1

  • =

Before

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-30
SLIDE 30

Eigenvalues/Eigenvectors Example: Flip X, Scale Y

−1 2 1 1

  • =

−1 2

  • No, changes direction

Before After

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-31
SLIDE 31

Rotation Matrix

What are the eigenvectors/eigenvalues of A = cos(θ) − sin(θ) sin(θ) cos(θ)

  • ?

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-32
SLIDE 32

Linear Eigen-Decomposition

Eigenvectors/eigenvalues (λ1, v1), (λ2, v2) Let v = av1 + bv2 Then Av = A(av1 + bv2) = aAv1 + bAv2 Av = λ1av1 + λ2bv2

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-33
SLIDE 33

Linear Eigen-Decomposition: Example

  • 4

−1 −1 4

  • v1 =

−1 −1

  • , λ1 = 3

v2 = −1 1

  • , λ2 = 5

Let v = 2

  • COMPSCI/MATH 290-04

Lecture 10: Point Clouds, Eigenvectors, PCA

slide-34
SLIDE 34

Spectral Theorem

If A is symmetric that is A = AT then all of its eigenvectors are orthogonal

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-35
SLIDE 35

Spectral Theorem: Example

1 2 2 1

  • v1 =

−1 1

  • , λ1 = −1

v2 = 1 1

  • , λ2 = 3

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-36
SLIDE 36

Solving for Eigenvalues/Eigenvectors

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-37
SLIDE 37

Table of Contents

⊲ Point Clouds Unit Overview ⊲ Eigenvalues / Eigenvectors ◮ Principal Component Analysis

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-38
SLIDE 38

Point Cloud Centroid

Given a collection of points X = { xi}|N

i=1

Centroid c is the “mean point” That is

  • c[k] = 1

N

N

  • i=1

xi[k] Average each coordinate independently

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-39
SLIDE 39

Point Cloud Centroid

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-40
SLIDE 40

Point Cloud Centroid

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-41
SLIDE 41

Directions of Variance

Organize point cloud into N × d matrix, each point along a row X =        −

  • x1

− −

  • x2

− −

  • x3

− . . . . . . . . . −

  • xN

−        Choose a unit column vector direction u ∈ Rd×1 Then d = Xu gives projections onto u

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-42
SLIDE 42

Directions of Variance

d = Xu gives projections onto u dTd = (Xu)T(Xu) = uT(X TX)u Gives the sum of squared projections onto u

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-43
SLIDE 43

Directions of Variance: Example

1000 point example in 2D, centroid is origin

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-44
SLIDE 44

Directions of Variance: Example

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-45
SLIDE 45

Directions of Variance: Example

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-46
SLIDE 46

Directions of Variance: Example

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-47
SLIDE 47

Directions of Variance: Example

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-48
SLIDE 48

Directions of Variance: Example

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-49
SLIDE 49

Directions of Variance: Some Math

dTd = (Xu)T(Xu) = uT(X TX)u = uTAu

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-50
SLIDE 50

Directions of Variance: Some Math

dTd = (Xu)T(Xu) = uT(X TX)u = uTAu ⊲ Note that A = X TX is symmetric.

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-51
SLIDE 51

Directions of Variance: Some Math

dTd = (Xu)T(Xu) = uT(X TX)u = uTAu ⊲ Note that A = X TX is symmetric. ⊲ It is also positive semi-definite. All eigenvalues are real and positive

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-52
SLIDE 52

Directions of Variance: Some Math

dTd = (Xu)T(Xu) = uT(X TX)u = uTAu ⊲ Note that A = X TX is symmetric. ⊲ It is also positive semi-definite. All eigenvalues are real and positive ⊲ Let (v1, λ1), (v2, λ2), ..., (vd, λd) be the d eigenvalue/eigenvector pairs. Then a direction u can be written as u = a1v1 + a2v2 + . . . + advd

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-53
SLIDE 53

Directions of Variance: Some Math

uT(X TX)u = uTAu Let u = a1v1 + a2v2 + . . . + advd and all vs are unit-norm eigenvectors

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-54
SLIDE 54

Directions of Variance: Some Math

uT(X TX)u = uTAu Let u = a1v1 + a2v2 + . . . + advd and all vs are unit-norm eigenvectors Then uTAu = (a1v1 + a2v2 + . . . + advd)TA(a1v1 + a2v2 + . . . + advd)

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-55
SLIDE 55

Directions of Variance: Some Math

uT(X TX)u = uTAu Let u = a1v1 + a2v2 + . . . + advd and all vs are unit-norm eigenvectors Then uTAu = (a1v1 + a2v2 + . . . + advd)TA(a1v1 + a2v2 + . . . + advd) uTAu = (a1v1+a2v2+. . .+advd)T(a1λ1v1+a2λ2v2+. . .+adλdvd)

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-56
SLIDE 56

Directions of Variance: Some Math

uT(X TX)u = uTAu Let u = a1v1 + a2v2 + . . . + advd and all vs are unit-norm eigenvectors Then uTAu = (a1v1 + a2v2 + . . . + advd)TA(a1v1 + a2v2 + . . . + advd) uTAu = (a1v1+a2v2+. . .+advd)T(a1λ1v1+a2λ2v2+. . .+adλdvd) Remembering that all vs are orthogonal

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-57
SLIDE 57

Directions of Variance: Some Math

uT(X TX)u = uTAu Let u = a1v1 + a2v2 + . . . + advd and all vs are unit-norm eigenvectors Then uTAu = (a1v1 + a2v2 + . . . + advd)TA(a1v1 + a2v2 + . . . + advd) uTAu = (a1v1+a2v2+. . .+advd)T(a1λ1v1+a2λ2v2+. . .+adλdvd) Remembering that all vs are orthogonal uTAu = a2

1λ1 + a2 2λ2 + . . . + a2 dλd

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-58
SLIDE 58

Directions of Variance: Some Math

uTAu = a2

1λ1 + a2 2λ2 + . . . + a2 dλd

Assuming that ||u|| = 1 = ⇒ (a2

1 + a2 2 + . . . a2 d) = 1

Also assume that λ1 > λ2 > . . . > λd Raffle Point Question: What should the as be to maximize the sum of squared dot products: uTAu ?

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-59
SLIDE 59

Principal Component Analysis

  • 1. Stack points in rows.
  • 2. Compute the “covariance matrix” A = X TX
  • 3. Compute eigenvalues/eigenvectors of A, sorted in

decreasing order Orthogonal directions of variance are in the eigenvectors Sum of squared dot products with directions are associated eigenvalues

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-60
SLIDE 60

PCA Example

Largest direction of variance, λ1 = 422

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-61
SLIDE 61

PCA Example

Smallest direction of variance, λ2 = 21.6

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-62
SLIDE 62

PCA: Interactive Examples 2D/3D

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-63
SLIDE 63

PCA Pitfalls

Outliers! (Show Demo Again)

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

slide-64
SLIDE 64

PCA Pitfalls

Close Eigenvectors “Multiplicity issues” for “near-isotropic” point clouds

COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA