CS325 Artificial Intelligence Computer Vision III Structure from - - PowerPoint PPT Presentation

▶
cs325 artificial intelligence computer vision iii
SMART_READER_LITE
LIVE PREVIEW

CS325 Artificial Intelligence Computer Vision III Structure from - - PowerPoint PPT Presentation

Nov 02, 2022 43 likes •465 views

CS325 Artificial Intelligence Computer Vision III Structure from Motion (Ch. 24) Dr. Cengiz Gnay, Emory Univ. Spring 2013 Gnay Computer Vision III Structure from Motion (Ch. 24) Spring 2013 1 / 13 Structure from Motion What??

slide-1
SLIDE 1

CS325 Artificial Intelligence Computer Vision III – Structure from Motion (Ch. 24)

  • Dr. Cengiz Günay, Emory Univ.

Spring 2013

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 1 / 13

slide-2
SLIDE 2

Structure from Motion

What??

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 2 / 13

slide-3
SLIDE 3

Structure from Motion

What?? Structure: 3D information Motion: Camera motion

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 2 / 13

slide-4
SLIDE 4

Structure from Motion

What?? Structure: 3D information Motion: Camera motion

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 2 / 13

slide-5
SLIDE 5

Structure from Motion

What?? Structure: 3D information Motion: Camera motion Looks familiar?

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 2 / 13

slide-6
SLIDE 6

Entry/Exit Surveys

Exit survey: Computer Vision II – 3D Vision

Why don’t we need to know the original object’s size when we have stereo vision? What’s the operating principle of the XBOX Kinect (R) motion tracker system?

Entry survey: Computer Vision III – Structure from Motion (0.25 pts)

Can you think of a way to apply the 3D vision alignment algorithms from last class for extracting structure from motion (SfM)? What would be a good application area for SfM?

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 3 / 13

slide-7
SLIDE 7

Triangulate from Camera Positions

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 4 / 13

slide-8
SLIDE 8

Triangulate from Camera Positions

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 4 / 13

slide-9
SLIDE 9

Triangulate from Camera Positions

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 4 / 13

slide-10
SLIDE 10

Triangulate from Camera Positions

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 4 / 13

slide-11
SLIDE 11

Triangulate from Camera Positions

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 4 / 13

slide-12
SLIDE 12

Triangulate from Camera Positions

Can we find locations of A, B, C?

1 Always 2 Sometimes 3 Never Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 4 / 13

slide-13
SLIDE 13

Triangulate from Camera Positions

Can we find locations of A, B, C?

1 Always 2 Sometimes 3 Never Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 4 / 13

slide-14
SLIDE 14

Example with Two Cameras

2 3 1

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 5 / 13

slide-15
SLIDE 15

Example with Two Cameras

1 2 3 2 3 1

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 5 / 13

slide-16
SLIDE 16

Example with Two Cameras

1 2 3 2 1 3 2 3 1

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 5 / 13

slide-17
SLIDE 17

Demos:

slide-18
SLIDE 18

SfM Examples: 3D Reconstruction From Snapshots

Lots of examples on the Wikipedia page: A Fountain Duomo of Pisa An alley Dots and texture

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 7 / 13

slide-19
SLIDE 19

SfM is Also Called “Camera Tracking”

Nowadays, it is even available in open-source programs: Blender 3D modeling software: see video of its camera tracking plugin More on the Wikipedia page

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 8 / 13

slide-20
SLIDE 20

So How Does SfM Work?

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 9 / 13

slide-21
SLIDE 21

So How Does SfM Work?

Here’s the math:

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 9 / 13

slide-22
SLIDE 22

So How Does SfM Work?

Here’s the math:

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 9 / 13

slide-23
SLIDE 23

So How Does SfM Work?

Here’s the math:

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 9 / 13

slide-24
SLIDE 24

SfM Math

Non-linear least-squares optimization problem: Gradient descent Conjugate gradient Gauss Newton methods (e.g., Levenberg-Marquardt) Singular Value Decomposition (e.g., PCA)

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 10 / 13

slide-25
SLIDE 25

How Many of Those Parameters Can We Recover?

Let’s assume we have m camera poses n points to recover

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 11 / 13

slide-26
SLIDE 26

How Many of Those Parameters Can We Recover?

Let’s assume we have m camera poses n points to recover Each camera pose has 6 parameters: 3 for x, y, z 3 for pointing angle α, β, φ Each point has 3 parameters: 3 for x, y, z

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 11 / 13

slide-27
SLIDE 27

How Many of Those Parameters Can We Recover?

Let’s assume we have m camera poses n points to recover Each camera pose has 6 parameters: 3 for x, y, z 3 for pointing angle α, β, φ Each point has 3 parameters: 3 for x, y, z Total: Unknown parameters: 6m + 3n. Constraints from 2D images: 2nm

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 11 / 13

slide-28
SLIDE 28

How Many of Those Parameters Can We Recover?

Let’s assume we have m camera poses n points to recover Each camera pose has 6 parameters: 3 for x, y, z 3 for pointing angle α, β, φ Each point has 3 parameters: 3 for x, y, z Total: Unknown parameters: 6m + 3n. Constraints from 2D images: 2nm To recover all points, must satisfy:

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 11 / 13

slide-29
SLIDE 29

How Many of Those Parameters Can We Recover?

Let’s assume we have m camera poses n points to recover Each camera pose has 6 parameters: 3 for x, y, z 3 for pointing angle α, β, φ Each point has 3 parameters: 3 for x, y, z Total: Unknown parameters: 6m + 3n. Constraints from 2D images: 2nm To recover all points, must satisfy: 6m + 3n ≤ 2nm

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 11 / 13

slide-30
SLIDE 30

How Many of Those Parameters Can We Recover?

Let’s assume we have m camera poses n points to recover Each camera pose has 6 parameters: 3 for x, y, z 3 for pointing angle α, β, φ Each point has 3 parameters: 3 for x, y, z Total: Unknown parameters: 6m + 3n. Constraints from 2D images: 2nm To recover all points, must satisfy: 6m + 3n ≤ 2nm Which parameters can’t we recover at all?

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 11 / 13

slide-31
SLIDE 31

How Many of Those Parameters Can We Recover?

Let’s assume we have m camera poses n points to recover Each camera pose has 6 parameters: 3 for x, y, z 3 for pointing angle α, β, φ Each point has 3 parameters: 3 for x, y, z Total: Unknown parameters: 6m + 3n. Constraints from 2D images: 2nm To recover all points, must satisfy: 6m + 3n ≤ 2nm Which parameters can’t we recover at all? Absolute frame of reference (x, y, z)

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 11 / 13

slide-32
SLIDE 32

How Many of Those Parameters Can We Recover?

Let’s assume we have m camera poses n points to recover Each camera pose has 6 parameters: 3 for x, y, z 3 for pointing angle α, β, φ Each point has 3 parameters: 3 for x, y, z Total: Unknown parameters: 6m + 3n. Constraints from 2D images: 2nm To recover all points, must satisfy: 6m + 3n ≤ 2nm Which parameters can’t we recover at all? Absolute frame of reference (x, y, z) Absolute orientation angle (α, β, φ)

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 11 / 13

slide-33
SLIDE 33

How Many of Those Parameters Can We Recover?

Let’s assume we have m camera poses n points to recover Each camera pose has 6 parameters: 3 for x, y, z 3 for pointing angle α, β, φ Each point has 3 parameters: 3 for x, y, z Total: Unknown parameters: 6m + 3n. Constraints from 2D images: 2nm To recover all points, must satisfy: 6m + 3n ≤ 2nm Which parameters can’t we recover at all? Absolute frame of reference (x, y, z) Absolute orientation angle (α, β, φ) Scale

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 11 / 13

slide-34
SLIDE 34

How About Motion of Subjects?

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 12 / 13

slide-35
SLIDE 35

How About Motion of Subjects?

Subjects assumed to be static.

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 12 / 13

slide-36
SLIDE 36

How About Motion of Subjects?

Subjects assumed to be static. Can we also recover structure of moving subjects? (Ask your neighbor)

1 Always 2 Sometimes 3 Never Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 12 / 13

slide-37
SLIDE 37

How About Motion of Subjects?

Subjects assumed to be static. Can we also recover structure of moving subjects? (Ask your neighbor)

1 Always 2 Sometimes 3 Never

Remember?

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 12 / 13

slide-38
SLIDE 38

How About Motion of Subjects?

Subjects assumed to be static. Can we also recover structure of moving subjects? (Ask your neighbor)

1 Always 2 Sometimes 3 Never

Remember? Need to have models of objects:

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 12 / 13

slide-39
SLIDE 39

How Does the Brain Do It?

SfM converts From camera images: To object locations:

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 13 / 13

slide-40
SLIDE 40

How Does the Brain Do It?

SfM converts From camera images: Egocentric or viewer-centered representation To object locations:

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 13 / 13

slide-41
SLIDE 41

How Does the Brain Do It?

SfM converts From camera images: Egocentric or viewer-centered representation To object locations: Allocentric or object-centered representation

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 13 / 13

slide-42
SLIDE 42

How Does the Brain Do It?

SfM converts From camera images: Egocentric or viewer-centered representation To object locations: Allocentric or object-centered representation The brain has two separate visual pathways for these: Ventral is allocentric and dorsal is egocentic. Read more here.

Günay Computer Vision III – Structure from Motion (Ch. 24) Spring 2013 13 / 13