Homographies and Estimating Extrinsics Instructor - Simon Lucey - - PowerPoint PPT Presentation

▶
homographies and estimating extrinsics
SMART_READER_LITE
LIVE PREVIEW

Homographies and Estimating Extrinsics Instructor - Simon Lucey - - PowerPoint PPT Presentation

Feb 15, 2024 250 likes •739 views

Homographies and Estimating Extrinsics Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Review: Motivation Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince Review: Pinhole Camera Instead model

slide-1
SLIDE 1

Homographies and Estimating Extrinsics

Instructor - Simon Lucey

16-423 - Designing Computer Vision Apps

slide-2
SLIDE 2

Review: Motivation

Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

slide-3
SLIDE 3

Review: Pinhole Camera

Real camera image is inverted Instead model impossible but more convenient virtual image

Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

slide-4
SLIDE 4

Today

  • Fitting Warp Functions.
  • Estimating Intrinsics.
slide-5
SLIDE 5

Euclidean warp

  • Consider viewing a fronto-parallel plane at a fixed

distance D.

  • In homogeneous coordinates, the imaging equations are:

3D rotation matrix becomes 2D (in plane) Plane at known distance D Point is on plane (w=0)

Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

slide-6
SLIDE 6

Euclidean warp

  • Simplifying
  • Rearranging the last equation

Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

slide-7
SLIDE 7

Euclidean warp

Homogeneous: Cartesian: For short:

Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

slide-8
SLIDE 8

Euclidean warp

Homogeneous: Cartesian: For short:

Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

How many unknowns?

slide-9
SLIDE 9

Estimating the Euclidean Warp ˆ Ω, ˆ τ = arg min

Ω,τ N

X

n=1

η{xn − euc[wn, Ω, τ]}

slide-10
SLIDE 10

Estimating the Euclidean Warp

ˆ Ω, ˆ τ = arg min

Ω,τ N

X

n=1

||xn − euc[wn, Ω, τ]||2

2

slide-11
SLIDE 11

Estimating the Euclidean Warp

ˆ Ω, ˆ τ = arg min

Ω,τ N

X

n=1

||xn − euc[wn, Ω, τ]||2

2

= arg min

Ω,τ N

X

n=1

||xn − Ωwn + τ||2

2

slide-12
SLIDE 12

Estimating the Euclidean Warp

ˆ Ω, ˆ τ = arg min

Ω,τ N

X

n=1

||xn − euc[wn, Ω, τ]||2

2

= arg min

Ω,τ N

X

n=1

||xn − Ωwn + τ||2

2

s.t. ΩT Ω = I, det(Ω) = 1

slide-13
SLIDE 13

Reminder: Non-Convex Set

9

slide-14
SLIDE 14

Reminder: Non-Convex Set

9

slide-15
SLIDE 15

Reminder: Non-Convex Set

9

slide-16
SLIDE 16

Estimating the Euclidean Warp s.t. ΩT Ω = I, det(Ω) = 1

µx = 1 N

N

X

n=1

xn, µw = 1 N

N

X

n=1

wn

ˆ Ω = arg min

Ω N

X

n=1

||(xn − µx) − Ω(wn − µw)||2

2

slide-17
SLIDE 17
  • In MATLAB this becomes,

>> [U,S,V] = svd(B*A’,0); >> Omega = U*V’;

  • SVD method works for ONLY very specific objectives.
  • Extremely useful as it allows us to find closed form solutions

to a non-convex problem (very rare).

Estimating the Euclidean Warp

slide-18
SLIDE 18

Similarity warp

  • Consider viewing fronto-parallel plane at unknown

distance D

  • By same logic as before we have

Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

slide-19
SLIDE 19

Similarity warp

  • Multiply each equation by :
  • Simplifying:

Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

slide-20
SLIDE 20

Similarity warp

  • Simplifying:
  • Incorporate the constants by defining:

Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

slide-21
SLIDE 21

Similarity warp

Homogeneous: Cartesian: For short:

slide-22
SLIDE 22

Similarity warp

Homogeneous: Cartesian: For short:

How many unknowns?

slide-23
SLIDE 23

Estimating the Similarity Warp

  • Since,
  • Simplifying:
slide-24
SLIDE 24

Estimating the Similarity Warp

  • Rearranging:
  • Form system of equations:
slide-25
SLIDE 25

Estimating the Similarity Warp

  • In MATLAB this becomes,

>> p = A\x;

  • Since p stems form a convex set, simply use “backslash”!!!
slide-26
SLIDE 26

Estimating the Similarity Warp

  • In MATLAB this becomes,

>> p = A\x;

  • Since p stems form a convex set, simply use “backslash”!!!
slide-27
SLIDE 27

Estimating the Similarity Warp

  • In MATLAB this becomes,

>> p = A\x;

  • Since p stems form a convex set, simply use “backslash”!!!
slide-28
SLIDE 28

Affine warp

Affine transform describes mapping well when the depth variation within the planar object is small and the camera is far away. When variation in depth is comparable to distance to object then the affine transformation is not a good model. Here we need the homography.

Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

slide-29
SLIDE 29

Affine warp

Homogeneous: Cartesian: For short:

slide-30
SLIDE 30

Affine warp

Homogeneous: Cartesian: For short:

How many unknowns?

slide-31
SLIDE 31

Estimating the Affine Warp

  • Rearranging:
  • Form system of equations:
  • In MATLAB this becomes,

>> p = A\x;

slide-32
SLIDE 32

Homography

  • Start with basic projection equation:
  • Combining these two matrices we get:

Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

slide-33
SLIDE 33

Homography

Homogeneous: Cartesian:

For short:

slide-34
SLIDE 34

Homography

Homogeneous: Cartesian:

For short:

How many unknowns?

slide-35
SLIDE 35

Homography Estimation

  • Re-arrange cartesian equations,

           −u1 −v1 −1 y1u1 y1v1 y1 u1 v1 1 −x1u1 −x1v1 −x1 −u2 −v2 −1 y2u2 y2v2 y2 u2 v2 1 −x2u2 −x2v2 −x2 . . . . . . . . . . . . . . . . . . . . . . . . . . . −uI −vI −1 yIuI yIvI yI uI vI 1 −xIuI −xIvI −xI                          φ11 φ12 φ13 φ21 φ22 φ23 φ31 φ32 φ33               = 0,

  • Form linear system

Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

slide-36
SLIDE 36

Homography Estimation

  • In MATLAB this becomes,

>> [U,S,V] = svd(A); >> Phi = reshape(V(:,end),[3,3])’;

  • Both sides are 3x1 vectors; should be parallel, so cross

product will be zero

  • For you to try MATLAB,

>> x = [randn(2,1);1]; cross(x,4*x)

slide-37
SLIDE 37

Caution…

  • Approach only minimizes algebraic error NOT the re-

projection error!!!

  • Need to employ non-linear optimization.
slide-38
SLIDE 38

Today

  • Fitting Warp Functions.
  • Estimating Extrinsics.
slide-39
SLIDE 39

Estimating Extrinsics

slide-40
SLIDE 40

Estimating Extrinsics

  • Writing out the camera equations in full
  • Estimate the homography from matched points
  • Factor out the intrinsic parameters

Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

slide-41
SLIDE 41

Estimating Extrinsics

  • Find the last column using the cross product of first two

columns

  • Make sure the determinant is 1. If it is -1, then multiply

last column by -1.

  • Find translation scaling factor between old and new

values

  • Finally, set

Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

slide-42
SLIDE 42

Augmented Reality

slide-43
SLIDE 43

Transformations between images

  • So far we have considered

transformations between the image and a plane in the world

  • Now consider two cameras

viewing the same plane

  • There is a homography between

camera 1 and the plane and a second homography between camera 2 and the plane

  • It follows that the relation between the two images is also a

homography

slide-44
SLIDE 44

Camera under pure rotation

Special case is camera under pure rotation. Homography can be showed to be

slide-45
SLIDE 45

Camera under pure rotation

Special case is camera under pure rotation. Homography can be showed to be

Why is this?

slide-46
SLIDE 46

Panorama Example

a) b) c) d)

slide-47
SLIDE 47

More to read…

  • Prince et al.
  • Chapter 14, Sections 1-4.
  • Chapter 15, Sections 1-6.
Core Hubs:
All Converters PDF Hub Video Hub Audio Hub Image Hub File Compressor