The Eight-Point Algorithm COMPSCI 527 Computer Vision COMPSCI 527 - - PowerPoint PPT Presentation

The Eight-Point Algorithm

COMPSCI 527 — Computer Vision

COMPSCI 527 — Computer Vision The Eight-Point Algorithm 1 / 17

Outline

1 Summary: The Epipolar Constraint 2 The Eight-Point Algorithm: t, R 3 Triangulation: Pm 4 Bundle Adjustment

COMPSCI 527 — Computer Vision The Eight-Point Algorithm 2 / 17

Summary: The Epipolar Constraint

Summary: The Epipolar Constraint

• l

• f

The two projection rays and the baseline are coplanar for corresponding points

• a = apa

, b = bpb , R = aRb , t = atb , e = aeb

• Epipolar constraint: bT E a = 0

where E = R [t]×

• Linear in E but not in R and t
• The epipole e belongs to every epipolar line

λTx = 0 with λT = bTE

COMPSCI 527 — Computer Vision The Eight-Point Algorithm 3 / 17

1 How tocompute E

2 How to break

up E into R E

T

O

O

O O

Summary: The Epipolar Constraint

The Structure of E

• E has rank 2 and null(E) = span(t) = span(e)
• Geometry:
• The epipole e is in all epipolar lines
• Therefore, λTe = 0 for all b in λT = bT E
• Therefore Ee = 0, so e 2 null(E)
• Algebra:

[t]×t = t ⇥ t = 0 t ⇥ v 6= 0 if v is not parallel to t

• Therefore, the rank of [t]× is 2 for nonzero t
• Since R is full rank, the solutions of [t]×x = 0 and Ex = 0

are the same

• Therefore, rank(E) = 2 for nonzero t
• null(E) = span(t) = span(e)

COMPSCI 527 — Computer Vision The Eight-Point Algorithm 4 / 17

TEe

00

if

00

E REE

Summary: The Epipolar Constraint

The Structure of E

• E has two equal singular values and one zero singular value
• Proof
• Let v be perpendicular to t. Then k[t]×vk = ktk kvk

(geometric definition of cross product)

• Let kvk = 1. Then k[t]×vk = ktk
• v ? t means that v 2 row space([t]×)
• Therefore, unit-norm vectors v 2 row space([t]×) are

mapped to a circle

• Therefore [t]× has two equal singular values
• Third is zero because t 2 null([t]×)
• Ditto for E, since E = R[t]× and R is orthogonal
• Therefore v3 ⇠ e ⇠ t
• If we have E, we can find t by SVD

COMPSCI 527 — Computer Vision The Eight-Point Algorithm 5 / 17

Ii iii it i

HIEIr E II

HEH

T

ft

UI VT

g

E R

Y

T

Summary: The Epipolar Constraint

A Fundamental Ambiguity

• The equation bT E a = 0 is homogeneous in E
• Therefore, we cannot tell the magnitude of E,
• r of t in E = R [t]×
• Absolute scale cannot be determined from images alone
• This ambiguity is general, has nothing to do with the

specifics of the formulation

• Cameras fundamentally measure angles, not distances
• This ambiguity is often exploited in movie special effects
• W.l.o.g., let ktk = 1
• Measure everything in units of inter-camera distance

COMPSCI 527 — Computer Vision The Eight-Point Algorithm 6 / 17

k

Summary: The Epipolar Constraint

Switching Cameras

• Everything holds when a and b are switched
• bT E a = 0
• r

aT ET b = 0

• Switch null space and row space of E with left null space

and column space

• Therefore, u3 ⇠ bea ⇠ bta
• Also, bEa = aET

b aEb ⇠ UΣV T =

⇥ u1 u2 u3 ⇤ diag(1, 1, 0) ⇥ v1 v2 v3 ⇤T

COMPSCI 527 — Computer Vision The Eight-Point Algorithm 7 / 17

be Eb

O

O

O

Summary: The Epipolar Constraint

Next Problem: How to Find E?

bT E a = 0

• Given pairs (a1, b1), . . . (an, bn) (tracking)
• Write one epipolar constraint equation per pair
• Linear in E

COMPSCI 527 — Computer Vision The Eight-Point Algorithm 8 / 17

The Eight-Point Algorithm: t, R

The Eight-Point Algorithm

• H. C. Longuet-Higgins, Nature, 293:133–135, 1981
• Needs at least 8 corresponding point pairs
• Preferably many more
• Overview:
• Solve bT

1 E a1 = 0, . . ., bT n E an = 0 for E

• Solve E = R [t]× for t, R
• Compute the 3D structure (points Pm) from am, bm, t, R
• The last step is called triangulation

COMPSCI 527 — Computer Vision The Eight-Point Algorithm 9 / 17

H

ein

• ps

BASELINE

The Eight-Point Algorithm: t, R

Rewriting the Epipolar Constraint

bT E a = 0 ⇥ b1 b2 b3 ⇤ 2 4 e11 e12 e13 e21 e22 e23 e31 e32 e33 3 5 2 4 a1 a2 a3 3 5 = 0 a1b1e11 + a1b2e21 + a1b3e31+ a2b1e12 + a2b2e22 + a2b3e32+ a3b1e13 + a3b2e23 + a3b3e33 = 0

⇥ a1b1 a1b2 a1b3 a2b1 a2b2 a2b3 a3b1 a3b2 a3b3 ⇤ 2 6 6 6 6 6 6 6 6 6 6 6 4 e11 e21 e31 e12 e22 e32 e13 e23 e33 3 7 7 7 7 7 7 7 7 7 7 7 5 = 0

cTη = 0

• With n point pairs, cT

mη = 0 for m = 1, . . . , n

COMPSCI 527 — Computer Vision The Eight-Point Algorithm 10 / 17

IEEE

f

y

up

• uter gb flatten

8

I

r

ee

l

O

The Eight-Point Algorithm: t, R

Solving for E

cT

mη = 0 for m = 1, . . . , n

Cη = 0 where C is n ⇥ 9

• Because of the scale ambiguity, we cannot tell the norm of η
• Set η = 1
• Homogeneous, least squares problem on the unit sphere
• We know how to solve that!

COMPSCI 527 — Computer Vision The Eight-Point Algorithm 11 / 17

Ill

• f e O

I

i

vases

The Eight-Point Algorithm: t, R

Solving for t

E = R [t]×

• We have E now
• We saw that null(E) = span(t)
• So we know how to find t with ktk = 1, up to a sign
• ±t

(and also ±[t]×)

COMPSCI 527 — Computer Vision The Eight-Point Algorithm 12 / 17

E

VE I E VI

E

I Vefi 3

O

ee I

I

The Eight-Point Algorithm: t, R

Solving for R

E = R [t]×

• We have both E and T = [t]×
• Linear system in R, but with the constraints RTR = RRT = I

and det(R) = 1

• The Procrustes problem, arg minRT R=RRT =I kE RTkF
• Appendix in the notes gives a solution based on the SVD
• Since T has rank 2, it turns out that the there are two

solutions, R1 and R2

COMPSCI 527 — Computer Vision The Eight-Point Algorithm 13 / 17

E

RI

xT RT ET

TR

ET

Tri

E

O

NON INVERTIBLE

00

00

The Eight-Point Algorithm: t, R

The Fourfold Ambiguity

(t, R1) , (−t, R2) , (t, R2) , (−t, R1)

• Only one solution places all world points in front of both cameras
• Try all four solutions, and reconstruct world points by triangulation
• Pick the one solution that makes sense

COMPSCI 527 — Computer Vision The Eight-Point Algorithm 14 / 17

15 ER

he

Triangulation: Pm

Triangulation

• For simplicity, divide a0 =

2 4 a0

1

a0

2

f 3 5 by f so that now a = 2 4 a1 a2 1 3 5

• Let α

def

=  a1 a2

• (coordinates in canonical image reference system)
• Ditto for b, β
• Projection equations in each camera reference frame: A is P in frame a

α =

1 A3

 A1 A2

• and β =

1 B3

 B1 B2

• Rewrite as αA3 =

 A1 A2

• and βB3 =

 B1 B2

• Plug B = R(A − t) into second set of equations
• All equations are linear. Four equations, 3 unknowns
• Solve in the LSE sense, get a modicum of noise rejection

COMPSCI 527 — Computer Vision The Eight-Point Algorithm 15 / 17

O

FEET're

Q

O

G A

I

4 3 34 4 1

ALEE.BEEllGAFH

Bundle Adjustment

Summary of Eight-Point Algorithm

• Given n ≥ 8 image point pairs (am, bm) for m = 1, . . . , n
• Solve n × 9 linear homogeneous system bT

m E am = 0 for E

• Compute ±t as the third right singular vector of ±E
• Solve ±E = R ± [t]× for R by Procrustes (linear problem

with orthogonality constraint) to obtain R1, R2

• Triangulate scene points Pm from am, bm, t, R and for all

four combinations of t and R (n separate problems, one per point pair)

• Choose the one combination of t, R that places world points

in front of both cameras

• Keep the corresponding triangulated Pm
• Everything is found up to a single, global scale factor

COMPSCI 527 — Computer Vision The Eight-Point Algorithm 16 / 17

Bundle Adjustment

Bundle Adjustment

• Let π be the perspective projection function. We are after

arg min

t,R,A1,...,An

1 n

n

X

m=1

⇥ (am − π(Am))2 + (bm − π(R(Am − t)))2⇤ | {z } reprojection error arg min

t,R,A1,...,An ρ(t, R, A1, . . . , An)

• Eight-point algorithm solves this single optimization problem

in multiple steps

• This greedy approach leads to a suboptimal solution
• Use solution t, R, P1, . . . , Pn to initialize a gradient-descent

search for an optimal solution to the full problem

• This fine-tuning step is called bundle adjustment

COMPSCI 527 — Computer Vision The Eight-Point Algorithm 17 / 17

HI