E Essential Matrix 16-385 Computer Vision (Kris Kitani) Carnegie - PowerPoint PPT Presentation

E Essential Matrix 16-385 Computer Vision (Kris Kitani) Carnegie Mellon University

Recall:Epipolar constraint p l 0 x x 0 l o 0 o e 0 e l 0 Potential matches for lie on the epipolar line x

The epipolar geometry is an important concept for stereo vision Task: Match point in left image to point in right image Left image Right image How would you do it?

The epipolar constraint is an important concept for stereo vision Task: Match point in left image to point in right image Left image Right image Epipolar constrain reduces search to a single line How do you compute the epipolar line?

Essential Matrix E The Essential Matrix is a 3 x 3 matrix that encodes epipolar geometry

Given a point in one image, multiplying by the essential matrix will tell us the epipolar line in the second view. E x = l 0 X l 0 x 0 x o 0 o e e 0

Representing the … Epipolar Line   a ax + by + c = 0 l = b in vector form   c l l If the point is on the epipolar line then x e x > l = ? x

Epipolar Line   a ax + by + c = 0 l = b in vector form   c l l If the point is on the epipolar line then x e x > l = 0 x

Recall: Dot Product c = a × b b a c · b = 0 c · a = 0 dot product of two orthogonal vectors is zero

vector representing the line is normal (orthogonal) to the plane   a l = b   c > l x o vector representing the point x is inside the plane x > l = 0 Therefore:

x > l = 0 E x = l 0 So if and then x 0> E x = ? X l 0 x 0 x o 0 o e e 0

x > l = 0 E x = l 0 So if and then x 0> E x = 0 X l 0 x 0 x o 0 o e e 0

Motivation The Essential Matrix is a 3 x 3 matrix that encodes epipolar geometry Given a point in one image, multiplying by the essential matrix will tell us the epipolar line in the second view.

Essential Matrix vs Homography What’s the difference between the essential matrix and a homography?

Essential Matrix vs Homography What’s the difference between the essential matrix and a homography? They are both 3 x 3 matrices but …

Essential Matrix vs Homography What’s the difference between the essential matrix and a homography? They are both 3 x 3 matrices but … l 0 = E x x 0 = H x Essential matrix maps a Homography maps a point to a line point to a point

Where does the Essential matrix come from?

X x 0 x o 0 o t R , t x 0 = R ( x − t )

X x 0 x o 0 o t R , t x 0 = R ( x − t ) Does this look familiar?

X x 0 x o 0 o t R , t x 0 = R ( x − t ) Camera-camera transform just like world-camera transform

X x 0 x o 0 o t These three vectors are coplanar x , t , x 0

X x 0 x o 0 o t x , t , x 0 If these three vectors are coplanar then x > ( t × x ) = ?

X x 0 x o 0 o t x , t , x 0 If these three vectors are coplanar then x > ( t × x ) = 0

Recall: Cross Product Vector (cross) product 
 takes two vectors and returns a vector perpendicular to both c = a × b b a c · b = 0 c · a = 0

X x 0 x o 0 o t x , t , x 0 If these three vectors are coplanar then ( x − t ) > ( t × x ) = ?

X x 0 x o 0 o t x , t , x 0 If these three vectors are coplanar then ( x − t ) > ( t × x ) = 0

putting it together rigid motion coplanarity x 0 = R ( x − t ) ( x − t ) > ( t × x ) = 0 ( x 0> R )( t × x ) = 0

putting it together rigid motion coplanarity x 0 = R ( x − t ) ( x − t ) > ( t × x ) = 0 ( x 0> R )( t × x ) = 0 ( x 0> R )([ t ⇥ ] x ) = 0

Cross product   a 2 b 3 − a 3 b 2 a 3 b 1 − a 1 b 3 a × b =   a 1 b 2 − a 2 b 1 Can also be written as a matrix multiplication     0 − a 3 a 2 b 1 0 a × b = [ a ] × b = a 3 − a 1 b 2     0 − a 2 a 1 b 3 Skew symmetric

putting it together rigid motion coplanarity x 0 = R ( x − t ) ( x − t ) > ( t × x ) = 0 ( x 0> R )( t × x ) = 0 ( x 0> R )([ t ⇥ ] x ) = 0 x 0> ( R [ t ⇥ ]) x = 0

putting it together rigid motion coplanarity x 0 = R ( x − t ) ( x − t ) > ( t × x ) = 0 ( x 0> R )( t × x ) = 0 ( x 0> R )([ t ⇥ ] x ) = 0 x 0> ( R [ t ⇥ ]) x = 0 x 0> E x = 0

putting it together rigid motion coplanarity x 0 = R ( x − t ) ( x − t ) > ( t × x ) = 0 ( x 0> R )( t × x ) = 0 ( x 0> R )([ t ⇥ ] x ) = 0 x 0> ( R [ t ⇥ ]) x = 0 x 0> E x = 0 Essential Matrix [Longuet-Higgins 1981]

properties of the E matrix x 0> E x = 0 Longuet-Higgins equation (points in normalized coordinates)

properties of the E matrix x 0> E x = 0 Longuet-Higgins equation x 0> l 0 = 0 x > l = 0 Epipolar lines l = E T x 0 l 0 = E x (points in normalized coordinates)

properties of the E matrix x 0> E x = 0 Longuet-Higgins equation x 0> l 0 = 0 x > l = 0 Epipolar lines l = E T x 0 l 0 = E x e 0> E = 0 E e = 0 Epipoles (points in normalized camera coordinates)

How do you generalize to uncalibrated cameras?