The Epipolar Geometry COMPSCI 527 Computer Vision COMPSCI 527 - - PowerPoint PPT Presentation

▶
the epipolar geometry
SMART_READER_LITE
LIVE PREVIEW

The Epipolar Geometry COMPSCI 527 Computer Vision COMPSCI 527 - - PowerPoint PPT Presentation

Nov 29, 2022 156 likes •285 views

The Epipolar Geometry COMPSCI 527 Computer Vision COMPSCI 527 Computer Vision The Epipolar Geometry 1 / 16 The Epipolar Geometry of a Pair of Cameras The Epipolar Geometry of a Pair of Cameras P projection ray y a r n o i t c

slide-1
SLIDE 1

The Epipolar Geometry

COMPSCI 527 — Computer Vision

COMPSCI 527 — Computer Vision The Epipolar Geometry 1 / 16

slide-2
SLIDE 2

The Epipolar Geometry of a Pair of Cameras

The Epipolar Geometry of a Pair of Cameras

epipolar plane p r

  • j

e c t i

  • n

r a y projection ray baseline epipolar line of p epipolar line of p epipole e epipole e center of projection center of projection P p p camera a camera b

I I

a a b a a b b b a b

COMPSCI 527 — Computer Vision The Epipolar Geometry 2 / 16

slide-3
SLIDE 3

The Epipolar Geometry of a Pair of Cameras

The Epipolar Constraint

epipolar plane projection ray projection ray baseline e p i p

  • l

a r l i n e

  • f

p epipolar line of p epipole e epipole e center of projection center of projection P p p camera a camera b

I I a a b a a b b b a b
  • The point pa in image a that corresponds to point pb in

image b is on the epipolar line of pb ... and vice versa

  • This is the only general constraint between two images of

the same scene; 3D reconstruction depends on it

  • Epipolar lines come in corresponding pairs
  • Two pencils of lines supported by the two epipoles

COMPSCI 527 — Computer Vision The Epipolar Geometry 3 / 16

I

slide-4
SLIDE 4

The Epipolar Geometry of a Pair of Cameras

Another Way to State the Epipolar Constraint

epipolar plane projection ray projection ray baseline e p i p

  • l

a r l i n e

  • f

p epipolar line of p epipole e epipole e center of projection center of projection P p p camera a camera b

I I a a b a a b b b a b

The two projection rays and the baseline are coplanar for corresponding points

COMPSCI 527 — Computer Vision The Epipolar Geometry 4 / 16

P

C

C

a

b

slide-5
SLIDE 5

The Epipolar Geometry of a Pair of Cameras

Two Uses of the Epipolar Constraint

  • Stereo vision:
  • Relative position and orientation of the two cameras are

known

  • So the epipoles can be determined
  • Given a point pa in image a, find its epipolar plane, and

therefore the epipolar line for the point pb in b

  • This reduces search for correspondence from the image

plane to the epipolar line

  • 3D camera motion and reconstruction:
  • Relative position and orientation of the two cameras are

unknown

  • Given corresponding points pa, pb (found, say, by tracking)

we can write one algebraic constraint on aP, aRb, atb

  • With enough pairs of corresponding points, we can write and

solve a system in these quantities

COMPSCI 527 — Computer Vision The Epipolar Geometry 5 / 16

slide-6
SLIDE 6

The Essential Matrix

The Essential Matrix

  • How to write the epipolar constraint algebraically?
  • The essential matrix E is a compact representation of the

relative position and orientation of two cameras

  • E is 3 × 3 and combines rotation and translation
  • The essential matrix E can be found by solving a

homogeneous linear system

  • Given E, rotation and translation can be found by additional

manipulation

COMPSCI 527 — Computer Vision The Epipolar Geometry 6 / 16

slide-7
SLIDE 7

The Essential Matrix

Coordinates

X

a

X

b

Y

a

Y

b

Z

a

Z

b

P p

a b

p

  • Image points as world points:

apa =

 

axa aya

f   and

bpb =

 

bxb byb

f  

  • Each camera measures a point in its own reference system
  • Transformation: bp = aRb(ap − atb)
  • Inverse:

ap = bRa(bp − bta)

where

bRa = aRT b

and

bta = −aRbatb

COMPSCI 527 — Computer Vision The Epipolar Geometry 7 / 16

  • O

08

Oo

  • r
slide-8
SLIDE 8

The Essential Matrix

Simplifying Notation

  • Too many super/subscripts to keep track of
  • Define: a = apa

, b = bpb , R = aRb , t = atb , e = aeb

X

a

X

b

Y

a

Y

b

Z

a

Z

b

a b e R, t

  • e and t are the same up to norm
  • ab as direction vector is RTb (all coordinates are in a now)

COMPSCI 527 — Computer Vision The Epipolar Geometry 8 / 16

0000

inHub

E_r

to

slide-9
SLIDE 9

The Essential Matrix

The Epipolar Constraint, Algebraically

X

a

X

b

Y

a

Y

b

Z

a

Z

b

a b e R, t

  • The two projection rays and the baseline are coplanar
  • The triple product of ab, t, and a is zero
  • We saw that ab = RTb
  • The triple product of RTb,

t, and a is zero: (RTb)T(t × a) = 0

COMPSCI 527 — Computer Vision The Epipolar Geometry 9 / 16

a it

IT

VIE

9Et_

F

OO O

Q

O

slide-10
SLIDE 10

The Essential Matrix

The Essential Matrix

(RTb)T(t × a) = 0 bTR (t × a) = 0 bTR [t]×a = 0 where t = (tx, ty, tz)T and [t]× =   −tz ty tz −tx −ty tx   bT E a = 0 where E = R [t]×

  • This equation is the epipolar constraint, written in algebra
  • Holds for any corresponding a, b in the two images (as

world vectors in their reference systems)

  • E is the essential matrix

COMPSCI 527 — Computer Vision The Epipolar Geometry 10 / 16

coCEO

e

slide-11
SLIDE 11

The Essential Matrix

The Epipolar Line in Image a

X

a

X

b

Y

a

Y

b

Z

a

Z

b

a b e R, t

  • Think of b as fixed
  • What vectors x in image a

satisfy the epipolar constraint? bT E x = 0

  • Let λT = bT E, a row vector
  • λTx = 0, a line!
  • a satisfies this homogeneous equation (epipolar constraint)
  • So does t:

λTt = bT Et = bT R [t]× t = 0 . . . and therefore e

  • So the line is the epipolar line of b

COMPSCI 527 — Computer Vision The Epipolar Geometry 11 / 16

II

a bTEE E