Camera Models Shao-Yi Chien Department of Electrical Engineering - - PowerPoint PPT Presentation

Camera Models

簡韶逸 Shao-Yi Chien Department of Electrical Engineering National Taiwan University Fall 2019

1

Outline

• Camera models

2

[Slides credit: Marc Pollefeys]

Camera Obscura: the Pre-Camera

• First idea: Mo-Ti, China (470BC to 390BC)
• First built: Alhazen, Iraq/Egypt (965 to 1039AD)

Illustration of Camera Obscura Freestanding camera obscura at UNC Chapel Hill

Photo by Seth Ilys

Camera Obscura, Reinerus Gemma-Frisius, 1544

Camera Obscura

T T

Z fY Z fX Z Y X ) / , / ( ) , , ( 

                                                 1 1 1 Z Y X f f Z fY fX Z Y X 

Pinhole Camera Model

Pinhole Camera Model

                                             1 1 1 1 1 Z Y X f f Z fY fX

PX x 

 

| I ) 1 , , ( diag P f f 

T y x T

p Z fY p Z fX Z Y X ) / , / ( ) , , (   

principal point

T y x p

p ) , (

                                                   1 1 1 Z Y X p f p f Z Zp fY Zp fX Z Y X

y x x x



Principal Point Offset

Principal Point Offset

 

cam

X | I K x 

           1

y x

p f p f K

Calibration Matrix

 

C ~

• X

~ R X ~

cam 

X 1 RC R 1 1 C ~ R R Xcam                               Z Y X

Camera Rotation and Translation

 

cam

X | I K x 

 X

C ~ | I KR x  

 

t | R K P  C ~ R t  

PX x 

Camera Rotation and Translation

Internal Camera Parameter Internal Orientation Intrinsic Matrix External Camera Parameter Exterior Orientation Extrinsic Matrix

           1

y x x x

p p K                        1 1

y x x x

p f p f m m K

CCD Camera

Non-square pixels 

𝑛𝑧 𝛽𝑧

           1

y x x x

p p s K              1

y x x x

p p K  

 

C ~ | I KR P  

non-singular 11 dof (5+3+3) decompose P in K,R,C?

 

4

p | M P 

4 1p

M C ~



 

   

M R K, RQ 

{finite cameras}={P4x3 | det M≠0} If rank P=3, but rank M<3, then cam at infinity

Finite Projective Camera

s: skew parameter, =0 for most normal cameras P4: Last column of P

Camera Anatomy

• Camera center
• Column points
• Principal plane
• Axis plane
• Principal point
• Principal ray

PC 

Camera Center (C): Null-space of camera projection matrix

λ)C (1 λA X    λ)PC (1 λPA PX x    

For all A all points on AC project on image of A, therefore C is camera center Image of camera center is (0,0,0)T, i.e. undefined Finite cameras:

        



1 p M

4 1

C

Infinite cameras:

Md , d           C

Camera Center

Proof: The camera center is a point at infinity

   

             1 p p p p p

4 3 2 1 2

p1, p2, p3: vanishing points of the world coordinate X, Y, and Z axes Image points corresponding to X,Y,Z directions and origin

Column Vectors

Row Vectors

                                   1 p p p

3 2 1

Z Y X y x

T T T

                                   1 p p p

3 2 1

Z Y X w y

T T T

note: p1,p2 dependent on image reparametrization

Row Vectors

principal point

 

, , , p ˆ

33 32 31 3

p p p 

3 3

Mm p ˆ P x   

The Principal Point

𝑛3

3

m

 

cam cam cam

X | I K X P x  

   

T

1 , , m M det v

3 



cam cam

P P k 

v v

4

k 

  



4

p | M C ~ | I KR P    k ) R det( 

vector defining front side of camera (direction unaffected) Direction unaffected because

The Principal Axis Vector

The principal axis vector v=det(M)m3 is directed towards the front of the camera

PX x 

 

Md D p | M PD x

4

  

Forward projection Back-projection

x P X





 

1

PP P P

   T T

I PP 



(pseudo-inverse)

PC 

 

λC x P λ X  



   

                           1 p

• μx

M 1 p M

• x

M μ λ X

4

• 1

4

• 1
• 1

x M d

• 1

 C D

Action of Projective Camera on Point

For finite camera

 

 

C ~ X ~ m C X P X P

T 3 T 3 T 3

     w

(dot product) (PC=0)

1 m ; det

3 

 M

If , then m3 unit vector in positive direction

 

3

m ) sign(detM P X; depth T w   

T

X X,Y,Z,T 

Depth of Points

Finding the camera center

PC 

(use SVD to find null-space)

 

 

4 3 2

p , p , p det  X

   

4 3 1

p , p , p det   Y

   

4 2 1

p , p , p det  Z

   

3 2 1

p , p , p det   T

Finding the camera orientation and internal parameters

KR M 

(use RQ decomposition ~QR) Q R

=( )-1= -1 -1

Q R (if only QR, invert)

Camera Matrix Decomposition

           1

y x x x

p p s K  

1 g arctan(1/s)

for CCD/CMOS, always s=0 Image from image, s≠0 possible (non coinciding principal axis)

HP

resulting camera:

When is Skew Non-zero?

   

homography 4 4 1 1 1 homography 3 3 P              general projective interpretation Meaningfull decomposition in K,R,t requires Euclidean image and space Camera center is still valid in projective space Principal plane requires affine image and space Principal ray requires affine image and Euclidean space

Euclidean vs. Projective

d P       

Camera center at infinity

M det  

Affine and non-affine cameras Definition: affine camera has P3T=(0,0,0,1)

Cameras at Infinity

Affine Cameras

 

                C ~ r r C ~ r r C ~ r r K C ~ | I KR P

3T 3T 2T 2T 1T 1T

C ~ r3T   d

     

                          

t t

d t t t

3T 2T 2T 1T 1T 3 3T 3T 3 2T 2T 3 1T 1T

r C ~ r r C ~ r r K r

• C

~ r r r

• C

~ r r r

• C

~ r r K P

modifying p34 corresponds to moving along principal ray

Affine Cameras

                                   

3T 2T 2T 1T 1T 3T 2T 2T 1T 1T

/ r C ~ r r C ~ r r K r C ~ r r C ~ r r 1 / / K P d d d d d d d d d d

t t t t t t

now adjust zoom to compensate              

   2T 2T 1T 1T

C ~ r r C ~ r r K P lim P d

t t

Affine Cameras

          1 βr αr X

2 1

            1 r βr αr X

3 2 1

X P X P X P

t 

 

point on plane parallel with principal plane and through origin, then general points              Δ ~ ~ K X P x

proj

d y x            

 affine

~ ~ K X P x d y x

proj

x

affine

x x

Error in Employing Affine Cameras

 

proj proj affine

x

• x

x

• x

d  

Approximation should only cause small error

• 1.  much smaller than d0
• 2. Points close to principal point

(i.e. small field of view)

Error in Employing Affine Cameras

            

 2x2

t ~ R ~ 1 x ~ K P d              1 t ~ R ~ 1 x ~ K

2x2

• 1

d

absorb d0 in K2x2                                     1 R ~ 1 x ~ t ~ K K 1 x ~ K t ~ R ~ 1 K 1 x ~ t ~ K R ~ K

2x2 2x2

• 1

2x2 2x2 2x2 2x2

                         



1 R ~ 1 x ~ K 1 t ~ R ~ 1 K P

2x2 2x2

alternatives, because 8dof (3+3+2), not more

Decomposition of P∞

Prefer this one

          



1 1 1 P canonical representation       



1 K K

2 2

calibration matrix principal point is not defined

Summary Parallel Projection

Orthographic projection Scaled orthographic projection           



1 1 1 P        1 t R H           



1 r r P

2 1T 1 1T

t t           



k t t / 1 r r P

2 1T 1 1T

(5dof) (6dof)

A Hierarchy of Affine Cameras

Weak perspective projection

                  



k t t

y x

/ 1 r r 1 α α P

2 1T 1 1T

(7dof)

A Hierarchy of Affine Cameras

1. Affine camera=camera with principal plane coinciding with P∞ 2. Affine camera maps parallel lines to parallel lines 3. No center of projection, but direction of projection PAD=0 (point on P∞)

Affine camera

                   k t t s

y x A

/ 1 r r 1 α α P

2 1T 1 1T

(8dof)

         1 P

2 23 22 21 1 13 12 11

t m m m t m m m

A

   

affine 4 4 1 1 1 affine 3 3 P           

A

A Hierarchy of Affine Cameras

T

) ( X X,Y,X,T 

T

) , , ( PX w y x 

T

) / , ( w y x

Straight lines are not mapped to straight lines! (otherwise it would be a projective camera)

(11dof)

Pushbroom Cameras

                     Z Y X p p p p p p y x

23 22 21 13 12 11

(5dof)

Null-space PC=0 yields camera center Also decomposition

 

c ~ | I R K P

2 2 2 2 2 2 3 2

 

   

Line Cameras