7-9 April 2020 / / / Amsterdam www.geospatialworldforum.org CLICK - - PowerPoint PPT Presentation

www.geospatialworldforum.org

#GWF2020

7-9 April 2020 / / / Amsterdam

CLICK TO KNOW MORE

Prof Is mat E lhas s an is mat@ks u.edu.s a is mat@ks u.edu.s a K S U, R iyadh, S audi Arabia

1

• Introduction
• Problem Definition
• Importance of Space Resection
• Importance of Space Resection
• Photogrammetry versus Computer Vision
• Mathematical Indirect Solutions
• Mathematical Indirect Solutions
• Direct Solutions
• Comparison
• Comparison
• Conclusions

2

Photogrammetry is defined as a measurement technique Photogrammetry is defined as a measurement technique where the coordinates of the points in 3D of an object are calculated after measurements made on 2D photographic images taken by metric camera. photographic images taken by metric camera. The position and attitude of the camera (camera exterior The position and attitude of the camera (camera exterior

• rientation elements) during the exposure is an important

factor in determining the required ground coordinates. The process of determining the position and attitude of the camera is called Space Resection (SR). the camera is called Space Resection (SR).

3

4

The diagram below shows the SR problem The diagram below shows the SR problem

Importance: The problem is important for both photogrammetry The problem is important for both photogrammetry and computer vision disciplines. Some of the photogrammetric applications of the space resection are: space resection are:

• Fixing ground coordinates by intersection from

single photo after solving the space resection problem. problem.

• Photo Triangulation, using multiple photos (Bundle

Adjustment)

• Camera calibration (Tsai, 1987)
• Head-Mounted Tracking System Positioning (Azuma

and Ward, 1991) and Ward, 1991)

• Object Recognition
• Ortho photo rectification

6

Importance of S R in C

• mputer Vision

Importance of S R in C

• mputer Vision

SR applications in computer vision include:: SR applications in computer vision include::

• Head-Mounted Tracking System Positioning

(Azuma and Ward, 1991) (Azuma and Ward, 1991)

• Robot picking and Robot navigation
• Robot picking and Robot navigation

(Linnainmaa et al. 1988)

• Visual surveying in 3D input devices
• Head pose computation
• Head pose computation

7

S pace R esection: S pace R esection: P hotogrammetry-C

• mputer Vision

Space resection is therefore dealt with in both Space resection is therefore dealt with in both photogrammetry & computer vision. Here is a comparison between the two treatments. Here is a comparison between the two treatments. Difference is shown in terminology and in processing: Terms: Terms: Photogrammetry V Computer Vision

Camera exterior orientation parameters Camera pose estimation

8

P hotogrammetry V C

• mputer Vision

P hotogrammetry V C

• mputer Vision

T erms, coordinates

Space resection Perspective n-point Space resection problem Perspective n-point problem 3D Cartesian Homogenous Coordinates 3D Cartesian Spatial coordinates Of camera persp center Camera pose angles Coordinates Camera line elements Camera attitiude

9

Camera pose angles Camera attitiude

P hotogrammetry V C

• mputer Vision

S

• lution C
• ncepts

S

• lution C
• ncepts

Non-linear problem Non-linear problem Iterative Solution

Linear problem Direct Solution Initial Approximations No need for initial Initial Approximations needed No need for initial approximations Collinearity Condition Collinearity Condition Closed Form

10

Approximate S

• lutions

Approximate S

• lutions

Here are some approximate solutions of the SR problem: problem:

1

• The Direct Linear Transformation (DLT), which is a

method frequently used in photogrammetry and remote method frequently used in photogrammetry and remote sensing. 2- The Church method proposed as a solution for single image resection (Slama, 1 980). image resection (Slama, 1 980). 3- A simplified absolute orientation method based on

• bject-distances and vertical lines used when no control
• bject-distances and vertical lines used when no control

points are available. This method is largely applied in archaeology and architecture by non-photogrammetrists due to its simplicity. due to its simplicity.

11

Approximate S

• lutions

Approximate S

• lutions

Continued: Continued:

4- A method of 3D conformal coordinate transformations (Dewitt, 1 996) where a special transformations (Dewitt, 1 996) where a special formulation of the rotation matrix as a function of the azimuth and tilt is proposed. the azimuth and tilt is proposed. 5- An approximate solution of the spatial transformation (Kraus, 1 997) which is particularly transformation (Kraus, 1 997) which is particularly suitable when incomplete control points are used.

12

Church’ solution (1 945) First iterative solution was published by Church, First iterative solution was published by Church, 1945, 1948. This is done by linearizing collinearity equations, (Wolf, 1980; Salama, 1980) This is done by linearizing collinearity equations, (Wolf, 1980; Salama, 1980) It needs a good starting value which constitutes an approximate solution. approximate solution. Approximate initial values which can be known to 10% accuracy for scale and distances and to within 1 5o for rotation angles would then be 10% accuracy for scale and distances and to within 1 5o for rotation angles would then be adjusted by the solution. See Seedahmed, 2008 for autonomous initial See Seedahmed, 2008 for autonomous initial values for exterior orientation parameters (EOP)

13

Collinearity Condition

Z y L f xa

Collinearity Condition

The exposure station

O x a Tilted photo plane f xa ya

The exposure station

• f a photograph (L), an object

point (A) and its photo

A ZL Y

point (A) and its photo image (a) all lie along a straight line (L, a, A).

A Za Xa Ya XL Ya XL YL X

14

Image & Ground Coordinate Systems Image & Ground Coordinate Systems

• Ground Coordinate System - X, Y, Z

z’ y’

• Ground Coordinate System - X, Y, Z

In Ground Coordinate System

• Exposure Station Coordinates L( XL, YL, ZL)

Z x’ y’ L za’ ya’ Xa’

• Object Point (A) Coordinates A( Xa, Ya, Za)
• Image coordinate system (x’, y’, z’ ) parallel to
• ground coordinate system (XYZ)

ZL X

• ground coordinate system (XYZ)

In image Coordinate System image point (a) coordinates a(xa’, ya’, za’)

Y A

Za

Xa ZL

xa’ , ya’ and za’ are related to the measured photo coordinates xa, ya, focal length (f) and the three rotation angles omega, phi and kappa.

Ya YL XL

15

X YL

Developed in a sequence of three independent two-dimensional rotations.

• rotation about x’ axis

x1 = x’

1

y1 = y’Cos + z’Sin z1 = -y’Sin + z’Cos

• f rotation about y’ axis
• f rotation about y’ axis

x2 = -z1Sinf + x1Cosf y2 = y1 z2 = z1Cosf + x1Sinf z2 = z1Cosf + x1Sinf

• rotation about z’ axis

x = x2Cos + y2Sin y = -x2Sin + y2Cos y = -x2Sin + y2Cos z = z2

16

x = x’(Cosf Cos ) + y’(Sin Sinf Cos + Cos Sin ) + z’(-Cos Sinf Cos + Sin Sin ) y = x’(-Cosf Sin ) + y’(-Sin Sinf Sin + Cos Cos ) + z’(Cos Sinf Sin + Sin Cos ) z = x’(Sinf ) + y’(-Sin Cosf ) + z’(Cos Cosf )

X = MX’ Rotation Matrix x = m11x’ + m12y’ + m13z’ y = m21x’ + m22y’ + m23z’ z = m x’ + m y’ + m z’

'

13 12 11

x m m m x

The sum of the squares of the three “direction cosines” in any row or in any z = m31x’ + m32y’ + m33z’

' '

33 32 31 23 22 21

z y m m m m m m z y

The sum of the squares of the three “direction cosines” in any row or in any column is unity. M -1 = MT X’ = MTX

17

Collinearity Condition Equations Collinearity Condition Equations from Similar Triangles

Collinearity condition equations developed from similar triangles (Wolf)

A L a L A a L A a

Z Z z Y Y y X X x ' ' '

x = m11x’ + m12y’ + m13z’ y = m21x’ + m22y’ + m23z’

* Dividing xa and ya by za

A L L A L A

Z Z Y Y X X

y = m21x’ + m22y’ + m23z’ z = m31x’ + m32y’ + m33z’

a L A L A a L A L A a L A L A a

z Z Z Z Z m z Z Z Y Y m z Z Z X X m x ' ' '

13 12 11

* Dividing xa and ya by za * Substitute –f for za * Correcting the offset of Principal point (xo, yo)

a L A a L A a L A a a L A L A a L A L A a L A L A a

z Z Z m z Y Y m z X X m z z Z Z Z Z m z Z Z Y Y m z Z Z X X m y ' ' ' ' ' '

33 32 31 23 22 21

point (xo, yo)

a L A a L A a L A a

z Z Z m z Z Z m z Z Z m z ' ' '

33 32 31

) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) (

23 22 21 33 32 31 13 12 11 L A L A L A L A L A L A L A L A L A

• a

Z Z m Y Y m X X m Z Z m Y Y m X X m Z Z m Y Y m X X m f x x

18

) ( ) ( ) ( ) ( ) ( ) (

33 32 31 23 22 21 L A L A L A L A L A L A

• a

Z Z m Y Y m X X m Z Z m Y Y m X X m f y y

C

• llinearity C
• ndition equations

from V ector C

• llinearity C
• ndition equations

from V ector T riangle

I prefer using this approach when teaching to allow students to understand the principle: XA = XO + xa XA = 3D object space coordinates XA = 3D object space coordinates XO = exposure station space coordinates = scale = scale xa=image coordinates

• Nonlinear
• Nine unknowns

) ( ) ( ) ( ) ( ) ( ) (

33 32 31 13 12 11 L A L A L A L A L A L A

• a

Z Z m Y Y m X X m Z Z m Y Y m X X m f x x

• , f ,
• XA, YA and ZA
• X , Y and Z

) ( ) ( ) ( ) ( ) ( ) (

33 32 31 23 22 21 L A L A L A L A L A L A

• a

Z Z m Y Y m X X m Z Z m Y Y m X X m f y y

• XL, YL and ZL

Taylor’s Theorem is used to linearize the nonlinear equations equations

substituting

) (

) ( ! ) (

n n n

a x n a f

) ( ) ( ) ( ) ( ) ( ) (

13 12 11 33 32 31 L A L A L A L A L A L A

Z Z m Y Y m X X m r Z Z m Y Y m X X m q

20

!

n

n

) ( ) ( ) ( ) ( ) ( ) (

23 22 21 13 12 11 L A L A L A L A L A L A

Z Z m Y Y m X X m s Z Z m Y Y m X X m r

Rewriting the Collinearity Equations Rewriting the Collinearity Equations

a

• x

q r f x F

L L L L L L

• dZ

Z F dY Y F dX X F d F d F d F F

a

• a
• y

q s f y G x q f x F

L L L L L L

• a

A A A A A A

dZ Z G dY Y G dX X G d G d G d G G x dZ Z F dY Y F dX X F Z Y X

Taylor’s Theorem

• F0 and G0 are functions F and G evaluated at the initial

approximations for the nine unknowns q

a A A A A A A

y dZ Z G dY Y G dX X G

approximations for the nine unknowns

• d , df , d are the unknown corrections to be applied to the

initial approximations

• The rest of the terms are the partial derivatives of F and G wrt
• The rest of the terms are the partial derivatives of F and G wrt

to their respective unknowns at the initial approximations

21

• Residual terms must be included in order to make the

equations consistent

V J dZ b dY b dX b dZ b dY b dX b d b d b d b

J = xa – Fo ; K = ya – Go

b terms are coefficients equal to the partial derivatives

ya A A A L L L xa A A A L L L

V K dZ b dY b dX b dZ b dY b dX b d b d b d b V J dZ b dY b dX b dZ b dY b dX b d b d b d b

26 25 24 26 25 24 23 22 21 16 15 14 16 15 14 13 12 11

b terms are coefficients equal to the partial derivatives Numerical values for these coefficient terms are obtained by using initial approximations for the unknowns. The terms must be solved iteratively (computed corrections are added to the initial approximations to obtain revised approximations) until the magnitudes of corrections to initial approximations become negligible.

22

negligible.

Formulate the collinearity equations for a number of control points whose X, Y and Z ground coordinates are known and whose images appear in the tilted photo. The equations are then solved for the six unknown elements of exterior orientation which appear in them. exterior orientation which appear in them. Space Resection collinearity equations for a point A

xa L L L

V J dZ b dY b dX b d b d b d b

16 15 14 13 12 11

A two dimensional conformal coordinate transformation is used X = ax’ – by’ + T

ya L L L

V K dZ b dY b dX b d b d b d b

26 25 24 23 22 21

X = ax’ – by’ + Tx

X, Y – ground control coordinates for the point

Y = ay’ + bx’ + Ty

x’, y’ – ground coordinates from a vertical photograph a, b, Tx, Ty – transformation parameters 23

This method does not require fiducial marks and can be solved without supplying initial approximations for the parameters Collinearity equations along with the correction for lens distortion dx, dy – lens distortion fx – pd in the x direction fy – pd in the y direction

) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) (

23 22 21 33 32 31 13 12 11 L A L A L A y y a L A L A L A L A L A L A x x a

Z Z m Y Y m X X m Z Z m Y Y m X X m f x y Z Z m Y Y m X X m Z Z m Y Y m X X m f x x

fy – pd in the y direction Rearranging the above two equations

) ( ) ( ) (

33 32 31 L A L A L A y y a

Z Z m Y Y m X X m f x y

L Z L Y L X L

L m f m x L L m f m x L L m f m x L

x x x

/ ) ( / ) ( / ) (

13 33 3 12 32 2 11 31 1

1 1

8 7 6 5 11 10 9 4 3 2 1

Z L Y L X L L Z L Y L X L y Z L Y L X L L Z L Y L X L x

y a x a

L m f m y L L m f m y L L m f m y L L Z m Y m X m f x L L m f m x L

y y L L L x x

/ ) ( / ) ( / ) ( / ) ( / ) (

22 32 6 21 31 5 13 12 11 4 13 33 3

24

1

11 10 9

Z L Y L X L

y a

L m f m y L

y

/ ) (

23 33 7

The resulting equations are solved iteratively using LSM

/ / / ) (

32 10 31 9 23 22 21 8 L L L y

L m L L m L L Z m Y m X m f y L

4 1 3 2 1

L L L L X L

) ( / /

33 32 31 33 11 32 10 L L L

Z m Y m X m L L m L L m L

1

8 4 11 10 9 7 6 5 3 2 1

L L L L L L L L L L L Z Y X

L L L

Advantages

• No initial approximations are required

for the unknowns. for the unknowns. Limitations

• Requirement of atleast six 3D object space control points
• Lower accuracy of the solution as compared with a rigorous bundle
• Lower accuracy of the solution as compared with a rigorous bundle

adjustment

25

Direct S

• lutions

Direct S

• lutions

Since the indirect solution requires initial approximate values for the exterior orientation parameters, and in computer vision problems the parameters, and in computer vision problems the approximate values are not known, a direct solution is a must. solution is a must. Six approaches of direct solution were presented and tested by Haralick, et al, 1 994. and tested by Haralick, et al, 1 994.

26

Direct R etrieval of E OP using 2D P rojective Direct R etrieval of E OP using 2D P rojective T ransformation

Seedahmed (2006) presented a direct algorithm to retrieve the EOPs from the 2D projective transformation, based on a direct relationship between transformation, based on a direct relationship between the 2D projective transformation and the collinearity model using a normalization process. This leads to a model using a normalization process. This leads to a direct matrix correspondence between the 2D projection parameters and the collinearity model parameters: hence, a direct matrix factorization to parameters: hence, a direct matrix factorization to retrieve the EOPs.

27

(no linearization, iterations or initial values) (no linearization, iterations or initial values)

Space resection in photogrammetry using Space resection in photogrammetry using collinearity condition without linearisation Said Easa (201 0) presented an optimization model for space resection with or without redundancy that space resection with or without redundancy that requires no linearization, iterations, or initial approximate values. approximate values. The model, which is nonlinear and noncovex, is solved using advanced Excel-based optimization software that has been recently developed. that has been recently developed. The proposed model is simple and converges to the global optimal solution very quickly. global optimal solution very quickly.

28

R

• drigues

Matrix S

• lution

(no initial values needed) (no initial values needed)

The classical iterative method based on Euler angle using The classical iterative method based on Euler angle using collinearity equations usually fail because the initial values are poor or unknown. are poor or unknown. For this reason, H. Zeng, (201 0) used Rodrigues matrix to represent the rotation matrix, and then established the mathematical model of space resection based on Rodrigues mathematical model of space resection based on Rodrigues matrix, finally, he presented the solution of the model. This algorithm need linearization and iterative process, but has algorithm need linearization and iterative process, but has no initial values problem, regardless of the size of the Euler angle, and is fast and effective

29

Newton-R aphson S earch S

• lution

Newton-R aphson S earch S

• lution

(no initial values)

Said Easa (2007) described the geometry of the 3point resection and the difficulties with N-R method. He presented a new Excel-based method that He presented a new Excel-based method that identifies the four solutions of the quartic polynomial. The method does not need initial estimates of the The method does not need initial estimates of the roots.

30

E xplicit S

• lution with LS

A to R edundant C

• ntrol

E xplicit S

• lution with LS

A to R edundant C

• ntrol

Smith 2006 suggested discrimination of the four Smith 2006 suggested discrimination of the four resulting solutions by using redundant control and applying least squares adjustment (LSA) for the general case where photography is not nearly vertical. general case where photography is not nearly vertical. All four solutions will be examined in the light of the All four solutions will be examined in the light of the available redundant control. The disadvantage of the method is the need for redundant control. redundant control.

Non-R igid S pace R esection by P arameterized Models by P arameterized Models

Wang et al proposed a model based approach to space

• resection. The method recovers a camera’

s location and

• resection. The method recovers a camera’

s location and

• rientation relative to an object coordinate system up to a

scale factor without the use of any GCP or vanishing point. The mathematical basis for this approach is the equivalence between the vector normal to the interpretation equivalence between the vector normal to the interpretation plane in the image space and the vector normal to the rotated interpretation plane in the object space. A two-step iterative scheme for recovering camera

• rientation that, unlike existing methods, does not require
• rientation that, unlike existing methods, does not require

a good initial guess for the rotation.

32

Non-R igid Approach, continued Non-R igid Approach, continued

Instead, the good initial estimate for the rotation is Instead, the good initial estimate for the rotation is computed directly by using coplanarity constraints. A non- linear least squares minimization procedure is then applied to determine camera orientation accurately. to determine camera orientation accurately. The camera translation and predefined model parameters The camera translation and predefined model parameters are determined based on the calculated rotation through a linear least squares minimization. Unlike existing methods, this method does not require a model-to-image fitting process, and is more effective and model-to-image fitting process, and is more effective and faster than previous approaches.

33

Non-R igid S

• lution

) , , (

R

Non-R igid S

• lution

Object line 6-7 & image line 6-7

z

Camera Coordinate System

y x

C Image edge 67{(x1, y1, -f), (x2, y2, - f)} Image plane Z

1 1 2 2

f)} t (X0, Y0,Z0 ) 5 6 Y Model edge 67(v, u) Object Coordinate System 2 1 3 4 8 5 6 7 X O

34

Object Coordinate System

S traight lines intersection approach

Lagunes & Battle (2009) proposed a technique based

• n determining the coordinates of the exposure

station by intersection of the straight lines through the station by intersection of the straight lines through the three known ground control points. The required azimuths of these lines are obtained The required azimuths of these lines are obtained from the geometric relationships between two similar triangles. Numerical solutions that show the good performance and accuracy of the solution were reported.

R ecursive S traight Lines S

• lution

R ecursive S traight Lines S

• lution

Tomaselli and Tozzi (1 996) developed a space resection Tomaselli and Tozzi (1 996) developed a space resection solution using an explicit math model relating straight lines as features, applying Kalman Filtering. An iterative process using sequential estimated camera An iterative process using sequential estimated camera location parameters to feed back to the feature extraction leading to a gradual reduction of image extraction leading to a gradual reduction of image space fro feature searching. Results show highly accurate space resection Results show highly accurate space resection parameters are obtained as well as a progressive processing time reduction.

36

Genetic E volution S

• lution

Urban and Stroner (201 2), Elnima (201 3) suggested an

• ptimizaation model based on genetic evolution

algorithm to solve the three point space resection algorithm to solve the three point space resection problem The solution does not need linearization nor The solution does not need linearization nor redundancy. The proposed model is simple and converges to the The proposed model is simple and converges to the global optimal solution. The disadvantage is the high computational demand.

38

Automobile Construction Machine Construction, Metalworking, Quality Control Mining Engineering Mining Engineering Objects in Motion Shipbuilding Shipbuilding Structures and Buildings Traffic Engineering Traffic Engineering Biostereometrics

39

Collinearity condition and collinearity equations are Collinearity condition and collinearity equations are more suitable for photogrammetric students who want to go deep in sight of the problem, considering single photo. photo. Iteration solution needing initial approximations of parameters give good accuracy suitable for surveying parameters give good accuracy suitable for surveying

• needs. It makes no problem with the use of

computers. computers. Initial approximations can now be easily obtained either using simple DLT or when GPS is used.

40

41