1 Error Sources: Systematic Biases Linear Scanner & - - PDF document

SLIDE 1

1

Quality Assurance and Quality Control

• f LiDAR Systems and Derived Data

A F H bib

Ayman F. Habib

Ayman F. Habib Digital Photogrammetry Research Group http://dprg.geomatics.ucalgary.ca Department of Geomatics Engineering University of Calgary, Canada

Introduction

Ayman F. Habib

2

Overview

• General background: LiDAR principles
• Error budget for LiDAR
• LiDAR quality assurance
• LiDAR quality control

Ayman F. Habib

3

– Internal/relative quality control (IQC) – External/absolute quality control (EQC)

• Experimental results
• Final remarks

Three Measurement Systems

1. GNSS 2. IMU 3 Laser scanner emits laser

LiDAR Principles

IN INS GNSS IMU

Ayman F. Habib

4

3. Laser scanner emits laser beams with high frequency and collects the reflections.

IN INS

GNSS

IMU X ⎡ ⎤ ⎡ ⎤

INS

R

scan

R

LiDAR Equation

Ayman F. Habib

5

i i i

i INS G INS scan

X G Y X R P R R R Z ρ

Δ

⎡ ⎤ ⎡ ⎤ ⎢ ⎥ ⎢ ⎥ = = + + ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ − ⎣ ⎦ ⎢ ⎥ ⎣ ⎦ r r r

X r

G

P r ρ −

i

G r RΔ

Range Data (Shaded Relief)

LiDAR Output

Ayman F. Habib

6

Intensity Data

SLIDE 2

2

• We would like to show the effect of biases in the

LiDAR measurements on the reconstructed object space.

• The effects will be derived through a simulation

process:

Si l d f & T j LiDAR

Error Sources: Systematic Biases

Ayman F. Habib

7

– Simulated surface & Trajectory LiDAR measurements Add biases Reconstructed surface.

• The effects will be shown through the difference

between the reconstructed footprints and the simulated surface (i.e., ground truth).

• These effects will be shown for linear LiDAR

systems.

50 50.05 50.1 50.15 Ground Truth & Biased Surface z-axis Ground Truth Biased Surface Trajectory

Linear Scanner & Boresighting Angular Bias

Ayman F. Habib

8

• 400
• 300
• 200
• 100

100 200 300 400 1000 2000 3000 49.95 x-axis y-axis

Linear Scanner & Boresighting Angular Bias

0.2 0.4 0.6 0.8 1 X,Y,Delta XYZ z-axis X,Y, Delta X X,Y, Delta Y X,Y, Delta Z

erence (m)

Ayman F. Habib

9

• Opposite Flight Directions & 30% Overlap
• Overlap area can be used to check the presence of biases
• 600
• 400
• 200

200 400 600

• 1
• 0.8
• 0.6
• 0.4
• 0.2

x-axis

Profile Diffe Flying Height Flying Direction Look Angle Boresighting Offset Bias Effect is independent of the Flying Height Effect is dependent on the Flying Direction (Except ΔZ) Effect is independent of the Look Angle Boresighting Angular Bias Effect Increases with the Flying Height Effect Changes with the Flying Direction Effect Changes with the Look Angle (Except ΔX)

Error Sources: Systematic Biases

Ayman F. Habib

10

Laser Beam Range Bias Effect is independent of the Flying Height Effect is independent of the Flying Direction Effect Depends on the Look Angle (Except ΔY) Laser Beam Angular Bias Effect Increases with the Flying Height Effect Changes with the Flying Direction (Except ΔY) Effect Changes with the Look Angle (Except ΔX)

• Assumption:

Linear Scanner Constant Attitude & Straight Line Trajectory Flying Direction Parallel to the Y axis Flat horizontal terrain

• The effect of random errors can be analyzed in
• ne of two different ways:

– Approach # I:

• Simulated surface & Trajectory LiDAR measurements

Add noise Reconstructed surface.

Error Sources: Random Errors

Ayman F. Habib

11

• Evaluate the difference between the reconstructed footprints

and the simulated surface (i.e., ground truth).

– Approach # II:

• Use the law of error propagation to evaluate the accuracy

(noise level) of the derived point cloud as it is determined by the accuracy (noise level) in the LiDAR measurements.

50 50.5 Ground Truth & Noisy Surface z-axis Ground Truth Noisy Surface Trajectory

Linear Scanner & Orientation Noise (I)

Ayman F. Habib

12

• 400
• 300
• 200
• 100

100 200 300 400

• 500

500 1000 49.5 x-axis y-axis

• Propagates with the flying height
• Dependent on the look angle

SLIDE 3

3

Linear Scanner & Orientation Noise (II)

0.8 1 1.2 1.4

uted Coordinates (m)

Accuracy of X Coordinates Accuracy of Y Coordinates Accuracy of Z Coordinates

Flying Height = 500m

Nadir Directions

Ayman F. Habib

13

20 40 60 80 100 120 140 160 180 200 0.2 0.4 0.6

Point Number (1 cycle) Accuracy of Compu Nadir Directions

• Propagates with the flying height
• Dependent on the look angle

One Scan

Linear Scanner & Orientation Noise (II)

Flying Height = 1000m

1.5 2 2.5

uted Coordinates (m)

Accuracy of X Coordinates Accuracy of Y Coordinates

Nadir Directions

Ayman F. Habib

14

20 40 60 80 100 120 140 160 180 200 0.5 1

Point Number (1 cycle) Accuracy of Compu

y Accuracy of Z Coordinates

• Propagates with the flying height
• Dependent on the look angle

One Scan

LiDAR Error Propagation Calculator

Ayman F. Habib

15

http://ilmbwww.gov.bc.ca/bmgs/pba/trim/specs

Quality Assurance & Control

• Quality assurance (before mission):

– Management activities to ensure that a process, item, or service is of the quality needed by the user. – It deals with creating management controls that cover planning, implementation, and review of data collection activities.

Ayman F. Habib

16

– Key activity in the quality assurance is the calibration calibration procedure procedure.

• Quality control (after mission):

– Provide routines and consistent checks to ensure data integrity, correctness, and completeness. – Check whether the desired quality has been achieved.

Photogrammetric Quality Assurance

• One of the key issues in quality assurance of data

acquisition systems is the calibration process.

• Camera Calibration.

– Laboratory calibration. – Indoor calibration.

Ayman F. Habib

17

– In-situ calibration.

• Total system calibration.

– Spatial and rotational offsets between various system components (e.g., camera, GPS, and IMU).

• Other QA measures include:

– Number & configuration of GCP, side lap percentage, distance to GPS base station.

Photogrammetric Quality Assurance

Ayman F. Habib

18

Laboratory Calibration: Multi-Collimators

SLIDE 4

4

• Photogrammetric reconstruction is based on redundant

measurements.

• Results from the photogrammetric triangulation gives

quantitative measures of the precision of the reconstruction

• utcome.

Photogrammetric Quality Control

Ayman F. Habib

19

– Variance component (overall measure of the quality of fit between the observed quantities and the used model). – Variance-covariance matrix for the derived object coordinates. – These values can be compared with expected nominal values.

• Independent measure for accuracy verification can be

established using check point analysis.

– Photogrammetric coordinates are compared with independently measured coordinates (e.g., GPS survey) RMSE analysis.

Photogrammetric Quality Control

Ayman F. Habib

20

Check Point Analysis

• Possible systematic errors:

– Spatial and rotational offsets between the various system components. – Range bias.

LiDAR QA: System Calibration

Ayman F. Habib

21

– Angular mirror bias.

• Calibration requires some control information.

– What are the most appropriate primitives?

• The appropriate configuration of the control

information and the flight mission.

LiDAR QA: System Calibration

• Target Function:

Target Function: minimize the normal distance between the laser point footprint and a known (control) surface.

• Use the LiDAR equation to

estimate the error parameters

firing point

Most appropriate primitives

Ayman F. Habib

22

p that minimize the cost of the target function.

• Caution: flight and control

surface configurations should be carefully established.

laser point

d

Only possible if we are dealing with a transparent system parameters (LAS ?)

• Quality control is a post-mission procedure to

ensure/verify the quality of collected data.

• Quality control procedures can be divided into two

main categories:

LiDAR Quality Control

Ayman F. Habib

23

– External/absolute QC measures: the LiDAR point cloud is compared with independently collected surface.

• Check point analysis.

– Internal/relative QC measures: the LiDAR point cloud from different flight lines is compared with each other to ensure data coherence, integrity, and correctness.

• External/absolute quality control measures (EQC):

– Similar to photogrammetric quality control, the derived LiDAR coordinates can be compared with independently surveyed targets.

• Check point analysis.

EQC: LiDAR Control Targets

Ayman F. Habib

24

p y

– Problem: How to correlate the non-selective LiDAR footprints to the utilized check points. – Solution: Use specially designed targets.

• The target design depends on the involved LiDAR system.

SLIDE 5

5

EQC: LiDAR Control Targets

Ayman F. Habib

25

EQC: LiDAR Control Targets

Ayman F. Habib

26

Range Data Intensity Data

• We have to implement a segmentation procedure to

derive the LiDAR coordinates of the target.

IQC: LiDAR Quality Control

• Surface reconstruction from LiDAR does not

have redundancy.

– Therefore, we do not have explicit measures in the derived surfaces to assess the quality of LIDAR- derived surfaces.

Ayman F. Habib

27

• Users should adopt other measures to evaluate

the internal quality internal quality of the derived LiDAR surfaces (IQC).

• Alternative methodologies are based on the:

– Coincidence of conjugate features in overlapping strips.

IQC: LiDAR Quality Control

Ayman F. Habib

28

Strip 2 Strip 3 Strip 4

IQC: LiDAR Quality Control

Ayman F. Habib

29

• Using interpolated intensity and range images:

– Interpolate the intensity and range data into a grid Intensity and range images. – Identify distinct features in the intensity images.

IQC: LiDAR Quality Control (#1)

Ayman F. Habib

30

y y g

• For these features, the X, Y, and Z coordinates can be derived.

– Compare the derived coordinates of the same feature from overlapping strips.

• Caution: Interpolation would lead to artifacts in

the interpolated images (especially at the vicinity

• f discontinuities in the intensity and range data).

SLIDE 6

6

IQC: LiDAR Quality Control (#1)

Ayman F. Habib

31

Intensity Images

DX(m) DY(m) DZ(m)

IQC: LiDAR Quality Control (#1)

Ayman F. Habib

32

• 0.97

0.00 1.92 DX(m) DY(m) DZ(m)

• 0.79

0.25 0.05

Average (m) Standard deviation (m)

IQC: LiDAR Quality Control (#1)

• The average and standard deviation of the estimated

discrepancies between 100 points in two overlapping strips

Ayman F. Habib

33

g ( ) ( ) X 0.45 ±0.36 Y 0.50 ±0.37 Z 0.22 ±0.28

• Interpolation and interpretation of the LiDAR data

might introduce artifacts, which will lead to unreliable quality control measures.

• Alternative procedures should be developed while

IQC: LiDAR Quality Control (#1)

Ayman F. Habib

34

p p relying on the raw data:

– Extract features from the raw LiDAR data. – Compare conjugate features in overlapping strips. – Deviations can be used as a quality control measure.

IQC: LiDAR Quality Control (#2)

Ayman F. Habib

35

• Check the quality of coincidence of conjugate planar patches.

7 1884 7.1884 7.1884 910 915 920 925 Z-Axis

7.1884 7.1884 910 915 920 925 Z-Axis

IQC: LiDAR Quality Control (#2)

Ayman F. Habib

36

2.2676 2.2676 2.2676 2.2676 2.2676 2.2676 2.2676 x 10

7

7.1884 7.1884 7.1884 7.1884 7.1884 x 10

6

X-Axis Y-Axis

2.2676 2.2676 2.2676 2.2676 2.2676 2.2676 2.2676 x 10 7 7.1884 7.1884 7.1884 7.1884 7.1884 7.1884 x 10 6 X-Axis Y-Axis

Strip # 3 Strip # 4

SLIDE 7

7

Ayman F. Habib

37 37 Strips 2 & 3 Strip 3&4 Strips 2 & 4 Transformation parameter / # of Patches 21 22 22 Scale Factor 1.0000 0.9996 0.9995 XT (m)

• 0.52

0.72 0.08 YT (m)

• 0 13
• 0 17
• 0 21

IQC: LiDAR Quality Control (#2)

Ayman F. Habib

38 YT (m) 0.13 0.17 0.21 ZT (m) 0.05 0.09 0.14 Ω (°) 0.0289

• 0.0561
• 0.0802

Φ (°) 0.0111

• 0.0139
• 0.0342

Κ (°) 0.0364 0.0288 0.0784 Normal Distance, m (After) 0.04 0.03 0.04

Estimated transformation parameters using conjugate planar patches in overlapping strips.

IQC: LiDAR Quality Control (#3)

Ayman F. Habib

39

Linear Feature Extraction

manual identification of LiDAR patches with the aid of imagery

IQC: LiDAR Quality Control (#3)

Ayman F. Habib

40

Ayman F. Habib

41 41

• 10
• 5

5 Original Lines 19 19 15 15 m) Model Lines Object Lines 10

• 5

5 Transformed Lines 19 19 15 15 m) Model Lines Object Lines

IQC: LiDAR Quality Control (#3)

Ayman F. Habib

42

75 80 85 90 95 100 105

• 25
• 20
• 15
• 10

18 18 14 13 13 14 X(m) Y(m 80 85 90 95 100 105 110

• 25
• 20
• 15
• 10

18 18 13 14 14 13 X(m) Y(m

SLIDE 8

8

Strips 2 & 3 Strips 3 & 4 Strips 2 & 4 Transformation parameter / # of Lines 24 36 24 Scale Factor 1.0002 1.0006 1.0013 XT (m)

• 0.56

0.75 0.10 YT (m) 0.04

• 0.17
• 0.16

IQC: LiDAR Quality Control (#3)

Ayman F. Habib

43

T ( )

ZT (m) 0.03 0.05 0.13 Ω (°) 0.0205

• 0.0386
• 0.0147

Φ (°) 0.0062

• 0.0125
• 0.0073

Κ (°) 0.0261

• 0.0145
• 0.0113

Normal Distance, m (Before) 0.38 ± 0.22 0.49 ± 0.24 0.26 ± 0.14 Normal Distance, m (After) 0.18 ± 0.19 0.18 ± 0.18 0.16 ± 0.11

Estimated transformation parameters using conjugate linear features in

• verlapping strips
• The previous IQC measures requires

preprocessing of the raw LiDAR data:

– Interpolation, planar patch segmentation, plane fitting, and/or intersection.

A th h b d i d hil i th

IQC: LiDAR Quality Control (# 1 – 3)

Ayman F. Habib

44

• Another approach can be devised while using the
• riginal point cloud.

– One strip is represented by a set of irregularly distributed points (LiDAR point cloud). – Second strip is represented by a TIN generated from the LiDAR point cloud. – Iterative Closest Patch (ICPatch).

) , , , , , , ( κ ϕ ω S Z Y X

T T T

i

q

' i

q

p

S

c p

S

a p

S

b p

S

IQC: LiDAR Quality Control (#4)

Ayman F. Habib

45

1

S

2

S

• Starting from a given set of approximate parameters, we

determine conjugate point-patch pairs in overlapping strips.

• Conjugate primitives are used to estimate an updated set
• f parameters, which are then used to determine new

correspondences.

• The approach is repeated until convergence.

IQC: LiDAR Quality Control (#4)

Ayman F. Habib

46

Register

IQC: LiDAR Quality Control (#4)

Ayman F. Habib

47

Green: Reference Surface Blue: Matches Red: Non-matches

Strips 2& 3 Strips 3& 4 Strips 2& 4 Scale Factor 0.9996 0.9998 0.9993 XT (m)

• 0.55

0.75 0.19 YT (m)

• 0.06
• 0.13
• 0.18

IQC: LiDAR Quality Control (#4)

Ayman F. Habib

48

YT (m) 0.06 0.13 0.18 ZT (m) 0.03 0.12 0.16 Ω (°) 0.0080

• 0.0267
• 0.0213

Φ (°) 0.0059

• 0.0088
• 0.0053

Κ (°)

• 0.0009
• 0.0003

0.0012 Average Normal Dist., m 0.09 0.09 0.10

SLIDE 9

9

IQC: LiDAR Quality Control (#5)

Ayman F. Habib

49 49

• Iterative Closest Point (ICP):

– Do we have conjugate points? – Is the performance impacted by the average point density?

• This approach is similar to the ICPatch procedure.

However, instead of using a TIN to represent the second strip, we use the original LiDAR point cloud.

• Iterative Closest Point (ICPoint) is used to

determine the correspondence between conjugate

IQC: LiDAR Quality Control (#5)

Ayman F. Habib

50

points in overlapping strips (starting from an approximate set of transformation parameters).

– Is there conjugate points?

• Conjugate points are used to estimate an updated

set of parameters, which are then used to determine new correspondences.

• The approach is repeated until convergence.

Strips 2& 3 Strips 3& 4 Strips 2& 4 Scale Factor 0.9997 1.0002 0.9994 XT (m)

• 0.47

0.70 0.26 YT (m)

• 0.27
• 0.32
• 0.41

IQC: LiDAR Quality Control (#5)

Ayman F. Habib

51

YT (m) 0.27 0.32 0.41 ZT (m) 0.00 0.04 0.15 Ω (°) 0.0132

• 0.0394
• 0.0302

Φ (°) 0.0082

• 0.0141
• 0.0059

Κ (°) 0.0039

• 0.0007
• 0.0100

Average Distance, m 0.51 0.51 0.60

Experimental Results

• Previous results were derived from three strips

captured in Brazil:

– Triple overlap, ~ 1000 m flying height, 50 KHZ pulse rate, ~ 0.70 m point spacing, 15 cm RMSEZ & 50 cm RMSEXY (manufacturer specification).

Ayman F. Habib

52

• The following results are derived from eleven

strips captured over the University of Calgary (UofC) Campus.

– 50% overlap, ~ 1200 m flying height, 50 KHZ pulse rate, ~ 0.75 m point spacing, 9 cm RMSEZ (reported by the data provider).

Experimental Results (UofC)

Ayman F. Habib

53

ICPoint ICPatch Lines Patches

Strips 3 & 4

Estimated Transformation parameters

SF XT (m) YT (m) ZT (m) Omega (deg) Phi (deg) Kappa (deg) Av_Dist Ndist(m)

Patches method

1.00019

• 0.02
• 0.02

0.02

• 0.0151

0.0023 0.0052 0.03

Method Parameters

Experimental Results (UofC)

Ayman F. Habib

54

08803 & 08804

Lines Collinearity

1.00009 0.04

• 0.08

0.02

• 0.0132

0.0020 0.0039 0.10

endpoint

0.99995 0.02

• 0.02

0.01

• 0.0084
• 0.0003

0.0068 0.08

ICPatch

0.99990

• 0.01
• 0.12

0.01

• 0.0023
• 0.0009

0.0029 0.04

ICPoint

0.99980

• 0.08
• 0.27

0.00

• 0.0036
• 0.0011

0.0022 0.51

Consistency in the results coming from various methods

SLIDE 10

10

Experimental Results (UofC)

Ayman F. Habib

55

ICPoint ICPatch Lines Patches

Strips 4 & 5

Estimated Transformation parameters

SF XT (m) YT (m) ZT (m) Omega (deg) Phi (deg) Kappa (deg) Av_Dist Ndist(m) Patches method 1.00003 0.76 0.14

• 0.01

0.0185 0.0060 0.0175 0.03

Method Parameters

Experimental Results (UofC)

Ayman F. Habib

56

08804 & 08805

Lines Collinearity 1.00037 0.80 0.10

• 0.03

0.0156 0.0022

• 0.0011

0.16 End point 0.99987 0.80 0.25

• 0.02

0.0164 0.0054 0.0270 0.13 ICPatch 1.00010 0.86 0.10

• 0.02

0.0039 0.0006 0.0073 0.04 ICPoint 1.00000 0.80

• 0.08
• 0.04

0.0089 0.0004 0.0080 0.57

Consistency in the results coming from various methods

IQC: LiDAR Quality Control

Ayman F. Habib

57

Before Adjustment

IQC: LiDAR Quality Control

Ayman F. Habib

58

After Adjustment

IQC: LiDAR Quality Control

Ayman F. Habib

59

IQC: LiDAR Quality Control

Ayman F. Habib

60

Before After

Points that belong to non-planar patches

SLIDE 11

11

IQC: LiDAR Quality Control

Segmentation Results (Biased Data)

Ayman F. Habib

61

IQC: LiDAR Quality Control

Segmentation Results (Adj. Data)

Ayman F. Habib

62

IQC: LiDAR Quality Control

Ayman F. Habib

63

IQC: LiDAR Quality Control

Ayman F. Habib

64

Before After

Points that belong to non-planar patches

IQC: LiDAR Quality Control

Segmentation Results (Biased Data)

Ayman F. Habib

65

IQC: LiDAR Quality Control

Segmentation Results (Adj. Data)

Ayman F. Habib

66

SLIDE 12

12

IQC: LiDAR Quality Control

Ayman F. Habib

67

IQC: LiDAR Quality Control

Segmentation Results (Biased Data)

Ayman F. Habib

68

IQC: LiDAR Quality Control

Segmentation Results (Adj. Data)

Ayman F. Habib

69

• The previous IQC measures can be used for EQC.
• In such a case, instead of comparing overlapping

strips, the EQC can be evaluated by comparing the LiDAR point cloud to an independently collected surface (ground truth).

LiDAR Quality Control (IQC & EQC)

Ayman F. Habib

70

• Approaches 2-4 will lead to more reliable

estimation of the internal and external quality of the LiDAR data.

• The ICPatch approach is preferred since it is based
• n the original/raw LiDAR point cloud without

the need for any preprocessing.

http://ilmbwww.gov.bc.ca/bmgs/pba/trim/specs

Concluding Remarks

• QA and QC procedures are essential for any

spatial data acquisition system.

• QA of LiDAR data is only possible for a

transparent system.

– Availability of the raw data.

Ayman F. Habib

71

• Quality control of LiDAR data can be conducted

by the end user.

– LiDAR derived data is not based on adjustment procedure. – Quality control measures, which are typically used in photogrammetry, are not applicable. – Alternative procedures are needed.

• The derived quality control procedures takes into

account the irregular and random nature of the LiDAR point cloud.

– Different measures with varying degrees of reliability and complexity.

• Current work is focusing on:

Concluding Remarks

Ayman F. Habib

72

• Current work is focusing on:

– Relating the derived discrepancies to systematic biases in the LiDAR system components. – Deriving methodologies for LiDAR calibration using control planar patches.

• The control patches can be derived from photogrammetric

data.