[PPT] - Institute of Systems Optimization Vision Based Landing System for a PowerPoint Presentation

SLIDE 1

www.ite.uni-karlsruhe.de

10.2008

Institute of Systems Optimization

Vision Based Landing System for a VTOL-MAV

N. Frietsch, O. Meister, C. Schlaile,
J. Seibold, G. F. Trommer

SLIDE 2

Natalie Frietsch Institute of Systems Optimization 1

Introduction

Operation without legal restrictions
Autonomous flight also in urban

environments

Teaming UAV/UAV and UAV/UGV
Tracking and geo-localization of
bjects

Technical Aims

GPS signals not always available Augmentation of navigation system with image based system

SLIDE 3

Natalie Frietsch Institute of Systems Optimization 2

Outline

AirQuad Image based navigation estimation Image based height estimation Simulation environment Results Conclusion

SLIDE 4

Natalie Frietsch Institute of Systems Optimization 3

AirQuad

Electrically powered
Max. dimensions 92 cm
Take-off weight 1000 g
Payload capacity 200 g
Operating time 25 min
Max. altitude ~ 500 m
Max. speed ~ 50 km/h
Operating range > 5 km

Specifications

SLIDE 5

Natalie Frietsch Institute of Systems Optimization 4

Image based navigation estimation

Augmentation of navigation system during hovering and landing situations

Assumption Optical flow estimation

Homographies suitable for motion estimation

Lucas-Kanade Algorithm
Optical flow with census transform (based on Stein 2004, Zabih and

Woodfill 1994) augmented with cross-correlation

SLIDE 6

Natalie Frietsch Institute of Systems Optimization 5

Image based navigation estimation

Estimation with RANSAC (RANdom SAmple Consensus)
Calibrated homography
Decomposition:

Homography estimation and decomposition

with rotation matrix translation distance to scene plane normal vector of in camera coordinates

Decomposition into

SLIDE 7

Natalie Frietsch Institute of Systems Optimization 6

Image based navigation estimation

To be known

– Initial position and attitude – Attitude

between Camera and MAV

– Distance to scene , in this case height above ground

Propagation of attitude and position

Estimation of distance to scene necessary

SLIDE 8

Natalie Frietsch Institute of Systems Optimization 7

Image based height above ground estimation

With theorem of intersecting lines
From two different positions

SLIDE 9

Natalie Frietsch Institute of Systems Optimization 8

Image based height above ground estimation

Orientation of MAV and camera compensated
Equation numerically well-conditioned
Equivalent is motion in vertical direction:

Estimation Conditions

from barometric pressure sensor
from optical flow

SLIDE 10

Natalie Frietsch Institute of Systems Optimization 9

Image based height above ground estimation

Motion in vertical direction:
e. g.
Displacement not from noise:
e. g.

Conditions

with Kalman filter:

known inputs, measurements

with optical flow:

Continuous estimation of height above ground

SLIDE 11

Natalie Frietsch Institute of Systems Optimization 10

Outline

AirQuad Image based navigation estimation Image based height estimation Simulation environment Results Conclusion

SLIDE 12

Natalie Frietsch Institute of Systems Optimization 11

Simulation environment

Essential for algorithm development and testing
MAV model included
Test of operational C-code

Navigation system Guidance Flight controller Motor/rotor model MAV dynamic model Generation of sensor data Disturbances Evaluation/ analysis

= software under test = MAV + sensor model

Generation of

ptical flow data

Image based system

SLIDE 13

Natalie Frietsch Institute of Systems Optimization 12

Results

Frame rate 25 fps, image size 640x480, 200 features, feature noise pix.

1. Simulation: Hovering at defined position and landing

Position Error of position Ground truth GPS/INS/Mag/Baro Vision

SLIDE 14

Natalie Frietsch Institute of Systems Optimization 13

Results

Positions divided by 1/25fps, Averaging with n = 6, data rate 4.16Hz

Hovering at defined position and landing

Ground truth GPS/INS/Mag/Baro Vision Velocity Error of velocity

SLIDE 15

Natalie Frietsch Institute of Systems Optimization 14

Results

Hovering at defined position and landing

Ground truth GPS/INS/Mag/Baro Vision Error of attitude Angular velocity of yaw angle Error of angular velocity

Magnetometer measurement can be corrupted by metallic surfaces.

Yaw angle is divided by 1/25fps

Improvement by vision system

Yaw angle

SLIDE 16

Natalie Frietsch Institute of Systems Optimization 15

Results

Hovering at defined position and landing

Ground truth Vision meas. Kalman filter Baro/Vision Height above ground estimation Error of height above ground Vision Baro rate 25Hz, baro offset -5m, baro noise m, baro drift 0.2 m/min

SLIDE 17

Natalie Frietsch Institute of Systems Optimization 16

Results

Hovering at defined position and landing

Height above ground estimation Error of height above ground Ground truth Vision meas. Kalman filter Baro/Vision Vision Simulated step of ground elevation of 2.5 m

SLIDE 18

Natalie Frietsch Institute of Systems Optimization 17

Results

2. Simulation: Waypoint flight

Waypoint flight

11 waypoints
Hover-and-stare points
~ 10 min
Height up to 30 m

Ground truth GPS/INS/Mag/Baro Vision

Last waypoint First waypoint

SLIDE 19

Natalie Frietsch Institute of Systems Optimization 18

Results

Frame rate 25 fps, image size 640x480, 200 features, feature noise pix.

Waypoint flight

GPS/INS/Mag/Baro Vision Error of position Error of velocity

SLIDE 20

Natalie Frietsch Institute of Systems Optimization 19

Results

Waypoint flight

Ground truth Vision meas. Kalman filter Baro/Vision Vision Height above ground estimation Error of height above ground Baro rate 25Hz, baro offset -5m, baro noise m, baro drift 0.2 m/min

SLIDE 21

Natalie Frietsch Institute of Systems Optimization 20

Results

3. Processing of in-flight data

Position Velocity GPS/INS/Mag/Baro Vision First results on processing of in-flight data Positions divided by 1/30fps, Averaging with n = 6, data rate 5Hz

SLIDE 22

Natalie Frietsch Institute of Systems Optimization 21

Results

Processing of in-flight data

Height above ground estimation

First tests with in-flight data confirm results of simulations

Barometric sensor data Vision Kalman filter Baro/Vision

Augmentation of navigation system possible

SLIDE 23

Natalie Frietsch Institute of Systems Optimization 22

Conclusion

Conclusion Future Work

Image based navigation aiding based on homographies in cases of

+ hovering and + landing

Height above ground estimation solely with

+ optical flow and + barometric sensor data

Integration in navigation and guidance modules
Implementation of algorithms on on-board image processing

hardware

SLIDE 24

Natalie Frietsch Institute of Systems Optimization 23

Institute of Systems Optimization Thank you for your attention.

SLIDE 25

Natalie Frietsch Institute of Systems Optimization 24

SLIDE 26

Natalie Frietsch Institute of Systems Optimization 25

Image based navigation

Comparison of gray values in neighborhood
Conversion of signature vector to decimal integer
Store points according to signature vector in table
Correspondences

, between images by comparing tables

Optical flow with census transform

222 142 127 127 235 191 15 33 69 2 1 1 2 2

‘ 20001122‘

SLIDE 27

Natalie Frietsch Institute of Systems Optimization 26

Image based navigation

Optical flow with census transform

Use neighbors in distance , e. g.
Filtering of signatures of one image

– Use only signatures including useful information

e. g. reject ‘11111111’

– Use only infrequent signatures e. g. less than 5 times in the image

Filtering of correspondences

– Hamming-Distance of 0 – Distance between points not too

large e. g. less than 50 pixels

– Gray values of pixels similar

e. g. less than 20% deviation

Result not robust

SLIDE 28

Natalie Frietsch Institute of Systems Optimization 27

Census: Census after

RANSAC:

LK: LK after

RANSAC:

Image based navigation estimation

Results of optical flow calculation

SLIDE 29

Natalie Frietsch Institute of Systems Optimization 28

Image based navigation estimation

Estimation with RANSAC (RANdom SAmple Consensus)
Calibrated homography
Decomposition:

Homography estimation and decomposition

with rotation matrix translation distance to scene plane normal vector of in camera coordinates

Rotation
Sign by
Singular value decomposition gives 2 physically possible solutions

SLIDE 30

Natalie Frietsch Institute of Systems Optimization 29

Image based navigation estimation

Integration in navigation coordinate system

with and from images

Camera fixed on MAV: = const, centers coincide

Propagation of attitude and position

SLIDE 31

Natalie Frietsch Institute of Systems Optimization 30

Image based height above ground estimation

Orientation of MAV and camera compensated
Motion in vertical direction:
e. g.
Displacement not from noise:
e. g.

Conditions

SLIDE 32

Natalie Frietsch Institute of Systems Optimization 31

Image based height above ground estimation

Motion in vertical direction:
e. g.
Displacement not from noise:
e. g.

Estimation Conditions

from barometric pressure sensor
from optical flow

Continuous estimation of height above ground with Kalman filter known inputs, measurements

SLIDE 33

Natalie Frietsch Institute of Systems Optimization 32

Simulation environment

Generation of feature points in camera coordinates
Projection on earth plane and storage in database
Search of features visible in last and current pose
Estimation of height above ground
Estimation of homography , rotation and

translation Adjustable parameters: image size, image rate, internal camera parameters, number of features, feature noise …

Simulation of vision system

Generation of

ptical flow data

Image based system

SLIDE 34

Natalie Frietsch Institute of Systems Optimization 33

Results

1. Simulation: Hovering at defined position and landing

Ground truth GPS/INS/Mag/Baro Vision Attitude Error of attitude

SLIDE 35

Natalie Frietsch Institute of Systems Optimization 34

Results

Hovering at defined position and landing

Ground truth GPS/INS/Mag/Baro Vision Angular velocity Error of angular velocity Angles divided by 1/25fps

SLIDE 36

Natalie Frietsch Institute of Systems Optimization 35

Results

Hovering at defined position and landing

Height above ground estimation Error of height above ground Baro rate 25Hz, baro offset -5m, baro noise m, baro drift 0.2 m/min Ground truth Vision Kalman filter Baro/Vision

SLIDE 37

Natalie Frietsch Institute of Systems Optimization 36

Results

Hovering at defined position and landing

Simulated step of ground elevation of 2.5 m Ground truth Vision Kalman filter Baro/Vision Height above ground estimation Error of height above ground

SLIDE 38

Natalie Frietsch Institute of Systems Optimization 37

Results

2. Simulation: Waypoint flight

Error of attitude Error of position Frame rate 25 fps, image size 640x480, 200 features, feature noise pix. GPS/INS/Mag/Baro Vision

SLIDE 39

Natalie Frietsch Institute of Systems Optimization 38

Results

Waypoint flight

GPS/INS/Mag/Baro Vision Error of velocity Error of angular velocity Positions divided by 1/25fps, Averaging with n = 6, data rate 4.16Hz Angles divided by 1/25fps

SLIDE 40

Natalie Frietsch Institute of Systems Optimization 39

Results

Waypoint flight

Height above ground estimation Error of height above ground Baro rate 25Hz, baro offset -5m, baro noise m, baro drift 0.2 m/min Ground truth Vision Kalman filter Baro/Vision

SLIDE 41

Natalie Frietsch Institute of Systems Optimization 40