A Constant-Time Efficient Stereo SLAM System Christopher Mei 1 , - - PowerPoint PPT Presentation

World representations Stereo Processing Experimental results Conclusion

A Constant-Time Efficient Stereo SLAM System

Christopher Mei1, Gabe Sibley2, Mark Cummins2, Paul Newman2 and Ian Reid1

1Active Vision Group, 2Mobile Robotics Group,

University of Oxford

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 1/25

World representations Stereo Processing Experimental results Conclusion

Simultaneous Localisation and Mapping

Goal of SLAM : to estimate the trajectory of a sensor in a iterative fashion, build a representation of the environment. What makes SLAM challenging ? computationally expensive (the complexity grows with the size of the environment), requires robust data fusion algorithms.

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 2/25

World representations Stereo Processing Experimental results Conclusion

A constant time stereo SLAM system

This talk will describe... a continuous relative representation (CRR) providing a way to represent precisely the local environment and improve accuracy at loop closure without global estimation, low-level vision tasks: image pyramids, quadtree feature selection, scale invariance through stereo, relocalisation and sub-pixel minimisation to provide improved accuracy and robustness. ... to obtain a constant-time precise and robust stereo SLAM system.

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 3/25

World representations Stereo Processing Experimental results Conclusion

Outline

1

World representations

2

Stereo Processing

3

Experimental results

4

Conclusion

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 4/25

World representations Stereo Processing Experimental results Conclusion

Outline

1

World representations

2

Stereo Processing

3

Experimental results

4

Conclusion

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 5/25

World representations Stereo Processing Experimental results Conclusion

Global representations

Global Representations “standard” representation: robot poses and landmarks are represented in a global frame, robocentric representation: global frame centred on the current robot position. Improves the consistency of EKF-SLAM [Castellanos et al IFAC 2004]

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 6/25

World representations Stereo Processing Experimental results Conclusion

Sub-mapping

Sub-maps sub-maps: robot poses and landmarks are grouped in smaller maps. Each sub-map uses a global or relative representation ([Estrada et al. ITRO 2005]

[Bosse et al. IJRR 2004] ),

reduces the complexity and improves consistency, limitations: sharing information between sub-maps is a difficult problem in particular when updating the estimates after loop closure.

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 7/25

World representations Stereo Processing Experimental results Conclusion

Continuous Relative Representation

Changing the representation has a profound impact on the mapping process. In particular, the maximum likelihood cost functions for the global representation and CRR are different: See [Sibley et. al RSS 2009]

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 8/25

World representations Stereo Processing Experimental results Conclusion

Example with a loop closure

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 9/25

World representations Stereo Processing Experimental results Conclusion

Change of perspective

Before loop closure After loop closure Change of perspective a robot should be able to evolve in an environment with non observable ego-motion, a challenge is to find the places where such events occur (lifts, trains, planes, ...)

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 10/25

World representations Stereo Processing Experimental results Conclusion

Advantages and limitations of the CRR

Advantages simplifies loop closure and data association, the precision at loop closure is the same as during exploration. Limitations to obtain a coherent global Euclidean representation, global estimation is required (with a complexity dependent

• n the size of the map), but is this often acquired?
• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 11/25

World representations Stereo Processing Experimental results Conclusion

Outline

1

World representations

2

Stereo Processing

3

Experimental results

4

Conclusion

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 12/25

World representations Stereo Processing Experimental results Conclusion

Stereo SLAM processing steps

Pre-processing Feature extraction Pose initialisation (SSD rotation) Image to map data association Robust estimation

• f the robot pose

Adding of new landmarks

Loop closure is done using a bag of words approach: FabMap [Cummins and Newman IJRR 2008].

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 13/25

World representations Stereo Processing Experimental results Conclusion

Components of the SLAM system

Processing steps that greatly effect the system’s performance Pose initialisation by image-based rotation estimation ([Mei

• et. al ITRO 2008]),

Better pose conditioning by spreading features using quadtrees, Sub-pixel minimisation for each landmark to image matching to improve precision, Efficient scale invariance by using the landmark distance (“true scale”).

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 14/25

World representations Stereo Processing Experimental results Conclusion

Improving conditioning using quadtrees

Without quadtrees With quadtrees

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 15/25

World representations Stereo Processing Experimental results Conclusion

Improving precision through sub-pixel minimisation

Applying sub-pixel minimisation (eg. Kanade-Lucas-Tomasi tracker) greatly improves precision and data association. Without sub-pixel minimisation With sub-pixel minimisation

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 16/25

World representations Stereo Processing Experimental results Conclusion

Efficient scale invariance

20 40 60 80 100 5 10 15 20 25 30 35 Distance (m) Patch size (pixels)

Scale invariance in SIFT: obtained by looking for a maximum score response at different scales in a DoG pyramid, this is an expensive process, Alternative: choose a fixed 3-D template size for computing the descriptors, (“true scale”). Problem: the camera image size is insufficient to represent large changes in depth, Solution: split the 3-D space in regions (“bands”) where the 3-D templates have the same size.

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 17/25

World representations Stereo Processing Experimental results Conclusion

Outline

1

World representations

2

Stereo Processing

3

Experimental results

4

Conclusion

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 18/25

World representations Stereo Processing Experimental results Conclusion

Experimental results

Begbroke Science Park (1.1km) New College (1.8km)

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 19/25

World representations Stereo Processing Experimental results Conclusion

Importance of loop closure on precision (1/2)

Trajectory estimates with/without loop closure. Begbroke New College

−40 −30 −20 −10 10 −10 −5 5 10 15 20 25 30 35 x (m) y (m) −200 −150 −100 −50 50 −50 50 100 150 200 250 x (m) y (m) −140 −120 −100 −80 −60 −40 −20 20 −50 50 100 150 200 250 y (m) x (m) −30 −20 −10 10 −10 10 20 30 −0.5 0.5 1 1.5 2 x (m) y (m) z (m) −200 −150 −100 −50 50 −200 200 400 −5 5 10 15 x (m) y (m) z (m) −200 −100 100 200 −100 100 200 300 −20 20 x (m) y (m) z (m)

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 20/25

World representations Stereo Processing Experimental results Conclusion

Importance of loop closure on precision (2/2)

Begbroke New College Distance Travelled 1.08 km 2.26 km Frames Processed 23K 51K Reprojection Error Min/Avg/Max 0.003 / 0.17 / 0.55 pixels 0.03 / 0.13 / 1.01 pixels Accuracy without loop closure ∼1m in (x-y) plane, ∼1m in z ∼15-25m in (x-y) plane, ∼15m in z Accuracy with loop closure ∼1cm in (x-y) plane, ∼1cm in z ∼10cm in (x-y) plane, ∼10cm in z

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 21/25

World representations Stereo Processing Experimental results Conclusion

Results on difficult sequences

Large changes in view- point Blur Lens Flare and Saturation

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 22/25

World representations Stereo Processing Experimental results Conclusion

Outline

1

World representations

2

Stereo Processing

3

Experimental results

4

Conclusion

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 23/25

World representations Stereo Processing Experimental results Conclusion

Advantages and limitations

Advantages constant-time computation, robust: relocalisation and loop closure, adapted to large scale exploration: the only limit is the memory and not the computational resources. Limitations loop closing is based on appearance and only triggered in distinctive environments: no guarantee of unicity or bounds on memory can be given, the visualisation of the map can be difficult for a user, imposing Euclidean constraints violates the constant-time performance guarantee.

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 24/25

World representations Stereo Processing Experimental results Conclusion

Future work

Future work detecting non-observable ego-motion between Euclidean maps (eg: lifts, cars, trains, ...) long-term mapping: how can we detect change from simple occlusions and tracking failures? [A demo will be running during tomorrow’s poster session.]

• C. Mei et. al

BMVC 2009 - Constant-Time Stereo SLAM - 25/25