Low-Drift, Efficient Visual Odometry and SLAM Utilizing - - PowerPoint PPT Presentation

Low-Drift, Efficient Visual Odometry and SLAM Utilizing Environmental Structures

1Seoul National University, South Korea 2Simon Fraser University, Canada

2nd International Workshop on Lines, Planes and Manhattan Models for 3-D Mapping (LPM 2019) May 23, 2019

Pyojin Kim2

pjinkim1215@gmail.com

Seung Jae Lee1

sjlazza@gmail.com

Motivation

□

Importance of Camera Rotational Motion

2

Estimated (left) and True (right) Camera Orientation

• The Main Source of Positional Inaccuracy in VO & SLAM
• Accurately Estimated Translational Motion[1]
• Rotations Causing Nonlinearity in VO & SLAM

Contents

□

Part 1: Absolute Camera Orientation from Multiple Lines and Planes[2-3]

3

□

Part 2: Linear SLAM Formulation with Absolute Camera Rotation[4]

Straub et al., '18[10] Bazin et al., '12[8] Zhou et al., '16[9]

Related Work

□

Separate Rotation & Translation Estimation

□

Drift-Free Rotation Estimation in Structured Environments

4

Kaess et al., '09[5] Bazin et al., '10[7] Cvisic et al., '15[6]

• They cannot estimate drift-free rotational motion of the camera.
• Sensitive and fragile to
• utlier lines
• Require at least two
• rthogonal planes
• Require GPU & two
• rthogonal planes

Main Contributions

□

A New Approach for Drift-Free Orientation from Both Lines and Planes

For full details, refer to [2-3]

□

A New Way for Accurate Translation on the De-Rotated Reprojection Error

□

Evaluation on the Public RGB-D and Author-collected Datasets

Structured Environment Exhibiting Orthogonal Regularities Projection Surface Normal Planes Lines

Structured Environment

Overview of the Proposed VO

□

LPVO[2] (Line and Plane based Visual Odometry)

6 Point Tracking Line Detection

Normal Extraction Depth Image RGB Image VD Extraction

Manhattan Frame Tracking

Point Cloud

De-rotated Reproj. Error Minimization

Drift-Free Rotation Estimation

□

Multiple Lines & Planes with Mean Shift

7 Gaussian Sphere Two Parallel Line Segments Vanishing Direction Surface Normal Vectors Normal Vectors of the Great Circles

Translation Estimation

□

De-rotated Reprojection Error Minimization

8

i-th Tracked Point Feature

: Translation : Rotation

Unknown Known

𝐮∗

Optimal 3-DoF Translation

: De-rotated Reproj. Error w/ Depth : De-rotated Reproj. Error w/o Depth : # of Points w/ Depth : # of Points w/o Depth

𝑠𝑗1 𝐮 , 𝑠𝑗2 𝐮 𝑠𝑗

′ 𝐮

𝐮∗ = arg min

𝐮

෍

𝑗=1 𝑁

𝑠𝑗1

2 𝐮 + 𝑠𝑗2 2 𝐮 + ෍ 𝑗=1 𝑂

𝑠𝑗

′2 𝐮

Experiment Setup

9

ICL-NUIM Dataset (~9.01 m) TUM RGB-D Dataset (~22.14 m) Building-scale Corridor Dataset (~120 m)

: only a single plane

• We compare LPVO[2] with ORB[11], DEMO[1], DVO[12], MWO[9], OPVO[13].

Qualitative Analysis with Floorplan

10

Only LPVO can estimate 6-DoF Nearly 8x more accurate

Qualitative Analysis with Floorplan

11 Video available at https://youtu.be/mt3kbv2TJZw

Quantitative Analysis with True Data

12

Frame Index Translation Error [m] Rotation Error [deg] Rotation error causes failure Average rotation error is ~0.2 deg On average, 5x more accurate

15 Hz @ 10 FPS

Linear RGB-D SLAM with Planar Features

Motivation

□

Development of Simple & Linear SLAM Approach

14

• SLAM is a High Dimensional Nonlinear Problem
• SLAM as A Linear Least Squares Given the Rotation[14]
• Planar Features in Low-Texture Indoor Environments

Effectiveness of the Prior Rotation Information[14] Odometry Initialization Optimum

Torus

Related Work

□

Recent Plane-based SLAM Approaches

15

Main Contributions

□

An Orthogonal Plane Detection Method in Structured Environments

□

A New, Linear Kalman Filter SLAM Formulation

□

Evaluation and Application to Augmented Reality (AR)

Linear RGB-D SLAM (L-SLAM) with a Global Planar Map

For full details, refer to [4]

Pipeline of the Proposed SLAM

□

L-SLAM[4] (Linear SLAM in Planar Environments)

17

Depth Linear SLAM within Kalman Filter RGB

Point Detection & Tracking Point Cloud Line Detection Surface Normals Vanishing Directions Orthogonal Plane Detection Drift-Free Rotation Tracking Translation Estimation

LPVO L-SLAM

Orthogonal Plane Detection

□

The Plane Model in RANSAC[18]

18

Detected Planes Overlaid on the RGB Image

: The Measured Disparity : The Normalized Image Coordinates

𝑣, 𝑤

Linear SLAM Formulation in KF

□

KF State Vector Definition

19

• State Vector in Linear KF
• 3-DoF Camera Translation
• 1-D Distance (Offset) of the Plane
• 3-DoF rotational motion is PERFECTLY compensated by LPVO[2].
• Camera, map position are expressed in global Manhattan map frame.

Linear SLAM Formulation in KF

□

Propagation Step (Predict) with LPVO

20

• Only 3-DoF camera translation is propagated with LPVO method.
• A constant position model is used in 1-D map position (& alignment).

where ,

• Process Model with LPVO

Linear SLAM Formulation in KF

□

Correction Step (Update) with Orthogonal Planes

21

• Observation model is nothing but a distance from the orthogonal plane.
• 1-D map positions are also updated in linear KF framework.

where

• Measurement Model

Evaluation & AR Application Results

□

Author-collected RGB-D Dataset (in SNU Building 301)

22 Video available at https://youtu.be/GO0Q0ZiBiSE

Evaluation & AR Application Results

□

AR Objects Rendering on ICL-NUIM Dataset

23 Video available at https://youtu.be/GO0Q0ZiBiSE

Quantitative Analysis on ICL-NUIM Dataset

24

□

Comparison of the Absolute Translation Error (meter)

lr-kt0n

• f-kt1n
• f-kt2n
• f-kt3n
• L-SLAM presents comparable results compared to other SLAM approaches.
• Estimated (magenta) and true (black) trajectories overlap significantly.

Pyojin Kim, Postdoctoral Fellow @ SFU Email: pjinkim1215@gmail.com Website: http://pyojinkim.me/ (Paper, Video, Code, etc.) Affiliation: GrUVi Lab. School of Computing Science Simon Fraser University (SFU), Burnaby, BC, Canada

Thank You for Your Time!  If there are some more questions…

Reference

1. Zhang, Ji, Michael Kaess, and Sanjiv Singh. "A real-time method for depth enhanced visual odometry." AURO 2017. 2. Kim, Pyojin, Brian Coltin, and H. Jin Kim. "Low-drift visual odometry in structured environments by decoupling rotational and translational motion." IEEE ICRA 2018. 3. Kim, Pyojin, Brian Coltin, and H. Jin Kim. "Indoor RGB-D Compass from a Single Line and Plane." IEEE CVPR 2018. 4. Kim, Pyojin, Brian Coltin, and H. Jin Kim. "Linear RGB-D SLAM for planar environments.“ ECCV 2018. 5. Kaess, Michael, Kai Ni, and Frank Dellaert. "Flow separation for fast and robust stereo odometry." IEEE ICRA 2009. 6. Cvišić, Igor, and Ivan Petrović. "Stereo odometry based on careful feature selection and tracking.“ IEEE ECMR 2015. 7. Bazin, Jean Charles, et al. "Motion estimation by decoupling rotation and translation in catadioptric vision.“ CVIU 2010. 8. Bazin, Jean-Charles, and Marc Pollefeys. "3-line ransac for orthogonal vanishing point detection." IEEE IROS 2012. 9. Zhou, Yi, et al. "Divide and conquer: Efficient density-based tracking of 3D sensors in Manhattan worlds." ACCV 2016. 10. Straub, Julian, et al. "The manhattan frame model—manhattan world inference in the space of surface normals." IEEE TPAMI 2018. 11. Mur-Artal, Raul, and Juan D. Tardós. "ORB-SLAM2: an open-source slam system for monocular, stereo, and RGB-D cameras." IEEE TRO 2017. 12. Kerl, Christian, Jürgen Sturm, and Daniel Cremers. "Robust odometry estimation for RGB-D cameras." IEEE ICRA 2013. 13. Kim, Pyojin, Brian Coltin, and H. Jin Kim. "Visual odometry with drift-free rotation estimation using indoor scene regularities." BMVC 2017. 14. Carlone, Luca, et al. "Initialization techniques for 3D SLAM: a survey on rotation estimation and its use in pose graph

• ptimization.“ IEEE ICRA 2015.

15. Hsiao, Ming, et al. "Keyframe-based dense planar SLAM." IEEE ICRA 2017. 16. Le, Phi-Hung, and Jana Košecka. "Dense piecewise planar RGB-D SLAM for indoor environments." IEEE IROS 2017. 17. Ma, Lingni, et al. "CPA-SLAM: Consistent plane-model alignment for direct RGB-D SLAM.“ IEEE ICRA 2016. 18. Taylor, Camillo J., and Anthony Cowley. "Parsing indoor scenes using rgb-d imagery." RSS 2013.