SLAM@NVIDIA Kari Pulli| Senior Director of Research Overview - - PowerPoint PPT Presentation

▶
slam nvidia
SMART_READER_LITE
LIVE PREVIEW

SLAM@NVIDIA Kari Pulli| Senior Director of Research Overview - - PowerPoint PPT Presentation

Mar 31, 2023 364 likes •727 views

SLAM@NVIDIA Kari Pulli| Senior Director of Research Overview Keyframe-based SlAM 3D rendering for Augmented Reality Problems with traditional keyframe-based SLAM Solution: Deferred Triangulation SLAM KeyFrame-based SLAM 3D

slide-1
SLIDE 1

SLAM@NVIDIA

Kari Pulli| Senior Director of Research

slide-2
SLIDE 2

Overview

  • Keyframe-based SlAM

3D rendering for Augmented Reality Problems with traditional keyframe-based SLAM Solution: Deferred Triangulation SLAM

slide-3
SLIDE 3

KeyFrame-based SLAM

3D Mapping

[Bundle Adjustment]

2D Tracking

[Optical Flow]

3D Tracking

[Pose Estimation]

Stereo Initialization

[Triangulation]

Rendering

[Overlaying (AR)] [Scene Reconstruction]

Optical flow Stereo triangulation 3D Tracking and pose estimation

BA

Incremental mapping and camera pose refinement

Time  Tracking Mapping Rendering

Rendering objects with the camera poses and geometry (map)

Bundle Adjustment

Adding Keyframes, data association, and recovery

BA BA BA BA

slide-4
SLIDE 4

Tracking example

slide-5
SLIDE 5
slide-6
SLIDE 6
slide-7
SLIDE 7
slide-8
SLIDE 8
slide-9
SLIDE 9
slide-10
SLIDE 10
slide-11
SLIDE 11
slide-12
SLIDE 12

DTAM

slide-13
SLIDE 13
slide-14
SLIDE 14

We have done Kinectfusion-type of processing using SoftKinetic range scanners, the quality and framerate of the depth is better than on Tango

slide-15
SLIDE 15
slide-16
SLIDE 16
slide-17
SLIDE 17
slide-18
SLIDE 18
slide-19
SLIDE 19

How to deal with the rotation?

ISMAR 2013 ISMAR 2012

slide-20
SLIDE 20

This is how

3DV 2014

slide-21
SLIDE 21

How to deal with the rotation?

Deferred triangulation

0.5x Speed for visualization Deferred 2D points Triangulated 3D points

slide-22
SLIDE 22

How to deal with the rotation?

Deferred triangulation Jointly (2D/3D) constrain a pose

0.5x Speed for visualization Deferred 2D points Triangulated 3D points

slide-23
SLIDE 23

How to overcome the rotation?

Deferred triangulation Jointly (2D/3D) constrain a pose Region merging

slide-24
SLIDE 24

Pose estimation

slide-25
SLIDE 25

Epipolar segment

slide-26
SLIDE 26

Epipolar segment

slide-27
SLIDE 27

Pose estimation

slide-28
SLIDE 28

Bundle Adjustment

slide-29
SLIDE 29
slide-30
SLIDE 30
slide-31
SLIDE 31

Quantitative evaluation

slide-32
SLIDE 32
slide-33
SLIDE 33
slide-34
SLIDE 34
slide-35
SLIDE 35

Summary

  • Keyframe-based SLAM is efficient

and can run in real time on mobile devices

But it has problems

A separate initialization phase is annoying Breaking with pure rotations is a critical failure

Both can be addressed by

tracking first in 2D deferring triangulation until there is enough baseline between the keyframes

Bonus: we plan to open source the implementation