Monocular Vision Based Obstacle Avoidance: A Literature Review - - PowerPoint PPT Presentation

Monocular Vision Based Obstacle Avoidance: A Literature Review

Outline

• Introduction
• Problem
• Relevant Work

○ Monocular Cues ○ Machine Learning ○ Expansion ○ Optical Flow ○ SLAM ○ GPU Processing

• Results
• Conclusion
• Q & A

Introduction

• Autonomous UAV operation in

complex environments requires:

○ Navigation ○ Homing ○ Obstacle detection/avoidance

• Detection and avoidance will be

the focus of this presentation

http://www.ros.org/news/robots/uavs/

Obstacle Detection/Avoidance

• What is obstacle detection/avoidance?
• Why is obstacle detection/avoidance

useful?

• The Problem

○ UAV drones are limited by: ■ carrying capacity ■ computational power on board ■ battery ○ Solutions are either heavy, computationally intensive, or energy inefficient

http://www.dronemagazine.it/1013-black-hornet-nano-drone-dellesercito-inglese-video/

Previous Solutions

• MAVs require:

○ Light weight ○ Computationally Inexpensive ○ Energy efficient

• Past solutions include:

○ Sensor Arrays, such as: LiDAR, radar, and sonar ○ Stereo Vision

Monocular Vision

• Use one camera to view the world
• Techniques for monocular vision

○ Monocular cues ○ Perception changes ○ Optical flow

• Less weight and energy

consumption than stereo

https://www.youtube.com/watch?v=V4r2HXGA8jw

Problem with Monocular Vision

• Single camera for vision

○ No inherent depth perception ○ Previous stereovision algorithms don’t apply

• Must rely on clever algorithms to make

up for its shortcoming

Proposed Solutions

• Researchers have attempted to tackle drawbacks of monocular vision
• Categories of research in this talk:

○ Monocular Cues ○ Machine Learning ○ Expansion ○ Optical Flow ○ SLAM ○ GPU Processing

Monocular Cues

• Use cues from scene in

image to estimate depth

• Cue types:

○ Accommodation ○ Motion parallax ○ Size constancy

https://www.pinterest.com/pin/46513808625067354/

Related Works: Monocular Cues

• Michels et al. - 2005

○ Monocular depth cues using a portable laser measuring system

• Ross et al. - 2013

○ Feature extraction in wooden environments for object detection

• Bills et al. - 2011

○ Specialized monocular cue algorithms for different environments

• Wu et al. - 2014

○ Synopsis of different monocular vision techniques

• Croon et al. - 2010

○ Texture variation for obstacle detection

Machine Learning

• Use learning capable computers to:

○ Improve accuracy of a detection and avoidance system ○ Basis of an detection and avoidance system

• Artificial Neural Networks is a

common machine learning technique used

https://www.quora.com/

Related Works: Machine Learning

• Michels et al. - 2005

○ Supervised and reinforcement learning on real and synthetic images for training depth

• Ross et al. - 2013

○ Iterative supervised learning using DAgger and an expert pilot’s movements

• Bills et al. - 2011

○ Classifier for identifying different indoor environments

• Oh et al. - 2004

○ Automation of gain tuning to improve light intensity variations using a neural network

• Smolyanskiy et al. - 2017

○ Two deep neural networks for navigation in nature trails

Expansion

• Use expansion of objects to detect
• bstacles
• Feature points such as SIFT and

SURF are used:

○ Approximate object location and dimension ○ Quickly expanding feature points belong to obstacles

http://www.mdpi.com/1424-8220/17/5/1061/htm

Related Works: Expansion

• Aguilar et al. - 2017

○ Matching features from known objects to determine change in scale

• Mori and Scherer - 2013

○ Scale change of SURF features for time to impact

• Al-Kaff et al. - 2016

○ Expansion of features and area of objects

• Chavez and Gustafson - 2009

○ Feature expansion using SIFT

Optical Flow

• Use movement in consecutive images to

understand environment

• Two versions - dense and sparse
• Dense

○ Calculates vector of displacement for each pixel

• Sparse

○ Calculates vector of displacement for selected features

• Most research for MAVs use sparse OF

http://drstyle.me/honey-bee-optical-flow-landing-airplane-plane/

Related Works: Optical Flow

• Sagar - 2014

○ Use OF on clusters of features to distinguish near and far obstacles

• Wu et al. - 2014

○ Detailed background, uses, and disadvantages of OF

• Oh et al. - 2004

○ OF sensitivity to illumination variation improved by using neural network

• Chavez and Gustafson - 2009

○ Use Lucas-Kanade OF to autonomously navigate a flapping wing MAV indoors

SLAM

• Simultaneous Localization and

Mapping (SLAM)

• Maps environment around UAV and

estimates location within it

○ Valuable information for detection/avoidance ○ Traditionally accomplished by complex sensor or stereo systems

• Monocular SLAM (MonoSLAM)

○ Use one camera to perform SLAM

http://www.asctec.de/en/uav-drone/ascending-technologies/ asctec-researchline/uav-uas-drone-computer-vision-slam/

Related Works: SLAM

• Park et al. - 2011

○ Novel parallel MonoSLAM method using GPU SIFT ○ Required fewer features than comparative methods

• Ha and Sattigeri - 2012

○ Combined image segmentation and MonoSLAM to avoid obstacles ○ Features from MonoSLAM linked to objects in segmented image ○ Simulation used as test bed

GPU Processing

• Improved embedded GPU capabilities

enables:

○ Increased detection/avoidance performance ○ Parallel execution of various detection/avoidance techniques

• These improvements equip

researchers the means to create more robust algorithms and execute algorithms in real-time

https://www.amazon.com/NVIDIA-Jetson-TX1-Development-Kit/dp/B017NWO6LG

Related Works: GPU Processing

• Ready and Taylor - 2007

○ GPU could track 500 features at 70% utilization vs CPU 40 features at 90% utilization

• Park et al. - 2011

○ MonoSLAM with GPU 5 to 9 times faster than CPU implementation

• Smolyanskiy et al. - 2017

○ GPU ran two neural networks and visual odometry to navigate trail and avoid obstacles

Results

• Most research discussed has focused on feature detection
• However:

○ Computation time is high ○ Usually offloaded to ground station for computing ○ Relatively small number of features

• GPUs could be used to address these issues:

○ Provide computational speed up for parallelizable algorithms ○ Embedded GPU’s can be used to do onboard computing ○ Increase in tracking ability of features

Our Research

• Assumption: Closer objects expand faster than farther away objects
• The metric used:

○ area of the features ○ area of convex hull produced from feature points

• Improvement of feature detection using GPUs

Detection Algorithm

1. Collect and match features between consecutive frames 2. Keep expanding features 3. Cluster neighboring features 4. Construct convex hulls from clusters 5. Keep expanding convex hulls 6. Issue avoidance command, if required

Avoidance Algorithm

1. Draw bounding rectangles around convex hulls 2. Determine quadrants the rectangles occupy 3. Determine path of avoidance

a. Calculate inverse vector sum of rectangles b. Navigate towards inverse vector sum

Conclusion

• Introduction to Monocular Vision
• Highlighted various approaches to using Monocular Vision
• Discussed our research
• Walk-through of our algorithms for detection and avoidance

Questions?

http://clipartall.com/clipart/11375-clipart-question.html