persistent self supervised learning principle from stereo
play

Persistent self-supervised learning principle: from stereo to - PowerPoint PPT Presentation

Dec 30, 2023 •359 likes •503 views

Persistent self-supervised learning principle: from stereo to monocular vision for obstacle avoidance by Kevin van Hecke, Guido de Croon, Laurens van der Maaten, Daniel Hennes, and Dario Izzo presented by Mike Kalaitzakis What is Persistent

  1. Persistent self-supervised learning principle: from stereo to monocular vision for obstacle avoidance by Kevin van Hecke, Guido de Croon, Laurens van der Maaten, Daniel Hennes, and Dario Izzo presented by Mike Kalaitzakis

  2. What is Persistent Self-Supervised Learning? ● What is Self- Supervised Learning ● What makes this persistent ● Differences and Similarities between other ML methods ● What are the advantages https://xkcd.com/1838/

  3. What is Persistent Self-Supervised Learning? ● Self-Supervised Learning is a mechanism that uses a trusted sensor cue for training to recognize a complementary sensor cue

  4. What is Persistent Self-Supervised Learning? ● Persistent Self- Supervised Learning is a Self- Supervised Learning method where the goal is to be able to replace the original trusted sensor cue if necessary

  5. What is Persistent Self-Supervised Learning? ● Shares some similarities with Unsupervised Learning (UL) and Learning from Demonstration (LfD) ● Does not need labeled data like UL ● Does not need an external “teacher” as LfD ● Does not need a reward function and many trial-and-error sessions as Reinforcement Learning ● Can autonomously decide which sensor cue to use or if re-training is needed

  6. Going from Stereo Vision to Monocular Vision ● The disparity is the difference between the same image feature in the two images of a calibrated stereo pair ● It is used to calculate the distance of an object from the cameras Stefano Mattoccia, University of Bolona ● A 4-gram custom made stereo camera is used (128x96 pixel, 10fps) ● The PSSL tries to estimate the average disparity and not the whole map TU Delft, MAV-Lab

  7. Visual Bag of Words (VBoW) and Textons ● In VboW a complex image is split in small image patches ● A dictionary is created by clustering the patches called textons ● When processing an image each patch is compared to the dictionary creating a texton occurrence histogram ● Both intensity and gradient textons are used

  8. Why Visual Bag of Words? ● Short Answer: Computationally efficient and limited computational power platform (Parrot AR Drone 2, 1GHz ARM cortex A8, 128MB RAM) ● Compared to the best available CNN the results were comparable and depended on the dataset

  9. System Overview ● Binary classification ● Determine the threshold (t λ ) from the ROC curve ● The threshold is chosen to minimize the probability of collision ● Since the data cannot be assumed identically and independently distributed a Markov process was used to model the system ● PSSL limitation factor: By design the system will encounter very few occasions where obstacles are detected

  10. System Overview

  11. Simulation Results ● SmartUAV simulation environment was used to choose regression method and learning scheme ● kNN, Linear regression, Gaussian process and Neural Networks were tested ● “Cold Turkey”, DAgger and “Training wheels” approaches to learning were tested

  12. Experiment Results

  13. ISS Video

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Depth from Stereo Dominic Cheng February 7, 2018 Agenda 1. Introduction to stereo 2.

Depth from Stereo Dominic Cheng February 7, 2018 Agenda 1. Introduction to stereo 2.

Depth from Stereo Dominic Cheng February 7, 2018 Agenda 1. Introduction to stereo 2. Efficient Deep Learning for Stereo Matching (W. Luo, A. Schwing, and R. Urtasun. In CVPR 2016.) 3. Cascade Residual Learning: A Two-stage Convolutional

647 views • 30 slides
Learning frameworks Self-supervised learning: (Auto)encoder networks Supervised learning Network

Learning frameworks Self-supervised learning: (Auto)encoder networks Supervised learning Network

Learning frameworks Self-supervised learning: (Auto)encoder networks Supervised learning Network must copy inputs to outputs through Assumes environment specifies correct output (targets) for each input a bottleneck (fewer hidden units)

407 views • 5 slides
Self-Supervised Feature Learning by Learning to Spot Artifacts Wonbin Kim Self-Supervised

Self-Supervised Feature Learning by Learning to Spot Artifacts Wonbin Kim Self-Supervised

Self-Supervised Feature Learning by Learning to Spot Artifacts Wonbin Kim Self-Supervised Learning To exploit different labelings that are freely available besides or within visual data, and to use them as intrinsic reward signals to

387 views • 24 slides
Efficient Deep Learning for Stereo Matching Wenjie Luo, Alex Schwing and Raquel Urtasun W. Luo

Efficient Deep Learning for Stereo Matching Wenjie Luo, Alex Schwing and Raquel Urtasun W. Luo

Efficient Deep Learning for Stereo Matching Wenjie Luo, Alex Schwing and Raquel Urtasun W. Luo et.al. (UofT) Stereo Matching 1 / 6 Stereo Estimation Desired Properties: Good enough to detect obstacles precisely Fast: real time Robust to:

119 views • 8 slides
Stacking for supervised learning Stacking for supervised learning Niall Rooney, NIKEL,

Stacking for supervised learning Stacking for supervised learning Niall Rooney, NIKEL,

Stacking for supervised learning Stacking for supervised learning Niall Rooney, NIKEL, University of Ulster 1 Ensemble learning Ensemble learning l Postulate multiple hypotheses to explain the data l Shortcomings of single model learning

681 views • 19 slides
Machine Learning for NLP Supervised Learning Aurlie Herbelot 2019 Centre for Mind/Brain

Machine Learning for NLP Supervised Learning Aurlie Herbelot 2019 Centre for Mind/Brain

Machine Learning for NLP Supervised Learning Aurlie Herbelot 2019 Centre for Mind/Brain Sciences University of Trento 1 Supervised learning Supervised: you know the result of the task you want to perform. Supervised learning mostly

832 views • 61 slides
PCA CS 446 Supervised learning So far, weve done supervised learning: Given (( x i , y i )) ,

PCA CS 446 Supervised learning So far, weve done supervised learning: Given (( x i , y i )) ,

PCA CS 446 Supervised learning So far, weve done supervised learning: Given (( x i , y i )) , find f with f ( x i ) y i . k -nn, decision trees, . . . 1 / 18 Supervised learning So far, weve done supervised learning: Given (( x i , y

1.01k views • 56 slides
Today Recap: epipolar constraint Stereo image rectification Stereo: Stereo

Today Recap: epipolar constraint Stereo image rectification Stereo: Stereo

3/29/2011 Today Recap: epipolar constraint Stereo image rectification Stereo: Stereo solutions Correspondence and Calibration Computing correspondences Non-geometric stereo constraints Mon, March 28 Calibration

342 views • 13 slides
3D Vision: Stereo Marc Pollefeys, Torsten Sattler Spring 2016

3D Vision: Stereo Marc Pollefeys, Torsten Sattler Spring 2016

3D Vision: Stereo Marc Pollefeys, Torsten Sattler Spring 2016 http://cvg.ethz.ch/teaching/3dvision/ Dense Correspondence / Stereo Tsukuba dataset http://cat.middlebury.edu/stereo/ Stereo Standard stereo geometry Stereo matching

478 views • 35 slides
Semi-Supervised Learning Maria-Florina Balcan 03/30/2015 Readings: Semi-Supervised Learning.

Semi-Supervised Learning Maria-Florina Balcan 03/30/2015 Readings: Semi-Supervised Learning.

Semi-Supervised Learning Maria-Florina Balcan 03/30/2015 Readings: Semi-Supervised Learning. Encyclopedia of Machine Learning. Jerry Zhu, 2010 Combining Labeled and Unlabeled Data with Co- Training. Avrim Blum, Tom Mitchell. COLT

953 views • 45 slides
RECSM Summer School: Machine Learning for Social Sciences Session 1.3: Supervised Learning and

RECSM Summer School: Machine Learning for Social Sciences Session 1.3: Supervised Learning and

RECSM Summer School: Machine Learning for Social Sciences Session 1.3: Supervised Learning and Model Accuracy Reto West Department of Political Science and International Relations University of Geneva 1 Supervised Learning Supervised

436 views • 42 slides
CSE 158 Lecture 1.5 Web Mining and Recommender Systems Supervised learning Regression

CSE 158 Lecture 1.5 Web Mining and Recommender Systems Supervised learning Regression

CSE 158 Lecture 1.5 Web Mining and Recommender Systems Supervised learning Regression What is supervised learning? Supervised learning is the process of trying to infer from labeled data the underlying function that produced the labels

784 views • 33 slides
CSE 158 Lecture 1.5 Web Mining and Recommender Systems Supervised learning Regression

CSE 158 Lecture 1.5 Web Mining and Recommender Systems Supervised learning Regression

CSE 158 Lecture 1.5 Web Mining and Recommender Systems Supervised learning Regression What is supervised learning? Supervised learning is the process of trying to infer from labeled data the underlying function that produced the labels

751 views • 25 slides
CSE 258 Lecture 1.5 Web Mining and Recommender Systems Supervised learning Regression

CSE 258 Lecture 1.5 Web Mining and Recommender Systems Supervised learning Regression

CSE 258 Lecture 1.5 Web Mining and Recommender Systems Supervised learning Regression What is supervised learning? Supervised learning is the process of trying to infer from labeled data the underlying function that produced the labels

599 views • 33 slides
Machine Learning Lecture 6 Unsupervised Learning with a bit of Supervised Learning Clustering

Machine Learning Lecture 6 Unsupervised Learning with a bit of Supervised Learning Clustering

Machine Learning Lecture 6 Unsupervised Learning with a bit of Supervised Learning Clustering and nearest neighbour classifiers. Justin Pearson 1 2020 1 http://user.it.uu.se/~justin/Teaching/MachineLearning/index.html 1 / 27 Supervised

813 views • 27 slides
1 Basic Stereo Derivations Correspondence It is fundamentally ambiguous, even with stereo

1 Basic Stereo Derivations Correspondence It is fundamentally ambiguous, even with stereo

Stereo Vision Reading: Chapter 11 Stereo matching computes depth from two or more images Subproblems: Calibrating camera positions. Finding all corresponding points (hardest part) Computing depth or surfaces. Slide credits

107 views • 7 slides
Monocular Vision Based Obstacle Avoidance: A Literature Review Outline Introduction

Monocular Vision Based Obstacle Avoidance: A Literature Review Outline Introduction

Monocular Vision Based Obstacle Avoidance: A Literature Review Outline Introduction Problem Relevant Work Monocular Cues Machine Learning Expansion Optical Flow SLAM GPU Processing

455 views • 26 slides
Obstacle Avoidance for Monocular Drones Table of Contents I. Technological context II. Neural

Obstacle Avoidance for Monocular Drones Table of Contents I. Technological context II. Neural

Obstacle Avoidance for Monocular Drones Table of Contents I. Technological context II. Neural Networks for obstacle avoidance i. Deep Reinforcement Learning ii. The two axis of development III. . Depth Net i. FOE Extraction pretraining ii.

509 views • 48 slides
R eal Time Lethbridge Entrepreneurial iGEM Team iGEM World Jamboree November 3 rd , 2013 Our

R eal Time Lethbridge Entrepreneurial iGEM Team iGEM World Jamboree November 3 rd , 2013 Our

B iologically E nhanced A ssay in R eal Time Lethbridge Entrepreneurial iGEM Team iGEM World Jamboree November 3 rd , 2013 Our Team Web Manager Chief Chief Co-Chief Co-Chief Science Officer Science Officer Executive Operational Officer

350 views • 18 slides
Impact as a driver for OA Future of Academic Impact Conference 4 th December 2012 Robert Kiley,

Impact as a driver for OA Future of Academic Impact Conference 4 th December 2012 Robert Kiley,

Impact as a driver for OA Future of Academic Impact Conference 4 th December 2012 Robert Kiley, Wellcome Trust r.kiley@wellcome.ac.uk Overview Brief overview of how Wellcome Trust undertakes evaluation Look at development of OA policy as

363 views • 18 slides
Monocular 3D Reconstruction Using Depth Cues from Planar Range Data Ben Eckart Heather Justice

Monocular 3D Reconstruction Using Depth Cues from Planar Range Data Ben Eckart Heather Justice

Monocular 3D Reconstruction Using Depth Cues from Planar Range Data Ben Eckart Heather Justice Joe Lisee 1 Motivation Eckart, Justice, Lisee 2 Motivation Laser-Level Obstacles Above-Laser Obstacles Eckart, Justice, Lisee 3 Project

231 views • 7 slides
Real-Time Monocular SLAM Andrew Davison Robot Vision Group Department of Computing Imperial

Real-Time Monocular SLAM Andrew Davison Robot Vision Group Department of Computing Imperial

Real-Time Monocular SLAM Andrew Davison Robot Vision Group Department of Computing Imperial College London March 30, 2011 Robot Vision in Real-Time Performance in robot vision is advancing fast . What are the reasons? Continued exponential

525 views • 34 slides
Agribot Sprayer, SLAM, and Robust Navigation Andrey Kurenkov, Troy ONeal, Pavel Komarov

Agribot Sprayer, SLAM, and Robust Navigation Andrey Kurenkov, Troy ONeal, Pavel Komarov

Agribot Sprayer, SLAM, and Robust Navigation Andrey Kurenkov, Troy ONeal, Pavel Komarov Agribot Robot Design 110 watt sprayer with 180 range of motion Problem Statements: 15 gallon tank 1) SLAM (localization, SICK 200 LIDAR mapping,

1.86k views • 30 slides
Deep Incremental Scene Understanding Federico Tombari & Christian Rupprecht Technical

Deep Incremental Scene Understanding Federico Tombari & Christian Rupprecht Technical

Deep Incremental Scene Understanding Federico Tombari & Christian Rupprecht Technical University of Munich, Germany Scene Understanding and SLAM Scene understanding with deep SLAM from RGB-D data allowing learning (typically frame-wise)

365 views • 34 slides

More recommend