From Robotic Systems to Robotics-grounded AI Systems Maxim - - PowerPoint PPT Presentation

From Robotic Systems to Robotics-grounded AI Systems

Maxim Likhachev Robotics Institute & NREC Carnegie Mellon University

Maxim Likhachev

Transitioning Robotics Systems into the Real World

Lots of progress in the last decade, with various robotics systems being transitioned into the real world…

Carnegie Mellon University 2

joint work with Stentz, Kelly, etc. at NREC Urban Challenge’07 CMU efforts joint project with Lockheed Martin

Maxim Likhachev

Transitioning Robotics Systems into the Real World

Lots of progress in the last decade, with various robotics systems being transitioned into the real world…

Carnegie Mellon University 3

joint work with Stentz, Kelly, etc. at NREC Urban Challenge’07 CMU efforts joint project with Lockheed Martin

All are well-done Robotics Systems, with no real AI in them! All are well-done Robotics Systems, with no real AI in them!

Maxim Likhachev

Typical Robotics Architecture

Carnegie Mellon University 4

Execution Monitor

Maxim Likhachev

Typical Robotics Architecture

Carnegie Mellon University 5

Execution Monitor

All modules are engineered for expected conditions and fail under unexpected conditions All modules are engineered for expected conditions and fail under unexpected conditions

Maxim Likhachev

Typical Robotics Architecture

Carnegie Mellon University 6

Execution Monitor

All modules are engineered for expected conditions and fail under unexpected conditions All modules are engineered for expected conditions and fail under unexpected conditions Robotics-grounded AI: Need to research how to introduce a deeper understanding of the world and the robot itself Robotics-grounded AI: Need to research how to introduce a deeper understanding of the world and the robot itself In Planning/Decision-making: need better ways to represent knowledge about the world and the robot itself, need to learn from experience and demonstrations, need to reason about its own capabilities, … In Planning/Decision-making: need better ways to represent knowledge about the world and the robot itself, need to learn from experience and demonstrations, need to reason about its own capabilities, …

Maxim Likhachev

Example I: Planning for Manipulation

Planning Representations Have to Be Dynamic

• we can’t model all potentially relevant factors in the real-world
• planner needs to figure out what, when and how it should model
• introduce “necessary” dimensions as part of planning
• learn relevant dimensions based on experience
• learn how to model environment from demonstrations

Carnegie Mellon University 7

Experience Graphs: planning with learning from experience and demonstrations [Phillips et al., RSS’13, ’12]

Maxim Likhachev

Example II: Planning for Aerial Vehicles

Planning Needs to Be Reason about the Models of Other Modules

• perception is brittle, controls are imperfect, localization is noisy,…
• planner needs to reason about strengths/weaknesses of these modules

Carnegie Mellon University 8

State Lattice with Controllers [Butzke et al., IROS’14]

Maxim Likhachev

Challenge Problem

General Purpose Mobile Manipulators for Manufacturing

• semi-structured environments but still plenty of uncertainty
• tasks require modeling object behavior and often dynamics
• potentially high impact on economy
• many aspects can be simulated
• common open software infrastructure exists, thanks to ROS!

Carnegie Mellon University 9