Some Very, Very Basic and/or Old Thoughts on Multimodality and - PowerPoint PPT Presentation

Some Very, Very Basic and/or Old Thoughts on Multimodality and Uncertainty Oliver Brock Robotics and Biology Laboratory

The Amazon Picking Challenge

The Problem

Possible Reasons for Winning… ► Luck ► Bigger team ► Graduate students versus undergraduates ► Cheating ► Embodiment ► Vertical integration (behavior) ► Better boxes: Factorization ► Embodiment ► Biases / priors / heuristics ► Project management ► Really smart people

Embodiment Choices

Boxes

VISION

Prior Knowledge for Segmentation

Projecting to 1D Color (Hue + B/W) Hue + white/gray/black

Probabilistic Segmentation P(Color | Object) Backprojection Bayes' rule

More Features! Color Height (3D) Edges Height (2D) Combine Segment 3D missing Distance to shelf

APC Segmentation Results

Importance of Each Feature

James J. Gibson (1904 – 1979) Eleanor J. Gibson (1910 – 2002) 1966

Prog. by Demo Prog. By Demo Kinematics Surveillance Activity Objects Manipulation

“Learning to See”

Experimental Design 576-dimensional observation 10/10/2015

Training (~ 50 minutes)

Test of Learned Behavior

What the Robot Sees and How It Learns

Learning to See robotics prior state reinforcement learning action 10/10/2015 32 Rico Jonschkowski

Learning to Perceive

Learning to Perceive

Moving in An Environment With Disturbances

Moving in a “Cluttered” Environment (Robot’s View)

Performance

Different Learned Representations

How Do We Do It? Rico Jonschkowski William of Ockham Sir Isaac Newton 1287 – 1347 1642 – 1727 1987 –

Five Robotic Priors Simplicity Only a small number of world properties are relevant. Task-relevant properties of the world change gradually. The task-relevant properties together with the action determine the reward. The amount of change in task-relevant properties resulting from an action is proportional to the magnitude of the action. The task-relevant properties and the action together determine the resulting change in these properties.

Functional Relationships between DOF

Online IP: Three Recursive Estimation Problems spatial coherency rigid body physics kinematics feature motion rigid body motion kinematic model measurement input input to process model

Visual Odometry and Online IP

spatial coherency rigid body physics kinematics feature motion rigid body motion kinematic model rigid body motion shape reconstruction (shape) rigid body physics shape appearance

Effect of Integrated Tracking on Reconstruction Integrated Tracking Non-integrated Tracking

Robotic Senses Considered as a Perceptual System PRIOR PRIOR PRIOR ESTIMATION ESTIMATION ESTIMATION PRIOR PRIOR PRIOR ESTIMATION ESTIMATION ESTIMATION PRIOR PRIOR PRIOR ESTIMATION ESTIMATION ESTIMATION

Layers of the Cortex / Connectivity of Visual Cortex

Uncertainty: the Ruler of Perception

Roberto Martín Martín Sebastian Höfer

Our Options Against Uncertainty ► Ignore it! Assume you know everything! (Ignore this.) ► Model it and then go back to the above ► Let the physics of interaction deal with it ► Stay in regions of the space (what space?) where uncertainty is not (that) relevant to task success

Modeling it and then Go Back to the Above Assumption: Only the world is a truthful and complete model of itself.

Explicit Reasoning about Uncertainty cost of modeling benefit/cost and computation High Low no uncertainty as much uncertainty as possible

Let the Physics of Interaction Deal with It

Pisa/IIT Soft Hand

How to Identify Regions of Robust Interaction? This slides was intentionally left blank.

Clemens Eppner Raphael Deimel Jessica Abele

Conclusion How? What? PRIOR PRIOR PRIOR ESTIMATION ESTIMATION ESTIMATION Don’t try to model the world! PRIOR PRIOR PRIOR Instead, try to model robust, ESTIMATION ESTIMATION ESTIMATION task-relevant correlations in action and perception space. PRIOR PRIOR PRIOR ESTIMATION ESTIMATION ESTIMATION INTERACTIVE PERCEPTION