Some Very, Very Basic and/or Old Thoughts on Multimodality and - - PowerPoint PPT Presentation

▶
some very very basic and or old thoughts
SMART_READER_LITE
LIVE PREVIEW

Some Very, Very Basic and/or Old Thoughts on Multimodality and - - PowerPoint PPT Presentation

Feb 20, 2023 634 likes •1.21k views

Some Very, Very Basic and/or Old Thoughts on Multimodality and Uncertainty Oliver Brock Robotics and Biology Laboratory The Amazon Picking Challenge The Problem Possible Reasons for Winning Luck Bigger team Graduate students

slide-1
SLIDE 1

Some Very, Very Basic and/or Old Thoughts

  • n Multimodality and Uncertainty

Oliver Brock Robotics and Biology Laboratory

slide-2
SLIDE 2
slide-3
SLIDE 3
slide-4
SLIDE 4
slide-5
SLIDE 5
slide-6
SLIDE 6
slide-7
SLIDE 7
slide-8
SLIDE 8

The Amazon Picking Challenge

slide-9
SLIDE 9
slide-10
SLIDE 10

The Problem

slide-11
SLIDE 11

Possible Reasons for Winning…

► Luck ► Bigger team ► Graduate students versus undergraduates ► Cheating ► Embodiment ► Vertical integration (behavior) ► Better boxes: Factorization ► Embodiment ► Biases / priors / heuristics ► Project management ► Really smart people

slide-12
SLIDE 12

Embodiment Choices

slide-13
SLIDE 13

Boxes

slide-14
SLIDE 14

VISION

slide-15
SLIDE 15

Prior Knowledge for Segmentation

slide-16
SLIDE 16

Hue + white/gray/black

Projecting to 1D Color (Hue + B/W)

slide-17
SLIDE 17

Bayes' rule Backprojection

P(Color | Object)

Probabilistic Segmentation

slide-18
SLIDE 18

Combine Segment

Color Height (3D) Edges Height (2D) 3D missing Distance to shelf

More Features!

slide-19
SLIDE 19

APC Segmentation Results

slide-20
SLIDE 20

Importance of Each Feature

slide-21
SLIDE 21
slide-22
SLIDE 22

James J. Gibson (1904–1979) Eleanor J. Gibson (1910–2002)

1966

slide-23
SLIDE 23

Objects Manipulation Activity Surveillance Kinematics

  • Prog. by Demo
  • Prog. By Demo
slide-24
SLIDE 24

“Learning to See”

slide-25
SLIDE 25
slide-26
SLIDE 26

10/10/2015

Experimental Design

576-dimensional observation

slide-27
SLIDE 27

Training (~ 50 minutes)

slide-28
SLIDE 28

Test of Learned Behavior

slide-29
SLIDE 29

What the Robot Sees and How It Learns

slide-30
SLIDE 30

10/10/2015 32 Rico Jonschkowski

Learning to See

action state

reinforcement learning

robotics prior

slide-31
SLIDE 31

Learning to Perceive

slide-32
SLIDE 32

Learning to Perceive

slide-33
SLIDE 33

Moving in An Environment With Disturbances

slide-34
SLIDE 34

Moving in a “Cluttered” Environment (Robot’s View)

slide-35
SLIDE 35

Performance

slide-36
SLIDE 36
slide-37
SLIDE 37

Different Learned Representations

slide-38
SLIDE 38

How Do We Do It?

William of Ockham 1287–1347 Sir Isaac Newton 1642–1727 Rico Jonschkowski 1987–

slide-39
SLIDE 39

Five Robotic Priors

Simplicity

Only a small number of world properties are relevant. The task-relevant properties and the action together determine the resulting change in these properties. The task-relevant properties together with the action determine the reward. The amount of change in task-relevant properties resulting from an action is proportional to the magnitude of the action. Task-relevant properties of the world change gradually.

slide-40
SLIDE 40

Functional Relationships between DOF

slide-41
SLIDE 41

feature motion

Online IP: Three Recursive Estimation Problems

measurement input input to process model

rigid body physics kinematics

rigid body motion kinematic model

spatial coherency

slide-42
SLIDE 42

Visual Odometry and Online IP

slide-43
SLIDE 43

feature motion

rigid body physics kinematics

rigid body motion kinematic model

spatial coherency

shape reconstruction rigid body motion (shape)

shape appearance rigid body physics

slide-44
SLIDE 44

Effect of Integrated Tracking

  • n Reconstruction

Integrated Tracking Non-integrated Tracking

slide-45
SLIDE 45

Robotic Senses Considered as a Perceptual System

ESTIMATION

PRIOR

ESTIMATION

PRIOR

ESTIMATION

PRIOR

ESTIMATION

PRIOR

ESTIMATION

PRIOR

ESTIMATION

PRIOR

ESTIMATION

PRIOR

ESTIMATION

PRIOR

ESTIMATION

PRIOR

slide-46
SLIDE 46

Layers of the Cortex / Connectivity of Visual Cortex

slide-47
SLIDE 47

Uncertainty: the Ruler of Perception

slide-48
SLIDE 48

Roberto Martín Martín Sebastian Höfer

slide-49
SLIDE 49

Our Options Against Uncertainty

► Ignore it! Assume you know everything! (Ignore this.) ► Model it and then go back to the above ► Let the physics of interaction deal with it ► Stay in regions of the space (what space?) where

uncertainty is not (that) relevant to task success

slide-50
SLIDE 50

Modeling it and then Go Back to the Above

Assumption: Only the world is a truthful and complete model of itself.

slide-51
SLIDE 51

Explicit Reasoning about Uncertainty

no uncertainty as much uncertainty as possible High benefit/cost cost of modeling and computation Low

slide-52
SLIDE 52

Let the Physics of Interaction Deal with It

slide-53
SLIDE 53

Pisa/IIT Soft Hand

slide-54
SLIDE 54

How to Identify Regions of Robust Interaction?

This slides was intentionally left blank.

slide-55
SLIDE 55

Clemens Eppner Jessica Abele Raphael Deimel

slide-56
SLIDE 56

Conclusion

ESTIMATION

PRIOR

ESTIMATION

PRIOR

ESTIMATION

PRIOR

ESTIMATION

PRIOR

ESTIMATION

PRIOR

ESTIMATION

PRIOR

ESTIMATION

PRIOR

ESTIMATION

PRIOR

ESTIMATION

PRIOR

Don’t try to model the world! Instead, try to model robust, task-relevant correlations in action and perception space.

INTERACTIVE PERCEPTION How? What?