The Curious Robot: Learning Visual Representa6ons via Physical - - PowerPoint PPT Presentation

▶
the curious robot learning visual representa6ons via
SMART_READER_LITE
LIVE PREVIEW

The Curious Robot: Learning Visual Representa6ons via Physical - - PowerPoint PPT Presentation

Nov 18, 2022 306 likes •534 views

The Curious Robot: Learning Visual Representa6ons via Physical Interac6ons Lerrel Pinto, Dhiraj Gandhi, Yuanfeng Han, Yong-Lae Park, Abhinav Gupta ECCV 2016 Presenter: Ginevra Gaudioso R B Problem Learning visual representa-ons of objects

slide-1
SLIDE 1

The Curious Robot: Learning Visual Representa6ons via Physical Interac6ons

Lerrel Pinto, Dhiraj Gandhi, Yuanfeng Han, Yong-Lae Park, Abhinav Gupta ECCV 2016 Presenter: Ginevra Gaudioso

R B

slide-2
SLIDE 2

Problem

  • Learning visual representa-ons of objects
  • By ac-vely interac-ng with the objects

Image source: hOps:// youtu.be/oSqHc0nLkm8?t=49

slide-3
SLIDE 3

Why do we care?

  • Robo6cs: robot needs to recognize objects
  • Vision: classifica6on

Image source: hOps:// youtu.be/oSqHc0nLkm8?t=47

slide-4
SLIDE 4

Related Work

  • Unsupervised Learning

– Other approaches use passive data – Here the robot ac6vely plays with the objects

  • Robo6c Tasks

– In robo6cs, we use vision to plan the best grasp – Here we use grasp data to classify the object

slide-5
SLIDE 5

Approach

Image source: www.robo6csbusinessreview.com

Experiments:

  • Grasping
  • Pushing
  • Poking
  • View at different angles
  • Use robot’s experiment results to label objects
  • Train Network to predict experiment results given

the picture of the object as input

slide-6
SLIDE 6

Grasp

Image source: paper

slide-7
SLIDE 7

Grasp

Image source: paper

slide-8
SLIDE 8

Push

Image source: paper

slide-9
SLIDE 9

Push

Image source: paper

slide-10
SLIDE 10

Poke

Image source: paper

slide-11
SLIDE 11

Poke

Image source: paper

slide-12
SLIDE 12

Complete Network

slide-13
SLIDE 13

Complete Network

Image source: paper

slide-14
SLIDE 14

Complete Network

Image source: paper

slide-15
SLIDE 15

Complete Network

Image source: paper

slide-16
SLIDE 16

Experiments – Root Network

Which images generate similar ac6va6on paOerns in the Root network?

slide-17
SLIDE 17

Experiments – Root Network

Network learns high level features of objects, such as shape.

Image source: paper

slide-18
SLIDE 18

Experiments – Image Retrieval

Recall@k : this approach leads to good retrieval levels

Image source: paper

slide-19
SLIDE 19

Experiments – Image Retrieval

Nearest Neighbors relies mostly on shape

Image source: paper

slide-20
SLIDE 20

Discussion

  • Strengths

– Robot learns all by itself – Able to learn meaningful features (shape) – Very good retrieval results

  • Weaknesses

– Physical interac6on is expensive – Network is heavily handcraged – Limited to objects physically available in training

slide-21
SLIDE 21

Extensions

  • Allow for passive data to enrich dataset
  • Ac6vely choose which training data to gather
  • Would this work with simpler network?