Robot attentional models for intuitive HRI. Verena Vanessa Hafner ! - - PowerPoint PPT Presentation

▶
robot attentional models for intuitive hri
SMART_READER_LITE
LIVE PREVIEW

Robot attentional models for intuitive HRI. Verena Vanessa Hafner ! - - PowerPoint PPT Presentation

Apr 11, 2024 123 likes •551 views

Robot attentional models for intuitive HRI. Verena Vanessa Hafner ! Kognitive Robotik, Institut fr Informatik, Humboldt-Universitt zu Berlin ! WS Attention Models in Robotics: Visual Systems for Better HRI, HRI 2014, March 3, 2014 Overview

slide-1
SLIDE 1

Robot attentional models for intuitive HRI.

Verena Vanessa Hafner! Kognitive Robotik, Institut für Informatik, Humboldt-Universität zu Berlin! WS Attention Models in Robotics: Visual Systems for Better HRI, HRI 2014, March 3, 2014

slide-2
SLIDE 2

Overview

  • Introduction & Motivation!
  • Joint Attention!
  • Robot Attentional Models!
  • Summary & Discussion

"2

slide-3
SLIDE 3

Overview

  • Introduction & Motivation!
  • Joint Attention!
  • Robot Attentional Models!
  • Summary & Discussion

"3

slide-4
SLIDE 4

Embodied Cognition

embodiment hypothesis ! intelligence emerges from the interaction of an agent with an environment and as a result

  • f sensorimotor activity.

"4

“Lara, 9 Monate, verschmiert Karottenbrei” (Foto: Peez, idw)

slide-5
SLIDE 5

Visual & auditory attention

EU project EARS on Embodied Audition for RobotS, 2014-2017, FP7 STREP

"5

slide-6
SLIDE 6

Overview

  • Introduction & Motivation!
  • Joint Attention!
  • Robot Attentional Models!
  • Summary & Discussion

"6

slide-7
SLIDE 7

Joint Attention

  • Strong interest in the robotics

community (HRI & devrob)!

  • Joint Attention skills are

important for:


  • Imitation

  • Social Cognition

  • Development of Language

  • Intuitive Interaction

"7 Kaplan, F. and Hafner, V. V. (2006), The Challenges of Joint Attention, Interaction Studies, 7:2, pp. 135-169!

slide-8
SLIDE 8

First Approaches

  • Gaze detection between a robot and a human


(Nagai et al. 2002+2003, Scassellati,1999, Carlson and Triesch, 2003)!

  • Pointing and gaze detection between a robot and

a human (Imai et al. 2001, Kozima et al. 2000)!

  • Pointing detection between two robots


(Hafner, Kaplan 2005)

"8

slide-9
SLIDE 9

What is Attention?

Process whereby an agent concentrates on some features of the environment to the (relative) exclusion of others.


  • Passive attention: a salient event automatically

triggers the attention of the agent.!

  • Active attention: the agent is involved in an

intentionally directed process and must actively select particular features of its environment.

slide-10
SLIDE 10

Attention by Sound Cue /
 Attention by Visual Cue

!10

slide-11
SLIDE 11

Defining Joint Attention:
 What Joint Attention is NOT

!11

Joint Attention is more than simultaneous looking !

slide-12
SLIDE 12

!12

Two robots look at the same things but do not share attention

slide-13
SLIDE 13

Defining 
 Joint Attention

!13

  • 1. Joint Attention is more than simultaneous looking
  • 2. Joint Attention is more than attention detection,

attention manipulation and social coordination!

  • 3. Joint Attention is mainly about intentional

understanding

slide-14
SLIDE 14

Prerequisites

  • f Joint

Attention

!14

  • Attention Detection!
  • Attention Manipulation!
  • Social Coordination!
  • Intentional Understanding
slide-15
SLIDE 15

!15

T3 Social coordination

0-3m Protoconversation, simple rhythmic interaction including turn-taking mediated by the caregiver 6m Shared games, conventional routines established between child and caregivers 9m Simple immediate imitation 18m Complex imitative games

T1 Attention detection

0-3m Mutual gaze 6m Discrimination of left/right 12m Gaze angle detection, interpretation of pointing 15m Gaze following and pointing detection toward object outside the field of view

T2 Attention manipulation

9m Imperative pointing as a request for reaching an object 12m Declarative pointing, attention manipulation using gestures 13m Referential words

T4 Intentional understanding

0-3m Early identification with other persons 6m Distinction between animate and inanimate entities 9m First goal-directed behaviour 12m Behavioural understanding of observed behaviour, intentional understanding of produced behaviour 18m Intentional understanding of observed behaviour

Human Developmental Timelines

slide-16
SLIDE 16

Pointing in Human Infants

!16 Imperative pointing (9 months)
 ! 
 ! Drawing attention as a request for reaching an object, 
 ! attention not monitored,! !

  • rigin: grasping?


! !


 Declarative pointing (12 months)
 ! 
 ! Drawing attention using gestures

slide-17
SLIDE 17

Interaction Game

!17

slide-18
SLIDE 18

However, intentional understanding 
 is still difficult

!18 [shoot-shorter.mov] [run1.mpg] [run2.mpg]

slide-19
SLIDE 19

(Joint) Attention in HRI

"19

slide-20
SLIDE 20

Overview

  • Introduction & Motivation!
  • Joint Attention!
  • Robot Attentional Models!
  • Summary & Discussion

"20

slide-21
SLIDE 21

Visual Attention and Attention Manipulation

  • n

"21 Schillaci,(G.,(Bodiroža,(S.(and(Hafner,(V.V.((2013),(Evaluating(the(Effect(of(Saliency(Detection(and(Attention(Manipulation( in(HumanERobot(Interaction,(International*Journal*of*Social*Robotics,*Springer,*Volume*5,*Issue*1*(2013),*pages*139@152,* OPEN*ACCESS.* Bodiroža,(S.,(Schillaci,(G.(and(Hafner,(V.V.((2011),(Robot(EgoEsphere:(An(Approach(for(Saliency(Detection(and(Attention( Manipulation(in(Humanoid(Robots(for(Intuitive(Interaction,(Proceedings*of*the*11th*IEEE@RAS*International*Conference*on* Humanoid*Robots*(Humanoids*2011),*pp.*689–694,*Bled,*Slovenia.*

slide-22
SLIDE 22

Robot Ego-Sphere

  • Saliency detection!
  • Multi-modal salient ego-sphere!
  • (tesselated sphere)

"22 J.(Ruesch,(M.(Lopes,(A.(Bernardino,(J.(Hornstein,(J.(SantosEVictor,(and(R.(Pfeifer,(“Multimodal(saliencyEbased(bottomEup(attention( a(framework(for(the(humanoid(robot(iCub,”(in(Proceedings*of*the*IEEE*International*Conference*on*Robotics*and*Automation* 2008*(ICRA*2008),(2008,(pp.(962–967.

slide-23
SLIDE 23
  • habituation, inhibition and forgetting!
  • different for motion and face detection!
  • saliency decays over time

Ego-sphere as a 
 short-term memory

"23

  • R. A. Peters, K. E. Hambuchen, K. Kawamura, and D. M. Wilkes, “The sensory ego-sphere as a short-term memory for

humanoids,” in Proceedings of the IEEE-RAS Conference on Humanoid Robots, 2001, pp. 451–460.

slide-24
SLIDE 24

Robot Attentional System

"24

slide-25
SLIDE 25

Robot Attention Manipulation

pointing

"25

slide-26
SLIDE 26

Experiments on Motor Babbling

Schillaci, G. and Hafner, V.V. (2011), Random Movement Strategies in Self-Exploration for a Humanoid Robot, Proceedings of the 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI 2011), pp. 245-246, Lausanne, Switzerland.

exploration strategies

"26

slide-27
SLIDE 27

Motor Babbling

"27

slide-28
SLIDE 28

Internal Models

!

(controller) (predictor)

"28 Schillaci, G., Hafner, V.V., Lara, B. (2012), Coupled Inverse-Forward Models for Action Execution Leading to Tool-Use in a Humanoid Robot, Proceedings of the 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI 2012), pp. 231-232, Boston, USA.

slide-29
SLIDE 29

Motor Babbling

"29

action spaces on a Nao robot

slide-30
SLIDE 30

Could pointing 
 emerge from grasping?

Hafner, V.V. and Schillaci, G. (2011), From field of view to field of reach - could pointing emerge from the development of grasping? Frontiers in Computational Neuroscience, Conference Abstract: IEEE ICDL-EPIROB 2011. "30

slide-31
SLIDE 31

Resulting Behaviour

  • bject outside the field of grasp

"31

slide-32
SLIDE 32

HRI! Robot Behaviours

  • Exploration!
  • Interaction!
  • Interaction Avoidance!
  • Full Interaction (combination)

"32

slide-33
SLIDE 33

"33

slide-34
SLIDE 34

Evaluation

  • Questionnaires (Godspeed) (N = 28)


Anthropomorphism, Animacy, Likeability, Perceived Intelligence, Perceived Safety, User Satisfaction!

  • Confirmed reliability and internal consistency – all

questionnaires have high Cronbach's alpha (α > 0.7)!

  • Proxemics

"34 C.Bartneck,(E.Croft,(and(D.Kulic,("Measurement(Instruments(for(the(Anthropomorphism,(Animacy,(Likeability,(Perceived( Intelligence,(and(Perceived(Safety(of(Robots,"(International*Journal*of*Social*Robots,*vol.*1,*2009,*pp.*71@81.

slide-35
SLIDE 35

Proxemics

  • personal spheres!
  • boundaries can be identified by factors like

gender, age and culture!

  • 4 spheres: Intimate Distance (0 to 45cm)


Personal Distance (45 to 120 cm)
 Social Distance (1.2 to 3.6 m)
 Public Distance (more than 3.6 m)

"35

slide-36
SLIDE 36

"36

Perceived Intelligence Likeability Animacy User Satisfaction Perceived Safety Anthropomorphism

slide-37
SLIDE 37

Study Results I

  • Positive correlation between anthropomorphic

attributes and perceived intelligence (expectations not taken into account)!

  • Interactiveness (exhibited with attentive

mechanisms) positively correlated with excitement, lifelikeness and intelligence!

  • Multi-modal interaction (interaction and full

interaction) increased the level of interactiveness

"43

slide-38
SLIDE 38

Study Results II

  • Negative correlation between: 

  • likeness and kindness, and variance of the

face-face distance


  • satisfaction and variance of the face-hand

distance!

  • Variance is higher during the interaction

avoidance than during the other behaviors

"44

slide-39
SLIDE 39

Overview

  • Introduction & Motivation!
  • Joint Attention!
  • Robot Attentional Models!
  • Summary & Discussion

"45

slide-40
SLIDE 40

Summary I

  • Intuitive HRI needs joint attention!
  • identified prerequisites!
  • attention manipulation through pointing!
  • attentional model based on saliency maps

& robot ego-sphere!

  • setup: human-robot interaction game

"46

slide-41
SLIDE 41

Summary II

  • different levels of interactiveness of the

robot!

  • pos. correlated with user experience

factors like excitement and robot factors like lifelikeness and intelligence!

  • robot feedback important for intuitive

interaction

"47

slide-42
SLIDE 42

Acknowledgements

Contributions: 
 all members and partners of the 
 Cognitive Robotics group at HU Berlin!

!

http://koro.informatik.hu-berlin.de

"48