A Cognitive Approach to Robot Self-Consciousness Antonio Chella and - - PowerPoint PPT Presentation

A Cognitive Approach to Robot Self-Consciousness

Antonio Chella and Salvatore Gaglio University of Palermo

Consciousness and Artificial Intelligence: Theoretical foundations and current approaches AAAI Symposium, Washington DC, 8-11 November 2007

• One of the major problems towards effective

robotic architectures is to give a robot the capabilities of introspection, i.e., to reflect about itself, its own perceptions and actions during its operating life.

• The robot introspection grows up from the

content of the agent perceptions, recalls, actions, reflections and so on in a coherent life long narrative.

McCarthy

• What am I doing?
• What is my goal?
• Observe physical body
• Do I know that proposition?
• Do I know what thing is?
• Did I ever do action? When and precisely

what?

• What is currently happening?
• What is the state of the actions I am

currently performing?

• What are my intentions?
• Did I ever do action? When and precisely

what?

• What does my belief in p depend on?
• What are my choices for action?
• Can I achieve possible-goal?
• Does my mental state up to now have

property p?

• How can I plan my thinking on this

problem?

Sloman

• We propose a model of robot introspection

based on higher order perceptions of the robot during time.

• First order robot perceptions are the

immediate perceptions of the outer world of a self reflective agent.

• Higher order perceptions are the

perceptions during time of the inner world of the agent.

• Higher order perception are at the basis of

the introspctive reasoning of the robot

The Cognitive Architecture

Chella, A.; Frixione, M.; and Gaglio, S. (1997). A cognitive architecture for artificial vision. Artificial Intelligence 89:73–111.

The Subconceptual Area

• Low level processing of data coming from

sensors.

• Information is not yet organized in terms of

conceptual structures and categories.

• Extraction of the 3-D model
• Kalman filters

The Conceptual Area

• A Conceptual Space CS

(Gärdenfors, 2000) is a metric space whose dimensions are strictly related to sensory based quantities (Color, pitch, spatial coordinates, etc.).

• Dimensions do not depend on any

specific linguistic description.

• The conceptual primitive is a

knoxel, i.e. a point in CS.

The Static CS

• A knoxel is a superquadric
• An object is a composition of

superquadrics

The linguistic area

• Hybrid formalism in the KL-ONE tradition
• T

erminological component

• terminological language: semantic networks (SINets)
• concept descriptions (general knowledge)
• Assertional component
• assertional language: ground atoms
• information about specific scene

Terminological component

• First order logic
• Concepts → One place predicates
• Roles → Two place predicates

Assertional component

Mapping

Generation of assertions

• Is driven by the focus of attention
• Implementation: artificial NNs
• two modalities:
• associative expectations
• linguistic expectations
• associative expectations are learned by NNs
• Hebbian mechanism
• linguistic expectations are driven by linguistic KB

Focus of attention

Linguistic expectations

A priori knowledge of the object shape

Cylinder-shaped(#k1) Box-shaped(#k2) Hammer (Hammer#1) has-part(Hammer#1,#k1) has-part(Hammer#1,#k2)

Associative expectations

Free associations among previously seen objects

Hammer (Hammer#1) Box (Box#1) Next-to(l#1) Has-part(l#1,Hammer#1) Has-part(l#1,Box#1)

System at work

Cylinder-shaped(#k1) Box-shaped(#k2) Hammer (Hammer#1) has-handle(Hammer#1,#k1) has-head(Hammer#1,#k2) Ball-shaped(#k3) Ball(Ball#1) has-part(Ball#1,#k3) Ellipsoid-shaped(#k4) Mouse(Mouse#1) has-part(Mouse#1,#k4)

Dynamic scenes

• Generic movements are made of smooth functions of

time separated by instantaneous discontinuities (Marr).

• A simple motion - delimited by two discontinuities -

can be approximated by the superimposition of frequency harmonics (FFT analysis)

Chella, A.; Frixione, M.; and Gaglio, S. (2000). Understanding dynamic scenes. Artificial Intelligence 123:89–132.

Parabolic motion

FFT Analysis of simple motion of a sq

FFT of the motion Recovered motion by the first frequencies

• f FFT

Static and Dynamic CS

Actions and Situations

• A Situation is a configuration of

knoxel in CS: objects maintain their motions states

• An (instantaneous) Action is a

scattering of knoxels in CS: an event occurs, and some objects may change their motion state

Dynamic focus of attention

• Synchronic attention:

scan operation in the same CS frame.

• Diachronic attention:

Scan operation in subsequent CS frames. A scattering aoccurs.

Terminological component

Seize part_of_Seize#1 Action Composite Simple Motion Grasp 1/1 1/1 Simple Motion Forearm stretching Upper arm stretching part_of_Stretch_out#2 1/1 1/1 * * * * * * * * * part_of_Seize#2 Arm approach Stretch out part_of_Action part_of_CMotion part_of_Stretch_out#1

System at work

A seizes an object

Mapping

Forearm_stretching(k_a) Upper_arm_stretching(k_b) Stretch_out(st1) part_of_Stretch_out#1(st1,k_a) part_of_Stretch_out#2(st1,k_b) Arm_approach(aa1) Grasp(g1) Seize(s1) part_of_Seize#1(s1,aa1) part_of_Seize#2(s1,g1) Assertions generated at the linguistic level

Robot introspection

• We propose that robot introspection is based on

higher order perceptions.

• First order perceptions: the perceptions of the
• uter world; they generate the agent conceptual

space

• Higher order perceptions: higher-order knoxels.

Second order perceptions

First order perception Second order perception

Second order knoxel

• A second order knoxel at time t

now describes the perception of the conceptual space of the agent at time t-d.

• The agent perceives itself and its

environment

kb ka

Ax(0) Ax(1)Ax(2) Ax(3) Ay(0) Ay(1) Ay(2) Ay(3) Az(0) Az(1) Az(2)Az(3)

k'a kb

Ax(0) Ax(1)Ax(2) Ax(3) Ay(0) Ay(1) Ay(2) Ay(3) Az(0) Az(1) Az(2)Az(3)

K a

t - δ

kb ka

Ax(0) Ax(1)Ax(2) Ax(3) Ay(0) Ay(1) Ay(2) Ay(3) Az(0) Az(1) Az(2)Az(3)

kb

Ax(0) Ax(1)Ax(2) Ax(3) Ay(0) Ay(1) Ay(2) Ay(3) Az(0) Az(1) Az(2)Az(3)

K a

t - δ

Sychronic Diachronic

Higher-order perceptions

Higher-order perceptions

• The outlined procedure may be generalized to

consider higher order knoxels

• Higher order knoxels correspond to the robot's higher
• rder perceptions of the knoxels of lower order at

previous d times.

• The union of first-order, second-order and higher-
• rder knoxels is at the basis of the robot introspection.
• The robot recursively embeds higher-order models of

its own CS's during its operating life.

Higher order knoxels

Higher order CSs

• Higher-order knoxels are mapped to meta-

predicates in the linguistic area, i.e. predicates describing the robot perceiving itself and its own actions.

• These meta-predicates form the basis of the

introspective reasoning of the robot

• The forms of introspection that are more directly

related to perceptual information can take great advantage from the proposed representation in the conceptual area.

• In the proposed framework the current and past

situations and actions, the goals, the plans the can be simply analyzed by geometric inspections in the CS

• The more “abstract” forms of introspection, that

are less perceptually constrained, are likely to be performed mainly within the linguistic area.

Introspection

Conclusions

• Open problems:
• 3D real time representation of the

perceived scene

• storing at time t the information of higher
• rder conceptual spaces at previous times,

starting from the beginning of the robot life

Thank you !