Ongoing Emergence: A Core Concept in Epigenetic Robotics - - PowerPoint PPT Presentation

http://www.cprince.com/PubRes/EpiRob05

Ongoing Emergence:

A Core Concept in Epigenetic Robotics

Christopher G. Prince Nathan A. Helder George J. Hollich

University of Minnesota Duluth,

• Dept. of Computer Science

Purdue University,

• Dept. of Psychological Sciences

http://www.cprince.com/PubRes/EpiRob05

Acknowledgments

Lakshmi Gogate, Eric Mislivec John Weng Supported in part by UROP grants and a donation from Digi-Key Corp.

2

http://www.cprince.com/PubRes/EpiRob05

Outline

Motivation

Infant example, goals review

Criteria for ongoing emergence Evaluation of recent robotics projects Bootstrapping & tasks Conclusions

3

http://www.cprince.com/PubRes/EpiRob05

Ongoing Emergence

Example cases: Infant locomotion, infant word learning, and infant

• bject skills

Each show progressive development

• f skills

4

http://www.cprince.com/PubRes/EpiRob05

Locomotion skills

Postures: sitting, crawling, cruising, walking

Age & order of acquisition highly variable

Skill developments occur gradually, over weeks of experience with the particular posture Adaptive responding depends on infants’ experience with the specific posture

5

(Adolph, 2005)

http://www.cprince.com/PubRes/EpiRob05

Crawling

(Adolph et al., 2000, 2003; Adolph, 2005; Berger & Adolph, 2003)

9 mo 9 mo

6

http://www.cprince.com/PubRes/EpiRob05

Cruising

11 mo

7

(Adolph et al., 2000, 2003; Adolph, 2005; Berger & Adolph, 2003)

http://www.cprince.com/PubRes/EpiRob05

First Steps

8

12 mo? 14 mo?

(Adolph et al., 2000, 2003; Adolph, 2005; Berger & Adolph, 2003)

http://www.cprince.com/PubRes/EpiRob05

Infants vs. Robots

Infants clearly display progressive developments of skills Potential for achieving ongoing emergence is a core reason why epigenetic robotics is interesting So, propose ongoing emergence as a core concept in epigenetic robotics

9

http://www.cprince.com/PubRes/EpiRob05

Goals of Recent Projects

Blank, Kumar, Meeden, & Marshall (2005) Dominey & Boucher (2005) Oudeyer, Kaplan, Hafner, & Whyte (2005) Weng and colleagues (2001, 2004; Chen & Weng, 2004)

10

Blank et al. (2005)

“allow a mobile robot to incrementally progress through levels of increasingly sophisticated behavior”

Developmental algorithms involve:

Abstraction: to focus agent on relevant environmental features Anticipation: enable prediction of environmental change Self-motivation: pushes system to develop more complex abstractions & anticipations

11

http://www.cprince.com/PubRes/EpiRob05

Dominey & Boucher(2005)

“successive emergence of behaviors in a developmental progression of increasing processing power and complexity” “From simple representations such as contact, support, and attachment … the infant [may]construct progressively more elaborate representations of visuospatial meaning” (p. 244; see also Mandler, 1999)

12

http://www.cprince.com/PubRes/EpiRob05

Oudeyer, Kaplan, Hafner, & Whyte (2005)

Progressive increases in complexity of activities & capabilities Autonomy, self-construction of development sequences, & intrinsic motivation (e.g., play)

13

http://www.cprince.com/PubRes/EpiRob05

Weng et al.

Autonomous construction of representations for previously unknown knowledge and skills (Weng et al., 2001)

Agents may thus “select rules when new situations arise, e.g. in uncontrolled environments” (p. 205, Weng, 2004).

Open ended and cumulative learning

• f complex skills by first learning

simpler skills (Weng et al., 2001)

14

http://www.cprince.com/PubRes/EpiRob05

How can we make progress?

• 1. Create robotic systems that attempt to

exhibit ongoing emergence

• 2. Analyze existing systems to see how they

fare, and see what is present, see what is missing

15

http://www.cprince.com/PubRes/EpiRob05

Criteria

Requirements for an epigenetic robot to be seen as exhibiting ongoing emergence

16

http://www.cprince.com/PubRes/EpiRob05

Criterion 1

Agent creates (acquires) behaviors, representations, and perceptual capacities

Call these “skills” Agent creates new skills based on current skill repertoire, physical & environmental resources

Properties of these new skills include

May not yet be independent components May not be usable to create further, new skills

17

http://www.cprince.com/PubRes/EpiRob05

Issue: Indefinite Skill Progress

Don’t want skill creation to stop In some sense, we want an indefinite progression of skills

But, how can that be measured? How can you tell if development is open-ended?

In a finite experiment, how can you make sure that skill development doesn’t stop?

18

http://www.cprince.com/PubRes/EpiRob05

Criterion 2

Make new skills part of repertoire of agent’s skills

New skills need to be incorporated into this repertoire

Properties of incorporated skills include

Is now a separate component, to some degree Can be used to create new skills, with primitive or other incorporated skills

19

http://www.cprince.com/PubRes/EpiRob05

Criterion 3

New environments may have new tasks

Perception (e.g., vision), and behaviors may vary depending on these tasks Issue: Why should the agent solve these tasks?

For example, why should an agent that develops skills to play soccer also learn to swim or learn social skills? The agent needs to be self-motivated Need semi-autonomous development of values & goals (motivations)

20

http://www.cprince.com/PubRes/EpiRob05

Criteria 4, 5, & 6

Skills need to start somewhere

Criterion 4: Bootstrapping, i.e., when system starts, some skills rapidly become available

Some skill invariance is needed to create new skills based on earlier skills

Criterion 5: Stability, i.e., skills persist over time

Characterization of scope of behaviors

Criterion 6: Reproducibility, i.e., robotic system started in similar initial states and in similar environments should produce similar effects

21

http://www.cprince.com/PubRes/EpiRob05

Criteria for Ongoing Emergence

1) New skill creation 2) Skills incorporated into repertoire 3) Autonomous development of motivations 4) Bootstrapping 5) Stability 6) Reproducibility

22

http://www.cprince.com/PubRes/EpiRob05

Systems & Evaluation

Scale

23

Meaning Notation Score Achieved criterion + 2 Partially achieved criterion ? 1 Criterion not achieved

–

http://www.cprince.com/PubRes/EpiRob05

Blank et al. (2005)

Connectionist models Sensor prototypes (Linaker, & Niklasson, 2000) Self-organizing maps Tasks: Wall following & navigation to goal locations

24

http://www.cprince.com/PubRes/EpiRob05

Evaluation

New skill creation (+) Learning wall following &

navigation to locations

Skill incorporation

(–) Independent sensory prototypes incrementally added;

• therwise 1 phase of learning

Autonomous development of motivations

(?) Self-organizing maps specify goals in location navigation

Bootstrapping

(+) Translation & rotation movements; sonar readings

Stability

(?) Apparently

Reproducibility

(+) 10 wall following simulations

25

http://www.cprince.com/PubRes/EpiRob05

Dominey & Boucher (2005)

Visual & auditory input

Connectionist models learn syntax, based on closed class words (e.g., “a”, “the”) Generalization to new sentences with same syntax Extension to new tasks by modification of architecture

26

http://www.cprince.com/PubRes/EpiRob05

Evaluation

New skill creation (+) Models acquire syntax Skill incorporation (–) Only 1 phase of learning Autonomous development of motivations

(–) No explicit value system

Bootstrapping

(+) Built-in speech and object recognition

Stability

(+) Ceiling performance & generalization on most tasks

Reproducibility

(+) Combined tasks; models seem to have limited random elements

27

http://www.cprince.com/PubRes/EpiRob05

Oudeyer et al. (2005)

Sensorimotor vectors used as input (8D) Learning ‘expert’ per vector space partition

f: SM -> S

Rate of expert learning used to select actions

28

http://www.cprince.com/PubRes/EpiRob05

Details on Results

Phase 1: exploration, body babbling Phase 2: looking, but not finding objects Phase 3: biting & bashing, but not

• riented to objects

29

Time step: 3-4 seconds

Evaluation

New skill creation (+) Robot learns various relations

between motors & sensors

Skill incorporation (?) Can new skills be used in

creating other skills?

Autonomous development of motivations

(+) Rate of learning used to predict sensory input

Bootstrapping

(+) (M) biting, bashing, head turn; (S) object, biting, oscillation

Stability

(?) Within phases

Reproducibility

(?) Similar developmental sequences in several experiments; sequences never exactly the same 30

http://www.cprince.com/PubRes/EpiRob05

Chen & Weng (2004)

“Drawbridge” task (Baillargeon et al., 1985)

IHDR learning (Weng & Hwang, 2003) Prediction of next sensory inputs Novelty computed from distance between actual sensory input and predicted input Higher novelty means higher action reinforcement

31

Evaluation

New skill creation (+) Novelty detection based on

visual stimuli

Skill incorporation (–) Single phase of development Autonomous development of motivations

(+) Novelty used as value system

Bootstrapping

(+) Fixed head positions, & turn head left, stay, turn head right

Stability

(?) Apparently

Reproducibility

(+) 12 “subjects”; 1 hr experience each before experiment (toys,

• bjects, people, room & corridor)

32

http://www.cprince.com/PubRes/EpiRob05

Summary

33

Criterion Score New skill creation 8/8 Skill incorporation 1/8 Autonomy 5/8 Bootstrapping 8/8 Stability 5/8 Reproducibility 7/8

http://www.cprince.com/PubRes/EpiRob05

Bootstrapping: Using Built-in Primitives

“one behavior among many” (Edsinger-Gonzales, 2005) One approach: Integration techniques

Cheng, Nagakubo & Kuniyoshi (2001) Triesch & von der Malsburg (2001) No apparent methods to incorporate or feedback results as separate, component skills which can be used for further development

34

http://www.cprince.com/PubRes/EpiRob05

Bootstrapping: No Built-in Primitives

Hypothesis: through self development

ICDL 05: Lungarella & Sporns (2005); Olsson, Nehaniv, & Polani (2005); Yoshikawa, Yoshimura, Hosoda, & Asada (2005) EpiRob 05: Edsinger-Gonzales (2005) Gold & Scassellati (2005); Michel, Gold & Scassellati (2004) Salunke (in preparation)

35

Tasks

What tasks are suited to demonstrate

• ngoing emergence?

Shouldn’t have a clearly defined end point Should be achievable incrementally

What about habituation?

Lots of psychological studies (e.g., Baillargeon et al., 1985) Computational models: Schöner & Thelen (in press) Robotic: Chen & Weng (2004); Lovett & Scassellati (2004)

36

Conclusions

Little research in ongoing emergence

Yet, vital to the goals of this new field

Present contribution: Start of a conceptual framework for this issue Need research into skill incorporation

Skills of a robot need to usable for development of new skills

37

http://www.cprince.com/PubRes/EpiRob05

References

http://www.cprince.com/PubRes/EpiRob05/talk/references.pdf

McMillan (2004)

1 yr 6 mo