Strategic Tradeoffs in Goal Reactivation Justin Li Computer Science - - PowerPoint PPT Presentation

Strategic Tradeoffs in Goal Reactivation

Justin Li

Computer Science and Engineering University of Michigan justinnh@umich.edu 2014-06-19

Introduction Strategies Experiments Results Summary ,

Introduction

2014-06-19

• Li. Goal Reactivation

2

Introduction Strategies Experiments Results Summary ,

Bruno Mars

◮ Tell the devil I said “Hey”

when you get back to where you’re from (Grenade, 2nd verse)

2014-06-19

• Li. Goal Reactivation

3

Introduction Strategies Experiments Results Summary ,

Bruno Mars

◮ Tell the devil I said “Hey”

when you get back to where you’re from (Grenade, 2nd verse)

◮ (Yes I listen to bad music)

2014-06-19

• Li. Goal Reactivation

3

Introduction Strategies Experiments Results Summary ,

Bruno Mars

◮ Tell the devil I said “Hey”

when you get back to where you’re from (Grenade, 2nd verse)

◮ (Yes I listen to bad music)

This talk: how do you remember to “tell the devil hey” when you see him?

2014-06-19

• Li. Goal Reactivation

3

Introduction Strategies Experiments Results Summary ,

Goal Reactivation in Everyday Life

◮ Passing on a message ◮ Buying milk after work ◮ Taking medication before bed

2014-06-19

• Li. Goal Reactivation

4

Introduction Strategies Experiments Results Summary ,

Goal Reactivation in Everyday Life

◮ Passing on a message ◮ Buying milk after work ◮ Taking medication before bed

Why is this a difficult?

2014-06-19

• Li. Goal Reactivation

4

Introduction Strategies Experiments Results Summary ,

Forgetting

◮ Forgetting keeps working

memory to a manageable size

◮ Memory elements are

forgotten if their activation falls below a threshold

◮ Activation increases with

recency and frequency of access

◮ Forgotten items can be

recovered from long-term memory

2014-06-19

• Li. Goal Reactivation

5

Introduction Strategies Experiments Results Summary ,

Research Question

Research question: How should an agent reactivate goals in different environments?

2014-06-19

• Li. Goal Reactivation

6

Introduction Strategies Experiments Results Summary ,

Formalizing Goals

A goal is made of:

◮ a target: when you see the

devil

◮ an action: tell him I said

“hey”

2014-06-19

• Li. Goal Reactivation

7

Introduction Strategies Experiments Results Summary ,

Formalizing Goals

A goal is made of:

◮ a target: when you see the

devil

◮ an action: tell him I said

“hey” Stages of completing a goal:

• 1. Encoding: Mr. Mars makes

his request

• 2. Retention: I do other things
• 3. Initiation: I see the devil
• 4. Execution: I tell him Mr.

Mars says “hey”

• 5. Completion: I forget about

this goal

2014-06-19

• Li. Goal Reactivation

7

Introduction Strategies Experiments Results Summary ,

Research Question

Research question: How should an agent reactivate goals in different environments? Strategies

◮ Procedural Strategy ◮ Preemptive Strategy ◮ Spontaneous Retrieval Strategy ◮ Noticing-Plus-Search Strategy

2014-06-19

• Li. Goal Reactivation

8

Introduction Strategies Experiments Results Summary ,

Research Question

Research question: How should an agent reactivate goals in different environments? Strategies

◮ Procedural Strategy ◮ Preemptive Strategy ◮ Spontaneous Retrieval Strategy ◮ Noticing-Plus-Search Strategy

2014-06-19

• Li. Goal Reactivation

9

Introduction Strategies Experiments Results Summary ,

Procedural Strategy

Learn a chunk that proposes the action if the target is present Example: practice until the goal becomes a habit

2014-06-19

• Li. Goal Reactivation

10

Introduction Strategies Experiments Results Summary ,

Procedural Strategy

Learn a chunk that proposes the action if the target is present Example: practice until the goal becomes a habit Encoding Learn a chunk Retention N/A Initiation chunk fires, proposing do-goal Execution do-goal is selected and applied Completion N/A

2014-06-19

• Li. Goal Reactivation

10

Introduction Strategies Experiments Results Summary ,

Preemptive Strategies

Periodically retrieve forgotten goals to see if they should be reactivated Example: “Today I have to buy milk, talk to John, and if I see the devil, ...”

2014-06-19

• Li. Goal Reactivation

11

Introduction Strategies Experiments Results Summary ,

Preemptive Strategies

Periodically retrieve forgotten goals to see if they should be reactivated Example: “Today I have to buy milk, talk to John, and if I see the devil, ...” Encoding store goal in SMem Retention periodically retrieve and boost goal Initiation rule matches target to percepts, proposing do-goal Execution do-goal is selected and applied Completion remove goal from SMem

2014-06-19

• Li. Goal Reactivation

11

Introduction Strategies Experiments Results Summary ,

Research Question

Research question: How should an agent reactivate goals in different environments?

2014-06-19

• Li. Goal Reactivation

12

Introduction Strategies Experiments Results Summary ,

Research Question

Research question: How should an agent reactivate goals in different environments?

◮ with different number of goals? ◮ with goals that never apply? ◮ different amounts of interference?

2014-06-19

• Li. Goal Reactivation

12

Introduction Strategies Experiments Results Summary ,

Experimental Domain

An abstract domain with fixed-length stages:

◮ Encoding: 2,000 cycles ◮ Retention: 13,000 cycles ◮ Initiation/Execution: 2,500

cycles

◮ Completion: 2,500 cycles

Randomly generate goal properties and interference events

2014-06-19

• Li. Goal Reactivation

13

Introduction Strategies Experiments Results Summary ,

Experimental Domain

An abstract domain with fixed-length stages:

◮ Encoding: 2,000 cycles ◮ Retention: 13,000 cycles ◮ Initiation/Execution: 2,500

cycles

◮ Completion: 2,500 cycles

Randomly generate goal properties and interference events Metrics:

◮ goal completion percentage ◮ computation required after

completion

2014-06-19

• Li. Goal Reactivation

13

Introduction Strategies Experiments Results Summary ,

Results: Preemptive Strategy Parameters

2014-06-19

• Li. Goal Reactivation

14

Introduction Strategies Experiments Results Summary ,

Results: Preemptive Strategy Parameters

2014-06-19

• Li. Goal Reactivation

15

Introduction Strategies Experiments Results Summary ,

Results: Preemptive Strategy Parameters

2014-06-19

• Li. Goal Reactivation

16

Introduction Strategies Experiments Results Summary ,

Results: Preemptive Strategy Parameters

2014-06-19

• Li. Goal Reactivation

17

Introduction Strategies Experiments Results Summary ,

Results: Preemptive Strategy Parameters

2014-06-19

• Li. Goal Reactivation

18

Introduction Strategies Experiments Results Summary ,

Results: Number of Goals

2014-06-19

• Li. Goal Reactivation

19

Introduction Strategies Experiments Results Summary ,

Results: Number of Goals

2014-06-19

• Li. Goal Reactivation

19

Introduction Strategies Experiments Results Summary ,

Results: Number of Goals

2014-06-19

• Li. Goal Reactivation

19

Introduction Strategies Experiments Results Summary ,

Results: Number of Goals

2014-06-19

• Li. Goal Reactivation

19

Introduction Strategies Experiments Results Summary ,

Results: Effects of Interference (at encoding)

2014-06-19

• Li. Goal Reactivation

20

Introduction Strategies Experiments Results Summary ,

Results: Effects of Interference (at retention)

2014-06-19

• Li. Goal Reactivation

21

Introduction Strategies Experiments Results Summary ,

Results: Post-Completion Costs

2014-06-19

• Li. Goal Reactivation

22

Introduction Strategies Experiments Results Summary ,

Results: Post-Completion Costs

2014-06-19

• Li. Goal Reactivation

23

Introduction Strategies Experiments Results Summary ,

Nuggets and Coal

Nuggets

◮ Flexible domain for testing

goal-reactivation strategies

◮ Appropriate metrics for

differentiating between strategies Coal

◮ Sparse data (ongoing work) ◮ Both tested strategies have

• ngoing costs

2014-06-19

• Li. Goal Reactivation

24

Introduction Strategies Experiments Results Summary ,

Questions?

2014-06-19

• Li. Goal Reactivation

25