Human Learning in Dynamic Human Learning in Dynamic Environments - - PowerPoint PPT Presentation

Human Learning in Dynamic Human Learning in Dynamic Environments

Cleotilde (Coty) Gonzalez

Dynamic Decision Making Laboratory d /DDML b www.cmu.edu/DDMLab Social and Decision Sciences Department Carnegie Mellon University Research supported by the National Science Foundation : Human and Social Dynamics: Decision, Risk, and Uncertainty

Dynamic Environments

• Combat missions, Production scheduling, Fire fighting,

Emergency dispatch, Air-traffic control

• Complex
• Number of components: alternatives, events, courses of

action, outcomes

• Uncertainty: All possible states of the world and
• utcomes are unavailable, incomplete, and difficult to

imagine g

• Constraints: limited time, knowledge, resources, human

capacity

• Dynamic Complexity
• Dynamic Complexity
• Arises from the interactions of components over time
• Environment is autonomous. All is change at many

g y different time scales

• Learning from our actions: feedback delays

Dynamic Decision Making: A Closed-Loop view

Hypothesize illnesses and Symptoms

delay

run tests

delay delay

Test results Health

External event

results Health Diagnosis Treatment

delay delay

Diagnosis Treatment

delay

Learning in dynamic systems is hard

• People remain suboptimal in these systems even

with repeated trials, unlimited time and performance incentives (Sterman,1994; Diehl & Sterman 1995) Sterman, 1995).

• We have difficulty processing feedback.

F db k d l bl f l Feedback delay is a problem for learning (Brehmer, 1992; Sterman, 1989).

But… how do we learn in dynamic environments? environments?

• Decision Makers recognize typical situations and typical

D i i k th i t k l d

• responses. Decision makers use their past knowledge

and adapt their strategies “on the fly”.

Chess studies Expertise: Chase & Simon 1973

• Chess studies, Expertise: Chase & Simon, 1973
• Adaptive Decision Making: Payne, Bettman, & Johnson, 1993
• Decision making under uncertainty: “Case-Based Decision
• Decision making under uncertainty Case Based Decision

Theory” , Gilboa and Schmeidler, 1995

• Theory of automaticity: Logan, 1988
• “Recognition-Primed Decision Making” (RPDM): Intuition, Mental

simulations, Klein et al., 1993; Klein, 1998

Pattern recognition is easier if you have i experience

Instance Based Learning Theory (Gonzalez, Lerch, &

Lebiere, 2003)

• RECOGNITION OF FAMILIAR PATTERNS
• Determining the similarity between a situation and past

experience

• Identifying ‘typical’ situations and responses

y g yp p

• ACQUIRING CAUSE-EFFECT KNOWLEDGE

Q

• Accumulation of instances with practice in a task
• Improvement of decision making by bootstrapping on previous

k l d knowledge

Implemented in ACT-R (Anderson and Lebiere, 1988)

IBLT: WHAT do we learn?

Situation Decision Outcome

Situation- Decision Cycle Action- Outcome Cycle Cycle Cycle

Future Decisions

S O D S D O S D O

Blending

• f past

Outcomes Similarity

S D O S D O

Time

Outcomes F db k

Environment

Feedback

IBLT: HOW do we learn?

ACT-R

(A d & L bi 1998) (Anderson & Lebiere, 1998)

h l l f

Declarative Memory Procedural Memory

The 2x2 levels of ACT-R

Chunks: declarative facts Productions: If (cond) Then (action) Symbolic facts (cond) Then (action) A ti ti f h k S bS b li Activation of chunks (likelihood of retrieval) Conflict Resolution (likelihood of use) SubSymbolic

IBLT models compare to human decision making:

• In dynamic resource allocation tasks (Gonzalez et

making:

al., 2003)

• In supply chain management control (Martin,

Gonzalez & Lebiere 2004) Gonzalez & Lebiere, 2004)

• In repeated choice tasks (Lebiere, Gonzalez & Martin,

2007) 2007)

• But there is long way to go to demonstrate:

generalizability and utility of IBLT g y y

Decision Making Games (DMGames) used for experimentation for experimentation

• DMGames embody the essential characteristics of
• DMGames embody the essential characteristics of

real-world decision environments

• Interactive
• Interactive
• Repeated and interrelated decisions

E t l t d t i t ti

• External events and team interactions
• Help compress time and space – speed up learning
• Help manipulate experience - learn from simulated

cases and on-demand repeated practice k d d l d h

• No risk to individuals and they are FUN.

DMGames used in behavioral research in the DDMlab

Military Command and Control Real-time resource allocation Military Command and Control Real-time resource allocation Real time resource allocation Real time resource allocation Medical Medical Supply- Chain ed ca Diagnosis Supply- Chain ed ca Diagnosis Chain Management Fire Fighting Chain Management Fire Fighting

MEDIC: Learning tools that represent the dynamics of medical diagnosis (Gonzalez & Vrbin, 2007) y f

g

( , )

• Concepts adapted from Kleinmuntz (1985):

Task complexity (numerous diseases and symptoms)

• Task complexity (numerous diseases and symptoms)
• Disease base rates
• Time pressure
• Test diagnosticity
• Treatment effectiveness
• Treatment risk
• Treatment risk
• Additions:
• Feedback delays (e.g. receiving test results)
• With the potential for:
• Dynamic diagnostic cues
• Dynamic symptoms

MEDIC demo

Factors that influence Learning in dynamic systems y

• Time constraints (Gonzalez, 2004)
• Workload (Gonzalez, 2005)
• The similarity and diversity of experiences (Gonzalez and

y y p

Quesada, 2004; Gonzalez and Madhavan, in preparation)

• Our inherent cognitive abilities (Gonzalez, Thomas and

Vanyukov 2004) Vanyukov, 2004)

• The type of feedback (Gonzalez, 2005)
• Our difficulty in understanding simple stock and flow

Our difficulty in understanding simple stock and flow structures (Cronin and Gonzalez, 2005; Cronin, Gonzalez and Sterman,

2006; Gonzalez, Sterman and Cronin, in preparation)

Experiment 1: probabilities

• MEDIC incorporated:
• Symptoms-disease associations from 0.1 to 0.9
• Delay in test results

y

• Time pressure due to patient’s declining health in

real-time

• Deterministic treatment needed to be provided
• N=12, students, paid flat rate

N , students, pa d flat rate

• Each student resolved 56 cases

Results

Treatment

Results- test diagnosticity

Disease base rates

Diagnosticity per disease

Experiment 1: Conclusions

• Students did learn – not perfectly
• Showed knowledge of probabilities, tested for the

more diagnostic cues, and diagnosed very closely to the real state of the diseases. f .

• What is the role of feedback and how would that

interact with the symptom-probability matrix?

Experiment 2: Probabilities and f db k feedback

• MEDIC:
• Symptomology table: Probability or Certainty
• either detailed feedback or no feedback
• Participants were assigned to one of four conditions:
• probabilities, full feedback (P1) -26
• certainty full feedback (P2)-30
• certainty, full feedback (P2) 30
• certainty, no feedback (P3)-25
• probabilities, no feedback (P4)- 29
• N= 110 Participants were paid a flat dollar amount

P b bili Probability C t i t

Disease 1 Disease 2 Disease 3 Disease 4 0.25 0.25 0.25 0.25 Base Rates

Certainty

0.0 0.0 0.0 0.0 Symptom 1 1.0 0.0 0.0 0.0 Symptom 2 1.0 1.0 0.0 0.0 Symptom 3 0.0 0.0 1.0 0.0 Symptom 4

Test diagnosticity - probability condition

Test diagnosticity – Certainty condition

Diagnosticity per disease

Experiment 2: Conclusions

• Full feedback was helpful in the probabilistic

i t d did t k diff i th environment and did not make a difference in the certain environment

• We now know that: with repeated trials, students

p , learn in probabilistic environments with time constraints and feedback delays

• Feedback helps in probabilistic environments

Feedback helps in probabilistic environments

• Probabilistic environments are not the main reason

for poor learning in dynamic tasks

Basic Building Blocks of Dynamic Decision Making Tasks Making Tasks

• Stocks (accumulations)
• Flows that increase (Inflow) or decrease (Outflow) the

stock

• Feedback Delays & multiple relationships
• Environmental or external effects
• Multiple decisions about flows

These problems of dynamic control over time are important to human life: keeping a healthy weight, bank p p g y g accounts, company inventory, stress levels, climate change etc.

Humans suffer of poor understanding of accumulation: Stock-Flow failure accumulation: Stock Flow failure

Cronin, Gonzalez & Sterman, 2008 ; Cronin & Gonzalez, 2007; Cronin, Gonzalez and Sterman, 2006; Sweeney & Sterman, 2000 St 2002 2000; Sterman, 2002;

Weight as balance between consumed and expended energy expended energy

• 1. When eaten most?
• 2. When exercised most?
• 3. When weight highest?
• 4. When weight lowest?

4 g

Blood glucose level as balance between glucagon and insulin production glucagon and insulin production

• 1. When most glucagon?

g g

• 2. When most insulin?
• 3. When glucose level
• 3. When glucose level

highest?

• 4. When glucose level
• 4. When glucose level

lowest?

Why? (Cronin & Gonzalez, 2007; Cronin, Gonzalez &

Sterman, 2008)

• Not an artifact of the graph

t rman, )

• Not due to the form of graphical presentation
• Not due to motivation
• Not due to motivation
• Not due to familiarity with the context
• Stock Flow failure is one important reason for
• Stock-Flow failure is one important reason for

learning problems in dynamic systems U f h i ti th t i t iti l li

• Use of heuristics that are intuitively appealing

but erroneous

Future work

• Further investigate the correlation heuristic

and the Stock Flow failure and the Stock-Flow failure

• Use DMGames of Dynamic Stocks and Flows to

d t d th i d l i bl understand the reasoning and learning problems in dynamic tasks

• Further develop the Instance-Based learning

theory to other dynamic problems, like the St k Fl Stock-Flow

• Further investigate ways to identify and
• vercome the problems in learning in dynamic

systems

DDMLab – February,