HPM in HMI Design Part II - Modelling Overview Part I - - - PowerPoint PPT Presentation

HPM in HMI Design Part II - Modelling

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 2

Overview

• Part I - Introduction

– Applied Cognitive Modelling – From Control Theory to Cognitive Functions

• Part II - Modelling

– HPM Engineering Life Cycle – ACT-R 6.0 for runaways – Advanced Modelling Tools

Human Performance Model Engineering Life Cycle

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 4

Problem Formulation Simulation results Conceptual Model Formal Model Computer Model

Mathematical modelling Implementation Proof of formalisation Program verification Conceptual modelling Proof of concept Experimentation Verification of Results Plausibility

(Fig. adopted from Lugner & Bub 1990)

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 5

Step 1 – Problem Formulation

What do we want to achieve with the HPM? How much effort can we afford? What confidence is necessary for the models predictions?

Problem Formulation Simulation results Conceptual Model Formal Model Computer Model

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 6

Step 2 – Conceptual Model

What Level of Detail is necessary? Which HPM method is appropriate? Conduct a Cognitive Task Analysis!

• Select appropriate CTA Method (see Schraagen et al. 2000)
• Identify goals, knowledge structures and information processing

strategies, heuristics, sources of control, ... Which cognitive theories and experimental results can we build upon? Which Architecture / Integrated Theory of Cognition is capable to ? Which additional assumptions and/or experimental Data is needed? How can we test our modelling assumptions?

Problem Formulation Simulation results Conceptual Model Formal Model Computer Model

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 7

Zeit (sec) Einheiten System Analyse Activities/ Processes World (theory) 105 days 104 hours Task Task analysis Subtasks Bounded 103 10 min Rationality 102 min Subtask Unit Task Analysis Unit Tasks 101 10 sec Unit task Cognitive task analysis Activities 100 1 sec Activity Embodied cognition Microstrategies Cognitive Band 10-1 100 ms Microstrategy Computational models

• f embodied cognition

Elements 10-2 10 ms Elements Architectural Parameters Biological 10-3 1 ms Parameters Band

(Gray & Boehm-Davis, 2000)

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 8

Fokus Informationsverarbeitung

subjective value formation Mental Information Processing Psychological mechanisms Physiologic Functions Anatomic Properties Physical Capabilities, Challenges Arrousal, Stress, Fatigue Cognitive Resources, Attention

(Fig. adapted from Rasmussen, 1986)

Behaviour Information, Task order Goals, Preferences

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 9

Theory Driven Human Performance Modelling

Theories Processes HPM describe Behaviour generate generate explain implement simulate

• HPM simulate processes

that generate observable behaviour according to some cognitive theory

• Unfortunately cognitive

theories were most often not developed with the goal to support computation

(Cooper 1999)

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 10

Theory Driven Human Performance Modelling

Theories Processes HPM describe Behaviour generate generate explain implement simulate

• HPM simulate processes

that generate observable behaviour according to some cognitive theory

• Unfortunately cognitive

theories were most often not developed with the goal to support computation

(Cooper 1999)

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 11

Conceptual Model

Rational Analysis (Anderson 1998) Goal Structures Knowledge Representation (Facts & Procedures) procedural (non-concious but observable) declarative (concious and explicable) Volitional Control of Behaviour Internal / External Control Conflict Resolution Predefined concurrent pathes of execution Learning Procedural: Subsymbolic Utility / Production Compilation Declarative: Subsymbolic Activation / Creation of new Facts

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 12

Step 3 – Formal Model

Translate concept on primitives of selected Architecture Implement Control Flow Extend Architecture if primitives are not sufficient?

Problem Formulation Simulation results Conceptual Model Formal Model Computer Model

A very short introduction to ACT-R

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 14

Productions (Basal Ganglia) Retrieval Buffer (VLPFC) Goal Buffer (DLPFC) Manual Buffer (Motor) Visual Buffer (Parietal) Deklarative Memory (Temporal/Hippocampus) Intentional Module (not identified) Visual Module (Occipital/Parietal) Motor Module (Motor/Cerebellum) External Task Environment

• 1. Evaluation (Striatum)
• 2. Selection (Pallidum)
• 3. Execution (Thalamus)

ACT-R 6.0

(Anderson et al. 2004)

• Production System
• Unified Cognitive Theory
• Buffers „hide“ complexity
• f computation in

modules and provide a common API

• Still Research: Mapping
• f Modules and Functions

to Brain Areas

(Abb. nach Taatgen 2004)

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 15

Declarative Memory Properties

Symbolic Level Associative Memory organized as a Semantic Net of CHUNKs with infinite Capacity (Anderson 1974, Anderson & Lebiere 1993) Subsymbolic Level

• Activation based Retrieval

Retrieval Time = f(Activation of Memoryelements)

• Activation Decay

Seldomly used Memory Elements are harder to retrieve than recently retrieved ones

• Activation Spreading

Usage of Memory Elements in Buffers rises activation of element and related chunks

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 16

Declarative Memory Syntax

CHUNK: typed data structure that has a unique name and may contain named references to other CHUNKS or terminal SYMBOLs like numbers. CHUNK-TYPE: Definition of CHUNK structure SLOT: Named Elements of a CHUNK (chunk-type NAME-OF-TYPE NAME-OF-SLOT-1 NAME-OF-SLOT-2 … ) (add-dm ( NAME-OF-CHUNK-1 isa NAME-OF-TYPE NAME-OF-SLOT-1 VALUE-OF-SLOT-1 NAME-OF-SLOT-2 VALUE-OF-SLOT-2 … ) ( NAME-OF-CHUNK-2 isa … ) )

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 17

Declarative Memory Example

(chunk-type number) (chunk-type addition-fact addend1 addend2 sum) (add-dm (seven isa number) (two isa number) (nine isa number) (fact-7+2=9 isa addition-fact addend1 seven addend2 two sum nine) )

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 18

Production Rules

Mental Information Processing is coded in form of Productions with a highly restricted set of information processing primitives IF Condition THEN Action Conditions: State and Content of Buffers Actions: Retrieval from Memory, Attention to Visual System, Issue Command to Motor System Modification of Retrieval/Goal Buffer Content Store to Memory (via clear Buffer)

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 19

Productions (Basal Ganglia) Retrieval Buffer (VLPFC) Goal Buffer (DLPFC) Manual Buffer (Motor) Visual Buffer (Parietal) Declarative Memory (Temporal/Hippocampus) Intentional Module (not identified) Visual Module (Occipital/Parietal) Motor Module (Motor/Cerebellum) External Task Environment

• 1. Evaluation (Striatum)
• 2. Selection (Pallidum)
• 3. Execution (Thalamus)

(Abb. nach Taatgen 2004)

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 20

Production Rule Syntax

(P NAME-OF-PRODUCTION

=BUFFER-1> NAME-OF-SLOT-1 ( Condition | Variable ) NAME-OF-SLOT-2 ( Condition | Variable ) =BUFFER-2> NAME-OF-SLOT-3 Condition … ==> =BUFFER-1> NAME-OF-SLOT-1 ( Symbol | Variable ) NAME-OF-SLOT-2 ( Symbol | Variable ) +BUFFER-2> NAME-OF-SLOT-3 ( Symbol | Variable ) )

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 21

Slot Content Primitives

Condition

• Slot contains specified Symbol
• Slot contains any Symbol (value bound to variable)
• two or more slots of any number of buffers contain the same symbol

Action

• set content to value of symbol
• set content to bound variable

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 22

Evaluate – Select – Execute Cycle

• All Productions are evaluated every 50 ms!
• Out of the set of matching productions (conflict set), one candidate

is elected to be executed.

• Conflict Resolution by comparing learned utility of production rule
• This kind of modelling is fun for 10 to 50 production rules
• more rules are hard to oversee and even harder to maintain
• In more complex domains: Lots of 'Hand Weaving' from Conceptual

Model to Formal Computer Model!

• Abstraction Layer provides no means for part-model reuse (but of

course one can program some lisp macros ;-)

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 23

Never trust a Cognitive Model...

• Essential primitives for HPM in

dynamic Systems are missing (jet), for Instance: – Duraton Estimation – Multitasking – Kinaesthetic Perception

• Ideal of PPS not reached?

– Coding in a kind of cognitive „Assembler“ (50 ms of cognition) – Models are hard to debug – Communication of models sometimes problematic

• 4 Modellierer derive at least 10

different Models, which „explain“ behaviour „very well“

Level Control Law (p apply-control-law-positive-error =goal> isa control-level state apply-control-law error > eps ?manual> state free ==> =goal> state sample-error +manual> ISA press-key key

• n

) ( apply-control-law-negative-error ...) ( apply-control-law-dead-band ...)

Tool Approaches to Raise Efficiency ACT-R

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 25

Supporting Model Engineering

ACT-R/State – Derivation of Control Flow Statechart from ACT-R Model (Urbas, Nekrasova & Leuchter 2005; Leuchter, Nekrasova & Urbas 2004) AGImap – Efficient Interfacing of technical System Simulator with ACT-R Perceptual-Motor Subsystem (Urbas & Leuchter 2006; Leuchter & Urbas, 2007) HTAmap – Direct Derivation of ACT-R Models from a cognitive Task Analysis via MDA Approaches (Heinath & Urbas 2007, Heinath & Urbas 2008) SIMTra – Multi-level Verification of empirical Data and Model Predictions (Dzaack & Urbas 2007, Dzaack 2008)

Slide 26

Running classes on modeling…

• High-Level Modeling Frameworks – Task Models
• Task Networks, GOMS, …
• imperative programming paradigm
• Explicit control flow, defined at design time
• Limited learning, adaptation, information processing capabilities, error

handling

• Low-Level Modeling Frameworks – Inf.Proc. Models
• ACT-R, (SOAR)
• Production system approach
• Implicit control flow, delayed to run-time
• The more descriptive, the harder to grasp
• Need for Support to understand control flow

Slide 27

Control Flow in imperative models

• model = sequence of instructions
• Sequence of instructions ≠ sequence of instruction

processing

• Control Flow: instructions, goto, conditional

I1: i=0 DO I2: incr i WHILE (i<10)

i = 0 incr i

i<10

Slide 28

Production Systems

• The select-execute-cycle chooses among all

productions those that match and executes at least one of them

• Firing of productions, i.e. execution of the

action part, changes memory  Control Flow delayed to run-time

• States of a Production Systems

– A state represents conditions on memory elements – A complete graph of a production system connects every state with its possible successors due to firing of productions

i < 10 incr i i == nil i=0

P0: i==nil → i=0 P1: i<10 → incr i

Slide 29

Memories in ACT-R

• Declarative Memory,

Motor, Perception

• Buffers
• Productions
• act2state:

– Complete matching & conflict resolution to complex – No Reduction of Complexity in state abstraction –  Goal Buffer only

Environment Visual Module Manual Module ACT-R Buffers Procedural Memory Declarative Memory Pattern matching Production execution ACT-R Buffers Visual Buffer Goal Buffer Retrieval Buffer Manual Buffer

Slide 30

From code to visualization:

ACT-R Code Intermediate format Object Model Visualization

(P initialize-addition =goal> ISA add arg1 =num1 arg2 =num2 sum nil ==> =goal> sum =num1 count 0 +retrieval> isa count-order first =num1 ) …

Slide 31

States from condition part of all Productions: C = { c | c = (name,GState)} States from action part of all Productions: A ={ a | a = (name, (GState-goalmodif, GState-subgoal))} Initial goal-state: I ={s | s– initial Goal-state} Conditions for transitions: Cp = {cp =(name, c*), c* - condition} Actions for transitions: Ap = {ap=(name, a*), a* - action} Goal-state: GState = (goal, Slots) Slots = { sl | sl = (name, symbol)} symbol = (type, value)

Intermediate Format

ACT-R Code Intermediate format Object Model Visualization

Slide 32

Objekt-Modell

Graph = (G, T) Goals G = {g | g = (name, states)} states = { s| s-Goal-State} Transitions T = { t | t =(name, STsource, STtarget, cond, act ), STsource ∈ State, STtarget ∈ State} State ={GState, g}

ACT-R Code Intermediate format Object Model Visualization

Slide 33

Slide 34

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 35

Closing the Gap

(Heinath & Urbas 2007)

Base for cog. modell is Cognitive Task Analysis Large Gap between CTA- Results and Formal Model No Formalisation of Formalisation! Solution Generative Programming Templates Model-Model-Transformation

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 36

Higher Efficiency by Abstraction

Main Points

• HTAmap = Additional

Level  Minimization of “Transformation Gap” between

• Task Modell
• Computational Model

1 2 3 4 4 5 6

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 37

Higher Efficiency by Model Fragment Reuse

Main Points

• HTAmap = Zusätzliche

Ebene  Minimiert die “Transformationslücke” zwischen

• Aufgabenmodell
• Kognitiven Modell
• Modell Engineering

 Higher Efficiency

• Reuse of adaptable

Patterns

1 2 3 4 4 5 6

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 38

Result of Sub Goal Templates CTA

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 39

Taxonomy of elementary Cognitive Activity Patterns

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 40

Transformation between SGT and HTAmap

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 41

Model-Model Transformation

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 42

Model Verification :: Startup of a Waste Water Treatment Plant

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 43

Empirical Results

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 44

Take Home Message

Human Performance Modelling in Cognitive Architectures for HMI Design is feasible, but

• takes often an unpredictable amount of time,
• suffers from cognitive science theories being incomplete (in respect

to computation) Application of HPM in HMI-Engineering calls for more efficiency!

• Model Analysis / Debugging Support
• Multiple Levels of Abstraction with tool chains

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 45

Thanks for your attention Members of the MoDyS Research Group

• Kindsmüller, M.C. (VolkswagenStiftung) – Lesen von Kurvenbildern
• Leuchter, S. (VolkswagenStiftung) – Modellierung & Software

Engineering

• Schulze-Kissing, D. (VolkswagenStiftung) – Empirie zur

Modellierung von Dauerschätzung

• Gauss, B. (EU) – e-Learning
• Huss, J. (TUB) – Kompensatorische Strategien
• Mahlke, S. (TUB) – Prospektive Gestaltung
• Dzaack, J. (DFG) – Aggregation und Auswertung von

Simulationsergebnissen

• Heinath, M. (DFG) – Integration von Handlungs- und

Informationsverarbeitungsmodellen

• Kiefer, J. (DFG) – Mikro- und Makrostrategien in

Mehraufgabenumgebungen

• Naumann, A (IBB) – Gestaltung von HMI für Vernetztes Fahren
• Pape, N. (VolkswagenStiftung) – Quantitative Modelle zur Schätzung

von Dauern

TU Dresden, 03/09/08 ICCL Summer School 2008, Urbas Slide 46

KogWis 2008 in Dresden

• 9. Fachtagung der

Gesellschaft für Kognitionswissenschaft Von den kognitionswissenschaftlichen Grundlagen zur Anwendung in Mensch-Maschine- Systemen und kognitiven technischen Systemen 28.9.2008-1.10.2008 Technische Universität Dresden http://www.kogwis08.de/