Toward Understanding Robust & Collaborative Monitoring - - PowerPoint PPT Presentation

▶
toward understanding robust collaborative monitoring
SMART_READER_LITE
LIVE PREVIEW

Toward Understanding Robust & Collaborative Monitoring - - PowerPoint PPT Presentation

Aug 01, 2023 273 likes •621 views

Toward Understanding Robust & Collaborative Monitoring (12RH05COR) PI: Dr. Chris Myers (AFRL/RHAC) Co-PI: Dr. Andrew Howes (U. of Birmingham) Co-PI: Dr. Rick Lewis (U. of Michigan) Additional Collaborators: Dr. Joseph Houpt (Wright State

slide-1
SLIDE 1

Toward Understanding Robust & Collaborative Monitoring

(12RH05COR) PI: Dr. Chris Myers (AFRL/RHAC) Co-PI: Dr. Andrew Howes (U. of Birmingham) Co-PI: Dr. Rick Lewis (U. of Michigan)

Additional Collaborators:

  • Dr. Joseph Houpt (Wright State Univ.)

AFOSR Program Review:

Mathematical and Computational Cognition Program Computational and Machine Intelligence Program Robust Decision Making in Human-System Interface Program (Jan 28 – Feb 1, 2013, Washington, DC)

slide-2
SLIDE 2

Dynamic Monitoring

2

slide-3
SLIDE 3

Dynamic Monitoring

3

slide-4
SLIDE 4

Dynamic Monitoring

  • Visual encoding
  • Eye movements
  • Decision making

– Is what is encoded a target? – What should be encoded next?

  • Mouse movements & clicking
  • Often involves coordinating with another

individual

4

slide-5
SLIDE 5

List of Project Goals Years 1-2:

– Adaptive boundedly optimal model of foveated vision Visual encoding, eye movements, decisions – Adaptive, boundedly optimal model of motor control Mouse movements & clicking

Years 2-3:

– Integrated models of foveated vision and motor control Model performs monitoring task – Collaborative version of integrated model Coordinating with another individual

Human—Model performance comparisons throughout

5

slide-6
SLIDE 6

List of Project Goals Years 1-2:

– Adaptive boundedly optimal model of foveated vision Visual encoding, eye movements, decisions – Adaptive, boundedly optimal model of motor control Mouse movements & clicking

Years 2-3:

– Integrated models of foveated vision and motor control Model Performs monitoring task – Collaborative version of integrated model Coordinating with another individual

Human—Model performance comparisons throughout

6

slide-7
SLIDE 7

Cognitively Bounded Rational Analysis

  • 1. Specify the architecture & environment

– Info available to process; – Processing constraints; – Explicit utility function;

  • 2. Specify space of possible task strategies
  • 3. Compute expected subjective utilities
  • 4. Determine strategies with highest payoff

& compare to human data

7

slide-8
SLIDE 8
  • Distractor ratio task – Find the O

– 15 different ratios – Potential for adaptive feature search

Saccadic Selectivity

8

3 color:45 shape 24 color:24 shape 45 color:3 shape

slide-9
SLIDE 9
  • Adaptive behavior*

Saccadic Selectivity

9

*Shen, J., Reingold, E. M., & Pomplun, M. (2000). Distractor ratio influences patterns of eye movements during search. Perception, 29, 241-250.

slide-10
SLIDE 10
  • Cognitively bounded rational searcher
  • Scaled down task – find the O:
  • Constraints:

– Foveated vision  Limited encoding in periphery – Feature noise  red or green? X or O? – Spatial noise  at which position?

Adaptive Visibility Model*

10

O O O O O O X 5:1 X O O X O X O 3:3 X O X X X X O 1:5

*Kowler, E. (2011). Eye movements: the past 25 years. Vision Research, 51(13), 1457-1483

slide-11
SLIDE 11

Adaptive Visibility Model

11

Computational Cognitive Process model Constraints on perceptual encoding Optimally integrated percepts

slide-12
SLIDE 12

Adaptive Visibility Model

12

Spatial & Feature Noise Bayes Rule Target | No Target Intelligent | Random

slide-13
SLIDE 13
  • Represents the likelihood that the shape

and color at each location is/is not the same as the target (O)

Object Percept

13

X O O X O X O Truth: Color: .4 .6 .3 .1 .4 .3 .2 Shape: .2 .4 .4 .5 .5 .6 .7 Model’s percept:

Fixated Location

slide-14
SLIDE 14

Obtain Percept: Spatial Noise

14

  • Feature for a location may

come from a neighbor

  • Fovea  µ = 0; σ = 0.1
  • Parafovea  µ = 0; σ = 10

X O O X O X O X O X X O X O

color shape

“Illusory conjunctions”

Neri & Levi, 2006; Levi, 2008; Poder & Wagemans, 2007

slide-15
SLIDE 15
  • Sampled from normal distribution
  • Fovea

 µ = 0; σ = 0.3

  • Parafovea

 µ = 0; σ = 1

Obtain Percept: Feature Noise

15

O O O O O O X

slide-16
SLIDE 16

Optimal Integration

  • Update posterior probabilities

– Across all possible displays – Given sample from current fixation location

  • Probability that a display

contains the target for all displays

16

slide-17
SLIDE 17

Decision Variable Calculation

  • Two decision variables:

– Present: sum over posteriors for all displays that contain a target – Absent: sum over posteriors for all displays that DO NOT contain a target

  • Threshold

– Set to 0.85 – Potential for strategic variability

17

slide-18
SLIDE 18

Different Strategies

  • Intelligent:

– Sequential – start at a location and move from left to right, and back around, until threshold reached – Posterior Driven – next fixation at location most likely to contain the target

  • Unintelligent:

– Random – uniform; sample location w/replacement

18

slide-19
SLIDE 19

Model Results: Random Strategy

19

slide-20
SLIDE 20

Model Results: Sequential Strategy

20

slide-21
SLIDE 21

Model Results: Posterior Strategy

21

slide-22
SLIDE 22

Model Results: Posterior Strategy

22

slide-23
SLIDE 23 ! ! ! ! ! ! !

1 2 3 4 5 6 7 4.8 5.0 5.2 5.4 5.6

Fixations per trial for random strategy

Number Same−Feature1 Distractors Number of Fixations

! ! ! ! ! ! ! !

Target absent Target present

! ! ! ! ! ! !

1 2 3 4 5 6 7 4.4 4.6 4.8 5.0 5.2 5.4

Fixations per trial for sequential strategy

Number Same−Feature1 Distractors Number of Fixations

! ! ! ! ! ! ! !

Target absent Target present

! ! ! ! ! ! !

1 2 3 4 5 6 7 3.0 3.2 3.4 3.6 3.8

Fixations per trial for look−for−targets strategy

Number Same−Feature1 Distractors Number of Fixations

! ! ! ! ! ! ! !

Target absent Target present

Model Results: No Spatial Noise

23

! ! ! ! ! ! !

1 2 3 4 5 6 7 2.0 2.1 2.2 2.3 2.4

Fixations per trial for random strategy

Number Same−Feature1 Distractors Number of Fixations

! ! ! ! ! ! ! !

Target absent Target present

! ! ! ! ! ! !

1 2 3 4 5 6 7 2.5 2.6 2.7 2.8 2.9 3.0

Fixations per trial for sequential strategy

Number Same−Feature1 Distractors Number of Fixations

! ! ! ! ! ! ! !

Target absent Target present

! ! ! ! ! ! !

1 2 3 4 5 6 7 1.9 2.0 2.1 2.2

Fixations per trial for look−for−targets strategy

Number Same−Feature1 Distractors Number of Fixations

! ! ! ! ! ! ! !

Target absent Target present

slide-24
SLIDE 24

What about salience?

  • Salience makes similar

predictions

– But only with explicit IOR – And aid from uncertainty for decision threshold

  • Hence, really salience+

24

slide-25
SLIDE 25

Differentiating b/t the Models

25

  • Find a target (e.g., X):

XO OX + +

♠♣ ♥♦

Current Trial: $0.005 Total Earned: $3.445 Initial fixation crosshairs 500 ms; either 8°, 12°, 14°, or 16° from + 500 ms 3000 ms

slide-26
SLIDE 26

Differentiating b/t the Models

26

  • Find a target (e.g., X):
  • Parafoveal calibration procedure to help account for

individual differences of the bounds in periphery

  • Results will:
  • Differentiate between two potential models
  • Indicate appropriate bounds for monitoring model

AVM Prediction Salience Prediction

XO OX

slide-27
SLIDE 27

Motor Control & Integration

27

  • 3 models, 2 developed, 1 under

development

  • Experiment to differentiate b/t models
  • Stay tuned

– Had a video teaser, but tough to run from Dayton…

slide-28
SLIDE 28

Summary

  • Adaptive visibility model for monitoring

– Empirical study to distinguish b/t it and salience

  • Motor control models for moving and

clicking under development

– 3 models – more on this next year

  • Integration of perceptual and motor

models

– Exploring the use of POMDPs

28

slide-29
SLIDE 29

Team

29

  • Dr. Andrew Howes (Co-PI; Univ. of Birmingham, UK)
  • Dr. Rick Lewis (Co-PI; Univ. of Michigan)
  • Dr. Joe Houpt (Asst. Professor; Wright State University)
  • Mr. Joe Benincasa (Research Assistant; UDRI)
  • Dr. Chris Myers (PI; Air Force Research Laboratory)
slide-30
SLIDE 30

List of Publications Attributed to the Grant

Lewis, R. L., Howes, A., & Singh, S. (submitted). A bounded optimality approach to psychological theory: Linking mechanism and behavior through utility maximization. TopiCS in Cognitive Science Myers, C. W., Lewis, R. L., & Howes, A. (in preparation). Boundedly optimal adaptation during visual

  • search. To be submitted to the 35th Annual Cognitive Science Conference; Berlin, Germany.

Myers, C. W., Lewis, R. L., & Howes, A. (in perparation). A boundedly optimal model of parafoveal

  • search. To be submitted to the 12th International Conference on Cognitive Modeling; Ottawa,

Canada. 30

slide-31
SLIDE 31

Robust Monitoring (Myers)

Research Objectives: DoD Benefits: Technical Approach: Budget ($k): YR 1 YR 2 YR 3 YR 4 150k 250k 150k 0k

Project Start Date: Project End Date: 31

  • Empirically investigate robust human

behavior during visual monitoring

  • Rigorously test cognitively bounded

rational analysis as a principled solution

  • Identify bounds on relevant processes
  • Apply bounds to processes
  • Determine optimal behavior given bounds
  • Compare cognitively bounded optimal

behavior to empirical data

  • Understanding of how humans achieve

robust adaptation will facilitate the design

  • f systems they operate
  • An understanding of robust human

adaptation will help toward the development of training curricula and methodology

  • Oct. 2012
  • Oct. 2014
slide-32
SLIDE 32

List of Project Goals 1. Adaptive boundedly rational model of foveated vision 2. Adaptive, boundedly rational model of motor control 3. Integrated models of foveated vision and motor control 4. Collaborative version of integrated model 5. Model performance comparison to humans

1. Visual search 2. Pointing and clicking moving targets 3. Complex monitoring task 4. Collaborative monitoring task

32

slide-33
SLIDE 33

Progress Towards Goals (or New Goals) 1. Adaptive boundedly rational model of foveated vision

Developed, tested in 2 task environments; working on 3rd

2. Adaptive, boundedly rational model of motor control

Bang-bang, proportional velocity; working toward adapting Saunders & Knill (2004)

3. Integrated models of foveated vision and motor control

Investigating using POMDPs for integration

4. Collaborative version of integrated model 5. Model performance comparison to humans

1. Visual search := Data collection underway 2. Pointing and clicking moving targets := Task developed 3. Complex monitoring task := Task developed 4. Collaborative monitoring task

33