Testing & Validation Human Workload Modeling for Autonomous - - PowerPoint PPT Presentation

testing validation
SMART_READER_LITE
LIVE PREVIEW

Testing & Validation Human Workload Modeling for Autonomous - - PowerPoint PPT Presentation

Modeling & Simulation, Testing & Validation Human Workload Modeling for Autonomous Ground Vehicles Presen enting ting: Dr. C.J. J. Hutto, Senior Research Scientist, Human Systems Engineering Branch, Georgia Tech Research Institute


slide-1
SLIDE 1

Modeling & Simulation, Testing & Validation

8/22/2018

Human Workload Modeling for Autonomous Ground Vehicles

Presen enting ting: Dr. C.J.

  • J. Hutto, Senior Research Scientist, Human Systems

Engineering Branch, Georgia Tech Research Institute (GTRI)

  • Dr. Vl

Vlad Pop, Former Research Faculty/Research Scientist, GTRI

  • W. Stuart

t Michel elson, son, Research Faculty/Research Scientist, GTRI

GTRI

slide-2
SLIDE 2

Modeling & Simulation, Testing & Validation

Motivation

  • Autonomous Ground Systems (AGS) play a significant role

in the DoD’s Third Offset Strategy.

8/22/2018 GTRI

slide-3
SLIDE 3

Modeling & Simulation, Testing & Validation

Motivation

  • AGS & Human-Automation Interaction
  • To achieve optimal workload in AGS, designers must be able

to assess the operator workload levels.

Pros

Decrease driving workload Increase in efficiency Improvements in safety

Cons

Introduce new cognitive demands Inconsistent attentional demands New type of driving mgt tasks

8/22/2018 GTRI

slide-4
SLIDE 4

Modeling & Simulation, Testing & Validation

Background

  • Human Workload Modeling: Multiple Resource Theory (MRT)
  • Wickens (1984) argued that human workload is not the result of one central processing

resource, but rather multiple resource channels (Wickens, C. D., 1984)

  • Tasks can be performed concurrently, but will interfere with each other
  • Increasing the resource demands exercised by one task will decrease resource

availability for another task (typically, with performance degradation)

8/22/2018 GTRI

slide-5
SLIDE 5

Modeling & Simulation, Testing & Validation

Background

  • Applying MRT via VCAP Workload Scales
  • Consistent with MRT, McCracken and Aldrich (1984) describe workload as a function of

several processing resources: visual, cognitive, auditory, and psychomotor (VCAP)

Visual Auditory 1.0 Visually Register/Detect (detect occurrence of image) 1.0 Detect/Register Sound (detect occurrence of sound) 3.7 Visually Discriminate (detect visual differences) 2.0 Orient to Sound (general orientation/attention) 4.0 Visually Inspect/Check (discrete inspection/static condition) 4.2 Orient to Sound (selective orientation/attention) 5.0 Visually Locate/Align (selective orientation) 4.3 Verify Auditory Feedback (detect anticipated sound) 5.4 Visually Track/Follow (maintain orientation) 4.9 Interpret Semantic Content (speech) 5.9 Visually Read (symbol) 6.6 Discriminate Sound Characteristics (detect auditory differences) 7.0 Visually Scan/Search/Monitor (continuous/serial inspection) 7.0 Interpret Sound Patterns (pulse rates, etc.) Cognitive Psychomotor 1.0 Automatic (simple association) 1.0 Speech 1.2 Alternative Selection 2.2 Discrete Actuation (button, toggle, trigger) 3.7 Sign/Signal Recognition 2.6 Continuous Adjustive (flight control, sensor control) 4.6 Evaluation/Judgment (consider single aspect) 4.6 Manipulative 5.3 Encoding/Decoding, Recall 5.8 Discrete Adjustive (rotary, thumbwheel, lever position) 6.8 Evaluation/Judgment (consider several aspects) 6.5 Symbolic Production (writing) 7.0 Estimation, Calculation, Conversion 7.0 Serial Discrete Manipulation (keyboard entries)

8/22/2018 GTRI

slide-6
SLIDE 6

Modeling & Simulation, Testing & Validation

Background

  • Computerized Discrete-Event Simulation (DES) and the

Task Network Model

Analyst input variables affecting workload:

  • System design alternatives (e.g., automation, layout, workflow

process, etc.)

  • Crew size, composition/attributes, & division of labor
  • Scripted, event-driven simulation scenarios, e.g.,
  • Typical mission scenario (demanding but realistic)
  • Atypical intense situation (time critical, dangerous)

DES TN model determinants of workload:

  • MRT (VCAP) related task demand
  • Task time distribution variability
  • Mean, standard deviation, distribution type
  • Rule-based task release conditions, e.g.,
  • Operator(s) must be capable (qualified/trained), able (reach, see,

etc), and available (not busy, not incapacitated, not off duty, etc)

  • Task sequencing/branching logic
  • Task beginning and ending effects, e.g.,
  • Make operator(s) “unavailable”
  • Increase/decrease or start/stop workload metrics
  • Task queue discipline (FIFO, LIFO, HAF)

8/22/2018 GTRI

slide-7
SLIDE 7

Modeling & Simulation, Testing & Validation

Surrogate AGS Example

  • Modeling Human Workload in a Toy Example

GM Cadillac Super Cruise Tesla Model S Autopilot

http://www.thedrive.com/tech/17083/the-battle-for-best-semi-autonomous-system-tesla-autopilot-vs-gm-supercruise-head-to-head 8/22/2018 GTRI

slide-8
SLIDE 8

Modeling & Simulation, Testing & Validation

Systems Comparison

  • Primary Differences
  • Domain (GPS-limited to pre-mapped roads vs intuitive cruise control-like conditions)
  • Updates (OTA w/ “Fleet Learning” vs Quarterly dealer-managed updates and company-only

learning)

  • Ease of System Activation/Engagement (and [accidental] disengagement)
  • Driving Mode Awareness, and Mode Transition Awareness
  • Hands-on vs Hands-off, and Driver Monitoring System
  • Lane Changing, Lane Keeping, and Radar Handling for Cut-ins

GM Cadillac Super Cruise Tesla Model S Autopilot

8/22/2018 GTRI

slide-9
SLIDE 9

Modeling & Simulation, Testing & Validation

Systems Comparison

  • Primary Differences
  • Driving Situation Awareness

GM Cadillac Super Cruise Tesla Model S Autopilot

8/22/2018 GTRI

slide-10
SLIDE 10

Modeling & Simulation, Testing & Validation

Surrogate Example Results

  • Found greater visual workload with the system indicating

that the automation is engaged by changing the color of an icon versus the additional presence of light or image.

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 Engaging Automation Confirming Engagement "Driving" with Automation Automation Disengagement Manual Disengagement

Visual Workload Rating

Visual Workload

Manual Driving Tesla Autopilot Cadillac Super Cruise™ 8/22/2018 GTRI

slide-11
SLIDE 11

Modeling & Simulation, Testing & Validation

Surrogate Example Results

  • The hands free system resulted in lower workload than the

hands on system

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 Engaging Automation Confirming Engagement "Driving" with Automation Automation Disengagement Manual Disengagement

Psychomotor Workload Rating

Psychomotor Workload

Manual Driving Tesla Autopilot Cadillac Super Cruise™ 8/22/2018 GTRI

slide-12
SLIDE 12

Modeling & Simulation, Testing & Validation

Surrogate Example Results

  • Only slight auditory differences between systems

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 Engaging Automation Confirming Engagement "Driving" with Automation Automation Disengagement Manual Disengagement

Auditory Workload Rating

Auditory Workload

Manual Driving Tesla Autopilot Cadillac Super Cruise™ 8/22/2018 GTRI

slide-13
SLIDE 13

Modeling & Simulation, Testing & Validation

Surrogate Example Results

  • Similar cognitive demand for both systems

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 Engaging Automation Confirming Engagement "Driving" with Automation Automation Disengagement Manual Disengagement

Cognitive Workload Rating

Cognitive Workload

Manual Driving Tesla Autopilot Cadillac Super Cruise™ 8/22/2018 GTRI

slide-14
SLIDE 14

Modeling & Simulation, Testing & Validation

Surrogate Example Results

  • Increased workload during automation engagement and

automation disengagement but decreased workload while automation was in use

0.0 7.0 14.0 21.0 28.0 Engaging Automation Confirming Engagement "Driving" with Automation Automation Disengagement Manual Disengagement

Total Workload Rating

Total Workload

Manual Driving Tesla Autopilot Cadillac Super Cruise™ 8/22/2018 GTRI

slide-15
SLIDE 15

Modeling & Simulation, Testing & Validation

Conclusions

  • A (purposely) simple example demonstrates several key concepts:

– Workload modeling can be useful for objectively comparing systems, comparing design alternatives, or evaluating automation design decisions to help optimize human-machine interaction. – Overall this type of analysis shows where the addition of automation is beneficial to operator workload and where it is not. – The analysis also reveals some differences between the two systems.

  • M&S approach scales easily to a variety of driving tasks and

scenarios, system design alternatives, environmental conditions, etc.

  • M&S approach extends beyond evaluation of system design;

simultaneously gain insights about human and system performance

8/22/2018 GTRI

slide-16
SLIDE 16

Modeling & Simulation, Testing & Validation

Questions

  • Contact the author:
  • Mr. Stuart Michelson

Stuart.Michelson@gtri.gatech.edu 404.407.6162 Georgia Tech Research Institute

  • Contact the presenter:
  • Dr. C.J. Hutto

cjhutto@gatech.edu 404.407.6887 Georgia Tech Research Institute