SLIDE 1
Adaptive Robotics for High-Stress Environments
Jamison Heard Vanderbilt University PhD Candidate May 11th 2018
Peer Mentor
Information
Consumer
Roles Human-Robotic Interaction
2
High-Stress Environments May 11 th 2018 Human-Robotic Interaction 2 - - PowerPoint PPT Presentation
High-Stress Environments May 11 th 2018 Human-Robotic Interaction 2 - - PowerPoint PPT Presentation
Adaptive Robotics for Jamison Heard Vanderbilt University PhD Candidate High-Stress Environments May 11 th 2018 Human-Robotic Interaction 2 Peer Mentor Roles Information Consumer High Stress Environments 3 Supervisory Peer Workload
SLIDE 2
SLIDE 3
High Stress Environments
Peer Supervisory
3
SLIDE 4
Workload and Performance
Overall Workload Performance
Poor Good
Underload Overload Normal Load Acceptable Level
4
SLIDE 5
Workload Components
Overall Workload Cognitive Physical Auditory Visual Speech
5
SLIDE 6
Objective Workload Metrics
Cognitive Physical Auditory Visual Speech
Brain-Activity Measures Cardiovascular Measures Eye-Tracking Measures Speech-Based Measures Respiration-Rate Noise Level Noise Level Speech-Response Time Postural Measures Heart-Rate Skin Temperature
6
SLIDE 7
Challenges with Physiological Signals
Individual Differences Day-to-Day Variability Age Physical Fitness Training Circadian Rhythms Stressful Events
7
SLIDE 8
Workload Assessment Algorithm Development and Validation
8
SLIDE 9
Workload Assessment Algorithm
9
SLIDE 10
Experimental Design
Peer-Based Evaluation Supervisory-Based Evaluation
10
SLIDE 11
Experimental Design
Independent Variables
- Workload
- Peer
- Low and High
- Supervisory
- Underload, Normal Load, and
Overload
- Peer manipulated teaming
partner
- Human or Robot
Dependent Variables
- Subjective Workload Metrics
- NASA TLX
- In-Situ Workload Ratings
- Objective Workload Metrics
- Heart-Rate
- Heart-Rate Variability
- Skin-Temperature
- Noise Level
- Respiration-Rate
- Posture Magnitude
11
SLIDE 12
Peer-Based Evaluation
18 Participants 4 Tasks:
- 1. Photo-Search
Hallway Search Liquid and Solid Containment Sampling
12
SLIDE 13
Supervisory-Based Evaluation
30 Participants 4 Concurrent Tasks: NASA MATB
- Tracking
- System Monitoring
- Resource Management
- Communications
13
SLIDE 14
Algorithm Evaluation
14
- Four Analysis:
- Population Generalizability
- Train on 70% of the Participants, test on the other 30%
- Cross-Teaming Generalizability
- Train on Peer Evaluation Data, Test on Supervisory
- Vice-Versa
- Cross-Task Generalizability
- Train on 3 Peer-based Tasks, Test on the 4th task
- 4-Fold Cross-Validation
- Cross-Interaction Generalizability
- Train on Peer Human-Human Teaming Data, Test on Human-Robot Teaming
- Vice-Versa
SLIDE 15
SUP PEER BOTH SUP PEER BOTH SUP PEER BOTH Cognitive Workload Physical Workload Overall Workload
88 96 95 63 90 81 90 96 95 98 63 98 99 66 98 100 67 100
Average Classification Accuracy (%) by Workload Component and Evaluation
Peer Evaluation Supervisory Evaluation
15
Results: Population & Cross-Teaming
SLIDE 16
Results: Cross-Task
T1 T2 T3 T4 T1 T2 T3 T4 T1 T2 T3 T4 Cognitive Workload Physical Workload Overall Workload
76 96 94 89 82 83 93 66 94 89 90 99
Average Classification Accuracy (%) by Workload Component and Peer Task
Peer Evaluation
16
T# represents peer-task number
SLIDE 17
Results: Cross-Interaction
H-H H-R HH-HR H-H H-R HH-HR H-H H-R HH-HR Cognitive Workload Physical Workload Overall Workload
94 95 96 87 90 86 94 95 93
Average Classification Accuracy (%) by Workload Component and Interaction Paradigm
Peer Evaluation
H-H: Human-Human, H-R: Human-Robot, HH-HR: Human-Human and Human-Robot
17
SLIDE 18
Workload Assessment Algorithm
Determine Adaptations Assess and Predict
Performance Prediction
i-CiFHaR
Interaction Decision Framework Workload Component Estimates Predicted Performance Task (Re-)Allocations Interaction Changes Workload Metrics Activity Recognition Workload Models
Adaptive Workload System Architecture
Current Tasks Future Tasks Communication Modality
Apply Adaptations
Envisioned System
18
SLIDE 19
Activity Recognition
19
SLIDE 20
Activity Recognition Background
20
External Sensors Wearable Sensors Cameras Environmental Physiological Accelerometer Gyroscope
SLIDE 21
Current EMS Hand-Off Process
https://www.mlrems.org/patient-handoff/
21
SLIDE 22
EMS Procedures
22
Airway Management
- Placing an IV
- Administer IV Medication
- IO Line
- Nasal Airway
- Oral Airway
- Intubation
- Crike Kit
- Chest Decompression
- Cardiopulmonary Resuscitation (CPR)
- Tourniquet
- Combat Gauze
- Splinting
- Stethoscope
- Placing Monitoring Equipment
Intravenous Therapy Vital Checking Wound and Fractures High Trauma
SLIDE 23
Hierarchical Task Analysis
23
CPR Preparation Give Breaths Compressions Check for Breathing Lift Patient’s Chin Use Bag-Valve Mask Use Mouth 30 Chest Compressions
SLIDE 24
Wearable Sensors
Apple Watch
- Accelerometer
- Gyroscope
MYO
- Accelerometer
- Gyroscope
- EMG
24
SLIDE 25
Accelerometer Data Example
Compressions Breath CPR
25
SLIDE 26
Video: Open Pose
26
SLIDE 27
Open Pose Challenges
27
SLIDE 28
Envisioned System
28
Wearable Sensors OpenPose Activity
Recognition Automatic EMS Procedure Detection Architecture
Heat Map Generation Health Database Triage Score Generation
SLIDE 29
Recap
29
- Workload Assessment
- Uses objective workload metrics to derive workload estimates for overall
workload and its contributing components
- Analyzed algorithm across populations, human-robotic teaming paradigms,
and cross-tasks
- Needs contextual information about the human’s current task
- Automatic EMS Procedure Detection
- Uses wearable and visual sensor data
- Procedures are decomposed into sub-tasks for easier recognition
SLIDE 30
Questions??
30