Adaptive Robotics for Jamison Heard Vanderbilt University PhD Candidate High-Stress Environments May 11 th 2018
Human-Robotic Interaction 2 Peer Mentor Roles Information Consumer
High Stress Environments 3 Supervisory Peer
Workload and Performance 4 Good Performance Acceptable Level Poor Underload Normal Load Overload Overall Workload
Workload Components 5 Overall Workload Cognitive Physical Auditory Visual Speech
Objective Workload Metrics 6 Cognitive Physical Postural Measures Brain-Activity Measures Heart-Rate Cardiovascular Measures Skin Temperature Visual Auditory Speech Noise Level Eye-Tracking Measures Speech-Response Time Speech-Based Measures Respiration-Rate Noise Level
Challenges with Physiological Signals 7 Age Individual Differences Physical Fitness Training Circadian Rhythms Day-to-Day Variability Stressful Events
8 Workload Assessment Algorithm Development and Validation
Workload Assessment Algorithm 9
Experimental Design 10 Supervisory-Based Evaluation Peer-Based Evaluation
Experimental Design 11 Independent Variables Dependent Variables • Workload • Subjective Workload Metrics • Peer • NASA TLX • In-Situ Workload Ratings • Low and High • Supervisory • Objective Workload Metrics • Underload, Normal Load, and • Heart-Rate Overload • Heart-Rate Variability • Peer manipulated teaming • Skin-Temperature partner • Noise Level • Human or Robot • Respiration-Rate • Posture Magnitude
Peer-Based Evaluation 12 18 Participants 4 Tasks: Liquid and Solid 1. Photo-Search Hallway Search Containment Sampling
Supervisory-Based Evaluation 13 30 Participants 4 Concurrent Tasks: • Tracking • System Monitoring • Resource Management • Communications NASA MATB
Algorithm Evaluation 14 • Four Analysis: • Population Generalizability • Train on 70% of the Participants, test on the other 30% • Cross-Teaming Generalizability • Train on Peer Evaluation Data, Test on Supervisory • Vice-Versa • Cross-Task Generalizability • Train on 3 Peer-based Tasks, Test on the 4 th task • 4-Fold Cross-Validation • Cross-Interaction Generalizability • Train on Peer Human-Human Teaming Data, Test on Human-Robot Teaming • Vice-Versa
Results: Population & Cross-Teaming 15 Average Classification Accuracy (%) by Workload Component and Evaluation 100 100 99 98 98 98 96 95 96 95 90 90 67 88 66 63 81 63 SUP PEER BOTH SUP PEER BOTH SUP PEER BOTH Cognitive Workload Physical Workload Overall Workload Peer Evaluation Supervisory Evaluation
Results: Cross-Task 16 Average Classification Accuracy (%) by Workload Component and Peer Task 99 96 94 94 93 90 89 89 83 82 76 66 T1 T2 T3 T4 T1 T2 T3 T4 T1 T2 T3 T4 Cognitive Workload Physical Workload Overall Workload Peer Evaluation T# represents peer-task number
Results: Cross-Interaction 17 Average Classification Accuracy (%) by Workload Component and Interaction Paradigm 96 95 95 94 94 93 90 87 86 H-H H-R HH-HR H-H H-R HH-HR H-H H-R HH-HR Cognitive Workload Physical Workload Overall Workload Peer Evaluation H-H : Human-Human, H-R : Human-Robot, HH-HR : Human-Human and Human-Robot
Envisioned System 18 Adaptive Workload System Architecture Future Tasks Activity Recognition Task Current Tasks (Re-)Allocations i-CiFHaR Workload Models Workload Component Workload Interaction Estimates Metrics Decision Workload Framework Assessment Algorithm Predicted Interaction Performance Changes Communication Performance Modality Prediction Assess and Predict Determine Adaptations Apply Adaptations
19 Activity Recognition
Activity Recognition Background 20 Cameras External Sensors Environmental Accelerometer Wearable Sensors Gyroscope Physiological
Current EMS Hand-Off Process 21 https://www.mlrems.org/patient-handoff/
EMS Procedures 22 Intravenous Airway Therapy Management • Nasal Airway • Placing an IV • Oral Airway • Administer IV Medication • Intubation • IO Line Wound and Vital Fractures Checking High Trauma • Tourniquet • Stethoscope • Combat Gauze • Placing Monitoring Equipment • Crike Kit • Splinting • Chest Decompression • Cardiopulmonary Resuscitation (CPR)
Hierarchical Task Analysis 23 Check for Breathing Preparation Lift Patient’s Chin Use Bag-Valve Mask CPR Give Breaths Use Mouth Compressions 30 Chest Compressions
Wearable Sensors 24 Apple Watch MYO • Accelerometer • Accelerometer • Gyroscope • Gyroscope • EMG
Accelerometer Data Example 25 CPR Compressions Breath
Video: Open Pose 26
Open Pose Challenges 27
Envisioned System 28 Automatic EMS Procedure Detection Architecture Wearable Sensors Triage Score Activity OpenPose Generation Recognition Heat Map Health Database Generation
Recap 29 • Workload Assessment • Uses objective workload metrics to derive workload estimates for overall workload and its contributing components • Analyzed algorithm across populations, human-robotic teaming paradigms, and cross-tasks • Needs contextual information about the human’s current task • Automatic EMS Procedure Detection • Uses wearable and visual sensor data • Procedures are decomposed into sub-tasks for easier recognition
30 Questions??
Recommend
More recommend
