Human Factors Research Some OSU examples 1 Human Factors Research - - PowerPoint PPT Presentation

1

Human Factors Research Some OSU examples

2

Users, Operators, Subject Matter Experts

Human Factors Research to Inform the

Human-Machine Systems Engineering Process

Needs, Problems, Opportunities Operation,Test & Evaluation Analysis Design Implementation Design Specifications Requirements HMS: Humans, Machines, Processes

(Model, Mockup, Prototype, Product) Data Collection Data Analysis & Hypothesis Testing Interpretation & Application of Results Generalizable Research Question(s) HFE Principles & Guidelines Research Design Hypothesis Formulation

3

Human Factors Research: An Overview

• Experimental Methods
• Relationships studies

– independent variables → dependent variables

• Comparative studies
• Descriptive Methods
• Literature Review
• Observation
• Surveys and Questionnaires
• Incident and Accident Analysis
• Modeling and Simulation
• Meta-Analysis
• Always involve human subjects/participants (directly or

indirectly)

4

Human Factors and Aviation Safety

Source: Boeing Commercial Airplanes

Primary Causes of Aircraft Accidents

Hull Loss Accidents – Worldwide Commercial Jet Fleet – 1994 Through 2005

Maintenance Airport/ATC Misc./Other Weather Airplane Flight Crew

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

3% 5% 7% 13% 17% 55%

5

Human Factors Research Methods In OSU's Cockpit Task Management (CTM) Research

CTM: Process by which pilots selectively attend to multiple, concurrent flight tasks to safely and effectively complete a flight.

Lockheed L1011 Boeing 777

6

Developing a Conceptual Framework For CTM:

Literature Review, Analysis, and Modeling

• Literature Review
• Cockpit Resource Management (e.g., Lauber, 1986)
• Human error in aviation (e.g, Nagel, 1988; Wiener, 1987; Ruffel-Smith, 1979)
• Cognitive psychology (e.g., Navon & Gopher, 1979; Wickens, 1984)
• Systems theory (e.g., Padulo & Arbib, 1979)
• A Model of CTM
• initiate tasks to achieve goals
• assess status of all tasks
• terminate completed tasks
• prioritize remaining tasks based on

–

importance:

• 1. aviate
• 2. navigate
• 3. communicate
• 4. manage systems

–

urgency

–

• ther factors (?)
• allocate resources (attend) to tasks in order of priority

Funk, K.H. (1991). Cockpit Task Management: Preliminary Definitions, Normative Theory, Error Taxonomy, and Design Recommendations, The International Journal of Aviation Psychology,

• Vol. 1, No. 4, pp. 271-285.

7

Determining the Significance of CTM: Accident Analysis

• CTM Error Taxonomy
• Task Initiation: early / late / incorrect / lacking
• Task Prioritization: incorrect
• Task Termination: early / late / incorrect / lacking
• Method:
• Reviewed 324 National Transportation Safety Board (NTSB) Aircraft Accident Reports

(1960 – 1989)

• Developed pre-impact timelines, classified CTM errors
• Findings: 80 CTM errors in 76 (23%) of the accidents

Chou, C.D., D. Madhavan, and K.H. Funk (1996). Studies of Cockpit Task Management Errors, International Journal of Aviation Psychology, Vol. 6, No. 4, pp. 307-320. CTM Error # Accidents % CTM Accidents # CTM Errors % of All CTM Errors

Task Initiation 35 46 35 44 Task Prioritization 24 32 24 30 Task Termination 21 28 21 26

8

Determining the Significance of CTM: Incident Analysis

• Method:
• Reviewed 470 Aviation Safety Reporting System (ASRS) incident reports:

–

Controlled Flight Toward Terrain incidents

–

In-flight engine emergency incidents

–

Terminal flight phase incidents

• Identified concurrent tasks, classified CTM errors
• Findings: 231 (49%) of the incidents involved CTM errors

Chou, C.D., D. Madhavan, and K.H. Funk (1996). Studies of Cockpit Task Management Errors, International Journal of Aviation Psychology, Vol. 6, No. 4, pp. 307-320.

Conclusion: CTM is a significant factor in flight safety.

CTM Error # Incidents % CTM Incidents # CTM Errors % of All CTM Errors

Task Initiation 137 59 145 42 Task Prioritization 133 58 122 35 Task Termination 83 36 82 23

9

Understanding CTM: Incident Analysis

• Does cockpit automation level

affect task performance?

• Method:
• Reviewed 420 NASA ASRS

incident reports

–

210 advanced technology + 210 conventional technology

• large commercial transport

aircraft

• 2 pilots
• 1988-89, 1990-91, 1992-93
• Reviewed narratives
• Constructed task models
• Classified errors
• Comparison with t-tests
• Findings:
• Error rate higher for advanced

technology aircraft (p = 0.036)

• Error rate decreasing (p = 0.032)

Task Prioritization Error Frequency Total Errors by Submission Period Advanced Technology Traditional Technology Submission Period

1988-1989 13 7 20 1990-1991 11 5 16 1992-1993 4 3 7

Total Errors by Aircraft Technology

28 15 Wilson, J. and K. Funk (1998). The Effect of Automation on the Frequency of Task Prioritization Errors on Commercial Aircraft Flight Decks: An ASRS Incident Report Study, Proceedings of the Second Workshop on Human Error, Safety, and System Development, Seattle, WA, April 1-2, 1998, pp. 6-16.

10

Understanding CTM: Simulator Study

• What are the factors that affect task

prioritization in the CTM process?

• Method: simulator study
• Professional pilot participants
• Difficult San Francisco approach scenarios
• Task prioritization Challenge Probe Points (CPPs)
• Stop sim or record & replay for interviews on CPPs:

“Why did you ...?”

• Analysis with ANOVA
• Findings: Prioritization Factors

1. Procedural compliance 2. Task importance 3. Task salience 4. Task status 5. Time/Effort requirements 6. Task urgency

C A R M E A N J E E S K U N K B O L D R M E N L O O A K V O R S F O 2 8 R I L S B i g S u r V O R E 1 - C A R M E ( S c e n a r i o E v e n t ) T u r n f r o m V 2 7 t o 3 3 1 R a d i a l D M E = 8 3 . 9 E 2 - D M E 4 2 ( A T C E v e n t ) V e c t o r a n d A l t . I n s t r u c t i o n s D M E = 4 2 E 3 - B O L D R ( M a l f u n c t i o n E v e n t ) B u s T i e C o n t a c t o r ( M 4 ) D M E = 3 4 E 4 - V e c t o r 3 6 0 ( A T C E v e n t ) V e c t o r i n s t r u c t i o n D M E = 2 5 E 5 - L o c a l i z e r ( S c e n a r i o E v e n t ) L o c a l i z e r N e e d l e " S w i n g s " D M E = 1 7 . 6 E 6 - F i n a l ( M a l f u n c t i o n E v e n t ) B o o s t P u m p F a i l u r e ( M 5 ) D M E = 1 3 F l i g h t P a t h

E 1 - S c e n a r i o E v e n t E 2 - A T C E v e n t E 3 - M a l f u n c t i o n E v e n t E 4 - A T C E v e n t E 5 - S c e n a r i o E v e n t E 6 - M a l f u n c i t o n E v e n t 3 6 0 ° 3 2 5 ° D i r e c t i o n o f F l i g h t

Colvin, K., K. Funk, & R. Braune (2005). Task Prioritization Factors: Two Part-Task Simulator Studies, International Journal of Aviation Psychology, Vol. 15, No. 4, pp. 321–338.

11

Improving CTM: Experimental Study (Training)

• Can task prioritization be trained?
• APE Mnemonic: Assess, Prioritize, Execute
• Simulator Experiment
• Licensed pilot participants
• Independent variable: training (Descriptive,

Prescriptive, None/Control)

• Dependent Variables

–

Task Prioritization Error Rate

–

Prospective Memory Recall

• Flight – training / no training – flight
• ANOVA of results
• –

– – – –

Bishara, S. and K. Funk (2002). Training Pilots to Prioritize Tasks, Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting, Baltimore, MD, September 30-October 4, 2002, pp. 96-100.

0.5 0.6 0.7 0.8 0.9 1 Pre Training Post Training

P r

• s

p e c t i v e M e m

• r

y P e r f .

Prescriptive Descriptive Control

12

Improving CTM: Experiment (System Comparison)

• Can CTM be facilitated by a cockpit aid?
• AgendaManager (CTM aid) vs. EICAS (conventional pilot warning/alerting system)
• Simulator Experiment
• Professional pilot participants
• Independent Variables: Alerting (AMgr vs. EICAS), Scenario
• Dependent Variables: CTM metrics
• Flight 1 (EICAS/AMgr) – Flight 2 (Amgr/EICAS)
• ANOVA of results

Funk, K. and Braune, R. (1999). The AgendaManager: A Knowledge-Based System to Facilitate the Management

• f Flight Deck Activities, SAE 1999-01-5536. 1999 World Aviation Congress, 19-21 October 1999, San

Francisco, CA.

Dependent Variable AMgr EICAS

sig.

Within subs. correct prioritization 100% 100% NS

• Subs. fault correction time (sec)

19.5 19.6 NS A/F programming time (sec) 7.9 5.9 NS goal conflicts % corrected 100% 70% 0.10 goal conflict resolution time (sec) 34.7 53.6 0.10 Subs./Aviate correct prioritization 72% 46% 0.05 Mean # unsatisfactory tasks 0.64 0.85 0.05 % time all tasks satisfactory 65% 52% 0.05 Mean participant rating (-5 - +5) 4.8 2.5 0.05

Findings

13

Flight Deck Automation Issues Research:

Literature Review, Surveys, Accident/Incident Analyses, Meta-Analysis

Funk, K., B. Lyall, J. Wilson, R. Vint, M. Niemczyk, C. Suroteguh, and G. Owen (1999). Flight Deck Automation Issues, International Journal of Aviation Psychology, Vol. 9, No. 2, pp. 109-123. 1. Automation may demand attention. 2. Automation behavior may be unexpected and unexplained. 3. Pilots may be overconfident in automation. 4. Behavior of automation may not be apparent. 5. Failure assessment may be difficult. 6. Mode transitions may be uncommanded. 7. Mode awareness may be lacking. 8. Mode selection may be incorrect. 9. Situation awareness may be reduced.

• 10. Understanding of automation

may be inadequate.

Top 10:

L1011 vs. B777

14

Operating Room Human Factors Research: Observation and Modeling

• Observed surgical procedures in Oregon hospital.
• Interviewed surgeons, nurses, assistants.
• Developed IDEF0 process (functional) model of

laparoscopic cholecystectomy (gall bladder removal using minimally invasive procedures) for future error identification.

Funk, K.H., T.L. Doolen, R. Botney, and J.D. Bauer, “A Functional Model of the Operating Room,” Proceedings of the Human Factors and Ergonomics Society 47th Annual Meeting, October 13-17, 2003, Denver, CO, pp. 1569-1573.

15

Operating Room Human Factors Research: Systemic Vulnerabilities Analysis

• Systemic vulnerabilities to Verres

Needle insertion

• Improperly visualize underlying anatomy.
• Miss important cue for needle

placement.

• Choose wrong insertion angle.
• Fail to stabilize abdominal wall.
• Misinterpret the degree of needle

resistance ...

• Err in sensing the click of the needle
• etc.
• Validated by post hoc literature review.

Funk, K.H., J.D. Bauer, T.L. Doolen, D. Telasha, R.J. Nicolalde, M. Reeber, N. Yodpijit, and M. Long (2010). The use of modeling to identify vulnerabilities to human error in laparoscopy, The Journal

• f Minimally Invasive Gynecology, Vol.

17, No. 3, pp. 311-320.

FMEA+

16

Operating Room Distractions and Interruptions

Research in collaboration with OHSU Department of Surgery

• Simulated laparoscopic cholecystectomy
• 18 OHSU 2nd & 3rd year surgical residents
• Independent Variable: Distracted vs non-

distracted

• Dependent Variables:
• Damage to organs
• Collateral blood loss
• Remembering to announce closure
• Total and cauterizing times
• Results: 8 out of 18 committed errors when

distracted versus 1 out of 18 when not distracted

Distractions/Interruptions # Errors

• Visual movement
• Ringing cell phone

1

• Question about “crashing” patient

4

• Side conversation

3

• Question about choice of profession

2

• Dropped metal tray

Feuerbacher, R.L.,Funk II, K.H., Spight, D.H., Diggs, B.S., Hunter, J.G. (2012). Realistic distractions and interruptions impair simulated surgical performance by novice surgeons, Archives of Surgery, http://archsurg.jamanetwork.com/article.aspx?articleid=1216543.

17

A Comparison of Human and Near-Optimal Task Management Behavior

• Objectives
• Framework for studying human TM behavior &

performance

• Study human TM strategy, tactics
• Compare human TM performance with near-
• ptimal heuristic (tabu search)
• Background
• Common TM situations
• Engineering models of TM
• Method: “Simulator” Experiment
• Apparatus: Tardast TM “game”
• Participants: 10 OSU students
• 5 randomized scenarios
• IVs: DRs, CRs, Ws
• DVs: scores, strategies, tactics
• Compared with tabu search

heuristic

• Results
• Human < tabu (not by much)
• Different strategies, tactics
• Conclusions
• Too many tasks → too few
• Over-attention to salient stimuli
• Tardast a useful framework

Shakeri, S., Funk, K. (2007). A comparison of human and near-optimal task management behavior, human factors, vol. 49, No. 3, pp. 400–416.

18

Human Factors Research: Summary

• Experimental Methods
• Relationships studies

– independent variables → dependent variables

• Comparative studies
• Descriptive Methods
• Literature Review
• Observation
• Surveys and Questionnaires
• Incident and Accident Analysis
• Modeling and Simulation
• Meta-Analysis
• Always involve human subjects/participants (directly or

indirectly)

19

Users, Operators, Subject Matter Experts

Human Factors Research to Inform the

Human-Machine Systems Engineering Process

Needs, Problems, Opportunities Operation,Test & Evaluation Analysis Design Implementation Design Specifications Requirements HMS: Humans, Machines, Processes

(Model, Mockup, Prototype, Product) Data Collection Data Analysis & Hypothesis Testing Interpretation & Application of Results Generalizable Research Question(s) HFE Principles & Guidelines Research Design Hypothesis Formulation