IE 507, Human-Centered Design Seminar Human Factors (and Human - - PowerPoint PPT Presentation

1

IE 507, Human-Centered Design Seminar Human Factors

(and Human Factors-Related)

Research (and Development)

• Dr. Kenneth Funk
• Mr. Sami Al-AbdRabbuh
• Mr. Steven Hattrup

School of Mechanical, Industrial, and Manufacturing Engineering Friday 25 May 2018

2

Human Factors and Aviation Safety

Source: Boeing Commercial Airplanes

Primary Causes of Aircraft Accidents

Hull Loss Accidents – Worldwide Commercial Jet Fleet – 1994 Through 2005

Maintenance Airport/ATC Misc./Other Weather Airplane Flight Crew

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

3% 5% 7% 13% 17% 55%

3

Cockpit Task Management (CTM) Research

CTM: Process by which pilots selectively attend to multiple, concurrent flight tasks to safely and effectively complete a flight.

Lockheed L1011 Boeing 777

4

Determining the Significance of CTM: Accident Analysis

• CTM Error Taxonomy
• Task Initiation: early / late / incorrect / lacking
• Task Prioritization: incorrect
• Task Termination: early / late / incorrect / lacking
• Method:
• Reviewed 324 National Transportation Safety Board (NTSB) Aircraft Accident Reports

(1960 – 1989)

• Developed pre-impact timelines, classified CTM errors
• Findings: 80 CTM errors in 76 (23%) of the accidents

Chou, C.D., D. Madhavan, and K.H. Funk (1996). Studies of Cockpit Task Management Errors, International Journal of Aviation Psychology, Vol. 6, No. 4, pp. 307-320. CTM Error # Accidents % CTM Accidents # CTM Errors % of All CTM Errors

Task Initiation 35 46 35 44 Task Prioritization 24 32 24 30 Task Termination 21 28 21 26

5

Improving CTM: Training Experiment

• Can task prioritization be trained?
• APE Mnemonic: Assess, Prioritize, Execute
• Simulator Experiment
• Licensed pilot participants
• Independent variable: training (Descriptive,

Prescriptive, None/Control)

• Dependent Variables

–

Task Prioritization Error Rate

–

Prospective Memory Recall

• Flight – training / no training – flight
• ANOVA of results
• –

– – – –

Bishara, S. and K. Funk (2002). Training Pilots to Prioritize Tasks, Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting, Baltimore, MD, September 30-October 4, 2002, pp. 96-100.

Interaction Plot

P r i

• r

i t i z a t i

• n

E r r

• r

R a t e

Control Descriptive Prescriptive

0.5 0.6 0.7 0.8 0.9 1 Pre Training Post Training

P r

• s

p e c t i v e M e m

• r

y P e r f .

Prescriptive Descriptive Control

6

Flight Simulator Studies of Task Prioritization Factors

7

Operating Room Distractions and Interruptions

Research in collaboration with OHSU Department of Surgery

• Simulated laparoscopic cholecystectomy
• 18 OHSU 2nd & 3rd year surgical residents
• Independent Variable: Distracted vs non-

distracted

• Dependent Variables:
• Damage to organs
• Collateral blood loss
• Remembering to announce closure
• Total and cauterizing times
• Results: 8 out of 18 committed errors when

distracted versus 1 out of 18 when not distracted

Distractions/Interruptions # Errors

• Visual movement
• Ringing cell phone

1

• Question about “crashing” patient

4

• Side conversation

3

• Question about choice of profession

2

• Dropped metal tray

Feuerbacher, R.L.,Funk II, K.H., Spight, D.H., Diggs, B.S., Hunter, J.G. (2012). Realistic distractions and interruptions impair simulated surgical performance by novice surgeons, Archives of Surgery, http://archsurg.jamanetwork.com/article.aspx?articleid=1216543.

8

Flight Deck Automation Issues Research:

Literature Review, Surveys, Accident/Incident Analyses, Meta-Analysis

Funk, K., B. Lyall, J. Wilson, R. Vint, M. Niemczyk, C. Suroteguh, and G. Owen (1999). Flight Deck Automation Issues, International Journal of Aviation Psychology, Vol. 9, No. 2, pp. 109-123. 1. Automation may demand attention. 2. Automation behavior may be unexpected and unexplained. 3. Pilots may be overconfident in automation. 4. Behavior of automation may not be apparent. 5. Failure assessment may be difficult. 6. Mode transitions may be uncommanded. 7. Mode awareness may be lacking. 8. Mode selection may be incorrect. 9. Situation awareness may be reduced.

• 10. Understanding of automation

may be inadequate.

Top 10:

L1011 vs. B777

9

Experiments & Trade Studies to Inform Targeting Device UI Specifications

Laser Target Locator Module (LTLM): TRIGR LTLM: MARK VIIE OSU Emulators: Targeting Device STORM (not available) Small Tactical Optical Rifle Mounted (STORM) Micro-Laser Rangefinder

(on forward rail of M4 carbine)

Medical Device R&D:

Healthcare Toolkit, West African Infectious Diseases Diagnosis Aid

Requirements Development for a Resilient Emergency Management System in a Local Jurisdiction

Sami Al-AbdRabbuh 25 May 2018

Outline

• What is the need?
• Conventional approaches to meeting the need?
• Why are conventional approaches inadequate?
• What is the science that might help meet the

need?

• Why is that inadequate by itself?
• What can Human factors approach contribute to

this?

• Objective and research question

What is the need?

• What is the need for a human-centered design

in Emergency Management (EM)?

– Emergency management systems often include time critical and cognitively demanding tasks that humans perform on machines, computers, and

• ther systems.

What is the role of an emergency manager?

• The role of a local emergency manager entails:
• 1. Managing resources before, during, and after a

major emergency or disaster.

• 2. Conducting activities related to the critical

components of emergency management.

• 3. Coordinating with all partners in the emergency

management process

https://emilms.fema.gov/IS0230d/FEM0104040text.htm

• What is the need?

For an emergency management person or entity to accomplish its tasks, there must be an understanding of what are the information requirements and decisions criteria that informs EM activities, EM systems design, and EM decisions.

Conventional approaches to meeting the need

• Homeland Security Exercise and Evaluation

Program (HSEEP)

Produces After-Action Reports/Improvement Plans (AAR/IP).

• Why are conventional approaches inadequate?

– It does not explicitly call out the requirements generation phase. – Rarely drives a deep understanding of what should emergency managers do in a local interagency level – Does not always help in driving needed change

What is the science that might help meet the need?

• Comprehensive emergency management

theory

• Task Analysis tools
• Balanced Scorecards

Emergency Management Theory

• Comprehensive Emergency Management

– Phases – Planning P Process

• Comprehensive functional emergency

management (CFEM) framework.

Source: FEMA

http://www.nfrmp.us/state/about.cfm

What tasks should an emergency manager focus on?

UTL All-Hazards Taxonomy of National Preparedness Tasks (Universal all-Hazards Taxonomy) 11

Task Analysis Methods

They are used to determine and describe the physical description of an activity performed within complex systems. Cognitive Tasks Analysis methods describe the cognitive processes used by agents. Task Analysis tools help in understanding the requirements for successful completion of these tasks. (Stanton,1960)

How to analyze tasks and generate requirements

Goals Decisions Information requirements

Cognitive Demand Table

Task Diagram Knowledge Audit Simulation Interview

GOAL DIRECTED COGNITIVE TASK ANALYSIS (GDTA)

APPLIED COGNITIVE TASK ANALYSIS (ACTA)

14

Prasanna, Yang, and King (2009)

How to analyze tasks and generate requirements of an EM system

Goal Directed Information Analysis (GDIA)

Obtain Information Requirements of specific EM roles in a specific type of hazard

Identify Decisions being made & information to make such decisions Prepare Goal-Decision-Information Diagrams Validate obtained records Goal Hierarchy validation à Obtain Goals (Cognitive Demands) Scenario Building à Identifying Physical Tasks

Context Discovery

Prasanna, Yang, and King (2009)

15

Obtain Information Requirements of specific EM roles in a specific type of hazard

Identify Decisions being made & information to make such decisions

Prepare Goal-Decision- Information Diagrams

Validate obtained records

Goal Hierarchy validation à

Obtain Goals (Cognitive Demands)

Scenario Building à Identifying Physical Tasks

Context Discovery

Prasanna, Yang, and King (2009)

16

Balanced Scorecards (BSC)

• BSCs in Business
• BSCs for natural

disaster management

Moe, T. L., Gehbauer, F., Senitz, S., & Mueller, M. (2007, 11)

17

Financial perspective Customer perspective Internal business perspective Innovation and learning perspective Donors’ perspective Target beneficiaries perspective Internal business perspective Innovation and learning perspective

Aspects to consider in a BSC

• Balanced Scorecard for natural disaster management

helps identify and prioritize strategies and actions.

Moe, T. L., Gehbauer, F., Senitz, S., & Mueller, M. (2007, 11)

18

• Revenue growth
• Productivity (Kaplan and Norton, 2000)

Donors’ perspective

• Time
• Quality
• Performance and service
• Costs (Kaplan and Norton, 1992)

Target beneficiaries perspective

• Reflected in measuring internal processes success (Kaplan and Norton, 2000)

Internal business perspective

• Human capital
• Information capital
• Organization capital (Kaplan and Norton, 2000)

Innovation and learning perspective

Moe, T. L., Gehbauer, F., Senitz, S., & Mueller, M. (2007, 11)

19

Phases BSC areas BSC measures G Y R Average Prediction Donors Matching policy priorities and raising the interests of key stakeholders G Donors Approval and commitment of sufficient resources to the project by key stakeholders G

Target beneficiaries

Addressing relevant risks and needs

• f the right target group of

beneficiaries G Internal Business Correct implementing agency, which is able and willing to deliver services Y IL Sharing previous lessons learned with stakeholders Y Total compliance 80% Warning … … Example of a balanced score card where information and decision requirements of an emergency management system are identified as BSC measures

Why EM theory is inadequate by itself?

• Systematic and human centered improvement

in an emergency management system cannot happen without integrating the theory with the appropriate tools, and the conventional approaches together.

What can Human factors approach this contribute?

We use task analysis to

• 1. develop a normative theory of EM (enhanced over UTL)
• 2. and to generate information requirements and develop

performance metrics)

Objective and research question

Objective: To develop a successful protocol to generate information requirements and decisions criteria a local jurisdiction (such as an Oregonian County) need to obtain for a successful Emergency Management planning, mitigation, and recovery.

Research question:

• How should an Emergency Management planning,

mitigation, and recovery work? What is the information (framework) needed to make it work?

The Design of Good Work

May 25, 2018

By: Steven Henry Hattrup

Major Professor: Kenneth H. Funk II, PhD College of Engineering: PhD Student in Industrial Engineering

THE NEED: Work can be stressful

May 25, 2018 1

Foxconn

• Highly standardized
• Highly efficient
• employee suicides
• Reaction: install nets
• (Perlin, 2013)
• Good Work?

THE NEED: Work can be harmful

May 25, 2018 2

Firgure 1: Foxconn Anti-Sucide Nets

https://www.wired.com/images_blogs/rawfile/2010/11/tl201009-foxconn24.jpg

Karoshi: Japanese term indicating death or disablement due to overwork (Iwasaki et al., 2006)

What is Good Work?

May 25, 2018 3

• Satisfying to the workers:
• Physical
• Psychological
• Social needs
• benefits the employer (Lee, 2014)
• “Prosperity for the employer cannot exist through a long term
• f years unless it is accompanied by prosperity for the

employee”

• Fredrick W. Taylor (1916)

Current Engineered Solution: Lee (2014)

May 25, 2018 4

Methods to perform:

• 1. Track: attrition,

turnover, productivity, quality, overhead

• 2. Survey & interview
• 3. Statistical

differences

• 4. Actions well

documented in literature

• 5. Implementation

performed by

• rganization

Figure 2: Adapted from Lee (2014)

Current Engineered Solution: Lee (2014) continued

• Work is measured through 12 controllable dimensions

May 25, 2018 5

• Dr. W.T. Lee’s 12 Work Dimensions

Compensation Accomplishment Safety Demand Social Interaction Autonomy Variety Value Aesthetics Technical Growth Feedback Personal Growth

Research Questions

• What examples of actions taken to improve the workers’ job

conditions can be identified in industry and literature?

• If implemented properly, how does Lee’s Work Improvement

Process improve work from the workers’ perspective?

• Does Lee’s Work Improvement Process decrease the mismatch between

work and workers? (Lee, 2014)

• What context behind, and reasons for, people’s work preferences can be

discovered from in person interviews?

May 25, 2018 6

Site and Participant Selection

• Site Selection Criteria:
• Organization in Willamette Valley (within 90 miles of Corvallis)
• Management that is on board with bettering work
• Participant Selection (Workers)
• Self selection
• Paper survey: Voluntary, anonymous
• In person interview: Voluntary, confidential

May 25, 2018 7

Methods – Mixed: Data Collection Qualitative: Grounded Theory

• In-person Interviews
• N=40
• ~45 minutes
• Based on participants thoughts, feelings, experiences about 12 dimensions
• Created note taking template to organize my notes and thoughts
• Artifact Analysis/ Document Analysis
• Position descriptions
• Employee handbooks
• Observations of workers while working

May 25, 2018 8

Quantitative

• Paper Survey

Methods: Analysis Quantitative

• Statistical tests: comparison tests, ANOVA

May 25, 2018 9

Qualitative

• Transcribe/ Type Interview notes, Observations, and Artifacts
• Make Notes while typing
• Coding: NVivo Program
• Coding First Cycle
• Codes: 12 Dimensions & key phrases from literature
• Coding Second Cycle
• Magnitude Coding: +/- does code help or prevent worker
• Axial Coding

Research Plan: Validate Lee’s (2014) Method

May 25, 2018 10

Start

Select Organizations

Perform 5 Step Process

1st iteration

Perform 5 Step Process

• 1. Track System health
• 2. Measure work and workers
• Survey
• Interview
• Observations
• 3. Identify Mismatches
• Statistics
• Coding
• 4. Identify Relevant Improvement Actions
• From Literature
• From Ideas brainstormed in interviews
• Collaboration meetings
• 5. Implement Actions
• Organization is ultimately the decider of

Actions

Research Plan: Validate Lee’s (2014) Method

May 25, 2018 11

Start

Select Organizations

Perform 5 Step Process Steady state system Perform 5 Step Process

Identify if method improved work

1st iteration 2nd iteration

Current Progress: Nearly ready to begin study

May 25, 2018 12

Antecedent Study: What are the Dimensions of Good Work? Organization Requirement:

• Two down
• One to go

Institutional Review Board:

• Expedited: Approval Granted
• Need to make revision to protocol
• Rephrase interview questions to be more open ended
• Add probing questions

Questions?

May 25, 2018 13

https://www.wired.com/images_blogs/rawfile/2010/11/tl201009- foxconn24.jpg

References

• Lee, W. (2014). About good work. Corvallis, Or.: Oregon State

University.

• Taylor, F. (1916). The principles of scientific management.

New York and London: Harper & Bros.

• Perlin, R. (2013). Chinese Workers Foxconned. Dissent, 60(2),

46–52. https://doi.org/10.1353/dss.2013.0024

May 25, 2018 14