Air Force Research Laboratory Trust in Automation Research: Current - - PowerPoint PPT Presentation

1

DISTRIBUTION STATEMENT A: Approved for Public Release

Air Force Research Laboratory

Integrity  Service  Excellence

Trust in Automation Research: Current Directions and Gaps

Date: 5 APR 17

Joseph Lyons, PhD Technical Advisor Human Trust & Interaction Branch Air Force Research Laboratory

2

DISTRIBUTION STATEMENT A: Approved for Public Release

• Personal Background
• Trust Background

– Definition & Measures – Trust Literature

• Auto-GCAS Project

– What is it? – Survey results – Interview results – Lessons Learned

• Future Directions & Gaps

– Shared awareness and shared intent

Outline

3

DISTRIBUTION STATEMENT A: Approved for Public Release

• Ph.D. I/O Psych, Minor HF Psych - Wright State

University 2005

• Senior Research Psychologist for Air Force Research

Laboratory – since 2005

• Research Interests: human-machine trust, interpersonal

trust, leadership, Org Science

• 2011-2013 served as Program Officer for Air Force Office
• f Scientific Research

– Chartered the Trust & Influence Portfolio – Air Force lead for Minerva

• 2013 Returned to WPAFB – started the Human Insight

and Trust (HIT) Team

– Dedicated to understanding human-machine trust for AF domains – 9 Civilians 1-2 Military, 1 post-doc, approx. 6 FTE contractors

Personal Background

4

DISTRIBUTION STATEMENT A: Approved for Public Release

HIT Video

5

DISTRIBUTION STATEMENT A: Approved for Public Release

• Prior/Current HIT Research

– Automated versus human aids (Lyons & Stokes, 2012) – Transparency and trust (Lyons, 2013; Lyons & Havig, 2014;

Lyons et al. 2016a)

– Trust and suspicion processes (Lyons et al., 2011; Bobko et al.,

2014)

– Trust of software code (Alarcon et al., in press; Hodgkin et al.,

2016)

– Impact of different error types on trust (Guznov et al. 2015;

2016)

– Human-Robot Interaction

• Signaling intent

– Dialogue, behavior, and rationale

• Social influences (Stokes et al., 2016)
• Human-Agent Teaming (Wynne & Lyons, in press)

– Pilot trust of landing aids in Degraded Visual Environments – Trust evaluations of fielded systems (Lyons et al., 2016b; Ho

et al., in press; Kolitai et al., 2014)

HIT Background

6

DISTRIBUTION STATEMENT A: Approved for Public Release

• Trust = willingness to be vulnerable w/out the ability or

capacity to monitoring (Mayer et al., 1995)

• Necessary Conditions

– Human intention – Relational – must have a referent (otherwise its dispositional) – Risk – without it trust is obsolete

• What to Measure?

– Intention to be vulnerable – Trustworthiness

• Ability, benevolence, integrity (Mayer et al., 1995)
• E.g., Machine characteristics, “Learned trust” (Hoff & Bashir, 2015)

– Reliance behavior

• Must be volitional, should be in risky context – more than mere cooperation

– Dispositional trust – Physiology?

Trust Background

7

DISTRIBUTION STATEMENT A: Approved for Public Release

• Management, e-commerce, Human Factors,

Robotics, Social Psych… trust research is pervasive

• Mayer et al. (1995)
• Lee & See (2004)
• Parasuraman & Manzey (2010)
• Hancock et al (2011)
• Chen & Barnes (2014)
• Technology Acceptance Model
• Merritt et al. (2016)
• Hoff & Bashir (2015)

Trust Background… “a small sliver”

8

DISTRIBUTION STATEMENT A: Approved for Public Release

• Why is this so complicated???

– Errors for highly automated systems can be catastrophic (Onnasch et al., 2014)

• Asiana flight 214 (2013) – overreliance
• Cruise ship Concordia (2012) – under-reliance

– Trust is dynamic & driven by multiple factors

• Social, Affective, Cognitive, Dispositional…
• Absolute values will change with changing context

– Importance of predictors may also change

– Perceptions of trustworthiness may be inaccurate – Unintended consequences (Parasuraman & Riley, 1997) – Few studies using real operators, real-world technology & real-world consequences (R3)

The Problem – Inaccurate Trust

9

DISTRIBUTION STATEMENT A: Approved for Public Release

• Time-to-collision-based automated system

– Components: Collision avoidance Algorithm, Digital Terrain Elevation Data, interface with flight control computer, HUD interface

• Behavior: wings-roll-to-level maneuver, 5-G pull up
• Objectives: do not interfere, do no harm, avoid collision

Automatic Ground Collision Avoidance System

10

DISTRIBUTION STATEMENT A: Approved for Public Release

AGCAS Video

11

DISTRIBUTION STATEMENT A: Approved for Public Release

• Why is this a good domain to study trust?

– Fielded automated DoD system – high level of automation

• Supports AF interests in autonomy & future technologies
• Supports pilot safety

– Real-life implications for safety

• “Most sophisticated AF safety system ever fielded”
• Four “saves” to date since 2014

– Trust-rich context

• Potential for false alarm/miss, prior system links, imperfect reliability, novelty

– Good place to test trust antecedents – ecological validity – Engineer/test concerns

• Distrust - Too much trust
• Impact of HUD display unknown
• Prior warning-based systems distrusted
• Acknowledgment of team: Dr. Nhut Ho, Lauren Hoffmann, Garrett Sadler, Capt

Eric Fergueson, 1Lt Anna Lee Van Abel, Samantha Cals, Kolina Koltai, Maj Casey Richardson, Mark “Tex” Wilkins, Air Vehicles Directorate, Air Force Flight Test Center

Why Auto-GCAS?

12

DISTRIBUTION STATEMENT A: Approved for Public Release

• AF Trust research lacks data on operator trust in

fielded systems with high levels of autonomy

• Four-year longitudinal study (started FY15)

– Examine operational pilot trust evolution in AGCAS

• vertime

– Identify and document user experience, concerns, impact, and benefits of technology as they emerge – Obtain operational pilots feedback on AFRL future automated/autonomous systems

• AACAS & Autonomous Wingman
• Uses survey and interview methods

– Visited 13 bases (6 were repeat visits in year 2) – Year 1: Survey (N = 142), Interview (N = 168) – Year 2: Survey (N = 100), Interview (N = 131)

AGCAS Field Study

13

DISTRIBUTION STATEMENT A: Approved for Public Release

Survey Measures

Rated on 7-point Likert Scales

Trust (4 items, α = .95) “I can count on Auto-GCAS to work when needed” Performance (3 items, α = .64) “Auto-GCAS is reliable” Transparency (3 items, α = .84) “I understand how Auto-GCAS works” Benevolence (3 items, α = .74) “I think Auto-GCAS was designed to help me” Benefits (2 items, r = .82) “I benefit from having Auto-GCAS installed on my plane” Confidence (3 items, α = .62) “I feel confident with Auto-GCAS installed on my plane” Aggressive flying (2 items, r = .75) “Auto-GCAS allows me to fly lower to the ground” Automation Schema (Merritt et al., 2016; 3 items, α = .69) “Automated systems rarely make mistakes”

14

DISTRIBUTION STATEMENT A: Approved for Public Release

Year 1 Correlations with trust

(N = 142)

Predictor r Outcome r Schema .29** Confidence .68** Performance .63** Aggressive .14 Transparency .23** Benevolence .43** Benefits .55** Note: ** p < .01.

15

DISTRIBUTION STATEMENT A: Approved for Public Release

Year 1 Unique Predictors of Trust

(N = 142; Lyons et al., 2016c)

Predictor β R2 .47** Schema .13† Performance .39** Transparency .10 Benevolence .06 Benefits .25** Note: † p < .10, ** p < .01. Consistent with Hoff & Bashir, 2015

16

DISTRIBUTION STATEMENT A: Approved for Public Release

Year 2 Correlations with trust

Predictor r Outcome r Schema .23* Confidence .79** Performance .62** Aggressive .18 Transparency .16 Benevolence .24* Benefits .45**

Note: * p < .05, ** p < .01

17

DISTRIBUTION STATEMENT A: Approved for Public Release

Year 2 Regression

Predictor β R2 .39** Schema .14 Performance .53** Transparency .05 Benevolence

• .05

Benefits .10

Note: ** p < .01

Less uncertainty – diminished impact of Individual differences Stronger performance perceptions with “Save” stories pervasive Likely ceiling effects for transparency, Benevolence, and benefits

18

DISTRIBUTION STATEMENT A: Approved for Public Release

Experienced Versus Novice Effects

• How does experience influence trust and

aggressive flying?

– Operationalized as:

• Experienced > 500 flt hrs., Novice < 500 flt hrs.

– Potential for complacency

• Generational differences in automation

trust and use

– Mostly DoD-irrelevant “older” samples – Focused on cognitive decay – Need research on military-relevant ages

19

DISTRIBUTION STATEMENT A: Approved for Public Release

Experienced Versus Novice Effects (Lyons et al., under review)

t(92) = 2.37, p < .05.

20

DISTRIBUTION STATEMENT A: Approved for Public Release

• System performance

– Operational “Save” videos

• 4 “saves” since 2014!

– Perceived nuisance-free operations – Test rides via Pilot Activated Recovery System

• Business case

– Need for system – Perceived benevolence via system extremely high

• Error attribution for early errors
• Transparency – HUD display

– Increases SA, indicates the system is working

• Pedigree of Test Community in the AF
• Activations are handled as learning opportunities

Key Drivers of Trust via Interview

(Ho et al., in press)

21

DISTRIBUTION STATEMENT A: Approved for Public Release

• Development

– Transparency is critical – Human-centered design

• Pre-implementation

– Think about the business case

• Why is it needed?
• Who is advocating for it?

– Study potential barriers

• Cultural, dispositional, organizational, process, similar

technologies

• Understand the narrative

– Think about evaluation metrics and how you would collect them – Educate – share performance data

Lessons Learned for Novel Technology Implementation

22

DISTRIBUTION STATEMENT A: Approved for Public Release

• Post-implementation

– Own the narrative

• Share success stories/videos - seeing is believing!
• Share failures

– Use metrics to evaluate success

• Establish evaluation processes
• Educate based on evidence
• Track unintended consequences
• Iterate and improve

– Anchor new behaviors in the culture – Understand that indirect knowledge can be a powerful trust (dis) antecedent

Lessons Learned for Novel Technology Implementation

23

DISTRIBUTION STATEMENT A: Approved for Public Release

• Longitudinal research with military operators

is challenging!

– Access

• SME’s on the team are critical for buy-in

– Attrition – Measure appropriateness/timing

• Existing trust models are robust

– Performance/error types – Transparency – Individual differences – Perceived benefits – Benevolence – “intent”

Lessons Learned for Trust Research

24

DISTRIBUTION STATEMENT A: Approved for Public Release

• More applied trust work

– Real operators, real tools, real consequences

• Better rationale for physiological metrics
• Need more research on intent-based

transparency for Human-Machine Teams

– Methods to signal positive intent – Understand the impact of intent – Methods for developed shared awareness

General Gaps & Future Directions

25

DISTRIBUTION STATEMENT A: Approved for Public Release

Transparency

• Human-Robot Transparency (Lyons, 2013)

Intent Environment Task Analytics Teamwork Human State Social Intent

26

DISTRIBUTION STATEMENT A: Approved for Public Release

• Task performance
• Predictability & performance are key facets of trust in

automation (Hancock et al., 2011)

• Capabilities and Limitations (Chen & Barnes, 2014)
• Error management will be important for human-machine

teams – the ability to learn from mistakes should foster greater trust (Ososky et al., 2013)

• Logic for errors – rationale
• Need to make errors predictable/understandable
• Understanding analytical underpinnings supports trust

repair (Dzindolet et al., 2003)

• Assumptions
• Shared contextual awareness is key for distributed robotics

(Stubbs et al., 2007)

• Level of urgency, goal priorities, timeline, level of automation

Shared Awareness – what is it doing, how well, and why

27

DISTRIBUTION STATEMENT A: Approved for Public Release

• Social style
• Anthropomorphic features impact trust/liking in cars (Waytz et

al., 2014) and trust of decision aids (Pak et al., 2012)

• Robotic speech style impacts social distance (Kim et al., 2012)
• Dialogue fosters team orientation (Fischer, 2011)
• Patient interaction style influences trust & performance

(Parasuraman & Miller, 2004)

• Conferred Benevolence
• Movement toward a shared goal
• Back up behavior
• Understanding of consequences
• Agent-based emotional displays – appraisal process is key (De

Melo et al., 2011)

• Transfer of authority

Intent – what is it going to do, how, and why

28

DISTRIBUTION STATEMENT A: Approved for Public Release

Intent – transfer of authority error

Automated Driving Technology: “I think you should take over now”

29

DISTRIBUTION STATEMENT A: Approved for Public Release

Expectations

• Appearance affords perceived function among users – may align
• r conflict with expectations of users (Wright, 2012)
• Spider legs vs. Wheels, Tracks (Sims et al., 2005)
• Male vs female robots (Eyssel & Hegel, 2012)
• Human-like faces most preferred, more social, perceived to

possess more ability (Broadbent et al., 2013)

• Need to avoid too much – “uncanny valley”
• Engagement
• Robot Gaze may signify engagement (Mutlu, 2011)
• Attraction
• Use of personal name (Moran et al., 2013)
• Liking liked to trust (Merritt, 2011)
• Polite robots are liked more (Mutlu, 2011)

Expectations and Engagement

30

DISTRIBUTION STATEMENT A: Approved for Public Release

Thanks!

Joseph Lyons, PhD joseph.lyons.6@US.AF.MIL (937) 255-8734