NOTE: The attached record is a presentation created by an outside - - PowerPoint PPT Presentation

▶
note the attached record is a presentation created by an
SMART_READER_LITE
LIVE PREVIEW

NOTE: The attached record is a presentation created by an outside - - PowerPoint PPT Presentation

Dec 02, 2023 219 likes •367 views

NOTE: The attached record is a presentation created by an outside third-party and provided to the National Security Commission on Artificial Intelligence (NSCAI) to complement an outside engagement. The record and its contents were not created,

slide-1
SLIDE 1

NOTE: The attached record is a presentation created by an outside third-party and provided to the National Security Commission on Artificial Intelligence (NSCAI) to complement an outside engagement. The record and its contents were not created, drafted, or developed by NSCAI and does not reflect the views or recommendations of NSCAI.

EPIC-2019-001-002607 epic.org EPIC-19-09-11-NSCAI-FOIA-20200731-7th-Production-pt1-AI-Psychology-Workforce-Presentation 001877

slide-2
SLIDE 2

How can psychology and AI together help prepare an AI-enabled workforce?

  • Goal 1: Develop and hire AI talent.
  • Identify relevant skills through work analysis
  • Measure those skills with valid and reliable assessments
  • Develop effective performance management systems
  • Goal 2: Combine psychology and AI to accomplish goals. For example:
  • Video interviews
  • Behavior tracking/monitoring
  • Human/AI teams

EPIC-2019-001-002608 epic.org EPIC-19-09-11-NSCAI-FOIA-20200731-7th-Production-pt1-AI-Psychology-Workforce-Presentation 001878

slide-3
SLIDE 3

Example 1: Video interviews

  • Video interviews can

be AI-based such that facial expressions, tone

  • f voice, responses, and
  • ther signals are

analyzed to build dynamic predictive models

  • Current vendors include

HireVue, SparkHire,

  • thers

EPIC-2019-001-002609 epic.org EPIC-19-09-11-NSCAI-FOIA-20200731-7th-Production-pt1-AI-Psychology-Workforce-Presentation 001879

Qu stion 2 of 6

Pl as d scrib how your skills, due tion, nd experie ce will h Ip you succ din this position .

H p

O

ng

v deo Respon

R ponse t m

2 49

i=ii½l:-8

J Done Answering

slide-4
SLIDE 4

Challenges

  • Uniform Guidelines on Employee Selection (EEOC)
  • APA Standards for Educational and Psychological Testing

(2014) requires that all assessments should have

  • Reliability
  • Validity
  • Fairness
  • Practically, reducing time-to-hire is as important as making

good decisions

EPIC-2019-001-002610 epic.org EPIC-19-09-11-NSCAI-FOIA-20200731-7th-Production-pt1-AI-Psychology-Workforce-Presentation 001880

slide-5
SLIDE 5

Challenges include fear about bias

EPIC-2019-001-002611 epic.org EPIC-19-09-11-NSCAI-FOIA-20200731-7th-Production-pt1-AI-Psychology-Workforce-Presentation 001881

IL HB2557

VIDEO INTERVIEW ACT Summary

Introduced In Committee Crossed Over Passed (2/13/2019) (5/1/2019) (3/27/2019) (5/29/2019)

Introduced Session: 101st General Assembly Bill Text Action History Vote History

Signed/Enacted > Dead/FailedNetoed

Views: 88 86 94

in the last Week Month Total

Associated Documents

Veto Overridden

Bill Summary: Creates the Artificial Intelligence Video Interview Act. Provides that an employer that asks applicants to record video interviews and uses an artificial intelligence analysis of applicant-submitted videos shall: notify each applicant in writing before the interview that artificia l intelligence may be used to analyze the applicant's facial expressions and consider the applicant's fitness for the position; provide each applicant with an information sheet before the interview explaining how the artificial intelligence works and what characteristics it uses to evaluate applicants ; and obtain written consent from the applicant to be evaluated by the artificial intelligence program. Provides that an employer may not use artificial intelligence to evaluate applicants who have not consented to the use of artificial intelligence analysis . Provides that an employer may not share applicant videos, except with persons whose expertise is necessary in order to evaluate an applicant's fitness for a position.

slide-6
SLIDE 6

Research Insights: AI in Hiring and Selection

  • Algorithmic combination of predictor variables is superior to clinical

(“human”) combination (Meehl, 1954)

  • People are happy to rely on AI advice in many cases, but not when

they think they are experts (Logg et al 2019)

  • Introducing variation in assessment medium between candidates is

not a good idea (Blacksmith, Willford, Behrend, 2016)

  • “Algorithmic bias” is a misnomer. Frequently the criterion (job

performance measure) is where bias is located.

  • Bottom line: AI will outperform human judges. Candidates will

probably accept it. Hiring managers probably won’t. Any tool should

  • nly be used when its validity can be demonstrated—must measure

things that are job-related.

EPIC-2019-001-002612 epic.org EPIC-19-09-11-NSCAI-FOIA-20200731-7th-Production-pt1-AI-Psychology-Workforce-Presentation 001882

slide-7
SLIDE 7

Example 2: Electronic performance management (EPM)

EPIC-2019-001-002613 epic.org EPIC-19-09-11-NSCAI-FOIA-20200731-7th-Production-pt1-AI-Psychology-Workforce-Presentation 001883

RTMONITORI SEE WHETHER THEY DRIVE SAFELY

The company w ill use the data to decide arguments between riders and drivers, it says

Hacks rebel after bosses secretly install motion sensors under desks

Well done, thanks for giving PHBs everywhere a great idea for 2016

By lain Thomson in San Francisco 12 Jan 2016 at 01:13

147 0 SHARE T

Eye spy OccupEye ... How the sensor box can be fitted to a desk

Staff at one of Britain's oldest nationa l newspape rs got a shock on Monday morning when they found monitoring sensors installed under their desks. The boxes, sold by OccupEye as a way to monitor how long staff are at their desks without relying "on coffee cups and coats on chairs," were installed in the offices of The

Daily Telegraph.

Staff weren't told anything 1stallation and soon kicked up a storm of protest.

slide-8
SLIDE 8

Consequences of data collection/AI on behavior

  • People will optimize their behavior toward any goal that they become

aware of.

  • Example: News coverage that people who “liked” curly fries on their facebook

profile were smarter. Everyone who heard the story immediately went and “liked” curly fries. The correlation then disappeared.

  • Choosing to measure something sends a message that it is important.

The rules are changed.

  • Example: Fitbits drive behavior but also redefine “fitness.”
  • Goal-setting research is psychology is clear that goals are motivating

but also narrow one’s focus, at a cost

EPIC-2019-001-002614 epic.org EPIC-19-09-11-NSCAI-FOIA-20200731-7th-Production-pt1-AI-Psychology-Workforce-Presentation 001884

slide-9
SLIDE 9

EPIC-2019-001-002615 epic.org EPIC-19-09-11-NSCAI-FOIA-20200731-7th-Production-pt1-AI-Psychology-Workforce-Presentation 001885

Descriptive vs Prescriptive Data Collection

  • Devices

like sociometers can be used to train Al about effective communication, but what works in one setting may be harmful n another

slide-10
SLIDE 10

Effects of Electronic Monitoring on Work Performance

EPIC-2019-001-002616 epic.org EPIC-19-09-11-NSCAI-FOIA-20200731-7th-Production-pt1-AI-Psychology-Workforce-Presentation 001886

Galinaky ........ & Pan. 1815

  • Hllnlll. KD11u1.

&

Baalh (aludy 3), 2009 NalNakar&Talum.1113

TDl'IIIZllk,. Y'lllbd. Whila. & Bahrand, 2018

Grillh.1113 .... 2.201•

Karim ( ). 2015

  • ■

'

Walllan,

Thompaan. Rudalph.

Whalan..

Bahrand, & Ginal, 2013

.,.,

  • i .............

,

a.ntan &

Julian,, 2002

  • a.ntan &

Salkar-Bamay. 2010 Malo,, NOldaln:lm.

BalWa.

Tnalar,, 2007

DauNt & Mallo. 2001

Kalb &Mallo. 1118

Jaalca &

Sanluzd

  • • ,.

DlWldlian & Handanlan, 2DDO

D'IJraD

(DiDarlaliaa, 2004

Jaalca (Diuarlalian). 2011 Paniml (Diaaartalian). 2002

Pollaf,

Bannalt.

Rabarla., & LDMJ. 2011 Kalb &Mao. 1N7 Mallo & Kalb. 1NS

  • Halnmn. 2002

Becker & Marique (Bludy 2). 201•

Jaalca &

Sarmmi, 2013

  • Thompaan. s.blllllianalli. & Murmy,

2009 Becker & Marique (Bludy 1). 2014 REMadal

I

..0.4

I ■: I

■

!

I

  • ■

I I

I

■

i

  • I

■

I

  • .

.

  • ,,__

................ .

  • --•--•· •

..0.2 0.0

'

I

0.2

' I

0.4 0.1

Con-elation

95% Cl

OA3(0.18. CUM) 0.18 (..0.00, 0.35) 0.17 (..0.32. 0.511

  • .

14 ( 0.00. 0.27) 0.14 (..0.25. 0.41) 0.12 (..0.03, 0.27) OJII ( 0.00. 0.17) 0.07 [..OJJI, 0.23) 0.08 (..0.21, 0.32)

O.Ot

(..0.

  • 11. 0.27)

0.03 (..0.13, 0.11) 0.01 [..0.20, 0.22) ..0.01 (..0.28. 0.2') ..0.02 (..0.18. 0.12) ..0.02 (..0.30. 0.27) ..0.03 (..0.14. 0.08) ..0.07 (..0.21. 0.07) ..0.07 (..0.

  • 11. O.Ot)

..0.10 (-0.19, ..0.01) ..0.10 (..0.34. 0.15) ..0.13 (..0.30. O.GS) ..0.19 (-0.27, ..0.11) ..0.11 (-0.35, ..0.02) ..0.20 (-0.31, ..0.03) ..0.20 (..OM, O.o8) ..0..21 (-0.41, ..O.Ot) ..0.02 (..0.07, 0.03)

slide-11
SLIDE 11

Effects of Electronic Monitoring on Job Attitudes

EPIC-2019-001-002617 epic.org EPIC-19-09-11-NSCAI-FOIA-20200731-7th-Production-pt1-AI-Psychology-Workforce-Presentation 001887

Aulhor(s), Year

  • Haala. Kalu. & Boalh (atudy 3). 2009
  • Slantan. 2000

Karim (Dlaaarlallan), 2015

DauNl &

  • Aillllo. 2001

Slantan & Julian, 2002 JHke & Sanluzzi, 2014 DIIVidaan &

  • Handanlan. 2DOO

JHke (Dlaaarlallan), 2011

Holland, Caapa. & Hackar. 2014 Chalykaff & Kochan. 1111 Panina (Dinarllllian). 2002

  • Cllllaan. car1aan.

Zhmullla. Hanill.

Harri&.

2017 JHke &

Sanluzzi, 2015 Thampllan.

  • li

■

Rli.

& Murmy.

200& Hafna'I, 2D02 T~

  • WBmd. While, & Balnnd, 2011

JHke & Sanluzzi, 2018 AdamB & Mulram. 2011 Nabaker & Tall.Im. 1113 McNal & Rach. 2007

Brawn.

  • Badgar. Bahrand.

Jenaan.

2012 Paa&,, Bannal. Rabarta. &

LCMIJ. 2011 D'Urao

(Dla ■ rtalian,

2DOI REMadal

  • I

! ■

t I

  • t
  • .

: .

  • .

'•

I

■

I : .

I

  • ..........

.,.._

...

,:

I

■

I :

  • .... - ...............

,,

  • I
  • .....

....

I

  • GA

I •

  • 0.2

.

I

I .

  • .

I .

  • I

0.0

'

I

0.2

  • .,

Cofflllation (96% CIJ

0.08 [-G.11. 0.211 O.ot f-0.Dl. 0.161 0.02 [-G.07. 0.111 0.02 [-G.11. 0.23)

  • G.07

[-G.D. 0.2G)

  • G.08

[-G.21, O.OI)

  • G.08

[-G.3l. 0.211

  • G.12

[-G.21. 0.02)

  • G.13

f,0.22. -G.CMJ

.a.

14 f,0.21, -G.07)

  • G.15

f,0.21. -G.03)

  • G.11

f,0.27, -G.OIJ

  • G.17

f,0.31, -G.02)

  • G.19

[-GAS. 0.07)

  • G.20

f,0.28, -G.12)

  • G.21

f,0.34. -G.07)

  • G.22

f,0.38, -G.05)

  • G.M

f,0.35. -G.12)

  • G.25

[-G.M. 0.251

  • G.27

f,0.38. -G.15)

  • G.34

f,0.44. -G.24)

  • G.31

f,OA7,

  • G.311
  • G.44

f,0.53, -G.35)

  • G.17

f,0.23. -G.111

slide-12
SLIDE 12
  • Fears about ubiquitous surveillance/privacy/psychological targeting
  • Reactance: when people’s autonomy is restricted they will seek to reassert

it by finding ways to act out that aren’t constrained

  • E.G., Uber drivers at National Airport
  • Mistrust of algorithmic decision making: people want to believe that

human judgment is fairer and more accurate, but it is not

  • Bottom line: Measuring behaviors will always change those same
  • behaviors. Must consider rationale, consequences, and effects on

models.

Consequences of data collection/AI on behavior

EPIC-2019-001-002618 epic.org EPIC-19-09-11-NSCAI-FOIA-20200731-7th-Production-pt1-AI-Psychology-Workforce-Presentation 001888

slide-13
SLIDE 13

Other issues requiring psychology expertise

  • Training, Development, and Education to support

human/AI teams

  • Developing both "taskwork" and "teamwork" skills
  • Avoiding automation surprise (e.g., Boeing)
  • Building trust
  • Communication, procedural justice, reactance,

psychological contracts/expectations

EPIC-2019-001-002619 epic.org EPIC-19-09-11-NSCAI-FOIA-20200731-7th-Production-pt1-AI-Psychology-Workforce-Presentation 001889

slide-14
SLIDE 14

Suggested Readings

  • Yost, A. B., Behrend, T. S., Howardson, G., Darrow, J. B., & Jensen, J. M. 2018.

Reactance to electronic surveillance: A test of antecedents and outcomes. Journal

  • f Business and Psychology, 34: 1-16.
  • Blacksmith, N., Willford, J. C., & Behrend, T. S. (2016). Technology in the

employment interview: A meta-analysis and future research agenda. Personnel Assessment and Decisions, 2(1), 2

  • Logg, J. M., Minson, J. A., & Moore, D. A. (2019). Algorithm appreciation: People

prefer algorithmic to human judgment. Organizational Behavior and Human Decision Processes, 151, 90-103.

  • Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical analysis and

a review of the evidence.

  • American Educational Research Association, American Psychological Association,

National Council on Measurement in Education, Joint Committee on Standards for Educational, & Psychological Testing (US). (2014). Standards for educational and psychological testing. Amer Educational Research Assn.

  • Onnela, J. P., Waber, B. N., Pentland, A., Schnorf, S., & Lazer, D. (2014). Using

sociometers to quantify social interaction patterns. Scientific reports, 4, 5604.

EPIC-2019-001-002620 epic.org EPIC-19-09-11-NSCAI-FOIA-20200731-7th-Production-pt1-AI-Psychology-Workforce-Presentation 001890