Under Pressure Sensing Stress of Computer Users Javier Hernandez - PowerPoint PPT Presentation

Under Pressure Sensing Stress of Computer Users Javier Hernandez javierhr@mit.edu Pablo Paredes pablo.paredes@berkeley.edu Asta Roseway astar@microsoft.com Mary Czerwinski marycz@microsoft.com

Headaches Fatigue Anxiety Insomnia Depression 2

Outline • Stress Measurement • Previous Work • User Study • Results • Conclusions 3

Stress Measurement Hormones Physiology Self-reports Behaviors Cortisol DSI EMG Keyboard Adrenaline PSS EDA Mouse … … … … Intrusive Subjective Instrumentation Existing Interactions Costly Cognitive Attention Unfamiliar Continuous Monitoring Slow Recall Problems Caring (Zimmermann et al, 2003) 4

Keyboard & Mouse Pressure-sensitive Keyboard Capacitive Mouse (Dietz et al., 2009) (Villar et al., 2009 Benko et al., 2010) 5

Keyboard & Mouse Non-Pressure Dynamics Non-Pressure Dynamics • Neutral/positive VS negative (Khanna et al, 2010) • Increased speed and acceleration during More typing mistakes and slower speeds high arousal (Maehr, 2008) • 15 emotional states (Epp, 2011) • Increased movement during stress • Cognitive VS physical stress (Vizer et al, 2009) (Rodriges et al, 2013) Pressure Dynamics Pressure Dynamics • Increased pressure during stress 6 emotions (Lv et al, 2008) • (Wahlstrom et al 2012, Dennerlein et al, 2003) 6

Experimental Study Stress Protocol Tasks Design Measurement 7 7

Within-subject laboratory study Three tasks: • Expressive Writing • Text Transcription • Mouse Clicking Two conditions Stressed VS Relaxed Study Experimental Stress Protocol Tasks Measurement Design 8 8

Physiological Stress Affectiva Q TM Electrodermal Activity (Galvanic Skin Response) Self-reported Stress Stress Experimental Study Protocol Tasks Design Measurement 9 9

Task I: Text Transcription • Transcribe short biographical text • 3 minutes • During the stressed condition: 1. Type fast 2. Internal competition 3. Cursor blinks faster 4. Random font style 5. Timer 6. Loud traffic noise Experimental Study Stress Protocol Design Measurement Tasks 10

Task II: Expressive Writing • Re-experience a relaxing or stressful memory and write about it • Recommended time of 5 minutes • Allowed to make spelling, grammar and sentence errors Experimental Study Stress Protocol Design Measurement Tasks 11 11

Task III: Mouse Clicking • After keyboard tasks • Based on Fitts’ law task 90 times (3 dist. x 3 widths x 10 rep.) Experimental Study Stress Protocol Design Measurement Tasks 12 12

TT MC EW TT MC EW TT MC EW TT MC EW Training EW MC TT EW MC TT EW MC TT EW MC TT EW : Expressive Writing Stressed Condition 24 participants MC : Mouse Clicking Relaxed Condition Balanced gender TT : Text Transcription 13 Protocol Experimental Study Stress Tasks Design Measurement 13

Results

Did the tasks elicit the intended emotions?

Self-reports Valence Arousal Stress High Energy Very Stressed Very Pleasant 7 7 7 * * * * * 6 6 6 6 * * * 5 5 5 5 4 4 4 Not Stressed at All 4 Very Unpleasant 3 3 3 3 Low Energy 2 2 2 2 1 1 1 Expressive Text Mouse Expressive Text Mouse Expressive Text Mouse Writing Transcription Clicking Writing Transcription Clicking Writing Transcription Clicking Stressed Condition Relaxed Condition Standard Error *The two distributions were significantly different (Wilcoxon Rank Sum test , p<0.05) 16

How is typing pressure affected by stress?

Transcription Task 95 90 Avg. Pressure 85 80 75 Stressed condition Relaxed condition 70 0104 0105 0108 0103 0106 0109 0203 0204 0205 0206 0207 0208 0301 0302 0303 0304 0305 0306 0401 0402 0403 0404 0405 0406 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 x x x P ti i t Participants Avg. Stressed – Avg. Relaxed 8 6 Higher Pressure 4 During Stressed 2 0 -2 Higher Pressure -4 During Relaxed -6 -8 0104 0105 0108 0103 0106 0109 0203 0204 0205 0206 0207 0208 0301 0302 0303 0304 0305 0306 0401 0402 0403 0404 0405 0406 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 * * * * * * * * * * * * * * * * * * * * * P ti i t 22 out of 24 subjects (91.67%) show increased typing pressure under the stressed condition 18 The difference was computed from significantly different distributions (Wilcoxon Rank Sum test , p<0.05)

Expressive Writing 95 90 Avg. Pressure 85 80 75 Stressed condition Relaxed condition 70 4 5 8 3 6 9 3 4 5 6 7 8 1 2 3 4 5 6 1 2 3 4 5 6 0 1 0 2 3 0 0 4 5 0 0 6 0 7 0 8 0 9 10 0 11 0 0 12 13 0 14 0 15 0 16 0 17 0 18 0 0 19 20 21 0 0 22 0 23 24 0 0 1 1 1 1 1 1 2 2 2 2 2 2 3 3 3 3 3 3 4 4 4 4 4 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 x 0 0 0 0 0 x x x P ti i t Participants Avg. Stressed – Avg. Relaxed 8 6 Higher Pressure 4 During Stressed 2 0 -2 Higher Pressure -4 During Relaxed -6 -8 0104 0105 0108 0103 0106 0109 0203 0204 0205 0206 0207 0208 0301 0302 0303 0304 0305 0306 0401 0402 0403 0404 0405 0406 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 * * * * * * * * * * * * * * * * * * * * P ti i t 23 out of 24 subjects (95.83%) show increased typing pressure under the stressed condition 19 The difference was computed from significantly different distributions (Wilcoxon Rank Sum test , p<0.05)

How is mouse capacitance affected by stress?

Capacitive Mouse 13x15 Capacitive Pixels  21

Mouse Clicking Start Task End Task FITT_1 (1) FITT_1 (842) FITT_1 (1682) FITT_1 (2523) FITT_1 (3364) FITT_1 (4204) FITT_1 (5045) FITT_1 (5886) FITT_1 (6726) FITT_1 (7567) 2 2 2 2 2 2 2 2 2 2 4 4 4 4 4 4 4 4 4 4 6 6 6 6 6 … 6 6 6 6 6 8 8 8 8 8 8 8 8 8 8 10 10 10 10 10 10 10 10 10 10 12 12 12 12 12 12 12 12 12 12 14 14 14 14 14 14 14 14 14 14 5 10 15 5 10 15 5 10 15 5 10 15 5 10 15 5 10 15 5 10 15 5 10 15 5 10 15 5 10 15 FITT_2 (1) FITT_2 (762) FITT_2 (1523) FITT_2 (2284) FITT_2 (3045) FITT_2 (3805) FITT_2 (4566) FITT_2 (5327) FITT_2 (6088) FITT_2 (6849) 2 2 2 2 2 2 2 2 2 2 4 4 4 4 4 4 4 4 4 4 … 6 6 6 6 6 6 6 6 6 6 8 8 8 8 8 8 8 8 8 8 10 10 10 10 10 10 10 10 10 10 12 12 12 12 12 12 12 12 12 12 14 14 14 14 14 14 14 14 14 14 5 10 15 5 10 15 5 10 15 5 10 15 5 10 15 5 10 15 5 10 15 5 10 15 5 10 15 5 10 15 Stressed condition Relaxed condition Mouse Task 24 22 20 Average Average 18 Stressed 16 Relaxed 14 12 10 8 00:00 00:07 00:15 00:22 00:30 00:37 00:45 00:52 01:00 01:07 Time 22

Mouse Clicking 25 Avg. Mouse Contact 20 # Seconds 15 10 5 Stressed condition Relaxed condition 0 0104 0105 0108 0103 0106 0109 0203 0204 0205 0206 0207 0208 0301 0302 0303 0304 0305 0306 0401 0402 0403 0404 0405 0406 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Participants P ti i t Avg. Stressed – Avg. Relaxed 4 Mean Stress - Mean Relax More Mouse Contact 2 During Stressed 0 More Mouse Contact -2 During Relaxed -4 0104 0105 0108 0103 0106 0109 0203 0204 0205 0206 0207 0208 0301 0302 0303 0304 0305 0306 0401 0402 0403 0404 0405 0406 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 * * * * * * * * * * * * * * * * * * * * * * * * 18 out of 24 subjects (75%) show increased mouse contact under the stressed condition The difference was computed from significantly different distributions (Wilcoxon Rank Sum test , p<0.05) 23

How much data is required to differentiate between the stressed and relaxed conditions at any point in time?

N = 500 randomly segmented windows 24 Stabilizes 23 after 3.37 22 21 seconds 20 19 18 Average 17 # Participants with 16 Stabilizes 15 significantly more after 2.5 14 pressure/contact 13 minutes during the stressed 12 condition 11 10 9 8 7 6 5 Text Transcription 4 Expressive Memory 3 2 Mouse Clicking 1 0 0 50 100 150 200 250 300 Observation Window (seconds) 25

Considerations • Ill-defined ground truth • Laboratory • Many types of stress • Direction of causality • Individual differences 26

Conclusions • Lab study to test effectiveness of pressure-sensitive keyboard and capacitive mouse to sense stress of computer users • 3 tasks (expressive writing, text transcription, and mouse clicking) under 2 conditions (stressed and relaxed) • Self-reports showed the effectiveness of tasks • During the stressed condition: – >79% showed more forceful typing pressure – 75% showed greater amount of mouse contact Very small observation windows can be used to obtain similar results • 27

Acknowledgements Hrvoje Benko Kevin Hinckley Michel Pahud Paul H. Dietz 28