Wearable ESM Differences in Experience Sampling Across Wearable - PowerPoint PPT Presentation

Wearable ESM Differences in Experience Sampling Across Wearable Devices Javier Hernandez (javierhr@mit.edu) * Daniel J. McDuff Christian Infante Pattie Maes Karen Quigley * Rosalind W. Picard

Content Background Wearable ESM Tool Real-life Experiment Results Conclusions

Content Background Wearable ESM Tool Real-life Experiment Results Conclusions

Experience Sampling Method Minimize interactions Minimize biases Natural settings Uncontrolled (Csikszentmihalyi et al. 1977) (Larson and Csikszentmihalyi, 1983) (EMA, Shiffman et al., 2008; Bolger et al., 2003) 4

ESM: Applications Emotions Time Productivity Social Interactions Food Intake Conner et al, 2009 Berkman et al, 2014 Ayzenberg et al, 2012 Church et al 2014 Burgin et al, 2012 Timmermann et al, 2015 Scholl et al 2014 Hu et al, 2015 Adams et al, 2014 5

ESM: Devices Disruptive Missed Response Interruptions Biases Can we use wearable devices to minimize some of the problems? 6

Considered Locations Pocket Wrist-worn Head-worn Familiar Concealable Intimate Widespread Accessible Very Accessible Disruptive Small Display Cumbersome Biases Different Very Different Adams et al, 2014 Timmermann et al, 2015 Hu et al, 2015 Ayzenberg et al, 2012 Scholl et al 2014 Berkman et al, 2014 Burgin et al, 2012 Stress and mood measurement Church et al 2014 7

Content Background Wearable ESM Tool Real-life Experiment Results Conclusions

Experience Sampling Tool 2D Grid 5-Likert Scale 9

How were you feeling during the previous 5 minutes? VERY ENERGETIC VERY UNPLEASANT VERY PLEASANT LOW ENERGY

What situation best reflects your previous 5 minutes? HIGH DEMANDS HIGH RESOURCES LOW RESOURCES LOW DEMANDS

How stressed were you feeling during the previous 5 minutes?

Was this prompt disruptive?

Prompt Triggering 1. Time Distribution . ~1 prompt every 45 minutes 2. Time Variability . Distributed over the day 3. Device Variability . No more than two consecutive prompts on the same device 16

Content Background Wearable ESM Tool Real-life Experiment Results Conclusions

18

19

Google Glass 640x360 pixels Samsung Galaxy S4 1920x1080 pixels Samsung Gear Live 320x320 pixels 20

Starting Phase Data Collection Ending Phase • Tutorial • Usability of devices • Surveys • Free-form Interview 1 st day  $15, 2 nd day  $25, 3 rd day  $35, 4 th day  $45, 5 th day  $55 + $25 for successfully completing the whole study $200 for the whole study 21

Data Overview 15 participants (7 females and 8 males) from MIT Graduate students and one administrator 29.66 years (STD: 6.42) 5 (mostly) consecutive work days 22

Content Background Wearable ESM Tool Real-life Experiment Results Conclusions

Response Rate per Person over 5 Days * Average #Prompts Issued by Device Total: 627 prompts 24 * Statistically significant based on ANOVA and Bonferroni correction (p<0.007)

Response Rate per Person over 5 Days * Average Average (%) #Prompts Submitted Issued by Device Total: 627 prompts Missed: 111 prompts (~18%) 25 * Statistically significant based on ANOVA and Bonferroni correction (p<0.007)

Response Times Prompt Final Triggered Submission Time (seconds) Total Time 26

Response Times 1 st interaction Prompt Final Triggered with the application Submission Time (seconds) Total Time Starting Time * 27

Response Times 1 st interaction Prompt Final Triggered with the application Submission Time (seconds) Total Time Starting Time Answering Time * * 28

Response Distribution “I found it difficult to point out things [on the Gear] because my finger was on top of it” “The watch was the hardest [to point] because my finger may be too fat” 29

Usability of Devices Potential Difficulty Reports Device Affected Social Elicited Future Use of Use Quality Comfort Interactions Stress Qua ty o epo ts 10 10 10 10 10 10 8 8 8 8 8 8 6 6 6 6 6 6 4 4 4 4 4 4 2 2 2 2 2 2 0 0 0 0 0 0 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 10 10 10 10 10 10 8 8 8 8 8 8 6 6 6 6 6 6 4 4 4 4 4 4 2 2 2 2 2 2 0 0 0 0 0 0 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 10 10 10 10 10 10 8 8 8 8 8 8 6 6 6 6 6 6 4 4 4 4 4 4 2 2 2 2 2 2 0 0 0 0 0 0 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 Very vs Very Very vs Very Very vs Very Never vs All the time Not at all vs Never vs All the time Significantly Challenging Easy Inaccurate Accurate Uncomfortable Comfortable More 30

Usability of Devices Potential Difficulty Reports Device Affected Social Elicited Future Use of Use Quality Comfort Interactions Stress Qua ty o epo ts 10 10 10 10 10 10 8 8 8 8 8 8 6 6 6 6 6 6 4 4 4 4 4 4 2 2 2 2 2 2 0 0 0 0 0 0 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 10 10 10 10 10 10 8 8 8 8 8 8 6 6 6 6 6 6 4 4 4 4 4 4 2 2 2 2 2 2 0 0 0 0 0 0 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 10 10 10 10 10 10 8 8 8 8 8 8 6 6 6 6 6 6 4 4 4 4 4 4 2 2 2 2 2 2 0 0 0 0 0 0 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 Very vs Very Very vs Very Very vs Very Never vs All the time Not at all vs Never vs All the time Significantly Challenging Easy Inaccurate Accurate Uncomfortable Comfortable More “[The Glass] is painful, I wear glasses sometimes and they're not that uncomfortable...” “Surprisingly [the Glass] was not uncomfortable” 31

Usability of Devices Potential Difficulty Reports Device Affected Social Elicited Future Use of Use Quality Comfort Interactions Stress Qua ty o epo ts 10 10 10 10 10 10 8 8 8 8 8 8 6 6 6 6 6 6 4 4 4 4 4 4 2 2 2 2 2 2 0 0 0 0 0 0 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 10 10 10 10 10 10 8 8 8 8 8 8 6 6 6 6 6 6 4 4 4 4 4 4 2 2 2 2 2 2 0 0 0 0 0 0 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 10 10 10 10 10 10 8 8 8 8 8 8 6 6 6 6 6 6 4 4 4 4 4 4 2 2 2 2 2 2 0 0 0 0 0 0 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 Very vs Very Very vs Very Very vs Very Never vs All the time Not at all vs Never vs All the time Significantly Challenging Easy Inaccurate Accurate Uncomfortable Comfortable More “People would feel I was taking pictures of them and did not enjoy the conversation” “[The Glass] was a nice ice breaker” 32

Usability of Devices Potential Difficulty Reports Device Affected Social Elicited Future Use of Use Quality Comfort Interactions Stress Qua ty o epo ts 10 10 10 10 10 10 8 8 8 8 8 8 6 6 6 6 6 6 4 4 4 4 4 4 2 2 2 2 2 2 0 0 0 0 0 0 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 10 10 10 10 10 10 8 8 8 8 8 8 6 6 6 6 6 6 4 4 4 4 4 4 2 2 2 2 2 2 0 0 0 0 0 0 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 10 10 10 10 10 10 8 8 8 8 8 8 6 6 6 6 6 6 4 4 4 4 4 4 2 2 2 2 2 2 0 0 0 0 0 0 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 Very vs Very Very vs Very Very vs Very Never vs All the time Not at all vs Never vs All the time Significantly Challenging Easy Inaccurate Accurate Uncomfortable Comfortable More “I found it very annoying receiving notification through Glass when speaking with people because it was so noticeable” 33

Content Background Wearable ESM Tool Real-life Experiment Results Conclusions

Overview Response Rates Response Time Response Distribution Usability 35

Limitations & Future Work • Application improvements Modify submission Self-report on demand • Influencing factors Negative press of Glass Usability of devices • Other interactions Modalities (e.g., voice, camera) Input (e.g., text, numbers) 36

Summary Wearable Experience Sampling Tool Qualitative and Quantitative Analysis 37

Wearable ESM Differences in Experience Sampling Across Wearable Devices Javier Hernandez (javierhr@mit.edu) * Daniel J. McDuff Christian Infante Pattie Maes Karen Quigley * Rosalind W. Picard