WEB-BASED VR EXPERIMENTS POWERED BY THE CROWD Xiao Ma [1,2] Megan Cackett [2] Leslie Park [2] Eric Chien [1,2] Mor Naaman [1,2] The Web Conference 2018 [1] Social Technologies Lab, Cornell Tech [2] Cornell University xiao@jacobs.cornell.edu | maxiao.info
VIRTUAL REALITY The Big Bang Theory S9E20 � 3
A VISION Virtual Reality (VR) Crowdsourcing ? + = � 4
COMBINING THE BEST OF BOTH WORLDS Virtual Reality Crowdsourced VR Crowdsourcing (as of 2017) (this work) Manipulation Realism Measurement Granularity Participant Diversity Reproducibility � 5
COMBINING THE BEST OF BOTH WORLDS Virtual Reality Crowdsourced VR Crowdsourcing (as of 2017) (this work) Manipulation + High - Low Realism Measurement + High - Low Granularity Participant - Low + High Diversity Reproducibility - Low + High � 6
COMBINING THE BEST OF BOTH WORLDS Virtual Reality Crowdsourced VR Crowdsourcing (as of 2017) (this work) Manipulation + High - Low + High Realism Measurement + High - Low + High Granularity Participant - Low + High + High Diversity Reproducibility - Low + High + High � 7
HOW � 8
RESEARCH QUESTIONS RQ1: Are there VR-eligible reachable workers? RQ2: What is a good user flow? RQ3: What types of experiment manipulations can we deliver remotely? RQ4: What are the limitations and challenges? � 9
CONTRIBUTIONS 1. Validated a VR-eligible panel of 242 workers 2. Implemented a user flow between desktop and VR 3. Replicated three previous studies with different experiment manipulation remotely 4. Limitations and challenges Source code and data are available at: bit.ly/VRCrowdExperiments � 10
OUTLINE 1. Validated a VR-eligible panel of 242 workers 2. Implemented a user flow between desktop and VR 3. Replicated three previous studies with different experiment manipulation remotely 4. Limitations and challenges Source code and data are available at: bit.ly/VRCrowdExperiments � 11
CONSTRUCTING VR-READY PANEL Please take a picture of your device with lasts 4 digits of your worker ID handwritten on a piece of paper in view. � 12
(MORE) DIVERSE PARTICIPANTS N = 242 � 13
(MORE) DIVERSE PARTICIPANTS N = 242 14% HIGH SCHOOL OR BELOW 70% WHITE 29% SOME / 2-YEAR COLLEGE 14% ASIAN 6% BLACK 29% BACHELOR'S 10% OTHER 13% MASTER'S AND ABOVE 0% 25% 50% 75% 100% 10% 32.5% 90% U.S. 21% BELOW $30K Age : 18 - 78 (Median 32) 10% OTHER 57% $30K - $80K 0% 25% 50% 75% 100% 52% SUBURBAN 61% MALE 22% ABOVE $80K 30% URBAN 39% FEMALE* 0% 25% 50% 75% 18% RURAL 0% 25% 50% 75% 100% 0% 25% 50% 75% 100% * One worker self-identified as “other”. � 14
OUTLINE 1. Validated a VR-eligible panel of 242 workers 2. Implemented a user flow between desktop and VR 3. Replicated three previous studies with different experiment manipulation remotely 4. Limitations and challenges Source code and data are available at: bit.ly/VRCrowdExperiments � 15
DESIGN GOALS OF USER FLOW Direct workers to participate in VR and complete survey on desktop Usable Allows for online data collection and thus remote participation Web-Based Allows for easier replication and adaptation Low Technical Barrier � 16
TECHNICAL PLATFORM CHOICE + Source code and data are available at: https://facebook.github.io/react-vr/ � 17 https://nodejs.org/en/ bit.ly/VRCrowdExperiments
FLOW Experimenter approves payment Verification Code 2 VR Web App URL Verification Code 1 Worker accepts task Worker completes Worker answers survey in through Amazon experiment in VR Qualtrics Mechanical Turk � 18
OUTLINE 1. Validated a VR-eligible panel of 242 workers 2. Implemented a user flow between desktop and VR 3. Replicated three previous studies with different experiment manipulation remotely 4. Limitations and challenges Source code and data are available at: bit.ly/VRCrowdExperiments � 19
MODELS OF ILLUSIONS IN VR � 20
MODELS OF ILLUSIONS IN VR A user’s feeling of being transported into the rendered environment Place Illusion A user’s feeling of experiencing the virtual world through an avatar Embodiment Illusion A user’s feeling that events happening in the virtual world are real Plausibility Illusion Gonzalez-Franco, M., & Lanier, J. (2017). Model of Illusions and Virtual Reality. � 21 Frontiers in psychology, 8, 1125.
MODELS OF ILLUSIONS IN VR Study 1: Restorative Effects of Virtual Environments [Valtchanov et al. 2010] , N = 22 (original) v.s. 55 (ours) A user’s feeling of being transported into the rendered environment Place Illusion Study 2: Proteus Effect [Yee and Bailenson, 2007] , N = 50 (original) v.s. 59 (ours) A user’s feeling of experiencing the virtual world through an avatar Embodiment Illusion Study 3: Drawing Power of Crowds [Milgram et al. 1969] , N = 1,424 (original) v.s. 56 (ours) A user’s feeling that events happening in the virtual world are real Plausibility Illusion Gonzalez-Franco, M., & Lanier, J. (2017). Model of Illusions and Virtual Reality. � 22 Frontiers in psychology, 8, 1125.
MODELS OF ILLUSIONS IN VR Study 1: Restorative Effects of Virtual Environments [Valtchanov et al. 2010] , N = 22 (original) v.s. 55 (ours) A user’s feeling of being transported into the rendered environment Place Illusion Study 2: Proteus Effect [Yee and Bailenson, 2007] , N = 50 (original) v.s. 59 (ours) A user’s feeling of experiencing the virtual world through an avatar Embodiment Illusion Study 3: Drawing Power of Crowds [Milgram et al. 1969] , N = 1,424 (original) v.s. 56 (ours) A user’s feeling that events happening in the virtual world are real Plausibility Illusion Gonzalez-Franco, M., & Lanier, J. (2017). Model of Illusions and Virtual Reality. � 23 Frontiers in psychology, 8, 1125.
� 24
STUDY 2: PROTEUS EFFECT Van Der Heide, Brandon, et al. "The Proteus effect in dyadic communication: Examining the effect of avatar � 25 appearance in computer-mediated dyadic interaction." Communication Research 40.6 (2013): 838-860.
� 26
STUDY 3: DRAWING POWER OF CROWDS N = 1,424 Milgram, S., Bickman, L., & Berkowitz, L. (1969). Note on the drawing power of crowds of different size. Journal � 27 of personality and social psychology, 13(2), 79.
STUDY 3: DRAWING POWER OF CROWDS N = 1,424 Milgram, S., Bickman, L., & Berkowitz, L. (1969). Note on the drawing power of crowds of different size. Journal � 28 of personality and social psychology, 13(2), 79.
� 29
RESULTS Divided Into Four Zones Default Field of View (101°) Zone 1 Zone 2 Zone 4 Front Zone 3 � 30
Participant Male avatar FOUR CONDITIONS Female avatar Zero Low Medium High Size of stimulus crowd � 31
Participant GAZE* DISTRIBUTION 0% 100% Male avatar Female avatar Zero (N = 15) Low (N = 15) Medium (N = 13) High (N = 13) Size of stimulus crowd * Head rotation used as a proxy for gaze. � 32
Participant GAZE* DISTRIBUTION 0% 100% Male avatar Female avatar 60% 14% 18% 8% Zero (N = 15) Low (N = 15) Medium (N = 13) High (N = 13) Size of stimulus crowd * Head rotation used as a proxy for gaze. � 33
Participant GAZE* DISTRIBUTION 0% 100% Male avatar Female avatar *** statistically significant 47% 60% 54% 47% 16% 23% 14% 18% 12% 22% 16% 24% 14% 8% 12% 13% Zero (N = 15) Low (N = 15) Medium (N = 13) High (N = 13) Size of stimulus crowd * Head rotation used as a proxy for gaze. � 34
RECAP Study 1: Restorative Effects of Virtual Environments [Valtchanov et al. 2010] , 22 (original) v.s. 55 (ours) A user’s feeling of being transported into the rendered environment Place Illusion Study 2: Proteus Effect [Yee and Bailenson, 2007] , 50 (original) v.s. 59 (ours) A user’s feeling of experiencing the virtual world through an avatar Embodiment Illusion Study 3: Drawing Power of Crowds [Milgram et al. 1969] , 1,424 (original) v.s. 56 (ours) A user’s feeling that events happening in the virtual world are real Plausibility Illusion Source code and data are available at: bit.ly/VRCrowdExperiments � 35
DISCUSSION � 36
COMBINING THE BEST OF BOTH WORLDS Virtual Reality Crowdsourced VR Crowdsourcing (as of 2017) (this work) Manipulation + High - Low + High Realism Measurement + High - Low + High Granularity Participant - Low + High + High Diversity Reproducibility - Low + High + High � 37
1. MANIPULATION REALISIM Study 1: Restorative Effects of Virtual Environments [Valtchanov et al. 2010] , N = 22 (original) v.s. 55 (ours) Illusion Delivered Place Illusion Study 2: Proteus Effect [Yee and Bailenson, 2007] , N = 50 (original) v.s. 59 (ours) Illusion Delivered Embodiment Illusion Study 3: Drawing Power of Crowds [Milgram et al. 1969] , N = 1,424 (original) v.s. 56 (ours) Illusion Delivered Plausibility Illusion � 38
2. MEASUREMENT GRANULARITY Study 1: Restorative Effects of Virtual Environments [Valtchanov et al. 2010] , N = 22 (original) v.s. 55 (ours) Illusion Delivered Place Illusion Study 2: Proteus Effect [Yee and Bailenson, 2007] , N = 50 (original) v.s. 59 (ours) Illusion Delivered Embodiment Illusion Study 3: Drawing Power of Crowds [Milgram et al. 1969] , N = 1,424 (original) v.s. 56 (ours) Illusion Delivered Plausibility Illusion Head Rotation � 39
Recommend
More recommend
