Thomas A. Stoffregen Affordance Perception-Action Laboratory (APAL) - - PowerPoint PPT Presentation

Thomas A. Stoffregen

Affordance Perception-Action Laboratory (APAL) School of Kinesiology University of Minnesota

DSC Antibes September 11, 2020

Motion sickness in physical and virtual systems

Across platforms

 Motion sickness is widely known in VR: Experienced by millions of consumers  Motion sickness is very common in driving simulators: Among new drivers, and in

re-training older drivers

 Motion sickness will be common in autonomous vehicles  This conference acknowledges that motion sickness occurs across platforms  Physical and virtual motion  For this reason, this conference offers a unique opportunity to compare and

contrast

 Motion sickness in physical vehicles has been known for thousands of years  Motion sickness in virtual vehicles (a type of VIMS) is recent, but rapidly

increasing

 Some scholars consider these to be separate maladies  There are characteristic differences in symptom profiles  I consider them to be the same malady: Motion sickness

It’s all motion sickness

 Some risk factors are related to aspects of the technology

 IPD, closed-loop processing delays, field of view restrictions  In the VR community, most of the attention goes to these  Most of these do not apply to autonomous vehicles

 Other risk factors are related to behavioral issues  The “Driver-Passenger effect” Passengers are more likely to get sick than drivers  In physical vehicles: Drivers < Passengers  In virtual vehicles: Drivers < Passengers  In autonomous vehicles, everyone is a passenger

 That is, we must assume motion sickness will be more common in autonomous vehicles

Risk Factors

Chang, C.-H., Pan, W.-W., Tseng, L.-Y., & Stoffregen, T. A. (2012). Postural activity and motion sickness during video game play in children and adults. Experimental Brain Research, 217, 299-309. Chen, Y.-C., Dong, X., Chen, F.-C., & Stoffregen, T. A. (2012). Control of a virtual avatar influences postural activity and motion sickness. Ecological Psychology, 24, 279-299. Dong, X., Yoshida, K., & Stoffregen, T. A. (2011). Control of a virtual vehicle influences postural activity and motion sickness. Journal of Experimental Psychology: Applied, 17, 128-138. Draper, M. H., Viirre, E. S., Furness, T. A., & Gawron, V. J. (2001). Effects of image scale and system time delay on simulator sickness with head-coupled virtual environments. Human Factors, 43, 129-146 Stoffregen, T. A., Chen, Y.-C., & Koslucher, F. C. (2014). Motion control, motion sickness, and the postural dynamics of mobile devices. Experimental Brain Research, 232, 1389-1397. Stoffregen, T. A., Hettinger, L. J., Haas, M. W., Roe, M., & Smart, L. J. (2000). Postural instability and motion sickness in a fixed-base flight simulator. Human Factors, 42, 458-469. Stoffregen, T. A., Faugloire, E., Yoshida, K., Flanagan, M., & Merhi, O. (2008). Motion sickness and postural sway in console video games. Human Factors, 50, 322-331.

Sex differences

 One major risk factor is sex  In general, women > men  In seasickness the sex difference is 2:1  In virtual environments, it can be as high as 4:1  Research must address this problem  It may be related to virtual locomotion  It may interact with the “driver-passenger effect”

Curry, C., Li, R., Peterson, N. A., & Stoffregen, T. A. (2020). Cybersickness in virtual reality head-mounted displays: Examining the influence of sex differences and vehicle control. International Journal of Human-Computer Interaction, 36, 1161-1167. https://doi.org/10.1080/10447318.2020.1726108 Koslucher, F. C., Haaland, E., Malsch, A., Webeler, J., & Stoffregen, T. A. (2015). Sex differences in the incidence of motion sickness induced by linear visual oscillation. Aerospace Medicine and Human Performance, 86, 787-793. Munafo, J., Diedrick, M., & Stoffregen, T. A. (2017). The virtual reality head-mounted display Oculus Rift induces motion sickness and is sexist in its effects. Experimental Brain Research, 235, 889–901. DOI 10.1007/s00221-016-4846-7

Theoretical issues

 Commonly, motion sickness is understood through the Sensory Conflict

Theory

 The Sensory Conflict Theory motivates specific types of experimental

manipulations, and specific types of design interventions

 Our work is motivated by a different theory  The Postural Instability Theory motivates different experimental

manipulations, and different types of design interventions

 A central claim of the Postural Instability Theory is that there should exist

postural precursors of motion sickness

 Postural activity (standing, or seated) should differ between individuals who become

motion sick and those who do not, and those differences should exist before any subjective symptoms of motion sickness

Oman C. M. (1982). A heuristic mathematical model for the dynamics of sensory conflict and motion sickness. Acta Oto-Laryngol Suppl 392:44. Reason J. T. (1978). Motion sickness adaptation: A neural mismatch model. J Royal Soc Med 71:819– 829. Riccio, G. E., & Stoffregen, T. A. (1991). An ecological theory of motion sickness and postural instability. Ecological Psychology, 3, 195-240. Stoffregen, T. A., & Riccio, G. E. (1991). An ecological critique of the sensory conflict theory of motion sickness. Ecological Psychology, 3, 159-194.

Postural precursors are widespread

 Postural precursors of motion sickness have been identified in

many situations

 Postural precursors have been identified in postural activity

before subjects were exposed to any motion stimuli

 Physical vehicles (seasickness)  Fixed base flight simulators  Console video games (Xbox); driving games  Head mounted displays (Oculus Rift); driving games  Standing, but also seated

Bonnet, C. T., Faugloire, E. M., Riley, M. A., Bardy, B. G., & Stoffregen, T. A. (2006). Motion sickness preceded by unstable displacements of the center of pressure. Human Movement Science, 25, 800-820. Chen, Y.-C., Dong, X., Chen, F.-C., & Stoffregen, T. A. (2012). Control of a virtual avatar influences postural activity and motion sickness. Ecological Psychology, 24, 279-299. Dong, X., Yoshida, K., & Stoffregen, T. A. (2011). Control of a virtual vehicle influences postural activity and motion sickness. Journal of Experimental Psychology: Applied, 17, 128-138. Munafo, J., Diedrick, M., & Stoffregen, T. A. (2017). The virtual reality head-mounted display Oculus Rift induces motion sickness and is sexist in its effects. Experimental Brain Research, 235, 889–901. Stoffregen, T. A., Chen, Y.-C., & Koslucher, F. C. (2014). Motion control, motion sickness, and the postural dynamics of mobile devices. Experimental Brain Research, 232, 1389-1397. Stoffregen, T. A., Hettinger, L. J., Haas, M. W., Roe, M., & Smart, L. J. (2000). Postural instability and motion sickness in a fixed-base flight simulator. Human Factors, 42, 458-469. Stoffregen, T. A., Faugloire, E., Yoshida, K., Flanagan, M., & Merhi, O. (2008). Motion sickness and postural sway in console video games. Human Factors, 50, 322-331. Stoffregen, T. A., & Smart, L. J. (1998). Postural instability precedes motion sickness. Brain Research Bulletin, 47, 437-448.

Sample data: Postural sway



Raw data from 10-minute trials



Six subjects



Before anyone experienced any subjective symptoms



Top: Well



Bottom: Sick (later)

Bonnet, C. T., Faugloire, E. M., Riley, M. A., Bardy, B. G., & Stoffregen, T. A. (2006). Motion sickness preceded by unstable displacements of the center of pressure. Human Movement Science, 25, 800-820.

 On ships at sea, almost everyone is a passenger  We measured sway 24 hours before a sea voyage  Sway differed as a function of the incidence and severity of (later) seasickness  The effect was related to looking at the horizon

Postural precursors of seasickness

Nachum, Z., Shupak, A., Letichevsky, V., Ben-David, J., Tal, D., Tamir, A., Talmon, Y, Gordon, C. R., Luntz, M. (2004). Mal de debarquement and posture: Reduced reliance on vestibular and visual cues. The Lyaryngoscope, 114, 581-586. Stoffregen, T. A., Chen, F.-C., Varlet, M., Alcantara, C., & Bardy, B. G. (2013). Getting your sea legs. PLOS ONE, 8(6), e66949. Varlet, M., Bardy, B. G., Chen, F.-C., Alcantara, C., & Stoffregen, T. A. (2015). Coupling of postural activity with motion of a ship at sea. Experimental Brain Research, 233, 1607-1616.

 Motion sickness is more common

among women than men

 Physical vehicles  Virtual vehicles  For biomechanical reasons, women

and men sway differently

 Women and men have different

postural precursors of motion sickness

 Sex differences in postural precursors

• f motion sickness are qualitative

Sex differences

Curry, C., Peterson, N., Li, R., & Stoffregen, T. A. (2020). Postural precursors of motion sickness in head-mounted displays: Drivers and passengers, women and men. Ergonomics, in press. Koslucher, F. C., Munafo, J., & Stoffregen, T. A. (2016). Postural sway in men and women during nauseogenic motion of the illuminated environment. Experimental Brain Research, 234, 2709-2720. Munafo, J., Diedrick, M., & Stoffregen, T. A. (2017). The virtual reality head-mounted display Oculus Rift induces motion sickness and is sexist in its effects. Experimental Brain Research, 235, 889–901.

Women Men

Physical and virtual driving interact

 Does physical driving experience affect

movement during virtual driving?

 In virtual driving, are postural precursors

• f motion sickness affected by physical

driving experience?

 Seated subjects drove a virtual vehicle  Drivers had >20 years physical driving

experience

 Non-Drivers had no physical driving

experience

 Motion sickness did not differ between

Drivers and Non-Drivers

 Yet, we found found postural precursors

• f motion sickness in people with vs.

without physical driving experience

Chang, C.-H., Chen, F.-C., Kung, W.-C., & Stoffregen, T. A. (2017). Effects of physical driving experience on body movement and motion sickness during virtual driving. Aerospace Medicine & Human Performance, 88, 985-992. Stoffregen, T. A., Chang, C.-H., Chen, F.-C., & Zeng, W.-J. (2017). Effects of decades of physical driving on body movement and motion sickness during virtual driving. PLOS ONE, 12(11): e0187120. https://doi.org/10.1371/journal. pone.0187120

 Physical vehicles affect sensory stimulation, but they also affect

the physical body

 Virtual vehicles affect sensory stimulation, but they also affect

the physical body

Summary: Movement matters!

 In individuals  Not based on subjective reports  Not based on unreliable physiological data  Not based on time-consuming cognitive or sensory

testing

 Based on objective movement data  The user doesn’t need to do anything, or even know

that it is happening

We can predict motion sickness

 Monitor postural kinematics  Detect individuals who are at risk of becoming unstable  In real time, modify system dynamics to promote stability  Prevent instability, and (so) prevent motion sickness  The User doesn’t have to do anything, or even know that it is

happening

 The technology becomes more inclusive  More users  End sex discrimination

Can we prevent motion sickness?

 Motion sickness is associated with a particular range of motion

frequencies (0.1 – 0.4 Hz)

 When people are exposed to these frequencies, posture becomes

unstable, and they get sick

 When real time monitoring identifies an individual at risk, the

system could implement a band-block filter

 Cut out the frequencies that are associated with motion sickness  Yes, that’s right, deliberately reduce fidelity  But, only for individuals who are at risk!  The suspension system on automobiles suppresses those same

frequencies

 Do you want a car that has no suspension?

Example

Stoffregen, T. A., & Smart, L. J. (1998). Postural instability precedes motion sickness. Brain Research Bulletin, 47, 437-448.

Thank you for your attention

Questions?

 Advances in VE/VR technology very often are

justified in terms of the changes they cause in subjective experience

 It “feels more real”, is more immersive, and so on  Many studies, many hypotheses about relations

between cybersickness and presence, immersion, vection, and so on

 Postural control is outside subjective awareness  But, postural control can be affected by subjective

awareness (clip of guy falling over)

What’s presence got to do with it?



Chemero, A. (2009). Radical embodied cognitive science. MIT Press.



Gibson, J. J. (1979). The ecological approach to visual perception. Houghton-Mifflin.



Riccio, G. E., & Stoffregen, T. A. (1991). An ecological theory of motion sickness and postural instability. Ecological Psychology, 3, 195-240.



Stoffregen, T. A., & Bardy, B. G. (2001). On specification and the senses. Behavioral and Brain Sciences, 24, 195-261.



Stoffregen, T. A., Mantel, B., Bardy, B. G. (2017). The senses considered as one perceptual system. Ecological Psychology, 29, 165-197.



Stoffregen, T. A., & Riccio, G. E. (1988). An ecological theory of orientation and the vestibular system. Psychological Review, 95, 3-14.

Sensory conflict: Hypothetical