Supernumerary Hand Illusions in Multimodal Realities Nina Rosa - - PowerPoint PPT Presentation

Self & Body Augmented Reality Further Research

Supernumerary Hand Illusions in Multimodal Realities

Nina Rosa

• Feb. 27, 2019

Self & Body Augmented Reality Further Research

PROGRAM

Self & Body Augmented Reality Further Research

Self & Body Augmented Reality Further Research

RUBBER HAND ILLUSION

Botvinick, M., & Cohen, J. (1998). Rubber hands ‘feel’ touch that eyes see. Nature, 391(6669), 756.

Image from: IJsselsteijn, W. A., de Kort, Y. A. W., & Haans, A. (2006). Is this my hand I see before me? The rubber hand illusion in reality, virtual reality, and mixed reality. Presence: Teleoper- ators and Virtual Environments, 15(4), 455-464.

Self & Body Augmented Reality Further Research

SENSE OF EMBODIMENT*

Body ownership is just one feeling connecting self and body.

◮ Ownership ◮ Self-location ◮ Agency

Separate concepts, not independent

*Kilteni, K., Groten, R., & Slater, M. (2012). The sense of embodiment in virtual reality. Presence: Teleoperators and Virtual Environments, 21(4), 373-387.

Self & Body Augmented Reality Further Research

FACTORS

Recall these factors in the illusion:

◮ Anthropomorphism ◮ Movement/sensory synchronity ◮ Visual discontinuity ◮ Relative orientation

Self & Body Augmented Reality Further Research

EXAMPLE FACTOR: PERSPECTIVE

Marc Owens, Avatar Machine, Talk to Me @ MoMA, 2011

Self & Body Augmented Reality Further Research

REPLACEMENT

Typical illusion setups look at replacement scenarios

IJsselsteijn, W. A., de Kort, Y. A. W., & Haans, A. (2006). Is this my hand I see before me? The rubber hand illusion in reality, virtual reality, and mixed reality. Presence: Teleoperators and Virtual Environments, 15(4), 455-464. Normand, J. M., Giannopoulos, E., Spanlang, B., & Slater, M. (2011). Multisensory stimulation can induce an illusion

• f larger belly size in immersive virtual reality. PloS one, 6(1), e16128.

Self & Body Augmented Reality Further Research

SUPERNUMERARY HAND ILLUSION

Guterstam, A., Petkova, V. I., & Ehrsson, H. H. (2011). The illusion of owning a third arm. PloS one, 6(2), e17208.

Self & Body Augmented Reality Further Research

ILLUSIONS IN VR

What about AR?

Won, A. S., Bailenson, J., Lee, J., & Lanier, J. (2015). Homuncular flexibility in virtual reality. Journal of Computer-Mediated Communication, 20(3), 241-259. Egeberg, M., Lind, S. L., Serubugo, S., Skantarova, D., & Kraus, M. (2016, March). Extending the human body in virtual reality: effect of sensory feedback on agency and ownership of virtual wings. In Proceedings of the 2016 Virtual Reality International Conference (p. 30). ACM.

Self & Body Augmented Reality Further Research

SUPERNUMERARY HAND ILLUSION IN AR

Self & Body Augmented Reality Further Research

THIRD ARM - FIRST ATTEMPT

◮ How does MM

feedback affect the SoE in an SHI in AR?

◮ hands, VT, VM ◮ 3 minutes ◮ 21-point Q ◮ data of 27

participants

Gilbers, C. (2017) The Sense of Embodiment in Augmented Reality: A Third Hand Illusion. Master thesis, Utrecht University.

Self & Body Augmented Reality Further Research

RESULTS

◮ Overall low ownership scores, possibly due to bad virtual

hand appearance emphasized by real environment

◮ Agency differences by active VM ◮ Number of hands did not play a role in agency ◮ No shift when 3 hands ◮ Possible extra location ◮ Shift when active VM

Self & Body Augmented Reality Further Research

THIRD ARM - SECOND ATTEMPT

◮ New hand ◮ Same

indep.vars. but not all combis

◮ 2 minutes +

threat

◮ 9-point Q +

SCR

Self & Body Augmented Reality Further Research

NUMBER OF HANDS

C1 - 3 hands, synch VT, synch VM C2 - 2 hands, synch VT, synch VM

Self & Body Augmented Reality Further Research

VISUAL-TACTILE SYNCHRONY

C3 - 3 hands, synch VT, no VM C4 - 3 hands, asynch VT, no VM

Self & Body Augmented Reality Further Research

VISUAL-MOTOR SYNCHRONY

C5 - 3 hands, no VT, synch VM C6 - 3 hands, no VT, asynch VM

Self & Body Augmented Reality Further Research

THREAT

Self & Body Augmented Reality Further Research

RESULTS

◮ Generally, ownership and agency Q-responses were

highest for C1 and C2, and lowest for C6.

◮ Location Q-responses were less positive, but found

differences in shift/more locations depending on #hands.

◮ No differences in SCR data!? Not even after habituation

correction!

◮ Combining Q-data and SCR-data found 2 groups:

• 1. low Q means, low Q sds, low-med SCR means
• 2. high Q means, high Q sds, low-high SCR means

◮ Hand realism

Self & Body Augmented Reality Further Research

SCR

C6_logSCR_10b_10s C5_logSCR_10b_10s C4_logSCR_10b_10s C3_logSCR_10b_10s C2_logSCR_10b_10s C1_logSCR_10b_10s

1.0 0.8 0.6 0.4 0.2 0.0

• 0.2

11 28 28

sixth_logSCR_10b_10s fifth_logSCR_10b_10s fourth_logSCR_10b_10s third_logSCR_10b_10s second_logSCR_10b_10s first_logSCR_10b_10s

1.0 0.8 0.6 0.4 0.2 0.0

• 0.2

28 18 28 11 11 5

Self & Body Augmented Reality Further Research

FOLLOW-UP QUESTION

Does personality influence the extent of ownership in rubber hand illusions? Does ownership over a supernumerary virtual arm positively influence performance in a task using that arm? (For thesis starting approximately April 2019) Pilot: Test ownership factors in this setting (For small project starting approximately April 2019)

Self & Body Augmented Reality Further Research

RESEARCH CONTEXT

◮ Player-Avatar Relation in Multimodal AR Games ◮ n.e.rosa@uu.nl

Rosa, N. (2016, October). Player/Avatar body relations in multimodal augmented reality games. In Proceedings of the 18th ACM International Conference on Multimodal Interaction (pp. 550-553). ACM.

Self & Body Augmented Reality Further Research

SUMMARY

• 1. The rubber hand illusion paradigm allows us to examine the

sense of embodiment: ownership, agency, self-location.

• 2. Our supernumerary hand illusions in AR suggest that:

◮ it is possible to own a third arm ◮ more synchronous cues may lead to greater ownership ◮ virtual hand appearance plays a role, possibly because the

fakeness is emphasized by the real environment