VR-Based Toolsets for Instructors DR. Hashim Yaqub BMT , UK - - PowerPoint PPT Presentation

VR-Based Toolsets for Instructors

• DR. Hashim Yaqub

BMT , UK

Abstract

This paper outlines the development of a VR-based toolset which allow instructors to monitor the activity of students in a virtual-training scenario. They also allow instructors to issue aid and introduce dynamic obstacles or changes in

• task. The tool makes use of consumer VR technologies and incorporates gaze-

aided interaction as well as design principals found in games using an omniscient viewpoint.

ENGAGE

 Rapid Virtual Training Prototyping  Learning Record Store  Tools for students and instructors  QEC, Type 26, Caimen-90 Fast, Sub Projects

Use Case- QEC



PC Mouse + Keyboard



Trainees/Students



Work individually or collaboratively



Practice, assessment or free- roam



Trainers



Curate Scenarios (Procedures and Emergencies)



Observe and Interfere in Live Scenario

Instructor Toolset Requirements

 Monitoring multiple virtual students  PC or VR-based

 Consumer (non-specialist tools)  Using standard VR Kit (controllers and/or integrated hand-tracking)

 Communicating with one or more students  Interfering in live scenario i.e. changing objectives or introducing emergency

situations

 Automated and Manual guidance (e.g. Waypoints and hint beacons)  Easy search and selection of students, objects and areas

Instructor Toolset Requirements



Comprehensive Map Interaction



Swiftly view relevant areas on the map of the vessel



Isolate individual levels/decks.



Selecting and Directing



Identifying student avatars



Selecting Individual or Group



Communicating with Individual or Group



Targeting Waypointing and Direction



Curating Scenarios



Setting up procedures



Setting up routes



Setting up emergency situations



Activity and Progress Tracking



There should be a comprehensive UI for displaying the progress of each student and/or the task they are performing.



Using this same functionality the instructor can modify the task in real-time.

Omniscient VR

 God Games  RTS/RPG  The Sims  XCOM  Deism

PC vs VR

 PC interface is well established

 Mouse + Keyboard  Gaming  Universal System Requirements



VR-Guide: A Specific User Role for Asymmetric Virtual Reality Setups in Distributed Virtual Reality Applications (2018)



• Horst. R., Dorner, R. & Peter, M.

PC vs VR

 VR

 Can be more mobile (Oculus

Quest)

 Make use of virtual space to

visualise large amount of information

 Easier to manipulate and view

3D assets in using relatively more naturalistic interaction

 Gaze-aided interaction

Interaction Challenges

 VR Interaction Modalities

 Motion Controllers  Hand-Tracking  Gaze-Aided Interaction  Distal Pointing

 Scenario Monitoring

 After-Action Review  Student Progress  Student Location

A human motor behavior model for distal pointing tasks (2010)

Kopper et. al



Spacetime: Enabling Fluid Individual and Collaborative Editing in Virtual Reality (2018)



• Xia. H., Herscher. S., Perlin. K., Wigdor., D

Gaze Interaction

 Gaze input

 Primary method of interaction  Using focal point and/or as direct

input

 Blink, timed focus, other novel

methods

 Gaze-Aided Interaction

 Using focal point to predict and

augment interaction

 Typical controller  Hand Tracking

Gaze + pinch interaction in virtual reality (2017) Pfeuffer et. al

DualGaze: Addressing the Midas Touch Problem in Gaze Mediated VR Interaction (2018) Mohan et. al

Left Hand Right Hand EYE Ray Hit Right Ray Hit Left Ray Hit Gaze Influence Area

Gaze-Aided Interaction Setup

Yellow and green line indicating object being made ‘selectable’ for each hand. Gaze influence area represented by mesh sphere around the eye hit point.

Right Ray Hit Unaltered Left Ray Hit Altered To Eye Hit Gaze Influence Area Increased

Influencing Selectable Targets

Gaze influence comes into effect when eye hits a selectable target If hand ray hit falls within gaze influence area, the hand’s ‘selectable target’ is altered to eye target

Right Ray Hit Altered to Eye Hit Gaze Influence Area Increased

Influencing Selectable Targets

Gaze influence comes into effect when eye hits a selectable target If hand ray hit falls within gaze influence area, the hand’s ‘selectable target’ is altered to eye target

Left Hand Right Hand EYE Ray Hit Right Ray Hit Left Ray Hit Gaze Influence Area

Adjust Influence Area for Distance

Gaze influence area adjusted depending

• n distance from player

Adjust Influence Area for Distance

Experimental Conditions



Hardware and Modalities



PC vs VR



Normal Interaction vs Gaze-Aided Interaction



Hand Tracking vs Controller



Task

 Collocated or Remote 

Sequential or Free-form



Locating Actor



Different Floors

 Multiple Actor Selection Methods

 Direct Selection  UI Item

Experimental Task Design

 Checklist of Directives – Instruct students to…

 Move to location  Interact with objects  Look at point of interest

 Actors

 Simple AI  Mouse + Keyboard  VR  Mixture of all?

Working Toolkit [To be finished by final submission]

How it Benefits Instructors

 Enhanced instructor-student interaction  Designed to facilitate remote training and assessment  Comprehensive presentation of multiple student stats  Dynamic and customisable virtual workspace  Naturalistic and simple navigation of complex scaled virtual spaces

 Useful for interior environments (i.e. vessels)

 Provide multiple means of communication with students

 Individual and Group Instructions  VOIP  Waypointing and Signposting

Thank You! Any Questions?

 Dr. Hashim Yaqub  Hashim.Yaqub@bmtglobal.com  01225473484