Flying the Unfriendly Skies: Overcoming Obstacles in Legacy - - PDF document

IT2EC 2020 IT2EC Extended Abstract Template Presentation/Panel

Flying the Unfriendly Skies: Overcoming Obstacles in Legacy Simulations to Improve Training

John Williamson1, Jennifer Lewis2

1Game Development Specialist, SAIC, Seattle, USA 2 Modeling and Simulation Engineer, SAIC, Dallas, USA

Abstract — Modern training demands push legacy simulation systems to their breaking points. Currently, no single game or simulation engine provides a robust, flexible, and secure platform on which to integrate modern training requirements, such as Learning Management Systems, biometrics, data analytics, and synthetic prototypes. This presentation proposes a new approach, incorporating instructional systems design and data analysis, allowing a student’s successes and failures to affect their options later in the learning flow. This presentation focuses on the technical challenges of creating an immersive, realistic training world that allows open-ended interaction from a variety

• f data analytics, biometric and learning science tools, to include security, privacy, fidelity, extensibility, and high-

value student performance tracking.

1 Introduction

For the past two years, Learning Next initiatives out of the US Air Force (USAF) and US Army (USAR) graduated pilot candidates using experimental tools, techniques, and technologies. This military-focused experimentation produced valuable data-backed insights and lessons learned in a wide variety of functional areas, to include data analysis, human performance and the use of immersive extended reality (XR) technology. A key hindrance in the experimental programs, however, is the lack of an integrated learning platform that provides a physics-based world in which to conduct all lessons, skills, and activities in a single learning flow that can enable the use of a Train Learn Reflect Train Again (TLRTA) methodology.

2 Approach

Learning Next proposes a new approach, incorporating instructional systems design and data analysis, allowing a student’s successes and failures to affect their options later in the learning flow. This solution uses Commercial Off the Shelf (COTS) gaming technology to allow for rapid prototyping and demonstrates the ability to simulate the complete lifecycle of a mission. For example, a student pilot who failed to detect a hydraulic leak during the pre- flight inspection would need to manually lower and lock his landing gear prior to landing his next mission or address a more serious emergency. This approach allows students to train to specific learning objectives through consequences rather than through multiple-choice

• assessments. More importantly, this approach allows

detailed data collection for any action or decision the student makes, setting the stage for predictive analysis of a student’s potential piloting skill as well as his strategic thinking ability. Finally, the approach adds retention concepts from popular commercial games such as quest based learning, leaderboards and achievements to self- motivate and reward students to log additional and high quality flight hours in the simulation.

3 Results and Discussion

A typical learner journey through the Learning Next environment would start on the virtual reality (VR) flight line where students get their first look at the aircraft they are learning to fly. Figure 1 shows the integration of academic concepts into this introductory look, allowing students to maneuver throughout the aircraft while learning its basic systems.

• Fig. 1. Electrical system concepts integrated as part of mission

execution training.

Each interaction with a component of the aircraft, even at this introductory level, represents an exposure to academic concepts that the system logs and analyzes for progress tracking and predictive analysis. This detailed interaction tracking minimizes the need for traditional assessments and allows students to maintain a level of immersion that makes VR effective. By eliminating the need to stop and take a test, students can be more fully present during both their training and assessment. Anecdotally, higher presence is associated with higher transfer, and studies empirically demonstrated a positive association between presence and transfer [1,2].

IT2EC 2020 IT2EC Extended Abstract Template Presentation/Panel The system collects data about student progress using a variety of data stores, including traditional relational databases as well as a student-centric Learning Record Store (LRS), which implements the Experience API (xAPI) standard. Each data store integrates into a cloud- hosted analytic workflow that provides recommendations

• n the next activities a student should pursue in the course,

presented via the Learning Management System (LMS), a student’s online portal to hosted content and activities. The analytic workflow also informs role-based data visualizations for students, instructors and leadership, as shown in Figure 2. The integration of these systems presents major challenges, as discussed below.

• Fig. 2. Example student feedback integrating telemetry, eye

tracking and biometric data.

3.1. Security and Privacy Learning Next implements multi-layer security to ensure the integrity of the learning platform. Unlike most commercially available flight simulations, it implements secure socket layer (SSL) encryption to send flight and training data to the backend LRS. The flight simulator and training platform locally encrypts data to aid in after action review (AAR). However, this data is never permanently stored, and no locally stored data includes personally identifiable information (PII). The system maintains all PII, such as the student’s name and identification number, in the backend LRS and segments PII from the raw and modeled data about the student’s progress and ongoing learning profile. Unique user accounts, optionally using CAC or military email authentication, combined with salted and hashed user-defined passwords ensures a student’s secure access to the flight sim as well as the LMS. Unique user access levels (students, instructors, commanders, administrators) provide different levels of access to data while automatic validation of local program and data files ensures clients and servers can reject unknown or obsolete connections. Network and data safe programming techniques, such as packet and command validation, prevent injection of invalid data into the system. 3.2. Fidelity and Extensibility Learning Next engineers did not design the integrated training platform to compete with the highest fidelity flight models of multi-million dollar simulations. Instead, the team focused on creating an accurate flight model sufficient for training, incorporating all standard simulation variables such as lift, drag, thrust, weight, center of gravity, and fuel consumption, while maintaining an affordable, portable and modular architecture that generates a wealth of data useful for predictive analysis of the student’s progress and future learning journey. The team consistently performs tradeoff analysis between form and function in this space. The platform’s design also allows for formal validation using both pilot subject matter expertise and black box recordings. 3.3. Student Performance Tracking Although a great deal of modeling, simulation and extended reality (XR) work goes into creating this type of integrated learning platform, the ultimate goal is the predictive analysis capabilities required to personalize the learner’s journey. To reach this goal, the system captures timestamped data for every instrument, switch, engine state, flight control position and aircraft attitude as well as every learner interaction, eye movement, and biometric data change. Based on the captured raw data, the system determines if the learner, for example, completed all checklist items, looked at the cockpit warning system (CWS) annunciator panel at the appropriate time, or moved the power control lever (PCL) to idle within the expected timeframe. The answers to these questions then inform the student’s rate of progress toward competency- based learning objectives written in accordance with Bloom’s Taxonomy, a classification tool educators use to identify the complexity of the material a student is

• learning. The student’s rate of progress toward levels of

mastery, such as remembering, applying and creating, provides insight into the next activities the learner should

• attempt. This tracking and analysis complete the learning

cycle that allows students to TLRTA.

4 Future Work

The single largest driving factor for the creation of Learning Next’s flight simulator is the lack of extensibility in any other solution. Learning Next builds upon Unity 3D, a commercial game engine, to allow engineers to port its capabilities to the widest variety of hardware solutions, e.g. personal computer (PC), game consoles, augmented reality (AR) devices such as HoloLens or Magic Leap, VR devices such as Oculus Quest or HTC Vive, and Android and Apple tablets and phones. Moving forward, engineers will tailor the Learning Next capabilities to the limits and advantages of specific

• hardware. For example, AR or untethered VR devices

provide more portability for practicing skills that require a significant amount of space while tablets and phones provide accessible content for subject areas that do not translate well to VR. More importantly, converting gamified training content to the next generation of game consoles, such as Xbox and Playstation, will allow Learning Next to serve as an educational awareness platform by allowing an accessible, realistic flight sim that

• nly requires a GamePad to fly. Students can already fly

portions of the Learning Next flight simulation on the Oculus Quest, leading to a real possibility of student pilots

IT2EC 2020 IT2EC Extended Abstract Template Presentation/Panel arriving at flight school with a $400 head mounted display and a firm understanding of basic aviation concepts normally covered in the first weeks of training.

Acknowledgements

The authors would like to thank Lt Col Paul “Slew” Vicars for setting a culture of teamwork and diligence that provided our team with opportunities to grow, fail, and

• learn. We would also like to thank our incredible Learning

Next teammates for their contributions to this presentation and to the program.

References

[1] Winn, W., Windschitl, M., Fruland, R. and Lee, Y. (2002) When Does Immersion in a Virtual Environment Help Students Construct Understanding? Proceedings of the International Conference of the Learning Sciences, Mahwah, 2002, 497-503. [2] Mikropoulos, T. (2006) Presence: A Unique Characteristic in Educational Virtual Environments. Virtual Reality, 10, 197-206. http://dx.doi.org/10.1007/s10055-006-0039-1

Author/Speaker Biographies

John Williamson has worked as a commercial game designer and producer on nearly three dozen titles in nearly every gaming genre and platform. In addition, he has worked with virtual reality since the mid-1990’s, first as research toward his Masters and later to create serious games for the US military. Jennifer Lewis is a simulation engineer who has developed interoperability solutions for distributed simulation and training environments for the past 18 years. She holds a Master of Science in Computer Science from the University of Texas at Dallas and is a Certified Modeling and Simulation Professional.