The HoloRange AR Sandtable Prototype Lessons Learned April 29, 2020 - PowerPoint PPT Presentation

DRAFT 1-6-20 The HoloRange AR Sandtable Prototype Lessons Learned April 29, 2020 Presenters Rodney Choi Daniel J. Lacks, PhD Senior Program Manager Chief Scientist

Background • CESI recently created a USMC wargaming prototype Augmented Reality (AR) sand table called the HoloRange • HoloRange is built with Unity to allow portability to any XR device • The USMC wargaming prototype is currently ported to the Magic Leap One

Sand Tables Hasty setup Analog Twin Organized and Collaborative

AR Devices • Similar to eyeglasses but with an image superimposed in your line of sight. This allows the wearer to be able to walk around the room or table without fear of falling or tripping. • Features: • Computer vision • Spatial Audio • Prescription Insert • Controller • Operating System • CPU/GPU/Storage/RAM/Network • Rechargeable

Simulator Driver • Display requires some driver to place and move objects and model the digital twin • The demo AR capability is fed by a simulation feed from OneSAF • OneSAF is a US Army open source simulation toolkit used in training, experimentation, acquisition, and analysis domains • Communicates with the HoloRange using a combination of web service calls and a CESI-developed interoperability framework called MSSV

Spatial Computing • A spatial computer, like an AR device, knows where it is in space • It uses a variety of sensors and cameras to build an understanding Guided mesh of both its environment and its user to blend the digital world with the real world • Typically applications require Guided Meshing, i.e. scanning the room. Instead, we scan a QR Code • Rapid set-up • Dispersed participants QR code

Live Demo

Important Takeaways • While the prototype is built for wargaming, there is a lot of overlap with training applications. • Planning, runtime, and after action review. • Course of Action Development • Rehearsal of Concept (ROC) Drills • Common Operational Picture (COP) Representation • Terrain view is scalable. Can display at individual shooter level or theater level • The AR system is a 3D view. It is not limited to a maneuver view. It could show network topologies, population distribution, anything that can be depicted in 2D or 3D • Technology implementation (i.e., which headset) is less important than the system design philosophy. Must consider human factors to move beyond a science demonstration

Thank You Daniel J. Lacks, PhD Rodney Choi Chief Scientist Senior Program Manager Daniel.Lacks@cesicorp.com Rod.Choi@cesicorp.com +1-407-674-8326 +1-407-674-8307

Gaming Development Software • Software Development Kit (SDK) provides hooks for game engines into the AR OS and hardware • Program graphics and control • Similar to game development to control the camera, light, assets, prefabs, etc. but with additional AR feature support such as meshing with a spatial mapper in the scene, eye tracking, hand gestures, gaze, etc. • Your software may be published to an app store

Spatial Computing • A spatial computer, like an AR device, knows where it is in space • It uses a variety of sensors and cameras to build an understanding Guided mesh of both its environment and its user to blend the digital world with the real world • Typically applications require Guided Meshing, i.e. scanning the room. Instead, we scan a QR Code • Rapid set-up • Dispersed participants QR code

Usability and Design • The default view is the collaborative sand table map locked onto the table, floor, or wall • Participants may activate a personal mode so that they may interact with the simulation in their own space • The user is free to pan, tilt, and zoom the map without affecting the collaborative view • Users may interact with the game; shared map participants are aware of areas being worked on • Participants may place sticky notes

Terrain • CESI imported real world 3D terrain from Open Street Maps using a tool called Key Terrain • Key Terrain (http://keyterrain.io) is a CESI online website that generates terrain from open source data in a variety of simulation and gaming formats • The terrain supports 3D buildings in which simulated entities can enter buildings, terrain, and tunnels • The terrain supports calculations such as attenuated line of site, querying features • Terrain is correlated with OneSAF, however coordinate system conversions are a challenge • Unity terrain format supported by NGA’s Foundation GEOINT effort

Grease Pen Drawings • We also developed a “John Madden style” grease pen capability to draw freehand pictures in 3D • Depict routes, air corridors, arrows, etc. • The controller is used as a wand Freehand • Supports snapping an existing drawing to the ground Wand

Billboards • Integrate data and analytics into the range • Support for line, bar, pie charts • Monitor projected at an end of the sand table • Anticipating device support for web pages and video feeds Early Prototype

Symbology • Icons are served by a custom symbology web server prototype microservice which allows applications to 3D Models in range load symbols such as: Web service • MIL-STD-2525D symbology • OneSAF or simulator icons • FilmBox files • Sprites and other gaming graphics Symbology in range