The HoloRange AR Sandtable Prototype Lessons Learned April 29, 2020 - - PowerPoint PPT Presentation

The HoloRange AR Sandtable Prototype

Lessons Learned

April 29, 2020 Presenters

Daniel J. Lacks, PhD Chief Scientist Rodney Choi Senior Program Manager

DRAFT 1-6-20

Background

• CESI recently created a USMC

wargaming prototype Augmented Reality (AR) sand table called the HoloRange

• HoloRange is built with Unity to

allow portability to any XR device

• The USMC wargaming prototype

is currently ported to the Magic Leap One

Sand Tables

Analog Twin Hasty setup Organized and Collaborative

AR Devices

• Similar to eyeglasses but with an

image superimposed in your line of

• sight. This allows the wearer to be

able to walk around the room or table without fear of falling or tripping.

• Features:
• Computer vision
• Spatial Audio
• Prescription Insert
• Controller
• Operating System
• CPU/GPU/Storage/RAM/Network
• Rechargeable

Simulator Driver

• Display requires some driver to

place and move objects and model the digital twin

• The demo AR capability is fed by a

simulation feed from OneSAF

• OneSAF is a US Army open source

simulation toolkit used in training, experimentation, acquisition, and analysis domains

• Communicates with the HoloRange

using a combination of web service calls and a CESI-developed interoperability framework called MSSV

Spatial Computing

• A spatial computer, like an AR

device, knows where it is in space

• It uses a variety of sensors and

cameras to build an understanding

• f both its environment and its

user to blend the digital world with the real world

• Typically applications require

Guided Meshing, i.e. scanning the

• room. Instead, we scan a QR Code
• Rapid set-up
• Dispersed participants

Guided mesh QR code

Live Demo

Important Takeaways

• While the prototype is built for wargaming, there is

a lot of overlap with training applications.

• Planning, runtime, and after action review.
• Course of Action Development
• Rehearsal of Concept (ROC) Drills
• Common Operational Picture (COP) Representation
• Terrain view is scalable. Can display at individual

shooter level or theater level

• The AR system is a 3D view. It is not limited to a

maneuver view. It could show network topologies, population distribution, anything that can be depicted in 2D or 3D

• Technology implementation (i.e., which headset) is

less important than the system design philosophy. Must consider human factors to move beyond a science demonstration

Thank You

Daniel J. Lacks, PhD Chief Scientist Daniel.Lacks@cesicorp.com +1-407-674-8326 Rodney Choi Senior Program Manager Rod.Choi@cesicorp.com +1-407-674-8307

Gaming Development Software

• Software Development Kit (SDK)

provides hooks for game engines into the AR OS and hardware

• Program graphics and control
• Similar to game development to

control the camera, light, assets, prefabs, etc. but with additional AR feature support such as meshing with a spatial mapper in the scene, eye tracking, hand gestures, gaze, etc.

• Your software may be published to

an app store

Spatial Computing

• A spatial computer, like an AR

device, knows where it is in space

• It uses a variety of sensors and

cameras to build an understanding

• f both its environment and its

user to blend the digital world with the real world

• Typically applications require

Guided Meshing, i.e. scanning the

• room. Instead, we scan a QR Code
• Rapid set-up
• Dispersed participants

Guided mesh QR code

Usability and Design

• The default view is the

collaborative sand table map locked onto the table, floor, or wall

• Participants may activate a

personal mode so that they may interact with the simulation in their

• wn space
• The user is free to pan, tilt, and zoom

the map without affecting the collaborative view

• Users may interact with the game;

shared map participants are aware of areas being worked on

• Participants may place sticky notes

Terrain

• CESI imported real world 3D terrain from

Open Street Maps using a tool called Key Terrain

• Key Terrain (http://keyterrain.io) is a CESI
• nline website that generates terrain

from open source data in a variety of simulation and gaming formats

• The terrain supports 3D buildings in

which simulated entities can enter buildings, terrain, and tunnels

• The terrain supports calculations such as

attenuated line of site, querying features

• Terrain is correlated with OneSAF,

however coordinate system conversions are a challenge

• Unity terrain format supported by NGA’s

Foundation GEOINT effort

Grease Pen Drawings

• We also developed a “John

Madden style” grease pen capability to draw freehand pictures in 3D

• Depict routes, air corridors,

arrows, etc.

• The controller is used as a wand
• Supports snapping an existing

drawing to the ground

Freehand Wand

Billboards

• Integrate data and analytics into

the range

• Support for line, bar, pie charts
• Monitor projected at an end of

the sand table

• Anticipating device support for

web pages and video feeds

Early Prototype

Symbology

• Icons are served by a custom

symbology web server prototype microservice which allows applications to load symbols such as:

• MIL-STD-2525D symbology
• OneSAF or simulator icons
• FilmBox files
• Sprites and other gaming

graphics

Web service Symbology in range 3D Models in range