for Cognitive Performance Elaine M. Raybourn, Applied Cognitive - - PowerPoint PPT Presentation

exascaleproject.org ideas-productivity.org

XR Design Challenges and Opportunities for Cognitive Performance

Elaine M. Raybourn, Applied Cognitive Science emraybo@sandia.gov IT²EC International Training Technology Exhibition & Conference April 28-30, 2020

So your code will see the future. SAND2019-14911 C

2 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09

I help organizations tell their stories.

• Cultures: DARPA, DoD (Army, SOF, USMC, OSD, Team Orlando), BT Research, FhG FIT,

INRIA, DOE Office of Science, ECP, Academia, National Labs

• Sandia Site PI for Interoperable Design of Extreme-scale Application Software (IDEAS-ECP)
• Research focus on ECP team productivity since 2017, transmedia learning since 2010, games,

immersive simulations, dispersed (virtual) teams and intelligent community systems since 2000

• Passion: Seize opportunities that allow us to learn about ourselves and others
• Favorite question: Why not?
• I am a social scientist who

deepens understanding by being embedded in different cultures.

3 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09

4 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09

Why narrative? We are wired for it.

Your life is a narrative. We experience events with a time-structure, with cause-and-effect; a narrative adds context, detail, and meaning, unlike a set of bullet

• points. Information

embedded in a paragraph, which tells a story was better recalled than information a list of ingredients.

LaBar, K. S., & Cabeza, R. (2006). Cognitive neuroscience of emotional memory. Nature Reviews Neuroscience, 7(1), 54.

5 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09

Our cognition is

increasingly

di distribu stributed ted

6 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09

The cognitive challenge is purposeful and productive learning in the wild

Transmedia learning is the scalable system of messages representing a core experience that unfolds from the use of multiple media and emotionally engages learners by involving them personally in the story.

Raybourn, 2014, Journal of Computational Science

Raybourn, E.M. (2014). A new paradigm for serious games: Transmedia learning for more effective training & education. Journal of Computational Science, (5) 3, Elsevier, 471–481

7 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09

In Gene Roddenberry’s Star Trek, it was not just the pointer to futuristic technologies of the holodeck that inspired audiences around the globe, but rather the narratives,

• r stories, created in it.

A XR design challenge is storytelling for cognitive performance.

Raybourn E.M., et al. (2019) Information Design for XR Immersive Environments: Challenges and Opportunities. In: Chen J., Fragomeni G. (eds) Virtual, Augmented and Mixed Reality. Multimodal Interaction. HCII 2019. Lecture Notes in Computer Science, vol 11574. pp. 1-12. Springer, Cham https://doi.org/10.1007/978-3-030-21607-8_12

8 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09

What is the learner analysis? What is the core experience or story?

What is the context, or prior experience?

What co-creation

• f content will

reinforce learning? Simulation Experience Design Framework, Raybourn (2007)

Raybourn, E. M., (2007). Applying simulation experience design methods to creating serious game-based adaptive training systems. Interacting with Computers, 19, Elsevier, 207-14.

9 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09

• 1. Challenges and opportunities for cognitive performance

The communication space: Support quiet reflection and active immersion

Image source: Elaine Raybourn

10 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09

• 2. Challenges and opportunities for cognitive performance

The storytelling space: Achieve co- created immersion with and through interactivity

Image source: Elaine Raybourn

11 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09

• 3. Challenges and opportunities for cognitive performance

The space isn’t a place: Achieve immersion with contextual cues.

Image source: Kyle Cochrane

12 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09

• 4. Challenges and opportunities for cognitive performance

The space for diversity: Support multiple points

• f view and perceptions

in multiple spaces

Image source: https://tvline.com/2018/01/09/black-mirror-uss-callister-spaceship-tv-series-netflix/

13 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09

• 5. Challenges and opportunities for cognitive performance

A space for evaluation: measure the effectiveness of training, evaluate efficacy of the experience.

Image source: https://screenrant.com/mind-blowing-twists-black-mirror-tv-show/, Playtest

14 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09

Acknowledgements and citation

• This work was supported by the U.S. Department of Energy Office of Science, Office of

Advanced Scientific Computing Research (ASCR), and by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration.

• Sandia National Laboratories is a multimission laboratory managed and operated by

National Technology and Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA-0003525. Citation

• Raybourn, E.M., Stubblefield, W., Trumbo, M. Jones, A., Whetzel, J., and Fabian, N.

(2019). Information Design for XR Immersive Environments: Challenges and

• Opportunities. In J. Y. C. Chen and G. Fragomeni (Eds.) Virtual, Augmented and Mixed

Reality: Multimodal Interaction. HCII 2019, LNCS 11574, pp. 1–12, 2019. https://doi.org/10.1007/978-3-030-21607-8_12

exascaleproject.org ideas-productivity.org

Thank you for your participation!

@elaineraybourn

https://bssw.io So your code will see the future.

SAND2018-4036 C