XR Design Challenges and Opportunities for Cognitive Performance Elaine M. Raybourn, Applied Cognitive Science emraybo@sandia.gov IT²EC So your code will see the future. International Training Technology Exhibition & Conference April 28-30, 2020 exascaleproject.org ideas-productivity.org SAND2019-14911 C
I help organizations tell their stories. • I am a social scientist who deepens understanding by being embedded in different cultures. • Cultures: DARPA, DoD (Army, SOF, USMC, OSD, Team Orlando), BT Research, FhG FIT, INRIA, DOE Office of Science, ECP, Academia, National Labs • Sandia Site PI for Interoperable Design of Extreme-scale Application Software (IDEAS-ECP) • Research focus on ECP team productivity since 2017, transmedia learning since 2010, games, immersive simulations, dispersed (virtual) teams and intelligent community systems since 2000 • Passion: Seize opportunities that allow us to learn about ourselves and others • Favorite question: Why not? 2 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09
3 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09
Why narrative? We are wired for it. Your life is a narrative. We experience events with a time-structure, with cause-and-effect; a narrative adds context, detail, and meaning, unlike a set of bullet points. Information embedded in a paragraph, which tells a story was better recalled than information a list of ingredients. LaBar, K. S., & Cabeza, R. (2006). Cognitive neuroscience of emotional memory. Nature Reviews Neuroscience , 7 (1), 54. 4 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09
Our cognition is increasingly di distribu stributed ted 5 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09
The cognitive challenge is purposeful and productive learning in the wild Transmedia learning is the scalable system of messages representing a core experience that unfolds from the use of multiple media and emotionally engages learners by involving them personally in the story. Raybourn, 2014, Journal of Computational Science Raybourn, E.M. (2014). A new paradigm for serious games: Transmedia learning for more effective training & education. Journal of Computational Science , (5) 3, Elsevier, 471 – 481 6 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09
A XR design challenge is storytelling for cognitive performance . In Gene Roddenberry’s Star Trek , it was not just the pointer to futuristic technologies of the holodeck that inspired audiences around the globe, but rather the narratives, or stories, created in it . Raybourn E.M., et al. (2019) Information Design for XR Immersive Environments: Challenges and Opportunities. In: Chen J., Fragomeni G. (eds) Virtual, Augmented and Mixed Reality. Multimodal Interaction. HCII 2019. Lecture Notes in Computer Science, vol 11574. pp. 1-12. Springer, Cham https://doi.org/10.1007/978-3-030-21607-8_12 7 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09
What is the core What is the learner experience or story? analysis? What co-creation What is the context, or of content will prior experience? reinforce learning? Simulation Experience Design Framework, Raybourn (2007) Raybourn, E. M., (2007). Applying simulation experience design methods to creating serious game-based adaptive training systems. Interacting with Computers , 19, Elsevier, 207-14. 8 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09
1. Challenges and opportunities for cognitive performance The communication space: Support quiet reflection and active immersion Image source: Elaine Raybourn 9 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09
2. Challenges and opportunities for cognitive performance Image source: Elaine Raybourn The storytelling space: Achieve co- created immersion with and through interactivity 10 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09
3. Challenges and opportunities for cognitive performance The space isn’t a place: Achieve immersion with contextual cues. Image source: Kyle Cochrane 11 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09
4. Challenges and opportunities for cognitive performance The space for diversity: Support multiple points of view and perceptions in multiple spaces Image source: https://tvline.com/2018/01/09/black-mirror-uss-callister-spaceship-tv-series-netflix/ 12 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09
5. Challenges and opportunities for cognitive performance A space for evaluation: measure the effectiveness of training, evaluate efficacy of the experience. Image source: https://screenrant.com/mind-blowing-twists-black-mirror-tv-show/, Playtest 13 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09
Acknowledgements and citation • This work was supported by the U.S. Department of Energy Office of Science, Office of Advanced Scientific Computing Research (ASCR), and by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration. • Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA-0003525. Citation • Raybourn, E.M., Stubblefield, W., Trumbo, M. Jones, A., Whetzel, J., and Fabian, N. (2019). Information Design for XR Immersive Environments: Challenges and Opportunities. In J. Y. C. Chen and G. Fragomeni (Eds.) Virtual, Augmented and Mixed Reality: Multimodal Interaction. HCII 2019, LNCS 11574, pp. 1 – 12, 2019. https://doi.org/10.1007/978-3-030-21607-8_12 14 AR Design: Challenges and Opportunities for Cognitive Performance @ 11:00AM 2019-12-09
Thank you for your participation! @elaineraybourn https://bssw.io So your code will see the future. exascaleproject.org ideas-productivity.org SAND2018-4036 C
Recommend
More recommend
