Immersive Analytics of Honey Bee Data Interaction and Visualisation - PowerPoint PPT Presentation

Expanded Perception and Interaction Centre Immersive Analytics of Honey Bee Data Interaction and Visualisation Design Huyen Nguyen | Postdoctoral Fellow Sydney, May 2018 YOW! Data Conference http://epicentre.matters.today

Research Background • 3D Navigation and Interaction • Collaborative Ergonomic Techniques in Immersive Design Process Systems 2 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Acknowledgement • CSIRO Collaborators • Ulrich Engelke • Justin Baker • Paulo de Souza • Florence Wang • Peter Marendy • Matt Adcock • Ray Williams • Stuart Anderson • Students • Holly Hutson • Sarah Ketchell • Ben Ward 3 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Expanded Perception and Interaction Centre Global Initiative for Honey bee Health

GIHH https://www.youtube.com/watch?v=B2G4zmW8Gsk 5 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Issues • Economy depends on Bees • Demand/Supply Imbalance • $170B, 84% global crop production • Food production increased thanks rely on pollinators to • Bees play a key role in the – Better agricultural practices sustainable food production – Seed improvement – Expansion of planted area • In the next decade the world will need 60% more food 6 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Issues Over the last five decades global bee populations have steadily declined leading to significant global nutrition, health and economic warning signs 7 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Bees with Backpacks 8 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Bees with Backpacks https://www.youtube.com/watch?v=CfkNFo-Hysc 9 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Experiment Setup Weather station Solar panel Electronics box Hive scales Tag reader at the hives and feeders 10 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Experiments in Tasmania 11 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Experiments in Brazilian Amazon 12 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Locations 13 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Expanded Perception and Interaction Centre From Data to Insights Visual Analytics and Immersive Analytics

Facilitate comprehension and decision making 15 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Visual Analytics Facilitate comprehension and decision making 16 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Bee Activities 17 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Individual Bee Activities 18 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Bee Drifting Number of bees recorded H1 H2 H3 H4 H1 H2 H3 H4 Hive 19 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Immersive Analytics 20 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

AR/VR Head-mounted Displays 21 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

HoloBee (AR) • Bee drifting: the behaviour of bees which leave one colony and join another. • Visualise bee drift to help understand bee drifting behaviour over 6-month period in Geeveston using AR interface on HoloLens. • Provide an easy to use and intuitive interface for data exploration. 22 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

HoloBee (AR) DATA COLLECTION, POST-PROCESSING & ANALYSIS Cleaning, classification, and aggregation operations on bee data COLLABORATION HoloBee Collaborative sessions in a shared environment INFORMATION DISPLAY Overview+Detail paradigm COMMAND LANGUAGES & SYSTEM FEEDBACK Voice, gaze, gesture recognition Directional pointers, visual cursor, auditory channel 23 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

HoloBee (AR) – Interface Global View Site View Interface Screenshot 24 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

HoloBee (AR) – Collaboration 25 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

MelissAR [Melissa – ancient Greek for Honey Bee] 26 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

MelissAR – Interface Detailed Information Information Query Site Information 27 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Virtual Bee Keeper (VR) • Visualise spatial and temporal data of bee activities and health- related information of bee hives in a 3D world. • Brood temperature, number of workers and drones, hive weight, site state, hive drift, etc. • Map the virtual world with the real settings of bee hives to access, query and analyse bee data to study bee behaviours. 28 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Virtual Bee Keeper (VR) – Interface Detailed Information Information Query Video Stream 29 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

I-Flight • Virtual environment for visualising simulated data of bee flight trajectories depending on terrain, temperature, time of day, etc. • Interaction techniques to interact with bees, beehives. • Interface for analytics queries and graphical representations of results integrated into the virtual world 30 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

I-Flight – Desktop Interface 31 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Bee Roles Novice • Undertaking orientation flights Water carrier: • Exploiting a water source Scout: • Searching spontaneously for new food sources Expert: • Exploiting a current food source Recruit: • Searching for a food source by observing the waggle dance of an exploiter Inspector: • Reconnaissance flights to previously exhausted food sources to see if it is replenished 32 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Bee Activities Location dependent activity models: • Exploratory searching • Seeking a food source • Searching for a food source by visual or olfactory sensing • Foraging at a food source • Returning to the hive • Resting in the hive 33 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

I-Flight – VR Interface Trajectories mapped on realistic terrain Information Query Environmental Factor Vis. 34 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Reality – Virtuality Continuum Augmented Real Augmented Virtual Reality (AR) Environment Virtuality (AV) Environment 35 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Collaborative Design for Immersive Analytics • Facilitate collaboration amongst GIHH partners 36 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Collaborative Design for Immersive Analytics 37 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Human Factors in Mixed Reality • Multidisciplinary study of cognition, interaction design and immersive analytics. 38 | Immersive Analytics of Honey Bee Data | Huyen Nguyen

Expanded Perception and Interaction Centre Thank you • Division/Unit Name • Presenter Name Presenter Title – t +61 2 9123 4567 – e firstname.surname@csiro.au – w www.csiro.au/lorem • Division/Unit Name huyen.nguyen@unsw.edu.au • Presenter Name Presenter Title – t +61 2 9123 4567 – e firstname.surname@csiro.au – w www.csiro.au/lorem http://epicentre.matters.today

References U. Engelke, D. Darcy, G. Mulliken, S. Bosse, M. Martini, S. Arndt, J.-N. Antons, K. Y. Chan, N. Ramzan, and K. Brunnstroem , “Psychophysiology -based QoE Assessment: A Survey,” in IEEE Journal on Selected Topics in Signal Processing, vol. 11, no. 1, pp. 6-21, 2017. U. Engelke, P. Marendy, F. Susanto, R. Williams, S. Mahbub, H. Nguyen, and P. de Souza, “A Visual Analytics Framework to Study Honey Bee Behaviour,” in IEEE International Conference on Data Science Systems (DSS), Sydney, Dec 2016. E. Kandogan and U. Engelke, “Agile Visual Analytics in Data Science Systems,” in IEEE International Conference on Data Science and Systems (DSS), Sydney, Dec 2016. H. Nguyen, P. Marendy, and U. Engelke, “Collaborative Framework Design for Immersive Analytics,” in International Symposium on Big Data Visual Analytics (BDVA) , Sydney, Nov 2016. U. Engelke and E. Kandogan, “Towards an Algebra for the Visual Analytics Design Process,” in IEEE Visual Analytics Science and Technology (VAST), Poster Track , Baltimore, Oct 2016. U. Engelke, H. Hutson, H. Nguyen, and P. de Souza, “ MelissAR : Towards Augmented Visual Analytics of Honey Bee Behaviour,” in Proceedings of ACM CHI , San Jose, May 2016. F. Susanto, S. Budi, P. de Souza, U. Engelke, and J. He, “Design of Environmental Sensor Networks Using Evolutionary Algorith ms, “ IEEE Geoscience and Remote Sensing Letters , vol. 13, no. 4, pp. 575-579, 2016. U. Engelke, J. Vuong, and J. Heinrich, “Visual Performance in Multidimensional Data Characterisation with Scatterplots and Pa ral lel Coordinates,” in IS&T Human Vision and Electronic Imaging (HVEI) , San Francisco, Feb 2016. U. Engelke, F. Susanto, P. de Souza, and P. Marendy, “Visual Assessment of Spatial Data Interpolation,” in International Symposium on Big Data Visual Analytics (BDVA) , Hobart, Sep 2015. 40 | Immersive Analytics of Honey Bee Data | Huyen Nguyen