VU Augmented Reality on Mobile Devices VU Augmented Reality on - PowerPoint PPT Presentation

VU Augmented Reality on Mobile Devices VU Augmented Reality on Mobile Devices  Introduction  Introduction – What is AR What is AR  Interaction Techniques  Navigation Collaboration Navigation, Collaboration  … 1

AR using the iPad as input device AR using the iPad as input device  iPad used as target  iPad used as target  iPad used as input device (2D gestures) 2

Assembly instructions using AR Assembly instructions using AR  Step by step Instructions  Step by step Instructions  Feedback from the real world object  Simple acknowledgements Simple acknowledgements 3

Sixth Sense Projector Base AR Sixth Sense Projector Base AR 4

Navigation Navigation  Navigation consists of two parts:  Navigation consists of two parts:  Travel (motor component) Travel (motor component)  Wayfinding (cognitive component)  Travel already directly controlled in AR !  Viewpoint controlled by user  Therefore mostly passive displays in AR user interfaces interfaces. 5

Navigation tasks Navigation tasks  Exploration Exploration  travel which has no specific target  build knowledge of environment  Search Search  naïve: travel to find a target whose position is not known  primed: travel to a target whose position is known  build layout knowledge; move to task location y g ;  Maneuvering  travel to position viewpoint for task  short, precise movements short, precise movements  AR can provide hints  direction of travel  important landmarks important landmarks 6

7

Navigation Support Navigation Support  Direct Overlays  Direct Overlays  Information registered to Environment  Easy to interpret  Small field of view S ll fi ld f i  No overview no knowledge build-up  Map integration  Provides overview  May require mental rotation to align M i t l t ti t li  Occludes display 8

Integrating instructions into the view Integrating instructions into the view 1 [Kolbe’04] 2 [Chittaro’05] 3 [Hile’08] 4 [Walther-Franks’08] 9

At an App Store near you At an App Store near you  Wikitude Drive Wikit d D i  ACrossAir Nearest Tube 10

Maps Maps  Map and spatial p p knowledge  Rules for good map d design i [Hoellerer’01]  Provide you are here marker  Provide grid  Choose either north-up or forward-up map  Try mixing local and global maps  Often as World-in- Miniature (WIM) 11

Maps Maps  Choose either  Choose either  north-up (exo) or  forward-up (ego) map 12

Signpost – Indoor Navigation Signpost – Indoor Navigation 13

Case Study: Tinmith Case Study: Tinmith  Outdoor AR System for y modeling the environment  Complex 3D interaction  Working planes  CSG CSG  Intersections  Complex system control  many functions  object library object library  manipulation modes [Wayne Piekarski, UniSA] 14

Image plane interaction Image plane interaction  Selection and manipulation  Different gestures 15

Outdoor Modeling with Working planes Outdoor Modeling with Working planes  Extension to image plane techniques  Planes relative to  User User  View direction  Reference Object  Polygons  Polygons [Piekarski, ’04] 16

Complex system input Complex system input  Pinch gloves  Pinch gloves  Hierarchical menu  6D tracking with ARToolkit markers  2 points for image plane techniques [Piekarski, ’03] 17

Collaboration Collaboration  Example systems  Example systems  Distributed Graphics Distributed Graphics 18

Collaboration in the Future? Collaboration in the Future? Remote Conferencing Remote Conferencing F Face to face Conferencing f C f i 19

Today’s Technology Today s Technology  Video Conferencing  Video Conferencing  lack of spatial cues  limited participants  2D collaboration 2D ll b ti  Collaborative Virtual Reality  Collaborative Virtual Reality  separation from real world  reduced conversational cues 20

Problem of Seams Problem of Seams  Functional seams  Functional seams  between different functional workspaces  Loss of gaze information  Loss of gaze information  Degraded non-verbal cues  Cognitive seams  between different work practices  More difficult to learn  Frustration 21

22

Collaborative Augmented Reality Augmented Reality  Face to face interaction  Face to face interaction Studierstube [Schmalstieg 96]  Seamless Interaction  Natural Communication Natural Communication  Attributes:  Virtuality  Augmentation Shared Space [Billinghurst 96, 99]  Cooperation  Independence  Individuality  Merges Task and Communication Space Communication Space SeamlessDesign[Kiyokawa 99] 23

Geometry Education Geometry Education  Teach geometric concepts  Roles: teacher sees teacher sees solution, students d don’t ’t Construct 3D [Kaufmann 00] 24

Augmented Classroom Augmented Classroom  Projection screen j  Teacher or student works with HMD  Class watches projection  Use of markers for manipulating finished manipulating finished work without HMD 25

More collaborative AR More collaborative AR ARTHUR [Broll00] MagicMeeting [Regenbrecht01] MagicMeeting [Regenbrecht01] DWARF [Sandor03] MagicBook [Billinghurst] 26

EMMIE: Augmented conferencing EMMIE: Augmented conferencing  “Hybrid User Interface”  Conferencing assisted by multiple computing devices  HMD, projection screen, notebook, tablet/PDA  Place virtual objects in 3D space j p  Shared “virtual ether” metaphor [Butz 99] 27

Indoor/Outdoor Collaboration Indoor/Outdoor Collaboration Columbia Univ.’s MARS [Höllerer99]: Campus information system GUI GUI Indoor Outdoor AR AR 28

Studierstube Tour Guide Studierstube Tour Guide  Ad-hoc networking via WLAN Ad hoc networking via WLAN  Navigation with another user  Guiding  Meeting [Reitmayr 03]  Following  Information browsing g  Setting the filters of other users  Following the selection of other users  Annotation  Annotation  Icons appear for all users  Filtering based on author possible 29

Remote collaboration: AR Videoconferencing Videoconferencing HITLab „Wearable Studierstube „Augmented conference space“: desktop videoconferencing“: 3D Video billboards + spatial audio AR objects in video [Barakonyi 04] AR objects in video [Barakonyi 04] in AR [Billinghurst 99] in AR [Billinghurst 99]  Remote collaboration via video  Avoid seams by integrating video + 3D interaction A id b i t ti id 3D i t ti 30

Hand of God interaction Hand of God interaction  On line 3D reconstruction  On-line 3D reconstruction  Show in mobile user’s view Show in mobile user s view [Stafford 06] 31

Technology: Distributed AR? Technology: Distributed AR?  Collaborative AR requirements  Collaborative AR requirements  Multiple users, each with own computer  Hybrid user interfaces require multiple computer  Distribution of labor  E.g., tracker server, app. server, render clients  Need for a distributed system  Need for a distributed system  What kind of networking technology? 32

Networking for Collaborative AR Networking for Collaborative AR  Face to face AR similar to networked VR  Face-to-face AR similar to networked VR  Similar solutions applicable  Easier than typical networked VR yp  Co-located better bandwidth situation than remote VR  Multiple HMDs easier synchronization than for tiled displays, CAVEsTM etc.  But: hybrid user interfaces are more dynamic and complex than regular VR 33

Approaches Approaches  AR needs the following types of sharing  AR needs the following types of sharing  Symmetric: Between users  Asymmetric: Functional units (e.g. tracker server, tracker client)  Important aspects of networking  Easy to use for programmers  Adaptability to dynamic situation changes p y y g 34

Approach 1: Distributed Shared Scene Graph Distributed Shared Scene Graph Goal: Distribution without programming Keep existing API intact Host 1 Host 2 Host 1 Host 2 explicit App App. App. App A App. App. App Sync. App. specific data implicit implicit Sync. Scene Graph  Dual database D l d t b (app, scene)  Optimizations  Distributed shared memory semantics  Transparent distribution E.g.: COTERIE, DIV, Tinmith, Avango 35

Approach 2: Component Middleware Component Middleware  Mobile AR and hybrid UIs have lots of physical parts parts  Physical and logical parts are g p modeled as communicating software software components  E.g. DWARF, g , Tinmith 36

Example: DWARF Example: DWARF  CORBA-based [Bauer, 2003] Middleware  Spontaneous networking of t ki f components  Service matches Service matches needs & abilities 37 37