[PPT] - VU Augmented Reality on Mobile Devices VU Augmented Reality on PowerPoint Presentation

SLIDE 1

VU Augmented Reality on Mobile Devices VU Augmented Reality on Mobile Devices

Introduction

What is AR

Introduction – What is AR
Interaction Techniques
Navigation Collaboration

Navigation, Collaboration

…

1

SLIDE 2

AR using the iPad as input device AR using the iPad as input device

iPad used as target
iPad used as target
iPad used as input device (2D gestures)

2

SLIDE 3

Assembly instructions using AR Assembly instructions using AR

Step by step Instructions
Step by step Instructions
Feedback from the real world object
Simple acknowledgements

Simple acknowledgements

3

SLIDE 4

Sixth Sense Projector Base AR Sixth Sense Projector Base AR

4

SLIDE 5

Navigation Navigation

Navigation consists of two parts:
Navigation consists of two parts:
Travel (motor component)

Travel (motor component)

Wayfinding (cognitive component)
Travel already directly controlled in AR !
Viewpoint controlled by user
Therefore mostly passive displays in AR user

interfaces

5

interfaces.

SLIDE 6

Navigation tasks Navigation tasks

Exploration

Exploration

travel which has no specific target
build knowledge of environment
Search

Search

naïve: travel to find a target whose position is not known
primed: travel to a target whose position is known
build layout knowledge; move to task location

y g ;

Maneuvering
travel to position viewpoint for task
short, precise movements

short, precise movements

AR can provide hints
direction of travel
important landmarks

6

important landmarks

SLIDE 7

7

SLIDE 8

Navigation Support Navigation Support

Direct Overlays
Direct Overlays
Information registered to Environment
Easy to interpret

S ll fi ld f i

Small field of view
No overview no knowledge build-up
Map integration
Provides overview

M i t l t ti t li

May require mental rotation to align
Occludes display

8

SLIDE 9

Integrating instructions into the view Integrating instructions into the view

9

1 [Kolbe’04] 2 [Chittaro’05] 3 [Hile’08] 4 [Walther-Franks’08]

SLIDE 10

At an App Store near you At an App Store near you

Wikit d D i

Wikitude Drive
ACrossAir Nearest Tube

10

SLIDE 11

Maps Maps

Map and spatial

p p knowledge

Rules for good map

d i design

Provide you are here

marker

[Hoellerer’01]

Provide grid
Choose either north-up or

forward-up map

Try mixing local and

global maps

Often as World-in-

11

Miniature (WIM)

SLIDE 12

Maps Maps

Choose either
Choose either
north-up (exo) or
forward-up (ego) map

12

SLIDE 13

Signpost – Indoor Navigation Signpost – Indoor Navigation

13

SLIDE 14

Case Study: Tinmith Case Study: Tinmith

Outdoor AR System for

y modeling the environment

Complex 3D interaction
Working planes
CSG

CSG

Intersections
Complex system control
many functions
object library

14

bject library
manipulation modes

[Wayne Piekarski, UniSA]

SLIDE 15

Image plane interaction Image plane interaction

Selection and

manipulation

Different gestures

15

SLIDE 16

Outdoor Modeling with Working planes

Extension to

Outdoor Modeling with Working planes

image plane techniques

Planes relative to
User

User

View direction
Reference Object
Polygons
Polygons

16

[Piekarski, ’04]

SLIDE 17

Complex system input Complex system input

Pinch gloves
Pinch gloves
Hierarchical menu
6D tracking with ARToolkit markers
2 points for image plane techniques

17

[Piekarski, ’03]

SLIDE 18

Collaboration Collaboration

Example systems
Example systems
Distributed Graphics

Distributed Graphics

18

SLIDE 19

Collaboration in the Future? Collaboration in the Future?

Remote Conferencing Remote Conferencing F f C f i

19

Face to face Conferencing

SLIDE 20

Today’s Technology Today s Technology

Video Conferencing
Video Conferencing
lack of spatial cues
limited participants

2D ll b ti

2D collaboration
Collaborative Virtual Reality
Collaborative Virtual Reality
separation from real world
reduced conversational cues

20

SLIDE 21

Problem of Seams

Functional seams

Problem of Seams

Functional seams
between different

functional workspaces

Loss of gaze information
Loss of gaze information
Degraded non-verbal cues
Cognitive seams
between different

work practices

More difficult to learn
Frustration

21

SLIDE 22

22

SLIDE 23

Collaborative Augmented Reality

Face to face interaction

Augmented Reality

Face to face interaction
Seamless Interaction
Natural Communication

Studierstube [Schmalstieg 96]

Natural Communication

Attributes:
Virtuality
Augmentation
Cooperation
Independence

Shared Space [Billinghurst 96, 99]

Individuality
Merges Task and

Communication Space

23

Communication Space

SeamlessDesign[Kiyokawa 99]

SLIDE 24

Geometry Education Geometry Education

Teach

geometric concepts

Roles:

teacher sees teacher sees solution, students d ’t don’t

24

Construct 3D [Kaufmann 00]

SLIDE 25

Augmented Classroom Augmented Classroom

Projection screen

j

Teacher or student

works with HMD

Class watches projection
Use of markers for

manipulating finished manipulating finished work without HMD

25

SLIDE 26

More collaborative AR More collaborative AR

MagicMeeting [Regenbrecht01] ARTHUR [Broll00] MagicMeeting [Regenbrecht01]

26

DWARF [Sandor03] MagicBook [Billinghurst]

SLIDE 27

EMMIE: Augmented conferencing EMMIE: Augmented conferencing

“Hybrid User Interface”
Conferencing assisted by multiple computing devices
HMD, projection screen, notebook, tablet/PDA
Place virtual objects in 3D space

j p

Shared “virtual ether” metaphor

27

[Butz 99]

SLIDE 28

Indoor/Outdoor Collaboration Indoor/Outdoor Collaboration

Columbia Univ.’s MARS [Höllerer99]: Campus information system

GUI GUI Outdoor AR Indoor AR

28

SLIDE 29

Studierstube Tour Guide Studierstube Tour Guide

Ad-hoc networking via WLAN

Ad hoc networking via WLAN

Navigation with another user
Guiding
Meeting
Following
Information browsing

[Reitmayr 03]

g

Setting the filters of other users
Following the selection of other users
Annotation
Annotation
Icons appear for all users
Filtering based on author possible

29

SLIDE 30

Remote collaboration: AR Videoconferencing Videoconferencing

HITLab „Wearable conference space“: Video billboards + spatial audio in AR [Billinghurst 99] Studierstube „Augmented desktop videoconferencing“: 3D AR objects in video [Barakonyi 04]

Remote collaboration via video

A id b i t ti id 3D i t ti

in AR [Billinghurst 99] AR objects in video [Barakonyi 04]

30

Avoid seams by integrating video + 3D interaction

SLIDE 31

Hand of God interaction Hand of God interaction

On line 3D reconstruction
On-line 3D reconstruction
Show in mobile user’s view

Show in mobile user s view

31

[Stafford 06]

SLIDE 32

Technology: Distributed AR? Technology: Distributed AR?

Collaborative AR requirements
Collaborative AR requirements
Multiple users, each with own computer
Hybrid user interfaces require multiple computer
Distribution of labor
E.g., tracker server, app. server, render clients
Need for a distributed system
Need for a distributed system
What kind of networking technology?

32

SLIDE 33

Networking for Collaborative AR Networking for Collaborative AR

Face to face AR similar to networked VR
Face-to-face AR similar to networked VR
Similar solutions applicable
Easier than typical networked VR

yp

Co-located better bandwidth situation than remote VR
Multiple HMDs easier synchronization than for tiled displays,

CAVEsTM etc.

But: hybrid user interfaces are more dynamic and

complex than regular VR

33

SLIDE 34

Approaches Approaches

AR needs the following types of sharing
AR needs the following types of sharing
Symmetric: Between users
Asymmetric: Functional units (e.g. tracker server, tracker client)
Important aspects of networking
Easy to use for programmers
Adaptability to dynamic situation changes

p y y g

34

SLIDE 35

Approach 1: Distributed Shared Scene Graph Distributed Shared Scene Graph

Goal: Distribution without programming

App A Host 1 Host 2 explicit App App Host 1 Host 2

Keep existing API intact

App. App. specific data App. Sync. App. App. implicit Scene Graph implicit Sync.

D l d t b

Dual database

(app, scene)

Optimizations

35

Distributed shared memory semantics
Transparent distribution E.g.: COTERIE, DIV, Tinmith, Avango

SLIDE 36

Approach 2: Component Middleware Component Middleware

Mobile AR and

hybrid UIs have lots of physical parts parts

Physical and

logical parts are g p modeled as communicating software software components

E.g. DWARF,

36

g , Tinmith

SLIDE 37

Example: DWARF Example: DWARF

CORBA-based

Middleware

Spontaneous

t ki f

[Bauer, 2003]

networking of components

Service matches

Service matches needs & abilities

37

SLIDE 38

Social AR

Users create content &

Social AR

Users create content &

model the world

“YouTube” of AR
Supported with automated methods
Situated social networks
AR 2 0

Oh, no, officer - it’s not graffiti. it’s an analog real-time augmented reality

AR 2.0
Same Place / Object

analog real-time augmented reality application.

Same Place / Object

Different Time

40

4 Junaio

40