SLIDE 1
Intro – Mobile Augmented Reality
- Enhance the perception of the real world
Nearest Tube App AR Toolkit
Mobile Augmented Reality Tssio Knop de Castro Visgraf Lab, IMPA, - - PowerPoint PPT Presentation
Mobile Augmented Reality Tssio Knop de Castro Visgraf Lab, IMPA, - - PowerPoint PPT Presentation
Mobile Augmented Reality Tssio Knop de Castro Visgraf Lab, IMPA, Rio de Janeiro 2011 Intro Mobile Augmented Reality AR Toolkit Nearest Tube App Enhance the perception of the real world Motivation Trip Advisor How to provide
SLIDE 2
SLIDE 3
Motivation
- How to provide useful information without confusing the
user?
- Is it possible to really insert a virtual object onto a real
scene in a realistic way?
Trip Advisor
SLIDE 4
Goal
To explore the possibilities and limitations of Mobile Augmented Reality concerning information insertion and
- bjects rendering
SLIDE 5
Mobile Development
SLIDE 6
The device: Nokia N900
- 600 MHz ARM
processor
- 256 MB RAM
- 5MP programmable
camera
SLIDE 7
Maemo Development
- MADDE (Maemo Application Development and Debugging
Environment)
- Cross-compiling
- Multi-platform support (Linux (32-bit/64-bit),
Windows, Mac OS X)
- Comes with Nokia Qt SDK =)
- It's a technology preview =(
- Scratchbox
SLIDE 8
Maemo Development
- Main challenges:
- Small community, poor documentation
- How to compile and configure external libraries for
the N900 using this Qt-based environment? (however, Qt is a good tool to create interfaces)
SLIDE 9
AR Toolkit Plus
- Marker-based augmented reality:
- Given a physical marker, this toolkit is able to compute
the camera position and orientation in real time
- Hundreds of markers templates
- There are marker-less methods as well (more expensive)
- We just need a good transformation matrix.
SLIDE 10
AR Toolkit Plus
- Main challenges:
- It's an external library: really tricky on MADDE
- No longer updated since June 2006
SLIDE 11
Coding in OpengGL ES 2.0
- OpenGL ES 2.0
- Programmable Shaders (GLSL)
- No fixed function pipeline
SLIDE 12
OpenGL ES 2.0
SLIDE 13
Overview
Capture (GStreamer) Detect Markers, estimate
- transf. matrix
(ARToolkitPlus) Setup VBO Setup Texture
Objects
Compile and link shaders Render!
SLIDE 14
GStreamer
- Powerful pipeline-based multimedia framework
SLIDE 15
Experiments
- 2D Text
- Texture Overlay
- 3D Text
- Shadow Mapping
2D Text
SLIDE 16
Texture Overlay
SLIDE 17
3D text
- More immersive than the 2D approach
- Array of vertices
- Simple shader
Video: 3D text
SLIDE 18
Shadow Mapping
- Lance Williams, Casting curved shadows on curved surfaces.
–
SIGGRAPH Comput. Graph. 12, 3 (August 1978)
light camera
- bjects
SLIDE 19
Shadow Mapping
- Render objects from Light POV into a buffer
- This buffer shows which regions are illuminated
light camera
- bjects
SLIDE 20
Shadow Mapping
- Render from the Camera POV using the computed depth buffer
- The (x,y,z) coordinates which aren't visible from light POV are in shadow.
light camera
- bjects
SLIDE 21
Shadow Mapping
Video: Shadow Mapping
SLIDE 22
Main Challenges
- Tracking stability
- Limited memory and speed
- Interface-related: one hand is always
holding the device
- Small community
- OpenGL ES 2.0 and GLSL sintax
SLIDE 23
Future Work
- Improve shadow mapping and 2D text
- Experiment with arbitrary complex
meshes
- Acquire light position in real time
- Render soft shadows
- Marker-less AR
SLIDE 24
Special Thanks
- Luiz Velho
- Djalma Lucio
- Aldo Zang
- Leonardo Carvalho
SLIDE 25
References
- Tangible Interfaces using Handheld
Augmented Reality. P. Rojtberg and A. Olwal
- OpenGL ES specs:
http://www.khronos.org/opengles/spec/
- Understanding Virtual Reality. William R.
Sherman and Alan Craig
SLIDE 26
Thank you
- tknop@impa.br