Augmented Reality made More Real Amitabh Varshney Institute for - - PowerPoint PPT Presentation

Augmented Reality made More Real

Amitabh Varshney Institute for Advanced Computer Studies (UMIACS) and Computer Science

Augmented Reality

• An Interface to facilitate our awareness of the

Digital (Virtual) World coexisting with the Real World in a natural and intuitive manner

• All senses, but most advances thus far have

been visual.

• Location-based Spatio-Temporal reality

Head-Mounted Displays

Ron Azuma 1995

Augmented Reality in the Battlefield

Augmented Reality for Surgery

Augmented Reality in Daily Life

Augmented Reality in Daily Life

Visual Challenges in AR

• Thin, Lightweight, High resolution, wide field of

view displays

• Registration of real and virtual worlds
• Rendering of virtual worlds mixed with real

worlds

• Perception management for AR

GPUs will play a central role in all of the above!

AR Prototype (in 3 months)

• We came across this Oculus

HD Prototype (don't ask how...)

• 840 x 975 pixels for each

eye

• Approx. 110 degree

diagonal field of view (comparatively wide)

• Device orientation tracking

– accelerometer- gyroscope combo

• Prototype 0 – 20fps HD

webcams: horrible!

Customizing the AR Bracket

• We made a CAD

model of the front shell

• We modeled a

bracket around it

• We 3D printed it

Assembling the AR Prototype

• We got a couple of Point Grey Flea3 Cameras
• 3.2 MP @ 60 fps via USB 3
• Low-latency camera design (thanks to Point Grey)
• Added some wide-angle lenses (80-185° FOV)
• Wrote some code for it...

Greg Kramida

GPUs for De-Warping

• Lens distortion: really

bad in our case

• Have to calibrate and

"undistort": – compute mapping

• nce (calibrate)

– apply mapping to each successive frame (remap)

• Remap operation is a

simple kernel

• OpenCV 3.0 alpha

already provides this kernel for both CUDA and OpenCL

Top graphic adapted from "Computer Vision Acceleration Using GPUs" by Shehrzad Qureshi

Disparity Computation

Top graphic adapted from "Computer Vision Acceleration Using GPUs" by Shehrzad Qureshi

• Depth from stereo
• Compute how far

each point in right image is from the same point in left image

• OpenCV 3.0 alpha

routines for CUDA and OpenCL:

– Block matching – Belief propagation

Need for AR Registration

Dynamic Registration

AR Registration

Extraction of objects using contours

• 1. Find color range of the interesting objects

– HSV, RGB, LAB, LUB

• 2. Create mask based on the range

– Noise reduced with erosion, dilation, and blurring

• 3. Detect edges of the mask
• 4. Find contours

– Draw lines at the border of screen to find partial views

• 5. Draw contours

Original Image Range Mask Monitor Frame Extracted Filled Contours with partial view fix Edge Detection Final Output

Original Image Monitor Frame Extracted Range Mask Edge Detection Filled Contours Filled Contours Filled Contours Final Output Currently: 40-50 msec/frame with GPUs

Curing Phobias through Augmented Reality

Botella et al. 2010

Next Steps: Augmentarium

Last Night …

Last Night …

Closing Remarks

• Confluence of a number of technological drivers – robots, HMDs,

projectors, imaging sensors, and GPU processing, ..

• VR/AR tools and technologies need to co-develop with driving

applications in consumer applications, entertainment, science, medicine, and engineering.

• Critical to leverage human perceptual and cognitive processes in

development of VR/AR tools

• Very exciting times ahead!

Acknowledgements

Greg Kramida Sujal Bista Tim Gray Fritz McCall Derek Yarnell Mike van Opstal Eric Krokos Hsueh-Chien Cheng Patricia Sazama Barbara Brawn-Cinani Larry Rosenblum NSF, NVIDIA, Mpower Maryland, and UMD CMNS