Augmented Reality made More Real Amitabh Varshney Institute for Advanced Computer Studies (UMIACS) and Computer Science
Augmented Reality • An Interface to facilitate our awareness of the Digital (Virtual) World coexisting with the Real World in a natural and intuitive manner • All senses, but most advances thus far have been visual. • Location-based Spatio-Temporal reality
Head-Mounted Displays Ron Azuma 1995
Augmented Reality in the Battlefield
Augmented Reality for Surgery
Augmented Reality in Daily Life
Augmented Reality in Daily Life
Visual Challenges in AR • Thin, Lightweight, High resolution, wide field of view displays • Registration of real and virtual worlds • Rendering of virtual worlds mixed with real worlds • Perception management for AR GPUs will play a central role in all of the above!
AR Prototype (in 3 months) • We came across this Oculus HD Prototype (don't ask how...) • 840 x 975 pixels for each eye • Approx. 110 degree diagonal field of view (comparatively wide) • Device orientation tracking – accelerometer- gyroscope combo • Prototype 0 – 20fps HD webcams: horrible!
Customizing the AR Bracket • We made a CAD model of the front shell • We modeled a bracket around it • We 3D printed it
Assembling the AR Prototype • We got a couple of Point Grey Flea3 Cameras • 3.2 MP @ 60 fps via USB 3 • Low-latency camera design (thanks to Point Grey) • Added some wide-angle lenses (80-185 ° FOV) • Wrote some code for it...
Greg Kramida
GPUs for De-Warping • Lens distortion: really bad in our case • Have to calibrate and "undistort": – compute mapping once (calibrate) – apply mapping to each successive frame (remap) • Remap operation is a simple kernel • OpenCV 3.0 alpha already provides this kernel for both CUDA and OpenCL Top graphic adapted from "Computer Vision Acceleration Using GPUs" by Shehrzad Qureshi
Disparity Computation • Depth from stereo • Compute how far each point in right image is from the same point in left image • OpenCV 3.0 alpha routines for CUDA and OpenCL: – Block matching – Belief propagation Top graphic adapted from "Computer Vision Acceleration Using GPUs" by Shehrzad Qureshi
Need for AR Registration
Dynamic Registration
AR Registration Extraction of objects using contours 1. Find color range of the interesting objects – HSV, RGB, LAB, LUB 2. Create mask based on the range – Noise reduced with erosion, dilation, and blurring 3. Detect edges of the mask 4. Find contours – Draw lines at the border of screen to find partial views 5. Draw contours
Original Image Range Monitor Frame Mask Extracted Edge Filled Contours with Final Output Detection partial view fix
Original Image Range Mask Monitor Frame Edge Extracted Detection Filled Contours Filled Contours Filled Contours Final Output Currently: 40-50 msec/frame with GPUs
Curing Phobias through Augmented Reality Botella et al. 2010
Next Steps: Augmentarium
Last Night …
Last Night …
Closing Remarks • Confluence of a number of technological drivers – robots, HMDs, projectors, imaging sensors, and GPU processing, .. • VR/AR tools and technologies need to co-develop with driving applications in consumer applications, entertainment, science, medicine, and engineering. • Critical to leverage human perceptual and cognitive processes in development of VR/AR tools • Very exciting times ahead!
Acknowledgements Greg Kramida Sujal Bista Tim Gray Fritz McCall Derek Yarnell Mike van Opstal Eric Krokos Hsueh-Chien Cheng Patricia Sazama Barbara Brawn-Cinani Larry Rosenblum NSF, NVIDIA, Mpower Maryland, and UMD CMNS
Recommend
More recommend
