SLIDE 1
Spatial Reconstruction Using Microsoft HoloLens
GUPTA Aman ZAFAR Waleed
2
AGENDA
▫ Introduction ▫ Application Design ▫ Results ▫ Looking Back ▫ Moving Forward ▫ Conclusion ▫ Feedback & Demo
HISS
Holo Indoor Spatial Scanner
Spatial Reconstruction Using Microsoft HoloLens GUPTA Aman ZAFAR - - PowerPoint PPT Presentation
Spatial Reconstruction Using Microsoft HoloLens GUPTA Aman ZAFAR - - PowerPoint PPT Presentation
Spatial Reconstruction Using Microsoft HoloLens GUPTA Aman ZAFAR Waleed AGENDA Introduction Application Design Results Looking Back Moving Forward Conclusion Feedback & Demo HISS Holo Indoor
SLIDE 2
SLIDE 3
3
Real-Estate Agents Interior Designers and Engineers VR Environment Developers
WHAT? WHY? WHO?
WHY
A Challenge An Opportunity Tackling a Real Problem
WHAT
To Capture To Reconstruct To Visualize
WHO
SLIDE 4
4
Existing Technologies
LIDAR Microsoft Kinect Fusion Google Tango
SLIDE 5
5
The HoloLens
▫ Spatial Understanding ▫ Mobility ▫ Accuracy ▫ Cost Effectiveness ▫ 3D Visualization
5
SLIDE 6
Software and Technologies
SLIDE 7
Application Design
HISS: Holo Indoor Spatial Scanner
SLIDE 8
The Deliverables
HoloLens
Record & Visualize
Processing Server
Process and Store Meshes
- n the Cloud
Web Portal
Interact with the Server
Mesh Processing Server Web Portal HoloLens Application
8
SLIDE 9
HoloLens Application
A modular approach.
Recording
Takes care of the recording and mesh generation features using Spatial Understanding Prefab
Visualization
Handles the modelling and manipulation of generated models.
Library
Populates list of available blobs from Azure Storage
Input/Output
Handles the serialization and deserialization of OBJ files from Mesh
9
SLIDE 10
10
The Recording Module
Uses the Spatial Understanding DLLs built by Microsoft Gesture and Speech Input Helpful Mesh Insights Minimum Criteria for Mesh Quality
10
SLIDE 11
Visualization View
The Modelling Module
11
SLIDE 12
The Processing Server
RANSAC Algorithm Poisson Surface Reconstruction Web Portal Advancing Front Surface Reconstruction
CGAL Library Azure Storage and Functions
System Design
SLIDE 13
13 13
PRPU Design
▫
Core C++ processing library
▫
Managed Interface
▫
Azure Function App
SLIDE 14
14 14
RANSAC Algorithm
▫
Point set shape detection
▫
Plane detection using point and normal set
▫
Outer hull reconstruction using point set with oriented normal
SLIDE 15
Advancing Front Surface Reconstruction ▪
RANSAC for Plane Identification
▪
Priority Structure Functor
▪
Advancing Front Reconstruction
SLIDE 16
16 16
Poisson Surface Reconstruction
▫
Operates on 3D point sets with
- riented normal
▫
Computes an implicit Poisson function and extracts an isosurface
▫
Doesn’t handle sharp features, noise
- r outliers well
▫
Use for interior element reconstruction
SLIDE 17
Processing System Flow
Parse arguments
File Read
RANSAC Point Set Shape Detection Outer Hull Construction
Shape Detection
Advancing Front Poisson
Surface Reconstruction
Push to Azure
File Write
SLIDE 18
Web Portal
18
SLIDE 19
Spatial Reconstruction
Record Upload Process
Download
Visualize
19
SLIDE 20
Results
SLIDE 21
Dirk’s Office
Original Scan Processed Model
SLIDE 22
Results Originals
SLIDE 23
Results
HLA
Mesh Recording Visualization & Modelling IO Module
Web Portal
Server Connection Front End Response
Processing Server
Processing Algorithms Server Connection Throughput
SLIDE 24
24 24
Moving Forward
▫ Integration of Classification Algorithms (Machine Learning) ▫ Incorporate Textures and Coloring of surfaces ▫ Manual capture of model features
Looking Back
▫ Enhance Feasibility Study ▫ Better Integration ▫ Better User Interface
SLIDE 25
CONCLUSION
Spatial Reconstruction is possible! The proof of concept works! We implemented various modules and interactions with multiple systems! The system has amazing applications! We need to refine the system! It must be implemented fully to become user acceptable!
SLIDE 26
Thank You!
Demo & Questions
26
SLIDE 27
27
SLIDE 28
28
SLIDE 29
29
SLIDE 30