Spatial Reconstruction Using Microsoft HoloLens GUPTA Aman ZAFAR - PowerPoint PPT Presentation

Spatial Reconstruction Using Microsoft HoloLens GUPTA Aman ZAFAR Waleed

AGENDA ▫ Introduction ▫ Application Design ▫ Results ▫ Looking Back ▫ Moving Forward ▫ Conclusion ▫ Feedback & Demo HISS Holo Indoor Spatial Scanner 2

WHAT WHY To Capture A Challenge WHAT? To Reconstruct An Opportunity WHY? To Visualize Tackling a Real Problem WHO? WHO Interior Designers and VR Environment Real-Estate Agents Engineers Developers 3

Existing Technologies LIDAR Microsoft Kinect Fusion Google Tango 4

The HoloLens ▫ Spatial Understanding ▫ Mobility ▫ Accuracy ▫ Cost Effectiveness ▫ 3D Visualization 5 5

Software and Technologies

Application Design HISS: Holo Indoor Spatial Scanner

The Deliverables HoloLens Record & Visualize Processing Server Process and Store Meshes on the Cloud Web Portal Interact with the Server HoloLens Mesh Web Portal Application Processing Server 8

HoloLens Application A modular approach. Recording Library Takes care of the recording Populates list of available and mesh generation blobs from Azure Storage features using Spatial Understanding Prefab Input/Output Visualization Handles the serialization and Handles the modelling and deserialization of OBJ files manipulation of generated from Mesh models. 9

The Recording Module Uses the Spatial Understanding DLLs built by Microsoft Gesture and Speech Input Helpful Mesh Insights Minimum Criteria for Mesh Quality 10 10

Visualization View The Modelling Module 11

The Processing Server System Design CGAL Library Azure Storage and Functions RANSAC Algorithm Advancing Front Surface Reconstruction Poisson Surface Reconstruction Web Portal

PRPU Design Core C++ processing library ▫ Managed Interface ▫ Azure Function App ▫ 13 13

RANSAC Algorithm Point set shape detection ▫ Plane detection using point and ▫ normal set Outer hull reconstruction using point ▫ set with oriented normal 14 14

Advancing Front Surface Reconstruction ▪ RANSAC for Plane Identification ▪ Priority Structure Functor ▪ Advancing Front Reconstruction

Poisson Surface Reconstruction Operates on 3D point sets with ▫ oriented normal Computes an implicit Poisson function ▫ and extracts an isosurface Doesn’t handle sharp features, noise ▫ or outliers well Use for interior element ▫ reconstruction 16 16

Processing System Flow Shape Surface File Read File Write Detection Reconstruction Push to Azure Parse arguments RANSAC Advancing Front Point Set Shape Poisson Detection Outer Hull Construction

Web Portal 18

Spatial Reconstruction Visualize Record Process Download Upload 19

Results

Dirk’s Office Original Scan Processed Model

Originals Results

Results HLA Web Portal Processing Server Mesh Recording Server Connection Processing Algorithms Visualization & Modelling Server Connection Front End IO Module Throughput Response

Looking Back Moving Forward ▫ Enhance Feasibility Study ▫ Integration of Classification Algorithms (Machine Learning) ▫ Better Integration ▫ Incorporate Textures and Coloring of ▫ Better User Interface surfaces ▫ Manual capture of model features 24 24

CONCLUSION Spatial Reconstruction is possible! The proof of concept works! We implemented various modules and interactions with multiple systems! The system has amazing applications! We need to refine the system! It must be implemented fully to become user acceptable!

Thank You! Demo & Questions 26

27

28

29

Dirk’s Office Advancing Front 30