MASSIVE TIME-LAPSE POINT CLOUD RENDERING IN VIRTUAL REALITY Markus - PowerPoint PPT Presentation

MASSIVE TIME-LAPSE POINT CLOUD RENDERING IN VIRTUAL REALITY Markus Schuetz, 2016.07.26

• Why? Performance and Rendering Techniques • AGENDA • Rendering Quality Interaction in Virtual Reality • 2

NVIDIAS NEW HEADQUARTER CURRENTLY UNDER CONSTRUCTION 3

DRONE SCANS ~Daily • • Point cloud created from drone images ~decimeter resolution • • 20 to 60 million points per time-slice 200 time-slices within first year • Exterior only • 4

LASER SCANS ~Monthly • • Terrestrial laser scanning ~millimeter/centimeter resolution • • ~800 million points per time-slice 10 time-slices within first year • Interior & Ground Level Scans • 5

VIEWER PERFORMANCE REQUIREMENTS TRADITIONAL VIRTUAL REALITY 30-60 FPS 90 FPS • • ~2 Million Pixel Render scene twice, once for • • each eye • Anti-Aliasing nice to have >2 Million Pixel per Eye • Anti-Aliasing must-have! • (especially for point clouds) 6

MEETING PERFORMANCE REQUIREMENTS Too much data. Out-Of-Core structures necessary • • Multi-Resolution Octree Source: “ Domitilla Catacomb Walkthrough – Dealing with more than 1 Billion Points”, Claus Scheiblauer Load and render only visible parts up to desired Level of Detail • source: “ Potree : Rendering Large Point Clouds in Web Browsers”, Markus Schuetz 7

ADAPTIVE POINT SIZES Noticeable difference in point-density • and holes where LOD changes Adjust point size to level of detail • • Nodes with different level overlap -> LOD != node level LOD = local leaf-node level • • Find local leaf-node level through octree-traversal in vertex-shader 8

EYE-DOME-LIGHTING Most point clouds do not contain surface normals. Sometimes no colors, either. • • Colors may suffer from overexposure EDL does not require normals! • • Creates Illumination & Outlines Conceptually close to SSAO • See: • “Interactive Scientific Visualisation of Large Datasets: Towards a Perception- based Approach”, Christian Boucheny 9

POINT-INTERPOLATION Points usually rendered as squares or circles • • Occlusions can reduce readability Render as paraboloids instead • • By altering depth in fragment shader Disables early-z, recover some speed with: • “layout( depth_greater) out float gl_FragDepth ;” Results in nearest-neighbor-like interpolation • between points -> produces Voronoi Diagrams “ High-Quality Point-Based Rendering Using Fast Single-Pass Interpolation ”, Schütz M., Wimmer M. 10

QUALITY Strong aliasing inherent to Point Cloud • Rendering Surfaces made up of overlapping points that • occlude each other. Closest to camera wins. Aliasing more noticeable in VR due to • constant motion and low resolution • Perceived as “sparkling” 11

SOURCES OF ALIASING LEVEL OF DETAIL SILHOUETTES OCCLUSIONS Building Multi-Resolution Object Silhouettes Surface Patches made up Octree, only considering of overlapping points point coordinates Point Sprite Silhouettes Points fighting for Like Nearest-Neighbor visibility source: “ Potree: Rendering Large Point Clouds in Web Browsers”, Markus Schuetz 12

POINT CLOUD MIP-MAPS Additionally store averaged colors in lower • Levels-Of-Detail Like Mip-Mapping for point clouds • • Averaged colors partially reduce occlusion- aliasing 13

MSAA Multisample Anti-Aliasing • • Different sample sizes for quality vs. speed Reduces impact of noise • • Helps with inhomogeneous colors from merging multiple scan locations Reduces “sparkling” during motion ! • • Partially reduces occlusion-aliasing 14

ALIASING FROM OCCLUSIONS Largely solved through combination of adaptive point sizes, Mip-Maps and MSAA. • • Adaptive Sizes make points as big as necessary but not bigger Mip-Maps let otherwise unintentionally occluded points affect the result by • contributing to the average MSAA lets multiple points affect the same pixel • 15

ANTI-ALIASED POINT CLOUDS 16

POINT CLOUDS IN VR Interaction Challenges Point clouds often not dense enough for real-world scale • • Can’t just do arbitrary locomotion. Tracked area restricted to a few meters • • Movements in VR that are counter to what the body feels and expects can easily make users dizzy 17

POINT CLOUDS IN VR Interaction Challenges User stuck in a small room but arbitrary • exploration possible through squeezing/stretching/rotating/dragging the model • Drag & Drop using a single controller Pinch-To-Zoom like gesture to scale & rotate • • Predefined views to choose from 18

19