MASSIVE TIME-LAPSE POINT CLOUD RENDERING IN VIRTUAL REALITY Markus - - PowerPoint PPT Presentation

Markus Schuetz, 2016.07.26

MASSIVE TIME-LAPSE POINT CLOUD RENDERING IN VIRTUAL REALITY

2

AGENDA

• Why?
• Performance and Rendering Techniques
• Rendering Quality
• Interaction in Virtual Reality

3

NVIDIAS NEW HEADQUARTER CURRENTLY UNDER CONSTRUCTION

4

DRONE SCANS

• ~Daily
• Point cloud created from drone images
• ~decimeter resolution
• 20 to 60 million points per time-slice
• 200 time-slices within first year
• Exterior only

5

LASER SCANS

• ~Monthly
• Terrestrial laser scanning
• ~millimeter/centimeter resolution
• ~800 million points per time-slice
• 10 time-slices within first year
• Interior & Ground Level Scans

6

VIRTUAL REALITY TRADITIONAL

VIEWER PERFORMANCE REQUIREMENTS

• 30-60 FPS
• ~2 Million Pixel
• Anti-Aliasing nice to have
• 90 FPS
• Render scene twice, once for

each eye

• >2 Million Pixel per Eye
• Anti-Aliasing must-have!

(especially for point clouds)

7

MEETING PERFORMANCE REQUIREMENTS

• Too much data. Out-Of-Core structures necessary
• Multi-Resolution Octree

Source: “Domitilla Catacomb Walkthrough – Dealing with more than 1 Billion Points”, Claus Scheiblauer

• Load and render only visible parts up to desired Level of Detail

source: “Potree: Rendering Large Point Clouds in Web Browsers”, Markus Schuetz

8

ADAPTIVE POINT SIZES

• Noticeable difference in point-density

and holes where LOD changes

• Adjust point size to level of detail
• Nodes with different level overlap
• > LOD != node level
• LOD = local leaf-node level
• Find local leaf-node level through
• ctree-traversal in vertex-shader

9

EYE-DOME-LIGHTING

• Most point clouds do not contain surface normals. Sometimes no colors, either.
• Colors may suffer from overexposure
• EDL does not require normals!
• Creates Illumination & Outlines
• Conceptually close to SSAO
• See:

“Interactive Scientific Visualisation of Large Datasets: Towards a Perception- based Approach”, Christian Boucheny

10

POINT-INTERPOLATION

• Points usually rendered as squares or circles
• Occlusions can reduce readability
• Render as paraboloids instead
• By altering depth in fragment shader
• Disables early-z, recover some speed with:

“layout(depth_greater) out float gl_FragDepth;”

• Results in nearest-neighbor-like interpolation

between points -> produces Voronoi Diagrams

“High-Quality Point-Based Rendering Using Fast Single-Pass Interpolation ”, Schütz M., Wimmer M.

11

QUALITY

• Strong aliasing inherent to Point Cloud

Rendering

• Surfaces made up of overlapping points that
• cclude each other. Closest to camera wins.
• Aliasing more noticeable in VR due to

constant motion and low resolution

• Perceived as “sparkling”

12

SILHOUETTES LEVEL OF DETAIL

SOURCES OF ALIASING

Object Silhouettes Point Sprite Silhouettes Building Multi-Resolution Octree, only considering point coordinates Like Nearest-Neighbor

OCCLUSIONS

Surface Patches made up

• f overlapping points

Points fighting for visibility

source: “Potree: Rendering Large Point Clouds in Web Browsers”, Markus Schuetz

13

POINT CLOUD MIP-MAPS

• Additionally store averaged colors in lower

Levels-Of-Detail

• Like Mip-Mapping for point clouds
• Averaged colors partially reduce occlusion-

aliasing

14

MSAA

• Multisample Anti-Aliasing
• Different sample sizes for quality vs. speed
• Reduces impact of noise
• Helps with inhomogeneous colors from merging

multiple scan locations

• Reduces “sparkling” during motion!
• Partially reduces occlusion-aliasing

15

ALIASING FROM OCCLUSIONS

• Largely solved through combination of adaptive point sizes, Mip-Maps and MSAA.
• Adaptive Sizes make points as big as necessary but not bigger
• Mip-Maps let otherwise unintentionally occluded points affect the result by

contributing to the average

• MSAA lets multiple points affect the same pixel

16

ANTI-ALIASED POINT CLOUDS

17

POINT CLOUDS IN VR

• Point clouds often not dense enough for real-world scale
• Can’t just do arbitrary locomotion.
• Tracked area restricted to a few meters
• Movements in VR that are counter to what the body feels and expects can easily

make users dizzy

Interaction Challenges

18

POINT CLOUDS IN VR

• User stuck in a small room but arbitrary

exploration possible through squeezing/stretching/rotating/dragging the model

• Drag & Drop using a single controller
• Pinch-To-Zoom like gesture to scale & rotate
• Predefined views to choose from

Interaction Challenges

19