SIGGRAPH 2013 Shaping the Future of Visual Computing NVIDIA IndeX - - PowerPoint PPT Presentation

▶
siggraph 2013
SMART_READER_LITE
LIVE PREVIEW

SIGGRAPH 2013 Shaping the Future of Visual Computing NVIDIA IndeX - - PowerPoint PPT Presentation

Sep 14, 2022 244 likes •570 views

SIGGRAPH 2013 Shaping the Future of Visual Computing NVIDIA IndeX Enabling Interactive and Scalable Visualization for Large Data Marc Nienhaus, NVIDIA IndeX Engineering Manager and Chief Architect NVIDIA IndeX Positioning NVIDIA

slide-1
SLIDE 1

SIGGRAPH 2013

Shaping the Future

  • f Visual Computing

NVIDIA IndeX™ – Enabling Interactive and Scalable Visualization for Large Data

Marc Nienhaus, NVIDIA IndeX Engineering Manager and Chief Architect

slide-2
SLIDE 2

NVIDIA IndeX™ Positioning

NVIDIA IndeX is a commercial software product initially developed to serve the Hydrocarbon market. It is a cluster-based scalable software Platform as a Service (PaaS) ready for the cloud and enables distribution of large-scale data for compute and high quality visualization of volumetric and surface data with interactive frame- rates.

http://www.nvidia-arc.com/products/nvidia-index.html

slide-3
SLIDE 3

NVIDIA IndeX – Enabling Interactive and Scalable Visualization for Large Data

NVIDIA IndeX software leverages GPU-clusters for scalable large-scale data visualization NVIDIA IndeX is a GPU-cluster aware solution for interactive visual computing NVIDIA IndeX is a commercial software solution available and already deployed by customers for data interpretation

http://www.nvidia-arc.com/products/nvidia-index.html

slide-4
SLIDE 4

Visualization of a seismic volume with embedded height field and slices

Special thanks to Crown Minerals and the New Zealand Ministry of Economic Development for allowing us to display this Taranaki Basin dataset. Crown Minerals manages the New Zealand Government’s oil, gas, mineral and coal resources. More information is available at: www.crownminerals.govt.nz

Example: Exploration in Hydrocarbon Industries

Scanning earth’s subsurface structure Cost-effective drilling for

  • il reservoirs

Acquisition and preprocessing

  • f subsurface data

Huge (peta bytes) subsurface dataset sizes Automatic data processing

slide-5
SLIDE 5

Visualization of a seismic volume with embedded height field and slices

Special thanks to Crown Minerals and the New Zealand Ministry of Economic Development for allowing us to display this Taranaki Basin dataset. Crown Minerals manages the New Zealand Government’s oil, gas, mineral and coal resources. More information is available at: www.crownminerals.govt.nz

Efficient Data Interpretation

Knowledge and experience

  • f experts in this field

Visually assess subsurface data Interactive exploration

Real-time frame rates Visual quality

Especially in Oil & Gas domain

slide-6
SLIDE 6

Visualization of a seismic volume with embedded height field and slices

Special thanks to Crown Minerals and the New Zealand Ministry of Economic Development for allowing us to display this Taranaki Basin dataset. Crown Minerals manages the New Zealand Government’s oil, gas, mineral and coal resources. More information is available at: www.crownminerals.govt.nz

NVIDIA IndeX – Scalable Interactive Large-Scale Data Visualization

Distributed Rendering

  • n GPU clusters

Supports today’s and tomorrow's huge dataset sizes

Real-Time Rendering

260 GB volume 14 cluster machines with 4 Tesla K10 13 frames per second

slide-7
SLIDE 7

Visual Quality and Accuracy

Visualization at original data resolution

Avoiding distractions e.g., popping artifacts due to level-of-details Highly accurate visual assessment

Depth-correct transparency rendering

Example: height field embedded into volume

slide-8
SLIDE 8

Visual Quality

slide-9
SLIDE 9

Sort-Last Approach for Distributed and Scalable Rendering

Object-space subdivision Later compositing of intermediate renderings

slide-10
SLIDE 10

LAN

Distributed Rendering using GPU-Clusters

Rendering Rendering Rendering

Viewer

Rendering Rendering

slide-11
SLIDE 11

Parallel and Distributed Rendering

Compositing Phase Rendering NodeN-1 Rendering Node0 Viewer Node

Compositing Compositing

Subsurface Data Rendering

Provide intermediate rendering results

Send composited image Send composited image

(..)

Hierarchical Scene Decomposition

(..) (..) (..) (..)

Distribute subsurface sub region data Distribute subsurface sub region data

Data Distribution

(..) (..) (..) (..) Render Sub Region

Horizons Seismic Volume

Render Sub Region

Horizons Seismic Volume

Render Sub Region

Horizons Seismic Volume

Render Sub Region

Horizons Seismic Volume Provide intermediate rendering results

Render Sub Region

Horizons Seismic Volume

Render Sub Region

Horizons Seismic Volume

Render Sub Region

Horizons Seismic Volume

Render Sub Region

Horizons Seismic Volume Provide intermediate rendering results Provide intermediate rendering results

slide-12
SLIDE 12

Performance and Scalability

Cluster details

2-16 cluster machines 4 Tesla K10 per cluster machine 8 GB per K10 1k x 1k screen 10 Gigabit Ethernet

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 40 GB 1.363 20.82 26.33 31.23 34.84 38.06 40.75 43.5 46.22 48.36 49.27 49.73 50.92 51.99 52.38 80 GB 0.959 2.321 24.56 30.89 33.65 35.52 37.42 39.99 47.41 49.25 49.71 49.73 50.34 51.27 52.12 160 GB 0.23 0.393 0.543 1.024 1.934 6.737 44.39 46.08 50.72 54.47 55.01 57.11 57.76 58.29 59.87 40 GB (CPU) 0.46 0.66 0.88 1.02 1.22 1.32 1.52 1.65 1.7 1.83 2.08 2.23 2.23 2.18 2.23

10 20 30 40 50 60 frames per second (fps)

slide-13
SLIDE 13

Another Cluster Setup

Cluster details

2-35 cluster machines 2 Tesla M2090 (Fermi) per cluster machine 1k x 1k screen 10 Gigabit Ethernet

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 5 GB 16 18 22 25 28 33 34 37 38 40 10 GB 17 20 21 25 28 31 34 37 40 41 42 44 47 50 51 20 GB 19 21 24 27 30 34 36 39 41 44 46 48 49 51 51 53 55 56 40 GB 23 26 30 33 37 41 42 44 47 47 51 53 53 55 56 58 59 61 62 62 65 64 66 67 68 68 69 80 GB 1 1 1 2 3 4 6 38 40 42 41 42 45 45 48 49 49 52 54 52 55 55 56 57 58 58 59 59 160 GB 2 3 3 4 53 53 53 55 55 57 58 59 63 64 65 65 65 65 65

10 20 30 40 50 60 70 frames per second (fps)

slide-14
SLIDE 14

Dataset Scalability

Target performance

≥10 fps @ 1024x1024

Volume dataset sizes Cluster details

2-35 cluster machines 2 Tesla M2090 (Fermi) per cluster machine 1k x 1k screen 10 Gigabit Ethernet

5 10 20 40 80 160

20 40 60 80 100 120 140 160

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Dataset size (GB) Cluster size (number of cluster machines)

≥10 fps

slide-15
SLIDE 15

Scalability at Various Screen Resolutions

Target performance

≥10 fps, ≥20 fps, ≥30 fps

40 GB volume dataset Screen resolutions

1024x1024 (Baseline)

(1,048,576 pixels)

1920x1080 (Full HD)

(2,073,600 pixels, 1.98 x Baseline)

2560x1440 (WQHD)

(3,686,400 pixels, 3.5 x Baseline)

3840x2160 (QFHD)

(8,294,400 pixels, 7.9 x Baseline)

1,048,576 2,073,600 3,686,400 8,294,400 1,048,576 2,073,600 3,686,400 1,048,576 2,073,600 3,686,400

1,048,576.00 2,048,576.00 3,048,576.00 4,048,576.00 5,048,576.00 6,048,576.00 7,048,576.00 8,048,576.00

3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Resolution (number of pixel)

Cluster size (number of cluster machines)

≥10 fps ≥20 fps ≥30 fps

slide-16
SLIDE 16

Workstation GPU Clusters NVIDIA GRID Visual Computing Appliance (VCA)

Interactively on Workstations, Clusters, and Clouds

Single or multiple GPU(s) Smaller dataset sizes Interactive rendering performance

(..)

Unlimited number of GPUs Huge dataset sizes Increasing rendering performance

(..)

slide-17
SLIDE 17

Interactive GPU-Cluster aware Visual Computing

Interactive attribute generation for instantaneous visualization Applications

Flow simulations Atmospheric dynamics visualization Combustion simulation Molecular dynamics simulations Seismic attribute generation for survey visualization

slide-18
SLIDE 18

Architectural Challenges for Interactive GPU-Cluster aware Visual Computing

Raw n-dimensional data is huge

Multiple times larger than generated attributes

Process raw data using user-defined algorithms

Plethora of possible types of attribute Manifold parallel compute algorithms Diversity of algorithm-specific subdivision schemes

Interactive attribute generation for instantaneous visualization

Scalability

slide-19
SLIDE 19

Mapping Attribute onto Scene Geometries

slide-20
SLIDE 20

Mapping Attribute onto Scene Geometries

slide-21
SLIDE 21

Proxy Shapes for Attribute Visualization

Proxy shapes

Slices Height fields Triangle meshes Volumes

Part of the scene description Canvas for attribute visualization

slide-22
SLIDE 22

Distributed Attribute Visualization Process

User-defined attribute computation

Compute jobs launched per portion Proxy shape intersection Algorithm-specific subdivision schemes

Attribute mapping

Rendering proxy shapes Analogy: procedural texturing

slide-23
SLIDE 23

Attribute Generation and Visualization Process

LAN

(..) (..) (..)

Remote Compute Remote Compute Remote Compute Viewer

Rendering Rendering Rendering Rendering

slide-24
SLIDE 24

GPU Cluster Setup for Scalable Visual Computing

Asynchronous compute maximizes performance

Rendering and compute process run in parallel Compute integration into rendering

Example: GPU Cluster Layout for Visual Computing

slide-25
SLIDE 25

Extensible Software Architecture

Interactive Cluster-aware Visual Computing (NVIDIA IndeX Core) Base layer for networking, job scheduling, distributed data storage (DiCE library)

C++ API

… Remote Access Multi User Video Streaming E&P Domain Other Application Layer(s)

Application

Other Application Domains

slide-26
SLIDE 26

NVIDIA IndeX’s Building Blocks for Managing Distributed Data

Data locality information

Spatial query tells which cluster machine stores which portions of data Depends on dataset type

Accessing large-scale data

Assemble from cluster machines Can be restricted to portions

Editing large-scale data

Direct editing/compute to the distributed data Can be restricted to portions Simple example: user-defined filter

slide-27
SLIDE 27

OpenGL Integration

  • 1. Rasterize opaque geometry

Capture depth and color buffers

  • 2. Volume ray casting

OpenGL depth buffer

  • 3. Alpha-based color blending

Result of NVIDIA IndeX’s large-scale data rendering with alpha OpenGL color buffer Pseudo code

// OpenGL rendering to color- and // depth-buffer on local machine (gl_bufferRGBA, gl_bufferz) ← renderGL(); // Distributed, scalable rendering // with depth-buffer input resultRGBA ← nvidia_index->render(bufferz); // Blending rendering results blend(gl_bufferRGBA, resultRGBA);

slide-28
SLIDE 28

Proof-of-Concept Example

OpenGL color buffer OpenGL depth buffer (inverted) Final combined rendering Opaque OpenGL geometry integrated in NVIDIA IndeX

slide-29
SLIDE 29

Depth-correct OpenGL Integration

Z-buffer compressions Multicasting z-buffer data Possible extensions

Transparent OpenGL InfiniBand and RDMA GPUdirect support

slide-30
SLIDE 30

Remote Visualization

Video streaming

H.264 video encoding Hardware-accelerated

  • n Kepler

Private and public clouds

Web-based applications Thin clients (tablets)

Multi-user support for world-wide collaboration

slide-31
SLIDE 31

Live Demo

GPU cluster located in Berlin, Germany

8 cluster nodes 2 Tesla M2090 each 82 GB volume Height field with 250 million triangles H.264-based video streaming

http://www.nvidia-arc.com/products/nvidia-index.html

NVIDIA IndeX Berlin Cluster Anaheim, SIGGRAPH 2012

slide-32
SLIDE 32

Thank you …

Marc Nienhaus

NVIDIA IndeX Engineering Manager and Chief Architect

Christopher Lux

  • Sr. Graphics Software Engineer, NVIDIA IndeX

Jörg Mensmann

  • Sr. Graphics Software Engineer, NVIDIA IndeX

Eduardo Olivares

  • Sr. Graphics Software Engineer, NVIDIA IndeX

Mahendra Gopaludu Roopa

Graphics Software Engineer

Gunter Sprenger

  • Sr. Graphics Software Engineer

Hitoshi Yamauchi

  • Sr. Graphics Software Engineer, NVIDIA IndeX

Runa Löber

Director Software Engineering

Tom-Michael Thamm

Director Software Product Management Product Manager NVIDIA IndeX

http://www.nvidia-arc.com/products/nvidia-index.html