Global Static Indexing for Real-time Exploration of Very Large - - PowerPoint PPT Presentation

▶
global static indexing for real time exploration of very
SMART_READER_LITE
LIVE PREVIEW

Global Static Indexing for Real-time Exploration of Very Large - - PowerPoint PPT Presentation

Dec 14, 2022 126 likes •394 views

Global Static Indexing for Real-time Exploration of Very Large Regular Grids Valerio Pascucci and Randall J. Frank pascucci,rjfrank@llnl.gov Lawrence Livermore National Laboratory CASC VP 1 Outline Motivation Previous work (bla

slide-1
SLIDE 1

CASC

VP 1

Global Static Indexing for Real-time Exploration of Very Large Regular Grids

Valerio Pascucci and Randall J. Frank

pascucci,rjfrank@llnl.gov

Lawrence Livermore National Laboratory

slide-2
SLIDE 2

CASC

VP 2

Outline

  • Motivation
  • Previous work
  • Data layout
  • 2n tree indexing
  • Performance for

slicing large grids

  • Conclusions and future work

k I i j

(bla bla[vis`01], bla bla bla [sc`00] ……)

slide-3
SLIDE 3

CASC

VP 3

We must achieve real-time interaction with large datasets on a wide variety of platforms. The problem

  • Extremely large datasets

0.5TB/timestep (8k,8k,8k,(time)).

  • Interactive rendering for

real-time data exploration.

  • Target platforms: desktop,

parallel server, cluster.

slide-4
SLIDE 4

CASC

VP 4

Previous Work

  • Out-of-core geometric algorithms

[Goodrich, Tsay, Vengroff, Vitter ‘93] [Vitter ‘00][Matias, Segal, Vitter ‘00] [Asano, Ranjan, Roos, Welzl ’95][Arge Miltersen ’99]

  • Out-of-core visualization

[Chiang, Silva ‘97][Sutton, Hansen ‘99] [Livnat,Shen,Johnson ’96][El-Sana, Chiang’00] [Bajaj, Pascucci, Thompson, Zhang ‘99]

  • Space filling curves

(image processing, multidimensional databse, geomtric datastructure …) [Bandou, Kamata.’99][Balmelli, Kovacevic, Vetterli ’99] [Parashar, Browne, Edwards, Klimkowski ’97] [Niedermeier, Reinhardt, sanders ‘97][wise’00] [Hans Sagan ’94] [Lawder king ’00][Griebel Zumbusch ‘99]

slide-5
SLIDE 5

CASC

VP 5

We apply three fundamental techniques to the visualization of large simulation data.

Our approach

  • Multi-resolution geometric representation:

– adaptive view-dependent refinement; – minimal geometric output for selected error tolerance.

  • Cache oblivious external memory data layouts:

– exploit spatial and resolution coherency; – no need for complicated paging techniques.

  • Progressive processing:

– continuously improved rendering; – scalability with the resources without budgeting.

slide-6
SLIDE 6

CASC

VP 6

We focus on the progressive computation of slices (any orientation) of large 3D rectilinear grids.

  • Rectilinear grid
  • Sub-sampling octtree
  • 1D order = hierarchical

3D Z-order curve

  • Coarse to fine slice refinement
slide-7
SLIDE 7

CASC

VP 7

General Data Layout

Grouping the data by level

  • f resolution

Data coherent Progressive refinement of a hierarchical geometric data-structure

Grouping the data by geometric proximity

slide-8
SLIDE 8

CASC

VP 8

General Data Layout

slide-9
SLIDE 9

CASC

VP 9

General Data Layout

slide-10
SLIDE 10

CASC

VP 10

General Data Layout

slide-11
SLIDE 11

CASC

VP 11

General Data Layout

slide-12
SLIDE 12

CASC

VP 12

General Data Layout

slide-13
SLIDE 13

CASC

VP 13

General Data Layout

slide-14
SLIDE 14

CASC

VP 14

General Data Layout

nD to 1D mapping:

I → I* I → l find the level of resolution l Cl (pre)compute the number of elements in the levels coarser than l I → I’ index of the element within its level of resolution I* = Cl + I’

C0 C1 C2 C3

slide-15
SLIDE 15

CASC

VP 15

We exploit the correlation of bin/quad/oct-trees with the Lebesgue space-filling curves.

The Lebesgue curve is also known as Z-order, Morton, …. Curve. Special case of the general definition introduced by Guiseppe Peano in 1890.

slide-16
SLIDE 16

CASC

VP 16

We turn the recursive definition of the Z-order curve into a hierarchical subsampling scheme.

(a) (b) (c) (d) coarse data new level data

slide-17
SLIDE 17

CASC

VP 17

We obtain a multi-resolution hierarchical representation which is not exactly a 2n-tree.

  • Not exactly a quad-tree ……

Level 0 Level 1

slide-18
SLIDE 18

CASC

VP 18

We obtain a multi-resolution hierarchical representation which is not exactly a 2n-tree.

  • Not exactly a quad-tree ……

Level 0 Level 1 Level 2

slide-19
SLIDE 19

CASC

VP 19

Level 3

We obtain a multi-resolution hierarchical representation which is not exactly a 2n-tree.

  • Not exactly a quad-tree ……

Level 0 Level 1 Level 2

slide-20
SLIDE 20

CASC

VP 20

The 1D index I* can be computed in a simple and efficient way in any dimension.

I*

I

15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 15 14 13 11 10 9 7 6 5 3 2 1 12 8 4 Z-order 4 3 2 1 Z-order 2 Z-order 0

2 1

Cl = 22(l-1) _I_ 22l I’ = I* = Cl + I’ _I_ 22(l+1)

  • 1

I

i j

slide-21
SLIDE 21

CASC

VP 21

The 1D index I* can be computed in a simple and efficient way in any dimension. I* I

k i j

slide-22
SLIDE 22

CASC

VP 22

Overall the hierarchical Z-order yields a cache

  • blivious hierarchical data layout.

Distribution in the grid of each constant size block of data 0 1 2 3 4 5 6 7 8 9

slide-23
SLIDE 23

CASC

VP 23

Theoretical analysis shows a gain of orders of magnitude independently of the block size.

1 10 100 1000 10000 100000 1000000 10000000 100000000 000000000 1 2 4 8 16 32 64 128 256 512 1024 2048 4096 8192 32*32*32 3-bits shift 1-bit shift

1 10 100 1000 10000 100000 1000000 10000000 100000000 1000000000 1 4 16 64 256 1024 4096 32^3-BLOC 3-BITS 1-bit shift

32K blocks 64K blocks Cache oblivious !!!

slide-24
SLIDE 24

CASC

VP 24

Real speedup matches theoretical expectations: more than 10x improvement, platform scalable.

  • 2048x2048x1920 dataset (we have run up to 8192x8192x7680)
  • 20MB memory cache
  • Translation and rotation tests (average over 3 primary axis)

250Mhz SGI Onyx (1024x1024)

0.001 0.010 0.100 1.000 10.000 100.000 1000.000 1 10 100 Resolution Seconds

ARR-Rot BIT-Rot BLK-Rot ARR-Trans BIT-Trans BLK-Trans

500Mhz PIII Laptop (512x512)

0.001 0.010 0.100 1.000 10.000 100.000 1 10 100 Resolution Seconds ARR-Rot BIT-Rot BLK-Rot ARR-Trans BIT-Trans BLK-Trans

slide-25
SLIDE 25

CASC

VP 25

Conclusions and Future Work

  • Results: an implicit scheme for coherent spatial, multi-resolution regular grid

data access

  • Simple address remapping
  • Read/write access
  • No additional data overhead
  • Supports progressive access
  • Near term applications
  • Volume rendering
  • Time critical iso-contouring
  • Future work
  • “View dependent” parameterization
  • Unstructured/temporal hierarchies
  • Improved interpolation
  • Distributed implementation
slide-26
SLIDE 26

CASC

VP 26

UCRL-VG-143542

This work was performed under the auspices of the U.S. Department of Energy by University of California Lawrence Livermore National Laboratory under contract No. W-7405-ENG-48.