Global Static Indexing for Real-time Exploration of Very Large Regular Grids Valerio Pascucci and Randall J. Frank pascucci,rjfrank@llnl.gov Lawrence Livermore National Laboratory CASC VP 1
Outline • Motivation • Previous work (bla bla[vis`01], bla bla bla [sc`00] ……) • Data layout • 2 n tree indexing i j k I • Performance for slicing large grids • Conclusions and future work CASC VP 2
We must achieve real-time interaction with large datasets on a wide variety of platforms. The problem • Extremely large datasets 0.5TB/timestep (8k,8k,8k,(time)). • Interactive rendering for real-time data exploration. • Target platforms: desktop, parallel server, cluster. CASC VP 3
Previous Work Out-of-core geometric algorithms • [Goodrich, Tsay, Vengroff, Vitter ‘93] [Vitter ‘00][Matias, Segal, Vitter ‘00] [Asano, Ranjan, Roos, Welzl ’95][Arge Miltersen ’99] Out-of-core visualization • [Chiang, Silva ‘97][Sutton, Hansen ‘99] [Livnat,Shen,Johnson ’96][El-Sana, Chiang’00] [Bajaj, Pascucci, Thompson, Zhang ‘99] Space filling curves • (image processing, multidimensional databse, geomtric datastructure …) [Bandou, Kamata.’99][Balmelli, Kovacevic, Vetterli ’99] [Parashar, Browne, Edwards, Klimkowski ’97] [Niedermeier, Reinhardt, sanders ‘97][wise’00] [Hans Sagan ’94] [Lawder king ’00][Griebel Zumbusch ‘99] CASC VP 4
We apply three fundamental techniques to the visualization of large simulation data. Our approach • Multi-resolution geometric representation: – adaptive view-dependent refinement; – minimal geometric output for selected error tolerance. • Cache oblivious external memory data layouts: – exploit spatial and resolution coherency; – no need for complicated paging techniques. • Progressive processing: – continuously improved rendering; – scalability with the resources without budgeting. CASC VP 5
We focus on the progressive computation of slices (any orientation) of large 3D rectilinear grids. • Rectilinear grid • 1D order = hierarchical • Sub-sampling octtree 3D Z-order curve • Coarse to fine slice refinement CASC VP 6
General Data Layout Data coherent Progressive refinement of a hierarchical geometric data-structure Grouping the data by level Grouping the data by of resolution geometric proximity CASC VP 7
General Data Layout CASC VP 8
General Data Layout CASC VP 9
General Data Layout CASC VP 10
General Data Layout CASC VP 11
General Data Layout CASC VP 12
General Data Layout CASC VP 13
General Data Layout nD to 1D mapping: I → I* C 0 C 1 C 2 C 3 I → l find the level of resolution l C l (pre)compute the number of elements in the levels coarser than l I → I’ index of the element within its level of resolution I* = C l + I’ CASC VP 14
We exploit the correlation of bin/quad/oct-trees with the Lebesgue space-filling curves. The Lebesgue curve is also known as Z-order, Morton, …. Curve. Special case of the general definition introduced by Guiseppe Peano in 1890. CASC VP 15
We turn the recursive definition of the Z-order curve into a hierarchical subsampling scheme. (a) (b) (c) (d) coarse data new level data CASC VP 16
We obtain a multi-resolution hierarchical representation which is not exactly a 2 n -tree. • Not exactly a quad-tree …… Level 0 Level 1 CASC VP 17
We obtain a multi-resolution hierarchical representation which is not exactly a 2 n -tree. • Not exactly a quad-tree …… Level 0 Level 1 Level 2 CASC VP 18
We obtain a multi-resolution hierarchical representation which is not exactly a 2 n -tree. • Not exactly a quad-tree …… Level 0 Level 1 Level 2 Level 3 CASC VP 19
The 1D index I* can be computed in a simple and efficient way in any dimension. i j I 0 1 2 Z-order 0 0 I Z-order 2 0 1 2 3 Z-order 4 0 4 8 12 1 2 3 5 6 7 9 10 11 13 14 15 I* 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 _I_ _I_ C l = 2 2(l-1) - I* = C l + I’ - 1 I’ = 2 2(l+1) 2 2l CASC VP 20
The 1D index I* can be computed in a simple and efficient way in any dimension. i j k I I* CASC VP 21
Overall the hierarchical Z-order yields a cache oblivious hierarchical data layout. Distribution in the grid of each constant size block of data 0 1 2 3 4 5 6 7 8 9 CASC VP 22
Theoretical analysis shows a gain of orders of magnitude independently of the block size. Cache oblivious !!! 000000000 1000000000 100000000 100000000 10000000 10000000 1000000 1000000 32^3-BLOC 32*32*32 100000 100000 3-BITS 3-bits shift 10000 10000 1-bit shift 1-bit shift 1000 1000 100 100 10 10 1 1 256 1024 4096 1 4 16 64 1 2 4 8 16 32 64 128 256 512 1024 2048 4096 8192 32K blocks 64K blocks CASC VP 23
Real speedup matches theoretical expectations: more than 10x improvement, platform scalable. 2048x2048x1920 dataset (we have run up to 8192x8192x7680) • 20MB memory cache • Translation and rotation tests (average over 3 primary axis) • 500Mhz PIII Laptop (512x512) 250Mhz SGI Onyx (1024x1024) 100.000 1000.000 100.000 10.000 ARR-Rot ARR-Rot 10.000 BIT-Rot BIT-Rot 1.000 Seconds Seconds BLK-Rot BLK-Rot 1.000 ARR-Trans ARR-Trans 0.100 BIT-Trans BIT-Trans 0.100 BLK-Trans BLK-Trans 0.010 0.010 0.001 0.001 1 10 100 1 10 100 Resolution Resolution CASC VP 24
Conclusions and Future Work • Results: an implicit scheme for coherent spatial, multi-resolution regular grid data access • Simple address remapping • Read/write access • No additional data overhead • Supports progressive access • Near term applications • Volume rendering • Time critical iso-contouring • Future work • “View dependent” parameterization • Unstructured/temporal hierarchies • Improved interpolation • Distributed implementation CASC VP 25
UCRL-VG-143542 This work was performed under the auspices of the U.S. Department of Energy by University of California Lawrence Livermore National Laboratory under contract No. W-7405-ENG-48. CASC VP 26
Recommend
More recommend
