SLIDE 1
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Multilevel optimization by space-filling curves in adaptive - - PowerPoint PPT Presentation
Multilevel optimization by space-filling curves in adaptive - - PowerPoint PPT Presentation
Multilevel optimization by space-filling curves in adaptive atmospheric modeling Jrn Behrens Technische Universitt Mnchen Center for Math. Sciences (M3) 85747 Garching, Germany behrens@ma.tum.de www-m3.ma.tum.de/m3/behrens
SLIDE 2
SLIDE 3
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Adaptive Algorithm
no yes
SLIDE 4
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Modular Adaptive Software
main Sub-grid processes System solver Dynamic kernel (conserv. SLM) Diagnostics visualization I/O, Data management Grid generator amatos
(http://www.amatos.info)
SLIDE 5
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Refinement Strategy
Rivara (1984), Bänsch (1991), Grids created with amatos
2D 3D
SLIDE 6
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Complex Geometries
Grid created by amatos (with F. Klaschka)
Polygonal domain Bitmapped domain
SLIDE 7
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Data Management and Numerics
SLIDE 8
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Levels
Grid level: domain decomposition parallelization DD solvers System level: matrix ordering sparse storage prevention of fill-in Cache level: data layout access optimization
SLIDE 9
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Grid Level: Parallelization
Partitioning problem Distribute cells in equally sized sets (partitions) Partitions shall be connected Partitions have to be re-calculated frequently Data movement has to be minimized Algorithm has to be parallel/low computational effort
SLIDE 10
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Data Management and Parallelization
SLIDE 11
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Grid Level: Parallelization
Space-filling curve for load balancing
1 4 2 3 5 6 7 8 9
Proc. 1 Proc. 2 Proc. 3 Proc. 4
Roberts et al. 1997, Griebel & Zumbusch, 1999
SLIDE 12
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Grid Level: Parallelization
- J. B., J. Zimmermann (2000), N. Rakowski (2003)
0000 1000 0000 0100 0000 1000 1100 1010 0110 1110 1100
1101 1010 1011 0110 0111 1100
Algorithm for triangles
- 1. One bit per refinement level
- 2. Set bits while refining
SLIDE 13
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Results: Tracer Advection
Artificial tracer in Arctic stratosphere Load balancing Edge-cut
SFC: J. B., J. Zimmermann (2000), Metis: G. Karypis, V. Kumar (1998)
SLIDE 14
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Cache Level: Data Management
Connectivity matrix with different orderings Nearest neighbor communication (vertex-wise)
SLIDE 15
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Cache Level: Data Management
Connectivity matrix with different orderings Cache misses Distance structure
SLIDE 16
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Cache Level: Data Management
Nearest neighbor communication (element-wise) Cache misses Distance structure
SLIDE 17
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
System Level: FEM support
Main data objects: nodes, edges, triangles FEM-Signature:
- Unknowns on nodes
- Unknowns on edges
- Unknowns on triangles
- Position in barycentric coordinates
(for edges and triangles)
SLIDE 18
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
System Level: Matrix Ordering
- J. B., N. Rakowski, S. Frickenhaus, et al. (2003)
Structure of matrix
tree-sorted quotient minimum degree reverse Cuthill-McKee reverse SFC
System with ~200.000 unknowns Utilizes preconditioned BiCGStab ILU pre-conditioning
Iterations Time
SLIDE 19
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Example 1: linear advection
Simulation of tracer transport Resolution of wind data: 50 x 50 km Situation in January 1990, 70 hPA layer (18.000 m)
- A. Rinke et al., 1997
SLIDE 20
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Example 1: linear advection
- J. B., K. Dethloff, W. Hiller, A. Rinke (2000)
Simulation of tracer transport
Resolution: 50 km uniform Resolution: 5 km local
SLIDE 21
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Example 1: linear advection
Simulation of tracer transport
Costs: Uniform vs. adaptive
SLIDE 22
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Example 2: shallow water equations
- M. Läuter (2003)
Flow over isolated mountain
Vorticity Geopotential Equations in vorticity-divergence form
SLIDE 23
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Example 3: Inverse Modeling
Problem: Given:
- wind
- tracer density
distribution Question: source
- f tracer?
SLIDE 24
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de
Conclusions
Triangular grid generation for simplicity and complex domains Adaptive grid refinement for accuracy and efficiency SFC for partitioning in parallel applications SFC ordering for efficient data access and matrix reordering Examples from tracer transport to dynamical core
SLIDE 25
Jörn Behrens TU München Scientific Computing behrens@ma.tum.de