High performance mantle convection modeling Jens Weism uller, Bj - - PowerPoint PPT Presentation

Introduction Equations HPC methods Asthenosphere Conclusions

High performance mantle convection modeling

Jens Weism¨ uller, Bj¨

• rn Gmeiner, Siavash Ghelichkhan,

Markus Huber, Lorenz John, Barbara Wohlmuth, Ulrich R¨ ude and Hans-Peter Bunge

Geophysics Department of Earth- and Environmental Sciences Ludwig-Maximilians-Universit¨ at M¨ unchen

SPPEXA annual plenary meeting, 25.01.2016

TERRA

Weism¨ uller et al. High performance mantle convection modeling 1/26

Introduction Equations HPC methods Asthenosphere Conclusions

The Earth’s mantle

• Solid layer between crust

and core

• Makes up about 84% of

the Earth’s volume

• Solid, but ductile
• Creeping, viscous flow
• Convects on time scales
• f cm/a

Image: mail.colonial.net Weism¨ uller et al. High performance mantle convection modeling 2/26

Introduction Equations HPC methods Asthenosphere Conclusions

Mantle Convection

• Regulates the long term

thermal evolution of the Earth

• Controls heat loss of the

core

• Driving force behind

plate tectonics

Image: mail.colonial.net Weism¨ uller et al. High performance mantle convection modeling 3/26

Introduction Equations HPC methods Asthenosphere Conclusions

Plate tectonics

• Earth’s magnetic field

alternates polarity

• Solidifying rock takes on

the current magnetic field of the Earth

• Residual magnetization

can be used to reconstruct tectonic plate motion history

Image: geology.sdsu.edu Weism¨ uller et al. High performance mantle convection modeling 4/26

Introduction Equations HPC methods Asthenosphere Conclusions

Seismic tomography

• Seismic tomography

images shear velocities in the mantle

• Using mineralogic

models, they can be converted to buoyancies

Ritsema et al, S40RTS Weism¨ uller et al. High performance mantle convection modeling 5/26

Introduction Equations HPC methods Asthenosphere Conclusions

Seismic tomography

• Seismic tomography

images shear velocities in the mantle

• Using mineralogic

models, they can be converted to buoyancies Together with plate velocities: Terminal and boundary conditions

Weism¨ uller et al. High performance mantle convection modeling 6/26

Introduction Equations HPC methods Asthenosphere Conclusions

Brief history of mantle convection modeling

• Global models

became available in the late 80s

• They were parallelized

in the 90s Glatzmeier, 1988 Bunge and Baumgardner, 1995 Baumgardner, 1985 Tackley et al, 1993

Weism¨ uller et al. High performance mantle convection modeling 7/26

Introduction Equations HPC methods Asthenosphere Conclusions

• Convective planform (Earth and
• ther planets)
• Link to seismology
• Geodetic observations
• Mineralogy
• Rheological effects
• Thermal evolution
• . . .

Tackley, 2012 Zhong et al, 2007 Schuberth et al, 2009

Weism¨ uller et al. High performance mantle convection modeling 8/26

Introduction Equations HPC methods Asthenosphere Conclusions

Towards extreme resolutions

• Most classic codes scale to

∼ 1, 000 processors

• Current supercomputers have

∼ 1, 000, 000 cores

• New class of problems:

◮ Fluid dynamic inverse models ◮ Complex non-linear problems Weism¨ uller et al. High performance mantle convection modeling 9/26

Introduction Equations HPC methods Asthenosphere Conclusions

Future applications: Adjoint

• Reconstruct past state
• f the mantle with

inverse models

• Start from initial guess
• Iteratively run forward

and backwards simulations

Ghelichkhan et al, submitted

Weism¨ uller et al. High performance mantle convection modeling 10/26

Introduction Equations HPC methods Asthenosphere Conclusions

Complex, non-linear rheologies

• Classically, flow laws are

linear: η = f (x, T)

• Non-linear behaviour

increasingly in focus: η = f (x, T, v)

• Conventional

stress-strain-relations might not hold: η =? Deformation mechanisms of MgO:

Cordier et al, 2012

⇒ Will need to deal with computationally challenging rheologies

Weism¨ uller et al. High performance mantle convection modeling 11/26

Introduction Equations HPC methods Asthenosphere Conclusions

Governing equations

η∆u − ∇p + RaT = 0 ∇ · u = 0

• Stokes equations

∂tT = −u · ∇T − κ ∆T } Heat equation

• Simplified Navier-Stokes equations
• Slow deformation by creep

Weism¨ uller et al. High performance mantle convection modeling 12/26

Introduction Equations HPC methods Asthenosphere Conclusions

Governing equations

η∆u − ∇p + RaT = 0 ∇ · u = 0

• Stokes equations

∂tT = −u · ∇T − κ ∆T } Heat equation

• Simplified Navier-Stokes equations
• Slow deformation by creep
• Force balance between friction and buoyancy

Weism¨ uller et al. High performance mantle convection modeling 12/26

Introduction Equations HPC methods Asthenosphere Conclusions

Governing equations

η∆u − ∇p + RaT = 0 ∇ · u = 0

• Stokes equations

∂tT = −u · ∇T − κ ∆T } Heat equation

• Simplified Navier-Stokes equations
• Slow deformation by creep
• Force balance between friction and buoyancy
• Dominated by viscous term η∆u ⇒ Global models

Weism¨ uller et al. High performance mantle convection modeling 12/26

Introduction Equations HPC methods Asthenosphere Conclusions

Governing equations

η∆u − ∇p + RaT = 0 ∇ · u = 0

• Stokes equations

∂tT = −u · ∇T − κ ∆T } Heat equation

• Simplified Navier-Stokes equations
• Slow deformation by creep
• Force balance between friction and buoyancy
• Dominated by viscous term η∆u ⇒ Global models
• Time derivative only in heat equation

Weism¨ uller et al. High performance mantle convection modeling 12/26

Introduction Equations HPC methods Asthenosphere Conclusions

Governing equations

η∆u − ∇p + RaT = 0 ∇ · u = 0

• Stokes equations

∂tT = −u · ∇T − κ ∆T } Heat equation Start with initial T Solve Stokes equations for u and p Integrate heat equation for T forward in time

Weism¨ uller et al. High performance mantle convection modeling 13/26

Introduction Equations HPC methods Asthenosphere Conclusions

Governing equations

η∆u − ∇p + RaT = 0 ∇ · u = 0

• Stokes equations

∂tT = −u · ∇T − κ ∆T } Heat equation Start with initial T Solve Stokes equations for u and p Integrate heat equation for T forward in time

Weism¨ uller et al. High performance mantle convection modeling 13/26

Introduction Equations HPC methods Asthenosphere Conclusions

Solving the Stokes equations

η∆u − ∇p + RaT = 0 ∇ · u = 0

• Stokes equations

∂tT = −u · ∇T − κ ∆T } Heat equation

• So called saddle point problem:

A BT B u p

• =

f

• Many possible solution schemes
• Expensive part is solution of Au = f

Weism¨ uller et al. High performance mantle convection modeling 14/26

Introduction Equations HPC methods Asthenosphere Conclusions

Adaptive mesh refinement

• Flow structures evolve down to the

kilometer-scale

• Refine the mesh where things happen
• Administrational overhead

Burstedde et al, 2013 Kronbichler et al, 2012

Weism¨ uller et al. High performance mantle convection modeling 15/26

Introduction Equations HPC methods Asthenosphere Conclusions

Extreme resolutions

Alternative Concept:

• Equal resolution throughout the

mantle + No administrative overhead – Computational overhead in areas that need no high resolution

Weism¨ uller et al. High performance mantle convection modeling 16/26

Introduction Equations HPC methods Asthenosphere Conclusions

Hierarchical Hybrid Grids

• Discretizing a planet not possible

with fully structured grid

• More efficient solvers on structured

grids Block-structured grid:

• Unstructured input mesh as

multigrid’s coarse grid

• Regularly refined

Weism¨ uller et al. High performance mantle convection modeling 17/26

Introduction Equations HPC methods Asthenosphere Conclusions

Hierarchical Hybrid Grids

On input mesh: Geometric hierarchy of ...

• Volumes (tetrahedra)
• Faces (triangles)
• Edges
• Nodes

Communication reduction by

• Independent solution within

geometric elements

• Communication of ghost

layers

Weism¨ uller et al. High performance mantle convection modeling 18/26

Introduction Equations HPC methods Asthenosphere Conclusions

Hierarchical Hybrid Grids

Matrix free stencil code

• Fully stencil-based for

constant viscosity (30 FLOPS per grid point and operator evaluation)

• Efficient on-the-fly stencil

assembly for more complex cases

Weism¨ uller et al. High performance mantle convection modeling 19/26

Introduction Equations HPC methods Asthenosphere Conclusions

The asthenosphere

• Mechanically weak uppermost layer of the mantle
• Traditionally: plausible depth extent of 660 km
• Low viscosity: 1018 − 1020 Pa s

(compared to ≈ 1022 Pa s in the lower mantle)

• Possible causes: Partial melting, water, mineralogy (p/T)

Weism¨ uller et al. High performance mantle convection modeling 20/26

Introduction Equations HPC methods Asthenosphere Conclusions

Thickness of the asthenosphere from seismology

• Full waveform seismic tomography studies have imaged a thin

horizontal layer of 100 − 200 km depth extent

Fichtner, 2009

Weism¨ uller et al. High performance mantle convection modeling 21/26

Introduction Equations HPC methods Asthenosphere Conclusions

Thickness of the asthenosphere from seismology

• Full waveform seismic tomography studies have imaged a thin

horizontal layer of 100 − 200 km depth extent

Fichtner, 2009

Maybe the asthenosphere is only 100 − 200 km thick?

Weism¨ uller et al. High performance mantle convection modeling 21/26

Introduction Equations HPC methods Asthenosphere Conclusions

Fast flow velocities

• Buried landscape in seismic

reflection data on northwest european continental shelf

• Uplift history hints at passage of

sublithospheric material at 35 cm/a

Hartley et al, 2011

Weism¨ uller et al. High performance mantle convection modeling 22/26

Introduction Equations HPC methods Asthenosphere Conclusions

Fast flow velocities

• Buried landscape in seismic

reflection data on northwest european continental shelf

• Uplift history hints at passage of

sublithospheric material at 35 cm/a 35 cm/a: Dynamically plausible?

Weism¨ uller et al. High performance mantle convection modeling 22/26

Introduction Equations HPC methods Asthenosphere Conclusions

Linking channel thickness and flow velocities

Model setup:

500 1000 1500 2000 2500 1018 1019 1020 1021 1022 Depth [km] Viscosity [Pa s] Channel thickness 1000km 660km 410km 200km 100km

• 5 scenarios
• Viscosity profiles according to
• bservational constraints
• Computations on JUQUEEN with:

◮ 2.3 · 1010 degrees of freedom ◮ 32 numerical layers in the

asthenosphere

◮ 65 536 processes Weism¨ uller et al. High performance mantle convection modeling 23/26

Introduction Equations HPC methods Asthenosphere Conclusions

Linking channel thickness and flow velocities

• Synthetic simulations (left): with stagnant lid (no-slip) and

mobile lid (free slip)

• Tomography/plate-driven simulations (right)

Weism¨ uller et al. High performance mantle convection modeling 24/26

Introduction Equations HPC methods Asthenosphere Conclusions

Linking channel thickness and flow velocities

• thick channel

⇒ low velocity

• thin channel

⇒ high velocity

• Velocities correspond to

preditions by a simple Poiseuille flow model

2.5 5 7.5 10 12.5 15 100 200 410 660 1000 10,000 1250 145 34.8 10 Asthenosphere velocity [-] (normalized to 1000 km scenario) Channel thickness [km] Viscosity ratio (lower/upper mantle) [-] Poiseuille Observation driven Synthetic, stagnant lid Synthetic, mobile lid

Weism¨ uller et al. High performance mantle convection modeling 25/26

Introduction Equations HPC methods Asthenosphere Conclusions

Conclusions

• Modern computers become faster, but more heterogeneous

⇒ new software needed

• Hierarchical Hybrid Grids (HHG) are a promising approach for

mantle convection

• Using HHG, we can explain recent geological observations (fast

flow, thin asthenosphere) dynamically

Weism¨ uller et al. High performance mantle convection modeling 26/26