NCAR atmospheric component LANL ocean, land ice and sea ice - PowerPoint PPT Presentation

NCAR atmospheric component • LANL – ocean, land ice and sea ice MPAS-Atmosphere: • Nonhydrostatic global atmospheric model • Time integration as in Advanced Research WRF • Spatial discretization similar to ARW except for Voronoi mesh accommodations. NCAR: Bill Skamarock, Joe Klemp, Michael Duda, Laura Fowler, Sang-Hun Park

Motivation Mesh flexibility: North American refinement

Motivation Mesh flexibility: Refinement for equatorial convection

Motivation Mesh flexibility: Refinement around the Andes

Centroidal Voronoi Meshes Unstructured spherical centroidal Voronoi meshes • Mostly hexagons , some pentagons and 7-sided cells • Cell centers are at cell center-of-mass (centroidal). • Cell edges bisect and are orthogonal to the lines connecting cell centers. • Uniform resolution – traditional icosahedral mesh. C-grid • Solve for normal velocities on cell edges. • Gradient operators in the horizontal momentum equations are 2 nd -order accurate. • Velocity divergence is 2 nd -order accurate for edge-centered velocities.

Centroidal Voronoi Meshes: Lloyd’s Method Given an initial set of generating points, Lloyd’s method may be used to arrive at a CVT: 1. Begin with any set of initial points (the generating point set) 2. Construct a Voronoi diagram for the set 3. Locate the mass centroid of each Voronoi cell 4. Move each generating point to the mass centroid of its Voronoi cell 5. Repeat 2-4 to convergence From Du et al. (1999) MacQueen’s method, a randomized alternative to Lloyd’s method, may also be used; no Voronoi diagrams need to be constructed, but convergence is generally much slower.

Centroidal Voronoi Meshes: Mesh Generation Mesh generation beginning from an icosahedral mesh. All points are free .

Centroidal Voronoi Meshes: Mesh Generation Mesh generation beginning from an icosahedral mesh with constrained refinement. Only points in the refinement region are free .

MPAS Meshes

Parallel decomposition The dual mesh of a Voronoi tessellation is a Delaunay triangulation – essentially the connectivity graph of the cells Parallel decomposition of an MPAS mesh then becomes a graph partitioning problem : equally distribute nodes among partitions (give each process equal work) while minimizing the edge cut (minimizing parallel communication) Graph partitioning We use the Metis package for parallel graph decomposition • Currently done as a pre-processing step, but could be done “on-line” Metis also handles weighted graph partitioning • Given a priori estimates for the computational costs of each grid cell, we can better balance the load among processes

Parallel decomposition MPAS uses halo (or ghost) DM parallel decomposition cells; Halo values are of the MPAS mesh communicated (currently Block of cells owned by MPI) a process Block plus two layers of halo/ghost cells Multithreading: Block of cells (shared memory) divided into nThreads threads and advanced in parallel (using OpenMP) nThreads = 3

Parallel decomposition Given an assignment of cells to a process, any number of layers of halo (ghost) cells may be added Block of cells owned by a process Cells are stored in a 1d array (2d with vertical dimension, etc.), with halo cells at the end of the array; the order of real cells may be updated to provide better cache re-use Block plus one layer of halo/ghost cells With a complete list of cells stored in a block, adjacent edge and vertex locations can be found; we apply a simple rule to determine ownership of edges and vertices adjacent to real cells in different blocks Block plus two layers of halo/ghost cells

Nonhydrostatic Atmospheric Solver Nonhydrostatic Variables: formulation Vertical coordinate: Equations Prognostic equations: • Prognostic equations for coupled variables. • Generalized height coordinate. • Horizontally vector invariant eqn set. • Continuity equation for dry air mass. • Thermodynamic equation for coupled potential temperature. Shallow atmosphere: Time integration scheme As in Advanced Research WRF - Diagnostics and definitions: Split-explicit Runge-Kutta (3rd order)

MPAS Vertical Mesh Specification of terrain: • High resolution terrain data (30 arcsec) averaged over grid-cell area • Terrain smoothing with one pass of a 4 th order Laplacian Smoothed Terrain-Following (STF) hybrid Coordinate Controls rate at which terrain influences are attenuated with height BTF Terrain influence that represents increased smoothing of the actual terrain with height Multiple passes of simple Laplacian smoother at each level: STF STF progressively smooths coordinate surfaces while transitioning to a height coordinate

MPAS -Tibetan Plateau, 28 o N 15 km grid 7.5 km grid 3 km grid Smoothed hybrid terrain-following (STF) coordinate Basic terrain-following (BTF) coordinate (Model top is at 30 km)

Nonhydrostatic Atmospheric Solver Prognostic equations: (1) Gradient operators (2) Flux divergence operators (3) Nonlinear Coriolis term

Operators on the Voronoi Mesh Pressure and KE gradients On the Voronoi mesh, P 1 P 2 is perpendicular to v 1 v 2 and is bisected by v 1 v 2 , hence P x ~ ( P 2 -P 1 ) / dx e is 2 nd order accurate.

Operators on the Voronoi Mesh cell-center KE evaluation Cell center kinetic energy: KE i Vertex kinetic energy: KE v

Operators on the Voronoi Mesh cell-center KE evaluation Cell center kinetic energy: KE i Vertex kinetic energy: KE v

Operators on the Voronoi Mesh cell-center KE evaluation Cell center kinetic energy: KE i Vertex kinetic energy: KE v

Operators on the Voronoi Mesh cell-center KE evaluation Cell center kinetic energy: KE i Vertex kinetic energy: KE v

Operators on the Voronoi Mesh Flux divergence and transport Transport equation, conservative form: Finite-Volume formulation, Integrate over cell: Apply divergence theorem: Discretize in time and space: Velocity divergence operator is 2 nd -order accurate for edge-centered velocities.

Operators on the Voronoi Mesh Flux divergence and transport Scalar transport equation for cell i : 1. Scalar edge-flux value ψ is the weighted sum of cell values from cells that share edge and all their neighbors. 2. An individual edge-flux is used to update the two cells that share the edge. 3. Three edge-flux evaluations and cell updates are needed to complete the Runge-Kutta timestep. 4. Monotonic constraint requires checking the cell-value update and renormalizing edge- fluxes if the cell updates are outside specific bounds (on the final RK3 update).

Operators on the Voronoi Mesh ‘Nonlinear’ Coriolis force Tangential velocity reconstruction: Nonlinear term: The general tangential velocity reconstruction produces a consistent divergence on the primal and dual grids, and allows for PV, enstrophy and energy* conservation in the nonlinear SW solver.

Operators on the Voronoi Mesh ‘Nonlinear’ Coriolis force Example: absolute vorticity at e 13 Example: absolute vorticity at vertex a

Time Integration Dynamics and Scalar Transport Options Default time integration Call physics Allows for smaller dynamics Do dynamics_split_steps timesteps relative to scalar Do step_rk3 = 1, 3 transport timestep and main compute large-time-step tendency Do acoustic_steps physics timestep. update u update rho, theta and w End acoustic_steps We can use any transport scheme End rk3 step here (we are not tied to an RK-based End dynamics_split_steps scheme). Scalar transport and physics are the Do scalar step_rk3 = 1, 3 expensive pieces in most scalar RK3 transport applications. End scalar rk3 step Call microphysics

MPAS Release – V5.0, January 2017 (1) mesoscale_reference physics suite Surface Layer: (Monin Obukhov): module_sf_sfclay.F as in WRF 3.8.1 PBL: YSU as in WRF 3.8.1 Land Surface Model (NOAH 4-layers): as in WRF 3.3.1. Gravity Wave Drag: YSU gravity wave drag scheme . Convection: new Tiedtke (nTiedtke), as in WRFV3.8.1 Microphysics: WSM6: as in WRF 3.8.1 Radiation: RRTMG sw as in WRF 3.8.1; RRTMG lw as in WRF 3.8.1 (2) scale_aware_CP (Convection-Permitting) physics suite Surface Layer: module_sf_mynn.F as in WRF 3.6.1 PBL: Mellor-Yamada-Nakanishi-Niino (MYNN) as in WRF 3.6.1 Land Surface Model (NOAH 4-layers): as in WRF 3.3.1. Gravity Wave Drag: YSU gravity wave drag scheme, WRF 3.6.1 Convection: Grell-Freitas scale aware scheme (modified from WRF 3.6.1) Microphysics: Thompson scheme (non-aeosol aware): as in WRF 3.8.1 Radiation: RRTMG sw as in WRF 3.8.1; RRTMG lw as in WRF 3.8.1

10-day 500 hPa Relative Vorticity Forecast 15-60 km variable 2 resolution mesh 56 km 1 36 km 0 16 km -1 15 km uniform resolution mesh MPAS Physics: -2 • WSM6 cloud microphysics s -1 x 10 4 • Tiedtke convection scheme • Monin-Obukhov surface layer • YSU PBL • Noah land-surface • RRTMG lw and sw.

Hazardous Weather Testbed Spring Experiment 2015 Forecasts Results from MPAS MPAS mesh mean cell spacing (km) MPAS mesh: 50 – 3 km variable resolution. CONUS is the 3 km region. Very smooth transition. MPAS Physics: • WSM6 cloud microphysics • Grell-Freitas convection scheme (scale-aware) • Monin-Obukhov surface layer • MYNN PBL • Noah land-surface • RRTMG lw and sw. 3-50 km mesh, ∆ x contours 4, 8, 12, 20, 30 40 km approximately 6.85 million cells 68% have < 4 km spacing (158 pentagons, 146 septagons)