Parallel Time-Domain Boundary Element Method for 3-Dimensional Wave - - PowerPoint PPT Presentation

Parallel Time-Domain Boundary Element Method for 3-Dimensional Wave Equation

Space-Time Methods for PDEs, RICAM Linz, November 10, 2016

• D. Luk´

aˇ s, M. Merta, J. Zapletal, and A. Veit Vˇ SB–Technical University of Ostrava, Czech Rep. University of Chicago industry email: dalibor.lukas@vsb.cz

Parallel Time-Domain Boundary Element Method for 3-Dimensional Wave Equation

Outline

• Parallel fast BEM and applications
• Boundary integral formulation of sound-hard scattering
• Time-domain boundary element method
• Parallelization, preconditioning, numerical experiments
• Conclusion, outlook, references

Parallel Time-Domain Boundary Element Method for 3-Dimensional Wave Equation

Outline

• Parallel fast BEM and applications
• Boundary integral formulation of sound-hard scattering
• Time-domain boundary element method
• Parallelization, preconditioning, numerical experiments
• Conclusion, outlook, references

Parallel fast BEM and applications

Laplace equation in an unbounded domain Ω ⊂ R3 lipschitz domain    −△u( x) = 0,

• x ∈ Ωe := R3 \ Ω

γN u(x) := du

dn(x) = g(x),

x ∈ Γ := ∂Ω |u( x)| = O(1/| x|), | x| → ∞. Representation formula ∀ x ∈ Ω : u( x) = −

• Γ

γNu(y) G( x, y) dS(y)

• △(.)=0 in Ωe, |.|=O(1/|

x|)

+

• Γ

u(y) γN,yG( x, y) dS(y)

• △(.)=0 in Ωe, |.|=O(1/|

x|)

, where G( x, y) :=

1 4π| x−y|.

An indirect method Find an auxiliary double-layer density φ : Γ → R such that γN

• Γ

φ(y) γN,yG(x, y) dS(y) = g(x), x ∈ Γ u( x) =

• Γ

φ(y) γN,yG( x, y) dS(y), x ∈ Ωe.

Parallel fast BEM and applications

Shape optimization of a DC electromagnet, FEM-BEM coupling

Ωi: ferromagnetic yoke Ωi Ωe: air Ωe: air Ωe

J

coil Ωm sample Ωo: focusing optics Γ: boundary

L., Postava, ˇ Zivotsk´ y: J Magn Magn Mater ’10, Math Comput Simulat ’12

Parallel fast BEM and applications

Acoustics of a railway wheel H-matrices, ACA/FMM

Parallel fast BEM and applications

Parallel fast BEM using cyclic graph decompositions

1 2 3 4 5 6 7 8 9 10 11 12 rotate G0 G1 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10 11 11 12 12

L., Kov´ aˇ r, Kov´ aˇ rov´ a, Merta: Numer Alg ’15 Solution to the system of 2.7M DOFs on 273 cores in 16 minutes.

Parallel fast BEM and applications

• Elmg. forming of plates with Fraunhofer IWU Chemnitz, FEM-BEM

0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05 −0.015 −0.01 −0.005 0.005 0.01 0.015 0.02 r [m] z [m] Time 3e−05 s, Thalf 6e−05 s, Jmax 9.48183e+06 A/m2, Hmax 61.1986 A/m, Fmax 720.694 N/m3.

Parallel fast BEM and applications

Structural health monitoring of aircrafts with Honeywell Int. using 3d anisotropic mixed elements (TD-NNS) by [Pechstein (Sinwel), Sch¨

• berl ’12].

piezo-actuator crack piezo-sensor

Parallel Time-Domain Boundary Element Method for 3-Dimensional Wave Equation

Outline

• Parallel fast BEM and applications
• Boundary integral formulation of sound-hard scattering
• Time-domain boundary element method
• Parallelization, preconditioning, numerical experiments
• Conclusion, outlook, references

Boundary integral formulation of sound-hard scattering

Sound-hard scattering Given the scatterer Ω and the causal incident wave uinc (satisfying ¨ u − △u = 0), we look for the scattered field u: uinc u Γ Ω        ¨ u − △u = 0 in Ωe × [0, T], u(., 0) = ˙ u(., 0) = 0 in Ωe, ∂u ∂n = −∂uinc ∂n on Γ × [0, T].

Boundary integral formulation of sound-hard scattering

Double-layer indirect boundary integral method We search for u in the form of the retarded double-layer potential u(x, t) = − 1 4π

• Γ

n(y) · (x − y) x − y

• φ(y, t − x − y)

x − y2 + ˙ φ(y, t − x − y) x − y

• dS(y),

which satisfies the wave equation and the initial conditions. It remains to fulfill the Neumann boundary condition lim

Ω∋ x→x∈Γ n(x) · ∇ xu(

x, t)

• =:(Wφ)(x,t)

= g(x, t) on Γ × [0, T], where g := −∂uinc

∂n .

Boundary integral formulation of sound-hard scattering

Weak boundary integral formulation [Bamberger, HaDuong ’86] Find φ ∈ V such that a(ξ, φ) = b(ξ) ∀ξ ∈ V, where a(ξ, φ) := T

• Γ
• Γ
• n(x) · n(y)

4πx − y ˙ ξ(x, t) ¨ φ(y, t − x − y) + curlΓ ˙ ξ(x, t) · curlΓφ(y, t − x − y) 4πx − y

• dS(y) dS(x) dt,

b(ξ) := T

• Γ

g(x, t) ˙ ξ(x, t) dS(x) dt.

Parallel Time-Domain Boundary Element Method for 3-Dimensional Wave Equation

Outline

• Parallel fast BEM and applications
• Boundary integral formulation of sound-hard scattering
• Time-domain boundary element method
• Parallelization, preconditioning, numerical experiments
• Conclusion, outlook, references

Time-domain boundary element method

Discrete ansatz Replace V by a finite-dimensional subspace V h,∆t spanned by the tensor-product of N temporal and M spatial basis functions: φh,∆t(x, t) :=

N

• l=1

M

• j=1

αj

l ϕj(x) bl(t).

We arrive at the (N M) × (N M) block linear system   A1,1 . . . A1,N . . . ... . . . AN,1 . . . AN,N     α1 . . . αN   =   b1 . . . bN   , where (Ak,l)i,j := a (ϕi(x) bk(t), ϕj(y) bl(t)) , (bk)i := b (ϕi(x) bk(t)) , (αl)j := αj

l .

Time-domain boundary element method

Matrix: a deeper look (Ak,l)i,j =

• supp ϕi
• supp ϕj

n(x) · n(y) 4πx − y ϕi(x) ϕj(y)

=:Ψk,l(x−y)

• T

˙ bk(t)¨ bl(t − x − y) dt dS(y) dS(x) +

• supp ϕi
• supp ϕj

curlΓϕi(x) · curlΓϕj(y) 4πx − y T ˙ bk(t) bl(t − x − y) dt

• =:

Ψk,l(x−y)

dS(y) dS(x), Piecewise smooth time-ansatz expensive quadrature due to nontrivial intersection of the light cone supp Ψk,l, supp Ψk,l with supp ϕi × supp ϕj. [El Gharib ’99], [Stephan, Maischak, Ostermann ’08]

Time-domain boundary element method

C∞-smooth (partition of unity) temporal basis [Sauter, Veit ’12, ’14]

0.5 1.5 2.5 0.2 0.4 0.6 0.8

t [s]

b0 b1 b2 b3

• allows for Sauter-Schwab quadrature
• ver

supp ϕi × supp ϕj

• and for higher-order approximation in

time. (Ak,l)i,j =

• supp ϕi
• supp ϕj

n(x) · n(y) 4πx − y ϕi(x) ϕj(y)

=:Ψk,l(x−y)∈C∞(R)

• T

˙ bk(t)¨ bl(t − x − y) dt dS(y) dS(x) +

• supp ϕi
• supp ϕj

curlΓϕi(x) · curlΓϕj(y) 4πx − y T ˙ bk(t) bl(t − x − y) dt

• =:

Ψk,l(x−y)∈C∞(R)

dS(y) dS(x), To accelerate the assembly, Ψ and Ψ are replaced by piecewise Chebyshev interpolants.

Time-domain boundary element method

Convergence of φh,∆t(x, .) − φanalytical(x, .)L2(0,T) on the sphere [Veit ’12] Ω the unit sphere, φanalytical a spherical harmonic function, x ∈ Γ 1st-order time-basis functions 2nd-order time-basis functions

10

1

10

2

10

−3

10

−2

10

−1

1/N

Number of timesteps Error

5 10 20 40 10

−5

10

−4

10

−3

1/N 2

Number of timesteps Error

Time-domain boundary element method

Matrix structure The matrix is sparse and it has a block-Hessenberg structure.

Parallel Time-Domain Boundary Element Method for 3-Dimensional Wave Equation

Outline

• Parallel fast BEM and applications
• Boundary integral formulation of sound-hard scattering
• Time-domain boundary element method
• Parallelization, preconditioning, numerical experiments
• Conclusion, outlook, references

Parallelization, preconditioning, numerical experiments

Parallel implementation

• For equidistant time stepping only the blue

parts have to be assembled.

• We employ a hybrid MPI-OpenMP model:

– pairs

• f

triangles corresponding to nonzero entries are evenly distributed to MPI nodes, – on each node the assembly (quadrature) is performed using OpenMP.

• We use up to 64 Intel Xeon E5 nodes (1024

cores) of the cluster Anselm, Vˇ SB-TU Os- trava.

Parallelization, preconditioning, numerical experiments

Parallel implementation We distribute blocks among MPI processes (nodes) as follows: Each process does the following:

• 1. Precompute the sparsity pattern of the block.
• 2. Distribute pairs of elements among computational nodes using MPI.
• 3. On each computational node assemble its contribution to the block in a shared

memory using OpenMP.

• 4. Gather the data on the MPI rank(s) owning the block.

Parallelization, preconditioning, numerical experiments

Weak parallel scalability of the assembly Submarine surface decomposed into 5604 triangles, 40 time steps, 80 time DOFs

Parallelization, preconditioning, numerical experiments

Preconditioning To accelerate FGMRES iterations we approxi- mate the upper Hessenberg matrix by an alge- braic multilevel preconditioner: A := AI,I AI,II AII,I AII,II

• ≈

AI,I AII,I AII,II

• so that A−1

I,I and A−1 II,II are approximated by a

small fixed number of FGMRES iterations even- tually preconditioned the same again.

Parallelization, preconditioning, numerical experiments

Numerical experiments, ball, 162 nodes, 1st-order in time

Parallelization, preconditioning, numerical experiments

Numerical experiments, ball, 162 nodes, 2nd-order in time

Parallelization, preconditioning, numerical experiments

Numerical experiments, submarine 2804 nodes, 40 time steps, 1st-order in time. GMRES(500): 9636 iters., 1607 s FGMRES(500,1(40)): 243 iters., 962 s

Parallelization, preconditioning, numerical experiments

• M. Merta, J. Zapletal et al.: BEM4I library

Parallel Time-Domain Boundary Element Method for 3-Dimensional Wave Equation

Outline

• Parallel fast BEM and applications
• Boundary integral formulation of sound-hard scattering
• Time-domain boundary element method
• Parallelization, preconditioning, numerical experiments
• Conclusion, outlook, references

Parallel Time-Domain Boundary Element Method for 3-Dimensional Wave Equation

Conclusion, outlook Time-domain BEM for 3d wave equation, adaptivity in time, parallely scalable assembly and postprocessing, → mapping properties of the operator, preconditioning, → adaptivity in time, → extension to the elastic wave equation. References

• Analysis: Bamberger, Ha Duong, Math. Meth. Appl. Sci. ’86
• Numerics: Sauter, Veit, Numer. Math. ’14
• Parallelization: Veit, Merta, Zapletal, L., Int. J. Numer. Meth. Eng. ’16
• Parallel fast BEM: L., Kov´

aˇ r, Kov´ aˇ rov´ a, Merta, Numer. Alg. ’15