Speeding up SPH models: local adaptivity and parallelization for - - PowerPoint PPT Presentation

▶
speeding up sph models local adaptivity and
SMART_READER_LITE
LIVE PREVIEW

Speeding up SPH models: local adaptivity and parallelization for - - PowerPoint PPT Presentation

Apr 29, 2023 30 likes •421 views

Speeding up SPH models: local adaptivity and parallelization for soil simulations. Yaidel Reyes L opez Dirk Roose Carlos Recarey Morfa KU Leuven, Belgium UCLV, Santa Clara, Cuba June 30, 2014 Elastic-Plastic soil model in SPH Dynamic

slide-1
SLIDE 1

Speeding up SPH models: local adaptivity and parallelization for soil simulations.

Yaidel Reyes L´

  • pez

Dirk Roose Carlos Recarey Morfa

KU Leuven, Belgium UCLV, Santa Clara, Cuba

June 30, 2014

slide-2
SLIDE 2

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Simulation of real life problems

Large deformations and post-failure flow: SPH

slide-3
SLIDE 3

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Simulation of real life problems

Large deformations and post-failure flow: SPH

Computationally demanding simulations!
slide-4
SLIDE 4

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Simulation of real life problems

Large deformations and post-failure flow: SPH

Computationally demanding simulations! Adaptivity Parallelization
slide-5
SLIDE 5

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Outline

Elastic-Plastic soil model in SPH Dynamic refinement General aspects Error due to refinement Numerical instabilities Refinement parameters Results Cohesive soil: tensile instability Artificial stress method 3D results Parallelization Shared Memory parallelization Domain decomposition Scalability Application Conclusions

slide-6
SLIDE 6

Section Outline

Elastic-Plastic soil model in SPH Dynamic refinement General aspects Error due to refinement Numerical instabilities Refinement parameters Results Cohesive soil: tensile instability Artificial stress method 3D results Parallelization Shared Memory parallelization Domain decomposition Scalability Application Conclusions

slide-7
SLIDE 7

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Elastic-Plastic soil model in SPH [Bui et al. 2008]

Continuity equation (soil density)

Dρi Dt =

N

  • j=1

mj(vα

i − vα j )∂Wij

∂xα

i

, W: cubic spline kernel function ( α: Einstein notation)

Momentum equation

Dvα

i

Dt =

N

  • j=1

mj

  • σαβ

i

ρ2

i

+ σαβ

j

ρ2

j

− Πijδαβ

  • ∂Wij

∂xβ

i

+ F α

i

σ: total stress tensor; F: gravity force Πij: Artificial viscosity → reduces instabilities

slide-8
SLIDE 8

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Elastic-plastic model

Total deformation: strain rate tensor ˙ εαβ = ˙ εαβ

e

+ ˙ εαβ

p

˙ εe → elastic deformation (Hooke’s law) ˙ εp → plastic deformation (plastic flow rule) Drucker-Prager yield condition. Stress-strain relationship: ˙ σαβ = σαγ ˙ ωβγ + σγβ ˙ ωαγ + 2G˙ eαβ + K ˙ εγγδαβ + P αβ G, K: elastic moduli ˙ e: deviatoric strain rate ˙ ω: spin rate P → associated or non-associated plastic flow rule

slide-9
SLIDE 9

Section Outline

Elastic-Plastic soil model in SPH Dynamic refinement General aspects Error due to refinement Numerical instabilities Refinement parameters Results Cohesive soil: tensile instability Artificial stress method 3D results Parallelization Shared Memory parallelization Domain decomposition Scalability Application Conclusions

slide-10
SLIDE 10

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Dynamic refinement

Simulation of the collapse of a block of non-cohesive soil

coarse discretization fine discretization

coarse discretization: accurate enough for undeformed regions fine discretization: required for deformed regions
slide-11
SLIDE 11

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Dynamic refinement

Simulation of the collapse of a block of non-cohesive soil

coarse discretization fine discretization

coarse discretization: accurate enough for undeformed regions fine discretization: required for deformed regions

⇒ dynamic refinement of initially “coarse”discretization reduces computational cost and memory requirements

slide-12
SLIDE 12

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Dynamic refinement

Refinement criterion decides when particle is refined must ensure that dynamically refined simulation is

nearly as accurate as simulation using fine resolution nearly as fast as simulation using the coarse resolution

Refinement procedure (how to refine particle) must ensure that

local properties are only slightly changed and global properties

are conserved

Related work: astrophysics Bate et al., Kitsionas & Whitworth, Meglicki et al., Monaghan & Varnas fluid flows Feldman & Bonet, Lastiwka et al., Vacondio et al. solids Spreng et al.

slide-13
SLIDE 13

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Refinement criterion

Refinement criterion can be problem dependent. We use a criterion based on deformation: particle pi is refined if the trace of the strain tensor exceeds threshold (εxx

i )2 + (εyy i )2 + (εzz i )2 > εmax

slide-14
SLIDE 14

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Refinement procedure

particle p selected for refinement : replaced by daughter particles d = 1, ..., M.

daughter particles’ position

ǫ separation parameter α smoothing ratio

daughter particles’ properties

hd = αhp md = λpmp, λ = 1

4

vd = vp

Other properties are interpolated: ρd, σd, εd Corrected SPM [Chen at al.] f(x) =

N

  • j=1

mj ρj f(xj)W (x − xj, hj)

N

  • j=1

mj ρj W (x − xj, hj) ,

slide-15
SLIDE 15

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Error due to refinement: density & kernel gradient

Dρi Dt =

N

  • j=1

mj(vα

i − vα j )∂Wij

∂xα

i

, Local error: eρ

p(x) = Dρ(x)

Dt − Dρ(x) Dt ∗ = mp (v(x) − vp) ·

  • ∇Wp(x) −

M

  • d=1

λd∇Wd(x)

  • Global error:

Eρ

p = m2 p

  • Ω
  • (v(x) − vp) ·
  • ∇Wp(x) −

M

  • d=1

λd∇Wd(x) 2 dx. v(x) varies in space and time → difficult to analyze ⇒ we consider E∇W

p

=

  • Ω
  • ∂Wp(x)

∂xα −

M

  • d=1

λd ∂Wd(x) ∂xα 2 dx, ← Kernel gradient error

slide-16
SLIDE 16

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Kernel gradient error due to refinement

E∇W

p

=

  • Ω
  • ∂Wp(x)

∂xα −

M

  • d=1

λd ∂Wd(x) ∂xα 2 dx, depends on ref. params ǫ & α For h = hp and h = αhp: compute E∇W

p

numerically for refinement parameters in region in (ǫ, α)-plane around (0.5, 0.5) (ǫ, α) = (0.5, 0.5) : ‘natural’ values (‘uniform’ grid refinement) select (ǫ, α) so that E∇W

p

is minimal (or small)

h(x) = hp h(x) = αrhp

0.2 0.3 0.4 0.5 0.6 0.7 0.8

ǫr

0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8

αr

  • 8
  • 7
  • 6
  • 5
  • 4
  • 3
  • 2
  • 1
0.2 0.3 0.4 0.5 0.6 0.7 0.8

ǫr

0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8

αr

  • 8
  • 7
  • 6
  • 5
  • 4
  • 3
  • 2
  • 1
slide-17
SLIDE 17

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Numerical instabilities

Swegle’s instability condition: W ′′T > 0, T: total stress Cubic spline kernel: ’standard’ particle configurations: |xi − xj| > 2

3h ⇒ W ′′ > 0

⇒SPH is:

stable in compression

regime (T < 0)

unstable in tension regime

(T > 0)

W ′′ < 0

unstable in
  • mpression

W ′′ > 0

unstable in tension

W ′′ > 0

unstable in tension

W W ′

2 3h

− 2

3h

We should avoid instabilities in compression regime due to the refinement

slide-18
SLIDE 18

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Numerical instabilities

i0 i i1 j0 j j1 i0 i i1 j

Conditions for W ′′ < 0: (ǫ, α)-plane →

  • Cond. 1 :

i0i1 < 2 3 hi0i1 0 < 2 3 αra − ǫ

  • Cond. 2 :

i1j0 < 2 3 hi1j0 ij dref < 2 3 αra + ǫ

  • Cond. 3 :

i1j < 2 3 hi1j ij dref < (1 + α)ra 3 + ǫ 2 hp = radref hd = αhp d1d2 = ǫdref

αr ǫr ra = 1.0 αr ǫr ra = 1.2

ij dref = 1.0 ij dref = 0.85

0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.2 0.3 0.4 0.5 0.6 0.7 0.8

ij dref = 1.0 ij dref = 0.85

0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.2 0.3 0.4 0.5 0.6 0.7 0.8

slide-19
SLIDE 19

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Optimal refinement parameters ǫ and α

( ǫ, α)-plane light color: small kernel gradient error E∇W

p

Two cases: h(x) = hp and h(x) = αhp for ra = 1 and ra = 1.2 (hp = radref) Take into account stability conditions! Tests with (ǫ, α) values indicated by dots

ra = 1.0

h(x) = hp h(x) = αrhp 0.2 0.3 0.4 0.5 0.6 0.7 0.8 ǫr 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 αr

  • 8
  • 7
  • 6
  • 5
  • 4
  • 3
  • 2
  • 1

0.2 0.3 0.4 0.5 0.6 0.7 0.8 ǫr

ra = 1.2

h(x) = hp h(x) = αrhp 0.2 0.3 0.4 0.5 0.6 0.7 0.8 ǫr 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 αr

  • 8
  • 7
  • 6
  • 5
  • 4
  • 3
  • 2
  • 1

0.2 0.3 0.4 0.5 0.6 0.7 0.8 ǫr

slide-20
SLIDE 20

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Aluminum bars test case [H. Bui]

Parameters: ρ = 2650 kg/m3, υ = 0.3, K = 0.7 MPa, co = 0, φ = 19.8◦, ψ = 0◦

top to bottom: coarse; (ǫ, α) = (0.5, 0.5) (0.55, 0.6) (0.55, 0.7); fine

video

t = 0.060000 sec t = 0.110000 sec t = 0.600000 sec (ǫr, αr) = (0.5, 0.5) (ǫr, αr) = (0.55, 0.5) (ǫr, αr) = (0.55, 0.6) 0.02 0.04 0.06 0.08 0.1 1 2 3 4 5 6 7 8 0.02 0.04 0.06 0.08 0.1 0.02 0.04 0.06 0.08 0.1 0.02 0.04 0.06 0.08 0.1 0.05 0.1 0.15 0.2 0.02 0.04 0.06 0.08 0.1 0.05 0.1 0.15 0.2 0.25 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

slide-21
SLIDE 21

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Aluminum bars test case

Appearance of instabilities top to bottom: (ǫ, α) = (0.5, 0.5) (0.55, 0.6) (0.55, 0.7)

t = 0.600000 sec (ǫr, αr) = (0.5, 0.5) (ǫr, αr) = (0.55, 0.6) (ǫr, αr) = (0.55, 0.7) 0.02 0.04 0.06 0.08 0.1 0.02 0.04 0.06 0.08 0.1 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.02 0.04 0.06 0.08 0.1

slide-22
SLIDE 22

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Non-cohesive soil

Parameters: ρ = 1850 kg/m3, υ = 0.3, K = 1.5 MPa, co = 0, φ = 45◦, ψ = 0◦

top to bottom: coarse; (ǫ, α) = (0.5, 0.5) (0.55, 0.5) (0.55, 0.6); fine

t = 0.250000 sec t = 0.450000 sec t = 1.800000 sec (ǫr, αr) = (0.5, 0.5) (ǫr, αr) = (0.55, 0.5) (ǫr, αr) = (0.55, 0.6) 0.5 1 1.5 2 1 2 3 4 5 6 7 8 0.5 1 1.5 2 0.5 1 1.5 2 0.5 1 1.5 2 0.5 1 1.5 2 2.5 3 3.5 4 0.5 1 1.5 2 1 2 3 4 5 0 1 2 3 4 5 6 7 8 9

slide-23
SLIDE 23

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

3D non-cohesive soil

video

slide-24
SLIDE 24

Section Outline

Elastic-Plastic soil model in SPH Dynamic refinement General aspects Error due to refinement Numerical instabilities Refinement parameters Results Cohesive soil: tensile instability Artificial stress method 3D results Parallelization Shared Memory parallelization Domain decomposition Scalability Application Conclusions

slide-25
SLIDE 25

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Tensile instability in cohesive soil

Cohesion → tensile forces ⇒ instabilities

2D simulation cohesive soil [Bui et al. 2008] 3D simulation cohesive soil

slide-26
SLIDE 26

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Artificial stress method

[Monaghan 2000, Gray et al. 2001] → presented results in 2D

Dvα

i

Dt =

N

  • j=1

mj

  • σαβ

i

ρ2

i

+ σαβ

j

ρ2

j

− Πijδαβ + f n

ij(Rαβ i

+ Rαβ

j )

  • ∂Wij

∂xβ

i

+F α.

fij = Wij W(dref, h), ˆ Raa

i

=

  • −ǫas

ˆ σaa

i

ρ2

i

: ˆ σaa

i

> 0 : otherwise

ǫas, n : scalar parameters (numerically chosen) ˆ

σi : diagonal

slide-27
SLIDE 27

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Artificial stress method

Calculate Ri:

  • 1. find ˆ

σi : diagonalize σi

  • 2. calculate ˆ

Ri

  • 3. calculate Ri : rotate ˆ

Ri to original coord. system σ is symmetric → σi = Qiˆ σiQT

i

Find eigenvalues and eigenvectors Simple analytical solution in 2D, not so simple in 3D:

analytical solution: cumbersome, possibly large relative errors iterative methods: slow

Solution [Kopp 2008] Cardano’s formula (analytical solution) if risk of large rounding errors: QL

slide-28
SLIDE 28

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

3D elastic beam

Remove plasticity term.

video

  • 0.6
  • 0.4
  • 0.2
0.2 0.4 0.6 0.1 0.2 0.3 0.4 0.5 0.6 0.7

yL t

Theoreti al Sim ulation
slide-29
SLIDE 29

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

3D slope

video

Current work with Prof. Bui: Validation against FEM.

slide-30
SLIDE 30

Section Outline

Elastic-Plastic soil model in SPH Dynamic refinement General aspects Error due to refinement Numerical instabilities Refinement parameters Results Cohesive soil: tensile instability Artificial stress method 3D results Parallelization Shared Memory parallelization Domain decomposition Scalability Application Conclusions

slide-31
SLIDE 31

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Parallelization

Parallelization efforts mainly for highly parallel systems:

clusters, GPUs (SPH-flow, DualSPHPysics, GPUSPH)

Shared memory systems:

increasing number of cores complex memory hierarchies (shared caches, ccNUMA)

c

L1 L2 L3

c

L1 L2

c

L1 L2

c

L1 L2 Memory

Interconnect socket0 socket1

c

L1 L2 L3

c

L1 L2

c

L1 L2

c

L1 L2 Memory

Efficiency challenges:

data races, load imbalance, ... extra cache misses, NUMA-unfriendly memory allocation
slide-32
SLIDE 32

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Shared Memory parallelization

Efficiency

avoid cache misses → requires data locality

→ neighboring particles close in memory, but particles move! Then? dynamic redistribution of particles in the memory

NUMA-friendly memory allocation → per-thread local storage

→ domain decomposition Requires dynamic load balancing Space filling curve: sequentialization of multidimensional data preserves locality properties sort particles (Z-curve) → keep interacting particle close in memory

slide-33
SLIDE 33

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Domain decomposition

Space filling curve for domain decomposition: Computationally inexpensive load balancing

slide-34
SLIDE 34

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Domain decomposition

slide-35
SLIDE 35

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Strong scalability

8 x Xeon E7-2830 Westmere-EX, 8 cores @ 2.13GHz

8 16 32 64 number of threads 5 10 15 20 speedup

1638400 5898240

slide-36
SLIDE 36

Section Outline

Elastic-Plastic soil model in SPH Dynamic refinement General aspects Error due to refinement Numerical instabilities Refinement parameters Results Cohesive soil: tensile instability Artificial stress method 3D results Parallelization Shared Memory parallelization Domain decomposition Scalability Application Conclusions

slide-37
SLIDE 37

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Application: soil-tool interaction

video

slide-38
SLIDE 38

Section Outline

Elastic-Plastic soil model in SPH Dynamic refinement General aspects Error due to refinement Numerical instabilities Refinement parameters Results Cohesive soil: tensile instability Artificial stress method 3D results Parallelization Shared Memory parallelization Domain decomposition Scalability Application Conclusions

slide-39
SLIDE 39

Elastic-Plastic soil model in SPH Dynamic refinement Cohesive soil: tensile instability Parallelization Application Conclusions

Conclusions

Dynamic refinement:

adequate refinement parameters based on the error estimate

and the stability conditions

reduced computational requirements the accuracy is very similar to that of the simulation using the

fine discretization Elastic-plastic soil model:

3D simulations (non-cohesive and cohesive)

Software:

GSPHLib (Generic SPH Library): generic C++ classes to assemble different models (WCSPH,

elastic, elastic-plastic)

dimensional analysis (compile-time) parallel implementation (shared memory) python interface To be made available soon