libraries for large scale quadratic programming M. Čermák, V. Hapla, D. Horák, J. Kružík, L. Pospíšil*, R. Sojka, A. Vašatová + prof. Z. Dostál PETSc User Meeting 2016, Vienna, June 28-30, 2016 * Università dellaSvizzera italiana, Switzerland .it4i.cz
IT4Innovations • HPC infrastructure • 8 research programmes 2013 Czechia Prague Ostrava Brno 2014 Vienna Anselm - 94 TFLOPs system operational since June 2013 2015 Salomon – 2 PFLOPs system from June/July 2015 .it4i.cz
Salomon in numbers • 1008 compute nodes • 24,192 Intel Haswell cores 2 x 12 per node § • 129,024 GB RAM 128GB per node, 5.3GB per core § • 864 2 nd generation MICs (KNC) Intel Xeon Phi 7120P § Iargest European installation of MICs 52,704 cores § ⇒ interested in PETSc on MIC • Rpeak 2.01 Pflop/s • Rmax 1.46 Pflop/s (LINPACK) .it4i.cz
in a nutshell P arallel, E fficient, R obust, M odular, O bject-oriented, N umerical http://permon.it4i.cz/ • PermonQP generic QP solution framework § QP problems, transforms, solvers § • PermonFLLOP FETI DDM implementation § extends PermonQP § • both based on/extending PETSc open-source frameworkfor sci. comp. § together solve large contact mechanics problems § • started in 2011, ~30 000 lines of effective C code .it4i.cz
PETSc + PERMON Nonlinear solvers (SNES) PCPERMON? PermonFLLOP PCBDDC PermonQP solvers (QPS) PERMON = external PETSc extension TAO much like SLEPc PermonQP problems (QP) Linear solvers Preconditioners (KSP) (PC) Matrices Vectors Index Sets (Mat) (Vec) (IS) .it4i.cz
Build • easy • just build PETSc • set PETSC_DIR and PETSC_ARCH • set PERMON_DIR • make .it4i.cz
PermonFLLOP • implements FETI -5 x 10 2 non-overlappingDDM 1.5 1 0.5 0 • dual formulation -0.5 -1 -1.5 • unknown optimal Lagrange -2 0.5 0.5 0.5 0.4 0.3 0.4 0.2 0 0.1 0 multipliers found by -0.1 0.3 -0.2 -0.3 -0.4 -0.5 -0.5 0.2 0.1 iterative process 0 -0.1 -0.2 • they enforcecontinuity -0.3 -0.4 0.5 0.6 0.5 across subdomains 0.4 0.3 0.2 0 0.5 0.1 0 -0.1 -0.2 -0.3 0.4 -0.4 -0.5 -0.5 • mix iterative and 0.3 0.2 direct solvers 0.1 0 -0.1 0.5 0.5 0.4 0.3 0.2 0 0.1 0 .it4i.cz -0.1 -0.2 -0.3 -0.4 -0.5 -0.5
FETI: primal formulation 𝐯 𝐂 % 𝛍 = 𝐠 𝐋 𝐩 𝐂 𝐏 K SPD block-diagonal stiffness matrix § K (blocks ~ subdomains) § R columns span kernel of K f load vector § f § B signed boolean gluing matrix (can enforce also Dirichlet BC - TFETI) .it4i.cz
FETI: QP formulations 𝐯 𝐂 % 𝛍 = 𝐠 𝐋 (primal) § 𝐩 𝐂 𝐏 𝐂𝐋 0 𝐂 % 𝛃 = 𝐂𝐋 0 𝐠 𝛍 −𝐂𝐒 (dual) § −𝐒 % 𝐂 % −𝐒 % 𝐠 𝐏 ⁄ 𝐯 % 𝐋𝐯 − 𝐯 % 𝐠 s.t. 𝐂𝐯 = 𝐩 7 8 min (primal) § 𝐯 𝐆 = 𝐂𝐋 0 𝐂 % , 𝐞 = 𝐂𝐋 0 𝐠 , 𝐇 = −𝐒 % 𝐂 % , 𝐟 = −𝐒 % 𝐠 𝐯 = 𝐋 > 𝐠 − 𝐂 % 𝛍 + 𝐒𝛃 𝛃 = − 𝐇𝐇 % @7 𝐇(𝐆𝛍 − 𝐞) ⁄ 𝛍 % 𝐆𝛍 − 𝛍 % 𝐞 s.t. 𝐇𝛍 = 𝐟 7 8 min (dual) § 𝛍 .it4i.cz
FETI: improving the dual formulation ⁄ 𝛍 % 𝐆𝛍− 𝛍 % 𝐞 s.t. 𝐇𝛍 = 𝐟 7 8 1) min 𝛍 F + 𝛍 G 𝛍 = 𝛍 𝛍 G = 𝐇 % 𝐇𝐇 % @7 𝐟 ∈ Im𝐇 % 𝐞̅ = 𝐞 − 𝐆𝛍 G F ∈ Ker𝐇 unknown 𝛍 F % 𝐞̅ s.t. 𝐇𝛍 F % 𝐆𝛍 F − 𝛍 F = 𝐩 7 8 ⁄ 𝛍 2) min M 𝛍 𝐐 = 𝐉 − 𝐇 % 𝐇𝐇 % @7 𝐇 , [ 𝐇𝐇 % @7 is coarse problem] F % 𝐐𝐞̅ ⟺ 𝐐𝐆𝐐𝛍 F % 𝐐𝐆𝐐𝛍 F − 𝛍 F = 𝐐 7 8 ⁄ 𝛍 3) min [solve with CG] M 𝛍 .it4i.cz
FETI: inequality constraints • primal formulation ⁄ 𝐯 % 𝐋𝐯 − 𝐯 % 𝐠 7 8 min s.t. 𝐂 Q 𝐯 = 𝐩 and 𝐂 R 𝐯 ≤ 𝐝 R 𝐯 (nonpenetration -mortar conditions) • dual formulation F % 𝐐𝐞̅ F % 𝐐𝐆𝐐𝛍 F − 𝛍 F = 𝐩 and 𝛍 F ≥ 𝒎 7 8 ⁄ 𝛍 min s.t. 𝐇𝛍 M 𝛍 (is no more a linear system !) • use SMALXE + MPGP (discussed later) to solve this dual formulation .it4i.cz
Total FETI (TFETI) • Dirichlet boundary conditions handled same way as gluing • all subdomain floating all stiffness matrices singular § no need to detect it § same nullspaces in typical cases § uniform handling § • additional rows of B matrix • effect of richer coarse space? F .it4i.cz
PermonFLLOP calls PermonQP MPI_Comm comm; Mat K, BE, BI; Věc f, cE, cI, x; QP qp; QPCreate(comm, &qp); QPSetOperator(qp, K); PetscErrorCode QPFetiPrepare (QP qp) QPSetRhs(qp, f); { QPSetEq(qp, BE, cE); QPSetIneq(qp, BI, cI); … QPTDualize(qp); QPFetiPrepare(qp); QPTHomogenizeEq(qp); QPTEnforceEqByProjector(qp); QPSCreate(comm, &qps); … QPSSetQP(qps, qp); QPSSetFromOptions(qps); } QPSSolve(qps); Sequence of QP transforms QPGetSolutionVector(qp,&x); .it4i.cz
PermonQP • QP solution framework • separation of concerns user solver QP problems (QP) • parent parent QP transforms (QPT) • QP0 QP1 QP2 and corresponding injects injects reconstruct. reconstruct. backward function function reconstructions child child QP solvers (QPS) • • “gradually get rid of difficulties in the original formulation” • modularity/composability • automatic/manual choice of solver • applications: contact problems, data fitting, support vector machines, ... .it4i.cz
PermonQP and PETSc TAO TAO • general objective function • only unconstrained and box constrained PermonQP • specific objective function (convex quadratic), Mat, Vec • more like KSP, less like SNES • can be coupled with FETI DDM (PermonFLLOP) • more general constraints linear equality § linear inequality § separable convex constraints – next slide § • applicable to any subset of indices (IS) • simple bound / box constraints • quadratic constraints (spherical, elliptical) • conical constraints, … .it4i.cz
Generalizing box constraints - QPC • separable convex constraints • applicable to any subset of indices (IS) • simple bound / box constraints • quadratic constraints (spherical, elliptical) • conical constraints, … [ 𝐲 % 𝐁𝐲 − 𝐲 % 𝐜 min 1 2 QPCBOUND QPCBOX QPC lower bound lower bound index set s.t. … upper bound 𝐲 ≥ 𝒎 𝐦 ≤ 𝐲 ≤ 𝐯 QPCELLIPTIC major axis QPCQUADRATIC QPCCONICAL minor axis radii angles .it4i.cz
Recommend
More recommend
