Parallel-in-Time Optimization with the General-Purpose XBraid Package Stefanie Günther, Jacob Schroder , Robert Falgout, Nicolas Gauger 7th Workshop on Parallel-in-Time methods Wednesday, May 3 rd , 2018 LLNL-PRES-750301 This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344. Lawrence Livermore National Security, LLC Lawrence Livermore National Laboratory LLNL-PRES-750301 1
Outline § Motivation: parallelizing optimization problems § Parallel-in-time overview: Multigrid reduction in time (MGRIT) § XBraid-adjoint code interface : Open source implementation • Non-intrusively uses existing user code • Example: user-interface for simple scalar ODE problem Lawrence Livermore National Laboratory LLNL-PRES-750301 2
Motivation: PDE constrained optimization Base flow Example 1 § Objective: Lift maximization § Design: Amplitudes of actuation Runtimes § Simulation: 2.5h Optimized flow 54% lift increase § Optimization: 1,152h 1. Ötzkaya, Nemili et al., 2015 Lawrence Livermore National Laboratory LLNL-PRES-750301 3
Problem description: Optimization with unsteady PDEs § Optimize object function J, with a design variable u (continuous) Z T min 1 J ( y ( t ) , u ) dt T 0 § While satisfying constraint of the forward in time process, (continuous) with state variable y and initial condition g δ y ( t ) + c ( y ( t ) , u ) = 0 , ∀ t ∈ (0 , T ) δ t y (0) = g Lawrence Livermore National Laboratory LLNL-PRES-750301 4
Problem description: Optimization with unsteady PDEs § Optimize object function J, with a design variable u (discrete) N min 1 J n := J ( y n , u ) X J ( y i , u ) N i =1 § While satisfying constraint of the forward in time process, (discrete) with state variable y and initial condition g y n = Φ ( y n − 1 , u ) , n = 1 , . . . , N y 0 = g Lawrence Livermore National Laboratory LLNL-PRES-750301 5
First Order Optimality Conditions N ( J n + (¯ y n ) T ( Φ n − 1 − y n )) X Form Lagrangian L = i State equations: 1. y n = Φ ( y n − 1 , u ) , n = 1 , . . . , N y 0 = g Adjoint equations: 2. y n = r y n J n + ( δ y Φ n ) T ¯ y n , ¯ n = N, . . . , 1 y N +1 = 0 ¯ Design equation: 3. N r u J n + ( δ u Φ n − 1 ) T ¯ X y n − 1 � � = 0 n =1 Lawrence Livermore National Laboratory LLNL-PRES-750301 6
Nested Optimization Approach Initial design u i For i = 1, 2, ... y n Time State equations solve : 1. Parallel! y n = Φ ( y n − 1 , u i ) , n = 1 , . . . , N y 0 = g y n ¯ Time Adjoint equations solve : 2. Parallel! y n = r y n J n + ( δ y Φ n ) T ¯ y n , ¯ n = N, . . . , 1 y N +1 = 0 ¯ Design update : 3. N ! ( r u J n + ( δ u Φ n − 1 ) T ¯ X u i +1 = u i � B − 1 y n ) i n =1 Lawrence Livermore National Laboratory LLNL-PRES-750301 7
Outline § Motivation: parallelizing optimization problems § Parallel-in-time overview: Multigrid reduction in time (MGRIT) § XBraid-adjoint code interface : Open source implementation • Non-intrusively uses existing user code • Example: user-interface for simple scalar ODE problem Lawrence Livermore National Laboratory LLNL-PRES-750301 8
Parallel-in-time overview: Approach leverages spatial multigrid research The Multigrid V-cycle Motivation: Clock speeds § are stagnate but speedup smoothing still possible through more (relaxation) concurrency prolongation à Parallel-in-time is needed (interpolation) Error on the fine grid restriction Error approximated on a smaller coarse grid Lawrence Livermore National Laboratory LLNL-PRES-750301 9
Multigrid reduction in time (MGRIT) § General one-step method for a forward evolution process y n = Φ ( y n − 1 , u i ) , n = 1 , . . . , N § In the linear setting (for simplicity) , sequential marching forward solve ≡ y 0 g 0 I y 1 g 1 I − Φ A y ≡ ≡ g = . . ... ... . . . . y N g N I − Φ § We solve this system iteratively with multigrid reduction (MGR 1979) • Replace sequential O(N) method with O(N) parallel alternative • Coarsens only in time à non-intrusive, i.e. is "arbitrary" Φ Lawrence Livermore National Laboratory LLNL-PRES-750301 10 10
Multigrid reduction in time (MGRIT) § General one-step method for a backward evolution process y n = r y n J n + ( δ y Φ n ) T ¯ y n , ¯ n = N, . . . , 1 § In the linear setting (for simplicity) , sequential marching forward solve ≡ � Φ T y 0 r y 0 J 0 I ¯ � Φ T y 1 r y 1 J 1 I ¯ ¯ A ¯ y ⌘ = . . ... ... . . . . y N r y N J N � Φ T I ¯ § We solve this system iteratively with multigrid reduction (MGR 1979) • Replace sequential O(N) method with O(N) parallel alternative • Coarsens only in time à non-intrusive, i.e. is "arbitrary" Φ Lawrence Livermore National Laboratory LLNL-PRES-750301 11 11
MGRIT for forward solve D T = m d t T 0 T 1 F -point (fine grid only) C -point (coarse & fine grid) t 0 t 1 t 2 t 3 … d t t N F -relaxation § Relaxation is highly parallel • Alternates between F -points and C -points • F -relaxation = block Jacobi on each coarse time interval § Coarse system approximates fine system Approximate impractical with (e.g., time rediscretization with ) • Φ m ∆ T Φ ∆ I I − Φ m I − Φ ∆ I A ∆ = B ∆ = ⇒ ... ... ... ... I − Φ m − Φ ∆ I Apply recursively for multilevel hierarchy • Lawrence Livermore National Laboratory LLNL-PRES-750301 12 12
A broader summary of MGRIT Expose concurrency in the evolution dimension with multigrid I § − Φ I Non-intrusive , with unchanged fine-grid problem § ... ... Converges to same solution as sequential marching § − Φ I Only store C -points to minimize storage § t 0 t 1 t 2 t 3 … d t t N Optimal for variety of parabolic problems § Converges in ~10 iterations for any coarsening factor • Extends to nonlinear problems with FAS formulation § In specialized two-level setting, MGRIT Parareal § ≡ Large speedups available, but in a new way § Time stepping is already O(N) • Useful only beyond a crossover • More time steps à more speedup potential • Example strong scaling of the 2D heat eqn • Much future work to do... § Lawrence Livermore National Laboratory LLNL-PRES-750301 13 13
Outline § Motivation: parallelizing optimization problems § Parallel-in-time overview: Multigrid reduction in time (MGRIT) § XBraid-adjoint code interface : Open source implementation • Non-intrusively uses existing user code • Example: user-interface for simple scalar ODE problem Lawrence Livermore National Laboratory LLNL-PRES-750301 14 14
XBraid-adjoint: open source & non-intrusive u := ( δ J/ δ u ) T § Solve for (reduced gradient of J w.r.t. design u ) y , ¯ y , ¯ § Wrap existing user code to obtain time parallelism Standard XBraid: forward in time solve y n = Φ ( y n − 1 , u ) 1. Step: 2. Clone y m = α y n + β y m 3. Sum: Iteration k of XBraid: 4. SpatialNorm: k y n k y k +1 ← XBraid( y k , u k ) 5. Buf[Un]Pack J ← J ( y k , u k ) 6. Init 7. Free 8. Access Lawrence Livermore National Laboratory LLNL-PRES-750301 15 15
XBraid-adjoint: open source & non-intrusive u := ( δ J/ δ u ) T § Solve for (reduced gradient of J w.r.t. design u ) y , ¯ y , ¯ § Wrap existing user code to obtain time parallelism XBraid-Adjoint J ( y n , u ) 1. ObjectiveT: Iteration k of XBraid-adjoint: y k + 1 ← XBraid adjoint( y k , ¯ y k , u k ) ¯ y n = ( δ y Φ n ) T ¯ y n +1 2. Step_diff: ¯ u ← δ J ( y k , u k ) u +=( δ u Φ n ) T ¯ ¯ y n +1 ¯ δ u y n += r y n J n 3. ObjectiveT_diff: ¯ u += r u J n ¯ Functions 2 and 3 allow XBraid to compute Reduced gradient: ¯ u y n = r y n J n + ( δ y Φ n ) T ¯ y n +1 ¯ N ! ( r u J n + ( δ u Φ n − 1 ) T ¯ X u i +1 = u i � B − 1 y n ) i n =1 Lawrence Livermore National Laboratory LLNL-PRES-750301 16 16
XBraid example: ex-01-adjoint.c § Solve for the reduced gradient (later use in optimization) u := ( δ J/ δ u ) T ¯ Z T § Begin with simple problem: J ( y, λ ) = 1 /T k y k dt 0 s.t. y t = λ y y (0) = 1 § First, user must define objects: App and Vector Lawrence Livermore National Laboratory LLNL-PRES-750301 17 17
XBraid example: ex-01-adjoint.c y n = Φ ( y n − 1 , u ) § Step() : ... Lawrence Livermore National Laboratory LLNL-PRES-750301 18 18
XBraid example: ex-01-adjoint.c y n = ( δ y Φ n ) T ¯ § Step_diff() : y n +1 ¯ u +=( δ u Φ n ) T ¯ y n +1 ¯ ... Lawrence Livermore National Laboratory LLNL-PRES-750301 19 19
XBraid example: ex-01-adjoint.c § ObjectiveT() and ObjectiveT_diff() are similar § Initialize and run XBraid-adjoint: ... u ← ( δ J/ δ u ) T ¯ Lawrence Livermore National Laboratory LLNL-PRES-750301 20 20
XBraid example: ex-01-adjoint.c § Run it! Lawrence Livermore National Laboratory LLNL-PRES-750301 21 21
Recommend
More recommend
