Predictive Data Science for physical systems From model reduction - PowerPoint PPT Presentation

Predictive Data Science for physical systems From model reduction to scientific machine learning Professor Karen E. Willcox Mathematics of Reduced Order Models | ICERM | 2-20-20 2 1

The Team Funding sources: • US Air Force Computational Math Program (F. Fahroo) Prof. Boris Kramer Michael Kapteyn Dr. Parisa Khodabakhshi • US Air Force Center of MIT UCSD Oden Institute Excellence on Rocket Combustion (M. Birkan, F. Fahroo, R. Munipalli, D. Talley) • US Department of Energy AEOLUS MMICC (S. Lee, W. Spotz) • SUTD-MIT International Design Centre Prof. Benjamin Elizabeth Qian Renee Swischuk Peherstorfer MIT Caliper Courant Institute 2

1 Scientific Machine Learning What, Why & How? Outline 2 Lift & Learn Projection-based model reduction as a lens through which to learn predictive models 3 Conclusions & Outlook 3

Scientific Machine Learning “ Scientific machine learning (SciML) is a core component of artificial intelligence (AI) and a computational technology that can be trained, with scientific data, to augment or automate human skills. Across the Department of Energy (DOE), SciML has the potential to transform science and energy research. Breakthroughs and major progress will be enabled by harnessing DOE investments in massive data from scientific user facilities, software for predictive models and algorithms, high-performance computing platforms, and the national workforce. ” 4

Scientific Machine Learning What role for model reduction? 1 reduce the cost of training 2 foundational shift in ML perspectives Integrate Embed domain heterogeneous, noisy knowledge & incomplete data Get predictions with Respect physical Bring interpretability quantified constraints to results uncertainties 5

Predictive Digital Twin via component-based ROMs and interpretable machine learning ROMs embed predictive modeling and reduce the cost of training Of Offline: fline: Use model library to Construct library of train a classifier that ROMs representing predicts asset state different asset states based on sensor data Online: Online: sensor data Analysis Prediction Optimization updated Digital Twin current Digital Twin [Kapteyn, Knezevic, W. AIAA Scitech 2020]

Machine learning Reduced-order modeling “ The scientific study of algorithms & statistical “Model order reduction (MOR) is a models that computer systems use to perform a technique for reducing the computational specific task without using explicit instructions, complexity of mathematical models in relying on patterns & inference instead .” [Wikipedia] numerical simulations.” [Wikipedia] What is the connection between reduced-order modeling and machine learning? Model reduction methods have grown from Computational Science & Engineering, with focus on reducing high-dimensional models that arise from physics-based modeling, whereas machine learning has grown from Computer Science, with a focus on creating low-dimensional models from black-box data streams. [Swischuk et al., Computers & Fluids , 2019] 7

Machine learning Reduced-order modeling “ The scientific study of algorithms & statistical “Model order reduction (MOR) is a models that computer systems use to perform a technique for reducing the computational specific task without using explicit instructions, complexity of mathematical models in relying on patterns & inference instead .” [Wikipedia] numerical simulations.” [Wikipedia] Reduced-order modeling & machine learning: Can we get the best of both worlds? Discover hidden structure Embed governing equations Non-intrusive implementation Structure-preserving Black-box & flexible Predictive (error estimators) Accessible & available Stability-preserving 8

1 Scientific Machine Learning 2 Lift & Learn Lift & Learn 3 Conclusions & Outlook Projection-based model reduction as a lens through which to learn low-dimensional predictive models 9

P, kPa Lift & Learn: Ingredients T, K Y CH4 1. A physics-based model Typically described by a set of PDEs or ODEs Q, MW/m 3 2. Lens of projection to define a structure-preserving Rocket combustion low-dimensional model Temperature Order parameter 3. Non-intrusive learning of the reduced model 4. Variable transformations that expose polynomial structure in the model → can be exploited with non -intrusive learning 𝑦 Solidification process in additive manufacturing 10

ሶ Start with a physics-based model Example: modeling solidification in additive manufacturing Space/time evolution of temperature 𝑈 and phase parameter 𝜚 with Model based on Kobayashi, 1993; collaboration with Bao & Biros Figure from: https://www.bintoa.com/powder-bed-fusion/ discretized state 𝐲 contains 𝑈 1 Discretize : temperature and phase field ⋮ Spatially discretized order parameter at 𝑈 𝑂 𝑨 𝐲 = 𝐁𝐲 + 𝐂𝐯 + 𝐠(𝐲, 𝐯) 𝐲 = finite element model 𝑂 𝑨 spatial grid points 𝜚 1 ⋮ 𝑂 𝑨 ~𝑃(10 3 − 10 9 ) 𝜚 𝑂 𝑨 11

high-fidelity physics-based simulation reduced-order model = + = + dimension 10 3 − 10 9 dimension 10 1 − 10 3 solution time ~minutes / hours solution time ~seconds / minutes Projection-based model reduction 1 Train : Solve PDEs to generate training data (snapshots) 2 Identify structure : Compute a low-dimensional basis 3 Reduce : Project PDE model onto the low-dimensional subspace 12

ሶ ሶ Projecting a Full-order model (FOM) 𝐲 = 𝐁𝐲 + 𝐂𝐯 state 𝐲 ∈ ℝ 𝑂 linear system Approximate 𝐲 ≈ 𝐖𝐲 𝑠 𝑊 ∈ ℝ 𝑂×𝑠 Residual: 𝑶 eqs ≫ 𝒔 dof 𝐬 = 𝐖 ሶ 𝐲 𝑠 − 𝐁𝐖𝐲 𝑠 − 𝐂𝐯 Project 𝐗 ⊤ 𝐬 = 0 (Galerkin: 𝐗 = 𝐖 ) Reduced-order model 𝐁 𝑠 = 𝐖 ⊤ 𝐁𝐖 (ROM) 𝐲 𝑠 = 𝐁 𝑠 𝐲 𝑠 + 𝐂 𝑠 𝐯 state 𝐲 𝑠 ∈ ℝ 𝑠 𝐂 𝑠 = 𝐖 ⊤ 𝐂 13

ሶ ሶ ሶ ሶ Quadratic Model Linear Model 𝐲 = 𝐁𝐲 + 𝐂𝐯 𝐲 = 𝐁𝐲 + 𝐈 𝐲 ⊗ 𝐲 + 𝐂𝐯 FOM: FOM: 𝐲 𝑠 = 𝐁 𝑠 𝐲 𝑠 + 𝐂 𝑠 𝐯 𝐲 𝑠 = 𝐁 𝑠 𝐲 𝑠 + 𝐈 𝑠 𝐲 𝑠 ⊗ 𝐲 𝑠 + 𝐂 𝑠 𝐯 ROM: ROM: Precompute the ROM matrices: Precompute the ROM matrices and tensor: 𝐈 𝑠 = 𝐖 ⊤ 𝐈(𝐖 ⊗ 𝐖) 𝐁 𝑠 = 𝐖 ⊤ 𝐁𝐖 , 𝐂 𝑠 = 𝐖 ⊤ 𝐂 projection preserves structure ↔ structure embeds physical constraints 14

Operator inference Non-intrusive learning of reduced models from simulation snapshot data

ሶ ሶ ሶ Given reduced state data, 𝐘 ) and derivative data ( ሶ Given reduced state data ( ෡ ෡ 𝐘 ): learn the reduced model | | | | ෡ ෡ 𝐲(𝑢 1 ) ො … 𝐲(𝑢 𝐿 ) ො 𝐘 = 𝐘 = 𝐲(𝑢 1 ) ො … 𝐲(𝑢 𝐿 ) ො | | | | Operator Inference Find the operators ෡ 𝐁, ෡ 𝐂, ෡ 𝐈 using proper orthogonal by solving the least squares problem: decomposition (POD) aka PCA ⊤ ෡ 𝐁 ⊤ + ෡ 𝐈 ⊤ + 𝐕 ⊤ ෡ 𝐂 ⊤ − ሶ 𝐘 ⊤ ෡ ෡ 𝐘 ⊗ ෡ ෡ 𝐘 ⊤ min 𝐘 𝐁,෡ ෡ 𝐂,෡ Peherstorfer & W. 𝐈 Data-driven operator inference for • Generate ෡ 𝐘 data by projection of 𝐘 snapshot data nonintrusive projection-based model reduction, Computer onto POD basis Methods in Applied Mechanics and • If data are Markovian, Operator Inference recovers Engineering , 2016 the intrusive POD reduced model [Peherstorfer, 2019] 16

𝜍𝑣 𝜍 𝜖 + 𝜖 𝜍𝑣 2 + 𝑞 𝜍𝑣 = 0 𝜖𝑢 𝜖𝑦 𝐹 𝐹 + 𝑞 𝑣 𝜍 𝜖𝑣 𝜖𝑦 + 𝑣 𝜖𝜍 𝜖𝑦 𝛿 − 1 + 1 𝑞 𝜍 2 𝜍𝑣 2 𝜖 𝑣 𝜖𝑣 𝜖𝑦 + 1 𝜖𝑞 𝐹 = 𝑣 𝜖𝑣 𝜖𝑦 + 𝑟 𝜖𝑞 𝑣 + = 0 𝜖𝑢 𝜍 𝜖𝑦 𝜖𝑦 𝑞 𝑣 𝛿𝑞 𝜖𝑣 𝜖𝑦 + 𝑣 𝜖𝑞 𝜖 𝛿𝑞 𝜖𝑣 𝜖𝑦 + 𝑣 𝜖𝑞 𝑞 + = 0 𝜖𝑢 𝜖𝑦 𝜖𝑦 𝑟 𝑟 𝜖𝑣 𝜖𝑦 + 𝑣 𝜖𝑟 𝜖𝑦 Variable Transformations & Lifting The physical governing equations reveal variable transformations and manipulations that expose polynomial structure

ሶ ሶ 𝜍𝑥 𝜍 𝜍 𝜖𝑥 𝜖𝑨 + 𝑥 𝜖𝜍 𝜖 + 𝜖 𝜍𝑥 2 + 𝑞 𝜍𝑥 = 0 𝜖𝑨 There are 𝜍 𝜖𝑢 𝜖𝑨 𝜖 𝑥 𝜖𝑥 𝜖𝑨 + 1 𝜖𝑞 𝐹 𝐹 + 𝑞 𝑥 𝑥 + = 0 𝜖𝑢 𝜍 𝜖𝑨 multiple ways 𝑞 𝛿𝑞 𝜖𝑥 𝜖𝑨 + 𝑥 𝜖𝑞 𝛿 − 1 + 1 𝑞 2 𝜍𝑥 2 𝐹 = to write the 𝜖𝑨 conservative variables primitive variables Euler equations mass, momentum, energy mass, velocity, pressure • 1 𝜍 Define specific volume: 𝑟 = Τ 𝜖𝑟 −1 𝜖𝜍 −1 𝜖𝑣 𝜖𝜍 𝜖𝑣 𝜖𝑟 • 𝜖𝑢 = 𝜖𝑢 = 𝜍 2 −𝜍 𝜖𝑨 − 𝑣 = 𝑟 𝜖𝑨 − 𝑣 Take derivative: 𝜍 2 𝜖𝑨 𝜖𝑨 Different choices of variables leads to 𝑥 𝜖𝑥 𝜖𝑨 + 𝑟 𝜖𝑞 𝐲 = 𝐈 𝐲 ⊗ 𝐲 + 𝐂𝐯 𝜖𝑨 different structure in 𝑥 𝜖 𝛿𝑞 𝜖𝑥 𝜖𝑨 + 𝑥 𝜖𝑞 𝑞 transformed system + = 0 the discretized system 𝜖𝑢 𝜖𝑨 has quadratic structure 𝑟 𝑟 𝜖𝑥 𝜖𝑨 + 𝑥 𝜖𝑟 𝜖𝑨 𝐲 𝑠 = 𝐈 𝑠 𝐲 𝑠 ⊗ 𝐲 𝑠 + 𝐂 𝑠 𝐯 specific volume variables ROM has quadratic structure 18