Motion & Physics Niloy Mitra Iasonas Kokkinos Paul Guerrero - - PowerPoint PPT Presentation

▶
motion physics
SMART_READER_LITE
LIVE PREVIEW

Motion & Physics Niloy Mitra Iasonas Kokkinos Paul Guerrero - - PowerPoint PPT Presentation

Sep 01, 2023 677 likes •1.04k views

CreativeAI: Deep Learning for Graphics Motion & Physics Niloy Mitra Iasonas Kokkinos Paul Guerrero Nils Thuerey Tobias Ritschel UCL UCL/Facebook UCL TUM UCL Computer Animation Feature detection (image features, point features)

slide-1
SLIDE 1

Niloy Mitra Iasonas Kokkinos Paul Guerrero Nils Thuerey Tobias Ritschel UCL UCL/Facebook UCL TUM UCL

CreativeAI: Deep Learning for Graphics

Motion & Physics

slide-2
SLIDE 2

SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

Computer Animation

2

  • Feature detection (image features, point features)

  • Denoising, Smoothing, etc. 

  • Embedding, Distance computation

  • Rendering

  • Animation

  • Physical simulation

  • Generative models
  • Motion over time
  • Loads of data, expensive
  • Relationships between spatial

and temporal changes

slide-3
SLIDE 3

SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

Character Animation

3

[DeepLoco: Dynamic Locomotion Skills Using Hierarchical Deep Reinforcement Learning, SIGGRAPH 2017] [DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills, SIGGRAPH 2018] [Mode-Adaptive Neural Networks for Quadruped Motion Control, SIGGRAPH 2018] [A Deep Learning Framework for Character Motion Synthesis and Editing, SIGGRAPH 2016]

  • Learn controllers for character rigs
  • Powerful and natural
  • Beyond the scope of this course…
slide-4
SLIDE 4

Physics-Based Animation

  • Leverage physical models
  • Examples:
  • Rigid bodies
  • Cloth
  • Deformable objects
  • Fluids

4 SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

slide-5
SLIDE 5

SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

Physics-Based Animation

5

Experiment Theory Computation

Skip Theory with Deep Learning?

[No! More on that later…]

Observations / data Model equations Discrete representation

slide-6
SLIDE 6

SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

Physics-Based Animation

  • Better goal: support solving suitable physical models
  • Nature = Partial Differential Equations (PDEs)
  • Hence we are aiming for solving PDEs with deep learning (DL)
  • Requirement: “regularity” of the targeted function

6

“Bypass the solving of evolution equations when these equations conceptually exist but are not available or known in closed form.” [Kevrekidis et al.]

slide-7
SLIDE 7

SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

Partial Differential Equations

  • Typical problem formulation: unknown

function

  • PDE of the general form:
  • Solve in domain , with boundary

conditions on boundary

  • Traditionally: discretize & solve numerically.

Here: also discretize, but solve with DL…

7

f ⇣ x1, ..., xn; ∂u ∂x1 , ..., ∂u ∂xn ; ∂2u ∂2x1 , ∂2u ∂x1∂x2 , ... ⌘ = 0 u(x1, ..., xn) Ω Ω Γ Γ

slide-8
SLIDE 8

SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

Methodology 1

  • Viewpoints: holistic or partial

[partial also meaning “coarse graining” or “sub-grid / up-res”]

  • Influences complexity and non-linearity of solution space
  • Trade off computation vs accuracy:
  • Target most costly parts of solving
  • Often at the expense of accuracy

8

slide-9
SLIDE 9

SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

Methodology 2

9

  • Consider dimensionality & structure of discretization
  • Small & unstructured
  • Fully connected NNs only choice
  • Only if necessary…
  • Large & structured
  • Employ convolutional NNs
  • Usually well suited
slide-10
SLIDE 10

SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

Solving PDEs with DL

  • Practical example: airfoil flow
  • Given boundary conditions solve stationary flow problem on grid
  • Fully replace traditional solver
  • 2D data, no time dimension
  • I.e., holistic approach with structured data

10

slide-11
SLIDE 11

SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

Solving PDEs with DL

  • Data generation
  • Large number of pairs: input (BCs) - targets (solutions)

11 Airfoil profile Generated mesh Full simulation domain

Inference region

Different free stream Velocities

slide-12
SLIDE 12

Solving PDEs with DL

  • Data generation
  • Example pair
  • Note - boundary conditions (i.e.

input fields) are typically constant

  • Rasterized airfoil shape present

in all three input fields

12 SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

Target

Pressure Velocity X Velocity Y

128 x 128 x 1 128 x 128 x 1 128 x 128 x 1

Freestream X

Boundary Conditions

Freestream Y Mask

128 x 128 x 1 128 x 128 x 1 128 x 128 x 1

slide-13
SLIDE 13

SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

Solving PDEs with DL

13

Reduce spatial dimensions Skip connections Increase spatial dimensions

  • U-net NN architecture
slide-14
SLIDE 14

SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

Solving PDEs with DL

  • Unet structure highly suitable for PDE solving
  • Makes boundary condition information available throughout
  • Crucial for inference of solution

14

  • U-net NN architecture
slide-15
SLIDE 15

SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

Solving PDEs with DL

  • Training: 80.000 iterations with ADAM optimizer
  • Convolutions with enough data - no dropout necessary
  • Learning rate decay stabilizes models

15

slide-16
SLIDE 16

Results

16

Target (A) Regular data (B) Dimension less Pressure Velocity X Velocity Y

  • Use knowledge about

physics to simplify space of solutions: make quantities dimension- less

  • Significant gains in

inference accuracy

slide-17
SLIDE 17

SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

Solving PDEs with DL

  • Validation and test accuracy for different model sizes

17

Saturated, little gain from weights and data

slide-18
SLIDE 18

Code example

Solving PDEs with DL

18

slide-19
SLIDE 19

SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

Solving PDEs with DL

  • Source code and training data available
  • Requirements: numpy / pytorch , OpenFOAM for data generation
  • Details at: 


https://github.com/thunil/Deep-Flow-Prediction and 
 http://geometry.cs.ucl.ac.uk/creativeai/

19

slide-20
SLIDE 20

SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

Additional Examples

  • Elasticity: material models
  • Fluids: up-res algorithm & dimensionality reduction
  • By no means exhaustive…

20

slide-21
SLIDE 21

Neural Material - Elasticity

  • Learn correction of regular FEM simulation for complex materials

21

[Neural Material: Learning Elastic Constitutive Material and Damping Models from Sparse Data, arXiv 2018]

slide-22
SLIDE 22

Neural Material - Elasticity

  • Learn correction of regular FEM simulation for complex materials
  • “Partial” approach
  • Numerical simulation with flexible NN for material behavior

22

[Neural Material: Learning Elastic Constitutive Material and Damping Models from Sparse Data, arXiv 2018]

slide-23
SLIDE 23

Temporal Data

  • tempoGAN: 3D GAN

with temporal coherence

23

[tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow , SIGGRAPH 2018]

slide-24
SLIDE 24

Temporal Data

  • tempoGAN: 3D GAN

with temporal coherence

24

[tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow , SIGGRAPH 2018]

G

xa

slide-25
SLIDE 25

Temporal Data

25

[tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow , SIGGRAPH 2018]

G D_s

ya

G(xa) G(x ) G(x ) G(x )

xa

  • tempoGAN: 3D GAN

with temporal coherence

slide-26
SLIDE 26

26

[tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow , SIGGRAPH 2018]

G D_s D_t

xa xt-1 xt xt+1 ya yt-1 yt yt+1

G(xa) G(xt-1) G(xt) G(xt+1)

“Loss” for generator

Temporal Data

  • tempoGAN: 3D GAN

with temporal coherence

slide-27
SLIDE 27

27

[tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow , SIGGRAPH 2018]

G D_s D_t

xa xt-1 xt xt+1 ya yt-1 yt yt+1

G(xa) G(xt-1) G(xt) G(xt+1)

Temporal Data

  • tempoGAN: 3D GAN

with temporal coherence

Advection encoded in loss for G

slide-28
SLIDE 28

Temporal Data

28

[tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow , SIGGRAPH 2018]

slide-29
SLIDE 29

Latent Spaces

29

  • Learn flexible reduced representation for physics problems

[Latent-space Physics: Towards Learning the Temporal Evolution of Fluid Flow, arXiv 2018] [Deep Fluids: A Generative Network for Parameterized Fluid Simulations, arXiv 2018]

slide-30
SLIDE 30

Latent Spaces

30

  • Learn flexible reduced representation for physics problems
  • Employ Encoder part (E) of Autoencoder network to reduce dimensions
  • Predict future state in latent space with FC network
  • Use Decoder (D) of Autoencoder to retrieve volume data

[Latent-space Physics: Towards Learning the Temporal Evolution of Fluid Flow, arXiv 2018] [Deep Fluids: A Generative Network for Parameterized Fluid Simulations, arXiv 2018]

E

t t-1 t-2 . . .

E E FC

t+1

D

slide-31
SLIDE 31

Latent Spaces

31

  • Learn flexible reduced representation for physics problems

[Latent-space Physics: Towards Learning the Temporal Evolution of Fluid Flow, arXiv 2018] [Deep Fluids: A Generative Network for Parameterized Fluid Simulations, arXiv 2018]

slide-32
SLIDE 32

SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

  • Checklist for solving PDEs with DL:

✓ Model? (Typically given) ✓ Data? Can enough training data be generated? ✓ Which NN Architecture? ✓ Fine tuning: learning rate, number of layers & features? ✓ Hyper-parameters, activation functions etc.?

Summary

32

slide-33
SLIDE 33

SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

  • Approach PDE solving with DL like solving with traditional

numerical methods:

  • Find closest example in literature
  • Reproduce & test
  • Then vary, adjust, refine …

Summary

33

slide-34
SLIDE 34

SIGGRAPH Asia Course CreativeAI: Deep Learning for Graphics

Thank you!

http://geometry.cs.ucl.ac.uk/creativeai/

34