Inverse Design and Automatic Differentiation for Optical Devices Ian - - PowerPoint PPT Presentation

Inverse Design and Automatic Differentiation for Optical Devices

Ian Williamson and Momchil Minkov Stanford University Optical Society Workshop Nov 21, 2019

https://github.com/fancompute/workshop-invdesign

2

Workshop outline

• Brief crash course on optical simulation
• Theory of numerically solving Maxwell’s equations
• Practical demonstration with our open source code
• Inverse design theory / practical introduction
• Adjoint gradient computation
• Automatic differentiation
• Connection to machine learning
• Device parameterization techniques
• Binarizing features
• Optimization penalties

3

A history of optical simulation packages (from our group)

4

maxwellfdfd fdfd_suite

Wonseok Shin Yu (Jerry) Shi

fdfdpy

Ian Williamson

angler ceviche

Tyler Hughes Momchil Minkov Ian Williamson MATLAB Python

gradient-enabled

A quick crash course in

• ptical simulation

5

6

How do we simulate optical devices?

electric field magnetic field relative permittivity this represents our device! current source

... we solve Maxwell’s equations!

7

Stern Group (Northwestern Univ) Wang et al. Nat. Comm. 8, 2098 (2017) Witmer et al. Opt. Express 24, 5876-5885 (2016)

Welcome to the optical device zoo

Kocaman et al. Nat. Pho. 5, 499–505 (2011) Soler Penades et al. Opt. Express 24, 22908-22916 (2016)

Constructing a set of equations to solve

Some assumptions we make:

• A two-dimensional domain (invariant along z-direction)
• All fields are also invariant along z-direction
• Electric-field polarized only along z-direction

8

Constructing a set of equations to solve

9

Where can these equations take us?

10

Interesting / useful

• ptical devices

Optical devices we can analytically model Inverse-designed

• ptical devices

11

The key idea behind finite differences

...

• Very simple concept that can be scaled up to solve

complex PDEs

• Convergence to the exact solution at a rate

proportional to h (depends on “stencil”)

Discretizing Maxwell’s equations

12

continuous PDE discretized PDE Along just one dimension, this looks like... Along two dimensions, the representation is a bit more complex, but the same idea applies!

Discretizing Maxwell’s equations

13

discretized PDE spatial representation flattened representation sparse linear problem:

The ceviche package (at a high level)

14

Notebook #01

15

Inverse design

• Assume that we can “mold” the permittivity
• Try to get the best possible structure for a

particular desired functionality.

16

Piggott, A. et al., Nature Photonics, 374–377 (2015)

Inverse design

• We want the gradient of some objective

function w.r.t. the permittivity at every point

• Efficiently!

17

Piggott, A. et al., Nature Photonics, 374–377 (2015)

Automatic differentiation

18

• r: differentiable programming

Usual programming: library of functions f1(x)

x

y

x

y

fi(x)

x

y

fn(x)

Program = “computational graph” x1

x2

x4

x5

x3

x6

Automatic differentiation

Differentiable programming library: f1(x)

x

y

x

y

fi(x)

x

y

fn(x)

Track a way to compute any dxi/dxj df1 dx

dfi dx

dfn dx

19

x1

x2

x4

x5

x3

x6

Forward/reverse autodiff

20

x1

x2

x3

x4

x4 = f3(f2(f1(x1)))

dx4 dx1 = dx4 dx3 dx3 dx2 dx2 dx1

[N4 × N3]

[N3 × N2]

[N2 × N1]

Forward/reverse autodiff

21

dx4 dx1 = dx4 dx3 dx3 dx2 dx2 dx1

[N4 × N3]

[N3 × N2]

[N2 × N1]

dx4 dx1 = ✓dx4 dx3 dx3 dx2 ◆ dx2 dx1

dx4 dx1 = dx4 dx3 ✓dx3 dx2 dx2 dx1 ◆

O(N1N2N3 + N1N3N4)

O(N2N3N4 + N1N2N4)

22

dx4 dx1 = ✓dx4 dx3 dx3 dx2 ◆ dx2 dx1

dx4 dx1 = dx4 dx3 ✓dx3 dx2 dx2 dx1 ◆

O(N1N2N3 + N1N3N4)

O(N2N3N4 + N1N2N4)

Forward-mode Reverse-mode In optimization problems, this is always the case! Final objective function is just one number. N4 ⌧ N1, N

Reverse-mode autodiff always better when (few outputs, many inputs) x1

x2

x3

x4

Reverse-mode autodiff

23

It’s backpropagation in machine learning lingo!

“AlexNet”, Krizhevsky et al., NIPS (2012)

62 million parameters… one final output!

Reverse-mode autodiff

24

It’s backpropagation in machine learning lingo!

“AlexNet”, Krizhevsky et al., NIPS (2012)

TensorFlow, PyTorch, Autograd et al. are all, in their heart, automatic differentiation libraries

Adjoint variable method

25

In photonics inverse design, it refers to how we “backprop” through the field computation Ae = b

e = A−1b

dL d✏r = dL de de d✏r = dL de d(A−1b) d✏r = dL de dA−1 d✏r b

Adjoint variable method

26

In photonics inverse design, it refers to how we “backprop” through the field computation dL d✏r = dL de dA−1 d✏r b = −dL de A−1 dA d✏r A−1b = = eT

aj

dA d✏r e

“dude, trust me”

e

diagonal tensor

eaj = (AT)−1  −dL de T

Adjoint variable method

27

For a “reciprocal” system (very often), dL d✏r = =

eaj = (AT)−1  −dL de T

= eT

aj

dA d✏r e

AT = A

The adjoint field is a solution to the same Maxwell problem, but with a different source that depends on the objective function and the

• riginal field.

e = A−1b

eaj = A−1baj

Autograd

28

• The autograd package from Harvard (HIPS) is

an autodiff library for NumPy and SciPy

• In many cases, you can just use it blindly, and

then get all the gradients you need!

• You can also extend it by telling autograd how

to differentiate through custom functions github.com/HIPS/autograd

29

Ceviche

In ceviche we’ve interfaced the AVM with autograd This allows great flexibility in building

• ptimization graphs!

Notebook #02

30

31

32

Notebook #03

33

34

The adjoint variable method lets us efficiently compute the gradients in PDE-constrained

• ptimization problems (inverse design)

The backpropagation algorithm is the equivalent of the adjoint method for training neural networks Automatic differentiation is the generalization of both backpropagation and the adjoint method to arbitrary computational graphs (programs) We can utilize automatic differentiation to efficiently

• ptimize complex models that freely combine

physics, machine learning, and other computations

35

Final thoughts (homework?)

• Viewing parameterization as a sequence of

composable transformations is powerful!

• ML libraries can be useful for things other than ML!
• Hyper parameters are important!
• penalty term, binarization, etc.
• Scheduling (adjusting hyper params during

progress of optimization)

• Must go to simulations which capture 3D

features to make real devices

• but doesn’t necessarily mean that you can’t

prototype in 2D!

36

Other resources

37

metanet.stanford.edu Stanford University

• Shanhui Fan
• Jonathan Fan
• Jelena Vuckovic

UC Berkeley

• Eli Yablonovitch

Princeton University

• Alejandro Rodriguez

Technical University of Denmark

• Ole Sigmund

MIT

• Steven Johnson

... and many others!

A collection of resources being setup by Jiaqi Jiang / Jonathan Fan

https://github.com/fancompute/workshop-invdesign

38