Variational Auto-Encoders (VAE) Jonathan Pillow Lecture 21 slides - - PowerPoint PPT Presentation

Variational Auto-Encoders (VAE)

Jonathan Pillow Lecture 21 slides

NEU 560 Spring 2018

VAE Generative Model

N(0, I)

~ z

~ x

latent

N(fθ(~ z), 2I)

fθ

data

~ z ∼ N(0, I)

~ x = fθ(~ z) + ~ ✏

✏ ∼ N(0, 2I)

Recognition

qφ(~ z|~ x) ≈ p(~ z|~ x, ✓)

~ x

= N(gφ1(~ x), Hφ2(~ x))

gφ1

Hφ2

mean cov

qφ(~ z|~ x)

train recognition networks and to minimize

Variational EM: (in theory)

Hφ2

E-step:

KL

• qφ(~

z|~ x)|p(~ z|~ x, ✓)

• <latexit sha1_base64="TKRsiIDf3/gCL0KgE9oS5LBLHC0=">ADnicdZLbhMxFIad4VbCpSks2VhESIkE0UxBgmUFGyRQVSTSVorTyON4MlY8trE9QxJ3oEtL8IOdcGmPAJvgycdITKBI1n+fb7/+B4rzowNw1+t4Nr1Gzdv7dxu37l7/5uZ+/BsZG5JnRIJf6NMaGcibo0DL6anSFGcxpyfx/E3FTwqDZPio10qOs7wTLCEWx9atJ5/u49itkM9j5NkEpZDxUreA5Rsej7TlXDcz94imxKLe7DytufdLrhIFwH3BZRLbqgjqPJXusHmkqSZ1RYwrExoyhUduywtoxwWrZRbqjCZI5ndOSlwBk1Y7c+XQmf+MwUJlL7JixcZ/+ucDgzZpnF3plhm5omq5L/YqPcJq/GjgmVWyrI1UJzqGVsLoqOGWaEsuXmCimd8rJCnWmFh/oe02EvQzkVmGxdQhJorSnblnUVluAqtV6dwZslKVfgloNRZGSUM3bVLQlCelQ4mf30Wl29+a6I/F/tdzqDPq5MSzN1hExcLTwtK4KIJVjVYNcG8BvOy7Z89aj7ytjeH0ThIPrwonvwuv4AO+AReAx6IAIvwQF4C47AEBDwFVyAS/Az+BJ8C74HF1fWoFXPAQbEVz+Bu5D/8c=</latexit><latexit sha1_base64="TKRsiIDf3/gCL0KgE9oS5LBLHC0=">ADnicdZLbhMxFIad4VbCpSks2VhESIkE0UxBgmUFGyRQVSTSVorTyON4MlY8trE9QxJ3oEtL8IOdcGmPAJvgycdITKBI1n+fb7/+B4rzowNw1+t4Nr1Gzdv7dxu37l7/5uZ+/BsZG5JnRIJf6NMaGcibo0DL6anSFGcxpyfx/E3FTwqDZPio10qOs7wTLCEWx9atJ5/u49itkM9j5NkEpZDxUreA5Rsej7TlXDcz94imxKLe7DytufdLrhIFwH3BZRLbqgjqPJXusHmkqSZ1RYwrExoyhUduywtoxwWrZRbqjCZI5ndOSlwBk1Y7c+XQmf+MwUJlL7JixcZ/+ucDgzZpnF3plhm5omq5L/YqPcJq/GjgmVWyrI1UJzqGVsLoqOGWaEsuXmCimd8rJCnWmFh/oe02EvQzkVmGxdQhJorSnblnUVluAqtV6dwZslKVfgloNRZGSUM3bVLQlCelQ4mf30Wl29+a6I/F/tdzqDPq5MSzN1hExcLTwtK4KIJVjVYNcG8BvOy7Z89aj7ytjeH0ThIPrwonvwuv4AO+AReAx6IAIvwQF4C47AEBDwFVyAS/Az+BJ8C74HF1fWoFXPAQbEVz+Bu5D/8c=</latexit><latexit sha1_base64="TKRsiIDf3/gCL0KgE9oS5LBLHC0=">ADnicdZLbhMxFIad4VbCpSks2VhESIkE0UxBgmUFGyRQVSTSVorTyON4MlY8trE9QxJ3oEtL8IOdcGmPAJvgycdITKBI1n+fb7/+B4rzowNw1+t4Nr1Gzdv7dxu37l7/5uZ+/BsZG5JnRIJf6NMaGcibo0DL6anSFGcxpyfx/E3FTwqDZPio10qOs7wTLCEWx9atJ5/u49itkM9j5NkEpZDxUreA5Rsej7TlXDcz94imxKLe7DytufdLrhIFwH3BZRLbqgjqPJXusHmkqSZ1RYwrExoyhUduywtoxwWrZRbqjCZI5ndOSlwBk1Y7c+XQmf+MwUJlL7JixcZ/+ucDgzZpnF3plhm5omq5L/YqPcJq/GjgmVWyrI1UJzqGVsLoqOGWaEsuXmCimd8rJCnWmFh/oe02EvQzkVmGxdQhJorSnblnUVluAqtV6dwZslKVfgloNRZGSUM3bVLQlCelQ4mf30Wl29+a6I/F/tdzqDPq5MSzN1hExcLTwtK4KIJVjVYNcG8BvOy7Z89aj7ytjeH0ThIPrwonvwuv4AO+AReAx6IAIvwQF4C47AEBDwFVyAS/Az+BJ8C74HF1fWoFXPAQbEVz+Bu5D/8c=</latexit><latexit sha1_base64="TKRsiIDf3/gCL0KgE9oS5LBLHC0=">ADnicdZLbhMxFIad4VbCpSks2VhESIkE0UxBgmUFGyRQVSTSVorTyON4MlY8trE9QxJ3oEtL8IOdcGmPAJvgycdITKBI1n+fb7/+B4rzowNw1+t4Nr1Gzdv7dxu37l7/5uZ+/BsZG5JnRIJf6NMaGcibo0DL6anSFGcxpyfx/E3FTwqDZPio10qOs7wTLCEWx9atJ5/u49itkM9j5NkEpZDxUreA5Rsej7TlXDcz94imxKLe7DytufdLrhIFwH3BZRLbqgjqPJXusHmkqSZ1RYwrExoyhUduywtoxwWrZRbqjCZI5ndOSlwBk1Y7c+XQmf+MwUJlL7JixcZ/+ucDgzZpnF3plhm5omq5L/YqPcJq/GjgmVWyrI1UJzqGVsLoqOGWaEsuXmCimd8rJCnWmFh/oe02EvQzkVmGxdQhJorSnblnUVluAqtV6dwZslKVfgloNRZGSUM3bVLQlCelQ4mf30Wl29+a6I/F/tdzqDPq5MSzN1hExcLTwtK4KIJVjVYNcG8BvOy7Z89aj7ytjeH0ThIPrwonvwuv4AO+AReAx6IAIvwQF4C47AEBDwFVyAS/Az+BJ8C74HF1fWoFXPAQbEVz+Bu5D/8c=</latexit>

gφ1

M-step: train generative network to maximize

fθ

Z qφ(~ z|~ x) log p(~ x|~ z)d~ z

≈ 1

k k

X

j=1

log p(~ x|~ zj)

~ zj ∼ qφ(~ z|~ x)

≈ 1

k

X

1 2σ2 ||~

x − fθ(~ zj)||2

M step is fine, but E-step is not. (Not obvious how to minimize KL term for phi1 and phi2) Alternate approach: use “reparametrization trick” and then do gradient descent on variational free energy ∼ qφ(~ z|~ x)

Reparametrization trick: to get: do: ~ ✏j ∼ N(0, I)

~ zj ∼ z }| { N(gφ1(~ x), Hφ2(~ x))

~ zj = (Hφ2(~ x))

1 2 ~

✏ + gφ1(~ x)

VAE objective

log p(X) =

N

X

i=1

log p(~ xi)

≥

N

X

i=1

Z qφ(~ zi|~ xi) log p(~ xi, ~ zi|✓) qφ(~ zi|~ xi)

• d~

zi

=

N

X

i=1

Z qφ(~ zi|~ xi) ✓ log p(~ xi|~ zi, ✓) + log  p(~ zi) qφ(~ zi|~ xi) ◆ d~ zi

≈

N

X

i=1

@1 k

k

X

j=1

log p(~ xi|~ zij, ✓) − KL ⇣ (qφ(~ zi|~ xi)||p(~ zi) ⌘ 1 A

= N(gφ1(~ x), Hφ2(~ x))

N(0, I)

• can use analytic formula

for KL of 2 Gaussians

VAE objective

≈

N

X

i=1

@1 k

k

X

j=1

log p(~ xi|~ zij, ✓) − KL ⇣ (qφ(~ zi|~ xi)||p(~ zi) ⌘ 1 A

= N(gφ1(~ x), Hφ2(~ x))

N(0, I)

NOTE: this does not mean that VAE is trying to make

qφ(~ z|~ x) = p(~ z)

VAE objective

≈

N

X

i=1

@1 k

k

X

j=1

log p(~ xi|~ zij, ✓) − KL ⇣ (qφ(~ zi|~ xi)||p(~ zi) ⌘ 1 A

= N(gφ1(~ x), Hφ2(~ x))

N(0, I)

1 k

N

X

i=1 k

X

j=1

log p(~ xi|~ zij, ✓)

~ zij ∼ qφ(~ z|~ xi)

for = 1

k

X

ij 1 2σ2 ||~

xi − fθ(~ zij)||2

= 1

k

X

ij 1 2σ2 ||~

xi − fθ

• Hφ2(~

xi)

1 2~

✏ij + gφ1(~ xi)

• ||2

N(0, I)

1st term:

VAE objective

= 1

k

X

ij 1 2σ2 ||~

xi − fθ

• Hφ2(~

xi)

1 2~

✏ij + gφ1(~ xi)

• ||2

N(0, I)

This is the key expression! Learning Algorithm:

• 1. sample for i = 1:N, j = 1:k
• 2. compute gradient w.r.t.
• 3. take small step in direction of gradient
• 4. repeat to convergence

θ, φ1, φ2

{✏ij}

Terminology

ELBO = “evidence lower bound” Z q(z|φ) log p(x, z|θ) q(z|φ)

• dz

(Same thing we’ve been calling “neg. free energy”) which is a lower bound on log p(x|θ)

Variational EM = general family of algorithms in which q(z|φ) 6= p(z|x, θ)

so in E step we minimize but can’t set it to zero KL(q(z|φ)||p(z|x, θ))

• often refer to q(z) as “variational approximation” to p(z|x)

Results

[Kingma & Welling 2014]

2D latent z

fθ(z)

Results

[Kingma & Welling 2014]

Results

[Rezende, Mohamed, & Wierstra 2014]

gφ1(~ x)

z space x space

Neuroscience Application

[Gao, Archer, Paninski & Cunningham 2016]

linear dynamical system (LDS)

p(~ z)

fθ

spike trains

~ x|~ z ∼ Poiss(fθ(~ z))

https://www.dropbox.com/s/cluev4fzfsob4q9/video_fLDS.mp4?dl=0 video:

Temporal Latent Variable models

• you make a series of (noisy) measurements
• you’d like to “denoise” them, or figure out what

underlying signal gave rise to them

x1 x2 x3 x4

hidden:

• r “latent” variable

…

Examples:

• x = airplane position+speed, y = radar signal
• x = monkey hand position, y = M1 firing rates
• x = rat location, y = place cell firing rates
• x = spikes, y = calcium fluorescence

y1

measurements:

• r “observations”

… y2 y3 y4

time

• All arrows mean: linear operation + Gaussian noise

forward equation:

• Markov property: xi depends only on xi-1
• observation yi depends only on xi
• linear operations, so everything in sight is Gaussian!*

* So long as initial prior p(x1) is Gaussian

x1 x2 x3 x4

hidden:

• r “latent” variable

… y1

measurements:

• r “observations”

… y2 y3 y4

time

Linear Dynamical System (Kalman Filter)

• bservation equation:

Kalman Filtering vs. Kalman Smoothing

x1 x2 x3 x4

hidden variable:

… y1

• bserved variable:

… y2 y3 y4

Kalman Filtering:

• Recursive, on-line, real-time algorithm
• Use only observations up to current time step
• Example Application: motor prosthetic

Kalman Filtering vs. Kalman Smoothing

x1 x2 x3 x4

hidden variable:

… y1

• bserved variable:

… y2 y3 y4

Kalman Filtering:

• Recursive, on-line, real-time algorithm
• Use only observations up to current time step
• Example Application: motor prosthetic

Kalman Filtering vs. Kalman Smoothing

x1 x2 x3 x4

hidden variable:

… y1

• bserved variable:

… y2 y3 y4

Kalman Filtering:

• Recursive, on-line, real-time algorithm
• Use only observations up to current time step
• Example Application: motor prosthetic

Kalman Filtering vs. Kalman Smoothing

x1 x2 x3 x4

hidden variable:

… y1

• bserved variable:

… y2 y3 y4

Kalman Filtering:

• Recursive, on-line, real-time algorithm
• Use only observations up to current time step
• Example Application: motor prosthetic

Kalman Filtering vs. Kalman Smoothing

x1 x2 x3 x4

hidden variable:

y1

• bserved variable:

y2 y3 y4

Kalman Smoothing:

• Gives posterior over hidden trajectory given all observations
• Off-line, requires forward and backward pass through data
• Application: infer voltage from calcium imaging data

Beyond the Kalman Filter

Basic idea: approximate the (nonlinear) forward updates using a (linear) Taylor series expansion around the posterior mean.

• Extended Kalman Filter (“EKF”) - relaxes

requirement of linear updates

Beyond the Kalman Filter

• Basic idea: propagate certain carefully selected points

through the forward equation; use them to update the mean and covariance of the posterior.

• Generally better than EKF
• Many other approaches: involve tricks for finding the best

Gaussian approximation to the posterior, even when the updates are nonlinear and the noise is non-Gaussian.

• Unscented filter (or “UKF”)

Beyond the Kalman Filter

Alternative, good when posterior is highly non-Gaussian (e.g., multi-modal): Drawbacks:

• computationally intensive
• can be impractical for high-dimensional problems
• Particle filter