SLIDE 1
Rao-Blackwellized Stochastic Gradients for Discrete Distributions
Runjing (Bryan) Liu June 11, 2019
University of California, Berkeley
Objective
- We fit a discrete latent variable model.
Rao-Blackwellized Stochastic Gradients for Discrete Distributions - - PowerPoint PPT Presentation
Rao-Blackwellized Stochastic Gradients for Discrete Distributions - - PowerPoint PPT Presentation
Rao-Blackwellized Stochastic Gradients for Discrete Distributions Runjing (Bryan) Liu June 11, 2019 University of California, Berkeley Objective We fit a discrete latent variable model . Objective We fit a discrete latent variable
SLIDE 2
SLIDE 3
Objective
- We fit a discrete latent variable model.
- Fitting such a model involves finding
argmin
η
Eqη(z) [fη(z)] where z is a discrete random variable with K categories.
SLIDE 4
Objective
- We fit a discrete latent variable model.
- Fitting such a model involves finding
argmin
η
Eqη(z) [fη(z)] where z is a discrete random variable with K categories.
- Two common approaches are :
SLIDE 5
Objective
- We fit a discrete latent variable model.
- Fitting such a model involves finding
argmin
η
Eqη(z) [fη(z)] where z is a discrete random variable with K categories.
- Two common approaches are :
- 1. Analytically integrate out z.
SLIDE 6
Objective
- We fit a discrete latent variable model.
- Fitting such a model involves finding
argmin
η
Eqη(z) [fη(z)] where z is a discrete random variable with K categories.
- Two common approaches are :
- 1. Analytically integrate out z.
Problem: K might be large.
SLIDE 7
Objective
- We fit a discrete latent variable model.
- Fitting such a model involves finding
argmin
η
Eqη(z) [fη(z)] where z is a discrete random variable with K categories.
- Two common approaches are :
- 1. Analytically integrate out z.
Problem: K might be large.
- 2. Sample z ∼ qη(z), and estimate the gradient with g(z).
SLIDE 8
Objective
- We fit a discrete latent variable model.
- Fitting such a model involves finding
argmin
η
Eqη(z) [fη(z)] where z is a discrete random variable with K categories.
- Two common approaches are :
- 1. Analytically integrate out z.
Problem: K might be large.
- 2. Sample z ∼ qη(z), and estimate the gradient with g(z).
Problem: g(z) might have high variance.
SLIDE 9
Objective
- We fit a discrete latent variable model.
- Fitting such a model involves finding
argmin
η
Eqη(z) [fη(z)] where z is a discrete random variable with K categories.
- Two common approaches are :
- 1. Analytically integrate out z.
Problem: K might be large.
- 2. Sample z ∼ qη(z), and estimate the gradient with g(z).
Problem: g(z) might have high variance.
We propose a method that uses a combination of these two approaches to reduce the variance of any gradient estimator g(z).
SLIDE 10
Our method
Suppose g is an unbiased estimate of the gradient, so ∇ηL(η) = Eqη(z)[g(z)] =
K
- k=1
qη(k)g(k)
SLIDE 11
Our method
Suppose g is an unbiased estimate of the gradient, so ∇ηL(η) = Eqη(z)[g(z)] =
K
- k=1
qη(k)g(k) Key observation: In many applications (e.g. variational Bayes), qη(z) is concentrated on only a few categories.
SLIDE 12
Our method
Suppose g is an unbiased estimate of the gradient, so ∇ηL(η) = Eqη(z)[g(z)] =
K
- k=1
qη(k)g(k) Key observation: In many applications (e.g. variational Bayes), qη(z) is concentrated on only a few categories. Our idea: Let us analytically sum categories where qη(z) has high probability, and sample the remaining terms.
SLIDE 13
Our method
Suppose g is an unbiased estimate of the gradient, so ∇ηL(η) = Eqη(z)[g(z)] =
K
- k=1
qη(k)g(k) Key observation: In many applications (e.g. variational Bayes), qη(z) is concentrated on only a few categories. Our idea: Let us analytically sum categories where qη(z) has high probability, and sample the remaining terms.
SLIDE 14
Our method
In math,
K
- k=1
qη(k)g(k) =
- z∈Cα
qη(z)g(z)
- analytically sum
+ (1 − qη(Cα))
- small
Eqη(z)[g(z)|z / ∈ Cα]
- estimate by sampling
SLIDE 15
Our method
In math,
K
- k=1
qη(k)g(k) =
- z∈Cα
qη(z)g(z)
- analytically sum
+ (1 − qη(Cα))
- small
Eqη(z)[g(z)|z / ∈ Cα]
- estimate by sampling
The variance reduction is guaranteed by representing our estimator as an instance of Rao-Blackwellization.
SLIDE 16
Results: Generative semi-supervised classification
We train a classifier to classify the class label of MNIST digits and learn a generative model for MNIST digits conditional on the class label.
SLIDE 17
Results: Generative semi-supervised classification
We train a classifier to classify the class label of MNIST digits and learn a generative model for MNIST digits conditional on the class label. Our objective is to maximize the evidence lower bound (ELBO), pη(x) ≥ Eqη(z)[log pη(x, z) − log qη(z)] In this problem, the class label z has ten discrete categories.
SLIDE 18
Results: Generative semi-supervised classification
SLIDE 19
Results: Generative semi-supervised classification
SLIDE 20
Results: moving MNIST
We train a generative model for non-centered MNIST digits.
SLIDE 21
Results: moving MNIST
We train a generative model for non-centered MNIST digits. To do so, we must first learn the location of the MNIST digit. There are 68 × 68 discrete categories.
SLIDE 22
Results: moving MNIST
We train a generative model for non-centered MNIST digits. To do so, we must first learn the location of the MNIST digit. There are 68 × 68 discrete categories. Thus, computing the exact sum is intractable!
SLIDE 23
Results: moving MNIST
Trajectory of the negative ELBO Reconstruction of MNIST digits
SLIDE 24