Raising the Reliability of Estimates of Generative Performance of - - PowerPoint PPT Presentation

Raising the Reliability of Estimates of Generative Performance of MRFs

Yuri Burda, Fields Institute Joint work with Roger Grosse and Ruslan Salakhutdinov

Workshop on Big Data and Statistical Machine Learning, Fields Institute, January 28, 2015

Markov Random Fields

• Express relations between random variables

Markov Random Fields

• Express relations between random variables
• Are very powerful

Markov Random Fields

• Express relations between random variables
• Are very powerful
• Can be used as generative models

Markov Random Fields

• Express relations between random variables
• Are very powerful
• Can be used as generative models

Markov Random Fields

• Express relations between random variables
• Are very powerful
• Can be used as generative models

MRFs are powerful

Restricted Boltzmann Machines – example of an MRF

Image visible variables hidden variables

Pair-wise Unary

Generating Samples from RBMs

Run Markov chain (alternating Gibbs Sampling):

Generating Samples from RBMs

Run Markov chain (alternating Gibbs Sampling):

Random

Generating Samples from RBMs

Run Markov chain (alternating Gibbs Sampling):

Random

Generating Samples from RBMs

Run Markov chain (alternating Gibbs Sampling):

Random

Generating Samples from RBMs

Run Markov chain (alternating Gibbs Sampling):

Random V

Generating Samples from RBMs

Run Markov chain (alternating Gibbs Sampling):

Random V

Generating Samples from RBMs

Run Markov chain (alternating Gibbs Sampling):

…

Random V

Generating Samples from RBMs

Run Markov chain (alternating Gibbs Sampling):

…

Random V T= infinity

Equilibrium Distribution

Generating Samples from MRFs

• In general useful MRFs admit tractable MCMC

transition operators

• Running MCMC for many steps one can

generate (approximate) samples from the MRF equilibrium distribution

Model comparison/selection

You can’t judge a model by its samples

Model comparison/selection

You can’t judge a model by its samples

Model comparison/selection

Better to compare validation/test data log-likelihood

Computing log-likelihood for MRFs

Computing log-likelihood for MRFs

Computing log-likelihood for MRFs

Computing log-likelihood for MRFs

It’s hard to compute an exponentially large sum

Computing log-likelihood for MRFs

It’s hard to compute an exponentially large sum But simple enough to approximate

Computing log-likelihood for MRFs

It’s hard to compute an exponentially large sum But simple enough to approximate sum

Computing log-likelihood for MRFs

It’s hard to compute an exponentially large sum But simple enough to approximate sum

Importance sampling

Importance sampling

Importance sampling

Importance sampling

Importance sampling

Importance sampling

Variance of this estimator can be large

Importance sampling

Variance of this estimator can be large

Annealed Importance Sampling

does not depend on size of

Annealed Importance Sampling

does not depend on size of

Annealed Importance Sampling

does not depend on size of

Annealed Importance Sampling

does not depend on size of

Annealed Importance Sampling

does not depend on size of

Annealed Importance Sampling

Annealed Importance Sampling

MRF

Annealed Importance Sampling

MRF

Annealed Importance Sampling

MRF

Annealed Importance Sampling

MRF

Annealed Importance Sampling

— MCMC

• perator for an MRF

with energy

MRF

Annealed Importance Sampling

— MCMC

• perator for an MRF

with energy

MRF

— MCMC

• perator for an MRF

with energy

Annealed Importance Sampling

• AIS reduces the variance of the estimator

dramatically

Annealed Importance Sampling

• AIS reduces the variance of the estimator

dramatically

• However, like all importance samplers, it tends

to underestimate the sum

Annealed Importance Sampling

• AIS reduces the variance of the estimator

dramatically

• However, like all importance samplers, it tends

to underestimate the sum

• Hence it is unreliable for model

comparison/selection

Annealed Importance Sampling

• AIS reduces the variance of the estimator

dramatically

• However, like all importance samplers, it tends

to underestimate the sum

• Hence it is unreliable for model

comparison/selection

Annealed Importance Sampling

• AIS reduces the variance of the estimator

dramatically

• However, like all importance samplers, it tends

to underestimate the sum

• Hence it is unreliable for model

comparison/selection

underestimate

Annealed Importance Sampling

• AIS reduces the variance of the estimator

dramatically

• However, like all importance samplers, it tends

to underestimate the sum

• Hence it is unreliable for model

comparison/selection

underestimate

• verestimate

Importance Sampling underestimates

Importance Sampling underestimates

Importance Sampling underestimates

Importance Sampling underestimates

Jensen’s inequality

Importance Sampling underestimates

Markov’s inequality

Importance Sampling underestimates

Markov’s inequality

Importance Sampling underestimates

The proposal distribution q is likely to miss some modes of f

Importance Sampling underestimates and hence overestimates !

Importance Sampling underestimates and hence overestimates !

Contrast with situation in Directed Graphical Models

Contrast with situation in Directed Graphical Models

Contrast with situation in Directed Graphical Models

Contrast with situation in Directed Graphical Models

Easy to get exact samples

Contrast with situation in Directed Graphical Models

Easy to get exact samples Evaluating still requires evaluating an exponentially large sum

Importance Sampling to the rescue

Importance Sampling to the rescue

Importance Sampling to the rescue

Importance Sampling to the rescue

Importance Sampling to the rescue

Importance Sampling to the rescue

Moreover, the estimator we get is a stochastic lower bound on

Importance Sampling to the rescue

We just have to make sure that the proposal distribution is close to the posterior

Reverse Annealing: a directed graphical model out of an undirected one

Reverse Annealing: a directed graphical model out of an undirected one

Reverse Annealing: a directed graphical model out of an undirected one

— MCMC

• perator for an MRF

with energy — MCMC

• perator for an MRF

with energy

Reverse Annealing: a directed graphical model out of an undirected one

— MCMC

• perator for an MRF

with energy — MCMC

• perator for an MRF

with energy

Reverse Annealing Importance Sampling Estimator — RAISE

• As easy to implement as AIS

Reverse Annealing Importance Sampling Estimator — RAISE

• As easy to implement as AIS
• Gives a model that approximates the original

MRF

Reverse Annealing Importance Sampling Estimator — RAISE

• As easy to implement as AIS
• Gives a model that approximates the original

MRF

• Asymptotically equivalent to the original MRF

Reverse Annealing Importance Sampling Estimator — RAISE

• As easy to implement as AIS
• Gives a model that approximates the original

MRF

• Asymptotically equivalent to the original MRF
• Exact samples can be generated from the

approximate model

Reverse Annealing Importance Sampling Estimator — RAISE

• As easy to implement as AIS
• Gives a model that approximates the original

MRF

• Asymptotically equivalent to the original MRF
• Exact samples can be generated from the

approximate model

• RAISE is a stochastic lower bound on the log-

likelihood of the approximate model

In practice

Evaluated the procedure on

• Small RBMs
• Large RBMs trained with
• Contrastive Divergence with 1 step
• Contrastive Divergence with 25 steps
• Persistent Contrastive divergence
• Large Deep Belief Networks
• Large Deep Boltzmann Machines
• Models of handwritten digits and

models of handwritten characters

Image visible variables hidden variables

Typical Results

MNIST DBN MNIST DBM

Typical Results

MNIST, CD-1 trained RBM, 500 hidden units

Non-typical Results

MNIST CD-1 trained RBM with 20 units

MNIST and Omniglot RBMs Results

• CSL: Conservative Sampling-based Log-likelihood (CSL) estimator
• f Bengio et. al.
• Y. Bengio, L. Yao, and K. Cho. Bounding the test log-

likelihood of generative models.

• RAISE errs on the side of underestimating the log-likelihood.
• The gap is typically very small!

Empirical observations

• Annealing from the data base rates model typically

gives better AIS estimates than annealing from the uniform distribution

Empirical observations

• Annealing from the data base rates model typically

gives better AIS estimates than annealing from the uniform distribution

• RAISE model approximates the original MRF

reasonably well with 1,000 – 100,000 intermediate distributions

Empirical observations

• Annealing from the data base rates model typically

gives better AIS estimates than annealing from the uniform distribution

• RAISE model approximates the original MRF

reasonably well with 1,000 – 100,000 intermediate distributions

• For models that don’t model the data distribution

well (overfitting, undertrained etc) the RAISE model can be substantially better than the original MRF.

Empirical observations

• It’s really hard to know when AIS is or isn’t working,

and RAISE can give a clue about that

Empirical observations

• It’s really hard to know when AIS is or isn’t working,

and RAISE can give a clue about that

• It’s likely that most, but not all, published results

based on AIS estimates with enough intermediate distributions are reliable.

Computational Tricks

RAISE requires estimating a large sum for each test sample, which is computationally expensive

Computational Tricks

RAISE requires estimating a large sum for each test sample, which is computationally expensive Method of control variates gives a way of using few test samples to achieve reasonably reliable estimates

Computational Tricks

RAISE requires estimating a large sum for each test sample, which is computationally expensive Method of control variates gives a way of using few test samples to achieve reasonably reliable estimates RAISE estimates and MRF unnormalized probabilities tend to be tightly correlated

Computational Tricks

RAISE requires estimating a large sum for each test sample, which is computationally expensive Method of control variates gives a way of using few test samples to achieve reasonably reliable estimates RAISE estimates and MRF unnormalized probabilities tend to be tightly correlated is a low-variance estimator of Hence

Computational Tricks

RAISE requires estimating a large sum for each test sample, which is computationally expensive Method of control variates gives a way of using few test samples to achieve reasonably reliable estimates RAISE estimates and MRF unnormalized probabilities tend to be tightly correlated Here are random test set samples and k is small is a low-variance estimator of Hence

Pretraining Very Very Deep Models

• Train an RBM or a DBN

Pretraining Very Very Deep Models

• Train an RBM or a DBN
• Unroll the model using RAISE to create a sigmoid

belief network with 100 or 1000 layers

Pretraining Very Very Deep Models

• Train an RBM or a DBN
• Unroll the model using RAISE to create a sigmoid

belief network with 100 or 1000 layers

• Use p and q to fine-tune the model with an

appropriate algorithm: wake-sleep (Hinton et al, 95) reweighted wake-sleep (Bornschein, Bengio, 14) neural variational inference (Mnih, Gregor, 13)

Pretraining Very Very Deep Models

• Train an RBM or a DBN
• Unroll the model using RAISE to create a sigmoid

belief network with 100 or 1000 layers

• Use p and q to fine-tune the model with an

appropriate algorithm: wake-sleep (Hinton et al, 95) reweighted wake-sleep (Bornschein, Bengio, 14) neural variational inference (Mnih, Gregor, 13)

• Brag about the deepest network that has ever

been trained 

Conclusions

• Comparing MRF models using variants of

importance sampling (AIS, sequential Monte Carlo etc) is unreliable

Conclusions

• Comparing MRF models using variants of

importance sampling (AIS, sequential Monte Carlo etc) is unreliable

• RAISE is as easy to use as AIS, and should be

used either instead or in conjunction with AIS when comparing models

Conclusions

• Comparing MRF models using variants of

importance sampling (AIS, sequential Monte Carlo etc) is unreliable

• RAISE is as easy to use as AIS, and should be

used either instead or in conjunction with AIS when comparing models

• One can (in principle) replace any MRF with a

directed graphical model that has a tractable approximation to the posterior

Thank you

Burda Y., Grosse R., Salakhutdinov R. Accurate and Conservative Estimates of MRF Log-likelihood using Reverse Annealing AISTATS 2015