15-780 Graduate Artificial Intelligence: Probabilistic inference - - PowerPoint PPT Presentation

15-780 – Graduate Artificial Intelligence: Probabilistic inference

• J. Zico Kolter (this lecture) and Nihar Shah

Carnegie Mellon University Spring 2020

1

Outline

Probabilistic graphical models Probabilistic inference Exact inference Sample-based inference A brief look at deep generative models

2

Outline

Probabilistic graphical models Probabilistic inference Exact inference Sample-based inference A brief look at deep generative models

3

Probabilistic graphical models

Probabilistic graphical models are all about representing distributions 𝑞 𝑌 where 𝑌 represents some large set of random variables Example: suppose 𝑌 ∈ 0,1 푛 (𝑜-dimensional random variable), would take 2푛 − 1 parameters to describe the full joint distribution Graphical models offer a way to represent these same distributions more compactly, by exploiting conditional independencies in the distribution Note: I’m going to use “probabilistic graphical model” and “Bayesian network” interchangeably, even though there are differences

4

Bayesian networks

A Bayesian network is defined by

• 1. A directed acyclic graph, 𝐻 = {𝑊 =

𝑌1, … , 𝑌푛 , 𝐹}

• 2. A set of conditional distributions 𝑞 𝑌푖 Parents 𝑌푖

Defines the joint probability distribution 𝑞 𝑌 = ∏

푖=1 푛

𝑞 𝑌푖 Parents 𝑌푖 Equivalently: each node is conditionally independent of all non-descendants given its parents

5

Example Bayesian network

Conditional independencies let us simply the joint distribution: 𝑞 𝑌1, 𝑌2, 𝑌3, 𝑌4 = 𝑞 𝑌1 𝑞 𝑌2 𝑌1 𝑞 𝑌3 𝑌1, 𝑌2 𝑞 𝑌4 𝑌1, 𝑌2, 𝑌3 = 𝑞 𝑌1 𝑞 𝑌2 𝑌1)𝑞 𝑌3 𝑌2 𝑞 𝑌4 𝑌3

6

X1 X2 X3 X4

24 − 1 = 15 parameters (assuming binary variables)

Example Bayesian network

Conditional independencies let us simply the joint distribution: 𝑞 𝑌1, 𝑌2, 𝑌3, 𝑌4 = 𝑞 𝑌1 𝑞 𝑌2 𝑌1 𝑞 𝑌3 𝑌1, 𝑌2 𝑞 𝑌4 𝑌1, 𝑌2, 𝑌3 = 𝑞 𝑌1 𝑞 𝑌2 𝑌1)𝑞 𝑌3 𝑌2 𝑞 𝑌4 𝑌3

7

X1 X2 X3 X4

24 − 1 = 15 parameters (assuming binary variables) 1 parameter 2 parameters

Example Bayesian network

Conditional independencies let us simply the joint distribution: 𝑞 𝑌1, 𝑌2, 𝑌3, 𝑌4 = 𝑞 𝑌1 𝑞 𝑌2 𝑌1 𝑞 𝑌3 𝑌1, 𝑌2 𝑞 𝑌4 𝑌1, 𝑌2, 𝑌3 = 𝑞 𝑌1 𝑞 𝑌2 𝑌1)𝑞 𝑌3 𝑌2 𝑞 𝑌4 𝑌3

8

X1 X2 X3 X4

24 − 1 = 15 parameters (assuming binary variables) 7 parameters

Poll: Simple Bayesian network

What conditional independencies exist in the following Bayesian network? 1. 𝑌1 and 𝑌2 are marginally independent 2. 𝑌4 is conditionally independent of 𝑌1 given 𝑌3 3. 𝑌1 is conditionally independent of 𝑌4 given 𝑌3 4. 𝑌1 is conditionally independent of 𝑌2 given 𝑌3

9

X1 X2 X3 X4

Generative model

Can also describe the probabilistic distribution as a sequential “story”, this is called a generative model 𝑌1 ∼ Bernoulli 𝜚 1 𝑌2| 𝑌1 = 𝑦1 ∼ Bernoulli 𝜚푥1

2

𝑌3| 𝑌2 = 𝑦2 ∼ Bernoulli 𝜚푥2

3

𝑌4| 𝑌3 = 𝑦3 ∼ Bernoulli 𝜚푥3

3

“First sample 𝑌1 from a Bernoulli distribution with parameter 𝜚 1 , then sample 𝑌2 from a Bernoulli distribution with parameter 𝜚푥1

2 , where 𝑦1 is the value we sampled for 𝑌1, then

sample 𝑌3 from a Bernoulli …”

10

X1 X2 X3 X4

More general generative models

This notion of a “sequential story” (generative model) is extremely powerful for describing very general distributions Naive Bayes: 𝑍 ∼ Bernoulli 𝜚 𝑌푖|𝑍 = 𝑧 ∼ Categorical 𝜚푦

푖

Gaussian mixture model: 𝑎 ∼ Categorical 𝜚 𝑌|𝑎 = 𝑨 ∼ 𝒪 𝜈푧, Σ푧

11

More general generative models

Linear regression: 𝑍 |𝑌 = 𝑦 ∼ 𝒪 𝜄𝑈 𝑦, 𝜏2 Changepoint model: 𝑌 ∼ Uniform 0,1 𝑍 |𝑌 = 𝑦 ∼ {𝒪 𝜈1, 𝜏2 if 𝑦 < 𝑢 𝒪 𝜈2, 𝜏2 if 𝑦 ≥ 𝑢 Latent Dirichlet Allocation: 𝑁 documents, 𝐿 topics, 𝑂𝑗 words/document 𝜄𝑗 ∼ Dirichlet 𝛽 (topic distributions per document) 𝜚𝑙 ∼ Dirichlet 𝛾 (word distributions per topic) 𝑨𝑗,𝑘 ∼ Categorical 𝜄𝑗 (topic of 𝑘th word in document 𝑗) 𝑥𝑗,𝑘 ∼ Categorical 𝜚𝑨𝑗,𝑘 (𝑘th word in document 𝑗)

12

Outline

Probabilistic graphical models Probabilistic inference Exact inference Sample-based inference A brief look at deep generative models

13

The inference problem

Given observations (i.e., knowing the value of some of the variables in a model), what is the distribution over the other (hidden) variables? A relatively “easy” problem if we observe variables at the “beginning” of chains in a Bayesian network:

• If we observe the value of 𝑌1, then 𝑌2, 𝑌3, 𝑌4 have the same distribution

as before, just with 𝑌1 “fixed”

• But if we observe 𝑌4 what is the distribution over 𝑌1, 𝑌2, 𝑌3?

14

X1 X2 X3 X4 X1 X2 X3 X4

Many types of inference problems

Marginal inference: given a generative distribution for 𝑞 X over 𝑌 = {𝑌1, … , 𝑌푛}, determine 𝑞(𝑌ℐ) for ℐ ⊆ {1, … , 𝑜} MAP inference: determine assignment with the maximum probability Conditional variants: solve either of the two variants conditioned on some

• bservable variables, e.g.

𝑞(𝑌ℐ|𝑌ℰ = 𝑦ℰ)

15

Approaches to inference

There are three categories of common approaches to inference (more exist, but these are most common)

• 1. Exact methods: Bayes’ rule or variable elimination methods
• 2. Sampling approaches: draw samples from the the distribution over hidden

variables, without construction them explicitly

• 3. Approximate variational approaches: approximate distributions over hidden

variables using “simple” distributions, minimizing the difference between these distributions and the true distributions

16

Outline

Probabilistic graphical models Probabilistic inference Exact inference Sample-based inference A brief look at deep generative models

17

Exact inference example

Mixture of Gaussians model: 𝑎 ∼ Categorical 𝜚 𝑌|𝑎 = 𝑨 ∼ 𝒪 𝜈푧, Σ푧 Task: compute 𝑞(𝑎|𝑦) In this case, we can solve inference exactly with Bayes’ rule: 𝑞 𝑎 𝑦 = 𝑞 𝑦 𝑎 𝑞 𝑎 ∑푧 𝑞 𝑦 𝑨 𝑞 𝑨

18

Z X

Exact inference in graphical models

In some cases, it’s possible to exploit the structure of the graphical model to develop efficient exact inference methods Example: how can I compute 𝑞(𝑌4)? 𝑞 𝑌4 = ∑

푥1,푥2,푥3

𝑄 𝑦1 𝑄 𝑦2 𝑦1 𝑄 𝑦3 𝑦2 𝑄 𝑌4 𝑦3

19

X1 X2 X3 X4

Need for approximate inference

In most cases, the exact distribution over hidden variables cannot be computed, would require representing an exponentially large distribution over hidden variables (or infinite, in continuous case) 𝑎푖 ∼ Bernoulli 𝜚푖 , 𝑗 = 1, … , 𝑜 𝑌|𝑎 = 𝑨 ∼ 𝒪 𝜄푇 𝑨, 𝜏2 Distribution 𝑄 (𝑎|𝑦) is a full distribution over 𝑜 binary random variables

20

Z1 X Z2 Zn · · ·

Outline

Probabilistic graphical models Probabilistic inference Exact inference Sample-based inference A brief look at deep generative models

21

Sample-based inference

If we can draw samples from a posterior distribution, then we can approximate arbitrary probabilistic queries about that distribution A naive strategy (rejection sampling): draw samples from the generative model until we find one that matches the observed data, distribution over other variables will be samples of the hidden variables given observed variables As we get more complex models, and more observed variables, probability that we see our exact observations goes to zero

22

X1 X2 X3 X4

Markov Chain Monte Carlo

Let’s consider a generic technique for generating samples from a distribution 𝑞 𝑌 (suppose distribution is complex so that we cannot directly compute or sample) Our strategy is going to be to generate samples 𝑌푡 via some conditional distribution 𝑞(𝑌푡+1|𝑌푡), constructed to guarantee that 𝑞 𝑌푡 → 𝑞(𝑌)

23

Metropolis-Hastings Algorithm

One of the workhorses of modern probabilistic methods

• 1. Pick some 𝑦0 (e.g., completely randomly)
• 2. For 𝑢 = 1,2, …

Sample: ̃ 𝑦푡+1 ∼ 𝑟 𝑌′ 𝑌 = 𝑦푡 Set: 𝑦푡+1 ≔ ̃ 𝑦푡+1 𝑥. 𝑞. min 1, 𝑞 ̃ 𝑦푡+1 𝑟 𝑦푡 ̃ 𝑦푡+1 𝑞 𝑦푡 𝑟 ̃ 𝑦푡+1 𝑦푡 𝑦푡

• therwise

24

Notes on MH

We choose 𝑟(𝑌′|𝑌) so that we can easily sample from the distribution (e.g., for continuous distributions, it’s common to choose) 𝑟 𝑌′ 𝑌 = 𝑦 = 𝒪 𝑦′ 𝑦; 𝐽 Note that even if we cannot compute the probabilities 𝑞(𝑦푡) and 𝑞( ̃ 𝑦푡+1) we can

• ften compute their ratio 𝑞( ̃

𝑦푡+1)/𝑞(𝑦푡) (requires only being able to compute the unnormalized probabilities), e.g., consider the case

25

X1 X2 X3 X4

Proof of MH algorithm

Theorem: For samples generated by MH, 𝑞(𝑌푡) → 𝑞 𝑌 as 𝑢 → ∞ Proof: We’ll proceed in two parts.

• 1. (Detailed balance equations) First, we show that given any distribution

𝑞(𝑌) and a conditional distribution 𝑞 𝑌′ 𝑌 , then if 𝑞 𝑌 𝑞 𝑌′ 𝑌 = 𝑞 𝑌′ 𝑞 𝑌 𝑌′ and if 𝑞(𝑌′|𝑌) > 0, ∀𝑦, 𝑦′ then repeatedly sampling 𝑦푡+1 ∼ 𝑞 𝑌′ 𝑌 = 𝑦푡 gives 𝑞 𝑌푡 → 𝑞 𝑌

• 2. The Metropolis-Hastings update gives a distribution that satisfies the

detailed balance equations

26

Proof of MH algorithm (cont)

Part 1: (not a complete proof), detailed balance says that for 𝑦푡, 𝑦푡+1 𝑞 𝑦푡 𝑞 𝑦푡+1 𝑦푡 = 𝑞 𝑦푡+1 𝑞 𝑦푡 𝑦푡+1 Summing both sizes over over 𝑦푡 gives ∑

푥푡

𝑞 𝑦푡 𝑞 𝑦푡+1 𝑦푡 = 𝑞 𝑦푡+1 which is equivalent to the fact that 𝑞(𝑌) is a stationary distribution of the conditional distribution 𝑞(𝑌′|𝑌) Under some properties of conditional distributions that we won’t cover, repeated sampling from the conditional will converge to the stationary distribution, assuming e.g. conditional has positive probabilities

27

Proof of MH algorithm (cont)

Part 2: First, note that detailed balance is trivially satisfied for 𝑦푡+1 = 𝑦푡 𝑞 𝑦푡+1 𝑦푡 𝑞 𝑦푡 = 𝑞 𝑦푡 𝑦푡+1)𝑞 𝑦푡+1 Now assuming 𝑦푡+1 ≠ 𝑦푡, suppose that (opposite case proceeds in exactly the same manner) 𝑞 𝑦푡 𝑟 𝑦푡+1 𝑦푡 ≤ 𝑞 𝑦푡+1 𝑟(𝑦푡|𝑦푡+1) Then: min 1, 𝑞 𝑦푡+1 𝑟 𝑦푡 𝑦푡+1) 𝑞 𝑦푡 𝑟 𝑦푡+1 𝑦푡) = 1 min 1, 𝑞 𝑦푡 𝑟 𝑦푡+1 𝑦푡) 𝑞 𝑦푡+1 𝑟 𝑦푡 𝑦푡+1) 𝑞 𝑦푡+1 𝑟 𝑦푡 𝑦푡+1) = 𝑞 𝑦푡 𝑟 𝑦푡+1 𝑦푡)

28

Proof of MH algorithm (cont)

So finally, note that 𝑞 𝑦푡 𝑞 𝑦푡+1 𝑦푡 = 𝑞 𝑦푡 𝑟 𝑦푡+1 𝑦푡 min 1, 𝑞 𝑦푡+1 𝑟 𝑦푡 𝑦푡+1) 𝑞 𝑦푡 𝑟 𝑦푡+1 𝑦푡) = 𝑞 𝑦푡 𝑟 𝑦푡+1 𝑦푡 = min 1, 𝑞 𝑦푡 𝑟 𝑦푡+1 𝑦푡) 𝑞 𝑦푡+1 𝑟 𝑦푡 𝑦푡+1) 𝑞 𝑦푡+1 𝑟 𝑦푡 𝑦푡+1) = 𝑞 𝑦푡+1 𝑞 𝑦푡 𝑦푡+1) (first and last lines follow by definition of 𝑞(𝑦푡+1|𝑦푡) and 𝑞(𝑦푡|𝑦푡+1), second and third by equations on the previous slide) Which shows the transition probabilities satisfy detailed balance. ∎

29

Poll: Metropolis-Hastings

Given the following true distributions 𝑞 and sampling distributions 𝑟 would result in creating accurate samples from the true distribution?

• 1. 𝑞 𝑦 = 𝒪 0,1 , 𝑟 𝑦′ = 𝑉 0,1
• 2. 𝑞 𝑦 = 𝑉 0,1 , 𝑟 𝑦′ = 𝒪(0,1)
• 3. 𝑞 𝑦 = 𝒪 0,1 , 𝑦′|𝑦 = 𝑦 + 𝑉 −1,1

30

Gibbs sampling

An application of MH to graphical models leads to what is called Gibbs sampling Suppose we want to draw a sample from 𝑞(𝑎|𝑌 = 𝑦) (i.e., sample over unobserved variables given observed variables)

• 1. Initialize 𝑨 randomly
• 2. Repeat: pick some 𝑗 and sample

𝑨푖 ∼ 𝑄 (𝑎푖|𝑎¬푖 = 𝑨¬푖, 𝑌 = 𝑦) Practical to implement as long as we can sample from a variable given a fixed value of all other variables (can exploit independence structure)

31

Gibbs as Metropolis-Hastings

We can derive Gibbs sampling as an application of the MH algorithm, with the proposal distribution (omitting 𝑌 terms for simplicity) 𝑟푖 𝑎′ 𝑎 = { 𝑨푖

′ ∼ 𝑄 (𝑎푖|𝑎¬푖 = 𝑨¬푖)

𝑨푗

′ = 𝑨푗

Under this distribution, proposal is always accepted: 𝑞 𝑨′ 𝑟푖 𝑨 𝑨′ 𝑞 𝑨 𝑟푖 𝑨′ 𝑨 = 𝑞 𝑨푖

′|𝑨¬푖 ′

𝑞 𝑨¬푖

′

𝑞(𝑨푖|𝑨¬푖

′ )

𝑞 𝑨푖 𝑨¬푖 𝑞 𝑨¬푖 𝑞(𝑨푖

′|𝑨¬푖) = 𝑞 𝑨푖 ′|𝑨¬푖 ′

𝑞 𝑨¬푖

′

𝑞(𝑨푖|𝑨¬푖

′ )

𝑞 𝑨푖 𝑨¬푖

′

𝑞 𝑨¬푖 𝑞(𝑨푖

′|𝑨¬푖 ′ ) = 1

Technically, this uses a different 𝑟푖 selected at random for each 𝑎푖 variable, but we can show that the product of all these individual 𝑟푖’s lead to a single “global” 𝑟 that still has all the necessary properties

32

Outline

Probabilistic graphical models Probabilistic inference Exact inference Sample-based inference A brief look at deep generative models

33

Deep generative models

Probabilistic models + deep learning (what could be better?) A huge landscape, going back many years, and we will just briefly highlight two common current approaches

• Variational autoencoders
• Generative adversarial networks
• See also (not discussed): normalizing flow models

34

Generator networks

Starting point for most deep generative models is a “generator network” 𝐻, a generative model that takes random noise inputs and outputs elements from the desired distribution, e.g. 𝑨 ∼ 𝒪 0, 𝐽 𝑦 ∼ 𝒪(𝐻 𝑨; 𝜄 , 𝜏2𝐽) For these models, how do we train the network via e.g. maximum likelihood estimation? maximize

휃

∑

푖=1 푚

log 𝑞 𝑦 푖 ; 𝜄 ≡ maximize

휃

∑

푖=1 푚

log ∫

푧

𝑞 𝑦 푖 |𝑨; 𝜄 𝑞 𝑨 𝑒𝑨 Typically hard to optimize via “standard” approaches (e.g., sampling + MLE), so alternative approaches are needed

35

Variational autoencoders

Variational autoencoders (VAEs) approximate the MLE using the so-called variational lower bound: for any distribution 𝑟(𝑎|𝑌) we have log 𝑞 𝑦 푖 ≥ 𝐹푧∼푞 푍|푥 푖 log 𝑞 𝑦 푖 𝑨 − 𝐿𝑀[𝑟 𝑎|𝑦 푖 ||𝑞(𝑎)] where 𝐿𝑀 𝑞||𝑟 = ∫ 𝑞 𝑦 log 푝 푥

푞 푥 𝑒𝑦 is called the KL-divergence between two

distributions In general, finding the “right” distribution 𝑟(𝑎|𝑦) is the goal of what are called variational inference methods Key idea of variational autoencoders: use a neural network to predict the 𝑟(𝑎|𝑦) distribution

36

VAEs continued

VAEs compute two networks, the “encoder” network that predicts 𝑟(𝑎|𝑦) and a “decoder” (generator) network that models 𝑞(𝑌|𝑨) Train VAEs by maximizing the variational lower bound maximize

퐸,퐺

∑

푖=1 푚

𝐅푧∼푞 푍|푥 푖 [log 𝑞(𝑦 푖 |𝑨)] − 𝐿𝑀(𝑟(𝑎|𝑦(푖))| 𝑞 𝑎 (some tricks required to make this a differentiable process)

37

x E q(Z|x) z G p(X|z)

Generative adversarial models (GANs)

An alternative approach to training deep generative: try to build a classifier that can “tell apart” generated samples from real data minimize

𝐻

maximize

𝐸

1 𝑛 ∑

𝑗=1 𝑛

log 𝑞(𝑦 𝑗 ; 𝐸) + 𝐅𝑦∼𝑞(𝑦,𝑨;𝐻)[log(1 − 𝑞(𝑦 ; 𝐸))] Training requires solving a min-max optimization problem but current results suggest that it can generate very realistic samples “Avoids” challenges associated with MLE by not trying to approximate it at all, but considering a different loss function

38

Examples of GANs

Samples of faces generated by (quite complex) GAN

39

Figure from (Kerras et al., 2018)