CS786 Lecture 15: May 21, 2012 MAP inference [KF Chapter 13] CS786 P. - - PDF document

26/06/2012 1

CS786 Lecture 15: May 21, 2012

MAP inference [KF Chapter 13]

CS786 P. Poupart 2012 1

MAP Queries

• Recall: MAP stands for maximum a posteriori
• MAP queries:

– find best assignment of all non‐evidence variables – Pr |

• Marginal MAP queries

– Find best assignment of a subset of non‐evidence variables – Pr ∑ Pr , |

• CS786 P. Poupart 2012

2

26/06/2012 2

Applications

• Speech recognition

– Find best sequence of words given audio signal – Pr |

• Image processing

– Find best pixel labeling given image (pixel intensities) – Pr |

• Medical diagnosis

– Find best diagnosis given symptoms – Pr |

CS786 P. Poupart 2012 3

MAP Inference Techniques

• Variable elimination
• Message passing
• Optimization
• Graph cut

CS786 P. Poupart 2012 4

26/06/2012 3

Comparison

• Inference queries:

Pr ∑ Pr ,

• ∑ ∏

,

• Operations: sum‐product
• MAP queries:

max

Pr max

• ∏
• Operations: max‐product
• Idea: use same algorithms, but eliminate variables by

maximization instead of summation

CS786 P. Poupart 2012 5

Max‐product Variable Elimination

• Same as variable elimination, but variables are

eliminated by maximization

1. Restrict factors based on evidence 2. For each non‐evidence variable

do

a. Multiply factors that contain

:

∏

• ∈

a. Maxout

:

\

max

• Notes:

– no need for normalization – All non‐evidence variables are eliminated

CS786 P. Poupart 2012 6

26/06/2012 4

7

CS 786 Lecture Slides (c) 2012, P. Poupart

Maxout Operation

• Let , be a potential with var. ( is a set)
• We “maxout” from to produce a new potential h

which is defined: max∈ ,

f(A,B) h(B)=maxA f(A,B)

ab 0.9 b 0.9 a~b 0.1 ~b 0.6 ~ab 0.4 ~a~b 0.6

8

CS 786 Lecture Slides (c) 2012, P. Poupart

Maximizing values

• Since most MAP queries are not really interested in

the maximum probability, but rather the assignment

• f values to variables that maximize the probability,

store the maximizing values in the factors

f(A,B) h(B)=maxA f(A,B)

ab 0.9 b 0.9 (A=a) a~b 0.1 ~b 0.6 (A=~a) ~ab 0.4 ~a~b 0.6

26/06/2012 5

Max‐Product VE Example

CS786 P. Poupart 2012 9

Marginal MAP Queries

• MAP queries:

max

Pr max

• ∏
• Operations: max‐product

Complexity: NP

• Marginal MAP queries:

max

Pr max

• ∑ ∏

,

• Operations: max‐sum‐product

Complexity: NPPP

CS786 P. Poupart 2012 10

26/06/2012 6

Variable Elimination for Marginal MAP

• Eliminate all variables

– Eliminate the query variables by maximization – Eliminate the hidden variables by summation

• Elimination order:

– Exact inference: eliminate hidden variables before query variables – Approximate inference: eliminate variables in any order

CS786 P. Poupart 2012 11