CSEP 573: Artificial Intelligence Spring 2014 Hidden Markov Models - PowerPoint PPT Presentation

CSEP 573: Artificial Intelligence Spring 2014 Hidden Markov Models & Exact Inference Ali Farhadi Many slides adapted from Dan Weld, Pieter Abbeel, Dan Klein, Stuart Russell, Andrew Moore & Luke Zettlemoyer 1

Outline § Probabilistic sequence models (and inference) § Probability and Uncertainty – Preview § Markov Chains § Hidden Markov Models § Exact Inference § Particle Filters

Recap: Reasoning Over Time 0.5 § Stationary Markov models 0.3 rain sun 0.5 X 1 X 2 X 3 X 4 0.7 § Hidden Markov models X E P rain umbrella 0.9 X 1 X 2 X 3 X 4 X 5 rain no umbrella 0.1 sun umbrella 0.2 E 1 E 2 E 3 E 4 E 5 sun no umbrella 0.8

Hidden Markov Models X 1 X 2 X 3 X 4 X N X 5 E 1 E 2 E 3 E 4 E N E 5 § Defines a joint probability distribution:

HMM Computations: Inference § Given X 1 X 2 X 3 X 4 § joint P(X 1:n , E 1:n ) § evidence E 1:n =e 1:n E 1 E 1 E 3 E 4 § Inference problems include: § Filtering, find P(X t |e 1:t ) for current t § Smoothing, find P(X t |e 1:n ) for past t § Most probable explanation, find x* 1:n = argmax x 1:n P(x 1:n |e 1:n )

Inference Recap: Simple Cases X 1 E 1 That’s my rule!

Inference Recap: Simple Cases X 1 X 1 X 2 E 1

Passage of Time § We want to know: § We can derive the following updates P ( X t +1 | e 1: t ) X = P ( X t +1 , x t | e 1: t ) x t X = P ( X t +1 | x t , e 1: t ) P ( x t | e 1: t ) x t X = P ( X t +1 | x t ) P ( x t | e 1: t ) x t � � § To get , compute each entry and normalize

Passage of Time § Assume we have current belief P(X | evidence to date) X 1 X 2 § Then, after one time step passes: § Or, compactly: § Basic idea: beliefs get “pushed” through the transitions § With the “B” notation, we have to be careful about what time step t the belief is about, and what evidence it includes

Example: Passage of Time Without observations, uncertainty “accumulates” T = 1 T = 2 T = 5 Transition model: ghosts usually go clockwise

Observations § Assume we have current belief P(X | previous evidence): P ( X t +1 | e 1: t +1 ) = P ( X t +1 , e t +1 | e 1: t ) /P ( e t +1 | e 1: t ) X 1 ∝ X t +1 P ( X t +1 , e t +1 | e 1: t ) = P ( e t +1 | e 1: t , X t +1 ) P ( X t +1 | e 1: t ) E 1 = P ( e t +1 | X t +1 ) P ( X t +1 | e 1: t ) ! Bas 11

Observations § Assume we have current belief P(X | previous evidence): X 1 § Then: E 1 § Or: § Basic idea: beliefs reweighted by likelihood of evidence § Unlike passage of time, we have to renormalize

Example: Observation § As we get observations, beliefs get reweighted, uncertainty “decreases” Before observation After observation

Online Belief Updates § Every time step, we start with current P(X | evidence) § We update for time: X 1 X 2 X 2 § We update for evidence: E 2

The Forward Algorithm § We want to know: § We can derive the following updates � § To get , compute each entry and normalize ! Problem:#space#is#|X|#and#)me#is#|X| 2 #per#)me#step#

Example § An HMM is defined by: § Initial distribution: § Transitions: § Emissions:

Forward Algorithm

Example Pac-man

Summary: Filtering § Filtering is the inference process of finding a distribution over X T given e 1 through e T : P( X T | e 1:t ) § We first compute P( X 1 | e 1 ): § For each t from 2 to T, we have P( X t-1 | e 1:t-1 ) § Elapse time: compute P( X t | e 1:t-1 ) § Observe: compute P(X t | e 1:t-1 , e t ) = P( X t | e 1:t )