Hidden Markov Models COSI 114 Computational Linguistics James - PowerPoint PPT Presentation

Forward Procedure x 1 x t-1 x t x t+1 x T o 1 o t-1 o t o t+1 o T ( + t 1 ) α j P ( o ... o , x j ) = = 1 t 1 t 1 + + P ( o ... o | x j ) P ( x j ) = = = 1 t 1 t 1 t 1 + + + P ( o ... o | x j ) P ( o | x j ) P ( x j ) = = = = 1 t t 1 t 1 t 1 t 1 + + + + P ( o ... o , x j ) P ( o | x j ) = = = 1 t t 1 t 1 t 1 + + +

Forward Procedure x 1 x t-1 x t x t+1 x T o 1 o t-1 o t o t+1 o T ( + t 1 ) α j P ( o ... o , x j ) = = 1 t 1 t 1 + + P ( o ... o | x j ) P ( x j ) = = = 1 t 1 t 1 t 1 + + + P ( o ... o | x j ) P ( o | x j ) P ( x j ) = = = = 1 t t 1 t 1 t 1 t 1 + + + + P ( o ... o , x j ) P ( o | x j ) = = = 1 t t 1 t 1 t 1 + + +

Forward Procedure x 1 x t-1 x t x t+1 x T o 1 o t-1 o t o t+1 o T P ( o ... o , x i , x j ) P ( o | x j ) ∑ = = = = 1 t t t 1 t 1 t 1 + + + i 1 ... N = P ( o ... o , x j | x i ) P ( x i ) P ( o | x j ) ∑ = = = = = 1 t t 1 t t t 1 t 1 + + + i 1 ... N = P ( o ... o , x i ) P ( x j | x i ) P ( o | x j ) ∑ = = = = = 1 t t t 1 t t 1 t 1 + + + i 1 ... N = ( t ) a b ∑ = α i ij jo t 1 + i 1 ... N =

Forward Procedure x 1 x t-1 x t x t+1 x T o 1 o t-1 o t o t+1 o T P ( o ... o , x i , x j ) P ( o | x j ) ∑ = = = = 1 t t t 1 t 1 t 1 + + + i 1 ... N = P ( o ... o , x j | x i ) P ( x i ) P ( o | x j ) ∑ = = = = = 1 t t 1 t t t 1 t 1 + + + i 1 ... N = P ( o ... o , x i ) P ( x j | x i ) P ( o | x j ) ∑ = = = = = 1 t t t 1 t t 1 t 1 + + + i 1 ... N = ( t ) a b ∑ = α i ij jo t 1 + i 1 ... N =

Forward Procedure x 1 x t-1 x t x t+1 x T o 1 o t-1 o t o t+1 o T P ( o ... o , x i , x j ) P ( o | x j ) ∑ = = = = 1 t t t 1 t 1 t 1 + + + i 1 ... N = P ( o ... o , x j | x i ) P ( x i ) P ( o | x j ) ∑ = = = = = 1 t t 1 t t t 1 t 1 + + + i 1 ... N = P ( o ... o , x i ) P ( x j | x i ) P ( o | x j ) ∑ = = = = = 1 t t t 1 t t 1 t 1 + + + i 1 ... N = ( t ) a b ∑ = α i ij jo t 1 + i 1 ... N =

Forward Procedure x 1 x t-1 x t x t+1 x T o 1 o t-1 o t o t+1 o T P ( o ... o , x i , x j ) P ( o | x j ) ∑ = = = = 1 t t t 1 t 1 t 1 + + + i 1 ... N = P ( o ... o , x j | x i ) P ( x i ) P ( o | x j ) ∑ = = = = = 1 t t 1 t t t 1 t 1 + + + i 1 ... N = P ( o ... o , x i ) P ( x j | x i ) P ( o | x j ) ∑ = = = = = 1 t t t 1 t t 1 t 1 + + + i 1 ... N = ( t ) a b ∑ = α i ij jo t 1 + i 1 ... N =

Backward Procedure x 1 x t-1 x t x t+1 x T o 1 o t-1 o t o t+1 o T ( T 1 ) 1 β + = i Probability of the rest ( t ) P ( o ... o | x i ) β = = i t T t of the states given the first state ( t ) a b ( t 1 ) ∑ β = β + i ij io j t j 1 ... N =

Decoding Solution x 1 x t-1 x t x t+1 x T o 1 o t-1 o t o t+1 o T N P ( O | ) i T ( ) ∑ Forward Procedure µ = α i 1 = N P ( O | ) ( 1 ) ∑ µ = π β Backward Procedure i i i 1 = N P ( O | ) ( t ) ( t ) ∑ µ = α β Combination i i i 1 =

Best State Sequence o 1 o t-1 o t o t+1 o T — Find the state sequence that best explains the observations — Viterbi algorithm — arg max P ( X | O ) X

Viterbi Algorithm x 1 x t-1 j o 1 o t-1 o t o t+1 o T ( t ) max P ( x ... x , o ... o , x j , o ) δ = = j 1 t 1 1 t 1 t t − − x ... x 1 t 1 − The state sequence which maximizes the probability of seeing the observations to time t-1, landing in state j, and seeing the observation at time t

Viterbi Algorithm x 1 x t-1 x t x t+1 o 1 o t-1 o t o t+1 o T ( t ) max P ( x ... x , o ... o , x j , o ) δ = = j 1 t 1 1 t 1 t t − − x ... x 1 t 1 − ( t 1 ) max ( t ) a b δ + = δ j i ij jo Recursive t 1 i + Computation ( t 1 ) arg max ( t ) a b ψ + = δ j i ij jo t 1 + i

Viterbi Algorithm x 1 x t-1 x t x t+1 x T o 1 o t-1 o t o t+1 o T ˆ X arg max ( T ) = δ T Compute the most i i likely state sequence ˆ X + t ( 1 ) = ψ + by working t ^ X t 1 backwards ˆ P ( X ) arg max ( T ) = δ i i

Parameter Estimation A A A A B B B B B o 1 o t-1 o t o t+1 o T • Given an observation sequence, find the model that is most likely to produce that sequence. • No analytic method • Given a model and observation sequence, update the model parameters to better fit the observations.

Parameter Estimation A A A A B B B B B o 1 o t-1 o t o t+1 o T ( t ) a b ( t 1 ) α β + i ij jo j Probability of p ( i , j ) t 1 = + t ( t ) ( t ) ∑ α β traversing an arc m m m 1 ... N = Probability of being ( t ) p ( i , j ) ∑ γ = i t in state i j 1 ... N =

Parameter Estimation A A A A B B B B B o 1 o t-1 o t o t+1 o T ˆ ( 1 ) π = γ i i T p ( i , j ) ∑ Now we can t ˆ a t 1 = = compute the new ij T ( t ) ∑ γ estimates of the i t 1 = model parameters. ( i ) ∑ γ t ˆ { t : o k } = b t = ik T ( t ) ∑ γ i t 1 =

HMM Applications — Generating parameters for n-gram models — Tagging speech — Speech recognition

The Most Important Thing A A A A B B B B B o 1 o t-1 o t o t+1 o T We can use the special structure of this model to do a lot of neat math and solve problems that are otherwise not solvable.

Example of Hidden Markov Model 0.3 0.7 Low High 0.2 0.8 0.6 0.6 0.4 0.4 Rain Dry

Example of Markov Model 0.3 0.7 Rain Dry 0.2 0.8 • Two states : ‘ Rain ’ and ‘ Dry ’ . • Transition probabilities: P( ‘ Rain ’ | ‘ Rain ’ ) =0.3 , P( ‘ Dry ’ | ‘ Rain ’ ) =0.7 , P( ‘ Rain ’ | ‘ Dry ’ ) =0.2, P( ‘ Dry ’ | ‘ Dry ’ ) =0.8 • Initial probabilities: say P( ‘ Rain ’ ) =0.4 , P( ‘ Dry ’ ) =0.6 .

Calculation of sequence probability • By Markov chain property, probability of state sequence can be found by the formula: P ( s , s , … , s ) P ( s | s , s , … , s ) P ( s , s , … , s ) = i 1 i 2 ik ik i 1 i 2 ik 1 i 1 i 2 ik 1 − − P ( s | s ) P ( s , s , … , s ) … = = ik ik 1 i 1 i 2 ik 1 − − P ( s | s ) P ( s | s ) … P ( s | s ) P ( s ) = ik ik 1 ik 1 ik 2 i 2 i 1 i 1 − − − • Suppose we want to calculate a probability of a sequence of states in our example, { ‘ Dry ’ , ’ Dry ’ , ’ Rain ’ ,Rain ’ }. P( { ‘ Dry ’ , ’ Dry ’ , ’ Rain ’ ,Rain ’ } ) = P( ‘ Rain ’ | ’ Rain ’ ) P( ‘ Rain ’ | ’ Dry ’ ) P( ‘ Dry ’ | ’ Dry ’ ) P( ‘ Dry ’ )= = 0.3*0.2*0.8*0.6

Hidden Markov models … { s , s , , s } • Set of states: 1 2 N • Process moves from one state to another generating a sequence of states : s , s , … , s , … i 1 i 2 ik • Markov chain property: probability of each subsequent state depends only on what was the previous state: P ( s | s , s , … , s ) P ( s | s ) = ik i 1 i 2 ik 1 ik ik 1 − − • States are not visible, but each state randomly generates one of M observations (or visible states) { v , v , … , v } 1 2 M • To define hidden Markov model, the following probabilities have to be specified: matrix of transition probabilities A=(a ij ), a ij = P(s i | s j ) , matrix of observation probabilities B=(b i (v m )), b i (v m ) = P(v m | s i ) and a vector of initial probabilities π =( π i ), π i = P(s i ) . Model is represented by M=(A, B, π ).

Example of Hidden Markov Model • Two states : ‘ Low ’ and ‘ High ’ atmospheric pressure. • Two observations : ‘ Rain ’ and ‘ Dry ’ . • Transition probabilities: P( ‘ Low ’ | ‘ Low ’ ) =0.3 , P( ‘ High ’ | ‘ Low ’ ) =0.7 , P( ‘ Low ’ | ‘ High ’ ) =0.2, P( ‘ High ’ | ‘ High ’ ) =0.8 • Observation probabilities : P( ‘ Rain ’ | ‘ Low ’ ) =0.6 , P( ‘ Dry ’ | ‘ Low ’ ) =0.4 , P( ‘ Rain ’ | ‘ High ’ ) =0.4 , P( ‘ Dry ’ | ‘ High ’ ) =0.3 . • Initial probabilities: say P( ‘ Low ’ ) =0.4 , P( ‘ High ’ ) =0.6 .

Calculation of observation sequence probability • Suppose we want to calculate a probability of a sequence of observations in our example, { ‘ Dry ’ , ’ Rain ’ }. • Consider all possible hidden state sequences: P( { ‘ Dry ’ , ’ Rain ’ } ) = P( { ‘ Dry ’ , ’ Rain ’ } , { ‘ Low ’ , ’ Low ’ } ) + P( { ‘ Dry ’ , ’ Rain ’ } , { ‘ Low ’ , ’ High ’ } ) + P( { ‘ Dry ’ , ’ Rain ’ } , { ‘ High ’ , ’ Low ’ } ) + P( { ‘ Dry ’ , ’ Rain ’ } , { ‘ High ’ , ’ High ’ } ) where first term is : P( { ‘ Dry ’ , ’ Rain ’ } , { ‘ Low ’ , ’ Low ’ } )= P( { ‘ Dry ’ , ’ Rain ’ } | { ‘ Low ’ , ’ Low ’ } ) P( { ‘ Low ’ , ’ Low ’ } ) = P( ‘ Dry ’ | ’ Low ’ )P( ‘ Rain ’ | ’ Low ’ ) P( ‘ Low ’ )P( ‘ Low ’ | ’ Low ) = 0.4*0.4*0.6*0.4*0.3

Main issues using HMMs : Evaluation problem. Given the HMM M=(A, B, π ) and the observation sequence O=o 1 o 2 ... o K , calculate the probability that model M has generated sequence O . • Decoding problem. Given the HMM M=(A, B, π ) and the observation sequence O=o 1 o 2 ... o K , calculate the most likely sequence of hidden states s i that produced this observation sequence O . • Learning problem. Given some training observation sequences O=o 1 o 2 ... o K and general structure of HMM (numbers of hidden and visible states), determine HMM parameters M=(A, B, π ) that best fit training data. O=o 1 ...o K denotes a sequence of observations o k ∈ { v 1 ,…,v M }.

Word recognition example(1). • Typed word recognition, assume all characters are separated. • Character recognizer outputs probability of the image being particular character, P(image|character). a 0.5 0.03 b c 0.005 0.31 z Hidden state Observation

Word recognition example(2). • Hidden states of HMM = characters. • Observations = typed images of characters segmented from the v image . Note that there is an infinite number of observations α • Observation probabilities = character recognizer scores. ( ) ( ) B b ( v ) P ( v | s ) = = i i α α • Transition probabilities will be defined differently in two subsequent models.

Word recognition example(3). • If lexicon is given, we can construct separate HMM models for each lexicon word. Amherst a m r t h e s Buffalo b u a o f f l 0.03 0.5 0.4 0.6 • Here recognition of word image is equivalent to the problem of evaluating few HMM models. • This is an application of Evaluation problem.

Word recognition example(4). • We can construct a single HMM for all words. • Hidden states = all characters in the alphabet. • Transition probabilities and initial probabilities are calculated from language model. • Observations and observation probabilities are as before. m r a f t o v b h e s • Here we have to determine the best sequence of hidden states, the one that most likely produced word image. • This is an application of Decoding problem.

Character recognition with HMM example. • The structure of hidden states is chosen. • Observations are feature vectors extracted from vertical slices. • Probabilistic mapping from hidden state to feature vectors: 1. use mixture of Gaussian models 2. Quantize feature vector space.

Exercise: character recognition with HMM(1) • The structure of hidden states: s 1 s 2 s 3 • Observation = number of islands in the vertical slice. • HMM for character ‘ A ’ : ⎛ .8 .2 0 ⎞ Transition probabilities: {a ij }= ⏐ 0 .8 .2 ⏐ ⎝ 0 0 1 ⎠ Observation probabilities: {b jk }= ⎛ .9 .1 0 ⎞ ⏐ .1 .8 .1 ⏐ ⎝ .9 .1 0 ⎠ • HMM for character ‘ B ’ : ⎛ .8 .2 0 ⎞ Transition probabilities: {a ij }= ⏐ 0 .8 .2 ⏐ ⎝ 0 0 1 ⎠ Observation probabilities: {b jk }= ⎛ .9 .1 0 ⎞ ⏐ 0 .2 .8 ⏐ ⎝ .6 .4 0 ⎠

Exercise: character recognition with HMM(2) • Suppose that after character image segmentation the following sequence of island numbers in 4 slices was observed: { 1, 3, 2, 1} • What HMM is more likely to generate this observation sequence , HMM for ‘ A ’ or HMM for ‘ B ’ ?

Exercise: character recognition with HMM(3) Consider likelihood of generating given observation for each possible sequence of hidden states: • HMM for character ‘ A ’ : Hidden state sequence Transition probabilities Observation probabilities s 1 → s 1 → s 2 → s 3 .8 * .2 * .2 * .9 * 0 * .8 * .9 = 0 s 1 → s 2 → s 2 → s 3 .2 * .8 * .2 * .9 * .1 * .8 * .9 = 0.0020736 s 1 → s 2 → s 3 → s 3 .2 * .2 * 1 * .9 * .1 * .1 * .9 = 0.000324 Total = 0.0023976 • HMM for character ‘ B ’ : Hidden state sequence Transition probabilities Observation probabilities s 1 → s 1 → s 2 → s 3 .8 * .2 * .2 * .9 * 0 * .2 * .6 = 0 s 1 → s 2 → s 2 → s 3 .2 * .8 * .2 * .9 * .8 * .2 * .6 = 0.0027648 s 1 → s 2 → s 3 → s 3 .2 * .2 * 1 * .9 * .8 * .4 * .6 = 0.006912 Total = 0.0096768

Evaluation Problem. • Evaluation problem. Given the HMM M=(A, B, π ) and the observation sequence O=o 1 o 2 ... o K , calculate the probability that model M has generated sequence O . • Trying to find probability of observations O=o 1 o 2 ... o K by means of considering all hidden state sequences (as was done in example) is impractical: N K hidden state sequences - exponential complexity. • Use Forward-Backward HMM algorithms for efficient calculations. • Define the forward variable α k (i) as the joint probability of the partial observation sequence o 1 o 2 ... o k and that the hidden state at time k is s i : α k (i)= P(o 1 o 2 ... o k , q k = s i )

Trellis representation of an HMM o 1 o k o k+1 o K = Observations s 1 s 1 s 1 s 1 s 2 s 2 s 2 s 2 a 1j a 2j s i s i s j s i a ij a Nj s N s N s N s N Time= 1 k k+1 K

Forward recursion for HMM • Initialization: α 1 (i)= P(o 1 , q 1 = s i ) = π i b i (o 1 ) , 1<=i<=N. • Forward recursion: α k+1 (i)= P(o 1 o 2 ... o k+1 , q k+1 = s j ) = Σ i P(o 1 o 2 ... o k+1 , q k = s i , q k+1 = s j ) = Σ i P(o 1 o 2 ... o k , q k = s i ) a ij b j (o k+1 ) = [ Σ i α k (i) a ij ] b j (o k+1 ) , 1<=j<=N, 1<=k<=K-1. • Termination: P(o 1 o 2 ... o K ) = Σ i P(o 1 o 2 ... o K , q K = s i ) = Σ i α K (i) • Complexity : N 2 K operations.

Backward recursion for HMM • Define the forward variable β k (i) as the joint probability of the partial observation sequence o k+1 o k+2 ... o K given that the hidden state at time k is s i : β k (i)= P(o k+1 o k+2 ... o K |q k = s i ) • Initialization: β K (i)= 1 , 1<=i<=N. • Backward recursion: β k (j)= P(o k+1 o k+2 ... o K | q k = s j ) = Σ i P(o k+1 o k+2 ... o K , q k+1 = s i | q k = s j ) = Σ i P(o k+2 o k+3 ... o K | q k+1 = s i ) a ji b i (o k+1 ) = Σ i β k+1 (i) a ji b i (o k+1 ) , 1<=j<=N, 1<=k<=K-1. • Termination: P(o 1 o 2 ... o K ) = Σ i P(o 1 o 2 ... o K , q 1 = s i ) = Σ i P(o 1 o 2 ... o K |q 1 = s i ) P(q 1 = s i ) = Σ i β 1 (i) b i (o 1 ) π i

Decoding problem • Decoding problem. Given the HMM M=(A, B, π ) and the observation sequence O=o 1 o 2 ... o K , calculate the most likely sequence of hidden states s i that produced this observation sequence. • We want to find the state sequence Q = q 1 …q K which maximizes P(Q | o 1 o 2 ... o K ) , or equivalently P(Q , o 1 o 2 ... o K ) . • Brute force consideration of all paths takes exponential time. Use efficient Viterbi algorithm instead. • Define variable δ k (i) as the maximum probability of producing observation sequence o 1 o 2 ... o k when moving along any hidden state sequence q 1 … q k-1 and getting into q k = s i . δ k (i) = max P(q 1 … q k-1 , q k = s i , o 1 o 2 ... o k ) where max is taken over all possible paths q 1 … q k-1 .

Viterbi algorithm (1) • General idea: if best path ending in q k = s j goes through q k-1 = s i then it should coincide with best path ending in q k-1 = s i . q k-1 q k s 1 a 1j s i s j a ij a Nj s N • δ k (i) = max P(q 1 … q k-1 , q k = s j , o 1 o 2 ... o k ) = max i [ a ij b j (o k ) max P(q 1 … q k-1 = s i , o 1 o 2 ... o k-1 ) ] • To backtrack best path keep info that predecessor of s j was s i .

Viterbi algorithm (2) • Initialization: δ 1 (i) = max P(q 1 = s i , o 1 ) = π i b i (o 1 ) , 1<=i<=N. • Forward recursion: δ k (j) = max P(q 1 … q k-1 , q k = s j , o 1 o 2 ... o k ) = max i [ a ij b j (o k ) max P(q 1 … q k-1 = s i , o 1 o 2 ... o k-1 ) ] = max i [ a ij b j (o k ) δ k-1 (i) ] , 1<=j<=N, 2<=k<=K. • Termination: choose best path ending at time K max i [ δ K (i) ] • Backtrack best path. This algorithm is similar to the forward recursion of evaluation problem, with Σ replaced by max and additional backtracking.

Learning problem (1) • Learning problem. Given some training observation sequences O=o 1 o 2 ... o K and general structure of HMM (numbers of hidden and visible states), determine HMM parameters M=(A, B, π ) that best fit training data, that is maximizes P(O | M) . • There is no algorithm producing optimal parameter values. • Use iterative expectation-maximization algorithm to find local maximum of P(O | M) - Baum-Welch algorithm.

Learning problem (2) • If training data has information about sequence of hidden states (as in word recognition example), then use maximum likelihood estimation of parameters: a ij = P(s i | s j ) = Number of transitions from state s j to state s i Number of transitions out of state s j Number of times observation v m occurs in state s i b i (v m ) = P(v m | s i )= Number of times in state s i

Baum-Welch algorithm General idea: Expected number of transitions from state s j to state s i a ij = P(s i | s j ) = Expected number of transitions out of state s j Expected number of times observation v m occurs in state s i b i (v m ) = P(v m | s i )= Expected number of times in state s i π i = P(s i ) = Expected frequency in state s i at time k=1.

Baum-Welch algorithm: expectation step(1) • Define variable ξ k (i,j) as the probability of being in state s i at time k and in state s j at time k+1, given the observation sequence o 1 o 2 ... o K . ξ k (i,j)= P(q k = s i , q k+1 = s j | o 1 o 2 ... o K ) P(q k = s i , q k+1 = s j , o 1 o 2 ... o k ) ξ k (i,j)= = P(o 1 o 2 ... o k ) P(q k = s i , o 1 o 2 ... o k ) a ij b j (o k+1 ) P(o k+2 ... o K | q k+1 = s j ) = P(o 1 o 2 ... o k ) α k (i) a ij b j (o k+1 ) β k+1 (j) Σ i Σ j α k (i) a ij b j (o k+1 ) β k+1 (j)

Baum-Welch algorithm: expectation step(2) • Define variable γ k (i) as the probability of being in state s i at time k, given the observation sequence o 1 o 2 ... o K . γ k (i)= P(q k = s i | o 1 o 2 ... o K ) α k (i) β k (i) P(q k = s i , o 1 o 2 ... o k ) γ k (i)= = P(o 1 o 2 ... o k ) Σ i α k (i) β k (i)

Baum-Welch algorithm: expectation step(3) • We calculated ξ k (i,j) = P(q k = s i , q k+1 = s j | o 1 o 2 ... o K ) and γ k (i)= P(q k = s i | o 1 o 2 ... o K ) • Expected number of transitions from state s i to state s j = = Σ k ξ k (i,j) • Expected number of transitions out of state s i = Σ k γ k (i) • Expected number of times observation v m occurs in state s i = = Σ k γ k (i) , k is such that o k = v m • Expected frequency in state s i at time k=1 : γ 1 (i) .

Baum-Welch algorithm: maximization step Σ k ξ k (i,j) Expected number of transitions from state s j to state s i a ij = = Expected number of transitions out of state s j Σ k γ k (i) = Σ k ξ k (i,j) Expected number of times observation v m occurs in state s i b i (v m ) = Expected number of times in state s i Σ k,o k = v m γ k (i) π i = ( Expected frequency in state s i at time k=1) = γ 1 (i).

The Noisy Channel Model — Search through space of all possible sentences. — Pick the one that is most probable given the waveform.

The Noisy Channel Model (II) — What is the most likely sentence out of all sentences in the language L given some acoustic input O? — Treat acoustic input O as sequence of individual observations ◦ O = o 1 ,o 2 ,o 3 ,…,o t — Define a sentence as a sequence of words: ◦ W = w 1 ,w 2 ,w 3 ,…,w n

Noisy Channel Model (III) — Probabilistic implication: Pick the highest prob S: ˆ W = argmax P ( W | O ) W ∈ L — We can use Bayes rule to rewrite this: P ( O | W ) P ( W ) ˆ W = argmax P ( O ) W ∈ L — Since denominator is the same for each candidate sentence W, we can ignore it for the argmax: ˆ W = argmax P ( O | W ) P ( W ) W ∈ L

Noisy channel model likelihood prior ˆ W = argmax P ( O | W ) P ( W ) W ∈ L

The noisy channel model — Ignoring the denominator leaves us with two factors: P(Source) and P(Signal| Source)

Speech Architecture meets Noisy Channel

HMMs for speech

Phones are not homogeneous! 5000 0 0.48152 ay k 0.937203 Time (s)

Each phone has 3 subphones

Resulting HMM word model for “ six ”

HMMs more formally — Markov chains — A kind of weighted finite-state automaton

HMMs more formally — Markov chains — A kind of weighted finite-state automaton

Another Markov chain

Another view of Markov chains

An example with numbers: — What is probability of: ◦ Hot hot hot hot ◦ Cold hot cold hot

Hidden Markov Models

Hidden Markov Models

Hidden Markov Models — Bakis network Ergodic (fully-connected) network — Left-to-right network

The Jason Eisner task — You are a climatologist in 2799 studying the history of global warming — YOU can ’ t find records of the weather in Baltimore for summer 2006 — But you do find Jason Eisner ’ s diary — Which records how many ice creams he ate each day. — Can we use this to figure out the weather? ◦ Given a sequence of observations O, – each observation an integer = number of ice creams eaten – Figure out correct hidden sequence Q of weather states (H or C) which caused Jason to eat the ice cream

HMMs more formally — Three fundamental problems ◦ Jack Ferguson at IDA in the 1960s 1) Given a specific HMM, determine likelihood of observation sequence. 2) Given an observation sequence and an HMM, discover the best (most probable) hidden state sequence 3) Given only an observation sequence, learn the HMM parameters (A, B matrix)

The Three Basic Problems for HMMs — Problem 1 ( Evaluation ) : Given the observation sequence O=(o 1 o 2 …o T ), and an HMM model Φ = (A,B), how do we efficiently compute P(O| Φ ), the probability of the observation sequence, given the model — Problem 2 ( Decoding ) : Given the observation sequence O=(o 1 o 2 …o T ), and an HMM model Φ = (A,B), how do we choose a corresponding state sequence Q=(q 1 q 2 … q T ) that is optimal in some sense (i.e., best explains the observations) — Problem 3 ( Learning): How do we adjust the model parameters Φ = (A,B) to maximize P(O| Φ )?

Problem 1: computing the observation likelihood — Given the following HMM: — How likely is the sequence 3 1 3?

How to compute likelihood — For a Markov chain, we just follow the states 3 1 3 and multiply the probabilities — But for an HMM, we don ’ t know what the states are! — So let ’ s start with a simpler situation. — Computing the observation likelihood for a given hidden state sequence ◦ Suppose we knew the weather and wanted to predict how much ice cream Jason would eat. ◦ I.e. P( 3 1 3 | H H C)

Computing likelihood for 1 given hidden state sequence

Computing total likelihood of 3 1 3 — We would need to sum over ◦ Hot hot cold ◦ Hot hot hot ◦ Hot cold hot ◦ …. — How many possible hidden state sequences are there for this sequence? — How about in general for an HMM with N hidden states and a sequence of T observations? ◦ N T — So we can ’ t just do separate computation for each hidden state sequence.

Instead: the Forward algorithm — A kind of dynamic programming algorithm ◦ Uses a table to store intermediate values — Idea: ◦ Compute the likelihood of the observation sequence ◦ By summing over all possible hidden state sequences ◦ But doing this efficiently – By folding all the sequences into a single trellis

The Forward Trellis

The forward algorithm — Each cell of the forward algorithm trellis alpha t (j) ◦ Represents the probability of being in state j ◦ After seeing the first t observations ◦ Given the automaton — Each cell thus expresses the following probabilty