Hidden Markov models and dynamic programming Matthew Macauley - - PowerPoint PPT Presentation

Hidden Markov models and dynamic programming

Matthew Macauley Department of Mathematical Sciences Clemson University http://www.math.clemson.edu/~macaule/ Math 4500, Spring 2017

• M. Macauley (Clemson)

Hidden Markov models and dynamic programming Math 4500, Spring 2017 1 / 12

The occasionally dishonest casino

3 canonical questions

Given a sequence of roles by the casino: WWWLWLWLWLWWLWWLLLWWWWLWWLWWLWLWLLLWLWWLLWWLWLWLLWWLLLWLWWWWLWLWWWWL

• ne may ask:
• 1. Evaluation: How likely is this sequence given our model?
• 2. Decoding: When was the casino rolling the fair vs. the unfair die?
• 3. Learning: Can we deduce the probability parameters if we didn’t know them? (e.g.,

“how loaded are the die?”, and “how often does the casino switch?”)

Fair Unfair W : 2/3 L: 1/3 W : 0.4 L: 0.6 0.95 0.9 0.05 0.1

• M. Macauley (Clemson)

Hidden Markov models and dynamic programming Math 4500, Spring 2017 2 / 12

Problem #1: Evaluation

For CpG identification, we need the posterior probabilities P♣πt ✏k ⑤ xq, for each k P Q and t ✏ 1, 2, . . . , ℓ. By Bayes’ theorem, P♣πt ✏ k ⑤ xq ✏ P♣x, πt ✏ kq P♣xq We can compute P♣x, πt ✏ kq recursively: P♣x, πt ✏kq ✏ P♣x1x2 ☎ ☎ ☎ xt, π ✏kq ☎ P♣xt1xt2 ☎ ☎ ☎ xℓ ⑤ x1x2 ☎ ☎ ☎ xt, πt ✏kq ✏ P♣x1x2 ☎ ☎ ☎ xt, π ✏kq ☎ P♣xt1xt2 ☎ ☎ ☎ xℓ ⑤ πt ✏kq ✏ fk♣tq ☎ bk♣tq.

The forward-backward algorithm

Given an emitted sequence x ✏ x1x2x3 ☎ ☎ ☎ xℓ, we will use the forward algorithm to compute fk♣tq: the probability of getting x ✏ x1x2x3 ☎ ☎ ☎ xt and ending up in state k. backward algorithm to compute bj♣tq: the probability of observing xt1 ☎ ☎ ☎ xℓ from state k. It is also straightforwrd to compute P♣xq using either of these algorithms.

• M. Macauley (Clemson)

Hidden Markov models and dynamic programming Math 4500, Spring 2017 3 / 12

The forward algorithm

• Example. Compute P♣xq, for x ✏ LWW.

Fair Unfair W : 2/3 L: 1/3 W : 0.4 L: 0.6 0.7 0.6 .3 .4

t✏0 t✏1 t✏2 t✏3

B F U F U F U

0.5 0.5 0.7 0.3 0.4 0.6

Forward algorithm

• 1. Initialize (t ✏ 0): Set fB♣0q ✏ 1, and fj♣0q, for all j P Q.
• 2. Recursion: do for t ✏ 1, 2, . . . , ℓ:

for each k P Q, define fk♣tq :✏ ek♣xtq ➳

jPQ

fj♣t ✁ 1qajk

• 3. Termination: Set P♣xq ✏

➳

kPQ

fk♣ℓq.

• M. Macauley (Clemson)

Hidden Markov models and dynamic programming Math 4500, Spring 2017 4 / 12

The forward algorithm

• Example. Compute P♣xq, for x ✏ LWW.

Fair Unfair W : 2/3 L: 1/3 W : 0.4 L: 0.6 0.7 0.6 .3 .4

t✏0 t✏1 t✏2 t✏3

B F U F U F U

0.5 0.5 0.7 0.3 0.4 0.6

t=0. fB♣0q ✏ 1, fF ♣0q ✏ 0, fU♣0q ✏ 0. t=1. fF ♣1q ✏ P♣x1 ✏L, π1 ✏Fq ✏ fB♣0q ☎ aBF ☎ eF ♣Lq ✏ 1 ☎ 1

2 ☎ 1 3 ✏ 1 6 .

fU♣1q ✏ P♣x1 ✏L, π1 ✏Uq ✏ fB♣0q ☎ aBU ☎ eU♣Lq ✏ 1 ☎ 1

2 ☎ 6 10 ✏ 0.3.

• M. Macauley (Clemson)

Hidden Markov models and dynamic programming Math 4500, Spring 2017 5 / 12

The forward algorithm

• Example. Compute P♣xq, for x ✏ LWW.

Fair Unfair W : 2/3 L: 1/3 W : 0.4 L: 0.6 0.7 0.6 .3 .4

t✏0 t✏1 t✏2 t✏3

B F U F U F U

0.5 0.5 0.7 0.3 0.4 0.6

t ✏2 : fF ♣2q ✏ P♣x1x2 ✏LW , π2 ✏Fq ✏ fF ♣1q ☎ aFF ☎ eF ♣W q fU♣1q ☎ aUF ☎ eF ♣W q ✏ 1

6 ♣.7q 2 3 ♣.3q♣.4q 2 3 ✓ 0.1578.

fU♣2q ✏ P♣x1x2 ✏LW , π2 ✏Uq ✏ fF ♣1q ☎ aFU ☎ eU♣W q fU♣1q ☎ aUU ☎ eU♣W q ✏ 1

6 ♣.3q♣.4q ♣.3q♣.6q♣.4q ✏ 0.092.

t ✏2 : fF ♣3q ✏ P♣x1x2x3 ✏LWW , π3 ✏Fq ✏ fF ♣2q ☎ aFF ☎ eF ♣W q fU♣2q ☎ aUF ☎ eF ♣W q ✏ ♣.1578q♣.7q 2

3 ♣.092q♣.4q 2 3 ✓ .0982.

fU♣3q ✏ P♣x1x2 ✏LWW , π3 ✏Uq ✏ fF ♣2q ☎ aFU ☎ eU♣W q fU♣2q ☎ aUU ☎ eU♣W q ✏ ♣.1578q♣.3q♣.4q ♣.092q♣.6q♣.4q ✓ 0.0410. Now, P♣xq ✏ P♣x ✏LWW q ✏ fF ♣3q fU♣3q ✓ .0982 .0410 ✏ .1392.

• M. Macauley (Clemson)

Hidden Markov models and dynamic programming Math 4500, Spring 2017 6 / 12

The backward algorithm

• Example. Compute P♣xq, for x ✏ LWW.

Fair Unfair W : 2/3 L: 1/3 W : 0.4 L: 0.6 0.7 0.6 .3 .4

t✏0 t✏1 t✏2 t✏3

B F U F U F U

0.5 0.5 0.7 0.3 0.4 0.6

Backward algorithm and bj♣ℓq

• 1. Initialize (t ✏ ℓ): Set bk♣ℓq ✏ 1 for all j P Q.
• 2. Recursion: do for t ✏ ℓ ✁ 1, . . . , 2, 1:

for each j P Q, bj♣tq : ✏ P♣xt1xt2 ☎ ☎ ☎ xℓ ⑤ πt ✏jq ✏ ➳

kPQ

P♣πt1 ✏k ⑤ πt ✏jq ☎ ek♣xt1q ☎ P♣xt2 ☎ ☎ ☎ xℓ ⑤ πt1 ✏kq ✏ ➳

kPQ

ajkek♣xt1qbk♣t 1q.

• 3. Termination: Set P♣xq ✏

➳

kPQ

aBkek♣x1qbk♣1q.

• M. Macauley (Clemson)

Hidden Markov models and dynamic programming Math 4500, Spring 2017 7 / 12

The backward algorithm

• Example. Compute P♣xq, for x ✏ LWW.

Fair Unfair W : 2/3 L: 1/3 W : 0.4 L: 0.6 0.7 0.6 .3 .4

t=3. bF ♣3q ✏ 1, bU♣3q ✏ 1. t=2. bF ♣2q ✏ aFF eF ♣W qbF ♣3q aFUeU♣W qbU♣3q ✏ ♣.7q 2

3 ♣.3q♣.4q ✏ 44 75 ✓ 0.5866.

bU♣2q ✏ aUF eF ♣W qbF ♣3q aUUeU♣W qbU♣3q ✏ ♣.4q 2

3 ♣.6q♣.4q ✏ 38 75 ✓ 0.5067.

t=1. bF ♣1q ✏ aFF eF ♣W qbF ♣2q aFUeU♣W qbU♣2q ✏ ♣.7q 2

3 ☎ 44 75 ♣.3q♣.4q 38 75 ✓ 0.3346.

bU♣2q ✏ aUF eF ♣W qbF ♣2q aUUeU♣W qbU♣2q ✏ ♣.4q 2

3 ☎ 44 75 ♣.6q♣.4q 38 75 ✓ 0.2780.

• M. Macauley (Clemson)

Hidden Markov models and dynamic programming Math 4500, Spring 2017 8 / 12

The forward-backward algorithm

• Example. Compute P♣xq, for x ✏ LWW.

Fair Unfair W : 2/3 L: 1/3 W : 0.4 L: 0.6 0.7 0.6 .3 .4

P♣π1 ✏F ⑤ x1x2x3 ✏LWW q ✏ fF ♣1qbF ♣1q P♣xq ✓ ♣1④6q♣.3346q .1392 ✓ 0.4006 P♣π1 ✏U ⑤ x1x2x3 ✏LWW q ✏ fF ♣1qbF ♣1q P♣xq ✓ ♣.3q♣.2780q .1392 ✓ 0.5991 P♣π2 ✏F ⑤ x1x2x3 ✏LWW q ✏ fF ♣1qbF ♣1q P♣xq ✓ ♣.1578q♣.5866q .1392 ✓ 0.6650 P♣π1 ✏U ⑤ x1x2x3 ✏LWW q ✏ fF ♣1qbF ♣1q P♣xq ✓ ♣.092q♣.5067q .1392 ✓ 0.3349 P♣π3 ✏F ⑤ x1x2x3 ✏LWW q ✏ fF ♣1qbF ♣1q P♣xq ✓ ♣.0982q♣1q .1392 ✓ 0.7055 P♣π3 ✏U ⑤ x1x2x3 ✏LWW q ✏ fF ♣1qbF ♣1q P♣xq ✓ ♣.041q♣1q .1392 ✓ 0.2945.

• M. Macauley (Clemson)

Hidden Markov models and dynamic programming Math 4500, Spring 2017 9 / 12

Decoding and the Viterbi algorithm

Problem #2: Decoding

Given an observed path x ✏ x1x2x3 ☎ ☎ ☎ xℓ, what is the most likely hidden path π ✏ π1π2π3 ☎ ☎ ☎ πℓ to emit x? That is, compute πmax ✏ arg max

π

P♣π⑤xq ✏ arg max

π

P♣x, πq Assume that for each j P Q, we’ve computed π1π2 ☎ ☎ ☎ πt✁2πt✁1 of highest probability among those emitting x1x2 ☎ ☎ ☎ xt✁1. Denote the probability of this path by vj♣t ✁ 1q ✏ max

π✏π1☎☎☎πt✁1 P♣πt✁1 ✏ j, xt✁1q.

Then, for each k P Q, say emitting x1x2 ☎ ☎ ☎ xt: vk♣tq ✏ max

π1☎☎☎πt✁1 P♣πt✁1 ✏ k, xtq ✏ max jPQ

✥ vj♣t ✁ 1qajkek♣xtq ✭ ✏ ek♣xtq max

jPQ

✥ vj♣t ✁ 1qajk ✭ .

• M. Macauley (Clemson)

Hidden Markov models and dynamic programming Math 4500, Spring 2017 10 / 12

Decoding and the Viterbi algorithm

Viterbi algorithm

• 1. Initialize (t ✏ 0): Set vB♣0q ✏ 1, and vj♣0q, for all j P Q.
• 2. Recursion: do for t ✏ 1, 2, . . . , ℓ:

for each k P Q, define vk♣tq :✏ ek♣xtq max

jPQ tvj♣t ✁ 1qajk✉

Also, set ptrk♣tq ✏ r ✏ arg maxjtvj♣t ✁ 1qajk✉.

• 3. Termination: Set P♣x, π✝q ✏ max

π

P♣x, πq ✏ max

jPQ tvj♣ℓq✉, and ptrk♣ℓq ✏ π✝ ℓ .

The maximum probability path can be found by tracing back through the pointers.

• M. Macauley (Clemson)

Hidden Markov models and dynamic programming Math 4500, Spring 2017 11 / 12

Decoding and the Viterbi algorithm

• Example. Given x ✏ LWW, what is the

most likely path π ✏ π1π2π3?

Fair Unfair W : 2/3 L: 1/3 W : 0.4 L: 0.6 0.7 0.6 .3 .4

t✏0 t✏1 t✏2 t✏3

B F U F U F U

0.5 0.5 0.7 0.3 0.4 0.6

t=1. vF ♣1q ✏ max

π1 P♣π1 ✏F, x1 ✏Lq ✏ maxtvB♣0q ☎ aBF ☎ eF ♣Lq✉ ✏ 1 ☎ 1 2 ☎ 1 3 ✏ 1 6 .

vU♣1q ✏ max

π1 P♣π1 ✏U, x1 ✏Lq ✏ maxtvB♣0q ☎ aBU ☎ eU♣Lq✉ ✏ 1 ☎ 1 2 ♣.6q ✏ 0.3.

• M. Macauley (Clemson)

Hidden Markov models and dynamic programming Math 4500, Spring 2017 12 / 12