Introduction to Hidden Markov Models CMSC 473/673 UMBC Recap from - - PowerPoint PPT Presentation

Introduction to Hidden Markov Models

CMSC 473/673 UMBC

Recap from last time…

Expectation Maximization (EM)

• 0. Assume some value for your parameters

Two step, iterative algorithm

• 1. E-step: count under uncertainty, assuming these

parameters

• 2. M-step: maximize log-likelihood, assuming these

uncertain counts

estimated counts

Counting Requires Marginalizing

E-step: count under uncertainty, assuming these parameters

Counting Requires Marginalizing

w z1 & w z2 & w z3 & w z4 & w

E-step: count under uncertainty, assuming these parameters

break into 4 disjoint pieces

EM Example 1: Three Coins/Class-based Unigrams

Imagine three coins Flip 1st coin (penny) If heads: flip 2nd coin (dollar coin) If tails: flip 3rd coin (dime)

• bserved:

a, b, e, etc. We run the code, vs. The run failed unobserved: vowel or constonant? part of speech?

EM Example 2: Machine Translation Alignment

Want: P(f|e) But don’t know how to train this directly… Solution: Use P(a, f|e), where a is an alignment Remember:

Le chat est sur la chaise verte The cat is on the green chair

marginalizingacross all possible alignments

IBM Model 1 (1993)

f: vector of French words (visualization of alignment) e: vector of English words a: vector of alignment indices t(fj|ei) : translation probability

• f the word fj given the word

ei Le chat est sur la chaise verte The cat is on the green chair 0 1 2 3 4 6 5

Learning the Alignments through EM

• 0. Assume some value for

and compute other parameter values Two step, iterative algorithm

• 1. E-step: count alignments and translations under

uncertainty, assuming these parameters

• 2. M-step: maximize log-likelihood (update

parameters), using uncertain counts

estimated counts

P( | “the cat”) P( | “the cat”)

le chat le chat

Follow up: IBM Model 1 Parameters

For IBM model 1, we can compute all parameters given translation parameters: How many of these are there? |French vocabulary| x |English vocabulary| From Rebecca: See Sec. 31 of the Knight tutorial for more about space considerations

Alignment: Output and Complexities

Component of machine translation systems Produce a translation lexicon automatically Cross-lingual projection/extraction of information Supervision for training other models (for example, neural MT systems)

http://www.cis.upenn.edu/~ccb/figures/research-statement/pivoting.jpg

Hidden Markov Models

…

Agenda

HMM Motivation (Part of Speech) and Brief Definition What is Part of Speech? HMM Detailed Definition HMM Tasks

Hidden Markov Models

Class-based Model Use different distributions to explain groupings of

• bservations

Sequence Model Bigram model of the classes, not the observations Implicitly model all possible class sequences There are algorithms for finding best sequence, the marginal likelihood, and doing semi-/un-supervised learning

Hidden Markov Models: Part of Speech

p(British Left Waffles on Falkland Islands)

Adjective Noun Verb Noun Verb Noun Prep Noun Noun Prep Noun Noun (i): (ii):

Class-based model Bigram model

• f the classes

Model all class sequences

Hidden Markov Models: Part of Speech

p(British Left Waffles on Falkland Islands)

Adjective Noun Verb Noun Verb Noun Prep Noun Noun Prep Noun Noun (i): (ii):

Class-based model Bigram model

• f the classes

Model all class sequences

Hidden Markov Models: Part of Speech

p(British Left Waffles on Falkland Islands)

Adjective Noun Verb Noun Verb Noun Prep Noun Noun Prep Noun Noun (i): (ii):

Class-based model Bigram model

• f the classes

Model all class sequences

Hidden Markov Models: Part of Speech

p(British Left Waffles on Falkland Islands)

Adjective Noun Verb Noun Verb Noun Prep Noun Noun Prep Noun Noun (i): (ii):

Class-based model Bigram model

• f the classes

Model all class sequences

Hidden Markov Models: Part of Speech

p(British Left Waffles on Falkland Islands)

1. Explain this sentence as a sequence of (likely?) latent (unseen) tags (labels) 2. Produce a tag sequence for this sentence Adjective Noun Verb Noun Verb Noun Prep Noun Noun Prep Noun Noun (i): (ii):

Class-based model Bigram model

• f the classes

Model all class sequences

Agenda

HMM Motivation (Part of Speech) and Brief Definition What is Part of Speech? HMM Detailed Definition HMM Tasks

Brief Aside: Parts of Speech

Classes of words that behave like one another in similar syntactic contexts

Parts of Speech

Classes of words that behave like one another in similar syntactic contexts Pronunciation (stress) can differ: object (noun: OB-ject) vs. object (verb: ob-JECT) It can help improve the inputs to other systems (text-to-speech, syntactic parsing)

Parts of Speech

Adapted from Luke Zettlemoyer

Nouns milk cat cats UMBC Baltimore bread speak give Verbs run Adjectives would-be wettest large happy red fake

Parts of Speech

Adapted from Luke Zettlemoyer

Nouns milk cat cats UMBC Baltimore bread speak give Verbs run Adjectives would-be wettest large happy red fake Determiners Conjunctions a the every what and

• r

if because Prepositions in under top

Parts of Speech

Adapted from Luke Zettlemoyer

Nouns milk cat cats UMBC Baltimore bread speak give Verbs run Adjectives would-be wettest large happy red fake can do may Determiners Conjunctions a the every what and

• r

if because Prepositions in under top

“I can eat.”

Parts of Speech

Adapted from Luke Zettlemoyer

Nouns milk cat cats UMBC Baltimore bread speak give Verbs run Adjectives would-be wettest large happy red fake can do may Determiners Conjunctions a the every what and

• r

if because Prepositions in under top

Parts of Speech

Adapted from Luke Zettlemoyer

Nouns milk cat cats UMBC Baltimore bread speak give Verbs run Adjectives would-be wettest large happy red fake can do may Determiners Conjunctions a the every what and

• r

if because Prepositions in under top Adverbs recently happily

Parts of Speech

Adapted from Luke Zettlemoyer

Nouns milk cat cats UMBC Baltimore bread speak give Verbs run Adjectives would-be wettest large happy red fake can do may Determiners Conjunctions a the every what and

• r

if because Prepositions in under top Adverbs recently happily

“Today, we eat there.”

then there (location)

Parts of Speech

Adapted from Luke Zettlemoyer

Nouns milk cat cats UMBC Baltimore bread speak give Verbs run Adjectives would-be wettest large happy red fake can do may Determiners Conjunctions a the every what and

• r

if because Prepositions in under top Adverbs recently happily

“I ate.” “There is a cat.”

then there (location) I you there

Parts of Speech

Adapted from Luke Zettlemoyer

Nouns milk cat cats UMBC Baltimore bread speak give Verbs run Adjectives would-be wettest large happy red fake can do may Determiners Conjunctions a the every what and

• r

if because Prepositions in under top Adverbs recently happily then there (location) I you there Numbers

• ne

1,324

Closed class words Open class words

Parts of Speech

Adapted from Luke Zettlemoyer

Nouns milk cat cats UMBC Baltimore bread speak give Verbs run Adjectives would-be wettest large happy red fake can do may Determiners Conjunctions a the every what and

• r

if because Prepositions in under top Adverbs recently happily then there (location) I you there Numbers

• ne

1,324

Parts of Speech

Adapted from Luke Zettlemoyer

Closed class words Open class words Nouns milk cat cats UMBC Baltimore bread speak give can do may Verbs Adjectives would-be wettest large happy red fake

Kamp & Partee (1995)

Adverbs recently happily then there (location)

intransitive

run

ditransitive transitive subsective non- subsective modals, auxiliaries

I you Determiners Prepositions Conjunctions

Pronouns

a the every what in under top and

• r

if because there Numbers

• ne

1,324

Parts of Speech

Adapted from Luke Zettlemoyer

Closed class words Open class words Nouns milk cat cats UMBC Baltimore bread speak give can do may Verbs Adjectives would-be wettest large happy red fake

Kamp & Partee (1995)

Adverbs recently happily then there (location)

intransitive

run

ditransitive transitive subsective non- subsective modals, auxiliaries

I you Determiners Prepositions Conjunctions

Pronouns

a the every what in under top Particles

(set) up

so (far) not (call)

• ff

and

• r

if because there Numbers

• ne

1,324

Parts of Speech

Adapted from Luke Zettlemoyer

Closed class words Open class words Nouns milk cat cats UMBC Baltimore bread speak give can do may Verbs Adjectives would-be wettest large happy red fake

Kamp & Partee (1995)

Adverbs recently happily then there (location)

intransitive

run

ditransitive transitive subsective non- subsective modals, auxiliaries

Numbers I you

• ne

1,324 Determiners Prepositions Conjunctions

Pronouns

and

• r

if a the every what in under top Particles

(set) up

so (far) not (call)

• ff

Language evolves! “I’m reading this because I want to procrastinate.” → “I’m reading this because procrastination.”

https://www.theatlantic.com/technology/archive/2013/11/english-has-a-new-preposition-because-internet/281601/

because because

Agenda

HMM Motivation (Part of Speech) and Brief Definition What is Part of Speech? HMM Detailed Definition HMM Tasks

Hidden Markov Models: Part of Speech

p(British Left Waffles on Falkland Islands)

Adjective Noun Verb Noun Verb Noun Prep Noun Noun Prep Noun Noun (i): (ii):

Class-based model Bigram model

• f the classes

Model all class sequences

𝑞 𝑥𝑗|𝑨𝑗

Hidden Markov Models: Part of Speech

p(British Left Waffles on Falkland Islands)

Adjective Noun Verb Noun Verb Noun Prep Noun Noun Prep Noun Noun (i): (ii):

Class-based model Bigram model

• f the classes

Model all class sequences

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗|𝑨𝑗−1

Hidden Markov Models: Part of Speech

p(British Left Waffles on Falkland Islands)

Adjective Noun Verb Noun Verb Noun Prep Noun Noun Prep Noun Noun (i): (ii):

Class-based model Bigram model

• f the classes

Model all class sequences

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗|𝑨𝑗−1

෍

𝑨1,..,𝑨𝑂

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂

Hidden Markov Model

Goal: maximize (log-)likelihood In practice: we don’t actually observe these z values; we just see the words w

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1| 𝑨0 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂| 𝑨𝑂−1 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗| 𝑨𝑗−1

Hidden Markov Model

Goal: maximize (log-)likelihood In practice: we don’t actually observe these z values; we just see the words w

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1| 𝑨0 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂| 𝑨𝑂−1 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗| 𝑨𝑗−1

if we knew the probability parameters then we could estimate z and evaluate likelihood… but we don’t! :( if we did observe z, estimating the probability parameterswould be easy… but we don’t! :(

Hidden Markov Model Terminology

Each zi can take the value of one of K latent states

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1| 𝑨0 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂| 𝑨𝑂−1 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗| 𝑨𝑗−1

Hidden Markov Model Terminology

Each zi can take the value of one of K latent states

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1| 𝑨0 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂| 𝑨𝑂−1 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗| 𝑨𝑗−1

transition probabilities/parameters

Hidden Markov Model Terminology

Each zi can take the value of one of K latent states

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1| 𝑨0 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂| 𝑨𝑂−1 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗| 𝑨𝑗−1

emission probabilities/parameters transition probabilities/parameters

Hidden Markov Model Terminology

Each zi can take the value of one of K latent states Transition and emission distributions do not change

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1| 𝑨0 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂| 𝑨𝑂−1 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗| 𝑨𝑗−1

emission probabilities/parameters transition probabilities/parameters

Hidden Markov Model Terminology

Each zi can take the value of one of K latent states Transition and emission distributions do not change Q: How many different probability values are there with K states and V vocab items?

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1| 𝑨0 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂| 𝑨𝑂−1 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗| 𝑨𝑗−1

emission probabilities/parameters transition probabilities/parameters

Hidden Markov Model Terminology

Each zi can take the value of one of K latent states Transition and emission distributions do not change Q: How many different probability values are there with K states and V vocab items? A: VK emission values and K2 transition values

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1| 𝑨0 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂| 𝑨𝑂−1 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗| 𝑨𝑗−1

emission probabilities/parameters transition probabilities/parameters

Hidden Markov Model Representation

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1| 𝑨0 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂| 𝑨𝑂−1 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗| 𝑨𝑗−1

emission probabilities/parameters transition probabilities/parameters

z1

w1

…

w2 w3 w4

z2 z3 z4

represent the probabilities and independence assumptions in a graph

Hidden Markov Model Representation

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1| 𝑨0 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂| 𝑨𝑂−1 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗| 𝑨𝑗−1

emission probabilities/parameters transition probabilities/parameters

z1

w1

…

w2 w3 w4

z2 z3 z4

Graphical Models (see CMSC 478/678… and also CMSC 691: Graphical & Statistical Models of Learning)

Hidden Markov Model Representation

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1| 𝑨0 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂| 𝑨𝑂−1 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗| 𝑨𝑗−1

emission probabilities/parameters transition probabilities/parameters

z1

w1

…

w2 w3 w4

z2 z3 z4

𝑞 𝑥1|𝑨1 𝑞 𝑥2|𝑨2 𝑞 𝑥3|𝑨3 𝑞 𝑥4|𝑨4

Hidden Markov Model Representation

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1| 𝑨0 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂| 𝑨𝑂−1 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗| 𝑨𝑗−1

emission probabilities/parameters transition probabilities/parameters

z1

w1

…

w2 w3 w4

z2 z3 z4

𝑞 𝑥1|𝑨1 𝑞 𝑥2|𝑨2 𝑞 𝑥3|𝑨3 𝑞 𝑥4|𝑨4

𝑞 𝑨2| 𝑨1 𝑞 𝑨3| 𝑨2 𝑞 𝑨4| 𝑨3

Hidden Markov Model Representation

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1| 𝑨0 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂| 𝑨𝑂−1 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗| 𝑨𝑗−1

emission probabilities/parameters transition probabilities/parameters

z1

w1

…

w2 w3 w4

z2 z3 z4

𝑞 𝑥1|𝑨1 𝑞 𝑥2|𝑨2 𝑞 𝑥3|𝑨3 𝑞 𝑥4|𝑨4

𝑞 𝑨2| 𝑨1 𝑞 𝑨3| 𝑨2 𝑞 𝑨4| 𝑨3 𝑞 𝑨1| 𝑨0 initial starting distribution (“BOS”)

Hidden Markov Model Representation

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1| 𝑨0 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂| 𝑨𝑂−1 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗| 𝑨𝑗−1

emission probabilities/parameters transition probabilities/parameters

z1

w1

…

w2 w3 w4

z2 z3 z4

𝑞 𝑥1|𝑨1 𝑞 𝑥2|𝑨2 𝑞 𝑥3|𝑨3 𝑞 𝑥4|𝑨4

𝑞 𝑨2| 𝑨1 𝑞 𝑨3| 𝑨2 𝑞 𝑨4| 𝑨3 𝑞 𝑨1| 𝑨0 initial starting distribution (“BOS”)

Each zi can take the value of one of K latent states Transition and emission distributions do not change

Example: 2-state Hidden Markov Model as a Lattice

z1 = N

w1

…

w2 w3 w4

z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V

…

Example: 2-state Hidden Markov Model as a Lattice

z1 = N

w1

…

w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥2|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

z2 = N z3 = N z4 = N z1 = V z2 = V z4 = V

…

𝑞 𝑥4|𝑊 𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 𝑞 𝑥1|𝑊

z3 = V

Example: 2-state Hidden Markov Model as a Lattice

z1 = N

w1

…

w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥2|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑊 𝑞 𝑊| start

…

𝑞 𝑥4|𝑊 𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 𝑞 𝑥1|𝑊

𝑞 𝑂| 𝑂 𝑞 𝑂| 𝑂 𝑞 𝑂| 𝑂

Example: 2-state Hidden Markov Model as a Lattice

z1 = N

w1

…

w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥2|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑊 𝑞 𝑊| start

…

𝑞 𝑊| 𝑂 𝑞 𝑊| 𝑂 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊 𝑞 𝑂| 𝑊 𝑞 𝑂| 𝑊

𝑞 𝑥4|𝑊 𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 𝑞 𝑥1|𝑊

𝑞 𝑂| 𝑂 𝑞 𝑂| 𝑂 𝑞 𝑂| 𝑂

Unigram Language Model

Comparison of Joint Probabilities

𝑞 𝑥1, 𝑥2, … , 𝑥𝑂 = 𝑞 𝑥1 𝑞 𝑥2 ⋯ 𝑞 𝑥𝑂 = ෑ

𝑗

𝑞 𝑥𝑗

Unigram Class-based Language Model (“K” coins) Unigram Language Model

Comparison of Joint Probabilities

𝑞 𝑥1, 𝑥2, … , 𝑥𝑂 = 𝑞 𝑥1 𝑞 𝑥2 ⋯ 𝑞 𝑥𝑂 = ෑ

𝑗

𝑞 𝑥𝑗 𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … ,𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗

Hidden Markov Model Unigram Class-based Language Model (“K” coins) Unigram Language Model

Comparison of Joint Probabilities

𝑞 𝑥1, 𝑥2, … , 𝑥𝑂 = 𝑞 𝑥1 𝑞 𝑥2 ⋯ 𝑞 𝑥𝑂 = ෑ

𝑗

𝑞 𝑥𝑗 𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … ,𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗 𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1| 𝑨0 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂| 𝑨𝑂−1 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗| 𝑨𝑗−1

Estimating Parameters from Observed Data

z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 N V end start N V w1 w2 W3 w4 N V Transition Counts Emission Counts

end emission not shown

Estimating Parameters from Observed Data

z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 N V end start 2 N 1 2 2 V 2 1 w1 w2 W3 w4 N 2 1 2 V 2 1 Transition Counts Emission Counts

end emission not shown

Estimating Parameters from Observed Data

z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 N V end start 1 N .2 .4 .4 V 2/3 1/3 w1 w2 W3 w4 N .4 .2 .4 V 2/3 1/3 Transition MLE Emission MLE

end emission not shown

Estimating Parameters from Observed Data

z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 N V end start 1 N .2 .4 .4 V 2/3 1/3 w1 w2 W3 w4 N .4 .2 .4 V 2/3 1/3 Transition MLE Emission MLE

end emission not shown

smooth these values if needed

Agenda

HMM Motivation (Part of Speech) and Brief Definition What is Part of Speech? HMM Detailed Definition HMM Tasks

Hidden Markov Model Tasks

Calculate the (log) likelihood of an observed sequence w1, …, wN Calculate the most likely sequence of states (for an

• bserved sequence)

Learn the emission and transition parameters

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1| 𝑨0 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂| 𝑨𝑂−1 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗| 𝑨𝑗−1

emission probabilities/parameters transition probabilities/parameters

Hidden Markov Model Tasks

Calculate the (log) likelihood of an observed sequence w1, …, wN Calculate the most likely sequence of states (for an

• bserved sequence)

Learn the emission and transition parameters

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1| 𝑨0 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂| 𝑨𝑂−1 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗| 𝑨𝑗−1

emission probabilities/parameters transition probabilities/parameters

HMM Likelihood Task

Marginalize over all latent sequence joint likelihoods

𝑞 𝑥1, 𝑥2, … , 𝑥𝑂 = ෍

𝑨1,⋯,𝑨𝑂

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂

Q: In a K-state HMM for a length N observation sequence, how many summands (different latent sequences) are there?

HMM Likelihood Task

Marginalize over all latent sequence joint likelihoods

𝑞 𝑥1, 𝑥2, … , 𝑥𝑂 = ෍

𝑨1,⋯,𝑨𝑂

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂

Q: In a K-state HMM for a length N observation sequence, how many summands (different latent sequences) are there? A: KN

HMM Likelihood Task

Marginalize over all latent sequence joint likelihoods

𝑞 𝑥1, 𝑥2, … , 𝑥𝑂 = ෍

𝑨1,⋯,𝑨𝑂

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂

Q: In a K-state HMM for a length N observation sequence, how many summands (different latent sequences) are there? A: KN Goal: Find a way to compute this exponential sum efficiently (in polynomial time)

HMM Likelihood Task

Marginalize over all latent sequence joint likelihoods

𝑞 𝑥1, 𝑥2, … , 𝑥𝑂 = ෍

𝑨1,⋯,𝑨𝑂

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂

Q: In a K-state HMM for a length N observation sequence, how many summands (different latent sequences) are there? A: KN Goal: Find a way to compute this exponential sum efficiently (in polynomial time) Like in language modeling, you need to model when to stop generating. This ending state is generally not included in “K.”

2 (3)-State HMM Likelihood

z1 = N

w1

…

w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥2|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑊 𝑞 𝑊| start

…

𝑞 𝑊| 𝑂 𝑞 𝑊| 𝑂 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊 𝑞 𝑂| 𝑊 𝑞 𝑂| 𝑊

𝑞 𝑥4|𝑊 𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 𝑞 𝑥1|𝑊

𝑞 𝑂| 𝑂 𝑞 𝑂| 𝑂 𝑞 𝑂| 𝑂

Q: What are the latent sequences here (EOS excluded)?

2 (3)-State HMM Likelihood

z1 = N

w1

…

w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥2|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑊 𝑞 𝑊| start

…

𝑞 𝑊| 𝑂 𝑞 𝑊| 𝑂 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊 𝑞 𝑂| 𝑊 𝑞 𝑂| 𝑊

𝑞 𝑥4|𝑊 𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 𝑞 𝑥1|𝑊

𝑞 𝑂| 𝑂 𝑞 𝑂| 𝑂 𝑞 𝑂| 𝑂

(N, w1), (N, w2), (N, w3), (N, w4) (N, w1), (N, w2), (N, w3), (V, w4) (N, w1), (N, w2), (V, w3), (N, w4) (N, w1), (N, w2), (V, w3), (V, w4) A: (N, w1), (V, w2), (N, w3), (N, w4) (N, w1), (V, w2), (N, w3), (V, w4) (N, w1), (V, w2), (V, w3), (N, w4) (N, w1), (V, w2), (V, w3), (V, w4) (V, w1), (N, w2), (N, w3), (N, w4) (V, w1), (N, w2), (N, w3), (V, w4) … (six more) Q: What are the latent sequences here (EOS excluded)?

2 (3)-State HMM Likelihood

z1 = N

w1

…

w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥2|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑊 𝑞 𝑊| start

…

𝑞 𝑊| 𝑂 𝑞 𝑊| 𝑂 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊 𝑞 𝑂| 𝑊 𝑞 𝑂| 𝑊

𝑞 𝑥4|𝑊 𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 𝑞 𝑥1|𝑊

𝑞 𝑂| 𝑂 𝑞 𝑂| 𝑂 𝑞 𝑂| 𝑂

(N, w1), (N, w2), (N, w3), (N, w4) (N, w1), (N, w2), (N, w3), (V, w4) (N, w1), (N, w2), (V, w3), (N, w4) (N, w1), (N, w2), (V, w3), (V, w4) A: (N, w1), (V, w2), (N, w3), (N, w4) (N, w1), (V, w2), (N, w3), (V, w4) (N, w1), (V, w2), (V, w3), (N, w4) (N, w1), (V, w2), (V, w3), (V, w4) (V, w1), (N, w2), (N, w3), (N, w4) (V, w1), (N, w2), (N, w3), (V, w4) … (six more) Q: What are the latent sequences here (EOS excluded)?

2 (3)-State HMM Likelihood

z1 = N

w1

…

w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥2|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑊 𝑞 𝑊| start

…

𝑞 𝑊| 𝑂 𝑞 𝑊| 𝑂 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊 𝑞 𝑂| 𝑊 𝑞 𝑂| 𝑊

𝑞 𝑥4|𝑊 𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 𝑞 𝑥1|𝑊

𝑞 𝑂| 𝑂 𝑞 𝑂| 𝑂 𝑞 𝑂| 𝑂

N V end start .7 .2 .1 N .15 .8 .05 V .6 .35 .05 w1 w2 w3 w4 N .7 .2 .05 .05 V .2 .6 .1 .1

2 (3)-State HMM Likelihood

z1 = N

w1

…

w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥2|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑊 𝑞 𝑊| start

…

𝑞 𝑊| 𝑂 𝑞 𝑊| 𝑂 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊 𝑞 𝑂| 𝑊 𝑞 𝑂| 𝑊

𝑞 𝑥4|𝑊 𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 𝑞 𝑥1|𝑊

𝑞 𝑂| 𝑂 𝑞 𝑂| 𝑂 𝑞 𝑂| 𝑂

Q: What’s the probability of (N, w1), (V, w2), (V, w3), (N, w4)? N V end start .7 .2 .1 N .15 .8 .05 V .6 .35 .05 w1 w2 w3 w4 N .7 .2 .05 .05 V .2 .6 .1 .1

2 (3)-State HMM Likelihood

z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 Q: What’s the probability of (N, w1), (V, w2), (V, w3), (N, w4)? A: (.7*.7) * (.8*.6) * (.35*.1) * (.6*.05) = 0.0002822 N V end start .7 .2 .1 N .15 .8 .05 V .6 .35 .05 w1 w2 w3 w4 N .7 .2 .05 .05 V .2 .6 .1 .1

2 (3)-State HMM Likelihood

z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 Q: What’s the probability of (N, w1), (V, w2), (V, w3), (N, w4) with ending included (unique ending symbol “#”)? A: (.7*.7) * (.8*.6) * (.35*.1) * (.6*.05) * (.05 * 1) = 0.00001235 N V end start .7 .2 .1 N .15 .8 .05 V .6 .35 .05 w1 w2 w3 w4

#

N .7 .2 .05 .05 V .2 .6 .1 .1

end

1

2 (3)-State HMM Likelihood

z1 = N

w1

…

w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥2|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑊 𝑞 𝑊| start

…

𝑞 𝑊| 𝑂 𝑞 𝑊| 𝑂 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊 𝑞 𝑂| 𝑊 𝑞 𝑂| 𝑊

𝑞 𝑥4|𝑊 𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 𝑞 𝑥1|𝑊

𝑞 𝑂| 𝑂 𝑞 𝑂| 𝑂 𝑞 𝑂| 𝑂

Q: What’s the probability of (N, w1), (V, w2), (N, w3), (N, w4)? N V end start .7 .2 .1 N .15 .8 .05 V .6 .35 .05 w1 w2 w3 w4 N .7 .2 .05 .05 V .2 .6 .1 .1

2 (3)-State HMM Likelihood

z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑂

Q: What’s the probability of (N, w1), (V, w2), (N, w3), (N, w4)? A: (.7*.7) * (.8*.6) * (.6*.05) * (.15*.05) = 0.00007056 N V end start .7 .2 .1 N .15 .8 .05 V .6 .35 .05 w1 w2 w3 w4 N .7 .2 .05 .05 V .2 .6 .1 .1

2 (3)-State HMM Likelihood

z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑂

Q: What’s the probability of (N, w1), (V, w2), (N, w3), (N, w4) with ending (unique symbol “#”)? A: (.7*.7) * (.8*.6) * (.6*.05) * (.15*.05) * (.05 * 1) = 0.000002646 N V end start .7 .2 .1 N .15 .8 .05 V .6 .35 .05 w1 w2 w3 w4

#

N .7 .2 .05 .05 V .2 .6 .1 .1

end

1

2 (3)-State HMM Likelihood

z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊

2 (3)-State HMM Likelihood

z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 Up until here, all the computation was the same

2 (3)-State HMM Likelihood

z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 Up until here, all the computation was the same Let’s reuse what computations we can

2 (3)-State HMM Likelihood

z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 Solution: pass information “forward” in the graph, e.g., from time step 2 to 3…

2 (3)-State HMM Likelihood

z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 Solution: pass information “forward” in the graph, e.g., from time step 2 to 3… Issue: these highlighted paths are only 2 of the 16 possible paths through the trellis

2 (3)-State HMM Likelihood

z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊 Solution: pass information “forward” in the graph, e.g., from time step 2 to 3… Solution: marginalize out all information from previous timesteps Issue: these highlighted paths are only 2 of the 16 possible paths through the trellis

Reusing Computation

zi-1 = C zi-1 = B zi-1 = A zi = C zi = B zi = A

let’s first consider “any shared path ending with B (AB, BB, or CB)→ B”

zi-2 = C zi-2 = B zi-2 = A

Reusing Computation

zi-1 = C zi-1 = B zi-1 = A zi = C zi = B zi = A

let’s first consider “any shared path ending with B (AB, BB, or CB)→ B” Assume that all necessary information has been computed and stored in 𝛽(𝑗 − 1, 𝐵), 𝛽(𝑗 − 1, 𝐶), 𝛽(𝑗 − 1, 𝐷)

zi-2 = C zi-2 = B zi-2 = A

𝛽(𝑗 − 1, 𝐵) 𝛽(𝑗 − 1, 𝐶) 𝛽(𝑗 − 1, 𝐷)

Reusing Computation

zi-1 = C zi-1 = B zi-1 = A zi = C zi = B zi = A

let’s first consider “any shared path ending with B (AB, BB, or CB)→ B” Assume that all necessary information has been computed and stored in 𝛽(𝑗 − 1, 𝐵), 𝛽(𝑗 − 1, 𝐶), 𝛽(𝑗 − 1, 𝐷) Marginalize (sum) across the previous timestep’s possible states

zi-2 = C zi-2 = B zi-2 = A

𝛽(𝑗 − 1, 𝐵) 𝛽(𝑗 − 1, 𝐶) 𝛽(𝑗 − 1, 𝐷) 𝛽(𝑗, 𝐶)

Reusing Computation

let’s first consider “any shared path ending with B (AB, BB, or CB)→ B” marginalize across the previous hidden state values 𝛽 𝑗, 𝐶 = ෍

𝑡

𝛽 𝑗 − 1, 𝑡 ∗ 𝑞 𝐶 𝑡) ∗ 𝑞(obs at 𝑗 | 𝐶)

zi-1 = C zi-1 = B zi-1 = A zi = C zi = B zi = A zi-2 = C zi-2 = B zi-2 = A

𝛽(𝑗 − 1, 𝐵) 𝛽(𝑗 − 1, 𝐶) 𝛽(𝑗 − 1, 𝐷) 𝛽(𝑗, 𝐶)

Reusing Computation

let’s first consider “any shared path ending with B (AB, BB, or CB)→ B” marginalize across the previous hidden state values 𝛽 𝑗, 𝐶 = ෍

𝑡

𝛽 𝑗 − 1, 𝑡 ∗ 𝑞 𝐶 𝑡) ∗ 𝑞(obs at 𝑗 | 𝐶) computing α at time i-1 will correctly incorporate paths through time i-2: we correctly obey the Markov property

zi-1 = C zi-1 = B zi-1 = A zi = C zi = B zi = A zi-2 = C zi-2 = B zi-2 = A

𝛽(𝑗 − 1, 𝐵) 𝛽(𝑗 − 1, 𝐶) 𝛽(𝑗 − 1, 𝐷) 𝛽(𝑗, 𝐶)

Forward Probability

let’s first consider “any shared path ending with B (AB, BB, or CB)→ B” marginalize across the previous hidden state values 𝛽 𝑗, 𝐶 = ෍

𝑡′

𝛽 𝑗 − 1, 𝑡′ ∗ 𝑞 𝐶 𝑡′) ∗ 𝑞(obs at 𝑗 | 𝐶) computing α at time i-1 will correctly incorporate paths through time i-2: we correctly obey the Markov property α(i, B) is the total probability of all paths to that state B from the beginning

zi-1 = C zi-1 = B zi-1 = A zi = C zi = B zi = A zi-2 = C zi-2 = B zi-2 = A

Forward Probability

α(i, s) is the total probability of all paths:

• 1. that start from the beginning
• 2. that end (currently) in s at step i
• 3. that emit the observation obs at i

Forward Probability

α(i, s) is the total probability of all paths:

• 1. that start from the beginning
• 2. that end (currently) in s at step i
• 3. that emit the observation obs at i

how likely is it to get into state s this way? what are the immediate ways to get into state s? what’s the total probability up until now?

2 (3) -State HMM Likelihood with Forward Probabilities

w3 w4

𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

z3 = N z4 = N z3 = V z4 = V 𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑊

α[1, N] = (.7*.7) α[2, V] = α[1, N] * (.8*.6) + α[1, V] * (.35*.6)

N V end start .7 .2 .1 N .15 .8 .05 V .6 .35 .05 w1 w2 w3 w4 N .7 .2 .05 .05 V .2 .6 .1 .1

𝑞 𝑊| start

α[3, V] = α[2, V] * (.35*.1)+ α[2, N] * (.8*.1) α[3, N] = α[2, V] * (.6*.05) + α[2, N] * (.15*.05)

2 (3) -State HMM Likelihood with Forward Probabilities

w3 w4

𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

z3 = N z4 = N z3 = V z4 = V 𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑊

α[1, N] = (.7*.7) α[2, V] = α[1, N] * (.8*.6) + α[1, V] * (.35*.6) α[3, V] = α[2, V] * (.35*.1)+ α[2, N] * (.8*.1) α[3, N] = α[2, V] * (.6*.05) + α[2, N] * (.15*.05)

N V end start .7 .2 .1 N .15 .8 .05 V .6 .35 .05 w1 w2 w3 w4 N .7 .2 .05 .05 V .2 .6 .1 .1

𝑞 𝑊| start

α[1, V] = (.2*.2)

2 (3) -State HMM Likelihood with Forward Probabilities

w3 w4

𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

z3 = N z4 = N z3 = V z4 = V 𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑊

α[1, N] = (.7*.7) = .49 α[2, V] = α[1, N] * (.8*.6) + α[1, V] * (.35*.6) = 0.2436 α[3, V] = α[2, V] * (.35*.1)+ α[2, N] * (.8*.1) α[3, N] = α[2, V] * (.6*.05) + α[2, N] * (.2*.05)

N V end start .7 .2 .1 N .15 .8 .05 V .6 .35 .05 w1 w2 w3 w4 N .7 .2 .05 .05 V .2 .6 .1 .1

𝑞 𝑊| start

α[1, V] = (.2*.2) = .04 α[2, N] = α[1, N] * (.15*.2) + α[1, V] * (.6*.2) = .0195

2 (3) -State HMM Likelihood with Forward Probabilities

w3 w4

𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

z3 = N z4 = N z3 = V z4 = V 𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑊

α[1, N] = (.7*.7) = .49 α[2, V] = α[1, N] * (.8*.6) + α[1, V] * (.35*.6) = 0.2436 α[3, V] = α[2, V] * (.35*.1)+ α[2, N] * (.8*.1) α[3, N] = α[2, V] * (.6*.05) + α[2, N] * (.2*.05)

N V end start .7 .2 .1 N .15 .8 .05 V .6 .35 .05 w1 w2 w3 w4 N .7 .2 .05 .05 V .2 .6 .1 .1

𝑞 𝑊| start

α[1, V] = (.2*.2) = .04 α[2, N] = α[1, N] * (.15*.2) + α[1, V] * (.6*.2) = .0195

2 (3) -State HMM Likelihood with Forward Probabilities

w3 w4

𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

z3 = N z4 = N z3 = V z4 = V 𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑊

α[1, N] = (.7*.7) = .49 α[2, V] = α[1, N] * (.8*.6) + α[1, V] * (.35*.6) = 0.2436 α[3, V] = α[2, V] * (.35*.1)+ α[2, N] * (.8*.1) α[3, N] = α[2, V] * (.6*.05) + α[2, N] * (.2*.05)

N V end start .7 .2 .1 N .15 .8 .05 V .6 .35 .05 w1 w2 w3 w4 N .7 .2 .05 .05 V .2 .6 .1 .1

𝑞 𝑊| start

α[1, V] = (.2*.2) = .04 α[2, N] = α[1, N] * (.15*.2) + α[1, V] * (.6*.2) = .0195

Use dynamic programming to build the α left-to-right

Forward Algorithm

α: a 2D table, N+2 x K* N+2: number of observations (+2 for the BOS & EOS symbols) K*: number of states Use dynamic programming to build the α left-to- right

Forward Algorithm

α = double[N+2][K*] α[0][*] = 0.0 α[0][START] = 1.0 for(i = 1; i ≤ N+1; ++i) { }

Forward Algorithm

α = double[N+2][K*] α[0][*] = 0.0 α[0][START] = 1.0 for(i = 1; i ≤ N+1; ++i) { for(state = 0; state < K*; ++state) { } }

Forward Algorithm

α = double[N+2][K*] α[0][*] = 0.0 α[0][START] = 1.0 for(i = 1; i ≤ N+1; ++i) { for(state = 0; state < K*; ++state) { pobs = pemission(obsi | state) } }

Forward Algorithm

α = double[N+2][K*] α[0][*] = 0.0 α[0][START] = 1.0 for(i = 1; i ≤ N+1; ++i) { for(state = 0; state < K*; ++state) { pobs = pemission(obsi | state) for(old = 0; old < K*; ++old) { pmove = ptransition(state | old) α[i][state] += α[i-1][old] * pobs * pmove } } }

Forward Algorithm

α = double[N+2][K*] α[0][*] = 0.0 α[0][START] = 1.0 for(i = 1; i ≤ N+1; ++i) { for(state = 0; state < K*; ++state) { pobs = pemission(obsi | state) for(old = 0; old < K*; ++old) { pmove = ptransition(state | old) α[i][state] += α[i-1][old] * pobs * pmove } } }

we still need to learn these (EM if not observed)

Forward Algorithm

α = double[N+2][K*] α[0][*] = 0.0 α[0][START] = 1.0 for(i = 1; i ≤ N+1; ++i) { for(state = 0; state < K*; ++state) { pobs = pemission(obsi | state) for(old = 0; old < K*; ++old) { pmove = ptransition(state | old) α[i][state] += α[i-1][old] * pobs * pmove } } }

Q: What do we return? (How do we return the likelihood of the sequence?)

Forward Algorithm

α = double[N+2][K*] α[0][*] = 0.0 α[0][START] = 1.0 for(i = 1; i ≤ N+1; ++i) { for(state = 0; state < K*; ++state) { pobs = pemission(obsi | state) for(old = 0; old < K*; ++old) { pmove = ptransition(state | old) α[i][state] += α[i-1][old] * pobs * pmove } } }

Q: What do we return? (How do we return the likelihood of the sequence?)

A: α[N+1][end]

Interactive HMM Example

https://goo.gl/rbHEoc (Jason Eisner, 2002)

Original: http://www.cs.jhu.edu/~jason/465/PowerPoint/lect24-hmm.xls

Forward Algorithm in Log-Space

α = double[N+2][K*] α[0][*] = -∞ α[0][*] = 0.0 for(i = 1; i ≤ N+1; ++i) { for(state = 0; state < K*; ++state) { pobs = log pemission(obsi | state) for(old = 0; old < K*; ++old) { pmove = log ptransition(state | old) α[i][state] = logadd(α[i][state], α[i-1][old] + pobs + pmove) } } }

Forward Algorithm in Log-Space

α = double[N+2][K*] α[0][*] = -∞ α[0][*] = 0.0 for(i = 1; i ≤ N+1; ++i) { for(state = 0; state < K*; ++state) { pobs = log pemission(obsi | state) for(old = 0; old < K*; ++old) { pmove = log ptransition(state | old) α[i][state] = logadd(α[i][state], α[i-1][old] + pobs + pmove) } } } logadd 𝑚𝑞,𝑚𝑟 = log exp 𝑚𝑞 + exp(𝑚𝑟)

Forward Algorithm in Log-Space

α = double[N+2][K*] α[0][*] = -∞ α[0][*] = 0.0 for(i = 1; i ≤ N+1; ++i) { for(state = 0; state < K*; ++state) { pobs = log pemission(obsi | state) for(old = 0; old < K*; ++old) { pmove = log ptransition(state | old) α[i][state] = logadd(α[i][state], α[i-1][old] + pobs + pmove) } } } logadd 𝑚𝑞,𝑚𝑟 = log exp 𝑚𝑞 + exp(𝑚𝑟)

this can still overflow! (why?)

Forward Algorithm in Log-Space

α = double[N+2][K*] α[0][*] = -∞ α[0][*] = 0.0 for(i = 1; i ≤ N+1; ++i) { for(state = 0; state < K*; ++state) { pobs = log pemission(obsi | state) for(old = 0; old < K*; ++old) { pmove = log ptransition(state | old) α[i][state] = logadd(α[i][state], α[i-1][old] + pobs + pmove) } } }

scipy.misc.logsumexp

Hidden Markov Model Tasks

Calculate the (log) likelihood of an observed sequence w1, …, wN Calculate the most likely sequence of states (for an

• bserved sequence)

Learn the emission and transition parameters

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1| 𝑨0 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂| 𝑨𝑂−1 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗| 𝑨𝑗−1

emission probabilities/parameters transition probabilities/parameters

HMM Most-Likely Sequence Task

Maximize over all latent sequence joint likelihoods

max

𝑨1,⋯,𝑨𝑂 𝑞 𝑨1, 𝑥1,𝑨2, 𝑥2,… ,𝑨𝑂, 𝑥𝑂

Q: In a K-state HMM for a length N observation sequence, how many comparisons (different latent sequences) do we make?

HMM Most-Likely Sequence Task

Maximize over all latent sequence joint likelihoods

max

𝑨1,⋯,𝑨𝑂 𝑞 𝑨1, 𝑥1,𝑨2, 𝑥2,… ,𝑨𝑂, 𝑥𝑂

Q: In a K-state HMM for a length N observation sequence, how many comparisons (different latent sequences) do we make? A: KN

HMM Most-Likely Sequence Task

Maximize over all latent sequence joint likelihoods

max

𝑨1,⋯,𝑨𝑂 𝑞 𝑨1, 𝑥1,𝑨2, 𝑥2,… ,𝑨𝑂, 𝑥𝑂

Q: In a K-state HMM for a length N observation sequence, how many comparisons (different latent sequences) do we make? A: KN Goal: Find a way to compute this exponential comparison efficiently (in polynomial time)

HMM Most-Likely Sequence Task

Maximize over all latent sequence joint likelihoods

max

𝑨1,⋯,𝑨𝑂 𝑞 𝑨1, 𝑥1,𝑨2, 𝑥2,… ,𝑨𝑂, 𝑥𝑂

Q: In a K-state HMM for a length N observation sequence, how many comparisons (different latent sequences) do we make? A: KN Goal: Find a way to compute this exponential comparison efficiently (in polynomial time)

Viterbi Decoding

What’s the Maximum Value?

consider “any shared path ending with B (AB, BB, or CB)→ B” maximize across the previous hidden state values

𝑤 𝑗, 𝐶 = max

𝑡′

𝑤 𝑗 − 1, 𝑡′ ∗ 𝑞 𝐶 𝑡′) ∗ 𝑞(obs at 𝑗 | 𝐶)

v(i, B) is the maximum probability of any paths to that state B from the beginning (and emitting the

• bservation)

zi-1 = C zi-1 = B zi-1 = A zi = C zi = B zi = A

What’s the Maximum Value?

consider “any shared path ending with B (AB, BB, or CB)→ B” maximize across the previous hidden state values

zi-1 = C zi-1 = B zi-1 = A zi = C zi = B zi = A zi-2 = C zi-2 = B zi-2 = A

𝑤 𝑗, 𝐶 = max

𝑡′

𝑤 𝑗 − 1, 𝑡′ ∗ 𝑞 𝐶 𝑡′) ∗ 𝑞(obs at 𝑗 | 𝐶)

v(i, B) is the maximum probability of any paths to that state B from the beginning (and emitting the

• bservation)

What’s the Maximum Value?

consider “any shared path ending with B (AB, BB, or CB)→ B” maximize across the previous hidden state values

zi-1 = C zi-1 = B zi-1 = A zi = C zi = B zi = A zi-2 = C zi-2 = B zi-2 = A

computing v at time i-1 will correctly incorporate (maximize over) paths through time i-2: we correctly obey the Markov property

𝑤 𝑗, 𝐶 = max

𝑡′

𝑤 𝑗 − 1, 𝑡′ ∗ 𝑞 𝐶 𝑡′) ∗ 𝑞(obs at 𝑗 | 𝐶)

v(i, B) is the maximum probability of any paths to that state B from the beginning (and emitting the

• bservation)

2 (3)-State Viterbi

z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊

Up until here, all the computation was the same Let’s reuse what computations we can

2 (3) -State Viterbi

w3 w4

𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

z3 = N z4 = N z3 = V z4 = V 𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑊

v[1, N] = (.7*.7) = .49 v[2, V] = max{v[1, N] * (.8*.6), v[1, V] * (.35*.6)} = 0.2352 v[3, V] = max{v[2, V] * (.35*.1), v[2, N] * (.8*.1)} v[3, N] = max{v[2, V] * (.6*.05), v[2, N] * (.2*.05)}

N V end start .7 .2 .1 N .15 .8 .05 V .6 .35 .05 w1 w2 w3 w4 N .7 .2 .05 .05 V .2 .6 .1 .1

𝑞 𝑊| start

v[1, V] = (.2*.2) = .04 v[2, N] = max{v[1, N] * (.15*.2), v[1, V] * (.6*.2)} = .0588

2 (3) -State Viterbi

w3 w4

𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

z3 = N z4 = N z3 = V z4 = V 𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑊

v[1, N] = (.7*.7) = .49 v[2, V] = max{v[1, N] * (.8*.6), v[1, V] * (.35*.6)} = 0.2352 v[3, V] = max{v[2, V] * (.35*.1), v[2, N] * (.8*.1)} v[3, N] = max{v[2, V] * (.6*.05), v[2, N] * (.2*.05)}

N V end start .7 .2 .1 N .15 .8 .05 V .6 .35 .05 w1 w2 w3 w4 N .7 .2 .05 .05 V .2 .6 .1 .1

𝑞 𝑊| start

v[1, V] = (.2*.2) = .04 v[2, N] = max{v[1, N] * (.15*.2), v[1, V] * (.6*.2)} = .0588

Use dynamic programming to build the v left-to-right

2 (3) -State Viterbi

w3 w4

𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

z3 = N z4 = N z3 = V z4 = V 𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑊

v[1, N] = (.7*.7) = .49 v[2, V] = max{v[1, N] * (.8*.6), v[1, V] * (.35*.6)} = 0.2352 v[3, V] = max{v[2, V] * (.35*.1), v[2, N] * (.8*.1)} v[3, N] = max{v[2, V] * (.6*.05), v[2, N] * (.2*.05)}

N V end start .7 .2 .1 N .15 .8 .05 V .6 .35 .05 w1 w2 w3 w4 N .7 .2 .05 .05 V .2 .6 .1 .1

𝑞 𝑊| start

v[1, V] = (.2*.2) = .04 v[2, N] = max{v[1, N] * (.15*.2), v[1, V] * (.6*.2)} = .0588

keep backpointers: record the state that produced the maximum

Viterbi Algorithm

v = double[N+2][K*] b = int[N+2][K*] v[0][*] = 0 v[0][START] = 1.0

Viterbi Algorithm

v = double[N+2][K*] b = int[N+2][K*] v[0][*] = 0 v[0][START] = 1.0 for(i = 1; i ≤ N+1; ++i) { for(state = 0; state< K*; ++state) { } }

Viterbi Algorithm

v = double[N+2][K*] b = int[N+2][K*] v[0][*] = 0 v[0][START] = 1.0 for(i = 1; i ≤ N+1; ++i) { for(state = 0; state< K*; ++state) { pobs = pemission(obsi | state) for(old = 0; old < K*; ++old) { pmove = ptransition(state| old) } } }

Viterbi Algorithm

v = double[N+2][K*] b = int[N+2][K*] v[0][*] = 0 v[0][START] = 1.0 for(i = 1; i ≤ N+1; ++i) { for(state = 0; state< K*; ++state) { pobs = pemission(obsi | state) for(old = 0; old < K*; ++old) { pmove = ptransition(state| old) if(v[i-1][old] * pobs * pmove > v[i][state]) { v[i][state] = v[i-1][old] * pobs * pmove b[i][state] = old } } } }

Viterbi Algorithm

v = double[N+2][K*] b = int[N+2][K*] v[0][*] = 0 v[0][START] = 1.0 for(i = 1; i ≤ N+1; ++i) { for(state = 0; state< K*; ++state) { pobs = pemission(obsi | state) for(old = 0; old < K*; ++old) { pmove = ptransition(state| old) if(v[i-1][old] * pobs * pmove > v[i][state]) { v[i][state] = v[i-1][old] * pobs * pmove b[i][state] = old } } } } Q: How do we return the most likely tag sequence?

Viterbi Algorithm

v = double[N+2][K*] b = int[N+2][K*] v[0][*] = 0 v[0][START] = 1.0 for(i = 1; i ≤ N+1; ++i) { for(state = 0; state< K*; ++state) { pobs = pemission(obsi | state) for(old = 0; old < K*; ++old) { pmove = ptransition(state| old) if(v[i-1][old] * pobs * pmove > v[i][state]) { v[i][state] = v[i-1][old] * pobs * pmove b[i][state] = old } } } } Q: How do we return the most likely tag sequence?

A: (ti)i, where ti-1 = b[i][ti]

Viterbi Algorithm in Log-Space

v = double[N+2][K*] b = int[N+2][K*] v[0][*] = -∞ v[0][START] = 0.0 for(i = 1; i ≤ N+1; ++i) { for(state = 0; state< K*; ++state) { pobs = log pemission(obsi | state) for(old = 0; old < K*; ++old) { pmove = log ptransition(state| old) if(v[i-1][old] + pobs + pmove > v[i][state]) { v[i][state] = v[i-1][old] + pobs + pmove b[i][state] = old } } } }

Forward vs. Viterbi

α = double[N+2][K*] α[0][*] = 0.0 α[0][START] = 1.0 for(i = 1; i ≤ N+1; ++i) { for(state= 0; state< K*; ++state) { pobs = pemission(obsi | state) for(old = 0; old < K*; ++old) { pmove = ptransition(state| old) α[i][state] += α[i-1][old] * pobs * pmove } } } v = double[N+2][K*] b = int[N+2][K*] v[0][*] = 0.0 v[0][START] = 1.0 for(i = 1; i ≤ N+1; ++i) { for(state= 0; state< K*; ++state) { pobs = pemission(obsi | state) for(old = 0; old < K*; ++old) { pmove = ptransition(state| old) if(v[i-1][old] * pobs * pmove > v[i][state]) { v[i][state] = v[i-1][old] * pobs * pmove b[i][state] = old } } } }

Forward vs. Viterbi

α = double[N+2][K*] α[0][*] = 0.0 α[0][START] = 1.0 for(i = 1; i ≤ N+1; ++i) { for(state= 0; state< K*; ++state) { pobs = pemission(obsi | state) for(old = 0; old < K*; ++old) { pmove = ptransition(state| old) α[i][state] += α[i-1][old] * pobs * pmove } } } v = double[N+2][K*] b = int[N+2][K*] v[0][*] = 0.0 v[0][START] = 1.0 for(i = 1; i ≤ N+1; ++i) { for(state= 0; state< K*; ++state) { pobs = pemission(obsi | state) for(old = 0; old < K*; ++old) { pmove = ptransition(state| old) if(v[i-1][old] * pobs * pmove > v[i][state]) { v[i][state] = v[i-1][old] * pobs * pmove b[i][state] = old } } } }

Hidden Markov Model Tasks

Calculate the (log) likelihood of an observed sequence w1, …, wN Calculate the most likely sequence of states (for an

• bserved sequence)

Learn the emission and transition parameters

𝑞 𝑨1, 𝑥1, 𝑨2, 𝑥2, … , 𝑨𝑂, 𝑥𝑂 = 𝑞 𝑨1| 𝑨0 𝑞 𝑥1|𝑨1 ⋯ 𝑞 𝑨𝑂| 𝑨𝑂−1 𝑞 𝑥𝑂|𝑨𝑂 = ෑ

𝑗

𝑞 𝑥𝑗|𝑨𝑗 𝑞 𝑨𝑗| 𝑨𝑗−1

emission probabilities/parameters transition probabilities/parameters

“The farther backward you can look, the farther forward you can see.”

commonly attributed to Winston Churchill

HMM Probabilities

Forward Values α(i, s) is the total probability of all paths: 1. that start from the beginning 2. that end (currently) in s at step i 3. that emit the observation

• bs at i

𝛽 𝑗, 𝑡 = ෍

𝑡′

𝛽 𝑗 − 1, 𝑡′ ∗ 𝑞 𝑡 𝑡′) ∗ 𝑞(obs at 𝑗 | 𝑡)

HMM Probabilities

Forward Values α(i, s) is the total probability of all paths: 1. that start from the beginning 2. that end (currently) in s at step i 3. that emit the observation

• bs at i

Backward Values β(i, s) is the total probability of all paths: 1. that start at step i at state s 2. that terminate at the end

3. (that emit the observation obs at i+1) 𝛽 𝑗, 𝑡 = ෍

𝑡′

𝛽 𝑗 − 1, 𝑡′ ∗ 𝑞 𝑡 𝑡′) ∗ 𝑞(obs at 𝑗 | 𝑡) 𝛾 𝑗, 𝑡 = ෍

𝑡′

𝛾 𝑗 + 1, 𝑡′ ∗ 𝑞(𝑡′|𝑡) ∗ 𝑞 obs at 𝑗 + 1 𝑡′)

Backward Algorithm

β: a 2D table, (N+2) x K* N+2: number of observations (+2 for the BOS & EOS symbols) K*: number of states Use dynamic programming to build the β right- to-left

Backward Algorithm

β = double[N+2][K*] β[n+1][END] = 1.0 for(i = N; i ≥ 0; --i) { for(next = 0; next < K*; ++next) { pobs = pemission(obsi+1 | next) for(state = 0; state < K*; ++state) { pmove = ptransition(next | state) β[i][state] += β[i+1][next] * pobs * pmove } } }

Backward Algorithm

β = double[N+2][K*] β[n+1][END] = 1.0 for(i = N; i ≥ 0; --i) { for(next = 0; next < K*; ++next) { pobs = pemission(obsi+1 | next) for(state = 0; state < K*; ++state) { pmove = ptransition(next | state) β[i][state] += β[i+1][next] * pobs * pmove } } } Q: What does β[0][START] represent?

Backward Algorithm

β = double[N+2][K*] β[n+1][END] = 1.0 for(i = N; i ≥ 0; --i) { for(next = 0; next < K*; ++next) { pobs = pemission(obsi+1 | next) for(state = 0; state < K*; ++state) { pmove = ptransition(next | state) β[i][state] += β[i+1][next] * pobs * pmove } } } Q: What does β[0][START] represent? A: Total probability of all paths from stop to start, for the

• bserved sequence

Backward Algorithm

β = double[N+2][K*] β[n+1][END] = 1.0 for(i = N; i ≥ 0; --i) { for(next = 0; next < K*; ++next) { pobs = pemission(obsi+1 | next) for(state = 0; state < K*; ++state) { pmove = ptransition(next | state) β[i][state] += β[i+1][next] * pobs * pmove } } } Q: What does β[0][START] represent? A: The marginal likelihood of the observed sequence

Backward Algorithm

β = double[N+2][K*] β[n+1][END] = 1.0 for(i = N; i ≥ 0; --i) { for(next = 0; next < K*; ++next) { pobs = pemission(obsi+1 | next) for(state = 0; state < K*; ++state) { pmove = ptransition(next | state) β[i][state] += β[i+1][next] * pobs * pmove } } } Q: What does β[0][START] represent? A: α[N+1][END]

2 (3) -State HMM Likelihood with Backward Probabilities

w3 w4

𝑞 𝑥3|𝑂 𝑞 𝑥4|𝑂

z3 = N z4 = N z3 = V z4 = V 𝑞 𝑂| 𝑂 z1 = N

w1 w2 w3 w4

𝑞 𝑥1|𝑂 𝑞 𝑥4|𝑂

𝑞 𝑂| start z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞 𝑊| 𝑊 𝑞 𝑊| 𝑂 𝑞 𝑂| 𝑊

𝑞 𝑥3|𝑊 𝑞 𝑥2|𝑊

𝑞 𝑂| 𝑊

β[1, N] = β[2, N] * (.15*.2) + β[2, V] * (.8*.6) β[2, V] = β[3, N] * (.6*.05) + β[3, V] * (.35*.1) β[3, V] = β[4, V] * (.35*.1)+ β[4, N] * (.6*.05) β[3, N] = β[4, V] * (.8*.1) + β[4, N] * (.15*.05)

N V end start .7 .2 .1 N .15 .8 .05 V .6 .35 .05 w1 w2 w3 w4 N .7 .2 .05 .05 V .2 .6 .1 .1

𝑞 𝑊| start

β[1, V] = β[2, N] * (.6*.2) + β[2, V] * (.35*.6) β[2, N] = β[3, N] * (.15*.05) + β[3, V] * (.8*.1)

Why Do We Need Backward Values?

zi-1 = C zi-1 = B zi-1 = A zi = C zi = B zi = A zi+1 = C zi+1 = B zi+1 = A

α(i, s) is the total probability of all paths: 1. that start from the beginning 2. that end (currently) in s at step i 3. that emit the observation obs at i β(i, s) is the total probability of all paths: 1. that start at step i at state s 2. that terminate at the end

3. (that emit the observation obs at i+1)

Why Do We Need Backward Values?

zi-1 = C zi-1 = B zi-1 = A zi = C zi = B zi = A zi+1 = C zi+1 = B zi+1 = A

α(i, s) is the total probability of all paths: 1. that start from the beginning 2. that end (currently) in s at step i 3. that emit the observation obs at i β(i, s) is the total probability of all paths: 1. that start at step i at state s 2. that terminate at the end

3. (that emit the observation obs at i+1)

α(i, B) β(i, B)

Why Do We Need Backward Values?

zi-1 = C zi-1 = B zi-1 = A zi = C zi = B zi = A zi+1 = C zi+1 = B zi+1 = A

α(i, s) is the total probability of all paths: 1. that start from the beginning 2. that end (currently) in s at step i 3. that emit the observation obs at i β(i, s) is the total probability of all paths: 1. that start at step i at state s 2. that terminate at the end

3. (that emit the observation obs at i+1)

α(i, B) β(i, B) α(i, B) * β(i, B) = total probability of paths through state B at step i

Why Do We Need Backward Values?

zi-1 = C zi-1 = B zi-1 = A zi = C zi = B zi = A zi+1 = C zi+1 = B zi+1 = A

α(i, s) is the total probability of all paths: 1. that start from the beginning 2. that end (currently) in s at step i 3. that emit the observation obs at i β(i, s) is the total probability of all paths: 1. that start at step i at state s 2. that terminate at the end

3. (that emit the observation obs at i+1)

α(i, B) β(i, B) α(i, s) * β(i, s) = total probability of paths through state s at step i

we can compute posterior state probabilities

(normalize by marginal likelihood)

Why Do We Need Backward Values?

zi-1 = C zi-1 = B zi-1 = A zi = C zi = B zi = A

α(i, s) is the total probability of all paths: 1. that start from the beginning 2. that end (currently) in s at step i 3. that emit the observation obs at i β(i, s) is the total probability of all paths: 1. that start at step i at state s 2. that terminate at the end

3. (that emit the observation obs at i+1)

α(i, B) β(i+1, s)

zi+1 = C zi+1 = B zi+1 = A

Why Do We Need Backward Values?

zi-1 = C zi-1 = B zi-1 = A zi = C zi = B zi = A

α(i, B) β(i+1, s’)

zi+1 = C zi+1 = B zi+1 = A

α(i, s) is the total probability of all paths: 1. that start from the beginning 2. that end (currently) in s at step i 3. that emit the observation obs at i β(i, s) is the total probability of all paths: 1. that start at step i at state s 2. that terminate at the end

3. (that emit the observation obs at i+1)

α(i, B) * p(s’ | B) * p(obs at i+1 | s’) * β(i+1, s’) = total probability of paths through the B→s’ arc (at time i)

Why Do We Need Backward Values?

zi-1 = C zi-1 = B zi-1 = A zi = C zi = B zi = A

α(i, B) β(i+1, s’)

zi+1 = C zi+1 = B zi+1 = A

α(i, s) is the total probability of all paths: 1. that start from the beginning 2. that end (currently) in s at step i 3. that emit the observation obs at i β(i, s) is the total probability of all paths: 1. that start at step i at state s 2. that terminate at the end

3. (that emit the observation obs at i+1)

we can compute posterior transition probabilities

(normalize by marginal likelihood)

α(i, B) * p(s’ | B) * p(obs at i+1 | s’) * β(i+1, s’) = total probability of paths through the B→s’ arc (at time i)

With Both Forward and Backward Values

α(i, s) * β(i, s) = total probability of paths through state s at step i α(i, s) * p(s’ | B) * p(obs at i+1 | s’) * β(i+1, s’) = total probability of paths through the s→s’ arc (at time i)

With Both Forward and Backward Values

α(i, s) * β(i, s) = total probability of paths through state s at step i α(i, s) * p(s’ | B) * p(obs at i+1 | s’) * β(i+1, s’) = total probability of paths through the s→s’ arc (at time i)

With Both Forward and Backward Values

α(i, s) * p(s’ | B) * p(obs at i+1 | s’) * β(i+1, s’) = total probability of paths through the s→s’ arc (at time i) α(i, s) * β(i, s) = total probability of paths through state s at step i

𝑞 𝑨𝑗 = 𝑡, 𝑨𝑗+1 = 𝑡′ 𝑥1,⋯ , 𝑥𝑂) =

Expectation Maximization (EM)

• 0. Assume some value for your parameters

Two step, iterative algorithm

• 1. E-step: count under uncertainty, assuming these

parameters

• 2. M-step: maximize log-likelihood, assuming these

uncertain counts

estimated counts

Expectation Maximization (EM)

• 0. Assume some value for your parameters

Two step, iterative algorithm

• 1. E-step: count under uncertainty, assuming these

parameters

• 2. M-step: maximize log-likelihood, assuming these

uncertain counts

estimated counts

pobs(w | s) ptrans(s’ | s)

Expectation Maximization (EM)

• 0. Assume some value for your parameters

Two step, iterative algorithm

• 1. E-step: count under uncertainty, assuming these

parameters

• 2. M-step: maximize log-likelihood, assuming these

uncertain counts

estimated counts

pobs(w | s) ptrans(s’ | s)

𝑞∗ 𝑨𝑗 = 𝑡 𝑥1,⋯ , 𝑥𝑂) = 𝛽 𝑗, 𝑡 ∗ 𝛾(𝑗, 𝑡) 𝛽(𝑂 + 1, END) 𝑞∗ 𝑨𝑗 = 𝑡, 𝑨𝑗+1 = 𝑡′ 𝑥

1, ⋯ , 𝑥𝑂) =

𝛽 𝑗, 𝑡 ∗ 𝑞 𝑡′ 𝑡 ∗ 𝑞 obs𝑗+1 𝑡′ ∗ 𝛾(𝑗 + 1, 𝑡′) 𝛽(𝑂 + 1, END)

M-Step

“maximize log-likelihood, assuming these uncertain counts”

if we observed the hidden transitions…

M-Step

“maximize log-likelihood, assuming these uncertain counts”

we don’t the hidden transitions, but we can approximatelycount

M-Step

“maximize log-likelihood, assuming these uncertain counts”

we don’t the hidden transitions, but we can approximatelycount

we compute these in the E-step, with

• ur α and β values

Estimating Parameters from Unobserved Data

N V end start N V w1 w2 W3 w4 N V Expected Transition Counts Expected Emission Counts

end emission not shown

z1 = N

w1 w2 w3 w4

𝑞

∗ 𝑥1|𝑂

𝑞

∗ 𝑥2|𝑂

𝑞

∗ 𝑥3|𝑂

𝑞

∗ 𝑥4|𝑂

𝑞

∗ 𝑂| start

z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞

∗ 𝑊| 𝑊

𝑞

∗ 𝑊| start

𝑞

∗ 𝑊| 𝑂

𝑞

∗ 𝑊| 𝑂

𝑞

∗ 𝑊| 𝑂

𝑞

∗ 𝑂| 𝑊

𝑞

∗ 𝑂| 𝑊

𝑞

∗ 𝑂| 𝑊

𝑞

∗ 𝑥4|𝑊

𝑞

∗ 𝑥3|𝑊

𝑞

∗ 𝑥2|𝑊

𝑞

∗ 𝑥1|𝑊

𝑞

∗ 𝑂| 𝑂

𝑞

∗ 𝑂| 𝑂

𝑞

∗ 𝑂| 𝑂

𝑞

∗ 𝑊| 𝑊

𝑞

∗ 𝑊| 𝑊

Estimating Parameters from Unobserved Data

N V end start N V w1 w2 W3 w4 N V Expected Transition Counts Expected Emission Counts

end emission not shown

z1 = N

w1 w2 w3 w4

𝑞

∗ 𝑥1|𝑂

𝑞

∗ 𝑥2|𝑂

𝑞

∗ 𝑥3|𝑂

𝑞

∗ 𝑥4|𝑂

𝑞

∗ 𝑂| start

z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞

∗ 𝑊| 𝑊

𝑞

∗ 𝑊| start

𝑞

∗ 𝑊| 𝑂

𝑞

∗ 𝑊| 𝑂

𝑞

∗ 𝑊| 𝑂

𝑞

∗ 𝑂| 𝑊

𝑞

∗ 𝑂| 𝑊

𝑞

∗ 𝑂| 𝑊

𝑞

∗ 𝑥4|𝑊

𝑞

∗ 𝑥3|𝑊

𝑞

∗ 𝑥2|𝑊

𝑞

∗ 𝑥1|𝑊

𝑞

∗ 𝑂| 𝑂

𝑞

∗ 𝑂| 𝑂

𝑞

∗ 𝑂| 𝑂

𝑞

∗ 𝑊| 𝑊

𝑞

∗ 𝑊| 𝑊

all of these p* arcs are specific to a time-step

all of these p* arcs are specific to a time-step

Estimating Parameters from Unobserved Data

N V end start N V w1 w2 W3 w4 N V Expected Transition Counts Expected Emission Counts

end emission not shown

z1 = N

w1 w2 w3 w4

𝑞

∗ 𝑥1|𝑂

𝑞

∗ 𝑥2|𝑂

𝑞

∗ 𝑥3|𝑂

𝑞

∗ 𝑥4|𝑂

𝑞

∗ 𝑂| start

z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞

∗ 𝑊| 𝑊

=.5 𝑞

∗ 𝑊| start

𝑞

∗ 𝑊| 𝑂

𝑞

∗ 𝑊| 𝑂

𝑞

∗ 𝑊| 𝑂

𝑞

∗ 𝑂| 𝑊

𝑞

∗ 𝑂| 𝑊

𝑞

∗ 𝑂| 𝑊

𝑞

∗ 𝑥4|𝑊

𝑞

∗ 𝑥3|𝑊

𝑞

∗ 𝑥2|𝑊

𝑞

∗ 𝑥1|𝑊

𝑞

∗ 𝑂| 𝑂

=.4 𝑞

∗ 𝑂| 𝑂

=.6 𝑞

∗ 𝑂| 𝑂

=.5 𝑞

∗ 𝑊| 𝑊

=.3 𝑞

∗ 𝑊| 𝑊

=.3

all of these p* arcs are specific to a time-step

Estimating Parameters from Unobserved Data

N V end start N 1.5 V 1.1 w1 w2 W3 w4 N V Expected Transition Counts Expected Emission Counts

end emission not shown

z1 = N

w1 w2 w3 w4

𝑞

∗ 𝑥1|𝑂

𝑞

∗ 𝑥2|𝑂

𝑞

∗ 𝑥3|𝑂

𝑞

∗ 𝑥4|𝑂

𝑞

∗ 𝑂| start

z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞

∗ 𝑊| start

𝑞

∗ 𝑊| 𝑂

𝑞

∗ 𝑊| 𝑂

𝑞

∗ 𝑊| 𝑂

𝑞

∗ 𝑂| 𝑊

𝑞

∗ 𝑂| 𝑊

𝑞

∗ 𝑂| 𝑊

𝑞

∗ 𝑥4|𝑊

𝑞

∗ 𝑥3|𝑊

𝑞

∗ 𝑥2|𝑊

𝑞

∗ 𝑥1|𝑊

𝑞

∗ 𝑊| 𝑊

𝑞

∗ 𝑂| 𝑂

𝑞

∗ 𝑂| 𝑂

𝑞

∗ 𝑂| 𝑂

𝑞

∗ 𝑊| 𝑊

𝑞

∗ 𝑊| 𝑊

𝑞

∗ 𝑊| 𝑊

=.5 𝑞

∗ 𝑂| 𝑂

=.4 𝑞

∗ 𝑂| 𝑂

=.6 𝑞

∗ 𝑂| 𝑂

=.5 𝑞

∗ 𝑊| 𝑊

=.3 𝑞

∗ 𝑊| 𝑊

=.3

Estimating Parameters from Unobserved Data

N V end start 1.8 .1 .1 N 1.5 .8 .1 V 1.4 1.1 .4 w1 w2 W3 w4 N .4 .3 .2 .2 V .1 .6 .3 .3 Expected Transition Counts Expected Emission Counts

end emission not shown

z1 = N

w1 w2 w3 w4

𝑞

∗ 𝑥1|𝑂

𝑞

∗ 𝑥2|𝑂

𝑞

∗ 𝑥3|𝑂

𝑞

∗ 𝑥4|𝑂

𝑞

∗ 𝑂| start

z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞

∗ 𝑊| start

𝑞

∗ 𝑊| 𝑂

𝑞

∗ 𝑊| 𝑂

𝑞

∗ 𝑊| 𝑂

𝑞

∗ 𝑂| 𝑊

𝑞

∗ 𝑂| 𝑊

𝑞

∗ 𝑂| 𝑊

𝑞

∗ 𝑥4|𝑊

𝑞

∗ 𝑥3|𝑊

𝑞

∗ 𝑥2|𝑊

𝑞

∗ 𝑥1|𝑊

𝑞

∗ 𝑊| 𝑊

𝑞

∗ 𝑂| 𝑂

𝑞

∗ 𝑂| 𝑂

𝑞

∗ 𝑂| 𝑂

𝑞

∗ 𝑊| 𝑊

𝑞

∗ 𝑊| 𝑊

(these numbers are made up)

Estimating Parameters from Unobserved Data

N V end start 1.8/2 .1/2 .1/2 N 1.5/ 2.4 .8/ 2.4 .1/ 2.4 V 1.4/2.9 1.1/ 2.9 .4/ 2.9 w1 w2 W3 w4 N .4/ 1.1 .3/ 1.1 .2/ 1.1 .2/ 1.1 V .1/ 1.3 .6/ 1.3 .3/ 1.3 .3/ 1.3

Expected Transition MLE Expected Emission MLE

end emission not shown

z1 = N

w1 w2 w3 w4

𝑞

∗ 𝑥1|𝑂

𝑞

∗ 𝑥2|𝑂

𝑞

∗ 𝑥3|𝑂

𝑞

∗ 𝑥4|𝑂

𝑞

∗ 𝑂| start

z2 = N z3 = N z4 = N z1 = V z2 = V z3 = V z4 = V 𝑞

∗ 𝑊| start

𝑞

∗ 𝑊| 𝑂

𝑞

∗ 𝑊| 𝑂

𝑞

∗ 𝑊| 𝑂

𝑞

∗ 𝑂| 𝑊

𝑞

∗ 𝑂| 𝑊

𝑞

∗ 𝑂| 𝑊

𝑞

∗ 𝑥4|𝑊

𝑞

∗ 𝑥3|𝑊

𝑞

∗ 𝑥2|𝑊

𝑞

∗ 𝑥1|𝑊

𝑞

∗ 𝑊| 𝑊

𝑞

∗ 𝑂| 𝑂

𝑞

∗ 𝑂| 𝑂

𝑞

∗ 𝑂| 𝑂

𝑞

∗ 𝑊| 𝑊

𝑞

∗ 𝑊| 𝑊

(these numbers are made up)

EM For HMMs (Baum-Welch Algorithm)

α = computeForwards() β = computeBackwards() L = α[N+1][END] for(i = N; i ≥ 0; --i) { for(next = 0; next < K*; ++next) { cobs(obsi+1 | next) += α[i+1][next]* β[i+1][next]/L for(state = 0; state < K*; ++state) { u = pobs(obsi+1 | next) * ptrans(next | state) ctrans(next| state) += α[i][state] * u * β[i+1][next]/L } } }

Expectation Maximization (EM)

• 0. Assume some value for your parameters

Two step, iterative algorithm

• 1. E-step: count under uncertainty, assuming these

parameters

• 2. M-step: maximize log-likelihood, assuming these

uncertain counts

estimated counts

pobs(w | s) ptrans(s’ | s)

𝑞∗ 𝑨𝑗 = 𝑡 𝑥1,⋯ , 𝑥𝑂) = 𝛽 𝑗, 𝑡 ∗ 𝛾(𝑗, 𝑡) 𝛽(𝑂 + 1, END) 𝑞∗ 𝑨𝑗 = 𝑡, 𝑨𝑗+1 = 𝑡′ 𝑥

1, ⋯ , 𝑥𝑂) =

𝛽 𝑗, 𝑡 ∗ 𝑞 𝑡′ 𝑡 ∗ 𝑞 obs𝑗+1 𝑡′ ∗ 𝛾(𝑗 + 1, 𝑡′) 𝛽(𝑂 + 1, END)

Baum-Welch

Semi-Supervised Learning

      ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

labeled data:

• human annotated
• relatively small/few

examples unlabeled data:

• raw; not annotated
• plentiful

Semi-Supervised Learning

      ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

labeled data:

• human annotated
• relatively small/few

examples unlabeled data:

• raw; not annotated
• plentiful

EM

Semi-Supervised Learning

      ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

labeled data:

• human annotated
• relatively small/few

examples unlabeled data:

• raw; not annotated
• plentiful

Semi-Supervised Learning

      ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

labeled data:

• human annotated
• relatively small/few

examples unlabeled data:

• raw; not annotated
• plentiful

?

Semi-Supervised Parameter Estimation

N V end start 2 N 1 2 2 V 2 1 w1 w2 W3 w4 N 2 1 2 V 2 1 Transition Counts Emission Counts

Semi-Supervised Parameter Estimation

N V end start 1.8 .1 .1 N 1.5 .8 .1 V 1.4 1.1 .4 w1 w2 W3 w4 N .4 .3 .2 .2 V .1 .6 .3 .3 Expected Transition Counts Expected Emission Counts N V end start 2 N 1 2 2 V 2 1 w1 w2 W3 w4 N 2 1 2 V 2 1 Transition Counts Emission Counts

Semi-Supervised Parameter Estimation

N V end start 1.8 .1 .1 N 1.5 .8 .1 V 1.4 1.1 .4 w1 w2 W3 w4 N .4 .3 .2 .2 V .1 .6 .3 .3 Expected Transition Counts Expected Emission Counts N V end start 2 N 1 2 2 V 2 1 w1 w2 W3 w4 N 2 1 2 V 2 1 Transition Counts Emission Counts

N V end start 1.8 .1 .1 N 1.5 .8 .1 V 1.4 1.1 .4 w1 w2 W3 w4 N .4 .3 .2 .2 V .1 .6 .3 .3 Expected Transition Counts Expected Emission Counts N V end start 2 N 1 2 2 V 2 1 w1 w2 W3 w4 N 2 1 2 V 2 1 Transition Counts Emission Counts

Semi-Supervised Parameter Estimation

N V end start 3.8 .1 .1 N 2.5 2.8 2.1 V 3.4 2.1 .4 w1 w2 W3 w4 N 2.4 .3 1.2 2.2 V .1 2.6 1.3 .3 Mixed Transition Counts Mixed Emission Counts

Agenda

HMM Motivation (Part of Speech) and Brief Definition What is Part of Speech? HMM Detailed Definition HMM Tasks