CSCE 471/871 Lecture 3: CSCE 471/871 Lecture 3: Markov Chains Markov Chains and and Hidden Markov Models Hidden Markov Models Stephen Scott Markov Chains Stephen Scott Hidden Markov Models Specifying an HMM sscott@cse.unl.edu 1 / 26
Outline CSCE 471/871 Lecture 3: Markov Chains and Hidden Markov chains Markov Models Hidden Markov models (HMMs) Stephen Scott Formal definition Markov Finding most probable state path (Viterbi algorithm) Chains Forward and backward algorithms Hidden Markov Specifying an HMM Models State sequence known Specifying an HMM State sequence unknown Structure 2 / 26
Markov Chains An Example: CpG Islands CSCE 471/871 Lecture 3: Markov Chains and Hidden Markov Focus on nucleotide sequences Models Stephen Scott The sequence “CG” (written “CpG”) tends to appear more frequently in some places than in others Markov Chains Such CpG islands are usually 10 2 – 10 3 bases long Hidden Markov Questions: Models Given a short segment, is it from a CpG island? 1 Specifying an HMM Given a long segment, where are its islands? 2 3 / 26
Modeling CpG Islands CSCE 471/871 Lecture 3: Markov Chains and Hidden Markov Models Model will be a CpG generator Stephen Scott Want probability of next symbol to depend on current Markov symbol Chains Hidden Will use a standard (non-hidden) Markov model Markov Models Probabilistic state machine Each state emits a symbol Specifying an HMM 4 / 26
Modeling CpG Islands (2) CSCE 471/871 Lecture 3: Markov Chains and Hidden A C Markov Models Stephen Scott Markov Chains P(A | T) Hidden Markov Models Specifying an HMM T G 5 / 26
The Markov Property CSCE 471/871 Lecture 3: Markov A first-order Markov model (what we study) has the Chains and property that observing symbol x i while in state π i Hidden Markov depends only on the previous state π i − 1 (which Models Stephen Scott generated x i − 1 ) Standard model has 1-1 correspondence between Markov Chains symbols and states, thus Hidden Markov Models P ( x i | x i − 1 , . . . , x 1 ) = P ( x i | x i − 1 ) Specifying an HMM and L � P ( x 1 , . . . , x L ) = P ( x 1 ) P ( x i | x i − 1 ) i = 2 6 / 26
Begin and End States CSCE For convenience, can add special “begin” ( B ) and “end” 471/871 Lecture 3: ( E ) states to clarify equations and define a distribution Markov Chains and over sequence lengths Hidden Markov Emit empty (null) symbols x 0 and x L + 1 to mark ends of Models sequence Stephen Scott Markov Chains Hidden A C Markov Models Specifying an E B HMM T G 7 / 26 L + 1 �
Markov Chains for Discrimination CSCE How do we use this to differentiate islands from 471/871 Lecture 3: non-islands? Markov Define two Markov models: islands (“ + ”) and Chains and Hidden non-islands (“ − ”) Markov Models Each model gets 4 states (A, C, G, T) Stephen Scott Take training set of known islands and non-islands Let c + st = number of times symbol t followed symbol s in Markov Chains an island: c + Hidden ˆ st P + ( t | s ) = Markov � t ′ c + Models st ′ Specifying an Example probabilities in [Durbin et al., p. 51] HMM Now score a sequence X = � x 1 , . . . , x L � by summing the log-odds ratios: � � � � L + 1 ˆ ˆ P + ( x i | x i − 1 ) P ( X | +) � log = log ˆ ˆ P ( X | − ) P − ( x i | x i − 1 ) i = 1 8 / 26
Hidden Markov Models CSCE Second CpG question: Given a long sequence, where 471/871 Lecture 3: are its islands? Markov Could use tools just presented by passing a fixed-width Chains and Hidden window over the sequence and computing scores Markov Models Trouble if islands’ lengths vary Stephen Scott Prefer single, unified model for islands vs. non-islands Markov Chains A C T G + + + + Hidden Markov [complete connectivity Models Definition between all pairs] Viterbi Forward/Backward A C T G - - - - Specifying an HMM Within the + group, transition probabilities similar to those for the separate + model, but there is a small chance of switching to a state in the − group 9 / 26
What’s Hidden in an HMM? CSCE 471/871 Lecture 3: Markov Chains and Hidden No longer have one-to-one correspondence between Markov Models states and emitted characters Stephen Scott E.g., was C emitted by C + or C − ? Markov Must differentiate the symbol sequence X from the Chains state sequence π = � π 1 , . . . , π L � Hidden Markov State transition probabilities same as before: Models P ( π i = ℓ | π i − 1 = j ) (i.e., P ( ℓ | j ) ) Definition Viterbi Now each state has a prob. of emitting any value: Forward/Backward P ( x i = x | π i = j ) (i.e., P ( x | j ) ) Specifying an HMM 10 / 26
What’s Hidden in an HMM? (2) CSCE 471/871 Lecture 3: Markov Chains and Hidden Markov Models Stephen Scott Markov Chains Hidden Markov Models Definition Viterbi Forward/Backward Specifying an HMM [In CpG HMM, emission probs discrete and = 0 or 1 ] 11 / 26
Example: The Occasionally Dishonest Casino CSCE Assume that a casino is typically fair, but with 471/871 Lecture 3: probability 0.05 it switches to a loaded die, and Markov Chains and switches back with probability 0.1 Hidden Markov Fair Loaded Models Stephen Scott 1: 1/6 1: 1/10 0.05 2: 1/6 2: 1/10 Markov Chains 3: 1/6 3: 1/10 Hidden Markov 4: 1/6 4: 1/10 Models Definition Viterbi 5: 1/6 5: 1/10 Forward/Backward 0.1 6: 1/6 6: 1/2 Specifying an HMM 0.95 0.9 Given a sequence of rolls, what’s hidden? 12 / 26
The Viterbi Algorithm CSCE 471/871 Lecture 3: Probability of seeing symbol sequence X and state Markov Chains and sequence π is Hidden Markov Models L � Stephen Scott P ( X , π ) = P ( π 1 | 0 ) P ( x i | π i ) P ( π i + 1 | π i ) i = 1 Markov Chains Can use this to find most likely path: Hidden Markov Models π ∗ = argmax Definition P ( X , π ) Viterbi π Forward/Backward Specifying an and trace it to identify islands (paths through “ + ” states) HMM There are an exponential number of paths through chain, so how do we find the most likely one? 13 / 26
The Viterbi Algorithm (2) CSCE Assume that we know (for all k ) v k ( i ) = probability of 471/871 Lecture 3: most likely path ending in state k with observation x i Markov Chains and Then Hidden Markov Models v ℓ ( i + 1 ) = P ( x i + 1 | ℓ ) max k { v k ( i ) P ( ℓ | k ) } Stephen Scott All states at i Markov Chains Hidden Markov Models State at l Definition +1 i Viterbi Forward/Backward l Specifying an HMM 14 / 26
The Viterbi Algorithm (3) CSCE 471/871 Lecture 3: Given the formula, can fill in table with dynamic Markov Chains and programming: Hidden Markov Models v 0 ( 0 ) = 1 , v k ( 0 ) = 0 for k > 0 Stephen Scott For i = 1 to L ; for ℓ = 1 to M (# states) Markov v ℓ ( i ) = P ( x i | ℓ ) max k { v k ( i − 1 ) P ( ℓ | k ) } Chains ptr i ( ℓ ) = argmax k { v k ( i − 1 ) P ( ℓ | k ) } Hidden Markov P ( X , π ∗ ) = max k { v k ( L ) P ( 0 | k ) } Models Definition π ∗ L = argmax k { v k ( L ) P ( 0 | k ) } Viterbi Forward/Backward For i = L to 1 Specifying an HMM π ∗ i − 1 = ptr i ( π ∗ i ) To avoid underflow, use log ( v ℓ ( i )) and add 15 / 26
The Forward Algorithm CSCE 471/871 Lecture 3: Given a sequence X , find P ( X ) = � π P ( X , π ) Markov Chains and Hidden Use dynamic programming like Viterbi, replacing max with Markov Models sum, and v k ( i ) with f k ( i ) = P ( x 1 , . . . , x i , π i = k ) (= prob. of Stephen Scott observed sequence through x i , stopping in state k ) Markov Chains f 0 ( 0 ) = 1 , f k ( 0 ) = 0 for k > 0 Hidden For i = 1 to L ; for ℓ = 1 to M (# states) Markov Models f ℓ ( i ) = P ( x i | ℓ ) � k f k ( i − 1 ) P ( ℓ | k ) Definition Viterbi P ( X ) = � Forward/Backward k f k ( L ) P ( 0 | k ) Specifying an HMM To avoid underflow, can again use logs, though exactness of results compromised (Section 3.6) 16 / 26
The Backward Algorithm CSCE Given a sequence X , find the probability that x i was emitted 471/871 Lecture 3: by state k , i.e., Markov Chains and Hidden P ( π i = k | X ) = P ( π i = k , X ) Markov Models P ( X ) Stephen Scott f k ( i ) b k ( i ) Markov � �� � � �� � Chains P ( x 1 , . . . , x i , π i = k ) P ( x i + 1 , . . . , x L | π i = k ) = Hidden P ( X ) Markov Models ���� computed by forward alg Definition Viterbi Forward/Backward Algorithm: Specifying an HMM b k ( L ) = P ( 0 | k ) for all k For i = L − 1 to 1 ; for k = 1 to M (# states) b k ( i ) = � ℓ P ( ℓ | k ) P ( x i + 1 | ℓ ) b ℓ ( i + 1 ) 17 / 26
Example Use of Forward/Backward Algorithm CSCE 471/871 Lecture 3: Markov Chains and Define g ( k ) = 1 if k ∈ { A + , C + , G + , T + } and 0 otherwise Hidden Markov Then G ( i | X ) = � Models k P ( π i = k | X ) g ( k ) = probability that x i is Stephen Scott in an island Markov For each state k , compute P ( π i = k | X ) with Chains forward/backward algorithm Hidden Markov Models Technique applicable to any HMM where set of states is Definition Viterbi partitioned into classes Forward/Backward Specifying an HMM Use to label individual parts of a sequence 18 / 26
Recommend
More recommend
