Pr Probability obability an and d Ti Time: me: Hidden dden - - PowerPoint PPT Presentation

▶
pr probability obability an and d ti time me
SMART_READER_LITE
LIVE PREVIEW

Pr Probability obability an and d Ti Time: me: Hidden dden - - PowerPoint PPT Presentation

Feb 05, 2023 526 likes •804 views

Pr Probability obability an and d Ti Time: me: Hidden dden Markov arkov Model odels s (H (HMMs) MMs) Computer Co ter Sc Scienc nce e cpsc sc322 322, , Lectu cture e 32 (Te Text xtbo book ok Chpt 6.5.2 5.2) Nov, No ,

slide-1
SLIDE 1

CPSC 322, Lecture 32 Slide 1

Pr Probability

  • bability an

and d Ti Time: me: Hidden dden Markov arkov Model

  • dels

s (H (HMMs) MMs)

Co Computer ter Sc Scienc nce e cpsc sc322 322, , Lectu cture e 32 (Te Text xtbo book

  • k Chpt 6.5.2

5.2)

No Nov, , 25, 2013

slide-2
SLIDE 2

CPSC 322, Lecture 32 Slide 2

Lecture cture Ov Overview rview

  • Re

Recap ap

  • Markov Models
  • Markov Chain
  • Hidden

dden Markov kov Models dels

slide-3
SLIDE 3

CPSC 322, Lecture 32 Slide 3

Ans nswer wering ing Que ueries ies un unde der Unc ncertainty ertainty

Stati atic c Belief f Ne Netwo work rk

& Va & Variable iable El Elimination ination Dyna namic ic Bayesi esian an Network

  • rk

Probabi

  • babili

lity ty Theor eory Hidde den n Mark arkov

  • v Mode

dels ls Email il spam pam filter ers Diagn gnostic

  • stic

Syst stem ems s (e.g. g., medic dicine) ine) Natural ural Langua nguage ge Proc

  • cessing

ssing Student dent Tracing acing in tutori

  • ring

ng Sy Syst stem ems Monit nitori

  • ring

ng (e.g .g cred edit car ards) ds) BioInf nform

  • rmatics

ics Mar arkov kov Chains ains Robot botics ics

slide-4
SLIDE 4

Sta tatio tiona nary ry Ma Markov kov Cha hain in (SMC) MC)

A stati ation

  • nary

ry Markov

  • v Chain : for all t >0
  • P (St+1| S0,…,St) =

and

  • P (St +1|

We only need to specify and

  • Simple Model, easy to specify
  • Often the natural model
  • The network can extend indefinitely
  • Variati

ation

  • ns

s of SMC C are at the core e of most t Na Natural ral Language ge Proce cessi ssing ng (NL NLP) ) applicat ations! ns!

Slide 4 CPSC 322, Lecture 32

slide-5
SLIDE 5

CPSC 322, Lecture 32 Slide 5

Lecture cture Ov Overview rview

  • Re

Recap ap

  • Markov Models
  • Markov Chain
  • Hidden

dden Markov kov Models dels

slide-6
SLIDE 6

How

  • w can

an we e mi mini nima mally lly ex exte tend nd Ma Marko rkov v Cha hain ins? s?

  • Mainta

tain inin ing the Markov

  • v and stat

atio iona nary ry assumptions? A useful situation to model is the one in which:

  • the reasoning system does not have

e acce cess ss to the states

  • but can make obser

erva vatio tions ns that give some information about the current state

Slide 6 CPSC 322, Lecture 32

slide-7
SLIDE 7

CPSC 322, Lecture 32 Slide 7

B.

  • B. h

h x h

  • A. 2

2 x h C . C . k k x h D.

  • D. k

k x k

slide-8
SLIDE 8

CPSC 322, Lecture 32 Slide 8

Hidden Markov Model

  • P (S0) specifies initial conditions
  • P (St+1|St) specifies the dynamics
  • P (Ot |St) specifies the sensor model
  • A Hi

Hidden Markov kov Model l (HM HMM) M) starts with a Markov chain, and adds a noisy observation about the state at each time step:

  • |domain(S)| = k
  • |domain(O)| = h
  • B. h

h x h

  • A. 2

2 x h C . C . k k x h D.

  • D. k

k x k

slide-9
SLIDE 9

CPSC 322, Lecture 32 Slide 9

Hidden Markov Model

  • P (S0) specifies initial conditions
  • P (St+1|St) specifies the dynamics
  • P (Ot |St) specifies the sensor model
  • A Hi

Hidden Markov kov Model l (HM HMM) M) starts with a Markov chain, and adds a noisy observation about the state at each time step:

  • |domain(S)| = k
  • |domain(O)| = h
slide-10
SLIDE 10

CPSC 322, Lecture 32 Slide 10

Example: mple: Localization for “Pushed around” Robot

  • Locali

aliza zatio tion (where am I?) is a fundamental problem in roboti

  • tics

cs

  • Suppose a robot is in a circular corridor with 16

locations

  • There are four

r doors s at positions: 2, 4, 7, 11

  • The Robot initially doesn’t know where it is
  • The Robot is pushed

ed around. After a push it can stay in the same location, move left or right.

  • The Robot has a No

Noisy sensor

  • r telling whether it is in front
  • f a door
slide-11
SLIDE 11

This scenario can be represented as…

  • Examp

mple e Stoch chastic astic Dy Dynami mics cs: when pushed, it stays in the same location p=0.2, moves one step left or right with equal probability P(Loct + 1 | Loc t) Loc t= 10 B. A. C. C.

slide-12
SLIDE 12

CPSC 322, Lecture 32 Slide 12

This scenario can be represented as…

  • Examp

mple e Stoch chastic astic Dy Dynami mics cs: when pushed, it stays in the same location p=0.2, moves left or right with equal probability P(Loct + 1 | Loc t) P(Loc1)

slide-13
SLIDE 13

CPSC 322, Lecture 32 Slide 13

This scenario can be represented as…

Examp mple e of No Noisy senso sor r telling whether it is in front of a door.

  • If it is in front of a door P(O t = T) = .8
  • If not in front of a door P(O t = T) = .1

P(O t | Loc t)

slide-14
SLIDE 14

Useful eful in infe ference ence in in HM HMMs Ms

  • Local

aliz izati ation

  • n: Robot starts at an unknown

location and it is pushed around t times. It wants to determine where it is

  • In genera

ral: l: compute the posterior distribution over the current state given all evidence to date P(St | O0 … Ot)

Slide 14 CPSC 322, Lecture 32

slide-15
SLIDE 15

CPSC 322, Lecture 32 Slide 15

Example mple : : Rob

  • bot
  • t Lo

Local alizati ization

  • n
  • Suppose a robot wants to determine its location based on its

actions and its sensor readings

  • Three actions: goRight, goLeft, Stay
  • This can be represented by an augmented HMM
slide-16
SLIDE 16

CPSC 322, Lecture 32 Slide 16

Rob

  • bot
  • t Lo

Local aliz ization ation Sen ensor

  • r an

and D d Dynamics namics Mo Mode del

  • Sa

Sample le Se Sensor r Model l (assume same as for pushed around)

  • Sa

Sample le St Stochastic chastic Dy Dynami mics cs: P(Loct + 1 | Actiont , Loc t)

P(Loct + 1 = L | Action t = goRight , Loc t = L) = 0.1 P(Loct + 1 = L+1 | Action t = goRight , Loc t = L) = 0.8 P(Loct + 1 = L + 2 | Action t = goRight , Loc t = L) = 0.074 P(Loct + 1 = L’ | Action t = goRight , Loc t = L) = 0.002 for all other locations L’

  • All location arithmetic is modulo 16
  • The action goLeft works the same but to the left
slide-17
SLIDE 17

CPSC 322, Lecture 32 Slide 17

Dyna namics mics Mo Mode del Mo More e Det etai ails ls

  • Sample

e Stoch chastic astic Dy Dynami mics cs: P(Loct + 1 | Action, Loc t)

P(Loct + 1 = L | Action t = goRight , Loc t = L) = 0.1 P(Loct + 1 = L+1 | Action t = goRight , Loc t = L) = 0.8 P(Loct + 1 = L + 2 | Action t = goRight , Loc t = L) = 0.074 P(Loct + 1 = L’ | Action t = goRight , Loc t = L) = 0.002 for all other locations L’

slide-18
SLIDE 18

CPSC 322, Lecture 32 Slide 18

Rob

  • bot
  • t Lo

Local aliz izatio ation n ad addi diti tion

  • nal

al sen enso sor

  • Additi

tion

  • nal

al Light Sensor:

  • r: there is light coming through an
  • pening at location 10

P (Lt | Loct)

  • Info

fo fro rom m the two wo sensors

  • rs is combined

ined :“Sensor Fusion”

slide-19
SLIDE 19

CPSC 322, Lecture 32 Slide 19

Th The Ro Robot t starts rts at an unkno nown wn locati ation

  • n and must

t deter ermin ine where it is The model appears to be too ambiguous

  • Sensors are too noisy
  • Dynamics are too stochastic to infer anything

http://www.cs.ubc.ca/spider/poole/demos/localization /localization.html

But inference actually works pretty well. Let’s check:

You can use standard Bnet inference. However you typically take advantage of the fact that time moves forward (not in 322)

slide-20
SLIDE 20

CPSC 322, Lecture 32 Slide 20

Sampl mple e scenari enario

  • to

to explore plore in demo mo

  • Keep making observations without moving. What

happens?

  • Then keep moving without making observations.

What happens?

  • Assume you are at a certain position alternate

moves and observations

  • ….
slide-21
SLIDE 21

CPSC 322, Lecture 32 Slide 21

HMMs have many other applications….

Natura ral l Langua uage ge Pr Proces essi sing ng: : e.g., Speech Recognition

  • States:

phoneme \ word

  • Observations: acoustic signal \ phoneme

Bi Bioinfo nform rmatics atics: Gene Finding

  • States: coding / non-coding region
  • Observations: DNA Sequences

Fo For thes ese e problem lems s the critic itical al infere erenc nce e is: :

find the most likely sequence of states given a sequence of observations

slide-22
SLIDE 22

CPSC 322, Lecture 32 Slide 22

Markov kov Models dels

Markov Chains Hidden Markov Model Markov Decision Processes (MDPs) Simplest Possible Dynamic Bnet Add noisy Observations about the state at time t Add Actions and Values (Rewards)

slide-23
SLIDE 23

CPSC 322, Lecture 32 Slide 23

Learning Goals for today’s class

Yo You u can an:

  • Specify the components of an Hidden Markov

Model (HMM)

  • Justify and apply HMMs to Robot Localization

Clarifi ifica catio ion n on secon

  • nd LG for last

st class ss

Yo You can:

  • Justify and apply Markov Chains to compute the probability
  • f a Natural Language sentence (NOT to estimate the

conditional probs- slide 18)

slide-24
SLIDE 24

CPSC 322, Lecture 32 Slide 24

Next xt week ek

Enviro ronm nmen ent Pr Problem em

Query ry Planning De Determini rminist stic Stocha chast stic Search ch Arc Co Consisten stency cy Search ch Search ch Valu lue Iterat teration ion Var. . Eliminat ation Constr trai aint nt Satisfactio sfaction Logic ics STRI RIPS Belief f Nets Vars s + Co Constr traints aints De Decision

  • n Ne

Nets

Mar arkov kov Decisi cision

  • n Proc
  • cess

esses es

Var. . Eliminat ation Stati atic Sequenti ntial al Re Representatio sentation Re Reasoning ing Techniqu nique SLS

Mar arkov kov Chains ains and HMMs Ms

slide-25
SLIDE 25

CPSC 322, Lecture 32 Slide 25

Next xt Class ass

  • One

ne-off

  • ff de

decis isions ions(TextBook 9.2)

  • Sin

ingl gle e Sta tage ge Dec ecision ision ne netw tworks

  • rks ( 9.2.1)
slide-26
SLIDE 26

CPSC 322, Lecture 1 Slide 26

Pe People

  • ple

Instru tructo tor

  • Giuse

sepp ppe e Carenini ( carenini@cs.ubc.ca; office CICSR 105)

Te Teachin hing g As Assist istant ants

  • Kamya

yar Ardekan kani kamya amyar.arde r.ardekan kany@gmail.com y@gmail.com

  • Tatsuro

suro Oya toya@cs.ub a@cs.ubc. c.ca ca

  • Xin Ru

Ru (Na Nancy) y) Wang nancywa ywang ng19 1991@ 91@yahoo. yahoo.ca ca