HMM: Particle filters Lirong Xia Recap: Reasoning over Time Markov - - PowerPoint PPT Presentation

Lirong Xia

HMM: Particle filters

Recap: Reasoning over Time

2

ØMarkov models p(X1) p(X|X-1)

• Hidden Markov models

p(E|X)

X E p

rain umbrella 0.9 rain no umbrella 0.1 sun umbrella 0.2 sun no umbrella 0.8

• Filtering: Given time t and evidences e1,…,et,

compute p(Xt|e1:t)

Filtering algorithm

3

ØNotation

• B(Xt-1)=p(Xt-1|e1:t-1)
• B’(Xt)=p(Xt|e1:t-1)

ØEach time step, we start with p(Xt-1 | previous evidence): ØElapse of time B’(Xt)=Σxt-1p(Xt|xt-1)B(xt-1) ØObserve B(Xt) ∝p(et|Xt)B’(Xt) ØRenormalize B(Xt)

ØParticle filtering ØViterbi algorithm

4

Today

Particle Filtering

5

ØSometimes |X| is too big to use exact inference

• |X| may be too big to even store

B(X)

• E.g. X is continuous

ØSolution: approximate inference

• Track samples of X, not all values
• Samples are called particles
• Time per step is linear in the

number of samples

• But: number needed may be large
• In memory: list of particles

ØThis is how robot localization works in practice

Representation: Particles

6

Øp(X) is now represented by a list

• f N particles (samples)
• Generally, N << |X|
• Storing map from X to counts would

defeat the point

Øp(x) approximated by number of particles with value x

• Many x will have p(x)=0
• More particles, more accuracy

ØFor now, all particles have a weight of 1

Particles: (3,3) (2,3) (3,3) (3,2) (3,3) (3,2) (2,1) (3,3) (3,3) (2,1)

Particle Filtering: Elapse Time

7

ØEach particle is moved by sampling its next position from the transition model x’= sample(p(X’|x))

• Samples’ frequencies reflect the

transition probabilities

ØThis captures the passage of time

• If we have enough samples, close to

the exact values before and after (consistent)

Particle Filtering: Observe

8

ØSlightly trickier:

• Likelihood weighting
• Note that, as before, the probabilities

don’t sum to one, since most have been downweighted

( ) ( ) ( ) ( ) ( )

| | ' w x p e x B X p e X B X = ∝

Particle Filtering: Resample

9

Ø Rather than tracking weighted samples, we resample Ø N times, we choose from our weighted sample distribution (i.e. draw with replacement) Ø This is analogous to renormalizing the distribution Ø Now the update is complete for this time step, continue with the next one

Old Particles: (3,3) w=0.1 (2,1) w=0.9 (2,1) w=0.9 (3,1) w=0.4 (3,2) w=0.3 (2,2) w=0.4 (1,1) w=0.4 (3,1) w=0.4 (2,1) w=0.9 (3,2) w=0.3 New Particles: (2,1) w=1 (2,1) w=1 (2,1) w=1 (3,2) w=1 (2,2) w=1 (2,1) w=1 (1,1) w=1 (3,1) w=1 (2,1) w=1 (1,1) w=1

ØElapse of time

B’(Xt)=Σxt-1p(Xt|xt-1)B(xt-1) ØObserve B(Xt) ∝p(et|Xt)B’(Xt)

ØRenormalize

B(xt) sum up to 1

10

Forward algorithm vs. particle filtering

Forward algorithm Particle filtering

• Elapse of time

x--->x’

• Observe

w(x’)=p(et|x)

• Resample

resample N particles

Robot Localization

11

Ø In robot localization:

• We know the map, but not the robot’s position
• Observations may be vectors of range finder readings
• State space and readings are typically continuous (works

basically like a very fine grid) and so we cannot store B(X)

• Particle filtering is a main technique

SLAM

12

ØSLAM = Simultaneous Localization And Mapping

• We do not know the map or our location
• Our belief state is over maps and positions!
• Main techniques:
• Kalman filtering (Gaussian HMMs)
• particle methods
• http://www.cs.duke.edu/~parr/dpslam/D-Wing.html

HMMs: MLE Queries

13

ØHMMs defined by:

• States X
• Observations E
• Initial distribution: p(X1)
• Transitions: p(X|X-1)
• Emissions: p(E|X)

ØQuery: most likely explanation:

( )

1:

1: 1:

argmax |

t

t t x

p x e

State Path

14

Ø Graph of states and transitions over time Ø Each arc represents some transition xt-1→xt Ø Each arc has weight p(xt|xt-1)p(et|xt) Ø Each path is a sequence of states Ø The product of weights on a path is the seq’s probability Ø Forward algorithm

• computing the sum of all paths

Ø Viterbi algorithm

• computing the best paths

X1 X2 … XN

Viterbi Algorithm

15

x1:T

* = argmax x1:T

p x1:T | e1:T

( )

mt xt ! " # $= max

x1:t−1 p x1:t−1,xt,e1:t

( )

= max

x1:t−1 p x1:t−1,e1:t−1

( ) p xt | xt−1 ( ) p et | xt ( )

= p et | xt

( )max

xt−1 p xt | xt−1

( )max

x1:t−2 p x1:t−1,e1:t−1

( )

= p et | xt

( )max

xt−1 p xt | xt−1

( )mt−1 xt−1

! " # $

Example

16

X E p

+r +u 0.9 +r

• u

0.1

• r

+u 0.2

• r
• u

0.8