Advanced Algorithms (XII) Shanghai Jiao Tong University Chihao - - PowerPoint PPT Presentation

Advanced Algorithms (XII)

Shanghai Jiao Tong University

Chihao Zhang

May 25, 2020

Random Walk on a Graph

Random Walk on a Graph

1 3 2

1 2 1 8 1 4 3 8 1 3 2 3 3 4

Random Walk on a Graph

P = [pij]1≤i,j≤n =

1 2 3 8 1 8 1 3 2 3 1 3 3 4

1 3 2

1 2 1 8 1 4 3 8 1 3 2 3 3 4

Random Walk on a Graph

P = [pij]1≤i,j≤n =

1 2 3 8 1 8 1 3 2 3 1 3 3 4

pij = Pr[Xt+1 = j ∣ Xt = i]

1 3 2

1 2 1 8 1 4 3 8 1 3 2 3 3 4

Random Walk on a Graph

P = [pij]1≤i,j≤n =

1 2 3 8 1 8 1 3 2 3 1 3 3 4

pij = Pr[Xt+1 = j ∣ Xt = i] ∀t ≥ 0, μT

t = μT 0 Pt

1 3 2

1 2 1 8 1 4 3 8 1 3 2 3 3 4

Random Walk on a Graph

P = [pij]1≤i,j≤n =

1 2 3 8 1 8 1 3 2 3 1 3 3 4

pij = Pr[Xt+1 = j ∣ Xt = i] Stationary distribution :

π πTP = πT

∀t ≥ 0, μT

t = μT 0 Pt

1 3 2

1 2 1 8 1 4 3 8 1 3 2 3 3 4

Fundamental Theorem of Markov Chains

Fundamental Theorem of Markov Chains

We study a few basic questions regarding a chain:

Fundamental Theorem of Markov Chains

We study a few basic questions regarding a chain:

• Does a stationary distribution always exist?

Fundamental Theorem of Markov Chains

We study a few basic questions regarding a chain:

• Does a stationary distribution always exist?
• If so, is the stationary distribution unique?

Fundamental Theorem of Markov Chains

We study a few basic questions regarding a chain:

• Does a stationary distribution always exist?
• If so, is the stationary distribution unique?
• If so, does any initial distribution converge to it?

Existence of Stationary Distribution

Existence of Stationary Distribution

Yes, any Markov chain has a stationary distribution

Existence of Stationary Distribution

Yes, any Markov chain has a stationary distribution

Perron-Frobenius

Any positive matrix matrix has a positive real eigenvalue with . Moreover, its eigenvector is positive.

n × n A λ ρ(A) = λ

Existence of Stationary Distribution

Yes, any Markov chain has a stationary distribution

Perron-Frobenius

Any positive matrix matrix has a positive real eigenvalue with . Moreover, its eigenvector is positive.

n × n A λ ρ(A) = λ λ(PT) = λ(P) = 1

Existence of Stationary Distribution

Yes, any Markov chain has a stationary distribution

Perron-Frobenius

Any positive matrix matrix has a positive real eigenvalue with . Moreover, its eigenvector is positive.

n × n A λ ρ(A) = λ λ(PT) = λ(P) = 1 The positive eigenvector is π

Uniqueness and Convergence

Uniqueness and Convergence

1 2

1 − p p 1 − q q

Uniqueness and Convergence

P = [ 1 − p p q 1 − q]

1 2

1 − p p 1 − q q

Uniqueness and Convergence

P = [ 1 − p p q 1 − q] is a stationary dist. of

π = ( q p + q, p p + q )

T

P

1 2

1 − p p 1 − q q

Uniqueness and Convergence

P = [ 1 − p p q 1 − q] is a stationary dist. of

π = ( q p + q, p p + q )

T

P

Start from an arbitrary μ0 = (μ(1), μ(2))

T

1 2

1 − p p 1 − q q

Uniqueness and Convergence

P = [ 1 − p p q 1 − q] is a stationary dist. of

π = ( q p + q, p p + q )

T

P

Start from an arbitrary μ0 = (μ(1), μ(2))

T

Compute ∥μT

0 Pt − πT∥

1 2

1 − p p 1 − q q

Δt = |μt(1) − π(1)|

Δt = |μt(1) − π(1)| Δt+1 = μt+1(1) − q p + q = μt(1 − p) + (1 − μt(1))q − q p + q = (1 − p − q) μt(1) − q p + q = (1 − p − q) ⋅ Δt

Δt = |μt(1) − π(1)| Δt+1 = μt+1(1) − q p + q = μt(1 − p) + (1 − μt(1))q − q p + q = (1 − p − q) μt(1) − q p + q = (1 − p − q) ⋅ Δt Since , there are two ways to prohibit :

• r

p, q ∈ [0,1] Δt → 0 p = q = 1 p = q = 0

p = q = 0

p = q = 0

1 2

1 1

p = q = 0

1 2

1 1

∀t, Δt = Δ0

p = q = 0

1 2

1 1

The graph is disconnected ∀t, Δt = Δ0

p = q = 0

1 2

1 1

The graph is disconnected The chain is called reducible ∀t, Δt = Δ0

p = q = 0

1 2

1 1

The graph is disconnected The chain is called reducible In this case, the stationary distribution is not unique ∀t, Δt = Δ0

p = q = 0

1 2

1 1

The graph is disconnected The chain is called reducible In this case, the stationary distribution is not unique

Chain = convex combination of small chains

∀t, Δt = Δ0

p = q = 0

1 2

1 1

The graph is disconnected The chain is called reducible In this case, the stationary distribution is not unique

Chain = convex combination of small chains

∀t, Δt = Δ0

Stationary distribution=convex combination of “small” distributions

p = q = 1

p = q = 1

1 2

1 1

p = q = 1

1 2

1 1

∀t, Δt = − Δt−1

p = q = 1

1 2

1 1

∀t, Δt = − Δt−1 The graph is bipartite

p = q = 1

1 2

1 1

∀t, Δt = − Δt−1 The graph is bipartite The chain is called periodic

p = q = 1

1 2

1 1

∀t, Δt = − Δt−1 The graph is bipartite The chain is called periodic Formally, ∃v, gcdC∈Cv|C| > 1

p = q = 1

1 2

1 1

∀t, Δt = − Δt−1 The graph is bipartite The chain is called periodic In this case, not all initial distribution converges to the stationary distribution Formally, ∃v, gcdC∈Cv|C| > 1

Fundamental Theorem of Markov Chains

Fundamental Theorem of Markov Chains

If a finite chain is irreducible and aperiodic, then it has a unique stationary distribution . Moreover, for any initial distribution , it holds that

P π μ

lim

t→∞ μTPt = πT

Fundamental Theorem of Markov Chains

If a finite chain is irreducible and aperiodic, then it has a unique stationary distribution . Moreover, for any initial distribution , it holds that

P π μ

lim

t→∞ μTPt = πT

(Show on board, see the note for details)

Reversible Chains

Reversible Chains

We study a special family of Markov chains called reversible chains

Reversible Chains

We study a special family of Markov chains called reversible chains Their transition graphs are undirected x → y ⟺ y → x

Reversible Chains

We study a special family of Markov chains called reversible chains Their transition graphs are undirected x → y ⟺ y → x A chain and a distribution satisfies detailed balance condition:

P π

Reversible Chains

We study a special family of Markov chains called reversible chains Their transition graphs are undirected x → y ⟺ y → x A chain and a distribution satisfies detailed balance condition:

P π

∀x, y ∈ V, π(x) ⋅ P(x, y) = π(y) ⋅ P(y, x)

Reversible Chains

We study a special family of Markov chains called reversible chains Their transition graphs are undirected x → y ⟺ y → x A chain and a distribution satisfies detailed balance condition:

P π

∀x, y ∈ V, π(x) ⋅ P(x, y) = π(y) ⋅ P(y, x) Then is a stationary distribution of

π P

We study reversible chains because

We study reversible chains because

• They are quite general. For any , one can define

an reversible whose stationary distribution is

π P π

We study reversible chains because

• They are quite general. For any , one can define

an reversible whose stationary distribution is

π P π

Helpful for Sampling

We study reversible chains because

• They are quite general. For any , one can define

an reversible whose stationary distribution is

π P π

Helpful for Sampling

• We have powerful tools (spectral method) to

analyze reversible chains

Spectral Decomposition Theorem

Spectral Decomposition Theorem

An symmetric matrix has real eigenvalues with corresponding eigenvectors which are orthogonal. Moreover, it holds that

n × n A n λ1, …, λn v1, …, vn

A = VΛVT

Spectral Decomposition Theorem

An symmetric matrix has real eigenvalues with corresponding eigenvectors which are orthogonal. Moreover, it holds that

n × n A n λ1, …, λn v1, …, vn

A = VΛVT where and

V = [v1, …, vn] Λ = diag(λ1, …, λn)

Spectral Decomposition Theorem

An symmetric matrix has real eigenvalues with corresponding eigenvectors which are orthogonal. Moreover, it holds that

n × n A n λ1, …, λn v1, …, vn

A = VΛVT where and

V = [v1, …, vn] Λ = diag(λ1, …, λn)

Equivalently, A =

n

∑

i=1

λivivT

i

Spectral Decomposition Theorem for Reversible Chains

Spectral Decomposition Theorem for Reversible Chains

is a stationary distribution of a reversible chain

π P

Spectral Decomposition Theorem for Reversible Chains

is a stationary distribution of a reversible chain

π P

Define an inner product

• n

:

⟨ ⋅ , ⋅ ⟩π ℝn

Spectral Decomposition Theorem for Reversible Chains

is a stationary distribution of a reversible chain

π P

Define an inner product

• n

:

⟨ ⋅ , ⋅ ⟩π ℝn

⟨x, y⟩π =

n

∑

i=1

π(i) ⋅ x(i) ⋅ y(i) = xTDπy,

Spectral Decomposition Theorem for Reversible Chains

is a stationary distribution of a reversible chain

π P

Define an inner product

• n

:

⟨ ⋅ , ⋅ ⟩π ℝn

⟨x, y⟩π =

n

∑

i=1

π(i) ⋅ x(i) ⋅ y(i) = xTDπy, where Dπ = diag(π1, …, πn)

Spectral Decomposition Theorem for Reversible Chains

is a stationary distribution of a reversible chain

π P

Define an inner product

• n

:

⟨ ⋅ , ⋅ ⟩π ℝn

⟨x, y⟩π =

n

∑

i=1

π(i) ⋅ x(i) ⋅ y(i) = xTDπy, where Dπ = diag(π1, …, πn) Consider the Hilbert space endowed with

ℝn ⟨ ⋅ , ⋅ ⟩π

Let be reversible with respect to . It has real eigenvalues with corresponding eigenvectors which are orthogonal in . Moreover

P ∈ ℝn×n π n λ1, …, λn v1, …, vn (ℝn, ⟨ ⋅ , ⟩π)

P =

n

∑

i=1

λivivT

i Dπ

Let be reversible with respect to . It has real eigenvalues with corresponding eigenvectors which are orthogonal in . Moreover

P ∈ ℝn×n π n λ1, …, λn v1, …, vn (ℝn, ⟨ ⋅ , ⟩π)

P =

n

∑

i=1

λivivT

i Dπ

Let be reversible with respect to . It has real eigenvalues with corresponding eigenvectors which are orthogonal in . Moreover

P ∈ ℝn×n π n λ1, …, λn v1, …, vn (ℝn, ⟨ ⋅ , ⟩π)

P =

n

∑

i=1

λivivT

i Dπ

• Proof. Reduce to the

symmetric case.

Properties of Eigenvalues

Properties of Eigenvalues

is a stationary distribution of a reversible chain

π P

Properties of Eigenvalues

is a stationary distribution of a reversible chain

π P

The eigenvalues of are

P λ1 ≤ λ2… ≤ λn

Properties of Eigenvalues

is a stationary distribution of a reversible chain

π P

The eigenvalues of are

P λ1 ≤ λ2… ≤ λn

• λn = 1

Properties of Eigenvalues

is a stationary distribution of a reversible chain

π P

The eigenvalues of are

P λ1 ≤ λ2… ≤ λn

• λn = 1
• and

if and only if is bipartite

λ1 ≥ − 1 λ1 = − 1 P

Properties of Eigenvalues

is a stationary distribution of a reversible chain

π P

The eigenvalues of are

P λ1 ≤ λ2… ≤ λn

• λn = 1
• and

if and only if is bipartite

λ1 ≥ − 1 λ1 = − 1 P

• if and only if is reducible

λn−1 = 1 P

Properties of Eigenvalues

is a stationary distribution of a reversible chain

π P

The eigenvalues of are

P λ1 ≤ λ2… ≤ λn

• λn = 1
• and

if and only if is bipartite

λ1 ≥ − 1 λ1 = − 1 P

• if and only if is reducible

λn−1 = 1 P

Proof next week!

P =

n

∑

i=1

λivivT

i Dπ

P =

n

∑

i=1

λivivT

i Dπ

Pt =

n

∑

i=1

λt

ivivT i Dπ

P =

n

∑

i=1

λivivT

i Dπ

Pt =

n

∑

i=1

λt

ivivT i Dπ

P =

n

∑

i=1

λivivT

i Dπ

Pt =

n

∑

i=1

λt

ivivT i Dπ

If is irreducible ( ) and aperiodic ( )

P λn−1 < 1 λ1 > − 1

P =

n

∑

i=1

λivivT

i Dπ

Pt =

n

∑

i=1

λt

ivivT i Dπ

If is irreducible ( ) and aperiodic ( )

P λn−1 < 1 λ1 > − 1

lim

t→∞ Pt = 11TDπ =

πT πT ⋮ πT