Advanced Algorithms (XIII) Shanghai Jiao Tong University Chihao - - PowerPoint PPT Presentation

Advanced Algorithms (XIII)

Shanghai Jiao Tong University

Chihao Zhang

June 1, 2020

Total Variation Distance

Total Variation Distance

Let and be two distributions on

μ ν Ω

Total Variation Distance

Let and be two distributions on

μ ν Ω

Their total variation distance is

Total Variation Distance

Let and be two distributions on

μ ν Ω

Their total variation distance is dTV(μ, ν) = 1 2 ∑

x∈Ω

μ(x) − ν(x) = max

A⊆Ω μ(A) − ν(A)

Total Variation Distance

Let and be two distributions on

μ ν Ω

Their total variation distance is dTV(μ, ν) = 1 2 ∑

x∈Ω

μ(x) − ν(x) = max

A⊆Ω μ(A) − ν(A)

A

μ ν

Total Variation Distance

Let and be two distributions on

μ ν Ω

Their total variation distance is dTV(μ, ν) = 1 2 ∑

x∈Ω

μ(x) − ν(x) = max

A⊆Ω μ(A) − ν(A)

A

μ ν

• distance scaled by

ℓ1 1 2

Coupling

Coupling

Let and be two distributions on

μ ν Ω

Coupling

Let and be two distributions on

μ ν Ω

A coupling of and is a joint distribution on such that:

μ ν ω Ω × Ω

Coupling

Let and be two distributions on

μ ν Ω

A coupling of and is a joint distribution on such that:

μ ν ω Ω × Ω

∀x ∈ Ω, μ(x) = ∑

y∈Ω

ω(x, y)

Coupling

Let and be two distributions on

μ ν Ω

A coupling of and is a joint distribution on such that:

μ ν ω Ω × Ω

∀x ∈ Ω, μ(x) = ∑

y∈Ω

ω(x, y) ∀y ∈ Ω, ν(x) = ∑

x∈Ω

ω(x, y)

Coupling Lemma

Coupling Lemma

Let be a coupling of and

ω μ ν

Coupling Lemma

Let be a coupling of and

ω μ ν

and

(X, Y) ∼ ω ⟹ X ∼ μ Y ∼ ν

Coupling Lemma

Let be a coupling of and

ω μ ν

and

(X, Y) ∼ ω ⟹ X ∼ μ Y ∼ ν

Then Pr

(X,Y)∼ω [X ≠ Y] ≥ dTV(μ, ν)

Coupling Lemma

Let be a coupling of and

ω μ ν

and

(X, Y) ∼ ω ⟹ X ∼ μ Y ∼ ν

Then Pr

(X,Y)∼ω [X ≠ Y] ≥ dTV(μ, ν)

Moreover, there exists such that

ω*

Coupling Lemma

Let be a coupling of and

ω μ ν

and

(X, Y) ∼ ω ⟹ X ∼ μ Y ∼ ν

Then Pr

(X,Y)∼ω [X ≠ Y] ≥ dTV(μ, ν)

Moreover, there exists such that

ω*

Pr

(X,Y)∼ω* [X ≠ Y] = dTV(μ, ν)

Proof of Coupling Lemma

Proof of Coupling Lemma

For finite , designing a coupling is equivalent to filling a matrix so that the marginals are correct

Ω Ω × Ω

Proof of Coupling Lemma

For finite , designing a coupling is equivalent to filling a matrix so that the marginals are correct

Ω Ω × Ω

Ω = {1,2}, μ = (1/2,1/2), ν = (1/3,2/3)

Proof of Coupling Lemma

For finite , designing a coupling is equivalent to filling a matrix so that the marginals are correct

Ω Ω × Ω

Ω = {1,2}, μ = (1/2,1/2), ν = (1/3,2/3)

μ ν

1 2 1 2 1 3 2 3

Proof of Coupling Lemma

For finite , designing a coupling is equivalent to filling a matrix so that the marginals are correct

Ω Ω × Ω

Ω = {1,2}, μ = (1/2,1/2), ν = (1/3,2/3)

μ ν

1 2 1 2 1 3 2 3 1 3

Proof of Coupling Lemma

For finite , designing a coupling is equivalent to filling a matrix so that the marginals are correct

Ω Ω × Ω

Ω = {1,2}, μ = (1/2,1/2), ν = (1/3,2/3)

μ ν

1 2 1 2 1 3 2 3 1 3 1 2

Proof of Coupling Lemma

For finite , designing a coupling is equivalent to filling a matrix so that the marginals are correct

Ω Ω × Ω

Ω = {1,2}, μ = (1/2,1/2), ν = (1/3,2/3)

μ ν

1 2 1 2 1 3 2 3 1 3 1 2

Proof of Coupling Lemma

For finite , designing a coupling is equivalent to filling a matrix so that the marginals are correct

Ω Ω × Ω

Ω = {1,2}, μ = (1/2,1/2), ν = (1/3,2/3)

μ ν

1 2 1 2 1 3 2 3 1 3 1 2 1 6

Proof of Coupling Lemma

For finite , designing a coupling is equivalent to filling a matrix so that the marginals are correct

Ω Ω × Ω

Ω = {1,2}, μ = (1/2,1/2), ν = (1/3,2/3)

μ ν

1 2 1 2 1 3 2 3 1 3 1 2 1 6

is the one maximizing the sum of diagonals

ω*

Coupling of Markov Chains

Coupling of Markov Chains

Consider two copies of the chain :

P

Coupling of Markov Chains

Consider two copies of the chain :

P

• The initial distribution is

and

• and

μ0 ν0 μT

t = μT 0 Pt

νT

t = νT 0 Pt

Coupling of Markov Chains

Consider two copies of the chain :

P

A coupling of the two chains is joint distribution

• f

and satisfying the following conditions

ω {μt}t≥0 {νt}t≥0

• The initial distribution is

and

• and

μ0 ν0 μT

t = μT 0 Pt

νT

t = νT 0 Pt

is a pair of processes such that

{(Xt, Yt)}t≥0 ∼ ω

is a pair of processes such that

{(Xt, Yt)}t≥0 ∼ ω

∀a, b ∈ Ω, Pr[Xt+1 = b ∣ Xt = a] = P(a, b)

is a pair of processes such that

{(Xt, Yt)}t≥0 ∼ ω

∀a, b ∈ Ω, Pr[Xt+1 = b ∣ Xt = a] = P(a, b) ∀a, b ∈ Ω, Pr[Yt+1 = b ∣ Xt = a] = P(a, b)

is a pair of processes such that

{(Xt, Yt)}t≥0 ∼ ω

∀a, b ∈ Ω, Pr[Xt+1 = b ∣ Xt = a] = P(a, b) ∀a, b ∈ Ω, Pr[Yt+1 = b ∣ Xt = a] = P(a, b)

Marginally, and are both chain

{Xt} {Yt} P

is a pair of processes such that

{(Xt, Yt)}t≥0 ∼ ω

∀a, b ∈ Ω, Pr[Xt+1 = b ∣ Xt = a] = P(a, b) ∀a, b ∈ Ω, Pr[Yt+1 = b ∣ Xt = a] = P(a, b)

Marginally, and are both chain

{Xt} {Yt} P

∀t ≥ 0, Xt = Yt ⟹ Xt′ = Yt′ for all t′ > t

is a pair of processes such that

{(Xt, Yt)}t≥0 ∼ ω

∀a, b ∈ Ω, Pr[Xt+1 = b ∣ Xt = a] = P(a, b) ∀a, b ∈ Ω, Pr[Yt+1 = b ∣ Xt = a] = P(a, b)

Marginally, and are both chain

{Xt} {Yt} P

∀t ≥ 0, Xt = Yt ⟹ Xt′ = Yt′ for all t′ > t

Two chains coalesce once they meet

Fundamental Theorem via Coupling

Fundamental Theorem via Coupling

If a finite chain is irreducible and aperiodic, then it has a unique stationary distribution . Moreover, for any initial distribution , it holds that

P π μ

lim

t→∞ μTPt = πT

Fundamental Theorem via Coupling

If a finite chain is irreducible and aperiodic, then it has a unique stationary distribution . Moreover, for any initial distribution , it holds that

P π μ

lim

t→∞ μTPt = πT

Consider two chains and

{Xt}t≥0 {Yt}t≥0

Fundamental Theorem via Coupling

If a finite chain is irreducible and aperiodic, then it has a unique stationary distribution . Moreover, for any initial distribution , it holds that

P π μ

lim

t→∞ μTPt = πT

Consider two chains and

{Xt}t≥0 {Yt}t≥0

• ,

for arbitrary

X0 ∼ π Y0 ∼ μ0 μ0

Fundamental Theorem via Coupling

If a finite chain is irreducible and aperiodic, then it has a unique stationary distribution . Moreover, for any initial distribution , it holds that

P π μ

lim

t→∞ μTPt = πT

Consider two chains and

{Xt}t≥0 {Yt}t≥0

• ,

for arbitrary

X0 ∼ π Y0 ∼ μ0 μ0

• A coupling where

and run independently

Xt Yt

irreducible + aperiodic

⟹ ∃t, ∀x, y, Pt(x, y) > 0

irreducible + aperiodic

⟹ ∃t, ∀x, y, Pt(x, y) > 0

Then for any , there exists some s.t.

z ∈ Ω θ > 0

irreducible + aperiodic

⟹ ∃t, ∀x, y, Pt(x, y) > 0

Pr[Xt = Yt] ≥ Pr[Xt = Yt = z] = Pr[Xt = z] ⋅ Pr[Yt = z] = π(z) ⋅ Pt(Y0, z) ≥ θ > 0

Then for any , there exists some s.t.

z ∈ Ω θ > 0

irreducible + aperiodic

⟹ ∃t, ∀x, y, Pt(x, y) > 0

Pr[Xt = Yt] ≥ Pr[Xt = Yt = z] = Pr[Xt = z] ⋅ Pr[Yt = z] = π(z) ⋅ Pt(Y0, z) ≥ θ > 0

Then for any , there exists some s.t.

z ∈ Ω θ > 0

Pr[Xt ≠ Yt] ≤ 1 − θ < 1

irreducible + aperiodic

⟹ ∃t, ∀x, y, Pt(x, y) > 0

Pr[Xt = Yt] ≥ Pr[Xt = Yt = z] = Pr[Xt = z] ⋅ Pr[Yt = z] = π(z) ⋅ Pt(Y0, z) ≥ θ > 0

Then for any , there exists some s.t.

z ∈ Ω θ > 0

Pr[X2t ≠ Y2t] = Pr[X2t ≠ Y2t ∧ Xt = Yt] + Pr[X2t ≠ Y2t ∧ Xt ≠ Yt] = Pr[X2t ≠ Y2t ∣ Xt ≠ Yt] ⋅ Pr[Xt ≠ Yt] ≤ (1 − θ)2 Pr[Xt ≠ Yt] ≤ 1 − θ < 1

irreducible + aperiodic

⟹ ∃t, ∀x, y, Pt(x, y) > 0

Pr[Xt = Yt] ≥ Pr[Xt = Yt = z] = Pr[Xt = z] ⋅ Pr[Yt = z] = π(z) ⋅ Pt(Y0, z) ≥ θ > 0

Then for any , there exists some s.t.

z ∈ Ω θ > 0

Pr[X2t ≠ Y2t] = Pr[X2t ≠ Y2t ∧ Xt = Yt] + Pr[X2t ≠ Y2t ∧ Xt ≠ Yt] = Pr[X2t ≠ Y2t ∣ Xt ≠ Yt] ⋅ Pr[Xt ≠ Yt] ≤ (1 − θ)2 Pr[Xt ≠ Yt] ≤ 1 − θ < 1

…

irreducible + aperiodic

⟹ ∃t, ∀x, y, Pt(x, y) > 0

Pr[Xt = Yt] ≥ Pr[Xt = Yt = z] = Pr[Xt = z] ⋅ Pr[Yt = z] = π(z) ⋅ Pt(Y0, z) ≥ θ > 0

Then for any , there exists some s.t.

z ∈ Ω θ > 0

Pr[X2t ≠ Y2t] = Pr[X2t ≠ Y2t ∧ Xt = Yt] + Pr[X2t ≠ Y2t ∧ Xt ≠ Yt] = Pr[X2t ≠ Y2t ∣ Xt ≠ Yt] ⋅ Pr[Xt ≠ Yt] ≤ (1 − θ)2 Pr[Xt ≠ Yt] ≤ 1 − θ < 1

…

Pr[Xkt ≠ Ykt] ≤ (1 − θ)k

irreducible + aperiodic

⟹ ∃t, ∀x, y, Pt(x, y) > 0

Pr[Xt = Yt] ≥ Pr[Xt = Yt = z] = Pr[Xt = z] ⋅ Pr[Yt = z] = π(z) ⋅ Pt(Y0, z) ≥ θ > 0

Then for any , there exists some s.t.

z ∈ Ω θ > 0

Pr[X2t ≠ Y2t] = Pr[X2t ≠ Y2t ∧ Xt = Yt] + Pr[X2t ≠ Y2t ∧ Xt ≠ Yt] = Pr[X2t ≠ Y2t ∣ Xt ≠ Yt] ⋅ Pr[Xt ≠ Yt] ≤ (1 − θ)2 Pr[Xt ≠ Yt] ≤ 1 − θ < 1

…

Pr[Xkt ≠ Ykt] ≤ (1 − θ)k

lim

n→∞ Pr[Xn ≠ Yn] = 0

Mixing Time

Mixing Time

The mixing time is the the first time such that the total variation distance between and is at most , for any initial

τmix(ε) t Xt π ε X0

Mixing Time

The mixing time is the the first time such that the total variation distance between and is at most , for any initial

τmix(ε) t Xt π ε X0

τmix(ε) = max

μ0

min

t≥0 dTV(μT 0 Pt, π) ≤ ε

Mixing Time

The mixing time is the the first time such that the total variation distance between and is at most , for any initial

τmix(ε) t Xt π ε X0

τmix(ε) = max

μ0

min

t≥0 dTV(μT 0 Pt, π) ≤ ε

τmix = τmix(1/4)

Random Walk on Hyper Cube

Random Walk on Hyper Cube

• V = {0,1}n

Random Walk on Hyper Cube

• V = {0,1}n
• iff

x ∼ y ∥x − y∥1 = 1

Random Walk on Hyper Cube

• V = {0,1}n
• iff

x ∼ y ∥x − y∥1 = 1

Standing at

• with prob. , do nothing
• otherwise, choose

u.a.r and flip

x ∈ {0,1}n 1 2 i ∈ [n] x(i)

Lazy walk on G

The chain is equivalent to

The chain is equivalent to

• choose

and u.a.r.

• change

i ∈ [n] b ∈ {0,1} x(i) ← b

The chain is equivalent to

• choose

and u.a.r.

• change

i ∈ [n] b ∈ {0,1} x(i) ← b

Let and be two walks

Xt Yt

The chain is equivalent to

• choose

and u.a.r.

• change

i ∈ [n] b ∈ {0,1} x(i) ← b

Let and be two walks

Xt Yt

We couple them by choosing the same and

i b

What is the probability that ?

Xt ≠ Yt

What is the probability that ?

Xt ≠ Yt

Coupon Collector!

What is the probability that ?

Xt ≠ Yt

Coupon Collector! If , then

t ≥ n log n + cn Pr[Xt ≠ Yt] ≤ e−c

What is the probability that ?

Xt ≠ Yt

Coupon Collector! If , then

t ≥ n log n + cn Pr[Xt ≠ Yt] ≤ e−c

Coupling lemma implies that

What is the probability that ?

Xt ≠ Yt

Coupon Collector! If , then

t ≥ n log n + cn Pr[Xt ≠ Yt] ≤ e−c

Coupling lemma implies that τmix(ε) ≤ n log n + n log ε−1

Another Random Walk

Another Random Walk

Standing at

• with prob.

, do nothing

• otherwise, choose

u.a.r and flip

x ∈ {0,1}n 1 n + 1 i ∈ [n] x(i)

Lazy walk on G

Another Random Walk

Standing at

• with prob.

, do nothing

• otherwise, choose

u.a.r and flip

x ∈ {0,1}n 1 n + 1 i ∈ [n] x(i)

Lazy walk on G

A coupling argument implies τmix ≤ 1

2 n log n + O(n)

Reversible Chain

Reversible Chain

Recall that we say a Markov chain is reversible with respect to if

P π

Reversible Chain

Recall that we say a Markov chain is reversible with respect to if

P π

∀x, y ∈ Ω, π(x)P(x, y) = π(y)P(y, x)

Reversible Chain

Recall that we say a Markov chain is reversible with respect to if

P π

∀x, y ∈ Ω, π(x)P(x, y) = π(y)P(y, x) Then is a stationary distribution of

π P

Reversible Chain

Recall that we say a Markov chain is reversible with respect to if

P π

∀x, y ∈ Ω, π(x)P(x, y) = π(y)P(y, x) Then is a stationary distribution of

π P

We showed that spectral decomposition is a powerful tool to analyze reversible chains

Relaxation Time

Relaxation Time

Relaxation Time

λ⋆ := max

1≤i≤n−1 |λi|

Relaxation Time

λ⋆ := max

1≤i≤n−1 |λi|

τrel := 1 1 − λ⋆

Relaxation Time

λ⋆ := max

1≤i≤n−1 |λi|

τrel := 1 1 − λ⋆ For reversible, irreducible, aperiodic chains:

Relaxation Time

λ⋆ := max

1≤i≤n−1 |λi|

τrel := 1 1 − λ⋆ For reversible, irreducible, aperiodic chains: (τrel − 1)log ( 1 2ε) ≤ τmix(ε) ≤ τrel log ( 1 επmin)

Relaxation Time

λ⋆ := max

1≤i≤n−1 |λi|

τrel := 1 1 − λ⋆ For reversible, irreducible, aperiodic chains: (τrel − 1)log ( 1 2ε) ≤ τmix(ε) ≤ τrel log ( 1 επmin)

πmin := min

x π(x)