SLIDE 1
Advanced Algorithms (X)
Shanghai Jiao Tong University
Chihao Zhang
May 11, 2020
Advanced Algorithms (X) Shanghai Jiao Tong University Chihao Zhang - - PowerPoint PPT Presentation
Advanced Algorithms (X) Shanghai Jiao Tong University Chihao Zhang - - PowerPoint PPT Presentation
Advanced Algorithms (X) Shanghai Jiao Tong University Chihao Zhang May 11, 2020 Estimate Estimate One can design a Monte-Carlo algorithm to estimate the value of Estimate One can design a Monte-Carlo algorithm to estimate the value
SLIDE 2
SLIDE 3
Estimate π
One can design a Monte-Carlo algorithm to estimate the value of π
SLIDE 4
Estimate π
One can design a Monte-Carlo algorithm to estimate the value of π
SLIDE 5
Estimate π
One can design a Monte-Carlo algorithm to estimate the value of π
- Xi ∈ [−1,1] × [−1,1]
Zn =
n
∑
i=1
1[∥Xi∥ ≤ 1]
SLIDE 6
SLIDE 7
Xi ∼ Ber ( π 4 ), E[Zn] = π 4 ⋅ n
SLIDE 8
Xi ∼ Ber ( π 4 ), E[Zn] = π 4 ⋅ n
Therefore, by Chernoff bound
SLIDE 9
Xi ∼ Ber ( π 4 ), E[Zn] = π 4 ⋅ n
Therefore, by Chernoff bound Pr [ Zn − π 4 ⋅ n ≥ ε ⋅ π 4 ⋅ n] ≤ 2 exp (− ε2πn 12 )
SLIDE 10
Xi ∼ Ber ( π 4 ), E[Zn] = π 4 ⋅ n
Therefore, by Chernoff bound Pr [ Zn − π 4 ⋅ n ≥ ε ⋅ π 4 ⋅ n] ≤ 2 exp (− ε2πn 12 ) If , we have an approximation
- f with probability at least
n ≥ 12 ε2π log 2 δ 1 ± ε π 1 − δ
SLIDE 11
Rejection Sampling
SLIDE 12
Rejection Sampling
The method is often called rejection sampling
SLIDE 13
Rejection Sampling
The method is often called rejection sampling It is useful to estimate the size of some good sets in a large set
SLIDE 14
Rejection Sampling
The method is often called rejection sampling It is useful to estimate the size of some good sets in a large set A
B
SLIDE 15
Rejection Sampling
The method is often called rejection sampling It is useful to estimate the size of some good sets in a large set The number of samples is proportional to
|A| |B|
A
B
SLIDE 16
Counting DNF
SLIDE 17
Counting DNF
A DNF formula ,
φ = C1 ∨ C2 ∨ ⋯ ∨ Cm
SLIDE 18
Counting DNF
A DNF formula ,
φ = C1 ∨ C2 ∨ ⋯ ∨ Cm Ci =
ℓi
⋀
j=1
xij
SLIDE 19
Counting DNF
A DNF formula ,
φ = C1 ∨ C2 ∨ ⋯ ∨ Cm Ci =
ℓi
⋀
j=1
xij
SLIDE 20
Counting DNF
A DNF formula ,
φ = C1 ∨ C2 ∨ ⋯ ∨ Cm Ci =
ℓi
⋀
j=1
xij
B = satisfying assignments
SLIDE 21
Counting DNF
A DNF formula ,
φ = C1 ∨ C2 ∨ ⋯ ∨ Cm Ci =
ℓi
⋀
j=1
xij
B = satisfying assignments A = all assignments
SLIDE 22
Counting DNF
A DNF formula ,
φ = C1 ∨ C2 ∨ ⋯ ∨ Cm Ci =
ℓi
⋀
j=1
xij may contain only polynomial many solutions
φ
B = satisfying assignments A = all assignments
SLIDE 23
Counting DNF
A DNF formula ,
φ = C1 ∨ C2 ∨ ⋯ ∨ Cm Ci =
ℓi
⋀
j=1
xij may contain only polynomial many solutions
φ
The Monte Carlo method using rejection sampling is slow!
B = satisfying assignments A = all assignments
SLIDE 24
SLIDE 25
For each clause , define the set
Ci
SLIDE 26
For each clause , define the set
Ci
Si := the set of assignments satisfying Ci
SLIDE 27
For each clause , define the set
Ci
Si := the set of assignments satisfying Ci We want to estimate
⋃
1≤i≤m
Si
SLIDE 28
For each clause , define the set
Ci
Si := the set of assignments satisfying Ci We want to estimate
⋃
1≤i≤m
Si
SLIDE 29
For each clause , define the set
Ci
Si := the set of assignments satisfying Ci We want to estimate
⋃
1≤i≤m
Si
B = ⋃
1≤i≤m
Si
SLIDE 30
For each clause , define the set
Ci
Si := the set of assignments satisfying Ci We want to estimate
⋃
1≤i≤m
Si
B = ⋃
1≤i≤m
Si A =
⋅
⋃
1≤i≤m
Si
SLIDE 31
For each clause , define the set
Ci
Si := the set of assignments satisfying Ci We want to estimate
⋃
1≤i≤m
Si
B = ⋃
1≤i≤m
Si A =
⋅
⋃
1≤i≤m
Si
(disjoint union)
SLIDE 32
How about CNF?
SLIDE 33
How about CNF?
We consider a very special case: monotone 2-CNF
SLIDE 34
How about CNF?
We consider a very special case: monotone 2-CNF φ = (x ∨ y) ∧ (x ∨ z) ∧ (x ∨ w) ∧ (y ∨ w)
SLIDE 35
How about CNF?
We consider a very special case: monotone 2-CNF φ = (x ∨ y) ∧ (x ∨ z) ∧ (x ∨ w) ∧ (y ∨ w)
x z y w
SLIDE 36
How about CNF?
We consider a very special case: monotone 2-CNF φ = (x ∨ y) ∧ (x ∨ z) ∧ (x ∨ w) ∧ (y ∨ w)
x z y w
x = 𝚞𝚜𝚟𝚏, y = 𝚐𝚋𝚖𝚝𝚏 z = 𝚐𝚋𝚖𝚝𝚏, w = 𝚞𝚜𝚟𝚏
SLIDE 37
How about CNF?
We consider a very special case: monotone 2-CNF φ = (x ∨ y) ∧ (x ∨ z) ∧ (x ∨ w) ∧ (y ∨ w)
x z y w
x = 𝚞𝚜𝚟𝚏, y = 𝚐𝚋𝚖𝚝𝚏 z = 𝚐𝚋𝚖𝚝𝚏, w = 𝚞𝚜𝚟𝚏
x z y w
SLIDE 38
How about CNF?
We consider a very special case: monotone 2-CNF φ = (x ∨ y) ∧ (x ∨ z) ∧ (x ∨ w) ∧ (y ∨ w)
x z y w
x = 𝚞𝚜𝚟𝚏, y = 𝚐𝚋𝚖𝚝𝚏 z = 𝚐𝚋𝚖𝚝𝚏, w = 𝚞𝚜𝚟𝚏
x z y w
#φ = # of independent sets
SLIDE 39
SLIDE 40
Sampling seems to be harder than DNF case…
SLIDE 41
Sampling seems to be harder than DNF case… Rejection sampling is correct but inefficient
SLIDE 42
Sampling seems to be harder than DNF case… Rejection sampling is correct but inefficient A natural idea is to resample those violated edges…
SLIDE 43
Sampling seems to be harder than DNF case… Rejection sampling is correct but inefficient A natural idea is to resample those violated edges… Unfortunately, this is not correct.
SLIDE 44
Sampling seems to be harder than DNF case… Rejection sampling is correct but inefficient A natural idea is to resample those violated edges… Unfortunately, this is not correct. Think about
SLIDE 45
Sampling seems to be harder than DNF case… Rejection sampling is correct but inefficient A natural idea is to resample those violated edges… Unfortunately, this is not correct.
x z y
Think about
SLIDE 46
Partial Rejection Sampling
SLIDE 47
Partial Rejection Sampling
Guo, Jerrum and Liu (JACM, 2019) proposed the following fix:
SLIDE 48
Partial Rejection Sampling
Guo, Jerrum and Liu (JACM, 2019) proposed the following fix: “Resample violated vertices and their neighbors”
SLIDE 49
Partial Rejection Sampling
Guo, Jerrum and Liu (JACM, 2019) proposed the following fix: “Resample violated vertices and their neighbors” We will prove the correctness and analyze its efficiency next week
SLIDE 50
From Sampling to Counting
SLIDE 51
SLIDE 52
We will show that, in many cases, if one can sample from a space, then he can also estimate the size of the space
SLIDE 53
We will show that, in many cases, if one can sample from a space, then he can also estimate the size of the space Consider independent sets again
SLIDE 54
We will show that, in many cases, if one can sample from a space, then he can also estimate the size of the space Consider independent sets again ,
G = (V, E) E = {e1, e2, …, em}
SLIDE 55
We will show that, in many cases, if one can sample from a space, then he can also estimate the size of the space Consider independent sets again ,
G = (V, E) E = {e1, e2, …, em}
We want to estimate , the number of i.s. in
I(G) G
SLIDE 56
SLIDE 57
Define
G0 = G, Gi = Gi−1 − ei
SLIDE 58
Define
G0 = G, Gi = Gi−1 − ei
|I(G)| = |I(G0)| = |I(G0)| |I(G1)| ⋅ |I(G1)| |I(G2)| … |I(Gm−1)| |I(Gm)| ⋅ |I(Gm)|
SLIDE 59
Define
G0 = G, Gi = Gi−1 − ei
|I(G)| = |I(G0)| = |I(G0)| |I(G1)| ⋅ |I(G1)| |I(G2)| … |I(Gm−1)| |I(Gm)| ⋅ |I(Gm)|
SLIDE 60
Define
G0 = G, Gi = Gi−1 − ei
|I(G)| = |I(G0)| = |I(G0)| |I(G1)| ⋅ |I(G1)| |I(G2)| … |I(Gm−1)| |I(Gm)| ⋅ |I(Gm)| ||
SLIDE 61
Define
G0 = G, Gi = Gi−1 − ei
|I(G)| = |I(G0)| = |I(G0)| |I(G1)| ⋅ |I(G1)| |I(G2)| … |I(Gm−1)| |I(Gm)| ⋅ |I(Gm)| 2n ||
SLIDE 62
Define
G0 = G, Gi = Gi−1 − ei
|I(G)| = |I(G0)| = |I(G0)| |I(G1)| ⋅ |I(G1)| |I(G2)| … |I(Gm−1)| |I(Gm)| ⋅ |I(Gm)| 2n ||
SLIDE 63
Define
G0 = G, Gi = Gi−1 − ei
|I(G)| = |I(G0)| = |I(G0)| |I(G1)| ⋅ |I(G1)| |I(G2)| … |I(Gm−1)| |I(Gm)| ⋅ |I(Gm)| 2n || A = I(Gi)
SLIDE 64
Define
G0 = G, Gi = Gi−1 − ei
|I(G)| = |I(G0)| = |I(G0)| |I(G1)| ⋅ |I(G1)| |I(G2)| … |I(Gm−1)| |I(Gm)| ⋅ |I(Gm)| 2n || A = I(Gi) B = I(Gi+1)
SLIDE 65
Define
G0 = G, Gi = Gi−1 − ei
|I(G)| = |I(G0)| = |I(G0)| |I(G1)| ⋅ |I(G1)| |I(G2)| … |I(Gm−1)| |I(Gm)| ⋅ |I(Gm)| 2n || A = I(Gi) B = I(Gi+1) can’t be too large!
|A| |B|
SLIDE 66
Define
G0 = G, Gi = Gi−1 − ei
|I(G)| = |I(G0)| = |I(G0)| |I(G1)| ⋅ |I(G1)| |I(G2)| … |I(Gm−1)| |I(Gm)| ⋅ |I(Gm)| 2n || A = I(Gi) B = I(Gi+1) can’t be too large!
|A| |B|
I(Gi) I(Gi+1) ≤ 2
SLIDE 67
From Counting to Sampling
SLIDE 68
SLIDE 69
On the other hand, one can consecutively sample each vertex as long as is known
Pr[v ∈ I]
SLIDE 70
On the other hand, one can consecutively sample each vertex as long as is known
Pr[v ∈ I]
The value can be obtained via a counting oracle
SLIDE 71