Advanced Algorithms (I) Chihao Zhang Shanghai Jiao Tong University - - PowerPoint PPT Presentation

advanced algorithms i
SMART_READER_LITE
LIVE PREVIEW

Advanced Algorithms (I) Chihao Zhang Shanghai Jiao Tong University - - PowerPoint PPT Presentation

Advanced Algorithms (I) Chihao Zhang Shanghai Jiao Tong University Feb. 25, 2019 Advanced Algorithms (I) 1/14 Advanced Algorithms In the course, we will learn Approximation Algorithms linear programming, semi-definite programming spectral


slide-1
SLIDE 1

Advanced Algorithms (I)

Chihao Zhang

Shanghai Jiao Tong University

  • Feb. 25, 2019

Advanced Algorithms (I) 1/14

slide-2
SLIDE 2

Advanced Algorithms

In the course, we will learn Approximation Algorithms linear programming, semi-definite programming spectral method random walks … We will emphasize on tools for designing approximation algorithms rigorous analysis of algorithms

Advanced Algorithms (I) 2/14

slide-3
SLIDE 3

Advanced Algorithms

In the course, we will learn Approximation Algorithms linear programming, semi-definite programming spectral method random walks … We will emphasize on tools for designing approximation algorithms rigorous analysis of algorithms

Advanced Algorithms (I) 2/14

slide-4
SLIDE 4

Advanced Algorithms

In the course, we will learn Approximation Algorithms ▶ linear programming, semi-definite programming ▶ spectral method ▶ random walks ▶ … We will emphasize on tools for designing approximation algorithms rigorous analysis of algorithms

Advanced Algorithms (I) 2/14

slide-5
SLIDE 5

Advanced Algorithms

In the course, we will learn Approximation Algorithms ▶ linear programming, semi-definite programming ▶ spectral method ▶ random walks ▶ … We will emphasize on ▶ tools for designing approximation algorithms ▶ rigorous analysis of algorithms

Advanced Algorithms (I) 2/14

slide-6
SLIDE 6

Course Info

Instructor: Chihao Zhang Course Homepage: http://chihaozhang.com/teaching/AA2019spring/ Ofgice Hour: every Monday, 7:00pm - 9:00pm Grading Policy Homework 30% Mid-term Exam 30% Course Project 40%

Advanced Algorithms (I) 3/14

slide-7
SLIDE 7

Course Info

▶ Instructor: Chihao Zhang ▶ Course Homepage: http://chihaozhang.com/teaching/AA2019spring/ ▶ Ofgice Hour: every Monday, 7:00pm - 9:00pm Grading Policy Homework 30% Mid-term Exam 30% Course Project 40%

Advanced Algorithms (I) 3/14

slide-8
SLIDE 8

Course Info

▶ Instructor: Chihao Zhang ▶ Course Homepage: http://chihaozhang.com/teaching/AA2019spring/ ▶ Ofgice Hour: every Monday, 7:00pm - 9:00pm Grading Policy ▶ Homework 30% ▶ Mid-term Exam 30% ▶ Course Project 40%

Advanced Algorithms (I) 3/14

slide-9
SLIDE 9

MaxSAT

Given a CNF formula , is it satisfiable? x x x x x x x x x NP-hard, we look at its optimization version. MaxSAT Input: A CNF formula C C Cm. Problem: Compute an assignment that satisfies maximum number of clauses. Harder than SAT, so we look for an approximate solution.

Advanced Algorithms (I) 4/14

slide-10
SLIDE 10

MaxSAT

Given a CNF formula ϕ, is it satisfiable? x x x x x x x x x NP-hard, we look at its optimization version. MaxSAT Input: A CNF formula C C Cm. Problem: Compute an assignment that satisfies maximum number of clauses. Harder than SAT, so we look for an approximate solution.

Advanced Algorithms (I) 4/14

slide-11
SLIDE 11

MaxSAT

Given a CNF formula ϕ, is it satisfiable? ϕ = (x1 ∨ x3 ∨ ¯ x29) ∧ (¯ x3 ∨ x7) ∧ · · · ∧ (¯ x33 ∨ ¯ x34 ∨ x90 ∨ x126) NP-hard, we look at its optimization version. MaxSAT Input: A CNF formula C C Cm. Problem: Compute an assignment that satisfies maximum number of clauses. Harder than SAT, so we look for an approximate solution.

Advanced Algorithms (I) 4/14

slide-12
SLIDE 12

MaxSAT

Given a CNF formula ϕ, is it satisfiable? ϕ = (x1 ∨ x3 ∨ ¯ x29) ∧ (¯ x3 ∨ x7) ∧ · · · ∧ (¯ x33 ∨ ¯ x34 ∨ x90 ∨ x126) NP-hard, we look at its optimization version. MaxSAT Input: A CNF formula C C Cm. Problem: Compute an assignment that satisfies maximum number of clauses. Harder than SAT, so we look for an approximate solution.

Advanced Algorithms (I) 4/14

slide-13
SLIDE 13

MaxSAT

Given a CNF formula ϕ, is it satisfiable? ϕ = (x1 ∨ x3 ∨ ¯ x29) ∧ (¯ x3 ∨ x7) ∧ · · · ∧ (¯ x33 ∨ ¯ x34 ∨ x90 ∨ x126) NP-hard, we look at its optimization version. MaxSAT Input: A CNF formula ϕ = C1 ∧ C2 · · · ∧ Cm. Problem: Compute an assignment that satisfies maximum number of clauses. Harder than SAT, so we look for an approximate solution.

Advanced Algorithms (I) 4/14

slide-14
SLIDE 14

MaxSAT

Given a CNF formula ϕ, is it satisfiable? ϕ = (x1 ∨ x3 ∨ ¯ x29) ∧ (¯ x3 ∨ x7) ∧ · · · ∧ (¯ x33 ∨ ¯ x34 ∨ x90 ∨ x126) NP-hard, we look at its optimization version. MaxSAT Input: A CNF formula ϕ = C1 ∧ C2 · · · ∧ Cm. Problem: Compute an assignment that satisfies maximum number of clauses. Harder than SAT, so we look for an approximate solution.

Advanced Algorithms (I) 4/14

slide-15
SLIDE 15

Tossing a Coin

An instance

The variable sets V x x xn The set of clauses C C Cm Each clause Ci contains

i literals

We first cosnider the following simple algorithm: For each variable xi, toss an independent fair coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 5/14

slide-16
SLIDE 16

Tossing a Coin

An instance ϕ

▶ The variable sets V = {x1, x2, . . . , xn} ▶ The set of clauses C = {C1, C2, . . . , Cm} ▶ Each clause Ci contains ℓi literals We first cosnider the following simple algorithm: For each variable xi, toss an independent fair coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 5/14

slide-17
SLIDE 17

Tossing a Coin

An instance ϕ

▶ The variable sets V = {x1, x2, . . . , xn} ▶ The set of clauses C = {C1, C2, . . . , Cm} ▶ Each clause Ci contains ℓi literals We first cosnider the following simple algorithm: For each variable xi, toss an independent fair coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 5/14

slide-18
SLIDE 18

Tossing a Coin

An instance ϕ

▶ The variable sets V = {x1, x2, . . . , xn} ▶ The set of clauses C = {C1, C2, . . . , Cm} ▶ Each clause Ci contains ℓi literals We first cosnider the following simple algorithm: ▶ For each variable xi, toss an independent fair coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 5/14

slide-19
SLIDE 19

Tossing a Coin

An instance ϕ

▶ The variable sets V = {x1, x2, . . . , xn} ▶ The set of clauses C = {C1, C2, . . . , Cm} ▶ Each clause Ci contains ℓi literals We first cosnider the following simple algorithm: ▶ For each variable xi, toss an independent fair coin. ▶ If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 5/14

slide-20
SLIDE 20

Analysis

The outcome of the algorithm is random, we are interested in its expectation. For this particular algorithm, it can be derandomized. E X

m i

Pr Ci is satisfied

m i

i

m On the otherhand, OPT m Therefore, E X OPT

Advanced Algorithms (I) 6/14

slide-21
SLIDE 21

Analysis

The outcome of the algorithm is random, we are interested in its expectation. For this particular algorithm, it can be derandomized. E X

m i

Pr Ci is satisfied

m i

i

m On the otherhand, OPT m Therefore, E X OPT

Advanced Algorithms (I) 6/14

slide-22
SLIDE 22

Analysis

The outcome of the algorithm is random, we are interested in its expectation. For this particular algorithm, it can be derandomized. E X

m i

Pr Ci is satisfied

m i

i

m On the otherhand, OPT m Therefore, E X OPT

Advanced Algorithms (I) 6/14

slide-23
SLIDE 23

Analysis

The outcome of the algorithm is random, we are interested in its expectation. For this particular algorithm, it can be derandomized. E [X] =

m

i=1

Pr [Ci is satisfied]

m i

i

m On the otherhand, OPT m Therefore, E X OPT

Advanced Algorithms (I) 6/14

slide-24
SLIDE 24

Analysis

The outcome of the algorithm is random, we are interested in its expectation. For this particular algorithm, it can be derandomized. E [X] =

m

i=1

Pr [Ci is satisfied] =

m

i=1

( 1 − 2−ℓi ) ≥ m 2 . On the otherhand, OPT m Therefore, E X OPT

Advanced Algorithms (I) 6/14

slide-25
SLIDE 25

Analysis

The outcome of the algorithm is random, we are interested in its expectation. For this particular algorithm, it can be derandomized. E [X] =

m

i=1

Pr [Ci is satisfied] =

m

i=1

( 1 − 2−ℓi ) ≥ m 2 . On the otherhand, OPT ≤ m. Therefore, E X OPT

Advanced Algorithms (I) 6/14

slide-26
SLIDE 26

Analysis

The outcome of the algorithm is random, we are interested in its expectation. For this particular algorithm, it can be derandomized. E [X] =

m

i=1

Pr [Ci is satisfied] =

m

i=1

( 1 − 2−ℓi ) ≥ m 2 . On the otherhand, OPT ≤ m. Therefore, E [X] ≥ 1 2 · OPT.

Advanced Algorithms (I) 6/14

slide-27
SLIDE 27

Can we improve the previous algorithm?

Observations

the worst case happens when for some singleton clause, i.e.,

i

; for a singleton C x, if there is no C x, then we can increase the probability of x to be true;

  • therwise, we can improve the upper bound for OPT! (x and x

cannot be both satisfied)

Advanced Algorithms (I) 7/14

slide-28
SLIDE 28

Can we improve the previous algorithm?

Observations

▶ the worst case happens when for some singleton clause, i.e., ℓi = 1; for a singleton C x, if there is no C x, then we can increase the probability of x to be true;

  • therwise, we can improve the upper bound for OPT! (x and x

cannot be both satisfied)

Advanced Algorithms (I) 7/14

slide-29
SLIDE 29

Can we improve the previous algorithm?

Observations

▶ the worst case happens when for some singleton clause, i.e., ℓi = 1; ▶ for a singleton C = x, if there is no C′ = ¯ x, then we can increase the probability of x to be true;

  • therwise, we can improve the upper bound for OPT! (x and x

cannot be both satisfied)

Advanced Algorithms (I) 7/14

slide-30
SLIDE 30

Can we improve the previous algorithm?

Observations

▶ the worst case happens when for some singleton clause, i.e., ℓi = 1; ▶ for a singleton C = x, if there is no C′ = ¯ x, then we can increase the probability of x to be true; ▶ otherwise, we can improve the upper bound for OPT! (x and ¯ x cannot be both satisfied)

Advanced Algorithms (I) 7/14

slide-31
SLIDE 31

Tossing a Biased Coin

Let the p-biased coin be with probability p to HEAD, p to TAIL. For each variable xi, toss an independent p-biased coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 8/14

slide-32
SLIDE 32

Tossing a Biased Coin

Let the p-biased coin be with probability p to HEAD, 1 − p to TAIL. For each variable xi, toss an independent p-biased coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 8/14

slide-33
SLIDE 33

Tossing a Biased Coin

Let the p-biased coin be with probability p to HEAD, 1 − p to TAIL. For each variable xi, toss an independent p-biased coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 8/14

slide-34
SLIDE 34

Tossing a Biased Coin

Let the p-biased coin be with probability p to HEAD, 1 − p to TAIL. ▶ For each variable xi, toss an independent p-biased coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 8/14

slide-35
SLIDE 35

Tossing a Biased Coin

Let the p-biased coin be with probability p to HEAD, 1 − p to TAIL. ▶ For each variable xi, toss an independent p-biased coin. ▶ If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 8/14

slide-36
SLIDE 36

Analysis

Assumption for Clauses of

More positive singletons than negative singletons. There are t pairs of x and x. p . E X t m t min p p The new upper bound for OPT: OPT m t Combine them and obtain E X OPT where .

Advanced Algorithms (I) 9/14

slide-37
SLIDE 37

Analysis

Assumption for Clauses of ϕ

▶ More positive singletons than negative singletons. There are t pairs of x and x. p . E X t m t min p p The new upper bound for OPT: OPT m t Combine them and obtain E X OPT where .

Advanced Algorithms (I) 9/14

slide-38
SLIDE 38

Analysis

Assumption for Clauses of ϕ

▶ More positive singletons than negative singletons. ▶ There are t pairs of x and ¯ x. p . E X t m t min p p The new upper bound for OPT: OPT m t Combine them and obtain E X OPT where .

Advanced Algorithms (I) 9/14

slide-39
SLIDE 39

Analysis

Assumption for Clauses of ϕ

▶ More positive singletons than negative singletons. ▶ There are t pairs of x and ¯ x. ▶ p ≥ 1

2.

E [X] ≥ t + (m − 2t) min { p, 1 − p2} . The new upper bound for OPT: OPT m t Combine them and obtain E X OPT where .

Advanced Algorithms (I) 9/14

slide-40
SLIDE 40

Analysis

Assumption for Clauses of ϕ

▶ More positive singletons than negative singletons. ▶ There are t pairs of x and ¯ x. ▶ p ≥ 1

2.

E [X] ≥ t + (m − 2t) min { p, 1 − p2} . The new upper bound for OPT: OPT ≤ m − t Combine them and obtain E X OPT where .

Advanced Algorithms (I) 9/14

slide-41
SLIDE 41

Analysis

Assumption for Clauses of ϕ

▶ More positive singletons than negative singletons. ▶ There are t pairs of x and ¯ x. ▶ p ≥ 1

2.

E [X] ≥ t + (m − 2t) min { p, 1 − p2} . The new upper bound for OPT: OPT ≤ m − t Combine them and obtain E [X] ≥ α · OPT where α ≈ 0.618.

Advanced Algorithms (I) 9/14

slide-42
SLIDE 42

Can we further improve the algorithm? We can use difgerent coins for each variable, e.g., in x x x we prefer to set x to true. How can we make use of the information? Linear Programming helps.

Advanced Algorithms (I) 10/14

slide-43
SLIDE 43

Can we further improve the algorithm? We can use difgerent coins for each variable, e.g., in ϕ = x1 ∧ (x1 ∨ ¯ x2), we prefer to set x1 to true. How can we make use of the information? Linear Programming helps.

Advanced Algorithms (I) 10/14

slide-44
SLIDE 44

Can we further improve the algorithm? We can use difgerent coins for each variable, e.g., in ϕ = x1 ∧ (x1 ∨ ¯ x2), we prefer to set x1 to true. How can we make use of the information? Linear Programming helps.

Advanced Algorithms (I) 10/14

slide-45
SLIDE 45

Can we further improve the algorithm? We can use difgerent coins for each variable, e.g., in ϕ = x1 ∧ (x1 ∨ ¯ x2), we prefer to set x1 to true. How can we make use of the information? Linear Programming helps.

Advanced Algorithms (I) 10/14

slide-46
SLIDE 46

Tossing Clever Coins

We introduce the following variables. for every i n , yi indicates whether xi is true; for every j m , zj indicates whether Cj is satisfied. max

m j

zj subject to

i Pj

yi

k Nj

yk zj j m s.t. Cj

i Pj

xi

k Nj

xk zj j m yi i n This integer program is equivalent to MaxSAT.

Advanced Algorithms (I) 11/14

slide-47
SLIDE 47

Tossing Clever Coins

We introduce the following variables. for every i n , yi indicates whether xi is true; for every j m , zj indicates whether Cj is satisfied. max

m j

zj subject to

i Pj

yi

k Nj

yk zj j m s.t. Cj

i Pj

xi

k Nj

xk zj j m yi i n This integer program is equivalent to MaxSAT.

Advanced Algorithms (I) 11/14

slide-48
SLIDE 48

Tossing Clever Coins

We introduce the following variables. ▶ for every i ∈ [n], yi indicates whether xi is true; ▶ for every j ∈ [m], zj indicates whether Cj is satisfied. max

m j

zj subject to

i Pj

yi

k Nj

yk zj j m s.t. Cj

i Pj

xi

k Nj

xk zj j m yi i n This integer program is equivalent to MaxSAT.

Advanced Algorithms (I) 11/14

slide-49
SLIDE 49

Tossing Clever Coins

We introduce the following variables. ▶ for every i ∈ [n], yi indicates whether xi is true; ▶ for every j ∈ [m], zj indicates whether Cj is satisfied. max

m

j=1

zj subject to ∑

i∈Pj

yi + ∑

k∈Nj

(1 − yk) ≥ zj, ∀j ∈ [m] s.t. Cj = ∨

i∈Pj

xi ∨ ∨

k∈Nj

¯ xk zj ∈ {0, 1}, ∀j ∈ [m] yi ∈ {0, 1}, ∀i ∈ [n] This integer program is equivalent to MaxSAT.

Advanced Algorithms (I) 11/14

slide-50
SLIDE 50

Tossing Clever Coins

We introduce the following variables. ▶ for every i ∈ [n], yi indicates whether xi is true; ▶ for every j ∈ [m], zj indicates whether Cj is satisfied. max

m

j=1

zj subject to ∑

i∈Pj

yi + ∑

k∈Nj

(1 − yk) ≥ zj, ∀j ∈ [m] s.t. Cj = ∨

i∈Pj

xi ∨ ∨

k∈Nj

¯ xk zj ∈ {0, 1}, ∀j ∈ [m] yi ∈ {0, 1}, ∀i ∈ [n] This integer program is equivalent to MaxSAT.

Advanced Algorithms (I) 11/14

slide-51
SLIDE 51

Relaxation

There is no efgicient algorithm for integer programming in general Therefore, we relax its non-linear constriants max

m j

zj subject to

i Pj

yi

k Pj

yk zj j m s.t. Cj

i Pj

xi

k Nj

xk zj j m yi i n We can solve this LP in poly-time

Advanced Algorithms (I) 12/14

slide-52
SLIDE 52

Relaxation

There is no efgicient algorithm for integer programming in general Therefore, we relax its non-linear constriants max

m j

zj subject to

i Pj

yi

k Pj

yk zj j m s.t. Cj

i Pj

xi

k Nj

xk zj j m yi i n We can solve this LP in poly-time

Advanced Algorithms (I) 12/14

slide-53
SLIDE 53

Relaxation

There is no efgicient algorithm for integer programming in general Therefore, we relax its non-linear constriants max

m j

zj subject to

i Pj

yi

k Pj

yk zj j m s.t. Cj

i Pj

xi

k Nj

xk zj j m yi i n We can solve this LP in poly-time

Advanced Algorithms (I) 12/14

slide-54
SLIDE 54

Relaxation

There is no efgicient algorithm for integer programming in general Therefore, we relax its non-linear constriants max

m

j=1

zj subject to ∑

i∈Pj

yi + ∑

k∈Pj

(1 − yk) ≥ zj, ∀j ∈ [m] s.t. Cj = ∨

i∈Pj

xi ∨ ∨

k∈Nj

¯ xk 0 ≤ zj ≤ 1, ∀j ∈ [m] 0 ≤ yi ≤ 1, ∀i ∈ [n] We can solve this LP in poly-time

Advanced Algorithms (I) 12/14

slide-55
SLIDE 55

Relaxation

There is no efgicient algorithm for integer programming in general Therefore, we relax its non-linear constriants max

m

j=1

zj subject to ∑

i∈Pj

yi + ∑

k∈Pj

(1 − yk) ≥ zj, ∀j ∈ [m] s.t. Cj = ∨

i∈Pj

xi ∨ ∨

k∈Nj

¯ xk 0 ≤ zj ≤ 1, ∀j ∈ [m] 0 ≤ yi ≤ 1, ∀i ∈ [n] We can solve this LP in poly-time

Advanced Algorithms (I) 12/14

slide-56
SLIDE 56

Algorithm

Let yi

i n

zj

j m

be an optimal solution of the LP. For each variable xi, toss an independent yi -biased coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 13/14

slide-57
SLIDE 57

Algorithm

Let ({ y∗

i

}

i∈[n] ,

{ z∗

j

}

j∈[m]

) be an optimal solution of the LP. For each variable xi, toss an independent yi -biased coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 13/14

slide-58
SLIDE 58

Algorithm

Let ({ y∗

i

}

i∈[n] ,

{ z∗

j

}

j∈[m]

) be an optimal solution of the LP. For each variable xi, toss an independent yi -biased coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 13/14

slide-59
SLIDE 59

Algorithm

Let ({ y∗

i

}

i∈[n] ,

{ z∗

j

}

j∈[m]

) be an optimal solution of the LP. ▶ For each variable xi, toss an independent y∗

i -biased

coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 13/14

slide-60
SLIDE 60

Algorithm

Let ({ y∗

i

}

i∈[n] ,

{ z∗

j

}

j∈[m]

) be an optimal solution of the LP. ▶ For each variable xi, toss an independent y∗

i -biased

coin. ▶ If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 13/14

slide-61
SLIDE 61

Analysis

A typical upper bound of OPT for LP based algorithms is OPT OPT LP

m j

zj We can further establish E X e

m j

zj Therefore, the LP rounding is a

e -approximation algorithm for

MaxSAT

Advanced Algorithms (I) 14/14

slide-62
SLIDE 62

Analysis

A typical upper bound of OPT for LP based algorithms is OPT ≤ OPT(LP) =

m

j=1

z∗

j

We can further establish E X e

m j

zj Therefore, the LP rounding is a

e -approximation algorithm for

MaxSAT

Advanced Algorithms (I) 14/14

slide-63
SLIDE 63

Analysis

A typical upper bound of OPT for LP based algorithms is OPT ≤ OPT(LP) =

m

j=1

z∗

j

We can further establish E [X] ≥ ( 1 − 1 e )

m

j=1

z∗

j .

Therefore, the LP rounding is a

e -approximation algorithm for

MaxSAT

Advanced Algorithms (I) 14/14

slide-64
SLIDE 64

Analysis

A typical upper bound of OPT for LP based algorithms is OPT ≤ OPT(LP) =

m

j=1

z∗

j

We can further establish E [X] ≥ ( 1 − 1 e )

m

j=1

z∗

j .

Therefore, the LP rounding is a (1 − 1

e

)-approximation algorithm for MaxSAT

Advanced Algorithms (I) 14/14