Advanced Algorithms (I) Chihao Zhang Shanghai Jiao Tong University - - PowerPoint PPT Presentation

Advanced Algorithms (I)

Chihao Zhang

Shanghai Jiao Tong University

• Feb. 25, 2019

Advanced Algorithms (I) 1/14

Advanced Algorithms

In the course, we will learn Approximation Algorithms linear programming, semi-definite programming spectral method random walks … We will emphasize on tools for designing approximation algorithms rigorous analysis of algorithms

Advanced Algorithms (I) 2/14

Advanced Algorithms

In the course, we will learn Approximation Algorithms linear programming, semi-definite programming spectral method random walks … We will emphasize on tools for designing approximation algorithms rigorous analysis of algorithms

Advanced Algorithms (I) 2/14

Advanced Algorithms

In the course, we will learn Approximation Algorithms ▶ linear programming, semi-definite programming ▶ spectral method ▶ random walks ▶ … We will emphasize on tools for designing approximation algorithms rigorous analysis of algorithms

Advanced Algorithms (I) 2/14

Advanced Algorithms

In the course, we will learn Approximation Algorithms ▶ linear programming, semi-definite programming ▶ spectral method ▶ random walks ▶ … We will emphasize on ▶ tools for designing approximation algorithms ▶ rigorous analysis of algorithms

Advanced Algorithms (I) 2/14

Course Info

Instructor: Chihao Zhang Course Homepage: http://chihaozhang.com/teaching/AA2019spring/ Ofgice Hour: every Monday, 7:00pm - 9:00pm Grading Policy Homework 30% Mid-term Exam 30% Course Project 40%

Advanced Algorithms (I) 3/14

Course Info

▶ Instructor: Chihao Zhang ▶ Course Homepage: http://chihaozhang.com/teaching/AA2019spring/ ▶ Ofgice Hour: every Monday, 7:00pm - 9:00pm Grading Policy Homework 30% Mid-term Exam 30% Course Project 40%

Advanced Algorithms (I) 3/14

Course Info

▶ Instructor: Chihao Zhang ▶ Course Homepage: http://chihaozhang.com/teaching/AA2019spring/ ▶ Ofgice Hour: every Monday, 7:00pm - 9:00pm Grading Policy ▶ Homework 30% ▶ Mid-term Exam 30% ▶ Course Project 40%

Advanced Algorithms (I) 3/14

MaxSAT

Given a CNF formula , is it satisfiable? x x x x x x x x x NP-hard, we look at its optimization version. MaxSAT Input: A CNF formula C C Cm. Problem: Compute an assignment that satisfies maximum number of clauses. Harder than SAT, so we look for an approximate solution.

Advanced Algorithms (I) 4/14

MaxSAT

Given a CNF formula ϕ, is it satisfiable? x x x x x x x x x NP-hard, we look at its optimization version. MaxSAT Input: A CNF formula C C Cm. Problem: Compute an assignment that satisfies maximum number of clauses. Harder than SAT, so we look for an approximate solution.

Advanced Algorithms (I) 4/14

MaxSAT

Given a CNF formula ϕ, is it satisfiable? ϕ = (x1 ∨ x3 ∨ ¯ x29) ∧ (¯ x3 ∨ x7) ∧ · · · ∧ (¯ x33 ∨ ¯ x34 ∨ x90 ∨ x126) NP-hard, we look at its optimization version. MaxSAT Input: A CNF formula C C Cm. Problem: Compute an assignment that satisfies maximum number of clauses. Harder than SAT, so we look for an approximate solution.

Advanced Algorithms (I) 4/14

MaxSAT

Given a CNF formula ϕ, is it satisfiable? ϕ = (x1 ∨ x3 ∨ ¯ x29) ∧ (¯ x3 ∨ x7) ∧ · · · ∧ (¯ x33 ∨ ¯ x34 ∨ x90 ∨ x126) NP-hard, we look at its optimization version. MaxSAT Input: A CNF formula C C Cm. Problem: Compute an assignment that satisfies maximum number of clauses. Harder than SAT, so we look for an approximate solution.

Advanced Algorithms (I) 4/14

MaxSAT

Given a CNF formula ϕ, is it satisfiable? ϕ = (x1 ∨ x3 ∨ ¯ x29) ∧ (¯ x3 ∨ x7) ∧ · · · ∧ (¯ x33 ∨ ¯ x34 ∨ x90 ∨ x126) NP-hard, we look at its optimization version. MaxSAT Input: A CNF formula ϕ = C1 ∧ C2 · · · ∧ Cm. Problem: Compute an assignment that satisfies maximum number of clauses. Harder than SAT, so we look for an approximate solution.

Advanced Algorithms (I) 4/14

MaxSAT

Given a CNF formula ϕ, is it satisfiable? ϕ = (x1 ∨ x3 ∨ ¯ x29) ∧ (¯ x3 ∨ x7) ∧ · · · ∧ (¯ x33 ∨ ¯ x34 ∨ x90 ∨ x126) NP-hard, we look at its optimization version. MaxSAT Input: A CNF formula ϕ = C1 ∧ C2 · · · ∧ Cm. Problem: Compute an assignment that satisfies maximum number of clauses. Harder than SAT, so we look for an approximate solution.

Advanced Algorithms (I) 4/14

Tossing a Coin

An instance

The variable sets V x x xn The set of clauses C C Cm Each clause Ci contains

i literals

We first cosnider the following simple algorithm: For each variable xi, toss an independent fair coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 5/14

Tossing a Coin

An instance ϕ

▶ The variable sets V = {x1, x2, . . . , xn} ▶ The set of clauses C = {C1, C2, . . . , Cm} ▶ Each clause Ci contains ℓi literals We first cosnider the following simple algorithm: For each variable xi, toss an independent fair coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 5/14

Tossing a Coin

An instance ϕ

▶ The variable sets V = {x1, x2, . . . , xn} ▶ The set of clauses C = {C1, C2, . . . , Cm} ▶ Each clause Ci contains ℓi literals We first cosnider the following simple algorithm: For each variable xi, toss an independent fair coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 5/14

Tossing a Coin

An instance ϕ

▶ The variable sets V = {x1, x2, . . . , xn} ▶ The set of clauses C = {C1, C2, . . . , Cm} ▶ Each clause Ci contains ℓi literals We first cosnider the following simple algorithm: ▶ For each variable xi, toss an independent fair coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 5/14

Tossing a Coin

An instance ϕ

▶ The variable sets V = {x1, x2, . . . , xn} ▶ The set of clauses C = {C1, C2, . . . , Cm} ▶ Each clause Ci contains ℓi literals We first cosnider the following simple algorithm: ▶ For each variable xi, toss an independent fair coin. ▶ If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 5/14

Analysis

The outcome of the algorithm is random, we are interested in its expectation. For this particular algorithm, it can be derandomized. E X

m i

Pr Ci is satisfied

m i

i

m On the otherhand, OPT m Therefore, E X OPT

Advanced Algorithms (I) 6/14

Analysis

The outcome of the algorithm is random, we are interested in its expectation. For this particular algorithm, it can be derandomized. E X

m i

Pr Ci is satisfied

m i

i

m On the otherhand, OPT m Therefore, E X OPT

Advanced Algorithms (I) 6/14

Analysis

The outcome of the algorithm is random, we are interested in its expectation. For this particular algorithm, it can be derandomized. E X

m i

Pr Ci is satisfied

m i

i

m On the otherhand, OPT m Therefore, E X OPT

Advanced Algorithms (I) 6/14

Analysis

The outcome of the algorithm is random, we are interested in its expectation. For this particular algorithm, it can be derandomized. E [X] =

m

∑

i=1

Pr [Ci is satisfied]

m i

i

m On the otherhand, OPT m Therefore, E X OPT

Advanced Algorithms (I) 6/14

Analysis

The outcome of the algorithm is random, we are interested in its expectation. For this particular algorithm, it can be derandomized. E [X] =

m

∑

i=1

Pr [Ci is satisfied] =

m

∑

i=1

( 1 − 2−ℓi ) ≥ m 2 . On the otherhand, OPT m Therefore, E X OPT

Advanced Algorithms (I) 6/14

Analysis

The outcome of the algorithm is random, we are interested in its expectation. For this particular algorithm, it can be derandomized. E [X] =

m

∑

i=1

Pr [Ci is satisfied] =

m

∑

i=1

( 1 − 2−ℓi ) ≥ m 2 . On the otherhand, OPT ≤ m. Therefore, E X OPT

Advanced Algorithms (I) 6/14

Analysis

The outcome of the algorithm is random, we are interested in its expectation. For this particular algorithm, it can be derandomized. E [X] =

m

∑

i=1

Pr [Ci is satisfied] =

m

∑

i=1

( 1 − 2−ℓi ) ≥ m 2 . On the otherhand, OPT ≤ m. Therefore, E [X] ≥ 1 2 · OPT.

Advanced Algorithms (I) 6/14

Can we improve the previous algorithm?

Observations

the worst case happens when for some singleton clause, i.e.,

i

; for a singleton C x, if there is no C x, then we can increase the probability of x to be true;

• therwise, we can improve the upper bound for OPT! (x and x

cannot be both satisfied)

Advanced Algorithms (I) 7/14

Can we improve the previous algorithm?

Observations

▶ the worst case happens when for some singleton clause, i.e., ℓi = 1; for a singleton C x, if there is no C x, then we can increase the probability of x to be true;

• therwise, we can improve the upper bound for OPT! (x and x

cannot be both satisfied)

Advanced Algorithms (I) 7/14

Can we improve the previous algorithm?

Observations

▶ the worst case happens when for some singleton clause, i.e., ℓi = 1; ▶ for a singleton C = x, if there is no C′ = ¯ x, then we can increase the probability of x to be true;

• therwise, we can improve the upper bound for OPT! (x and x

cannot be both satisfied)

Advanced Algorithms (I) 7/14

Can we improve the previous algorithm?

Observations

▶ the worst case happens when for some singleton clause, i.e., ℓi = 1; ▶ for a singleton C = x, if there is no C′ = ¯ x, then we can increase the probability of x to be true; ▶ otherwise, we can improve the upper bound for OPT! (x and ¯ x cannot be both satisfied)

Advanced Algorithms (I) 7/14

Tossing a Biased Coin

Let the p-biased coin be with probability p to HEAD, p to TAIL. For each variable xi, toss an independent p-biased coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 8/14

Tossing a Biased Coin

Let the p-biased coin be with probability p to HEAD, 1 − p to TAIL. For each variable xi, toss an independent p-biased coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 8/14

Tossing a Biased Coin

Let the p-biased coin be with probability p to HEAD, 1 − p to TAIL. For each variable xi, toss an independent p-biased coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 8/14

Tossing a Biased Coin

Let the p-biased coin be with probability p to HEAD, 1 − p to TAIL. ▶ For each variable xi, toss an independent p-biased coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 8/14

Tossing a Biased Coin

Let the p-biased coin be with probability p to HEAD, 1 − p to TAIL. ▶ For each variable xi, toss an independent p-biased coin. ▶ If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 8/14

Analysis

Assumption for Clauses of

More positive singletons than negative singletons. There are t pairs of x and x. p . E X t m t min p p The new upper bound for OPT: OPT m t Combine them and obtain E X OPT where .

Advanced Algorithms (I) 9/14

Analysis

Assumption for Clauses of ϕ

▶ More positive singletons than negative singletons. There are t pairs of x and x. p . E X t m t min p p The new upper bound for OPT: OPT m t Combine them and obtain E X OPT where .

Advanced Algorithms (I) 9/14

Analysis

Assumption for Clauses of ϕ

▶ More positive singletons than negative singletons. ▶ There are t pairs of x and ¯ x. p . E X t m t min p p The new upper bound for OPT: OPT m t Combine them and obtain E X OPT where .

Advanced Algorithms (I) 9/14

Analysis

Assumption for Clauses of ϕ

▶ More positive singletons than negative singletons. ▶ There are t pairs of x and ¯ x. ▶ p ≥ 1

2.

E [X] ≥ t + (m − 2t) min { p, 1 − p2} . The new upper bound for OPT: OPT m t Combine them and obtain E X OPT where .

Advanced Algorithms (I) 9/14

Analysis

Assumption for Clauses of ϕ

▶ More positive singletons than negative singletons. ▶ There are t pairs of x and ¯ x. ▶ p ≥ 1

2.

E [X] ≥ t + (m − 2t) min { p, 1 − p2} . The new upper bound for OPT: OPT ≤ m − t Combine them and obtain E X OPT where .

Advanced Algorithms (I) 9/14

Analysis

Assumption for Clauses of ϕ

▶ More positive singletons than negative singletons. ▶ There are t pairs of x and ¯ x. ▶ p ≥ 1

2.

E [X] ≥ t + (m − 2t) min { p, 1 − p2} . The new upper bound for OPT: OPT ≤ m − t Combine them and obtain E [X] ≥ α · OPT where α ≈ 0.618.

Advanced Algorithms (I) 9/14

Can we further improve the algorithm? We can use difgerent coins for each variable, e.g., in x x x we prefer to set x to true. How can we make use of the information? Linear Programming helps.

Advanced Algorithms (I) 10/14

Can we further improve the algorithm? We can use difgerent coins for each variable, e.g., in ϕ = x1 ∧ (x1 ∨ ¯ x2), we prefer to set x1 to true. How can we make use of the information? Linear Programming helps.

Advanced Algorithms (I) 10/14

Can we further improve the algorithm? We can use difgerent coins for each variable, e.g., in ϕ = x1 ∧ (x1 ∨ ¯ x2), we prefer to set x1 to true. How can we make use of the information? Linear Programming helps.

Advanced Algorithms (I) 10/14

Can we further improve the algorithm? We can use difgerent coins for each variable, e.g., in ϕ = x1 ∧ (x1 ∨ ¯ x2), we prefer to set x1 to true. How can we make use of the information? Linear Programming helps.

Advanced Algorithms (I) 10/14

Tossing Clever Coins

We introduce the following variables. for every i n , yi indicates whether xi is true; for every j m , zj indicates whether Cj is satisfied. max

m j

zj subject to

i Pj

yi

k Nj

yk zj j m s.t. Cj

i Pj

xi

k Nj

xk zj j m yi i n This integer program is equivalent to MaxSAT.

Advanced Algorithms (I) 11/14

Tossing Clever Coins

We introduce the following variables. for every i n , yi indicates whether xi is true; for every j m , zj indicates whether Cj is satisfied. max

m j

zj subject to

i Pj

yi

k Nj

yk zj j m s.t. Cj

i Pj

xi

k Nj

xk zj j m yi i n This integer program is equivalent to MaxSAT.

Advanced Algorithms (I) 11/14

Tossing Clever Coins

We introduce the following variables. ▶ for every i ∈ [n], yi indicates whether xi is true; ▶ for every j ∈ [m], zj indicates whether Cj is satisfied. max

m j

zj subject to

i Pj

yi

k Nj

yk zj j m s.t. Cj

i Pj

xi

k Nj

xk zj j m yi i n This integer program is equivalent to MaxSAT.

Advanced Algorithms (I) 11/14

Tossing Clever Coins

We introduce the following variables. ▶ for every i ∈ [n], yi indicates whether xi is true; ▶ for every j ∈ [m], zj indicates whether Cj is satisfied. max

m

∑

j=1

zj subject to ∑

i∈Pj

yi + ∑

k∈Nj

(1 − yk) ≥ zj, ∀j ∈ [m] s.t. Cj = ∨

i∈Pj

xi ∨ ∨

k∈Nj

¯ xk zj ∈ {0, 1}, ∀j ∈ [m] yi ∈ {0, 1}, ∀i ∈ [n] This integer program is equivalent to MaxSAT.

Advanced Algorithms (I) 11/14

Tossing Clever Coins

We introduce the following variables. ▶ for every i ∈ [n], yi indicates whether xi is true; ▶ for every j ∈ [m], zj indicates whether Cj is satisfied. max

m

∑

j=1

zj subject to ∑

i∈Pj

yi + ∑

k∈Nj

(1 − yk) ≥ zj, ∀j ∈ [m] s.t. Cj = ∨

i∈Pj

xi ∨ ∨

k∈Nj

¯ xk zj ∈ {0, 1}, ∀j ∈ [m] yi ∈ {0, 1}, ∀i ∈ [n] This integer program is equivalent to MaxSAT.

Advanced Algorithms (I) 11/14

Relaxation

There is no efgicient algorithm for integer programming in general Therefore, we relax its non-linear constriants max

m j

zj subject to

i Pj

yi

k Pj

yk zj j m s.t. Cj

i Pj

xi

k Nj

xk zj j m yi i n We can solve this LP in poly-time

Advanced Algorithms (I) 12/14

Relaxation

There is no efgicient algorithm for integer programming in general Therefore, we relax its non-linear constriants max

m j

zj subject to

i Pj

yi

k Pj

yk zj j m s.t. Cj

i Pj

xi

k Nj

xk zj j m yi i n We can solve this LP in poly-time

Advanced Algorithms (I) 12/14

Relaxation

There is no efgicient algorithm for integer programming in general Therefore, we relax its non-linear constriants max

m j

zj subject to

i Pj

yi

k Pj

yk zj j m s.t. Cj

i Pj

xi

k Nj

xk zj j m yi i n We can solve this LP in poly-time

Advanced Algorithms (I) 12/14

Relaxation

There is no efgicient algorithm for integer programming in general Therefore, we relax its non-linear constriants max

m

∑

j=1

zj subject to ∑

i∈Pj

yi + ∑

k∈Pj

(1 − yk) ≥ zj, ∀j ∈ [m] s.t. Cj = ∨

i∈Pj

xi ∨ ∨

k∈Nj

¯ xk 0 ≤ zj ≤ 1, ∀j ∈ [m] 0 ≤ yi ≤ 1, ∀i ∈ [n] We can solve this LP in poly-time

Advanced Algorithms (I) 12/14

Relaxation

There is no efgicient algorithm for integer programming in general Therefore, we relax its non-linear constriants max

m

∑

j=1

zj subject to ∑

i∈Pj

yi + ∑

k∈Pj

(1 − yk) ≥ zj, ∀j ∈ [m] s.t. Cj = ∨

i∈Pj

xi ∨ ∨

k∈Nj

¯ xk 0 ≤ zj ≤ 1, ∀j ∈ [m] 0 ≤ yi ≤ 1, ∀i ∈ [n] We can solve this LP in poly-time

Advanced Algorithms (I) 12/14

Algorithm

Let yi

i n

zj

j m

be an optimal solution of the LP. For each variable xi, toss an independent yi -biased coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 13/14

Algorithm

Let ({ y∗

i

}

i∈[n] ,

{ z∗

j

}

j∈[m]

) be an optimal solution of the LP. For each variable xi, toss an independent yi -biased coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 13/14

Algorithm

Let ({ y∗

i

}

i∈[n] ,

{ z∗

j

}

j∈[m]

) be an optimal solution of the LP. For each variable xi, toss an independent yi -biased coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 13/14

Algorithm

Let ({ y∗

i

}

i∈[n] ,

{ z∗

j

}

j∈[m]

) be an optimal solution of the LP. ▶ For each variable xi, toss an independent y∗

i -biased

coin. If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 13/14

Algorithm

Let ({ y∗

i

}

i∈[n] ,

{ z∗

j

}

j∈[m]

) be an optimal solution of the LP. ▶ For each variable xi, toss an independent y∗

i -biased

coin. ▶ If the coin goes HEAD, we set xi true, otherwise we set xi false.

Advanced Algorithms (I) 13/14

Analysis

A typical upper bound of OPT for LP based algorithms is OPT OPT LP

m j

zj We can further establish E X e

m j

zj Therefore, the LP rounding is a

e -approximation algorithm for

MaxSAT

Advanced Algorithms (I) 14/14

Analysis

A typical upper bound of OPT for LP based algorithms is OPT ≤ OPT(LP) =

m

∑

j=1

z∗

j

We can further establish E X e

m j

zj Therefore, the LP rounding is a

e -approximation algorithm for

MaxSAT

Advanced Algorithms (I) 14/14

Analysis

A typical upper bound of OPT for LP based algorithms is OPT ≤ OPT(LP) =

m

∑

j=1

z∗

j

We can further establish E [X] ≥ ( 1 − 1 e )

m

∑

j=1

z∗

j .

Therefore, the LP rounding is a

e -approximation algorithm for

MaxSAT

Advanced Algorithms (I) 14/14

Analysis

A typical upper bound of OPT for LP based algorithms is OPT ≤ OPT(LP) =

m

∑

j=1

z∗

j

We can further establish E [X] ≥ ( 1 − 1 e )

m

∑

j=1

z∗

j .

Therefore, the LP rounding is a (1 − 1

e

)-approximation algorithm for MaxSAT

Advanced Algorithms (I) 14/14