CSC373 Weeks 9 & 10: Approximation Algorithms & Local - - PowerPoint PPT Presentation

CSC373 Weeks 9 & 10: Approximation Algorithms & Local Search

373F19 - Nisarg Shah & Karan Singh 1

NP-Completeness

373F19 - Nisarg Shah & Karan Singh 2

• We saw that many problems are NP-complete

➢ Unlikely to have polynomial time algorithms to solve them ➢ What can we do?

• One idea:

➢ Instead of solving them exactly, solve them approximately ➢ Sometimes, we might want to use an approximation

algorithm even when we can compute an exact solution in polynomial time (WHY?)

Approximation Algorithms

373F19 - Nisarg Shah & Karan Singh 3

• We’ll focus on optimization problems

➢ Decision problem: “Is there…where…≥ 𝑙?”

• E.g. “Is there an assignment which satisfies at least 𝑙 clauses of a

given formula 𝜒?”

➢ Optimization problem: “Find…which maximizes…”

• E.g. “Find an assignment which satisfies the maximum possible

number of clauses from a given formula 𝜒.”

➢ Recall that if the decision problem is hard, then the

• ptimization problem is hard too

Approximation Algorithms

373F19 - Nisarg Shah & Karan Singh 4

• There is a function 𝑄𝑠𝑝𝑔𝑗𝑢 we want to maximize or a

function 𝐷𝑝𝑡𝑢 we want to minimize

• Given input instance 𝐽…

➢ Our algorithm returns a solution 𝐵𝑀𝐻(𝐽) ➢ An optimal solution maximizing 𝑄𝑠𝑝𝑔𝑗𝑢 or minimizing 𝐷𝑝𝑡𝑢

is 𝑃𝑄𝑈(𝐽)

➢ Then, the approximation ratio of 𝐵𝑀𝐻 on instance 𝐽 is

𝑄𝑠𝑝𝑔𝑗𝑢 𝑃𝑄𝑈 𝐽 𝑄𝑠𝑝𝑔𝑗𝑢 𝐵𝑀𝐻 𝐽

• r

𝐷𝑝𝑡𝑢 𝐵𝑀𝐻 𝐽 𝐷𝑝𝑡𝑢 𝑃𝑄𝑈 𝐽

Approximation Algorithms

373F19 - Nisarg Shah & Karan Singh 5

• Approximation ratio of 𝐵𝑀𝐻 on instance 𝐽 is

𝑄𝑠𝑝𝑔𝑗𝑢 𝑃𝑄𝑈 𝐽 𝑄𝑠𝑝𝑔𝑗𝑢 𝐵𝑀𝐻 𝐽

• r

𝐷𝑝𝑡𝑢 𝐵𝑀𝐻 𝐽 𝐷𝑝𝑡𝑢 𝑃𝑄𝑈 𝐽

➢ Note: These are defined to be ≥ 1 in each case.

• 2-approximation = half the optimal profit / twice the optimal cost
• 𝐵𝑀𝐻 has worst-case 𝑑-approximation if for each

instance 𝐽…

𝑄𝑠𝑝𝑔𝑗𝑢 𝐵𝑀𝐻 𝐽 ≥ 1 𝑑 ⋅ 𝑄𝑠𝑝𝑔𝑗𝑢 𝑃𝑄𝑈 𝐽 𝑝𝑠 𝐷𝑝𝑡𝑢 𝐵𝑀𝐻 𝐽 ≤ 𝑑 ⋅ 𝐷𝑝𝑡𝑢 𝑃𝑄𝑈 𝐽

Note

373F19 - Nisarg Shah & Karan Singh 6

• By default, when we say 𝑑-approximation, we will

always mean 𝑑-approximation in the worst case

➢ Also interesting to look at approximation in the average

case when your inputs are drawn from some distribution

• Our use of approximation ratios ≥ 1 is just a

convention

➢ Some books and papers use approximation ratios ≤ 1

convention

➢ E.g. they might say 0.5-approximation to mean that the

algorithm generates at least half the optimal profit or has at most twice the optimal cost

PTAS and FPTAS

373F19 - Nisarg Shah & Karan Singh 7

• Arbitrarily close to 1 approximations
• FPTAS: Fully polynomial time approximation

scheme

➢ For every 𝜗 > 0, there is a 1 + 𝜗 -approximation

algorithm that runs in time 𝑞𝑝𝑚𝑧 𝑜, Τ 1 𝜗 on instances of size 𝑜

• PTAS: Polynomial time approximation scheme

➢ For every 𝜗 > 0, there is a 1 + 𝜗 -approximation

algorithm that runs in time 𝑞𝑝𝑚𝑧 𝑜 on instances of size 𝑜

• Note: Could have exponential dependence on Τ

1 𝜗

Approximation Landscape

373F19 - Nisarg Shah & Karan Singh 8

➢ An FPTAS

• E.g. the knapsack problem

➢ A PTAS but no FPTAS

• E.g. the makespan problem (we’ll see)

➢ 𝑑-approximation for a constant 𝑑 > 1 but no PTAS

• E.g. vertex cover and JISP (we’ll see)

➢ Θ log 𝑜 -approximation but no constant approximation

• E.g. set cover

➢ No 𝑜1−𝜗-approximation for any 𝜗 > 0

• E.g. graph coloring and maximum independent set

Impossibility of better approximations assuming widely held beliefs like P ≠ NP 𝑜 = parameter of problem at hand

373F19 - Nisarg Shah & Karan Singh 9

Makespan Minimization

Makespan

373F19 - Nisarg Shah & Karan Singh 10

• Problem

➢ Input: 𝑛 identical machines, 𝑜 jobs, job 𝑘 requires

processing time 𝑢𝑘

➢ Output: Assign jobs to machines to minimize makespan ➢ Let 𝑇 𝑗 = set of jobs assigned to machine 𝑗 in a solution ➢ Constraints:

• Each job must run contiguously on one machine
• Each machine can process at most one job at a time

➢ Load on machine 𝑗 : 𝑀𝑗 = σ𝑘∈𝑇 𝑗 𝑢𝑘 ➢ Goal: minimize makespan 𝑀 = max

𝑗

𝑀𝑗

Makespan

373F19 - Nisarg Shah & Karan Singh 11

• Even the special case of 𝑛 = 2 machines is already

NP-hard by reduction from PARTITION

• PARTITION

➢ Input: Set 𝑇 containing 𝑜 integers ➢ Output: Can we partition 𝑇 into two sets with equal sum (i.e.

𝑇 = 𝑇1 ∩ 𝑇2, 𝑇1 ∩ 𝑇2 = ∅, and σ𝑥∈𝑇1 𝑥 = σ𝑥∈𝑇2 𝑥 )?

➢ Exercise!

• Show that PARTITION is NP-complete by reduction from SUBSET-SUM
• Show that if there is a polynomial-time algorithm for solving

MAKESPAN with 2 machines, then you can solve PARTITION in polynomial-time

Makespan

373F19 - Nisarg Shah & Karan Singh 12

• Greedy list-scheduling algorithm

➢ Consider the 𝑜 jobs in some “nice” sorted order. ➢ Assign each job 𝑘 to a machine with the smallest load so far

• Note

➢ Implementable in 𝑃 𝑜 log 𝑛 using priority queue

• Back to greedy…?

➢ But this time, we can’t hope that greedy will be optimal ➢ We can still hope that it is approximately optimal

• Which order?

Makespan

373F19 - Nisarg Shah & Karan Singh 13

• Theorem [Graham 1966]

➢ Regardless of the order, greedy gives a 2-approximation. ➢ This was the first worst-case approximation analysis

• Let optimal makespan = 𝑀∗
• To show that makespan under greedy solution is not

much worse than 𝑀∗, we need to show that 𝑀∗ isn’t too low

Makespan

373F19 - Nisarg Shah & Karan Singh 14

• Theorem [Graham 1966]

➢ Regardless of the order, greedy gives a 2-approximation.

• Fact 1: 𝑀∗ ≥ max

𝑘

𝑢𝑘

➢ Some machine must process job with highest processing time

• Fact 2: 𝑀∗ ≥

1 𝑛 σ𝑘 𝑢𝑘

➢ Total processing time is σ𝑘 𝑢𝑘 ➢ At least one machine must do at least 1/𝑛 of this work

(pigeonhole principle)

Makespan

373F19 - Nisarg Shah & Karan Singh 15

• Theorem [Graham 1966]

➢ Regardless of the order, greedy gives a 2-approximation.

• Proof:

➢ Suppose machine 𝑗 is bottleneck under greedy (so load = 𝑀𝑗) ➢ Let 𝑘∗ = last job scheduled on 𝑗 by greedy ➢ Right before 𝑘∗ was assigned to 𝑗, 𝑗 had the smallest load

• Load of other machines could have only increased from then
• 𝑀𝑗 − 𝑢𝑘∗ ≤ 𝑀𝑙, ∀𝑙

➢ Average over all 𝑙 : 𝑀𝑗 − 𝑢𝑘∗ ≤

1 𝑛 σ𝑘 𝑢𝑘

➢ 𝑀𝑗 ≤ 𝑢𝑘∗ +

1 𝑛 σ𝑘 𝑢𝑘 ≤ 𝑀∗ + 𝑀∗ = 2𝑀∗ Fact 1

Fact 2

Makespan

373F19 - Nisarg Shah & Karan Singh 16

• Theorem [Graham 1966]

➢ Regardless of the order, greedy gives a 2-approximation.

• Is our analysis tight?

➢ Essentially. ➢ There is an example where greedy does perform this badly. ➢ Note: In the upcoming example, greedy is only as bad as

2 − 1/𝑛, but you can also improve earlier analysis to show that greedy always gives 2 − 1/𝑛 approximation.

➢ So 2 − 1/𝑛 is exactly tight.

Makespan

373F19 - Nisarg Shah & Karan Singh 17

• Theorem [Graham 1966]

➢ Regardless of the order, greedy gives a 2-approximation.

• Is our analysis tight?

➢ Example:

• 𝑛(𝑛 − 1) jobs of length 1, followed by one job of length 𝑛
• Greedy evenly distributes unit length jobs on all 𝑛 machines, and

assigning the last heavy job makes makespan 𝑛 − 1 + 𝑛 = 2𝑛 − 1

• Optimal makespan is 𝑛 by evenly distributing unit length jobs among

𝑛 − 1 machines and putting the single heavy job on the remaining

➢ Idea: It seems keeping heavy jobs at the end is bad. So just

start with them first!

Makespan

373F19 - Nisarg Shah & Karan Singh 18

• Longest Processing Time (LPT) First

➢ Run the greedy algorithm but consider jobs in the

decreasing order of their processing time

• Need more facts about what the optimal cannot

beat

• Fact 3: If the bottleneck machine has only one job,

then the solution is optimal.

➢ The optimal solution must schedule that job on some

machine

Makespan

373F19 - Nisarg Shah & Karan Singh 19

• Longest Processing Time (LPT) First

➢ Run the greedy algorithm but consider jobs in the

decreasing order of their processing time

➢ Suppose 𝑢1 ≥ 𝑢2 ≥ ⋯ ≥ 𝑢𝑜

• Fact 4: If there are more than 𝑛 jobs, 𝑀∗ ≥ 2 ⋅ 𝑢𝑛+1

➢ Consider the first 𝑛 + 1 jobs ➢ All of them require processing time at least 𝑢𝑛+1 ➢ By pigeonhole principle, in the optimal solution, at least

two of them end up on the same machine

Makespan

373F19 - Nisarg Shah & Karan Singh 20

• Theorem

➢ Greedy with longest processing time first gives 3/2-

approximation

• Proof:

➢ Similar to the proof for arbitrary ordering ➢ Consider bottleneck machine 𝑗 and job 𝑘∗ that was last

scheduled on this machine by greedy

➢ Case 1: Machine 𝑗 has only one job 𝑘∗

• By Fact 3, greedy is optimal in this case (i.e. 1-approximation)

Makespan

373F19 - Nisarg Shah & Karan Singh 21

• Theorem

➢ Greedy with longest processing time first gives 3/2-

approximation

• Proof:

➢ Similar to the proof for arbitrary ordering ➢ Consider bottleneck machine 𝑗 and job 𝑘∗ that was last

scheduled on this machine by greedy

➢ Case 2: Machine 𝑗 has at least two jobs

• Job 𝑘∗ must have 𝑢𝑘∗ ≤ 𝑢𝑛+1
• As before, 𝑀 = 𝑀𝑗 = 𝑀𝑗 − 𝑢𝑘∗ + 𝑢𝑘∗ ≤ 1.5 𝑀∗

Same as before ≤ 𝑀∗ ≤ 𝑀∗/2 𝑢𝑘∗ ≤ 𝑢𝑛+1 and Fact 4

Makespan

373F19 - Nisarg Shah & Karan Singh 22

• Theorem

➢ Greedy with LPT rule gives 3/2-approximation ➢ Is our analysis tight? No!

• Theorem [Graham 1966]

➢ Greedy with LPT rule gives 4/3-approximation ➢ Is Graham’s 4/3 approximation tight?

• Essentially.
• In the upcoming example, greedy is only as bad as

4 3 − 1 3𝑛

• But Graham actually proves

4 3 − 1 3𝑛 upper bound. So this is exactly

tight.

Makespan

373F19 - Nisarg Shah & Karan Singh 23

• Theorem

➢ Greedy with LPT rule gives 3/2-approximation ➢ Is our analysis tight? No!

• Theorem [Graham 1966]

➢ Greedy with LPT rule gives 4/3-approximation ➢ Tight example:

• 2 jobs of lengths 𝑛, 𝑛 + 1, … , 2𝑛 − 1, one more job of length 𝑛
• Greedy-LPT has makespan 4𝑛 − 1 (verify!)
• OPT has makespan 3𝑛 (verify!)
• Thus, approximation ratio is at least as bad as

4𝑛−1 3𝑛 = 4 3 − 1 3𝑛

373F19 - Nisarg Shah & Karan Singh 24

Unweighted Vertex Cover

Unweighted Vertex Cover

373F19 - Nisarg Shah & Karan Singh 25

• Problem

➢ Input: Undirected graph 𝐻 = (𝑊, 𝐹) ➢ Output: Vertex cover 𝑇 of minimum cardinality ➢ Recall: 𝑇 is vertex cover if every edge has at least one

endpoint in 𝑇

➢ We already saw that this problem is NP-hard

• Q: What would be a good greedy algorithm for this

problem?

Unweighted Vertex Cover

373F19 - Nisarg Shah & Karan Singh 26

• Greedy edge-selection algorithm:

➢ Start with 𝑇 = ∅ ➢ While there exists an edge whose both endpoints are not

in 𝑇, add both its endpoints to 𝑇

• Hmm…

➢ Why are we selecting edges rather than vertices? ➢ Why are we adding both endpoints? ➢ We’ll see..

Unweighted Vertex Cover

373F19 - Nisarg Shah & Karan Singh 27

Unweighted Vertex Cover

373F19 - Nisarg Shah & Karan Singh 28

• Theorem:

➢ Greedy edge-selection algorithm for unweighted vertex

cover gives 2-approximation.

• Question:

➢ If 𝑇 is any vertex cover (containing 𝑇 vertices), 𝑁 is any

matching (containing |𝑁| edges), then what is the relation between |𝑇| and 𝑁 ?

➢ Answer: 𝑇 ≥ |𝑁|.

Unweighted Vertex Cover

373F19 - Nisarg Shah & Karan Singh 29

• Theorem:

➢ Greedy edge-selection algorithm for unweighted vertex

cover gives 2-approximation.

• Proof:

➢ Let 𝑇∗ = min vertex cover, 𝑇 = solution returned by greedy ➢ By design, 𝑇 = 2 ⋅ |𝑁| ➢ Because 𝑁 is a matching, 𝑇∗ ≥ |𝑁|

(By last slide)

➢ Hence, 𝑇 ≤ 2|𝑇∗| ∎

Unweighted Vertex Cover

373F19 - Nisarg Shah & Karan Singh 30

• Theorem:

➢ Greedy edge-selection algorithm for unweighted vertex

cover gives 2-approximation.

• Corollary:

➢ If 𝑁∗ is maximum matching, then greedy finds matching

𝑁 with 𝑁 ≥

1 2 𝑁∗

• Proof:

➢ By design, 𝑁 =

1 2 |𝑇|

➢ 𝑇 ≥ 𝑁∗

(Same reason again!)

➢ Hence, 𝑁 ≥

1 2 𝑁∗ ∎ This is a so-called maximal matching which cannot be extended

Unweighted Vertex Cover

373F19 - Nisarg Shah & Karan Singh 31

• What about a greedy vertex selection algorithm?

➢ Start with 𝑇 = ∅ ➢ While 𝑇 is not a vertex cover:

• Choose a vertex 𝑤 which maximizes the number of uncovered

edges incident on it

• Add 𝑤 to 𝑇

➢ Interestingly, this only gives log 𝑒max approximation,

where 𝑒max is the maximum degree of any vertex

• But unlike the edge-selection version, this generalizes to set cover,

and gives provably best possible approximation ratio for set cover in polynomial time (unless P=NP)

Unweighted Vertex Cover

373F19 - Nisarg Shah & Karan Singh 32

• Theorem [Dinur-Safra 2004]:

➢ Unless P = NP, there is no 𝜍-approximation polynomial-

time algorithm for unweighted vertex cover for any 𝜍 < 1.3606.

• Q: How can something like this be proven?

➢ We’ll see later. ➢ Basically, reduce “solving a hard problem” (e.g. 3SAT) to

“finding any good approximation of current problem”

Unweighted Vertex Cover

373F19 - Nisarg Shah & Karan Singh 33

• Theorem [Dinur-Safra 2004]:

➢ Unless P = NP, there is no 𝜍-approximation polynomial-

time algorithm for unweighted vertex cover for any 𝜍 < 1.3606.

• Q: How can something like this be proven?

➢ We’ll see later. ➢ Basically, reduce “solving a hard problem” (e.g. 3SAT) to

“finding any good approximation of current problem”

373F19 - Nisarg Shah & Karan Singh 34

Weighted Vertex Cover

Weighted Vertex Cover

373F19 - Nisarg Shah & Karan Singh 35

• Problem

➢ Input: Undirected graph 𝐻 = (𝑊, 𝐹), weights 𝑥 ∶ 𝑊 → 𝑆≥0 ➢ Output: Vertex cover 𝑇 of minimum total weight

• The same greedy algorithm doesn’t work

➢ Gives arbitrarily bad approximation ➢ Obvious modification which try to take weights into

account also don’t work

➢ Need another strategy…

ILP Formulation

373F19 - Nisarg Shah & Karan Singh 36

➢ For each vertex 𝑤, create a binary variable 𝑦𝑤 ∈ {0,1}

indicating whether vertex 𝑤 is chosen in the vertex cover

➢ Then, computing min weight vertex cover is equivalent to

solving the following integer linear program min Σ𝑤 𝑥𝑤 ⋅ 𝑦𝑤 subject to 𝑦𝑣 + 𝑦𝑤 ≥ 1, ∀ 𝑣, 𝑤 ∈ 𝐹 𝑦𝑤 ∈ 0,1 , ∀𝑤 ∈ 𝑊

LP Relaxation

373F19 - Nisarg Shah & Karan Singh 37

• What if we solve this LP instead of the original ILP?

min Σ𝑤 𝑥𝑤 ⋅ 𝑦𝑤 subject to 𝑦𝑣 + 𝑦𝑤 ≥ 1, ∀ 𝑣, 𝑤 ∈ 𝐹 𝑦𝑤 ∈ 0,1 , ∀𝑤 ∈ 𝑊 min Σ𝑤 𝑥𝑤 ⋅ 𝑦𝑤 subject to 𝑦𝑣 + 𝑦𝑤 ≥ 1, ∀ 𝑣, 𝑤 ∈ 𝐹 𝑦𝑤 ≥ 0, ∀𝑤 ∈ 𝑊 ILP with binary variables LP with real variables

Rounding LP Solution

373F19 - Nisarg Shah & Karan Singh 38

• What if we solve this LP instead of the original ILP?

➢ Minimizes objective over a larger feasible space ➢ Optimal LP objective value ≤ optimal ILP objective value ➢ But optimal LP solution 𝑦∗ is not a binary vector

• Can we round it to some binary vector ො

𝑦 without increasing the

• bjective value too much?

min Σ𝑤 𝑥𝑤 ⋅ 𝑦𝑤 subject to 𝑦𝑣 + 𝑦𝑤 ≥ 1, ∀ 𝑣, 𝑤 ∈ 𝐹 𝑦𝑤 ≥ 0, ∀𝑤 ∈ 𝑊 LP with real variables min Σ𝑤 𝑥𝑤 ⋅ 𝑦𝑤 subject to 𝑦𝑣 + 𝑦𝑤 ≥ 1, ∀ 𝑣, 𝑤 ∈ 𝐹 𝑦𝑤 ∈ 0,1 , ∀𝑤 ∈ 𝑊 ILP with binary variables

Rounding LP Solution

373F19 - Nisarg Shah & Karan Singh 39

• Consider LP optimal solution 𝑦∗

➢ Let ො

𝑦𝑤 = 1 whenever 𝑦𝑤

∗ ≥ 0.5 and ො

𝑦𝑤 = 0 otherwise

➢ Claim 1: ො

𝑦 is a feasible solution of ILP (i.e. a vertex cover)

• For every edge 𝑣, 𝑤 ∈ 𝐹, at least one of 𝑦𝑣

∗, 𝑦𝑤 ∗ is at least 0.5

• So at least one of ො

𝑦𝑣, ො 𝑦𝑤 is 1 ∎

min Σ𝑤 𝑥𝑤 ⋅ 𝑦𝑤 subject to 𝑦𝑣 + 𝑦𝑤 ≥ 1, ∀ 𝑣, 𝑤 ∈ 𝐹 𝑦𝑤 ≥ 0, ∀𝑤 ∈ 𝑊 LP with real variables min Σ𝑤 𝑥𝑤 ⋅ 𝑦𝑤 subject to 𝑦𝑣 + 𝑦𝑤 ≥ 1, ∀ 𝑣, 𝑤 ∈ 𝐹 𝑦𝑤 ∈ 0,1 , ∀𝑤 ∈ 𝑊 ILP with binary variables

Rounding LP Solution

373F19 - Nisarg Shah & Karan Singh 40

• Consider LP optimal solution 𝑦∗

➢ Let ො

𝑦𝑤 = 1 whenever 𝑦𝑤

∗ ≥ 0.5 and ො

𝑦𝑤 = 0 otherwise

➢ Claim 2: σ𝑤 𝑥𝑤 ⋅ ො

𝑦𝑤 ≤ 2 ∗ σ𝑤 𝑥𝑤 ⋅ 𝑦𝑤

∗

• Weight only increases when some 𝑦𝑤

∗ ∈ [0.5,1] is shifted up to 1

• At most doubling the variable, so at least doubling the weight ∎

min Σ𝑤 𝑥𝑤 ⋅ 𝑦𝑤 subject to 𝑦𝑣 + 𝑦𝑤 ≥ 1, ∀ 𝑣, 𝑤 ∈ 𝐹 𝑦𝑤 ≥ 0, ∀𝑤 ∈ 𝑊 LP with real variables min Σ𝑤 𝑥𝑤 ⋅ 𝑦𝑤 subject to 𝑦𝑣 + 𝑦𝑤 ≥ 1, ∀ 𝑣, 𝑤 ∈ 𝐹 𝑦𝑤 ∈ 0,1 , ∀𝑤 ∈ 𝑊 ILP with binary variables

Rounding LP Solution

373F19 - Nisarg Shah & Karan Singh 41

• Consider LP optimal solution 𝑦∗

➢ Let ො

𝑦𝑤 = 1 whenever 𝑦𝑤

∗ ≥ 0.5 and ො

𝑦𝑤 = 0 otherwise

➢ Hence, ො

𝑦 is a vertex cover with weight at most 2 ∗ LP optimal value ≤ 2 ∗ ILP optimal value min Σ𝑤 𝑥𝑤 ⋅ 𝑦𝑤 subject to 𝑦𝑣 + 𝑦𝑤 ≥ 1, ∀ 𝑣, 𝑤 ∈ 𝐹 𝑦𝑤 ≥ 0, ∀𝑤 ∈ 𝑊 LP with real variables min Σ𝑤 𝑥𝑤 ⋅ 𝑦𝑤 subject to 𝑦𝑣 + 𝑦𝑤 ≥ 1, ∀ 𝑣, 𝑤 ∈ 𝐹 𝑦𝑤 ∈ 0,1 , ∀𝑤 ∈ 𝑊 ILP with binary variables

General LP Relaxation Strategy

373F19 - Nisarg Shah & Karan Singh 42

• Your NP-complete problem amounts to solving

➢ Max 𝑑𝑈𝑦 subject to 𝐵𝑦 ≤ 𝑐, 𝑦 ∈ ℕ (need not be binary)

• Instead, solve:

➢ Max 𝑑𝑈𝑦 subject to 𝐵𝑦 ≤ 𝑐, 𝑦 ∈ ℝ≥0 (LP relaxation)

• LP optimal value ≥ ILP optimal value (for maximization)

➢ 𝑦∗ = LP optimal solution ➢ Round 𝑦∗ to ො

𝑦 such that 𝑑𝑈 ො 𝑦 ≥

𝑑𝑈𝑦∗ 𝜍

≥

ILP optimal value 𝜍

➢ Gives 𝜍-approximation

• Info: Best 𝜍 you can hope to get via this approach for a particular

LP-ILP combination is called the integrality gap

373F19 - Nisarg Shah & Karan Singh 43

𝑙-Center Problem

𝑙-Center Problem

373F19 - Nisarg Shah & Karan Singh 44

• Problem

➢ Input: Set of 𝑜 sites 𝑡1, … , 𝑡𝑜 and an integer 𝑙 ➢ Output: Return a set 𝐷 of 𝑙 centers s.t. the maximum

distance of any site from its nearest center is minimized

• Minimize 𝑠 𝐷 =

max

𝑗∈ 1,…,𝑜 𝑒(𝑡𝑗, 𝐷), where 𝑒 𝑡𝑗, 𝐷 = min 𝑑∈𝐷 𝑒 𝑡𝑗, 𝑑

➢ Sites are points in some metric space with distance 𝑒

satisfying:

• Identity: 𝑒 𝑦, 𝑦 = 0 for all 𝑦
• Symmetry: 𝑒 𝑦, 𝑧 = 𝑒(𝑧, 𝑦) for all 𝑦, 𝑧
• Triangle inequality: 𝑒 𝑦, 𝑨 ≤ 𝑒 𝑦, 𝑧 + 𝑒 𝑧, 𝑨 for all 𝑦, 𝑧, 𝑨

𝑙-Center Problem

373F19 - Nisarg Shah & Karan Singh 45

• Problem

➢ Input: Set of 𝑜 sites 𝑡1, … , 𝑡𝑜 and an integer 𝑙 ➢ Output: Return a set 𝐷 of 𝑙 centers s.t. the maximum

distance of any site from its nearest center is minimized

• Minimize 𝑠 𝐷 =

max

𝑗∈ 1,…,𝑜 𝑒(𝑡𝑗, 𝐷), where 𝑒 𝑡𝑗, 𝐷 = min 𝑑∈𝐷 𝑒 𝑡𝑗, 𝑑

➢ Given 𝐷, note that 𝑠(𝐷) is the minimum radius 𝑠 such

that if we draw a ball of radius 𝑠 around every center in 𝐷, then the balls collectively cover all the sites

373F19 - Nisarg Shah & Karan Singh 46

𝑙-Center Problem

Objective value

Bad Greedy

373F19 - Nisarg Shah & Karan Singh 47

• Bad greedy (forget about running time)

➢ Put the first center at the optimal location for 𝑙 = 1 ➢ Put every next center to reduce the objective value as

much as possible given the centers already placed

• Arbitrarily bad approximation

Good Greedy

373F19 - Nisarg Shah & Karan Singh 48

• Good greedy

➢ Put the first center at an arbitrary site ➢ Put every next center at a site whose distance to its

nearest center is maximum among all sites

❑ Good Greedy ➢ 𝐷1 ← 𝑡1

(arbitrary site works)

➢ For 𝑘 = 2, … , 𝑙: ➢ 𝑡𝑗 ← argmax

𝑡

𝑒(𝑡, 𝐷

𝑘−1); Δ𝑘 = 𝑒 𝑡𝑗, 𝐷 𝑘−1

➢ 𝐷j ← 𝐷j−1 ∪ 𝑡𝑗 ➢ Return 𝐷k

Good Greedy

373F19 - Nisarg Shah & Karan Singh 49

• For reasons that will soon become clear…

➢ Imagine that we run good greedy for 𝑙 + 1 steps rather

than 𝑙 steps, and obtain 𝐷𝑙+1

➢ Note: The 𝑙 + 1 points in 𝐷𝑙+1 are sites

❑

Good Greedy

➢ 𝐷1 ← 𝑡1

(arbitrary site works)

➢ For 𝑘 = 2, … , 𝑙: ➢ 𝑡𝑗 ← argmax

𝑡

𝑒(𝑡, 𝐷

𝑘−1); Δ𝑘 = 𝑒 𝑡𝑗, 𝐷 𝑘−1

➢ 𝐷j ← 𝐷j−1 ∪ 𝑡𝑗 ➢ Return 𝐷k

Good Greedy

373F19 - Nisarg Shah & Karan Singh 50

• Claim: 𝑒 𝑡𝑗, 𝑡

𝑘 ≥ 𝑠 𝐷𝑙 for all 𝑡𝑗, 𝑡 𝑘 ∈ 𝐷𝑙+1

➢ Proof: By construction of the algorithm.

• At each iteration 𝑘, we add a new center that is at least Δ𝑘 far from

all previous centers

• Δ𝑘 decreases as 𝑘 increases (Why?)
• Δ𝑙+1 = 𝑠 𝐷𝑙

❑

Good Greedy

➢ 𝐷1 ← 𝑡1

(arbitrary site works)

➢ For 𝑘 = 2, … , 𝑙: ➢ 𝑡𝑗 ← argmax

𝑡

𝑒(𝑡, 𝐷

𝑘−1); Δ𝑘 = 𝑒 𝑡𝑗, 𝐷 𝑘−1

➢ 𝐷j ← 𝐷j−1 ∪ 𝑡𝑗 ➢ Return 𝐷k

373F19 - Nisarg Shah & Karan Singh 51

𝑙-Center Problem

373F19 - Nisarg Shah & Karan Singh 52

𝑙-Center Problem

373F19 - Nisarg Shah & Karan Singh 53

𝑙-Center Problem

373F19 - Nisarg Shah & Karan Singh 54

𝑙-Center Problem

373F19 - Nisarg Shah & Karan Singh 55

𝑙-Center Problem

Good Greedy

373F19 - Nisarg Shah & Karan Singh 56

• Theorem: If 𝐷∗ is the optimal set of 𝑙 centers, then

𝑠 𝐷𝑙 ≤ 2 ⋅ 𝑠 𝐷∗

• Proof:

➢ Draw a ball of radius 𝑠(𝐷∗) from each center in 𝐷∗ ➢ By pigeonhole principle, at least two 𝑡𝑗, 𝑡

𝑘 ∈ 𝐷𝑙+1 must

belong to the same ball (say centered at 𝑑∗ ∈ 𝐷∗)

• Hence, 𝑒 𝑡𝑗, 𝑑∗ , 𝑒 𝑡

𝑘, 𝑑∗ ≤ 𝑠 𝐷∗

➢ But by our claim:

𝑠 𝐷𝑙 ≤ 𝑒 𝑡𝑗, 𝑡

𝑘

≤ 𝑒 𝑡𝑗, 𝑑∗ + 𝑒 𝑡

𝑘, 𝑑∗ ≤ 2 ⋅ 𝑠 𝐷∗

➢ Done!

Hardness of Approximation

373F19 - Nisarg Shah & Karan Singh 57

• Best polynomial time approximation?

➢ Good greedy gives 2-approximation in polynomial time ➢ Can we get a better approximation?

• Theorem: Unless P=NP, there is no polynomial time

algorithm which gives 𝜍-approximation for the 𝑙- center problem for 𝜍 < 2.

• How do we prove this?

Hardness of Approximation

373F19 - Nisarg Shah & Karan Singh 58

• Theorem: Unless P=NP, there is no polynomial time

algorithm which gives 𝜍-approximation for the 𝑙- center problem for 𝜍 < 2.

• How do we prove this?

➢ Same reduction idea:

• Show that if there is a polytime algorithm which gives 𝜍-apx to 𝑙-

center for some 𝜍 < 2, then using this algorithm, we can solve a known NP-complete problem in polytime.

• Hmm. Which NP-complete problem should we use?
• How about FriendlyRepresentatives problem from assignment 3?

Hardness of Approximation

373F19 - Nisarg Shah & Karan Singh 59

• Theorem: Unless P=NP, there is no polynomial time

algorithm which gives 𝜍-approximation for the 𝑙- center problem for 𝜍 < 2.

• Proof:

➢ Consider an instance of FriendlyRepresentatives

• Given a set of people 𝑂, a friendship relation 𝐺, and an integer 𝑛,

we want to check if there exists a subset 𝑇 ⊆ 𝑂 of 𝑛 people such that every person not in 𝑇 is friends with someone in 𝑇.

• Denote this by (𝑂, 𝐺, 𝑛)

Hardness of Approximation

373F19 - Nisarg Shah & Karan Singh 60

• Theorem: Unless P=NP, there is no polynomial time

algorithm which gives 𝜍-approximation for the 𝑙- center problem for 𝜍 < 2.

• Proof:

➢ Consider an instance (𝑂, 𝐺, 𝑛) of FriendlyRepresentatives ➢ Create an instance of 𝑙-Center as follows

• Create a site 𝑡𝑗 for each person 𝑗 ∈ 𝑂
• Define 𝑒 𝑡𝑗, 𝑡

𝑘 = 1 if 𝑗, 𝑘 ∈ 𝐺 and 2 if 𝑗, 𝑘 ∉ 𝐺

• Check that this satisfies triangle inequality
• Set 𝑙 = 𝑛
• Note: There are no other points in this metric space, so you must

place centers on sites.

Hardness of Approximation

373F19 - Nisarg Shah & Karan Singh 61

• Theorem: Unless P=NP, there is no polynomial time

algorithm which gives 𝜍-approximation for the 𝑙- center problem for 𝜍 < 2.

• Proof:

➢ 𝐷 is a set of friendly representatives if and only if 𝑠 𝐷 = 1

• Every center is obviously at distance 0 from itself
• Every non-center 𝑡

𝑘 is at distance at most 1 from some 𝑡𝑗 ∈ 𝐷 if and

• nly if every person not in 𝐷 is friends with someone in 𝐷

➢ There are only two possibilities:

• YES: There exists 𝐷 with 𝑠 𝐷 = 1
• NO: Every 𝐷 has 𝑠 𝐷 = 2

Hardness of Approximation

373F19 - Nisarg Shah & Karan Singh 62

• Theorem: Unless P=NP, there is no polynomial time

algorithm which gives 𝜍-approximation for the 𝑙- center problem for 𝜍 < 2.

• Proof:

➢ YES: There exists 𝐷 with 𝑠 𝐷 = 1

• Since our algorithm gives 𝜍-approximation with 𝜍 < 2, it must

return a set 𝐷 with 𝑠 𝐷 < 2

• But 𝑠 𝐷 < 2 means that 𝑠 𝐷 = 1
• So the algorithm returns 𝐷 with 𝑠 𝐷 = 1

➢ NO: Our algorithm returns a 𝐷 with 𝑠 𝐷 = 2 ➢ So checking 𝑠(𝐷) of the 𝐷 returned by algorithm allows

solving FriendlyRepresentatives!

373F19 - Nisarg Shah & Karan Singh 63

Weighted Set Packing

Weighted Set Packing

373F19 - Nisarg Shah & Karan Singh 64

• Problem

➢ Input: A collection of sets 𝒯 = 𝑇1, … , 𝑇𝑜 with values

𝑤1, … , 𝑤𝑜 ≥ 0. There are m set elements.

➢ Output: Pick disjoint sets with maximum total value, i.e.

pick 𝑋 ⊆ {1, … , 𝑜} to maximize σ𝑗∈𝑋 𝑤𝑗 subject to the constraint that for all 𝑗, 𝑘 ∈ 𝑋, 𝑇𝑗 ∩ 𝑇

𝑘 = ∅.

➢ This is known to be an NP-hard problem ➢ It is also known that for any constant 𝜗 > 0, you cannot

get 𝑃 𝑛 Τ

1 2−𝜗 approximation in polynomial time unless

NP=ZPP (widely believed to be not true)

Greedy Template

CSC304 - Nisarg Shah 65

• Sort the sets in some order, consider them one-by-
• ne, and take any set that you can along the way.
• Greedy Algorithm:

➢ Sort the sets in a specific order. ➢ Relabel them as 1,2, … , 𝑜 in this order. ➢ 𝑋 ← ∅ ➢ For 𝑗 = 1, … , 𝑜:

• If 𝑇𝑗 ∩ 𝑇

𝑘 = ∅ for every 𝑘 ∈ 𝑋, then 𝑋 ← 𝑋 ∪ {𝑗}

➢ Return 𝑋.

Greedy Algorithm

CSC304 - Nisarg Shah 66

• What order should we sort the sets by?
• We want to take sets with high values.

➢ 𝑤1 ≥ 𝑤2 ≥ ⋯ ≥ 𝑤𝑜? Only 𝑛-approximation 

• We don’t want to exhaust many items too soon.

➢

𝑤1 𝑇1 ≥ 𝑤2 𝑇2 ≥ ⋯ 𝑤𝑜 𝑇𝑜 ? Also 𝑛-approximation 

• 𝑛-approximation :

𝑤1 𝑇1 ≥ 𝑤2 𝑇2 ≥ ⋯ 𝑤𝑜 𝑇𝑜 ? [Lehmann et al. 2011]

Proof of Approximation

CSC2556 - Nisarg Shah 67

• Definitions

➢ 𝑃𝑄𝑈 = Some optimal solution ➢ 𝑋 = Solution returned by our greedy algorithm ➢ For 𝑗 ∈ 𝑋,

𝑃𝑄𝑈𝑗 = 𝑘 ∈ 𝑃𝑄𝑈, 𝑘 ≥ 𝑗 ∶ 𝑇𝑗 ∩ 𝑇

𝑘 ≠ ∅

OPTi has future j in OPT blocked for inclusion in greedy W because of choosing I (i is also in OPTi).

• Claim 1: 𝑃𝑄𝑈 ⊆ڂ𝑗∈𝑋 𝑃𝑄𝑈𝑗

If j from OPT is in W => j in OPTj, else j must be in some OPTi or the greedy algorithm would have chosen it.

• Claim 2: It is enough to show that ∀𝑗 ∈ 𝑋

𝑛 ⋅ 𝑤𝑗 ≥ Σ𝑘∈𝑃𝑄𝑈𝑗 𝑤𝑘

The value of greedy choice i is at least as good 1/ 𝑛 * the values from the optimal solution it blocks, and all elements of OPT will be accounted for by the union of OPTi’s.

• Observation: For 𝑘 ∈ 𝑃𝑄𝑈𝑗, 𝑤𝑘 ≤ 𝑤𝑗 ⋅

𝑇𝑘 𝑇𝑗

Greedy ordering.

Proof of Approximation

CSC2556 - Nisarg Shah 68

• Summing over all 𝑘 ∈ 𝑃𝑄𝑈𝑗 :

Σ𝑘∈𝑃𝑄𝑈𝑗 𝑤𝑘 ≤ 𝑤𝑗 𝑇𝑗 ⋅ Σ𝑘∈𝑃𝑄𝑈𝑗 𝑇

𝑘

• Using Cauchy-Schwarz (Σ𝑗 𝑦𝑗𝑧𝑗 ≤

Σ𝑗 𝑦𝑗

2 ⋅

Σ𝑗 𝑧𝑗

2)

Σ𝑘∈𝑃𝑄𝑈𝑗

• 1. 𝑇

𝑘 ≤

𝑃𝑄𝑈𝑗 ⋅ Σ𝑘∈𝑃𝑄𝑈𝑗 𝑇

𝑘

≤ 𝑇𝑗 ⋅ 𝑛

Every element in Si can block at most one set in OPT => OPTi <= Si. Also note every Sj in OPTi is disjoint because it belongs to OPT. so the sum of these Sj’s is at most m.

373F19 - Nisarg Shah & Karan Singh 69

Local Search Paradigm

Local Search

373F19 - Nisarg Shah & Karan Singh 70

• A heuristic paradigm for solving complex problems

➢ Sometimes it might provably return an optimal solution ➢ But even if not, it might give a good approximation

• Idea:

➢ Start with some solution 𝑇 ➢ While there is a “better” solution 𝑇′ in the local neighborhood of 𝑇 ➢

Switch to 𝑇’

• Need to define what is “better” and what is a “local

neighborhood”

Local Search

373F19 - Nisarg Shah & Karan Singh 71

• Sometimes local search provably returns an
• ptimal solution
• We already saw such an example: network flow

➢ Start with zero flow ➢ “Local neighborhood”

• Set of all flows which can be obtained by augmenting the current

flow along a path in the residual graph

➢ “Better”

• Higher flow value

Local Search

373F19 - Nisarg Shah & Karan Singh 72

• But sometimes it doesn’t return an optimal

solution, and “gets stuck” in a local maxima

Local Search

373F19 - Nisarg Shah & Karan Singh 73

• In that case, we want to bound the ratio between

the optimal solution and the worst solution local search might return

Worst ratio

373F19 - Nisarg Shah & Karan Singh 74

Max-Cut

Max-Cut

373F19 - Nisarg Shah & Karan Singh 75

• Problem

➢ Input: An undirected graph 𝐻 = (𝑊, 𝐹) ➢ Output: A partition (𝐵, 𝐶) of 𝑊 that maximizes the

number of edges going across the cut, i.e., maximizes |𝐹′| where 𝐹′ = 𝑣, 𝑤 ∈ 𝐹 𝑣 ∈ 𝐵, 𝑤 ∈ 𝐶}

➢ This is also known to be an NP-hard problem ➢ What is a natural local search algorithm for this problem?

• Given a current partition, what small change can you do to

improve the objective value?

Max-Cut

373F19 - Nisarg Shah & Karan Singh 76

• Local Search

➢ Initialize (𝐵, 𝐶) arbitrarily. ➢ While there is a vertex 𝑣 such that moving 𝑣 to the other

side improves the objective value:

• Move 𝑣 to the other side.

➢ When does moving 𝑣, say from 𝐵 to 𝐶, improve the

• bjective value?
• When 𝑣 has more incident edges going within the cut than across

the cut, i.e., when 𝑣, 𝑤 ∈ 𝐹 𝑤 ∈ 𝐵 > 𝑣, 𝑤 ∈ 𝐹 𝑤 ∈ 𝐶

Max-Cut

373F19 - Nisarg Shah & Karan Singh 77

• Local Search

➢ Initialize (𝐵, 𝐶) arbitrarily. ➢ While there is a vertex 𝑣 such that moving 𝑣 to the other

side improves the objective value:

• Move 𝑣 to the other side.

➢ Why does the algorithm stop?

• Every iteration increases the number of edges across the cut by at

least 1, so the algorithm must stop in at most |𝐹| iterations

Max-Cut

373F19 - Nisarg Shah & Karan Singh 78

• Local Search

➢ Initialize (𝐵, 𝐶) arbitrarily. ➢ While there is a vertex 𝑣 such that moving 𝑣 to the other

side improves the objective value:

• Move 𝑣 to the other side.

➢ Approximation ratio?

• At the end, every vertex has at least as many edges going across

the cut as within the cut

• Hence, at least half of all edges must be going across the cut
• Exercise: Prove this formally by writing equations.

Weighted Max-Cut

373F19 - Nisarg Shah & Karan Singh 79

• Variant

➢ Now we’re given integral edge weights 𝑥: 𝐹 → ℕ ➢ The goal is to maximize the total weight of edges going

across the cut

• Algorithm

➢ The same algorithm works, but now we move 𝑣 to the

• ther side if the total weight of its incident edges going

within the cut is greater than the total weight of its incident edges going across the cut

Weighted Max-Cut

373F19 - Nisarg Shah & Karan Singh 80

• Number of iterations?

➢ In the unweighted case, we said that the number of

edges going across the cut must increase by at least 1, so it takes at most |𝐹| iterations

➢ In the weighted case, the total weight of edges going

across the cut increases by at least 1, but this could take up to σ𝑓∈𝐹 𝑥𝑓 iterations, which is exponential in the input length

• There are examples where the local search actually takes

exponentially many steps

Weighted Max-Cut

373F19 - Nisarg Shah & Karan Singh 81

• Number of iterations?

➢ But we can find a 2 + 𝜗 approximation in time polynomial

in the input length and

1 𝜗

➢ The idea is to only move vertices when it “sufficiently

improves” the objective value

Weighted Max-Cut

373F19 - Nisarg Shah & Karan Singh 82

• Better approximations?

➢ Theorem [Goemans-Williamson]: There exists a

polynomial time algorithm for max-cut with approximation ratio

2 𝜌 ⋅ min 0≤𝜄≤𝜌 𝜄 1−cos 𝜄 ≈ 0.878

• Uses “semidefinite programming” and “randomized rounding”
• Note: The literature from here on uses approximation ratios ≤ 1,

so we will follow that convention in the remaining slides.

➢ If the unique games conjecture is true, then this is tight

373F19 - Nisarg Shah & Karan Singh 83

Exact Max-𝑙-SAT

Exact Max-𝑙-SAT

373F19 - Nisarg Shah & Karan Singh 84

• Problem

➢ Input: An exact 𝑙-SAT formula 𝜒 = 𝐷1 ∧ 𝐷2 ∧ ⋯ ∧ 𝐷𝑛,

where each clause 𝐷𝑗 has exactly 𝑙 literals, and a weight 𝑥𝑗 ≥ 0 of each clause 𝐷𝑗

➢ Output: A truth assignment 𝜐 maximizing the number (or

total weight) of clauses satisfied under 𝜐

➢ Let us denote by 𝑋(𝜐) the total weight of clauses

satisfied under 𝜐

➢ What is a good definition of “local neighborhood”?

Exact Max-𝑙-SAT

373F19 - Nisarg Shah & Karan Singh 85

• Local neighborhood:

➢ 𝑂𝑒(𝜐) = set of all truth assignments which can be

• btained by changing the value of at most 𝑒 variables in 𝜐
• Theorem: The local search with 𝑒 = 1 gives a Τ

2 3

approximation to Exact Max-2-SAT.

Exact Max-𝑙-SAT

373F19 - Nisarg Shah & Karan Singh 86

• Theorem: The local search with 𝑒 = 1 gives a Τ

2 3

approximation to Exact Max-2-SAT.

• Proof:

➢ Let 𝜐 be a local optimum

• 𝑇0 = set of clauses not satisfied under 𝜐
• 𝑇1 = set of clauses from which exactly one literal is true under 𝜐
• 𝑇2 = set of clauses from which both literals are true under 𝜐
• 𝑋 𝑇0 , 𝑋 𝑇1 , 𝑋 𝑇2 be the corresponding total weights
• Goal: 𝑋 𝑇1 + 𝑋 𝑇2 ≥ Τ

2 3 ⋅ 𝑋 𝑇0 + 𝑋 𝑇1 + 𝑋 𝑇2

• Equivalently, 𝑋 𝑇0 ≤ Τ

1 3 ⋅ 𝑋 𝑇0 + 𝑋 𝑇1 + 𝑋 𝑇2

Exact Max-𝑙-SAT

373F19 - Nisarg Shah & Karan Singh 87

• Theorem: The local search with 𝑒 = 1 gives a Τ

2 3

approximation to Exact Max-2-SAT.

• Proof:

➢ Let 𝜐 be a local optimum

• 𝑇0 = set of clauses not satisfied under 𝜐
• 𝑇1 = set of clauses from which exactly one literal is true under 𝜐
• 𝑇2 = set of clauses from which both literals are true under 𝜐
• 𝑋 𝑇0 , 𝑋 𝑇1 , 𝑋 𝑇2 be the corresponding total weights
• Goal: 𝑋 𝑇1 + 𝑋 𝑇2 ≥ Τ

2 3 ⋅ 𝑋 𝑇0 + 𝑋 𝑇1 + 𝑋 𝑇2

• Equivalently, 𝑋 𝑇0 ≤ Τ

1 3 ⋅ 𝑋 𝑇0 + 𝑋 𝑇1 + 𝑋 𝑇2

Exact Max-𝑙-SAT

373F19 - Nisarg Shah & Karan Singh 88

• Theorem: The local search with 𝑒 = 1 gives a Τ

2 3

approximation to Exact Max-2-SAT.

• Proof:

➢ Clause 𝐷 involves variable 𝑘 if it contains 𝑦𝑘 or ഥ

𝑦𝑘

• 𝐵𝑘 = set of clauses in 𝑇0 involving variable 𝑘
• 𝐶

𝑘 = set of clauses in 𝑇1 involving variable 𝑘 such that it is the

literal of variable 𝑘 that is true under 𝜐

• 𝐷

𝑘 = set of clauses in 𝑇2 involving variable 𝑘

• 𝑋 𝐵𝑘 , 𝑋 𝐶

𝑘 , 𝑋 𝐷 𝑘 be the corresponding total weights

Exact Max-𝑙-SAT

373F19 - Nisarg Shah & Karan Singh 89

• Theorem: The local search with 𝑒 = 1 gives a Τ

2 3

approximation to Exact Max-2-SAT.

• Proof:

➢ 2 𝑋 𝑇0 = σ𝑘 𝑋 𝐵𝑘

• Every clause in 𝑇0 is counted twice on the RHS

➢ 𝑋 𝑇1 = σ𝑘 𝑋 𝐶

𝑘

• Every clause in 𝑇1 is only counted once on the RHS for the variable

whose literal was true under 𝜐

➢ For each 𝑘 : 𝑋 𝐵𝑘 ≤ 𝑋 𝐶

𝑘

• From local optimality of 𝜐, since otherwise flipping the truth value
• f variable 𝑘 would have increased the total weight

Exact Max-𝑙-SAT

373F19 - Nisarg Shah & Karan Singh 90

• Theorem: The local search with 𝑒 = 1 gives a Τ

2 3

approximation to Exact Max-2-SAT.

• Proof:

➢ 2 𝑋 𝑇0 ≤ 𝑋 𝑇1

• Summing the third equation on the last slide over all 𝑘, and then

using the first two equations on the last slide

➢ Hence:

• 3 𝑋 𝑇0 ≤ 𝑋 𝑇0 + 𝑋 𝑇1 ≤ 𝑋 𝑇0 + 𝑋 𝑇1 + 𝑋 𝑇2
• Precisely the condition we wanted to prove…

Exact Max-𝑙-SAT

373F19 - Nisarg Shah & Karan Singh 91

• Higher 𝑒?

➢ Searches over a larger neighborhood ➢ May get a better approximation ratio, but increases the

running time as we now need to check if any neighbor in a large neighborhood provides a better objective

➢ The bound is still 2/3 for 𝑒 = 𝑝(𝑜) ➢ It is no better than 4/5 for 𝑒 < Τ

𝑜 2

➢ It can be shown that with 𝑒 = Τ

𝑜 2, the algorithm always

terminates at an optimal solution

Exact Max-𝑙-SAT

373F19 - Nisarg Shah & Karan Singh 92

• Better approximation?

➢ We can learn something from our proof ➢ Note that we did not use anything about 𝑋 𝑇2 , and

simply added it at the end

➢ If we could also guarantee that 𝑋 𝑇0 ≤ 𝑋 𝑇2 …

• Then we would get 4 𝑋 𝑇0 ≤ 𝑋 𝑇0 + 𝑋 𝑇1 + 𝑋 𝑇2 , which

would give a Τ

3 4 approximation

➢ Result (without proof): This can be done by including just

• ne more assignment in the neighborhood: 𝑂 𝜐 =

𝑂1 𝜐 ∪ 𝜐𝑑 , where 𝜐𝑑 = complement of 𝜐

Exact Max-𝑙-SAT

373F19 - Nisarg Shah & Karan Singh 93

• What if we do not want to modify the

neighborhood?

➢ A slightly different tweak also works ➢ We want to weigh clauses in 𝑋(𝑇2) more because when

we get a clause through 𝑇2, we get more robustness (it can withstand changes in single variables)

• Modified local search:

➢ Start at arbitrary 𝜐 ➢ While there is an assignment in 𝑂1 𝜐 that improves the

potential 1.5 𝑋 𝑇1 + 2 𝑋(𝑇2)

• Switch to that assignment

Exact Max-𝑙-SAT

373F19 - Nisarg Shah & Karan Singh 94

• Modified local search:

➢ Start at arbitrary 𝜐 ➢ While there is an assignment in 𝑂1 𝜐 that improves the

potential 1.5 𝑋 𝑇1 + 2 𝑋(𝑇2)

• Switch to that assignment
• Note:

➢ This is the first time that we’re using a definition of

“better” in local search paradigm that does not quite align with the ultimate objective we want to maximize

➢ This is called “non-oblivious local search”

Exact Max-𝑙-SAT

373F19 - Nisarg Shah & Karan Singh 95

• Modified local search:

➢ Start at arbitrary 𝜐 ➢ While there is an assignment in 𝑂1 𝜐 that improves the

potential 1.5 𝑋 𝑇1 + 2 𝑋(𝑇2)

• Switch to that assignment
• Result (without proof):

➢ Modified local search gives Τ

3 4-approximation to Exact

Max-2-SAT

Exact Max-𝑙-SAT

373F19 - Nisarg Shah & Karan Singh 96

• More generally:

➢ The same technique works for higher values of 𝑙 ➢ Gives

2𝑙−1 2𝑙 approximation for Exact Max-𝑙-SAT

• We’ll see how to achieve the same approximation using a much

simpler technique

• Note: This is Τ

7 8 for Exact Max-3-SAT

➢ Theorem [Håstad]: Achieving Τ

7 8 + 𝜗 approximation

where 𝜗 > 0 is NP-hard.

• Uses PCP (probabilistically checkable proofs) technique