Al Algo gori rithms thms fo for r in inst stan ance ce-st - - PowerPoint PPT Presentation

Al Algo gori rithms thms fo for r in inst stan ance ce-st stable able an and pe d pert rtur urbat bation ion-res resili ilient ent pr prob

• blems

ems

Haris Angelidakis, TTIC Konstantin Makarychev, Northwestern Yury Makarychev, TTIC Aravindan Vijayaraghavan, Northwestern

QTW on Beyond Worst-Case Analysis Northwestern, May 24, 2017

Motivation

• Practice: Need to solve clustering and combinatorial
• ptimization problems.
• Theory:
• Many problems are NP-hard. Cannot solve them

exactly.

• Design approximation algorithms for worst case.

Can we get better algorithms for real-world instances than for worst-case instances?

Motivation

• Answer: Yes!

When we solve problems that arise in practice, we often get much better approximation than it is theoretically possible for worst case instances.

• Want to design algorithms with provable

performance guarantees for solving real-world instances.

Motivation

• Need a model for real-world instances.
• Many different models have been proposed.
• It’s unrealistic that one model will capture all

instances that arise in different applications.

This work

• Assumption: instances are stable/perturbation-

resilient

• Consider several problems:
• 𝑙-means
• 𝑙-median
• Max Cut
• Multiway Cut
• Get exact polynomial-time algorithms

𝑙-means and 𝑙-median

Given a set of points 𝑌, distance 𝑒(⋅,⋅) on 𝑌, and 𝑙 Partition 𝑌 into 𝑙 clusters 𝐷1, … , 𝐷𝑙 and find a “center” 𝑑𝑗 in each 𝐷𝑗 so as to minimize ෍

𝑗=1 𝑙

෍

𝑣∈𝐷𝑗

𝑒(𝑣, 𝑑𝑗) ෍

𝑗=1 𝑙

෍

𝑣∈𝐷𝑗

𝑒 𝑣, 𝑑𝑗 2 (𝑙-means) (𝑙-median)

Multiway Cut

Given

• a graph 𝐻 = (𝑊, 𝐹, 𝑥)
• a set of terminals 𝑢1, … , 𝑢𝑙

Find a partition of 𝑊 into sets 𝑇1, … , 𝑇𝑙 that minimizes the weight of cut edges s.t. 𝑢𝑗 ∈ 𝑇𝑗.

𝑢1 𝑢2 𝑢3 𝑢4

Instance-stability & perturbation- resilience

➢ Consider an instance ℐ of an optimization or clustering problem. ➢ ℐ′ is a 𝛿-perturbation of ℐ if it can be obtained from ℐ by “perturbing the parameters” — multiplying each parameter by a number from 1 to 𝛿.

• 𝑥 𝑓 ≤ 𝑥′ 𝑓 ≤ 𝛿 ⋅ 𝑥 𝑓
• 𝑒(𝑣, 𝑤) ≤ 𝑒′ 𝑣, 𝑤 ≤ 𝛿 ⋅ 𝑒(𝑣, 𝑤)

Instance-stability & perturbation- resilience

An instance ℐ of an optimization or clustering problem is perturbation-resilient/instance-stable if the optimal solution remains the same when we perturb the instance: every γ-perturbation ℐ′ has the same optimal solution as ℐ

Instance-stability & perturbation- resilience

Every γ-perturbation ℐ′ has the same optimal solution as ℐ

• In practice, we are interested in solving instances

where the optimal solution “stands out” among all solutions [Bilu, Linial]

• Objective function is an approximation to the “true”
• bjective function.
• “Practically interesting instance” ⇒ it is stable

Results

History

Instance-stability & perturbation-resilience was introduced

in discrete optimization: by Bilu and Linial `10 in clustering: by Awasthi, Blum, and Sheffet `12

Results (clustering)

𝛿 ≥ 3

𝒍-center, 𝒍-means, 𝒍-median

[Awasthi, Blum, Sheffet `12]

𝛿 ≥ 1 + 2

𝒍-center, 𝒍-means, 𝒍-median

[Balcan, Liang `13]

𝛿 ≥ 2

• sym. /asym.

𝒍-center

[Balcan, Haghtalab, White `16]

𝛿 ≥ 2

𝒍-means, 𝒍-median

[AMM `17]

Results (optimization)

𝛿 ≥ 𝑑𝑜

Max Cut

[Bilu, Linial `10]

𝛿 ≥ 𝑑 𝑜

Max Cut

[Bilu, Daniely, Linial, Saks `13]

𝛿 ≥ 𝑑 log 𝑜 log log 𝑜

Max Cut

[MMV `13]

𝛿 ≥ 4

Multiway

[MMV `13]

𝛿 ≥ 2 − 2/𝑙

Multiway

[AMM `17]

Results (optimization)

Our algorithms are robust.

• Find the optimal solution, if the instance is stable.
• Find an optimal solution or detects that the instance is

not stable, otherwise.

• Never output an incorrect answer.

Solve weakly stable instances.

Assume that when we perturb the instance

• the optimal solution changes only slightly, or
• there is a core that changes only slightly.

[Balcan, Haghtalab, White `16] No polynomial-time algorithm for (2 − 𝜁)-perturbation-resilient instances

• f 𝑙-center (𝑂𝑄 ≠ 𝑆𝑄).

[Ben-David, Reyzin `14] No polynomial-time algorithm for instances of 𝑙-means, 𝑙-median, 𝑙-center satisfying (2 − 𝜁)-center proximity property (𝑄 ≠ 𝑂𝑄).

Hardness results for center-based

• bejctives

Hardness results for optimization problems

Set Cover, Vertex Cover, Min 2-Horn Deletion

There is no robust algorithm for 𝑃(𝑜1−𝜁)-stable instances unless P = NP [AMM `17].

Provide evidence that [MMV `13, AMM `17]

• No robust algorithm for Max Cut when

𝛿 < 𝑃 log 𝑜 log log 𝑜

• Multiway cut is hard when 𝛿 < 4

3 − 𝑃 1 𝑙 .

Algorithm for Clustering Problems

Center proximity property

[Awasthi, Blum, Sheffet `12] A clustering 𝐷1, …, 𝐷𝑙 with centers 𝑑1, …, 𝑑𝑙 satisfies the center proximity property if for every 𝑞 ∈ 𝐷𝑗: 𝑒 𝑞, 𝑑

𝑘

> 𝛿 𝑒 𝑞, 𝑑𝑗 𝑑𝑗 𝐷

𝑘

𝐷𝑗 𝑑

𝑘

𝑞

i. 𝛿-perturbation resilience ⇒ 𝛿-center proximity ii. 2-center proximity ⇒ each cluster is a subtree of the MST

• iii. use single-linkage + DP to find 𝐷1, … , 𝐷𝑙

Plan [AMM `17]

Perturbation resilience: the optimal clustering doesn’t change when we perturb the distances. 𝑒 𝑣, 𝑤 /𝛿 ≤ 𝑒′ 𝑣, 𝑤 ≤ 𝑒(𝑣, 𝑤) [ABS `12] 𝑒′(⋅,⋅) doesn’t have to be a metric [AMM `17] 𝑒′(⋅,⋅) is a metric

Metric perturbation resilience is a more natural notion.

Perturbation resilience ⇒ center proximity

Assume center proximity doesn’t hold. Then 𝑒 𝑞, 𝑑

𝑘 ≤ 𝛿 𝑒 𝑞, 𝑑𝑗 .

Perturbation resilience ⇒ center proximity [ABS `12, AMM `17]

𝑑𝑗 𝐷

𝑘

𝐷𝑗 𝑑

𝑘

𝑞

Assume center proximity doesn’t hold.

• Let 𝑒′ 𝑞, 𝑑

𝑘 = 𝑒 𝑞, 𝑑𝑗 ≥ 𝛿−1𝑒(𝑞, 𝑑 𝑘).

• Don’t change other distances.
• Consider the shortest-path closure.

Perturbation resilience ⇒ center proximity [ABS `12, AMM `17]

𝑑𝑗 𝐷𝑗 𝑞 𝐷

𝑘

𝑑

𝑘

Assume center proximity doesn’t hold.

• Let 𝑒′ 𝑞, 𝑑

𝑘 = 𝑒 𝑞, 𝑑𝑗 ≥ 𝛿−1𝑒(𝑞, 𝑑 𝑘).

• Don’t change other distances.
• Consider the shortest-path closure.

Perturbation resilience ⇒ center proximity [ABS `12, AMM `17]

𝑑𝑗 𝐷𝑗 𝑞 𝐷

𝑘

𝑑

𝑘

Assume center proximity doesn’t hold.

• Let 𝑒′ 𝑞, 𝑑

𝑘 = 𝑒 𝑞, 𝑑𝑗 ≥ 𝛿−1𝑒(𝑞, 𝑑 𝑘).

• Don’t change other distances.
• Consider the shortest-path closure.

Perturbation resilience ⇒ center proximity [ABS `12, AMM `17]

𝑑𝑗 𝐷𝑗 𝑞 𝐷

𝑘

𝑑

𝑘

This is a 𝛿-perturbation.

Distances inside clusters 𝑫𝒋 and 𝑫𝒌 don’t change.

Consider 𝑣, 𝑤 ∈ 𝐷𝑗. 𝑒′ 𝑣, 𝑤 = min 𝑒 𝑣, 𝑤 , 𝑒 𝑣, 𝑞 + 𝑒′ 𝑞, 𝑑

𝑘 + 𝑒 𝑑 𝑘, 𝑤

Perturbation resilience ⇒ center proximity [ABS `12, AMM `17]

𝑑𝑗 𝐷𝑗 𝑞 𝐷

𝑘

𝑑

𝑘

𝑣 𝑤

Distances inside clusters 𝑫𝒋 and 𝑫𝒌 don’t change.

Consider 𝑣, 𝑤 ∈ 𝐷𝑗. 𝑒′ 𝑣, 𝑤 = min 𝑒 𝑣, 𝑤 , 𝑒 𝑣, 𝑞 + 𝑒′ 𝑞, 𝑑

𝑘 + 𝑒 𝑑 𝑘, 𝑤

Perturbation resilience ⇒ center proximity [ABS `12, AMM `17]

𝑑𝑗 𝐷𝑗 𝑞 𝐷

𝑘

𝑑

𝑘

𝑣 𝑤

Since the instance is 𝛿-stable, 𝐷1, … , 𝐷𝑙 must be the unique

• ptimal solution for distance 𝑒′.

Still, 𝑑𝑗 and 𝑑

𝑘 are optimal centers for 𝐷𝑗 and 𝐷 𝑘.

𝑒′ 𝑞, 𝑑𝑗 = 𝑒′ 𝑞, 𝑑

𝑘 ⇒ can move 𝑞 from 𝐷𝑗 to 𝐷 𝑘

Perturbation resilience ⇒ center proximity [ABS `12, AMM `17]

𝑑𝑗 𝐷𝑗 𝑞 𝐷

𝑘

𝑑

𝑘

[ABS `12] 2-center proximity ⇒ every 𝑣 ∈ 𝐷𝑗 is closer to 𝑑𝑗 than to any 𝑤 ∉ 𝐷𝑗 Assume the path from 𝑣 ∈ 𝐷𝑗 to 𝑑𝑗 in MST, leaves 𝐷𝑗.

Each cluster is a subtree of MST

𝑣 𝑑𝑗 𝑤

[ABS `12] 2-center proximity ⇒ every 𝑣 ∈ 𝐷𝑗 is closer to 𝑑𝑗 than to any 𝑤 ∉ 𝐷𝑗 Assume the path from 𝑣 ∈ 𝐷𝑗 to 𝑑𝑗 in MST, leaves 𝐷𝑗.

Each cluster is a subtree of MST

𝑣 𝑑𝑗 𝑤

Root MST at some 𝑠. 𝑈 𝑣 is the subtree rooted at 𝑣. cost𝑣(𝑘, 𝑑): the cost of the partitioning of 𝑈 𝑣

• into 𝑘 clusters (subtrees)
• so that 𝑑 is the center of the cluster containing 𝑣.

Dynamic programming algorithm

𝑣 𝑠 𝑈(𝑣) 𝑑

Fill out the DP table bottom-up. Example: 𝑙-median, 𝑣 has 2 children 𝑣1 and 𝑣2.

Dynamic programming algorithm

𝑣 𝑈(𝑣) 𝑣2 𝑣1

Fill out the DP table bottom-up. Example: 𝑙-median, 𝑣 has 2 children 𝑣1 and 𝑣2.

Dynamic programming algorithm

𝑣 𝑈(𝑣)

Fill out the DP table bottom-up. Example: 𝑙-median, 𝑣 has 2 children 𝑣1 and 𝑣2.

Dynamic programming algorithm

𝑣 𝑈(𝑣)

Fill out the DP table bottom-up. Example: 𝑙-median, 𝑣 has 2 children 𝑣1 and 𝑣2.

Dynamic programming algorithm

𝑣 𝑈(𝑣)

𝑣, 𝑣1, 𝑣2 lie in the same cluster cost𝑣 𝑘, 𝑑 = 𝑒 𝑣, 𝑑 + cost𝑣1 𝑘1, 𝑑 + cost𝑣2 𝑘2, 𝑑 where 𝑘1 + 𝑘2 = 𝑘 + 1 𝑣, 𝑣1, 𝑣2 lie in different clusters cost𝑣 𝑘, 𝑑 = 𝑒 𝑣, 𝑑 + cost𝑣1 𝑘1, 𝑑1 + cost𝑣2 𝑘2, 𝑑2 where 𝑘1 + 𝑘2 = 𝑘 − 1, 𝑑1 ∈ 𝑈 𝑣1 , 𝑑2 ∈ 𝑈 𝑣2 𝑣, 𝑣1 lie in the same clusters, 𝑣2 in a different

cost𝑣 𝑘, 𝑑 = 𝑒 𝑣, 𝑑 + cost𝑣1 𝑘1, 𝑑 + cost𝑣2 𝑘1, 𝑑2

where 𝑘1 + 𝑘2 = 𝑘, 𝑑2 ∈ 𝑈 𝑣2

Dynamic programming algorithm

Algorithms for Max Cut and Multiway Cut

Algorithms for Max Cut and Multiway Cut [MMV `13]

Write an SDP or LP relaxation for the problem. Show that it is integral if the instance is 𝛿-stable.

solve the relaxation if the SDP/LP solution is integral return the solution else return that the instance is not 𝛿-stable

The algorithm is robust: it never returns an incorrect answer.

Multiway Cut

Write the relaxation for Multiway Cut by

Călinescu, Karloff, and Rabani [CKR `98]

To get an 𝛽-approximation, we would design a rounding scheme with

Pr 𝑣, 𝑤 is cut ≤ 𝛽 𝑒 𝑣, 𝑤

Then

𝔽 weight of cut edges ≤ 𝛽 ෍

𝑣,𝑤 ∈𝐹

𝑥𝑣𝑤𝑒(𝑣, 𝑤)

Multiway Cut: complementary objective

If we want to maximize the weight of uncut edges, we would we would design a rounding scheme with

Pr 𝑣, 𝑤 is not cut ≥ 𝛾 (1 − 𝑒 𝑣, 𝑤 )

Then

𝔽 wt. of uncut edges ≥ 𝛾 ෍

𝑣,𝑤 ∈𝐹

𝑥𝑣𝑤(1 − 𝑒 𝑣, 𝑤 )

Write an LP or SDP relaxation for the problem. Design a rounding procedure s.t.

Pr 𝑣, 𝑤 is cut ≤ 𝛽 𝑒 𝑣, 𝑤 Pr 𝑣, 𝑤 is not cut ≥ 𝛾 1 − 𝑒 𝑣, 𝑤

• r

Pr 𝑣, 𝑤 is cut ≥ 𝛾 𝑒 𝑣, 𝑤 Pr 𝑣, 𝑤 is not cut ≤ 𝛽 1 − 𝑒 𝑣, 𝑤

Then the relaxation for 𝛿-stable instances is integral, when 𝛿 ≥ 𝛽/𝛾

General approach to solving stable instances of graph partitioning

minimization maximization

!

Solving Max Cut [MMV `13]

Use the Goemans–Williamson SDP relaxation with ℓ2

2-triangle inequalities.

Design a rounding procedure with 𝛽 𝛾 = 𝑃 log 𝑜 log log 𝑜 , which is a combination of two algorithms:

• the algorithm for Sparsest Cut with Nonuniform Demands

by Arora, Lee, and Naor `08,

• the algorithm for Min Uncut by Agarwal, Charikar,

Makarychev, M `05

Solving Multiway Cut [AMM `17]

Rounding procedures for Multiway Cut by

• Sharma and Vondrák `14
• Buchbinder, Schwartz, and Weizman `17

are highly non-trivial. Show: need a rounding procedure only for LP solutions that are almost integral. Design a simple rounding procedure with

𝛽 𝛾 = 2 − 2 𝑙 .

Summary

• Algorithms for 2-perturbation-resilient instances of

problems with a natural center based objective: 𝑙-means, 𝑙-median, facility location

• Robust algorithms for 𝑃

log 𝑜 log log 𝑜 -stable instance of Max Cut and 2 −

2 𝑙 -stable instances of

Multiway Cut.

• Negative results for stable instances of Max Cut,

Multiway Cut, Max 𝑙-Cut, Multi Cut, Set Cover, Vertex Cover, Min 2-Horn Deletion.