Algorithms with provable guarantees for clustering problems Ola - - PowerPoint PPT Presentation

Algorithms with provable guarantees for clustering problems

Ola Svensson

Where to place rescue centers?

Build k centers so as to minimize sum of travel distances

Where to place rescue centers?

Build k centers so as to minimize sum of travel distances

• ptimize some objective

Median and Center

MEDIAN: Open point/facility on real line so as to minimize sum of distances from clients ( )

Median and Center

MEDIAN: Open point/facility on real line so as to minimize sum of distances from clients ( )

Median and Center

MEDIAN: Open point/facility on real line so as to minimize sum of distances from clients ( )

Median and Center

decrease distance for 3 clients increase distance for 6 clients

MEDIAN: Open point/facility on real line so as to minimize sum of distances from clients ( )

Median and Center

decrease distance for 3 clients increase distance for 6 clients decrease distance for 6 clients increase distance for 3 clients

MEDIAN: Open point/facility on real line so as to minimize sum of distances from clients ( )

Median and Center

MEDIAN: Open point/facility on real line so as to minimize sum of distances from clients ( )

Median and Center

CENTER: Open point/facility on real line so as to minimize max distance

• ver all clients ( )

MEDIAN: Open point/facility on real line so as to minimize sum of distances from clients ( )

Median and Center

CENTER: Open point/facility on real line so as to minimize max distance

• ver all clients ( )

MEDIAN: Open point/facility on real line so as to minimize sum of distances from clients ( )

Median and Center

CENTER: Open point/facility on real line so as to minimize max distance

• ver all clients ( )

x x

MEDIAN: Open point/facility on real line so as to minimize sum of distances from clients ( )

K-Median and K-Center

K-MEDIAN: Open k points/facilities in a metric space so as to minimize sum of distances from clients ( )

K-Median and K-Center

K-MEDIAN: Open k points/facilities in a metric space so as to minimize sum of distances from clients ( )

K-Median and K-Center

K-MEDIAN: Open k points/facilities in a metric space so as to minimize sum of distances from clients ( )

K-Median and K-Center

K-CENTER: Open k points/facilities in a metric space so as to minimize max distance over all clients ( ) K-MEDIAN: Open k points/facilities in a metric space so as to minimize sum of distances from clients ( )

K-Median and K-Center

K-CENTER: Open k points/facilities in a metric space so as to minimize max distance over all clients ( ) K-MEDIAN: Open k points/facilities in a metric space so as to minimize sum of distances from clients ( )

Mathematical formulation of objective functions

Mathematical formulation of objective functions

General Problem parameterized by 𝒒 ≥ 𝟐: Find a set 𝑻 of k points/facilities in a metric space so as to minimize

𝒌 𝒅𝒎𝒋𝒇𝒐𝒖

𝒆 𝒌, 𝑻 𝒒

𝟐/𝒒

Mathematical formulation of objective functions

General Problem parameterized by 𝒒 ≥ 𝟐: Find a set 𝑻 of k points/facilities in a metric space so as to minimize

𝒌 𝒅𝒎𝒋𝒇𝒐𝒖

𝒆 𝒌, 𝑻 𝒒

𝟐/𝒒

Distance from client j to closest facility in S

K-MEDIAN: 𝒒 = 𝟐 K-CENTER: 𝒒 = ∞ K-MEANS: 𝒒 = 𝟑 Actually, 𝑘 𝑑𝑚𝑗𝑓𝑜𝑢 𝑒 𝑘, 𝑇 2 and Euclidean metric

Facility Location

Facility Location: Open facilities in a metric space so as to minimize sum of distances from clients + opening costs

ALL THESE PROBLEMS ARE INTRACTABLE (NP-HARD) IN THE WORST CASE

Solving intractable problems

• Heuristics
• good for “typical” instances
• bad instances do not happen too often

1 4 16 64 256 1024 4096 16384 50's 70's 80's 90's 00's

Dantzig, Fulkerson, and Johnson solve a 49- city instance to optimality Applegate, Bixby, Chvatal, Cook, and Helsgaun solve a 24978-city instance

!

Sweden has only 9 million inhabitants ≈ 360 persons/city

Solving intractable problems

• Approximation Algorithms
• Perhaps we can efficiently find a reasonably good solution?

Approximation Ratio: worst case over all instances

• α=1 is an exact polynomial time algorithm
• α=1.01 then algorithm finds a solution with at most 1% higher cost

GOAL: Complete understanding of worst case behavior

State of the Art

Approximation Hardness Facility Location 1.488

[Li’11]

1.463

[Guha & Khuller’98]

K-Center 2

[Gonzales’85, Hochbaum & Shmoys’85]

2

[Hsu & Nemhauser’79]

K-Median 2.67

[Byrka et al’15]

1+2/e

[Jain et al.’02]

K-Means 9

[Kanungo et al’2004]

1.0013

[Lee. Schmidt, Wright’15]

Even better: Approximation algorithms (can be) achieved by standard LP relaxations and techniques transfer between problems

A 2-APPROXIMATION ALGORITHM FOR K-CENTER

Greedy K-Center

Open any point For 𝑗 = 2, … , 𝑙 Open point farthest away from already

• pened points

Greedy K-Center

Open any point For 𝑗 = 2, … , 𝑙 Open point farthest away from already

• pened points

Greedy K-Center

Open any point For 𝑗 = 2, … , 𝑙 Open point farthest away from already

• pened points

Greedy K-Center

Open any point For 𝑗 = 2, … , 𝑙 Open point farthest away from already

• pened points

Analysis

Open any point For 𝑗 = 2, … , 𝑙 Open point farthest away from already

• pened points

Consider optimal solution and corresponding Voronoi diagram

Analysis

Open any point For 𝑗 = 2, … , 𝑙 Open point farthest away from already

• pened points

Case 1: We opened up one point in each cell

Analysis

Open any point For 𝑗 = 2, … , 𝑙 Open point farthest away from already

• pened points

Case 1: We opened up one point in each cell

Analysis

Open any point For 𝑗 = 2, … , 𝑙 Open point farthest away from already

• pened points

Case 1: We opened up one point in each cell

≤ 𝑃𝑄𝑈 ≤ 𝑃𝑄𝑈

Analysis

Open any point For 𝑗 = 2, … , 𝑙 Open point farthest away from already

• pened points

Case 1: We opened up one point in each cell

≤ 𝑃𝑄𝑈 ≤ 𝑃𝑄𝑈 ≤ 2 ⋅ 𝑃𝑄𝑈

In this case any client is connected within distance ≤ 𝟑 ⋅ 𝑷𝑸𝑼

Analysis

Open any point For 𝑗 = 2, … , 𝑙 Open point farthest away from already

• pened points

Case 1I: We did not open up one point in each cell

Analysis

Open any point For 𝑗 = 2, … , 𝑙 Open point farthest away from already

• pened points

Case 1I: We opened up two points in a single cell

Analysis

Open any point For 𝑗 = 2, … , 𝑙 Open point farthest away from already

• pened points

Case 1I: We opened up two points in a single cell

≤ 𝑃𝑄𝑈 ≤ 𝑃𝑄𝑈

Analysis

Open any point For 𝑗 = 2, … , 𝑙 Open point farthest away from already

• pened points

Case 1I: We opened up two points in a single cell

≤ 𝑃𝑄𝑈 ≤ 2 ⋅ 𝑃𝑄𝑈

Also in this case any client is connected within distance ≤ 𝟑 ⋅ 𝑷𝑸𝑼

≤ 𝑃𝑄𝑈

Open any point For 𝑗 = 2, … , 𝑙 Open point farthest away from already

• pened points

THEOREM:

The above greedy algorithm is a 2-approximation for k-Center

Gonzales, Hochbaum & Shmoys’85

ALGORITHMS FOR FACILITY LOCATION AND K-MEDIAN

LINEAR PROGRAMMING RELAXATION

LINEAR PROGRAM:

• yi takes value 1 if i is opened and 0 otherwise
• xij takes value 1 if j is connected to i and 0 otherwise

LP Relaxation for Facility Location

LINEAR PROGRAM:

• yi takes value 1 if i is opened and 0 otherwise
• xij takes value 1 if j is connected to i and 0 otherwise
• pening cost

connection cost

LP Relaxation for Facility Location

minimize 𝑗∈𝐺 𝑔

𝑗𝑧𝑗 + 𝑗∈𝐺,𝑘∈𝐷 𝑒𝑗𝑘𝑦𝑗𝑘

subject to

𝑗∈𝐺 𝑦𝑗𝑘 = 1 𝑘 ∈ 𝐷 𝑦𝑗𝑘 ≤ 𝑧𝑗 i ∈ 𝐺, 𝑘 ∈ 𝐷 𝑦𝑗𝑘, 𝑧𝑗 ∈ [0,1] i ∈ 𝐺, 𝑘 ∈ 𝐷

LINEAR PROGRAM:

• yi takes value 1 if i is opened and 0 otherwise
• xij takes value 1 if j is connected to i and 0 otherwise

LP Relaxation for Facility Location

minimize 𝑗∈𝐺 𝑔

𝑗𝑧𝑗 + 𝑗∈𝐺,𝑘∈𝐷 𝑒𝑗𝑘𝑦𝑗𝑘

subject to

𝑗∈𝐺 𝑦𝑗𝑘 = 1 𝑘 ∈ 𝐷 𝑦𝑗𝑘 ≤ 𝑧𝑗 i ∈ 𝐺, 𝑘 ∈ 𝐷 𝑦𝑗𝑘, 𝑧𝑗 ∈ [0,1] i ∈ 𝐺, 𝑘 ∈ 𝐷

Every client is connected

LINEAR PROGRAM:

• yi takes value 1 if i is opened and 0 otherwise
• xij takes value 1 if j is connected to i and 0 otherwise

LP Relaxation for Facility Location

minimize 𝑗∈𝐺 𝑔

𝑗𝑧𝑗 + 𝑗∈𝐺,𝑘∈𝐷 𝑒𝑗𝑘𝑦𝑗𝑘

subject to

𝑗∈𝐺 𝑦𝑗𝑘 = 1 𝑘 ∈ 𝐷 𝑦𝑗𝑘 ≤ 𝑧𝑗 i ∈ 𝐺, 𝑘 ∈ 𝐷 𝑦𝑗𝑘, 𝑧𝑗 ∈ [0,1] i ∈ 𝐺, 𝑘 ∈ 𝐷

Clients connected to open facilities

LINEAR PROGRAM:

• yi takes value 1 if i is opened and 0 otherwise
• xij takes value 1 if j is connected to i and 0 otherwise

LP Relaxation for Facility Location

minimize 𝑗∈𝐺 𝑔

𝑗𝑧𝑗 + 𝑗∈𝐺,𝑘∈𝐷 𝑒𝑗𝑘𝑦𝑗𝑘

subject to

𝑗∈𝐺 𝑦𝑗𝑘 = 1 𝑘 ∈ 𝐷 𝑦𝑗𝑘 ≤ 𝑧𝑗 i ∈ 𝐺, 𝑘 ∈ 𝐷 𝑦𝑗𝑘, 𝑧𝑗 ∈ [0,1] i ∈ 𝐺, 𝑘 ∈ 𝐷

ALGORITHMS USING RELAXATION

Randomized Rounding

Interpret yi as the probability that facility i is opened

Randomized Rounding

Interpret yi as the probability that facility i is opened

Open each facility i with probability yi Connect client to closest opened facility

Randomized Rounding

Interpret yi as the probability that facility i is opened PROBLEM:

• With constant probability: a client has no facility opened close to it

Open each facility i with probability yi Connect client to closest opened facility

Dependent Rounding

Grow and select balls Open each facility i with probability yi subject to a facility is opened in each ball Connect client to closest opened facility

Dependent Rounding

Grow and select balls Open each facility i with probability yi subject to a facility is opened in each ball Connect client to closest opened facility

Dependent Rounding

Grow and select balls Open each facility i with probability yi subject to a facility is opened in each ball Connect client to closest opened facility

While possible select ball with smallest radius that is disjoint from selected balls

Dependent Rounding

Grow and select balls Open each facility i with probability yi subject to a facility is opened in each ball Connect client to closest opened facility

While possible select ball with smallest radius that is disjoint from selected balls => Every client has a “fall back” path of length 3 times it radius

Dependent Rounding

Grow and select balls Open each facility i with probability yi subject to a facility is opened in each ball Connect client to closest opened facility

While possible select ball with smallest radius that is disjoint from selected balls => Every client has a “fall back” path of length 3 times it radius

Dependent Rounding

Grow and select balls Open each facility i with probability yi subject to a facility is opened in each ball Connect client to closest opened facility

Dependent Rounding

Grow and select balls Open each facility i with probability yi subject to a facility is opened in each ball Connect client to closest opened facility

Dependent Rounding

Grow and select balls Open each facility i with probability yi subject to a facility is opened in each ball Connect client to closest opened facility

Dependent Rounding

Grow and select balls Open each facility i with probability yi subject to a facility is opened in each ball Connect client to closest opened facility

Dependent Rounding

Grow and select balls Open each facility i with probability yi subject to a facility is opened in each ball Connect client to closest opened facility

First constant approximation algorithm

THEOREM:

“dependent rounding” gives 3.16-approximation algorithm

Shmoys, Tardos, Aardal’97

Impressive progress based on same LP

THEOREM:

“dependent rounding” gives (1+2/e)-approximation algorithm

Chudak & Shmoys’99

THEOREM:

Primal-dual gives 3-approximation algorithm

Jain & Vazirani’01, Jain et al’03, Mahdian et al.’02

Impressive progress based on same LP

THEOREM:

“dependent rounding” gives (1+2/e)-approximation algorithm

Chudak & Shmoys’99

THEOREM:

Primal-dual gives 1.6-approximation algorithm

Jain & Vazirani’01, Jain et al’03, Mahdian et al.’02

Impressive progress based on same LP

THEOREM:

“dependent rounding” gives (1+2/e)-approximation algorithm

Chudak & Shmoys’99

THEOREM:

Primal-dual gives 1.52-approximation algorithm

Jain & Vazirani’01, Jain et al’03, Mahdian et al.’02

Impressive progress based on same LP

THEOREM:

“dependent rounding” gives (1+2/e)-approximation algorithm

Chudak & Shmoys’99

THEOREM:

Primal-dual gives 1.52-approximation algorithm

Jain & Vazirani’01, Jain et al’03, Mahdian et al.’02

THEOREM:

“dependent rounding”+primal-dual gives 1.5-approximation algorithm

Byrka’07

Impressive progress based on same LP

THEOREM:

Primal-dual gives 1.52-approximation algorithm

Jain & Vazirani’01, Jain et al’03, Mahdian et al.’02

THEOREM:

“dependent rounding”+primal-dual gives 1.5-approximation algorithm

Byrka’07

THEOREM:

“dependent rounding”+primal-dual gives 1.488-approximation algorithm

Li’11

Impressive progress based on same LP

THEOREM:

“dependent rounding”+primal-dual gives 1.488-approximation algorithm

Li’11

ALMOST TIGHT: It is NP-hard to do better than 1.463 Guha and Kuller’99

Relation to k-Median

K-MEDIAN: same as facility location but hard constraint that at most k facilities are opened.

Relation to k-Median

K-MEDIAN: same as facility location but hard constraint that at most k facilities are opened. Relationship to facility location: Simple economy

• If the price of opening facilities is cheap, many facilities will be opened
• If the price of opening facilities is expensive, few facilities will be opened

Relation to k-Median

K-MEDIAN: same as facility location but hard constraint that at most k facilities are opened. Relationship to facility location: Simple economy

• If the price of opening facilities is cheap, many facilities will be opened
• If the price of opening facilities is expensive, few facilities will be opened

=> Find price so that ≈ k facilities are opened

Relation to k-Median

K-MEDIAN: same as facility location but hard constraint that at most k facilities are opened. Relationship to facility location: Simple economy

• If the price of opening facilities is cheap, many facilities will be opened
• If the price of opening facilities is expensive, few facilities will be opened

=> Find price so that ≈ k facilities are opened

First exploited by Jain & Vazirani’01 to give fast and elegant approximation algorithms for k-median based on algorithms for facility location

Relaxing hard constraint for k-Median

• Difficulty is the hard constraint that we can open at most k facilities

THEOREM:

An r-pseudo-approximation algorithm that opens k+c facilities can be turned into a r+ε-approximation algorithm that opens k facilities and runs in time nO(c/ε)

Li & S.’12 Together with an improved “pseudo-approximation” gives THEOREM:

There is a 2.73- approximation algorithm for k-Median

Li & S.’12

Relaxing hard constraint for k-Median

• Difficulty is the hard constraint that we can open at most k facilities

THEOREM:

An r-pseudo-approximation algorithm that opens k+c facilities can be turned into a r+ε-approximation algorithm that opens k facilities and runs in time nO(c/ε)

Li & S.’12 Together with an improved “pseudo-approximation” gives THEOREM:

There is a 2.73- approximation algorithm for k-Median

Li & S.’12 THEOREM:

There is a 2.67- approximation algorithm for k-Median

Byrka et al’15

State of the Art

Approximation Hardness Facility Location 1.488

[Li’11]

1.463

[Guha & Khuller’98]

K-Center 2

[Gonzales’85, Hochbaum & Shmoys’85]

2

[Hsu & Nemhauser’79]

K-Median 2.6

[Byrka et al’15]

1+2/e

[Jain et al.’02]

K-Means 9

[Kanungo et al.’04]

1.0013

[Lee. Schmidt, Wright’15]

Techniques developed transfers to the different problems

State of the Art

Approximation Hardness Facility Location 1.488

[Li’11]

1.463

[Guha & Khuller’98]

K-Center 2

[Gonzales’85, Hochbaum & Shmoys’85]

2

[Hsu & Nemhauser’79]

K-Median 2.6

[Byrka et al’15]

1+2/e

[Jain et al.’02]

K-Means 9

[Kanungo et al.’04]

1.0013

[Lee. Schmidt, Wright’15]

Techniques developed transfers to the different problems

What is his problem?

Facilities have Capacities

Facilities have Capacities

Each potential facility i has a capacity Ui that regulates how many clients facility can accept 3 3 3 3

Facilities have Capacities

Each potential facility i has a capacity Ui that regulates how many clients facility can accept 3 3 3 3

Facilities have Capacities

Each potential facility i has a capacity Ui that regulates how many clients facility can accept 3 3 3 3

State of the Art

Capacitated Approximation Hardness Facility Location 5

[Bansal, Garg, Gupta’12]

1.463

[Guha & Khuller’98]

K-Center 9

[An et al.’14]

3

[Cygan et al.’12]

K-Median

• 1+2/e

[Jain et al.’02]

K-Means

• 1.0013

[Lee, Schmidt, Wright’15]

No “uniform” approach

Standard LP has unbounded integrality gap

APPRECIATE THE DIFFICULTY

Special case of Capacitated Facility Location

Special case: all distances are 0

Special case: all distances are 0

INPUT: n clients, set of facilities with capacities and opening costs

Special case: all distances are 0

INPUT: n clients, set of facilities with capacities and opening costs GOAL: find a subset of facilities so that 1. Total capacity is at least n 2. Opening costs are minimized

Special case: all distances are 0

INPUT: n clients, set of facilities with capacities and opening costs GOAL: find a subset of facilities so that 1. Total capacity is at least n 2. Opening costs are minimized

Minimum Knapsack Problem

Standard LP has bad integrality gap Strengthened using knapsack-cover inequalities

Add a constraint for each subset of facilities “that we suppose to open”

Knapsack-Cover Inequalities (Wolsey’75)

… 20 clients

€2 ≤8 €0 ≤5 €1 ≤3 €10 ≤19 €0 ≤2

Knapsack-Cover Inequalities (Wolsey’75)

• Suppose a subset S of facilities was already included in the solution

… 20 clients

€2 ≤8 €0 ≤5 €1 ≤3 €10 ≤19 €0 ≤2

S

Knapsack-Cover Inequalities (Wolsey’75)

• Suppose a subset S of facilities was already included in the solution
• Among the remaining facilities must open capacity

… 20 clients

€2 ≤8 €0 ≤5 €1 ≤3 €10 ≤19 €0 ≤2

S

Knapsack-Cover Inequalities (Wolsey’75)

• Suppose a subset S of facilities was already included in the solution
• Among the remaining facilities must open capacity
• Strengthen since no need to have higher capacity than right-hand-side

… 20 clients

€2 ≤8 €0 ≤5 €1 ≤3 €10 ≤19 €0 ≤2

S

Knapsack-Cover Inequalities (Wolsey’75)

• Suppose a subset S of facilities was already included in the solution
• Among the remaining facilities must open capacity
• Strengthen since no need to have higher capacity than right-hand-side

… 20 clients

€2 ≤8 €0 ≤5 €1 ≤3 €10 ≤19 €0 ≤2

S

Non-Trivial to Generalize to Facility Location

• Several proposed inequalities
• Leung and Magnanti’89, Cornuejols, Sridharan, Thizy’91. Aardal’92, Aardal, Pochet and Wolsey’93, Deng and

Simchi-Levi’93

• Many recently proved insufficient Kolliopoulos & Moysoglou’13
• Sequence of local search algorithms that give 5-approximation algorithm
• Uniform capacities: Korupolu, Plaxton, Rajaraman’00, Chudak & Williamson’05, Aggarwal et al.’13
• General capacities: Pal, Tardos, Wexler’01, Bansal, Garg, Gupta’12

Recent progress

THEOREM:

A generalization of the knapsack cover inequalities yields a “good” LP- relaxation for capacitated facility location. Polynomial time rounding algorithm that finds a solution whose cost is no more than a constant times LP-OPT.

An, Singh, Svensson’14

Constant should be improved; not optimized constant is 288  No known large lower bound on the integrality gap Rich family of techniques to tap into to analyze the relaxation Are the techniques flexible enough to apply to related problems?

TIME TO SUMMARIZE

• Many interesting techniques developed by studying these problems
• Quite good understanding of uncapacitated problems
• Increased understanding of capacitated ones

Better algorithms for k-Median and Facility Location? More uniform treatment of capacitated problems?

• Integrality gap of relaxation for capacitated facility location?
• Is there a “good” compact relaxation?
• Constant factor for capacitated k-Median?

What about k-Means?