Convex Combinatorial Optimization ORSIS Prize 2005 Shmuel Onn and - - PowerPoint PPT Presentation

Convex Combinatorial Optimization

ORSIS Prize 2005

Shmuel Onn and Uri Rothblum

Technion – Israel Institute of Technology http://ie.technion.ac.il/~onn Supported in part by ISF – Israel Science Foundation

Linear Combinatorial Optimization (LCO)

6 3) 1 5 2 3 4

⊆ ∈

LCO: Given family F 2N of subsets of N = {1,…,n} and real weighting w:N , , find F F F of maximum weight w(F)= w(j) Example: Spanning Trees e.g. n=6, graph G=K4 gives the family F F = {{1,2,4}, {1,2,5}, … {3,5,6}, {4,5,6}}

∑

∈F j

Shmuel Onn

3

• 1

1 2

• 2 3)

Linear Combinatorial Optimization (LCO)

⊆ ∈

LCO: Given family F 2N of subsets of N = {1,…,n} and real weighting w:N , , find F F F of maximum weight w(F)= w(j) Example: Spanning Trees e.g. n=6, graph G=K4 gives the family F F = {{1,2,4}, {1,2,5}, … {3,5,6}, {4,5,6}} Now consider weighting w:{1,…,6}

∑

∈F j

Shmuel Onn

3

• 1

1 2

• 2 3)

Linear Combinatorial Optimization (LCO)

⊆ ∈

LCO: Given family F 2N of subsets of N = {1,…,n} and real weighting w:N , , find F F F of maximum weight w(F)= w(j) Example: Spanning Trees e.g. n=6, graph G=K4 gives the family F F = {{1,2,4}, {1,2,5}, … {3,5,6}, {4,5,6}} Now consider weighting w:{1,…,6}

.

The maximum weight tree is easily

• btained by the greedy algorithm

∑

∈F j

Shmuel Onn

Convex Combinatorial Optimization (CCO)

CCO: Given family F 2N , vectorial weighting w:N , , and convex functional c: , find F F F of maximum value c(w(F)).

d d

∈ ⊆

(3 -2) (-1 2) (1 0) (2 -1) (-2 3) (0 1) Example: Spanning Trees Consider again n=6, the graph G=K4 , d=2, weighting w and convex function c: defined by c(x) = |x| = x1 + x2

2 2 2 2

The objective value of the optimal tree F* is c(w(F*)) = c((-3 6)) = 9 + 36 = 45 where w(F*) = (0 1) + (-1 2) + (-2 3) = (-3 6)

Shmuel Onn

⊆ ∈

LCO: Given family F F 2N of subsets of N = {1,…,n} and real weighting w:N , , find F F F of maximum weight w(F)= w(j)

∑

∈F j

Some bad news, good news, and questions

Very broad expressive power Generally intractable even for d=1 For variable d, intractable even for F F =2N We provide broad polynomial time solvabe setup Approximation algorithms Classification of “edge-well-behaved” families F F Convex minimization

? ? ?

CCO: Given family F 2N , vectorial weighting w:N , , and convex functional c: , find F F F of maximum value c(w(F)).

d d

∈ ⊆

Shmuel Onn

Application 1: Matroids

Here F is the family of independent sets or bases of a matroid over N presented by a membership oracle. In particular, includes the spanning trees of before.

Corollary 1: CCO over matroids is polynomially solvable

(studied by Hassin and Tamir, Onn)

Shmuel Onn

Application 2: Positive Semidefinite Quadratic Assignment

Quadratic Assignment: Given nxn matrix M, find vector x {0,1}n maximizing xTMx

Corollary 2: PSD Quadratic Assignment is polynomially solvable

F = 2N is the family of all subsets Special Case (yet NP-hard): M positive semidefinite of rank d, so M=WTW with W given dxn matrix

Formulation as CCO:

w(j)=Wj is the j-th column of W c: is defined by c(x) = |x|2 = x1

2 + … + xd 2 d

∈ (studied by Allemand, Fukuda, Liebling and Steiner)

Shmuel Onn

Application 3: Partitioning Problems

Partition m items, each evaluated by k criteria, to p players, to maximize social utility which is a convex function

• f the sums of values of items each player gets

Convex functional on kxp matrices c:

kxp

The data for the problem consists of: Criteria-item table given as kxm matrix A Restrictions on the number of items each player gets

Shmuel Onn

Example and demonstration of the utility computation:

The convex functional on kxp matrices is c(X) = ∑ Xij

3

Consider m=6 items, k=2 criteria, p=3 players Each player gets 2 items The criteria -item matrix is:

items criteria

The social utility of π is c(Aπ) = 244432 The matrix of a partition such as π = (34, 56, 12) is:

players criteria Shmuel Onn

All 90 partitions π

• f items {1, …,6} To

3 players where each player gets 2 items

Shmuel Onn

All 90 corresponding 2x3 partition matrices Aπ on which the convex c is to be evaluated

Shmuel Onn

The optimal partition is among 30 special extremal partitions, which are directly enumerable by our CCO algorithm. π = (34, 56, 12)

players criteria

c(Aπ) = 244432

Shmuel Onn

Corollary 3: Partitioning problems are polynomially solvable

Formulation as CCO:

The convex functional c on kxp matrices is as given.

(studied by Aviran, Granot, Barnes, Hoffman, Hwang, Onn, Rothblum, and more)

Define n=mp, d=kp The ground set is N = { (i,j) : i=1, …, m, j=1, …, p } Each partition π = (π1, …, πp) is encoded as Fπ = { (i,j) : i πj };

∈

F 2N consists of all Fπ of partitions π obyeing restrictions

⊆

The weight function w:N is given by

kxp

≅

d

⊗

w(i,j) = Ai 1j = i=1, …, m, j=1, …, p

Ai

k p j

Shmuel Onn

Application 4: Minimum Variance Clustering

Special case of partitioning, hence CCO problem with n=mp, d=kp, weighting w of elements by kxp matrices, and convex function c assigning to each kxp matrix X its Euclidean norm c(X) = ∑ Xij

2

Formulation as CCO:

Corollary 4: Minimum variance clustering is polynomially solvable

(numerous applications in the analysis of statistical data and other areas)

Given m points v1, …, vm in k, group them into p (balanced) clusters so as to minimize the sum of cluster variances .

P=3, k=3, m large

Shmuel Onn

The Geometric Approach to Combinatorial Optimization

Denote by 1j the j-th unit vector in Rn.

⊆

∑

∈F j

For a subset F N denote by 1F = 1j its indicator. The family polytope of F F 2N is the 0-1 polytope PF

F = conv { 1F : F F

F }

⊆

∈ The The linear and convex combinatorial optimization linear and convex combinatorial optimization problems now ask for the problems now ask for the best vertex best vertex of P

• f PF

F Shmuel Onn

The GrÖtschel – Lovász – Schrijver Theory

Ellipsoid method (equivalence of separation and optimization) If PF

F is facet-well-behaved then linear combinatorial optimization (LCO)

• ver the family F

F is solvable in polynomial time. The proof makes use of:

Shmuel Onn

Our Work

If PF

F is edge-well-behaved then convex combinatorial optimization (CCO)

• ver the family F

F is solvable in strongly polynomial time.

Shmuel Onn

Our Work

• Zonotope construction

Equivalence of augmentation and optimization (via scaling and diophantine approximation) If PF

F is edge-well-behaved then convex combinatorial optimization (CCO)

• ver the family F

F is solvable in strongly polynomial time. The proof makes use of:

Shmuel Onn

The Main Theorem

Now, recall CCO: Given family F 2N , vectorial weighting w:N Rd, , and convex functional c:Rd R, find F F F of maximum value c(w(F)).

∈ ⊆

Theorem: Theorem: Fix

Fix d

• d. Then

. Then convex combinatorial optimization (CCO) is strongly polynomially time solvable over any edge-guaranteed family F F given by a membership oracle and convex c given by an evaluation oracle. A family F F is edge-guaranteed if it comes with a set of vectors E = {e1, …, em} Rn containing every edge direction of the polytope PF

F.

⊆

Typically, have edge-well-behaved classes C = (F Fn) of families having “uniform” edge-direction sets En = {e1, …, em(n)} with m(n) polynomial in n.

Shmuel Onn

Back to the Applications

• 1. Matroids over n elements: edge-well-behaved with

with En = { 1i - 1j : 1 ≤ i <j ≤ n } so m(n) = O(n2)

Corollary: All are strongly polynomially time solvable

• 2. PSD quadratic assignment over nxn matrices: edge-well-behaved

with with En = { 1i : 1 ≤ i ≤ n } so m(n) = n

• 3. Partitioning n items to p players: edge-well-behaved with

with En the set of all nxp circuit matrices so m(n) = O(np)

• 4. Minimum variance clustering: same as partitioning, m(n) = O(np)

Shmuel Onn

Proof ingredient 1: zonotope refinement and construction

Lemma 1: If E = {e1, …, em} contains all edge directions of a polytope P then the zonotope Z = [-1, 1] e1 + … + [-1, 1] em is a refinement of P. Lemma 2: In Rd, the zonotope Z can be constructed from E = {e1, …, em} along with a vector ai in the cone of every vertex in O(md-1) operations.

(Edelsbrunner, Gritzmann, Orourk, Seidel, Sharir, Sturmfels, …)

E E e e1

1

e e2

2

e e3

3

a a5

5

a a4

4

a a6

6

a a2

2

a a3

3

a a1

1

Z

a a1

1

a a5

5

a a4

4

a a3

3

P

a a2

2

a a6

6 Shmuel Onn

Consider LCO with weight w over family F F edge-guaranteed by E

Proof ingredient 2: membership augmentation optimization

Lemma 3: Membership Augmentation Proof: F in F F can be improved if and only if there is an edge direction e in E such that w● e > 0 and 1F + e = 1G for some G in F F. Lemma 4: Augmentation (linear) Optimization Proof: Schulz-Weismantel-Ziegler and GrÖtschel–Lovász using scaling ideas going back to Edmonds-Karp. Lemma 5: Polynomial time LCO Strongly polynomial time LCO Proof: Frank-Tárdos show that using Diophantine approximation can replace w by w’ of bit size depending polynomially only on n.

Shmuel Onn

Proof - combining all ingredients: the CCO algorithm

Input: Family F F given by membership oracle, edge-guaranteed by E in Rn, dxn weight matrix w, convex functional c on Rd given by evaluation oracle

Rn Rd

w

a a4

4

a a3

3

a a5

5

a a1

1

a a6

6

Z a a2

2

PF

F

b bi

i=wT●a

ai

i

1Fi

• 1. Construct the zonotope Z generated by the

projection w●E, and find ai in each normal cone

• 2. Lift each ai in Rd to bi = wT● ai in Rn and solve

LCO with weight bi over F F using membership oracle

• 3. Obtain the vertex 1Fi of PF

F

and the vertex w●1Fi of w●PF

F

• 4. Output an Fi

attaining maximum value c(w●1Fi) using evaluation oracle w●1Fi w● PF

F

a ai

i Shmuel Onn

Conclusion Conclusion

Approximation algorithms for classes of families Classification of edge-well-behaved classes of families Convex minimization: our results enable polynomial time

construction of inequality presentation of w●PF

F providing first step

Convex combinatorial optimization is a broadly expressive and useful framework It is strongly polynomial time solvable

• ver any edge-guaranteed family F

F

Some Broad Research Directions Some Broad Research Directions

Shmuel Onn