Advanced Algorithms (III) Chihao Zhang Shanghai Jiao Tong - - PowerPoint PPT Presentation

Advanced Algorithms (III)

Chihao Zhang

Shanghai Jiao Tong University

• Mar. 11, 2019

Advanced Algorithms (III) 1/11

MaxCut

MaxCut Input: An undirected graph G V E . Problem: A set S V that maximizes E S S . NP-hard Similar to Max2SAT, tossing a fair coin yields an

• approximation.

(Exercise) Can we find clever coins via LP relaxation…?

Advanced Algorithms (III) 2/11

MaxCut

MaxCut Input: An undirected graph G = (V, E). Problem: A set S ⊆ V that maximizes

• E(S, ¯

S)

• .

NP-hard Similar to Max2SAT, tossing a fair coin yields an

• approximation.

(Exercise) Can we find clever coins via LP relaxation…?

Advanced Algorithms (III) 2/11

MaxCut

MaxCut Input: An undirected graph G = (V, E). Problem: A set S ⊆ V that maximizes

• E(S, ¯

S)

• .

NP-hard Similar to Max2SAT, tossing a fair coin yields an

• approximation.

(Exercise) Can we find clever coins via LP relaxation…?

Advanced Algorithms (III) 2/11

MaxCut

MaxCut Input: An undirected graph G = (V, E). Problem: A set S ⊆ V that maximizes

• E(S, ¯

S)

• .

NP-hard Similar to Max2SAT, tossing a fair coin yields an 1

2-approximation.

(Exercise) Can we find clever coins via LP relaxation…?

Advanced Algorithms (III) 2/11

MaxCut

MaxCut Input: An undirected graph G = (V, E). Problem: A set S ⊆ V that maximizes

• E(S, ¯

S)

• .

NP-hard Similar to Max2SAT, tossing a fair coin yields an 1

2-approximation.

(Exercise) Can we find clever coins via LP relaxation…?

Advanced Algorithms (III) 2/11

LP for MaxCut

introduce a vairable xu for every u V. introduce a variable yu v for every edge e u v . The cost function is

e u v E yu v.

How to write linear constraints for a cut? idea: Let F u v E yu v , we view S S F as a bipartite subgraph of G.

Advanced Algorithms (III) 3/11

LP for MaxCut

▶ introduce a vairable xu ∈ {0, 1} for every u ∈ V. introduce a variable yu v for every edge e u v . The cost function is

e u v E yu v.

How to write linear constraints for a cut? idea: Let F u v E yu v , we view S S F as a bipartite subgraph of G.

Advanced Algorithms (III) 3/11

LP for MaxCut

▶ introduce a vairable xu ∈ {0, 1} for every u ∈ V. ▶ introduce a variable yu,v ∈ {0, 1} for every edge e = {u, v}. The cost function is

e u v E yu v.

How to write linear constraints for a cut? idea: Let F u v E yu v , we view S S F as a bipartite subgraph of G.

Advanced Algorithms (III) 3/11

LP for MaxCut

▶ introduce a vairable xu ∈ {0, 1} for every u ∈ V. ▶ introduce a variable yu,v ∈ {0, 1} for every edge e = {u, v}. The cost function is ∑

e={u,v}∈E yu,v.

How to write linear constraints for a cut? idea: Let F u v E yu v , we view S S F as a bipartite subgraph of G.

Advanced Algorithms (III) 3/11

LP for MaxCut

▶ introduce a vairable xu ∈ {0, 1} for every u ∈ V. ▶ introduce a variable yu,v ∈ {0, 1} for every edge e = {u, v}. The cost function is ∑

e={u,v}∈E yu,v.

How to write linear constraints for a cut? idea: Let F u v E yu v , we view S S F as a bipartite subgraph of G.

Advanced Algorithms (III) 3/11

LP for MaxCut

▶ introduce a vairable xu ∈ {0, 1} for every u ∈ V. ▶ introduce a variable yu,v ∈ {0, 1} for every edge e = {u, v}. The cost function is ∑

e={u,v}∈E yu,v.

How to write linear constraints for a cut? idea: Let F = { {u, v} ∈ E : yu,v = 1 } , we view (S, ¯ S, F) as a bipartite subgraph of G.

Advanced Algorithms (III) 3/11

▶ introduce yu,v and yv,u for every {u, v} ∈ (V

2

). introduce constaints for being bipartite graph metric. introduce constraints to rule out odd cycles. max

u v E

yu v s.t. yu v yu w yw v u v w V yu v yu w yw v u v w V yu v yv u u v V

Advanced Algorithms (III) 4/11

▶ introduce yu,v and yv,u for every {u, v} ∈ (V

2

). ▶ introduce constaints for being bipartite graph metric. introduce constraints to rule out odd cycles. max

u v E

yu v s.t. yu v yu w yw v u v w V yu v yu w yw v u v w V yu v yv u u v V

Advanced Algorithms (III) 4/11

▶ introduce yu,v and yv,u for every {u, v} ∈ (V

2

). ▶ introduce constaints for being bipartite graph metric. ▶ introduce constraints to rule out odd cycles. max

u v E

yu v s.t. yu v yu w yw v u v w V yu v yu w yw v u v w V yu v yv u u v V

Advanced Algorithms (III) 4/11

▶ introduce yu,v and yv,u for every {u, v} ∈ (V

2

). ▶ introduce constaints for being bipartite graph metric. ▶ introduce constraints to rule out odd cycles. max ∑

{u,v}∈E

yu,v s.t. yu v yu w yw v u v w V yu v yu w yw v u v w V yu v yv u u v V

Advanced Algorithms (III) 4/11

▶ introduce yu,v and yv,u for every {u, v} ∈ (V

2

). ▶ introduce constaints for being bipartite graph metric. ▶ introduce constraints to rule out odd cycles. max ∑

{u,v}∈E

yu,v s.t. yu,v ≤ yu,w + yw,v, ∀u, v, w ∈ V yu v yu w yw v u v w V yu v yv u u v V

Advanced Algorithms (III) 4/11

▶ introduce yu,v and yv,u for every {u, v} ∈ (V

2

). ▶ introduce constaints for being bipartite graph metric. ▶ introduce constraints to rule out odd cycles. max ∑

{u,v}∈E

yu,v s.t. yu,v ≤ yu,w + yw,v, ∀u, v, w ∈ V ∑

e={u,v}∈C

yu,v ≤ |C| − 1, ∀odd cycle C yu v yu w yw v u v w V yu v yv u u v V

Advanced Algorithms (III) 4/11

▶ introduce yu,v and yv,u for every {u, v} ∈ (V

2

). ▶ introduce constaints for being bipartite graph metric. ▶ introduce constraints to rule out odd cycles. max ∑

{u,v}∈E

yu,v s.t. yu,v ≤ yu,w + yw,v, ∀u, v, w ∈ V yu,v + yu,w + yw,v ≤ 2, ∀u, v, w, ∈ V yu v yv u u v V

Advanced Algorithms (III) 4/11

▶ introduce yu,v and yv,u for every {u, v} ∈ (V

2

). ▶ introduce constaints for being bipartite graph metric. ▶ introduce constraints to rule out odd cycles. max ∑

{u,v}∈E

yu,v s.t. yu,v ≤ yu,w + yw,v, ∀u, v, w ∈ V yu,v + yu,w + yw,v ≤ 2, ∀u, v, w, ∈ V yu,v = yv,u, ∀u, v ∈ V

Advanced Algorithms (III) 4/11

▶ introduce yu,v and yv,u for every {u, v} ∈ (V

2

). ▶ introduce constaints for being bipartite graph metric. ▶ introduce constraints to rule out odd cycles. max ∑

{u,v}∈E

yu,v s.t. yu,v ≤ yu,w + yw,v, ∀u, v, w ∈ V yu,v + yu,w + yw,v ≤ 2, ∀u, v, w, ∈ V yu,v = yv,u, ∀u, v ∈ V yu,v ∈ {0, 1} , ∀u, v ∈ V

Advanced Algorithms (III) 4/11

▶ introduce yu,v and yv,u for every {u, v} ∈ (V

2

). ▶ introduce constaints for being bipartite graph metric. ▶ introduce constraints to rule out odd cycles. max ∑

{u,v}∈E

yu,v s.t. yu,v ≤ yu,w + yw,v, ∀u, v, w ∈ V yu,v + yu,w + yw,v ≤ 2, ∀u, v, w, ∈ V yu,v = yv,u, ∀u, v ∈ V yu,v ∈ [0, 1], ∀u, v ∈ V

Advanced Algorithms (III) 4/11

Interality Gap

Theorem

For every , there exists a graph G such that LP G MaxCut G Random graph n p for proper p…

Advanced Algorithms (III) 5/11

Interality Gap

Theorem

For every ε > 0, there exists a graph G such that LP(G) MaxCut(G) ≥ 2 − ε Random graph n p for proper p…

Advanced Algorithms (III) 5/11

Interality Gap

Theorem

For every ε > 0, there exists a graph G such that LP(G) MaxCut(G) ≥ 2 − ε Random graph G(n, p) for proper p…

Advanced Algorithms (III) 5/11

LP in Matrix Form

max x y s.t. x y x y x y max x y s.t. x y x y x y

Hadamard Product

A B

i j n

aij bij

Advanced Algorithms (III) 6/11

LP in Matrix Form

max 2x − 3y s.t. x + y ≤ 2 3x − y ≤ 1 x ≥ 0 y ≥ 0 max x y s.t. x y x y x y

Hadamard Product

A B

i j n

aij bij

Advanced Algorithms (III) 6/11

LP in Matrix Form

max 2x − 3y s.t. x + y ≤ 2 3x − y ≤ 1 x ≥ 0 y ≥ 0 max [2 −3 ]

• [x

−y ] s.t. [1 1 ]

• [x

y ] ≤ 2 [3 −1 ]

• [x

y ] ≤ 1 [x y ] ⪰ 0

Hadamard Product

A B

i j n

aij bij

Advanced Algorithms (III) 6/11

LP in Matrix Form

max 2x − 3y s.t. x + y ≤ 2 3x − y ≤ 1 x ≥ 0 y ≥ 0 max [2 −3 ]

• [x

−y ] s.t. [1 1 ]

• [x

y ] ≤ 2 [3 −1 ]

• [x

y ] ≤ 1 [x y ] ⪰ 0

Hadamard Product

A • B ≜ ∑

1≤i,j≤n

aij · bij.

Advanced Algorithms (III) 6/11

Positive Semi-definite Matrix

Definition

An n × n symmetric matrix A is positive semi-definite if xTAx ≥ 0 for every vector x. We write it as A ⪰ 0.

Linear Programming

max cTx s.t. aT

i x

bi i m xj j n

PSD Programming

max C X s.t. Ai X bi i m X

Advanced Algorithms (III) 7/11

Positive Semi-definite Matrix

Definition

An n × n symmetric matrix A is positive semi-definite if xTAx ≥ 0 for every vector x. We write it as A ⪰ 0.

Linear Programming

max cTx s.t. aT

i x ≤ bi,

∀i ∈ [m] xj ≥ 0, ∀j ∈ [n]

PSD Programming

max C X s.t. Ai X bi i m X

Advanced Algorithms (III) 7/11

Positive Semi-definite Matrix

Definition

An n × n symmetric matrix A is positive semi-definite if xTAx ≥ 0 for every vector x. We write it as A ⪰ 0.

Linear Programming

max cTx s.t. aT

i x ≤ bi,

∀i ∈ [m] xj ≥ 0, ∀j ∈ [n]

PSD Programming

max C • X s.t. Ai • X ≤ bi, ∀i ∈ [m] X ⪰ 0

Advanced Algorithms (III) 7/11

Property of PSD Matrix

Theorem

For an n n symmetric matrix, the followings are equivalent

• 1. A

;

• 2. A has n non-negative eigenvalues;
• 3. A

VTV for some n n matrix V v v vn . We now prove the theorem using spectral theorem for symmetric matrices.

Advanced Algorithms (III) 8/11

Property of PSD Matrix

Theorem

For an n × n symmetric matrix, the followings are equivalent

• 1. A ⪰ 0;
• 2. A has n non-negative eigenvalues;
• 3. A = VTV for some n × n matrix V =

[v1 v2 . . . vn ] . We now prove the theorem using spectral theorem for symmetric matrices.

Advanced Algorithms (III) 8/11

Property of PSD Matrix

Theorem

For an n × n symmetric matrix, the followings are equivalent

• 1. A ⪰ 0;
• 2. A has n non-negative eigenvalues;
• 3. A = VTV for some n × n matrix V =

[v1 v2 . . . vn ] . We now prove the theorem using spectral theorem for symmetric matrices.

Advanced Algorithms (III) 8/11

Vector Programming

If we write X VTV for some V v v vn , then

PSD Programming

max C X s.t. Ak X bk k m X

Vector Programming

max

i j n

c i j vT

i vj

s.t.

i j n

ak i j vT

i vj

bk k m vi

n

i n

Advanced Algorithms (III) 9/11

Vector Programming

If we write X = VTV for some V = [v1 v2 . . . vn ] , then

PSD Programming

max C • X s.t. Ak • X ≤ bk, ∀k ∈ [m] X ⪰ 0

Vector Programming

max ∑

1≤i,j≤n

c(i, j) · vT

i vj

s.t. ∑

1≤i,j≤n

ak(i, j) · vT

i vj ≤ bk,

∀k ∈ [m] vi ∈ Rn, ∀i ∈ [n]

Advanced Algorithms (III) 9/11

Back to MaxCut

It is easy to model MaxCut as the following quadratic programming. max

e u v E

xuxv s.t. xu u V We relax it to a vector programming, which is equivalent to an SDP. max

e u v E

wT

uwv

s.t. wu

n

u V wu u V

Advanced Algorithms (III) 10/11

Back to MaxCut

It is easy to model MaxCut as the following quadratic programming. max

e u v E

xuxv s.t. xu u V We relax it to a vector programming, which is equivalent to an SDP. max

e u v E

wT

uwv

s.t. wu

n

u V wu u V

Advanced Algorithms (III) 10/11

Back to MaxCut

It is easy to model MaxCut as the following quadratic programming. max 1 2 ∑

e={u,v}∈E

(1 − xuxv) s.t. xu ∈ {−1, 1} , ∀u ∈ V We relax it to a vector programming, which is equivalent to an SDP. max

e u v E

wT

uwv

s.t. wu

n

u V wu u V

Advanced Algorithms (III) 10/11

Back to MaxCut

It is easy to model MaxCut as the following quadratic programming. max 1 2 ∑

e={u,v}∈E

(1 − xuxv) s.t. xu ∈ {−1, 1} , ∀u ∈ V We relax it to a vector programming, which is equivalent to an SDP. max

e u v E

wT

uwv

s.t. wu

n

u V wu u V

Advanced Algorithms (III) 10/11

Back to MaxCut

It is easy to model MaxCut as the following quadratic programming. max 1 2 ∑

e={u,v}∈E

(1 − xuxv) s.t. xu ∈ {−1, 1} , ∀u ∈ V We relax it to a vector programming, which is equivalent to an SDP. max 1 2 ∑

e={u,v}∈E

( 1 − wT

uwv

) s.t. wu ∈ Rn, ∀u ∈ V ∥wu∥2 = 1, ∀u ∈ V

Advanced Algorithms (III) 10/11

Back to MaxCut

It is easy to model MaxCut as the following quadratic programming. max 1 2 ∑

e={u,v}∈E

(1 − xuxv) s.t. xu ∈ {−1, 1} , ∀u ∈ V We relax it to a vector programming, which is equivalent to an SDP. max 1 2 ∑

e={u,v}∈E

( 1 − wT

uwv

) s.t. wu ∈ Rn, ∀u ∈ V ∥wu∥2 = 1, ∀u ∈ V

Advanced Algorithms (III) 10/11

Rounding

Let {wu} be an optimal solution of the SDP. Choose a random hyperplane to divide the vectors into two classes… Next week: How to implement the rounding? How to analyze the performance?

Advanced Algorithms (III) 11/11

Rounding

Let {wu} be an optimal solution of the SDP. Choose a random hyperplane to divide the vectors into two classes… Next week: How to implement the rounding? How to analyze the performance?

Advanced Algorithms (III) 11/11

Rounding

Let {wu} be an optimal solution of the SDP. Choose a random hyperplane to divide the vectors into two classes… Next week: How to implement the rounding? How to analyze the performance?

Advanced Algorithms (III) 11/11