Approximation Algorithms (Part I) P : an optimization problem I : an - - PDF document

SLIDE 1

1

Approximation Algorithms (Part I)

P : an optimization problem I : an instance of P A : an algorithm for solving P V(I) : the value for I that is obtained by A V*(I) : the optimal value for I A is an approximation algorithm for P, if for each I, A can generate a feasible solution with V(I) close to V*(I).

SLIDE 2

2

• Classification

Suppose that A is an approximation algorithm for an optimization problem P. ♣ ♣ ♣ ♣ A is an absolute approximation algorithm for P if and only if for each I, |V*(I) − − − − V(I)| ≤ ≤ ≤ ≤ c, where c is a constant. ♣ ♣ ♣ ♣ A is an h(n)-approximate algorithm for P if and only if for each I,

V * I V I V * I ( ) ( ) ( )

− − − −

≤

≤ ≤ ≤ h(n), where n is the size of I.

SLIDE 3

3

♣ ♣ ♣ ♣ A is a c-approximate algorithm for P if and only if for each I,

V * I V I V * I ( ) ( ) ( )

− − − −

≤

≤ ≤ ≤ c, where c is a constant. Notice that

V * I V I V * I ( ) ( ) ( )

− − − −

≤

≤ ≤ ≤ 1, if P is a maximization problem. Hence, c < 1 is required for maximization problems. ♣ ♣ ♣ ♣ A is an approximation scheme for P if and

• nly if for each given ε

ε ε ε > 0 and for each I, A can generate a feasible solution with

V * I V I V * I ( ) ( ) ( )

− − − −

≤

≤ ≤ ≤ ε ε ε ε. The time complexity of A relies on the value of ε ε ε ε, in addition to the size of I.

SLIDE 4

4

♣ ♣ ♣ ♣ An approximation scheme is referred to as a polynomial time approximation scheme if and

• nly if its time complexity is polynomial to the

size of I. ♣ ♣ ♣ ♣ A polynomial time approximation scheme is further referred to as a fully polynomial time approximation scheme if and only if its time complexity is also polynomial to 1

ε ε ε ε .

For some NP-hard problems, designing efficient and “accurate” approximation algorithms is as hard as designing efficient exact algorithms.

SLIDE 5

5

Ex. 0/1 Knapsack Instance : A finite set U = {u1, u2, …, un}, a “size” s(ui) ∈ ∈ ∈ ∈ Z+

and a “value”

v(ui) ∈ ∈ ∈ ∈ Z+

for each ui ∈

∈ ∈ ∈ U, and a size constraint b ∈ ∈ ∈ ∈ Z+. Question : What is the subset U’ ⊆ ⊆ ⊆ ⊆ U such that

i

i

u U'

s u

∑

( )

∈ ∈ ∈ ∈

≤

≤ ≤ ≤ b and

i

i

u U'

v u

∑

( )

∈ ∈ ∈ ∈

is

maximized ? There is an approximation algorithm as follows : examine the elements ui in nonincreasing order

• f

i i

v u s u ( ) ( )

and add ui to U’ if feasible.

For the instance of U = {u1, u2}, v(u1) = 100, v(u2) = 20, s(u1) = 4, s(u2) = 1, and b = 4. ⇒ ⇒ ⇒ ⇒ U’ = {u1} and V(I) = 100 = V*(I).

SLIDE 6

6

Consider the following instance : U = {u1, u2}, v(u1) = 2, v(u2) = r, s(u1) = 1, s(u2) = r, and b = r, where r > 2. ⇒ ⇒ ⇒ ⇒ U’ = {u1}, V(I) = 2, and V*(I) = r. This algorithm is not an absolute approximation algorithm, because |V*(I) − − − − V(I)| = r − − − − 2 is not a constant. This algorithm is a 1-approximate algorithm, because

V * I V I V * I ( ) ( ) ( )

− − − −

=

r − 2 1

≤

≤ ≤ ≤ 1. This algorithm is not a c-approximate algorithm for any c < 1.

SLIDE 7

7

• Absolute Approximation

Absolute approximation algorithms are the most desirable approximation algorithms. However, there are very few NP-hard problems whose polynomial-time absolute approximation algorithms are available. In particular, designing polynomial-time absolute approximation algorithms for some NP-hard problems was shown NP-hard.

• Ex. Consider the problem of finding the minimum

number d of colors needed to color a planar graph G = (V, E). We have d = 0 if V is empty, d = 1 if E is empty (V is not empty), and d = 2 if G is bipartite (neither V nor E is empty).

SLIDE 8

8

Determining whether d = 3 is NP-hard, if G is not bipartite and neither V nor E is empty. Since every planar graph is 4-colorable, an easy absolute approximation algorithm with |V*(I) − − − − V(I)| ≤ ≤ ≤ ≤ 1 is as follows. Step 1. Return d = 0 if V is empty. Step 2. Return d = 1 if E is empty. Step 3. Return d = 2 if G is bipartite. Step 4. Return d = 4. The time complexity of the algorithm is dominated by Step 3, which takes O(|V| + |E|) time.

SLIDE 9

9

• Ex. Consider the following NP-hard problem,

which is a restricted subproblem of the well known bin packing problem. “Given two disks, each of capacity c, and n programs with storage requirements l1, l2, …, ln, respectively, determine the maximum number of programs that can be stored in the two disks (no program stored in two disks).” There is an O(n log n) time absolute approximation algorithm with |V*(I) − − − − V(I)| ≤ ≤ ≤ ≤ 1 as follows Step 1. Arrange the programs in a nondecreasing

• rder of li’s.

Step 2. Examine the programs in the sorted order and assign them to the first disk if it has enough space, and to the second disk else, until no further assignment is possible.

SLIDE 10

10

For example, if n = 4, (l1, l2, l3, l4) = (2, 4, 5, 6), and c = 10, then the first two programs are stored in the first disk and the third program is stored in the second disk. For the example, an optimal solution is to store the first and third (or fourth) disks in one disk and the other two in the other disk. ♣ ♣ ♣ ♣ Proof of |V*(I) − − − − V(I)| ≤ ≤ ≤ ≤ 1 Let p be the maximum number of programs stored in a disk of capacity 2c, which happens when programs are stored in a nondecreasing

• rder of li’s.

⇒ V*(I) ≤ ≤ ≤ ≤ p Assume that the p programs stored in the disk have storage requirements l1 ≤ ≤ ≤ ≤ l2 ≤ ≤ ≤ ≤ … ≤ ≤ ≤ ≤ lp, and p’ is the greatest index with

p' i i

l

=

∑

1

≤

≤ ≤ ≤ c.

SLIDE 11

11

Since

p i i p'+

l

− =∑ 1 1

≤

≤ ≤ ≤

p i i p'+

l

=∑ 2

≤

≤ ≤ ≤ c, we have V(I) ≥ ≥ ≥ ≥ p − − − − 1 (i.e., store the first p’ programs in the first disk and the (p’ + 1)th to (p − − − − 1)th programs to the second disk). Therefore, |V*(I) − − − − V(I)| ≤ ≤ ≤ ≤ 1. When k ≥ ≥ ≥ ≥ 2 disks are used, |V*(I) − − − − V(I)| ≤ ≤ ≤ ≤ k − − − − 1 can be proved similarly.

SLIDE 12

12

• NP-Hardness of Absolute

Approximation

Ex. 0/1 Knapsack Instance : A finite set U = {u1, u2, …, un}, a “size” s(ui) ∈ ∈ ∈ ∈ Z+

and a “value”

v(ui) ∈ ∈ ∈ ∈ Z+

for each ui ∈

∈ ∈ ∈ U, and a size constraint b ∈ ∈ ∈ ∈ Z+. Question : What is the subset U’ ⊆ ⊆ ⊆ ⊆ U such that

i

i

u U'

s u

∑

( )

∈ ∈ ∈ ∈

≤

≤ ≤ ≤ b and

i

i

u U'

v u

∑

( )

∈ ∈ ∈ ∈

is

maximized ? Π Π Π Π : the problem of designing a polynomial-time absolute approximation algorithm for 0/1 Knapsack. We show below that Π Π Π Π is NP-hard.

SLIDE 13

13

It suffices to show that if there exists a polynomial- time absolute approximation algorithm for 0/1 Knapsack, then 0/1 Knapsack can be solved in polynomial time (i.e., 0/1 Knapsack ∝ ∝ ∝ ∝ Π Π Π Π). Suppose that A is a polynomial-time absolute approximation algorithm for 0/1 Knapsack with |V*(I) − − − − V(I)| ≤ ≤ ≤ ≤ k, where k is a constant. Let I be any instance of 0/1 Knapsack, and I

• be

the instance of 0/1 Knapsack that is obtained by multiplying each “value” (i.e., v(ui)) of I by k + 1. ⇒ ⇒ ⇒ ⇒ (1) |V*( I ) − − − − V( I )| is a multiple of k + 1. (2) I and I

• have the same optimal solution

(i.e., the same subset U’) and V*( I ) = (k + 1)V*(I).

SLIDE 14

14

When applying A to I , we have |V*( I ) − − − − V( I )| ≤ ≤ ≤ ≤ k, which together with (1) assures V( I ) = V*( I ). ⇒ ⇒ ⇒ ⇒ A can generate an optimal solution for I

• (and hence an optimal solution for I).

For example, consider I as follows : U = {u1, u2, u3}, (v(u1), v(u2), v(u3)) = (1, 2, 3), (s(u1), s(u2), s(u3)) = (50, 60, 30), and b = 100, for which U’ = {u2, u3} is the optimal solution and V*(I) = 5. When I

• changes (v(u1), v(u2), v(u3)) to

(1 × × × × 5, 2 × × × × 5, 3 × × × × 5) = (5, 10, 15), the optimal U’ remains the same, but V*( I ) = 5 × × × × 5 = 25. If A can guarantee |V*( I ) − − − − V( I )| ≤ ≤ ≤ ≤ 4, then A would compute V( I ) = 25 and output U’ = {u2, u3} for I , which is also optimal for I.

SLIDE 15

15

• Ex. Clique

Instance : An undirected graph G = (V, E). Question : What is the size of a maximum clique

• f G ?

Π Π Π Π : the problem of designing a polynomial-time absolute approximation algorithm for Clique. In order to show that Π Π Π Π is NP-hard, it suffices to show that if there exists a polynomial-time absolute approximation algorithm A for Clique, then Clique can be solved in polynomial time. Assume that A guarantees |V*(I) − − − − V(I)| ≤ ≤ ≤ ≤ k for each instance I of Clique, where k is a constant.

SLIDE 16

16

Let I denote any instance G = (V, E) of Clique, and I’ denote the instance G’ = (V’, E’) of Clique, where G’ contains k + 1 copies of G and every two vertices in distinct copies are connected by an edge. For example, when k = 1, the following graph G

1 2 3 4

will induce G’ as follows.

1 2 3 4 1' 2' 3' 4'

⇒ ⇒ ⇒ ⇒ The maximum clique in G has size 3 and the maximum clique in G’ has size 2 × × × × 3 = 6.

SLIDE 17

17

In general, G has a clique of size q if and only if G’ has a clique of size (k + 1) × × × × q. When applying A to G’, we have |V*(I’) − − − − V(I’)| ≤ ≤ ≤ ≤ k. Since |V*(I) − − − − V(I)| is a multiple of k + 1, V(I’) = V*(I’) is implied, i.e., A can generate an optimal solution for I’ (and hence for I).

SLIDE 18

18

• h(n)-Approximation
• Ex. Given m identical processors, denoted by Pi

(1 ≤ ≤ ≤ ≤ i ≤ ≤ ≤ ≤ m), and n jobs, denoted by Jk (1 ≤ ≤ ≤ ≤ k ≤ ≤ ≤ ≤ n), a schedule is to assign each job with a time interval and a processor for processing. Each job is not allowed to be processed by more than one processor at the same time. A schedule is nonpreemptive if each job is required to be processed continuously from start to end by the same processor, and preemptive else. tk : the amount of processing time required for Jk; Fi : the time when Pi completes the processing

• f all the jobs assigned to it.

It is an NP-hard problem to find a nonpreemptive schedule that can minimize max{Fi | 1 ≤ ≤ ≤ ≤ i ≤ ≤ ≤ ≤ m}.

SLIDE 19

19

An LPT (longest processing time) schedule is to assign a free job with the longest processing time to a processor whenever it becomes available. For example, the following is an LPT schedule for m = 3, n = 6, and (t1, t2, t3, t4, t5, t6) = (8, 7, 6, 5, 4, 3).

6 7 8 11 J1 P1 J2 J3 J4 J6 J5 P2 P3 time

⇒ ⇒ ⇒ ⇒ max{F1, F2, F3} = 11, which is minimized.

SLIDE 20

20

An LPT schedule for m = 3, n = 7, and (t1, t2, t3, t4, t5, t6, t7) = (5, 5, 4, 4, 3, 3, 3) is shown below.

4 5 8 11 J1 P1 J2 J3 J4 J6 J5 P2 P3 time J7

This schedule has max{F1, F2, F3} = 11, which is not optimal. An optimal schedule is shown as follow.

5 9 J1 P1 J2 J3 J4 J6 J5 P2 P3 time J7

For this instance, we have

V * I V I V * I ( ) ( ) ( )

− − − −

= (11 −

− − − 9) / 9 = 2/9.

SLIDE 21

21

It takes O(n log n) time to generate an LPT schedule, which can guarantee

V * I V I V * I ( ) ( ) ( )

− − − −

≤ ≤ ≤ ≤

m − 1 1 3 3

. The upper bound is tight, because it equals 2

9 as

m = 3 (refer to the instance above). ♣ ♣ ♣ ♣ Proof of

V * I V I V * I ( ) ( ) ( )

− − − −

≤

≤ ≤ ≤

m − 1 1 3 3

When m = 1, the inequality holds (V*(I) = V(I)). So, assume m ≥ ≥ ≥ ≥ 2 below. Suppose that the inequality is violated for some m and (t1, t2, …, tn). Besides, it is assumed that n is minimum, while violating the inequality. Assume t1 ≥ ≥ ≥ ≥ t2 ≥ ≥ ≥ ≥ … ≥ ≥ ≥ ≥ tn, i.e., J1, J2, …, Jn are processed in this sequence.

SLIDE 22

22

fk : the time when the processing of Jk is finished. ⇒ ⇒ ⇒ ⇒ max{fi | 1 ≤ ≤ ≤ ≤ i ≤ ≤ ≤ ≤ n} = max{Fi | 1 ≤ ≤ ≤ ≤ i ≤ ≤ ≤ ≤ m}. ♦ ♦ ♦ ♦ We first show fn = max{fi | 1 ≤ ≤ ≤ ≤ i ≤ ≤ ≤ ≤ n} below. Suppose fn’ = max{fi | 1 ≤ ≤ ≤ ≤ i ≤ ≤ ≤ ≤ n}, where n’ < n. ⇒ ⇒ ⇒ ⇒ V(I’) = fn’ = max{Fi | 1 ≤ ≤ ≤ ≤ i ≤ ≤ ≤ ≤ m} = V(I), where I’ denotes the instance of (t1, t2, …, tn’) and I denotes the instance of (t1, t2, …, tn). Since V*(I’) ≤ ≤ ≤ ≤ V*(I), we have

V * I' V I' V * I' ( ) ( ) ( )

− − − −

≥

≥ ≥ ≥

V * I V I V * I ( ) ( ) ( )

− − − −

(>

m − 1 1 3 3

), which contradicts the assumption of n. ♦ ♦ ♦ ♦ Next we show V*(I) < 3 × × × × tn below, meaning that at most two jobs are assigned to each processor in an optimal schedule for I.

SLIDE 23

23

Since fn = V(I), the processing of Jn starts at time V(I) − − − − tn, i.e., all processors are busy between time 0 and time V(I) − − − − tn. ⇒ ⇒ ⇒ ⇒ V(I) − − − − tn ≤ ≤ ≤ ≤

n i i

m

t

−

∑

1 =1

1 ×

× × ×

⇒ V(I) ≤ ≤ ≤ ≤

n i i

m

t

∑

=1

1 ×

× × ×

+

n

m m

t

− 1 ×

× × ×

⇒ V(I) ≤ ≤ ≤ ≤ V*(I) +

n

m m

t

− 1 ×

× × ×

(

n i i

m

t

∑

=1

1 ×

× × ×

≤

≤ ≤ ≤ V*(I)) ⇒

V * I V I V * I ( ) ( ) ( )

− − − −

=

V I V * I ( ) ( )

1 −

− − −

≤ ≤ ≤ ≤ (

) *( )

n

m m V I

t

− 1 ×

× × × × × × ×

Now that

V * I V I V * I ( ) ( ) ( )

− − − −

>

m − 1 1 3 3

, we have

m − 1 1 3 3

< (

) *( )

n

m m V I

t

− 1 ×

× × × × × × ×

, from which V*(I) < 3 × × × × tn can be derived.

SLIDE 24

24

It was proved that an LPT schedule is optimal, if any optimal schedule has at most two jobs assigned to each processor (refer to page 568 of

• Ref. (2)).

⇒ ⇒ ⇒ ⇒

V * I V I V * I ( ) ( ) ( )

− − − −

= 0,

contradicting the assumption about I !

SLIDE 25

25

• c-Approximation

Ex. Bin Packing Instance : A capacity value b > 0 and a finite set U of items whose sizes do not exceed b. Question : What is the minimum number of bins of equal capacity b to put away all items of U (each item is placed entirely in one bin) ? For example, given b = 10 and six items of sizes 5, 6, 3, 7, 5 and 4, an optimal packing is shown below.

5 5 6 4 7 3

SLIDE 26

26

Four heuristics for Bin Packing :

• First Fit (FF) :

Arrange the bins in sequence and place each item into the first bin in which it fits.

• Best Fit (BF) :

Arrange the bins in sequence and place each item into the most nearly full bin in which it fits.

• First Fit Decreasing (FFD) :

the same as FF, except that the items are placed in a nonincreasing sequence of sizes.

• Best Fit Decreasing (BFD) :

the same as BF, except that the items are placed in a nonincreasing sequence of sizes.

SLIDE 27

27

For the example of b = 10 and sizes 5, 6, 3, 7, 5, 4, FF :

5 3 4 6 7 5

BF :

5 3 4 6 7 5

FFD, BFD :

5 3 4 6 7 5

SLIDE 28

28

Properties : (P1) For FF or BF, V(I) ≤ ≤ ≤ ≤ 17

10

×

× × × V*(I) + 2; (P2) For FFD or BFD, V(I) ≤ ≤ ≤ ≤ 11

9

×

× × × V*(I) + 4. The proofs of (P1) and (P2), which are rather lengthy and complex, can be found below. Johnson, Demers, Ullman, Garey, and Graham, “Worst-Case Performance Bounds for Simple One-Dimensional Packing Algorithms,” SIAM Journal on Computing, vol. 3, no. 4, 1974, 299-325. The upper bound of (P1) was further improved to    17

10

×

× × × V*(I)    below. Garey, Graham, Johnson, and Yao, “Resource Constrained Scheduling as Generalized Bin Packing,” Journal of Combinatorial Theory (A),

• vol. 21, 1976, 257-298.

SLIDE 29

29

♣ ♣ ♣ ♣ A simple proof of V(I) < 2 × × × × V*(I) (i.e.,

V * I V I V * I ( ) ( ) ( )

− − − −

< 1) for FF

Assume U = {u1, u2, …, un}, and let si be the size

• f ui, where 1 ≤

≤ ≤ ≤ i ≤ ≤ ≤ ≤ n. When V(I) = 1, we have V*(I) = 1 (V(I) < 2 × × × × V*(I)). We consider V(I) > 1 below. For even V(I), s1 + s2 + … + sn >

( ) V I 2

×

× × × b ⇒ ⇒ ⇒ ⇒ V*(I) ≥ ≥ ≥ ≥

( ) V I 2

+ 1

⇒ ⇒ ⇒ ⇒ V(I) ≤ ≤ ≤ ≤ 2 × × × × V*(I) − − − − 2. For odd V(I), s1 + s2 + … + sn >

( ) V I − 1 2

×

× × × b ⇒ ⇒ ⇒ ⇒ V*(I) ≥ ≥ ≥ ≥

( ) V I − 1 2

+ 1

⇒ ⇒ ⇒ ⇒ V(I) ≤ ≤ ≤ ≤ 2 × × × × V*(I) − − − − 1.

SLIDE 30

30

Ex. Traveling Salesman Problem (TSP) Instance : A set C of m cities and distances di,j > 0 for all pairs of cities i, j ∈ ∈ ∈ ∈ C. Question : What is the length of a shortest tour that starts at any city, visits each of the other m − − − − 1 cities exactly once, and returns to the initial city ? When all di,j’s satisfy triangle inequality, i.e., di,j ≤ ≤ ≤ ≤ di,k + dk,j for all i, j, k ∈ ∈ ∈ ∈ C, the metric TSP results. The Euclidean TSP, where di,j is the geodesic distance between i and j, is an instance of the metric TSP. Both the metric TSP and the Euclidean TSP are NP-hard.

SLIDE 31

31

♣ A heuristic with V(I) < 2 × × × × V*(I) For example,

1 2 3 4 5 6

Step 1. Find an MST.

1 2 3 4 5 6

Let d* be the total distance of it. Any feasible tour is a spanning tree augmented with an edge. ⇒ ⇒ ⇒ ⇒ d* < V*(I)

SLIDE 32

32

Step 2. Replace each edge of the tree with two

• pposite arcs.

1 2 3 4 5 6

Step 3. Construct an Euler circuit of total distance 2 × × × × d* on the graph above. 1 → → → → 2 → → → → 3 → → → → 2 → → → → 4 → → → → 6 → → → → 4 → → → → 5 → → → → 4 → → → → 2 → → → → 1 Step 4. Arbitrarily find a feasible tour from the circuit above. 1 → → → → 2 → → → → 3 − − − −− − − −− − − −→ → → → 4 → → → → 6 − − − −− − − −− − − −→ → → → 5 − − − −− − − −− − − −− − − −− − − −− − − −→ → → → 1 The feasible tour has V(I) < 2 × × × × d* < 2 × × × × V*(I).

SLIDE 33

33

The time complexity is dominated by Step 1, which takes O(n log n) time for the Euclidean TSP. ♣ ♣ ♣ ♣ A heuristic with V(I) < 3

2

×

× × × V*(I) Consider the same example above. Step 1. Find an MST.

1 2 3 4 5 6

Let d* be the total distance of it. ⇒ ⇒ ⇒ ⇒ d* < V*(I) (1) There are an even number of odd-degree vertices in the tree.

SLIDE 34

34

Step 2. Find a maximum matching with minimum cost on the set of odd-degree vertices.

1 2 3 4 5 6

Step 3. Add the edges of the matching to the MST. (All the vertices are of even degrees.)

1 2 3 4 5 6

Step 4. Construct an Euler circuit on the graph above. 1 → → → → 3 → → → → 2 → → → → 4 → → → → 6 → → → → 5 → → → → 4 → → → → 2 → → → → 1 Step 5. Arbitrarily find a feasible tour from the circuit above. 1 → → → → 3 → → → → 2 → → → → 4 → → → → 6 → → → → 5 − − − −− − − −− − − −− − − −− − − −− − − −− − − −→ → → → 1

SLIDE 35

35

The time complexity is dominated by Step 2, which takes O(m3) time for the Euclidean TSP (refer to Combinatorial Optimization : Networks and Matroids, by E. L. Lawler, 1976). Suppose that there are 2k odd-degree vertices : a1, a2, …, a2k, in the MST. (It is also assumed that these 2k vertices appear in some shortest tour with the same sequence.) M1 = {(a1, a2), (a3, a4), …, (a2k−

− − −1, a2k)}

M2 = {(a2, a3), (a4, a5), …, (a2k−

− − −2, a2k− − − −1), (a2k, a1)}

(M1 and M2 are two maximum matchings.) c1 (c2) : the cost of M1 (M2) ⇒ ⇒ ⇒ ⇒ c1 + c2 ≤ ≤ ≤ ≤ V*(I) (2) (guaranteed by triangle inequality)

SLIDE 36

36

c* : the minimum cost of a maximum matching

• n {a1, a2, …, a2k}

⇒ ⇒ ⇒ ⇒ 2 × × × × c* ≤ ≤ ≤ ≤ c1 + c2 (3) l : the total distance of the Euler circuit ⇒ ⇒ ⇒ ⇒ l = c* + d* (4) and V(I) ≤ ≤ ≤ ≤ l (5) (1), (2), (3), (4), (5) ⇒ ⇒ ⇒ ⇒ V(I) < 3

2

×

× × × V*(I) Exercise 11. Read Sec. 9-1 of the textbook. (1) Illustrate the approximation algorithm by an example. (2) Show V(I) ≤ ≤ ≤ ≤ 2 × × × × V*(I). Exercise 12. Read Sec. 9-6 of the textbook. (1) Illustrate the approximation algorithm by an example. (2) Show V(I) ≤ ≤ ≤ ≤ 2 × × × × V*(I).

SLIDE 37

37

• NP-Hardness of c-Approximation
• Ex. TSP

Let Π Π Π Π be the problem of designing a c-approximate algorithm for TSP. We show Hamiltonian Cycle ∝ ∝ ∝ ∝ Π Π Π Π below. Hamiltonian Cycle Instance : An undirected graph G = (V, E). Question : Does G contain a Hamiltonian cycle, i.e., an ordering (v1, v2, …, v|V|) of the vertices of G such that (v1, v|V|) ∈ ∈ ∈ ∈ E and (vi, vi+1) ∈ ∈ ∈ ∈ E for all 1 ≤ ≤ ≤ ≤ i < |V| ? Suppose that G = (V, E) is an arbitrary instance of Hamiltonian Cycle, where |V| = n is assumed. Construct as an instance of TSP a weighted complete graph G

• = (V, E

) whose weights are as follows.

SLIDE 38

38

w(u, v) = 1, if (u, v) ∈ ∈ ∈ ∈ E; w(u, v) = k, if (u, v) ∈ ∈ ∈ ∈ E − − − − E, where k > (1 + c) × × × × n. An illustrative example of G and G

• is below.

1 2 3 4 5 1 2 3 4 5

1 1 1 1 1 k k k k 1

G has a Hamiltonian cycle if and only if V*(I) = n for G . If there is a c-approximate algorithm for TSP, then whether or not V*(I) = n for G

• can be

decided from V(I) as follows.

SLIDE 39

39

♦ V(I) = n implies V*(I) = n, because V*(I) ≤ ≤ ≤ ≤ V(I). ♦ V(I) > n implies V*(I) > n (equivalently, V*(I) = n implies V(I) = n), as explained below. V(I) ∈ ∈ ∈ ∈ {n, n + (k − − − − 1), n + 2 × × × × (k − − − − 1), …, n + n × × × × (k − − − − 1)}. (1)

V * I V I V * I ( ) ( ) ( )

− − − −

≤

≤ ≤ ≤ c. (2) When V*(I) = n, a unique value of V(I) from (1), i.e., V(I) = n, can satisfy (2). Therefore, if there is a polynomial-time c-approximate algorithm for TSP, then Hamiltonian Cycle is polynomial time solvable.

SLIDE 40

40

• Ex. 0/1 Integer Programming

min (or max) p0 +

n j j j

p x

∑

=1

subject to

, n i j j j

a x

∑

=1

≤

≤ ≤ ≤ bi, for 1 ≤ ≤ ≤ ≤ i ≤ ≤ ≤ ≤ m xj ∈ ∈ ∈ ∈ {0, 1}, for 1 ≤ ≤ ≤ ≤ j ≤ ≤ ≤ ≤ n (p0, pj, ai,j, bi are integers) 0/1 Integer Programming is an NP-hard problem (0/1 Knapsack is an instance of it). Let Π Π Π Π be the problem of designing a c-approximate algorithm for 0/1 Integer Programming. We show Partition ∝ ∝ ∝ ∝ Π Π Π Π below.

SLIDE 41

41

Partition Instance : A multiset B = {b1, b2, …, bn} of positive integers. Question : Is there a subset B’ ⊆ ⊆ ⊆ ⊆ B such that

i

i

b B'

b

∑

∈ ∈ ∈ ∈

=

j

j

b B B'

b

−

∑

∈ ∈ ∈ ∈

?

Suppose that {b1, b2, …, bn} is an arbitrary instance of Partition. Construct an instance of 0/1 Integer Programming as follows, where k =

n j j

b

∑

=1

1 2

and t > c is an integer.

min 1 + t × × × × (k − − − −

n j j j

b x

∑

=1

) (≥ ≥ ≥ ≥ 1) subject to

n j j j

b x

∑

=1

≤

≤ ≤ ≤ k xj ∈ ∈ ∈ ∈ {0, 1}, for 1 ≤ ≤ ≤ ≤ j ≤ ≤ ≤ ≤ n

SLIDE 42

42

The answer of Partition is yes if and only if V*(I) = 1 for the constructed instance. If there is a c-approximate algorithm for 0/1 Integer Programming, then whether V*(I) = 1

• r not can be decided from V(I) as follows.

♦ V(I) = 1 implies V*(I) = 1, because V*(I) ≤ ≤ ≤ ≤ V(I). ♦ V(I) > 1 implies V*(I) > 1 (equivalently, V*(I) = 1 implies V(I) = 1), as explained below. V(I) ∈ ∈ ∈ ∈ {1, 1 + t, 1 + 2 × × × × t, …, 1 + k × × × × t}. (1)

V * I V I V * I ( ) ( ) ( )

− − − −

≤

≤ ≤ ≤ c. (2) When V*(I) = 1, a unique value of V(I) from (1), i.e., V(I) = 1, can satisfy (2).

SLIDE 43

43

Therefore, if there is a polynomial-time c-approximate algorithm for 0/1 Integer Programming, then Partition is polynomial time solvable. Exercise 13. Read Sec. 9-3 of the textbook. (1) Illustrate the approximation algorithm by an example. (2) Show V(I) ≤ ≤ ≤ ≤ 2 × × × × V*(I). (3) Show that the problem of designing a c-approximate algorithm with c < 2 is NP-hard.

SLIDE 44

44

• More Examples

Ex. Minimum Routing Cost Spanning Tree (an NP-hard problem) Instance : A weighted complete graph G whose edge costs obey the triangle inequality. Question : What is the spanning tree of G whose total routing cost between all pairs of vertices is minimum ? The routing cost of two vertices u, v on a tree, denoted by r(u, v), is the total edge cost along the u-v path. Consider the following example where the left graph is G and the right graph is the spanning tree with the minimum total routing cost.

SLIDE 45

45

The total routing cost of the tree is computed as r(a, b) + r(a, c) + r(a, d) + r(b, c) + r(b, d) + r(c, d) = 1 + 2 + 2 + 1 + 1 + 2 = 9. ♣ ♣ ♣ ♣ A heuristic with V(I) < 2 × × × × V*(I) Construct the 1-star (also a spanning tree),

denoted by S*, of G that has the minimum

total routing cost among the n 1-stars of G, where n is the number of vertices in G. For example, the following are K4 and its four 1-stars.

SLIDE 46

46

♣ ♣ ♣ ♣ Proof of V(I) < 2 × × × × V*(I) Suppose that T* is the spanning tree of G that has the minimum total routing cost. Let C(T) denote the total routing cost induced by a spanning tree T. ⇒ ⇒ ⇒ ⇒ V(I) = C(S*), V*(I) = C(T*) A vertex of a tree is a centroid, if each subtree resulting from its deletion has at most 

  n/2   

vertices.

SLIDE 47

47

For example, vertex m is a centroid of the following tree. Suppose that m* is the centroid of T* and ˆ S is the 1-star rooted at m*. ⇒ ⇒ ⇒ ⇒ C(S*) ≤ ≤ ≤ ≤ C( ˆ S ) It suffices to show C( ˆ S ) ≤ ≤ ≤ ≤ 2C(T*). Let w(v, m*) be the edge cost of (v, m*). Fact 1. C( ˆ S ) = (n − − − − 1) × × × ×

*

*

( , )

v m w v m ≠

∑

all

.

SLIDE 48

48

For example, refer to the T* (= ˆ S ) on page 45, where b = m*. C( ˆ S ) = r(a, b) + r(a, c) + r(a, d) + r(b, c) + r(b, d) + r(c, d) = w(a, b) + (w(a, b) + w(b, c)) + (w(a, b) + w(b, d)) + w(b, c) + w(b, d) + (w(c, b) + w(b, d)) = 3(w(a, b) + w(c, b) + w(d, b)) Fact 2. C(T*) > 

  n/2    ×

× × ×

*

*

( , )

u m w u m ≠

∑

all

. Suppose that m* is of degree d. Draw T* as follows, where each subtree Ti has at most 

  n/2    vertices.

SLIDE 49

49

If u ≠ ≠ ≠ ≠ m* and v ≠ ≠ ≠ ≠ m* are not located in the same Ti, then r(u, v) > r(u, m*) ≥ ≥ ≥ ≥ w(u, m*), where “≥ ≥ ≥ ≥” is assured by triangle inequality. For each u≠ ≠ ≠ ≠ m*, there are at least 

  n/2    v’s

such that v ≠ ≠ ≠ ≠ m* and u, v are not in the same Ti. ⇒ ⇒ ⇒ ⇒ C(T*) = ( , )

x y r x y

∑

all

≠ ≠ ≠ ≠

> ( , )

i

T

u m v m u v

r u v ∑

all *, *, and , are not in the same

≠ ≠ ≠ ≠ ≠ ≠ ≠ ≠

>

   n/2    ×

× × × ( , *)

u m w u m

∑

all *

≠ ≠ ≠ ≠

With Fact 1 and Fact 2, C( ˆ S ) ≤ ≤ ≤ ≤ 2C(T*) is derived. The heuristic takes O(n2) time, as explained below. S* can be determined by comparing the total routing costs of the n 1-stars of G.

SLIDE 50

50

The total routing cost of each 1-star can be computed in O(n) time according to Fact 1 (Fact 1 holds for any 1-star). Finally, there are two comments. (1) There is an O(n2k) time heuristic that constructs a minimum total routing cost k-star on a general graph and guarantees V(I) < k

k + +

3 1

×

× × × V*(I), where k ≥ ≥ ≥ ≥ 1. a 3-star (a spanning tree with 3 internal nodes)

SLIDE 51

51

In this heuristic, centroids are replaced with subgraphs of G, named separators. The details can be found in Wu, Lancia, Bafna, Chao, Ravi, and Tang, “A polynomial time approximation scheme for minimum routing cost spanning trees,” SIAM Journal on Computing,

• vol. 29, no. 3, 2000, 761-778.

(2) When k = 2, there is an improvement over (1), where O(n3) time is required and the ratio is bounded by 1.577. The details can be found in Wu, Chao, and Tang, “Approximation algorithms for some optimum communication spanning tree problems,” Discrete Applied Mathematics, vol. 102, no. 3, 2000, 245- 266.

SLIDE 52

52

Ex. Sequence Alignment Instance : A set S = {S1, S2, …, Sk} of k sequences. Question : What are the aligned sequences, denoted by S1, S2, …,

k

S , of S1, S2, …, Sk such that

,

i j i j k d S

S ∑

1 <

( )

≤ ≤ ≤ ≤ ≤ ≤ ≤ ≤

is

minimum (

i

S

includes Si), where d()

computes the distance of

i

S

and

j

S

?

The distance of two (aligned) sequences is the number of mismatches between them. For example, d(GCCAT, GAT) = 4. G C C A T G A T - - (“-“ means blank) The distance can be reduced to 2 by alignment (i.e., adding blanks). G C C A T G - - A T

SLIDE 53

53

Property 1.

,

i i

d S S ( ) = 0 and

,

i j

d S S ( ) +

,

i k

d S S ( ) ≥ ≥ ≥ ≥

,

j k

d S S ( ) (triangle inequality). Let

,

i j

*

d S S ( ) be the minimum distance between all occurrences of

i

S

and

j

S , which can be found in O(|Si| × × × × |Sj|) time by dynamic programming (refer to Sec. 7-3 of the textbook). Sc is the center of S, if

,

c j j k

*

d S S ∑

1

( )

≤ ≤ ≤ ≤ ≤ ≤ ≤ ≤

is minimum,

where 1 ≤ ≤ ≤ ≤ c ≤ ≤ ≤ ≤ k. For example, if S1 = ATGCTC, S2 = AGAGC, S3 = TTCTG, and S4 = ATTGCATGC, then

,

*

d S S

1 2

( )= d*(ATGCTC, A - GAGC) = 3;

,

*

d S S

1 3

( )= d*(ATGCTC, TT - CTG) = 3;

,

*

d S S

1 4

( )= d*(AT - GC - T - C, ATTGCATGC) = 3;

,

*

d S S

2 3

( )= d*(AGAGC, TTCTG) = 5;

SLIDE 54

54

,

*

d S S

2 4

( )= d*(A - - G - A - GC, ATTGCATGC) = 4;

,

*

d S S

3 4

( )= d*( - TT - C - TG -, ATTGCATGC) = 4. ⇒ ⇒ ⇒ ⇒

,

j j

*

d S S ∑

1 1 4

( )

≤ ≤ ≤ ≤ ≤ ≤ ≤ ≤

= 9,

,

j j

*

d S S ∑

2 1 4

( )

≤ ≤ ≤ ≤ ≤ ≤ ≤ ≤

= 12

,

j j

*

d S S ∑

3 1 4

( )

≤ ≤ ≤ ≤ ≤ ≤ ≤ ≤

= 12,

,

j j

*

d S S ∑

4 1 4

( )

≤ ≤ ≤ ≤ ≤ ≤ ≤ ≤

= 11

⇒ ⇒ ⇒ ⇒ S1 is the center. ♣ ♣ ♣ ♣ A heuristic with V(I) < 2 × × × × V*(I) Step 1. Find the center (assume S1) of S. Step 2. For i = 2, 3, …, k, 2.1 find an optimal alignment between S1 and Si; 2.2 add blanks to the already aligned sequences of S1, S2, …, Si−

− − −1.

SLIDE 55

55

For example, S1 = ATGCTC, S2 = AGAGC, S3 = TTCTG, and S4 = ATTGCATGC. Initially, S1 is the center. i = 2 : S1= ATGCTC and S2= A - GAGC result. i = 3 : S1= ATGCTC and S3= - TTCTG result. i = 4 : S1= AT - GC - T - C and S4 = ATTGCATGC

• result. Moreover, S2= A - - GA - G - C and

S3= - T - TC - T - G result, as a consequence

• f a new S1.

♣ ♣ ♣ ♣ Proof of V(I) < 2 × × × × V*(I)

,

i j

D S S ( ): the distance between

i

S

and

j

S induced by the heuristic;

,

i j

*

D S S ( ) : the distance between

i

S

and

j

S induced by an optimal algorithm.

SLIDE 56

56

⇒ ⇒ ⇒ ⇒ V(I) =

,

i j i j k D S

S ∑

1 <

( )

≤ ≤ ≤ ≤ ≤ ≤ ≤ ≤

; V*(I) =

,

i j i j k

*

D S S ∑

1 <

( )

≤ ≤ ≤ ≤ ≤ ≤ ≤ ≤

. By Property 1 (triangle inequality), V(I) ≤ ≤ ≤ ≤

, ,

i j k

i j

D S S D S S ∑

1 1 1 <

( ( ) + ( ))

≤ ≤ ≤ ≤ ≤ ≤ ≤ ≤

. (1) Since

,

i j

*

D S S ( ) ≥ ≥ ≥ ≥

,

i j

*

d S S ( ), we have V*(I) ≥ ≥ ≥ ≥

,

i j i j k

*

d S S ∑

1 <

( )

≤ ≤ ≤ ≤ ≤ ≤ ≤ ≤

. (2)

SLIDE 57

57

Property 2.

, ,

i j k

i j

D S S D S S ∑

1 1 1 <

( ( ) + ( ))

≤ ≤ ≤ ≤ ≤ ≤ ≤ ≤

= ( ) ,

i=

i

k

k D S S − × ∑

1 1

1 ( ). For example, assume k = 5.

, ,

i j

i j

D S S D S S ∑

1 1 1 < 5(

( ) + ( ))

≤ ≤ ≤ ≤ ≤ ≤ ≤ ≤

=

, , , ,

D S S D S S D S S D S S

1 1 1 2 1 1 1 3

[ ( ) + ( ) + ( ) + ( ) +

, , , ,

D S S D S S D S S D S S

1 1 1 4 1 1 1 5

( ) + ( ) + ( ) + ( )]+

, , , ,

D S S D S S D S S D S S

1 2 1 3 1 2 1 4

[ ( ) + ( ) + ( ) + ( ) +

, , , ,

D S S D S S D S S D S S

1 2 1 5 1 3 1 4

( ) + ( )]+[ ( ) + ( ) +

, , , ,

D S S D S S D S S D S S

1 3 1 5 1 4 1 5

( ) + ( )]+[ ( ) + ( )] = 4 × × × ×

, , , ,

D S S D S S D S S D S S

1 1 1 2 1 3 1 4

[ ( ) + ( ) + ( ) + ( ) +

,

D S S

1 5

( )] = ( ) ,

i=

i

D S S − × ∑

1 1

5

5 1 ( ) .

SLIDE 58

58

Property 3.

,

i

D S S

1

( ) =

,

i

*

d S S

1

( ). Refer to the example of page 55 again. When i = 2, S1= ATGCTC and S2= A - GAGC were determined optimally, i.e., d(S1, S2) at i = 2 is equal to

,

*

d S S

1 2

( ). Then, when i = 4, S1

and S2 were changed to

AT - GC - T - C and A - - GA - G - C, respectively, where three blanks were added at the same positions. Therefore, d(S1, S2) (=

,

D S S

1

2

( ) ) at i = 4 is equal to d(S1, S2) (=

,

*

d S S

1 2

( )) at i = 2. By Property 2 and Property 3, (1) can be written as follows: V(I) ≤ ≤ ≤ ≤ ( ) ,

i=

i

k *

k S S

d

− × ∑

1 1

1 ( ) . (3)

SLIDE 59

59

Property 4.

,

i j i j k

*

d S S ∑

1 <

2 ( )

≤ ≤ ≤ ≤ ≤ ≤ ≤ ≤

× × × ×

≥

≥ ≥ ≥ ,

i=

i

k *

k S S

d

× ∑

1 1

( ). For example, assume k = 5.

,

i j

*

d S S ( )’s included in

,

i j i j

*

d S S ∑

1 < 5

( )

≤ ≤ ≤ ≤ ≤ ≤ ≤ ≤

are

denoted by 1’s in either of the following two matrices (

,

i j

*

d S S ( ) =

,

j i

*

d S S ( )). ⇒ ⇒ ⇒ ⇒

,

i j

*

d S S ( )’s included in

,

i j i j

*

d S S ∑

1 < 5

2 ( )

≤ ≤ ≤ ≤ ≤ ≤ ≤ ≤

× × × × are denoted by 1’s in the following matrix.

SLIDE 60

60

Let Ri be the summation of the ith

row in the

matrix above, where 1 ≤ ≤ ≤ ≤ i ≤ ≤ ≤ ≤ 5. ⇒ ⇒ ⇒ ⇒

,

i j i j

*

d S S ∑

1 < 5

2 ( )

≤ ≤ ≤ ≤ ≤ ≤ ≤ ≤

× × × ×

=

i=

i

R ∑

1

5

. Since S1 is the center, we have Ri ≥ ≥ ≥ ≥ R1 (= ,

i=

i

* S

S

d

∑

1 1

5

( ) ). ⇒ ⇒ ⇒ ⇒

i=

i

R ∑

1

5

≥

≥ ≥ ≥ ,

i=

i

* S

S

d

∑

1 1

5

5 ( ) × × × × .

,

i j i j

*

d S S ∑

1 < 5

2 ( )

≤ ≤ ≤ ≤ ≤ ≤ ≤ ≤

× × × ×

≥

≥ ≥ ≥ ,

i=

i

* S

S

d

∑

1 2

5

5 ( ) × × × × . By Property 4, (2) can be written as follows: V*(I) ≥ ≥ ≥ ≥ ,

i=

i

k *

k S S

d

× ∑

1 1

( ) 2 . (4) Combining (3) and (4), we derive V(I) < 2 × × × × V*(I).