Advanced Algorithms Knapsack Problem Instance : n items i =1,2, - - PowerPoint PPT Presentation

Advanced Algorithms

南京大学 尹一通

Knapsack Problem

Instance: n items i=1,2, ..., n;

weights w1, w2, ..., wn ∈ ℤ+; values v1, v2, ..., vn ∈ ℤ+;

knapsack capacity B ∈ ℤ+; Find a subset of items whose total weight is bounded by B and total value is maximized.

weight1 value1 weight2 value2 weight3 value3 weight4 value4 weight5 value5 weight6 value6 capacity B

Knapsack Problem

Find an S ⊆ {1,2, ..., n} that maximizes ∑i∈S vi subject to ∑i∈S wi ≤ B.

• 0-1 Knapsack

problem

• one of Karp’s 21

NP-complete problems

weight1 value1 weight2 value2 weight3 value3 weight4 value4 weight5 value5 weight6 value6 capacity B

Instance: n items i=1,2, ..., n;

weights w1, w2, ..., wn ∈ ℤ+; values v1, v2, ..., vn ∈ ℤ+;

knapsack capacity B ∈ ℤ+;

Greedy Heuristics

Find an S ⊆ {1,2, ..., n} that maximizes ∑i∈S vi subject to ∑i∈S wi ≤ B. Sort all items according to the ratio ri = vi/wi so that r1 ≥ r2 ≥ … ≥ rn ; for i=1,2, ..., n item i joins S if the resulting total weight ≤ B; approximation ratio: arbitrarily bad Instance: n items i=1,2, ..., n;

weights w1, w2, ..., wn ∈ ℤ+; values v1, v2, ..., vn ∈ ℤ+;

knapsack capacity B ∈ ℤ+;

Dynamic Programming

A(i, v) = minimum total weight of S ⊆ {1,2, ..., i} with total value exactly v A(i, v) = ∞ if no such S exists define: Instance: n items i=1,2, ..., n;

weights w1, w2, ..., wn ∈ ℤ+; values v1, v2, ..., vn ∈ ℤ+;

knapsack capacity B ∈ ℤ+;

Dynamic Programming

A(i, v) = if ∃S ⊆ {1,2, ..., i}, v = ∑j∈S vj min

S⊆{1,2,...,i} P j∈S vj =v

X

j∈S

wj

(

∞

• therwise

define: Instance: n items i=1,2, ..., n;

weights w1, w2, ..., wn ∈ ℤ+; values v1, v2, ..., vn ∈ ℤ+;

knapsack capacity B ∈ ℤ+;

Dynamic Programming

A(i, v) = minimum total weight of S ⊆ {1,2, ..., i} with total value exactly v recursion: A(1, v) = if v = v1 w1

(

∞

• therwise

A(i, v) = min{ A(i-1, v), A(i-1, v-vi) + wi } for i > 1 Instance: n items i=1,2, ..., n;

weights w1, w2, ..., wn ∈ ℤ+; values v1, v2, ..., vn ∈ ℤ+;

knapsack capacity B ∈ ℤ+; 1 ≤ i ≤ n, 1 ≤ v ≤ V = ∑i vi Dynamic programming: table size O(nV) time complexity O(nV)

Dynamic Programming

A(i, v) = minimum total weight of S ⊆ {1,2, ..., i} with total value exactly v Dynamic programming: table size O(nV) time complexity O(nV)

Polynomial Time?

Instance: n items i=1,2, ..., n;

weights w1, w2, ..., wn ∈ ℤ+; values v1, v2, ..., vn ∈ ℤ+;

knapsack capacity B ∈ ℤ+; Find a subset of items whose total weight is bounded by B and total value is maximized. knapsack: max v that A(n,v)≤B

Polynomial Time

• polynomial-time Algorithm A:

∃constant c, ∀ input x∈{0,1}*, A(x) terminates in |x|c steps |x| = length of input x (in binary code) Instance: n items i=1,2, ..., n;

weights w1, w2, ..., wn ∈ ℤ+; values v1, v2, ..., vn ∈ ℤ+;

knapsack capacity B ∈ ℤ+; time complexity: O(nV) where V = ∑i vi

• pseudopolynomial-time Algorithm A:

∃constant c, ∀ input x∈{0,1}*, A(x) terminates in |x|c steps |x| = length of input x (in unary code)

Dynamic Programming

A(i, v) = minimum total weight of S ⊆ {1,2, ..., i} with total value exactly v Dynamic programming: time complexity O(nV) where V = ∑i vi

Pseudo- Polynomial Time!

Instance: n items i=1,2, ..., n;

weights w1, w2, ..., wn ∈ ℤ+; values v1, v2, ..., vn ∈ ℤ+;

knapsack capacity B ∈ ℤ+; Find a subset of items whose total weight is bounded by B and total value is maximized. knapsack: max{v: A(n,v)≤B} A(i, v) = min{ A(i-1, v), A(i-1, v-vi) + wi } A(1, v) =

if v = v1

w1

(

∞

• therwise

Scaling & Rounding

Instance: n items i=1,2, ..., n;

weights w1, w2, ..., wn ∈ ℤ+; values v1, v2, ..., vn ∈ ℤ+;

knapsack capacity B ∈ ℤ+; Set k = ; for i=1,2, ..., n, let v’i = ⌊vi / k⌋; return the knapsack solution found by dynamic programming with new values v’i ;

(to be fixed)

vmax = max

1≤i≤n vi

vi :

n

k

Scaling & Rounding

Instance: n items i=1,2, ..., n;

weights w1, w2, ..., wn ∈ ℤ+; values v1, v2, ..., vn ∈ ℤ+;

knapsack capacity B ∈ ℤ+; Set k = ; for i=1,2, ..., n, let v’i = ⌊vi / k⌋; return the knapsack solution found by dynamic programming with new values v’i ;

(to be fixed)

time complexity: O(n V’) where V’ = ∑i v’i = ∑i⌊vi / k⌋ = O(V/k) = O(nV/k) and V = ∑i vi

jvi k k

Instance: n items i=1,2, ..., n;

weights w1, w2, ..., wn ∈ ℤ+; values v1, v2, ..., vn ∈ ℤ+;

knapsack capacity B ∈ ℤ+; Set k = ; for i=1,2, ..., n, let v’i = ⌊vi / k⌋; return the knapsack solution found by dynamic programming with new values v’i ;

(to be fixed)

S*: optimal knapsack solution of the original instance S: the solution returned by the algorithm

(optimal solution of the scaled instance)

OPT = X

i∈S∗

vi

SOL = X

i∈S

vi jvi k k X

i∈S

X

i∈S∗

≥

≥ k X

i∈S

jvi k k ≥ k X

i∈S∗

jvi k k

= k X

i∈S∗

vi k

≤ k X

i∈S∗

⇣jvi k k + 1 ⌘

O(nV/k) where V = ∑i vi time complexity:

≤ k X

i∈S∗

jvi k k + nk

≥ OPT − nk

Instance: n items i=1,2, ..., n;

weights w1, w2, ..., wn ∈ ℤ+; values v1, v2, ..., vn ∈ ℤ+;

knapsack capacity B ∈ ℤ+; Set k = ; for i=1,2, ..., n, let v’i = ⌊vi / k⌋; return the knapsack solution found by dynamic programming with new values v’i ;

(to be fixed)

OPT: optimal value of the original instance SOL: value of the solution returned by the algorithm

SOL

WLOG:

OPT ≥ vmax = max

1≤i≤n vi

≤ nvmax O(nV/k) where V = ∑i vi time complexity:

≥ OPT − nk

≥ 1 − nk OPT ≥ 1 − nk

vmax

SOL OPT

Instance: n items i=1,2, ..., n;

weights w1, w2, ..., wn ∈ ℤ+; values v1, v2, ..., vn ∈ ℤ+;

knapsack capacity B ∈ ℤ+; Set k = ; for i=1,2, ..., n, let v’i = ⌊vi / k⌋; return the knapsack solution found by dynamic programming with new values v’i ; time complexity: OPT: optimal value of the original instance SOL: value of the solution returned by the algorithm SOL OPT ≥ 1 − nk

vmax

for any 0 ≤ ε ≤ 1:

vmax = max

1≤i≤n vi

where

j✏vmax n k

O ✓n2vmax k ◆

= O ✓n3 ✏ ◆

≥ 1 − ✏

Approximation Ratio

Optimization problem:

• instance I:

OPT(I) =

• ptimum of instance I
• algorithm A:

SOLA(I) =

value returned by A on instance I

minimization:

approximation ratio of algorithm A is α if ∀ instance I :

SOLA(I) OPT(I) ≤ α

maximization: approximation ratio of algorithm A is α

if ∀ instance I :

SOLA(I) OPT(I) ≥ α

ε-approximation: (1-ε) OPT(I) ≤ SOLA(I) ≤ (1+ε) OPT(I)

(minimization) (maximization)

returns a solution s for every instance I

Approximation Ratio

Optimization problem:

• instance I:

OPT(I) =

• ptimum of instance I

SOLA(ε, I) =

value returned by A on instance I

(1-ε) OPT(I) ≤ SOLA(ε, I) ≤ (1+ε) OPT(I)

(minimization) (maximization)

• algorithm A: returns a solution s for every instance I and 0≤ ε ≤1

and ε

• A is a Polynomial-Time Approximation Scheme (PTAS) if:

∀0≤ ε ≤1, A returns in polynomial time and

• A is a Fully Polynomial-Time Approximation Scheme (FPTAS) if:

furthermore, A returns in time Poly(1/ε, n) where n = |I|

(in binary code)

Instance: n items i=1,2, ..., n;

weights w1, w2, ..., wn ∈ ℤ+; values v1, v2, ..., vn ∈ ℤ+;

knapsack capacity B ∈ ℤ+; Set k = ; for i=1,2, ..., n, let v’i = ⌊vi / k⌋; return the knapsack solution found by dynamic programming with new values v’i ; time complexity: SOL OPT for any 0 ≤ ε ≤ 1:

vmax = max

1≤i≤n vi

where

j✏vmax n k

≥ 1 − ✏

O ✓n3 ✏ ◆

FPTAS

approximation ratio: Are FPTASs the “best” approximation algorithms?

Bin Packing

Instance: n items i=1,2, ..., n; with sizes s1, s2, ..., sn ∈ ℤ+; Find a packing of the n items into smallest number

• f bins with capacity B ∈ ℤ+.

items: bins:

Bin Packing

Instance: n items i=1,2, ..., n; with sizes s1, s2, ..., sn ∈(0, 1]; Find a packing of the n items into smallest number

• f unit-sized bins.

items: bins:

Bin Packing

Instance: n items i=1,2, ..., n; with sizes s1, s2, ..., sn ∈(0, 1]; Find a φ: [n]→[m] with smallest m such that ∀j ∈ [m], ∑i: φ(i) = j si ≤ 1. items: bins:

First Fit

• NP-hard.

Instance: n items i=1,2, ..., n; with sizes s1, s2, ..., sn ∈(0, 1]; Find a packing of the n items into smallest number

• f unit-sized bins.

FirstFit Initially k=1; for i=1,2, ..., n item i joins the first bin among 1,2, ..., k in which it fits; if item i can fit into none of these k bins

• pen a new bin k++ and item i joins it;

FirstFit Initially k=1; for i=1,2, ..., n item i joins the first bin among 1,2, ..., k in which it fits; if item i can fit into none of these k bins

• pen a new bin k++ and item i joins it;

items:

Observation: All but at most one bin are more than half full. Instance: n items with sizes s1, s2, ..., sn ∈(0, 1]; Pack them into smallest number of unit-sized bins. OPT ≥ ∑i si ∑i si > (SOL - 1) / 2 SOL - 1 < 2 ∑i si ≤ 2 OPT SOL ≤ 2 OPT FirstFit Initially k=1; for i=1,2, ..., n item i joins the first bin among 1,2, ..., k in which it fits; if item i can fit into none of these k bins

• pen a new bin k++ and item i joins it;

Instance: n items with sizes s1, s2, ..., sn ∈(0, 1]; Pack them into smallest number of unit-sized bins. FirstFit Initially k=1; for i=1,2, ..., n item i joins the first bin among 1,2, ..., k in which it fits; if item i can fit into none of these k bins

• pen a new bin k++ and item i joins it;

Observation: All but at most one bin are more than (1-γ) full. ∑i si > (1-γ)(SOL - 1) SOL ≤ OPT / (1-γ) + 1 Assumption: If all items are small, si < γ < 0.5

} γ } γ

Instance: n items with sizes s1, s2, ..., sn ∈(0, 1]; Pack them into smallest number of unit-sized bins. Input: n numbers x1, x2, ..., xn ∈ ℤ+. Determine whether ∃ a partition of {1, 2, ... , n} into A and B such that ∑i∈A xi = ∑i∈B xi.

Theorem

Unless P=NP, there is no poly-time approximation algorithm for bin packing with approximation ratio <3/2. reduction from the partition problem: n items with sizes s1, s2, ..., sn where si = 2xi / ∑j xj ∃ a packing into 2 unit-sized bins “yes” partition problem “no” all packings use ≥3 unit-sized bins

reduction

Instance: n items with sizes s1, s2, ..., sn ∈(0, 1]; Pack them into smallest number of unit-sized bins. It is NP-hard to distinguish between:

• the instances with OPT = 2;
• the instances with OPT ≥ 3.

FirstFitDecreasing (FFD) Sort items in non-increasing order of sizes; run FirstFit;

Theorem

Unless P=NP, there is no poly-time approximation algorithm for bin packing with approximation ratio <3/2. FFD returns a packing into ≤11/9 OPT +1 bins (1+ε) OPT +1?

Dynamic Programming

Instance: n items with sizes s1, s2, ..., sn ∈(0, 1]; Pack them into smallest number of unit-sized bins. Assumption:

• |{s1, s2, ..., sn}| = k

n1, n2, ..., nk where ∑j nj = n There are exactly nj items of size s(j) for j = 1,2, ... , k. There are k distinct sizes: s(1), s (2), ..., s(k) Bins(i1, i2, . . . , ik) Let = minimum # of bins to pack: i1× items of size s(1) i2× items of size s(2) ik× items of size s(k) OPT = Bins(n1, n2, . . . , nk)

Dynamic Programming

Instance: n items with sizes s1, s2, ..., sn ∈(0, 1]; Pack them into smallest number of unit-sized bins. Assumption:

• |{s1, s2, ..., sn}| = k

= minimum # of bins to pack: nj× items of size s(j), 1≤ j ≤ k k distinct sizes: s(1), s (2), ..., s(k) min

~ x=(x1,...,xk) Bins(x1,...,xk)=1

Bins(i1 − x1, . . . , ik − xk) Recursion:

enumerable in time O(nk)

OPT = Bins(n1, n2, . . . , nk)

Bins(i1, i2, . . . , ik) = 1+

Dynamic programming: table size O(nk) time complexity O(n2k)

si :

1

Grouping & Rounding

Instance I: n items with sizes s1, s2, ..., sn ∈(0, 1]; Instance Ifinite: n items with sizes s1, s2, ..., sn ∈(0, 1] where |{s1, s2, ..., sn}| = k

• Sort items in non-decreasing order: s1 ≤ s2 ≤ … ≤ sn
• Partition them into k groups, each with ≤⌈n/k⌉ items.
• Round up the size of each item to the size of the

largest item in its group. linear grouping:

Instance I: n items with sizes s1, s2, ..., sn ∈(0, 1];

≤ ⌈n/k⌉ items

n

k groups

• any packing of Ifinite must be a feasible packing of I : OPT(I) ≤ OPT(Ifinite)
• consider rounding down version J of Ifinite :

1

si : Ifinite: J:

OPT(J) ≤ OPT(I) any packing of J corresponds to a packing of Ifinite except for the last group OPT(Ifinite) ≤ OPT(J)

+ ⌈n/k⌉ si :

1

OPT(I) ≤ OPT(Ifinite) ≤ OPT(I) + ⌈n/k⌉ Lemma:

OPT(I) ≤ OPT(Ifinite) ≤ OPT(I) + ⌈n/k⌉ Lemma: Instance I: n items with sizes s1, s2, ..., sn ∈(0, 1]; = SOL of DP in time O(n2k) ≤ 1 + ✏ ? SOL OPT  1 + dn/ke OPT

≤ ⌈n/k⌉ items

n

k groups

1

si : si :

1

Ifinite: J:

Assumption:

• ∀1≤ i ≤ n: si ≥ γ

each bin contains ≤ 1/γ items OPT ≥ γn Instance I: n items with sizes s1, s2, ..., sn ∈(0, 1];

≤ ⌊n/k⌋ items

n

k groups linear grouping

si :

1

SOL OPT  1 + dn/ke OPT ≤ 1 + 2 γk Ifinite:

Instance I: n items with sizes s1, s2, ..., sn ∈(0, 1]; Instance Ibig: n items with sizes s1, s2, ..., sn ∈(0, 1] where si ≥ γ for all 1≤ i ≤ n. m’ ≤ max{ m, OPT(I)/(1-γ) + 1 } Lemma: Given any packing of Ibig ⊆ I into m bins, using

FirstFit to pack items in I of sizes < γ, altogether uses m’ bins:

• Case.1: FirstFit does not open a new bin: m’= m
• Case.2: FirstFit opens a new bin:

All but at most one bin are more than (1-γ) full.

} γ last new bin

∑i si > (1-γ)(m’ - 1) m’ ≤ ∑i si /(1-γ) + 1 ≤ OPT(I)/(1-γ) + 1

Ibig : 0

1

si : γ

remove items of size < γ

1

si : γ Instance I: n items with sizes s1, s2, ..., sn ∈(0, 1];

1

k groups linear grouping & rounding up:

Remove all items of size < γ : I ⇒ Ibig; apply linear grouping with k groups & round up: Ibig ⇒ Ifinite; find OPT(Ifinite) by dynamic programming in O(n2k) time; pack items of size < γ by FirstFit;

Algorithm

remove items of size < γ

Instance Ibig: n items with sizes s1, s2, ..., sn ∈(0, 1] where si ≥ γ for all 1≤ i ≤ n. Ibig : Ifinite:

Remove all items of size < γ : I ⇒ I’’; apply linear grouping with k groups & round up: I’’ ⇒ I’; find OPT(I’) by dynamic programming in O(n2k) time; pack items of size < γ by FirstFit;

Algorithm m’ ≤ max{ m, OPT(I)/(1-γ) + 1 } Lemma: Given any packing of I’’ into m bins, using FirstFit

to pack items in I of sizes < γ, altogether uses m’ bins:

SOL ≤ max{ OPT(I’), OPT(I)/(1-γ) + 1 } OPT(I’’) ≤ OPT(I’) ≤ OPT(I’’) + ⌈n(I’’)/k⌉ Lemma: SOL ≤ max{ OPT(I)/(1-γ) + 1 }

OPT(I’’) + ⌈n(I’’)/k⌉,

≤ max{ OPT(I)/(1-γ) + 1 } OPT(I’’),

(1+ 2/γk)

Remove all items of size < γ : I ⇒ I’’; apply linear grouping with k groups & round up: I’’ ⇒ I’; find OPT(I’) by dynamic programming in O(n2k) time; pack items of size < γ by FirstFit;

Algorithm m’ ≤ max{ m, OPT(I)/(1-γ) + 1 } Lemma: Given any packing of I’’ into m bins, using FirstFit

to pack items in I of sizes < γ, altogether uses m’ bins:

SOL ≤ max{ OPT(I’), OPT(I)/(1-γ) + 1 } OPT(I’’) ≤ OPT(I’) ≤ OPT(I’’) + ⌈n(I’’)/k⌉ Lemma: SOL OPT(I’’) ≥ γn(I’’) large items:

(1+ 2/γk) OPT(I) ,

Remove all items of size < γ : I ⇒ I’’; apply linear grouping with k groups & round up: I’’ ⇒ I’; find OPT(I’) by dynamic programming in O(n2k) time; pack items of size < γ by FirstFit;

Algorithm m’ ≤ max{ m, OPT(I)/(1-γ) + 1 } Lemma: Given any packing of I’’ into m bins, using FirstFit

to pack items in I of sizes < γ, altogether uses m’ bins:

SOL ≤ max{ OPT(I’), OPT(I)/(1-γ) + 1 } OPT(I’’) ≤ OPT(I’) ≤ OPT(I’’) + ⌈n(I’’)/k⌉ Lemma: SOL OPT(I’’) ≥ γn(I’’) large items: ≤ max{ OPT(I)/(1-γ) + 1 } OPT(I’’) ≤ OPT(I) trivially:

choose γ = ε/2 and k = 4/ε2 ≤ (1 + ✏)OPT + 1 time complexity: nO(1/✏2) Asymptotic PTAS approximation: SOL ≤ (1 + ✏)OPT + 1 SOL ≤ max{ OPT(I)/(1-γ) + 1 }

(1+ 2/γk)OPT(I) ,

Remove all items of size < γ : I ⇒ Ibig; apply linear grouping with k groups & round up: Ibig ⇒ Ifinite; find OPT(Ifinite) by dynamic programming in O(n2k) time; pack items of size < γ by FirstFit;

Algorithm

Scheduling

n jobs m machines

makespan

Scheduling

n jobs m machines

processing time pj with

makespan

Instance: n jobs j=1, 2, ..., n each with processing time pj ∈ ℤ+. Find a schedule of n jobs to m machines that minimizes the makespan Cmax.

Instance: n jobs j=1, 2, ..., n each with processing time pj ∈ ℤ+. Find a schedule of n jobs to m machines that minimizes the makespan Cmax. Scheduling:

• The List algorithm has approximation ratio 2.
• The LPT (Longest Processing

Time) algorithm has approximation ratio 4/3.

• The problem of minimum makespan on identical

machines has a PTAS (Polynomial Time Approximation Scheme).

Instance: n items i=1,2, ..., n; with sizes p1, p2, ..., pn ∈ ℤ+; Find a packing of the n items into smallest number

• f bins with capacity B ∈ ℤ+.

Instance: n jobs j=1, 2, ..., n each with processing time pj ∈ ℤ+. Find a schedule of n jobs to m machines that minimizes the makespan Cmax. Bin packing: Scheduling: Cmax(I) = min{ B : Bins(I, B) ≤ m } Given an instance I: p1, p2, ..., pn

Instance: n jobs j=1, 2, ..., n each with processing time pj ∈ ℤ+. Find a schedule of n jobs to m machines that minimizes the makespan Cmax. Scheduling: Cmax(I) = min{ B : Bins(I, B) ≤ m } Given an instance I: p1, p2, ..., pn

L = max 8 < : max

1≤j≤n pj, 1

m

n

X

j=1

pj 9 = ; ≤ OPT ≤ 2 · L

idea for algorithm: binary search for B between [L, 2L] to find the minimum B such that Bins(I, B) ≤ m;

Cmax(I) = min{ B : Bins(I, B) ≤ m } Given an instance I: p1, p2, ..., pn idea for algorithm: binary search for B between [L, 2L] to find the minimum B such that Bins(Ifinite, B) ≤ m; Ifinite: • group jobs into k intervals [B/(1+ε)t+1, B/(1+ε)t];

• round down;

to further control the time complexity:

• deal with small jobs (pj < B/(1+ε)k+1) separately;

This will give a PTAS for scheduling.