Programming for the 0/1 Knapsack Problem Nirmal Prajapati Sanjay - - PowerPoint PPT Presentation

Revisiting Sparse Dynamic Programming for the 0/1 Knapsack Problem

Tarequl Islam Sifat Tarequl.Sifat@colostate.edu Corespeq Inc Fort Collins, Colorado, USA Nirmal Prajapati prajapati@lanl.gov Los Alamos National Laboratory Los Alamos, New Mexico, USA Sanjay Rajopadhye Sanjay.Rajopadhye@colostate.edu Department of Computer Science Colorado State University Fort Collins, Colorado, USA

0/1 Knapsack Problem Statement

◼ Given a set of 𝑂 items numbered from 1 up to 𝑂, each with a weight

𝑥𝑗 and a profit 𝑞𝑗, along with maximum capacity 𝐷, we must 𝑛𝑏𝑦𝑗𝑛𝑗𝑨𝑓 ෍

𝑗=1 𝑂+1

𝑞𝑗𝑦𝑗 𝑡𝑣𝑐𝑘𝑓𝑑𝑢 𝑢𝑝 ෍

𝑗=1 𝑂+1

𝑥𝑗𝑦𝑗 < 𝐷 𝑏𝑜𝑒 𝑦𝑗 ∈ {0,1}

2

Sparse DP Algorithm for solving 0/1- Knapsack Problem

◼ A “sparse” KPDP algorithm (SKPDP) has been known for a while. ◼ Conventional KPDP algorithm generates a DP table that contains

many repeated values.

◼ SKPDP does not calculate the repeated values in the DP table. ◼ So far there has been no quantitative analysis of its benefits.

3

Contributions

◼ Quantitative analysis of Sequential SKPDP ◼ Exploration of two parallelization techniques for SKPDP and their

performance analysis

◼ Comparison of SKPDP with Branch-and-Bound

4

Problem Instance Generation For Quantitative Analysis

◼ Uncorrelated : 𝑞𝑗 = 𝑠𝑏𝑜𝑒𝑝𝑛(𝑠𝑏𝑜𝑕𝑓) ◼ Weakly Correlated : 𝑞𝑗 = 𝛽𝑥𝑗 + 𝑠𝑏𝑜𝑒𝑝𝑛(𝑠𝑏𝑜𝑕𝑓) ◼ Strongly Correlated: 𝑞𝑗 = 𝛽𝑥𝑗 + 𝛾 [Hardest Problem Instances] ◼ Subset Sum: 𝑞𝑗 = 𝑥𝑗

where 𝛽, 𝛾 are constants, 𝑞𝑗 is profit and 𝑥𝑗 is weight of each item.

5

Punch Line

◼ For KP instances with significantly large capacity than the number of

items (C >> N)

◼ If the problem instance is weakly correlated, the operation count of

SKPDP algorithm is invariant with respect to capacity (𝐷).

◼ If the problem instance is strongly correlated, the operation count of

SKPDP algorithm is exponentially less than KPDP.

6

Punch Line

7

Weakly Correlated Instances Strongly Correlated Instances

Dynamic Programming Solution

8

for ( k=1 ; k < N ; k++ ) { for ( c=0 ; c <= C ; c++ ) { if( c < weights[k] ) M[k,c] = M[k-1,c]; else M[k,c]=MAX(M[k-1,c], M[k-1,c-weights[k]]+profits[k]); } }

𝑁 𝑙, 𝑑 = ቊ 𝑁 𝑙 − 1, 𝑑 𝑗𝑔 𝑥𝑙 > 𝑑 max 𝑁 𝑙 − 1, 𝑑 , 𝑁 𝑙 − 1, 𝑑 − 𝑥𝑙 + 𝑞𝑙 𝑓𝑚𝑡𝑓

Example Problem Instance

Item No. Profits Weights 1 1 1 2 6 2 3 18 5 4 22 6 5 28 7

9

𝑂 = 5 𝐷 = 11

Dynamic Programming Table

10

𝑁 𝑙, 𝑑 = ቊ 𝑁 𝑙 − 1, 𝑑 𝑗𝑔 𝑥𝑙 > 𝑑 max 𝑁 𝑙 − 1, 𝑑 , 𝑁 𝑙 − 1, 𝑑 − 𝑥𝑙 + 𝑞𝑙 𝑓𝑚𝑡𝑓

Capacity

Item# (weight,profit)

1 2 3 4 5 6 7 8 9 10 11 1 (1,1) 1 1 1 1 1 1 1 1 1 1 1 2 (2,6) 1 6 7 7 7 7 7 7 7 7 7 3 (5,18) 1 6 7 7 18 19 24 25 25 25 25 4 (6,22) 1 6 7 7 18 22 24 28 29 29 40 5 (7,28) 1 6 7 7 18 22 28 29 34 35 40

c k

Memory Efficient KPDP

◼ Only need the current row of the DP table to calculate the next

row

◼ The whole table does not have to be stored ◼ This way we can find the optimal profit value ◼ We can find the exact solution including which items are taken in

the optimal solution by using a divide-and-conquer strategy

◼ Divide-and-Conquer strategy doubles the number of

computations, 2𝑂𝐷

◼ Reduces memory requirement by a factor of 𝑂/2

11

“Sparsity” in the current context

12

“Sparsity” in the current context

Capacity

Item# (weight,profit)

1 2 3 4 5 6 7 8 9 10 11 1 (1,1) 1 1 1 1 1 1 1 1 1 1 1 2 (2,6) 1 6 7 7 7 7 7 7 7 7 7 3 (5,18) 1 6 7 7 18 19 24 25 25 25 25 4 (6,22) 1 6 7 7 18 22 24 28 29 29 40 5 (7,28) 1 6 7 7 18 22 28 29 34 35 40

13

Item# <0,0> 1 (1,1) <0,0>,<1,1> 2 (2,6) <0,0>,<1,1>,<2,6>,<3,7> 3 (5,18) <0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25> 4 (6,22) <0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,22>,<7,24>,<8,28>,<9,29>,<11,40> 5 (7,28) <0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,22>,<7,28>,<8,29>,<9,34>,<10,35>,<11,40>

Building the Sparse Table Add-Merge-Kill

14

{1,2,3} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,19>, <7,24>, <8,25> {1,2,3,4} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,22>, <7,24>, <8,28>, <9,29>, <11,40>

When we include the 4th item (Weight: 6, Profit: 22) in our choice

<0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25>

Building the Sparse Table Add-Merge-Kill

14

{1,2,3} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,19>, <7,24>, <8,25> {1,2,3,4} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,22>, <7,24>, <8,28>, <9,29>, <11,40>

When we include the 4th item (Weight: 6, Profit: 22) in our choice

<0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25>

Building the Sparse Table Add-Merge-Kill

14

{1,2,3} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,19>, <7,24>, <8,25> {1,2,3,4} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,22>, <7,24>, <8,28>, <9,29>, <11,40>

When we include the 4th item (Weight: 6, Profit: 22) in our choice Add (6,22) to each pair,

<6,22>,<7,23>,<8,28>,<9,29>,<11,40>,<12,41>,<13,46>,<14,47>

<0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25>

Building the Sparse Table Add-Merge-Kill

14

{1,2,3} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,19>, <7,24>, <8,25> {1,2,3,4} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,22>, <7,24>, <8,28>, <9,29>, <11,40>

When we include the 4th item (Weight: 6, Profit: 22) in our choice Add (6,22) to each pair,

<6,22>,<7,23>,<8,28>,<9,29>,<11,40>

<0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25>

Building the Sparse Table Add-Merge-Kill

14

{1,2,3} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,19>, <7,24>, <8,25> {1,2,3,4} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,22>, <7,24>, <8,28>, <9,29>, <11,40>

When we include the 4th item (Weight: 6, Profit: 22) in our choice

<6,22>,<7,23>,<8,28>,<9,29>,<11,40>

<0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25>

Building the Sparse Table Add-Merge-Kill

14

{1,2,3} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,19>, <7,24>, <8,25> {1,2,3,4} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,22>, <7,24>, <8,28>, <9,29>, <11,40>

When we include the 4th item (Weight: 6, Profit: 22) in our choice

<6,22>,<7,23>,<8,28>,<9,29>,<11,40>

Building the Sparse Table Add-Merge-Kill

15

Generation of Problem Instances

◼ The fraction of objects that can fit in the knapsack on average, 1/𝜇

𝑋

𝑏𝑤𝑕 = 𝜇𝐷 𝑂

◼ The set of weights, 𝑥𝑗 is generated with a normal distribution that

has a mean of 𝑋

𝑏𝑤𝑕

◼ For weakly correlated problem instances, the correlation between

the weights and profits is controlled by a noise factor, 𝜏 𝑞𝑗 = 𝛽𝑥𝑗 + 𝑆𝑏𝑜𝑒𝑝𝑛 𝐽𝑜𝑢𝑓𝑕𝑓𝑠 𝑐𝑓𝑢𝑥𝑓𝑓𝑜 [−𝜏𝑋

𝑏𝑤𝑕, 𝜏𝑋 𝑏𝑤𝑕]

◼ For strongly correlated problem instances 𝜏 is irrelevant,

𝑞𝑗 = 𝛽𝑥𝑗 + 𝛾

16

Gain

𝐻𝑏𝑗𝑜 = 1 − 𝐽𝑢𝑓𝑠𝑏𝑢𝑗𝑝𝑜𝑡 𝑗𝑜 𝑇𝐿𝑄𝐸𝑄 𝐽𝑢𝑓𝑠𝑏𝑢𝑗𝑝𝑜𝑡 𝑗𝑜 𝐿𝑄𝐸𝑄 (= 2𝑂𝐷) The range of Gain is (-1, 1)

◼ A value of gain close to 1 means that the number of iterations

in SKPDP is insignificant compared to KPDP

◼ A value of gain close to -1 mean that we have the worst-case

scenario for SKPDP.

17

Gain

18

𝜇 = 2, 𝜏 = 0.1%

SKPDP vs KPDP

19

Weakly Correlated Instances Strongly Correlated Instances 𝜇 = 2, 𝜏 = 0.1% 𝑂 = 256, 𝜇 = 8, 𝜏 = 0.1%

Impact of 𝜏 on the sparsity

20

𝑞 = 𝛽𝑥 + 𝑆𝑏𝑜𝑒𝑝𝑛 𝐽𝑜𝑢. 𝐽𝑜 −𝜏𝑋

𝑏𝑤𝑕, 𝜏𝑋 𝑏𝑤𝑕 ; 𝜇 = 2

Impact of 𝜇 on the sparsity

21

𝑋

𝑏𝑤𝑕 = 𝜇𝐷

𝑂 ; 𝜏 = 0.1%

Impact of 𝜏 and 𝜇 on the sparsity

22

𝑂 = 210, 212 < C < 250

Parallelization of SKPDP

◼ Fine-Grained Parallelization ◼ Coarse-Grained Parallelization

23

Fine-Grained Parallelization

24

Fine-Grained Parallelization

Item# <0,0> 1 (1,1) <0,0>,<1,1> 2 (2,6) <0,0>,<1,1>,<2,6>,<3,7> 3 (5,18) <0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25> 4 (6,22) <0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,22>,<7,24>,<8,28>,<9,29>,<11,40> 5 (7,28) <0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,22>,<7,28>,<8,29>,<9,34>,<10,35>,<11,40>

25

{1,2,3} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,19>, <7,24>, <8,25> {1,2,3,4} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,22>, <7,24>, <8,28>, <9,29>, <11,40>

When we include the 4th item (Weight: 6, Profit: 22) in our choice

Remember the example used to demonstrate Add-Merge-Kill?

Fine-Grained Parallelization

26

<0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25>

4th row:

Fine-Grained Parallelization

26

<0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25>

4th row:

<5,18>,<6,19>,<7,24>,<8,25> <0,0>,<1,1>,<2,6>,<3,7>

Fine-Grained Parallelization

26

<0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25>

4th row:

<5,18>,<6,19>,<7,24>,<8,25> <0,0>,<1,1>,<2,6>,<3,7> <0,0>,<1,1>,<2,6>,<3,7>,<6,22>,<7,23>,<8,28>,<9,29> <5,18>,<6,19>,<7,24>,<8,25>,<11,40>

Fine-Grained Parallelization

26

<0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25>

4th row:

<5,18>,<6,19>,<7,24>,<8,25> <0,0>,<1,1>,<2,6>,<3,7> <0,0>,<1,1>,<2,6>,<3,7>,<6,22>,<7,23>,<8,28>,<9,29> <5,18>,<6,19>,<7,24>,<8,25>,<11,40>

At most 𝑥𝑗 pairs from the suffix of the preceding part

Fine-Grained Parallelization

26

<0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25>

4th row:

<5,18>,<6,19>,<7,24>,<8,25> <0,0>,<1,1>,<2,6>,<3,7> <0,0>,<1,1>,<2,6>,<3,7>,<6,22>,<7,23>,<8,28>,<9,29> <5,18>,<6,19>,<7,24>,<8,25>,<11,40>

At most 𝑥𝑗 pairs from the suffix of the preceding part At most 𝑥𝑗 pairs from the prefix of the following part

Fine-Grained Parallelization

26

<0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25>

4th row:

<5,18>,<6,19>,<7,24>,<8,25> <0,0>,<1,1>,<2,6>,<3,7> <0,0>,<1,1>,<2,6>,<3,7>,<6,22>,<7,23>,<8,28>,<9,29> <5,18>,<6,19>,<7,24>,<8,25>,<11,40>

At most 𝑥𝑗 pairs from the suffix of the preceding part At most 𝑥𝑗 pairs from the prefix of the following part In the worst-case extra merge-kill we must do at each point of stitching is 2𝑥𝑗

Fine-Grained Parallelization

26

<0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25>

4th row:

<5,18>,<6,19>,<7,24>,<8,25> <0,0>,<1,1>,<2,6>,<3,7> <0,0>,<1,1>,<2,6>,<3,7>,<6,22>,<7,23>,<8,28>,<9,29> <5,18>,<6,19>,<7,24>,<8,25>,<11,40> <0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,22>,<7,24>,<8,28>,<9,29>,<11,40>

Coarse-Grained Parallelization

27

Coarse-Grained Parallelization

◼ The computation occurs in batches of P rows. ◼ Every 𝑄𝑢ℎ row of the DP table is stored (completely) in the

memory.

◼ Other 𝑄 − 1 rows only stores a local buffer of size 𝑥𝑗

28

<0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25>

4rd row: k j At most 𝑥𝑗

Coarse-Grained Parallelization

◼ The local buffers are basically a FIFO stack and implemented

with a rotating array.

◼ These rotating arrays have a length of 𝑋

𝑛𝑏𝑦 = max(𝑥𝑗)

◼ Extra memory required by the implementation is 𝑄 ∗ 𝑋

𝑛𝑏𝑦

29

Performance of Coarse-Grained Parallelization

30

𝜇 = 8, 𝜏 = 0.1%

SKPDP vs BnB

31

Future work

◼ A model to predict the performance gain by SKPDP ◼ Ordering the items by weight values to maximize sparsity ◼ Choosing the best input parameters for the coarse-grained

parallelization of SKPDP

32

Conclusion

◼ SKPDP shows exponential performance improvement using

SKPDP over KPDP.

◼ Coarse-grained parallelization shows promise. ◼ For most problem instances BnB with proper heuristics does

better than SKPDP.

◼ For some hard problem instances SKPDP performs better

than BnB.

33

34

Thank you!