Sequential and Parallel Algorithms for Mixed Packing and Covering - - PowerPoint PPT Presentation

Sequential and Parallel Algorithms for Mixed Packing and Covering

(from FOCS 2001)

Neal E. Young UC Riverside

2

Marek is being punished. He can eat only bacon, beans, and beets! Can Marek get enough of what he needs without getting too much of what he doesn’t?

3

Nutrition Facts

Serving Size 4 pc Servings per container 10 Amount Per Serving Calories 110 Calories from Fat 80 % Daily Value Total Fat 10g 30 %

Saturated Fat 9g 50 % Cholesterol 3mg 10 % Sodium 500mg 40 % Total Carbohydrates 0g 0 % Dietary Fiber 0g 0 % Sugars 0g Protein 5g 30 % Vitamin A 0% Vitamin B 7% Vitamin C 2% Iron 16%

Bacon

Nutrition Facts

Serving Size 1 cup Servings per container 2 Amount Per Serving Calories 200 Calories from Fat 16 % Daily Value Total Fat 2g 5 %

Saturated Fat 0g 0 % Cholesterol 0mg 0 % Sodium 0 mg 0 % Total Carbohydrates 6g 20 % Dietary Fiber 2g 15 % Sugars 0g Protein 6g 35 % Vitamin A 40% Vitamin B 0% Vitamin C 22% Iron 2%

Beans

Nutrition Facts

Serving Size 3 oz Servings per container 5 Amount Per Serving Calories 180 Calories from Fat 0 % Daily Value Total Fat 0g 0 %

Saturated Fat 0g 0 % Cholesterol 3mg 15 % Sodium 30mg 2 % Total Carbohydrates 8g 30 % Dietary Fiber 2g 15 % Sugars 6g Protein 0g 0 % Vitamin A 40% Vitamin B 52% Vitamin C 26% Iron 3%

Beets

4

protein: 30 bacon + 35 bean > 100 vitamin A: 40 bean + 43 beet > 100 vitamin B: 7 bacon + 52 beet > 100 vitamin C: 2 bacon + 22 bean + 26 beet > 100 fat: 30 bacon + 5 bean < 100 sugar: 15 bean + 37 beet < 100 salt: 40 bacon + 2 beet < 100 cholesterol: 10 bacon + 10 bean + 15 beet < 100

constraints

unknowns bacon, bean, beet

1 serving beans has 35%

• f the RDA of protein

5

protein: 30 bacon + 35 bean > (1-ε)100 vitamin A: 40 bean + 43 beet > (1-ε)100 vitamin B: 7 bacon + 52 beet > (1-ε)100 vitamin C: 2 bacon + 22 bean + 26 beet > (1-ε)100 fat: 30 bacon + 5 bean < (1+ε)100 sugar: 15 bean + 37 beet < (1+ε)100 salt: 40 bacon + 2 beet < (1+ε)100 cholesterol: 10 bacon + 10 bean + 15 beet < (1+ε)100

ε-approximate solutions:

6

Bibliography

[1950] von Neumann. Numerical method for determination of the value and the best strategies of a zero- sum two-person game with large numbers of strategies. [1950] Brown and von Neumann. Solutions of games by differential equations. [1952] Chernoff. A measure of asymptotic efficiency for tests of a hypothesis based on the sum of

• bservations. --- Chernoff bound, implicit use of Lmax-like functions

[1958] Ford and Fulkerson. A suggested computation for maximal multicommodity flow. [1960] Dantzig and Wolfe. Decomposition principle for linear programs. [1962] Benders. Partitioning procedures for solving mixed-variables programming problems. [1971] Held and Karp. The traveling salesman problem and minimum spanning trees. [1977] Khachiyan. Convergence rate of the game processes for solving matrix games. ... [1979] Shapiro. A survey of Lagrangean techniques for discrete optimization. Annals of Discrete Mathematics, 5:113--138, 1979.

7

Bibliography

Grigoriadis and Khachiyan. A sublinear-time randomized approximation algorithm for matrix games. OR Research Letters, 1995. An exponential-function reduction method for block-angular convex programs. Networks, 1995.

• Young. Randomized rounding without solving the linear program. SODA, 1995.

Karger and Plotkin. Adding multiple cost constraints to combinatorial optimization problems, with applications to multicommodity flows. STOC, 1995. Grigoriadis and Khachiyan. Coordination complexity of parallel price-directive decomposition. MOR, 1996. Approximate minimum-cost multicommodity flows in o(knm/ε^2) time. Math. Programming, 1996. Garg and Konemann. Faster and simpler algorithms for multicommodity flow and other fractional packing problems. FOCS, 1998. --- variable-size increments

• Konemann. Fast Combinatorial Algorithms for Packing and Covering Problems

PhD thesis, Max-Planck-Institute for Informatik, 2000. --- dropping met covering constraints

• Fleischer. Approximating fractional multicommodity flow independent of the number of commodities.

SIAM J. Discrete Math, 2000. --- partitioning increments into phases

8 30 bacon + 35 bean > 100 40 bean + 43 beet > 100 7 bacon + 52 beet > 100 2 bacon + 22 bean + 26 beet > 100 30 bacon + 5 bean < 100 15 bean + 37 beet < 100 40 bacon + 2 beet < 100 10 bacon + 10 bean + 15 beet < 100

linear program

fat sugar salt chol. vit A vit B vit C prot.

beets

?

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

beans

?

bacon

?

9

fat? sugar? salt? chol? vit A? vit B? vit C? prot?

beets

1

30 7 2 35 40 22 26 52 43 30 40

10

5 15 10 15 2 37

beans bacon

One serving of beets?

10

0% 37% 2% 15% 43% 52% 26% 0%

beets

1

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

beans bacon

After one serving of beets

RDA vit A RDA vit B RDA vit C RDA prot. RDA fat RDA sugar RDA salt RDA chol.

11

0% 37% 2% 15% 43% 52% 26% 0%

beets

1

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

beans bacon

get protein avoid sugar

Greedy approach: Get protein, avoid sugar … … eat bacon

min(…) max(…)

37

12

30 37 42 25 43 59 28 30

beets

1

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

beans bacon

1

get vitamin C avoid salt

Get vitamin C, avoid salt … … eat beets

min(…)

28

max(…)

42

13

30 74 44 40 86 111 54 30

beets

2

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

beans bacon

1

Protein low Sugar high

Get protein, avoid sugar … … eat bacon

min(…)

30

max(…)

74

14

60 74 84 50 86 118 56 60

beets

2

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

beans bacon

2

Vitamin C low Salt high

Get vitamin C, avoid salt … … eat beans

min(…)

56

max(…)

84

15

65 89 84 60 126 118 78 95

beets

2

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

beans

1

bacon

2

vitamin C low Sugar high

Get vitamin C, avoid sugar … … eat beans?

min(…)

78

max(…)

89

16

69 100 84 67 156 118 94.5 121

beets

2

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

beans

1.75

bacon

2

vitamin C low

Get vitamin C, completely avoid sugar… … eat bacon?

sugar maxed out

min(…)

94.5

max(…)

100

17

80.7 100 100 71 156 121 95.3 133

beets

2

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

beans

1.75

bacon

2.4

vitamin C low

Get vitamin C, completely avoid sugar and salt … … stuck!

sugar maxed out salt maxed out

min(…)

96.6

max(…)

100

18

Making a greedy approach work

 Balance all needs in each step.  Take small bites.

19

Balancing all needs… Smooth approximations of Max() and Min()

20

Lmax(x,-x) Max(x,-x)

21

lmax < max + ln m

ln(m)

22

Change in Lmax() when inputs change:

gradient estimate is ε-approximate within ε-neighborhood

+ε Lmax true g true < (1+ε)g g = change estimated by gradient

23 beets

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

beans bacon

Algorithm: use Lmin and Lmax instead of min and max. Choose increments so Lmin increases by > (1-ε) times as much as Lmax.

Lmin(…)

• 1.38

Lmax(…)

1.38

initially ln m

24

Choose increments so Lmin increases by > (1-ε) times as much as Lmax. Stop when Lmin > ln(m)/ε (= 13.8) to get ε-approximate solution:

30 bacon + 35 bean > ln(m)/ε 40 bean + 43 beet > 13.8 7 bacon + 52 beet > 13.8 2 bacon + 22 bean + 26 beet > 13.8 30 bacon + 5 bean < ln(m) + [ln(m)/ε + ln(m)]/(1-ε) = (1+O(ε))ln(m)/ε 15 bean + 37 beet < (1+O(ε))13.8 40 bacon + 2 beet < (1+O(ε))13.8 10 bacon + 10 bean + 15 beet < (1+O(ε))13.8

target

25

Use gradients to estimate increase in Lmin and Lmax.

beets

+δ ?

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

beans bacon Lmin(…)

+30.2δ

Lmax (…)

+13.5δ

Variable ok to raise if est. increase in lmin > .9 est. increase in lmax

Partial derivative of lmin w.r.t. beets is 30.2 Estimate is ε-accurate if inputs to lmin change < ε

26 beets

• k

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

beans

+δ?

bacon

Beets are ok, what about beans?

Lmin(…)

+24.2δ

Lmax(…)

+7.5δ

27 beets

• k

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

• k

beans bacon

+δ? What about bacon?

Lmin(…)

+9.7δ

Lmax(…)

+20δ

28 beets

• k

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

• k

beans bacon

No!

Beets and beans are okay to raise, but not bacon.

Lmin(…)

• 1.38

Lmax(…)

1.38

29

Raise beets. Now: by how much? For gradient estimates to be ε-accurate, need inputs to lmin and lmax to change by < ε

beets

+δ ?

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

beans bacon Lmin(…)

+30.2δ

Lmax (…)

+13.5δ

Partial derivative of lmin w.r.t. beets is 30.2 Estimate is ε-accurate if inputs to lmin change < ε

30

0.07 0.00 0.03 0.08 0.1 0.05

beets

.002

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

beans bacon

Raise beets just enough so some constraint increases by ε. (ε = 0.1)

Lmin(…)

• 1.33

Lmax(…)

1.41

Up .05 Up .03

Vitamin B up 0.1

31

0.07 0.00 0.03 0.08 0.1 0.05

beets

.002

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

beans bacon

Repeat. Check variables… and raise ok one so some constraint increases by ε

Lmin(…)

• 1.33

Lmax(…)

1.41

• k
• k

no

32

1.06 0.05 0.43 1.24 1.5 0.75

beets

.029

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

beans bacon Lmin(…)

• 0.69

Lmax(…)

1.87

no

• k

no Fifteen (1/520)-servings of beets later… Up .70 Up .49

33

0.42 2.34 0.06 1.28 4.64 1.5 2.62 3.0

beets

.029

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

.085

beans bacon Lmin(…)

1.03

Lmax(…)

2.81

no

• k
• k

and thirty-four (1/400)-servings of beans… Up 2.41 Up 1.42

34

4.65 8.46 5.31 5.39 13.9 8.97 8.25 10.0

beets

.155

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

.180

beans bacon

.125

Lmin(…)

7.73

Lmax(…)

8.57

After 204 rounds: (recall target = 13.8)

35

4.65 8.46 5.31 5.39 13.9 8.97 8.25 10.0

beets

.155

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

.180

beans bacon

.125

Lmin(…)

7.76

Lmax(…)

8.57

Delete covering constraints as they are satisfied

36

335 total rounds until lmin > target:

15 beet 34 bean 5 bacon 17 beet 12 bean 15 bacon 17 beet 12 bean 15 bacon 16 beet 12 bean 15 bacon 16 beet 12 bean 14 bacon 15 beet 13 bean 15 bacon 18 beet 9 bean 18 bacon 8 beet 10 bean

37

64 98 73 66 168 100 100 136

beets

1.69

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

2.39

beans bacon

1.74 At end, scale up by 100/target:

38

Algorithm

• 1. x = (0,0,…,0)
• 2. target = ln(m)/ε
• 3. Until lmin > target do:
• 4. Let x = x + δ, where vector δ satisfies:
• 5. δ[j]>0 only if ok to raise variable x[j],
• 6. and some constraint increases by ε.
• 7. Delete any satisfied covering constraints.
• 8. Return x, appropriately scaled.

39

Correctness

 Is there always a variable to raise?  Does the algorithm terminate?

40

Lemma: If the constraints can be satisfied, there is always a variable to raise.

Suppose there is a feasible solution x*: <100 <100 <100 <100 >100 >100 >100 >100

x[1] x*[1]

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

x*[2] x[2] x[3] x*[3]

At most 100 At least 100

41

Lemma: If the constraints can be satisfied, there is always a variable to raise.

Take vector δ = ε x*/100. <ε <ε <ε <ε >ε >ε >ε >ε

x[1] δ[1]

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

δ[2] x[2] x[3] δ[3]

42

Lemma: If the constraints can be satisfied, there is always a variable to raise.

Add δ to current x. Lmin increases by as much as lmax. +ε +ε +ε +ε +ε +ε +ε +ε

x[1] x+δ[1]

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

x+δ[2] x[2] x[3] x+δ[3] Lmin(…)

+ε

Lmax(…)

+ε

Raise by ε

• r less

Raise by ε

• r more

QED

43

Lemma: There are O(m log(m)/ε ) increments

Each increment increases at least one constraint by ε. +? +? +? +ε +? +? +? +?

x[1] +δ[1]

30 7 2 35 40 22 26 52 43 30 40 10 5 15 10 15 2 37

+δ[2] x[2] x[3] +δ[3]

Constraint < O(target) at end, so at most O(target/ε) increments per constraint. target = O(log(m)/ε)

2

QED

44

Summary

Solutions meet constraints within 1+O(ε) factor. O(m log(m)/ε ) linear-time increments. Faster implementations possible.

2