Linear Programming Chapter 1-2.2 Bjrn Morn 3 Convex Func- 1 - PowerPoint PPT Presentation

Linear Programming Chapter 1-2.2 Björn Morén

3 Convex Func- 1 Introductjon tjons 2 System of Linear Definitjons sets Equalitjes Definitjons func- Gauss Jordan tjons Linear Inequalitjes Propertjes

3 Convex Func- 1 Introductjon tjons 2 System of Linear Definitjons sets Equalitjes Definitjons func- Gauss Jordan tjons Linear Inequalitjes Propertjes

Linear Programming Björn Morén October 20, 2016 3 / 24 Background = x 1 + x 2 − x 3 5 = − 2 x 1 − x 2 + x 3 − 9 = x 1 + 3 x 2 − 3 x 3 7 • System of linear equalities, 2000 years ago • System of linear inequalities, 18th century • Linear programming, 20th century

1 1 -1 5 0 1 -1 1 0 2 -2 2 1 1 -1 5 0 1 -1 1 0 0 0 0 Feasible solution, one redundant row Linear Programming Björn Morén October 20, 2016 4 / 24 Repetjtjon Gauss Jordan (GJ) 1 1 -1 5 -2 -1 1 -9 1 3 -3 7

1 1 -1 5 0 1 -1 1 0 0 0 0 Feasible solution, one redundant row Linear Programming Björn Morén October 20, 2016 4 / 24 Repetjtjon Gauss Jordan (GJ) 1 1 -1 5 -2 -1 1 -9 1 3 -3 7 1 1 -1 5 0 1 -1 1 0 2 -2 2

Linear Programming Björn Morén October 20, 2016 4 / 24 Repetjtjon Gauss Jordan (GJ) 1 1 -1 5 -2 -1 1 -9 1 3 -3 7 1 1 -1 5 0 1 -1 1 0 2 -2 2 1 1 -1 5 0 1 -1 1 0 0 0 0 Feasible solution, one redundant row

1 1 -1 5 0 1 -1 1 0 2 -2 3 1 1 -1 5 0 1 -1 1 0 0 0 1 Last row shows infeasibility Linear Programming Björn Morén October 20, 2016 5 / 24 Repetjtjon Gauss Jordan (GJ) 1 1 -1 5 -2 -1 1 -9 1 3 -3 8

1 1 -1 5 0 1 -1 1 0 0 0 1 Last row shows infeasibility Linear Programming Björn Morén October 20, 2016 5 / 24 Repetjtjon Gauss Jordan (GJ) 1 1 -1 5 -2 -1 1 -9 1 3 -3 8 1 1 -1 5 0 1 -1 1 0 2 -2 3

Linear Programming Björn Morén October 20, 2016 5 / 24 Repetjtjon Gauss Jordan (GJ) 1 1 -1 5 -2 -1 1 -9 1 3 -3 8 1 1 -1 5 0 1 -1 1 0 2 -2 3 1 1 -1 5 0 1 -1 1 0 0 0 1 Last row shows infeasibility

3 Convex Func- 1 Introductjon tjons 2 System of Linear Definitjons sets Equalitjes Definitjons func- Gauss Jordan tjons Linear Inequalitjes Propertjes

Linear Programming Björn Morén October 20, 2016 7 / 24 Theorem of alternatjves for systems of linear equatjons Theorem 1. Excactly one of the following two systems has a solution. 1. Ax = b 2. π = ( π 1 , . . . , π m )

Linear Programming Björn Morén October 20, 2016 8 / 24 Memory Matrix in GJ

Linear Programming Björn Morén October 20, 2016 8 / 24 Memory Matrix in GJ

Linear Programming Björn Morén October 20, 2016 8 / 24 Memory Matrix in GJ

Linear Programming Björn Morén October 20, 2016 9 / 24 Memory Matrix in GJ: Example

Linear Programming Björn Morén October 20, 2016 9 / 24 Memory Matrix in GJ: Example

Linear Programming Björn Morén October 20, 2016 9 / 24 Memory Matrix in GJ: Example

Linear Programming Björn Morén October 20, 2016 9 / 24 Memory Matrix in GJ: Example

Linear Programming Björn Morén October 20, 2016 10 / 24 Memory Matrix in GJ: Example Last row is proof of infeasibility Where π = (-3 -5 0 -1 1) such that πA = 0 and πb = 6 � = 0

• Used in computer implementations to save memory • Similar to Dantzigs revised simplex method • Memory matrix referred to as basis inverse, denoted Linear Programming Björn Morén October 20, 2016 11 / 24 Revised GJ with Explicit Basis Inverse • ¯ A is not stored at each iteration • ¯ A can be computed: columns ¯ A .j = ¯ MA .j and rows ¯ A i. = ¯ M i. A

Linear Programming Björn Morén October 20, 2016 11 / 24 Revised GJ with Explicit Basis Inverse • ¯ A is not stored at each iteration • ¯ A can be computed: columns ¯ A .j = ¯ MA .j and rows ¯ A i. = ¯ M i. A • Used in computer implementations to save memory • Similar to Dantzigs revised simplex method • Memory matrix referred to as basis inverse, denoted B − 1

Linear Programming Björn Morén October 20, 2016 12 / 24 Revised GJ with Explicit Basis Inverse Method 1. Select pivot row i 2. Compute row i : ¯ A i. 3. If ¯ A i � = 0 , select nonzero pivot element j If ¯ A i = 0 , either row is redundant, go to 1 or problem is infeasible, method finishes. 4. Compute column j : ¯ A j. and perform pivot step 5. Stop when pivot step has been done for all rows

Linear Programming Björn Morén October 20, 2016 13 / 24 Revised GJ: Example

Linear Programming Björn Morén October 20, 2016 13 / 24 Revised GJ: Example

Linear Programming Björn Morén October 20, 2016 13 / 24 Revised GJ: Example

Linear Programming Björn Morén October 20, 2016 13 / 24 Revised GJ: Example

Linear Programming Björn Morén October 20, 2016 14 / 24 Systems of Linear Inequalitjes

In iteration 1. If is unique solution to system, terminate. 2. Let be basis for 3. If terminate 4. Otherwise, take such that for some . Find maximum such that is feasible. Linear Programming Björn Morén October 20, 2016 15 / 24 Systems of Linear Inequalitjes Start with x 0 and P 0 indices of active constraints. • If P 0 = ∅ : Select a constraint i and a point ¯ x on the constraint. If ¯ x is infeasible. Find maximum λ such that x 1 = x 0 + λ (¯ x − x 0 ) is feasible.

2. Let be basis for 3. If terminate 4. Otherwise, take such that for some . Find maximum such that is feasible. Linear Programming Björn Morén October 20, 2016 15 / 24 Systems of Linear Inequalitjes Start with x 0 and P 0 indices of active constraints. • If P 0 = ∅ : Select a constraint i and a point ¯ x on the constraint. If ¯ x is infeasible. Find maximum λ such that x 1 = x 0 + λ (¯ x − x 0 ) is feasible. In iteration r 1. If x r is unique solution to system, terminate.

3. If terminate 4. Otherwise, take such that for some . Find maximum such that is feasible. Linear Programming Björn Morén October 20, 2016 15 / 24 Systems of Linear Inequalitjes Start with x 0 and P 0 indices of active constraints. • If P 0 = ∅ : Select a constraint i and a point ¯ x on the constraint. If ¯ x is infeasible. Find maximum λ such that x 1 = x 0 + λ (¯ x − x 0 ) is feasible. In iteration r 1. If x r is unique solution to system, terminate. 2. Let { y } be basis for { A i. y = 0; i ∈ P r }

4. Otherwise, take such that for some . Find maximum such that is feasible. Linear Programming Björn Morén October 20, 2016 15 / 24 Systems of Linear Inequalitjes Start with x 0 and P 0 indices of active constraints. • If P 0 = ∅ : Select a constraint i and a point ¯ x on the constraint. If ¯ x is infeasible. Find maximum λ such that x 1 = x 0 + λ (¯ x − x 0 ) is feasible. In iteration r 1. If x r is unique solution to system, terminate. 2. Let { y } be basis for { A i. y = 0; i ∈ P r } 3. If { A i. y = 0; ∀ y, i } terminate

Linear Programming Björn Morén October 20, 2016 15 / 24 Systems of Linear Inequalitjes Start with x 0 and P 0 indices of active constraints. • If P 0 = ∅ : Select a constraint i and a point ¯ x on the constraint. If ¯ x is infeasible. Find maximum λ such that x 1 = x 0 + λ (¯ x − x 0 ) is feasible. In iteration r 1. If x r is unique solution to system, terminate. 2. Let { y } be basis for { A i. y = 0; i ∈ P r } 3. If { A i. y = 0; ∀ y, i } terminate 4. Otherwise, take ¯ y such that A i. ¯ y < 0 for some i . Find maximum λ such that x r +1 = x r + λ (¯ y − x r ) is feasible.

3 Convex Func- 1 Introductjon tjons 2 System of Linear Definitjons sets Equalitjes Definitjons func- Gauss Jordan tjons Linear Inequalitjes Propertjes

Linear Programming Björn Morén October 20, 2016 17 / 24 Convex sets Definition 1. A set K is convex if for x, y ∈ K, 0 ≤ α ≤ 1 , then z = αx + (1 − α ) y ∈ K

Linear Programming Björn Morén October 20, 2016 18 / 24 Convex functjons Jensen’s inequality Let 0 ≤ α ≤ 1 and y 1 , y 2 ∈ Γ where Γ is a convex set. Definition 2. A function g ( y ) is convex if g ( αy 1 + (1 − α ) y 2 ) ≤ αg ( y 1 ) + (1 − α ) g ( y 2 )

Linear Programming Björn Morén October 20, 2016 19 / 24 Concave functjons Let 0 ≤ α ≤ 1 and y 1 , y 2 ∈ Γ where Γ is a convex set. Definition 3. A function h ( y ) is concave if h ( αy 1 + (1 − α ) y 2 ) ≥ αh ( y 1 ) + (1 − α ) h ( y 2 )

Linear Programming Björn Morén October 20, 2016 20 / 24 Gradient support inequality Theorem 2. Let g ( y ) be a real-valued differentiable real-valued function defined on R n . Then g ( y ) is convex iff g ( y ) ≥ g (¯ y ) + ∇ g (¯ y )( y − ¯ y )

Linear Programming Björn Morén October 20, 2016 21 / 24 Gradient support inequality

Linear Programming Björn Morén October 20, 2016 22 / 24 Differentjable functjon Theorem 3. Let g ( y ) be a real-valued differentiable real-valued function defined on R n . Then g ( y ) is convex iff ( ∇ g ( y 2 ) − ∇ g ( y 1 ))( y 2 − y 1 ) ≥ 0