The Simplex Method Marco Chiarandini Department of Mathematics - PowerPoint PPT Presentation

DM545 Linear and Integer Programming Lecture 2 The Simplex Method Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination Outline Simplex Method 1. Definitions and Basics 2. Fundamental Theorem of LP 3. Gaussian Elimination 4. Simplex Method Standard Form Basic Feasible Solutions Algorithm 2

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination Mathematical Model Simplex Method Graphical Representation: Machines/Materials A and B x 2 Products 1 and 2 max 6 x 1 + 8 x 2 5 x 1 + 10 x 2 ≤ 60 4 x 1 + 4 x 2 ≤ 40 x 1 ≥ 0 ≥ x 2 0 5 x 1 + 10 x 2 ≤ 60 x 1 4 x 1 + 4 x 2 ≤ 40 6 x 1 + 8 x 2 = 16 3

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination Outline Simplex Method 1. Definitions and Basics 2. Fundamental Theorem of LP 3. Gaussian Elimination 4. Simplex Method Standard Form Basic Feasible Solutions Algorithm 4

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination Linear Programming Simplex Method Abstract mathematical model: Decision Variables Criterion Constraints max / min c T · x c ∈ R n objective func. � A ∈ R m × n , b ∈ R m constraints A · x b x ∈ R n , 0 ∈ R n x ≥ 0 ◮ Any vector x ∈ R n satisfying all constraints is a feasible solution. ◮ Each x ∗ ∈ R n that gives the best possible value for c T x among all feasible x is an optimal solution or optimum ◮ The value c T x ∗ is the optimum value 5

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination In Matrix Form Simplex Method max c 1 x 1 + c 2 x 2 + c 3 x 3 + . . . + c n x n = z ≤ s.t. a 11 x 1 + a 12 x 2 + a 13 x 3 + . . . + a 1 n x n b 1 ≤ a 21 x 1 + a 22 x 2 + a 23 x 3 + . . . + a 2 n x n b 2 . . . a m 1 x 1 a m 2 x 2 a m 3 x 3 a mn x n ≤ b m + + + . . . + x 1 , x 2 , . . . , x n ≥ 0 c T = c T x max z = � c 1 � c 2 . . . c n Ax = b x ≥ 0       a 11 a 12 . . . a 1 n x 1 b 1 a 21 a 22 . . . a 2 n x 2 b 2       A =  , x =  , b = . . .       . . .       . . .     a 31 a 32 . . . a mn x n b m 6

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination Definitions Simplex Method ◮ N natural numbers, Z integer numbers, Q rational numbers, R real numbers ◮ column vector and matrices scalar product: y T x = � n i = 1 y i x i ◮ linear combination x ∈ R k x = � k x 1 , . . . , x k ∈ R i = 1 λ i x i λ = ( λ 1 , . . . , λ k ) T ∈ R k moreover: λ ≥ 0 conic combination ( � k λ T 1 = 1 i = 1 λ i = 1 ) affine combination λ ≥ 0 and λ T 1 = 1 convex combination 7

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination Definitions Simplex Method ◮ set S is linear independent if no element of it can be expressed as combination of the others Eg: S ⊆ R = ⇒ max n lin. indep. ◮ rank of a matrix for columns (= for rows) if ( m , n ) -matrix has rank = min { m , n } then the matrix is full rank if ( n , n ) -matrix is full rank then it is regular and admits an inverse ◮ G ⊆ R n is an hyperplane if ∃ a ∈ R n \ { 0 } and α ∈ R : G = { x ∈ R n | a T x = α } ◮ H ⊆ R n is an halfspace if ∃ a ∈ R n \ { 0 } and α ∈ R : H = { x ∈ R n | a T x ≤ α } ( a T x = α is a supporting hyperplane of H ) 8

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination Definitions Simplex Method ◮ a set S ⊂ R is a polyhedron if ∃ m ∈ Z + , A ∈ R m × n , b ∈ R m : i = 1 { x ∈ R n | A i · x ≤ b i } P = { x ∈ R | Ax ≤ b } = ∩ m ◮ a polyhedron P is a polytope if it is bounded: ∃ B ∈ R , B > 0: p ⊆ { x ∈ R n |� x �≤ B } ◮ Theorem: every polyhedron P � = R n is determined by finitely many halfspaces 9

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination Definitions Simplex Method ◮ General optimization problem: max { ϕ ( x ) | x ∈ F } , F is feasible region for x ◮ If A and b are rational numbers, P = { x ∈ R n | Ax ≤ b } is a rational polyhedron ◮ convex set: if x , y ∈ P and 0 ≤ λ ≤ 1 then λ x + ( 1 − λ ) y ∈ P ◮ convex function if its epigraph { ( x , y ) ∈ R 2 : y ≥ f ( x ) } is a convex set or f : X → R , if ∀ x , y ∈ X , λ ∈ [ 0 , 1 ] it holds that f ( λ x + ( 1 − λ ) y ) ≤ λ f ( x ) + ( 1 − λ ) f ( y ) 10

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination Definitions Simplex Method ◮ Given a set of points X ⊆ R n the convex hull conv ( X ) is the convex linear combination of the points x i ∈ X ; λ 1 , λ 2 , . . . , λ n ≥ 0 and � conv ( X ) = { λ 1 � x 1 + λ 2 x 2 + . . . + λ n x n | � i λ i = 1 } 11

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination Definitions Simplex Method ◮ A face of P is F = { x ∈ P | ax = α } . Hence F is either P itself or the intersection of P with a supporting hyperplane. It is said to be proper if F � = ∅ and F � = P . ◮ A point x for which { x } is a face is called a vertex of P and also a basic solution of Ax ≤ b ◮ A facet is a maximal face distinct from P cx ≤ d is facet defining if cx = d is a supporting hyperplane of P 12

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination Linear Programming Problem Simplex Method Input: a matrix A ∈ R m × n and column vectors b ∈ R m , c ∈ R n Task: 1. decide that { x ∈ R n ; Ax ≤ b } is empty (prob. infeasible), or 2. find a column vector x ∈ R n such that Ax ≤ b and c T x is max, or 3. decide that for all α ∈ R there is an x ∈ R n with Ax ≤ b and c T x > α (prob. unbounded) 1. F = ∅ 2. F � = ∅ and ∃ solution 1. one solution 2. infinite solution 3. F � = ∅ and � ∃ solution 13

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination Linear Programming and Linear Algebra Simplex Method ◮ Linear algebra: linear equations (Gaussian elimination) ◮ Integer linear algebra: linear diophantine equations ◮ Linear programming: linear inequalities (simplex method) ◮ Integer linear programming: linear diophantine inequalities 14

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination Outline Simplex Method 1. Definitions and Basics 2. Fundamental Theorem of LP 3. Gaussian Elimination 4. Simplex Method Standard Form Basic Feasible Solutions Algorithm 15

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination Fundamental Theorem of LP Simplex Method Theorem (Fundamental Theorem of Linear Programming) Given: min { c T x | x ∈ P } where P = { x ∈ R n | Ax ≤ b } If P is a bounded polyhedron and not empty and x ∗ is an optimal solution to the problem, then: ◮ x ∗ is an extreme point (vertex) of P, or ◮ x ∗ lies on a face F ⊂ P of optimal solution Proof: ◮ assume x ∗ not a vertex of P then ∃ a ball around it still in P . Show that a point in the ball has better cost ◮ if x ∗ is not a vertex then it is a convex combination of vertices. Show that all points are also optimal. 16

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination Simplex Method Implications: ◮ the optimal solution is at the intersection of hyperplanes supporting halfspaces. ◮ hence finitely many possibilities � n ◮ Solution method: write all inequalities as equalities and solve all � m systems of linear equalities ( n # variables, m # constraints) ◮ for each point we need then to check if feasible and if best in cost. ◮ each system is solved by Gaussian elimination 17

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination Simplex Method Simplex Method 1. find a solution that is at the intersection of some n hyperplanes 2. try systematically to produce the other points by exchanging one hyperplane with another 3. check optimality, proof provided by duality theory 18

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination Demo Simplex Method 19

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination Outline Simplex Method 1. Definitions and Basics 2. Fundamental Theorem of LP 3. Gaussian Elimination 4. Simplex Method Standard Form Basic Feasible Solutions Algorithm 20

Definitions and Basics Fundamental Theorem of LP Gaussian Elimination Gaussian Elimination Simplex Method 1. Forward elimination reduces the system to triangular (row echelon) form (or degenerate) elementary row operations (or LU decomposition) 2. back substitution Example: 2 x + y − z = 8 ( I ) − 3 x − y + 2 z = − 11 ( II ) − 2 x + y + 2 z = − 3 ( III ) 21