UNIT 2 Non-Linear Programming A NLP problem An engineering factory - - PowerPoint PPT Presentation

UNIT 2

Non-Linear Programming

A NLP problem

An engineering factory makes 4 products (PROD1 to PROD4) on the following machines: 4 grinders, 2 drills and 1 planer. Each product yields a certain contribution to profit (defined as €/unit selling price minus cost of raw materials). These quantities together with the unit production times (hours) required on each process are given in the table below. PROD 1 PROD 2 PROD 3 PROD 4 Contribution to profit 8+0.1x1 6 8 6-0.2x4 Grinding 0.5 0.7

• Drilling

0.1 0.2

• 0.3

Planing

• 0.01
• If there are 8 working hours in a day, what should the factory

produce (daily) in order to maximize the total profit?

A NLP model

Nonlinear objective function

Introduction

 In order to find the optimal solutions of an NLP

problem, we will study a special type of points called Kuhn-Tucker points.

 If our problem does satisfy some particular

conditions, we will be sure that one of these special points will be the optimal solution.

Lagrangian function

 It is a function that is built adding some terms to the

• bjective function:

) ) ,..., ( ( ) ,..., ( ) ,..., , ,..., (

1 1 1 1 1

∑

=

− + =

m i n i i i n m n

x x g b x x f x x L λ λ λ

• .f.

Lagrange multipliers RHS of the constraints constraints functions

Lagrangian function: example

9 3 . .

2 2 2

≥ = + + ≤ + + y z y x z y x t s xyz Max

y z y x z y x xyz z y x L

3 2 2 2 2 1 3 2 1

) 9 ( ) 3 ( ) , , , , , ( λ λ λ λ λ λ − − − − + − − − + =

Kuhn-Tucker (K-T) points

 For a point to be K-T, it must satisfy 4 conditions

(the K-T conditions) :

 Feasibility  Stationary point  Sign  Complementary slackness

K-T conditions

 Feasibility:

 The point must satisfy all the constraints of the problem (i.e., it must be

a feasible solution).

 Stationary point:

 The partial derivatives of the Lagrangian function with respect to the

problem variables must take value 0 at this point.

 Sign:

 If a constraint is canonical, its associated multiplier must be

• nonnegative. If it is not canonical, it must be nonpositive.

 Complementary slackness:

 For each constraint, either it is non-binding or its associated multiplier

takes value 0.

)) ( ( = − x g b λ

K-T conditions: example

9 3 . .

2 2 2

≥ = + + ≤ + + y z y x z y x t s xyz Max

     ≥ = + + ≤ + + 9 3

2 2 2

y z y x z y x

Feasibility:

K-T conditions: example

9 3 . .

2 2 2

≥ = + + ≤ + + y z y x z y x t s xyz Max

y z y x z y x xyz z y x L

3 2 2 2 2 1 3 2 1

) 9 ( ) 3 ( ) , , , , , ( λ λ λ λ λ λ − − − − + − − − + =          = − − = ∂ ∂ = − − − = ∂ ∂ = − − = ∂ ∂ 2 2 2

2 1 3 2 1 2 1

λ λ λ λ λ λ λ z xy z L y xz y L x yz x L

Stationary point:

K-T conditions: example

9 3 . .

2 2 2

≥ = + + ≤ + + y z y x z y x t s xyz Max

   ≤ ≥

3 1

λ λ

Sign:

K-T conditions: example

9 3 . .

2 2 2

≥ = + + ≤ + + y z y x z y x t s xyz Max

   = = − − − ) 3 (

3 1

y z y x λ λ

Complementary slackness:

How to proceed to check if a point is K-T?

 The best order to check the K-T conditions is the

following one:

Feasibility Complementary slackness Stationary point Sign

Checking K-T conditions: example

9 3 . .

2 2 2

≥ = + + ≤ + + y z y x z y x t s xyz Max

Is (0,3,0) a K-T point?

Feasibility:

     ≥ ≥ = + + = + + ≤ + + ≤ + + 3 9 3 9 3 3 3

2 2 2 2

y z y x z y x

  

Checking K-T conditions: example

9 3 . .

2 2 2

≥ = + + ≤ + + y z y x z y x t s xyz Max

   = = − − − ) 3 (

3 1

y z y x λ λ

   = ⋅ = = ⋅ = − − − 3 ) 3 3 (

3 3 1 1

λ λ λ λ y

3 =

λ

Complementary slackness:

Is (0,3,0) a K-T point?



Checking K-T conditions: example

9 3 . .

2 2 2

≥ = + + ≤ + + y z y x z y x t s xyz Max          = = ⋅ − − ⋅ = − − = ∂ ∂ = − − = − ⋅ − − ⋅ = − − − = ∂ ∂ = − = ⋅ ⋅ − − ⋅ = − − = ∂ ∂ 2 3 2 6 3 2 2 2 3 2

1 2 1 2 1 2 1 2 1 3 2 1 1 2 1 2 1

λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ z xy z L y xz y L x yz x L

3 2 1

= = = λ λ λ

Stationary point:

Checking K-T conditions: example

9 3 . .

2 2 2

≥ = + + ≤ + + y z y x z y x t s xyz Max

   ≤ ≥

3 1

λ λ

Sign:

 

So (0,3,0) is a K-T point, and its associated multipliers are (0,0,0).

Obtaining K-T points

 If we are not given any candidates for K-T points, we can obtain

them by solving the system of equations obtained from the K-T conditions.

 Example:

, 5 2 . .

2 2

≥ ≤ + + y x y x t s y x Max

Note that all the functions are C2 and the second constraint qualification (linearity) is satisfied, thus all the optimal solutions (if there is any) must be K-T points.

• Lagrangian function
• Feasibility
• Stationary point
• Sign
• Complementary slackness

Obtaining K-T points

, 5 2 . .

2 2

≥ ≤ + + y x y x t s y x Max

y x y x y x y x L

3 2 1 2 2 3 2 1

) 2 5 ( ) , , , , ( λ λ λ λ λ λ − − − − + + =

, , 5 2 ≥ ≥ ≤ + y x y x

2 2 , 2

3 1 2 1

= − − = − − λ λ λ λ y x

, ,

3 2 1

≤ ≤ ≥ λ λ λ

, , ) 2 5 (

3 2 1

= = = − − y x y x λ λ λ

Obtaining K-T points

 We begin with the complementary slackness conditions (if there is any):

     = = =      = = =      = − − = = − − 2 5 ) 2 5 (

3 3 2 2 1 1

y

• r

y x

• r

x y x

• r

y x λ λ λ λ λ λ

, , 5 2 , , 5 2 , , 5 2 , , 5 2 , , , , , , , ,

3 2 3 2 1 3 1 2 1 3 2 1

= = = + = = = + = = = + = = = + = = = = = = = = = = = = y x y x x y x y y x y x y x x y λ λ λ λ λ λ λ λ λ λ λ λ

Case 1: Case 2: Case 3: Case 4: Case 5: Case 6: Case 7: Case 8:

Obtaining K-T points



Then we analyze the stationary point conditions and the equality constraints of the problem (if any) for each one of the cases obtained earlier:

2 2 , 2

3 1 2 1

= − − = − − λ λ λ λ y x

, ,

3 2 1

= = = λ λ λ

Case 1: Stationary point:

, 2 2 = = ⇒    = = y x y x

Obtaining K-T points



Then we analyze the stationary point conditions and the equality constraints of the problem (if any) for each one of the cases obtained earlier:

2 2 , 2

3 1 2 1

= − − = − − λ λ λ λ y x Case 2:

, 2

3 3

= = ⇒    = − = λ λ x x

, ,

2 1

= = = y λ λ

Stationary point:

Obtaining K-T points



Then we analyze the stationary point conditions and the equality constraints of the problem (if any) for each one of the cases obtained earlier:

2 2 , 2

3 1 2 1

= − − = − − λ λ λ λ y x

, ,

3 1

= = = λ λ x

Case 3:

, 2

2 2

= = ⇒    = = − λ λ y y

Stationary point:

Obtaining K-T points



Then we analyze the stationary point conditions and the equality constraints of the problem (if any) for each one of the cases obtained earlier:

2 2 , 2

3 1 2 1

= − − = − − λ λ λ λ y x Case 4:

,

3 2 3 2

= = ⇒    = − = − λ λ λ λ

, ,

1

= = = y x λ

Stationary point:

Obtaining K-T points



Then we analyze the stationary point conditions and the equality constraints of the problem (if any) for each one of the cases obtained earlier:

2 2 , 2

3 1 2 1

= − − = − − λ λ λ λ y x

, , 5 2

3 2

= = = + λ λ y x

Case 5:

1 , 2 , 2 2 2 4 10

1 1 1

= = = ⇒    = − = − − x y y y λ λ λ

Stationary point:

Obtaining K-T points



Then we analyze the stationary point conditions and the equality constraints of the problem (if any) for each one of the cases obtained earlier:

2 2 , 2

3 1 2 1

= − − = − − λ λ λ λ y x Case 6:

20 , 10 , 5 2 10

3 1 3 1 1

− = = = ⇒    = − − = − λ λ λ λ λ x

, , 5 2

2

= = = + y y x λ

Stationary point:

Obtaining K-T points



Then we analyze the stationary point conditions and the equality constraints of the problem (if any) for each one of the cases obtained earlier:

2 2 , 2

3 1 2 1

= − − = − − λ λ λ λ y x

, , 5 2

3 =

= = + λ x y x

Case 7:

5 . 2 , 5 . 2 , 5 . 2 2 5

2 1 1 2 1

− = = = ⇒    = − = − − λ λ λ λ λ y

Stationary point:

Obtaining K-T points



Then we analyze the stationary point conditions and the equality constraints of the problem (if any) for each one of the cases obtained earlier:

2 2 , 2

3 1 2 1

= − − = − − λ λ λ λ y x Case 8:

, , 5 2 = = = + y x y x

infeasible

Stationary point:

Obtaining K-T points



Now we check feasibility and sign conditions for the points and multipliers obtained:

Point Multipliers Cases 1,2,3,4 (0,0) (0,0,0) Case 5 (1,2) (2,0,0) Case 6 (5,0) (10,0,-20) Case 7 (0,2.5) (2.5,-2.5,0)

, , 5 2 ≥ ≥ ≤ + y x y x

Feasibility: , ,

3 2 1

≤ ≤ ≥ λ λ λ Sign:



All the obtained points are K-T points.

K-T points in CP problems

 In CP points there are no complementary slackness

• r sign conditions.

 We only have to state and solve the equations

system formed by the stationary point and feasibility conditions.

Example

21 . . 10 4 2

2 2

= + + + + + y x t s y x y x Min

) 21 ( 10 4 2 ) , , (

1 2 2 1

y x y x y x y x L − − + + + + + = λ λ

       = − + = ∂ ∂ = − + = ∂ ∂ 4 2 2 2

1 1

λ λ y y L x x L

21 = + y x

24 10 11

1 =

= = λ y x

Stationary point: Feasibility: K-T point

Regular solution

 A solution x is called regular if:

 It is an interior point

• r

 It is a boundary (border) point and the gradients of the

binding constraints for x are linearly independent.

Constraint qualifications

 1st const. qual.: there are no constraints.  2nd const. qual.(linearity): all constraints are linear.  3rd const. qual.(regularity): all feasible solutions are regular.

We would like to prove that at least one of these conditions is satisfied.

K-T necessary condition

If we have a NLP problem with C2 functions that satisfies at least one constraint qualification: For each solution x* that is a local optimum, there is a vector of Lagrangian multipliers λ such that (x*, λ) is a K-T point.

Beware: If the K-T necessary condition is satisfied, then if x is an optimal solution x is a K-T point, but if x is a K-T point x is an optimal solution (we cannot even say if it is a local optimum)

Sufficient conditions for optimality

 When can we be sure that a K-T point is an optimal

solution?

 2 methods:

 Concavity/convexity (K-T sufficient condition theorem)  Weierstrass theorem + constraint qualification

K-T Sufficient conditions for optimality

Theorem (K-T sufficient condition): Let us consider a NLP problem such that:

• All the functions involved are C2
• S is a convex set
• The o.f. is convex (concave) and the problem is a minimization

(maximization) problem Then every K-T point is a global optimum.

Beware: this theorem is quite similar to the Local-Global theorem, but not exactly the same (the Local-Global theorem is for local optima, while this one is for K-T points).

K-T Sufficient conditions for

• ptimality

 Example:

, 5 2 . .

2 2

≥ ≤ + + y x y x t s y x Max

• All the functions are C2.
• S is defined by semispaces, thus it is a convex set.
• The objective function is convex, but we have a maximization problem.

Therefore, we cannot apply K-T sufficient condition

Weierstrass theorem + constraint qualification

 If its conditions are met, W. theorem states that the problem

will have a global optimum.

 If, in addition, one of the constraint qualifications is satisfied,

this optimum will be a K-T point.

 In this case, we will have to evaluate all the K-T points for

the objective function, and the one that gives the best value will be the optimal solution (notice that there can be more than one optimal solution).

Weierstrass theorem + constraint qualification

 Example:

, 5 2 . .

2 2

≥ ≤ + + y x y x t s y x Max

• The objective function is continuous.
• S is closed and bounded, thus compact.
• S is not empty (the point (0,0), e.g., is in S).

Therefore we can apply Weierstrass theorem, which says that the problem has a global optimum.

• Since the second constraint qualification (linearity) is satisfied, the global
• ptimum must be a K-T point.

Weierstrass theorem + constraint qualification

 Example:

, 5 2 . .

2 2

≥ ≤ + + y x y x t s y x Max

• We must evaluate the objective function F for all the K-T points:
• F(0,0)=0
• F(1,2)=5
• F(5,0)=25
• F(0,2.5)=6.25
• Since we have a maximization problem, the best point is (5,0), so this is

the optimal solution for the problem.

Economical meaning of the K-T multipliers

 The multipliers can be used to approximate the value the o.f will

take in the optimal solution if we modify the right-hand side of the associated constraint.

 If the right-hand side of the ith constraint increases by one unit,

the o.f value will approximately increase by units.

 The approximation is valid if the increase of the right-hand side

is small, and it becomes worse as the increase gets bigger.

 If the multiplier of a binding constraint is zero, we cannot use any

multiplier to estimate the variation of the o.f..