UNIT 1 An introduction to optimization An optimization problem An - - PowerPoint PPT Presentation

UNIT 1

An introduction to optimization

An optimization problem

An engineering factory makes 4 products (PROD1 to PROD4) on the following machines: 4 grinders, 2 drills and 1 planer. Each product yields a certain contribution to profit (defined as €/unit selling price minus cost of raw materials). These quantities together with the unit production times (hours) required on each process are given in the table below. PROD 1 PROD 2 PROD 3 PROD 4 Contribution to profit 10 6 8 4 Grinding 0.5 0.7 0.2

• Drilling

0.1 0.2

• 0.3

Planing

• 0.1
• If there are 8 working hours in a day, how many units of each

product should the factory produce (daily) in order to maximize the total profit?

An optimization model

• Variables: represent the unknown quantities for which we

want to find the optimal value

• Objective function: represents the expression of the quantity

that we want to be maximal or minimal

• Constraints: represent the conditions that the values of the

variables must satisfy (non-negativity constraints are a special case)

• Optimal solution: among all the solutions that satisfy all the

constraints, the one with the best value for the objective function:

Problem types

 Non Linear Programming (NLP): functions can be of any type, but variables must

be continuous.

 Linear Programming (LP): all the functions in the model (objective function and

constraints) are linear and the variables are continuous.

 Classical Programming (CP): the model has only = constraints (or no constraints at

all) and the variables are continuous.

 Integer Linear Programming (ILP): all the functions are linear and the variables

may take only integer variables.

Remarks:

• A problem can be in more than one class. For example, it can

be a NLP, LP and CP problem at the same time.

• LP and CP problems are also NLP problems, but ILP are not.
• There are more types of mathematical programming problems

but we will not discuss them here.

Problem types: examples

NLP and LP NLP and CP ILP

Feasible and unfeasible solutions

 A solution is a set of values for the variables of the problem.  If a solution satisfies all the constraints of the problem, it is

called feasible solution.

 If it does not satisfy one ore more constraints (it violates a

constraint), it is called infeasible.

 The set of all feasible solutions of a problem is called

solution set (or choice set).

Interior and boundary solutions

 If a feasible solution satisfies one or more constraints

with equality, it is called boundary solution. The constraints that are satisfied with equality for this solution are called binding constraints. The constraints that are not satisfied with equality are called non- binding constraints.

 If a feasible solution does not satisfy any constraint with

equality, it is called interior solution.

Maxima and minima

 Given a function , a solution set and a

feasible solution :

 is a non-strict global maximum of in if for all

.

 is a strict global maximum of in if for all

.

 is a non-strict local maximum of in if there is a

such that for all satisfying .

 is a strict local maximum of in if there is a such

that for all satisfying .

Optimal solutions

 In a maximization problem, an optimal solution is a global

maximum of the objective function in the solution set.

 In a minimization problem, an optimal solution is a global

minimum of the objective function in the solution set.

Problem types according to their solutions

 A problem is said to be feasible if it has at least one

feasible solution, and it is called infeasible if it has no feasible solutions.

 A problem is unbounded if it is feasible but it has no optimal

solution (for any feasible solution, there is always another feasible solution that gives a better value of the objective function).

Graphical resolution

 When the problem has only 2 variables, it can be solved

graphically.

 Here, we will only solve graphically problems that have a linear

• bjective function.

 Let us consider, for example, the following LP problem:

Graphical resolution

 First, we must draw the solution set.  To do this, we draw each constraint

considering it an equality constraint.

(1,1)

Graphical resolution

 We must find out which side of the line corresponds with

the constraint.

 We can do this by choosing an arbitrary point and

checking if it satisfies the inequality or not.

 For example, point (1,1) satisfies the inequality, so the

right side must be the lower one.

Graphical resolution

 After drawing all the constraints and determining the right

side, we get the solution set as the intersection of all the regions.

S

Graphical resolution



Now we draw the objective function.



First, we set it equal to a certain value (for instance 0) and draw it.



Then we choose a different value and draw it again.



So we can see in which direction it increases and in which direction it decreases.

S

Graphical resolution



Since we want to maximize the objective function, we move the line in the direction of increase, but without leaving the solution set.



The last point of the solution set that is “touched” by the line is the

• ptimal solution.



To obtain the exact values for the optimal solution, we must solve the linear system of equations formed by the constraints that intersect at the optimal solution, in this case:

S

(1,4.5)

Problem transformations

 As we will see later, sometimes it is convenient to

modify a problem.

 Let us see how to transform variables, constraints

and objective of a problem.

Problem transformations: objective

 If we have a maximization problem, we can change it into a

minimization one by changing the sign of the coefficients of the objective function:

 The inverse transformation is valid as well.  But, after solving the transformed problem, we will have to

change the sign of the optimal value of the objective function.

Problem transformations: elimination of constants

 We can remove any constant value from the objective

function:

 The optimal value of the objective function of the second

problem will be 10 units less than that of the first problem.

Problem transformations: constraints

 We can change the sign of a constraint multiplying all its

members by -1:

Problem transformations: turning inequalities into equalities

 We can transform a or constraint into a = one by

adding new variables (slack variables):

 If the the constraint is a inequality, we add a slack

variable.

 If the constraint is a inequality, we subtract a slack

variable.

Problem transformations: turning equalities into inequalities

 We can transform a = constraint into two inequalities, one

and one :

Problem transformations: changing the sign of the variables

 We can substitute a non-positive variable by a non-

negative one with opposite sign in all the functions of the problem:

 We can also substitute a non-negative variable by a non-

positive one, but this is rarely done.

Problem transformations: changing the sign of the variables

 We can substitute a free variable (with no specific sign) by

two non-negative ones:

Basic theorems of Mathematical Programming

 Now we will see some theorems that will be used to

guarantee that a certain optimization problem has an optimal solution

Weierstrass theorem

 When does a MP problem have at least one optimal solution?

Weierstrass theorem: A continous function defined over a non-empty compact solution set has at least one global maximum and one global minimum.

What does compact mean?

Compact sets

 S is closed The complementary of S is open  S is bounded All the variables are bounded  S is compact S is closed and bounded

Theorem: Every set S defined by constraints and continuous functions is a closed set.

= ≥ ≤ , ,

Compact sets

 Example:

 S is defined by constraints  are continuous functions

{ }

20 5 3 2 / ) , , (

3

≥ ∧ ≥ ∧ ≥ ∧ ≤ + + ℜ ∈ = z y x z y x z y x S

≥ ≤,    + + z y x z y x , , 5 3 2

S is a closed set

Compact sets

 Example:

 

{ }

20 5 3 2 / ) , , (

3

≥ ∧ ≥ ∧ ≥ ∧ ≤ + + ℜ ∈ = z y x z y x z y x S S ∈ ) , , (

       ≤ ≤ ≤ ≤ ≤ ≤ 4 3 20 10 z y x

S is bounded and closed S is compact S is non-empty S is bounded

Local-Global theorem

 When is a local minimum (maximum) also a global minimum (maximum)?

Local-Global theorem:

• If S is convex and the objective function is (strictly)

convex in S, then every local minimum is a (strict) global minimum.

• If S is convex and the objective function is (strictly)

concave in S, then every local maximum is a (strict) global maximum.

What does convex mean?