24. Nonlinear programming Overview Example: making tires Example: - PowerPoint PPT Presentation

CS/ECE/ISyE 524 Introduction to Optimization Spring 2017–18 24. Nonlinear programming ❼ Overview ❼ Example: making tires ❼ Example: largest inscribed polygon ❼ Example: navigation using ranges Laurent Lessard (www.laurentlessard.com)

First things first The labels nonlinear or nonconvex are not particularly informative or helpful in practice. ❼ Throughout the course we studied properties of linear constraints, convex quadratics, even MIPs. We can’t expect there to be a rigorous science for “everything else”. ❼ It doesn’t really make sense to define something as not having a particular property. ❼ “I’m an ECE professor” is a very informative statement. But using the label “non-(ECE professor)” is virtually meaningless. It could be a student, a horse, a tomato,... 24-2

Important categories ❼ Continuous vs discrete: As with LPs, the presence of binary or integer constraints is an important feature. ❼ Smoothness: Are the constraints and the objective function differentiable? twice-differentiable? ❼ Qualitative shape: Are there many local minima? ❼ Problem scale: A few variables? hundreds? thousands? This sort of information is very useful in practice. It helps you decide on an appropriate solution approach. 24-3

This lecture: examples! ❼ It doesn’t make sense to enumerate all the tips and trick for solving nonlinear/nonconvex problems. Too many! ❼ Instead, we will look at a few specific examples in detail. Each example will highlight some important lessons about dealing with nonconvex/nonlinear problems. 24-4

Example: making tires ❼ Tires are made by combining rubber, oil, and carbon. ❼ Tires must have a hardness of between 25 and 35. ❼ Tires must have an elasticity of at least 16. ❼ Tires must have a tensile strength of at least 12. ❼ To make a set of four tires, we require 100 pounds of total product (rubber, oil, and carbon). ◮ At least 50 pounds of carbon. ◮ Between 25 and 60 pounds of rubber. 24-5

Example: making tires ❼ Chemical Engineers tell you that the tensile strength, elasticity, and hardness of tires made of r pounds of rubber, h pounds of oil, and c pounds of carbon are... ◮ Tensile strength = 12 . 5 − 0 . 1 h − 0 . 001 h 2 ◮ Elasticity = 17 + . 35 r − 0 . 04 h − 0 . 002 r 2 ◮ Hardness = 34 + 0 . 1 r + 0 . 06 h − 0 . 3 c + 0 . 01 rh + 0 . 005 h 2 + 0 . 001 c 1 . 95 ❼ The Purchasing Department says rubber costs ✩ .04/pound, oil costs ✩ .01/pound, and carbon costs ✩ .07/pound. 24-6

Example: making tires minimize 0 . 04 r + 0 . 01 h + 0 . 07 c r , h , c total: r + h + c = 100 12 . 5 − 0 . 1 h − 0 . 001 h 2 ≥ 12 tensile: 17 + . 35 r − 0 . 04 h − 0 . 002 r 2 ≥ 16 elasticity: hardness: 25 ≤ 34 + 0 . 1 r + 0 . 06 h − 0 . 3 c + 0 . 01 rh + 0 . 005 h 2 + 0 . 001 c 1 . 95 ≤ 35 25 ≤ r ≤ 60 , h ≥ 0 , c ≥ 50 ❼ Problem is smooth and continuous. Julia: Tires.ipynb ❼ Fairly typical of something you might encounter in practice. Can we simplify it? Can we learn something useful? 24-7

Example: making tires minimize 0 . 04 r + 0 . 01 h + 0 . 07 c r , h , c total: r + h + c = 100 12 . 5 − 0 . 1 h − 0 . 001 h 2 ≥ 12 tensile: 17 + . 35 r − 0 . 04 h − 0 . 002 r 2 ≥ 16 elasticity: hardness: 25 ≤ 34 + 0 . 1 r + 0 . 06 h − 0 . 3 c + 0 . 01 rh + 0 . 005 h 2 + 0 . 001 c 1 . 95 ≤ 35 25 ≤ r ≤ 60 , h ≥ 0 , c ≥ 50 ❼ Optimal solution is: ( r ⋆ , h ⋆ , c ⋆ ) = (45 . 23 , 4 . 77 , 50). ❼ Only tensile constraint is tight! ❼ Does this mean elasticity and hardness don’t matter? 24-8

Example: making tires minimize 0 . 04 r + 0 . 01 h + 0 . 07 c r , h , c total: r + h + c = 100 12 . 5 − 0 . 1 h − 0 . 001 h 2 ≥ 12 tensile: 17 + . 35 r − 0 . 04 h − 0 . 002 r 2 ≥ 16 elasticity: hardness: 25 ≤ 34 + 0 . 1 r + 0 . 06 h − 0 . 3 c + 0 . 01 rh + 0 . 005 h 2 + 0 . 001 c 1 . 95 ≤ 35 25 ≤ r ≤ 60 , h ≥ 0 , c ≥ 50 ❼ Tensile constraint only depends on h . ❼ Can we simplify it? 24-9

Example: making tires Tensile constraint: 12 . 5 − 0 . 1 h − 0 . 001 h 2 ≥ 12 Tensile strength ❼ Since h ≥ 0, only a small range of h is admissible 14 12 ❼ If we solve for equality 10 (quadratic formula), the 8 positive solution is h = 4 . 77 6 h - 150 - 100 - 50 0 50 We can replace the tensile constraint by 0 ≤ h ≤ 4 . 77. 24-10

Example: making tires minimize 0 . 04 r + 0 . 01 h + 0 . 07 c r , h , c total: r + h + c = 100 tensile: 0 ≤ h ≤ 4 . 77 17 + . 35 r − 0 . 04 h − 0 . 002 r 2 ≥ 16 elasticity: hardness: 25 ≤ 34 + 0 . 1 r + 0 . 06 h − 0 . 3 c + 0 . 01 rh + 0 . 005 h 2 + 0 . 001 c 1 . 95 ≤ 35 25 ≤ r ≤ 60 , c ≥ 50 ❼ We can’t independently choose r , h , c ... ❼ Let’s eliminate r . Replace r by (100 − h − c ). 24-11

Example: making tires Objective function: 0 . 04 r + 0 . 01 h + 0 . 07 c = 0 . 04(100 − h − c ) + 0 . 01 h + 0 . 07 c = 4 − 0 . 03 h + 0 . 03 c Elasticity and hardness: (similar substitutions) 32 + 0 . 05 c − 0 . 002 c 2 + 0 . 01 h − 0 . 004 ch − 0 . 002 h 2 ≥ 16 25 ≤ 44 + 0 . 96 h − 0 . 4 c − 0 . 01 ch − 0 . 005 h 2 + 0 . 001 c 1 . 95 ≤ 35 Original bounds: 25 ≤ r ≤ 60 and c ≥ 50. ⇐ ⇒ 25 ≤ 100 − h − c ≤ 60 and c ≥ 50 ⇐ ⇒ 40 ≤ h + c ≤ 75 and c ≥ 50 ⇐ ⇒ 50 ≤ h + c ≤ 75 and c ≥ 50 24-12

Example: making tires minimize 4 − 0 . 03 h + 0 . 03 c h , c tensile: 0 ≤ h ≤ 4 . 77 bound: 50 ≤ h + c ≤ 75 , c ≥ 50 32 + 0 . 05 c − 0 . 002 c 2 + 0 . 01 h elasticity: − 0 . 004 ch − 0 . 002 h 2 ≥ 16 hardness: 25 ≤ 44 + 0 . 96 h − 0 . 4 c − 0 . 01 ch − 0 . 005 h 2 + 0 . 001 c 1 . 95 ≤ 35 ❼ tensile constraint is now linear ❼ elasticity constraint is a convex quadratic ❼ Only two variables! Let’s draw a picture... 24-13

Example: making tires carbon ( c ) 100 80 lin. constr. elasticity 60 hardness 40 20 80 oil ( h ) 0 20 40 60 ❼ Feasible region is quite small. Let’s zoom in... 24-14

Example: making tires carbon ( c ) 60 58 56 54 52 50 48 46 oil ( h ) 0 1 2 3 4 5 6 ❼ Objective is to minimize 4 − 0 . 03 h + 0 . 03 c ❼ Solution doesn’t involve hardness or elasticity constraints. 24-15

Example: making tires carbon ( c ) 60 ❼ Objective function is: A 58 ( p h − p r ) h + ( p c − p r ) c 56 B where p i is the price of i . 54 52 50 D ❼ Normal vector for objective: 48 C � p h − p r � 46 oil ( h ) n = 0 1 2 3 4 5 6 p c − p r Simple solution: ❼ Is rubber the cheapest ingredient? if so, choose C . ❼ Otherwise: is rubber the most expensive? if so, choose A . ❼ Otherwise: is oil cheaper than carbon? if so, choose D . ❼ Is rubber cheaper than the avg price of carbon and oil? if so, choose B . Otherwise, choose A . 24-16

Making tires, what did we learn? ❼ Sometimes constraints that look complicated aren’t actually complicated. ❼ Sometimes a constraint won’t matter. You can examine dual variables to quickly check which constraints are active. ❼ If you can draw a picture, draw a picture! ❼ Complicated-looking problems can have simple solutions. 24-17

Example: largest inscribed polygon What is the polygon ( n sides) of maximal area that can be completely contained inside a circle of radius 1? ❼ A pretty famous problem. The solution is a regular polygon. All sides have equal length with vertices on the unit circle. ❼ How can we solve this using optimization? 24-18

Example: largest inscribed polygon B First model r 2 A Express the vertices of the polygon in r 1 polar coordinates ( r i , θ i ) where the origin θ 2 is the center of the circle and angles are θ 1 measured with respect to (1 , 0). O ❼ What are the constraints? ❼ How do we compute the area? ❼ We must have r i ≤ 1 to ensure all points are inscribed. ❼ Calculate the area one triangle at a time. For example, triangle (OAB) has area 1 2 r 1 r 2 sin( θ 2 − θ 1 ). ❼ Is this enough? Let’s see... Polygon.ipynb 24-19

Example: largest inscribed polygon Model Result Solution is incorrect! n 1 � ❼ Adding θ i ≥ 0 doesn’t help. max r i r i +1 sin( θ i +1 − θ i ) 2 r ,θ ❼ Adding θ i ≤ 2 π doesn’t help. i =1 s.t. 0 ≤ r i ≤ 1 ❼ Adding θ 1 = 0 doesn’t help. ❼ can obtain a single-point solution ❼ can obtain polygons that cross each other ❼ can obtain other suboptimal polygons The reason is local maxima . More on this later... 24-20

Example: largest inscribed polygon Model 1 finalized: By assigning an order to the angles, we obtain the model: n 1 � maximize r i r i +1 sin( θ i +1 − θ i ) 2 r ,θ i =1 subject to: 0 ≤ r i ≤ 1 0 = θ 1 ≤ θ 2 ≤ · · · ≤ θ n ≤ 2 π This model produces the correct solution! 24-21

Example: largest inscribed polygon B Second model r 2 A This time use relative angles . α i is the α 2 r 1 angle between a pair of adjacent edges. r 3 α 1 This automatically encodes ordering! O ❼ What are the constraints? ❼ How do we compute the area? ❼ We must have r i ≤ 1 to ensure all points are inscribed. ❼ Angles must sum to the full circle: α 1 + · · · + α n = 2 π . ❼ Calculate the area one triangle at a time. For example, triangle (OAB) has area 1 2 r 1 r 2 sin( α i ). 24-22

Example: largest inscribed polygon Model 2 finalized: n 1 � maximize r i r i +1 sin( α i ) 2 r ,α i =1 subject to: 0 ≤ r i ≤ 1 α 1 + · · · + α n = 2 π α i ≥ 0 This model produces the correct solution as well! 24-23