In case you missed it - Who am I ? Name: S ebastien Gros - - PowerPoint PPT Presentation

In case you missed it - Who am I ?

Name: S´ ebastien Gros Nationality: Swiss Residence: G¨

• teborg, Sweden

Affiliation: Chalmers University of Technology Department: Signals & Systems Position: Assistant Professor Email: grosse@chalmers.se Tel. +46 31 772 15 55 Recent research topics: distributed & parallelized methods for optimal control, estimation & system identification, NMPC & Economic NMPC, optimal control for complex mechanical systems, integrators for real-time optimal control, robust optimal control, aerospace applications, airborne wind energy, wind turbine control, smart grids, traffic control

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 1 / 32

Numerical Optimal Control with DAEs Lecture 5: Newton method & SQP

S´ ebastien Gros

AWESCO PhD course

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 2 / 32

Survival map of Direct Optimal Control

OCP Collocation Single-Shooting Multiple-Shooting NLP Interior-Point Interior-Point SQP Active-Set QP solver QP solver

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 3 / 32

Survival map of Direct Optimal Control

OCP Collocation Single-Shooting Multiple-Shooting NLP Interior-Point Interior-Point SQP Active-Set QP solver QP solver

Newton - a general-purpose sledgehammer for algebraic equations...

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 3 / 32

Survival map of Direct Optimal Control

OCP Collocation Single-Shooting Multiple-Shooting NLP Interior-Point Interior-Point SQP Active-Set QP solver QP solver

Newton - a general-purpose sledgehammer for algebraic equations... ... will be used to solve the KKT conditions !!

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 3 / 32

Outline

1

KKT conditions - Quick Reminder

2

The Newton method

3

Newton on the KKT conditions

4

Sequential Quadratic Programming

5

Hessian approximation

6

Maratos effect

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 4 / 32

Outline

1

KKT conditions - Quick Reminder

2

The Newton method

3

Newton on the KKT conditions

4

Sequential Quadratic Programming

5

Hessian approximation

6

Maratos effect

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 5 / 32

KKT point

Consider the NLP problem: min

w

Φ (w) s.t. g (w) = 0 h (w) ≤ 0

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 6 / 32

KKT point

Consider the NLP problem: min

w

Φ (w) s.t. g (w) = 0 h (w) ≤ 0 A point {w∗, µ∗, λ∗ } is called a KKT point if it satisfies: where L = Φ (w) + λTg (w) + µTh (w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 6 / 32

KKT point

Consider the NLP problem: min

w

Φ (w) s.t. g (w) = 0 h (w) ≤ 0 A point {w∗, µ∗, λ∗ } is called a KKT point if it satisfies: Dual Feasibility: ∇wL (w∗, µ∗, λ∗ ) = 0, µ∗ ≥ 0, where L = Φ (w) + λTg (w) + µTh (w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 6 / 32

KKT point

Consider the NLP problem: min

w

Φ (w) s.t. g (w) = 0 h (w) ≤ 0 A point {w∗, µ∗, λ∗ } is called a KKT point if it satisfies: Dual Feasibility: ∇wL (w∗, µ∗, λ∗ ) = 0, µ∗ ≥ 0, Primal Feasibility: g (w∗) = 0, h (w∗) ≤ 0, where L = Φ (w) + λTg (w) + µTh (w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 6 / 32

KKT point

Consider the NLP problem: min

w

Φ (w) s.t. g (w) = 0 h (w) ≤ 0 A point {w∗, µ∗, λ∗ } is called a KKT point if it satisfies: Dual Feasibility: ∇wL (w∗, µ∗, λ∗ ) = 0, µ∗ ≥ 0, Primal Feasibility: g (w∗) = 0, h (w∗) ≤ 0, Complementary Slackness: µ∗

i hi(w∗) = 0,

∀ i where L = Φ (w) + λTg (w) + µTh (w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 6 / 32

KKT point

Consider the NLP problem: min

w

Φ (w) s.t. g (w) = 0 h (w) ≤ 0 A point {w∗, µ∗, λ∗ } is called a KKT point if it satisfies: Dual Feasibility: ∇wL (w∗, µ∗, λ∗ ) = 0, µ∗ ≥ 0, Primal Feasibility: g (w∗) = 0, h (w∗) ≤ 0, Complementary Slackness: µ∗

i hi(w∗) = 0,

∀ i where L = Φ (w) + λTg (w) + µTh (w) Optimality conditions for NLP with equality and/or inequality constraints:

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 6 / 32

KKT point

Consider the NLP problem: min

w

Φ (w) s.t. g (w) = 0 h (w) ≤ 0 A point {w∗, µ∗, λ∗ } is called a KKT point if it satisfies: Dual Feasibility: ∇wL (w∗, µ∗, λ∗ ) = 0, µ∗ ≥ 0, Primal Feasibility: g (w∗) = 0, h (w∗) ≤ 0, Complementary Slackness: µ∗

i hi(w∗) = 0,

∀ i where L = Φ (w) + λTg (w) + µTh (w) Optimality conditions for NLP with equality and/or inequality constraints: 1st-Order Necessary Conditions: A (local) optimum w⋆ satisfying LICQ of a (differentiable) NLP corresponds to a unique KKT point

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 6 / 32

KKT point

Consider the NLP problem: min

w

Φ (w) s.t. g (w) = 0 h (w) ≤ 0 A point {w∗, µ∗, λ∗ } is called a KKT point if it satisfies: Dual Feasibility: ∇wL (w∗, µ∗, λ∗ ) = 0, µ∗ ≥ 0, Primal Feasibility: g (w∗) = 0, h (w∗) ≤ 0, Complementary Slackness: µ∗

i hi(w∗) = 0,

∀ i where L = Φ (w) + λTg (w) + µTh (w) Optimality conditions for NLP with equality and/or inequality constraints: 1st-Order Necessary Conditions: A (local) optimum w⋆ satisfying LICQ of a (differentiable) NLP corresponds to a unique KKT point 2nd-Order Sufficient Conditions require positivity of the Hessian ∇2

wL in all

critical feasible directions at the solution

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 6 / 32

KKT point

Consider the NLP problem: min

w

Φ (w) s.t. g (w) = 0 h (w) ≤ 0 Most NLP solvers are in essence ”KKT solvers” A point {w∗, µ∗, λ∗ } is called a KKT point if it satisfies: Dual Feasibility: ∇wL (w∗, µ∗, λ∗ ) = 0, µ∗ ≥ 0, Primal Feasibility: g (w∗) = 0, h (w∗) ≤ 0, Complementary Slackness: µ∗

i hi(w∗) = 0,

∀ i where L = Φ (w) + λTg (w) + µTh (w) Optimality conditions for NLP with equality and/or inequality constraints: 1st-Order Necessary Conditions: A (local) optimum w⋆ satisfying LICQ of a (differentiable) NLP corresponds to a unique KKT point 2nd-Order Sufficient Conditions require positivity of the Hessian ∇2

wL in all

critical feasible directions at the solution

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 6 / 32

Outline

1

KKT conditions - Quick Reminder

2

The Newton method

3

Newton on the KKT conditions

4

Sequential Quadratic Programming

5

Hessian approximation

6

Maratos effect

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 7 / 32

Core idea

Goal: solve r (w) = 0... how ?!?

• 0.4
• 0.2

0.2

w r (w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 8 / 32

Core idea

Goal: solve r (w) = 0... how ?!?

• 0.4
• 0.2

0.2

w w r (w) Key idea: guess w, iterate the linear model: r (w + ∆w) ≈ r (w) + ∇r (w)⊤ ∆w = 0

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 8 / 32

Core idea

Goal: solve r (w) = 0... how ?!?

• 0.4
• 0.2

0.2

w w r (w) Key idea: guess w, iterate the linear model: r (w + ∆w) ≈ r (w) + ∇r (w)⊤ ∆w = 0

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 8 / 32

Core idea

Goal: solve r (w) = 0... how ?!?

• 0.4
• 0.2

0.2

w w r (w) Key idea: guess w, iterate the linear model: r (w + ∆w) ≈ r (w) + ∇r (w)⊤ ∆w = 0

Algorithm: Newton method Input: w, tol while r (w) ∞ ≥ tol do Compute r (w) and ∇r (w) Compute the Newton direction ∇r (w)T ∆w = −r (w) Newton step w ← w + ∆w return w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 8 / 32

Core idea

Goal: solve r (w) = 0... how ?!?

• 0.4
• 0.2

0.2

w w r (w) Key idea: guess w, iterate the linear model: r (w + ∆w) ≈ r (w) + ∇r (w)⊤ ∆w = 0

Algorithm: Newton method Input: w, tol while r (w) ∞ ≥ tol do Compute r (w) and ∇r (w) Compute the Newton direction ∇r (w)T ∆w = −r (w) Newton step w ← w + ∆w return w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 8 / 32

Core idea

Goal: solve r (w) = 0... how ?!?

• 0.4
• 0.2

0.2

w w r (w) Key idea: guess w, iterate the linear model: r (w + ∆w) ≈ r (w) + ∇r (w)⊤ ∆w = 0

Algorithm: Newton method Input: w, tol while r (w) ∞ ≥ tol do Compute r (w) and ∇r (w) Compute the Newton direction ∇r (w)T ∆w = −r (w) Newton step w ← w + ∆w return w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 8 / 32

Core idea

Goal: solve r (w) = 0... how ?!?

• 0.4
• 0.2

0.2

w w r (w) Key idea: guess w, iterate the linear model: r (w + ∆w) ≈ r (w) + ∇r (w)⊤ ∆w = 0

Algorithm: Newton method Input: w, tol while r (w) ∞ ≥ tol do Compute r (w) and ∇r (w) Compute the Newton direction ∇r (w)T ∆w = −r (w) Newton step w ← w + ∆w return w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 8 / 32

Core idea

Goal: solve r (w) = 0... how ?!?

• 0.4
• 0.2

0.2

w w r (w) Key idea: guess w, iterate the linear model: r (w + ∆w) ≈ r (w) + ∇r (w)⊤ ∆w = 0

Algorithm: Newton method Input: w, tol while r (w) ∞ ≥ tol do Compute r (w) and ∇r (w) Compute the Newton direction ∇r (w)T ∆w = −r (w) Newton step w ← w + ∆w return w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 8 / 32

Core idea

Goal: solve r (w) = 0... how ?!?

• 0.4
• 0.2

0.2

w w r (w) Key idea: guess w, iterate the linear model: r (w + ∆w) ≈ r (w) + ∇r (w)⊤ ∆w = 0

Algorithm: Newton method Input: w, tol while r (w) ∞ ≥ tol do Compute r (w) and ∇r (w) Compute the Newton direction ∇r (w)T ∆w = −r (w) Newton step w ← w + ∆w return w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 8 / 32

Core idea

Goal: solve r (w) = 0... how ?!?

• 0.4
• 0.2

0.2

w w r (w) Key idea: guess w, iterate the linear model: r (w + ∆w) ≈ r (w) + ∇r (w)⊤ ∆w = 0

Algorithm: Newton method Input: w, tol while r (w) ∞ ≥ tol do Compute r (w) and ∇r (w) Compute the Newton direction ∇r (w)T ∆w = −r (w) Newton step w ← w + ∆w return w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 8 / 32

Core idea

Goal: solve r (w) = 0... how ?!?

• 0.4
• 0.2

0.2

w w r (w) Key idea: guess w, iterate the linear model: r (w + ∆w) ≈ r (w) + ∇r (w)⊤ ∆w = 0

Algorithm: Newton method Input: w, tol while r (w) ∞ ≥ tol do Compute r (w) and ∇r (w) Compute the Newton direction ∇r (w)T ∆w = −r (w) Newton step w ← w + ∆w return w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 8 / 32

Core idea

Goal: solve r (w) = 0... how ?!?

• 0.4
• 0.2

0.2

w w r (w) Key idea: guess w, iterate the linear model: r (w + ∆w) ≈ r (w) + ∇r (w)⊤ ∆w = 0

Algorithm: Newton method Input: w, tol while r (w) ∞ ≥ tol do Compute r (w) and ∇r (w) Compute the Newton direction ∇r (w)T ∆w = −r (w) Newton step w ← w + ∆w return w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 8 / 32

Core idea

Goal: solve r (w) = 0... how ?!?

• 0.4
• 0.2

0.2

w w r (w) Key idea: guess w, iterate the linear model: r (w + ∆w) ≈ r (w) + ∇r (w)⊤ ∆w = 0 This is a full-step Newton iteration

Algorithm: Newton method Input: w, tol while r (w) ∞ ≥ tol do Compute r (w) and ∇r (w) Compute the Newton direction ∇r (w)T ∆w = −r (w) Newton step w ← w + ∆w return w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 8 / 32

Core idea

Goal: solve r (w) = 0... how ?!?

• 0.4
• 0.2

0.2

w w r (w) Key idea: guess w, iterate the linear model: r (w + ∆w) ≈ r (w) + ∇r (w)⊤ ∆w = 0 This is a full-step Newton iteration Reduced steps are often needed

Algorithm: Newton method Input: w, tol while r (w) ∞ ≥ tol do Compute r (w) and ∇r (w) Compute the Newton direction ∇r (w)T ∆w = −r (w) Newton step, t ∈ ]0, 1] w ← w + t∆w return w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 8 / 32

Why reduced steps ?

Newton step with t ∈ ]0, 1]: ∇r (w)⊤ ∆w = −r (w) w ← w + t∆w

• 1.5
• 1
• 0.5

0.5 1 1.5 w r (w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 9 / 32

Why reduced steps ?

Newton step with t ∈ ]0, 1]: ∇r (w)⊤ ∆w = −r (w) w ← w + t∆w

• 1.5
• 1
• 0.5

0.5 1 1.5 t = 1 w w r (w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 9 / 32

Why reduced steps ?

Newton step with t ∈ ]0, 1]: ∇r (w)⊤ ∆w = −r (w) w ← w + t∆w

• 1.5
• 1
• 0.5

0.5 1 1.5 t = 1 w w r (w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 9 / 32

Why reduced steps ?

Newton step with t ∈ ]0, 1]: ∇r (w)⊤ ∆w = −r (w) w ← w + t∆w

• 1.5
• 1
• 0.5

0.5 1 1.5 t = 1 w w r (w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 9 / 32

Why reduced steps ?

Newton step with t ∈ ]0, 1]: ∇r (w)⊤ ∆w = −r (w) w ← w + t∆w

• 1.5
• 1
• 0.5

0.5 1 1.5 t = 1 w w r (w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 9 / 32

Why reduced steps ?

Newton step with t ∈ ]0, 1]: ∇r (w)⊤ ∆w = −r (w) w ← w + t∆w

• 1.5
• 1
• 0.5

0.5 1 1.5 t = 1 w w r (w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 9 / 32

Why reduced steps ?

Newton step with t ∈ ]0, 1]: ∇r (w)⊤ ∆w = −r (w) w ← w + t∆w

• 1.5
• 1
• 0.5

0.5 1 1.5 t = 1 w w r (w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 9 / 32

Why reduced steps ?

Newton step with t ∈ ]0, 1]: ∇r (w)⊤ ∆w = −r (w) w ← w + t∆w

• 1.5
• 1
• 0.5

0.5 1 1.5 t = 1 w w r (w) The full-step Newton iteration can be unstable !!

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 9 / 32

Why reduced steps ?

Newton step with t ∈ ]0, 1]: ∇r (w)⊤ ∆w = −r (w) w ← w + t∆w

• 1.5
• 1
• 0.5

0.5 1 1.5 t = 0.8 w w r (w) The full-step Newton iteration can be unstable !!

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 9 / 32

Why reduced steps ?

Newton step with t ∈ ]0, 1]: ∇r (w)⊤ ∆w = −r (w) w ← w + t∆w

• 1.5
• 1
• 0.5

0.5 1 1.5 t = 0.8 w w r (w) The full-step Newton iteration can be unstable !!

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 9 / 32

Why reduced steps ?

Newton step with t ∈ ]0, 1]: ∇r (w)⊤ ∆w = −r (w) w ← w + t∆w

• 1.5
• 1
• 0.5

0.5 1 1.5 t = 0.8 w w r (w) The full-step Newton iteration can be unstable !!

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 9 / 32

Why reduced steps ?

Newton step with t ∈ ]0, 1]: ∇r (w)⊤ ∆w = −r (w) w ← w + t∆w

• 1.5
• 1
• 0.5

0.5 1 1.5 t = 0.8 w w r (w) The full-step Newton iteration can be unstable !! While the reduced-steps Newton iteration is stable...

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 9 / 32

Does Newton always work ?

Is the Newton step ∆w always providing a direction ”improving” r (w) ?

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 10 / 32

Does Newton always work ?

Is the Newton step ∆w always providing a direction ”improving” r (w) ? I.e. is there always a t > 0 s.t. r (w + t∆w) < r (w) is true ?

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 10 / 32

Does Newton always work ?

Is the Newton step ∆w always providing a direction ”improving” r (w) ? I.e. is there always a t > 0 s.t. r (w + t∆w) < r (w) is true ? Yes... but

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 10 / 32

Does Newton always work ?

Is the Newton step ∆w always providing a direction ”improving” r (w) ? I.e. is there always a t > 0 s.t. r (w + t∆w) < r (w) is true ? Yes... but Proof: r (w + t∆w) < r (w) holds for some t > 0 if d dt r (w + t∆w) 2

• t=0

< 0 with r (w) 2 differentiable.

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 10 / 32

Does Newton always work ?

Is the Newton step ∆w always providing a direction ”improving” r (w) ? I.e. is there always a t > 0 s.t. r (w + t∆w) < r (w) is true ? Yes... but Proof: r (w + t∆w) < r (w) holds for some t > 0 if d dt r (w + t∆w) 2

• t=0

< 0 with r (w) 2 differentiable. I.e. 2r (w)T d dt r (w + t∆w)t=0 < 0

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 10 / 32

Does Newton always work ?

Is the Newton step ∆w always providing a direction ”improving” r (w) ? I.e. is there always a t > 0 s.t. r (w + t∆w) < r (w) is true ? Yes... but Proof: r (w + t∆w) < r (w) holds for some t > 0 if d dt r (w + t∆w) 2

• t=0

< 0 with r (w) 2 differentiable. I.e. 2r (w)T d dt r (w + t∆w)t=0 < 0 We have d dt r (w + t∆w)t=0 = ∇r (w)T∆w = −∇r (w)T∇r (w)−Tr (w) = −r (w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 10 / 32

Does Newton always work ?

Is the Newton step ∆w always providing a direction ”improving” r (w) ? I.e. is there always a t > 0 s.t. r (w + t∆w) < r (w) is true ? Yes... but Proof: r (w + t∆w) < r (w) holds for some t > 0 if d dt r (w + t∆w) 2

• t=0

< 0 with r (w) 2 differentiable. I.e. 2r (w)T d dt r (w + t∆w)t=0 < 0 We have d dt r (w + t∆w)t=0 = ∇r (w)T∆w = −∇r (w)T∇r (w)−Tr (w) = −r (w) Then d dt r (w + t∆w) 2

• t=0

= −2r (w) 2 < 0

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 10 / 32

Does Newton always work ?

Is the Newton step ∆w always providing a direction ”improving” r (w) ? I.e. is there always a t > 0 s.t. r (w + t∆w) < r (w) is true ? Yes... but How to select the step size t ∈ ]0, 1] ? Globalization... Line-search: reduce t until some criteria of progression on r are met Trust region: confine the step ∆w within a region where ∇r (w) provides a good model of r (w) Filter techniques: monitor progress on specific components of r (w) separately ... ... ensures that progress is made in one way or another. Note: most of these techniques are specific to optimization.

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 10 / 32

But still, Newton can fail...

Solve r (w) = 0

r(w)

• 0.4
• 0.2

0.2 0.4 0.6

w w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 11 / 32

But still, Newton can fail...

Solve r (w) = 0

r(w)

• 0.4
• 0.2

0.2 0.4 0.6

w w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 11 / 32

But still, Newton can fail...

Solve r (w) = 0

r(w)

• 0.4
• 0.2

0.2 0.4 0.6

w w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 11 / 32

But still, Newton can fail...

Solve r (w) = 0

r(w)

• 0.4
• 0.2

0.2 0.4 0.6

w w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 11 / 32

But still, Newton can fail...

Solve r (w) = 0

r(w)

• 0.4
• 0.2

0.2 0.4 0.6

w w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 11 / 32

But still, Newton can fail...

Solve r (w) = 0

r(w)

• 0.4
• 0.2

0.2 0.4 0.6

w w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 11 / 32

But still, Newton can fail...

Solve r (w) = 0

r(w)

• 0.4
• 0.2

0.2 0.4 0.6

w w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 11 / 32

But still, Newton can fail...

Solve r (w) = 0

r(w)

• 0.4
• 0.2

0.2 0.4 0.6

w w Newton stops with r (w) = 0 and ∇r (w) singular i.e. the Newton direction ∆w given by ∇r (w)⊤ ∆w = −r (w) is undefined...

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 11 / 32

But still, Newton can fail...

Solve r (w) = 0

r(w)

• 0.4
• 0.2

0.2 0.4 0.6

w w Newton stops with r (w) = 0 and ∇r (w) singular i.e. the Newton direction ∆w given by ∇r (w)⊤ ∆w = −r (w) is undefined... This is a common failure mode for Newton-based solvers when tackling very non-linear r and starting with a poor initial guess !!

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 11 / 32

Convergence of full-step Newton methods

Newton method: ∇r (w)⊤ ∆w = −r (w) w ← w + ∆w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 12 / 32

Convergence of full-step Newton methods

Newton method: ∇r (w)⊤ ∆w = −r (w) w ← w + ∆w Yields the iteration k = 0, 1, ....: wk+1 ← wk − ∇r (wk)−⊤ r (wk)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 12 / 32

Convergence of full-step Newton methods

Newton method: ∇r (w)⊤ ∆w = −r (w) w ← w + ∆w Yields the iteration k = 0, 1, ....: wk+1 ← wk − ∇r (wk)−⊤ r (wk) Newton-type method (Jacobian approx.) M∆w = −r (w) w ← w + ∆w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 12 / 32

Convergence of full-step Newton methods

Newton method: ∇r (w)⊤ ∆w = −r (w) w ← w + ∆w Yields the iteration k = 0, 1, ....: wk+1 ← wk − ∇r (wk)−⊤ r (wk) Newton-type method (Jacobian approx.) M∆w = −r (w) w ← w + ∆w Yields the iteration k = 0, 1, ....: wk+1 ← wk − M−1

k r (wk)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 12 / 32

Convergence of full-step Newton methods

Newton method: ∇r (w)⊤ ∆w = −r (w) w ← w + ∆w Yields the iteration k = 0, 1, ....: wk+1 ← wk − ∇r (wk)−⊤ r (wk) Newton-type method (Jacobian approx.) M∆w = −r (w) w ← w + ∆w Yields the iteration k = 0, 1, ....: wk+1 ← wk − M−1

k r (wk)

Theorem: assume Nonlinearity of r:

• M−1

k

• ∇r(w)T − ∇r(w∗)T
• ≤ ω w − w∗, for

w ∈ [wk, w⋆]

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 12 / 32

Convergence of full-step Newton methods

Newton method: ∇r (w)⊤ ∆w = −r (w) w ← w + ∆w Yields the iteration k = 0, 1, ....: wk+1 ← wk − ∇r (wk)−⊤ r (wk) Newton-type method (Jacobian approx.) M∆w = −r (w) w ← w + ∆w Yields the iteration k = 0, 1, ....: wk+1 ← wk − M−1

k r (wk)

Theorem: assume Nonlinearity of r:

• M−1

k

• ∇r(w)T − ∇r(w∗)T
• ≤ ω w − w∗, for

w ∈ [wk, w⋆] Jacobian approximation error:

• M−1

k (∇r(wk)T − Mk)

• ≤ κk < 1
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 12 / 32

Convergence of full-step Newton methods

Newton method: ∇r (w)⊤ ∆w = −r (w) w ← w + ∆w Yields the iteration k = 0, 1, ....: wk+1 ← wk − ∇r (wk)−⊤ r (wk) Newton-type method (Jacobian approx.) M∆w = −r (w) w ← w + ∆w Yields the iteration k = 0, 1, ....: wk+1 ← wk − M−1

k r (wk)

Theorem: assume Nonlinearity of r:

• M−1

k

• ∇r(w)T − ∇r(w∗)T
• ≤ ω w − w∗, for

w ∈ [wk, w⋆] Jacobian approximation error:

• M−1

k (∇r(wk)T − Mk)

• ≤ κk < 1

Good initial guess w0 − w∗ ≤ 2

ω (1 − max {κk})

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 12 / 32

Convergence of full-step Newton methods

Newton method: ∇r (w)⊤ ∆w = −r (w) w ← w + ∆w Yields the iteration k = 0, 1, ....: wk+1 ← wk − ∇r (wk)−⊤ r (wk) Newton-type method (Jacobian approx.) M∆w = −r (w) w ← w + ∆w Yields the iteration k = 0, 1, ....: wk+1 ← wk − M−1

k r (wk)

Theorem: assume Nonlinearity of r:

• M−1

k

• ∇r(w)T − ∇r(w∗)T
• ≤ ω w − w∗, for

w ∈ [wk, w⋆] Jacobian approximation error:

• M−1

k (∇r(wk)T − Mk)

• ≤ κk < 1

Good initial guess w0 − w∗ ≤ 2

ω (1 − max {κk})

Then wk → w∗ with the following linear-quadratic contraction in each iteration: wk+1 − w∗ ≤

• κk + ω

2 wk − w∗

• wk − w∗.
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 12 / 32

Convergence of full-step Newton methods

Newton method: ∇r (w)⊤ ∆w = −r (w) w ← w + ∆w Yields the iteration k = 0, 1, ....: wk+1 ← wk − ∇r (wk)−⊤ r (wk) Newton-type method (Jacobian approx.) M∆w = −r (w) w ← w + ∆w Yields the iteration k = 0, 1, ....: wk+1 ← wk − M−1

k r (wk)

Theorem: assume Nonlinearity of r:

• M−1

k

• ∇r(w)T − ∇r(w∗)T
• ≤ ω w − w∗, for

w ∈ [wk, w⋆] Jacobian approximation error:

• M−1

k (∇r(wk)T − Mk)

• ≤ κk < 1

Good initial guess w0 − w∗ ≤ 2

ω (1 − max {κk})

Then wk → w∗ with the following linear-quadratic contraction in each iteration: wk+1 − w∗ ≤

• κk + ω

2 wk − w∗

• wk − w∗.

What about reduced steps ? Slow convergence when t < 1 (damped phase). When full steps become feasible, fast convergence to the solution.

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 12 / 32

Newton methods - Short Survival Guide

Exact Newton method: ∇r (w)⊤ ∆w = −r (w) w ← w + t∆w Newton-type method M∆w = −r (w) w ← w + t∆w

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 13 / 32

Newton methods - Short Survival Guide

Exact Newton method: ∇r (w)⊤ ∆w = −r (w) w ← w + t∆w Newton-type method M∆w = −r (w) w ← w + t∆w Exact Newton direction ∆w improves r for a sufficiently small step size t ∈ ]0, 1]

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 13 / 32

Newton methods - Short Survival Guide

Exact Newton method: ∇r (w)⊤ ∆w = −r (w) w ← w + t∆w Newton-type method M∆w = −r (w) w ← w + t∆w Exact Newton direction ∆w improves r for a sufficiently small step size t ∈ ]0, 1] Inexact Newton direction ∆w improves r for a sufficiently small step size t ∈ ]0, 1] if M > 0

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 13 / 32

Newton methods - Short Survival Guide

Exact Newton method: ∇r (w)⊤ ∆w = −r (w) w ← w + t∆w Newton-type method M∆w = −r (w) w ← w + t∆w Exact Newton direction ∆w improves r for a sufficiently small step size t ∈ ]0, 1] Inexact Newton direction ∆w improves r for a sufficiently small step size t ∈ ]0, 1] if M > 0 Exact full (t = 1) Newton steps converge quadratically if close enough to the solution

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 13 / 32

Newton methods - Short Survival Guide

Exact Newton method: ∇r (w)⊤ ∆w = −r (w) w ← w + t∆w Newton-type method M∆w = −r (w) w ← w + t∆w Exact Newton direction ∆w improves r for a sufficiently small step size t ∈ ]0, 1] Inexact Newton direction ∆w improves r for a sufficiently small step size t ∈ ]0, 1] if M > 0 Exact full (t = 1) Newton steps converge quadratically if close enough to the solution Inexact full (t = 1) Newton steps converge linearly if close enough to the solution and if the Jacobian approximation is ”sufficiently good”

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 13 / 32

Newton methods - Short Survival Guide

Exact Newton method: ∇r (w)⊤ ∆w = −r (w) w ← w + t∆w Newton-type method M∆w = −r (w) w ← w + t∆w Exact Newton direction ∆w improves r for a sufficiently small step size t ∈ ]0, 1] Inexact Newton direction ∆w improves r for a sufficiently small step size t ∈ ]0, 1] if M > 0 Exact full (t = 1) Newton steps converge quadratically if close enough to the solution Inexact full (t = 1) Newton steps converge linearly if close enough to the solution and if the Jacobian approximation is ”sufficiently good” Newton iteration fails if ∇r becomes singular

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 13 / 32

Newton methods - Short Survival Guide

Exact Newton method: ∇r (w)⊤ ∆w = −r (w) w ← w + t∆w Newton-type method M∆w = −r (w) w ← w + t∆w Exact Newton direction ∆w improves r for a sufficiently small step size t ∈ ]0, 1] Inexact Newton direction ∆w improves r for a sufficiently small step size t ∈ ]0, 1] if M > 0 Exact full (t = 1) Newton steps converge quadratically if close enough to the solution Inexact full (t = 1) Newton steps converge linearly if close enough to the solution and if the Jacobian approximation is ”sufficiently good” Newton iteration fails if ∇r becomes singular Newton methods with globalization converge in two phases: damped (slow) phase when reduced steps (t < 1) are needed, quadratic/ linear when full steps are possible.

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 13 / 32

Outline

1

KKT conditions - Quick Reminder

2

The Newton method

3

Newton on the KKT conditions

4

Sequential Quadratic Programming

5

Hessian approximation

6

Maratos effect

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 14 / 32

Core idea

A vast majority of solvers try to find a KKT point w, µ, λ i.e: Primal Feasibility: g (w) = 0, h (w) ≤ 0, Dual Feasibility: ∇wL (w, µ, λ ) = 0, µ ≥ 0, Complementarity Slackness: µihi(w) = 0, i = 1, ... where L = Φ (w) + λ⊤g (w) + µ⊤h (w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 15 / 32

Core idea

A vast majority of solvers try to find a KKT point w, µ, λ i.e: Primal Feasibility: g (w) = 0, h (w) ≤ 0, Dual Feasibility: ∇wL (w, µ, λ ) = 0, µ ≥ 0, Complementarity Slackness: µihi(w) = 0, i = 1, ... where L = Φ (w) + λ⊤g (w) + µ⊤h (w) Let’s consider for now equality constrained problems, i.e. find w, λ s.t.: ∇wL(w, λ) = 0 g(w) = 0

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 15 / 32

Core idea

A vast majority of solvers try to find a KKT point w, µ, λ i.e: Primal Feasibility: g (w) = 0, h (w) ≤ 0, Dual Feasibility: ∇wL (w, µ, λ ) = 0, µ ≥ 0, Complementarity Slackness: µihi(w) = 0, i = 1, ... where L = Φ (w) + λ⊤g (w) + µ⊤h (w) Let’s consider for now equality constrained problems, i.e. find w, λ s.t.: ∇wL(w, λ) = 0 g(w) = 0 Idea: apply the Newton method on the KKT conditions, i.e. Solve... r (w, λ) = ∇wL(w, λ) g(w)

• = 0
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 15 / 32

Core idea

A vast majority of solvers try to find a KKT point w, µ, λ i.e: Primal Feasibility: g (w) = 0, h (w) ≤ 0, Dual Feasibility: ∇wL (w, µ, λ ) = 0, µ ≥ 0, Complementarity Slackness: µihi(w) = 0, i = 1, ... where L = Φ (w) + λ⊤g (w) + µ⊤h (w) Let’s consider for now equality constrained problems, i.e. find w, λ s.t.: ∇wL(w, λ) = 0 g(w) = 0 Idea: apply the Newton method on the KKT conditions, i.e. Solve... r (w, λ) = ∇wL(w, λ) g(w)

• = 0

... by iterating ∇r (w, λ)T ∆w ∆λ

• = −r (w, λ)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 15 / 32

Newton method on the KKT conditions

KKT conditions r (w, λ) = ∇wL(w, λ) g(w)

• = 0

Newton direction ∇r (w, λ)T ∆w ∆λ

• = −r (w, λ)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 16 / 32

Newton method on the KKT conditions

KKT conditions r (w, λ) = ∇wL(w, λ) g(w)

• = 0

Newton direction ∇r (w, λ)T ∆w ∆λ

• = −r (w, λ)

Given by: ∇2

wL(w, λ)∆w

+ ∇w,λL(w, λ)∆λ = −∇wL(w, λ) ∇g(w)T∆w = −g(w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 16 / 32

Newton method on the KKT conditions

KKT conditions r (w, λ) = ∇wL(w, λ) g(w)

• = 0

Newton direction ∇r (w, λ)T ∆w ∆λ

• = −r (w, λ)

Given by: using ∇wL(w, λ) = ∇Φ(w) + ∇g(w)λ ∇2

wL(w, λ)∆w

+ ∇w,λL(w, λ)∆λ = −∇wL(w, λ) ∇g(w)T∆w = −g(w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 16 / 32

Newton method on the KKT conditions

KKT conditions r (w, λ) = ∇wL(w, λ) g(w)

• = 0

Newton direction ∇r (w, λ)T ∆w ∆λ

• = −r (w, λ)

Given by: using ∇wL(w, λ) = ∇Φ(w) + ∇g(w)λ ∇2

wL(w, λ)∆w

+ ∇g(w)∆λ = −∇wL(w, λ) ∇g(w)T∆w = −g(w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 16 / 32

Newton method on the KKT conditions

KKT conditions r (w, λ) = ∇wL(w, λ) g(w)

• = 0

Newton direction ∇r (w, λ)T ∆w ∆λ

• = −r (w, λ)

Given by: using ∇wL(w, λ) = ∇Φ(w) + ∇g(w)λ ∇2

wL(w, λ)∆w

+ ∇g(w)∆λ = −∇Φ(w) − ∇g(w)λ ∇g(w)T∆w = −g(w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 16 / 32

Newton method on the KKT conditions

KKT conditions r (w, λ) = ∇wL(w, λ) g(w)

• = 0

Newton direction ∇r (w, λ)T ∆w ∆λ

• = −r (w, λ)

Given by: using ∇wL(w, λ) = ∇Φ(w) + ∇g(w)λ ∇2

wL(w, λ)∆w

+ ∇g(w) (λ + ∆λ) = −∇Φ(w) ∇g(w)T∆w = −g(w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 16 / 32

Newton method on the KKT conditions

KKT conditions r (w, λ) = ∇wL(w, λ) g(w)

• = 0

Newton direction ∇r (w, λ)T ∆w ∆λ

• = −r (w, λ)

Given by: using ∇wL(w, λ) = ∇Φ(w) + ∇g(w)λ ∇2

wL(w, λ)∆w

+ ∇g(w) (λ + ∆λ) = −∇Φ(w) ∇g(w)T∆w = −g(w)

The Newton direction on the KKT conditions

∇2

wL(w, λ)

∇g(w) ∇g(w)T

• KKT matrix (symmetric indefinite)
• ∆w

λ + ∆λ

• = −

∇Φ(w) g(w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 16 / 32

Newton method on the KKT conditions

KKT conditions r (w, λ) = ∇wL(w, λ) g(w)

• = 0

Newton direction ∇r (w, λ)T ∆w ∆λ

• = −r (w, λ)

Given by: using ∇wL(w, λ) = ∇Φ(w) + ∇g(w)λ ∇2

wL(w, λ)∆w

+ ∇g(w) (λ + ∆λ) = −∇Φ(w) ∇g(w)T∆w = −g(w)

The Newton direction on the KKT conditions

H (w, λ) ∇g(w) ∇g(w)T

• KKT matrix (symmetric indefinite)
• ∆w

λ + ∆λ

• = −

∇Φ(w) g(w)

• where H (w, λ) = ∇2

wL(w, λ) is the Hessian of the problem.

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 16 / 32

Newton method on the KKT conditions

KKT conditions r (w, λ) = ∇wL(w, λ) g(w)

• = 0

Newton direction ∇r (w, λ)T ∆w ∆λ

• = −r (w, λ)

Given by: using ∇wL(w, λ) = ∇Φ(w) + ∇g(w)λ ∇2

wL(w, λ)∆w

+ ∇g(w) (λ + ∆λ) = −∇Φ(w) ∇g(w)T∆w = −g(w)

The Newton direction on the KKT conditions

H (w, λ) ∇g(w) ∇g(w)T

• KKT matrix (symmetric indefinite)

∆w λ+

• = −

∇Φ(w) g(w)

• where H (w, λ) = ∇2

wL(w, λ) is the Hessian of the problem. Note: update of the dual

variable is λ+ = λ + ∆λ

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 16 / 32

Newton method on the KKT conditions

KKT conditions r (w, λ) = ∇wL(w, λ) g(w)

• = 0

Newton direction ∇r (w, λ)T ∆w ∆λ

• = −r (w, λ)

Given by: using ∇wL(w, λ) = ∇Φ(w) + ∇g(w)λ ∇2

wL(w, λ)∆w

+ ∇g(w) (λ + ∆λ) = −∇Φ(w) ∇g(w)T∆w = −g(w)

The Newton direction on the KKT conditions

H (w, λ) ∇g(w) ∇g(w)T

• KKT matrix (symmetric indefinite)

∆w λ+

• = −

∇Φ(w) g(w)

• where H (w, λ) = ∇2

wL(w, λ) is the Hessian of the problem. Note: update of the dual

variable is λ+ = λ + ∆λ ∇wL(w, λ) is not needed for computing the Newton step

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 16 / 32

Newton method on the KKT conditions

KKT conditions r (w, λ) = ∇wL(w, λ) g(w)

• = 0

Newton direction ∇r (w, λ)T ∆w ∆λ

• = −r (w, λ)

Given by: using ∇wL(w, λ) = ∇Φ(w) + ∇g(w)λ ∇2

wL(w, λ)∆w

+ ∇g(w) (λ + ∆λ) = −∇Φ(w) ∇g(w)T∆w = −g(w)

The Newton direction on the KKT conditions

H (w, λ) ∇g(w) ∇g(w)T

• KKT matrix (symmetric indefinite)

∆w λ+

• = −

∇Φ(w) g(w)

• where H (w, λ) = ∇2

wL(w, λ) is the Hessian of the problem. Note: update of the dual

variable is λ+ = λ + ∆λ ∇wL(w, λ) is not needed for computing the Newton step The updated dual variables λ+ are readily provided !

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 16 / 32

Newton Iteration for Optimization - Example

min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0
• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Iterate:

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0
• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Iterate:

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• with:

∇g (w) = 2w = 2w1 2w2

• min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0
• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Iterate:

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• with:

∇g (w) = 2w = 2w1 2w2

• L (w, λ) = Φ (w) + λg (w)

∇wL (w, λ) = 2 1 1 4

• w +

1

• + 2λw

min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0
• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Iterate:

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• with:

∇g (w) = 2w = 2w1 2w2

• L (w, λ) = Φ (w) + λg (w)

∇wL (w, λ) = 2 1 1 4

• w +

1

• + 2λw

H (w, λ) = 2 + 2λ 1 1 4 + 2λ

• min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0
• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Iterate:

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• with:

∇g (w) = 2w = 2w1 2w2

• L (w, λ) = Φ (w) + λg (w)

∇wL (w, λ) = 2 1 1 4

• w +

1

• + 2λw

H (w, λ) = 2 + 2λ 1 1 4 + 2λ

• ∇Φ (w) =
• 2w1 + w2 + 1

w1 + 4w2

• min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0
• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Newton Iteration for Optimization - Example

Algorithm: Newton method

Input: guess w, λ while ∇L or g ≥ tol do Compute H (w, λ) , ∇g (w) , ∇Φ (w) , g (w) Compute Newton direction

• H

∇g ∇gT ∆w λ+

• = −

∇Φ g

• ∆λ = λ+ − λ

Compute Newton step, t ∈ ]0, 1] w ← w + t∆w, λ ← λ + t∆λ return w, λ min

w

1 2wT 2 1 1 4

• w + wT

1

• s.t. g (w) = wTw − 1 = 0

Guess λ = 0, step t = 1

• 2
• 1

1 2

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2

w1 w1 Your initial guess matters !!

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 17 / 32

Invertibility of the KKT matrix

The Newton direction of the KKT conditions

H (w, λ) ∇g(w) ∇g(w)T

• KKT matrix (symmetric indefinite)

∆w λ+

• = −

∇Φ(w) g(w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 18 / 32

Invertibility of the KKT matrix

The Newton direction of the KKT conditions

H (w, λ) ∇g(w) ∇g(w)T

• KKT matrix (symmetric indefinite)

∆w λ+

• = −

∇Φ(w) g(w)

• The KKT matrix is invertible if
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 18 / 32

Invertibility of the KKT matrix

The Newton direction of the KKT conditions

H (w, λ) ∇g(w) ∇g(w)T

• KKT matrix (symmetric indefinite)

∆w λ+

• = −

∇Φ(w) g(w)

• The KKT matrix is invertible if

∇g(w) is full column rank (LICQ)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 18 / 32

Invertibility of the KKT matrix

The Newton direction of the KKT conditions

H (w, λ) ∇g(w) ∇g(w)T

• KKT matrix (symmetric indefinite)

∆w λ+

• = −

∇Φ(w) g(w)

• The KKT matrix is invertible if

∇g(w) is full column rank (LICQ) ∀d = 0, such that ∇g(w)⊤d = 0 d⊤H (w, λ) d > 0 (SOSC)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 18 / 32

Invertibility of the KKT matrix

The Newton direction of the KKT conditions

H (w, λ) ∇g(w) ∇g(w)T

• KKT matrix (symmetric indefinite)

∆w λ+

• = −

∇Φ(w) g(w)

• The KKT matrix is invertible if

∇g(w) is full column rank (LICQ) ∀d = 0, such that ∇g(w)⊤d = 0 d⊤H (w, λ) d > 0 (SOSC) If (w, λ) is LICQ & SOSC, then the KKT matrix is invertible in a neighborhood of (w, λ)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 18 / 32

Invertibility of the KKT matrix

The Newton direction of the KKT conditions

H (w, λ) ∇g(w) ∇g(w)T

• KKT matrix (symmetric indefinite)

∆w λ+

• = −

∇Φ(w) g(w)

• The KKT matrix is invertible if

∇g(w) is full column rank (LICQ) ∀d = 0, such that ∇g(w)⊤d = 0 d⊤H (w, λ) d > 0 (SOSC) If (w, λ) is LICQ & SOSC, then the KKT matrix is invertible in a neighborhood of (w, λ) If LICQ & SOSC hold at the solution, then the Newton iteration is well defined in its neighborhood

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 18 / 32

Invertibility of the KKT matrix

The Newton direction of the KKT conditions

H (w, λ) ∇g(w) ∇g(w)T

• KKT matrix (symmetric indefinite)

∆w λ+

• = −

∇Φ(w) g(w)

• The KKT matrix is invertible if

∇g(w) is full column rank (LICQ) ∀d = 0, such that ∇g(w)⊤d = 0 d⊤H (w, λ) d > 0 (SOSC) If (w, λ) is LICQ & SOSC, then the KKT matrix is invertible in a neighborhood of (w, λ) In practice, when the solution fails LICQ/SOSC, it is common to

• bserve the solver struggling

numerically, as the KKT matrix becomes increasingly ill-conditioned !! If LICQ & SOSC hold at the solution, then the Newton iteration is well defined in its neighborhood

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 18 / 32

Quadratic model interpretation

Problem: min

w

Φ (w) s.t. g (w) = 0 The Newton direction is given by H(w, λ) ∇g (w) ∇g (w)T ∆w λ+

• = −

∇Φ (w) g (w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 19 / 32

Quadratic model interpretation

Problem: min

w

Φ (w) s.t. g (w) = 0 The Newton direction is given by H(w, λ) ∇g (w) ∇g (w)T ∆w λ+

• = −

∇Φ (w) g (w)

• The Newton direction is also given by the Quadratic Program (QP):

min

∆w

1 2∆wTH(w, λ)∆w + ∇Φ (w)T ∆w s.t. g (w) + ∇g (w)T ∆w = 0

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 19 / 32

Quadratic model interpretation

Problem: min

w

Φ (w) s.t. g (w) = 0 The Newton direction is given by H(w, λ) ∇g (w) ∇g (w)T ∆w λ+

• = −

∇Φ (w) g (w)

• The Newton direction is also given by the Quadratic Program (QP):

min

∆w

1 2∆wTH(w, λ)∆w + ∇Φ (w)T ∆w s.t. g (w) + ∇g (w)T ∆w = 0 Dual variables λ+ given by the dual variables of the QP, i.e. λ+ = λQP

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 19 / 32

Quadratic model interpretation

Problem: min

w

Φ (w) s.t. g (w) = 0 The Newton direction is given by H(w, λ) ∇g (w) ∇g (w)T ∆w λ+

• = −

∇Φ (w) g (w)

• The Newton direction is also given by the Quadratic Program (QP):

min

∆w

1 2∆wTH(w, λ)∆w + ∇Φ (w)T ∆w s.t. g (w) + ∇g (w)T ∆w = 0 Dual variables λ+ given by the dual variables of the QP, i.e. λ+ = λQP Proof: the KKT conditions of the QP are equivalent to the system providing the Newton direction

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 19 / 32

Quadratic model interpretation

Problem: min

w

Φ (w) s.t. g (w) = 0 The Newton direction is given by H(w, λ) ∇g (w) ∇g (w)T ∆w λ+

• = −

∇Φ (w) g (w)

• The Newton direction is also given by the Quadratic Program (QP):

min

∆w

1 2∆wTH(w, λ)∆w + ∇Φ (w)T ∆w s.t. g (w) + ∇g (w)T ∆w = 0 Dual variables λ+ given by the dual variables of the QP, i.e. λ+ = λQP Proof: the KKT conditions of the QP are equivalent to the system providing the Newton direction The Newton direction is given by solving a quadratic models of the original problem !!

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 19 / 32

Outline

1

KKT conditions - Quick Reminder

2

The Newton method

3

Newton on the KKT conditions

4

Sequential Quadratic Programming

5

Hessian approximation

6

Maratos effect

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 20 / 32

What about inequality constraints ?

Find the ”primal-dual” variables x, µ, λ such that: Primal Feasibility: g (w) = 0, h (w) ≤ 0, Dual Feasibility: ∇wL (w, µ, λ ) = 0, µ ≥ 0, Complementarity Slackness: µihi(w) = 0, i = 1, ...

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 21 / 32

What about inequality constraints ?

Find the ”primal-dual” variables x, µ, λ such that: Primal Feasibility: g (w) = 0, h (w) ≤ 0, Dual Feasibility: ∇wL (w, µ, λ ) = 0, µ ≥ 0, Complementarity Slackness: µihi(w) = 0, i = 1, ...

• 5
• 4
• 3
• 2
• 1

1

• 1

1 2 3 4 5

hi(w) µi hi(w) not active hi(w) active Solution manifold of µihi(w) = 0

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 21 / 32

What about inequality constraints ?

Find the ”primal-dual” variables x, µ, λ such that: Primal Feasibility: g (w) = 0, h (w) ≤ 0, Dual Feasibility: ∇wL (w, µ, λ ) = 0, µ ≥ 0, Complementarity Slackness: µihi(w) = 0, i = 1, ...

• 5
• 4
• 3
• 2
• 1

1

• 1

1 2 3 4 5

hi(w) µi hi(w) not active hi(w) active Solution manifold of µihi(w) = 0

Manifold generated by the Complementary Slackness condition is not smooth, Newton can not be used !!

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 21 / 32

Quadratic model interpretation

NLP min

w

Φ (w) s.t. g (w) = 0 The Newton direction is given by H(w, λ) ∇g (w) ∇g (w)T ∆w λ+

• = −

∇Φ (w) g (w)

• with H(w, λ) = ∇2

wL(w, λ)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 22 / 32

Quadratic model interpretation

NLP min

w

Φ (w) s.t. g (w) = 0 The Newton direction is given by H(w, λ) ∇g (w) ∇g (w)T ∆w λ+

• = −

∇Φ (w) g (w)

• with H(w, λ) = ∇2

wL(w, λ)

The Newton direction is given by the Quadratic Program (QP): min

∆w

1 2∆wTH(w, λ)∆w + ∇Φ (w)T ∆w s.t. g (w) + ∇g (w)T ∆w = 0

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 22 / 32

Quadratic model interpretation

NLP min

w

Φ (w) s.t. g (w) = 0 The Newton direction is given by H(w, λ) ∇g (w) ∇g (w)T ∆w λ+

• = −

∇Φ (w) g (w)

• with H(w, λ) = ∇2

wL(w, λ)

The Newton direction is given by the Quadratic Program (QP): min

∆w

1 2∆wTH(w, λ)∆w + ∇Φ (w)T ∆w s.t. g (w) + ∇g (w)T ∆w = 0 Dual variables λ+ given by the dual variables of the QP, i.e. λ+ = λQP

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 22 / 32

Quadratic interpretation for inequality constraints

Problem: min

w

Φ (w) s.t. g (w) = 0 s.t. h (w) ≤ 0

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 23 / 32

Quadratic interpretation for inequality constraints

Problem: min

w

Φ (w) s.t. g (w) = 0 s.t. h (w) ≤ 0 The Newton direction is given by the Quadratic Program (QP): min

∆w

1 2∆wTH(w, λ, µ)∆w + ∇Φ (w)T ∆w s.t. g (w) + ∇g (w)T ∆w = 0 h (w) + ∇h (w)T ∆w ≤ 0 with H(w, λ) = ∇2

wL(w, λ)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 23 / 32

Quadratic interpretation for inequality constraints

Problem: min

w

Φ (w) s.t. g (w) = 0 s.t. h (w) ≤ 0 The Newton direction is given by the Quadratic Program (QP): min

∆w

1 2∆wTH(w, λ, µ)∆w + ∇Φ (w)T ∆w s.t. g (w) + ∇g (w)T ∆w = 0 h (w) + ∇h (w)T ∆w ≤ 0 with H(w, λ) = ∇2

wL(w, λ)

Dual variables λ+ and µ+ given by the dual variables of the QP, i.e. λ+ = λQP, µ+ = µQP

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 23 / 32

SQP Algorithm

Algorithm: SQP with line-search

Input: guess w, λ, µ while ∇L∞ or g∞ or max (0, hi) ≥ tol do Compute g, h, ∇Φ(w), ∇g(w), ∇h(w), H (w, µ, λ) Compute Newton direction by solving the QP min

∆w

1 2∆wTH(w, λ, µ)∆w + ∇Φ (w)T ∆w s.t. g (w) + ∇g (w)T ∆w = 0 h (w) + ∇h (w)T ∆w ≤ 0 Select step size t to ensure progress (c.f. globalization / line-search) Take primal step: w ← w + t∆w Take dual step: λ ← (1 − t)λ + tλQP, µ ← (1 − t)µ + tµQP return w, λ, µ

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 24 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0 QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0 µ2 = 0

• Res. = 2.21

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0 µ2 = 0

• Res. = 2.21

Linearized constraints

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0 µ2 = 0

• Res. = 2.21

Contours of QP cost

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0 µ2 = 0

• Res. = 2.21

Step with t = 1

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0.48789 µ2 = 0.4285

• Res. = 0.4025

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0.48789 µ2 = 0.4285

• Res. = 0.4025

Linearized constraints

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0.48789 µ2 = 0.4285

• Res. = 0.4025

Contours of QP cost

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0.48789 µ2 = 0.4285

• Res. = 0.4025

Step with t = 1

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0.60331 µ2 = 0.25659

• Res. = 0.12986

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0.60331 µ2 = 0.25659

• Res. = 0.12986

Linearized constraints

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0.60331 µ2 = 0.25659

• Res. = 0.12986

Contours of QP cost

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0.60331 µ2 = 0.25659

• Res. = 0.12986

Step with t = 1

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0.68358 µ2 = 0.16672

• Res. = 0.013072

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0.68358 µ2 = 0.16672

• Res. = 0.013072

Linearized constraints

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0.68358 µ2 = 0.16672

• Res. = 0.013072

Contours of QP cost

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0.68358 µ2 = 0.16672

• Res. = 0.013072

Step with t = 1

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0.68999 µ2 = 0.16001

• Res. = 3.0916e-05

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0.68999 µ2 = 0.16001

• Res. = 3.0916e-05

Linearized constraints

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0.68999 µ2 = 0.16001

• Res. = 3.0916e-05

Contours of QP cost

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0.68999 µ2 = 0.16001

• Res. = 3.0916e-05

Step with t = 1

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

µ1 = 0.69 µ2 = 0.16

• Res. = 6.0624e-11

QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

SQP - Illustration

NLP: min

w

1 2 w − w02

Q

s.t. h (w) ≤ 0

1 2 3 4 5 6 10-10 10-5 100 1 2 3 4 5 0.5 1

Res

Iteration Iteration

Step-size t QP: min

w

1 2∆w⊤H (w, µ) ∆w + ∇Φ (w)⊤ ∆w s.t. h (w) + ∇h (w)⊤ ∆w ≤ 0 Hessian: H (w, µ) = ∇2

wΦ (w) + ∇2 w

• µ⊤h (w)
• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 25 / 32

Some remarks

The Newton direction is given by the Quadratic Program (QP): min

∆w

1 2∆wTH(w, λ, µ)∆w + ∇Φ (w)T ∆w s.t. g (w) + ∇g (w)T ∆w = 0 h (w) + ∇h (w)T ∆w ≤ 0 with H(w, λ) = ∇2

wL(w, λ)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 26 / 32

Some remarks

The Newton direction is given by the Quadratic Program (QP): min

∆w

1 2∆wTH(w, λ, µ)∆w + ∇Φ (w)T ∆w s.t. g (w) + ∇g (w)T ∆w = 0 h (w) + ∇h (w)T ∆w ≤ 0 with H(w, λ) = ∇2

wL(w, λ)

SQP inherits the convergence properties of the Newton method

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 26 / 32

Some remarks

The Newton direction is given by the Quadratic Program (QP): min

∆w

1 2∆wTH(w, λ, µ)∆w + ∇Φ (w)T ∆w s.t. g (w) + ∇g (w)T ∆w = 0 h (w) + ∇h (w)T ∆w ≤ 0 with H(w, λ) = ∇2

wL(w, λ)

SQP inherits the convergence properties of the Newton method What happens if SOSC fails during the iterations ? I.e. for an iterate w, λ, µ: d⊤H(w, λ, µ)d ≯ 0 for some d = 0 being a critical feasible direction ?

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 26 / 32

Some remarks

The Newton direction is given by the Quadratic Program (QP): min

∆w

1 2∆wTH(w, λ, µ)∆w + ∇Φ (w)T ∆w s.t. g (w) + ∇g (w)T ∆w = 0 h (w) + ∇h (w)T ∆w ≤ 0 with H(w, λ) = ∇2

wL(w, λ)

SQP inherits the convergence properties of the Newton method What happens if SOSC fails during the iterations ? I.e. for an iterate w, λ, µ: d⊤H(w, λ, µ)d ≯ 0 for some d = 0 being a critical feasible direction ? QP unbounded !! Heuristics are used in SQP methods to modify H(w, λ, µ) and recover an adequate curvature in the QP cost (regularization).

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 26 / 32

Outline

1

KKT conditions - Quick Reminder

2

The Newton method

3

Newton on the KKT conditions

4

Sequential Quadratic Programming

5

Hessian approximation

6

Maratos effect

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 27 / 32

Newton-type Methods - Gauss-Newton Hessian approximation

Cost function of the type Φ(w) = 1

2R(w)2, with R(w) ∈ I

Rm

Gauss-Newton Hessian approximation

Observe that ∇2

wΦ (w) =

∂ ∂w (∇R(w)R(w)) = ∇R(w)∇R(w)⊤ +

m

• i=1

∇2Ri(w)Ri(w)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 28 / 32

Newton-type Methods - Gauss-Newton Hessian approximation

Cost function of the type Φ(w) = 1

2R(w)2, with R(w) ∈ I

Rm

Gauss-Newton Hessian approximation

Observe that ∇2

wΦ (w) =

∂ ∂w (∇R(w)R(w)) = ∇R(w)∇R(w)⊤ +

m

• i=1

∇2Ri(w)Ri(w) Gauss-Newton method proposes to use: Bk = ∇R(wk)∇R(wk)T + αkI

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 28 / 32

Newton-type Methods - Gauss-Newton Hessian approximation

Cost function of the type Φ(w) = 1

2R(w)2, with R(w) ∈ I

Rm

Gauss-Newton Hessian approximation

Observe that ∇2

wΦ (w) =

∂ ∂w (∇R(w)R(w)) = ∇R(w)∇R(w)⊤ +

m

• i=1

∇2Ri(w)Ri(w) Gauss-Newton method proposes to use: Bk = ∇R(wk)∇R(wk)T + αkI Bk is a good approximation if:

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 28 / 32

Newton-type Methods - Gauss-Newton Hessian approximation

Cost function of the type Φ(w) = 1

2R(w)2, with R(w) ∈ I

Rm

Gauss-Newton Hessian approximation

Observe that ∇2

wΦ (w) =

∂ ∂w (∇R(w)R(w)) = ∇R(w)∇R(w)⊤ +

m

• i=1

∇2Ri(w)Ri(w) Gauss-Newton method proposes to use: Bk = ∇R(wk)∇R(wk)T + αkI Bk is a good approximation if: constraints are close to linear or Φ(w⋆) ≈ 0 (implies λ, µ ≈ 0)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 28 / 32

Newton-type Methods - Gauss-Newton Hessian approximation

Cost function of the type Φ(w) = 1

2R(w)2, with R(w) ∈ I

Rm

Gauss-Newton Hessian approximation

Observe that ∇2

wΦ (w) =

∂ ∂w (∇R(w)R(w)) = ∇R(w)∇R(w)⊤ +

m

• i=1

∇2Ri(w)Ri(w) Gauss-Newton method proposes to use: Bk = ∇R(wk)∇R(wk)T + αkI Bk is a good approximation if: constraints are close to linear or Φ(w⋆) ≈ 0 (implies λ, µ ≈ 0) and

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 28 / 32

Newton-type Methods - Gauss-Newton Hessian approximation

Cost function of the type Φ(w) = 1

2R(w)2, with R(w) ∈ I

Rm

Gauss-Newton Hessian approximation

Observe that ∇2

wΦ (w) =

∂ ∂w (∇R(w)R(w)) = ∇R(w)∇R(w)⊤ +

m

• i=1

∇2Ri(w)Ri(w) Gauss-Newton method proposes to use: Bk = ∇R(wk)∇R(wk)T + αkI Bk is a good approximation if: constraints are close to linear or Φ(w⋆) ≈ 0 (implies λ, µ ≈ 0) and all ∇2Ri(w) are small (R close to linear), or all Ri(w) are small, i.e. Φ(w⋆) ≈ 0

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 28 / 32

Newton-type Methods - Gauss-Newton Hessian approximation

Cost function of the type Φ(w) = 1

2R(w)2, with R(w) ∈ I

Rm

Gauss-Newton Hessian approximation

Observe that ∇2

wΦ (w) =

∂ ∂w (∇R(w)R(w)) = ∇R(w)∇R(w)⊤ +

m

• i=1

∇2Ri(w)Ri(w) Gauss-Newton method proposes to use: Bk = ∇R(wk)∇R(wk)T + αkI Bk is a good approximation if: constraints are close to linear or Φ(w⋆) ≈ 0 (implies λ, µ ≈ 0) and all ∇2Ri(w) are small (R close to linear), or all Ri(w) are small, i.e. Φ(w⋆) ≈ 0 Typical application to tracking & fitting problems: R(w) = y(w) − ¯ y

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 28 / 32

Newton-type Methods - Gauss-Newton Hessian approximation

Cost function of the type Φ(w) = 1

2R(w)2, with R(w) ∈ I

Rm

Gauss-Newton Hessian approximation

Observe that ∇2

wΦ (w) =

∂ ∂w (∇R(w)R(w)) = ∇R(w)∇R(w)⊤ +

m

• i=1

∇2Ri(w)Ri(w) Gauss-Newton method proposes to use: Bk = ∇R(wk)∇R(wk)T + αkI Bk is a good approximation if: constraints are close to linear or Φ(w⋆) ≈ 0 (implies λ, µ ≈ 0) and all ∇2Ri(w) are small (R close to linear), or all Ri(w) are small, i.e. Φ(w⋆) ≈ 0 Typical application to tracking & fitting problems: R(w) = y(w) − ¯ y

Convergence

If Φ (wk) → 0 then κk → 0 Can get superlinear convergence...

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 28 / 32

Newton-type Methods - BFGS

Compute numerical derivative of H(w) in an efficient (iterative) way

BFGS

Define sk = wk+1 − wk yk = ∇L(wk+1) − ∇L(wk) Idea: Update Bk → Bk+1 such that Bk+1sk = yk (secant condition)

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 29 / 32

Newton-type Methods - BFGS

Compute numerical derivative of H(w) in an efficient (iterative) way

BFGS

Define sk = wk+1 − wk yk = ∇L(wk+1) − ∇L(wk) Idea: Update Bk → Bk+1 such that Bk+1sk = yk (secant condition) BFGS formula: Bk+1 = Bk − BkssTBk sTBks + yyT sTy , B0 = I

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 29 / 32

Newton-type Methods - BFGS

Compute numerical derivative of H(w) in an efficient (iterative) way

BFGS

Define sk = wk+1 − wk yk = ∇L(wk+1) − ∇L(wk) Idea: Update Bk → Bk+1 such that Bk+1sk = yk (secant condition) BFGS formula: Bk+1 = Bk − BkssTBk sTBks + yyT sTy , B0 = I See ”Powell’s trick” to make sure that Bk+1 > 0

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 29 / 32

Newton-type Methods - BFGS

Compute numerical derivative of H(w) in an efficient (iterative) way

BFGS

Define sk = wk+1 − wk yk = ∇L(wk+1) − ∇L(wk) Idea: Update Bk → Bk+1 such that Bk+1sk = yk (secant condition) BFGS formula: Bk+1 = Bk − BkssTBk sTBks + yyT sTy , B0 = I See ”Powell’s trick” to make sure that Bk+1 > 0

Convergence

It can be shown that Bk → H(w), then κk → 0 Can get superlinear convergence...

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 29 / 32

Outline

1

KKT conditions - Quick Reminder

2

The Newton method

3

Newton on the KKT conditions

4

Sequential Quadratic Programming

5

Hessian approximation

6

Maratos effect

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 30 / 32

Maratos effect - Some NLPs can yield ”creeping” convergence

5 10 15 20 25 30 35 10-10 10-5 100 5 10 15 20 25 30 35 0.5 1

Res

Iteration Iteration

Step-size t

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 31 / 32

Maratos effect - Some NLPs can yield ”creeping” convergence

5 10 15 20 25 30 35 10-10 10-5 100 5 10 15 20 25 30 35 0.5 1

Res

Iteration Iteration

Step-size t What is going on ?!? This is a case of the Maratos effect, can happen with nonlinear constraints...

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 31 / 32

Maratos effect

Consider the NLP : min

u,v

Φ = 3v 2 − 2u s.t. g = u − v 2 = 0 Optimum: w∗ =

• .

Consider the iterate: wk =

• a2

a

• The Newton step is:

∆wk = −

• 2a2

a

• for λ = 2...

Define: w =

• u

v

• −0.25

−0.2 −0.15 −0.1 −0.05 0.05 0.1 0.15 0.2 0.25 −0.25 −0.2 −0.15 −0.1 −0.05 0.05 0.1 0.15 0.2 0.25

wk wk+1 ∆wk w∗ g = 0 u v

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 32 / 32

Maratos effect

Consider the NLP : min

u,v

Φ = 3v 2 − 2u s.t. g = u − v 2 = 0 Optimum: w∗ =

• .

Consider the iterate: wk =

• a2

a

• The Newton step is:

∆wk = −

• 2a2

a

• for λ = 2...

The full Newton step: wk+1 = wk + ∆wk is much closer to w∗ than wk. Define: w =

• u

v

• −0.25

−0.2 −0.15 −0.1 −0.05 0.05 0.1 0.15 0.2 0.25 −0.25 −0.2 −0.15 −0.1 −0.05 0.05 0.1 0.15 0.2 0.25

wk wk+1 ∆wk w∗ g = 0 u v

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 32 / 32

Maratos effect

Consider the NLP : min

u,v

Φ = 3v 2 − 2u s.t. g = u − v 2 = 0 Optimum: w∗ =

• .

Consider the iterate: wk =

• a2

a

• The Newton step is:

∆wk = −

• 2a2

a

• for λ = 2...

The full Newton step: wk+1 = wk + ∆wk is much closer to w∗ than wk. Define: w =

• u

v

• −0.25

−0.2 −0.15 −0.1 −0.05 0.05 0.1 0.15 0.2 0.25 −0.25 −0.2 −0.15 −0.1 −0.05 0.05 0.1 0.15 0.2 0.25

wk wk+1 ∆wk w∗ g = 0 u v But: Φ(wk+1) > Φ(wk) |g(wk+1)| > |g(wk)| No penalty function can accept ∆wk !!

• S. Gros

Optimal Control with DAEs, lecture 5 17th of February, 2016 32 / 32