CS 287 Advanced Robotics (Fall 2019) Lecture 7: Constrained - PowerPoint PPT Presentation

CS 287 Advanced Robotics (Fall 2019) Lecture 7: Constrained Optimization Pieter Abbeel UC Berkeley EECS [optional] Boyd and Vandenberghe, Convex Optimization, Chapters 9 – 11 [optional] Nocedal and Wright, Chapter 18

Outline n Constrained Optimization n Penalty Formulation n Convex Programs and Solvers n Dual Descent

Outline n Constrained Optimization n Penalty Formulation n Convex Programs and Solvers n Dual Descent

Constrained Optimization min g 0 ( x ) x s . t . g i ( x ) ≤ 0 ∀ i h j ( x ) = 0 ∀ j

Outline n Constrained Optimization n Penalty Formulation n Convex Programs and Solvers n Dual Descent

Penalty Formulation Original: Penalty Formulation: | g i ( x ) | + + µ min g 0 ( x ) X X min g 0 ( x ) + µ | h j ( x ) | x x i j s . t . g i ( x ) ≤ 0 ∀ i h j ( x ) = 0 ∀ j - constrained - now unconstrained - same solution for mu large enough

Penalty Method n Inner loop: optimize merit function merit function | g i ( x ) | + + µ X X min g 0 ( x ) + µ | h j ( x ) | = min f µ ( x ) x x i j and increase μ in an outer loop until the two sums equal zero. n Inner loop optimization can be done by any of: n Gradient descent n Newton or quasi-Newton method n Trust region method

Penalty Method w/Trust Region Inner Loop n Inner loop: optimize merit function merit function | g i ( x ) | + + µ X X min g 0 ( x ) + µ | h j ( x ) | = min f µ ( x ) x x i j and increase μ in an outer loop until the two sums equal zero. n Trust region method repeatedly solves: x ) | + + µ X X min g 0 (¯ x ) + r x g 0 (¯ x )( x � ¯ x ) + µ | g i (¯ x ) + r x g i (¯ x )( x � ¯ | h j (¯ x ) + r x h j (¯ x )( x � ¯ x ) | x i j (trust region constraint) : current point s . t . k x � ¯ x k 2  ε ¯ x

Inputs: x, µ = 1 , ε 0 , α ∈ (0 . 5 , 1) , β ∈ (0 , 1) , t ∈ (1 , ∞ ) ¯ x ) | + + X X WHILE ( AND ) | g i (¯ | h j (¯ x ) | ≥ δ µ < µ MAX i j // increase penalty coefficient for constraints; re-init trust region size µ ← tµ, ε ← ε 0 WHILE (1) // [2] loop that optimizes Compute terms of first-order approximations: g 0 (¯ x ) , r x g 0 (¯ x ) , g i (¯ x ) , r x g i (¯ x ) , h j (¯ x ) , r x h j (¯ x ) , 8 i, j WHILE (1) // [3] loop that does trust-region size search Call convex program solver to solve: IF THEN: Update AND Update (Grow) trust region: AND BREAK out of while [3] ε ← ε / β x ← ¯ ¯ x next? ELSE: No update to AND Update (Shrink) trust region: IF below some threshold THEN: BREAK out of while [3] and while [2] ε

Tweak: Retain Convex Terms Exactly n Non-convex optimization with convex parts separated: min f 0 ( x ) + g 0 ( x ) x with: s . t . f i ( x ) ≤ 0 ∀ i f i convex g k non-convex Ax − b = 0 ∀ j h l nonlinear g k ( x ) ≤ 0 ∀ k h l ( x ) = 0 ∀ l n Retain convex parts and in inner loop solve: | g k ( x ) | + + µ X X min f 0 ( x ) + g 0 ( x ) + µ | h l ( x ) | x k l s . t . f i ( x ) ≤ 0 ∀ i Ax − b = 0 ∀ j

Outline n Constrained Optimization n Penalty Formulation n Convex Programs and Solvers n Dual Descent

Convex Optimization Problems n Convex optimization problems are a special class of optimization problems, of the following form: x ∈ R n f 0 ( x ) min s . t . f i ( x ) ≤ 0 i = 1 , . . . , n Ax = b with f i (x) convex for i = 0, 1, …, n

Convex Functions n A function f is convex if and only if ∀ x 1 , x 2 ∈ Domain( f ) , ∀ t ∈ [0 , 1] : f ( tx 1 + (1 − t ) x 2 ) ≤ tf ( x 1 ) + (1 − t ) f ( x 2 ) Image source: wikipedia

Convex Functions • Unique minimum • Set of points for which f(x) <= a is convex Source: Thomas Jungblut’s Blog

Outline n Constrained Optimization n Penalty Formulation n Convex Programs and Solvers n Equality Constraints n Inequality Constraints n Dual Descent

Convex Problems: Equality Constrained Minimization n Problem to be solved: n We will cover three solution methods: n Elimination n Newton’s method n Infeasible start Newton method

Method 1: Elimination From linear algebra we know that there exist a matrix F (in fact infinitely many) n such that: : any solution to Ax = b F: spans the null-space of A A way to find an F: compute SVD of A, A = U S V’, for A having k nonzero singular values, set F = U(:, k+1:end) So we can solve the equality constrained minimization problem by solving an n unconstrained minimization problem over a new variable z : Potential cons: (i) need to first find a solution to Ax=b, (ii) need to find F, (iii) n elimination might destroy sparsity in original problem structure

Methods 2 and 3 --- First Consider Optimality Condition n Recall problem to be solved: x* with Ax*=b is (local) optimum if and only if: Equivalently:

Methods 2 and 3 --- First Consider Optimality Condition n Recall problem to be solved:

Method 2: Newton’s Method Problem to be solved: n n Assume x is feasible, i.e., satisfies Ax = b, now use 2 nd order approximation of f: n Optimality condition for 2 nd order approximation: n

Method 2: Newton’s Method With Newton step obtained by solving a linear system of equations: Feasible descent method:

Method 3: Infeasible Start Newton Method Problem to be solved: n n Use 1 st order approximation of the optimality conditions at current x: n Equivalently: n

Outline n Constrained Optimization n Penalty Formulation n Convex Programs and Solvers n Equality Constraints n Inequality Constraints: Barrier Method n Dual Descent

Convex Problems: Equality and Inequality Constrained Minimization n Recall the problem to be solved:

Equality and Inequality Constrained Minimization Approximation via logarithmic barrier: Problem to be solved: n n Reformulation via indicator function n * for t > 0, -(1/t) log(-u) is a smooth approximation of I_ (u) * approximation improves for t à infinity * better conditioned for smaller t à No inequality constraints anymore, but very poorly conditioned objective function

Equality and Inequality Constrained Minimization

Barrier Method Given: strictly feasible x, t=t (0) > 0, μ > 1, tolerance ε > 0 n Repeat n 1. Centering Step. Compute x * (t) by solving starting from x 2. Update. x := x * (t). 3. Stopping Criterion. Quit if m/t < ε 4. Increase t. t := μt

Example 1: Inequality Form LP

Example 2: Geometric Program

Example 3: Standard LPs

Initialization Basic phase I method: n Initialize by first solving: Easy to initialize above problem, pick some x such that Ax = b, and then simply set s = max i f i (x) n Can stop early---whenever s < 0 n

Initialization Sum of infeasibilities phase I method: n Initialize by first solving: n Easy to initialize above problem, pick some x such that Ax = b, and then simply set s i = n max(0, f i (x)) For infeasible problems, produces a solution that satisfies many more inequalities than n basic phase I method

Other methods for convex problems n We have covered a primal interior point method / barrier method n one of several optimization approaches n Examples of others: n Primal-dual interior point methods n Primal-dual infeasible interior point methods

Outline n Constrained Optimization n Penalty Formulation n Convex Programs and Solvers n Equality Constraints n Inequality Constraints: Barrier Method n Dual Descent

Formulation Dual-Descent Formulation: Original: Penalty Formulation: | g i ( x ) | + + µ min g 0 ( x ) X X min g 0 ( x ) + µ | h j ( x ) | x x i j s . t . g i ( x ) ≤ 0 ∀ i Penalty Method iterates: h j ( x ) = 0 ∀ j Dual Descent iterates: - Optimize over x - Optimize over x - Gradient descent step for lambda and nu - Increase mu as needed New, equivalent problem with same solution: Dual-Descent Formulation of new, equivalent problem almost identical to penalty formulation, but individual additive updates to lambda and nu, rather than scaling up of a single mu

Next Lecture Optimization-based Optimal Control! J