SLIDE 1
Next Problem
Consider Next Hardest Problem How could we adapt gradient projection
- r other linear equality constrained
Computational Optimization Constrained Optimization Algorithms - - PowerPoint PPT Presentation
Computational Optimization Constrained Optimization Algorithms - - PowerPoint PPT Presentation
Computational Optimization Constrained Optimization Algorithms Feasible Descent Methods Continued Next Problem Consider Next Hardest Problem min ( ) f x . . s t Ax b How could we adapt gradient projection or other linear
SLIDE 2
SLIDE 3
Change one item of working set at a time
Active Set Methods MW 16.5
SLIDE 4
Traverse interior of set (a little more later)
Interior point algorithms MW 16.6
SLIDE 5
Change many elements of working set at once
Gradient Projection MW 16.7
SLIDE 6
Gradient Projection Method NW 16.7
For equality constrained approach we projected the gradient back to the feasible region. We could do this for Inequalities too if projection is cheap
) (xk f ∇ −
xk
u x l t s c x Qx x ≤ ≤ + . . ' ' 2 1 min
SLIDE 7
Projection for bounds constraints
Projection is closest point in the set to x
⎪ ⎩ ⎪ ⎨ ⎧ > ≤ ≤ < = ≤ ≤ −
i i i i i i i i i i i i s
x u x if u u x if x x if s u s t s s x
- .
. || || 2 1 min
2
- u
SLIDE 8
Cauchy Point
Take a step The projected step is a function of t Cauchy point finds t that minimizes e.g. An exact linesearch along the projected direction
i i
tg x − ( ) ( )
i i
x t P x tg = − )) ( ( t x q
SLIDE 9
Plus: Cauchy point can change many constraints in working set
xk x(t) g
SLIDE 10
Gradient Projection Method for QP NW 16.5
Start with feasible x0 For x = 0,1,2,… if xk satisfies KKT then optimal Set x =xk and find cauchy point xc; xk+1 is an approximate solution of QP using active set of xc fixed and rest feasible. approximation just needs to find feasible decrease. End;
SLIDE 11
KKT have nice form
KKT of Lagrangian is Primal Feasibility Dual feasibility Complimentarity
min ( ) . . f x s t l x u ≤ ≤ ( , , ) ( ) '( ) ( ) L x f x x l u x λ α λ α = − − − − l x u ≤ ≤ ( ) 0, f x λ α λ α ∇ − + = ≥ ≥ ( ) ( ) 1,..,
i i i i i i
x l u x i n λ α − = − = =
SLIDE 12
KKT have nice form
Need So if complimentarity holds Point is optimal
( ) ( ) 1,..,
i i i i i i
x l u x i n λ α − = − = = ( ) ,
i i i i i
f x x α λ α λ ∂ = − ≥ ∂ ( ) ( ) 0, , ( ) ( ) 0, 0,
i i i i i i i i
f x f x So if x x f x f x if x x α λ α λ ∂ ∂ ≥ = = ∂ ∂ ∂ ∂ < = = − ∂ ∂
SLIDE 13