SLIDE 1
Line Search 2 Lecture 4 ME EN 575 Andrew Ning aning@byu.edu - - PDF document
Line Search 2 Lecture 4 ME EN 575 Andrew Ning aning@byu.edu - - PDF document
Line Search 2 Lecture 4 ME EN 575 Andrew Ning aning@byu.edu Outline Root Finding Methods 1D Optimization Methods Root Finding Methods Root Finding Methods How do we know when we have reached a local minimum? Bisection Example:
SLIDE 2
SLIDE 3
Bisection Example: Refrigeration Tank
Minimize the cost of a cylindrical refrigeration tank with a volume of 50 m3.
- Circular ends cost $10 per m2
- Cylindrical walls cost $6 per m2
- Refrigerator costs $80 per m2 over its life
SLIDE 4
minimize 45πd2 + 17200 d with respect to d subject to d ≥ 0
Newton’s Method
We can do better by using gradient information
SLIDE 5
1 2 3 x f(x)
xk+1 = xk − f ′(xk) f ′′(xk)
SLIDE 6
f(x) x 1 2 3 x f(x) 2 1
Brent’s Method
SLIDE 7
1D Optimization Methods Golden Section Search
SLIDE 8
Extreme 1: bisection-like Extreme 2: small improvement
SLIDE 9
I1 I2 I2 I2 = τI1 I1 I2 I2 I3 I3 I3 = τI2
SLIDE 10
I1 = I2 + I3
Polynomial Methods
Approximate function locally as a polynomial (in this case quadratic): ˜ f = 1 2ax2 + bx + c If a > 0, the minimum of this function is x∗ = −b/a.
SLIDE 11
Brent’s Method
Combines quadratic polynomial method with golden section search.
Where are we going?
Optimization N-D Optimization 1-D Optimization
- Bisection Search
- Fibonnacci Search
- Golden Section Search
- Newton’s Method
- Polynomial Interpolation
- Brent’s Method
Unconstrained Constrained
- Lagrange Multipliers
- Exterior Penalty Methods
- Interior Point Methods
- SQP
Line-search Methods
- Steepest Descent
- Conjugate Gradient
- Newton’s Method
- Quasi-Newton Methods
Smooth Non-smooth
- Nelder-Mead Simplex
- Genetic Algorithms
Trust-region Methods
- Polynomial Fits
- Nonparametric Fits
SLIDE 12
Intuition in Higher Dimensions
Consider a hypersphere inscribed inside a hypercube volume of sphere ? volume of cube
1 2 3 4 5 6 7 8 9 dimension 0.0 0.2 0.4 0.6 0.8 1.0 Vsphere/Vcube