optimization introduction optimization
play

Optimization (Introduction) Optimization Goal: Find the minimizer - PowerPoint PPT Presentation

Optimization (Introduction) Optimization Goal: Find the minimizer that minimizes the objective (cost) function : " Unconstrained Optimization Optimization Goal: Find the minimizer that minimizes the


  1. Optimization (Introduction)

  2. Optimization Goal: Find the minimizer π’š βˆ— that minimizes the objective (cost) function 𝑔 π’š : β„› " β†’ β„› Unconstrained Optimization

  3. Optimization Goal: Find the minimizer π’š βˆ— that minimizes the objective (cost) function 𝑔 π’š : β„› " β†’ β„› Constrained Optimization

  4. Unconstrained Optimization β€’ What if we are looking for a maximizer π’š βˆ— ? 𝑔 π’š βˆ— = max 𝑔 π’š π’š

  5. Calculus problem: maximize the rectangle area subject to perimeter constraint max 𝒆 ∈ β„› !

  6. 𝐡𝑠𝑓𝑏 = 𝑒 $ 𝑒 % 𝑒 % 𝑒 $ 𝑄𝑓𝑠𝑗𝑛𝑓𝑒𝑓𝑠 = 2(𝑒 $ + 𝑒 % ) 𝑒 % 𝑒 $

  7. Unconstrained Optimization 1D

  8. What is the optimal solution? (1D) 𝑔 𝑦 βˆ— = min 𝑔 𝑦 " (First-order) Necessary condition (Second-order) Sufficient condition

  9. Types of optimization problems 𝑔 𝑦 βˆ— = min 𝑔: nonlinear, continuous 𝑔 𝑦 and smooth " Gradient-free methods Evaluate 𝑔 𝑦 Gradient (first-derivative) methods Evaluate 𝑔 𝑦 , 𝑔′ 𝑦 Second-derivative methods Evaluate 𝑔 𝑦 , 𝑔′ 𝑦 , 𝑔′′ 𝑦

  10. Does the solution exists? Local or global solution?

  11. Example (1D) Consider the function 𝑔 π’š = 7 ! 8 βˆ’ 7 " 9 βˆ’ 11 𝑦 : + 40𝑦 . Find the stationary point and check the sufficient condition 100 - 6 - 4 - 2 2 4 6 - 100 - 200

  12. Optimization in 1D: Golden Section Search β€’ Similar idea of bisection method for root finding β€’ Needs to bracket the minimum inside an interval β€’ Required the function to be unimodal A function 𝑔: β„› β†’ β„› is unimodal on an interval [𝑏, 𝑐] ΓΌ There is a unique π’š βˆ— ∈ [𝑏, 𝑐] such that 𝑔(π’š βˆ— ) is the minimum in [𝑏, 𝑐] ΓΌ For any 𝑦 ; , 𝑦 : ∈ [𝑏, 𝑐] with 𝑦 ; < 𝑦 : 𝑦 : < π’š βˆ— ⟹ 𝑔(𝑦 ; ) > 𝑔(𝑦 : ) Β§ 𝑦 ; > π’š βˆ— ⟹ 𝑔(𝑦 ; ) < 𝑔(𝑦 : ) Β§

  13. 𝑔 𝑔 $ $ 𝑔 𝑔 & & 𝑔 ' 𝑔 𝑔 % % 𝑔 ' 𝑦 $ 𝑦 $ 𝑦 & 𝑦 & 𝑦 % 𝑦 ' 𝑦 % 𝑦 ' 𝑑 𝑑 𝑏 𝑏 𝑐 𝑐

  14. Golden Section Search

  15. Golden Section Search What happens with the length of the interval after one iteration? β„Ž ! = 𝜐 β„Ž " Or in general: β„Ž #$! = 𝜐 β„Ž # Hence the interval gets reduced by 𝝊 (for bisection method to solve nonlinear equations, 𝜐 =0.5) For recursion: 𝜐 β„Ž ! = (1 βˆ’ 𝜐) β„Ž " 𝜐 𝜐 β„Ž " = (1 βˆ’ 𝜐) β„Ž " 𝜐 % = (1 βˆ’ 𝜐) 𝝊 = 𝟏. πŸ•πŸπŸ—

  16. Golden Section Search β€’ Derivative free method! β€’ Slow convergence: |𝑓 ?B; | lim = 0.618 𝑠 = 1 (π‘šπ‘—π‘œπ‘“π‘π‘  π‘‘π‘π‘œπ‘€π‘“π‘ π‘•π‘“π‘œπ‘‘π‘“) 𝑓 ? ?β†’A β€’ Only one function evaluation per iteration

  17. Example

  18. Newton’s Method Using Taylor Expansion, we can approximate the function 𝑔 with a quadratic function about 𝑦 C 𝑔 𝑦 β‰ˆ 𝑔 𝑦 C + 𝑔 D 𝑦 C (𝑦 βˆ’ 𝑦 C ) + ; : 𝑔 D β€² 𝑦 C (𝑦 βˆ’ 𝑦 C ) : And we want to find the minimum of the quadratic function using the first-order necessary condition

  19. Newton’s Method β€’ Algorithm: 𝑦 0 = starting guess 𝑦 123 = 𝑦 1 βˆ’ 𝑔′ 𝑦 1 /𝑔′′ 𝑦 1 β€’ Convergence: β€’ Typical quadratic convergence β€’ Local convergence (start guess close to solution) β€’ May fail to converge, or converge to a maximum or point of inflection

  20. Newton’s Method (Graphical Representation)

  21. Example Consider the function 𝑔 𝑦 = 4 𝑦 9 + 2 𝑦 : + 5 𝑦 + 40 If we use the initial guess 𝑦 C = 2 , what would be the value of 𝑦 after one iteration of the Newton’s method?

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend


More recommend