in case you missed it who am i
play

In case you missed it - Who am I ? Name: S ebastien Gros - PowerPoint PPT Presentation

Dec 18, 2023 •11 likes •1.97k views

In case you missed it - Who am I ? Name: S ebastien Gros Nationality: Swiss Residence: G oteborg, Sweden Affiliation: Chalmers University of Technology Department: Signals & Systems Position: Assistant Professor Email:

  1. Core idea Goal : solve r ( w ) = 0... how ?!? 0.2 Algorithm: Newton method Input : w , tol w while � r ( w ) � ∞ ≥ tol do 0 Compute r ( w ) r ( w ) and ∇ r ( w ) -0.2 Compute the Newton direction ∇ r ( w ) T ∆ w = − r ( w ) -0.4 Newton step w w ← w + ∆ w Key idea : guess w , iterate the linear model: return w r ( w + ∆ w ) ≈ r ( w ) + ∇ r ( w ) ⊤ ∆ w = 0 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 8 / 32

  2. Core idea Goal : solve r ( w ) = 0... how ?!? 0.2 Algorithm: Newton method Input : w , tol w while � r ( w ) � ∞ ≥ tol do 0 Compute r ( w ) r ( w ) and ∇ r ( w ) -0.2 Compute the Newton direction ∇ r ( w ) T ∆ w = − r ( w ) -0.4 Newton step w w ← w + ∆ w Key idea : guess w , iterate the linear model: return w r ( w + ∆ w ) ≈ r ( w ) + ∇ r ( w ) ⊤ ∆ w = 0 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 8 / 32

  3. Core idea Goal : solve r ( w ) = 0... how ?!? 0.2 Algorithm: Newton method Input : w , tol w while � r ( w ) � ∞ ≥ tol do 0 Compute r ( w ) r ( w ) and ∇ r ( w ) -0.2 Compute the Newton direction ∇ r ( w ) T ∆ w = − r ( w ) -0.4 Newton step w w ← w + ∆ w Key idea : guess w , iterate the linear model: return w r ( w + ∆ w ) ≈ r ( w ) + ∇ r ( w ) ⊤ ∆ w = 0 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 8 / 32

  4. Core idea Goal : solve r ( w ) = 0... how ?!? 0.2 Algorithm: Newton method Input : w , tol w while � r ( w ) � ∞ ≥ tol do 0 Compute r ( w ) r ( w ) and ∇ r ( w ) -0.2 Compute the Newton direction ∇ r ( w ) T ∆ w = − r ( w ) -0.4 Newton step w w ← w + ∆ w Key idea : guess w , iterate the linear model: return w r ( w + ∆ w ) ≈ r ( w ) + ∇ r ( w ) ⊤ ∆ w = 0 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 8 / 32

  5. Core idea Goal : solve r ( w ) = 0... how ?!? 0.2 Algorithm: Newton method Input : w , tol w while � r ( w ) � ∞ ≥ tol do 0 Compute r ( w ) r ( w ) and ∇ r ( w ) -0.2 Compute the Newton direction ∇ r ( w ) T ∆ w = − r ( w ) -0.4 Newton step w w ← w + ∆ w Key idea : guess w , iterate the linear model: return w r ( w + ∆ w ) ≈ r ( w ) + ∇ r ( w ) ⊤ ∆ w = 0 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 8 / 32

  6. Core idea Goal : solve r ( w ) = 0... how ?!? 0.2 Algorithm: Newton method Input : w , tol w while � r ( w ) � ∞ ≥ tol do 0 Compute r ( w ) r ( w ) and ∇ r ( w ) -0.2 Compute the Newton direction ∇ r ( w ) T ∆ w = − r ( w ) -0.4 Newton step w w ← w + ∆ w Key idea : guess w , iterate the linear model: return w r ( w + ∆ w ) ≈ r ( w ) + ∇ r ( w ) ⊤ ∆ w = 0 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 8 / 32

  7. Core idea Goal : solve r ( w ) = 0... how ?!? 0.2 Algorithm: Newton method Input : w , tol w while � r ( w ) � ∞ ≥ tol do 0 Compute r ( w ) r ( w ) and ∇ r ( w ) -0.2 Compute the Newton direction ∇ r ( w ) T ∆ w = − r ( w ) -0.4 Newton step w w ← w + ∆ w Key idea : guess w , iterate the linear model: return w r ( w + ∆ w ) ≈ r ( w ) + ∇ r ( w ) ⊤ ∆ w = 0 This is a full-step Newton iteration 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 8 / 32

  8. Core idea Goal : solve r ( w ) = 0... how ?!? 0.2 Algorithm: Newton method Input : w , tol w while � r ( w ) � ∞ ≥ tol do 0 Compute r ( w ) r ( w ) and ∇ r ( w ) -0.2 Compute the Newton direction ∇ r ( w ) T ∆ w = − r ( w ) -0.4 Newton step, t ∈ ]0 , 1] w w ← w + t ∆ w Key idea : guess w , iterate the linear model: return w r ( w + ∆ w ) ≈ r ( w ) + ∇ r ( w ) ⊤ ∆ w = 0 This is a full-step Newton iteration Reduced steps are often needed 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 8 / 32

  9. Why reduced steps ? Newton step with t ∈ ]0 , 1]: ∇ r ( w ) ⊤ ∆ w = − r ( w ) w ← w + t ∆ w 1.5 1 0.5 r ( w ) 0 -0.5 -1 -1.5 w 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 9 / 32

  10. Why reduced steps ? Newton step with t ∈ ]0 , 1]: ∇ r ( w ) ⊤ ∆ w = − r ( w ) w ← w + t ∆ w 1.5 1 t = 1 0.5 r ( w ) 0 w -0.5 -1 -1.5 w 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 9 / 32

  11. Why reduced steps ? Newton step with t ∈ ]0 , 1]: ∇ r ( w ) ⊤ ∆ w = − r ( w ) w ← w + t ∆ w 1.5 1 t = 1 0.5 r ( w ) w 0 -0.5 -1 -1.5 w 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 9 / 32

  12. Why reduced steps ? Newton step with t ∈ ]0 , 1]: ∇ r ( w ) ⊤ ∆ w = − r ( w ) w ← w + t ∆ w 1.5 1 t = 1 0.5 r ( w ) 0 w -0.5 -1 -1.5 w 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 9 / 32

  13. Why reduced steps ? Newton step with t ∈ ]0 , 1]: ∇ r ( w ) ⊤ ∆ w = − r ( w ) w ← w + t ∆ w 1.5 1 t = 1 0.5 r ( w ) w 0 -0.5 -1 -1.5 w 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 9 / 32

  14. Why reduced steps ? Newton step with t ∈ ]0 , 1]: ∇ r ( w ) ⊤ ∆ w = − r ( w ) w ← w + t ∆ w 1.5 1 t = 1 0.5 r ( w ) 0 w -0.5 -1 -1.5 w 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 9 / 32

  15. Why reduced steps ? Newton step with t ∈ ]0 , 1]: ∇ r ( w ) ⊤ ∆ w = − r ( w ) w ← w + t ∆ w 1.5 1 t = 1 0.5 r ( w ) w 0 -0.5 -1 -1.5 w 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 9 / 32

  16. Why reduced steps ? Newton step with t ∈ ]0 , 1]: ∇ r ( w ) ⊤ ∆ w = − r ( w ) w ← w + t ∆ w 1.5 1 t = 1 0.5 r ( w ) w 0 -0.5 -1 -1.5 w The full-step Newton iteration can be unstable !! 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 9 / 32

  17. Why reduced steps ? Newton step with t ∈ ]0 , 1]: ∇ r ( w ) ⊤ ∆ w = − r ( w ) w ← w + t ∆ w 1.5 1 t = 0.8 0.5 r ( w ) 0 w -0.5 -1 -1.5 w The full-step Newton iteration can be unstable !! 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 9 / 32

  18. Why reduced steps ? Newton step with t ∈ ]0 , 1]: ∇ r ( w ) ⊤ ∆ w = − r ( w ) w ← w + t ∆ w 1.5 1 t = 0.8 0.5 r ( w ) w 0 -0.5 -1 -1.5 w The full-step Newton iteration can be unstable !! 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 9 / 32

  19. Why reduced steps ? Newton step with t ∈ ]0 , 1]: ∇ r ( w ) ⊤ ∆ w = − r ( w ) w ← w + t ∆ w 1.5 1 t = 0.8 0.5 r ( w ) 0 w -0.5 -1 -1.5 w The full-step Newton iteration can be unstable !! 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 9 / 32

  20. Why reduced steps ? Newton step with t ∈ ]0 , 1]: ∇ r ( w ) ⊤ ∆ w = − r ( w ) w ← w + t ∆ w 1.5 1 t = 0.8 0.5 r ( w ) 0 w -0.5 -1 -1.5 w The full-step Newton iteration can be unstable !! While the reduced-steps Newton iteration is stable... 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 9 / 32

  21. Does Newton always work ? Is the Newton step ∆ w always providing a direction ”improving” r ( w ) ? 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 10 / 32

  22. Does Newton always work ? Is the Newton step ∆ w always providing a direction ”improving” r ( w ) ? I.e. is there always a t > 0 s.t. � r ( w + t ∆ w ) � < � r ( w ) � is true ? 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 10 / 32

  23. Does Newton always work ? Is the Newton step ∆ w always providing a direction ”improving” r ( w ) ? I.e. is there always a t > 0 s.t. � r ( w + t ∆ w ) � < � r ( w ) � is true ? Yes... but 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 10 / 32

  24. Does Newton always work ? Is the Newton step ∆ w always providing a direction ”improving” r ( w ) ? I.e. is there always a t > 0 s.t. � r ( w + t ∆ w ) � < � r ( w ) � is true ? Yes... but Proof : � r ( w + t ∆ w ) � < � r ( w ) � holds for some t > 0 if � d � d t � r ( w + t ∆ w ) � 2 < 0 � � t =0 with � r ( w ) � 2 differentiable. 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 10 / 32

  25. Does Newton always work ? Is the Newton step ∆ w always providing a direction ”improving” r ( w ) ? I.e. is there always a t > 0 s.t. � r ( w + t ∆ w ) � < � r ( w ) � is true ? Yes... but Proof : � r ( w + t ∆ w ) � < � r ( w ) � holds for some t > 0 if � d � d t � r ( w + t ∆ w ) � 2 < 0 � � t =0 with � r ( w ) � 2 differentiable. I.e. 2 r ( w ) T d d t r ( w + t ∆ w ) t =0 < 0 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 10 / 32

  26. Does Newton always work ? Is the Newton step ∆ w always providing a direction ”improving” r ( w ) ? I.e. is there always a t > 0 s.t. � r ( w + t ∆ w ) � < � r ( w ) � is true ? Yes... but Proof : � r ( w + t ∆ w ) � < � r ( w ) � holds for some t > 0 if � d � d t � r ( w + t ∆ w ) � 2 < 0 � � t =0 with � r ( w ) � 2 differentiable. I.e. 2 r ( w ) T d d t r ( w + t ∆ w ) t =0 < 0 We have d d t r ( w + t ∆ w ) t =0 = ∇ r ( w ) T ∆ w = −∇ r ( w ) T ∇ r ( w ) − T r ( w ) = − r ( w ) 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 10 / 32

  27. Does Newton always work ? Is the Newton step ∆ w always providing a direction ”improving” r ( w ) ? I.e. is there always a t > 0 s.t. � r ( w + t ∆ w ) � < � r ( w ) � is true ? Yes... but Proof : � r ( w + t ∆ w ) � < � r ( w ) � holds for some t > 0 if � d � d t � r ( w + t ∆ w ) � 2 < 0 � � t =0 with � r ( w ) � 2 differentiable. I.e. 2 r ( w ) T d d t r ( w + t ∆ w ) t =0 < 0 We have d d t r ( w + t ∆ w ) t =0 = ∇ r ( w ) T ∆ w = −∇ r ( w ) T ∇ r ( w ) − T r ( w ) = − r ( w ) Then � d � = − 2 � r ( w ) � 2 < 0 d t � r ( w + t ∆ w ) � 2 � � t =0 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 10 / 32

  28. Does Newton always work ? Is the Newton step ∆ w always providing a direction ”improving” r ( w ) ? I.e. is there always a t > 0 s.t. � r ( w + t ∆ w ) � < � r ( w ) � is true ? Yes... but How to select the step size t ∈ ]0 , 1] ? Globalization... Line-search : reduce t until some criteria of progression on � r � are met Trust region : confine the step ∆ w within a region where ∇ r ( w ) provides a good model of r ( w ) Filter techniques : monitor progress on specific components of r ( w ) separately ... ... ensures that progress is made in one way or another . Note: most of these techniques are specific to optimization. 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 10 / 32

  29. But still, Newton can fail... Solve r ( w ) = 0 0.6 0.4 0.2 r(w) 0 w -0.2 -0.4 w 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 11 / 32

  30. But still, Newton can fail... Solve r ( w ) = 0 0.6 0.4 0.2 r(w) 0 w -0.2 -0.4 w 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 11 / 32

  31. But still, Newton can fail... Solve r ( w ) = 0 0.6 0.4 0.2 r(w) 0 w -0.2 -0.4 w 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 11 / 32

  32. But still, Newton can fail... Solve r ( w ) = 0 0.6 0.4 0.2 r(w) 0 w -0.2 -0.4 w 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 11 / 32

  33. But still, Newton can fail... Solve r ( w ) = 0 0.6 0.4 0.2 r(w) 0 w -0.2 -0.4 w 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 11 / 32

  34. But still, Newton can fail... Solve r ( w ) = 0 0.6 0.4 0.2 r(w) 0 w -0.2 -0.4 w 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 11 / 32

  35. But still, Newton can fail... Solve r ( w ) = 0 0.6 0.4 0.2 r(w) 0 w -0.2 -0.4 w 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 11 / 32

  36. But still, Newton can fail... Solve r ( w ) = 0 Newton stops with r ( w ) � = 0 and ∇ r ( w ) singular 0.6 i.e. the Newton direction ∆ w given by 0.4 ∇ r ( w ) ⊤ ∆ w = − r ( w ) 0.2 r(w) is undefined... 0 w -0.2 -0.4 w 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 11 / 32

  37. But still, Newton can fail... Solve r ( w ) = 0 Newton stops with r ( w ) � = 0 and ∇ r ( w ) singular 0.6 i.e. the Newton direction ∆ w given by 0.4 ∇ r ( w ) ⊤ ∆ w = − r ( w ) 0.2 r(w) is undefined... 0 w -0.2 -0.4 w This is a common failure mode for Newton-based solvers when tackling very non-linear r and starting with a poor initial guess !! 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 11 / 32

  38. Convergence of full-step Newton methods Newton method: ∇ r ( w ) ⊤ ∆ w = − r ( w ) w ← w + ∆ w 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 12 / 32

  39. Convergence of full-step Newton methods Newton method: ∇ r ( w ) ⊤ ∆ w = − r ( w ) w ← w + ∆ w Yields the iteration k = 0 , 1 , .... : w k +1 ← w k − ∇ r ( w k ) −⊤ r ( w k ) 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 12 / 32

  40. Convergence of full-step Newton methods Newton method: Newton-type method (Jacobian approx.) ∇ r ( w ) ⊤ ∆ w = − r ( w ) M ∆ w = − r ( w ) w ← w + ∆ w w ← w + ∆ w Yields the iteration k = 0 , 1 , .... : w k +1 ← w k − ∇ r ( w k ) −⊤ r ( w k ) 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 12 / 32

  41. Convergence of full-step Newton methods Newton method: Newton-type method (Jacobian approx.) ∇ r ( w ) ⊤ ∆ w = − r ( w ) M ∆ w = − r ( w ) w ← w + ∆ w w ← w + ∆ w Yields the iteration k = 0 , 1 , .... : Yields the iteration k = 0 , 1 , .... : w k +1 ← w k − ∇ r ( w k ) −⊤ r ( w k ) w k +1 ← w k − M − 1 k r ( w k ) 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 12 / 32

  42. Convergence of full-step Newton methods Newton method: Newton-type method (Jacobian approx.) ∇ r ( w ) ⊤ ∆ w = − r ( w ) M ∆ w = − r ( w ) w ← w + ∆ w w ← w + ∆ w Yields the iteration k = 0 , 1 , .... : Yields the iteration k = 0 , 1 , .... : w k +1 ← w k − ∇ r ( w k ) −⊤ r ( w k ) w k +1 ← w k − M − 1 k r ( w k ) Theorem : assume � � ∇ r ( w ) T − ∇ r ( w ∗ ) T �� � � M − 1 � � ≤ ω � w − w ∗ � , for Nonlinearity of r : k w ∈ [ w k , w ⋆ ] 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 12 / 32

  43. Convergence of full-step Newton methods Newton method: Newton-type method (Jacobian approx.) ∇ r ( w ) ⊤ ∆ w = − r ( w ) M ∆ w = − r ( w ) w ← w + ∆ w w ← w + ∆ w Yields the iteration k = 0 , 1 , .... : Yields the iteration k = 0 , 1 , .... : w k +1 ← w k − ∇ r ( w k ) −⊤ r ( w k ) w k +1 ← w k − M − 1 k r ( w k ) Theorem : assume � � ∇ r ( w ) T − ∇ r ( w ∗ ) T �� � � M − 1 � ≤ ω � w − w ∗ � , for � Nonlinearity of r : k w ∈ [ w k , w ⋆ ] � � k ( ∇ r ( w k ) T − M k ) � � M − 1 � Jacobian approximation error: � ≤ κ k < 1 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 12 / 32

  44. Convergence of full-step Newton methods Newton method: Newton-type method (Jacobian approx.) ∇ r ( w ) ⊤ ∆ w = − r ( w ) M ∆ w = − r ( w ) w ← w + ∆ w w ← w + ∆ w Yields the iteration k = 0 , 1 , .... : Yields the iteration k = 0 , 1 , .... : w k +1 ← w k − ∇ r ( w k ) −⊤ r ( w k ) w k +1 ← w k − M − 1 k r ( w k ) Theorem : assume � � ∇ r ( w ) T − ∇ r ( w ∗ ) T �� � � M − 1 � � ≤ ω � w − w ∗ � , for Nonlinearity of r : k w ∈ [ w k , w ⋆ ] � � k ( ∇ r ( w k ) T − M k ) � M − 1 � � Jacobian approximation error: � ≤ κ k < 1 Good initial guess � w 0 − w ∗ � ≤ 2 ω (1 − max { κ k } ) 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 12 / 32

  45. Convergence of full-step Newton methods Newton method: Newton-type method (Jacobian approx.) ∇ r ( w ) ⊤ ∆ w = − r ( w ) M ∆ w = − r ( w ) w ← w + ∆ w w ← w + ∆ w Yields the iteration k = 0 , 1 , .... : Yields the iteration k = 0 , 1 , .... : w k +1 ← w k − ∇ r ( w k ) −⊤ r ( w k ) w k +1 ← w k − M − 1 k r ( w k ) Theorem : assume � � ∇ r ( w ) T − ∇ r ( w ∗ ) T �� � � M − 1 � � ≤ ω � w − w ∗ � , for Nonlinearity of r : k w ∈ [ w k , w ⋆ ] � � k ( ∇ r ( w k ) T − M k ) � � M − 1 � Jacobian approximation error: � ≤ κ k < 1 Good initial guess � w 0 − w ∗ � ≤ 2 ω (1 − max { κ k } ) Then w k → w ∗ with the following linear-quadratic contraction in each iteration: � � κ k + ω � w k +1 − w ∗ � 2 � w k − w ∗ � � w k − w ∗ � . ≤ 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 12 / 32

  46. Convergence of full-step Newton methods Newton method: Newton-type method (Jacobian approx.) ∇ r ( w ) ⊤ ∆ w = − r ( w ) M ∆ w = − r ( w ) w ← w + ∆ w w ← w + ∆ w Yields the iteration k = 0 , 1 , .... : Yields the iteration k = 0 , 1 , .... : w k +1 ← w k − ∇ r ( w k ) −⊤ r ( w k ) w k +1 ← w k − M − 1 k r ( w k ) Theorem : assume � � ∇ r ( w ) T − ∇ r ( w ∗ ) T �� � � M − 1 � � ≤ ω � w − w ∗ � , for Nonlinearity of r : k w ∈ [ w k , w ⋆ ] � � k ( ∇ r ( w k ) T − M k ) � � M − 1 � Jacobian approximation error: � ≤ κ k < 1 Good initial guess � w 0 − w ∗ � ≤ 2 ω (1 − max { κ k } ) Then w k → w ∗ with the following linear-quadratic contraction in each iteration: � � κ k + ω � w k +1 − w ∗ � 2 � w k − w ∗ � � w k − w ∗ � . ≤ What about reduced steps ? Slow convergence when t < 1 (damped phase). When full steps become feasible, fast convergence to the solution. 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 12 / 32

  47. Newton methods - Short Survival Guide Exact Newton method: Newton-type method ∇ r ( w ) ⊤ ∆ w = − r ( w ) M ∆ w = − r ( w ) w ← w + t ∆ w w ← w + t ∆ w 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 13 / 32

  48. Newton methods - Short Survival Guide Exact Newton method: Newton-type method ∇ r ( w ) ⊤ ∆ w = − r ( w ) M ∆ w = − r ( w ) w ← w + t ∆ w w ← w + t ∆ w Exact Newton direction ∆ w improves r for a sufficiently small step size t ∈ ]0 , 1] 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 13 / 32

  49. Newton methods - Short Survival Guide Exact Newton method: Newton-type method ∇ r ( w ) ⊤ ∆ w = − r ( w ) M ∆ w = − r ( w ) w ← w + t ∆ w w ← w + t ∆ w Exact Newton direction ∆ w improves r for a sufficiently small step size t ∈ ]0 , 1] Inexact Newton direction ∆ w improves r for a sufficiently small step size t ∈ ]0 , 1] if M > 0 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 13 / 32

  50. Newton methods - Short Survival Guide Exact Newton method: Newton-type method ∇ r ( w ) ⊤ ∆ w = − r ( w ) M ∆ w = − r ( w ) w ← w + t ∆ w w ← w + t ∆ w Exact Newton direction ∆ w improves r for a sufficiently small step size t ∈ ]0 , 1] Inexact Newton direction ∆ w improves r for a sufficiently small step size t ∈ ]0 , 1] if M > 0 Exact full ( t = 1) Newton steps converge quadratically if close enough to the solution 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 13 / 32

  51. Newton methods - Short Survival Guide Exact Newton method: Newton-type method ∇ r ( w ) ⊤ ∆ w = − r ( w ) M ∆ w = − r ( w ) w ← w + t ∆ w w ← w + t ∆ w Exact Newton direction ∆ w improves r for a sufficiently small step size t ∈ ]0 , 1] Inexact Newton direction ∆ w improves r for a sufficiently small step size t ∈ ]0 , 1] if M > 0 Exact full ( t = 1) Newton steps converge quadratically if close enough to the solution Inexact full ( t = 1) Newton steps converge linearly if close enough to the solution and if the Jacobian approximation is ”sufficiently good” 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 13 / 32

  52. Newton methods - Short Survival Guide Exact Newton method: Newton-type method ∇ r ( w ) ⊤ ∆ w = − r ( w ) M ∆ w = − r ( w ) w ← w + t ∆ w w ← w + t ∆ w Exact Newton direction ∆ w improves r for a sufficiently small step size t ∈ ]0 , 1] Inexact Newton direction ∆ w improves r for a sufficiently small step size t ∈ ]0 , 1] if M > 0 Exact full ( t = 1) Newton steps converge quadratically if close enough to the solution Inexact full ( t = 1) Newton steps converge linearly if close enough to the solution and if the Jacobian approximation is ”sufficiently good” Newton iteration fails if ∇ r becomes singular 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 13 / 32

  53. Newton methods - Short Survival Guide Exact Newton method: Newton-type method ∇ r ( w ) ⊤ ∆ w = − r ( w ) M ∆ w = − r ( w ) w ← w + t ∆ w w ← w + t ∆ w Exact Newton direction ∆ w improves r for a sufficiently small step size t ∈ ]0 , 1] Inexact Newton direction ∆ w improves r for a sufficiently small step size t ∈ ]0 , 1] if M > 0 Exact full ( t = 1) Newton steps converge quadratically if close enough to the solution Inexact full ( t = 1) Newton steps converge linearly if close enough to the solution and if the Jacobian approximation is ”sufficiently good” Newton iteration fails if ∇ r becomes singular Newton methods with globalization converge in two phases: damped (slow) phase when reduced steps ( t < 1) are needed, quadratic/ linear when full steps are possible. 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 13 / 32

  54. Outline 1 KKT conditions - Quick Reminder The Newton method 2 Newton on the KKT conditions 3 Sequential Quadratic Programming 4 Hessian approximation 5 Maratos effect 6 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 14 / 32

  55. Core idea A vast majority of solvers try to find a KKT point w , µ , λ i.e: g ( w ) = 0 , h ( w ) ≤ 0 , Primal Feasibility: ∇ w L ( w , µ , λ ) = 0 , µ ≥ 0 , Dual Feasibility: Complementarity Slackness: µ i h i ( w ) = 0 , i = 1 , ... where L = Φ ( w ) + λ ⊤ g ( w ) + µ ⊤ h ( w ) 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 15 / 32

  56. Core idea A vast majority of solvers try to find a KKT point w , µ , λ i.e: g ( w ) = 0 , h ( w ) ≤ 0 , Primal Feasibility: ∇ w L ( w , µ , λ ) = 0 , µ ≥ 0 , Dual Feasibility: Complementarity Slackness: µ i h i ( w ) = 0 , i = 1 , ... where L = Φ ( w ) + λ ⊤ g ( w ) + µ ⊤ h ( w ) Let’s consider for now equality constrained problems, i.e. find w , λ s.t.: ∇ w L ( w , λ ) = 0 g ( w ) = 0 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 15 / 32

  57. Core idea A vast majority of solvers try to find a KKT point w , µ , λ i.e: g ( w ) = 0 , h ( w ) ≤ 0 , Primal Feasibility: ∇ w L ( w , µ , λ ) = 0 , µ ≥ 0 , Dual Feasibility: Complementarity Slackness: µ i h i ( w ) = 0 , i = 1 , ... where L = Φ ( w ) + λ ⊤ g ( w ) + µ ⊤ h ( w ) Let’s consider for now equality constrained problems, i.e. find w , λ s.t.: ∇ w L ( w , λ ) = 0 g ( w ) = 0 Idea: apply the Newton method on the KKT conditions, i.e. Solve... � ∇ w L ( w , λ ) � r ( w , λ ) = = 0 g ( w ) 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 15 / 32

  58. Core idea A vast majority of solvers try to find a KKT point w , µ , λ i.e: g ( w ) = 0 , h ( w ) ≤ 0 , Primal Feasibility: ∇ w L ( w , µ , λ ) = 0 , µ ≥ 0 , Dual Feasibility: Complementarity Slackness: µ i h i ( w ) = 0 , i = 1 , ... where L = Φ ( w ) + λ ⊤ g ( w ) + µ ⊤ h ( w ) Let’s consider for now equality constrained problems, i.e. find w , λ s.t.: ∇ w L ( w , λ ) = 0 g ( w ) = 0 Idea: apply the Newton method on the KKT conditions, i.e. Solve... ... by iterating � ∇ w L ( w , λ ) � ∆ w � � r ( w , λ ) = = 0 ∇ r ( w , λ ) T = − r ( w , λ ) g ( w ) ∆ λ 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 15 / 32

  59. Newton method on the KKT conditions KKT conditions Newton direction � ∇ w L ( w , λ ) � ∆ w � � ∇ r ( w , λ ) T r ( w , λ ) = = 0 = − r ( w , λ ) g ( w ) ∆ λ 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 16 / 32

  60. Newton method on the KKT conditions KKT conditions Newton direction � ∇ w L ( w , λ ) � ∆ w � � ∇ r ( w , λ ) T r ( w , λ ) = = 0 = − r ( w , λ ) g ( w ) ∆ λ Given by: ∇ 2 w L ( w , λ )∆ w + ∇ w , λ L ( w , λ )∆ λ = −∇ w L ( w , λ ) ∇ g ( w ) T ∆ w = − g ( w ) 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 16 / 32

  61. Newton method on the KKT conditions KKT conditions Newton direction � ∇ w L ( w , λ ) � ∆ w � � ∇ r ( w , λ ) T r ( w , λ ) = = 0 = − r ( w , λ ) g ( w ) ∆ λ Given by: using ∇ w L ( w , λ ) = ∇ Φ( w ) + ∇ g ( w ) λ ∇ 2 w L ( w , λ )∆ w + ∇ w , λ L ( w , λ )∆ λ = −∇ w L ( w , λ ) ∇ g ( w ) T ∆ w = − g ( w ) 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 16 / 32

  62. Newton method on the KKT conditions KKT conditions Newton direction � ∇ w L ( w , λ ) � ∆ w � � ∇ r ( w , λ ) T r ( w , λ ) = = 0 = − r ( w , λ ) g ( w ) ∆ λ Given by: using ∇ w L ( w , λ ) = ∇ Φ( w ) + ∇ g ( w ) λ ∇ 2 w L ( w , λ )∆ w + ∇ g ( w )∆ λ = −∇ w L ( w , λ ) ∇ g ( w ) T ∆ w = − g ( w ) 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 16 / 32

  63. Newton method on the KKT conditions KKT conditions Newton direction � ∇ w L ( w , λ ) � ∆ w � � ∇ r ( w , λ ) T r ( w , λ ) = = 0 = − r ( w , λ ) g ( w ) ∆ λ Given by: using ∇ w L ( w , λ ) = ∇ Φ( w ) + ∇ g ( w ) λ ∇ 2 w L ( w , λ )∆ w + ∇ g ( w )∆ λ = −∇ Φ( w ) − ∇ g ( w ) λ ∇ g ( w ) T ∆ w = − g ( w ) 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 16 / 32

  64. Newton method on the KKT conditions KKT conditions Newton direction � ∇ w L ( w , λ ) � ∆ w � � ∇ r ( w , λ ) T r ( w , λ ) = = 0 = − r ( w , λ ) g ( w ) ∆ λ Given by: using ∇ w L ( w , λ ) = ∇ Φ( w ) + ∇ g ( w ) λ ∇ 2 w L ( w , λ )∆ w + ∇ g ( w ) ( λ + ∆ λ ) = −∇ Φ( w ) ∇ g ( w ) T ∆ w = − g ( w ) 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 16 / 32

  65. Newton method on the KKT conditions KKT conditions Newton direction � ∇ w L ( w , λ ) � ∆ w � � ∇ r ( w , λ ) T r ( w , λ ) = = 0 = − r ( w , λ ) g ( w ) ∆ λ Given by: using ∇ w L ( w , λ ) = ∇ Φ( w ) + ∇ g ( w ) λ ∇ 2 w L ( w , λ )∆ w + ∇ g ( w ) ( λ + ∆ λ ) = −∇ Φ( w ) ∇ g ( w ) T ∆ w = − g ( w ) The Newton direction on the KKT conditions � ∇ 2 � ∇ Φ( w ) � � � � w L ( w , λ ) ∇ g ( w ) ∆ w = − ∇ g ( w ) T 0 λ + ∆ λ g ( w ) � �� � KKT matrix (symmetric indefinite) 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 16 / 32

  66. Newton method on the KKT conditions KKT conditions Newton direction � ∇ w L ( w , λ ) � ∆ w � � ∇ r ( w , λ ) T r ( w , λ ) = = 0 = − r ( w , λ ) g ( w ) ∆ λ Given by: using ∇ w L ( w , λ ) = ∇ Φ( w ) + ∇ g ( w ) λ ∇ 2 w L ( w , λ )∆ w + ∇ g ( w ) ( λ + ∆ λ ) = −∇ Φ( w ) ∇ g ( w ) T ∆ w = − g ( w ) The Newton direction on the KKT conditions � H ( w , λ ) � ∇ Φ( w ) � � � � ∇ g ( w ) ∆ w = − ∇ g ( w ) T 0 λ + ∆ λ g ( w ) � �� � KKT matrix (symmetric indefinite) where H ( w , λ ) = ∇ 2 w L ( w , λ ) is the Hessian of the problem. 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 16 / 32

  67. Newton method on the KKT conditions KKT conditions Newton direction � ∇ w L ( w , λ ) � ∆ w � � ∇ r ( w , λ ) T r ( w , λ ) = = 0 = − r ( w , λ ) g ( w ) ∆ λ Given by: using ∇ w L ( w , λ ) = ∇ Φ( w ) + ∇ g ( w ) λ ∇ 2 w L ( w , λ )∆ w + ∇ g ( w ) ( λ + ∆ λ ) = −∇ Φ( w ) ∇ g ( w ) T ∆ w = − g ( w ) The Newton direction on the KKT conditions � H ( w , λ ) � ∆ w � ∇ Φ( w ) � � � ∇ g ( w ) = − ∇ g ( w ) T λ + 0 g ( w ) � �� � KKT matrix (symmetric indefinite) where H ( w , λ ) = ∇ 2 w L ( w , λ ) is the Hessian of the problem. Note: update of the dual variable is λ + = λ + ∆ λ 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 16 / 32

  68. Newton method on the KKT conditions KKT conditions Newton direction � ∇ w L ( w , λ ) � ∆ w � � ∇ r ( w , λ ) T r ( w , λ ) = = 0 = − r ( w , λ ) g ( w ) ∆ λ Given by: using ∇ w L ( w , λ ) = ∇ Φ( w ) + ∇ g ( w ) λ ∇ 2 w L ( w , λ )∆ w + ∇ g ( w ) ( λ + ∆ λ ) = −∇ Φ( w ) ∇ g ( w ) T ∆ w = − g ( w ) The Newton direction on the KKT conditions � H ( w , λ ) � ∆ w � ∇ Φ( w ) � � � ∇ g ( w ) = − ∇ g ( w ) T λ + 0 g ( w ) � �� � KKT matrix (symmetric indefinite) where H ( w , λ ) = ∇ 2 w L ( w , λ ) is the Hessian of the problem. Note: update of the dual variable is λ + = λ + ∆ λ ∇ w L ( w , λ ) is not needed for computing the Newton step 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 16 / 32

  69. Newton method on the KKT conditions KKT conditions Newton direction � ∇ w L ( w , λ ) � ∆ w � � ∇ r ( w , λ ) T r ( w , λ ) = = 0 = − r ( w , λ ) g ( w ) ∆ λ Given by: using ∇ w L ( w , λ ) = ∇ Φ( w ) + ∇ g ( w ) λ ∇ 2 w L ( w , λ )∆ w + ∇ g ( w ) ( λ + ∆ λ ) = −∇ Φ( w ) ∇ g ( w ) T ∆ w = − g ( w ) The Newton direction on the KKT conditions � H ( w , λ ) � ∆ w � ∇ Φ( w ) � � � ∇ g ( w ) = − ∇ g ( w ) T λ + 0 g ( w ) � �� � KKT matrix (symmetric indefinite) where H ( w , λ ) = ∇ 2 w L ( w , λ ) is the Hessian of the problem. Note: update of the dual variable is λ + = λ + ∆ λ ∇ w L ( w , λ ) is not needed for computing the Newton step The updated dual variables λ + are readily provided ! 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 16 / 32

  70. Newton Iteration for Optimization - Example � 2 � 1 � � 1 1 2 w T w + w T min 1 4 0 w s.t. g ( w ) = w T w − 1 = 0 2 1.5 1 0.5 w 1 0 -0.5 -1 -1.5 -2 -2 -1 0 1 2 w 1 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 17 / 32

  71. Newton Iteration for Optimization - Example � 2 � 1 � � 1 1 2 w T w + w T Iterate: min 1 4 0 w � � ∆ w � ∇ Φ � � � H ∇ g s.t. g ( w ) = w T w − 1 = 0 = − ∇ g T λ + 0 g 2 1.5 1 0.5 w 1 0 -0.5 -1 -1.5 -2 -2 -1 0 1 2 w 1 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 17 / 32

  72. Newton Iteration for Optimization - Example � 2 � 1 � � 1 1 2 w T w + w T Iterate: min 1 4 0 w � � ∆ w � ∇ Φ � � � H ∇ g s.t. g ( w ) = w T w − 1 = 0 = − ∇ g T λ + 0 g � 2 w 1 with: � 2 ∇ g ( w ) = 2 w = 2 w 2 1.5 1 0.5 w 1 0 -0.5 -1 -1.5 -2 -2 -1 0 1 2 w 1 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 17 / 32

  73. Newton Iteration for Optimization - Example � 2 � 1 � � 1 1 2 w T w + w T Iterate: min 1 4 0 w � � ∆ w � ∇ Φ � � � H ∇ g s.t. g ( w ) = w T w − 1 = 0 = − ∇ g T λ + 0 g � 2 w 1 with: � 2 ∇ g ( w ) = 2 w = 2 w 2 1.5 1 L ( w , λ ) = Φ ( w ) + λ g ( w ) � 2 � 1 0.5 � � 1 w 1 0 ∇ w L ( w , λ ) = w + + 2 λ w 1 4 0 -0.5 -1 -1.5 -2 -2 -1 0 1 2 w 1 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 17 / 32

  74. Newton Iteration for Optimization - Example � 2 � 1 � � 1 1 2 w T w + w T Iterate: min 1 4 0 w � � ∆ w � ∇ Φ � � � H ∇ g s.t. g ( w ) = w T w − 1 = 0 = − ∇ g T λ + 0 g � 2 w 1 with: � 2 ∇ g ( w ) = 2 w = 2 w 2 1.5 1 L ( w , λ ) = Φ ( w ) + λ g ( w ) � 2 � 1 0.5 � � 1 w 1 0 ∇ w L ( w , λ ) = w + + 2 λ w 1 4 0 -0.5 -1 � 2 + 2 λ -1.5 � 1 H ( w , λ ) = -2 1 4 + 2 λ -2 -1 0 1 2 w 1 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 17 / 32

  75. Newton Iteration for Optimization - Example � 2 � 1 � � 1 1 2 w T w + w T Iterate: min 1 4 0 w � � ∆ w � ∇ Φ � � � H ∇ g s.t. g ( w ) = w T w − 1 = 0 = − ∇ g T λ + 0 g � 2 w 1 with: � 2 ∇ g ( w ) = 2 w = 2 w 2 1.5 1 L ( w , λ ) = Φ ( w ) + λ g ( w ) � 2 � 1 0.5 � � 1 w 1 0 ∇ w L ( w , λ ) = w + + 2 λ w 1 4 0 -0.5 -1 � 2 + 2 λ -1.5 � 1 H ( w , λ ) = -2 1 4 + 2 λ -2 -1 0 1 2 w 1 � � 2 w 1 + w 2 + 1 ∇ Φ ( w ) = w 1 + 4 w 2 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 17 / 32

  76. Newton Iteration for Optimization - Example � 2 � 1 � � 1 1 2 w T w + w T min Algorithm: Newton method 1 4 0 w Input : guess w , λ s.t. g ( w ) = w T w − 1 = 0 while �∇L� or � g � ≥ tol do Guess λ = 0, step t = 1 Compute 2 H ( w , λ ) , ∇ g ( w ) , ∇ Φ ( w ) , g ( w ) 1.5 1 Compute Newton direction 0.5 � � ∆ w � ∇ Φ � � � ∇ g H w 1 0 = − ∇ g T λ + 0 g -0.5 -1 ∆ λ = λ + − λ -1.5 -2 Compute Newton step, t ∈ ]0 , 1] -2 -1 0 1 2 w 1 w ← w + t ∆ w , λ ← λ + t ∆ λ return w , λ 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 17 / 32

  77. Newton Iteration for Optimization - Example � 2 � 1 � � 1 1 2 w T w + w T min Algorithm: Newton method 1 4 0 w Input : guess w , λ s.t. g ( w ) = w T w − 1 = 0 while �∇L� or � g � ≥ tol do Guess λ = 0, step t = 1 Compute 2 H ( w , λ ) , ∇ g ( w ) , ∇ Φ ( w ) , g ( w ) 1.5 1 Compute Newton direction 0.5 � � ∆ w � ∇ Φ � � � ∇ g H w 1 0 = − ∇ g T λ + 0 g -0.5 -1 ∆ λ = λ + − λ -1.5 -2 Compute Newton step, t ∈ ]0 , 1] -2 -1 0 1 2 w 1 w ← w + t ∆ w , λ ← λ + t ∆ λ return w , λ 17 th of February, 2016 S. Gros Optimal Control with DAEs, lecture 5 17 / 32

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Capacity markets and the internal market: the French court case (C-543/15 ) A missed opportunity?

Capacity markets and the internal market: the French court case (C-543/15 ) A missed opportunity?

UiO/UiOEnergi/EnergiNorge seminar 26.04.2016 Capacity markets and the internal market: the French court case (C-543/15 ) A missed opportunity? Catherine Banet Frsteamanuensis, Ph.D, LL.M Petroleums- og energirett avdelingen

158 views • 15 slides
Missed Pay Study of the Dutch Offshore EBN Dutch Exploration Day, 23 May 2016 Why Missed Pay

Missed Pay Study of the Dutch Offshore EBN Dutch Exploration Day, 23 May 2016 Why Missed Pay

Missed Pay Study of the Dutch Offshore EBN Dutch Exploration Day, 23 May 2016 Why Missed Pay study in the Netherlands? Many unsuccessful exploration wells Classified as dry Significant shows Tested hydrocarbon

570 views • 8 slides
The Incas built at least 23,000 km of roads throughout their empire but missed to invent the

The Incas built at least 23,000 km of roads throughout their empire but missed to invent the

The Incas built at least 23,000 km of roads throughout their empire but missed to invent the wheel. Let us imagine that in musical time theory we missed to discover that we can use durational ratios q q q q q q q q 2 X e = q q 2 = e

1.28k views • 20 slides
Acing the Interview 1 Acing the Interview (MAR 2013) Ive missed more than 9,000 shots in my

Acing the Interview 1 Acing the Interview (MAR 2013) Ive missed more than 9,000 shots in my

Acing the Interview 1 Acing the Interview (MAR 2013) Ive missed more than 9,000 shots in my career. Ive lost almost 300 games. 26 times, Ive been trusted to take the game winning shot and missed. Ive failed over and over and over

689 views • 26 slides
1 Case 1: Scalp Laceration Case 1 Have I missed any additional scalp lacerations? How do you

1 Case 1: Scalp Laceration Case 1 Have I missed any additional scalp lacerations? How do you

AcademicLifeinEM.blogspot.com Head and Neck Tricks of the Trade: Finalist for the MedGadgets Best New Helpful hints for the daily practice of EM Medical Blog of 2009 Featured EM blog on MedPedia (equivalent of Wikipedia,

406 views • 18 slides
1 Case 1 CASE 1: SCALP Have I missed any additional scalp lacerations? LACERATION How do you

1 Case 1 CASE 1: SCALP Have I missed any additional scalp lacerations? LACERATION How do you

PORTION DISTORTION: MORE ALCOHOL IN TUMBLER Tricks of the Trade: Helpful hints for the daily practice of EM Rachel Chin, MD Professor of Clinical Emergency Medicine, UC San Francisco San Francisco General Hospital Hawaii 2013

531 views • 13 slides
Case Presentation Brigitte M. Ronnett, MD Department of Pathology The Johns Hopkins University

Case Presentation Brigitte M. Ronnett, MD Department of Pathology The Johns Hopkins University

Case Presentation Brigitte M. Ronnett, MD Department of Pathology The Johns Hopkins University School of Medicine Baltimore, MD USA Case Presentation The patient is a 39 year-old woman, G2P1, who presented with a missed abortion at 8-10

1.17k views • 59 slides
Lecture 8 ANNOUNCEMENTS A summary of frequently misunderstood/missed concepts is A summary

Lecture 8 ANNOUNCEMENTS A summary of frequently misunderstood/missed concepts is A summary

Lecture 8 ANNOUNCEMENTS A summary of frequently misunderstood/missed concepts is A summary of frequently misunderstood/missed concepts is now posted on the class website, and will be updated regularly. Graded HW assignments can be

271 views • 25 slides
Contributions and Missed Opportunities S T . C O L U M B A S ( O A K L A N D , C A ) F R .

Contributions and Missed Opportunities S T . C O L U M B A S ( O A K L A N D , C A ) F R .

The 2018 Racism Pastoral Letter: Contributions and Missed Opportunities S T . C O L U M B A S ( O A K L A N D , C A ) F R . B R Y A N M A S S I N G A L E F O R D H A M U N I V E R S I T Y November 2018: The first since 1979 Overview

750 views • 53 slides
Case Presentation 19 yo, Soph Linebacker, Starter Clavicle fx, Left in fall camp 2014,

Case Presentation 19 yo, Soph Linebacker, Starter Clavicle fx, Left in fall camp 2014,

4/12/2016 Case Presentation 19 yo, Soph Linebacker, Starter Clavicle fx, Left in fall camp 2014, required ORIF Case Presentation Missed 3 games Returned for remaining 10 games Multiple subluxations Right Shoulder during JP

249 views • 4 slides
1 Welcome and introductions Indicate if you are: EC, NC Pre-K, Title I, Head Start, anybody we

1 Welcome and introductions Indicate if you are: EC, NC Pre-K, Title I, Head Start, anybody we

1 Welcome and introductions Indicate if you are: EC, NC Pre-K, Title I, Head Start, anybody we missed? Indicate if you are a preschool coordinator? EC director? Other administrators? Anybody we missed? 2 3 Almost every state has developed

805 views • 52 slides
TURN IT UP TO 11: LP IN THE DIAGNOSIS OF SAH Matt Greer February 10 th , 2015 IN CASE YOU

TURN IT UP TO 11: LP IN THE DIAGNOSIS OF SAH Matt Greer February 10 th , 2015 IN CASE YOU

TURN IT UP TO 11: LP IN THE DIAGNOSIS OF SAH Matt Greer February 10 th , 2015 IN CASE YOU MISSED THE REFERENCE HEADACHES IN THE ED Account for approximately 2% of ED visits 1% of these are due to SAH Approximately 14% get imaging

800 views • 31 slides
What if I missed the 2018 Mandatory Educational Presentation? 1) You will need to submit a

What if I missed the 2018 Mandatory Educational Presentation? 1) You will need to submit a

What if I missed the 2018 Mandatory Educational Presentation? 1) You will need to submit a letter to the Sale Committee stating your reason for requesting excusal. This must be completed by the next scheduled meeting (July 1 st , 2018) 2)

845 views • 53 slides
Opportunity is missed by most people because it is dressed in overalls and looks like work.

Opportunity is missed by most people because it is dressed in overalls and looks like work.

Opportunity is missed by most people because it is dressed in overalls and looks like work. Thomas Edison Calcaneus Scapula (superior angle, inferior angle, spine of the Fibula (head, lateral malleolus) scapula, root, coracoid process)

222 views • 20 slides
even more ways to generate self log missed opportunities data lost forever different modes of

even more ways to generate self log missed opportunities data lost forever different modes of

even more ways to generate self log missed opportunities data lost forever different modes of self logging keep calm and thank google what i did cant you tell i was falling in love? how i did it timestamp | sender | message chat

307 views • 28 slides
Raytracer 2 We start with some bits we missed in the previous lecture General camera in

Raytracer 2 We start with some bits we missed in the previous lecture General camera in

Raytracer 2 We start with some bits we missed in the previous lecture General camera in word coordinates (we positioned in (0,0,0) in the previous slides Computing refraction term without trigo (by means of dot products) Some

622 views • 19 slides
Quadratic Programming Seminar Andreas Potschka Interdisciplinary Center for Scientific Computing,

Quadratic Programming Seminar Andreas Potschka Interdisciplinary Center for Scientific Computing,

Quadratic Programming Seminar Andreas Potschka Interdisciplinary Center for Scientific Computing, Heidelberg University WS 2018/19 October 17, 2018 Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar 1

460 views • 20 slides
A good recipe for solving MINLPs The Problem The Ingredients VNS Local Branching Leo Liberti 1

A good recipe for solving MINLPs The Problem The Ingredients VNS Local Branching Leo Liberti 1

A good recipe for solving MINLPs L. Liberti, G. Nannicini, N. Mladenovic A good recipe for solving MINLPs The Problem The Ingredients VNS Local Branching Leo Liberti 1 , Giacomo Nannicini 1 , Nenad Mladenovi c 2 BB for MINLPs SQP The

595 views • 42 slides
Accurate Clock Mesh Sizing via Sequential Quadratic Programming Venkata Rajesh Mekala, Yifang

Accurate Clock Mesh Sizing via Sequential Quadratic Programming Venkata Rajesh Mekala, Yifang

Accurate Clock Mesh Sizing via Sequential Quadratic Programming Venkata Rajesh Mekala, Yifang Liu, XiaojiYe, Jiang Hu, Peng Li Department of ECE Texas A&amp;M University 1 ISPD 2010 3/18/2010 OUTLINE Introduction Previous Works

765 views • 29 slides
Sequential Quadratic Programming 1 Lecture 17 ME EN 575 Andrew Ning aning@byu.edu Outline

Sequential Quadratic Programming 1 Lecture 17 ME EN 575 Andrew Ning aning@byu.edu Outline

Sequential Quadratic Programming 1 Lecture 17 ME EN 575 Andrew Ning aning@byu.edu Outline Equality Constrained Quadratic Programming Equality Constrained Sequential Quadratic Programming Quadratic Programming (QP) Equality Constrained

132 views • 9 slides
1/31/2007 Massachusetts Institute of Technology Context Model Selection Finite Horizon

1/31/2007 Massachusetts Institute of Technology Context Model Selection Finite Horizon

1/31/2007 Massachusetts Institute of Technology Context Model Selection Finite Horizon Control Design for Model Identification Optimal Discrimination between Which model best explains a given data set? Several Models 1. Parameter

346 views • 8 slides
A two-step sequential linear programming algorithm for MINLP problems: An application to gas

A two-step sequential linear programming algorithm for MINLP problems: An application to gas

A two-step sequential linear programming algorithm for MINLP problems: An application to gas transmission networks Julio Gonz alez-D az Angel M. Gonz alez-Rueda Mar a P. Fern andez de C ordoba University of Santiago

1.66k views • 151 slides
Designs of Orthogonal Filter Banks and Orthogonal Cosine-Modulated Filter Banks Jie Yan

Designs of Orthogonal Filter Banks and Orthogonal Cosine-Modulated Filter Banks Jie Yan

Designs of Orthogonal Filter Banks and Orthogonal Cosine-Modulated Filter Banks Jie Yan Department of Electrical and Computer Engineering University of Victoria April 16, 2010 1 / 45 OUTLINE INTRODUCTION 1 LS DESIGN OF ORTHOGONAL FILTER

584 views • 45 slides
1 Feature Extraction and Description Visual Vocabulary Construction From a database of

1 Feature Extraction and Description Visual Vocabulary Construction From a database of

Visual Categorization With Bags Basic Problem Addressed of Keypoints. ECCV, 2004 . Find a method for Generic Visual G. Csurka, C. Bray, C. Dance, and L. Fan. Categorization Visual Categorization: Identifying whether objects of one or

172 views • 5 slides

More recommend