Quadratic Programming Seminar Andreas Potschka Interdisciplinary - - PowerPoint PPT Presentation

Quadratic Programming Seminar

Andreas Potschka

Interdisciplinary Center for Scientific Computing, Heidelberg University

WS 2018/19 October 17, 2018

Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 1

Overview

Organizational matters Introduction Presentation topics Preparation guidelines for presentations Introductory round

Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 2

Organizational matters

◮ Wednesdays, 14–16 h ◮ Kickoff: October 17, 2018 ◮ Location: INF 205, SR1 ◮ Target group: BSc/MSc

◮ Mathematics ◮ Scientific Computing ◮ Computer Science

◮ Language: English ◮ One presentation per session (45–75 min plus discussion) ◮ Credit Points: 6 CP ◮ Prerequisites: Presentation, regular attendance

Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 3

Grading criteria

◮ Quality of contents

◮ Mathematical precision ◮ Focus on the essential aspects, adapted to audience ◮ Clear structure

◮ Presentation style

◮ Comprehensible pronounciation ◮ Adequate tempo of presentation ◮ Responsiveness to questions from the audience

◮ Presentation technique

◮ Choice: Black board, PowerPoint, L

A

T EXbeamer, etc.

◮ Readable, well-structured, meaningful black board and slides ◮ Focus on one message per slide ◮ Nominal style instead of full sentences ◮ Avoid clutter ◮ Handout Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 4

Introduction

Linear Program min gTx

• ver x ∈ Rn

s.t. x 0, Ax = b ∈ Rm Quadratic Program min 1

2xTHx + gTx

• ver x ∈ Rn

s.t. x 0, Ax = b ∈ Rm

◮ QP: Extension of LP ◮ Hessian matrix H ∈ Rn×n symmetric ◮ H positive semi-definite ⇒ convex QP ◮ Nonconvex QPs are very difficult ◮ Many applications: Portfolio optimization, model predictive control, . . . ◮ Important subproblem for nonlinear optimization (Sequential Quadratic Programming) ◮ Different properties require different numerical methods: Convexity, sparsity, inequalities ◮ Two main approaches: Interior point and active set ◮ Strongly linked with linear algebra of saddle point problems (symmetric indefinite matrices)

Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 5

Presentation topics

Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 6

• 1. Problem formulation and applications

Contents:

◮ Problem formulation ◮ Portfolio optimization ◮ Model predictive control (MPC) ◮ Sequential quadratic programming (SQP) for nonlinear optimization

Literature references:

◮ J. Nocedal, S.J. Wright. Numerical Optimization, Springer, 2006. Ch. 16.1, pp. 449–450,

529–534.

◮ C. Geiger, C. Kanzow. Theorie und Numerik restringierter Optimierungsaufgaben.

Springer, 2002. Ch. 5.1, pp. 197–198.

◮ A. Alession, A. Bemporad. A Survey on Explicit Model Predictive Control. In: Nonlinear

Model Predictive Control. Lecture Notes in Control and Information Sciences, Springer, 2009, Volume 384, pp. 345–347, 362–363.

Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 7

• 2. Optimality conditions for QPs

Contents:

◮ Mathematical derivation of necessary and sufficient conditions of optimality for convex and

nonconvex QPs

◮ Degeneracy ◮ Basis for following numerical methods

Literature references:

◮ P

.E. Gill, W. Murray, M.H. Wright. Practical Optimization. Elsevier, 2004. Ch. 3, pp. 59–77.

◮ C. Geiger, C. Kanzow. Theorie und Numerik restringierter Optimierungsaufgaben.

Springer, 2002. Ch. 2.2, pp. 40–55, Ch. 5, pp. 197–205

◮ J. Nocedal, S.J. Wright. Numerical Optimization, Springer, 2006. Ch. 16.4, pp. 465–467.

Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 8

• 3. QP duality

Contents:

◮ Primal and dual QP

, convex and nonconvex case

◮ Duality theorems (Lagrange, Franke-Wolfe, Dorn)

Literature references:

◮ S. Boyd, L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004,

Chapter 5 (available online)

◮ R.W. Cottle, J.-S. Pang, R.E. Stone. The Linear Complementarity Problem, Classics in

Applied Mathematics 60, SIAM, 2009. pp. 113–117

◮ O.L. Mangasarian. Duality in quadratic programming. In ‘Nonlinear Programming’, chapter

8-2, pp. 123–126. McGraw-Hill, New York, USA, 1969. Reprinted as Classics in Applied Mathematics 10, SIAM, Philadelphia, USA, 1994.

◮ J.J. Júdice. The duality theory of general quadratic programs. Portugaliae Mathematica,

42, 113–121, 1984.

Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 9

• 4. Complexity theory

Contents:

◮ Combinatorial, complexity, and computability aspects for determination of QP solutions

and their active constraint sets Literature references:

◮ P

.M. Pardalos and G. Schnitger. Checking local optimality in constrained quadratic programming is NP-hard. Operations Research Letters, 7(1), 33–35, 1988.

◮ K.G. Murty and S.N. Kabadi. Some NP-complete problems in quadratic and nonlinear

• programming. Mathematical Programming, 39(2), 117–129, 1987.

◮ M.R. Garey and D.S. Johnson. Computers and Intractability. A Guide to the Theory of

NP-Completeness. W.H. Freeman and Co, 1979.

Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 10

• 5. Direct linear algebra for equality constrained QPs

Contents:

◮ Range and null space methods ◮ Symmetric indefinite decomposition

Literature references:

◮ J. Nocedal, S.J. Wright. Numerical Optimization, Springer, 2006. Ch. 16.1, 16.2,

• pp. 451–459.

◮ M. Benzi, G.H. Golub, J. Liesen. Numerical solution of saddle point problems. Acta

Numerica 14, pp. 1–137, 2005. Focus pp. 30–34, 40–43.

Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 11

• 6. Iterative linear algebra for equality constrained QPs

Contents:

◮ Spectral properties of saddle-point matrices ◮ Iterative solution methods (projected CG, MINRES) ◮ Preconditioning

Literature references:

◮ J. Nocedal, S.J. Wright. Numerical Optimization, Springer, 2006. Ch. 16.3, pp. 459–463. ◮ M. Benzi, G.H. Golub, J. Liesen. Numerical solution of saddle point problems. Acta

Numerica 14, pp. 1–137, 2005. Focus pp. 14–29, 43–96.

Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 12

• 7. Bound constrained QPs

Contents:

◮ Gradient projection method for QPs with only bound constraints

Literature references:

◮ J.J. Moré and G. Toraldo. On the solution of large quadratic programming problems with

bound constraints. SIAM Journal on Optimization, 1(1), 93–113, 1991.

◮ J.J. Moré and G. Toraldo. Algorithms for bound constrained quadratic programming

• problems. Numerische Mathematik, 55(4), 377–400, 1989.

◮ J. Nocedal, S.J. Wright. Numerical Optimization, Springer, 2006. Ch. 16.7, pp. 485–490.

Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 13

• 8. Active set methods for inequality constrained QPs

Contents:

◮ Primal active set method for QPs with general linear inequalities ◮ Dual active set method for convex QPs with general linear inequalities ◮ Updates for matrix decompositions

Literature references:

◮ J. Nocedal, S.J. Wright. Numerical Optimization, Springer, 2006. Ch. 16.5, pp. 467–480.

Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 14

• 9. Interior point methods for inequality constrained QPs

Contents:

◮ Interior point method for QPs with general linear inequalities ◮ Numerical treatment of saddle-point systems

Literature references:

◮ J. Nocedal, S.J. Wright. Numerical Optimization, Springer, 2006. Ch. 16.6, pp. 480–485. ◮ M. Benzi, G.H. Golub, J. Liesen. Numerical solution of saddle point problems. Acta

Numerica 14, pp. 1–137, 2005. Focus pp. 12–14.

Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 15

• 10. Homotopy methods for parametric QPs

Contents:

◮ Parametric QPs ◮ Primal-dual active set method for convex parametric QPs

Literature references:

◮ M.J. Best. An Algorithm for the Solution of the Parametric Quadratic Programming

• Problem. Applied Mathematics and Parallel Computing – Festschrift for Klaus Ritter,

Physica-Verlag, 1996, ch. 3, pp. 57–76.

Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 16

• 11. Linear complementarity problems for QPs

Contents:

◮ Linear complementarity problems (LCP) ◮ Optimality conditions ◮ Connection with Quadratic Programming ◮ Dantzig-Wolfe algorithm for LCP and QP

Literature references:

◮ P

. Wolfe. The simplex method for quadratic programming. Econometrica 27, pp. 382–398, 1959.

◮ R.W. Cottle, J.-S. Pang, R.E. Stone. The Linear Complementarity Problem, Classics in

Applied Mathematics 60, SIAM, 2009. pp. 3–5, 23, 29, 113–117, 138.

Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 17

List of topics

Nr Date Topic 1 14.11.2018 Problem formulation and applications 2 21.11.2018 Optimality conditions for QPs 3 CANCELLED QP duality 4 28.11.2018 Complexity theory 5 05.12.2018 Direct linear algebra for equality constrained QPs 6 12.12.2018 Iterative linear algebra for equality constrained QPs 7 19.12.2018 Bound constrained QPs 8 09.01.2019 Active set methods for inequality constrained QPs 9 16.01.2019 Interior point methods for inequality constrained QPs 10 23.01.2019 Homotopy methods for parametric QPs 11 30.01.2019 Linear complementarity problems for QPs

Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 18

Preparation guidelines for presentations

◮ Who is my audience?

Imagine one or two concrete persons!

◮ How much time do I have? ◮ Structure: Overview, main part, summary ◮ One week before presentation:

Meet me to discuss slides/black board

◮ Your presentation is more than your slides

Deliver at least one, better two exercise presentations

Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 19

Introductory round

◮ Name ◮ Country ◮ Semester ◮ Study program ◮ Possible topics for seminar presentation

Andreas Potschka, potschka@iwr.uni-heidelberg.de Quadratic Programming Seminar – 20