convex optimization lecture notes for ee 227bt draft fall
play

Convex Optimization Lecture Notes for EE 227BT Draft, Fall 2013 - PDF document

Oct 29, 2022 •39 likes •989 views

Convex Optimization Lecture Notes for EE 227BT Draft, Fall 2013 Laurent El Ghaoui August 29, 2013 2 Contents 1 Introduction 7 1.1 Optimization problems . . . . . . . . . . . . . . . . . . . . . . . . 7 1.1.1 The model . . . . . . . .

  1. Convex Optimization Lecture Notes for EE 227BT Draft, Fall 2013 Laurent El Ghaoui August 29, 2013

  2. 2

  3. Contents 1 Introduction 7 1.1 Optimization problems . . . . . . . . . . . . . . . . . . . . . . . . 7 1.1.1 The model . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.1.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.1.3 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.2 Convex optimization problems . . . . . . . . . . . . . . . . . . . 9 1.2.1 Convexity . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.2.2 Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.2.3 A brief history of convex optimization . . . . . . . . . . . 9 1.3 Course objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2 Linear Algebra Review 11 2.1 Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.1.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.1.2 Scalar product and norms . . . . . . . . . . . . . . . . . . 11 2.2 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.2.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.2.2 Matrix norms . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.2.3 Matrix description of subspaces . . . . . . . . . . . . . . . 15 2.2.4 Singular value decomposition . . . . . . . . . . . . . . . . 15 2.3 Symmetric Matrices . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.3.1 Definition and examples . . . . . . . . . . . . . . . . . . . 15 2.3.2 Eigenvalue decomposition . . . . . . . . . . . . . . . . . . 16 2.3.3 Positive semi-definite matrices . . . . . . . . . . . . . . . 17 3 Convex Optimization Problems 19 3.1 Convex Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.1.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.1.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.1.3 Support and indicator functions . . . . . . . . . . . . . . 20 3.1.4 Operations that preserve convexity . . . . . . . . . . . . . 20 3.1.5 Separation theorems . . . . . . . . . . . . . . . . . . . . . 21 3.2 Convex Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.2.1 Domain of a function . . . . . . . . . . . . . . . . . . . . . 21 3.2.2 Definition of convexity . . . . . . . . . . . . . . . . . . . . 21 3.2.3 Alternate characterizations of convexity . . . . . . . . . . 22 3.2.4 Operations that preserve convexity . . . . . . . . . . . . . 22 3.2.5 Conjugate function . . . . . . . . . . . . . . . . . . . . . . 23 3.3 Convex Optimization Problems . . . . . . . . . . . . . . . . . . . 23 3.3.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.3.2 Optimality . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.3.3 Equivalent problems . . . . . . . . . . . . . . . . . . . . . 25 3

  4. 4 CONTENTS 3.3.4 Maximization of convex functions . . . . . . . . . . . . . . 27 3.4 Linear Optimization . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.4.1 Definition and standard forms . . . . . . . . . . . . . . . . 28 3.4.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.5 Overview of conic optimization . . . . . . . . . . . . . . . . . . . 31 3.5.1 Conic optimization models. . . . . . . . . . . . . . . . . . 31 3.5.2 Tractable conic optimization. . . . . . . . . . . . . . . . . 32 3.6 Second-order cone optimization . . . . . . . . . . . . . . . . . . . 33 3.6.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . 33 3.6.2 Standard form. . . . . . . . . . . . . . . . . . . . . . . . . 33 3.6.3 Special case: convex quadratic optimization. . . . . . . . 33 3.6.4 Quadratically constrained, convex quadratic optimization. 33 3.6.5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3.6.6 Risk-return trade-off in portfolio optimization . . . . . . . 34 3.6.7 Robust half-space constraint . . . . . . . . . . . . . . . . 34 3.6.8 Robust linear programming . . . . . . . . . . . . . . . . . 34 3.6.9 Robust separation . . . . . . . . . . . . . . . . . . . . . . 35 3.7 Semidefinite optimization . . . . . . . . . . . . . . . . . . . . . . 36 3.7.1 Definition and standard forms . . . . . . . . . . . . . . . . 36 3.7.2 Special cases . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.7.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.8 More on non-convex quadratic optimization . . . . . . . . . . . . 39 3.8.1 More on rank relaxation. . . . . . . . . . . . . . . . . . . 39 3.8.2 Lagrange relaxation . . . . . . . . . . . . . . . . . . . . . 40 3.8.3 The S -lemma . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.9 Optimization over ellipsoids . . . . . . . . . . . . . . . . . . . . . 41 3.9.1 Parametrizations of ellipsoids . . . . . . . . . . . . . . . . 41 3.9.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3.9.3 Maximum volume ellipsoid in a polyhedron. . . . . . . . . 42 3.9.4 Minimum trace ellipsoid containing the sum of ellipsoids. 43 3.9.5 Application: reachable sets for discrete-time dynamical systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.10 Geometric Programming . . . . . . . . . . . . . . . . . . . . . . . 44 3.10.1 Standard form . . . . . . . . . . . . . . . . . . . . . . . . 44 3.10.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 3.10.3 Generalized geometric programming . . . . . . . . . . . . 48 4 Duality 49 4.1 Weak Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.1.1 Primal problem . . . . . . . . . . . . . . . . . . . . . . . . 49 4.1.2 Dual problem . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.1.3 Geometric interpretation . . . . . . . . . . . . . . . . . . . 51 4.1.4 Minimax inequality . . . . . . . . . . . . . . . . . . . . . . 51 4.1.5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 4.1.6 Semidefinite optimization problem . . . . . . . . . . . . . 54 4.2 Strong duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 4.2.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . 55 4.2.2 A strong duality theorem . . . . . . . . . . . . . . . . . . 55 4.2.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 4.3 Strong duality for convex problems . . . . . . . . . . . . . . . . . 58 4.3.1 Primal and dual problems . . . . . . . . . . . . . . . . . . 58 4.3.2 Strong duality via Slater’s condition . . . . . . . . . . . . 58 4.3.3 Geometric interpretation . . . . . . . . . . . . . . . . . . . 59 4.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 9: Convex Optimization Problems

CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 9: Convex Optimization Problems

CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 9: Convex Optimization Problems Instructor: Shaddin Dughmi Announcements Homework: Due beginning of next class Must submit a hard copy, unless you have a good excuse If using late

934 views • 62 slides
CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 13: Optimality Conditions for

CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 13: Optimality Conditions for

CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 13: Optimality Conditions for Convex Optimization Instructor: Shaddin Dughmi Announcements Today: short lecture wrapping up convex optimization Thursday: We begin combinatorial

396 views • 20 slides
CS675: Convex and Combinatorial Optimization Fall 2019 Convex Optimization Problems Instructor:

CS675: Convex and Combinatorial Optimization Fall 2019 Convex Optimization Problems Instructor:

CS675: Convex and Combinatorial Optimization Fall 2019 Convex Optimization Problems Instructor: Shaddin Dughmi Outline Convex Optimization Basics 1 Common Classes 2 Interlude: Positive Semi-Definite Matrices 3 More Convex Optimization

897 views • 62 slides
CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 25: Unconstrained Submodular

CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 25: Unconstrained Submodular

CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 25: Unconstrained Submodular Function Minimization Instructor: Shaddin Dughmi Announcements Outline Introduction 1 The Convex Closure and the Lovasz Extension 2 Wrapping up 3

1.19k views • 66 slides
CS675: Convex and Combinatorial Optimization Fall 2019 Duality of Convex Optimization Problems

CS675: Convex and Combinatorial Optimization Fall 2019 Duality of Convex Optimization Problems

CS675: Convex and Combinatorial Optimization Fall 2019 Duality of Convex Optimization Problems Instructor: Shaddin Dughmi Outline The Lagrange Dual Problem 1 Duality 2 Optimality Conditions 3 Recall: Optimization Problem in Standard Form

685 views • 67 slides
CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 17: Combinatorial Problems as

CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 17: Combinatorial Problems as

CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 17: Combinatorial Problems as Linear Programs III Instructor: Shaddin Dughmi Announcements Today: Spanning Trees and Flows Flexibility awarded by polyhedral perspective Outline

584 views • 35 slides
CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 8: Convex Functions Wrapup

CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 8: Convex Functions Wrapup

CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 8: Convex Functions Wrapup Instructor: Shaddin Dughmi Outline Quasiconvex Functions 1 Log-Concave Functions 2 Quasiconvex Functions A function f : R n R is quasiconvex if

407 views • 38 slides
CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 16: Combinatorial Problems as

CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 16: Combinatorial Problems as

CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 16: Combinatorial Problems as Linear Programs II Instructor: Shaddin Dughmi Announcements Project announced Choose a topic and partner(s) by Nov 1 Choose papers by Nov 8 Short

679 views • 44 slides
CS675: Convex and Combinatorial Optimization Fall 2019 Convex Functions Instructor: Shaddin

CS675: Convex and Combinatorial Optimization Fall 2019 Convex Functions Instructor: Shaddin

CS675: Convex and Combinatorial Optimization Fall 2019 Convex Functions Instructor: Shaddin Dughmi Outline Convex Functions 1 Examples of Convex and Concave Functions 2 Convexity-Preserving Operations 3 Convex Functions A function f : R n

932 views • 54 slides
CS675: Convex and Combinatorial Optimization Fall 2014 Convex Functions Instructor: Shaddin

CS675: Convex and Combinatorial Optimization Fall 2014 Convex Functions Instructor: Shaddin

CS675: Convex and Combinatorial Optimization Fall 2014 Convex Functions Instructor: Shaddin Dughmi Outline Convex Functions 1 Examples of Convex and Concave Functions 2 Convexity-Preserving Operations 3 Convex Functions A function f : R n

572 views • 54 slides
Dimension Free Optimization and Non-Convex Optimization Instructor: Sham Kakade 1 Non-convex

Dimension Free Optimization and Non-Convex Optimization Instructor: Sham Kakade 1 Non-convex

CSE 547/Stat 548: Machine Learning for Big Data Lecture Dimension Free Optimization and Non-Convex Optimization Instructor: Sham Kakade 1 Non-convex optimization and Black-Box Oracle Complexity Suppose we are trying to minimize the function F (

273 views • 4 slides
CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 1: Introduction to Optimization

CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 1: Introduction to Optimization

CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 1: Introduction to Optimization Instructor: Shaddin Dughmi Outline Course Overview 1 Administrivia 2 Linear Programming 3 Outline Course Overview 1 Administrivia 2 Linear

794 views • 40 slides
CS675: Convex and Combinatorial Optimization Fall 2019 Convex Sets Instructor: Shaddin Dughmi

CS675: Convex and Combinatorial Optimization Fall 2019 Convex Sets Instructor: Shaddin Dughmi

CS675: Convex and Combinatorial Optimization Fall 2019 Convex Sets Instructor: Shaddin Dughmi Outline Convex sets, Affine sets, and Cones 1 Examples of Convex Sets 2 Convexity-Preserving Operations 3 Separation Theorems 4 Convex Sets A

515 views • 38 slides
CS675: Convex and Combinatorial Optimization Fall 2019 Submodular Function Optimization

CS675: Convex and Combinatorial Optimization Fall 2019 Submodular Function Optimization

CS675: Convex and Combinatorial Optimization Fall 2019 Submodular Function Optimization Instructor: Shaddin Dughmi Outline Introduction to Submodular Functions 1 Unconstrained Submodular Minimization 2 Definition and Examples The Convex

2.22k views • 173 slides
CS675: Convex and Combinatorial Optimization Spring 2018 Convex Optimization Problems

CS675: Convex and Combinatorial Optimization Spring 2018 Convex Optimization Problems

CS675: Convex and Combinatorial Optimization Spring 2018 Convex Optimization Problems Instructor: Shaddin Dughmi Outline Convex Optimization Basics 1 Common Classes 2 Interlude: Positive Semi-Definite Matrices 3 More Convex Optimization

887 views • 61 slides
CS675: Convex and Combinatorial Optimization Fall 2014 Optimality Conditions for Convex

CS675: Convex and Combinatorial Optimization Fall 2014 Optimality Conditions for Convex

CS675: Convex and Combinatorial Optimization Fall 2014 Optimality Conditions for Convex Optimization Instructor: Shaddin Dughmi Outline Optimality Conditions 1 Recall: Lagrangian Duality Primal Problem Dual Problem min f 0 ( x ) max g ( ,

510 views • 18 slides
Problems for Ivan Corwins OOPS lectures Read this first: Problems should be completed by groups

Problems for Ivan Corwins OOPS lectures Read this first: Problems should be completed by groups

Problems for Ivan Corwins OOPS lectures Read this first: Problems should be completed by groups of students and submitted by 14:00UTC the next day to the TAs. Please identify the members of your group. The TAs will collate and select the

171 views • 4 slides
Diverse Particle Selection for High-Dimensional Inference in Graphical Models Erik Sudderth UC

Diverse Particle Selection for High-Dimensional Inference in Graphical Models Erik Sudderth UC

Diverse Particle Selection for High-Dimensional Inference in Graphical Models Erik Sudderth UC Irvine Computer Science Collaborators: Particle Max-Product: Jason Pacheco, MIT Human Pose: Silvia Zuffi & Michael Black, MPI Tubingen

830 views • 69 slides
Individual Choice Behavior: This is a large, sprawling literature, in economics and psychology,

Individual Choice Behavior: This is a large, sprawling literature, in economics and psychology,

Individual Choice Behavior: This is a large, sprawling literature, in economics and psychology, much of which is devoted to testing the predictions of different theories of rational behavior, and to documenting deviations from those predictions.

285 views • 13 slides
Quantitative Sabotage Games Paul Hunter Universit Libre de Bruxelles Work of: Thomas Brihaye,

Quantitative Sabotage Games Paul Hunter Universit Libre de Bruxelles Work of: Thomas Brihaye,

Quantitative Sabotage Games Paul Hunter Universit Libre de Bruxelles Work of: Thomas Brihaye, Gilles Geeraerts, Axel Haddad, Benjamin Monmege, Guillermo Prez, Gabriel Renault GRASTA October 2015 Sabotage games Sabotage games Sabotage

848 views • 47 slides
Recent work in Truncated Statistics Andrew Ilyas Motivation: Poincar and the Baker

Recent work in Truncated Statistics Andrew Ilyas Motivation: Poincar and the Baker

Recent work in Truncated Statistics Andrew Ilyas Motivation: Poincar and the Baker Motivation: Poincar and the Baker Motivation: Poincar and the Baker Claimed weight: 1 kg/loaf Motivation: Poincar and the Baker Claimed weight: 1

1.97k views • 154 slides
Benefits and Contributions Update Key Decisions Day-to-Day Chronic Hospitalisation Expenses

Benefits and Contributions Update Key Decisions Day-to-Day Chronic Hospitalisation Expenses

2019 Medical Scheme Benefits and Contributions Update Key Decisions Day-to-Day Chronic Hospitalisation Expenses Would you be prepared to be limited to a Would you What rate of To what extent network of accept Do you need hospital do

441 views • 20 slides
A Multilevel Vertex Separator Algorithm Based on the Solution of Bilinear Programs William Hager 1

A Multilevel Vertex Separator Algorithm Based on the Solution of Bilinear Programs William Hager 1

A Multilevel Vertex Separator Algorithm Based on the Solution of Bilinear Programs William Hager 1 James Hungerford 2 Ilya Safro 3 1 University of Florida 2 M.A.I.O.R. Srl, Lucca, Italy 3 Clemson University October 12th, 2016 CSC16 Albuquerque,

778 views • 56 slides
SuperCDMS Soudan 15 Ge iZIP detectors (9 kg) installed in CDMS II apparatus in Soudan

SuperCDMS Soudan 15 Ge iZIP detectors (9 kg) installed in CDMS II apparatus in Soudan

SuperCDMS Soudan: High Threshold Analysis Brett Cornell Caltech SuperCDMS Soudan 15 Ge iZIP detectors (9 kg) installed in CDMS II apparatus in Soudan Underground Lab Data taken March 2012 July 2014: 510 total live-days 496

411 views • 26 slides

More recommend