COMP331/COMP557: Optimisation Martin Gairing Computer Science Department University of Liverpool 1st Semester 2018/19 Material adapted from a course by Martin Skutella at TU Berlin 1
My Background FH Esslingen ◮ 1995-2000: Diplom (Electrical Engineering) Clemson University ◮ 2000-2001: MSc (Computer Science) University of Paderborn ◮ 2002-2007: PhD + Postdoc International Computer Science Institute Berkeley ◮ 2007- 2009: Postdoc Liverpool University ◮ Since 2009: Lecturer/Senior Lecturer 2
Administrative Details Lectures: ◮ Mondays, 11:00 - 12:00 ◮ Tuesdays, 10:00 - 11:00 ◮ Thursdays, 12:00 -13:00 Tutorials: ◮ Flávia Alves (F.Alves@liverpool.ac.uk) ◮ starting from Friday 28 September Assessment: ◮ 25 % continuous assessment ◮ 75 % final exam 3
The webpage for this module ◮ https://cgi.csc.liv.ac.uk/~gairing/COMP557/ ◮ lecture notes ◮ resources ◮ announcements 4
Course Aims ◮ To provide a foundation for modelling various continuous and discrete optimisation problems. ◮ To provide the tools and paradigms for the design and analysis of algorithms for continuous and discrete optimisation problems. Apply these tools to real-world problems. ◮ To review the links and interconnections between optimisation and computational complexity theory. ◮ To provide an in-depth, systematic and critical understanding of selected significant topics at the intersection of optimisation, algorithms and (to a lesser extent) complexity theory, together with the related research issues. 5
Learning Outcomes Upon completion of the module you should have: ◮ A critical awareness of current problems and research issues in the field of optimisation. ◮ The ability to formulate optimisation models for the purpose of modelling particular applications. ◮ The ability to use appropriate algorithmic paradigms and techniques in context of a particular optimisation model. ◮ The ability to read, understand and communicate research literature in the field of optimisation. ◮ The ability to recognise potential research opportunities and research directions. 6
Outline Introduction 1 Linear Programming Basics 2 The Geometry of Linear Programming 3 The Simplex Method 4 Duality 5 Applications of Linear Programming 6 7
Chapter 1: Introduction 8
A Motivating (and Refreshing) Example Small brewery produces ale and beer. ◮ Production limited by scarce resources: corn, hops, barley malt. ◮ Recipes for ale and beer require different proportions of resources. Beverage Corn (lb) Hops (oz) Malt (lb) Profit (£) Ale (barrel) 5 4 35 13 Beer (barrel) 15 4 20 23 Quantity 480 160 1190 ◮ Devote all resources to ale: 34 barrels of ale = ⇒ £442 ◮ Devote all resources to beer: 32 barrels of beer = ⇒ £736 ◮ 7.5 barrels of ale, 29.5 barrels of beer = ⇒ £776 ◮ 12 barrels of ale, 28 barrels of beer ⇒ £800 = Is this best possible? 9
A Motivating (and Refreshing) Example Beverage Corn (lb) Hops (oz) Malt (lb) Profit (£) Ale (barrel) 5 4 35 13 Beer (barrel) 15 4 20 23 Quantity 480 160 1190 ◮ Mathematical Formulation: 13 A 23 B Profit max + s.t. 5 A + 15 B ≤ 480 Corn 4 A + 4 B ≤ 160 Hops 35 A + 20 B ≤ 1190 Malt A , B ≥ 0 10
A Motivating (and Refreshing) Example Beer Malt: 35 A + 20 B ≤ 1190 35 30 Corn: 5 A + 15 B ≤ 480 25 20 15 Feasible region 10 Hops: 4 A + 4 B ≤ 160 5 Ale 0 0 5 10 15 20 25 30 35 11
A Motivating (and Refreshing) Example Beer Profit ( 0 , 32 ) 35 ( 12 , 28 ) 30 ⇐ Optimum 25 13 A + 23 B = 1000 20 ( 26 , 14 ) 15 Feasible region 13 A + 23 B = 800 10 13 A + 23 B = 700 5 ( 0 , 0 ) ( 34 , 0 ) Ale 0 0 5 10 15 20 25 30 35 12
A Motivating (and Refreshing) Example Beer ( 0 , 32 ) 35 ( 12 , 28 ) 30 25 20 ( 26 , 14 ) 15 Feasible region 10 5 ( 0 , 0 ) ( 34 , 0 ) Ale 0 0 5 10 15 20 25 30 35 Observation: Regardless of objective function coefficients, an optimal solution occurs at an extreme point (vertex). 13
Terminology and Notation Numbers: ◮ R . . . set of real numbers ◮ R ≥ 0 or R + . . . set of non-negative real numbers ◮ R n . . . n-dimensional real vector space ◮ Z , Z ≥ 0 , Z n . . . set of integers, non-negative integers, n-dimensional ... Sets: ◮ S = { s 1 , s 2 , · · · , s k } . . . a set of k elements ◮ S = { x | P ( x ) } . . . set of elements x for which condition P is true ◮ Example: Z ≥ 0 = { i | i ∈ Z and i ≥ 0 } ◮ | S | . . . size (number of elements) of a finite set S ◮ 2 S . . . set of all subsets of S ◮ e.g.: 2 { a , b , c } = {∅ , { a } , { b } , { c } , { a , b } , { a , c } , { b , c } , { a , b , c }} ◮ µ : S �→ T . . . µ is a mapping (or function) from set S to set T 14
Terminology and Notation – Linear Algebra ◮ matrix of dimension m × n : � a 11 a 12 ... a 1 n � | — a T � 1 — a 21 a 22 ... a 2 n | | � . . . . A = = = . A 1 A 2 ... A n . . . . . . . | | | — a T a m 1 a m 2 ... a mn m — � a 11 a 21 ... a m 1 � a 12 a 22 ... a m 2 ◮ and its transpose: A T = . . . . . . . . . a 1 n a 2 n ... a mn � x 1 � . ; row vector x T (the transpose of x ) ◮ Column vector x = . . x n x T y = y T x = � n ◮ Inner product of x , y ∈ R n : i = 1 x i y i ◮ Matrix equation Ax = b is equivalent to a T i x = b i for all i ∈ { 1 , . . . , m } ( b is an m-vector, b i is its i ’th component) 15
Terminology and Notation – Linear Algebra ◮ det( A ) . . . determinant of a matrix ◮ e.g.: det ( a 11 a 21 a 12 a 22 ) = a 11 · a 22 − a 12 · a 21 ◮ e i . . . unit vector (dimension from context) ◮ 1 in i ’th component, 0 else � 1 � 0 � 0 � � � ◮ e.g. (dimension 3): e 1 = e 2 = e 3 = 0 1 0 0 0 1 � � | | | ◮ I = . . . identity matrix (dimension from context, here n ) e 1 e 2 ... e n | | | ◮ 1 on main diagonal, 0 else � 1 0 0 � ◮ e.g. (dimension 3): I = 0 1 0 0 0 1 ◮ rank ( A ) = size of the largest set of linearly independent columns = size of the largest set of linearly independent rows ◮ A − 1 . . . matrix inverse of square matrix A ◮ A − 1 A = A A − 1 = I ◮ A − 1 exists if and only if det( A ) � = 0 16
Optimization Problems Generic optimization problem Given: set X , function f : X → R Task: find x ∗ ∈ X maximizing (minimizing) f ( x ∗ ) , i. e., f ( x ∗ ) ≥ f ( x ) ( f ( x ∗ ) ≤ f ( x )) for all x ∈ X . ◮ An x ∗ with these properties is called optimal solution (optimum). ◮ Here, X is the set of feasible solutions, f is the objective function. Short form: maximize f ( x ) subject to x ∈ X or simply: max { f ( x ) | x ∈ X } . Problem: Too general to say anything meaningful! 17
Convex Optimization Problems Definition 1.1. Let X ⊆ R n and f : X → R . a X is convex if for all x , y ∈ X and 0 ≤ λ ≤ 1 it holds that λ · x + ( 1 − λ ) · y ∈ X . b f is convex if for all x , y ∈ X and 0 ≤ λ ≤ 1 with λ · x + ( 1 − λ ) · y ∈ X it holds that λ · f ( x ) + ( 1 − λ ) · f ( y ) ≥ f ( λ · x + ( 1 − λ ) · y ) . c If X and f are both convex, then min { f ( x ) | x ∈ X } is a convex optimization problem. Note: f : X �→ R is called concave if − f is convex. 18
Local and Global Optimality Definition 1.2. Let X ⊆ R n and f : X �→ R . x ′ ∈ X is a local optimum of the optimization problem min { f ( x ) | x ∈ X } if there is an ε > 0 such that for all x ∈ X with � x ′ − x � 2 ≤ ε . f ( x ′ ) ≤ f ( x ) Theorem 1.3. For a convex optimization problem, every local optimum is a (global) optimum. 19
Optimization Problems Considered in this Course: maximize f ( x ) subject to x ∈ X ◮ X ⊆ R n polyhedron, f linear function − → linear optimization problem (in particular convex) ◮ X ⊆ Z n integer points of a polyhedron, f linear function − → integer linear optimization problem ◮ X related to some combinatorial structure (e. g., graph) − → combinatorial optimization problem ◮ X finite (but usually huge) − → discrete optimization problem 20
Example: Shortest Path Problem Given: directed graph D = ( V , A ) , s weight function w : A → R ≥ 0 , 7 start node s ∈ V , c 5 b 8 destination node t ∈ V . 9 7 5 3 a e t Task: find s - t -path of minimum weight. 4 6 8 9 6 g f 11 That is, X = { P ⊆ A | P is s - t -path in D } and f : X → R is given by � f ( P ) = w ( a ) . a ∈ P Remark. Note that the finite set of feasible solutions X is only implicitly given by D . This holds for all interesting problems in combinatorial optimization! 21
Example: Minimum Spanning Tree (MST) Problem Given: undirected graph G = ( V , E ) , weight a function w : E → R ≥ 0 . 7 8 Task: find connected subgraph of G containing c 5 b 7 all nodes in V with minimum total weight. 9 5 15 e d 6 9 8 11 g f That is, X = { E ′ ⊆ E | E ′ connects all nodes in V } and f : X → R is given by f ( E ′ ) = � w ( e ) . e ∈ E ′ Remarks ◮ Notice that there always exists an optimal solution without cycles. ◮ A connected graph without cycles is called a tree. ◮ A subgraph of G containing all nodes in V is called spanning. 22
Recommend
More recommend
